2024-06-10T02:26:11.313 INFO:root:teuthology version: 0.0.1.dev303+g861a8dc 2024-06-10T02:26:11.313 DEBUG:teuthology.run:Teuthology command: teuthology --owner scheduled_teuthology@teuthology --name teuthology-2024-06-09_21:24:02-fs-squid-distro-default-smithi --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/no} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/no 5-quiesce/with-quiesce 6-workunit/suites/blogbench}} --verbose --archive /home/teuthworker/archive/teuthology-2024-06-09_21:24:02-fs-squid-distro-default-smithi/7747454 -- /home/teuthworker/archive/teuthology-2024-06-09_21:24:02-fs-squid-distro-default-smithi/7747454/orig.config.yaml 2024-06-10T02:26:11.366 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-06-10T02:26:11.458 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-06-09_21:24:02-fs-squid-distro-default-smithi/7747454 branch: squid description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/no} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/no 5-quiesce/with-quiesce 6-workunit/suites/blogbench}} email: ceph-qa@ceph.com first_in_suite: false job_id: '7747454' kernel: client: sha1: distro kdb: 1 sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: teuthology-2024-06-09_21:24:02-fs-squid-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: squid ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 5 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_bal_replicate_threshold: 1 mds_export_ephemeral_random_max: 0.1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - slow metadata IO - SLOW_OPS - slow request sha1: 3d89accfe3201286f000e065a1faea2a88c7a6ba subvols: create: 2 subvol_options: --size 25000000000 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - min: 3 name: mds_cache.dir_update - min: 3 name: mds_cache.dir_update_receipt - mds.dir_split install: ceph: flavor: default sha1: 3d89accfe3201286f000e065a1faea2a88c7a6ba kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=secure - nowsync syntax: v2 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: 7b7a5d375cf4e4afab96b9d4d229b699ae8304bb owner: scheduled_teuthology@teuthology priority: 100 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 178 sha1: 3d89accfe3201286f000e065a1faea2a88c7a6ba sleep_before_teardown: 0 subset: 130/512 suite: fs suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_7b7a5d375cf4e4afab96b9d4d229b699ae8304bb/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: 7b7a5d375cf4e4afab96b9d4d229b699ae8304bb syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi052.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOdUI5Es9BQD3sGZ7KzadEBnXSY4B0/OY66gV6xLjlb1/0I7JieLNBAM//4HkvgqoFDKl/2Y8E+f4S1q2VK5Sxk= smithi120.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCh1pXdHSg8VXOw7ICOZ2C9ahh/vbalIdLG2G1Hh2S+o7tAodgI0/h0wGkXeF+exHxJh0XE6mPq2oTnDQmgFSz8= smithi188.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOEIYxh468S/wbsGnpaqIvzATcLV0T7vdMWJ/+bsYDLDy+YMBWHGm1h9sNH09/Hwx5kkCy/xYAGFVwR9Vhyjd2A= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate false - ceph fs subvolumegroup pin cephfs qa random 0.10 - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - quiescer: null - workunit: clients: all: - suites/blogbench.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 861a8dcf7aa816a26e13f039336f7ed0a9aec0fa timestamp: 2024-06-09_21:24:02 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2698841 2024-06-10T02:26:11.458 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_7b7a5d375cf4e4afab96b9d4d229b699ae8304bb/qa; will attempt to use it 2024-06-10T02:26:11.459 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_7b7a5d375cf4e4afab96b9d4d229b699ae8304bb/qa/tasks 2024-06-10T02:26:11.459 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-06-10T02:26:11.461 INFO:teuthology.task.internal:Checking packages... 2024-06-10T02:26:11.481 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '3d89accfe3201286f000e065a1faea2a88c7a6ba' 2024-06-10T02:26:11.481 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-06-10T02:26:11.481 INFO:teuthology.packaging:ref: None 2024-06-10T02:26:11.481 INFO:teuthology.packaging:tag: None 2024-06-10T02:26:11.481 INFO:teuthology.packaging:branch: squid 2024-06-10T02:26:11.481 INFO:teuthology.packaging:sha1: 3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-10T02:26:11.482 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=squid 2024-06-10T02:26:11.649 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-2970.g7b7a5d37 2024-06-10T02:26:11.650 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-06-10T02:26:11.657 INFO:teuthology.task.internal:no buildpackages task found 2024-06-10T02:26:11.657 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-06-10T02:26:11.668 INFO:teuthology.task.internal:Saving configuration 2024-06-10T02:26:11.687 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-06-10T02:26:11.694 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-06-10T02:26:11.719 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi052.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-06-09_21:24:02-fs-squid-distro-default-smithi/7747454', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-06-10 02:21:17.393502', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOdUI5Es9BQD3sGZ7KzadEBnXSY4B0/OY66gV6xLjlb1/0I7JieLNBAM//4HkvgqoFDKl/2Y8E+f4S1q2VK5Sxk='} 2024-06-10T02:26:11.737 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi120.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-06-09_21:24:02-fs-squid-distro-default-smithi/7747454', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-06-10 02:21:17.394543', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCh1pXdHSg8VXOw7ICOZ2C9ahh/vbalIdLG2G1Hh2S+o7tAodgI0/h0wGkXeF+exHxJh0XE6mPq2oTnDQmgFSz8='} 2024-06-10T02:26:11.759 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi188.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-06-09_21:24:02-fs-squid-distro-default-smithi/7747454', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-06-10 02:21:17.392302', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOEIYxh468S/wbsGnpaqIvzATcLV0T7vdMWJ/+bsYDLDy+YMBWHGm1h9sNH09/Hwx5kkCy/xYAGFVwR9Vhyjd2A='} 2024-06-10T02:26:11.759 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-06-10T02:26:11.765 INFO:teuthology.task.internal:roles: ubuntu@smithi052.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-06-10T02:26:11.766 INFO:teuthology.task.internal:roles: ubuntu@smithi120.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-06-10T02:26:11.766 INFO:teuthology.task.internal:roles: ubuntu@smithi188.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-06-10T02:26:11.766 INFO:teuthology.run_tasks:Running task console_log... 2024-06-10T02:26:11.855 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fa4e7ea7be0>, signals=[15]) 2024-06-10T02:26:11.855 INFO:teuthology.run_tasks:Running task internal.connect... 2024-06-10T02:26:11.861 INFO:teuthology.task.internal:Opening connections... 2024-06-10T02:26:11.861 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi052.front.sepia.ceph.com 2024-06-10T02:26:11.863 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi052.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-10T02:26:11.941 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi120.front.sepia.ceph.com 2024-06-10T02:26:11.942 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi120.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-10T02:26:12.018 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi188.front.sepia.ceph.com 2024-06-10T02:26:12.018 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi188.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-10T02:26:12.091 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-06-10T02:26:12.098 DEBUG:teuthology.orchestra.run.smithi052:> uname -m 2024-06-10T02:26:12.117 INFO:teuthology.orchestra.run.smithi052.stdout:x86_64 2024-06-10T02:26:12.118 DEBUG:teuthology.orchestra.run.smithi052:> cat /etc/os-release 2024-06-10T02:26:12.175 INFO:teuthology.orchestra.run.smithi052.stdout:NAME="CentOS Stream" 2024-06-10T02:26:12.175 INFO:teuthology.orchestra.run.smithi052.stdout:VERSION="9" 2024-06-10T02:26:12.175 INFO:teuthology.orchestra.run.smithi052.stdout:ID="centos" 2024-06-10T02:26:12.175 INFO:teuthology.orchestra.run.smithi052.stdout:ID_LIKE="rhel fedora" 2024-06-10T02:26:12.175 INFO:teuthology.orchestra.run.smithi052.stdout:VERSION_ID="9" 2024-06-10T02:26:12.175 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM_ID="platform:el9" 2024-06-10T02:26:12.176 INFO:teuthology.orchestra.run.smithi052.stdout:PRETTY_NAME="CentOS Stream 9" 2024-06-10T02:26:12.176 INFO:teuthology.orchestra.run.smithi052.stdout:ANSI_COLOR="0;31" 2024-06-10T02:26:12.176 INFO:teuthology.orchestra.run.smithi052.stdout:LOGO="fedora-logo-icon" 2024-06-10T02:26:12.176 INFO:teuthology.orchestra.run.smithi052.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-06-10T02:26:12.176 INFO:teuthology.orchestra.run.smithi052.stdout:HOME_URL="https://centos.org/" 2024-06-10T02:26:12.176 INFO:teuthology.orchestra.run.smithi052.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-06-10T02:26:12.176 INFO:teuthology.orchestra.run.smithi052.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-06-10T02:26:12.176 INFO:teuthology.orchestra.run.smithi052.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-06-10T02:26:12.176 INFO:teuthology.lock.ops:Updating smithi052.front.sepia.ceph.com on lock server 2024-06-10T02:26:12.202 DEBUG:teuthology.orchestra.run.smithi120:> uname -m 2024-06-10T02:26:12.220 INFO:teuthology.orchestra.run.smithi120.stdout:x86_64 2024-06-10T02:26:12.220 DEBUG:teuthology.orchestra.run.smithi120:> cat /etc/os-release 2024-06-10T02:26:12.277 INFO:teuthology.orchestra.run.smithi120.stdout:NAME="CentOS Stream" 2024-06-10T02:26:12.277 INFO:teuthology.orchestra.run.smithi120.stdout:VERSION="9" 2024-06-10T02:26:12.277 INFO:teuthology.orchestra.run.smithi120.stdout:ID="centos" 2024-06-10T02:26:12.277 INFO:teuthology.orchestra.run.smithi120.stdout:ID_LIKE="rhel fedora" 2024-06-10T02:26:12.278 INFO:teuthology.orchestra.run.smithi120.stdout:VERSION_ID="9" 2024-06-10T02:26:12.278 INFO:teuthology.orchestra.run.smithi120.stdout:PLATFORM_ID="platform:el9" 2024-06-10T02:26:12.278 INFO:teuthology.orchestra.run.smithi120.stdout:PRETTY_NAME="CentOS Stream 9" 2024-06-10T02:26:12.278 INFO:teuthology.orchestra.run.smithi120.stdout:ANSI_COLOR="0;31" 2024-06-10T02:26:12.278 INFO:teuthology.orchestra.run.smithi120.stdout:LOGO="fedora-logo-icon" 2024-06-10T02:26:12.278 INFO:teuthology.orchestra.run.smithi120.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-06-10T02:26:12.278 INFO:teuthology.orchestra.run.smithi120.stdout:HOME_URL="https://centos.org/" 2024-06-10T02:26:12.278 INFO:teuthology.orchestra.run.smithi120.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-06-10T02:26:12.278 INFO:teuthology.orchestra.run.smithi120.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-06-10T02:26:12.278 INFO:teuthology.orchestra.run.smithi120.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-06-10T02:26:12.279 INFO:teuthology.lock.ops:Updating smithi120.front.sepia.ceph.com on lock server 2024-06-10T02:26:12.302 DEBUG:teuthology.orchestra.run.smithi188:> uname -m 2024-06-10T02:26:12.319 INFO:teuthology.orchestra.run.smithi188.stdout:x86_64 2024-06-10T02:26:12.320 DEBUG:teuthology.orchestra.run.smithi188:> cat /etc/os-release 2024-06-10T02:26:12.376 INFO:teuthology.orchestra.run.smithi188.stdout:NAME="CentOS Stream" 2024-06-10T02:26:12.377 INFO:teuthology.orchestra.run.smithi188.stdout:VERSION="9" 2024-06-10T02:26:12.377 INFO:teuthology.orchestra.run.smithi188.stdout:ID="centos" 2024-06-10T02:26:12.377 INFO:teuthology.orchestra.run.smithi188.stdout:ID_LIKE="rhel fedora" 2024-06-10T02:26:12.377 INFO:teuthology.orchestra.run.smithi188.stdout:VERSION_ID="9" 2024-06-10T02:26:12.377 INFO:teuthology.orchestra.run.smithi188.stdout:PLATFORM_ID="platform:el9" 2024-06-10T02:26:12.377 INFO:teuthology.orchestra.run.smithi188.stdout:PRETTY_NAME="CentOS Stream 9" 2024-06-10T02:26:12.377 INFO:teuthology.orchestra.run.smithi188.stdout:ANSI_COLOR="0;31" 2024-06-10T02:26:12.377 INFO:teuthology.orchestra.run.smithi188.stdout:LOGO="fedora-logo-icon" 2024-06-10T02:26:12.377 INFO:teuthology.orchestra.run.smithi188.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-06-10T02:26:12.377 INFO:teuthology.orchestra.run.smithi188.stdout:HOME_URL="https://centos.org/" 2024-06-10T02:26:12.377 INFO:teuthology.orchestra.run.smithi188.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-06-10T02:26:12.377 INFO:teuthology.orchestra.run.smithi188.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-06-10T02:26:12.377 INFO:teuthology.orchestra.run.smithi188.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-06-10T02:26:12.378 INFO:teuthology.lock.ops:Updating smithi188.front.sepia.ceph.com on lock server 2024-06-10T02:26:12.402 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-06-10T02:26:12.412 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-06-10T02:26:12.417 INFO:teuthology.task.internal:Checking for old test directory... 2024-06-10T02:26:12.417 DEBUG:teuthology.orchestra.run.smithi052:> test '!' -e /home/ubuntu/cephtest 2024-06-10T02:26:12.419 DEBUG:teuthology.orchestra.run.smithi120:> test '!' -e /home/ubuntu/cephtest 2024-06-10T02:26:12.422 DEBUG:teuthology.orchestra.run.smithi188:> test '!' -e /home/ubuntu/cephtest 2024-06-10T02:26:12.438 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-06-10T02:26:12.444 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-06-10T02:26:12.444 DEBUG:teuthology.orchestra.run.smithi052:> test -z $(ls -A /var/lib/ceph) 2024-06-10T02:26:12.477 DEBUG:teuthology.orchestra.run.smithi120:> test -z $(ls -A /var/lib/ceph) 2024-06-10T02:26:12.480 DEBUG:teuthology.orchestra.run.smithi188:> test -z $(ls -A /var/lib/ceph) 2024-06-10T02:26:12.543 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-06-10T02:26:12.614 INFO:teuthology.run_tasks:Running task kernel... 2024-06-10T02:26:12.628 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': 1, 'sha1': 'distro'} 2024-06-10T02:26:12.628 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2024-06-10T02:26:12.629 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2024-06-10T02:26:12.629 DEBUG:teuthology.orchestra.run.smithi052:> test -f /run/.containerenv -o -f /.dockerenv 2024-06-10T02:26:12.646 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-10T02:26:12.646 DEBUG:teuthology.orchestra.run.smithi052:> uname -r 2024-06-10T02:26:12.704 INFO:teuthology.orchestra.run.smithi052.stdout:5.14.0-437.el9.x86_64 2024-06-10T02:26:12.705 INFO:teuthology.task.kernel:Running kernel on smithi052: 5.14.0-437.el9.x86_64 2024-06-10T02:26:12.705 DEBUG:teuthology.orchestra.run.smithi052:> sudo yum install -y kernel 2024-06-10T02:26:15.064 INFO:teuthology.orchestra.run.smithi052.stdout:CentOS Stream 9 - BaseOS 10 MB/s | 8.1 MB 00:00 2024-06-10T02:26:18.608 INFO:teuthology.orchestra.run.smithi052.stdout:CentOS Stream 9 - AppStream 8.0 MB/s | 19 MB 00:02 2024-06-10T02:26:22.824 INFO:teuthology.orchestra.run.smithi052.stdout:CentOS Stream 9 - CRB 12 MB/s | 6.2 MB 00:00 2024-06-10T02:26:24.357 INFO:teuthology.orchestra.run.smithi052.stdout:CentOS Stream 9 - Extras packages 56 kB/s | 17 kB 00:00 2024-06-10T02:26:25.920 INFO:teuthology.orchestra.run.smithi052.stdout:Extra Packages for Enterprise Linux 15 MB/s | 21 MB 00:01 2024-06-10T02:26:31.334 INFO:teuthology.orchestra.run.smithi052.stdout:lab-extras 42 kB/s | 1.7 kB 00:00 2024-06-10T02:26:33.423 INFO:teuthology.orchestra.run.smithi052.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-06-10T02:26:33.423 INFO:teuthology.orchestra.run.smithi052.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-06-10T02:26:33.512 INFO:teuthology.orchestra.run.smithi052.stdout:Dependencies resolved. 2024-06-10T02:26:33.522 INFO:teuthology.orchestra.run.smithi052.stdout:================================================================================ 2024-06-10T02:26:33.523 INFO:teuthology.orchestra.run.smithi052.stdout: Package Architecture Version Repository Size 2024-06-10T02:26:33.523 INFO:teuthology.orchestra.run.smithi052.stdout:================================================================================ 2024-06-10T02:26:33.523 INFO:teuthology.orchestra.run.smithi052.stdout:Installing: 2024-06-10T02:26:33.523 INFO:teuthology.orchestra.run.smithi052.stdout: kernel x86_64 5.14.0-452.el9 baseos 893 k 2024-06-10T02:26:33.523 INFO:teuthology.orchestra.run.smithi052.stdout:Installing dependencies: 2024-06-10T02:26:33.523 INFO:teuthology.orchestra.run.smithi052.stdout: kernel-core x86_64 5.14.0-452.el9 baseos 16 M 2024-06-10T02:26:33.523 INFO:teuthology.orchestra.run.smithi052.stdout: kernel-modules x86_64 5.14.0-452.el9 baseos 35 M 2024-06-10T02:26:33.523 INFO:teuthology.orchestra.run.smithi052.stdout: kernel-modules-core x86_64 5.14.0-452.el9 baseos 29 M 2024-06-10T02:26:33.523 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-10T02:26:33.523 INFO:teuthology.orchestra.run.smithi052.stdout:Transaction Summary 2024-06-10T02:26:33.523 INFO:teuthology.orchestra.run.smithi052.stdout:================================================================================ 2024-06-10T02:26:33.523 INFO:teuthology.orchestra.run.smithi052.stdout:Install 4 Packages 2024-06-10T02:26:33.524 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-10T02:26:33.524 INFO:teuthology.orchestra.run.smithi052.stdout:Total download size: 80 M 2024-06-10T02:26:33.524 INFO:teuthology.orchestra.run.smithi052.stdout:Installed size: 124 M 2024-06-10T02:26:33.524 INFO:teuthology.orchestra.run.smithi052.stdout:Downloading Packages: 2024-06-10T02:26:33.685 INFO:teuthology.orchestra.run.smithi052.stdout:[MIRROR] kernel-5.14.0-452.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-5.14.0-452.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-10T02:26:33.686 INFO:teuthology.orchestra.run.smithi052.stdout:[MIRROR] kernel-core-5.14.0-452.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-core-5.14.0-452.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-10T02:26:33.690 INFO:teuthology.orchestra.run.smithi052.stdout:[MIRROR] kernel-modules-5.14.0-452.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-modules-5.14.0-452.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-10T02:26:33.876 INFO:teuthology.orchestra.run.smithi052.stdout:(1/4): kernel-5.14.0-452.el9.x86_64.rpm 3.2 MB/s | 893 kB 00:00 2024-06-10T02:26:34.218 INFO:teuthology.orchestra.run.smithi052.stdout:(2/4): kernel-core-5.14.0-452.el9.x86_64.rpm 26 MB/s | 16 MB 00:00 2024-06-10T02:26:34.743 INFO:teuthology.orchestra.run.smithi052.stdout:(3/4): kernel-modules-5.14.0-452.el9.x86_64.rpm 30 MB/s | 35 MB 00:01 2024-06-10T02:26:35.027 INFO:teuthology.orchestra.run.smithi052.stdout:(4/4): kernel-modules-core-5.14.0-452.el9.x86_6 25 MB/s | 29 MB 00:01 2024-06-10T02:26:35.029 INFO:teuthology.orchestra.run.smithi052.stdout:-------------------------------------------------------------------------------- 2024-06-10T02:26:35.030 INFO:teuthology.orchestra.run.smithi052.stdout:Total 53 MB/s | 80 MB 00:01 2024-06-10T02:26:35.756 INFO:teuthology.orchestra.run.smithi052.stdout:Running transaction check 2024-06-10T02:26:36.167 INFO:teuthology.orchestra.run.smithi052.stdout:Transaction check succeeded. 2024-06-10T02:26:36.167 INFO:teuthology.orchestra.run.smithi052.stdout:Running transaction test 2024-06-10T02:26:37.144 INFO:teuthology.orchestra.run.smithi052.stdout:Transaction test succeeded. 2024-06-10T02:26:37.145 INFO:teuthology.orchestra.run.smithi052.stdout:Running transaction 2024-06-10T02:26:39.697 INFO:teuthology.orchestra.run.smithi052.stdout: Preparing : 1/1 2024-06-10T02:26:40.374 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : kernel-modules-core-5.14.0-452.el9.x86_64 1/4 2024-06-10T02:26:40.469 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : kernel-core-5.14.0-452.el9.x86_64 2/4 2024-06-10T02:26:43.254 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: kernel-core-5.14.0-452.el9.x86_64 2/4 2024-06-10T02:26:43.389 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : kernel-modules-5.14.0-452.el9.x86_64 3/4 2024-06-10T02:26:48.484 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: kernel-modules-5.14.0-452.el9.x86_64 3/4 2024-06-10T02:26:48.537 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : kernel-5.14.0-452.el9.x86_64 4/4 2024-06-10T02:26:53.652 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: kernel-modules-core-5.14.0-452.el9.x86_64 4/4 2024-06-10T02:27:24.857 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: kernel-core-5.14.0-452.el9.x86_64 4/4 2024-06-10T02:27:24.899 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: kernel-modules-5.14.0-452.el9.x86_64 4/4 2024-06-10T02:27:25.252 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: kernel-5.14.0-452.el9.x86_64 4/4 2024-06-10T02:27:25.253 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : kernel-5.14.0-452.el9.x86_64 1/4 2024-06-10T02:27:25.253 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : kernel-core-5.14.0-452.el9.x86_64 2/4 2024-06-10T02:27:25.253 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : kernel-modules-5.14.0-452.el9.x86_64 3/4 2024-06-10T02:27:26.059 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : kernel-modules-core-5.14.0-452.el9.x86_64 4/4 2024-06-10T02:27:26.060 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-10T02:27:26.060 INFO:teuthology.orchestra.run.smithi052.stdout:Installed: 2024-06-10T02:27:26.060 INFO:teuthology.orchestra.run.smithi052.stdout: kernel-5.14.0-452.el9.x86_64 2024-06-10T02:27:26.060 INFO:teuthology.orchestra.run.smithi052.stdout: kernel-core-5.14.0-452.el9.x86_64 2024-06-10T02:27:26.060 INFO:teuthology.orchestra.run.smithi052.stdout: kernel-modules-5.14.0-452.el9.x86_64 2024-06-10T02:27:26.060 INFO:teuthology.orchestra.run.smithi052.stdout: kernel-modules-core-5.14.0-452.el9.x86_64 2024-06-10T02:27:26.060 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-10T02:27:26.060 INFO:teuthology.orchestra.run.smithi052.stdout:Complete! 2024-06-10T02:27:26.378 DEBUG:teuthology.orchestra.run.smithi052:> rpm -q kernel | sort -rV | head -n 1 2024-06-10T02:27:26.494 INFO:teuthology.orchestra.run.smithi052.stdout:kernel-5.14.0-452.el9.x86_64 2024-06-10T02:27:26.494 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-452.el9.x86_64 2024-06-10T02:27:26.494 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-437.el9.x86_64 Expected: 5.14.0-452.el9.x86_64 2024-06-10T02:27:26.495 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-06-10T02:27:26.495 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-06-10T02:27:26.495 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2024-06-10T02:27:26.495 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi052.front.sepia.ceph.com, path=None, version=distro) 2024-06-10T02:27:26.495 DEBUG:teuthology.orchestra.run.smithi052:> rpm -q kernel | sort -rV | head -n 1 2024-06-10T02:27:26.549 INFO:teuthology.orchestra.run.smithi052.stdout:kernel-5.14.0-452.el9.x86_64 2024-06-10T02:27:26.549 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-452.el9.x86_64 2024-06-10T02:27:26.550 DEBUG:teuthology.orchestra.run.smithi052:> sudo rpm -qi grub2-tools 2024-06-10T02:27:26.587 INFO:teuthology.orchestra.run.smithi052.stdout:Name : grub2-tools 2024-06-10T02:27:26.587 INFO:teuthology.orchestra.run.smithi052.stdout:Epoch : 1 2024-06-10T02:27:26.587 INFO:teuthology.orchestra.run.smithi052.stdout:Version : 2.06 2024-06-10T02:27:26.587 INFO:teuthology.orchestra.run.smithi052.stdout:Release : 68.el9 2024-06-10T02:27:26.587 INFO:teuthology.orchestra.run.smithi052.stdout:Architecture: x86_64 2024-06-10T02:27:26.587 INFO:teuthology.orchestra.run.smithi052.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-06-10T02:27:26.587 INFO:teuthology.orchestra.run.smithi052.stdout:Group : Unspecified 2024-06-10T02:27:26.587 INFO:teuthology.orchestra.run.smithi052.stdout:Size : 8264254 2024-06-10T02:27:26.587 INFO:teuthology.orchestra.run.smithi052.stdout:License : GPLv3+ 2024-06-10T02:27:26.588 INFO:teuthology.orchestra.run.smithi052.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-06-10T02:27:26.588 INFO:teuthology.orchestra.run.smithi052.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-06-10T02:27:26.588 INFO:teuthology.orchestra.run.smithi052.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-06-10T02:27:26.588 INFO:teuthology.orchestra.run.smithi052.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-06-10T02:27:26.588 INFO:teuthology.orchestra.run.smithi052.stdout:Packager : builder@centos.org 2024-06-10T02:27:26.588 INFO:teuthology.orchestra.run.smithi052.stdout:Vendor : CentOS 2024-06-10T02:27:26.588 INFO:teuthology.orchestra.run.smithi052.stdout:URL : http://www.gnu.org/software/grub/ 2024-06-10T02:27:26.588 INFO:teuthology.orchestra.run.smithi052.stdout:Summary : Support tools for GRUB. 2024-06-10T02:27:26.588 INFO:teuthology.orchestra.run.smithi052.stdout:Description : 2024-06-10T02:27:26.588 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-10T02:27:26.588 INFO:teuthology.orchestra.run.smithi052.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-06-10T02:27:26.588 INFO:teuthology.orchestra.run.smithi052.stdout:customizable bootloader with modular architecture. It supports a rich 2024-06-10T02:27:26.588 INFO:teuthology.orchestra.run.smithi052.stdout:variety of kernel formats, file systems, computer architectures and 2024-06-10T02:27:26.588 INFO:teuthology.orchestra.run.smithi052.stdout:hardware devices. 2024-06-10T02:27:26.589 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-10T02:27:26.589 INFO:teuthology.orchestra.run.smithi052.stdout:This subpackage provides tools for support of all platforms. 2024-06-10T02:27:26.589 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-06-10T02:27:26.590 INFO:teuthology.task.kernel:Updating grub on smithi052 to boot 5.14.0-452.el9.x86_64 2024-06-10T02:27:26.590 DEBUG:teuthology.orchestra.run.smithi052:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-06-10T02:27:27.313 INFO:teuthology.orchestra.run.smithi052.stderr:Generating grub configuration file ... 2024-06-10T02:27:28.736 INFO:teuthology.orchestra.run.smithi052.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-06-10T02:27:28.754 INFO:teuthology.orchestra.run.smithi052.stderr:done 2024-06-10T02:27:28.756 DEBUG:teuthology.orchestra.run.smithi052:> mktemp 2024-06-10T02:27:28.812 INFO:teuthology.orchestra.run.smithi052.stdout:/tmp/tmp.A4L6DDHtbn 2024-06-10T02:27:28.813 DEBUG:teuthology.orchestra.run.smithi052:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.A4L6DDHtbn 2024-06-10T02:27:28.879 DEBUG:teuthology.orchestra.run.smithi052:> sudo chmod 0666 /tmp/tmp.A4L6DDHtbn 2024-06-10T02:27:29.023 DEBUG:teuthology.orchestra.remote:smithi052:/tmp/tmp.A4L6DDHtbn is 6KB 2024-06-10T02:27:29.072 DEBUG:teuthology.orchestra.run.smithi052:> rm -fr /tmp/tmp.A4L6DDHtbn 2024-06-10T02:27:29.088 DEBUG:teuthology.orchestra.run.smithi052:> sudo /bin/ls /boot/loader/entries || true 2024-06-10T02:27:29.158 INFO:teuthology.orchestra.run.smithi052.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-06-10T02:27:29.158 INFO:teuthology.orchestra.run.smithi052.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-06-10T02:27:29.158 INFO:teuthology.orchestra.run.smithi052.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-06-10T02:27:29.158 INFO:teuthology.orchestra.run.smithi052.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-452.el9.x86_64.conf 2024-06-10T02:27:29.159 DEBUG:teuthology.orchestra.run.smithi052:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-5.14.0-452.el9.x86_64 2024-06-10T02:27:29.306 DEBUG:teuthology.orchestra.run.smithi052:> sudo shutdown -r now 2024-06-10T02:27:59.309 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-06-10T02:27:59.309 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi052.front.sepia.ceph.com' 2024-06-10T02:27:59.310 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi052.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-10T02:28:59.313 DEBUG:teuthology.orchestra.remote:timed out 2024-06-10T02:29:08.314 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi052.front.sepia.ceph.com' 2024-06-10T02:29:08.315 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi052.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-10T02:29:08.739 DEBUG:teuthology.orchestra.run.smithi052:> true 2024-06-10T02:29:09.200 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi052.front.sepia.ceph.com' 2024-06-10T02:29:09.200 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5.14.0-452.el9.x86_64"... 2024-06-10T02:29:09.200 DEBUG:teuthology.orchestra.run.smithi052:> uname -r 2024-06-10T02:29:09.215 INFO:teuthology.orchestra.run.smithi052.stdout:5.14.0-452.el9.x86_64 2024-06-10T02:29:09.215 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-452.el9.x86_64 vs 5.14.0-452.el9.x86_64 2024-06-10T02:29:09.216 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-06-10T02:29:09.216 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-06-10T02:29:10.217 DEBUG:teuthology.parallel:result is None 2024-06-10T02:29:10.217 INFO:teuthology.run_tasks:Running task internal.base... 2024-06-10T02:29:10.225 INFO:teuthology.task.internal:Creating test directory... 2024-06-10T02:29:10.225 DEBUG:teuthology.orchestra.run.smithi052:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-06-10T02:29:10.228 DEBUG:teuthology.orchestra.run.smithi120:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-06-10T02:29:10.231 DEBUG:teuthology.orchestra.run.smithi188:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-06-10T02:29:10.249 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-06-10T02:29:10.255 INFO:teuthology.run_tasks:Running task internal.archive... 2024-06-10T02:29:10.262 INFO:teuthology.task.internal:Creating archive directory... 2024-06-10T02:29:10.263 DEBUG:teuthology.orchestra.run.smithi052:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-06-10T02:29:10.290 DEBUG:teuthology.orchestra.run.smithi120:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-06-10T02:29:10.292 DEBUG:teuthology.orchestra.run.smithi188:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-06-10T02:29:10.399 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-06-10T02:29:10.406 INFO:teuthology.task.internal:Enabling coredump saving... 2024-06-10T02:29:10.407 DEBUG:teuthology.orchestra.run.smithi120:> test -f /run/.containerenv -o -f /.dockerenv 2024-06-10T02:29:10.423 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-10T02:29:10.424 DEBUG:teuthology.orchestra.run.smithi188:> test -f /run/.containerenv -o -f /.dockerenv 2024-06-10T02:29:10.440 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-10T02:29:10.440 DEBUG:teuthology.orchestra.run.smithi052:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-06-10T02:29:10.442 DEBUG:teuthology.orchestra.run.smithi120:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-06-10T02:29:10.467 DEBUG:teuthology.orchestra.run.smithi188:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-06-10T02:29:10.501 INFO:teuthology.orchestra.run.smithi120.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-06-10T02:29:10.523 INFO:teuthology.orchestra.run.smithi188.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-06-10T02:29:10.526 INFO:teuthology.orchestra.run.smithi120.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-06-10T02:29:10.555 INFO:teuthology.orchestra.run.smithi052.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-06-10T02:29:10.561 INFO:teuthology.orchestra.run.smithi188.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-06-10T02:29:10.770 INFO:teuthology.orchestra.run.smithi052.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-06-10T02:29:10.772 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-06-10T02:29:10.779 INFO:teuthology.task.internal:Configuring sudo... 2024-06-10T02:29:10.779 DEBUG:teuthology.orchestra.run.smithi052:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-06-10T02:29:10.781 DEBUG:teuthology.orchestra.run.smithi120:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-06-10T02:29:10.783 DEBUG:teuthology.orchestra.run.smithi188:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-06-10T02:29:10.811 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-06-10T02:29:10.826 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-06-10T02:29:10.826 DEBUG:teuthology.orchestra.run.smithi052:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-06-10T02:29:10.851 DEBUG:teuthology.orchestra.run.smithi120:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-06-10T02:29:10.853 DEBUG:teuthology.orchestra.run.smithi188:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-06-10T02:29:10.870 DEBUG:teuthology.orchestra.run.smithi052:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-06-10T02:29:10.953 DEBUG:teuthology.orchestra.run.smithi052:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-06-10T02:29:11.142 DEBUG:teuthology.orchestra.run.smithi052:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-06-10T02:29:11.201 DEBUG:teuthology.orchestra.run.smithi052:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-06-10T02:29:11.274 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-06-10T02:29:11.274 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-06-10T02:29:11.332 DEBUG:teuthology.orchestra.run.smithi120:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-06-10T02:29:11.378 DEBUG:teuthology.orchestra.run.smithi120:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-06-10T02:29:11.428 DEBUG:teuthology.orchestra.run.smithi120:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-06-10T02:29:11.511 DEBUG:teuthology.orchestra.run.smithi120:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-06-10T02:29:11.553 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-06-10T02:29:11.553 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-06-10T02:29:11.619 DEBUG:teuthology.orchestra.run.smithi188:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-06-10T02:29:11.664 DEBUG:teuthology.orchestra.run.smithi188:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-06-10T02:29:11.715 DEBUG:teuthology.orchestra.run.smithi188:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-06-10T02:29:11.799 DEBUG:teuthology.orchestra.run.smithi188:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-06-10T02:29:11.840 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-06-10T02:29:11.840 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-06-10T02:29:11.906 DEBUG:teuthology.orchestra.run.smithi052:> sudo service rsyslog restart 2024-06-10T02:29:11.909 DEBUG:teuthology.orchestra.run.smithi120:> sudo service rsyslog restart 2024-06-10T02:29:11.911 DEBUG:teuthology.orchestra.run.smithi188:> sudo service rsyslog restart 2024-06-10T02:29:11.942 INFO:teuthology.orchestra.run.smithi120.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-06-10T02:29:11.989 INFO:teuthology.orchestra.run.smithi188.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-06-10T02:29:11.990 INFO:teuthology.orchestra.run.smithi052.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-06-10T02:29:12.488 INFO:teuthology.run_tasks:Running task internal.timer... 2024-06-10T02:29:12.497 INFO:teuthology.task.internal:Starting timer... 2024-06-10T02:29:12.497 INFO:teuthology.run_tasks:Running task pcp... 2024-06-10T02:29:12.549 INFO:teuthology.run_tasks:Running task selinux... 2024-06-10T02:29:12.558 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-06-10T02:29:12.558 DEBUG:teuthology.orchestra.run.smithi052:> sudo service auditd rotate 2024-06-10T02:29:12.746 INFO:teuthology.orchestra.run.smithi052.stdout:Rotating logs: 2024-06-10T02:29:12.749 DEBUG:teuthology.orchestra.run.smithi120:> sudo service auditd rotate 2024-06-10T02:29:12.796 INFO:teuthology.orchestra.run.smithi120.stdout:Rotating logs: 2024-06-10T02:29:12.797 DEBUG:teuthology.orchestra.run.smithi188:> sudo service auditd rotate 2024-06-10T02:29:12.864 INFO:teuthology.orchestra.run.smithi188.stdout:Rotating logs: 2024-06-10T02:29:12.866 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-06-10T02:29:12.866 DEBUG:teuthology.orchestra.run.smithi052:> /usr/sbin/getenforce 2024-06-10T02:29:12.890 INFO:teuthology.orchestra.run.smithi052.stdout:Permissive 2024-06-10T02:29:12.890 DEBUG:teuthology.orchestra.run.smithi120:> /usr/sbin/getenforce 2024-06-10T02:29:12.915 INFO:teuthology.orchestra.run.smithi120.stdout:Permissive 2024-06-10T02:29:12.915 DEBUG:teuthology.orchestra.run.smithi188:> /usr/sbin/getenforce 2024-06-10T02:29:12.953 INFO:teuthology.orchestra.run.smithi188.stdout:Permissive 2024-06-10T02:29:12.954 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi052.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi120.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi188.front.sepia.ceph.com': 'permissive'} 2024-06-10T02:29:12.954 DEBUG:teuthology.orchestra.run.smithi052:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-06-10T02:29:12.980 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-10T02:29:12.981 DEBUG:teuthology.orchestra.run.smithi120:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-06-10T02:29:13.006 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-10T02:29:13.007 DEBUG:teuthology.orchestra.run.smithi188:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-06-10T02:29:13.033 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-10T02:29:13.033 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-06-10T02:29:13.033 DEBUG:teuthology.orchestra.run.smithi052:> sudo /usr/sbin/setenforce permissive 2024-06-10T02:29:13.101 DEBUG:teuthology.orchestra.run.smithi120:> sudo /usr/sbin/setenforce permissive 2024-06-10T02:29:13.127 DEBUG:teuthology.orchestra.run.smithi188:> sudo /usr/sbin/setenforce permissive 2024-06-10T02:29:13.155 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-06-10T02:29:13.165 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-06-10T02:29:13.176 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-06-10T02:29:13.177 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi052.front.sepia.ceph.com,smithi120.front.sepia.ceph.com,smithi188.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-06-10T02:33:51.814 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi052.front.sepia.ceph.com'), Remote(name='ubuntu@smithi120.front.sepia.ceph.com'), Remote(name='ubuntu@smithi188.front.sepia.ceph.com')] 2024-06-10T02:33:51.815 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi052.front.sepia.ceph.com' 2024-06-10T02:33:51.817 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi052.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-10T02:33:51.899 DEBUG:teuthology.orchestra.run.smithi052:> true 2024-06-10T02:33:51.990 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi052.front.sepia.ceph.com' 2024-06-10T02:33:51.991 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi120.front.sepia.ceph.com' 2024-06-10T02:33:51.991 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi120.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-10T02:33:52.066 DEBUG:teuthology.orchestra.run.smithi120:> true 2024-06-10T02:33:52.145 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi120.front.sepia.ceph.com' 2024-06-10T02:33:52.146 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi188.front.sepia.ceph.com' 2024-06-10T02:33:52.146 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi188.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-10T02:33:52.221 DEBUG:teuthology.orchestra.run.smithi188:> true 2024-06-10T02:33:52.298 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi188.front.sepia.ceph.com' 2024-06-10T02:33:52.299 INFO:teuthology.run_tasks:Running task clock... 2024-06-10T02:33:52.309 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-06-10T02:33:52.309 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-06-10T02:33:52.310 DEBUG:teuthology.orchestra.run.smithi052:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-06-10T02:33:52.312 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-06-10T02:33:52.313 DEBUG:teuthology.orchestra.run.smithi120:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-06-10T02:33:52.315 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-06-10T02:33:52.315 DEBUG:teuthology.orchestra.run.smithi188:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-06-10T02:33:52.340 INFO:teuthology.orchestra.run.smithi120.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-06-10T02:33:52.342 INFO:teuthology.orchestra.run.smithi052.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-06-10T02:33:52.353 INFO:teuthology.orchestra.run.smithi120.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-06-10T02:33:52.357 INFO:teuthology.orchestra.run.smithi052.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-06-10T02:33:52.368 INFO:teuthology.orchestra.run.smithi188.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-06-10T02:33:52.376 INFO:teuthology.orchestra.run.smithi120.stderr:sudo: ntpd: command not found 2024-06-10T02:33:52.382 INFO:teuthology.orchestra.run.smithi188.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-06-10T02:33:52.383 INFO:teuthology.orchestra.run.smithi052.stderr:sudo: ntpd: command not found 2024-06-10T02:33:52.386 INFO:teuthology.orchestra.run.smithi120.stdout:506 Cannot talk to daemon 2024-06-10T02:33:52.395 INFO:teuthology.orchestra.run.smithi052.stdout:506 Cannot talk to daemon 2024-06-10T02:33:52.399 INFO:teuthology.orchestra.run.smithi120.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-06-10T02:33:52.406 INFO:teuthology.orchestra.run.smithi188.stderr:sudo: ntpd: command not found 2024-06-10T02:33:52.411 INFO:teuthology.orchestra.run.smithi052.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-06-10T02:33:52.411 INFO:teuthology.orchestra.run.smithi120.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-06-10T02:33:52.416 INFO:teuthology.orchestra.run.smithi188.stdout:506 Cannot talk to daemon 2024-06-10T02:33:52.426 INFO:teuthology.orchestra.run.smithi052.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-06-10T02:33:52.430 INFO:teuthology.orchestra.run.smithi188.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-06-10T02:33:52.443 INFO:teuthology.orchestra.run.smithi188.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-06-10T02:33:52.471 INFO:teuthology.orchestra.run.smithi120.stderr:bash: line 1: ntpq: command not found 2024-06-10T02:33:52.475 INFO:teuthology.orchestra.run.smithi120.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-06-10T02:33:52.475 INFO:teuthology.orchestra.run.smithi120.stdout:=============================================================================== 2024-06-10T02:33:52.475 INFO:teuthology.orchestra.run.smithi120.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-10T02:33:52.475 INFO:teuthology.orchestra.run.smithi120.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-10T02:33:52.475 INFO:teuthology.orchestra.run.smithi120.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-10T02:33:52.476 INFO:teuthology.orchestra.run.smithi120.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-10T02:33:52.487 INFO:teuthology.orchestra.run.smithi052.stderr:bash: line 1: ntpq: command not found 2024-06-10T02:33:52.492 INFO:teuthology.orchestra.run.smithi052.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-06-10T02:33:52.492 INFO:teuthology.orchestra.run.smithi052.stdout:=============================================================================== 2024-06-10T02:33:52.492 INFO:teuthology.orchestra.run.smithi052.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-10T02:33:52.492 INFO:teuthology.orchestra.run.smithi052.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-10T02:33:52.492 INFO:teuthology.orchestra.run.smithi052.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-10T02:33:52.492 INFO:teuthology.orchestra.run.smithi052.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-10T02:33:52.500 INFO:teuthology.orchestra.run.smithi188.stderr:bash: line 1: ntpq: command not found 2024-06-10T02:33:52.504 INFO:teuthology.orchestra.run.smithi188.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-06-10T02:33:52.504 INFO:teuthology.orchestra.run.smithi188.stdout:=============================================================================== 2024-06-10T02:33:52.504 INFO:teuthology.orchestra.run.smithi188.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-10T02:33:52.504 INFO:teuthology.orchestra.run.smithi188.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-10T02:33:52.504 INFO:teuthology.orchestra.run.smithi188.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-10T02:33:52.504 INFO:teuthology.orchestra.run.smithi188.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-10T02:33:52.505 INFO:teuthology.run_tasks:Running task install... 2024-06-10T02:33:52.513 DEBUG:teuthology.task.install:project ceph 2024-06-10T02:33:52.513 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '3d89accfe3201286f000e065a1faea2a88c7a6ba'}} 2024-06-10T02:33:52.513 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '3d89accfe3201286f000e065a1faea2a88c7a6ba'} 2024-06-10T02:33:52.513 INFO:teuthology.task.install:Using flavor: default 2024-06-10T02:33:52.518 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-06-10T02:33:52.519 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-06-10T02:33:52.519 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '3d89accfe3201286f000e065a1faea2a88c7a6ba', 'tag': None, 'wait_for_package': False} 2024-06-10T02:33:52.519 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-10T02:33:52.521 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '3d89accfe3201286f000e065a1faea2a88c7a6ba', 'tag': None, 'wait_for_package': False} 2024-06-10T02:33:52.521 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-10T02:33:52.522 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '3d89accfe3201286f000e065a1faea2a88c7a6ba', 'tag': None, 'wait_for_package': False} 2024-06-10T02:33:52.523 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-10T02:33:52.666 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/ 2024-06-10T02:33:52.666 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2946.g3d89accf 2024-06-10T02:33:52.678 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/ 2024-06-10T02:33:52.678 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2946.g3d89accf 2024-06-10T02:33:52.738 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/ 2024-06-10T02:33:52.738 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2946.g3d89accf 2024-06-10T02:33:52.790 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-06-10T02:33:52.790 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-06-10T02:33:52.790 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-06-10T02:33:52.815 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-06-10T02:33:52.815 DEBUG:teuthology.orchestra.run.smithi188:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/3d89accfe3201286f000e065a1faea2a88c7a6ba/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-06-10T02:33:52.839 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-06-10T02:33:52.840 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-06-10T02:33:52.840 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-06-10T02:33:52.847 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-06-10T02:33:52.848 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-06-10T02:33:52.848 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-06-10T02:33:52.869 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-06-10T02:33:52.869 DEBUG:teuthology.orchestra.run.smithi052:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/3d89accfe3201286f000e065a1faea2a88c7a6ba/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-06-10T02:33:52.872 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-06-10T02:33:52.872 DEBUG:teuthology.orchestra.run.smithi120:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/3d89accfe3201286f000e065a1faea2a88c7a6ba/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-06-10T02:33:52.880 DEBUG:teuthology.orchestra.run.smithi188:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-06-10T02:33:52.936 DEBUG:teuthology.orchestra.run.smithi052:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-06-10T02:33:52.937 DEBUG:teuthology.orchestra.run.smithi120:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-06-10T02:33:52.953 DEBUG:teuthology.orchestra.run.smithi188:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-06-10T02:33:53.010 DEBUG:teuthology.orchestra.run.smithi120:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-06-10T02:33:53.013 DEBUG:teuthology.orchestra.run.smithi052:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-06-10T02:33:53.015 INFO:teuthology.orchestra.run.smithi188.stdout:check_obsoletes = 1 2024-06-10T02:33:53.017 DEBUG:teuthology.orchestra.run.smithi188:> sudo yum clean all 2024-06-10T02:33:53.073 INFO:teuthology.orchestra.run.smithi120.stdout:check_obsoletes = 1 2024-06-10T02:33:53.075 DEBUG:teuthology.orchestra.run.smithi120:> sudo yum clean all 2024-06-10T02:33:53.077 INFO:teuthology.orchestra.run.smithi052.stdout:check_obsoletes = 1 2024-06-10T02:33:53.079 DEBUG:teuthology.orchestra.run.smithi052:> sudo yum clean all 2024-06-10T02:33:53.240 INFO:teuthology.orchestra.run.smithi188.stdout:45 files removed 2024-06-10T02:33:53.268 DEBUG:teuthology.orchestra.run.smithi188:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-06-10T02:33:53.298 INFO:teuthology.orchestra.run.smithi120.stdout:45 files removed 2024-06-10T02:33:53.321 INFO:teuthology.orchestra.run.smithi052.stdout:45 files removed 2024-06-10T02:33:53.324 DEBUG:teuthology.orchestra.run.smithi120:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-06-10T02:33:53.349 DEBUG:teuthology.orchestra.run.smithi052:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-06-10T02:33:53.856 INFO:teuthology.orchestra.run.smithi188.stdout:ceph packages for x86_64 255 kB/s | 83 kB 00:00 2024-06-10T02:33:53.992 INFO:teuthology.orchestra.run.smithi120.stdout:ceph packages for x86_64 204 kB/s | 83 kB 00:00 2024-06-10T02:33:54.057 INFO:teuthology.orchestra.run.smithi052.stdout:ceph packages for x86_64 200 kB/s | 83 kB 00:00 2024-06-10T02:33:54.298 INFO:teuthology.orchestra.run.smithi188.stdout:ceph noarch packages 29 kB/s | 11 kB 00:00 2024-06-10T02:33:54.384 INFO:teuthology.orchestra.run.smithi120.stdout:ceph noarch packages 33 kB/s | 11 kB 00:00 2024-06-10T02:33:54.440 INFO:teuthology.orchestra.run.smithi052.stdout:ceph noarch packages 35 kB/s | 11 kB 00:00 2024-06-10T02:33:54.685 INFO:teuthology.orchestra.run.smithi120.stdout:ceph source packages 7.1 kB/s | 1.9 kB 00:00 2024-06-10T02:33:54.715 INFO:teuthology.orchestra.run.smithi188.stdout:ceph source packages 4.9 kB/s | 1.9 kB 00:00 2024-06-10T02:33:54.799 INFO:teuthology.orchestra.run.smithi052.stdout:ceph source packages 6.0 kB/s | 1.9 kB 00:00 2024-06-10T02:33:55.259 INFO:teuthology.orchestra.run.smithi188.stdout:CentOS Stream 9 - BaseOS 16 MB/s | 8.1 MB 00:00 2024-06-10T02:33:56.496 INFO:teuthology.orchestra.run.smithi120.stdout:CentOS Stream 9 - BaseOS 4.6 MB/s | 8.1 MB 00:01 2024-06-10T02:33:56.827 INFO:teuthology.orchestra.run.smithi052.stdout:CentOS Stream 9 - BaseOS 4.1 MB/s | 8.1 MB 00:01 2024-06-10T02:33:57.252 INFO:teuthology.orchestra.run.smithi188.stdout:CentOS Stream 9 - AppStream 21 MB/s | 19 MB 00:00 2024-06-10T02:34:00.162 INFO:teuthology.orchestra.run.smithi052.stdout:CentOS Stream 9 - AppStream 8.8 MB/s | 19 MB 00:02 2024-06-10T02:34:02.435 INFO:teuthology.orchestra.run.smithi188.stdout:CentOS Stream 9 - CRB 4.0 MB/s | 6.2 MB 00:01 2024-06-10T02:34:03.860 INFO:teuthology.orchestra.run.smithi188.stdout:CentOS Stream 9 - Extras packages 63 kB/s | 17 kB 00:00 2024-06-10T02:34:05.045 INFO:teuthology.orchestra.run.smithi052.stdout:CentOS Stream 9 - CRB 5.5 MB/s | 6.2 MB 00:01 2024-06-10T02:34:05.130 INFO:teuthology.orchestra.run.smithi120.stdout:CentOS Stream 9 - AppStream 2.5 MB/s | 19 MB 00:07 2024-06-10T02:34:06.628 INFO:teuthology.orchestra.run.smithi052.stdout:CentOS Stream 9 - Extras packages 47 kB/s | 17 kB 00:00 2024-06-10T02:34:08.209 INFO:teuthology.orchestra.run.smithi052.stdout:Extra Packages for Enterprise Linux 15 MB/s | 21 MB 00:01 2024-06-10T02:34:10.330 INFO:teuthology.orchestra.run.smithi120.stdout:CentOS Stream 9 - CRB 3.9 MB/s | 6.2 MB 00:01 2024-06-10T02:34:11.897 INFO:teuthology.orchestra.run.smithi120.stdout:CentOS Stream 9 - Extras packages 42 kB/s | 17 kB 00:00 2024-06-10T02:34:12.091 INFO:teuthology.orchestra.run.smithi188.stdout:Extra Packages for Enterprise Linux 2.6 MB/s | 21 MB 00:08 2024-06-10T02:34:13.301 INFO:teuthology.orchestra.run.smithi120.stdout:Extra Packages for Enterprise Linux 17 MB/s | 21 MB 00:01 2024-06-10T02:34:13.605 INFO:teuthology.orchestra.run.smithi052.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-06-10T02:34:16.007 INFO:teuthology.orchestra.run.smithi052.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-06-10T02:34:16.008 INFO:teuthology.orchestra.run.smithi052.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-06-10T02:34:16.019 INFO:teuthology.orchestra.run.smithi052.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-06-10T02:34:16.020 INFO:teuthology.orchestra.run.smithi052.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-06-10T02:34:16.021 INFO:teuthology.orchestra.run.smithi052.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-06-10T02:34:16.022 INFO:teuthology.orchestra.run.smithi052.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-06-10T02:34:16.023 INFO:teuthology.orchestra.run.smithi052.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-06-10T02:34:16.026 INFO:teuthology.orchestra.run.smithi052.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-06-10T02:34:16.027 INFO:teuthology.orchestra.run.smithi052.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-06-10T02:34:16.093 INFO:teuthology.orchestra.run.smithi052.stdout:Dependencies resolved. 2024-06-10T02:34:16.116 INFO:teuthology.orchestra.run.smithi052.stdout:======================================================================================= 2024-06-10T02:34:16.116 INFO:teuthology.orchestra.run.smithi052.stdout: Package Arch Version Repository Size 2024-06-10T02:34:16.116 INFO:teuthology.orchestra.run.smithi052.stdout:======================================================================================= 2024-06-10T02:34:16.116 INFO:teuthology.orchestra.run.smithi052.stdout:Installing: 2024-06-10T02:34:16.116 INFO:teuthology.orchestra.run.smithi052.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-06-10T02:34:16.116 INFO:teuthology.orchestra.run.smithi052.stdout: ceph x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 6.6 k 2024-06-10T02:34:16.116 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-base x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 5.4 M 2024-06-10T02:34:16.117 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-fuse x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 890 k 2024-06-10T02:34:16.117 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 145 k 2024-06-10T02:34:16.117 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-mgr x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 1.5 M 2024-06-10T02:34:16.117 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 142 k 2024-06-10T02:34:16.117 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 3.5 M 2024-06-10T02:34:16.117 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 7.4 M 2024-06-10T02:34:16.117 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-mgr-rook noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 49 k 2024-06-10T02:34:16.117 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-radosgw x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 10 M 2024-06-10T02:34:16.117 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-test x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 48 M 2024-06-10T02:34:16.117 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-volume noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 264 k 2024-06-10T02:34:16.117 INFO:teuthology.orchestra.run.smithi052.stdout: cephadm noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 763 k 2024-06-10T02:34:16.117 INFO:teuthology.orchestra.run.smithi052.stdout: cephfs-mirror x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 221 k 2024-06-10T02:34:16.118 INFO:teuthology.orchestra.run.smithi052.stdout: cephfs-shell x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 27 k 2024-06-10T02:34:16.118 INFO:teuthology.orchestra.run.smithi052.stdout: cephfs-top noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 18 k 2024-06-10T02:34:16.118 INFO:teuthology.orchestra.run.smithi052.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-06-10T02:34:16.118 INFO:teuthology.orchestra.run.smithi052.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-06-10T02:34:16.118 INFO:teuthology.orchestra.run.smithi052.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-06-10T02:34:16.118 INFO:teuthology.orchestra.run.smithi052.stdout: libcephfs-devel x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 32 k 2024-06-10T02:34:16.118 INFO:teuthology.orchestra.run.smithi052.stdout: libcephfs2 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 747 k 2024-06-10T02:34:16.118 INFO:teuthology.orchestra.run.smithi052.stdout: librados-devel x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 127 k 2024-06-10T02:34:16.118 INFO:teuthology.orchestra.run.smithi052.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-06-10T02:34:16.118 INFO:teuthology.orchestra.run.smithi052.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-06-10T02:34:16.118 INFO:teuthology.orchestra.run.smithi052.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-06-10T02:34:16.118 INFO:teuthology.orchestra.run.smithi052.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-06-10T02:34:16.119 INFO:teuthology.orchestra.run.smithi052.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-06-10T02:34:16.119 INFO:teuthology.orchestra.run.smithi052.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-06-10T02:34:16.119 INFO:teuthology.orchestra.run.smithi052.stdout: python3-cephfs x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 162 k 2024-06-10T02:34:16.119 INFO:teuthology.orchestra.run.smithi052.stdout: python3-rados x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 321 k 2024-06-10T02:34:16.119 INFO:teuthology.orchestra.run.smithi052.stdout: python3-rbd x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 300 k 2024-06-10T02:34:16.119 INFO:teuthology.orchestra.run.smithi052.stdout: python3-rgw x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 99 k 2024-06-10T02:34:16.119 INFO:teuthology.orchestra.run.smithi052.stdout: rbd-fuse x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 85 k 2024-06-10T02:34:16.119 INFO:teuthology.orchestra.run.smithi052.stdout: rbd-mirror x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 3.1 M 2024-06-10T02:34:16.119 INFO:teuthology.orchestra.run.smithi052.stdout: rbd-nbd x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 171 k 2024-06-10T02:34:16.119 INFO:teuthology.orchestra.run.smithi052.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-06-10T02:34:16.119 INFO:teuthology.orchestra.run.smithi052.stdout:Upgrading: 2024-06-10T02:34:16.119 INFO:teuthology.orchestra.run.smithi052.stdout: NetworkManager x86_64 1:1.47.91-1.el9 baseos 2.3 M 2024-06-10T02:34:16.120 INFO:teuthology.orchestra.run.smithi052.stdout: NetworkManager-libnm x86_64 1:1.47.91-1.el9 baseos 1.8 M 2024-06-10T02:34:16.120 INFO:teuthology.orchestra.run.smithi052.stdout: NetworkManager-team x86_64 1:1.47.91-1.el9 baseos 41 k 2024-06-10T02:34:16.120 INFO:teuthology.orchestra.run.smithi052.stdout: NetworkManager-tui x86_64 1:1.47.91-1.el9 baseos 248 k 2024-06-10T02:34:16.120 INFO:teuthology.orchestra.run.smithi052.stdout: elfutils-debuginfod-client x86_64 0.191-4.el9 baseos 37 k 2024-06-10T02:34:16.120 INFO:teuthology.orchestra.run.smithi052.stdout: elfutils-libelf x86_64 0.191-4.el9 baseos 207 k 2024-06-10T02:34:16.120 INFO:teuthology.orchestra.run.smithi052.stdout: elfutils-libs x86_64 0.191-4.el9 baseos 260 k 2024-06-10T02:34:16.120 INFO:teuthology.orchestra.run.smithi052.stdout: librados2 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 3.4 M 2024-06-10T02:34:16.120 INFO:teuthology.orchestra.run.smithi052.stdout: librbd1 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 3.1 M 2024-06-10T02:34:16.120 INFO:teuthology.orchestra.run.smithi052.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-06-10T02:34:16.120 INFO:teuthology.orchestra.run.smithi052.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-06-10T02:34:16.120 INFO:teuthology.orchestra.run.smithi052.stdout:Installing dependencies: 2024-06-10T02:34:16.120 INFO:teuthology.orchestra.run.smithi052.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-06-10T02:34:16.121 INFO:teuthology.orchestra.run.smithi052.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-06-10T02:34:16.121 INFO:teuthology.orchestra.run.smithi052.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-06-10T02:34:16.121 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-common x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 21 M 2024-06-10T02:34:16.121 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 32 k 2024-06-10T02:34:16.121 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-mds x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 2.4 M 2024-06-10T02:34:16.121 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 249 k 2024-06-10T02:34:16.121 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-mon x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 4.7 M 2024-06-10T02:34:16.121 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-osd x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 17 M 2024-06-10T02:34:16.121 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 16 k 2024-06-10T02:34:16.121 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-selinux x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 25 k 2024-06-10T02:34:16.121 INFO:teuthology.orchestra.run.smithi052.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-06-10T02:34:16.121 INFO:teuthology.orchestra.run.smithi052.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-06-10T02:34:16.122 INFO:teuthology.orchestra.run.smithi052.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-06-10T02:34:16.122 INFO:teuthology.orchestra.run.smithi052.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-06-10T02:34:16.122 INFO:teuthology.orchestra.run.smithi052.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-06-10T02:34:16.122 INFO:teuthology.orchestra.run.smithi052.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-06-10T02:34:16.122 INFO:teuthology.orchestra.run.smithi052.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-06-10T02:34:16.122 INFO:teuthology.orchestra.run.smithi052.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-06-10T02:34:16.122 INFO:teuthology.orchestra.run.smithi052.stdout: libcephsqlite x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 163 k 2024-06-10T02:34:16.122 INFO:teuthology.orchestra.run.smithi052.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-06-10T02:34:16.122 INFO:teuthology.orchestra.run.smithi052.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-06-10T02:34:16.122 INFO:teuthology.orchestra.run.smithi052.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-06-10T02:34:16.122 INFO:teuthology.orchestra.run.smithi052.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-06-10T02:34:16.122 INFO:teuthology.orchestra.run.smithi052.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-06-10T02:34:16.123 INFO:teuthology.orchestra.run.smithi052.stdout: libradosstriper1 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 501 k 2024-06-10T02:34:16.123 INFO:teuthology.orchestra.run.smithi052.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-06-10T02:34:16.123 INFO:teuthology.orchestra.run.smithi052.stdout: librgw2 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 5.3 M 2024-06-10T02:34:16.123 INFO:teuthology.orchestra.run.smithi052.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-06-10T02:34:16.123 INFO:teuthology.orchestra.run.smithi052.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-06-10T02:34:16.123 INFO:teuthology.orchestra.run.smithi052.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-06-10T02:34:16.123 INFO:teuthology.orchestra.run.smithi052.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-06-10T02:34:16.123 INFO:teuthology.orchestra.run.smithi052.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-06-10T02:34:16.123 INFO:teuthology.orchestra.run.smithi052.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-06-10T02:34:16.123 INFO:teuthology.orchestra.run.smithi052.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-06-10T02:34:16.124 INFO:teuthology.orchestra.run.smithi052.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-06-10T02:34:16.124 INFO:teuthology.orchestra.run.smithi052.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-06-10T02:34:16.124 INFO:teuthology.orchestra.run.smithi052.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-06-10T02:34:16.124 INFO:teuthology.orchestra.run.smithi052.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-06-10T02:34:16.124 INFO:teuthology.orchestra.run.smithi052.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-06-10T02:34:16.124 INFO:teuthology.orchestra.run.smithi052.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-06-10T02:34:16.124 INFO:teuthology.orchestra.run.smithi052.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-06-10T02:34:16.124 INFO:teuthology.orchestra.run.smithi052.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-06-10T02:34:16.124 INFO:teuthology.orchestra.run.smithi052.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-06-10T02:34:16.124 INFO:teuthology.orchestra.run.smithi052.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-06-10T02:34:16.124 INFO:teuthology.orchestra.run.smithi052.stdout: python3-ceph-argparse x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 45 k 2024-06-10T02:34:16.124 INFO:teuthology.orchestra.run.smithi052.stdout: python3-ceph-common x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 135 k 2024-06-10T02:34:16.125 INFO:teuthology.orchestra.run.smithi052.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-06-10T02:34:16.125 INFO:teuthology.orchestra.run.smithi052.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-06-10T02:34:16.125 INFO:teuthology.orchestra.run.smithi052.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-06-10T02:34:16.125 INFO:teuthology.orchestra.run.smithi052.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-06-10T02:34:16.125 INFO:teuthology.orchestra.run.smithi052.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-06-10T02:34:16.125 INFO:teuthology.orchestra.run.smithi052.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-06-10T02:34:16.125 INFO:teuthology.orchestra.run.smithi052.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-06-10T02:34:16.125 INFO:teuthology.orchestra.run.smithi052.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-06-10T02:34:16.125 INFO:teuthology.orchestra.run.smithi052.stdout: python3-google-auth noarch 1:2.29.0-2.el9 epel 205 k 2024-06-10T02:34:16.125 INFO:teuthology.orchestra.run.smithi052.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-06-10T02:34:16.125 INFO:teuthology.orchestra.run.smithi052.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-06-10T02:34:16.125 INFO:teuthology.orchestra.run.smithi052.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-06-10T02:34:16.125 INFO:teuthology.orchestra.run.smithi052.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-06-10T02:34:16.125 INFO:teuthology.orchestra.run.smithi052.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-06-10T02:34:16.125 INFO:teuthology.orchestra.run.smithi052.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-06-10T02:34:16.126 INFO:teuthology.orchestra.run.smithi052.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-06-10T02:34:16.126 INFO:teuthology.orchestra.run.smithi052.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-06-10T02:34:16.126 INFO:teuthology.orchestra.run.smithi052.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-06-10T02:34:16.126 INFO:teuthology.orchestra.run.smithi052.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-06-10T02:34:16.126 INFO:teuthology.orchestra.run.smithi052.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-06-10T02:34:16.126 INFO:teuthology.orchestra.run.smithi052.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-06-10T02:34:16.126 INFO:teuthology.orchestra.run.smithi052.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-06-10T02:34:16.126 INFO:teuthology.orchestra.run.smithi052.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-06-10T02:34:16.126 INFO:teuthology.orchestra.run.smithi052.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-06-10T02:34:16.127 INFO:teuthology.orchestra.run.smithi052.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-06-10T02:34:16.127 INFO:teuthology.orchestra.run.smithi052.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-06-10T02:34:16.127 INFO:teuthology.orchestra.run.smithi052.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-06-10T02:34:16.127 INFO:teuthology.orchestra.run.smithi052.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-06-10T02:34:16.127 INFO:teuthology.orchestra.run.smithi052.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-06-10T02:34:16.127 INFO:teuthology.orchestra.run.smithi052.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-06-10T02:34:16.127 INFO:teuthology.orchestra.run.smithi052.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-06-10T02:34:16.127 INFO:teuthology.orchestra.run.smithi052.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-06-10T02:34:16.127 INFO:teuthology.orchestra.run.smithi052.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-06-10T02:34:16.127 INFO:teuthology.orchestra.run.smithi052.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-06-10T02:34:16.127 INFO:teuthology.orchestra.run.smithi052.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-06-10T02:34:16.127 INFO:teuthology.orchestra.run.smithi052.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-06-10T02:34:16.127 INFO:teuthology.orchestra.run.smithi052.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-06-10T02:34:16.128 INFO:teuthology.orchestra.run.smithi052.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-06-10T02:34:16.128 INFO:teuthology.orchestra.run.smithi052.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-06-10T02:34:16.128 INFO:teuthology.orchestra.run.smithi052.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-06-10T02:34:16.128 INFO:teuthology.orchestra.run.smithi052.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-06-10T02:34:16.128 INFO:teuthology.orchestra.run.smithi052.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-06-10T02:34:16.128 INFO:teuthology.orchestra.run.smithi052.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-06-10T02:34:16.128 INFO:teuthology.orchestra.run.smithi052.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-06-10T02:34:16.128 INFO:teuthology.orchestra.run.smithi052.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-06-10T02:34:16.128 INFO:teuthology.orchestra.run.smithi052.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-06-10T02:34:16.128 INFO:teuthology.orchestra.run.smithi052.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-06-10T02:34:16.128 INFO:teuthology.orchestra.run.smithi052.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-06-10T02:34:16.128 INFO:teuthology.orchestra.run.smithi052.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-06-10T02:34:16.128 INFO:teuthology.orchestra.run.smithi052.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-06-10T02:34:16.129 INFO:teuthology.orchestra.run.smithi052.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-06-10T02:34:16.129 INFO:teuthology.orchestra.run.smithi052.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-06-10T02:34:16.129 INFO:teuthology.orchestra.run.smithi052.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-06-10T02:34:16.129 INFO:teuthology.orchestra.run.smithi052.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-06-10T02:34:16.129 INFO:teuthology.orchestra.run.smithi052.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-06-10T02:34:16.129 INFO:teuthology.orchestra.run.smithi052.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-06-10T02:34:16.129 INFO:teuthology.orchestra.run.smithi052.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-06-10T02:34:16.129 INFO:teuthology.orchestra.run.smithi052.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-06-10T02:34:16.129 INFO:teuthology.orchestra.run.smithi052.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-06-10T02:34:16.129 INFO:teuthology.orchestra.run.smithi052.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-06-10T02:34:16.129 INFO:teuthology.orchestra.run.smithi052.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-06-10T02:34:16.129 INFO:teuthology.orchestra.run.smithi052.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-06-10T02:34:16.129 INFO:teuthology.orchestra.run.smithi052.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-06-10T02:34:16.130 INFO:teuthology.orchestra.run.smithi052.stdout:Installing weak dependencies: 2024-06-10T02:34:16.130 INFO:teuthology.orchestra.run.smithi052.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-06-10T02:34:16.130 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-10T02:34:16.130 INFO:teuthology.orchestra.run.smithi052.stdout:Transaction Summary 2024-06-10T02:34:16.130 INFO:teuthology.orchestra.run.smithi052.stdout:======================================================================================= 2024-06-10T02:34:16.130 INFO:teuthology.orchestra.run.smithi052.stdout:Install 148 Packages 2024-06-10T02:34:16.130 INFO:teuthology.orchestra.run.smithi052.stdout:Upgrade 11 Packages 2024-06-10T02:34:16.130 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-10T02:34:16.131 INFO:teuthology.orchestra.run.smithi052.stdout:Total download size: 225 M 2024-06-10T02:34:16.131 INFO:teuthology.orchestra.run.smithi052.stdout:Downloading Packages: 2024-06-10T02:34:17.314 INFO:teuthology.orchestra.run.smithi052.stdout:(1/159): ceph-19.0.0-2946.g3d89accf.el9.x86_64. 38 kB/s | 6.6 kB 00:00 2024-06-10T02:34:17.334 INFO:teuthology.orchestra.run.smithi188.stdout:lab-extras 22 kB/s | 1.7 kB 00:00 2024-06-10T02:34:17.531 INFO:teuthology.orchestra.run.smithi052.stdout:(2/159): ceph-fuse-19.0.0-2946.g3d89accf.el9.x8 4.0 MB/s | 890 kB 00:00 2024-06-10T02:34:17.590 INFO:teuthology.orchestra.run.smithi052.stdout:(3/159): ceph-immutable-object-cache-19.0.0-294 2.4 MB/s | 145 kB 00:00 2024-06-10T02:34:17.890 INFO:teuthology.orchestra.run.smithi052.stdout:(4/159): ceph-mds-19.0.0-2946.g3d89accf.el9.x86 7.9 MB/s | 2.4 MB 00:00 2024-06-10T02:34:18.099 INFO:teuthology.orchestra.run.smithi052.stdout:(5/159): ceph-mgr-19.0.0-2946.g3d89accf.el9.x86 7.2 MB/s | 1.5 MB 00:00 2024-06-10T02:34:18.316 INFO:teuthology.orchestra.run.smithi052.stdout:(6/159): ceph-base-19.0.0-2946.g3d89accf.el9.x8 4.6 MB/s | 5.4 MB 00:01 2024-06-10T02:34:18.507 INFO:teuthology.orchestra.run.smithi120.stdout:lab-extras 17 kB/s | 1.7 kB 00:00 2024-06-10T02:34:18.724 INFO:teuthology.orchestra.run.smithi052.stdout:(7/159): ceph-mon-19.0.0-2946.g3d89accf.el9.x86 7.5 MB/s | 4.7 MB 00:00 2024-06-10T02:34:19.083 INFO:teuthology.orchestra.run.smithi052.stdout:(8/159): ceph-common-19.0.0-2946.g3d89accf.el9. 11 MB/s | 21 MB 00:01 2024-06-10T02:34:19.141 INFO:teuthology.orchestra.run.smithi052.stdout:(9/159): ceph-selinux-19.0.0-2946.g3d89accf.el9 433 kB/s | 25 kB 00:00 2024-06-10T02:34:19.628 INFO:teuthology.orchestra.run.smithi188.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-06-10T02:34:19.629 INFO:teuthology.orchestra.run.smithi188.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-06-10T02:34:19.639 INFO:teuthology.orchestra.run.smithi188.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-06-10T02:34:19.640 INFO:teuthology.orchestra.run.smithi188.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-06-10T02:34:19.641 INFO:teuthology.orchestra.run.smithi188.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-06-10T02:34:19.642 INFO:teuthology.orchestra.run.smithi188.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-06-10T02:34:19.643 INFO:teuthology.orchestra.run.smithi188.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-06-10T02:34:19.646 INFO:teuthology.orchestra.run.smithi188.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-06-10T02:34:19.646 INFO:teuthology.orchestra.run.smithi188.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-06-10T02:34:19.710 INFO:teuthology.orchestra.run.smithi188.stdout:Dependencies resolved. 2024-06-10T02:34:19.724 INFO:teuthology.orchestra.run.smithi188.stdout:======================================================================================= 2024-06-10T02:34:19.724 INFO:teuthology.orchestra.run.smithi188.stdout: Package Arch Version Repository Size 2024-06-10T02:34:19.724 INFO:teuthology.orchestra.run.smithi188.stdout:======================================================================================= 2024-06-10T02:34:19.724 INFO:teuthology.orchestra.run.smithi188.stdout:Installing: 2024-06-10T02:34:19.724 INFO:teuthology.orchestra.run.smithi188.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-06-10T02:34:19.724 INFO:teuthology.orchestra.run.smithi188.stdout: ceph x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 6.6 k 2024-06-10T02:34:19.724 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-base x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 5.4 M 2024-06-10T02:34:19.724 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-fuse x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 890 k 2024-06-10T02:34:19.724 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 145 k 2024-06-10T02:34:19.724 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 1.5 M 2024-06-10T02:34:19.724 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 142 k 2024-06-10T02:34:19.725 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 3.5 M 2024-06-10T02:34:19.725 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 7.4 M 2024-06-10T02:34:19.725 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-rook noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 49 k 2024-06-10T02:34:19.725 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-radosgw x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 10 M 2024-06-10T02:34:19.725 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-test x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 48 M 2024-06-10T02:34:19.725 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-volume noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 264 k 2024-06-10T02:34:19.725 INFO:teuthology.orchestra.run.smithi188.stdout: cephadm noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 763 k 2024-06-10T02:34:19.725 INFO:teuthology.orchestra.run.smithi188.stdout: cephfs-mirror x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 221 k 2024-06-10T02:34:19.725 INFO:teuthology.orchestra.run.smithi188.stdout: cephfs-shell x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 27 k 2024-06-10T02:34:19.725 INFO:teuthology.orchestra.run.smithi188.stdout: cephfs-top noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 18 k 2024-06-10T02:34:19.725 INFO:teuthology.orchestra.run.smithi188.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-06-10T02:34:19.725 INFO:teuthology.orchestra.run.smithi188.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-06-10T02:34:19.726 INFO:teuthology.orchestra.run.smithi188.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-06-10T02:34:19.726 INFO:teuthology.orchestra.run.smithi188.stdout: libcephfs-devel x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 32 k 2024-06-10T02:34:19.726 INFO:teuthology.orchestra.run.smithi188.stdout: libcephfs2 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 747 k 2024-06-10T02:34:19.726 INFO:teuthology.orchestra.run.smithi188.stdout: librados-devel x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 127 k 2024-06-10T02:34:19.726 INFO:teuthology.orchestra.run.smithi188.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-06-10T02:34:19.726 INFO:teuthology.orchestra.run.smithi188.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-06-10T02:34:19.726 INFO:teuthology.orchestra.run.smithi188.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-06-10T02:34:19.726 INFO:teuthology.orchestra.run.smithi188.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-06-10T02:34:19.726 INFO:teuthology.orchestra.run.smithi188.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-06-10T02:34:19.726 INFO:teuthology.orchestra.run.smithi188.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-06-10T02:34:19.726 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cephfs x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 162 k 2024-06-10T02:34:19.726 INFO:teuthology.orchestra.run.smithi188.stdout: python3-rados x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 321 k 2024-06-10T02:34:19.727 INFO:teuthology.orchestra.run.smithi188.stdout: python3-rbd x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 300 k 2024-06-10T02:34:19.727 INFO:teuthology.orchestra.run.smithi188.stdout: python3-rgw x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 99 k 2024-06-10T02:34:19.727 INFO:teuthology.orchestra.run.smithi188.stdout: rbd-fuse x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 85 k 2024-06-10T02:34:19.727 INFO:teuthology.orchestra.run.smithi188.stdout: rbd-mirror x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 3.1 M 2024-06-10T02:34:19.727 INFO:teuthology.orchestra.run.smithi188.stdout: rbd-nbd x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 171 k 2024-06-10T02:34:19.727 INFO:teuthology.orchestra.run.smithi188.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-06-10T02:34:19.727 INFO:teuthology.orchestra.run.smithi188.stdout:Upgrading: 2024-06-10T02:34:19.727 INFO:teuthology.orchestra.run.smithi188.stdout: NetworkManager x86_64 1:1.47.91-1.el9 baseos 2.3 M 2024-06-10T02:34:19.727 INFO:teuthology.orchestra.run.smithi188.stdout: NetworkManager-libnm x86_64 1:1.47.91-1.el9 baseos 1.8 M 2024-06-10T02:34:19.727 INFO:teuthology.orchestra.run.smithi188.stdout: NetworkManager-team x86_64 1:1.47.91-1.el9 baseos 41 k 2024-06-10T02:34:19.727 INFO:teuthology.orchestra.run.smithi188.stdout: NetworkManager-tui x86_64 1:1.47.91-1.el9 baseos 248 k 2024-06-10T02:34:19.727 INFO:teuthology.orchestra.run.smithi188.stdout: elfutils-debuginfod-client x86_64 0.191-4.el9 baseos 37 k 2024-06-10T02:34:19.728 INFO:teuthology.orchestra.run.smithi188.stdout: elfutils-libelf x86_64 0.191-4.el9 baseos 207 k 2024-06-10T02:34:19.728 INFO:teuthology.orchestra.run.smithi188.stdout: elfutils-libs x86_64 0.191-4.el9 baseos 260 k 2024-06-10T02:34:19.728 INFO:teuthology.orchestra.run.smithi188.stdout: librados2 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 3.4 M 2024-06-10T02:34:19.728 INFO:teuthology.orchestra.run.smithi188.stdout: librbd1 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 3.1 M 2024-06-10T02:34:19.728 INFO:teuthology.orchestra.run.smithi188.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-06-10T02:34:19.728 INFO:teuthology.orchestra.run.smithi188.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-06-10T02:34:19.728 INFO:teuthology.orchestra.run.smithi188.stdout:Installing dependencies: 2024-06-10T02:34:19.728 INFO:teuthology.orchestra.run.smithi188.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-06-10T02:34:19.728 INFO:teuthology.orchestra.run.smithi188.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-06-10T02:34:19.728 INFO:teuthology.orchestra.run.smithi188.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-06-10T02:34:19.728 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-common x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 21 M 2024-06-10T02:34:19.728 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 32 k 2024-06-10T02:34:19.728 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mds x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 2.4 M 2024-06-10T02:34:19.728 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 249 k 2024-06-10T02:34:19.729 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mon x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 4.7 M 2024-06-10T02:34:19.729 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-osd x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 17 M 2024-06-10T02:34:19.729 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 16 k 2024-06-10T02:34:19.729 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-selinux x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 25 k 2024-06-10T02:34:19.729 INFO:teuthology.orchestra.run.smithi188.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-06-10T02:34:19.729 INFO:teuthology.orchestra.run.smithi188.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-06-10T02:34:19.729 INFO:teuthology.orchestra.run.smithi188.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-06-10T02:34:19.729 INFO:teuthology.orchestra.run.smithi188.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-06-10T02:34:19.729 INFO:teuthology.orchestra.run.smithi188.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-06-10T02:34:19.729 INFO:teuthology.orchestra.run.smithi188.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-06-10T02:34:19.729 INFO:teuthology.orchestra.run.smithi188.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-06-10T02:34:19.729 INFO:teuthology.orchestra.run.smithi188.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-06-10T02:34:19.730 INFO:teuthology.orchestra.run.smithi188.stdout: libcephsqlite x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 163 k 2024-06-10T02:34:19.730 INFO:teuthology.orchestra.run.smithi188.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-06-10T02:34:19.730 INFO:teuthology.orchestra.run.smithi188.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-06-10T02:34:19.730 INFO:teuthology.orchestra.run.smithi188.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-06-10T02:34:19.730 INFO:teuthology.orchestra.run.smithi188.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-06-10T02:34:19.730 INFO:teuthology.orchestra.run.smithi188.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-06-10T02:34:19.730 INFO:teuthology.orchestra.run.smithi188.stdout: libradosstriper1 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 501 k 2024-06-10T02:34:19.730 INFO:teuthology.orchestra.run.smithi188.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-06-10T02:34:19.730 INFO:teuthology.orchestra.run.smithi188.stdout: librgw2 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 5.3 M 2024-06-10T02:34:19.730 INFO:teuthology.orchestra.run.smithi188.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-06-10T02:34:19.730 INFO:teuthology.orchestra.run.smithi188.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-06-10T02:34:19.730 INFO:teuthology.orchestra.run.smithi188.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-06-10T02:34:19.731 INFO:teuthology.orchestra.run.smithi188.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-06-10T02:34:19.731 INFO:teuthology.orchestra.run.smithi188.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-06-10T02:34:19.731 INFO:teuthology.orchestra.run.smithi188.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-06-10T02:34:19.731 INFO:teuthology.orchestra.run.smithi188.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-06-10T02:34:19.731 INFO:teuthology.orchestra.run.smithi188.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-06-10T02:34:19.731 INFO:teuthology.orchestra.run.smithi188.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-06-10T02:34:19.731 INFO:teuthology.orchestra.run.smithi188.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-06-10T02:34:19.731 INFO:teuthology.orchestra.run.smithi188.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-06-10T02:34:19.731 INFO:teuthology.orchestra.run.smithi188.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-06-10T02:34:19.731 INFO:teuthology.orchestra.run.smithi188.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-06-10T02:34:19.731 INFO:teuthology.orchestra.run.smithi188.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-06-10T02:34:19.731 INFO:teuthology.orchestra.run.smithi188.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-06-10T02:34:19.731 INFO:teuthology.orchestra.run.smithi188.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-06-10T02:34:19.731 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-06-10T02:34:19.732 INFO:teuthology.orchestra.run.smithi188.stdout: python3-ceph-argparse x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 45 k 2024-06-10T02:34:19.732 INFO:teuthology.orchestra.run.smithi188.stdout: python3-ceph-common x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 135 k 2024-06-10T02:34:19.732 INFO:teuthology.orchestra.run.smithi188.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-06-10T02:34:19.732 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-06-10T02:34:19.732 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-06-10T02:34:19.732 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-06-10T02:34:19.732 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-06-10T02:34:19.732 INFO:teuthology.orchestra.run.smithi188.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-06-10T02:34:19.732 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-06-10T02:34:19.732 INFO:teuthology.orchestra.run.smithi188.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-06-10T02:34:19.732 INFO:teuthology.orchestra.run.smithi188.stdout: python3-google-auth noarch 1:2.29.0-2.el9 epel 205 k 2024-06-10T02:34:19.732 INFO:teuthology.orchestra.run.smithi188.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-06-10T02:34:19.732 INFO:teuthology.orchestra.run.smithi188.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-06-10T02:34:19.732 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-06-10T02:34:19.733 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-06-10T02:34:19.733 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-06-10T02:34:19.733 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-06-10T02:34:19.733 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-06-10T02:34:19.733 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-06-10T02:34:19.733 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-06-10T02:34:19.733 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-06-10T02:34:19.733 INFO:teuthology.orchestra.run.smithi188.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-06-10T02:34:19.733 INFO:teuthology.orchestra.run.smithi188.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-06-10T02:34:19.733 INFO:teuthology.orchestra.run.smithi188.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-06-10T02:34:19.733 INFO:teuthology.orchestra.run.smithi188.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-06-10T02:34:19.733 INFO:teuthology.orchestra.run.smithi188.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-06-10T02:34:19.733 INFO:teuthology.orchestra.run.smithi188.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-06-10T02:34:19.733 INFO:teuthology.orchestra.run.smithi188.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-06-10T02:34:19.733 INFO:teuthology.orchestra.run.smithi188.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-06-10T02:34:19.734 INFO:teuthology.orchestra.run.smithi188.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-06-10T02:34:19.734 INFO:teuthology.orchestra.run.smithi188.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-06-10T02:34:19.734 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-06-10T02:34:19.734 INFO:teuthology.orchestra.run.smithi188.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-06-10T02:34:19.734 INFO:teuthology.orchestra.run.smithi188.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-06-10T02:34:19.734 INFO:teuthology.orchestra.run.smithi188.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-06-10T02:34:19.734 INFO:teuthology.orchestra.run.smithi188.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-06-10T02:34:19.734 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-06-10T02:34:19.734 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-06-10T02:34:19.734 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-06-10T02:34:19.734 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-06-10T02:34:19.734 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-06-10T02:34:19.734 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-06-10T02:34:19.735 INFO:teuthology.orchestra.run.smithi188.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-06-10T02:34:19.735 INFO:teuthology.orchestra.run.smithi188.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-06-10T02:34:19.735 INFO:teuthology.orchestra.run.smithi188.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-06-10T02:34:19.735 INFO:teuthology.orchestra.run.smithi188.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-06-10T02:34:19.735 INFO:teuthology.orchestra.run.smithi188.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-06-10T02:34:19.735 INFO:teuthology.orchestra.run.smithi188.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-06-10T02:34:19.735 INFO:teuthology.orchestra.run.smithi188.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-06-10T02:34:19.735 INFO:teuthology.orchestra.run.smithi188.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-06-10T02:34:19.735 INFO:teuthology.orchestra.run.smithi188.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-06-10T02:34:19.735 INFO:teuthology.orchestra.run.smithi188.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-06-10T02:34:19.735 INFO:teuthology.orchestra.run.smithi188.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-06-10T02:34:19.735 INFO:teuthology.orchestra.run.smithi188.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-06-10T02:34:19.735 INFO:teuthology.orchestra.run.smithi188.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-06-10T02:34:19.735 INFO:teuthology.orchestra.run.smithi188.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-06-10T02:34:19.735 INFO:teuthology.orchestra.run.smithi188.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-06-10T02:34:19.736 INFO:teuthology.orchestra.run.smithi188.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-06-10T02:34:19.736 INFO:teuthology.orchestra.run.smithi188.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-06-10T02:34:19.736 INFO:teuthology.orchestra.run.smithi188.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-06-10T02:34:19.736 INFO:teuthology.orchestra.run.smithi188.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-06-10T02:34:19.736 INFO:teuthology.orchestra.run.smithi188.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-06-10T02:34:19.736 INFO:teuthology.orchestra.run.smithi188.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-06-10T02:34:19.736 INFO:teuthology.orchestra.run.smithi188.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-06-10T02:34:19.736 INFO:teuthology.orchestra.run.smithi188.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-06-10T02:34:19.736 INFO:teuthology.orchestra.run.smithi188.stdout:Installing weak dependencies: 2024-06-10T02:34:19.736 INFO:teuthology.orchestra.run.smithi188.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-06-10T02:34:19.736 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-06-10T02:34:19.736 INFO:teuthology.orchestra.run.smithi188.stdout:Transaction Summary 2024-06-10T02:34:19.736 INFO:teuthology.orchestra.run.smithi188.stdout:======================================================================================= 2024-06-10T02:34:19.736 INFO:teuthology.orchestra.run.smithi188.stdout:Install 148 Packages 2024-06-10T02:34:19.737 INFO:teuthology.orchestra.run.smithi188.stdout:Upgrade 11 Packages 2024-06-10T02:34:19.737 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-06-10T02:34:19.737 INFO:teuthology.orchestra.run.smithi188.stdout:Total download size: 225 M 2024-06-10T02:34:19.737 INFO:teuthology.orchestra.run.smithi188.stdout:Downloading Packages: 2024-06-10T02:34:20.000 INFO:teuthology.orchestra.run.smithi052.stdout:(10/159): ceph-radosgw-19.0.0-2946.g3d89accf.el 8.0 MB/s | 10 MB 00:01 2024-06-10T02:34:20.167 INFO:teuthology.orchestra.run.smithi052.stdout:(11/159): ceph-osd-19.0.0-2946.g3d89accf.el9.x8 9.1 MB/s | 17 MB 00:01 2024-06-10T02:34:20.192 INFO:teuthology.orchestra.run.smithi052.stdout:(12/159): cephfs-mirror-19.0.0-2946.g3d89accf.e 1.1 MB/s | 221 kB 00:00 2024-06-10T02:34:20.243 INFO:teuthology.orchestra.run.smithi052.stdout:(13/159): cephfs-shell-19.0.0-2946.g3d89accf.el 360 kB/s | 27 kB 00:00 2024-06-10T02:34:20.268 INFO:teuthology.orchestra.run.smithi052.stdout:(14/159): libcephfs-devel-19.0.0-2946.g3d89accf 421 kB/s | 32 kB 00:00 2024-06-10T02:34:20.277 INFO:teuthology.orchestra.run.smithi188.stdout:(1/159): ceph-19.0.0-2946.g3d89accf.el9.x86_64. 38 kB/s | 6.6 kB 00:00 2024-06-10T02:34:20.343 INFO:teuthology.orchestra.run.smithi052.stdout:(15/159): libcephsqlite-19.0.0-2946.g3d89accf.e 2.1 MB/s | 163 kB 00:00 2024-06-10T02:34:20.502 INFO:teuthology.orchestra.run.smithi052.stdout:(16/159): libcephfs2-19.0.0-2946.g3d89accf.el9. 2.8 MB/s | 747 kB 00:00 2024-06-10T02:34:20.518 INFO:teuthology.orchestra.run.smithi052.stdout:(17/159): librados-devel-19.0.0-2946.g3d89accf. 723 kB/s | 127 kB 00:00 2024-06-10T02:34:20.603 INFO:teuthology.orchestra.run.smithi052.stdout:(18/159): libradosstriper1-19.0.0-2946.g3d89acc 4.9 MB/s | 501 kB 00:00 2024-06-10T02:34:20.703 INFO:teuthology.orchestra.run.smithi052.stdout:(19/159): python3-ceph-argparse-19.0.0-2946.g3d 452 kB/s | 45 kB 00:00 2024-06-10T02:34:20.791 INFO:teuthology.orchestra.run.smithi120.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-06-10T02:34:20.792 INFO:teuthology.orchestra.run.smithi120.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-06-10T02:34:20.803 INFO:teuthology.orchestra.run.smithi120.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-06-10T02:34:20.804 INFO:teuthology.orchestra.run.smithi120.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-06-10T02:34:20.805 INFO:teuthology.orchestra.run.smithi120.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-06-10T02:34:20.806 INFO:teuthology.orchestra.run.smithi120.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-06-10T02:34:20.807 INFO:teuthology.orchestra.run.smithi120.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-06-10T02:34:20.810 INFO:teuthology.orchestra.run.smithi120.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-06-10T02:34:20.811 INFO:teuthology.orchestra.run.smithi120.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-06-10T02:34:20.828 INFO:teuthology.orchestra.run.smithi052.stdout:(20/159): python3-ceph-common-19.0.0-2946.g3d89 1.1 MB/s | 135 kB 00:00 2024-06-10T02:34:20.844 INFO:teuthology.orchestra.run.smithi188.stdout:(2/159): ceph-fuse-19.0.0-2946.g3d89accf.el9.x8 1.5 MB/s | 890 kB 00:00 2024-06-10T02:34:20.874 INFO:teuthology.orchestra.run.smithi120.stdout:Dependencies resolved. 2024-06-10T02:34:20.903 INFO:teuthology.orchestra.run.smithi120.stdout:======================================================================================= 2024-06-10T02:34:20.903 INFO:teuthology.orchestra.run.smithi120.stdout: Package Arch Version Repository Size 2024-06-10T02:34:20.903 INFO:teuthology.orchestra.run.smithi120.stdout:======================================================================================= 2024-06-10T02:34:20.904 INFO:teuthology.orchestra.run.smithi120.stdout:Installing: 2024-06-10T02:34:20.904 INFO:teuthology.orchestra.run.smithi120.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-06-10T02:34:20.904 INFO:teuthology.orchestra.run.smithi120.stdout: ceph x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 6.6 k 2024-06-10T02:34:20.904 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-base x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 5.4 M 2024-06-10T02:34:20.904 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-fuse x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 890 k 2024-06-10T02:34:20.904 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 145 k 2024-06-10T02:34:20.904 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mgr x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 1.5 M 2024-06-10T02:34:20.904 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 142 k 2024-06-10T02:34:20.904 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 3.5 M 2024-06-10T02:34:20.904 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 7.4 M 2024-06-10T02:34:20.904 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mgr-rook noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 49 k 2024-06-10T02:34:20.904 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-radosgw x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 10 M 2024-06-10T02:34:20.904 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-test x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 48 M 2024-06-10T02:34:20.905 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-volume noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 264 k 2024-06-10T02:34:20.905 INFO:teuthology.orchestra.run.smithi120.stdout: cephadm noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 763 k 2024-06-10T02:34:20.905 INFO:teuthology.orchestra.run.smithi120.stdout: cephfs-mirror x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 221 k 2024-06-10T02:34:20.905 INFO:teuthology.orchestra.run.smithi120.stdout: cephfs-shell x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 27 k 2024-06-10T02:34:20.905 INFO:teuthology.orchestra.run.smithi120.stdout: cephfs-top noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 18 k 2024-06-10T02:34:20.905 INFO:teuthology.orchestra.run.smithi120.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-06-10T02:34:20.905 INFO:teuthology.orchestra.run.smithi120.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-06-10T02:34:20.905 INFO:teuthology.orchestra.run.smithi120.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-06-10T02:34:20.905 INFO:teuthology.orchestra.run.smithi120.stdout: libcephfs-devel x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 32 k 2024-06-10T02:34:20.905 INFO:teuthology.orchestra.run.smithi120.stdout: libcephfs2 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 747 k 2024-06-10T02:34:20.905 INFO:teuthology.orchestra.run.smithi120.stdout: librados-devel x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 127 k 2024-06-10T02:34:20.905 INFO:teuthology.orchestra.run.smithi120.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-06-10T02:34:20.905 INFO:teuthology.orchestra.run.smithi120.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-06-10T02:34:20.906 INFO:teuthology.orchestra.run.smithi120.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-06-10T02:34:20.906 INFO:teuthology.orchestra.run.smithi120.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-06-10T02:34:20.906 INFO:teuthology.orchestra.run.smithi120.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-06-10T02:34:20.906 INFO:teuthology.orchestra.run.smithi120.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-06-10T02:34:20.906 INFO:teuthology.orchestra.run.smithi120.stdout: python3-cephfs x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 162 k 2024-06-10T02:34:20.906 INFO:teuthology.orchestra.run.smithi120.stdout: python3-rados x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 321 k 2024-06-10T02:34:20.906 INFO:teuthology.orchestra.run.smithi120.stdout: python3-rbd x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 300 k 2024-06-10T02:34:20.906 INFO:teuthology.orchestra.run.smithi120.stdout: python3-rgw x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 99 k 2024-06-10T02:34:20.906 INFO:teuthology.orchestra.run.smithi120.stdout: rbd-fuse x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 85 k 2024-06-10T02:34:20.906 INFO:teuthology.orchestra.run.smithi120.stdout: rbd-mirror x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 3.1 M 2024-06-10T02:34:20.906 INFO:teuthology.orchestra.run.smithi120.stdout: rbd-nbd x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 171 k 2024-06-10T02:34:20.906 INFO:teuthology.orchestra.run.smithi120.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-06-10T02:34:20.907 INFO:teuthology.orchestra.run.smithi120.stdout:Upgrading: 2024-06-10T02:34:20.907 INFO:teuthology.orchestra.run.smithi120.stdout: NetworkManager x86_64 1:1.47.91-1.el9 baseos 2.3 M 2024-06-10T02:34:20.907 INFO:teuthology.orchestra.run.smithi120.stdout: NetworkManager-libnm x86_64 1:1.47.91-1.el9 baseos 1.8 M 2024-06-10T02:34:20.907 INFO:teuthology.orchestra.run.smithi120.stdout: NetworkManager-team x86_64 1:1.47.91-1.el9 baseos 41 k 2024-06-10T02:34:20.907 INFO:teuthology.orchestra.run.smithi120.stdout: NetworkManager-tui x86_64 1:1.47.91-1.el9 baseos 248 k 2024-06-10T02:34:20.907 INFO:teuthology.orchestra.run.smithi120.stdout: elfutils-debuginfod-client x86_64 0.191-4.el9 baseos 37 k 2024-06-10T02:34:20.907 INFO:teuthology.orchestra.run.smithi120.stdout: elfutils-libelf x86_64 0.191-4.el9 baseos 207 k 2024-06-10T02:34:20.907 INFO:teuthology.orchestra.run.smithi120.stdout: elfutils-libs x86_64 0.191-4.el9 baseos 260 k 2024-06-10T02:34:20.907 INFO:teuthology.orchestra.run.smithi120.stdout: librados2 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 3.4 M 2024-06-10T02:34:20.907 INFO:teuthology.orchestra.run.smithi120.stdout: librbd1 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 3.1 M 2024-06-10T02:34:20.907 INFO:teuthology.orchestra.run.smithi120.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-06-10T02:34:20.907 INFO:teuthology.orchestra.run.smithi120.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-06-10T02:34:20.907 INFO:teuthology.orchestra.run.smithi120.stdout:Installing dependencies: 2024-06-10T02:34:20.907 INFO:teuthology.orchestra.run.smithi120.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-06-10T02:34:20.908 INFO:teuthology.orchestra.run.smithi120.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-06-10T02:34:20.908 INFO:teuthology.orchestra.run.smithi120.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-06-10T02:34:20.908 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-common x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 21 M 2024-06-10T02:34:20.908 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 32 k 2024-06-10T02:34:20.908 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mds x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 2.4 M 2024-06-10T02:34:20.908 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 249 k 2024-06-10T02:34:20.908 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mon x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 4.7 M 2024-06-10T02:34:20.908 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-osd x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 17 M 2024-06-10T02:34:20.908 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 16 k 2024-06-10T02:34:20.908 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-selinux x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 25 k 2024-06-10T02:34:20.908 INFO:teuthology.orchestra.run.smithi120.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-06-10T02:34:20.908 INFO:teuthology.orchestra.run.smithi120.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-06-10T02:34:20.908 INFO:teuthology.orchestra.run.smithi120.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-06-10T02:34:20.909 INFO:teuthology.orchestra.run.smithi120.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-06-10T02:34:20.909 INFO:teuthology.orchestra.run.smithi120.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-06-10T02:34:20.909 INFO:teuthology.orchestra.run.smithi120.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-06-10T02:34:20.909 INFO:teuthology.orchestra.run.smithi120.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-06-10T02:34:20.909 INFO:teuthology.orchestra.run.smithi120.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-06-10T02:34:20.909 INFO:teuthology.orchestra.run.smithi120.stdout: libcephsqlite x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 163 k 2024-06-10T02:34:20.909 INFO:teuthology.orchestra.run.smithi120.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-06-10T02:34:20.909 INFO:teuthology.orchestra.run.smithi120.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-06-10T02:34:20.909 INFO:teuthology.orchestra.run.smithi120.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-06-10T02:34:20.909 INFO:teuthology.orchestra.run.smithi120.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-06-10T02:34:20.909 INFO:teuthology.orchestra.run.smithi120.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-06-10T02:34:20.909 INFO:teuthology.orchestra.run.smithi120.stdout: libradosstriper1 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 501 k 2024-06-10T02:34:20.910 INFO:teuthology.orchestra.run.smithi120.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-06-10T02:34:20.910 INFO:teuthology.orchestra.run.smithi120.stdout: librgw2 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 5.3 M 2024-06-10T02:34:20.910 INFO:teuthology.orchestra.run.smithi120.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-06-10T02:34:20.910 INFO:teuthology.orchestra.run.smithi120.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-06-10T02:34:20.910 INFO:teuthology.orchestra.run.smithi120.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-06-10T02:34:20.910 INFO:teuthology.orchestra.run.smithi120.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-06-10T02:34:20.910 INFO:teuthology.orchestra.run.smithi120.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-06-10T02:34:20.910 INFO:teuthology.orchestra.run.smithi120.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-06-10T02:34:20.910 INFO:teuthology.orchestra.run.smithi120.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-06-10T02:34:20.910 INFO:teuthology.orchestra.run.smithi120.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-06-10T02:34:20.910 INFO:teuthology.orchestra.run.smithi120.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-06-10T02:34:20.910 INFO:teuthology.orchestra.run.smithi120.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-06-10T02:34:20.911 INFO:teuthology.orchestra.run.smithi120.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-06-10T02:34:20.911 INFO:teuthology.orchestra.run.smithi120.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-06-10T02:34:20.911 INFO:teuthology.orchestra.run.smithi120.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-06-10T02:34:20.911 INFO:teuthology.orchestra.run.smithi120.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-06-10T02:34:20.911 INFO:teuthology.orchestra.run.smithi120.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-06-10T02:34:20.911 INFO:teuthology.orchestra.run.smithi120.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-06-10T02:34:20.911 INFO:teuthology.orchestra.run.smithi120.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-06-10T02:34:20.911 INFO:teuthology.orchestra.run.smithi120.stdout: python3-ceph-argparse x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 45 k 2024-06-10T02:34:20.911 INFO:teuthology.orchestra.run.smithi120.stdout: python3-ceph-common x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 135 k 2024-06-10T02:34:20.911 INFO:teuthology.orchestra.run.smithi120.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-06-10T02:34:20.911 INFO:teuthology.orchestra.run.smithi120.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-06-10T02:34:20.911 INFO:teuthology.orchestra.run.smithi120.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-06-10T02:34:20.911 INFO:teuthology.orchestra.run.smithi120.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-06-10T02:34:20.912 INFO:teuthology.orchestra.run.smithi120.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-06-10T02:34:20.912 INFO:teuthology.orchestra.run.smithi120.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-06-10T02:34:20.912 INFO:teuthology.orchestra.run.smithi120.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-06-10T02:34:20.912 INFO:teuthology.orchestra.run.smithi120.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-06-10T02:34:20.912 INFO:teuthology.orchestra.run.smithi120.stdout: python3-google-auth noarch 1:2.29.0-2.el9 epel 205 k 2024-06-10T02:34:20.912 INFO:teuthology.orchestra.run.smithi120.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-06-10T02:34:20.912 INFO:teuthology.orchestra.run.smithi120.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-06-10T02:34:20.912 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-06-10T02:34:20.912 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-06-10T02:34:20.912 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-06-10T02:34:20.912 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-06-10T02:34:20.912 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-06-10T02:34:20.912 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-06-10T02:34:20.912 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-06-10T02:34:20.913 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-06-10T02:34:20.913 INFO:teuthology.orchestra.run.smithi120.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-06-10T02:34:20.913 INFO:teuthology.orchestra.run.smithi120.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-06-10T02:34:20.913 INFO:teuthology.orchestra.run.smithi120.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-06-10T02:34:20.913 INFO:teuthology.orchestra.run.smithi120.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-06-10T02:34:20.913 INFO:teuthology.orchestra.run.smithi120.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-06-10T02:34:20.913 INFO:teuthology.orchestra.run.smithi120.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-06-10T02:34:20.913 INFO:teuthology.orchestra.run.smithi120.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-06-10T02:34:20.913 INFO:teuthology.orchestra.run.smithi120.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-06-10T02:34:20.913 INFO:teuthology.orchestra.run.smithi120.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-06-10T02:34:20.913 INFO:teuthology.orchestra.run.smithi120.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-06-10T02:34:20.913 INFO:teuthology.orchestra.run.smithi120.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-06-10T02:34:20.913 INFO:teuthology.orchestra.run.smithi120.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-06-10T02:34:20.913 INFO:teuthology.orchestra.run.smithi120.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-06-10T02:34:20.913 INFO:teuthology.orchestra.run.smithi120.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-06-10T02:34:20.914 INFO:teuthology.orchestra.run.smithi120.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-06-10T02:34:20.914 INFO:teuthology.orchestra.run.smithi120.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-06-10T02:34:20.914 INFO:teuthology.orchestra.run.smithi120.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-06-10T02:34:20.914 INFO:teuthology.orchestra.run.smithi120.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-06-10T02:34:20.914 INFO:teuthology.orchestra.run.smithi120.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-06-10T02:34:20.914 INFO:teuthology.orchestra.run.smithi120.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-06-10T02:34:20.914 INFO:teuthology.orchestra.run.smithi120.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-06-10T02:34:20.914 INFO:teuthology.orchestra.run.smithi120.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-06-10T02:34:20.914 INFO:teuthology.orchestra.run.smithi120.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-06-10T02:34:20.914 INFO:teuthology.orchestra.run.smithi120.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-06-10T02:34:20.914 INFO:teuthology.orchestra.run.smithi120.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-06-10T02:34:20.914 INFO:teuthology.orchestra.run.smithi120.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-06-10T02:34:20.914 INFO:teuthology.orchestra.run.smithi120.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-06-10T02:34:20.915 INFO:teuthology.orchestra.run.smithi120.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-06-10T02:34:20.915 INFO:teuthology.orchestra.run.smithi120.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-06-10T02:34:20.915 INFO:teuthology.orchestra.run.smithi120.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-06-10T02:34:20.915 INFO:teuthology.orchestra.run.smithi120.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-06-10T02:34:20.915 INFO:teuthology.orchestra.run.smithi120.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-06-10T02:34:20.915 INFO:teuthology.orchestra.run.smithi120.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-06-10T02:34:20.915 INFO:teuthology.orchestra.run.smithi120.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-06-10T02:34:20.915 INFO:teuthology.orchestra.run.smithi120.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-06-10T02:34:20.915 INFO:teuthology.orchestra.run.smithi120.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-06-10T02:34:20.915 INFO:teuthology.orchestra.run.smithi120.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-06-10T02:34:20.915 INFO:teuthology.orchestra.run.smithi120.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-06-10T02:34:20.915 INFO:teuthology.orchestra.run.smithi120.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-06-10T02:34:20.915 INFO:teuthology.orchestra.run.smithi120.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-06-10T02:34:20.915 INFO:teuthology.orchestra.run.smithi120.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-06-10T02:34:20.916 INFO:teuthology.orchestra.run.smithi120.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-06-10T02:34:20.916 INFO:teuthology.orchestra.run.smithi120.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-06-10T02:34:20.916 INFO:teuthology.orchestra.run.smithi120.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-06-10T02:34:20.916 INFO:teuthology.orchestra.run.smithi120.stdout:Installing weak dependencies: 2024-06-10T02:34:20.916 INFO:teuthology.orchestra.run.smithi120.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-06-10T02:34:20.916 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-06-10T02:34:20.916 INFO:teuthology.orchestra.run.smithi120.stdout:Transaction Summary 2024-06-10T02:34:20.916 INFO:teuthology.orchestra.run.smithi120.stdout:======================================================================================= 2024-06-10T02:34:20.916 INFO:teuthology.orchestra.run.smithi120.stdout:Install 148 Packages 2024-06-10T02:34:20.916 INFO:teuthology.orchestra.run.smithi120.stdout:Upgrade 11 Packages 2024-06-10T02:34:20.916 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-06-10T02:34:20.917 INFO:teuthology.orchestra.run.smithi120.stdout:Total download size: 225 M 2024-06-10T02:34:20.917 INFO:teuthology.orchestra.run.smithi120.stdout:Downloading Packages: 2024-06-10T02:34:20.928 INFO:teuthology.orchestra.run.smithi052.stdout:(21/159): python3-cephfs-19.0.0-2946.g3d89accf. 1.6 MB/s | 162 kB 00:00 2024-06-10T02:34:21.003 INFO:teuthology.orchestra.run.smithi188.stdout:(3/159): ceph-immutable-object-cache-19.0.0-294 914 kB/s | 145 kB 00:00 2024-06-10T02:34:21.045 INFO:teuthology.orchestra.run.smithi052.stdout:(22/159): python3-rados-19.0.0-2946.g3d89accf.e 2.7 MB/s | 321 kB 00:00 2024-06-10T02:34:21.162 INFO:teuthology.orchestra.run.smithi052.stdout:(23/159): python3-rbd-19.0.0-2946.g3d89accf.el9 2.5 MB/s | 300 kB 00:00 2024-06-10T02:34:21.263 INFO:teuthology.orchestra.run.smithi052.stdout:(24/159): librgw2-19.0.0-2946.g3d89accf.el9.x86 7.2 MB/s | 5.3 MB 00:00 2024-06-10T02:34:21.288 INFO:teuthology.orchestra.run.smithi052.stdout:(25/159): python3-rgw-19.0.0-2946.g3d89accf.el9 794 kB/s | 99 kB 00:00 2024-06-10T02:34:21.321 INFO:teuthology.orchestra.run.smithi052.stdout:(26/159): rbd-fuse-19.0.0-2946.g3d89accf.el9.x8 1.4 MB/s | 85 kB 00:00 2024-06-10T02:34:21.413 INFO:teuthology.orchestra.run.smithi052.stdout:(27/159): rbd-nbd-19.0.0-2946.g3d89accf.el9.x86 1.8 MB/s | 171 kB 00:00 2024-06-10T02:34:21.505 INFO:teuthology.orchestra.run.smithi052.stdout:(28/159): ceph-grafana-dashboards-19.0.0-2946.g 347 kB/s | 32 kB 00:00 2024-06-10T02:34:21.586 INFO:teuthology.orchestra.run.smithi188.stdout:(4/159): ceph-base-19.0.0-2946.g3d89accf.el9.x8 3.6 MB/s | 5.4 MB 00:01 2024-06-10T02:34:21.589 INFO:teuthology.orchestra.run.smithi052.stdout:(29/159): ceph-mgr-cephadm-19.0.0-2946.g3d89acc 1.7 MB/s | 142 kB 00:00 2024-06-10T02:34:21.722 INFO:teuthology.orchestra.run.smithi052.stdout:(30/159): rbd-mirror-19.0.0-2946.g3d89accf.el9. 7.1 MB/s | 3.1 MB 00:00 2024-06-10T02:34:21.953 INFO:teuthology.orchestra.run.smithi188.stdout:(5/159): ceph-mds-19.0.0-2946.g3d89accf.el9.x86 2.5 MB/s | 2.4 MB 00:00 2024-06-10T02:34:22.020 INFO:teuthology.orchestra.run.smithi188.stdout:(6/159): ceph-mgr-19.0.0-2946.g3d89accf.el9.x86 3.4 MB/s | 1.5 MB 00:00 2024-06-10T02:34:22.300 INFO:teuthology.orchestra.run.smithi120.stdout:(1/159): ceph-19.0.0-2946.g3d89accf.el9.x86_64. 12 kB/s | 6.6 kB 00:00 2024-06-10T02:34:22.373 INFO:teuthology.orchestra.run.smithi052.stdout:(31/159): ceph-mgr-dashboard-19.0.0-2946.g3d89a 4.5 MB/s | 3.5 MB 00:00 2024-06-10T02:34:22.473 INFO:teuthology.orchestra.run.smithi052.stdout:(32/159): ceph-mgr-modules-core-19.0.0-2946.g3d 2.4 MB/s | 249 kB 00:00 2024-06-10T02:34:22.565 INFO:teuthology.orchestra.run.smithi052.stdout:(33/159): ceph-mgr-rook-19.0.0-2946.g3d89accf.e 538 kB/s | 49 kB 00:00 2024-06-10T02:34:22.657 INFO:teuthology.orchestra.run.smithi052.stdout:(34/159): ceph-prometheus-alerts-19.0.0-2946.g3 179 kB/s | 16 kB 00:00 2024-06-10T02:34:22.757 INFO:teuthology.orchestra.run.smithi052.stdout:(35/159): ceph-volume-19.0.0-2946.g3d89accf.el9 2.6 MB/s | 264 kB 00:00 2024-06-10T02:34:22.866 INFO:teuthology.orchestra.run.smithi052.stdout:(36/159): ceph-mgr-diskprediction-local-19.0.0- 6.5 MB/s | 7.4 MB 00:01 2024-06-10T02:34:22.950 INFO:teuthology.orchestra.run.smithi052.stdout:(37/159): cephfs-top-19.0.0-2946.g3d89accf.el9. 218 kB/s | 18 kB 00:00 2024-06-10T02:34:22.984 INFO:teuthology.orchestra.run.smithi052.stdout:(38/159): cephadm-19.0.0-2946.g3d89accf.el9.noa 3.3 MB/s | 763 kB 00:00 2024-06-10T02:34:23.044 INFO:teuthology.orchestra.run.smithi052.stdout:[MIRROR] fuse-2.9.9-15.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/fuse-2.9.9-15.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-10T02:34:23.050 INFO:teuthology.orchestra.run.smithi052.stdout:[MIRROR] libgfortran-11.4.1-3.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libgfortran-11.4.1-3.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-10T02:34:23.275 INFO:teuthology.orchestra.run.smithi120.stdout:(2/159): ceph-fuse-19.0.0-2946.g3d89accf.el9.x8 913 kB/s | 890 kB 00:00 2024-06-10T02:34:23.359 INFO:teuthology.orchestra.run.smithi120.stdout:(3/159): ceph-immutable-object-cache-19.0.0-294 1.7 MB/s | 145 kB 00:00 2024-06-10T02:34:23.522 INFO:teuthology.orchestra.run.smithi188.stdout:(7/159): ceph-common-19.0.0-2946.g3d89accf.el9. 6.1 MB/s | 21 MB 00:03 2024-06-10T02:34:23.567 INFO:teuthology.orchestra.run.smithi052.stdout:(39/159): fuse-2.9.9-15.el9.x86_64.rpm 130 kB/s | 80 kB 00:00 2024-06-10T02:34:23.568 INFO:teuthology.orchestra.run.smithi052.stdout:[MIRROR] libquadmath-11.4.1-3.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libquadmath-11.4.1-3.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-10T02:34:23.587 INFO:teuthology.orchestra.run.smithi188.stdout:(8/159): ceph-mon-19.0.0-2946.g3d89accf.el9.x86 2.9 MB/s | 4.7 MB 00:01 2024-06-10T02:34:23.637 INFO:teuthology.orchestra.run.smithi188.stdout:(9/159): ceph-selinux-19.0.0-2946.g3d89accf.el9 507 kB/s | 25 kB 00:00 2024-06-10T02:34:23.684 INFO:teuthology.orchestra.run.smithi052.stdout:(40/159): libgfortran-11.4.1-3.el9.x86_64.rpm 1.1 MB/s | 803 kB 00:00 2024-06-10T02:34:23.776 INFO:teuthology.orchestra.run.smithi052.stdout:(41/159): libquadmath-11.4.1-3.el9.x86_64.rpm 937 kB/s | 195 kB 00:00 2024-06-10T02:34:23.901 INFO:teuthology.orchestra.run.smithi052.stdout:(42/159): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.1 MB/s | 253 kB 00:00 2024-06-10T02:34:24.060 INFO:teuthology.orchestra.run.smithi052.stdout:(43/159): python3-cryptography-36.0.1-4.el9.x86 4.3 MB/s | 1.2 MB 00:00 2024-06-10T02:34:24.085 INFO:teuthology.orchestra.run.smithi052.stdout:(44/159): python3-ply-3.11-14.el9.noarch.rpm 580 kB/s | 106 kB 00:00 2024-06-10T02:34:24.227 INFO:teuthology.orchestra.run.smithi052.stdout:(45/159): python3-pycparser-2.20-6.el9.noarch.r 810 kB/s | 135 kB 00:00 2024-06-10T02:34:24.252 INFO:teuthology.orchestra.run.smithi052.stdout:(46/159): python3-requests-2.25.1-8.el9.noarch. 748 kB/s | 125 kB 00:00 2024-06-10T02:34:24.253 INFO:teuthology.orchestra.run.smithi052.stdout:[MIRROR] automake-1.16.2-8.el9.noarch.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/automake-1.16.2-8.el9.noarch.rpm (IP: 8.43.85.72) 2024-06-10T02:34:24.411 INFO:teuthology.orchestra.run.smithi052.stdout:(47/159): python3-urllib3-1.26.5-5.el9.noarch.r 1.1 MB/s | 215 kB 00:00 2024-06-10T02:34:24.412 INFO:teuthology.orchestra.run.smithi052.stdout:[MIRROR] bison-3.7.4-5.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/bison-3.7.4-5.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-10T02:34:24.521 INFO:teuthology.orchestra.run.smithi188.stdout:(10/159): ceph-radosgw-19.0.0-2946.g3d89accf.el 10 MB/s | 10 MB 00:00 2024-06-10T02:34:24.533 INFO:teuthology.orchestra.run.smithi052.stdout:(48/159): automake-1.16.2-8.el9.noarch.rpm 2.4 MB/s | 689 kB 00:00 2024-06-10T02:34:24.534 INFO:teuthology.orchestra.run.smithi052.stdout:[MIRROR] boost-program-options-1.75.0-8.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/boost-program-options-1.75.0-8.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-10T02:34:24.587 INFO:teuthology.orchestra.run.smithi052.stdout:(49/159): boost-program-options-1.75.0-8.el9.x8 2.0 MB/s | 107 kB 00:00 2024-06-10T02:34:24.604 INFO:teuthology.orchestra.run.smithi188.stdout:(11/159): cephfs-mirror-19.0.0-2946.g3d89accf.e 2.6 MB/s | 221 kB 00:00 2024-06-10T02:34:24.617 INFO:teuthology.orchestra.run.smithi120.stdout:(4/159): ceph-mds-19.0.0-2946.g3d89accf.el9.x86 1.9 MB/s | 2.4 MB 00:01 2024-06-10T02:34:24.654 INFO:teuthology.orchestra.run.smithi188.stdout:(12/159): cephfs-shell-19.0.0-2946.g3d89accf.el 544 kB/s | 27 kB 00:00 2024-06-10T02:34:24.695 INFO:teuthology.orchestra.run.smithi052.stdout:(50/159): elfutils-libelf-devel-0.191-4.el9.x86 200 kB/s | 22 kB 00:00 2024-06-10T02:34:24.722 INFO:teuthology.orchestra.run.smithi188.stdout:(13/159): libcephfs-devel-19.0.0-2946.g3d89accf 475 kB/s | 32 kB 00:00 2024-06-10T02:34:24.754 INFO:teuthology.orchestra.run.smithi052.stdout:(51/159): bison-3.7.4-5.el9.x86_64.rpm 2.7 MB/s | 943 kB 00:00 2024-06-10T02:34:24.776 INFO:teuthology.orchestra.run.smithi120.stdout:(5/159): ceph-base-19.0.0-2946.g3d89accf.el9.x8 1.8 MB/s | 5.4 MB 00:03 2024-06-10T02:34:24.804 INFO:teuthology.orchestra.run.smithi052.stdout:(52/159): flexiblas-3.0.4-8.el9.x86_64.rpm 656 kB/s | 33 kB 00:00 2024-06-10T02:34:24.822 INFO:teuthology.orchestra.run.smithi188.stdout:(14/159): libcephfs2-19.0.0-2946.g3d89accf.el9. 7.3 MB/s | 747 kB 00:00 2024-06-10T02:34:24.838 INFO:teuthology.orchestra.run.smithi052.stdout:(53/159): flex-2.6.4-9.el9.x86_64.rpm 2.2 MB/s | 313 kB 00:00 2024-06-10T02:34:24.897 INFO:teuthology.orchestra.run.smithi188.stdout:(15/159): libcephsqlite-19.0.0-2946.g3d89accf.e 2.1 MB/s | 163 kB 00:00 2024-06-10T02:34:24.964 INFO:teuthology.orchestra.run.smithi188.stdout:(16/159): librados-devel-19.0.0-2946.g3d89accf. 1.9 MB/s | 127 kB 00:00 2024-06-10T02:34:24.980 INFO:teuthology.orchestra.run.smithi052.stdout:(54/159): flexiblas-openblas-openmp-3.0.4-8.el9 128 kB/s | 18 kB 00:00 2024-06-10T02:34:25.030 INFO:teuthology.orchestra.run.smithi052.stdout:(55/159): libaio-devel-0.3.111-13.el9.x86_64.rp 223 kB/s | 11 kB 00:00 2024-06-10T02:34:25.048 INFO:teuthology.orchestra.run.smithi188.stdout:(17/159): libradosstriper1-19.0.0-2946.g3d89acc 5.9 MB/s | 501 kB 00:00 2024-06-10T02:34:25.147 INFO:teuthology.orchestra.run.smithi052.stdout:(56/159): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.3 MB/s | 160 kB 00:00 2024-06-10T02:34:25.206 INFO:teuthology.orchestra.run.smithi052.stdout:(57/159): librabbitmq-0.11.0-7.el9.x86_64.rpm 777 kB/s | 45 kB 00:00 2024-06-10T02:34:25.276 INFO:teuthology.orchestra.run.smithi120.stdout:(6/159): ceph-mgr-19.0.0-2946.g3d89accf.el9.x86 2.3 MB/s | 1.5 MB 00:00 2024-06-10T02:34:25.340 INFO:teuthology.orchestra.run.smithi052.stdout:(58/159): flexiblas-netlib-3.0.4-8.el9.x86_64.r 5.6 MB/s | 3.0 MB 00:00 2024-06-10T02:34:25.416 INFO:teuthology.orchestra.run.smithi052.stdout:(59/159): librdkafka-1.6.1-102.el9.x86_64.rpm 3.1 MB/s | 662 kB 00:00 2024-06-10T02:34:25.473 INFO:teuthology.orchestra.run.smithi052.stdout:(60/159): libtool-2.4.6-45.el9.x86_64.rpm 4.3 MB/s | 582 kB 00:00 2024-06-10T02:34:25.499 INFO:teuthology.orchestra.run.smithi052.stdout:(61/159): libuuid-devel-2.37.4-18.el9.x86_64.rp 305 kB/s | 25 kB 00:00 2024-06-10T02:34:25.541 INFO:teuthology.orchestra.run.smithi052.stdout:(62/159): libzstd-devel-1.5.1-2.el9.x86_64.rpm 709 kB/s | 47 kB 00:00 2024-06-10T02:34:25.590 INFO:teuthology.orchestra.run.smithi188.stdout:(18/159): librgw2-19.0.0-2946.g3d89accf.el9.x86 9.8 MB/s | 5.3 MB 00:00 2024-06-10T02:34:25.599 INFO:teuthology.orchestra.run.smithi052.stdout:(63/159): lua-5.4.4-4.el9.x86_64.rpm 3.2 MB/s | 188 kB 00:00 2024-06-10T02:34:25.625 INFO:teuthology.orchestra.run.smithi052.stdout:(64/159): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.3 MB/s | 292 kB 00:00 2024-06-10T02:34:25.648 INFO:teuthology.orchestra.run.smithi188.stdout:(19/159): python3-ceph-argparse-19.0.0-2946.g3d 778 kB/s | 45 kB 00:00 2024-06-10T02:34:25.650 INFO:teuthology.orchestra.run.smithi052.stdout:(65/159): openblas-0.3.21-2.el9.x86_64.rpm 699 kB/s | 35 kB 00:00 2024-06-10T02:34:25.724 INFO:teuthology.orchestra.run.smithi188.stdout:(20/159): python3-ceph-common-19.0.0-2946.g3d89 1.8 MB/s | 135 kB 00:00 2024-06-10T02:34:25.815 INFO:teuthology.orchestra.run.smithi188.stdout:(21/159): python3-cephfs-19.0.0-2946.g3d89accf. 1.7 MB/s | 162 kB 00:00 2024-06-10T02:34:25.891 INFO:teuthology.orchestra.run.smithi188.stdout:(22/159): python3-rados-19.0.0-2946.g3d89accf.e 4.2 MB/s | 321 kB 00:00 2024-06-10T02:34:25.966 INFO:teuthology.orchestra.run.smithi188.stdout:(23/159): python3-rbd-19.0.0-2946.g3d89accf.el9 3.9 MB/s | 300 kB 00:00 2024-06-10T02:34:26.033 INFO:teuthology.orchestra.run.smithi188.stdout:(24/159): python3-rgw-19.0.0-2946.g3d89accf.el9 1.5 MB/s | 99 kB 00:00 2024-06-10T02:34:26.100 INFO:teuthology.orchestra.run.smithi188.stdout:(25/159): rbd-fuse-19.0.0-2946.g3d89accf.el9.x8 1.2 MB/s | 85 kB 00:00 2024-06-10T02:34:26.159 INFO:teuthology.orchestra.run.smithi120.stdout:(7/159): ceph-mon-19.0.0-2946.g3d89accf.el9.x86 3.4 MB/s | 4.7 MB 00:01 2024-06-10T02:34:26.226 INFO:teuthology.orchestra.run.smithi052.stdout:(66/159): ceph-test-19.0.0-2946.g3d89accf.el9.x 6.8 MB/s | 48 MB 00:07 2024-06-10T02:34:26.392 INFO:teuthology.orchestra.run.smithi188.stdout:(26/159): rbd-mirror-19.0.0-2946.g3d89accf.el9. 11 MB/s | 3.1 MB 00:00 2024-06-10T02:34:26.467 INFO:teuthology.orchestra.run.smithi188.stdout:(27/159): rbd-nbd-19.0.0-2946.g3d89accf.el9.x86 2.2 MB/s | 171 kB 00:00 2024-06-10T02:34:26.534 INFO:teuthology.orchestra.run.smithi188.stdout:(28/159): ceph-grafana-dashboards-19.0.0-2946.g 476 kB/s | 32 kB 00:00 2024-06-10T02:34:26.601 INFO:teuthology.orchestra.run.smithi188.stdout:(29/159): ceph-mgr-cephadm-19.0.0-2946.g3d89acc 2.1 MB/s | 142 kB 00:00 2024-06-10T02:34:26.634 INFO:teuthology.orchestra.run.smithi052.stdout:(67/159): postgresql-13.14-1.el9.x86_64.rpm 3.9 MB/s | 1.6 MB 00:00 2024-06-10T02:34:26.834 INFO:teuthology.orchestra.run.smithi052.stdout:(68/159): openssl-devel-3.2.1-1.el9.x86_64.rpm 3.7 MB/s | 4.4 MB 00:01 2024-06-10T02:34:26.876 INFO:teuthology.orchestra.run.smithi052.stdout:(69/159): postgresql-contrib-13.14-1.el9.x86_64 3.7 MB/s | 911 kB 00:00 2024-06-10T02:34:26.918 INFO:teuthology.orchestra.run.smithi052.stdout:(70/159): postgresql-private-libs-13.14-1.el9.x 1.6 MB/s | 133 kB 00:00 2024-06-10T02:34:26.960 INFO:teuthology.orchestra.run.smithi188.stdout:(30/159): ceph-mgr-dashboard-19.0.0-2946.g3d89a 9.8 MB/s | 3.5 MB 00:00 2024-06-10T02:34:26.985 INFO:teuthology.orchestra.run.smithi052.stdout:(71/159): openblas-openmp-0.3.21-2.el9.x86_64.r 3.5 MB/s | 4.7 MB 00:01 2024-06-10T02:34:27.035 INFO:teuthology.orchestra.run.smithi052.stdout:(72/159): python3-attrs-20.3.0-7.el9.noarch.rpm 1.7 MB/s | 87 kB 00:00 2024-06-10T02:34:27.244 INFO:teuthology.orchestra.run.smithi052.stdout:(73/159): protobuf-3.14.0-13.el9.x86_64.rpm 3.1 MB/s | 1.0 MB 00:00 2024-06-10T02:34:27.332 INFO:teuthology.orchestra.run.smithi052.stdout:(74/159): python3-colorama-0.4.6-3.el9.noarch.r 1.0 MB/s | 61 kB 00:00 2024-06-10T02:34:27.420 INFO:teuthology.orchestra.run.smithi052.stdout:(75/159): python3-devel-3.9.18-3.el9.x86_64.rpm 2.0 MB/s | 244 kB 00:00 2024-06-10T02:34:27.652 INFO:teuthology.orchestra.run.smithi188.stdout:(31/159): ceph-mgr-diskprediction-local-19.0.0- 11 MB/s | 7.4 MB 00:00 2024-06-10T02:34:27.678 INFO:teuthology.orchestra.run.smithi052.stdout:(76/159): python3-jinja2-2.11.3-6.el9.noarch.rp 961 kB/s | 248 kB 00:00 2024-06-10T02:34:27.735 INFO:teuthology.orchestra.run.smithi188.stdout:(32/159): ceph-mgr-modules-core-19.0.0-2946.g3d 2.9 MB/s | 249 kB 00:00 2024-06-10T02:34:27.737 INFO:teuthology.orchestra.run.smithi052.stdout:(77/159): python3-jsonpatch-1.21-16.el9.noarch. 439 kB/s | 26 kB 00:00 2024-06-10T02:34:27.794 INFO:teuthology.orchestra.run.smithi188.stdout:(33/159): ceph-mgr-rook-19.0.0-2946.g3d89accf.e 846 kB/s | 49 kB 00:00 2024-06-10T02:34:27.795 INFO:teuthology.orchestra.run.smithi052.stdout:(78/159): python3-jsonpointer-2.0-4.el9.noarch. 329 kB/s | 19 kB 00:00 2024-06-10T02:34:27.870 INFO:teuthology.orchestra.run.smithi188.stdout:(34/159): ceph-prometheus-alerts-19.0.0-2946.g3 218 kB/s | 16 kB 00:00 2024-06-10T02:34:27.879 INFO:teuthology.orchestra.run.smithi052.stdout:(79/159): python3-mako-1.1.4-6.el9.noarch.rpm 2.0 MB/s | 172 kB 00:00 2024-06-10T02:34:27.938 INFO:teuthology.orchestra.run.smithi052.stdout:(80/159): python3-markupsafe-1.1.1-12.el9.x86_6 598 kB/s | 35 kB 00:00 2024-06-10T02:34:27.945 INFO:teuthology.orchestra.run.smithi188.stdout:(35/159): ceph-volume-19.0.0-2946.g3d89accf.el9 3.4 MB/s | 264 kB 00:00 2024-06-10T02:34:28.038 INFO:teuthology.orchestra.run.smithi052.stdout:(81/159): postgresql-server-13.14-1.el9.x86_64. 5.0 MB/s | 5.8 MB 00:01 2024-06-10T02:34:28.045 INFO:teuthology.orchestra.run.smithi188.stdout:(36/159): cephadm-19.0.0-2946.g3d89accf.el9.noa 7.5 MB/s | 763 kB 00:00 2024-06-10T02:34:28.112 INFO:teuthology.orchestra.run.smithi188.stdout:(37/159): cephfs-top-19.0.0-2946.g3d89accf.el9. 273 kB/s | 18 kB 00:00 2024-06-10T02:34:28.130 INFO:teuthology.orchestra.run.smithi052.stdout:(82/159): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.7 MB/s | 250 kB 00:00 2024-06-10T02:34:28.150 INFO:teuthology.orchestra.run.smithi188.stdout:[MIRROR] fuse-2.9.9-15.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/fuse-2.9.9-15.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-10T02:34:28.251 INFO:teuthology.orchestra.run.smithi052.stdout:(83/159): python3-oauthlib-3.1.1-5.el9.noarch.r 1.8 MB/s | 222 kB 00:00 2024-06-10T02:34:28.297 INFO:teuthology.orchestra.run.smithi052.stdout:(84/159): python3-packaging-20.9-5.el9.noarch.r 1.6 MB/s | 77 kB 00:00 2024-06-10T02:34:28.347 INFO:teuthology.orchestra.run.smithi052.stdout:(85/159): python3-prettytable-0.7.2-27.el9.noar 844 kB/s | 42 kB 00:00 2024-06-10T02:34:28.431 INFO:teuthology.orchestra.run.smithi052.stdout:(86/159): python3-protobuf-3.14.0-13.el9.noarch 3.2 MB/s | 269 kB 00:00 2024-06-10T02:34:28.487 INFO:teuthology.orchestra.run.smithi188.stdout:(38/159): ceph-osd-19.0.0-2946.g3d89accf.el9.x8 2.6 MB/s | 17 MB 00:06 2024-06-10T02:34:28.489 INFO:teuthology.orchestra.run.smithi188.stdout:[MIRROR] libgfortran-11.4.1-3.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libgfortran-11.4.1-3.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-10T02:34:28.601 INFO:teuthology.orchestra.run.smithi120.stdout:(8/159): ceph-radosgw-19.0.0-2946.g3d89accf.el9 4.2 MB/s | 10 MB 00:02 2024-06-10T02:34:28.623 INFO:teuthology.orchestra.run.smithi052.stdout:(87/159): python3-pyasn1-0.4.8-6.el9.noarch.rpm 829 kB/s | 159 kB 00:00 2024-06-10T02:34:28.651 INFO:teuthology.orchestra.run.smithi120.stdout:(9/159): ceph-selinux-19.0.0-2946.g3d89accf.el9 505 kB/s | 25 kB 00:00 2024-06-10T02:34:28.723 INFO:teuthology.orchestra.run.smithi052.stdout:(88/159): python3-babel-2.9.1-2.el9.noarch.rpm 3.5 MB/s | 6.0 MB 00:01 2024-06-10T02:34:28.749 INFO:teuthology.orchestra.run.smithi052.stdout:(89/159): python3-pyasn1-modules-0.4.8-6.el9.no 2.2 MB/s | 279 kB 00:00 2024-06-10T02:34:28.774 INFO:teuthology.orchestra.run.smithi052.stdout:(90/159): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-06-10T02:34:28.799 INFO:teuthology.orchestra.run.smithi052.stdout:(91/159): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-06-10T02:34:28.850 INFO:teuthology.orchestra.run.smithi052.stdout:(92/159): python3-toml-0.10.2-6.el9.noarch.rpm 834 kB/s | 42 kB 00:00 2024-06-10T02:34:28.885 INFO:teuthology.orchestra.run.smithi120.stdout:(10/159): ceph-common-19.0.0-2946.g3d89accf.el9 2.9 MB/s | 21 MB 00:07 2024-06-10T02:34:28.900 INFO:teuthology.orchestra.run.smithi052.stdout:(93/159): python3-wcwidth-0.2.5-8.el9.noarch.rp 888 kB/s | 44 kB 00:00 2024-06-10T02:34:28.961 INFO:teuthology.orchestra.run.smithi120.stdout:(11/159): cephfs-mirror-19.0.0-2946.g3d89accf.e 2.9 MB/s | 221 kB 00:00 2024-06-10T02:34:28.984 INFO:teuthology.orchestra.run.smithi052.stdout:(94/159): qatlib-24.02.0-1.el9.x86_64.rpm 2.6 MB/s | 221 kB 00:00 2024-06-10T02:34:29.011 INFO:teuthology.orchestra.run.smithi120.stdout:(12/159): cephfs-shell-19.0.0-2946.g3d89accf.el 542 kB/s | 27 kB 00:00 2024-06-10T02:34:29.034 INFO:teuthology.orchestra.run.smithi052.stdout:(95/159): qatlib-service-24.02.0-1.el9.x86_64.r 719 kB/s | 36 kB 00:00 2024-06-10T02:34:29.061 INFO:teuthology.orchestra.run.smithi120.stdout:(13/159): libcephfs-devel-19.0.0-2946.g3d89accf 638 kB/s | 32 kB 00:00 2024-06-10T02:34:29.134 INFO:teuthology.orchestra.run.smithi052.stdout:(96/159): qatzip-libs-1.2.0-1.el9.x86_64.rpm 472 kB/s | 47 kB 00:00 2024-06-10T02:34:29.218 INFO:teuthology.orchestra.run.smithi052.stdout:(97/159): socat-1.7.4.1-5.el9.x86_64.rpm 3.6 MB/s | 305 kB 00:00 2024-06-10T02:34:29.236 INFO:teuthology.orchestra.run.smithi120.stdout:(14/159): libcephfs2-19.0.0-2946.g3d89accf.el9. 4.2 MB/s | 747 kB 00:00 2024-06-10T02:34:29.277 INFO:teuthology.orchestra.run.smithi052.stdout:(98/159): uuid-1.6.2-55.el9.x86_64.rpm 997 kB/s | 58 kB 00:00 2024-06-10T02:34:29.287 INFO:teuthology.orchestra.run.smithi120.stdout:(15/159): libcephsqlite-19.0.0-2946.g3d89accf.e 3.2 MB/s | 163 kB 00:00 2024-06-10T02:34:29.335 INFO:teuthology.orchestra.run.smithi052.stdout:(99/159): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.6 MB/s | 94 kB 00:00 2024-06-10T02:34:29.337 INFO:teuthology.orchestra.run.smithi120.stdout:(16/159): librados-devel-19.0.0-2946.g3d89accf. 2.5 MB/s | 127 kB 00:00 2024-06-10T02:34:29.394 INFO:teuthology.orchestra.run.smithi052.stdout:(100/159): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2024-06-10T02:34:29.444 INFO:teuthology.orchestra.run.smithi052.stdout:(101/159): zlib-devel-1.2.11-41.el9.x86_64.rpm 895 kB/s | 45 kB 00:00 2024-06-10T02:34:29.446 INFO:teuthology.orchestra.run.smithi052.stdout:[MIRROR] lua-devel-5.4.4-4.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/CRB/x86_64/os/Packages/lua-devel-5.4.4-4.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-10T02:34:29.588 INFO:teuthology.orchestra.run.smithi120.stdout:(17/159): ceph-osd-19.0.0-2946.g3d89accf.el9.x8 3.9 MB/s | 17 MB 00:04 2024-06-10T02:34:29.613 INFO:teuthology.orchestra.run.smithi120.stdout:(18/159): libradosstriper1-19.0.0-2946.g3d89acc 1.8 MB/s | 501 kB 00:00 2024-06-10T02:34:29.653 INFO:teuthology.orchestra.run.smithi052.stdout:(102/159): python3-numpy-1.20.1-5.el9.x86_64.rp 3.0 MB/s | 5.1 MB 00:01 2024-06-10T02:34:29.663 INFO:teuthology.orchestra.run.smithi120.stdout:(19/159): python3-ceph-argparse-19.0.0-2946.g3d 906 kB/s | 45 kB 00:00 2024-06-10T02:34:29.678 INFO:teuthology.orchestra.run.smithi052.stdout:(103/159): lua-devel-5.4.4-4.el9.x86_64.rpm 95 kB/s | 22 kB 00:00 2024-06-10T02:34:29.682 INFO:teuthology.orchestra.run.smithi052.stdout:[MIRROR] protobuf-compiler-3.14.0-13.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/CRB/x86_64/os/Packages/protobuf-compiler-3.14.0-13.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-10T02:34:29.714 INFO:teuthology.orchestra.run.smithi120.stdout:(20/159): python3-ceph-common-19.0.0-2946.g3d89 2.6 MB/s | 135 kB 00:00 2024-06-10T02:34:29.720 INFO:teuthology.orchestra.run.smithi052.stdout:(104/159): abseil-cpp-20211102.0-3.el9.x86_64.r 13 MB/s | 548 kB 00:00 2024-06-10T02:34:29.745 INFO:teuthology.orchestra.run.smithi052.stdout:(105/159): gperftools-libs-2.9.1-2.el9.x86_64.r 12 MB/s | 309 kB 00:00 2024-06-10T02:34:29.771 INFO:teuthology.orchestra.run.smithi052.stdout:(106/159): grpc-data-1.46.7-10.el9.noarch.rpm 780 kB/s | 19 kB 00:00 2024-06-10T02:34:29.772 INFO:teuthology.orchestra.run.smithi120.stdout:(21/159): python3-cephfs-19.0.0-2946.g3d89accf. 2.7 MB/s | 162 kB 00:00 2024-06-10T02:34:29.865 INFO:teuthology.orchestra.run.smithi120.stdout:(22/159): python3-rados-19.0.0-2946.g3d89accf.e 3.4 MB/s | 321 kB 00:00 2024-06-10T02:34:29.871 INFO:teuthology.orchestra.run.smithi052.stdout:(107/159): libarrow-9.0.0-11.el9.x86_64.rpm 44 MB/s | 4.4 MB 00:00 2024-06-10T02:34:29.896 INFO:teuthology.orchestra.run.smithi052.stdout:(108/159): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-06-10T02:34:29.922 INFO:teuthology.orchestra.run.smithi052.stdout:(109/159): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-06-10T02:34:29.947 INFO:teuthology.orchestra.run.smithi052.stdout:(110/159): libunwind-1.6.2-1.el9.x86_64.rpm 2.6 MB/s | 67 kB 00:00 2024-06-10T02:34:29.948 INFO:teuthology.orchestra.run.smithi120.stdout:(23/159): python3-rbd-19.0.0-2946.g3d89accf.el9 3.5 MB/s | 300 kB 00:00 2024-06-10T02:34:29.981 INFO:teuthology.orchestra.run.smithi052.stdout:(111/159): protobuf-compiler-3.14.0-13.el9.x86_ 2.6 MB/s | 863 kB 00:00 2024-06-10T02:34:30.006 INFO:teuthology.orchestra.run.smithi052.stdout:(112/159): luarocks-3.9.2-5.el9.noarch.rpm 2.5 MB/s | 151 kB 00:00 2024-06-10T02:34:30.007 INFO:teuthology.orchestra.run.smithi120.stdout:(24/159): python3-rgw-19.0.0-2946.g3d89accf.el9 1.7 MB/s | 99 kB 00:00 2024-06-10T02:34:30.057 INFO:teuthology.orchestra.run.smithi120.stdout:(25/159): rbd-fuse-19.0.0-2946.g3d89accf.el9.x8 1.7 MB/s | 85 kB 00:00 2024-06-10T02:34:30.064 INFO:teuthology.orchestra.run.smithi052.stdout:(113/159): parquet-libs-9.0.0-11.el9.x86_64.rpm 9.8 MB/s | 839 kB 00:00 2024-06-10T02:34:30.090 INFO:teuthology.orchestra.run.smithi052.stdout:(114/159): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.9 MB/s | 43 kB 00:00 2024-06-10T02:34:30.115 INFO:teuthology.orchestra.run.smithi052.stdout:(115/159): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-06-10T02:34:30.140 INFO:teuthology.orchestra.run.smithi052.stdout:(116/159): python3-certifi-2021.10.8-2.el9.noar 597 kB/s | 15 kB 00:00 2024-06-10T02:34:30.166 INFO:teuthology.orchestra.run.smithi052.stdout:(117/159): python3-cheroot-8.6.0-4.el9.noarch.r 6.7 MB/s | 172 kB 00:00 2024-06-10T02:34:30.200 INFO:teuthology.orchestra.run.smithi052.stdout:(118/159): python3-asyncssh-2.13.2-5.el9.noarch 2.8 MB/s | 548 kB 00:00 2024-06-10T02:34:30.225 INFO:teuthology.orchestra.run.smithi052.stdout:(119/159): python3-cherrypy-18.6.1-2.el9.noarch 6.0 MB/s | 358 kB 00:00 2024-06-10T02:34:30.250 INFO:teuthology.orchestra.run.smithi052.stdout:(120/159): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.9 MB/s | 303 kB 00:00 2024-06-10T02:34:30.276 INFO:teuthology.orchestra.run.smithi052.stdout:(121/159): python3-google-auth-2.29.0-2.el9.noa 4.0 MB/s | 205 kB 00:00 2024-06-10T02:34:30.301 INFO:teuthology.orchestra.run.smithi052.stdout:(122/159): python3-grpcio-tools-1.46.7-10.el9.x 5.7 MB/s | 144 kB 00:00 2024-06-10T02:34:30.326 INFO:teuthology.orchestra.run.smithi052.stdout:(123/159): python3-jaraco-8.2.1-3.el9.noarch.rp 429 kB/s | 11 kB 00:00 2024-06-10T02:34:30.351 INFO:teuthology.orchestra.run.smithi052.stdout:(124/159): python3-jaraco-classes-3.2.1-5.el9.n 707 kB/s | 18 kB 00:00 2024-06-10T02:34:30.377 INFO:teuthology.orchestra.run.smithi052.stdout:(125/159): python3-jaraco-collections-3.0.0-8.e 931 kB/s | 23 kB 00:00 2024-06-10T02:34:30.391 INFO:teuthology.orchestra.run.smithi120.stdout:(26/159): librgw2-19.0.0-2946.g3d89accf.el9.x86 6.6 MB/s | 5.3 MB 00:00 2024-06-10T02:34:30.402 INFO:teuthology.orchestra.run.smithi052.stdout:(126/159): python3-jaraco-functools-3.5.0-2.el9 780 kB/s | 19 kB 00:00 2024-06-10T02:34:30.427 INFO:teuthology.orchestra.run.smithi052.stdout:(127/159): python3-jaraco-text-3.2.0-6.el9.noar 793 kB/s | 20 kB 00:00 2024-06-10T02:34:30.449 INFO:teuthology.orchestra.run.smithi120.stdout:(27/159): rbd-nbd-19.0.0-2946.g3d89accf.el9.x86 2.9 MB/s | 171 kB 00:00 2024-06-10T02:34:30.477 INFO:teuthology.orchestra.run.smithi052.stdout:(128/159): python3-kubernetes-26.1.0-2.el9.noar 20 MB/s | 1.0 MB 00:00 2024-06-10T02:34:30.500 INFO:teuthology.orchestra.run.smithi120.stdout:(28/159): ceph-grafana-dashboards-19.0.0-2946.g 634 kB/s | 32 kB 00:00 2024-06-10T02:34:30.528 INFO:teuthology.orchestra.run.smithi052.stdout:(129/159): python3-grpcio-1.46.7-10.el9.x86_64. 7.3 MB/s | 2.0 MB 00:00 2024-06-10T02:34:30.550 INFO:teuthology.orchestra.run.smithi120.stdout:(29/159): rbd-mirror-19.0.0-2946.g3d89accf.el9. 6.3 MB/s | 3.1 MB 00:00 2024-06-10T02:34:30.553 INFO:teuthology.orchestra.run.smithi052.stdout:(130/159): python3-logutils-0.3.5-21.el9.noarch 615 kB/s | 46 kB 00:00 2024-06-10T02:34:30.578 INFO:teuthology.orchestra.run.smithi052.stdout:(131/159): python3-more-itertools-8.12.0-2.el9. 1.5 MB/s | 79 kB 00:00 2024-06-10T02:34:30.584 INFO:teuthology.orchestra.run.smithi120.stdout:(30/159): ceph-mgr-cephadm-19.0.0-2946.g3d89acc 1.7 MB/s | 142 kB 00:00 2024-06-10T02:34:30.604 INFO:teuthology.orchestra.run.smithi052.stdout:(132/159): python3-natsort-7.1.1-5.el9.noarch.r 1.1 MB/s | 58 kB 00:00 2024-06-10T02:34:30.629 INFO:teuthology.orchestra.run.smithi052.stdout:(133/159): python3-portend-3.1.0-2.el9.noarch.r 660 kB/s | 16 kB 00:00 2024-06-10T02:34:30.654 INFO:teuthology.orchestra.run.smithi052.stdout:(134/159): python3-pyOpenSSL-21.0.0-1.el9.noarc 3.5 MB/s | 90 kB 00:00 2024-06-10T02:34:30.688 INFO:teuthology.orchestra.run.smithi052.stdout:(135/159): python3-pecan-1.4.2-3.el9.noarch.rpm 2.4 MB/s | 272 kB 00:00 2024-06-10T02:34:30.713 INFO:teuthology.orchestra.run.smithi052.stdout:(136/159): python3-pyperclip-1.8.2-2.el9.noarch 482 kB/s | 28 kB 00:00 2024-06-10T02:34:30.739 INFO:teuthology.orchestra.run.smithi052.stdout:(137/159): python3-repoze-lru-0.7-16.el9.noarch 611 kB/s | 31 kB 00:00 2024-06-10T02:34:30.754 INFO:teuthology.orchestra.run.smithi188.stdout:(39/159): ceph-test-19.0.0-2946.g3d89accf.el9.x 6.8 MB/s | 48 MB 00:07 2024-06-10T02:34:30.755 INFO:teuthology.orchestra.run.smithi188.stdout:[MIRROR] libquadmath-11.4.1-3.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libquadmath-11.4.1-3.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-10T02:34:30.764 INFO:teuthology.orchestra.run.smithi052.stdout:(138/159): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-06-10T02:34:30.789 INFO:teuthology.orchestra.run.smithi052.stdout:(139/159): python3-tempora-5.0.0-2.el9.noarch.r 1.4 MB/s | 36 kB 00:00 2024-06-10T02:34:30.814 INFO:teuthology.orchestra.run.smithi052.stdout:(140/159): python3-routes-2.5.1-5.el9.noarch.rp 1.8 MB/s | 188 kB 00:00 2024-06-10T02:34:30.840 INFO:teuthology.orchestra.run.smithi052.stdout:(141/159): python3-typing-extensions-4.4.0-2.el 1.0 MB/s | 51 kB 00:00 2024-06-10T02:34:30.865 INFO:teuthology.orchestra.run.smithi052.stdout:(142/159): python3-websocket-client-1.2.3-2.el9 3.5 MB/s | 90 kB 00:00 2024-06-10T02:34:30.899 INFO:teuthology.orchestra.run.smithi052.stdout:(143/159): python3-webob-1.8.7-6.el9.noarch.rpm 2.7 MB/s | 230 kB 00:00 2024-06-10T02:34:30.924 INFO:teuthology.orchestra.run.smithi052.stdout:(144/159): python3-zc-lockfile-2.0-10.el9.noarc 806 kB/s | 20 kB 00:00 2024-06-10T02:34:30.949 INFO:teuthology.orchestra.run.smithi052.stdout:(145/159): re2-20211101-3.el9.x86_64.rpm 7.5 MB/s | 192 kB 00:00 2024-06-10T02:34:30.975 INFO:teuthology.orchestra.run.smithi052.stdout:(146/159): python3-werkzeug-2.0.3-3.el9.1.noarc 3.8 MB/s | 427 kB 00:00 2024-06-10T02:34:31.001 INFO:teuthology.orchestra.run.smithi120.stdout:(31/159): ceph-mgr-dashboard-19.0.0-2946.g3d89a 7.8 MB/s | 3.5 MB 00:00 2024-06-10T02:34:31.025 INFO:teuthology.orchestra.run.smithi052.stdout:(147/159): thrift-0.15.0-2.el9.x86_64.rpm 21 MB/s | 1.6 MB 00:00 2024-06-10T02:34:31.068 INFO:teuthology.orchestra.run.smithi120.stdout:(32/159): ceph-mgr-modules-core-19.0.0-2946.g3d 3.7 MB/s | 249 kB 00:00 2024-06-10T02:34:31.110 INFO:teuthology.orchestra.run.smithi120.stdout:(33/159): ceph-mgr-rook-19.0.0-2946.g3d89accf.e 1.2 MB/s | 49 kB 00:00 2024-06-10T02:34:31.152 INFO:teuthology.orchestra.run.smithi120.stdout:(34/159): ceph-prometheus-alerts-19.0.0-2946.g3 394 kB/s | 16 kB 00:00 2024-06-10T02:34:31.260 INFO:teuthology.orchestra.run.smithi120.stdout:(35/159): ceph-volume-19.0.0-2946.g3d89accf.el9 2.4 MB/s | 264 kB 00:00 2024-06-10T02:34:31.484 INFO:teuthology.orchestra.run.smithi052.stdout:(148/159): librados2-19.0.0-2946.g3d89accf.el9. 6.7 MB/s | 3.4 MB 00:00 2024-06-10T02:34:31.494 INFO:teuthology.orchestra.run.smithi120.stdout:(36/159): cephadm-19.0.0-2946.g3d89accf.el9.noa 3.2 MB/s | 763 kB 00:00 2024-06-10T02:34:31.544 INFO:teuthology.orchestra.run.smithi120.stdout:(37/159): cephfs-top-19.0.0-2946.g3d89accf.el9. 365 kB/s | 18 kB 00:00 2024-06-10T02:34:31.567 INFO:teuthology.orchestra.run.smithi052.stdout:(149/159): librbd1-19.0.0-2946.g3d89accf.el9.x8 5.8 MB/s | 3.1 MB 00:00 2024-06-10T02:34:31.582 INFO:teuthology.orchestra.run.smithi120.stdout:[MIRROR] fuse-2.9.9-15.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/fuse-2.9.9-15.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-10T02:34:31.703 INFO:teuthology.orchestra.run.smithi120.stdout:(38/159): ceph-mgr-diskprediction-local-19.0.0- 6.6 MB/s | 7.4 MB 00:01 2024-06-10T02:34:31.704 INFO:teuthology.orchestra.run.smithi120.stdout:[MIRROR] libgfortran-11.4.1-3.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libgfortran-11.4.1-3.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-10T02:34:31.851 INFO:teuthology.orchestra.run.smithi052.stdout:(150/159): NetworkManager-1.47.91-1.el9.x86_64. 6.3 MB/s | 2.3 MB 00:00 2024-06-10T02:34:31.893 INFO:teuthology.orchestra.run.smithi052.stdout:(151/159): NetworkManager-libnm-1.47.91-1.el9.x 5.5 MB/s | 1.8 MB 00:00 2024-06-10T02:34:32.002 INFO:teuthology.orchestra.run.smithi052.stdout:(152/159): NetworkManager-team-1.47.91-1.el9.x8 276 kB/s | 41 kB 00:00 2024-06-10T02:34:32.045 INFO:teuthology.orchestra.run.smithi120.stdout:(39/159): fuse-2.9.9-15.el9.x86_64.rpm 160 kB/s | 80 kB 00:00 2024-06-10T02:34:32.046 INFO:teuthology.orchestra.run.smithi120.stdout:[MIRROR] libquadmath-11.4.1-3.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libquadmath-11.4.1-3.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-10T02:34:32.085 INFO:teuthology.orchestra.run.smithi052.stdout:(153/159): NetworkManager-tui-1.47.91-1.el9.x86 1.3 MB/s | 248 kB 00:00 2024-06-10T02:34:32.102 INFO:teuthology.orchestra.run.smithi052.stdout:(154/159): elfutils-debuginfod-client-0.191-4.e 367 kB/s | 37 kB 00:00 2024-06-10T02:34:32.237 INFO:teuthology.orchestra.run.smithi120.stdout:(40/159): libquadmath-11.4.1-3.el9.x86_64.rpm 1.0 MB/s | 195 kB 00:00 2024-06-10T02:34:32.277 INFO:teuthology.orchestra.run.smithi052.stdout:(155/159): elfutils-libelf-0.191-4.el9.x86_64.r 1.1 MB/s | 207 kB 00:00 2024-06-10T02:34:32.311 INFO:teuthology.orchestra.run.smithi052.stdout:(156/159): elfutils-libs-0.191-4.el9.x86_64.rpm 1.2 MB/s | 260 kB 00:00 2024-06-10T02:34:32.421 INFO:teuthology.orchestra.run.smithi120.stdout:(41/159): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.3 MB/s | 253 kB 00:00 2024-06-10T02:34:32.570 INFO:teuthology.orchestra.run.smithi052.stdout:(157/159): openssl-3.2.1-1.el9.x86_64.rpm 4.6 MB/s | 1.3 MB 00:00 2024-06-10T02:34:32.613 INFO:teuthology.orchestra.run.smithi120.stdout:(42/159): libgfortran-11.4.1-3.el9.x86_64.rpm 883 kB/s | 803 kB 00:00 2024-06-10T02:34:32.670 INFO:teuthology.orchestra.run.smithi052.stdout:(158/159): openssl-libs-3.2.1-1.el9.x86_64.rpm 6.7 MB/s | 2.4 MB 00:00 2024-06-10T02:34:32.721 INFO:teuthology.orchestra.run.smithi120.stdout:(43/159): python3-ply-3.11-14.el9.noarch.rpm 984 kB/s | 106 kB 00:00 2024-06-10T02:34:32.822 INFO:teuthology.orchestra.run.smithi120.stdout:(44/159): python3-pycparser-2.20-6.el9.noarch.r 1.3 MB/s | 135 kB 00:00 2024-06-10T02:34:32.887 INFO:teuthology.orchestra.run.smithi052.stdout:(159/159): python3-scipy-1.6.2-8.el9.x86_64.rpm 4.0 MB/s | 16 MB 00:04 2024-06-10T02:34:32.892 INFO:teuthology.orchestra.run.smithi052.stdout:-------------------------------------------------------------------------------- 2024-06-10T02:34:32.893 INFO:teuthology.orchestra.run.smithi052.stdout:Total 13 MB/s | 225 MB 00:16 2024-06-10T02:34:32.931 INFO:teuthology.orchestra.run.smithi120.stdout:(45/159): python3-requests-2.25.1-8.el9.noarch. 1.1 MB/s | 125 kB 00:00 2024-06-10T02:34:33.039 INFO:teuthology.orchestra.run.smithi120.stdout:(46/159): python3-urllib3-1.26.5-5.el9.noarch.r 1.9 MB/s | 215 kB 00:00 2024-06-10T02:34:33.040 INFO:teuthology.orchestra.run.smithi120.stdout:[MIRROR] automake-1.16.2-8.el9.noarch.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/automake-1.16.2-8.el9.noarch.rpm (IP: 8.43.85.72) 2024-06-10T02:34:33.198 INFO:teuthology.orchestra.run.smithi120.stdout:(47/159): python3-cryptography-36.0.1-4.el9.x86 1.6 MB/s | 1.2 MB 00:00 2024-06-10T02:34:33.199 INFO:teuthology.orchestra.run.smithi120.stdout:[MIRROR] bison-3.7.4-5.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/bison-3.7.4-5.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-10T02:34:33.273 INFO:teuthology.orchestra.run.smithi120.stdout:(48/159): automake-1.16.2-8.el9.noarch.rpm 2.9 MB/s | 689 kB 00:00 2024-06-10T02:34:33.274 INFO:teuthology.orchestra.run.smithi120.stdout:[MIRROR] boost-program-options-1.75.0-8.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/boost-program-options-1.75.0-8.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-10T02:34:33.324 INFO:teuthology.orchestra.run.smithi120.stdout:(49/159): boost-program-options-1.75.0-8.el9.x8 2.1 MB/s | 107 kB 00:00 2024-06-10T02:34:33.374 INFO:teuthology.orchestra.run.smithi120.stdout:(50/159): elfutils-libelf-devel-0.191-4.el9.x86 435 kB/s | 22 kB 00:00 2024-06-10T02:34:33.424 INFO:teuthology.orchestra.run.smithi120.stdout:(51/159): bison-3.7.4-5.el9.x86_64.rpm 4.1 MB/s | 943 kB 00:00 2024-06-10T02:34:33.450 INFO:teuthology.orchestra.run.smithi120.stdout:(52/159): flex-2.6.4-9.el9.x86_64.rpm 4.1 MB/s | 313 kB 00:00 2024-06-10T02:34:33.475 INFO:teuthology.orchestra.run.smithi120.stdout:(53/159): flexiblas-3.0.4-8.el9.x86_64.rpm 651 kB/s | 33 kB 00:00 2024-06-10T02:34:33.525 INFO:teuthology.orchestra.run.smithi120.stdout:(54/159): flexiblas-openblas-openmp-3.0.4-8.el9 365 kB/s | 18 kB 00:00 2024-06-10T02:34:33.584 INFO:teuthology.orchestra.run.smithi120.stdout:(55/159): flexiblas-netlib-3.0.4-8.el9.x86_64.r 22 MB/s | 3.0 MB 00:00 2024-06-10T02:34:33.601 INFO:teuthology.orchestra.run.smithi120.stdout:(56/159): libaio-devel-0.3.111-13.el9.x86_64.rp 148 kB/s | 11 kB 00:00 2024-06-10T02:34:33.635 INFO:teuthology.orchestra.run.smithi120.stdout:(57/159): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 160 kB 00:00 2024-06-10T02:34:33.660 INFO:teuthology.orchestra.run.smithi120.stdout:(58/159): librabbitmq-0.11.0-7.el9.x86_64.rpm 771 kB/s | 45 kB 00:00 2024-06-10T02:34:33.694 INFO:teuthology.orchestra.run.smithi120.stdout:(59/159): librdkafka-1.6.1-102.el9.x86_64.rpm 11 MB/s | 662 kB 00:00 2024-06-10T02:34:33.727 INFO:teuthology.orchestra.run.smithi120.stdout:(60/159): libtool-2.4.6-45.el9.x86_64.rpm 8.5 MB/s | 582 kB 00:00 2024-06-10T02:34:33.752 INFO:teuthology.orchestra.run.smithi120.stdout:(61/159): libuuid-devel-2.37.4-18.el9.x86_64.rp 435 kB/s | 25 kB 00:00 2024-06-10T02:34:33.778 INFO:teuthology.orchestra.run.smithi120.stdout:(62/159): libzstd-devel-1.5.1-2.el9.x86_64.rpm 941 kB/s | 47 kB 00:00 2024-06-10T02:34:33.803 INFO:teuthology.orchestra.run.smithi120.stdout:(63/159): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.7 MB/s | 292 kB 00:00 2024-06-10T02:34:33.828 INFO:teuthology.orchestra.run.smithi120.stdout:(64/159): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 188 kB 00:00 2024-06-10T02:34:33.854 INFO:teuthology.orchestra.run.smithi120.stdout:(65/159): openblas-0.3.21-2.el9.x86_64.rpm 698 kB/s | 35 kB 00:00 2024-06-10T02:34:34.013 INFO:teuthology.orchestra.run.smithi120.stdout:(66/159): openblas-openmp-0.3.21-2.el9.x86_64.r 26 MB/s | 4.7 MB 00:00 2024-06-10T02:34:34.088 INFO:teuthology.orchestra.run.smithi120.stdout:(67/159): openssl-devel-3.2.1-1.el9.x86_64.rpm 19 MB/s | 4.4 MB 00:00 2024-06-10T02:34:34.130 INFO:teuthology.orchestra.run.smithi120.stdout:(68/159): postgresql-13.14-1.el9.x86_64.rpm 14 MB/s | 1.6 MB 00:00 2024-06-10T02:34:34.141 INFO:teuthology.orchestra.run.smithi052.stdout:Running transaction check 2024-06-10T02:34:34.172 INFO:teuthology.orchestra.run.smithi120.stdout:(69/159): postgresql-contrib-13.14-1.el9.x86_64 11 MB/s | 911 kB 00:00 2024-06-10T02:34:34.197 INFO:teuthology.orchestra.run.smithi120.stdout:(70/159): postgresql-private-libs-13.14-1.el9.x 1.9 MB/s | 133 kB 00:00 2024-06-10T02:34:34.316 INFO:teuthology.orchestra.run.smithi052.stdout:Transaction check succeeded. 2024-06-10T02:34:34.316 INFO:teuthology.orchestra.run.smithi052.stdout:Running transaction test 2024-06-10T02:34:34.615 INFO:teuthology.orchestra.run.smithi120.stdout:(71/159): ceph-test-19.0.0-2946.g3d89accf.el9.x 8.1 MB/s | 48 MB 00:05 2024-06-10T02:34:34.648 INFO:teuthology.orchestra.run.smithi120.stdout:(72/159): protobuf-3.14.0-13.el9.x86_64.rpm 2.2 MB/s | 1.0 MB 00:00 2024-06-10T02:34:34.732 INFO:teuthology.orchestra.run.smithi120.stdout:(73/159): postgresql-server-13.14-1.el9.x86_64. 10 MB/s | 5.8 MB 00:00 2024-06-10T02:34:34.782 INFO:teuthology.orchestra.run.smithi120.stdout:(74/159): python3-attrs-20.3.0-7.el9.noarch.rpm 523 kB/s | 87 kB 00:00 2024-06-10T02:34:34.807 INFO:teuthology.orchestra.run.smithi120.stdout:(75/159): python3-colorama-0.4.6-3.el9.noarch.r 816 kB/s | 61 kB 00:00 2024-06-10T02:34:34.891 INFO:teuthology.orchestra.run.smithi120.stdout:(76/159): python3-babel-2.9.1-2.el9.noarch.rpm 25 MB/s | 6.0 MB 00:00 2024-06-10T02:34:34.916 INFO:teuthology.orchestra.run.smithi120.stdout:(77/159): python3-devel-3.9.18-3.el9.x86_64.rpm 1.8 MB/s | 244 kB 00:00 2024-06-10T02:34:34.941 INFO:teuthology.orchestra.run.smithi120.stdout:(78/159): python3-jinja2-2.11.3-6.el9.noarch.rp 1.8 MB/s | 248 kB 00:00 2024-06-10T02:34:34.967 INFO:teuthology.orchestra.run.smithi120.stdout:(79/159): python3-jsonpatch-1.21-16.el9.noarch. 340 kB/s | 26 kB 00:00 2024-06-10T02:34:34.992 INFO:teuthology.orchestra.run.smithi120.stdout:(80/159): python3-jsonpointer-2.0-4.el9.noarch. 254 kB/s | 19 kB 00:00 2024-06-10T02:34:35.017 INFO:teuthology.orchestra.run.smithi120.stdout:(81/159): python3-markupsafe-1.1.1-12.el9.x86_6 695 kB/s | 35 kB 00:00 2024-06-10T02:34:35.043 INFO:teuthology.orchestra.run.smithi120.stdout:(82/159): python3-mako-1.1.4-6.el9.noarch.rpm 2.2 MB/s | 172 kB 00:00 2024-06-10T02:34:35.068 INFO:teuthology.orchestra.run.smithi120.stdout:(83/159): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.9 MB/s | 250 kB 00:00 2024-06-10T02:34:35.093 INFO:teuthology.orchestra.run.smithi120.stdout:(84/159): python3-oauthlib-3.1.1-5.el9.noarch.r 4.3 MB/s | 222 kB 00:00 2024-06-10T02:34:35.119 INFO:teuthology.orchestra.run.smithi120.stdout:(85/159): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-06-10T02:34:35.144 INFO:teuthology.orchestra.run.smithi120.stdout:(86/159): python3-prettytable-0.7.2-27.el9.noar 838 kB/s | 42 kB 00:00 2024-06-10T02:34:35.178 INFO:teuthology.orchestra.run.smithi120.stdout:(87/159): python3-protobuf-3.14.0-13.el9.noarch 4.5 MB/s | 269 kB 00:00 2024-06-10T02:34:35.203 INFO:teuthology.orchestra.run.smithi120.stdout:(88/159): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.6 MB/s | 159 kB 00:00 2024-06-10T02:34:35.228 INFO:teuthology.orchestra.run.smithi120.stdout:(89/159): python3-pyasn1-modules-0.4.8-6.el9.no 5.4 MB/s | 279 kB 00:00 2024-06-10T02:34:35.245 INFO:teuthology.orchestra.run.smithi120.stdout:(90/159): python3-pytz-2021.1-5.el9.noarch.rpm 1.2 MB/s | 51 kB 00:00 2024-06-10T02:34:35.329 INFO:teuthology.orchestra.run.smithi120.stdout:(91/159): python3-numpy-1.20.1-5.el9.x86_64.rpm 15 MB/s | 5.1 MB 00:00 2024-06-10T02:34:35.363 INFO:teuthology.orchestra.run.smithi120.stdout:(92/159): python3-requests-oauthlib-1.3.0-12.el 400 kB/s | 54 kB 00:00 2024-06-10T02:34:35.380 INFO:teuthology.orchestra.run.smithi120.stdout:(93/159): python3-toml-0.10.2-6.el9.noarch.rpm 831 kB/s | 42 kB 00:00 2024-06-10T02:34:35.405 INFO:teuthology.orchestra.run.smithi120.stdout:(94/159): python3-wcwidth-0.2.5-8.el9.noarch.rp 1.0 MB/s | 44 kB 00:00 2024-06-10T02:34:35.439 INFO:teuthology.orchestra.run.smithi120.stdout:(95/159): qatlib-24.02.0-1.el9.x86_64.rpm 3.7 MB/s | 221 kB 00:00 2024-06-10T02:34:35.464 INFO:teuthology.orchestra.run.smithi120.stdout:(96/159): qatlib-service-24.02.0-1.el9.x86_64.r 610 kB/s | 36 kB 00:00 2024-06-10T02:34:35.490 INFO:teuthology.orchestra.run.smithi120.stdout:(97/159): qatzip-libs-1.2.0-1.el9.x86_64.rpm 942 kB/s | 47 kB 00:00 2024-06-10T02:34:35.515 INFO:teuthology.orchestra.run.smithi120.stdout:(98/159): socat-1.7.4.1-5.el9.x86_64.rpm 5.9 MB/s | 305 kB 00:00 2024-06-10T02:34:35.540 INFO:teuthology.orchestra.run.smithi120.stdout:(99/159): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-06-10T02:34:35.565 INFO:teuthology.orchestra.run.smithi120.stdout:(100/159): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.8 MB/s | 94 kB 00:00 2024-06-10T02:34:35.591 INFO:teuthology.orchestra.run.smithi120.stdout:(101/159): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-06-10T02:34:35.618 INFO:teuthology.orchestra.run.smithi120.stdout:(102/159): zlib-devel-1.2.11-41.el9.x86_64.rpm 889 kB/s | 45 kB 00:00 2024-06-10T02:34:35.618 INFO:teuthology.orchestra.run.smithi120.stdout:[MIRROR] lua-devel-5.4.4-4.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/CRB/x86_64/os/Packages/lua-devel-5.4.4-4.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-10T02:34:35.618 INFO:teuthology.orchestra.run.smithi120.stdout:[MIRROR] protobuf-compiler-3.14.0-13.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/CRB/x86_64/os/Packages/protobuf-compiler-3.14.0-13.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-10T02:34:35.758 INFO:teuthology.orchestra.run.smithi120.stdout:(103/159): lua-devel-5.4.4-4.el9.x86_64.rpm 133 kB/s | 22 kB 00:00 2024-06-10T02:34:35.925 INFO:teuthology.orchestra.run.smithi120.stdout:(104/159): python3-scipy-1.6.2-8.el9.x86_64.rpm 24 MB/s | 16 MB 00:00 2024-06-10T02:34:35.984 INFO:teuthology.orchestra.run.smithi120.stdout:(105/159): abseil-cpp-20211102.0-3.el9.x86_64.r 2.4 MB/s | 548 kB 00:00 2024-06-10T02:34:36.009 INFO:teuthology.orchestra.run.smithi120.stdout:(106/159): grpc-data-1.46.7-10.el9.noarch.rpm 862 kB/s | 19 kB 00:00 2024-06-10T02:34:36.043 INFO:teuthology.orchestra.run.smithi120.stdout:(107/159): protobuf-compiler-3.14.0-13.el9.x86_ 2.0 MB/s | 863 kB 00:00 2024-06-10T02:34:36.076 INFO:teuthology.orchestra.run.smithi120.stdout:(108/159): gperftools-libs-2.9.1-2.el9.x86_64.r 2.0 MB/s | 309 kB 00:00 2024-06-10T02:34:36.127 INFO:teuthology.orchestra.run.smithi120.stdout:(109/159): libarrow-doc-9.0.0-11.el9.noarch.rpm 519 kB/s | 26 kB 00:00 2024-06-10T02:34:36.160 INFO:teuthology.orchestra.run.smithi120.stdout:(110/159): libunwind-1.6.2-1.el9.x86_64.rpm 2.1 MB/s | 67 kB 00:00 2024-06-10T02:34:36.186 INFO:teuthology.orchestra.run.smithi120.stdout:(111/159): liboath-2.6.7-2.el9.x86_64.rpm 446 kB/s | 49 kB 00:00 2024-06-10T02:34:36.211 INFO:teuthology.orchestra.run.smithi120.stdout:(112/159): luarocks-3.9.2-5.el9.noarch.rpm 2.9 MB/s | 151 kB 00:00 2024-06-10T02:34:36.253 INFO:teuthology.orchestra.run.smithi120.stdout:(113/159): python3-asyncssh-2.13.2-5.el9.noarch 13 MB/s | 548 kB 00:00 2024-06-10T02:34:36.270 INFO:teuthology.orchestra.run.smithi120.stdout:(114/159): python3-bcrypt-3.2.2-1.el9.x86_64.rp 2.6 MB/s | 43 kB 00:00 2024-06-10T02:34:36.287 INFO:teuthology.orchestra.run.smithi120.stdout:(115/159): python3-cachetools-4.2.4-1.el9.noarc 1.9 MB/s | 32 kB 00:00 2024-06-10T02:34:36.312 INFO:teuthology.orchestra.run.smithi120.stdout:(116/159): python3-certifi-2021.10.8-2.el9.noar 592 kB/s | 15 kB 00:00 2024-06-10T02:34:36.371 INFO:teuthology.orchestra.run.smithi120.stdout:(117/159): parquet-libs-9.0.0-11.el9.x86_64.rpm 4.4 MB/s | 839 kB 00:00 2024-06-10T02:34:36.396 INFO:teuthology.orchestra.run.smithi120.stdout:(118/159): python3-cheroot-8.6.0-4.el9.noarch.r 2.0 MB/s | 172 kB 00:00 2024-06-10T02:34:36.472 INFO:teuthology.orchestra.run.smithi120.stdout:(119/159): libarrow-9.0.0-11.el9.x86_64.rpm 9.6 MB/s | 4.4 MB 00:00 2024-06-10T02:34:36.497 INFO:teuthology.orchestra.run.smithi120.stdout:(120/159): python3-cherrypy-18.6.1-2.el9.noarch 3.5 MB/s | 358 kB 00:00 2024-06-10T02:34:36.522 INFO:teuthology.orchestra.run.smithi120.stdout:(121/159): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.4 MB/s | 303 kB 00:00 2024-06-10T02:34:36.546 INFO:teuthology.orchestra.run.smithi052.stdout:Transaction test succeeded. 2024-06-10T02:34:36.547 INFO:teuthology.orchestra.run.smithi120.stdout:(122/159): python3-google-auth-2.29.0-2.el9.noa 2.7 MB/s | 205 kB 00:00 2024-06-10T02:34:36.548 INFO:teuthology.orchestra.run.smithi052.stdout:Running transaction 2024-06-10T02:34:36.573 INFO:teuthology.orchestra.run.smithi120.stdout:(123/159): python3-jaraco-8.2.1-3.el9.noarch.rp 426 kB/s | 11 kB 00:00 2024-06-10T02:34:36.599 INFO:teuthology.orchestra.run.smithi120.stdout:(124/159): python3-grpcio-tools-1.46.7-10.el9.x 1.9 MB/s | 144 kB 00:00 2024-06-10T02:34:36.624 INFO:teuthology.orchestra.run.smithi120.stdout:(125/159): python3-jaraco-classes-3.2.1-5.el9.n 351 kB/s | 18 kB 00:00 2024-06-10T02:34:36.649 INFO:teuthology.orchestra.run.smithi120.stdout:(126/159): python3-jaraco-collections-3.0.0-8.e 461 kB/s | 23 kB 00:00 2024-06-10T02:34:36.674 INFO:teuthology.orchestra.run.smithi120.stdout:(127/159): python3-jaraco-functools-3.5.0-2.el9 385 kB/s | 19 kB 00:00 2024-06-10T02:34:36.729 INFO:teuthology.orchestra.run.smithi120.stdout:(128/159): python3-jaraco-text-3.2.0-6.el9.noar 247 kB/s | 20 kB 00:00 2024-06-10T02:34:36.758 INFO:teuthology.orchestra.run.smithi120.stdout:(129/159): python3-logutils-0.3.5-21.el9.noarch 1.5 MB/s | 46 kB 00:00 2024-06-10T02:34:36.783 INFO:teuthology.orchestra.run.smithi120.stdout:(130/159): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-06-10T02:34:36.809 INFO:teuthology.orchestra.run.smithi120.stdout:(131/159): python3-natsort-7.1.1-5.el9.noarch.r 2.2 MB/s | 58 kB 00:00 2024-06-10T02:34:36.843 INFO:teuthology.orchestra.run.smithi120.stdout:(132/159): python3-kubernetes-26.1.0-2.el9.noar 6.1 MB/s | 1.0 MB 00:00 2024-06-10T02:34:36.868 INFO:teuthology.orchestra.run.smithi120.stdout:(133/159): python3-portend-3.1.0-2.el9.noarch.r 658 kB/s | 16 kB 00:00 2024-06-10T02:34:36.893 INFO:teuthology.orchestra.run.smithi120.stdout:(134/159): python3-pecan-1.4.2-3.el9.noarch.rpm 3.2 MB/s | 272 kB 00:00 2024-06-10T02:34:36.919 INFO:teuthology.orchestra.run.smithi120.stdout:(135/159): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.7 MB/s | 90 kB 00:00 2024-06-10T02:34:36.943 INFO:teuthology.orchestra.run.smithi120.stdout:(136/159): python3-pyperclip-1.8.2-2.el9.noarch 565 kB/s | 28 kB 00:00 2024-06-10T02:34:36.994 INFO:teuthology.orchestra.run.smithi120.stdout:(137/159): python3-grpcio-1.46.7-10.el9.x86_64. 4.1 MB/s | 2.0 MB 00:00 2024-06-10T02:34:37.019 INFO:teuthology.orchestra.run.smithi120.stdout:(138/159): python3-repoze-lru-0.7-16.el9.noarch 408 kB/s | 31 kB 00:00 2024-06-10T02:34:37.045 INFO:teuthology.orchestra.run.smithi120.stdout:(139/159): python3-rsa-4.9-2.el9.noarch.rpm 1.1 MB/s | 59 kB 00:00 2024-06-10T02:34:37.070 INFO:teuthology.orchestra.run.smithi120.stdout:(140/159): python3-tempora-5.0.0-2.el9.noarch.r 711 kB/s | 36 kB 00:00 2024-06-10T02:34:37.095 INFO:teuthology.orchestra.run.smithi120.stdout:(141/159): python3-typing-extensions-4.4.0-2.el 1.0 MB/s | 51 kB 00:00 2024-06-10T02:34:37.121 INFO:teuthology.orchestra.run.smithi120.stdout:(142/159): python3-routes-2.5.1-5.el9.noarch.rp 1.0 MB/s | 188 kB 00:00 2024-06-10T02:34:37.155 INFO:teuthology.orchestra.run.smithi120.stdout:(143/159): python3-websocket-client-1.2.3-2.el9 1.5 MB/s | 90 kB 00:00 2024-06-10T02:34:37.180 INFO:teuthology.orchestra.run.smithi120.stdout:(144/159): python3-zc-lockfile-2.0-10.el9.noarc 805 kB/s | 20 kB 00:00 2024-06-10T02:34:37.205 INFO:teuthology.orchestra.run.smithi120.stdout:(145/159): python3-webob-1.8.7-6.el9.noarch.rpm 1.7 MB/s | 230 kB 00:00 2024-06-10T02:34:37.230 INFO:teuthology.orchestra.run.smithi120.stdout:(146/159): re2-20211101-3.el9.x86_64.rpm 3.7 MB/s | 192 kB 00:00 2024-06-10T02:34:37.264 INFO:teuthology.orchestra.run.smithi120.stdout:(147/159): python3-werkzeug-2.0.3-3.el9.1.noarc 2.9 MB/s | 427 kB 00:00 2024-06-10T02:34:37.306 INFO:teuthology.orchestra.run.smithi120.stdout:(148/159): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-06-10T02:34:37.615 INFO:teuthology.orchestra.run.smithi120.stdout:(149/159): librbd1-19.0.0-2946.g3d89accf.el9.x8 8.9 MB/s | 3.1 MB 00:00 2024-06-10T02:34:37.715 INFO:teuthology.orchestra.run.smithi120.stdout:(150/159): librados2-19.0.0-2946.g3d89accf.el9. 7.0 MB/s | 3.4 MB 00:00 2024-06-10T02:34:38.049 INFO:teuthology.orchestra.run.smithi120.stdout:(151/159): NetworkManager-team-1.47.91-1.el9.x8 124 kB/s | 41 kB 00:00 2024-06-10T02:34:38.391 INFO:teuthology.orchestra.run.smithi120.stdout:(152/159): NetworkManager-tui-1.47.91-1.el9.x86 726 kB/s | 248 kB 00:00 2024-06-10T02:34:38.492 INFO:teuthology.orchestra.run.smithi120.stdout:(153/159): elfutils-debuginfod-client-0.191-4.e 367 kB/s | 37 kB 00:00 2024-06-10T02:34:38.616 INFO:teuthology.orchestra.run.smithi120.stdout:(154/159): NetworkManager-libnm-1.47.91-1.el9.x 1.8 MB/s | 1.8 MB 00:01 2024-06-10T02:34:38.675 INFO:teuthology.orchestra.run.smithi120.stdout:(155/159): elfutils-libelf-0.191-4.el9.x86_64.r 1.1 MB/s | 207 kB 00:00 2024-06-10T02:34:38.725 INFO:teuthology.orchestra.run.smithi120.stdout:(156/159): elfutils-libs-0.191-4.el9.x86_64.rpm 2.3 MB/s | 260 kB 00:00 2024-06-10T02:34:38.901 INFO:teuthology.orchestra.run.smithi120.stdout:(157/159): NetworkManager-1.47.91-1.el9.x86_64. 1.4 MB/s | 2.3 MB 00:01 2024-06-10T02:34:39.126 INFO:teuthology.orchestra.run.smithi120.stdout:(158/159): openssl-libs-3.2.1-1.el9.x86_64.rpm 6.0 MB/s | 2.4 MB 00:00 2024-06-10T02:34:39.167 INFO:teuthology.orchestra.run.smithi120.stdout:(159/159): openssl-3.2.1-1.el9.x86_64.rpm 2.7 MB/s | 1.3 MB 00:00 2024-06-10T02:34:39.174 INFO:teuthology.orchestra.run.smithi120.stdout:-------------------------------------------------------------------------------- 2024-06-10T02:34:39.174 INFO:teuthology.orchestra.run.smithi120.stdout:Total 12 MB/s | 225 MB 00:18 2024-06-10T02:34:40.394 INFO:teuthology.orchestra.run.smithi120.stdout:Running transaction check 2024-06-10T02:34:40.552 INFO:teuthology.orchestra.run.smithi120.stdout:Transaction check succeeded. 2024-06-10T02:34:40.552 INFO:teuthology.orchestra.run.smithi120.stdout:Running transaction test 2024-06-10T02:34:42.631 INFO:teuthology.orchestra.run.smithi120.stdout:Transaction test succeeded. 2024-06-10T02:34:42.631 INFO:teuthology.orchestra.run.smithi120.stdout:Running transaction 2024-06-10T02:34:42.970 INFO:teuthology.orchestra.run.smithi052.stdout: Preparing : 1/1 2024-06-10T02:34:43.050 INFO:teuthology.orchestra.run.smithi052.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/170 2024-06-10T02:34:43.114 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/170 2024-06-10T02:34:43.179 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/170 2024-06-10T02:34:43.218 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/170 2024-06-10T02:34:43.243 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/170 2024-06-10T02:34:43.299 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/170 2024-06-10T02:34:43.662 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/170 2024-06-10T02:34:43.777 INFO:teuthology.orchestra.run.smithi052.stdout: Upgrading : librados2-2:19.0.0-2946.g3d89accf.el9.x86_64 8/170 2024-06-10T02:34:43.885 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: librados2-2:19.0.0-2946.g3d89accf.el9.x86_64 8/170 2024-06-10T02:34:43.911 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : libcephfs2-2:19.0.0-2946.g3d89accf.el9.x86_64 9/170 2024-06-10T02:34:44.014 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: libcephfs2-2:19.0.0-2946.g3d89accf.el9.x86_64 9/170 2024-06-10T02:34:44.049 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-rados-2:19.0.0-2946.g3d89accf.el9.x86_64 10/170 2024-06-10T02:34:44.093 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/170 2024-06-10T02:34:44.307 INFO:teuthology.orchestra.run.smithi052.stdout: Upgrading : elfutils-libelf-0.191-4.el9.x86_64 12/170 2024-06-10T02:34:44.356 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : postgresql-13.14-1.el9.x86_64 13/170 2024-06-10T02:34:44.383 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : libcephsqlite-2:19.0.0-2946.g3d89accf.el9.x86_64 14/170 2024-06-10T02:34:44.473 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2946.g3d89accf.el9.x86_64 14/170 2024-06-10T02:34:44.501 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : libradosstriper1-2:19.0.0-2946.g3d89accf.el9.x86 15/170 2024-06-10T02:34:44.547 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2946.g3d89accf.el9.x86 15/170 2024-06-10T02:34:44.577 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-06-10T02:34:44.604 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-06-10T02:34:44.747 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/170 2024-06-10T02:34:44.927 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/170 2024-06-10T02:34:45.091 INFO:teuthology.orchestra.run.smithi052.stdout: Upgrading : NetworkManager-libnm-1:1.47.91-1.el9.x86_64 18/170 2024-06-10T02:34:45.152 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.91-1.el9.x86_64 18/170 2024-06-10T02:34:45.260 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: NetworkManager-1:1.47.91-1.el9.x86_64 19/170 2024-06-10T02:34:45.336 INFO:teuthology.orchestra.run.smithi052.stdout: Upgrading : NetworkManager-1:1.47.91-1.el9.x86_64 19/170 2024-06-10T02:34:45.540 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: NetworkManager-1:1.47.91-1.el9.x86_64 19/170 2024-06-10T02:34:45.646 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : re2-1:20211101-3.el9.x86_64 20/170 2024-06-10T02:34:45.751 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/170 2024-06-10T02:34:45.783 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 22/170 2024-06-10T02:34:45.819 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : liboath-2.6.7-2.el9.x86_64 23/170 2024-06-10T02:34:45.895 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 24/170 2024-06-10T02:34:45.935 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 25/170 2024-06-10T02:34:45.978 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/170 2024-06-10T02:34:46.010 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/170 2024-06-10T02:34:46.046 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/170 2024-06-10T02:34:46.119 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/170 2024-06-10T02:34:46.170 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 30/170 2024-06-10T02:34:46.199 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/170 2024-06-10T02:34:46.243 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 32/170 2024-06-10T02:34:46.321 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 33/170 2024-06-10T02:34:46.353 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-ceph-common-2:19.0.0-2946.g3d89accf.el9. 34/170 2024-06-10T02:34:46.397 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-ceph-argparse-2:19.0.0-2946.g3d89accf.el 35/170 2024-06-10T02:34:46.425 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-cephfs-2:19.0.0-2946.g3d89accf.el9.x86_6 36/170 2024-06-10T02:34:46.492 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/170 2024-06-10T02:34:46.540 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/170 2024-06-10T02:34:46.619 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/170 2024-06-10T02:34:46.776 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/170 2024-06-10T02:34:46.845 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/170 2024-06-10T02:34:47.416 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/170 2024-06-10T02:34:47.509 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/170 2024-06-10T02:34:47.542 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/170 2024-06-10T02:34:47.579 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/170 2024-06-10T02:34:47.627 INFO:teuthology.orchestra.run.smithi052.stdout: Upgrading : elfutils-libs-0.191-4.el9.x86_64 46/170 2024-06-10T02:34:47.666 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : librados-devel-2:19.0.0-2946.g3d89accf.el9.x86_6 47/170 2024-06-10T02:34:47.708 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 48/170 2024-06-10T02:34:47.875 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 49/170 2024-06-10T02:34:47.927 INFO:teuthology.orchestra.run.smithi052.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 50/170 2024-06-10T02:34:47.984 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 51/170 2024-06-10T02:34:48.058 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/170 2024-06-10T02:34:48.094 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 53/170 2024-06-10T02:34:48.131 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 54/170 2024-06-10T02:34:48.178 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/170 2024-06-10T02:34:48.221 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/170 2024-06-10T02:34:48.265 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/170 2024-06-10T02:34:48.307 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/170 2024-06-10T02:34:48.453 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/170 2024-06-10T02:34:48.492 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/170 2024-06-10T02:34:48.609 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 61/170 2024-06-10T02:34:48.803 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/170 2024-06-10T02:34:48.833 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-google-auth-1:2.29.0-2.el9.noarch 63/170 2024-06-10T02:34:48.874 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/170 2024-06-10T02:34:48.915 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 65/170 2024-06-10T02:34:48.939 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/170 2024-06-10T02:34:49.032 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/170 2024-06-10T02:34:49.102 INFO:teuthology.orchestra.run.smithi120.stdout: Preparing : 1/1 2024-06-10T02:34:49.167 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/170 2024-06-10T02:34:49.197 INFO:teuthology.orchestra.run.smithi120.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/170 2024-06-10T02:34:49.199 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/170 2024-06-10T02:34:49.227 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/170 2024-06-10T02:34:49.235 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/170 2024-06-10T02:34:49.277 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/170 2024-06-10T02:34:49.288 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/170 2024-06-10T02:34:49.318 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/170 2024-06-10T02:34:49.323 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/170 2024-06-10T02:34:49.349 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/170 2024-06-10T02:34:49.357 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/170 2024-06-10T02:34:49.387 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/170 2024-06-10T02:34:49.391 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/170 2024-06-10T02:34:49.421 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/170 2024-06-10T02:34:49.455 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/170 2024-06-10T02:34:49.497 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/170 2024-06-10T02:34:49.531 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/170 2024-06-10T02:34:49.623 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/170 2024-06-10T02:34:49.671 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/170 2024-06-10T02:34:49.701 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/170 2024-06-10T02:34:49.749 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/170 2024-06-10T02:34:49.861 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-packaging-20.9-5.el9.noarch 83/170 2024-06-10T02:34:49.913 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 84/170 2024-06-10T02:34:49.942 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/170 2024-06-10T02:34:50.041 INFO:teuthology.orchestra.run.smithi120.stdout: Upgrading : librados2-2:19.0.0-2946.g3d89accf.el9.x86_64 8/170 2024-06-10T02:34:50.165 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: librados2-2:19.0.0-2946.g3d89accf.el9.x86_64 8/170 2024-06-10T02:34:50.193 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : libcephfs2-2:19.0.0-2946.g3d89accf.el9.x86_64 9/170 2024-06-10T02:34:50.276 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: libcephfs2-2:19.0.0-2946.g3d89accf.el9.x86_64 9/170 2024-06-10T02:34:50.305 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-rados-2:19.0.0-2946.g3d89accf.el9.x86_64 10/170 2024-06-10T02:34:50.340 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/170 2024-06-10T02:34:50.538 INFO:teuthology.orchestra.run.smithi120.stdout: Upgrading : elfutils-libelf-0.191-4.el9.x86_64 12/170 2024-06-10T02:34:50.588 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : postgresql-13.14-1.el9.x86_64 13/170 2024-06-10T02:34:50.617 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : libcephsqlite-2:19.0.0-2946.g3d89accf.el9.x86_64 14/170 2024-06-10T02:34:50.711 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 85/170 2024-06-10T02:34:50.712 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2946.g3d89accf.el9.x86_64 14/170 2024-06-10T02:34:50.734 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : libradosstriper1-2:19.0.0-2946.g3d89accf.el9.x86 15/170 2024-06-10T02:34:50.758 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 86/170 2024-06-10T02:34:50.780 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2946.g3d89accf.el9.x86 15/170 2024-06-10T02:34:50.800 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 87/170 2024-06-10T02:34:50.802 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-06-10T02:34:50.828 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-06-10T02:34:50.915 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 88/170 2024-06-10T02:34:50.965 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 89/170 2024-06-10T02:34:51.010 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 90/170 2024-06-10T02:34:51.018 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/170 2024-06-10T02:34:51.110 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 91/170 2024-06-10T02:34:51.160 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/170 2024-06-10T02:34:51.161 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 92/170 2024-06-10T02:34:51.316 INFO:teuthology.orchestra.run.smithi120.stdout: Upgrading : NetworkManager-libnm-1:1.47.91-1.el9.x86_64 18/170 2024-06-10T02:34:51.346 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : openblas-0.3.21-2.el9.x86_64 93/170 2024-06-10T02:34:51.377 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.91-1.el9.x86_64 18/170 2024-06-10T02:34:51.384 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 94/170 2024-06-10T02:34:51.468 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 95/170 2024-06-10T02:34:51.488 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: NetworkManager-1:1.47.91-1.el9.x86_64 19/170 2024-06-10T02:34:51.585 INFO:teuthology.orchestra.run.smithi120.stdout: Upgrading : NetworkManager-1:1.47.91-1.el9.x86_64 19/170 2024-06-10T02:34:51.781 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: NetworkManager-1:1.47.91-1.el9.x86_64 19/170 2024-06-10T02:34:51.881 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : re2-1:20211101-3.el9.x86_64 20/170 2024-06-10T02:34:51.988 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/170 2024-06-10T02:34:52.024 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 22/170 2024-06-10T02:34:52.067 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : liboath-2.6.7-2.el9.x86_64 23/170 2024-06-10T02:34:52.132 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 24/170 2024-06-10T02:34:52.165 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 96/170 2024-06-10T02:34:52.166 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 25/170 2024-06-10T02:34:52.210 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/170 2024-06-10T02:34:52.242 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/170 2024-06-10T02:34:52.270 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/170 2024-06-10T02:34:52.279 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 97/170 2024-06-10T02:34:52.338 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/170 2024-06-10T02:34:52.392 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 30/170 2024-06-10T02:34:52.423 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/170 2024-06-10T02:34:52.458 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 32/170 2024-06-10T02:34:52.533 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 33/170 2024-06-10T02:34:52.568 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-ceph-common-2:19.0.0-2946.g3d89accf.el9. 34/170 2024-06-10T02:34:52.611 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-ceph-argparse-2:19.0.0-2946.g3d89accf.el 35/170 2024-06-10T02:34:52.640 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-cephfs-2:19.0.0-2946.g3d89accf.el9.x86_6 36/170 2024-06-10T02:34:52.712 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/170 2024-06-10T02:34:52.764 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/170 2024-06-10T02:34:52.838 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/170 2024-06-10T02:34:52.973 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/170 2024-06-10T02:34:53.025 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/170 2024-06-10T02:34:53.542 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/170 2024-06-10T02:34:53.628 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/170 2024-06-10T02:34:53.636 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 98/170 2024-06-10T02:34:53.674 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/170 2024-06-10T02:34:53.710 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/170 2024-06-10T02:34:53.721 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 99/170 2024-06-10T02:34:53.750 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 100/170 2024-06-10T02:34:53.767 INFO:teuthology.orchestra.run.smithi120.stdout: Upgrading : elfutils-libs-0.191-4.el9.x86_64 46/170 2024-06-10T02:34:53.783 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 101/170 2024-06-10T02:34:53.805 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : librados-devel-2:19.0.0-2946.g3d89accf.el9.x86_6 47/170 2024-06-10T02:34:53.855 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 48/170 2024-06-10T02:34:54.010 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 49/170 2024-06-10T02:34:54.058 INFO:teuthology.orchestra.run.smithi120.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 50/170 2024-06-10T02:34:54.111 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 102/170 2024-06-10T02:34:54.112 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 51/170 2024-06-10T02:34:54.154 INFO:teuthology.orchestra.run.smithi052.stdout: Upgrading : librbd1-2:19.0.0-2946.g3d89accf.el9.x86_64 103/170 2024-06-10T02:34:54.186 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/170 2024-06-10T02:34:54.217 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 53/170 2024-06-10T02:34:54.254 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 54/170 2024-06-10T02:34:54.271 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: librbd1-2:19.0.0-2946.g3d89accf.el9.x86_64 103/170 2024-06-10T02:34:54.300 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/170 2024-06-10T02:34:54.300 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-rbd-2:19.0.0-2946.g3d89accf.el9.x86_64 104/170 2024-06-10T02:34:54.336 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/170 2024-06-10T02:34:54.337 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 105/170 2024-06-10T02:34:54.379 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/170 2024-06-10T02:34:54.421 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/170 2024-06-10T02:34:54.557 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/170 2024-06-10T02:34:54.607 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/170 2024-06-10T02:34:54.647 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 61/170 2024-06-10T02:34:54.751 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/170 2024-06-10T02:34:54.790 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-google-auth-1:2.29.0-2.el9.noarch 63/170 2024-06-10T02:34:54.829 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/170 2024-06-10T02:34:54.835 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 106/170 2024-06-10T02:34:54.857 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 65/170 2024-06-10T02:34:54.865 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : librgw2-2:19.0.0-2946.g3d89accf.el9.x86_64 107/170 2024-06-10T02:34:54.879 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/170 2024-06-10T02:34:54.937 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: librgw2-2:19.0.0-2946.g3d89accf.el9.x86_64 107/170 2024-06-10T02:34:54.954 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/170 2024-06-10T02:34:54.967 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-rgw-2:19.0.0-2946.g3d89accf.el9.x86_64 108/170 2024-06-10T02:34:55.080 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/170 2024-06-10T02:34:55.120 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/170 2024-06-10T02:34:55.148 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/170 2024-06-10T02:34:55.182 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/170 2024-06-10T02:34:55.221 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/170 2024-06-10T02:34:55.253 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/170 2024-06-10T02:34:55.283 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/170 2024-06-10T02:34:55.328 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/170 2024-06-10T02:34:55.359 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/170 2024-06-10T02:34:55.400 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/170 2024-06-10T02:34:55.435 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/170 2024-06-10T02:34:55.521 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/170 2024-06-10T02:34:55.566 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/170 2024-06-10T02:34:55.597 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/170 2024-06-10T02:34:55.644 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/170 2024-06-10T02:34:55.742 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-packaging-20.9-5.el9.noarch 83/170 2024-06-10T02:34:55.798 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 84/170 2024-06-10T02:34:56.520 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 85/170 2024-06-10T02:34:56.562 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 86/170 2024-06-10T02:34:56.596 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 87/170 2024-06-10T02:34:56.694 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 88/170 2024-06-10T02:34:56.759 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 89/170 2024-06-10T02:34:56.804 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 90/170 2024-06-10T02:34:56.891 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 91/170 2024-06-10T02:34:56.923 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 92/170 2024-06-10T02:34:57.100 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : openblas-0.3.21-2.el9.x86_64 93/170 2024-06-10T02:34:57.130 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 94/170 2024-06-10T02:34:57.206 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 95/170 2024-06-10T02:34:57.315 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 109/170 2024-06-10T02:34:57.539 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 109/170 2024-06-10T02:34:57.855 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 96/170 2024-06-10T02:34:57.968 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 97/170 2024-06-10T02:34:58.134 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 109/170 2024-06-10T02:34:58.167 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : ceph-base-2:19.0.0-2946.g3d89accf.el9.x86_64 110/170 2024-06-10T02:34:58.255 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: ceph-base-2:19.0.0-2946.g3d89accf.el9.x86_64 110/170 2024-06-10T02:34:58.256 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-06-10T02:34:58.256 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-06-10T02:34:58.256 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-10T02:34:58.285 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : ceph-selinux-2:19.0.0-2946.g3d89accf.el9.x86_64 111/170 2024-06-10T02:34:59.190 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 98/170 2024-06-10T02:34:59.266 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 99/170 2024-06-10T02:34:59.294 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 100/170 2024-06-10T02:34:59.327 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 101/170 2024-06-10T02:34:59.637 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 102/170 2024-06-10T02:34:59.681 INFO:teuthology.orchestra.run.smithi120.stdout: Upgrading : librbd1-2:19.0.0-2946.g3d89accf.el9.x86_64 103/170 2024-06-10T02:34:59.773 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: librbd1-2:19.0.0-2946.g3d89accf.el9.x86_64 103/170 2024-06-10T02:34:59.803 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-rbd-2:19.0.0-2946.g3d89accf.el9.x86_64 104/170 2024-06-10T02:34:59.839 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 105/170 2024-06-10T02:35:00.313 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 106/170 2024-06-10T02:35:00.343 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : librgw2-2:19.0.0-2946.g3d89accf.el9.x86_64 107/170 2024-06-10T02:35:00.397 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: librgw2-2:19.0.0-2946.g3d89accf.el9.x86_64 107/170 2024-06-10T02:35:00.419 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-rgw-2:19.0.0-2946.g3d89accf.el9.x86_64 108/170 2024-06-10T02:35:02.681 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 109/170 2024-06-10T02:35:02.724 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 109/170 2024-06-10T02:35:03.261 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 109/170 2024-06-10T02:35:03.327 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-base-2:19.0.0-2946.g3d89accf.el9.x86_64 110/170 2024-06-10T02:35:03.420 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-base-2:19.0.0-2946.g3d89accf.el9.x86_64 110/170 2024-06-10T02:35:03.420 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-06-10T02:35:03.420 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-06-10T02:35:03.421 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-06-10T02:35:03.453 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-selinux-2:19.0.0-2946.g3d89accf.el9.x86_64 111/170 2024-06-10T02:35:10.104 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2946.g3d89accf.el9.x86_64 111/170 2024-06-10T02:35:10.104 INFO:teuthology.orchestra.run.smithi052.stdout:skipping the directory /sys 2024-06-10T02:35:10.104 INFO:teuthology.orchestra.run.smithi052.stdout:skipping the directory /proc 2024-06-10T02:35:10.104 INFO:teuthology.orchestra.run.smithi052.stdout:skipping the directory /mnt 2024-06-10T02:35:10.104 INFO:teuthology.orchestra.run.smithi052.stdout:skipping the directory /var/tmp 2024-06-10T02:35:10.104 INFO:teuthology.orchestra.run.smithi052.stdout:skipping the directory /home 2024-06-10T02:35:10.104 INFO:teuthology.orchestra.run.smithi052.stdout:skipping the directory /root 2024-06-10T02:35:10.104 INFO:teuthology.orchestra.run.smithi052.stdout:skipping the directory /tmp 2024-06-10T02:35:10.105 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-10T02:35:10.283 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : ceph-volume-2:19.0.0-2946.g3d89accf.el9.noarch 112/170 2024-06-10T02:35:10.299 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: ceph-volume-2:19.0.0-2946.g3d89accf.el9.noarch 112/170 2024-06-10T02:35:10.299 INFO:teuthology.orchestra.run.smithi052.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-10T02:35:10.299 INFO:teuthology.orchestra.run.smithi052.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-06-10T02:35:10.299 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-10T02:35:11.836 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : ceph-osd-2:19.0.0-2946.g3d89accf.el9.x86_64 113/170 2024-06-10T02:35:11.864 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: ceph-osd-2:19.0.0-2946.g3d89accf.el9.x86_64 113/170 2024-06-10T02:35:11.865 INFO:teuthology.orchestra.run.smithi052.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-10T02:35:11.865 INFO:teuthology.orchestra.run.smithi052.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-06-10T02:35:11.865 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-06-10T02:35:11.865 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-06-10T02:35:11.865 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-10T02:35:12.103 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : ceph-mds-2:19.0.0-2946.g3d89accf.el9.x86_64 114/170 2024-06-10T02:35:12.128 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: ceph-mds-2:19.0.0-2946.g3d89accf.el9.x86_64 114/170 2024-06-10T02:35:12.128 INFO:teuthology.orchestra.run.smithi052.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-10T02:35:12.128 INFO:teuthology.orchestra.run.smithi052.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-06-10T02:35:12.128 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-06-10T02:35:12.128 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-06-10T02:35:12.128 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-10T02:35:12.604 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : ceph-mon-2:19.0.0-2946.g3d89accf.el9.x86_64 115/170 2024-06-10T02:35:12.629 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: ceph-mon-2:19.0.0-2946.g3d89accf.el9.x86_64 115/170 2024-06-10T02:35:12.629 INFO:teuthology.orchestra.run.smithi052.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-10T02:35:12.629 INFO:teuthology.orchestra.run.smithi052.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-06-10T02:35:12.629 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-06-10T02:35:12.629 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-06-10T02:35:12.629 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-10T02:35:12.735 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : automake-1.16.2-8.el9.noarch 116/170 2024-06-10T02:35:12.794 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-ply-3.11-14.el9.noarch 117/170 2024-06-10T02:35:12.859 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 118/170 2024-06-10T02:35:13.066 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 119/170 2024-06-10T02:35:13.114 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 120/170 2024-06-10T02:35:13.196 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 121/170 2024-06-10T02:35:13.295 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 122/170 2024-06-10T02:35:13.440 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 123/170 2024-06-10T02:35:13.481 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 124/170 2024-06-10T02:35:13.513 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 125/170 2024-06-10T02:35:13.548 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : fuse-2.9.9-15.el9.x86_64 126/170 2024-06-10T02:35:13.980 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 127/170 2024-06-10T02:35:14.006 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 127/170 2024-06-10T02:35:14.038 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2946.g3d89accf.e 128/170 2024-06-10T02:35:14.119 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2946.g3d89accf. 129/170 2024-06-10T02:35:14.145 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2946.g3d89accf.el9.noa 130/170 2024-06-10T02:35:15.244 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2946.g3d89accf.el9.noa 130/170 2024-06-10T02:35:15.288 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2946.g3d89accf.el9.n 131/170 2024-06-10T02:35:15.604 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2946.g3d89accf.el9.x86_64 111/170 2024-06-10T02:35:15.604 INFO:teuthology.orchestra.run.smithi120.stdout:skipping the directory /sys 2024-06-10T02:35:15.604 INFO:teuthology.orchestra.run.smithi120.stdout:skipping the directory /proc 2024-06-10T02:35:15.604 INFO:teuthology.orchestra.run.smithi120.stdout:skipping the directory /mnt 2024-06-10T02:35:15.604 INFO:teuthology.orchestra.run.smithi120.stdout:skipping the directory /var/tmp 2024-06-10T02:35:15.604 INFO:teuthology.orchestra.run.smithi120.stdout:skipping the directory /home 2024-06-10T02:35:15.604 INFO:teuthology.orchestra.run.smithi120.stdout:skipping the directory /root 2024-06-10T02:35:15.604 INFO:teuthology.orchestra.run.smithi120.stdout:skipping the directory /tmp 2024-06-10T02:35:15.604 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-06-10T02:35:15.770 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-volume-2:19.0.0-2946.g3d89accf.el9.noarch 112/170 2024-06-10T02:35:15.785 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-volume-2:19.0.0-2946.g3d89accf.el9.noarch 112/170 2024-06-10T02:35:15.785 INFO:teuthology.orchestra.run.smithi120.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-10T02:35:15.785 INFO:teuthology.orchestra.run.smithi120.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-06-10T02:35:15.786 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-06-10T02:35:16.309 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2946.g3d89accf.el9.n 131/170 2024-06-10T02:35:16.339 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2946.g3d8 132/170 2024-06-10T02:35:16.449 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2946.g3d8 132/170 2024-06-10T02:35:16.620 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2946.g3d89accf.el 133/170 2024-06-10T02:35:16.651 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : ceph-mgr-2:19.0.0-2946.g3d89accf.el9.x86_64 134/170 2024-06-10T02:35:16.676 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2946.g3d89accf.el9.x86_64 134/170 2024-06-10T02:35:16.677 INFO:teuthology.orchestra.run.smithi052.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-10T02:35:16.677 INFO:teuthology.orchestra.run.smithi052.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-06-10T02:35:16.677 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-06-10T02:35:16.677 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-06-10T02:35:16.677 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-10T02:35:16.771 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : ceph-mgr-rook-2:19.0.0-2946.g3d89accf.el9.noarch 135/170 2024-06-10T02:35:16.783 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2946.g3d89accf.el9.noarch 135/170 2024-06-10T02:35:16.888 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : ceph-2:19.0.0-2946.g3d89accf.el9.x86_64 136/170 2024-06-10T02:35:16.951 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : ceph-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 137/170 2024-06-10T02:35:17.274 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-osd-2:19.0.0-2946.g3d89accf.el9.x86_64 113/170 2024-06-10T02:35:17.300 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-osd-2:19.0.0-2946.g3d89accf.el9.x86_64 113/170 2024-06-10T02:35:17.300 INFO:teuthology.orchestra.run.smithi120.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-10T02:35:17.300 INFO:teuthology.orchestra.run.smithi120.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-06-10T02:35:17.300 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-06-10T02:35:17.301 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-06-10T02:35:17.301 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-06-10T02:35:17.525 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-mds-2:19.0.0-2946.g3d89accf.el9.x86_64 114/170 2024-06-10T02:35:17.547 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-mds-2:19.0.0-2946.g3d89accf.el9.x86_64 114/170 2024-06-10T02:35:17.547 INFO:teuthology.orchestra.run.smithi120.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-10T02:35:17.547 INFO:teuthology.orchestra.run.smithi120.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-06-10T02:35:17.548 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-06-10T02:35:17.548 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-06-10T02:35:17.548 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-06-10T02:35:17.898 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : libtool-2.4.6-45.el9.x86_64 138/170 2024-06-10T02:35:17.928 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : ceph-radosgw-2:19.0.0-2946.g3d89accf.el9.x86_64 139/170 2024-06-10T02:35:17.953 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2946.g3d89accf.el9.x86_64 139/170 2024-06-10T02:35:17.953 INFO:teuthology.orchestra.run.smithi052.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-10T02:35:17.953 INFO:teuthology.orchestra.run.smithi052.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-06-10T02:35:17.954 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-06-10T02:35:17.954 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-06-10T02:35:17.954 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-10T02:35:17.992 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-mon-2:19.0.0-2946.g3d89accf.el9.x86_64 115/170 2024-06-10T02:35:18.012 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2946.g3d89a 140/170 2024-06-10T02:35:18.015 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-mon-2:19.0.0-2946.g3d89accf.el9.x86_64 115/170 2024-06-10T02:35:18.015 INFO:teuthology.orchestra.run.smithi120.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-10T02:35:18.015 INFO:teuthology.orchestra.run.smithi120.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-06-10T02:35:18.015 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-06-10T02:35:18.015 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-06-10T02:35:18.015 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-06-10T02:35:18.036 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2946.g3d89a 140/170 2024-06-10T02:35:18.036 INFO:teuthology.orchestra.run.smithi052.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-10T02:35:18.036 INFO:teuthology.orchestra.run.smithi052.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-06-10T02:35:18.036 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-10T02:35:18.087 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : cephfs-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 141/170 2024-06-10T02:35:18.111 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 141/170 2024-06-10T02:35:18.111 INFO:teuthology.orchestra.run.smithi052.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-10T02:35:18.111 INFO:teuthology.orchestra.run.smithi052.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-06-10T02:35:18.112 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-06-10T02:35:18.112 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-06-10T02:35:18.112 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-10T02:35:18.122 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : automake-1.16.2-8.el9.noarch 116/170 2024-06-10T02:35:18.171 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-ply-3.11-14.el9.noarch 117/170 2024-06-10T02:35:18.226 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 118/170 2024-06-10T02:35:18.412 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 119/170 2024-06-10T02:35:18.429 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : rbd-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 142/170 2024-06-10T02:35:18.454 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 142/170 2024-06-10T02:35:18.454 INFO:teuthology.orchestra.run.smithi052.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-10T02:35:18.454 INFO:teuthology.orchestra.run.smithi052.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-06-10T02:35:18.454 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-06-10T02:35:18.454 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-06-10T02:35:18.454 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-10T02:35:18.461 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 120/170 2024-06-10T02:35:18.538 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 121/170 2024-06-10T02:35:18.626 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 122/170 2024-06-10T02:35:18.756 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 123/170 2024-06-10T02:35:18.803 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 124/170 2024-06-10T02:35:18.835 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 125/170 2024-06-10T02:35:18.862 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : fuse-2.9.9-15.el9.x86_64 126/170 2024-06-10T02:35:19.306 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 127/170 2024-06-10T02:35:19.336 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 127/170 2024-06-10T02:35:19.368 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2946.g3d89accf.e 128/170 2024-06-10T02:35:19.429 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2946.g3d89accf. 129/170 2024-06-10T02:35:19.459 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2946.g3d89accf.el9.noa 130/170 2024-06-10T02:35:20.488 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2946.g3d89accf.el9.noa 130/170 2024-06-10T02:35:20.520 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2946.g3d89accf.el9.n 131/170 2024-06-10T02:35:21.480 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2946.g3d89accf.el9.n 131/170 2024-06-10T02:35:21.512 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2946.g3d8 132/170 2024-06-10T02:35:21.613 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2946.g3d8 132/170 2024-06-10T02:35:21.776 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2946.g3d89accf.el 133/170 2024-06-10T02:35:21.814 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-mgr-2:19.0.0-2946.g3d89accf.el9.x86_64 134/170 2024-06-10T02:35:21.837 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2946.g3d89accf.el9.x86_64 134/170 2024-06-10T02:35:21.837 INFO:teuthology.orchestra.run.smithi120.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-10T02:35:21.837 INFO:teuthology.orchestra.run.smithi120.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-06-10T02:35:21.837 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-06-10T02:35:21.837 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-06-10T02:35:21.837 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-06-10T02:35:21.890 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-mgr-rook-2:19.0.0-2946.g3d89accf.el9.noarch 135/170 2024-06-10T02:35:21.901 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2946.g3d89accf.el9.noarch 135/170 2024-06-10T02:35:21.998 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-2:19.0.0-2946.g3d89accf.el9.x86_64 136/170 2024-06-10T02:35:22.046 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 137/170 2024-06-10T02:35:22.946 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : libtool-2.4.6-45.el9.x86_64 138/170 2024-06-10T02:35:22.976 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-radosgw-2:19.0.0-2946.g3d89accf.el9.x86_64 139/170 2024-06-10T02:35:22.999 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2946.g3d89accf.el9.x86_64 139/170 2024-06-10T02:35:22.999 INFO:teuthology.orchestra.run.smithi120.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-10T02:35:22.999 INFO:teuthology.orchestra.run.smithi120.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-06-10T02:35:22.999 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-06-10T02:35:22.999 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-06-10T02:35:22.999 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-06-10T02:35:23.043 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2946.g3d89a 140/170 2024-06-10T02:35:23.065 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2946.g3d89a 140/170 2024-06-10T02:35:23.065 INFO:teuthology.orchestra.run.smithi120.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-10T02:35:23.066 INFO:teuthology.orchestra.run.smithi120.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-06-10T02:35:23.066 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-06-10T02:35:23.111 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : cephfs-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 141/170 2024-06-10T02:35:23.133 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 141/170 2024-06-10T02:35:23.133 INFO:teuthology.orchestra.run.smithi120.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-10T02:35:23.133 INFO:teuthology.orchestra.run.smithi120.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-06-10T02:35:23.133 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-06-10T02:35:23.133 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-06-10T02:35:23.134 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-06-10T02:35:23.241 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : ceph-test-2:19.0.0-2946.g3d89accf.el9.x86_64 143/170 2024-06-10T02:35:23.283 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : rbd-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 144/170 2024-06-10T02:35:23.325 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : rbd-nbd-2:19.0.0-2946.g3d89accf.el9.x86_64 145/170 2024-06-10T02:35:23.356 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 146/170 2024-06-10T02:35:23.383 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 147/170 2024-06-10T02:35:23.445 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : rbd-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 142/170 2024-06-10T02:35:23.467 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 142/170 2024-06-10T02:35:23.467 INFO:teuthology.orchestra.run.smithi120.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-10T02:35:23.467 INFO:teuthology.orchestra.run.smithi120.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-06-10T02:35:23.467 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-06-10T02:35:23.467 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-06-10T02:35:23.468 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-06-10T02:35:23.597 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : cephfs-shell-2:19.0.0-2946.g3d89accf.el9.x86_64 148/170 2024-06-10T02:35:23.645 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 149/170 2024-06-10T02:35:23.678 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : libcephfs-devel-2:19.0.0-2946.g3d89accf.el9.x86_ 150/170 2024-06-10T02:35:23.709 INFO:teuthology.orchestra.run.smithi052.stdout: Upgrading : elfutils-debuginfod-client-0.191-4.el9.x86_64 151/170 2024-06-10T02:35:23.739 INFO:teuthology.orchestra.run.smithi052.stdout: Upgrading : NetworkManager-team-1:1.47.91-1.el9.x86_64 152/170 2024-06-10T02:35:23.776 INFO:teuthology.orchestra.run.smithi052.stdout: Upgrading : NetworkManager-tui-1:1.47.91-1.el9.x86_64 153/170 2024-06-10T02:35:24.393 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-06-10T02:35:24.440 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 154/170 2024-06-10T02:35:24.462 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-06-10T02:35:25.402 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : cephfs-top-2:19.0.0-2946.g3d89accf.el9.noarch 155/170 2024-06-10T02:35:25.486 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 156/170 2024-06-10T02:35:25.530 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 157/170 2024-06-10T02:35:25.636 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : flex-2.6.4-9.el9.x86_64 158/170 2024-06-10T02:35:25.673 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : bison-3.7.4-5.el9.x86_64 159/170 2024-06-10T02:35:25.698 INFO:teuthology.orchestra.run.smithi052.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-06-10T02:35:25.723 INFO:teuthology.orchestra.run.smithi052.stdout: Cleanup : elfutils-debuginfod-client-0.190-2.el9.x86_64 161/170 2024-06-10T02:35:25.750 INFO:teuthology.orchestra.run.smithi052.stdout: Cleanup : elfutils-libs-0.190-2.el9.x86_64 162/170 2024-06-10T02:35:25.750 INFO:teuthology.orchestra.run.smithi052.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-06-10T02:35:25.826 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-06-10T02:35:25.859 INFO:teuthology.orchestra.run.smithi052.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 164/170 2024-06-10T02:35:25.859 INFO:teuthology.orchestra.run.smithi052.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 165/170 2024-06-10T02:35:25.935 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 165/170 2024-06-10T02:35:25.959 INFO:teuthology.orchestra.run.smithi052.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 166/170 2024-06-10T02:35:25.968 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-06-10T02:35:25.969 INFO:teuthology.orchestra.run.smithi052.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-06-10T02:35:26.153 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-06-10T02:35:26.153 INFO:teuthology.orchestra.run.smithi052.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-06-10T02:35:26.229 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-06-10T02:35:26.254 INFO:teuthology.orchestra.run.smithi052.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 169/170 2024-06-10T02:35:26.322 INFO:teuthology.orchestra.run.smithi052.stdout: Cleanup : elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-06-10T02:35:27.957 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-test-2:19.0.0-2946.g3d89accf.el9.x86_64 143/170 2024-06-10T02:35:27.998 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : rbd-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 144/170 2024-06-10T02:35:28.039 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : rbd-nbd-2:19.0.0-2946.g3d89accf.el9.x86_64 145/170 2024-06-10T02:35:28.071 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 146/170 2024-06-10T02:35:28.098 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 147/170 2024-06-10T02:35:28.281 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : cephfs-shell-2:19.0.0-2946.g3d89accf.el9.x86_64 148/170 2024-06-10T02:35:28.329 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 149/170 2024-06-10T02:35:28.352 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : libcephfs-devel-2:19.0.0-2946.g3d89accf.el9.x86_ 150/170 2024-06-10T02:35:28.402 INFO:teuthology.orchestra.run.smithi120.stdout: Upgrading : elfutils-debuginfod-client-0.191-4.el9.x86_64 151/170 2024-06-10T02:35:28.431 INFO:teuthology.orchestra.run.smithi120.stdout: Upgrading : NetworkManager-team-1:1.47.91-1.el9.x86_64 152/170 2024-06-10T02:35:28.469 INFO:teuthology.orchestra.run.smithi120.stdout: Upgrading : NetworkManager-tui-1:1.47.91-1.el9.x86_64 153/170 2024-06-10T02:35:29.089 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-06-10T02:35:29.274 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 154/170 2024-06-10T02:35:29.275 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-06-10T02:35:29.972 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : cephfs-top-2:19.0.0-2946.g3d89accf.el9.noarch 155/170 2024-06-10T02:35:30.066 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 156/170 2024-06-10T02:35:30.106 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 157/170 2024-06-10T02:35:30.188 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : flex-2.6.4-9.el9.x86_64 158/170 2024-06-10T02:35:30.236 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : bison-3.7.4-5.el9.x86_64 159/170 2024-06-10T02:35:30.260 INFO:teuthology.orchestra.run.smithi120.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-06-10T02:35:30.286 INFO:teuthology.orchestra.run.smithi120.stdout: Cleanup : elfutils-debuginfod-client-0.190-2.el9.x86_64 161/170 2024-06-10T02:35:30.312 INFO:teuthology.orchestra.run.smithi120.stdout: Cleanup : elfutils-libs-0.190-2.el9.x86_64 162/170 2024-06-10T02:35:30.312 INFO:teuthology.orchestra.run.smithi120.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-06-10T02:35:30.397 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-06-10T02:35:30.430 INFO:teuthology.orchestra.run.smithi120.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 164/170 2024-06-10T02:35:30.430 INFO:teuthology.orchestra.run.smithi120.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 165/170 2024-06-10T02:35:30.514 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 165/170 2024-06-10T02:35:30.539 INFO:teuthology.orchestra.run.smithi120.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 166/170 2024-06-10T02:35:30.545 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-06-10T02:35:30.545 INFO:teuthology.orchestra.run.smithi120.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-06-10T02:35:30.725 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-06-10T02:35:30.725 INFO:teuthology.orchestra.run.smithi120.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-06-10T02:35:30.793 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-06-10T02:35:30.817 INFO:teuthology.orchestra.run.smithi120.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 169/170 2024-06-10T02:35:30.882 INFO:teuthology.orchestra.run.smithi120.stdout: Cleanup : elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-06-10T02:35:31.720 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-06-10T02:35:31.720 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : ceph-2:19.0.0-2946.g3d89accf.el9.x86_64 1/170 2024-06-10T02:35:31.720 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : ceph-base-2:19.0.0-2946.g3d89accf.el9.x86_64 2/170 2024-06-10T02:35:31.720 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 3/170 2024-06-10T02:35:31.720 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : ceph-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 4/170 2024-06-10T02:35:31.720 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2946.g3d89a 5/170 2024-06-10T02:35:31.720 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : ceph-mds-2:19.0.0-2946.g3d89accf.el9.x86_64 6/170 2024-06-10T02:35:31.720 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : ceph-mgr-2:19.0.0-2946.g3d89accf.el9.x86_64 7/170 2024-06-10T02:35:31.721 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : ceph-mon-2:19.0.0-2946.g3d89accf.el9.x86_64 8/170 2024-06-10T02:35:31.721 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : ceph-osd-2:19.0.0-2946.g3d89accf.el9.x86_64 9/170 2024-06-10T02:35:31.721 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : ceph-radosgw-2:19.0.0-2946.g3d89accf.el9.x86_64 10/170 2024-06-10T02:35:31.721 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : ceph-selinux-2:19.0.0-2946.g3d89accf.el9.x86_64 11/170 2024-06-10T02:35:31.721 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : ceph-test-2:19.0.0-2946.g3d89accf.el9.x86_64 12/170 2024-06-10T02:35:31.721 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : cephfs-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 13/170 2024-06-10T02:35:31.721 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : cephfs-shell-2:19.0.0-2946.g3d89accf.el9.x86_64 14/170 2024-06-10T02:35:31.721 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libcephfs-devel-2:19.0.0-2946.g3d89accf.el9.x86_ 15/170 2024-06-10T02:35:31.721 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libcephfs2-2:19.0.0-2946.g3d89accf.el9.x86_64 16/170 2024-06-10T02:35:31.722 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libcephsqlite-2:19.0.0-2946.g3d89accf.el9.x86_64 17/170 2024-06-10T02:35:31.722 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : librados-devel-2:19.0.0-2946.g3d89accf.el9.x86_6 18/170 2024-06-10T02:35:31.722 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libradosstriper1-2:19.0.0-2946.g3d89accf.el9.x86 19/170 2024-06-10T02:35:31.722 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : librgw2-2:19.0.0-2946.g3d89accf.el9.x86_64 20/170 2024-06-10T02:35:31.722 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2946.g3d89accf.el 21/170 2024-06-10T02:35:31.722 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-ceph-common-2:19.0.0-2946.g3d89accf.el9. 22/170 2024-06-10T02:35:31.722 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-cephfs-2:19.0.0-2946.g3d89accf.el9.x86_6 23/170 2024-06-10T02:35:31.722 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-rados-2:19.0.0-2946.g3d89accf.el9.x86_64 24/170 2024-06-10T02:35:31.722 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-rbd-2:19.0.0-2946.g3d89accf.el9.x86_64 25/170 2024-06-10T02:35:31.722 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-rgw-2:19.0.0-2946.g3d89accf.el9.x86_64 26/170 2024-06-10T02:35:31.722 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : rbd-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 27/170 2024-06-10T02:35:31.722 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : rbd-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 28/170 2024-06-10T02:35:31.723 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : rbd-nbd-2:19.0.0-2946.g3d89accf.el9.x86_64 29/170 2024-06-10T02:35:31.735 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2946.g3d89accf. 30/170 2024-06-10T02:35:31.735 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2946.g3d89accf.el9.noa 31/170 2024-06-10T02:35:31.735 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2946.g3d89accf.el9.n 32/170 2024-06-10T02:35:31.735 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2946.g3d8 33/170 2024-06-10T02:35:31.735 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2946.g3d89accf.el 34/170 2024-06-10T02:35:31.735 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2946.g3d89accf.el9.noarch 35/170 2024-06-10T02:35:31.735 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2946.g3d89accf.e 36/170 2024-06-10T02:35:31.735 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : ceph-volume-2:19.0.0-2946.g3d89accf.el9.noarch 37/170 2024-06-10T02:35:31.735 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 38/170 2024-06-10T02:35:31.736 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : cephfs-top-2:19.0.0-2946.g3d89accf.el9.noarch 39/170 2024-06-10T02:35:31.736 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/170 2024-06-10T02:35:31.736 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/170 2024-06-10T02:35:31.736 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/170 2024-06-10T02:35:31.736 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/170 2024-06-10T02:35:31.736 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/170 2024-06-10T02:35:31.736 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/170 2024-06-10T02:35:31.736 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/170 2024-06-10T02:35:31.736 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/170 2024-06-10T02:35:31.736 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/170 2024-06-10T02:35:31.736 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/170 2024-06-10T02:35:31.736 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/170 2024-06-10T02:35:31.737 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/170 2024-06-10T02:35:31.737 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/170 2024-06-10T02:35:31.737 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/170 2024-06-10T02:35:31.737 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/170 2024-06-10T02:35:31.737 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/170 2024-06-10T02:35:31.737 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/170 2024-06-10T02:35:31.737 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/170 2024-06-10T02:35:31.737 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/170 2024-06-10T02:35:31.737 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/170 2024-06-10T02:35:31.737 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/170 2024-06-10T02:35:31.737 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/170 2024-06-10T02:35:31.737 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/170 2024-06-10T02:35:31.738 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/170 2024-06-10T02:35:31.738 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/170 2024-06-10T02:35:31.738 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/170 2024-06-10T02:35:31.738 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/170 2024-06-10T02:35:31.738 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/170 2024-06-10T02:35:31.738 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/170 2024-06-10T02:35:31.738 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/170 2024-06-10T02:35:31.738 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/170 2024-06-10T02:35:31.738 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/170 2024-06-10T02:35:31.738 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/170 2024-06-10T02:35:31.738 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/170 2024-06-10T02:35:31.738 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/170 2024-06-10T02:35:31.739 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/170 2024-06-10T02:35:31.739 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/170 2024-06-10T02:35:31.739 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 77/170 2024-06-10T02:35:31.739 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/170 2024-06-10T02:35:31.739 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/170 2024-06-10T02:35:31.739 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/170 2024-06-10T02:35:31.739 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/170 2024-06-10T02:35:31.739 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/170 2024-06-10T02:35:31.739 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/170 2024-06-10T02:35:31.739 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/170 2024-06-10T02:35:31.739 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/170 2024-06-10T02:35:31.739 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/170 2024-06-10T02:35:31.740 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/170 2024-06-10T02:35:31.740 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/170 2024-06-10T02:35:31.740 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/170 2024-06-10T02:35:31.740 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/170 2024-06-10T02:35:31.740 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/170 2024-06-10T02:35:31.740 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/170 2024-06-10T02:35:31.740 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/170 2024-06-10T02:35:31.740 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/170 2024-06-10T02:35:31.740 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/170 2024-06-10T02:35:31.740 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/170 2024-06-10T02:35:31.740 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/170 2024-06-10T02:35:31.740 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/170 2024-06-10T02:35:31.740 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 99/170 2024-06-10T02:35:31.740 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/170 2024-06-10T02:35:31.741 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 101/170 2024-06-10T02:35:31.741 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/170 2024-06-10T02:35:31.741 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/170 2024-06-10T02:35:31.741 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/170 2024-06-10T02:35:31.741 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/170 2024-06-10T02:35:31.741 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/170 2024-06-10T02:35:31.741 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/170 2024-06-10T02:35:31.741 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/170 2024-06-10T02:35:31.741 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/170 2024-06-10T02:35:31.741 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/170 2024-06-10T02:35:31.741 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/170 2024-06-10T02:35:31.741 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/170 2024-06-10T02:35:31.742 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/170 2024-06-10T02:35:31.742 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/170 2024-06-10T02:35:31.742 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/170 2024-06-10T02:35:31.742 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/170 2024-06-10T02:35:31.742 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/170 2024-06-10T02:35:31.742 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 118/170 2024-06-10T02:35:31.742 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/170 2024-06-10T02:35:31.742 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/170 2024-06-10T02:35:31.742 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/170 2024-06-10T02:35:31.742 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-google-auth-1:2.29.0-2.el9.noarch 122/170 2024-06-10T02:35:31.742 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/170 2024-06-10T02:35:31.742 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/170 2024-06-10T02:35:31.742 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/170 2024-06-10T02:35:31.743 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/170 2024-06-10T02:35:31.743 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/170 2024-06-10T02:35:31.743 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/170 2024-06-10T02:35:31.743 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/170 2024-06-10T02:35:31.743 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/170 2024-06-10T02:35:31.743 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/170 2024-06-10T02:35:31.743 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/170 2024-06-10T02:35:31.743 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/170 2024-06-10T02:35:31.743 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/170 2024-06-10T02:35:31.743 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/170 2024-06-10T02:35:31.743 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/170 2024-06-10T02:35:31.743 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/170 2024-06-10T02:35:31.744 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/170 2024-06-10T02:35:31.744 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/170 2024-06-10T02:35:31.744 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/170 2024-06-10T02:35:31.744 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/170 2024-06-10T02:35:31.744 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/170 2024-06-10T02:35:31.744 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/170 2024-06-10T02:35:31.744 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/170 2024-06-10T02:35:31.744 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/170 2024-06-10T02:35:31.744 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/170 2024-06-10T02:35:31.744 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/170 2024-06-10T02:35:31.744 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/170 2024-06-10T02:35:31.744 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : librados2-2:19.0.0-2946.g3d89accf.el9.x86_64 149/170 2024-06-10T02:35:31.744 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/170 2024-06-10T02:35:31.745 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : librbd1-2:19.0.0-2946.g3d89accf.el9.x86_64 151/170 2024-06-10T02:35:31.745 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/170 2024-06-10T02:35:31.745 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : NetworkManager-1:1.47.91-1.el9.x86_64 153/170 2024-06-10T02:35:31.745 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/170 2024-06-10T02:35:31.745 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : NetworkManager-libnm-1:1.47.91-1.el9.x86_64 155/170 2024-06-10T02:35:31.745 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/170 2024-06-10T02:35:31.745 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : NetworkManager-team-1:1.47.91-1.el9.x86_64 157/170 2024-06-10T02:35:31.745 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/170 2024-06-10T02:35:31.745 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : NetworkManager-tui-1:1.47.91-1.el9.x86_64 159/170 2024-06-10T02:35:31.745 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-06-10T02:35:31.745 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : elfutils-debuginfod-client-0.191-4.el9.x86_64 161/170 2024-06-10T02:35:31.745 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 162/170 2024-06-10T02:35:31.745 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : elfutils-libelf-0.191-4.el9.x86_64 163/170 2024-06-10T02:35:31.746 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 164/170 2024-06-10T02:35:31.746 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : elfutils-libs-0.191-4.el9.x86_64 165/170 2024-06-10T02:35:31.746 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 166/170 2024-06-10T02:35:31.746 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 167/170 2024-06-10T02:35:31.746 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 168/170 2024-06-10T02:35:31.746 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 169/170 2024-06-10T02:35:34.530 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 170/170 2024-06-10T02:35:34.530 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-10T02:35:34.530 INFO:teuthology.orchestra.run.smithi052.stdout:Upgraded: 2024-06-10T02:35:34.530 INFO:teuthology.orchestra.run.smithi052.stdout: NetworkManager-1:1.47.91-1.el9.x86_64 2024-06-10T02:35:34.530 INFO:teuthology.orchestra.run.smithi052.stdout: NetworkManager-libnm-1:1.47.91-1.el9.x86_64 2024-06-10T02:35:34.530 INFO:teuthology.orchestra.run.smithi052.stdout: NetworkManager-team-1:1.47.91-1.el9.x86_64 2024-06-10T02:35:34.530 INFO:teuthology.orchestra.run.smithi052.stdout: NetworkManager-tui-1:1.47.91-1.el9.x86_64 2024-06-10T02:35:34.530 INFO:teuthology.orchestra.run.smithi052.stdout: elfutils-debuginfod-client-0.191-4.el9.x86_64 2024-06-10T02:35:34.531 INFO:teuthology.orchestra.run.smithi052.stdout: elfutils-libelf-0.191-4.el9.x86_64 2024-06-10T02:35:34.531 INFO:teuthology.orchestra.run.smithi052.stdout: elfutils-libs-0.191-4.el9.x86_64 2024-06-10T02:35:34.531 INFO:teuthology.orchestra.run.smithi052.stdout: librados2-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T02:35:34.531 INFO:teuthology.orchestra.run.smithi052.stdout: librbd1-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T02:35:34.531 INFO:teuthology.orchestra.run.smithi052.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-06-10T02:35:34.531 INFO:teuthology.orchestra.run.smithi052.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-06-10T02:35:34.531 INFO:teuthology.orchestra.run.smithi052.stdout:Installed: 2024-06-10T02:35:34.531 INFO:teuthology.orchestra.run.smithi052.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-06-10T02:35:34.531 INFO:teuthology.orchestra.run.smithi052.stdout: automake-1.16.2-8.el9.noarch 2024-06-10T02:35:34.531 INFO:teuthology.orchestra.run.smithi052.stdout: bison-3.7.4-5.el9.x86_64 2024-06-10T02:35:34.531 INFO:teuthology.orchestra.run.smithi052.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-06-10T02:35:34.531 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T02:35:34.531 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-base-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T02:35:34.531 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T02:35:34.532 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T02:35:34.532 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-grafana-dashboards-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-10T02:35:34.532 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-immutable-object-cache-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T02:35:34.532 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-mds-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T02:35:34.532 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-mgr-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T02:35:34.532 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-mgr-cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-10T02:35:34.532 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-mgr-dashboard-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-10T02:35:34.532 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-10T02:35:34.532 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-mgr-modules-core-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-10T02:35:34.532 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-mgr-rook-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-10T02:35:34.532 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-mon-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T02:35:34.532 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-osd-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T02:35:34.532 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-prometheus-alerts-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-10T02:35:34.532 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-radosgw-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T02:35:34.533 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-selinux-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T02:35:34.533 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-test-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T02:35:34.533 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-volume-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-10T02:35:34.533 INFO:teuthology.orchestra.run.smithi052.stdout: cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-10T02:35:34.533 INFO:teuthology.orchestra.run.smithi052.stdout: cephfs-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T02:35:34.533 INFO:teuthology.orchestra.run.smithi052.stdout: cephfs-shell-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T02:35:34.533 INFO:teuthology.orchestra.run.smithi052.stdout: cephfs-top-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-10T02:35:34.533 INFO:teuthology.orchestra.run.smithi052.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-06-10T02:35:34.533 INFO:teuthology.orchestra.run.smithi052.stdout: flex-2.6.4-9.el9.x86_64 2024-06-10T02:35:34.533 INFO:teuthology.orchestra.run.smithi052.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-06-10T02:35:34.533 INFO:teuthology.orchestra.run.smithi052.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-06-10T02:35:34.533 INFO:teuthology.orchestra.run.smithi052.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-06-10T02:35:34.533 INFO:teuthology.orchestra.run.smithi052.stdout: fuse-2.9.9-15.el9.x86_64 2024-06-10T02:35:34.533 INFO:teuthology.orchestra.run.smithi052.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-06-10T02:35:34.534 INFO:teuthology.orchestra.run.smithi052.stdout: grpc-data-1.46.7-10.el9.noarch 2024-06-10T02:35:34.534 INFO:teuthology.orchestra.run.smithi052.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-06-10T02:35:34.534 INFO:teuthology.orchestra.run.smithi052.stdout: libarrow-9.0.0-11.el9.x86_64 2024-06-10T02:35:34.534 INFO:teuthology.orchestra.run.smithi052.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-06-10T02:35:34.534 INFO:teuthology.orchestra.run.smithi052.stdout: libcephfs-devel-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T02:35:34.534 INFO:teuthology.orchestra.run.smithi052.stdout: libcephfs2-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T02:35:34.534 INFO:teuthology.orchestra.run.smithi052.stdout: libcephsqlite-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T02:35:34.534 INFO:teuthology.orchestra.run.smithi052.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-06-10T02:35:34.534 INFO:teuthology.orchestra.run.smithi052.stdout: liboath-2.6.7-2.el9.x86_64 2024-06-10T02:35:34.534 INFO:teuthology.orchestra.run.smithi052.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-06-10T02:35:34.534 INFO:teuthology.orchestra.run.smithi052.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-06-10T02:35:34.534 INFO:teuthology.orchestra.run.smithi052.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-06-10T02:35:34.535 INFO:teuthology.orchestra.run.smithi052.stdout: librados-devel-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T02:35:34.535 INFO:teuthology.orchestra.run.smithi052.stdout: libradosstriper1-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T02:35:34.535 INFO:teuthology.orchestra.run.smithi052.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-06-10T02:35:34.535 INFO:teuthology.orchestra.run.smithi052.stdout: librgw2-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T02:35:34.535 INFO:teuthology.orchestra.run.smithi052.stdout: libtool-2.4.6-45.el9.x86_64 2024-06-10T02:35:34.535 INFO:teuthology.orchestra.run.smithi052.stdout: libunwind-1.6.2-1.el9.x86_64 2024-06-10T02:35:34.535 INFO:teuthology.orchestra.run.smithi052.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-06-10T02:35:34.535 INFO:teuthology.orchestra.run.smithi052.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-06-10T02:35:34.535 INFO:teuthology.orchestra.run.smithi052.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-06-10T02:35:34.535 INFO:teuthology.orchestra.run.smithi052.stdout: lua-5.4.4-4.el9.x86_64 2024-06-10T02:35:34.535 INFO:teuthology.orchestra.run.smithi052.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-06-10T02:35:34.535 INFO:teuthology.orchestra.run.smithi052.stdout: luarocks-3.9.2-5.el9.noarch 2024-06-10T02:35:34.535 INFO:teuthology.orchestra.run.smithi052.stdout: openblas-0.3.21-2.el9.x86_64 2024-06-10T02:35:34.536 INFO:teuthology.orchestra.run.smithi052.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-06-10T02:35:34.536 INFO:teuthology.orchestra.run.smithi052.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-06-10T02:35:34.536 INFO:teuthology.orchestra.run.smithi052.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-06-10T02:35:34.536 INFO:teuthology.orchestra.run.smithi052.stdout: postgresql-13.14-1.el9.x86_64 2024-06-10T02:35:34.536 INFO:teuthology.orchestra.run.smithi052.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-06-10T02:35:34.536 INFO:teuthology.orchestra.run.smithi052.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-06-10T02:35:34.536 INFO:teuthology.orchestra.run.smithi052.stdout: postgresql-server-13.14-1.el9.x86_64 2024-06-10T02:35:34.536 INFO:teuthology.orchestra.run.smithi052.stdout: protobuf-3.14.0-13.el9.x86_64 2024-06-10T02:35:34.536 INFO:teuthology.orchestra.run.smithi052.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-06-10T02:35:34.536 INFO:teuthology.orchestra.run.smithi052.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-06-10T02:35:34.536 INFO:teuthology.orchestra.run.smithi052.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-06-10T02:35:34.536 INFO:teuthology.orchestra.run.smithi052.stdout: python3-babel-2.9.1-2.el9.noarch 2024-06-10T02:35:34.536 INFO:teuthology.orchestra.run.smithi052.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-06-10T02:35:34.537 INFO:teuthology.orchestra.run.smithi052.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-06-10T02:35:34.537 INFO:teuthology.orchestra.run.smithi052.stdout: python3-ceph-argparse-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T02:35:34.537 INFO:teuthology.orchestra.run.smithi052.stdout: python3-ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T02:35:34.537 INFO:teuthology.orchestra.run.smithi052.stdout: python3-cephfs-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T02:35:34.537 INFO:teuthology.orchestra.run.smithi052.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-06-10T02:35:34.537 INFO:teuthology.orchestra.run.smithi052.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-06-10T02:35:34.537 INFO:teuthology.orchestra.run.smithi052.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-06-10T02:35:34.537 INFO:teuthology.orchestra.run.smithi052.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-06-10T02:35:34.537 INFO:teuthology.orchestra.run.smithi052.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-06-10T02:35:34.537 INFO:teuthology.orchestra.run.smithi052.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-06-10T02:35:34.537 INFO:teuthology.orchestra.run.smithi052.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-06-10T02:35:34.537 INFO:teuthology.orchestra.run.smithi052.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-06-10T02:35:34.537 INFO:teuthology.orchestra.run.smithi052.stdout: python3-google-auth-1:2.29.0-2.el9.noarch 2024-06-10T02:35:34.537 INFO:teuthology.orchestra.run.smithi052.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-06-10T02:35:34.538 INFO:teuthology.orchestra.run.smithi052.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-06-10T02:35:34.538 INFO:teuthology.orchestra.run.smithi052.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-06-10T02:35:34.538 INFO:teuthology.orchestra.run.smithi052.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-06-10T02:35:34.538 INFO:teuthology.orchestra.run.smithi052.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-06-10T02:35:34.538 INFO:teuthology.orchestra.run.smithi052.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-06-10T02:35:34.538 INFO:teuthology.orchestra.run.smithi052.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-06-10T02:35:34.538 INFO:teuthology.orchestra.run.smithi052.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-06-10T02:35:34.538 INFO:teuthology.orchestra.run.smithi052.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-06-10T02:35:34.538 INFO:teuthology.orchestra.run.smithi052.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-06-10T02:35:34.538 INFO:teuthology.orchestra.run.smithi052.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-06-10T02:35:34.538 INFO:teuthology.orchestra.run.smithi052.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-06-10T02:35:34.538 INFO:teuthology.orchestra.run.smithi052.stdout: python3-mako-1.1.4-6.el9.noarch 2024-06-10T02:35:34.539 INFO:teuthology.orchestra.run.smithi052.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-06-10T02:35:34.539 INFO:teuthology.orchestra.run.smithi052.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-06-10T02:35:34.539 INFO:teuthology.orchestra.run.smithi052.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-06-10T02:35:34.539 INFO:teuthology.orchestra.run.smithi052.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-06-10T02:35:34.539 INFO:teuthology.orchestra.run.smithi052.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-06-10T02:35:34.539 INFO:teuthology.orchestra.run.smithi052.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-06-10T02:35:34.539 INFO:teuthology.orchestra.run.smithi052.stdout: python3-packaging-20.9-5.el9.noarch 2024-06-10T02:35:34.539 INFO:teuthology.orchestra.run.smithi052.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-06-10T02:35:34.539 INFO:teuthology.orchestra.run.smithi052.stdout: python3-ply-3.11-14.el9.noarch 2024-06-10T02:35:34.539 INFO:teuthology.orchestra.run.smithi052.stdout: python3-portend-3.1.0-2.el9.noarch 2024-06-10T02:35:34.539 INFO:teuthology.orchestra.run.smithi052.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-06-10T02:35:34.539 INFO:teuthology.orchestra.run.smithi052.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-06-10T02:35:34.539 INFO:teuthology.orchestra.run.smithi052.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-06-10T02:35:34.539 INFO:teuthology.orchestra.run.smithi052.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-06-10T02:35:34.540 INFO:teuthology.orchestra.run.smithi052.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-06-10T02:35:34.540 INFO:teuthology.orchestra.run.smithi052.stdout: python3-pycparser-2.20-6.el9.noarch 2024-06-10T02:35:34.540 INFO:teuthology.orchestra.run.smithi052.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-06-10T02:35:34.540 INFO:teuthology.orchestra.run.smithi052.stdout: python3-pytz-2021.1-5.el9.noarch 2024-06-10T02:35:34.540 INFO:teuthology.orchestra.run.smithi052.stdout: python3-rados-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T02:35:34.540 INFO:teuthology.orchestra.run.smithi052.stdout: python3-rbd-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T02:35:34.540 INFO:teuthology.orchestra.run.smithi052.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-06-10T02:35:34.540 INFO:teuthology.orchestra.run.smithi052.stdout: python3-requests-2.25.1-8.el9.noarch 2024-06-10T02:35:34.540 INFO:teuthology.orchestra.run.smithi052.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-06-10T02:35:34.540 INFO:teuthology.orchestra.run.smithi052.stdout: python3-rgw-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T02:35:34.540 INFO:teuthology.orchestra.run.smithi052.stdout: python3-routes-2.5.1-5.el9.noarch 2024-06-10T02:35:34.540 INFO:teuthology.orchestra.run.smithi052.stdout: python3-rsa-4.9-2.el9.noarch 2024-06-10T02:35:34.540 INFO:teuthology.orchestra.run.smithi052.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-06-10T02:35:34.541 INFO:teuthology.orchestra.run.smithi052.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-06-10T02:35:34.541 INFO:teuthology.orchestra.run.smithi052.stdout: python3-toml-0.10.2-6.el9.noarch 2024-06-10T02:35:34.541 INFO:teuthology.orchestra.run.smithi052.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-06-10T02:35:34.541 INFO:teuthology.orchestra.run.smithi052.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-06-10T02:35:34.541 INFO:teuthology.orchestra.run.smithi052.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-06-10T02:35:34.541 INFO:teuthology.orchestra.run.smithi052.stdout: python3-webob-1.8.7-6.el9.noarch 2024-06-10T02:35:34.541 INFO:teuthology.orchestra.run.smithi052.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-06-10T02:35:34.541 INFO:teuthology.orchestra.run.smithi052.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-06-10T02:35:34.541 INFO:teuthology.orchestra.run.smithi052.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-06-10T02:35:34.541 INFO:teuthology.orchestra.run.smithi052.stdout: qatlib-24.02.0-1.el9.x86_64 2024-06-10T02:35:34.541 INFO:teuthology.orchestra.run.smithi052.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-06-10T02:35:34.541 INFO:teuthology.orchestra.run.smithi052.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-06-10T02:35:34.541 INFO:teuthology.orchestra.run.smithi052.stdout: rbd-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T02:35:34.541 INFO:teuthology.orchestra.run.smithi052.stdout: rbd-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T02:35:34.542 INFO:teuthology.orchestra.run.smithi052.stdout: rbd-nbd-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T02:35:34.542 INFO:teuthology.orchestra.run.smithi052.stdout: re2-1:20211101-3.el9.x86_64 2024-06-10T02:35:34.542 INFO:teuthology.orchestra.run.smithi052.stdout: socat-1.7.4.1-5.el9.x86_64 2024-06-10T02:35:34.542 INFO:teuthology.orchestra.run.smithi052.stdout: thrift-0.15.0-2.el9.x86_64 2024-06-10T02:35:34.542 INFO:teuthology.orchestra.run.smithi052.stdout: uuid-1.6.2-55.el9.x86_64 2024-06-10T02:35:34.542 INFO:teuthology.orchestra.run.smithi052.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-06-10T02:35:34.542 INFO:teuthology.orchestra.run.smithi052.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-06-10T02:35:34.542 INFO:teuthology.orchestra.run.smithi052.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-06-10T02:35:34.542 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-10T02:35:34.543 INFO:teuthology.orchestra.run.smithi052.stdout:Complete! 2024-06-10T02:35:34.954 DEBUG:teuthology.parallel:result is None 2024-06-10T02:35:36.540 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-06-10T02:35:36.540 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-2:19.0.0-2946.g3d89accf.el9.x86_64 1/170 2024-06-10T02:35:36.541 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-base-2:19.0.0-2946.g3d89accf.el9.x86_64 2/170 2024-06-10T02:35:36.541 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 3/170 2024-06-10T02:35:36.541 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 4/170 2024-06-10T02:35:36.541 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2946.g3d89a 5/170 2024-06-10T02:35:36.541 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-mds-2:19.0.0-2946.g3d89accf.el9.x86_64 6/170 2024-06-10T02:35:36.541 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-mgr-2:19.0.0-2946.g3d89accf.el9.x86_64 7/170 2024-06-10T02:35:36.541 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-mon-2:19.0.0-2946.g3d89accf.el9.x86_64 8/170 2024-06-10T02:35:36.541 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-osd-2:19.0.0-2946.g3d89accf.el9.x86_64 9/170 2024-06-10T02:35:36.541 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-radosgw-2:19.0.0-2946.g3d89accf.el9.x86_64 10/170 2024-06-10T02:35:36.541 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-selinux-2:19.0.0-2946.g3d89accf.el9.x86_64 11/170 2024-06-10T02:35:36.541 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-test-2:19.0.0-2946.g3d89accf.el9.x86_64 12/170 2024-06-10T02:35:36.542 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : cephfs-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 13/170 2024-06-10T02:35:36.542 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : cephfs-shell-2:19.0.0-2946.g3d89accf.el9.x86_64 14/170 2024-06-10T02:35:36.542 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : libcephfs-devel-2:19.0.0-2946.g3d89accf.el9.x86_ 15/170 2024-06-10T02:35:36.542 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : libcephfs2-2:19.0.0-2946.g3d89accf.el9.x86_64 16/170 2024-06-10T02:35:36.542 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : libcephsqlite-2:19.0.0-2946.g3d89accf.el9.x86_64 17/170 2024-06-10T02:35:36.542 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : librados-devel-2:19.0.0-2946.g3d89accf.el9.x86_6 18/170 2024-06-10T02:35:36.542 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : libradosstriper1-2:19.0.0-2946.g3d89accf.el9.x86 19/170 2024-06-10T02:35:36.542 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : librgw2-2:19.0.0-2946.g3d89accf.el9.x86_64 20/170 2024-06-10T02:35:36.542 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2946.g3d89accf.el 21/170 2024-06-10T02:35:36.542 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-ceph-common-2:19.0.0-2946.g3d89accf.el9. 22/170 2024-06-10T02:35:36.542 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-cephfs-2:19.0.0-2946.g3d89accf.el9.x86_6 23/170 2024-06-10T02:35:36.542 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-rados-2:19.0.0-2946.g3d89accf.el9.x86_64 24/170 2024-06-10T02:35:36.542 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-rbd-2:19.0.0-2946.g3d89accf.el9.x86_64 25/170 2024-06-10T02:35:36.543 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-rgw-2:19.0.0-2946.g3d89accf.el9.x86_64 26/170 2024-06-10T02:35:36.543 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : rbd-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 27/170 2024-06-10T02:35:36.560 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : rbd-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 28/170 2024-06-10T02:35:36.560 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : rbd-nbd-2:19.0.0-2946.g3d89accf.el9.x86_64 29/170 2024-06-10T02:35:36.560 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2946.g3d89accf. 30/170 2024-06-10T02:35:36.560 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2946.g3d89accf.el9.noa 31/170 2024-06-10T02:35:36.560 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2946.g3d89accf.el9.n 32/170 2024-06-10T02:35:36.560 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2946.g3d8 33/170 2024-06-10T02:35:36.560 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2946.g3d89accf.el 34/170 2024-06-10T02:35:36.560 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2946.g3d89accf.el9.noarch 35/170 2024-06-10T02:35:36.561 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2946.g3d89accf.e 36/170 2024-06-10T02:35:36.561 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-volume-2:19.0.0-2946.g3d89accf.el9.noarch 37/170 2024-06-10T02:35:36.561 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 38/170 2024-06-10T02:35:36.561 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : cephfs-top-2:19.0.0-2946.g3d89accf.el9.noarch 39/170 2024-06-10T02:35:36.561 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/170 2024-06-10T02:35:36.561 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/170 2024-06-10T02:35:36.561 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/170 2024-06-10T02:35:36.561 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/170 2024-06-10T02:35:36.561 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/170 2024-06-10T02:35:36.561 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/170 2024-06-10T02:35:36.561 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/170 2024-06-10T02:35:36.561 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/170 2024-06-10T02:35:36.561 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/170 2024-06-10T02:35:36.562 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/170 2024-06-10T02:35:36.562 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/170 2024-06-10T02:35:36.562 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/170 2024-06-10T02:35:36.562 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/170 2024-06-10T02:35:36.562 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/170 2024-06-10T02:35:36.562 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/170 2024-06-10T02:35:36.562 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/170 2024-06-10T02:35:36.562 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/170 2024-06-10T02:35:36.562 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/170 2024-06-10T02:35:36.562 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/170 2024-06-10T02:35:36.562 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/170 2024-06-10T02:35:36.562 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/170 2024-06-10T02:35:36.563 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/170 2024-06-10T02:35:36.563 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/170 2024-06-10T02:35:36.563 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/170 2024-06-10T02:35:36.563 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/170 2024-06-10T02:35:36.563 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/170 2024-06-10T02:35:36.563 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/170 2024-06-10T02:35:36.563 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/170 2024-06-10T02:35:36.563 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/170 2024-06-10T02:35:36.563 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/170 2024-06-10T02:35:36.563 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/170 2024-06-10T02:35:36.563 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/170 2024-06-10T02:35:36.563 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/170 2024-06-10T02:35:36.563 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/170 2024-06-10T02:35:36.564 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/170 2024-06-10T02:35:36.564 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/170 2024-06-10T02:35:36.564 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/170 2024-06-10T02:35:36.564 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 77/170 2024-06-10T02:35:36.564 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/170 2024-06-10T02:35:36.564 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/170 2024-06-10T02:35:36.564 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/170 2024-06-10T02:35:36.564 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/170 2024-06-10T02:35:36.564 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/170 2024-06-10T02:35:36.564 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/170 2024-06-10T02:35:36.564 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/170 2024-06-10T02:35:36.564 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/170 2024-06-10T02:35:36.564 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/170 2024-06-10T02:35:36.565 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/170 2024-06-10T02:35:36.565 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/170 2024-06-10T02:35:36.565 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/170 2024-06-10T02:35:36.565 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/170 2024-06-10T02:35:36.565 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/170 2024-06-10T02:35:36.565 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/170 2024-06-10T02:35:36.565 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/170 2024-06-10T02:35:36.565 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/170 2024-06-10T02:35:36.565 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/170 2024-06-10T02:35:36.565 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/170 2024-06-10T02:35:36.565 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/170 2024-06-10T02:35:36.565 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/170 2024-06-10T02:35:36.565 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 99/170 2024-06-10T02:35:36.565 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/170 2024-06-10T02:35:36.566 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 101/170 2024-06-10T02:35:36.566 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/170 2024-06-10T02:35:36.566 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/170 2024-06-10T02:35:36.566 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/170 2024-06-10T02:35:36.566 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/170 2024-06-10T02:35:36.566 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/170 2024-06-10T02:35:36.566 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/170 2024-06-10T02:35:36.566 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/170 2024-06-10T02:35:36.566 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/170 2024-06-10T02:35:36.566 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/170 2024-06-10T02:35:36.566 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/170 2024-06-10T02:35:36.567 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/170 2024-06-10T02:35:36.567 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/170 2024-06-10T02:35:36.567 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/170 2024-06-10T02:35:36.567 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/170 2024-06-10T02:35:36.567 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/170 2024-06-10T02:35:36.567 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/170 2024-06-10T02:35:36.567 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 118/170 2024-06-10T02:35:36.567 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/170 2024-06-10T02:35:36.567 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/170 2024-06-10T02:35:36.567 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/170 2024-06-10T02:35:36.567 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-google-auth-1:2.29.0-2.el9.noarch 122/170 2024-06-10T02:35:36.567 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/170 2024-06-10T02:35:36.567 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/170 2024-06-10T02:35:36.567 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/170 2024-06-10T02:35:36.568 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/170 2024-06-10T02:35:36.568 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/170 2024-06-10T02:35:36.568 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/170 2024-06-10T02:35:36.568 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/170 2024-06-10T02:35:36.568 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/170 2024-06-10T02:35:36.568 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/170 2024-06-10T02:35:36.568 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/170 2024-06-10T02:35:36.568 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/170 2024-06-10T02:35:36.568 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/170 2024-06-10T02:35:36.568 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/170 2024-06-10T02:35:36.568 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/170 2024-06-10T02:35:36.568 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/170 2024-06-10T02:35:36.568 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/170 2024-06-10T02:35:36.569 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/170 2024-06-10T02:35:36.569 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/170 2024-06-10T02:35:36.569 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/170 2024-06-10T02:35:36.569 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/170 2024-06-10T02:35:36.569 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/170 2024-06-10T02:35:36.569 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/170 2024-06-10T02:35:36.569 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/170 2024-06-10T02:35:36.569 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/170 2024-06-10T02:35:36.569 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/170 2024-06-10T02:35:36.569 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/170 2024-06-10T02:35:36.569 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : librados2-2:19.0.0-2946.g3d89accf.el9.x86_64 149/170 2024-06-10T02:35:36.569 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/170 2024-06-10T02:35:36.570 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : librbd1-2:19.0.0-2946.g3d89accf.el9.x86_64 151/170 2024-06-10T02:35:36.570 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/170 2024-06-10T02:35:36.570 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : NetworkManager-1:1.47.91-1.el9.x86_64 153/170 2024-06-10T02:35:36.570 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/170 2024-06-10T02:35:36.570 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : NetworkManager-libnm-1:1.47.91-1.el9.x86_64 155/170 2024-06-10T02:35:36.570 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/170 2024-06-10T02:35:36.570 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : NetworkManager-team-1:1.47.91-1.el9.x86_64 157/170 2024-06-10T02:35:36.570 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/170 2024-06-10T02:35:36.570 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : NetworkManager-tui-1:1.47.91-1.el9.x86_64 159/170 2024-06-10T02:35:36.570 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-06-10T02:35:36.570 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : elfutils-debuginfod-client-0.191-4.el9.x86_64 161/170 2024-06-10T02:35:36.570 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 162/170 2024-06-10T02:35:36.570 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : elfutils-libelf-0.191-4.el9.x86_64 163/170 2024-06-10T02:35:36.571 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 164/170 2024-06-10T02:35:36.571 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : elfutils-libs-0.191-4.el9.x86_64 165/170 2024-06-10T02:35:36.571 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 166/170 2024-06-10T02:35:36.571 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 167/170 2024-06-10T02:35:36.571 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 168/170 2024-06-10T02:35:36.571 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 169/170 2024-06-10T02:35:39.395 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 170/170 2024-06-10T02:35:39.395 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-06-10T02:35:39.395 INFO:teuthology.orchestra.run.smithi120.stdout:Upgraded: 2024-06-10T02:35:39.395 INFO:teuthology.orchestra.run.smithi120.stdout: NetworkManager-1:1.47.91-1.el9.x86_64 2024-06-10T02:35:39.395 INFO:teuthology.orchestra.run.smithi120.stdout: NetworkManager-libnm-1:1.47.91-1.el9.x86_64 2024-06-10T02:35:39.395 INFO:teuthology.orchestra.run.smithi120.stdout: NetworkManager-team-1:1.47.91-1.el9.x86_64 2024-06-10T02:35:39.395 INFO:teuthology.orchestra.run.smithi120.stdout: NetworkManager-tui-1:1.47.91-1.el9.x86_64 2024-06-10T02:35:39.395 INFO:teuthology.orchestra.run.smithi120.stdout: elfutils-debuginfod-client-0.191-4.el9.x86_64 2024-06-10T02:35:39.395 INFO:teuthology.orchestra.run.smithi120.stdout: elfutils-libelf-0.191-4.el9.x86_64 2024-06-10T02:35:39.395 INFO:teuthology.orchestra.run.smithi120.stdout: elfutils-libs-0.191-4.el9.x86_64 2024-06-10T02:35:39.395 INFO:teuthology.orchestra.run.smithi120.stdout: librados2-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T02:35:39.396 INFO:teuthology.orchestra.run.smithi120.stdout: librbd1-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T02:35:39.396 INFO:teuthology.orchestra.run.smithi120.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-06-10T02:35:39.396 INFO:teuthology.orchestra.run.smithi120.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-06-10T02:35:39.396 INFO:teuthology.orchestra.run.smithi120.stdout:Installed: 2024-06-10T02:35:39.396 INFO:teuthology.orchestra.run.smithi120.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-06-10T02:35:39.396 INFO:teuthology.orchestra.run.smithi120.stdout: automake-1.16.2-8.el9.noarch 2024-06-10T02:35:39.396 INFO:teuthology.orchestra.run.smithi120.stdout: bison-3.7.4-5.el9.x86_64 2024-06-10T02:35:39.396 INFO:teuthology.orchestra.run.smithi120.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-06-10T02:35:39.396 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T02:35:39.396 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-base-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T02:35:39.396 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T02:35:39.396 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T02:35:39.396 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-grafana-dashboards-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-10T02:35:39.397 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-immutable-object-cache-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T02:35:39.397 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mds-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T02:35:39.397 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mgr-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T02:35:39.397 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mgr-cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-10T02:35:39.397 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mgr-dashboard-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-10T02:35:39.397 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-10T02:35:39.397 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mgr-modules-core-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-10T02:35:39.397 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mgr-rook-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-10T02:35:39.397 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mon-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T02:35:39.397 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-osd-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T02:35:39.397 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-prometheus-alerts-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-10T02:35:39.397 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-radosgw-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T02:35:39.397 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-selinux-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T02:35:39.398 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-test-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T02:35:39.398 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-volume-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-10T02:35:39.398 INFO:teuthology.orchestra.run.smithi120.stdout: cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-10T02:35:39.398 INFO:teuthology.orchestra.run.smithi120.stdout: cephfs-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T02:35:39.398 INFO:teuthology.orchestra.run.smithi120.stdout: cephfs-shell-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T02:35:39.398 INFO:teuthology.orchestra.run.smithi120.stdout: cephfs-top-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-10T02:35:39.398 INFO:teuthology.orchestra.run.smithi120.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-06-10T02:35:39.398 INFO:teuthology.orchestra.run.smithi120.stdout: flex-2.6.4-9.el9.x86_64 2024-06-10T02:35:39.398 INFO:teuthology.orchestra.run.smithi120.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-06-10T02:35:39.398 INFO:teuthology.orchestra.run.smithi120.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-06-10T02:35:39.398 INFO:teuthology.orchestra.run.smithi120.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-06-10T02:35:39.398 INFO:teuthology.orchestra.run.smithi120.stdout: fuse-2.9.9-15.el9.x86_64 2024-06-10T02:35:39.399 INFO:teuthology.orchestra.run.smithi120.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-06-10T02:35:39.399 INFO:teuthology.orchestra.run.smithi120.stdout: grpc-data-1.46.7-10.el9.noarch 2024-06-10T02:35:39.399 INFO:teuthology.orchestra.run.smithi120.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-06-10T02:35:39.399 INFO:teuthology.orchestra.run.smithi120.stdout: libarrow-9.0.0-11.el9.x86_64 2024-06-10T02:35:39.399 INFO:teuthology.orchestra.run.smithi120.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-06-10T02:35:39.399 INFO:teuthology.orchestra.run.smithi120.stdout: libcephfs-devel-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T02:35:39.399 INFO:teuthology.orchestra.run.smithi120.stdout: libcephfs2-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T02:35:39.399 INFO:teuthology.orchestra.run.smithi120.stdout: libcephsqlite-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T02:35:39.399 INFO:teuthology.orchestra.run.smithi120.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-06-10T02:35:39.399 INFO:teuthology.orchestra.run.smithi120.stdout: liboath-2.6.7-2.el9.x86_64 2024-06-10T02:35:39.399 INFO:teuthology.orchestra.run.smithi120.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-06-10T02:35:39.399 INFO:teuthology.orchestra.run.smithi120.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-06-10T02:35:39.399 INFO:teuthology.orchestra.run.smithi120.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-06-10T02:35:39.400 INFO:teuthology.orchestra.run.smithi120.stdout: librados-devel-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T02:35:39.400 INFO:teuthology.orchestra.run.smithi120.stdout: libradosstriper1-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T02:35:39.400 INFO:teuthology.orchestra.run.smithi120.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-06-10T02:35:39.400 INFO:teuthology.orchestra.run.smithi120.stdout: librgw2-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T02:35:39.400 INFO:teuthology.orchestra.run.smithi120.stdout: libtool-2.4.6-45.el9.x86_64 2024-06-10T02:35:39.400 INFO:teuthology.orchestra.run.smithi120.stdout: libunwind-1.6.2-1.el9.x86_64 2024-06-10T02:35:39.400 INFO:teuthology.orchestra.run.smithi120.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-06-10T02:35:39.400 INFO:teuthology.orchestra.run.smithi120.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-06-10T02:35:39.400 INFO:teuthology.orchestra.run.smithi120.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-06-10T02:35:39.400 INFO:teuthology.orchestra.run.smithi120.stdout: lua-5.4.4-4.el9.x86_64 2024-06-10T02:35:39.400 INFO:teuthology.orchestra.run.smithi120.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-06-10T02:35:39.400 INFO:teuthology.orchestra.run.smithi120.stdout: luarocks-3.9.2-5.el9.noarch 2024-06-10T02:35:39.400 INFO:teuthology.orchestra.run.smithi120.stdout: openblas-0.3.21-2.el9.x86_64 2024-06-10T02:35:39.401 INFO:teuthology.orchestra.run.smithi120.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-06-10T02:35:39.401 INFO:teuthology.orchestra.run.smithi120.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-06-10T02:35:39.401 INFO:teuthology.orchestra.run.smithi120.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-06-10T02:35:39.401 INFO:teuthology.orchestra.run.smithi120.stdout: postgresql-13.14-1.el9.x86_64 2024-06-10T02:35:39.401 INFO:teuthology.orchestra.run.smithi120.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-06-10T02:35:39.401 INFO:teuthology.orchestra.run.smithi120.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-06-10T02:35:39.401 INFO:teuthology.orchestra.run.smithi120.stdout: postgresql-server-13.14-1.el9.x86_64 2024-06-10T02:35:39.401 INFO:teuthology.orchestra.run.smithi120.stdout: protobuf-3.14.0-13.el9.x86_64 2024-06-10T02:35:39.401 INFO:teuthology.orchestra.run.smithi120.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-06-10T02:35:39.401 INFO:teuthology.orchestra.run.smithi120.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-06-10T02:35:39.401 INFO:teuthology.orchestra.run.smithi120.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-06-10T02:35:39.401 INFO:teuthology.orchestra.run.smithi120.stdout: python3-babel-2.9.1-2.el9.noarch 2024-06-10T02:35:39.401 INFO:teuthology.orchestra.run.smithi120.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-06-10T02:35:39.402 INFO:teuthology.orchestra.run.smithi120.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-06-10T02:35:39.402 INFO:teuthology.orchestra.run.smithi120.stdout: python3-ceph-argparse-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T02:35:39.402 INFO:teuthology.orchestra.run.smithi120.stdout: python3-ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T02:35:39.402 INFO:teuthology.orchestra.run.smithi120.stdout: python3-cephfs-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T02:35:39.402 INFO:teuthology.orchestra.run.smithi120.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-06-10T02:35:39.402 INFO:teuthology.orchestra.run.smithi120.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-06-10T02:35:39.402 INFO:teuthology.orchestra.run.smithi120.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-06-10T02:35:39.402 INFO:teuthology.orchestra.run.smithi120.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-06-10T02:35:39.402 INFO:teuthology.orchestra.run.smithi120.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-06-10T02:35:39.402 INFO:teuthology.orchestra.run.smithi120.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-06-10T02:35:39.402 INFO:teuthology.orchestra.run.smithi120.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-06-10T02:35:39.402 INFO:teuthology.orchestra.run.smithi120.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-06-10T02:35:39.402 INFO:teuthology.orchestra.run.smithi120.stdout: python3-google-auth-1:2.29.0-2.el9.noarch 2024-06-10T02:35:39.403 INFO:teuthology.orchestra.run.smithi120.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-06-10T02:35:39.403 INFO:teuthology.orchestra.run.smithi120.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-06-10T02:35:39.403 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-06-10T02:35:39.403 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-06-10T02:35:39.403 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-06-10T02:35:39.403 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-06-10T02:35:39.403 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-06-10T02:35:39.403 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-06-10T02:35:39.403 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-06-10T02:35:39.403 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-06-10T02:35:39.403 INFO:teuthology.orchestra.run.smithi120.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-06-10T02:35:39.403 INFO:teuthology.orchestra.run.smithi120.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-06-10T02:35:39.403 INFO:teuthology.orchestra.run.smithi120.stdout: python3-mako-1.1.4-6.el9.noarch 2024-06-10T02:35:39.404 INFO:teuthology.orchestra.run.smithi120.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-06-10T02:35:39.404 INFO:teuthology.orchestra.run.smithi120.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-06-10T02:35:39.404 INFO:teuthology.orchestra.run.smithi120.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-06-10T02:35:39.404 INFO:teuthology.orchestra.run.smithi120.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-06-10T02:35:39.404 INFO:teuthology.orchestra.run.smithi120.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-06-10T02:35:39.404 INFO:teuthology.orchestra.run.smithi120.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-06-10T02:35:39.404 INFO:teuthology.orchestra.run.smithi120.stdout: python3-packaging-20.9-5.el9.noarch 2024-06-10T02:35:39.404 INFO:teuthology.orchestra.run.smithi120.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-06-10T02:35:39.404 INFO:teuthology.orchestra.run.smithi120.stdout: python3-ply-3.11-14.el9.noarch 2024-06-10T02:35:39.404 INFO:teuthology.orchestra.run.smithi120.stdout: python3-portend-3.1.0-2.el9.noarch 2024-06-10T02:35:39.404 INFO:teuthology.orchestra.run.smithi120.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-06-10T02:35:39.404 INFO:teuthology.orchestra.run.smithi120.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-06-10T02:35:39.404 INFO:teuthology.orchestra.run.smithi120.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-06-10T02:35:39.405 INFO:teuthology.orchestra.run.smithi120.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-06-10T02:35:39.405 INFO:teuthology.orchestra.run.smithi120.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-06-10T02:35:39.405 INFO:teuthology.orchestra.run.smithi120.stdout: python3-pycparser-2.20-6.el9.noarch 2024-06-10T02:35:39.405 INFO:teuthology.orchestra.run.smithi120.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-06-10T02:35:39.405 INFO:teuthology.orchestra.run.smithi120.stdout: python3-pytz-2021.1-5.el9.noarch 2024-06-10T02:35:39.405 INFO:teuthology.orchestra.run.smithi120.stdout: python3-rados-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T02:35:39.405 INFO:teuthology.orchestra.run.smithi120.stdout: python3-rbd-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T02:35:39.405 INFO:teuthology.orchestra.run.smithi120.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-06-10T02:35:39.405 INFO:teuthology.orchestra.run.smithi120.stdout: python3-requests-2.25.1-8.el9.noarch 2024-06-10T02:35:39.405 INFO:teuthology.orchestra.run.smithi120.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-06-10T02:35:39.405 INFO:teuthology.orchestra.run.smithi120.stdout: python3-rgw-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T02:35:39.405 INFO:teuthology.orchestra.run.smithi120.stdout: python3-routes-2.5.1-5.el9.noarch 2024-06-10T02:35:39.405 INFO:teuthology.orchestra.run.smithi120.stdout: python3-rsa-4.9-2.el9.noarch 2024-06-10T02:35:39.406 INFO:teuthology.orchestra.run.smithi120.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-06-10T02:35:39.406 INFO:teuthology.orchestra.run.smithi120.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-06-10T02:35:39.406 INFO:teuthology.orchestra.run.smithi120.stdout: python3-toml-0.10.2-6.el9.noarch 2024-06-10T02:35:39.406 INFO:teuthology.orchestra.run.smithi120.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-06-10T02:35:39.406 INFO:teuthology.orchestra.run.smithi120.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-06-10T02:35:39.406 INFO:teuthology.orchestra.run.smithi120.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-06-10T02:35:39.406 INFO:teuthology.orchestra.run.smithi120.stdout: python3-webob-1.8.7-6.el9.noarch 2024-06-10T02:35:39.406 INFO:teuthology.orchestra.run.smithi120.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-06-10T02:35:39.406 INFO:teuthology.orchestra.run.smithi120.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-06-10T02:35:39.406 INFO:teuthology.orchestra.run.smithi120.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-06-10T02:35:39.406 INFO:teuthology.orchestra.run.smithi120.stdout: qatlib-24.02.0-1.el9.x86_64 2024-06-10T02:35:39.406 INFO:teuthology.orchestra.run.smithi120.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-06-10T02:35:39.406 INFO:teuthology.orchestra.run.smithi120.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-06-10T02:35:39.407 INFO:teuthology.orchestra.run.smithi120.stdout: rbd-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T02:35:39.407 INFO:teuthology.orchestra.run.smithi120.stdout: rbd-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T02:35:39.407 INFO:teuthology.orchestra.run.smithi120.stdout: rbd-nbd-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T02:35:39.407 INFO:teuthology.orchestra.run.smithi120.stdout: re2-1:20211101-3.el9.x86_64 2024-06-10T02:35:39.407 INFO:teuthology.orchestra.run.smithi120.stdout: socat-1.7.4.1-5.el9.x86_64 2024-06-10T02:35:39.407 INFO:teuthology.orchestra.run.smithi120.stdout: thrift-0.15.0-2.el9.x86_64 2024-06-10T02:35:39.407 INFO:teuthology.orchestra.run.smithi120.stdout: uuid-1.6.2-55.el9.x86_64 2024-06-10T02:35:39.407 INFO:teuthology.orchestra.run.smithi120.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-06-10T02:35:39.407 INFO:teuthology.orchestra.run.smithi120.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-06-10T02:35:39.408 INFO:teuthology.orchestra.run.smithi120.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-06-10T02:35:39.408 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-06-10T02:35:39.408 INFO:teuthology.orchestra.run.smithi120.stdout:Complete! 2024-06-10T02:35:39.860 DEBUG:teuthology.parallel:result is None 2024-06-10T02:39:28.187 INFO:teuthology.orchestra.run.smithi188.stdout:[MIRROR] fuse-2.9.9-15.el9.x86_64.rpm: Curl error (28): Timeout was reached for http://mirror.datto.com/CentOS-Stream/9-stream/BaseOS/x86_64/os/Packages/fuse-2.9.9-15.el9.x86_64.rpm [Operation too slow. Less than 1000 bytes/sec transferred the last 300 seconds] 2024-06-10T02:39:28.528 INFO:teuthology.orchestra.run.smithi188.stdout:[MIRROR] libgfortran-11.4.1-3.el9.x86_64.rpm: Curl error (28): Timeout was reached for http://mirror.datto.com/CentOS-Stream/9-stream/BaseOS/x86_64/os/Packages/libgfortran-11.4.1-3.el9.x86_64.rpm [Operation too slow. Less than 1000 bytes/sec transferred the last 300 seconds] 2024-06-10T02:39:28.589 INFO:teuthology.orchestra.run.smithi188.stdout:(40/159): fuse-2.9.9-15.el9.x86_64.rpm 274 B/s | 80 kB 05:00 2024-06-10T02:39:28.873 INFO:teuthology.orchestra.run.smithi188.stdout:(41/159): python3-cffi-1.14.5-5.el9.x86_64.rpm 894 kB/s | 253 kB 00:00 2024-06-10T02:39:29.473 INFO:teuthology.orchestra.run.smithi188.stdout:(42/159): libgfortran-11.4.1-3.el9.x86_64.rpm 2.7 kB/s | 803 kB 05:00 2024-06-10T02:39:29.581 INFO:teuthology.orchestra.run.smithi188.stdout:(43/159): python3-ply-3.11-14.el9.noarch.rpm 985 kB/s | 106 kB 00:00 2024-06-10T02:39:29.774 INFO:teuthology.orchestra.run.smithi188.stdout:(44/159): python3-pycparser-2.20-6.el9.noarch.r 705 kB/s | 135 kB 00:00 2024-06-10T02:39:29.849 INFO:teuthology.orchestra.run.smithi188.stdout:(45/159): python3-cryptography-36.0.1-4.el9.x86 1.3 MB/s | 1.2 MB 00:00 2024-06-10T02:39:29.882 INFO:teuthology.orchestra.run.smithi188.stdout:(46/159): python3-requests-2.25.1-8.el9.noarch. 1.1 MB/s | 125 kB 00:00 2024-06-10T02:39:29.922 INFO:teuthology.orchestra.run.smithi188.stdout:[MIRROR] automake-1.16.2-8.el9.noarch.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/automake-1.16.2-8.el9.noarch.rpm (IP: 8.43.85.72) 2024-06-10T02:39:30.049 INFO:teuthology.orchestra.run.smithi188.stdout:(47/159): python3-urllib3-1.26.5-5.el9.noarch.r 1.0 MB/s | 215 kB 00:00 2024-06-10T02:39:30.050 INFO:teuthology.orchestra.run.smithi188.stdout:[MIRROR] bison-3.7.4-5.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/bison-3.7.4-5.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-10T02:39:30.683 INFO:teuthology.orchestra.run.smithi188.stdout:(48/159): automake-1.16.2-8.el9.noarch.rpm 861 kB/s | 689 kB 00:00 2024-06-10T02:39:30.684 INFO:teuthology.orchestra.run.smithi188.stdout:[MIRROR] boost-program-options-1.75.0-8.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/boost-program-options-1.75.0-8.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-10T02:39:30.772 INFO:teuthology.orchestra.run.smithi188.stdout:[MIRROR] libquadmath-11.4.1-3.el9.x86_64.rpm: Curl error (28): Timeout was reached for http://mirror.datto.com/CentOS-Stream/9-stream/BaseOS/x86_64/os/Packages/libquadmath-11.4.1-3.el9.x86_64.rpm [Operation too slow. Less than 1000 bytes/sec transferred the last 300 seconds] 2024-06-10T02:39:30.800 INFO:teuthology.orchestra.run.smithi188.stdout:(49/159): boost-program-options-1.75.0-8.el9.x8 916 kB/s | 107 kB 00:00 2024-06-10T02:39:30.875 INFO:teuthology.orchestra.run.smithi188.stdout:(50/159): bison-3.7.4-5.el9.x86_64.rpm 1.1 MB/s | 943 kB 00:00 2024-06-10T02:39:30.901 INFO:teuthology.orchestra.run.smithi188.stdout:(51/159): elfutils-libelf-devel-0.191-4.el9.x86 216 kB/s | 22 kB 00:00 2024-06-10T02:39:30.984 INFO:teuthology.orchestra.run.smithi188.stdout:(52/159): flex-2.6.4-9.el9.x86_64.rpm 2.8 MB/s | 313 kB 00:00 2024-06-10T02:39:31.010 INFO:teuthology.orchestra.run.smithi188.stdout:(53/159): flexiblas-3.0.4-8.el9.x86_64.rpm 301 kB/s | 33 kB 00:00 2024-06-10T02:39:31.052 INFO:teuthology.orchestra.run.smithi188.stdout:(54/159): libquadmath-11.4.1-3.el9.x86_64.rpm 666 B/s | 195 kB 05:00 2024-06-10T02:39:31.118 INFO:teuthology.orchestra.run.smithi188.stdout:(55/159): flexiblas-openblas-openmp-3.0.4-8.el9 168 kB/s | 18 kB 00:00 2024-06-10T02:39:31.227 INFO:teuthology.orchestra.run.smithi188.stdout:(56/159): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.4 MB/s | 160 kB 00:00 2024-06-10T02:39:31.244 INFO:teuthology.orchestra.run.smithi188.stdout:(57/159): libaio-devel-0.3.111-13.el9.x86_64.rp 58 kB/s | 11 kB 00:00 2024-06-10T02:39:31.328 INFO:teuthology.orchestra.run.smithi188.stdout:(58/159): librabbitmq-0.11.0-7.el9.x86_64.rpm 451 kB/s | 45 kB 00:00 2024-06-10T02:39:31.470 INFO:teuthology.orchestra.run.smithi188.stdout:(59/159): flexiblas-netlib-3.0.4-8.el9.x86_64.r 6.2 MB/s | 3.0 MB 00:00 2024-06-10T02:39:31.578 INFO:teuthology.orchestra.run.smithi188.stdout:(60/159): libuuid-devel-2.37.4-18.el9.x86_64.rp 235 kB/s | 25 kB 00:00 2024-06-10T02:39:31.604 INFO:teuthology.orchestra.run.smithi188.stdout:(61/159): libtool-2.4.6-45.el9.x86_64.rpm 2.1 MB/s | 582 kB 00:00 2024-06-10T02:39:31.679 INFO:teuthology.orchestra.run.smithi188.stdout:(62/159): libzstd-devel-1.5.1-2.el9.x86_64.rpm 473 kB/s | 47 kB 00:00 2024-06-10T02:39:31.721 INFO:teuthology.orchestra.run.smithi188.stdout:(63/159): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.4 MB/s | 292 kB 00:00 2024-06-10T02:39:31.788 INFO:teuthology.orchestra.run.smithi188.stdout:(64/159): lua-5.4.4-4.el9.x86_64.rpm 1.7 MB/s | 188 kB 00:00 2024-06-10T02:39:31.830 INFO:teuthology.orchestra.run.smithi188.stdout:(65/159): openblas-0.3.21-2.el9.x86_64.rpm 324 kB/s | 35 kB 00:00 2024-06-10T02:39:31.897 INFO:teuthology.orchestra.run.smithi188.stdout:(66/159): librdkafka-1.6.1-102.el9.x86_64.rpm 1.0 MB/s | 662 kB 00:00 2024-06-10T02:39:32.264 INFO:teuthology.orchestra.run.smithi188.stdout:(67/159): postgresql-13.14-1.el9.x86_64.rpm 4.3 MB/s | 1.6 MB 00:00 2024-06-10T02:39:32.389 INFO:teuthology.orchestra.run.smithi188.stdout:(68/159): postgresql-contrib-13.14-1.el9.x86_64 7.1 MB/s | 911 kB 00:00 2024-06-10T02:39:32.497 INFO:teuthology.orchestra.run.smithi188.stdout:(69/159): postgresql-private-libs-13.14-1.el9.x 1.2 MB/s | 133 kB 00:00 2024-06-10T02:39:32.564 INFO:teuthology.orchestra.run.smithi188.stdout:(70/159): openssl-devel-3.2.1-1.el9.x86_64.rpm 6.0 MB/s | 4.4 MB 00:00 2024-06-10T02:39:33.966 INFO:teuthology.orchestra.run.smithi188.stdout:(71/159): protobuf-3.14.0-13.el9.x86_64.rpm 738 kB/s | 1.0 MB 00:01 2024-06-10T02:39:34.148 INFO:teuthology.orchestra.run.smithi188.stdout:(72/159): python3-attrs-20.3.0-7.el9.noarch.rpm 477 kB/s | 87 kB 00:00 2024-06-10T02:39:35.948 INFO:teuthology.orchestra.run.smithi188.stdout:(73/159): openblas-openmp-0.3.21-2.el9.x86_64.r 1.1 MB/s | 4.7 MB 00:04 2024-06-10T02:39:36.065 INFO:teuthology.orchestra.run.smithi188.stdout:(74/159): python3-colorama-0.4.6-3.el9.noarch.r 531 kB/s | 61 kB 00:00 2024-06-10T02:39:36.257 INFO:teuthology.orchestra.run.smithi188.stdout:(75/159): postgresql-server-13.14-1.el9.x86_64. 1.5 MB/s | 5.8 MB 00:03 2024-06-10T02:39:36.441 INFO:teuthology.orchestra.run.smithi188.stdout:(76/159): python3-devel-3.9.18-3.el9.x86_64.rpm 650 kB/s | 244 kB 00:00 2024-06-10T02:39:36.566 INFO:teuthology.orchestra.run.smithi188.stdout:(77/159): python3-jsonpatch-1.21-16.el9.noarch. 205 kB/s | 26 kB 00:00 2024-06-10T02:39:36.599 INFO:teuthology.orchestra.run.smithi188.stdout:(78/159): python3-jinja2-2.11.3-6.el9.noarch.rp 726 kB/s | 248 kB 00:00 2024-06-10T02:39:36.675 INFO:teuthology.orchestra.run.smithi188.stdout:(79/159): python3-jsonpointer-2.0-4.el9.noarch. 176 kB/s | 19 kB 00:00 2024-06-10T02:39:36.784 INFO:teuthology.orchestra.run.smithi188.stdout:(80/159): python3-markupsafe-1.1.1-12.el9.x86_6 320 kB/s | 35 kB 00:00 2024-06-10T02:39:36.892 INFO:teuthology.orchestra.run.smithi188.stdout:(81/159): python3-mako-1.1.4-6.el9.noarch.rpm 589 kB/s | 172 kB 00:00 2024-06-10T02:39:37.259 INFO:teuthology.orchestra.run.smithi188.stdout:(82/159): python3-numpy-f2py-1.20.1-5.el9.x86_6 684 kB/s | 250 kB 00:00 2024-06-10T02:39:37.601 INFO:teuthology.orchestra.run.smithi188.stdout:(83/159): python3-oauthlib-3.1.1-5.el9.noarch.r 651 kB/s | 222 kB 00:00 2024-06-10T02:39:37.709 INFO:teuthology.orchestra.run.smithi188.stdout:(84/159): python3-packaging-20.9-5.el9.noarch.r 712 kB/s | 77 kB 00:00 2024-06-10T02:39:37.810 INFO:teuthology.orchestra.run.smithi188.stdout:(85/159): python3-prettytable-0.7.2-27.el9.noar 422 kB/s | 42 kB 00:00 2024-06-10T02:39:37.927 INFO:teuthology.orchestra.run.smithi188.stdout:(86/159): python3-protobuf-3.14.0-13.el9.noarch 2.3 MB/s | 269 kB 00:00 2024-06-10T02:39:38.027 INFO:teuthology.orchestra.run.smithi188.stdout:(87/159): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.5 MB/s | 159 kB 00:00 2024-06-10T02:39:38.136 INFO:teuthology.orchestra.run.smithi188.stdout:(88/159): python3-pyasn1-modules-0.4.8-6.el9.no 2.5 MB/s | 279 kB 00:00 2024-06-10T02:39:38.244 INFO:teuthology.orchestra.run.smithi188.stdout:(89/159): python3-pytz-2021.1-5.el9.noarch.rpm 470 kB/s | 51 kB 00:00 2024-06-10T02:39:38.353 INFO:teuthology.orchestra.run.smithi188.stdout:(90/159): python3-requests-oauthlib-1.3.0-12.el 495 kB/s | 54 kB 00:00 2024-06-10T02:39:38.945 INFO:teuthology.orchestra.run.smithi188.stdout:(91/159): python3-scipy-1.6.2-8.el9.x86_64.rpm 27 MB/s | 16 MB 00:00 2024-06-10T02:39:39.070 INFO:teuthology.orchestra.run.smithi188.stdout:(92/159): python3-toml-0.10.2-6.el9.noarch.rpm 333 kB/s | 42 kB 00:00 2024-06-10T02:39:39.170 INFO:teuthology.orchestra.run.smithi188.stdout:(93/159): python3-wcwidth-0.2.5-8.el9.noarch.rp 443 kB/s | 44 kB 00:00 2024-06-10T02:39:39.279 INFO:teuthology.orchestra.run.smithi188.stdout:(94/159): qatlib-24.02.0-1.el9.x86_64.rpm 2.0 MB/s | 221 kB 00:00 2024-06-10T02:39:39.379 INFO:teuthology.orchestra.run.smithi188.stdout:(95/159): qatlib-service-24.02.0-1.el9.x86_64.r 358 kB/s | 36 kB 00:00 2024-06-10T02:39:39.480 INFO:teuthology.orchestra.run.smithi188.stdout:(96/159): qatzip-libs-1.2.0-1.el9.x86_64.rpm 473 kB/s | 47 kB 00:00 2024-06-10T02:39:39.588 INFO:teuthology.orchestra.run.smithi188.stdout:(97/159): socat-1.7.4.1-5.el9.x86_64.rpm 2.8 MB/s | 305 kB 00:00 2024-06-10T02:39:39.680 INFO:teuthology.orchestra.run.smithi188.stdout:(98/159): uuid-1.6.2-55.el9.x86_64.rpm 631 kB/s | 58 kB 00:00 2024-06-10T02:39:39.789 INFO:teuthology.orchestra.run.smithi188.stdout:(99/159): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 866 kB/s | 94 kB 00:00 2024-06-10T02:39:39.889 INFO:teuthology.orchestra.run.smithi188.stdout:(100/159): xmlstarlet-1.6.1-20.el9.x86_64.rpm 637 kB/s | 64 kB 00:00 2024-06-10T02:39:39.990 INFO:teuthology.orchestra.run.smithi188.stdout:(101/159): zlib-devel-1.2.11-41.el9.x86_64.rpm 447 kB/s | 45 kB 00:00 2024-06-10T02:39:40.027 INFO:teuthology.orchestra.run.smithi188.stdout:[MIRROR] lua-devel-5.4.4-4.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/CRB/x86_64/os/Packages/lua-devel-5.4.4-4.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-10T02:39:40.132 INFO:teuthology.orchestra.run.smithi188.stdout:(102/159): lua-devel-5.4.4-4.el9.x86_64.rpm 157 kB/s | 22 kB 00:00 2024-06-10T02:39:40.133 INFO:teuthology.orchestra.run.smithi188.stdout:[MIRROR] protobuf-compiler-3.14.0-13.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/CRB/x86_64/os/Packages/protobuf-compiler-3.14.0-13.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-10T02:39:40.257 INFO:teuthology.orchestra.run.smithi188.stdout:(103/159): protobuf-compiler-3.14.0-13.el9.x86_ 6.7 MB/s | 863 kB 00:00 2024-06-10T02:39:40.309 INFO:teuthology.orchestra.run.smithi188.stdout:(104/159): abseil-cpp-20211102.0-3.el9.x86_64.r 11 MB/s | 548 kB 00:00 2024-06-10T02:39:40.341 INFO:teuthology.orchestra.run.smithi188.stdout:(105/159): gperftools-libs-2.9.1-2.el9.x86_64.r 9.0 MB/s | 309 kB 00:00 2024-06-10T02:39:40.366 INFO:teuthology.orchestra.run.smithi188.stdout:(106/159): grpc-data-1.46.7-10.el9.noarch.rpm 780 kB/s | 19 kB 00:00 2024-06-10T02:39:40.483 INFO:teuthology.orchestra.run.smithi188.stdout:(107/159): libarrow-9.0.0-11.el9.x86_64.rpm 38 MB/s | 4.4 MB 00:00 2024-06-10T02:39:40.508 INFO:teuthology.orchestra.run.smithi188.stdout:(108/159): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-06-10T02:39:40.534 INFO:teuthology.orchestra.run.smithi188.stdout:(109/159): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-06-10T02:39:40.559 INFO:teuthology.orchestra.run.smithi188.stdout:(110/159): libunwind-1.6.2-1.el9.x86_64.rpm 2.6 MB/s | 67 kB 00:00 2024-06-10T02:39:40.584 INFO:teuthology.orchestra.run.smithi188.stdout:(111/159): luarocks-3.9.2-5.el9.noarch.rpm 6.0 MB/s | 151 kB 00:00 2024-06-10T02:39:40.618 INFO:teuthology.orchestra.run.smithi188.stdout:(112/159): parquet-libs-9.0.0-11.el9.x86_64.rpm 24 MB/s | 839 kB 00:00 2024-06-10T02:39:40.651 INFO:teuthology.orchestra.run.smithi188.stdout:(113/159): python3-asyncssh-2.13.2-5.el9.noarch 16 MB/s | 548 kB 00:00 2024-06-10T02:39:40.677 INFO:teuthology.orchestra.run.smithi188.stdout:(114/159): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-06-10T02:39:40.702 INFO:teuthology.orchestra.run.smithi188.stdout:(115/159): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-06-10T02:39:40.727 INFO:teuthology.orchestra.run.smithi188.stdout:(116/159): python3-certifi-2021.10.8-2.el9.noar 598 kB/s | 15 kB 00:00 2024-06-10T02:39:40.753 INFO:teuthology.orchestra.run.smithi188.stdout:(117/159): python3-cheroot-8.6.0-4.el9.noarch.r 6.7 MB/s | 172 kB 00:00 2024-06-10T02:39:40.786 INFO:teuthology.orchestra.run.smithi188.stdout:(118/159): python3-cherrypy-18.6.1-2.el9.noarch 11 MB/s | 358 kB 00:00 2024-06-10T02:39:40.811 INFO:teuthology.orchestra.run.smithi188.stdout:(119/159): python3-cmd2-2.4.2-2.el9.noarch.rpm 12 MB/s | 303 kB 00:00 2024-06-10T02:39:40.837 INFO:teuthology.orchestra.run.smithi188.stdout:(120/159): python3-google-auth-2.29.0-2.el9.noa 8.0 MB/s | 205 kB 00:00 2024-06-10T02:39:40.895 INFO:teuthology.orchestra.run.smithi188.stdout:(121/159): python3-grpcio-1.46.7-10.el9.x86_64. 35 MB/s | 2.0 MB 00:00 2024-06-10T02:39:40.921 INFO:teuthology.orchestra.run.smithi188.stdout:(122/159): python3-grpcio-tools-1.46.7-10.el9.x 5.6 MB/s | 144 kB 00:00 2024-06-10T02:39:40.946 INFO:teuthology.orchestra.run.smithi188.stdout:(123/159): python3-jaraco-8.2.1-3.el9.noarch.rp 429 kB/s | 11 kB 00:00 2024-06-10T02:39:40.972 INFO:teuthology.orchestra.run.smithi188.stdout:(124/159): python3-jaraco-classes-3.2.1-5.el9.n 707 kB/s | 18 kB 00:00 2024-06-10T02:39:40.997 INFO:teuthology.orchestra.run.smithi188.stdout:(125/159): python3-jaraco-collections-3.0.0-8.e 925 kB/s | 23 kB 00:00 2024-06-10T02:39:41.022 INFO:teuthology.orchestra.run.smithi188.stdout:(126/159): python3-jaraco-functools-3.5.0-2.el9 781 kB/s | 19 kB 00:00 2024-06-10T02:39:41.047 INFO:teuthology.orchestra.run.smithi188.stdout:(127/159): python3-jaraco-text-3.2.0-6.el9.noar 791 kB/s | 20 kB 00:00 2024-06-10T02:39:41.089 INFO:teuthology.orchestra.run.smithi188.stdout:(128/159): python3-kubernetes-26.1.0-2.el9.noar 24 MB/s | 1.0 MB 00:00 2024-06-10T02:39:41.115 INFO:teuthology.orchestra.run.smithi188.stdout:(129/159): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-06-10T02:39:41.140 INFO:teuthology.orchestra.run.smithi188.stdout:(130/159): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-06-10T02:39:41.165 INFO:teuthology.orchestra.run.smithi188.stdout:(131/159): python3-natsort-7.1.1-5.el9.noarch.r 2.2 MB/s | 58 kB 00:00 2024-06-10T02:39:41.191 INFO:teuthology.orchestra.run.smithi188.stdout:(132/159): python3-pecan-1.4.2-3.el9.noarch.rpm 11 MB/s | 272 kB 00:00 2024-06-10T02:39:41.216 INFO:teuthology.orchestra.run.smithi188.stdout:(133/159): python3-portend-3.1.0-2.el9.noarch.r 659 kB/s | 16 kB 00:00 2024-06-10T02:39:41.241 INFO:teuthology.orchestra.run.smithi188.stdout:(134/159): python3-pyOpenSSL-21.0.0-1.el9.noarc 3.5 MB/s | 90 kB 00:00 2024-06-10T02:39:41.266 INFO:teuthology.orchestra.run.smithi188.stdout:(135/159): python3-pyperclip-1.8.2-2.el9.noarch 1.1 MB/s | 28 kB 00:00 2024-06-10T02:39:41.292 INFO:teuthology.orchestra.run.smithi188.stdout:(136/159): python3-repoze-lru-0.7-16.el9.noarch 1.2 MB/s | 31 kB 00:00 2024-06-10T02:39:41.317 INFO:teuthology.orchestra.run.smithi188.stdout:(137/159): python3-routes-2.5.1-5.el9.noarch.rp 7.4 MB/s | 188 kB 00:00 2024-06-10T02:39:41.342 INFO:teuthology.orchestra.run.smithi188.stdout:(138/159): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-06-10T02:39:41.376 INFO:teuthology.orchestra.run.smithi188.stdout:(139/159): python3-tempora-5.0.0-2.el9.noarch.r 1.0 MB/s | 36 kB 00:00 2024-06-10T02:39:41.401 INFO:teuthology.orchestra.run.smithi188.stdout:(140/159): python3-typing-extensions-4.4.0-2.el 2.0 MB/s | 51 kB 00:00 2024-06-10T02:39:41.426 INFO:teuthology.orchestra.run.smithi188.stdout:(141/159): python3-webob-1.8.7-6.el9.noarch.rpm 9.0 MB/s | 230 kB 00:00 2024-06-10T02:39:41.452 INFO:teuthology.orchestra.run.smithi188.stdout:(142/159): python3-websocket-client-1.2.3-2.el9 3.5 MB/s | 90 kB 00:00 2024-06-10T02:39:41.486 INFO:teuthology.orchestra.run.smithi188.stdout:(143/159): python3-werkzeug-2.0.3-3.el9.1.noarc 13 MB/s | 427 kB 00:00 2024-06-10T02:39:41.502 INFO:teuthology.orchestra.run.smithi188.stdout:(144/159): python3-zc-lockfile-2.0-10.el9.noarc 1.2 MB/s | 20 kB 00:00 2024-06-10T02:39:41.536 INFO:teuthology.orchestra.run.smithi188.stdout:(145/159): re2-20211101-3.el9.x86_64.rpm 5.6 MB/s | 192 kB 00:00 2024-06-10T02:39:41.586 INFO:teuthology.orchestra.run.smithi188.stdout:(146/159): thrift-0.15.0-2.el9.x86_64.rpm 32 MB/s | 1.6 MB 00:00 2024-06-10T02:39:42.712 INFO:teuthology.orchestra.run.smithi188.stdout:(147/159): python3-babel-2.9.1-2.el9.noarch.rpm 713 kB/s | 6.0 MB 00:08 2024-06-10T02:39:42.912 INFO:teuthology.orchestra.run.smithi188.stdout:(148/159): librados2-19.0.0-2946.g3d89accf.el9. 2.6 MB/s | 3.4 MB 00:01 2024-06-10T02:39:44.312 INFO:teuthology.orchestra.run.smithi188.stdout:(149/159): librbd1-19.0.0-2946.g3d89accf.el9.x8 2.0 MB/s | 3.1 MB 00:01 2024-06-10T02:39:44.379 INFO:teuthology.orchestra.run.smithi188.stdout:(150/159): python3-numpy-1.20.1-5.el9.x86_64.rp 687 kB/s | 5.1 MB 00:07 2024-06-10T02:39:44.746 INFO:teuthology.orchestra.run.smithi188.stdout:(151/159): NetworkManager-team-1.47.91-1.el9.x8 113 kB/s | 41 kB 00:00 2024-06-10T02:39:45.071 INFO:teuthology.orchestra.run.smithi188.stdout:(152/159): NetworkManager-tui-1.47.91-1.el9.x86 763 kB/s | 248 kB 00:00 2024-06-10T02:39:45.171 INFO:teuthology.orchestra.run.smithi188.stdout:(153/159): NetworkManager-1.47.91-1.el9.x86_64. 1.0 MB/s | 2.3 MB 00:02 2024-06-10T02:39:45.254 INFO:teuthology.orchestra.run.smithi188.stdout:(154/159): elfutils-debuginfod-client-0.191-4.e 200 kB/s | 37 kB 00:00 2024-06-10T02:39:45.371 INFO:teuthology.orchestra.run.smithi188.stdout:(155/159): elfutils-libelf-0.191-4.el9.x86_64.r 1.0 MB/s | 207 kB 00:00 2024-06-10T02:39:45.547 INFO:teuthology.orchestra.run.smithi188.stdout:(156/159): elfutils-libs-0.191-4.el9.x86_64.rpm 889 kB/s | 260 kB 00:00 2024-06-10T02:39:46.014 INFO:teuthology.orchestra.run.smithi188.stdout:(157/159): NetworkManager-libnm-1.47.91-1.el9.x 1.1 MB/s | 1.8 MB 00:01 2024-06-10T02:39:46.572 INFO:teuthology.orchestra.run.smithi188.stdout:(158/159): openssl-3.2.1-1.el9.x86_64.rpm 1.1 MB/s | 1.3 MB 00:01 2024-06-10T02:39:47.506 INFO:teuthology.orchestra.run.smithi188.stdout:(159/159): openssl-libs-3.2.1-1.el9.x86_64.rpm 1.2 MB/s | 2.4 MB 00:01 2024-06-10T02:39:47.509 INFO:teuthology.orchestra.run.smithi188.stdout:-------------------------------------------------------------------------------- 2024-06-10T02:39:47.509 INFO:teuthology.orchestra.run.smithi188.stdout:Total 703 kB/s | 225 MB 05:27 2024-06-10T02:39:48.653 INFO:teuthology.orchestra.run.smithi188.stdout:Running transaction check 2024-06-10T02:39:48.806 INFO:teuthology.orchestra.run.smithi188.stdout:Transaction check succeeded. 2024-06-10T02:39:48.807 INFO:teuthology.orchestra.run.smithi188.stdout:Running transaction test 2024-06-10T02:39:50.593 INFO:teuthology.orchestra.run.smithi188.stdout:Transaction test succeeded. 2024-06-10T02:39:50.593 INFO:teuthology.orchestra.run.smithi188.stdout:Running transaction 2024-06-10T02:39:56.775 INFO:teuthology.orchestra.run.smithi188.stdout: Preparing : 1/1 2024-06-10T02:39:56.842 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/170 2024-06-10T02:39:56.880 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/170 2024-06-10T02:39:56.934 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/170 2024-06-10T02:39:56.976 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/170 2024-06-10T02:39:57.002 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/170 2024-06-10T02:39:57.044 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/170 2024-06-10T02:39:57.377 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/170 2024-06-10T02:39:57.495 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : librados2-2:19.0.0-2946.g3d89accf.el9.x86_64 8/170 2024-06-10T02:39:57.617 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: librados2-2:19.0.0-2946.g3d89accf.el9.x86_64 8/170 2024-06-10T02:39:57.646 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libcephfs2-2:19.0.0-2946.g3d89accf.el9.x86_64 9/170 2024-06-10T02:39:57.737 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: libcephfs2-2:19.0.0-2946.g3d89accf.el9.x86_64 9/170 2024-06-10T02:39:57.767 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-rados-2:19.0.0-2946.g3d89accf.el9.x86_64 10/170 2024-06-10T02:39:57.799 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/170 2024-06-10T02:39:57.981 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : elfutils-libelf-0.191-4.el9.x86_64 12/170 2024-06-10T02:39:58.031 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : postgresql-13.14-1.el9.x86_64 13/170 2024-06-10T02:39:58.060 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libcephsqlite-2:19.0.0-2946.g3d89accf.el9.x86_64 14/170 2024-06-10T02:39:58.163 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2946.g3d89accf.el9.x86_64 14/170 2024-06-10T02:39:58.186 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libradosstriper1-2:19.0.0-2946.g3d89accf.el9.x86 15/170 2024-06-10T02:39:58.248 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2946.g3d89accf.el9.x86 15/170 2024-06-10T02:39:58.272 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-06-10T02:39:58.290 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-06-10T02:39:58.481 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/170 2024-06-10T02:39:58.615 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/170 2024-06-10T02:39:58.769 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : NetworkManager-libnm-1:1.47.91-1.el9.x86_64 18/170 2024-06-10T02:39:58.838 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.91-1.el9.x86_64 18/170 2024-06-10T02:39:59.034 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: NetworkManager-1:1.47.91-1.el9.x86_64 19/170 2024-06-10T02:39:59.214 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : NetworkManager-1:1.47.91-1.el9.x86_64 19/170 2024-06-10T02:39:59.405 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: NetworkManager-1:1.47.91-1.el9.x86_64 19/170 2024-06-10T02:39:59.509 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : re2-1:20211101-3.el9.x86_64 20/170 2024-06-10T02:39:59.597 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/170 2024-06-10T02:39:59.644 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 22/170 2024-06-10T02:39:59.678 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : liboath-2.6.7-2.el9.x86_64 23/170 2024-06-10T02:39:59.742 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 24/170 2024-06-10T02:39:59.778 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 25/170 2024-06-10T02:39:59.821 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/170 2024-06-10T02:39:59.854 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/170 2024-06-10T02:39:59.882 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/170 2024-06-10T02:39:59.941 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/170 2024-06-10T02:40:00.003 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 30/170 2024-06-10T02:40:00.035 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/170 2024-06-10T02:40:00.070 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 32/170 2024-06-10T02:40:00.145 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 33/170 2024-06-10T02:40:00.181 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-ceph-common-2:19.0.0-2946.g3d89accf.el9. 34/170 2024-06-10T02:40:00.224 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-ceph-argparse-2:19.0.0-2946.g3d89accf.el 35/170 2024-06-10T02:40:00.253 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-cephfs-2:19.0.0-2946.g3d89accf.el9.x86_6 36/170 2024-06-10T02:40:00.330 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/170 2024-06-10T02:40:00.375 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/170 2024-06-10T02:40:00.449 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/170 2024-06-10T02:40:00.580 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/170 2024-06-10T02:40:00.636 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/170 2024-06-10T02:40:01.146 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/170 2024-06-10T02:40:01.231 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/170 2024-06-10T02:40:01.261 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/170 2024-06-10T02:40:01.289 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/170 2024-06-10T02:40:01.337 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : elfutils-libs-0.191-4.el9.x86_64 46/170 2024-06-10T02:40:01.375 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : librados-devel-2:19.0.0-2946.g3d89accf.el9.x86_6 47/170 2024-06-10T02:40:01.446 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 48/170 2024-06-10T02:40:01.598 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 49/170 2024-06-10T02:40:01.645 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 50/170 2024-06-10T02:40:01.700 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 51/170 2024-06-10T02:40:01.773 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/170 2024-06-10T02:40:01.804 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 53/170 2024-06-10T02:40:01.841 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 54/170 2024-06-10T02:40:01.887 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/170 2024-06-10T02:40:01.931 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/170 2024-06-10T02:40:01.974 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/170 2024-06-10T02:40:02.016 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/170 2024-06-10T02:40:02.151 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/170 2024-06-10T02:40:02.193 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/170 2024-06-10T02:40:02.234 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 61/170 2024-06-10T02:40:02.328 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/170 2024-06-10T02:40:02.359 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-google-auth-1:2.29.0-2.el9.noarch 63/170 2024-06-10T02:40:02.400 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/170 2024-06-10T02:40:02.425 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 65/170 2024-06-10T02:40:02.450 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/170 2024-06-10T02:40:02.523 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/170 2024-06-10T02:40:02.650 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/170 2024-06-10T02:40:02.682 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/170 2024-06-10T02:40:02.710 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/170 2024-06-10T02:40:02.744 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/170 2024-06-10T02:40:02.783 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/170 2024-06-10T02:40:02.815 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/170 2024-06-10T02:40:02.845 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/170 2024-06-10T02:40:02.879 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/170 2024-06-10T02:40:02.912 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/170 2024-06-10T02:40:02.953 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/170 2024-06-10T02:40:02.988 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/170 2024-06-10T02:40:03.074 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/170 2024-06-10T02:40:03.119 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/170 2024-06-10T02:40:03.158 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/170 2024-06-10T02:40:03.205 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/170 2024-06-10T02:40:03.303 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-packaging-20.9-5.el9.noarch 83/170 2024-06-10T02:40:03.351 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 84/170 2024-06-10T02:40:04.067 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 85/170 2024-06-10T02:40:04.115 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 86/170 2024-06-10T02:40:04.148 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 87/170 2024-06-10T02:40:04.247 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 88/170 2024-06-10T02:40:04.304 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 89/170 2024-06-10T02:40:04.348 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 90/170 2024-06-10T02:40:04.435 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 91/170 2024-06-10T02:40:04.484 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 92/170 2024-06-10T02:40:04.660 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : openblas-0.3.21-2.el9.x86_64 93/170 2024-06-10T02:40:04.691 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 94/170 2024-06-10T02:40:04.768 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 95/170 2024-06-10T02:40:05.405 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 96/170 2024-06-10T02:40:05.529 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 97/170 2024-06-10T02:40:06.733 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 98/170 2024-06-10T02:40:06.819 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 99/170 2024-06-10T02:40:06.846 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 100/170 2024-06-10T02:40:06.871 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 101/170 2024-06-10T02:40:07.181 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 102/170 2024-06-10T02:40:07.225 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : librbd1-2:19.0.0-2946.g3d89accf.el9.x86_64 103/170 2024-06-10T02:40:07.299 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: librbd1-2:19.0.0-2946.g3d89accf.el9.x86_64 103/170 2024-06-10T02:40:07.330 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-rbd-2:19.0.0-2946.g3d89accf.el9.x86_64 104/170 2024-06-10T02:40:07.366 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 105/170 2024-06-10T02:40:07.839 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 106/170 2024-06-10T02:40:07.870 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : librgw2-2:19.0.0-2946.g3d89accf.el9.x86_64 107/170 2024-06-10T02:40:07.933 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: librgw2-2:19.0.0-2946.g3d89accf.el9.x86_64 107/170 2024-06-10T02:40:07.955 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-rgw-2:19.0.0-2946.g3d89accf.el9.x86_64 108/170 2024-06-10T02:40:10.197 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 109/170 2024-06-10T02:40:10.226 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 109/170 2024-06-10T02:40:10.765 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 109/170 2024-06-10T02:40:10.804 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-base-2:19.0.0-2946.g3d89accf.el9.x86_64 110/170 2024-06-10T02:40:10.897 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-base-2:19.0.0-2946.g3d89accf.el9.x86_64 110/170 2024-06-10T02:40:10.897 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-06-10T02:40:10.897 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-06-10T02:40:10.897 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-06-10T02:40:10.922 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-selinux-2:19.0.0-2946.g3d89accf.el9.x86_64 111/170 2024-06-10T02:40:22.923 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2946.g3d89accf.el9.x86_64 111/170 2024-06-10T02:40:22.924 INFO:teuthology.orchestra.run.smithi188.stdout:skipping the directory /sys 2024-06-10T02:40:22.924 INFO:teuthology.orchestra.run.smithi188.stdout:skipping the directory /proc 2024-06-10T02:40:22.924 INFO:teuthology.orchestra.run.smithi188.stdout:skipping the directory /mnt 2024-06-10T02:40:22.924 INFO:teuthology.orchestra.run.smithi188.stdout:skipping the directory /var/tmp 2024-06-10T02:40:22.924 INFO:teuthology.orchestra.run.smithi188.stdout:skipping the directory /home 2024-06-10T02:40:22.924 INFO:teuthology.orchestra.run.smithi188.stdout:skipping the directory /root 2024-06-10T02:40:22.924 INFO:teuthology.orchestra.run.smithi188.stdout:skipping the directory /tmp 2024-06-10T02:40:22.924 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-06-10T02:40:23.097 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-volume-2:19.0.0-2946.g3d89accf.el9.noarch 112/170 2024-06-10T02:40:23.112 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-volume-2:19.0.0-2946.g3d89accf.el9.noarch 112/170 2024-06-10T02:40:23.112 INFO:teuthology.orchestra.run.smithi188.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-10T02:40:23.112 INFO:teuthology.orchestra.run.smithi188.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-06-10T02:40:23.112 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-06-10T02:40:24.600 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-osd-2:19.0.0-2946.g3d89accf.el9.x86_64 113/170 2024-06-10T02:40:24.627 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-osd-2:19.0.0-2946.g3d89accf.el9.x86_64 113/170 2024-06-10T02:40:24.627 INFO:teuthology.orchestra.run.smithi188.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-10T02:40:24.628 INFO:teuthology.orchestra.run.smithi188.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-06-10T02:40:24.628 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-06-10T02:40:24.628 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-06-10T02:40:24.628 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-06-10T02:40:24.859 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-mds-2:19.0.0-2946.g3d89accf.el9.x86_64 114/170 2024-06-10T02:40:24.880 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-mds-2:19.0.0-2946.g3d89accf.el9.x86_64 114/170 2024-06-10T02:40:24.881 INFO:teuthology.orchestra.run.smithi188.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-10T02:40:24.881 INFO:teuthology.orchestra.run.smithi188.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-06-10T02:40:24.881 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-06-10T02:40:24.881 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-06-10T02:40:24.881 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-06-10T02:40:25.318 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-mon-2:19.0.0-2946.g3d89accf.el9.x86_64 115/170 2024-06-10T02:40:25.340 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-mon-2:19.0.0-2946.g3d89accf.el9.x86_64 115/170 2024-06-10T02:40:25.341 INFO:teuthology.orchestra.run.smithi188.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-10T02:40:25.341 INFO:teuthology.orchestra.run.smithi188.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-06-10T02:40:25.341 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-06-10T02:40:25.341 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-06-10T02:40:25.341 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-06-10T02:40:25.456 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : automake-1.16.2-8.el9.noarch 116/170 2024-06-10T02:40:25.504 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-ply-3.11-14.el9.noarch 117/170 2024-06-10T02:40:25.560 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 118/170 2024-06-10T02:40:25.745 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 119/170 2024-06-10T02:40:25.794 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 120/170 2024-06-10T02:40:25.871 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 121/170 2024-06-10T02:40:25.959 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 122/170 2024-06-10T02:40:26.097 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 123/170 2024-06-10T02:40:26.135 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 124/170 2024-06-10T02:40:26.176 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 125/170 2024-06-10T02:40:26.203 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : fuse-2.9.9-15.el9.x86_64 126/170 2024-06-10T02:40:26.655 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 127/170 2024-06-10T02:40:26.685 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 127/170 2024-06-10T02:40:26.717 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2946.g3d89accf.e 128/170 2024-06-10T02:40:26.779 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2946.g3d89accf. 129/170 2024-06-10T02:40:26.808 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2946.g3d89accf.el9.noa 130/170 2024-06-10T02:40:27.841 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2946.g3d89accf.el9.noa 130/170 2024-06-10T02:40:27.877 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2946.g3d89accf.el9.n 131/170 2024-06-10T02:40:28.844 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2946.g3d89accf.el9.n 131/170 2024-06-10T02:40:28.877 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2946.g3d8 132/170 2024-06-10T02:40:28.978 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2946.g3d8 132/170 2024-06-10T02:40:29.140 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2946.g3d89accf.el 133/170 2024-06-10T02:40:29.171 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-mgr-2:19.0.0-2946.g3d89accf.el9.x86_64 134/170 2024-06-10T02:40:29.194 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2946.g3d89accf.el9.x86_64 134/170 2024-06-10T02:40:29.194 INFO:teuthology.orchestra.run.smithi188.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-10T02:40:29.194 INFO:teuthology.orchestra.run.smithi188.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-06-10T02:40:29.194 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-06-10T02:40:29.194 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-06-10T02:40:29.195 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-06-10T02:40:29.239 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-mgr-rook-2:19.0.0-2946.g3d89accf.el9.noarch 135/170 2024-06-10T02:40:29.249 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2946.g3d89accf.el9.noarch 135/170 2024-06-10T02:40:29.348 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-2:19.0.0-2946.g3d89accf.el9.x86_64 136/170 2024-06-10T02:40:29.396 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 137/170 2024-06-10T02:40:30.300 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libtool-2.4.6-45.el9.x86_64 138/170 2024-06-10T02:40:30.867 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-radosgw-2:19.0.0-2946.g3d89accf.el9.x86_64 139/170 2024-06-10T02:40:30.868 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2946.g3d89accf.el9.x86_64 139/170 2024-06-10T02:40:30.868 INFO:teuthology.orchestra.run.smithi188.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-10T02:40:30.868 INFO:teuthology.orchestra.run.smithi188.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-06-10T02:40:30.868 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-06-10T02:40:30.868 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-06-10T02:40:30.868 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-06-10T02:40:30.868 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2946.g3d89a 140/170 2024-06-10T02:40:30.868 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2946.g3d89a 140/170 2024-06-10T02:40:30.868 INFO:teuthology.orchestra.run.smithi188.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-10T02:40:30.868 INFO:teuthology.orchestra.run.smithi188.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-06-10T02:40:30.868 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-06-10T02:40:30.868 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : cephfs-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 141/170 2024-06-10T02:40:30.869 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 141/170 2024-06-10T02:40:30.869 INFO:teuthology.orchestra.run.smithi188.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-10T02:40:30.869 INFO:teuthology.orchestra.run.smithi188.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-06-10T02:40:30.869 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-06-10T02:40:30.869 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-06-10T02:40:30.869 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-06-10T02:40:30.869 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : rbd-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 142/170 2024-06-10T02:40:30.869 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 142/170 2024-06-10T02:40:30.869 INFO:teuthology.orchestra.run.smithi188.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-10T02:40:30.869 INFO:teuthology.orchestra.run.smithi188.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-06-10T02:40:30.869 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-06-10T02:40:30.869 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-06-10T02:40:30.870 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-06-10T02:40:35.330 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-test-2:19.0.0-2946.g3d89accf.el9.x86_64 143/170 2024-06-10T02:40:35.372 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : rbd-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 144/170 2024-06-10T02:40:35.412 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : rbd-nbd-2:19.0.0-2946.g3d89accf.el9.x86_64 145/170 2024-06-10T02:40:35.444 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 146/170 2024-06-10T02:40:35.472 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 147/170 2024-06-10T02:40:35.651 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : cephfs-shell-2:19.0.0-2946.g3d89accf.el9.x86_64 148/170 2024-06-10T02:40:35.693 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 149/170 2024-06-10T02:40:35.725 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libcephfs-devel-2:19.0.0-2946.g3d89accf.el9.x86_ 150/170 2024-06-10T02:40:35.757 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : elfutils-debuginfod-client-0.191-4.el9.x86_64 151/170 2024-06-10T02:40:35.787 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : NetworkManager-team-1:1.47.91-1.el9.x86_64 152/170 2024-06-10T02:40:35.833 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : NetworkManager-tui-1:1.47.91-1.el9.x86_64 153/170 2024-06-10T02:40:36.455 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-06-10T02:40:36.498 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 154/170 2024-06-10T02:40:36.517 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-06-10T02:40:37.345 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : cephfs-top-2:19.0.0-2946.g3d89accf.el9.noarch 155/170 2024-06-10T02:40:37.436 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 156/170 2024-06-10T02:40:37.478 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 157/170 2024-06-10T02:40:37.558 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : flex-2.6.4-9.el9.x86_64 158/170 2024-06-10T02:40:37.590 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : bison-3.7.4-5.el9.x86_64 159/170 2024-06-10T02:40:37.615 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-06-10T02:40:37.641 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : elfutils-debuginfod-client-0.190-2.el9.x86_64 161/170 2024-06-10T02:40:37.667 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : elfutils-libs-0.190-2.el9.x86_64 162/170 2024-06-10T02:40:37.667 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-06-10T02:40:37.751 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-06-10T02:40:37.776 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 164/170 2024-06-10T02:40:37.776 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 165/170 2024-06-10T02:40:37.860 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 165/170 2024-06-10T02:40:37.885 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 166/170 2024-06-10T02:40:37.890 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-06-10T02:40:37.890 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-06-10T02:40:38.071 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-06-10T02:40:38.071 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-06-10T02:40:38.163 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-06-10T02:40:38.188 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 169/170 2024-06-10T02:40:38.251 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-06-10T02:40:43.907 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-06-10T02:40:43.907 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-2:19.0.0-2946.g3d89accf.el9.x86_64 1/170 2024-06-10T02:40:43.907 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-base-2:19.0.0-2946.g3d89accf.el9.x86_64 2/170 2024-06-10T02:40:43.907 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 3/170 2024-06-10T02:40:43.908 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 4/170 2024-06-10T02:40:43.908 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2946.g3d89a 5/170 2024-06-10T02:40:43.908 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-mds-2:19.0.0-2946.g3d89accf.el9.x86_64 6/170 2024-06-10T02:40:43.908 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-mgr-2:19.0.0-2946.g3d89accf.el9.x86_64 7/170 2024-06-10T02:40:43.908 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-mon-2:19.0.0-2946.g3d89accf.el9.x86_64 8/170 2024-06-10T02:40:43.908 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-osd-2:19.0.0-2946.g3d89accf.el9.x86_64 9/170 2024-06-10T02:40:43.908 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-radosgw-2:19.0.0-2946.g3d89accf.el9.x86_64 10/170 2024-06-10T02:40:43.908 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-selinux-2:19.0.0-2946.g3d89accf.el9.x86_64 11/170 2024-06-10T02:40:43.908 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-test-2:19.0.0-2946.g3d89accf.el9.x86_64 12/170 2024-06-10T02:40:43.908 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : cephfs-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 13/170 2024-06-10T02:40:43.908 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : cephfs-shell-2:19.0.0-2946.g3d89accf.el9.x86_64 14/170 2024-06-10T02:40:43.909 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libcephfs-devel-2:19.0.0-2946.g3d89accf.el9.x86_ 15/170 2024-06-10T02:40:43.909 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libcephfs2-2:19.0.0-2946.g3d89accf.el9.x86_64 16/170 2024-06-10T02:40:43.909 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libcephsqlite-2:19.0.0-2946.g3d89accf.el9.x86_64 17/170 2024-06-10T02:40:43.909 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : librados-devel-2:19.0.0-2946.g3d89accf.el9.x86_6 18/170 2024-06-10T02:40:43.909 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libradosstriper1-2:19.0.0-2946.g3d89accf.el9.x86 19/170 2024-06-10T02:40:43.909 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : librgw2-2:19.0.0-2946.g3d89accf.el9.x86_64 20/170 2024-06-10T02:40:43.909 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2946.g3d89accf.el 21/170 2024-06-10T02:40:43.909 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-ceph-common-2:19.0.0-2946.g3d89accf.el9. 22/170 2024-06-10T02:40:43.909 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-cephfs-2:19.0.0-2946.g3d89accf.el9.x86_6 23/170 2024-06-10T02:40:43.909 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-rados-2:19.0.0-2946.g3d89accf.el9.x86_64 24/170 2024-06-10T02:40:43.909 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-rbd-2:19.0.0-2946.g3d89accf.el9.x86_64 25/170 2024-06-10T02:40:43.909 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-rgw-2:19.0.0-2946.g3d89accf.el9.x86_64 26/170 2024-06-10T02:40:43.910 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : rbd-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 27/170 2024-06-10T02:40:43.910 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : rbd-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 28/170 2024-06-10T02:40:43.910 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : rbd-nbd-2:19.0.0-2946.g3d89accf.el9.x86_64 29/170 2024-06-10T02:40:43.910 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2946.g3d89accf. 30/170 2024-06-10T02:40:43.910 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2946.g3d89accf.el9.noa 31/170 2024-06-10T02:40:43.910 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2946.g3d89accf.el9.n 32/170 2024-06-10T02:40:43.910 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2946.g3d8 33/170 2024-06-10T02:40:43.910 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2946.g3d89accf.el 34/170 2024-06-10T02:40:43.910 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2946.g3d89accf.el9.noarch 35/170 2024-06-10T02:40:43.910 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2946.g3d89accf.e 36/170 2024-06-10T02:40:43.910 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-volume-2:19.0.0-2946.g3d89accf.el9.noarch 37/170 2024-06-10T02:40:43.910 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 38/170 2024-06-10T02:40:43.911 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : cephfs-top-2:19.0.0-2946.g3d89accf.el9.noarch 39/170 2024-06-10T02:40:43.911 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/170 2024-06-10T02:40:43.911 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/170 2024-06-10T02:40:43.911 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/170 2024-06-10T02:40:43.911 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/170 2024-06-10T02:40:43.911 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/170 2024-06-10T02:40:43.911 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/170 2024-06-10T02:40:43.911 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/170 2024-06-10T02:40:43.911 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/170 2024-06-10T02:40:43.911 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/170 2024-06-10T02:40:43.911 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/170 2024-06-10T02:40:43.912 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/170 2024-06-10T02:40:43.912 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/170 2024-06-10T02:40:43.912 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/170 2024-06-10T02:40:43.912 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/170 2024-06-10T02:40:43.912 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/170 2024-06-10T02:40:43.912 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/170 2024-06-10T02:40:43.912 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/170 2024-06-10T02:40:43.912 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/170 2024-06-10T02:40:43.912 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/170 2024-06-10T02:40:43.912 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/170 2024-06-10T02:40:43.912 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/170 2024-06-10T02:40:43.912 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/170 2024-06-10T02:40:43.912 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/170 2024-06-10T02:40:43.913 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/170 2024-06-10T02:40:43.913 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/170 2024-06-10T02:40:43.913 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/170 2024-06-10T02:40:43.913 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/170 2024-06-10T02:40:43.913 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/170 2024-06-10T02:40:43.913 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/170 2024-06-10T02:40:43.913 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/170 2024-06-10T02:40:43.913 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/170 2024-06-10T02:40:43.913 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/170 2024-06-10T02:40:43.913 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/170 2024-06-10T02:40:43.913 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/170 2024-06-10T02:40:43.913 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/170 2024-06-10T02:40:43.913 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/170 2024-06-10T02:40:43.914 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/170 2024-06-10T02:40:43.914 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 77/170 2024-06-10T02:40:43.914 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/170 2024-06-10T02:40:43.914 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/170 2024-06-10T02:40:43.914 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/170 2024-06-10T02:40:43.914 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/170 2024-06-10T02:40:43.914 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/170 2024-06-10T02:40:43.914 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/170 2024-06-10T02:40:43.914 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/170 2024-06-10T02:40:43.914 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/170 2024-06-10T02:40:43.914 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/170 2024-06-10T02:40:43.914 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/170 2024-06-10T02:40:43.915 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/170 2024-06-10T02:40:43.915 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/170 2024-06-10T02:40:43.915 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/170 2024-06-10T02:40:43.915 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/170 2024-06-10T02:40:43.915 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/170 2024-06-10T02:40:43.915 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/170 2024-06-10T02:40:43.915 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/170 2024-06-10T02:40:43.915 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/170 2024-06-10T02:40:43.915 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/170 2024-06-10T02:40:43.915 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/170 2024-06-10T02:40:43.915 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/170 2024-06-10T02:40:43.915 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 99/170 2024-06-10T02:40:43.915 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/170 2024-06-10T02:40:43.915 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 101/170 2024-06-10T02:40:43.916 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/170 2024-06-10T02:40:43.916 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/170 2024-06-10T02:40:43.916 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/170 2024-06-10T02:40:43.916 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/170 2024-06-10T02:40:43.916 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/170 2024-06-10T02:40:43.916 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/170 2024-06-10T02:40:43.916 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/170 2024-06-10T02:40:43.916 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/170 2024-06-10T02:40:43.916 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/170 2024-06-10T02:40:43.916 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/170 2024-06-10T02:40:43.916 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/170 2024-06-10T02:40:43.916 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/170 2024-06-10T02:40:43.917 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/170 2024-06-10T02:40:43.917 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/170 2024-06-10T02:40:43.917 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/170 2024-06-10T02:40:43.917 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/170 2024-06-10T02:40:43.917 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 118/170 2024-06-10T02:40:43.917 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/170 2024-06-10T02:40:43.917 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/170 2024-06-10T02:40:43.917 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/170 2024-06-10T02:40:43.917 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-google-auth-1:2.29.0-2.el9.noarch 122/170 2024-06-10T02:40:43.917 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/170 2024-06-10T02:40:43.917 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/170 2024-06-10T02:40:43.917 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/170 2024-06-10T02:40:43.917 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/170 2024-06-10T02:40:43.917 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/170 2024-06-10T02:40:43.918 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/170 2024-06-10T02:40:43.918 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/170 2024-06-10T02:40:43.918 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/170 2024-06-10T02:40:43.918 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/170 2024-06-10T02:40:43.918 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/170 2024-06-10T02:40:43.918 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/170 2024-06-10T02:40:43.918 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/170 2024-06-10T02:40:43.918 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/170 2024-06-10T02:40:43.918 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/170 2024-06-10T02:40:43.918 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/170 2024-06-10T02:40:43.918 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/170 2024-06-10T02:40:43.919 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/170 2024-06-10T02:40:43.919 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/170 2024-06-10T02:40:43.919 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/170 2024-06-10T02:40:43.919 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/170 2024-06-10T02:40:43.919 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/170 2024-06-10T02:40:43.919 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/170 2024-06-10T02:40:43.919 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/170 2024-06-10T02:40:43.919 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/170 2024-06-10T02:40:43.919 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/170 2024-06-10T02:40:43.919 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/170 2024-06-10T02:40:43.919 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : librados2-2:19.0.0-2946.g3d89accf.el9.x86_64 149/170 2024-06-10T02:40:43.919 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/170 2024-06-10T02:40:43.920 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : librbd1-2:19.0.0-2946.g3d89accf.el9.x86_64 151/170 2024-06-10T02:40:43.920 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/170 2024-06-10T02:40:43.920 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : NetworkManager-1:1.47.91-1.el9.x86_64 153/170 2024-06-10T02:40:43.920 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/170 2024-06-10T02:40:43.920 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : NetworkManager-libnm-1:1.47.91-1.el9.x86_64 155/170 2024-06-10T02:40:43.920 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/170 2024-06-10T02:40:43.920 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : NetworkManager-team-1:1.47.91-1.el9.x86_64 157/170 2024-06-10T02:40:43.920 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/170 2024-06-10T02:40:43.920 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : NetworkManager-tui-1:1.47.91-1.el9.x86_64 159/170 2024-06-10T02:40:43.920 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-06-10T02:40:43.920 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : elfutils-debuginfod-client-0.191-4.el9.x86_64 161/170 2024-06-10T02:40:43.920 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 162/170 2024-06-10T02:40:43.920 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : elfutils-libelf-0.191-4.el9.x86_64 163/170 2024-06-10T02:40:43.921 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 164/170 2024-06-10T02:40:43.921 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : elfutils-libs-0.191-4.el9.x86_64 165/170 2024-06-10T02:40:43.921 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 166/170 2024-06-10T02:40:43.921 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 167/170 2024-06-10T02:40:43.921 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 168/170 2024-06-10T02:40:43.921 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 169/170 2024-06-10T02:40:46.529 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 170/170 2024-06-10T02:40:46.529 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-06-10T02:40:46.529 INFO:teuthology.orchestra.run.smithi188.stdout:Upgraded: 2024-06-10T02:40:46.529 INFO:teuthology.orchestra.run.smithi188.stdout: NetworkManager-1:1.47.91-1.el9.x86_64 2024-06-10T02:40:46.529 INFO:teuthology.orchestra.run.smithi188.stdout: NetworkManager-libnm-1:1.47.91-1.el9.x86_64 2024-06-10T02:40:46.529 INFO:teuthology.orchestra.run.smithi188.stdout: NetworkManager-team-1:1.47.91-1.el9.x86_64 2024-06-10T02:40:46.529 INFO:teuthology.orchestra.run.smithi188.stdout: NetworkManager-tui-1:1.47.91-1.el9.x86_64 2024-06-10T02:40:46.529 INFO:teuthology.orchestra.run.smithi188.stdout: elfutils-debuginfod-client-0.191-4.el9.x86_64 2024-06-10T02:40:46.530 INFO:teuthology.orchestra.run.smithi188.stdout: elfutils-libelf-0.191-4.el9.x86_64 2024-06-10T02:40:46.530 INFO:teuthology.orchestra.run.smithi188.stdout: elfutils-libs-0.191-4.el9.x86_64 2024-06-10T02:40:46.530 INFO:teuthology.orchestra.run.smithi188.stdout: librados2-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T02:40:46.530 INFO:teuthology.orchestra.run.smithi188.stdout: librbd1-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T02:40:46.530 INFO:teuthology.orchestra.run.smithi188.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-06-10T02:40:46.530 INFO:teuthology.orchestra.run.smithi188.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-06-10T02:40:46.530 INFO:teuthology.orchestra.run.smithi188.stdout:Installed: 2024-06-10T02:40:46.530 INFO:teuthology.orchestra.run.smithi188.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-06-10T02:40:46.530 INFO:teuthology.orchestra.run.smithi188.stdout: automake-1.16.2-8.el9.noarch 2024-06-10T02:40:46.530 INFO:teuthology.orchestra.run.smithi188.stdout: bison-3.7.4-5.el9.x86_64 2024-06-10T02:40:46.530 INFO:teuthology.orchestra.run.smithi188.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-06-10T02:40:46.531 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T02:40:46.531 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-base-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T02:40:46.531 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T02:40:46.531 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T02:40:46.531 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-grafana-dashboards-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-10T02:40:46.531 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-immutable-object-cache-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T02:40:46.531 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mds-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T02:40:46.531 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T02:40:46.531 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-10T02:40:46.531 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-dashboard-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-10T02:40:46.531 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-10T02:40:46.531 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-modules-core-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-10T02:40:46.531 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-rook-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-10T02:40:46.532 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mon-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T02:40:46.532 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-osd-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T02:40:46.532 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-prometheus-alerts-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-10T02:40:46.532 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-radosgw-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T02:40:46.532 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-selinux-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T02:40:46.532 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-test-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T02:40:46.532 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-volume-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-10T02:40:46.532 INFO:teuthology.orchestra.run.smithi188.stdout: cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-10T02:40:46.532 INFO:teuthology.orchestra.run.smithi188.stdout: cephfs-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T02:40:46.532 INFO:teuthology.orchestra.run.smithi188.stdout: cephfs-shell-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T02:40:46.532 INFO:teuthology.orchestra.run.smithi188.stdout: cephfs-top-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-10T02:40:46.532 INFO:teuthology.orchestra.run.smithi188.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-06-10T02:40:46.532 INFO:teuthology.orchestra.run.smithi188.stdout: flex-2.6.4-9.el9.x86_64 2024-06-10T02:40:46.533 INFO:teuthology.orchestra.run.smithi188.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-06-10T02:40:46.533 INFO:teuthology.orchestra.run.smithi188.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-06-10T02:40:46.533 INFO:teuthology.orchestra.run.smithi188.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-06-10T02:40:46.533 INFO:teuthology.orchestra.run.smithi188.stdout: fuse-2.9.9-15.el9.x86_64 2024-06-10T02:40:46.533 INFO:teuthology.orchestra.run.smithi188.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-06-10T02:40:46.533 INFO:teuthology.orchestra.run.smithi188.stdout: grpc-data-1.46.7-10.el9.noarch 2024-06-10T02:40:46.533 INFO:teuthology.orchestra.run.smithi188.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-06-10T02:40:46.533 INFO:teuthology.orchestra.run.smithi188.stdout: libarrow-9.0.0-11.el9.x86_64 2024-06-10T02:40:46.533 INFO:teuthology.orchestra.run.smithi188.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-06-10T02:40:46.533 INFO:teuthology.orchestra.run.smithi188.stdout: libcephfs-devel-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T02:40:46.533 INFO:teuthology.orchestra.run.smithi188.stdout: libcephfs2-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T02:40:46.533 INFO:teuthology.orchestra.run.smithi188.stdout: libcephsqlite-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T02:40:46.533 INFO:teuthology.orchestra.run.smithi188.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-06-10T02:40:46.534 INFO:teuthology.orchestra.run.smithi188.stdout: liboath-2.6.7-2.el9.x86_64 2024-06-10T02:40:46.534 INFO:teuthology.orchestra.run.smithi188.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-06-10T02:40:46.534 INFO:teuthology.orchestra.run.smithi188.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-06-10T02:40:46.534 INFO:teuthology.orchestra.run.smithi188.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-06-10T02:40:46.534 INFO:teuthology.orchestra.run.smithi188.stdout: librados-devel-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T02:40:46.534 INFO:teuthology.orchestra.run.smithi188.stdout: libradosstriper1-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T02:40:46.534 INFO:teuthology.orchestra.run.smithi188.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-06-10T02:40:46.534 INFO:teuthology.orchestra.run.smithi188.stdout: librgw2-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T02:40:46.534 INFO:teuthology.orchestra.run.smithi188.stdout: libtool-2.4.6-45.el9.x86_64 2024-06-10T02:40:46.534 INFO:teuthology.orchestra.run.smithi188.stdout: libunwind-1.6.2-1.el9.x86_64 2024-06-10T02:40:46.534 INFO:teuthology.orchestra.run.smithi188.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-06-10T02:40:46.534 INFO:teuthology.orchestra.run.smithi188.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-06-10T02:40:46.534 INFO:teuthology.orchestra.run.smithi188.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-06-10T02:40:46.535 INFO:teuthology.orchestra.run.smithi188.stdout: lua-5.4.4-4.el9.x86_64 2024-06-10T02:40:46.535 INFO:teuthology.orchestra.run.smithi188.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-06-10T02:40:46.535 INFO:teuthology.orchestra.run.smithi188.stdout: luarocks-3.9.2-5.el9.noarch 2024-06-10T02:40:46.535 INFO:teuthology.orchestra.run.smithi188.stdout: openblas-0.3.21-2.el9.x86_64 2024-06-10T02:40:46.535 INFO:teuthology.orchestra.run.smithi188.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-06-10T02:40:46.535 INFO:teuthology.orchestra.run.smithi188.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-06-10T02:40:46.535 INFO:teuthology.orchestra.run.smithi188.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-06-10T02:40:46.535 INFO:teuthology.orchestra.run.smithi188.stdout: postgresql-13.14-1.el9.x86_64 2024-06-10T02:40:46.535 INFO:teuthology.orchestra.run.smithi188.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-06-10T02:40:46.535 INFO:teuthology.orchestra.run.smithi188.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-06-10T02:40:46.535 INFO:teuthology.orchestra.run.smithi188.stdout: postgresql-server-13.14-1.el9.x86_64 2024-06-10T02:40:46.535 INFO:teuthology.orchestra.run.smithi188.stdout: protobuf-3.14.0-13.el9.x86_64 2024-06-10T02:40:46.535 INFO:teuthology.orchestra.run.smithi188.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-06-10T02:40:46.536 INFO:teuthology.orchestra.run.smithi188.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-06-10T02:40:46.536 INFO:teuthology.orchestra.run.smithi188.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-06-10T02:40:46.536 INFO:teuthology.orchestra.run.smithi188.stdout: python3-babel-2.9.1-2.el9.noarch 2024-06-10T02:40:46.536 INFO:teuthology.orchestra.run.smithi188.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-06-10T02:40:46.536 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-06-10T02:40:46.536 INFO:teuthology.orchestra.run.smithi188.stdout: python3-ceph-argparse-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T02:40:46.536 INFO:teuthology.orchestra.run.smithi188.stdout: python3-ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T02:40:46.536 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cephfs-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T02:40:46.536 INFO:teuthology.orchestra.run.smithi188.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-06-10T02:40:46.536 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-06-10T02:40:46.536 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-06-10T02:40:46.536 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-06-10T02:40:46.536 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-06-10T02:40:46.537 INFO:teuthology.orchestra.run.smithi188.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-06-10T02:40:46.537 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-06-10T02:40:46.537 INFO:teuthology.orchestra.run.smithi188.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-06-10T02:40:46.537 INFO:teuthology.orchestra.run.smithi188.stdout: python3-google-auth-1:2.29.0-2.el9.noarch 2024-06-10T02:40:46.537 INFO:teuthology.orchestra.run.smithi188.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-06-10T02:40:46.537 INFO:teuthology.orchestra.run.smithi188.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-06-10T02:40:46.537 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-06-10T02:40:46.537 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-06-10T02:40:46.537 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-06-10T02:40:46.537 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-06-10T02:40:46.537 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-06-10T02:40:46.537 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-06-10T02:40:46.537 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-06-10T02:40:46.538 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-06-10T02:40:46.538 INFO:teuthology.orchestra.run.smithi188.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-06-10T02:40:46.538 INFO:teuthology.orchestra.run.smithi188.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-06-10T02:40:46.538 INFO:teuthology.orchestra.run.smithi188.stdout: python3-mako-1.1.4-6.el9.noarch 2024-06-10T02:40:46.538 INFO:teuthology.orchestra.run.smithi188.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-06-10T02:40:46.538 INFO:teuthology.orchestra.run.smithi188.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-06-10T02:40:46.538 INFO:teuthology.orchestra.run.smithi188.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-06-10T02:40:46.538 INFO:teuthology.orchestra.run.smithi188.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-06-10T02:40:46.538 INFO:teuthology.orchestra.run.smithi188.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-06-10T02:40:46.538 INFO:teuthology.orchestra.run.smithi188.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-06-10T02:40:46.538 INFO:teuthology.orchestra.run.smithi188.stdout: python3-packaging-20.9-5.el9.noarch 2024-06-10T02:40:46.538 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-06-10T02:40:46.538 INFO:teuthology.orchestra.run.smithi188.stdout: python3-ply-3.11-14.el9.noarch 2024-06-10T02:40:46.538 INFO:teuthology.orchestra.run.smithi188.stdout: python3-portend-3.1.0-2.el9.noarch 2024-06-10T02:40:46.539 INFO:teuthology.orchestra.run.smithi188.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-06-10T02:40:46.539 INFO:teuthology.orchestra.run.smithi188.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-06-10T02:40:46.539 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-06-10T02:40:46.539 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-06-10T02:40:46.539 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-06-10T02:40:46.539 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pycparser-2.20-6.el9.noarch 2024-06-10T02:40:46.539 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-06-10T02:40:46.539 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pytz-2021.1-5.el9.noarch 2024-06-10T02:40:46.539 INFO:teuthology.orchestra.run.smithi188.stdout: python3-rados-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T02:40:46.539 INFO:teuthology.orchestra.run.smithi188.stdout: python3-rbd-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T02:40:46.539 INFO:teuthology.orchestra.run.smithi188.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-06-10T02:40:46.539 INFO:teuthology.orchestra.run.smithi188.stdout: python3-requests-2.25.1-8.el9.noarch 2024-06-10T02:40:46.539 INFO:teuthology.orchestra.run.smithi188.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-06-10T02:40:46.540 INFO:teuthology.orchestra.run.smithi188.stdout: python3-rgw-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T02:40:46.540 INFO:teuthology.orchestra.run.smithi188.stdout: python3-routes-2.5.1-5.el9.noarch 2024-06-10T02:40:46.540 INFO:teuthology.orchestra.run.smithi188.stdout: python3-rsa-4.9-2.el9.noarch 2024-06-10T02:40:46.540 INFO:teuthology.orchestra.run.smithi188.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-06-10T02:40:46.540 INFO:teuthology.orchestra.run.smithi188.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-06-10T02:40:46.540 INFO:teuthology.orchestra.run.smithi188.stdout: python3-toml-0.10.2-6.el9.noarch 2024-06-10T02:40:46.540 INFO:teuthology.orchestra.run.smithi188.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-06-10T02:40:46.540 INFO:teuthology.orchestra.run.smithi188.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-06-10T02:40:46.540 INFO:teuthology.orchestra.run.smithi188.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-06-10T02:40:46.540 INFO:teuthology.orchestra.run.smithi188.stdout: python3-webob-1.8.7-6.el9.noarch 2024-06-10T02:40:46.540 INFO:teuthology.orchestra.run.smithi188.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-06-10T02:40:46.540 INFO:teuthology.orchestra.run.smithi188.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-06-10T02:40:46.540 INFO:teuthology.orchestra.run.smithi188.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-06-10T02:40:46.540 INFO:teuthology.orchestra.run.smithi188.stdout: qatlib-24.02.0-1.el9.x86_64 2024-06-10T02:40:46.541 INFO:teuthology.orchestra.run.smithi188.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-06-10T02:40:46.541 INFO:teuthology.orchestra.run.smithi188.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-06-10T02:40:46.541 INFO:teuthology.orchestra.run.smithi188.stdout: rbd-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T02:40:46.541 INFO:teuthology.orchestra.run.smithi188.stdout: rbd-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T02:40:46.541 INFO:teuthology.orchestra.run.smithi188.stdout: rbd-nbd-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T02:40:46.541 INFO:teuthology.orchestra.run.smithi188.stdout: re2-1:20211101-3.el9.x86_64 2024-06-10T02:40:46.541 INFO:teuthology.orchestra.run.smithi188.stdout: socat-1.7.4.1-5.el9.x86_64 2024-06-10T02:40:46.541 INFO:teuthology.orchestra.run.smithi188.stdout: thrift-0.15.0-2.el9.x86_64 2024-06-10T02:40:46.541 INFO:teuthology.orchestra.run.smithi188.stdout: uuid-1.6.2-55.el9.x86_64 2024-06-10T02:40:46.541 INFO:teuthology.orchestra.run.smithi188.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-06-10T02:40:46.541 INFO:teuthology.orchestra.run.smithi188.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-06-10T02:40:46.542 INFO:teuthology.orchestra.run.smithi188.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-06-10T02:40:46.542 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-06-10T02:40:46.542 INFO:teuthology.orchestra.run.smithi188.stdout:Complete! 2024-06-10T02:40:47.013 DEBUG:teuthology.parallel:result is None 2024-06-10T02:40:47.013 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-10T02:40:47.153 DEBUG:teuthology.orchestra.run.smithi052:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-06-10T02:40:47.181 INFO:teuthology.orchestra.run.smithi052.stdout:19.0.0-2946.g3d89accf.el9 2024-06-10T02:40:47.181 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2946.g3d89accf.el9 2024-06-10T02:40:47.181 INFO:teuthology.task.install:The correct ceph version 19.0.0-2946.g3d89accf is installed. 2024-06-10T02:40:47.183 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-10T02:40:47.417 DEBUG:teuthology.orchestra.run.smithi120:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-06-10T02:40:47.443 INFO:teuthology.orchestra.run.smithi120.stdout:19.0.0-2946.g3d89accf.el9 2024-06-10T02:40:47.443 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2946.g3d89accf.el9 2024-06-10T02:40:47.443 INFO:teuthology.task.install:The correct ceph version 19.0.0-2946.g3d89accf is installed. 2024-06-10T02:40:47.445 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-10T02:40:47.658 DEBUG:teuthology.orchestra.run.smithi188:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-06-10T02:40:47.683 INFO:teuthology.orchestra.run.smithi188.stdout:19.0.0-2946.g3d89accf.el9 2024-06-10T02:40:47.683 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2946.g3d89accf.el9 2024-06-10T02:40:47.683 INFO:teuthology.task.install:The correct ceph version 19.0.0-2946.g3d89accf is installed. 2024-06-10T02:40:47.685 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-06-10T02:40:47.685 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-06-10T02:40:47.685 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-06-10T02:40:47.719 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-06-10T02:40:47.719 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-06-10T02:40:47.751 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-06-10T02:40:47.751 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-06-10T02:40:47.780 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-06-10T02:40:47.781 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-06-10T02:40:47.781 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd of=/usr/bin/daemon-helper 2024-06-10T02:40:47.809 DEBUG:teuthology.orchestra.run.smithi052:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-06-10T02:40:47.879 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-06-10T02:40:47.879 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd of=/usr/bin/daemon-helper 2024-06-10T02:40:47.907 DEBUG:teuthology.orchestra.run.smithi120:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-06-10T02:40:47.973 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-06-10T02:40:47.974 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd of=/usr/bin/daemon-helper 2024-06-10T02:40:48.001 DEBUG:teuthology.orchestra.run.smithi188:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-06-10T02:40:48.068 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-06-10T02:40:48.068 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-06-10T02:40:48.069 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd of=/usr/bin/adjust-ulimits 2024-06-10T02:40:48.100 DEBUG:teuthology.orchestra.run.smithi052:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-06-10T02:40:48.168 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-06-10T02:40:48.168 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd of=/usr/bin/adjust-ulimits 2024-06-10T02:40:48.196 DEBUG:teuthology.orchestra.run.smithi120:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-06-10T02:40:48.263 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-06-10T02:40:48.264 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd of=/usr/bin/adjust-ulimits 2024-06-10T02:40:48.292 DEBUG:teuthology.orchestra.run.smithi188:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-06-10T02:40:48.360 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-06-10T02:40:48.360 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-06-10T02:40:48.360 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd of=/usr/bin/stdin-killer 2024-06-10T02:40:48.390 DEBUG:teuthology.orchestra.run.smithi052:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-06-10T02:40:48.460 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-06-10T02:40:48.460 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd of=/usr/bin/stdin-killer 2024-06-10T02:40:48.488 DEBUG:teuthology.orchestra.run.smithi120:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-06-10T02:40:48.555 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-06-10T02:40:48.555 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd of=/usr/bin/stdin-killer 2024-06-10T02:40:48.583 DEBUG:teuthology.orchestra.run.smithi188:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-06-10T02:40:48.649 INFO:teuthology.run_tasks:Running task cephadm... 2024-06-10T02:40:48.741 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_bal_replicate_threshold': 1, 'mds_export_ephemeral_random_max': 0.1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': '3d89accfe3201286f000e065a1faea2a88c7a6ba', 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2024-06-10T02:40:48.741 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-10T02:40:48.742 INFO:tasks.cephadm:Cluster fsid is d8676e9e-26d2-11ef-bc9b-c7b262605968 2024-06-10T02:40:48.742 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-06-10T02:40:48.742 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.52', 'mon.b': '172.21.15.120', 'mon.c': '172.21.15.188'} 2024-06-10T02:40:48.742 INFO:tasks.cephadm:First mon is mon.a on smithi052 2024-06-10T02:40:48.742 INFO:tasks.cephadm:First mgr is x 2024-06-10T02:40:48.742 INFO:tasks.cephadm:Normalizing hostnames... 2024-06-10T02:40:48.742 DEBUG:teuthology.orchestra.run.smithi052:> sudo hostname $(hostname -s) 2024-06-10T02:40:48.773 DEBUG:teuthology.orchestra.run.smithi120:> sudo hostname $(hostname -s) 2024-06-10T02:40:48.801 DEBUG:teuthology.orchestra.run.smithi188:> sudo hostname $(hostname -s) 2024-06-10T02:40:48.829 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-06-10T02:40:48.830 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-10T02:40:49.056 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '3d89accfe3201286f000e065a1faea2a88c7a6ba', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/55273/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-2946-g3d89accf', 'node_name': '172.21.2.14+braggi14', 'job_name': 'ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-2946.g3d89accf'}, 'url': 'https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-05-30 17:04:17.570799', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'squid', 'chacra_url': 'https://3.chacra.ceph.com/repos/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2024-06-10T02:40:49.265 INFO:tasks.util.chacra:got chacra host 3.chacra.ceph.com, ref squid, sha1 3d89accfe3201286f000e065a1faea2a88c7a6ba from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-10T02:40:49.266 INFO:tasks.cephadm:Discovered cachra url: https://3.chacra.ceph.com/binaries/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/x86_64/flavors/default/cephadm 2024-06-10T02:40:49.266 DEBUG:teuthology.orchestra.run.smithi052:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-06-10T02:40:49.794 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r--. 1 ubuntu ubuntu 781907 Jun 10 02:40 /home/ubuntu/cephtest/cephadm 2024-06-10T02:40:49.794 DEBUG:teuthology.orchestra.run.smithi120:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-06-10T02:40:50.196 INFO:teuthology.orchestra.run.smithi120.stdout:-rw-r--r--. 1 ubuntu ubuntu 781907 Jun 10 02:40 /home/ubuntu/cephtest/cephadm 2024-06-10T02:40:50.196 DEBUG:teuthology.orchestra.run.smithi188:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-06-10T02:40:50.594 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r--. 1 ubuntu ubuntu 781907 Jun 10 02:40 /home/ubuntu/cephtest/cephadm 2024-06-10T02:40:50.595 DEBUG:teuthology.orchestra.run.smithi052:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-06-10T02:40:50.615 DEBUG:teuthology.orchestra.run.smithi120:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-06-10T02:40:50.633 DEBUG:teuthology.orchestra.run.smithi188:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-06-10T02:40:50.670 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba on all hosts... 2024-06-10T02:40:50.671 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba pull 2024-06-10T02:40:50.673 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba pull 2024-06-10T02:40:50.675 DEBUG:teuthology.orchestra.run.smithi188:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba pull 2024-06-10T02:40:51.448 INFO:teuthology.orchestra.run.smithi120.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba... 2024-06-10T02:40:51.571 INFO:teuthology.orchestra.run.smithi188.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba... 2024-06-10T02:40:51.693 INFO:teuthology.orchestra.run.smithi052.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba... 2024-06-10T02:41:30.785 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-10T02:41:30.785 INFO:teuthology.orchestra.run.smithi052.stdout: "ceph_version": "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)", 2024-06-10T02:41:30.785 INFO:teuthology.orchestra.run.smithi052.stdout: "image_id": "b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1", 2024-06-10T02:41:30.785 INFO:teuthology.orchestra.run.smithi052.stdout: "repo_digests": [ 2024-06-10T02:41:30.785 INFO:teuthology.orchestra.run.smithi052.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9" 2024-06-10T02:41:30.786 INFO:teuthology.orchestra.run.smithi052.stdout: ] 2024-06-10T02:41:30.786 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-10T02:41:33.514 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-06-10T02:41:33.514 INFO:teuthology.orchestra.run.smithi120.stdout: "ceph_version": "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)", 2024-06-10T02:41:33.514 INFO:teuthology.orchestra.run.smithi120.stdout: "image_id": "b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1", 2024-06-10T02:41:33.514 INFO:teuthology.orchestra.run.smithi120.stdout: "repo_digests": [ 2024-06-10T02:41:33.514 INFO:teuthology.orchestra.run.smithi120.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9" 2024-06-10T02:41:33.514 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-06-10T02:41:33.515 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-06-10T02:41:34.119 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-06-10T02:41:34.119 INFO:teuthology.orchestra.run.smithi188.stdout: "ceph_version": "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)", 2024-06-10T02:41:34.119 INFO:teuthology.orchestra.run.smithi188.stdout: "image_id": "b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1", 2024-06-10T02:41:34.119 INFO:teuthology.orchestra.run.smithi188.stdout: "repo_digests": [ 2024-06-10T02:41:34.119 INFO:teuthology.orchestra.run.smithi188.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9" 2024-06-10T02:41:34.119 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-06-10T02:41:34.119 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-06-10T02:41:34.144 DEBUG:teuthology.orchestra.run.smithi052:> sudo mkdir -p /etc/ceph 2024-06-10T02:41:34.178 DEBUG:teuthology.orchestra.run.smithi120:> sudo mkdir -p /etc/ceph 2024-06-10T02:41:34.204 DEBUG:teuthology.orchestra.run.smithi188:> sudo mkdir -p /etc/ceph 2024-06-10T02:41:34.231 DEBUG:teuthology.orchestra.run.smithi052:> sudo chmod 777 /etc/ceph 2024-06-10T02:41:34.261 DEBUG:teuthology.orchestra.run.smithi120:> sudo chmod 777 /etc/ceph 2024-06-10T02:41:34.285 DEBUG:teuthology.orchestra.run.smithi188:> sudo chmod 777 /etc/ceph 2024-06-10T02:41:34.308 INFO:tasks.cephadm:Writing seed config... 2024-06-10T02:41:34.309 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-06-10T02:41:34.309 INFO:tasks.cephadm: override: [client] debug client = 20 2024-06-10T02:41:34.309 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-06-10T02:41:34.309 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-06-10T02:41:34.309 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-06-10T02:41:34.309 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-06-10T02:41:34.309 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-06-10T02:41:34.309 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-06-10T02:41:34.309 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-06-10T02:41:34.309 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-06-10T02:41:34.309 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-06-10T02:41:34.310 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-06-10T02:41:34.310 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-06-10T02:41:34.310 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-06-10T02:41:34.310 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-06-10T02:41:34.310 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-06-10T02:41:34.310 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-06-10T02:41:34.310 INFO:tasks.cephadm: override: [mds] mds_bal_replicate_threshold = 1 2024-06-10T02:41:34.310 INFO:tasks.cephadm: override: [mds] mds_export_ephemeral_random_max = 0.1 2024-06-10T02:41:34.310 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-06-10T02:41:34.310 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-06-10T02:41:34.310 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-06-10T02:41:34.310 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-06-10T02:41:34.310 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-06-10T02:41:34.310 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-06-10T02:41:34.311 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-06-10T02:41:34.311 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-06-10T02:41:34.311 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-06-10T02:41:34.311 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-06-10T02:41:34.311 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-06-10T02:41:34.311 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-06-10T02:41:34.311 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-06-10T02:41:34.311 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2024-06-10T02:41:34.311 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-06-10T02:41:34.311 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-06-10T02:41:34.311 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-06-10T02:41:34.311 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-06-10T02:41:34.311 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-06-10T02:41:34.312 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-06-10T02:41:34.312 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-06-10T02:41:34.312 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-06-10T02:41:34.312 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-06-10T02:41:34.312 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-06-10T02:41:34.312 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-06-10T02:41:34.312 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-06-10T02:41:34.312 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-06-10T02:41:34.312 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-06-10T02:41:34.313 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-06-10T02:41:34.313 DEBUG:teuthology.orchestra.run.smithi052:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-06-10T02:41:34.331 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = d8676e9e-26d2-11ef-bc9b-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_bal_replicate_threshold = 1 mds_export_ephemeral_random_max = 0.1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-06-10T02:41:34.331 DEBUG:teuthology.orchestra.run.smithi052:mon.a> sudo journalctl -f -n 0 -u ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968@mon.a.service 2024-06-10T02:41:34.374 DEBUG:teuthology.orchestra.run.smithi052:mgr.x> sudo journalctl -f -n 0 -u ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968@mgr.x.service 2024-06-10T02:41:34.417 INFO:tasks.cephadm:Bootstrapping... 2024-06-10T02:41:34.417 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba -v bootstrap --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.52 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-06-10T02:41:34.618 INFO:teuthology.orchestra.run.smithi052.stdout:-------------------------------------------------------------------------------- 2024-06-10T02:41:34.618 INFO:teuthology.orchestra.run.smithi052.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba', '-v', 'bootstrap', '--fsid', 'd8676e9e-26d2-11ef-bc9b-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.52', '--skip-admin-label'] 2024-06-10T02:41:34.619 INFO:teuthology.orchestra.run.smithi052.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-06-10T02:41:34.619 INFO:teuthology.orchestra.run.smithi052.stdout:Verifying podman|docker is present... 2024-06-10T02:41:34.645 INFO:teuthology.orchestra.run.smithi052.stdout:/bin/podman: stdout 5.0.2 2024-06-10T02:41:34.645 INFO:teuthology.orchestra.run.smithi052.stdout:Verifying lvm2 is present... 2024-06-10T02:41:34.645 INFO:teuthology.orchestra.run.smithi052.stdout:Verifying time synchronization is in place... 2024-06-10T02:41:34.653 INFO:teuthology.orchestra.run.smithi052.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-06-10T02:41:34.653 INFO:teuthology.orchestra.run.smithi052.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-06-10T02:41:34.660 INFO:teuthology.orchestra.run.smithi052.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-06-10T02:41:34.660 INFO:teuthology.orchestra.run.smithi052.stdout:systemctl: stdout inactive 2024-06-10T02:41:34.667 INFO:teuthology.orchestra.run.smithi052.stdout:systemctl: stdout enabled 2024-06-10T02:41:34.673 INFO:teuthology.orchestra.run.smithi052.stdout:systemctl: stdout active 2024-06-10T02:41:34.673 INFO:teuthology.orchestra.run.smithi052.stdout:Unit chronyd.service is enabled and running 2024-06-10T02:41:34.674 INFO:teuthology.orchestra.run.smithi052.stdout:Repeating the final host check... 2024-06-10T02:41:34.699 INFO:teuthology.orchestra.run.smithi052.stdout:/bin/podman: stdout 5.0.2 2024-06-10T02:41:34.699 INFO:teuthology.orchestra.run.smithi052.stdout:podman (/bin/podman) version 5.0.2 is present 2024-06-10T02:41:34.699 INFO:teuthology.orchestra.run.smithi052.stdout:systemctl is present 2024-06-10T02:41:34.699 INFO:teuthology.orchestra.run.smithi052.stdout:lvcreate is present 2024-06-10T02:41:34.706 INFO:teuthology.orchestra.run.smithi052.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-06-10T02:41:34.706 INFO:teuthology.orchestra.run.smithi052.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-06-10T02:41:34.712 INFO:teuthology.orchestra.run.smithi052.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-06-10T02:41:34.712 INFO:teuthology.orchestra.run.smithi052.stdout:systemctl: stdout inactive 2024-06-10T02:41:34.719 INFO:teuthology.orchestra.run.smithi052.stdout:systemctl: stdout enabled 2024-06-10T02:41:34.726 INFO:teuthology.orchestra.run.smithi052.stdout:systemctl: stdout active 2024-06-10T02:41:34.726 INFO:teuthology.orchestra.run.smithi052.stdout:Unit chronyd.service is enabled and running 2024-06-10T02:41:34.726 INFO:teuthology.orchestra.run.smithi052.stdout:Host looks OK 2024-06-10T02:41:34.726 INFO:teuthology.orchestra.run.smithi052.stdout:Cluster fsid: d8676e9e-26d2-11ef-bc9b-c7b262605968 2024-06-10T02:41:34.726 INFO:teuthology.orchestra.run.smithi052.stdout:Acquiring lock 140605011486080 on /run/cephadm/d8676e9e-26d2-11ef-bc9b-c7b262605968.lock 2024-06-10T02:41:34.726 INFO:teuthology.orchestra.run.smithi052.stdout:Lock 140605011486080 acquired on /run/cephadm/d8676e9e-26d2-11ef-bc9b-c7b262605968.lock 2024-06-10T02:41:34.727 INFO:teuthology.orchestra.run.smithi052.stdout:Verifying IP 172.21.15.52 port 3300 ... 2024-06-10T02:41:34.727 INFO:teuthology.orchestra.run.smithi052.stdout:Verifying IP 172.21.15.52 port 6789 ... 2024-06-10T02:41:34.727 INFO:teuthology.orchestra.run.smithi052.stdout:Base mon IP(s) is [172.21.15.52:3300, 172.21.15.52:6789], mon addrv is [v2:172.21.15.52:3300,v1:172.21.15.52:6789] 2024-06-10T02:41:34.730 INFO:teuthology.orchestra.run.smithi052.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.52 metric 100 2024-06-10T02:41:34.730 INFO:teuthology.orchestra.run.smithi052.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.52 metric 100 2024-06-10T02:41:34.733 INFO:teuthology.orchestra.run.smithi052.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-06-10T02:41:34.733 INFO:teuthology.orchestra.run.smithi052.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 1024 pref medium 2024-06-10T02:41:34.733 INFO:teuthology.orchestra.run.smithi052.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 100 pref medium 2024-06-10T02:41:34.735 INFO:teuthology.orchestra.run.smithi052.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-06-10T02:41:34.735 INFO:teuthology.orchestra.run.smithi052.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-06-10T02:41:34.736 INFO:teuthology.orchestra.run.smithi052.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-06-10T02:41:34.736 INFO:teuthology.orchestra.run.smithi052.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-06-10T02:41:34.736 INFO:teuthology.orchestra.run.smithi052.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:febd:15de/64 scope link noprefixroute 2024-06-10T02:41:34.736 INFO:teuthology.orchestra.run.smithi052.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-06-10T02:41:34.736 INFO:teuthology.orchestra.run.smithi052.stdout:Mon IP `172.21.15.52` is in CIDR network `172.21.0.0/20` 2024-06-10T02:41:34.736 INFO:teuthology.orchestra.run.smithi052.stdout:Mon IP `172.21.15.52` is in CIDR network `172.21.0.0/20` 2024-06-10T02:41:34.737 INFO:teuthology.orchestra.run.smithi052.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-06-10T02:41:34.737 INFO:teuthology.orchestra.run.smithi052.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-06-10T02:41:34.738 INFO:teuthology.orchestra.run.smithi052.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba... 2024-06-10T02:41:35.320 INFO:teuthology.orchestra.run.smithi052.stdout:/bin/podman: stdout b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 2024-06-10T02:41:35.320 INFO:teuthology.orchestra.run.smithi052.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba... 2024-06-10T02:41:35.320 INFO:teuthology.orchestra.run.smithi052.stdout:/bin/podman: stderr Getting image source signatures 2024-06-10T02:41:35.321 INFO:teuthology.orchestra.run.smithi052.stdout:/bin/podman: stderr Copying blob sha256:53b2d4191269e9d20972bfa652565296fad6514cf6a3d99f19db644875f67876 2024-06-10T02:41:35.321 INFO:teuthology.orchestra.run.smithi052.stdout:/bin/podman: stderr Copying blob sha256:3ca969c7b6d79946c6f10490fffc00028babe7df67f50ed53c7bc209325e0612 2024-06-10T02:41:35.321 INFO:teuthology.orchestra.run.smithi052.stdout:/bin/podman: stderr Copying config sha256:b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 2024-06-10T02:41:35.321 INFO:teuthology.orchestra.run.smithi052.stdout:/bin/podman: stderr Writing manifest to image destination 2024-06-10T02:41:36.250 INFO:teuthology.orchestra.run.smithi052.stdout:ceph: stdout ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc) 2024-06-10T02:41:36.250 INFO:teuthology.orchestra.run.smithi052.stdout:Ceph version: ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc) 2024-06-10T02:41:36.250 INFO:teuthology.orchestra.run.smithi052.stdout:Extracting ceph user uid/gid from container image... 2024-06-10T02:41:37.105 INFO:teuthology.orchestra.run.smithi052.stdout:stat: stdout 167 167 2024-06-10T02:41:37.105 INFO:teuthology.orchestra.run.smithi052.stdout:Creating initial keys... 2024-06-10T02:41:38.003 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph-authtool: stdout AQDhZ2ZmF6AuGRAAOw2n79IMzRpyA/4aglzlxQ== 2024-06-10T02:41:38.875 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph-authtool: stdout AQDiZ2ZmhKUpFBAAmrJPAdb98z0KLgIYNIbsGQ== 2024-06-10T02:41:39.755 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph-authtool: stdout AQDjZ2ZmnbziCxAAEp+1dvtUbUTPBogNYcPmuw== 2024-06-10T02:41:39.756 INFO:teuthology.orchestra.run.smithi052.stdout:Creating initial monmap... 2024-06-10T02:41:40.595 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-06-10T02:41:40.595 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-06-10T02:41:40.595 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to d8676e9e-26d2-11ef-bc9b-c7b262605968 2024-06-10T02:41:40.595 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-06-10T02:41:40.595 INFO:teuthology.orchestra.run.smithi052.stdout:monmaptool for a [v2:172.21.15.52:3300,v1:172.21.15.52:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-06-10T02:41:40.595 INFO:teuthology.orchestra.run.smithi052.stdout:setting min_mon_release = quincy 2024-06-10T02:41:40.595 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/monmaptool: set fsid to d8676e9e-26d2-11ef-bc9b-c7b262605968 2024-06-10T02:41:40.595 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-06-10T02:41:40.595 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-10T02:41:40.595 INFO:teuthology.orchestra.run.smithi052.stdout:Creating mon... 2024-06-10T02:41:41.433 INFO:teuthology.orchestra.run.smithi052.stdout:create mon.a on 2024-06-10T02:41:41.655 INFO:teuthology.orchestra.run.smithi052.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-06-10T02:41:41.805 INFO:teuthology.orchestra.run.smithi052.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-06-10T02:41:42.106 INFO:teuthology.orchestra.run.smithi052.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968.target → /etc/systemd/system/ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968.target. 2024-06-10T02:41:42.106 INFO:teuthology.orchestra.run.smithi052.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968.target → /etc/systemd/system/ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968.target. 2024-06-10T02:41:42.344 INFO:teuthology.orchestra.run.smithi052.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968@mon.a 2024-06-10T02:41:42.344 INFO:teuthology.orchestra.run.smithi052.stdout:systemctl: stderr Failed to reset failed state of unit ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968@mon.a.service: Unit ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968@mon.a.service not loaded. 2024-06-10T02:41:42.516 INFO:teuthology.orchestra.run.smithi052.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968.target.wants/ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968@mon.a.service → /etc/systemd/system/ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968@.service. 2024-06-10T02:41:43.116 INFO:teuthology.orchestra.run.smithi052.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-06-10T02:41:43.116 INFO:teuthology.orchestra.run.smithi052.stdout:systemctl: stdout disabled 2024-06-10T02:41:43.123 INFO:teuthology.orchestra.run.smithi052.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-06-10T02:41:43.123 INFO:teuthology.orchestra.run.smithi052.stdout:systemctl: stdout inactive 2024-06-10T02:41:43.124 INFO:teuthology.orchestra.run.smithi052.stdout:firewalld.service is not enabled 2024-06-10T02:41:43.124 INFO:teuthology.orchestra.run.smithi052.stdout:Not possible to enable service . firewalld.service is not available 2024-06-10T02:41:43.124 INFO:teuthology.orchestra.run.smithi052.stdout:Waiting for mon to start... 2024-06-10T02:41:43.124 INFO:teuthology.orchestra.run.smithi052.stdout:Waiting for mon... 2024-06-10T02:41:44.175 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout cluster: 2024-06-10T02:41:44.175 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout id: d8676e9e-26d2-11ef-bc9b-c7b262605968 2024-06-10T02:41:44.175 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-06-10T02:41:44.175 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout 2024-06-10T02:41:44.175 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout services: 2024-06-10T02:41:44.176 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.558249s) 2024-06-10T02:41:44.176 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-06-10T02:41:44.176 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-06-10T02:41:44.176 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout 2024-06-10T02:41:44.176 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout data: 2024-06-10T02:41:44.176 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-06-10T02:41:44.176 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-06-10T02:41:44.176 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-06-10T02:41:44.176 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout pgs: 2024-06-10T02:41:44.176 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout 2024-06-10T02:41:44.176 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:43.553+0000 7f46877c9700 1 Processor -- start 2024-06-10T02:41:44.176 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:43.553+0000 7f46877c9700 1 -- start start 2024-06-10T02:41:44.176 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:43.553+0000 7f46877c9700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4680106600 0x7f4680106a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:41:44.177 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:43.553+0000 7f46877c9700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f4680106ff0 con 0x7f4680106600 2024-06-10T02:41:44.177 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:43.553+0000 7f4685565700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4680106600 0x7f4680106a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:41:44.177 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:43.553+0000 7f4685565700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4680106600 0x7f4680106a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54452/0 (socket says 172.21.15.52:54452) 2024-06-10T02:41:44.177 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:43.553+0000 7f4685565700 1 -- 172.21.15.52:0/3776783675 learned_addr learned my addr 172.21.15.52:0/3776783675 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:41:44.177 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:43.554+0000 7f4685565700 1 -- 172.21.15.52:0/3776783675 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4680107800 con 0x7f4680106600 2024-06-10T02:41:44.177 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:43.554+0000 7f4685565700 1 --2- 172.21.15.52:0/3776783675 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4680106600 0x7f4680106a20 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f467c0098a0 tx=0x7f467c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cfa643401a8f900 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:41:44.177 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:43.554+0000 7f4677fff700 1 -- 172.21.15.52:0/3776783675 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f467c004160 con 0x7f4680106600 2024-06-10T02:41:44.177 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:43.554+0000 7f4677fff700 1 -- 172.21.15.52:0/3776783675 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f467c004300 con 0x7f4680106600 2024-06-10T02:41:44.177 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:43.554+0000 7f4677fff700 1 -- 172.21.15.52:0/3776783675 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f467c0036a0 con 0x7f4680106600 2024-06-10T02:41:44.177 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:43.555+0000 7f46877c9700 1 -- 172.21.15.52:0/3776783675 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4680106600 msgr2=0x7f4680106a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:41:44.177 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:43.555+0000 7f46877c9700 1 --2- 172.21.15.52:0/3776783675 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4680106600 0x7f4680106a20 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f467c0098a0 tx=0x7f467c009bb0 comp rx=0 tx=0).stop 2024-06-10T02:41:44.177 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:43.555+0000 7f46877c9700 1 -- 172.21.15.52:0/3776783675 shutdown_connections 2024-06-10T02:41:44.178 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:43.555+0000 7f46877c9700 1 --2- 172.21.15.52:0/3776783675 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4680106600 0x7f4680106a20 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:41:44.178 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:43.555+0000 7f46877c9700 1 -- 172.21.15.52:0/3776783675 >> 172.21.15.52:0/3776783675 conn(0x7f4680101ba0 msgr2=0x7f4680103fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:41:44.178 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:43.555+0000 7f46877c9700 1 -- 172.21.15.52:0/3776783675 shutdown_connections 2024-06-10T02:41:44.178 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:43.555+0000 7f46877c9700 1 -- 172.21.15.52:0/3776783675 wait complete. 2024-06-10T02:41:44.178 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:43.555+0000 7f46877c9700 1 Processor -- start 2024-06-10T02:41:44.178 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:43.555+0000 7f46877c9700 1 -- start start 2024-06-10T02:41:44.178 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:43.556+0000 7f46877c9700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4680106600 0x7f468019b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:41:44.178 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:43.556+0000 7f46877c9700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f4680107d30 con 0x7f4680106600 2024-06-10T02:41:44.178 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:43.556+0000 7f4685565700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4680106600 0x7f468019b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:41:44.178 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:43.556+0000 7f4685565700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4680106600 0x7f468019b670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54458/0 (socket says 172.21.15.52:54458) 2024-06-10T02:41:44.178 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:43.556+0000 7f4685565700 1 -- 172.21.15.52:0/3655148187 learned_addr learned my addr 172.21.15.52:0/3655148187 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:41:44.178 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:43.556+0000 7f4685565700 1 -- 172.21.15.52:0/3655148187 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f468019bbb0 con 0x7f4680106600 2024-06-10T02:41:44.178 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:43.556+0000 7f4685565700 1 --2- 172.21.15.52:0/3655148187 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4680106600 0x7f468019b670 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f467c000c00 tx=0x7f467c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:41:44.179 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:43.556+0000 7f46767fc700 1 -- 172.21.15.52:0/3655148187 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f467c003db0 con 0x7f4680106600 2024-06-10T02:41:44.179 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:43.556+0000 7f46767fc700 1 -- 172.21.15.52:0/3655148187 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f467c011630 con 0x7f4680106600 2024-06-10T02:41:44.179 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:43.556+0000 7f46767fc700 1 -- 172.21.15.52:0/3655148187 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f467c011950 con 0x7f4680106600 2024-06-10T02:41:44.179 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:43.556+0000 7f46877c9700 1 -- 172.21.15.52:0/3655148187 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f468019be40 con 0x7f4680106600 2024-06-10T02:41:44.179 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:43.556+0000 7f46877c9700 1 -- 172.21.15.52:0/3655148187 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f468019eb30 con 0x7f4680106600 2024-06-10T02:41:44.179 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:43.557+0000 7f46767fc700 1 -- 172.21.15.52:0/3655148187 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f467c011af0 con 0x7f4680106600 2024-06-10T02:41:44.179 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:43.557+0000 7f46767fc700 1 -- 172.21.15.52:0/3655148187 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f467c01c450 con 0x7f4680106600 2024-06-10T02:41:44.179 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:43.557+0000 7f46877c9700 1 -- 172.21.15.52:0/3655148187 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4680063af0 con 0x7f4680106600 2024-06-10T02:41:44.179 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:43.560+0000 7f46767fc700 1 -- 172.21.15.52:0/3655148187 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f467c032020 con 0x7f4680106600 2024-06-10T02:41:44.179 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:43.633+0000 7f46877c9700 1 -- 172.21.15.52:0/3655148187 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f468002de20 con 0x7f4680106600 2024-06-10T02:41:44.179 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:43.633+0000 7f46767fc700 1 -- 172.21.15.52:0/3655148187 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7f467c004260 con 0x7f4680106600 2024-06-10T02:41:44.179 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:43.634+0000 7f46877c9700 1 -- 172.21.15.52:0/3655148187 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4680106600 msgr2=0x7f468019b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:41:44.179 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:43.634+0000 7f46877c9700 1 --2- 172.21.15.52:0/3655148187 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4680106600 0x7f468019b670 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f467c000c00 tx=0x7f467c004750 comp rx=0 tx=0).stop 2024-06-10T02:41:44.180 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:43.634+0000 7f46877c9700 1 -- 172.21.15.52:0/3655148187 shutdown_connections 2024-06-10T02:41:44.180 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:43.634+0000 7f46877c9700 1 --2- 172.21.15.52:0/3655148187 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4680106600 0x7f468019b670 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:41:44.180 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:43.634+0000 7f46877c9700 1 -- 172.21.15.52:0/3655148187 >> 172.21.15.52:0/3655148187 conn(0x7f4680101ba0 msgr2=0x7f468018d990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:41:44.180 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:43.634+0000 7f46877c9700 1 -- 172.21.15.52:0/3655148187 shutdown_connections 2024-06-10T02:41:44.180 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:43.634+0000 7f46877c9700 1 -- 172.21.15.52:0/3655148187 wait complete. 2024-06-10T02:41:44.180 INFO:teuthology.orchestra.run.smithi052.stdout:mon is available 2024-06-10T02:41:44.180 INFO:teuthology.orchestra.run.smithi052.stdout:Assimilating anything we can from ceph.conf... 2024-06-10T02:41:45.222 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:45 smithi052 ceph-mon[31990]: from='client.? 172.21.15.52:0/1479762352' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-06-10T02:41:45.264 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout 2024-06-10T02:41:45.265 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout [global] 2024-06-10T02:41:45.265 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout fsid = d8676e9e-26d2-11ef-bc9b-c7b262605968 2024-06-10T02:41:45.265 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.52:3300,v1:172.21.15.52:6789] 2024-06-10T02:41:45.265 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-06-10T02:41:45.265 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-06-10T02:41:45.265 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-06-10T02:41:45.265 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-06-10T02:41:45.265 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout 2024-06-10T02:41:45.265 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout [mgr] 2024-06-10T02:41:45.265 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-06-10T02:41:45.265 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout 2024-06-10T02:41:45.266 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout [osd] 2024-06-10T02:41:45.266 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-06-10T02:41:45.266 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-06-10T02:41:45.266 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-06-10T02:41:45.266 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-06-10T02:41:45.266 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-06-10T02:41:45.266 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:44.630+0000 7f8c11ad0700 1 Processor -- start 2024-06-10T02:41:45.266 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:44.630+0000 7f8c11ad0700 1 -- start start 2024-06-10T02:41:45.266 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:44.630+0000 7f8c11ad0700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8c0c107fc0 0x7f8c0c1083e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:41:45.266 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:44.630+0000 7f8c11ad0700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f8c0c1089b0 con 0x7f8c0c107fc0 2024-06-10T02:41:45.266 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:44.631+0000 7f8c0b7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8c0c107fc0 0x7f8c0c1083e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:41:45.266 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:44.631+0000 7f8c0b7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8c0c107fc0 0x7f8c0c1083e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54462/0 (socket says 172.21.15.52:54462) 2024-06-10T02:41:45.267 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:44.631+0000 7f8c0b7fe700 1 -- 172.21.15.52:0/847179311 learned_addr learned my addr 172.21.15.52:0/847179311 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:41:45.267 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:44.631+0000 7f8c0b7fe700 1 -- 172.21.15.52:0/847179311 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c0c109170 con 0x7f8c0c107fc0 2024-06-10T02:41:45.267 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:44.631+0000 7f8c0b7fe700 1 --2- 172.21.15.52:0/847179311 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8c0c107fc0 0x7f8c0c1083e0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f8bfc009e80 tx=0x7f8bfc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95f2396bcb694292 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:41:45.267 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:44.631+0000 7f8c0a7fc700 1 -- 172.21.15.52:0/847179311 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8bfc017070 con 0x7f8c0c107fc0 2024-06-10T02:41:45.267 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:44.631+0000 7f8c0a7fc700 1 -- 172.21.15.52:0/847179311 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8bfc003a20 con 0x7f8c0c107fc0 2024-06-10T02:41:45.267 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:44.631+0000 7f8c0a7fc700 1 -- 172.21.15.52:0/847179311 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8bfc003d20 con 0x7f8c0c107fc0 2024-06-10T02:41:45.267 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:44.632+0000 7f8c11ad0700 1 -- 172.21.15.52:0/847179311 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8c0c107fc0 msgr2=0x7f8c0c1083e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:41:45.267 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:44.632+0000 7f8c11ad0700 1 --2- 172.21.15.52:0/847179311 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8c0c107fc0 0x7f8c0c1083e0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f8bfc009e80 tx=0x7f8bfc00b330 comp rx=0 tx=0).stop 2024-06-10T02:41:45.267 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:44.632+0000 7f8c11ad0700 1 -- 172.21.15.52:0/847179311 shutdown_connections 2024-06-10T02:41:45.267 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:44.632+0000 7f8c11ad0700 1 --2- 172.21.15.52:0/847179311 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8c0c107fc0 0x7f8c0c1083e0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:41:45.267 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:44.632+0000 7f8c11ad0700 1 -- 172.21.15.52:0/847179311 >> 172.21.15.52:0/847179311 conn(0x7f8c0c103560 msgr2=0x7f8c0c1059a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:41:45.267 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:44.632+0000 7f8c11ad0700 1 -- 172.21.15.52:0/847179311 shutdown_connections 2024-06-10T02:41:45.267 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:44.632+0000 7f8c11ad0700 1 -- 172.21.15.52:0/847179311 wait complete. 2024-06-10T02:41:45.267 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:44.632+0000 7f8c11ad0700 1 Processor -- start 2024-06-10T02:41:45.268 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:44.633+0000 7f8c11ad0700 1 -- start start 2024-06-10T02:41:45.268 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:44.633+0000 7f8c11ad0700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8c0c107fc0 0x7f8c0c07e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:41:45.268 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:44.633+0000 7f8c11ad0700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f8c0c1096a0 con 0x7f8c0c107fc0 2024-06-10T02:41:45.268 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:44.633+0000 7f8c0b7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8c0c107fc0 0x7f8c0c07e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:41:45.268 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:44.633+0000 7f8c0b7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8c0c107fc0 0x7f8c0c07e410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54474/0 (socket says 172.21.15.52:54474) 2024-06-10T02:41:45.268 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:44.633+0000 7f8c0b7fe700 1 -- 172.21.15.52:0/1479762352 learned_addr learned my addr 172.21.15.52:0/1479762352 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:41:45.268 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:44.633+0000 7f8c0b7fe700 1 -- 172.21.15.52:0/1479762352 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c0c07e950 con 0x7f8c0c107fc0 2024-06-10T02:41:45.268 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:44.633+0000 7f8c0b7fe700 1 --2- 172.21.15.52:0/1479762352 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8c0c107fc0 0x7f8c0c07e410 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f8bfc004000 tx=0x7f8bfc004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:41:45.268 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:44.634+0000 7f8c08ff9700 1 -- 172.21.15.52:0/1479762352 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8bfc022020 con 0x7f8c0c107fc0 2024-06-10T02:41:45.268 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:44.634+0000 7f8c08ff9700 1 -- 172.21.15.52:0/1479762352 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8bfc018040 con 0x7f8c0c107fc0 2024-06-10T02:41:45.268 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:44.634+0000 7f8c08ff9700 1 -- 172.21.15.52:0/1479762352 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8bfc017070 con 0x7f8c0c107fc0 2024-06-10T02:41:45.268 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:44.634+0000 7f8c11ad0700 1 -- 172.21.15.52:0/1479762352 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c0c07cb40 con 0x7f8c0c107fc0 2024-06-10T02:41:45.268 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:44.634+0000 7f8c11ad0700 1 -- 172.21.15.52:0/1479762352 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c0c07d080 con 0x7f8c0c107fc0 2024-06-10T02:41:45.268 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:44.634+0000 7f8c08ff9700 1 -- 172.21.15.52:0/1479762352 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f8bfc025020 con 0x7f8c0c107fc0 2024-06-10T02:41:45.269 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:44.634+0000 7f8c08ff9700 1 -- 172.21.15.52:0/1479762352 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f8bfc00f050 con 0x7f8c0c107fc0 2024-06-10T02:41:45.269 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:44.635+0000 7f8c11ad0700 1 -- 172.21.15.52:0/1479762352 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c0c063af0 con 0x7f8c0c107fc0 2024-06-10T02:41:45.269 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:44.637+0000 7f8c08ff9700 1 -- 172.21.15.52:0/1479762352 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f8bfc02c030 con 0x7f8c0c107fc0 2024-06-10T02:41:45.269 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:44.709+0000 7f8c11ad0700 1 -- 172.21.15.52:0/1479762352 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f8c0c194f70 con 0x7f8c0c107fc0 2024-06-10T02:41:45.269 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:44.712+0000 7f8c08ff9700 1 -- 172.21.15.52:0/1479762352 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+429 (secure 0 0 0) 0x7f8bfc01e480 con 0x7f8c0c107fc0 2024-06-10T02:41:45.269 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:44.712+0000 7f8c08ff9700 1 -- 172.21.15.52:0/1479762352 <== mon.0 v2:172.21.15.52:3300/0 8 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8bfc035070 con 0x7f8c0c107fc0 2024-06-10T02:41:45.269 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:44.713+0000 7f8c11ad0700 1 -- 172.21.15.52:0/1479762352 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8c0c107fc0 msgr2=0x7f8c0c07e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:41:45.269 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:44.713+0000 7f8c11ad0700 1 --2- 172.21.15.52:0/1479762352 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8c0c107fc0 0x7f8c0c07e410 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f8bfc004000 tx=0x7f8bfc004ad0 comp rx=0 tx=0).stop 2024-06-10T02:41:45.269 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:44.713+0000 7f8c11ad0700 1 -- 172.21.15.52:0/1479762352 shutdown_connections 2024-06-10T02:41:45.269 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:44.713+0000 7f8c11ad0700 1 --2- 172.21.15.52:0/1479762352 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8c0c107fc0 0x7f8c0c07e410 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:41:45.269 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:44.713+0000 7f8c11ad0700 1 -- 172.21.15.52:0/1479762352 >> 172.21.15.52:0/1479762352 conn(0x7f8c0c103560 msgr2=0x7f8c0c192370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:41:45.269 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:44.713+0000 7f8c11ad0700 1 -- 172.21.15.52:0/1479762352 shutdown_connections 2024-06-10T02:41:45.269 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:44.713+0000 7f8c11ad0700 1 -- 172.21.15.52:0/1479762352 wait complete. 2024-06-10T02:41:45.270 INFO:teuthology.orchestra.run.smithi052.stdout:Generating new minimal ceph.conf... 2024-06-10T02:41:46.304 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:45.704+0000 7fe53db42700 1 Processor -- start 2024-06-10T02:41:46.305 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:45.704+0000 7fe53db42700 1 -- start start 2024-06-10T02:41:46.305 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:45.704+0000 7fe53db42700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe538107fc0 0x7fe5381083e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:41:46.305 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:45.704+0000 7fe53db42700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fe5381089b0 con 0x7fe538107fc0 2024-06-10T02:41:46.305 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:45.704+0000 7fe5377fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe538107fc0 0x7fe5381083e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:41:46.305 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:45.704+0000 7fe5377fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe538107fc0 0x7fe5381083e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54482/0 (socket says 172.21.15.52:54482) 2024-06-10T02:41:46.305 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:45.704+0000 7fe5377fe700 1 -- 172.21.15.52:0/2923027148 learned_addr learned my addr 172.21.15.52:0/2923027148 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:41:46.305 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:45.705+0000 7fe5377fe700 1 -- 172.21.15.52:0/2923027148 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe538109170 con 0x7fe538107fc0 2024-06-10T02:41:46.305 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:45.705+0000 7fe5377fe700 1 --2- 172.21.15.52:0/2923027148 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe538107fc0 0x7fe5381083e0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fe5280098a0 tx=0x7fe528009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=434b96bb21fc8eae server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:41:46.305 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:45.705+0000 7fe5367fc700 1 -- 172.21.15.52:0/2923027148 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe528004160 con 0x7fe538107fc0 2024-06-10T02:41:46.305 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:45.705+0000 7fe5367fc700 1 -- 172.21.15.52:0/2923027148 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fe528004300 con 0x7fe538107fc0 2024-06-10T02:41:46.305 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:45.705+0000 7fe5367fc700 1 -- 172.21.15.52:0/2923027148 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe528011b50 con 0x7fe538107fc0 2024-06-10T02:41:46.306 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:45.706+0000 7fe53db42700 1 -- 172.21.15.52:0/2923027148 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe538107fc0 msgr2=0x7fe5381083e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:41:46.306 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:45.706+0000 7fe53db42700 1 --2- 172.21.15.52:0/2923027148 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe538107fc0 0x7fe5381083e0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fe5280098a0 tx=0x7fe528009bb0 comp rx=0 tx=0).stop 2024-06-10T02:41:46.306 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:45.706+0000 7fe53db42700 1 -- 172.21.15.52:0/2923027148 shutdown_connections 2024-06-10T02:41:46.306 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:45.706+0000 7fe53db42700 1 --2- 172.21.15.52:0/2923027148 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe538107fc0 0x7fe5381083e0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:41:46.306 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:45.706+0000 7fe53db42700 1 -- 172.21.15.52:0/2923027148 >> 172.21.15.52:0/2923027148 conn(0x7fe538103560 msgr2=0x7fe5381059a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:41:46.306 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:45.706+0000 7fe53db42700 1 -- 172.21.15.52:0/2923027148 shutdown_connections 2024-06-10T02:41:46.306 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:45.706+0000 7fe53db42700 1 -- 172.21.15.52:0/2923027148 wait complete. 2024-06-10T02:41:46.306 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:45.706+0000 7fe53db42700 1 Processor -- start 2024-06-10T02:41:46.306 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:45.707+0000 7fe53db42700 1 -- start start 2024-06-10T02:41:46.306 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:45.707+0000 7fe53db42700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe538107fc0 0x7fe53819bf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:41:46.306 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:45.707+0000 7fe53db42700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fe53810c520 con 0x7fe538107fc0 2024-06-10T02:41:46.306 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:45.707+0000 7fe5377fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe538107fc0 0x7fe53819bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:41:46.307 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:45.707+0000 7fe5377fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe538107fc0 0x7fe53819bf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54490/0 (socket says 172.21.15.52:54490) 2024-06-10T02:41:46.307 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:45.707+0000 7fe5377fe700 1 -- 172.21.15.52:0/1894306523 learned_addr learned my addr 172.21.15.52:0/1894306523 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:41:46.307 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:45.707+0000 7fe5377fe700 1 -- 172.21.15.52:0/1894306523 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe53819c460 con 0x7fe538107fc0 2024-06-10T02:41:46.307 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:45.707+0000 7fe5377fe700 1 --2- 172.21.15.52:0/1894306523 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe538107fc0 0x7fe53819bf20 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fe528000c00 tx=0x7fe528011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:41:46.307 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:45.708+0000 7fe534ff9700 1 -- 172.21.15.52:0/1894306523 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe528003b20 con 0x7fe538107fc0 2024-06-10T02:41:46.307 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:45.708+0000 7fe534ff9700 1 -- 172.21.15.52:0/1894306523 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fe528003cc0 con 0x7fe538107fc0 2024-06-10T02:41:46.307 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:45.708+0000 7fe53db42700 1 -- 172.21.15.52:0/1894306523 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe53819c6f0 con 0x7fe538107fc0 2024-06-10T02:41:46.307 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:45.708+0000 7fe534ff9700 1 -- 172.21.15.52:0/1894306523 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe528024910 con 0x7fe538107fc0 2024-06-10T02:41:46.307 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:45.708+0000 7fe53db42700 1 -- 172.21.15.52:0/1894306523 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe53819f3e0 con 0x7fe538107fc0 2024-06-10T02:41:46.307 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:45.708+0000 7fe534ff9700 1 -- 172.21.15.52:0/1894306523 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fe528022020 con 0x7fe538107fc0 2024-06-10T02:41:46.307 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:45.708+0000 7fe534ff9700 1 -- 172.21.15.52:0/1894306523 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fe528011e00 con 0x7fe538107fc0 2024-06-10T02:41:46.307 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:45.709+0000 7fe53db42700 1 -- 172.21.15.52:0/1894306523 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe538063af0 con 0x7fe538107fc0 2024-06-10T02:41:46.307 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:45.711+0000 7fe534ff9700 1 -- 172.21.15.52:0/1894306523 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fe528042dd0 con 0x7fe538107fc0 2024-06-10T02:41:46.308 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:45.782+0000 7fe53db42700 1 -- 172.21.15.52:0/1894306523 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7fe53819f5b0 con 0x7fe538107fc0 2024-06-10T02:41:46.308 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:45.782+0000 7fe534ff9700 1 -- 172.21.15.52:0/1894306523 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+175 (secure 0 0 0) 0x7fe528031020 con 0x7fe538107fc0 2024-06-10T02:41:46.308 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:45.783+0000 7fe53db42700 1 -- 172.21.15.52:0/1894306523 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe538107fc0 msgr2=0x7fe53819bf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:41:46.308 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:45.783+0000 7fe53db42700 1 --2- 172.21.15.52:0/1894306523 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe538107fc0 0x7fe53819bf20 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fe528000c00 tx=0x7fe528011c10 comp rx=0 tx=0).stop 2024-06-10T02:41:46.308 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:45.783+0000 7fe53db42700 1 -- 172.21.15.52:0/1894306523 shutdown_connections 2024-06-10T02:41:46.308 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:45.783+0000 7fe53db42700 1 --2- 172.21.15.52:0/1894306523 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe538107fc0 0x7fe53819bf20 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:41:46.308 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:45.783+0000 7fe53db42700 1 -- 172.21.15.52:0/1894306523 >> 172.21.15.52:0/1894306523 conn(0x7fe538103560 msgr2=0x7fe538104030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:41:46.308 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:45.783+0000 7fe53db42700 1 -- 172.21.15.52:0/1894306523 shutdown_connections 2024-06-10T02:41:46.308 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:45.783+0000 7fe53db42700 1 -- 172.21.15.52:0/1894306523 wait complete. 2024-06-10T02:41:46.308 INFO:teuthology.orchestra.run.smithi052.stdout:Restarting the monitor... 2024-06-10T02:41:46.317 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:46 smithi052 ceph-mon[31990]: from='client.? 172.21.15.52:0/1894306523' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:41:46.583 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:46 smithi052 systemd[1]: Stopping Ceph mon.a for d8676e9e-26d2-11ef-bc9b-c7b262605968... 2024-06-10T02:41:46.972 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:46 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mon-a[31985]: 2024-06-10T02:41:46.586+0000 7fdabb3cd700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-06-10T02:41:46.972 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:46 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mon-a[31985]: 2024-06-10T02:41:46.587+0000 7fdabb3cd700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-06-10T02:41:46.973 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:46 smithi052 podman[32236]: 2024-06-10 02:41:46.77221343 +0000 UTC m=+0.292183693 container died 237686c5efe41e7d0b65a628c24911860f63459f796b25436de3af59084fef5a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mon-a, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, org.label-schema.build-date=20240528) 2024-06-10T02:41:47.275 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 podman[32236]: 2024-06-10 02:41:47.103472208 +0000 UTC m=+0.623442454 container cleanup 237686c5efe41e7d0b65a628c24911860f63459f796b25436de3af59084fef5a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mon-a, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-06-10T02:41:47.275 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 bash[32236]: ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mon-a 2024-06-10T02:41:47.700 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 podman[32249]: 2024-06-10 02:41:47.280572474 +0000 UTC m=+0.508711585 container remove 237686c5efe41e7d0b65a628c24911860f63459f796b25436de3af59084fef5a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mon-a, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-10T02:41:47.700 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 systemd[1]: ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968@mon.a.service: Deactivated successfully. 2024-06-10T02:41:47.700 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 systemd[1]: Stopped Ceph mon.a for d8676e9e-26d2-11ef-bc9b-c7b262605968. 2024-06-10T02:41:47.700 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 systemd[1]: Starting Ceph mon.a for d8676e9e-26d2-11ef-bc9b-c7b262605968... 2024-06-10T02:41:47.952 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 podman[32343]: 2024-06-10 02:41:47.705419788 +0000 UTC m=+0.088779848 container create cb4e54e38ee793a7d4a80b5b4ec7bed9005451170e7a0aaaaa917e8fddd8cc68 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mon-a, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-10T02:41:47.953 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 podman[32343]: 2024-06-10 02:41:47.634439162 +0000 UTC m=+0.017799242 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-10T02:41:47.953 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 podman[32343]: 2024-06-10 02:41:47.890868685 +0000 UTC m=+0.274228746 container init cb4e54e38ee793a7d4a80b5b4ec7bed9005451170e7a0aaaaa917e8fddd8cc68 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mon-a, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-06-10T02:41:47.953 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 podman[32343]: 2024-06-10 02:41:47.893916023 +0000 UTC m=+0.277276084 container start cb4e54e38ee793a7d4a80b5b4ec7bed9005451170e7a0aaaaa917e8fddd8cc68 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mon-a, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-10T02:41:47.953 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: set uid:gid to 167:167 (ceph:ceph) 2024-06-10T02:41:47.953 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc), process ceph-mon, pid 2 2024-06-10T02:41:47.953 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: pidfile_write: ignore empty --pid-file 2024-06-10T02:41:47.953 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: load: jerasure load: lrc 2024-06-10T02:41:47.953 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: RocksDB version: 7.9.2 2024-06-10T02:41:47.953 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: Git sha 0 2024-06-10T02:41:47.953 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: Compile date 2024-05-30 15:42:47 2024-06-10T02:41:47.953 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: DB SUMMARY 2024-06-10T02:41:47.954 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: DB Session ID: 94MZS7RWYMM4T2UUHR3W 2024-06-10T02:41:47.954 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: CURRENT file: CURRENT 2024-06-10T02:41:47.954 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: IDENTITY file: IDENTITY 2024-06-10T02:41:47.954 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-06-10T02:41:47.954 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-06-10T02:41:47.954 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 105452 ; 2024-06-10T02:41:47.954 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: Options.error_if_exists: 0 2024-06-10T02:41:47.955 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: Options.create_if_missing: 0 2024-06-10T02:41:47.955 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: Options.paranoid_checks: 1 2024-06-10T02:41:47.955 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: Options.flush_verify_memtable_count: 1 2024-06-10T02:41:47.955 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-06-10T02:41:47.955 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-06-10T02:41:47.955 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: Options.env: 0x557c2e5a8c60 2024-06-10T02:41:47.955 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: Options.fs: PosixFileSystem 2024-06-10T02:41:47.955 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: Options.info_log: 0x557c2ef55b20 2024-06-10T02:41:47.956 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: Options.max_file_opening_threads: 16 2024-06-10T02:41:47.956 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: Options.statistics: (nil) 2024-06-10T02:41:47.956 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: Options.use_fsync: 0 2024-06-10T02:41:47.956 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: Options.max_log_file_size: 0 2024-06-10T02:41:47.956 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-06-10T02:41:47.956 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: Options.log_file_time_to_roll: 0 2024-06-10T02:41:47.956 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: Options.keep_log_file_num: 1000 2024-06-10T02:41:47.956 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: Options.recycle_log_file_num: 0 2024-06-10T02:41:47.956 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: Options.allow_fallocate: 1 2024-06-10T02:41:47.956 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: Options.allow_mmap_reads: 0 2024-06-10T02:41:47.956 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: Options.allow_mmap_writes: 0 2024-06-10T02:41:47.957 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: Options.use_direct_reads: 0 2024-06-10T02:41:47.957 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-06-10T02:41:47.957 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: Options.create_missing_column_families: 0 2024-06-10T02:41:47.957 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: Options.db_log_dir: 2024-06-10T02:41:47.957 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: Options.wal_dir: 2024-06-10T02:41:47.957 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: Options.table_cache_numshardbits: 6 2024-06-10T02:41:47.957 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: Options.WAL_ttl_seconds: 0 2024-06-10T02:41:47.957 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: Options.WAL_size_limit_MB: 0 2024-06-10T02:41:47.957 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-06-10T02:41:47.957 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-06-10T02:41:47.957 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: Options.is_fd_close_on_exec: 1 2024-06-10T02:41:47.957 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: Options.advise_random_on_open: 1 2024-06-10T02:41:47.957 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: Options.db_write_buffer_size: 0 2024-06-10T02:41:47.958 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: Options.write_buffer_manager: 0x557c2ef261e0 2024-06-10T02:41:47.958 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-06-10T02:41:47.958 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-06-10T02:41:47.958 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: Options.use_adaptive_mutex: 0 2024-06-10T02:41:47.958 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: Options.rate_limiter: (nil) 2024-06-10T02:41:47.958 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-06-10T02:41:47.958 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: Options.wal_recovery_mode: 2 2024-06-10T02:41:47.958 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: Options.enable_thread_tracking: 0 2024-06-10T02:41:47.958 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: Options.enable_pipelined_write: 0 2024-06-10T02:41:47.958 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: Options.unordered_write: 0 2024-06-10T02:41:47.958 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-06-10T02:41:47.958 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-06-10T02:41:47.959 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-06-10T02:41:47.959 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-06-10T02:41:47.959 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: Options.row_cache: None 2024-06-10T02:41:47.959 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: Options.wal_filter: None 2024-06-10T02:41:47.959 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-06-10T02:41:47.959 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: Options.allow_ingest_behind: 0 2024-06-10T02:41:47.959 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: Options.two_write_queues: 0 2024-06-10T02:41:47.959 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: Options.manual_wal_flush: 0 2024-06-10T02:41:47.959 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: Options.wal_compression: 0 2024-06-10T02:41:47.959 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: Options.atomic_flush: 0 2024-06-10T02:41:47.959 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-06-10T02:41:47.959 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: Options.persist_stats_to_disk: 0 2024-06-10T02:41:47.959 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: Options.write_dbid_to_manifest: 0 2024-06-10T02:41:47.959 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: Options.log_readahead_size: 0 2024-06-10T02:41:47.960 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-06-10T02:41:47.960 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: Options.best_efforts_recovery: 0 2024-06-10T02:41:47.960 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-06-10T02:41:47.960 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-06-10T02:41:47.960 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: Options.allow_data_in_errors: 0 2024-06-10T02:41:47.960 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: Options.db_host_id: __hostname__ 2024-06-10T02:41:47.960 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: Options.enforce_single_del_contracts: true 2024-06-10T02:41:47.960 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: Options.max_background_jobs: 2 2024-06-10T02:41:47.960 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: Options.max_background_compactions: -1 2024-06-10T02:41:47.960 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: Options.max_subcompactions: 1 2024-06-10T02:41:47.960 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-06-10T02:41:47.960 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-06-10T02:41:47.960 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: Options.delayed_write_rate : 16777216 2024-06-10T02:41:47.960 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: Options.max_total_wal_size: 0 2024-06-10T02:41:47.961 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-06-10T02:41:47.961 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: Options.stats_dump_period_sec: 600 2024-06-10T02:41:47.961 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: Options.stats_persist_period_sec: 600 2024-06-10T02:41:47.961 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-06-10T02:41:47.961 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: Options.max_open_files: -1 2024-06-10T02:41:47.961 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: Options.bytes_per_sync: 0 2024-06-10T02:41:47.961 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: Options.wal_bytes_per_sync: 0 2024-06-10T02:41:47.961 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: Options.strict_bytes_per_sync: 0 2024-06-10T02:41:47.961 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: Options.compaction_readahead_size: 0 2024-06-10T02:41:47.961 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: Options.max_background_flushes: -1 2024-06-10T02:41:47.961 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: Compression algorithms supported: 2024-06-10T02:41:47.961 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-06-10T02:41:47.961 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: kZSTD supported: 0 2024-06-10T02:41:47.962 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: kXpressCompression supported: 0 2024-06-10T02:41:47.962 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: kLZ4HCCompression supported: 1 2024-06-10T02:41:47.962 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: kZlibCompression supported: 1 2024-06-10T02:41:47.962 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: kSnappyCompression supported: 1 2024-06-10T02:41:47.962 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: kLZ4Compression supported: 1 2024-06-10T02:41:47.962 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: kBZip2Compression supported: 0 2024-06-10T02:41:47.962 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: Fast CRC32 supported: Supported on x86 2024-06-10T02:41:47.962 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: DMutex implementation: pthread_mutex_t 2024-06-10T02:41:47.962 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-06-10T02:41:47.962 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-06-10T02:41:47.962 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-06-10T02:41:47.962 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: Options.merge_operator: 2024-06-10T02:41:47.962 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: Options.compaction_filter: None 2024-06-10T02:41:47.963 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: Options.compaction_filter_factory: None 2024-06-10T02:41:47.963 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: Options.sst_partitioner_factory: None 2024-06-10T02:41:47.963 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: Options.memtable_factory: SkipListFactory 2024-06-10T02:41:47.963 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: Options.table_factory: BlockBasedTable 2024-06-10T02:41:47.963 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x557c2ef55c40) 2024-06-10T02:41:47.963 INFO:journalctl@ceph.mon.a.smithi052.stdout: cache_index_and_filter_blocks: 1 2024-06-10T02:41:47.963 INFO:journalctl@ceph.mon.a.smithi052.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-06-10T02:41:47.963 INFO:journalctl@ceph.mon.a.smithi052.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-06-10T02:41:47.963 INFO:journalctl@ceph.mon.a.smithi052.stdout: pin_top_level_index_and_filter: 1 2024-06-10T02:41:47.963 INFO:journalctl@ceph.mon.a.smithi052.stdout: index_type: 0 2024-06-10T02:41:47.963 INFO:journalctl@ceph.mon.a.smithi052.stdout: data_block_index_type: 0 2024-06-10T02:41:47.963 INFO:journalctl@ceph.mon.a.smithi052.stdout: index_shortening: 1 2024-06-10T02:41:47.963 INFO:journalctl@ceph.mon.a.smithi052.stdout: data_block_hash_table_util_ratio: 0.750000 2024-06-10T02:41:47.964 INFO:journalctl@ceph.mon.a.smithi052.stdout: checksum: 4 2024-06-10T02:41:47.964 INFO:journalctl@ceph.mon.a.smithi052.stdout: no_block_cache: 0 2024-06-10T02:41:47.964 INFO:journalctl@ceph.mon.a.smithi052.stdout: block_cache: 0x557c2ef57090 2024-06-10T02:41:47.964 INFO:journalctl@ceph.mon.a.smithi052.stdout: block_cache_name: BinnedLRUCache 2024-06-10T02:41:47.964 INFO:journalctl@ceph.mon.a.smithi052.stdout: block_cache_options: 2024-06-10T02:41:47.964 INFO:journalctl@ceph.mon.a.smithi052.stdout: capacity : 536870912 2024-06-10T02:41:47.964 INFO:journalctl@ceph.mon.a.smithi052.stdout: num_shard_bits : 4 2024-06-10T02:41:47.964 INFO:journalctl@ceph.mon.a.smithi052.stdout: strict_capacity_limit : 0 2024-06-10T02:41:47.964 INFO:journalctl@ceph.mon.a.smithi052.stdout: high_pri_pool_ratio: 0.000 2024-06-10T02:41:47.964 INFO:journalctl@ceph.mon.a.smithi052.stdout: block_cache_compressed: (nil) 2024-06-10T02:41:47.964 INFO:journalctl@ceph.mon.a.smithi052.stdout: persistent_cache: (nil) 2024-06-10T02:41:47.964 INFO:journalctl@ceph.mon.a.smithi052.stdout: block_size: 4096 2024-06-10T02:41:47.964 INFO:journalctl@ceph.mon.a.smithi052.stdout: block_size_deviation: 10 2024-06-10T02:41:47.965 INFO:journalctl@ceph.mon.a.smithi052.stdout: block_restart_interval: 16 2024-06-10T02:41:47.965 INFO:journalctl@ceph.mon.a.smithi052.stdout: index_block_restart_interval: 1 2024-06-10T02:41:47.965 INFO:journalctl@ceph.mon.a.smithi052.stdout: metadata_block_size: 4096 2024-06-10T02:41:47.965 INFO:journalctl@ceph.mon.a.smithi052.stdout: partition_filters: 0 2024-06-10T02:41:47.965 INFO:journalctl@ceph.mon.a.smithi052.stdout: use_delta_encoding: 1 2024-06-10T02:41:47.965 INFO:journalctl@ceph.mon.a.smithi052.stdout: filter_policy: bloomfilter 2024-06-10T02:41:47.965 INFO:journalctl@ceph.mon.a.smithi052.stdout: whole_key_filtering: 1 2024-06-10T02:41:47.965 INFO:journalctl@ceph.mon.a.smithi052.stdout: verify_compression: 0 2024-06-10T02:41:47.965 INFO:journalctl@ceph.mon.a.smithi052.stdout: read_amp_bytes_per_bit: 0 2024-06-10T02:41:47.965 INFO:journalctl@ceph.mon.a.smithi052.stdout: format_version: 5 2024-06-10T02:41:47.965 INFO:journalctl@ceph.mon.a.smithi052.stdout: enable_index_compression: 1 2024-06-10T02:41:47.965 INFO:journalctl@ceph.mon.a.smithi052.stdout: block_align: 0 2024-06-10T02:41:47.965 INFO:journalctl@ceph.mon.a.smithi052.stdout: max_auto_readahead_size: 262144 2024-06-10T02:41:47.965 INFO:journalctl@ceph.mon.a.smithi052.stdout: prepopulate_block_cache: 0 2024-06-10T02:41:47.966 INFO:journalctl@ceph.mon.a.smithi052.stdout: initial_auto_readahead_size: 8192 2024-06-10T02:41:47.966 INFO:journalctl@ceph.mon.a.smithi052.stdout: num_file_reads_for_auto_readahead: 2 2024-06-10T02:41:47.966 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: Options.write_buffer_size: 33554432 2024-06-10T02:41:47.966 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: Options.max_write_buffer_number: 2 2024-06-10T02:41:47.966 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: Options.compression: NoCompression 2024-06-10T02:41:47.966 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: Options.bottommost_compression: Disabled 2024-06-10T02:41:47.966 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: Options.prefix_extractor: nullptr 2024-06-10T02:41:47.966 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-06-10T02:41:47.966 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: Options.num_levels: 7 2024-06-10T02:41:47.966 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-06-10T02:41:47.966 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-06-10T02:41:47.967 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-06-10T02:41:47.967 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-06-10T02:41:47.967 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-06-10T02:41:47.967 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-06-10T02:41:47.967 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-06-10T02:41:47.967 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-06-10T02:41:47.967 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-06-10T02:41:47.967 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-06-10T02:41:47.967 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-06-10T02:41:47.967 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-06-10T02:41:47.967 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: Options.compression_opts.window_bits: -14 2024-06-10T02:41:47.967 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: Options.compression_opts.level: 32767 2024-06-10T02:41:47.967 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: Options.compression_opts.strategy: 0 2024-06-10T02:41:47.967 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-06-10T02:41:47.968 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-06-10T02:41:47.968 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-06-10T02:41:47.968 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-06-10T02:41:47.968 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: Options.compression_opts.enabled: false 2024-06-10T02:41:47.968 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-06-10T02:41:47.968 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-06-10T02:41:47.969 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-06-10T02:41:47.969 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-06-10T02:41:47.969 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: Options.target_file_size_base: 67108864 2024-06-10T02:41:47.969 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: Options.target_file_size_multiplier: 1 2024-06-10T02:41:47.969 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-06-10T02:41:47.969 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-06-10T02:41:47.969 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-06-10T02:41:47.969 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-06-10T02:41:47.969 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-06-10T02:41:47.969 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-06-10T02:41:47.970 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-06-10T02:41:47.970 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-06-10T02:41:47.970 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-06-10T02:41:47.970 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-06-10T02:41:47.970 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-06-10T02:41:47.970 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-06-10T02:41:47.970 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-06-10T02:41:47.970 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: Options.arena_block_size: 1048576 2024-06-10T02:41:47.970 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-06-10T02:41:47.970 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-06-10T02:41:47.970 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: Options.disable_auto_compactions: 0 2024-06-10T02:41:47.970 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-06-10T02:41:47.971 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-06-10T02:41:47.971 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-06-10T02:41:47.971 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-06-10T02:41:47.971 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-06-10T02:41:47.971 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-06-10T02:41:47.971 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-06-10T02:41:47.971 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-06-10T02:41:47.971 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-06-10T02:41:47.971 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-06-10T02:41:47.971 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: Options.table_properties_collectors: 2024-06-10T02:41:47.971 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: Options.inplace_update_support: 0 2024-06-10T02:41:47.971 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: Options.inplace_update_num_locks: 10000 2024-06-10T02:41:47.971 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-06-10T02:41:47.972 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-06-10T02:41:47.972 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: Options.memtable_huge_page_size: 0 2024-06-10T02:41:47.972 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: Options.bloom_locality: 0 2024-06-10T02:41:47.972 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: Options.max_successive_merges: 0 2024-06-10T02:41:47.972 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: Options.optimize_filters_for_hits: 0 2024-06-10T02:41:47.972 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: Options.paranoid_file_checks: 0 2024-06-10T02:41:47.972 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: Options.force_consistency_checks: 1 2024-06-10T02:41:47.972 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: Options.report_bg_io_stats: 0 2024-06-10T02:41:47.972 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: Options.ttl: 2592000 2024-06-10T02:41:47.972 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: Options.periodic_compaction_seconds: 0 2024-06-10T02:41:47.972 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-06-10T02:41:47.972 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-06-10T02:41:47.973 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: Options.enable_blob_files: false 2024-06-10T02:41:47.973 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: Options.min_blob_size: 0 2024-06-10T02:41:47.973 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: Options.blob_file_size: 268435456 2024-06-10T02:41:47.973 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: Options.blob_compression_type: NoCompression 2024-06-10T02:41:47.973 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: Options.enable_blob_garbage_collection: false 2024-06-10T02:41:47.973 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-06-10T02:41:47.973 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-06-10T02:41:47.973 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-06-10T02:41:47.973 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: Options.blob_file_starting_level: 0 2024-06-10T02:41:47.973 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-06-10T02:41:47.973 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-06-10T02:41:47.973 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-06-10T02:41:47.973 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: b32f44b8-1434-4928-8a41-3ed43aefb1e9 2024-06-10T02:41:47.974 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: EVENT_LOG_v1 {"time_micros": 1717987307924071, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-06-10T02:41:47.974 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-06-10T02:41:47.974 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: EVENT_LOG_v1 {"time_micros": 1717987307925082, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 100517, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 328, "table_properties": {"data_size": 98428, "index_size": 307, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14328, "raw_average_key_size": 49, "raw_value_size": 90639, "raw_average_value_size": 315, "num_data_blocks": 12, "num_entries": 287, "num_filter_entries": 287, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1717987307, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "b32f44b8-1434-4928-8a41-3ed43aefb1e9", "db_session_id": "94MZS7RWYMM4T2UUHR3W", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-06-10T02:41:47.974 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: EVENT_LOG_v1 {"time_micros": 1717987307925167, "job": 1, "event": "recovery_finished"} 2024-06-10T02:41:47.974 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-06-10T02:41:47.974 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-06-10T02:41:47.974 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x557c2f046000 2024-06-10T02:41:47.974 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: DB pointer 0x557c2f030000 2024-06-10T02:41:47.974 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-06-10T02:41:47.974 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-06-10T02:41:47.974 INFO:journalctl@ceph.mon.a.smithi052.stdout: ** DB Stats ** 2024-06-10T02:41:47.974 INFO:journalctl@ceph.mon.a.smithi052.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-06-10T02:41:47.975 INFO:journalctl@ceph.mon.a.smithi052.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-06-10T02:41:47.975 INFO:journalctl@ceph.mon.a.smithi052.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-06-10T02:41:47.975 INFO:journalctl@ceph.mon.a.smithi052.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-06-10T02:41:47.975 INFO:journalctl@ceph.mon.a.smithi052.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-06-10T02:41:47.975 INFO:journalctl@ceph.mon.a.smithi052.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-06-10T02:41:47.975 INFO:journalctl@ceph.mon.a.smithi052.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-06-10T02:41:47.975 INFO:journalctl@ceph.mon.a.smithi052.stdout: 2024-06-10T02:41:47.975 INFO:journalctl@ceph.mon.a.smithi052.stdout: ** Compaction Stats [default] ** 2024-06-10T02:41:47.975 INFO:journalctl@ceph.mon.a.smithi052.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-06-10T02:41:47.975 INFO:journalctl@ceph.mon.a.smithi052.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-06-10T02:41:47.975 INFO:journalctl@ceph.mon.a.smithi052.stdout: L0 2/0 100.00 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 160.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-06-10T02:41:47.975 INFO:journalctl@ceph.mon.a.smithi052.stdout: Sum 2/0 100.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 160.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-06-10T02:41:47.975 INFO:journalctl@ceph.mon.a.smithi052.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 160.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-06-10T02:41:47.976 INFO:journalctl@ceph.mon.a.smithi052.stdout: 2024-06-10T02:41:47.976 INFO:journalctl@ceph.mon.a.smithi052.stdout: ** Compaction Stats [default] ** 2024-06-10T02:41:47.976 INFO:journalctl@ceph.mon.a.smithi052.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-06-10T02:41:47.976 INFO:journalctl@ceph.mon.a.smithi052.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-06-10T02:41:47.976 INFO:journalctl@ceph.mon.a.smithi052.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 160.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-06-10T02:41:47.976 INFO:journalctl@ceph.mon.a.smithi052.stdout: 2024-06-10T02:41:47.976 INFO:journalctl@ceph.mon.a.smithi052.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-06-10T02:41:47.976 INFO:journalctl@ceph.mon.a.smithi052.stdout: 2024-06-10T02:41:47.976 INFO:journalctl@ceph.mon.a.smithi052.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-06-10T02:41:47.976 INFO:journalctl@ceph.mon.a.smithi052.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-06-10T02:41:47.976 INFO:journalctl@ceph.mon.a.smithi052.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-06-10T02:41:47.976 INFO:journalctl@ceph.mon.a.smithi052.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-06-10T02:41:47.976 INFO:journalctl@ceph.mon.a.smithi052.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-06-10T02:41:47.977 INFO:journalctl@ceph.mon.a.smithi052.stdout: AddFile(Keys): cumulative 0, interval 0 2024-06-10T02:41:47.977 INFO:journalctl@ceph.mon.a.smithi052.stdout: Cumulative compaction: 0.00 GB write, 30.58 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-06-10T02:41:47.977 INFO:journalctl@ceph.mon.a.smithi052.stdout: Interval compaction: 0.00 GB write, 30.58 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-06-10T02:41:47.977 INFO:journalctl@ceph.mon.a.smithi052.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-06-10T02:41:47.977 INFO:journalctl@ceph.mon.a.smithi052.stdout: Block cache BinnedLRUCache@0x557c2ef57090#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 8e-06 secs_since: 0 2024-06-10T02:41:47.977 INFO:journalctl@ceph.mon.a.smithi052.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-06-10T02:41:47.977 INFO:journalctl@ceph.mon.a.smithi052.stdout: 2024-06-10T02:41:47.977 INFO:journalctl@ceph.mon.a.smithi052.stdout: ** File Read Latency Histogram By Level [default] ** 2024-06-10T02:41:47.977 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: starting mon.a rank 0 at public addrs [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] at bind addrs [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 2024-06-10T02:41:47.977 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: mon.a@-1(???) e1 preinit fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 2024-06-10T02:41:47.977 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: mon.a@-1(???).mds e1 new map 2024-06-10T02:41:47.977 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: mon.a@-1(???).mds e1 print_map 2024-06-10T02:41:47.977 INFO:journalctl@ceph.mon.a.smithi052.stdout: e1 2024-06-10T02:41:47.977 INFO:journalctl@ceph.mon.a.smithi052.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-06-10T02:41:47.978 INFO:journalctl@ceph.mon.a.smithi052.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-06-10T02:41:47.978 INFO:journalctl@ceph.mon.a.smithi052.stdout: legacy client fscid: -1 2024-06-10T02:41:47.978 INFO:journalctl@ceph.mon.a.smithi052.stdout: 2024-06-10T02:41:47.978 INFO:journalctl@ceph.mon.a.smithi052.stdout: No filesystems configured 2024-06-10T02:41:47.978 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-06-10T02:41:47.978 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-06-10T02:41:47.978 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-06-10T02:41:47.978 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-06-10T02:41:47.978 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-06-10T02:41:47.978 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 bash[32343]: cb4e54e38ee793a7d4a80b5b4ec7bed9005451170e7a0aaaaa917e8fddd8cc68 2024-06-10T02:41:47.978 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 systemd[1]: Started Ceph mon.a for d8676e9e-26d2-11ef-bc9b-c7b262605968. 2024-06-10T02:41:47.979 INFO:teuthology.orchestra.run.smithi052.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-06-10T02:41:48.223 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: mon.a is new leader, mons a in quorum (ranks 0) 2024-06-10T02:41:48.223 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: monmap epoch 1 2024-06-10T02:41:48.223 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 2024-06-10T02:41:48.223 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: last_changed 2024-06-10T02:41:40.094306+0000 2024-06-10T02:41:48.223 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: created 2024-06-10T02:41:40.094306+0000 2024-06-10T02:41:48.223 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: min_mon_release 19 (squid) 2024-06-10T02:41:48.223 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: election_strategy: 1 2024-06-10T02:41:48.223 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: 0: [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] mon.a 2024-06-10T02:41:48.223 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: fsmap 2024-06-10T02:41:48.223 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: osdmap e1: 0 total, 0 up, 0 in 2024-06-10T02:41:48.223 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:47 smithi052 ceph-mon[32362]: mgrmap e1: no daemons active 2024-06-10T02:41:48.974 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:48.350+0000 7f7bff382700 1 Processor -- start 2024-06-10T02:41:48.974 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:48.350+0000 7f7bff382700 1 -- start start 2024-06-10T02:41:48.974 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:48.350+0000 7f7bff382700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7bf8107fc0 0x7f7bf81083e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:41:48.974 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:48.350+0000 7f7bff382700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f7bf81089b0 con 0x7f7bf8107fc0 2024-06-10T02:41:48.974 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:48.350+0000 7f7bfd11e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7bf8107fc0 0x7f7bf81083e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:41:48.974 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:48.351+0000 7f7bfd11e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7bf8107fc0 0x7f7bf81083e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54500/0 (socket says 172.21.15.52:54500) 2024-06-10T02:41:48.974 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:48.351+0000 7f7bfd11e700 1 -- 172.21.15.52:0/2807966189 learned_addr learned my addr 172.21.15.52:0/2807966189 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:41:48.974 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:48.351+0000 7f7bfd11e700 1 -- 172.21.15.52:0/2807966189 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bf8109170 con 0x7f7bf8107fc0 2024-06-10T02:41:48.975 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:48.351+0000 7f7bfd11e700 1 --2- 172.21.15.52:0/2807966189 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7bf8107fc0 0x7f7bf81083e0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f7bf40098a0 tx=0x7f7bf4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e39181ff3d98cc46 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:41:48.975 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:48.351+0000 7f7beffff700 1 -- 172.21.15.52:0/2807966189 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7bf4004160 con 0x7f7bf8107fc0 2024-06-10T02:41:48.975 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:48.351+0000 7f7beffff700 1 -- 172.21.15.52:0/2807966189 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7bf4004300 con 0x7f7bf8107fc0 2024-06-10T02:41:48.975 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:48.351+0000 7f7beffff700 1 -- 172.21.15.52:0/2807966189 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7bf4011b50 con 0x7f7bf8107fc0 2024-06-10T02:41:48.975 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:48.352+0000 7f7bff382700 1 -- 172.21.15.52:0/2807966189 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7bf8107fc0 msgr2=0x7f7bf81083e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:41:48.975 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:48.352+0000 7f7bff382700 1 --2- 172.21.15.52:0/2807966189 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7bf8107fc0 0x7f7bf81083e0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f7bf40098a0 tx=0x7f7bf4009bb0 comp rx=0 tx=0).stop 2024-06-10T02:41:48.975 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:48.352+0000 7f7bff382700 1 -- 172.21.15.52:0/2807966189 shutdown_connections 2024-06-10T02:41:48.975 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:48.352+0000 7f7bff382700 1 --2- 172.21.15.52:0/2807966189 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7bf8107fc0 0x7f7bf81083e0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:41:48.975 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:48.352+0000 7f7bff382700 1 -- 172.21.15.52:0/2807966189 >> 172.21.15.52:0/2807966189 conn(0x7f7bf8103560 msgr2=0x7f7bf81059a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:41:48.975 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:48.352+0000 7f7bff382700 1 -- 172.21.15.52:0/2807966189 shutdown_connections 2024-06-10T02:41:48.975 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:48.352+0000 7f7bff382700 1 -- 172.21.15.52:0/2807966189 wait complete. 2024-06-10T02:41:48.975 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:48.353+0000 7f7bff382700 1 Processor -- start 2024-06-10T02:41:48.975 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:48.353+0000 7f7bff382700 1 -- start start 2024-06-10T02:41:48.976 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:48.353+0000 7f7bff382700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7bf8107fc0 0x7f7bf807e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:41:48.976 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:48.353+0000 7f7bff382700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f7bf810c520 con 0x7f7bf8107fc0 2024-06-10T02:41:48.976 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:48.353+0000 7f7bfd11e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7bf8107fc0 0x7f7bf807e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:41:48.976 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:48.353+0000 7f7bfd11e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7bf8107fc0 0x7f7bf807e410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54506/0 (socket says 172.21.15.52:54506) 2024-06-10T02:41:48.976 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:48.353+0000 7f7bfd11e700 1 -- 172.21.15.52:0/1852031592 learned_addr learned my addr 172.21.15.52:0/1852031592 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:41:48.976 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:48.353+0000 7f7bfd11e700 1 -- 172.21.15.52:0/1852031592 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bf807e950 con 0x7f7bf8107fc0 2024-06-10T02:41:48.976 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:48.353+0000 7f7bfd11e700 1 --2- 172.21.15.52:0/1852031592 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7bf8107fc0 0x7f7bf807e410 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f7bf4000c00 tx=0x7f7bf4011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:41:48.976 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:48.353+0000 7f7bee7fc700 1 -- 172.21.15.52:0/1852031592 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7bf4003b20 con 0x7f7bf8107fc0 2024-06-10T02:41:48.976 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:48.353+0000 7f7bee7fc700 1 -- 172.21.15.52:0/1852031592 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7bf4003cc0 con 0x7f7bf8107fc0 2024-06-10T02:41:48.976 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:48.353+0000 7f7bff382700 1 -- 172.21.15.52:0/1852031592 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7bf807cb40 con 0x7f7bf8107fc0 2024-06-10T02:41:48.976 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:48.354+0000 7f7bee7fc700 1 -- 172.21.15.52:0/1852031592 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7bf4024910 con 0x7f7bf8107fc0 2024-06-10T02:41:48.976 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:48.354+0000 7f7bff382700 1 -- 172.21.15.52:0/1852031592 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7bf807d080 con 0x7f7bf8107fc0 2024-06-10T02:41:48.976 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:48.354+0000 7f7bee7fc700 1 -- 172.21.15.52:0/1852031592 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f7bf4022020 con 0x7f7bf8107fc0 2024-06-10T02:41:48.977 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:48.354+0000 7f7bee7fc700 1 -- 172.21.15.52:0/1852031592 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f7bf4011e00 con 0x7f7bf8107fc0 2024-06-10T02:41:48.977 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:48.355+0000 7f7bff382700 1 -- 172.21.15.52:0/1852031592 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7bf8063af0 con 0x7f7bf8107fc0 2024-06-10T02:41:48.977 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:48.357+0000 7f7bee7fc700 1 -- 172.21.15.52:0/1852031592 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f7bf402ddf0 con 0x7f7bf8107fc0 2024-06-10T02:41:48.977 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:48.427+0000 7f7bff382700 1 -- 172.21.15.52:0/1852031592 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7f7bf807d5b0 con 0x7f7bf8107fc0 2024-06-10T02:41:48.977 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:48.429+0000 7f7bee7fc700 1 -- 172.21.15.52:0/1852031592 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7f7bf4032030 con 0x7f7bf8107fc0 2024-06-10T02:41:48.977 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:48.430+0000 7f7bff382700 1 -- 172.21.15.52:0/1852031592 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7bf8107fc0 msgr2=0x7f7bf807e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:41:48.977 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:48.430+0000 7f7bff382700 1 --2- 172.21.15.52:0/1852031592 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7bf8107fc0 0x7f7bf807e410 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f7bf4000c00 tx=0x7f7bf4011c10 comp rx=0 tx=0).stop 2024-06-10T02:41:48.977 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:48.430+0000 7f7bff382700 1 -- 172.21.15.52:0/1852031592 shutdown_connections 2024-06-10T02:41:48.977 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:48.430+0000 7f7bff382700 1 --2- 172.21.15.52:0/1852031592 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7bf8107fc0 0x7f7bf807e410 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:41:48.977 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:48.430+0000 7f7bff382700 1 -- 172.21.15.52:0/1852031592 >> 172.21.15.52:0/1852031592 conn(0x7f7bf8103560 msgr2=0x7f7bf8104030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:41:48.977 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:48.430+0000 7f7bff382700 1 -- 172.21.15.52:0/1852031592 shutdown_connections 2024-06-10T02:41:48.977 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:48.430+0000 7f7bff382700 1 -- 172.21.15.52:0/1852031592 wait complete. 2024-06-10T02:41:48.977 INFO:teuthology.orchestra.run.smithi052.stdout:Wrote config to /etc/ceph/ceph.conf 2024-06-10T02:41:49.000 INFO:teuthology.orchestra.run.smithi052.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-06-10T02:41:49.000 INFO:teuthology.orchestra.run.smithi052.stdout:Creating mgr... 2024-06-10T02:41:49.000 INFO:teuthology.orchestra.run.smithi052.stdout:Verifying port 0.0.0.0:9283 ... 2024-06-10T02:41:49.001 INFO:teuthology.orchestra.run.smithi052.stdout:Verifying port 0.0.0.0:8765 ... 2024-06-10T02:41:49.265 INFO:teuthology.orchestra.run.smithi052.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968@mgr.x 2024-06-10T02:41:49.265 INFO:teuthology.orchestra.run.smithi052.stdout:systemctl: stderr Failed to reset failed state of unit ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968@mgr.x.service: Unit ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968@mgr.x.service not loaded. 2024-06-10T02:41:49.419 INFO:teuthology.orchestra.run.smithi052.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968.target.wants/ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968@.service. 2024-06-10T02:41:49.652 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:49 smithi052 ceph-mon[32362]: from='client.? 172.21.15.52:0/1852031592' entity='client.admin' 2024-06-10T02:41:49.653 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 10 02:41:49 smithi052 systemd[1]: Starting Ceph mgr.x for d8676e9e-26d2-11ef-bc9b-c7b262605968... 2024-06-10T02:41:49.921 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 10 02:41:49 smithi052 podman[32580]: 2024-06-10 02:41:49.657425185 +0000 UTC m=+0.104640607 container create c194861d5c8bae344a78836907d4a118fc2c6cbfd0e5fc5a27e6dc0bb5527d0e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mgr-x, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-06-10T02:41:49.921 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 10 02:41:49 smithi052 podman[32580]: 2024-06-10 02:41:49.570445349 +0000 UTC m=+0.017660760 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-10T02:41:49.921 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 10 02:41:49 smithi052 podman[32580]: 2024-06-10 02:41:49.85117179 +0000 UTC m=+0.298387199 container init c194861d5c8bae344a78836907d4a118fc2c6cbfd0e5fc5a27e6dc0bb5527d0e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mgr-x, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-06-10T02:41:49.922 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 10 02:41:49 smithi052 podman[32580]: 2024-06-10 02:41:49.853882666 +0000 UTC m=+0.301098068 container start c194861d5c8bae344a78836907d4a118fc2c6cbfd0e5fc5a27e6dc0bb5527d0e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mgr-x, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-10T02:41:49.935 INFO:teuthology.orchestra.run.smithi052.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-06-10T02:41:49.935 INFO:teuthology.orchestra.run.smithi052.stdout:systemctl: stdout disabled 2024-06-10T02:41:49.943 INFO:teuthology.orchestra.run.smithi052.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-06-10T02:41:49.943 INFO:teuthology.orchestra.run.smithi052.stdout:systemctl: stdout inactive 2024-06-10T02:41:49.943 INFO:teuthology.orchestra.run.smithi052.stdout:firewalld.service is not enabled 2024-06-10T02:41:49.944 INFO:teuthology.orchestra.run.smithi052.stdout:Not possible to enable service . firewalld.service is not available 2024-06-10T02:41:49.953 INFO:teuthology.orchestra.run.smithi052.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-06-10T02:41:49.953 INFO:teuthology.orchestra.run.smithi052.stdout:systemctl: stdout disabled 2024-06-10T02:41:49.961 INFO:teuthology.orchestra.run.smithi052.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-06-10T02:41:49.961 INFO:teuthology.orchestra.run.smithi052.stdout:systemctl: stdout inactive 2024-06-10T02:41:49.962 INFO:teuthology.orchestra.run.smithi052.stdout:firewalld.service is not enabled 2024-06-10T02:41:49.962 INFO:teuthology.orchestra.run.smithi052.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-06-10T02:41:49.962 INFO:teuthology.orchestra.run.smithi052.stdout:Waiting for mgr to start... 2024-06-10T02:41:49.962 INFO:teuthology.orchestra.run.smithi052.stdout:Waiting for mgr... 2024-06-10T02:41:50.187 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 10 02:41:49 smithi052 bash[32580]: c194861d5c8bae344a78836907d4a118fc2c6cbfd0e5fc5a27e6dc0bb5527d0e 2024-06-10T02:41:50.188 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 10 02:41:49 smithi052 systemd[1]: Started Ceph mgr.x for d8676e9e-26d2-11ef-bc9b-c7b262605968. 2024-06-10T02:41:50.188 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 10 02:41:50 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mgr-x[32595]: 2024-06-10T02:41:50.025+0000 7f9486242200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-06-10T02:41:50.472 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 10 02:41:50 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mgr-x[32595]: 2024-06-10T02:41:50.220+0000 7f9486242200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-06-10T02:41:50.473 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 10 02:41:50 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mgr-x[32595]: 2024-06-10T02:41:50.328+0000 7f9486242200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-06-10T02:41:50.745 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:50 smithi052 ceph-mon[32362]: from='client.? 172.21.15.52:0/346821063' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-06-10T02:41:50.745 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 10 02:41:50 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mgr-x[32595]: 2024-06-10T02:41:50.659+0000 7f9486242200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-06-10T02:41:50.745 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 10 02:41:50 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mgr-x[32595]: 2024-06-10T02:41:50.749+0000 7f9486242200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-06-10T02:41:51.089 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout 2024-06-10T02:41:51.089 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout { 2024-06-10T02:41:51.089 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "fsid": "d8676e9e-26d2-11ef-bc9b-c7b262605968", 2024-06-10T02:41:51.089 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "health": { 2024-06-10T02:41:51.089 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-06-10T02:41:51.090 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-06-10T02:41:51.090 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-06-10T02:41:51.090 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout }, 2024-06-10T02:41:51.090 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-06-10T02:41:51.090 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-06-10T02:41:51.090 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout 0 2024-06-10T02:41:51.090 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout ], 2024-06-10T02:41:51.090 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-06-10T02:41:51.090 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "a" 2024-06-10T02:41:51.090 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout ], 2024-06-10T02:41:51.090 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-06-10T02:41:51.090 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "monmap": { 2024-06-10T02:41:51.090 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-10T02:41:51.091 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-06-10T02:41:51.091 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-06-10T02:41:51.091 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout }, 2024-06-10T02:41:51.091 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-06-10T02:41:51.091 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-10T02:41:51.091 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-06-10T02:41:51.091 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-06-10T02:41:51.091 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-06-10T02:41:51.091 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-06-10T02:41:51.091 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-06-10T02:41:51.091 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-06-10T02:41:51.091 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout }, 2024-06-10T02:41:51.091 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-06-10T02:41:51.092 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-06-10T02:41:51.092 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-06-10T02:41:51.092 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-06-10T02:41:51.092 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-06-10T02:41:51.092 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-06-10T02:41:51.092 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-06-10T02:41:51.092 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-06-10T02:41:51.092 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-06-10T02:41:51.092 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout }, 2024-06-10T02:41:51.092 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-06-10T02:41:51.092 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-10T02:41:51.092 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-06-10T02:41:51.092 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-06-10T02:41:51.092 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout }, 2024-06-10T02:41:51.092 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-06-10T02:41:51.093 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "available": false, 2024-06-10T02:41:51.093 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-06-10T02:41:51.093 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "modules": [ 2024-06-10T02:41:51.093 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "iostat", 2024-06-10T02:41:51.093 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "nfs", 2024-06-10T02:41:51.093 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "restful" 2024-06-10T02:41:51.093 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout ], 2024-06-10T02:41:51.093 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "services": {} 2024-06-10T02:41:51.093 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout }, 2024-06-10T02:41:51.093 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-06-10T02:41:51.093 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-10T02:41:51.093 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "modified": "2024-06-10T02:41:43.077320+0000", 2024-06-10T02:41:51.093 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "services": {} 2024-06-10T02:41:51.093 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout }, 2024-06-10T02:41:51.094 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-06-10T02:41:51.094 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout } 2024-06-10T02:41:51.094 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:50.407+0000 7f3f87fdc700 1 Processor -- start 2024-06-10T02:41:51.094 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:50.407+0000 7f3f87fdc700 1 -- start start 2024-06-10T02:41:51.094 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:50.407+0000 7f3f87fdc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3f801062a0 0x7f3f801066c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:41:51.094 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:50.407+0000 7f3f87fdc700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f3f80106c90 con 0x7f3f801062a0 2024-06-10T02:41:51.094 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:50.408+0000 7f3f85d78700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3f801062a0 0x7f3f801066c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:41:51.094 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:50.408+0000 7f3f85d78700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3f801062a0 0x7f3f801066c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54536/0 (socket says 172.21.15.52:54536) 2024-06-10T02:41:51.094 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:50.408+0000 7f3f85d78700 1 -- 172.21.15.52:0/3438421291 learned_addr learned my addr 172.21.15.52:0/3438421291 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:41:51.094 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:50.408+0000 7f3f85d78700 1 -- 172.21.15.52:0/3438421291 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f80109230 con 0x7f3f801062a0 2024-06-10T02:41:51.094 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:50.408+0000 7f3f85d78700 1 --2- 172.21.15.52:0/3438421291 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3f801062a0 0x7f3f801066c0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f3f7c0098a0 tx=0x7f3f7c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=831099a0e16bdbe0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:41:51.094 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:50.408+0000 7f3f84d76700 1 -- 172.21.15.52:0/3438421291 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3f7c004160 con 0x7f3f801062a0 2024-06-10T02:41:51.094 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:50.409+0000 7f3f84d76700 1 -- 172.21.15.52:0/3438421291 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f3f7c004300 con 0x7f3f801062a0 2024-06-10T02:41:51.095 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:50.409+0000 7f3f84d76700 1 -- 172.21.15.52:0/3438421291 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3f7c011b50 con 0x7f3f801062a0 2024-06-10T02:41:51.095 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:50.409+0000 7f3f87fdc700 1 -- 172.21.15.52:0/3438421291 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3f801062a0 msgr2=0x7f3f801066c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:41:51.095 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:50.409+0000 7f3f87fdc700 1 --2- 172.21.15.52:0/3438421291 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3f801062a0 0x7f3f801066c0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f3f7c0098a0 tx=0x7f3f7c009bb0 comp rx=0 tx=0).stop 2024-06-10T02:41:51.095 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:50.409+0000 7f3f87fdc700 1 -- 172.21.15.52:0/3438421291 shutdown_connections 2024-06-10T02:41:51.095 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:50.409+0000 7f3f87fdc700 1 --2- 172.21.15.52:0/3438421291 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3f801062a0 0x7f3f801066c0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:41:51.095 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:50.409+0000 7f3f87fdc700 1 -- 172.21.15.52:0/3438421291 >> 172.21.15.52:0/3438421291 conn(0x7f3f80101bd0 msgr2=0x7f3f80104010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:41:51.095 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:50.410+0000 7f3f87fdc700 1 -- 172.21.15.52:0/3438421291 shutdown_connections 2024-06-10T02:41:51.095 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:50.410+0000 7f3f87fdc700 1 -- 172.21.15.52:0/3438421291 wait complete. 2024-06-10T02:41:51.095 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:50.410+0000 7f3f87fdc700 1 Processor -- start 2024-06-10T02:41:51.095 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:50.410+0000 7f3f87fdc700 1 -- start start 2024-06-10T02:41:51.095 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:50.410+0000 7f3f87fdc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3f801062a0 0x7f3f8019bef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:41:51.095 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:50.410+0000 7f3f87fdc700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f3f8010c5e0 con 0x7f3f801062a0 2024-06-10T02:41:51.095 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:50.411+0000 7f3f85d78700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3f801062a0 0x7f3f8019bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:41:51.095 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:50.411+0000 7f3f85d78700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3f801062a0 0x7f3f8019bef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54540/0 (socket says 172.21.15.52:54540) 2024-06-10T02:41:51.096 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:50.411+0000 7f3f85d78700 1 -- 172.21.15.52:0/346821063 learned_addr learned my addr 172.21.15.52:0/346821063 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:41:51.096 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:50.411+0000 7f3f85d78700 1 -- 172.21.15.52:0/346821063 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f8019c430 con 0x7f3f801062a0 2024-06-10T02:41:51.096 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:50.411+0000 7f3f85d78700 1 --2- 172.21.15.52:0/346821063 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3f801062a0 0x7f3f8019bef0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f3f7c000c00 tx=0x7f3f7c011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:41:51.096 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:50.411+0000 7f3f72ffd700 1 -- 172.21.15.52:0/346821063 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3f7c003b20 con 0x7f3f801062a0 2024-06-10T02:41:51.096 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:50.411+0000 7f3f72ffd700 1 -- 172.21.15.52:0/346821063 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f3f7c003cc0 con 0x7f3f801062a0 2024-06-10T02:41:51.096 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:50.411+0000 7f3f87fdc700 1 -- 172.21.15.52:0/346821063 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f8019c6c0 con 0x7f3f801062a0 2024-06-10T02:41:51.096 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:50.411+0000 7f3f87fdc700 1 -- 172.21.15.52:0/346821063 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f8019f3b0 con 0x7f3f801062a0 2024-06-10T02:41:51.096 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:50.411+0000 7f3f72ffd700 1 -- 172.21.15.52:0/346821063 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3f7c024910 con 0x7f3f801062a0 2024-06-10T02:41:51.096 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:50.412+0000 7f3f72ffd700 1 -- 172.21.15.52:0/346821063 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f3f7c022020 con 0x7f3f801062a0 2024-06-10T02:41:51.096 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:50.412+0000 7f3f72ffd700 1 -- 172.21.15.52:0/346821063 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f3f7c011e00 con 0x7f3f801062a0 2024-06-10T02:41:51.096 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:50.412+0000 7f3f87fdc700 1 -- 172.21.15.52:0/346821063 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f80063af0 con 0x7f3f801062a0 2024-06-10T02:41:51.096 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:50.414+0000 7f3f72ffd700 1 -- 172.21.15.52:0/346821063 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f3f7c01acd0 con 0x7f3f801062a0 2024-06-10T02:41:51.096 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:50.488+0000 7f3f87fdc700 1 -- 172.21.15.52:0/346821063 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f3f8019f580 con 0x7f3f801062a0 2024-06-10T02:41:51.097 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:50.489+0000 7f3f72ffd700 1 -- 172.21.15.52:0/346821063 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f3f7c02d3f0 con 0x7f3f801062a0 2024-06-10T02:41:51.097 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:50.490+0000 7f3f87fdc700 1 -- 172.21.15.52:0/346821063 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3f801062a0 msgr2=0x7f3f8019bef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:41:51.097 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:50.490+0000 7f3f87fdc700 1 --2- 172.21.15.52:0/346821063 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3f801062a0 0x7f3f8019bef0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f3f7c000c00 tx=0x7f3f7c011c10 comp rx=0 tx=0).stop 2024-06-10T02:41:51.097 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:50.490+0000 7f3f87fdc700 1 -- 172.21.15.52:0/346821063 shutdown_connections 2024-06-10T02:41:51.097 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:50.490+0000 7f3f87fdc700 1 --2- 172.21.15.52:0/346821063 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3f801062a0 0x7f3f8019bef0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:41:51.097 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:50.490+0000 7f3f87fdc700 1 -- 172.21.15.52:0/346821063 >> 172.21.15.52:0/346821063 conn(0x7f3f80101bd0 msgr2=0x7f3f80102600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:41:51.097 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:50.490+0000 7f3f87fdc700 1 -- 172.21.15.52:0/346821063 shutdown_connections 2024-06-10T02:41:51.097 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:50.490+0000 7f3f87fdc700 1 -- 172.21.15.52:0/346821063 wait complete. 2024-06-10T02:41:51.097 INFO:teuthology.orchestra.run.smithi052.stdout:mgr not available, waiting (1/15)... 2024-06-10T02:41:52.360 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 10 02:41:52 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mgr-x[32595]: 2024-06-10T02:41:52.013+0000 7f9486242200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-06-10T02:41:52.360 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 10 02:41:52 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mgr-x[32595]: 2024-06-10T02:41:52.189+0000 7f9486242200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-06-10T02:41:52.361 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 10 02:41:52 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mgr-x[32595]: 2024-06-10T02:41:52.365+0000 7f9486242200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-06-10T02:41:52.966 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 10 02:41:52 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mgr-x[32595]: 2024-06-10T02:41:52.664+0000 7f9486242200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-06-10T02:41:52.966 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 10 02:41:52 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mgr-x[32595]: 2024-06-10T02:41:52.761+0000 7f9486242200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-06-10T02:41:52.966 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 10 02:41:52 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mgr-x[32595]: 2024-06-10T02:41:52.970+0000 7f9486242200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-06-10T02:41:53.222 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 10 02:41:53 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mgr-x[32595]: 2024-06-10T02:41:53.173+0000 7f9486242200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-06-10T02:41:54.183 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 10 02:41:53 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mgr-x[32595]: 2024-06-10T02:41:53.805+0000 7f9486242200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-06-10T02:41:54.472 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 10 02:41:54 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mgr-x[32595]: 2024-06-10T02:41:54.244+0000 7f9486242200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-06-10T02:41:54.473 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 10 02:41:54 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mgr-x[32595]: 2024-06-10T02:41:54.345+0000 7f9486242200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-06-10T02:41:54.972 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 10 02:41:54 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mgr-x[32595]: 2024-06-10T02:41:54.554+0000 7f9486242200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-06-10T02:41:55.445 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 10 02:41:55 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mgr-x[32595]: 2024-06-10T02:41:55.100+0000 7f9486242200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-06-10T02:41:55.445 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 10 02:41:55 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mgr-x[32595]: 2024-06-10T02:41:55.196+0000 7f9486242200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-06-10T02:41:55.722 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 10 02:41:55 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mgr-x[32595]: 2024-06-10T02:41:55.449+0000 7f9486242200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-06-10T02:41:55.723 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 10 02:41:55 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mgr-x[32595]: 2024-06-10T02:41:55.589+0000 7f9486242200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-06-10T02:41:55.723 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 10 02:41:55 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mgr-x[32595]: 2024-06-10T02:41:55.687+0000 7f9486242200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-06-10T02:41:56.120 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 10 02:41:55 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mgr-x[32595]: 2024-06-10T02:41:55.808+0000 7f9486242200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-06-10T02:41:56.120 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 10 02:41:56 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mgr-x[32595]: 2024-06-10T02:41:56.124+0000 7f9486242200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-06-10T02:41:56.472 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:56 smithi052 ceph-mon[32362]: from='client.? 172.21.15.52:0/2001851799' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-06-10T02:41:57.021 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 10 02:41:56 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mgr-x[32595]: 2024-06-10T02:41:56.758+0000 7f9486242200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-06-10T02:41:57.021 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 10 02:41:57 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mgr-x[32595]: 2024-06-10T02:41:56.983+0000 7f9486242200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-06-10T02:41:57.473 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:57 smithi052 ceph-mon[32362]: Activating manager daemon x 2024-06-10T02:41:57.473 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:57 smithi052 ceph-mon[32362]: mgrmap e2: x(active, starting, since 0.00287546s) 2024-06-10T02:41:57.473 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:57 smithi052 ceph-mon[32362]: from='mgr.14100 172.21.15.52:0/448441699' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-06-10T02:41:57.473 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:57 smithi052 ceph-mon[32362]: from='mgr.14100 172.21.15.52:0/448441699' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-06-10T02:41:57.473 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:57 smithi052 ceph-mon[32362]: from='mgr.14100 172.21.15.52:0/448441699' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-06-10T02:41:57.473 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:57 smithi052 ceph-mon[32362]: from='mgr.14100 172.21.15.52:0/448441699' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-10T02:41:57.473 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:57 smithi052 ceph-mon[32362]: from='mgr.14100 172.21.15.52:0/448441699' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-06-10T02:41:57.473 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:57 smithi052 ceph-mon[32362]: Manager daemon x is now available 2024-06-10T02:41:57.473 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:57 smithi052 ceph-mon[32362]: from='mgr.14100 172.21.15.52:0/448441699' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-06-10T02:41:57.473 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:57 smithi052 ceph-mon[32362]: from='mgr.14100 172.21.15.52:0/448441699' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-06-10T02:41:57.473 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:57 smithi052 ceph-mon[32362]: from='mgr.14100 172.21.15.52:0/448441699' entity='mgr.x' 2024-06-10T02:41:57.474 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:57 smithi052 ceph-mon[32362]: from='mgr.14100 172.21.15.52:0/448441699' entity='mgr.x' 2024-06-10T02:41:57.474 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:57 smithi052 ceph-mon[32362]: from='mgr.14100 172.21.15.52:0/448441699' entity='mgr.x' 2024-06-10T02:41:59.361 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:58 smithi052 ceph-mon[32362]: mgrmap e3: x(active, since 1.00749s) 2024-06-10T02:42:00.472 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:41:59 smithi052 ceph-mon[32362]: mgrmap e4: x(active, since 2s) 2024-06-10T02:42:02.866 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout 2024-06-10T02:42:02.877 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout { 2024-06-10T02:42:02.877 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "fsid": "d8676e9e-26d2-11ef-bc9b-c7b262605968", 2024-06-10T02:42:02.877 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "health": { 2024-06-10T02:42:02.877 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-06-10T02:42:02.877 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-06-10T02:42:02.877 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-06-10T02:42:02.877 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout }, 2024-06-10T02:42:02.878 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-06-10T02:42:02.878 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-06-10T02:42:02.878 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout 0 2024-06-10T02:42:02.878 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout ], 2024-06-10T02:42:02.878 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-06-10T02:42:02.878 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "a" 2024-06-10T02:42:02.878 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout ], 2024-06-10T02:42:02.878 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "quorum_age": 8, 2024-06-10T02:42:02.878 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "monmap": { 2024-06-10T02:42:02.878 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-10T02:42:02.878 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-06-10T02:42:02.879 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-06-10T02:42:02.879 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout }, 2024-06-10T02:42:02.879 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-06-10T02:42:02.879 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-10T02:42:02.879 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-06-10T02:42:02.879 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-06-10T02:42:02.879 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-06-10T02:42:02.880 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-06-10T02:42:02.880 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-06-10T02:42:02.880 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-06-10T02:42:02.880 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout }, 2024-06-10T02:42:02.880 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-06-10T02:42:02.880 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-06-10T02:42:02.880 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-06-10T02:42:02.880 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-06-10T02:42:02.880 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-06-10T02:42:02.880 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-06-10T02:42:02.881 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-06-10T02:42:02.881 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-06-10T02:42:02.881 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-06-10T02:42:02.881 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout }, 2024-06-10T02:42:02.881 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-06-10T02:42:02.881 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-10T02:42:02.881 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-06-10T02:42:02.881 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-06-10T02:42:02.881 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout }, 2024-06-10T02:42:02.881 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-06-10T02:42:02.881 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "available": false, 2024-06-10T02:42:02.881 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-06-10T02:42:02.881 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "modules": [ 2024-06-10T02:42:02.881 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "iostat", 2024-06-10T02:42:02.882 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "nfs", 2024-06-10T02:42:02.882 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "restful" 2024-06-10T02:42:02.882 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout ], 2024-06-10T02:42:02.882 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "services": {} 2024-06-10T02:42:02.882 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout }, 2024-06-10T02:42:02.882 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-06-10T02:42:02.882 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-10T02:42:02.882 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "modified": "2024-06-10T02:41:43.077320+0000", 2024-06-10T02:42:02.882 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "services": {} 2024-06-10T02:42:02.882 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout }, 2024-06-10T02:42:02.882 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-06-10T02:42:02.882 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout } 2024-06-10T02:42:02.883 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:56.108+0000 7f85ee043700 1 Processor -- start 2024-06-10T02:42:02.883 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:56.108+0000 7f85ee043700 1 -- start start 2024-06-10T02:42:02.883 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:56.108+0000 7f85ee043700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f85e8106e60 0x7f85e8107280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:42:02.883 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:56.108+0000 7f85ee043700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f85e81077c0 con 0x7f85e8106e60 2024-06-10T02:42:02.883 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:56.109+0000 7f85e77fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f85e8106e60 0x7f85e8107280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:42:02.883 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:56.109+0000 7f85e77fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f85e8106e60 0x7f85e8107280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:34118/0 (socket says 172.21.15.52:34118) 2024-06-10T02:42:02.883 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:56.109+0000 7f85e77fe700 1 -- 172.21.15.52:0/4172814061 learned_addr learned my addr 172.21.15.52:0/4172814061 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:42:02.883 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:56.109+0000 7f85e77fe700 1 -- 172.21.15.52:0/4172814061 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85e8108050 con 0x7f85e8106e60 2024-06-10T02:42:02.883 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:56.109+0000 7f85e77fe700 1 --2- 172.21.15.52:0/4172814061 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f85e8106e60 0x7f85e8107280 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f85d80098a0 tx=0x7f85d8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59cf1f7b72fcaeeb server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:42:02.883 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:56.109+0000 7f85e6ffd700 1 -- 172.21.15.52:0/4172814061 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f85d8004160 con 0x7f85e8106e60 2024-06-10T02:42:02.883 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:56.109+0000 7f85e6ffd700 1 -- 172.21.15.52:0/4172814061 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f85d8004300 con 0x7f85e8106e60 2024-06-10T02:42:02.883 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:56.109+0000 7f85e6ffd700 1 -- 172.21.15.52:0/4172814061 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f85d8011b50 con 0x7f85e8106e60 2024-06-10T02:42:02.883 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:56.110+0000 7f85ee043700 1 -- 172.21.15.52:0/4172814061 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f85e8106e60 msgr2=0x7f85e8107280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:42:02.884 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:56.110+0000 7f85ee043700 1 --2- 172.21.15.52:0/4172814061 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f85e8106e60 0x7f85e8107280 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f85d80098a0 tx=0x7f85d8009bb0 comp rx=0 tx=0).stop 2024-06-10T02:42:02.884 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:56.110+0000 7f85ee043700 1 -- 172.21.15.52:0/4172814061 shutdown_connections 2024-06-10T02:42:02.884 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:56.110+0000 7f85ee043700 1 --2- 172.21.15.52:0/4172814061 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f85e8106e60 0x7f85e8107280 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:42:02.884 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:56.110+0000 7f85ee043700 1 -- 172.21.15.52:0/4172814061 >> 172.21.15.52:0/4172814061 conn(0x7f85e8102450 msgr2=0x7f85e8104890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:42:02.884 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:56.110+0000 7f85ee043700 1 -- 172.21.15.52:0/4172814061 shutdown_connections 2024-06-10T02:42:02.884 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:56.110+0000 7f85ee043700 1 -- 172.21.15.52:0/4172814061 wait complete. 2024-06-10T02:42:02.884 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:56.111+0000 7f85ee043700 1 Processor -- start 2024-06-10T02:42:02.884 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:56.111+0000 7f85ee043700 1 -- start start 2024-06-10T02:42:02.884 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:56.111+0000 7f85ee043700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f85e8106e60 0x7f85e807bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:42:02.884 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:56.111+0000 7f85ee043700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f85e810b310 con 0x7f85e8106e60 2024-06-10T02:42:02.884 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:56.111+0000 7f85e77fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f85e8106e60 0x7f85e807bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:42:02.884 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:56.111+0000 7f85e77fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f85e8106e60 0x7f85e807bac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:34134/0 (socket says 172.21.15.52:34134) 2024-06-10T02:42:02.884 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:56.111+0000 7f85e77fe700 1 -- 172.21.15.52:0/2001851799 learned_addr learned my addr 172.21.15.52:0/2001851799 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:42:02.885 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:56.111+0000 7f85e77fe700 1 -- 172.21.15.52:0/2001851799 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85e819ec00 con 0x7f85e8106e60 2024-06-10T02:42:02.885 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:56.112+0000 7f85e77fe700 1 --2- 172.21.15.52:0/2001851799 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f85e8106e60 0x7f85e807bac0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f85d8000c00 tx=0x7f85d8011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:42:02.885 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:56.112+0000 7f85e57fa700 1 -- 172.21.15.52:0/2001851799 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f85d8003b20 con 0x7f85e8106e60 2024-06-10T02:42:02.885 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:56.112+0000 7f85e57fa700 1 -- 172.21.15.52:0/2001851799 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f85d8003cc0 con 0x7f85e8106e60 2024-06-10T02:42:02.885 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:56.112+0000 7f85ee043700 1 -- 172.21.15.52:0/2001851799 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85e819edd0 con 0x7f85e8106e60 2024-06-10T02:42:02.885 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:56.112+0000 7f85e57fa700 1 -- 172.21.15.52:0/2001851799 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f85d8024910 con 0x7f85e8106e60 2024-06-10T02:42:02.885 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:56.112+0000 7f85ee043700 1 -- 172.21.15.52:0/2001851799 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85e819f1f0 con 0x7f85e8106e60 2024-06-10T02:42:02.885 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:56.112+0000 7f85e57fa700 1 -- 172.21.15.52:0/2001851799 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f85d8022020 con 0x7f85e8106e60 2024-06-10T02:42:02.885 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:56.112+0000 7f85e57fa700 1 -- 172.21.15.52:0/2001851799 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f85d8011e00 con 0x7f85e8106e60 2024-06-10T02:42:02.885 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:56.113+0000 7f85ee043700 1 -- 172.21.15.52:0/2001851799 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85e810b490 con 0x7f85e8106e60 2024-06-10T02:42:02.885 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:56.115+0000 7f85e57fa700 1 -- 172.21.15.52:0/2001851799 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f85d801acd0 con 0x7f85e8106e60 2024-06-10T02:42:02.885 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:56.190+0000 7f85ee043700 1 -- 172.21.15.52:0/2001851799 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f85e819f4a0 con 0x7f85e8106e60 2024-06-10T02:42:02.885 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:56.191+0000 7f85e57fa700 1 -- 172.21.15.52:0/2001851799 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f85d8032030 con 0x7f85e8106e60 2024-06-10T02:42:02.886 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:56.192+0000 7f85ee043700 1 -- 172.21.15.52:0/2001851799 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f85e8106e60 msgr2=0x7f85e807bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:42:02.886 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:56.192+0000 7f85ee043700 1 --2- 172.21.15.52:0/2001851799 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f85e8106e60 0x7f85e807bac0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f85d8000c00 tx=0x7f85d8011c10 comp rx=0 tx=0).stop 2024-06-10T02:42:02.886 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:56.192+0000 7f85ee043700 1 -- 172.21.15.52:0/2001851799 shutdown_connections 2024-06-10T02:42:02.886 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:56.192+0000 7f85ee043700 1 --2- 172.21.15.52:0/2001851799 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f85e8106e60 0x7f85e807bac0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:42:02.886 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:56.192+0000 7f85ee043700 1 -- 172.21.15.52:0/2001851799 >> 172.21.15.52:0/2001851799 conn(0x7f85e8102450 msgr2=0x7f85e8102ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:42:02.886 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:56.192+0000 7f85ee043700 1 -- 172.21.15.52:0/2001851799 shutdown_connections 2024-06-10T02:42:02.886 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:41:56.192+0000 7f85ee043700 1 -- 172.21.15.52:0/2001851799 wait complete. 2024-06-10T02:42:02.886 INFO:teuthology.orchestra.run.smithi052.stdout:mgr not available, waiting (2/15)... 2024-06-10T02:42:05.920 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:42:05 smithi052 ceph-mon[32362]: from='client.? 172.21.15.52:0/302683026' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-06-10T02:42:06.217 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout 2024-06-10T02:42:06.217 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout { 2024-06-10T02:42:06.217 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "fsid": "d8676e9e-26d2-11ef-bc9b-c7b262605968", 2024-06-10T02:42:06.217 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "health": { 2024-06-10T02:42:06.217 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-06-10T02:42:06.217 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-06-10T02:42:06.217 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-06-10T02:42:06.217 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout }, 2024-06-10T02:42:06.217 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-06-10T02:42:06.217 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-06-10T02:42:06.218 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout 0 2024-06-10T02:42:06.218 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout ], 2024-06-10T02:42:06.218 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-06-10T02:42:06.218 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "a" 2024-06-10T02:42:06.218 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout ], 2024-06-10T02:42:06.218 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "quorum_age": 17, 2024-06-10T02:42:06.218 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "monmap": { 2024-06-10T02:42:06.218 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-10T02:42:06.218 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-06-10T02:42:06.218 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-06-10T02:42:06.218 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout }, 2024-06-10T02:42:06.218 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-06-10T02:42:06.219 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-10T02:42:06.219 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-06-10T02:42:06.219 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-06-10T02:42:06.219 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-06-10T02:42:06.228 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-06-10T02:42:06.229 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-06-10T02:42:06.229 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-06-10T02:42:06.229 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout }, 2024-06-10T02:42:06.229 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-06-10T02:42:06.229 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-06-10T02:42:06.229 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-06-10T02:42:06.229 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-06-10T02:42:06.229 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-06-10T02:42:06.229 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-06-10T02:42:06.229 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-06-10T02:42:06.229 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-06-10T02:42:06.230 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-06-10T02:42:06.230 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout }, 2024-06-10T02:42:06.230 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-06-10T02:42:06.230 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-10T02:42:06.230 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-06-10T02:42:06.230 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-06-10T02:42:06.230 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout }, 2024-06-10T02:42:06.230 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-06-10T02:42:06.230 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "available": true, 2024-06-10T02:42:06.230 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-06-10T02:42:06.231 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "modules": [ 2024-06-10T02:42:06.231 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "iostat", 2024-06-10T02:42:06.231 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "nfs", 2024-06-10T02:42:06.231 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "restful" 2024-06-10T02:42:06.231 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout ], 2024-06-10T02:42:06.231 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "services": {} 2024-06-10T02:42:06.231 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout }, 2024-06-10T02:42:06.231 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-06-10T02:42:06.231 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-10T02:42:06.231 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "modified": "2024-06-10T02:41:43.077320+0000", 2024-06-10T02:42:06.231 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "services": {} 2024-06-10T02:42:06.231 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout }, 2024-06-10T02:42:06.231 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-06-10T02:42:06.231 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout } 2024-06-10T02:42:06.232 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:05.315+0000 7fd5e8752700 1 Processor -- start 2024-06-10T02:42:06.232 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:05.315+0000 7fd5e8752700 1 -- start start 2024-06-10T02:42:06.232 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:05.316+0000 7fd5e8752700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd5e0105d00 0x7fd5e0106120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:42:06.232 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:05.316+0000 7fd5e8752700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fd5e0106660 con 0x7fd5e0105d00 2024-06-10T02:42:06.232 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:05.316+0000 7fd5e64ee700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd5e0105d00 0x7fd5e0106120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:42:06.232 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:05.316+0000 7fd5e64ee700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd5e0105d00 0x7fd5e0106120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52314/0 (socket says 172.21.15.52:52314) 2024-06-10T02:42:06.232 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:05.316+0000 7fd5e64ee700 1 -- 172.21.15.52:0/3506637941 learned_addr learned my addr 172.21.15.52:0/3506637941 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:42:06.232 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:05.316+0000 7fd5e64ee700 1 -- 172.21.15.52:0/3506637941 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5e01067e0 con 0x7fd5e0105d00 2024-06-10T02:42:06.232 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:05.316+0000 7fd5e64ee700 1 --2- 172.21.15.52:0/3506637941 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd5e0105d00 0x7fd5e0106120 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fd5dc0098a0 tx=0x7fd5dc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5bb46583d251b38 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:42:06.232 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:05.317+0000 7fd5e54ec700 1 -- 172.21.15.52:0/3506637941 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd5dc004160 con 0x7fd5e0105d00 2024-06-10T02:42:06.232 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:05.317+0000 7fd5e54ec700 1 -- 172.21.15.52:0/3506637941 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd5dc004300 con 0x7fd5e0105d00 2024-06-10T02:42:06.232 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:05.317+0000 7fd5e54ec700 1 -- 172.21.15.52:0/3506637941 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd5dc011b50 con 0x7fd5e0105d00 2024-06-10T02:42:06.233 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:05.317+0000 7fd5e8752700 1 -- 172.21.15.52:0/3506637941 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd5e0105d00 msgr2=0x7fd5e0106120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:42:06.234 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:05.317+0000 7fd5e8752700 1 --2- 172.21.15.52:0/3506637941 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd5e0105d00 0x7fd5e0106120 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fd5dc0098a0 tx=0x7fd5dc009bb0 comp rx=0 tx=0).stop 2024-06-10T02:42:06.234 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:05.317+0000 7fd5e8752700 1 -- 172.21.15.52:0/3506637941 shutdown_connections 2024-06-10T02:42:06.234 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:05.317+0000 7fd5e8752700 1 --2- 172.21.15.52:0/3506637941 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd5e0105d00 0x7fd5e0106120 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:42:06.234 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:05.317+0000 7fd5e8752700 1 -- 172.21.15.52:0/3506637941 >> 172.21.15.52:0/3506637941 conn(0x7fd5e0101740 msgr2=0x7fd5e0103ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:42:06.235 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:05.317+0000 7fd5e8752700 1 -- 172.21.15.52:0/3506637941 shutdown_connections 2024-06-10T02:42:06.235 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:05.317+0000 7fd5e8752700 1 -- 172.21.15.52:0/3506637941 wait complete. 2024-06-10T02:42:06.235 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:05.318+0000 7fd5e8752700 1 Processor -- start 2024-06-10T02:42:06.235 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:05.318+0000 7fd5e8752700 1 -- start start 2024-06-10T02:42:06.235 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:05.318+0000 7fd5e8752700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd5e0105d00 0x7fd5e019e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:42:06.235 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:05.318+0000 7fd5e8752700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fd5e010c3a0 con 0x7fd5e0105d00 2024-06-10T02:42:06.235 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:05.318+0000 7fd5e64ee700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd5e0105d00 0x7fd5e019e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:42:06.235 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:05.318+0000 7fd5e64ee700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd5e0105d00 0x7fd5e019e1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52316/0 (socket says 172.21.15.52:52316) 2024-06-10T02:42:06.235 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:05.319+0000 7fd5e64ee700 1 -- 172.21.15.52:0/302683026 learned_addr learned my addr 172.21.15.52:0/302683026 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:42:06.235 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:05.319+0000 7fd5e64ee700 1 -- 172.21.15.52:0/302683026 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5e019e6f0 con 0x7fd5e0105d00 2024-06-10T02:42:06.235 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:05.319+0000 7fd5e64ee700 1 --2- 172.21.15.52:0/302683026 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd5e0105d00 0x7fd5e019e1b0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fd5dc011b60 tx=0x7fd5dc011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:42:06.235 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:05.319+0000 7fd5d77fe700 1 -- 172.21.15.52:0/302683026 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd5dc003970 con 0x7fd5e0105d00 2024-06-10T02:42:06.236 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:05.319+0000 7fd5d77fe700 1 -- 172.21.15.52:0/302683026 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd5dc003b10 con 0x7fd5e0105d00 2024-06-10T02:42:06.236 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:05.319+0000 7fd5e8752700 1 -- 172.21.15.52:0/302683026 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5e019e980 con 0x7fd5e0105d00 2024-06-10T02:42:06.236 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:05.319+0000 7fd5d77fe700 1 -- 172.21.15.52:0/302683026 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd5dc024610 con 0x7fd5e0105d00 2024-06-10T02:42:06.236 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:05.319+0000 7fd5e8752700 1 -- 172.21.15.52:0/302683026 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5e01a1670 con 0x7fd5e0105d00 2024-06-10T02:42:06.236 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:05.320+0000 7fd5d77fe700 1 -- 172.21.15.52:0/302683026 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7fd5dc01a430 con 0x7fd5e0105d00 2024-06-10T02:42:06.236 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:05.320+0000 7fd5e8752700 1 -- 172.21.15.52:0/302683026 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5e0063af0 con 0x7fd5e0105d00 2024-06-10T02:42:06.236 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:05.320+0000 7fd5d77fe700 1 --2- 172.21.15.52:0/302683026 >> [v2:172.21.15.52:6800/3934479642,v1:172.21.15.52:6801/3934479642] conn(0x7fd5cc039050 0x7fd5cc03b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:42:06.236 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:05.320+0000 7fd5d77fe700 1 -- 172.21.15.52:0/302683026 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fd5dc011e00 con 0x7fd5e0105d00 2024-06-10T02:42:06.236 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:05.320+0000 7fd5e5ced700 1 --2- 172.21.15.52:0/302683026 >> [v2:172.21.15.52:6800/3934479642,v1:172.21.15.52:6801/3934479642] conn(0x7fd5cc039050 0x7fd5cc03b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:42:06.236 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:05.321+0000 7fd5e5ced700 1 --2- 172.21.15.52:0/302683026 >> [v2:172.21.15.52:6800/3934479642,v1:172.21.15.52:6801/3934479642] conn(0x7fd5cc039050 0x7fd5cc03b510 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fd5d0006fd0 tx=0x7fd5d0006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:42:06.236 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:05.326+0000 7fd5d77fe700 1 -- 172.21.15.52:0/302683026 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fd5dc0248c0 con 0x7fd5e0105d00 2024-06-10T02:42:06.236 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:05.618+0000 7fd5e8752700 1 -- 172.21.15.52:0/302683026 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fd5e01a1cb0 con 0x7fd5e0105d00 2024-06-10T02:42:06.237 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:05.618+0000 7fd5d77fe700 1 -- 172.21.15.52:0/302683026 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7fd5dc01a9f0 con 0x7fd5e0105d00 2024-06-10T02:42:06.237 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:05.621+0000 7fd5e8752700 1 -- 172.21.15.52:0/302683026 >> [v2:172.21.15.52:6800/3934479642,v1:172.21.15.52:6801/3934479642] conn(0x7fd5cc039050 msgr2=0x7fd5cc03b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:42:06.237 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:05.621+0000 7fd5e8752700 1 --2- 172.21.15.52:0/302683026 >> [v2:172.21.15.52:6800/3934479642,v1:172.21.15.52:6801/3934479642] conn(0x7fd5cc039050 0x7fd5cc03b510 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fd5d0006fd0 tx=0x7fd5d0006e40 comp rx=0 tx=0).stop 2024-06-10T02:42:06.237 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:05.621+0000 7fd5e8752700 1 -- 172.21.15.52:0/302683026 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd5e0105d00 msgr2=0x7fd5e019e1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:42:06.237 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:05.621+0000 7fd5e8752700 1 --2- 172.21.15.52:0/302683026 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd5e0105d00 0x7fd5e019e1b0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fd5dc011b60 tx=0x7fd5dc011c40 comp rx=0 tx=0).stop 2024-06-10T02:42:06.237 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:05.621+0000 7fd5e8752700 1 -- 172.21.15.52:0/302683026 shutdown_connections 2024-06-10T02:42:06.237 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:05.621+0000 7fd5e8752700 1 --2- 172.21.15.52:0/302683026 >> [v2:172.21.15.52:6800/3934479642,v1:172.21.15.52:6801/3934479642] conn(0x7fd5cc039050 0x7fd5cc03b510 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:42:06.237 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:05.621+0000 7fd5e8752700 1 --2- 172.21.15.52:0/302683026 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd5e0105d00 0x7fd5e019e1b0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:42:06.237 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:05.621+0000 7fd5e8752700 1 -- 172.21.15.52:0/302683026 >> 172.21.15.52:0/302683026 conn(0x7fd5e0101740 msgr2=0x7fd5e0102250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:42:06.237 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:05.621+0000 7fd5e8752700 1 -- 172.21.15.52:0/302683026 shutdown_connections 2024-06-10T02:42:06.237 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:05.621+0000 7fd5e8752700 1 -- 172.21.15.52:0/302683026 wait complete. 2024-06-10T02:42:06.237 INFO:teuthology.orchestra.run.smithi052.stdout:mgr is available 2024-06-10T02:42:07.473 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:42:07 smithi052 ceph-mon[32362]: from='client.? 172.21.15.52:0/2153285541' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-06-10T02:42:07.571 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout 2024-06-10T02:42:07.571 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout [global] 2024-06-10T02:42:07.571 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout fsid = d8676e9e-26d2-11ef-bc9b-c7b262605968 2024-06-10T02:42:07.571 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-06-10T02:42:07.571 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-06-10T02:42:07.571 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-06-10T02:42:07.571 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-06-10T02:42:07.571 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout 2024-06-10T02:42:07.572 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout [mgr] 2024-06-10T02:42:07.572 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-06-10T02:42:07.572 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout 2024-06-10T02:42:07.572 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout [osd] 2024-06-10T02:42:07.572 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-06-10T02:42:07.572 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-06-10T02:42:07.572 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-06-10T02:42:07.572 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-06-10T02:42:07.572 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-06-10T02:42:07.572 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:06.837+0000 7fbd640fb700 1 Processor -- start 2024-06-10T02:42:07.572 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:06.837+0000 7fbd640fb700 1 -- start start 2024-06-10T02:42:07.573 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:06.837+0000 7fbd640fb700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbd5c107fc0 0x7fbd5c1083e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:42:07.573 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:06.837+0000 7fbd640fb700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fbd5c1089b0 con 0x7fbd5c107fc0 2024-06-10T02:42:07.573 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:06.837+0000 7fbd61e97700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbd5c107fc0 0x7fbd5c1083e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:42:07.573 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:06.837+0000 7fbd61e97700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbd5c107fc0 0x7fbd5c1083e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52320/0 (socket says 172.21.15.52:52320) 2024-06-10T02:42:07.573 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:06.838+0000 7fbd61e97700 1 -- 172.21.15.52:0/970453910 learned_addr learned my addr 172.21.15.52:0/970453910 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:42:07.573 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:06.838+0000 7fbd61e97700 1 -- 172.21.15.52:0/970453910 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd5c109170 con 0x7fbd5c107fc0 2024-06-10T02:42:07.573 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:06.838+0000 7fbd61e97700 1 --2- 172.21.15.52:0/970453910 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbd5c107fc0 0x7fbd5c1083e0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fbd580098a0 tx=0x7fbd58009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bb306edde4d91ed server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:42:07.573 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:06.838+0000 7fbd60e95700 1 -- 172.21.15.52:0/970453910 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbd58004160 con 0x7fbd5c107fc0 2024-06-10T02:42:07.573 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:06.838+0000 7fbd60e95700 1 -- 172.21.15.52:0/970453910 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fbd58004300 con 0x7fbd5c107fc0 2024-06-10T02:42:07.573 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:06.838+0000 7fbd60e95700 1 -- 172.21.15.52:0/970453910 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbd58011b50 con 0x7fbd5c107fc0 2024-06-10T02:42:07.573 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:06.839+0000 7fbd640fb700 1 -- 172.21.15.52:0/970453910 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbd5c107fc0 msgr2=0x7fbd5c1083e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:42:07.573 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:06.839+0000 7fbd640fb700 1 --2- 172.21.15.52:0/970453910 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbd5c107fc0 0x7fbd5c1083e0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fbd580098a0 tx=0x7fbd58009bb0 comp rx=0 tx=0).stop 2024-06-10T02:42:07.574 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:06.839+0000 7fbd640fb700 1 -- 172.21.15.52:0/970453910 shutdown_connections 2024-06-10T02:42:07.574 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:06.839+0000 7fbd640fb700 1 --2- 172.21.15.52:0/970453910 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbd5c107fc0 0x7fbd5c1083e0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:42:07.574 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:06.839+0000 7fbd640fb700 1 -- 172.21.15.52:0/970453910 >> 172.21.15.52:0/970453910 conn(0x7fbd5c103560 msgr2=0x7fbd5c1059a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:42:07.574 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:06.839+0000 7fbd640fb700 1 -- 172.21.15.52:0/970453910 shutdown_connections 2024-06-10T02:42:07.574 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:06.839+0000 7fbd640fb700 1 -- 172.21.15.52:0/970453910 wait complete. 2024-06-10T02:42:07.574 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:06.839+0000 7fbd640fb700 1 Processor -- start 2024-06-10T02:42:07.574 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:06.840+0000 7fbd640fb700 1 -- start start 2024-06-10T02:42:07.574 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:06.840+0000 7fbd640fb700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbd5c107fc0 0x7fbd5c19bf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:42:07.574 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:06.840+0000 7fbd640fb700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fbd5c10c520 con 0x7fbd5c107fc0 2024-06-10T02:42:07.574 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:06.840+0000 7fbd61e97700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbd5c107fc0 0x7fbd5c19bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:42:07.574 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:06.840+0000 7fbd61e97700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbd5c107fc0 0x7fbd5c19bf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52330/0 (socket says 172.21.15.52:52330) 2024-06-10T02:42:07.575 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:06.840+0000 7fbd61e97700 1 -- 172.21.15.52:0/2153285541 learned_addr learned my addr 172.21.15.52:0/2153285541 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:42:07.575 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:06.840+0000 7fbd61e97700 1 -- 172.21.15.52:0/2153285541 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd5c19c4a0 con 0x7fbd5c107fc0 2024-06-10T02:42:07.575 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:06.840+0000 7fbd61e97700 1 --2- 172.21.15.52:0/2153285541 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbd5c107fc0 0x7fbd5c19bf60 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fbd58000c00 tx=0x7fbd58011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:42:07.575 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:06.840+0000 7fbd52ffd700 1 -- 172.21.15.52:0/2153285541 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbd58003b20 con 0x7fbd5c107fc0 2024-06-10T02:42:07.575 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:06.840+0000 7fbd52ffd700 1 -- 172.21.15.52:0/2153285541 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fbd58003cc0 con 0x7fbd5c107fc0 2024-06-10T02:42:07.575 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:06.840+0000 7fbd640fb700 1 -- 172.21.15.52:0/2153285541 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd5c19c730 con 0x7fbd5c107fc0 2024-06-10T02:42:07.575 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:06.841+0000 7fbd52ffd700 1 -- 172.21.15.52:0/2153285541 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbd58024910 con 0x7fbd5c107fc0 2024-06-10T02:42:07.575 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:06.841+0000 7fbd640fb700 1 -- 172.21.15.52:0/2153285541 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd5c19f420 con 0x7fbd5c107fc0 2024-06-10T02:42:07.575 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:06.841+0000 7fbd52ffd700 1 -- 172.21.15.52:0/2153285541 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7fbd58022020 con 0x7fbd5c107fc0 2024-06-10T02:42:07.575 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:06.842+0000 7fbd640fb700 1 -- 172.21.15.52:0/2153285541 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd5c063af0 con 0x7fbd5c107fc0 2024-06-10T02:42:07.575 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:06.842+0000 7fbd52ffd700 1 --2- 172.21.15.52:0/2153285541 >> [v2:172.21.15.52:6800/3934479642,v1:172.21.15.52:6801/3934479642] conn(0x7fbd48039050 0x7fbd4803b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:42:07.576 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:06.842+0000 7fbd52ffd700 1 -- 172.21.15.52:0/2153285541 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fbd58011e00 con 0x7fbd5c107fc0 2024-06-10T02:42:07.576 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:06.842+0000 7fbd61696700 1 --2- 172.21.15.52:0/2153285541 >> [v2:172.21.15.52:6800/3934479642,v1:172.21.15.52:6801/3934479642] conn(0x7fbd48039050 0x7fbd4803b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:42:07.576 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:06.842+0000 7fbd61696700 1 --2- 172.21.15.52:0/2153285541 >> [v2:172.21.15.52:6800/3934479642,v1:172.21.15.52:6801/3934479642] conn(0x7fbd48039050 0x7fbd4803b510 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fbd4c006fd0 tx=0x7fbd4c006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:42:07.576 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:06.847+0000 7fbd52ffd700 1 -- 172.21.15.52:0/2153285541 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fbd5804ca70 con 0x7fbd5c107fc0 2024-06-10T02:42:07.576 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:07.051+0000 7fbd640fb700 1 -- 172.21.15.52:0/2153285541 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7fbd5c19f640 con 0x7fbd5c107fc0 2024-06-10T02:42:07.576 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:07.052+0000 7fbd52ffd700 1 -- 172.21.15.52:0/2153285541 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+373 (secure 0 0 0) 0x7fbd5c19f640 con 0x7fbd5c107fc0 2024-06-10T02:42:07.576 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:07.054+0000 7fbd640fb700 1 -- 172.21.15.52:0/2153285541 >> [v2:172.21.15.52:6800/3934479642,v1:172.21.15.52:6801/3934479642] conn(0x7fbd48039050 msgr2=0x7fbd4803b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:42:07.576 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:07.054+0000 7fbd640fb700 1 --2- 172.21.15.52:0/2153285541 >> [v2:172.21.15.52:6800/3934479642,v1:172.21.15.52:6801/3934479642] conn(0x7fbd48039050 0x7fbd4803b510 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fbd4c006fd0 tx=0x7fbd4c006e40 comp rx=0 tx=0).stop 2024-06-10T02:42:07.576 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:07.054+0000 7fbd640fb700 1 -- 172.21.15.52:0/2153285541 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbd5c107fc0 msgr2=0x7fbd5c19bf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:42:07.576 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:07.054+0000 7fbd640fb700 1 --2- 172.21.15.52:0/2153285541 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbd5c107fc0 0x7fbd5c19bf60 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fbd58000c00 tx=0x7fbd58011c10 comp rx=0 tx=0).stop 2024-06-10T02:42:07.576 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:07.054+0000 7fbd640fb700 1 -- 172.21.15.52:0/2153285541 shutdown_connections 2024-06-10T02:42:07.577 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:07.054+0000 7fbd640fb700 1 --2- 172.21.15.52:0/2153285541 >> [v2:172.21.15.52:6800/3934479642,v1:172.21.15.52:6801/3934479642] conn(0x7fbd48039050 0x7fbd4803b510 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:42:07.577 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:07.054+0000 7fbd640fb700 1 --2- 172.21.15.52:0/2153285541 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbd5c107fc0 0x7fbd5c19bf60 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:42:07.577 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:07.054+0000 7fbd640fb700 1 -- 172.21.15.52:0/2153285541 >> 172.21.15.52:0/2153285541 conn(0x7fbd5c103560 msgr2=0x7fbd5c104050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:42:07.577 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:07.054+0000 7fbd640fb700 1 -- 172.21.15.52:0/2153285541 shutdown_connections 2024-06-10T02:42:07.577 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:07.054+0000 7fbd640fb700 1 -- 172.21.15.52:0/2153285541 wait complete. 2024-06-10T02:42:07.577 INFO:teuthology.orchestra.run.smithi052.stdout:Enabling cephadm module... 2024-06-10T02:42:08.660 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:42:08 smithi052 ceph-mon[32362]: from='client.? 172.21.15.52:0/3829932776' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-06-10T02:42:08.661 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 10 02:42:08 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mgr-x[32595]: ignoring --setuser ceph since I am not root 2024-06-10T02:42:08.661 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 10 02:42:08 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mgr-x[32595]: ignoring --setgroup ceph since I am not root 2024-06-10T02:42:08.661 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 10 02:42:08 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mgr-x[32595]: 2024-06-10T02:42:08.473+0000 7fa1b9835200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-06-10T02:42:08.661 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 10 02:42:08 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mgr-x[32595]: 2024-06-10T02:42:08.663+0000 7fa1b9835200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-06-10T02:42:08.972 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 10 02:42:08 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mgr-x[32595]: 2024-06-10T02:42:08.771+0000 7fa1b9835200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-06-10T02:42:09.416 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:42:09 smithi052 ceph-mon[32362]: from='client.? 172.21.15.52:0/3829932776' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-06-10T02:42:09.416 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:42:09 smithi052 ceph-mon[32362]: mgrmap e5: x(active, since 11s) 2024-06-10T02:42:09.417 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 10 02:42:09 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mgr-x[32595]: 2024-06-10T02:42:09.093+0000 7fa1b9835200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-06-10T02:42:09.417 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 10 02:42:09 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mgr-x[32595]: 2024-06-10T02:42:09.182+0000 7fa1b9835200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-06-10T02:42:09.424 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:07.968+0000 7f9781372700 1 Processor -- start 2024-06-10T02:42:09.424 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:07.968+0000 7f9781372700 1 -- start start 2024-06-10T02:42:09.424 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:07.969+0000 7f9781372700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f977c107fc0 0x7f977c1083e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:42:09.424 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:07.969+0000 7f9781372700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f977c1089b0 con 0x7f977c107fc0 2024-06-10T02:42:09.424 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:07.969+0000 7f977affd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f977c107fc0 0x7f977c1083e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:42:09.424 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:07.969+0000 7f977affd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f977c107fc0 0x7f977c1083e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52332/0 (socket says 172.21.15.52:52332) 2024-06-10T02:42:09.424 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:07.969+0000 7f977affd700 1 -- 172.21.15.52:0/2105796319 learned_addr learned my addr 172.21.15.52:0/2105796319 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:42:09.425 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:07.969+0000 7f977affd700 1 -- 172.21.15.52:0/2105796319 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f977c109170 con 0x7f977c107fc0 2024-06-10T02:42:09.425 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:07.969+0000 7f977affd700 1 --2- 172.21.15.52:0/2105796319 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f977c107fc0 0x7f977c1083e0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f976c009e80 tx=0x7f976c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dac58f28d560cdbf server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:42:09.425 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:07.970+0000 7f9779ffb700 1 -- 172.21.15.52:0/2105796319 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f976c017070 con 0x7f977c107fc0 2024-06-10T02:42:09.425 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:07.970+0000 7f9779ffb700 1 -- 172.21.15.52:0/2105796319 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f976c00b8c0 con 0x7f977c107fc0 2024-06-10T02:42:09.425 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:07.970+0000 7f9779ffb700 1 -- 172.21.15.52:0/2105796319 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f976c0045c0 con 0x7f977c107fc0 2024-06-10T02:42:09.425 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:07.970+0000 7f9781372700 1 -- 172.21.15.52:0/2105796319 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f977c107fc0 msgr2=0x7f977c1083e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:42:09.425 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:07.970+0000 7f9781372700 1 --2- 172.21.15.52:0/2105796319 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f977c107fc0 0x7f977c1083e0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f976c009e80 tx=0x7f976c00b330 comp rx=0 tx=0).stop 2024-06-10T02:42:09.425 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:07.970+0000 7f9781372700 1 -- 172.21.15.52:0/2105796319 shutdown_connections 2024-06-10T02:42:09.425 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:07.970+0000 7f9781372700 1 --2- 172.21.15.52:0/2105796319 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f977c107fc0 0x7f977c1083e0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:42:09.425 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:07.970+0000 7f9781372700 1 -- 172.21.15.52:0/2105796319 >> 172.21.15.52:0/2105796319 conn(0x7f977c103560 msgr2=0x7f977c1059a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:42:09.425 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:07.970+0000 7f9781372700 1 -- 172.21.15.52:0/2105796319 shutdown_connections 2024-06-10T02:42:09.425 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:07.970+0000 7f9781372700 1 -- 172.21.15.52:0/2105796319 wait complete. 2024-06-10T02:42:09.426 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:07.971+0000 7f9781372700 1 Processor -- start 2024-06-10T02:42:09.426 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:07.971+0000 7f9781372700 1 -- start start 2024-06-10T02:42:09.426 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:07.971+0000 7f9781372700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f977c107fc0 0x7f977c07e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:42:09.426 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:07.971+0000 7f9781372700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f977c10c520 con 0x7f977c107fc0 2024-06-10T02:42:09.426 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:07.971+0000 7f977affd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f977c107fc0 0x7f977c07e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:42:09.426 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:07.971+0000 7f977affd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f977c107fc0 0x7f977c07e410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52342/0 (socket says 172.21.15.52:52342) 2024-06-10T02:42:09.426 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:07.971+0000 7f977affd700 1 -- 172.21.15.52:0/3829932776 learned_addr learned my addr 172.21.15.52:0/3829932776 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:42:09.426 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:07.972+0000 7f977affd700 1 -- 172.21.15.52:0/3829932776 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f977c07e950 con 0x7f977c107fc0 2024-06-10T02:42:09.426 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:07.972+0000 7f977affd700 1 --2- 172.21.15.52:0/3829932776 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f977c107fc0 0x7f977c07e410 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f976c009530 tx=0x7f976c00bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:42:09.426 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:07.972+0000 7f9773fff700 1 -- 172.21.15.52:0/3829932776 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f976c020020 con 0x7f977c107fc0 2024-06-10T02:42:09.426 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:07.972+0000 7f9773fff700 1 -- 172.21.15.52:0/3829932776 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f976c003b50 con 0x7f977c107fc0 2024-06-10T02:42:09.426 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:07.972+0000 7f9781372700 1 -- 172.21.15.52:0/3829932776 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f977c07cb40 con 0x7f977c107fc0 2024-06-10T02:42:09.427 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:07.972+0000 7f9773fff700 1 -- 172.21.15.52:0/3829932776 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f976c017040 con 0x7f977c107fc0 2024-06-10T02:42:09.427 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:07.972+0000 7f9781372700 1 -- 172.21.15.52:0/3829932776 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f977c07d080 con 0x7f977c107fc0 2024-06-10T02:42:09.427 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:07.973+0000 7f9773fff700 1 -- 172.21.15.52:0/3829932776 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7f976c023020 con 0x7f977c107fc0 2024-06-10T02:42:09.427 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:07.973+0000 7f9773fff700 1 --2- 172.21.15.52:0/3829932776 >> [v2:172.21.15.52:6800/3934479642,v1:172.21.15.52:6801/3934479642] conn(0x7f9768039050 0x7f976803b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:42:09.427 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:07.973+0000 7f9773fff700 1 -- 172.21.15.52:0/3829932776 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f976c00f050 con 0x7f977c107fc0 2024-06-10T02:42:09.427 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:07.973+0000 7f9781372700 1 -- 172.21.15.52:0/3829932776 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f977c063af0 con 0x7f977c107fc0 2024-06-10T02:42:09.427 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:07.973+0000 7f977a7fc700 1 --2- 172.21.15.52:0/3829932776 >> [v2:172.21.15.52:6800/3934479642,v1:172.21.15.52:6801/3934479642] conn(0x7f9768039050 0x7f976803b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:42:09.427 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:07.974+0000 7f977a7fc700 1 --2- 172.21.15.52:0/3829932776 >> [v2:172.21.15.52:6800/3934479642,v1:172.21.15.52:6801/3934479642] conn(0x7f9768039050 0x7f976803b510 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9764006fd0 tx=0x7f9764006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:42:09.427 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:07.978+0000 7f9773fff700 1 -- 172.21.15.52:0/3829932776 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f976c02a020 con 0x7f977c107fc0 2024-06-10T02:42:09.427 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:08.243+0000 7f9781372700 1 -- 172.21.15.52:0/3829932776 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7f977c07d250 con 0x7f977c107fc0 2024-06-10T02:42:09.427 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:08.295+0000 7f9773fff700 1 -- 172.21.15.52:0/3829932776 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7f976c04c980 con 0x7f977c107fc0 2024-06-10T02:42:09.427 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:08.296+0000 7f9773fff700 1 -- 172.21.15.52:0/3829932776 <== mon.0 v2:172.21.15.52:3300/0 8 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7f976c025400 con 0x7f977c107fc0 2024-06-10T02:42:09.427 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:08.298+0000 7f9781372700 1 -- 172.21.15.52:0/3829932776 >> [v2:172.21.15.52:6800/3934479642,v1:172.21.15.52:6801/3934479642] conn(0x7f9768039050 msgr2=0x7f976803b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:42:09.428 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:08.298+0000 7f9781372700 1 --2- 172.21.15.52:0/3829932776 >> [v2:172.21.15.52:6800/3934479642,v1:172.21.15.52:6801/3934479642] conn(0x7f9768039050 0x7f976803b510 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9764006fd0 tx=0x7f9764006e40 comp rx=0 tx=0).stop 2024-06-10T02:42:09.428 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:08.298+0000 7f9781372700 1 -- 172.21.15.52:0/3829932776 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f977c107fc0 msgr2=0x7f977c07e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:42:09.428 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:08.298+0000 7f9781372700 1 --2- 172.21.15.52:0/3829932776 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f977c107fc0 0x7f977c07e410 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f976c009530 tx=0x7f976c00bd50 comp rx=0 tx=0).stop 2024-06-10T02:42:09.428 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:08.298+0000 7f9781372700 1 -- 172.21.15.52:0/3829932776 shutdown_connections 2024-06-10T02:42:09.428 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:08.298+0000 7f9781372700 1 --2- 172.21.15.52:0/3829932776 >> [v2:172.21.15.52:6800/3934479642,v1:172.21.15.52:6801/3934479642] conn(0x7f9768039050 0x7f976803b510 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:42:09.428 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:08.298+0000 7f9781372700 1 --2- 172.21.15.52:0/3829932776 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f977c107fc0 0x7f977c07e410 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:42:09.428 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:08.298+0000 7f9781372700 1 -- 172.21.15.52:0/3829932776 >> 172.21.15.52:0/3829932776 conn(0x7f977c103560 msgr2=0x7f977c103fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:42:09.428 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:08.298+0000 7f9781372700 1 -- 172.21.15.52:0/3829932776 shutdown_connections 2024-06-10T02:42:09.428 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:08.298+0000 7f9781372700 1 -- 172.21.15.52:0/3829932776 wait complete. 2024-06-10T02:42:10.452 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:42:10 smithi052 ceph-mon[32362]: from='client.? 172.21.15.52:0/2285227698' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-06-10T02:42:10.706 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 10 02:42:10 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mgr-x[32595]: 2024-06-10T02:42:10.455+0000 7fa1b9835200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-06-10T02:42:10.706 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 10 02:42:10 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mgr-x[32595]: 2024-06-10T02:42:10.632+0000 7fa1b9835200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-06-10T02:42:10.742 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout { 2024-06-10T02:42:10.742 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-06-10T02:42:10.742 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "available": true, 2024-06-10T02:42:10.742 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-06-10T02:42:10.742 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-06-10T02:42:10.743 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout } 2024-06-10T02:42:10.743 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:09.851+0000 7f1927f09700 1 Processor -- start 2024-06-10T02:42:10.743 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:09.851+0000 7f1927f09700 1 -- start start 2024-06-10T02:42:10.743 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:09.851+0000 7f1927f09700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f19201062a0 0x7f19201066c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:42:10.743 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:09.851+0000 7f1927f09700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f1920106c90 con 0x7f19201062a0 2024-06-10T02:42:10.743 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:09.851+0000 7f1925ca5700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f19201062a0 0x7f19201066c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:42:10.743 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:09.851+0000 7f1925ca5700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f19201062a0 0x7f19201066c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52360/0 (socket says 172.21.15.52:52360) 2024-06-10T02:42:10.743 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:09.851+0000 7f1925ca5700 1 -- 172.21.15.52:0/280004126 learned_addr learned my addr 172.21.15.52:0/280004126 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:42:10.743 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:09.852+0000 7f1925ca5700 1 -- 172.21.15.52:0/280004126 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1920109230 con 0x7f19201062a0 2024-06-10T02:42:10.743 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:09.852+0000 7f1925ca5700 1 --2- 172.21.15.52:0/280004126 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f19201062a0 0x7f19201066c0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f191c0098a0 tx=0x7f191c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d728e8c420f8a62 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:42:10.743 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:09.852+0000 7f1924ca3700 1 -- 172.21.15.52:0/280004126 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f191c004160 con 0x7f19201062a0 2024-06-10T02:42:10.743 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:09.852+0000 7f1924ca3700 1 -- 172.21.15.52:0/280004126 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f191c004300 con 0x7f19201062a0 2024-06-10T02:42:10.743 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:09.852+0000 7f1924ca3700 1 -- 172.21.15.52:0/280004126 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f191c011b50 con 0x7f19201062a0 2024-06-10T02:42:10.744 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:09.853+0000 7f1927f09700 1 -- 172.21.15.52:0/280004126 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f19201062a0 msgr2=0x7f19201066c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:42:10.744 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:09.853+0000 7f1927f09700 1 --2- 172.21.15.52:0/280004126 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f19201062a0 0x7f19201066c0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f191c0098a0 tx=0x7f191c009bb0 comp rx=0 tx=0).stop 2024-06-10T02:42:10.744 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:09.853+0000 7f1927f09700 1 -- 172.21.15.52:0/280004126 shutdown_connections 2024-06-10T02:42:10.744 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:09.853+0000 7f1927f09700 1 --2- 172.21.15.52:0/280004126 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f19201062a0 0x7f19201066c0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:42:10.744 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:09.853+0000 7f1927f09700 1 -- 172.21.15.52:0/280004126 >> 172.21.15.52:0/280004126 conn(0x7f1920101bd0 msgr2=0x7f1920104010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:42:10.744 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:09.853+0000 7f1927f09700 1 -- 172.21.15.52:0/280004126 shutdown_connections 2024-06-10T02:42:10.744 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:09.853+0000 7f1927f09700 1 -- 172.21.15.52:0/280004126 wait complete. 2024-06-10T02:42:10.745 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:09.853+0000 7f1927f09700 1 Processor -- start 2024-06-10T02:42:10.745 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:09.853+0000 7f1927f09700 1 -- start start 2024-06-10T02:42:10.745 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:09.854+0000 7f1927f09700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f19201062a0 0x7f192019bef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:42:10.745 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:09.854+0000 7f1927f09700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f192010c5e0 con 0x7f19201062a0 2024-06-10T02:42:10.745 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:09.854+0000 7f1925ca5700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f19201062a0 0x7f192019bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:42:10.746 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:09.854+0000 7f1925ca5700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f19201062a0 0x7f192019bef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52364/0 (socket says 172.21.15.52:52364) 2024-06-10T02:42:10.746 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:09.854+0000 7f1925ca5700 1 -- 172.21.15.52:0/2285227698 learned_addr learned my addr 172.21.15.52:0/2285227698 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:42:10.746 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:09.854+0000 7f1925ca5700 1 -- 172.21.15.52:0/2285227698 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f192019c430 con 0x7f19201062a0 2024-06-10T02:42:10.746 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:09.854+0000 7f1925ca5700 1 --2- 172.21.15.52:0/2285227698 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f19201062a0 0x7f192019bef0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f191c000c00 tx=0x7f191c011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:42:10.746 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:09.854+0000 7f1916ffd700 1 -- 172.21.15.52:0/2285227698 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f191c003b20 con 0x7f19201062a0 2024-06-10T02:42:10.747 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:09.854+0000 7f1916ffd700 1 -- 172.21.15.52:0/2285227698 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f191c003cc0 con 0x7f19201062a0 2024-06-10T02:42:10.747 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:09.854+0000 7f1927f09700 1 -- 172.21.15.52:0/2285227698 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f192019c6c0 con 0x7f19201062a0 2024-06-10T02:42:10.747 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:09.854+0000 7f1916ffd700 1 -- 172.21.15.52:0/2285227698 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f191c024910 con 0x7f19201062a0 2024-06-10T02:42:10.747 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:09.855+0000 7f1927f09700 1 -- 172.21.15.52:0/2285227698 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f192019f3b0 con 0x7f19201062a0 2024-06-10T02:42:10.747 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:09.855+0000 7f1927f09700 1 -- 172.21.15.52:0/2285227698 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1920063af0 con 0x7f19201062a0 2024-06-10T02:42:10.747 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:09.856+0000 7f1916ffd700 1 -- 172.21.15.52:0/2285227698 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7f191c022020 con 0x7f19201062a0 2024-06-10T02:42:10.747 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:09.856+0000 7f1916ffd700 1 --2- 172.21.15.52:0/2285227698 >> [v2:172.21.15.52:6800/3934479642,v1:172.21.15.52:6801/3934479642] conn(0x7f190c0390a0 0x7f190c03b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:42:10.748 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:09.856+0000 7f1916ffd700 1 -- 172.21.15.52:0/2285227698 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f191c011e00 con 0x7f19201062a0 2024-06-10T02:42:10.748 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:09.856+0000 7f19254a4700 1 -- 172.21.15.52:0/2285227698 >> [v2:172.21.15.52:6800/3934479642,v1:172.21.15.52:6801/3934479642] conn(0x7f190c0390a0 msgr2=0x7f190c03b560 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.52:6800/3934479642 2024-06-10T02:42:10.748 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:09.856+0000 7f19254a4700 1 --2- 172.21.15.52:0/2285227698 >> [v2:172.21.15.52:6800/3934479642,v1:172.21.15.52:6801/3934479642] conn(0x7f190c0390a0 0x7f190c03b560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-06-10T02:42:10.748 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:09.865+0000 7f1916ffd700 1 -- 172.21.15.52:0/2285227698 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f191c01ad80 con 0x7f19201062a0 2024-06-10T02:42:10.748 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:10.057+0000 7f19254a4700 1 -- 172.21.15.52:0/2285227698 >> [v2:172.21.15.52:6800/3934479642,v1:172.21.15.52:6801/3934479642] conn(0x7f190c0390a0 msgr2=0x7f190c03b560 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.52:6800/3934479642 2024-06-10T02:42:10.749 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:10.057+0000 7f19254a4700 1 --2- 172.21.15.52:0/2285227698 >> [v2:172.21.15.52:6800/3934479642,v1:172.21.15.52:6801/3934479642] conn(0x7f190c0390a0 0x7f190c03b560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-06-10T02:42:10.749 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:10.134+0000 7f1927f09700 1 -- 172.21.15.52:0/2285227698 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f192019f580 con 0x7f19201062a0 2024-06-10T02:42:10.749 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:10.134+0000 7f1916ffd700 1 -- 172.21.15.52:0/2285227698 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7f192019f580 con 0x7f19201062a0 2024-06-10T02:42:10.749 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:10.136+0000 7f1927f09700 1 -- 172.21.15.52:0/2285227698 >> [v2:172.21.15.52:6800/3934479642,v1:172.21.15.52:6801/3934479642] conn(0x7f190c0390a0 msgr2=0x7f190c03b560 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-10T02:42:10.749 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:10.136+0000 7f1927f09700 1 --2- 172.21.15.52:0/2285227698 >> [v2:172.21.15.52:6800/3934479642,v1:172.21.15.52:6801/3934479642] conn(0x7f190c0390a0 0x7f190c03b560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:42:10.749 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:10.137+0000 7f1927f09700 1 -- 172.21.15.52:0/2285227698 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f19201062a0 msgr2=0x7f192019bef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:42:10.750 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:10.137+0000 7f1927f09700 1 --2- 172.21.15.52:0/2285227698 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f19201062a0 0x7f192019bef0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f191c000c00 tx=0x7f191c011c10 comp rx=0 tx=0).stop 2024-06-10T02:42:10.750 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:10.137+0000 7f1927f09700 1 -- 172.21.15.52:0/2285227698 shutdown_connections 2024-06-10T02:42:10.750 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:10.137+0000 7f1927f09700 1 --2- 172.21.15.52:0/2285227698 >> [v2:172.21.15.52:6800/3934479642,v1:172.21.15.52:6801/3934479642] conn(0x7f190c0390a0 0x7f190c03b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:42:10.750 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:10.137+0000 7f1927f09700 1 --2- 172.21.15.52:0/2285227698 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f19201062a0 0x7f192019bef0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:42:10.750 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:10.137+0000 7f1927f09700 1 -- 172.21.15.52:0/2285227698 >> 172.21.15.52:0/2285227698 conn(0x7f1920101bd0 msgr2=0x7f1920102600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:42:10.750 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:10.137+0000 7f1927f09700 1 -- 172.21.15.52:0/2285227698 shutdown_connections 2024-06-10T02:42:10.751 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:10.137+0000 7f1927f09700 1 -- 172.21.15.52:0/2285227698 wait complete. 2024-06-10T02:42:10.751 INFO:teuthology.orchestra.run.smithi052.stdout:Waiting for the mgr to restart... 2024-06-10T02:42:10.751 INFO:teuthology.orchestra.run.smithi052.stdout:Waiting for mgr epoch 5... 2024-06-10T02:42:10.968 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 10 02:42:10 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mgr-x[32595]: 2024-06-10T02:42:10.820+0000 7fa1b9835200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-06-10T02:42:11.222 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 10 02:42:11 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mgr-x[32595]: 2024-06-10T02:42:11.125+0000 7fa1b9835200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-06-10T02:42:11.222 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 10 02:42:11 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mgr-x[32595]: 2024-06-10T02:42:11.224+0000 7fa1b9835200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-06-10T02:42:11.472 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 10 02:42:11 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mgr-x[32595]: 2024-06-10T02:42:11.436+0000 7fa1b9835200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-06-10T02:42:11.972 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 10 02:42:11 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mgr-x[32595]: 2024-06-10T02:42:11.637+0000 7fa1b9835200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-06-10T02:42:12.581 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 10 02:42:12 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mgr-x[32595]: 2024-06-10T02:42:12.268+0000 7fa1b9835200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-06-10T02:42:12.972 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 10 02:42:12 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mgr-x[32595]: 2024-06-10T02:42:12.707+0000 7fa1b9835200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-06-10T02:42:12.973 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 10 02:42:12 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mgr-x[32595]: 2024-06-10T02:42:12.808+0000 7fa1b9835200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-06-10T02:42:13.473 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 10 02:42:13 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mgr-x[32595]: 2024-06-10T02:42:13.017+0000 7fa1b9835200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-06-10T02:42:13.931 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 10 02:42:13 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mgr-x[32595]: 2024-06-10T02:42:13.584+0000 7fa1b9835200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-06-10T02:42:13.931 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 10 02:42:13 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mgr-x[32595]: 2024-06-10T02:42:13.680+0000 7fa1b9835200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-06-10T02:42:14.182 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 10 02:42:13 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mgr-x[32595]: 2024-06-10T02:42:13.935+0000 7fa1b9835200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-06-10T02:42:14.182 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 10 02:42:14 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mgr-x[32595]: 2024-06-10T02:42:14.075+0000 7fa1b9835200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-06-10T02:42:14.182 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 10 02:42:14 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mgr-x[32595]: 2024-06-10T02:42:14.175+0000 7fa1b9835200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-06-10T02:42:14.472 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 10 02:42:14 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mgr-x[32595]: 2024-06-10T02:42:14.299+0000 7fa1b9835200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-06-10T02:42:14.972 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 10 02:42:14 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mgr-x[32595]: 2024-06-10T02:42:14.640+0000 7fa1b9835200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-06-10T02:42:15.544 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 10 02:42:15 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mgr-x[32595]: 2024-06-10T02:42:15.268+0000 7fa1b9835200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-06-10T02:42:15.545 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 10 02:42:15 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mgr-x[32595]: 2024-06-10T02:42:15.496+0000 7fa1b9835200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-06-10T02:42:15.973 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:42:15 smithi052 ceph-mon[32362]: Active manager daemon x restarted 2024-06-10T02:42:15.973 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:42:15 smithi052 ceph-mon[32362]: Activating manager daemon x 2024-06-10T02:42:15.973 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:42:15 smithi052 ceph-mon[32362]: osdmap e2: 0 total, 0 up, 0 in 2024-06-10T02:42:15.973 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:42:15 smithi052 ceph-mon[32362]: mgrmap e6: x(active, starting, since 0.00347608s) 2024-06-10T02:42:15.973 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:42:15 smithi052 ceph-mon[32362]: from='mgr.14118 172.21.15.52:0/1358068050' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-10T02:42:15.973 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:42:15 smithi052 ceph-mon[32362]: from='mgr.14118 172.21.15.52:0/1358068050' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-06-10T02:42:15.973 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:42:15 smithi052 ceph-mon[32362]: from='mgr.14118 172.21.15.52:0/1358068050' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-06-10T02:42:15.973 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:42:15 smithi052 ceph-mon[32362]: from='mgr.14118 172.21.15.52:0/1358068050' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-06-10T02:42:15.973 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:42:15 smithi052 ceph-mon[32362]: from='mgr.14118 172.21.15.52:0/1358068050' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-06-10T02:42:15.974 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:42:15 smithi052 ceph-mon[32362]: Manager daemon x is now available 2024-06-10T02:42:15.974 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:42:15 smithi052 ceph-mon[32362]: from='mgr.14118 172.21.15.52:0/1358068050' entity='mgr.x' 2024-06-10T02:42:15.974 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:42:15 smithi052 ceph-mon[32362]: from='mgr.14118 172.21.15.52:0/1358068050' entity='mgr.x' 2024-06-10T02:42:15.974 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:42:15 smithi052 ceph-mon[32362]: from='mgr.14118 172.21.15.52:0/1358068050' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T02:42:15.974 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:42:15 smithi052 ceph-mon[32362]: from='mgr.14118 172.21.15.52:0/1358068050' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T02:42:16.846 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:42:16 smithi052 ceph-mon[32362]: Found migration_current of "None". Setting to last migration. 2024-06-10T02:42:16.846 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:42:16 smithi052 ceph-mon[32362]: from='mgr.14118 172.21.15.52:0/1358068050' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-06-10T02:42:16.847 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:42:16 smithi052 ceph-mon[32362]: from='mgr.14118 172.21.15.52:0/1358068050' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-06-10T02:42:16.847 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:42:16 smithi052 ceph-mon[32362]: mgrmap e7: x(active, since 1.00578s) 2024-06-10T02:42:17.089 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout { 2024-06-10T02:42:17.089 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-06-10T02:42:17.089 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "initialized": true 2024-06-10T02:42:17.089 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout } 2024-06-10T02:42:17.089 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:11.175+0000 7f36048db700 1 Processor -- start 2024-06-10T02:42:17.089 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:11.175+0000 7f36048db700 1 -- start start 2024-06-10T02:42:17.089 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:11.175+0000 7f36048db700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3600108030 0x7f3600108450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:42:17.089 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:11.175+0000 7f36048db700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f3600108a20 con 0x7f3600108030 2024-06-10T02:42:17.089 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:11.176+0000 7f35fe59c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3600108030 0x7f3600108450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:42:17.090 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:11.176+0000 7f35fe59c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3600108030 0x7f3600108450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37398/0 (socket says 172.21.15.52:37398) 2024-06-10T02:42:17.090 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:11.176+0000 7f35fe59c700 1 -- 172.21.15.52:0/2621639896 learned_addr learned my addr 172.21.15.52:0/2621639896 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:42:17.090 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:11.176+0000 7f35fe59c700 1 -- 172.21.15.52:0/2621639896 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3600109190 con 0x7f3600108030 2024-06-10T02:42:17.090 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:11.176+0000 7f35fe59c700 1 --2- 172.21.15.52:0/2621639896 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3600108030 0x7f3600108450 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f35f00098a0 tx=0x7f35f0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=859391ae20367adb server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:42:17.090 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:11.176+0000 7f35fd59a700 1 -- 172.21.15.52:0/2621639896 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f35f0004160 con 0x7f3600108030 2024-06-10T02:42:17.090 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:11.176+0000 7f35fd59a700 1 -- 172.21.15.52:0/2621639896 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f35f0004300 con 0x7f3600108030 2024-06-10T02:42:17.090 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:11.176+0000 7f35fd59a700 1 -- 172.21.15.52:0/2621639896 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f35f0011b50 con 0x7f3600108030 2024-06-10T02:42:17.090 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:11.177+0000 7f36048db700 1 -- 172.21.15.52:0/2621639896 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3600108030 msgr2=0x7f3600108450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:42:17.090 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:11.177+0000 7f36048db700 1 --2- 172.21.15.52:0/2621639896 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3600108030 0x7f3600108450 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f35f00098a0 tx=0x7f35f0009bb0 comp rx=0 tx=0).stop 2024-06-10T02:42:17.090 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:11.177+0000 7f36048db700 1 -- 172.21.15.52:0/2621639896 shutdown_connections 2024-06-10T02:42:17.090 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:11.177+0000 7f36048db700 1 --2- 172.21.15.52:0/2621639896 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3600108030 0x7f3600108450 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:42:17.090 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:11.177+0000 7f36048db700 1 -- 172.21.15.52:0/2621639896 >> 172.21.15.52:0/2621639896 conn(0x7f36001035b0 msgr2=0x7f3600105a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:42:17.090 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:11.177+0000 7f36048db700 1 -- 172.21.15.52:0/2621639896 shutdown_connections 2024-06-10T02:42:17.091 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:11.177+0000 7f36048db700 1 -- 172.21.15.52:0/2621639896 wait complete. 2024-06-10T02:42:17.091 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:11.178+0000 7f36048db700 1 Processor -- start 2024-06-10T02:42:17.091 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:11.178+0000 7f36048db700 1 -- start start 2024-06-10T02:42:17.091 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:11.178+0000 7f36048db700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3600108030 0x7f360007e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:42:17.091 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:11.178+0000 7f36048db700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f360010c540 con 0x7f3600108030 2024-06-10T02:42:17.091 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:11.179+0000 7f35fe59c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3600108030 0x7f360007e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:42:17.091 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:11.179+0000 7f35fe59c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3600108030 0x7f360007e410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37414/0 (socket says 172.21.15.52:37414) 2024-06-10T02:42:17.091 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:11.179+0000 7f35fe59c700 1 -- 172.21.15.52:0/1752158006 learned_addr learned my addr 172.21.15.52:0/1752158006 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:42:17.091 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:11.179+0000 7f35fe59c700 1 -- 172.21.15.52:0/1752158006 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f360007e950 con 0x7f3600108030 2024-06-10T02:42:17.091 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:11.179+0000 7f35fe59c700 1 --2- 172.21.15.52:0/1752158006 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3600108030 0x7f360007e410 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f35f0011670 tx=0x7f35f0011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:42:17.091 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:11.179+0000 7f35f77fe700 1 -- 172.21.15.52:0/1752158006 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f35f00039c0 con 0x7f3600108030 2024-06-10T02:42:17.091 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:11.179+0000 7f35f77fe700 1 -- 172.21.15.52:0/1752158006 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f35f0011e50 con 0x7f3600108030 2024-06-10T02:42:17.092 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:11.179+0000 7f36048db700 1 -- 172.21.15.52:0/1752158006 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f360007cb40 con 0x7f3600108030 2024-06-10T02:42:17.092 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:11.179+0000 7f35f77fe700 1 -- 172.21.15.52:0/1752158006 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f35f001a5c0 con 0x7f3600108030 2024-06-10T02:42:17.092 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:11.179+0000 7f36048db700 1 -- 172.21.15.52:0/1752158006 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f360007d080 con 0x7f3600108030 2024-06-10T02:42:17.092 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:11.180+0000 7f35f77fe700 1 -- 172.21.15.52:0/1752158006 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7f35f001a8c0 con 0x7f3600108030 2024-06-10T02:42:17.092 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:11.181+0000 7f35f77fe700 1 --2- 172.21.15.52:0/1752158006 >> [v2:172.21.15.52:6800/3934479642,v1:172.21.15.52:6801/3934479642] conn(0x7f35ec039000 0x7f35ec03b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:42:17.092 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:11.181+0000 7f35f77fe700 1 -- 172.21.15.52:0/1752158006 --> [v2:172.21.15.52:6800/3934479642,v1:172.21.15.52:6801/3934479642] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f35ec03bbd0 con 0x7f35ec039000 2024-06-10T02:42:17.092 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:11.181+0000 7f35f77fe700 1 -- 172.21.15.52:0/1752158006 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f35f004d3e0 con 0x7f3600108030 2024-06-10T02:42:17.092 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:11.181+0000 7f35fdd9b700 1 -- 172.21.15.52:0/1752158006 >> [v2:172.21.15.52:6800/3934479642,v1:172.21.15.52:6801/3934479642] conn(0x7f35ec039000 msgr2=0x7f35ec03b4c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.52:6800/3934479642 2024-06-10T02:42:17.092 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:11.181+0000 7f35fdd9b700 1 --2- 172.21.15.52:0/1752158006 >> [v2:172.21.15.52:6800/3934479642,v1:172.21.15.52:6801/3934479642] conn(0x7f35ec039000 0x7f35ec03b4c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-06-10T02:42:17.092 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:11.382+0000 7f35fdd9b700 1 -- 172.21.15.52:0/1752158006 >> [v2:172.21.15.52:6800/3934479642,v1:172.21.15.52:6801/3934479642] conn(0x7f35ec039000 msgr2=0x7f35ec03b4c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.52:6800/3934479642 2024-06-10T02:42:17.092 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:11.382+0000 7f35fdd9b700 1 --2- 172.21.15.52:0/1752158006 >> [v2:172.21.15.52:6800/3934479642,v1:172.21.15.52:6801/3934479642] conn(0x7f35ec039000 0x7f35ec03b4c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-06-10T02:42:17.092 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:11.783+0000 7f35fdd9b700 1 -- 172.21.15.52:0/1752158006 >> [v2:172.21.15.52:6800/3934479642,v1:172.21.15.52:6801/3934479642] conn(0x7f35ec039000 msgr2=0x7f35ec03b4c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.52:6800/3934479642 2024-06-10T02:42:17.092 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:11.783+0000 7f35fdd9b700 1 --2- 172.21.15.52:0/1752158006 >> [v2:172.21.15.52:6800/3934479642,v1:172.21.15.52:6801/3934479642] conn(0x7f35ec039000 0x7f35ec03b4c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-06-10T02:42:17.092 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:12.585+0000 7f35fdd9b700 1 -- 172.21.15.52:0/1752158006 >> [v2:172.21.15.52:6800/3934479642,v1:172.21.15.52:6801/3934479642] conn(0x7f35ec039000 msgr2=0x7f35ec03b4c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.52:6800/3934479642 2024-06-10T02:42:17.093 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:12.585+0000 7f35fdd9b700 1 --2- 172.21.15.52:0/1752158006 >> [v2:172.21.15.52:6800/3934479642,v1:172.21.15.52:6801/3934479642] conn(0x7f35ec039000 0x7f35ec03b4c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-06-10T02:42:17.093 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:14.186+0000 7f35fdd9b700 1 -- 172.21.15.52:0/1752158006 >> [v2:172.21.15.52:6800/3934479642,v1:172.21.15.52:6801/3934479642] conn(0x7f35ec039000 msgr2=0x7f35ec03b4c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.52:6800/3934479642 2024-06-10T02:42:17.093 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:14.186+0000 7f35fdd9b700 1 --2- 172.21.15.52:0/1752158006 >> [v2:172.21.15.52:6800/3934479642,v1:172.21.15.52:6801/3934479642] conn(0x7f35ec039000 0x7f35ec03b4c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-06-10T02:42:17.093 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:15.499+0000 7f35f77fe700 1 -- 172.21.15.52:0/1752158006 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mgrmap(e 6) ==== 46054+0+0 (secure 0 0 0) 0x7f35f001ac00 con 0x7f3600108030 2024-06-10T02:42:17.093 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:15.499+0000 7f35f77fe700 1 -- 172.21.15.52:0/1752158006 >> [v2:172.21.15.52:6800/3934479642,v1:172.21.15.52:6801/3934479642] conn(0x7f35ec039000 msgr2=0x7f35ec03b4c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-10T02:42:17.093 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:15.499+0000 7f35f77fe700 1 --2- 172.21.15.52:0/1752158006 >> [v2:172.21.15.52:6800/3934479642,v1:172.21.15.52:6801/3934479642] conn(0x7f35ec039000 0x7f35ec03b4c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:42:17.093 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:16.500+0000 7f35f77fe700 1 -- 172.21.15.52:0/1752158006 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mgrmap(e 7) ==== 46181+0+0 (secure 0 0 0) 0x7f35f004e860 con 0x7f3600108030 2024-06-10T02:42:17.093 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:16.501+0000 7f35f77fe700 1 --2- 172.21.15.52:0/1752158006 >> [v2:172.21.15.52:6800/896317057,v1:172.21.15.52:6801/896317057] conn(0x7f35ec03ca50 0x7f35ec03ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:42:17.093 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:16.501+0000 7f35f77fe700 1 -- 172.21.15.52:0/1752158006 --> [v2:172.21.15.52:6800/896317057,v1:172.21.15.52:6801/896317057] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f35ec03bbd0 con 0x7f35ec03ca50 2024-06-10T02:42:17.093 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:16.501+0000 7f35fdd9b700 1 --2- 172.21.15.52:0/1752158006 >> [v2:172.21.15.52:6800/896317057,v1:172.21.15.52:6801/896317057] conn(0x7f35ec03ca50 0x7f35ec03ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:42:17.093 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:16.502+0000 7f35fdd9b700 1 --2- 172.21.15.52:0/1752158006 >> [v2:172.21.15.52:6800/896317057,v1:172.21.15.52:6801/896317057] conn(0x7f35ec03ca50 0x7f35ec03ee40 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f35e8003a10 tx=0x7f35e8009290 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:42:17.093 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:16.503+0000 7f35f77fe700 1 -- 172.21.15.52:0/1752158006 <== mgr.14118 v2:172.21.15.52:6800/896317057 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7f35ec03bbd0 con 0x7f35ec03ca50 2024-06-10T02:42:17.093 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:16.509+0000 7f36048db700 1 -- 172.21.15.52:0/1752158006 --> [v2:172.21.15.52:6800/896317057,v1:172.21.15.52:6801/896317057] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f3600063af0 con 0x7f35ec03ca50 2024-06-10T02:42:17.093 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:16.510+0000 7f35f77fe700 1 -- 172.21.15.52:0/1752158006 <== mgr.14118 v2:172.21.15.52:6800/896317057 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7f3600063af0 con 0x7f35ec03ca50 2024-06-10T02:42:17.094 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:16.510+0000 7f36048db700 1 -- 172.21.15.52:0/1752158006 >> [v2:172.21.15.52:6800/896317057,v1:172.21.15.52:6801/896317057] conn(0x7f35ec03ca50 msgr2=0x7f35ec03ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:42:17.094 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:16.510+0000 7f36048db700 1 --2- 172.21.15.52:0/1752158006 >> [v2:172.21.15.52:6800/896317057,v1:172.21.15.52:6801/896317057] conn(0x7f35ec03ca50 0x7f35ec03ee40 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f35e8003a10 tx=0x7f35e8009290 comp rx=0 tx=0).stop 2024-06-10T02:42:17.094 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:16.510+0000 7f36048db700 1 -- 172.21.15.52:0/1752158006 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3600108030 msgr2=0x7f360007e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:42:17.094 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:16.510+0000 7f36048db700 1 --2- 172.21.15.52:0/1752158006 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3600108030 0x7f360007e410 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f35f0011670 tx=0x7f35f0011b60 comp rx=0 tx=0).stop 2024-06-10T02:42:17.094 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:16.510+0000 7f36048db700 1 -- 172.21.15.52:0/1752158006 shutdown_connections 2024-06-10T02:42:17.094 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:16.510+0000 7f36048db700 1 --2- 172.21.15.52:0/1752158006 >> [v2:172.21.15.52:6800/896317057,v1:172.21.15.52:6801/896317057] conn(0x7f35ec03ca50 0x7f35ec03ee40 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:42:17.094 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:16.510+0000 7f36048db700 1 --2- 172.21.15.52:0/1752158006 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3600108030 0x7f360007e410 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:42:17.094 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:16.510+0000 7f36048db700 1 --2- 172.21.15.52:0/1752158006 >> [v2:172.21.15.52:6800/3934479642,v1:172.21.15.52:6801/3934479642] conn(0x7f35ec039000 0x7f35ec03b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:42:17.094 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:16.510+0000 7f36048db700 1 -- 172.21.15.52:0/1752158006 >> 172.21.15.52:0/1752158006 conn(0x7f36001035b0 msgr2=0x7f36001040a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:42:17.094 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:16.510+0000 7f36048db700 1 -- 172.21.15.52:0/1752158006 shutdown_connections 2024-06-10T02:42:17.094 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:16.510+0000 7f36048db700 1 -- 172.21.15.52:0/1752158006 wait complete. 2024-06-10T02:42:17.094 INFO:teuthology.orchestra.run.smithi052.stdout:mgr epoch 5 is available 2024-06-10T02:42:17.094 INFO:teuthology.orchestra.run.smithi052.stdout:Setting orchestrator backend to cephadm... 2024-06-10T02:42:17.973 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:42:17 smithi052 ceph-mon[32362]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-06-10T02:42:17.973 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:42:17 smithi052 ceph-mon[32362]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-06-10T02:42:17.973 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:42:17 smithi052 ceph-mon[32362]: from='mgr.14118 172.21.15.52:0/1358068050' entity='mgr.x' 2024-06-10T02:42:17.973 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:42:17 smithi052 ceph-mon[32362]: from='mgr.14118 172.21.15.52:0/1358068050' entity='mgr.x' 2024-06-10T02:42:17.973 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:42:17 smithi052 ceph-mon[32362]: from='mgr.14118 172.21.15.52:0/1358068050' entity='mgr.x' 2024-06-10T02:42:17.973 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:42:17 smithi052 ceph-mon[32362]: from='mgr.14118 172.21.15.52:0/1358068050' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T02:42:18.253 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:17.457+0000 7f8411db0700 1 Processor -- start 2024-06-10T02:42:18.253 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:17.457+0000 7f8411db0700 1 -- start start 2024-06-10T02:42:18.253 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:17.457+0000 7f8411db0700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f840c106e60 0x7f840c107280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:42:18.253 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:17.457+0000 7f8411db0700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f840c1077c0 con 0x7f840c106e60 2024-06-10T02:42:18.253 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:17.457+0000 7f840b7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f840c106e60 0x7f840c107280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:42:18.253 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:17.457+0000 7f840b7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f840c106e60 0x7f840c107280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37476/0 (socket says 172.21.15.52:37476) 2024-06-10T02:42:18.253 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:17.457+0000 7f840b7fe700 1 -- 172.21.15.52:0/470854035 learned_addr learned my addr 172.21.15.52:0/470854035 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:42:18.253 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:17.458+0000 7f840b7fe700 1 -- 172.21.15.52:0/470854035 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f840c108050 con 0x7f840c106e60 2024-06-10T02:42:18.253 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:17.458+0000 7f840b7fe700 1 --2- 172.21.15.52:0/470854035 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f840c106e60 0x7f840c107280 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f83fc0098a0 tx=0x7f83fc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2ed025780b83884 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:42:18.254 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:17.458+0000 7f840affd700 1 -- 172.21.15.52:0/470854035 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f83fc004160 con 0x7f840c106e60 2024-06-10T02:42:18.254 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:17.458+0000 7f840affd700 1 -- 172.21.15.52:0/470854035 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f83fc004300 con 0x7f840c106e60 2024-06-10T02:42:18.254 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:17.458+0000 7f840affd700 1 -- 172.21.15.52:0/470854035 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f83fc011b50 con 0x7f840c106e60 2024-06-10T02:42:18.254 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:17.459+0000 7f8411db0700 1 -- 172.21.15.52:0/470854035 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f840c106e60 msgr2=0x7f840c107280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:42:18.254 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:17.459+0000 7f8411db0700 1 --2- 172.21.15.52:0/470854035 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f840c106e60 0x7f840c107280 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f83fc0098a0 tx=0x7f83fc009bb0 comp rx=0 tx=0).stop 2024-06-10T02:42:18.254 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:17.459+0000 7f8411db0700 1 -- 172.21.15.52:0/470854035 shutdown_connections 2024-06-10T02:42:18.254 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:17.459+0000 7f8411db0700 1 --2- 172.21.15.52:0/470854035 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f840c106e60 0x7f840c107280 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:42:18.254 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:17.459+0000 7f8411db0700 1 -- 172.21.15.52:0/470854035 >> 172.21.15.52:0/470854035 conn(0x7f840c102450 msgr2=0x7f840c104890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:42:18.254 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:17.459+0000 7f8411db0700 1 -- 172.21.15.52:0/470854035 shutdown_connections 2024-06-10T02:42:18.254 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:17.459+0000 7f8411db0700 1 -- 172.21.15.52:0/470854035 wait complete. 2024-06-10T02:42:18.254 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:17.460+0000 7f8411db0700 1 Processor -- start 2024-06-10T02:42:18.254 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:17.460+0000 7f8411db0700 1 -- start start 2024-06-10T02:42:18.254 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:17.460+0000 7f8411db0700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f840c106e60 0x7f840c19bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:42:18.255 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:17.460+0000 7f8411db0700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f840c10b310 con 0x7f840c106e60 2024-06-10T02:42:18.255 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:17.460+0000 7f840b7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f840c106e60 0x7f840c19bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:42:18.255 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:17.460+0000 7f840b7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f840c106e60 0x7f840c19bf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37486/0 (socket says 172.21.15.52:37486) 2024-06-10T02:42:18.255 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:17.460+0000 7f840b7fe700 1 -- 172.21.15.52:0/2147778433 learned_addr learned my addr 172.21.15.52:0/2147778433 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:42:18.255 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:17.460+0000 7f840b7fe700 1 -- 172.21.15.52:0/2147778433 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f840c19c4d0 con 0x7f840c106e60 2024-06-10T02:42:18.255 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:17.460+0000 7f840b7fe700 1 --2- 172.21.15.52:0/2147778433 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f840c106e60 0x7f840c19bf90 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f83fc011670 tx=0x7f83fc011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:42:18.255 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:17.461+0000 7f84097fa700 1 -- 172.21.15.52:0/2147778433 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f83fc0039c0 con 0x7f840c106e60 2024-06-10T02:42:18.255 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:17.461+0000 7f84097fa700 1 -- 172.21.15.52:0/2147778433 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f83fc011e50 con 0x7f840c106e60 2024-06-10T02:42:18.255 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:17.461+0000 7f8411db0700 1 -- 172.21.15.52:0/2147778433 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f840c19c760 con 0x7f840c106e60 2024-06-10T02:42:18.255 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:17.461+0000 7f8411db0700 1 -- 172.21.15.52:0/2147778433 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f840c19f400 con 0x7f840c106e60 2024-06-10T02:42:18.255 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:17.461+0000 7f84097fa700 1 -- 172.21.15.52:0/2147778433 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f83fc01a5c0 con 0x7f840c106e60 2024-06-10T02:42:18.255 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:17.462+0000 7f84097fa700 1 -- 172.21.15.52:0/2147778433 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 7) ==== 46181+0+0 (secure 0 0 0) 0x7f83fc01a8e0 con 0x7f840c106e60 2024-06-10T02:42:18.255 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:17.462+0000 7f8411db0700 1 -- 172.21.15.52:0/2147778433 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83ec005330 con 0x7f840c106e60 2024-06-10T02:42:18.255 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:17.462+0000 7f84097fa700 1 --2- 172.21.15.52:0/2147778433 >> [v2:172.21.15.52:6800/896317057,v1:172.21.15.52:6801/896317057] conn(0x7f83f8038ef0 0x7f83f803b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:42:18.256 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:17.462+0000 7f84097fa700 1 -- 172.21.15.52:0/2147778433 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f83fc04d1d0 con 0x7f840c106e60 2024-06-10T02:42:18.256 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:17.462+0000 7f8402ffd700 1 --2- 172.21.15.52:0/2147778433 >> [v2:172.21.15.52:6800/896317057,v1:172.21.15.52:6801/896317057] conn(0x7f83f8038ef0 0x7f83f803b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:42:18.256 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:17.463+0000 7f8402ffd700 1 --2- 172.21.15.52:0/2147778433 >> [v2:172.21.15.52:6800/896317057,v1:172.21.15.52:6801/896317057] conn(0x7f83f8038ef0 0x7f83f803b3b0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f83f4006fd0 tx=0x7f83f4006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:42:18.256 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:17.468+0000 7f84097fa700 1 -- 172.21.15.52:0/2147778433 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f83fc019c70 con 0x7f840c106e60 2024-06-10T02:42:18.256 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:17.691+0000 7f8411db0700 1 -- 172.21.15.52:0/2147778433 --> [v2:172.21.15.52:6800/896317057,v1:172.21.15.52:6801/896317057] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7f83ec000bc0 con 0x7f83f8038ef0 2024-06-10T02:42:18.256 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:17.696+0000 7f84097fa700 1 -- 172.21.15.52:0/2147778433 <== mgr.14118 v2:172.21.15.52:6800/896317057 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f83ec000bc0 con 0x7f83f8038ef0 2024-06-10T02:42:18.256 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:17.698+0000 7f8411db0700 1 -- 172.21.15.52:0/2147778433 >> [v2:172.21.15.52:6800/896317057,v1:172.21.15.52:6801/896317057] conn(0x7f83f8038ef0 msgr2=0x7f83f803b3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:42:18.256 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:17.698+0000 7f8411db0700 1 --2- 172.21.15.52:0/2147778433 >> [v2:172.21.15.52:6800/896317057,v1:172.21.15.52:6801/896317057] conn(0x7f83f8038ef0 0x7f83f803b3b0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f83f4006fd0 tx=0x7f83f4006e40 comp rx=0 tx=0).stop 2024-06-10T02:42:18.256 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:17.698+0000 7f8411db0700 1 -- 172.21.15.52:0/2147778433 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f840c106e60 msgr2=0x7f840c19bf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:42:18.256 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:17.698+0000 7f8411db0700 1 --2- 172.21.15.52:0/2147778433 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f840c106e60 0x7f840c19bf90 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f83fc011670 tx=0x7f83fc011b60 comp rx=0 tx=0).stop 2024-06-10T02:42:18.256 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:17.699+0000 7f8411db0700 1 -- 172.21.15.52:0/2147778433 shutdown_connections 2024-06-10T02:42:18.256 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:17.699+0000 7f8411db0700 1 --2- 172.21.15.52:0/2147778433 >> [v2:172.21.15.52:6800/896317057,v1:172.21.15.52:6801/896317057] conn(0x7f83f8038ef0 0x7f83f803b3b0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:42:18.256 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:17.699+0000 7f8411db0700 1 --2- 172.21.15.52:0/2147778433 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f840c106e60 0x7f840c19bf90 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:42:18.257 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:17.699+0000 7f8411db0700 1 -- 172.21.15.52:0/2147778433 >> 172.21.15.52:0/2147778433 conn(0x7f840c102450 msgr2=0x7f840c103290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:42:18.257 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:17.699+0000 7f8411db0700 1 -- 172.21.15.52:0/2147778433 shutdown_connections 2024-06-10T02:42:18.257 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:17.699+0000 7f8411db0700 1 -- 172.21.15.52:0/2147778433 wait complete. 2024-06-10T02:42:18.881 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:42:18 smithi052 ceph-mon[32362]: [10/Jun/2024:02:42:17] ENGINE Bus STARTING 2024-06-10T02:42:18.881 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:42:18 smithi052 ceph-mon[32362]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T02:42:18.882 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:42:18 smithi052 ceph-mon[32362]: [10/Jun/2024:02:42:17] ENGINE Serving on https://172.21.15.52:7150 2024-06-10T02:42:18.882 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:42:18 smithi052 ceph-mon[32362]: [10/Jun/2024:02:42:17] ENGINE Serving on http://172.21.15.52:8765 2024-06-10T02:42:18.882 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:42:18 smithi052 ceph-mon[32362]: [10/Jun/2024:02:42:17] ENGINE Bus STARTED 2024-06-10T02:42:18.882 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:42:18 smithi052 ceph-mon[32362]: mgrmap e8: x(active, since 2s) 2024-06-10T02:42:18.882 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:42:18 smithi052 ceph-mon[32362]: from='mgr.14118 172.21.15.52:0/1358068050' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T02:42:19.451 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout value unchanged 2024-06-10T02:42:19.452 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:18.662+0000 7f9d6545e700 1 Processor -- start 2024-06-10T02:42:19.452 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:18.662+0000 7f9d6545e700 1 -- start start 2024-06-10T02:42:19.452 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:18.662+0000 7f9d6545e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9d60108070 0x7f9d60108490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:42:19.452 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:18.662+0000 7f9d6545e700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f9d60108a60 con 0x7f9d60108070 2024-06-10T02:42:19.452 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:18.662+0000 7f9d5effd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9d60108070 0x7f9d60108490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:42:19.452 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:18.662+0000 7f9d5effd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9d60108070 0x7f9d60108490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37488/0 (socket says 172.21.15.52:37488) 2024-06-10T02:42:19.452 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:18.662+0000 7f9d5effd700 1 -- 172.21.15.52:0/352735656 learned_addr learned my addr 172.21.15.52:0/352735656 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:42:19.452 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:18.662+0000 7f9d5effd700 1 -- 172.21.15.52:0/352735656 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d601091d0 con 0x7f9d60108070 2024-06-10T02:42:19.452 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:18.662+0000 7f9d5effd700 1 --2- 172.21.15.52:0/352735656 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9d60108070 0x7f9d60108490 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f9d500098a0 tx=0x7f9d50009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abc25078155b680f server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:42:19.453 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:18.663+0000 7f9d5dffb700 1 -- 172.21.15.52:0/352735656 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9d50004160 con 0x7f9d60108070 2024-06-10T02:42:19.453 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:18.663+0000 7f9d5dffb700 1 -- 172.21.15.52:0/352735656 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f9d50004300 con 0x7f9d60108070 2024-06-10T02:42:19.453 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:18.663+0000 7f9d5dffb700 1 -- 172.21.15.52:0/352735656 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9d50011b50 con 0x7f9d60108070 2024-06-10T02:42:19.453 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:18.663+0000 7f9d6545e700 1 -- 172.21.15.52:0/352735656 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9d60108070 msgr2=0x7f9d60108490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:42:19.453 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:18.663+0000 7f9d6545e700 1 --2- 172.21.15.52:0/352735656 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9d60108070 0x7f9d60108490 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f9d500098a0 tx=0x7f9d50009bb0 comp rx=0 tx=0).stop 2024-06-10T02:42:19.453 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:18.663+0000 7f9d6545e700 1 -- 172.21.15.52:0/352735656 shutdown_connections 2024-06-10T02:42:19.453 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:18.663+0000 7f9d6545e700 1 --2- 172.21.15.52:0/352735656 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9d60108070 0x7f9d60108490 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:42:19.453 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:18.663+0000 7f9d6545e700 1 -- 172.21.15.52:0/352735656 >> 172.21.15.52:0/352735656 conn(0x7f9d601035f0 msgr2=0x7f9d60105a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:42:19.453 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:18.663+0000 7f9d6545e700 1 -- 172.21.15.52:0/352735656 shutdown_connections 2024-06-10T02:42:19.453 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:18.663+0000 7f9d6545e700 1 -- 172.21.15.52:0/352735656 wait complete. 2024-06-10T02:42:19.453 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:18.664+0000 7f9d6545e700 1 Processor -- start 2024-06-10T02:42:19.454 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:18.664+0000 7f9d6545e700 1 -- start start 2024-06-10T02:42:19.454 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:18.664+0000 7f9d6545e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9d60108070 0x7f9d6019bed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:42:19.454 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:18.664+0000 7f9d6545e700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f9d6010c580 con 0x7f9d60108070 2024-06-10T02:42:19.454 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:18.664+0000 7f9d5effd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9d60108070 0x7f9d6019bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:42:19.454 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:18.664+0000 7f9d5effd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9d60108070 0x7f9d6019bed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37500/0 (socket says 172.21.15.52:37500) 2024-06-10T02:42:19.454 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:18.664+0000 7f9d5effd700 1 -- 172.21.15.52:0/480026580 learned_addr learned my addr 172.21.15.52:0/480026580 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:42:19.454 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:18.665+0000 7f9d5effd700 1 -- 172.21.15.52:0/480026580 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d6019c410 con 0x7f9d60108070 2024-06-10T02:42:19.455 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:18.665+0000 7f9d5effd700 1 --2- 172.21.15.52:0/480026580 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9d60108070 0x7f9d6019bed0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f9d50000c00 tx=0x7f9d50011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:42:19.455 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:18.665+0000 7f9d47fff700 1 -- 172.21.15.52:0/480026580 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9d50003b20 con 0x7f9d60108070 2024-06-10T02:42:19.455 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:18.665+0000 7f9d47fff700 1 -- 172.21.15.52:0/480026580 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f9d50003cc0 con 0x7f9d60108070 2024-06-10T02:42:19.455 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:18.665+0000 7f9d47fff700 1 -- 172.21.15.52:0/480026580 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9d50024910 con 0x7f9d60108070 2024-06-10T02:42:19.455 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:18.665+0000 7f9d6545e700 1 -- 172.21.15.52:0/480026580 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d6019c6a0 con 0x7f9d60108070 2024-06-10T02:42:19.455 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:18.665+0000 7f9d6545e700 1 -- 172.21.15.52:0/480026580 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d6019f390 con 0x7f9d60108070 2024-06-10T02:42:19.455 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:18.666+0000 7f9d47fff700 1 -- 172.21.15.52:0/480026580 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f9d50022020 con 0x7f9d60108070 2024-06-10T02:42:19.455 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:18.666+0000 7f9d6545e700 1 -- 172.21.15.52:0/480026580 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d60063af0 con 0x7f9d60108070 2024-06-10T02:42:19.455 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:18.666+0000 7f9d47fff700 1 --2- 172.21.15.52:0/480026580 >> [v2:172.21.15.52:6800/896317057,v1:172.21.15.52:6801/896317057] conn(0x7f9d480390a0 0x7f9d4803b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:42:19.455 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:18.666+0000 7f9d47fff700 1 -- 172.21.15.52:0/480026580 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f9d50011e00 con 0x7f9d60108070 2024-06-10T02:42:19.455 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:18.666+0000 7f9d5e7fc700 1 --2- 172.21.15.52:0/480026580 >> [v2:172.21.15.52:6800/896317057,v1:172.21.15.52:6801/896317057] conn(0x7f9d480390a0 0x7f9d4803b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:42:19.455 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:18.667+0000 7f9d5e7fc700 1 --2- 172.21.15.52:0/480026580 >> [v2:172.21.15.52:6800/896317057,v1:172.21.15.52:6801/896317057] conn(0x7f9d480390a0 0x7f9d4803b560 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9d54006fd0 tx=0x7f9d54006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:42:19.456 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:18.672+0000 7f9d47fff700 1 -- 172.21.15.52:0/480026580 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f9d5002cdf0 con 0x7f9d60108070 2024-06-10T02:42:19.456 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:18.882+0000 7f9d6545e700 1 -- 172.21.15.52:0/480026580 --> [v2:172.21.15.52:6800/896317057,v1:172.21.15.52:6801/896317057] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7f9d60104810 con 0x7f9d480390a0 2024-06-10T02:42:19.456 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:18.883+0000 7f9d47fff700 1 -- 172.21.15.52:0/480026580 <== mgr.14118 v2:172.21.15.52:6800/896317057 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7f9d60104810 con 0x7f9d480390a0 2024-06-10T02:42:19.456 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:18.885+0000 7f9d6545e700 1 -- 172.21.15.52:0/480026580 >> [v2:172.21.15.52:6800/896317057,v1:172.21.15.52:6801/896317057] conn(0x7f9d480390a0 msgr2=0x7f9d4803b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:42:19.456 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:18.885+0000 7f9d6545e700 1 --2- 172.21.15.52:0/480026580 >> [v2:172.21.15.52:6800/896317057,v1:172.21.15.52:6801/896317057] conn(0x7f9d480390a0 0x7f9d4803b560 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9d54006fd0 tx=0x7f9d54006e40 comp rx=0 tx=0).stop 2024-06-10T02:42:19.456 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:18.885+0000 7f9d6545e700 1 -- 172.21.15.52:0/480026580 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9d60108070 msgr2=0x7f9d6019bed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:42:19.456 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:18.885+0000 7f9d6545e700 1 --2- 172.21.15.52:0/480026580 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9d60108070 0x7f9d6019bed0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f9d50000c00 tx=0x7f9d50011c10 comp rx=0 tx=0).stop 2024-06-10T02:42:19.456 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:18.885+0000 7f9d6545e700 1 -- 172.21.15.52:0/480026580 shutdown_connections 2024-06-10T02:42:19.456 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:18.885+0000 7f9d6545e700 1 --2- 172.21.15.52:0/480026580 >> [v2:172.21.15.52:6800/896317057,v1:172.21.15.52:6801/896317057] conn(0x7f9d480390a0 0x7f9d4803b560 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:42:19.456 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:18.885+0000 7f9d6545e700 1 --2- 172.21.15.52:0/480026580 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9d60108070 0x7f9d6019bed0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:42:19.457 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:18.885+0000 7f9d6545e700 1 -- 172.21.15.52:0/480026580 >> 172.21.15.52:0/480026580 conn(0x7f9d601035f0 msgr2=0x7f9d60104100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:42:19.457 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:18.885+0000 7f9d6545e700 1 -- 172.21.15.52:0/480026580 shutdown_connections 2024-06-10T02:42:19.457 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:18.885+0000 7f9d6545e700 1 -- 172.21.15.52:0/480026580 wait complete. 2024-06-10T02:42:19.457 INFO:teuthology.orchestra.run.smithi052.stdout:Generating ssh key... 2024-06-10T02:42:20.102 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:42:19 smithi052 ceph-mon[32362]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T02:42:20.675 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 10 02:42:20 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mgr-x[32595]: Generating public/private rsa key pair. 2024-06-10T02:42:20.675 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 10 02:42:20 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mgr-x[32595]: Your identification has been saved in /tmp/tmpu3k53gwz/key. 2024-06-10T02:42:20.675 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 10 02:42:20 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mgr-x[32595]: Your public key has been saved in /tmp/tmpu3k53gwz/key.pub. 2024-06-10T02:42:20.675 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 10 02:42:20 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mgr-x[32595]: The key fingerprint is: 2024-06-10T02:42:20.675 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 10 02:42:20 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mgr-x[32595]: SHA256:CcIRrbb82SDhrU3LN1gV3+R+0fijTctHRvE3HZINl2I ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968 2024-06-10T02:42:20.675 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 10 02:42:20 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mgr-x[32595]: The key's randomart image is: 2024-06-10T02:42:20.675 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 10 02:42:20 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mgr-x[32595]: +---[RSA 3072]----+ 2024-06-10T02:42:20.675 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 10 02:42:20 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mgr-x[32595]: | oo .+..| 2024-06-10T02:42:20.675 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 10 02:42:20 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mgr-x[32595]: | . .. . Eo= | 2024-06-10T02:42:20.675 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 10 02:42:20 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mgr-x[32595]: | o.. o.+o.*| 2024-06-10T02:42:20.676 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 10 02:42:20 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mgr-x[32595]: | +. . .. . +o*| 2024-06-10T02:42:20.676 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 10 02:42:20 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mgr-x[32595]: | + + S. . o+| 2024-06-10T02:42:20.676 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 10 02:42:20 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mgr-x[32595]: | = + . .o=| 2024-06-10T02:42:20.676 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 10 02:42:20 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mgr-x[32595]: | B B =+o| 2024-06-10T02:42:20.676 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 10 02:42:20 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mgr-x[32595]: | . B + . +.| 2024-06-10T02:42:20.676 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 10 02:42:20 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mgr-x[32595]: | . . .| 2024-06-10T02:42:20.676 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 10 02:42:20 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mgr-x[32595]: +----[SHA256]-----+ 2024-06-10T02:42:20.973 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:42:20 smithi052 ceph-mon[32362]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T02:42:20.973 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:42:20 smithi052 ceph-mon[32362]: Generating ssh key... 2024-06-10T02:42:20.973 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:42:20 smithi052 ceph-mon[32362]: from='mgr.14118 172.21.15.52:0/1358068050' entity='mgr.x' 2024-06-10T02:42:20.973 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:42:20 smithi052 ceph-mon[32362]: from='mgr.14118 172.21.15.52:0/1358068050' entity='mgr.x' 2024-06-10T02:42:20.999 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:19.886+0000 7ff60159e700 1 Processor -- start 2024-06-10T02:42:20.999 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:19.886+0000 7ff60159e700 1 -- start start 2024-06-10T02:42:20.999 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:19.886+0000 7ff60159e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff5fc07e6f0 0x7ff5fc07cba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:42:20.999 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:19.886+0000 7ff60159e700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ff5fc07d170 con 0x7ff5fc07e6f0 2024-06-10T02:42:20.999 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:19.886+0000 7ff5faffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff5fc07e6f0 0x7ff5fc07cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:42:21.000 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:19.886+0000 7ff5faffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff5fc07e6f0 0x7ff5fc07cba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37514/0 (socket says 172.21.15.52:37514) 2024-06-10T02:42:21.000 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:19.886+0000 7ff5faffd700 1 -- 172.21.15.52:0/3580390813 learned_addr learned my addr 172.21.15.52:0/3580390813 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:42:21.000 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:19.886+0000 7ff5faffd700 1 -- 172.21.15.52:0/3580390813 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5fc07d2f0 con 0x7ff5fc07e6f0 2024-06-10T02:42:21.000 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:19.886+0000 7ff5faffd700 1 --2- 172.21.15.52:0/3580390813 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff5fc07e6f0 0x7ff5fc07cba0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7ff5ec0098a0 tx=0x7ff5ec009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac97409749c8aedf server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:42:21.000 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:19.887+0000 7ff5f9ffb700 1 -- 172.21.15.52:0/3580390813 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff5ec004160 con 0x7ff5fc07e6f0 2024-06-10T02:42:21.000 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:19.887+0000 7ff5f9ffb700 1 -- 172.21.15.52:0/3580390813 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff5ec004300 con 0x7ff5fc07e6f0 2024-06-10T02:42:21.000 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:19.887+0000 7ff5f9ffb700 1 -- 172.21.15.52:0/3580390813 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff5ec011b50 con 0x7ff5fc07e6f0 2024-06-10T02:42:21.000 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:19.887+0000 7ff60159e700 1 -- 172.21.15.52:0/3580390813 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff5fc07e6f0 msgr2=0x7ff5fc07cba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:42:21.000 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:19.887+0000 7ff60159e700 1 --2- 172.21.15.52:0/3580390813 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff5fc07e6f0 0x7ff5fc07cba0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7ff5ec0098a0 tx=0x7ff5ec009bb0 comp rx=0 tx=0).stop 2024-06-10T02:42:21.000 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:19.887+0000 7ff60159e700 1 -- 172.21.15.52:0/3580390813 shutdown_connections 2024-06-10T02:42:21.000 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:19.887+0000 7ff60159e700 1 --2- 172.21.15.52:0/3580390813 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff5fc07e6f0 0x7ff5fc07cba0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:42:21.000 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:19.887+0000 7ff60159e700 1 -- 172.21.15.52:0/3580390813 >> 172.21.15.52:0/3580390813 conn(0x7ff5fc103560 msgr2=0x7ff5fc1059a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:42:21.000 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:19.887+0000 7ff60159e700 1 -- 172.21.15.52:0/3580390813 shutdown_connections 2024-06-10T02:42:21.001 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:19.887+0000 7ff60159e700 1 -- 172.21.15.52:0/3580390813 wait complete. 2024-06-10T02:42:21.001 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:19.888+0000 7ff60159e700 1 Processor -- start 2024-06-10T02:42:21.001 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:19.888+0000 7ff60159e700 1 -- start start 2024-06-10T02:42:21.001 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:19.888+0000 7ff60159e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff5fc07e6f0 0x7ff5fc19e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:42:21.001 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:19.889+0000 7ff60159e700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ff5fc10e570 con 0x7ff5fc07e6f0 2024-06-10T02:42:21.001 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:19.889+0000 7ff5faffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff5fc07e6f0 0x7ff5fc19e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:42:21.001 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:19.889+0000 7ff5faffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff5fc07e6f0 0x7ff5fc19e100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37526/0 (socket says 172.21.15.52:37526) 2024-06-10T02:42:21.001 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:19.889+0000 7ff5faffd700 1 -- 172.21.15.52:0/1136756332 learned_addr learned my addr 172.21.15.52:0/1136756332 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:42:21.001 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:19.889+0000 7ff5faffd700 1 -- 172.21.15.52:0/1136756332 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5fc19e640 con 0x7ff5fc07e6f0 2024-06-10T02:42:21.001 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:19.889+0000 7ff5faffd700 1 --2- 172.21.15.52:0/1136756332 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff5fc07e6f0 0x7ff5fc19e100 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ff5ec011670 tx=0x7ff5ec011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:42:21.001 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:19.889+0000 7ff5e3fff700 1 -- 172.21.15.52:0/1136756332 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff5ec0039c0 con 0x7ff5fc07e6f0 2024-06-10T02:42:21.001 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:19.889+0000 7ff60159e700 1 -- 172.21.15.52:0/1136756332 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5fc19e8d0 con 0x7ff5fc07e6f0 2024-06-10T02:42:21.002 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:19.889+0000 7ff5e3fff700 1 -- 172.21.15.52:0/1136756332 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff5ec011e50 con 0x7ff5fc07e6f0 2024-06-10T02:42:21.002 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:19.889+0000 7ff5e3fff700 1 -- 172.21.15.52:0/1136756332 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff5ec01a5c0 con 0x7ff5fc07e6f0 2024-06-10T02:42:21.002 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:19.889+0000 7ff60159e700 1 -- 172.21.15.52:0/1136756332 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5fc1a15c0 con 0x7ff5fc07e6f0 2024-06-10T02:42:21.002 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:19.890+0000 7ff5e3fff700 1 -- 172.21.15.52:0/1136756332 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7ff5ec01a8c0 con 0x7ff5fc07e6f0 2024-06-10T02:42:21.002 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:19.890+0000 7ff5e3fff700 1 --2- 172.21.15.52:0/1136756332 >> [v2:172.21.15.52:6800/896317057,v1:172.21.15.52:6801/896317057] conn(0x7ff5e4039000 0x7ff5e403b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:42:21.002 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:19.890+0000 7ff60159e700 1 -- 172.21.15.52:0/1136756332 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5fc063af0 con 0x7ff5fc07e6f0 2024-06-10T02:42:21.002 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:19.890+0000 7ff5e3fff700 1 -- 172.21.15.52:0/1136756332 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7ff5ec04d580 con 0x7ff5fc07e6f0 2024-06-10T02:42:21.002 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:19.891+0000 7ff5fa7fc700 1 --2- 172.21.15.52:0/1136756332 >> [v2:172.21.15.52:6800/896317057,v1:172.21.15.52:6801/896317057] conn(0x7ff5e4039000 0x7ff5e403b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:42:21.002 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:19.891+0000 7ff5fa7fc700 1 --2- 172.21.15.52:0/1136756332 >> [v2:172.21.15.52:6800/896317057,v1:172.21.15.52:6801/896317057] conn(0x7ff5e4039000 0x7ff5e403b4c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7ff5f0006fd0 tx=0x7ff5f0006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:42:21.002 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:19.896+0000 7ff5e3fff700 1 -- 172.21.15.52:0/1136756332 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7ff5ec02d910 con 0x7ff5fc07e6f0 2024-06-10T02:42:21.002 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:20.106+0000 7ff60159e700 1 -- 172.21.15.52:0/1136756332 --> [v2:172.21.15.52:6800/896317057,v1:172.21.15.52:6801/896317057] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7ff5fc1047c0 con 0x7ff5e4039000 2024-06-10T02:42:21.002 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:20.382+0000 7ff5e3fff700 1 -- 172.21.15.52:0/1136756332 <== mgr.14118 v2:172.21.15.52:6800/896317057 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7ff5fc1047c0 con 0x7ff5e4039000 2024-06-10T02:42:21.003 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:20.385+0000 7ff60159e700 1 -- 172.21.15.52:0/1136756332 >> [v2:172.21.15.52:6800/896317057,v1:172.21.15.52:6801/896317057] conn(0x7ff5e4039000 msgr2=0x7ff5e403b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:42:21.003 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:20.385+0000 7ff60159e700 1 --2- 172.21.15.52:0/1136756332 >> [v2:172.21.15.52:6800/896317057,v1:172.21.15.52:6801/896317057] conn(0x7ff5e4039000 0x7ff5e403b4c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7ff5f0006fd0 tx=0x7ff5f0006e40 comp rx=0 tx=0).stop 2024-06-10T02:42:21.003 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:20.385+0000 7ff60159e700 1 -- 172.21.15.52:0/1136756332 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff5fc07e6f0 msgr2=0x7ff5fc19e100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:42:21.003 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:20.385+0000 7ff60159e700 1 --2- 172.21.15.52:0/1136756332 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff5fc07e6f0 0x7ff5fc19e100 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ff5ec011670 tx=0x7ff5ec011b60 comp rx=0 tx=0).stop 2024-06-10T02:42:21.003 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:20.385+0000 7ff60159e700 1 -- 172.21.15.52:0/1136756332 shutdown_connections 2024-06-10T02:42:21.003 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:20.385+0000 7ff60159e700 1 --2- 172.21.15.52:0/1136756332 >> [v2:172.21.15.52:6800/896317057,v1:172.21.15.52:6801/896317057] conn(0x7ff5e4039000 0x7ff5e403b4c0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:42:21.003 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:20.385+0000 7ff60159e700 1 --2- 172.21.15.52:0/1136756332 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff5fc07e6f0 0x7ff5fc19e100 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:42:21.003 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:20.385+0000 7ff60159e700 1 -- 172.21.15.52:0/1136756332 >> 172.21.15.52:0/1136756332 conn(0x7ff5fc103560 msgr2=0x7ff5fc1040b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:42:21.003 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:20.385+0000 7ff60159e700 1 -- 172.21.15.52:0/1136756332 shutdown_connections 2024-06-10T02:42:21.003 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:20.385+0000 7ff60159e700 1 -- 172.21.15.52:0/1136756332 wait complete. 2024-06-10T02:42:22.173 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCplfU1Jbyk6R5JXXM2hZMcXCEloEITPToJJkgS2OMIo3MFaHRT8Xt9FWIMeifAkphay+L1SKm2aJsDP+AIwNPZBFNOpbO4mEKyP5ABu+UN+ZbpmNmRvcCxpdM0WaY3pruFcDHEfcV3V7le1rRyK9AU1xD8HoXJNBRmZlN6kt43nyRI/B/Lh5k+gepOOdnstqJ353C5wgVZ0Om1Pyqz6q+tRS/w8ROfVeM59EPlXVM0koP0rzeRAnzSvk9CK0XbruSL0jZYW+BtEgRHSG2jrGDJHkWwV7hSVTpbJEnUmf5XrNaKuneOHjxhaJGHUmNn9xc4q1rUsZVBJzo8p5D+Ug+wRgi7LRGWJK7OTYLNd7G+OtrZoMFw3K1JjpQtVBLkMqVyzNGhu0pz/y0gcd9m1bjOwVUogX8Lp0bcfczOSyJ59hzSYXgFfakzNfdRBVpvxmex7EH8VczAHdjG+S4oSu/vmWa9wy65hPRemHNifWtwA/kv0i53fvlHJxledeMSRe8= ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968 2024-06-10T02:42:22.173 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:21.419+0000 7f11d42fc700 1 Processor -- start 2024-06-10T02:42:22.173 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:21.419+0000 7f11d42fc700 1 -- start start 2024-06-10T02:42:22.173 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:21.419+0000 7f11d42fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f11cc107fc0 0x7f11cc1083e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:42:22.173 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:21.419+0000 7f11d42fc700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f11cc1089b0 con 0x7f11cc107fc0 2024-06-10T02:42:22.173 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:21.420+0000 7f11d2098700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f11cc107fc0 0x7f11cc1083e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:42:22.173 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:21.420+0000 7f11d2098700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f11cc107fc0 0x7f11cc1083e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57566/0 (socket says 172.21.15.52:57566) 2024-06-10T02:42:22.174 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:21.420+0000 7f11d2098700 1 -- 172.21.15.52:0/721054305 learned_addr learned my addr 172.21.15.52:0/721054305 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:42:22.174 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:21.420+0000 7f11d2098700 1 -- 172.21.15.52:0/721054305 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11cc109170 con 0x7f11cc107fc0 2024-06-10T02:42:22.174 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:21.420+0000 7f11d2098700 1 --2- 172.21.15.52:0/721054305 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f11cc107fc0 0x7f11cc1083e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f11bc009e80 tx=0x7f11bc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a1a645e4cfe1db7 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:42:22.174 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:21.420+0000 7f11d1096700 1 -- 172.21.15.52:0/721054305 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f11bc017070 con 0x7f11cc107fc0 2024-06-10T02:42:22.174 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:21.420+0000 7f11d1096700 1 -- 172.21.15.52:0/721054305 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f11bc00b8c0 con 0x7f11cc107fc0 2024-06-10T02:42:22.174 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:21.420+0000 7f11d1096700 1 -- 172.21.15.52:0/721054305 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f11bc0045c0 con 0x7f11cc107fc0 2024-06-10T02:42:22.174 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:21.421+0000 7f11d42fc700 1 -- 172.21.15.52:0/721054305 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f11cc107fc0 msgr2=0x7f11cc1083e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:42:22.174 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:21.421+0000 7f11d42fc700 1 --2- 172.21.15.52:0/721054305 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f11cc107fc0 0x7f11cc1083e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f11bc009e80 tx=0x7f11bc00b330 comp rx=0 tx=0).stop 2024-06-10T02:42:22.174 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:21.421+0000 7f11d42fc700 1 -- 172.21.15.52:0/721054305 shutdown_connections 2024-06-10T02:42:22.174 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:21.421+0000 7f11d42fc700 1 --2- 172.21.15.52:0/721054305 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f11cc107fc0 0x7f11cc1083e0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:42:22.174 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:21.421+0000 7f11d42fc700 1 -- 172.21.15.52:0/721054305 >> 172.21.15.52:0/721054305 conn(0x7f11cc103560 msgr2=0x7f11cc1059a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:42:22.174 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:21.421+0000 7f11d42fc700 1 -- 172.21.15.52:0/721054305 shutdown_connections 2024-06-10T02:42:22.175 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:21.421+0000 7f11d42fc700 1 -- 172.21.15.52:0/721054305 wait complete. 2024-06-10T02:42:22.175 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:21.422+0000 7f11d42fc700 1 Processor -- start 2024-06-10T02:42:22.175 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:21.422+0000 7f11d42fc700 1 -- start start 2024-06-10T02:42:22.178 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:21.422+0000 7f11d42fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f11cc107fc0 0x7f11cc19bea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:42:22.178 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:21.422+0000 7f11d42fc700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f11cc10c520 con 0x7f11cc107fc0 2024-06-10T02:42:22.178 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:21.422+0000 7f11d2098700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f11cc107fc0 0x7f11cc19bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:42:22.179 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:21.422+0000 7f11d2098700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f11cc107fc0 0x7f11cc19bea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57576/0 (socket says 172.21.15.52:57576) 2024-06-10T02:42:22.179 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:21.422+0000 7f11d2098700 1 -- 172.21.15.52:0/4066448283 learned_addr learned my addr 172.21.15.52:0/4066448283 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:42:22.179 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:21.422+0000 7f11d2098700 1 -- 172.21.15.52:0/4066448283 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11cc19c3e0 con 0x7f11cc107fc0 2024-06-10T02:42:22.179 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:21.423+0000 7f11d2098700 1 --2- 172.21.15.52:0/4066448283 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f11cc107fc0 0x7f11cc19bea0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f11bc0072a0 tx=0x7f11bc00bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:42:22.179 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:21.423+0000 7f11cb7fe700 1 -- 172.21.15.52:0/4066448283 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f11bc020020 con 0x7f11cc107fc0 2024-06-10T02:42:22.179 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:21.423+0000 7f11cb7fe700 1 -- 172.21.15.52:0/4066448283 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f11bc003b50 con 0x7f11cc107fc0 2024-06-10T02:42:22.179 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:21.423+0000 7f11d42fc700 1 -- 172.21.15.52:0/4066448283 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11cc19c670 con 0x7f11cc107fc0 2024-06-10T02:42:22.179 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:21.423+0000 7f11cb7fe700 1 -- 172.21.15.52:0/4066448283 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f11bc017040 con 0x7f11cc107fc0 2024-06-10T02:42:22.179 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:21.423+0000 7f11d42fc700 1 -- 172.21.15.52:0/4066448283 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11cc19f360 con 0x7f11cc107fc0 2024-06-10T02:42:22.179 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:21.424+0000 7f11d42fc700 1 -- 172.21.15.52:0/4066448283 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11cc063af0 con 0x7f11cc107fc0 2024-06-10T02:42:22.179 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:21.424+0000 7f11cb7fe700 1 -- 172.21.15.52:0/4066448283 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f11bc023020 con 0x7f11cc107fc0 2024-06-10T02:42:22.179 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:21.424+0000 7f11cb7fe700 1 --2- 172.21.15.52:0/4066448283 >> [v2:172.21.15.52:6800/896317057,v1:172.21.15.52:6801/896317057] conn(0x7f11b4039050 0x7f11b403b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:42:22.179 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:21.424+0000 7f11cb7fe700 1 -- 172.21.15.52:0/4066448283 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f11bc00f050 con 0x7f11cc107fc0 2024-06-10T02:42:22.180 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:21.424+0000 7f11d1897700 1 --2- 172.21.15.52:0/4066448283 >> [v2:172.21.15.52:6800/896317057,v1:172.21.15.52:6801/896317057] conn(0x7f11b4039050 0x7f11b403b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:42:22.180 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:21.425+0000 7f11d1897700 1 --2- 172.21.15.52:0/4066448283 >> [v2:172.21.15.52:6800/896317057,v1:172.21.15.52:6801/896317057] conn(0x7f11b4039050 0x7f11b403b510 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f11c0006fd0 tx=0x7f11c0006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:42:22.180 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:21.429+0000 7f11cb7fe700 1 -- 172.21.15.52:0/4066448283 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f11bc02a020 con 0x7f11cc107fc0 2024-06-10T02:42:22.180 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:21.637+0000 7f11d42fc700 1 -- 172.21.15.52:0/4066448283 --> [v2:172.21.15.52:6800/896317057,v1:172.21.15.52:6801/896317057] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7f11cc1046d0 con 0x7f11b4039050 2024-06-10T02:42:22.180 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:21.637+0000 7f11cb7fe700 1 -- 172.21.15.52:0/4066448283 <== mgr.14118 v2:172.21.15.52:6800/896317057 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7f11cc1046d0 con 0x7f11b4039050 2024-06-10T02:42:22.180 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:21.640+0000 7f11d42fc700 1 -- 172.21.15.52:0/4066448283 >> [v2:172.21.15.52:6800/896317057,v1:172.21.15.52:6801/896317057] conn(0x7f11b4039050 msgr2=0x7f11b403b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:42:22.180 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:21.640+0000 7f11d42fc700 1 --2- 172.21.15.52:0/4066448283 >> [v2:172.21.15.52:6800/896317057,v1:172.21.15.52:6801/896317057] conn(0x7f11b4039050 0x7f11b403b510 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f11c0006fd0 tx=0x7f11c0006e40 comp rx=0 tx=0).stop 2024-06-10T02:42:22.180 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:21.640+0000 7f11d42fc700 1 -- 172.21.15.52:0/4066448283 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f11cc107fc0 msgr2=0x7f11cc19bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:42:22.180 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:21.640+0000 7f11d42fc700 1 --2- 172.21.15.52:0/4066448283 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f11cc107fc0 0x7f11cc19bea0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f11bc0072a0 tx=0x7f11bc00bd50 comp rx=0 tx=0).stop 2024-06-10T02:42:22.180 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:21.640+0000 7f11d42fc700 1 -- 172.21.15.52:0/4066448283 shutdown_connections 2024-06-10T02:42:22.180 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:21.640+0000 7f11d42fc700 1 --2- 172.21.15.52:0/4066448283 >> [v2:172.21.15.52:6800/896317057,v1:172.21.15.52:6801/896317057] conn(0x7f11b4039050 0x7f11b403b510 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:42:22.180 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:21.640+0000 7f11d42fc700 1 --2- 172.21.15.52:0/4066448283 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f11cc107fc0 0x7f11cc19bea0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:42:22.180 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:21.640+0000 7f11d42fc700 1 -- 172.21.15.52:0/4066448283 >> 172.21.15.52:0/4066448283 conn(0x7f11cc103560 msgr2=0x7f11cc103fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:42:22.181 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:21.640+0000 7f11d42fc700 1 -- 172.21.15.52:0/4066448283 shutdown_connections 2024-06-10T02:42:22.181 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:21.640+0000 7f11d42fc700 1 -- 172.21.15.52:0/4066448283 wait complete. 2024-06-10T02:42:22.181 INFO:teuthology.orchestra.run.smithi052.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-06-10T02:42:22.181 INFO:teuthology.orchestra.run.smithi052.stdout:Adding key to root@localhost authorized_keys... 2024-06-10T02:42:22.181 INFO:teuthology.orchestra.run.smithi052.stdout:Adding host smithi052... 2024-06-10T02:42:22.473 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:42:22 smithi052 ceph-mon[32362]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T02:42:23.472 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:42:23 smithi052 ceph-mon[32362]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi052", "addr": "172.21.15.52", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T02:42:24.722 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:42:24 smithi052 ceph-mon[32362]: Deploying cephadm binary to smithi052 2024-06-10T02:42:27.823 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout Added host 'smithi052' with addr '172.21.15.52' 2024-06-10T02:42:27.823 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:22.623+0000 7f91ba85e700 1 Processor -- start 2024-06-10T02:42:27.823 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:22.623+0000 7f91ba85e700 1 -- start start 2024-06-10T02:42:27.823 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:22.623+0000 7f91ba85e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f91b407e6f0 0x7f91b407cba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:42:27.823 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:22.623+0000 7f91ba85e700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f91b407d170 con 0x7f91b407e6f0 2024-06-10T02:42:27.823 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:22.624+0000 7f91b3fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f91b407e6f0 0x7f91b407cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:42:27.823 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:22.624+0000 7f91b3fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f91b407e6f0 0x7f91b407cba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57590/0 (socket says 172.21.15.52:57590) 2024-06-10T02:42:27.823 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:22.624+0000 7f91b3fff700 1 -- 172.21.15.52:0/4222929080 learned_addr learned my addr 172.21.15.52:0/4222929080 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:42:27.823 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:22.624+0000 7f91b3fff700 1 -- 172.21.15.52:0/4222929080 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91b407d2f0 con 0x7f91b407e6f0 2024-06-10T02:42:27.823 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:22.624+0000 7f91b3fff700 1 --2- 172.21.15.52:0/4222929080 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f91b407e6f0 0x7f91b407cba0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f91a40098a0 tx=0x7f91a4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=721ea00ca8c2666a server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:42:27.823 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:22.624+0000 7f91b2ffd700 1 -- 172.21.15.52:0/4222929080 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f91a4004160 con 0x7f91b407e6f0 2024-06-10T02:42:27.824 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:22.624+0000 7f91b2ffd700 1 -- 172.21.15.52:0/4222929080 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f91a4004300 con 0x7f91b407e6f0 2024-06-10T02:42:27.824 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:22.624+0000 7f91b2ffd700 1 -- 172.21.15.52:0/4222929080 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f91a4011b50 con 0x7f91b407e6f0 2024-06-10T02:42:27.824 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:22.625+0000 7f91ba85e700 1 -- 172.21.15.52:0/4222929080 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f91b407e6f0 msgr2=0x7f91b407cba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:42:27.824 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:22.625+0000 7f91ba85e700 1 --2- 172.21.15.52:0/4222929080 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f91b407e6f0 0x7f91b407cba0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f91a40098a0 tx=0x7f91a4009bb0 comp rx=0 tx=0).stop 2024-06-10T02:42:27.824 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:22.625+0000 7f91ba85e700 1 -- 172.21.15.52:0/4222929080 shutdown_connections 2024-06-10T02:42:27.824 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:22.625+0000 7f91ba85e700 1 --2- 172.21.15.52:0/4222929080 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f91b407e6f0 0x7f91b407cba0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:42:27.824 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:22.625+0000 7f91ba85e700 1 -- 172.21.15.52:0/4222929080 >> 172.21.15.52:0/4222929080 conn(0x7f91b41035a0 msgr2=0x7f91b41059e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:42:27.824 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:22.625+0000 7f91ba85e700 1 -- 172.21.15.52:0/4222929080 shutdown_connections 2024-06-10T02:42:27.824 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:22.625+0000 7f91ba85e700 1 -- 172.21.15.52:0/4222929080 wait complete. 2024-06-10T02:42:27.824 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:22.626+0000 7f91ba85e700 1 Processor -- start 2024-06-10T02:42:27.824 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:22.626+0000 7f91ba85e700 1 -- start start 2024-06-10T02:42:27.824 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:22.626+0000 7f91ba85e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f91b407e6f0 0x7f91b419e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:42:27.824 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:22.626+0000 7f91ba85e700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f91b410e550 con 0x7f91b407e6f0 2024-06-10T02:42:27.824 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:22.626+0000 7f91b3fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f91b407e6f0 0x7f91b419e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:42:27.825 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:22.626+0000 7f91b3fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f91b407e6f0 0x7f91b419e0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57592/0 (socket says 172.21.15.52:57592) 2024-06-10T02:42:27.825 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:22.626+0000 7f91b3fff700 1 -- 172.21.15.52:0/1693326726 learned_addr learned my addr 172.21.15.52:0/1693326726 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:42:27.825 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:22.626+0000 7f91b3fff700 1 -- 172.21.15.52:0/1693326726 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91b419e620 con 0x7f91b407e6f0 2024-06-10T02:42:27.825 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:22.627+0000 7f91b3fff700 1 --2- 172.21.15.52:0/1693326726 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f91b407e6f0 0x7f91b419e0e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f91a4011b60 tx=0x7f91a4011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:42:27.825 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:22.627+0000 7f91b17fa700 1 -- 172.21.15.52:0/1693326726 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f91a4003b20 con 0x7f91b407e6f0 2024-06-10T02:42:27.825 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:22.627+0000 7f91b17fa700 1 -- 172.21.15.52:0/1693326726 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f91a4003cc0 con 0x7f91b407e6f0 2024-06-10T02:42:27.825 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:22.627+0000 7f91ba85e700 1 -- 172.21.15.52:0/1693326726 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91b419e8b0 con 0x7f91b407e6f0 2024-06-10T02:42:27.825 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:22.627+0000 7f91b17fa700 1 -- 172.21.15.52:0/1693326726 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f91a4024840 con 0x7f91b407e6f0 2024-06-10T02:42:27.825 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:22.627+0000 7f91ba85e700 1 -- 172.21.15.52:0/1693326726 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91b41a1550 con 0x7f91b407e6f0 2024-06-10T02:42:27.825 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:22.628+0000 7f91b17fa700 1 -- 172.21.15.52:0/1693326726 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f91a4022020 con 0x7f91b407e6f0 2024-06-10T02:42:27.825 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:22.628+0000 7f91ba85e700 1 -- 172.21.15.52:0/1693326726 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91b4063af0 con 0x7f91b407e6f0 2024-06-10T02:42:27.825 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:22.628+0000 7f91b17fa700 1 --2- 172.21.15.52:0/1693326726 >> [v2:172.21.15.52:6800/896317057,v1:172.21.15.52:6801/896317057] conn(0x7f919c039050 0x7f919c03b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:42:27.825 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:22.628+0000 7f91b17fa700 1 -- 172.21.15.52:0/1693326726 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f91a4011e00 con 0x7f91b407e6f0 2024-06-10T02:42:27.826 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:22.628+0000 7f91b37fe700 1 --2- 172.21.15.52:0/1693326726 >> [v2:172.21.15.52:6800/896317057,v1:172.21.15.52:6801/896317057] conn(0x7f919c039050 0x7f919c03b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:42:27.826 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:22.629+0000 7f91b37fe700 1 --2- 172.21.15.52:0/1693326726 >> [v2:172.21.15.52:6800/896317057,v1:172.21.15.52:6801/896317057] conn(0x7f919c039050 0x7f919c03b510 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f91a8006fd0 tx=0x7f91a8006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:42:27.826 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:22.633+0000 7f91b17fa700 1 -- 172.21.15.52:0/1693326726 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f91a400ba30 con 0x7f91b407e6f0 2024-06-10T02:42:27.826 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:22.842+0000 7f91ba85e700 1 -- 172.21.15.52:0/1693326726 --> [v2:172.21.15.52:6800/896317057,v1:172.21.15.52:6801/896317057] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi052", "addr": "172.21.15.52", "target": ["mon-mgr", ""]}) -- 0x7f91b41047d0 con 0x7f919c039050 2024-06-10T02:42:27.826 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:27.237+0000 7f91b17fa700 1 -- 172.21.15.52:0/1693326726 <== mgr.14118 v2:172.21.15.52:6800/896317057 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f91b41047d0 con 0x7f919c039050 2024-06-10T02:42:27.826 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:27.240+0000 7f91ba85e700 1 -- 172.21.15.52:0/1693326726 >> [v2:172.21.15.52:6800/896317057,v1:172.21.15.52:6801/896317057] conn(0x7f919c039050 msgr2=0x7f919c03b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:42:27.826 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:27.240+0000 7f91ba85e700 1 --2- 172.21.15.52:0/1693326726 >> [v2:172.21.15.52:6800/896317057,v1:172.21.15.52:6801/896317057] conn(0x7f919c039050 0x7f919c03b510 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f91a8006fd0 tx=0x7f91a8006e40 comp rx=0 tx=0).stop 2024-06-10T02:42:27.826 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:27.240+0000 7f91ba85e700 1 -- 172.21.15.52:0/1693326726 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f91b407e6f0 msgr2=0x7f91b419e0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:42:27.826 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:27.240+0000 7f91ba85e700 1 --2- 172.21.15.52:0/1693326726 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f91b407e6f0 0x7f91b419e0e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f91a4011b60 tx=0x7f91a4011c40 comp rx=0 tx=0).stop 2024-06-10T02:42:27.826 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:27.240+0000 7f91ba85e700 1 -- 172.21.15.52:0/1693326726 shutdown_connections 2024-06-10T02:42:27.826 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:27.240+0000 7f91ba85e700 1 --2- 172.21.15.52:0/1693326726 >> [v2:172.21.15.52:6800/896317057,v1:172.21.15.52:6801/896317057] conn(0x7f919c039050 0x7f919c03b510 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:42:27.826 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:27.240+0000 7f91ba85e700 1 --2- 172.21.15.52:0/1693326726 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f91b407e6f0 0x7f91b419e0e0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:42:27.827 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:27.240+0000 7f91ba85e700 1 -- 172.21.15.52:0/1693326726 >> 172.21.15.52:0/1693326726 conn(0x7f91b41035a0 msgr2=0x7f91b41040c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:42:27.827 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:27.240+0000 7f91ba85e700 1 -- 172.21.15.52:0/1693326726 shutdown_connections 2024-06-10T02:42:27.827 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:27.240+0000 7f91ba85e700 1 -- 172.21.15.52:0/1693326726 wait complete. 2024-06-10T02:42:27.827 INFO:teuthology.orchestra.run.smithi052.stdout:Deploying unmanaged mon service... 2024-06-10T02:42:28.463 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:42:28 smithi052 ceph-mon[32362]: from='mgr.14118 172.21.15.52:0/1358068050' entity='mgr.x' 2024-06-10T02:42:28.464 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:42:28 smithi052 ceph-mon[32362]: Added host smithi052 2024-06-10T02:42:28.464 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:42:28 smithi052 ceph-mon[32362]: from='mgr.14118 172.21.15.52:0/1358068050' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T02:42:29.208 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-06-10T02:42:29.208 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:28.230+0000 7f73f63c5700 1 Processor -- start 2024-06-10T02:42:29.208 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:28.230+0000 7f73f63c5700 1 -- start start 2024-06-10T02:42:29.208 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:28.230+0000 7f73f63c5700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f73f0106e60 0x7f73f0107280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:42:29.208 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:28.230+0000 7f73f63c5700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f73f01077c0 con 0x7f73f0106e60 2024-06-10T02:42:29.208 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:28.231+0000 7f73effff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f73f0106e60 0x7f73f0107280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:42:29.208 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:28.231+0000 7f73effff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f73f0106e60 0x7f73f0107280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57594/0 (socket says 172.21.15.52:57594) 2024-06-10T02:42:29.208 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:28.231+0000 7f73effff700 1 -- 172.21.15.52:0/4276990042 learned_addr learned my addr 172.21.15.52:0/4276990042 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:42:29.208 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:28.231+0000 7f73effff700 1 -- 172.21.15.52:0/4276990042 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73f0107fc0 con 0x7f73f0106e60 2024-06-10T02:42:29.209 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:28.231+0000 7f73effff700 1 --2- 172.21.15.52:0/4276990042 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f73f0106e60 0x7f73f0107280 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f73e00098a0 tx=0x7f73e0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ba753b25e34502e server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:42:29.209 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:28.231+0000 7f73eeffd700 1 -- 172.21.15.52:0/4276990042 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f73e0004160 con 0x7f73f0106e60 2024-06-10T02:42:29.209 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:28.231+0000 7f73eeffd700 1 -- 172.21.15.52:0/4276990042 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f73e0004300 con 0x7f73f0106e60 2024-06-10T02:42:29.209 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:28.231+0000 7f73eeffd700 1 -- 172.21.15.52:0/4276990042 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f73e0011b50 con 0x7f73f0106e60 2024-06-10T02:42:29.209 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:28.232+0000 7f73f63c5700 1 -- 172.21.15.52:0/4276990042 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f73f0106e60 msgr2=0x7f73f0107280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:42:29.209 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:28.232+0000 7f73f63c5700 1 --2- 172.21.15.52:0/4276990042 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f73f0106e60 0x7f73f0107280 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f73e00098a0 tx=0x7f73e0009bb0 comp rx=0 tx=0).stop 2024-06-10T02:42:29.210 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:28.232+0000 7f73f63c5700 1 -- 172.21.15.52:0/4276990042 shutdown_connections 2024-06-10T02:42:29.210 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:28.232+0000 7f73f63c5700 1 --2- 172.21.15.52:0/4276990042 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f73f0106e60 0x7f73f0107280 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:42:29.210 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:28.232+0000 7f73f63c5700 1 -- 172.21.15.52:0/4276990042 >> 172.21.15.52:0/4276990042 conn(0x7f73f0102440 msgr2=0x7f73f0104860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:42:29.210 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:28.232+0000 7f73f63c5700 1 -- 172.21.15.52:0/4276990042 shutdown_connections 2024-06-10T02:42:29.210 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:28.232+0000 7f73f63c5700 1 -- 172.21.15.52:0/4276990042 wait complete. 2024-06-10T02:42:29.210 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:28.233+0000 7f73f63c5700 1 Processor -- start 2024-06-10T02:42:29.210 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:28.233+0000 7f73f63c5700 1 -- start start 2024-06-10T02:42:29.210 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:28.233+0000 7f73f63c5700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f73f0106e60 0x7f73f007c310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:42:29.210 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:28.233+0000 7f73f63c5700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f73f010b400 con 0x7f73f0106e60 2024-06-10T02:42:29.210 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:28.233+0000 7f73effff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f73f0106e60 0x7f73f007c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:42:29.210 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:28.233+0000 7f73effff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f73f0106e60 0x7f73f007c310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57606/0 (socket says 172.21.15.52:57606) 2024-06-10T02:42:29.210 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:28.233+0000 7f73effff700 1 -- 172.21.15.52:0/3624817992 learned_addr learned my addr 172.21.15.52:0/3624817992 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:42:29.211 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:28.234+0000 7f73effff700 1 -- 172.21.15.52:0/3624817992 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73f019ec10 con 0x7f73f0106e60 2024-06-10T02:42:29.211 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:28.234+0000 7f73effff700 1 --2- 172.21.15.52:0/3624817992 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f73f0106e60 0x7f73f007c310 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f73e0011670 tx=0x7f73e0011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:42:29.211 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:28.234+0000 7f73ed7fa700 1 -- 172.21.15.52:0/3624817992 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f73e00039c0 con 0x7f73f0106e60 2024-06-10T02:42:29.211 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:28.234+0000 7f73ed7fa700 1 -- 172.21.15.52:0/3624817992 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f73e0011e50 con 0x7f73f0106e60 2024-06-10T02:42:29.211 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:28.234+0000 7f73f63c5700 1 -- 172.21.15.52:0/3624817992 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73f019ede0 con 0x7f73f0106e60 2024-06-10T02:42:29.211 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:28.234+0000 7f73ed7fa700 1 -- 172.21.15.52:0/3624817992 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f73e001a5c0 con 0x7f73f0106e60 2024-06-10T02:42:29.211 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:28.234+0000 7f73f63c5700 1 -- 172.21.15.52:0/3624817992 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73f019f200 con 0x7f73f0106e60 2024-06-10T02:42:29.211 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:28.235+0000 7f73ed7fa700 1 -- 172.21.15.52:0/3624817992 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f73e001a8c0 con 0x7f73f0106e60 2024-06-10T02:42:29.211 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:28.235+0000 7f73f63c5700 1 -- 172.21.15.52:0/3624817992 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73f0063af0 con 0x7f73f0106e60 2024-06-10T02:42:29.211 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:28.235+0000 7f73ed7fa700 1 --2- 172.21.15.52:0/3624817992 >> [v2:172.21.15.52:6800/896317057,v1:172.21.15.52:6801/896317057] conn(0x7f73d8039000 0x7f73d803b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:42:29.211 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:28.235+0000 7f73ed7fa700 1 -- 172.21.15.52:0/3624817992 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f73e004d5b0 con 0x7f73f0106e60 2024-06-10T02:42:29.211 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:28.236+0000 7f73ef7fe700 1 --2- 172.21.15.52:0/3624817992 >> [v2:172.21.15.52:6800/896317057,v1:172.21.15.52:6801/896317057] conn(0x7f73d8039000 0x7f73d803b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:42:29.212 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:28.236+0000 7f73ef7fe700 1 --2- 172.21.15.52:0/3624817992 >> [v2:172.21.15.52:6800/896317057,v1:172.21.15.52:6801/896317057] conn(0x7f73d8039000 0x7f73d803b4c0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f73e4006fd0 tx=0x7f73e4006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:42:29.212 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:28.241+0000 7f73ed7fa700 1 -- 172.21.15.52:0/3624817992 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f73e0019c70 con 0x7f73f0106e60 2024-06-10T02:42:29.212 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:28.467+0000 7f73f63c5700 1 -- 172.21.15.52:0/3624817992 --> [v2:172.21.15.52:6800/896317057,v1:172.21.15.52:6801/896317057] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f73f01035f0 con 0x7f73d8039000 2024-06-10T02:42:29.212 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:28.472+0000 7f73ed7fa700 1 -- 172.21.15.52:0/3624817992 <== mgr.14118 v2:172.21.15.52:6800/896317057 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f73f01035f0 con 0x7f73d8039000 2024-06-10T02:42:29.212 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:28.475+0000 7f73f63c5700 1 -- 172.21.15.52:0/3624817992 >> [v2:172.21.15.52:6800/896317057,v1:172.21.15.52:6801/896317057] conn(0x7f73d8039000 msgr2=0x7f73d803b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:42:29.212 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:28.475+0000 7f73f63c5700 1 --2- 172.21.15.52:0/3624817992 >> [v2:172.21.15.52:6800/896317057,v1:172.21.15.52:6801/896317057] conn(0x7f73d8039000 0x7f73d803b4c0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f73e4006fd0 tx=0x7f73e4006e40 comp rx=0 tx=0).stop 2024-06-10T02:42:29.212 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:28.475+0000 7f73f63c5700 1 -- 172.21.15.52:0/3624817992 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f73f0106e60 msgr2=0x7f73f007c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:42:29.212 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:28.475+0000 7f73f63c5700 1 --2- 172.21.15.52:0/3624817992 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f73f0106e60 0x7f73f007c310 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f73e0011670 tx=0x7f73e0011b60 comp rx=0 tx=0).stop 2024-06-10T02:42:29.212 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:28.475+0000 7f73f63c5700 1 -- 172.21.15.52:0/3624817992 shutdown_connections 2024-06-10T02:42:29.212 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:28.475+0000 7f73f63c5700 1 --2- 172.21.15.52:0/3624817992 >> [v2:172.21.15.52:6800/896317057,v1:172.21.15.52:6801/896317057] conn(0x7f73d8039000 0x7f73d803b4c0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:42:29.212 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:28.475+0000 7f73f63c5700 1 --2- 172.21.15.52:0/3624817992 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f73f0106e60 0x7f73f007c310 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:42:29.212 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:28.475+0000 7f73f63c5700 1 -- 172.21.15.52:0/3624817992 >> 172.21.15.52:0/3624817992 conn(0x7f73f0102440 msgr2=0x7f73f0102ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:42:29.212 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:28.475+0000 7f73f63c5700 1 -- 172.21.15.52:0/3624817992 shutdown_connections 2024-06-10T02:42:29.212 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:28.475+0000 7f73f63c5700 1 -- 172.21.15.52:0/3624817992 wait complete. 2024-06-10T02:42:29.213 INFO:teuthology.orchestra.run.smithi052.stdout:Deploying unmanaged mgr service... 2024-06-10T02:42:29.719 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:42:29 smithi052 ceph-mon[32362]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T02:42:29.719 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:42:29 smithi052 ceph-mon[32362]: Saving service mon spec with placement count:5 2024-06-10T02:42:29.719 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:42:29 smithi052 ceph-mon[32362]: from='mgr.14118 172.21.15.52:0/1358068050' entity='mgr.x' 2024-06-10T02:42:31.055 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:42:30 smithi052 ceph-mon[32362]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T02:42:31.056 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:42:30 smithi052 ceph-mon[32362]: Saving service mgr spec with placement count:2 2024-06-10T02:42:31.056 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:42:30 smithi052 ceph-mon[32362]: from='mgr.14118 172.21.15.52:0/1358068050' entity='mgr.x' 2024-06-10T02:42:31.056 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:42:30 smithi052 ceph-mon[32362]: from='mgr.14118 172.21.15.52:0/1358068050' entity='mgr.x' 2024-06-10T02:42:31.263 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-06-10T02:42:31.263 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:29.713+0000 7feef362f700 1 Processor -- start 2024-06-10T02:42:31.263 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:29.713+0000 7feef362f700 1 -- start start 2024-06-10T02:42:31.263 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:29.714+0000 7feef362f700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feeec108a20 0x7feeec10ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:42:31.263 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:29.714+0000 7feef362f700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7feeec07bec0 con 0x7feeec108a20 2024-06-10T02:42:31.263 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:29.714+0000 7feef13cb700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feeec108a20 0x7feeec10ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:42:31.263 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:29.714+0000 7feef13cb700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feeec108a20 0x7feeec10ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57616/0 (socket says 172.21.15.52:57616) 2024-06-10T02:42:31.263 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:29.714+0000 7feef13cb700 1 -- 172.21.15.52:0/1341623279 learned_addr learned my addr 172.21.15.52:0/1341623279 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:42:31.263 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:29.714+0000 7feef13cb700 1 -- 172.21.15.52:0/1341623279 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feeec10b350 con 0x7feeec108a20 2024-06-10T02:42:31.264 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:29.714+0000 7feef13cb700 1 --2- 172.21.15.52:0/1341623279 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feeec108a20 0x7feeec10ae10 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7feee8009e80 tx=0x7feee800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee9f09e85c0e9d9c server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:42:31.264 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:29.714+0000 7feee3fff700 1 -- 172.21.15.52:0/1341623279 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feee8017070 con 0x7feeec108a20 2024-06-10T02:42:31.264 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:29.714+0000 7feee3fff700 1 -- 172.21.15.52:0/1341623279 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7feee800b8c0 con 0x7feeec108a20 2024-06-10T02:42:31.264 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:29.714+0000 7feee3fff700 1 -- 172.21.15.52:0/1341623279 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feee80045c0 con 0x7feeec108a20 2024-06-10T02:42:31.264 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:29.715+0000 7feef362f700 1 -- 172.21.15.52:0/1341623279 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feeec108a20 msgr2=0x7feeec10ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:42:31.264 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:29.715+0000 7feef362f700 1 --2- 172.21.15.52:0/1341623279 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feeec108a20 0x7feeec10ae10 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7feee8009e80 tx=0x7feee800b330 comp rx=0 tx=0).stop 2024-06-10T02:42:31.264 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:29.715+0000 7feef362f700 1 -- 172.21.15.52:0/1341623279 shutdown_connections 2024-06-10T02:42:31.264 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:29.715+0000 7feef362f700 1 --2- 172.21.15.52:0/1341623279 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feeec108a20 0x7feeec10ae10 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:42:31.264 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:29.715+0000 7feef362f700 1 -- 172.21.15.52:0/1341623279 >> 172.21.15.52:0/1341623279 conn(0x7feeec102450 msgr2=0x7feeec104890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:42:31.264 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:29.715+0000 7feef362f700 1 -- 172.21.15.52:0/1341623279 shutdown_connections 2024-06-10T02:42:31.264 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:29.715+0000 7feef362f700 1 -- 172.21.15.52:0/1341623279 wait complete. 2024-06-10T02:42:31.264 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:29.716+0000 7feef362f700 1 Processor -- start 2024-06-10T02:42:31.264 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:29.716+0000 7feef362f700 1 -- start start 2024-06-10T02:42:31.264 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:29.716+0000 7feef362f700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feeec108a20 0x7feeec19be60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:42:31.265 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:29.716+0000 7feef362f700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7feeec10c380 con 0x7feeec108a20 2024-06-10T02:42:31.265 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:29.716+0000 7feef13cb700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feeec108a20 0x7feeec19be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:42:31.265 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:29.716+0000 7feef13cb700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feeec108a20 0x7feeec19be60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57632/0 (socket says 172.21.15.52:57632) 2024-06-10T02:42:31.265 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:29.716+0000 7feef13cb700 1 -- 172.21.15.52:0/3707278317 learned_addr learned my addr 172.21.15.52:0/3707278317 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:42:31.265 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:29.716+0000 7feef13cb700 1 -- 172.21.15.52:0/3707278317 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feeec19c3a0 con 0x7feeec108a20 2024-06-10T02:42:31.265 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:29.717+0000 7feef13cb700 1 --2- 172.21.15.52:0/3707278317 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feeec108a20 0x7feeec19be60 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7feee80072a0 tx=0x7feee800bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:42:31.265 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:29.717+0000 7feee27fc700 1 -- 172.21.15.52:0/3707278317 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feee8020020 con 0x7feeec108a20 2024-06-10T02:42:31.265 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:29.717+0000 7feef362f700 1 -- 172.21.15.52:0/3707278317 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feeec19c630 con 0x7feeec108a20 2024-06-10T02:42:31.265 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:29.717+0000 7feee27fc700 1 -- 172.21.15.52:0/3707278317 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7feee8003b50 con 0x7feeec108a20 2024-06-10T02:42:31.265 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:29.717+0000 7feee27fc700 1 -- 172.21.15.52:0/3707278317 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feee8017040 con 0x7feeec108a20 2024-06-10T02:42:31.265 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:29.717+0000 7feef362f700 1 -- 172.21.15.52:0/3707278317 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feeec19f320 con 0x7feeec108a20 2024-06-10T02:42:31.265 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:29.718+0000 7feee27fc700 1 -- 172.21.15.52:0/3707278317 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7feee8023020 con 0x7feeec108a20 2024-06-10T02:42:31.265 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:29.718+0000 7feef362f700 1 -- 172.21.15.52:0/3707278317 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feeec063af0 con 0x7feeec108a20 2024-06-10T02:42:31.265 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:29.718+0000 7feee27fc700 1 --2- 172.21.15.52:0/3707278317 >> [v2:172.21.15.52:6800/896317057,v1:172.21.15.52:6801/896317057] conn(0x7feed8039000 0x7feed803b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:42:31.266 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:29.718+0000 7feee27fc700 1 -- 172.21.15.52:0/3707278317 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7feee800f050 con 0x7feeec108a20 2024-06-10T02:42:31.266 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:29.718+0000 7feef0bca700 1 --2- 172.21.15.52:0/3707278317 >> [v2:172.21.15.52:6800/896317057,v1:172.21.15.52:6801/896317057] conn(0x7feed8039000 0x7feed803b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:42:31.266 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:29.719+0000 7feef0bca700 1 --2- 172.21.15.52:0/3707278317 >> [v2:172.21.15.52:6800/896317057,v1:172.21.15.52:6801/896317057] conn(0x7feed8039000 0x7feed803b4c0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7feedc006fd0 tx=0x7feedc006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:42:31.266 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:29.723+0000 7feee27fc700 1 -- 172.21.15.52:0/3707278317 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7feee802a020 con 0x7feeec108a20 2024-06-10T02:42:31.266 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:29.951+0000 7feef362f700 1 -- 172.21.15.52:0/3707278317 --> [v2:172.21.15.52:6800/896317057,v1:172.21.15.52:6801/896317057] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7feeec103560 con 0x7feed8039000 2024-06-10T02:42:31.266 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:29.956+0000 7feee27fc700 1 -- 172.21.15.52:0/3707278317 <== mgr.14118 v2:172.21.15.52:6800/896317057 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7feeec103560 con 0x7feed8039000 2024-06-10T02:42:31.266 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:29.958+0000 7feef362f700 1 -- 172.21.15.52:0/3707278317 >> [v2:172.21.15.52:6800/896317057,v1:172.21.15.52:6801/896317057] conn(0x7feed8039000 msgr2=0x7feed803b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:42:31.266 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:29.958+0000 7feef362f700 1 --2- 172.21.15.52:0/3707278317 >> [v2:172.21.15.52:6800/896317057,v1:172.21.15.52:6801/896317057] conn(0x7feed8039000 0x7feed803b4c0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7feedc006fd0 tx=0x7feedc006e40 comp rx=0 tx=0).stop 2024-06-10T02:42:31.266 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:29.958+0000 7feef362f700 1 -- 172.21.15.52:0/3707278317 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feeec108a20 msgr2=0x7feeec19be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:42:31.266 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:29.958+0000 7feef362f700 1 --2- 172.21.15.52:0/3707278317 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feeec108a20 0x7feeec19be60 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7feee80072a0 tx=0x7feee800bd50 comp rx=0 tx=0).stop 2024-06-10T02:42:31.266 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:29.958+0000 7feef362f700 1 -- 172.21.15.52:0/3707278317 shutdown_connections 2024-06-10T02:42:31.266 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:29.958+0000 7feef362f700 1 --2- 172.21.15.52:0/3707278317 >> [v2:172.21.15.52:6800/896317057,v1:172.21.15.52:6801/896317057] conn(0x7feed8039000 0x7feed803b4c0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:42:31.267 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:29.958+0000 7feef362f700 1 --2- 172.21.15.52:0/3707278317 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feeec108a20 0x7feeec19be60 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:42:31.267 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:29.958+0000 7feef362f700 1 -- 172.21.15.52:0/3707278317 >> 172.21.15.52:0/3707278317 conn(0x7feeec102450 msgr2=0x7feeec102e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:42:31.267 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:29.958+0000 7feef362f700 1 -- 172.21.15.52:0/3707278317 shutdown_connections 2024-06-10T02:42:31.267 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:29.958+0000 7feef362f700 1 -- 172.21.15.52:0/3707278317 wait complete. 2024-06-10T02:42:32.477 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:31.698+0000 7fbd8aad2700 1 Processor -- start 2024-06-10T02:42:32.477 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:31.698+0000 7fbd8aad2700 1 -- start start 2024-06-10T02:42:32.477 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:31.698+0000 7fbd8aad2700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbd84107ff0 0x7fbd84108410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:42:32.477 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:31.698+0000 7fbd8aad2700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fbd841089e0 con 0x7fbd84107ff0 2024-06-10T02:42:32.477 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:31.699+0000 7fbd8886e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbd84107ff0 0x7fbd84108410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:42:32.477 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:31.699+0000 7fbd8886e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbd84107ff0 0x7fbd84108410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49404/0 (socket says 172.21.15.52:49404) 2024-06-10T02:42:32.478 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:31.699+0000 7fbd8886e700 1 -- 172.21.15.52:0/2086283680 learned_addr learned my addr 172.21.15.52:0/2086283680 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:42:32.478 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:31.699+0000 7fbd8886e700 1 -- 172.21.15.52:0/2086283680 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd841091a0 con 0x7fbd84107ff0 2024-06-10T02:42:32.478 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:31.699+0000 7fbd8886e700 1 --2- 172.21.15.52:0/2086283680 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbd84107ff0 0x7fbd84108410 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fbd780098a0 tx=0x7fbd78009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a618da2dbad71137 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:42:32.478 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:31.699+0000 7fbd837fe700 1 -- 172.21.15.52:0/2086283680 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbd78004160 con 0x7fbd84107ff0 2024-06-10T02:42:32.478 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:31.699+0000 7fbd837fe700 1 -- 172.21.15.52:0/2086283680 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd78004300 con 0x7fbd84107ff0 2024-06-10T02:42:32.478 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:31.699+0000 7fbd837fe700 1 -- 172.21.15.52:0/2086283680 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbd78011b20 con 0x7fbd84107ff0 2024-06-10T02:42:32.478 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:31.700+0000 7fbd8aad2700 1 -- 172.21.15.52:0/2086283680 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbd84107ff0 msgr2=0x7fbd84108410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:42:32.478 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:31.700+0000 7fbd8aad2700 1 --2- 172.21.15.52:0/2086283680 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbd84107ff0 0x7fbd84108410 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fbd780098a0 tx=0x7fbd78009bb0 comp rx=0 tx=0).stop 2024-06-10T02:42:32.478 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:31.700+0000 7fbd8aad2700 1 -- 172.21.15.52:0/2086283680 shutdown_connections 2024-06-10T02:42:32.478 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:31.700+0000 7fbd8aad2700 1 --2- 172.21.15.52:0/2086283680 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbd84107ff0 0x7fbd84108410 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:42:32.478 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:31.700+0000 7fbd8aad2700 1 -- 172.21.15.52:0/2086283680 >> 172.21.15.52:0/2086283680 conn(0x7fbd84103590 msgr2=0x7fbd841059d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:42:32.478 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:31.700+0000 7fbd8aad2700 1 -- 172.21.15.52:0/2086283680 shutdown_connections 2024-06-10T02:42:32.479 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:31.700+0000 7fbd8aad2700 1 -- 172.21.15.52:0/2086283680 wait complete. 2024-06-10T02:42:32.479 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:31.701+0000 7fbd8aad2700 1 Processor -- start 2024-06-10T02:42:32.479 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:31.701+0000 7fbd8aad2700 1 -- start start 2024-06-10T02:42:32.479 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:31.701+0000 7fbd8aad2700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbd84107ff0 0x7fbd8419bf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:42:32.479 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:31.701+0000 7fbd8aad2700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fbd8410c550 con 0x7fbd84107ff0 2024-06-10T02:42:32.479 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:31.701+0000 7fbd8886e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbd84107ff0 0x7fbd8419bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:42:32.479 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:31.701+0000 7fbd8886e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbd84107ff0 0x7fbd8419bf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49406/0 (socket says 172.21.15.52:49406) 2024-06-10T02:42:32.479 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:31.701+0000 7fbd8886e700 1 -- 172.21.15.52:0/1991581378 learned_addr learned my addr 172.21.15.52:0/1991581378 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:42:32.479 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:31.701+0000 7fbd8886e700 1 -- 172.21.15.52:0/1991581378 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd8419c470 con 0x7fbd84107ff0 2024-06-10T02:42:32.479 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:31.702+0000 7fbd8886e700 1 --2- 172.21.15.52:0/1991581378 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbd84107ff0 0x7fbd8419bf30 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fbd780116a0 tx=0x7fbd78011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:42:32.479 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:31.702+0000 7fbd81ffb700 1 -- 172.21.15.52:0/1991581378 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbd78003a40 con 0x7fbd84107ff0 2024-06-10T02:42:32.479 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:31.702+0000 7fbd81ffb700 1 -- 172.21.15.52:0/1991581378 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd78003be0 con 0x7fbd84107ff0 2024-06-10T02:42:32.479 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:31.702+0000 7fbd8aad2700 1 -- 172.21.15.52:0/1991581378 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd8419c700 con 0x7fbd84107ff0 2024-06-10T02:42:32.480 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:31.702+0000 7fbd81ffb700 1 -- 172.21.15.52:0/1991581378 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbd7801a720 con 0x7fbd84107ff0 2024-06-10T02:42:32.480 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:31.702+0000 7fbd8aad2700 1 -- 172.21.15.52:0/1991581378 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd8419cb60 con 0x7fbd84107ff0 2024-06-10T02:42:32.480 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:31.703+0000 7fbd8aad2700 1 -- 172.21.15.52:0/1991581378 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd84063af0 con 0x7fbd84107ff0 2024-06-10T02:42:32.480 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:31.703+0000 7fbd81ffb700 1 -- 172.21.15.52:0/1991581378 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fbd7801aa40 con 0x7fbd84107ff0 2024-06-10T02:42:32.480 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:31.703+0000 7fbd81ffb700 1 --2- 172.21.15.52:0/1991581378 >> [v2:172.21.15.52:6800/896317057,v1:172.21.15.52:6801/896317057] conn(0x7fbd74039000 0x7fbd7403b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:42:32.480 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:31.704+0000 7fbd81ffb700 1 -- 172.21.15.52:0/1991581378 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fbd7804d6a0 con 0x7fbd84107ff0 2024-06-10T02:42:32.480 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:31.704+0000 7fbd83fff700 1 --2- 172.21.15.52:0/1991581378 >> [v2:172.21.15.52:6800/896317057,v1:172.21.15.52:6801/896317057] conn(0x7fbd74039000 0x7fbd7403b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:42:32.480 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:31.704+0000 7fbd83fff700 1 --2- 172.21.15.52:0/1991581378 >> [v2:172.21.15.52:6800/896317057,v1:172.21.15.52:6801/896317057] conn(0x7fbd74039000 0x7fbd7403b4c0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fbd70006fd0 tx=0x7fbd70006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:42:32.480 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:31.709+0000 7fbd81ffb700 1 -- 172.21.15.52:0/1991581378 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fbd78011e20 con 0x7fbd84107ff0 2024-06-10T02:42:32.480 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:31.920+0000 7fbd8aad2700 1 -- 172.21.15.52:0/1991581378 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7fbd8419f4b0 con 0x7fbd84107ff0 2024-06-10T02:42:32.480 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:31.923+0000 7fbd81ffb700 1 -- 172.21.15.52:0/1991581378 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7fbd78020240 con 0x7fbd84107ff0 2024-06-10T02:42:32.480 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:31.926+0000 7fbd8aad2700 1 -- 172.21.15.52:0/1991581378 >> [v2:172.21.15.52:6800/896317057,v1:172.21.15.52:6801/896317057] conn(0x7fbd74039000 msgr2=0x7fbd7403b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:42:32.480 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:31.926+0000 7fbd8aad2700 1 --2- 172.21.15.52:0/1991581378 >> [v2:172.21.15.52:6800/896317057,v1:172.21.15.52:6801/896317057] conn(0x7fbd74039000 0x7fbd7403b4c0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fbd70006fd0 tx=0x7fbd70006e40 comp rx=0 tx=0).stop 2024-06-10T02:42:32.481 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:31.926+0000 7fbd8aad2700 1 -- 172.21.15.52:0/1991581378 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbd84107ff0 msgr2=0x7fbd8419bf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:42:32.481 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:31.926+0000 7fbd8aad2700 1 --2- 172.21.15.52:0/1991581378 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbd84107ff0 0x7fbd8419bf30 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fbd780116a0 tx=0x7fbd78011ba0 comp rx=0 tx=0).stop 2024-06-10T02:42:32.481 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:31.926+0000 7fbd8aad2700 1 -- 172.21.15.52:0/1991581378 shutdown_connections 2024-06-10T02:42:32.481 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:31.926+0000 7fbd8aad2700 1 --2- 172.21.15.52:0/1991581378 >> [v2:172.21.15.52:6800/896317057,v1:172.21.15.52:6801/896317057] conn(0x7fbd74039000 0x7fbd7403b4c0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:42:32.481 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:31.926+0000 7fbd8aad2700 1 --2- 172.21.15.52:0/1991581378 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbd84107ff0 0x7fbd8419bf30 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:42:32.481 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:31.926+0000 7fbd8aad2700 1 -- 172.21.15.52:0/1991581378 >> 172.21.15.52:0/1991581378 conn(0x7fbd84103590 msgr2=0x7fbd84104080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:42:32.481 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:31.926+0000 7fbd8aad2700 1 -- 172.21.15.52:0/1991581378 shutdown_connections 2024-06-10T02:42:32.481 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:31.926+0000 7fbd8aad2700 1 -- 172.21.15.52:0/1991581378 wait complete. 2024-06-10T02:42:32.632 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:42:32 smithi052 ceph-mon[32362]: from='mgr.14118 172.21.15.52:0/1358068050' entity='mgr.x' 2024-06-10T02:42:32.632 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:42:32 smithi052 ceph-mon[32362]: from='client.? 172.21.15.52:0/1991581378' entity='client.admin' 2024-06-10T02:42:34.080 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:33.310+0000 7f8d89f47700 1 Processor -- start 2024-06-10T02:42:34.080 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:33.310+0000 7f8d89f47700 1 -- start start 2024-06-10T02:42:34.080 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:33.310+0000 7f8d89f47700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8d84107ff0 0x7f8d84108410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:42:34.080 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:33.310+0000 7f8d89f47700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f8d841089e0 con 0x7f8d84107ff0 2024-06-10T02:42:34.080 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:33.311+0000 7f8d837fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8d84107ff0 0x7f8d84108410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:42:34.080 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:33.311+0000 7f8d837fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8d84107ff0 0x7f8d84108410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49408/0 (socket says 172.21.15.52:49408) 2024-06-10T02:42:34.081 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:33.311+0000 7f8d837fe700 1 -- 172.21.15.52:0/4227617506 learned_addr learned my addr 172.21.15.52:0/4227617506 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:42:34.081 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:33.311+0000 7f8d837fe700 1 -- 172.21.15.52:0/4227617506 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d841091a0 con 0x7f8d84107ff0 2024-06-10T02:42:34.081 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:33.311+0000 7f8d837fe700 1 --2- 172.21.15.52:0/4227617506 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8d84107ff0 0x7f8d84108410 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f8d740098a0 tx=0x7f8d74009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eed327cc166aac14 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:42:34.081 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:33.311+0000 7f8d827fc700 1 -- 172.21.15.52:0/4227617506 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8d74004160 con 0x7f8d84107ff0 2024-06-10T02:42:34.081 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:33.311+0000 7f8d827fc700 1 -- 172.21.15.52:0/4227617506 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d74004300 con 0x7f8d84107ff0 2024-06-10T02:42:34.081 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:33.311+0000 7f8d827fc700 1 -- 172.21.15.52:0/4227617506 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8d74011b20 con 0x7f8d84107ff0 2024-06-10T02:42:34.081 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:33.312+0000 7f8d89f47700 1 -- 172.21.15.52:0/4227617506 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8d84107ff0 msgr2=0x7f8d84108410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:42:34.081 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:33.312+0000 7f8d89f47700 1 --2- 172.21.15.52:0/4227617506 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8d84107ff0 0x7f8d84108410 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f8d740098a0 tx=0x7f8d74009bb0 comp rx=0 tx=0).stop 2024-06-10T02:42:34.081 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:33.312+0000 7f8d89f47700 1 -- 172.21.15.52:0/4227617506 shutdown_connections 2024-06-10T02:42:34.081 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:33.312+0000 7f8d89f47700 1 --2- 172.21.15.52:0/4227617506 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8d84107ff0 0x7f8d84108410 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:42:34.081 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:33.312+0000 7f8d89f47700 1 -- 172.21.15.52:0/4227617506 >> 172.21.15.52:0/4227617506 conn(0x7f8d84103590 msgr2=0x7f8d841059d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:42:34.081 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:33.312+0000 7f8d89f47700 1 -- 172.21.15.52:0/4227617506 shutdown_connections 2024-06-10T02:42:34.082 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:33.312+0000 7f8d89f47700 1 -- 172.21.15.52:0/4227617506 wait complete. 2024-06-10T02:42:34.082 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:33.313+0000 7f8d89f47700 1 Processor -- start 2024-06-10T02:42:34.082 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:33.313+0000 7f8d89f47700 1 -- start start 2024-06-10T02:42:34.082 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:33.313+0000 7f8d89f47700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8d84107ff0 0x7f8d8419bed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:42:34.082 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:33.313+0000 7f8d89f47700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f8d8410c550 con 0x7f8d84107ff0 2024-06-10T02:42:34.082 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:33.314+0000 7f8d837fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8d84107ff0 0x7f8d8419bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:42:34.082 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:33.314+0000 7f8d837fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8d84107ff0 0x7f8d8419bed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49416/0 (socket says 172.21.15.52:49416) 2024-06-10T02:42:34.082 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:33.314+0000 7f8d837fe700 1 -- 172.21.15.52:0/1393303851 learned_addr learned my addr 172.21.15.52:0/1393303851 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:42:34.082 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:33.314+0000 7f8d837fe700 1 -- 172.21.15.52:0/1393303851 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d8419c410 con 0x7f8d84107ff0 2024-06-10T02:42:34.082 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:33.314+0000 7f8d837fe700 1 --2- 172.21.15.52:0/1393303851 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8d84107ff0 0x7f8d8419bed0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f8d740116a0 tx=0x7f8d74011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:42:34.082 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:33.314+0000 7f8d80ff9700 1 -- 172.21.15.52:0/1393303851 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8d74003a40 con 0x7f8d84107ff0 2024-06-10T02:42:34.082 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:33.314+0000 7f8d80ff9700 1 -- 172.21.15.52:0/1393303851 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d74003be0 con 0x7f8d84107ff0 2024-06-10T02:42:34.082 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:33.314+0000 7f8d89f47700 1 -- 172.21.15.52:0/1393303851 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d8419c6a0 con 0x7f8d84107ff0 2024-06-10T02:42:34.083 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:33.314+0000 7f8d80ff9700 1 -- 172.21.15.52:0/1393303851 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8d7401a720 con 0x7f8d84107ff0 2024-06-10T02:42:34.083 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:33.314+0000 7f8d89f47700 1 -- 172.21.15.52:0/1393303851 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d8419cb00 con 0x7f8d84107ff0 2024-06-10T02:42:34.083 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:33.315+0000 7f8d80ff9700 1 -- 172.21.15.52:0/1393303851 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f8d7401aa20 con 0x7f8d84107ff0 2024-06-10T02:42:34.083 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:33.315+0000 7f8d89f47700 1 -- 172.21.15.52:0/1393303851 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d84063af0 con 0x7f8d84107ff0 2024-06-10T02:42:34.083 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:33.316+0000 7f8d80ff9700 1 --2- 172.21.15.52:0/1393303851 >> [v2:172.21.15.52:6800/896317057,v1:172.21.15.52:6801/896317057] conn(0x7f8d70039000 0x7f8d7003b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:42:34.083 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:33.316+0000 7f8d80ff9700 1 -- 172.21.15.52:0/1393303851 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f8d7404d6d0 con 0x7f8d84107ff0 2024-06-10T02:42:34.083 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:33.316+0000 7f8d82ffd700 1 --2- 172.21.15.52:0/1393303851 >> [v2:172.21.15.52:6800/896317057,v1:172.21.15.52:6801/896317057] conn(0x7f8d70039000 0x7f8d7003b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:42:34.083 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:33.316+0000 7f8d82ffd700 1 --2- 172.21.15.52:0/1393303851 >> [v2:172.21.15.52:6800/896317057,v1:172.21.15.52:6801/896317057] conn(0x7f8d70039000 0x7f8d7003b4c0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f8d6c006fd0 tx=0x7f8d6c006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:42:34.083 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:33.321+0000 7f8d80ff9700 1 -- 172.21.15.52:0/1393303851 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f8d74011e20 con 0x7f8d84107ff0 2024-06-10T02:42:34.083 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:33.542+0000 7f8d89f47700 1 -- 172.21.15.52:0/1393303851 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7f8d8419f4d0 con 0x7f8d84107ff0 2024-06-10T02:42:34.083 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:33.545+0000 7f8d80ff9700 1 -- 172.21.15.52:0/1393303851 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7f8d7401adc0 con 0x7f8d84107ff0 2024-06-10T02:42:34.083 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:33.547+0000 7f8d89f47700 1 -- 172.21.15.52:0/1393303851 >> [v2:172.21.15.52:6800/896317057,v1:172.21.15.52:6801/896317057] conn(0x7f8d70039000 msgr2=0x7f8d7003b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:42:34.084 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:33.547+0000 7f8d89f47700 1 --2- 172.21.15.52:0/1393303851 >> [v2:172.21.15.52:6800/896317057,v1:172.21.15.52:6801/896317057] conn(0x7f8d70039000 0x7f8d7003b4c0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f8d6c006fd0 tx=0x7f8d6c006e40 comp rx=0 tx=0).stop 2024-06-10T02:42:34.084 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:33.547+0000 7f8d89f47700 1 -- 172.21.15.52:0/1393303851 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8d84107ff0 msgr2=0x7f8d8419bed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:42:34.084 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:33.547+0000 7f8d89f47700 1 --2- 172.21.15.52:0/1393303851 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8d84107ff0 0x7f8d8419bed0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f8d740116a0 tx=0x7f8d74011ba0 comp rx=0 tx=0).stop 2024-06-10T02:42:34.084 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:33.547+0000 7f8d89f47700 1 -- 172.21.15.52:0/1393303851 shutdown_connections 2024-06-10T02:42:34.084 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:33.547+0000 7f8d89f47700 1 --2- 172.21.15.52:0/1393303851 >> [v2:172.21.15.52:6800/896317057,v1:172.21.15.52:6801/896317057] conn(0x7f8d70039000 0x7f8d7003b4c0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:42:34.084 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:33.547+0000 7f8d89f47700 1 --2- 172.21.15.52:0/1393303851 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8d84107ff0 0x7f8d8419bed0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:42:34.084 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:33.547+0000 7f8d89f47700 1 -- 172.21.15.52:0/1393303851 >> 172.21.15.52:0/1393303851 conn(0x7f8d84103590 msgr2=0x7f8d84104030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:42:34.084 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:33.547+0000 7f8d89f47700 1 -- 172.21.15.52:0/1393303851 shutdown_connections 2024-06-10T02:42:34.084 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:33.548+0000 7f8d89f47700 1 -- 172.21.15.52:0/1393303851 wait complete. 2024-06-10T02:42:34.084 INFO:teuthology.orchestra.run.smithi052.stdout:Enabling the dashboard module... 2024-06-10T02:42:34.473 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:42:34 smithi052 ceph-mon[32362]: from='mgr.14118 172.21.15.52:0/1358068050' entity='mgr.x' 2024-06-10T02:42:34.473 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:42:34 smithi052 ceph-mon[32362]: from='client.? 172.21.15.52:0/1393303851' entity='client.admin' 2024-06-10T02:42:35.886 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:42:35 smithi052 ceph-mon[32362]: from='mgr.14118 172.21.15.52:0/1358068050' entity='mgr.x' 2024-06-10T02:42:35.886 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:42:35 smithi052 ceph-mon[32362]: from='client.? 172.21.15.52:0/103105982' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-06-10T02:42:35.886 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 10 02:42:35 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mgr-x[32595]: ignoring --setuser ceph since I am not root 2024-06-10T02:42:35.887 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 10 02:42:35 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mgr-x[32595]: ignoring --setgroup ceph since I am not root 2024-06-10T02:42:35.887 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 10 02:42:35 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mgr-x[32595]: 2024-06-10T02:42:35.890+0000 7fe32b7e6200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-06-10T02:42:36.191 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 10 02:42:36 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mgr-x[32595]: 2024-06-10T02:42:36.088+0000 7fe32b7e6200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-06-10T02:42:36.299 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:34.513+0000 7f4456028700 1 Processor -- start 2024-06-10T02:42:36.299 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:34.513+0000 7f4456028700 1 -- start start 2024-06-10T02:42:36.299 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:34.513+0000 7f4456028700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4450107ff0 0x7f4450108410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:42:36.299 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:34.513+0000 7f4456028700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f44501089e0 con 0x7f4450107ff0 2024-06-10T02:42:36.299 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:34.514+0000 7f444f7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4450107ff0 0x7f4450108410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:42:36.299 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:34.514+0000 7f444f7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4450107ff0 0x7f4450108410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49428/0 (socket says 172.21.15.52:49428) 2024-06-10T02:42:36.299 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:34.514+0000 7f444f7fe700 1 -- 172.21.15.52:0/3715620328 learned_addr learned my addr 172.21.15.52:0/3715620328 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:42:36.299 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:34.514+0000 7f444f7fe700 1 -- 172.21.15.52:0/3715620328 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44501091a0 con 0x7f4450107ff0 2024-06-10T02:42:36.300 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:34.514+0000 7f444f7fe700 1 --2- 172.21.15.52:0/3715620328 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4450107ff0 0x7f4450108410 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f44400098a0 tx=0x7f4440009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f602dc4b93921ece server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:42:36.300 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:34.514+0000 7f444e7fc700 1 -- 172.21.15.52:0/3715620328 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4440004160 con 0x7f4450107ff0 2024-06-10T02:42:36.300 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:34.514+0000 7f444e7fc700 1 -- 172.21.15.52:0/3715620328 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4440004300 con 0x7f4450107ff0 2024-06-10T02:42:36.300 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:34.514+0000 7f444e7fc700 1 -- 172.21.15.52:0/3715620328 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4440011b20 con 0x7f4450107ff0 2024-06-10T02:42:36.300 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:34.515+0000 7f4456028700 1 -- 172.21.15.52:0/3715620328 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4450107ff0 msgr2=0x7f4450108410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:42:36.300 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:34.515+0000 7f4456028700 1 --2- 172.21.15.52:0/3715620328 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4450107ff0 0x7f4450108410 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f44400098a0 tx=0x7f4440009bb0 comp rx=0 tx=0).stop 2024-06-10T02:42:36.300 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:34.515+0000 7f4456028700 1 -- 172.21.15.52:0/3715620328 shutdown_connections 2024-06-10T02:42:36.300 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:34.515+0000 7f4456028700 1 --2- 172.21.15.52:0/3715620328 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4450107ff0 0x7f4450108410 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:42:36.300 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:34.515+0000 7f4456028700 1 -- 172.21.15.52:0/3715620328 >> 172.21.15.52:0/3715620328 conn(0x7f4450103590 msgr2=0x7f44501059d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:42:36.300 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:34.515+0000 7f4456028700 1 -- 172.21.15.52:0/3715620328 shutdown_connections 2024-06-10T02:42:36.300 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:34.515+0000 7f4456028700 1 -- 172.21.15.52:0/3715620328 wait complete. 2024-06-10T02:42:36.300 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:34.516+0000 7f4456028700 1 Processor -- start 2024-06-10T02:42:36.300 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:34.516+0000 7f4456028700 1 -- start start 2024-06-10T02:42:36.301 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:34.517+0000 7f4456028700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4450107ff0 0x7f445007e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:42:36.301 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:34.517+0000 7f4456028700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f445010c550 con 0x7f4450107ff0 2024-06-10T02:42:36.301 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:34.517+0000 7f444f7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4450107ff0 0x7f445007e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:42:36.301 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:34.517+0000 7f444f7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4450107ff0 0x7f445007e410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49440/0 (socket says 172.21.15.52:49440) 2024-06-10T02:42:36.301 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:34.517+0000 7f444f7fe700 1 -- 172.21.15.52:0/103105982 learned_addr learned my addr 172.21.15.52:0/103105982 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:42:36.301 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:34.517+0000 7f444f7fe700 1 -- 172.21.15.52:0/103105982 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f445007e950 con 0x7f4450107ff0 2024-06-10T02:42:36.301 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:34.517+0000 7f444f7fe700 1 --2- 172.21.15.52:0/103105982 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4450107ff0 0x7f445007e410 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f4440000c00 tx=0x7f4440011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:42:36.301 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:34.518+0000 7f444cff9700 1 -- 172.21.15.52:0/103105982 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4440003b60 con 0x7f4450107ff0 2024-06-10T02:42:36.301 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:34.518+0000 7f444cff9700 1 -- 172.21.15.52:0/103105982 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f444001a910 con 0x7f4450107ff0 2024-06-10T02:42:36.301 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:34.518+0000 7f4456028700 1 -- 172.21.15.52:0/103105982 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f445007cb40 con 0x7f4450107ff0 2024-06-10T02:42:36.301 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:34.518+0000 7f444cff9700 1 -- 172.21.15.52:0/103105982 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4440024930 con 0x7f4450107ff0 2024-06-10T02:42:36.301 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:34.518+0000 7f4456028700 1 -- 172.21.15.52:0/103105982 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f445007d000 con 0x7f4450107ff0 2024-06-10T02:42:36.301 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:34.519+0000 7f444cff9700 1 -- 172.21.15.52:0/103105982 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f4440022020 con 0x7f4450107ff0 2024-06-10T02:42:36.302 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:34.519+0000 7f4456028700 1 -- 172.21.15.52:0/103105982 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4450063af0 con 0x7f4450107ff0 2024-06-10T02:42:36.302 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:34.519+0000 7f444cff9700 1 --2- 172.21.15.52:0/103105982 >> [v2:172.21.15.52:6800/896317057,v1:172.21.15.52:6801/896317057] conn(0x7f443c0390a0 0x7f443c03b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:42:36.302 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:34.519+0000 7f444cff9700 1 -- 172.21.15.52:0/103105982 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f444004d1d0 con 0x7f4450107ff0 2024-06-10T02:42:36.302 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:34.519+0000 7f444effd700 1 --2- 172.21.15.52:0/103105982 >> [v2:172.21.15.52:6800/896317057,v1:172.21.15.52:6801/896317057] conn(0x7f443c0390a0 0x7f443c03b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:42:36.302 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:34.519+0000 7f444effd700 1 --2- 172.21.15.52:0/103105982 >> [v2:172.21.15.52:6800/896317057,v1:172.21.15.52:6801/896317057] conn(0x7f443c0390a0 0x7f443c03b560 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f4438006fd0 tx=0x7f4438006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:42:36.302 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:34.524+0000 7f444cff9700 1 -- 172.21.15.52:0/103105982 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f444002c510 con 0x7f4450107ff0 2024-06-10T02:42:36.302 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:34.793+0000 7f4456028700 1 -- 172.21.15.52:0/103105982 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7f445007d280 con 0x7f4450107ff0 2024-06-10T02:42:36.302 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:35.706+0000 7f444cff9700 1 -- 172.21.15.52:0/103105982 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7f444002ad60 con 0x7f4450107ff0 2024-06-10T02:42:36.302 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:35.708+0000 7f444cff9700 1 -- 172.21.15.52:0/103105982 <== mon.0 v2:172.21.15.52:3300/0 8 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7f444002dc90 con 0x7f4450107ff0 2024-06-10T02:42:36.302 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:35.710+0000 7f4456028700 1 -- 172.21.15.52:0/103105982 >> [v2:172.21.15.52:6800/896317057,v1:172.21.15.52:6801/896317057] conn(0x7f443c0390a0 msgr2=0x7f443c03b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:42:36.302 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:35.710+0000 7f4456028700 1 --2- 172.21.15.52:0/103105982 >> [v2:172.21.15.52:6800/896317057,v1:172.21.15.52:6801/896317057] conn(0x7f443c0390a0 0x7f443c03b560 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f4438006fd0 tx=0x7f4438006e40 comp rx=0 tx=0).stop 2024-06-10T02:42:36.302 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:35.710+0000 7f4456028700 1 -- 172.21.15.52:0/103105982 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4450107ff0 msgr2=0x7f445007e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:42:36.303 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:35.710+0000 7f4456028700 1 --2- 172.21.15.52:0/103105982 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4450107ff0 0x7f445007e410 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f4440000c00 tx=0x7f4440011c10 comp rx=0 tx=0).stop 2024-06-10T02:42:36.303 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:35.710+0000 7f4456028700 1 -- 172.21.15.52:0/103105982 shutdown_connections 2024-06-10T02:42:36.303 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:35.710+0000 7f4456028700 1 --2- 172.21.15.52:0/103105982 >> [v2:172.21.15.52:6800/896317057,v1:172.21.15.52:6801/896317057] conn(0x7f443c0390a0 0x7f443c03b560 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:42:36.303 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:35.710+0000 7f4456028700 1 --2- 172.21.15.52:0/103105982 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4450107ff0 0x7f445007e410 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:42:36.303 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:35.710+0000 7f4456028700 1 -- 172.21.15.52:0/103105982 >> 172.21.15.52:0/103105982 conn(0x7f4450103590 msgr2=0x7f4450104030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:42:36.303 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:35.710+0000 7f4456028700 1 -- 172.21.15.52:0/103105982 shutdown_connections 2024-06-10T02:42:36.303 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:35.710+0000 7f4456028700 1 -- 172.21.15.52:0/103105982 wait complete. 2024-06-10T02:42:36.472 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 10 02:42:36 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mgr-x[32595]: 2024-06-10T02:42:36.195+0000 7fe32b7e6200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-06-10T02:42:36.837 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:42:36 smithi052 ceph-mon[32362]: from='client.? 172.21.15.52:0/103105982' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-06-10T02:42:36.837 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:42:36 smithi052 ceph-mon[32362]: mgrmap e9: x(active, since 20s) 2024-06-10T02:42:36.837 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 10 02:42:36 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mgr-x[32595]: 2024-06-10T02:42:36.523+0000 7fe32b7e6200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-06-10T02:42:36.838 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 10 02:42:36 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mgr-x[32595]: 2024-06-10T02:42:36.612+0000 7fe32b7e6200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-06-10T02:42:37.973 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:42:37 smithi052 ceph-mon[32362]: from='client.? 172.21.15.52:0/1324569372' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-06-10T02:42:37.973 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 10 02:42:37 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mgr-x[32595]: 2024-06-10T02:42:37.877+0000 7fe32b7e6200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-06-10T02:42:37.984 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout { 2024-06-10T02:42:37.985 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-06-10T02:42:37.985 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "available": true, 2024-06-10T02:42:37.985 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-06-10T02:42:37.985 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-06-10T02:42:37.985 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout } 2024-06-10T02:42:37.985 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:37.150+0000 7ff4bb5e6700 1 Processor -- start 2024-06-10T02:42:37.985 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:37.150+0000 7ff4bb5e6700 1 -- start start 2024-06-10T02:42:37.985 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:37.150+0000 7ff4bb5e6700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff4b4108a50 0x7ff4b410ae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:42:37.985 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:37.150+0000 7ff4bb5e6700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ff4b407c6c0 con 0x7ff4b4108a50 2024-06-10T02:42:37.985 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:37.151+0000 7ff4b9382700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff4b4108a50 0x7ff4b410ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:42:37.985 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:37.151+0000 7ff4b9382700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff4b4108a50 0x7ff4b410ae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49464/0 (socket says 172.21.15.52:49464) 2024-06-10T02:42:37.985 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:37.151+0000 7ff4b9382700 1 -- 172.21.15.52:0/3259224277 learned_addr learned my addr 172.21.15.52:0/3259224277 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:42:37.986 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:37.151+0000 7ff4b9382700 1 -- 172.21.15.52:0/3259224277 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4b410b380 con 0x7ff4b4108a50 2024-06-10T02:42:37.986 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:37.151+0000 7ff4b9382700 1 --2- 172.21.15.52:0/3259224277 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff4b4108a50 0x7ff4b410ae40 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff4b00098a0 tx=0x7ff4b0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6294c3581ef82ccd server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:42:37.986 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:37.151+0000 7ff4a7fff700 1 -- 172.21.15.52:0/3259224277 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff4b0004160 con 0x7ff4b4108a50 2024-06-10T02:42:37.986 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:37.151+0000 7ff4a7fff700 1 -- 172.21.15.52:0/3259224277 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff4b0004300 con 0x7ff4b4108a50 2024-06-10T02:42:37.986 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:37.151+0000 7ff4a7fff700 1 -- 172.21.15.52:0/3259224277 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff4b0011b20 con 0x7ff4b4108a50 2024-06-10T02:42:37.986 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:37.152+0000 7ff4bb5e6700 1 -- 172.21.15.52:0/3259224277 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff4b4108a50 msgr2=0x7ff4b410ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:42:37.986 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:37.152+0000 7ff4bb5e6700 1 --2- 172.21.15.52:0/3259224277 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff4b4108a50 0x7ff4b410ae40 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff4b00098a0 tx=0x7ff4b0009bb0 comp rx=0 tx=0).stop 2024-06-10T02:42:37.986 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:37.152+0000 7ff4bb5e6700 1 -- 172.21.15.52:0/3259224277 shutdown_connections 2024-06-10T02:42:37.986 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:37.152+0000 7ff4bb5e6700 1 --2- 172.21.15.52:0/3259224277 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff4b4108a50 0x7ff4b410ae40 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:42:37.986 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:37.152+0000 7ff4bb5e6700 1 -- 172.21.15.52:0/3259224277 >> 172.21.15.52:0/3259224277 conn(0x7ff4b41024a0 msgr2=0x7ff4b41048c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:42:37.987 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:37.152+0000 7ff4bb5e6700 1 -- 172.21.15.52:0/3259224277 shutdown_connections 2024-06-10T02:42:37.987 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:37.152+0000 7ff4bb5e6700 1 -- 172.21.15.52:0/3259224277 wait complete. 2024-06-10T02:42:37.987 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:37.153+0000 7ff4bb5e6700 1 Processor -- start 2024-06-10T02:42:37.987 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:37.153+0000 7ff4bb5e6700 1 -- start start 2024-06-10T02:42:37.987 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:37.153+0000 7ff4bb5e6700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff4b4108a50 0x7ff4b41082d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:42:37.987 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:37.153+0000 7ff4bb5e6700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ff4b410c380 con 0x7ff4b4108a50 2024-06-10T02:42:37.987 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:37.153+0000 7ff4b9382700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff4b4108a50 0x7ff4b41082d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:42:37.987 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:37.153+0000 7ff4b9382700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff4b4108a50 0x7ff4b41082d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49470/0 (socket says 172.21.15.52:49470) 2024-06-10T02:42:37.987 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:37.153+0000 7ff4b9382700 1 -- 172.21.15.52:0/1324569372 learned_addr learned my addr 172.21.15.52:0/1324569372 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:42:37.987 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:37.153+0000 7ff4b9382700 1 -- 172.21.15.52:0/1324569372 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4b4108810 con 0x7ff4b4108a50 2024-06-10T02:42:37.987 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:37.154+0000 7ff4b9382700 1 --2- 172.21.15.52:0/1324569372 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff4b4108a50 0x7ff4b41082d0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff4b00117e0 tx=0x7ff4b0011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:42:37.987 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:37.154+0000 7ff4a67fc700 1 -- 172.21.15.52:0/1324569372 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff4b0003ba0 con 0x7ff4b4108a50 2024-06-10T02:42:37.987 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:37.154+0000 7ff4a67fc700 1 -- 172.21.15.52:0/1324569372 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff4b001a910 con 0x7ff4b4108a50 2024-06-10T02:42:37.988 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:37.154+0000 7ff4a67fc700 1 -- 172.21.15.52:0/1324569372 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff4b00248f0 con 0x7ff4b4108a50 2024-06-10T02:42:37.988 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:37.154+0000 7ff4bb5e6700 1 -- 172.21.15.52:0/1324569372 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4b4106a00 con 0x7ff4b4108a50 2024-06-10T02:42:37.988 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:37.154+0000 7ff4bb5e6700 1 -- 172.21.15.52:0/1324569372 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4b4106ec0 con 0x7ff4b4108a50 2024-06-10T02:42:37.988 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:37.155+0000 7ff4a67fc700 1 -- 172.21.15.52:0/1324569372 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7ff4b0022020 con 0x7ff4b4108a50 2024-06-10T02:42:37.988 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:37.155+0000 7ff4bb5e6700 1 -- 172.21.15.52:0/1324569372 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4b4063af0 con 0x7ff4b4108a50 2024-06-10T02:42:37.988 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:37.155+0000 7ff4a67fc700 1 --2- 172.21.15.52:0/1324569372 >> [v2:172.21.15.52:6800/896317057,v1:172.21.15.52:6801/896317057] conn(0x7ff4a0039100 0x7ff4a003b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:42:37.988 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:37.155+0000 7ff4a67fc700 1 -- 172.21.15.52:0/1324569372 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7ff4b004e450 con 0x7ff4b4108a50 2024-06-10T02:42:37.988 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:37.155+0000 7ff4b8b81700 1 -- 172.21.15.52:0/1324569372 >> [v2:172.21.15.52:6800/896317057,v1:172.21.15.52:6801/896317057] conn(0x7ff4a0039100 msgr2=0x7ff4a003b5c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.52:6800/896317057 2024-06-10T02:42:37.988 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:37.155+0000 7ff4b8b81700 1 --2- 172.21.15.52:0/1324569372 >> [v2:172.21.15.52:6800/896317057,v1:172.21.15.52:6801/896317057] conn(0x7ff4a0039100 0x7ff4a003b5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-06-10T02:42:37.988 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:37.160+0000 7ff4a67fc700 1 -- 172.21.15.52:0/1324569372 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7ff4b000b3e0 con 0x7ff4b4108a50 2024-06-10T02:42:37.988 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:37.356+0000 7ff4b8b81700 1 -- 172.21.15.52:0/1324569372 >> [v2:172.21.15.52:6800/896317057,v1:172.21.15.52:6801/896317057] conn(0x7ff4a0039100 msgr2=0x7ff4a003b5c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.52:6800/896317057 2024-06-10T02:42:37.989 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:37.356+0000 7ff4b8b81700 1 --2- 172.21.15.52:0/1324569372 >> [v2:172.21.15.52:6800/896317057,v1:172.21.15.52:6801/896317057] conn(0x7ff4a0039100 0x7ff4a003b5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-06-10T02:42:37.989 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:37.425+0000 7ff4bb5e6700 1 -- 172.21.15.52:0/1324569372 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7ff4b4107660 con 0x7ff4b4108a50 2024-06-10T02:42:37.989 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:37.425+0000 7ff4a67fc700 1 -- 172.21.15.52:0/1324569372 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7ff4b0020c10 con 0x7ff4b4108a50 2024-06-10T02:42:37.989 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:37.428+0000 7ff4bb5e6700 1 -- 172.21.15.52:0/1324569372 >> [v2:172.21.15.52:6800/896317057,v1:172.21.15.52:6801/896317057] conn(0x7ff4a0039100 msgr2=0x7ff4a003b5c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-10T02:42:37.989 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:37.428+0000 7ff4bb5e6700 1 --2- 172.21.15.52:0/1324569372 >> [v2:172.21.15.52:6800/896317057,v1:172.21.15.52:6801/896317057] conn(0x7ff4a0039100 0x7ff4a003b5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:42:37.989 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:37.428+0000 7ff4bb5e6700 1 -- 172.21.15.52:0/1324569372 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff4b4108a50 msgr2=0x7ff4b41082d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:42:37.989 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:37.428+0000 7ff4bb5e6700 1 --2- 172.21.15.52:0/1324569372 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff4b4108a50 0x7ff4b41082d0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff4b00117e0 tx=0x7ff4b0011ba0 comp rx=0 tx=0).stop 2024-06-10T02:42:37.989 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:37.428+0000 7ff4bb5e6700 1 -- 172.21.15.52:0/1324569372 shutdown_connections 2024-06-10T02:42:37.989 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:37.428+0000 7ff4bb5e6700 1 --2- 172.21.15.52:0/1324569372 >> [v2:172.21.15.52:6800/896317057,v1:172.21.15.52:6801/896317057] conn(0x7ff4a0039100 0x7ff4a003b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:42:37.989 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:37.428+0000 7ff4bb5e6700 1 --2- 172.21.15.52:0/1324569372 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff4b4108a50 0x7ff4b41082d0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:42:37.989 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:37.428+0000 7ff4bb5e6700 1 -- 172.21.15.52:0/1324569372 >> 172.21.15.52:0/1324569372 conn(0x7ff4b41024a0 msgr2=0x7ff4b4102e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:42:37.990 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:37.428+0000 7ff4bb5e6700 1 -- 172.21.15.52:0/1324569372 shutdown_connections 2024-06-10T02:42:37.990 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:37.428+0000 7ff4bb5e6700 1 -- 172.21.15.52:0/1324569372 wait complete. 2024-06-10T02:42:37.990 INFO:teuthology.orchestra.run.smithi052.stdout:Waiting for the mgr to restart... 2024-06-10T02:42:37.990 INFO:teuthology.orchestra.run.smithi052.stdout:Waiting for mgr epoch 9... 2024-06-10T02:42:38.235 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 10 02:42:38 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mgr-x[32595]: 2024-06-10T02:42:38.059+0000 7fe32b7e6200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-06-10T02:42:38.236 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 10 02:42:38 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mgr-x[32595]: 2024-06-10T02:42:38.239+0000 7fe32b7e6200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-06-10T02:42:38.849 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 10 02:42:38 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mgr-x[32595]: 2024-06-10T02:42:38.545+0000 7fe32b7e6200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-06-10T02:42:38.849 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 10 02:42:38 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mgr-x[32595]: 2024-06-10T02:42:38.642+0000 7fe32b7e6200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-06-10T02:42:38.849 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 10 02:42:38 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mgr-x[32595]: 2024-06-10T02:42:38.853+0000 7fe32b7e6200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-06-10T02:42:39.223 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 10 02:42:39 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mgr-x[32595]: 2024-06-10T02:42:39.055+0000 7fe32b7e6200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-06-10T02:42:39.973 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 10 02:42:39 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mgr-x[32595]: 2024-06-10T02:42:39.685+0000 7fe32b7e6200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-06-10T02:42:40.433 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 10 02:42:40 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mgr-x[32595]: 2024-06-10T02:42:40.125+0000 7fe32b7e6200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-06-10T02:42:40.433 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 10 02:42:40 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mgr-x[32595]: 2024-06-10T02:42:40.227+0000 7fe32b7e6200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-06-10T02:42:40.433 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 10 02:42:40 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mgr-x[32595]: 2024-06-10T02:42:40.436+0000 7fe32b7e6200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-06-10T02:42:41.223 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 10 02:42:40 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mgr-x[32595]: 2024-06-10T02:42:40.984+0000 7fe32b7e6200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-06-10T02:42:41.223 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 10 02:42:41 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mgr-x[32595]: 2024-06-10T02:42:41.081+0000 7fe32b7e6200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-06-10T02:42:41.697 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 10 02:42:41 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mgr-x[32595]: 2024-06-10T02:42:41.335+0000 7fe32b7e6200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-06-10T02:42:41.697 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 10 02:42:41 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mgr-x[32595]: 2024-06-10T02:42:41.475+0000 7fe32b7e6200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-06-10T02:42:41.697 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 10 02:42:41 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mgr-x[32595]: 2024-06-10T02:42:41.579+0000 7fe32b7e6200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-06-10T02:42:41.697 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 10 02:42:41 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mgr-x[32595]: 2024-06-10T02:42:41.700+0000 7fe32b7e6200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-06-10T02:42:42.473 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 10 02:42:42 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mgr-x[32595]: 2024-06-10T02:42:42.011+0000 7fe32b7e6200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-06-10T02:42:42.919 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 10 02:42:42 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mgr-x[32595]: 2024-06-10T02:42:42.644+0000 7fe32b7e6200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-06-10T02:42:42.919 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 10 02:42:42 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mgr-x[32595]: 2024-06-10T02:42:42.870+0000 7fe32b7e6200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-06-10T02:42:43.223 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:42:42 smithi052 ceph-mon[32362]: Active manager daemon x restarted 2024-06-10T02:42:43.223 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:42:42 smithi052 ceph-mon[32362]: Activating manager daemon x 2024-06-10T02:42:43.223 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:42:42 smithi052 ceph-mon[32362]: osdmap e3: 0 total, 0 up, 0 in 2024-06-10T02:42:43.223 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:42:42 smithi052 ceph-mon[32362]: mgrmap e10: x(active, starting, since 0.00337293s) 2024-06-10T02:42:43.223 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:42:42 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-10T02:42:43.223 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:42:42 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-06-10T02:42:43.223 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:42:42 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-06-10T02:42:43.223 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:42:42 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-06-10T02:42:43.224 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:42:42 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-06-10T02:42:43.224 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:42:42 smithi052 ceph-mon[32362]: Manager daemon x is now available 2024-06-10T02:42:43.224 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:42:42 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T02:42:44.099 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:42:43 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-06-10T02:42:44.099 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:42:43 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-06-10T02:42:44.099 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:42:43 smithi052 ceph-mon[32362]: mgrmap e11: x(active, since 1.00654s) 2024-06-10T02:42:44.813 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout { 2024-06-10T02:42:44.813 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-06-10T02:42:44.813 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "initialized": true 2024-06-10T02:42:44.813 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout } 2024-06-10T02:42:44.813 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:38.388+0000 7f3b941c9700 1 Processor -- start 2024-06-10T02:42:44.813 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:38.388+0000 7f3b941c9700 1 -- start start 2024-06-10T02:42:44.813 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:38.388+0000 7f3b941c9700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3b8c108a50 0x7f3b8c10ae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:42:44.814 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:38.388+0000 7f3b941c9700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f3b8c07c6d0 con 0x7f3b8c108a50 2024-06-10T02:42:44.814 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:38.389+0000 7f3b91f65700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3b8c108a50 0x7f3b8c10ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:42:44.814 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:38.389+0000 7f3b91f65700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3b8c108a50 0x7f3b8c10ae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49474/0 (socket says 172.21.15.52:49474) 2024-06-10T02:42:44.814 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:38.389+0000 7f3b91f65700 1 -- 172.21.15.52:0/725641940 learned_addr learned my addr 172.21.15.52:0/725641940 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:42:44.814 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:38.389+0000 7f3b91f65700 1 -- 172.21.15.52:0/725641940 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b8c10b380 con 0x7f3b8c108a50 2024-06-10T02:42:44.814 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:38.389+0000 7f3b91f65700 1 --2- 172.21.15.52:0/725641940 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3b8c108a50 0x7f3b8c10ae40 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f3b880098a0 tx=0x7f3b88009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=794cfdb3f61636b3 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:42:44.814 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:38.389+0000 7f3b90f63700 1 -- 172.21.15.52:0/725641940 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3b88004160 con 0x7f3b8c108a50 2024-06-10T02:42:44.814 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:38.389+0000 7f3b90f63700 1 -- 172.21.15.52:0/725641940 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b88004300 con 0x7f3b8c108a50 2024-06-10T02:42:44.814 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:38.389+0000 7f3b90f63700 1 -- 172.21.15.52:0/725641940 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3b88011b20 con 0x7f3b8c108a50 2024-06-10T02:42:44.814 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:38.390+0000 7f3b941c9700 1 -- 172.21.15.52:0/725641940 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3b8c108a50 msgr2=0x7f3b8c10ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:42:44.814 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:38.390+0000 7f3b941c9700 1 --2- 172.21.15.52:0/725641940 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3b8c108a50 0x7f3b8c10ae40 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f3b880098a0 tx=0x7f3b88009bb0 comp rx=0 tx=0).stop 2024-06-10T02:42:44.815 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:38.390+0000 7f3b941c9700 1 -- 172.21.15.52:0/725641940 shutdown_connections 2024-06-10T02:42:44.815 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:38.390+0000 7f3b941c9700 1 --2- 172.21.15.52:0/725641940 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3b8c108a50 0x7f3b8c10ae40 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:42:44.815 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:38.390+0000 7f3b941c9700 1 -- 172.21.15.52:0/725641940 >> 172.21.15.52:0/725641940 conn(0x7f3b8c102440 msgr2=0x7f3b8c104860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:42:44.815 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:38.390+0000 7f3b941c9700 1 -- 172.21.15.52:0/725641940 shutdown_connections 2024-06-10T02:42:44.815 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:38.390+0000 7f3b941c9700 1 -- 172.21.15.52:0/725641940 wait complete. 2024-06-10T02:42:44.815 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:38.391+0000 7f3b941c9700 1 Processor -- start 2024-06-10T02:42:44.815 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:38.391+0000 7f3b941c9700 1 -- start start 2024-06-10T02:42:44.815 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:38.391+0000 7f3b941c9700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3b8c108a50 0x7f3b8c19be70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:42:44.815 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:38.391+0000 7f3b941c9700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f3b8c10c380 con 0x7f3b8c108a50 2024-06-10T02:42:44.815 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:38.391+0000 7f3b91f65700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3b8c108a50 0x7f3b8c19be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:42:44.815 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:38.391+0000 7f3b91f65700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3b8c108a50 0x7f3b8c19be70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49488/0 (socket says 172.21.15.52:49488) 2024-06-10T02:42:44.815 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:38.391+0000 7f3b91f65700 1 -- 172.21.15.52:0/2999035399 learned_addr learned my addr 172.21.15.52:0/2999035399 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:42:44.816 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:38.391+0000 7f3b91f65700 1 -- 172.21.15.52:0/2999035399 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b8c19c3b0 con 0x7f3b8c108a50 2024-06-10T02:42:44.816 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:38.392+0000 7f3b91f65700 1 --2- 172.21.15.52:0/2999035399 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3b8c108a50 0x7f3b8c19be70 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f3b880117e0 tx=0x7f3b88011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:42:44.816 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:38.392+0000 7f3b82ffd700 1 -- 172.21.15.52:0/2999035399 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3b88003bf0 con 0x7f3b8c108a50 2024-06-10T02:42:44.816 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:38.392+0000 7f3b82ffd700 1 -- 172.21.15.52:0/2999035399 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b8801a910 con 0x7f3b8c108a50 2024-06-10T02:42:44.816 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:38.392+0000 7f3b941c9700 1 -- 172.21.15.52:0/2999035399 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b8c19c640 con 0x7f3b8c108a50 2024-06-10T02:42:44.816 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:38.392+0000 7f3b82ffd700 1 -- 172.21.15.52:0/2999035399 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3b88024970 con 0x7f3b8c108a50 2024-06-10T02:42:44.816 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:38.392+0000 7f3b941c9700 1 -- 172.21.15.52:0/2999035399 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b8c19caa0 con 0x7f3b8c108a50 2024-06-10T02:42:44.816 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:38.393+0000 7f3b82ffd700 1 -- 172.21.15.52:0/2999035399 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7f3b88022020 con 0x7f3b8c108a50 2024-06-10T02:42:44.816 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:38.393+0000 7f3b82ffd700 1 --2- 172.21.15.52:0/2999035399 >> [v2:172.21.15.52:6800/896317057,v1:172.21.15.52:6801/896317057] conn(0x7f3b780390b0 0x7f3b7803b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:42:44.816 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:38.393+0000 7f3b82ffd700 1 -- 172.21.15.52:0/2999035399 --> [v2:172.21.15.52:6800/896317057,v1:172.21.15.52:6801/896317057] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f3b7803bc80 con 0x7f3b780390b0 2024-06-10T02:42:44.816 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:38.393+0000 7f3b91764700 1 -- 172.21.15.52:0/2999035399 >> [v2:172.21.15.52:6800/896317057,v1:172.21.15.52:6801/896317057] conn(0x7f3b780390b0 msgr2=0x7f3b7803b570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.52:6800/896317057 2024-06-10T02:42:44.816 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:38.393+0000 7f3b82ffd700 1 -- 172.21.15.52:0/2999035399 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f3b8804d360 con 0x7f3b8c108a50 2024-06-10T02:42:44.816 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:38.393+0000 7f3b91764700 1 --2- 172.21.15.52:0/2999035399 >> [v2:172.21.15.52:6800/896317057,v1:172.21.15.52:6801/896317057] conn(0x7f3b780390b0 0x7f3b7803b570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-06-10T02:42:44.817 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:38.594+0000 7f3b91764700 1 -- 172.21.15.52:0/2999035399 >> [v2:172.21.15.52:6800/896317057,v1:172.21.15.52:6801/896317057] conn(0x7f3b780390b0 msgr2=0x7f3b7803b570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.52:6800/896317057 2024-06-10T02:42:44.817 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:38.594+0000 7f3b91764700 1 --2- 172.21.15.52:0/2999035399 >> [v2:172.21.15.52:6800/896317057,v1:172.21.15.52:6801/896317057] conn(0x7f3b780390b0 0x7f3b7803b570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-06-10T02:42:44.817 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:38.995+0000 7f3b91764700 1 -- 172.21.15.52:0/2999035399 >> [v2:172.21.15.52:6800/896317057,v1:172.21.15.52:6801/896317057] conn(0x7f3b780390b0 msgr2=0x7f3b7803b570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.52:6800/896317057 2024-06-10T02:42:44.817 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:38.995+0000 7f3b91764700 1 --2- 172.21.15.52:0/2999035399 >> [v2:172.21.15.52:6800/896317057,v1:172.21.15.52:6801/896317057] conn(0x7f3b780390b0 0x7f3b7803b570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-06-10T02:42:44.817 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:39.796+0000 7f3b91764700 1 -- 172.21.15.52:0/2999035399 >> [v2:172.21.15.52:6800/896317057,v1:172.21.15.52:6801/896317057] conn(0x7f3b780390b0 msgr2=0x7f3b7803b570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.52:6800/896317057 2024-06-10T02:42:44.817 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:39.796+0000 7f3b91764700 1 --2- 172.21.15.52:0/2999035399 >> [v2:172.21.15.52:6800/896317057,v1:172.21.15.52:6801/896317057] conn(0x7f3b780390b0 0x7f3b7803b570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-06-10T02:42:44.818 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:41.397+0000 7f3b91764700 1 -- 172.21.15.52:0/2999035399 >> [v2:172.21.15.52:6800/896317057,v1:172.21.15.52:6801/896317057] conn(0x7f3b780390b0 msgr2=0x7f3b7803b570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.52:6800/896317057 2024-06-10T02:42:44.818 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:41.397+0000 7f3b91764700 1 --2- 172.21.15.52:0/2999035399 >> [v2:172.21.15.52:6800/896317057,v1:172.21.15.52:6801/896317057] conn(0x7f3b780390b0 0x7f3b7803b570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-06-10T02:42:44.818 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:42.873+0000 7f3b82ffd700 1 -- 172.21.15.52:0/2999035399 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mgrmap(e 10) ==== 46067+0+0 (secure 0 0 0) 0x7f3b8801a430 con 0x7f3b8c108a50 2024-06-10T02:42:44.818 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:42.873+0000 7f3b82ffd700 1 -- 172.21.15.52:0/2999035399 >> [v2:172.21.15.52:6800/896317057,v1:172.21.15.52:6801/896317057] conn(0x7f3b780390b0 msgr2=0x7f3b7803b570 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-10T02:42:44.818 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:42.873+0000 7f3b82ffd700 1 --2- 172.21.15.52:0/2999035399 >> [v2:172.21.15.52:6800/896317057,v1:172.21.15.52:6801/896317057] conn(0x7f3b780390b0 0x7f3b7803b570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:42:44.818 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:43.875+0000 7f3b82ffd700 1 -- 172.21.15.52:0/2999035399 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mgrmap(e 11) ==== 46194+0+0 (secure 0 0 0) 0x7f3b8802bb50 con 0x7f3b8c108a50 2024-06-10T02:42:44.818 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:43.875+0000 7f3b82ffd700 1 --2- 172.21.15.52:0/2999035399 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f3b7803cb70 0x7f3b7803ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:42:44.818 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:43.875+0000 7f3b82ffd700 1 -- 172.21.15.52:0/2999035399 --> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f3b88019e50 con 0x7f3b7803cb70 2024-06-10T02:42:44.818 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:43.876+0000 7f3b91764700 1 --2- 172.21.15.52:0/2999035399 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f3b7803cb70 0x7f3b7803ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:42:44.818 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:43.877+0000 7f3b91764700 1 --2- 172.21.15.52:0/2999035399 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f3b7803cb70 0x7f3b7803ef60 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f3b7c003a10 tx=0x7f3b7c009270 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:42:44.818 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:43.877+0000 7f3b82ffd700 1 -- 172.21.15.52:0/2999035399 <== mgr.14150 v2:172.21.15.52:6800/3214763969 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7f3b88019e50 con 0x7f3b7803cb70 2024-06-10T02:42:44.818 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:43.889+0000 7f3b941c9700 1 -- 172.21.15.52:0/2999035399 --> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f3b8c063af0 con 0x7f3b7803cb70 2024-06-10T02:42:44.819 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:43.889+0000 7f3b82ffd700 1 -- 172.21.15.52:0/2999035399 <== mgr.14150 v2:172.21.15.52:6800/3214763969 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7f3b8c063af0 con 0x7f3b7803cb70 2024-06-10T02:42:44.819 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:43.890+0000 7f3b941c9700 1 -- 172.21.15.52:0/2999035399 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f3b7803cb70 msgr2=0x7f3b7803ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:42:44.819 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:43.890+0000 7f3b941c9700 1 --2- 172.21.15.52:0/2999035399 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f3b7803cb70 0x7f3b7803ef60 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f3b7c003a10 tx=0x7f3b7c009270 comp rx=0 tx=0).stop 2024-06-10T02:42:44.819 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:43.890+0000 7f3b941c9700 1 -- 172.21.15.52:0/2999035399 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3b8c108a50 msgr2=0x7f3b8c19be70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:42:44.819 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:43.890+0000 7f3b941c9700 1 --2- 172.21.15.52:0/2999035399 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3b8c108a50 0x7f3b8c19be70 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f3b880117e0 tx=0x7f3b88011c10 comp rx=0 tx=0).stop 2024-06-10T02:42:44.819 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:43.890+0000 7f3b941c9700 1 -- 172.21.15.52:0/2999035399 shutdown_connections 2024-06-10T02:42:44.819 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:43.890+0000 7f3b941c9700 1 --2- 172.21.15.52:0/2999035399 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f3b7803cb70 0x7f3b7803ef60 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:42:44.819 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:43.890+0000 7f3b941c9700 1 --2- 172.21.15.52:0/2999035399 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3b8c108a50 0x7f3b8c19be70 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:42:44.819 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:43.890+0000 7f3b941c9700 1 --2- 172.21.15.52:0/2999035399 >> [v2:172.21.15.52:6800/896317057,v1:172.21.15.52:6801/896317057] conn(0x7f3b780390b0 0x7f3b7803b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:42:44.819 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:43.890+0000 7f3b941c9700 1 -- 172.21.15.52:0/2999035399 >> 172.21.15.52:0/2999035399 conn(0x7f3b8c102440 msgr2=0x7f3b8c102e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:42:44.819 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:43.890+0000 7f3b941c9700 1 -- 172.21.15.52:0/2999035399 shutdown_connections 2024-06-10T02:42:44.819 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:43.890+0000 7f3b941c9700 1 -- 172.21.15.52:0/2999035399 wait complete. 2024-06-10T02:42:44.819 INFO:teuthology.orchestra.run.smithi052.stdout:mgr epoch 9 is available 2024-06-10T02:42:44.819 INFO:teuthology.orchestra.run.smithi052.stdout:Generating a dashboard self-signed certificate... 2024-06-10T02:42:45.723 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:42:45 smithi052 ceph-mon[32362]: [10/Jun/2024:02:42:44] ENGINE Bus STARTING 2024-06-10T02:42:45.723 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:42:45 smithi052 ceph-mon[32362]: [10/Jun/2024:02:42:44] ENGINE Serving on https://172.21.15.52:7150 2024-06-10T02:42:45.723 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:42:45 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:42:45.723 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:42:45 smithi052 ceph-mon[32362]: [10/Jun/2024:02:42:44] ENGINE Serving on http://172.21.15.52:8765 2024-06-10T02:42:45.723 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:42:45 smithi052 ceph-mon[32362]: [10/Jun/2024:02:42:44] ENGINE Bus STARTED 2024-06-10T02:42:46.126 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-06-10T02:42:46.126 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:45.233+0000 7ff5ea5c2700 1 Processor -- start 2024-06-10T02:42:46.126 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:45.233+0000 7ff5ea5c2700 1 -- start start 2024-06-10T02:42:46.126 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:45.234+0000 7ff5ea5c2700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff5e4107ff0 0x7ff5e4108410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:42:46.126 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:45.234+0000 7ff5ea5c2700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ff5e41089e0 con 0x7ff5e4107ff0 2024-06-10T02:42:46.126 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:45.234+0000 7ff5e3fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff5e4107ff0 0x7ff5e4108410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:42:46.126 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:45.234+0000 7ff5e3fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff5e4107ff0 0x7ff5e4108410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:44270/0 (socket says 172.21.15.52:44270) 2024-06-10T02:42:46.127 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:45.234+0000 7ff5e3fff700 1 -- 172.21.15.52:0/3450403647 learned_addr learned my addr 172.21.15.52:0/3450403647 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:42:46.127 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:45.234+0000 7ff5e3fff700 1 -- 172.21.15.52:0/3450403647 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5e41091a0 con 0x7ff5e4107ff0 2024-06-10T02:42:46.127 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:45.234+0000 7ff5e3fff700 1 --2- 172.21.15.52:0/3450403647 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff5e4107ff0 0x7ff5e4108410 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ff5d40098a0 tx=0x7ff5d4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7218c43327908d05 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:42:46.127 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:45.234+0000 7ff5e2ffd700 1 -- 172.21.15.52:0/3450403647 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff5d4004160 con 0x7ff5e4107ff0 2024-06-10T02:42:46.127 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:45.235+0000 7ff5e2ffd700 1 -- 172.21.15.52:0/3450403647 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5d4004300 con 0x7ff5e4107ff0 2024-06-10T02:42:46.127 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:45.235+0000 7ff5e2ffd700 1 -- 172.21.15.52:0/3450403647 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff5d4011b20 con 0x7ff5e4107ff0 2024-06-10T02:42:46.127 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:45.235+0000 7ff5ea5c2700 1 -- 172.21.15.52:0/3450403647 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff5e4107ff0 msgr2=0x7ff5e4108410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:42:46.127 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:45.235+0000 7ff5ea5c2700 1 --2- 172.21.15.52:0/3450403647 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff5e4107ff0 0x7ff5e4108410 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ff5d40098a0 tx=0x7ff5d4009bb0 comp rx=0 tx=0).stop 2024-06-10T02:42:46.127 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:45.235+0000 7ff5ea5c2700 1 -- 172.21.15.52:0/3450403647 shutdown_connections 2024-06-10T02:42:46.127 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:45.235+0000 7ff5ea5c2700 1 --2- 172.21.15.52:0/3450403647 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff5e4107ff0 0x7ff5e4108410 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:42:46.127 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:45.235+0000 7ff5ea5c2700 1 -- 172.21.15.52:0/3450403647 >> 172.21.15.52:0/3450403647 conn(0x7ff5e4103590 msgr2=0x7ff5e41059d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:42:46.127 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:45.235+0000 7ff5ea5c2700 1 -- 172.21.15.52:0/3450403647 shutdown_connections 2024-06-10T02:42:46.127 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:45.236+0000 7ff5ea5c2700 1 -- 172.21.15.52:0/3450403647 wait complete. 2024-06-10T02:42:46.127 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:45.236+0000 7ff5ea5c2700 1 Processor -- start 2024-06-10T02:42:46.128 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:45.236+0000 7ff5ea5c2700 1 -- start start 2024-06-10T02:42:46.128 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:45.237+0000 7ff5ea5c2700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff5e4107ff0 0x7ff5e419bed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:42:46.128 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:45.237+0000 7ff5ea5c2700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ff5e410c550 con 0x7ff5e4107ff0 2024-06-10T02:42:46.128 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:45.237+0000 7ff5e3fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff5e4107ff0 0x7ff5e419bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:42:46.128 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:45.237+0000 7ff5e3fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff5e4107ff0 0x7ff5e419bed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:44280/0 (socket says 172.21.15.52:44280) 2024-06-10T02:42:46.128 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:45.237+0000 7ff5e3fff700 1 -- 172.21.15.52:0/750567269 learned_addr learned my addr 172.21.15.52:0/750567269 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:42:46.128 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:45.237+0000 7ff5e3fff700 1 -- 172.21.15.52:0/750567269 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5e419c410 con 0x7ff5e4107ff0 2024-06-10T02:42:46.128 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:45.237+0000 7ff5e3fff700 1 --2- 172.21.15.52:0/750567269 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff5e4107ff0 0x7ff5e419bed0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7ff5d4000c00 tx=0x7ff5d4011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:42:46.128 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:45.238+0000 7ff5e17fa700 1 -- 172.21.15.52:0/750567269 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff5d4003b60 con 0x7ff5e4107ff0 2024-06-10T02:42:46.128 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:45.238+0000 7ff5e17fa700 1 -- 172.21.15.52:0/750567269 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5d401a910 con 0x7ff5e4107ff0 2024-06-10T02:42:46.128 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:45.238+0000 7ff5ea5c2700 1 -- 172.21.15.52:0/750567269 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5e419c6a0 con 0x7ff5e4107ff0 2024-06-10T02:42:46.128 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:45.238+0000 7ff5e17fa700 1 -- 172.21.15.52:0/750567269 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff5d4024930 con 0x7ff5e4107ff0 2024-06-10T02:42:46.128 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:45.238+0000 7ff5ea5c2700 1 -- 172.21.15.52:0/750567269 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5e419cb00 con 0x7ff5e4107ff0 2024-06-10T02:42:46.128 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:45.239+0000 7ff5e17fa700 1 -- 172.21.15.52:0/750567269 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 11) ==== 46194+0+0 (secure 0 0 0) 0x7ff5d4022020 con 0x7ff5e4107ff0 2024-06-10T02:42:46.129 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:45.239+0000 7ff5e17fa700 1 --2- 172.21.15.52:0/750567269 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7ff5cc038ff0 0x7ff5cc03b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:42:46.129 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:45.239+0000 7ff5e17fa700 1 -- 172.21.15.52:0/750567269 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7ff5d404d130 con 0x7ff5e4107ff0 2024-06-10T02:42:46.129 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:45.239+0000 7ff5e37fe700 1 --2- 172.21.15.52:0/750567269 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7ff5cc038ff0 0x7ff5cc03b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:42:46.129 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:45.239+0000 7ff5ea5c2700 1 -- 172.21.15.52:0/750567269 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5e4063af0 con 0x7ff5e4107ff0 2024-06-10T02:42:46.129 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:45.239+0000 7ff5e37fe700 1 --2- 172.21.15.52:0/750567269 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7ff5cc038ff0 0x7ff5cc03b4b0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff5d8006fd0 tx=0x7ff5d8006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:42:46.129 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:45.244+0000 7ff5e17fa700 1 -- 172.21.15.52:0/750567269 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7ff5d400e260 con 0x7ff5e4107ff0 2024-06-10T02:42:46.129 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:45.460+0000 7ff5ea5c2700 1 -- 172.21.15.52:0/750567269 --> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7ff5e4104700 con 0x7ff5cc038ff0 2024-06-10T02:42:46.129 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:45.481+0000 7ff5e17fa700 1 -- 172.21.15.52:0/750567269 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7ff5d40193c0 con 0x7ff5e4107ff0 2024-06-10T02:42:46.129 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:45.563+0000 7ff5e17fa700 1 -- 172.21.15.52:0/750567269 <== mgr.14150 v2:172.21.15.52:6800/3214763969 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7ff5e4104700 con 0x7ff5cc038ff0 2024-06-10T02:42:46.129 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:45.565+0000 7ff5ea5c2700 1 -- 172.21.15.52:0/750567269 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7ff5cc038ff0 msgr2=0x7ff5cc03b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:42:46.129 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:45.565+0000 7ff5ea5c2700 1 --2- 172.21.15.52:0/750567269 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7ff5cc038ff0 0x7ff5cc03b4b0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff5d8006fd0 tx=0x7ff5d8006e40 comp rx=0 tx=0).stop 2024-06-10T02:42:46.129 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:45.565+0000 7ff5ea5c2700 1 -- 172.21.15.52:0/750567269 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff5e4107ff0 msgr2=0x7ff5e419bed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:42:46.129 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:45.565+0000 7ff5ea5c2700 1 --2- 172.21.15.52:0/750567269 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff5e4107ff0 0x7ff5e419bed0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7ff5d4000c00 tx=0x7ff5d4011c10 comp rx=0 tx=0).stop 2024-06-10T02:42:46.130 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:45.566+0000 7ff5ea5c2700 1 -- 172.21.15.52:0/750567269 shutdown_connections 2024-06-10T02:42:46.130 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:45.566+0000 7ff5ea5c2700 1 --2- 172.21.15.52:0/750567269 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7ff5cc038ff0 0x7ff5cc03b4b0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:42:46.130 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:45.566+0000 7ff5ea5c2700 1 --2- 172.21.15.52:0/750567269 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff5e4107ff0 0x7ff5e419bed0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:42:46.130 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:45.566+0000 7ff5ea5c2700 1 -- 172.21.15.52:0/750567269 >> 172.21.15.52:0/750567269 conn(0x7ff5e4103590 msgr2=0x7ff5e4103ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:42:46.130 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:45.566+0000 7ff5ea5c2700 1 -- 172.21.15.52:0/750567269 shutdown_connections 2024-06-10T02:42:46.130 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:45.566+0000 7ff5ea5c2700 1 -- 172.21.15.52:0/750567269 wait complete. 2024-06-10T02:42:46.130 INFO:teuthology.orchestra.run.smithi052.stdout:Creating initial admin user... 2024-06-10T02:42:46.766 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:42:46 smithi052 ceph-mon[32362]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T02:42:46.766 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:42:46 smithi052 ceph-mon[32362]: mgrmap e12: x(active, since 2s) 2024-06-10T02:42:46.766 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:42:46 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:42:46.766 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:42:46 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:42:46.766 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:42:46 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:42:47.646 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$PW92tqYneUGABAFJihDQluBS6oh9Wl4tis9xF.kUaozyHZFURtC.G", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1717987367, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-06-10T02:42:47.647 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:46.531+0000 7fb3b7cbd700 1 Processor -- start 2024-06-10T02:42:47.647 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:46.532+0000 7fb3b7cbd700 1 -- start start 2024-06-10T02:42:47.647 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:46.532+0000 7fb3b7cbd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb3b007e6a0 0x7fb3b007cba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:42:47.647 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:46.532+0000 7fb3b7cbd700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb3b007d0e0 con 0x7fb3b007e6a0 2024-06-10T02:42:47.647 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:46.532+0000 7fb3b5a59700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb3b007e6a0 0x7fb3b007cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:42:47.647 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:46.532+0000 7fb3b5a59700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb3b007e6a0 0x7fb3b007cba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:44292/0 (socket says 172.21.15.52:44292) 2024-06-10T02:42:47.647 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:46.532+0000 7fb3b5a59700 1 -- 172.21.15.52:0/499459714 learned_addr learned my addr 172.21.15.52:0/499459714 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:42:47.647 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:46.532+0000 7fb3b5a59700 1 -- 172.21.15.52:0/499459714 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3b007d260 con 0x7fb3b007e6a0 2024-06-10T02:42:47.647 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:46.533+0000 7fb3b5a59700 1 --2- 172.21.15.52:0/499459714 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb3b007e6a0 0x7fb3b007cba0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb3ac0098a0 tx=0x7fb3ac009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f6d8d02c4d60d1 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:42:47.647 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:46.533+0000 7fb3b4a57700 1 -- 172.21.15.52:0/499459714 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb3ac004160 con 0x7fb3b007e6a0 2024-06-10T02:42:47.647 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:46.533+0000 7fb3b4a57700 1 -- 172.21.15.52:0/499459714 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb3ac004300 con 0x7fb3b007e6a0 2024-06-10T02:42:47.648 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:46.533+0000 7fb3b4a57700 1 -- 172.21.15.52:0/499459714 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb3ac011b20 con 0x7fb3b007e6a0 2024-06-10T02:42:47.648 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:46.534+0000 7fb3b7cbd700 1 -- 172.21.15.52:0/499459714 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb3b007e6a0 msgr2=0x7fb3b007cba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:42:47.648 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:46.534+0000 7fb3b7cbd700 1 --2- 172.21.15.52:0/499459714 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb3b007e6a0 0x7fb3b007cba0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb3ac0098a0 tx=0x7fb3ac009bb0 comp rx=0 tx=0).stop 2024-06-10T02:42:47.648 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:46.534+0000 7fb3b7cbd700 1 -- 172.21.15.52:0/499459714 shutdown_connections 2024-06-10T02:42:47.648 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:46.534+0000 7fb3b7cbd700 1 --2- 172.21.15.52:0/499459714 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb3b007e6a0 0x7fb3b007cba0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:42:47.648 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:46.534+0000 7fb3b7cbd700 1 -- 172.21.15.52:0/499459714 >> 172.21.15.52:0/499459714 conn(0x7fb3b01035b0 msgr2=0x7fb3b0105a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:42:47.648 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:46.534+0000 7fb3b7cbd700 1 -- 172.21.15.52:0/499459714 shutdown_connections 2024-06-10T02:42:47.648 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:46.534+0000 7fb3b7cbd700 1 -- 172.21.15.52:0/499459714 wait complete. 2024-06-10T02:42:47.648 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:46.534+0000 7fb3b7cbd700 1 Processor -- start 2024-06-10T02:42:47.648 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:46.535+0000 7fb3b7cbd700 1 -- start start 2024-06-10T02:42:47.648 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:46.535+0000 7fb3b7cbd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb3b007e6a0 0x7fb3b019e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:42:47.648 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:46.535+0000 7fb3b7cbd700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb3b010e5d0 con 0x7fb3b007e6a0 2024-06-10T02:42:47.648 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:46.535+0000 7fb3b5a59700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb3b007e6a0 0x7fb3b019e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:42:47.649 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:46.535+0000 7fb3b5a59700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb3b007e6a0 0x7fb3b019e080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:44302/0 (socket says 172.21.15.52:44302) 2024-06-10T02:42:47.649 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:46.535+0000 7fb3b5a59700 1 -- 172.21.15.52:0/1688629356 learned_addr learned my addr 172.21.15.52:0/1688629356 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:42:47.649 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:46.535+0000 7fb3b5a59700 1 -- 172.21.15.52:0/1688629356 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3b019e5c0 con 0x7fb3b007e6a0 2024-06-10T02:42:47.649 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:46.535+0000 7fb3b5a59700 1 --2- 172.21.15.52:0/1688629356 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb3b007e6a0 0x7fb3b019e080 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb3ac0117e0 tx=0x7fb3ac011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:42:47.649 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:46.535+0000 7fb3a6ffd700 1 -- 172.21.15.52:0/1688629356 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb3ac003ba0 con 0x7fb3b007e6a0 2024-06-10T02:42:47.649 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:46.535+0000 7fb3a6ffd700 1 -- 172.21.15.52:0/1688629356 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb3ac01a910 con 0x7fb3b007e6a0 2024-06-10T02:42:47.649 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:46.535+0000 7fb3b7cbd700 1 -- 172.21.15.52:0/1688629356 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3b019e850 con 0x7fb3b007e6a0 2024-06-10T02:42:47.649 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:46.536+0000 7fb3a6ffd700 1 -- 172.21.15.52:0/1688629356 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb3ac0248f0 con 0x7fb3b007e6a0 2024-06-10T02:42:47.649 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:46.536+0000 7fb3b7cbd700 1 -- 172.21.15.52:0/1688629356 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3b019ecb0 con 0x7fb3b007e6a0 2024-06-10T02:42:47.649 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:46.536+0000 7fb3a6ffd700 1 -- 172.21.15.52:0/1688629356 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7fb3ac022020 con 0x7fb3b007e6a0 2024-06-10T02:42:47.649 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:46.537+0000 7fb3a6ffd700 1 --2- 172.21.15.52:0/1688629356 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fb39c0390a0 0x7fb39c03b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:42:47.649 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:46.537+0000 7fb3b7cbd700 1 -- 172.21.15.52:0/1688629356 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3b0063af0 con 0x7fb3b007e6a0 2024-06-10T02:42:47.650 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:46.537+0000 7fb3a6ffd700 1 -- 172.21.15.52:0/1688629356 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fb3ac04d9f0 con 0x7fb3b007e6a0 2024-06-10T02:42:47.650 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:46.537+0000 7fb3b5258700 1 --2- 172.21.15.52:0/1688629356 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fb39c0390a0 0x7fb39c03b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:42:47.650 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:46.537+0000 7fb3b5258700 1 --2- 172.21.15.52:0/1688629356 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fb39c0390a0 0x7fb39c03b560 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb3a0006fd0 tx=0x7fb3a0006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:42:47.650 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:46.543+0000 7fb3a6ffd700 1 -- 172.21.15.52:0/1688629356 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fb3ac00b3e0 con 0x7fb3b007e6a0 2024-06-10T02:42:47.650 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:46.770+0000 7fb3b7cbd700 1 -- 172.21.15.52:0/1688629356 --> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7fb3b01a16c0 con 0x7fb39c0390a0 2024-06-10T02:42:47.650 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:47.017+0000 7fb3a6ffd700 1 -- 172.21.15.52:0/1688629356 <== mgr.14150 v2:172.21.15.52:6800/3214763969 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7fb3b01a16c0 con 0x7fb39c0390a0 2024-06-10T02:42:47.650 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:47.019+0000 7fb3b7cbd700 1 -- 172.21.15.52:0/1688629356 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fb39c0390a0 msgr2=0x7fb39c03b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:42:47.650 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:47.019+0000 7fb3b7cbd700 1 --2- 172.21.15.52:0/1688629356 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fb39c0390a0 0x7fb39c03b560 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb3a0006fd0 tx=0x7fb3a0006e40 comp rx=0 tx=0).stop 2024-06-10T02:42:47.650 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:47.019+0000 7fb3b7cbd700 1 -- 172.21.15.52:0/1688629356 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb3b007e6a0 msgr2=0x7fb3b019e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:42:47.650 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:47.019+0000 7fb3b7cbd700 1 --2- 172.21.15.52:0/1688629356 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb3b007e6a0 0x7fb3b019e080 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb3ac0117e0 tx=0x7fb3ac011ba0 comp rx=0 tx=0).stop 2024-06-10T02:42:47.650 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:47.019+0000 7fb3b7cbd700 1 -- 172.21.15.52:0/1688629356 shutdown_connections 2024-06-10T02:42:47.650 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:47.019+0000 7fb3b7cbd700 1 --2- 172.21.15.52:0/1688629356 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fb39c0390a0 0x7fb39c03b560 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:42:47.651 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:47.019+0000 7fb3b7cbd700 1 --2- 172.21.15.52:0/1688629356 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb3b007e6a0 0x7fb3b019e080 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:42:47.651 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:47.019+0000 7fb3b7cbd700 1 -- 172.21.15.52:0/1688629356 >> 172.21.15.52:0/1688629356 conn(0x7fb3b01035b0 msgr2=0x7fb3b0104060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:42:47.651 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:47.020+0000 7fb3b7cbd700 1 -- 172.21.15.52:0/1688629356 shutdown_connections 2024-06-10T02:42:47.651 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:47.020+0000 7fb3b7cbd700 1 -- 172.21.15.52:0/1688629356 wait complete. 2024-06-10T02:42:47.651 INFO:teuthology.orchestra.run.smithi052.stdout:Fetching dashboard port number... 2024-06-10T02:42:48.209 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:42:48 smithi052 ceph-mon[32362]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T02:42:48.209 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:42:48 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:42:49.179 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout 8443 2024-06-10T02:42:49.179 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:48.460+0000 7fdf970a6700 1 Processor -- start 2024-06-10T02:42:49.179 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:48.460+0000 7fdf970a6700 1 -- start start 2024-06-10T02:42:49.180 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:48.460+0000 7fdf970a6700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdf90108a30 0x7fdf9010ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:42:49.180 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:48.460+0000 7fdf970a6700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fdf9007c6f0 con 0x7fdf90108a30 2024-06-10T02:42:49.180 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:48.461+0000 7fdf94e42700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdf90108a30 0x7fdf9010ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:42:49.180 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:48.461+0000 7fdf94e42700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdf90108a30 0x7fdf9010ae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:44308/0 (socket says 172.21.15.52:44308) 2024-06-10T02:42:49.180 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:48.461+0000 7fdf94e42700 1 -- 172.21.15.52:0/2990381192 learned_addr learned my addr 172.21.15.52:0/2990381192 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:42:49.180 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:48.461+0000 7fdf94e42700 1 -- 172.21.15.52:0/2990381192 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf9010b360 con 0x7fdf90108a30 2024-06-10T02:42:49.180 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:48.461+0000 7fdf94e42700 1 --2- 172.21.15.52:0/2990381192 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdf90108a30 0x7fdf9010ae20 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fdf8c0098a0 tx=0x7fdf8c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=629153a35bd04377 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:42:49.180 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:48.461+0000 7fdf877fe700 1 -- 172.21.15.52:0/2990381192 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdf8c004160 con 0x7fdf90108a30 2024-06-10T02:42:49.180 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:48.461+0000 7fdf877fe700 1 -- 172.21.15.52:0/2990381192 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf8c004300 con 0x7fdf90108a30 2024-06-10T02:42:49.180 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:48.461+0000 7fdf877fe700 1 -- 172.21.15.52:0/2990381192 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdf8c011b20 con 0x7fdf90108a30 2024-06-10T02:42:49.180 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:48.462+0000 7fdf970a6700 1 -- 172.21.15.52:0/2990381192 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdf90108a30 msgr2=0x7fdf9010ae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:42:49.180 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:48.462+0000 7fdf970a6700 1 --2- 172.21.15.52:0/2990381192 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdf90108a30 0x7fdf9010ae20 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fdf8c0098a0 tx=0x7fdf8c009bb0 comp rx=0 tx=0).stop 2024-06-10T02:42:49.181 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:48.462+0000 7fdf970a6700 1 -- 172.21.15.52:0/2990381192 shutdown_connections 2024-06-10T02:42:49.181 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:48.462+0000 7fdf970a6700 1 --2- 172.21.15.52:0/2990381192 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdf90108a30 0x7fdf9010ae20 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:42:49.181 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:48.462+0000 7fdf970a6700 1 -- 172.21.15.52:0/2990381192 >> 172.21.15.52:0/2990381192 conn(0x7fdf90102440 msgr2=0x7fdf90104860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:42:49.181 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:48.462+0000 7fdf970a6700 1 -- 172.21.15.52:0/2990381192 shutdown_connections 2024-06-10T02:42:49.181 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:48.462+0000 7fdf970a6700 1 -- 172.21.15.52:0/2990381192 wait complete. 2024-06-10T02:42:49.181 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:48.462+0000 7fdf970a6700 1 Processor -- start 2024-06-10T02:42:49.181 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:48.462+0000 7fdf970a6700 1 -- start start 2024-06-10T02:42:49.181 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:48.463+0000 7fdf970a6700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdf90108a30 0x7fdf901082b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:42:49.181 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:48.463+0000 7fdf970a6700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fdf9010c370 con 0x7fdf90108a30 2024-06-10T02:42:49.181 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:48.463+0000 7fdf94e42700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdf90108a30 0x7fdf901082b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:42:49.181 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:48.463+0000 7fdf94e42700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdf90108a30 0x7fdf901082b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:44320/0 (socket says 172.21.15.52:44320) 2024-06-10T02:42:49.181 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:48.463+0000 7fdf94e42700 1 -- 172.21.15.52:0/1670506632 learned_addr learned my addr 172.21.15.52:0/1670506632 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:42:49.181 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:48.463+0000 7fdf94e42700 1 -- 172.21.15.52:0/1670506632 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf901087f0 con 0x7fdf90108a30 2024-06-10T02:42:49.182 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:48.463+0000 7fdf94e42700 1 --2- 172.21.15.52:0/1670506632 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdf90108a30 0x7fdf901082b0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fdf8c0117e0 tx=0x7fdf8c011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:42:49.182 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:48.463+0000 7fdf85ffb700 1 -- 172.21.15.52:0/1670506632 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdf8c003ba0 con 0x7fdf90108a30 2024-06-10T02:42:49.182 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:48.463+0000 7fdf85ffb700 1 -- 172.21.15.52:0/1670506632 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf8c01a910 con 0x7fdf90108a30 2024-06-10T02:42:49.182 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:48.463+0000 7fdf970a6700 1 -- 172.21.15.52:0/1670506632 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf901069e0 con 0x7fdf90108a30 2024-06-10T02:42:49.182 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:48.464+0000 7fdf85ffb700 1 -- 172.21.15.52:0/1670506632 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdf8c0248f0 con 0x7fdf90108a30 2024-06-10T02:42:49.182 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:48.464+0000 7fdf970a6700 1 -- 172.21.15.52:0/1670506632 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf90106ea0 con 0x7fdf90108a30 2024-06-10T02:42:49.182 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:48.465+0000 7fdf85ffb700 1 -- 172.21.15.52:0/1670506632 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7fdf8c022020 con 0x7fdf90108a30 2024-06-10T02:42:49.182 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:48.465+0000 7fdf85ffb700 1 --2- 172.21.15.52:0/1670506632 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fdf78039100 0x7fdf7803b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:42:49.182 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:48.465+0000 7fdf85ffb700 1 -- 172.21.15.52:0/1670506632 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fdf8c04d5a0 con 0x7fdf90108a30 2024-06-10T02:42:49.182 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:48.465+0000 7fdf87fff700 1 --2- 172.21.15.52:0/1670506632 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fdf78039100 0x7fdf7803b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:42:49.182 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:48.465+0000 7fdf970a6700 1 -- 172.21.15.52:0/1670506632 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf90063af0 con 0x7fdf90108a30 2024-06-10T02:42:49.182 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:48.465+0000 7fdf87fff700 1 --2- 172.21.15.52:0/1670506632 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fdf78039100 0x7fdf7803b5c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fdf80006fd0 tx=0x7fdf80006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:42:49.182 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:48.470+0000 7fdf85ffb700 1 -- 172.21.15.52:0/1670506632 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fdf8c00b3e0 con 0x7fdf90108a30 2024-06-10T02:42:49.183 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:48.677+0000 7fdf970a6700 1 -- 172.21.15.52:0/1670506632 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7fdf90107560 con 0x7fdf90108a30 2024-06-10T02:42:49.183 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:48.677+0000 7fdf85ffb700 1 -- 172.21.15.52:0/1670506632 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7fdf8c020c10 con 0x7fdf90108a30 2024-06-10T02:42:49.183 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:48.680+0000 7fdf970a6700 1 -- 172.21.15.52:0/1670506632 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fdf78039100 msgr2=0x7fdf7803b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:42:49.183 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:48.680+0000 7fdf970a6700 1 --2- 172.21.15.52:0/1670506632 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fdf78039100 0x7fdf7803b5c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fdf80006fd0 tx=0x7fdf80006e40 comp rx=0 tx=0).stop 2024-06-10T02:42:49.183 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:48.680+0000 7fdf970a6700 1 -- 172.21.15.52:0/1670506632 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdf90108a30 msgr2=0x7fdf901082b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:42:49.183 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:48.680+0000 7fdf970a6700 1 --2- 172.21.15.52:0/1670506632 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdf90108a30 0x7fdf901082b0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fdf8c0117e0 tx=0x7fdf8c011ba0 comp rx=0 tx=0).stop 2024-06-10T02:42:49.183 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:48.680+0000 7fdf970a6700 1 -- 172.21.15.52:0/1670506632 shutdown_connections 2024-06-10T02:42:49.183 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:48.680+0000 7fdf970a6700 1 --2- 172.21.15.52:0/1670506632 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fdf78039100 0x7fdf7803b5c0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:42:49.183 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:48.680+0000 7fdf970a6700 1 --2- 172.21.15.52:0/1670506632 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdf90108a30 0x7fdf901082b0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:42:49.183 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:48.680+0000 7fdf970a6700 1 -- 172.21.15.52:0/1670506632 >> 172.21.15.52:0/1670506632 conn(0x7fdf90102440 msgr2=0x7fdf90102ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:42:49.183 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:48.680+0000 7fdf970a6700 1 -- 172.21.15.52:0/1670506632 shutdown_connections 2024-06-10T02:42:49.183 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:48.680+0000 7fdf970a6700 1 -- 172.21.15.52:0/1670506632 wait complete. 2024-06-10T02:42:49.187 INFO:teuthology.orchestra.run.smithi052.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-06-10T02:42:49.187 INFO:teuthology.orchestra.run.smithi052.stdout:systemctl: stdout disabled 2024-06-10T02:42:49.195 INFO:teuthology.orchestra.run.smithi052.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-06-10T02:42:49.195 INFO:teuthology.orchestra.run.smithi052.stdout:systemctl: stdout inactive 2024-06-10T02:42:49.195 INFO:teuthology.orchestra.run.smithi052.stdout:firewalld.service is not enabled 2024-06-10T02:42:49.195 INFO:teuthology.orchestra.run.smithi052.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-06-10T02:42:49.197 INFO:teuthology.orchestra.run.smithi052.stdout:Ceph Dashboard is now available at: 2024-06-10T02:42:49.197 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-10T02:42:49.197 INFO:teuthology.orchestra.run.smithi052.stdout: URL: https://smithi052.front.sepia.ceph.com:8443/ 2024-06-10T02:42:49.197 INFO:teuthology.orchestra.run.smithi052.stdout: User: admin 2024-06-10T02:42:49.197 INFO:teuthology.orchestra.run.smithi052.stdout: Password: bqvhwbi8x9 2024-06-10T02:42:49.197 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-10T02:42:49.197 INFO:teuthology.orchestra.run.smithi052.stdout:Saving cluster configuration to /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/config directory 2024-06-10T02:42:49.198 INFO:teuthology.orchestra.run.smithi052.stdout:Enabling autotune for osd_memory_target 2024-06-10T02:42:49.321 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:42:49 smithi052 ceph-mon[32362]: from='client.? 172.21.15.52:0/1670506632' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-06-10T02:42:50.136 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:42:50 smithi052 ceph-mon[32362]: mgrmap e13: x(active, since 6s) 2024-06-10T02:42:50.404 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:49.623+0000 7fa062ab6700 1 Processor -- start 2024-06-10T02:42:50.404 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:49.623+0000 7fa062ab6700 1 -- start start 2024-06-10T02:42:50.404 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:49.623+0000 7fa062ab6700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa05c108a10 0x7fa05c10ae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:42:50.405 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:49.623+0000 7fa062ab6700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fa05c07c710 con 0x7fa05c108a10 2024-06-10T02:42:50.405 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:49.623+0000 7fa060852700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa05c108a10 0x7fa05c10ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:42:50.405 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:49.623+0000 7fa060852700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa05c108a10 0x7fa05c10ae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:44330/0 (socket says 172.21.15.52:44330) 2024-06-10T02:42:50.405 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:49.623+0000 7fa060852700 1 -- 172.21.15.52:0/579642980 learned_addr learned my addr 172.21.15.52:0/579642980 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:42:50.405 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:49.624+0000 7fa060852700 1 -- 172.21.15.52:0/579642980 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa05c10b340 con 0x7fa05c108a10 2024-06-10T02:42:50.405 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:49.624+0000 7fa060852700 1 --2- 172.21.15.52:0/579642980 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa05c108a10 0x7fa05c10ae00 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa0500098a0 tx=0x7fa050009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a37e00eabcecde2 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:42:50.405 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:49.624+0000 7fa05b7fe700 1 -- 172.21.15.52:0/579642980 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa050004160 con 0x7fa05c108a10 2024-06-10T02:42:50.405 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:49.624+0000 7fa05b7fe700 1 -- 172.21.15.52:0/579642980 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa050004300 con 0x7fa05c108a10 2024-06-10T02:42:50.405 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:49.624+0000 7fa05b7fe700 1 -- 172.21.15.52:0/579642980 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa050011b20 con 0x7fa05c108a10 2024-06-10T02:42:50.405 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:49.624+0000 7fa062ab6700 1 -- 172.21.15.52:0/579642980 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa05c108a10 msgr2=0x7fa05c10ae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:42:50.405 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:49.624+0000 7fa062ab6700 1 --2- 172.21.15.52:0/579642980 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa05c108a10 0x7fa05c10ae00 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa0500098a0 tx=0x7fa050009bb0 comp rx=0 tx=0).stop 2024-06-10T02:42:50.405 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:49.625+0000 7fa062ab6700 1 -- 172.21.15.52:0/579642980 shutdown_connections 2024-06-10T02:42:50.406 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:49.625+0000 7fa062ab6700 1 --2- 172.21.15.52:0/579642980 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa05c108a10 0x7fa05c10ae00 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:42:50.406 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:49.625+0000 7fa062ab6700 1 -- 172.21.15.52:0/579642980 >> 172.21.15.52:0/579642980 conn(0x7fa05c102440 msgr2=0x7fa05c104860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:42:50.406 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:49.625+0000 7fa062ab6700 1 -- 172.21.15.52:0/579642980 shutdown_connections 2024-06-10T02:42:50.406 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:49.625+0000 7fa062ab6700 1 -- 172.21.15.52:0/579642980 wait complete. 2024-06-10T02:42:50.406 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:49.625+0000 7fa062ab6700 1 Processor -- start 2024-06-10T02:42:50.406 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:49.625+0000 7fa062ab6700 1 -- start start 2024-06-10T02:42:50.406 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:49.626+0000 7fa062ab6700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa05c108a10 0x7fa05c19be30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:42:50.406 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:49.626+0000 7fa062ab6700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fa05c10c3a0 con 0x7fa05c108a10 2024-06-10T02:42:50.406 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:49.626+0000 7fa060852700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa05c108a10 0x7fa05c19be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:42:50.406 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:49.626+0000 7fa060852700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa05c108a10 0x7fa05c19be30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:44334/0 (socket says 172.21.15.52:44334) 2024-06-10T02:42:50.406 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:49.626+0000 7fa060852700 1 -- 172.21.15.52:0/2469130655 learned_addr learned my addr 172.21.15.52:0/2469130655 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:42:50.406 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:49.626+0000 7fa060852700 1 -- 172.21.15.52:0/2469130655 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa05c19c370 con 0x7fa05c108a10 2024-06-10T02:42:50.407 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:49.626+0000 7fa060852700 1 --2- 172.21.15.52:0/2469130655 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa05c108a10 0x7fa05c19be30 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fa0500117e0 tx=0x7fa050011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:42:50.407 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:49.626+0000 7fa059ffb700 1 -- 172.21.15.52:0/2469130655 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa050003ba0 con 0x7fa05c108a10 2024-06-10T02:42:50.407 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:49.626+0000 7fa059ffb700 1 -- 172.21.15.52:0/2469130655 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa05001a910 con 0x7fa05c108a10 2024-06-10T02:42:50.407 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:49.626+0000 7fa062ab6700 1 -- 172.21.15.52:0/2469130655 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa05c19c600 con 0x7fa05c108a10 2024-06-10T02:42:50.407 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:49.626+0000 7fa059ffb700 1 -- 172.21.15.52:0/2469130655 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa0500248f0 con 0x7fa05c108a10 2024-06-10T02:42:50.407 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:49.627+0000 7fa062ab6700 1 -- 172.21.15.52:0/2469130655 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa05c19ca60 con 0x7fa05c108a10 2024-06-10T02:42:50.407 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:49.627+0000 7fa059ffb700 1 -- 172.21.15.52:0/2469130655 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fa050022020 con 0x7fa05c108a10 2024-06-10T02:42:50.407 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:49.628+0000 7fa059ffb700 1 --2- 172.21.15.52:0/2469130655 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fa04c039140 0x7fa04c03b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:42:50.407 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:49.628+0000 7fa059ffb700 1 -- 172.21.15.52:0/2469130655 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fa05004dad0 con 0x7fa05c108a10 2024-06-10T02:42:50.407 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:49.628+0000 7fa062ab6700 1 -- 172.21.15.52:0/2469130655 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa05c063af0 con 0x7fa05c108a10 2024-06-10T02:42:50.407 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:49.628+0000 7fa05bfff700 1 --2- 172.21.15.52:0/2469130655 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fa04c039140 0x7fa04c03b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:42:50.407 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:49.628+0000 7fa05bfff700 1 --2- 172.21.15.52:0/2469130655 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fa04c039140 0x7fa04c03b600 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fa048006fd0 tx=0x7fa048006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:42:50.407 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:49.633+0000 7fa059ffb700 1 -- 172.21.15.52:0/2469130655 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fa05004de20 con 0x7fa05c108a10 2024-06-10T02:42:50.408 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:49.840+0000 7fa062ab6700 1 -- 172.21.15.52:0/2469130655 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) -- 0x7fa05c19f4d0 con 0x7fa05c108a10 2024-06-10T02:42:50.408 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:49.840+0000 7fa059ffb700 1 -- 172.21.15.52:0/2469130655 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) ==== 127+0+0 (secure 0 0 0) 0x7fa050020c10 con 0x7fa05c108a10 2024-06-10T02:42:50.408 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:49.843+0000 7fa062ab6700 1 -- 172.21.15.52:0/2469130655 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fa04c039140 msgr2=0x7fa04c03b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:42:50.408 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:49.843+0000 7fa062ab6700 1 --2- 172.21.15.52:0/2469130655 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fa04c039140 0x7fa04c03b600 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fa048006fd0 tx=0x7fa048006e40 comp rx=0 tx=0).stop 2024-06-10T02:42:50.408 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:49.843+0000 7fa062ab6700 1 -- 172.21.15.52:0/2469130655 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa05c108a10 msgr2=0x7fa05c19be30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:42:50.408 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:49.843+0000 7fa062ab6700 1 --2- 172.21.15.52:0/2469130655 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa05c108a10 0x7fa05c19be30 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fa0500117e0 tx=0x7fa050011ba0 comp rx=0 tx=0).stop 2024-06-10T02:42:50.408 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:49.843+0000 7fa062ab6700 1 -- 172.21.15.52:0/2469130655 shutdown_connections 2024-06-10T02:42:50.408 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:49.843+0000 7fa062ab6700 1 --2- 172.21.15.52:0/2469130655 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fa04c039140 0x7fa04c03b600 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:42:50.408 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:49.843+0000 7fa062ab6700 1 --2- 172.21.15.52:0/2469130655 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa05c108a10 0x7fa05c19be30 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:42:50.408 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:49.843+0000 7fa062ab6700 1 -- 172.21.15.52:0/2469130655 >> 172.21.15.52:0/2469130655 conn(0x7fa05c102440 msgr2=0x7fa05c102e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:42:50.408 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:49.843+0000 7fa062ab6700 1 -- 172.21.15.52:0/2469130655 shutdown_connections 2024-06-10T02:42:50.408 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:49.843+0000 7fa062ab6700 1 -- 172.21.15.52:0/2469130655 wait complete. 2024-06-10T02:42:51.726 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:50.848+0000 7f21d286b700 1 Processor -- start 2024-06-10T02:42:51.726 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:50.848+0000 7f21d286b700 1 -- start start 2024-06-10T02:42:51.726 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:50.848+0000 7f21d286b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f21cc07e710 0x7f21cc07cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:42:51.727 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:50.848+0000 7f21d286b700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f21cc07d190 con 0x7f21cc07e710 2024-06-10T02:42:51.727 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:50.849+0000 7f21cbfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f21cc07e710 0x7f21cc07cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:42:51.727 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:50.849+0000 7f21cbfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f21cc07e710 0x7f21cc07cbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:46208/0 (socket says 172.21.15.52:46208) 2024-06-10T02:42:51.727 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:50.849+0000 7f21cbfff700 1 -- 172.21.15.52:0/3731552131 learned_addr learned my addr 172.21.15.52:0/3731552131 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:42:51.727 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:50.849+0000 7f21cbfff700 1 -- 172.21.15.52:0/3731552131 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21cc07d310 con 0x7f21cc07e710 2024-06-10T02:42:51.727 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:50.849+0000 7f21cbfff700 1 --2- 172.21.15.52:0/3731552131 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f21cc07e710 0x7f21cc07cbc0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f21bc0098a0 tx=0x7f21bc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dd227bd691637fd server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:42:51.727 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:50.849+0000 7f21caffd700 1 -- 172.21.15.52:0/3731552131 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f21bc004160 con 0x7f21cc07e710 2024-06-10T02:42:51.727 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:50.849+0000 7f21caffd700 1 -- 172.21.15.52:0/3731552131 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f21bc004300 con 0x7f21cc07e710 2024-06-10T02:42:51.727 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:50.849+0000 7f21caffd700 1 -- 172.21.15.52:0/3731552131 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f21bc011b20 con 0x7f21cc07e710 2024-06-10T02:42:51.727 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:50.850+0000 7f21d286b700 1 -- 172.21.15.52:0/3731552131 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f21cc07e710 msgr2=0x7f21cc07cbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:42:51.727 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:50.850+0000 7f21d286b700 1 --2- 172.21.15.52:0/3731552131 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f21cc07e710 0x7f21cc07cbc0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f21bc0098a0 tx=0x7f21bc009bb0 comp rx=0 tx=0).stop 2024-06-10T02:42:51.727 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:50.850+0000 7f21d286b700 1 -- 172.21.15.52:0/3731552131 shutdown_connections 2024-06-10T02:42:51.727 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:50.850+0000 7f21d286b700 1 --2- 172.21.15.52:0/3731552131 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f21cc07e710 0x7f21cc07cbc0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:42:51.728 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:50.850+0000 7f21d286b700 1 -- 172.21.15.52:0/3731552131 >> 172.21.15.52:0/3731552131 conn(0x7f21cc103560 msgr2=0x7f21cc1059a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:42:51.728 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:50.850+0000 7f21d286b700 1 -- 172.21.15.52:0/3731552131 shutdown_connections 2024-06-10T02:42:51.728 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:50.850+0000 7f21d286b700 1 -- 172.21.15.52:0/3731552131 wait complete. 2024-06-10T02:42:51.728 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:50.851+0000 7f21d286b700 1 Processor -- start 2024-06-10T02:42:51.728 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:50.851+0000 7f21d286b700 1 -- start start 2024-06-10T02:42:51.728 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:50.851+0000 7f21d286b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f21cc07e710 0x7f21cc19e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:42:51.728 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:50.851+0000 7f21d286b700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f21cc10e5a0 con 0x7f21cc07e710 2024-06-10T02:42:51.728 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:50.851+0000 7f21cbfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f21cc07e710 0x7f21cc19e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:42:51.728 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:50.851+0000 7f21cbfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f21cc07e710 0x7f21cc19e070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:46222/0 (socket says 172.21.15.52:46222) 2024-06-10T02:42:51.728 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:50.851+0000 7f21cbfff700 1 -- 172.21.15.52:0/3694975812 learned_addr learned my addr 172.21.15.52:0/3694975812 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:42:51.728 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:50.851+0000 7f21cbfff700 1 -- 172.21.15.52:0/3694975812 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21cc19e5b0 con 0x7f21cc07e710 2024-06-10T02:42:51.728 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:50.852+0000 7f21cbfff700 1 --2- 172.21.15.52:0/3694975812 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f21cc07e710 0x7f21cc19e070 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f21bc0117e0 tx=0x7f21bc011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:42:51.729 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:50.852+0000 7f21c97fa700 1 -- 172.21.15.52:0/3694975812 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f21bc003bf0 con 0x7f21cc07e710 2024-06-10T02:42:51.729 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:50.852+0000 7f21c97fa700 1 -- 172.21.15.52:0/3694975812 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f21bc01a910 con 0x7f21cc07e710 2024-06-10T02:42:51.729 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:50.852+0000 7f21d286b700 1 -- 172.21.15.52:0/3694975812 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21cc19e840 con 0x7f21cc07e710 2024-06-10T02:42:51.729 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:50.852+0000 7f21c97fa700 1 -- 172.21.15.52:0/3694975812 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f21bc024970 con 0x7f21cc07e710 2024-06-10T02:42:51.729 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:50.852+0000 7f21d286b700 1 -- 172.21.15.52:0/3694975812 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21cc19eca0 con 0x7f21cc07e710 2024-06-10T02:42:51.729 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:50.853+0000 7f21c97fa700 1 -- 172.21.15.52:0/3694975812 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f21bc022020 con 0x7f21cc07e710 2024-06-10T02:42:51.729 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:50.853+0000 7f21d286b700 1 -- 172.21.15.52:0/3694975812 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21cc063af0 con 0x7f21cc07e710 2024-06-10T02:42:51.729 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:50.853+0000 7f21c97fa700 1 --2- 172.21.15.52:0/3694975812 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f21b4039150 0x7f21b403b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:42:51.729 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:50.853+0000 7f21c97fa700 1 -- 172.21.15.52:0/3694975812 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f21bc04d550 con 0x7f21cc07e710 2024-06-10T02:42:51.729 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:50.853+0000 7f21cb7fe700 1 --2- 172.21.15.52:0/3694975812 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f21b4039150 0x7f21b403b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:42:51.729 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:50.853+0000 7f21cb7fe700 1 --2- 172.21.15.52:0/3694975812 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f21b4039150 0x7f21b403b610 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f21c0006fd0 tx=0x7f21c0006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:42:51.729 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:50.858+0000 7f21c97fa700 1 -- 172.21.15.52:0/3694975812 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f21bc011e20 con 0x7f21cc07e710 2024-06-10T02:42:51.729 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:51.183+0000 7f21d286b700 1 -- 172.21.15.52:0/3694975812 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7f21cc1a1670 con 0x7f21cc07e710 2024-06-10T02:42:51.730 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:51.187+0000 7f21c97fa700 1 -- 172.21.15.52:0/3694975812 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) ==== 153+0+0 (secure 0 0 0) 0x7f21bc0172b0 con 0x7f21cc07e710 2024-06-10T02:42:51.730 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:51.189+0000 7f21d286b700 1 -- 172.21.15.52:0/3694975812 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f21b4039150 msgr2=0x7f21b403b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:42:51.730 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:51.189+0000 7f21d286b700 1 --2- 172.21.15.52:0/3694975812 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f21b4039150 0x7f21b403b610 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f21c0006fd0 tx=0x7f21c0006e40 comp rx=0 tx=0).stop 2024-06-10T02:42:51.730 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:51.189+0000 7f21d286b700 1 -- 172.21.15.52:0/3694975812 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f21cc07e710 msgr2=0x7f21cc19e070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:42:51.730 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:51.189+0000 7f21d286b700 1 --2- 172.21.15.52:0/3694975812 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f21cc07e710 0x7f21cc19e070 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f21bc0117e0 tx=0x7f21bc011c10 comp rx=0 tx=0).stop 2024-06-10T02:42:51.730 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:51.189+0000 7f21d286b700 1 -- 172.21.15.52:0/3694975812 shutdown_connections 2024-06-10T02:42:51.730 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:51.189+0000 7f21d286b700 1 --2- 172.21.15.52:0/3694975812 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f21b4039150 0x7f21b403b610 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:42:51.730 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:51.189+0000 7f21d286b700 1 --2- 172.21.15.52:0/3694975812 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f21cc07e710 0x7f21cc19e070 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:42:51.730 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:51.189+0000 7f21d286b700 1 -- 172.21.15.52:0/3694975812 >> 172.21.15.52:0/3694975812 conn(0x7f21cc103560 msgr2=0x7f21cc103fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:42:51.730 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:51.189+0000 7f21d286b700 1 -- 172.21.15.52:0/3694975812 shutdown_connections 2024-06-10T02:42:51.730 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-10T02:42:51.189+0000 7f21d286b700 1 -- 172.21.15.52:0/3694975812 wait complete. 2024-06-10T02:42:51.730 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-06-10T02:42:51.730 INFO:teuthology.orchestra.run.smithi052.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-06-10T02:42:51.731 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-10T02:42:51.731 INFO:teuthology.orchestra.run.smithi052.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-06-10T02:42:51.731 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-10T02:42:51.731 INFO:teuthology.orchestra.run.smithi052.stdout:Or, if you are only running a single cluster on this host: 2024-06-10T02:42:51.731 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-10T02:42:51.731 INFO:teuthology.orchestra.run.smithi052.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-06-10T02:42:51.731 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-10T02:42:51.731 INFO:teuthology.orchestra.run.smithi052.stdout:Please consider enabling telemetry to help improve Ceph: 2024-06-10T02:42:51.731 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-10T02:42:51.731 INFO:teuthology.orchestra.run.smithi052.stdout: ceph telemetry on 2024-06-10T02:42:51.731 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-10T02:42:51.731 INFO:teuthology.orchestra.run.smithi052.stdout:For more information see: 2024-06-10T02:42:51.731 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-10T02:42:51.731 INFO:teuthology.orchestra.run.smithi052.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-06-10T02:42:51.732 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-10T02:42:51.732 INFO:teuthology.orchestra.run.smithi052.stdout:Bootstrap complete. 2024-06-10T02:42:51.771 INFO:tasks.cephadm:Fetching config... 2024-06-10T02:42:51.771 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-06-10T02:42:51.771 DEBUG:teuthology.orchestra.run.smithi052:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-06-10T02:42:51.790 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-06-10T02:42:51.790 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-06-10T02:42:51.790 DEBUG:teuthology.orchestra.run.smithi052:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-06-10T02:42:51.846 INFO:tasks.cephadm:Fetching mon keyring... 2024-06-10T02:42:51.847 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-06-10T02:42:51.847 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd if=/var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/keyring of=/dev/stdout 2024-06-10T02:42:51.914 INFO:tasks.cephadm:Fetching pub ssh key... 2024-06-10T02:42:51.914 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-06-10T02:42:51.914 DEBUG:teuthology.orchestra.run.smithi052:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-06-10T02:42:51.970 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-06-10T02:42:51.970 DEBUG:teuthology.orchestra.run.smithi052:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCplfU1Jbyk6R5JXXM2hZMcXCEloEITPToJJkgS2OMIo3MFaHRT8Xt9FWIMeifAkphay+L1SKm2aJsDP+AIwNPZBFNOpbO4mEKyP5ABu+UN+ZbpmNmRvcCxpdM0WaY3pruFcDHEfcV3V7le1rRyK9AU1xD8HoXJNBRmZlN6kt43nyRI/B/Lh5k+gepOOdnstqJ353C5wgVZ0Om1Pyqz6q+tRS/w8ROfVeM59EPlXVM0koP0rzeRAnzSvk9CK0XbruSL0jZYW+BtEgRHSG2jrGDJHkWwV7hSVTpbJEnUmf5XrNaKuneOHjxhaJGHUmNn9xc4q1rUsZVBJzo8p5D+Ug+wRgi7LRGWJK7OTYLNd7G+OtrZoMFw3K1JjpQtVBLkMqVyzNGhu0pz/y0gcd9m1bjOwVUogX8Lp0bcfczOSyJ59hzSYXgFfakzNfdRBVpvxmex7EH8VczAHdjG+S4oSu/vmWa9wy65hPRemHNifWtwA/kv0i53fvlHJxledeMSRe8= ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-06-10T02:42:52.051 INFO:teuthology.orchestra.run.smithi052.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCplfU1Jbyk6R5JXXM2hZMcXCEloEITPToJJkgS2OMIo3MFaHRT8Xt9FWIMeifAkphay+L1SKm2aJsDP+AIwNPZBFNOpbO4mEKyP5ABu+UN+ZbpmNmRvcCxpdM0WaY3pruFcDHEfcV3V7le1rRyK9AU1xD8HoXJNBRmZlN6kt43nyRI/B/Lh5k+gepOOdnstqJ353C5wgVZ0Om1Pyqz6q+tRS/w8ROfVeM59EPlXVM0koP0rzeRAnzSvk9CK0XbruSL0jZYW+BtEgRHSG2jrGDJHkWwV7hSVTpbJEnUmf5XrNaKuneOHjxhaJGHUmNn9xc4q1rUsZVBJzo8p5D+Ug+wRgi7LRGWJK7OTYLNd7G+OtrZoMFw3K1JjpQtVBLkMqVyzNGhu0pz/y0gcd9m1bjOwVUogX8Lp0bcfczOSyJ59hzSYXgFfakzNfdRBVpvxmex7EH8VczAHdjG+S4oSu/vmWa9wy65hPRemHNifWtwA/kv0i53fvlHJxledeMSRe8= ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968 2024-06-10T02:42:52.064 DEBUG:teuthology.orchestra.run.smithi120:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCplfU1Jbyk6R5JXXM2hZMcXCEloEITPToJJkgS2OMIo3MFaHRT8Xt9FWIMeifAkphay+L1SKm2aJsDP+AIwNPZBFNOpbO4mEKyP5ABu+UN+ZbpmNmRvcCxpdM0WaY3pruFcDHEfcV3V7le1rRyK9AU1xD8HoXJNBRmZlN6kt43nyRI/B/Lh5k+gepOOdnstqJ353C5wgVZ0Om1Pyqz6q+tRS/w8ROfVeM59EPlXVM0koP0rzeRAnzSvk9CK0XbruSL0jZYW+BtEgRHSG2jrGDJHkWwV7hSVTpbJEnUmf5XrNaKuneOHjxhaJGHUmNn9xc4q1rUsZVBJzo8p5D+Ug+wRgi7LRGWJK7OTYLNd7G+OtrZoMFw3K1JjpQtVBLkMqVyzNGhu0pz/y0gcd9m1bjOwVUogX8Lp0bcfczOSyJ59hzSYXgFfakzNfdRBVpvxmex7EH8VczAHdjG+S4oSu/vmWa9wy65hPRemHNifWtwA/kv0i53fvlHJxledeMSRe8= ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-06-10T02:42:52.102 INFO:teuthology.orchestra.run.smithi120.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCplfU1Jbyk6R5JXXM2hZMcXCEloEITPToJJkgS2OMIo3MFaHRT8Xt9FWIMeifAkphay+L1SKm2aJsDP+AIwNPZBFNOpbO4mEKyP5ABu+UN+ZbpmNmRvcCxpdM0WaY3pruFcDHEfcV3V7le1rRyK9AU1xD8HoXJNBRmZlN6kt43nyRI/B/Lh5k+gepOOdnstqJ353C5wgVZ0Om1Pyqz6q+tRS/w8ROfVeM59EPlXVM0koP0rzeRAnzSvk9CK0XbruSL0jZYW+BtEgRHSG2jrGDJHkWwV7hSVTpbJEnUmf5XrNaKuneOHjxhaJGHUmNn9xc4q1rUsZVBJzo8p5D+Ug+wRgi7LRGWJK7OTYLNd7G+OtrZoMFw3K1JjpQtVBLkMqVyzNGhu0pz/y0gcd9m1bjOwVUogX8Lp0bcfczOSyJ59hzSYXgFfakzNfdRBVpvxmex7EH8VczAHdjG+S4oSu/vmWa9wy65hPRemHNifWtwA/kv0i53fvlHJxledeMSRe8= ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968 2024-06-10T02:42:52.114 DEBUG:teuthology.orchestra.run.smithi188:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCplfU1Jbyk6R5JXXM2hZMcXCEloEITPToJJkgS2OMIo3MFaHRT8Xt9FWIMeifAkphay+L1SKm2aJsDP+AIwNPZBFNOpbO4mEKyP5ABu+UN+ZbpmNmRvcCxpdM0WaY3pruFcDHEfcV3V7le1rRyK9AU1xD8HoXJNBRmZlN6kt43nyRI/B/Lh5k+gepOOdnstqJ353C5wgVZ0Om1Pyqz6q+tRS/w8ROfVeM59EPlXVM0koP0rzeRAnzSvk9CK0XbruSL0jZYW+BtEgRHSG2jrGDJHkWwV7hSVTpbJEnUmf5XrNaKuneOHjxhaJGHUmNn9xc4q1rUsZVBJzo8p5D+Ug+wRgi7LRGWJK7OTYLNd7G+OtrZoMFw3K1JjpQtVBLkMqVyzNGhu0pz/y0gcd9m1bjOwVUogX8Lp0bcfczOSyJ59hzSYXgFfakzNfdRBVpvxmex7EH8VczAHdjG+S4oSu/vmWa9wy65hPRemHNifWtwA/kv0i53fvlHJxledeMSRe8= ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-06-10T02:42:52.154 INFO:teuthology.orchestra.run.smithi188.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCplfU1Jbyk6R5JXXM2hZMcXCEloEITPToJJkgS2OMIo3MFaHRT8Xt9FWIMeifAkphay+L1SKm2aJsDP+AIwNPZBFNOpbO4mEKyP5ABu+UN+ZbpmNmRvcCxpdM0WaY3pruFcDHEfcV3V7le1rRyK9AU1xD8HoXJNBRmZlN6kt43nyRI/B/Lh5k+gepOOdnstqJ353C5wgVZ0Om1Pyqz6q+tRS/w8ROfVeM59EPlXVM0koP0rzeRAnzSvk9CK0XbruSL0jZYW+BtEgRHSG2jrGDJHkWwV7hSVTpbJEnUmf5XrNaKuneOHjxhaJGHUmNn9xc4q1rUsZVBJzo8p5D+Ug+wRgi7LRGWJK7OTYLNd7G+OtrZoMFw3K1JjpQtVBLkMqVyzNGhu0pz/y0gcd9m1bjOwVUogX8Lp0bcfczOSyJ59hzSYXgFfakzNfdRBVpvxmex7EH8VczAHdjG+S4oSu/vmWa9wy65hPRemHNifWtwA/kv0i53fvlHJxledeMSRe8= ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968 2024-06-10T02:42:52.165 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-06-10T02:42:52.416 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:42:52.473 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:42:52 smithi052 ceph-mon[32362]: from='client.? 172.21.15.52:0/3694975812' entity='client.admin' 2024-06-10T02:42:54.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:42:54.868+0000 7f5751e40700 1 -- 172.21.15.52:0/810228133 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f574c100df0 msgr2=0x7f574c101210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:42:54.866 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:42:54.868+0000 7f5751e40700 1 --2- 172.21.15.52:0/810228133 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f574c100df0 0x7f574c101210 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f573c009910 tx=0x7f573c009c20 comp rx=0 tx=0).stop 2024-06-10T02:42:54.866 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:42:54.868+0000 7f5751e40700 1 -- 172.21.15.52:0/810228133 shutdown_connections 2024-06-10T02:42:54.866 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:42:54.868+0000 7f5751e40700 1 --2- 172.21.15.52:0/810228133 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f574c100df0 0x7f574c101210 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:42:54.866 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:42:54.868+0000 7f5751e40700 1 -- 172.21.15.52:0/810228133 >> 172.21.15.52:0/810228133 conn(0x7f574c0fc390 msgr2=0x7f574c0fe7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:42:54.866 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:42:54.868+0000 7f5751e40700 1 -- 172.21.15.52:0/810228133 shutdown_connections 2024-06-10T02:42:54.866 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:42:54.868+0000 7f5751e40700 1 -- 172.21.15.52:0/810228133 wait complete. 2024-06-10T02:42:54.866 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:42:54 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:42:54.866 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:42:54 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:42:54.866 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:42:54 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi052", "name": "osd_memory_target"}]: dispatch 2024-06-10T02:42:54.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:42:54 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:42:54.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:42:54 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T02:42:54.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:42:54 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:42:54.868 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:42:54.869+0000 7f5751e40700 1 Processor -- start 2024-06-10T02:42:54.868 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:42:54.870+0000 7f5751e40700 1 -- start start 2024-06-10T02:42:54.868 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:42:54.870+0000 7f5751e40700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f574c100df0 0x7f574c0fe250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:42:54.868 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:42:54.870+0000 7f5751e40700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f574c106e80 con 0x7f574c100df0 2024-06-10T02:42:54.868 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:42:54.870+0000 7f574b7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f574c100df0 0x7f574c0fe250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:42:54.868 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:42:54.870+0000 7f574b7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f574c100df0 0x7f574c0fe250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:46226/0 (socket says 172.21.15.52:46226) 2024-06-10T02:42:54.869 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:42:54.870+0000 7f574b7fe700 1 -- 172.21.15.52:0/2095374077 learned_addr learned my addr 172.21.15.52:0/2095374077 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:42:54.869 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:42:54.870+0000 7f574b7fe700 1 -- 172.21.15.52:0/2095374077 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f574c0fe790 con 0x7f574c100df0 2024-06-10T02:42:54.869 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:42:54.871+0000 7f574b7fe700 1 --2- 172.21.15.52:0/2095374077 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f574c100df0 0x7f574c0fe250 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f573c005850 tx=0x7f573c005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:42:54.869 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:42:54.871+0000 7f5748ff9700 1 -- 172.21.15.52:0/2095374077 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f573c026070 con 0x7f574c100df0 2024-06-10T02:42:54.869 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:42:54.871+0000 7f5748ff9700 1 -- 172.21.15.52:0/2095374077 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f573c0044a0 con 0x7f574c100df0 2024-06-10T02:42:54.869 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:42:54.871+0000 7f5751e40700 1 -- 172.21.15.52:0/2095374077 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f574c0fc980 con 0x7f574c100df0 2024-06-10T02:42:54.869 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:42:54.871+0000 7f5748ff9700 1 -- 172.21.15.52:0/2095374077 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f573c003ab0 con 0x7f574c100df0 2024-06-10T02:42:54.869 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:42:54.871+0000 7f5751e40700 1 -- 172.21.15.52:0/2095374077 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f574c0fcec0 con 0x7f574c100df0 2024-06-10T02:42:54.869 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:42:54.872+0000 7f5751e40700 1 -- 172.21.15.52:0/2095374077 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f574c107600 con 0x7f574c100df0 2024-06-10T02:42:54.869 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:42:54.872+0000 7f5748ff9700 1 -- 172.21.15.52:0/2095374077 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f573c029070 con 0x7f574c100df0 2024-06-10T02:42:54.870 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:42:54.873+0000 7f5748ff9700 1 --2- 172.21.15.52:0/2095374077 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f5738039150 0x7f573803b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:42:54.870 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:42:54.873+0000 7f5748ff9700 1 -- 172.21.15.52:0/2095374077 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f573c01e070 con 0x7f574c100df0 2024-06-10T02:42:54.870 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:42:54.873+0000 7f574affd700 1 --2- 172.21.15.52:0/2095374077 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f5738039150 0x7f573803b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:42:54.870 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:42:54.873+0000 7f574affd700 1 --2- 172.21.15.52:0/2095374077 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f5738039150 0x7f573803b610 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f5734006fd0 tx=0x7f5734006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:42:54.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:42:54.882+0000 7f5748ff9700 1 -- 172.21.15.52:0/2095374077 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f573c013070 con 0x7f574c100df0 2024-06-10T02:42:55.087 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:42:55.091+0000 7f5751e40700 1 -- 172.21.15.52:0/2095374077 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7f574c190b80 con 0x7f574c100df0 2024-06-10T02:42:55.090 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:42:55.094+0000 7f5748ff9700 1 -- 172.21.15.52:0/2095374077 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7f573c030070 con 0x7f574c100df0 2024-06-10T02:42:55.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:42:55.097+0000 7f5751e40700 1 -- 172.21.15.52:0/2095374077 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f5738039150 msgr2=0x7f573803b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:42:55.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:42:55.097+0000 7f5751e40700 1 --2- 172.21.15.52:0/2095374077 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f5738039150 0x7f573803b610 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f5734006fd0 tx=0x7f5734006e40 comp rx=0 tx=0).stop 2024-06-10T02:42:55.094 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:42:55.097+0000 7f5751e40700 1 -- 172.21.15.52:0/2095374077 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f574c100df0 msgr2=0x7f574c0fe250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:42:55.094 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:42:55.097+0000 7f5751e40700 1 --2- 172.21.15.52:0/2095374077 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f574c100df0 0x7f574c0fe250 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f573c005850 tx=0x7f573c005030 comp rx=0 tx=0).stop 2024-06-10T02:42:55.094 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:42:55.097+0000 7f5751e40700 1 -- 172.21.15.52:0/2095374077 shutdown_connections 2024-06-10T02:42:55.094 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:42:55.097+0000 7f5751e40700 1 --2- 172.21.15.52:0/2095374077 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f5738039150 0x7f573803b610 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:42:55.094 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:42:55.097+0000 7f5751e40700 1 --2- 172.21.15.52:0/2095374077 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f574c100df0 0x7f574c0fe250 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:42:55.094 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:42:55.097+0000 7f5751e40700 1 -- 172.21.15.52:0/2095374077 >> 172.21.15.52:0/2095374077 conn(0x7f574c0fc390 msgr2=0x7f574c069cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:42:55.094 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:42:55.097+0000 7f5751e40700 1 -- 172.21.15.52:0/2095374077 shutdown_connections 2024-06-10T02:42:55.094 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:42:55.097+0000 7f5751e40700 1 -- 172.21.15.52:0/2095374077 wait complete. 2024-06-10T02:42:55.671 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-06-10T02:42:55.671 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-06-10T02:42:56.261 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:42:56.293 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:42:56 smithi052 ceph-mon[32362]: from='client.? 172.21.15.52:0/2095374077' entity='client.admin' 2024-06-10T02:42:56.293 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:42:56 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T02:42:56.293 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:42:56 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:42:57.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:42:57.674+0000 7fa679937700 1 -- 172.21.15.52:0/3199334933 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa674103510 msgr2=0x7fa674103930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:42:57.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:42:57.674+0000 7fa679937700 1 --2- 172.21.15.52:0/3199334933 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa674103510 0x7fa674103930 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fa664009910 tx=0x7fa664009c20 comp rx=0 tx=0).stop 2024-06-10T02:42:57.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:42:57.674+0000 7fa679937700 1 -- 172.21.15.52:0/3199334933 shutdown_connections 2024-06-10T02:42:57.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:42:57.674+0000 7fa679937700 1 --2- 172.21.15.52:0/3199334933 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa674103510 0x7fa674103930 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:42:57.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:42:57.674+0000 7fa679937700 1 -- 172.21.15.52:0/3199334933 >> 172.21.15.52:0/3199334933 conn(0x7fa6740fd4c0 msgr2=0x7fa6740ff8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:42:57.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:42:57.674+0000 7fa679937700 1 -- 172.21.15.52:0/3199334933 shutdown_connections 2024-06-10T02:42:57.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:42:57.674+0000 7fa679937700 1 -- 172.21.15.52:0/3199334933 wait complete. 2024-06-10T02:42:57.673 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:42:57.675+0000 7fa679937700 1 Processor -- start 2024-06-10T02:42:57.673 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:42:57.675+0000 7fa679937700 1 -- start start 2024-06-10T02:42:57.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:42:57.675+0000 7fa679937700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa674103510 0x7fa6741976e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:42:57.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:42:57.675+0000 7fa679937700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fa674107fd0 con 0x7fa674103510 2024-06-10T02:42:57.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:42:57.676+0000 7fa672ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa674103510 0x7fa6741976e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:42:57.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:42:57.676+0000 7fa672ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa674103510 0x7fa6741976e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:46240/0 (socket says 172.21.15.52:46240) 2024-06-10T02:42:57.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:42:57.676+0000 7fa672ffd700 1 -- 172.21.15.52:0/3371662702 learned_addr learned my addr 172.21.15.52:0/3371662702 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:42:57.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:42:57.676+0000 7fa672ffd700 1 -- 172.21.15.52:0/3371662702 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa674197c20 con 0x7fa674103510 2024-06-10T02:42:57.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:42:57.676+0000 7fa672ffd700 1 --2- 172.21.15.52:0/3371662702 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa674103510 0x7fa6741976e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa664006010 tx=0x7fa664005050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:42:57.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:42:57.676+0000 7fa678935700 1 -- 172.21.15.52:0/3371662702 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa664027070 con 0x7fa674103510 2024-06-10T02:42:57.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:42:57.676+0000 7fa678935700 1 -- 172.21.15.52:0/3371662702 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa66400be40 con 0x7fa674103510 2024-06-10T02:42:57.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:42:57.676+0000 7fa679937700 1 -- 172.21.15.52:0/3371662702 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa674197eb0 con 0x7fa674103510 2024-06-10T02:42:57.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:42:57.676+0000 7fa678935700 1 -- 172.21.15.52:0/3371662702 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa66400f460 con 0x7fa674103510 2024-06-10T02:42:57.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:42:57.676+0000 7fa679937700 1 -- 172.21.15.52:0/3371662702 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa67419aba0 con 0x7fa674103510 2024-06-10T02:42:57.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:42:57.677+0000 7fa678935700 1 -- 172.21.15.52:0/3371662702 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fa66402a070 con 0x7fa674103510 2024-06-10T02:42:57.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:42:57.677+0000 7fa679937700 1 -- 172.21.15.52:0/3371662702 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa674108750 con 0x7fa674103510 2024-06-10T02:42:57.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:42:57.677+0000 7fa678935700 1 --2- 172.21.15.52:0/3371662702 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fa660039150 0x7fa66003b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:42:57.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:42:57.677+0000 7fa678935700 1 -- 172.21.15.52:0/3371662702 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fa66401e030 con 0x7fa674103510 2024-06-10T02:42:57.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:42:57.678+0000 7fa66bfff700 1 --2- 172.21.15.52:0/3371662702 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fa660039150 0x7fa66003b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:42:57.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:42:57.678+0000 7fa66bfff700 1 --2- 172.21.15.52:0/3371662702 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fa660039150 0x7fa66003b610 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fa65c006fd0 tx=0x7fa65c006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:42:57.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:42:57.682+0000 7fa678935700 1 -- 172.21.15.52:0/3371662702 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fa664013070 con 0x7fa674103510 2024-06-10T02:42:57.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:42:57.893+0000 7fa679937700 1 -- 172.21.15.52:0/3371662702 --> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7fa6740628d0 con 0x7fa660039150 2024-06-10T02:42:57.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:42:57.896+0000 7fa678935700 1 -- 172.21.15.52:0/3371662702 <== mgr.14150 v2:172.21.15.52:6800/3214763969 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fa6740628d0 con 0x7fa660039150 2024-06-10T02:42:57.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:42:57.898+0000 7fa679937700 1 -- 172.21.15.52:0/3371662702 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fa660039150 msgr2=0x7fa66003b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:42:57.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:42:57.898+0000 7fa679937700 1 --2- 172.21.15.52:0/3371662702 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fa660039150 0x7fa66003b610 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fa65c006fd0 tx=0x7fa65c006e40 comp rx=0 tx=0).stop 2024-06-10T02:42:57.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:42:57.898+0000 7fa679937700 1 -- 172.21.15.52:0/3371662702 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa674103510 msgr2=0x7fa6741976e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:42:57.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:42:57.898+0000 7fa679937700 1 --2- 172.21.15.52:0/3371662702 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa674103510 0x7fa6741976e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa664006010 tx=0x7fa664005050 comp rx=0 tx=0).stop 2024-06-10T02:42:57.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:42:57.898+0000 7fa679937700 1 -- 172.21.15.52:0/3371662702 shutdown_connections 2024-06-10T02:42:57.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:42:57.898+0000 7fa679937700 1 --2- 172.21.15.52:0/3371662702 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fa660039150 0x7fa66003b610 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:42:57.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:42:57.898+0000 7fa679937700 1 --2- 172.21.15.52:0/3371662702 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa674103510 0x7fa6741976e0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:42:57.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:42:57.898+0000 7fa679937700 1 -- 172.21.15.52:0/3371662702 >> 172.21.15.52:0/3371662702 conn(0x7fa6740fd4c0 msgr2=0x7fa6740fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:42:57.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:42:57.899+0000 7fa679937700 1 -- 172.21.15.52:0/3371662702 shutdown_connections 2024-06-10T02:42:57.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:42:57.899+0000 7fa679937700 1 -- 172.21.15.52:0/3371662702 wait complete. 2024-06-10T02:42:58.626 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi120 2024-06-10T02:42:58.627 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-06-10T02:42:58.627 DEBUG:teuthology.orchestra.run.smithi120:> dd of=/etc/ceph/ceph.conf 2024-06-10T02:42:58.646 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-06-10T02:42:58.646 DEBUG:teuthology.orchestra.run.smithi120:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-06-10T02:42:58.703 INFO:tasks.cephadm:Adding host smithi120 to orchestrator... 2024-06-10T02:42:58.703 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph orch host add smithi120 2024-06-10T02:42:58.957 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:42:59.080 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:42:58 smithi052 ceph-mon[32362]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T02:42:59.080 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:42:58 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:42:59.080 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:42:58 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T02:42:59.080 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:42:58 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:42:59.080 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:42:58 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T02:42:59.080 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:42:58 smithi052 ceph-mon[32362]: Updating smithi052:/etc/ceph/ceph.conf 2024-06-10T02:42:59.081 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:42:58 smithi052 ceph-mon[32362]: Updating smithi052:/var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-10T02:42:59.081 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:42:58 smithi052 ceph-mon[32362]: Updating smithi052:/etc/ceph/ceph.client.admin.keyring 2024-06-10T02:42:59.081 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:42:58 smithi052 ceph-mon[32362]: Updating smithi052:/var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-06-10T02:43:00.105 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:42:59 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:43:00.105 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:42:59 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:43:00.105 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:42:59 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:43:00.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:00.402+0000 7fed48aca700 1 -- 172.21.15.52:0/2604638306 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fed44100bf0 msgr2=0x7fed44100fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:43:00.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:00.402+0000 7fed48aca700 1 --2- 172.21.15.52:0/2604638306 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fed44100bf0 0x7fed44100fd0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fed34009910 tx=0x7fed34009c20 comp rx=0 tx=0).stop 2024-06-10T02:43:00.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:00.402+0000 7fed48aca700 1 -- 172.21.15.52:0/2604638306 shutdown_connections 2024-06-10T02:43:00.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:00.402+0000 7fed48aca700 1 --2- 172.21.15.52:0/2604638306 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fed44100bf0 0x7fed44100fd0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:43:00.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:00.402+0000 7fed48aca700 1 -- 172.21.15.52:0/2604638306 >> 172.21.15.52:0/2604638306 conn(0x7fed440fc8b0 msgr2=0x7fed440fecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:43:00.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:00.402+0000 7fed48aca700 1 -- 172.21.15.52:0/2604638306 shutdown_connections 2024-06-10T02:43:00.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:00.402+0000 7fed48aca700 1 -- 172.21.15.52:0/2604638306 wait complete. 2024-06-10T02:43:00.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:00.403+0000 7fed48aca700 1 Processor -- start 2024-06-10T02:43:00.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:00.403+0000 7fed48aca700 1 -- start start 2024-06-10T02:43:00.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:00.403+0000 7fed48aca700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fed44100bf0 0x7fed4419f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:43:00.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:00.403+0000 7fed48aca700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fed4410e730 con 0x7fed44100bf0 2024-06-10T02:43:00.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:00.403+0000 7fed4259c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fed44100bf0 0x7fed4419f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:43:00.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:00.403+0000 7fed4259c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fed44100bf0 0x7fed4419f820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:46266/0 (socket says 172.21.15.52:46266) 2024-06-10T02:43:00.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:00.403+0000 7fed4259c700 1 -- 172.21.15.52:0/2182190426 learned_addr learned my addr 172.21.15.52:0/2182190426 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:43:00.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:00.403+0000 7fed4259c700 1 -- 172.21.15.52:0/2182190426 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed4419fd60 con 0x7fed44100bf0 2024-06-10T02:43:00.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:00.404+0000 7fed4259c700 1 --2- 172.21.15.52:0/2182190426 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fed44100bf0 0x7fed4419f820 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fed34005850 tx=0x7fed34005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:43:00.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:00.404+0000 7fed337fe700 1 -- 172.21.15.52:0/2182190426 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fed34026070 con 0x7fed44100bf0 2024-06-10T02:43:00.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:00.404+0000 7fed337fe700 1 -- 172.21.15.52:0/2182190426 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed3400bc30 con 0x7fed44100bf0 2024-06-10T02:43:00.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:00.404+0000 7fed337fe700 1 -- 172.21.15.52:0/2182190426 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fed34003a20 con 0x7fed44100bf0 2024-06-10T02:43:00.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:00.404+0000 7fed48aca700 1 -- 172.21.15.52:0/2182190426 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed4419fff0 con 0x7fed44100bf0 2024-06-10T02:43:00.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:00.404+0000 7fed48aca700 1 -- 172.21.15.52:0/2182190426 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed441999e0 con 0x7fed44100bf0 2024-06-10T02:43:00.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:00.405+0000 7fed337fe700 1 -- 172.21.15.52:0/2182190426 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fed34029070 con 0x7fed44100bf0 2024-06-10T02:43:00.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:00.405+0000 7fed48aca700 1 -- 172.21.15.52:0/2182190426 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed4406ac20 con 0x7fed44100bf0 2024-06-10T02:43:00.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:00.405+0000 7fed337fe700 1 --2- 172.21.15.52:0/2182190426 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fed2c0390f0 0x7fed2c03b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:43:00.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:00.405+0000 7fed337fe700 1 -- 172.21.15.52:0/2182190426 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fed3401e070 con 0x7fed44100bf0 2024-06-10T02:43:00.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:00.405+0000 7fed41d9b700 1 --2- 172.21.15.52:0/2182190426 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fed2c0390f0 0x7fed2c03b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:43:00.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:00.406+0000 7fed41d9b700 1 --2- 172.21.15.52:0/2182190426 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fed2c0390f0 0x7fed2c03b5b0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fed38006fd0 tx=0x7fed38006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:43:00.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:00.410+0000 7fed337fe700 1 -- 172.21.15.52:0/2182190426 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fed34013070 con 0x7fed44100bf0 2024-06-10T02:43:00.616 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:00.620+0000 7fed48aca700 1 -- 172.21.15.52:0/2182190426 --> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi120", "target": ["mon-mgr", ""]}) -- 0x7fed4419af10 con 0x7fed2c0390f0 2024-06-10T02:43:01.473 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:00 smithi052 ceph-mon[32362]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi120", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T02:43:02.473 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:01 smithi052 ceph-mon[32362]: Deploying cephadm binary to smithi120 2024-06-10T02:43:04.473 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:03 smithi052 ceph-mon[32362]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T02:43:04.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:04.875+0000 7fed337fe700 1 -- 172.21.15.52:0/2182190426 <== mgr.14150 v2:172.21.15.52:6800/3214763969 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7fed4419af10 con 0x7fed2c0390f0 2024-06-10T02:43:04.871 INFO:teuthology.orchestra.run.smithi052.stdout:Added host 'smithi120' with addr '172.21.15.120' 2024-06-10T02:43:04.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:04.878+0000 7fed48aca700 1 -- 172.21.15.52:0/2182190426 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fed2c0390f0 msgr2=0x7fed2c03b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:43:04.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:04.878+0000 7fed48aca700 1 --2- 172.21.15.52:0/2182190426 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fed2c0390f0 0x7fed2c03b5b0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fed38006fd0 tx=0x7fed38006e40 comp rx=0 tx=0).stop 2024-06-10T02:43:04.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:04.878+0000 7fed48aca700 1 -- 172.21.15.52:0/2182190426 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fed44100bf0 msgr2=0x7fed4419f820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:43:04.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:04.878+0000 7fed48aca700 1 --2- 172.21.15.52:0/2182190426 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fed44100bf0 0x7fed4419f820 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fed34005850 tx=0x7fed34005030 comp rx=0 tx=0).stop 2024-06-10T02:43:04.875 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:04.878+0000 7fed48aca700 1 -- 172.21.15.52:0/2182190426 shutdown_connections 2024-06-10T02:43:04.875 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:04.878+0000 7fed48aca700 1 --2- 172.21.15.52:0/2182190426 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fed2c0390f0 0x7fed2c03b5b0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:43:04.875 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:04.878+0000 7fed48aca700 1 --2- 172.21.15.52:0/2182190426 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fed44100bf0 0x7fed4419f820 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:43:04.875 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:04.878+0000 7fed48aca700 1 -- 172.21.15.52:0/2182190426 >> 172.21.15.52:0/2182190426 conn(0x7fed440fc8b0 msgr2=0x7fed440fd880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:43:04.875 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:04.878+0000 7fed48aca700 1 -- 172.21.15.52:0/2182190426 shutdown_connections 2024-06-10T02:43:04.875 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:04.878+0000 7fed48aca700 1 -- 172.21.15.52:0/2182190426 wait complete. 2024-06-10T02:43:05.562 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph orch host ls --format=json 2024-06-10T02:43:05.813 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:43:06.124 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:05 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:43:06.124 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:05 smithi052 ceph-mon[32362]: Added host smithi120 2024-06-10T02:43:06.124 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:05 smithi052 ceph-mon[32362]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T02:43:06.124 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:05 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T02:43:06.125 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:05 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:43:07.006 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:06 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:43:07.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:07.245+0000 7faf15316700 1 -- 172.21.15.52:0/605128684 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faf10102100 msgr2=0x7faf101024e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:43:07.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:07.245+0000 7faf15316700 1 --2- 172.21.15.52:0/605128684 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faf10102100 0x7faf101024e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7faf0000b0a0 tx=0x7faf0000b3b0 comp rx=0 tx=0).stop 2024-06-10T02:43:07.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:07.245+0000 7faf15316700 1 -- 172.21.15.52:0/605128684 shutdown_connections 2024-06-10T02:43:07.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:07.245+0000 7faf15316700 1 --2- 172.21.15.52:0/605128684 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faf10102100 0x7faf101024e0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:43:07.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:07.245+0000 7faf15316700 1 -- 172.21.15.52:0/605128684 >> 172.21.15.52:0/605128684 conn(0x7faf100fd9b0 msgr2=0x7faf100ffdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:43:07.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:07.245+0000 7faf15316700 1 -- 172.21.15.52:0/605128684 shutdown_connections 2024-06-10T02:43:07.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:07.245+0000 7faf15316700 1 -- 172.21.15.52:0/605128684 wait complete. 2024-06-10T02:43:07.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:07.246+0000 7faf15316700 1 Processor -- start 2024-06-10T02:43:07.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:07.246+0000 7faf15316700 1 -- start start 2024-06-10T02:43:07.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:07.246+0000 7faf15316700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faf10102100 0x7faf1019d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:43:07.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:07.246+0000 7faf15316700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7faf1010c4d0 con 0x7faf10102100 2024-06-10T02:43:07.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:07.246+0000 7faf0effd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faf10102100 0x7faf1019d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:43:07.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:07.246+0000 7faf0effd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faf10102100 0x7faf1019d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:41260/0 (socket says 172.21.15.52:41260) 2024-06-10T02:43:07.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:07.246+0000 7faf0effd700 1 -- 172.21.15.52:0/1952484869 learned_addr learned my addr 172.21.15.52:0/1952484869 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:43:07.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:07.246+0000 7faf0effd700 1 -- 172.21.15.52:0/1952484869 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf1019dba0 con 0x7faf10102100 2024-06-10T02:43:07.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:07.246+0000 7faf0effd700 1 --2- 172.21.15.52:0/1952484869 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faf10102100 0x7faf1019d660 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7faf00015040 tx=0x7faf00003d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:43:07.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:07.247+0000 7faf07fff700 1 -- 172.21.15.52:0/1952484869 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faf00017040 con 0x7faf10102100 2024-06-10T02:43:07.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:07.247+0000 7faf07fff700 1 -- 172.21.15.52:0/1952484869 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf00022070 con 0x7faf10102100 2024-06-10T02:43:07.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:07.247+0000 7faf15316700 1 -- 172.21.15.52:0/1952484869 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf1019de30 con 0x7faf10102100 2024-06-10T02:43:07.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:07.247+0000 7faf07fff700 1 -- 172.21.15.52:0/1952484869 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faf0001d5d0 con 0x7faf10102100 2024-06-10T02:43:07.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:07.247+0000 7faf15316700 1 -- 172.21.15.52:0/1952484869 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf10197730 con 0x7faf10102100 2024-06-10T02:43:07.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:07.248+0000 7faf07fff700 1 -- 172.21.15.52:0/1952484869 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7faf0001d880 con 0x7faf10102100 2024-06-10T02:43:07.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:07.248+0000 7faf15316700 1 -- 172.21.15.52:0/1952484869 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf101035e0 con 0x7faf10102100 2024-06-10T02:43:07.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:07.248+0000 7faf07fff700 1 --2- 172.21.15.52:0/1952484869 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7faefc039150 0x7faefc03b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:43:07.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:07.248+0000 7faf07fff700 1 -- 172.21.15.52:0/1952484869 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7faf0000f050 con 0x7faf10102100 2024-06-10T02:43:07.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:07.248+0000 7faf0e7fc700 1 --2- 172.21.15.52:0/1952484869 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7faefc039150 0x7faefc03b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:43:07.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:07.248+0000 7faf0e7fc700 1 --2- 172.21.15.52:0/1952484869 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7faefc039150 0x7faefc03b610 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7faef8006fd0 tx=0x7faef8006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:43:07.249 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:07.253+0000 7faf07fff700 1 -- 172.21.15.52:0/1952484869 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7faf00007e20 con 0x7faf10102100 2024-06-10T02:43:07.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:07.465+0000 7faf15316700 1 -- 172.21.15.52:0/1952484869 --> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7faf10198c60 con 0x7faefc039150 2024-06-10T02:43:07.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:07.466+0000 7faf07fff700 1 -- 172.21.15.52:0/1952484869 <== mgr.14150 v2:172.21.15.52:6800/3214763969 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+160 (secure 0 0 0) 0x7faf10198c60 con 0x7faefc039150 2024-06-10T02:43:07.462 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-10T02:43:07.462 INFO:teuthology.orchestra.run.smithi052.stdout:[{"addr": "172.21.15.52", "hostname": "smithi052", "labels": [], "status": ""}, {"addr": "172.21.15.120", "hostname": "smithi120", "labels": [], "status": ""}] 2024-06-10T02:43:07.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:07.468+0000 7faf15316700 1 -- 172.21.15.52:0/1952484869 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7faefc039150 msgr2=0x7faefc03b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:43:07.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:07.468+0000 7faf15316700 1 --2- 172.21.15.52:0/1952484869 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7faefc039150 0x7faefc03b610 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7faef8006fd0 tx=0x7faef8006e40 comp rx=0 tx=0).stop 2024-06-10T02:43:07.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:07.468+0000 7faf15316700 1 -- 172.21.15.52:0/1952484869 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faf10102100 msgr2=0x7faf1019d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:43:07.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:07.468+0000 7faf15316700 1 --2- 172.21.15.52:0/1952484869 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faf10102100 0x7faf1019d660 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7faf00015040 tx=0x7faf00003d50 comp rx=0 tx=0).stop 2024-06-10T02:43:07.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:07.468+0000 7faf15316700 1 -- 172.21.15.52:0/1952484869 shutdown_connections 2024-06-10T02:43:07.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:07.468+0000 7faf15316700 1 --2- 172.21.15.52:0/1952484869 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7faefc039150 0x7faefc03b610 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:43:07.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:07.468+0000 7faf15316700 1 --2- 172.21.15.52:0/1952484869 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faf10102100 0x7faf1019d660 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:43:07.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:07.468+0000 7faf15316700 1 -- 172.21.15.52:0/1952484869 >> 172.21.15.52:0/1952484869 conn(0x7faf100fd9b0 msgr2=0x7faf10103ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:43:07.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:07.468+0000 7faf15316700 1 -- 172.21.15.52:0/1952484869 shutdown_connections 2024-06-10T02:43:07.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:07.468+0000 7faf15316700 1 -- 172.21.15.52:0/1952484869 wait complete. 2024-06-10T02:43:08.051 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi188 2024-06-10T02:43:08.051 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-06-10T02:43:08.051 DEBUG:teuthology.orchestra.run.smithi188:> dd of=/etc/ceph/ceph.conf 2024-06-10T02:43:08.071 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-06-10T02:43:08.071 DEBUG:teuthology.orchestra.run.smithi188:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-06-10T02:43:08.128 INFO:tasks.cephadm:Adding host smithi188 to orchestrator... 2024-06-10T02:43:08.129 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph orch host add smithi188 2024-06-10T02:43:08.376 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:43:08.723 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:08 smithi052 ceph-mon[32362]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T02:43:08.723 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:08 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:43:08.723 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:08 smithi052 ceph-mon[32362]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-10T02:43:09.880 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:09.883+0000 7f537879b700 1 -- 172.21.15.52:0/2145464579 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5370108020 msgr2=0x7f5370108400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:43:09.880 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:09.883+0000 7f537879b700 1 --2- 172.21.15.52:0/2145464579 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5370108020 0x7f5370108400 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f536c00b0a0 tx=0x7f536c00b3b0 comp rx=0 tx=0).stop 2024-06-10T02:43:09.880 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:09.883+0000 7f537879b700 1 -- 172.21.15.52:0/2145464579 shutdown_connections 2024-06-10T02:43:09.880 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:09.883+0000 7f537879b700 1 --2- 172.21.15.52:0/2145464579 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5370108020 0x7f5370108400 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:43:09.880 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:09.883+0000 7f537879b700 1 -- 172.21.15.52:0/2145464579 >> 172.21.15.52:0/2145464579 conn(0x7f5370077510 msgr2=0x7f5370077920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:43:09.880 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:09.883+0000 7f537879b700 1 -- 172.21.15.52:0/2145464579 shutdown_connections 2024-06-10T02:43:09.880 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:09.883+0000 7f537879b700 1 -- 172.21.15.52:0/2145464579 wait complete. 2024-06-10T02:43:09.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:09.884+0000 7f537879b700 1 Processor -- start 2024-06-10T02:43:09.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:09.884+0000 7f537879b700 1 -- start start 2024-06-10T02:43:09.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:09.884+0000 7f537879b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5370108020 0x7f537019d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:43:09.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:09.884+0000 7f537879b700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f537010c620 con 0x7f5370108020 2024-06-10T02:43:09.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:09.884+0000 7f5376537700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5370108020 0x7f537019d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:43:09.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:09.884+0000 7f5376537700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5370108020 0x7f537019d720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:41288/0 (socket says 172.21.15.52:41288) 2024-06-10T02:43:09.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:09.884+0000 7f5376537700 1 -- 172.21.15.52:0/2590961265 learned_addr learned my addr 172.21.15.52:0/2590961265 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:43:09.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:09.884+0000 7f5376537700 1 -- 172.21.15.52:0/2590961265 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f537019dc60 con 0x7f5370108020 2024-06-10T02:43:09.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:09.884+0000 7f5376537700 1 --2- 172.21.15.52:0/2590961265 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5370108020 0x7f537019d720 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f536c0062a0 tx=0x7f536c003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:43:09.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:09.885+0000 7f53677fe700 1 -- 172.21.15.52:0/2590961265 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f536c017040 con 0x7f5370108020 2024-06-10T02:43:09.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:09.885+0000 7f53677fe700 1 -- 172.21.15.52:0/2590961265 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f536c022070 con 0x7f5370108020 2024-06-10T02:43:09.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:09.885+0000 7f53677fe700 1 -- 172.21.15.52:0/2590961265 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f536c01d440 con 0x7f5370108020 2024-06-10T02:43:09.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:09.885+0000 7f537879b700 1 -- 172.21.15.52:0/2590961265 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f537019def0 con 0x7f5370108020 2024-06-10T02:43:09.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:09.885+0000 7f537879b700 1 -- 172.21.15.52:0/2590961265 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5370197930 con 0x7f5370108020 2024-06-10T02:43:09.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:09.886+0000 7f53677fe700 1 -- 172.21.15.52:0/2590961265 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f536c004000 con 0x7f5370108020 2024-06-10T02:43:09.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:09.886+0000 7f537879b700 1 -- 172.21.15.52:0/2590961265 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5370103740 con 0x7f5370108020 2024-06-10T02:43:09.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:09.886+0000 7f53677fe700 1 --2- 172.21.15.52:0/2590961265 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f535c039150 0x7f535c03b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:43:09.884 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:09.886+0000 7f53677fe700 1 -- 172.21.15.52:0/2590961265 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f536c00f050 con 0x7f5370108020 2024-06-10T02:43:09.884 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:09.886+0000 7f5375d36700 1 --2- 172.21.15.52:0/2590961265 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f535c039150 0x7f535c03b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:43:09.884 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:09.887+0000 7f5375d36700 1 --2- 172.21.15.52:0/2590961265 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f535c039150 0x7f535c03b610 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f5360006fd0 tx=0x7f5360006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:43:09.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:09.891+0000 7f53677fe700 1 -- 172.21.15.52:0/2590961265 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f536c007e20 con 0x7f5370108020 2024-06-10T02:43:10.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:10.103+0000 7f537879b700 1 -- 172.21.15.52:0/2590961265 --> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi188", "target": ["mon-mgr", ""]}) -- 0x7f5370198e10 con 0x7f535c039150 2024-06-10T02:43:10.723 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:10 smithi052 ceph-mon[32362]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T02:43:10.723 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:10 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:43:10.723 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:10 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:43:10.723 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:10 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:43:10.723 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:10 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:43:10.723 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:10 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi120", "name": "osd_memory_target"}]: dispatch 2024-06-10T02:43:10.724 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:10 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:43:10.724 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:10 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T02:43:11.723 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:11 smithi052 ceph-mon[32362]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi188", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T02:43:11.723 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:11 smithi052 ceph-mon[32362]: Updating smithi120:/etc/ceph/ceph.conf 2024-06-10T02:43:11.723 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:11 smithi052 ceph-mon[32362]: Updating smithi120:/var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-10T02:43:11.723 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:11 smithi052 ceph-mon[32362]: Deploying cephadm binary to smithi188 2024-06-10T02:43:12.723 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:12 smithi052 ceph-mon[32362]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T02:43:12.723 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:12 smithi052 ceph-mon[32362]: Updating smithi120:/etc/ceph/ceph.client.admin.keyring 2024-06-10T02:43:12.723 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:12 smithi052 ceph-mon[32362]: Updating smithi120:/var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-06-10T02:43:14.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:14.397+0000 7f53677fe700 1 -- 172.21.15.52:0/2590961265 <== mgr.14150 v2:172.21.15.52:6800/3214763969 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f5370198e10 con 0x7f535c039150 2024-06-10T02:43:14.394 INFO:teuthology.orchestra.run.smithi052.stdout:Added host 'smithi188' with addr '172.21.15.188' 2024-06-10T02:43:14.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:14.400+0000 7f537879b700 1 -- 172.21.15.52:0/2590961265 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f535c039150 msgr2=0x7f535c03b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:43:14.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:14.400+0000 7f537879b700 1 --2- 172.21.15.52:0/2590961265 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f535c039150 0x7f535c03b610 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f5360006fd0 tx=0x7f5360006e40 comp rx=0 tx=0).stop 2024-06-10T02:43:14.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:14.400+0000 7f537879b700 1 -- 172.21.15.52:0/2590961265 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5370108020 msgr2=0x7f537019d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:43:14.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:14.400+0000 7f537879b700 1 --2- 172.21.15.52:0/2590961265 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5370108020 0x7f537019d720 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f536c0062a0 tx=0x7f536c003ce0 comp rx=0 tx=0).stop 2024-06-10T02:43:14.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:14.400+0000 7f537879b700 1 -- 172.21.15.52:0/2590961265 shutdown_connections 2024-06-10T02:43:14.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:14.400+0000 7f537879b700 1 --2- 172.21.15.52:0/2590961265 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f535c039150 0x7f535c03b610 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:43:14.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:14.400+0000 7f537879b700 1 --2- 172.21.15.52:0/2590961265 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5370108020 0x7f537019d720 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:43:14.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:14.400+0000 7f537879b700 1 -- 172.21.15.52:0/2590961265 >> 172.21.15.52:0/2590961265 conn(0x7f5370077510 msgr2=0x7f5370103ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:43:14.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:14.400+0000 7f537879b700 1 -- 172.21.15.52:0/2590961265 shutdown_connections 2024-06-10T02:43:14.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:14.400+0000 7f537879b700 1 -- 172.21.15.52:0/2590961265 wait complete. 2024-06-10T02:43:14.692 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:14 smithi052 ceph-mon[32362]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T02:43:14.692 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:14 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:43:14.692 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:14 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:43:14.693 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:14 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:43:14.693 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:14 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:43:14.693 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:14 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T02:43:14.962 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph orch host ls --format=json 2024-06-10T02:43:15.212 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:43:15.596 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:15 smithi052 ceph-mon[32362]: Added host smithi188 2024-06-10T02:43:15.596 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:15 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:43:16.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:16 smithi052 ceph-mon[32362]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T02:43:16.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:16 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:43:16.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:16.697+0000 7f6529900700 1 -- 172.21.15.52:0/3077612702 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6524100870 msgr2=0x7f6524100c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:43:16.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:16.697+0000 7f6529900700 1 --2- 172.21.15.52:0/3077612702 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6524100870 0x7f6524100c50 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f651400aa20 tx=0x7f651400ad30 comp rx=0 tx=0).stop 2024-06-10T02:43:16.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:16.697+0000 7f6529900700 1 -- 172.21.15.52:0/3077612702 shutdown_connections 2024-06-10T02:43:16.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:16.697+0000 7f6529900700 1 --2- 172.21.15.52:0/3077612702 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6524100870 0x7f6524100c50 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:43:16.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:16.697+0000 7f6529900700 1 -- 172.21.15.52:0/3077612702 >> 172.21.15.52:0/3077612702 conn(0x7f65240fc000 msgr2=0x7f65240fe420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:43:16.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:16.697+0000 7f6529900700 1 -- 172.21.15.52:0/3077612702 shutdown_connections 2024-06-10T02:43:16.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:16.697+0000 7f6529900700 1 -- 172.21.15.52:0/3077612702 wait complete. 2024-06-10T02:43:16.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:16.697+0000 7f6529900700 1 Processor -- start 2024-06-10T02:43:16.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:16.698+0000 7f6529900700 1 -- start start 2024-06-10T02:43:16.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:16.698+0000 7f6529900700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6524100870 0x7f65240fc4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:43:16.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:16.698+0000 7f6529900700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f652410a3c0 con 0x7f6524100870 2024-06-10T02:43:16.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:16.698+0000 7f6522ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6524100870 0x7f65240fc4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:43:16.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:16.698+0000 7f6522ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6524100870 0x7f65240fc4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49402/0 (socket says 172.21.15.52:49402) 2024-06-10T02:43:16.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:16.698+0000 7f6522ffd700 1 -- 172.21.15.52:0/4192225127 learned_addr learned my addr 172.21.15.52:0/4192225127 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:43:16.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:16.698+0000 7f6522ffd700 1 -- 172.21.15.52:0/4192225127 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65240fe3a0 con 0x7f6524100870 2024-06-10T02:43:16.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:16.698+0000 7f6522ffd700 1 --2- 172.21.15.52:0/4192225127 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6524100870 0x7f65240fc4b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f651400c6d0 tx=0x7f6514004f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:43:16.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:16.699+0000 7f65288fe700 1 -- 172.21.15.52:0/4192225127 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6514005490 con 0x7f6524100870 2024-06-10T02:43:16.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:16.699+0000 7f6529900700 1 -- 172.21.15.52:0/4192225127 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65240fca50 con 0x7f6524100870 2024-06-10T02:43:16.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:16.699+0000 7f65288fe700 1 -- 172.21.15.52:0/4192225127 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6514005630 con 0x7f6524100870 2024-06-10T02:43:16.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:16.699+0000 7f65288fe700 1 -- 172.21.15.52:0/4192225127 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f651400e750 con 0x7f6524100870 2024-06-10T02:43:16.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:16.699+0000 7f6529900700 1 -- 172.21.15.52:0/4192225127 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65240fcf10 con 0x7f6524100870 2024-06-10T02:43:16.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:16.699+0000 7f65288fe700 1 -- 172.21.15.52:0/4192225127 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f651400ea10 con 0x7f6524100870 2024-06-10T02:43:16.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:16.700+0000 7f65288fe700 1 --2- 172.21.15.52:0/4192225127 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f650c0390f0 0x7f650c03b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:43:16.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:16.700+0000 7f65288fe700 1 -- 172.21.15.52:0/4192225127 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f651401e070 con 0x7f6524100870 2024-06-10T02:43:16.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:16.700+0000 7f6529900700 1 -- 172.21.15.52:0/4192225127 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6524101c70 con 0x7f6524100870 2024-06-10T02:43:16.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:16.700+0000 7f65227fc700 1 --2- 172.21.15.52:0/4192225127 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f650c0390f0 0x7f650c03b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:43:16.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:16.700+0000 7f65227fc700 1 --2- 172.21.15.52:0/4192225127 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f650c0390f0 0x7f650c03b5b0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f6518006fd0 tx=0x7f6518006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:43:16.702 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:16.705+0000 7f65288fe700 1 -- 172.21.15.52:0/4192225127 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f6514020370 con 0x7f6524100870 2024-06-10T02:43:16.913 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:16.916+0000 7f6529900700 1 -- 172.21.15.52:0/4192225127 --> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f65240fd9e0 con 0x7f650c0390f0 2024-06-10T02:43:16.914 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:16.917+0000 7f65288fe700 1 -- 172.21.15.52:0/4192225127 <== mgr.14150 v2:172.21.15.52:6800/3214763969 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+240 (secure 0 0 0) 0x7f65240fd9e0 con 0x7f650c0390f0 2024-06-10T02:43:16.914 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-10T02:43:16.915 INFO:teuthology.orchestra.run.smithi052.stdout:[{"addr": "172.21.15.52", "hostname": "smithi052", "labels": [], "status": ""}, {"addr": "172.21.15.120", "hostname": "smithi120", "labels": [], "status": ""}, {"addr": "172.21.15.188", "hostname": "smithi188", "labels": [], "status": ""}] 2024-06-10T02:43:16.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:16.919+0000 7f6529900700 1 -- 172.21.15.52:0/4192225127 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f650c0390f0 msgr2=0x7f650c03b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:43:16.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:16.920+0000 7f6529900700 1 --2- 172.21.15.52:0/4192225127 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f650c0390f0 0x7f650c03b5b0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f6518006fd0 tx=0x7f6518006e40 comp rx=0 tx=0).stop 2024-06-10T02:43:16.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:16.920+0000 7f6529900700 1 -- 172.21.15.52:0/4192225127 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6524100870 msgr2=0x7f65240fc4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:43:16.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:16.920+0000 7f6529900700 1 --2- 172.21.15.52:0/4192225127 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6524100870 0x7f65240fc4b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f651400c6d0 tx=0x7f6514004f20 comp rx=0 tx=0).stop 2024-06-10T02:43:16.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:16.920+0000 7f6529900700 1 -- 172.21.15.52:0/4192225127 shutdown_connections 2024-06-10T02:43:16.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:16.920+0000 7f6529900700 1 --2- 172.21.15.52:0/4192225127 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f650c0390f0 0x7f650c03b5b0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:43:16.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:16.920+0000 7f6529900700 1 --2- 172.21.15.52:0/4192225127 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6524100870 0x7f65240fc4b0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:43:16.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:16.920+0000 7f6529900700 1 -- 172.21.15.52:0/4192225127 >> 172.21.15.52:0/4192225127 conn(0x7f65240fc000 msgr2=0x7f652418f850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:43:16.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:16.920+0000 7f6529900700 1 -- 172.21.15.52:0/4192225127 shutdown_connections 2024-06-10T02:43:16.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:16.920+0000 7f6529900700 1 -- 172.21.15.52:0/4192225127 wait complete. 2024-06-10T02:43:17.460 INFO:tasks.cephadm:Setting crush tunables to default 2024-06-10T02:43:17.460 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph osd crush tunables default 2024-06-10T02:43:17.703 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:43:18.084 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:18 smithi052 ceph-mon[32362]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T02:43:18.084 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:18 smithi052 ceph-mon[32362]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-10T02:43:18.084 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:18 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:43:19.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:19.158+0000 7fa41b520700 1 -- 172.21.15.52:0/3987030815 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa414100e40 msgr2=0x7fa414101220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:43:19.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:19.158+0000 7fa41b520700 1 --2- 172.21.15.52:0/3987030815 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa414100e40 0x7fa414101220 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fa410009910 tx=0x7fa410009c20 comp rx=0 tx=0).stop 2024-06-10T02:43:19.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:19.158+0000 7fa41b520700 1 -- 172.21.15.52:0/3987030815 shutdown_connections 2024-06-10T02:43:19.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:19.158+0000 7fa41b520700 1 --2- 172.21.15.52:0/3987030815 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa414100e40 0x7fa414101220 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:43:19.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:19.158+0000 7fa41b520700 1 -- 172.21.15.52:0/3987030815 >> 172.21.15.52:0/3987030815 conn(0x7fa4140fc8c0 msgr2=0x7fa4140fece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:43:19.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:19.158+0000 7fa41b520700 1 -- 172.21.15.52:0/3987030815 shutdown_connections 2024-06-10T02:43:19.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:19.158+0000 7fa41b520700 1 -- 172.21.15.52:0/3987030815 wait complete. 2024-06-10T02:43:19.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:19.159+0000 7fa41b520700 1 Processor -- start 2024-06-10T02:43:19.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:19.159+0000 7fa41b520700 1 -- start start 2024-06-10T02:43:19.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:19.159+0000 7fa41b520700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa414100e40 0x7fa41419dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:43:19.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:19.159+0000 7fa41b520700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fa414112b40 con 0x7fa414100e40 2024-06-10T02:43:19.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:19.159+0000 7fa4192bc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa414100e40 0x7fa41419dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:43:19.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:19.159+0000 7fa4192bc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa414100e40 0x7fa41419dfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49420/0 (socket says 172.21.15.52:49420) 2024-06-10T02:43:19.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:19.159+0000 7fa4192bc700 1 -- 172.21.15.52:0/2701410684 learned_addr learned my addr 172.21.15.52:0/2701410684 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:43:19.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:19.159+0000 7fa4192bc700 1 -- 172.21.15.52:0/2701410684 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa41419e4e0 con 0x7fa414100e40 2024-06-10T02:43:19.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:19.160+0000 7fa4192bc700 1 --2- 172.21.15.52:0/2701410684 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa414100e40 0x7fa41419dfa0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fa410006010 tx=0x7fa410005050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:43:19.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:19.160+0000 7fa40a7fc700 1 -- 172.21.15.52:0/2701410684 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa410026070 con 0x7fa414100e40 2024-06-10T02:43:19.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:19.160+0000 7fa41b520700 1 -- 172.21.15.52:0/2701410684 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa41419e770 con 0x7fa414100e40 2024-06-10T02:43:19.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:19.160+0000 7fa40a7fc700 1 -- 172.21.15.52:0/2701410684 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa41000bc30 con 0x7fa414100e40 2024-06-10T02:43:19.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:19.160+0000 7fa40a7fc700 1 -- 172.21.15.52:0/2701410684 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa410003ab0 con 0x7fa414100e40 2024-06-10T02:43:19.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:19.160+0000 7fa41b520700 1 -- 172.21.15.52:0/2701410684 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa41419ebd0 con 0x7fa414100e40 2024-06-10T02:43:19.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:19.161+0000 7fa40a7fc700 1 -- 172.21.15.52:0/2701410684 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fa410029070 con 0x7fa414100e40 2024-06-10T02:43:19.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:19.161+0000 7fa41b520700 1 -- 172.21.15.52:0/2701410684 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa41406ac20 con 0x7fa414100e40 2024-06-10T02:43:19.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:19.161+0000 7fa40a7fc700 1 --2- 172.21.15.52:0/2701410684 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fa4000390f0 0x7fa40003b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:43:19.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:19.161+0000 7fa40a7fc700 1 -- 172.21.15.52:0/2701410684 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fa41001e070 con 0x7fa414100e40 2024-06-10T02:43:19.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:19.161+0000 7fa418abb700 1 --2- 172.21.15.52:0/2701410684 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fa4000390f0 0x7fa40003b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:43:19.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:19.162+0000 7fa418abb700 1 --2- 172.21.15.52:0/2701410684 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fa4000390f0 0x7fa40003b5b0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fa404006fd0 tx=0x7fa404006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:43:19.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:19.166+0000 7fa40a7fc700 1 -- 172.21.15.52:0/2701410684 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fa410013070 con 0x7fa414100e40 2024-06-10T02:43:19.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:19.379+0000 7fa41b520700 1 -- 172.21.15.52:0/2701410684 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7fa41419f300 con 0x7fa414100e40 2024-06-10T02:43:20.008 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:20.011+0000 7fa40a7fc700 1 -- 172.21.15.52:0/2701410684 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7fa410030090 con 0x7fa414100e40 2024-06-10T02:43:20.013 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:20.015+0000 7fa41b520700 1 -- 172.21.15.52:0/2701410684 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fa4000390f0 msgr2=0x7fa40003b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:43:20.013 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:20.015+0000 7fa41b520700 1 --2- 172.21.15.52:0/2701410684 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fa4000390f0 0x7fa40003b5b0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fa404006fd0 tx=0x7fa404006e40 comp rx=0 tx=0).stop 2024-06-10T02:43:20.013 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:20.015+0000 7fa41b520700 1 -- 172.21.15.52:0/2701410684 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa414100e40 msgr2=0x7fa41419dfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:43:20.013 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:20.015+0000 7fa41b520700 1 --2- 172.21.15.52:0/2701410684 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa414100e40 0x7fa41419dfa0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fa410006010 tx=0x7fa410005050 comp rx=0 tx=0).stop 2024-06-10T02:43:20.013 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:20.016+0000 7fa41b520700 1 -- 172.21.15.52:0/2701410684 shutdown_connections 2024-06-10T02:43:20.013 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:20.016+0000 7fa41b520700 1 --2- 172.21.15.52:0/2701410684 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fa4000390f0 0x7fa40003b5b0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:43:20.013 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:20.016+0000 7fa41b520700 1 --2- 172.21.15.52:0/2701410684 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa414100e40 0x7fa41419dfa0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:43:20.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:20.016+0000 7fa41b520700 1 -- 172.21.15.52:0/2701410684 >> 172.21.15.52:0/2701410684 conn(0x7fa4140fc8c0 msgr2=0x7fa4140fcca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:43:20.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:20.016+0000 7fa41b520700 1 -- 172.21.15.52:0/2701410684 shutdown_connections 2024-06-10T02:43:20.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:20.016+0000 7fa41b520700 1 -- 172.21.15.52:0/2701410684 wait complete. 2024-06-10T02:43:20.014 INFO:teuthology.orchestra.run.smithi052.stderr:adjusted tunables profile to default 2024-06-10T02:43:20.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:20 smithi052 ceph-mon[32362]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T02:43:20.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:20 smithi052 ceph-mon[32362]: from='client.? 172.21.15.52:0/2701410684' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-06-10T02:43:20.336 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:20 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:43:20.336 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:20 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:43:20.336 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:20 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:43:20.336 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:20 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:43:20.336 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:20 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi188", "name": "osd_memory_target"}]: dispatch 2024-06-10T02:43:20.336 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:20 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:43:20.336 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:20 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T02:43:20.651 INFO:tasks.cephadm:Adding mon.a on smithi052 2024-06-10T02:43:20.651 INFO:tasks.cephadm:Adding mon.b on smithi120 2024-06-10T02:43:20.651 INFO:tasks.cephadm:Adding mon.c on smithi188 2024-06-10T02:43:20.651 DEBUG:teuthology.orchestra.run.smithi188:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph orch apply mon '3;smithi052:172.21.15.52=a;smithi120:172.21.15.120=b;smithi188:172.21.15.188=c' 2024-06-10T02:43:20.882 INFO:teuthology.orchestra.run.smithi188.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-10T02:43:21.473 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:21 smithi052 ceph-mon[32362]: Updating smithi188:/etc/ceph/ceph.conf 2024-06-10T02:43:21.473 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:21 smithi052 ceph-mon[32362]: from='client.? 172.21.15.52:0/2701410684' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-06-10T02:43:21.473 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:21 smithi052 ceph-mon[32362]: osdmap e4: 0 total, 0 up, 0 in 2024-06-10T02:43:21.473 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:21 smithi052 ceph-mon[32362]: Updating smithi188:/var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-10T02:43:21.473 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:21 smithi052 ceph-mon[32362]: Updating smithi188:/etc/ceph/ceph.client.admin.keyring 2024-06-10T02:43:21.473 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:21 smithi052 ceph-mon[32362]: Updating smithi188:/var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-06-10T02:43:21.474 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:21 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:43:21.474 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:21 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:43:21.474 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:21 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:43:22.363 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:22.364+0000 7f2e3e5c6700 1 -- 172.21.15.188:0/3946375108 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2e381030a0 msgr2=0x7f2e38103480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:43:22.363 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:22.364+0000 7f2e3e5c6700 1 --2- 172.21.15.188:0/3946375108 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2e381030a0 0x7f2e38103480 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f2e28009910 tx=0x7f2e28009c20 comp rx=0 tx=0).stop 2024-06-10T02:43:22.364 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:22.364+0000 7f2e3e5c6700 1 -- 172.21.15.188:0/3946375108 shutdown_connections 2024-06-10T02:43:22.364 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:22.364+0000 7f2e3e5c6700 1 --2- 172.21.15.188:0/3946375108 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2e381030a0 0x7f2e38103480 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:43:22.364 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:22.364+0000 7f2e3e5c6700 1 -- 172.21.15.188:0/3946375108 >> 172.21.15.188:0/3946375108 conn(0x7f2e380fc890 msgr2=0x7f2e380fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:43:22.364 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:22.364+0000 7f2e3e5c6700 1 -- 172.21.15.188:0/3946375108 shutdown_connections 2024-06-10T02:43:22.364 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:22.365+0000 7f2e3e5c6700 1 -- 172.21.15.188:0/3946375108 wait complete. 2024-06-10T02:43:22.364 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:22.365+0000 7f2e3e5c6700 1 Processor -- start 2024-06-10T02:43:22.364 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:22.365+0000 7f2e3e5c6700 1 -- start start 2024-06-10T02:43:22.364 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:22.365+0000 7f2e3e5c6700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2e381030a0 0x7f2e38100d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:43:22.364 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:22.365+0000 7f2e3e5c6700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f2e3810c520 con 0x7f2e381030a0 2024-06-10T02:43:22.364 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:22.366+0000 7f2e37fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2e381030a0 0x7f2e38100d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:43:22.364 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:22.366+0000 7f2e37fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2e381030a0 0x7f2e38100d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.188:56232/0 (socket says 172.21.15.188:56232) 2024-06-10T02:43:22.364 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:22.366+0000 7f2e37fff700 1 -- 172.21.15.188:0/4113580483 learned_addr learned my addr 172.21.15.188:0/4113580483 (peer_addr_for_me v2:172.21.15.188:0/0) 2024-06-10T02:43:22.365 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:22.366+0000 7f2e37fff700 1 -- 172.21.15.188:0/4113580483 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e38102bf0 con 0x7f2e381030a0 2024-06-10T02:43:22.365 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:22.366+0000 7f2e37fff700 1 --2- 172.21.15.188:0/4113580483 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2e381030a0 0x7f2e38100d00 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f2e28006010 tx=0x7f2e28005050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:43:22.365 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:22.366+0000 7f2e357fa700 1 -- 172.21.15.188:0/4113580483 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2e28026070 con 0x7f2e381030a0 2024-06-10T02:43:22.365 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:22.366+0000 7f2e3e5c6700 1 -- 172.21.15.188:0/4113580483 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e38101240 con 0x7f2e381030a0 2024-06-10T02:43:22.365 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:22.366+0000 7f2e357fa700 1 -- 172.21.15.188:0/4113580483 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e2800bc30 con 0x7f2e381030a0 2024-06-10T02:43:22.365 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:22.366+0000 7f2e357fa700 1 -- 172.21.15.188:0/4113580483 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2e28003ab0 con 0x7f2e381030a0 2024-06-10T02:43:22.365 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:22.366+0000 7f2e3e5c6700 1 -- 172.21.15.188:0/4113580483 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e38101700 con 0x7f2e381030a0 2024-06-10T02:43:22.366 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:22.367+0000 7f2e357fa700 1 -- 172.21.15.188:0/4113580483 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f2e28029070 con 0x7f2e381030a0 2024-06-10T02:43:22.366 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:22.367+0000 7f2e357fa700 1 --2- 172.21.15.188:0/4113580483 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f2e200390f0 0x7f2e2003b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:43:22.366 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:22.367+0000 7f2e3e5c6700 1 -- 172.21.15.188:0/4113580483 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e3806a9c0 con 0x7f2e381030a0 2024-06-10T02:43:22.366 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:22.368+0000 7f2e357fa700 1 -- 172.21.15.188:0/4113580483 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f2e2801e070 con 0x7f2e381030a0 2024-06-10T02:43:22.366 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:22.368+0000 7f2e377fe700 1 --2- 172.21.15.188:0/4113580483 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f2e200390f0 0x7f2e2003b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:43:22.366 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:22.368+0000 7f2e377fe700 1 --2- 172.21.15.188:0/4113580483 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f2e200390f0 0x7f2e2003b5b0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f2e2c006fd0 tx=0x7f2e2c006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:43:22.370 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:22.373+0000 7f2e357fa700 1 -- 172.21.15.188:0/4113580483 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f2e28013070 con 0x7f2e381030a0 2024-06-10T02:43:22.473 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:22 smithi052 ceph-mon[32362]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T02:43:22.580 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:22.583+0000 7f2e3e5c6700 1 -- 172.21.15.188:0/4113580483 --> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi052:172.21.15.52=a;smithi120:172.21.15.120=b;smithi188:172.21.15.188=c", "target": ["mon-mgr", ""]}) -- 0x7f2e381022f0 con 0x7f2e200390f0 2024-06-10T02:43:22.587 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:22.591+0000 7f2e357fa700 1 -- 172.21.15.188:0/4113580483 <== mgr.14150 v2:172.21.15.52:6800/3214763969 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f2e381022f0 con 0x7f2e200390f0 2024-06-10T02:43:22.587 INFO:teuthology.orchestra.run.smithi188.stdout:Scheduled mon update... 2024-06-10T02:43:22.590 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:22.592+0000 7f2e3e5c6700 1 -- 172.21.15.188:0/4113580483 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f2e200390f0 msgr2=0x7f2e2003b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:43:22.590 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:22.592+0000 7f2e3e5c6700 1 --2- 172.21.15.188:0/4113580483 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f2e200390f0 0x7f2e2003b5b0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f2e2c006fd0 tx=0x7f2e2c006e40 comp rx=0 tx=0).stop 2024-06-10T02:43:22.590 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:22.592+0000 7f2e3e5c6700 1 -- 172.21.15.188:0/4113580483 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2e381030a0 msgr2=0x7f2e38100d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:43:22.590 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:22.592+0000 7f2e3e5c6700 1 --2- 172.21.15.188:0/4113580483 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2e381030a0 0x7f2e38100d00 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f2e28006010 tx=0x7f2e28005050 comp rx=0 tx=0).stop 2024-06-10T02:43:22.590 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:22.593+0000 7f2e3e5c6700 1 -- 172.21.15.188:0/4113580483 shutdown_connections 2024-06-10T02:43:22.590 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:22.593+0000 7f2e3e5c6700 1 --2- 172.21.15.188:0/4113580483 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f2e200390f0 0x7f2e2003b5b0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:43:22.590 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:22.593+0000 7f2e3e5c6700 1 --2- 172.21.15.188:0/4113580483 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2e381030a0 0x7f2e38100d00 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:43:22.590 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:22.593+0000 7f2e3e5c6700 1 -- 172.21.15.188:0/4113580483 >> 172.21.15.188:0/4113580483 conn(0x7f2e380fc890 msgr2=0x7f2e38103e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:43:22.590 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:22.593+0000 7f2e3e5c6700 1 -- 172.21.15.188:0/4113580483 shutdown_connections 2024-06-10T02:43:22.590 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:22.593+0000 7f2e3e5c6700 1 -- 172.21.15.188:0/4113580483 wait complete. 2024-06-10T02:43:23.143 DEBUG:teuthology.orchestra.run.smithi120:mon.b> sudo journalctl -f -n 0 -u ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968@mon.b.service 2024-06-10T02:43:23.146 DEBUG:teuthology.orchestra.run.smithi188:mon.c> sudo journalctl -f -n 0 -u ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968@mon.c.service 2024-06-10T02:43:23.149 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-06-10T02:43:23.149 DEBUG:teuthology.orchestra.run.smithi188:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph mon dump -f json 2024-06-10T02:43:23.823 INFO:teuthology.orchestra.run.smithi188.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-10T02:43:23.973 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:23 smithi052 ceph-mon[32362]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi052:172.21.15.52=a;smithi120:172.21.15.120=b;smithi188:172.21.15.188=c", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T02:43:23.973 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:23 smithi052 ceph-mon[32362]: Saving service mon spec with placement smithi052:172.21.15.52=a;smithi120:172.21.15.120=b;smithi188:172.21.15.188=c;count:3 2024-06-10T02:43:23.973 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:23 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:43:23.973 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:23 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T02:43:23.974 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:23 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:43:23.974 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:23 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T02:43:23.974 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:23 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:43:23.974 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:23 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-10T02:43:23.974 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:23 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:43:23.974 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:23 smithi052 ceph-mon[32362]: Deploying daemon mon.c on smithi188 2024-06-10T02:43:24.973 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:24 smithi052 ceph-mon[32362]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T02:43:26.973 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:26 smithi052 ceph-mon[32362]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T02:43:27.090 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:27.092+0000 7efcab8ce700 1 -- 172.21.15.188:0/2712663079 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efca4100fe0 msgr2=0x7efca41013c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:43:27.090 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:27.092+0000 7efcab8ce700 1 --2- 172.21.15.188:0/2712663079 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efca4100fe0 0x7efca41013c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7efca0009910 tx=0x7efca0009c20 comp rx=0 tx=0).stop 2024-06-10T02:43:27.091 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:27.092+0000 7efcab8ce700 1 -- 172.21.15.188:0/2712663079 shutdown_connections 2024-06-10T02:43:27.091 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:27.092+0000 7efcab8ce700 1 --2- 172.21.15.188:0/2712663079 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efca4100fe0 0x7efca41013c0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:43:27.091 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:27.092+0000 7efcab8ce700 1 -- 172.21.15.188:0/2712663079 >> 172.21.15.188:0/2712663079 conn(0x7efca40fc890 msgr2=0x7efca40fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:43:27.091 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:27.092+0000 7efcab8ce700 1 -- 172.21.15.188:0/2712663079 shutdown_connections 2024-06-10T02:43:27.091 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:27.092+0000 7efcab8ce700 1 -- 172.21.15.188:0/2712663079 wait complete. 2024-06-10T02:43:27.091 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:27.092+0000 7efcab8ce700 1 Processor -- start 2024-06-10T02:43:27.091 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:27.092+0000 7efcab8ce700 1 -- start start 2024-06-10T02:43:27.091 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:27.093+0000 7efcab8ce700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efca4100fe0 0x7efca419d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:43:27.091 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:27.093+0000 7efcab8ce700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7efca410b3b0 con 0x7efca4100fe0 2024-06-10T02:43:27.091 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:27.093+0000 7efca966a700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efca4100fe0 0x7efca419d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:43:27.091 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:27.093+0000 7efca966a700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efca4100fe0 0x7efca419d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.188:56254/0 (socket says 172.21.15.188:56254) 2024-06-10T02:43:27.091 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:27.093+0000 7efca966a700 1 -- 172.21.15.188:0/3178890066 learned_addr learned my addr 172.21.15.188:0/3178890066 (peer_addr_for_me v2:172.21.15.188:0/0) 2024-06-10T02:43:27.092 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:27.093+0000 7efca966a700 1 -- 172.21.15.188:0/3178890066 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efca419dba0 con 0x7efca4100fe0 2024-06-10T02:43:27.092 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:27.094+0000 7efca966a700 1 --2- 172.21.15.188:0/3178890066 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efca4100fe0 0x7efca419d660 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7efca0005850 tx=0x7efca0005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:43:27.093 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:27.094+0000 7efc9a7fc700 1 -- 172.21.15.188:0/3178890066 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efca0026070 con 0x7efca4100fe0 2024-06-10T02:43:27.093 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:27.094+0000 7efc9a7fc700 1 -- 172.21.15.188:0/3178890066 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efca000bbd0 con 0x7efca4100fe0 2024-06-10T02:43:27.093 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:27.094+0000 7efcab8ce700 1 -- 172.21.15.188:0/3178890066 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efca419de90 con 0x7efca4100fe0 2024-06-10T02:43:27.093 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:27.094+0000 7efc9a7fc700 1 -- 172.21.15.188:0/3178890066 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efca0003970 con 0x7efca4100fe0 2024-06-10T02:43:27.093 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:27.094+0000 7efcab8ce700 1 -- 172.21.15.188:0/3178890066 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efca41977e0 con 0x7efca4100fe0 2024-06-10T02:43:27.093 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:27.095+0000 7efcab8ce700 1 -- 172.21.15.188:0/3178890066 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efca41024c0 con 0x7efca4100fe0 2024-06-10T02:43:27.093 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:27.095+0000 7efc9a7fc700 1 -- 172.21.15.188:0/3178890066 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7efca0029070 con 0x7efca4100fe0 2024-06-10T02:43:27.093 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:27.095+0000 7efc9a7fc700 1 --2- 172.21.15.188:0/3178890066 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7efc900390f0 0x7efc9003b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:43:27.094 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:27.095+0000 7efc9a7fc700 1 -- 172.21.15.188:0/3178890066 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7efca001e070 con 0x7efca4100fe0 2024-06-10T02:43:27.094 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:27.095+0000 7efca8e69700 1 --2- 172.21.15.188:0/3178890066 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7efc900390f0 0x7efc9003b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:43:27.094 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:27.096+0000 7efca8e69700 1 --2- 172.21.15.188:0/3178890066 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7efc900390f0 0x7efc9003b5b0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7efc94006fd0 tx=0x7efc94006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:43:27.098 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:27.101+0000 7efc9a7fc700 1 -- 172.21.15.188:0/3178890066 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7efca0013070 con 0x7efca4100fe0 2024-06-10T02:43:27.392 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:27.394+0000 7efcab8ce700 1 -- 172.21.15.188:0/3178890066 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7efca4198d10 con 0x7efca4100fe0 2024-06-10T02:43:27.392 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:27.395+0000 7efc9a7fc700 1 -- 172.21.15.188:0/3178890066 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+745 (secure 0 0 0) 0x7efca0030020 con 0x7efca4100fe0 2024-06-10T02:43:27.392 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-06-10T02:43:27.393 INFO:teuthology.orchestra.run.smithi188.stdout:{"epoch":1,"fsid":"d8676e9e-26d2-11ef-bc9b-c7b262605968","modified":"2024-06-10T02:41:40.094306Z","created":"2024-06-10T02:41:40.094306Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:3300","nonce":0},{"type":"v1","addr":"172.21.15.52:6789","nonce":0}]},"addr":"172.21.15.52:6789/0","public_addr":"172.21.15.52:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-06-10T02:43:27.395 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:27.398+0000 7efcab8ce700 1 -- 172.21.15.188:0/3178890066 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7efc900390f0 msgr2=0x7efc9003b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:43:27.395 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:27.398+0000 7efcab8ce700 1 --2- 172.21.15.188:0/3178890066 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7efc900390f0 0x7efc9003b5b0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7efc94006fd0 tx=0x7efc94006e40 comp rx=0 tx=0).stop 2024-06-10T02:43:27.395 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:27.398+0000 7efcab8ce700 1 -- 172.21.15.188:0/3178890066 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efca4100fe0 msgr2=0x7efca419d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:43:27.395 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:27.398+0000 7efcab8ce700 1 --2- 172.21.15.188:0/3178890066 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efca4100fe0 0x7efca419d660 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7efca0005850 tx=0x7efca0005030 comp rx=0 tx=0).stop 2024-06-10T02:43:27.395 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:27.398+0000 7efcab8ce700 1 -- 172.21.15.188:0/3178890066 shutdown_connections 2024-06-10T02:43:27.395 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:27.398+0000 7efcab8ce700 1 --2- 172.21.15.188:0/3178890066 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7efc900390f0 0x7efc9003b5b0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:43:27.395 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:27.398+0000 7efcab8ce700 1 --2- 172.21.15.188:0/3178890066 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efca4100fe0 0x7efca419d660 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:43:27.396 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:27.398+0000 7efcab8ce700 1 -- 172.21.15.188:0/3178890066 >> 172.21.15.188:0/3178890066 conn(0x7efca40fc890 msgr2=0x7efca406a360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:43:27.396 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:27.398+0000 7efcab8ce700 1 -- 172.21.15.188:0/3178890066 shutdown_connections 2024-06-10T02:43:27.396 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:27.398+0000 7efcab8ce700 1 -- 172.21.15.188:0/3178890066 wait complete. 2024-06-10T02:43:27.396 INFO:teuthology.orchestra.run.smithi188.stderr:dumped monmap epoch 1 2024-06-10T02:43:27.973 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:27 smithi052 ceph-mon[32362]: from='client.? 172.21.15.188:0/3178890066' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-06-10T02:43:29.199 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-06-10T02:43:29.200 DEBUG:teuthology.orchestra.run.smithi188:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph mon dump -f json 2024-06-10T02:43:29.443 INFO:teuthology.orchestra.run.smithi188.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.c/config 2024-06-10T02:43:33.200 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:43:32 smithi188 ceph-mon[36599]: Deploying daemon mon.b on smithi120 2024-06-10T02:43:33.223 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:32 smithi052 ceph-mon[32362]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T02:43:33.224 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:32 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-10T02:43:33.224 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:32 smithi052 ceph-mon[32362]: mon.a calling monitor election 2024-06-10T02:43:33.224 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:32 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-10T02:43:33.224 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:32 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-10T02:43:33.224 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:32 smithi052 ceph-mon[32362]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T02:43:33.224 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:32 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-10T02:43:33.224 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:32 smithi052 ceph-mon[32362]: mon.c calling monitor election 2024-06-10T02:43:33.224 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:32 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-10T02:43:33.224 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:32 smithi052 ceph-mon[32362]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T02:43:33.224 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:32 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-10T02:43:33.224 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:32 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-10T02:43:33.224 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:32 smithi052 ceph-mon[32362]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-06-10T02:43:33.224 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:32 smithi052 ceph-mon[32362]: monmap epoch 2 2024-06-10T02:43:33.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:32 smithi052 ceph-mon[32362]: fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 2024-06-10T02:43:33.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:32 smithi052 ceph-mon[32362]: last_changed 2024-06-10T02:43:27.712881+0000 2024-06-10T02:43:33.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:32 smithi052 ceph-mon[32362]: created 2024-06-10T02:41:40.094306+0000 2024-06-10T02:43:33.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:32 smithi052 ceph-mon[32362]: min_mon_release 19 (squid) 2024-06-10T02:43:33.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:32 smithi052 ceph-mon[32362]: election_strategy: 1 2024-06-10T02:43:33.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:32 smithi052 ceph-mon[32362]: 0: [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] mon.a 2024-06-10T02:43:33.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:32 smithi052 ceph-mon[32362]: 1: [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] mon.c 2024-06-10T02:43:33.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:32 smithi052 ceph-mon[32362]: fsmap 2024-06-10T02:43:33.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:32 smithi052 ceph-mon[32362]: osdmap e4: 0 total, 0 up, 0 in 2024-06-10T02:43:33.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:32 smithi052 ceph-mon[32362]: mgrmap e13: x(active, since 49s) 2024-06-10T02:43:33.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:32 smithi052 ceph-mon[32362]: overall HEALTH_OK 2024-06-10T02:43:33.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:32 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:43:33.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:32 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:43:33.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:32 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:43:33.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:32 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-10T02:43:33.226 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:32 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:43:33.226 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:32 smithi052 ceph-mon[32362]: Deploying daemon mon.b on smithi120 2024-06-10T02:43:34.027 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:34.030+0000 7f76bd92f700 1 -- 172.21.15.188:0/2047537493 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f76b8101100 msgr2=0x7f76a4005610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:43:34.027 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:34.030+0000 7f76bd92f700 1 --2- 172.21.15.188:0/2047537493 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f76b8101100 0x7f76a4005610 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f76a8005fa0 tx=0x7f76a800ffc0 comp rx=0 tx=0).stop 2024-06-10T02:43:34.027 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:34.030+0000 7f76bd92f700 1 -- 172.21.15.188:0/2047537493 shutdown_connections 2024-06-10T02:43:34.027 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:34.030+0000 7f76bd92f700 1 --2- 172.21.15.188:0/2047537493 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f76b8101100 0x7f76a4005610 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:43:34.027 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:34.030+0000 7f76bd92f700 1 -- 172.21.15.188:0/2047537493 >> 172.21.15.188:0/2047537493 conn(0x7f76b80fc9d0 msgr2=0x7f76b80fedf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:43:34.027 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:34.030+0000 7f76bd92f700 1 -- 172.21.15.188:0/2047537493 shutdown_connections 2024-06-10T02:43:34.028 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:34.030+0000 7f76bd92f700 1 -- 172.21.15.188:0/2047537493 wait complete. 2024-06-10T02:43:34.028 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:34.031+0000 7f76bd92f700 1 Processor -- start 2024-06-10T02:43:34.028 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:34.031+0000 7f76bd92f700 1 -- start start 2024-06-10T02:43:34.029 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:34.031+0000 7f76bd92f700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f76b8101100 0x7f76b8198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:43:34.029 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:34.031+0000 7f76bd92f700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f76b8198db0 0x7f76b8199230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:43:34.029 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:34.031+0000 7f76bd92f700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f76b8108f30 con 0x7f76b8101100 2024-06-10T02:43:34.030 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:34.031+0000 7f76bd92f700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f76b8108db0 con 0x7f76b8198db0 2024-06-10T02:43:34.030 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:34.031+0000 7f76affff700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f76b8198db0 0x7f76b8199230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:43:34.030 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:34.032+0000 7f76b6ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f76b8101100 0x7f76b8198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:43:34.030 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:34.032+0000 7f76affff700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f76b8198db0 0x7f76b8199230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.188:55548/0 (socket says 172.21.15.188:55548) 2024-06-10T02:43:34.030 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:34.032+0000 7f76affff700 1 -- 172.21.15.188:0/1226306924 learned_addr learned my addr 172.21.15.188:0/1226306924 (peer_addr_for_me v2:172.21.15.188:0/0) 2024-06-10T02:43:34.030 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:34.032+0000 7f76b6ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f76b8101100 0x7f76b8198870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.188:58396/0 (socket says 172.21.15.188:58396) 2024-06-10T02:43:34.030 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:34.032+0000 7f76affff700 1 -- 172.21.15.188:0/1226306924 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f76b8198db0 msgr2=0x7f76b8199230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-06-10T02:43:34.030 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:34.032+0000 7f76affff700 1 -- 172.21.15.188:0/1226306924 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f76b8198db0 msgr2=0x7f76b8199230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-06-10T02:43:34.030 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:34.032+0000 7f76affff700 1 --2- 172.21.15.188:0/1226306924 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f76b8198db0 0x7f76b8199230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-06-10T02:43:34.030 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:34.032+0000 7f76affff700 1 --2- 172.21.15.188:0/1226306924 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f76b8198db0 0x7f76b8199230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-06-10T02:43:34.030 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:34.032+0000 7f76b6ffd700 1 -- 172.21.15.188:0/1226306924 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f76b8198db0 msgr2=0x7f76b8199230 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-10T02:43:34.031 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:34.032+0000 7f76b6ffd700 1 --2- 172.21.15.188:0/1226306924 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f76b8198db0 0x7f76b8199230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:43:34.031 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:34.032+0000 7f76b6ffd700 1 -- 172.21.15.188:0/1226306924 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76b81a18e0 con 0x7f76b8101100 2024-06-10T02:43:34.031 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:34.032+0000 7f76b6ffd700 1 --2- 172.21.15.188:0/1226306924 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f76b8101100 0x7f76b8198870 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f76a8000c00 tx=0x7f76a8004d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:43:34.031 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:34.032+0000 7f76bc92d700 1 -- 172.21.15.188:0/1226306924 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f76a80053b0 con 0x7f76b8101100 2024-06-10T02:43:34.031 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:34.032+0000 7f76bc92d700 1 -- 172.21.15.188:0/1226306924 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76a8005550 con 0x7f76b8101100 2024-06-10T02:43:34.031 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:34.032+0000 7f76bd92f700 1 -- 172.21.15.188:0/1226306924 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76b81a1ab0 con 0x7f76b8101100 2024-06-10T02:43:34.031 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:34.032+0000 7f76bc92d700 1 -- 172.21.15.188:0/1226306924 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f76a8011670 con 0x7f76b8101100 2024-06-10T02:43:34.031 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:34.032+0000 7f76bd92f700 1 -- 172.21.15.188:0/1226306924 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76b81a2010 con 0x7f76b8101100 2024-06-10T02:43:34.031 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:34.033+0000 7f76bc92d700 1 -- 172.21.15.188:0/1226306924 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f76a8011810 con 0x7f76b8101100 2024-06-10T02:43:34.032 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:34.034+0000 7f76bd92f700 1 -- 172.21.15.188:0/1226306924 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76b81025e0 con 0x7f76b8101100 2024-06-10T02:43:34.032 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:34.034+0000 7f76bc92d700 1 --2- 172.21.15.188:0/1226306924 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f7698038f00 0x7f769803b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:43:34.032 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:34.034+0000 7f76bc92d700 1 -- 172.21.15.188:0/1226306924 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f76a804e7e0 con 0x7f76b8101100 2024-06-10T02:43:34.032 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:34.034+0000 7f76affff700 1 --2- 172.21.15.188:0/1226306924 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f7698038f00 0x7f769803b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:43:34.032 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:34.034+0000 7f76affff700 1 --2- 172.21.15.188:0/1226306924 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f7698038f00 0x7f769803b3c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f76a0009180 tx=0x7f76a0008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:43:34.032 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:43:33 smithi188 ceph-mon[36599]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T02:43:34.033 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:43:33 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-10T02:43:34.035 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:34.039+0000 7f76bc92d700 1 -- 172.21.15.188:0/1226306924 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f76a8022070 con 0x7f76b8101100 2024-06-10T02:43:34.223 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:33 smithi052 ceph-mon[32362]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T02:43:34.223 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:33 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-10T02:43:34.312 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:34.316+0000 7f76bd92f700 1 -- 172.21.15.188:0/1226306924 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f76b802dca0 con 0x7f76b8101100 2024-06-10T02:43:34.312 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:34.316+0000 7f76bc92d700 1 -- 172.21.15.188:0/1226306924 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7f76a8019bf0 con 0x7f76b8101100 2024-06-10T02:43:34.312 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-06-10T02:43:34.312 INFO:teuthology.orchestra.run.smithi188.stdout:{"epoch":2,"fsid":"d8676e9e-26d2-11ef-bc9b-c7b262605968","modified":"2024-06-10T02:43:27.712881Z","created":"2024-06-10T02:41:40.094306Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:3300","nonce":0},{"type":"v1","addr":"172.21.15.52:6789","nonce":0}]},"addr":"172.21.15.52:6789/0","public_addr":"172.21.15.52:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:3300","nonce":0},{"type":"v1","addr":"172.21.15.188:6789","nonce":0}]},"addr":"172.21.15.188:6789/0","public_addr":"172.21.15.188:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-06-10T02:43:34.315 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:34.318+0000 7f76bd92f700 1 -- 172.21.15.188:0/1226306924 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f7698038f00 msgr2=0x7f769803b3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:43:34.315 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:34.318+0000 7f76bd92f700 1 --2- 172.21.15.188:0/1226306924 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f7698038f00 0x7f769803b3c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f76a0009180 tx=0x7f76a0008040 comp rx=0 tx=0).stop 2024-06-10T02:43:34.315 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:34.318+0000 7f76bd92f700 1 -- 172.21.15.188:0/1226306924 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f76b8101100 msgr2=0x7f76b8198870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:43:34.315 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:34.318+0000 7f76bd92f700 1 --2- 172.21.15.188:0/1226306924 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f76b8101100 0x7f76b8198870 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f76a8000c00 tx=0x7f76a8004d60 comp rx=0 tx=0).stop 2024-06-10T02:43:34.315 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:34.318+0000 7f76bd92f700 1 -- 172.21.15.188:0/1226306924 shutdown_connections 2024-06-10T02:43:34.315 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:34.318+0000 7f76bd92f700 1 --2- 172.21.15.188:0/1226306924 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f7698038f00 0x7f769803b3c0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:43:34.315 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:34.318+0000 7f76bd92f700 1 --2- 172.21.15.188:0/1226306924 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f76b8101100 0x7f76b8198870 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:43:34.315 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:34.318+0000 7f76bd92f700 1 --2- 172.21.15.188:0/1226306924 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f76b8198db0 0x7f76b8199230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:43:34.315 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:34.318+0000 7f76bd92f700 1 -- 172.21.15.188:0/1226306924 >> 172.21.15.188:0/1226306924 conn(0x7f76b80fc9d0 msgr2=0x7f76b80ffe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:43:34.315 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:34.318+0000 7f76bd92f700 1 -- 172.21.15.188:0/1226306924 shutdown_connections 2024-06-10T02:43:34.315 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:34.318+0000 7f76bd92f700 1 -- 172.21.15.188:0/1226306924 wait complete. 2024-06-10T02:43:34.316 INFO:teuthology.orchestra.run.smithi188.stderr:dumped monmap epoch 2 2024-06-10T02:43:34.942 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:43:34 smithi188 ceph-mon[36599]: from='client.? 172.21.15.188:0/1226306924' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-06-10T02:43:35.223 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:34 smithi052 ceph-mon[32362]: from='client.? 172.21.15.188:0/1226306924' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-06-10T02:43:35.970 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-06-10T02:43:35.971 DEBUG:teuthology.orchestra.run.smithi188:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph mon dump -f json 2024-06-10T02:43:36.200 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:43:35 smithi188 ceph-mon[36599]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T02:43:36.213 INFO:teuthology.orchestra.run.smithi188.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.c/config 2024-06-10T02:43:36.223 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:35 smithi052 ceph-mon[32362]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T02:43:42.004 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:42.008+0000 7f376b91d700 1 -- 172.21.15.188:0/2959022573 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f37500035c0 msgr2=0x7f3750005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:43:42.005 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:42.008+0000 7f376b91d700 1 --2- 172.21.15.188:0/2959022573 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f37500035c0 0x7f3750005a50 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f3764101ee0 tx=0x7f375c009e80 comp rx=0 tx=0).stop 2024-06-10T02:43:42.005 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:42.008+0000 7f376b91d700 1 -- 172.21.15.188:0/2959022573 shutdown_connections 2024-06-10T02:43:42.005 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:42.008+0000 7f376b91d700 1 --2- 172.21.15.188:0/2959022573 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f37500035c0 0x7f3750005a50 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:43:42.005 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:42.008+0000 7f376b91d700 1 -- 172.21.15.188:0/2959022573 >> 172.21.15.188:0/2959022573 conn(0x7f37640fdaa0 msgr2=0x7f37640ffec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:43:42.005 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:42.008+0000 7f376b91d700 1 -- 172.21.15.188:0/2959022573 shutdown_connections 2024-06-10T02:43:42.005 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:42.008+0000 7f376b91d700 1 -- 172.21.15.188:0/2959022573 wait complete. 2024-06-10T02:43:42.007 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:42.009+0000 7f376b91d700 1 Processor -- start 2024-06-10T02:43:42.007 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:42.009+0000 7f376b91d700 1 -- start start 2024-06-10T02:43:42.007 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:42.009+0000 7f376b91d700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f37500035c0 0x7f3764074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:43:42.007 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:42.009+0000 7f376b91d700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3764074cd0 0x7f3764075150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:43:42.007 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:42.009+0000 7f376b91d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f37641a3d70 0x7f37641a6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:43:42.007 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:42.009+0000 7f376b91d700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f3764109ff0 con 0x7f37641a3d70 2024-06-10T02:43:42.008 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:42.009+0000 7f376b91d700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f3764109e70 con 0x7f37500035c0 2024-06-10T02:43:42.008 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:42.009+0000 7f376b91d700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f376410a170 con 0x7f3764074cd0 2024-06-10T02:43:42.008 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:42.009+0000 7f3768eb8700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3764074cd0 0x7f3764075150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:43:42.008 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:42.009+0000 7f3769eba700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f37641a3d70 0x7f37641a6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:43:42.008 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:42.009+0000 7f3768eb8700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3764074cd0 0x7f3764075150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.188:37822/0 (socket says 172.21.15.188:37822) 2024-06-10T02:43:42.008 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:42.009+0000 7f3768eb8700 1 -- 172.21.15.188:0/2524694335 learned_addr learned my addr 172.21.15.188:0/2524694335 (peer_addr_for_me v2:172.21.15.188:0/0) 2024-06-10T02:43:42.008 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:42.009+0000 7f3769eba700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f37641a3d70 0x7f37641a6160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.188:49886/0 (socket says 172.21.15.188:49886) 2024-06-10T02:43:42.008 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:42.009+0000 7f37696b9700 1 --2- 172.21.15.188:0/2524694335 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f37500035c0 0x7f3764074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:43:42.008 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:42.009+0000 7f3768eb8700 1 -- 172.21.15.188:0/2524694335 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3764074cd0 msgr2=0x7f3764075150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-06-10T02:43:42.008 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:42.009+0000 7f3768eb8700 1 -- 172.21.15.188:0/2524694335 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3764074cd0 msgr2=0x7f3764075150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-06-10T02:43:42.008 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:42.009+0000 7f3768eb8700 1 --2- 172.21.15.188:0/2524694335 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3764074cd0 0x7f3764075150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-06-10T02:43:42.008 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:42.009+0000 7f37696b9700 1 -- 172.21.15.188:0/2524694335 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f37500035c0 msgr2=0x7f3764074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-06-10T02:43:42.009 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:42.009+0000 7f37696b9700 1 -- 172.21.15.188:0/2524694335 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f37500035c0 msgr2=0x7f3764074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-06-10T02:43:42.009 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:42.009+0000 7f37696b9700 1 --2- 172.21.15.188:0/2524694335 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f37500035c0 0x7f3764074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-06-10T02:43:42.009 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:42.010+0000 7f37696b9700 1 --2- 172.21.15.188:0/2524694335 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f37500035c0 0x7f3764074790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-06-10T02:43:42.009 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:42.010+0000 7f3768eb8700 1 --2- 172.21.15.188:0/2524694335 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3764074cd0 0x7f3764075150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-06-10T02:43:42.009 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:42.010+0000 7f3769eba700 1 -- 172.21.15.188:0/2524694335 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3764074cd0 msgr2=0x7f3764075150 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-10T02:43:42.009 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:42.010+0000 7f3769eba700 1 --2- 172.21.15.188:0/2524694335 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3764074cd0 0x7f3764075150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:43:42.009 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:42.010+0000 7f3769eba700 1 -- 172.21.15.188:0/2524694335 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f37500035c0 msgr2=0x7f3764074790 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-10T02:43:42.009 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:42.010+0000 7f3769eba700 1 --2- 172.21.15.188:0/2524694335 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f37500035c0 0x7f3764074790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:43:42.009 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:42.010+0000 7f3769eba700 1 -- 172.21.15.188:0/2524694335 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37641a6890 con 0x7f37641a3d70 2024-06-10T02:43:42.009 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:42.010+0000 7f3769eba700 1 --2- 172.21.15.188:0/2524694335 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f37641a3d70 0x7f37641a6160 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f3760006fd0 tx=0x7f376000b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:43:42.009 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:42.010+0000 7f375a7fc700 1 -- 172.21.15.188:0/2524694335 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3760015020 con 0x7f37641a3d70 2024-06-10T02:43:42.009 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:42.010+0000 7f375a7fc700 1 -- 172.21.15.188:0/2524694335 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3760011d10 con 0x7f37641a3d70 2024-06-10T02:43:42.009 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:42.010+0000 7f375a7fc700 1 -- 172.21.15.188:0/2524694335 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3760010e10 con 0x7f37641a3d70 2024-06-10T02:43:42.009 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:42.010+0000 7f376b91d700 1 -- 172.21.15.188:0/2524694335 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37641a6b80 con 0x7f37641a3d70 2024-06-10T02:43:42.010 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:42.010+0000 7f376b91d700 1 -- 172.21.15.188:0/2524694335 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37641a7070 con 0x7f37641a3d70 2024-06-10T02:43:42.010 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:42.011+0000 7f375a7fc700 1 -- 172.21.15.188:0/2524694335 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f3760018020 con 0x7f37641a3d70 2024-06-10T02:43:42.010 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:42.011+0000 7f375a7fc700 1 --2- 172.21.15.188:0/2524694335 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f3748039580 0x7f374803ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:43:42.010 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:42.011+0000 7f375a7fc700 1 -- 172.21.15.188:0/2524694335 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f376004bc30 con 0x7f37641a3d70 2024-06-10T02:43:42.010 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:42.012+0000 7f37696b9700 1 --2- 172.21.15.188:0/2524694335 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f3748039580 0x7f374803ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:43:42.011 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:42.012+0000 7f376b91d700 1 -- 172.21.15.188:0/2524694335 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3764063af0 con 0x7f37641a3d70 2024-06-10T02:43:42.011 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:42.012+0000 7f37696b9700 1 --2- 172.21.15.188:0/2524694335 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f3748039580 0x7f374803ba40 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f375c009a30 tx=0x7f375c00b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:43:42.013 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:42.017+0000 7f375a7fc700 1 -- 172.21.15.188:0/2524694335 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f376001f020 con 0x7f37641a3d70 2024-06-10T02:43:42.200 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:43:41 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-10T02:43:42.200 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:43:41 smithi188 ceph-mon[36599]: mon.c calling monitor election 2024-06-10T02:43:42.201 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:43:41 smithi188 ceph-mon[36599]: mon.a calling monitor election 2024-06-10T02:43:42.201 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:43:41 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-10T02:43:42.201 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:43:41 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-10T02:43:42.201 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:43:41 smithi188 ceph-mon[36599]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T02:43:42.201 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:43:41 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-10T02:43:42.201 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:43:41 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-10T02:43:42.201 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:43:41 smithi188 ceph-mon[36599]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T02:43:42.201 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:43:41 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-10T02:43:42.201 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:43:41 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-10T02:43:42.201 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:43:41 smithi188 ceph-mon[36599]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T02:43:42.201 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:43:41 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-10T02:43:42.201 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:43:41 smithi188 ceph-mon[36599]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-06-10T02:43:42.201 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:43:41 smithi188 ceph-mon[36599]: monmap epoch 3 2024-06-10T02:43:42.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:43:41 smithi188 ceph-mon[36599]: fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 2024-06-10T02:43:42.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:43:41 smithi188 ceph-mon[36599]: last_changed 2024-06-10T02:43:36.729060+0000 2024-06-10T02:43:42.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:43:41 smithi188 ceph-mon[36599]: created 2024-06-10T02:41:40.094306+0000 2024-06-10T02:43:42.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:43:41 smithi188 ceph-mon[36599]: min_mon_release 19 (squid) 2024-06-10T02:43:42.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:43:41 smithi188 ceph-mon[36599]: election_strategy: 1 2024-06-10T02:43:42.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:43:41 smithi188 ceph-mon[36599]: 0: [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] mon.a 2024-06-10T02:43:42.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:43:41 smithi188 ceph-mon[36599]: 1: [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] mon.c 2024-06-10T02:43:42.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:43:41 smithi188 ceph-mon[36599]: 2: [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] mon.b 2024-06-10T02:43:42.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:43:41 smithi188 ceph-mon[36599]: fsmap 2024-06-10T02:43:42.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:43:41 smithi188 ceph-mon[36599]: osdmap e4: 0 total, 0 up, 0 in 2024-06-10T02:43:42.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:43:41 smithi188 ceph-mon[36599]: mgrmap e13: x(active, since 58s) 2024-06-10T02:43:42.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:43:41 smithi188 ceph-mon[36599]: overall HEALTH_OK 2024-06-10T02:43:42.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:43:41 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:43:42.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:43:41 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:43:42.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:43:41 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:43:42.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:43:41 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:43:42.223 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:41 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-10T02:43:42.224 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:41 smithi052 ceph-mon[32362]: mon.c calling monitor election 2024-06-10T02:43:42.224 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:41 smithi052 ceph-mon[32362]: mon.a calling monitor election 2024-06-10T02:43:42.224 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:41 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-10T02:43:42.224 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:41 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-10T02:43:42.224 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:41 smithi052 ceph-mon[32362]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T02:43:42.224 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:41 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-10T02:43:42.224 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:41 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-10T02:43:42.224 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:41 smithi052 ceph-mon[32362]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T02:43:42.224 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:41 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-10T02:43:42.224 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:41 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-10T02:43:42.224 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:41 smithi052 ceph-mon[32362]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T02:43:42.224 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:41 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-10T02:43:42.224 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:41 smithi052 ceph-mon[32362]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-06-10T02:43:42.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:41 smithi052 ceph-mon[32362]: monmap epoch 3 2024-06-10T02:43:42.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:41 smithi052 ceph-mon[32362]: fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 2024-06-10T02:43:42.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:41 smithi052 ceph-mon[32362]: last_changed 2024-06-10T02:43:36.729060+0000 2024-06-10T02:43:42.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:41 smithi052 ceph-mon[32362]: created 2024-06-10T02:41:40.094306+0000 2024-06-10T02:43:42.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:41 smithi052 ceph-mon[32362]: min_mon_release 19 (squid) 2024-06-10T02:43:42.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:41 smithi052 ceph-mon[32362]: election_strategy: 1 2024-06-10T02:43:42.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:41 smithi052 ceph-mon[32362]: 0: [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] mon.a 2024-06-10T02:43:42.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:41 smithi052 ceph-mon[32362]: 1: [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] mon.c 2024-06-10T02:43:42.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:41 smithi052 ceph-mon[32362]: 2: [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] mon.b 2024-06-10T02:43:42.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:41 smithi052 ceph-mon[32362]: fsmap 2024-06-10T02:43:42.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:41 smithi052 ceph-mon[32362]: osdmap e4: 0 total, 0 up, 0 in 2024-06-10T02:43:42.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:41 smithi052 ceph-mon[32362]: mgrmap e13: x(active, since 58s) 2024-06-10T02:43:42.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:41 smithi052 ceph-mon[32362]: overall HEALTH_OK 2024-06-10T02:43:42.226 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:41 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:43:42.226 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:41 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:43:42.226 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:41 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:43:42.226 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:41 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:43:42.301 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:42.304+0000 7f376b91d700 1 -- 172.21.15.188:0/2524694335 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f376402dca0 con 0x7f37641a3d70 2024-06-10T02:43:42.301 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:42.305+0000 7f375a7fc700 1 -- 172.21.15.188:0/2524694335 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1283 (secure 0 0 0) 0x7f3760018310 con 0x7f37641a3d70 2024-06-10T02:43:42.301 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-06-10T02:43:42.301 INFO:teuthology.orchestra.run.smithi188.stdout:{"epoch":3,"fsid":"d8676e9e-26d2-11ef-bc9b-c7b262605968","modified":"2024-06-10T02:43:36.729060Z","created":"2024-06-10T02:41:40.094306Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:3300","nonce":0},{"type":"v1","addr":"172.21.15.52:6789","nonce":0}]},"addr":"172.21.15.52:6789/0","public_addr":"172.21.15.52:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:3300","nonce":0},{"type":"v1","addr":"172.21.15.188:6789","nonce":0}]},"addr":"172.21.15.188:6789/0","public_addr":"172.21.15.188:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:3300","nonce":0},{"type":"v1","addr":"172.21.15.120:6789","nonce":0}]},"addr":"172.21.15.120:6789/0","public_addr":"172.21.15.120:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-06-10T02:43:42.304 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:42.307+0000 7f376b91d700 1 -- 172.21.15.188:0/2524694335 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f3748039580 msgr2=0x7f374803ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:43:42.304 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:42.307+0000 7f376b91d700 1 --2- 172.21.15.188:0/2524694335 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f3748039580 0x7f374803ba40 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f375c009a30 tx=0x7f375c00b040 comp rx=0 tx=0).stop 2024-06-10T02:43:42.304 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:42.308+0000 7f376b91d700 1 -- 172.21.15.188:0/2524694335 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f37641a3d70 msgr2=0x7f37641a6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:43:42.304 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:42.308+0000 7f376b91d700 1 --2- 172.21.15.188:0/2524694335 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f37641a3d70 0x7f37641a6160 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f3760006fd0 tx=0x7f376000b7e0 comp rx=0 tx=0).stop 2024-06-10T02:43:42.304 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:42.308+0000 7f376b91d700 1 -- 172.21.15.188:0/2524694335 shutdown_connections 2024-06-10T02:43:42.304 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:42.308+0000 7f376b91d700 1 --2- 172.21.15.188:0/2524694335 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f3748039580 0x7f374803ba40 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:43:42.304 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:42.308+0000 7f376b91d700 1 --2- 172.21.15.188:0/2524694335 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f37641a3d70 0x7f37641a6160 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:43:42.304 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:42.308+0000 7f376b91d700 1 --2- 172.21.15.188:0/2524694335 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f37500035c0 0x7f3764074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:43:42.305 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:42.308+0000 7f376b91d700 1 --2- 172.21.15.188:0/2524694335 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3764074cd0 0x7f3764075150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:43:42.305 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:42.308+0000 7f376b91d700 1 -- 172.21.15.188:0/2524694335 >> 172.21.15.188:0/2524694335 conn(0x7f37640fdaa0 msgr2=0x7f37640febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:43:42.305 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:42.308+0000 7f376b91d700 1 -- 172.21.15.188:0/2524694335 shutdown_connections 2024-06-10T02:43:42.305 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:42.308+0000 7f376b91d700 1 -- 172.21.15.188:0/2524694335 wait complete. 2024-06-10T02:43:42.305 INFO:teuthology.orchestra.run.smithi188.stderr:dumped monmap epoch 3 2024-06-10T02:43:42.870 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:43:42 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T02:43:42.870 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:43:42 smithi188 ceph-mon[36599]: from='client.? 172.21.15.188:0/2524694335' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-06-10T02:43:42.870 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:43:42 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-10T02:43:42.894 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-06-10T02:43:42.894 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph config generate-minimal-conf 2024-06-10T02:43:43.154 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:43:43.223 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:42 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T02:43:43.223 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:42 smithi052 ceph-mon[32362]: from='client.? 172.21.15.188:0/2524694335' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-06-10T02:43:43.223 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:42 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-10T02:43:44.135 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:43:43 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-10T02:43:44.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:44.770+0000 7f39951ff700 1 -- 172.21.15.52:0/1444286239 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3990101040 msgr2=0x7f3990101420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:43:44.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:44.770+0000 7f39951ff700 1 --2- 172.21.15.52:0/1444286239 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3990101040 0x7f3990101420 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f3980009910 tx=0x7f3980009c20 comp rx=0 tx=0).stop 2024-06-10T02:43:44.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:44.770+0000 7f39951ff700 1 -- 172.21.15.52:0/1444286239 shutdown_connections 2024-06-10T02:43:44.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:44.770+0000 7f39951ff700 1 --2- 172.21.15.52:0/1444286239 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3990101040 0x7f3990101420 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:43:44.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:44.770+0000 7f39951ff700 1 -- 172.21.15.52:0/1444286239 >> 172.21.15.52:0/1444286239 conn(0x7f39900fc8f0 msgr2=0x7f39900fed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:43:44.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:44.770+0000 7f39951ff700 1 -- 172.21.15.52:0/1444286239 shutdown_connections 2024-06-10T02:43:44.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:44.770+0000 7f39951ff700 1 -- 172.21.15.52:0/1444286239 wait complete. 2024-06-10T02:43:44.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:44.771+0000 7f39951ff700 1 Processor -- start 2024-06-10T02:43:44.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:44.771+0000 7f39951ff700 1 -- start start 2024-06-10T02:43:44.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:44.771+0000 7f39951ff700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3990101040 0x7f3990198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:43:44.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:44.771+0000 7f39951ff700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3990198ff0 0x7f3990199470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:43:44.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:44.771+0000 7f39951ff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f39901a1b20 0x7f39901a3f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:43:44.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:44.771+0000 7f39951ff700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f399010b3d0 con 0x7f39901a1b20 2024-06-10T02:43:44.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:44.771+0000 7f39951ff700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f399010b250 con 0x7f3990101040 2024-06-10T02:43:44.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:44.771+0000 7f39951ff700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f399010b550 con 0x7f3990198ff0 2024-06-10T02:43:44.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:44.771+0000 7f398f59e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f39901a1b20 0x7f39901a3f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:43:44.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:44.771+0000 7f398f59e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f39901a1b20 0x7f39901a3f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:33694/0 (socket says 172.21.15.52:33694) 2024-06-10T02:43:44.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:44.771+0000 7f398f59e700 1 -- 172.21.15.52:0/1695517455 learned_addr learned my addr 172.21.15.52:0/1695517455 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:43:44.770 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:44.771+0000 7f398ed9d700 1 --2- 172.21.15.52:0/1695517455 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3990101040 0x7f3990198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:43:44.770 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:44.771+0000 7f398e59c700 1 --2- 172.21.15.52:0/1695517455 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3990198ff0 0x7f3990199470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:43:44.770 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:44.772+0000 7f398f59e700 1 -- 172.21.15.52:0/1695517455 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3990198ff0 msgr2=0x7f3990199470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:43:44.770 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:44.772+0000 7f398f59e700 1 --2- 172.21.15.52:0/1695517455 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3990198ff0 0x7f3990199470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:43:44.770 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:44.772+0000 7f398f59e700 1 -- 172.21.15.52:0/1695517455 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3990101040 msgr2=0x7f3990198ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:43:44.770 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:44.772+0000 7f398f59e700 1 --2- 172.21.15.52:0/1695517455 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3990101040 0x7f3990198ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:43:44.770 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:44.772+0000 7f398f59e700 1 -- 172.21.15.52:0/1695517455 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39901a4750 con 0x7f39901a1b20 2024-06-10T02:43:44.770 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:44.772+0000 7f398ed9d700 1 --2- 172.21.15.52:0/1695517455 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3990101040 0x7f3990198ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T02:43:44.770 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:44.772+0000 7f398f59e700 1 --2- 172.21.15.52:0/1695517455 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f39901a1b20 0x7f39901a3f10 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f398800ba50 tx=0x7f398800bd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:43:44.770 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:44.772+0000 7f3977fff700 1 -- 172.21.15.52:0/1695517455 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3988019070 con 0x7f39901a1b20 2024-06-10T02:43:44.770 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:44.772+0000 7f39951ff700 1 -- 172.21.15.52:0/1695517455 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39901a4a40 con 0x7f39901a1b20 2024-06-10T02:43:44.770 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:44.772+0000 7f3977fff700 1 -- 172.21.15.52:0/1695517455 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f398800ce10 con 0x7f39901a1b20 2024-06-10T02:43:44.770 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:44.772+0000 7f3977fff700 1 -- 172.21.15.52:0/1695517455 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3988007350 con 0x7f39901a1b20 2024-06-10T02:43:44.771 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:44.772+0000 7f39951ff700 1 -- 172.21.15.52:0/1695517455 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39901a4f80 con 0x7f39901a1b20 2024-06-10T02:43:44.771 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:44.773+0000 7f3977fff700 1 -- 172.21.15.52:0/1695517455 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f398801c070 con 0x7f39901a1b20 2024-06-10T02:43:44.771 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:44.774+0000 7f3977fff700 1 --2- 172.21.15.52:0/1695517455 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f3978039580 0x7f397803ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:43:44.771 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:44.774+0000 7f39951ff700 1 -- 172.21.15.52:0/1695517455 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3990063af0 con 0x7f39901a1b20 2024-06-10T02:43:44.772 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:44.774+0000 7f3977fff700 1 -- 172.21.15.52:0/1695517455 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f398804b6b0 con 0x7f39901a1b20 2024-06-10T02:43:44.772 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:44.774+0000 7f398ed9d700 1 --2- 172.21.15.52:0/1695517455 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f3978039580 0x7f397803ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:43:44.772 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:44.774+0000 7f398ed9d700 1 --2- 172.21.15.52:0/1695517455 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f3978039580 0x7f397803ba40 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f3980009750 tx=0x7f398000b540 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:43:44.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:44.779+0000 7f3977fff700 1 -- 172.21.15.52:0/1695517455 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f3988012500 con 0x7f39901a1b20 2024-06-10T02:43:44.885 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:43:44 smithi120 ceph-mon[36634]: mon.b calling monitor election 2024-06-10T02:43:44.885 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:43:44 smithi120 ceph-mon[36634]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T02:43:44.885 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:43:44 smithi120 ceph-mon[36634]: Updating smithi052:/etc/ceph/ceph.conf 2024-06-10T02:43:44.886 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:43:44 smithi120 ceph-mon[36634]: Updating smithi120:/etc/ceph/ceph.conf 2024-06-10T02:43:44.886 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:43:44 smithi120 ceph-mon[36634]: Updating smithi188:/etc/ceph/ceph.conf 2024-06-10T02:43:44.886 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:43:44 smithi120 ceph-mon[36634]: mon.b calling monitor election 2024-06-10T02:43:44.886 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:43:44 smithi120 ceph-mon[36634]: mon.c calling monitor election 2024-06-10T02:43:44.886 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:43:44 smithi120 ceph-mon[36634]: mon.a calling monitor election 2024-06-10T02:43:44.886 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:43:44 smithi120 ceph-mon[36634]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-06-10T02:43:44.886 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:43:44 smithi120 ceph-mon[36634]: monmap epoch 3 2024-06-10T02:43:44.886 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:43:44 smithi120 ceph-mon[36634]: fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 2024-06-10T02:43:44.886 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:43:44 smithi120 ceph-mon[36634]: last_changed 2024-06-10T02:43:36.729060+0000 2024-06-10T02:43:44.886 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:43:44 smithi120 ceph-mon[36634]: created 2024-06-10T02:41:40.094306+0000 2024-06-10T02:43:44.886 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:43:44 smithi120 ceph-mon[36634]: min_mon_release 19 (squid) 2024-06-10T02:43:44.886 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:43:44 smithi120 ceph-mon[36634]: election_strategy: 1 2024-06-10T02:43:44.886 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:43:44 smithi120 ceph-mon[36634]: 0: [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] mon.a 2024-06-10T02:43:44.887 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:43:44 smithi120 ceph-mon[36634]: 1: [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] mon.c 2024-06-10T02:43:44.887 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:43:44 smithi120 ceph-mon[36634]: 2: [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] mon.b 2024-06-10T02:43:44.887 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:43:44 smithi120 ceph-mon[36634]: fsmap 2024-06-10T02:43:44.887 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:43:44 smithi120 ceph-mon[36634]: osdmap e4: 0 total, 0 up, 0 in 2024-06-10T02:43:44.887 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:43:44 smithi120 ceph-mon[36634]: mgrmap e13: x(active, since 60s) 2024-06-10T02:43:44.887 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:43:44 smithi120 ceph-mon[36634]: overall HEALTH_OK 2024-06-10T02:43:44.887 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:43:44 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:43:44.887 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:43:44 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:43:44.887 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:43:44 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:43:44.887 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:43:44 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:43:44.887 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:43:44 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:43:44.887 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:43:44 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:43:44.887 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:43:44 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:43:44.887 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:43:44 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:43:44.887 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:43:44 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:43:44.888 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:43:44 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:43:44.888 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:43:44 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:43:44.888 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:43:44 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-10T02:43:44.888 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:43:44 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-10T02:43:44.888 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:43:44 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:43:44.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:44 smithi052 ceph-mon[32362]: mon.b calling monitor election 2024-06-10T02:43:44.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:44 smithi052 ceph-mon[32362]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T02:43:44.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:44 smithi052 ceph-mon[32362]: Updating smithi052:/etc/ceph/ceph.conf 2024-06-10T02:43:44.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:44 smithi052 ceph-mon[32362]: Updating smithi120:/etc/ceph/ceph.conf 2024-06-10T02:43:44.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:44 smithi052 ceph-mon[32362]: Updating smithi188:/etc/ceph/ceph.conf 2024-06-10T02:43:44.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:44 smithi052 ceph-mon[32362]: mon.b calling monitor election 2024-06-10T02:43:44.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:44 smithi052 ceph-mon[32362]: mon.c calling monitor election 2024-06-10T02:43:44.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:44 smithi052 ceph-mon[32362]: mon.a calling monitor election 2024-06-10T02:43:44.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:44 smithi052 ceph-mon[32362]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-06-10T02:43:44.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:44 smithi052 ceph-mon[32362]: monmap epoch 3 2024-06-10T02:43:44.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:44 smithi052 ceph-mon[32362]: fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 2024-06-10T02:43:44.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:44 smithi052 ceph-mon[32362]: last_changed 2024-06-10T02:43:36.729060+0000 2024-06-10T02:43:44.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:44 smithi052 ceph-mon[32362]: created 2024-06-10T02:41:40.094306+0000 2024-06-10T02:43:44.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:44 smithi052 ceph-mon[32362]: min_mon_release 19 (squid) 2024-06-10T02:43:44.901 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:44 smithi052 ceph-mon[32362]: election_strategy: 1 2024-06-10T02:43:44.901 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:44 smithi052 ceph-mon[32362]: 0: [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] mon.a 2024-06-10T02:43:44.901 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:44 smithi052 ceph-mon[32362]: 1: [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] mon.c 2024-06-10T02:43:44.901 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:44 smithi052 ceph-mon[32362]: 2: [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] mon.b 2024-06-10T02:43:44.901 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:44 smithi052 ceph-mon[32362]: fsmap 2024-06-10T02:43:44.901 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:44 smithi052 ceph-mon[32362]: osdmap e4: 0 total, 0 up, 0 in 2024-06-10T02:43:44.901 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:44 smithi052 ceph-mon[32362]: mgrmap e13: x(active, since 60s) 2024-06-10T02:43:44.901 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:44 smithi052 ceph-mon[32362]: overall HEALTH_OK 2024-06-10T02:43:44.901 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:44 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:43:44.901 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:44 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:43:44.901 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:44 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:43:44.901 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:44 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:43:44.901 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:44 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:43:44.901 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:44 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:43:44.902 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:44 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:43:44.902 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:44 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:43:44.902 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:44 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:43:44.902 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:44 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:43:44.902 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:44 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:43:44.902 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:44 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-10T02:43:44.902 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:44 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-10T02:43:44.902 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:44 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:43:44.950 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:43:44 smithi188 ceph-mon[36599]: mon.b calling monitor election 2024-06-10T02:43:44.950 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:43:44 smithi188 ceph-mon[36599]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T02:43:44.950 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:43:44 smithi188 ceph-mon[36599]: Updating smithi052:/etc/ceph/ceph.conf 2024-06-10T02:43:44.951 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:43:44 smithi188 ceph-mon[36599]: Updating smithi120:/etc/ceph/ceph.conf 2024-06-10T02:43:44.951 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:43:44 smithi188 ceph-mon[36599]: Updating smithi188:/etc/ceph/ceph.conf 2024-06-10T02:43:44.951 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:43:44 smithi188 ceph-mon[36599]: mon.b calling monitor election 2024-06-10T02:43:44.951 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:43:44 smithi188 ceph-mon[36599]: mon.c calling monitor election 2024-06-10T02:43:44.951 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:43:44 smithi188 ceph-mon[36599]: mon.a calling monitor election 2024-06-10T02:43:44.951 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:43:44 smithi188 ceph-mon[36599]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-06-10T02:43:44.951 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:43:44 smithi188 ceph-mon[36599]: monmap epoch 3 2024-06-10T02:43:44.951 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:43:44 smithi188 ceph-mon[36599]: fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 2024-06-10T02:43:44.951 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:43:44 smithi188 ceph-mon[36599]: last_changed 2024-06-10T02:43:36.729060+0000 2024-06-10T02:43:44.951 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:43:44 smithi188 ceph-mon[36599]: created 2024-06-10T02:41:40.094306+0000 2024-06-10T02:43:44.951 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:43:44 smithi188 ceph-mon[36599]: min_mon_release 19 (squid) 2024-06-10T02:43:44.951 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:43:44 smithi188 ceph-mon[36599]: election_strategy: 1 2024-06-10T02:43:44.951 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:43:44 smithi188 ceph-mon[36599]: 0: [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] mon.a 2024-06-10T02:43:44.951 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:43:44 smithi188 ceph-mon[36599]: 1: [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] mon.c 2024-06-10T02:43:44.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:43:44 smithi188 ceph-mon[36599]: 2: [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] mon.b 2024-06-10T02:43:44.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:43:44 smithi188 ceph-mon[36599]: fsmap 2024-06-10T02:43:44.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:43:44 smithi188 ceph-mon[36599]: osdmap e4: 0 total, 0 up, 0 in 2024-06-10T02:43:44.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:43:44 smithi188 ceph-mon[36599]: mgrmap e13: x(active, since 60s) 2024-06-10T02:43:44.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:43:44 smithi188 ceph-mon[36599]: overall HEALTH_OK 2024-06-10T02:43:44.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:43:44 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:43:44.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:43:44 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:43:44.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:43:44 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:43:44.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:43:44 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:43:44.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:43:44 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:43:44.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:43:44 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:43:44.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:43:44 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:43:44.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:43:44 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:43:44.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:43:44 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:43:44.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:43:44 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:43:44.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:43:44 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:43:44.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:43:44 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-10T02:43:44.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:43:44 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-10T02:43:44.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:43:44 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:43:45.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:45.004+0000 7f39951ff700 1 -- 172.21.15.52:0/1695517455 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f39901a5260 con 0x7f39901a1b20 2024-06-10T02:43:45.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:45.004+0000 7f3977fff700 1 -- 172.21.15.52:0/1695517455 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+275 (secure 0 0 0) 0x7f3988014440 con 0x7f39901a1b20 2024-06-10T02:43:45.001 INFO:teuthology.orchestra.run.smithi052.stdout:# minimal ceph.conf for d8676e9e-26d2-11ef-bc9b-c7b262605968 2024-06-10T02:43:45.001 INFO:teuthology.orchestra.run.smithi052.stdout:[global] 2024-06-10T02:43:45.001 INFO:teuthology.orchestra.run.smithi052.stdout: fsid = d8676e9e-26d2-11ef-bc9b-c7b262605968 2024-06-10T02:43:45.001 INFO:teuthology.orchestra.run.smithi052.stdout: mon_host = [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] 2024-06-10T02:43:45.003 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:45.006+0000 7f39951ff700 1 -- 172.21.15.52:0/1695517455 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f3978039580 msgr2=0x7f397803ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:43:45.003 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:45.006+0000 7f39951ff700 1 --2- 172.21.15.52:0/1695517455 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f3978039580 0x7f397803ba40 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f3980009750 tx=0x7f398000b540 comp rx=0 tx=0).stop 2024-06-10T02:43:45.003 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:45.006+0000 7f39951ff700 1 -- 172.21.15.52:0/1695517455 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f39901a1b20 msgr2=0x7f39901a3f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:43:45.003 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:45.006+0000 7f39951ff700 1 --2- 172.21.15.52:0/1695517455 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f39901a1b20 0x7f39901a3f10 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f398800ba50 tx=0x7f398800bd60 comp rx=0 tx=0).stop 2024-06-10T02:43:45.003 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:45.006+0000 7f39951ff700 1 -- 172.21.15.52:0/1695517455 shutdown_connections 2024-06-10T02:43:45.003 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:45.006+0000 7f39951ff700 1 --2- 172.21.15.52:0/1695517455 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f3978039580 0x7f397803ba40 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:43:45.003 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:45.006+0000 7f39951ff700 1 --2- 172.21.15.52:0/1695517455 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f39901a1b20 0x7f39901a3f10 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:43:45.003 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:45.006+0000 7f39951ff700 1 --2- 172.21.15.52:0/1695517455 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3990101040 0x7f3990198ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:43:45.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:45.006+0000 7f39951ff700 1 --2- 172.21.15.52:0/1695517455 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3990198ff0 0x7f3990199470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:43:45.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:45.006+0000 7f39951ff700 1 -- 172.21.15.52:0/1695517455 >> 172.21.15.52:0/1695517455 conn(0x7f39900fc8f0 msgr2=0x7f3990102a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:43:45.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:45.006+0000 7f39951ff700 1 -- 172.21.15.52:0/1695517455 shutdown_connections 2024-06-10T02:43:45.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:45.006+0000 7f39951ff700 1 -- 172.21.15.52:0/1695517455 wait complete. 2024-06-10T02:43:45.672 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-06-10T02:43:45.673 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-06-10T02:43:45.673 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd of=/etc/ceph/ceph.conf 2024-06-10T02:43:45.702 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-06-10T02:43:45.702 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-06-10T02:43:45.768 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-06-10T02:43:45.768 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd of=/etc/ceph/ceph.conf 2024-06-10T02:43:45.799 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-06-10T02:43:45.799 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-06-10T02:43:45.863 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:43:45 smithi120 ceph-mon[36634]: Updating smithi188:/var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-10T02:43:45.863 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:43:45 smithi120 ceph-mon[36634]: Updating smithi052:/var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-10T02:43:45.863 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:43:45 smithi120 ceph-mon[36634]: Updating smithi120:/var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-10T02:43:45.863 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:43:45 smithi120 ceph-mon[36634]: Reconfiguring mon.a (unknown last config time)... 2024-06-10T02:43:45.864 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:43:45 smithi120 ceph-mon[36634]: Reconfiguring daemon mon.a on smithi052 2024-06-10T02:43:45.864 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:43:45 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-10T02:43:45.864 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:43:45 smithi120 ceph-mon[36634]: from='client.? 172.21.15.52:0/1695517455' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:43:45.867 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-06-10T02:43:45.867 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd of=/etc/ceph/ceph.conf 2024-06-10T02:43:45.894 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:43:45 smithi188 ceph-mon[36599]: Updating smithi188:/var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-10T02:43:45.894 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:43:45 smithi188 ceph-mon[36599]: Updating smithi052:/var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-10T02:43:45.894 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:43:45 smithi188 ceph-mon[36599]: Updating smithi120:/var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-10T02:43:45.894 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:43:45 smithi188 ceph-mon[36599]: Reconfiguring mon.a (unknown last config time)... 2024-06-10T02:43:45.894 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:43:45 smithi188 ceph-mon[36599]: Reconfiguring daemon mon.a on smithi052 2024-06-10T02:43:45.894 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:43:45 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-10T02:43:45.894 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:43:45 smithi188 ceph-mon[36599]: from='client.? 172.21.15.52:0/1695517455' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:43:45.899 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-06-10T02:43:45.899 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-06-10T02:43:45.968 INFO:tasks.cephadm:Adding mgr.x on smithi052 2024-06-10T02:43:45.968 INFO:tasks.cephadm:Adding mgr.y on smithi120 2024-06-10T02:43:45.968 INFO:tasks.cephadm:Adding mgr.z on smithi188 2024-06-10T02:43:45.969 DEBUG:teuthology.orchestra.run.smithi188:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph orch apply mgr '3;smithi052=x;smithi120=y;smithi188=z' 2024-06-10T02:43:45.973 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:45 smithi052 ceph-mon[32362]: Updating smithi188:/var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-10T02:43:45.973 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:45 smithi052 ceph-mon[32362]: Updating smithi052:/var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-10T02:43:45.973 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:45 smithi052 ceph-mon[32362]: Updating smithi120:/var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-10T02:43:45.973 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:45 smithi052 ceph-mon[32362]: Reconfiguring mon.a (unknown last config time)... 2024-06-10T02:43:45.973 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:45 smithi052 ceph-mon[32362]: Reconfiguring daemon mon.a on smithi052 2024-06-10T02:43:45.973 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:45 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-10T02:43:45.974 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:45 smithi052 ceph-mon[32362]: from='client.? 172.21.15.52:0/1695517455' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:43:46.240 INFO:teuthology.orchestra.run.smithi188.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.c/config 2024-06-10T02:43:46.885 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:43:46 smithi120 ceph-mon[36634]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T02:43:46.885 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:43:46 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:43:46.885 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:43:46 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:43:46.885 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:43:46 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-10T02:43:46.885 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:43:46 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-10T02:43:46.885 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:43:46 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:43:46.950 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:43:46 smithi188 ceph-mon[36599]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T02:43:46.950 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:43:46 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:43:46.950 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:43:46 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:43:46.950 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:43:46 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-10T02:43:46.950 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:43:46 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-10T02:43:46.950 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:43:46 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:43:46.973 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:46 smithi052 ceph-mon[32362]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T02:43:46.973 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:46 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:43:46.973 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:46 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:43:46.973 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:46 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-10T02:43:46.974 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:46 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-10T02:43:46.974 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:46 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:43:47.697 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:47.699+0000 7f10f40ca700 1 -- 172.21.15.188:0/2489972595 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f10ec1025a0 msgr2=0x7f10ec1029c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:43:47.697 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:47.699+0000 7f10f40ca700 1 --2- 172.21.15.188:0/2489972595 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f10ec1025a0 0x7f10ec1029c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f10dc009910 tx=0x7f10dc009c20 comp rx=0 tx=0).stop 2024-06-10T02:43:47.697 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:47.699+0000 7f10f40ca700 1 -- 172.21.15.188:0/2489972595 shutdown_connections 2024-06-10T02:43:47.697 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:47.699+0000 7f10f40ca700 1 --2- 172.21.15.188:0/2489972595 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f10ec1025a0 0x7f10ec1029c0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:43:47.697 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:47.699+0000 7f10f40ca700 1 -- 172.21.15.188:0/2489972595 >> 172.21.15.188:0/2489972595 conn(0x7f10ec0fbc40 msgr2=0x7f10ec0fe080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:43:47.698 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:47.699+0000 7f10f40ca700 1 -- 172.21.15.188:0/2489972595 shutdown_connections 2024-06-10T02:43:47.698 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:47.699+0000 7f10f40ca700 1 -- 172.21.15.188:0/2489972595 wait complete. 2024-06-10T02:43:47.698 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:47.699+0000 7f10f40ca700 1 Processor -- start 2024-06-10T02:43:47.698 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:47.700+0000 7f10f40ca700 1 -- start start 2024-06-10T02:43:47.698 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:47.700+0000 7f10f40ca700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f10ec1025a0 0x7f10ec197880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:43:47.698 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:47.700+0000 7f10f40ca700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f10ec197dc0 0x7f10ec19ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:43:47.698 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:47.700+0000 7f10f40ca700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f10ec198240 0x7f10ec19f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:43:47.698 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:47.700+0000 7f10f40ca700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f10ec107070 con 0x7f10ec198240 2024-06-10T02:43:47.698 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:47.700+0000 7f10f40ca700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f10ec106ef0 con 0x7f10ec197dc0 2024-06-10T02:43:47.698 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:47.700+0000 7f10f40ca700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f10ec1071f0 con 0x7f10ec1025a0 2024-06-10T02:43:47.698 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:47.700+0000 7f10f1e66700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f10ec1025a0 0x7f10ec197880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:43:47.698 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:47.700+0000 7f10f1e66700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f10ec1025a0 0x7f10ec197880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.188:37836/0 (socket says 172.21.15.188:37836) 2024-06-10T02:43:47.699 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:47.700+0000 7f10f1e66700 1 -- 172.21.15.188:0/742438276 learned_addr learned my addr 172.21.15.188:0/742438276 (peer_addr_for_me v2:172.21.15.188:0/0) 2024-06-10T02:43:47.699 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:47.700+0000 7f10f2667700 1 --2- 172.21.15.188:0/742438276 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f10ec198240 0x7f10ec19f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:43:47.699 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:47.700+0000 7f10f1665700 1 --2- 172.21.15.188:0/742438276 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f10ec197dc0 0x7f10ec19ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:43:47.699 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:47.700+0000 7f10f1e66700 1 -- 172.21.15.188:0/742438276 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f10ec197dc0 msgr2=0x7f10ec19ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:43:47.699 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:47.700+0000 7f10f1e66700 1 --2- 172.21.15.188:0/742438276 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f10ec197dc0 0x7f10ec19ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:43:47.699 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:47.700+0000 7f10f1e66700 1 -- 172.21.15.188:0/742438276 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f10ec198240 msgr2=0x7f10ec19f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:43:47.699 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:47.700+0000 7f10f1e66700 1 --2- 172.21.15.188:0/742438276 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f10ec198240 0x7f10ec19f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:43:47.699 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:47.700+0000 7f10f1e66700 1 -- 172.21.15.188:0/742438276 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10ec19fc90 con 0x7f10ec1025a0 2024-06-10T02:43:47.699 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:47.700+0000 7f10f1665700 1 --2- 172.21.15.188:0/742438276 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f10ec197dc0 0x7f10ec19ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T02:43:47.699 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:47.700+0000 7f10f1e66700 1 --2- 172.21.15.188:0/742438276 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f10ec1025a0 0x7f10ec197880 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f10dc005950 tx=0x7f10dc004f80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:43:47.699 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:47.700+0000 7f10e2ffd700 1 -- 172.21.15.188:0/742438276 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10dc026070 con 0x7f10ec1025a0 2024-06-10T02:43:47.699 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:47.700+0000 7f10e2ffd700 1 -- 172.21.15.188:0/742438276 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f10dc00b750 con 0x7f10ec1025a0 2024-06-10T02:43:47.700 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:47.701+0000 7f10f40ca700 1 -- 172.21.15.188:0/742438276 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10ec19ff20 con 0x7f10ec1025a0 2024-06-10T02:43:47.700 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:47.701+0000 7f10e2ffd700 1 -- 172.21.15.188:0/742438276 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10dc003a80 con 0x7f10ec1025a0 2024-06-10T02:43:47.700 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:47.701+0000 7f10f40ca700 1 -- 172.21.15.188:0/742438276 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10ec1a0390 con 0x7f10ec1025a0 2024-06-10T02:43:47.700 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:47.701+0000 7f10e2ffd700 1 -- 172.21.15.188:0/742438276 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f10dc029070 con 0x7f10ec1025a0 2024-06-10T02:43:47.701 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:47.701+0000 7f10f40ca700 1 -- 172.21.15.188:0/742438276 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10ec063af0 con 0x7f10ec1025a0 2024-06-10T02:43:47.701 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:47.702+0000 7f10e2ffd700 1 --2- 172.21.15.188:0/742438276 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f10d8039580 0x7f10d803ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:43:47.701 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:47.702+0000 7f10e2ffd700 1 -- 172.21.15.188:0/742438276 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f10dc04d030 con 0x7f10ec1025a0 2024-06-10T02:43:47.701 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:47.702+0000 7f10f1665700 1 --2- 172.21.15.188:0/742438276 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f10d8039580 0x7f10d803ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:43:47.701 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:47.702+0000 7f10f1665700 1 --2- 172.21.15.188:0/742438276 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f10d8039580 0x7f10d803ba40 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f10e8009910 tx=0x7f10e8008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:43:47.703 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:47.707+0000 7f10e2ffd700 1 -- 172.21.15.188:0/742438276 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f10dc013070 con 0x7f10ec1025a0 2024-06-10T02:43:47.885 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:43:47 smithi120 ceph-mon[36634]: Reconfiguring mon.b (monmap changed)... 2024-06-10T02:43:47.885 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:43:47 smithi120 ceph-mon[36634]: Reconfiguring daemon mon.b on smithi120 2024-06-10T02:43:47.910 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:47.913+0000 7f10f40ca700 1 -- 172.21.15.188:0/742438276 --> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi052=x;smithi120=y;smithi188=z", "target": ["mon-mgr", ""]}) -- 0x7f10ec062960 con 0x7f10d8039580 2024-06-10T02:43:47.910 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:43:47 smithi188 ceph-mon[36599]: Reconfiguring mon.b (monmap changed)... 2024-06-10T02:43:47.910 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:43:47 smithi188 ceph-mon[36599]: Reconfiguring daemon mon.b on smithi120 2024-06-10T02:43:47.915 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:47.919+0000 7f10e2ffd700 1 -- 172.21.15.188:0/742438276 <== mgr.14150 v2:172.21.15.52:6800/3214763969 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f10ec062960 con 0x7f10d8039580 2024-06-10T02:43:47.916 INFO:teuthology.orchestra.run.smithi188.stdout:Scheduled mgr update... 2024-06-10T02:43:47.918 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:47.922+0000 7f10f40ca700 1 -- 172.21.15.188:0/742438276 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f10d8039580 msgr2=0x7f10d803ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:43:47.918 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:47.922+0000 7f10f40ca700 1 --2- 172.21.15.188:0/742438276 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f10d8039580 0x7f10d803ba40 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f10e8009910 tx=0x7f10e8008040 comp rx=0 tx=0).stop 2024-06-10T02:43:47.918 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:47.922+0000 7f10f40ca700 1 -- 172.21.15.188:0/742438276 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f10ec1025a0 msgr2=0x7f10ec197880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:43:47.918 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:47.922+0000 7f10f40ca700 1 --2- 172.21.15.188:0/742438276 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f10ec1025a0 0x7f10ec197880 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f10dc005950 tx=0x7f10dc004f80 comp rx=0 tx=0).stop 2024-06-10T02:43:47.918 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:47.922+0000 7f10f40ca700 1 -- 172.21.15.188:0/742438276 shutdown_connections 2024-06-10T02:43:47.919 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:47.922+0000 7f10f40ca700 1 --2- 172.21.15.188:0/742438276 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f10d8039580 0x7f10d803ba40 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:43:47.919 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:47.922+0000 7f10f40ca700 1 --2- 172.21.15.188:0/742438276 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f10ec198240 0x7f10ec19f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:43:47.919 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:47.922+0000 7f10f40ca700 1 --2- 172.21.15.188:0/742438276 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f10ec1025a0 0x7f10ec197880 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:43:47.919 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:47.922+0000 7f10f40ca700 1 --2- 172.21.15.188:0/742438276 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f10ec197dc0 0x7f10ec19ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:43:47.919 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:47.922+0000 7f10f40ca700 1 -- 172.21.15.188:0/742438276 >> 172.21.15.188:0/742438276 conn(0x7f10ec0fbc40 msgr2=0x7f10ec100580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:43:47.919 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:47.922+0000 7f10f40ca700 1 -- 172.21.15.188:0/742438276 shutdown_connections 2024-06-10T02:43:47.919 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:43:47.922+0000 7f10f40ca700 1 -- 172.21.15.188:0/742438276 wait complete. 2024-06-10T02:43:47.973 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:47 smithi052 ceph-mon[32362]: Reconfiguring mon.b (monmap changed)... 2024-06-10T02:43:47.973 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:47 smithi052 ceph-mon[32362]: Reconfiguring daemon mon.b on smithi120 2024-06-10T02:43:48.611 DEBUG:teuthology.orchestra.run.smithi120:mgr.y> sudo journalctl -f -n 0 -u ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968@mgr.y.service 2024-06-10T02:43:48.614 DEBUG:teuthology.orchestra.run.smithi188:mgr.z> sudo journalctl -f -n 0 -u ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968@mgr.z.service 2024-06-10T02:43:48.616 INFO:tasks.cephadm:Deploying OSDs... 2024-06-10T02:43:48.616 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-06-10T02:43:48.616 DEBUG:teuthology.orchestra.run.smithi052:> dd if=/scratch_devs of=/dev/stdout 2024-06-10T02:43:48.636 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-06-10T02:43:48.636 DEBUG:teuthology.orchestra.run.smithi052:> stat /dev/vg_nvme/lv_1 2024-06-10T02:43:48.695 INFO:teuthology.orchestra.run.smithi052.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-06-10T02:43:48.695 INFO:teuthology.orchestra.run.smithi052.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-10T02:43:48.696 INFO:teuthology.orchestra.run.smithi052.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-06-10T02:43:48.696 INFO:teuthology.orchestra.run.smithi052.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-10T02:43:48.696 INFO:teuthology.orchestra.run.smithi052.stdout:Context: system_u:object_r:device_t:s0 2024-06-10T02:43:48.696 INFO:teuthology.orchestra.run.smithi052.stdout:Access: 2024-06-10 02:42:55.095272907 +0000 2024-06-10T02:43:48.696 INFO:teuthology.orchestra.run.smithi052.stdout:Modify: 2024-06-10 02:32:58.515959443 +0000 2024-06-10T02:43:48.696 INFO:teuthology.orchestra.run.smithi052.stdout:Change: 2024-06-10 02:32:58.515959443 +0000 2024-06-10T02:43:48.696 INFO:teuthology.orchestra.run.smithi052.stdout: Birth: 2024-06-10 02:32:58.515959443 +0000 2024-06-10T02:43:48.696 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-06-10T02:43:48.765 INFO:teuthology.orchestra.run.smithi052.stderr:1+0 records in 2024-06-10T02:43:48.765 INFO:teuthology.orchestra.run.smithi052.stderr:1+0 records out 2024-06-10T02:43:48.765 INFO:teuthology.orchestra.run.smithi052.stderr:512 bytes copied, 0.00011523 s, 4.4 MB/s 2024-06-10T02:43:48.767 DEBUG:teuthology.orchestra.run.smithi052:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-06-10T02:43:48.826 DEBUG:teuthology.orchestra.run.smithi052:> stat /dev/vg_nvme/lv_2 2024-06-10T02:43:48.883 INFO:teuthology.orchestra.run.smithi052.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-06-10T02:43:48.883 INFO:teuthology.orchestra.run.smithi052.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-10T02:43:48.883 INFO:teuthology.orchestra.run.smithi052.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-06-10T02:43:48.883 INFO:teuthology.orchestra.run.smithi052.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-10T02:43:48.883 INFO:teuthology.orchestra.run.smithi052.stdout:Context: system_u:object_r:device_t:s0 2024-06-10T02:43:48.883 INFO:teuthology.orchestra.run.smithi052.stdout:Access: 2024-06-10 02:42:55.140272302 +0000 2024-06-10T02:43:48.884 INFO:teuthology.orchestra.run.smithi052.stdout:Modify: 2024-06-10 02:32:58.851954989 +0000 2024-06-10T02:43:48.884 INFO:teuthology.orchestra.run.smithi052.stdout:Change: 2024-06-10 02:32:58.851954989 +0000 2024-06-10T02:43:48.884 INFO:teuthology.orchestra.run.smithi052.stdout: Birth: 2024-06-10 02:32:58.851954989 +0000 2024-06-10T02:43:48.884 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-06-10T02:43:48.885 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:43:48 smithi120 ceph-mon[36634]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T02:43:48.885 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:43:48 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:43:48.885 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:43:48 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:43:48.885 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:43:48 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:43:48.885 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:43:48 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-10T02:43:48.885 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:43:48 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-10T02:43:48.885 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:43:48 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:43:48.929 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:43:48 smithi188 ceph-mon[36599]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T02:43:48.930 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:43:48 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:43:48.930 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:43:48 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:43:48.930 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:43:48 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:43:48.930 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:43:48 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-10T02:43:48.930 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:43:48 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-10T02:43:48.930 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:43:48 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:43:48.947 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:48 smithi052 ceph-mon[32362]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T02:43:48.947 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:48 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:43:48.947 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:48 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:43:48.947 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:48 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:43:48.947 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:48 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-10T02:43:48.947 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:48 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-10T02:43:48.947 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:48 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:43:48.949 INFO:teuthology.orchestra.run.smithi052.stderr:1+0 records in 2024-06-10T02:43:48.949 INFO:teuthology.orchestra.run.smithi052.stderr:1+0 records out 2024-06-10T02:43:48.949 INFO:teuthology.orchestra.run.smithi052.stderr:512 bytes copied, 0.000108239 s, 4.7 MB/s 2024-06-10T02:43:48.950 DEBUG:teuthology.orchestra.run.smithi052:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-06-10T02:43:49.008 DEBUG:teuthology.orchestra.run.smithi052:> stat /dev/vg_nvme/lv_3 2024-06-10T02:43:49.064 INFO:teuthology.orchestra.run.smithi052.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-06-10T02:43:49.064 INFO:teuthology.orchestra.run.smithi052.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-10T02:43:49.064 INFO:teuthology.orchestra.run.smithi052.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-06-10T02:43:49.064 INFO:teuthology.orchestra.run.smithi052.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-10T02:43:49.064 INFO:teuthology.orchestra.run.smithi052.stdout:Context: system_u:object_r:device_t:s0 2024-06-10T02:43:49.064 INFO:teuthology.orchestra.run.smithi052.stdout:Access: 2024-06-10 02:42:55.180271765 +0000 2024-06-10T02:43:49.065 INFO:teuthology.orchestra.run.smithi052.stdout:Modify: 2024-06-10 02:32:59.181950615 +0000 2024-06-10T02:43:49.065 INFO:teuthology.orchestra.run.smithi052.stdout:Change: 2024-06-10 02:32:59.181950615 +0000 2024-06-10T02:43:49.065 INFO:teuthology.orchestra.run.smithi052.stdout: Birth: 2024-06-10 02:32:59.181950615 +0000 2024-06-10T02:43:49.065 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-06-10T02:43:49.128 INFO:teuthology.orchestra.run.smithi052.stderr:1+0 records in 2024-06-10T02:43:49.128 INFO:teuthology.orchestra.run.smithi052.stderr:1+0 records out 2024-06-10T02:43:49.128 INFO:teuthology.orchestra.run.smithi052.stderr:512 bytes copied, 0.00010775 s, 4.8 MB/s 2024-06-10T02:43:49.130 DEBUG:teuthology.orchestra.run.smithi052:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-06-10T02:43:49.187 DEBUG:teuthology.orchestra.run.smithi052:> stat /dev/vg_nvme/lv_4 2024-06-10T02:43:49.243 INFO:teuthology.orchestra.run.smithi052.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-06-10T02:43:49.243 INFO:teuthology.orchestra.run.smithi052.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-10T02:43:49.243 INFO:teuthology.orchestra.run.smithi052.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-06-10T02:43:49.243 INFO:teuthology.orchestra.run.smithi052.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-10T02:43:49.243 INFO:teuthology.orchestra.run.smithi052.stdout:Context: system_u:object_r:device_t:s0 2024-06-10T02:43:49.243 INFO:teuthology.orchestra.run.smithi052.stdout:Access: 2024-06-10 02:42:55.214271309 +0000 2024-06-10T02:43:49.243 INFO:teuthology.orchestra.run.smithi052.stdout:Modify: 2024-06-10 02:32:59.531945975 +0000 2024-06-10T02:43:49.243 INFO:teuthology.orchestra.run.smithi052.stdout:Change: 2024-06-10 02:32:59.531945975 +0000 2024-06-10T02:43:49.244 INFO:teuthology.orchestra.run.smithi052.stdout: Birth: 2024-06-10 02:32:59.531945975 +0000 2024-06-10T02:43:49.244 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-06-10T02:43:49.307 INFO:teuthology.orchestra.run.smithi052.stderr:1+0 records in 2024-06-10T02:43:49.307 INFO:teuthology.orchestra.run.smithi052.stderr:1+0 records out 2024-06-10T02:43:49.307 INFO:teuthology.orchestra.run.smithi052.stderr:512 bytes copied, 9.2484e-05 s, 5.5 MB/s 2024-06-10T02:43:49.308 DEBUG:teuthology.orchestra.run.smithi052:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-06-10T02:43:49.364 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-06-10T02:43:49.365 DEBUG:teuthology.orchestra.run.smithi120:> dd if=/scratch_devs of=/dev/stdout 2024-06-10T02:43:49.383 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-06-10T02:43:49.383 DEBUG:teuthology.orchestra.run.smithi120:> stat /dev/vg_nvme/lv_1 2024-06-10T02:43:49.441 INFO:teuthology.orchestra.run.smithi120.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-06-10T02:43:49.442 INFO:teuthology.orchestra.run.smithi120.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-10T02:43:49.442 INFO:teuthology.orchestra.run.smithi120.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-06-10T02:43:49.442 INFO:teuthology.orchestra.run.smithi120.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-10T02:43:49.442 INFO:teuthology.orchestra.run.smithi120.stdout:Context: system_u:object_r:device_t:s0 2024-06-10T02:43:49.442 INFO:teuthology.orchestra.run.smithi120.stdout:Access: 2024-06-10 02:43:36.099254529 +0000 2024-06-10T02:43:49.442 INFO:teuthology.orchestra.run.smithi120.stdout:Modify: 2024-06-10 02:33:10.046251684 +0000 2024-06-10T02:43:49.442 INFO:teuthology.orchestra.run.smithi120.stdout:Change: 2024-06-10 02:33:10.046251684 +0000 2024-06-10T02:43:49.442 INFO:teuthology.orchestra.run.smithi120.stdout: Birth: 2024-06-10 02:33:10.046251684 +0000 2024-06-10T02:43:49.442 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-06-10T02:43:49.507 INFO:teuthology.orchestra.run.smithi120.stderr:1+0 records in 2024-06-10T02:43:49.507 INFO:teuthology.orchestra.run.smithi120.stderr:1+0 records out 2024-06-10T02:43:49.507 INFO:teuthology.orchestra.run.smithi120.stderr:512 bytes copied, 0.000317631 s, 1.6 MB/s 2024-06-10T02:43:49.508 DEBUG:teuthology.orchestra.run.smithi120:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-06-10T02:43:49.566 DEBUG:teuthology.orchestra.run.smithi120:> stat /dev/vg_nvme/lv_2 2024-06-10T02:43:49.621 INFO:teuthology.orchestra.run.smithi120.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-06-10T02:43:49.621 INFO:teuthology.orchestra.run.smithi120.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-10T02:43:49.622 INFO:teuthology.orchestra.run.smithi120.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-06-10T02:43:49.622 INFO:teuthology.orchestra.run.smithi120.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-10T02:43:49.622 INFO:teuthology.orchestra.run.smithi120.stdout:Context: system_u:object_r:device_t:s0 2024-06-10T02:43:49.622 INFO:teuthology.orchestra.run.smithi120.stdout:Access: 2024-06-10 02:43:36.100254508 +0000 2024-06-10T02:43:49.622 INFO:teuthology.orchestra.run.smithi120.stdout:Modify: 2024-06-10 02:33:10.380243915 +0000 2024-06-10T02:43:49.622 INFO:teuthology.orchestra.run.smithi120.stdout:Change: 2024-06-10 02:33:10.380243915 +0000 2024-06-10T02:43:49.622 INFO:teuthology.orchestra.run.smithi120.stdout: Birth: 2024-06-10 02:33:10.380243915 +0000 2024-06-10T02:43:49.622 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-06-10T02:43:49.685 INFO:teuthology.orchestra.run.smithi120.stderr:1+0 records in 2024-06-10T02:43:49.686 INFO:teuthology.orchestra.run.smithi120.stderr:1+0 records out 2024-06-10T02:43:49.686 INFO:teuthology.orchestra.run.smithi120.stderr:512 bytes copied, 0.000275275 s, 1.9 MB/s 2024-06-10T02:43:49.687 DEBUG:teuthology.orchestra.run.smithi120:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-06-10T02:43:49.743 DEBUG:teuthology.orchestra.run.smithi120:> stat /dev/vg_nvme/lv_3 2024-06-10T02:43:49.799 INFO:teuthology.orchestra.run.smithi120.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-06-10T02:43:49.799 INFO:teuthology.orchestra.run.smithi120.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-10T02:43:49.799 INFO:teuthology.orchestra.run.smithi120.stdout:Device: 5h/5d Inode: 701 Links: 1 2024-06-10T02:43:49.799 INFO:teuthology.orchestra.run.smithi120.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-10T02:43:49.799 INFO:teuthology.orchestra.run.smithi120.stdout:Context: system_u:object_r:device_t:s0 2024-06-10T02:43:49.799 INFO:teuthology.orchestra.run.smithi120.stdout:Access: 2024-06-10 02:43:36.100254508 +0000 2024-06-10T02:43:49.800 INFO:teuthology.orchestra.run.smithi120.stdout:Modify: 2024-06-10 02:33:10.702236426 +0000 2024-06-10T02:43:49.800 INFO:teuthology.orchestra.run.smithi120.stdout:Change: 2024-06-10 02:33:10.702236426 +0000 2024-06-10T02:43:49.800 INFO:teuthology.orchestra.run.smithi120.stdout: Birth: 2024-06-10 02:33:10.702236426 +0000 2024-06-10T02:43:49.800 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-06-10T02:43:49.860 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:43:49 smithi120 ceph-mon[36634]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi052=x;smithi120=y;smithi188=z", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T02:43:49.860 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:43:49 smithi120 ceph-mon[36634]: Saving service mgr spec with placement smithi052=x;smithi120=y;smithi188=z;count:3 2024-06-10T02:43:49.860 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:43:49 smithi120 ceph-mon[36634]: Reconfiguring mon.c (monmap changed)... 2024-06-10T02:43:49.861 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:43:49 smithi120 ceph-mon[36634]: Reconfiguring daemon mon.c on smithi188 2024-06-10T02:43:49.862 INFO:teuthology.orchestra.run.smithi120.stderr:1+0 records in 2024-06-10T02:43:49.863 INFO:teuthology.orchestra.run.smithi120.stderr:1+0 records out 2024-06-10T02:43:49.863 INFO:teuthology.orchestra.run.smithi120.stderr:512 bytes copied, 0.000192544 s, 2.7 MB/s 2024-06-10T02:43:49.864 DEBUG:teuthology.orchestra.run.smithi120:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-06-10T02:43:49.920 DEBUG:teuthology.orchestra.run.smithi120:> stat /dev/vg_nvme/lv_4 2024-06-10T02:43:49.950 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:43:49 smithi188 ceph-mon[36599]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi052=x;smithi120=y;smithi188=z", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T02:43:49.950 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:43:49 smithi188 ceph-mon[36599]: Saving service mgr spec with placement smithi052=x;smithi120=y;smithi188=z;count:3 2024-06-10T02:43:49.950 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:43:49 smithi188 ceph-mon[36599]: Reconfiguring mon.c (monmap changed)... 2024-06-10T02:43:49.950 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:43:49 smithi188 ceph-mon[36599]: Reconfiguring daemon mon.c on smithi188 2024-06-10T02:43:49.973 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:49 smithi052 ceph-mon[32362]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi052=x;smithi120=y;smithi188=z", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T02:43:49.973 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:49 smithi052 ceph-mon[32362]: Saving service mgr spec with placement smithi052=x;smithi120=y;smithi188=z;count:3 2024-06-10T02:43:49.973 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:49 smithi052 ceph-mon[32362]: Reconfiguring mon.c (monmap changed)... 2024-06-10T02:43:49.973 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:49 smithi052 ceph-mon[32362]: Reconfiguring daemon mon.c on smithi188 2024-06-10T02:43:49.975 INFO:teuthology.orchestra.run.smithi120.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-06-10T02:43:49.975 INFO:teuthology.orchestra.run.smithi120.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-10T02:43:49.975 INFO:teuthology.orchestra.run.smithi120.stdout:Device: 5h/5d Inode: 721 Links: 1 2024-06-10T02:43:49.976 INFO:teuthology.orchestra.run.smithi120.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-10T02:43:49.976 INFO:teuthology.orchestra.run.smithi120.stdout:Context: system_u:object_r:device_t:s0 2024-06-10T02:43:49.976 INFO:teuthology.orchestra.run.smithi120.stdout:Access: 2024-06-10 02:43:36.101254485 +0000 2024-06-10T02:43:49.976 INFO:teuthology.orchestra.run.smithi120.stdout:Modify: 2024-06-10 02:33:11.029228820 +0000 2024-06-10T02:43:49.976 INFO:teuthology.orchestra.run.smithi120.stdout:Change: 2024-06-10 02:33:11.029228820 +0000 2024-06-10T02:43:49.976 INFO:teuthology.orchestra.run.smithi120.stdout: Birth: 2024-06-10 02:33:11.029228820 +0000 2024-06-10T02:43:49.976 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-06-10T02:43:50.038 INFO:teuthology.orchestra.run.smithi120.stderr:1+0 records in 2024-06-10T02:43:50.038 INFO:teuthology.orchestra.run.smithi120.stderr:1+0 records out 2024-06-10T02:43:50.038 INFO:teuthology.orchestra.run.smithi120.stderr:512 bytes copied, 0.00023751 s, 2.2 MB/s 2024-06-10T02:43:50.039 DEBUG:teuthology.orchestra.run.smithi120:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-06-10T02:43:50.095 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-06-10T02:43:50.095 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/scratch_devs of=/dev/stdout 2024-06-10T02:43:50.112 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-06-10T02:43:50.112 DEBUG:teuthology.orchestra.run.smithi188:> stat /dev/vg_nvme/lv_1 2024-06-10T02:43:50.169 INFO:teuthology.orchestra.run.smithi188.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-06-10T02:43:50.169 INFO:teuthology.orchestra.run.smithi188.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-10T02:43:50.169 INFO:teuthology.orchestra.run.smithi188.stdout:Device: 5h/5d Inode: 664 Links: 1 2024-06-10T02:43:50.169 INFO:teuthology.orchestra.run.smithi188.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-10T02:43:50.169 INFO:teuthology.orchestra.run.smithi188.stdout:Context: system_u:object_r:device_t:s0 2024-06-10T02:43:50.169 INFO:teuthology.orchestra.run.smithi188.stdout:Access: 2024-06-10 02:43:26.795600685 +0000 2024-06-10T02:43:50.169 INFO:teuthology.orchestra.run.smithi188.stdout:Modify: 2024-06-10 02:33:07.972217267 +0000 2024-06-10T02:43:50.170 INFO:teuthology.orchestra.run.smithi188.stdout:Change: 2024-06-10 02:33:07.972217267 +0000 2024-06-10T02:43:50.170 INFO:teuthology.orchestra.run.smithi188.stdout: Birth: 2024-06-10 02:33:07.972217267 +0000 2024-06-10T02:43:50.170 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-06-10T02:43:50.234 INFO:teuthology.orchestra.run.smithi188.stderr:1+0 records in 2024-06-10T02:43:50.234 INFO:teuthology.orchestra.run.smithi188.stderr:1+0 records out 2024-06-10T02:43:50.234 INFO:teuthology.orchestra.run.smithi188.stderr:512 bytes copied, 0.000295185 s, 1.7 MB/s 2024-06-10T02:43:50.235 DEBUG:teuthology.orchestra.run.smithi188:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-06-10T02:43:50.293 DEBUG:teuthology.orchestra.run.smithi188:> stat /dev/vg_nvme/lv_2 2024-06-10T02:43:50.348 INFO:teuthology.orchestra.run.smithi188.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-06-10T02:43:50.348 INFO:teuthology.orchestra.run.smithi188.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-10T02:43:50.348 INFO:teuthology.orchestra.run.smithi188.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-06-10T02:43:50.348 INFO:teuthology.orchestra.run.smithi188.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-10T02:43:50.348 INFO:teuthology.orchestra.run.smithi188.stdout:Context: system_u:object_r:device_t:s0 2024-06-10T02:43:50.348 INFO:teuthology.orchestra.run.smithi188.stdout:Access: 2024-06-10 02:43:26.796600668 +0000 2024-06-10T02:43:50.349 INFO:teuthology.orchestra.run.smithi188.stdout:Modify: 2024-06-10 02:33:08.306211145 +0000 2024-06-10T02:43:50.349 INFO:teuthology.orchestra.run.smithi188.stdout:Change: 2024-06-10 02:33:08.306211145 +0000 2024-06-10T02:43:50.349 INFO:teuthology.orchestra.run.smithi188.stdout: Birth: 2024-06-10 02:33:08.306211145 +0000 2024-06-10T02:43:50.349 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-06-10T02:43:50.410 INFO:teuthology.orchestra.run.smithi188.stderr:1+0 records in 2024-06-10T02:43:50.410 INFO:teuthology.orchestra.run.smithi188.stderr:1+0 records out 2024-06-10T02:43:50.410 INFO:teuthology.orchestra.run.smithi188.stderr:512 bytes copied, 0.000195065 s, 2.6 MB/s 2024-06-10T02:43:50.412 DEBUG:teuthology.orchestra.run.smithi188:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-06-10T02:43:50.468 DEBUG:teuthology.orchestra.run.smithi188:> stat /dev/vg_nvme/lv_3 2024-06-10T02:43:50.522 INFO:teuthology.orchestra.run.smithi188.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-06-10T02:43:50.522 INFO:teuthology.orchestra.run.smithi188.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-10T02:43:50.523 INFO:teuthology.orchestra.run.smithi188.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-06-10T02:43:50.523 INFO:teuthology.orchestra.run.smithi188.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-10T02:43:50.523 INFO:teuthology.orchestra.run.smithi188.stdout:Context: system_u:object_r:device_t:s0 2024-06-10T02:43:50.523 INFO:teuthology.orchestra.run.smithi188.stdout:Access: 2024-06-10 02:43:26.796600668 +0000 2024-06-10T02:43:50.523 INFO:teuthology.orchestra.run.smithi188.stdout:Modify: 2024-06-10 02:33:08.639205041 +0000 2024-06-10T02:43:50.523 INFO:teuthology.orchestra.run.smithi188.stdout:Change: 2024-06-10 02:33:08.639205041 +0000 2024-06-10T02:43:50.523 INFO:teuthology.orchestra.run.smithi188.stdout: Birth: 2024-06-10 02:33:08.639205041 +0000 2024-06-10T02:43:50.523 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-06-10T02:43:50.585 INFO:teuthology.orchestra.run.smithi188.stderr:1+0 records in 2024-06-10T02:43:50.585 INFO:teuthology.orchestra.run.smithi188.stderr:1+0 records out 2024-06-10T02:43:50.585 INFO:teuthology.orchestra.run.smithi188.stderr:512 bytes copied, 0.000194293 s, 2.6 MB/s 2024-06-10T02:43:50.587 DEBUG:teuthology.orchestra.run.smithi188:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-06-10T02:43:50.642 DEBUG:teuthology.orchestra.run.smithi188:> stat /dev/vg_nvme/lv_4 2024-06-10T02:43:50.697 INFO:teuthology.orchestra.run.smithi188.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-06-10T02:43:50.697 INFO:teuthology.orchestra.run.smithi188.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-10T02:43:50.697 INFO:teuthology.orchestra.run.smithi188.stdout:Device: 5h/5d Inode: 721 Links: 1 2024-06-10T02:43:50.697 INFO:teuthology.orchestra.run.smithi188.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-10T02:43:50.697 INFO:teuthology.orchestra.run.smithi188.stdout:Context: system_u:object_r:device_t:s0 2024-06-10T02:43:50.698 INFO:teuthology.orchestra.run.smithi188.stdout:Access: 2024-06-10 02:43:26.796600668 +0000 2024-06-10T02:43:50.698 INFO:teuthology.orchestra.run.smithi188.stdout:Modify: 2024-06-10 02:33:08.958199193 +0000 2024-06-10T02:43:50.698 INFO:teuthology.orchestra.run.smithi188.stdout:Change: 2024-06-10 02:33:08.958199193 +0000 2024-06-10T02:43:50.698 INFO:teuthology.orchestra.run.smithi188.stdout: Birth: 2024-06-10 02:33:08.958199193 +0000 2024-06-10T02:43:50.698 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-06-10T02:43:50.762 INFO:teuthology.orchestra.run.smithi188.stderr:1+0 records in 2024-06-10T02:43:50.762 INFO:teuthology.orchestra.run.smithi188.stderr:1+0 records out 2024-06-10T02:43:50.762 INFO:teuthology.orchestra.run.smithi188.stderr:512 bytes copied, 0.000178296 s, 2.9 MB/s 2024-06-10T02:43:50.764 DEBUG:teuthology.orchestra.run.smithi188:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-06-10T02:43:50.819 INFO:tasks.cephadm:Deploying osd.0 on smithi052 with /dev/vg_nvme/lv_4... 2024-06-10T02:43:50.819 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-06-10T02:43:50.892 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:43:50 smithi188 ceph-mon[36599]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T02:43:50.892 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:43:50 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:43:50.892 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:43:50 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:43:50.892 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:43:50 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T02:43:50.893 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:43:50 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:43:50.893 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:43:50 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T02:43:50.893 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:43:50 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:43:50.893 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:43:50 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-10T02:43:50.893 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:43:50 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-06-10T02:43:50.893 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:43:50 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-10T02:43:50.893 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:43:50 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:43:50.973 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:50 smithi052 ceph-mon[32362]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T02:43:50.973 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:50 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:43:50.973 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:50 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:43:50.973 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:50 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T02:43:50.974 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:50 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:43:50.974 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:50 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T02:43:50.974 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:50 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:43:50.974 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:50 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-10T02:43:50.974 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:50 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-06-10T02:43:50.974 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:50 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-10T02:43:50.974 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:50 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:43:51.072 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:43:51.135 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:43:50 smithi120 ceph-mon[36634]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T02:43:51.135 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:43:50 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:43:51.135 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:43:50 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:43:51.135 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:43:50 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T02:43:51.135 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:43:50 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:43:51.135 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:43:50 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T02:43:51.135 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:43:50 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:43:51.135 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:43:50 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-10T02:43:51.135 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:43:50 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-06-10T02:43:51.136 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:43:50 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-10T02:43:51.136 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:43:50 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:43:51.723 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:51 smithi052 ceph-mon[32362]: Deploying daemon mgr.z on smithi188 2024-06-10T02:43:51.950 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:43:51 smithi188 ceph-mon[36599]: Deploying daemon mgr.z on smithi188 2024-06-10T02:43:52.135 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:43:51 smithi120 ceph-mon[36634]: Deploying daemon mgr.z on smithi188 2024-06-10T02:43:52.432 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Jun 10 02:43:52 smithi188 systemd[1]: Starting Ceph mgr.z for d8676e9e-26d2-11ef-bc9b-c7b262605968... 2024-06-10T02:43:52.700 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:43:52 smithi188 ceph-mon[36599]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T02:43:52.701 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Jun 10 02:43:52 smithi188 podman[38412]: 2024-06-10 02:43:52.436549659 +0000 UTC m=+0.102908093 container create 2ede476ce097933a1031babcad5a744c62310e7ba1669bc78aaf5349564a5d40 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mgr-z, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-06-10T02:43:52.701 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Jun 10 02:43:52 smithi188 podman[38412]: 2024-06-10 02:43:52.350708423 +0000 UTC m=+0.017066871 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-10T02:43:52.701 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Jun 10 02:43:52 smithi188 podman[38412]: 2024-06-10 02:43:52.655466116 +0000 UTC m=+0.321824549 container init 2ede476ce097933a1031babcad5a744c62310e7ba1669bc78aaf5349564a5d40 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mgr-z, ceph=True, io.buildah.version=1.33.5, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-10T02:43:52.701 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Jun 10 02:43:52 smithi188 podman[38412]: 2024-06-10 02:43:52.658093019 +0000 UTC m=+0.324451454 container start 2ede476ce097933a1031babcad5a744c62310e7ba1669bc78aaf5349564a5d40 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mgr-z, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc) 2024-06-10T02:43:52.701 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Jun 10 02:43:52 smithi188 ceph-mgr[38431]: -- 172.21.15.188:0/2976054901 <== mon.1 v2:172.21.15.188:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55666069e5a0 con 0x556660672800 2024-06-10T02:43:52.850 INFO:teuthology.orchestra.run.smithi052.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-06-10T02:43:52.850 INFO:teuthology.orchestra.run.smithi052.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-06-10T02:43:52.850 INFO:teuthology.orchestra.run.smithi052.stderr: stderr: 10+0 records in 2024-06-10T02:43:52.850 INFO:teuthology.orchestra.run.smithi052.stderr:10+0 records out 2024-06-10T02:43:52.850 INFO:teuthology.orchestra.run.smithi052.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00830939 s, 1.3 GB/s 2024-06-10T02:43:52.851 INFO:teuthology.orchestra.run.smithi052.stderr:--> Zapping successful for: 2024-06-10T02:43:52.973 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:52 smithi052 ceph-mon[32362]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T02:43:52.998 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Jun 10 02:43:52 smithi188 bash[38412]: 2ede476ce097933a1031babcad5a744c62310e7ba1669bc78aaf5349564a5d40 2024-06-10T02:43:52.998 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Jun 10 02:43:52 smithi188 systemd[1]: Started Ceph mgr.z for d8676e9e-26d2-11ef-bc9b-c7b262605968. 2024-06-10T02:43:52.999 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Jun 10 02:43:52 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mgr-z[38427]: 2024-06-10T02:43:52.817+0000 7f57659d5200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-06-10T02:43:52.999 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Jun 10 02:43:53 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mgr-z[38427]: 2024-06-10T02:43:53.002+0000 7f57659d5200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-06-10T02:43:53.036 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:43:52 smithi120 ceph-mon[36634]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T02:43:53.410 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Jun 10 02:43:53 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mgr-z[38427]: 2024-06-10T02:43:53.104+0000 7f57659d5200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-06-10T02:43:53.427 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi052:vg_nvme/lv_4 2024-06-10T02:43:53.672 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:43:53.700 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Jun 10 02:43:53 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mgr-z[38427]: 2024-06-10T02:43:53.414+0000 7f57659d5200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-06-10T02:43:53.700 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Jun 10 02:43:53 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mgr-z[38427]: 2024-06-10T02:43:53.499+0000 7f57659d5200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-06-10T02:43:54.003 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:43:53 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:43:54.003 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:43:53 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:43:54.003 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:43:53 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:43:54.003 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:43:53 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-10T02:43:54.004 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:43:53 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-06-10T02:43:54.004 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:43:53 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-10T02:43:54.004 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:43:53 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:43:54.004 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:43:53 smithi120 ceph-mon[36634]: Deploying daemon mgr.y on smithi120 2024-06-10T02:43:54.004 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:43:53 smithi120 ceph-mon[36634]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T02:43:54.108 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:53 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:43:54.108 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:53 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:43:54.109 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:53 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:43:54.109 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:53 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-10T02:43:54.109 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:53 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-06-10T02:43:54.109 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:53 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-10T02:43:54.109 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:53 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:43:54.109 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:53 smithi052 ceph-mon[32362]: Deploying daemon mgr.y on smithi120 2024-06-10T02:43:54.109 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:53 smithi052 ceph-mon[32362]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T02:43:54.200 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:43:53 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:43:54.200 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:43:53 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:43:54.200 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:43:53 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:43:54.200 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:43:53 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-10T02:43:54.201 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:43:53 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-06-10T02:43:54.201 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:43:53 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-10T02:43:54.201 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:43:53 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:43:54.201 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:43:53 smithi188 ceph-mon[36599]: Deploying daemon mgr.y on smithi120 2024-06-10T02:43:54.201 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:43:53 smithi188 ceph-mon[36599]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T02:43:54.950 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Jun 10 02:43:54 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mgr-z[38427]: 2024-06-10T02:43:54.689+0000 7f57659d5200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-06-10T02:43:54.950 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Jun 10 02:43:54 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mgr-z[38427]: 2024-06-10T02:43:54.861+0000 7f57659d5200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-06-10T02:43:55.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:55.204+0000 7fdee1fad700 1 -- 172.21.15.52:0/274489145 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdedc104090 msgr2=0x7fdedc10a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:43:55.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:55.204+0000 7fdee1fad700 1 --2- 172.21.15.52:0/274489145 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdedc104090 0x7fdedc10a940 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fded400b0d0 tx=0x7fded400b3e0 comp rx=0 tx=0).stop 2024-06-10T02:43:55.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:55.204+0000 7fdee1fad700 1 -- 172.21.15.52:0/274489145 shutdown_connections 2024-06-10T02:43:55.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:55.204+0000 7fdee1fad700 1 --2- 172.21.15.52:0/274489145 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdedc104090 0x7fdedc10a940 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:43:55.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:55.204+0000 7fdee1fad700 1 --2- 172.21.15.52:0/274489145 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdedc1024f0 0x7fdedc102910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:43:55.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:55.204+0000 7fdee1fad700 1 --2- 172.21.15.52:0/274489145 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdedc1036f0 0x7fdedc103b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:43:55.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:55.204+0000 7fdee1fad700 1 -- 172.21.15.52:0/274489145 >> 172.21.15.52:0/274489145 conn(0x7fdedc0fda90 msgr2=0x7fdedc0ffed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:43:55.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:55.204+0000 7fdee1fad700 1 -- 172.21.15.52:0/274489145 shutdown_connections 2024-06-10T02:43:55.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:55.204+0000 7fdee1fad700 1 -- 172.21.15.52:0/274489145 wait complete. 2024-06-10T02:43:55.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:55.205+0000 7fdee1fad700 1 Processor -- start 2024-06-10T02:43:55.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:55.205+0000 7fdee1fad700 1 -- start start 2024-06-10T02:43:55.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:55.205+0000 7fdee1fad700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdedc1024f0 0x7fdedc19a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:43:55.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:55.205+0000 7fdee1fad700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdedc1036f0 0x7fdedc19ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:43:55.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:55.205+0000 7fdee1fad700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdedc104090 0x7fdedc1a1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:43:55.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:55.205+0000 7fdee1fad700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fdedc10e150 con 0x7fdedc1024f0 2024-06-10T02:43:55.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:55.205+0000 7fdee1fad700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fdedc10dfd0 con 0x7fdedc104090 2024-06-10T02:43:55.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:55.205+0000 7fdee1fad700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fdedc10e2d0 con 0x7fdedc1036f0 2024-06-10T02:43:55.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:55.205+0000 7fdedb7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdedc1024f0 0x7fdedc19a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:43:55.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:55.205+0000 7fdedb7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdedc1024f0 0x7fdedc19a910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52468/0 (socket says 172.21.15.52:52468) 2024-06-10T02:43:55.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:55.205+0000 7fdedb7fe700 1 -- 172.21.15.52:0/3339990466 learned_addr learned my addr 172.21.15.52:0/3339990466 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:43:55.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:55.205+0000 7fdedaffd700 1 --2- 172.21.15.52:0/3339990466 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdedc1036f0 0x7fdedc19ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:43:55.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:55.205+0000 7fdedbfff700 1 --2- 172.21.15.52:0/3339990466 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdedc104090 0x7fdedc1a1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:43:55.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:55.205+0000 7fdedb7fe700 1 -- 172.21.15.52:0/3339990466 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdedc1036f0 msgr2=0x7fdedc19ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:43:55.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:55.205+0000 7fdedb7fe700 1 --2- 172.21.15.52:0/3339990466 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdedc1036f0 0x7fdedc19ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:43:55.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:55.205+0000 7fdedb7fe700 1 -- 172.21.15.52:0/3339990466 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdedc104090 msgr2=0x7fdedc1a1ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:43:55.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:55.205+0000 7fdedb7fe700 1 --2- 172.21.15.52:0/3339990466 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdedc104090 0x7fdedc1a1ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:43:55.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:55.205+0000 7fdedb7fe700 1 -- 172.21.15.52:0/3339990466 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdedc1a2570 con 0x7fdedc1024f0 2024-06-10T02:43:55.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:55.206+0000 7fdedbfff700 1 --2- 172.21.15.52:0/3339990466 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdedc104090 0x7fdedc1a1ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T02:43:55.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:55.206+0000 7fdedb7fe700 1 --2- 172.21.15.52:0/3339990466 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdedc1024f0 0x7fdedc19a910 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fdecc00e980 tx=0x7fdecc00ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:43:55.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:55.206+0000 7fded8ff9700 1 -- 172.21.15.52:0/3339990466 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdecc00cbd0 con 0x7fdedc1024f0 2024-06-10T02:43:55.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:55.206+0000 7fded8ff9700 1 -- 172.21.15.52:0/3339990466 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdecc004510 con 0x7fdedc1024f0 2024-06-10T02:43:55.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:55.206+0000 7fdee1fad700 1 -- 172.21.15.52:0/3339990466 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdedc1a2860 con 0x7fdedc1024f0 2024-06-10T02:43:55.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:55.206+0000 7fded8ff9700 1 -- 172.21.15.52:0/3339990466 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdecc010430 con 0x7fdedc1024f0 2024-06-10T02:43:55.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:55.206+0000 7fdee1fad700 1 -- 172.21.15.52:0/3339990466 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdedc1a2da0 con 0x7fdedc1024f0 2024-06-10T02:43:55.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:55.207+0000 7fded8ff9700 1 -- 172.21.15.52:0/3339990466 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fdecc0105d0 con 0x7fdedc1024f0 2024-06-10T02:43:55.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:55.207+0000 7fdee1fad700 1 -- 172.21.15.52:0/3339990466 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdedc069180 con 0x7fdedc1024f0 2024-06-10T02:43:55.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:55.207+0000 7fded8ff9700 1 --2- 172.21.15.52:0/3339990466 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fdec4039330 0x7fdec403b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:43:55.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:55.207+0000 7fded8ff9700 1 -- 172.21.15.52:0/3339990466 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fdecc04c5f0 con 0x7fdedc1024f0 2024-06-10T02:43:55.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:55.207+0000 7fdedaffd700 1 --2- 172.21.15.52:0/3339990466 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fdec4039330 0x7fdec403b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:43:55.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:55.208+0000 7fdedaffd700 1 --2- 172.21.15.52:0/3339990466 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fdec4039330 0x7fdec403b7f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fded0009a10 tx=0x7fded0009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:43:55.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:55.212+0000 7fded8ff9700 1 -- 172.21.15.52:0/3339990466 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fdecc01fc00 con 0x7fdedc1024f0 2024-06-10T02:43:55.341 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Jun 10 02:43:55 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mgr-z[38427]: 2024-06-10T02:43:55.039+0000 7f57659d5200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-06-10T02:43:55.341 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Jun 10 02:43:55 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mgr-z[38427]: 2024-06-10T02:43:55.344+0000 7f57659d5200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-06-10T02:43:55.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:55.440+0000 7fdee1fad700 1 -- 172.21.15.52:0/3339990466 --> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi052:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fdedc075de0 con 0x7fdec4039330 2024-06-10T02:43:55.576 INFO:journalctl@ceph.mgr.y.smithi120.stdout:Jun 10 02:43:55 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mgr-y[38015]: 2024-06-10T02:43:55.390+0000 7f7eb50f2200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-06-10T02:43:55.577 INFO:journalctl@ceph.mgr.y.smithi120.stdout:Jun 10 02:43:55 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mgr-y[38015]: 2024-06-10T02:43:55.579+0000 7f7eb50f2200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-06-10T02:43:55.654 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Jun 10 02:43:55 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mgr-z[38427]: 2024-06-10T02:43:55.444+0000 7f57659d5200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-06-10T02:43:55.655 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Jun 10 02:43:55 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mgr-z[38427]: 2024-06-10T02:43:55.658+0000 7f57659d5200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-06-10T02:43:55.885 INFO:journalctl@ceph.mgr.y.smithi120.stdout:Jun 10 02:43:55 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mgr-y[38015]: 2024-06-10T02:43:55.688+0000 7f7eb50f2200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-06-10T02:43:55.950 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Jun 10 02:43:55 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mgr-z[38427]: 2024-06-10T02:43:55.862+0000 7f57659d5200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-06-10T02:43:56.308 INFO:journalctl@ceph.mgr.y.smithi120.stdout:Jun 10 02:43:56 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mgr-y[38015]: 2024-06-10T02:43:56.015+0000 7f7eb50f2200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-06-10T02:43:56.308 INFO:journalctl@ceph.mgr.y.smithi120.stdout:Jun 10 02:43:56 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mgr-y[38015]: 2024-06-10T02:43:56.114+0000 7f7eb50f2200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-06-10T02:43:56.596 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Jun 10 02:43:56 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mgr-z[38427]: 2024-06-10T02:43:56.501+0000 7f57659d5200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-06-10T02:43:56.596 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:43:56 smithi188 ceph-mon[36599]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T02:43:56.596 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:43:56 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:43:56.596 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:43:56 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:43:56.596 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:43:56 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:43:56.596 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:43:56 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:43:56.596 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:43:56 smithi188 ceph-mon[36599]: from='client.14220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi052:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T02:43:56.596 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:43:56 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-10T02:43:56.597 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:43:56 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-10T02:43:56.597 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:43:56 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:43:56.597 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:43:56 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T02:43:56.635 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:43:56 smithi120 ceph-mon[36634]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T02:43:56.635 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:43:56 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:43:56.636 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:43:56 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:43:56.636 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:43:56 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:43:56.636 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:43:56 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:43:56.636 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:43:56 smithi120 ceph-mon[36634]: from='client.14220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi052:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T02:43:56.636 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:43:56 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-10T02:43:56.636 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:43:56 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-10T02:43:56.636 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:43:56 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:43:56.636 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:43:56 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T02:43:56.687 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:56 smithi052 ceph-mon[32362]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T02:43:56.688 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:56 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:43:56.688 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:56 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:43:56.688 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:56 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:43:56.688 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:56 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:43:56.688 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:56 smithi052 ceph-mon[32362]: from='client.14220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi052:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T02:43:56.688 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:56 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-10T02:43:56.688 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:56 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-10T02:43:56.688 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:56 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:43:56.688 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:56 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T02:43:57.200 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Jun 10 02:43:56 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mgr-z[38427]: 2024-06-10T02:43:56.936+0000 7f57659d5200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-06-10T02:43:57.200 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Jun 10 02:43:57 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mgr-z[38427]: 2024-06-10T02:43:57.033+0000 7f57659d5200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-06-10T02:43:57.635 INFO:journalctl@ceph.mgr.y.smithi120.stdout:Jun 10 02:43:57 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mgr-y[38015]: 2024-06-10T02:43:57.336+0000 7f7eb50f2200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-06-10T02:43:57.635 INFO:journalctl@ceph.mgr.y.smithi120.stdout:Jun 10 02:43:57 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mgr-y[38015]: 2024-06-10T02:43:57.503+0000 7f7eb50f2200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-06-10T02:43:57.700 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Jun 10 02:43:57 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mgr-z[38427]: 2024-06-10T02:43:57.235+0000 7f57659d5200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-06-10T02:43:57.955 INFO:journalctl@ceph.mgr.y.smithi120.stdout:Jun 10 02:43:57 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mgr-y[38015]: 2024-06-10T02:43:57.672+0000 7f7eb50f2200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-06-10T02:43:57.955 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:43:57 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:43:57.955 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:43:57 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:43:57.956 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:43:57 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:43:57.956 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:43:57 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:43:57.956 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:43:57 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:43:57.956 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:43:57 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T02:43:57.956 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:43:57 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:43:57.956 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:43:57 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-10T02:43:57.956 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:43:57 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-10T02:43:57.956 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:43:57 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:43:58.082 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Jun 10 02:43:57 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mgr-z[38427]: 2024-06-10T02:43:57.750+0000 7f57659d5200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-06-10T02:43:58.082 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Jun 10 02:43:57 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mgr-z[38427]: 2024-06-10T02:43:57.842+0000 7f57659d5200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-06-10T02:43:58.082 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:43:57 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:43:58.082 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:43:57 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:43:58.082 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:43:57 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:43:58.082 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:43:57 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:43:58.082 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:43:57 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:43:58.082 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:43:57 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T02:43:58.082 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:43:57 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:43:58.083 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:43:57 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-10T02:43:58.083 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:43:57 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-10T02:43:58.083 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:43:57 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:43:58.223 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:57 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:43:58.224 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:57 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:43:58.224 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:57 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:43:58.224 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:57 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:43:58.224 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:57 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:43:58.224 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:57 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T02:43:58.224 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:57 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:43:58.224 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:57 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-10T02:43:58.224 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:57 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-10T02:43:58.224 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:57 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:43:58.245 INFO:journalctl@ceph.mgr.y.smithi120.stdout:Jun 10 02:43:57 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mgr-y[38015]: 2024-06-10T02:43:57.958+0000 7f7eb50f2200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-06-10T02:43:58.245 INFO:journalctl@ceph.mgr.y.smithi120.stdout:Jun 10 02:43:58 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mgr-y[38015]: 2024-06-10T02:43:58.049+0000 7f7eb50f2200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-06-10T02:43:58.245 INFO:journalctl@ceph.mgr.y.smithi120.stdout:Jun 10 02:43:58 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mgr-y[38015]: 2024-06-10T02:43:58.248+0000 7f7eb50f2200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-06-10T02:43:58.430 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Jun 10 02:43:58 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mgr-z[38427]: 2024-06-10T02:43:58.085+0000 7f57659d5200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-06-10T02:43:58.430 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Jun 10 02:43:58 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mgr-z[38427]: 2024-06-10T02:43:58.221+0000 7f57659d5200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-06-10T02:43:58.430 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Jun 10 02:43:58 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mgr-z[38427]: 2024-06-10T02:43:58.316+0000 7f57659d5200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-06-10T02:43:58.430 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Jun 10 02:43:58 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mgr-z[38427]: 2024-06-10T02:43:58.433+0000 7f57659d5200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-06-10T02:43:58.635 INFO:journalctl@ceph.mgr.y.smithi120.stdout:Jun 10 02:43:58 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mgr-y[38015]: 2024-06-10T02:43:58.437+0000 7f7eb50f2200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-06-10T02:43:59.026 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:43:58 smithi120 ceph-mon[36634]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T02:43:59.026 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:43:58 smithi120 ceph-mon[36634]: Reconfiguring mgr.x (unknown last config time)... 2024-06-10T02:43:59.026 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:43:58 smithi120 ceph-mon[36634]: Reconfiguring daemon mgr.x on smithi052 2024-06-10T02:43:59.027 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:43:58 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:43:59.027 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:43:58 smithi120 ceph-mon[36634]: from='client.? 172.21.15.52:0/4133640758' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e03c0691-cc5a-497a-bbc2-ce18c337f074"}]: dispatch 2024-06-10T02:43:59.027 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:43:58 smithi120 ceph-mon[36634]: from='client.? 172.21.15.52:0/4133640758' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e03c0691-cc5a-497a-bbc2-ce18c337f074"}]': finished 2024-06-10T02:43:59.027 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:43:58 smithi120 ceph-mon[36634]: osdmap e5: 1 total, 0 up, 1 in 2024-06-10T02:43:59.027 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:43:58 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-10T02:43:59.200 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Jun 10 02:43:58 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mgr-z[38427]: 2024-06-10T02:43:58.734+0000 7f57659d5200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-06-10T02:43:59.200 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:43:58 smithi188 ceph-mon[36599]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T02:43:59.200 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:43:58 smithi188 ceph-mon[36599]: Reconfiguring mgr.x (unknown last config time)... 2024-06-10T02:43:59.201 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:43:58 smithi188 ceph-mon[36599]: Reconfiguring daemon mgr.x on smithi052 2024-06-10T02:43:59.201 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:43:58 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:43:59.201 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:43:58 smithi188 ceph-mon[36599]: from='client.? 172.21.15.52:0/4133640758' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e03c0691-cc5a-497a-bbc2-ce18c337f074"}]: dispatch 2024-06-10T02:43:59.201 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:43:58 smithi188 ceph-mon[36599]: from='client.? 172.21.15.52:0/4133640758' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e03c0691-cc5a-497a-bbc2-ce18c337f074"}]': finished 2024-06-10T02:43:59.201 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:43:58 smithi188 ceph-mon[36599]: osdmap e5: 1 total, 0 up, 1 in 2024-06-10T02:43:59.201 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:43:58 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-10T02:43:59.224 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:58 smithi052 ceph-mon[32362]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T02:43:59.224 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:58 smithi052 ceph-mon[32362]: Reconfiguring mgr.x (unknown last config time)... 2024-06-10T02:43:59.224 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:58 smithi052 ceph-mon[32362]: Reconfiguring daemon mgr.x on smithi052 2024-06-10T02:43:59.224 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:58 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:43:59.224 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:58 smithi052 ceph-mon[32362]: from='client.? 172.21.15.52:0/4133640758' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e03c0691-cc5a-497a-bbc2-ce18c337f074"}]: dispatch 2024-06-10T02:43:59.224 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:58 smithi052 ceph-mon[32362]: from='client.? 172.21.15.52:0/4133640758' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e03c0691-cc5a-497a-bbc2-ce18c337f074"}]': finished 2024-06-10T02:43:59.224 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:58 smithi052 ceph-mon[32362]: osdmap e5: 1 total, 0 up, 1 in 2024-06-10T02:43:59.224 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:58 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-10T02:43:59.385 INFO:journalctl@ceph.mgr.y.smithi120.stdout:Jun 10 02:43:59 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mgr-y[38015]: 2024-06-10T02:43:59.029+0000 7f7eb50f2200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-06-10T02:43:59.700 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Jun 10 02:43:59 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mgr-z[38427]: 2024-06-10T02:43:59.336+0000 7f57659d5200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-06-10T02:43:59.700 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Jun 10 02:43:59 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mgr-z[38427]: 2024-06-10T02:43:59.554+0000 7f57659d5200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-06-10T02:43:59.733 INFO:journalctl@ceph.mgr.y.smithi120.stdout:Jun 10 02:43:59 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mgr-y[38015]: 2024-06-10T02:43:59.445+0000 7f7eb50f2200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-06-10T02:43:59.734 INFO:journalctl@ceph.mgr.y.smithi120.stdout:Jun 10 02:43:59 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mgr-y[38015]: 2024-06-10T02:43:59.540+0000 7f7eb50f2200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-06-10T02:43:59.734 INFO:journalctl@ceph.mgr.y.smithi120.stdout:Jun 10 02:43:59 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mgr-y[38015]: 2024-06-10T02:43:59.736+0000 7f7eb50f2200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-06-10T02:43:59.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:43:59.782+0000 7fded8ff9700 1 -- 172.21.15.52:0/3339990466 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mgrmap(e 14) ==== 92060+0+0 (secure 0 0 0) 0x7fdecc0217c0 con 0x7fdedc1024f0 2024-06-10T02:44:00.135 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:43:59 smithi120 ceph-mon[36634]: from='client.? 172.21.15.52:0/2859735636' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-10T02:44:00.135 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:43:59 smithi120 ceph-mon[36634]: Standby manager daemon z started 2024-06-10T02:44:00.135 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:43:59 smithi120 ceph-mon[36634]: from='mgr.? 172.21.15.188:0/3294187188' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-06-10T02:44:00.135 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:43:59 smithi120 ceph-mon[36634]: from='mgr.? 172.21.15.188:0/3294187188' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-10T02:44:00.135 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:43:59 smithi120 ceph-mon[36634]: from='mgr.? 172.21.15.188:0/3294187188' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-06-10T02:44:00.136 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:43:59 smithi120 ceph-mon[36634]: from='mgr.? 172.21.15.188:0/3294187188' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-10T02:44:00.200 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:43:59 smithi188 ceph-mon[36599]: from='client.? 172.21.15.52:0/2859735636' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-10T02:44:00.200 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:43:59 smithi188 ceph-mon[36599]: Standby manager daemon z started 2024-06-10T02:44:00.200 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:43:59 smithi188 ceph-mon[36599]: from='mgr.? 172.21.15.188:0/3294187188' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-06-10T02:44:00.200 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:43:59 smithi188 ceph-mon[36599]: from='mgr.? 172.21.15.188:0/3294187188' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-10T02:44:00.201 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:43:59 smithi188 ceph-mon[36599]: from='mgr.? 172.21.15.188:0/3294187188' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-06-10T02:44:00.201 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:43:59 smithi188 ceph-mon[36599]: from='mgr.? 172.21.15.188:0/3294187188' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-10T02:44:00.223 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:59 smithi052 ceph-mon[32362]: from='client.? 172.21.15.52:0/2859735636' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-10T02:44:00.224 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:59 smithi052 ceph-mon[32362]: Standby manager daemon z started 2024-06-10T02:44:00.224 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:59 smithi052 ceph-mon[32362]: from='mgr.? 172.21.15.188:0/3294187188' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-06-10T02:44:00.224 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:59 smithi052 ceph-mon[32362]: from='mgr.? 172.21.15.188:0/3294187188' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-10T02:44:00.224 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:59 smithi052 ceph-mon[32362]: from='mgr.? 172.21.15.188:0/3294187188' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-06-10T02:44:00.224 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:43:59 smithi052 ceph-mon[32362]: from='mgr.? 172.21.15.188:0/3294187188' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-10T02:44:00.578 INFO:journalctl@ceph.mgr.y.smithi120.stdout:Jun 10 02:44:00 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mgr-y[38015]: 2024-06-10T02:44:00.253+0000 7f7eb50f2200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-06-10T02:44:00.578 INFO:journalctl@ceph.mgr.y.smithi120.stdout:Jun 10 02:44:00 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mgr-y[38015]: 2024-06-10T02:44:00.344+0000 7f7eb50f2200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-06-10T02:44:00.885 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:44:00 smithi120 ceph-mon[36634]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T02:44:00.885 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:44:00 smithi120 ceph-mon[36634]: mgrmap e14: x(active, since 76s), standbys: z 2024-06-10T02:44:00.885 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:44:00 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-06-10T02:44:00.885 INFO:journalctl@ceph.mgr.y.smithi120.stdout:Jun 10 02:44:00 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mgr-y[38015]: 2024-06-10T02:44:00.581+0000 7f7eb50f2200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-06-10T02:44:00.885 INFO:journalctl@ceph.mgr.y.smithi120.stdout:Jun 10 02:44:00 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mgr-y[38015]: 2024-06-10T02:44:00.713+0000 7f7eb50f2200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-06-10T02:44:00.885 INFO:journalctl@ceph.mgr.y.smithi120.stdout:Jun 10 02:44:00 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mgr-y[38015]: 2024-06-10T02:44:00.806+0000 7f7eb50f2200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-06-10T02:44:01.200 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:44:00 smithi188 ceph-mon[36599]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T02:44:01.200 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:44:00 smithi188 ceph-mon[36599]: mgrmap e14: x(active, since 76s), standbys: z 2024-06-10T02:44:01.200 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:44:00 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-06-10T02:44:01.213 INFO:journalctl@ceph.mgr.y.smithi120.stdout:Jun 10 02:44:00 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mgr-y[38015]: 2024-06-10T02:44:00.921+0000 7f7eb50f2200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-06-10T02:44:01.223 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:44:00 smithi052 ceph-mon[32362]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T02:44:01.223 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:44:00 smithi052 ceph-mon[32362]: mgrmap e14: x(active, since 76s), standbys: z 2024-06-10T02:44:01.224 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:44:00 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-06-10T02:44:02.130 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:44:01 smithi052 ceph-mon[32362]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T02:44:02.135 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:44:01 smithi120 ceph-mon[36634]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T02:44:02.135 INFO:journalctl@ceph.mgr.y.smithi120.stdout:Jun 10 02:44:01 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mgr-y[38015]: 2024-06-10T02:44:01.216+0000 7f7eb50f2200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-06-10T02:44:02.135 INFO:journalctl@ceph.mgr.y.smithi120.stdout:Jun 10 02:44:01 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mgr-y[38015]: 2024-06-10T02:44:01.810+0000 7f7eb50f2200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-06-10T02:44:02.135 INFO:journalctl@ceph.mgr.y.smithi120.stdout:Jun 10 02:44:02 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mgr-y[38015]: 2024-06-10T02:44:02.021+0000 7f7eb50f2200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-06-10T02:44:02.200 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:44:01 smithi188 ceph-mon[36599]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T02:44:02.794 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:44:02.797+0000 7fded8ff9700 1 -- 172.21.15.52:0/3339990466 <== mon.0 v2:172.21.15.52:3300/0 8 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7fdecc016370 con 0x7fdedc1024f0 2024-06-10T02:44:03.135 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:44:02 smithi120 ceph-mon[36634]: Standby manager daemon y started 2024-06-10T02:44:03.135 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:44:02 smithi120 ceph-mon[36634]: from='mgr.? 172.21.15.120:0/3095244017' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-06-10T02:44:03.135 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:44:02 smithi120 ceph-mon[36634]: from='mgr.? 172.21.15.120:0/3095244017' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-10T02:44:03.136 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:44:02 smithi120 ceph-mon[36634]: from='mgr.? 172.21.15.120:0/3095244017' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-06-10T02:44:03.136 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:44:02 smithi120 ceph-mon[36634]: from='mgr.? 172.21.15.120:0/3095244017' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-10T02:44:03.200 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:44:02 smithi188 ceph-mon[36599]: Standby manager daemon y started 2024-06-10T02:44:03.200 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:44:02 smithi188 ceph-mon[36599]: from='mgr.? 172.21.15.120:0/3095244017' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-06-10T02:44:03.200 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:44:02 smithi188 ceph-mon[36599]: from='mgr.? 172.21.15.120:0/3095244017' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-10T02:44:03.200 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:44:02 smithi188 ceph-mon[36599]: from='mgr.? 172.21.15.120:0/3095244017' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-06-10T02:44:03.200 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:44:02 smithi188 ceph-mon[36599]: from='mgr.? 172.21.15.120:0/3095244017' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-10T02:44:03.223 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:44:02 smithi052 ceph-mon[32362]: Standby manager daemon y started 2024-06-10T02:44:03.223 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:44:02 smithi052 ceph-mon[32362]: from='mgr.? 172.21.15.120:0/3095244017' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-06-10T02:44:03.224 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:44:02 smithi052 ceph-mon[32362]: from='mgr.? 172.21.15.120:0/3095244017' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-10T02:44:03.224 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:44:02 smithi052 ceph-mon[32362]: from='mgr.? 172.21.15.120:0/3095244017' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-06-10T02:44:03.224 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:44:02 smithi052 ceph-mon[32362]: from='mgr.? 172.21.15.120:0/3095244017' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-10T02:44:03.893 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:44:03 smithi052 ceph-mon[32362]: mgrmap e15: x(active, since 79s), standbys: z, y 2024-06-10T02:44:03.893 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:44:03 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-06-10T02:44:03.893 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:44:03 smithi052 ceph-mon[32362]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T02:44:03.893 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:44:03 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:44:03.893 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:44:03 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:44:03.893 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:44:03 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T02:44:03.893 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:44:03 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:44:03.893 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:44:03 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T02:44:03.893 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:44:03 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:44:04.135 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:44:03 smithi120 ceph-mon[36634]: mgrmap e15: x(active, since 79s), standbys: z, y 2024-06-10T02:44:04.135 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:44:03 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-06-10T02:44:04.135 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:44:03 smithi120 ceph-mon[36634]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T02:44:04.135 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:44:03 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:44:04.135 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:44:03 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:44:04.135 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:44:03 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T02:44:04.136 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:44:03 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:44:04.136 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:44:03 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T02:44:04.136 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:44:03 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:44:04.200 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:44:03 smithi188 ceph-mon[36599]: mgrmap e15: x(active, since 79s), standbys: z, y 2024-06-10T02:44:04.200 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:44:03 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-06-10T02:44:04.200 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:44:03 smithi188 ceph-mon[36599]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T02:44:04.201 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:44:03 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:44:04.201 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:44:03 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:44:04.201 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:44:03 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T02:44:04.201 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:44:03 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:44:04.201 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:44:03 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T02:44:04.201 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:44:03 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:44:06.200 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:44:05 smithi188 ceph-mon[36599]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T02:44:06.223 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:44:05 smithi052 ceph-mon[32362]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T02:44:06.385 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:44:05 smithi120 ceph-mon[36634]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T02:44:07.200 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:44:06 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-06-10T02:44:07.200 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:44:06 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:44:07.200 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:44:06 smithi188 ceph-mon[36599]: Deploying daemon osd.0 on smithi052 2024-06-10T02:44:07.223 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:44:06 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-06-10T02:44:07.224 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:44:06 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:44:07.224 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:44:06 smithi052 ceph-mon[32362]: Deploying daemon osd.0 on smithi052 2024-06-10T02:44:07.385 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:44:06 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-06-10T02:44:07.385 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:44:06 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:44:07.385 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:44:06 smithi120 ceph-mon[36634]: Deploying daemon osd.0 on smithi052 2024-06-10T02:44:08.200 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:44:07 smithi188 ceph-mon[36599]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T02:44:08.223 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:44:07 smithi052 ceph-mon[32362]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T02:44:08.385 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:44:07 smithi120 ceph-mon[36634]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T02:44:10.200 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:44:09 smithi188 ceph-mon[36599]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T02:44:10.224 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:44:09 smithi052 ceph-mon[32362]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T02:44:10.385 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:44:09 smithi120 ceph-mon[36634]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T02:44:12.848 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:44:12 smithi052 ceph-mon[32362]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T02:44:12.848 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:44:12 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:44:12.848 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:44:12 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:44:12.848 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:44:12 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T02:44:12.885 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:44:12 smithi120 ceph-mon[36634]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T02:44:12.885 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:44:12 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:44:12.885 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:44:12 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:44:12.885 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:44:12 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T02:44:12.950 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:44:12 smithi188 ceph-mon[36599]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T02:44:12.950 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:44:12 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:44:12.950 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:44:12 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:44:12.950 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:44:12 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T02:44:15.042 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:44:14 smithi052 ceph-mon[32362]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T02:44:15.042 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:44:14 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:44:15.043 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:44:14 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:44:15.043 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:44:14 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:44:15.043 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:44:14 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T02:44:15.043 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:44:14 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:44:15.093 INFO:teuthology.orchestra.run.smithi052.stdout:Created osd(s) 0 on host 'smithi052' 2024-06-10T02:44:15.094 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:44:15.097+0000 7fded8ff9700 1 -- 172.21.15.52:0/3339990466 <== mgr.14150 v2:172.21.15.52:6800/3214763969 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fdedc075de0 con 0x7fdec4039330 2024-06-10T02:44:15.096 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:44:15.099+0000 7fdee1fad700 1 -- 172.21.15.52:0/3339990466 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fdec4039330 msgr2=0x7fdec403b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:44:15.096 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:44:15.099+0000 7fdee1fad700 1 --2- 172.21.15.52:0/3339990466 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fdec4039330 0x7fdec403b7f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fded0009a10 tx=0x7fded0009190 comp rx=0 tx=0).stop 2024-06-10T02:44:15.096 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:44:15.099+0000 7fdee1fad700 1 -- 172.21.15.52:0/3339990466 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdedc1024f0 msgr2=0x7fdedc19a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:44:15.096 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:44:15.099+0000 7fdee1fad700 1 --2- 172.21.15.52:0/3339990466 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdedc1024f0 0x7fdedc19a910 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fdecc00e980 tx=0x7fdecc00ec90 comp rx=0 tx=0).stop 2024-06-10T02:44:15.096 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:44:15.099+0000 7fdee1fad700 1 -- 172.21.15.52:0/3339990466 shutdown_connections 2024-06-10T02:44:15.096 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:44:15.099+0000 7fdee1fad700 1 --2- 172.21.15.52:0/3339990466 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdedc104090 0x7fdedc1a1ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:44:15.096 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:44:15.099+0000 7fdee1fad700 1 --2- 172.21.15.52:0/3339990466 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fdec4039330 0x7fdec403b7f0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:44:15.096 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:44:15.099+0000 7fdee1fad700 1 --2- 172.21.15.52:0/3339990466 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdedc1024f0 0x7fdedc19a910 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:44:15.096 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:44:15.099+0000 7fdee1fad700 1 --2- 172.21.15.52:0/3339990466 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdedc1036f0 0x7fdedc19ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:44:15.096 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:44:15.099+0000 7fdee1fad700 1 -- 172.21.15.52:0/3339990466 >> 172.21.15.52:0/3339990466 conn(0x7fdedc0fda90 msgr2=0x7fdedc0746c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:44:15.097 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:44:15.099+0000 7fdee1fad700 1 -- 172.21.15.52:0/3339990466 shutdown_connections 2024-06-10T02:44:15.097 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:44:15.099+0000 7fdee1fad700 1 -- 172.21.15.52:0/3339990466 wait complete. 2024-06-10T02:44:15.135 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:44:14 smithi120 ceph-mon[36634]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T02:44:15.135 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:44:14 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:44:15.135 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:44:14 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:44:15.135 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:44:14 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:44:15.135 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:44:14 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T02:44:15.135 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:44:14 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:44:15.200 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:44:14 smithi188 ceph-mon[36599]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T02:44:15.200 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:44:14 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:44:15.201 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:44:14 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:44:15.201 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:44:14 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:44:15.201 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:44:14 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T02:44:15.201 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:44:14 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:44:15.699 DEBUG:teuthology.orchestra.run.smithi052:osd.0> sudo journalctl -f -n 0 -u ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968@osd.0.service 2024-06-10T02:44:15.701 INFO:tasks.cephadm:Deploying osd.1 on smithi120 with /dev/vg_nvme/lv_4... 2024-06-10T02:44:15.702 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-06-10T02:44:15.946 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.b/config 2024-06-10T02:44:16.367 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:44:16 smithi120 ceph-mon[36634]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T02:44:16.368 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:44:16 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:44:16.368 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:44:16 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:44:16.368 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:44:16 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T02:44:16.368 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:44:16 smithi120 ceph-mon[36634]: from='osd.0 [v2:172.21.15.52:6802/4255833408,v1:172.21.15.52:6803/4255833408]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-06-10T02:44:16.450 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:44:16 smithi188 ceph-mon[36599]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T02:44:16.450 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:44:16 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:44:16.450 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:44:16 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:44:16.451 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:44:16 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T02:44:16.451 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:44:16 smithi188 ceph-mon[36599]: from='osd.0 [v2:172.21.15.52:6802/4255833408,v1:172.21.15.52:6803/4255833408]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-06-10T02:44:16.473 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:44:16 smithi052 ceph-mon[32362]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T02:44:16.474 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:44:16 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:44:16.474 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:44:16 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:44:16.474 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:44:16 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T02:44:16.474 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:44:16 smithi052 ceph-mon[32362]: from='osd.0 [v2:172.21.15.52:6802/4255833408,v1:172.21.15.52:6803/4255833408]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-06-10T02:44:17.244 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:44:17 smithi120 ceph-mon[36634]: from='osd.0 [v2:172.21.15.52:6802/4255833408,v1:172.21.15.52:6803/4255833408]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-06-10T02:44:17.244 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:44:17 smithi120 ceph-mon[36634]: osdmap e6: 1 total, 0 up, 1 in 2024-06-10T02:44:17.244 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:44:17 smithi120 ceph-mon[36634]: from='osd.0 [v2:172.21.15.52:6802/4255833408,v1:172.21.15.52:6803/4255833408]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]: dispatch 2024-06-10T02:44:17.244 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:44:17 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-10T02:44:17.450 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:44:17 smithi188 ceph-mon[36599]: from='osd.0 [v2:172.21.15.52:6802/4255833408,v1:172.21.15.52:6803/4255833408]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-06-10T02:44:17.450 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:44:17 smithi188 ceph-mon[36599]: osdmap e6: 1 total, 0 up, 1 in 2024-06-10T02:44:17.450 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:44:17 smithi188 ceph-mon[36599]: from='osd.0 [v2:172.21.15.52:6802/4255833408,v1:172.21.15.52:6803/4255833408]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]: dispatch 2024-06-10T02:44:17.451 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:44:17 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-10T02:44:17.474 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:44:17 smithi052 ceph-mon[32362]: from='osd.0 [v2:172.21.15.52:6802/4255833408,v1:172.21.15.52:6803/4255833408]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-06-10T02:44:17.474 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:44:17 smithi052 ceph-mon[32362]: osdmap e6: 1 total, 0 up, 1 in 2024-06-10T02:44:17.474 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:44:17 smithi052 ceph-mon[32362]: from='osd.0 [v2:172.21.15.52:6802/4255833408,v1:172.21.15.52:6803/4255833408]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]: dispatch 2024-06-10T02:44:17.474 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:44:17 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-10T02:44:17.474 INFO:journalctl@ceph.osd.0.smithi052.stdout:Jun 10 02:44:17 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-0[46981]: 2024-06-10T02:44:17.091+0000 7f280b858700 -1 osd.0 0 waiting for initial osdmap 2024-06-10T02:44:17.474 INFO:journalctl@ceph.osd.0.smithi052.stdout:Jun 10 02:44:17 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-0[46981]: 2024-06-10T02:44:17.094+0000 7f280764b700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-10T02:44:17.778 INFO:teuthology.orchestra.run.smithi120.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-06-10T02:44:17.778 INFO:teuthology.orchestra.run.smithi120.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-06-10T02:44:17.778 INFO:teuthology.orchestra.run.smithi120.stderr: stderr: 10+0 records in 2024-06-10T02:44:17.778 INFO:teuthology.orchestra.run.smithi120.stderr:10+0 records out 2024-06-10T02:44:17.778 INFO:teuthology.orchestra.run.smithi120.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0110727 s, 947 MB/s 2024-06-10T02:44:17.778 INFO:teuthology.orchestra.run.smithi120.stderr:--> Zapping successful for: 2024-06-10T02:44:18.223 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:44:18 smithi052 ceph-mon[32362]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T02:44:18.224 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:44:18 smithi052 ceph-mon[32362]: from='osd.0 [v2:172.21.15.52:6802/4255833408,v1:172.21.15.52:6803/4255833408]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]': finished 2024-06-10T02:44:18.224 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:44:18 smithi052 ceph-mon[32362]: osdmap e7: 1 total, 0 up, 1 in 2024-06-10T02:44:18.224 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:44:18 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-10T02:44:18.224 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:44:18 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-10T02:44:18.385 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:44:18 smithi120 ceph-mon[36634]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T02:44:18.385 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:44:18 smithi120 ceph-mon[36634]: from='osd.0 [v2:172.21.15.52:6802/4255833408,v1:172.21.15.52:6803/4255833408]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]': finished 2024-06-10T02:44:18.385 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:44:18 smithi120 ceph-mon[36634]: osdmap e7: 1 total, 0 up, 1 in 2024-06-10T02:44:18.385 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:44:18 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-10T02:44:18.385 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:44:18 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-10T02:44:18.450 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:44:18 smithi188 ceph-mon[36599]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T02:44:18.450 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:44:18 smithi188 ceph-mon[36599]: from='osd.0 [v2:172.21.15.52:6802/4255833408,v1:172.21.15.52:6803/4255833408]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]': finished 2024-06-10T02:44:18.450 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:44:18 smithi188 ceph-mon[36599]: osdmap e7: 1 total, 0 up, 1 in 2024-06-10T02:44:18.450 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:44:18 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-10T02:44:18.451 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:44:18 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-10T02:44:18.473 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi120:vg_nvme/lv_4 2024-06-10T02:44:18.715 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.b/config 2024-06-10T02:44:19.383 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:44:19 smithi120 ceph-mon[36634]: purged_snaps scrub starts 2024-06-10T02:44:19.383 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:44:19 smithi120 ceph-mon[36634]: purged_snaps scrub ok 2024-06-10T02:44:19.383 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:44:19 smithi120 ceph-mon[36634]: osd.0 [v2:172.21.15.52:6802/4255833408,v1:172.21.15.52:6803/4255833408] boot 2024-06-10T02:44:19.383 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:44:19 smithi120 ceph-mon[36634]: osdmap e8: 1 total, 1 up, 1 in 2024-06-10T02:44:19.383 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:44:19 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-10T02:44:19.384 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:44:19 smithi120 ceph-mon[36634]: Detected new or changed devices on smithi052 2024-06-10T02:44:19.384 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:44:19 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:44:19.384 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:44:19 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:44:19.384 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:44:19 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-10T02:44:19.384 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:44:19 smithi120 ceph-mon[36634]: Adjusting osd_memory_target on smithi052 to 17092M 2024-06-10T02:44:19.384 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:44:19 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:44:19.384 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:44:19 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:44:19.384 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:44:19 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T02:44:19.384 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:44:19 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:44:19.450 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:44:19 smithi188 ceph-mon[36599]: purged_snaps scrub starts 2024-06-10T02:44:19.451 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:44:19 smithi188 ceph-mon[36599]: purged_snaps scrub ok 2024-06-10T02:44:19.451 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:44:19 smithi188 ceph-mon[36599]: osd.0 [v2:172.21.15.52:6802/4255833408,v1:172.21.15.52:6803/4255833408] boot 2024-06-10T02:44:19.451 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:44:19 smithi188 ceph-mon[36599]: osdmap e8: 1 total, 1 up, 1 in 2024-06-10T02:44:19.451 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:44:19 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-10T02:44:19.451 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:44:19 smithi188 ceph-mon[36599]: Detected new or changed devices on smithi052 2024-06-10T02:44:19.451 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:44:19 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:44:19.451 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:44:19 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:44:19.451 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:44:19 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-10T02:44:19.452 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:44:19 smithi188 ceph-mon[36599]: Adjusting osd_memory_target on smithi052 to 17092M 2024-06-10T02:44:19.452 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:44:19 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:44:19.452 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:44:19 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:44:19.452 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:44:19 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T02:44:19.452 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:44:19 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:44:19.473 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:44:19 smithi052 ceph-mon[32362]: purged_snaps scrub starts 2024-06-10T02:44:19.474 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:44:19 smithi052 ceph-mon[32362]: purged_snaps scrub ok 2024-06-10T02:44:19.474 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:44:19 smithi052 ceph-mon[32362]: osd.0 [v2:172.21.15.52:6802/4255833408,v1:172.21.15.52:6803/4255833408] boot 2024-06-10T02:44:19.474 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:44:19 smithi052 ceph-mon[32362]: osdmap e8: 1 total, 1 up, 1 in 2024-06-10T02:44:19.474 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:44:19 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-10T02:44:19.474 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:44:19 smithi052 ceph-mon[32362]: Detected new or changed devices on smithi052 2024-06-10T02:44:19.474 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:44:19 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:44:19.474 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:44:19 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:44:19.474 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:44:19 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-10T02:44:19.474 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:44:19 smithi052 ceph-mon[32362]: Adjusting osd_memory_target on smithi052 to 17092M 2024-06-10T02:44:19.474 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:44:19 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:44:19.474 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:44:19 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:44:19.474 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:44:19 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T02:44:19.475 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:44:19 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:44:20.150 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:44:20.151+0000 7f6797bc4700 1 -- 172.21.15.120:0/691362784 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f67901036f0 msgr2=0x7f6790103b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:44:20.151 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:44:20.151+0000 7f6797bc4700 1 --2- 172.21.15.120:0/691362784 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f67901036f0 0x7f6790103b50 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f67840099c0 tx=0x7f6784009cd0 comp rx=0 tx=0).stop 2024-06-10T02:44:20.151 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:44:20.151+0000 7f6797bc4700 1 -- 172.21.15.120:0/691362784 shutdown_connections 2024-06-10T02:44:20.151 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:44:20.151+0000 7f6797bc4700 1 --2- 172.21.15.120:0/691362784 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6790104090 0x7f679010a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:44:20.151 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:44:20.151+0000 7f6797bc4700 1 --2- 172.21.15.120:0/691362784 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f67901024f0 0x7f6790102910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:44:20.151 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:44:20.151+0000 7f6797bc4700 1 --2- 172.21.15.120:0/691362784 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f67901036f0 0x7f6790103b50 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:44:20.151 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:44:20.151+0000 7f6797bc4700 1 -- 172.21.15.120:0/691362784 >> 172.21.15.120:0/691362784 conn(0x7f67900fda90 msgr2=0x7f67900ffed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:44:20.151 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:44:20.151+0000 7f6797bc4700 1 -- 172.21.15.120:0/691362784 shutdown_connections 2024-06-10T02:44:20.151 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:44:20.151+0000 7f6797bc4700 1 -- 172.21.15.120:0/691362784 wait complete. 2024-06-10T02:44:20.151 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:44:20.151+0000 7f6797bc4700 1 Processor -- start 2024-06-10T02:44:20.151 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:44:20.152+0000 7f6797bc4700 1 -- start start 2024-06-10T02:44:20.151 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:44:20.152+0000 7f6797bc4700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f67901024f0 0x7f67901987f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:44:20.151 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:44:20.152+0000 7f6797bc4700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f67901036f0 0x7f6790198d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:44:20.152 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:44:20.152+0000 7f6797bc4700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6790104090 0x7f679019fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:44:20.152 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:44:20.152+0000 7f6797bc4700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f679010e1a0 con 0x7f67901036f0 2024-06-10T02:44:20.152 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:44:20.152+0000 7f6797bc4700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f679010e020 con 0x7f6790104090 2024-06-10T02:44:20.152 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:44:20.152+0000 7f6797bc4700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f679010e320 con 0x7f67901024f0 2024-06-10T02:44:20.152 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:44:20.152+0000 7f6796161700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6790104090 0x7f679019fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:44:20.152 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:44:20.152+0000 7f6796161700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6790104090 0x7f679019fdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:35600/0 (socket says 172.21.15.120:35600) 2024-06-10T02:44:20.152 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:44:20.152+0000 7f6796161700 1 -- 172.21.15.120:0/3536399890 learned_addr learned my addr 172.21.15.120:0/3536399890 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-06-10T02:44:20.152 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:44:20.152+0000 7f6795960700 1 --2- 172.21.15.120:0/3536399890 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f67901024f0 0x7f67901987f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:44:20.152 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:44:20.152+0000 7f679515f700 1 --2- 172.21.15.120:0/3536399890 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f67901036f0 0x7f6790198d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:44:20.152 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:44:20.152+0000 7f6796161700 1 -- 172.21.15.120:0/3536399890 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f67901024f0 msgr2=0x7f67901987f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:44:20.152 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:44:20.152+0000 7f6796161700 1 --2- 172.21.15.120:0/3536399890 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f67901024f0 0x7f67901987f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:44:20.152 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:44:20.152+0000 7f6796161700 1 -- 172.21.15.120:0/3536399890 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f67901036f0 msgr2=0x7f6790198d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:44:20.153 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:44:20.152+0000 7f6796161700 1 --2- 172.21.15.120:0/3536399890 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f67901036f0 0x7f6790198d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:44:20.153 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:44:20.152+0000 7f6796161700 1 -- 172.21.15.120:0/3536399890 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67901a0450 con 0x7f6790104090 2024-06-10T02:44:20.153 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:44:20.152+0000 7f6795960700 1 --2- 172.21.15.120:0/3536399890 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f67901024f0 0x7f67901987f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T02:44:20.153 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:44:20.152+0000 7f6796161700 1 --2- 172.21.15.120:0/3536399890 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6790104090 0x7f679019fdb0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f678800c970 tx=0x7f678800cd30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:44:20.153 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:44:20.152+0000 7f678effd700 1 -- 172.21.15.120:0/3536399890 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6788007b00 con 0x7f6790104090 2024-06-10T02:44:20.153 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:44:20.153+0000 7f6797bc4700 1 -- 172.21.15.120:0/3536399890 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67901a0740 con 0x7f6790104090 2024-06-10T02:44:20.153 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:44:20.153+0000 7f678effd700 1 -- 172.21.15.120:0/3536399890 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6788004d10 con 0x7f6790104090 2024-06-10T02:44:20.153 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:44:20.153+0000 7f678effd700 1 -- 172.21.15.120:0/3536399890 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6788005650 con 0x7f6790104090 2024-06-10T02:44:20.153 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:44:20.153+0000 7f6797bc4700 1 -- 172.21.15.120:0/3536399890 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67901a0d00 con 0x7f6790104090 2024-06-10T02:44:20.153 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:44:20.154+0000 7f6797bc4700 1 -- 172.21.15.120:0/3536399890 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6790069180 con 0x7f6790104090 2024-06-10T02:44:20.154 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:44:20.155+0000 7f678effd700 1 -- 172.21.15.120:0/3536399890 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7f6788007ca0 con 0x7f6790104090 2024-06-10T02:44:20.154 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:44:20.156+0000 7f678effd700 1 --2- 172.21.15.120:0/3536399890 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f67780a2b90 0x7f67780a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:44:20.154 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:44:20.156+0000 7f678effd700 1 -- 172.21.15.120:0/3536399890 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1767+0+0 (secure 0 0 0) 0x7f67880cd630 con 0x7f6790104090 2024-06-10T02:44:20.154 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:44:20.156+0000 7f6795960700 1 --2- 172.21.15.120:0/3536399890 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f67780a2b90 0x7f67780a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:44:20.155 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:44:20.156+0000 7f6795960700 1 --2- 172.21.15.120:0/3536399890 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f67780a2b90 0x7f67780a5050 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f6790103550 tx=0x7f6780009570 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:44:20.156 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:44:20.160+0000 7f678effd700 1 -- 172.21.15.120:0/3536399890 <== mon.2 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f6788014030 con 0x7f6790104090 2024-06-10T02:44:20.357 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:44:20.360+0000 7f6797bc4700 1 -- 172.21.15.120:0/3536399890 --> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi120:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f67900628d0 con 0x7f67780a2b90 2024-06-10T02:44:20.357 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:44:20 smithi120 ceph-mon[36634]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T02:44:20.357 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:44:20 smithi120 ceph-mon[36634]: osdmap e9: 1 total, 1 up, 1 in 2024-06-10T02:44:20.450 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:44:20 smithi188 ceph-mon[36599]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T02:44:20.451 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:44:20 smithi188 ceph-mon[36599]: osdmap e9: 1 total, 1 up, 1 in 2024-06-10T02:44:20.473 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:44:20 smithi052 ceph-mon[32362]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T02:44:20.473 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:44:20 smithi052 ceph-mon[32362]: osdmap e9: 1 total, 1 up, 1 in 2024-06-10T02:44:21.263 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:44:21 smithi120 ceph-mon[36634]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi120:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T02:44:21.263 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:44:21 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-10T02:44:21.264 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:44:21 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-10T02:44:21.264 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:44:21 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:44:21.450 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:44:21 smithi188 ceph-mon[36599]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi120:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T02:44:21.450 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:44:21 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-10T02:44:21.450 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:44:21 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-10T02:44:21.450 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:44:21 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:44:21.473 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:44:21 smithi052 ceph-mon[32362]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi120:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T02:44:21.473 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:44:21 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-10T02:44:21.474 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:44:21 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-10T02:44:21.474 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:44:21 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:44:22.320 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:44:22 smithi120 ceph-mon[36634]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-10T02:44:22.365 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:44:22 smithi188 ceph-mon[36599]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-10T02:44:22.473 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:44:22 smithi052 ceph-mon[32362]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-10T02:44:23.385 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:44:23 smithi120 ceph-mon[36634]: from='client.? 172.21.15.120:0/189654099' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a24d8da8-c5ec-44d7-ab30-56f6ce3efda7"}]: dispatch 2024-06-10T02:44:23.385 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:44:23 smithi120 ceph-mon[36634]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a24d8da8-c5ec-44d7-ab30-56f6ce3efda7"}]: dispatch 2024-06-10T02:44:23.386 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:44:23 smithi120 ceph-mon[36634]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a24d8da8-c5ec-44d7-ab30-56f6ce3efda7"}]': finished 2024-06-10T02:44:23.386 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:44:23 smithi120 ceph-mon[36634]: osdmap e10: 2 total, 1 up, 2 in 2024-06-10T02:44:23.386 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:44:23 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-10T02:44:23.450 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:44:23 smithi188 ceph-mon[36599]: from='client.? 172.21.15.120:0/189654099' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a24d8da8-c5ec-44d7-ab30-56f6ce3efda7"}]: dispatch 2024-06-10T02:44:23.451 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:44:23 smithi188 ceph-mon[36599]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a24d8da8-c5ec-44d7-ab30-56f6ce3efda7"}]: dispatch 2024-06-10T02:44:23.451 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:44:23 smithi188 ceph-mon[36599]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a24d8da8-c5ec-44d7-ab30-56f6ce3efda7"}]': finished 2024-06-10T02:44:23.451 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:44:23 smithi188 ceph-mon[36599]: osdmap e10: 2 total, 1 up, 2 in 2024-06-10T02:44:23.451 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:44:23 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-10T02:44:23.474 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:44:23 smithi052 ceph-mon[32362]: from='client.? 172.21.15.120:0/189654099' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a24d8da8-c5ec-44d7-ab30-56f6ce3efda7"}]: dispatch 2024-06-10T02:44:23.474 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:44:23 smithi052 ceph-mon[32362]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a24d8da8-c5ec-44d7-ab30-56f6ce3efda7"}]: dispatch 2024-06-10T02:44:23.474 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:44:23 smithi052 ceph-mon[32362]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a24d8da8-c5ec-44d7-ab30-56f6ce3efda7"}]': finished 2024-06-10T02:44:23.474 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:44:23 smithi052 ceph-mon[32362]: osdmap e10: 2 total, 1 up, 2 in 2024-06-10T02:44:23.474 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:44:23 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-10T02:44:24.385 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:44:24 smithi120 ceph-mon[36634]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-10T02:44:24.385 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:44:24 smithi120 ceph-mon[36634]: from='client.? 172.21.15.120:0/644780593' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-10T02:44:24.450 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:44:24 smithi188 ceph-mon[36599]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-10T02:44:24.451 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:44:24 smithi188 ceph-mon[36599]: from='client.? 172.21.15.120:0/644780593' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-10T02:44:24.474 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:44:24 smithi052 ceph-mon[32362]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-10T02:44:24.474 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:44:24 smithi052 ceph-mon[32362]: from='client.? 172.21.15.120:0/644780593' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-10T02:44:26.385 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:44:26 smithi120 ceph-mon[36634]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-10T02:44:26.450 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:44:26 smithi188 ceph-mon[36599]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-10T02:44:26.473 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:44:26 smithi052 ceph-mon[32362]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-10T02:44:28.369 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:44:28 smithi120 ceph-mon[36634]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-10T02:44:28.450 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:44:28 smithi188 ceph-mon[36599]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-10T02:44:28.474 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:44:28 smithi052 ceph-mon[32362]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-10T02:44:30.385 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:44:30 smithi120 ceph-mon[36634]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-10T02:44:30.385 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:44:30 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-06-10T02:44:30.385 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:44:30 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:44:30.450 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:44:30 smithi188 ceph-mon[36599]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-10T02:44:30.450 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:44:30 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-06-10T02:44:30.450 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:44:30 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:44:30.474 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:44:30 smithi052 ceph-mon[32362]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-10T02:44:30.474 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:44:30 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-06-10T02:44:30.474 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:44:30 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:44:31.326 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:44:31 smithi120 ceph-mon[36634]: Deploying daemon osd.1 on smithi120 2024-06-10T02:44:31.450 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:44:31 smithi188 ceph-mon[36599]: Deploying daemon osd.1 on smithi120 2024-06-10T02:44:31.474 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:44:31 smithi052 ceph-mon[32362]: Deploying daemon osd.1 on smithi120 2024-06-10T02:44:32.231 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:44:32 smithi120 ceph-mon[36634]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-10T02:44:32.450 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:44:32 smithi188 ceph-mon[36599]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-10T02:44:32.473 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:44:32 smithi052 ceph-mon[32362]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-10T02:44:34.451 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:44:34 smithi188 ceph-mon[36599]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-10T02:44:34.474 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:44:34 smithi052 ceph-mon[32362]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-10T02:44:34.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:44:34 smithi120 ceph-mon[36634]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-10T02:44:36.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:44:36 smithi052 ceph-mon[32362]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-10T02:44:36.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:44:36 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:44:36.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:44:36 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:44:36.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:44:36 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T02:44:36.623 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:44:36 smithi120 ceph-mon[36634]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-10T02:44:36.623 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:44:36 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:44:36.623 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:44:36 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:44:36.624 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:44:36 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T02:44:36.700 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:44:36 smithi188 ceph-mon[36599]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-10T02:44:36.701 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:44:36 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:44:36.701 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:44:36 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:44:36.701 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:44:36 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T02:44:38.591 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:44:38 smithi120 ceph-mon[36634]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-10T02:44:38.675 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:44:38.678+0000 7f678effd700 1 -- 172.21.15.120:0/3536399890 <== mgr.14150 v2:172.21.15.52:6800/3214763969 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f67900628d0 con 0x7f67780a2b90 2024-06-10T02:44:38.676 INFO:teuthology.orchestra.run.smithi120.stdout:Created osd(s) 1 on host 'smithi120' 2024-06-10T02:44:38.678 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:44:38.681+0000 7f6797bc4700 1 -- 172.21.15.120:0/3536399890 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f67780a2b90 msgr2=0x7f67780a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:44:38.678 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:44:38.681+0000 7f6797bc4700 1 --2- 172.21.15.120:0/3536399890 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f67780a2b90 0x7f67780a5050 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f6790103550 tx=0x7f6780009570 comp rx=0 tx=0).stop 2024-06-10T02:44:38.678 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:44:38.681+0000 7f6797bc4700 1 -- 172.21.15.120:0/3536399890 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6790104090 msgr2=0x7f679019fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:44:38.678 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:44:38.681+0000 7f6797bc4700 1 --2- 172.21.15.120:0/3536399890 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6790104090 0x7f679019fdb0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f678800c970 tx=0x7f678800cd30 comp rx=0 tx=0).stop 2024-06-10T02:44:38.678 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:44:38.681+0000 7f6797bc4700 1 -- 172.21.15.120:0/3536399890 shutdown_connections 2024-06-10T02:44:38.678 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:44:38.681+0000 7f6797bc4700 1 --2- 172.21.15.120:0/3536399890 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6790104090 0x7f679019fdb0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:44:38.678 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:44:38.681+0000 7f6797bc4700 1 --2- 172.21.15.120:0/3536399890 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f67901024f0 0x7f67901987f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:44:38.678 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:44:38.681+0000 7f6797bc4700 1 --2- 172.21.15.120:0/3536399890 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f67780a2b90 0x7f67780a5050 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:44:38.679 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:44:38.681+0000 7f6797bc4700 1 --2- 172.21.15.120:0/3536399890 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f67901036f0 0x7f6790198d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:44:38.679 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:44:38.681+0000 7f6797bc4700 1 -- 172.21.15.120:0/3536399890 >> 172.21.15.120:0/3536399890 conn(0x7f67900fda90 msgr2=0x7f6790106a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:44:38.679 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:44:38.681+0000 7f6797bc4700 1 -- 172.21.15.120:0/3536399890 shutdown_connections 2024-06-10T02:44:38.679 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:44:38.681+0000 7f6797bc4700 1 -- 172.21.15.120:0/3536399890 wait complete. 2024-06-10T02:44:38.700 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:44:38 smithi188 ceph-mon[36599]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-10T02:44:38.723 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:44:38 smithi052 ceph-mon[32362]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-10T02:44:39.274 DEBUG:teuthology.orchestra.run.smithi120:osd.1> sudo journalctl -f -n 0 -u ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968@osd.1.service 2024-06-10T02:44:39.276 INFO:tasks.cephadm:Deploying osd.2 on smithi188 with /dev/vg_nvme/lv_4... 2024-06-10T02:44:39.277 DEBUG:teuthology.orchestra.run.smithi188:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-06-10T02:44:39.517 INFO:teuthology.orchestra.run.smithi188.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.c/config 2024-06-10T02:44:39.885 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:44:39 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:44:39.885 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:44:39 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:44:39.885 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:44:39 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:44:39.885 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:44:39 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:44:39.885 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:44:39 smithi120 ceph-mon[36634]: from='osd.1 [v2:172.21.15.120:6800/4116540053,v1:172.21.15.120:6801/4116540053]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-06-10T02:44:39.886 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:44:39 smithi120 ceph-mon[36634]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-06-10T02:44:39.926 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:44:39 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:44:39.926 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:44:39 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:44:39.926 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:44:39 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:44:39.926 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:44:39 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:44:39.926 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:44:39 smithi188 ceph-mon[36599]: from='osd.1 [v2:172.21.15.120:6800/4116540053,v1:172.21.15.120:6801/4116540053]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-06-10T02:44:39.926 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:44:39 smithi188 ceph-mon[36599]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-06-10T02:44:39.973 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:44:39 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:44:39.974 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:44:39 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:44:39.974 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:44:39 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:44:39.974 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:44:39 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:44:39.974 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:44:39 smithi052 ceph-mon[32362]: from='osd.1 [v2:172.21.15.120:6800/4116540053,v1:172.21.15.120:6801/4116540053]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-06-10T02:44:39.974 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:44:39 smithi052 ceph-mon[32362]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-06-10T02:44:40.796 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:44:40 smithi188 ceph-mon[36599]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-10T02:44:40.796 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:44:40 smithi188 ceph-mon[36599]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-06-10T02:44:40.796 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:44:40 smithi188 ceph-mon[36599]: from='osd.1 [v2:172.21.15.120:6800/4116540053,v1:172.21.15.120:6801/4116540053]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]: dispatch 2024-06-10T02:44:40.797 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:44:40 smithi188 ceph-mon[36599]: osdmap e11: 2 total, 1 up, 2 in 2024-06-10T02:44:40.797 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:44:40 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-10T02:44:40.797 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:44:40 smithi188 ceph-mon[36599]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]: dispatch 2024-06-10T02:44:41.135 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:44:40 smithi120 ceph-mon[36634]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-10T02:44:41.135 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:44:40 smithi120 ceph-mon[36634]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-06-10T02:44:41.136 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:44:40 smithi120 ceph-mon[36634]: from='osd.1 [v2:172.21.15.120:6800/4116540053,v1:172.21.15.120:6801/4116540053]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]: dispatch 2024-06-10T02:44:41.136 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:44:40 smithi120 ceph-mon[36634]: osdmap e11: 2 total, 1 up, 2 in 2024-06-10T02:44:41.136 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:44:40 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-10T02:44:41.136 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:44:40 smithi120 ceph-mon[36634]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]: dispatch 2024-06-10T02:44:41.224 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:44:40 smithi052 ceph-mon[32362]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-10T02:44:41.224 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:44:40 smithi052 ceph-mon[32362]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-06-10T02:44:41.224 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:44:40 smithi052 ceph-mon[32362]: from='osd.1 [v2:172.21.15.120:6800/4116540053,v1:172.21.15.120:6801/4116540053]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]: dispatch 2024-06-10T02:44:41.224 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:44:40 smithi052 ceph-mon[32362]: osdmap e11: 2 total, 1 up, 2 in 2024-06-10T02:44:41.224 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:44:40 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-10T02:44:41.224 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:44:40 smithi052 ceph-mon[32362]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]: dispatch 2024-06-10T02:44:41.370 INFO:teuthology.orchestra.run.smithi188.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-06-10T02:44:41.370 INFO:teuthology.orchestra.run.smithi188.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-06-10T02:44:41.370 INFO:teuthology.orchestra.run.smithi188.stderr: stderr: 10+0 records in 2024-06-10T02:44:41.370 INFO:teuthology.orchestra.run.smithi188.stderr:10+0 records out 2024-06-10T02:44:41.370 INFO:teuthology.orchestra.run.smithi188.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0129784 s, 808 MB/s 2024-06-10T02:44:41.370 INFO:teuthology.orchestra.run.smithi188.stderr:--> Zapping successful for: 2024-06-10T02:44:41.563 INFO:journalctl@ceph.osd.1.smithi120.stdout:Jun 10 02:44:41 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-1[41185]: 2024-06-10T02:44:41.178+0000 7fe5a366a700 -1 osd.1 0 waiting for initial osdmap 2024-06-10T02:44:41.563 INFO:journalctl@ceph.osd.1.smithi120.stdout:Jun 10 02:44:41 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-1[41185]: 2024-06-10T02:44:41.182+0000 7fe59b455700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-10T02:44:41.882 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:44:41 smithi120 ceph-mon[36634]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]': finished 2024-06-10T02:44:41.882 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:44:41 smithi120 ceph-mon[36634]: osdmap e12: 2 total, 1 up, 2 in 2024-06-10T02:44:41.882 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:44:41 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-10T02:44:41.882 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:44:41 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-10T02:44:41.882 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:44:41 smithi120 ceph-mon[36634]: from='osd.1 ' entity='osd.1' 2024-06-10T02:44:42.026 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:44:41 smithi188 ceph-mon[36599]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]': finished 2024-06-10T02:44:42.027 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:44:41 smithi188 ceph-mon[36599]: osdmap e12: 2 total, 1 up, 2 in 2024-06-10T02:44:42.027 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:44:41 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-10T02:44:42.027 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:44:41 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-10T02:44:42.027 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:44:41 smithi188 ceph-mon[36599]: from='osd.1 ' entity='osd.1' 2024-06-10T02:44:42.053 DEBUG:teuthology.orchestra.run.smithi188:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi188:vg_nvme/lv_4 2024-06-10T02:44:42.224 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:44:41 smithi052 ceph-mon[32362]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]': finished 2024-06-10T02:44:42.224 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:44:41 smithi052 ceph-mon[32362]: osdmap e12: 2 total, 1 up, 2 in 2024-06-10T02:44:42.224 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:44:41 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-10T02:44:42.224 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:44:41 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-10T02:44:42.224 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:44:41 smithi052 ceph-mon[32362]: from='osd.1 ' entity='osd.1' 2024-06-10T02:44:42.295 INFO:teuthology.orchestra.run.smithi188.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.c/config 2024-06-10T02:44:42.951 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:44:42 smithi188 ceph-mon[36599]: purged_snaps scrub starts 2024-06-10T02:44:42.951 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:44:42 smithi188 ceph-mon[36599]: purged_snaps scrub ok 2024-06-10T02:44:42.951 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:44:42 smithi188 ceph-mon[36599]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-10T02:44:42.951 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:44:42 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-10T02:44:42.951 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:44:42 smithi188 ceph-mon[36599]: osd.1 [v2:172.21.15.120:6800/4116540053,v1:172.21.15.120:6801/4116540053] boot 2024-06-10T02:44:42.951 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:44:42 smithi188 ceph-mon[36599]: osdmap e13: 2 total, 2 up, 2 in 2024-06-10T02:44:42.951 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:44:42 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-10T02:44:42.951 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:44:42 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:44:42.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:44:42 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:44:42.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:44:42 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-10T02:44:42.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:44:42 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:44:42.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:44:42 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:44:42.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:44:42 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T02:44:42.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:44:42 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:44:42.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:44:42 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T02:44:42.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:44:42 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:44:42.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:44:42 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T02:44:42.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:44:42 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:44:43.135 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:44:42 smithi120 ceph-mon[36634]: purged_snaps scrub starts 2024-06-10T02:44:43.136 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:44:42 smithi120 ceph-mon[36634]: purged_snaps scrub ok 2024-06-10T02:44:43.136 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:44:42 smithi120 ceph-mon[36634]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-10T02:44:43.136 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:44:42 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-10T02:44:43.136 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:44:42 smithi120 ceph-mon[36634]: osd.1 [v2:172.21.15.120:6800/4116540053,v1:172.21.15.120:6801/4116540053] boot 2024-06-10T02:44:43.136 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:44:42 smithi120 ceph-mon[36634]: osdmap e13: 2 total, 2 up, 2 in 2024-06-10T02:44:43.136 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:44:42 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-10T02:44:43.136 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:44:42 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:44:43.136 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:44:42 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:44:43.136 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:44:42 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-10T02:44:43.136 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:44:42 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:44:43.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:44:42 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:44:43.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:44:42 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T02:44:43.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:44:42 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:44:43.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:44:42 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T02:44:43.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:44:42 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:44:43.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:44:42 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T02:44:43.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:44:42 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:44:43.224 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:44:42 smithi052 ceph-mon[32362]: purged_snaps scrub starts 2024-06-10T02:44:43.224 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:44:42 smithi052 ceph-mon[32362]: purged_snaps scrub ok 2024-06-10T02:44:43.224 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:44:42 smithi052 ceph-mon[32362]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-10T02:44:43.224 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:44:42 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-10T02:44:43.224 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:44:42 smithi052 ceph-mon[32362]: osd.1 [v2:172.21.15.120:6800/4116540053,v1:172.21.15.120:6801/4116540053] boot 2024-06-10T02:44:43.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:44:42 smithi052 ceph-mon[32362]: osdmap e13: 2 total, 2 up, 2 in 2024-06-10T02:44:43.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:44:42 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-10T02:44:43.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:44:42 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:44:43.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:44:42 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:44:43.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:44:42 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-10T02:44:43.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:44:42 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:44:43.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:44:42 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:44:43.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:44:42 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T02:44:43.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:44:42 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:44:43.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:44:42 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T02:44:43.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:44:42 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:44:43.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:44:42 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T02:44:43.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:44:42 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:44:43.750 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:44:43.751+0000 7f70820e1700 1 -- 172.21.15.188:0/3018676470 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f707c1036f0 msgr2=0x7f707c103b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:44:43.750 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:44:43.751+0000 7f70820e1700 1 --2- 172.21.15.188:0/3018676470 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f707c1036f0 0x7f707c103b50 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f7064009990 tx=0x7f7064009ca0 comp rx=0 tx=0).stop 2024-06-10T02:44:43.750 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:44:43.751+0000 7f70820e1700 1 -- 172.21.15.188:0/3018676470 shutdown_connections 2024-06-10T02:44:43.750 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:44:43.751+0000 7f70820e1700 1 --2- 172.21.15.188:0/3018676470 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f707c104090 0x7f707c10a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:44:43.750 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:44:43.751+0000 7f70820e1700 1 --2- 172.21.15.188:0/3018676470 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f707c1024f0 0x7f707c102910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:44:43.751 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:44:43.751+0000 7f70820e1700 1 --2- 172.21.15.188:0/3018676470 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f707c1036f0 0x7f707c103b50 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:44:43.751 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:44:43.751+0000 7f70820e1700 1 -- 172.21.15.188:0/3018676470 >> 172.21.15.188:0/3018676470 conn(0x7f707c0fda90 msgr2=0x7f707c0ffed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:44:43.751 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:44:43.751+0000 7f70820e1700 1 -- 172.21.15.188:0/3018676470 shutdown_connections 2024-06-10T02:44:43.751 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:44:43.751+0000 7f70820e1700 1 -- 172.21.15.188:0/3018676470 wait complete. 2024-06-10T02:44:43.751 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:44:43.751+0000 7f70820e1700 1 Processor -- start 2024-06-10T02:44:43.751 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:44:43.752+0000 7f70820e1700 1 -- start start 2024-06-10T02:44:43.751 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:44:43.752+0000 7f70820e1700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f707c1024f0 0x7f707c076230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:44:43.751 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:44:43.752+0000 7f70820e1700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f707c1036f0 0x7f707c074880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:44:43.751 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:44:43.752+0000 7f70820e1700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f707c104090 0x7f707c074dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:44:43.751 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:44:43.752+0000 7f70820e1700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f707c10e220 con 0x7f707c1024f0 2024-06-10T02:44:43.752 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:44:43.752+0000 7f70820e1700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f707c10e0a0 con 0x7f707c1036f0 2024-06-10T02:44:43.752 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:44:43.752+0000 7f70820e1700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f707c10e3a0 con 0x7f707c104090 2024-06-10T02:44:43.752 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:44:43.752+0000 7f707bfff700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f707c104090 0x7f707c074dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:44:43.752 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:44:43.752+0000 7f707bfff700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f707c104090 0x7f707c074dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.188:55262/0 (socket says 172.21.15.188:55262) 2024-06-10T02:44:43.752 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:44:43.752+0000 7f707bfff700 1 -- 172.21.15.188:0/2969617104 learned_addr learned my addr 172.21.15.188:0/2969617104 (peer_addr_for_me v2:172.21.15.188:0/0) 2024-06-10T02:44:43.752 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:44:43.752+0000 7f707b7fe700 1 --2- 172.21.15.188:0/2969617104 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f707c1024f0 0x7f707c076230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:44:43.752 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:44:43.752+0000 7f7072ffd700 1 --2- 172.21.15.188:0/2969617104 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f707c1036f0 0x7f707c074880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:44:43.752 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:44:43.752+0000 7f707bfff700 1 -- 172.21.15.188:0/2969617104 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f707c1036f0 msgr2=0x7f707c074880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:44:43.752 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:44:43.752+0000 7f707bfff700 1 --2- 172.21.15.188:0/2969617104 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f707c1036f0 0x7f707c074880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:44:43.752 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:44:43.752+0000 7f707bfff700 1 -- 172.21.15.188:0/2969617104 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f707c1024f0 msgr2=0x7f707c076230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:44:43.752 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:44:43.752+0000 7f707bfff700 1 --2- 172.21.15.188:0/2969617104 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f707c1024f0 0x7f707c076230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:44:43.753 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:44:43.752+0000 7f707bfff700 1 -- 172.21.15.188:0/2969617104 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f707c075670 con 0x7f707c104090 2024-06-10T02:44:43.753 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:44:43.752+0000 7f7072ffd700 1 --2- 172.21.15.188:0/2969617104 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f707c1036f0 0x7f707c074880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T02:44:43.753 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:44:43.752+0000 7f707bfff700 1 --2- 172.21.15.188:0/2969617104 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f707c104090 0x7f707c074dc0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f707400b970 tx=0x7f707400bc80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:44:43.753 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:44:43.753+0000 7f70797fa700 1 -- 172.21.15.188:0/2969617104 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7074019070 con 0x7f707c104090 2024-06-10T02:44:43.753 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:44:43.753+0000 7f70820e1700 1 -- 172.21.15.188:0/2969617104 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f707c1a2a50 con 0x7f707c104090 2024-06-10T02:44:43.753 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:44:43.753+0000 7f70797fa700 1 -- 172.21.15.188:0/2969617104 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f707400ce10 con 0x7f707c104090 2024-06-10T02:44:43.753 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:44:43.753+0000 7f70797fa700 1 -- 172.21.15.188:0/2969617104 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7074007370 con 0x7f707c104090 2024-06-10T02:44:43.753 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:44:43.753+0000 7f70820e1700 1 -- 172.21.15.188:0/2969617104 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f707c1a2fe0 con 0x7f707c104090 2024-06-10T02:44:43.754 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:44:43.754+0000 7f70820e1700 1 -- 172.21.15.188:0/2969617104 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f707c069180 con 0x7f707c104090 2024-06-10T02:44:43.754 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:44:43.755+0000 7f70797fa700 1 -- 172.21.15.188:0/2969617104 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7f707401c070 con 0x7f707c104090 2024-06-10T02:44:43.754 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:44:43.755+0000 7f70797fa700 1 --2- 172.21.15.188:0/2969617104 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f70680a2b90 0x7f70680a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:44:43.754 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:44:43.755+0000 7f70797fa700 1 -- 172.21.15.188:0/2969617104 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2309+0+0 (secure 0 0 0) 0x7f70740ccb30 con 0x7f707c104090 2024-06-10T02:44:43.754 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:44:43.756+0000 7f707b7fe700 1 --2- 172.21.15.188:0/2969617104 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f70680a2b90 0x7f70680a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:44:43.754 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:44:43.756+0000 7f707b7fe700 1 --2- 172.21.15.188:0/2969617104 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f70680a2b90 0x7f70680a5050 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f707c103550 tx=0x7f706c009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:44:43.756 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:44:43.760+0000 7f70797fa700 1 -- 172.21.15.188:0/2969617104 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f707409b2e0 con 0x7f707c104090 2024-06-10T02:44:43.957 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:44:43.960+0000 7f70820e1700 1 -- 172.21.15.188:0/2969617104 --> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi188:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f707c0628d0 con 0x7f70680a2b90 2024-06-10T02:44:44.135 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:44:43 smithi120 ceph-mon[36634]: Detected new or changed devices on smithi120 2024-06-10T02:44:44.135 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:44:43 smithi120 ceph-mon[36634]: Adjusting osd_memory_target on smithi120 to 17088M 2024-06-10T02:44:44.200 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:44:43 smithi188 ceph-mon[36599]: Detected new or changed devices on smithi120 2024-06-10T02:44:44.200 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:44:43 smithi188 ceph-mon[36599]: Adjusting osd_memory_target on smithi120 to 17088M 2024-06-10T02:44:44.224 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:44:43 smithi052 ceph-mon[32362]: Detected new or changed devices on smithi120 2024-06-10T02:44:44.224 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:44:43 smithi052 ceph-mon[32362]: Adjusting osd_memory_target on smithi120 to 17088M 2024-06-10T02:44:44.813 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:44:44 smithi188 ceph-mon[36599]: pgmap v63: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-06-10T02:44:44.813 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:44:44 smithi188 ceph-mon[36599]: osdmap e14: 2 total, 2 up, 2 in 2024-06-10T02:44:44.813 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:44:44 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-10T02:44:44.814 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:44:44 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-10T02:44:44.814 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:44:44 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:44:45.135 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:44:44 smithi120 ceph-mon[36634]: pgmap v63: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-06-10T02:44:45.135 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:44:44 smithi120 ceph-mon[36634]: osdmap e14: 2 total, 2 up, 2 in 2024-06-10T02:44:45.135 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:44:44 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-10T02:44:45.135 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:44:44 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-10T02:44:45.136 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:44:44 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:44:45.224 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:44:44 smithi052 ceph-mon[32362]: pgmap v63: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-06-10T02:44:45.224 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:44:44 smithi052 ceph-mon[32362]: osdmap e14: 2 total, 2 up, 2 in 2024-06-10T02:44:45.224 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:44:44 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-10T02:44:45.224 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:44:44 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-10T02:44:45.224 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:44:44 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:44:45.950 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:44:45 smithi188 ceph-mon[36599]: from='client.24169 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi188:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T02:44:46.068 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:44:45 smithi052 ceph-mon[32362]: from='client.24169 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi188:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T02:44:46.135 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:44:45 smithi120 ceph-mon[36634]: from='client.24169 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi188:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T02:44:46.974 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:44:46 smithi052 ceph-mon[32362]: pgmap v65: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-06-10T02:44:46.974 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:44:46 smithi052 ceph-mon[32362]: from='client.? 172.21.15.188:0/2302173691' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a7162106-ccbf-45bd-bc0e-051ee43f4214"}]: dispatch 2024-06-10T02:44:46.974 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:44:46 smithi052 ceph-mon[32362]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a7162106-ccbf-45bd-bc0e-051ee43f4214"}]: dispatch 2024-06-10T02:44:46.974 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:44:46 smithi052 ceph-mon[32362]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a7162106-ccbf-45bd-bc0e-051ee43f4214"}]': finished 2024-06-10T02:44:46.974 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:44:46 smithi052 ceph-mon[32362]: osdmap e15: 3 total, 2 up, 3 in 2024-06-10T02:44:46.974 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:44:46 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-10T02:44:47.135 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:44:46 smithi120 ceph-mon[36634]: pgmap v65: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-06-10T02:44:47.135 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:44:46 smithi120 ceph-mon[36634]: from='client.? 172.21.15.188:0/2302173691' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a7162106-ccbf-45bd-bc0e-051ee43f4214"}]: dispatch 2024-06-10T02:44:47.135 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:44:46 smithi120 ceph-mon[36634]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a7162106-ccbf-45bd-bc0e-051ee43f4214"}]: dispatch 2024-06-10T02:44:47.135 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:44:46 smithi120 ceph-mon[36634]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a7162106-ccbf-45bd-bc0e-051ee43f4214"}]': finished 2024-06-10T02:44:47.136 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:44:46 smithi120 ceph-mon[36634]: osdmap e15: 3 total, 2 up, 3 in 2024-06-10T02:44:47.136 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:44:46 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-10T02:44:47.200 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:44:46 smithi188 ceph-mon[36599]: pgmap v65: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-06-10T02:44:47.201 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:44:46 smithi188 ceph-mon[36599]: from='client.? 172.21.15.188:0/2302173691' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a7162106-ccbf-45bd-bc0e-051ee43f4214"}]: dispatch 2024-06-10T02:44:47.201 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:44:46 smithi188 ceph-mon[36599]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a7162106-ccbf-45bd-bc0e-051ee43f4214"}]: dispatch 2024-06-10T02:44:47.201 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:44:46 smithi188 ceph-mon[36599]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a7162106-ccbf-45bd-bc0e-051ee43f4214"}]': finished 2024-06-10T02:44:47.201 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:44:46 smithi188 ceph-mon[36599]: osdmap e15: 3 total, 2 up, 3 in 2024-06-10T02:44:47.201 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:44:46 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-10T02:44:48.135 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:44:47 smithi120 ceph-mon[36634]: from='client.? 172.21.15.188:0/2589084513' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-10T02:44:48.201 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:44:47 smithi188 ceph-mon[36599]: from='client.? 172.21.15.188:0/2589084513' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-10T02:44:48.224 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:44:47 smithi052 ceph-mon[32362]: from='client.? 172.21.15.188:0/2589084513' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-10T02:44:49.135 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:44:48 smithi120 ceph-mon[36634]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-06-10T02:44:49.200 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:44:48 smithi188 ceph-mon[36599]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-06-10T02:44:49.224 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:44:48 smithi052 ceph-mon[32362]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-06-10T02:44:51.135 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:44:50 smithi120 ceph-mon[36634]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-06-10T02:44:51.200 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:44:50 smithi188 ceph-mon[36599]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-06-10T02:44:51.223 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:44:50 smithi052 ceph-mon[32362]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-06-10T02:44:52.977 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:44:52 smithi188 ceph-mon[36599]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-06-10T02:44:53.135 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:44:52 smithi120 ceph-mon[36634]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-06-10T02:44:53.224 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:44:52 smithi052 ceph-mon[32362]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-06-10T02:44:54.037 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:44:53 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-06-10T02:44:54.037 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:44:53 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:44:54.153 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:44:53 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-06-10T02:44:54.153 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:44:53 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:44:54.224 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:44:53 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-06-10T02:44:54.224 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:44:53 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:44:55.009 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:44:54 smithi188 ceph-mon[36599]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-06-10T02:44:55.009 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:44:54 smithi188 ceph-mon[36599]: Deploying daemon osd.2 on smithi188 2024-06-10T02:44:55.049 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:44:54 smithi120 ceph-mon[36634]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-06-10T02:44:55.049 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:44:54 smithi120 ceph-mon[36634]: Deploying daemon osd.2 on smithi188 2024-06-10T02:44:55.224 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:44:54 smithi052 ceph-mon[32362]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-06-10T02:44:55.224 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:44:54 smithi052 ceph-mon[32362]: Deploying daemon osd.2 on smithi188 2024-06-10T02:44:56.950 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:44:56 smithi188 ceph-mon[36599]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-06-10T02:44:57.135 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:44:56 smithi120 ceph-mon[36634]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-06-10T02:44:57.224 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:44:56 smithi052 ceph-mon[32362]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-06-10T02:44:58.951 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:44:58 smithi188 ceph-mon[36599]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-06-10T02:44:58.951 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:44:58 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:44:58.951 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:44:58 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T02:44:58.951 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:44:58 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:44:59.135 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:44:58 smithi120 ceph-mon[36634]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-06-10T02:44:59.136 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:44:58 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:44:59.136 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:44:58 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T02:44:59.136 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:44:58 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:44:59.224 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:44:58 smithi052 ceph-mon[32362]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-06-10T02:44:59.224 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:44:58 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:44:59.224 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:44:58 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T02:44:59.224 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:44:58 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:45:00.951 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:00 smithi188 ceph-mon[36599]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-06-10T02:45:01.135 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:00 smithi120 ceph-mon[36634]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-06-10T02:45:01.224 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:00 smithi052 ceph-mon[32362]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-06-10T02:45:01.974 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:45:01.976+0000 7f70797fa700 1 -- 172.21.15.188:0/2969617104 <== mgr.14150 v2:172.21.15.52:6800/3214763969 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f707c0628d0 con 0x7f70680a2b90 2024-06-10T02:45:01.974 INFO:teuthology.orchestra.run.smithi188.stdout:Created osd(s) 2 on host 'smithi188' 2024-06-10T02:45:01.976 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:45:01.978+0000 7f70820e1700 1 -- 172.21.15.188:0/2969617104 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f70680a2b90 msgr2=0x7f70680a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:45:01.976 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:45:01.978+0000 7f70820e1700 1 --2- 172.21.15.188:0/2969617104 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f70680a2b90 0x7f70680a5050 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f707c103550 tx=0x7f706c009190 comp rx=0 tx=0).stop 2024-06-10T02:45:01.976 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:45:01.978+0000 7f70820e1700 1 -- 172.21.15.188:0/2969617104 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f707c104090 msgr2=0x7f707c074dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:45:01.976 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:45:01.978+0000 7f70820e1700 1 --2- 172.21.15.188:0/2969617104 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f707c104090 0x7f707c074dc0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f707400b970 tx=0x7f707400bc80 comp rx=0 tx=0).stop 2024-06-10T02:45:01.977 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:45:01.979+0000 7f70820e1700 1 -- 172.21.15.188:0/2969617104 shutdown_connections 2024-06-10T02:45:01.977 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:45:01.979+0000 7f70820e1700 1 --2- 172.21.15.188:0/2969617104 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f707c104090 0x7f707c074dc0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:45:01.977 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:45:01.979+0000 7f70820e1700 1 --2- 172.21.15.188:0/2969617104 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f70680a2b90 0x7f70680a5050 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:45:01.977 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:45:01.979+0000 7f70820e1700 1 --2- 172.21.15.188:0/2969617104 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f707c1024f0 0x7f707c076230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:45:01.977 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:45:01.979+0000 7f70820e1700 1 --2- 172.21.15.188:0/2969617104 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f707c1036f0 0x7f707c074880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:45:01.977 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:45:01.979+0000 7f70820e1700 1 -- 172.21.15.188:0/2969617104 >> 172.21.15.188:0/2969617104 conn(0x7f707c0fda90 msgr2=0x7f707c106a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:45:01.977 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:45:01.979+0000 7f70820e1700 1 -- 172.21.15.188:0/2969617104 shutdown_connections 2024-06-10T02:45:01.977 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:45:01.979+0000 7f70820e1700 1 -- 172.21.15.188:0/2969617104 wait complete. 2024-06-10T02:45:02.506 DEBUG:teuthology.orchestra.run.smithi188:osd.2> sudo journalctl -f -n 0 -u ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968@osd.2.service 2024-06-10T02:45:02.508 INFO:tasks.cephadm:Deploying osd.3 on smithi052 with /dev/vg_nvme/lv_3... 2024-06-10T02:45:02.508 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-06-10T02:45:02.766 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:45:02.771 INFO:journalctl@ceph.osd.2.smithi188.stdout:Jun 10 02:45:02 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-2[41721]: 2024-06-10T02:45:02.596+0000 7f9d6edea7c0 -1 osd.2 0 log_to_monitors true 2024-06-10T02:45:02.889 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:02 smithi052 ceph-mon[32362]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-06-10T02:45:02.889 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:02 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:45:02.889 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:02 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:45:02.889 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:02 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:45:02.889 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:02 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:45:02.889 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:02 smithi052 ceph-mon[32362]: from='osd.2 [v2:172.21.15.188:6800/2624326999,v1:172.21.15.188:6801/2624326999]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-06-10T02:45:02.889 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:02 smithi052 ceph-mon[32362]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-06-10T02:45:03.026 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:02 smithi188 ceph-mon[36599]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-06-10T02:45:03.026 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:02 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:45:03.026 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:02 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:45:03.027 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:02 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:45:03.027 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:02 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:45:03.027 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:02 smithi188 ceph-mon[36599]: from='osd.2 [v2:172.21.15.188:6800/2624326999,v1:172.21.15.188:6801/2624326999]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-06-10T02:45:03.027 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:02 smithi188 ceph-mon[36599]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-06-10T02:45:03.135 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:02 smithi120 ceph-mon[36634]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-06-10T02:45:03.135 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:02 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:45:03.135 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:02 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:45:03.136 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:02 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:45:03.136 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:02 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:45:03.136 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:02 smithi120 ceph-mon[36634]: from='osd.2 [v2:172.21.15.188:6800/2624326999,v1:172.21.15.188:6801/2624326999]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-06-10T02:45:03.136 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:02 smithi120 ceph-mon[36634]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-06-10T02:45:04.451 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:04 smithi188 ceph-mon[36599]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-06-10T02:45:04.451 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:04 smithi188 ceph-mon[36599]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-06-10T02:45:04.452 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:04 smithi188 ceph-mon[36599]: osdmap e16: 3 total, 2 up, 3 in 2024-06-10T02:45:04.452 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:04 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-10T02:45:04.452 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:04 smithi188 ceph-mon[36599]: from='osd.2 [v2:172.21.15.188:6800/2624326999,v1:172.21.15.188:6801/2624326999]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]: dispatch 2024-06-10T02:45:04.452 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:04 smithi188 ceph-mon[36599]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]: dispatch 2024-06-10T02:45:04.452 INFO:journalctl@ceph.osd.2.smithi188.stdout:Jun 10 02:45:04 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-2[41721]: 2024-06-10T02:45:04.203+0000 7f9d6444e700 -1 osd.2 0 waiting for initial osdmap 2024-06-10T02:45:04.452 INFO:journalctl@ceph.osd.2.smithi188.stdout:Jun 10 02:45:04 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-2[41721]: 2024-06-10T02:45:04.209+0000 7f9d60241700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-10T02:45:04.474 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:04 smithi052 ceph-mon[32362]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-06-10T02:45:04.474 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:04 smithi052 ceph-mon[32362]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-06-10T02:45:04.474 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:04 smithi052 ceph-mon[32362]: osdmap e16: 3 total, 2 up, 3 in 2024-06-10T02:45:04.474 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:04 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-10T02:45:04.474 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:04 smithi052 ceph-mon[32362]: from='osd.2 [v2:172.21.15.188:6800/2624326999,v1:172.21.15.188:6801/2624326999]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]: dispatch 2024-06-10T02:45:04.474 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:04 smithi052 ceph-mon[32362]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]: dispatch 2024-06-10T02:45:04.635 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:04 smithi120 ceph-mon[36634]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-06-10T02:45:04.636 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:04 smithi120 ceph-mon[36634]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-06-10T02:45:04.636 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:04 smithi120 ceph-mon[36634]: osdmap e16: 3 total, 2 up, 3 in 2024-06-10T02:45:04.636 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:04 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-10T02:45:04.636 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:04 smithi120 ceph-mon[36634]: from='osd.2 [v2:172.21.15.188:6800/2624326999,v1:172.21.15.188:6801/2624326999]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]: dispatch 2024-06-10T02:45:04.636 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:04 smithi120 ceph-mon[36634]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]: dispatch 2024-06-10T02:45:04.722 INFO:teuthology.orchestra.run.smithi052.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-06-10T02:45:04.722 INFO:teuthology.orchestra.run.smithi052.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-06-10T02:45:04.722 INFO:teuthology.orchestra.run.smithi052.stderr: stderr: 10+0 records in 2024-06-10T02:45:04.722 INFO:teuthology.orchestra.run.smithi052.stderr:10+0 records out 2024-06-10T02:45:04.722 INFO:teuthology.orchestra.run.smithi052.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00826388 s, 1.3 GB/s 2024-06-10T02:45:04.722 INFO:teuthology.orchestra.run.smithi052.stderr:--> Zapping successful for: 2024-06-10T02:45:05.339 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:05 smithi052 ceph-mon[32362]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]': finished 2024-06-10T02:45:05.339 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:05 smithi052 ceph-mon[32362]: osdmap e17: 3 total, 2 up, 3 in 2024-06-10T02:45:05.339 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:05 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-10T02:45:05.339 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:05 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-10T02:45:05.370 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi052:vg_nvme/lv_3 2024-06-10T02:45:05.624 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:45:05.635 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:05 smithi120 ceph-mon[36634]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]': finished 2024-06-10T02:45:05.635 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:05 smithi120 ceph-mon[36634]: osdmap e17: 3 total, 2 up, 3 in 2024-06-10T02:45:05.635 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:05 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-10T02:45:05.636 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:05 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-10T02:45:05.699 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:05 smithi188 ceph-mon[36599]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]': finished 2024-06-10T02:45:05.699 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:05 smithi188 ceph-mon[36599]: osdmap e17: 3 total, 2 up, 3 in 2024-06-10T02:45:05.699 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:05 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-10T02:45:05.699 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:05 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-10T02:45:06.224 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:06 smithi052 ceph-mon[32362]: purged_snaps scrub starts 2024-06-10T02:45:06.224 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:06 smithi052 ceph-mon[32362]: purged_snaps scrub ok 2024-06-10T02:45:06.224 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:06 smithi052 ceph-mon[32362]: pgmap v78: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-06-10T02:45:06.224 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:06 smithi052 ceph-mon[32362]: osd.2 [v2:172.21.15.188:6800/2624326999,v1:172.21.15.188:6801/2624326999] boot 2024-06-10T02:45:06.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:06 smithi052 ceph-mon[32362]: osdmap e18: 3 total, 3 up, 3 in 2024-06-10T02:45:06.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:06 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-10T02:45:06.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:06 smithi052 ceph-mon[32362]: Detected new or changed devices on smithi188 2024-06-10T02:45:06.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:06 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:45:06.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:06 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:45:06.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:06 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-10T02:45:06.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:06 smithi052 ceph-mon[32362]: Adjusting osd_memory_target on smithi188 to 17088M 2024-06-10T02:45:06.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:06 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:45:06.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:06 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:45:06.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:06 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T02:45:06.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:06 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:45:06.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:06 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T02:45:06.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:06 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:45:06.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:06 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T02:45:06.226 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:06 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:45:06.636 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:06 smithi120 ceph-mon[36634]: purged_snaps scrub starts 2024-06-10T02:45:06.636 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:06 smithi120 ceph-mon[36634]: purged_snaps scrub ok 2024-06-10T02:45:06.636 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:06 smithi120 ceph-mon[36634]: pgmap v78: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-06-10T02:45:06.636 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:06 smithi120 ceph-mon[36634]: osd.2 [v2:172.21.15.188:6800/2624326999,v1:172.21.15.188:6801/2624326999] boot 2024-06-10T02:45:06.636 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:06 smithi120 ceph-mon[36634]: osdmap e18: 3 total, 3 up, 3 in 2024-06-10T02:45:06.636 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:06 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-10T02:45:06.636 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:06 smithi120 ceph-mon[36634]: Detected new or changed devices on smithi188 2024-06-10T02:45:06.636 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:06 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:45:06.636 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:06 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:45:06.637 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:06 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-10T02:45:06.637 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:06 smithi120 ceph-mon[36634]: Adjusting osd_memory_target on smithi188 to 17088M 2024-06-10T02:45:06.637 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:06 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:45:06.637 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:06 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:45:06.637 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:06 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T02:45:06.637 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:06 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:45:06.637 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:06 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T02:45:06.637 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:06 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:45:06.637 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:06 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T02:45:06.637 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:06 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:45:06.701 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:06 smithi188 ceph-mon[36599]: purged_snaps scrub starts 2024-06-10T02:45:06.701 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:06 smithi188 ceph-mon[36599]: purged_snaps scrub ok 2024-06-10T02:45:06.701 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:06 smithi188 ceph-mon[36599]: pgmap v78: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-06-10T02:45:06.701 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:06 smithi188 ceph-mon[36599]: osd.2 [v2:172.21.15.188:6800/2624326999,v1:172.21.15.188:6801/2624326999] boot 2024-06-10T02:45:06.701 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:06 smithi188 ceph-mon[36599]: osdmap e18: 3 total, 3 up, 3 in 2024-06-10T02:45:06.701 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:06 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-10T02:45:06.701 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:06 smithi188 ceph-mon[36599]: Detected new or changed devices on smithi188 2024-06-10T02:45:06.701 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:06 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:45:06.701 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:06 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:45:06.702 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:06 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-10T02:45:06.702 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:06 smithi188 ceph-mon[36599]: Adjusting osd_memory_target on smithi188 to 17088M 2024-06-10T02:45:06.702 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:06 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:45:06.702 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:06 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:45:06.702 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:06 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T02:45:06.702 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:06 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:45:06.702 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:06 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T02:45:06.702 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:06 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:45:06.702 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:06 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T02:45:06.702 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:06 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:45:07.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:45:07.104+0000 7fbee0a91700 1 -- 172.21.15.52:0/2978102550 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbedc1013f0 msgr2=0x7fbedc101810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:45:07.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:45:07.104+0000 7fbee0a91700 1 --2- 172.21.15.52:0/2978102550 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbedc1013f0 0x7fbedc101810 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fbecc009990 tx=0x7fbecc009ca0 comp rx=0 tx=0).stop 2024-06-10T02:45:07.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:45:07.104+0000 7fbee0a91700 1 -- 172.21.15.52:0/2978102550 shutdown_connections 2024-06-10T02:45:07.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:45:07.104+0000 7fbee0a91700 1 --2- 172.21.15.52:0/2978102550 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbedc102f90 0x7fbedc109840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:45:07.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:45:07.104+0000 7fbee0a91700 1 --2- 172.21.15.52:0/2978102550 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbedc1013f0 0x7fbedc101810 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:45:07.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:45:07.104+0000 7fbee0a91700 1 --2- 172.21.15.52:0/2978102550 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbedc1025f0 0x7fbedc102a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:45:07.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:45:07.104+0000 7fbee0a91700 1 -- 172.21.15.52:0/2978102550 >> 172.21.15.52:0/2978102550 conn(0x7fbedc0fc990 msgr2=0x7fbedc0fedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:45:07.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:45:07.104+0000 7fbee0a91700 1 -- 172.21.15.52:0/2978102550 shutdown_connections 2024-06-10T02:45:07.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:45:07.104+0000 7fbee0a91700 1 -- 172.21.15.52:0/2978102550 wait complete. 2024-06-10T02:45:07.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:45:07.105+0000 7fbee0a91700 1 Processor -- start 2024-06-10T02:45:07.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:45:07.105+0000 7fbee0a91700 1 -- start start 2024-06-10T02:45:07.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:45:07.105+0000 7fbee0a91700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbedc1013f0 0x7fbedc1987e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:45:07.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:45:07.105+0000 7fbee0a91700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbedc1025f0 0x7fbedc198d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:45:07.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:45:07.105+0000 7fbee0a91700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbedc102f90 0x7fbedc19fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:45:07.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:45:07.105+0000 7fbee0a91700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fbedc10cfd0 con 0x7fbedc102f90 2024-06-10T02:45:07.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:45:07.105+0000 7fbee0a91700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fbedc10ce50 con 0x7fbedc1025f0 2024-06-10T02:45:07.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:45:07.105+0000 7fbee0a91700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fbedc10d150 con 0x7fbedc1013f0 2024-06-10T02:45:07.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:45:07.105+0000 7fbedad9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbedc102f90 0x7fbedc19fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:45:07.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:45:07.106+0000 7fbedad9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbedc102f90 0x7fbedc19fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:43742/0 (socket says 172.21.15.52:43742) 2024-06-10T02:45:07.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:45:07.106+0000 7fbedad9d700 1 -- 172.21.15.52:0/1145387884 learned_addr learned my addr 172.21.15.52:0/1145387884 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:45:07.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:45:07.106+0000 7fbeda59c700 1 --2- 172.21.15.52:0/1145387884 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbedc1013f0 0x7fbedc1987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:45:07.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:45:07.106+0000 7fbed9d9b700 1 --2- 172.21.15.52:0/1145387884 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbedc1025f0 0x7fbedc198d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:45:07.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:45:07.106+0000 7fbedad9d700 1 -- 172.21.15.52:0/1145387884 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbedc1013f0 msgr2=0x7fbedc1987e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:45:07.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:45:07.106+0000 7fbedad9d700 1 --2- 172.21.15.52:0/1145387884 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbedc1013f0 0x7fbedc1987e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:45:07.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:45:07.106+0000 7fbedad9d700 1 -- 172.21.15.52:0/1145387884 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbedc1025f0 msgr2=0x7fbedc198d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:45:07.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:45:07.106+0000 7fbedad9d700 1 --2- 172.21.15.52:0/1145387884 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbedc1025f0 0x7fbedc198d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:45:07.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:45:07.106+0000 7fbedad9d700 1 -- 172.21.15.52:0/1145387884 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbedc1a0440 con 0x7fbedc102f90 2024-06-10T02:45:07.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:45:07.106+0000 7fbed9d9b700 1 --2- 172.21.15.52:0/1145387884 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbedc1025f0 0x7fbedc198d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T02:45:07.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:45:07.106+0000 7fbedad9d700 1 --2- 172.21.15.52:0/1145387884 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbedc102f90 0x7fbedc19fda0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fbed400ef60 tx=0x7fbed400ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:45:07.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:45:07.394+0000 7fbecb7fe700 1 -- 172.21.15.52:0/1145387884 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbed40092e0 con 0x7fbedc102f90 2024-06-10T02:45:07.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:45:07.394+0000 7fbecb7fe700 1 -- 172.21.15.52:0/1145387884 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbed4004bb0 con 0x7fbedc102f90 2024-06-10T02:45:07.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:45:07.394+0000 7fbee0a91700 1 -- 172.21.15.52:0/1145387884 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbedc1a06d0 con 0x7fbedc102f90 2024-06-10T02:45:07.392 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:45:07.394+0000 7fbecb7fe700 1 -- 172.21.15.52:0/1145387884 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbed4004280 con 0x7fbedc102f90 2024-06-10T02:45:07.392 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:45:07.394+0000 7fbee0a91700 1 -- 172.21.15.52:0/1145387884 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbedc1a0c10 con 0x7fbedc102f90 2024-06-10T02:45:07.392 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:45:07.395+0000 7fbee0a91700 1 -- 172.21.15.52:0/1145387884 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbedc069180 con 0x7fbedc102f90 2024-06-10T02:45:07.393 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:45:07.397+0000 7fbecb7fe700 1 -- 172.21.15.52:0/1145387884 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7fbed401b030 con 0x7fbedc102f90 2024-06-10T02:45:07.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:45:07.397+0000 7fbecb7fe700 1 --2- 172.21.15.52:0/1145387884 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fbec40a2c10 0x7fbec40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:45:07.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:45:07.397+0000 7fbecb7fe700 1 -- 172.21.15.52:0/1145387884 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 2851+0+0 (secure 0 0 0) 0x7fbed40cddf0 con 0x7fbedc102f90 2024-06-10T02:45:07.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:45:07.397+0000 7fbeda59c700 1 --2- 172.21.15.52:0/1145387884 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fbec40a2c10 0x7fbec40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:45:07.395 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:45:07.398+0000 7fbeda59c700 1 --2- 172.21.15.52:0/1145387884 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fbec40a2c10 0x7fbec40a50d0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fbecc00b2e0 tx=0x7fbecc00b910 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:45:07.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:45:07.404+0000 7fbecb7fe700 1 -- 172.21.15.52:0/1145387884 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fbed4022020 con 0x7fbedc102f90 2024-06-10T02:45:07.617 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:45:07.620+0000 7fbee0a91700 1 -- 172.21.15.52:0/1145387884 --> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi052:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fbedc0febc0 con 0x7fbec40a2c10 2024-06-10T02:45:07.700 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:07 smithi188 ceph-mon[36599]: osdmap e19: 3 total, 3 up, 3 in 2024-06-10T02:45:07.723 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:07 smithi052 ceph-mon[32362]: osdmap e19: 3 total, 3 up, 3 in 2024-06-10T02:45:07.885 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:07 smithi120 ceph-mon[36634]: osdmap e19: 3 total, 3 up, 3 in 2024-06-10T02:45:08.560 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:08 smithi052 ceph-mon[32362]: pgmap v81: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-06-10T02:45:08.560 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:08 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-06-10T02:45:08.560 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:08 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-06-10T02:45:08.560 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:08 smithi052 ceph-mon[32362]: osdmap e20: 3 total, 3 up, 3 in 2024-06-10T02:45:08.561 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:08 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-06-10T02:45:08.561 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:08 smithi052 ceph-mon[32362]: from='client.14277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi052:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T02:45:08.561 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:08 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-10T02:45:08.561 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:08 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-10T02:45:08.561 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:08 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:45:08.700 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:08 smithi188 ceph-mon[36599]: pgmap v81: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-06-10T02:45:08.701 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:08 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-06-10T02:45:08.701 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:08 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-06-10T02:45:08.701 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:08 smithi188 ceph-mon[36599]: osdmap e20: 3 total, 3 up, 3 in 2024-06-10T02:45:08.701 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:08 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-06-10T02:45:08.701 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:08 smithi188 ceph-mon[36599]: from='client.14277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi052:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T02:45:08.701 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:08 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-10T02:45:08.701 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:08 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-10T02:45:08.701 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:08 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:45:08.885 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:08 smithi120 ceph-mon[36634]: pgmap v81: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-06-10T02:45:08.886 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:08 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-06-10T02:45:08.886 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:08 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-06-10T02:45:08.886 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:08 smithi120 ceph-mon[36634]: osdmap e20: 3 total, 3 up, 3 in 2024-06-10T02:45:08.886 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:08 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-06-10T02:45:08.886 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:08 smithi120 ceph-mon[36634]: from='client.14277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi052:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T02:45:08.886 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:08 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-10T02:45:08.886 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:08 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-10T02:45:08.886 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:08 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:45:09.256 INFO:journalctl@ceph.osd.0.smithi052.stdout:Jun 10 02:45:08 smithi052 sudo[51755]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-06-10T02:45:09.256 INFO:journalctl@ceph.osd.0.smithi052.stdout:Jun 10 02:45:08 smithi052 sudo[51755]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-10T02:45:09.256 INFO:journalctl@ceph.osd.0.smithi052.stdout:Jun 10 02:45:08 smithi052 sudo[51755]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-10T02:45:09.256 INFO:journalctl@ceph.osd.0.smithi052.stdout:Jun 10 02:45:09 smithi052 sudo[51755]: pam_unix(sudo:session): session closed for user root 2024-06-10T02:45:09.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:45:09.459+0000 7fbecb7fe700 1 -- 172.21.15.52:0/1145387884 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fbed4012050 con 0x7fbedc102f90 2024-06-10T02:45:09.505 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:09 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-06-10T02:45:09.506 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:09 smithi052 ceph-mon[32362]: osdmap e21: 3 total, 3 up, 3 in 2024-06-10T02:45:09.506 INFO:journalctl@ceph.osd.0.smithi052.stdout:Jun 10 02:45:09 smithi052 sudo[51780]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-06-10T02:45:09.506 INFO:journalctl@ceph.osd.0.smithi052.stdout:Jun 10 02:45:09 smithi052 sudo[51780]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-10T02:45:09.506 INFO:journalctl@ceph.osd.0.smithi052.stdout:Jun 10 02:45:09 smithi052 sudo[51780]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-10T02:45:09.506 INFO:journalctl@ceph.osd.0.smithi052.stdout:Jun 10 02:45:09 smithi052 sudo[51780]: pam_unix(sudo:session): session closed for user root 2024-06-10T02:45:09.713 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:09 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-06-10T02:45:09.714 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:09 smithi120 ceph-mon[36634]: osdmap e21: 3 total, 3 up, 3 in 2024-06-10T02:45:09.950 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:09 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-06-10T02:45:09.951 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:09 smithi188 ceph-mon[36599]: osdmap e21: 3 total, 3 up, 3 in 2024-06-10T02:45:09.969 INFO:journalctl@ceph.osd.1.smithi120.stdout:Jun 10 02:45:09 smithi120 sudo[44863]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-06-10T02:45:09.969 INFO:journalctl@ceph.osd.1.smithi120.stdout:Jun 10 02:45:09 smithi120 sudo[44863]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-10T02:45:09.969 INFO:journalctl@ceph.osd.1.smithi120.stdout:Jun 10 02:45:09 smithi120 sudo[44863]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-10T02:45:09.969 INFO:journalctl@ceph.osd.1.smithi120.stdout:Jun 10 02:45:09 smithi120 sudo[44863]: pam_unix(sudo:session): session closed for user root 2024-06-10T02:45:10.385 INFO:journalctl@ceph.osd.1.smithi120.stdout:Jun 10 02:45:09 smithi120 sudo[44866]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-06-10T02:45:10.385 INFO:journalctl@ceph.osd.1.smithi120.stdout:Jun 10 02:45:09 smithi120 sudo[44866]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-10T02:45:10.386 INFO:journalctl@ceph.osd.1.smithi120.stdout:Jun 10 02:45:09 smithi120 sudo[44866]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-10T02:45:10.386 INFO:journalctl@ceph.osd.1.smithi120.stdout:Jun 10 02:45:09 smithi120 sudo[44866]: pam_unix(sudo:session): session closed for user root 2024-06-10T02:45:10.491 INFO:journalctl@ceph.osd.2.smithi188.stdout:Jun 10 02:45:10 smithi188 sudo[45746]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-06-10T02:45:10.491 INFO:journalctl@ceph.osd.2.smithi188.stdout:Jun 10 02:45:10 smithi188 sudo[45746]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-10T02:45:10.491 INFO:journalctl@ceph.osd.2.smithi188.stdout:Jun 10 02:45:10 smithi188 sudo[45746]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-10T02:45:10.491 INFO:journalctl@ceph.osd.2.smithi188.stdout:Jun 10 02:45:10 smithi188 sudo[45746]: pam_unix(sudo:session): session closed for user root 2024-06-10T02:45:10.492 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:10 smithi188 ceph-mon[36599]: pgmap v84: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-06-10T02:45:10.492 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:10 smithi188 ceph-mon[36599]: osdmap e22: 3 total, 3 up, 3 in 2024-06-10T02:45:10.492 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:10 smithi188 ceph-mon[36599]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-06-10T02:45:10.492 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:10 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:45:10.492 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:10 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:45:10.492 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:10 smithi188 ceph-mon[36599]: from='client.? 172.21.15.52:0/3806161052' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "53680244-b49a-4120-b24a-9ad7a6341dc5"}]: dispatch 2024-06-10T02:45:10.492 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:10 smithi188 ceph-mon[36599]: from='client.? 172.21.15.52:0/3806161052' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "53680244-b49a-4120-b24a-9ad7a6341dc5"}]': finished 2024-06-10T02:45:10.492 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:10 smithi188 ceph-mon[36599]: osdmap e23: 4 total, 3 up, 4 in 2024-06-10T02:45:10.492 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:10 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-10T02:45:10.724 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:10 smithi052 ceph-mon[32362]: pgmap v84: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-06-10T02:45:10.725 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:10 smithi052 ceph-mon[32362]: osdmap e22: 3 total, 3 up, 3 in 2024-06-10T02:45:10.725 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:10 smithi052 ceph-mon[32362]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-06-10T02:45:10.725 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:10 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:45:10.725 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:10 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:45:10.725 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:10 smithi052 ceph-mon[32362]: from='client.? 172.21.15.52:0/3806161052' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "53680244-b49a-4120-b24a-9ad7a6341dc5"}]: dispatch 2024-06-10T02:45:10.725 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:10 smithi052 ceph-mon[32362]: from='client.? 172.21.15.52:0/3806161052' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "53680244-b49a-4120-b24a-9ad7a6341dc5"}]': finished 2024-06-10T02:45:10.725 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:10 smithi052 ceph-mon[32362]: osdmap e23: 4 total, 3 up, 4 in 2024-06-10T02:45:10.725 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:10 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-10T02:45:10.885 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:10 smithi120 ceph-mon[36634]: pgmap v84: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-06-10T02:45:10.886 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:10 smithi120 ceph-mon[36634]: osdmap e22: 3 total, 3 up, 3 in 2024-06-10T02:45:10.886 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:10 smithi120 ceph-mon[36634]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-06-10T02:45:10.886 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:10 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:45:10.886 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:10 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:45:10.886 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:10 smithi120 ceph-mon[36634]: from='client.? 172.21.15.52:0/3806161052' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "53680244-b49a-4120-b24a-9ad7a6341dc5"}]: dispatch 2024-06-10T02:45:10.886 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:10 smithi120 ceph-mon[36634]: from='client.? 172.21.15.52:0/3806161052' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "53680244-b49a-4120-b24a-9ad7a6341dc5"}]': finished 2024-06-10T02:45:10.886 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:10 smithi120 ceph-mon[36634]: osdmap e23: 4 total, 3 up, 4 in 2024-06-10T02:45:10.886 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:10 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-10T02:45:10.950 INFO:journalctl@ceph.osd.2.smithi188.stdout:Jun 10 02:45:10 smithi188 sudo[45749]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-06-10T02:45:10.951 INFO:journalctl@ceph.osd.2.smithi188.stdout:Jun 10 02:45:10 smithi188 sudo[45749]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-10T02:45:10.951 INFO:journalctl@ceph.osd.2.smithi188.stdout:Jun 10 02:45:10 smithi188 sudo[45749]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-10T02:45:10.951 INFO:journalctl@ceph.osd.2.smithi188.stdout:Jun 10 02:45:10 smithi188 sudo[45749]: pam_unix(sudo:session): session closed for user root 2024-06-10T02:45:11.224 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:10 smithi052 sudo[51859]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-06-10T02:45:11.224 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:10 smithi052 sudo[51859]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-10T02:45:11.224 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:10 smithi052 sudo[51859]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-10T02:45:11.224 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:10 smithi052 sudo[51859]: pam_unix(sudo:session): session closed for user root 2024-06-10T02:45:11.724 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:11 smithi052 sudo[51862]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-06-10T02:45:11.724 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:11 smithi052 sudo[51862]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-10T02:45:11.724 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:11 smithi052 sudo[51862]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-10T02:45:11.724 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:11 smithi052 sudo[51862]: pam_unix(sudo:session): session closed for user root 2024-06-10T02:45:11.908 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:11 smithi188 sudo[45752]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-06-10T02:45:11.909 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:11 smithi188 sudo[45752]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-10T02:45:11.909 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:11 smithi188 sudo[45752]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-10T02:45:11.909 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:11 smithi188 sudo[45752]: pam_unix(sudo:session): session closed for user root 2024-06-10T02:45:12.201 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:11 smithi188 sudo[45755]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-06-10T02:45:12.201 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:11 smithi188 sudo[45755]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-10T02:45:12.201 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:11 smithi188 sudo[45755]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-10T02:45:12.201 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:11 smithi188 sudo[45755]: pam_unix(sudo:session): session closed for user root 2024-06-10T02:45:12.201 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:11 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:45:12.201 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:11 smithi188 ceph-mon[36599]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-10T02:45:12.201 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:11 smithi188 ceph-mon[36599]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-10T02:45:12.201 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:11 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-10T02:45:12.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:11 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-10T02:45:12.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:11 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-10T02:45:12.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:11 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-10T02:45:12.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:11 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-10T02:45:12.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:11 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-10T02:45:12.224 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:11 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:45:12.224 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:11 smithi052 ceph-mon[32362]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-10T02:45:12.224 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:11 smithi052 ceph-mon[32362]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-10T02:45:12.224 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:11 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-10T02:45:12.224 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:11 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-10T02:45:12.224 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:11 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-10T02:45:12.224 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:11 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-10T02:45:12.224 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:11 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-10T02:45:12.224 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:11 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-10T02:45:12.410 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:12 smithi120 sudo[44869]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-06-10T02:45:12.411 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:12 smithi120 sudo[44869]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-10T02:45:12.411 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:12 smithi120 sudo[44869]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-10T02:45:12.411 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:12 smithi120 sudo[44869]: pam_unix(sudo:session): session closed for user root 2024-06-10T02:45:12.885 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:12 smithi120 sudo[44872]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-06-10T02:45:12.886 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:12 smithi120 sudo[44872]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-10T02:45:12.886 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:12 smithi120 sudo[44872]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-10T02:45:12.886 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:12 smithi120 sudo[44872]: pam_unix(sudo:session): session closed for user root 2024-06-10T02:45:12.886 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:12 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:45:12.886 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:12 smithi120 ceph-mon[36634]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-10T02:45:12.886 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:12 smithi120 ceph-mon[36634]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-10T02:45:12.886 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:12 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-10T02:45:12.886 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:12 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-10T02:45:12.886 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:12 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-10T02:45:12.886 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:12 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-10T02:45:12.887 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:12 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-10T02:45:12.887 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:12 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-10T02:45:13.201 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:12 smithi188 ceph-mon[36599]: pgmap v87: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-06-10T02:45:13.201 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:12 smithi188 ceph-mon[36599]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-10T02:45:13.201 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:12 smithi188 ceph-mon[36599]: from='client.? 172.21.15.52:0/753546621' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-10T02:45:13.201 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:12 smithi188 ceph-mon[36599]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-10T02:45:13.201 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:12 smithi188 ceph-mon[36599]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-10T02:45:13.201 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:12 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-10T02:45:13.201 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:12 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-10T02:45:13.201 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:12 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-10T02:45:13.201 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:12 smithi188 ceph-mon[36599]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-10T02:45:13.224 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:12 smithi052 ceph-mon[32362]: pgmap v87: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-06-10T02:45:13.224 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:12 smithi052 ceph-mon[32362]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-10T02:45:13.224 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:12 smithi052 ceph-mon[32362]: from='client.? 172.21.15.52:0/753546621' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-10T02:45:13.224 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:12 smithi052 ceph-mon[32362]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-10T02:45:13.224 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:12 smithi052 ceph-mon[32362]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-10T02:45:13.224 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:12 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-10T02:45:13.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:12 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-10T02:45:13.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:12 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-10T02:45:13.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:12 smithi052 ceph-mon[32362]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-10T02:45:13.386 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:12 smithi120 ceph-mon[36634]: pgmap v87: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-06-10T02:45:13.386 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:12 smithi120 ceph-mon[36634]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-10T02:45:13.386 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:12 smithi120 ceph-mon[36634]: from='client.? 172.21.15.52:0/753546621' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-10T02:45:13.386 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:12 smithi120 ceph-mon[36634]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-10T02:45:13.386 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:12 smithi120 ceph-mon[36634]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-10T02:45:13.386 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:12 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-10T02:45:13.386 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:12 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-10T02:45:13.386 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:12 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-10T02:45:13.386 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:12 smithi120 ceph-mon[36634]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-10T02:45:14.200 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:13 smithi188 ceph-mon[36599]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-06-10T02:45:14.224 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:13 smithi052 ceph-mon[32362]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-06-10T02:45:14.385 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:13 smithi120 ceph-mon[36634]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-06-10T02:45:16.184 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:15 smithi052 ceph-mon[32362]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-06-10T02:45:16.200 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:15 smithi188 ceph-mon[36599]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-06-10T02:45:16.385 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:15 smithi120 ceph-mon[36634]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-06-10T02:45:18.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:17 smithi052 ceph-mon[32362]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-06-10T02:45:18.200 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:17 smithi188 ceph-mon[36599]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-06-10T02:45:18.385 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:17 smithi120 ceph-mon[36634]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-06-10T02:45:19.198 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:18 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-06-10T02:45:19.198 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:18 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:45:19.198 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:18 smithi052 ceph-mon[32362]: Deploying daemon osd.3 on smithi052 2024-06-10T02:45:19.200 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:18 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-06-10T02:45:19.200 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:18 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:45:19.200 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:18 smithi188 ceph-mon[36599]: Deploying daemon osd.3 on smithi052 2024-06-10T02:45:19.385 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:18 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-06-10T02:45:19.386 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:18 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:45:19.386 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:18 smithi120 ceph-mon[36634]: Deploying daemon osd.3 on smithi052 2024-06-10T02:45:20.045 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:19 smithi052 ceph-mon[32362]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-06-10T02:45:20.201 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:19 smithi188 ceph-mon[36599]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-06-10T02:45:20.385 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:19 smithi120 ceph-mon[36634]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-06-10T02:45:21.974 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:21 smithi052 ceph-mon[32362]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-06-10T02:45:22.201 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:21 smithi188 ceph-mon[36599]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-06-10T02:45:22.385 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:21 smithi120 ceph-mon[36634]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-06-10T02:45:24.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:23 smithi188 ceph-mon[36599]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-06-10T02:45:24.224 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:23 smithi052 ceph-mon[32362]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-06-10T02:45:24.385 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:23 smithi120 ceph-mon[36634]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-06-10T02:45:25.635 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:25 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:45:25.636 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:25 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:45:25.636 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:25 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T02:45:25.701 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:25 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:45:25.701 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:25 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:45:25.701 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:25 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T02:45:25.724 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:25 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:45:25.724 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:25 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:45:25.724 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:25 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T02:45:26.408 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:26 smithi052 ceph-mon[32362]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-06-10T02:45:26.635 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:26 smithi120 ceph-mon[36634]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-06-10T02:45:26.701 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:26 smithi188 ceph-mon[36599]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-06-10T02:45:27.978 INFO:teuthology.orchestra.run.smithi052.stdout:Created osd(s) 3 on host 'smithi052' 2024-06-10T02:45:27.978 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:45:27.979+0000 7fbecb7fe700 1 -- 172.21.15.52:0/1145387884 <== mgr.14150 v2:172.21.15.52:6800/3214763969 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fbedc0febc0 con 0x7fbec40a2c10 2024-06-10T02:45:27.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:45:27.982+0000 7fbee0a91700 1 -- 172.21.15.52:0/1145387884 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fbec40a2c10 msgr2=0x7fbec40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:45:27.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:45:27.982+0000 7fbee0a91700 1 --2- 172.21.15.52:0/1145387884 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fbec40a2c10 0x7fbec40a50d0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fbecc00b2e0 tx=0x7fbecc00b910 comp rx=0 tx=0).stop 2024-06-10T02:45:27.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:45:27.982+0000 7fbee0a91700 1 -- 172.21.15.52:0/1145387884 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbedc102f90 msgr2=0x7fbedc19fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:45:27.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:45:27.982+0000 7fbee0a91700 1 --2- 172.21.15.52:0/1145387884 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbedc102f90 0x7fbedc19fda0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fbed400ef60 tx=0x7fbed400ef90 comp rx=0 tx=0).stop 2024-06-10T02:45:27.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:45:27.982+0000 7fbee0a91700 1 -- 172.21.15.52:0/1145387884 shutdown_connections 2024-06-10T02:45:27.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:45:27.982+0000 7fbee0a91700 1 --2- 172.21.15.52:0/1145387884 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fbec40a2c10 0x7fbec40a50d0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:45:27.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:45:27.982+0000 7fbee0a91700 1 --2- 172.21.15.52:0/1145387884 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbedc102f90 0x7fbedc19fda0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:45:27.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:45:27.982+0000 7fbee0a91700 1 --2- 172.21.15.52:0/1145387884 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbedc1013f0 0x7fbedc1987e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:45:27.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:45:27.982+0000 7fbee0a91700 1 --2- 172.21.15.52:0/1145387884 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbedc1025f0 0x7fbedc198d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:45:27.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:45:27.982+0000 7fbee0a91700 1 -- 172.21.15.52:0/1145387884 >> 172.21.15.52:0/1145387884 conn(0x7fbedc0fc990 msgr2=0x7fbedc0fe4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:45:27.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:45:27.983+0000 7fbee0a91700 1 -- 172.21.15.52:0/1145387884 shutdown_connections 2024-06-10T02:45:27.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:45:27.983+0000 7fbee0a91700 1 -- 172.21.15.52:0/1145387884 wait complete. 2024-06-10T02:45:28.189 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:28 smithi052 ceph-mon[32362]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-06-10T02:45:28.189 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:28 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:45:28.189 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:28 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:45:28.190 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:28 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:45:28.190 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:28 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T02:45:28.190 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:28 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:45:28.190 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:28 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:45:28.190 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:28 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:45:28.190 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:28 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T02:45:28.385 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:28 smithi120 ceph-mon[36634]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-06-10T02:45:28.386 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:28 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:45:28.386 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:28 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:45:28.386 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:28 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:45:28.386 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:28 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T02:45:28.386 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:28 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:45:28.386 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:28 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:45:28.386 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:28 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:45:28.386 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:28 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T02:45:28.451 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:28 smithi188 ceph-mon[36599]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-06-10T02:45:28.451 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:28 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:45:28.451 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:28 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:45:28.451 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:28 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:45:28.451 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:28 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T02:45:28.451 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:28 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:45:28.451 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:28 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:45:28.451 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:28 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:45:28.452 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:28 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T02:45:28.825 DEBUG:teuthology.orchestra.run.smithi052:osd.3> sudo journalctl -f -n 0 -u ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968@osd.3.service 2024-06-10T02:45:28.827 INFO:tasks.cephadm:Deploying osd.4 on smithi120 with /dev/vg_nvme/lv_3... 2024-06-10T02:45:28.828 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-06-10T02:45:29.073 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.b/config 2024-06-10T02:45:29.135 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:29 smithi120 ceph-mon[36634]: from='osd.3 [v2:172.21.15.52:6810/116897011,v1:172.21.15.52:6811/116897011]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-06-10T02:45:29.326 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:29 smithi052 ceph-mon[32362]: from='osd.3 [v2:172.21.15.52:6810/116897011,v1:172.21.15.52:6811/116897011]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-06-10T02:45:29.451 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:29 smithi188 ceph-mon[36599]: from='osd.3 [v2:172.21.15.52:6810/116897011,v1:172.21.15.52:6811/116897011]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-06-10T02:45:30.224 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:30 smithi052 ceph-mon[32362]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-06-10T02:45:30.224 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:30 smithi052 ceph-mon[32362]: from='osd.3 [v2:172.21.15.52:6810/116897011,v1:172.21.15.52:6811/116897011]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-06-10T02:45:30.224 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:30 smithi052 ceph-mon[32362]: osdmap e24: 4 total, 3 up, 4 in 2024-06-10T02:45:30.224 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:30 smithi052 ceph-mon[32362]: from='osd.3 [v2:172.21.15.52:6810/116897011,v1:172.21.15.52:6811/116897011]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]: dispatch 2024-06-10T02:45:30.224 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:30 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-10T02:45:30.386 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:30 smithi120 ceph-mon[36634]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-06-10T02:45:30.386 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:30 smithi120 ceph-mon[36634]: from='osd.3 [v2:172.21.15.52:6810/116897011,v1:172.21.15.52:6811/116897011]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-06-10T02:45:30.386 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:30 smithi120 ceph-mon[36634]: osdmap e24: 4 total, 3 up, 4 in 2024-06-10T02:45:30.386 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:30 smithi120 ceph-mon[36634]: from='osd.3 [v2:172.21.15.52:6810/116897011,v1:172.21.15.52:6811/116897011]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]: dispatch 2024-06-10T02:45:30.386 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:30 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-10T02:45:30.451 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:30 smithi188 ceph-mon[36599]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-06-10T02:45:30.451 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:30 smithi188 ceph-mon[36599]: from='osd.3 [v2:172.21.15.52:6810/116897011,v1:172.21.15.52:6811/116897011]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-06-10T02:45:30.451 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:30 smithi188 ceph-mon[36599]: osdmap e24: 4 total, 3 up, 4 in 2024-06-10T02:45:30.451 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:30 smithi188 ceph-mon[36599]: from='osd.3 [v2:172.21.15.52:6810/116897011,v1:172.21.15.52:6811/116897011]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]: dispatch 2024-06-10T02:45:30.451 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:30 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-10T02:45:30.975 INFO:journalctl@ceph.osd.3.smithi052.stdout:Jun 10 02:45:30 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-3[53862]: 2024-06-10T02:45:30.541+0000 7f3c44a71700 -1 osd.3 0 waiting for initial osdmap 2024-06-10T02:45:30.975 INFO:journalctl@ceph.osd.3.smithi052.stdout:Jun 10 02:45:30 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-3[53862]: 2024-06-10T02:45:30.547+0000 7f3c3c05b700 -1 osd.3 25 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-10T02:45:30.987 INFO:teuthology.orchestra.run.smithi120.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-06-10T02:45:30.987 INFO:teuthology.orchestra.run.smithi120.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-06-10T02:45:30.987 INFO:teuthology.orchestra.run.smithi120.stderr: stderr: 10+0 records in 2024-06-10T02:45:30.987 INFO:teuthology.orchestra.run.smithi120.stderr:10+0 records out 2024-06-10T02:45:30.987 INFO:teuthology.orchestra.run.smithi120.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0123178 s, 851 MB/s 2024-06-10T02:45:30.987 INFO:teuthology.orchestra.run.smithi120.stderr:--> Zapping successful for: 2024-06-10T02:45:31.286 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:31 smithi120 ceph-mon[36634]: from='osd.3 [v2:172.21.15.52:6810/116897011,v1:172.21.15.52:6811/116897011]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]': finished 2024-06-10T02:45:31.286 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:31 smithi120 ceph-mon[36634]: osdmap e25: 4 total, 3 up, 4 in 2024-06-10T02:45:31.286 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:31 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-10T02:45:31.287 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:31 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-10T02:45:31.287 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:31 smithi120 ceph-mon[36634]: from='osd.3 [v2:172.21.15.52:6810/116897011,v1:172.21.15.52:6811/116897011]' entity='osd.3' 2024-06-10T02:45:31.451 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:31 smithi188 ceph-mon[36599]: from='osd.3 [v2:172.21.15.52:6810/116897011,v1:172.21.15.52:6811/116897011]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]': finished 2024-06-10T02:45:31.451 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:31 smithi188 ceph-mon[36599]: osdmap e25: 4 total, 3 up, 4 in 2024-06-10T02:45:31.451 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:31 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-10T02:45:31.451 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:31 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-10T02:45:31.451 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:31 smithi188 ceph-mon[36599]: from='osd.3 [v2:172.21.15.52:6810/116897011,v1:172.21.15.52:6811/116897011]' entity='osd.3' 2024-06-10T02:45:31.474 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:31 smithi052 ceph-mon[32362]: from='osd.3 [v2:172.21.15.52:6810/116897011,v1:172.21.15.52:6811/116897011]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]': finished 2024-06-10T02:45:31.474 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:31 smithi052 ceph-mon[32362]: osdmap e25: 4 total, 3 up, 4 in 2024-06-10T02:45:31.474 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:31 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-10T02:45:31.474 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:31 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-10T02:45:31.474 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:31 smithi052 ceph-mon[32362]: from='osd.3 [v2:172.21.15.52:6810/116897011,v1:172.21.15.52:6811/116897011]' entity='osd.3' 2024-06-10T02:45:31.663 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi120:vg_nvme/lv_3 2024-06-10T02:45:31.900 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.b/config 2024-06-10T02:45:32.379 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:32 smithi120 ceph-mon[36634]: purged_snaps scrub starts 2024-06-10T02:45:32.380 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:32 smithi120 ceph-mon[36634]: purged_snaps scrub ok 2024-06-10T02:45:32.380 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:32 smithi120 ceph-mon[36634]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-06-10T02:45:32.380 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:32 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-10T02:45:32.380 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:32 smithi120 ceph-mon[36634]: osd.3 [v2:172.21.15.52:6810/116897011,v1:172.21.15.52:6811/116897011] boot 2024-06-10T02:45:32.380 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:32 smithi120 ceph-mon[36634]: osdmap e26: 4 total, 4 up, 4 in 2024-06-10T02:45:32.380 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:32 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-10T02:45:32.380 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:32 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:45:32.380 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:32 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:45:32.380 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:32 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-10T02:45:32.380 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:32 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-10T02:45:32.380 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:32 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:45:32.380 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:32 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:45:32.381 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:32 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T02:45:32.451 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:32 smithi188 ceph-mon[36599]: purged_snaps scrub starts 2024-06-10T02:45:32.451 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:32 smithi188 ceph-mon[36599]: purged_snaps scrub ok 2024-06-10T02:45:32.451 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:32 smithi188 ceph-mon[36599]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-06-10T02:45:32.451 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:32 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-10T02:45:32.451 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:32 smithi188 ceph-mon[36599]: osd.3 [v2:172.21.15.52:6810/116897011,v1:172.21.15.52:6811/116897011] boot 2024-06-10T02:45:32.451 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:32 smithi188 ceph-mon[36599]: osdmap e26: 4 total, 4 up, 4 in 2024-06-10T02:45:32.451 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:32 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-10T02:45:32.451 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:32 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:45:32.452 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:32 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:45:32.452 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:32 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-10T02:45:32.452 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:32 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-10T02:45:32.452 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:32 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:45:32.452 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:32 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:45:32.452 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:32 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T02:45:32.474 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:32 smithi052 ceph-mon[32362]: purged_snaps scrub starts 2024-06-10T02:45:32.474 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:32 smithi052 ceph-mon[32362]: purged_snaps scrub ok 2024-06-10T02:45:32.475 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:32 smithi052 ceph-mon[32362]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-06-10T02:45:32.475 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:32 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-10T02:45:32.475 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:32 smithi052 ceph-mon[32362]: osd.3 [v2:172.21.15.52:6810/116897011,v1:172.21.15.52:6811/116897011] boot 2024-06-10T02:45:32.475 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:32 smithi052 ceph-mon[32362]: osdmap e26: 4 total, 4 up, 4 in 2024-06-10T02:45:32.475 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:32 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-10T02:45:32.475 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:32 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:45:32.475 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:32 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:45:32.475 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:32 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-10T02:45:32.475 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:32 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-10T02:45:32.475 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:32 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:45:32.475 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:32 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:45:32.475 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:32 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T02:45:33.321 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:33 smithi120 ceph-mon[36634]: Detected new or changed devices on smithi052 2024-06-10T02:45:33.321 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:33 smithi120 ceph-mon[36634]: Adjusting osd_memory_target on smithi052 to 8546M 2024-06-10T02:45:33.322 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:33 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:45:33.322 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:33 smithi120 ceph-mon[36634]: osdmap e27: 4 total, 4 up, 4 in 2024-06-10T02:45:33.451 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:33 smithi188 ceph-mon[36599]: Detected new or changed devices on smithi052 2024-06-10T02:45:33.451 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:33 smithi188 ceph-mon[36599]: Adjusting osd_memory_target on smithi052 to 8546M 2024-06-10T02:45:33.451 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:33 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:45:33.451 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:33 smithi188 ceph-mon[36599]: osdmap e27: 4 total, 4 up, 4 in 2024-06-10T02:45:33.474 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:33 smithi052 ceph-mon[32362]: Detected new or changed devices on smithi052 2024-06-10T02:45:33.474 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:33 smithi052 ceph-mon[32362]: Adjusting osd_memory_target on smithi052 to 8546M 2024-06-10T02:45:33.474 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:33 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:45:33.474 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:33 smithi052 ceph-mon[32362]: osdmap e27: 4 total, 4 up, 4 in 2024-06-10T02:45:33.516 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:45:33.516+0000 7fb65ca83700 1 -- 172.21.15.120:0/2201304827 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb658103720 msgr2=0x7fb658103b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:45:33.517 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:45:33.516+0000 7fb65ca83700 1 --2- 172.21.15.120:0/2201304827 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb658103720 0x7fb658103b80 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fb64c0099c0 tx=0x7fb64c009cd0 comp rx=0 tx=0).stop 2024-06-10T02:45:33.517 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:45:33.516+0000 7fb65ca83700 1 -- 172.21.15.120:0/2201304827 shutdown_connections 2024-06-10T02:45:33.517 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:45:33.516+0000 7fb65ca83700 1 --2- 172.21.15.120:0/2201304827 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb6581040c0 0x7fb65810a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:45:33.517 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:45:33.516+0000 7fb65ca83700 1 --2- 172.21.15.120:0/2201304827 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb658102520 0x7fb658102940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:45:33.517 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:45:33.516+0000 7fb65ca83700 1 --2- 172.21.15.120:0/2201304827 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb658103720 0x7fb658103b80 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:45:33.518 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:45:33.516+0000 7fb65ca83700 1 -- 172.21.15.120:0/2201304827 >> 172.21.15.120:0/2201304827 conn(0x7fb6580fdaa0 msgr2=0x7fb6580fff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:45:33.518 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:45:33.516+0000 7fb65ca83700 1 -- 172.21.15.120:0/2201304827 shutdown_connections 2024-06-10T02:45:33.518 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:45:33.516+0000 7fb65ca83700 1 -- 172.21.15.120:0/2201304827 wait complete. 2024-06-10T02:45:33.518 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:45:33.517+0000 7fb65ca83700 1 Processor -- start 2024-06-10T02:45:33.518 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:45:33.517+0000 7fb65ca83700 1 -- start start 2024-06-10T02:45:33.518 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:45:33.517+0000 7fb65ca83700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb658102520 0x7fb65819a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:45:33.518 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:45:33.517+0000 7fb65ca83700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb658103720 0x7fb65819af20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:45:33.518 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:45:33.517+0000 7fb65ca83700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb6581040c0 0x7fb6581a1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:45:33.518 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:45:33.517+0000 7fb65ca83700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb65810e190 con 0x7fb6581040c0 2024-06-10T02:45:33.518 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:45:33.517+0000 7fb65ca83700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fb65810e010 con 0x7fb658102520 2024-06-10T02:45:33.518 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:45:33.517+0000 7fb65ca83700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fb65810e310 con 0x7fb658103720 2024-06-10T02:45:33.518 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:45:33.517+0000 7fb65659c700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb658102520 0x7fb65819a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:45:33.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:45:33.517+0000 7fb65659c700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb658102520 0x7fb65819a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:46820/0 (socket says 172.21.15.120:46820) 2024-06-10T02:45:33.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:45:33.517+0000 7fb65659c700 1 -- 172.21.15.120:0/3695254668 learned_addr learned my addr 172.21.15.120:0/3695254668 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-06-10T02:45:33.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:45:33.517+0000 7fb655d9b700 1 --2- 172.21.15.120:0/3695254668 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb658103720 0x7fb65819af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:45:33.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:45:33.517+0000 7fb656d9d700 1 --2- 172.21.15.120:0/3695254668 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb6581040c0 0x7fb6581a1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:45:33.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:45:33.517+0000 7fb65659c700 1 -- 172.21.15.120:0/3695254668 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb658103720 msgr2=0x7fb65819af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:45:33.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:45:33.517+0000 7fb65659c700 1 --2- 172.21.15.120:0/3695254668 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb658103720 0x7fb65819af20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:45:33.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:45:33.517+0000 7fb65659c700 1 -- 172.21.15.120:0/3695254668 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb6581040c0 msgr2=0x7fb6581a1fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:45:33.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:45:33.517+0000 7fb65659c700 1 --2- 172.21.15.120:0/3695254668 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb6581040c0 0x7fb6581a1fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:45:33.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:45:33.517+0000 7fb65659c700 1 -- 172.21.15.120:0/3695254668 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6581a2640 con 0x7fb658102520 2024-06-10T02:45:33.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:45:33.517+0000 7fb656d9d700 1 --2- 172.21.15.120:0/3695254668 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb6581040c0 0x7fb6581a1fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T02:45:33.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:45:33.518+0000 7fb65659c700 1 --2- 172.21.15.120:0/3695254668 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb658102520 0x7fb65819a9e0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fb64800b500 tx=0x7fb64800b810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:45:33.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:45:33.518+0000 7fb6477fe700 1 -- 172.21.15.120:0/3695254668 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb648015020 con 0x7fb658102520 2024-06-10T02:45:33.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:45:33.518+0000 7fb65ca83700 1 -- 172.21.15.120:0/3695254668 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6581a2930 con 0x7fb658102520 2024-06-10T02:45:33.520 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:45:33.518+0000 7fb6477fe700 1 -- 172.21.15.120:0/3695254668 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb648011d10 con 0x7fb658102520 2024-06-10T02:45:33.520 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:45:33.518+0000 7fb6477fe700 1 -- 172.21.15.120:0/3695254668 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb64800f2e0 con 0x7fb658102520 2024-06-10T02:45:33.520 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:45:33.518+0000 7fb65ca83700 1 -- 172.21.15.120:0/3695254668 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6581a2ef0 con 0x7fb658102520 2024-06-10T02:45:33.520 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:45:33.519+0000 7fb65ca83700 1 -- 172.21.15.120:0/3695254668 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb658069180 con 0x7fb658102520 2024-06-10T02:45:33.521 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:45:33.520+0000 7fb6477fe700 1 -- 172.21.15.120:0/3695254668 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fb648018020 con 0x7fb658102520 2024-06-10T02:45:33.521 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:45:33.520+0000 7fb6477fe700 1 --2- 172.21.15.120:0/3695254668 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fb6400a2c20 0x7fb6400a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:45:33.521 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:45:33.520+0000 7fb6477fe700 1 -- 172.21.15.120:0/3695254668 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 3698+0+0 (secure 0 0 0) 0x7fb6480cdb80 con 0x7fb658102520 2024-06-10T02:45:33.521 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:45:33.521+0000 7fb655d9b700 1 --2- 172.21.15.120:0/3695254668 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fb6400a2c20 0x7fb6400a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:45:33.521 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:45:33.521+0000 7fb655d9b700 1 --2- 172.21.15.120:0/3695254668 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fb6400a2c20 0x7fb6400a50e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb64c000c00 tx=0x7fb64c005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:45:33.521 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:45:33.524+0000 7fb6477fe700 1 -- 172.21.15.120:0/3695254668 <== mon.2 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fb64801f020 con 0x7fb658102520 2024-06-10T02:45:33.724 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:45:33.727+0000 7fb65ca83700 1 -- 172.21.15.120:0/3695254668 --> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi120:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fb658075de0 con 0x7fb6400a2c20 2024-06-10T02:45:34.345 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:34 smithi120 ceph-mon[36634]: pgmap v102: 1 pgs: 1 remapped+peering; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-06-10T02:45:34.345 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:34 smithi120 ceph-mon[36634]: osdmap e28: 4 total, 4 up, 4 in 2024-06-10T02:45:34.345 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:34 smithi120 ceph-mon[36634]: from='client.24218 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi120:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T02:45:34.346 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:34 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-10T02:45:34.346 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:34 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-10T02:45:34.346 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:34 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:45:34.375 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:34 smithi052 ceph-mon[32362]: pgmap v102: 1 pgs: 1 remapped+peering; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-06-10T02:45:34.375 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:34 smithi052 ceph-mon[32362]: osdmap e28: 4 total, 4 up, 4 in 2024-06-10T02:45:34.375 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:34 smithi052 ceph-mon[32362]: from='client.24218 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi120:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T02:45:34.375 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:34 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-10T02:45:34.375 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:34 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-10T02:45:34.375 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:34 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:45:34.451 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:34 smithi188 ceph-mon[36599]: pgmap v102: 1 pgs: 1 remapped+peering; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-06-10T02:45:34.451 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:34 smithi188 ceph-mon[36599]: osdmap e28: 4 total, 4 up, 4 in 2024-06-10T02:45:34.451 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:34 smithi188 ceph-mon[36599]: from='client.24218 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi120:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T02:45:34.451 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:34 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-10T02:45:34.451 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:34 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-10T02:45:34.451 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:34 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:45:35.359 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:35 smithi120 ceph-mon[36634]: osdmap e29: 4 total, 4 up, 4 in 2024-06-10T02:45:35.451 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:35 smithi188 ceph-mon[36599]: osdmap e29: 4 total, 4 up, 4 in 2024-06-10T02:45:35.474 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:35 smithi052 ceph-mon[32362]: osdmap e29: 4 total, 4 up, 4 in 2024-06-10T02:45:36.386 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:36 smithi120 ceph-mon[36634]: pgmap v105: 1 pgs: 1 remapped+peering; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-06-10T02:45:36.386 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:36 smithi120 ceph-mon[36634]: osdmap e30: 4 total, 4 up, 4 in 2024-06-10T02:45:36.451 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:36 smithi188 ceph-mon[36599]: pgmap v105: 1 pgs: 1 remapped+peering; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-06-10T02:45:36.451 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:36 smithi188 ceph-mon[36599]: osdmap e30: 4 total, 4 up, 4 in 2024-06-10T02:45:36.474 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:36 smithi052 ceph-mon[32362]: pgmap v105: 1 pgs: 1 remapped+peering; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-06-10T02:45:36.474 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:36 smithi052 ceph-mon[32362]: osdmap e30: 4 total, 4 up, 4 in 2024-06-10T02:45:37.385 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:37 smithi120 ceph-mon[36634]: from='client.? 172.21.15.120:0/3010328961' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9f72f1f4-2ef3-49c5-8410-c108767d2ef5"}]: dispatch 2024-06-10T02:45:37.386 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:37 smithi120 ceph-mon[36634]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9f72f1f4-2ef3-49c5-8410-c108767d2ef5"}]: dispatch 2024-06-10T02:45:37.386 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:37 smithi120 ceph-mon[36634]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9f72f1f4-2ef3-49c5-8410-c108767d2ef5"}]': finished 2024-06-10T02:45:37.386 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:37 smithi120 ceph-mon[36634]: osdmap e31: 5 total, 4 up, 5 in 2024-06-10T02:45:37.386 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:37 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-10T02:45:37.386 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:37 smithi120 ceph-mon[36634]: from='client.? 172.21.15.120:0/1118950591' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-10T02:45:37.451 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:37 smithi188 ceph-mon[36599]: from='client.? 172.21.15.120:0/3010328961' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9f72f1f4-2ef3-49c5-8410-c108767d2ef5"}]: dispatch 2024-06-10T02:45:37.451 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:37 smithi188 ceph-mon[36599]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9f72f1f4-2ef3-49c5-8410-c108767d2ef5"}]: dispatch 2024-06-10T02:45:37.451 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:37 smithi188 ceph-mon[36599]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9f72f1f4-2ef3-49c5-8410-c108767d2ef5"}]': finished 2024-06-10T02:45:37.451 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:37 smithi188 ceph-mon[36599]: osdmap e31: 5 total, 4 up, 5 in 2024-06-10T02:45:37.451 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:37 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-10T02:45:37.451 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:37 smithi188 ceph-mon[36599]: from='client.? 172.21.15.120:0/1118950591' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-10T02:45:37.474 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:37 smithi052 ceph-mon[32362]: from='client.? 172.21.15.120:0/3010328961' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9f72f1f4-2ef3-49c5-8410-c108767d2ef5"}]: dispatch 2024-06-10T02:45:37.474 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:37 smithi052 ceph-mon[32362]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9f72f1f4-2ef3-49c5-8410-c108767d2ef5"}]: dispatch 2024-06-10T02:45:37.474 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:37 smithi052 ceph-mon[32362]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9f72f1f4-2ef3-49c5-8410-c108767d2ef5"}]': finished 2024-06-10T02:45:37.474 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:37 smithi052 ceph-mon[32362]: osdmap e31: 5 total, 4 up, 5 in 2024-06-10T02:45:37.475 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:37 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-10T02:45:37.475 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:37 smithi052 ceph-mon[32362]: from='client.? 172.21.15.120:0/1118950591' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-10T02:45:38.700 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:38 smithi188 ceph-mon[36599]: pgmap v108: 1 pgs: 1 remapped+peering; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-06-10T02:45:38.724 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:38 smithi052 ceph-mon[32362]: pgmap v108: 1 pgs: 1 remapped+peering; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-06-10T02:45:38.885 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:38 smithi120 ceph-mon[36634]: pgmap v108: 1 pgs: 1 remapped+peering; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-06-10T02:45:40.635 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:40 smithi120 ceph-mon[36634]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 99 KiB/s, 0 objects/s recovering 2024-06-10T02:45:40.700 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:40 smithi188 ceph-mon[36599]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 99 KiB/s, 0 objects/s recovering 2024-06-10T02:45:40.724 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:40 smithi052 ceph-mon[32362]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 99 KiB/s, 0 objects/s recovering 2024-06-10T02:45:42.604 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:42 smithi120 ceph-mon[36634]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-06-10T02:45:42.701 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:42 smithi188 ceph-mon[36599]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-06-10T02:45:42.724 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:42 smithi052 ceph-mon[32362]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-06-10T02:45:44.636 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:44 smithi120 ceph-mon[36634]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-06-10T02:45:44.636 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:44 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-06-10T02:45:44.636 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:44 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:45:44.636 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:44 smithi120 ceph-mon[36634]: Deploying daemon osd.4 on smithi120 2024-06-10T02:45:44.701 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:44 smithi188 ceph-mon[36599]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-06-10T02:45:44.701 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:44 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-06-10T02:45:44.701 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:44 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:45:44.701 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:44 smithi188 ceph-mon[36599]: Deploying daemon osd.4 on smithi120 2024-06-10T02:45:44.724 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:44 smithi052 ceph-mon[32362]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-06-10T02:45:44.724 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:44 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-06-10T02:45:44.724 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:44 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:45:44.724 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:44 smithi052 ceph-mon[32362]: Deploying daemon osd.4 on smithi120 2024-06-10T02:45:46.700 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:46 smithi188 ceph-mon[36599]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-06-10T02:45:46.724 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:46 smithi052 ceph-mon[32362]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-06-10T02:45:46.765 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:46 smithi120 ceph-mon[36634]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-06-10T02:45:48.494 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:48 smithi120 ceph-mon[36634]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 55 KiB/s, 0 objects/s recovering 2024-06-10T02:45:48.701 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:48 smithi188 ceph-mon[36599]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 55 KiB/s, 0 objects/s recovering 2024-06-10T02:45:48.724 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:48 smithi052 ceph-mon[32362]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 55 KiB/s, 0 objects/s recovering 2024-06-10T02:45:50.385 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:50 smithi120 ceph-mon[36634]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-06-10T02:45:50.386 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:50 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:45:50.386 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:50 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:45:50.386 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:50 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T02:45:50.451 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:50 smithi188 ceph-mon[36599]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-06-10T02:45:50.451 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:50 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:45:50.451 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:50 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:45:50.451 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:50 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T02:45:50.474 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:50 smithi052 ceph-mon[32362]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-06-10T02:45:50.474 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:50 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:45:50.474 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:50 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:45:50.474 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:50 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T02:45:52.386 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:52 smithi120 ceph-mon[36634]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-06-10T02:45:52.386 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:52 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:45:52.386 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:52 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:45:52.386 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:52 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:45:52.387 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:52 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T02:45:52.387 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:52 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:45:52.451 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:52 smithi188 ceph-mon[36599]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-06-10T02:45:52.451 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:52 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:45:52.451 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:52 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:45:52.451 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:52 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:45:52.451 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:52 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T02:45:52.451 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:52 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:45:52.474 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:52 smithi052 ceph-mon[32362]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-06-10T02:45:52.474 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:52 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:45:52.474 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:52 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:45:52.474 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:52 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:45:52.475 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:52 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T02:45:52.475 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:52 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:45:52.992 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:45:52.995+0000 7fb6477fe700 1 -- 172.21.15.120:0/3695254668 <== mgr.14150 v2:172.21.15.52:6800/3214763969 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fb658075de0 con 0x7fb6400a2c20 2024-06-10T02:45:52.993 INFO:teuthology.orchestra.run.smithi120.stdout:Created osd(s) 4 on host 'smithi120' 2024-06-10T02:45:52.995 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:45:52.997+0000 7fb65ca83700 1 -- 172.21.15.120:0/3695254668 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fb6400a2c20 msgr2=0x7fb6400a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:45:52.995 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:45:52.997+0000 7fb65ca83700 1 --2- 172.21.15.120:0/3695254668 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fb6400a2c20 0x7fb6400a50e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb64c000c00 tx=0x7fb64c005fb0 comp rx=0 tx=0).stop 2024-06-10T02:45:52.995 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:45:52.997+0000 7fb65ca83700 1 -- 172.21.15.120:0/3695254668 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb658102520 msgr2=0x7fb65819a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:45:52.995 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:45:52.997+0000 7fb65ca83700 1 --2- 172.21.15.120:0/3695254668 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb658102520 0x7fb65819a9e0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fb64800b500 tx=0x7fb64800b810 comp rx=0 tx=0).stop 2024-06-10T02:45:52.995 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:45:52.997+0000 7fb65ca83700 1 -- 172.21.15.120:0/3695254668 shutdown_connections 2024-06-10T02:45:52.995 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:45:52.997+0000 7fb65ca83700 1 --2- 172.21.15.120:0/3695254668 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fb6400a2c20 0x7fb6400a50e0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:45:52.995 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:45:52.997+0000 7fb65ca83700 1 --2- 172.21.15.120:0/3695254668 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb6581040c0 0x7fb6581a1fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:45:52.995 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:45:52.997+0000 7fb65ca83700 1 --2- 172.21.15.120:0/3695254668 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb658102520 0x7fb65819a9e0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:45:52.995 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:45:52.997+0000 7fb65ca83700 1 --2- 172.21.15.120:0/3695254668 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb658103720 0x7fb65819af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:45:52.996 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:45:52.997+0000 7fb65ca83700 1 -- 172.21.15.120:0/3695254668 >> 172.21.15.120:0/3695254668 conn(0x7fb6580fdaa0 msgr2=0x7fb6580746c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:45:52.996 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:45:52.997+0000 7fb65ca83700 1 -- 172.21.15.120:0/3695254668 shutdown_connections 2024-06-10T02:45:52.996 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:45:52.997+0000 7fb65ca83700 1 -- 172.21.15.120:0/3695254668 wait complete. 2024-06-10T02:45:53.694 DEBUG:teuthology.orchestra.run.smithi120:osd.4> sudo journalctl -f -n 0 -u ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968@osd.4.service 2024-06-10T02:45:53.696 INFO:tasks.cephadm:Deploying osd.5 on smithi188 with /dev/vg_nvme/lv_3... 2024-06-10T02:45:53.696 DEBUG:teuthology.orchestra.run.smithi188:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-06-10T02:45:53.939 INFO:teuthology.orchestra.run.smithi188.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.c/config 2024-06-10T02:45:54.062 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:53 smithi188 ceph-mon[36599]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-06-10T02:45:54.062 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:53 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:45:54.062 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:53 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T02:45:54.062 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:53 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:45:54.062 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:53 smithi188 ceph-mon[36599]: from='osd.4 [v2:172.21.15.120:6808/1320690621,v1:172.21.15.120:6809/1320690621]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-06-10T02:45:54.062 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:53 smithi188 ceph-mon[36599]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-06-10T02:45:54.224 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:53 smithi052 ceph-mon[32362]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-06-10T02:45:54.224 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:53 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:45:54.224 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:53 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T02:45:54.224 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:53 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:45:54.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:53 smithi052 ceph-mon[32362]: from='osd.4 [v2:172.21.15.120:6808/1320690621,v1:172.21.15.120:6809/1320690621]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-06-10T02:45:54.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:53 smithi052 ceph-mon[32362]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-06-10T02:45:54.255 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:53 smithi120 ceph-mon[36634]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-06-10T02:45:54.255 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:53 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:45:54.255 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:53 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T02:45:54.256 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:53 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:45:54.256 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:53 smithi120 ceph-mon[36634]: from='osd.4 [v2:172.21.15.120:6808/1320690621,v1:172.21.15.120:6809/1320690621]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-06-10T02:45:54.256 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:53 smithi120 ceph-mon[36634]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-06-10T02:45:55.224 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:54 smithi052 ceph-mon[32362]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-06-10T02:45:55.224 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:54 smithi052 ceph-mon[32362]: osdmap e32: 5 total, 4 up, 5 in 2024-06-10T02:45:55.224 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:54 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-10T02:45:55.224 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:54 smithi052 ceph-mon[32362]: from='osd.4 [v2:172.21.15.120:6808/1320690621,v1:172.21.15.120:6809/1320690621]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]: dispatch 2024-06-10T02:45:55.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:54 smithi052 ceph-mon[32362]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]: dispatch 2024-06-10T02:45:55.235 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:54 smithi120 ceph-mon[36634]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-06-10T02:45:55.235 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:54 smithi120 ceph-mon[36634]: osdmap e32: 5 total, 4 up, 5 in 2024-06-10T02:45:55.236 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:54 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-10T02:45:55.236 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:54 smithi120 ceph-mon[36634]: from='osd.4 [v2:172.21.15.120:6808/1320690621,v1:172.21.15.120:6809/1320690621]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]: dispatch 2024-06-10T02:45:55.236 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:54 smithi120 ceph-mon[36634]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]: dispatch 2024-06-10T02:45:55.236 INFO:journalctl@ceph.osd.4.smithi120.stdout:Jun 10 02:45:54 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-4[47679]: 2024-06-10T02:45:54.974+0000 7f4dfd70f700 -1 osd.4 0 waiting for initial osdmap 2024-06-10T02:45:55.236 INFO:journalctl@ceph.osd.4.smithi120.stdout:Jun 10 02:45:54 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-4[47679]: 2024-06-10T02:45:54.984+0000 7f4df9502700 -1 osd.4 33 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-10T02:45:55.290 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:54 smithi188 ceph-mon[36599]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-06-10T02:45:55.290 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:54 smithi188 ceph-mon[36599]: osdmap e32: 5 total, 4 up, 5 in 2024-06-10T02:45:55.290 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:54 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-10T02:45:55.291 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:54 smithi188 ceph-mon[36599]: from='osd.4 [v2:172.21.15.120:6808/1320690621,v1:172.21.15.120:6809/1320690621]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]: dispatch 2024-06-10T02:45:55.291 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:54 smithi188 ceph-mon[36599]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]: dispatch 2024-06-10T02:45:55.882 INFO:teuthology.orchestra.run.smithi188.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-06-10T02:45:55.882 INFO:teuthology.orchestra.run.smithi188.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-06-10T02:45:55.882 INFO:teuthology.orchestra.run.smithi188.stderr: stderr: 10+0 records in 2024-06-10T02:45:55.882 INFO:teuthology.orchestra.run.smithi188.stderr:10+0 records out 2024-06-10T02:45:55.883 INFO:teuthology.orchestra.run.smithi188.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.012252 s, 856 MB/s 2024-06-10T02:45:55.883 INFO:teuthology.orchestra.run.smithi188.stderr:--> Zapping successful for: 2024-06-10T02:45:56.224 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:55 smithi052 ceph-mon[32362]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-06-10T02:45:56.224 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:55 smithi052 ceph-mon[32362]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]': finished 2024-06-10T02:45:56.224 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:55 smithi052 ceph-mon[32362]: osdmap e33: 5 total, 4 up, 5 in 2024-06-10T02:45:56.224 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:55 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-10T02:45:56.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:55 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-10T02:45:56.296 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:55 smithi188 ceph-mon[36599]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-06-10T02:45:56.296 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:55 smithi188 ceph-mon[36599]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]': finished 2024-06-10T02:45:56.296 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:55 smithi188 ceph-mon[36599]: osdmap e33: 5 total, 4 up, 5 in 2024-06-10T02:45:56.296 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:55 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-10T02:45:56.296 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:55 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-10T02:45:56.322 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:55 smithi120 ceph-mon[36634]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-06-10T02:45:56.322 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:55 smithi120 ceph-mon[36634]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]': finished 2024-06-10T02:45:56.322 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:55 smithi120 ceph-mon[36634]: osdmap e33: 5 total, 4 up, 5 in 2024-06-10T02:45:56.322 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:55 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-10T02:45:56.322 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:55 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-10T02:45:56.577 DEBUG:teuthology.orchestra.run.smithi188:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi188:vg_nvme/lv_3 2024-06-10T02:45:56.825 INFO:teuthology.orchestra.run.smithi188.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.c/config 2024-06-10T02:45:57.125 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:56 smithi188 ceph-mon[36599]: purged_snaps scrub starts 2024-06-10T02:45:57.125 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:56 smithi188 ceph-mon[36599]: purged_snaps scrub ok 2024-06-10T02:45:57.125 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:56 smithi188 ceph-mon[36599]: osd.4 [v2:172.21.15.120:6808/1320690621,v1:172.21.15.120:6809/1320690621] boot 2024-06-10T02:45:57.125 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:56 smithi188 ceph-mon[36599]: osdmap e34: 5 total, 5 up, 5 in 2024-06-10T02:45:57.125 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:56 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-10T02:45:57.125 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:56 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:45:57.125 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:56 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:45:57.125 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:56 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-10T02:45:57.125 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:56 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-10T02:45:57.126 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:56 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:45:57.126 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:56 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:45:57.126 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:56 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T02:45:57.126 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:56 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:45:57.135 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:56 smithi120 ceph-mon[36634]: purged_snaps scrub starts 2024-06-10T02:45:57.136 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:56 smithi120 ceph-mon[36634]: purged_snaps scrub ok 2024-06-10T02:45:57.136 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:56 smithi120 ceph-mon[36634]: osd.4 [v2:172.21.15.120:6808/1320690621,v1:172.21.15.120:6809/1320690621] boot 2024-06-10T02:45:57.136 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:56 smithi120 ceph-mon[36634]: osdmap e34: 5 total, 5 up, 5 in 2024-06-10T02:45:57.136 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:56 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-10T02:45:57.136 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:56 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:45:57.136 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:56 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:45:57.136 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:56 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-10T02:45:57.136 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:56 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-10T02:45:57.136 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:56 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:45:57.136 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:56 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:45:57.136 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:56 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T02:45:57.136 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:56 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:45:57.224 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:56 smithi052 ceph-mon[32362]: purged_snaps scrub starts 2024-06-10T02:45:57.224 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:56 smithi052 ceph-mon[32362]: purged_snaps scrub ok 2024-06-10T02:45:57.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:56 smithi052 ceph-mon[32362]: osd.4 [v2:172.21.15.120:6808/1320690621,v1:172.21.15.120:6809/1320690621] boot 2024-06-10T02:45:57.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:56 smithi052 ceph-mon[32362]: osdmap e34: 5 total, 5 up, 5 in 2024-06-10T02:45:57.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:56 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-10T02:45:57.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:56 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:45:57.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:56 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:45:57.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:56 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-10T02:45:57.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:56 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-10T02:45:57.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:56 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:45:57.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:56 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:45:57.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:56 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T02:45:57.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:56 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:45:58.190 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:57 smithi188 ceph-mon[36599]: Detected new or changed devices on smithi120 2024-06-10T02:45:58.190 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:57 smithi188 ceph-mon[36599]: Adjusting osd_memory_target on smithi120 to 8544M 2024-06-10T02:45:58.190 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:57 smithi188 ceph-mon[36599]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-06-10T02:45:58.190 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:57 smithi188 ceph-mon[36599]: osdmap e35: 5 total, 5 up, 5 in 2024-06-10T02:45:58.386 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:57 smithi120 ceph-mon[36634]: Detected new or changed devices on smithi120 2024-06-10T02:45:58.386 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:57 smithi120 ceph-mon[36634]: Adjusting osd_memory_target on smithi120 to 8544M 2024-06-10T02:45:58.386 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:57 smithi120 ceph-mon[36634]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-06-10T02:45:58.386 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:57 smithi120 ceph-mon[36634]: osdmap e35: 5 total, 5 up, 5 in 2024-06-10T02:45:58.410 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:45:58.411+0000 7f39dc060700 1 -- 172.21.15.188:0/3285990308 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f39d41024c0 msgr2=0x7f39d41028e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:45:58.410 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:45:58.411+0000 7f39dc060700 1 --2- 172.21.15.188:0/3285990308 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f39d41024c0 0x7f39d41028e0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f39d0009990 tx=0x7f39d0009ca0 comp rx=0 tx=0).stop 2024-06-10T02:45:58.410 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:45:58.411+0000 7f39dc060700 1 -- 172.21.15.188:0/3285990308 shutdown_connections 2024-06-10T02:45:58.411 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:45:58.411+0000 7f39dc060700 1 --2- 172.21.15.188:0/3285990308 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f39d4104060 0x7f39d410a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:45:58.411 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:45:58.411+0000 7f39dc060700 1 --2- 172.21.15.188:0/3285990308 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f39d41024c0 0x7f39d41028e0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:45:58.411 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:45:58.411+0000 7f39dc060700 1 --2- 172.21.15.188:0/3285990308 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f39d41036c0 0x7f39d4103b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:45:58.411 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:45:58.411+0000 7f39dc060700 1 -- 172.21.15.188:0/3285990308 >> 172.21.15.188:0/3285990308 conn(0x7f39d40fda60 msgr2=0x7f39d40ffea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:45:58.411 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:45:58.411+0000 7f39dc060700 1 -- 172.21.15.188:0/3285990308 shutdown_connections 2024-06-10T02:45:58.411 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:45:58.411+0000 7f39dc060700 1 -- 172.21.15.188:0/3285990308 wait complete. 2024-06-10T02:45:58.411 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:45:58.411+0000 7f39dc060700 1 Processor -- start 2024-06-10T02:45:58.411 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:45:58.411+0000 7f39dc060700 1 -- start start 2024-06-10T02:45:58.411 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:45:58.411+0000 7f39dc060700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f39d41024c0 0x7f39d4198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:45:58.411 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:45:58.411+0000 7f39dc060700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f39d41036c0 0x7f39d4198ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:45:58.411 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:45:58.411+0000 7f39dc060700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f39d4104060 0x7f39d419fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:45:58.411 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:45:58.412+0000 7f39dc060700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f39d410e110 con 0x7f39d41036c0 2024-06-10T02:45:58.411 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:45:58.412+0000 7f39dc060700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f39d410df90 con 0x7f39d41024c0 2024-06-10T02:45:58.412 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:45:58.412+0000 7f39dc060700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f39d410e290 con 0x7f39d4104060 2024-06-10T02:45:58.412 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:45:58.412+0000 7f39da5fd700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f39d4104060 0x7f39d419fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:45:58.412 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:45:58.412+0000 7f39da5fd700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f39d4104060 0x7f39d419fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.188:47808/0 (socket says 172.21.15.188:47808) 2024-06-10T02:45:58.412 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:45:58.412+0000 7f39da5fd700 1 -- 172.21.15.188:0/775786833 learned_addr learned my addr 172.21.15.188:0/775786833 (peer_addr_for_me v2:172.21.15.188:0/0) 2024-06-10T02:45:58.412 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:45:58.412+0000 7f39d95fb700 1 --2- 172.21.15.188:0/775786833 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f39d41036c0 0x7f39d4198ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:45:58.412 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:45:58.412+0000 7f39d9dfc700 1 --2- 172.21.15.188:0/775786833 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f39d41024c0 0x7f39d4198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:45:58.412 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:45:58.412+0000 7f39da5fd700 1 -- 172.21.15.188:0/775786833 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f39d41024c0 msgr2=0x7f39d4198760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:45:58.412 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:45:58.412+0000 7f39da5fd700 1 --2- 172.21.15.188:0/775786833 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f39d41024c0 0x7f39d4198760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:45:58.412 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:45:58.412+0000 7f39da5fd700 1 -- 172.21.15.188:0/775786833 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f39d41036c0 msgr2=0x7f39d4198ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:45:58.412 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:45:58.412+0000 7f39da5fd700 1 --2- 172.21.15.188:0/775786833 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f39d41036c0 0x7f39d4198ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:45:58.412 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:45:58.412+0000 7f39da5fd700 1 -- 172.21.15.188:0/775786833 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39d41a03c0 con 0x7f39d4104060 2024-06-10T02:45:58.412 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:45:58.412+0000 7f39d9dfc700 1 --2- 172.21.15.188:0/775786833 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f39d41024c0 0x7f39d4198760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T02:45:58.412 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:45:58.412+0000 7f39da5fd700 1 --2- 172.21.15.188:0/775786833 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f39d4104060 0x7f39d419fd20 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f39cc00ef60 tx=0x7f39cc00ef90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:45:58.413 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:45:58.412+0000 7f39caffd700 1 -- 172.21.15.188:0/775786833 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39cc0092e0 con 0x7f39d4104060 2024-06-10T02:45:58.413 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:45:58.412+0000 7f39dc060700 1 -- 172.21.15.188:0/775786833 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39d41a0650 con 0x7f39d4104060 2024-06-10T02:45:58.413 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:45:58.412+0000 7f39caffd700 1 -- 172.21.15.188:0/775786833 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f39cc004bb0 con 0x7f39d4104060 2024-06-10T02:45:58.413 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:45:58.412+0000 7f39caffd700 1 -- 172.21.15.188:0/775786833 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39cc004280 con 0x7f39d4104060 2024-06-10T02:45:58.413 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:45:58.412+0000 7f39dc060700 1 -- 172.21.15.188:0/775786833 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39d41a0bc0 con 0x7f39d4104060 2024-06-10T02:45:58.414 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:45:58.413+0000 7f39dc060700 1 -- 172.21.15.188:0/775786833 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39d4069180 con 0x7f39d4104060 2024-06-10T02:45:58.414 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:45:58.414+0000 7f39caffd700 1 -- 172.21.15.188:0/775786833 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f39cc01b030 con 0x7f39d4104060 2024-06-10T02:45:58.414 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:45:58.415+0000 7f39caffd700 1 --2- 172.21.15.188:0/775786833 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f39c00a2c20 0x7f39c00a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:45:58.414 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:45:58.415+0000 7f39caffd700 1 -- 172.21.15.188:0/775786833 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 4101+0+0 (secure 0 0 0) 0x7f39cc0ce4d0 con 0x7f39d4104060 2024-06-10T02:45:58.414 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:45:58.416+0000 7f39d9dfc700 1 --2- 172.21.15.188:0/775786833 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f39c00a2c20 0x7f39c00a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:45:58.414 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:45:58.416+0000 7f39d9dfc700 1 --2- 172.21.15.188:0/775786833 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f39c00a2c20 0x7f39c00a50e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f39d0006010 tx=0x7f39d0005c60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:45:58.417 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:45:58.419+0000 7f39caffd700 1 -- 172.21.15.188:0/775786833 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f39cc022020 con 0x7f39d4104060 2024-06-10T02:45:58.474 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:57 smithi052 ceph-mon[32362]: Detected new or changed devices on smithi120 2024-06-10T02:45:58.474 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:57 smithi052 ceph-mon[32362]: Adjusting osd_memory_target on smithi120 to 8544M 2024-06-10T02:45:58.474 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:57 smithi052 ceph-mon[32362]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-06-10T02:45:58.474 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:57 smithi052 ceph-mon[32362]: osdmap e35: 5 total, 5 up, 5 in 2024-06-10T02:45:58.619 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:45:58.621+0000 7f39dc060700 1 -- 172.21.15.188:0/775786833 --> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi188:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f39d40628d0 con 0x7f39c00a2c20 2024-06-10T02:45:59.386 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:58 smithi120 ceph-mon[36634]: from='client.24244 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi188:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T02:45:59.386 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:58 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-10T02:45:59.386 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:58 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-10T02:45:59.386 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:58 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:45:59.451 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:58 smithi188 ceph-mon[36599]: from='client.24244 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi188:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T02:45:59.451 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:58 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-10T02:45:59.451 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:58 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-10T02:45:59.451 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:58 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:45:59.474 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:58 smithi052 ceph-mon[32362]: from='client.24244 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi188:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T02:45:59.474 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:58 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-10T02:45:59.474 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:58 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-10T02:45:59.474 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:58 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:46:00.354 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:45:59 smithi188 ceph-mon[36599]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-06-10T02:46:00.385 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:45:59 smithi120 ceph-mon[36634]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-06-10T02:46:00.474 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:45:59 smithi052 ceph-mon[32362]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-06-10T02:46:02.201 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:46:01 smithi188 ceph-mon[36599]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-06-10T02:46:02.201 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:46:01 smithi188 ceph-mon[36599]: from='client.? 172.21.15.188:0/4229999181' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "001b5706-7f27-4fa4-9b69-969e6d765abc"}]: dispatch 2024-06-10T02:46:02.201 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:46:01 smithi188 ceph-mon[36599]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "001b5706-7f27-4fa4-9b69-969e6d765abc"}]: dispatch 2024-06-10T02:46:02.201 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:46:01 smithi188 ceph-mon[36599]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "001b5706-7f27-4fa4-9b69-969e6d765abc"}]': finished 2024-06-10T02:46:02.201 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:46:01 smithi188 ceph-mon[36599]: osdmap e36: 6 total, 5 up, 6 in 2024-06-10T02:46:02.201 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:46:01 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-10T02:46:02.386 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:46:01 smithi120 ceph-mon[36634]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-06-10T02:46:02.386 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:46:01 smithi120 ceph-mon[36634]: from='client.? 172.21.15.188:0/4229999181' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "001b5706-7f27-4fa4-9b69-969e6d765abc"}]: dispatch 2024-06-10T02:46:02.386 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:46:01 smithi120 ceph-mon[36634]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "001b5706-7f27-4fa4-9b69-969e6d765abc"}]: dispatch 2024-06-10T02:46:02.386 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:46:01 smithi120 ceph-mon[36634]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "001b5706-7f27-4fa4-9b69-969e6d765abc"}]': finished 2024-06-10T02:46:02.386 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:46:01 smithi120 ceph-mon[36634]: osdmap e36: 6 total, 5 up, 6 in 2024-06-10T02:46:02.386 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:46:01 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-10T02:46:02.474 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:46:01 smithi052 ceph-mon[32362]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-06-10T02:46:02.474 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:46:01 smithi052 ceph-mon[32362]: from='client.? 172.21.15.188:0/4229999181' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "001b5706-7f27-4fa4-9b69-969e6d765abc"}]: dispatch 2024-06-10T02:46:02.474 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:46:01 smithi052 ceph-mon[32362]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "001b5706-7f27-4fa4-9b69-969e6d765abc"}]: dispatch 2024-06-10T02:46:02.474 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:46:01 smithi052 ceph-mon[32362]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "001b5706-7f27-4fa4-9b69-969e6d765abc"}]': finished 2024-06-10T02:46:02.474 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:46:01 smithi052 ceph-mon[32362]: osdmap e36: 6 total, 5 up, 6 in 2024-06-10T02:46:02.474 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:46:01 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-10T02:46:03.386 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:46:02 smithi120 ceph-mon[36634]: from='client.? 172.21.15.188:0/1424935296' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-10T02:46:03.451 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:46:03 smithi188 ceph-mon[36599]: from='client.? 172.21.15.188:0/1424935296' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-10T02:46:03.474 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:46:02 smithi052 ceph-mon[32362]: from='client.? 172.21.15.188:0/1424935296' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-10T02:46:04.375 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:46:03 smithi052 ceph-mon[32362]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-06-10T02:46:04.385 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:46:03 smithi120 ceph-mon[36634]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-06-10T02:46:04.451 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:46:04 smithi188 ceph-mon[36599]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-06-10T02:46:06.201 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:46:06 smithi188 ceph-mon[36599]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-06-10T02:46:06.386 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:46:06 smithi120 ceph-mon[36634]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-06-10T02:46:06.474 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:46:06 smithi052 ceph-mon[32362]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-06-10T02:46:08.115 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:46:08 smithi188 ceph-mon[36599]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-06-10T02:46:08.386 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:46:08 smithi120 ceph-mon[36634]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-06-10T02:46:08.474 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:46:08 smithi052 ceph-mon[32362]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-06-10T02:46:09.201 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:46:09 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-06-10T02:46:09.201 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:46:09 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:46:09.386 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:46:09 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-06-10T02:46:09.386 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:46:09 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:46:09.474 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:46:09 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-06-10T02:46:09.474 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:46:09 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:46:10.009 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:46:10 smithi188 ceph-mon[36599]: Deploying daemon osd.5 on smithi188 2024-06-10T02:46:10.009 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:46:10 smithi188 ceph-mon[36599]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-06-10T02:46:10.386 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:46:10 smithi120 ceph-mon[36634]: Deploying daemon osd.5 on smithi188 2024-06-10T02:46:10.386 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:46:10 smithi120 ceph-mon[36634]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-06-10T02:46:10.474 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:46:10 smithi052 ceph-mon[32362]: Deploying daemon osd.5 on smithi188 2024-06-10T02:46:10.474 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:46:10 smithi052 ceph-mon[32362]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-06-10T02:46:12.201 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:46:12 smithi188 ceph-mon[36599]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-06-10T02:46:12.386 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:46:12 smithi120 ceph-mon[36634]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-06-10T02:46:12.474 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:46:12 smithi052 ceph-mon[32362]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-06-10T02:46:14.304 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:46:14 smithi188 ceph-mon[36599]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-06-10T02:46:14.386 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:46:14 smithi120 ceph-mon[36634]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-06-10T02:46:14.474 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:46:14 smithi052 ceph-mon[32362]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-06-10T02:46:15.886 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:46:15 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:46:15.886 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:46:15 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:46:15.886 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:46:15 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T02:46:15.951 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:46:15 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:46:15.951 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:46:15 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:46:15.951 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:46:15 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T02:46:15.974 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:46:15 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:46:15.975 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:46:15 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:46:15.975 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:46:15 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T02:46:16.701 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:46:16 smithi188 ceph-mon[36599]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-06-10T02:46:16.886 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:46:16 smithi120 ceph-mon[36634]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-06-10T02:46:16.974 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:46:16 smithi052 ceph-mon[32362]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-06-10T02:46:18.344 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:46:18.346+0000 7f39caffd700 1 -- 172.21.15.188:0/775786833 <== mgr.14150 v2:172.21.15.52:6800/3214763969 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f39d40628d0 con 0x7f39c00a2c20 2024-06-10T02:46:18.344 INFO:teuthology.orchestra.run.smithi188.stdout:Created osd(s) 5 on host 'smithi188' 2024-06-10T02:46:18.346 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:46:18.348+0000 7f39dc060700 1 -- 172.21.15.188:0/775786833 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f39c00a2c20 msgr2=0x7f39c00a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:46:18.346 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:46:18.348+0000 7f39dc060700 1 --2- 172.21.15.188:0/775786833 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f39c00a2c20 0x7f39c00a50e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f39d0006010 tx=0x7f39d0005c60 comp rx=0 tx=0).stop 2024-06-10T02:46:18.346 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:46:18.348+0000 7f39dc060700 1 -- 172.21.15.188:0/775786833 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f39d4104060 msgr2=0x7f39d419fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:46:18.346 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:46:18.348+0000 7f39dc060700 1 --2- 172.21.15.188:0/775786833 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f39d4104060 0x7f39d419fd20 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f39cc00ef60 tx=0x7f39cc00ef90 comp rx=0 tx=0).stop 2024-06-10T02:46:18.346 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:46:18.348+0000 7f39dc060700 1 -- 172.21.15.188:0/775786833 shutdown_connections 2024-06-10T02:46:18.347 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:46:18.348+0000 7f39dc060700 1 --2- 172.21.15.188:0/775786833 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f39d4104060 0x7f39d419fd20 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:46:18.347 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:46:18.348+0000 7f39dc060700 1 --2- 172.21.15.188:0/775786833 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f39d41024c0 0x7f39d4198760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:46:18.347 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:46:18.348+0000 7f39dc060700 1 --2- 172.21.15.188:0/775786833 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f39c00a2c20 0x7f39c00a50e0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:46:18.347 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:46:18.348+0000 7f39dc060700 1 --2- 172.21.15.188:0/775786833 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f39d41036c0 0x7f39d4198ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:46:18.347 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:46:18.348+0000 7f39dc060700 1 -- 172.21.15.188:0/775786833 >> 172.21.15.188:0/775786833 conn(0x7f39d40fda60 msgr2=0x7f39d4106a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:46:18.347 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:46:18.348+0000 7f39dc060700 1 -- 172.21.15.188:0/775786833 shutdown_connections 2024-06-10T02:46:18.347 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:46:18.349+0000 7f39dc060700 1 -- 172.21.15.188:0/775786833 wait complete. 2024-06-10T02:46:18.666 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:46:18 smithi188 ceph-mon[36599]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-06-10T02:46:18.667 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:46:18 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:46:18.667 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:46:18 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:46:18.667 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:46:18 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:46:18.667 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:46:18 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T02:46:18.667 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:46:18 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:46:18.667 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:46:18 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:46:18.667 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:46:18 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:46:18.667 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:46:18 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T02:46:18.724 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:46:18 smithi052 ceph-mon[32362]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-06-10T02:46:18.724 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:46:18 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:46:18.725 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:46:18 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:46:18.725 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:46:18 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:46:18.725 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:46:18 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T02:46:18.725 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:46:18 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:46:18.725 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:46:18 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:46:18.725 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:46:18 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:46:18.725 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:46:18 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T02:46:18.886 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:46:18 smithi120 ceph-mon[36634]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-06-10T02:46:18.886 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:46:18 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:46:18.886 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:46:18 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:46:18.886 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:46:18 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:46:18.887 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:46:18 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T02:46:18.887 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:46:18 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:46:18.887 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:46:18 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:46:18.887 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:46:18 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:46:18.887 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:46:18 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T02:46:19.026 DEBUG:teuthology.orchestra.run.smithi188:osd.5> sudo journalctl -f -n 0 -u ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968@osd.5.service 2024-06-10T02:46:19.029 INFO:tasks.cephadm:Deploying osd.6 on smithi052 with /dev/vg_nvme/lv_2... 2024-06-10T02:46:19.029 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-06-10T02:46:19.293 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:46:19.579 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:46:19 smithi188 ceph-mon[36599]: from='osd.5 [v2:172.21.15.188:6808/3951947483,v1:172.21.15.188:6809/3951947483]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-06-10T02:46:19.579 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:46:19 smithi188 ceph-mon[36599]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-06-10T02:46:19.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:46:19 smithi052 ceph-mon[32362]: from='osd.5 [v2:172.21.15.188:6808/3951947483,v1:172.21.15.188:6809/3951947483]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-06-10T02:46:19.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:46:19 smithi052 ceph-mon[32362]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-06-10T02:46:19.886 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:46:19 smithi120 ceph-mon[36634]: from='osd.5 [v2:172.21.15.188:6808/3951947483,v1:172.21.15.188:6809/3951947483]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-06-10T02:46:19.886 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:46:19 smithi120 ceph-mon[36634]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-06-10T02:46:20.426 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:46:20 smithi188 ceph-mon[36599]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-06-10T02:46:20.426 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:46:20 smithi188 ceph-mon[36599]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-06-10T02:46:20.426 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:46:20 smithi188 ceph-mon[36599]: osdmap e37: 6 total, 5 up, 6 in 2024-06-10T02:46:20.426 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:46:20 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-10T02:46:20.426 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:46:20 smithi188 ceph-mon[36599]: from='osd.5 [v2:172.21.15.188:6808/3951947483,v1:172.21.15.188:6809/3951947483]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]: dispatch 2024-06-10T02:46:20.427 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:46:20 smithi188 ceph-mon[36599]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]: dispatch 2024-06-10T02:46:20.427 INFO:journalctl@ceph.osd.5.smithi188.stdout:Jun 10 02:46:20 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-5[48629]: 2024-06-10T02:46:20.412+0000 7f5d46fb1700 -1 osd.5 0 waiting for initial osdmap 2024-06-10T02:46:20.427 INFO:journalctl@ceph.osd.5.smithi188.stdout:Jun 10 02:46:20 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-5[48629]: 2024-06-10T02:46:20.418+0000 7f5d42da4700 -1 osd.5 38 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-10T02:46:20.657 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:46:20 smithi052 ceph-mon[32362]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-06-10T02:46:20.658 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:46:20 smithi052 ceph-mon[32362]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-06-10T02:46:20.658 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:46:20 smithi052 ceph-mon[32362]: osdmap e37: 6 total, 5 up, 6 in 2024-06-10T02:46:20.658 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:46:20 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-10T02:46:20.658 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:46:20 smithi052 ceph-mon[32362]: from='osd.5 [v2:172.21.15.188:6808/3951947483,v1:172.21.15.188:6809/3951947483]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]: dispatch 2024-06-10T02:46:20.658 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:46:20 smithi052 ceph-mon[32362]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]: dispatch 2024-06-10T02:46:20.886 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:46:20 smithi120 ceph-mon[36634]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-06-10T02:46:20.886 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:46:20 smithi120 ceph-mon[36634]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-06-10T02:46:20.886 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:46:20 smithi120 ceph-mon[36634]: osdmap e37: 6 total, 5 up, 6 in 2024-06-10T02:46:20.886 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:46:20 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-10T02:46:20.886 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:46:20 smithi120 ceph-mon[36634]: from='osd.5 [v2:172.21.15.188:6808/3951947483,v1:172.21.15.188:6809/3951947483]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]: dispatch 2024-06-10T02:46:20.886 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:46:20 smithi120 ceph-mon[36634]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]: dispatch 2024-06-10T02:46:21.249 INFO:teuthology.orchestra.run.smithi052.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-06-10T02:46:21.249 INFO:teuthology.orchestra.run.smithi052.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-06-10T02:46:21.249 INFO:teuthology.orchestra.run.smithi052.stderr: stderr: 10+0 records in 2024-06-10T02:46:21.249 INFO:teuthology.orchestra.run.smithi052.stderr:10+0 records out 2024-06-10T02:46:21.249 INFO:teuthology.orchestra.run.smithi052.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00907545 s, 1.2 GB/s 2024-06-10T02:46:21.249 INFO:teuthology.orchestra.run.smithi052.stderr:--> Zapping successful for: 2024-06-10T02:46:21.564 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:46:21 smithi052 ceph-mon[32362]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]': finished 2024-06-10T02:46:21.564 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:46:21 smithi052 ceph-mon[32362]: osdmap e38: 6 total, 5 up, 6 in 2024-06-10T02:46:21.564 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:46:21 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-10T02:46:21.564 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:46:21 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-10T02:46:21.564 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:46:21 smithi052 ceph-mon[32362]: osd.5 [v2:172.21.15.188:6808/3951947483,v1:172.21.15.188:6809/3951947483] boot 2024-06-10T02:46:21.564 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:46:21 smithi052 ceph-mon[32362]: osdmap e39: 6 total, 6 up, 6 in 2024-06-10T02:46:21.564 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:46:21 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-10T02:46:21.615 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:46:21 smithi188 ceph-mon[36599]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]': finished 2024-06-10T02:46:21.616 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:46:21 smithi188 ceph-mon[36599]: osdmap e38: 6 total, 5 up, 6 in 2024-06-10T02:46:21.616 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:46:21 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-10T02:46:21.616 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:46:21 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-10T02:46:21.616 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:46:21 smithi188 ceph-mon[36599]: osd.5 [v2:172.21.15.188:6808/3951947483,v1:172.21.15.188:6809/3951947483] boot 2024-06-10T02:46:21.616 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:46:21 smithi188 ceph-mon[36599]: osdmap e39: 6 total, 6 up, 6 in 2024-06-10T02:46:21.616 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:46:21 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-10T02:46:21.886 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:46:21 smithi120 ceph-mon[36634]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]': finished 2024-06-10T02:46:21.886 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:46:21 smithi120 ceph-mon[36634]: osdmap e38: 6 total, 5 up, 6 in 2024-06-10T02:46:21.886 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:46:21 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-10T02:46:21.886 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:46:21 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-10T02:46:21.886 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:46:21 smithi120 ceph-mon[36634]: osd.5 [v2:172.21.15.188:6808/3951947483,v1:172.21.15.188:6809/3951947483] boot 2024-06-10T02:46:21.887 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:46:21 smithi120 ceph-mon[36634]: osdmap e39: 6 total, 6 up, 6 in 2024-06-10T02:46:21.887 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:46:21 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-10T02:46:21.934 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi052:vg_nvme/lv_2 2024-06-10T02:46:22.203 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:46:22.586 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:46:22 smithi052 ceph-mon[32362]: purged_snaps scrub starts 2024-06-10T02:46:22.586 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:46:22 smithi052 ceph-mon[32362]: purged_snaps scrub ok 2024-06-10T02:46:22.586 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:46:22 smithi052 ceph-mon[32362]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-06-10T02:46:22.586 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:46:22 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:46:22.586 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:46:22 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:46:22.586 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:46:22 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-10T02:46:22.587 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:46:22 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-10T02:46:22.587 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:46:22 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:46:22.587 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:46:22 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:46:22.587 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:46:22 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T02:46:22.587 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:46:22 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:46:22.587 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:46:22 smithi052 ceph-mon[32362]: osdmap e40: 6 total, 6 up, 6 in 2024-06-10T02:46:22.701 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:46:22 smithi188 ceph-mon[36599]: purged_snaps scrub starts 2024-06-10T02:46:22.701 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:46:22 smithi188 ceph-mon[36599]: purged_snaps scrub ok 2024-06-10T02:46:22.701 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:46:22 smithi188 ceph-mon[36599]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-06-10T02:46:22.701 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:46:22 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:46:22.701 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:46:22 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:46:22.701 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:46:22 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-10T02:46:22.701 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:46:22 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-10T02:46:22.701 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:46:22 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:46:22.702 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:46:22 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:46:22.702 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:46:22 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T02:46:22.702 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:46:22 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:46:22.702 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:46:22 smithi188 ceph-mon[36599]: osdmap e40: 6 total, 6 up, 6 in 2024-06-10T02:46:22.886 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:46:22 smithi120 ceph-mon[36634]: purged_snaps scrub starts 2024-06-10T02:46:22.886 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:46:22 smithi120 ceph-mon[36634]: purged_snaps scrub ok 2024-06-10T02:46:22.886 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:46:22 smithi120 ceph-mon[36634]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-06-10T02:46:22.886 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:46:22 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:46:22.886 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:46:22 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:46:22.886 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:46:22 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-10T02:46:22.886 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:46:22 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-10T02:46:22.886 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:46:22 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:46:22.886 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:46:22 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:46:22.887 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:46:22 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T02:46:22.887 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:46:22 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:46:22.887 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:46:22 smithi120 ceph-mon[36634]: osdmap e40: 6 total, 6 up, 6 in 2024-06-10T02:46:23.478 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:46:23 smithi188 ceph-mon[36599]: Detected new or changed devices on smithi188 2024-06-10T02:46:23.478 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:46:23 smithi188 ceph-mon[36599]: Adjusting osd_memory_target on smithi188 to 8544M 2024-06-10T02:46:23.479 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:46:23 smithi188 ceph-mon[36599]: osdmap e41: 6 total, 6 up, 6 in 2024-06-10T02:46:23.566 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:46:23 smithi052 ceph-mon[32362]: Detected new or changed devices on smithi188 2024-06-10T02:46:23.567 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:46:23 smithi052 ceph-mon[32362]: Adjusting osd_memory_target on smithi188 to 8544M 2024-06-10T02:46:23.567 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:46:23 smithi052 ceph-mon[32362]: osdmap e41: 6 total, 6 up, 6 in 2024-06-10T02:46:23.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:46:23.806+0000 7f7808ba2700 1 -- 172.21.15.52:0/2741338023 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7804104140 msgr2=0x7f780410a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:46:23.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:46:23.806+0000 7f7808ba2700 1 --2- 172.21.15.52:0/2741338023 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7804104140 0x7f780410a9f0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f77fc00b0a0 tx=0x7f77fc00b3b0 comp rx=0 tx=0).stop 2024-06-10T02:46:23.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:46:23.806+0000 7f7808ba2700 1 -- 172.21.15.52:0/2741338023 shutdown_connections 2024-06-10T02:46:23.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:46:23.806+0000 7f7808ba2700 1 --2- 172.21.15.52:0/2741338023 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7804104140 0x7f780410a9f0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:46:23.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:46:23.806+0000 7f7808ba2700 1 --2- 172.21.15.52:0/2741338023 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f78041007c0 0x7f7804100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:46:23.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:46:23.806+0000 7f7808ba2700 1 --2- 172.21.15.52:0/2741338023 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f78041037a0 0x7f7804103c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:46:23.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:46:23.806+0000 7f7808ba2700 1 -- 172.21.15.52:0/2741338023 >> 172.21.15.52:0/2741338023 conn(0x7f78040fc100 msgr2=0x7f78040fe560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:46:23.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:46:23.806+0000 7f7808ba2700 1 -- 172.21.15.52:0/2741338023 shutdown_connections 2024-06-10T02:46:23.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:46:23.806+0000 7f7808ba2700 1 -- 172.21.15.52:0/2741338023 wait complete. 2024-06-10T02:46:23.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:46:23.807+0000 7f7808ba2700 1 Processor -- start 2024-06-10T02:46:23.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:46:23.807+0000 7f7808ba2700 1 -- start start 2024-06-10T02:46:23.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:46:23.807+0000 7f7808ba2700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f78041007c0 0x7f7804103100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:46:23.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:46:23.807+0000 7f7808ba2700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f78041037a0 0x7f7804101750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:46:23.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:46:23.807+0000 7f7808ba2700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7804104140 0x7f7804101c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:46:23.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:46:23.807+0000 7f7808ba2700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f780410e0c0 con 0x7f7804104140 2024-06-10T02:46:23.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:46:23.807+0000 7f7808ba2700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f780410df40 con 0x7f78041007c0 2024-06-10T02:46:23.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:46:23.807+0000 7f7808ba2700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f780410e240 con 0x7f78041037a0 2024-06-10T02:46:23.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:46:23.807+0000 7f7802d9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7804104140 0x7f7804101c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:46:23.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:46:23.807+0000 7f7802d9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7804104140 0x7f7804101c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47258/0 (socket says 172.21.15.52:47258) 2024-06-10T02:46:23.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:46:23.807+0000 7f7802d9d700 1 -- 172.21.15.52:0/149395783 learned_addr learned my addr 172.21.15.52:0/149395783 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:46:23.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:46:23.807+0000 7f7801d9b700 1 --2- 172.21.15.52:0/149395783 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f78041037a0 0x7f7804101750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:46:23.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:46:23.807+0000 7f780259c700 1 --2- 172.21.15.52:0/149395783 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f78041007c0 0x7f7804103100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:46:23.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:46:23.807+0000 7f7802d9d700 1 -- 172.21.15.52:0/149395783 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f78041037a0 msgr2=0x7f7804101750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:46:23.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:46:23.807+0000 7f7802d9d700 1 --2- 172.21.15.52:0/149395783 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f78041037a0 0x7f7804101750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:46:23.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:46:23.807+0000 7f7802d9d700 1 -- 172.21.15.52:0/149395783 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f78041007c0 msgr2=0x7f7804103100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:46:23.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:46:23.807+0000 7f7802d9d700 1 --2- 172.21.15.52:0/149395783 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f78041007c0 0x7f7804103100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:46:23.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:46:23.807+0000 7f7802d9d700 1 -- 172.21.15.52:0/149395783 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7804102540 con 0x7f7804104140 2024-06-10T02:46:23.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:46:23.808+0000 7f7802d9d700 1 --2- 172.21.15.52:0/149395783 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7804104140 0x7f7804101c90 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f77fc007f00 tx=0x7f77fc004dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:46:23.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:46:23.808+0000 7f77fb7fe700 1 -- 172.21.15.52:0/149395783 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77fc021070 con 0x7f7804104140 2024-06-10T02:46:23.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:46:23.808+0000 7f7808ba2700 1 -- 172.21.15.52:0/149395783 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78041a27a0 con 0x7f7804104140 2024-06-10T02:46:23.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:46:23.808+0000 7f77fb7fe700 1 -- 172.21.15.52:0/149395783 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f77fc007b80 con 0x7f7804104140 2024-06-10T02:46:23.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:46:23.808+0000 7f77fb7fe700 1 -- 172.21.15.52:0/149395783 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77fc01d450 con 0x7f7804104140 2024-06-10T02:46:23.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:46:23.808+0000 7f7808ba2700 1 -- 172.21.15.52:0/149395783 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78041a2c50 con 0x7f7804104140 2024-06-10T02:46:23.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:46:23.809+0000 7f7808ba2700 1 -- 172.21.15.52:0/149395783 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7804069180 con 0x7f7804104140 2024-06-10T02:46:23.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:46:23.810+0000 7f77fb7fe700 1 -- 172.21.15.52:0/149395783 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f77fc004120 con 0x7f7804104140 2024-06-10T02:46:23.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:46:23.811+0000 7f77fb7fe700 1 --2- 172.21.15.52:0/149395783 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f77f00a2c20 0x7f77f00a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:46:23.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:46:23.811+0000 7f780259c700 1 --2- 172.21.15.52:0/149395783 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f77f00a2c20 0x7f77f00a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:46:23.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:46:23.811+0000 7f77fb7fe700 1 -- 172.21.15.52:0/149395783 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 4562+0+0 (secure 0 0 0) 0x7f77fc00f020 con 0x7f7804104140 2024-06-10T02:46:23.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:46:23.811+0000 7f780259c700 1 --2- 172.21.15.52:0/149395783 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f77f00a2c20 0x7f77f00a50e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f780406aa40 tx=0x7f77f400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:46:23.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:46:23.815+0000 7f77fb7fe700 1 -- 172.21.15.52:0/149395783 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f77fc02a020 con 0x7f7804104140 2024-06-10T02:46:23.885 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:46:23 smithi120 ceph-mon[36634]: Detected new or changed devices on smithi188 2024-06-10T02:46:23.886 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:46:23 smithi120 ceph-mon[36634]: Adjusting osd_memory_target on smithi188 to 8544M 2024-06-10T02:46:23.886 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:46:23 smithi120 ceph-mon[36634]: osdmap e41: 6 total, 6 up, 6 in 2024-06-10T02:46:24.030 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:46:24.033+0000 7f7808ba2700 1 -- 172.21.15.52:0/149395783 --> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi052:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f78040628d0 con 0x7f77f00a2c20 2024-06-10T02:46:24.701 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:46:24 smithi188 ceph-mon[36599]: pgmap v140: 1 pgs: 1 remapped+peering; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-06-10T02:46:24.701 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:46:24 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-10T02:46:24.701 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:46:24 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-10T02:46:24.701 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:46:24 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:46:24.724 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:46:24 smithi052 ceph-mon[32362]: pgmap v140: 1 pgs: 1 remapped+peering; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-06-10T02:46:24.725 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:46:24 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-10T02:46:24.725 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:46:24 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-10T02:46:24.725 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:46:24 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:46:24.787 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:46:24 smithi120 ceph-mon[36634]: pgmap v140: 1 pgs: 1 remapped+peering; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-06-10T02:46:24.787 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:46:24 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-10T02:46:24.787 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:46:24 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-10T02:46:24.787 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:46:24 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:46:25.701 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:46:25 smithi188 ceph-mon[36599]: from='client.14343 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi052:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T02:46:25.701 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:46:25 smithi188 ceph-mon[36599]: osdmap e42: 6 total, 6 up, 6 in 2024-06-10T02:46:25.724 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:46:25 smithi052 ceph-mon[32362]: from='client.14343 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi052:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T02:46:25.724 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:46:25 smithi052 ceph-mon[32362]: osdmap e42: 6 total, 6 up, 6 in 2024-06-10T02:46:25.886 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:46:25 smithi120 ceph-mon[36634]: from='client.14343 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi052:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T02:46:25.886 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:46:25 smithi120 ceph-mon[36634]: osdmap e42: 6 total, 6 up, 6 in 2024-06-10T02:46:26.701 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:46:26 smithi188 ceph-mon[36599]: pgmap v143: 1 pgs: 1 remapped+peering; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-06-10T02:46:26.701 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:46:26 smithi188 ceph-mon[36599]: osdmap e43: 6 total, 6 up, 6 in 2024-06-10T02:46:26.724 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:46:26 smithi052 ceph-mon[32362]: pgmap v143: 1 pgs: 1 remapped+peering; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-06-10T02:46:26.725 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:46:26 smithi052 ceph-mon[32362]: osdmap e43: 6 total, 6 up, 6 in 2024-06-10T02:46:26.887 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:46:26 smithi120 ceph-mon[36634]: pgmap v143: 1 pgs: 1 remapped+peering; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-06-10T02:46:26.887 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:46:26 smithi120 ceph-mon[36634]: osdmap e43: 6 total, 6 up, 6 in 2024-06-10T02:46:27.701 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:46:27 smithi188 ceph-mon[36599]: from='client.? 172.21.15.52:0/1165264715' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "140c903e-9b31-4a81-9ccd-2edc0a2decf9"}]: dispatch 2024-06-10T02:46:27.701 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:46:27 smithi188 ceph-mon[36599]: from='client.? 172.21.15.52:0/1165264715' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "140c903e-9b31-4a81-9ccd-2edc0a2decf9"}]': finished 2024-06-10T02:46:27.701 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:46:27 smithi188 ceph-mon[36599]: osdmap e44: 7 total, 6 up, 7 in 2024-06-10T02:46:27.701 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:46:27 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-10T02:46:27.724 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:46:27 smithi052 ceph-mon[32362]: from='client.? 172.21.15.52:0/1165264715' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "140c903e-9b31-4a81-9ccd-2edc0a2decf9"}]: dispatch 2024-06-10T02:46:27.725 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:46:27 smithi052 ceph-mon[32362]: from='client.? 172.21.15.52:0/1165264715' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "140c903e-9b31-4a81-9ccd-2edc0a2decf9"}]': finished 2024-06-10T02:46:27.725 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:46:27 smithi052 ceph-mon[32362]: osdmap e44: 7 total, 6 up, 7 in 2024-06-10T02:46:27.725 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:46:27 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-10T02:46:27.886 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:46:27 smithi120 ceph-mon[36634]: from='client.? 172.21.15.52:0/1165264715' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "140c903e-9b31-4a81-9ccd-2edc0a2decf9"}]: dispatch 2024-06-10T02:46:27.886 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:46:27 smithi120 ceph-mon[36634]: from='client.? 172.21.15.52:0/1165264715' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "140c903e-9b31-4a81-9ccd-2edc0a2decf9"}]': finished 2024-06-10T02:46:27.886 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:46:27 smithi120 ceph-mon[36634]: osdmap e44: 7 total, 6 up, 7 in 2024-06-10T02:46:27.886 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:46:27 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-10T02:46:28.701 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:46:28 smithi188 ceph-mon[36599]: pgmap v145: 1 pgs: 1 remapped+peering; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-06-10T02:46:28.701 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:46:28 smithi188 ceph-mon[36599]: from='client.? 172.21.15.52:0/3484741513' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-10T02:46:28.724 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:46:28 smithi052 ceph-mon[32362]: pgmap v145: 1 pgs: 1 remapped+peering; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-06-10T02:46:28.724 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:46:28 smithi052 ceph-mon[32362]: from='client.? 172.21.15.52:0/3484741513' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-10T02:46:28.886 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:46:28 smithi120 ceph-mon[36634]: pgmap v145: 1 pgs: 1 remapped+peering; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-06-10T02:46:28.886 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:46:28 smithi120 ceph-mon[36634]: from='client.? 172.21.15.52:0/3484741513' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-10T02:46:30.701 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:46:30 smithi188 ceph-mon[36599]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 105 KiB/s, 0 objects/s recovering 2024-06-10T02:46:30.724 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:46:30 smithi052 ceph-mon[32362]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 105 KiB/s, 0 objects/s recovering 2024-06-10T02:46:30.886 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:46:30 smithi120 ceph-mon[36634]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 105 KiB/s, 0 objects/s recovering 2024-06-10T02:46:32.643 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:46:32 smithi052 ceph-mon[32362]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 89 KiB/s, 0 objects/s recovering 2024-06-10T02:46:32.701 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:46:32 smithi188 ceph-mon[36599]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 89 KiB/s, 0 objects/s recovering 2024-06-10T02:46:32.886 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:46:32 smithi120 ceph-mon[36634]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 89 KiB/s, 0 objects/s recovering 2024-06-10T02:46:34.701 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:46:34 smithi188 ceph-mon[36599]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-06-10T02:46:34.724 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:46:34 smithi052 ceph-mon[32362]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-06-10T02:46:34.886 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:46:34 smithi120 ceph-mon[36634]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-06-10T02:46:35.710 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:46:35 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-06-10T02:46:35.711 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:46:35 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:46:35.711 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:46:35 smithi052 ceph-mon[32362]: Deploying daemon osd.6 on smithi052 2024-06-10T02:46:35.886 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:46:35 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-06-10T02:46:35.886 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:46:35 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:46:35.886 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:46:35 smithi120 ceph-mon[36634]: Deploying daemon osd.6 on smithi052 2024-06-10T02:46:35.951 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:46:35 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-06-10T02:46:35.951 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:46:35 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:46:35.951 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:46:35 smithi188 ceph-mon[36599]: Deploying daemon osd.6 on smithi052 2024-06-10T02:46:36.724 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:46:36 smithi052 ceph-mon[32362]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-06-10T02:46:36.886 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:46:36 smithi120 ceph-mon[36634]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-06-10T02:46:36.951 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:46:36 smithi188 ceph-mon[36599]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-06-10T02:46:38.725 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:46:38 smithi052 ceph-mon[32362]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-06-10T02:46:38.886 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:46:38 smithi120 ceph-mon[36634]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-06-10T02:46:38.951 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:46:38 smithi188 ceph-mon[36599]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-06-10T02:46:40.724 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:46:40 smithi052 ceph-mon[32362]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-06-10T02:46:40.886 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:46:40 smithi120 ceph-mon[36634]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-06-10T02:46:40.951 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:46:40 smithi188 ceph-mon[36599]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-06-10T02:46:41.886 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:46:41 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:46:41.886 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:46:41 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:46:41.886 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:46:41 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T02:46:41.951 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:46:41 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:46:41.951 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:46:41 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:46:41.951 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:46:41 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T02:46:41.977 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:46:41 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:46:41.977 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:46:41 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:46:41.977 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:46:41 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T02:46:42.886 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:46:42 smithi120 ceph-mon[36634]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-06-10T02:46:42.951 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:46:42 smithi188 ceph-mon[36599]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-06-10T02:46:42.974 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:46:42 smithi052 ceph-mon[32362]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-06-10T02:46:44.706 INFO:teuthology.orchestra.run.smithi052.stdout:Created osd(s) 6 on host 'smithi052' 2024-06-10T02:46:44.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:46:44.707+0000 7f77fb7fe700 1 -- 172.21.15.52:0/149395783 <== mgr.14150 v2:172.21.15.52:6800/3214763969 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f78040628d0 con 0x7f77f00a2c20 2024-06-10T02:46:44.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:46:44.710+0000 7f7808ba2700 1 -- 172.21.15.52:0/149395783 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f77f00a2c20 msgr2=0x7f77f00a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:46:44.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:46:44.710+0000 7f7808ba2700 1 --2- 172.21.15.52:0/149395783 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f77f00a2c20 0x7f77f00a50e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f780406aa40 tx=0x7f77f400b410 comp rx=0 tx=0).stop 2024-06-10T02:46:44.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:46:44.710+0000 7f7808ba2700 1 -- 172.21.15.52:0/149395783 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7804104140 msgr2=0x7f7804101c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:46:44.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:46:44.710+0000 7f7808ba2700 1 --2- 172.21.15.52:0/149395783 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7804104140 0x7f7804101c90 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f77fc007f00 tx=0x7f77fc004dd0 comp rx=0 tx=0).stop 2024-06-10T02:46:44.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:46:44.710+0000 7f7808ba2700 1 -- 172.21.15.52:0/149395783 shutdown_connections 2024-06-10T02:46:44.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:46:44.710+0000 7f7808ba2700 1 --2- 172.21.15.52:0/149395783 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f77f00a2c20 0x7f77f00a50e0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:46:44.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:46:44.710+0000 7f7808ba2700 1 --2- 172.21.15.52:0/149395783 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7804104140 0x7f7804101c90 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:46:44.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:46:44.710+0000 7f7808ba2700 1 --2- 172.21.15.52:0/149395783 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f78041007c0 0x7f7804103100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:46:44.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:46:44.710+0000 7f7808ba2700 1 --2- 172.21.15.52:0/149395783 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f78041037a0 0x7f7804101750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:46:44.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:46:44.710+0000 7f7808ba2700 1 -- 172.21.15.52:0/149395783 >> 172.21.15.52:0/149395783 conn(0x7f78040fc100 msgr2=0x7f7804106a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:46:44.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:46:44.711+0000 7f7808ba2700 1 -- 172.21.15.52:0/149395783 shutdown_connections 2024-06-10T02:46:44.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:46:44.711+0000 7f7808ba2700 1 -- 172.21.15.52:0/149395783 wait complete. 2024-06-10T02:46:44.724 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:46:44 smithi052 ceph-mon[32362]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-06-10T02:46:44.724 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:46:44 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:46:44.724 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:46:44 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:46:44.725 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:46:44 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:46:44.725 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:46:44 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T02:46:44.725 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:46:44 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:46:44.725 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:46:44 smithi052 ceph-mon[32362]: from='osd.6 [v2:172.21.15.52:6818/359084919,v1:172.21.15.52:6819/359084919]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-06-10T02:46:44.951 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:46:44 smithi188 ceph-mon[36599]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-06-10T02:46:44.951 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:46:44 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:46:44.951 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:46:44 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:46:44.951 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:46:44 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:46:44.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:46:44 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T02:46:44.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:46:44 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:46:44.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:46:44 smithi188 ceph-mon[36599]: from='osd.6 [v2:172.21.15.52:6818/359084919,v1:172.21.15.52:6819/359084919]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-06-10T02:46:45.136 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:46:44 smithi120 ceph-mon[36634]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-06-10T02:46:45.136 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:46:44 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:46:45.136 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:46:44 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:46:45.136 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:46:44 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:46:45.136 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:46:44 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T02:46:45.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:46:44 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:46:45.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:46:44 smithi120 ceph-mon[36634]: from='osd.6 [v2:172.21.15.52:6818/359084919,v1:172.21.15.52:6819/359084919]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-06-10T02:46:45.427 DEBUG:teuthology.orchestra.run.smithi052:osd.6> sudo journalctl -f -n 0 -u ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968@osd.6.service 2024-06-10T02:46:45.430 INFO:tasks.cephadm:Deploying osd.7 on smithi120 with /dev/vg_nvme/lv_2... 2024-06-10T02:46:45.430 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-06-10T02:46:45.659 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:46:45 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:46:45.659 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:46:45 smithi052 ceph-mon[32362]: from='osd.6 [v2:172.21.15.52:6818/359084919,v1:172.21.15.52:6819/359084919]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-06-10T02:46:45.659 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:46:45 smithi052 ceph-mon[32362]: osdmap e45: 7 total, 6 up, 7 in 2024-06-10T02:46:45.659 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:46:45 smithi052 ceph-mon[32362]: from='osd.6 [v2:172.21.15.52:6818/359084919,v1:172.21.15.52:6819/359084919]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]: dispatch 2024-06-10T02:46:45.659 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:46:45 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-10T02:46:45.659 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:46:45 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:46:45.660 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:46:45 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T02:46:45.677 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.b/config 2024-06-10T02:46:45.799 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:46:45 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:46:45.799 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:46:45 smithi120 ceph-mon[36634]: from='osd.6 [v2:172.21.15.52:6818/359084919,v1:172.21.15.52:6819/359084919]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-06-10T02:46:45.799 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:46:45 smithi120 ceph-mon[36634]: osdmap e45: 7 total, 6 up, 7 in 2024-06-10T02:46:45.799 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:46:45 smithi120 ceph-mon[36634]: from='osd.6 [v2:172.21.15.52:6818/359084919,v1:172.21.15.52:6819/359084919]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]: dispatch 2024-06-10T02:46:45.799 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:46:45 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-10T02:46:45.799 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:46:45 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:46:45.799 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:46:45 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T02:46:45.951 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:46:45 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:46:45.951 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:46:45 smithi188 ceph-mon[36599]: from='osd.6 [v2:172.21.15.52:6818/359084919,v1:172.21.15.52:6819/359084919]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-06-10T02:46:45.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:46:45 smithi188 ceph-mon[36599]: osdmap e45: 7 total, 6 up, 7 in 2024-06-10T02:46:45.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:46:45 smithi188 ceph-mon[36599]: from='osd.6 [v2:172.21.15.52:6818/359084919,v1:172.21.15.52:6819/359084919]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]: dispatch 2024-06-10T02:46:45.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:46:45 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-10T02:46:45.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:46:45 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:46:45.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:46:45 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T02:46:46.225 INFO:journalctl@ceph.osd.6.smithi052.stdout:Jun 10 02:46:46 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-6[60649]: 2024-06-10T02:46:46.134+0000 7f4ceae1a700 -1 osd.6 0 waiting for initial osdmap 2024-06-10T02:46:46.225 INFO:journalctl@ceph.osd.6.smithi052.stdout:Jun 10 02:46:46 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-6[60649]: 2024-06-10T02:46:46.145+0000 7f4ce2404700 -1 osd.6 46 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-10T02:46:46.903 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:46:46 smithi120 ceph-mon[36634]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-06-10T02:46:46.903 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:46:46 smithi120 ceph-mon[36634]: from='osd.6 [v2:172.21.15.52:6818/359084919,v1:172.21.15.52:6819/359084919]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]': finished 2024-06-10T02:46:46.903 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:46:46 smithi120 ceph-mon[36634]: osdmap e46: 7 total, 6 up, 7 in 2024-06-10T02:46:46.903 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:46:46 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-10T02:46:46.903 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:46:46 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-10T02:46:46.903 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:46:46 smithi120 ceph-mon[36634]: from='osd.6 [v2:172.21.15.52:6818/359084919,v1:172.21.15.52:6819/359084919]' entity='osd.6' 2024-06-10T02:46:46.909 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:46:46 smithi052 ceph-mon[32362]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-06-10T02:46:46.909 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:46:46 smithi052 ceph-mon[32362]: from='osd.6 [v2:172.21.15.52:6818/359084919,v1:172.21.15.52:6819/359084919]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]': finished 2024-06-10T02:46:46.909 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:46:46 smithi052 ceph-mon[32362]: osdmap e46: 7 total, 6 up, 7 in 2024-06-10T02:46:46.910 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:46:46 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-10T02:46:46.910 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:46:46 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-10T02:46:46.910 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:46:46 smithi052 ceph-mon[32362]: from='osd.6 [v2:172.21.15.52:6818/359084919,v1:172.21.15.52:6819/359084919]' entity='osd.6' 2024-06-10T02:46:46.951 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:46:46 smithi188 ceph-mon[36599]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-06-10T02:46:46.951 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:46:46 smithi188 ceph-mon[36599]: from='osd.6 [v2:172.21.15.52:6818/359084919,v1:172.21.15.52:6819/359084919]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]': finished 2024-06-10T02:46:46.951 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:46:46 smithi188 ceph-mon[36599]: osdmap e46: 7 total, 6 up, 7 in 2024-06-10T02:46:46.951 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:46:46 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-10T02:46:46.951 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:46:46 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-10T02:46:46.951 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:46:46 smithi188 ceph-mon[36599]: from='osd.6 [v2:172.21.15.52:6818/359084919,v1:172.21.15.52:6819/359084919]' entity='osd.6' 2024-06-10T02:46:47.818 INFO:teuthology.orchestra.run.smithi120.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-06-10T02:46:47.818 INFO:teuthology.orchestra.run.smithi120.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-06-10T02:46:47.818 INFO:teuthology.orchestra.run.smithi120.stderr: stderr: 10+0 records in 2024-06-10T02:46:47.818 INFO:teuthology.orchestra.run.smithi120.stderr:10+0 records out 2024-06-10T02:46:47.818 INFO:teuthology.orchestra.run.smithi120.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0116142 s, 903 MB/s 2024-06-10T02:46:47.818 INFO:teuthology.orchestra.run.smithi120.stderr:--> Zapping successful for: 2024-06-10T02:46:47.919 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:46:47 smithi052 ceph-mon[32362]: purged_snaps scrub starts 2024-06-10T02:46:47.919 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:46:47 smithi052 ceph-mon[32362]: purged_snaps scrub ok 2024-06-10T02:46:47.919 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:46:47 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-10T02:46:47.919 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:46:47 smithi052 ceph-mon[32362]: osd.6 [v2:172.21.15.52:6818/359084919,v1:172.21.15.52:6819/359084919] boot 2024-06-10T02:46:47.919 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:46:47 smithi052 ceph-mon[32362]: osdmap e47: 7 total, 7 up, 7 in 2024-06-10T02:46:47.920 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:46:47 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-10T02:46:47.951 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:46:47 smithi188 ceph-mon[36599]: purged_snaps scrub starts 2024-06-10T02:46:47.951 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:46:47 smithi188 ceph-mon[36599]: purged_snaps scrub ok 2024-06-10T02:46:47.951 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:46:47 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-10T02:46:47.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:46:47 smithi188 ceph-mon[36599]: osd.6 [v2:172.21.15.52:6818/359084919,v1:172.21.15.52:6819/359084919] boot 2024-06-10T02:46:47.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:46:47 smithi188 ceph-mon[36599]: osdmap e47: 7 total, 7 up, 7 in 2024-06-10T02:46:47.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:46:47 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-10T02:46:48.124 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:46:47 smithi120 ceph-mon[36634]: purged_snaps scrub starts 2024-06-10T02:46:48.125 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:46:47 smithi120 ceph-mon[36634]: purged_snaps scrub ok 2024-06-10T02:46:48.125 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:46:47 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-10T02:46:48.125 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:46:47 smithi120 ceph-mon[36634]: osd.6 [v2:172.21.15.52:6818/359084919,v1:172.21.15.52:6819/359084919] boot 2024-06-10T02:46:48.125 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:46:47 smithi120 ceph-mon[36634]: osdmap e47: 7 total, 7 up, 7 in 2024-06-10T02:46:48.125 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:46:47 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-10T02:46:48.471 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi120:vg_nvme/lv_2 2024-06-10T02:46:48.716 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.b/config 2024-06-10T02:46:48.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:46:48 smithi120 ceph-mon[36634]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-06-10T02:46:48.754 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:46:48 smithi120 ceph-mon[36634]: osdmap e48: 7 total, 7 up, 7 in 2024-06-10T02:46:48.951 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:46:48 smithi188 ceph-mon[36599]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-06-10T02:46:48.951 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:46:48 smithi188 ceph-mon[36599]: osdmap e48: 7 total, 7 up, 7 in 2024-06-10T02:46:48.974 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:46:48 smithi052 ceph-mon[32362]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-06-10T02:46:48.974 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:46:48 smithi052 ceph-mon[32362]: osdmap e48: 7 total, 7 up, 7 in 2024-06-10T02:46:49.759 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:46:49 smithi120 ceph-mon[36634]: Detected new or changed devices on smithi052 2024-06-10T02:46:49.759 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:46:49 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:46:49.759 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:46:49 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:46:49.759 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:46:49 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-10T02:46:49.759 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:46:49 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-10T02:46:49.759 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:46:49 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-10T02:46:49.759 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:46:49 smithi120 ceph-mon[36634]: Adjusting osd_memory_target on smithi052 to 5697M 2024-06-10T02:46:49.759 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:46:49 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:46:49.760 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:46:49 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:46:49.760 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:46:49 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T02:46:49.760 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:46:49 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:46:49.951 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:46:49 smithi188 ceph-mon[36599]: Detected new or changed devices on smithi052 2024-06-10T02:46:49.951 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:46:49 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:46:49.951 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:46:49 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:46:49.951 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:46:49 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-10T02:46:49.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:46:49 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-10T02:46:49.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:46:49 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-10T02:46:49.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:46:49 smithi188 ceph-mon[36599]: Adjusting osd_memory_target on smithi052 to 5697M 2024-06-10T02:46:49.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:46:49 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:46:49.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:46:49 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:46:49.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:46:49 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T02:46:49.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:46:49 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:46:49.975 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:46:49 smithi052 ceph-mon[32362]: Detected new or changed devices on smithi052 2024-06-10T02:46:49.975 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:46:49 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:46:49.975 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:46:49 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:46:49.975 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:46:49 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-10T02:46:49.975 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:46:49 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-10T02:46:49.975 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:46:49 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-10T02:46:49.975 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:46:49 smithi052 ceph-mon[32362]: Adjusting osd_memory_target on smithi052 to 5697M 2024-06-10T02:46:49.975 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:46:49 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:46:49.975 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:46:49 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:46:49.975 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:46:49 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T02:46:49.975 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:46:49 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:46:50.293 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:46:50.294+0000 7f708200b700 1 -- 172.21.15.120:0/1512297762 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f707c102510 msgr2=0x7f707c102930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:46:50.293 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:46:50.294+0000 7f708200b700 1 --2- 172.21.15.120:0/1512297762 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f707c102510 0x7f707c102930 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f706c0099c0 tx=0x7f706c009cd0 comp rx=0 tx=0).stop 2024-06-10T02:46:50.293 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:46:50.294+0000 7f708200b700 1 -- 172.21.15.120:0/1512297762 shutdown_connections 2024-06-10T02:46:50.293 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:46:50.294+0000 7f708200b700 1 --2- 172.21.15.120:0/1512297762 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f707c1040b0 0x7f707c10a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:46:50.293 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:46:50.294+0000 7f708200b700 1 --2- 172.21.15.120:0/1512297762 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f707c102510 0x7f707c102930 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:46:50.293 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:46:50.294+0000 7f708200b700 1 --2- 172.21.15.120:0/1512297762 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f707c103710 0x7f707c103b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:46:50.293 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:46:50.294+0000 7f708200b700 1 -- 172.21.15.120:0/1512297762 >> 172.21.15.120:0/1512297762 conn(0x7f707c0fdab0 msgr2=0x7f707c0ffef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:46:50.293 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:46:50.294+0000 7f708200b700 1 -- 172.21.15.120:0/1512297762 shutdown_connections 2024-06-10T02:46:50.293 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:46:50.294+0000 7f708200b700 1 -- 172.21.15.120:0/1512297762 wait complete. 2024-06-10T02:46:50.293 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:46:50.295+0000 7f708200b700 1 Processor -- start 2024-06-10T02:46:50.295 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:46:50.295+0000 7f708200b700 1 -- start start 2024-06-10T02:46:50.296 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:46:50.295+0000 7f708200b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f707c102510 0x7f707c076230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:46:50.296 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:46:50.295+0000 7f708200b700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f707c103710 0x7f707c074880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:46:50.296 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:46:50.295+0000 7f708200b700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f707c1040b0 0x7f707c074dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:46:50.296 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:46:50.295+0000 7f708200b700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f707c10e100 con 0x7f707c102510 2024-06-10T02:46:50.296 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:46:50.295+0000 7f708200b700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f707c10df80 con 0x7f707c103710 2024-06-10T02:46:50.296 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:46:50.295+0000 7f708200b700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f707c10e280 con 0x7f707c1040b0 2024-06-10T02:46:50.296 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:46:50.296+0000 7f707affd700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f707c103710 0x7f707c074880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:46:50.296 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:46:50.296+0000 7f707affd700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f707c103710 0x7f707c074880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:54746/0 (socket says 172.21.15.120:54746) 2024-06-10T02:46:50.296 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:46:50.296+0000 7f707affd700 1 -- 172.21.15.120:0/380546446 learned_addr learned my addr 172.21.15.120:0/380546446 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-06-10T02:46:50.296 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:46:50.296+0000 7f707b7fe700 1 --2- 172.21.15.120:0/380546446 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f707c102510 0x7f707c076230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:46:50.296 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:46:50.296+0000 7f707bfff700 1 --2- 172.21.15.120:0/380546446 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f707c1040b0 0x7f707c074dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:46:50.296 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:46:50.296+0000 7f707b7fe700 1 -- 172.21.15.120:0/380546446 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f707c1040b0 msgr2=0x7f707c074dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:46:50.296 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:46:50.296+0000 7f707b7fe700 1 --2- 172.21.15.120:0/380546446 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f707c1040b0 0x7f707c074dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:46:50.297 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:46:50.296+0000 7f707b7fe700 1 -- 172.21.15.120:0/380546446 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f707c103710 msgr2=0x7f707c074880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:46:50.297 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:46:50.296+0000 7f707b7fe700 1 --2- 172.21.15.120:0/380546446 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f707c103710 0x7f707c074880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:46:50.297 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:46:50.296+0000 7f707b7fe700 1 -- 172.21.15.120:0/380546446 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f707c075670 con 0x7f707c102510 2024-06-10T02:46:50.297 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:46:50.296+0000 7f707affd700 1 --2- 172.21.15.120:0/380546446 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f707c103710 0x7f707c074880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T02:46:50.297 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:46:50.296+0000 7f707bfff700 1 --2- 172.21.15.120:0/380546446 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f707c1040b0 0x7f707c074dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T02:46:50.297 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:46:50.296+0000 7f707b7fe700 1 --2- 172.21.15.120:0/380546446 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f707c102510 0x7f707c076230 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f706c0098e0 tx=0x7f706c0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:46:50.297 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:46:50.296+0000 7f7078ff9700 1 -- 172.21.15.120:0/380546446 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f706c027070 con 0x7f707c102510 2024-06-10T02:46:50.297 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:46:50.296+0000 7f708200b700 1 -- 172.21.15.120:0/380546446 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f707c1a2820 con 0x7f707c102510 2024-06-10T02:46:50.297 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:46:50.296+0000 7f708200b700 1 -- 172.21.15.120:0/380546446 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f707c1a2d00 con 0x7f707c102510 2024-06-10T02:46:50.297 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:46:50.296+0000 7f7078ff9700 1 -- 172.21.15.120:0/380546446 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f706c004210 con 0x7f707c102510 2024-06-10T02:46:50.297 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:46:50.296+0000 7f7078ff9700 1 -- 172.21.15.120:0/380546446 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f706c022e40 con 0x7f707c102510 2024-06-10T02:46:50.297 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:46:50.297+0000 7f708200b700 1 -- 172.21.15.120:0/380546446 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f707c069180 con 0x7f707c102510 2024-06-10T02:46:50.298 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:46:50.299+0000 7f7078ff9700 1 -- 172.21.15.120:0/380546446 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f706c02a070 con 0x7f707c102510 2024-06-10T02:46:50.298 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:46:50.299+0000 7f7078ff9700 1 --2- 172.21.15.120:0/380546446 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f70640a2c20 0x7f70640a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:46:50.298 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:46:50.299+0000 7f7078ff9700 1 -- 172.21.15.120:0/380546446 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 4965+0+0 (secure 0 0 0) 0x7f706c0cf590 con 0x7f707c102510 2024-06-10T02:46:50.298 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:46:50.300+0000 7f707affd700 1 --2- 172.21.15.120:0/380546446 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f70640a2c20 0x7f70640a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:46:50.298 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:46:50.300+0000 7f707affd700 1 --2- 172.21.15.120:0/380546446 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f70640a2c20 0x7f70640a50e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f707000f4d0 tx=0x7f7070005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:46:50.301 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:46:50.303+0000 7f7078ff9700 1 -- 172.21.15.120:0/380546446 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f706c013070 con 0x7f707c102510 2024-06-10T02:46:50.505 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:46:50.507+0000 7f708200b700 1 -- 172.21.15.120:0/380546446 --> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi120:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f707c0628d0 con 0x7f70640a2c20 2024-06-10T02:46:50.886 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:46:50 smithi120 ceph-mon[36634]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-06-10T02:46:50.886 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:46:50 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-10T02:46:50.886 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:46:50 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-10T02:46:50.886 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:46:50 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:46:50.974 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:46:50 smithi052 ceph-mon[32362]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-06-10T02:46:50.975 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:46:50 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-10T02:46:50.975 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:46:50 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-10T02:46:50.975 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:46:50 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:46:51.201 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:46:50 smithi188 ceph-mon[36599]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-06-10T02:46:51.201 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:46:50 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-10T02:46:51.201 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:46:50 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-10T02:46:51.201 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:46:50 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:46:51.974 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:46:51 smithi052 ceph-mon[32362]: from='client.14367 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi120:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T02:46:52.102 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:46:51 smithi188 ceph-mon[36599]: from='client.14367 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi120:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T02:46:52.136 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:46:51 smithi120 ceph-mon[36634]: from='client.14367 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi120:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T02:46:52.974 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:46:52 smithi052 ceph-mon[32362]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-06-10T02:46:53.136 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:46:52 smithi120 ceph-mon[36634]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-06-10T02:46:53.201 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:46:52 smithi188 ceph-mon[36599]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-06-10T02:46:53.975 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:46:53 smithi052 ceph-mon[32362]: from='client.? 172.21.15.120:0/2990950138' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "57b735d4-5072-4ca2-a3ff-3763ad84f15a"}]: dispatch 2024-06-10T02:46:53.975 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:46:53 smithi052 ceph-mon[32362]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "57b735d4-5072-4ca2-a3ff-3763ad84f15a"}]: dispatch 2024-06-10T02:46:53.975 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:46:53 smithi052 ceph-mon[32362]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "57b735d4-5072-4ca2-a3ff-3763ad84f15a"}]': finished 2024-06-10T02:46:53.975 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:46:53 smithi052 ceph-mon[32362]: osdmap e49: 8 total, 7 up, 8 in 2024-06-10T02:46:53.975 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:46:53 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-10T02:46:54.037 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:46:53 smithi120 ceph-mon[36634]: from='client.? 172.21.15.120:0/2990950138' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "57b735d4-5072-4ca2-a3ff-3763ad84f15a"}]: dispatch 2024-06-10T02:46:54.037 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:46:53 smithi120 ceph-mon[36634]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "57b735d4-5072-4ca2-a3ff-3763ad84f15a"}]: dispatch 2024-06-10T02:46:54.038 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:46:53 smithi120 ceph-mon[36634]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "57b735d4-5072-4ca2-a3ff-3763ad84f15a"}]': finished 2024-06-10T02:46:54.038 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:46:53 smithi120 ceph-mon[36634]: osdmap e49: 8 total, 7 up, 8 in 2024-06-10T02:46:54.038 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:46:53 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-10T02:46:54.201 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:46:53 smithi188 ceph-mon[36599]: from='client.? 172.21.15.120:0/2990950138' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "57b735d4-5072-4ca2-a3ff-3763ad84f15a"}]: dispatch 2024-06-10T02:46:54.201 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:46:53 smithi188 ceph-mon[36599]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "57b735d4-5072-4ca2-a3ff-3763ad84f15a"}]: dispatch 2024-06-10T02:46:54.201 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:46:53 smithi188 ceph-mon[36599]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "57b735d4-5072-4ca2-a3ff-3763ad84f15a"}]': finished 2024-06-10T02:46:54.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:46:53 smithi188 ceph-mon[36599]: osdmap e49: 8 total, 7 up, 8 in 2024-06-10T02:46:54.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:46:53 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-10T02:46:54.974 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:46:54 smithi052 ceph-mon[32362]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-06-10T02:46:54.975 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:46:54 smithi052 ceph-mon[32362]: from='client.? 172.21.15.120:0/23814373' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-10T02:46:55.047 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:46:54 smithi120 ceph-mon[36634]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-06-10T02:46:55.048 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:46:54 smithi120 ceph-mon[36634]: from='client.? 172.21.15.120:0/23814373' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-10T02:46:55.201 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:46:54 smithi188 ceph-mon[36599]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-06-10T02:46:55.201 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:46:54 smithi188 ceph-mon[36599]: from='client.? 172.21.15.120:0/23814373' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-10T02:46:56.974 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:46:56 smithi052 ceph-mon[32362]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-06-10T02:46:56.976 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:46:56 smithi120 ceph-mon[36634]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-06-10T02:46:57.201 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:46:56 smithi188 ceph-mon[36599]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-06-10T02:46:58.974 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:46:58 smithi052 ceph-mon[32362]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-06-10T02:46:59.136 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:46:58 smithi120 ceph-mon[36634]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-06-10T02:46:59.201 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:46:58 smithi188 ceph-mon[36599]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-06-10T02:47:00.974 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:47:00 smithi052 ceph-mon[32362]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-06-10T02:47:01.136 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:47:00 smithi120 ceph-mon[36634]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-06-10T02:47:01.201 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:47:00 smithi188 ceph-mon[36599]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-06-10T02:47:02.136 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:47:01 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-06-10T02:47:02.136 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:47:01 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:47:02.136 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:47:01 smithi120 ceph-mon[36634]: Deploying daemon osd.7 on smithi120 2024-06-10T02:47:02.201 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:47:01 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-06-10T02:47:02.201 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:47:01 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:47:02.201 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:47:01 smithi188 ceph-mon[36599]: Deploying daemon osd.7 on smithi120 2024-06-10T02:47:02.224 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:47:01 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-06-10T02:47:02.224 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:47:01 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:47:02.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:47:01 smithi052 ceph-mon[32362]: Deploying daemon osd.7 on smithi120 2024-06-10T02:47:03.038 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:47:02 smithi120 ceph-mon[36634]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-06-10T02:47:03.201 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:47:02 smithi188 ceph-mon[36599]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-06-10T02:47:03.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:47:02 smithi052 ceph-mon[32362]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-06-10T02:47:04.848 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:47:04 smithi120 ceph-mon[36634]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-06-10T02:47:05.027 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:47:04 smithi052 ceph-mon[32362]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-06-10T02:47:05.201 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:47:04 smithi188 ceph-mon[36599]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-06-10T02:47:07.029 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:47:06 smithi120 ceph-mon[36634]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-06-10T02:47:07.029 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:47:06 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:47:07.029 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:47:06 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:47:07.029 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:47:06 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T02:47:07.201 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:47:06 smithi188 ceph-mon[36599]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-06-10T02:47:07.201 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:47:06 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:47:07.201 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:47:06 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:47:07.201 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:47:06 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T02:47:07.224 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:47:06 smithi052 ceph-mon[32362]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-06-10T02:47:07.224 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:47:06 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:47:07.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:47:06 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:47:07.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:47:06 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T02:47:08.886 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:47:08 smithi120 ceph-mon[36634]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-06-10T02:47:09.201 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:47:08 smithi188 ceph-mon[36599]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-06-10T02:47:09.224 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:47:08 smithi052 ceph-mon[32362]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-06-10T02:47:10.530 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:47:10.532+0000 7f7078ff9700 1 -- 172.21.15.120:0/380546446 <== mgr.14150 v2:172.21.15.52:6800/3214763969 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f707c0628d0 con 0x7f70640a2c20 2024-06-10T02:47:10.531 INFO:teuthology.orchestra.run.smithi120.stdout:Created osd(s) 7 on host 'smithi120' 2024-06-10T02:47:10.533 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:47:10.535+0000 7f708200b700 1 -- 172.21.15.120:0/380546446 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f70640a2c20 msgr2=0x7f70640a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:47:10.533 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:47:10.535+0000 7f708200b700 1 --2- 172.21.15.120:0/380546446 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f70640a2c20 0x7f70640a50e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f707000f4d0 tx=0x7f7070005f90 comp rx=0 tx=0).stop 2024-06-10T02:47:10.533 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:47:10.535+0000 7f708200b700 1 -- 172.21.15.120:0/380546446 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f707c102510 msgr2=0x7f707c076230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:47:10.533 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:47:10.535+0000 7f708200b700 1 --2- 172.21.15.120:0/380546446 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f707c102510 0x7f707c076230 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f706c0098e0 tx=0x7f706c0047b0 comp rx=0 tx=0).stop 2024-06-10T02:47:10.533 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:47:10.535+0000 7f708200b700 1 -- 172.21.15.120:0/380546446 shutdown_connections 2024-06-10T02:47:10.534 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:47:10.535+0000 7f708200b700 1 --2- 172.21.15.120:0/380546446 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f707c1040b0 0x7f707c074dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:47:10.534 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:47:10.535+0000 7f708200b700 1 --2- 172.21.15.120:0/380546446 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f70640a2c20 0x7f70640a50e0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:47:10.534 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:47:10.535+0000 7f708200b700 1 --2- 172.21.15.120:0/380546446 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f707c102510 0x7f707c076230 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:47:10.534 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:47:10.535+0000 7f708200b700 1 --2- 172.21.15.120:0/380546446 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f707c103710 0x7f707c074880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:47:10.534 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:47:10.535+0000 7f708200b700 1 -- 172.21.15.120:0/380546446 >> 172.21.15.120:0/380546446 conn(0x7f707c0fdab0 msgr2=0x7f707c1069b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:47:10.534 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:47:10.535+0000 7f708200b700 1 -- 172.21.15.120:0/380546446 shutdown_connections 2024-06-10T02:47:10.534 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:47:10.535+0000 7f708200b700 1 -- 172.21.15.120:0/380546446 wait complete. 2024-06-10T02:47:10.886 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:47:10 smithi120 ceph-mon[36634]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-06-10T02:47:10.886 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:47:10 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:47:10.887 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:47:10 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:47:10.887 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:47:10 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:47:10.887 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:47:10 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T02:47:10.887 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:47:10 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:47:10.887 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:47:10 smithi120 ceph-mon[36634]: from='osd.7 [v2:172.21.15.120:6816/3737722011,v1:172.21.15.120:6817/3737722011]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-06-10T02:47:10.887 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:47:10 smithi120 ceph-mon[36634]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-06-10T02:47:10.887 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:47:10 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:47:10.887 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:47:10 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:47:10.887 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:47:10 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T02:47:10.975 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:47:10 smithi052 ceph-mon[32362]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-06-10T02:47:10.975 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:47:10 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:47:10.975 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:47:10 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:47:10.975 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:47:10 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:47:10.976 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:47:10 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T02:47:10.976 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:47:10 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:47:10.976 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:47:10 smithi052 ceph-mon[32362]: from='osd.7 [v2:172.21.15.120:6816/3737722011,v1:172.21.15.120:6817/3737722011]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-06-10T02:47:10.976 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:47:10 smithi052 ceph-mon[32362]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-06-10T02:47:10.976 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:47:10 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:47:10.976 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:47:10 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:47:10.976 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:47:10 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T02:47:11.201 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:47:10 smithi188 ceph-mon[36599]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-06-10T02:47:11.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:47:10 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:47:11.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:47:10 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:47:11.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:47:10 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:47:11.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:47:10 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T02:47:11.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:47:10 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:47:11.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:47:10 smithi188 ceph-mon[36599]: from='osd.7 [v2:172.21.15.120:6816/3737722011,v1:172.21.15.120:6817/3737722011]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-06-10T02:47:11.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:47:10 smithi188 ceph-mon[36599]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-06-10T02:47:11.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:47:10 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:47:11.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:47:10 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:47:11.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:47:10 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T02:47:11.269 DEBUG:teuthology.orchestra.run.smithi120:osd.7> sudo journalctl -f -n 0 -u ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968@osd.7.service 2024-06-10T02:47:11.271 INFO:tasks.cephadm:Deploying osd.8 on smithi188 with /dev/vg_nvme/lv_2... 2024-06-10T02:47:11.271 DEBUG:teuthology.orchestra.run.smithi188:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-06-10T02:47:11.514 INFO:teuthology.orchestra.run.smithi188.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.c/config 2024-06-10T02:47:11.757 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:47:11 smithi120 ceph-mon[36634]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-06-10T02:47:11.757 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:47:11 smithi120 ceph-mon[36634]: osdmap e50: 8 total, 7 up, 8 in 2024-06-10T02:47:11.757 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:47:11 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-10T02:47:11.757 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:47:11 smithi120 ceph-mon[36634]: from='osd.7 [v2:172.21.15.120:6816/3737722011,v1:172.21.15.120:6817/3737722011]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]: dispatch 2024-06-10T02:47:11.757 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:47:11 smithi120 ceph-mon[36634]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]: dispatch 2024-06-10T02:47:11.947 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:47:11 smithi188 ceph-mon[36599]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-06-10T02:47:11.947 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:47:11 smithi188 ceph-mon[36599]: osdmap e50: 8 total, 7 up, 8 in 2024-06-10T02:47:11.947 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:47:11 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-10T02:47:11.947 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:47:11 smithi188 ceph-mon[36599]: from='osd.7 [v2:172.21.15.120:6816/3737722011,v1:172.21.15.120:6817/3737722011]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]: dispatch 2024-06-10T02:47:11.947 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:47:11 smithi188 ceph-mon[36599]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]: dispatch 2024-06-10T02:47:12.136 INFO:journalctl@ceph.osd.7.smithi120.stdout:Jun 10 02:47:11 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-7[54219]: 2024-06-10T02:47:11.764+0000 7f3b278ac700 -1 osd.7 0 waiting for initial osdmap 2024-06-10T02:47:12.136 INFO:journalctl@ceph.osd.7.smithi120.stdout:Jun 10 02:47:11 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-7[54219]: 2024-06-10T02:47:11.772+0000 7f3b2369f700 -1 osd.7 51 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-10T02:47:12.224 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:47:11 smithi052 ceph-mon[32362]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-06-10T02:47:12.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:47:11 smithi052 ceph-mon[32362]: osdmap e50: 8 total, 7 up, 8 in 2024-06-10T02:47:12.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:47:11 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-10T02:47:12.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:47:11 smithi052 ceph-mon[32362]: from='osd.7 [v2:172.21.15.120:6816/3737722011,v1:172.21.15.120:6817/3737722011]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]: dispatch 2024-06-10T02:47:12.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:47:11 smithi052 ceph-mon[32362]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]: dispatch 2024-06-10T02:47:12.951 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:47:12 smithi188 ceph-mon[36599]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-06-10T02:47:12.951 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:47:12 smithi188 ceph-mon[36599]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]': finished 2024-06-10T02:47:12.951 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:47:12 smithi188 ceph-mon[36599]: osdmap e51: 8 total, 7 up, 8 in 2024-06-10T02:47:12.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:47:12 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-10T02:47:12.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:47:12 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-10T02:47:13.089 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:47:12 smithi120 ceph-mon[36634]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-06-10T02:47:13.089 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:47:12 smithi120 ceph-mon[36634]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]': finished 2024-06-10T02:47:13.090 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:47:12 smithi120 ceph-mon[36634]: osdmap e51: 8 total, 7 up, 8 in 2024-06-10T02:47:13.090 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:47:12 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-10T02:47:13.090 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:47:12 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-10T02:47:13.224 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:47:12 smithi052 ceph-mon[32362]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-06-10T02:47:13.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:47:12 smithi052 ceph-mon[32362]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]': finished 2024-06-10T02:47:13.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:47:12 smithi052 ceph-mon[32362]: osdmap e51: 8 total, 7 up, 8 in 2024-06-10T02:47:13.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:47:12 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-10T02:47:13.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:47:12 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-10T02:47:13.554 INFO:teuthology.orchestra.run.smithi188.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-06-10T02:47:13.554 INFO:teuthology.orchestra.run.smithi188.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-06-10T02:47:13.554 INFO:teuthology.orchestra.run.smithi188.stderr: stderr: 10+0 records in 2024-06-10T02:47:13.554 INFO:teuthology.orchestra.run.smithi188.stderr:10+0 records out 2024-06-10T02:47:13.554 INFO:teuthology.orchestra.run.smithi188.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0129698 s, 808 MB/s 2024-06-10T02:47:13.554 INFO:teuthology.orchestra.run.smithi188.stderr:--> Zapping successful for: 2024-06-10T02:47:13.886 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:47:13 smithi120 ceph-mon[36634]: purged_snaps scrub starts 2024-06-10T02:47:13.886 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:47:13 smithi120 ceph-mon[36634]: purged_snaps scrub ok 2024-06-10T02:47:13.886 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:47:13 smithi120 ceph-mon[36634]: osd.7 [v2:172.21.15.120:6816/3737722011,v1:172.21.15.120:6817/3737722011] boot 2024-06-10T02:47:13.887 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:47:13 smithi120 ceph-mon[36634]: osdmap e52: 8 total, 8 up, 8 in 2024-06-10T02:47:13.887 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:47:13 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-10T02:47:13.887 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:47:13 smithi120 ceph-mon[36634]: osdmap e53: 8 total, 8 up, 8 in 2024-06-10T02:47:14.185 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:47:13 smithi188 ceph-mon[36599]: purged_snaps scrub starts 2024-06-10T02:47:14.185 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:47:13 smithi188 ceph-mon[36599]: purged_snaps scrub ok 2024-06-10T02:47:14.185 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:47:13 smithi188 ceph-mon[36599]: osd.7 [v2:172.21.15.120:6816/3737722011,v1:172.21.15.120:6817/3737722011] boot 2024-06-10T02:47:14.185 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:47:13 smithi188 ceph-mon[36599]: osdmap e52: 8 total, 8 up, 8 in 2024-06-10T02:47:14.185 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:47:13 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-10T02:47:14.185 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:47:13 smithi188 ceph-mon[36599]: osdmap e53: 8 total, 8 up, 8 in 2024-06-10T02:47:14.212 DEBUG:teuthology.orchestra.run.smithi188:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi188:vg_nvme/lv_2 2024-06-10T02:47:14.224 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:47:13 smithi052 ceph-mon[32362]: purged_snaps scrub starts 2024-06-10T02:47:14.224 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:47:13 smithi052 ceph-mon[32362]: purged_snaps scrub ok 2024-06-10T02:47:14.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:47:13 smithi052 ceph-mon[32362]: osd.7 [v2:172.21.15.120:6816/3737722011,v1:172.21.15.120:6817/3737722011] boot 2024-06-10T02:47:14.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:47:13 smithi052 ceph-mon[32362]: osdmap e52: 8 total, 8 up, 8 in 2024-06-10T02:47:14.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:47:13 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-10T02:47:14.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:47:13 smithi052 ceph-mon[32362]: osdmap e53: 8 total, 8 up, 8 in 2024-06-10T02:47:14.450 INFO:teuthology.orchestra.run.smithi188.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.c/config 2024-06-10T02:47:14.830 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:47:14 smithi188 ceph-mon[36599]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-06-10T02:47:14.830 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:47:14 smithi188 ceph-mon[36599]: osdmap e54: 8 total, 8 up, 8 in 2024-06-10T02:47:14.830 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:47:14 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:47:14.830 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:47:14 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:47:14.830 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:47:14 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-10T02:47:14.830 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:47:14 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-10T02:47:14.830 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:47:14 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-10T02:47:14.831 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:47:14 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:47:14.831 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:47:14 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:47:14.831 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:47:14 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T02:47:14.831 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:47:14 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:47:15.136 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:47:14 smithi120 ceph-mon[36634]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-06-10T02:47:15.136 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:47:14 smithi120 ceph-mon[36634]: osdmap e54: 8 total, 8 up, 8 in 2024-06-10T02:47:15.136 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:47:14 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:47:15.136 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:47:14 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:47:15.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:47:14 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-10T02:47:15.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:47:14 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-10T02:47:15.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:47:14 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-10T02:47:15.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:47:14 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:47:15.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:47:14 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:47:15.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:47:14 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T02:47:15.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:47:14 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:47:15.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:47:14 smithi052 ceph-mon[32362]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-06-10T02:47:15.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:47:14 smithi052 ceph-mon[32362]: osdmap e54: 8 total, 8 up, 8 in 2024-06-10T02:47:15.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:47:14 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:47:15.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:47:14 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:47:15.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:47:14 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-10T02:47:15.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:47:14 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-10T02:47:15.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:47:14 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-10T02:47:15.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:47:14 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:47:15.226 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:47:14 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:47:15.226 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:47:14 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T02:47:15.226 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:47:14 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:47:15.835 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:47:15 smithi188 ceph-mon[36599]: Detected new or changed devices on smithi120 2024-06-10T02:47:15.836 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:47:15 smithi188 ceph-mon[36599]: Adjusting osd_memory_target on smithi120 to 5696M 2024-06-10T02:47:15.836 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:47:15 smithi188 ceph-mon[36599]: osdmap e55: 8 total, 8 up, 8 in 2024-06-10T02:47:16.034 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:47:16.035+0000 7fdab1ffa700 1 -- 172.21.15.188:0/3540347922 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdaac105900 msgr2=0x7fdaac107cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:47:16.034 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:47:16.035+0000 7fdab1ffa700 1 --2- 172.21.15.188:0/3540347922 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdaac105900 0x7fdaac107cf0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fdaa0009990 tx=0x7fdaa0009ca0 comp rx=0 tx=0).stop 2024-06-10T02:47:16.034 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:47:16.035+0000 7fdab1ffa700 1 -- 172.21.15.188:0/3540347922 shutdown_connections 2024-06-10T02:47:16.034 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:47:16.035+0000 7fdab1ffa700 1 --2- 172.21.15.188:0/3540347922 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdaac108230 0x7fdaac10a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:47:16.034 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:47:16.035+0000 7fdab1ffa700 1 --2- 172.21.15.188:0/3540347922 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdaac102fd0 0x7fdaac1053c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:47:16.034 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:47:16.035+0000 7fdab1ffa700 1 --2- 172.21.15.188:0/3540347922 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdaac105900 0x7fdaac107cf0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:47:16.035 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:47:16.035+0000 7fdab1ffa700 1 -- 172.21.15.188:0/3540347922 >> 172.21.15.188:0/3540347922 conn(0x7fdaac0fc980 msgr2=0x7fdaac0fedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:47:16.035 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:47:16.035+0000 7fdab1ffa700 1 -- 172.21.15.188:0/3540347922 shutdown_connections 2024-06-10T02:47:16.035 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:47:16.035+0000 7fdab1ffa700 1 -- 172.21.15.188:0/3540347922 wait complete. 2024-06-10T02:47:16.035 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:47:16.036+0000 7fdab1ffa700 1 Processor -- start 2024-06-10T02:47:16.035 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:47:16.036+0000 7fdab1ffa700 1 -- start start 2024-06-10T02:47:16.035 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:47:16.036+0000 7fdab1ffa700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdaac102fd0 0x7fdaac102930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:47:16.035 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:47:16.036+0000 7fdab1ffa700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdaac105900 0x7fdaac100f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:47:16.035 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:47:16.036+0000 7fdab1ffa700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdaac108230 0x7fdaac1014c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:47:16.035 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:47:16.036+0000 7fdab1ffa700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fdaac10e0f0 con 0x7fdaac108230 2024-06-10T02:47:16.035 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:47:16.036+0000 7fdab1ffa700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fdaac10df70 con 0x7fdaac105900 2024-06-10T02:47:16.035 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:47:16.036+0000 7fdab1ffa700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fdaac10e270 con 0x7fdaac102fd0 2024-06-10T02:47:16.035 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:47:16.037+0000 7fdaab7fe700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdaac102fd0 0x7fdaac102930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:47:16.035 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:47:16.037+0000 7fdaaaffd700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdaac105900 0x7fdaac100f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:47:16.036 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:47:16.037+0000 7fdaab7fe700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdaac102fd0 0x7fdaac102930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.188:49470/0 (socket says 172.21.15.188:49470) 2024-06-10T02:47:16.036 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:47:16.037+0000 7fdaab7fe700 1 -- 172.21.15.188:0/3619047975 learned_addr learned my addr 172.21.15.188:0/3619047975 (peer_addr_for_me v2:172.21.15.188:0/0) 2024-06-10T02:47:16.036 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:47:16.037+0000 7fdaabfff700 1 --2- 172.21.15.188:0/3619047975 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdaac108230 0x7fdaac1014c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:47:16.037 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:47:16.037+0000 7fdaab7fe700 1 -- 172.21.15.188:0/3619047975 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdaac105900 msgr2=0x7fdaac100f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:47:16.037 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:47:16.037+0000 7fdaab7fe700 1 --2- 172.21.15.188:0/3619047975 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdaac105900 0x7fdaac100f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:47:16.037 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:47:16.037+0000 7fdaab7fe700 1 -- 172.21.15.188:0/3619047975 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdaac108230 msgr2=0x7fdaac1014c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:47:16.037 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:47:16.037+0000 7fdaab7fe700 1 --2- 172.21.15.188:0/3619047975 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdaac108230 0x7fdaac1014c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:47:16.037 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:47:16.037+0000 7fdaab7fe700 1 -- 172.21.15.188:0/3619047975 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdaac101d70 con 0x7fdaac102fd0 2024-06-10T02:47:16.037 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:47:16.037+0000 7fdaab7fe700 1 --2- 172.21.15.188:0/3619047975 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdaac102fd0 0x7fdaac102930 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fda9c00d950 tx=0x7fda9c00dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:47:16.037 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:47:16.037+0000 7fdaa8ff9700 1 -- 172.21.15.188:0/3619047975 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda9c016070 con 0x7fdaac102fd0 2024-06-10T02:47:16.037 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:47:16.037+0000 7fdaa8ff9700 1 -- 172.21.15.188:0/3619047975 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda9c00fd70 con 0x7fdaac102fd0 2024-06-10T02:47:16.037 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:47:16.037+0000 7fdab1ffa700 1 -- 172.21.15.188:0/3619047975 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdaac1a2880 con 0x7fdaac102fd0 2024-06-10T02:47:16.038 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:47:16.037+0000 7fdaa8ff9700 1 -- 172.21.15.188:0/3619047975 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda9c00e610 con 0x7fdaac102fd0 2024-06-10T02:47:16.038 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:47:16.037+0000 7fdab1ffa700 1 -- 172.21.15.188:0/3619047975 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdaac1a2d90 con 0x7fdaac102fd0 2024-06-10T02:47:16.038 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:47:16.038+0000 7fdab1ffa700 1 -- 172.21.15.188:0/3619047975 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdaac069180 con 0x7fdaac102fd0 2024-06-10T02:47:16.038 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:47:16.039+0000 7fdaa8ff9700 1 -- 172.21.15.188:0/3619047975 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fda9c019070 con 0x7fdaac102fd0 2024-06-10T02:47:16.038 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:47:16.040+0000 7fdaa8ff9700 1 --2- 172.21.15.188:0/3619047975 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fda940a2c20 0x7fda940a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:47:16.039 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:47:16.040+0000 7fdaa8ff9700 1 -- 172.21.15.188:0/3619047975 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5397+0+0 (secure 0 0 0) 0x7fda9c0cdf70 con 0x7fdaac102fd0 2024-06-10T02:47:16.039 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:47:16.040+0000 7fdaaaffd700 1 --2- 172.21.15.188:0/3619047975 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fda940a2c20 0x7fda940a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:47:16.039 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:47:16.041+0000 7fdaaaffd700 1 --2- 172.21.15.188:0/3619047975 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fda940a2c20 0x7fda940a50e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fdaa000b5c0 tx=0x7fdaa0005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:47:16.042 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:47:16.045+0000 7fdaa8ff9700 1 -- 172.21.15.188:0/3619047975 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fda9c09ba30 con 0x7fdaac102fd0 2024-06-10T02:47:16.136 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:47:15 smithi120 ceph-mon[36634]: Detected new or changed devices on smithi120 2024-06-10T02:47:16.136 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:47:15 smithi120 ceph-mon[36634]: Adjusting osd_memory_target on smithi120 to 5696M 2024-06-10T02:47:16.136 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:47:15 smithi120 ceph-mon[36634]: osdmap e55: 8 total, 8 up, 8 in 2024-06-10T02:47:16.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:47:15 smithi052 ceph-mon[32362]: Detected new or changed devices on smithi120 2024-06-10T02:47:16.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:47:15 smithi052 ceph-mon[32362]: Adjusting osd_memory_target on smithi120 to 5696M 2024-06-10T02:47:16.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:47:15 smithi052 ceph-mon[32362]: osdmap e55: 8 total, 8 up, 8 in 2024-06-10T02:47:16.246 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:47:16.249+0000 7fdab1ffa700 1 -- 172.21.15.188:0/3619047975 --> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi188:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fdaac02e0a0 con 0x7fda940a2c20 2024-06-10T02:47:17.073 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:47:16 smithi188 ceph-mon[36599]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-06-10T02:47:17.073 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:47:16 smithi188 ceph-mon[36599]: osdmap e56: 8 total, 8 up, 8 in 2024-06-10T02:47:17.073 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:47:16 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-10T02:47:17.073 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:47:16 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-10T02:47:17.073 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:47:16 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:47:17.136 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:47:16 smithi120 ceph-mon[36634]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-06-10T02:47:17.136 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:47:16 smithi120 ceph-mon[36634]: osdmap e56: 8 total, 8 up, 8 in 2024-06-10T02:47:17.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:47:16 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-10T02:47:17.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:47:16 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-10T02:47:17.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:47:16 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:47:17.224 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:47:16 smithi052 ceph-mon[32362]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-06-10T02:47:17.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:47:16 smithi052 ceph-mon[32362]: osdmap e56: 8 total, 8 up, 8 in 2024-06-10T02:47:17.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:47:16 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-10T02:47:17.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:47:16 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-10T02:47:17.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:47:16 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:47:18.122 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:47:17 smithi188 ceph-mon[36599]: from='client.24310 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi188:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T02:47:18.136 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:47:17 smithi120 ceph-mon[36634]: from='client.24310 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi188:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T02:47:18.224 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:47:17 smithi052 ceph-mon[32362]: from='client.24310 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi188:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T02:47:19.136 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:47:18 smithi120 ceph-mon[36634]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-06-10T02:47:19.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:47:18 smithi188 ceph-mon[36599]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-06-10T02:47:19.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:47:18 smithi052 ceph-mon[32362]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-06-10T02:47:20.136 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:47:19 smithi120 ceph-mon[36634]: from='client.? 172.21.15.188:0/1323116794' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4ffbb43b-854a-4eac-99cf-7af89bc65409"}]: dispatch 2024-06-10T02:47:20.136 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:47:19 smithi120 ceph-mon[36634]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4ffbb43b-854a-4eac-99cf-7af89bc65409"}]: dispatch 2024-06-10T02:47:20.136 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:47:19 smithi120 ceph-mon[36634]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4ffbb43b-854a-4eac-99cf-7af89bc65409"}]': finished 2024-06-10T02:47:20.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:47:19 smithi120 ceph-mon[36634]: osdmap e57: 9 total, 8 up, 9 in 2024-06-10T02:47:20.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:47:19 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-10T02:47:20.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:47:19 smithi120 ceph-mon[36634]: from='client.? 172.21.15.188:0/2491634702' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-10T02:47:20.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:47:19 smithi188 ceph-mon[36599]: from='client.? 172.21.15.188:0/1323116794' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4ffbb43b-854a-4eac-99cf-7af89bc65409"}]: dispatch 2024-06-10T02:47:20.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:47:19 smithi188 ceph-mon[36599]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4ffbb43b-854a-4eac-99cf-7af89bc65409"}]: dispatch 2024-06-10T02:47:20.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:47:19 smithi188 ceph-mon[36599]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4ffbb43b-854a-4eac-99cf-7af89bc65409"}]': finished 2024-06-10T02:47:20.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:47:19 smithi188 ceph-mon[36599]: osdmap e57: 9 total, 8 up, 9 in 2024-06-10T02:47:20.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:47:19 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-10T02:47:20.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:47:19 smithi188 ceph-mon[36599]: from='client.? 172.21.15.188:0/2491634702' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-10T02:47:20.224 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:47:19 smithi052 ceph-mon[32362]: from='client.? 172.21.15.188:0/1323116794' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4ffbb43b-854a-4eac-99cf-7af89bc65409"}]: dispatch 2024-06-10T02:47:20.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:47:19 smithi052 ceph-mon[32362]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4ffbb43b-854a-4eac-99cf-7af89bc65409"}]: dispatch 2024-06-10T02:47:20.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:47:19 smithi052 ceph-mon[32362]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4ffbb43b-854a-4eac-99cf-7af89bc65409"}]': finished 2024-06-10T02:47:20.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:47:19 smithi052 ceph-mon[32362]: osdmap e57: 9 total, 8 up, 9 in 2024-06-10T02:47:20.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:47:19 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-10T02:47:20.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:47:19 smithi052 ceph-mon[32362]: from='client.? 172.21.15.188:0/2491634702' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-10T02:47:21.136 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:47:20 smithi120 ceph-mon[36634]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-06-10T02:47:21.201 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:47:20 smithi188 ceph-mon[36599]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-06-10T02:47:21.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:47:20 smithi052 ceph-mon[32362]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-06-10T02:47:22.951 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:47:22 smithi188 ceph-mon[36599]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-06-10T02:47:23.139 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:47:22 smithi120 ceph-mon[36634]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-06-10T02:47:23.224 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:47:22 smithi052 ceph-mon[32362]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-06-10T02:47:25.107 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:47:24 smithi188 ceph-mon[36599]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-06-10T02:47:25.136 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:47:24 smithi120 ceph-mon[36634]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-06-10T02:47:25.224 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:47:24 smithi052 ceph-mon[32362]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-06-10T02:47:27.136 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:47:26 smithi120 ceph-mon[36634]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-06-10T02:47:27.136 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:47:26 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-06-10T02:47:27.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:47:26 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:47:27.151 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:47:26 smithi188 ceph-mon[36599]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-06-10T02:47:27.151 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:47:26 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-06-10T02:47:27.151 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:47:26 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:47:27.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:47:26 smithi052 ceph-mon[32362]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-06-10T02:47:27.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:47:26 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-06-10T02:47:27.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:47:26 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:47:28.136 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:47:27 smithi120 ceph-mon[36634]: Deploying daemon osd.8 on smithi188 2024-06-10T02:47:28.155 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:47:27 smithi188 ceph-mon[36599]: Deploying daemon osd.8 on smithi188 2024-06-10T02:47:28.224 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:47:27 smithi052 ceph-mon[32362]: Deploying daemon osd.8 on smithi188 2024-06-10T02:47:29.136 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:47:28 smithi120 ceph-mon[36634]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-06-10T02:47:29.155 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:47:28 smithi188 ceph-mon[36599]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-06-10T02:47:29.224 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:47:28 smithi052 ceph-mon[32362]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-06-10T02:47:31.100 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:47:30 smithi188 ceph-mon[36599]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-06-10T02:47:31.136 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:47:30 smithi120 ceph-mon[36634]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-06-10T02:47:31.224 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:47:30 smithi052 ceph-mon[32362]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-06-10T02:47:32.925 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:47:32 smithi188 ceph-mon[36599]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-06-10T02:47:32.925 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:47:32 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:47:32.925 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:47:32 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T02:47:32.926 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:47:32 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:47:33.136 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:47:32 smithi120 ceph-mon[36634]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-06-10T02:47:33.136 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:47:32 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:47:33.136 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:47:32 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T02:47:33.136 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:47:32 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:47:33.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:47:32 smithi052 ceph-mon[32362]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-06-10T02:47:33.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:47:32 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:47:33.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:47:32 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T02:47:33.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:47:32 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:47:34.951 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:47:34 smithi188 ceph-mon[36599]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-06-10T02:47:35.136 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:47:34 smithi120 ceph-mon[36634]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-06-10T02:47:35.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:47:34 smithi052 ceph-mon[32362]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-06-10T02:47:35.715 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:47:35.717+0000 7fdaa8ff9700 1 -- 172.21.15.188:0/3619047975 <== mgr.14150 v2:172.21.15.52:6800/3214763969 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fdaac02e0a0 con 0x7fda940a2c20 2024-06-10T02:47:35.715 INFO:teuthology.orchestra.run.smithi188.stdout:Created osd(s) 8 on host 'smithi188' 2024-06-10T02:47:35.717 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:47:35.720+0000 7fdab1ffa700 1 -- 172.21.15.188:0/3619047975 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fda940a2c20 msgr2=0x7fda940a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:47:35.717 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:47:35.720+0000 7fdab1ffa700 1 --2- 172.21.15.188:0/3619047975 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fda940a2c20 0x7fda940a50e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fdaa000b5c0 tx=0x7fdaa0005fd0 comp rx=0 tx=0).stop 2024-06-10T02:47:35.718 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:47:35.720+0000 7fdab1ffa700 1 -- 172.21.15.188:0/3619047975 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdaac102fd0 msgr2=0x7fdaac102930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:47:35.718 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:47:35.720+0000 7fdab1ffa700 1 --2- 172.21.15.188:0/3619047975 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdaac102fd0 0x7fdaac102930 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fda9c00d950 tx=0x7fda9c00dc60 comp rx=0 tx=0).stop 2024-06-10T02:47:35.718 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:47:35.720+0000 7fdab1ffa700 1 -- 172.21.15.188:0/3619047975 shutdown_connections 2024-06-10T02:47:35.718 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:47:35.720+0000 7fdab1ffa700 1 --2- 172.21.15.188:0/3619047975 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fda940a2c20 0x7fda940a50e0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:47:35.718 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:47:35.720+0000 7fdab1ffa700 1 --2- 172.21.15.188:0/3619047975 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdaac108230 0x7fdaac1014c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:47:35.718 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:47:35.720+0000 7fdab1ffa700 1 --2- 172.21.15.188:0/3619047975 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdaac102fd0 0x7fdaac102930 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:47:35.718 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:47:35.720+0000 7fdab1ffa700 1 --2- 172.21.15.188:0/3619047975 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdaac105900 0x7fdaac100f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:47:35.718 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:47:35.720+0000 7fdab1ffa700 1 -- 172.21.15.188:0/3619047975 >> 172.21.15.188:0/3619047975 conn(0x7fdaac0fc980 msgr2=0x7fdaac103910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:47:35.718 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:47:35.720+0000 7fdab1ffa700 1 -- 172.21.15.188:0/3619047975 shutdown_connections 2024-06-10T02:47:35.718 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:47:35.720+0000 7fdab1ffa700 1 -- 172.21.15.188:0/3619047975 wait complete. 2024-06-10T02:47:36.037 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:47:35 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:47:36.037 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:47:35 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:47:36.037 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:47:35 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:47:36.037 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:47:35 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T02:47:36.037 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:47:35 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:47:36.037 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:47:35 smithi188 ceph-mon[36599]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-06-10T02:47:36.037 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:47:35 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:47:36.037 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:47:35 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:47:36.037 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:47:35 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T02:47:36.136 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:47:35 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:47:36.136 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:47:35 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:47:36.136 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:47:35 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:47:36.136 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:47:35 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T02:47:36.136 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:47:35 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:47:36.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:47:35 smithi120 ceph-mon[36634]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-06-10T02:47:36.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:47:35 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:47:36.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:47:35 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:47:36.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:47:35 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T02:47:36.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:47:35 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:47:36.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:47:35 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:47:36.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:47:35 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:47:36.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:47:35 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T02:47:36.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:47:35 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:47:36.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:47:35 smithi052 ceph-mon[32362]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-06-10T02:47:36.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:47:35 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:47:36.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:47:35 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:47:36.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:47:35 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T02:47:36.373 DEBUG:teuthology.orchestra.run.smithi188:osd.8> sudo journalctl -f -n 0 -u ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968@osd.8.service 2024-06-10T02:47:36.376 INFO:tasks.cephadm:Deploying osd.9 on smithi052 with /dev/vg_nvme/lv_1... 2024-06-10T02:47:36.376 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-06-10T02:47:36.636 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:47:36.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:47:36 smithi188 ceph-mon[36599]: from='osd.8 [v2:172.21.15.188:6816/1444828441,v1:172.21.15.188:6817/1444828441]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-06-10T02:47:36.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:47:36 smithi188 ceph-mon[36599]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-06-10T02:47:37.057 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:47:36 smithi052 ceph-mon[32362]: from='osd.8 [v2:172.21.15.188:6816/1444828441,v1:172.21.15.188:6817/1444828441]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-06-10T02:47:37.057 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:47:36 smithi052 ceph-mon[32362]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-06-10T02:47:37.386 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:47:36 smithi120 ceph-mon[36634]: from='osd.8 [v2:172.21.15.188:6816/1444828441,v1:172.21.15.188:6817/1444828441]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-06-10T02:47:37.386 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:47:36 smithi120 ceph-mon[36634]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-06-10T02:47:38.068 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:47:37 smithi052 ceph-mon[32362]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-06-10T02:47:38.068 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:47:37 smithi052 ceph-mon[32362]: osdmap e58: 9 total, 8 up, 9 in 2024-06-10T02:47:38.068 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:47:37 smithi052 ceph-mon[32362]: from='osd.8 [v2:172.21.15.188:6816/1444828441,v1:172.21.15.188:6817/1444828441]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]: dispatch 2024-06-10T02:47:38.068 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:47:37 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-10T02:47:38.068 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:47:37 smithi052 ceph-mon[32362]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]: dispatch 2024-06-10T02:47:38.068 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:47:37 smithi052 ceph-mon[32362]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-06-10T02:47:38.201 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:47:37 smithi188 ceph-mon[36599]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-06-10T02:47:38.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:47:37 smithi188 ceph-mon[36599]: osdmap e58: 9 total, 8 up, 9 in 2024-06-10T02:47:38.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:47:37 smithi188 ceph-mon[36599]: from='osd.8 [v2:172.21.15.188:6816/1444828441,v1:172.21.15.188:6817/1444828441]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]: dispatch 2024-06-10T02:47:38.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:47:37 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-10T02:47:38.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:47:37 smithi188 ceph-mon[36599]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]: dispatch 2024-06-10T02:47:38.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:47:37 smithi188 ceph-mon[36599]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-06-10T02:47:38.202 INFO:journalctl@ceph.osd.8.smithi188.stdout:Jun 10 02:47:37 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-8[55165]: 2024-06-10T02:47:37.904+0000 7fc333494700 -1 osd.8 0 waiting for initial osdmap 2024-06-10T02:47:38.202 INFO:journalctl@ceph.osd.8.smithi188.stdout:Jun 10 02:47:37 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-8[55165]: 2024-06-10T02:47:37.915+0000 7fc32f287700 -1 osd.8 59 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-10T02:47:38.386 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:47:37 smithi120 ceph-mon[36634]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-06-10T02:47:38.387 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:47:37 smithi120 ceph-mon[36634]: osdmap e58: 9 total, 8 up, 9 in 2024-06-10T02:47:38.387 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:47:37 smithi120 ceph-mon[36634]: from='osd.8 [v2:172.21.15.188:6816/1444828441,v1:172.21.15.188:6817/1444828441]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]: dispatch 2024-06-10T02:47:38.387 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:47:37 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-10T02:47:38.387 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:47:37 smithi120 ceph-mon[36634]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]: dispatch 2024-06-10T02:47:38.387 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:47:37 smithi120 ceph-mon[36634]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-06-10T02:47:38.776 INFO:teuthology.orchestra.run.smithi052.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-06-10T02:47:38.776 INFO:teuthology.orchestra.run.smithi052.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-06-10T02:47:38.776 INFO:teuthology.orchestra.run.smithi052.stderr: stderr: 10+0 records in 2024-06-10T02:47:38.776 INFO:teuthology.orchestra.run.smithi052.stderr:10+0 records out 2024-06-10T02:47:38.777 INFO:teuthology.orchestra.run.smithi052.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0109213 s, 960 MB/s 2024-06-10T02:47:38.777 INFO:teuthology.orchestra.run.smithi052.stderr:--> Zapping successful for: 2024-06-10T02:47:38.975 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:47:38 smithi052 ceph-mon[32362]: purged_snaps scrub starts 2024-06-10T02:47:38.976 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:47:38 smithi052 ceph-mon[32362]: purged_snaps scrub ok 2024-06-10T02:47:38.976 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:47:38 smithi052 ceph-mon[32362]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]': finished 2024-06-10T02:47:38.976 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:47:38 smithi052 ceph-mon[32362]: osdmap e59: 9 total, 8 up, 9 in 2024-06-10T02:47:38.976 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:47:38 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-10T02:47:38.976 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:47:38 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-10T02:47:38.976 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:47:38 smithi052 ceph-mon[32362]: osd.8 [v2:172.21.15.188:6816/1444828441,v1:172.21.15.188:6817/1444828441] boot 2024-06-10T02:47:38.976 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:47:38 smithi052 ceph-mon[32362]: osdmap e60: 9 total, 9 up, 9 in 2024-06-10T02:47:38.976 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:47:38 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-10T02:47:39.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:47:38 smithi188 ceph-mon[36599]: purged_snaps scrub starts 2024-06-10T02:47:39.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:47:38 smithi188 ceph-mon[36599]: purged_snaps scrub ok 2024-06-10T02:47:39.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:47:38 smithi188 ceph-mon[36599]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]': finished 2024-06-10T02:47:39.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:47:38 smithi188 ceph-mon[36599]: osdmap e59: 9 total, 8 up, 9 in 2024-06-10T02:47:39.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:47:38 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-10T02:47:39.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:47:38 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-10T02:47:39.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:47:38 smithi188 ceph-mon[36599]: osd.8 [v2:172.21.15.188:6816/1444828441,v1:172.21.15.188:6817/1444828441] boot 2024-06-10T02:47:39.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:47:38 smithi188 ceph-mon[36599]: osdmap e60: 9 total, 9 up, 9 in 2024-06-10T02:47:39.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:47:38 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-10T02:47:39.386 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:47:38 smithi120 ceph-mon[36634]: purged_snaps scrub starts 2024-06-10T02:47:39.386 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:47:38 smithi120 ceph-mon[36634]: purged_snaps scrub ok 2024-06-10T02:47:39.387 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:47:38 smithi120 ceph-mon[36634]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]': finished 2024-06-10T02:47:39.387 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:47:38 smithi120 ceph-mon[36634]: osdmap e59: 9 total, 8 up, 9 in 2024-06-10T02:47:39.387 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:47:38 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-10T02:47:39.387 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:47:38 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-10T02:47:39.387 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:47:38 smithi120 ceph-mon[36634]: osd.8 [v2:172.21.15.188:6816/1444828441,v1:172.21.15.188:6817/1444828441] boot 2024-06-10T02:47:39.387 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:47:38 smithi120 ceph-mon[36634]: osdmap e60: 9 total, 9 up, 9 in 2024-06-10T02:47:39.387 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:47:38 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-10T02:47:39.482 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi052:vg_nvme/lv_1 2024-06-10T02:47:39.733 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:47:40.386 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:47:40 smithi120 ceph-mon[36634]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-06-10T02:47:40.387 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:47:40 smithi120 ceph-mon[36634]: osdmap e61: 9 total, 9 up, 9 in 2024-06-10T02:47:40.387 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:47:40 smithi120 ceph-mon[36634]: Detected new or changed devices on smithi188 2024-06-10T02:47:40.387 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:47:40 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:47:40.387 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:47:40 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:47:40.387 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:47:40 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-10T02:47:40.387 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:47:40 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-10T02:47:40.387 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:47:40 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-10T02:47:40.387 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:47:40 smithi120 ceph-mon[36634]: Adjusting osd_memory_target on smithi188 to 5696M 2024-06-10T02:47:40.387 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:47:40 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:47:40.387 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:47:40 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:47:40.387 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:47:40 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T02:47:40.387 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:47:40 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:47:40.451 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:47:40 smithi188 ceph-mon[36599]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-06-10T02:47:40.452 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:47:40 smithi188 ceph-mon[36599]: osdmap e61: 9 total, 9 up, 9 in 2024-06-10T02:47:40.452 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:47:40 smithi188 ceph-mon[36599]: Detected new or changed devices on smithi188 2024-06-10T02:47:40.452 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:47:40 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:47:40.452 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:47:40 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:47:40.452 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:47:40 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-10T02:47:40.452 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:47:40 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-10T02:47:40.452 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:47:40 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-10T02:47:40.452 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:47:40 smithi188 ceph-mon[36599]: Adjusting osd_memory_target on smithi188 to 5696M 2024-06-10T02:47:40.452 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:47:40 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:47:40.452 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:47:40 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:47:40.452 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:47:40 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T02:47:40.452 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:47:40 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:47:40.475 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:47:40 smithi052 ceph-mon[32362]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-06-10T02:47:40.475 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:47:40 smithi052 ceph-mon[32362]: osdmap e61: 9 total, 9 up, 9 in 2024-06-10T02:47:40.475 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:47:40 smithi052 ceph-mon[32362]: Detected new or changed devices on smithi188 2024-06-10T02:47:40.475 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:47:40 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:47:40.475 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:47:40 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:47:40.475 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:47:40 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-10T02:47:40.476 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:47:40 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-10T02:47:40.476 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:47:40 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-10T02:47:40.476 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:47:40 smithi052 ceph-mon[32362]: Adjusting osd_memory_target on smithi188 to 5696M 2024-06-10T02:47:40.476 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:47:40 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:47:40.476 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:47:40 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:47:40.476 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:47:40 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T02:47:40.476 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:47:40 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:47:41.122 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:47:41 smithi052 ceph-mon[32362]: osdmap e62: 9 total, 9 up, 9 in 2024-06-10T02:47:41.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:47:41.341+0000 7f4a125ef700 1 -- 172.21.15.52:0/3356263874 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4a0c1025f0 msgr2=0x7f4a0c102a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:47:41.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:47:41.341+0000 7f4a125ef700 1 --2- 172.21.15.52:0/3356263874 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4a0c1025f0 0x7f4a0c102a50 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f49f40099c0 tx=0x7f49f4009cd0 comp rx=0 tx=0).stop 2024-06-10T02:47:41.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:47:41.341+0000 7f4a125ef700 1 -- 172.21.15.52:0/3356263874 shutdown_connections 2024-06-10T02:47:41.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:47:41.341+0000 7f4a125ef700 1 --2- 172.21.15.52:0/3356263874 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4a0c102f90 0x7f4a0c109840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:47:41.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:47:41.341+0000 7f4a125ef700 1 --2- 172.21.15.52:0/3356263874 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4a0c1013f0 0x7f4a0c101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:47:41.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:47:41.341+0000 7f4a125ef700 1 --2- 172.21.15.52:0/3356263874 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4a0c1025f0 0x7f4a0c102a50 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:47:41.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:47:41.341+0000 7f4a125ef700 1 -- 172.21.15.52:0/3356263874 >> 172.21.15.52:0/3356263874 conn(0x7f4a0c0fc990 msgr2=0x7f4a0c0fedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:47:41.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:47:41.341+0000 7f4a125ef700 1 -- 172.21.15.52:0/3356263874 shutdown_connections 2024-06-10T02:47:41.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:47:41.341+0000 7f4a125ef700 1 -- 172.21.15.52:0/3356263874 wait complete. 2024-06-10T02:47:41.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:47:41.342+0000 7f4a125ef700 1 Processor -- start 2024-06-10T02:47:41.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:47:41.342+0000 7f4a125ef700 1 -- start start 2024-06-10T02:47:41.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:47:41.342+0000 7f4a125ef700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4a0c1013f0 0x7f4a0c1986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:47:41.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:47:41.342+0000 7f4a125ef700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4a0c1025f0 0x7f4a0c198c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:47:41.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:47:41.342+0000 7f4a125ef700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4a0c102f90 0x7f4a0c19fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:47:41.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:47:41.342+0000 7f4a125ef700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f4a0c10cfc0 con 0x7f4a0c102f90 2024-06-10T02:47:41.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:47:41.342+0000 7f4a125ef700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f4a0c10ce40 con 0x7f4a0c1013f0 2024-06-10T02:47:41.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:47:41.342+0000 7f4a125ef700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f4a0c10d140 con 0x7f4a0c1025f0 2024-06-10T02:47:41.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:47:41.343+0000 7f4a10b8c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4a0c102f90 0x7f4a0c19fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:47:41.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:47:41.343+0000 7f4a10b8c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4a0c102f90 0x7f4a0c19fcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60694/0 (socket says 172.21.15.52:60694) 2024-06-10T02:47:41.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:47:41.343+0000 7f4a10b8c700 1 -- 172.21.15.52:0/3379413109 learned_addr learned my addr 172.21.15.52:0/3379413109 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:47:41.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:47:41.343+0000 7f4a03fff700 1 --2- 172.21.15.52:0/3379413109 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4a0c1025f0 0x7f4a0c198c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:47:41.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:47:41.343+0000 7f4a0bfff700 1 --2- 172.21.15.52:0/3379413109 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4a0c1013f0 0x7f4a0c1986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:47:41.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:47:41.343+0000 7f4a10b8c700 1 -- 172.21.15.52:0/3379413109 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4a0c1025f0 msgr2=0x7f4a0c198c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:47:41.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:47:41.343+0000 7f4a10b8c700 1 --2- 172.21.15.52:0/3379413109 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4a0c1025f0 0x7f4a0c198c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:47:41.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:47:41.343+0000 7f4a10b8c700 1 -- 172.21.15.52:0/3379413109 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4a0c1013f0 msgr2=0x7f4a0c1986f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:47:41.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:47:41.343+0000 7f4a10b8c700 1 --2- 172.21.15.52:0/3379413109 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4a0c1013f0 0x7f4a0c1986f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:47:41.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:47:41.343+0000 7f4a10b8c700 1 -- 172.21.15.52:0/3379413109 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a0c1a0350 con 0x7f4a0c102f90 2024-06-10T02:47:41.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:47:41.343+0000 7f4a10b8c700 1 --2- 172.21.15.52:0/3379413109 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4a0c102f90 0x7f4a0c19fcb0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f4a04007f10 tx=0x7f4a0400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:47:41.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:47:41.343+0000 7f4a09ffb700 1 -- 172.21.15.52:0/3379413109 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a04004bb0 con 0x7f4a0c102f90 2024-06-10T02:47:41.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:47:41.343+0000 7f4a09ffb700 1 -- 172.21.15.52:0/3379413109 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a04004d50 con 0x7f4a0c102f90 2024-06-10T02:47:41.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:47:41.343+0000 7f4a09ffb700 1 -- 172.21.15.52:0/3379413109 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a040059f0 con 0x7f4a0c102f90 2024-06-10T02:47:41.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:47:41.343+0000 7f4a125ef700 1 -- 172.21.15.52:0/3379413109 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a0c103870 con 0x7f4a0c102f90 2024-06-10T02:47:41.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:47:41.343+0000 7f4a125ef700 1 -- 172.21.15.52:0/3379413109 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a0c103e00 con 0x7f4a0c102f90 2024-06-10T02:47:41.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:47:41.344+0000 7f4a125ef700 1 -- 172.21.15.52:0/3379413109 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a0c069180 con 0x7f4a0c102f90 2024-06-10T02:47:41.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:47:41.345+0000 7f4a09ffb700 1 -- 172.21.15.52:0/3379413109 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f4a0401a020 con 0x7f4a0c102f90 2024-06-10T02:47:41.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:47:41.346+0000 7f4a09ffb700 1 --2- 172.21.15.52:0/3379413109 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f49f80a2ca0 0x7f49f80a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:47:41.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:47:41.346+0000 7f4a0bfff700 1 --2- 172.21.15.52:0/3379413109 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f49f80a2ca0 0x7f49f80a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:47:41.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:47:41.346+0000 7f4a09ffb700 1 -- 172.21.15.52:0/3379413109 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5829+0+0 (secure 0 0 0) 0x7f4a040cdd10 con 0x7f4a0c102f90 2024-06-10T02:47:41.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:47:41.347+0000 7f4a0bfff700 1 --2- 172.21.15.52:0/3379413109 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f49f80a2ca0 0x7f49f80a5160 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f4a0c102450 tx=0x7f49fc00f820 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:47:41.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:47:41.351+0000 7f4a09ffb700 1 -- 172.21.15.52:0/3379413109 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f4a04010070 con 0x7f4a0c102f90 2024-06-10T02:47:41.386 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:47:41 smithi120 ceph-mon[36634]: osdmap e62: 9 total, 9 up, 9 in 2024-06-10T02:47:41.451 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:47:41 smithi188 ceph-mon[36599]: osdmap e62: 9 total, 9 up, 9 in 2024-06-10T02:47:41.566 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:47:41.569+0000 7f4a125ef700 1 -- 172.21.15.52:0/3379413109 --> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi052:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f4a0c0feb60 con 0x7f49f80a2ca0 2024-06-10T02:47:42.322 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:47:42 smithi052 ceph-mon[32362]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-06-10T02:47:42.322 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:47:42 smithi052 ceph-mon[32362]: osdmap e63: 9 total, 9 up, 9 in 2024-06-10T02:47:42.322 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:47:42 smithi052 ceph-mon[32362]: from='client.14400 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi052:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T02:47:42.322 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:47:42 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-10T02:47:42.322 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:47:42 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-10T02:47:42.322 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:47:42 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:47:42.386 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:47:42 smithi120 ceph-mon[36634]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-06-10T02:47:42.386 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:47:42 smithi120 ceph-mon[36634]: osdmap e63: 9 total, 9 up, 9 in 2024-06-10T02:47:42.386 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:47:42 smithi120 ceph-mon[36634]: from='client.14400 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi052:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T02:47:42.386 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:47:42 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-10T02:47:42.386 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:47:42 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-10T02:47:42.386 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:47:42 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:47:42.451 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:47:42 smithi188 ceph-mon[36599]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-06-10T02:47:42.451 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:47:42 smithi188 ceph-mon[36599]: osdmap e63: 9 total, 9 up, 9 in 2024-06-10T02:47:42.451 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:47:42 smithi188 ceph-mon[36599]: from='client.14400 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi052:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T02:47:42.452 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:47:42 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-10T02:47:42.452 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:47:42 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-10T02:47:42.452 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:47:42 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:47:43.297 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:47:43 smithi052 ceph-mon[32362]: osdmap e64: 9 total, 9 up, 9 in 2024-06-10T02:47:43.386 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:47:43 smithi120 ceph-mon[36634]: osdmap e64: 9 total, 9 up, 9 in 2024-06-10T02:47:43.451 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:47:43 smithi188 ceph-mon[36599]: osdmap e64: 9 total, 9 up, 9 in 2024-06-10T02:47:44.386 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:47:44 smithi120 ceph-mon[36634]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-06-10T02:47:44.451 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:47:44 smithi188 ceph-mon[36599]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-06-10T02:47:44.475 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:47:44 smithi052 ceph-mon[32362]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-06-10T02:47:45.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:47:45 smithi052 ceph-mon[32362]: from='client.? 172.21.15.52:0/1955624308' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5bfd985c-8acc-4d00-8561-40bbd541c07b"}]: dispatch 2024-06-10T02:47:45.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:47:45 smithi052 ceph-mon[32362]: from='client.? 172.21.15.52:0/1955624308' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5bfd985c-8acc-4d00-8561-40bbd541c07b"}]': finished 2024-06-10T02:47:45.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:47:45 smithi052 ceph-mon[32362]: osdmap e65: 10 total, 9 up, 10 in 2024-06-10T02:47:45.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:47:45 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-10T02:47:45.386 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:47:45 smithi120 ceph-mon[36634]: from='client.? 172.21.15.52:0/1955624308' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5bfd985c-8acc-4d00-8561-40bbd541c07b"}]: dispatch 2024-06-10T02:47:45.386 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:47:45 smithi120 ceph-mon[36634]: from='client.? 172.21.15.52:0/1955624308' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5bfd985c-8acc-4d00-8561-40bbd541c07b"}]': finished 2024-06-10T02:47:45.387 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:47:45 smithi120 ceph-mon[36634]: osdmap e65: 10 total, 9 up, 10 in 2024-06-10T02:47:45.387 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:47:45 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-10T02:47:45.452 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:47:45 smithi188 ceph-mon[36599]: from='client.? 172.21.15.52:0/1955624308' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5bfd985c-8acc-4d00-8561-40bbd541c07b"}]: dispatch 2024-06-10T02:47:45.452 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:47:45 smithi188 ceph-mon[36599]: from='client.? 172.21.15.52:0/1955624308' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5bfd985c-8acc-4d00-8561-40bbd541c07b"}]': finished 2024-06-10T02:47:45.452 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:47:45 smithi188 ceph-mon[36599]: osdmap e65: 10 total, 9 up, 10 in 2024-06-10T02:47:45.452 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:47:45 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-10T02:47:46.386 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:47:46 smithi120 ceph-mon[36634]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 117 KiB/s, 0 objects/s recovering 2024-06-10T02:47:46.386 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:47:46 smithi120 ceph-mon[36634]: from='client.? 172.21.15.52:0/1771039119' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-10T02:47:46.451 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:47:46 smithi188 ceph-mon[36599]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 117 KiB/s, 0 objects/s recovering 2024-06-10T02:47:46.451 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:47:46 smithi188 ceph-mon[36599]: from='client.? 172.21.15.52:0/1771039119' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-10T02:47:46.475 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:47:46 smithi052 ceph-mon[32362]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 117 KiB/s, 0 objects/s recovering 2024-06-10T02:47:46.475 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:47:46 smithi052 ceph-mon[32362]: from='client.? 172.21.15.52:0/1771039119' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-10T02:47:48.386 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:47:48 smithi120 ceph-mon[36634]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-06-10T02:47:48.451 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:47:48 smithi188 ceph-mon[36599]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-06-10T02:47:48.474 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:47:48 smithi052 ceph-mon[32362]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-06-10T02:47:50.224 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:47:50 smithi052 ceph-mon[32362]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-06-10T02:47:50.386 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:47:50 smithi120 ceph-mon[36634]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-06-10T02:47:50.451 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:47:50 smithi188 ceph-mon[36599]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-06-10T02:47:52.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:47:52 smithi052 ceph-mon[32362]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-06-10T02:47:52.386 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:47:52 smithi120 ceph-mon[36634]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-06-10T02:47:52.451 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:47:52 smithi188 ceph-mon[36599]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-06-10T02:47:53.198 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:47:53 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-06-10T02:47:53.198 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:47:53 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:47:53.198 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:47:53 smithi052 ceph-mon[32362]: Deploying daemon osd.9 on smithi052 2024-06-10T02:47:53.386 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:47:53 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-06-10T02:47:53.386 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:47:53 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:47:53.386 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:47:53 smithi120 ceph-mon[36634]: Deploying daemon osd.9 on smithi052 2024-06-10T02:47:53.451 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:47:53 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-06-10T02:47:53.451 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:47:53 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:47:53.451 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:47:53 smithi188 ceph-mon[36599]: Deploying daemon osd.9 on smithi052 2024-06-10T02:47:54.360 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:47:54 smithi052 ceph-mon[32362]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-06-10T02:47:54.386 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:47:54 smithi120 ceph-mon[36634]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-06-10T02:47:54.451 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:47:54 smithi188 ceph-mon[36599]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-06-10T02:47:56.386 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:47:56 smithi120 ceph-mon[36634]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-06-10T02:47:56.393 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:47:56 smithi052 ceph-mon[32362]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-06-10T02:47:56.451 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:47:56 smithi188 ceph-mon[36599]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-06-10T02:47:58.198 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:47:58 smithi052 ceph-mon[32362]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-06-10T02:47:58.386 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:47:58 smithi120 ceph-mon[36634]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-06-10T02:47:58.451 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:47:58 smithi188 ceph-mon[36599]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-06-10T02:47:59.701 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:47:59 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:47:59.702 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:47:59 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:47:59.702 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:47:59 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T02:47:59.724 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:47:59 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:47:59.725 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:47:59 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:47:59.725 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:47:59 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T02:47:59.886 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:47:59 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:47:59.886 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:47:59 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:47:59.887 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:47:59 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T02:48:00.701 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:48:00 smithi188 ceph-mon[36599]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-06-10T02:48:00.725 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:48:00 smithi052 ceph-mon[32362]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-06-10T02:48:00.886 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:48:00 smithi120 ceph-mon[36634]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-06-10T02:48:02.431 INFO:teuthology.orchestra.run.smithi052.stdout:Created osd(s) 9 on host 'smithi052' 2024-06-10T02:48:02.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:48:02.432+0000 7f4a09ffb700 1 -- 172.21.15.52:0/3379413109 <== mgr.14150 v2:172.21.15.52:6800/3214763969 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f4a0c0feb60 con 0x7f49f80a2ca0 2024-06-10T02:48:02.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:48:02.435+0000 7f4a125ef700 1 -- 172.21.15.52:0/3379413109 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f49f80a2ca0 msgr2=0x7f49f80a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:48:02.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:48:02.435+0000 7f4a125ef700 1 --2- 172.21.15.52:0/3379413109 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f49f80a2ca0 0x7f49f80a5160 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f4a0c102450 tx=0x7f49fc00f820 comp rx=0 tx=0).stop 2024-06-10T02:48:02.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:48:02.435+0000 7f4a125ef700 1 -- 172.21.15.52:0/3379413109 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4a0c102f90 msgr2=0x7f4a0c19fcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:48:02.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:48:02.435+0000 7f4a125ef700 1 --2- 172.21.15.52:0/3379413109 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4a0c102f90 0x7f4a0c19fcb0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f4a04007f10 tx=0x7f4a0400d3b0 comp rx=0 tx=0).stop 2024-06-10T02:48:02.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:48:02.435+0000 7f4a125ef700 1 -- 172.21.15.52:0/3379413109 shutdown_connections 2024-06-10T02:48:02.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:48:02.435+0000 7f4a125ef700 1 --2- 172.21.15.52:0/3379413109 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f49f80a2ca0 0x7f49f80a5160 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:48:02.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:48:02.435+0000 7f4a125ef700 1 --2- 172.21.15.52:0/3379413109 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4a0c102f90 0x7f4a0c19fcb0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:48:02.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:48:02.435+0000 7f4a125ef700 1 --2- 172.21.15.52:0/3379413109 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4a0c1013f0 0x7f4a0c1986f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:48:02.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:48:02.435+0000 7f4a125ef700 1 --2- 172.21.15.52:0/3379413109 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4a0c1025f0 0x7f4a0c198c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:48:02.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:48:02.435+0000 7f4a125ef700 1 -- 172.21.15.52:0/3379413109 >> 172.21.15.52:0/3379413109 conn(0x7f4a0c0fc990 msgr2=0x7f4a0c0fe450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:48:02.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:48:02.435+0000 7f4a125ef700 1 -- 172.21.15.52:0/3379413109 shutdown_connections 2024-06-10T02:48:02.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:48:02.435+0000 7f4a125ef700 1 -- 172.21.15.52:0/3379413109 wait complete. 2024-06-10T02:48:02.725 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:48:02 smithi052 ceph-mon[32362]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-06-10T02:48:02.725 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:48:02 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:48:02.725 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:48:02 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:48:02.725 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:48:02 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:48:02.725 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:48:02 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T02:48:02.725 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:48:02 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:48:02.725 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:48:02 smithi052 ceph-mon[32362]: from='osd.9 [v2:172.21.15.52:6826/924440658,v1:172.21.15.52:6827/924440658]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-06-10T02:48:02.725 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:48:02 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:48:02.725 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:48:02 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:48:02.725 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:48:02 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T02:48:02.886 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:48:02 smithi120 ceph-mon[36634]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-06-10T02:48:02.887 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:48:02 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:48:02.887 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:48:02 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:48:02.887 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:48:02 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:48:02.887 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:48:02 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T02:48:02.887 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:48:02 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:48:02.887 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:48:02 smithi120 ceph-mon[36634]: from='osd.9 [v2:172.21.15.52:6826/924440658,v1:172.21.15.52:6827/924440658]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-06-10T02:48:02.887 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:48:02 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:48:02.887 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:48:02 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:48:02.887 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:48:02 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T02:48:02.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:48:02 smithi188 ceph-mon[36599]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-06-10T02:48:02.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:48:02 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:48:02.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:48:02 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:48:02.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:48:02 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:48:02.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:48:02 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T02:48:02.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:48:02 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:48:02.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:48:02 smithi188 ceph-mon[36599]: from='osd.9 [v2:172.21.15.52:6826/924440658,v1:172.21.15.52:6827/924440658]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-06-10T02:48:02.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:48:02 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:48:02.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:48:02 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:48:02.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:48:02 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T02:48:03.131 DEBUG:teuthology.orchestra.run.smithi052:osd.9> sudo journalctl -f -n 0 -u ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968@osd.9.service 2024-06-10T02:48:03.134 INFO:tasks.cephadm:Deploying osd.10 on smithi120 with /dev/vg_nvme/lv_1... 2024-06-10T02:48:03.134 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-06-10T02:48:03.378 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.b/config 2024-06-10T02:48:03.725 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:48:03 smithi052 ceph-mon[32362]: from='osd.9 [v2:172.21.15.52:6826/924440658,v1:172.21.15.52:6827/924440658]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-06-10T02:48:03.725 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:48:03 smithi052 ceph-mon[32362]: osdmap e66: 10 total, 9 up, 10 in 2024-06-10T02:48:03.725 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:48:03 smithi052 ceph-mon[32362]: from='osd.9 [v2:172.21.15.52:6826/924440658,v1:172.21.15.52:6827/924440658]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]: dispatch 2024-06-10T02:48:03.725 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:48:03 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-10T02:48:03.825 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:48:03 smithi120 ceph-mon[36634]: from='osd.9 [v2:172.21.15.52:6826/924440658,v1:172.21.15.52:6827/924440658]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-06-10T02:48:03.825 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:48:03 smithi120 ceph-mon[36634]: osdmap e66: 10 total, 9 up, 10 in 2024-06-10T02:48:03.825 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:48:03 smithi120 ceph-mon[36634]: from='osd.9 [v2:172.21.15.52:6826/924440658,v1:172.21.15.52:6827/924440658]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]: dispatch 2024-06-10T02:48:03.825 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:48:03 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-10T02:48:03.951 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:48:03 smithi188 ceph-mon[36599]: from='osd.9 [v2:172.21.15.52:6826/924440658,v1:172.21.15.52:6827/924440658]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-06-10T02:48:03.951 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:48:03 smithi188 ceph-mon[36599]: osdmap e66: 10 total, 9 up, 10 in 2024-06-10T02:48:03.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:48:03 smithi188 ceph-mon[36599]: from='osd.9 [v2:172.21.15.52:6826/924440658,v1:172.21.15.52:6827/924440658]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]: dispatch 2024-06-10T02:48:03.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:48:03 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-10T02:48:04.225 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 02:48:04 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T02:48:04.044+0000 7f98742cf700 -1 osd.9 0 waiting for initial osdmap 2024-06-10T02:48:04.225 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 02:48:04 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T02:48:04.055+0000 7f986c0ba700 -1 osd.9 67 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-10T02:48:04.800 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:48:04 smithi120 ceph-mon[36634]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-06-10T02:48:04.800 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:48:04 smithi120 ceph-mon[36634]: from='osd.9 [v2:172.21.15.52:6826/924440658,v1:172.21.15.52:6827/924440658]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]': finished 2024-06-10T02:48:04.800 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:48:04 smithi120 ceph-mon[36634]: osdmap e67: 10 total, 9 up, 10 in 2024-06-10T02:48:04.800 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:48:04 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-10T02:48:04.800 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:48:04 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-10T02:48:04.800 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:48:04 smithi120 ceph-mon[36634]: from='osd.9 [v2:172.21.15.52:6826/924440658,v1:172.21.15.52:6827/924440658]' entity='osd.9' 2024-06-10T02:48:04.859 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:48:04 smithi052 ceph-mon[32362]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-06-10T02:48:04.859 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:48:04 smithi052 ceph-mon[32362]: from='osd.9 [v2:172.21.15.52:6826/924440658,v1:172.21.15.52:6827/924440658]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]': finished 2024-06-10T02:48:04.859 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:48:04 smithi052 ceph-mon[32362]: osdmap e67: 10 total, 9 up, 10 in 2024-06-10T02:48:04.859 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:48:04 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-10T02:48:04.859 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:48:04 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-10T02:48:04.859 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:48:04 smithi052 ceph-mon[32362]: from='osd.9 [v2:172.21.15.52:6826/924440658,v1:172.21.15.52:6827/924440658]' entity='osd.9' 2024-06-10T02:48:04.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:48:04 smithi188 ceph-mon[36599]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-06-10T02:48:04.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:48:04 smithi188 ceph-mon[36599]: from='osd.9 [v2:172.21.15.52:6826/924440658,v1:172.21.15.52:6827/924440658]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]': finished 2024-06-10T02:48:04.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:48:04 smithi188 ceph-mon[36599]: osdmap e67: 10 total, 9 up, 10 in 2024-06-10T02:48:04.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:48:04 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-10T02:48:04.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:48:04 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-10T02:48:04.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:48:04 smithi188 ceph-mon[36599]: from='osd.9 [v2:172.21.15.52:6826/924440658,v1:172.21.15.52:6827/924440658]' entity='osd.9' 2024-06-10T02:48:05.380 INFO:teuthology.orchestra.run.smithi120.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-06-10T02:48:05.380 INFO:teuthology.orchestra.run.smithi120.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-06-10T02:48:05.380 INFO:teuthology.orchestra.run.smithi120.stderr: stderr: 10+0 records in 2024-06-10T02:48:05.380 INFO:teuthology.orchestra.run.smithi120.stderr:10+0 records out 2024-06-10T02:48:05.380 INFO:teuthology.orchestra.run.smithi120.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0112399 s, 933 MB/s 2024-06-10T02:48:05.380 INFO:teuthology.orchestra.run.smithi120.stderr:--> Zapping successful for: 2024-06-10T02:48:05.725 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:48:05 smithi052 ceph-mon[32362]: purged_snaps scrub starts 2024-06-10T02:48:05.725 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:48:05 smithi052 ceph-mon[32362]: purged_snaps scrub ok 2024-06-10T02:48:05.725 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:48:05 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-10T02:48:05.726 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:48:05 smithi052 ceph-mon[32362]: osd.9 [v2:172.21.15.52:6826/924440658,v1:172.21.15.52:6827/924440658] boot 2024-06-10T02:48:05.726 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:48:05 smithi052 ceph-mon[32362]: osdmap e68: 10 total, 10 up, 10 in 2024-06-10T02:48:05.726 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:48:05 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-10T02:48:05.876 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:48:05 smithi120 ceph-mon[36634]: purged_snaps scrub starts 2024-06-10T02:48:05.876 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:48:05 smithi120 ceph-mon[36634]: purged_snaps scrub ok 2024-06-10T02:48:05.877 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:48:05 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-10T02:48:05.877 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:48:05 smithi120 ceph-mon[36634]: osd.9 [v2:172.21.15.52:6826/924440658,v1:172.21.15.52:6827/924440658] boot 2024-06-10T02:48:05.877 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:48:05 smithi120 ceph-mon[36634]: osdmap e68: 10 total, 10 up, 10 in 2024-06-10T02:48:05.877 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:48:05 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-10T02:48:05.951 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:48:05 smithi188 ceph-mon[36599]: purged_snaps scrub starts 2024-06-10T02:48:05.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:48:05 smithi188 ceph-mon[36599]: purged_snaps scrub ok 2024-06-10T02:48:05.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:48:05 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-10T02:48:05.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:48:05 smithi188 ceph-mon[36599]: osd.9 [v2:172.21.15.52:6826/924440658,v1:172.21.15.52:6827/924440658] boot 2024-06-10T02:48:05.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:48:05 smithi188 ceph-mon[36599]: osdmap e68: 10 total, 10 up, 10 in 2024-06-10T02:48:05.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:48:05 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-10T02:48:06.221 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi120:vg_nvme/lv_1 2024-06-10T02:48:06.460 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.b/config 2024-06-10T02:48:06.847 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:48:06 smithi120 ceph-mon[36634]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-06-10T02:48:06.847 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:48:06 smithi120 ceph-mon[36634]: osdmap e69: 10 total, 10 up, 10 in 2024-06-10T02:48:06.951 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:48:06 smithi188 ceph-mon[36599]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-06-10T02:48:06.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:48:06 smithi188 ceph-mon[36599]: osdmap e69: 10 total, 10 up, 10 in 2024-06-10T02:48:06.974 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:48:06 smithi052 ceph-mon[32362]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-06-10T02:48:06.975 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:48:06 smithi052 ceph-mon[32362]: osdmap e69: 10 total, 10 up, 10 in 2024-06-10T02:48:07.886 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:48:07 smithi120 ceph-mon[36634]: Detected new or changed devices on smithi052 2024-06-10T02:48:07.887 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:48:07 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:48:07.887 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:48:07 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:48:07.887 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:48:07 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-10T02:48:07.887 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:48:07 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-10T02:48:07.887 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:48:07 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-10T02:48:07.887 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:48:07 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-06-10T02:48:07.887 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:48:07 smithi120 ceph-mon[36634]: Adjusting osd_memory_target on smithi052 to 4273M 2024-06-10T02:48:07.887 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:48:07 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:48:07.887 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:48:07 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:48:07.887 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:48:07 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T02:48:07.888 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:48:07 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:48:08.165 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:48:08.164+0000 7f5af88da700 1 -- 172.21.15.120:0/2288581636 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5af41013d0 msgr2=0x7f5af41017f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:48:08.165 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:48:08.164+0000 7f5af88da700 1 --2- 172.21.15.120:0/2288581636 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5af41013d0 0x7f5af41017f0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f5ae40099c0 tx=0x7f5ae4009cd0 comp rx=0 tx=0).stop 2024-06-10T02:48:08.165 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:48:08.164+0000 7f5af88da700 1 -- 172.21.15.120:0/2288581636 shutdown_connections 2024-06-10T02:48:08.165 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:48:08.164+0000 7f5af88da700 1 --2- 172.21.15.120:0/2288581636 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5af4102f70 0x7f5af4109820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:48:08.165 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:48:08.164+0000 7f5af88da700 1 --2- 172.21.15.120:0/2288581636 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5af41013d0 0x7f5af41017f0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:48:08.165 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:48:08.164+0000 7f5af88da700 1 --2- 172.21.15.120:0/2288581636 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5af41025d0 0x7f5af4102a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:48:08.165 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:48:08.164+0000 7f5af88da700 1 -- 172.21.15.120:0/2288581636 >> 172.21.15.120:0/2288581636 conn(0x7f5af40fc990 msgr2=0x7f5af40fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:48:08.165 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:48:08.164+0000 7f5af88da700 1 -- 172.21.15.120:0/2288581636 shutdown_connections 2024-06-10T02:48:08.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:48:08.164+0000 7f5af88da700 1 -- 172.21.15.120:0/2288581636 wait complete. 2024-06-10T02:48:08.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:48:08.164+0000 7f5af88da700 1 Processor -- start 2024-06-10T02:48:08.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:48:08.165+0000 7f5af88da700 1 -- start start 2024-06-10T02:48:08.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:48:08.165+0000 7f5af88da700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5af41013d0 0x7f5af406b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:48:08.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:48:08.165+0000 7f5af88da700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5af41025d0 0x7f5af419d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:48:08.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:48:08.165+0000 7f5af88da700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5af4102f70 0x7f5af419fb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:48:08.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:48:08.165+0000 7f5af88da700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f5af410d010 con 0x7f5af4102f70 2024-06-10T02:48:08.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:48:08.165+0000 7f5af88da700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f5af410ce90 con 0x7f5af41025d0 2024-06-10T02:48:08.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:48:08.165+0000 7f5af88da700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f5af410d190 con 0x7f5af41013d0 2024-06-10T02:48:08.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:48:08.165+0000 7f5af1d9b700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5af41025d0 0x7f5af419d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:48:08.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:48:08.165+0000 7f5af1d9b700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5af41025d0 0x7f5af419d520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:53896/0 (socket says 172.21.15.120:53896) 2024-06-10T02:48:08.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:48:08.165+0000 7f5af1d9b700 1 -- 172.21.15.120:0/226208249 learned_addr learned my addr 172.21.15.120:0/226208249 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-06-10T02:48:08.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:48:08.165+0000 7f5af259c700 1 --2- 172.21.15.120:0/226208249 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5af41013d0 0x7f5af406b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:48:08.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:48:08.165+0000 7f5af1d9b700 1 -- 172.21.15.120:0/226208249 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5af41013d0 msgr2=0x7f5af406b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:48:08.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:48:08.165+0000 7f5af2d9d700 1 --2- 172.21.15.120:0/226208249 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5af4102f70 0x7f5af419fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:48:08.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:48:08.165+0000 7f5af1d9b700 1 --2- 172.21.15.120:0/226208249 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5af41013d0 0x7f5af406b740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:48:08.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:48:08.165+0000 7f5af1d9b700 1 -- 172.21.15.120:0/226208249 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5af4102f70 msgr2=0x7f5af419fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:48:08.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:48:08.165+0000 7f5af1d9b700 1 --2- 172.21.15.120:0/226208249 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5af4102f70 0x7f5af419fb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:48:08.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:48:08.165+0000 7f5af1d9b700 1 -- 172.21.15.120:0/226208249 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5af41a0430 con 0x7f5af41025d0 2024-06-10T02:48:08.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:48:08.165+0000 7f5af1d9b700 1 --2- 172.21.15.120:0/226208249 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5af41025d0 0x7f5af419d520 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f5ae800d980 tx=0x7f5ae800dd40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:48:08.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:48:08.165+0000 7f5ae37fe700 1 -- 172.21.15.120:0/226208249 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ae8016070 con 0x7f5af41025d0 2024-06-10T02:48:08.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:48:08.165+0000 7f5ae37fe700 1 -- 172.21.15.120:0/226208249 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ae8004500 con 0x7f5af41025d0 2024-06-10T02:48:08.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:48:08.165+0000 7f5af88da700 1 -- 172.21.15.120:0/226208249 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5af41a0720 con 0x7f5af41025d0 2024-06-10T02:48:08.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:48:08.166+0000 7f5ae37fe700 1 -- 172.21.15.120:0/226208249 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ae8004d00 con 0x7f5af41025d0 2024-06-10T02:48:08.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:48:08.166+0000 7f5af88da700 1 -- 172.21.15.120:0/226208249 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5af41a0c60 con 0x7f5af41025d0 2024-06-10T02:48:08.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:48:08.166+0000 7f5af88da700 1 -- 172.21.15.120:0/226208249 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5af4069180 con 0x7f5af41025d0 2024-06-10T02:48:08.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:48:08.168+0000 7f5ae37fe700 1 -- 172.21.15.120:0/226208249 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f5ae800bbf0 con 0x7f5af41025d0 2024-06-10T02:48:08.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:48:08.169+0000 7f5ae37fe700 1 --2- 172.21.15.120:0/226208249 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f5adc0a2c20 0x7f5adc0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:48:08.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:48:08.169+0000 7f5ae37fe700 1 -- 172.21.15.120:0/226208249 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6261+0+0 (secure 0 0 0) 0x7f5ae80cec50 con 0x7f5af41025d0 2024-06-10T02:48:08.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:48:08.169+0000 7f5af259c700 1 --2- 172.21.15.120:0/226208249 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f5adc0a2c20 0x7f5adc0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:48:08.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:48:08.170+0000 7f5af259c700 1 --2- 172.21.15.120:0/226208249 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f5adc0a2c20 0x7f5adc0a50e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f5ae40098e0 tx=0x7f5ae4000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:48:08.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:48:08.172+0000 7f5ae37fe700 1 -- 172.21.15.120:0/226208249 <== mon.2 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f5ae801f020 con 0x7f5af41025d0 2024-06-10T02:48:08.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:48:07 smithi188 ceph-mon[36599]: Detected new or changed devices on smithi052 2024-06-10T02:48:08.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:48:07 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:48:08.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:48:07 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:48:08.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:48:07 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-10T02:48:08.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:48:07 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-10T02:48:08.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:48:07 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-10T02:48:08.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:48:07 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-06-10T02:48:08.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:48:07 smithi188 ceph-mon[36599]: Adjusting osd_memory_target on smithi052 to 4273M 2024-06-10T02:48:08.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:48:07 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:48:08.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:48:07 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:48:08.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:48:07 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T02:48:08.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:48:07 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:48:08.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:48:07 smithi052 ceph-mon[32362]: Detected new or changed devices on smithi052 2024-06-10T02:48:08.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:48:07 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:48:08.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:48:07 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:48:08.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:48:07 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-10T02:48:08.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:48:07 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-10T02:48:08.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:48:07 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-10T02:48:08.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:48:07 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-06-10T02:48:08.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:48:07 smithi052 ceph-mon[32362]: Adjusting osd_memory_target on smithi052 to 4273M 2024-06-10T02:48:08.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:48:07 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:48:08.226 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:48:07 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:48:08.226 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:48:07 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T02:48:08.226 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:48:07 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:48:08.374 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:48:08.376+0000 7f5af88da700 1 -- 172.21.15.120:0/226208249 --> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi120:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f5af40628d0 con 0x7f5adc0a2c20 2024-06-10T02:48:09.033 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:48:08 smithi120 ceph-mon[36634]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-06-10T02:48:09.033 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:48:08 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-10T02:48:09.033 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:48:08 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-10T02:48:09.033 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:48:08 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:48:09.201 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:48:08 smithi188 ceph-mon[36599]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-06-10T02:48:09.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:48:08 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-10T02:48:09.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:48:08 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-10T02:48:09.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:48:08 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:48:09.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:48:08 smithi052 ceph-mon[32362]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-06-10T02:48:09.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:48:08 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-10T02:48:09.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:48:08 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-10T02:48:09.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:48:08 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:48:10.036 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:48:09 smithi120 ceph-mon[36634]: from='client.24332 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi120:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T02:48:10.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:48:09 smithi188 ceph-mon[36599]: from='client.24332 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi120:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T02:48:10.224 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:48:09 smithi052 ceph-mon[32362]: from='client.24332 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi120:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T02:48:10.886 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:48:10 smithi120 ceph-mon[36634]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-06-10T02:48:11.201 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:48:10 smithi188 ceph-mon[36599]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-06-10T02:48:11.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:48:10 smithi052 ceph-mon[32362]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-06-10T02:48:12.136 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:48:11 smithi120 ceph-mon[36634]: from='client.? 172.21.15.120:0/2395913322' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0180cddf-e109-42cc-829a-185604276f02"}]: dispatch 2024-06-10T02:48:12.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:48:11 smithi120 ceph-mon[36634]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0180cddf-e109-42cc-829a-185604276f02"}]: dispatch 2024-06-10T02:48:12.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:48:11 smithi120 ceph-mon[36634]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0180cddf-e109-42cc-829a-185604276f02"}]': finished 2024-06-10T02:48:12.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:48:11 smithi120 ceph-mon[36634]: osdmap e70: 11 total, 10 up, 11 in 2024-06-10T02:48:12.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:48:11 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-10T02:48:12.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:48:11 smithi188 ceph-mon[36599]: from='client.? 172.21.15.120:0/2395913322' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0180cddf-e109-42cc-829a-185604276f02"}]: dispatch 2024-06-10T02:48:12.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:48:11 smithi188 ceph-mon[36599]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0180cddf-e109-42cc-829a-185604276f02"}]: dispatch 2024-06-10T02:48:12.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:48:11 smithi188 ceph-mon[36599]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0180cddf-e109-42cc-829a-185604276f02"}]': finished 2024-06-10T02:48:12.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:48:11 smithi188 ceph-mon[36599]: osdmap e70: 11 total, 10 up, 11 in 2024-06-10T02:48:12.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:48:11 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-10T02:48:12.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:48:11 smithi052 ceph-mon[32362]: from='client.? 172.21.15.120:0/2395913322' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0180cddf-e109-42cc-829a-185604276f02"}]: dispatch 2024-06-10T02:48:12.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:48:11 smithi052 ceph-mon[32362]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0180cddf-e109-42cc-829a-185604276f02"}]: dispatch 2024-06-10T02:48:12.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:48:11 smithi052 ceph-mon[32362]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0180cddf-e109-42cc-829a-185604276f02"}]': finished 2024-06-10T02:48:12.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:48:11 smithi052 ceph-mon[32362]: osdmap e70: 11 total, 10 up, 11 in 2024-06-10T02:48:12.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:48:11 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-10T02:48:13.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:48:12 smithi120 ceph-mon[36634]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-06-10T02:48:13.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:48:12 smithi120 ceph-mon[36634]: from='client.? 172.21.15.120:0/3696789248' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-10T02:48:13.201 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:48:12 smithi188 ceph-mon[36599]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-06-10T02:48:13.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:48:12 smithi188 ceph-mon[36599]: from='client.? 172.21.15.120:0/3696789248' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-10T02:48:13.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:48:12 smithi052 ceph-mon[32362]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-06-10T02:48:13.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:48:12 smithi052 ceph-mon[32362]: from='client.? 172.21.15.120:0/3696789248' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-10T02:48:15.136 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:48:14 smithi120 ceph-mon[36634]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-06-10T02:48:15.201 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:48:14 smithi188 ceph-mon[36599]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-06-10T02:48:15.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:48:14 smithi052 ceph-mon[32362]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-06-10T02:48:16.886 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:48:16 smithi120 ceph-mon[36634]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-06-10T02:48:17.201 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:48:16 smithi188 ceph-mon[36599]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-06-10T02:48:17.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:48:16 smithi052 ceph-mon[32362]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-06-10T02:48:19.136 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:48:18 smithi120 ceph-mon[36634]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-06-10T02:48:19.201 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:48:18 smithi188 ceph-mon[36599]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-06-10T02:48:19.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:48:18 smithi052 ceph-mon[32362]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-06-10T02:48:20.136 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:48:19 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-06-10T02:48:20.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:48:19 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:48:20.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:48:19 smithi120 ceph-mon[36634]: Deploying daemon osd.10 on smithi120 2024-06-10T02:48:20.201 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:48:19 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-06-10T02:48:20.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:48:19 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:48:20.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:48:19 smithi188 ceph-mon[36599]: Deploying daemon osd.10 on smithi120 2024-06-10T02:48:20.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:48:19 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-06-10T02:48:20.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:48:19 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:48:20.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:48:19 smithi052 ceph-mon[32362]: Deploying daemon osd.10 on smithi120 2024-06-10T02:48:21.090 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:48:20 smithi120 ceph-mon[36634]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-06-10T02:48:21.201 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:48:20 smithi188 ceph-mon[36599]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-06-10T02:48:21.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:48:20 smithi052 ceph-mon[32362]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-06-10T02:48:23.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:48:22 smithi120 ceph-mon[36634]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-06-10T02:48:23.201 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:48:22 smithi188 ceph-mon[36599]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-06-10T02:48:23.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:48:22 smithi052 ceph-mon[32362]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-06-10T02:48:25.136 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:48:24 smithi120 ceph-mon[36634]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-06-10T02:48:25.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:48:24 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:48:25.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:48:24 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T02:48:25.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:48:24 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:48:25.201 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:48:24 smithi188 ceph-mon[36599]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-06-10T02:48:25.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:48:24 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:48:25.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:48:24 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T02:48:25.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:48:24 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:48:25.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:48:24 smithi052 ceph-mon[32362]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-06-10T02:48:25.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:48:24 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:48:25.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:48:24 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T02:48:25.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:48:24 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:48:27.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:48:26 smithi120 ceph-mon[36634]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-06-10T02:48:27.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:48:26 smithi188 ceph-mon[36599]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-06-10T02:48:27.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:48:26 smithi052 ceph-mon[32362]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-06-10T02:48:27.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:48:27.697+0000 7f5ae37fe700 1 -- 172.21.15.120:0/226208249 <== mgr.14150 v2:172.21.15.52:6800/3214763969 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f5af40628d0 con 0x7f5adc0a2c20 2024-06-10T02:48:27.696 INFO:teuthology.orchestra.run.smithi120.stdout:Created osd(s) 10 on host 'smithi120' 2024-06-10T02:48:27.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:48:27.700+0000 7f5af88da700 1 -- 172.21.15.120:0/226208249 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f5adc0a2c20 msgr2=0x7f5adc0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:48:27.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:48:27.700+0000 7f5af88da700 1 --2- 172.21.15.120:0/226208249 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f5adc0a2c20 0x7f5adc0a50e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f5ae40098e0 tx=0x7f5ae4000bc0 comp rx=0 tx=0).stop 2024-06-10T02:48:27.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:48:27.700+0000 7f5af88da700 1 -- 172.21.15.120:0/226208249 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5af41025d0 msgr2=0x7f5af419d520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:48:27.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:48:27.700+0000 7f5af88da700 1 --2- 172.21.15.120:0/226208249 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5af41025d0 0x7f5af419d520 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f5ae800d980 tx=0x7f5ae800dd40 comp rx=0 tx=0).stop 2024-06-10T02:48:27.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:48:27.700+0000 7f5af88da700 1 -- 172.21.15.120:0/226208249 shutdown_connections 2024-06-10T02:48:27.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:48:27.700+0000 7f5af88da700 1 --2- 172.21.15.120:0/226208249 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f5adc0a2c20 0x7f5adc0a50e0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:48:27.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:48:27.700+0000 7f5af88da700 1 --2- 172.21.15.120:0/226208249 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5af4102f70 0x7f5af419fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:48:27.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:48:27.700+0000 7f5af88da700 1 --2- 172.21.15.120:0/226208249 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5af41013d0 0x7f5af406b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:48:27.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:48:27.700+0000 7f5af88da700 1 --2- 172.21.15.120:0/226208249 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5af41025d0 0x7f5af419d520 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:48:27.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:48:27.700+0000 7f5af88da700 1 -- 172.21.15.120:0/226208249 >> 172.21.15.120:0/226208249 conn(0x7f5af40fc990 msgr2=0x7f5af41058f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:48:27.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:48:27.700+0000 7f5af88da700 1 -- 172.21.15.120:0/226208249 shutdown_connections 2024-06-10T02:48:27.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-10T02:48:27.700+0000 7f5af88da700 1 -- 172.21.15.120:0/226208249 wait complete. 2024-06-10T02:48:28.820 DEBUG:teuthology.orchestra.run.smithi120:osd.10> sudo journalctl -f -n 0 -u ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968@osd.10.service 2024-06-10T02:48:28.822 INFO:tasks.cephadm:Deploying osd.11 on smithi188 with /dev/vg_nvme/lv_1... 2024-06-10T02:48:28.822 DEBUG:teuthology.orchestra.run.smithi188:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-06-10T02:48:28.951 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:48:28 smithi188 ceph-mon[36599]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-06-10T02:48:28.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:48:28 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:48:28.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:48:28 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:48:28.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:48:28 smithi188 ceph-mon[36599]: from='osd.10 [v2:172.21.15.120:6824/1531769083,v1:172.21.15.120:6825/1531769083]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-06-10T02:48:28.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:48:28 smithi188 ceph-mon[36599]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-06-10T02:48:28.975 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:48:28 smithi052 ceph-mon[32362]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-06-10T02:48:28.975 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:48:28 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:48:28.975 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:48:28 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:48:28.975 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:48:28 smithi052 ceph-mon[32362]: from='osd.10 [v2:172.21.15.120:6824/1531769083,v1:172.21.15.120:6825/1531769083]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-06-10T02:48:28.975 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:48:28 smithi052 ceph-mon[32362]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-06-10T02:48:29.064 INFO:teuthology.orchestra.run.smithi188.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.c/config 2024-06-10T02:48:29.136 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:48:28 smithi120 ceph-mon[36634]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-06-10T02:48:29.136 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:48:28 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:48:29.136 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:48:28 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:48:29.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:48:28 smithi120 ceph-mon[36634]: from='osd.10 [v2:172.21.15.120:6824/1531769083,v1:172.21.15.120:6825/1531769083]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-06-10T02:48:29.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:48:28 smithi120 ceph-mon[36634]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-06-10T02:48:29.824 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:48:29 smithi120 ceph-mon[36634]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-06-10T02:48:29.824 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:48:29 smithi120 ceph-mon[36634]: osdmap e71: 11 total, 10 up, 11 in 2024-06-10T02:48:29.824 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:48:29 smithi120 ceph-mon[36634]: from='osd.10 [v2:172.21.15.120:6824/1531769083,v1:172.21.15.120:6825/1531769083]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]: dispatch 2024-06-10T02:48:29.825 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:48:29 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-10T02:48:29.825 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:48:29 smithi120 ceph-mon[36634]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]: dispatch 2024-06-10T02:48:29.825 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:48:29 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:48:29.825 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:48:29 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:48:30.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:48:29 smithi188 ceph-mon[36599]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-06-10T02:48:30.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:48:29 smithi188 ceph-mon[36599]: osdmap e71: 11 total, 10 up, 11 in 2024-06-10T02:48:30.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:48:29 smithi188 ceph-mon[36599]: from='osd.10 [v2:172.21.15.120:6824/1531769083,v1:172.21.15.120:6825/1531769083]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]: dispatch 2024-06-10T02:48:30.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:48:29 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-10T02:48:30.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:48:29 smithi188 ceph-mon[36599]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]: dispatch 2024-06-10T02:48:30.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:48:29 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:48:30.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:48:29 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:48:30.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:48:29 smithi052 ceph-mon[32362]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-06-10T02:48:30.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:48:29 smithi052 ceph-mon[32362]: osdmap e71: 11 total, 10 up, 11 in 2024-06-10T02:48:30.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:48:29 smithi052 ceph-mon[32362]: from='osd.10 [v2:172.21.15.120:6824/1531769083,v1:172.21.15.120:6825/1531769083]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]: dispatch 2024-06-10T02:48:30.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:48:29 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-10T02:48:30.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:48:29 smithi052 ceph-mon[32362]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]: dispatch 2024-06-10T02:48:30.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:48:29 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:48:30.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:48:29 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:48:30.591 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 02:48:30 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T02:48:30.224+0000 7ff995c79700 -1 osd.10 0 waiting for initial osdmap 2024-06-10T02:48:30.591 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 02:48:30 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T02:48:30.235+0000 7ff98da64700 -1 osd.10 72 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-10T02:48:30.886 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:48:30 smithi120 ceph-mon[36634]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-06-10T02:48:30.887 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:48:30 smithi120 ceph-mon[36634]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]': finished 2024-06-10T02:48:30.887 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:48:30 smithi120 ceph-mon[36634]: osdmap e72: 11 total, 10 up, 11 in 2024-06-10T02:48:30.887 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:48:30 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-10T02:48:30.887 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:48:30 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-10T02:48:30.887 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:48:30 smithi120 ceph-mon[36634]: from='osd.10 ' entity='osd.10' 2024-06-10T02:48:30.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:48:30 smithi188 ceph-mon[36599]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-06-10T02:48:30.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:48:30 smithi188 ceph-mon[36599]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]': finished 2024-06-10T02:48:30.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:48:30 smithi188 ceph-mon[36599]: osdmap e72: 11 total, 10 up, 11 in 2024-06-10T02:48:30.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:48:30 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-10T02:48:30.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:48:30 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-10T02:48:30.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:48:30 smithi188 ceph-mon[36599]: from='osd.10 ' entity='osd.10' 2024-06-10T02:48:31.176 INFO:teuthology.orchestra.run.smithi188.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-06-10T02:48:31.176 INFO:teuthology.orchestra.run.smithi188.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-06-10T02:48:31.176 INFO:teuthology.orchestra.run.smithi188.stderr: stderr: 10+0 records in 2024-06-10T02:48:31.176 INFO:teuthology.orchestra.run.smithi188.stderr:10+0 records out 2024-06-10T02:48:31.176 INFO:teuthology.orchestra.run.smithi188.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0123471 s, 849 MB/s 2024-06-10T02:48:31.176 INFO:teuthology.orchestra.run.smithi188.stderr:--> Zapping successful for: 2024-06-10T02:48:31.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:48:30 smithi052 ceph-mon[32362]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-06-10T02:48:31.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:48:30 smithi052 ceph-mon[32362]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]': finished 2024-06-10T02:48:31.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:48:30 smithi052 ceph-mon[32362]: osdmap e72: 11 total, 10 up, 11 in 2024-06-10T02:48:31.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:48:30 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-10T02:48:31.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:48:30 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-10T02:48:31.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:48:30 smithi052 ceph-mon[32362]: from='osd.10 ' entity='osd.10' 2024-06-10T02:48:31.842 DEBUG:teuthology.orchestra.run.smithi188:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi188:vg_nvme/lv_1 2024-06-10T02:48:32.086 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:48:31 smithi120 ceph-mon[36634]: purged_snaps scrub starts 2024-06-10T02:48:32.086 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:48:31 smithi120 ceph-mon[36634]: purged_snaps scrub ok 2024-06-10T02:48:32.086 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:48:31 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-10T02:48:32.086 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:48:31 smithi120 ceph-mon[36634]: osd.10 [v2:172.21.15.120:6824/1531769083,v1:172.21.15.120:6825/1531769083] boot 2024-06-10T02:48:32.086 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:48:31 smithi120 ceph-mon[36634]: osdmap e73: 11 total, 11 up, 11 in 2024-06-10T02:48:32.090 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:48:31 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-10T02:48:32.090 INFO:teuthology.orchestra.run.smithi188.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.c/config 2024-06-10T02:48:32.201 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:48:31 smithi188 ceph-mon[36599]: purged_snaps scrub starts 2024-06-10T02:48:32.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:48:31 smithi188 ceph-mon[36599]: purged_snaps scrub ok 2024-06-10T02:48:32.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:48:31 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-10T02:48:32.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:48:31 smithi188 ceph-mon[36599]: osd.10 [v2:172.21.15.120:6824/1531769083,v1:172.21.15.120:6825/1531769083] boot 2024-06-10T02:48:32.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:48:31 smithi188 ceph-mon[36599]: osdmap e73: 11 total, 11 up, 11 in 2024-06-10T02:48:32.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:48:31 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-10T02:48:32.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:48:31 smithi052 ceph-mon[32362]: purged_snaps scrub starts 2024-06-10T02:48:32.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:48:31 smithi052 ceph-mon[32362]: purged_snaps scrub ok 2024-06-10T02:48:32.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:48:31 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-10T02:48:32.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:48:31 smithi052 ceph-mon[32362]: osd.10 [v2:172.21.15.120:6824/1531769083,v1:172.21.15.120:6825/1531769083] boot 2024-06-10T02:48:32.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:48:31 smithi052 ceph-mon[32362]: osdmap e73: 11 total, 11 up, 11 in 2024-06-10T02:48:32.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:48:31 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-10T02:48:33.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:48:32 smithi120 ceph-mon[36634]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-06-10T02:48:33.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:48:32 smithi120 ceph-mon[36634]: osdmap e74: 11 total, 11 up, 11 in 2024-06-10T02:48:33.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:48:32 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:48:33.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:48:32 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:48:33.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:48:32 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-10T02:48:33.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:48:32 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-10T02:48:33.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:48:32 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-10T02:48:33.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:48:32 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-06-10T02:48:33.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:48:32 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:48:33.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:48:32 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:48:33.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:48:32 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T02:48:33.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:48:32 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:48:33.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:48:32 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T02:48:33.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:48:32 smithi188 ceph-mon[36599]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-06-10T02:48:33.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:48:32 smithi188 ceph-mon[36599]: osdmap e74: 11 total, 11 up, 11 in 2024-06-10T02:48:33.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:48:32 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:48:33.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:48:32 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:48:33.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:48:32 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-10T02:48:33.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:48:32 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-10T02:48:33.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:48:32 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-10T02:48:33.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:48:32 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-06-10T02:48:33.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:48:32 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:48:33.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:48:32 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:48:33.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:48:32 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T02:48:33.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:48:32 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:48:33.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:48:32 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T02:48:33.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:48:32 smithi052 ceph-mon[32362]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-06-10T02:48:33.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:48:32 smithi052 ceph-mon[32362]: osdmap e74: 11 total, 11 up, 11 in 2024-06-10T02:48:33.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:48:32 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:48:33.226 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:48:32 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:48:33.226 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:48:32 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-10T02:48:33.226 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:48:32 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-10T02:48:33.226 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:48:32 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-10T02:48:33.226 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:48:32 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-06-10T02:48:33.226 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:48:32 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:48:33.226 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:48:32 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:48:33.226 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:48:32 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T02:48:33.226 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:48:32 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:48:33.226 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:48:32 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T02:48:33.746 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:48:33.747+0000 7f4c612f3700 1 -- 172.21.15.188:0/3116073180 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c5c1024f0 msgr2=0x7f4c5c102910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:48:33.747 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:48:33.747+0000 7f4c612f3700 1 --2- 172.21.15.188:0/3116073180 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c5c1024f0 0x7f4c5c102910 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f4c4c009990 tx=0x7f4c4c009ca0 comp rx=0 tx=0).stop 2024-06-10T02:48:33.747 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:48:33.747+0000 7f4c612f3700 1 -- 172.21.15.188:0/3116073180 shutdown_connections 2024-06-10T02:48:33.747 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:48:33.747+0000 7f4c612f3700 1 --2- 172.21.15.188:0/3116073180 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4c5c104090 0x7f4c5c10a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:48:33.747 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:48:33.747+0000 7f4c612f3700 1 --2- 172.21.15.188:0/3116073180 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c5c1024f0 0x7f4c5c102910 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:48:33.747 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:48:33.747+0000 7f4c612f3700 1 --2- 172.21.15.188:0/3116073180 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4c5c1036f0 0x7f4c5c103b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:48:33.747 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:48:33.747+0000 7f4c612f3700 1 -- 172.21.15.188:0/3116073180 >> 172.21.15.188:0/3116073180 conn(0x7f4c5c0fda90 msgr2=0x7f4c5c0ffed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:48:33.747 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:48:33.747+0000 7f4c612f3700 1 -- 172.21.15.188:0/3116073180 shutdown_connections 2024-06-10T02:48:33.747 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:48:33.747+0000 7f4c612f3700 1 -- 172.21.15.188:0/3116073180 wait complete. 2024-06-10T02:48:33.747 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:48:33.747+0000 7f4c612f3700 1 Processor -- start 2024-06-10T02:48:33.747 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:48:33.747+0000 7f4c612f3700 1 -- start start 2024-06-10T02:48:33.747 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:48:33.748+0000 7f4c612f3700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4c5c1024f0 0x7f4c5c198620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:48:33.748 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:48:33.748+0000 7f4c612f3700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c5c1036f0 0x7f4c5c198b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:48:33.748 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:48:33.748+0000 7f4c612f3700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4c5c104090 0x7f4c5c19fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:48:33.748 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:48:33.748+0000 7f4c612f3700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f4c5c10e0d0 con 0x7f4c5c104090 2024-06-10T02:48:33.748 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:48:33.748+0000 7f4c612f3700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f4c5c10df50 con 0x7f4c5c1024f0 2024-06-10T02:48:33.748 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:48:33.748+0000 7f4c612f3700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f4c5c10e250 con 0x7f4c5c1036f0 2024-06-10T02:48:33.748 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:48:33.748+0000 7f4c5a7fc700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c5c1036f0 0x7f4c5c198b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:48:33.748 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:48:33.748+0000 7f4c5a7fc700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c5c1036f0 0x7f4c5c198b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.188:53912/0 (socket says 172.21.15.188:53912) 2024-06-10T02:48:33.748 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:48:33.748+0000 7f4c5a7fc700 1 -- 172.21.15.188:0/3438440331 learned_addr learned my addr 172.21.15.188:0/3438440331 (peer_addr_for_me v2:172.21.15.188:0/0) 2024-06-10T02:48:33.748 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:48:33.748+0000 7f4c5b7fe700 1 --2- 172.21.15.188:0/3438440331 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4c5c104090 0x7f4c5c19fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:48:33.748 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:48:33.748+0000 7f4c5affd700 1 --2- 172.21.15.188:0/3438440331 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4c5c1024f0 0x7f4c5c198620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:48:33.748 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:48:33.748+0000 7f4c5a7fc700 1 -- 172.21.15.188:0/3438440331 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4c5c1024f0 msgr2=0x7f4c5c198620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:48:33.748 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:48:33.748+0000 7f4c5a7fc700 1 --2- 172.21.15.188:0/3438440331 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4c5c1024f0 0x7f4c5c198620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:48:33.748 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:48:33.748+0000 7f4c5a7fc700 1 -- 172.21.15.188:0/3438440331 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4c5c104090 msgr2=0x7f4c5c19fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:48:33.749 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:48:33.748+0000 7f4c5a7fc700 1 --2- 172.21.15.188:0/3438440331 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4c5c104090 0x7f4c5c19fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:48:33.749 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:48:33.748+0000 7f4c5a7fc700 1 -- 172.21.15.188:0/3438440331 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c5c1a0280 con 0x7f4c5c1036f0 2024-06-10T02:48:33.749 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:48:33.748+0000 7f4c5affd700 1 --2- 172.21.15.188:0/3438440331 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4c5c1024f0 0x7f4c5c198620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T02:48:33.749 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:48:33.748+0000 7f4c5a7fc700 1 --2- 172.21.15.188:0/3438440331 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c5c1036f0 0x7f4c5c198b60 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f4c5000c9e0 tx=0x7f4c5000ccf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:48:33.749 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:48:33.748+0000 7f4c43fff700 1 -- 172.21.15.188:0/3438440331 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c50007ac0 con 0x7f4c5c1036f0 2024-06-10T02:48:33.749 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:48:33.748+0000 7f4c612f3700 1 -- 172.21.15.188:0/3438440331 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c5c1a0570 con 0x7f4c5c1036f0 2024-06-10T02:48:33.749 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:48:33.748+0000 7f4c43fff700 1 -- 172.21.15.188:0/3438440331 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c50007c60 con 0x7f4c5c1036f0 2024-06-10T02:48:33.749 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:48:33.748+0000 7f4c43fff700 1 -- 172.21.15.188:0/3438440331 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c50018730 con 0x7f4c5c1036f0 2024-06-10T02:48:33.749 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:48:33.748+0000 7f4c612f3700 1 -- 172.21.15.188:0/3438440331 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c5c1a0b00 con 0x7f4c5c1036f0 2024-06-10T02:48:33.750 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:48:33.749+0000 7f4c612f3700 1 -- 172.21.15.188:0/3438440331 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c5c069180 con 0x7f4c5c1036f0 2024-06-10T02:48:33.750 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:48:33.751+0000 7f4c43fff700 1 -- 172.21.15.188:0/3438440331 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f4c500188d0 con 0x7f4c5c1036f0 2024-06-10T02:48:33.750 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:48:33.751+0000 7f4c43fff700 1 --2- 172.21.15.188:0/3438440331 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f4c440a2c20 0x7f4c440a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:48:33.750 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:48:33.751+0000 7f4c43fff700 1 -- 172.21.15.188:0/3438440331 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 6694+0+0 (secure 0 0 0) 0x7f4c500cec70 con 0x7f4c5c1036f0 2024-06-10T02:48:33.750 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:48:33.751+0000 7f4c5affd700 1 --2- 172.21.15.188:0/3438440331 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f4c440a2c20 0x7f4c440a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:48:33.750 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:48:33.752+0000 7f4c5affd700 1 --2- 172.21.15.188:0/3438440331 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f4c440a2c20 0x7f4c440a50e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f4c4c0039c0 tx=0x7f4c4c005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:48:33.753 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:48:33.755+0000 7f4c43fff700 1 -- 172.21.15.188:0/3438440331 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f4c50014030 con 0x7f4c5c1036f0 2024-06-10T02:48:33.807 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:48:33 smithi188 ceph-mon[36599]: Detected new or changed devices on smithi120 2024-06-10T02:48:33.807 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:48:33 smithi188 ceph-mon[36599]: Adjusting osd_memory_target on smithi120 to 4272M 2024-06-10T02:48:33.807 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:48:33 smithi188 ceph-mon[36599]: osdmap e75: 11 total, 11 up, 11 in 2024-06-10T02:48:33.807 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:48:33 smithi188 ceph-mon[36599]: osdmap e76: 11 total, 11 up, 11 in 2024-06-10T02:48:33.807 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:48:33 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:48:33.807 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:48:33 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T02:48:33.807 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:48:33 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:48:33.963 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:48:33.965+0000 7f4c612f3700 1 -- 172.21.15.188:0/3438440331 --> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi188:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f4c5c0628d0 con 0x7f4c440a2c20 2024-06-10T02:48:34.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:48:33 smithi120 ceph-mon[36634]: Detected new or changed devices on smithi120 2024-06-10T02:48:34.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:48:33 smithi120 ceph-mon[36634]: Adjusting osd_memory_target on smithi120 to 4272M 2024-06-10T02:48:34.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:48:33 smithi120 ceph-mon[36634]: osdmap e75: 11 total, 11 up, 11 in 2024-06-10T02:48:34.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:48:33 smithi120 ceph-mon[36634]: osdmap e76: 11 total, 11 up, 11 in 2024-06-10T02:48:34.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:48:33 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:48:34.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:48:33 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T02:48:34.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:48:33 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:48:34.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:48:33 smithi052 ceph-mon[32362]: Detected new or changed devices on smithi120 2024-06-10T02:48:34.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:48:33 smithi052 ceph-mon[32362]: Adjusting osd_memory_target on smithi120 to 4272M 2024-06-10T02:48:34.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:48:33 smithi052 ceph-mon[32362]: osdmap e75: 11 total, 11 up, 11 in 2024-06-10T02:48:34.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:48:33 smithi052 ceph-mon[32362]: osdmap e76: 11 total, 11 up, 11 in 2024-06-10T02:48:34.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:48:33 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:48:34.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:48:33 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T02:48:34.226 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:48:33 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:48:34.862 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:48:34 smithi188 ceph-mon[36599]: pgmap v240: 1 pgs: 1 remapped+peering; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-06-10T02:48:34.862 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:48:34 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-10T02:48:34.862 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:48:34 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-10T02:48:34.862 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:48:34 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:48:34.862 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:48:34 smithi188 ceph-mon[36599]: osdmap e77: 11 total, 11 up, 11 in 2024-06-10T02:48:35.126 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:48:34 smithi052 ceph-mon[32362]: pgmap v240: 1 pgs: 1 remapped+peering; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-06-10T02:48:35.127 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:48:34 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-10T02:48:35.127 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:48:34 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-10T02:48:35.127 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:48:34 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:48:35.127 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:48:34 smithi052 ceph-mon[32362]: osdmap e77: 11 total, 11 up, 11 in 2024-06-10T02:48:35.136 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:48:34 smithi120 ceph-mon[36634]: pgmap v240: 1 pgs: 1 remapped+peering; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-06-10T02:48:35.136 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:48:34 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-10T02:48:35.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:48:34 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-10T02:48:35.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:48:34 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:48:35.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:48:34 smithi120 ceph-mon[36634]: osdmap e77: 11 total, 11 up, 11 in 2024-06-10T02:48:36.051 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:48:35 smithi188 ceph-mon[36599]: from='client.24379 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi188:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T02:48:36.136 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:48:35 smithi120 ceph-mon[36634]: from='client.24379 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi188:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T02:48:36.139 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:48:35 smithi052 ceph-mon[32362]: from='client.24379 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi188:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T02:48:36.886 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:48:36 smithi120 ceph-mon[36634]: pgmap v243: 1 pgs: 1 remapped+peering; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-06-10T02:48:37.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:48:36 smithi188 ceph-mon[36599]: pgmap v243: 1 pgs: 1 remapped+peering; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-06-10T02:48:37.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:48:36 smithi052 ceph-mon[32362]: pgmap v243: 1 pgs: 1 remapped+peering; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-06-10T02:48:38.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:48:37 smithi120 ceph-mon[36634]: from='client.? 172.21.15.188:0/60193542' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "73aec750-82d1-4800-b6a0-4e305912f9ee"}]: dispatch 2024-06-10T02:48:38.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:48:37 smithi120 ceph-mon[36634]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "73aec750-82d1-4800-b6a0-4e305912f9ee"}]: dispatch 2024-06-10T02:48:38.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:48:37 smithi120 ceph-mon[36634]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "73aec750-82d1-4800-b6a0-4e305912f9ee"}]': finished 2024-06-10T02:48:38.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:48:37 smithi120 ceph-mon[36634]: osdmap e78: 12 total, 11 up, 12 in 2024-06-10T02:48:38.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:48:37 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-10T02:48:38.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:48:37 smithi120 ceph-mon[36634]: from='client.? 172.21.15.188:0/2946769253' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-10T02:48:38.201 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:48:37 smithi188 ceph-mon[36599]: from='client.? 172.21.15.188:0/60193542' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "73aec750-82d1-4800-b6a0-4e305912f9ee"}]: dispatch 2024-06-10T02:48:38.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:48:37 smithi188 ceph-mon[36599]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "73aec750-82d1-4800-b6a0-4e305912f9ee"}]: dispatch 2024-06-10T02:48:38.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:48:37 smithi188 ceph-mon[36599]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "73aec750-82d1-4800-b6a0-4e305912f9ee"}]': finished 2024-06-10T02:48:38.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:48:37 smithi188 ceph-mon[36599]: osdmap e78: 12 total, 11 up, 12 in 2024-06-10T02:48:38.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:48:37 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-10T02:48:38.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:48:37 smithi188 ceph-mon[36599]: from='client.? 172.21.15.188:0/2946769253' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-10T02:48:38.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:48:37 smithi052 ceph-mon[32362]: from='client.? 172.21.15.188:0/60193542' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "73aec750-82d1-4800-b6a0-4e305912f9ee"}]: dispatch 2024-06-10T02:48:38.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:48:37 smithi052 ceph-mon[32362]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "73aec750-82d1-4800-b6a0-4e305912f9ee"}]: dispatch 2024-06-10T02:48:38.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:48:37 smithi052 ceph-mon[32362]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "73aec750-82d1-4800-b6a0-4e305912f9ee"}]': finished 2024-06-10T02:48:38.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:48:37 smithi052 ceph-mon[32362]: osdmap e78: 12 total, 11 up, 12 in 2024-06-10T02:48:38.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:48:37 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-10T02:48:38.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:48:37 smithi052 ceph-mon[32362]: from='client.? 172.21.15.188:0/2946769253' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-10T02:48:39.136 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:48:38 smithi120 ceph-mon[36634]: pgmap v245: 1 pgs: 1 remapped+peering; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-06-10T02:48:39.201 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:48:38 smithi188 ceph-mon[36599]: pgmap v245: 1 pgs: 1 remapped+peering; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-06-10T02:48:39.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:48:38 smithi052 ceph-mon[32362]: pgmap v245: 1 pgs: 1 remapped+peering; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-06-10T02:48:41.136 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:48:40 smithi120 ceph-mon[36634]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-06-10T02:48:41.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:48:40 smithi188 ceph-mon[36599]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-06-10T02:48:41.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:48:40 smithi052 ceph-mon[32362]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-06-10T02:48:42.136 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:48:41 smithi120 ceph-mon[36634]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-06-10T02:48:42.161 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:48:41 smithi188 ceph-mon[36599]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-06-10T02:48:42.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:48:41 smithi052 ceph-mon[32362]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-06-10T02:48:44.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:48:43 smithi188 ceph-mon[36599]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-06-10T02:48:44.386 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:48:43 smithi120 ceph-mon[36634]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-06-10T02:48:44.475 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:48:43 smithi052 ceph-mon[32362]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-06-10T02:48:45.257 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:48:44 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-06-10T02:48:45.257 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:48:44 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:48:45.257 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:48:44 smithi188 ceph-mon[36599]: Deploying daemon osd.11 on smithi188 2024-06-10T02:48:45.386 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:48:44 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-06-10T02:48:45.387 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:48:44 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:48:45.387 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:48:44 smithi120 ceph-mon[36634]: Deploying daemon osd.11 on smithi188 2024-06-10T02:48:45.475 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:48:44 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-06-10T02:48:45.475 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:48:44 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:48:45.475 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:48:44 smithi052 ceph-mon[32362]: Deploying daemon osd.11 on smithi188 2024-06-10T02:48:46.260 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:48:46 smithi188 ceph-mon[36599]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-06-10T02:48:46.386 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:48:46 smithi120 ceph-mon[36634]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-06-10T02:48:46.475 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:48:46 smithi052 ceph-mon[32362]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-06-10T02:48:48.139 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:48:48 smithi188 ceph-mon[36599]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 57 KiB/s, 0 objects/s recovering 2024-06-10T02:48:48.386 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:48:48 smithi120 ceph-mon[36634]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 57 KiB/s, 0 objects/s recovering 2024-06-10T02:48:48.475 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:48:48 smithi052 ceph-mon[32362]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 57 KiB/s, 0 objects/s recovering 2024-06-10T02:48:50.339 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:48:50 smithi188 ceph-mon[36599]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-06-10T02:48:50.386 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:48:50 smithi120 ceph-mon[36634]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-06-10T02:48:50.475 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:48:50 smithi052 ceph-mon[32362]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-06-10T02:48:51.887 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:48:51 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:48:51.887 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:48:51 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T02:48:51.887 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:48:51 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:48:51.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:48:51 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:48:51.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:48:51 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T02:48:51.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:48:51 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:48:51.975 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:48:51 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:48:51.975 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:48:51 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T02:48:51.975 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:48:51 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:48:52.886 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:48:52 smithi120 ceph-mon[36634]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-06-10T02:48:52.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:48:52 smithi188 ceph-mon[36599]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-06-10T02:48:52.975 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:48:52 smithi052 ceph-mon[32362]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-06-10T02:48:54.633 INFO:teuthology.orchestra.run.smithi188.stdout:Created osd(s) 11 on host 'smithi188' 2024-06-10T02:48:54.633 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:48:54.634+0000 7f4c43fff700 1 -- 172.21.15.188:0/3438440331 <== mgr.14150 v2:172.21.15.52:6800/3214763969 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f4c5c0628d0 con 0x7f4c440a2c20 2024-06-10T02:48:54.636 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:48:54.637+0000 7f4c612f3700 1 -- 172.21.15.188:0/3438440331 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f4c440a2c20 msgr2=0x7f4c440a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:48:54.636 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:48:54.637+0000 7f4c612f3700 1 --2- 172.21.15.188:0/3438440331 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f4c440a2c20 0x7f4c440a50e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f4c4c0039c0 tx=0x7f4c4c005fb0 comp rx=0 tx=0).stop 2024-06-10T02:48:54.637 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:48:54.638+0000 7f4c612f3700 1 -- 172.21.15.188:0/3438440331 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c5c1036f0 msgr2=0x7f4c5c198b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:48:54.637 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:48:54.638+0000 7f4c612f3700 1 --2- 172.21.15.188:0/3438440331 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c5c1036f0 0x7f4c5c198b60 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f4c5000c9e0 tx=0x7f4c5000ccf0 comp rx=0 tx=0).stop 2024-06-10T02:48:54.637 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:48:54.638+0000 7f4c612f3700 1 -- 172.21.15.188:0/3438440331 shutdown_connections 2024-06-10T02:48:54.637 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:48:54.638+0000 7f4c612f3700 1 --2- 172.21.15.188:0/3438440331 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f4c440a2c20 0x7f4c440a50e0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:48:54.637 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:48:54.638+0000 7f4c612f3700 1 --2- 172.21.15.188:0/3438440331 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4c5c104090 0x7f4c5c19fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:48:54.637 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:48:54.638+0000 7f4c612f3700 1 --2- 172.21.15.188:0/3438440331 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4c5c1024f0 0x7f4c5c198620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:48:54.637 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:48:54.638+0000 7f4c612f3700 1 --2- 172.21.15.188:0/3438440331 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c5c1036f0 0x7f4c5c198b60 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:48:54.637 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:48:54.638+0000 7f4c612f3700 1 -- 172.21.15.188:0/3438440331 >> 172.21.15.188:0/3438440331 conn(0x7f4c5c0fda90 msgr2=0x7f4c5c1069f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:48:54.637 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:48:54.638+0000 7f4c612f3700 1 -- 172.21.15.188:0/3438440331 shutdown_connections 2024-06-10T02:48:54.637 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:48:54.638+0000 7f4c612f3700 1 -- 172.21.15.188:0/3438440331 wait complete. 2024-06-10T02:48:54.887 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:48:54 smithi120 ceph-mon[36634]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-06-10T02:48:54.887 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:48:54 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:48:54.887 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:48:54 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:48:54.887 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:48:54 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:48:54.887 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:48:54 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T02:48:54.887 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:48:54 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:48:54.887 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:48:54 smithi120 ceph-mon[36634]: from='osd.11 [v2:172.21.15.188:6824/3769050170,v1:172.21.15.188:6825/3769050170]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-06-10T02:48:54.887 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:48:54 smithi120 ceph-mon[36634]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-06-10T02:48:54.887 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:48:54 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:48:54.887 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:48:54 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:48:54.888 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:48:54 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T02:48:54.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:48:54 smithi188 ceph-mon[36599]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-06-10T02:48:54.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:48:54 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:48:54.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:48:54 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:48:54.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:48:54 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:48:54.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:48:54 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T02:48:54.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:48:54 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:48:54.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:48:54 smithi188 ceph-mon[36599]: from='osd.11 [v2:172.21.15.188:6824/3769050170,v1:172.21.15.188:6825/3769050170]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-06-10T02:48:54.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:48:54 smithi188 ceph-mon[36599]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-06-10T02:48:54.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:48:54 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:48:54.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:48:54 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:48:54.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:48:54 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T02:48:54.975 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:48:54 smithi052 ceph-mon[32362]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-06-10T02:48:54.975 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:48:54 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:48:54.975 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:48:54 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:48:54.975 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:48:54 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:48:54.976 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:48:54 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T02:48:54.976 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:48:54 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:48:54.976 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:48:54 smithi052 ceph-mon[32362]: from='osd.11 [v2:172.21.15.188:6824/3769050170,v1:172.21.15.188:6825/3769050170]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-06-10T02:48:54.976 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:48:54 smithi052 ceph-mon[32362]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-06-10T02:48:54.976 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:48:54 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:48:54.976 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:48:54 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:48:54.976 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:48:54 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T02:48:55.302 DEBUG:teuthology.orchestra.run.smithi188:osd.11> sudo journalctl -f -n 0 -u ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968@osd.11.service 2024-06-10T02:48:55.304 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-06-10T02:48:55.305 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph osd stat -f json 2024-06-10T02:48:55.566 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:48:55.940 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:48:55 smithi052 ceph-mon[32362]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-06-10T02:48:55.940 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:48:55 smithi052 ceph-mon[32362]: osdmap e79: 12 total, 11 up, 12 in 2024-06-10T02:48:55.940 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:48:55 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-10T02:48:55.940 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:48:55 smithi052 ceph-mon[32362]: from='osd.11 [v2:172.21.15.188:6824/3769050170,v1:172.21.15.188:6825/3769050170]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]: dispatch 2024-06-10T02:48:55.940 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:48:55 smithi052 ceph-mon[32362]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]: dispatch 2024-06-10T02:48:55.940 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:48:55 smithi052 ceph-mon[32362]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-06-10T02:48:55.951 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:48:55 smithi188 ceph-mon[36599]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-06-10T02:48:55.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:48:55 smithi188 ceph-mon[36599]: osdmap e79: 12 total, 11 up, 12 in 2024-06-10T02:48:55.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:48:55 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-10T02:48:55.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:48:55 smithi188 ceph-mon[36599]: from='osd.11 [v2:172.21.15.188:6824/3769050170,v1:172.21.15.188:6825/3769050170]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]: dispatch 2024-06-10T02:48:55.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:48:55 smithi188 ceph-mon[36599]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]: dispatch 2024-06-10T02:48:55.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:48:55 smithi188 ceph-mon[36599]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-06-10T02:48:56.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:48:55 smithi120 ceph-mon[36634]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-06-10T02:48:56.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:48:55 smithi120 ceph-mon[36634]: osdmap e79: 12 total, 11 up, 12 in 2024-06-10T02:48:56.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:48:55 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-10T02:48:56.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:48:55 smithi120 ceph-mon[36634]: from='osd.11 [v2:172.21.15.188:6824/3769050170,v1:172.21.15.188:6825/3769050170]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]: dispatch 2024-06-10T02:48:56.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:48:55 smithi120 ceph-mon[36634]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]: dispatch 2024-06-10T02:48:56.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:48:55 smithi120 ceph-mon[36634]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-06-10T02:48:56.452 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 02:48:56 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T02:48:56.294+0000 7f1e29200700 -1 osd.11 0 waiting for initial osdmap 2024-06-10T02:48:56.452 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 02:48:56 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T02:48:56.307+0000 7f1e207ea700 -1 osd.11 80 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-10T02:48:56.975 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:48:56 smithi052 ceph-mon[32362]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]': finished 2024-06-10T02:48:56.975 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:48:56 smithi052 ceph-mon[32362]: osdmap e80: 12 total, 11 up, 12 in 2024-06-10T02:48:56.975 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:48:56 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-10T02:48:56.976 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:48:56 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-10T02:48:56.976 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:48:56 smithi052 ceph-mon[32362]: from='osd.11 ' entity='osd.11' 2024-06-10T02:48:57.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:48:56 smithi120 ceph-mon[36634]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]': finished 2024-06-10T02:48:57.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:48:56 smithi120 ceph-mon[36634]: osdmap e80: 12 total, 11 up, 12 in 2024-06-10T02:48:57.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:48:56 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-10T02:48:57.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:48:56 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-10T02:48:57.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:48:56 smithi120 ceph-mon[36634]: from='osd.11 ' entity='osd.11' 2024-06-10T02:48:57.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:48:56 smithi188 ceph-mon[36599]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]': finished 2024-06-10T02:48:57.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:48:56 smithi188 ceph-mon[36599]: osdmap e80: 12 total, 11 up, 12 in 2024-06-10T02:48:57.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:48:56 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-10T02:48:57.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:48:56 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-10T02:48:57.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:48:56 smithi188 ceph-mon[36599]: from='osd.11 ' entity='osd.11' 2024-06-10T02:48:57.619 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:48:57.619+0000 7f8c15653700 1 -- 172.21.15.52:0/3777368930 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8c101025e0 msgr2=0x7f8c10102a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:48:57.620 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:48:57.619+0000 7f8c15653700 1 --2- 172.21.15.52:0/3777368930 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8c101025e0 0x7f8c10102a40 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f8bf80099c0 tx=0x7f8bf8009cd0 comp rx=0 tx=0).stop 2024-06-10T02:48:57.620 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:48:57.619+0000 7f8c15653700 1 -- 172.21.15.52:0/3777368930 shutdown_connections 2024-06-10T02:48:57.620 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:48:57.619+0000 7f8c15653700 1 --2- 172.21.15.52:0/3777368930 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8c10102f80 0x7f8c10109830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:48:57.620 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:48:57.619+0000 7f8c15653700 1 --2- 172.21.15.52:0/3777368930 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8c101013e0 0x7f8c10101800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:48:57.620 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:48:57.619+0000 7f8c15653700 1 --2- 172.21.15.52:0/3777368930 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8c101025e0 0x7f8c10102a40 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:48:57.620 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:48:57.619+0000 7f8c15653700 1 -- 172.21.15.52:0/3777368930 >> 172.21.15.52:0/3777368930 conn(0x7f8c100fc980 msgr2=0x7f8c100fedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:48:57.620 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:48:57.619+0000 7f8c15653700 1 -- 172.21.15.52:0/3777368930 shutdown_connections 2024-06-10T02:48:57.620 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:48:57.619+0000 7f8c15653700 1 -- 172.21.15.52:0/3777368930 wait complete. 2024-06-10T02:48:57.620 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:48:57.620+0000 7f8c15653700 1 Processor -- start 2024-06-10T02:48:57.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:48:57.620+0000 7f8c15653700 1 -- start start 2024-06-10T02:48:57.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:48:57.620+0000 7f8c15653700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8c101013e0 0x7f8c1006b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:48:57.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:48:57.620+0000 7f8c15653700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8c101025e0 0x7f8c1019d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:48:57.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:48:57.620+0000 7f8c15653700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8c10102f80 0x7f8c1019fa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:48:57.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:48:57.620+0000 7f8c15653700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f8c1010cfd0 con 0x7f8c10102f80 2024-06-10T02:48:57.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:48:57.620+0000 7f8c15653700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f8c1010ce50 con 0x7f8c101013e0 2024-06-10T02:48:57.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:48:57.620+0000 7f8c15653700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f8c1010d150 con 0x7f8c101025e0 2024-06-10T02:48:57.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:48:57.620+0000 7f8c0e7fc700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8c101025e0 0x7f8c1019d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:48:57.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:48:57.620+0000 7f8c0e7fc700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8c101025e0 0x7f8c1019d420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.52:55286/0 (socket says 172.21.15.52:55286) 2024-06-10T02:48:57.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:48:57.620+0000 7f8c0e7fc700 1 -- 172.21.15.52:0/2818513796 learned_addr learned my addr 172.21.15.52:0/2818513796 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:48:57.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:48:57.620+0000 7f8c0f7fe700 1 --2- 172.21.15.52:0/2818513796 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8c10102f80 0x7f8c1019fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:48:57.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:48:57.621+0000 7f8c0effd700 1 --2- 172.21.15.52:0/2818513796 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8c101013e0 0x7f8c1006b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:48:57.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:48:57.621+0000 7f8c0e7fc700 1 -- 172.21.15.52:0/2818513796 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8c101013e0 msgr2=0x7f8c1006b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:48:57.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:48:57.621+0000 7f8c0e7fc700 1 --2- 172.21.15.52:0/2818513796 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8c101013e0 0x7f8c1006b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:48:57.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:48:57.621+0000 7f8c0e7fc700 1 -- 172.21.15.52:0/2818513796 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8c10102f80 msgr2=0x7f8c1019fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:48:57.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:48:57.621+0000 7f8c0e7fc700 1 --2- 172.21.15.52:0/2818513796 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8c10102f80 0x7f8c1019fa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:48:57.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:48:57.621+0000 7f8c0e7fc700 1 -- 172.21.15.52:0/2818513796 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c101a0330 con 0x7f8c101025e0 2024-06-10T02:48:57.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:48:57.621+0000 7f8c0e7fc700 1 --2- 172.21.15.52:0/2818513796 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8c101025e0 0x7f8c1019d420 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f8bf8005910 tx=0x7f8bf8009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:48:57.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:48:57.621+0000 7f8c07fff700 1 -- 172.21.15.52:0/2818513796 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bf8027070 con 0x7f8c101025e0 2024-06-10T02:48:57.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:48:57.621+0000 7f8c07fff700 1 -- 172.21.15.52:0/2818513796 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8bf8004150 con 0x7f8c101025e0 2024-06-10T02:48:57.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:48:57.621+0000 7f8c15653700 1 -- 172.21.15.52:0/2818513796 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c101971f0 con 0x7f8c101025e0 2024-06-10T02:48:57.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:48:57.621+0000 7f8c07fff700 1 -- 172.21.15.52:0/2818513796 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bf8022da0 con 0x7f8c101025e0 2024-06-10T02:48:57.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:48:57.621+0000 7f8c15653700 1 -- 172.21.15.52:0/2818513796 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c101976d0 con 0x7f8c101025e0 2024-06-10T02:48:57.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:48:57.622+0000 7f8c15653700 1 -- 172.21.15.52:0/2818513796 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c10069180 con 0x7f8c101025e0 2024-06-10T02:48:57.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:48:57.624+0000 7f8c07fff700 1 -- 172.21.15.52:0/2818513796 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f8bf802a070 con 0x7f8c101025e0 2024-06-10T02:48:57.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:48:57.624+0000 7f8c07fff700 1 --2- 172.21.15.52:0/2818513796 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f8bfc0a2c20 0x7f8bfc0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:48:57.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:48:57.624+0000 7f8c07fff700 1 -- 172.21.15.52:0/2818513796 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f8bf80cfe10 con 0x7f8c101025e0 2024-06-10T02:48:57.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:48:57.624+0000 7f8c0effd700 1 --2- 172.21.15.52:0/2818513796 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f8bfc0a2c20 0x7f8bfc0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:48:57.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:48:57.625+0000 7f8c0effd700 1 --2- 172.21.15.52:0/2818513796 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f8bfc0a2c20 0x7f8bfc0a50e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f8c10102440 tx=0x7f8c0000f820 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:48:57.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:48:57.628+0000 7f8c07fff700 1 -- 172.21.15.52:0/2818513796 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f8bf8031020 con 0x7f8c101025e0 2024-06-10T02:48:57.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:48:57.837+0000 7f8c15653700 1 -- 172.21.15.52:0/2818513796 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f8c1002dca0 con 0x7f8c101025e0 2024-06-10T02:48:57.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:48:57.838+0000 7f8c07fff700 1 -- 172.21.15.52:0/2818513796 <== mon.1 v2:172.21.15.188:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v81) ==== 74+0+133 (secure 0 0 0) 0x7f8bf8013070 con 0x7f8c101025e0 2024-06-10T02:48:57.836 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-10T02:48:57.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:48:57.840+0000 7f8c15653700 1 -- 172.21.15.52:0/2818513796 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f8bfc0a2c20 msgr2=0x7f8bfc0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:48:57.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:48:57.840+0000 7f8c15653700 1 --2- 172.21.15.52:0/2818513796 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f8bfc0a2c20 0x7f8bfc0a50e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f8c10102440 tx=0x7f8c0000f820 comp rx=0 tx=0).stop 2024-06-10T02:48:57.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:48:57.840+0000 7f8c15653700 1 -- 172.21.15.52:0/2818513796 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8c101025e0 msgr2=0x7f8c1019d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:48:57.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:48:57.840+0000 7f8c15653700 1 --2- 172.21.15.52:0/2818513796 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8c101025e0 0x7f8c1019d420 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f8bf8005910 tx=0x7f8bf8009580 comp rx=0 tx=0).stop 2024-06-10T02:48:57.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:48:57.840+0000 7f8c15653700 1 -- 172.21.15.52:0/2818513796 shutdown_connections 2024-06-10T02:48:57.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:48:57.840+0000 7f8c15653700 1 --2- 172.21.15.52:0/2818513796 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f8bfc0a2c20 0x7f8bfc0a50e0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:48:57.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:48:57.840+0000 7f8c15653700 1 --2- 172.21.15.52:0/2818513796 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8c10102f80 0x7f8c1019fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:48:57.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:48:57.840+0000 7f8c15653700 1 --2- 172.21.15.52:0/2818513796 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8c101013e0 0x7f8c1006b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:48:57.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:48:57.840+0000 7f8c15653700 1 --2- 172.21.15.52:0/2818513796 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8c101025e0 0x7f8c1019d420 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:48:57.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:48:57.840+0000 7f8c15653700 1 -- 172.21.15.52:0/2818513796 >> 172.21.15.52:0/2818513796 conn(0x7f8c100fc980 msgr2=0x7f8c10105960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:48:57.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:48:57.840+0000 7f8c15653700 1 -- 172.21.15.52:0/2818513796 shutdown_connections 2024-06-10T02:48:57.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:48:57.840+0000 7f8c15653700 1 -- 172.21.15.52:0/2818513796 wait complete. 2024-06-10T02:48:58.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:48:57 smithi120 ceph-mon[36634]: purged_snaps scrub starts 2024-06-10T02:48:58.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:48:57 smithi120 ceph-mon[36634]: purged_snaps scrub ok 2024-06-10T02:48:58.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:48:57 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-10T02:48:58.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:48:57 smithi120 ceph-mon[36634]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-06-10T02:48:58.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:48:57 smithi120 ceph-mon[36634]: osd.11 [v2:172.21.15.188:6824/3769050170,v1:172.21.15.188:6825/3769050170] boot 2024-06-10T02:48:58.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:48:57 smithi120 ceph-mon[36634]: osdmap e81: 12 total, 12 up, 12 in 2024-06-10T02:48:58.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:48:57 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-10T02:48:58.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:48:57 smithi120 ceph-mon[36634]: from='client.? 172.21.15.52:0/2818513796' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-06-10T02:48:58.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:48:57 smithi188 ceph-mon[36599]: purged_snaps scrub starts 2024-06-10T02:48:58.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:48:57 smithi188 ceph-mon[36599]: purged_snaps scrub ok 2024-06-10T02:48:58.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:48:57 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-10T02:48:58.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:48:57 smithi188 ceph-mon[36599]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-06-10T02:48:58.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:48:57 smithi188 ceph-mon[36599]: osd.11 [v2:172.21.15.188:6824/3769050170,v1:172.21.15.188:6825/3769050170] boot 2024-06-10T02:48:58.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:48:57 smithi188 ceph-mon[36599]: osdmap e81: 12 total, 12 up, 12 in 2024-06-10T02:48:58.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:48:57 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-10T02:48:58.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:48:57 smithi188 ceph-mon[36599]: from='client.? 172.21.15.52:0/2818513796' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-06-10T02:48:58.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:48:57 smithi052 ceph-mon[32362]: purged_snaps scrub starts 2024-06-10T02:48:58.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:48:57 smithi052 ceph-mon[32362]: purged_snaps scrub ok 2024-06-10T02:48:58.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:48:57 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-10T02:48:58.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:48:57 smithi052 ceph-mon[32362]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-06-10T02:48:58.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:48:57 smithi052 ceph-mon[32362]: osd.11 [v2:172.21.15.188:6824/3769050170,v1:172.21.15.188:6825/3769050170] boot 2024-06-10T02:48:58.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:48:57 smithi052 ceph-mon[32362]: osdmap e81: 12 total, 12 up, 12 in 2024-06-10T02:48:58.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:48:57 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-10T02:48:58.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:48:57 smithi052 ceph-mon[32362]: from='client.? 172.21.15.52:0/2818513796' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-06-10T02:48:58.632 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":81,"num_osds":12,"num_up_osds":12,"osd_up_since":1717987737,"num_in_osds":12,"osd_in_since":1717987716,"num_remapped_pgs":0} 2024-06-10T02:48:58.633 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-10T02:48:58.881 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:48:59.637 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:48:59 smithi120 ceph-mon[36634]: Detected new or changed devices on smithi188 2024-06-10T02:48:59.637 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:48:59 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:48:59.637 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:48:59 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:48:59.637 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:48:59 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-10T02:48:59.637 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:48:59 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-10T02:48:59.637 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:48:59 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-10T02:48:59.637 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:48:59 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-06-10T02:48:59.637 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:48:59 smithi120 ceph-mon[36634]: Adjusting osd_memory_target on smithi188 to 4272M 2024-06-10T02:48:59.638 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:48:59 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:48:59.638 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:48:59 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:48:59.638 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:48:59 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T02:48:59.638 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:48:59 smithi120 ceph-mon[36634]: osdmap e82: 12 total, 12 up, 12 in 2024-06-10T02:48:59.638 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:48:59 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:48:59.702 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:48:59 smithi188 ceph-mon[36599]: Detected new or changed devices on smithi188 2024-06-10T02:48:59.702 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:48:59 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:48:59.702 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:48:59 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:48:59.702 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:48:59 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-10T02:48:59.702 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:48:59 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-10T02:48:59.702 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:48:59 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-10T02:48:59.702 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:48:59 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-06-10T02:48:59.702 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:48:59 smithi188 ceph-mon[36599]: Adjusting osd_memory_target on smithi188 to 4272M 2024-06-10T02:48:59.702 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:48:59 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:48:59.703 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:48:59 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:48:59.703 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:48:59 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T02:48:59.703 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:48:59 smithi188 ceph-mon[36599]: osdmap e82: 12 total, 12 up, 12 in 2024-06-10T02:48:59.703 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:48:59 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:48:59.725 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:48:59 smithi052 ceph-mon[32362]: Detected new or changed devices on smithi188 2024-06-10T02:48:59.725 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:48:59 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:48:59.725 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:48:59 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:48:59.726 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:48:59 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-10T02:48:59.726 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:48:59 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-10T02:48:59.726 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:48:59 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-10T02:48:59.726 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:48:59 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-06-10T02:48:59.726 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:48:59 smithi052 ceph-mon[32362]: Adjusting osd_memory_target on smithi188 to 4272M 2024-06-10T02:48:59.726 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:48:59 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:48:59.726 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:48:59 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:48:59.726 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:48:59 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T02:48:59.726 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:48:59 smithi052 ceph-mon[32362]: osdmap e82: 12 total, 12 up, 12 in 2024-06-10T02:48:59.726 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:48:59 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:49:00.361 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:00 smithi052 ceph-mon[32362]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T02:49:00.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:00.573+0000 7ff8e8ae5700 1 -- 172.21.15.52:0/2019830388 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff8e406b620 msgr2=0x7ff8e410f350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:49:00.572 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:00.573+0000 7ff8e8ae5700 1 --2- 172.21.15.52:0/2019830388 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff8e406b620 0x7ff8e410f350 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7ff8dc00b0d0 tx=0x7ff8dc00b3e0 comp rx=0 tx=0).stop 2024-06-10T02:49:00.572 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:00.573+0000 7ff8e8ae5700 1 -- 172.21.15.52:0/2019830388 shutdown_connections 2024-06-10T02:49:00.572 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:00.573+0000 7ff8e8ae5700 1 --2- 172.21.15.52:0/2019830388 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff8e406b620 0x7ff8e410f350 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:49:00.572 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:00.573+0000 7ff8e8ae5700 1 --2- 172.21.15.52:0/2019830388 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff8e4108580 0x7ff8e4108960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:49:00.572 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:00.573+0000 7ff8e8ae5700 1 --2- 172.21.15.52:0/2019830388 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff8e406ac60 0x7ff8e406b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:49:00.572 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:00.573+0000 7ff8e8ae5700 1 -- 172.21.15.52:0/2019830388 >> 172.21.15.52:0/2019830388 conn(0x7ff8e4078430 msgr2=0x7ff8e4078840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:49:00.572 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:00.573+0000 7ff8e8ae5700 1 -- 172.21.15.52:0/2019830388 shutdown_connections 2024-06-10T02:49:00.572 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:00.573+0000 7ff8e8ae5700 1 -- 172.21.15.52:0/2019830388 wait complete. 2024-06-10T02:49:00.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:00.574+0000 7ff8e8ae5700 1 Processor -- start 2024-06-10T02:49:00.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:00.574+0000 7ff8e8ae5700 1 -- start start 2024-06-10T02:49:00.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:00.574+0000 7ff8e8ae5700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff8e406ac60 0x7ff8e419e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:49:00.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:00.574+0000 7ff8e8ae5700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff8e406b620 0x7ff8e419ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:49:00.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:00.574+0000 7ff8e8ae5700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff8e4108580 0x7ff8e4198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:49:00.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:00.574+0000 7ff8e8ae5700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ff8e41127c0 con 0x7ff8e4108580 2024-06-10T02:49:00.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:00.574+0000 7ff8e8ae5700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff8e4112640 con 0x7ff8e406b620 2024-06-10T02:49:00.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:00.574+0000 7ff8e8ae5700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ff8e4112940 con 0x7ff8e406ac60 2024-06-10T02:49:00.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:00.574+0000 7ff8e2d9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff8e4108580 0x7ff8e4198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:49:00.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:00.574+0000 7ff8e2d9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff8e4108580 0x7ff8e4198920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53490/0 (socket says 172.21.15.52:53490) 2024-06-10T02:49:00.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:00.574+0000 7ff8e2d9d700 1 -- 172.21.15.52:0/3945449327 learned_addr learned my addr 172.21.15.52:0/3945449327 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:49:00.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:00.575+0000 7ff8e259c700 1 --2- 172.21.15.52:0/3945449327 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff8e406ac60 0x7ff8e419e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:49:00.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:00.575+0000 7ff8d9d9b700 1 --2- 172.21.15.52:0/3945449327 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff8e406b620 0x7ff8e419ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:49:00.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:00.575+0000 7ff8e2d9d700 1 -- 172.21.15.52:0/3945449327 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff8e406ac60 msgr2=0x7ff8e419e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:49:00.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:00.575+0000 7ff8e2d9d700 1 --2- 172.21.15.52:0/3945449327 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff8e406ac60 0x7ff8e419e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:49:00.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:00.575+0000 7ff8e2d9d700 1 -- 172.21.15.52:0/3945449327 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff8e406b620 msgr2=0x7ff8e419ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:49:00.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:00.575+0000 7ff8e2d9d700 1 --2- 172.21.15.52:0/3945449327 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff8e406b620 0x7ff8e419ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:49:00.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:00.575+0000 7ff8e2d9d700 1 -- 172.21.15.52:0/3945449327 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8e4199080 con 0x7ff8e4108580 2024-06-10T02:49:00.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:00.575+0000 7ff8e2d9d700 1 --2- 172.21.15.52:0/3945449327 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff8e4108580 0x7ff8e4198920 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7ff8dc006070 tx=0x7ff8dc004ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:49:00.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:00.575+0000 7ff8dbfff700 1 -- 172.21.15.52:0/3945449327 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8dc021070 con 0x7ff8e4108580 2024-06-10T02:49:00.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:00.575+0000 7ff8e8ae5700 1 -- 172.21.15.52:0/3945449327 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8e4199310 con 0x7ff8e4108580 2024-06-10T02:49:00.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:00.575+0000 7ff8dbfff700 1 -- 172.21.15.52:0/3945449327 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff8dc007df0 con 0x7ff8e4108580 2024-06-10T02:49:00.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:00.575+0000 7ff8dbfff700 1 -- 172.21.15.52:0/3945449327 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8dc005470 con 0x7ff8e4108580 2024-06-10T02:49:00.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:00.575+0000 7ff8e8ae5700 1 -- 172.21.15.52:0/3945449327 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8e41a55e0 con 0x7ff8e4108580 2024-06-10T02:49:00.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:00.576+0000 7ff8e8ae5700 1 -- 172.21.15.52:0/3945449327 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8e4069180 con 0x7ff8e4108580 2024-06-10T02:49:00.578 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:00.577+0000 7ff8dbfff700 1 -- 172.21.15.52:0/3945449327 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7ff8dc024070 con 0x7ff8e4108580 2024-06-10T02:49:00.578 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:00.578+0000 7ff8dbfff700 1 --2- 172.21.15.52:0/3945449327 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7ff8d00a2c20 0x7ff8d00a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:49:00.578 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:00.578+0000 7ff8e259c700 1 --2- 172.21.15.52:0/3945449327 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7ff8d00a2c20 0x7ff8d00a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:49:00.578 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:00.578+0000 7ff8dbfff700 1 -- 172.21.15.52:0/3945449327 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7ff8dc00f020 con 0x7ff8e4108580 2024-06-10T02:49:00.578 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:00.578+0000 7ff8e259c700 1 --2- 172.21.15.52:0/3945449327 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7ff8d00a2c20 0x7ff8d00a50e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ff8d4005ea0 tx=0x7ff8d4005e30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:49:00.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:00.583+0000 7ff8dbfff700 1 -- 172.21.15.52:0/3945449327 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7ff8dc02b020 con 0x7ff8e4108580 2024-06-10T02:49:00.636 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:00 smithi120 ceph-mon[36634]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T02:49:00.701 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:00 smithi188 ceph-mon[36599]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T02:49:00.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:00.793+0000 7ff8e8ae5700 1 -- 172.21.15.52:0/3945449327 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff8e41a57b0 con 0x7ff8e4108580 2024-06-10T02:49:00.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:00.794+0000 7ff8dbfff700 1 -- 172.21.15.52:0/3945449327 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+18456 (secure 0 0 0) 0x7ff8dc0074e0 con 0x7ff8e4108580 2024-06-10T02:49:00.793 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-10T02:49:00.793 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":82,"fsid":"d8676e9e-26d2-11ef-bc9b-c7b262605968","created":"2024-06-10T02:41:43.076837+0000","modified":"2024-06-10T02:48:58.299163+0000","last_up_change":"2024-06-10T02:48:57.297829+0000","last_in_change":"2024-06-10T02:48:36.865802+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-10T02:45:07.394690+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e03c0691-cc5a-497a-bbc2-ce18c337f074","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":4255833408},{"type":"v1","addr":"172.21.15.52:6803","nonce":4255833408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":4255833408},{"type":"v1","addr":"172.21.15.52:6805","nonce":4255833408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":4255833408},{"type":"v1","addr":"172.21.15.52:6809","nonce":4255833408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":4255833408},{"type":"v1","addr":"172.21.15.52:6807","nonce":4255833408}]},"public_addr":"172.21.15.52:6803/4255833408","cluster_addr":"172.21.15.52:6805/4255833408","heartbeat_back_addr":"172.21.15.52:6809/4255833408","heartbeat_front_addr":"172.21.15.52:6807/4255833408","state":["exists","up"]},{"osd":1,"uuid":"a24d8da8-c5ec-44d7-ab30-56f6ce3efda7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":4116540053},{"type":"v1","addr":"172.21.15.120:6801","nonce":4116540053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":4116540053},{"type":"v1","addr":"172.21.15.120:6803","nonce":4116540053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":4116540053},{"type":"v1","addr":"172.21.15.120:6807","nonce":4116540053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":4116540053},{"type":"v1","addr":"172.21.15.120:6805","nonce":4116540053}]},"public_addr":"172.21.15.120:6801/4116540053","cluster_addr":"172.21.15.120:6803/4116540053","heartbeat_back_addr":"172.21.15.120:6807/4116540053","heartbeat_front_addr":"172.21.15.120:6805/4116540053","state":["exists","up"]},{"osd":2,"uuid":"a7162106-ccbf-45bd-bc0e-051ee43f4214","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":2624326999},{"type":"v1","addr":"172.21.15.188:6801","nonce":2624326999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":2624326999},{"type":"v1","addr":"172.21.15.188:6803","nonce":2624326999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":2624326999},{"type":"v1","addr":"172.21.15.188:6807","nonce":2624326999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":2624326999},{"type":"v1","addr":"172.21.15.188:6805","nonce":2624326999}]},"public_addr":"172.21.15.188:6801/2624326999","cluster_addr":"172.21.15.188:6803/2624326999","heartbeat_back_addr":"172.21.15.188:6807/2624326999","heartbeat_front_addr":"172.21.15.188:6805/2624326999","state":["exists","up"]},{"osd":3,"uuid":"53680244-b49a-4120-b24a-9ad7a6341dc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":116897011},{"type":"v1","addr":"172.21.15.52:6811","nonce":116897011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":116897011},{"type":"v1","addr":"172.21.15.52:6813","nonce":116897011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":116897011},{"type":"v1","addr":"172.21.15.52:6817","nonce":116897011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":116897011},{"type":"v1","addr":"172.21.15.52:6815","nonce":116897011}]},"public_addr":"172.21.15.52:6811/116897011","cluster_addr":"172.21.15.52:6813/116897011","heartbeat_back_addr":"172.21.15.52:6817/116897011","heartbeat_front_addr":"172.21.15.52:6815/116897011","state":["exists","up"]},{"osd":4,"uuid":"9f72f1f4-2ef3-49c5-8410-c108767d2ef5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":1320690621},{"type":"v1","addr":"172.21.15.120:6809","nonce":1320690621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":1320690621},{"type":"v1","addr":"172.21.15.120:6811","nonce":1320690621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":1320690621},{"type":"v1","addr":"172.21.15.120:6815","nonce":1320690621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":1320690621},{"type":"v1","addr":"172.21.15.120:6813","nonce":1320690621}]},"public_addr":"172.21.15.120:6809/1320690621","cluster_addr":"172.21.15.120:6811/1320690621","heartbeat_back_addr":"172.21.15.120:6815/1320690621","heartbeat_front_addr":"172.21.15.120:6813/1320690621","state":["exists","up"]},{"osd":5,"uuid":"001b5706-7f27-4fa4-9b69-969e6d765abc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3951947483},{"type":"v1","addr":"172.21.15.188:6809","nonce":3951947483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3951947483},{"type":"v1","addr":"172.21.15.188:6811","nonce":3951947483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3951947483},{"type":"v1","addr":"172.21.15.188:6815","nonce":3951947483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3951947483},{"type":"v1","addr":"172.21.15.188:6813","nonce":3951947483}]},"public_addr":"172.21.15.188:6809/3951947483","cluster_addr":"172.21.15.188:6811/3951947483","heartbeat_back_addr":"172.21.15.188:6815/3951947483","heartbeat_front_addr":"172.21.15.188:6813/3951947483","state":["exists","up"]},{"osd":6,"uuid":"140c903e-9b31-4a81-9ccd-2edc0a2decf9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":359084919},{"type":"v1","addr":"172.21.15.52:6819","nonce":359084919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":359084919},{"type":"v1","addr":"172.21.15.52:6821","nonce":359084919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":359084919},{"type":"v1","addr":"172.21.15.52:6825","nonce":359084919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":359084919},{"type":"v1","addr":"172.21.15.52:6823","nonce":359084919}]},"public_addr":"172.21.15.52:6819/359084919","cluster_addr":"172.21.15.52:6821/359084919","heartbeat_back_addr":"172.21.15.52:6825/359084919","heartbeat_front_addr":"172.21.15.52:6823/359084919","state":["exists","up"]},{"osd":7,"uuid":"57b735d4-5072-4ca2-a3ff-3763ad84f15a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":3737722011},{"type":"v1","addr":"172.21.15.120:6817","nonce":3737722011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6818","nonce":3737722011},{"type":"v1","addr":"172.21.15.120:6819","nonce":3737722011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":3737722011},{"type":"v1","addr":"172.21.15.120:6823","nonce":3737722011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":3737722011},{"type":"v1","addr":"172.21.15.120:6821","nonce":3737722011}]},"public_addr":"172.21.15.120:6817/3737722011","cluster_addr":"172.21.15.120:6819/3737722011","heartbeat_back_addr":"172.21.15.120:6823/3737722011","heartbeat_front_addr":"172.21.15.120:6821/3737722011","state":["exists","up"]},{"osd":8,"uuid":"4ffbb43b-854a-4eac-99cf-7af89bc65409","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1444828441},{"type":"v1","addr":"172.21.15.188:6817","nonce":1444828441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1444828441},{"type":"v1","addr":"172.21.15.188:6819","nonce":1444828441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1444828441},{"type":"v1","addr":"172.21.15.188:6823","nonce":1444828441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1444828441},{"type":"v1","addr":"172.21.15.188:6821","nonce":1444828441}]},"public_addr":"172.21.15.188:6817/1444828441","cluster_addr":"172.21.15.188:6819/1444828441","heartbeat_back_addr":"172.21.15.188:6823/1444828441","heartbeat_front_addr":"172.21.15.188:6821/1444828441","state":["exists","up"]},{"osd":9,"uuid":"5bfd985c-8acc-4d00-8561-40bbd541c07b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":924440658},{"type":"v1","addr":"172.21.15.52:6827","nonce":924440658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":924440658},{"type":"v1","addr":"172.21.15.52:6829","nonce":924440658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":924440658},{"type":"v1","addr":"172.21.15.52:6833","nonce":924440658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":924440658},{"type":"v1","addr":"172.21.15.52:6831","nonce":924440658}]},"public_addr":"172.21.15.52:6827/924440658","cluster_addr":"172.21.15.52:6829/924440658","heartbeat_back_addr":"172.21.15.52:6833/924440658","heartbeat_front_addr":"172.21.15.52:6831/924440658","state":["exists","up"]},{"osd":10,"uuid":"0180cddf-e109-42cc-829a-185604276f02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6824","nonce":1531769083},{"type":"v1","addr":"172.21.15.120:6825","nonce":1531769083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":1531769083},{"type":"v1","addr":"172.21.15.120:6827","nonce":1531769083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":1531769083},{"type":"v1","addr":"172.21.15.120:6831","nonce":1531769083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6828","nonce":1531769083},{"type":"v1","addr":"172.21.15.120:6829","nonce":1531769083}]},"public_addr":"172.21.15.120:6825/1531769083","cluster_addr":"172.21.15.120:6827/1531769083","heartbeat_back_addr":"172.21.15.120:6831/1531769083","heartbeat_front_addr":"172.21.15.120:6829/1531769083","state":["exists","up"]},{"osd":11,"uuid":"73aec750-82d1-4800-b6a0-4e305912f9ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":3769050170},{"type":"v1","addr":"172.21.15.188:6825","nonce":3769050170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":3769050170},{"type":"v1","addr":"172.21.15.188:6827","nonce":3769050170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":3769050170},{"type":"v1","addr":"172.21.15.188:6831","nonce":3769050170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":3769050170},{"type":"v1","addr":"172.21.15.188:6829","nonce":3769050170}]},"public_addr":"172.21.15.188:6825/3769050170","cluster_addr":"172.21.15.188:6827/3769050170","heartbeat_back_addr":"172.21.15.188:6831/3769050170","heartbeat_front_addr":"172.21.15.188:6829/3769050170","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:44:16.455752+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:44:40.219735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:45:03.558899+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:45:29.155004+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:45:53.981098+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:46:19.412020+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:46:45.462640+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:47:11.484578+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:47:36.879054+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:48:03.392446+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:48:29.422173+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:48:55.523423+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.52:0/3535271671":"2024-06-11T02:42:42.872312+0000","172.21.15.52:0/1092934380":"2024-06-11T02:42:42.872312+0000","172.21.15.52:6800/896317057":"2024-06-11T02:42:42.872312+0000","172.21.15.52:6801/3934479642":"2024-06-11T02:42:15.498118+0000","172.21.15.52:6800/3934479642":"2024-06-11T02:42:15.498118+0000","172.21.15.52:0/2759694132":"2024-06-11T02:42:15.498118+0000","172.21.15.52:0/2593665631":"2024-06-11T02:42:15.498118+0000","172.21.15.52:0/3151504634":"2024-06-11T02:42:15.498118+0000","172.21.15.52:6801/896317057":"2024-06-11T02:42:42.872312+0000","172.21.15.52:0/1254480982":"2024-06-11T02:42:42.872312+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-10T02:49:00.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:00.796+0000 7ff8e8ae5700 1 -- 172.21.15.52:0/3945449327 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7ff8d00a2c20 msgr2=0x7ff8d00a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:49:00.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:00.796+0000 7ff8e8ae5700 1 --2- 172.21.15.52:0/3945449327 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7ff8d00a2c20 0x7ff8d00a50e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ff8d4005ea0 tx=0x7ff8d4005e30 comp rx=0 tx=0).stop 2024-06-10T02:49:00.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:00.796+0000 7ff8e8ae5700 1 -- 172.21.15.52:0/3945449327 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff8e4108580 msgr2=0x7ff8e4198920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:49:00.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:00.796+0000 7ff8e8ae5700 1 --2- 172.21.15.52:0/3945449327 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff8e4108580 0x7ff8e4198920 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7ff8dc006070 tx=0x7ff8dc004ef0 comp rx=0 tx=0).stop 2024-06-10T02:49:00.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:00.797+0000 7ff8e8ae5700 1 -- 172.21.15.52:0/3945449327 shutdown_connections 2024-06-10T02:49:00.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:00.797+0000 7ff8e8ae5700 1 --2- 172.21.15.52:0/3945449327 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7ff8d00a2c20 0x7ff8d00a50e0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:49:00.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:00.797+0000 7ff8e8ae5700 1 --2- 172.21.15.52:0/3945449327 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff8e4108580 0x7ff8e4198920 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:49:00.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:00.797+0000 7ff8e8ae5700 1 --2- 172.21.15.52:0/3945449327 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff8e406ac60 0x7ff8e419e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:49:00.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:00.797+0000 7ff8e8ae5700 1 --2- 172.21.15.52:0/3945449327 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff8e406b620 0x7ff8e419ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:49:00.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:00.797+0000 7ff8e8ae5700 1 -- 172.21.15.52:0/3945449327 >> 172.21.15.52:0/3945449327 conn(0x7ff8e4078430 msgr2=0x7ff8e40ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:49:00.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:00.797+0000 7ff8e8ae5700 1 -- 172.21.15.52:0/3945449327 shutdown_connections 2024-06-10T02:49:00.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:00.797+0000 7ff8e8ae5700 1 -- 172.21.15.52:0/3945449327 wait complete. 2024-06-10T02:49:01.500 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:01 smithi052 ceph-mon[32362]: from='client.? 172.21.15.52:0/3945449327' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T02:49:01.502 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-06-10T02:45:07.394690+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-06-10T02:49:01.503 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-06-10T02:49:01.636 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:01 smithi120 ceph-mon[36634]: from='client.? 172.21.15.52:0/3945449327' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T02:49:01.702 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:01 smithi188 ceph-mon[36599]: from='client.? 172.21.15.52:0/3945449327' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T02:49:01.752 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:49:02.475 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:02 smithi052 ceph-mon[32362]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T02:49:02.637 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:02 smithi120 ceph-mon[36634]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T02:49:02.702 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:02 smithi188 ceph-mon[36599]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T02:49:03.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:03.491+0000 7fec316f6700 1 -- 172.21.15.52:0/1358482173 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fec2c06b620 msgr2=0x7fec2c10f350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:49:03.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:03.491+0000 7fec316f6700 1 --2- 172.21.15.52:0/1358482173 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fec2c06b620 0x7fec2c10f350 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fec2400b0d0 tx=0x7fec2400b3e0 comp rx=0 tx=0).stop 2024-06-10T02:49:03.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:03.491+0000 7fec316f6700 1 -- 172.21.15.52:0/1358482173 shutdown_connections 2024-06-10T02:49:03.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:03.491+0000 7fec316f6700 1 --2- 172.21.15.52:0/1358482173 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fec2c06b620 0x7fec2c10f350 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:49:03.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:03.491+0000 7fec316f6700 1 --2- 172.21.15.52:0/1358482173 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fec2c108580 0x7fec2c108960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:49:03.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:03.491+0000 7fec316f6700 1 --2- 172.21.15.52:0/1358482173 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fec2c06ac60 0x7fec2c06b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:49:03.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:03.491+0000 7fec316f6700 1 -- 172.21.15.52:0/1358482173 >> 172.21.15.52:0/1358482173 conn(0x7fec2c078430 msgr2=0x7fec2c078840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:49:03.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:03.491+0000 7fec316f6700 1 -- 172.21.15.52:0/1358482173 shutdown_connections 2024-06-10T02:49:03.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:03.491+0000 7fec316f6700 1 -- 172.21.15.52:0/1358482173 wait complete. 2024-06-10T02:49:03.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:03.492+0000 7fec316f6700 1 Processor -- start 2024-06-10T02:49:03.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:03.492+0000 7fec316f6700 1 -- start start 2024-06-10T02:49:03.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:03.492+0000 7fec316f6700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fec2c06ac60 0x7fec2c19e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:49:03.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:03.492+0000 7fec316f6700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fec2c06b620 0x7fec2c19ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:49:03.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:03.492+0000 7fec316f6700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fec2c108580 0x7fec2c1988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:49:03.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:03.492+0000 7fec316f6700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fec2c112810 con 0x7fec2c108580 2024-06-10T02:49:03.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:03.492+0000 7fec316f6700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fec2c112690 con 0x7fec2c06ac60 2024-06-10T02:49:03.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:03.492+0000 7fec316f6700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fec2c112990 con 0x7fec2c06b620 2024-06-10T02:49:03.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:03.493+0000 7fec2b7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fec2c108580 0x7fec2c1988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:49:03.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:03.493+0000 7fec2b7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fec2c108580 0x7fec2c1988d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55666/0 (socket says 172.21.15.52:55666) 2024-06-10T02:49:03.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:03.493+0000 7fec2b7fe700 1 -- 172.21.15.52:0/3108266454 learned_addr learned my addr 172.21.15.52:0/3108266454 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:49:03.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:03.493+0000 7fec2a7fc700 1 --2- 172.21.15.52:0/3108266454 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fec2c06b620 0x7fec2c19ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:49:03.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:03.493+0000 7fec2affd700 1 --2- 172.21.15.52:0/3108266454 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fec2c06ac60 0x7fec2c19e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:49:03.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:03.493+0000 7fec2b7fe700 1 -- 172.21.15.52:0/3108266454 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fec2c06b620 msgr2=0x7fec2c19ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:49:03.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:03.493+0000 7fec2b7fe700 1 --2- 172.21.15.52:0/3108266454 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fec2c06b620 0x7fec2c19ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:49:03.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:03.493+0000 7fec2b7fe700 1 -- 172.21.15.52:0/3108266454 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fec2c06ac60 msgr2=0x7fec2c19e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:49:03.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:03.493+0000 7fec2b7fe700 1 --2- 172.21.15.52:0/3108266454 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fec2c06ac60 0x7fec2c19e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:49:03.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:03.493+0000 7fec2b7fe700 1 -- 172.21.15.52:0/3108266454 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec2c199030 con 0x7fec2c108580 2024-06-10T02:49:03.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:03.493+0000 7fec2affd700 1 --2- 172.21.15.52:0/3108266454 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fec2c06ac60 0x7fec2c19e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T02:49:03.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:03.493+0000 7fec2b7fe700 1 --2- 172.21.15.52:0/3108266454 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fec2c108580 0x7fec2c1988d0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fec24004dc0 tx=0x7fec24004df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:49:03.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:03.493+0000 7fec13fff700 1 -- 172.21.15.52:0/3108266454 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec24021020 con 0x7fec2c108580 2024-06-10T02:49:03.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:03.493+0000 7fec13fff700 1 -- 172.21.15.52:0/3108266454 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec24007c00 con 0x7fec2c108580 2024-06-10T02:49:03.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:03.493+0000 7fec316f6700 1 -- 172.21.15.52:0/3108266454 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec2c1992c0 con 0x7fec2c108580 2024-06-10T02:49:03.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:03.493+0000 7fec13fff700 1 -- 172.21.15.52:0/3108266454 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec24005370 con 0x7fec2c108580 2024-06-10T02:49:03.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:03.493+0000 7fec316f6700 1 -- 172.21.15.52:0/3108266454 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec2c1a55e0 con 0x7fec2c108580 2024-06-10T02:49:03.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:03.494+0000 7fec316f6700 1 -- 172.21.15.52:0/3108266454 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec2c069180 con 0x7fec2c108580 2024-06-10T02:49:03.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:03.495+0000 7fec13fff700 1 -- 172.21.15.52:0/3108266454 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fec24024020 con 0x7fec2c108580 2024-06-10T02:49:03.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:03.496+0000 7fec13fff700 1 --2- 172.21.15.52:0/3108266454 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fec140a2ca0 0x7fec140a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:49:03.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:03.497+0000 7fec2affd700 1 --2- 172.21.15.52:0/3108266454 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fec140a2ca0 0x7fec140a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:49:03.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:03.497+0000 7fec13fff700 1 -- 172.21.15.52:0/3108266454 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fec2400f020 con 0x7fec2c108580 2024-06-10T02:49:03.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:03.497+0000 7fec2affd700 1 --2- 172.21.15.52:0/3108266454 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fec140a2ca0 0x7fec140a5160 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fec1c005fd0 tx=0x7fec1c005e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:49:03.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:03.500+0000 7fec13fff700 1 -- 172.21.15.52:0/3108266454 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fec24007db0 con 0x7fec2c108580 2024-06-10T02:49:03.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:03.707+0000 7fec316f6700 1 -- 172.21.15.52:0/3108266454 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fec2c1a5810 con 0x7fec2c108580 2024-06-10T02:49:03.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:03.707+0000 7fec13fff700 1 -- 172.21.15.52:0/3108266454 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) ==== 93+0+10 (secure 0 0 0) 0x7fec24097e20 con 0x7fec2c108580 2024-06-10T02:49:03.706 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 1 2024-06-10T02:49:03.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:03.709+0000 7fec316f6700 1 -- 172.21.15.52:0/3108266454 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fec140a2ca0 msgr2=0x7fec140a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:49:03.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:03.709+0000 7fec316f6700 1 --2- 172.21.15.52:0/3108266454 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fec140a2ca0 0x7fec140a5160 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fec1c005fd0 tx=0x7fec1c005e40 comp rx=0 tx=0).stop 2024-06-10T02:49:03.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:03.709+0000 7fec316f6700 1 -- 172.21.15.52:0/3108266454 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fec2c108580 msgr2=0x7fec2c1988d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:49:03.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:03.709+0000 7fec316f6700 1 --2- 172.21.15.52:0/3108266454 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fec2c108580 0x7fec2c1988d0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fec24004dc0 tx=0x7fec24004df0 comp rx=0 tx=0).stop 2024-06-10T02:49:03.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:03.709+0000 7fec316f6700 1 -- 172.21.15.52:0/3108266454 shutdown_connections 2024-06-10T02:49:03.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:03.709+0000 7fec316f6700 1 --2- 172.21.15.52:0/3108266454 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fec140a2ca0 0x7fec140a5160 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:49:03.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:03.709+0000 7fec316f6700 1 --2- 172.21.15.52:0/3108266454 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fec2c108580 0x7fec2c1988d0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:49:03.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:03.709+0000 7fec316f6700 1 --2- 172.21.15.52:0/3108266454 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fec2c06ac60 0x7fec2c19e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:49:03.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:03.709+0000 7fec316f6700 1 --2- 172.21.15.52:0/3108266454 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fec2c06b620 0x7fec2c19ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:49:03.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:03.709+0000 7fec316f6700 1 -- 172.21.15.52:0/3108266454 >> 172.21.15.52:0/3108266454 conn(0x7fec2c078430 msgr2=0x7fec2c0ffbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:49:03.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:03.710+0000 7fec316f6700 1 -- 172.21.15.52:0/3108266454 shutdown_connections 2024-06-10T02:49:03.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:03.710+0000 7fec316f6700 1 -- 172.21.15.52:0/3108266454 wait complete. 2024-06-10T02:49:04.452 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:04 smithi188 ceph-mon[36599]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T02:49:04.452 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:04 smithi188 ceph-mon[36599]: from='client.? 172.21.15.52:0/3108266454' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-06-10T02:49:04.475 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:04 smithi052 ceph-mon[32362]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T02:49:04.475 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:04 smithi052 ceph-mon[32362]: from='client.? 172.21.15.52:0/3108266454' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-06-10T02:49:04.507 INFO:tasks.cephadm:Adding mds.a on smithi052 2024-06-10T02:49:04.507 INFO:tasks.cephadm:Adding mds.d on smithi052 2024-06-10T02:49:04.507 INFO:tasks.cephadm:Adding mds.g on smithi052 2024-06-10T02:49:04.507 INFO:tasks.cephadm:Adding mds.j on smithi052 2024-06-10T02:49:04.507 INFO:tasks.cephadm:Adding mds.b on smithi120 2024-06-10T02:49:04.507 INFO:tasks.cephadm:Adding mds.e on smithi120 2024-06-10T02:49:04.507 INFO:tasks.cephadm:Adding mds.h on smithi120 2024-06-10T02:49:04.507 INFO:tasks.cephadm:Adding mds.k on smithi120 2024-06-10T02:49:04.507 INFO:tasks.cephadm:Adding mds.c on smithi188 2024-06-10T02:49:04.507 INFO:tasks.cephadm:Adding mds.f on smithi188 2024-06-10T02:49:04.507 INFO:tasks.cephadm:Adding mds.i on smithi188 2024-06-10T02:49:04.508 INFO:tasks.cephadm:Adding mds.l on smithi188 2024-06-10T02:49:04.508 DEBUG:teuthology.orchestra.run.smithi188:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph orch apply mds all '12;smithi052=a;smithi052=d;smithi052=g;smithi052=j;smithi120=b;smithi120=e;smithi120=h;smithi120=k;smithi188=c;smithi188=f;smithi188=i;smithi188=l' 2024-06-10T02:49:04.636 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:04 smithi120 ceph-mon[36634]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T02:49:04.637 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:04 smithi120 ceph-mon[36634]: from='client.? 172.21.15.52:0/3108266454' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-06-10T02:49:04.758 INFO:teuthology.orchestra.run.smithi188.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.c/config 2024-06-10T02:49:06.411 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:49:06.412+0000 7f92a9f13700 1 -- 172.21.15.188:0/2184977301 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f92a4100b80 msgr2=0x7f92a4100fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:49:06.412 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:49:06.412+0000 7f92a9f13700 1 --2- 172.21.15.188:0/2184977301 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f92a4100b80 0x7f92a4100fa0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f9294009990 tx=0x7f9294009ca0 comp rx=0 tx=0).stop 2024-06-10T02:49:06.412 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:49:06.412+0000 7f92a9f13700 1 -- 172.21.15.188:0/2184977301 shutdown_connections 2024-06-10T02:49:06.412 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:49:06.412+0000 7f92a9f13700 1 --2- 172.21.15.188:0/2184977301 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f92a4102720 0x7f92a4108fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:49:06.412 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:49:06.412+0000 7f92a9f13700 1 --2- 172.21.15.188:0/2184977301 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f92a4100b80 0x7f92a4100fa0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:49:06.412 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:49:06.412+0000 7f92a9f13700 1 --2- 172.21.15.188:0/2184977301 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f92a4101d80 0x7f92a41021e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:49:06.412 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:49:06.412+0000 7f92a9f13700 1 -- 172.21.15.188:0/2184977301 >> 172.21.15.188:0/2184977301 conn(0x7f92a40fc100 msgr2=0x7f92a40fe560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:49:06.412 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:49:06.412+0000 7f92a9f13700 1 -- 172.21.15.188:0/2184977301 shutdown_connections 2024-06-10T02:49:06.412 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:49:06.412+0000 7f92a9f13700 1 -- 172.21.15.188:0/2184977301 wait complete. 2024-06-10T02:49:06.412 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:49:06.412+0000 7f92a9f13700 1 Processor -- start 2024-06-10T02:49:06.412 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:49:06.412+0000 7f92a9f13700 1 -- start start 2024-06-10T02:49:06.412 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:49:06.413+0000 7f92a9f13700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f92a4100b80 0x7f92a419a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:49:06.413 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:49:06.413+0000 7f92a9f13700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f92a4101d80 0x7f92a419ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:49:06.413 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:49:06.413+0000 7f92a9f13700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f92a4102720 0x7f92a41a1f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:49:06.413 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:49:06.413+0000 7f92a9f13700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f92a410cfc0 con 0x7f92a4101d80 2024-06-10T02:49:06.413 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:49:06.413+0000 7f92a9f13700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f92a410ce40 con 0x7f92a4102720 2024-06-10T02:49:06.413 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:49:06.413+0000 7f92a9f13700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f92a410d140 con 0x7f92a4100b80 2024-06-10T02:49:06.413 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:49:06.413+0000 7f92a37fe700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f92a4100b80 0x7f92a419a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:49:06.413 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:49:06.413+0000 7f92a37fe700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f92a4100b80 0x7f92a419a950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.188:33708/0 (socket says 172.21.15.188:33708) 2024-06-10T02:49:06.413 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:49:06.413+0000 7f92a37fe700 1 -- 172.21.15.188:0/2050601807 learned_addr learned my addr 172.21.15.188:0/2050601807 (peer_addr_for_me v2:172.21.15.188:0/0) 2024-06-10T02:49:06.413 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:49:06.413+0000 7f92a3fff700 1 --2- 172.21.15.188:0/2050601807 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f92a4102720 0x7f92a41a1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:49:06.413 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:49:06.413+0000 7f92a37fe700 1 -- 172.21.15.188:0/2050601807 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f92a4102720 msgr2=0x7f92a41a1f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:49:06.413 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:49:06.413+0000 7f92a37fe700 1 --2- 172.21.15.188:0/2050601807 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f92a4102720 0x7f92a41a1f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:49:06.413 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:49:06.413+0000 7f92a37fe700 1 -- 172.21.15.188:0/2050601807 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f92a4101d80 msgr2=0x7f92a419ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:49:06.413 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:49:06.413+0000 7f92a2ffd700 1 --2- 172.21.15.188:0/2050601807 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f92a4101d80 0x7f92a419ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:49:06.414 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:49:06.413+0000 7f92a37fe700 1 --2- 172.21.15.188:0/2050601807 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f92a4101d80 0x7f92a419ae90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:49:06.414 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:49:06.413+0000 7f92a37fe700 1 -- 172.21.15.188:0/2050601807 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92a41a25b0 con 0x7f92a4100b80 2024-06-10T02:49:06.414 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:49:06.413+0000 7f92a37fe700 1 --2- 172.21.15.188:0/2050601807 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f92a4100b80 0x7f92a419a950 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f9294000c00 tx=0x7f9294009ee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:49:06.414 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:49:06.413+0000 7f92a0ff9700 1 -- 172.21.15.188:0/2050601807 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9294027070 con 0x7f92a4100b80 2024-06-10T02:49:06.414 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:49:06.413+0000 7f92a0ff9700 1 -- 172.21.15.188:0/2050601807 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9294004150 con 0x7f92a4100b80 2024-06-10T02:49:06.414 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:49:06.413+0000 7f92a9f13700 1 -- 172.21.15.188:0/2050601807 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92a41a2840 con 0x7f92a4100b80 2024-06-10T02:49:06.414 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:49:06.413+0000 7f92a0ff9700 1 -- 172.21.15.188:0/2050601807 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f929400f6c0 con 0x7f92a4100b80 2024-06-10T02:49:06.414 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:49:06.413+0000 7f92a9f13700 1 -- 172.21.15.188:0/2050601807 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92a41a2c90 con 0x7f92a4100b80 2024-06-10T02:49:06.415 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:49:06.414+0000 7f92a9f13700 1 -- 172.21.15.188:0/2050601807 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92a4069180 con 0x7f92a4100b80 2024-06-10T02:49:06.415 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:49:06.415+0000 7f92a0ff9700 1 -- 172.21.15.188:0/2050601807 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f929402a070 con 0x7f92a4100b80 2024-06-10T02:49:06.415 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:49:06.416+0000 7f92a0ff9700 1 --2- 172.21.15.188:0/2050601807 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f928c0a2c20 0x7f928c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:49:06.415 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:49:06.416+0000 7f92a0ff9700 1 -- 172.21.15.188:0/2050601807 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f92940d03e0 con 0x7f92a4100b80 2024-06-10T02:49:06.415 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:49:06.416+0000 7f92a2ffd700 1 --2- 172.21.15.188:0/2050601807 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f928c0a2c20 0x7f928c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:49:06.416 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:49:06.417+0000 7f92a2ffd700 1 --2- 172.21.15.188:0/2050601807 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f928c0a2c20 0x7f928c0a50e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f9298005950 tx=0x7f9298005cf0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:49:06.417 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:49:06.420+0000 7f92a0ff9700 1 -- 172.21.15.188:0/2050601807 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f9294031020 con 0x7f92a4100b80 2024-06-10T02:49:06.452 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:06 smithi188 ceph-mon[36599]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T02:49:06.626 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:49:06.628+0000 7f92a9f13700 1 -- 172.21.15.188:0/2050601807 --> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi052=a;smithi052=d;smithi052=g;smithi052=j;smithi120=b;smithi120=e;smithi120=h;smithi120=k;smithi188=c;smithi188=f;smithi188=i;smithi188=l", "target": ["mon-mgr", ""]}) -- 0x7f92a40fe3c0 con 0x7f928c0a2c20 2024-06-10T02:49:06.634 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:49:06.637+0000 7f92a0ff9700 1 -- 172.21.15.188:0/2050601807 <== mgr.14150 v2:172.21.15.52:6800/3214763969 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f92a40fe3c0 con 0x7f928c0a2c20 2024-06-10T02:49:06.635 INFO:teuthology.orchestra.run.smithi188.stdout:Scheduled mds.all update... 2024-06-10T02:49:06.637 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:06 smithi120 ceph-mon[36634]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T02:49:06.637 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:49:06.639+0000 7f92a9f13700 1 -- 172.21.15.188:0/2050601807 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f928c0a2c20 msgr2=0x7f928c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:49:06.637 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:49:06.639+0000 7f92a9f13700 1 --2- 172.21.15.188:0/2050601807 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f928c0a2c20 0x7f928c0a50e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f9298005950 tx=0x7f9298005cf0 comp rx=0 tx=0).stop 2024-06-10T02:49:06.637 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:49:06.639+0000 7f92a9f13700 1 -- 172.21.15.188:0/2050601807 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f92a4100b80 msgr2=0x7f92a419a950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:49:06.637 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:49:06.639+0000 7f92a9f13700 1 --2- 172.21.15.188:0/2050601807 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f92a4100b80 0x7f92a419a950 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f9294000c00 tx=0x7f9294009ee0 comp rx=0 tx=0).stop 2024-06-10T02:49:06.637 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:49:06.639+0000 7f92a9f13700 1 -- 172.21.15.188:0/2050601807 shutdown_connections 2024-06-10T02:49:06.637 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:49:06.639+0000 7f92a9f13700 1 --2- 172.21.15.188:0/2050601807 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f92a4102720 0x7f92a41a1f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:49:06.637 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:49:06.639+0000 7f92a9f13700 1 --2- 172.21.15.188:0/2050601807 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f92a4100b80 0x7f92a419a950 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:49:06.637 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:49:06.639+0000 7f92a9f13700 1 --2- 172.21.15.188:0/2050601807 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f928c0a2c20 0x7f928c0a50e0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:49:06.638 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:49:06.639+0000 7f92a9f13700 1 --2- 172.21.15.188:0/2050601807 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f92a4101d80 0x7f92a419ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:49:06.638 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:49:06.639+0000 7f92a9f13700 1 -- 172.21.15.188:0/2050601807 >> 172.21.15.188:0/2050601807 conn(0x7f92a40fc100 msgr2=0x7f92a40fdbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:49:06.638 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:49:06.639+0000 7f92a9f13700 1 -- 172.21.15.188:0/2050601807 shutdown_connections 2024-06-10T02:49:06.638 INFO:teuthology.orchestra.run.smithi188.stderr:2024-06-10T02:49:06.639+0000 7f92a9f13700 1 -- 172.21.15.188:0/2050601807 wait complete. 2024-06-10T02:49:06.725 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:06 smithi052 ceph-mon[32362]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T02:49:07.357 DEBUG:teuthology.orchestra.run.smithi052:mds.a> sudo journalctl -f -n 0 -u ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968@mds.a.service 2024-06-10T02:49:07.360 DEBUG:teuthology.orchestra.run.smithi052:mds.d> sudo journalctl -f -n 0 -u ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968@mds.d.service 2024-06-10T02:49:07.403 DEBUG:teuthology.orchestra.run.smithi052:mds.g> sudo journalctl -f -n 0 -u ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968@mds.g.service 2024-06-10T02:49:07.446 DEBUG:teuthology.orchestra.run.smithi052:mds.j> sudo journalctl -f -n 0 -u ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968@mds.j.service 2024-06-10T02:49:07.489 DEBUG:teuthology.orchestra.run.smithi120:mds.b> sudo journalctl -f -n 0 -u ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968@mds.b.service 2024-06-10T02:49:07.493 DEBUG:teuthology.orchestra.run.smithi120:mds.e> sudo journalctl -f -n 0 -u ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968@mds.e.service 2024-06-10T02:49:07.536 DEBUG:teuthology.orchestra.run.smithi120:mds.h> sudo journalctl -f -n 0 -u ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968@mds.h.service 2024-06-10T02:49:07.579 DEBUG:teuthology.orchestra.run.smithi120:mds.k> sudo journalctl -f -n 0 -u ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968@mds.k.service 2024-06-10T02:49:07.622 DEBUG:teuthology.orchestra.run.smithi188:mds.c> sudo journalctl -f -n 0 -u ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968@mds.c.service 2024-06-10T02:49:07.624 DEBUG:teuthology.orchestra.run.smithi188:mds.f> sudo journalctl -f -n 0 -u ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968@mds.f.service 2024-06-10T02:49:07.666 DEBUG:teuthology.orchestra.run.smithi188:mds.i> sudo journalctl -f -n 0 -u ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968@mds.i.service 2024-06-10T02:49:07.711 DEBUG:teuthology.orchestra.run.smithi188:mds.l> sudo journalctl -f -n 0 -u ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968@mds.l.service 2024-06-10T02:49:07.753 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-06-10T02:49:07.754 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T02:49:07.887 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:07 smithi120 ceph-mon[36634]: from='client.24418 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi052=a;smithi052=d;smithi052=g;smithi052=j;smithi120=b;smithi120=e;smithi120=h;smithi120=k;smithi188=c;smithi188=f;smithi188=i;smithi188=l", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T02:49:07.887 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:07 smithi120 ceph-mon[36634]: Saving service mds.all spec with placement smithi052=a;smithi052=d;smithi052=g;smithi052=j;smithi120=b;smithi120=e;smithi120=h;smithi120=k;smithi188=c;smithi188=f;smithi188=i;smithi188=l;count:12 2024-06-10T02:49:07.887 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:07 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:49:07.887 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:07 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T02:49:07.887 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:07 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:49:07.887 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:07 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T02:49:07.887 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:07 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:49:07.887 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:07 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-10T02:49:07.887 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:07 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-10T02:49:07.887 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:07 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:49:07.906 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:07 smithi188 ceph-mon[36599]: from='client.24418 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi052=a;smithi052=d;smithi052=g;smithi052=j;smithi120=b;smithi120=e;smithi120=h;smithi120=k;smithi188=c;smithi188=f;smithi188=i;smithi188=l", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T02:49:07.906 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:07 smithi188 ceph-mon[36599]: Saving service mds.all spec with placement smithi052=a;smithi052=d;smithi052=g;smithi052=j;smithi120=b;smithi120=e;smithi120=h;smithi120=k;smithi188=c;smithi188=f;smithi188=i;smithi188=l;count:12 2024-06-10T02:49:07.906 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:07 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:49:07.906 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:07 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T02:49:07.906 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:07 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:49:07.906 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:07 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T02:49:07.907 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:07 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:49:07.907 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:07 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-10T02:49:07.907 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:07 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-10T02:49:07.907 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:07 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:49:07.975 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:07 smithi052 ceph-mon[32362]: from='client.24418 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi052=a;smithi052=d;smithi052=g;smithi052=j;smithi120=b;smithi120=e;smithi120=h;smithi120=k;smithi188=c;smithi188=f;smithi188=i;smithi188=l", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T02:49:07.975 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:07 smithi052 ceph-mon[32362]: Saving service mds.all spec with placement smithi052=a;smithi052=d;smithi052=g;smithi052=j;smithi120=b;smithi120=e;smithi120=h;smithi120=k;smithi188=c;smithi188=f;smithi188=i;smithi188=l;count:12 2024-06-10T02:49:07.975 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:07 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:49:07.975 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:07 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T02:49:07.975 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:07 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:49:07.975 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:07 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T02:49:07.976 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:07 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:49:07.976 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:07 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-10T02:49:07.976 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:07 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-10T02:49:07.976 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:07 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:49:08.002 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:49:08.725 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:08 smithi052 ceph-mon[32362]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T02:49:08.725 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:08 smithi052 ceph-mon[32362]: Deploying daemon mds.l on smithi188 2024-06-10T02:49:08.914 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:08 smithi188 ceph-mon[36599]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T02:49:08.914 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:08 smithi188 ceph-mon[36599]: Deploying daemon mds.l on smithi188 2024-06-10T02:49:09.136 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:08 smithi120 ceph-mon[36634]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T02:49:09.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:08 smithi120 ceph-mon[36634]: Deploying daemon mds.l on smithi188 2024-06-10T02:49:09.452 INFO:journalctl@ceph.mds.l.smithi188.stdout:Jun 10 02:49:09 smithi188 systemd[1]: Starting Ceph mds.l for d8676e9e-26d2-11ef-bc9b-c7b262605968... 2024-06-10T02:49:09.755 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:09.755+0000 7fbc3d8ac700 1 -- 172.21.15.52:0/3133305541 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbc38107400 msgr2=0x7fbc381077e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:49:09.755 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:09.755+0000 7fbc3d8ac700 1 --2- 172.21.15.52:0/3133305541 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbc38107400 0x7fbc381077e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fbc28009990 tx=0x7fbc28009ca0 comp rx=0 tx=0).stop 2024-06-10T02:49:09.755 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:09.755+0000 7fbc3d8ac700 1 -- 172.21.15.52:0/3133305541 shutdown_connections 2024-06-10T02:49:09.755 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:09.755+0000 7fbc3d8ac700 1 --2- 172.21.15.52:0/3133305541 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbc38101d50 0x7fbc3810e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:49:09.755 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:09.755+0000 7fbc3d8ac700 1 --2- 172.21.15.52:0/3133305541 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbc38107400 0x7fbc381077e0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:49:09.756 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:09.755+0000 7fbc3d8ac700 1 --2- 172.21.15.52:0/3133305541 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbc38101390 0x7fbc38101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:49:09.756 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:09.755+0000 7fbc3d8ac700 1 -- 172.21.15.52:0/3133305541 >> 172.21.15.52:0/3133305541 conn(0x7fbc38078340 msgr2=0x7fbc38078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:49:09.756 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:09.755+0000 7fbc3d8ac700 1 -- 172.21.15.52:0/3133305541 shutdown_connections 2024-06-10T02:49:09.756 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:09.755+0000 7fbc3d8ac700 1 -- 172.21.15.52:0/3133305541 wait complete. 2024-06-10T02:49:09.756 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:09.756+0000 7fbc3d8ac700 1 Processor -- start 2024-06-10T02:49:09.756 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:09.756+0000 7fbc3d8ac700 1 -- start start 2024-06-10T02:49:09.756 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:09.756+0000 7fbc3d8ac700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbc38101390 0x7fbc38113a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:49:09.756 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:09.756+0000 7fbc3d8ac700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbc38101d50 0x7fbc38111ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:49:09.756 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:09.756+0000 7fbc3d8ac700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbc38107400 0x7fbc381124c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:49:09.756 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:09.756+0000 7fbc3d8ac700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fbc38111710 con 0x7fbc38101390 2024-06-10T02:49:09.756 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:09.756+0000 7fbc3d8ac700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fbc38111590 con 0x7fbc38101d50 2024-06-10T02:49:09.756 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:09.756+0000 7fbc3d8ac700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fbc38111890 con 0x7fbc38107400 2024-06-10T02:49:09.756 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:09.756+0000 7fbc36ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbc38101390 0x7fbc38113a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:49:09.757 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:09.756+0000 7fbc36ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbc38101390 0x7fbc38113a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55694/0 (socket says 172.21.15.52:55694) 2024-06-10T02:49:09.757 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:09.756+0000 7fbc36ffd700 1 -- 172.21.15.52:0/1617275825 learned_addr learned my addr 172.21.15.52:0/1617275825 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:49:09.757 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:09.756+0000 7fbc377fe700 1 --2- 172.21.15.52:0/1617275825 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbc38107400 0x7fbc381124c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:49:09.757 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:09.756+0000 7fbc367fc700 1 --2- 172.21.15.52:0/1617275825 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbc38101d50 0x7fbc38111ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:49:09.757 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:09.757+0000 7fbc36ffd700 1 -- 172.21.15.52:0/1617275825 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbc38107400 msgr2=0x7fbc381124c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:49:09.757 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:09.757+0000 7fbc36ffd700 1 --2- 172.21.15.52:0/1617275825 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbc38107400 0x7fbc381124c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:49:09.757 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:09.757+0000 7fbc36ffd700 1 -- 172.21.15.52:0/1617275825 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbc38101d50 msgr2=0x7fbc38111ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:49:09.757 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:09.757+0000 7fbc36ffd700 1 --2- 172.21.15.52:0/1617275825 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbc38101d50 0x7fbc38111ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:49:09.757 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:09.757+0000 7fbc36ffd700 1 -- 172.21.15.52:0/1617275825 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc381a7270 con 0x7fbc38101390 2024-06-10T02:49:09.757 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:09.757+0000 7fbc367fc700 1 --2- 172.21.15.52:0/1617275825 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbc38101d50 0x7fbc38111ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T02:49:09.757 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:09.757+0000 7fbc36ffd700 1 --2- 172.21.15.52:0/1617275825 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbc38101390 0x7fbc38113a20 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fbc280038e0 tx=0x7fbc28005590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:49:09.757 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:09.757+0000 7fbc3c8aa700 1 -- 172.21.15.52:0/1617275825 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc28026070 con 0x7fbc38101390 2024-06-10T02:49:09.757 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:09.757+0000 7fbc3c8aa700 1 -- 172.21.15.52:0/1617275825 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc28004030 con 0x7fbc38101390 2024-06-10T02:49:09.758 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:09.757+0000 7fbc3d8ac700 1 -- 172.21.15.52:0/1617275825 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc381a7440 con 0x7fbc38101390 2024-06-10T02:49:09.759 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:09.757+0000 7fbc3c8aa700 1 -- 172.21.15.52:0/1617275825 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc28021410 con 0x7fbc38101390 2024-06-10T02:49:09.759 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:09.757+0000 7fbc3d8ac700 1 -- 172.21.15.52:0/1617275825 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc381a7890 con 0x7fbc38101390 2024-06-10T02:49:09.759 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:09.758+0000 7fbc3d8ac700 1 -- 172.21.15.52:0/1617275825 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc38069180 con 0x7fbc38101390 2024-06-10T02:49:09.759 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:09.759+0000 7fbc3c8aa700 1 -- 172.21.15.52:0/1617275825 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fbc28029070 con 0x7fbc38101390 2024-06-10T02:49:09.759 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:09.760+0000 7fbc3c8aa700 1 --2- 172.21.15.52:0/1617275825 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fbc200a2c20 0x7fbc200a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:49:09.759 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:09.760+0000 7fbc367fc700 1 --2- 172.21.15.52:0/1617275825 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fbc200a2c20 0x7fbc200a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:49:09.759 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:09.760+0000 7fbc3c8aa700 1 -- 172.21.15.52:0/1617275825 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fbc280cfa10 con 0x7fbc38101390 2024-06-10T02:49:09.759 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:09.760+0000 7fbc367fc700 1 --2- 172.21.15.52:0/1617275825 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fbc200a2c20 0x7fbc200a50e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fbc380fe590 tx=0x7fbc2c008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:49:09.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:09.764+0000 7fbc3c8aa700 1 -- 172.21.15.52:0/1617275825 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fbc28030080 con 0x7fbc38101390 2024-06-10T02:49:09.874 INFO:journalctl@ceph.mds.l.smithi188.stdout:Jun 10 02:49:09 smithi188 podman[66010]: 2024-06-10 02:49:09.50595021 +0000 UTC m=+0.016842216 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-10T02:49:09.874 INFO:journalctl@ceph.mds.l.smithi188.stdout:Jun 10 02:49:09 smithi188 podman[66010]: 2024-06-10 02:49:09.61445281 +0000 UTC m=+0.125344861 container create 14557ac164f932444d000162d6bea3fae330fb441b9a1dbb1ed599d0b2786b37 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mds-l, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-06-10T02:49:09.874 INFO:journalctl@ceph.mds.l.smithi188.stdout:Jun 10 02:49:09 smithi188 podman[66010]: 2024-06-10 02:49:09.850488295 +0000 UTC m=+0.361380292 container init 14557ac164f932444d000162d6bea3fae330fb441b9a1dbb1ed599d0b2786b37 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mds-l, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240528, GIT_CLEAN=True) 2024-06-10T02:49:09.874 INFO:journalctl@ceph.mds.l.smithi188.stdout:Jun 10 02:49:09 smithi188 podman[66010]: 2024-06-10 02:49:09.853030084 +0000 UTC m=+0.363922083 container start 14557ac164f932444d000162d6bea3fae330fb441b9a1dbb1ed599d0b2786b37 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mds-l, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-06-10T02:49:10.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:10.017+0000 7fbc3d8ac700 1 -- 172.21.15.52:0/1617275825 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbc381a7bb0 con 0x7fbc38101390 2024-06-10T02:49:10.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:10.017+0000 7fbc3c8aa700 1 -- 172.21.15.52:0/1617275825 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1393 (secure 0 0 0) 0x7fbc28013070 con 0x7fbc38101390 2024-06-10T02:49:10.016 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-10T02:49:10.017 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/529564979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":529564979},{"type":"v1","addr":"172.21.15.188:6833","nonce":529564979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-06-10T02:49:10.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:10.020+0000 7fbc3d8ac700 1 -- 172.21.15.52:0/1617275825 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fbc200a2c20 msgr2=0x7fbc200a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:49:10.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:10.020+0000 7fbc3d8ac700 1 --2- 172.21.15.52:0/1617275825 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fbc200a2c20 0x7fbc200a50e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fbc380fe590 tx=0x7fbc2c008040 comp rx=0 tx=0).stop 2024-06-10T02:49:10.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:10.020+0000 7fbc3d8ac700 1 -- 172.21.15.52:0/1617275825 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbc38101390 msgr2=0x7fbc38113a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:49:10.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:10.020+0000 7fbc3d8ac700 1 --2- 172.21.15.52:0/1617275825 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbc38101390 0x7fbc38113a20 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fbc280038e0 tx=0x7fbc28005590 comp rx=0 tx=0).stop 2024-06-10T02:49:10.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:10.020+0000 7fbc3d8ac700 1 -- 172.21.15.52:0/1617275825 shutdown_connections 2024-06-10T02:49:10.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:10.020+0000 7fbc3d8ac700 1 --2- 172.21.15.52:0/1617275825 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbc38107400 0x7fbc381124c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:49:10.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:10.020+0000 7fbc3d8ac700 1 --2- 172.21.15.52:0/1617275825 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fbc200a2c20 0x7fbc200a50e0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:49:10.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:10.020+0000 7fbc3d8ac700 1 --2- 172.21.15.52:0/1617275825 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbc38101390 0x7fbc38113a20 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:49:10.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:10.020+0000 7fbc3d8ac700 1 --2- 172.21.15.52:0/1617275825 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbc38101d50 0x7fbc38111ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:49:10.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:10.020+0000 7fbc3d8ac700 1 -- 172.21.15.52:0/1617275825 >> 172.21.15.52:0/1617275825 conn(0x7fbc38078340 msgr2=0x7fbc380fedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:49:10.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:10.020+0000 7fbc3d8ac700 1 -- 172.21.15.52:0/1617275825 shutdown_connections 2024-06-10T02:49:10.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:10.020+0000 7fbc3d8ac700 1 -- 172.21.15.52:0/1617275825 wait complete. 2024-06-10T02:49:10.019 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 2 2024-06-10T02:49:10.202 INFO:journalctl@ceph.mds.l.smithi188.stdout:Jun 10 02:49:09 smithi188 ceph-mds[66029]: -- 172.21.15.188:0/385080858 <== mon.1 v2:172.21.15.188:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c151fe5a40 con 0x55c151fb9000 2024-06-10T02:49:10.202 INFO:journalctl@ceph.mds.l.smithi188.stdout:Jun 10 02:49:09 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mds-l[66025]: starting mds.l at 2024-06-10T02:49:10.202 INFO:journalctl@ceph.mds.l.smithi188.stdout:Jun 10 02:49:09 smithi188 bash[66010]: 14557ac164f932444d000162d6bea3fae330fb441b9a1dbb1ed599d0b2786b37 2024-06-10T02:49:10.202 INFO:journalctl@ceph.mds.l.smithi188.stdout:Jun 10 02:49:09 smithi188 systemd[1]: Started Ceph mds.l for d8676e9e-26d2-11ef-bc9b-c7b262605968. 2024-06-10T02:49:10.914 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:10 smithi120 ceph-mon[36634]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T02:49:10.914 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:10 smithi120 ceph-mon[36634]: mds.? [v2:172.21.15.188:6832/529564979,v1:172.21.15.188:6833/529564979] up:boot 2024-06-10T02:49:10.914 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:10 smithi120 ceph-mon[36634]: fsmap 1 up:standby 2024-06-10T02:49:10.914 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:10 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-06-10T02:49:10.914 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:10 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:49:10.914 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:10 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:49:10.914 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:10 smithi120 ceph-mon[36634]: from='client.? 172.21.15.52:0/1617275825' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T02:49:10.914 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:10 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:49:10.914 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:10 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-10T02:49:10.914 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:10 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-10T02:49:10.914 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:10 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:49:10.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:10 smithi188 ceph-mon[36599]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T02:49:10.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:10 smithi188 ceph-mon[36599]: mds.? [v2:172.21.15.188:6832/529564979,v1:172.21.15.188:6833/529564979] up:boot 2024-06-10T02:49:10.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:10 smithi188 ceph-mon[36599]: fsmap 1 up:standby 2024-06-10T02:49:10.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:10 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-06-10T02:49:10.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:10 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:49:10.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:10 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:49:10.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:10 smithi188 ceph-mon[36599]: from='client.? 172.21.15.52:0/1617275825' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T02:49:10.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:10 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:49:10.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:10 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-10T02:49:10.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:10 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-10T02:49:10.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:10 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:49:10.977 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:10 smithi052 ceph-mon[32362]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T02:49:10.978 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:10 smithi052 ceph-mon[32362]: mds.? [v2:172.21.15.188:6832/529564979,v1:172.21.15.188:6833/529564979] up:boot 2024-06-10T02:49:10.978 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:10 smithi052 ceph-mon[32362]: fsmap 1 up:standby 2024-06-10T02:49:10.978 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:10 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-06-10T02:49:10.978 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:10 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:49:10.978 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:10 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:49:10.978 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:10 smithi052 ceph-mon[32362]: from='client.? 172.21.15.52:0/1617275825' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T02:49:10.978 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:10 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:49:10.978 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:10 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-10T02:49:10.978 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:10 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-10T02:49:10.978 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:10 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:49:11.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:11 smithi188 ceph-mon[36599]: Deploying daemon mds.k on smithi120 2024-06-10T02:49:11.975 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:11 smithi052 ceph-mon[32362]: Deploying daemon mds.k on smithi120 2024-06-10T02:49:12.053 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:11 smithi120 ceph-mon[36634]: Deploying daemon mds.k on smithi120 2024-06-10T02:49:12.387 INFO:journalctl@ceph.mds.k.smithi120.stdout:Jun 10 02:49:12 smithi120 systemd[1]: Starting Ceph mds.k for d8676e9e-26d2-11ef-bc9b-c7b262605968... 2024-06-10T02:49:12.715 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:12 smithi120 ceph-mon[36634]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T02:49:12.716 INFO:journalctl@ceph.mds.k.smithi120.stdout:Jun 10 02:49:12 smithi120 podman[65060]: 2024-06-10 02:49:12.350703857 +0000 UTC m=+0.017832443 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-10T02:49:12.716 INFO:journalctl@ceph.mds.k.smithi120.stdout:Jun 10 02:49:12 smithi120 podman[65060]: 2024-06-10 02:49:12.465198317 +0000 UTC m=+0.132326949 container create 87d4e55dc881189ff24119696bed15ea75240f677a08f32425e6adedf06111c6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mds-k, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , ceph=True, RELEASE=squid-3d89acc, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-06-10T02:49:12.716 INFO:journalctl@ceph.mds.k.smithi120.stdout:Jun 10 02:49:12 smithi120 podman[65060]: 2024-06-10 02:49:12.718106403 +0000 UTC m=+0.385234980 container init 87d4e55dc881189ff24119696bed15ea75240f677a08f32425e6adedf06111c6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mds-k, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-10T02:49:12.741 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T02:49:12.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:12 smithi188 ceph-mon[36599]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T02:49:12.975 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:12 smithi052 ceph-mon[32362]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T02:49:13.010 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:49:13.137 INFO:journalctl@ceph.mds.k.smithi120.stdout:Jun 10 02:49:12 smithi120 podman[65060]: 2024-06-10 02:49:12.720950498 +0000 UTC m=+0.388079087 container start 87d4e55dc881189ff24119696bed15ea75240f677a08f32425e6adedf06111c6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mds-k, ceph=True, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5) 2024-06-10T02:49:13.137 INFO:journalctl@ceph.mds.k.smithi120.stdout:Jun 10 02:49:12 smithi120 ceph-mds[65079]: -- 172.21.15.120:0/2703133294 <== mon.2 v2:172.21.15.120:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b1f9cf9a40 con 0x55b1f9ccd000 2024-06-10T02:49:13.137 INFO:journalctl@ceph.mds.k.smithi120.stdout:Jun 10 02:49:12 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mds-k[65075]: starting mds.k at 2024-06-10T02:49:13.137 INFO:journalctl@ceph.mds.k.smithi120.stdout:Jun 10 02:49:12 smithi120 bash[65060]: 87d4e55dc881189ff24119696bed15ea75240f677a08f32425e6adedf06111c6 2024-06-10T02:49:13.137 INFO:journalctl@ceph.mds.k.smithi120.stdout:Jun 10 02:49:12 smithi120 systemd[1]: Started Ceph mds.k for d8676e9e-26d2-11ef-bc9b-c7b262605968. 2024-06-10T02:49:14.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:13 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:49:14.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:13 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:49:14.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:13 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:49:14.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:13 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-10T02:49:14.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:13 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-10T02:49:14.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:13 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:49:14.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:13 smithi120 ceph-mon[36634]: Deploying daemon mds.j on smithi052 2024-06-10T02:49:14.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:13 smithi120 ceph-mon[36634]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T02:49:14.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:13 smithi120 ceph-mon[36634]: mds.? [v2:172.21.15.120:6832/2609002896,v1:172.21.15.120:6833/2609002896] up:boot 2024-06-10T02:49:14.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:13 smithi120 ceph-mon[36634]: fsmap 2 up:standby 2024-06-10T02:49:14.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:13 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-06-10T02:49:14.162 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:13 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:49:14.162 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:13 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:49:14.162 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:13 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:49:14.162 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:13 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-10T02:49:14.162 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:13 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-10T02:49:14.163 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:13 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:49:14.163 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:13 smithi052 ceph-mon[32362]: Deploying daemon mds.j on smithi052 2024-06-10T02:49:14.163 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:13 smithi052 ceph-mon[32362]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T02:49:14.163 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:13 smithi052 ceph-mon[32362]: mds.? [v2:172.21.15.120:6832/2609002896,v1:172.21.15.120:6833/2609002896] up:boot 2024-06-10T02:49:14.163 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:13 smithi052 ceph-mon[32362]: fsmap 2 up:standby 2024-06-10T02:49:14.163 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:13 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-06-10T02:49:14.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:13 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:49:14.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:13 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:49:14.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:13 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:49:14.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:13 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-10T02:49:14.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:13 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-10T02:49:14.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:13 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:49:14.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:13 smithi188 ceph-mon[36599]: Deploying daemon mds.j on smithi052 2024-06-10T02:49:14.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:13 smithi188 ceph-mon[36599]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T02:49:14.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:13 smithi188 ceph-mon[36599]: mds.? [v2:172.21.15.120:6832/2609002896,v1:172.21.15.120:6833/2609002896] up:boot 2024-06-10T02:49:14.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:13 smithi188 ceph-mon[36599]: fsmap 2 up:standby 2024-06-10T02:49:14.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:13 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-06-10T02:49:14.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:14.769+0000 7f37650fb700 1 -- 172.21.15.52:0/4160956443 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f376006b520 msgr2=0x7f376010f340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:49:14.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:14.769+0000 7f37650fb700 1 --2- 172.21.15.52:0/4160956443 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f376006b520 0x7f376010f340 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f3758009990 tx=0x7f3758009ca0 comp rx=0 tx=0).stop 2024-06-10T02:49:14.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:14.769+0000 7f37650fb700 1 -- 172.21.15.52:0/4160956443 shutdown_connections 2024-06-10T02:49:14.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:14.769+0000 7f37650fb700 1 --2- 172.21.15.52:0/4160956443 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f376006b520 0x7f376010f340 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:49:14.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:14.769+0000 7f37650fb700 1 --2- 172.21.15.52:0/4160956443 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3760108520 0x7f3760108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:49:14.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:14.769+0000 7f37650fb700 1 --2- 172.21.15.52:0/4160956443 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f376006ab60 0x7f376006afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:49:14.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:14.769+0000 7f37650fb700 1 -- 172.21.15.52:0/4160956443 >> 172.21.15.52:0/4160956443 conn(0x7f3760078340 msgr2=0x7f3760078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:49:14.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:14.769+0000 7f37650fb700 1 -- 172.21.15.52:0/4160956443 shutdown_connections 2024-06-10T02:49:14.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:14.769+0000 7f37650fb700 1 -- 172.21.15.52:0/4160956443 wait complete. 2024-06-10T02:49:14.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:14.770+0000 7f37650fb700 1 Processor -- start 2024-06-10T02:49:14.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:14.770+0000 7f37650fb700 1 -- start start 2024-06-10T02:49:14.770 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:14.770+0000 7f37650fb700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f376006ab60 0x7f3760101840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:49:14.770 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:14.770+0000 7f37650fb700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f376006b520 0x7f3760101d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:49:14.770 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:14.770+0000 7f37650fb700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3760108520 0x7f37601022c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:49:14.770 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:14.770+0000 7f37650fb700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f3760112800 con 0x7f3760108520 2024-06-10T02:49:14.771 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:14.770+0000 7f37650fb700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f3760112680 con 0x7f376006ab60 2024-06-10T02:49:14.771 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:14.770+0000 7f37650fb700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f3760112980 con 0x7f376006b520 2024-06-10T02:49:14.771 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:14.770+0000 7f375f59e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3760108520 0x7f37601022c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:49:14.771 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:14.770+0000 7f375f59e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3760108520 0x7f37601022c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57644/0 (socket says 172.21.15.52:57644) 2024-06-10T02:49:14.771 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:14.770+0000 7f375f59e700 1 -- 172.21.15.52:0/3715346700 learned_addr learned my addr 172.21.15.52:0/3715346700 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:49:14.771 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:14.770+0000 7f375e59c700 1 --2- 172.21.15.52:0/3715346700 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f376006b520 0x7f3760101d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:49:14.771 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:14.770+0000 7f375ed9d700 1 --2- 172.21.15.52:0/3715346700 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f376006ab60 0x7f3760101840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:49:14.771 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:14.771+0000 7f375f59e700 1 -- 172.21.15.52:0/3715346700 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f376006b520 msgr2=0x7f3760101d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:49:14.771 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:14.771+0000 7f375f59e700 1 --2- 172.21.15.52:0/3715346700 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f376006b520 0x7f3760101d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:49:14.771 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:14.771+0000 7f375f59e700 1 -- 172.21.15.52:0/3715346700 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f376006ab60 msgr2=0x7f3760101840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:49:14.771 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:14.771+0000 7f375f59e700 1 --2- 172.21.15.52:0/3715346700 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f376006ab60 0x7f3760101840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:49:14.771 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:14.771+0000 7f375f59e700 1 -- 172.21.15.52:0/3715346700 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37601a7130 con 0x7f3760108520 2024-06-10T02:49:14.772 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:14.771+0000 7f375e59c700 1 --2- 172.21.15.52:0/3715346700 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f376006b520 0x7f3760101d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T02:49:14.772 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:14.771+0000 7f375f59e700 1 --2- 172.21.15.52:0/3715346700 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3760108520 0x7f37601022c0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f37580047b0 tx=0x7f3758009ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:49:14.772 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:14.771+0000 7f3757fff700 1 -- 172.21.15.52:0/3715346700 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3758004830 con 0x7f3760108520 2024-06-10T02:49:14.772 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:14.771+0000 7f37650fb700 1 -- 172.21.15.52:0/3715346700 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37601a73c0 con 0x7f3760108520 2024-06-10T02:49:14.772 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:14.771+0000 7f3757fff700 1 -- 172.21.15.52:0/3715346700 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3758005020 con 0x7f3760108520 2024-06-10T02:49:14.772 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:14.771+0000 7f3757fff700 1 -- 172.21.15.52:0/3715346700 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3758003e60 con 0x7f3760108520 2024-06-10T02:49:14.772 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:14.771+0000 7f37650fb700 1 -- 172.21.15.52:0/3715346700 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37601a7810 con 0x7f3760108520 2024-06-10T02:49:14.772 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:14.772+0000 7f37650fb700 1 -- 172.21.15.52:0/3715346700 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3760103be0 con 0x7f3760108520 2024-06-10T02:49:14.773 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:14.773+0000 7f3757fff700 1 -- 172.21.15.52:0/3715346700 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f37580051d0 con 0x7f3760108520 2024-06-10T02:49:14.773 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:14.774+0000 7f3757fff700 1 --2- 172.21.15.52:0/3715346700 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f374c0a2c20 0x7f374c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:49:14.773 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:14.774+0000 7f375ed9d700 1 --2- 172.21.15.52:0/3715346700 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f374c0a2c20 0x7f374c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:49:14.773 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:14.774+0000 7f3757fff700 1 -- 172.21.15.52:0/3715346700 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f37580cfa20 con 0x7f3760108520 2024-06-10T02:49:14.773 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:14.774+0000 7f375ed9d700 1 --2- 172.21.15.52:0/3715346700 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f374c0a2c20 0x7f374c0a50e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f3750009d10 tx=0x7f3750009040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:49:14.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:14.779+0000 7f3757fff700 1 -- 172.21.15.52:0/3715346700 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f3758013070 con 0x7f3760108520 2024-06-10T02:49:15.035 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:15.037+0000 7f37650fb700 1 -- 172.21.15.52:0/3715346700 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f376002dca0 con 0x7f3760108520 2024-06-10T02:49:15.036 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:15.037+0000 7f3757fff700 1 -- 172.21.15.52:0/3715346700 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2221 (secure 0 0 0) 0x7f375809ceb0 con 0x7f3760108520 2024-06-10T02:49:15.036 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-10T02:49:15.036 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24383,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6833/2609002896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":2609002896},{"type":"v1","addr":"172.21.15.120:6833","nonce":2609002896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/529564979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":529564979},{"type":"v1","addr":"172.21.15.188:6833","nonce":529564979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-06-10T02:49:15.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:15.039+0000 7f37650fb700 1 -- 172.21.15.52:0/3715346700 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f374c0a2c20 msgr2=0x7f374c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:49:15.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:15.039+0000 7f37650fb700 1 --2- 172.21.15.52:0/3715346700 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f374c0a2c20 0x7f374c0a50e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f3750009d10 tx=0x7f3750009040 comp rx=0 tx=0).stop 2024-06-10T02:49:15.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:15.039+0000 7f37650fb700 1 -- 172.21.15.52:0/3715346700 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3760108520 msgr2=0x7f37601022c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:49:15.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:15.039+0000 7f37650fb700 1 --2- 172.21.15.52:0/3715346700 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3760108520 0x7f37601022c0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f37580047b0 tx=0x7f3758009ee0 comp rx=0 tx=0).stop 2024-06-10T02:49:15.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:15.039+0000 7f37650fb700 1 -- 172.21.15.52:0/3715346700 shutdown_connections 2024-06-10T02:49:15.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:15.039+0000 7f37650fb700 1 --2- 172.21.15.52:0/3715346700 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f374c0a2c20 0x7f374c0a50e0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:49:15.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:15.039+0000 7f37650fb700 1 --2- 172.21.15.52:0/3715346700 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3760108520 0x7f37601022c0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:49:15.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:15.039+0000 7f37650fb700 1 --2- 172.21.15.52:0/3715346700 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f376006ab60 0x7f3760101840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:49:15.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:15.039+0000 7f37650fb700 1 --2- 172.21.15.52:0/3715346700 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f376006b520 0x7f3760101d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:49:15.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:15.039+0000 7f37650fb700 1 -- 172.21.15.52:0/3715346700 >> 172.21.15.52:0/3715346700 conn(0x7f3760078340 msgr2=0x7f37601003c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:49:15.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:15.040+0000 7f37650fb700 1 -- 172.21.15.52:0/3715346700 shutdown_connections 2024-06-10T02:49:15.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:15.040+0000 7f37650fb700 1 -- 172.21.15.52:0/3715346700 wait complete. 2024-06-10T02:49:15.039 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 3 2024-06-10T02:49:16.198 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:16 smithi052 ceph-mon[32362]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T02:49:16.198 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:16 smithi052 ceph-mon[32362]: from='client.? 172.21.15.52:0/3715346700' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T02:49:16.386 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:16 smithi120 ceph-mon[36634]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T02:49:16.387 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:16 smithi120 ceph-mon[36634]: from='client.? 172.21.15.52:0/3715346700' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T02:49:16.452 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:16 smithi188 ceph-mon[36599]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T02:49:16.452 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:16 smithi188 ceph-mon[36599]: from='client.? 172.21.15.52:0/3715346700' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T02:49:17.422 INFO:journalctl@ceph.mds.j.smithi052.stdout:Jun 10 02:49:17 smithi052 systemd[1]: Starting Ceph mds.j for d8676e9e-26d2-11ef-bc9b-c7b262605968... 2024-06-10T02:49:17.685 INFO:journalctl@ceph.mds.j.smithi052.stdout:Jun 10 02:49:17 smithi052 podman[72892]: 2024-06-10 02:49:17.424533218 +0000 UTC m=+0.113799686 container create 6ba576331e94db1d4f7f22b03deba8d69e93243eea22e4980e7abeab01a874a2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mds-j, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-10T02:49:17.685 INFO:journalctl@ceph.mds.j.smithi052.stdout:Jun 10 02:49:17 smithi052 podman[72892]: 2024-06-10 02:49:17.328109749 +0000 UTC m=+0.017376242 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-10T02:49:17.686 INFO:journalctl@ceph.mds.j.smithi052.stdout:Jun 10 02:49:17 smithi052 podman[72892]: 2024-06-10 02:49:17.660548376 +0000 UTC m=+0.349814845 container init 6ba576331e94db1d4f7f22b03deba8d69e93243eea22e4980e7abeab01a874a2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mds-j, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-06-10T02:49:17.686 INFO:journalctl@ceph.mds.j.smithi052.stdout:Jun 10 02:49:17 smithi052 podman[72892]: 2024-06-10 02:49:17.663503433 +0000 UTC m=+0.352769911 container start 6ba576331e94db1d4f7f22b03deba8d69e93243eea22e4980e7abeab01a874a2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mds-j, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, RELEASE=squid-3d89acc, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=) 2024-06-10T02:49:17.686 INFO:journalctl@ceph.mds.j.smithi052.stdout:Jun 10 02:49:17 smithi052 ceph-mds[72911]: -- 172.21.15.52:0/747056654 <== mon.0 v2:172.21.15.52:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ac10f29a40 con 0x55ac10efc800 2024-06-10T02:49:17.975 INFO:journalctl@ceph.mds.j.smithi052.stdout:Jun 10 02:49:17 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mds-j[72907]: starting mds.j at 2024-06-10T02:49:17.975 INFO:journalctl@ceph.mds.j.smithi052.stdout:Jun 10 02:49:17 smithi052 bash[72892]: 6ba576331e94db1d4f7f22b03deba8d69e93243eea22e4980e7abeab01a874a2 2024-06-10T02:49:17.975 INFO:journalctl@ceph.mds.j.smithi052.stdout:Jun 10 02:49:17 smithi052 systemd[1]: Started Ceph mds.j for d8676e9e-26d2-11ef-bc9b-c7b262605968. 2024-06-10T02:49:18.236 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T02:49:18.861 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:18 smithi052 ceph-mon[32362]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T02:49:18.861 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:18 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:49:18.861 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:18 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:49:18.861 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:18 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:49:18.861 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:18 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-10T02:49:18.861 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:18 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-10T02:49:18.861 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:18 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:49:18.861 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:18 smithi052 ceph-mon[32362]: mds.? [v2:172.21.15.52:6834/889673247,v1:172.21.15.52:6835/889673247] up:boot 2024-06-10T02:49:18.862 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:18 smithi052 ceph-mon[32362]: fsmap 3 up:standby 2024-06-10T02:49:18.862 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:18 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-06-10T02:49:18.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:18 smithi188 ceph-mon[36599]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T02:49:18.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:18 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:49:18.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:18 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:49:18.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:18 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:49:18.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:18 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-10T02:49:18.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:18 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-10T02:49:18.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:18 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:49:18.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:18 smithi188 ceph-mon[36599]: mds.? [v2:172.21.15.52:6834/889673247,v1:172.21.15.52:6835/889673247] up:boot 2024-06-10T02:49:18.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:18 smithi188 ceph-mon[36599]: fsmap 3 up:standby 2024-06-10T02:49:18.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:18 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-06-10T02:49:19.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:18 smithi120 ceph-mon[36634]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T02:49:19.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:18 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:49:19.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:18 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:49:19.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:18 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:49:19.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:18 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-10T02:49:19.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:18 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-10T02:49:19.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:18 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:49:19.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:18 smithi120 ceph-mon[36634]: mds.? [v2:172.21.15.52:6834/889673247,v1:172.21.15.52:6835/889673247] up:boot 2024-06-10T02:49:19.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:18 smithi120 ceph-mon[36634]: fsmap 3 up:standby 2024-06-10T02:49:19.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:18 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-06-10T02:49:19.200 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:49:19.975 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:19 smithi052 ceph-mon[32362]: Deploying daemon mds.d on smithi052 2024-06-10T02:49:20.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:19 smithi120 ceph-mon[36634]: Deploying daemon mds.d on smithi052 2024-06-10T02:49:20.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:19 smithi188 ceph-mon[36599]: Deploying daemon mds.d on smithi052 2024-06-10T02:49:20.966 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:20 smithi052 ceph-mon[32362]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T02:49:21.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:20 smithi120 ceph-mon[36634]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T02:49:21.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:20 smithi188 ceph-mon[36599]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T02:49:21.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:21.519+0000 7f22085e0700 1 -- 172.21.15.52:0/790995702 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2200107840 msgr2=0x7f220010f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:49:21.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:21.519+0000 7f22085e0700 1 --2- 172.21.15.52:0/790995702 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2200107840 0x7f220010f3c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f21f800b0d0 tx=0x7f21f800b3e0 comp rx=0 tx=0).stop 2024-06-10T02:49:21.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:21.520+0000 7f22085e0700 1 -- 172.21.15.52:0/790995702 shutdown_connections 2024-06-10T02:49:21.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:21.520+0000 7f22085e0700 1 --2- 172.21.15.52:0/790995702 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2200107840 0x7f220010f3c0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:49:21.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:21.520+0000 7f22085e0700 1 --2- 172.21.15.52:0/790995702 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2200102740 0x7f2200102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:49:21.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:21.520+0000 7f22085e0700 1 --2- 172.21.15.52:0/790995702 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f22001030f0 0x7f2200107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:49:21.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:21.520+0000 7f22085e0700 1 -- 172.21.15.52:0/790995702 >> 172.21.15.52:0/790995702 conn(0x7f22000795f0 msgr2=0x7f2200079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:49:21.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:21.520+0000 7f22085e0700 1 -- 172.21.15.52:0/790995702 shutdown_connections 2024-06-10T02:49:21.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:21.520+0000 7f22085e0700 1 -- 172.21.15.52:0/790995702 wait complete. 2024-06-10T02:49:21.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:21.520+0000 7f22085e0700 1 Processor -- start 2024-06-10T02:49:21.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:21.520+0000 7f22085e0700 1 -- start start 2024-06-10T02:49:21.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:21.520+0000 7f22085e0700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2200102740 0x7f220019e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:49:21.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:21.521+0000 7f22085e0700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f22001030f0 0x7f220019ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:49:21.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:21.521+0000 7f22085e0700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2200107840 0x7f2200198b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:49:21.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:21.521+0000 7f22085e0700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f2200112850 con 0x7f22001030f0 2024-06-10T02:49:21.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:21.521+0000 7f22085e0700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f22001126d0 con 0x7f2200102740 2024-06-10T02:49:21.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:21.521+0000 7f22085e0700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f22001129d0 con 0x7f2200107840 2024-06-10T02:49:21.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:21.521+0000 7f2205b7b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f22001030f0 0x7f220019ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:49:21.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:21.521+0000 7f2205b7b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f22001030f0 0x7f220019ee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:41516/0 (socket says 172.21.15.52:41516) 2024-06-10T02:49:21.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:21.521+0000 7f2205b7b700 1 -- 172.21.15.52:0/2360084206 learned_addr learned my addr 172.21.15.52:0/2360084206 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:49:21.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:21.521+0000 7f2206b7d700 1 --2- 172.21.15.52:0/2360084206 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2200107840 0x7f2200198b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:49:21.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:21.521+0000 7f220637c700 1 --2- 172.21.15.52:0/2360084206 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2200102740 0x7f220019e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:49:21.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:21.521+0000 7f2205b7b700 1 -- 172.21.15.52:0/2360084206 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2200107840 msgr2=0x7f2200198b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:49:21.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:21.521+0000 7f2205b7b700 1 --2- 172.21.15.52:0/2360084206 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2200107840 0x7f2200198b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:49:21.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:21.521+0000 7f2205b7b700 1 -- 172.21.15.52:0/2360084206 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2200102740 msgr2=0x7f220019e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:49:21.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:21.521+0000 7f2205b7b700 1 --2- 172.21.15.52:0/2360084206 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2200102740 0x7f220019e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:49:21.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:21.521+0000 7f2205b7b700 1 -- 172.21.15.52:0/2360084206 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2200199290 con 0x7f22001030f0 2024-06-10T02:49:21.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:21.521+0000 7f2206b7d700 1 --2- 172.21.15.52:0/2360084206 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2200107840 0x7f2200198b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T02:49:21.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:21.521+0000 7f2205b7b700 1 --2- 172.21.15.52:0/2360084206 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f22001030f0 0x7f220019ee60 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f21f000b530 tx=0x7f21f000b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:49:21.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:21.521+0000 7f21f77fe700 1 -- 172.21.15.52:0/2360084206 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21f0015020 con 0x7f22001030f0 2024-06-10T02:49:21.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:21.521+0000 7f22085e0700 1 -- 172.21.15.52:0/2360084206 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2200199580 con 0x7f22001030f0 2024-06-10T02:49:21.523 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:21.521+0000 7f21f77fe700 1 -- 172.21.15.52:0/2360084206 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f21f0011d10 con 0x7f22001030f0 2024-06-10T02:49:21.523 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:21.521+0000 7f21f77fe700 1 -- 172.21.15.52:0/2360084206 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21f0010e50 con 0x7f22001030f0 2024-06-10T02:49:21.523 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:21.521+0000 7f22085e0700 1 -- 172.21.15.52:0/2360084206 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22001a57f0 con 0x7f22001030f0 2024-06-10T02:49:21.524 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:21.522+0000 7f22085e0700 1 -- 172.21.15.52:0/2360084206 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2200069180 con 0x7f22001030f0 2024-06-10T02:49:21.524 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:21.524+0000 7f21f77fe700 1 -- 172.21.15.52:0/2360084206 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f21f0018020 con 0x7f22001030f0 2024-06-10T02:49:21.524 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:21.525+0000 7f21f77fe700 1 --2- 172.21.15.52:0/2360084206 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f21ec0a2dc0 0x7f21ec0a5280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:49:21.524 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:21.525+0000 7f220637c700 1 --2- 172.21.15.52:0/2360084206 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f21ec0a2dc0 0x7f21ec0a5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:49:21.524 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:21.525+0000 7f21f77fe700 1 -- 172.21.15.52:0/2360084206 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f21f00ce9f0 con 0x7f22001030f0 2024-06-10T02:49:21.524 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:21.526+0000 7f220637c700 1 --2- 172.21.15.52:0/2360084206 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f21ec0a2dc0 0x7f21ec0a5280 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f21fc006fd0 tx=0x7f21fc008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:49:21.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:21.529+0000 7f21f77fe700 1 -- 172.21.15.52:0/2360084206 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f21f001f020 con 0x7f22001030f0 2024-06-10T02:49:21.782 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:21.783+0000 7f22085e0700 1 -- 172.21.15.52:0/2360084206 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f220002dca0 con 0x7f22001030f0 2024-06-10T02:49:21.782 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:21.784+0000 7f21f77fe700 1 -- 172.21.15.52:0/2360084206 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3043 (secure 0 0 0) 0x7f21f009bdf0 con 0x7f22001030f0 2024-06-10T02:49:21.782 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-10T02:49:21.783 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/889673247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":889673247},{"type":"v1","addr":"172.21.15.52:6835","nonce":889673247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24383,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6833/2609002896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":2609002896},{"type":"v1","addr":"172.21.15.120:6833","nonce":2609002896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/529564979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":529564979},{"type":"v1","addr":"172.21.15.188:6833","nonce":529564979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-06-10T02:49:21.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:21.786+0000 7f22085e0700 1 -- 172.21.15.52:0/2360084206 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f21ec0a2dc0 msgr2=0x7f21ec0a5280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:49:21.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:21.786+0000 7f22085e0700 1 --2- 172.21.15.52:0/2360084206 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f21ec0a2dc0 0x7f21ec0a5280 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f21fc006fd0 tx=0x7f21fc008040 comp rx=0 tx=0).stop 2024-06-10T02:49:21.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:21.786+0000 7f22085e0700 1 -- 172.21.15.52:0/2360084206 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f22001030f0 msgr2=0x7f220019ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:49:21.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:21.786+0000 7f22085e0700 1 --2- 172.21.15.52:0/2360084206 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f22001030f0 0x7f220019ee60 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f21f000b530 tx=0x7f21f000b8f0 comp rx=0 tx=0).stop 2024-06-10T02:49:21.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:21.786+0000 7f22085e0700 1 -- 172.21.15.52:0/2360084206 shutdown_connections 2024-06-10T02:49:21.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:21.786+0000 7f22085e0700 1 --2- 172.21.15.52:0/2360084206 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2200107840 0x7f2200198b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:49:21.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:21.786+0000 7f22085e0700 1 --2- 172.21.15.52:0/2360084206 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2200102740 0x7f220019e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:49:21.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:21.786+0000 7f22085e0700 1 --2- 172.21.15.52:0/2360084206 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f21ec0a2dc0 0x7f21ec0a5280 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:49:21.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:21.786+0000 7f22085e0700 1 --2- 172.21.15.52:0/2360084206 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f22001030f0 0x7f220019ee60 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:49:21.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:21.786+0000 7f22085e0700 1 -- 172.21.15.52:0/2360084206 >> 172.21.15.52:0/2360084206 conn(0x7f22000795f0 msgr2=0x7f2200100700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:49:21.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:21.786+0000 7f22085e0700 1 -- 172.21.15.52:0/2360084206 shutdown_connections 2024-06-10T02:49:21.786 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:21.787+0000 7f22085e0700 1 -- 172.21.15.52:0/2360084206 wait complete. 2024-06-10T02:49:21.786 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 4 2024-06-10T02:49:21.916 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:21 smithi052 ceph-mon[32362]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T02:49:21.916 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:21 smithi052 ceph-mon[32362]: from='client.? 172.21.15.52:0/2360084206' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T02:49:22.136 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:21 smithi120 ceph-mon[36634]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T02:49:22.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:21 smithi120 ceph-mon[36634]: from='client.? 172.21.15.52:0/2360084206' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T02:49:22.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:21 smithi188 ceph-mon[36599]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T02:49:22.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:21 smithi188 ceph-mon[36599]: from='client.? 172.21.15.52:0/2360084206' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T02:49:23.372 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:23 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:49:23.372 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:23 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:49:23.372 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:23 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:49:23.372 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:23 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-10T02:49:23.372 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:23 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-10T02:49:23.373 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:23 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:49:23.373 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:23 smithi188 ceph-mon[36599]: Deploying daemon mds.e on smithi120 2024-06-10T02:49:23.373 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:23 smithi188 ceph-mon[36599]: mds.? [v2:172.21.15.52:6836/3615663511,v1:172.21.15.52:6837/3615663511] up:boot 2024-06-10T02:49:23.373 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:23 smithi188 ceph-mon[36599]: fsmap 4 up:standby 2024-06-10T02:49:23.373 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:23 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-06-10T02:49:23.387 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:23 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:49:23.387 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:23 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:49:23.387 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:23 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:49:23.387 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:23 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-10T02:49:23.387 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:23 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-10T02:49:23.387 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:23 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:49:23.387 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:23 smithi120 ceph-mon[36634]: Deploying daemon mds.e on smithi120 2024-06-10T02:49:23.387 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:23 smithi120 ceph-mon[36634]: mds.? [v2:172.21.15.52:6836/3615663511,v1:172.21.15.52:6837/3615663511] up:boot 2024-06-10T02:49:23.387 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:23 smithi120 ceph-mon[36634]: fsmap 4 up:standby 2024-06-10T02:49:23.387 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:23 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-06-10T02:49:23.475 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:23 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:49:23.475 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:23 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:49:23.475 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:23 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:49:23.476 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:23 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-10T02:49:23.476 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:23 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-10T02:49:23.476 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:23 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:49:23.476 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:23 smithi052 ceph-mon[32362]: Deploying daemon mds.e on smithi120 2024-06-10T02:49:23.476 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:23 smithi052 ceph-mon[32362]: mds.? [v2:172.21.15.52:6836/3615663511,v1:172.21.15.52:6837/3615663511] up:boot 2024-06-10T02:49:23.476 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:23 smithi052 ceph-mon[32362]: fsmap 4 up:standby 2024-06-10T02:49:23.476 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:23 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-06-10T02:49:24.340 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:24 smithi120 ceph-mon[36634]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T02:49:24.452 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:24 smithi188 ceph-mon[36599]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T02:49:24.475 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:24 smithi052 ceph-mon[32362]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T02:49:24.637 INFO:journalctl@ceph.mds.e.smithi120.stdout:Jun 10 02:49:24 smithi120 podman[65373]: 2024-06-10 02:49:24.577934938 +0000 UTC m=+0.133993590 container create e38b88a6eb058e63a998296d4d14bbe6e4418284f925720c67fdd23acf4f1781 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mds-e, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-06-10T02:49:24.688 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T02:49:24.956 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:49:25.137 INFO:journalctl@ceph.mds.e.smithi120.stdout:Jun 10 02:49:24 smithi120 podman[65373]: 2024-06-10 02:49:24.822388097 +0000 UTC m=+0.378446744 container init e38b88a6eb058e63a998296d4d14bbe6e4418284f925720c67fdd23acf4f1781 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mds-e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-06-10T02:49:25.137 INFO:journalctl@ceph.mds.e.smithi120.stdout:Jun 10 02:49:24 smithi120 podman[65373]: 2024-06-10 02:49:24.824964398 +0000 UTC m=+0.381023054 container start e38b88a6eb058e63a998296d4d14bbe6e4418284f925720c67fdd23acf4f1781 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mds-e, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-10T02:49:25.137 INFO:journalctl@ceph.mds.e.smithi120.stdout:Jun 10 02:49:24 smithi120 ceph-mds[65393]: -- 172.21.15.120:0/964654172 <== mon.2 v2:172.21.15.120:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557929695a40 con 0x557929668c00 2024-06-10T02:49:25.137 INFO:journalctl@ceph.mds.e.smithi120.stdout:Jun 10 02:49:24 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mds-e[65389]: starting mds.e at 2024-06-10T02:49:25.137 INFO:journalctl@ceph.mds.e.smithi120.stdout:Jun 10 02:49:24 smithi120 bash[65373]: e38b88a6eb058e63a998296d4d14bbe6e4418284f925720c67fdd23acf4f1781 2024-06-10T02:49:25.137 INFO:journalctl@ceph.mds.e.smithi120.stdout:Jun 10 02:49:24 smithi120 systemd[1]: Started Ceph mds.e for d8676e9e-26d2-11ef-bc9b-c7b262605968. 2024-06-10T02:49:26.054 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:25 smithi052 ceph-mon[32362]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T02:49:26.054 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:25 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:49:26.054 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:25 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:49:26.054 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:25 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:49:26.054 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:25 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-10T02:49:26.054 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:25 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-10T02:49:26.054 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:25 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:49:26.054 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:25 smithi052 ceph-mon[32362]: Deploying daemon mds.f on smithi188 2024-06-10T02:49:26.054 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:25 smithi052 ceph-mon[32362]: mds.? [v2:172.21.15.120:6834/1996370288,v1:172.21.15.120:6835/1996370288] up:boot 2024-06-10T02:49:26.054 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:25 smithi052 ceph-mon[32362]: fsmap 5 up:standby 2024-06-10T02:49:26.054 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:25 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-06-10T02:49:26.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:25 smithi188 ceph-mon[36599]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T02:49:26.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:25 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:49:26.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:25 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:49:26.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:25 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:49:26.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:25 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-10T02:49:26.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:25 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-10T02:49:26.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:25 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:49:26.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:25 smithi188 ceph-mon[36599]: Deploying daemon mds.f on smithi188 2024-06-10T02:49:26.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:25 smithi188 ceph-mon[36599]: mds.? [v2:172.21.15.120:6834/1996370288,v1:172.21.15.120:6835/1996370288] up:boot 2024-06-10T02:49:26.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:25 smithi188 ceph-mon[36599]: fsmap 5 up:standby 2024-06-10T02:49:26.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:25 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-06-10T02:49:26.387 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:25 smithi120 ceph-mon[36634]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T02:49:26.387 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:25 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:49:26.387 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:25 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:49:26.387 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:25 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:49:26.387 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:25 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-10T02:49:26.387 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:25 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-10T02:49:26.388 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:25 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:49:26.388 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:25 smithi120 ceph-mon[36634]: Deploying daemon mds.f on smithi188 2024-06-10T02:49:26.388 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:25 smithi120 ceph-mon[36634]: mds.? [v2:172.21.15.120:6834/1996370288,v1:172.21.15.120:6835/1996370288] up:boot 2024-06-10T02:49:26.388 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:25 smithi120 ceph-mon[36634]: fsmap 5 up:standby 2024-06-10T02:49:26.388 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:25 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-06-10T02:49:26.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:26.578+0000 7f88f30f3700 1 -- 172.21.15.52:0/3455766983 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f88ec06ab30 msgr2=0x7f88ec10f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:49:26.578 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:26.578+0000 7f88f30f3700 1 --2- 172.21.15.52:0/3455766983 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f88ec06ab30 0x7f88ec10f320 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f88e400b0a0 tx=0x7f88e400b3b0 comp rx=0 tx=0).stop 2024-06-10T02:49:26.578 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:26.578+0000 7f88f30f3700 1 -- 172.21.15.52:0/3455766983 shutdown_connections 2024-06-10T02:49:26.578 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:26.578+0000 7f88f30f3700 1 --2- 172.21.15.52:0/3455766983 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f88ec06ab30 0x7f88ec10f320 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:49:26.578 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:26.578+0000 7f88f30f3700 1 --2- 172.21.15.52:0/3455766983 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f88ec108500 0x7f88ec1088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:49:26.578 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:26.578+0000 7f88f30f3700 1 --2- 172.21.15.52:0/3455766983 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f88ec06a170 0x7f88ec06a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:49:26.578 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:26.578+0000 7f88f30f3700 1 -- 172.21.15.52:0/3455766983 >> 172.21.15.52:0/3455766983 conn(0x7f88ec078340 msgr2=0x7f88ec078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:49:26.578 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:26.578+0000 7f88f30f3700 1 -- 172.21.15.52:0/3455766983 shutdown_connections 2024-06-10T02:49:26.578 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:26.578+0000 7f88f30f3700 1 -- 172.21.15.52:0/3455766983 wait complete. 2024-06-10T02:49:26.578 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:26.579+0000 7f88f30f3700 1 Processor -- start 2024-06-10T02:49:26.578 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:26.579+0000 7f88f30f3700 1 -- start start 2024-06-10T02:49:26.580 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:26.580+0000 7f88f30f3700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f88ec06a170 0x7f88ec198cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:49:26.580 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:26.580+0000 7f88f30f3700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f88ec06ab30 0x7f88ec199210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:49:26.580 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:26.580+0000 7f88f30f3700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f88ec108500 0x7f88ec19d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:49:26.580 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:26.580+0000 7f88f30f3700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f88ec112760 con 0x7f88ec108500 2024-06-10T02:49:26.580 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:26.580+0000 7f88f30f3700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f88ec1125e0 con 0x7f88ec06a170 2024-06-10T02:49:26.580 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:26.580+0000 7f88f30f3700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f88ec1128e0 con 0x7f88ec06ab30 2024-06-10T02:49:26.580 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:26.580+0000 7f88f1690700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f88ec108500 0x7f88ec19d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:49:26.580 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:26.580+0000 7f88f1690700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f88ec108500 0x7f88ec19d580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:41550/0 (socket says 172.21.15.52:41550) 2024-06-10T02:49:26.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:26.580+0000 7f88f0e8f700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f88ec06a170 0x7f88ec198cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:49:26.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:26.580+0000 7f88f1690700 1 -- 172.21.15.52:0/615453967 learned_addr learned my addr 172.21.15.52:0/615453967 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:49:26.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:26.580+0000 7f88ebfff700 1 --2- 172.21.15.52:0/615453967 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f88ec06ab30 0x7f88ec199210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:49:26.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:26.580+0000 7f88f1690700 1 -- 172.21.15.52:0/615453967 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f88ec06ab30 msgr2=0x7f88ec199210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:49:26.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:26.580+0000 7f88f1690700 1 --2- 172.21.15.52:0/615453967 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f88ec06ab30 0x7f88ec199210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:49:26.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:26.580+0000 7f88f1690700 1 -- 172.21.15.52:0/615453967 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f88ec06a170 msgr2=0x7f88ec198cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:49:26.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:26.580+0000 7f88f1690700 1 --2- 172.21.15.52:0/615453967 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f88ec06a170 0x7f88ec198cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:49:26.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:26.580+0000 7f88f1690700 1 -- 172.21.15.52:0/615453967 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88ec19dc80 con 0x7f88ec108500 2024-06-10T02:49:26.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:26.580+0000 7f88f1690700 1 --2- 172.21.15.52:0/615453967 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f88ec108500 0x7f88ec19d580 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f88e40062a0 tx=0x7f88e4004d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:49:26.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:26.580+0000 7f88e9ffb700 1 -- 172.21.15.52:0/615453967 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88e4021070 con 0x7f88ec108500 2024-06-10T02:49:26.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:26.581+0000 7f88f30f3700 1 -- 172.21.15.52:0/615453967 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88ec19df10 con 0x7f88ec108500 2024-06-10T02:49:26.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:26.581+0000 7f88e9ffb700 1 -- 172.21.15.52:0/615453967 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f88e4003c00 con 0x7f88ec108500 2024-06-10T02:49:26.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:26.581+0000 7f88e9ffb700 1 -- 172.21.15.52:0/615453967 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88e4004880 con 0x7f88ec108500 2024-06-10T02:49:26.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:26.581+0000 7f88f30f3700 1 -- 172.21.15.52:0/615453967 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88ec1a5830 con 0x7f88ec108500 2024-06-10T02:49:26.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:26.582+0000 7f88f30f3700 1 -- 172.21.15.52:0/615453967 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88ec069180 con 0x7f88ec108500 2024-06-10T02:49:26.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:26.583+0000 7f88e9ffb700 1 -- 172.21.15.52:0/615453967 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f88e4003db0 con 0x7f88ec108500 2024-06-10T02:49:26.583 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:26.584+0000 7f88e9ffb700 1 --2- 172.21.15.52:0/615453967 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f88d40a2ca0 0x7f88d40a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:49:26.583 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:26.584+0000 7f88f0e8f700 1 --2- 172.21.15.52:0/615453967 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f88d40a2ca0 0x7f88d40a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:49:26.583 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:26.584+0000 7f88e9ffb700 1 -- 172.21.15.52:0/615453967 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f88e400f020 con 0x7f88ec108500 2024-06-10T02:49:26.583 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:26.585+0000 7f88f0e8f700 1 --2- 172.21.15.52:0/615453967 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f88d40a2ca0 0x7f88d40a5160 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f88dc007b60 tx=0x7f88dc0073d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:49:26.586 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:26.588+0000 7f88e9ffb700 1 -- 172.21.15.52:0/615453967 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f88e402a020 con 0x7f88ec108500 2024-06-10T02:49:26.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:26.836+0000 7f88f30f3700 1 -- 172.21.15.52:0/615453967 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f88ec199fe0 con 0x7f88ec108500 2024-06-10T02:49:26.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:26.837+0000 7f88e9ffb700 1 -- 172.21.15.52:0/615453967 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4696 (secure 0 0 0) 0x7f88e409d4a0 con 0x7f88ec108500 2024-06-10T02:49:26.836 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-10T02:49:26.836 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/889673247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":889673247},{"type":"v1","addr":"172.21.15.52:6835","nonce":889673247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/3615663511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":3615663511},{"type":"v1","addr":"172.21.15.52:6837","nonce":3615663511}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24383,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6833/2609002896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":2609002896},{"type":"v1","addr":"172.21.15.120:6833","nonce":2609002896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24401,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/1996370288","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":1996370288},{"type":"v1","addr":"172.21.15.120:6835","nonce":1996370288}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/529564979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":529564979},{"type":"v1","addr":"172.21.15.188:6833","nonce":529564979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-06-10T02:49:26.838 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:26.839+0000 7f88f30f3700 1 -- 172.21.15.52:0/615453967 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f88d40a2ca0 msgr2=0x7f88d40a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:49:26.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:26.839+0000 7f88f30f3700 1 --2- 172.21.15.52:0/615453967 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f88d40a2ca0 0x7f88d40a5160 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f88dc007b60 tx=0x7f88dc0073d0 comp rx=0 tx=0).stop 2024-06-10T02:49:26.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:26.839+0000 7f88f30f3700 1 -- 172.21.15.52:0/615453967 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f88ec108500 msgr2=0x7f88ec19d580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:49:26.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:26.839+0000 7f88f30f3700 1 --2- 172.21.15.52:0/615453967 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f88ec108500 0x7f88ec19d580 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f88e40062a0 tx=0x7f88e4004d70 comp rx=0 tx=0).stop 2024-06-10T02:49:26.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:26.839+0000 7f88f30f3700 1 -- 172.21.15.52:0/615453967 shutdown_connections 2024-06-10T02:49:26.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:26.839+0000 7f88f30f3700 1 --2- 172.21.15.52:0/615453967 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f88d40a2ca0 0x7f88d40a5160 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:49:26.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:26.839+0000 7f88f30f3700 1 --2- 172.21.15.52:0/615453967 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f88ec108500 0x7f88ec19d580 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:49:26.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:26.839+0000 7f88f30f3700 1 --2- 172.21.15.52:0/615453967 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f88ec06a170 0x7f88ec198cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:49:26.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:26.839+0000 7f88f30f3700 1 --2- 172.21.15.52:0/615453967 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f88ec06ab30 0x7f88ec199210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:49:26.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:26.839+0000 7f88f30f3700 1 -- 172.21.15.52:0/615453967 >> 172.21.15.52:0/615453967 conn(0x7f88ec078340 msgr2=0x7f88ec0fffa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:49:26.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:26.839+0000 7f88f30f3700 1 -- 172.21.15.52:0/615453967 shutdown_connections 2024-06-10T02:49:26.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:26.839+0000 7f88f30f3700 1 -- 172.21.15.52:0/615453967 wait complete. 2024-06-10T02:49:26.840 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 6 2024-06-10T02:49:27.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:26 smithi188 ceph-mon[36599]: from='client.? 172.21.15.52:0/615453967' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T02:49:27.202 INFO:journalctl@ceph.mds.f.smithi188.stdout:Jun 10 02:49:27 smithi188 systemd[1]: Starting Ceph mds.f for d8676e9e-26d2-11ef-bc9b-c7b262605968... 2024-06-10T02:49:27.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:26 smithi052 ceph-mon[32362]: from='client.? 172.21.15.52:0/615453967' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T02:49:27.319 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:26 smithi120 ceph-mon[36634]: from='client.? 172.21.15.52:0/615453967' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T02:49:27.621 INFO:journalctl@ceph.mds.f.smithi188.stdout:Jun 10 02:49:27 smithi188 podman[66324]: 2024-06-10 02:49:27.256849978 +0000 UTC m=+0.017330501 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-10T02:49:27.622 INFO:journalctl@ceph.mds.f.smithi188.stdout:Jun 10 02:49:27 smithi188 podman[66324]: 2024-06-10 02:49:27.370280481 +0000 UTC m=+0.130760994 container create d833706531e889d802362fdafb3fd63f13f00b56823861510e78ba58fd1d9a04 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mds-f, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-06-10T02:49:27.622 INFO:journalctl@ceph.mds.f.smithi188.stdout:Jun 10 02:49:27 smithi188 podman[66324]: 2024-06-10 02:49:27.597904213 +0000 UTC m=+0.358384723 container init d833706531e889d802362fdafb3fd63f13f00b56823861510e78ba58fd1d9a04 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mds-f, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-06-10T02:49:27.622 INFO:journalctl@ceph.mds.f.smithi188.stdout:Jun 10 02:49:27 smithi188 podman[66324]: 2024-06-10 02:49:27.600571783 +0000 UTC m=+0.361052296 container start d833706531e889d802362fdafb3fd63f13f00b56823861510e78ba58fd1d9a04 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mds-f, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=) 2024-06-10T02:49:27.622 INFO:journalctl@ceph.mds.f.smithi188.stdout:Jun 10 02:49:27 smithi188 ceph-mds[66343]: -- 172.21.15.188:0/202580976 <== mon.1 v2:172.21.15.188:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558e1d5b1a40 con 0x558e1d584c00 2024-06-10T02:49:27.952 INFO:journalctl@ceph.mds.f.smithi188.stdout:Jun 10 02:49:27 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mds-f[66339]: starting mds.f at 2024-06-10T02:49:27.952 INFO:journalctl@ceph.mds.f.smithi188.stdout:Jun 10 02:49:27 smithi188 bash[66324]: d833706531e889d802362fdafb3fd63f13f00b56823861510e78ba58fd1d9a04 2024-06-10T02:49:27.952 INFO:journalctl@ceph.mds.f.smithi188.stdout:Jun 10 02:49:27 smithi188 systemd[1]: Started Ceph mds.f for d8676e9e-26d2-11ef-bc9b-c7b262605968. 2024-06-10T02:49:28.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:28 smithi188 ceph-mon[36599]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T02:49:28.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:28 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:49:28.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:28 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:49:28.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:28 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:49:28.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:28 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-10T02:49:28.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:28 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-10T02:49:28.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:28 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:49:28.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:28 smithi188 ceph-mon[36599]: Deploying daemon mds.c on smithi188 2024-06-10T02:49:28.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:28 smithi188 ceph-mon[36599]: mds.? [v2:172.21.15.188:6834/3904544259,v1:172.21.15.188:6835/3904544259] up:boot 2024-06-10T02:49:28.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:28 smithi188 ceph-mon[36599]: fsmap 6 up:standby 2024-06-10T02:49:28.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:28 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-06-10T02:49:29.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:28 smithi120 ceph-mon[36634]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T02:49:29.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:28 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:49:29.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:28 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:49:29.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:28 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:49:29.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:28 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-10T02:49:29.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:28 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-10T02:49:29.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:28 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:49:29.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:28 smithi120 ceph-mon[36634]: Deploying daemon mds.c on smithi188 2024-06-10T02:49:29.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:28 smithi120 ceph-mon[36634]: mds.? [v2:172.21.15.188:6834/3904544259,v1:172.21.15.188:6835/3904544259] up:boot 2024-06-10T02:49:29.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:28 smithi120 ceph-mon[36634]: fsmap 6 up:standby 2024-06-10T02:49:29.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:28 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-06-10T02:49:29.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:28 smithi052 ceph-mon[32362]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T02:49:29.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:28 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:49:29.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:28 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:49:29.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:28 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:49:29.226 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:28 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-10T02:49:29.226 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:28 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-10T02:49:29.226 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:28 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:49:29.226 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:28 smithi052 ceph-mon[32362]: Deploying daemon mds.c on smithi188 2024-06-10T02:49:29.226 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:28 smithi052 ceph-mon[32362]: mds.? [v2:172.21.15.188:6834/3904544259,v1:172.21.15.188:6835/3904544259] up:boot 2024-06-10T02:49:29.226 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:28 smithi052 ceph-mon[32362]: fsmap 6 up:standby 2024-06-10T02:49:29.226 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:28 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-06-10T02:49:29.533 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T02:49:29.797 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:49:30.128 INFO:journalctl@ceph.mds.c.smithi188.stdout:Jun 10 02:49:29 smithi188 systemd[1]: Starting Ceph mds.c for d8676e9e-26d2-11ef-bc9b-c7b262605968... 2024-06-10T02:49:30.385 INFO:journalctl@ceph.mds.c.smithi188.stdout:Jun 10 02:49:30 smithi188 podman[66627]: 2024-06-10 02:49:30.03344698 +0000 UTC m=+0.018036971 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-10T02:49:30.386 INFO:journalctl@ceph.mds.c.smithi188.stdout:Jun 10 02:49:30 smithi188 podman[66627]: 2024-06-10 02:49:30.135263894 +0000 UTC m=+0.119853916 container create 714d5a7132bb0e96d2c39b78cd361de88975ce3fb7a258a0d18ddbffee93c38b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mds-c, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc) 2024-06-10T02:49:30.702 INFO:journalctl@ceph.mds.c.smithi188.stdout:Jun 10 02:49:30 smithi188 podman[66627]: 2024-06-10 02:49:30.387951986 +0000 UTC m=+0.372541969 container init 714d5a7132bb0e96d2c39b78cd361de88975ce3fb7a258a0d18ddbffee93c38b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mds-c, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-06-10T02:49:30.702 INFO:journalctl@ceph.mds.c.smithi188.stdout:Jun 10 02:49:30 smithi188 podman[66627]: 2024-06-10 02:49:30.390824963 +0000 UTC m=+0.375414953 container start 714d5a7132bb0e96d2c39b78cd361de88975ce3fb7a258a0d18ddbffee93c38b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mds-c, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-06-10T02:49:30.702 INFO:journalctl@ceph.mds.c.smithi188.stdout:Jun 10 02:49:30 smithi188 ceph-mds[66646]: -- 172.21.15.188:0/3591464418 <== mon.1 v2:172.21.15.188:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557c75baba40 con 0x557c75b7f000 2024-06-10T02:49:30.702 INFO:journalctl@ceph.mds.c.smithi188.stdout:Jun 10 02:49:30 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mds-c[66642]: starting mds.c at 2024-06-10T02:49:30.703 INFO:journalctl@ceph.mds.c.smithi188.stdout:Jun 10 02:49:30 smithi188 bash[66627]: 714d5a7132bb0e96d2c39b78cd361de88975ce3fb7a258a0d18ddbffee93c38b 2024-06-10T02:49:30.703 INFO:journalctl@ceph.mds.c.smithi188.stdout:Jun 10 02:49:30 smithi188 systemd[1]: Started Ceph mds.c for d8676e9e-26d2-11ef-bc9b-c7b262605968. 2024-06-10T02:49:31.106 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:30 smithi052 ceph-mon[32362]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T02:49:31.106 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:30 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:49:31.106 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:30 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:49:31.106 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:30 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:49:31.106 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:30 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-10T02:49:31.106 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:30 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-10T02:49:31.106 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:30 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:49:31.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:30 smithi120 ceph-mon[36634]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T02:49:31.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:30 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:49:31.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:30 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:49:31.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:30 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:49:31.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:30 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-10T02:49:31.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:30 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-10T02:49:31.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:30 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:49:31.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:30 smithi188 ceph-mon[36599]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T02:49:31.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:30 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:49:31.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:30 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:49:31.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:30 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:49:31.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:30 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-10T02:49:31.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:30 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-10T02:49:31.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:30 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:49:31.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:31.454+0000 7f4640f97700 1 -- 172.21.15.52:0/9642224 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f463c102ef0 msgr2=0x7f463c10f3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:49:31.454 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:31.454+0000 7f4640f97700 1 --2- 172.21.15.52:0/9642224 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f463c102ef0 0x7f463c10f3a0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f463400b0d0 tx=0x7f463400b3e0 comp rx=0 tx=0).stop 2024-06-10T02:49:31.454 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:31.454+0000 7f4640f97700 1 -- 172.21.15.52:0/9642224 shutdown_connections 2024-06-10T02:49:31.454 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:31.454+0000 7f4640f97700 1 --2- 172.21.15.52:0/9642224 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f463c102ef0 0x7f463c10f3a0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:49:31.454 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:31.454+0000 7f4640f97700 1 --2- 172.21.15.52:0/9642224 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f463c1085d0 0x7f463c1089b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:49:31.454 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:31.454+0000 7f4640f97700 1 --2- 172.21.15.52:0/9642224 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f463c102530 0x7f463c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:49:31.454 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:31.454+0000 7f4640f97700 1 -- 172.21.15.52:0/9642224 >> 172.21.15.52:0/9642224 conn(0x7f463c079640 msgr2=0x7f463c079a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:49:31.454 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:31.454+0000 7f4640f97700 1 -- 172.21.15.52:0/9642224 shutdown_connections 2024-06-10T02:49:31.454 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:31.454+0000 7f4640f97700 1 -- 172.21.15.52:0/9642224 wait complete. 2024-06-10T02:49:31.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:31.455+0000 7f4640f97700 1 Processor -- start 2024-06-10T02:49:31.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:31.455+0000 7f4640f97700 1 -- start start 2024-06-10T02:49:31.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:31.455+0000 7f4640f97700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f463c102530 0x7f463c074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:49:31.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:31.455+0000 7f4640f97700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f463c102ef0 0x7f463c074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:49:31.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:31.455+0000 7f4640f97700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f463c1085d0 0x7f463c075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:49:31.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:31.455+0000 7f4640f97700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f463c112810 con 0x7f463c1085d0 2024-06-10T02:49:31.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:31.455+0000 7f4640f97700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f463c112690 con 0x7f463c102530 2024-06-10T02:49:31.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:31.455+0000 7f4640f97700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f463c112990 con 0x7f463c102ef0 2024-06-10T02:49:31.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:31.455+0000 7f463ad9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f463c1085d0 0x7f463c075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:49:31.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:31.455+0000 7f463ad9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f463c1085d0 0x7f463c075210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55562/0 (socket says 172.21.15.52:55562) 2024-06-10T02:49:31.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:31.455+0000 7f463ad9d700 1 -- 172.21.15.52:0/2611648196 learned_addr learned my addr 172.21.15.52:0/2611648196 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:49:31.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:31.456+0000 7f4631d9b700 1 --2- 172.21.15.52:0/2611648196 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f463c102ef0 0x7f463c074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:49:31.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:31.456+0000 7f463a59c700 1 --2- 172.21.15.52:0/2611648196 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f463c102530 0x7f463c074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:49:31.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:31.456+0000 7f463ad9d700 1 -- 172.21.15.52:0/2611648196 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f463c102ef0 msgr2=0x7f463c074cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:49:31.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:31.456+0000 7f463ad9d700 1 --2- 172.21.15.52:0/2611648196 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f463c102ef0 0x7f463c074cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:49:31.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:31.456+0000 7f463ad9d700 1 -- 172.21.15.52:0/2611648196 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f463c102530 msgr2=0x7f463c074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:49:31.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:31.456+0000 7f463ad9d700 1 --2- 172.21.15.52:0/2611648196 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f463c102530 0x7f463c074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:49:31.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:31.456+0000 7f463ad9d700 1 -- 172.21.15.52:0/2611648196 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f463c1a7060 con 0x7f463c1085d0 2024-06-10T02:49:31.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:31.456+0000 7f463ad9d700 1 --2- 172.21.15.52:0/2611648196 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f463c1085d0 0x7f463c075210 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f4634006070 tx=0x7f4634004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:49:31.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:31.456+0000 7f4633fff700 1 -- 172.21.15.52:0/2611648196 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4634021070 con 0x7f463c1085d0 2024-06-10T02:49:31.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:31.456+0000 7f4633fff700 1 -- 172.21.15.52:0/2611648196 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4634007b80 con 0x7f463c1085d0 2024-06-10T02:49:31.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:31.456+0000 7f4640f97700 1 -- 172.21.15.52:0/2611648196 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f463c1a72f0 con 0x7f463c1085d0 2024-06-10T02:49:31.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:31.456+0000 7f4633fff700 1 -- 172.21.15.52:0/2611648196 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46340052a0 con 0x7f463c1085d0 2024-06-10T02:49:31.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:31.456+0000 7f4640f97700 1 -- 172.21.15.52:0/2611648196 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f463c1a77a0 con 0x7f463c1085d0 2024-06-10T02:49:31.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:31.457+0000 7f4640f97700 1 -- 172.21.15.52:0/2611648196 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f463c103c40 con 0x7f463c1085d0 2024-06-10T02:49:31.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:31.459+0000 7f4633fff700 1 -- 172.21.15.52:0/2611648196 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f4634024070 con 0x7f463c1085d0 2024-06-10T02:49:31.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:31.459+0000 7f4633fff700 1 --2- 172.21.15.52:0/2611648196 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f46280a2c20 0x7f46280a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:49:31.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:31.459+0000 7f463a59c700 1 --2- 172.21.15.52:0/2611648196 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f46280a2c20 0x7f46280a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:49:31.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:31.459+0000 7f4633fff700 1 -- 172.21.15.52:0/2611648196 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f463400f020 con 0x7f463c1085d0 2024-06-10T02:49:31.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:31.460+0000 7f463a59c700 1 --2- 172.21.15.52:0/2611648196 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f46280a2c20 0x7f46280a50e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f462c007b60 tx=0x7f462c0073d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:49:31.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:31.463+0000 7f4633fff700 1 -- 172.21.15.52:0/2611648196 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f463402b020 con 0x7f463c1085d0 2024-06-10T02:49:31.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:31.713+0000 7f4640f97700 1 -- 172.21.15.52:0/2611648196 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f463c075c70 con 0x7f463c1085d0 2024-06-10T02:49:31.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:31.714+0000 7f4633fff700 1 -- 172.21.15.52:0/2611648196 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6349 (secure 0 0 0) 0x7f463409da40 con 0x7f463c1085d0 2024-06-10T02:49:31.713 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-10T02:49:31.713 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/889673247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":889673247},{"type":"v1","addr":"172.21.15.52:6835","nonce":889673247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/3615663511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":3615663511},{"type":"v1","addr":"172.21.15.52:6837","nonce":3615663511}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24383,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6833/2609002896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":2609002896},{"type":"v1","addr":"172.21.15.120:6833","nonce":2609002896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24401,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/1996370288","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":1996370288},{"type":"v1","addr":"172.21.15.120:6835","nonce":1996370288}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/529564979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":529564979},{"type":"v1","addr":"172.21.15.188:6833","nonce":529564979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24463,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/3904544259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":3904544259},{"type":"v1","addr":"172.21.15.188:6835","nonce":3904544259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24469,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/819317726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":819317726},{"type":"v1","addr":"172.21.15.188:6837","nonce":819317726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-06-10T02:49:31.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:31.716+0000 7f4640f97700 1 -- 172.21.15.52:0/2611648196 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f46280a2c20 msgr2=0x7f46280a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:49:31.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:31.716+0000 7f4640f97700 1 --2- 172.21.15.52:0/2611648196 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f46280a2c20 0x7f46280a50e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f462c007b60 tx=0x7f462c0073d0 comp rx=0 tx=0).stop 2024-06-10T02:49:31.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:31.716+0000 7f4640f97700 1 -- 172.21.15.52:0/2611648196 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f463c1085d0 msgr2=0x7f463c075210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:49:31.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:31.716+0000 7f4640f97700 1 --2- 172.21.15.52:0/2611648196 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f463c1085d0 0x7f463c075210 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f4634006070 tx=0x7f4634004d00 comp rx=0 tx=0).stop 2024-06-10T02:49:31.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:31.716+0000 7f4640f97700 1 -- 172.21.15.52:0/2611648196 shutdown_connections 2024-06-10T02:49:31.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:31.716+0000 7f4640f97700 1 --2- 172.21.15.52:0/2611648196 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f46280a2c20 0x7f46280a50e0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:49:31.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:31.716+0000 7f4640f97700 1 --2- 172.21.15.52:0/2611648196 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f463c1085d0 0x7f463c075210 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:49:31.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:31.716+0000 7f4640f97700 1 --2- 172.21.15.52:0/2611648196 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f463c102530 0x7f463c074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:49:31.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:31.716+0000 7f4640f97700 1 --2- 172.21.15.52:0/2611648196 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f463c102ef0 0x7f463c074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:49:31.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:31.716+0000 7f4640f97700 1 -- 172.21.15.52:0/2611648196 >> 172.21.15.52:0/2611648196 conn(0x7f463c079640 msgr2=0x7f463c100fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:49:31.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:31.716+0000 7f4640f97700 1 -- 172.21.15.52:0/2611648196 shutdown_connections 2024-06-10T02:49:31.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:31.716+0000 7f4640f97700 1 -- 172.21.15.52:0/2611648196 wait complete. 2024-06-10T02:49:31.717 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 8 2024-06-10T02:49:31.975 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:31 smithi052 ceph-mon[32362]: Deploying daemon mds.h on smithi120 2024-06-10T02:49:31.975 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:31 smithi052 ceph-mon[32362]: mds.? [v2:172.21.15.188:6836/819317726,v1:172.21.15.188:6837/819317726] up:boot 2024-06-10T02:49:31.975 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:31 smithi052 ceph-mon[32362]: fsmap 7 up:standby 2024-06-10T02:49:31.976 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:31 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-06-10T02:49:31.976 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:31 smithi052 ceph-mon[32362]: from='client.? 172.21.15.52:0/2611648196' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T02:49:32.136 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:31 smithi120 ceph-mon[36634]: Deploying daemon mds.h on smithi120 2024-06-10T02:49:32.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:31 smithi120 ceph-mon[36634]: mds.? [v2:172.21.15.188:6836/819317726,v1:172.21.15.188:6837/819317726] up:boot 2024-06-10T02:49:32.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:31 smithi120 ceph-mon[36634]: fsmap 7 up:standby 2024-06-10T02:49:32.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:31 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-06-10T02:49:32.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:31 smithi120 ceph-mon[36634]: from='client.? 172.21.15.52:0/2611648196' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T02:49:32.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:31 smithi188 ceph-mon[36599]: Deploying daemon mds.h on smithi120 2024-06-10T02:49:32.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:31 smithi188 ceph-mon[36599]: mds.? [v2:172.21.15.188:6836/819317726,v1:172.21.15.188:6837/819317726] up:boot 2024-06-10T02:49:32.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:31 smithi188 ceph-mon[36599]: fsmap 7 up:standby 2024-06-10T02:49:32.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:31 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-06-10T02:49:32.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:31 smithi188 ceph-mon[36599]: from='client.? 172.21.15.52:0/2611648196' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T02:49:32.887 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:32 smithi120 ceph-mon[36634]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T02:49:32.975 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:32 smithi052 ceph-mon[32362]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T02:49:33.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:32 smithi188 ceph-mon[36599]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T02:49:33.637 INFO:journalctl@ceph.mds.h.smithi120.stdout:Jun 10 02:49:33 smithi120 systemd[1]: Started Ceph mds.h for d8676e9e-26d2-11ef-bc9b-c7b262605968. 2024-06-10T02:49:34.382 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T02:49:34.650 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:49:34.702 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:34 smithi188 ceph-mon[36599]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T02:49:34.702 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:34 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:49:34.702 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:34 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:49:34.702 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:34 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:49:34.702 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:34 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-10T02:49:34.702 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:34 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-10T02:49:34.703 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:34 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:49:34.703 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:34 smithi188 ceph-mon[36599]: Deploying daemon mds.i on smithi188 2024-06-10T02:49:34.703 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:34 smithi188 ceph-mon[36599]: mds.? [v2:172.21.15.120:6836/4132440952,v1:172.21.15.120:6837/4132440952] up:boot 2024-06-10T02:49:34.703 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:34 smithi188 ceph-mon[36599]: fsmap 8 up:standby 2024-06-10T02:49:34.703 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:34 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-06-10T02:49:34.725 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:34 smithi052 ceph-mon[32362]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T02:49:34.725 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:34 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:49:34.725 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:34 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:49:34.725 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:34 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:49:34.725 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:34 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-10T02:49:34.725 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:34 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-10T02:49:34.726 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:34 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:49:34.726 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:34 smithi052 ceph-mon[32362]: Deploying daemon mds.i on smithi188 2024-06-10T02:49:34.726 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:34 smithi052 ceph-mon[32362]: mds.? [v2:172.21.15.120:6836/4132440952,v1:172.21.15.120:6837/4132440952] up:boot 2024-06-10T02:49:34.726 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:34 smithi052 ceph-mon[32362]: fsmap 8 up:standby 2024-06-10T02:49:34.726 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:34 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-06-10T02:49:34.887 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:34 smithi120 ceph-mon[36634]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T02:49:34.887 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:34 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:49:34.887 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:34 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:49:34.887 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:34 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:49:34.888 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:34 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-10T02:49:34.888 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:34 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-10T02:49:34.888 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:34 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:49:34.888 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:34 smithi120 ceph-mon[36634]: Deploying daemon mds.i on smithi188 2024-06-10T02:49:34.888 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:34 smithi120 ceph-mon[36634]: mds.? [v2:172.21.15.120:6836/4132440952,v1:172.21.15.120:6837/4132440952] up:boot 2024-06-10T02:49:34.888 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:34 smithi120 ceph-mon[36634]: fsmap 8 up:standby 2024-06-10T02:49:34.888 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:34 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-06-10T02:49:35.887 INFO:journalctl@ceph.mds.i.smithi188.stdout:Jun 10 02:49:35 smithi188 systemd[1]: Starting Ceph mds.i for d8676e9e-26d2-11ef-bc9b-c7b262605968... 2024-06-10T02:49:36.150 INFO:journalctl@ceph.mds.i.smithi188.stdout:Jun 10 02:49:35 smithi188 podman[66931]: 2024-06-10 02:49:35.792416677 +0000 UTC m=+0.017529944 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-10T02:49:36.151 INFO:journalctl@ceph.mds.i.smithi188.stdout:Jun 10 02:49:35 smithi188 podman[66931]: 2024-06-10 02:49:35.932840452 +0000 UTC m=+0.157953703 container create 843470528b5be402a8e6ed580c3d719a17dd4bcfd9fa141870596e14cf8b5e02 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mds-i, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD) 2024-06-10T02:49:36.151 INFO:journalctl@ceph.mds.i.smithi188.stdout:Jun 10 02:49:36 smithi188 podman[66931]: 2024-06-10 02:49:36.152631884 +0000 UTC m=+0.377745133 container init 843470528b5be402a8e6ed580c3d719a17dd4bcfd9fa141870596e14cf8b5e02 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mds-i, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-10T02:49:36.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:36.307+0000 7f659894e700 1 -- 172.21.15.52:0/2076390631 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6594101d30 msgr2=0x7f659410e1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:49:36.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:36.307+0000 7f659894e700 1 --2- 172.21.15.52:0/2076390631 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6594101d30 0x7f659410e1d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f658c00b0a0 tx=0x7f658c00b3b0 comp rx=0 tx=0).stop 2024-06-10T02:49:36.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:36.307+0000 7f659894e700 1 -- 172.21.15.52:0/2076390631 shutdown_connections 2024-06-10T02:49:36.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:36.307+0000 7f659894e700 1 --2- 172.21.15.52:0/2076390631 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6594101d30 0x7f659410e1d0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:49:36.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:36.307+0000 7f659894e700 1 --2- 172.21.15.52:0/2076390631 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6594107420 0x7f6594107800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:49:36.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:36.307+0000 7f659894e700 1 --2- 172.21.15.52:0/2076390631 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6594101370 0x7f65941017f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:49:36.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:36.307+0000 7f659894e700 1 -- 172.21.15.52:0/2076390631 >> 172.21.15.52:0/2076390631 conn(0x7f6594078340 msgr2=0x7f6594078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:49:36.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:36.307+0000 7f659894e700 1 -- 172.21.15.52:0/2076390631 shutdown_connections 2024-06-10T02:49:36.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:36.307+0000 7f659894e700 1 -- 172.21.15.52:0/2076390631 wait complete. 2024-06-10T02:49:36.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:36.308+0000 7f659894e700 1 Processor -- start 2024-06-10T02:49:36.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:36.308+0000 7f659894e700 1 -- start start 2024-06-10T02:49:36.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:36.308+0000 7f659894e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6594101370 0x7f6594199950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:49:36.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:36.308+0000 7f659894e700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6594101d30 0x7f6594199e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:49:36.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:36.308+0000 7f659894e700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6594107420 0x7f65941a49d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:49:36.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:36.308+0000 7f659894e700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f6594111630 con 0x7f6594101370 2024-06-10T02:49:36.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:36.308+0000 7f659894e700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f65941114b0 con 0x7f6594107420 2024-06-10T02:49:36.314 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:36.308+0000 7f659894e700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f65941117b0 con 0x7f6594101d30 2024-06-10T02:49:36.314 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:36.308+0000 7f659259c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6594101370 0x7f6594199950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:49:36.314 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:36.308+0000 7f659259c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6594101370 0x7f6594199950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55578/0 (socket says 172.21.15.52:55578) 2024-06-10T02:49:36.314 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:36.308+0000 7f659259c700 1 -- 172.21.15.52:0/2297329540 learned_addr learned my addr 172.21.15.52:0/2297329540 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:49:36.314 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:36.308+0000 7f6591d9b700 1 --2- 172.21.15.52:0/2297329540 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6594101d30 0x7f6594199e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:49:36.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:36.308+0000 7f6592d9d700 1 --2- 172.21.15.52:0/2297329540 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6594107420 0x7f65941a49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:49:36.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:36.308+0000 7f659259c700 1 -- 172.21.15.52:0/2297329540 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6594101d30 msgr2=0x7f6594199e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:49:36.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:36.308+0000 7f659259c700 1 --2- 172.21.15.52:0/2297329540 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6594101d30 0x7f6594199e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:49:36.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:36.308+0000 7f659259c700 1 -- 172.21.15.52:0/2297329540 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6594107420 msgr2=0x7f65941a49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:49:36.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:36.308+0000 7f659259c700 1 --2- 172.21.15.52:0/2297329540 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6594107420 0x7f65941a49d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:49:36.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:36.308+0000 7f659259c700 1 -- 172.21.15.52:0/2297329540 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65941a4f10 con 0x7f6594101370 2024-06-10T02:49:36.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:36.309+0000 7f659259c700 1 --2- 172.21.15.52:0/2297329540 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6594101370 0x7f6594199950 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f658400c920 tx=0x7f658400cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:49:36.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:36.309+0000 7f65837fe700 1 -- 172.21.15.52:0/2297329540 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6584007b00 con 0x7f6594101370 2024-06-10T02:49:36.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:36.309+0000 7f65837fe700 1 -- 172.21.15.52:0/2297329540 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6584004d10 con 0x7f6594101370 2024-06-10T02:49:36.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:36.309+0000 7f659894e700 1 -- 172.21.15.52:0/2297329540 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65941a5140 con 0x7f6594101370 2024-06-10T02:49:36.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:36.309+0000 7f65837fe700 1 -- 172.21.15.52:0/2297329540 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6584005680 con 0x7f6594101370 2024-06-10T02:49:36.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:36.309+0000 7f659894e700 1 -- 172.21.15.52:0/2297329540 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65941a5650 con 0x7f6594101370 2024-06-10T02:49:36.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:36.310+0000 7f659894e700 1 -- 172.21.15.52:0/2297329540 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6594069180 con 0x7f6594101370 2024-06-10T02:49:36.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:36.311+0000 7f65837fe700 1 -- 172.21.15.52:0/2297329540 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f6584004750 con 0x7f6594101370 2024-06-10T02:49:36.318 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:36.312+0000 7f65837fe700 1 --2- 172.21.15.52:0/2297329540 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f657c0a2c20 0x7f657c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:49:36.318 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:36.312+0000 7f6591d9b700 1 --2- 172.21.15.52:0/2297329540 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f657c0a2c20 0x7f657c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:49:36.318 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:36.312+0000 7f65837fe700 1 -- 172.21.15.52:0/2297329540 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f65840ceaa0 con 0x7f6594101370 2024-06-10T02:49:36.318 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:36.312+0000 7f6591d9b700 1 --2- 172.21.15.52:0/2297329540 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f657c0a2c20 0x7f657c0a50e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f65940789b0 tx=0x7f65880073d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:49:36.318 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:36.316+0000 7f65837fe700 1 -- 172.21.15.52:0/2297329540 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f6584014030 con 0x7f6594101370 2024-06-10T02:49:36.452 INFO:journalctl@ceph.mds.i.smithi188.stdout:Jun 10 02:49:36 smithi188 podman[66931]: 2024-06-10 02:49:36.155791494 +0000 UTC m=+0.380904747 container start 843470528b5be402a8e6ed580c3d719a17dd4bcfd9fa141870596e14cf8b5e02 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mds-i, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-06-10T02:49:36.452 INFO:journalctl@ceph.mds.i.smithi188.stdout:Jun 10 02:49:36 smithi188 ceph-mds[66950]: -- 172.21.15.188:0/3207386665 <== mon.1 v2:172.21.15.188:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55de99643a40 con 0x55de99616c00 2024-06-10T02:49:36.452 INFO:journalctl@ceph.mds.i.smithi188.stdout:Jun 10 02:49:36 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mds-i[66946]: starting mds.i at 2024-06-10T02:49:36.452 INFO:journalctl@ceph.mds.i.smithi188.stdout:Jun 10 02:49:36 smithi188 bash[66931]: 843470528b5be402a8e6ed580c3d719a17dd4bcfd9fa141870596e14cf8b5e02 2024-06-10T02:49:36.452 INFO:journalctl@ceph.mds.i.smithi188.stdout:Jun 10 02:49:36 smithi188 systemd[1]: Started Ceph mds.i for d8676e9e-26d2-11ef-bc9b-c7b262605968. 2024-06-10T02:49:36.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:36.587+0000 7f659894e700 1 -- 172.21.15.52:0/2297329540 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f659402de60 con 0x7f6594101370 2024-06-10T02:49:36.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:36.588+0000 7f65837fe700 1 -- 172.21.15.52:0/2297329540 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8004 (secure 0 0 0) 0x7f658409bea0 con 0x7f6594101370 2024-06-10T02:49:36.587 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-10T02:49:36.588 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/889673247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":889673247},{"type":"v1","addr":"172.21.15.52:6835","nonce":889673247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/3615663511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":3615663511},{"type":"v1","addr":"172.21.15.52:6837","nonce":3615663511}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24383,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6833/2609002896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":2609002896},{"type":"v1","addr":"172.21.15.120:6833","nonce":2609002896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24401,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/1996370288","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":1996370288},{"type":"v1","addr":"172.21.15.120:6835","nonce":1996370288}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/529564979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":529564979},{"type":"v1","addr":"172.21.15.188:6833","nonce":529564979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24428,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/4132440952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":4132440952},{"type":"v1","addr":"172.21.15.120:6837","nonce":4132440952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24463,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/3904544259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":3904544259},{"type":"v1","addr":"172.21.15.188:6835","nonce":3904544259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24469,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/819317726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":819317726},{"type":"v1","addr":"172.21.15.188:6837","nonce":819317726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24484,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6839/310444836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":310444836},{"type":"v1","addr":"172.21.15.188:6839","nonce":310444836}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-06-10T02:49:36.590 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:36.590+0000 7f659894e700 1 -- 172.21.15.52:0/2297329540 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f657c0a2c20 msgr2=0x7f657c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:49:36.590 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:36.590+0000 7f659894e700 1 --2- 172.21.15.52:0/2297329540 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f657c0a2c20 0x7f657c0a50e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f65940789b0 tx=0x7f65880073d0 comp rx=0 tx=0).stop 2024-06-10T02:49:36.590 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:36.590+0000 7f659894e700 1 -- 172.21.15.52:0/2297329540 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6594101370 msgr2=0x7f6594199950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:49:36.590 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:36.590+0000 7f659894e700 1 --2- 172.21.15.52:0/2297329540 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6594101370 0x7f6594199950 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f658400c920 tx=0x7f658400cc30 comp rx=0 tx=0).stop 2024-06-10T02:49:36.590 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:36.590+0000 7f659894e700 1 -- 172.21.15.52:0/2297329540 shutdown_connections 2024-06-10T02:49:36.590 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:36.590+0000 7f659894e700 1 --2- 172.21.15.52:0/2297329540 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6594107420 0x7f65941a49d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:49:36.591 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:36.590+0000 7f659894e700 1 --2- 172.21.15.52:0/2297329540 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f657c0a2c20 0x7f657c0a50e0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:49:36.591 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:36.590+0000 7f659894e700 1 --2- 172.21.15.52:0/2297329540 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6594101370 0x7f6594199950 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:49:36.591 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:36.590+0000 7f659894e700 1 --2- 172.21.15.52:0/2297329540 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6594101d30 0x7f6594199e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:49:36.591 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:36.590+0000 7f659894e700 1 -- 172.21.15.52:0/2297329540 >> 172.21.15.52:0/2297329540 conn(0x7f6594078340 msgr2=0x7f65941000c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:49:36.591 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:36.591+0000 7f659894e700 1 -- 172.21.15.52:0/2297329540 shutdown_connections 2024-06-10T02:49:36.591 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:36.591+0000 7f659894e700 1 -- 172.21.15.52:0/2297329540 wait complete. 2024-06-10T02:49:36.591 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 10 2024-06-10T02:49:36.826 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:36 smithi120 ceph-mon[36634]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T02:49:36.826 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:36 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:49:36.827 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:36 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:49:36.827 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:36 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:49:36.827 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:36 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-10T02:49:36.827 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:36 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-10T02:49:36.827 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:36 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:49:36.902 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:36 smithi052 ceph-mon[32362]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T02:49:36.903 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:36 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:49:36.903 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:36 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:49:36.903 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:36 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:49:36.903 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:36 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-10T02:49:36.903 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:36 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-10T02:49:36.903 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:36 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:49:36.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:36 smithi188 ceph-mon[36599]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T02:49:36.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:36 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:49:36.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:36 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:49:36.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:36 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:49:36.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:36 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-10T02:49:36.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:36 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-10T02:49:36.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:36 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:49:37.599 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:37 smithi188 ceph-mon[36599]: Deploying daemon mds.g on smithi052 2024-06-10T02:49:37.599 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:37 smithi188 ceph-mon[36599]: mds.? [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] up:boot 2024-06-10T02:49:37.600 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:37 smithi188 ceph-mon[36599]: fsmap 9 up:standby 2024-06-10T02:49:37.600 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:37 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-06-10T02:49:37.600 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:37 smithi188 ceph-mon[36599]: from='client.? 172.21.15.52:0/2297329540' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T02:49:37.887 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:37 smithi120 ceph-mon[36634]: Deploying daemon mds.g on smithi052 2024-06-10T02:49:37.887 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:37 smithi120 ceph-mon[36634]: mds.? [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] up:boot 2024-06-10T02:49:37.887 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:37 smithi120 ceph-mon[36634]: fsmap 9 up:standby 2024-06-10T02:49:37.887 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:37 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-06-10T02:49:37.887 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:37 smithi120 ceph-mon[36634]: from='client.? 172.21.15.52:0/2297329540' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T02:49:37.975 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:37 smithi052 ceph-mon[32362]: Deploying daemon mds.g on smithi052 2024-06-10T02:49:37.975 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:37 smithi052 ceph-mon[32362]: mds.? [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] up:boot 2024-06-10T02:49:37.976 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:37 smithi052 ceph-mon[32362]: fsmap 9 up:standby 2024-06-10T02:49:37.976 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:37 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-06-10T02:49:37.976 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:37 smithi052 ceph-mon[32362]: from='client.? 172.21.15.52:0/2297329540' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T02:49:38.622 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:38 smithi052 ceph-mon[32362]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T02:49:38.887 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:38 smithi120 ceph-mon[36634]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T02:49:38.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:38 smithi188 ceph-mon[36599]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T02:49:38.975 INFO:journalctl@ceph.mds.g.smithi052.stdout:Jun 10 02:49:38 smithi052 systemd[1]: Starting Ceph mds.g for d8676e9e-26d2-11ef-bc9b-c7b262605968... 2024-06-10T02:49:39.224 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T02:49:39.307 INFO:journalctl@ceph.mds.g.smithi052.stdout:Jun 10 02:49:39 smithi052 podman[74131]: 2024-06-10 02:49:38.955087802 +0000 UTC m=+0.020907555 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-10T02:49:39.308 INFO:journalctl@ceph.mds.g.smithi052.stdout:Jun 10 02:49:39 smithi052 podman[74131]: 2024-06-10 02:49:39.057406227 +0000 UTC m=+0.123225956 container create 76f195dccc988bd64f4212ed1badb6021f602f5339e3cffb48a3d0daf32636b3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mds-g, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc) 2024-06-10T02:49:39.308 INFO:journalctl@ceph.mds.g.smithi052.stdout:Jun 10 02:49:39 smithi052 podman[74131]: 2024-06-10 02:49:39.310068257 +0000 UTC m=+0.375887998 container init 76f195dccc988bd64f4212ed1badb6021f602f5339e3cffb48a3d0daf32636b3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mds-g, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240528, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-06-10T02:49:39.493 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:49:39.617 INFO:journalctl@ceph.mds.g.smithi052.stdout:Jun 10 02:49:39 smithi052 podman[74131]: 2024-06-10 02:49:39.313373756 +0000 UTC m=+0.379193501 container start 76f195dccc988bd64f4212ed1badb6021f602f5339e3cffb48a3d0daf32636b3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mds-g, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=squid-3d89acc, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux ) 2024-06-10T02:49:39.617 INFO:journalctl@ceph.mds.g.smithi052.stdout:Jun 10 02:49:39 smithi052 ceph-mds[74169]: -- 172.21.15.52:0/3394426351 <== mon.0 v2:172.21.15.52:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55bfd8b63a40 con 0x55bfd8b37000 2024-06-10T02:49:39.617 INFO:journalctl@ceph.mds.g.smithi052.stdout:Jun 10 02:49:39 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mds-g[74147]: starting mds.g at 2024-06-10T02:49:39.617 INFO:journalctl@ceph.mds.g.smithi052.stdout:Jun 10 02:49:39 smithi052 bash[74131]: 76f195dccc988bd64f4212ed1badb6021f602f5339e3cffb48a3d0daf32636b3 2024-06-10T02:49:39.617 INFO:journalctl@ceph.mds.g.smithi052.stdout:Jun 10 02:49:39 smithi052 systemd[1]: Started Ceph mds.g for d8676e9e-26d2-11ef-bc9b-c7b262605968. 2024-06-10T02:49:40.608 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:40 smithi052 ceph-mon[32362]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T02:49:40.608 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:40 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:49:40.608 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:40 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:49:40.608 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:40 smithi052 ceph-mon[32362]: mds.? [v2:172.21.15.52:6838/3059599042,v1:172.21.15.52:6839/3059599042] up:boot 2024-06-10T02:49:40.608 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:40 smithi052 ceph-mon[32362]: fsmap 10 up:standby 2024-06-10T02:49:40.608 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:40 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-06-10T02:49:40.608 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:40 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:49:40.608 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:40 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-10T02:49:40.608 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:40 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-10T02:49:40.609 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:40 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:49:40.609 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:40 smithi052 ceph-mon[32362]: Deploying daemon mds.a on smithi052 2024-06-10T02:49:40.887 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:40 smithi120 ceph-mon[36634]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T02:49:40.887 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:40 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:49:40.887 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:40 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:49:40.887 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:40 smithi120 ceph-mon[36634]: mds.? [v2:172.21.15.52:6838/3059599042,v1:172.21.15.52:6839/3059599042] up:boot 2024-06-10T02:49:40.887 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:40 smithi120 ceph-mon[36634]: fsmap 10 up:standby 2024-06-10T02:49:40.887 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:40 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-06-10T02:49:40.887 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:40 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:49:40.888 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:40 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-10T02:49:40.888 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:40 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-10T02:49:40.888 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:40 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:49:40.888 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:40 smithi120 ceph-mon[36634]: Deploying daemon mds.a on smithi052 2024-06-10T02:49:40.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:40 smithi188 ceph-mon[36599]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T02:49:40.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:40 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:49:40.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:40 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:49:40.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:40 smithi188 ceph-mon[36599]: mds.? [v2:172.21.15.52:6838/3059599042,v1:172.21.15.52:6839/3059599042] up:boot 2024-06-10T02:49:40.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:40 smithi188 ceph-mon[36599]: fsmap 10 up:standby 2024-06-10T02:49:40.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:40 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-06-10T02:49:40.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:40 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:49:40.954 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:40 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-10T02:49:40.954 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:40 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-10T02:49:40.954 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:40 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:49:40.954 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:40 smithi188 ceph-mon[36599]: Deploying daemon mds.a on smithi052 2024-06-10T02:49:41.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:41.347+0000 7ff1be595700 1 -- 172.21.15.52:0/2999997249 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff1b81024e0 msgr2=0x7ff1b8102960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:49:41.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:41.347+0000 7ff1be595700 1 --2- 172.21.15.52:0/2999997249 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff1b81024e0 0x7ff1b8102960 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7ff1ac009990 tx=0x7ff1ac009ca0 comp rx=0 tx=0).stop 2024-06-10T02:49:41.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:41.347+0000 7ff1be595700 1 -- 172.21.15.52:0/2999997249 shutdown_connections 2024-06-10T02:49:41.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:41.347+0000 7ff1be595700 1 --2- 172.21.15.52:0/2999997249 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff1b8102ea0 0x7ff1b810f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:49:41.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:41.347+0000 7ff1be595700 1 --2- 172.21.15.52:0/2999997249 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff1b81085f0 0x7ff1b81089d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:49:41.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:41.347+0000 7ff1be595700 1 --2- 172.21.15.52:0/2999997249 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff1b81024e0 0x7ff1b8102960 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:49:41.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:41.347+0000 7ff1be595700 1 -- 172.21.15.52:0/2999997249 >> 172.21.15.52:0/2999997249 conn(0x7ff1b8079640 msgr2=0x7ff1b8079a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:49:41.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:41.347+0000 7ff1be595700 1 -- 172.21.15.52:0/2999997249 shutdown_connections 2024-06-10T02:49:41.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:41.347+0000 7ff1be595700 1 -- 172.21.15.52:0/2999997249 wait complete. 2024-06-10T02:49:41.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:41.347+0000 7ff1be595700 1 Processor -- start 2024-06-10T02:49:41.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:41.348+0000 7ff1be595700 1 -- start start 2024-06-10T02:49:41.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:41.348+0000 7ff1be595700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff1b81024e0 0x7ff1b819e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:49:41.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:41.348+0000 7ff1be595700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff1b8102ea0 0x7ff1b819ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:49:41.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:41.348+0000 7ff1be595700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff1b81085f0 0x7ff1b8198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:49:41.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:41.348+0000 7ff1be595700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ff1b81127b0 con 0x7ff1b81085f0 2024-06-10T02:49:41.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:41.348+0000 7ff1be595700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff1b8112630 con 0x7ff1b81024e0 2024-06-10T02:49:41.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:41.348+0000 7ff1be595700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ff1b8112930 con 0x7ff1b8102ea0 2024-06-10T02:49:41.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:41.348+0000 7ff1bcb32700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff1b81085f0 0x7ff1b8198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:49:41.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:41.348+0000 7ff1bcb32700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff1b81085f0 0x7ff1b8198a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60778/0 (socket says 172.21.15.52:60778) 2024-06-10T02:49:41.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:41.348+0000 7ff1bcb32700 1 -- 172.21.15.52:0/1391660365 learned_addr learned my addr 172.21.15.52:0/1391660365 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:49:41.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:41.348+0000 7ff1b77fe700 1 --2- 172.21.15.52:0/1391660365 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff1b8102ea0 0x7ff1b819ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:49:41.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:41.348+0000 7ff1b7fff700 1 --2- 172.21.15.52:0/1391660365 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff1b81024e0 0x7ff1b819e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:49:41.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:41.349+0000 7ff1bcb32700 1 -- 172.21.15.52:0/1391660365 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff1b8102ea0 msgr2=0x7ff1b819ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:49:41.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:41.349+0000 7ff1bcb32700 1 --2- 172.21.15.52:0/1391660365 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff1b8102ea0 0x7ff1b819ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:49:41.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:41.349+0000 7ff1bcb32700 1 -- 172.21.15.52:0/1391660365 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff1b81024e0 msgr2=0x7ff1b819e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:49:41.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:41.349+0000 7ff1bcb32700 1 --2- 172.21.15.52:0/1391660365 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff1b81024e0 0x7ff1b819e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:49:41.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:41.349+0000 7ff1bcb32700 1 -- 172.21.15.52:0/1391660365 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1b81991b0 con 0x7ff1b81085f0 2024-06-10T02:49:41.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:41.349+0000 7ff1bcb32700 1 --2- 172.21.15.52:0/1391660365 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff1b81085f0 0x7ff1b8198a50 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ff1b000cca0 tx=0x7ff1b000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:49:41.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:41.349+0000 7ff1b57fa700 1 -- 172.21.15.52:0/1391660365 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1b0016020 con 0x7ff1b81085f0 2024-06-10T02:49:41.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:41.349+0000 7ff1be595700 1 -- 172.21.15.52:0/1391660365 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1b81994a0 con 0x7ff1b81085f0 2024-06-10T02:49:41.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:41.349+0000 7ff1b57fa700 1 -- 172.21.15.52:0/1391660365 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff1b0012420 con 0x7ff1b81085f0 2024-06-10T02:49:41.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:41.349+0000 7ff1b57fa700 1 -- 172.21.15.52:0/1391660365 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1b00115e0 con 0x7ff1b81085f0 2024-06-10T02:49:41.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:41.349+0000 7ff1be595700 1 -- 172.21.15.52:0/1391660365 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1b81a5710 con 0x7ff1b81085f0 2024-06-10T02:49:41.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:41.350+0000 7ff1be595700 1 -- 172.21.15.52:0/1391660365 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1b8069180 con 0x7ff1b81085f0 2024-06-10T02:49:41.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:41.351+0000 7ff1b57fa700 1 -- 172.21.15.52:0/1391660365 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7ff1b0019020 con 0x7ff1b81085f0 2024-06-10T02:49:41.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:41.352+0000 7ff1b57fa700 1 --2- 172.21.15.52:0/1391660365 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7ff1a00a2c20 0x7ff1a00a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:49:41.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:41.352+0000 7ff1b57fa700 1 -- 172.21.15.52:0/1391660365 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7ff1b00cea60 con 0x7ff1b81085f0 2024-06-10T02:49:41.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:41.352+0000 7ff1b7fff700 1 --2- 172.21.15.52:0/1391660365 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7ff1a00a2c20 0x7ff1a00a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:49:41.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:41.353+0000 7ff1b7fff700 1 --2- 172.21.15.52:0/1391660365 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7ff1a00a2c20 0x7ff1a00a50e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7ff1b8108e00 tx=0x7ff1a800b3c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:49:41.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:41.357+0000 7ff1b57fa700 1 -- 172.21.15.52:0/1391660365 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7ff1b0020020 con 0x7ff1b81085f0 2024-06-10T02:49:41.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:41.616+0000 7ff1be595700 1 -- 172.21.15.52:0/1391660365 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff1b802dca0 con 0x7ff1b81085f0 2024-06-10T02:49:41.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:41.617+0000 7ff1b57fa700 1 -- 172.21.15.52:0/1391660365 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8830 (secure 0 0 0) 0x7ff1b009bdc0 con 0x7ff1b81085f0 2024-06-10T02:49:41.779 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-10T02:49:41.779 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/889673247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":889673247},{"type":"v1","addr":"172.21.15.52:6835","nonce":889673247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/3615663511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":3615663511},{"type":"v1","addr":"172.21.15.52:6837","nonce":3615663511}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14571,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6839/3059599042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":3059599042},{"type":"v1","addr":"172.21.15.52:6839","nonce":3059599042}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24383,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6833/2609002896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":2609002896},{"type":"v1","addr":"172.21.15.120:6833","nonce":2609002896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24401,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/1996370288","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":1996370288},{"type":"v1","addr":"172.21.15.120:6835","nonce":1996370288}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/529564979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":529564979},{"type":"v1","addr":"172.21.15.188:6833","nonce":529564979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24428,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/4132440952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":4132440952},{"type":"v1","addr":"172.21.15.120:6837","nonce":4132440952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24463,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/3904544259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":3904544259},{"type":"v1","addr":"172.21.15.188:6835","nonce":3904544259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24469,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/819317726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":819317726},{"type":"v1","addr":"172.21.15.188:6837","nonce":819317726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24484,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6839/310444836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":310444836},{"type":"v1","addr":"172.21.15.188:6839","nonce":310444836}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-06-10T02:49:41.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:41.619+0000 7ff1be595700 1 -- 172.21.15.52:0/1391660365 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7ff1a00a2c20 msgr2=0x7ff1a00a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:49:41.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:41.619+0000 7ff1be595700 1 --2- 172.21.15.52:0/1391660365 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7ff1a00a2c20 0x7ff1a00a50e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7ff1b8108e00 tx=0x7ff1a800b3c0 comp rx=0 tx=0).stop 2024-06-10T02:49:41.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:41.619+0000 7ff1be595700 1 -- 172.21.15.52:0/1391660365 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff1b81085f0 msgr2=0x7ff1b8198a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:49:41.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:41.619+0000 7ff1be595700 1 --2- 172.21.15.52:0/1391660365 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff1b81085f0 0x7ff1b8198a50 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ff1b000cca0 tx=0x7ff1b000a060 comp rx=0 tx=0).stop 2024-06-10T02:49:41.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:41.619+0000 7ff1be595700 1 -- 172.21.15.52:0/1391660365 shutdown_connections 2024-06-10T02:49:41.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:41.619+0000 7ff1be595700 1 --2- 172.21.15.52:0/1391660365 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7ff1a00a2c20 0x7ff1a00a50e0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:49:41.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:41.619+0000 7ff1be595700 1 --2- 172.21.15.52:0/1391660365 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff1b81085f0 0x7ff1b8198a50 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:49:41.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:41.619+0000 7ff1be595700 1 --2- 172.21.15.52:0/1391660365 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff1b81024e0 0x7ff1b819e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:49:41.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:41.619+0000 7ff1be595700 1 --2- 172.21.15.52:0/1391660365 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff1b8102ea0 0x7ff1b819ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:49:41.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:41.619+0000 7ff1be595700 1 -- 172.21.15.52:0/1391660365 >> 172.21.15.52:0/1391660365 conn(0x7ff1b8079640 msgr2=0x7ff1b8100f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:49:41.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:41.619+0000 7ff1be595700 1 -- 172.21.15.52:0/1391660365 shutdown_connections 2024-06-10T02:49:41.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:41.619+0000 7ff1be595700 1 -- 172.21.15.52:0/1391660365 wait complete. 2024-06-10T02:49:41.780 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 11 2024-06-10T02:49:42.725 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:42 smithi052 ceph-mon[32362]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T02:49:42.725 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:42 smithi052 ceph-mon[32362]: from='client.? 172.21.15.52:0/1391660365' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T02:49:42.887 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:42 smithi120 ceph-mon[36634]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T02:49:42.887 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:42 smithi120 ceph-mon[36634]: from='client.? 172.21.15.52:0/1391660365' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T02:49:42.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:42 smithi188 ceph-mon[36599]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T02:49:42.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:42 smithi188 ceph-mon[36599]: from='client.? 172.21.15.52:0/1391660365' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T02:49:43.475 INFO:journalctl@ceph.mds.a.smithi052.stdout:Jun 10 02:49:43 smithi052 systemd[1]: Starting Ceph mds.a for d8676e9e-26d2-11ef-bc9b-c7b262605968... 2024-06-10T02:49:43.807 INFO:journalctl@ceph.mds.a.smithi052.stdout:Jun 10 02:49:43 smithi052 podman[74592]: 2024-06-10 02:49:43.457118334 +0000 UTC m=+0.017971192 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-10T02:49:43.807 INFO:journalctl@ceph.mds.a.smithi052.stdout:Jun 10 02:49:43 smithi052 podman[74592]: 2024-06-10 02:49:43.57076751 +0000 UTC m=+0.131620357 container create 20535a539d25316c74f9e9fc1c30c803e6a62f9a60c575fa97717355850608cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mds-a, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-06-10T02:49:43.807 INFO:journalctl@ceph.mds.a.smithi052.stdout:Jun 10 02:49:43 smithi052 podman[74592]: 2024-06-10 02:49:43.781614344 +0000 UTC m=+0.342467191 container init 20535a539d25316c74f9e9fc1c30c803e6a62f9a60c575fa97717355850608cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mds-a, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528) 2024-06-10T02:49:43.807 INFO:journalctl@ceph.mds.a.smithi052.stdout:Jun 10 02:49:43 smithi052 podman[74592]: 2024-06-10 02:49:43.78455633 +0000 UTC m=+0.345409175 container start 20535a539d25316c74f9e9fc1c30c803e6a62f9a60c575fa97717355850608cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mds-a, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-10T02:49:44.225 INFO:journalctl@ceph.mds.a.smithi052.stdout:Jun 10 02:49:43 smithi052 ceph-mds[74611]: -- 172.21.15.52:0/3937824756 <== mon.0 v2:172.21.15.52:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562912a0fa40 con 0x5629129e3000 2024-06-10T02:49:44.225 INFO:journalctl@ceph.mds.a.smithi052.stdout:Jun 10 02:49:43 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mds-a[74607]: starting mds.a at 2024-06-10T02:49:44.226 INFO:journalctl@ceph.mds.a.smithi052.stdout:Jun 10 02:49:43 smithi052 bash[74592]: 20535a539d25316c74f9e9fc1c30c803e6a62f9a60c575fa97717355850608cf 2024-06-10T02:49:44.226 INFO:journalctl@ceph.mds.a.smithi052.stdout:Jun 10 02:49:43 smithi052 systemd[1]: Started Ceph mds.a for d8676e9e-26d2-11ef-bc9b-c7b262605968. 2024-06-10T02:49:44.369 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T02:49:44.628 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:49:44.725 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:44 smithi052 ceph-mon[32362]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T02:49:44.725 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:44 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:49:44.725 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:44 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:49:44.725 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:44 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:49:44.725 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:44 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-10T02:49:44.726 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:44 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-10T02:49:44.726 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:44 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:49:44.741 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:44 smithi120 ceph-mon[36634]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T02:49:44.741 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:44 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:49:44.741 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:44 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:49:44.741 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:44 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:49:44.741 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:44 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-10T02:49:44.741 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:44 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-10T02:49:44.741 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:44 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:49:44.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:44 smithi188 ceph-mon[36599]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T02:49:44.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:44 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:49:44.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:44 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:49:44.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:44 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:49:44.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:44 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-10T02:49:44.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:44 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-10T02:49:44.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:44 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:49:45.740 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:45 smithi052 ceph-mon[32362]: Deploying daemon mds.b on smithi120 2024-06-10T02:49:45.741 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:45 smithi052 ceph-mon[32362]: mds.? [v2:172.21.15.52:6840/3039275521,v1:172.21.15.52:6841/3039275521] up:boot 2024-06-10T02:49:45.741 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:45 smithi052 ceph-mon[32362]: fsmap 11 up:standby 2024-06-10T02:49:45.741 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:45 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-06-10T02:49:45.784 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:45 smithi120 ceph-mon[36634]: Deploying daemon mds.b on smithi120 2024-06-10T02:49:45.784 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:45 smithi120 ceph-mon[36634]: mds.? [v2:172.21.15.52:6840/3039275521,v1:172.21.15.52:6841/3039275521] up:boot 2024-06-10T02:49:45.784 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:45 smithi120 ceph-mon[36634]: fsmap 11 up:standby 2024-06-10T02:49:45.784 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:45 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-06-10T02:49:45.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:45 smithi188 ceph-mon[36599]: Deploying daemon mds.b on smithi120 2024-06-10T02:49:45.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:45 smithi188 ceph-mon[36599]: mds.? [v2:172.21.15.52:6840/3039275521,v1:172.21.15.52:6841/3039275521] up:boot 2024-06-10T02:49:45.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:45 smithi188 ceph-mon[36599]: fsmap 11 up:standby 2024-06-10T02:49:45.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:45 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-06-10T02:49:46.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:46.282+0000 7f7382f57700 1 -- 172.21.15.52:0/2530569577 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f737c108500 msgr2=0x7f737c1088e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:49:46.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:46.282+0000 7f7382f57700 1 --2- 172.21.15.52:0/2530569577 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f737c108500 0x7f737c1088e0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f7378009990 tx=0x7f7378009ca0 comp rx=0 tx=0).stop 2024-06-10T02:49:46.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:46.282+0000 7f7382f57700 1 -- 172.21.15.52:0/2530569577 shutdown_connections 2024-06-10T02:49:46.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:46.282+0000 7f7382f57700 1 --2- 172.21.15.52:0/2530569577 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f737c06ab30 0x7f737c10f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:49:46.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:46.282+0000 7f7382f57700 1 --2- 172.21.15.52:0/2530569577 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f737c108500 0x7f737c1088e0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:49:46.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:46.282+0000 7f7382f57700 1 --2- 172.21.15.52:0/2530569577 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f737c06a170 0x7f737c06a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:49:46.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:46.282+0000 7f7382f57700 1 -- 172.21.15.52:0/2530569577 >> 172.21.15.52:0/2530569577 conn(0x7f737c078340 msgr2=0x7f737c078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:49:46.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:46.282+0000 7f7382f57700 1 -- 172.21.15.52:0/2530569577 shutdown_connections 2024-06-10T02:49:46.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:46.282+0000 7f7382f57700 1 -- 172.21.15.52:0/2530569577 wait complete. 2024-06-10T02:49:46.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:46.283+0000 7f7382f57700 1 Processor -- start 2024-06-10T02:49:46.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:46.283+0000 7f7382f57700 1 -- start start 2024-06-10T02:49:46.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:46.283+0000 7f7382f57700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f737c06a170 0x7f737c101460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:49:46.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:46.283+0000 7f7382f57700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f737c06ab30 0x7f737c1019a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:49:46.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:46.283+0000 7f7382f57700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f737c108500 0x7f737c101ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:49:46.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:46.283+0000 7f7382f57700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f737c112760 con 0x7f737c06ab30 2024-06-10T02:49:46.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:46.283+0000 7f7382f57700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f737c1125e0 con 0x7f737c06a170 2024-06-10T02:49:46.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:46.283+0000 7f7382f57700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f737c1128e0 con 0x7f737c108500 2024-06-10T02:49:46.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:46.284+0000 7f7373fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f737c06ab30 0x7f737c1019a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:49:46.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:46.284+0000 7f7373fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f737c06ab30 0x7f737c1019a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60804/0 (socket says 172.21.15.52:60804) 2024-06-10T02:49:46.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:46.284+0000 7f7373fff700 1 -- 172.21.15.52:0/2790049374 learned_addr learned my addr 172.21.15.52:0/2790049374 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:49:46.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:46.284+0000 7f7380cf3700 1 --2- 172.21.15.52:0/2790049374 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f737c06a170 0x7f737c101460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:49:46.286 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:46.284+0000 7f73814f4700 1 --2- 172.21.15.52:0/2790049374 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f737c108500 0x7f737c101ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:49:46.286 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:46.284+0000 7f7373fff700 1 -- 172.21.15.52:0/2790049374 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f737c108500 msgr2=0x7f737c101ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:49:46.286 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:46.284+0000 7f7373fff700 1 --2- 172.21.15.52:0/2790049374 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f737c108500 0x7f737c101ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:49:46.286 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:46.284+0000 7f7373fff700 1 -- 172.21.15.52:0/2790049374 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f737c06a170 msgr2=0x7f737c101460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:49:46.286 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:46.284+0000 7f7373fff700 1 --2- 172.21.15.52:0/2790049374 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f737c06a170 0x7f737c101460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:49:46.286 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:46.284+0000 7f7373fff700 1 -- 172.21.15.52:0/2790049374 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f737c1a7140 con 0x7f737c06ab30 2024-06-10T02:49:46.286 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:46.284+0000 7f7373fff700 1 --2- 172.21.15.52:0/2790049374 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f737c06ab30 0x7f737c1019a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f736c00c920 tx=0x7f736c00cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:49:46.286 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:46.284+0000 7f7371ffb700 1 -- 172.21.15.52:0/2790049374 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f736c007b00 con 0x7f737c06ab30 2024-06-10T02:49:46.286 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:46.284+0000 7f7371ffb700 1 -- 172.21.15.52:0/2790049374 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f736c004d10 con 0x7f737c06ab30 2024-06-10T02:49:46.286 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:46.284+0000 7f7382f57700 1 -- 172.21.15.52:0/2790049374 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f737c1a7430 con 0x7f737c06ab30 2024-06-10T02:49:46.287 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:46.284+0000 7f7371ffb700 1 -- 172.21.15.52:0/2790049374 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f736c005680 con 0x7f737c06ab30 2024-06-10T02:49:46.287 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:46.285+0000 7f7382f57700 1 -- 172.21.15.52:0/2790049374 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f737c1a79c0 con 0x7f737c06ab30 2024-06-10T02:49:46.287 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:46.285+0000 7f7382f57700 1 -- 172.21.15.52:0/2790049374 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f737c10a9e0 con 0x7f737c06ab30 2024-06-10T02:49:46.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:46.287+0000 7f7371ffb700 1 -- 172.21.15.52:0/2790049374 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f736c004750 con 0x7f737c06ab30 2024-06-10T02:49:46.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:46.287+0000 7f7371ffb700 1 --2- 172.21.15.52:0/2790049374 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f73640a2ca0 0x7f73640a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:49:46.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:46.288+0000 7f7380cf3700 1 --2- 172.21.15.52:0/2790049374 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f73640a2ca0 0x7f73640a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:49:46.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:46.288+0000 7f7371ffb700 1 -- 172.21.15.52:0/2790049374 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f736c0ce730 con 0x7f737c06ab30 2024-06-10T02:49:46.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:46.288+0000 7f7380cf3700 1 --2- 172.21.15.52:0/2790049374 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f73640a2ca0 0x7f73640a5160 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f737800b5c0 tx=0x7f7378005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:49:46.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:46.292+0000 7f7371ffb700 1 -- 172.21.15.52:0/2790049374 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f736c014030 con 0x7f737c06ab30 2024-06-10T02:49:46.492 INFO:journalctl@ceph.mds.b.smithi120.stdout:Jun 10 02:49:46 smithi120 podman[65993]: 2024-06-10 02:49:46.397017493 +0000 UTC m=+0.140790948 container create 920e913c3ac785cbb9088617e1ef9b3bdd425117a4df8ac647c4f323ba53888c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mds-b, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0) 2024-06-10T02:49:46.546 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:46.548+0000 7f7382f57700 1 -- 172.21.15.52:0/2790049374 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f737c069180 con 0x7f737c06ab30 2024-06-10T02:49:46.547 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:46.548+0000 7f7371ffb700 1 -- 172.21.15.52:0/2790049374 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9656 (secure 0 0 0) 0x7f736c09ba90 con 0x7f737c06ab30 2024-06-10T02:49:46.547 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-10T02:49:46.547 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/889673247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":889673247},{"type":"v1","addr":"172.21.15.52:6835","nonce":889673247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/3615663511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":3615663511},{"type":"v1","addr":"172.21.15.52:6837","nonce":3615663511}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14571,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6839/3059599042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":3059599042},{"type":"v1","addr":"172.21.15.52:6839","nonce":3059599042}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14583,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6841/3039275521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":3039275521},{"type":"v1","addr":"172.21.15.52:6841","nonce":3039275521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24383,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6833/2609002896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":2609002896},{"type":"v1","addr":"172.21.15.120:6833","nonce":2609002896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24401,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/1996370288","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":1996370288},{"type":"v1","addr":"172.21.15.120:6835","nonce":1996370288}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/529564979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":529564979},{"type":"v1","addr":"172.21.15.188:6833","nonce":529564979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24428,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/4132440952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":4132440952},{"type":"v1","addr":"172.21.15.120:6837","nonce":4132440952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24463,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/3904544259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":3904544259},{"type":"v1","addr":"172.21.15.188:6835","nonce":3904544259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24469,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/819317726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":819317726},{"type":"v1","addr":"172.21.15.188:6837","nonce":819317726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24484,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6839/310444836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":310444836},{"type":"v1","addr":"172.21.15.188:6839","nonce":310444836}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-06-10T02:49:46.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:46.550+0000 7f7382f57700 1 -- 172.21.15.52:0/2790049374 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f73640a2ca0 msgr2=0x7f73640a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:49:46.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:46.550+0000 7f7382f57700 1 --2- 172.21.15.52:0/2790049374 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f73640a2ca0 0x7f73640a5160 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f737800b5c0 tx=0x7f7378005960 comp rx=0 tx=0).stop 2024-06-10T02:49:46.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:46.550+0000 7f7382f57700 1 -- 172.21.15.52:0/2790049374 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f737c06ab30 msgr2=0x7f737c1019a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:49:46.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:46.550+0000 7f7382f57700 1 --2- 172.21.15.52:0/2790049374 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f737c06ab30 0x7f737c1019a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f736c00c920 tx=0x7f736c00cc30 comp rx=0 tx=0).stop 2024-06-10T02:49:46.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:46.550+0000 7f7382f57700 1 -- 172.21.15.52:0/2790049374 shutdown_connections 2024-06-10T02:49:46.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:46.550+0000 7f7382f57700 1 --2- 172.21.15.52:0/2790049374 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f737c108500 0x7f737c101ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:49:46.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:46.550+0000 7f7382f57700 1 --2- 172.21.15.52:0/2790049374 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f737c06a170 0x7f737c101460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:49:46.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:46.550+0000 7f7382f57700 1 --2- 172.21.15.52:0/2790049374 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f73640a2ca0 0x7f73640a5160 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:49:46.550 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:46.550+0000 7f7382f57700 1 --2- 172.21.15.52:0/2790049374 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f737c06ab30 0x7f737c1019a0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:49:46.550 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:46.550+0000 7f7382f57700 1 -- 172.21.15.52:0/2790049374 >> 172.21.15.52:0/2790049374 conn(0x7f737c078340 msgr2=0x7f737c0fffa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:49:46.550 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:46.551+0000 7f7382f57700 1 -- 172.21.15.52:0/2790049374 shutdown_connections 2024-06-10T02:49:46.550 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:46.551+0000 7f7382f57700 1 -- 172.21.15.52:0/2790049374 wait complete. 2024-06-10T02:49:46.550 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 12 2024-06-10T02:49:46.755 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:46 smithi120 ceph-mon[36634]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T02:49:46.755 INFO:journalctl@ceph.mds.b.smithi120.stdout:Jun 10 02:49:46 smithi120 podman[65993]: 2024-06-10 02:49:46.657783613 +0000 UTC m=+0.401557068 container init 920e913c3ac785cbb9088617e1ef9b3bdd425117a4df8ac647c4f323ba53888c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mds-b, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-06-10T02:49:46.755 INFO:journalctl@ceph.mds.b.smithi120.stdout:Jun 10 02:49:46 smithi120 podman[65993]: 2024-06-10 02:49:46.660335328 +0000 UTC m=+0.404108785 container start 920e913c3ac785cbb9088617e1ef9b3bdd425117a4df8ac647c4f323ba53888c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mds-b, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-10T02:49:46.755 INFO:journalctl@ceph.mds.b.smithi120.stdout:Jun 10 02:49:46 smithi120 ceph-mds[66012]: -- 172.21.15.120:0/2940824136 <== mon.2 v2:172.21.15.120:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5610a3c79a40 con 0x5610a3c4c800 2024-06-10T02:49:46.756 INFO:journalctl@ceph.mds.b.smithi120.stdout:Jun 10 02:49:46 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mds-b[66008]: starting mds.b at 2024-06-10T02:49:46.756 INFO:journalctl@ceph.mds.b.smithi120.stdout:Jun 10 02:49:46 smithi120 bash[65993]: 920e913c3ac785cbb9088617e1ef9b3bdd425117a4df8ac647c4f323ba53888c 2024-06-10T02:49:46.913 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:46 smithi052 ceph-mon[32362]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T02:49:46.951 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:46 smithi188 ceph-mon[36599]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T02:49:47.137 INFO:journalctl@ceph.mds.b.smithi120.stdout:Jun 10 02:49:46 smithi120 systemd[1]: Started Ceph mds.b for d8676e9e-26d2-11ef-bc9b-c7b262605968. 2024-06-10T02:49:47.830 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:47 smithi188 ceph-mon[36599]: from='client.? 172.21.15.52:0/2790049374' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T02:49:47.830 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:47 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:49:47.830 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:47 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:49:47.831 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:47 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:49:47.831 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:47 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:49:47.831 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:47 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:49:47.831 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:47 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T02:49:47.835 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:47 smithi120 ceph-mon[36634]: from='client.? 172.21.15.52:0/2790049374' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T02:49:47.835 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:47 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:49:47.835 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:47 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:49:47.835 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:47 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:49:47.836 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:47 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:49:47.836 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:47 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:49:47.836 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:47 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T02:49:47.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:47 smithi052 ceph-mon[32362]: from='client.? 172.21.15.52:0/2790049374' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T02:49:47.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:47 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:49:47.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:47 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:49:47.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:47 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:49:47.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:47 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:49:47.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:47 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:49:47.868 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:47 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T02:49:48.887 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:48 smithi120 ceph-mon[36634]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T02:49:48.887 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:48 smithi120 ceph-mon[36634]: mds.? [v2:172.21.15.120:6838/1672436487,v1:172.21.15.120:6839/1672436487] up:boot 2024-06-10T02:49:48.887 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:48 smithi120 ceph-mon[36634]: fsmap 12 up:standby 2024-06-10T02:49:48.887 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:48 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-06-10T02:49:48.887 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:48 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:49:48.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:48 smithi188 ceph-mon[36599]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T02:49:48.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:48 smithi188 ceph-mon[36599]: mds.? [v2:172.21.15.120:6838/1672436487,v1:172.21.15.120:6839/1672436487] up:boot 2024-06-10T02:49:48.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:48 smithi188 ceph-mon[36599]: fsmap 12 up:standby 2024-06-10T02:49:48.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:48 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-06-10T02:49:48.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:48 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:49:48.975 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:48 smithi052 ceph-mon[32362]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T02:49:48.975 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:48 smithi052 ceph-mon[32362]: mds.? [v2:172.21.15.120:6838/1672436487,v1:172.21.15.120:6839/1672436487] up:boot 2024-06-10T02:49:48.975 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:48 smithi052 ceph-mon[32362]: fsmap 12 up:standby 2024-06-10T02:49:48.975 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:48 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-06-10T02:49:48.975 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:48 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:49:49.280 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T02:49:49.535 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:49:49.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:49 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:49:49.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:49 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:49:49.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:49 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:49:49.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:49 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:49:49.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:49 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:49:49.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:49 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:49:49.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:49 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:49:49.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:49 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T02:49:49.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:49 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:49:49.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:49 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T02:49:49.971 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:49 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:49:49.972 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:49 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:49:49.972 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:49 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:49:49.972 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:49 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:49:49.972 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:49 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:49:49.972 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:49 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:49:49.972 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:49 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:49:49.972 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:49 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T02:49:49.972 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:49 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:49:49.972 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:49 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T02:49:50.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:49 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:49:50.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:49 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:49:50.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:49 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:49:50.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:49 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:49:50.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:49 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:49:50.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:49 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:49:50.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:49 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:49:50.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:49 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T02:49:50.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:49 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:49:50.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:49 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T02:49:50.951 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:50 smithi052 ceph-mon[32362]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T02:49:51.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:50 smithi120 ceph-mon[36634]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T02:49:51.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:51.187+0000 7f8f0441b700 1 -- 172.21.15.52:0/3217725657 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8efc1084f0 msgr2=0x7f8efc1088d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:49:51.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:51.187+0000 7f8f0441b700 1 --2- 172.21.15.52:0/3217725657 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8efc1084f0 0x7f8efc1088d0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f8ef8009990 tx=0x7f8ef8009ca0 comp rx=0 tx=0).stop 2024-06-10T02:49:51.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:51.187+0000 7f8f0441b700 1 -- 172.21.15.52:0/3217725657 shutdown_connections 2024-06-10T02:49:51.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:51.187+0000 7f8f0441b700 1 --2- 172.21.15.52:0/3217725657 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8efc102f10 0x7f8efc10f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:49:51.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:51.187+0000 7f8f0441b700 1 --2- 172.21.15.52:0/3217725657 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8efc1084f0 0x7f8efc1088d0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:49:51.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:51.187+0000 7f8f0441b700 1 --2- 172.21.15.52:0/3217725657 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8efc102550 0x7f8efc1029d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:49:51.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:51.187+0000 7f8f0441b700 1 -- 172.21.15.52:0/3217725657 >> 172.21.15.52:0/3217725657 conn(0x7f8efc079650 msgr2=0x7f8efc079a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:49:51.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:51.187+0000 7f8f0441b700 1 -- 172.21.15.52:0/3217725657 shutdown_connections 2024-06-10T02:49:51.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:51.187+0000 7f8f0441b700 1 -- 172.21.15.52:0/3217725657 wait complete. 2024-06-10T02:49:51.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:51.188+0000 7f8f0441b700 1 Processor -- start 2024-06-10T02:49:51.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:51.188+0000 7f8f0441b700 1 -- start start 2024-06-10T02:49:51.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:51.188+0000 7f8f0441b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8efc102550 0x7f8efc19ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:49:51.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:51.188+0000 7f8f0441b700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8efc102f10 0x7f8efc19b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:49:51.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:51.188+0000 7f8f0441b700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8efc1084f0 0x7f8efc19f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:49:51.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:51.188+0000 7f8f0441b700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f8efc112770 con 0x7f8efc102550 2024-06-10T02:49:51.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:51.188+0000 7f8f0441b700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f8efc1125f0 con 0x7f8efc102f10 2024-06-10T02:49:51.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:51.188+0000 7f8f0441b700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f8efc1128f0 con 0x7f8efc1084f0 2024-06-10T02:49:51.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:51.188+0000 7f8f021b7700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8efc102550 0x7f8efc19ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:49:51.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:51.188+0000 7f8f021b7700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8efc102550 0x7f8efc19ae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:38416/0 (socket says 172.21.15.52:38416) 2024-06-10T02:49:51.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:51.188+0000 7f8f021b7700 1 -- 172.21.15.52:0/2793396327 learned_addr learned my addr 172.21.15.52:0/2793396327 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:49:51.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:51.188+0000 7f8f019b6700 1 --2- 172.21.15.52:0/2793396327 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8efc102f10 0x7f8efc19b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:49:51.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:51.188+0000 7f8f029b8700 1 --2- 172.21.15.52:0/2793396327 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8efc1084f0 0x7f8efc19f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:49:51.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:51.188+0000 7f8f021b7700 1 -- 172.21.15.52:0/2793396327 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8efc1084f0 msgr2=0x7f8efc19f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:49:51.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:51.188+0000 7f8f021b7700 1 --2- 172.21.15.52:0/2793396327 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8efc1084f0 0x7f8efc19f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:49:51.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:51.188+0000 7f8f021b7700 1 -- 172.21.15.52:0/2793396327 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8efc102f10 msgr2=0x7f8efc19b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:49:51.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:51.188+0000 7f8f021b7700 1 --2- 172.21.15.52:0/2793396327 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8efc102f10 0x7f8efc19b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:49:51.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:51.188+0000 7f8f021b7700 1 -- 172.21.15.52:0/2793396327 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8efc19fe90 con 0x7f8efc102550 2024-06-10T02:49:51.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:51.188+0000 7f8f019b6700 1 --2- 172.21.15.52:0/2793396327 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8efc102f10 0x7f8efc19b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T02:49:51.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:51.189+0000 7f8f021b7700 1 --2- 172.21.15.52:0/2793396327 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8efc102550 0x7f8efc19ae80 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f8ef8000c00 tx=0x7f8ef800bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:49:51.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:51.189+0000 7f8ef37fe700 1 -- 172.21.15.52:0/2793396327 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ef8027070 con 0x7f8efc102550 2024-06-10T02:49:51.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:51.189+0000 7f8f0441b700 1 -- 172.21.15.52:0/2793396327 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8efc1a0120 con 0x7f8efc102550 2024-06-10T02:49:51.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:51.189+0000 7f8ef37fe700 1 -- 172.21.15.52:0/2793396327 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ef8003d70 con 0x7f8efc102550 2024-06-10T02:49:51.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:51.189+0000 7f8ef37fe700 1 -- 172.21.15.52:0/2793396327 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ef800f460 con 0x7f8efc102550 2024-06-10T02:49:51.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:51.189+0000 7f8f0441b700 1 -- 172.21.15.52:0/2793396327 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8efc1a7a00 con 0x7f8efc102550 2024-06-10T02:49:51.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:51.190+0000 7f8f0441b700 1 -- 172.21.15.52:0/2793396327 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8efc103ba0 con 0x7f8efc102550 2024-06-10T02:49:51.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:51.191+0000 7f8ef37fe700 1 -- 172.21.15.52:0/2793396327 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f8ef802a070 con 0x7f8efc102550 2024-06-10T02:49:51.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:51.192+0000 7f8ef37fe700 1 --2- 172.21.15.52:0/2793396327 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f8ee80a2b00 0x7f8ee80a4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:49:51.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:51.192+0000 7f8ef37fe700 1 -- 172.21.15.52:0/2793396327 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f8ef80cf770 con 0x7f8efc102550 2024-06-10T02:49:51.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:51.192+0000 7f8f019b6700 1 --2- 172.21.15.52:0/2793396327 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f8ee80a2b00 0x7f8ee80a4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:49:51.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:51.192+0000 7f8f019b6700 1 --2- 172.21.15.52:0/2793396327 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f8ee80a2b00 0x7f8ee80a4fc0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f8efc076720 tx=0x7f8eec009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:49:51.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:51.197+0000 7f8ef37fe700 1 -- 172.21.15.52:0/2793396327 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f8ef8013070 con 0x7f8efc102550 2024-06-10T02:49:51.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:50 smithi188 ceph-mon[36599]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T02:49:51.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:51.466+0000 7f8f0441b700 1 -- 172.21.15.52:0/2793396327 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8efc069180 con 0x7f8efc102550 2024-06-10T02:49:51.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:51.467+0000 7f8ef37fe700 1 -- 172.21.15.52:0/2793396327 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10485 (secure 0 0 0) 0x7f8ef809cc00 con 0x7f8efc102550 2024-06-10T02:49:51.466 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-10T02:49:51.466 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/889673247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":889673247},{"type":"v1","addr":"172.21.15.52:6835","nonce":889673247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/3615663511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":3615663511},{"type":"v1","addr":"172.21.15.52:6837","nonce":3615663511}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14571,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6839/3059599042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":3059599042},{"type":"v1","addr":"172.21.15.52:6839","nonce":3059599042}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14583,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6841/3039275521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":3039275521},{"type":"v1","addr":"172.21.15.52:6841","nonce":3039275521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24383,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6833/2609002896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":2609002896},{"type":"v1","addr":"172.21.15.120:6833","nonce":2609002896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24401,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/1996370288","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":1996370288},{"type":"v1","addr":"172.21.15.120:6835","nonce":1996370288}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/529564979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":529564979},{"type":"v1","addr":"172.21.15.188:6833","nonce":529564979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24428,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/4132440952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":4132440952},{"type":"v1","addr":"172.21.15.120:6837","nonce":4132440952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24449,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6839/1672436487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1672436487},{"type":"v1","addr":"172.21.15.120:6839","nonce":1672436487}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":24463,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/3904544259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":3904544259},{"type":"v1","addr":"172.21.15.188:6835","nonce":3904544259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24469,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/819317726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":819317726},{"type":"v1","addr":"172.21.15.188:6837","nonce":819317726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24484,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6839/310444836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":310444836},{"type":"v1","addr":"172.21.15.188:6839","nonce":310444836}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-06-10T02:49:51.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:51.470+0000 7f8f0441b700 1 -- 172.21.15.52:0/2793396327 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f8ee80a2b00 msgr2=0x7f8ee80a4fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:49:51.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:51.470+0000 7f8f0441b700 1 --2- 172.21.15.52:0/2793396327 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f8ee80a2b00 0x7f8ee80a4fc0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f8efc076720 tx=0x7f8eec009260 comp rx=0 tx=0).stop 2024-06-10T02:49:51.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:51.470+0000 7f8f0441b700 1 -- 172.21.15.52:0/2793396327 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8efc102550 msgr2=0x7f8efc19ae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:49:51.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:51.470+0000 7f8f0441b700 1 --2- 172.21.15.52:0/2793396327 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8efc102550 0x7f8efc19ae80 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f8ef8000c00 tx=0x7f8ef800bab0 comp rx=0 tx=0).stop 2024-06-10T02:49:51.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:51.470+0000 7f8f0441b700 1 -- 172.21.15.52:0/2793396327 shutdown_connections 2024-06-10T02:49:51.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:51.470+0000 7f8f0441b700 1 --2- 172.21.15.52:0/2793396327 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8efc1084f0 0x7f8efc19f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:49:51.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:51.470+0000 7f8f0441b700 1 --2- 172.21.15.52:0/2793396327 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f8ee80a2b00 0x7f8ee80a4fc0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:49:51.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:51.470+0000 7f8f0441b700 1 --2- 172.21.15.52:0/2793396327 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8efc102550 0x7f8efc19ae80 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:49:51.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:51.470+0000 7f8f0441b700 1 --2- 172.21.15.52:0/2793396327 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8efc102f10 0x7f8efc19b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:49:51.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:51.470+0000 7f8f0441b700 1 -- 172.21.15.52:0/2793396327 >> 172.21.15.52:0/2793396327 conn(0x7f8efc079650 msgr2=0x7f8efc1003c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:49:51.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:51.470+0000 7f8f0441b700 1 -- 172.21.15.52:0/2793396327 shutdown_connections 2024-06-10T02:49:51.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:51.470+0000 7f8f0441b700 1 -- 172.21.15.52:0/2793396327 wait complete. 2024-06-10T02:49:51.470 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 13 2024-06-10T02:49:52.549 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:52 smithi052 ceph-mon[32362]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T02:49:52.549 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:52 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:49:52.549 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:52 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:49:52.549 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:52 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:49:52.549 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:52 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:49:52.550 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:52 smithi052 ceph-mon[32362]: from='client.? 172.21.15.52:0/2793396327' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T02:49:52.579 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph --format=json-pretty osd lspools 2024-06-10T02:49:52.637 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:52 smithi120 ceph-mon[36634]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T02:49:52.637 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:52 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:49:52.637 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:52 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:49:52.637 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:52 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:49:52.637 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:52 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:49:52.637 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:52 smithi120 ceph-mon[36634]: from='client.? 172.21.15.52:0/2793396327' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T02:49:52.702 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:52 smithi188 ceph-mon[36599]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T02:49:52.702 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:52 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:49:52.702 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:52 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:49:52.703 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:52 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:49:52.703 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:52 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:49:52.703 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:52 smithi188 ceph-mon[36599]: from='client.? 172.21.15.52:0/2793396327' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T02:49:52.872 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:49:54.533 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:54.534+0000 7fea5c26b700 1 -- 172.21.15.52:0/1786997473 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fea541084f0 msgr2=0x7fea541088d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:49:54.533 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:54.534+0000 7fea5c26b700 1 --2- 172.21.15.52:0/1786997473 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fea541084f0 0x7fea541088d0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fea50009990 tx=0x7fea50009ca0 comp rx=0 tx=0).stop 2024-06-10T02:49:54.533 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:54.534+0000 7fea5c26b700 1 -- 172.21.15.52:0/1786997473 shutdown_connections 2024-06-10T02:49:54.533 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:54.534+0000 7fea5c26b700 1 --2- 172.21.15.52:0/1786997473 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fea54102e80 0x7fea5410f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:49:54.533 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:54.534+0000 7fea5c26b700 1 --2- 172.21.15.52:0/1786997473 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fea541084f0 0x7fea541088d0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:49:54.533 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:54.534+0000 7fea5c26b700 1 --2- 172.21.15.52:0/1786997473 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fea541024e0 0x7fea54102940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:49:54.534 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:54.534+0000 7fea5c26b700 1 -- 172.21.15.52:0/1786997473 >> 172.21.15.52:0/1786997473 conn(0x7fea54079530 msgr2=0x7fea54079940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:49:54.534 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:54.535+0000 7fea5c26b700 1 -- 172.21.15.52:0/1786997473 shutdown_connections 2024-06-10T02:49:54.534 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:54.535+0000 7fea5c26b700 1 -- 172.21.15.52:0/1786997473 wait complete. 2024-06-10T02:49:54.535 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:54.535+0000 7fea5c26b700 1 Processor -- start 2024-06-10T02:49:54.536 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:54.535+0000 7fea5c26b700 1 -- start start 2024-06-10T02:49:54.536 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:54.535+0000 7fea5c26b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fea541024e0 0x7fea5419e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:49:54.536 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:54.535+0000 7fea5c26b700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fea54102e80 0x7fea5419ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:49:54.536 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:54.535+0000 7fea5c26b700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fea541084f0 0x7fea54198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:49:54.536 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:54.535+0000 7fea5c26b700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fea541127b0 con 0x7fea541024e0 2024-06-10T02:49:54.536 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:54.535+0000 7fea5c26b700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fea54112630 con 0x7fea54102e80 2024-06-10T02:49:54.536 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:54.535+0000 7fea5c26b700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fea54112930 con 0x7fea541084f0 2024-06-10T02:49:54.536 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:54.535+0000 7fea5a007700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fea541024e0 0x7fea5419e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:49:54.536 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:54.536+0000 7fea5a007700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fea541024e0 0x7fea5419e8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:38438/0 (socket says 172.21.15.52:38438) 2024-06-10T02:49:54.536 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:54.536+0000 7fea5a007700 1 -- 172.21.15.52:0/2064223337 learned_addr learned my addr 172.21.15.52:0/2064223337 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:49:54.536 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:54.536+0000 7fea5a808700 1 --2- 172.21.15.52:0/2064223337 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fea541084f0 0x7fea54198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:49:54.536 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:54.536+0000 7fea59806700 1 --2- 172.21.15.52:0/2064223337 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fea54102e80 0x7fea5419ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:49:54.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:54.536+0000 7fea5a007700 1 -- 172.21.15.52:0/2064223337 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fea541084f0 msgr2=0x7fea54198a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:49:54.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:54.536+0000 7fea5a007700 1 --2- 172.21.15.52:0/2064223337 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fea541084f0 0x7fea54198a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:49:54.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:54.536+0000 7fea5a007700 1 -- 172.21.15.52:0/2064223337 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fea54102e80 msgr2=0x7fea5419ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:49:54.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:54.536+0000 7fea5a007700 1 --2- 172.21.15.52:0/2064223337 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fea54102e80 0x7fea5419ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:49:54.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:54.536+0000 7fea5a007700 1 -- 172.21.15.52:0/2064223337 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea541991f0 con 0x7fea541024e0 2024-06-10T02:49:54.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:54.536+0000 7fea5a808700 1 --2- 172.21.15.52:0/2064223337 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fea541084f0 0x7fea54198a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T02:49:54.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:54.536+0000 7fea59806700 1 --2- 172.21.15.52:0/2064223337 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fea54102e80 0x7fea5419ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T02:49:54.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:54.536+0000 7fea5a007700 1 --2- 172.21.15.52:0/2064223337 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fea541024e0 0x7fea5419e8d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fea50000c00 tx=0x7fea500047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:49:54.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:54.536+0000 7fea4b7fe700 1 -- 172.21.15.52:0/2064223337 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea50027070 con 0x7fea541024e0 2024-06-10T02:49:54.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:54.536+0000 7fea5c26b700 1 -- 172.21.15.52:0/2064223337 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea54199480 con 0x7fea541024e0 2024-06-10T02:49:54.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:54.536+0000 7fea5c26b700 1 -- 172.21.15.52:0/2064223337 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea541a57a0 con 0x7fea541024e0 2024-06-10T02:49:54.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:54.536+0000 7fea4b7fe700 1 -- 172.21.15.52:0/2064223337 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea50004050 con 0x7fea541024e0 2024-06-10T02:49:54.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:54.536+0000 7fea4b7fe700 1 -- 172.21.15.52:0/2064223337 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea5000f460 con 0x7fea541024e0 2024-06-10T02:49:54.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:54.537+0000 7fea5c26b700 1 -- 172.21.15.52:0/2064223337 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea54069180 con 0x7fea541024e0 2024-06-10T02:49:54.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:54.539+0000 7fea4b7fe700 1 -- 172.21.15.52:0/2064223337 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fea5002a070 con 0x7fea541024e0 2024-06-10T02:49:54.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:54.540+0000 7fea4b7fe700 1 --2- 172.21.15.52:0/2064223337 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fea400a2c20 0x7fea400a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:49:54.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:54.540+0000 7fea59806700 1 --2- 172.21.15.52:0/2064223337 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fea400a2c20 0x7fea400a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:49:54.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:54.540+0000 7fea4b7fe700 1 -- 172.21.15.52:0/2064223337 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fea500cffb0 con 0x7fea541024e0 2024-06-10T02:49:54.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:54.541+0000 7fea59806700 1 --2- 172.21.15.52:0/2064223337 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fea400a2c20 0x7fea400a50e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fea54079ba0 tx=0x7fea44008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:49:54.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:54.544+0000 7fea4b7fe700 1 -- 172.21.15.52:0/2064223337 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fea50031070 con 0x7fea541024e0 2024-06-10T02:49:54.750 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:54.751+0000 7fea5c26b700 1 -- 172.21.15.52:0/2064223337 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fea5402dca0 con 0x7fea541024e0 2024-06-10T02:49:54.750 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:54.752+0000 7fea4b7fe700 1 -- 172.21.15.52:0/2064223337 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v82) ==== 84+0+65 (secure 0 0 0) 0x7fea50013070 con 0x7fea541024e0 2024-06-10T02:49:54.750 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-10T02:49:54.751 INFO:teuthology.orchestra.run.smithi052.stdout:[ 2024-06-10T02:49:54.751 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-10T02:49:54.751 INFO:teuthology.orchestra.run.smithi052.stdout: "poolnum": 1, 2024-06-10T02:49:54.751 INFO:teuthology.orchestra.run.smithi052.stdout: "poolname": ".mgr" 2024-06-10T02:49:54.751 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T02:49:54.751 INFO:teuthology.orchestra.run.smithi052.stdout:] 2024-06-10T02:49:54.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:54.754+0000 7fea5c26b700 1 -- 172.21.15.52:0/2064223337 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fea400a2c20 msgr2=0x7fea400a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:49:54.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:54.754+0000 7fea5c26b700 1 --2- 172.21.15.52:0/2064223337 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fea400a2c20 0x7fea400a50e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fea54079ba0 tx=0x7fea44008040 comp rx=0 tx=0).stop 2024-06-10T02:49:54.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:54.754+0000 7fea5c26b700 1 -- 172.21.15.52:0/2064223337 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fea541024e0 msgr2=0x7fea5419e8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:49:54.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:54.754+0000 7fea5c26b700 1 --2- 172.21.15.52:0/2064223337 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fea541024e0 0x7fea5419e8d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fea50000c00 tx=0x7fea500047b0 comp rx=0 tx=0).stop 2024-06-10T02:49:54.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:54.754+0000 7fea5c26b700 1 -- 172.21.15.52:0/2064223337 shutdown_connections 2024-06-10T02:49:54.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:54.754+0000 7fea5c26b700 1 --2- 172.21.15.52:0/2064223337 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fea541084f0 0x7fea54198a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:49:54.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:54.754+0000 7fea5c26b700 1 --2- 172.21.15.52:0/2064223337 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fea400a2c20 0x7fea400a50e0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:49:54.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:54.754+0000 7fea5c26b700 1 --2- 172.21.15.52:0/2064223337 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fea541024e0 0x7fea5419e8d0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:49:54.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:54.754+0000 7fea5c26b700 1 --2- 172.21.15.52:0/2064223337 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fea54102e80 0x7fea5419ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:49:54.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:54.754+0000 7fea5c26b700 1 -- 172.21.15.52:0/2064223337 >> 172.21.15.52:0/2064223337 conn(0x7fea54079530 msgr2=0x7fea54100590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:49:54.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:54.754+0000 7fea5c26b700 1 -- 172.21.15.52:0/2064223337 shutdown_connections 2024-06-10T02:49:54.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:54.754+0000 7fea5c26b700 1 -- 172.21.15.52:0/2064223337 wait complete. 2024-06-10T02:49:54.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:54 smithi188 ceph-mon[36599]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T02:49:54.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:54 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:49:54.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:54 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:49:54.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:54 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:49:54.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:54 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T02:49:54.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:54 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:49:54.975 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:54 smithi052 ceph-mon[32362]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T02:49:54.975 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:54 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:49:54.975 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:54 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:49:54.975 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:54 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:49:54.975 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:54 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T02:49:54.975 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:54 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:49:55.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:54 smithi120 ceph-mon[36634]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T02:49:55.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:54 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:49:55.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:54 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:49:55.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:54 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:49:55.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:54 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T02:49:55.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:54 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:49:55.357 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-06-10T02:49:55.357 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph osd pool create cephfs_metadata --pg_num_min 64 2024-06-10T02:49:55.611 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:49:55.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:55 smithi188 ceph-mon[36599]: from='client.? 172.21.15.52:0/2064223337' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-06-10T02:49:55.970 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:55 smithi052 ceph-mon[32362]: from='client.? 172.21.15.52:0/2064223337' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-06-10T02:49:56.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:55 smithi120 ceph-mon[36634]: from='client.? 172.21.15.52:0/2064223337' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-06-10T02:49:56.717 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:56 smithi052 ceph-mon[32362]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T02:49:56.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:56 smithi188 ceph-mon[36599]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T02:49:57.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:56 smithi120 ceph-mon[36634]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T02:49:57.267 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:57.268+0000 7f0f16e9b700 1 -- 172.21.15.52:0/3414225861 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0f10101fd0 msgr2=0x7f0f10105fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:49:57.267 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:57.268+0000 7f0f16e9b700 1 --2- 172.21.15.52:0/3414225861 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0f10101fd0 0x7f0f10105fd0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f0efc0099c0 tx=0x7f0efc009cd0 comp rx=0 tx=0).stop 2024-06-10T02:49:57.267 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:57.268+0000 7f0f16e9b700 1 -- 172.21.15.52:0/3414225861 shutdown_connections 2024-06-10T02:49:57.267 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:57.268+0000 7f0f16e9b700 1 --2- 172.21.15.52:0/3414225861 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0f101066b0 0x7f0f1010e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:49:57.267 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:57.268+0000 7f0f16e9b700 1 --2- 172.21.15.52:0/3414225861 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0f10101620 0x7f0f10101a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:49:57.267 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:57.268+0000 7f0f16e9b700 1 --2- 172.21.15.52:0/3414225861 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0f10101fd0 0x7f0f10105fd0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:49:57.267 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:57.268+0000 7f0f16e9b700 1 -- 172.21.15.52:0/3414225861 >> 172.21.15.52:0/3414225861 conn(0x7f0f10078340 msgr2=0x7f0f10078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:49:57.267 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:57.268+0000 7f0f16e9b700 1 -- 172.21.15.52:0/3414225861 shutdown_connections 2024-06-10T02:49:57.267 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:57.268+0000 7f0f16e9b700 1 -- 172.21.15.52:0/3414225861 wait complete. 2024-06-10T02:49:57.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:57.268+0000 7f0f16e9b700 1 Processor -- start 2024-06-10T02:49:57.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:57.269+0000 7f0f16e9b700 1 -- start start 2024-06-10T02:49:57.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:57.269+0000 7f0f16e9b700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0f10101620 0x7f0f1019d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:49:57.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:57.269+0000 7f0f16e9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0f10101fd0 0x7f0f1019dd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:49:57.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:57.269+0000 7f0f16e9b700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0f101066b0 0x7f0f10197990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:49:57.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:57.269+0000 7f0f16e9b700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f0f10111690 con 0x7f0f10101fd0 2024-06-10T02:49:57.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:57.269+0000 7f0f16e9b700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f0f10111510 con 0x7f0f10101620 2024-06-10T02:49:57.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:57.269+0000 7f0f16e9b700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f0f10111810 con 0x7f0f101066b0 2024-06-10T02:49:57.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:57.269+0000 7f0f0ffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0f10101fd0 0x7f0f1019dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:49:57.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:57.269+0000 7f0f0ffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0f10101fd0 0x7f0f1019dd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:38454/0 (socket says 172.21.15.52:38454) 2024-06-10T02:49:57.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:57.269+0000 7f0f0ffff700 1 -- 172.21.15.52:0/1878288657 learned_addr learned my addr 172.21.15.52:0/1878288657 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:49:57.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:57.269+0000 7f0f14c37700 1 --2- 172.21.15.52:0/1878288657 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0f10101620 0x7f0f1019d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:49:57.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:57.269+0000 7f0f15438700 1 --2- 172.21.15.52:0/1878288657 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0f101066b0 0x7f0f10197990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:49:57.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:57.269+0000 7f0f0ffff700 1 -- 172.21.15.52:0/1878288657 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0f101066b0 msgr2=0x7f0f10197990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:49:57.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:57.269+0000 7f0f0ffff700 1 --2- 172.21.15.52:0/1878288657 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0f101066b0 0x7f0f10197990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:49:57.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:57.269+0000 7f0f0ffff700 1 -- 172.21.15.52:0/1878288657 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0f10101620 msgr2=0x7f0f1019d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:49:57.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:57.269+0000 7f0f0ffff700 1 --2- 172.21.15.52:0/1878288657 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0f10101620 0x7f0f1019d7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:49:57.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:57.269+0000 7f0f0ffff700 1 -- 172.21.15.52:0/1878288657 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f101980f0 con 0x7f0f10101fd0 2024-06-10T02:49:57.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:57.270+0000 7f0f0ffff700 1 --2- 172.21.15.52:0/1878288657 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0f10101fd0 0x7f0f1019dd10 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f0efc00c010 tx=0x7f0efc0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:49:57.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:57.270+0000 7f0f0dffb700 1 -- 172.21.15.52:0/1878288657 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0efc027070 con 0x7f0f10101fd0 2024-06-10T02:49:57.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:57.270+0000 7f0f0dffb700 1 -- 172.21.15.52:0/1878288657 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0efc003730 con 0x7f0f10101fd0 2024-06-10T02:49:57.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:57.270+0000 7f0f16e9b700 1 -- 172.21.15.52:0/1878288657 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f10198380 con 0x7f0f10101fd0 2024-06-10T02:49:57.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:57.270+0000 7f0f0dffb700 1 -- 172.21.15.52:0/1878288657 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0efc022d40 con 0x7f0f10101fd0 2024-06-10T02:49:57.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:57.270+0000 7f0f16e9b700 1 -- 172.21.15.52:0/1878288657 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f101a46a0 con 0x7f0f10101fd0 2024-06-10T02:49:57.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:57.271+0000 7f0f16e9b700 1 -- 172.21.15.52:0/1878288657 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f10069180 con 0x7f0f10101fd0 2024-06-10T02:49:57.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:57.272+0000 7f0f0dffb700 1 -- 172.21.15.52:0/1878288657 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f0efc02a070 con 0x7f0f10101fd0 2024-06-10T02:49:57.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:57.273+0000 7f0f0dffb700 1 --2- 172.21.15.52:0/1878288657 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f0f000a2c20 0x7f0f000a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:49:57.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:57.273+0000 7f0f0dffb700 1 -- 172.21.15.52:0/1878288657 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f0efc0d00c0 con 0x7f0f10101fd0 2024-06-10T02:49:57.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:57.273+0000 7f0f14c37700 1 --2- 172.21.15.52:0/1878288657 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f0f000a2c20 0x7f0f000a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:49:57.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:57.274+0000 7f0f14c37700 1 --2- 172.21.15.52:0/1878288657 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f0f000a2c20 0x7f0f000a50e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f0f10198ad0 tx=0x7f0f0400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:49:57.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:57.276+0000 7f0f0dffb700 1 -- 172.21.15.52:0/1878288657 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f0efc031070 con 0x7f0f10101fd0 2024-06-10T02:49:57.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:57.493+0000 7f0f16e9b700 1 -- 172.21.15.52:0/1878288657 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f0f1002dca0 con 0x7f0f10101fd0 2024-06-10T02:49:57.976 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:57.977+0000 7f0f0dffb700 1 -- 172.21.15.52:0/1878288657 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v83) ==== 138+0+0 (secure 0 0 0) 0x7f0efc013070 con 0x7f0f10101fd0 2024-06-10T02:49:57.979 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:57.980+0000 7f0f16e9b700 1 -- 172.21.15.52:0/1878288657 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f0f000a2c20 msgr2=0x7f0f000a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:49:57.979 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:57.980+0000 7f0f16e9b700 1 --2- 172.21.15.52:0/1878288657 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f0f000a2c20 0x7f0f000a50e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f0f10198ad0 tx=0x7f0f0400b410 comp rx=0 tx=0).stop 2024-06-10T02:49:57.979 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:57.980+0000 7f0f16e9b700 1 -- 172.21.15.52:0/1878288657 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0f10101fd0 msgr2=0x7f0f1019dd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:49:57.979 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:57.980+0000 7f0f16e9b700 1 --2- 172.21.15.52:0/1878288657 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0f10101fd0 0x7f0f1019dd10 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f0efc00c010 tx=0x7f0efc0047b0 comp rx=0 tx=0).stop 2024-06-10T02:49:57.979 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:57.980+0000 7f0f16e9b700 1 -- 172.21.15.52:0/1878288657 shutdown_connections 2024-06-10T02:49:57.979 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:57.980+0000 7f0f16e9b700 1 --2- 172.21.15.52:0/1878288657 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0f101066b0 0x7f0f10197990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:49:57.979 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:57.980+0000 7f0f16e9b700 1 --2- 172.21.15.52:0/1878288657 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0f10101620 0x7f0f1019d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:49:57.979 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:57.980+0000 7f0f16e9b700 1 --2- 172.21.15.52:0/1878288657 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f0f000a2c20 0x7f0f000a50e0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:49:57.979 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:57.980+0000 7f0f16e9b700 1 --2- 172.21.15.52:0/1878288657 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0f10101fd0 0x7f0f1019dd10 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:49:57.979 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:57.980+0000 7f0f16e9b700 1 -- 172.21.15.52:0/1878288657 >> 172.21.15.52:0/1878288657 conn(0x7f0f10078340 msgr2=0x7f0f100ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:49:57.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:57.981+0000 7f0f16e9b700 1 -- 172.21.15.52:0/1878288657 shutdown_connections 2024-06-10T02:49:57.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:49:57.981+0000 7f0f16e9b700 1 -- 172.21.15.52:0/1878288657 wait complete. 2024-06-10T02:49:57.980 INFO:teuthology.orchestra.run.smithi052.stderr:pool 'cephfs_metadata' created 2024-06-10T02:49:58.300 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:57 smithi052 ceph-mon[32362]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T02:49:58.300 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:57 smithi052 ceph-mon[32362]: from='client.? 172.21.15.52:0/1878288657' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-06-10T02:49:58.386 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:57 smithi120 ceph-mon[36634]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T02:49:58.387 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:57 smithi120 ceph-mon[36634]: from='client.? 172.21.15.52:0/1878288657' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-06-10T02:49:58.452 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:57 smithi188 ceph-mon[36599]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T02:49:58.452 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:57 smithi188 ceph-mon[36599]: from='client.? 172.21.15.52:0/1878288657' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-06-10T02:49:58.589 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-06-10T02:49:58.840 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:49:59.213 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:58 smithi052 ceph-mon[32362]: from='client.? 172.21.15.52:0/1878288657' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-06-10T02:49:59.213 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:58 smithi052 ceph-mon[32362]: osdmap e83: 12 total, 12 up, 12 in 2024-06-10T02:49:59.214 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:58 smithi052 ceph-mon[32362]: osdmap e84: 12 total, 12 up, 12 in 2024-06-10T02:49:59.387 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:58 smithi120 ceph-mon[36634]: from='client.? 172.21.15.52:0/1878288657' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-06-10T02:49:59.387 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:58 smithi120 ceph-mon[36634]: osdmap e83: 12 total, 12 up, 12 in 2024-06-10T02:49:59.387 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:58 smithi120 ceph-mon[36634]: osdmap e84: 12 total, 12 up, 12 in 2024-06-10T02:49:59.452 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:58 smithi188 ceph-mon[36599]: from='client.? 172.21.15.52:0/1878288657' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-06-10T02:49:59.452 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:58 smithi188 ceph-mon[36599]: osdmap e83: 12 total, 12 up, 12 in 2024-06-10T02:49:59.452 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:58 smithi188 ceph-mon[36599]: osdmap e84: 12 total, 12 up, 12 in 2024-06-10T02:50:00.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:59 smithi052 ceph-mon[32362]: pgmap v291: 33 pgs: 6 creating+peering, 1 active+clean, 26 unknown; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T02:50:00.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:59 smithi052 ceph-mon[32362]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-10T02:50:00.226 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:49:59 smithi052 ceph-mon[32362]: osdmap e85: 12 total, 12 up, 12 in 2024-06-10T02:50:00.387 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:59 smithi120 ceph-mon[36634]: pgmap v291: 33 pgs: 6 creating+peering, 1 active+clean, 26 unknown; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T02:50:00.387 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:59 smithi120 ceph-mon[36634]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-10T02:50:00.387 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:49:59 smithi120 ceph-mon[36634]: osdmap e85: 12 total, 12 up, 12 in 2024-06-10T02:50:00.452 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:59 smithi188 ceph-mon[36599]: pgmap v291: 33 pgs: 6 creating+peering, 1 active+clean, 26 unknown; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T02:50:00.452 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:59 smithi188 ceph-mon[36599]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-10T02:50:00.452 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:49:59 smithi188 ceph-mon[36599]: osdmap e85: 12 total, 12 up, 12 in 2024-06-10T02:50:00.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:00.479+0000 7f328b7bc700 1 -- 172.21.15.52:0/3303550156 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3284102770 msgr2=0x7f3284102b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:50:00.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:00.479+0000 7f328b7bc700 1 --2- 172.21.15.52:0/3303550156 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3284102770 0x7f3284102b50 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f32800099c0 tx=0x7f3280009cd0 comp rx=0 tx=0).stop 2024-06-10T02:50:00.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:00.479+0000 7f328b7bc700 1 -- 172.21.15.52:0/3303550156 shutdown_connections 2024-06-10T02:50:00.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:00.479+0000 7f328b7bc700 1 --2- 172.21.15.52:0/3303550156 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3284107860 0x7f328410f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:50:00.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:00.479+0000 7f328b7bc700 1 --2- 172.21.15.52:0/3303550156 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3284102770 0x7f3284102b50 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:50:00.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:00.479+0000 7f328b7bc700 1 --2- 172.21.15.52:0/3303550156 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3284103120 0x7f3284107130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:50:00.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:00.479+0000 7f328b7bc700 1 -- 172.21.15.52:0/3303550156 >> 172.21.15.52:0/3303550156 conn(0x7f3284079600 msgr2=0x7f3284079a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:50:00.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:00.479+0000 7f328b7bc700 1 -- 172.21.15.52:0/3303550156 shutdown_connections 2024-06-10T02:50:00.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:00.479+0000 7f328b7bc700 1 -- 172.21.15.52:0/3303550156 wait complete. 2024-06-10T02:50:00.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:00.480+0000 7f328b7bc700 1 Processor -- start 2024-06-10T02:50:00.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:00.480+0000 7f328b7bc700 1 -- start start 2024-06-10T02:50:00.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:00.480+0000 7f328b7bc700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3284102770 0x7f328419e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:50:00.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:00.480+0000 7f328b7bc700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3284103120 0x7f328419ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:50:00.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:00.480+0000 7f328b7bc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3284107860 0x7f3284198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:50:00.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:00.480+0000 7f328b7bc700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f32841127d0 con 0x7f3284107860 2024-06-10T02:50:00.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:00.480+0000 7f328b7bc700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f3284112650 con 0x7f3284103120 2024-06-10T02:50:00.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:00.480+0000 7f328b7bc700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f3284112950 con 0x7f3284102770 2024-06-10T02:50:00.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:00.480+0000 7f3289d59700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3284107860 0x7f3284198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:50:00.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:00.480+0000 7f3289d59700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3284107860 0x7f3284198a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:38472/0 (socket says 172.21.15.52:38472) 2024-06-10T02:50:00.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:00.480+0000 7f3289d59700 1 -- 172.21.15.52:0/1455462303 learned_addr learned my addr 172.21.15.52:0/1455462303 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:50:00.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:00.480+0000 7f3289558700 1 --2- 172.21.15.52:0/1455462303 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3284102770 0x7f328419e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:50:00.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:00.480+0000 7f3288d57700 1 --2- 172.21.15.52:0/1455462303 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3284103120 0x7f328419ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:50:00.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:00.481+0000 7f3289d59700 1 -- 172.21.15.52:0/1455462303 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3284102770 msgr2=0x7f328419e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:50:00.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:00.481+0000 7f3289d59700 1 --2- 172.21.15.52:0/1455462303 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3284102770 0x7f328419e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:50:00.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:00.481+0000 7f3289d59700 1 -- 172.21.15.52:0/1455462303 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3284103120 msgr2=0x7f328419ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:50:00.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:00.481+0000 7f3289d59700 1 --2- 172.21.15.52:0/1455462303 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3284103120 0x7f328419ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:50:00.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:00.481+0000 7f3289d59700 1 -- 172.21.15.52:0/1455462303 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3284199170 con 0x7f3284107860 2024-06-10T02:50:00.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:00.481+0000 7f3289d59700 1 --2- 172.21.15.52:0/1455462303 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3284107860 0x7f3284198a10 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f327c007d70 tx=0x7f327c00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:50:00.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:00.481+0000 7f32767fc700 1 -- 172.21.15.52:0/1455462303 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f327c016020 con 0x7f3284107860 2024-06-10T02:50:00.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:00.481+0000 7f32767fc700 1 -- 172.21.15.52:0/1455462303 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f327c00ac90 con 0x7f3284107860 2024-06-10T02:50:00.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:00.481+0000 7f328b7bc700 1 -- 172.21.15.52:0/1455462303 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3284199460 con 0x7f3284107860 2024-06-10T02:50:00.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:00.481+0000 7f32767fc700 1 -- 172.21.15.52:0/1455462303 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f327c0115c0 con 0x7f3284107860 2024-06-10T02:50:00.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:00.481+0000 7f328b7bc700 1 -- 172.21.15.52:0/1455462303 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32841a5720 con 0x7f3284107860 2024-06-10T02:50:00.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:00.482+0000 7f328b7bc700 1 -- 172.21.15.52:0/1455462303 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f328410aaa0 con 0x7f3284107860 2024-06-10T02:50:00.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:00.483+0000 7f32767fc700 1 -- 172.21.15.52:0/1455462303 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f327c019020 con 0x7f3284107860 2024-06-10T02:50:00.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:00.484+0000 7f32767fc700 1 --2- 172.21.15.52:0/1455462303 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f32700a2ca0 0x7f32700a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:50:00.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:00.484+0000 7f3289558700 1 --2- 172.21.15.52:0/1455462303 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f32700a2ca0 0x7f32700a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:50:00.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:00.484+0000 7f32767fc700 1 -- 172.21.15.52:0/1455462303 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7497+0+0 (secure 0 0 0) 0x7f327c0cec30 con 0x7f3284107860 2024-06-10T02:50:00.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:00.484+0000 7f3289558700 1 --2- 172.21.15.52:0/1455462303 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f32700a2ca0 0x7f32700a5160 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f3280000c00 tx=0x7f3280005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:50:00.487 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:00.489+0000 7f32767fc700 1 -- 172.21.15.52:0/1455462303 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f327c020020 con 0x7f3284107860 2024-06-10T02:50:00.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:00.694+0000 7f328b7bc700 1 -- 172.21.15.52:0/1455462303 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) -- 0x7f3284199f70 con 0x7f3284107860 2024-06-10T02:50:01.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:01.181+0000 7f32767fc700 1 -- 172.21.15.52:0/1455462303 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v86) ==== 171+0+0 (secure 0 0 0) 0x7f3284199f70 con 0x7f3284107860 2024-06-10T02:50:01.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:01.184+0000 7f328b7bc700 1 -- 172.21.15.52:0/1455462303 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f32700a2ca0 msgr2=0x7f32700a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:50:01.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:01.184+0000 7f328b7bc700 1 --2- 172.21.15.52:0/1455462303 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f32700a2ca0 0x7f32700a5160 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f3280000c00 tx=0x7f3280005fb0 comp rx=0 tx=0).stop 2024-06-10T02:50:01.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:01.185+0000 7f328b7bc700 1 -- 172.21.15.52:0/1455462303 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3284107860 msgr2=0x7f3284198a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:50:01.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:01.185+0000 7f328b7bc700 1 --2- 172.21.15.52:0/1455462303 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3284107860 0x7f3284198a10 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f327c007d70 tx=0x7f327c00a340 comp rx=0 tx=0).stop 2024-06-10T02:50:01.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:01.185+0000 7f328b7bc700 1 -- 172.21.15.52:0/1455462303 shutdown_connections 2024-06-10T02:50:01.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:01.185+0000 7f328b7bc700 1 --2- 172.21.15.52:0/1455462303 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f32700a2ca0 0x7f32700a5160 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:50:01.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:01.185+0000 7f328b7bc700 1 --2- 172.21.15.52:0/1455462303 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3284107860 0x7f3284198a10 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:50:01.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:01.185+0000 7f328b7bc700 1 --2- 172.21.15.52:0/1455462303 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3284102770 0x7f328419e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:50:01.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:01.185+0000 7f328b7bc700 1 --2- 172.21.15.52:0/1455462303 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3284103120 0x7f328419ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:50:01.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:01.185+0000 7f328b7bc700 1 -- 172.21.15.52:0/1455462303 >> 172.21.15.52:0/1455462303 conn(0x7f3284079600 msgr2=0x7f3284100520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:50:01.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:01.185+0000 7f328b7bc700 1 -- 172.21.15.52:0/1455462303 shutdown_connections 2024-06-10T02:50:01.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:01.185+0000 7f328b7bc700 1 -- 172.21.15.52:0/1455462303 wait complete. 2024-06-10T02:50:01.185 INFO:teuthology.orchestra.run.smithi052.stderr:pool 'cephfs_data' created 2024-06-10T02:50:01.387 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:50:01 smithi120 ceph-mon[36634]: Health detail: HEALTH_WARN 1 pool(s) do not have an application enabled 2024-06-10T02:50:01.387 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:50:01 smithi120 ceph-mon[36634]: [WRN] POOL_APP_NOT_ENABLED: 1 pool(s) do not have an application enabled 2024-06-10T02:50:01.387 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:50:01 smithi120 ceph-mon[36634]: application not enabled on pool 'cephfs_metadata' 2024-06-10T02:50:01.387 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:50:01 smithi120 ceph-mon[36634]: use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-06-10T02:50:01.452 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:50:01 smithi188 ceph-mon[36599]: Health detail: HEALTH_WARN 1 pool(s) do not have an application enabled 2024-06-10T02:50:01.452 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:50:01 smithi188 ceph-mon[36599]: [WRN] POOL_APP_NOT_ENABLED: 1 pool(s) do not have an application enabled 2024-06-10T02:50:01.452 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:50:01 smithi188 ceph-mon[36599]: application not enabled on pool 'cephfs_metadata' 2024-06-10T02:50:01.452 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:50:01 smithi188 ceph-mon[36599]: use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-06-10T02:50:01.475 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:50:01 smithi052 ceph-mon[32362]: Health detail: HEALTH_WARN 1 pool(s) do not have an application enabled 2024-06-10T02:50:01.475 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:50:01 smithi052 ceph-mon[32362]: [WRN] POOL_APP_NOT_ENABLED: 1 pool(s) do not have an application enabled 2024-06-10T02:50:01.475 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:50:01 smithi052 ceph-mon[32362]: application not enabled on pool 'cephfs_metadata' 2024-06-10T02:50:01.475 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:50:01 smithi052 ceph-mon[32362]: use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-06-10T02:50:01.849 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph fs new cephfs cephfs_metadata cephfs_data 2024-06-10T02:50:02.102 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:50:02.387 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:50:02 smithi120 ceph-mon[36634]: from='client.? 172.21.15.52:0/1455462303' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-06-10T02:50:02.387 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:50:02 smithi120 ceph-mon[36634]: pgmap v294: 33 pgs: 6 creating+peering, 4 active+clean, 23 unknown; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T02:50:02.387 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:50:02 smithi120 ceph-mon[36634]: from='client.? 172.21.15.52:0/1455462303' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-06-10T02:50:02.388 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:50:02 smithi120 ceph-mon[36634]: osdmap e86: 12 total, 12 up, 12 in 2024-06-10T02:50:02.452 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:50:02 smithi188 ceph-mon[36599]: from='client.? 172.21.15.52:0/1455462303' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-06-10T02:50:02.452 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:50:02 smithi188 ceph-mon[36599]: pgmap v294: 33 pgs: 6 creating+peering, 4 active+clean, 23 unknown; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T02:50:02.452 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:50:02 smithi188 ceph-mon[36599]: from='client.? 172.21.15.52:0/1455462303' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-06-10T02:50:02.452 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:50:02 smithi188 ceph-mon[36599]: osdmap e86: 12 total, 12 up, 12 in 2024-06-10T02:50:02.455 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:50:02 smithi052 ceph-mon[32362]: from='client.? 172.21.15.52:0/1455462303' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-06-10T02:50:02.455 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:50:02 smithi052 ceph-mon[32362]: pgmap v294: 33 pgs: 6 creating+peering, 4 active+clean, 23 unknown; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T02:50:02.455 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:50:02 smithi052 ceph-mon[32362]: from='client.? 172.21.15.52:0/1455462303' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-06-10T02:50:02.455 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:50:02 smithi052 ceph-mon[32362]: osdmap e86: 12 total, 12 up, 12 in 2024-06-10T02:50:03.383 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:50:03 smithi052 ceph-mon[32362]: osdmap e87: 12 total, 12 up, 12 in 2024-06-10T02:50:03.452 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:50:03 smithi188 ceph-mon[36599]: osdmap e87: 12 total, 12 up, 12 in 2024-06-10T02:50:03.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:03.624+0000 7fb8f0c80700 1 -- 172.21.15.52:0/1616607831 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb8ec101390 msgr2=0x7fb8ec101810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:50:03.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:03.624+0000 7fb8f0c80700 1 --2- 172.21.15.52:0/1616607831 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb8ec101390 0x7fb8ec101810 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fb8e0009990 tx=0x7fb8e0009ca0 comp rx=0 tx=0).stop 2024-06-10T02:50:03.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:03.624+0000 7fb8f0c80700 1 -- 172.21.15.52:0/1616607831 shutdown_connections 2024-06-10T02:50:03.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:03.624+0000 7fb8f0c80700 1 --2- 172.21.15.52:0/1616607831 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb8ec101d50 0x7fb8ec10e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:50:03.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:03.624+0000 7fb8f0c80700 1 --2- 172.21.15.52:0/1616607831 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb8ec107400 0x7fb8ec1077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:50:03.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:03.624+0000 7fb8f0c80700 1 --2- 172.21.15.52:0/1616607831 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb8ec101390 0x7fb8ec101810 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:50:03.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:03.624+0000 7fb8f0c80700 1 -- 172.21.15.52:0/1616607831 >> 172.21.15.52:0/1616607831 conn(0x7fb8ec078340 msgr2=0x7fb8ec078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:50:03.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:03.624+0000 7fb8f0c80700 1 -- 172.21.15.52:0/1616607831 shutdown_connections 2024-06-10T02:50:03.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:03.625+0000 7fb8f0c80700 1 -- 172.21.15.52:0/1616607831 wait complete. 2024-06-10T02:50:03.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:03.625+0000 7fb8f0c80700 1 Processor -- start 2024-06-10T02:50:03.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:03.625+0000 7fb8f0c80700 1 -- start start 2024-06-10T02:50:03.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:03.626+0000 7fb8f0c80700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb8ec101390 0x7fb8ec103070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:50:03.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:03.626+0000 7fb8f0c80700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb8ec101d50 0x7fb8ec1035b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:50:03.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:03.626+0000 7fb8f0c80700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb8ec107400 0x7fb8ec103af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:50:03.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:03.626+0000 7fb8f0c80700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb8ec1116f0 con 0x7fb8ec101d50 2024-06-10T02:50:03.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:03.626+0000 7fb8f0c80700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fb8ec111570 con 0x7fb8ec107400 2024-06-10T02:50:03.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:03.626+0000 7fb8f0c80700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fb8ec111870 con 0x7fb8ec101390 2024-06-10T02:50:03.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:03.626+0000 7fb8e9d9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb8ec101d50 0x7fb8ec1035b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:50:03.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:03.626+0000 7fb8e9d9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb8ec101d50 0x7fb8ec1035b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:41192/0 (socket says 172.21.15.52:41192) 2024-06-10T02:50:03.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:03.626+0000 7fb8e9d9b700 1 -- 172.21.15.52:0/3734372837 learned_addr learned my addr 172.21.15.52:0/3734372837 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:50:03.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:03.626+0000 7fb8ead9d700 1 --2- 172.21.15.52:0/3734372837 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb8ec107400 0x7fb8ec103af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:50:03.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:03.626+0000 7fb8ea59c700 1 --2- 172.21.15.52:0/3734372837 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb8ec101390 0x7fb8ec103070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:50:03.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:03.626+0000 7fb8e9d9b700 1 -- 172.21.15.52:0/3734372837 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb8ec101390 msgr2=0x7fb8ec103070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:50:03.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:03.626+0000 7fb8e9d9b700 1 --2- 172.21.15.52:0/3734372837 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb8ec101390 0x7fb8ec103070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:50:03.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:03.626+0000 7fb8e9d9b700 1 -- 172.21.15.52:0/3734372837 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb8ec107400 msgr2=0x7fb8ec103af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:50:03.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:03.626+0000 7fb8e9d9b700 1 --2- 172.21.15.52:0/3734372837 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb8ec107400 0x7fb8ec103af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:50:03.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:03.626+0000 7fb8e9d9b700 1 -- 172.21.15.52:0/3734372837 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8ec1a7190 con 0x7fb8ec101d50 2024-06-10T02:50:03.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:03.626+0000 7fb8ea59c700 1 --2- 172.21.15.52:0/3734372837 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb8ec101390 0x7fb8ec103070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T02:50:03.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:03.626+0000 7fb8e9d9b700 1 --2- 172.21.15.52:0/3734372837 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb8ec101d50 0x7fb8ec1035b0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fb8e0000c00 tx=0x7fb8e00047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:50:03.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:03.626+0000 7fb8db7fe700 1 -- 172.21.15.52:0/3734372837 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8e0027070 con 0x7fb8ec101d50 2024-06-10T02:50:03.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:03.626+0000 7fb8db7fe700 1 -- 172.21.15.52:0/3734372837 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb8e0004050 con 0x7fb8ec101d50 2024-06-10T02:50:03.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:03.626+0000 7fb8f0c80700 1 -- 172.21.15.52:0/3734372837 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8ec1a7420 con 0x7fb8ec101d50 2024-06-10T02:50:03.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:03.626+0000 7fb8db7fe700 1 -- 172.21.15.52:0/3734372837 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8e000f460 con 0x7fb8ec101d50 2024-06-10T02:50:03.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:03.627+0000 7fb8f0c80700 1 -- 172.21.15.52:0/3734372837 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8ec1a7870 con 0x7fb8ec101d50 2024-06-10T02:50:03.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:03.627+0000 7fb8f0c80700 1 -- 172.21.15.52:0/3734372837 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8ec069180 con 0x7fb8ec101d50 2024-06-10T02:50:03.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:03.629+0000 7fb8db7fe700 1 -- 172.21.15.52:0/3734372837 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fb8e002a070 con 0x7fb8ec101d50 2024-06-10T02:50:03.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:03.629+0000 7fb8db7fe700 1 --2- 172.21.15.52:0/3734372837 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fb8d40a2c20 0x7fb8d40a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:50:03.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:03.630+0000 7fb8ea59c700 1 --2- 172.21.15.52:0/3734372837 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fb8d40a2c20 0x7fb8d40a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:50:03.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:03.630+0000 7fb8db7fe700 1 -- 172.21.15.52:0/3734372837 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7879+0+0 (secure 0 0 0) 0x7fb8e00d02e0 con 0x7fb8ec101d50 2024-06-10T02:50:03.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:03.630+0000 7fb8ea59c700 1 --2- 172.21.15.52:0/3734372837 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fb8d40a2c20 0x7fb8d40a50e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fb8dc005fd0 tx=0x7fb8dc005ee0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:50:03.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:03.635+0000 7fb8db7fe700 1 -- 172.21.15.52:0/3734372837 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fb8e0031070 con 0x7fb8ec101d50 2024-06-10T02:50:03.637 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:50:03 smithi120 ceph-mon[36634]: osdmap e87: 12 total, 12 up, 12 in 2024-06-10T02:50:03.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:03.888+0000 7fb8f0c80700 1 -- 172.21.15.52:0/3734372837 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fb8ec1a7c00 con 0x7fb8ec101d50 2024-06-10T02:50:04.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:04.192+0000 7fb8db7fe700 1 -- 172.21.15.52:0/3734372837 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7fb8e0013070 con 0x7fb8ec101d50 2024-06-10T02:50:04.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:04.195+0000 7fb8f0c80700 1 -- 172.21.15.52:0/3734372837 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fb8d40a2c20 msgr2=0x7fb8d40a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:50:04.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:04.195+0000 7fb8f0c80700 1 --2- 172.21.15.52:0/3734372837 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fb8d40a2c20 0x7fb8d40a50e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fb8dc005fd0 tx=0x7fb8dc005ee0 comp rx=0 tx=0).stop 2024-06-10T02:50:04.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:04.195+0000 7fb8f0c80700 1 -- 172.21.15.52:0/3734372837 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb8ec101d50 msgr2=0x7fb8ec1035b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:50:04.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:04.195+0000 7fb8f0c80700 1 --2- 172.21.15.52:0/3734372837 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb8ec101d50 0x7fb8ec1035b0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fb8e0000c00 tx=0x7fb8e00047b0 comp rx=0 tx=0).stop 2024-06-10T02:50:04.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:04.195+0000 7fb8f0c80700 1 -- 172.21.15.52:0/3734372837 shutdown_connections 2024-06-10T02:50:04.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:04.195+0000 7fb8f0c80700 1 --2- 172.21.15.52:0/3734372837 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb8ec107400 0x7fb8ec103af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:50:04.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:04.195+0000 7fb8f0c80700 1 --2- 172.21.15.52:0/3734372837 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb8ec101390 0x7fb8ec103070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:50:04.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:04.195+0000 7fb8f0c80700 1 --2- 172.21.15.52:0/3734372837 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fb8d40a2c20 0x7fb8d40a50e0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:50:04.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:04.195+0000 7fb8f0c80700 1 --2- 172.21.15.52:0/3734372837 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb8ec101d50 0x7fb8ec1035b0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:50:04.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:04.195+0000 7fb8f0c80700 1 -- 172.21.15.52:0/3734372837 >> 172.21.15.52:0/3734372837 conn(0x7fb8ec078340 msgr2=0x7fb8ec100580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:50:04.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:04.195+0000 7fb8f0c80700 1 -- 172.21.15.52:0/3734372837 shutdown_connections 2024-06-10T02:50:04.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:04.195+0000 7fb8f0c80700 1 -- 172.21.15.52:0/3734372837 wait complete. 2024-06-10T02:50:04.195 INFO:teuthology.orchestra.run.smithi052.stderr:new fs with metadata pool 2 and data pool 3 2024-06-10T02:50:04.452 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:50:04 smithi188 ceph-mon[36599]: pgmap v297: 97 pgs: 27 creating+peering, 54 active+clean, 16 unknown; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T02:50:04.452 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:50:04 smithi188 ceph-mon[36599]: osdmap e88: 12 total, 12 up, 12 in 2024-06-10T02:50:04.452 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:50:04 smithi188 ceph-mon[36599]: from='client.? 172.21.15.52:0/3734372837' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-06-10T02:50:04.452 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:50:04 smithi188 ceph-mon[36599]: daemon mds.i assigned to filesystem cephfs as rank 0 2024-06-10T02:50:04.475 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:50:04 smithi052 ceph-mon[32362]: pgmap v297: 97 pgs: 27 creating+peering, 54 active+clean, 16 unknown; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T02:50:04.475 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:50:04 smithi052 ceph-mon[32362]: osdmap e88: 12 total, 12 up, 12 in 2024-06-10T02:50:04.475 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:50:04 smithi052 ceph-mon[32362]: from='client.? 172.21.15.52:0/3734372837' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-06-10T02:50:04.475 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:50:04 smithi052 ceph-mon[32362]: daemon mds.i assigned to filesystem cephfs as rank 0 2024-06-10T02:50:04.637 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:50:04 smithi120 ceph-mon[36634]: pgmap v297: 97 pgs: 27 creating+peering, 54 active+clean, 16 unknown; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T02:50:04.637 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:50:04 smithi120 ceph-mon[36634]: osdmap e88: 12 total, 12 up, 12 in 2024-06-10T02:50:04.637 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:50:04 smithi120 ceph-mon[36634]: from='client.? 172.21.15.52:0/3734372837' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-06-10T02:50:04.637 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:50:04 smithi120 ceph-mon[36634]: daemon mds.i assigned to filesystem cephfs as rank 0 2024-06-10T02:50:04.948 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-06-10T02:50:04.948 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-06-10T02:50:05.201 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:50:05.453 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:50:05 smithi052 ceph-mon[32362]: from='client.? 172.21.15.52:0/3734372837' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-06-10T02:50:05.453 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:50:05 smithi052 ceph-mon[32362]: osdmap e89: 12 total, 12 up, 12 in 2024-06-10T02:50:05.453 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:50:05 smithi052 ceph-mon[32362]: fsmap cephfs:1 {0=i=up:creating} 11 up:standby 2024-06-10T02:50:05.453 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:50:05 smithi052 ceph-mon[32362]: daemon mds.i is now active in filesystem cephfs as rank 0 2024-06-10T02:50:05.637 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:50:05 smithi120 ceph-mon[36634]: from='client.? 172.21.15.52:0/3734372837' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-06-10T02:50:05.637 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:50:05 smithi120 ceph-mon[36634]: osdmap e89: 12 total, 12 up, 12 in 2024-06-10T02:50:05.637 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:50:05 smithi120 ceph-mon[36634]: fsmap cephfs:1 {0=i=up:creating} 11 up:standby 2024-06-10T02:50:05.637 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:50:05 smithi120 ceph-mon[36634]: daemon mds.i is now active in filesystem cephfs as rank 0 2024-06-10T02:50:05.663 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:50:05 smithi188 ceph-mon[36599]: from='client.? 172.21.15.52:0/3734372837' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-06-10T02:50:05.663 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:50:05 smithi188 ceph-mon[36599]: osdmap e89: 12 total, 12 up, 12 in 2024-06-10T02:50:05.664 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:50:05 smithi188 ceph-mon[36599]: fsmap cephfs:1 {0=i=up:creating} 11 up:standby 2024-06-10T02:50:05.664 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:50:05 smithi188 ceph-mon[36599]: daemon mds.i is now active in filesystem cephfs as rank 0 2024-06-10T02:50:06.378 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:50:06 smithi052 ceph-mon[32362]: pgmap v300: 97 pgs: 21 creating+peering, 72 active+clean, 4 unknown; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T02:50:06.379 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:50:06 smithi052 ceph-mon[32362]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-06-10T02:50:06.379 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:50:06 smithi052 ceph-mon[32362]: Cluster is now healthy 2024-06-10T02:50:06.379 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:50:06 smithi052 ceph-mon[32362]: mds.? [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] up:active 2024-06-10T02:50:06.379 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:50:06 smithi052 ceph-mon[32362]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-06-10T02:50:06.379 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:50:06 smithi052 ceph-mon[32362]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-06-10T02:50:06.508 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:50:06 smithi188 ceph-mon[36599]: pgmap v300: 97 pgs: 21 creating+peering, 72 active+clean, 4 unknown; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T02:50:06.508 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:50:06 smithi188 ceph-mon[36599]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-06-10T02:50:06.508 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:50:06 smithi188 ceph-mon[36599]: Cluster is now healthy 2024-06-10T02:50:06.508 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:50:06 smithi188 ceph-mon[36599]: mds.? [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] up:active 2024-06-10T02:50:06.508 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:50:06 smithi188 ceph-mon[36599]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-06-10T02:50:06.508 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:50:06 smithi188 ceph-mon[36599]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-06-10T02:50:06.637 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:50:06 smithi120 ceph-mon[36634]: pgmap v300: 97 pgs: 21 creating+peering, 72 active+clean, 4 unknown; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T02:50:06.637 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:50:06 smithi120 ceph-mon[36634]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-06-10T02:50:06.637 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:50:06 smithi120 ceph-mon[36634]: Cluster is now healthy 2024-06-10T02:50:06.637 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:50:06 smithi120 ceph-mon[36634]: mds.? [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] up:active 2024-06-10T02:50:06.637 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:50:06 smithi120 ceph-mon[36634]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-06-10T02:50:06.637 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:50:06 smithi120 ceph-mon[36634]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-06-10T02:50:06.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:06.938+0000 7fbb36865700 1 -- 172.21.15.52:0/4252471920 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbb30101d40 msgr2=0x7fbb3010e230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:50:06.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:06.938+0000 7fbb36865700 1 --2- 172.21.15.52:0/4252471920 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbb30101d40 0x7fbb3010e230 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fbb2800b0a0 tx=0x7fbb2800b3b0 comp rx=0 tx=0).stop 2024-06-10T02:50:06.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:06.938+0000 7fbb36865700 1 -- 172.21.15.52:0/4252471920 shutdown_connections 2024-06-10T02:50:06.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:06.938+0000 7fbb36865700 1 --2- 172.21.15.52:0/4252471920 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbb30101d40 0x7fbb3010e230 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:50:06.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:06.938+0000 7fbb36865700 1 --2- 172.21.15.52:0/4252471920 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbb301073f0 0x7fbb301077d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:50:06.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:06.939+0000 7fbb36865700 1 --2- 172.21.15.52:0/4252471920 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbb30101380 0x7fbb30101800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:50:06.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:06.939+0000 7fbb36865700 1 -- 172.21.15.52:0/4252471920 >> 172.21.15.52:0/4252471920 conn(0x7fbb30078300 msgr2=0x7fbb30078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:50:06.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:06.939+0000 7fbb36865700 1 -- 172.21.15.52:0/4252471920 shutdown_connections 2024-06-10T02:50:06.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:06.939+0000 7fbb36865700 1 -- 172.21.15.52:0/4252471920 wait complete. 2024-06-10T02:50:06.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:06.939+0000 7fbb36865700 1 Processor -- start 2024-06-10T02:50:06.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:06.940+0000 7fbb36865700 1 -- start start 2024-06-10T02:50:06.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:06.940+0000 7fbb36865700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbb30101380 0x7fbb3019e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:50:06.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:06.940+0000 7fbb36865700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbb30101d40 0x7fbb3019ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:50:06.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:06.940+0000 7fbb36865700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbb301073f0 0x7fbb30198a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:50:06.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:06.940+0000 7fbb36865700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fbb30111680 con 0x7fbb30101380 2024-06-10T02:50:06.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:06.940+0000 7fbb36865700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fbb30111500 con 0x7fbb301073f0 2024-06-10T02:50:06.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:06.940+0000 7fbb36865700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fbb30111800 con 0x7fbb30101d40 2024-06-10T02:50:06.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:06.940+0000 7fbb2ffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbb30101380 0x7fbb3019e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:50:06.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:06.940+0000 7fbb2ffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbb30101380 0x7fbb3019e810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:41212/0 (socket says 172.21.15.52:41212) 2024-06-10T02:50:06.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:06.940+0000 7fbb2ffff700 1 -- 172.21.15.52:0/412507967 learned_addr learned my addr 172.21.15.52:0/412507967 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:50:06.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:06.940+0000 7fbb27fff700 1 --2- 172.21.15.52:0/412507967 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbb30101d40 0x7fbb3019ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:50:06.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:06.940+0000 7fbb34e02700 1 --2- 172.21.15.52:0/412507967 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbb301073f0 0x7fbb30198a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:50:06.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:06.940+0000 7fbb2ffff700 1 -- 172.21.15.52:0/412507967 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbb30101d40 msgr2=0x7fbb3019ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:50:06.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:06.940+0000 7fbb2ffff700 1 --2- 172.21.15.52:0/412507967 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbb30101d40 0x7fbb3019ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:50:06.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:06.940+0000 7fbb2ffff700 1 -- 172.21.15.52:0/412507967 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbb301073f0 msgr2=0x7fbb30198a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:50:06.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:06.940+0000 7fbb2ffff700 1 --2- 172.21.15.52:0/412507967 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbb301073f0 0x7fbb30198a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:50:06.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:06.940+0000 7fbb2ffff700 1 -- 172.21.15.52:0/412507967 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb30199180 con 0x7fbb30101380 2024-06-10T02:50:06.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:06.941+0000 7fbb2ffff700 1 --2- 172.21.15.52:0/412507967 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbb30101380 0x7fbb3019e810 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fbb2000d980 tx=0x7fbb2000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:50:06.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:06.941+0000 7fbb2dffb700 1 -- 172.21.15.52:0/412507967 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb20017070 con 0x7fbb30101380 2024-06-10T02:50:06.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:06.941+0000 7fbb2dffb700 1 -- 172.21.15.52:0/412507967 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb20005020 con 0x7fbb30101380 2024-06-10T02:50:06.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:06.941+0000 7fbb36865700 1 -- 172.21.15.52:0/412507967 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb30199470 con 0x7fbb30101380 2024-06-10T02:50:06.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:06.941+0000 7fbb2dffb700 1 -- 172.21.15.52:0/412507967 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb2000fbf0 con 0x7fbb30101380 2024-06-10T02:50:06.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:06.941+0000 7fbb36865700 1 -- 172.21.15.52:0/412507967 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb30102f90 con 0x7fbb30101380 2024-06-10T02:50:06.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:06.942+0000 7fbb36865700 1 -- 172.21.15.52:0/412507967 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb30069180 con 0x7fbb30101380 2024-06-10T02:50:06.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:06.943+0000 7fbb2dffb700 1 -- 172.21.15.52:0/412507967 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fbb200051d0 con 0x7fbb30101380 2024-06-10T02:50:06.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:06.944+0000 7fbb2dffb700 1 --2- 172.21.15.52:0/412507967 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fbb1c0a2ca0 0x7fbb1c0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:50:06.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:06.944+0000 7fbb27fff700 1 --2- 172.21.15.52:0/412507967 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fbb1c0a2ca0 0x7fbb1c0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:50:06.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:06.944+0000 7fbb2dffb700 1 -- 172.21.15.52:0/412507967 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 7979+0+0 (secure 0 0 0) 0x7fbb200cec60 con 0x7fbb30101380 2024-06-10T02:50:06.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:06.944+0000 7fbb27fff700 1 --2- 172.21.15.52:0/412507967 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fbb1c0a2ca0 0x7fbb1c0a5160 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fbb30078900 tx=0x7fbb18009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:50:06.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:06.948+0000 7fbb2dffb700 1 -- 172.21.15.52:0/412507967 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fbb20020020 con 0x7fbb30101380 2024-06-10T02:50:06.952 INFO:journalctl@ceph.mds.i.smithi188.stdout:Jun 10 02:50:06 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mds-i[66946]: 2024-06-10T02:50:06.508+0000 7f087a23e700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-06-10T02:50:07.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:07.168+0000 7fbb36865700 1 -- 172.21.15.52:0/412507967 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) -- 0x7fbb300630a0 con 0x7fbb30101380 2024-06-10T02:50:07.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:07.210+0000 7fbb2dffb700 1 -- 172.21.15.52:0/412507967 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v90) ==== 157+0+0 (secure 0 0 0) 0x7fbb2009bd10 con 0x7fbb30101380 2024-06-10T02:50:07.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:07.213+0000 7fbb36865700 1 -- 172.21.15.52:0/412507967 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fbb1c0a2ca0 msgr2=0x7fbb1c0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:50:07.212 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:07.213+0000 7fbb36865700 1 --2- 172.21.15.52:0/412507967 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fbb1c0a2ca0 0x7fbb1c0a5160 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fbb30078900 tx=0x7fbb18009260 comp rx=0 tx=0).stop 2024-06-10T02:50:07.212 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:07.213+0000 7fbb36865700 1 -- 172.21.15.52:0/412507967 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbb30101380 msgr2=0x7fbb3019e810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:50:07.212 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:07.213+0000 7fbb36865700 1 --2- 172.21.15.52:0/412507967 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbb30101380 0x7fbb3019e810 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fbb2000d980 tx=0x7fbb2000dc90 comp rx=0 tx=0).stop 2024-06-10T02:50:07.212 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:07.213+0000 7fbb36865700 1 -- 172.21.15.52:0/412507967 shutdown_connections 2024-06-10T02:50:07.212 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:07.213+0000 7fbb36865700 1 --2- 172.21.15.52:0/412507967 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbb301073f0 0x7fbb30198a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:50:07.212 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:07.213+0000 7fbb36865700 1 --2- 172.21.15.52:0/412507967 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fbb1c0a2ca0 0x7fbb1c0a5160 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:50:07.212 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:07.213+0000 7fbb36865700 1 --2- 172.21.15.52:0/412507967 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbb30101380 0x7fbb3019e810 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:50:07.212 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:07.213+0000 7fbb36865700 1 --2- 172.21.15.52:0/412507967 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbb30101d40 0x7fbb3019ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:50:07.212 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:07.213+0000 7fbb36865700 1 -- 172.21.15.52:0/412507967 >> 172.21.15.52:0/412507967 conn(0x7fbb30078300 msgr2=0x7fbb30100680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:50:07.212 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:07.213+0000 7fbb36865700 1 -- 172.21.15.52:0/412507967 shutdown_connections 2024-06-10T02:50:07.212 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:07.213+0000 7fbb36865700 1 -- 172.21.15.52:0/412507967 wait complete. 2024-06-10T02:50:07.475 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:50:07 smithi052 ceph-mon[32362]: from='client.? 172.21.15.52:0/412507967' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-06-10T02:50:07.637 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:50:07 smithi120 ceph-mon[36634]: from='client.? 172.21.15.52:0/412507967' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-06-10T02:50:07.702 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:50:07 smithi188 ceph-mon[36599]: from='client.? 172.21.15.52:0/412507967' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-06-10T02:50:07.866 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-06-10T02:50:08.116 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:50:08.474 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:50:08 smithi052 ceph-mon[32362]: pgmap v301: 97 pgs: 8 creating+peering, 89 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 3 op/s 2024-06-10T02:50:08.474 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:50:08 smithi052 ceph-mon[32362]: from='client.? 172.21.15.52:0/412507967' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-06-10T02:50:08.474 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:50:08 smithi052 ceph-mon[32362]: osdmap e90: 12 total, 12 up, 12 in 2024-06-10T02:50:08.637 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:50:08 smithi120 ceph-mon[36634]: pgmap v301: 97 pgs: 8 creating+peering, 89 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 3 op/s 2024-06-10T02:50:08.637 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:50:08 smithi120 ceph-mon[36634]: from='client.? 172.21.15.52:0/412507967' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-06-10T02:50:08.637 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:50:08 smithi120 ceph-mon[36634]: osdmap e90: 12 total, 12 up, 12 in 2024-06-10T02:50:08.702 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:50:08 smithi188 ceph-mon[36599]: pgmap v301: 97 pgs: 8 creating+peering, 89 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 3 op/s 2024-06-10T02:50:08.702 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:50:08 smithi188 ceph-mon[36599]: from='client.? 172.21.15.52:0/412507967' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-06-10T02:50:08.703 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:50:08 smithi188 ceph-mon[36599]: osdmap e90: 12 total, 12 up, 12 in 2024-06-10T02:50:09.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:09.917+0000 7f3b1014d700 1 -- 172.21.15.52:0/2915435406 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3b081015f0 msgr2=0x7f3b081019d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:50:09.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:09.917+0000 7f3b1014d700 1 --2- 172.21.15.52:0/2915435406 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3b081015f0 0x7f3b081019d0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f3b040099c0 tx=0x7f3b04009cd0 comp rx=0 tx=0).stop 2024-06-10T02:50:09.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:09.917+0000 7f3b1014d700 1 -- 172.21.15.52:0/2915435406 shutdown_connections 2024-06-10T02:50:09.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:09.917+0000 7f3b1014d700 1 --2- 172.21.15.52:0/2915435406 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3b081066c0 0x7f3b0810e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:50:09.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:09.917+0000 7f3b1014d700 1 --2- 172.21.15.52:0/2915435406 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3b081015f0 0x7f3b081019d0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:50:09.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:09.917+0000 7f3b1014d700 1 --2- 172.21.15.52:0/2915435406 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3b08101fa0 0x7f3b08105fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:50:09.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:09.917+0000 7f3b1014d700 1 -- 172.21.15.52:0/2915435406 >> 172.21.15.52:0/2915435406 conn(0x7f3b08078330 msgr2=0x7f3b08078740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:50:09.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:09.917+0000 7f3b1014d700 1 -- 172.21.15.52:0/2915435406 shutdown_connections 2024-06-10T02:50:09.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:09.917+0000 7f3b1014d700 1 -- 172.21.15.52:0/2915435406 wait complete. 2024-06-10T02:50:09.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:09.918+0000 7f3b1014d700 1 Processor -- start 2024-06-10T02:50:09.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:09.918+0000 7f3b1014d700 1 -- start start 2024-06-10T02:50:09.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:09.918+0000 7f3b1014d700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3b081015f0 0x7f3b08103080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:50:09.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:09.918+0000 7f3b1014d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3b08101fa0 0x7f3b081035c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:50:09.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:09.918+0000 7f3b1014d700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3b081066c0 0x7f3b08103b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:50:09.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:09.918+0000 7f3b1014d700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f3b081116d0 con 0x7f3b08101fa0 2024-06-10T02:50:09.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:09.918+0000 7f3b1014d700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f3b08111550 con 0x7f3b081015f0 2024-06-10T02:50:09.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:09.918+0000 7f3b1014d700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f3b08111850 con 0x7f3b081066c0 2024-06-10T02:50:09.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:09.918+0000 7f3b0d6e8700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3b08101fa0 0x7f3b081035c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:50:09.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:09.918+0000 7f3b0d6e8700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3b08101fa0 0x7f3b081035c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:41236/0 (socket says 172.21.15.52:41236) 2024-06-10T02:50:09.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:09.918+0000 7f3b0d6e8700 1 -- 172.21.15.52:0/2099764350 learned_addr learned my addr 172.21.15.52:0/2099764350 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:50:09.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:09.918+0000 7f3b0dee9700 1 --2- 172.21.15.52:0/2099764350 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3b081015f0 0x7f3b08103080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:50:09.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:09.918+0000 7f3b0e6ea700 1 --2- 172.21.15.52:0/2099764350 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3b081066c0 0x7f3b08103b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:50:09.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:09.919+0000 7f3b0d6e8700 1 -- 172.21.15.52:0/2099764350 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3b081066c0 msgr2=0x7f3b08103b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:50:09.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:09.919+0000 7f3b0d6e8700 1 --2- 172.21.15.52:0/2099764350 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3b081066c0 0x7f3b08103b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:50:09.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:09.919+0000 7f3b0d6e8700 1 -- 172.21.15.52:0/2099764350 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3b081015f0 msgr2=0x7f3b08103080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:50:09.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:09.919+0000 7f3b0d6e8700 1 --2- 172.21.15.52:0/2099764350 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3b081015f0 0x7f3b08103080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:50:09.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:09.919+0000 7f3b0d6e8700 1 -- 172.21.15.52:0/2099764350 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b081a7100 con 0x7f3b08101fa0 2024-06-10T02:50:09.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:09.919+0000 7f3b0e6ea700 1 --2- 172.21.15.52:0/2099764350 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3b081066c0 0x7f3b08103b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T02:50:09.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:09.919+0000 7f3b0d6e8700 1 --2- 172.21.15.52:0/2099764350 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3b08101fa0 0x7f3b081035c0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f3af800d950 tx=0x7f3af800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:50:09.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:09.919+0000 7f3afeffd700 1 -- 172.21.15.52:0/2099764350 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3af8016070 con 0x7f3b08101fa0 2024-06-10T02:50:09.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:09.919+0000 7f3afeffd700 1 -- 172.21.15.52:0/2099764350 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3af8004d10 con 0x7f3b08101fa0 2024-06-10T02:50:09.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:09.919+0000 7f3afeffd700 1 -- 172.21.15.52:0/2099764350 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3af80055e0 con 0x7f3b08101fa0 2024-06-10T02:50:09.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:09.919+0000 7f3b1014d700 1 -- 172.21.15.52:0/2099764350 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b081a73f0 con 0x7f3b08101fa0 2024-06-10T02:50:09.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:09.919+0000 7f3b1014d700 1 -- 172.21.15.52:0/2099764350 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b081a7980 con 0x7f3b08101fa0 2024-06-10T02:50:09.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:09.920+0000 7f3b1014d700 1 -- 172.21.15.52:0/2099764350 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b08050a00 con 0x7f3b08101fa0 2024-06-10T02:50:09.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:09.921+0000 7f3afeffd700 1 -- 172.21.15.52:0/2099764350 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f3af8004750 con 0x7f3b08101fa0 2024-06-10T02:50:09.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:09.922+0000 7f3afeffd700 1 --2- 172.21.15.52:0/2099764350 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f3af40a2d40 0x7f3af40a5200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:50:09.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:09.922+0000 7f3afeffd700 1 -- 172.21.15.52:0/2099764350 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f3af80ceda0 con 0x7f3b08101fa0 2024-06-10T02:50:09.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:09.922+0000 7f3b0dee9700 1 --2- 172.21.15.52:0/2099764350 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f3af40a2d40 0x7f3af40a5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:50:09.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:09.922+0000 7f3b0dee9700 1 --2- 172.21.15.52:0/2099764350 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f3af40a2d40 0x7f3af40a5200 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f3b04000c00 tx=0x7f3b04005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:50:09.925 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:09.927+0000 7f3afeffd700 1 -- 172.21.15.52:0/2099764350 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f3af801f020 con 0x7f3b08101fa0 2024-06-10T02:50:10.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:10.134+0000 7f3b1014d700 1 -- 172.21.15.52:0/2099764350 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f3b0802dca0 con 0x7f3b08101fa0 2024-06-10T02:50:10.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:50:10 smithi188 ceph-mon[36599]: pgmap v303: 97 pgs: 97 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 7 op/s 2024-06-10T02:50:10.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:50:10 smithi188 ceph-mon[36599]: from='client.? 172.21.15.52:0/2099764350' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-06-10T02:50:10.975 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:50:10 smithi052 ceph-mon[32362]: pgmap v303: 97 pgs: 97 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 7 op/s 2024-06-10T02:50:10.975 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:50:10 smithi052 ceph-mon[32362]: from='client.? 172.21.15.52:0/2099764350' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-06-10T02:50:11.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:50:10 smithi120 ceph-mon[36634]: pgmap v303: 97 pgs: 97 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 7 op/s 2024-06-10T02:50:11.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:50:10 smithi120 ceph-mon[36634]: from='client.? 172.21.15.52:0/2099764350' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-06-10T02:50:11.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:11.217+0000 7f3afeffd700 1 -- 172.21.15.52:0/2099764350 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v92) ==== 228+0+0 (secure 0 0 0) 0x7f3af809bcd0 con 0x7f3b08101fa0 2024-06-10T02:50:11.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:11.220+0000 7f3b1014d700 1 -- 172.21.15.52:0/2099764350 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f3af40a2d40 msgr2=0x7f3af40a5200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:50:11.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:11.220+0000 7f3b1014d700 1 --2- 172.21.15.52:0/2099764350 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f3af40a2d40 0x7f3af40a5200 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f3b04000c00 tx=0x7f3b04005fb0 comp rx=0 tx=0).stop 2024-06-10T02:50:11.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:11.220+0000 7f3b1014d700 1 -- 172.21.15.52:0/2099764350 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3b08101fa0 msgr2=0x7f3b081035c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:50:11.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:11.220+0000 7f3b1014d700 1 --2- 172.21.15.52:0/2099764350 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3b08101fa0 0x7f3b081035c0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f3af800d950 tx=0x7f3af800dc60 comp rx=0 tx=0).stop 2024-06-10T02:50:11.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:11.220+0000 7f3b1014d700 1 -- 172.21.15.52:0/2099764350 shutdown_connections 2024-06-10T02:50:11.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:11.220+0000 7f3b1014d700 1 --2- 172.21.15.52:0/2099764350 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3b081066c0 0x7f3b08103b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:50:11.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:11.220+0000 7f3b1014d700 1 --2- 172.21.15.52:0/2099764350 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3b081015f0 0x7f3b08103080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:50:11.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:11.220+0000 7f3b1014d700 1 --2- 172.21.15.52:0/2099764350 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f3af40a2d40 0x7f3af40a5200 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:50:11.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:11.220+0000 7f3b1014d700 1 --2- 172.21.15.52:0/2099764350 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3b08101fa0 0x7f3b081035c0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:50:11.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:11.220+0000 7f3b1014d700 1 -- 172.21.15.52:0/2099764350 >> 172.21.15.52:0/2099764350 conn(0x7f3b08078330 msgr2=0x7f3b0806a3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:50:11.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:11.220+0000 7f3b1014d700 1 -- 172.21.15.52:0/2099764350 shutdown_connections 2024-06-10T02:50:11.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:11.220+0000 7f3b1014d700 1 -- 172.21.15.52:0/2099764350 wait complete. 2024-06-10T02:50:11.220 INFO:teuthology.orchestra.run.smithi052.stderr:pool 'cephfs_data_ec' created 2024-06-10T02:50:11.831 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:50:11 smithi052 ceph-mon[32362]: osdmap e91: 12 total, 12 up, 12 in 2024-06-10T02:50:11.831 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:50:11 smithi052 ceph-mon[32362]: from='client.? 172.21.15.52:0/2099764350' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-06-10T02:50:11.831 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:50:11 smithi052 ceph-mon[32362]: osdmap e92: 12 total, 12 up, 12 in 2024-06-10T02:50:11.863 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-06-10T02:50:11.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:50:11 smithi188 ceph-mon[36599]: osdmap e91: 12 total, 12 up, 12 in 2024-06-10T02:50:11.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:50:11 smithi188 ceph-mon[36599]: from='client.? 172.21.15.52:0/2099764350' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-06-10T02:50:11.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:50:11 smithi188 ceph-mon[36599]: osdmap e92: 12 total, 12 up, 12 in 2024-06-10T02:50:12.119 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:50:12.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:50:11 smithi120 ceph-mon[36634]: osdmap e91: 12 total, 12 up, 12 in 2024-06-10T02:50:12.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:50:11 smithi120 ceph-mon[36634]: from='client.? 172.21.15.52:0/2099764350' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-06-10T02:50:12.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:50:11 smithi120 ceph-mon[36634]: osdmap e92: 12 total, 12 up, 12 in 2024-06-10T02:50:12.946 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:50:12 smithi052 ceph-mon[32362]: pgmap v305: 97 pgs: 97 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s wr, 6 op/s 2024-06-10T02:50:12.947 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:50:12 smithi052 ceph-mon[32362]: osdmap e93: 12 total, 12 up, 12 in 2024-06-10T02:50:12.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:50:12 smithi188 ceph-mon[36599]: pgmap v305: 97 pgs: 97 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s wr, 6 op/s 2024-06-10T02:50:12.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:50:12 smithi188 ceph-mon[36599]: osdmap e93: 12 total, 12 up, 12 in 2024-06-10T02:50:13.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:50:12 smithi120 ceph-mon[36634]: pgmap v305: 97 pgs: 97 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s wr, 6 op/s 2024-06-10T02:50:13.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:50:12 smithi120 ceph-mon[36634]: osdmap e93: 12 total, 12 up, 12 in 2024-06-10T02:50:13.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:13.813+0000 7fef78273700 1 -- 172.21.15.52:0/2809063050 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fef70102030 msgr2=0x7fef70105ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:50:13.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:13.813+0000 7fef78273700 1 --2- 172.21.15.52:0/2809063050 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fef70102030 0x7fef70105ff0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fef600099c0 tx=0x7fef60009cd0 comp rx=0 tx=0).stop 2024-06-10T02:50:13.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:13.813+0000 7fef78273700 1 -- 172.21.15.52:0/2809063050 shutdown_connections 2024-06-10T02:50:13.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:13.813+0000 7fef78273700 1 --2- 172.21.15.52:0/2809063050 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fef701066d0 0x7fef7010e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:50:13.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:13.813+0000 7fef78273700 1 --2- 172.21.15.52:0/2809063050 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fef70101680 0x7fef70101a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:50:13.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:13.813+0000 7fef78273700 1 --2- 172.21.15.52:0/2809063050 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fef70102030 0x7fef70105ff0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:50:13.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:13.813+0000 7fef78273700 1 -- 172.21.15.52:0/2809063050 >> 172.21.15.52:0/2809063050 conn(0x7fef700773e0 msgr2=0x7fef700777f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:50:13.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:13.813+0000 7fef78273700 1 -- 172.21.15.52:0/2809063050 shutdown_connections 2024-06-10T02:50:13.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:13.813+0000 7fef78273700 1 -- 172.21.15.52:0/2809063050 wait complete. 2024-06-10T02:50:13.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:13.814+0000 7fef78273700 1 Processor -- start 2024-06-10T02:50:13.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:13.814+0000 7fef78273700 1 -- start start 2024-06-10T02:50:13.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:13.814+0000 7fef78273700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fef70101680 0x7fef7019e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:50:13.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:13.814+0000 7fef78273700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fef70102030 0x7fef7019ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:50:13.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:13.814+0000 7fef78273700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fef701066d0 0x7fef70198ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:50:13.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:13.814+0000 7fef78273700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fef70111710 con 0x7fef70102030 2024-06-10T02:50:13.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:13.814+0000 7fef78273700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fef70111590 con 0x7fef70101680 2024-06-10T02:50:13.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:13.814+0000 7fef78273700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fef70111890 con 0x7fef701066d0 2024-06-10T02:50:13.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:13.814+0000 7fef7580e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fef70102030 0x7fef7019ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:50:13.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:13.814+0000 7fef7580e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fef70102030 0x7fef7019ee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:45654/0 (socket says 172.21.15.52:45654) 2024-06-10T02:50:13.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:13.814+0000 7fef7580e700 1 -- 172.21.15.52:0/3590969502 learned_addr learned my addr 172.21.15.52:0/3590969502 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:50:13.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:13.814+0000 7fef7600f700 1 --2- 172.21.15.52:0/3590969502 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fef70101680 0x7fef7019e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:50:13.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:13.814+0000 7fef7580e700 1 -- 172.21.15.52:0/3590969502 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fef701066d0 msgr2=0x7fef70198ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:50:13.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:13.814+0000 7fef7580e700 1 --2- 172.21.15.52:0/3590969502 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fef701066d0 0x7fef70198ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:50:13.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:13.814+0000 7fef7580e700 1 -- 172.21.15.52:0/3590969502 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fef70101680 msgr2=0x7fef7019e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:50:13.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:13.814+0000 7fef7580e700 1 --2- 172.21.15.52:0/3590969502 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fef70101680 0x7fef7019e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:50:13.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:13.814+0000 7fef7580e700 1 -- 172.21.15.52:0/3590969502 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef70199240 con 0x7fef70102030 2024-06-10T02:50:13.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:13.815+0000 7fef7580e700 1 --2- 172.21.15.52:0/3590969502 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fef70102030 0x7fef7019ee60 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fef600059d0 tx=0x7fef60005640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:50:13.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:13.815+0000 7fef677fe700 1 -- 172.21.15.52:0/3590969502 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef60027070 con 0x7fef70102030 2024-06-10T02:50:13.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:13.815+0000 7fef677fe700 1 -- 172.21.15.52:0/3590969502 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef600040d0 con 0x7fef70102030 2024-06-10T02:50:13.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:13.815+0000 7fef78273700 1 -- 172.21.15.52:0/3590969502 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef701994d0 con 0x7fef70102030 2024-06-10T02:50:13.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:13.815+0000 7fef78273700 1 -- 172.21.15.52:0/3590969502 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef701a57f0 con 0x7fef70102030 2024-06-10T02:50:13.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:13.815+0000 7fef677fe700 1 -- 172.21.15.52:0/3590969502 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef6000f460 con 0x7fef70102030 2024-06-10T02:50:13.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:13.816+0000 7fef78273700 1 -- 172.21.15.52:0/3590969502 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef70069180 con 0x7fef70102030 2024-06-10T02:50:13.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:13.817+0000 7fef677fe700 1 -- 172.21.15.52:0/3590969502 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fef6002a070 con 0x7fef70102030 2024-06-10T02:50:13.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:13.818+0000 7fef677fe700 1 --2- 172.21.15.52:0/3590969502 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fef5c0a2ca0 0x7fef5c0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:50:13.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:13.818+0000 7fef7600f700 1 --2- 172.21.15.52:0/3590969502 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fef5c0a2ca0 0x7fef5c0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:50:13.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:13.818+0000 7fef677fe700 1 -- 172.21.15.52:0/3590969502 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8770+0+0 (secure 0 0 0) 0x7fef600d0860 con 0x7fef70102030 2024-06-10T02:50:13.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:13.818+0000 7fef7600f700 1 --2- 172.21.15.52:0/3590969502 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fef5c0a2ca0 0x7fef5c0a5160 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fef6c006fd0 tx=0x7fef6c008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:50:13.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:13.822+0000 7fef677fe700 1 -- 172.21.15.52:0/3590969502 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fef60031020 con 0x7fef70102030 2024-06-10T02:50:14.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:14.039+0000 7fef78273700 1 -- 172.21.15.52:0/3590969502 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) -- 0x7fef701a5a30 con 0x7fef70102030 2024-06-10T02:50:14.475 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:50:14 smithi052 ceph-mon[32362]: pgmap v308: 129 pgs: 1 creating+activating, 10 creating+peering, 14 unknown, 104 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 177 B/s wr, 4 op/s 2024-06-10T02:50:14.476 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:50:14 smithi052 ceph-mon[32362]: osdmap e94: 12 total, 12 up, 12 in 2024-06-10T02:50:14.476 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:50:14 smithi052 ceph-mon[32362]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-10T02:50:14.476 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:50:14 smithi052 ceph-mon[32362]: from='client.? 172.21.15.52:0/3590969502' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-06-10T02:50:14.637 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:50:14 smithi120 ceph-mon[36634]: pgmap v308: 129 pgs: 1 creating+activating, 10 creating+peering, 14 unknown, 104 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 177 B/s wr, 4 op/s 2024-06-10T02:50:14.637 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:50:14 smithi120 ceph-mon[36634]: osdmap e94: 12 total, 12 up, 12 in 2024-06-10T02:50:14.637 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:50:14 smithi120 ceph-mon[36634]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-10T02:50:14.637 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:50:14 smithi120 ceph-mon[36634]: from='client.? 172.21.15.52:0/3590969502' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-06-10T02:50:14.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:14.669+0000 7fef677fe700 1 -- 172.21.15.52:0/3590969502 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v95) ==== 169+0+0 (secure 0 0 0) 0x7fef60013070 con 0x7fef70102030 2024-06-10T02:50:14.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:14.673+0000 7fef78273700 1 -- 172.21.15.52:0/3590969502 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fef5c0a2ca0 msgr2=0x7fef5c0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:50:14.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:14.673+0000 7fef78273700 1 --2- 172.21.15.52:0/3590969502 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fef5c0a2ca0 0x7fef5c0a5160 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fef6c006fd0 tx=0x7fef6c008040 comp rx=0 tx=0).stop 2024-06-10T02:50:14.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:14.673+0000 7fef78273700 1 -- 172.21.15.52:0/3590969502 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fef70102030 msgr2=0x7fef7019ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:50:14.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:14.673+0000 7fef78273700 1 --2- 172.21.15.52:0/3590969502 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fef70102030 0x7fef7019ee60 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fef600059d0 tx=0x7fef60005640 comp rx=0 tx=0).stop 2024-06-10T02:50:14.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:14.673+0000 7fef78273700 1 -- 172.21.15.52:0/3590969502 shutdown_connections 2024-06-10T02:50:14.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:14.673+0000 7fef78273700 1 --2- 172.21.15.52:0/3590969502 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fef701066d0 0x7fef70198ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:50:14.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:14.673+0000 7fef78273700 1 --2- 172.21.15.52:0/3590969502 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fef70101680 0x7fef7019e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:50:14.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:14.673+0000 7fef78273700 1 --2- 172.21.15.52:0/3590969502 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fef5c0a2ca0 0x7fef5c0a5160 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:50:14.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:14.673+0000 7fef78273700 1 --2- 172.21.15.52:0/3590969502 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fef70102030 0x7fef7019ee60 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:50:14.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:14.673+0000 7fef78273700 1 -- 172.21.15.52:0/3590969502 >> 172.21.15.52:0/3590969502 conn(0x7fef700773e0 msgr2=0x7fef70105230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:50:14.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:14.673+0000 7fef78273700 1 -- 172.21.15.52:0/3590969502 shutdown_connections 2024-06-10T02:50:14.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:14.673+0000 7fef78273700 1 -- 172.21.15.52:0/3590969502 wait complete. 2024-06-10T02:50:14.672 INFO:teuthology.orchestra.run.smithi052.stderr:set pool 4 allow_ec_overwrites to true 2024-06-10T02:50:14.702 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:50:14 smithi188 ceph-mon[36599]: pgmap v308: 129 pgs: 1 creating+activating, 10 creating+peering, 14 unknown, 104 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 177 B/s wr, 4 op/s 2024-06-10T02:50:14.702 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:50:14 smithi188 ceph-mon[36599]: osdmap e94: 12 total, 12 up, 12 in 2024-06-10T02:50:14.702 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:50:14 smithi188 ceph-mon[36599]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-10T02:50:14.702 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:50:14 smithi188 ceph-mon[36599]: from='client.? 172.21.15.52:0/3590969502' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-06-10T02:50:15.456 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph fs add_data_pool cephfs cephfs_data_ec 2024-06-10T02:50:15.717 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:50:15.725 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:50:15 smithi052 ceph-mon[32362]: from='client.? 172.21.15.52:0/3590969502' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-06-10T02:50:15.725 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:50:15 smithi052 ceph-mon[32362]: osdmap e95: 12 total, 12 up, 12 in 2024-06-10T02:50:15.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:50:15 smithi188 ceph-mon[36599]: from='client.? 172.21.15.52:0/3590969502' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-06-10T02:50:15.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:50:15 smithi188 ceph-mon[36599]: osdmap e95: 12 total, 12 up, 12 in 2024-06-10T02:50:16.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:50:15 smithi120 ceph-mon[36634]: from='client.? 172.21.15.52:0/3590969502' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-06-10T02:50:16.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:50:15 smithi120 ceph-mon[36634]: osdmap e95: 12 total, 12 up, 12 in 2024-06-10T02:50:16.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:50:16 smithi052 ceph-mon[32362]: pgmap v311: 129 pgs: 1 creating+activating, 10 creating+peering, 11 unknown, 107 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T02:50:16.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:50:16 smithi188 ceph-mon[36599]: pgmap v311: 129 pgs: 1 creating+activating, 10 creating+peering, 11 unknown, 107 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T02:50:17.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:50:16 smithi120 ceph-mon[36634]: pgmap v311: 129 pgs: 1 creating+activating, 10 creating+peering, 11 unknown, 107 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T02:50:17.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:17.450+0000 7f0d016cc700 1 -- 172.21.15.52:0/4257806502 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0cfc107840 msgr2=0x7f0cfc10f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:50:17.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:17.450+0000 7f0d016cc700 1 --2- 172.21.15.52:0/4257806502 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0cfc107840 0x7f0cfc10f3c0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f0cf400b0d0 tx=0x7f0cf400b3e0 comp rx=0 tx=0).stop 2024-06-10T02:50:17.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:17.450+0000 7f0d016cc700 1 -- 172.21.15.52:0/4257806502 shutdown_connections 2024-06-10T02:50:17.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:17.450+0000 7f0d016cc700 1 --2- 172.21.15.52:0/4257806502 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0cfc107840 0x7f0cfc10f3c0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:50:17.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:17.450+0000 7f0d016cc700 1 --2- 172.21.15.52:0/4257806502 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0cfc102740 0x7f0cfc102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:50:17.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:17.450+0000 7f0d016cc700 1 --2- 172.21.15.52:0/4257806502 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0cfc1030f0 0x7f0cfc107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:50:17.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:17.450+0000 7f0d016cc700 1 -- 172.21.15.52:0/4257806502 >> 172.21.15.52:0/4257806502 conn(0x7f0cfc0795f0 msgr2=0x7f0cfc079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:50:17.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:17.450+0000 7f0d016cc700 1 -- 172.21.15.52:0/4257806502 shutdown_connections 2024-06-10T02:50:17.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:17.450+0000 7f0d016cc700 1 -- 172.21.15.52:0/4257806502 wait complete. 2024-06-10T02:50:17.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:17.450+0000 7f0d016cc700 1 Processor -- start 2024-06-10T02:50:17.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:17.451+0000 7f0d016cc700 1 -- start start 2024-06-10T02:50:17.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:17.451+0000 7f0d016cc700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0cfc102740 0x7f0cfc19e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:50:17.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:17.451+0000 7f0d016cc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0cfc1030f0 0x7f0cfc19ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:50:17.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:17.451+0000 7f0d016cc700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0cfc107840 0x7f0cfc198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:50:17.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:17.451+0000 7f0d016cc700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f0cfc112780 con 0x7f0cfc1030f0 2024-06-10T02:50:17.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:17.451+0000 7f0d016cc700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f0cfc112600 con 0x7f0cfc102740 2024-06-10T02:50:17.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:17.451+0000 7f0d016cc700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f0cfc112900 con 0x7f0cfc107840 2024-06-10T02:50:17.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:17.451+0000 7f0cfa7fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0cfc1030f0 0x7f0cfc19ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:50:17.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:17.451+0000 7f0cfa7fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0cfc1030f0 0x7f0cfc19ecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:45664/0 (socket says 172.21.15.52:45664) 2024-06-10T02:50:17.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:17.451+0000 7f0cfaffd700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0cfc102740 0x7f0cfc19e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:50:17.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:17.451+0000 7f0cfa7fc700 1 -- 172.21.15.52:0/3121877635 learned_addr learned my addr 172.21.15.52:0/3121877635 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:50:17.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:17.451+0000 7f0cfb7fe700 1 --2- 172.21.15.52:0/3121877635 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0cfc107840 0x7f0cfc198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:50:17.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:17.451+0000 7f0cfa7fc700 1 -- 172.21.15.52:0/3121877635 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0cfc107840 msgr2=0x7f0cfc198940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:50:17.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:17.451+0000 7f0cfa7fc700 1 --2- 172.21.15.52:0/3121877635 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0cfc107840 0x7f0cfc198940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:50:17.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:17.451+0000 7f0cfa7fc700 1 -- 172.21.15.52:0/3121877635 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0cfc102740 msgr2=0x7f0cfc19e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:50:17.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:17.451+0000 7f0cfa7fc700 1 --2- 172.21.15.52:0/3121877635 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0cfc102740 0x7f0cfc19e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:50:17.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:17.451+0000 7f0cfa7fc700 1 -- 172.21.15.52:0/3121877635 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cfc1990a0 con 0x7f0cfc1030f0 2024-06-10T02:50:17.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:17.452+0000 7f0cfa7fc700 1 --2- 172.21.15.52:0/3121877635 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0cfc1030f0 0x7f0cfc19ecc0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f0cec00d980 tx=0x7f0cec00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:50:17.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:17.452+0000 7f0cf3fff700 1 -- 172.21.15.52:0/3121877635 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cec017070 con 0x7f0cfc1030f0 2024-06-10T02:50:17.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:17.452+0000 7f0d016cc700 1 -- 172.21.15.52:0/3121877635 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0cfc199390 con 0x7f0cfc1030f0 2024-06-10T02:50:17.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:17.452+0000 7f0cf3fff700 1 -- 172.21.15.52:0/3121877635 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0cec005020 con 0x7f0cfc1030f0 2024-06-10T02:50:17.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:17.452+0000 7f0cf3fff700 1 -- 172.21.15.52:0/3121877635 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cec00fbf0 con 0x7f0cfc1030f0 2024-06-10T02:50:17.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:17.452+0000 7f0d016cc700 1 -- 172.21.15.52:0/3121877635 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0cfc1a5650 con 0x7f0cfc1030f0 2024-06-10T02:50:17.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:17.453+0000 7f0d016cc700 1 -- 172.21.15.52:0/3121877635 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0cfc069180 con 0x7f0cfc1030f0 2024-06-10T02:50:17.454 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:17.454+0000 7f0cf3fff700 1 -- 172.21.15.52:0/3121877635 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f0cec0051d0 con 0x7f0cfc1030f0 2024-06-10T02:50:17.454 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:17.455+0000 7f0cf3fff700 1 --2- 172.21.15.52:0/3121877635 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f0ce80a2c20 0x7f0ce80a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:50:17.454 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:17.455+0000 7f0cfaffd700 1 --2- 172.21.15.52:0/3121877635 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f0ce80a2c20 0x7f0ce80a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:50:17.454 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:17.455+0000 7f0cf3fff700 1 -- 172.21.15.52:0/3121877635 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8770+0+0 (secure 0 0 0) 0x7f0cec0cf470 con 0x7f0cfc1030f0 2024-06-10T02:50:17.454 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:17.455+0000 7f0cfaffd700 1 --2- 172.21.15.52:0/3121877635 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f0ce80a2c20 0x7f0ce80a50e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc199a80 tx=0x7f0ce400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:50:17.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:17.460+0000 7f0cf3fff700 1 -- 172.21.15.52:0/3121877635 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f0cec020020 con 0x7f0cfc1030f0 2024-06-10T02:50:17.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:17.717+0000 7f0d016cc700 1 -- 172.21.15.52:0/3121877635 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) -- 0x7f0cfc1a5820 con 0x7f0cfc1030f0 2024-06-10T02:50:18.685 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:18.686+0000 7f0cf3fff700 1 -- 172.21.15.52:0/3121877635 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v17) ==== 137+0+0 (secure 0 0 0) 0x7f0cec09c1b0 con 0x7f0cfc1030f0 2024-06-10T02:50:18.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:18.689+0000 7f0d016cc700 1 -- 172.21.15.52:0/3121877635 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f0ce80a2c20 msgr2=0x7f0ce80a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:50:18.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:18.689+0000 7f0d016cc700 1 --2- 172.21.15.52:0/3121877635 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f0ce80a2c20 0x7f0ce80a50e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc199a80 tx=0x7f0ce400b410 comp rx=0 tx=0).stop 2024-06-10T02:50:18.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:18.689+0000 7f0d016cc700 1 -- 172.21.15.52:0/3121877635 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0cfc1030f0 msgr2=0x7f0cfc19ecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:50:18.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:18.689+0000 7f0d016cc700 1 --2- 172.21.15.52:0/3121877635 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0cfc1030f0 0x7f0cfc19ecc0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f0cec00d980 tx=0x7f0cec00dc90 comp rx=0 tx=0).stop 2024-06-10T02:50:18.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:18.689+0000 7f0d016cc700 1 -- 172.21.15.52:0/3121877635 shutdown_connections 2024-06-10T02:50:18.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:18.689+0000 7f0d016cc700 1 --2- 172.21.15.52:0/3121877635 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0cfc107840 0x7f0cfc198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:50:18.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:18.689+0000 7f0d016cc700 1 --2- 172.21.15.52:0/3121877635 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0cfc102740 0x7f0cfc19e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:50:18.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:18.689+0000 7f0d016cc700 1 --2- 172.21.15.52:0/3121877635 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f0ce80a2c20 0x7f0ce80a50e0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:50:18.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:18.689+0000 7f0d016cc700 1 --2- 172.21.15.52:0/3121877635 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0cfc1030f0 0x7f0cfc19ecc0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:50:18.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:18.689+0000 7f0d016cc700 1 -- 172.21.15.52:0/3121877635 >> 172.21.15.52:0/3121877635 conn(0x7f0cfc0795f0 msgr2=0x7f0cfc100450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:50:18.689 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:18.689+0000 7f0d016cc700 1 -- 172.21.15.52:0/3121877635 shutdown_connections 2024-06-10T02:50:18.689 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:18.689+0000 7f0d016cc700 1 -- 172.21.15.52:0/3121877635 wait complete. 2024-06-10T02:50:18.689 INFO:teuthology.orchestra.run.smithi052.stderr:added data pool 4 to fsmap 2024-06-10T02:50:18.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:50:18 smithi188 ceph-mon[36599]: pgmap v312: 129 pgs: 1 creating+activating, 2 creating+peering, 126 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T02:50:18.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:50:18 smithi188 ceph-mon[36599]: from='client.? 172.21.15.52:0/3121877635' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-06-10T02:50:18.975 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:50:18 smithi052 ceph-mon[32362]: pgmap v312: 129 pgs: 1 creating+activating, 2 creating+peering, 126 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T02:50:18.975 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:50:18 smithi052 ceph-mon[32362]: from='client.? 172.21.15.52:0/3121877635' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-06-10T02:50:19.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:50:18 smithi120 ceph-mon[36634]: pgmap v312: 129 pgs: 1 creating+activating, 2 creating+peering, 126 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T02:50:19.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:50:18 smithi120 ceph-mon[36634]: from='client.? 172.21.15.52:0/3121877635' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-06-10T02:50:19.484 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T02:50:19.725 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:50:19 smithi052 ceph-mon[32362]: osdmap e96: 12 total, 12 up, 12 in 2024-06-10T02:50:19.725 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:50:19 smithi052 ceph-mon[32362]: from='client.? 172.21.15.52:0/3121877635' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-06-10T02:50:19.725 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:50:19 smithi052 ceph-mon[32362]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-06-10T02:50:19.738 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:50:19.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:50:19 smithi188 ceph-mon[36599]: osdmap e96: 12 total, 12 up, 12 in 2024-06-10T02:50:19.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:50:19 smithi188 ceph-mon[36599]: from='client.? 172.21.15.52:0/3121877635' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-06-10T02:50:19.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:50:19 smithi188 ceph-mon[36599]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-06-10T02:50:20.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:50:19 smithi120 ceph-mon[36634]: osdmap e96: 12 total, 12 up, 12 in 2024-06-10T02:50:20.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:50:19 smithi120 ceph-mon[36634]: from='client.? 172.21.15.52:0/3121877635' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-06-10T02:50:20.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:50:19 smithi120 ceph-mon[36634]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-06-10T02:50:20.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:50:20 smithi188 ceph-mon[36599]: pgmap v314: 129 pgs: 129 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T02:50:20.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:50:20 smithi188 ceph-mon[36599]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-06-10T02:50:20.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:50:20 smithi188 ceph-mon[36599]: Cluster is now healthy 2024-06-10T02:50:20.960 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:50:20 smithi052 ceph-mon[32362]: pgmap v314: 129 pgs: 129 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T02:50:20.961 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:50:20 smithi052 ceph-mon[32362]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-06-10T02:50:20.961 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:50:20 smithi052 ceph-mon[32362]: Cluster is now healthy 2024-06-10T02:50:21.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:50:20 smithi120 ceph-mon[36634]: pgmap v314: 129 pgs: 129 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T02:50:21.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:50:20 smithi120 ceph-mon[36634]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-06-10T02:50:21.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:50:20 smithi120 ceph-mon[36634]: Cluster is now healthy 2024-06-10T02:50:21.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:21.498+0000 7f48de78d700 1 -- 172.21.15.52:0/4167645101 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f48d81030f0 msgr2=0x7f48d8107110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:50:21.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:21.498+0000 7f48de78d700 1 --2- 172.21.15.52:0/4167645101 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f48d81030f0 0x7f48d8107110 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f48c00099c0 tx=0x7f48c0009cd0 comp rx=0 tx=0).stop 2024-06-10T02:50:21.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:21.498+0000 7f48de78d700 1 -- 172.21.15.52:0/4167645101 shutdown_connections 2024-06-10T02:50:21.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:21.498+0000 7f48de78d700 1 --2- 172.21.15.52:0/4167645101 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f48d8107840 0x7f48d810f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:50:21.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:21.498+0000 7f48de78d700 1 --2- 172.21.15.52:0/4167645101 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f48d8102740 0x7f48d8102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:50:21.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:21.498+0000 7f48de78d700 1 --2- 172.21.15.52:0/4167645101 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f48d81030f0 0x7f48d8107110 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:50:21.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:21.498+0000 7f48de78d700 1 -- 172.21.15.52:0/4167645101 >> 172.21.15.52:0/4167645101 conn(0x7f48d80795f0 msgr2=0x7f48d8079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:50:21.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:21.498+0000 7f48de78d700 1 -- 172.21.15.52:0/4167645101 shutdown_connections 2024-06-10T02:50:21.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:21.498+0000 7f48de78d700 1 -- 172.21.15.52:0/4167645101 wait complete. 2024-06-10T02:50:21.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:21.499+0000 7f48de78d700 1 Processor -- start 2024-06-10T02:50:21.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:21.499+0000 7f48de78d700 1 -- start start 2024-06-10T02:50:21.500 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:21.499+0000 7f48de78d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f48d8102740 0x7f48d819e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:50:21.500 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:21.499+0000 7f48de78d700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f48d81030f0 0x7f48d819ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:50:21.500 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:21.500+0000 7f48de78d700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f48d8107840 0x7f48d81989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:50:21.500 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:21.500+0000 7f48de78d700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f48d8112770 con 0x7f48d8102740 2024-06-10T02:50:21.500 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:21.500+0000 7f48de78d700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f48d81125f0 con 0x7f48d8107840 2024-06-10T02:50:21.500 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:21.500+0000 7f48de78d700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f48d81128f0 con 0x7f48d81030f0 2024-06-10T02:50:21.500 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:21.500+0000 7f48d7fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f48d8102740 0x7f48d819e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:50:21.500 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:21.500+0000 7f48d7fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f48d8102740 0x7f48d819e830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37484/0 (socket says 172.21.15.52:37484) 2024-06-10T02:50:21.500 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:21.500+0000 7f48d7fff700 1 -- 172.21.15.52:0/2150870164 learned_addr learned my addr 172.21.15.52:0/2150870164 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:50:21.500 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:21.500+0000 7f48cffff700 1 --2- 172.21.15.52:0/2150870164 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f48d81030f0 0x7f48d819ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:50:21.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:21.500+0000 7f48dcd2a700 1 --2- 172.21.15.52:0/2150870164 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f48d8107840 0x7f48d81989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:50:21.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:21.500+0000 7f48d7fff700 1 -- 172.21.15.52:0/2150870164 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f48d81030f0 msgr2=0x7f48d819ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:50:21.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:21.500+0000 7f48d7fff700 1 --2- 172.21.15.52:0/2150870164 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f48d81030f0 0x7f48d819ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:50:21.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:21.500+0000 7f48d7fff700 1 -- 172.21.15.52:0/2150870164 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f48d8107840 msgr2=0x7f48d81989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:50:21.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:21.500+0000 7f48d7fff700 1 --2- 172.21.15.52:0/2150870164 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f48d8107840 0x7f48d81989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:50:21.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:21.500+0000 7f48d7fff700 1 -- 172.21.15.52:0/2150870164 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48d8199150 con 0x7f48d8102740 2024-06-10T02:50:21.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:21.500+0000 7f48d7fff700 1 --2- 172.21.15.52:0/2150870164 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f48d8102740 0x7f48d819e830 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f48c800e9a0 tx=0x7f48c800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:50:21.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:21.500+0000 7f48d5ffb700 1 -- 172.21.15.52:0/2150870164 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48c800ca10 con 0x7f48d8102740 2024-06-10T02:50:21.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:21.500+0000 7f48d5ffb700 1 -- 172.21.15.52:0/2150870164 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f48c800cbb0 con 0x7f48d8102740 2024-06-10T02:50:21.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:21.500+0000 7f48d5ffb700 1 -- 172.21.15.52:0/2150870164 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48c8018680 con 0x7f48d8102740 2024-06-10T02:50:21.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:21.500+0000 7f48de78d700 1 -- 172.21.15.52:0/2150870164 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48d8199440 con 0x7f48d8102740 2024-06-10T02:50:21.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:21.500+0000 7f48de78d700 1 -- 172.21.15.52:0/2150870164 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48d80746c0 con 0x7f48d8102740 2024-06-10T02:50:21.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:21.501+0000 7f48de78d700 1 -- 172.21.15.52:0/2150870164 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48d8069180 con 0x7f48d8102740 2024-06-10T02:50:21.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:21.503+0000 7f48d5ffb700 1 -- 172.21.15.52:0/2150870164 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f48c8010430 con 0x7f48d8102740 2024-06-10T02:50:21.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:21.503+0000 7f48d5ffb700 1 --2- 172.21.15.52:0/2150870164 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f48c40a2ca0 0x7f48c40a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:50:21.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:21.503+0000 7f48cffff700 1 --2- 172.21.15.52:0/2150870164 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f48c40a2ca0 0x7f48c40a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:50:21.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:21.504+0000 7f48d5ffb700 1 -- 172.21.15.52:0/2150870164 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8802+0+0 (secure 0 0 0) 0x7f48c80cf8f0 con 0x7f48d8102740 2024-06-10T02:50:21.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:21.504+0000 7f48cffff700 1 --2- 172.21.15.52:0/2150870164 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f48c40a2ca0 0x7f48c40a5160 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f48d81070e0 tx=0x7f48c0005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:50:21.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:21.507+0000 7f48d5ffb700 1 -- 172.21.15.52:0/2150870164 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f48c809c610 con 0x7f48d8102740 2024-06-10T02:50:21.758 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:21.759+0000 7f48de78d700 1 -- 172.21.15.52:0/2150870164 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f48d802dca0 con 0x7f48d8102740 2024-06-10T02:50:21.758 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:21.760+0000 7f48d5ffb700 1 -- 172.21.15.52:0/2150870164 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11803 (secure 0 0 0) 0x7f48c809bf10 con 0x7f48d8102740 2024-06-10T02:50:21.759 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-10T02:50:21.759 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/889673247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":889673247},{"type":"v1","addr":"172.21.15.52:6835","nonce":889673247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/3615663511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":3615663511},{"type":"v1","addr":"172.21.15.52:6837","nonce":3615663511}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14571,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6839/3059599042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":3059599042},{"type":"v1","addr":"172.21.15.52:6839","nonce":3059599042}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14583,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6841/3039275521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":3039275521},{"type":"v1","addr":"172.21.15.52:6841","nonce":3039275521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24383,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6833/2609002896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":2609002896},{"type":"v1","addr":"172.21.15.120:6833","nonce":2609002896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24401,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/1996370288","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":1996370288},{"type":"v1","addr":"172.21.15.120:6835","nonce":1996370288}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/529564979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":529564979},{"type":"v1","addr":"172.21.15.188:6833","nonce":529564979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24428,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/4132440952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":4132440952},{"type":"v1","addr":"172.21.15.120:6837","nonce":4132440952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24449,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6839/1672436487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1672436487},{"type":"v1","addr":"172.21.15.120:6839","nonce":1672436487}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":24463,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/3904544259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":3904544259},{"type":"v1","addr":"172.21.15.188:6835","nonce":3904544259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24469,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/819317726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":819317726},{"type":"v1","addr":"172.21.15.188:6837","nonce":819317726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-06-10T02:50:03.889766+0000","modified":"2024-06-10T02:50:17.718706+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24484},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24484":{"gid":24484,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.188:6839/310444836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":310444836},{"type":"v1","addr":"172.21.15.188:6839","nonce":310444836}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24484,"qdb_cluster":[24484]},"id":1}]} 2024-06-10T02:50:21.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:21.763+0000 7f48de78d700 1 -- 172.21.15.52:0/2150870164 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f48c40a2ca0 msgr2=0x7f48c40a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:50:21.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:21.763+0000 7f48de78d700 1 --2- 172.21.15.52:0/2150870164 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f48c40a2ca0 0x7f48c40a5160 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f48d81070e0 tx=0x7f48c0005fb0 comp rx=0 tx=0).stop 2024-06-10T02:50:21.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:21.763+0000 7f48de78d700 1 -- 172.21.15.52:0/2150870164 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f48d8102740 msgr2=0x7f48d819e830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:50:21.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:21.763+0000 7f48de78d700 1 --2- 172.21.15.52:0/2150870164 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f48d8102740 0x7f48d819e830 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f48c800e9a0 tx=0x7f48c800ecb0 comp rx=0 tx=0).stop 2024-06-10T02:50:21.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:21.763+0000 7f48de78d700 1 -- 172.21.15.52:0/2150870164 shutdown_connections 2024-06-10T02:50:21.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:21.763+0000 7f48de78d700 1 --2- 172.21.15.52:0/2150870164 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f48d8107840 0x7f48d81989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:50:21.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:21.763+0000 7f48de78d700 1 --2- 172.21.15.52:0/2150870164 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f48c40a2ca0 0x7f48c40a5160 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:50:21.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:21.763+0000 7f48de78d700 1 --2- 172.21.15.52:0/2150870164 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f48d8102740 0x7f48d819e830 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:50:21.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:21.763+0000 7f48de78d700 1 --2- 172.21.15.52:0/2150870164 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f48d81030f0 0x7f48d819ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:50:21.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:21.763+0000 7f48de78d700 1 -- 172.21.15.52:0/2150870164 >> 172.21.15.52:0/2150870164 conn(0x7f48d80795f0 msgr2=0x7f48d8100510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:50:21.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:21.763+0000 7f48de78d700 1 -- 172.21.15.52:0/2150870164 shutdown_connections 2024-06-10T02:50:21.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:21.763+0000 7f48de78d700 1 -- 172.21.15.52:0/2150870164 wait complete. 2024-06-10T02:50:21.763 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 17 2024-06-10T02:50:22.622 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-10T02:50:22.879 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:50:22.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:50:22 smithi188 ceph-mon[36599]: pgmap v315: 129 pgs: 129 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T02:50:22.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:50:22 smithi188 ceph-mon[36599]: from='client.? 172.21.15.52:0/2150870164' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T02:50:22.975 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:50:22 smithi052 ceph-mon[32362]: pgmap v315: 129 pgs: 129 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T02:50:22.975 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:50:22 smithi052 ceph-mon[32362]: from='client.? 172.21.15.52:0/2150870164' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T02:50:23.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:50:22 smithi120 ceph-mon[36634]: pgmap v315: 129 pgs: 129 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T02:50:23.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:50:22 smithi120 ceph-mon[36634]: from='client.? 172.21.15.52:0/2150870164' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T02:50:24.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:24.516+0000 7fe47fc3d700 1 -- 172.21.15.52:0/2198948493 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe478102740 msgr2=0x7fe478102b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:50:24.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:24.516+0000 7fe47fc3d700 1 --2- 172.21.15.52:0/2198948493 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe478102740 0x7fe478102b20 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fe46800b0d0 tx=0x7fe46800b3e0 comp rx=0 tx=0).stop 2024-06-10T02:50:24.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:24.516+0000 7fe47fc3d700 1 -- 172.21.15.52:0/2198948493 shutdown_connections 2024-06-10T02:50:24.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:24.516+0000 7fe47fc3d700 1 --2- 172.21.15.52:0/2198948493 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe478107840 0x7fe47810f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:50:24.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:24.516+0000 7fe47fc3d700 1 --2- 172.21.15.52:0/2198948493 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe478102740 0x7fe478102b20 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:50:24.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:24.516+0000 7fe47fc3d700 1 --2- 172.21.15.52:0/2198948493 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe4781030f0 0x7fe478107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:50:24.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:24.516+0000 7fe47fc3d700 1 -- 172.21.15.52:0/2198948493 >> 172.21.15.52:0/2198948493 conn(0x7fe4780795f0 msgr2=0x7fe478079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:50:24.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:24.516+0000 7fe47fc3d700 1 -- 172.21.15.52:0/2198948493 shutdown_connections 2024-06-10T02:50:24.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:24.516+0000 7fe47fc3d700 1 -- 172.21.15.52:0/2198948493 wait complete. 2024-06-10T02:50:24.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:24.517+0000 7fe47fc3d700 1 Processor -- start 2024-06-10T02:50:24.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:24.517+0000 7fe47fc3d700 1 -- start start 2024-06-10T02:50:24.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:24.517+0000 7fe47fc3d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe478102740 0x7fe47819e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:50:24.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:24.517+0000 7fe47fc3d700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe4781030f0 0x7fe47819ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:50:24.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:24.517+0000 7fe47fc3d700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe478107840 0x7fe478198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:50:24.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:24.517+0000 7fe47fc3d700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fe4781127e0 con 0x7fe478102740 2024-06-10T02:50:24.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:24.517+0000 7fe47fc3d700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe478112660 con 0x7fe4781030f0 2024-06-10T02:50:24.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:24.517+0000 7fe47fc3d700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fe478112960 con 0x7fe478107840 2024-06-10T02:50:24.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:24.517+0000 7fe47d9d9700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe478102740 0x7fe47819e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:50:24.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:24.518+0000 7fe47d9d9700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe478102740 0x7fe47819e7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37500/0 (socket says 172.21.15.52:37500) 2024-06-10T02:50:24.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:24.518+0000 7fe47d9d9700 1 -- 172.21.15.52:0/2645461967 learned_addr learned my addr 172.21.15.52:0/2645461967 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:50:24.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:24.518+0000 7fe47d1d8700 1 --2- 172.21.15.52:0/2645461967 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe4781030f0 0x7fe47819ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:50:24.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:24.518+0000 7fe47e1da700 1 --2- 172.21.15.52:0/2645461967 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe478107840 0x7fe478198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:50:24.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:24.518+0000 7fe47d9d9700 1 -- 172.21.15.52:0/2645461967 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe478107840 msgr2=0x7fe478198a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:50:24.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:24.518+0000 7fe47d9d9700 1 --2- 172.21.15.52:0/2645461967 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe478107840 0x7fe478198a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:50:24.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:24.518+0000 7fe47d9d9700 1 -- 172.21.15.52:0/2645461967 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe4781030f0 msgr2=0x7fe47819ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:50:24.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:24.518+0000 7fe47d9d9700 1 --2- 172.21.15.52:0/2645461967 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe4781030f0 0x7fe47819ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:50:24.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:24.518+0000 7fe47d9d9700 1 -- 172.21.15.52:0/2645461967 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe478199160 con 0x7fe478102740 2024-06-10T02:50:24.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:24.518+0000 7fe47d9d9700 1 --2- 172.21.15.52:0/2645461967 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe478102740 0x7fe47819e7f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fe46800b820 tx=0x7fe468004ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:50:24.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:24.518+0000 7fe476ffd700 1 -- 172.21.15.52:0/2645461967 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe468021020 con 0x7fe478102740 2024-06-10T02:50:24.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:24.518+0000 7fe47fc3d700 1 -- 172.21.15.52:0/2645461967 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4781993f0 con 0x7fe478102740 2024-06-10T02:50:24.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:24.518+0000 7fe476ffd700 1 -- 172.21.15.52:0/2645461967 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe468007d00 con 0x7fe478102740 2024-06-10T02:50:24.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:24.518+0000 7fe476ffd700 1 -- 172.21.15.52:0/2645461967 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe468005500 con 0x7fe478102740 2024-06-10T02:50:24.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:24.518+0000 7fe47fc3d700 1 -- 172.21.15.52:0/2645461967 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4781a56c0 con 0x7fe478102740 2024-06-10T02:50:24.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:24.519+0000 7fe47fc3d700 1 -- 172.21.15.52:0/2645461967 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe478069180 con 0x7fe478102740 2024-06-10T02:50:24.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:24.521+0000 7fe476ffd700 1 -- 172.21.15.52:0/2645461967 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fe468024020 con 0x7fe478102740 2024-06-10T02:50:24.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:24.521+0000 7fe476ffd700 1 --2- 172.21.15.52:0/2645461967 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fe4600a2c20 0x7fe4600a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:50:24.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:24.521+0000 7fe47d1d8700 1 --2- 172.21.15.52:0/2645461967 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fe4600a2c20 0x7fe4600a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:50:24.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:24.521+0000 7fe476ffd700 1 -- 172.21.15.52:0/2645461967 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8802+0+0 (secure 0 0 0) 0x7fe46800f020 con 0x7fe478102740 2024-06-10T02:50:24.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:24.522+0000 7fe47d1d8700 1 --2- 172.21.15.52:0/2645461967 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fe4600a2c20 0x7fe4600a50e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fe46c005ea0 tx=0x7fe46c005e30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:50:24.523 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:24.525+0000 7fe476ffd700 1 -- 172.21.15.52:0/2645461967 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fe46809d780 con 0x7fe478102740 2024-06-10T02:50:24.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:24.730+0000 7fe47fc3d700 1 -- 172.21.15.52:0/2645461967 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe47802dca0 con 0x7fe478102740 2024-06-10T02:50:24.730 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:24.732+0000 7fe476ffd700 1 -- 172.21.15.52:0/2645461967 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) ==== 74+0+24128 (secure 0 0 0) 0x7fe46809d080 con 0x7fe478102740 2024-06-10T02:50:24.731 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-10T02:50:24.731 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":96,"fsid":"d8676e9e-26d2-11ef-bc9b-c7b262605968","created":"2024-06-10T02:41:43.076837+0000","modified":"2024-06-10T02:50:17.718578+0000","last_up_change":"2024-06-10T02:48:57.297829+0000","last_in_change":"2024-06-10T02:48:36.865802+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-10T02:45:07.394690+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-10T02:49:57.926838+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-10T02:50:01.130393+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-10T02:50:10.655588+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e03c0691-cc5a-497a-bbc2-ce18c337f074","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":4255833408},{"type":"v1","addr":"172.21.15.52:6803","nonce":4255833408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":4255833408},{"type":"v1","addr":"172.21.15.52:6805","nonce":4255833408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":4255833408},{"type":"v1","addr":"172.21.15.52:6809","nonce":4255833408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":4255833408},{"type":"v1","addr":"172.21.15.52:6807","nonce":4255833408}]},"public_addr":"172.21.15.52:6803/4255833408","cluster_addr":"172.21.15.52:6805/4255833408","heartbeat_back_addr":"172.21.15.52:6809/4255833408","heartbeat_front_addr":"172.21.15.52:6807/4255833408","state":["exists","up"]},{"osd":1,"uuid":"a24d8da8-c5ec-44d7-ab30-56f6ce3efda7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":4116540053},{"type":"v1","addr":"172.21.15.120:6801","nonce":4116540053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":4116540053},{"type":"v1","addr":"172.21.15.120:6803","nonce":4116540053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":4116540053},{"type":"v1","addr":"172.21.15.120:6807","nonce":4116540053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":4116540053},{"type":"v1","addr":"172.21.15.120:6805","nonce":4116540053}]},"public_addr":"172.21.15.120:6801/4116540053","cluster_addr":"172.21.15.120:6803/4116540053","heartbeat_back_addr":"172.21.15.120:6807/4116540053","heartbeat_front_addr":"172.21.15.120:6805/4116540053","state":["exists","up"]},{"osd":2,"uuid":"a7162106-ccbf-45bd-bc0e-051ee43f4214","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":2624326999},{"type":"v1","addr":"172.21.15.188:6801","nonce":2624326999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":2624326999},{"type":"v1","addr":"172.21.15.188:6803","nonce":2624326999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":2624326999},{"type":"v1","addr":"172.21.15.188:6807","nonce":2624326999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":2624326999},{"type":"v1","addr":"172.21.15.188:6805","nonce":2624326999}]},"public_addr":"172.21.15.188:6801/2624326999","cluster_addr":"172.21.15.188:6803/2624326999","heartbeat_back_addr":"172.21.15.188:6807/2624326999","heartbeat_front_addr":"172.21.15.188:6805/2624326999","state":["exists","up"]},{"osd":3,"uuid":"53680244-b49a-4120-b24a-9ad7a6341dc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":116897011},{"type":"v1","addr":"172.21.15.52:6811","nonce":116897011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":116897011},{"type":"v1","addr":"172.21.15.52:6813","nonce":116897011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":116897011},{"type":"v1","addr":"172.21.15.52:6817","nonce":116897011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":116897011},{"type":"v1","addr":"172.21.15.52:6815","nonce":116897011}]},"public_addr":"172.21.15.52:6811/116897011","cluster_addr":"172.21.15.52:6813/116897011","heartbeat_back_addr":"172.21.15.52:6817/116897011","heartbeat_front_addr":"172.21.15.52:6815/116897011","state":["exists","up"]},{"osd":4,"uuid":"9f72f1f4-2ef3-49c5-8410-c108767d2ef5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":1320690621},{"type":"v1","addr":"172.21.15.120:6809","nonce":1320690621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":1320690621},{"type":"v1","addr":"172.21.15.120:6811","nonce":1320690621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":1320690621},{"type":"v1","addr":"172.21.15.120:6815","nonce":1320690621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":1320690621},{"type":"v1","addr":"172.21.15.120:6813","nonce":1320690621}]},"public_addr":"172.21.15.120:6809/1320690621","cluster_addr":"172.21.15.120:6811/1320690621","heartbeat_back_addr":"172.21.15.120:6815/1320690621","heartbeat_front_addr":"172.21.15.120:6813/1320690621","state":["exists","up"]},{"osd":5,"uuid":"001b5706-7f27-4fa4-9b69-969e6d765abc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3951947483},{"type":"v1","addr":"172.21.15.188:6809","nonce":3951947483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3951947483},{"type":"v1","addr":"172.21.15.188:6811","nonce":3951947483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3951947483},{"type":"v1","addr":"172.21.15.188:6815","nonce":3951947483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3951947483},{"type":"v1","addr":"172.21.15.188:6813","nonce":3951947483}]},"public_addr":"172.21.15.188:6809/3951947483","cluster_addr":"172.21.15.188:6811/3951947483","heartbeat_back_addr":"172.21.15.188:6815/3951947483","heartbeat_front_addr":"172.21.15.188:6813/3951947483","state":["exists","up"]},{"osd":6,"uuid":"140c903e-9b31-4a81-9ccd-2edc0a2decf9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":359084919},{"type":"v1","addr":"172.21.15.52:6819","nonce":359084919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":359084919},{"type":"v1","addr":"172.21.15.52:6821","nonce":359084919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":359084919},{"type":"v1","addr":"172.21.15.52:6825","nonce":359084919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":359084919},{"type":"v1","addr":"172.21.15.52:6823","nonce":359084919}]},"public_addr":"172.21.15.52:6819/359084919","cluster_addr":"172.21.15.52:6821/359084919","heartbeat_back_addr":"172.21.15.52:6825/359084919","heartbeat_front_addr":"172.21.15.52:6823/359084919","state":["exists","up"]},{"osd":7,"uuid":"57b735d4-5072-4ca2-a3ff-3763ad84f15a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":3737722011},{"type":"v1","addr":"172.21.15.120:6817","nonce":3737722011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6818","nonce":3737722011},{"type":"v1","addr":"172.21.15.120:6819","nonce":3737722011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":3737722011},{"type":"v1","addr":"172.21.15.120:6823","nonce":3737722011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":3737722011},{"type":"v1","addr":"172.21.15.120:6821","nonce":3737722011}]},"public_addr":"172.21.15.120:6817/3737722011","cluster_addr":"172.21.15.120:6819/3737722011","heartbeat_back_addr":"172.21.15.120:6823/3737722011","heartbeat_front_addr":"172.21.15.120:6821/3737722011","state":["exists","up"]},{"osd":8,"uuid":"4ffbb43b-854a-4eac-99cf-7af89bc65409","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1444828441},{"type":"v1","addr":"172.21.15.188:6817","nonce":1444828441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1444828441},{"type":"v1","addr":"172.21.15.188:6819","nonce":1444828441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1444828441},{"type":"v1","addr":"172.21.15.188:6823","nonce":1444828441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1444828441},{"type":"v1","addr":"172.21.15.188:6821","nonce":1444828441}]},"public_addr":"172.21.15.188:6817/1444828441","cluster_addr":"172.21.15.188:6819/1444828441","heartbeat_back_addr":"172.21.15.188:6823/1444828441","heartbeat_front_addr":"172.21.15.188:6821/1444828441","state":["exists","up"]},{"osd":9,"uuid":"5bfd985c-8acc-4d00-8561-40bbd541c07b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":924440658},{"type":"v1","addr":"172.21.15.52:6827","nonce":924440658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":924440658},{"type":"v1","addr":"172.21.15.52:6829","nonce":924440658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":924440658},{"type":"v1","addr":"172.21.15.52:6833","nonce":924440658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":924440658},{"type":"v1","addr":"172.21.15.52:6831","nonce":924440658}]},"public_addr":"172.21.15.52:6827/924440658","cluster_addr":"172.21.15.52:6829/924440658","heartbeat_back_addr":"172.21.15.52:6833/924440658","heartbeat_front_addr":"172.21.15.52:6831/924440658","state":["exists","up"]},{"osd":10,"uuid":"0180cddf-e109-42cc-829a-185604276f02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6824","nonce":1531769083},{"type":"v1","addr":"172.21.15.120:6825","nonce":1531769083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":1531769083},{"type":"v1","addr":"172.21.15.120:6827","nonce":1531769083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":1531769083},{"type":"v1","addr":"172.21.15.120:6831","nonce":1531769083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6828","nonce":1531769083},{"type":"v1","addr":"172.21.15.120:6829","nonce":1531769083}]},"public_addr":"172.21.15.120:6825/1531769083","cluster_addr":"172.21.15.120:6827/1531769083","heartbeat_back_addr":"172.21.15.120:6831/1531769083","heartbeat_front_addr":"172.21.15.120:6829/1531769083","state":["exists","up"]},{"osd":11,"uuid":"73aec750-82d1-4800-b6a0-4e305912f9ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":3769050170},{"type":"v1","addr":"172.21.15.188:6825","nonce":3769050170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":3769050170},{"type":"v1","addr":"172.21.15.188:6827","nonce":3769050170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":3769050170},{"type":"v1","addr":"172.21.15.188:6831","nonce":3769050170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":3769050170},{"type":"v1","addr":"172.21.15.188:6829","nonce":3769050170}]},"public_addr":"172.21.15.188:6825/3769050170","cluster_addr":"172.21.15.188:6827/3769050170","heartbeat_back_addr":"172.21.15.188:6831/3769050170","heartbeat_front_addr":"172.21.15.188:6829/3769050170","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:44:16.455752+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:44:40.219735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:45:03.558899+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:45:29.155004+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:45:53.981098+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:46:19.412020+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:46:45.462640+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:47:11.484578+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:47:36.879054+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:48:03.392446+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:48:29.422173+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:48:55.523423+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.52:0/3535271671":"2024-06-11T02:42:42.872312+0000","172.21.15.52:0/1092934380":"2024-06-11T02:42:42.872312+0000","172.21.15.52:6800/896317057":"2024-06-11T02:42:42.872312+0000","172.21.15.52:6801/3934479642":"2024-06-11T02:42:15.498118+0000","172.21.15.52:6800/3934479642":"2024-06-11T02:42:15.498118+0000","172.21.15.52:0/2759694132":"2024-06-11T02:42:15.498118+0000","172.21.15.52:0/2593665631":"2024-06-11T02:42:15.498118+0000","172.21.15.52:0/3151504634":"2024-06-11T02:42:15.498118+0000","172.21.15.52:6801/896317057":"2024-06-11T02:42:42.872312+0000","172.21.15.52:0/1254480982":"2024-06-11T02:42:42.872312+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-10T02:50:24.732 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:24.734+0000 7fe47fc3d700 1 -- 172.21.15.52:0/2645461967 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fe4600a2c20 msgr2=0x7fe4600a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:50:24.732 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:24.734+0000 7fe47fc3d700 1 --2- 172.21.15.52:0/2645461967 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fe4600a2c20 0x7fe4600a50e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fe46c005ea0 tx=0x7fe46c005e30 comp rx=0 tx=0).stop 2024-06-10T02:50:24.733 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:24.734+0000 7fe47fc3d700 1 -- 172.21.15.52:0/2645461967 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe478102740 msgr2=0x7fe47819e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:50:24.733 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:24.734+0000 7fe47fc3d700 1 --2- 172.21.15.52:0/2645461967 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe478102740 0x7fe47819e7f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fe46800b820 tx=0x7fe468004ed0 comp rx=0 tx=0).stop 2024-06-10T02:50:24.733 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:24.734+0000 7fe47fc3d700 1 -- 172.21.15.52:0/2645461967 shutdown_connections 2024-06-10T02:50:24.733 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:24.734+0000 7fe47fc3d700 1 --2- 172.21.15.52:0/2645461967 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe478107840 0x7fe478198a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:50:24.733 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:24.734+0000 7fe47fc3d700 1 --2- 172.21.15.52:0/2645461967 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fe4600a2c20 0x7fe4600a50e0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:50:24.733 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:24.734+0000 7fe47fc3d700 1 --2- 172.21.15.52:0/2645461967 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe478102740 0x7fe47819e7f0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:50:24.733 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:24.734+0000 7fe47fc3d700 1 --2- 172.21.15.52:0/2645461967 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe4781030f0 0x7fe47819ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:50:24.733 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:24.734+0000 7fe47fc3d700 1 -- 172.21.15.52:0/2645461967 >> 172.21.15.52:0/2645461967 conn(0x7fe4780795f0 msgr2=0x7fe478100530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:50:24.733 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:24.734+0000 7fe47fc3d700 1 -- 172.21.15.52:0/2645461967 shutdown_connections 2024-06-10T02:50:24.733 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:24.734+0000 7fe47fc3d700 1 -- 172.21.15.52:0/2645461967 wait complete. 2024-06-10T02:50:24.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:50:24 smithi188 ceph-mon[36599]: pgmap v316: 129 pgs: 129 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T02:50:24.975 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:50:24 smithi052 ceph-mon[32362]: pgmap v316: 129 pgs: 129 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T02:50:25.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:50:24 smithi120 ceph-mon[36634]: pgmap v316: 129 pgs: 129 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T02:50:25.437 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-10T02:50:25.691 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:50:25.947 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:50:25 smithi188 ceph-mon[36599]: from='client.? 172.21.15.52:0/2645461967' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T02:50:25.975 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:50:25 smithi052 ceph-mon[32362]: from='client.? 172.21.15.52:0/2645461967' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T02:50:26.038 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:50:25 smithi120 ceph-mon[36634]: from='client.? 172.21.15.52:0/2645461967' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T02:50:26.780 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:50:26 smithi052 ceph-mon[32362]: pgmap v317: 129 pgs: 129 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T02:50:26.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:50:26 smithi188 ceph-mon[36599]: pgmap v317: 129 pgs: 129 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T02:50:27.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:50:26 smithi120 ceph-mon[36634]: pgmap v317: 129 pgs: 129 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T02:50:27.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:27.352+0000 7f4ef0104700 1 -- 172.21.15.52:0/698199409 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4ee8100e50 msgr2=0x7f4ee81012d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:50:27.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:27.352+0000 7f4ef0104700 1 --2- 172.21.15.52:0/698199409 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4ee8100e50 0x7f4ee81012d0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f4ed8009990 tx=0x7f4ed8009ca0 comp rx=0 tx=0).stop 2024-06-10T02:50:27.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:27.352+0000 7f4ef0104700 1 -- 172.21.15.52:0/698199409 shutdown_connections 2024-06-10T02:50:27.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:27.352+0000 7f4ef0104700 1 --2- 172.21.15.52:0/698199409 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4ee8101810 0x7f4ee810f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:50:27.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:27.352+0000 7f4ef0104700 1 --2- 172.21.15.52:0/698199409 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4ee8107570 0x7f4ee8107950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:50:27.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:27.352+0000 7f4ef0104700 1 --2- 172.21.15.52:0/698199409 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4ee8100e50 0x7f4ee81012d0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:50:27.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:27.352+0000 7f4ef0104700 1 -- 172.21.15.52:0/698199409 >> 172.21.15.52:0/698199409 conn(0x7f4ee80766b0 msgr2=0x7f4ee8076ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:50:27.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:27.352+0000 7f4ef0104700 1 -- 172.21.15.52:0/698199409 shutdown_connections 2024-06-10T02:50:27.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:27.352+0000 7f4ef0104700 1 -- 172.21.15.52:0/698199409 wait complete. 2024-06-10T02:50:27.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:27.353+0000 7f4ef0104700 1 Processor -- start 2024-06-10T02:50:27.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:27.353+0000 7f4ef0104700 1 -- start start 2024-06-10T02:50:27.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:27.353+0000 7f4ef0104700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4ee8100e50 0x7f4ee8102030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:50:27.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:27.353+0000 7f4ef0104700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4ee8101810 0x7f4ee8102570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:50:27.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:27.353+0000 7f4ef0104700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4ee8107570 0x7f4ee8102ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:50:27.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:27.353+0000 7f4ef0104700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f4ee8112850 con 0x7f4ee8100e50 2024-06-10T02:50:27.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:27.353+0000 7f4ef0104700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f4ee81126d0 con 0x7f4ee8101810 2024-06-10T02:50:27.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:27.353+0000 7f4ef0104700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f4ee81129d0 con 0x7f4ee8107570 2024-06-10T02:50:27.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:27.353+0000 7f4eedea0700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4ee8100e50 0x7f4ee8102030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:50:27.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:27.353+0000 7f4eedea0700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4ee8100e50 0x7f4ee8102030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37510/0 (socket says 172.21.15.52:37510) 2024-06-10T02:50:27.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:27.353+0000 7f4eedea0700 1 -- 172.21.15.52:0/1750955676 learned_addr learned my addr 172.21.15.52:0/1750955676 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:50:27.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:27.353+0000 7f4eed69f700 1 --2- 172.21.15.52:0/1750955676 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4ee8101810 0x7f4ee8102570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:50:27.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:27.353+0000 7f4eee6a1700 1 --2- 172.21.15.52:0/1750955676 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4ee8107570 0x7f4ee8102ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:50:27.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:27.353+0000 7f4eedea0700 1 -- 172.21.15.52:0/1750955676 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4ee8107570 msgr2=0x7f4ee8102ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:50:27.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:27.353+0000 7f4eedea0700 1 --2- 172.21.15.52:0/1750955676 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4ee8107570 0x7f4ee8102ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:50:27.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:27.353+0000 7f4eedea0700 1 -- 172.21.15.52:0/1750955676 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4ee8101810 msgr2=0x7f4ee8102570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:50:27.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:27.353+0000 7f4eedea0700 1 --2- 172.21.15.52:0/1750955676 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4ee8101810 0x7f4ee8102570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:50:27.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:27.353+0000 7f4eedea0700 1 -- 172.21.15.52:0/1750955676 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ee81a71c0 con 0x7f4ee8100e50 2024-06-10T02:50:27.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:27.353+0000 7f4eee6a1700 1 --2- 172.21.15.52:0/1750955676 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4ee8107570 0x7f4ee8102ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T02:50:27.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:27.354+0000 7f4eedea0700 1 --2- 172.21.15.52:0/1750955676 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4ee8100e50 0x7f4ee8102030 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f4ee400b500 tx=0x7f4ee400b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:50:27.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:27.354+0000 7f4edeffd700 1 -- 172.21.15.52:0/1750955676 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ee4015020 con 0x7f4ee8100e50 2024-06-10T02:50:27.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:27.354+0000 7f4ef0104700 1 -- 172.21.15.52:0/1750955676 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ee81a74b0 con 0x7f4ee8100e50 2024-06-10T02:50:27.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:27.354+0000 7f4edeffd700 1 -- 172.21.15.52:0/1750955676 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ee4011d10 con 0x7f4ee8100e50 2024-06-10T02:50:27.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:27.354+0000 7f4edeffd700 1 -- 172.21.15.52:0/1750955676 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ee4010e50 con 0x7f4ee8100e50 2024-06-10T02:50:27.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:27.354+0000 7f4ef0104700 1 -- 172.21.15.52:0/1750955676 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ee81a7a10 con 0x7f4ee8100e50 2024-06-10T02:50:27.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:27.355+0000 7f4ef0104700 1 -- 172.21.15.52:0/1750955676 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ee8069180 con 0x7f4ee8100e50 2024-06-10T02:50:27.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:27.356+0000 7f4edeffd700 1 -- 172.21.15.52:0/1750955676 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f4ee4018020 con 0x7f4ee8100e50 2024-06-10T02:50:27.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:27.357+0000 7f4edeffd700 1 --2- 172.21.15.52:0/1750955676 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f4ed40a2e40 0x7f4ed40a5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:50:27.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:27.357+0000 7f4eed69f700 1 --2- 172.21.15.52:0/1750955676 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f4ed40a2e40 0x7f4ed40a5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:50:27.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:27.357+0000 7f4edeffd700 1 -- 172.21.15.52:0/1750955676 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8802+0+0 (secure 0 0 0) 0x7f4ee40cefa0 con 0x7f4ee8100e50 2024-06-10T02:50:27.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:27.357+0000 7f4eed69f700 1 --2- 172.21.15.52:0/1750955676 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f4ed40a2e40 0x7f4ed40a5300 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f4ed800b5c0 tx=0x7f4ed8005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:50:27.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:27.360+0000 7f4edeffd700 1 -- 172.21.15.52:0/1750955676 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f4ee401f020 con 0x7f4ee8100e50 2024-06-10T02:50:27.566 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:27.567+0000 7f4ef0104700 1 -- 172.21.15.52:0/1750955676 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4ee8103510 con 0x7f4ee8100e50 2024-06-10T02:50:27.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:27.569+0000 7f4edeffd700 1 -- 172.21.15.52:0/1750955676 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) ==== 74+0+24128 (secure 0 0 0) 0x7f4ee409bc60 con 0x7f4ee8100e50 2024-06-10T02:50:27.568 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-10T02:50:27.568 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":96,"fsid":"d8676e9e-26d2-11ef-bc9b-c7b262605968","created":"2024-06-10T02:41:43.076837+0000","modified":"2024-06-10T02:50:17.718578+0000","last_up_change":"2024-06-10T02:48:57.297829+0000","last_in_change":"2024-06-10T02:48:36.865802+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-10T02:45:07.394690+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-10T02:49:57.926838+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-10T02:50:01.130393+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-10T02:50:10.655588+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e03c0691-cc5a-497a-bbc2-ce18c337f074","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":4255833408},{"type":"v1","addr":"172.21.15.52:6803","nonce":4255833408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":4255833408},{"type":"v1","addr":"172.21.15.52:6805","nonce":4255833408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":4255833408},{"type":"v1","addr":"172.21.15.52:6809","nonce":4255833408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":4255833408},{"type":"v1","addr":"172.21.15.52:6807","nonce":4255833408}]},"public_addr":"172.21.15.52:6803/4255833408","cluster_addr":"172.21.15.52:6805/4255833408","heartbeat_back_addr":"172.21.15.52:6809/4255833408","heartbeat_front_addr":"172.21.15.52:6807/4255833408","state":["exists","up"]},{"osd":1,"uuid":"a24d8da8-c5ec-44d7-ab30-56f6ce3efda7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":4116540053},{"type":"v1","addr":"172.21.15.120:6801","nonce":4116540053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":4116540053},{"type":"v1","addr":"172.21.15.120:6803","nonce":4116540053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":4116540053},{"type":"v1","addr":"172.21.15.120:6807","nonce":4116540053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":4116540053},{"type":"v1","addr":"172.21.15.120:6805","nonce":4116540053}]},"public_addr":"172.21.15.120:6801/4116540053","cluster_addr":"172.21.15.120:6803/4116540053","heartbeat_back_addr":"172.21.15.120:6807/4116540053","heartbeat_front_addr":"172.21.15.120:6805/4116540053","state":["exists","up"]},{"osd":2,"uuid":"a7162106-ccbf-45bd-bc0e-051ee43f4214","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":2624326999},{"type":"v1","addr":"172.21.15.188:6801","nonce":2624326999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":2624326999},{"type":"v1","addr":"172.21.15.188:6803","nonce":2624326999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":2624326999},{"type":"v1","addr":"172.21.15.188:6807","nonce":2624326999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":2624326999},{"type":"v1","addr":"172.21.15.188:6805","nonce":2624326999}]},"public_addr":"172.21.15.188:6801/2624326999","cluster_addr":"172.21.15.188:6803/2624326999","heartbeat_back_addr":"172.21.15.188:6807/2624326999","heartbeat_front_addr":"172.21.15.188:6805/2624326999","state":["exists","up"]},{"osd":3,"uuid":"53680244-b49a-4120-b24a-9ad7a6341dc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":116897011},{"type":"v1","addr":"172.21.15.52:6811","nonce":116897011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":116897011},{"type":"v1","addr":"172.21.15.52:6813","nonce":116897011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":116897011},{"type":"v1","addr":"172.21.15.52:6817","nonce":116897011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":116897011},{"type":"v1","addr":"172.21.15.52:6815","nonce":116897011}]},"public_addr":"172.21.15.52:6811/116897011","cluster_addr":"172.21.15.52:6813/116897011","heartbeat_back_addr":"172.21.15.52:6817/116897011","heartbeat_front_addr":"172.21.15.52:6815/116897011","state":["exists","up"]},{"osd":4,"uuid":"9f72f1f4-2ef3-49c5-8410-c108767d2ef5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":1320690621},{"type":"v1","addr":"172.21.15.120:6809","nonce":1320690621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":1320690621},{"type":"v1","addr":"172.21.15.120:6811","nonce":1320690621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":1320690621},{"type":"v1","addr":"172.21.15.120:6815","nonce":1320690621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":1320690621},{"type":"v1","addr":"172.21.15.120:6813","nonce":1320690621}]},"public_addr":"172.21.15.120:6809/1320690621","cluster_addr":"172.21.15.120:6811/1320690621","heartbeat_back_addr":"172.21.15.120:6815/1320690621","heartbeat_front_addr":"172.21.15.120:6813/1320690621","state":["exists","up"]},{"osd":5,"uuid":"001b5706-7f27-4fa4-9b69-969e6d765abc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3951947483},{"type":"v1","addr":"172.21.15.188:6809","nonce":3951947483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3951947483},{"type":"v1","addr":"172.21.15.188:6811","nonce":3951947483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3951947483},{"type":"v1","addr":"172.21.15.188:6815","nonce":3951947483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3951947483},{"type":"v1","addr":"172.21.15.188:6813","nonce":3951947483}]},"public_addr":"172.21.15.188:6809/3951947483","cluster_addr":"172.21.15.188:6811/3951947483","heartbeat_back_addr":"172.21.15.188:6815/3951947483","heartbeat_front_addr":"172.21.15.188:6813/3951947483","state":["exists","up"]},{"osd":6,"uuid":"140c903e-9b31-4a81-9ccd-2edc0a2decf9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":359084919},{"type":"v1","addr":"172.21.15.52:6819","nonce":359084919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":359084919},{"type":"v1","addr":"172.21.15.52:6821","nonce":359084919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":359084919},{"type":"v1","addr":"172.21.15.52:6825","nonce":359084919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":359084919},{"type":"v1","addr":"172.21.15.52:6823","nonce":359084919}]},"public_addr":"172.21.15.52:6819/359084919","cluster_addr":"172.21.15.52:6821/359084919","heartbeat_back_addr":"172.21.15.52:6825/359084919","heartbeat_front_addr":"172.21.15.52:6823/359084919","state":["exists","up"]},{"osd":7,"uuid":"57b735d4-5072-4ca2-a3ff-3763ad84f15a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":3737722011},{"type":"v1","addr":"172.21.15.120:6817","nonce":3737722011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6818","nonce":3737722011},{"type":"v1","addr":"172.21.15.120:6819","nonce":3737722011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":3737722011},{"type":"v1","addr":"172.21.15.120:6823","nonce":3737722011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":3737722011},{"type":"v1","addr":"172.21.15.120:6821","nonce":3737722011}]},"public_addr":"172.21.15.120:6817/3737722011","cluster_addr":"172.21.15.120:6819/3737722011","heartbeat_back_addr":"172.21.15.120:6823/3737722011","heartbeat_front_addr":"172.21.15.120:6821/3737722011","state":["exists","up"]},{"osd":8,"uuid":"4ffbb43b-854a-4eac-99cf-7af89bc65409","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1444828441},{"type":"v1","addr":"172.21.15.188:6817","nonce":1444828441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1444828441},{"type":"v1","addr":"172.21.15.188:6819","nonce":1444828441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1444828441},{"type":"v1","addr":"172.21.15.188:6823","nonce":1444828441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1444828441},{"type":"v1","addr":"172.21.15.188:6821","nonce":1444828441}]},"public_addr":"172.21.15.188:6817/1444828441","cluster_addr":"172.21.15.188:6819/1444828441","heartbeat_back_addr":"172.21.15.188:6823/1444828441","heartbeat_front_addr":"172.21.15.188:6821/1444828441","state":["exists","up"]},{"osd":9,"uuid":"5bfd985c-8acc-4d00-8561-40bbd541c07b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":924440658},{"type":"v1","addr":"172.21.15.52:6827","nonce":924440658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":924440658},{"type":"v1","addr":"172.21.15.52:6829","nonce":924440658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":924440658},{"type":"v1","addr":"172.21.15.52:6833","nonce":924440658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":924440658},{"type":"v1","addr":"172.21.15.52:6831","nonce":924440658}]},"public_addr":"172.21.15.52:6827/924440658","cluster_addr":"172.21.15.52:6829/924440658","heartbeat_back_addr":"172.21.15.52:6833/924440658","heartbeat_front_addr":"172.21.15.52:6831/924440658","state":["exists","up"]},{"osd":10,"uuid":"0180cddf-e109-42cc-829a-185604276f02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6824","nonce":1531769083},{"type":"v1","addr":"172.21.15.120:6825","nonce":1531769083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":1531769083},{"type":"v1","addr":"172.21.15.120:6827","nonce":1531769083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":1531769083},{"type":"v1","addr":"172.21.15.120:6831","nonce":1531769083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6828","nonce":1531769083},{"type":"v1","addr":"172.21.15.120:6829","nonce":1531769083}]},"public_addr":"172.21.15.120:6825/1531769083","cluster_addr":"172.21.15.120:6827/1531769083","heartbeat_back_addr":"172.21.15.120:6831/1531769083","heartbeat_front_addr":"172.21.15.120:6829/1531769083","state":["exists","up"]},{"osd":11,"uuid":"73aec750-82d1-4800-b6a0-4e305912f9ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":3769050170},{"type":"v1","addr":"172.21.15.188:6825","nonce":3769050170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":3769050170},{"type":"v1","addr":"172.21.15.188:6827","nonce":3769050170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":3769050170},{"type":"v1","addr":"172.21.15.188:6831","nonce":3769050170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":3769050170},{"type":"v1","addr":"172.21.15.188:6829","nonce":3769050170}]},"public_addr":"172.21.15.188:6825/3769050170","cluster_addr":"172.21.15.188:6827/3769050170","heartbeat_back_addr":"172.21.15.188:6831/3769050170","heartbeat_front_addr":"172.21.15.188:6829/3769050170","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:44:16.455752+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:44:40.219735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:45:03.558899+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:45:29.155004+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:45:53.981098+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:46:19.412020+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:46:45.462640+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:47:11.484578+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:47:36.879054+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:48:03.392446+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:48:29.422173+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:48:55.523423+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.52:0/3535271671":"2024-06-11T02:42:42.872312+0000","172.21.15.52:0/1092934380":"2024-06-11T02:42:42.872312+0000","172.21.15.52:6800/896317057":"2024-06-11T02:42:42.872312+0000","172.21.15.52:6801/3934479642":"2024-06-11T02:42:15.498118+0000","172.21.15.52:6800/3934479642":"2024-06-11T02:42:15.498118+0000","172.21.15.52:0/2759694132":"2024-06-11T02:42:15.498118+0000","172.21.15.52:0/2593665631":"2024-06-11T02:42:15.498118+0000","172.21.15.52:0/3151504634":"2024-06-11T02:42:15.498118+0000","172.21.15.52:6801/896317057":"2024-06-11T02:42:42.872312+0000","172.21.15.52:0/1254480982":"2024-06-11T02:42:42.872312+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-10T02:50:27.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:27.571+0000 7f4ef0104700 1 -- 172.21.15.52:0/1750955676 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f4ed40a2e40 msgr2=0x7f4ed40a5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:50:27.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:27.571+0000 7f4ef0104700 1 --2- 172.21.15.52:0/1750955676 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f4ed40a2e40 0x7f4ed40a5300 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f4ed800b5c0 tx=0x7f4ed8005960 comp rx=0 tx=0).stop 2024-06-10T02:50:27.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:27.571+0000 7f4ef0104700 1 -- 172.21.15.52:0/1750955676 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4ee8100e50 msgr2=0x7f4ee8102030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:50:27.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:27.571+0000 7f4ef0104700 1 --2- 172.21.15.52:0/1750955676 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4ee8100e50 0x7f4ee8102030 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f4ee400b500 tx=0x7f4ee400b810 comp rx=0 tx=0).stop 2024-06-10T02:50:27.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:27.571+0000 7f4ef0104700 1 -- 172.21.15.52:0/1750955676 shutdown_connections 2024-06-10T02:50:27.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:27.571+0000 7f4ef0104700 1 --2- 172.21.15.52:0/1750955676 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4ee8107570 0x7f4ee8102ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:50:27.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:27.571+0000 7f4ef0104700 1 --2- 172.21.15.52:0/1750955676 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f4ed40a2e40 0x7f4ed40a5300 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:50:27.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:27.571+0000 7f4ef0104700 1 --2- 172.21.15.52:0/1750955676 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4ee8100e50 0x7f4ee8102030 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:50:27.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:27.571+0000 7f4ef0104700 1 --2- 172.21.15.52:0/1750955676 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4ee8101810 0x7f4ee8102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:50:27.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:27.571+0000 7f4ef0104700 1 -- 172.21.15.52:0/1750955676 >> 172.21.15.52:0/1750955676 conn(0x7f4ee80766b0 msgr2=0x7f4ee80ff9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:50:27.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:27.571+0000 7f4ef0104700 1 -- 172.21.15.52:0/1750955676 shutdown_connections 2024-06-10T02:50:27.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:27.571+0000 7f4ef0104700 1 -- 172.21.15.52:0/1750955676 wait complete. 2024-06-10T02:50:27.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:50:27 smithi188 ceph-mon[36599]: from='client.? 172.21.15.52:0/1750955676' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T02:50:27.967 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:50:27 smithi052 ceph-mon[32362]: from='client.? 172.21.15.52:0/1750955676' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T02:50:28.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:50:27 smithi120 ceph-mon[36634]: from='client.? 172.21.15.52:0/1750955676' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T02:50:28.461 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T02:50:28.723 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:50:28.947 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:50:28 smithi052 ceph-mon[32362]: pgmap v318: 129 pgs: 129 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T02:50:28.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:50:28 smithi188 ceph-mon[36599]: pgmap v318: 129 pgs: 129 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T02:50:29.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:50:28 smithi120 ceph-mon[36634]: pgmap v318: 129 pgs: 129 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T02:50:30.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:30.426+0000 7f0f1b7af700 1 -- 172.21.15.52:0/2068687927 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0f14101d30 msgr2=0x7f0f1410e210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:50:30.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:30.426+0000 7f0f1b7af700 1 --2- 172.21.15.52:0/2068687927 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0f14101d30 0x7f0f1410e210 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f0f0c00b0a0 tx=0x7f0f0c00b3b0 comp rx=0 tx=0).stop 2024-06-10T02:50:30.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:30.426+0000 7f0f1b7af700 1 -- 172.21.15.52:0/2068687927 shutdown_connections 2024-06-10T02:50:30.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:30.426+0000 7f0f1b7af700 1 --2- 172.21.15.52:0/2068687927 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0f14101d30 0x7f0f1410e210 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:50:30.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:30.426+0000 7f0f1b7af700 1 --2- 172.21.15.52:0/2068687927 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0f141073d0 0x7f0f141077b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:50:30.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:30.426+0000 7f0f1b7af700 1 --2- 172.21.15.52:0/2068687927 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0f14101390 0x7f0f141017f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:50:30.428 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:30.426+0000 7f0f1b7af700 1 -- 172.21.15.52:0/2068687927 >> 172.21.15.52:0/2068687927 conn(0x7f0f140782c0 msgr2=0x7f0f140786d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:50:30.428 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:30.426+0000 7f0f1b7af700 1 -- 172.21.15.52:0/2068687927 shutdown_connections 2024-06-10T02:50:30.428 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:30.426+0000 7f0f1b7af700 1 -- 172.21.15.52:0/2068687927 wait complete. 2024-06-10T02:50:30.428 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:30.427+0000 7f0f1b7af700 1 Processor -- start 2024-06-10T02:50:30.428 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:30.427+0000 7f0f1b7af700 1 -- start start 2024-06-10T02:50:30.428 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:30.427+0000 7f0f1b7af700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0f14101390 0x7f0f1419ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:50:30.428 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:30.427+0000 7f0f1b7af700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0f14101d30 0x7f0f1419b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:50:30.428 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:30.427+0000 7f0f1b7af700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0f141073d0 0x7f0f1419f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:50:30.428 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:30.427+0000 7f0f1b7af700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f0f141116d0 con 0x7f0f14101390 2024-06-10T02:50:30.428 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:30.427+0000 7f0f1b7af700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f0f14111550 con 0x7f0f14101d30 2024-06-10T02:50:30.428 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:30.427+0000 7f0f1b7af700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f0f14111850 con 0x7f0f141073d0 2024-06-10T02:50:30.428 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:30.427+0000 7f0f1954b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0f14101390 0x7f0f1419ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:50:30.428 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:30.428+0000 7f0f1954b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0f14101390 0x7f0f1419ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37528/0 (socket says 172.21.15.52:37528) 2024-06-10T02:50:30.428 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:30.428+0000 7f0f1954b700 1 -- 172.21.15.52:0/1769062171 learned_addr learned my addr 172.21.15.52:0/1769062171 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:50:30.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:30.428+0000 7f0f18d4a700 1 --2- 172.21.15.52:0/1769062171 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0f14101d30 0x7f0f1419b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:50:30.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:30.428+0000 7f0f19d4c700 1 --2- 172.21.15.52:0/1769062171 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0f141073d0 0x7f0f1419f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:50:30.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:30.428+0000 7f0f1954b700 1 -- 172.21.15.52:0/1769062171 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0f141073d0 msgr2=0x7f0f1419f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:50:30.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:30.428+0000 7f0f1954b700 1 --2- 172.21.15.52:0/1769062171 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0f141073d0 0x7f0f1419f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:50:30.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:30.428+0000 7f0f1954b700 1 -- 172.21.15.52:0/1769062171 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0f14101d30 msgr2=0x7f0f1419b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:50:30.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:30.428+0000 7f0f1954b700 1 --2- 172.21.15.52:0/1769062171 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0f14101d30 0x7f0f1419b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:50:30.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:30.428+0000 7f0f1954b700 1 -- 172.21.15.52:0/1769062171 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f1419fe20 con 0x7f0f14101390 2024-06-10T02:50:30.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:30.428+0000 7f0f18d4a700 1 --2- 172.21.15.52:0/1769062171 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0f14101d30 0x7f0f1419b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T02:50:30.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:30.428+0000 7f0f1954b700 1 --2- 172.21.15.52:0/1769062171 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0f14101390 0x7f0f1419ae10 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f0f1000d950 tx=0x7f0f1000dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:50:30.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:30.428+0000 7f0f0a7fc700 1 -- 172.21.15.52:0/1769062171 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f10016070 con 0x7f0f14101390 2024-06-10T02:50:30.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:30.428+0000 7f0f0a7fc700 1 -- 172.21.15.52:0/1769062171 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f10009b40 con 0x7f0f14101390 2024-06-10T02:50:30.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:30.428+0000 7f0f1b7af700 1 -- 172.21.15.52:0/1769062171 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f141a0110 con 0x7f0f14101390 2024-06-10T02:50:30.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:30.428+0000 7f0f0a7fc700 1 -- 172.21.15.52:0/1769062171 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f1000f620 con 0x7f0f14101390 2024-06-10T02:50:30.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:30.428+0000 7f0f1b7af700 1 -- 172.21.15.52:0/1769062171 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f141a7990 con 0x7f0f14101390 2024-06-10T02:50:30.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:30.429+0000 7f0f1b7af700 1 -- 172.21.15.52:0/1769062171 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f14069180 con 0x7f0f14101390 2024-06-10T02:50:30.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:30.431+0000 7f0f0a7fc700 1 -- 172.21.15.52:0/1769062171 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f0f10019070 con 0x7f0f14101390 2024-06-10T02:50:30.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:30.432+0000 7f0f0a7fc700 1 --2- 172.21.15.52:0/1769062171 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f0f000a2ca0 0x7f0f000a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:50:30.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:30.432+0000 7f0f18d4a700 1 --2- 172.21.15.52:0/1769062171 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f0f000a2ca0 0x7f0f000a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:50:30.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:30.432+0000 7f0f0a7fc700 1 -- 172.21.15.52:0/1769062171 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8802+0+0 (secure 0 0 0) 0x7f0f100ceff0 con 0x7f0f14101390 2024-06-10T02:50:30.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:30.433+0000 7f0f18d4a700 1 --2- 172.21.15.52:0/1769062171 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f0f000a2ca0 0x7f0f000a5160 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f0f141012d0 tx=0x7f0f04008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:50:30.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:30.436+0000 7f0f0a7fc700 1 -- 172.21.15.52:0/1769062171 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f0f1009bd10 con 0x7f0f14101390 2024-06-10T02:50:30.684 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:30.685+0000 7f0f1b7af700 1 -- 172.21.15.52:0/1769062171 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0f1419c170 con 0x7f0f14101390 2024-06-10T02:50:30.685 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:30.686+0000 7f0f0a7fc700 1 -- 172.21.15.52:0/1769062171 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11803 (secure 0 0 0) 0x7f0f1419c170 con 0x7f0f14101390 2024-06-10T02:50:30.685 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-10T02:50:30.685 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/889673247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":889673247},{"type":"v1","addr":"172.21.15.52:6835","nonce":889673247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/3615663511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":3615663511},{"type":"v1","addr":"172.21.15.52:6837","nonce":3615663511}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14571,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6839/3059599042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":3059599042},{"type":"v1","addr":"172.21.15.52:6839","nonce":3059599042}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14583,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6841/3039275521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":3039275521},{"type":"v1","addr":"172.21.15.52:6841","nonce":3039275521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24383,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6833/2609002896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":2609002896},{"type":"v1","addr":"172.21.15.120:6833","nonce":2609002896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24401,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/1996370288","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":1996370288},{"type":"v1","addr":"172.21.15.120:6835","nonce":1996370288}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/529564979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":529564979},{"type":"v1","addr":"172.21.15.188:6833","nonce":529564979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24428,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/4132440952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":4132440952},{"type":"v1","addr":"172.21.15.120:6837","nonce":4132440952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24449,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6839/1672436487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1672436487},{"type":"v1","addr":"172.21.15.120:6839","nonce":1672436487}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":24463,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/3904544259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":3904544259},{"type":"v1","addr":"172.21.15.188:6835","nonce":3904544259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24469,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/819317726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":819317726},{"type":"v1","addr":"172.21.15.188:6837","nonce":819317726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-06-10T02:50:03.889766+0000","modified":"2024-06-10T02:50:17.718706+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24484},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24484":{"gid":24484,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.188:6839/310444836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":310444836},{"type":"v1","addr":"172.21.15.188:6839","nonce":310444836}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24484,"qdb_cluster":[24484]},"id":1}]} 2024-06-10T02:50:30.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:30.688+0000 7f0f1b7af700 1 -- 172.21.15.52:0/1769062171 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f0f000a2ca0 msgr2=0x7f0f000a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:50:30.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:30.688+0000 7f0f1b7af700 1 --2- 172.21.15.52:0/1769062171 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f0f000a2ca0 0x7f0f000a5160 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f0f141012d0 tx=0x7f0f04008040 comp rx=0 tx=0).stop 2024-06-10T02:50:30.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:30.688+0000 7f0f1b7af700 1 -- 172.21.15.52:0/1769062171 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0f14101390 msgr2=0x7f0f1419ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:50:30.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:30.688+0000 7f0f1b7af700 1 --2- 172.21.15.52:0/1769062171 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0f14101390 0x7f0f1419ae10 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f0f1000d950 tx=0x7f0f1000dd10 comp rx=0 tx=0).stop 2024-06-10T02:50:30.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:30.688+0000 7f0f1b7af700 1 -- 172.21.15.52:0/1769062171 shutdown_connections 2024-06-10T02:50:30.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:30.688+0000 7f0f1b7af700 1 --2- 172.21.15.52:0/1769062171 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0f141073d0 0x7f0f1419f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:50:30.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:30.688+0000 7f0f1b7af700 1 --2- 172.21.15.52:0/1769062171 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f0f000a2ca0 0x7f0f000a5160 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:50:30.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:30.688+0000 7f0f1b7af700 1 --2- 172.21.15.52:0/1769062171 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0f14101390 0x7f0f1419ae10 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:50:30.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:30.688+0000 7f0f1b7af700 1 --2- 172.21.15.52:0/1769062171 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0f14101d30 0x7f0f1419b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:50:30.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:30.688+0000 7f0f1b7af700 1 -- 172.21.15.52:0/1769062171 >> 172.21.15.52:0/1769062171 conn(0x7f0f140782c0 msgr2=0x7f0f1406a190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:50:30.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:30.688+0000 7f0f1b7af700 1 -- 172.21.15.52:0/1769062171 shutdown_connections 2024-06-10T02:50:30.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:30.688+0000 7f0f1b7af700 1 -- 172.21.15.52:0/1769062171 wait complete. 2024-06-10T02:50:30.688 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 17 2024-06-10T02:50:30.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:50:30 smithi188 ceph-mon[36599]: pgmap v319: 129 pgs: 129 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T02:50:30.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:50:30 smithi188 ceph-mon[36599]: from='client.? 172.21.15.52:0/1769062171' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T02:50:30.975 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:50:30 smithi052 ceph-mon[32362]: pgmap v319: 129 pgs: 129 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T02:50:30.975 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:50:30 smithi052 ceph-mon[32362]: from='client.? 172.21.15.52:0/1769062171' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T02:50:31.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:50:30 smithi120 ceph-mon[36634]: pgmap v319: 129 pgs: 129 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T02:50:31.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:50:30 smithi120 ceph-mon[36634]: from='client.? 172.21.15.52:0/1769062171' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T02:50:31.431 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-10T02:50:31.686 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:50:32.817 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:50:32 smithi052 ceph-mon[32362]: pgmap v320: 129 pgs: 129 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T02:50:33.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:50:32 smithi120 ceph-mon[36634]: pgmap v320: 129 pgs: 129 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T02:50:33.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:50:32 smithi188 ceph-mon[36599]: pgmap v320: 129 pgs: 129 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T02:50:33.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:33.328+0000 7fca973de700 1 -- 172.21.15.52:0/2918385270 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fca901032e0 msgr2=0x7fca901036c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:50:33.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:33.328+0000 7fca973de700 1 --2- 172.21.15.52:0/2918385270 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fca901032e0 0x7fca901036c0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fca8c009990 tx=0x7fca8c009ca0 comp rx=0 tx=0).stop 2024-06-10T02:50:33.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:33.329+0000 7fca973de700 1 -- 172.21.15.52:0/2918385270 shutdown_connections 2024-06-10T02:50:33.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:33.329+0000 7fca973de700 1 --2- 172.21.15.52:0/2918385270 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fca90101d80 0x7fca9010e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:50:33.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:33.329+0000 7fca973de700 1 --2- 172.21.15.52:0/2918385270 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fca901032e0 0x7fca901036c0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:50:33.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:33.329+0000 7fca973de700 1 --2- 172.21.15.52:0/2918385270 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fca901013c0 0x7fca90101840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:50:33.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:33.329+0000 7fca973de700 1 -- 172.21.15.52:0/2918385270 >> 172.21.15.52:0/2918385270 conn(0x7fca90077400 msgr2=0x7fca90077810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:50:33.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:33.329+0000 7fca973de700 1 -- 172.21.15.52:0/2918385270 shutdown_connections 2024-06-10T02:50:33.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:33.329+0000 7fca973de700 1 -- 172.21.15.52:0/2918385270 wait complete. 2024-06-10T02:50:33.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:33.329+0000 7fca973de700 1 Processor -- start 2024-06-10T02:50:33.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:33.329+0000 7fca973de700 1 -- start start 2024-06-10T02:50:33.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:33.330+0000 7fca973de700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fca901013c0 0x7fca90104180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:50:33.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:33.330+0000 7fca973de700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fca90101d80 0x7fca901046c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:50:33.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:33.330+0000 7fca973de700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fca901032e0 0x7fca90104c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:50:33.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:33.330+0000 7fca973de700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fca901117f0 con 0x7fca901032e0 2024-06-10T02:50:33.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:33.330+0000 7fca973de700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fca90111670 con 0x7fca90101d80 2024-06-10T02:50:33.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:33.330+0000 7fca973de700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fca90111970 con 0x7fca901013c0 2024-06-10T02:50:33.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:33.330+0000 7fca9597b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fca901032e0 0x7fca90104c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:50:33.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:33.330+0000 7fca9597b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fca901032e0 0x7fca90104c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:41114/0 (socket says 172.21.15.52:41114) 2024-06-10T02:50:33.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:33.330+0000 7fca9597b700 1 -- 172.21.15.52:0/2588078046 learned_addr learned my addr 172.21.15.52:0/2588078046 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:50:33.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:33.330+0000 7fca94979700 1 --2- 172.21.15.52:0/2588078046 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fca90101d80 0x7fca901046c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:50:33.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:33.330+0000 7fca9517a700 1 --2- 172.21.15.52:0/2588078046 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fca901013c0 0x7fca90104180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:50:33.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:33.330+0000 7fca9597b700 1 -- 172.21.15.52:0/2588078046 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fca901013c0 msgr2=0x7fca90104180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:50:33.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:33.330+0000 7fca9597b700 1 --2- 172.21.15.52:0/2588078046 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fca901013c0 0x7fca90104180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:50:33.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:33.330+0000 7fca9597b700 1 -- 172.21.15.52:0/2588078046 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fca90101d80 msgr2=0x7fca901046c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:50:33.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:33.330+0000 7fca9597b700 1 --2- 172.21.15.52:0/2588078046 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fca90101d80 0x7fca901046c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:50:33.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:33.330+0000 7fca9597b700 1 -- 172.21.15.52:0/2588078046 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca901a71f0 con 0x7fca901032e0 2024-06-10T02:50:33.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:33.330+0000 7fca94979700 1 --2- 172.21.15.52:0/2588078046 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fca90101d80 0x7fca901046c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T02:50:33.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:33.330+0000 7fca9597b700 1 --2- 172.21.15.52:0/2588078046 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fca901032e0 0x7fca90104c00 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fca88007d70 tx=0x7fca8800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:50:33.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:33.330+0000 7fca867fc700 1 -- 172.21.15.52:0/2588078046 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca88016020 con 0x7fca901032e0 2024-06-10T02:50:33.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:33.330+0000 7fca867fc700 1 -- 172.21.15.52:0/2588078046 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca8800ad40 con 0x7fca901032e0 2024-06-10T02:50:33.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:33.330+0000 7fca973de700 1 -- 172.21.15.52:0/2588078046 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca901a74e0 con 0x7fca901032e0 2024-06-10T02:50:33.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:33.330+0000 7fca867fc700 1 -- 172.21.15.52:0/2588078046 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca880116f0 con 0x7fca901032e0 2024-06-10T02:50:33.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:33.330+0000 7fca973de700 1 -- 172.21.15.52:0/2588078046 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca901a79d0 con 0x7fca901032e0 2024-06-10T02:50:33.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:33.332+0000 7fca973de700 1 -- 172.21.15.52:0/2588078046 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca90069180 con 0x7fca901032e0 2024-06-10T02:50:33.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:33.333+0000 7fca867fc700 1 -- 172.21.15.52:0/2588078046 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fca88019020 con 0x7fca901032e0 2024-06-10T02:50:33.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:33.334+0000 7fca867fc700 1 --2- 172.21.15.52:0/2588078046 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fca7c0a2c20 0x7fca7c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:50:33.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:33.334+0000 7fca9517a700 1 --2- 172.21.15.52:0/2588078046 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fca7c0a2c20 0x7fca7c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:50:33.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:33.334+0000 7fca867fc700 1 -- 172.21.15.52:0/2588078046 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8802+0+0 (secure 0 0 0) 0x7fca880cf6f0 con 0x7fca901032e0 2024-06-10T02:50:33.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:33.334+0000 7fca9517a700 1 --2- 172.21.15.52:0/2588078046 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fca7c0a2c20 0x7fca7c0a50e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fca8c009750 tx=0x7fca8c00b540 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:50:33.336 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:33.337+0000 7fca867fc700 1 -- 172.21.15.52:0/2588078046 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fca88020020 con 0x7fca901032e0 2024-06-10T02:50:33.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:33.544+0000 7fca973de700 1 -- 172.21.15.52:0/2588078046 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fca9002dca0 con 0x7fca901032e0 2024-06-10T02:50:33.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:33.545+0000 7fca867fc700 1 -- 172.21.15.52:0/2588078046 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) ==== 74+0+24128 (secure 0 0 0) 0x7fca88022300 con 0x7fca901032e0 2024-06-10T02:50:33.544 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-10T02:50:33.544 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":96,"fsid":"d8676e9e-26d2-11ef-bc9b-c7b262605968","created":"2024-06-10T02:41:43.076837+0000","modified":"2024-06-10T02:50:17.718578+0000","last_up_change":"2024-06-10T02:48:57.297829+0000","last_in_change":"2024-06-10T02:48:36.865802+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-10T02:45:07.394690+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-10T02:49:57.926838+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-10T02:50:01.130393+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-10T02:50:10.655588+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e03c0691-cc5a-497a-bbc2-ce18c337f074","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":4255833408},{"type":"v1","addr":"172.21.15.52:6803","nonce":4255833408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":4255833408},{"type":"v1","addr":"172.21.15.52:6805","nonce":4255833408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":4255833408},{"type":"v1","addr":"172.21.15.52:6809","nonce":4255833408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":4255833408},{"type":"v1","addr":"172.21.15.52:6807","nonce":4255833408}]},"public_addr":"172.21.15.52:6803/4255833408","cluster_addr":"172.21.15.52:6805/4255833408","heartbeat_back_addr":"172.21.15.52:6809/4255833408","heartbeat_front_addr":"172.21.15.52:6807/4255833408","state":["exists","up"]},{"osd":1,"uuid":"a24d8da8-c5ec-44d7-ab30-56f6ce3efda7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":4116540053},{"type":"v1","addr":"172.21.15.120:6801","nonce":4116540053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":4116540053},{"type":"v1","addr":"172.21.15.120:6803","nonce":4116540053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":4116540053},{"type":"v1","addr":"172.21.15.120:6807","nonce":4116540053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":4116540053},{"type":"v1","addr":"172.21.15.120:6805","nonce":4116540053}]},"public_addr":"172.21.15.120:6801/4116540053","cluster_addr":"172.21.15.120:6803/4116540053","heartbeat_back_addr":"172.21.15.120:6807/4116540053","heartbeat_front_addr":"172.21.15.120:6805/4116540053","state":["exists","up"]},{"osd":2,"uuid":"a7162106-ccbf-45bd-bc0e-051ee43f4214","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":2624326999},{"type":"v1","addr":"172.21.15.188:6801","nonce":2624326999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":2624326999},{"type":"v1","addr":"172.21.15.188:6803","nonce":2624326999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":2624326999},{"type":"v1","addr":"172.21.15.188:6807","nonce":2624326999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":2624326999},{"type":"v1","addr":"172.21.15.188:6805","nonce":2624326999}]},"public_addr":"172.21.15.188:6801/2624326999","cluster_addr":"172.21.15.188:6803/2624326999","heartbeat_back_addr":"172.21.15.188:6807/2624326999","heartbeat_front_addr":"172.21.15.188:6805/2624326999","state":["exists","up"]},{"osd":3,"uuid":"53680244-b49a-4120-b24a-9ad7a6341dc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":116897011},{"type":"v1","addr":"172.21.15.52:6811","nonce":116897011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":116897011},{"type":"v1","addr":"172.21.15.52:6813","nonce":116897011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":116897011},{"type":"v1","addr":"172.21.15.52:6817","nonce":116897011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":116897011},{"type":"v1","addr":"172.21.15.52:6815","nonce":116897011}]},"public_addr":"172.21.15.52:6811/116897011","cluster_addr":"172.21.15.52:6813/116897011","heartbeat_back_addr":"172.21.15.52:6817/116897011","heartbeat_front_addr":"172.21.15.52:6815/116897011","state":["exists","up"]},{"osd":4,"uuid":"9f72f1f4-2ef3-49c5-8410-c108767d2ef5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":1320690621},{"type":"v1","addr":"172.21.15.120:6809","nonce":1320690621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":1320690621},{"type":"v1","addr":"172.21.15.120:6811","nonce":1320690621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":1320690621},{"type":"v1","addr":"172.21.15.120:6815","nonce":1320690621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":1320690621},{"type":"v1","addr":"172.21.15.120:6813","nonce":1320690621}]},"public_addr":"172.21.15.120:6809/1320690621","cluster_addr":"172.21.15.120:6811/1320690621","heartbeat_back_addr":"172.21.15.120:6815/1320690621","heartbeat_front_addr":"172.21.15.120:6813/1320690621","state":["exists","up"]},{"osd":5,"uuid":"001b5706-7f27-4fa4-9b69-969e6d765abc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3951947483},{"type":"v1","addr":"172.21.15.188:6809","nonce":3951947483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3951947483},{"type":"v1","addr":"172.21.15.188:6811","nonce":3951947483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3951947483},{"type":"v1","addr":"172.21.15.188:6815","nonce":3951947483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3951947483},{"type":"v1","addr":"172.21.15.188:6813","nonce":3951947483}]},"public_addr":"172.21.15.188:6809/3951947483","cluster_addr":"172.21.15.188:6811/3951947483","heartbeat_back_addr":"172.21.15.188:6815/3951947483","heartbeat_front_addr":"172.21.15.188:6813/3951947483","state":["exists","up"]},{"osd":6,"uuid":"140c903e-9b31-4a81-9ccd-2edc0a2decf9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":359084919},{"type":"v1","addr":"172.21.15.52:6819","nonce":359084919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":359084919},{"type":"v1","addr":"172.21.15.52:6821","nonce":359084919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":359084919},{"type":"v1","addr":"172.21.15.52:6825","nonce":359084919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":359084919},{"type":"v1","addr":"172.21.15.52:6823","nonce":359084919}]},"public_addr":"172.21.15.52:6819/359084919","cluster_addr":"172.21.15.52:6821/359084919","heartbeat_back_addr":"172.21.15.52:6825/359084919","heartbeat_front_addr":"172.21.15.52:6823/359084919","state":["exists","up"]},{"osd":7,"uuid":"57b735d4-5072-4ca2-a3ff-3763ad84f15a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":3737722011},{"type":"v1","addr":"172.21.15.120:6817","nonce":3737722011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6818","nonce":3737722011},{"type":"v1","addr":"172.21.15.120:6819","nonce":3737722011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":3737722011},{"type":"v1","addr":"172.21.15.120:6823","nonce":3737722011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":3737722011},{"type":"v1","addr":"172.21.15.120:6821","nonce":3737722011}]},"public_addr":"172.21.15.120:6817/3737722011","cluster_addr":"172.21.15.120:6819/3737722011","heartbeat_back_addr":"172.21.15.120:6823/3737722011","heartbeat_front_addr":"172.21.15.120:6821/3737722011","state":["exists","up"]},{"osd":8,"uuid":"4ffbb43b-854a-4eac-99cf-7af89bc65409","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1444828441},{"type":"v1","addr":"172.21.15.188:6817","nonce":1444828441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1444828441},{"type":"v1","addr":"172.21.15.188:6819","nonce":1444828441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1444828441},{"type":"v1","addr":"172.21.15.188:6823","nonce":1444828441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1444828441},{"type":"v1","addr":"172.21.15.188:6821","nonce":1444828441}]},"public_addr":"172.21.15.188:6817/1444828441","cluster_addr":"172.21.15.188:6819/1444828441","heartbeat_back_addr":"172.21.15.188:6823/1444828441","heartbeat_front_addr":"172.21.15.188:6821/1444828441","state":["exists","up"]},{"osd":9,"uuid":"5bfd985c-8acc-4d00-8561-40bbd541c07b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":924440658},{"type":"v1","addr":"172.21.15.52:6827","nonce":924440658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":924440658},{"type":"v1","addr":"172.21.15.52:6829","nonce":924440658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":924440658},{"type":"v1","addr":"172.21.15.52:6833","nonce":924440658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":924440658},{"type":"v1","addr":"172.21.15.52:6831","nonce":924440658}]},"public_addr":"172.21.15.52:6827/924440658","cluster_addr":"172.21.15.52:6829/924440658","heartbeat_back_addr":"172.21.15.52:6833/924440658","heartbeat_front_addr":"172.21.15.52:6831/924440658","state":["exists","up"]},{"osd":10,"uuid":"0180cddf-e109-42cc-829a-185604276f02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6824","nonce":1531769083},{"type":"v1","addr":"172.21.15.120:6825","nonce":1531769083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":1531769083},{"type":"v1","addr":"172.21.15.120:6827","nonce":1531769083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":1531769083},{"type":"v1","addr":"172.21.15.120:6831","nonce":1531769083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6828","nonce":1531769083},{"type":"v1","addr":"172.21.15.120:6829","nonce":1531769083}]},"public_addr":"172.21.15.120:6825/1531769083","cluster_addr":"172.21.15.120:6827/1531769083","heartbeat_back_addr":"172.21.15.120:6831/1531769083","heartbeat_front_addr":"172.21.15.120:6829/1531769083","state":["exists","up"]},{"osd":11,"uuid":"73aec750-82d1-4800-b6a0-4e305912f9ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":3769050170},{"type":"v1","addr":"172.21.15.188:6825","nonce":3769050170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":3769050170},{"type":"v1","addr":"172.21.15.188:6827","nonce":3769050170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":3769050170},{"type":"v1","addr":"172.21.15.188:6831","nonce":3769050170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":3769050170},{"type":"v1","addr":"172.21.15.188:6829","nonce":3769050170}]},"public_addr":"172.21.15.188:6825/3769050170","cluster_addr":"172.21.15.188:6827/3769050170","heartbeat_back_addr":"172.21.15.188:6831/3769050170","heartbeat_front_addr":"172.21.15.188:6829/3769050170","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:44:16.455752+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:44:40.219735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:45:03.558899+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:45:29.155004+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:45:53.981098+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:46:19.412020+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:46:45.462640+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:47:11.484578+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:47:36.879054+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:48:03.392446+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:48:29.422173+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:48:55.523423+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.52:0/3535271671":"2024-06-11T02:42:42.872312+0000","172.21.15.52:0/1092934380":"2024-06-11T02:42:42.872312+0000","172.21.15.52:6800/896317057":"2024-06-11T02:42:42.872312+0000","172.21.15.52:6801/3934479642":"2024-06-11T02:42:15.498118+0000","172.21.15.52:6800/3934479642":"2024-06-11T02:42:15.498118+0000","172.21.15.52:0/2759694132":"2024-06-11T02:42:15.498118+0000","172.21.15.52:0/2593665631":"2024-06-11T02:42:15.498118+0000","172.21.15.52:0/3151504634":"2024-06-11T02:42:15.498118+0000","172.21.15.52:6801/896317057":"2024-06-11T02:42:42.872312+0000","172.21.15.52:0/1254480982":"2024-06-11T02:42:42.872312+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-10T02:50:33.546 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:33.548+0000 7fca973de700 1 -- 172.21.15.52:0/2588078046 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fca7c0a2c20 msgr2=0x7fca7c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:50:33.546 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:33.548+0000 7fca973de700 1 --2- 172.21.15.52:0/2588078046 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fca7c0a2c20 0x7fca7c0a50e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fca8c009750 tx=0x7fca8c00b540 comp rx=0 tx=0).stop 2024-06-10T02:50:33.547 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:33.548+0000 7fca973de700 1 -- 172.21.15.52:0/2588078046 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fca901032e0 msgr2=0x7fca90104c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:50:33.547 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:33.548+0000 7fca973de700 1 --2- 172.21.15.52:0/2588078046 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fca901032e0 0x7fca90104c00 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fca88007d70 tx=0x7fca8800a340 comp rx=0 tx=0).stop 2024-06-10T02:50:33.547 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:33.548+0000 7fca973de700 1 -- 172.21.15.52:0/2588078046 shutdown_connections 2024-06-10T02:50:33.547 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:33.548+0000 7fca973de700 1 --2- 172.21.15.52:0/2588078046 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fca7c0a2c20 0x7fca7c0a50e0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:50:33.547 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:33.548+0000 7fca973de700 1 --2- 172.21.15.52:0/2588078046 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fca901032e0 0x7fca90104c00 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:50:33.547 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:33.548+0000 7fca973de700 1 --2- 172.21.15.52:0/2588078046 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fca901013c0 0x7fca90104180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:50:33.547 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:33.548+0000 7fca973de700 1 --2- 172.21.15.52:0/2588078046 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fca90101d80 0x7fca901046c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:50:33.547 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:33.548+0000 7fca973de700 1 -- 172.21.15.52:0/2588078046 >> 172.21.15.52:0/2588078046 conn(0x7fca90077400 msgr2=0x7fca9006aec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:50:33.547 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:33.548+0000 7fca973de700 1 -- 172.21.15.52:0/2588078046 shutdown_connections 2024-06-10T02:50:33.547 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:33.548+0000 7fca973de700 1 -- 172.21.15.52:0/2588078046 wait complete. 2024-06-10T02:50:33.858 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:50:33 smithi052 ceph-mon[32362]: from='client.? 172.21.15.52:0/2588078046' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T02:50:34.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:50:33 smithi120 ceph-mon[36634]: from='client.? 172.21.15.52:0/2588078046' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T02:50:34.174 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-06-10T02:50:03.889766+0000', 'modified': '2024-06-10T02:50:17.718706+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24484}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24484': {'gid': 24484, 'name': 'i', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.188:6839/310444836', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6838', 'nonce': 310444836}, {'type': 'v1', 'addr': '172.21.15.188:6839', 'nonce': 310444836}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 24484, 'qdb_cluster': [24484]} 2024-06-10T02:50:34.174 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-06-10T02:50:34.174 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph tell mds.i status 2024-06-10T02:50:34.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:50:33 smithi188 ceph-mon[36599]: from='client.? 172.21.15.52:0/2588078046' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T02:50:34.433 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:50:34.726 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:50:34 smithi052 ceph-mon[32362]: pgmap v321: 129 pgs: 129 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T02:50:35.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:50:34 smithi120 ceph-mon[36634]: pgmap v321: 129 pgs: 129 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T02:50:35.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:50:34 smithi188 ceph-mon[36599]: pgmap v321: 129 pgs: 129 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T02:50:36.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.131+0000 7f1d37bed700 1 -- 172.21.15.52:0/1077616794 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1d3006ac00 msgr2=0x7f1d3006b080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:50:36.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.131+0000 7f1d37bed700 1 --2- 172.21.15.52:0/1077616794 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1d3006ac00 0x7f1d3006b080 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f1d20009990 tx=0x7f1d20009ca0 comp rx=0 tx=0).stop 2024-06-10T02:50:36.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.131+0000 7f1d37bed700 1 -- 172.21.15.52:0/1077616794 shutdown_connections 2024-06-10T02:50:36.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.131+0000 7f1d37bed700 1 --2- 172.21.15.52:0/1077616794 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1d3006b5c0 0x7f1d3010f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:50:36.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.131+0000 7f1d37bed700 1 --2- 172.21.15.52:0/1077616794 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1d30108530 0x7f1d30108910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:50:36.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.131+0000 7f1d37bed700 1 --2- 172.21.15.52:0/1077616794 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1d3006ac00 0x7f1d3006b080 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:50:36.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.131+0000 7f1d37bed700 1 -- 172.21.15.52:0/1077616794 >> 172.21.15.52:0/1077616794 conn(0x7f1d30077400 msgr2=0x7f1d30077810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:50:36.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.131+0000 7f1d37bed700 1 -- 172.21.15.52:0/1077616794 shutdown_connections 2024-06-10T02:50:36.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.131+0000 7f1d37bed700 1 -- 172.21.15.52:0/1077616794 wait complete. 2024-06-10T02:50:36.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.132+0000 7f1d37bed700 1 Processor -- start 2024-06-10T02:50:36.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.132+0000 7f1d37bed700 1 -- start start 2024-06-10T02:50:36.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.132+0000 7f1d37bed700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1d3006ac00 0x7f1d30198d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:50:36.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.132+0000 7f1d37bed700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1d3006b5c0 0x7f1d301992d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:50:36.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.132+0000 7f1d37bed700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1d30108530 0x7f1d3019d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:50:36.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.132+0000 7f1d37bed700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f1d30112860 con 0x7f1d3006ac00 2024-06-10T02:50:36.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.132+0000 7f1d37bed700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1d301126e0 con 0x7f1d30108530 2024-06-10T02:50:36.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.132+0000 7f1d37bed700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f1d301129e0 con 0x7f1d3006b5c0 2024-06-10T02:50:36.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.132+0000 7f1d35989700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1d3006ac00 0x7f1d30198d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:50:36.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.133+0000 7f1d35989700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1d3006ac00 0x7f1d30198d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:41136/0 (socket says 172.21.15.52:41136) 2024-06-10T02:50:36.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.133+0000 7f1d35188700 1 --2- 172.21.15.52:0/1004805548 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1d3006b5c0 0x7f1d301992d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:50:36.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.133+0000 7f1d35989700 1 -- 172.21.15.52:0/1004805548 learned_addr learned my addr 172.21.15.52:0/1004805548 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:50:36.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.133+0000 7f1d3618a700 1 --2- 172.21.15.52:0/1004805548 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1d30108530 0x7f1d3019d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:50:36.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.133+0000 7f1d35989700 1 -- 172.21.15.52:0/1004805548 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1d3006b5c0 msgr2=0x7f1d301992d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:50:36.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.133+0000 7f1d35989700 1 --2- 172.21.15.52:0/1004805548 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1d3006b5c0 0x7f1d301992d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:50:36.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.133+0000 7f1d35989700 1 -- 172.21.15.52:0/1004805548 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1d30108530 msgr2=0x7f1d3019d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:50:36.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.133+0000 7f1d35989700 1 --2- 172.21.15.52:0/1004805548 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1d30108530 0x7f1d3019d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:50:36.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.133+0000 7f1d35989700 1 -- 172.21.15.52:0/1004805548 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d3019dda0 con 0x7f1d3006ac00 2024-06-10T02:50:36.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.133+0000 7f1d35989700 1 --2- 172.21.15.52:0/1004805548 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1d3006ac00 0x7f1d30198d90 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f1d2c00ba50 tx=0x7f1d2c00bd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:50:36.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.133+0000 7f1d26ffd700 1 -- 172.21.15.52:0/1004805548 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d2c019070 con 0x7f1d3006ac00 2024-06-10T02:50:36.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.133+0000 7f1d26ffd700 1 -- 172.21.15.52:0/1004805548 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d2c014440 con 0x7f1d3006ac00 2024-06-10T02:50:36.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.133+0000 7f1d37bed700 1 -- 172.21.15.52:0/1004805548 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d3019e090 con 0x7f1d3006ac00 2024-06-10T02:50:36.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.133+0000 7f1d26ffd700 1 -- 172.21.15.52:0/1004805548 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d2c0075d0 con 0x7f1d3006ac00 2024-06-10T02:50:36.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.133+0000 7f1d37bed700 1 -- 172.21.15.52:0/1004805548 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d301a5910 con 0x7f1d3006ac00 2024-06-10T02:50:36.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.136+0000 7f1d26ffd700 1 -- 172.21.15.52:0/1004805548 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f1d2c01c070 con 0x7f1d3006ac00 2024-06-10T02:50:36.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.136+0000 7f1d26ffd700 1 --2- 172.21.15.52:0/1004805548 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f1d1c0a2c20 0x7f1d1c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:50:36.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.136+0000 7f1d26ffd700 1 -- 172.21.15.52:0/1004805548 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8802+0+0 (secure 0 0 0) 0x7f1d2c0ce660 con 0x7f1d3006ac00 2024-06-10T02:50:36.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.136+0000 7f1d35188700 1 --2- 172.21.15.52:0/1004805548 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f1d1c0a2c20 0x7f1d1c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:50:36.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.137+0000 7f1d35188700 1 --2- 172.21.15.52:0/1004805548 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f1d1c0a2c20 0x7f1d1c0a50e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f1d2000b5c0 tx=0x7f1d20005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:50:36.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.137+0000 7f1d42470b80 1 Processor -- start 2024-06-10T02:50:36.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.137+0000 7f1d42470b80 1 -- start start 2024-06-10T02:50:36.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.137+0000 7f1d42470b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d8678385f0 0x55d8678389d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:50:36.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.137+0000 7f1d42470b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d867838f10 0x55d867832990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:50:36.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.137+0000 7f1d42470b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d867832ed0 0x55d8678332f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:50:36.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.137+0000 7f1d42470b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55d8677847c0 con 0x55d867838f10 2024-06-10T02:50:36.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.137+0000 7f1d42470b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55d86772c6d0 con 0x55d8678385f0 2024-06-10T02:50:36.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.137+0000 7f1d42470b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55d867780780 con 0x55d867832ed0 2024-06-10T02:50:36.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.137+0000 7f1d35188700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d867838f10 0x55d867832990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:50:36.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.137+0000 7f1d35188700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d867838f10 0x55d867832990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:41138/0 (socket says 172.21.15.52:41138) 2024-06-10T02:50:36.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.138+0000 7f1d35188700 1 -- 172.21.15.52:0/3473808593 learned_addr learned my addr 172.21.15.52:0/3473808593 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:50:36.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.138+0000 7f1d3618a700 1 --2- 172.21.15.52:0/3473808593 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d867832ed0 0x55d8678332f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:50:36.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.138+0000 7f1d35989700 1 --2- 172.21.15.52:0/3473808593 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d8678385f0 0x55d8678389d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:50:36.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.138+0000 7f1d35188700 1 -- 172.21.15.52:0/3473808593 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d867832ed0 msgr2=0x55d8678332f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:50:36.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.138+0000 7f1d35188700 1 --2- 172.21.15.52:0/3473808593 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d867832ed0 0x55d8678332f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:50:36.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.138+0000 7f1d35188700 1 -- 172.21.15.52:0/3473808593 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d8678385f0 msgr2=0x55d8678389d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:50:36.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.138+0000 7f1d35188700 1 --2- 172.21.15.52:0/3473808593 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d8678385f0 0x55d8678389d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:50:36.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.138+0000 7f1d35188700 1 -- 172.21.15.52:0/3473808593 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d867722f00 con 0x55d867838f10 2024-06-10T02:50:36.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.138+0000 7f1d35989700 1 --2- 172.21.15.52:0/3473808593 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d8678385f0 0x55d8678389d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T02:50:36.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.138+0000 7f1d35188700 1 --2- 172.21.15.52:0/3473808593 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d867838f10 0x55d867832990 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f1d20015000 tx=0x7f1d20020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2c312e27f102187 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:50:36.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.138+0000 7f1d1bfff700 1 -- 172.21.15.52:0/3473808593 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d2000bda0 con 0x55d867838f10 2024-06-10T02:50:36.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.138+0000 7f1d1bfff700 1 -- 172.21.15.52:0/3473808593 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d2000fd50 con 0x55d867838f10 2024-06-10T02:50:36.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.138+0000 7f1d1bfff700 1 -- 172.21.15.52:0/3473808593 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d2002f400 con 0x55d867838f10 2024-06-10T02:50:36.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.139+0000 7f1d42470b80 1 -- 172.21.15.52:0/3473808593 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d867838f10 msgr2=0x55d867832990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:50:36.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.139+0000 7f1d42470b80 1 --2- 172.21.15.52:0/3473808593 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d867838f10 0x55d867832990 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f1d20015000 tx=0x7f1d20020000 comp rx=0 tx=0).stop 2024-06-10T02:50:36.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.139+0000 7f1d42470b80 1 -- 172.21.15.52:0/3473808593 shutdown_connections 2024-06-10T02:50:36.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.139+0000 7f1d42470b80 1 --2- 172.21.15.52:0/3473808593 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d867832ed0 0x55d8678332f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:50:36.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.139+0000 7f1d42470b80 1 --2- 172.21.15.52:0/3473808593 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d8678385f0 0x55d8678389d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:50:36.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.139+0000 7f1d42470b80 1 --2- 172.21.15.52:0/3473808593 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d867838f10 0x55d867832990 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:50:36.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.139+0000 7f1d42470b80 1 -- 172.21.15.52:0/3473808593 >> 172.21.15.52:0/3473808593 conn(0x55d86775dc70 msgr2=0x55d86782d800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:50:36.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.139+0000 7f1d42470b80 1 -- 172.21.15.52:0/3473808593 shutdown_connections 2024-06-10T02:50:36.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.139+0000 7f1d42470b80 1 -- 172.21.15.52:0/3473808593 wait complete. 2024-06-10T02:50:36.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.140+0000 7f1d42470b80 1 -- start start 2024-06-10T02:50:36.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.140+0000 7f1d42470b80 1 Processor -- start 2024-06-10T02:50:36.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.140+0000 7f1d42470b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d867832ed0 0x55d86783ec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:50:36.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.140+0000 7f1d42470b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d8678385f0 0x55d86783bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:50:36.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.140+0000 7f1d42470b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d867838f10 0x55d86783c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:50:36.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.140+0000 7f1d42470b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55d86783f8c0 con 0x55d867838f10 2024-06-10T02:50:36.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.140+0000 7f1d42470b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55d867724ba0 con 0x55d8678385f0 2024-06-10T02:50:36.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.140+0000 7f1d42470b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55d86783fa40 con 0x55d867832ed0 2024-06-10T02:50:36.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.140+0000 7f1d42470b80 10 client.0 _ll_register_callbacks cb 0x55d86771e8d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T02:50:36.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.140+0000 7f1d3618a700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d867838f10 0x55d86783c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:50:36.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.140+0000 7f1d3618a700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d867838f10 0x55d86783c4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:41142/0 (socket says 172.21.15.52:41142) 2024-06-10T02:50:36.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.140+0000 7f1d35989700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d867832ed0 0x55d86783ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:50:36.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.140+0000 7f1d3618a700 1 -- 172.21.15.52:0/1816731452 learned_addr learned my addr 172.21.15.52:0/1816731452 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:50:36.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.140+0000 7f1d35188700 1 --2- 172.21.15.52:0/1816731452 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d8678385f0 0x55d86783bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:50:36.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.141+0000 7f1d3618a700 1 -- 172.21.15.52:0/1816731452 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d867832ed0 msgr2=0x55d86783ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:50:36.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.141+0000 7f1d3618a700 1 --2- 172.21.15.52:0/1816731452 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d867832ed0 0x55d86783ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:50:36.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.141+0000 7f1d3618a700 1 -- 172.21.15.52:0/1816731452 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d8678385f0 msgr2=0x55d86783bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:50:36.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.141+0000 7f1d3618a700 1 --2- 172.21.15.52:0/1816731452 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d8678385f0 0x55d86783bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:50:36.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.141+0000 7f1d3618a700 1 -- 172.21.15.52:0/1816731452 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d8676d3880 con 0x55d867838f10 2024-06-10T02:50:36.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.141+0000 7f1d3618a700 1 --2- 172.21.15.52:0/1816731452 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d867838f10 0x55d86783c4b0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f1d28010500 tx=0x7f1d28010810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1ceb4fdcc616110 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:50:36.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.141+0000 7f1d197fa700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-10T02:50:36.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.141+0000 7f1d197fa700 1 -- 172.21.15.52:0/1816731452 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d2801a020 con 0x55d867838f10 2024-06-10T02:50:36.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.141+0000 7f1d197fa700 1 -- 172.21.15.52:0/1816731452 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8802+0+0 (secure 0 0 0) 0x7f1d28016480 con 0x55d867838f10 2024-06-10T02:50:36.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.141+0000 7f1d42470b80 1 -- 172.21.15.52:0/1816731452 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1d2000fd50 con 0x55d867838f10 2024-06-10T02:50:36.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.141+0000 7f1d197fa700 1 -- 172.21.15.52:0/1816731452 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f1d28025050 con 0x55d867838f10 2024-06-10T02:50:36.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.141+0000 7f1d42470b80 10 client.14691 fetch_fsmap learned FSMap version 17 2024-06-10T02:50:36.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.141+0000 7f1d42470b80 1 -- 172.21.15.52:0/1816731452 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x55d86784a220 con 0x55d867838f10 2024-06-10T02:50:36.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.141+0000 7f1d197fa700 1 -- 172.21.15.52:0/1816731452 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d2000fd50 con 0x55d867838f10 2024-06-10T02:50:36.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.142+0000 7f1d197fa700 1 -- 172.21.15.52:0/1816731452 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 17) ==== 8342+0+0 (secure 0 0 0) 0x7f1d280149c0 con 0x55d867838f10 2024-06-10T02:50:36.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.142+0000 7f1d42470b80 10 client.14691 fetch_fsmap finished waiting for FSMap version 17 2024-06-10T02:50:36.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.142+0000 7f1d42470b80 10 client.14691 resolve_mds: resolved name 'i' to daemon mds.i 2024-06-10T02:50:36.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.142+0000 7f1d42470b80 20 client.14691 populate_metadata read hostname 'smithi052' 2024-06-10T02:50:36.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.142+0000 7f1d42470b80 1 --2- 172.21.15.52:0/1816731452 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55d86784c880 0x55d86784dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:50:36.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.142+0000 7f1d42470b80 4 client.14691 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T02:50:36.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.142+0000 7f1d42470b80 1 -- 172.21.15.52:0/1816731452 --> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d867791cc0 con 0x55d86784c880 2024-06-10T02:50:36.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.142+0000 7f1d35989700 1 --2- 172.21.15.52:0/1816731452 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55d86784c880 0x55d86784dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:50:36.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.142+0000 7f1d35989700 1 --2- 172.21.15.52:0/1816731452 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55d86784c880 0x55d86784dcf0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=91d2ea6f94f133da server_cookie=8f06328f83ac6a3b in_seq=0 out_seq=0 2024-06-10T02:50:36.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.142+0000 7f1d197fa700 10 client.14691 ms_handle_connect on v2:172.21.15.188:6838/310444836 2024-06-10T02:50:36.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.143+0000 7f1d197fa700 1 -- 172.21.15.52:0/1816731452 <== mds.0 v2:172.21.15.188:6838/310444836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55d867791cc0 con 0x55d86784c880 2024-06-10T02:50:36.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.143+0000 7f1d197fa700 10 client.14691 handle_command_reply: tid=0 2024-06-10T02:50:36.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.143+0000 7f1d42470b80 1 client.14691 shutdown 2024-06-10T02:50:36.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.143+0000 7f1d42470b80 1 -- 172.21.15.52:0/1816731452 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d867838f10 msgr2=0x55d86783c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:50:36.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.143+0000 7f1d42470b80 1 --2- 172.21.15.52:0/1816731452 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d867838f10 0x55d86783c4b0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f1d28010500 tx=0x7f1d28010810 comp rx=0 tx=0).stop 2024-06-10T02:50:36.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.143+0000 7f1d42470b80 1 -- 172.21.15.52:0/1816731452 shutdown_connections 2024-06-10T02:50:36.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.143+0000 7f1d42470b80 1 --2- 172.21.15.52:0/1816731452 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55d86784c880 0x55d86784dcf0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:50:36.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.143+0000 7f1d42470b80 1 --2- 172.21.15.52:0/1816731452 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d867838f10 0x55d86783c4b0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:50:36.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.143+0000 7f1d42470b80 1 --2- 172.21.15.52:0/1816731452 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d867832ed0 0x55d86783ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:50:36.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.143+0000 7f1d42470b80 1 --2- 172.21.15.52:0/1816731452 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d8678385f0 0x55d86783bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:50:36.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.143+0000 7f1d197fa700 0 client.14691 ms_handle_reset on v2:172.21.15.188:6838/310444836 2024-06-10T02:50:36.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.143+0000 7f1d42470b80 1 -- 172.21.15.52:0/1816731452 >> 172.21.15.52:0/1816731452 conn(0x55d86775dc70 msgr2=0x55d86782d5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:50:36.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.143+0000 7f1d42470b80 1 -- 172.21.15.52:0/1816731452 shutdown_connections 2024-06-10T02:50:36.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.143+0000 7f1d42470b80 1 -- 172.21.15.52:0/1816731452 wait complete. 2024-06-10T02:50:36.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.144+0000 7f1d42470b80 20 client.14691 trim_cache size 0 max 16384 2024-06-10T02:50:36.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.165+0000 7f1d42470b80 1 Processor -- start 2024-06-10T02:50:36.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.165+0000 7f1d42470b80 1 -- start start 2024-06-10T02:50:36.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.165+0000 7f1d42470b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d867832ed0 0x55d867832990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:50:36.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.166+0000 7f1d42470b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d8678385f0 0x55d86783c520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:50:36.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.166+0000 7f1d42470b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d867838f10 0x55d867857540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:50:36.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.166+0000 7f1d42470b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55d86784c030 con 0x55d867838f10 2024-06-10T02:50:36.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.166+0000 7f1d42470b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55d86784c700 con 0x55d8678385f0 2024-06-10T02:50:36.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.166+0000 7f1d42470b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55d86784cdd0 con 0x55d867832ed0 2024-06-10T02:50:36.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.166+0000 7f1d3618a700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d867838f10 0x55d867857540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:50:36.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.166+0000 7f1d3618a700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d867838f10 0x55d867857540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:41154/0 (socket says 172.21.15.52:41154) 2024-06-10T02:50:36.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.166+0000 7f1d3618a700 1 -- 172.21.15.52:0/1018264893 learned_addr learned my addr 172.21.15.52:0/1018264893 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:50:36.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.166+0000 7f1d35989700 1 --2- 172.21.15.52:0/1018264893 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d867832ed0 0x55d867832990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:50:36.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.166+0000 7f1d35188700 1 --2- 172.21.15.52:0/1018264893 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d8678385f0 0x55d86783c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:50:36.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.166+0000 7f1d3618a700 1 -- 172.21.15.52:0/1018264893 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d867832ed0 msgr2=0x55d867832990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:50:36.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.166+0000 7f1d3618a700 1 --2- 172.21.15.52:0/1018264893 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d867832ed0 0x55d867832990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:50:36.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.166+0000 7f1d3618a700 1 -- 172.21.15.52:0/1018264893 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d8678385f0 msgr2=0x55d86783c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:50:36.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.166+0000 7f1d3618a700 1 --2- 172.21.15.52:0/1018264893 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d8678385f0 0x55d86783c520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:50:36.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.166+0000 7f1d3618a700 1 -- 172.21.15.52:0/1018264893 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d8678380f0 con 0x55d867838f10 2024-06-10T02:50:36.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.166+0000 7f1d35989700 1 --2- 172.21.15.52:0/1018264893 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d867832ed0 0x55d867832990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T02:50:36.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.166+0000 7f1d35188700 1 --2- 172.21.15.52:0/1018264893 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d8678385f0 0x55d86783c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T02:50:36.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.166+0000 7f1d3618a700 1 --2- 172.21.15.52:0/1018264893 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d867838f10 0x55d867857540 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f1d28014e60 tx=0x7f1d28010b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e20237b2774e1f1 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:50:36.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.166+0000 7f1d1b7fe700 1 -- 172.21.15.52:0/1018264893 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d2801a040 con 0x55d867838f10 2024-06-10T02:50:36.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.166+0000 7f1d1b7fe700 1 -- 172.21.15.52:0/1018264893 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d28014910 con 0x55d867838f10 2024-06-10T02:50:36.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.166+0000 7f1d1b7fe700 1 -- 172.21.15.52:0/1018264893 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d28016de0 con 0x55d867838f10 2024-06-10T02:50:36.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.167+0000 7f1d42470b80 1 -- 172.21.15.52:0/1018264893 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d867838f10 msgr2=0x55d867857540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:50:36.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.167+0000 7f1d42470b80 1 --2- 172.21.15.52:0/1018264893 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d867838f10 0x55d867857540 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f1d28014e60 tx=0x7f1d28010b60 comp rx=0 tx=0).stop 2024-06-10T02:50:36.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.167+0000 7f1d42470b80 1 -- 172.21.15.52:0/1018264893 shutdown_connections 2024-06-10T02:50:36.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.167+0000 7f1d42470b80 1 --2- 172.21.15.52:0/1018264893 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d867838f10 0x55d867857540 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:50:36.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.167+0000 7f1d42470b80 1 --2- 172.21.15.52:0/1018264893 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d867832ed0 0x55d867832990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:50:36.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.167+0000 7f1d42470b80 1 --2- 172.21.15.52:0/1018264893 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d8678385f0 0x55d86783c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:50:36.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.167+0000 7f1d42470b80 1 -- 172.21.15.52:0/1018264893 >> 172.21.15.52:0/1018264893 conn(0x55d86775dc70 msgr2=0x55d86783a8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:50:36.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.167+0000 7f1d42470b80 1 -- 172.21.15.52:0/1018264893 shutdown_connections 2024-06-10T02:50:36.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.167+0000 7f1d42470b80 1 -- 172.21.15.52:0/1018264893 wait complete. 2024-06-10T02:50:36.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.167+0000 7f1d42470b80 1 -- start start 2024-06-10T02:50:36.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.167+0000 7f1d42470b80 1 Processor -- start 2024-06-10T02:50:36.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.168+0000 7f1d42470b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d867832ed0 0x55d86783b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:50:36.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.168+0000 7f1d42470b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d8678385f0 0x55d86783be30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:50:36.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.168+0000 7f1d42470b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d867838f10 0x55d8678bb140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:50:36.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.168+0000 7f1d42470b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55d86784db70 con 0x55d8678385f0 2024-06-10T02:50:36.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.168+0000 7f1d42470b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55d86784d4a0 con 0x55d867838f10 2024-06-10T02:50:36.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.168+0000 7f1d42470b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55d86784e240 con 0x55d867832ed0 2024-06-10T02:50:36.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.168+0000 7f1d42470b80 10 client.0 _ll_register_callbacks cb 0x55d86771e8d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T02:50:36.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.168+0000 7f1d35188700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d8678385f0 0x55d86783be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:50:36.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.168+0000 7f1d35188700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d8678385f0 0x55d86783be30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:41156/0 (socket says 172.21.15.52:41156) 2024-06-10T02:50:36.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.168+0000 7f1d35188700 1 -- 172.21.15.52:0/4293040614 learned_addr learned my addr 172.21.15.52:0/4293040614 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:50:36.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.168+0000 7f1d3618a700 1 --2- 172.21.15.52:0/4293040614 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d867838f10 0x55d8678bb140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:50:36.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.168+0000 7f1d35989700 1 --2- 172.21.15.52:0/4293040614 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d867832ed0 0x55d86783b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:50:36.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.168+0000 7f1d35188700 1 -- 172.21.15.52:0/4293040614 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d867832ed0 msgr2=0x55d86783b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:50:36.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.168+0000 7f1d35188700 1 --2- 172.21.15.52:0/4293040614 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d867832ed0 0x55d86783b8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:50:36.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.168+0000 7f1d35188700 1 -- 172.21.15.52:0/4293040614 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d867838f10 msgr2=0x55d8678bb140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:50:36.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.168+0000 7f1d35188700 1 --2- 172.21.15.52:0/4293040614 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d867838f10 0x55d8678bb140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:50:36.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.168+0000 7f1d35188700 1 -- 172.21.15.52:0/4293040614 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d86783c710 con 0x55d8678385f0 2024-06-10T02:50:36.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.169+0000 7f1d35188700 1 --2- 172.21.15.52:0/4293040614 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d8678385f0 0x55d86783be30 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f1d2000ba90 tx=0x7f1d2000bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32c074dd276f963d server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:50:36.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.169+0000 7f1d197fa700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-10T02:50:36.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.169+0000 7f1d197fa700 1 -- 172.21.15.52:0/4293040614 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d2003c070 con 0x55d8678385f0 2024-06-10T02:50:36.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.169+0000 7f1d197fa700 1 -- 172.21.15.52:0/4293040614 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8802+0+0 (secure 0 0 0) 0x7f1d20024cc0 con 0x55d8678385f0 2024-06-10T02:50:36.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.169+0000 7f1d42470b80 1 -- 172.21.15.52:0/4293040614 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1d28014910 con 0x55d8678385f0 2024-06-10T02:50:36.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.169+0000 7f1d197fa700 1 -- 172.21.15.52:0/4293040614 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f1d2000f870 con 0x55d8678385f0 2024-06-10T02:50:36.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.169+0000 7f1d42470b80 10 client.14697 fetch_fsmap learned FSMap version 17 2024-06-10T02:50:36.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.169+0000 7f1d42470b80 1 -- 172.21.15.52:0/4293040614 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x55d8678bc6e0 con 0x55d8678385f0 2024-06-10T02:50:36.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.169+0000 7f1d197fa700 1 -- 172.21.15.52:0/4293040614 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d28014910 con 0x55d8678385f0 2024-06-10T02:50:36.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.169+0000 7f1d197fa700 1 -- 172.21.15.52:0/4293040614 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 17) ==== 8342+0+0 (secure 0 0 0) 0x7f1d2003f030 con 0x55d8678385f0 2024-06-10T02:50:36.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.169+0000 7f1d42470b80 10 client.14697 fetch_fsmap finished waiting for FSMap version 17 2024-06-10T02:50:36.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.169+0000 7f1d42470b80 10 client.14697 resolve_mds: resolved name 'i' to daemon mds.i 2024-06-10T02:50:36.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.169+0000 7f1d42470b80 20 client.14697 populate_metadata read hostname 'smithi052' 2024-06-10T02:50:36.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.169+0000 7f1d42470b80 1 --2- 172.21.15.52:0/4293040614 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55d8678c0e20 0x55d8678bfb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:50:36.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.169+0000 7f1d42470b80 4 client.14697 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "status"}] 2024-06-10T02:50:36.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.169+0000 7f1d42470b80 1 -- 172.21.15.52:0/4293040614 --> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] -- command(tid 0: {"prefix": "status"}) -- 0x55d86778c240 con 0x55d8678c0e20 2024-06-10T02:50:36.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.170+0000 7f1d35989700 1 --2- 172.21.15.52:0/4293040614 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55d8678c0e20 0x55d8678bfb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:50:36.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.170+0000 7f1d35989700 1 --2- 172.21.15.52:0/4293040614 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55d8678c0e20 0x55d8678bfb30 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=77b26fc468c79f53 server_cookie=59628486404c84aa in_seq=0 out_seq=0 2024-06-10T02:50:36.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.170+0000 7f1d197fa700 10 client.14697 ms_handle_connect on v2:172.21.15.188:6838/310444836 2024-06-10T02:50:36.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.170+0000 7f1d197fa700 1 -- 172.21.15.52:0/4293040614 <== mds.0 v2:172.21.15.188:6838/310444836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x55d86778c240 con 0x55d8678c0e20 2024-06-10T02:50:36.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.170+0000 7f1d197fa700 10 client.14697 handle_command_reply: tid=0 2024-06-10T02:50:36.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.170+0000 7f1d42470b80 1 client.14697 shutdown 2024-06-10T02:50:36.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.170+0000 7f1d42470b80 1 -- 172.21.15.52:0/4293040614 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d8678385f0 msgr2=0x55d86783be30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:50:36.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.170+0000 7f1d42470b80 1 --2- 172.21.15.52:0/4293040614 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d8678385f0 0x55d86783be30 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f1d2000ba90 tx=0x7f1d2000bfd0 comp rx=0 tx=0).stop 2024-06-10T02:50:36.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.170+0000 7f1d42470b80 1 -- 172.21.15.52:0/4293040614 shutdown_connections 2024-06-10T02:50:36.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.170+0000 7f1d42470b80 1 --2- 172.21.15.52:0/4293040614 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55d8678c0e20 0x55d8678bfb30 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:50:36.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.170+0000 7f1d42470b80 1 --2- 172.21.15.52:0/4293040614 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d867838f10 0x55d8678bb140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:50:36.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.170+0000 7f1d197fa700 0 client.14697 ms_handle_reset on v2:172.21.15.188:6838/310444836 2024-06-10T02:50:36.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.170+0000 7f1d42470b80 1 --2- 172.21.15.52:0/4293040614 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d867832ed0 0x55d86783b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:50:36.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.170+0000 7f1d42470b80 1 --2- 172.21.15.52:0/4293040614 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d8678385f0 0x55d86783be30 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:50:36.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.170+0000 7f1d42470b80 1 -- 172.21.15.52:0/4293040614 >> 172.21.15.52:0/4293040614 conn(0x55d86775dc70 msgr2=0x55d8678393b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:50:36.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.170+0000 7f1d42470b80 1 -- 172.21.15.52:0/4293040614 shutdown_connections 2024-06-10T02:50:36.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.171+0000 7f1d42470b80 1 -- 172.21.15.52:0/4293040614 wait complete. 2024-06-10T02:50:36.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.171+0000 7f1d42470b80 20 client.14697 trim_cache size 0 max 16384 2024-06-10T02:50:36.175 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-10T02:50:36.175 INFO:teuthology.orchestra.run.smithi052.stdout: "cluster_fsid": "d8676e9e-26d2-11ef-bc9b-c7b262605968", 2024-06-10T02:50:36.175 INFO:teuthology.orchestra.run.smithi052.stdout: "whoami": 0, 2024-06-10T02:50:36.175 INFO:teuthology.orchestra.run.smithi052.stdout: "id": 24484, 2024-06-10T02:50:36.175 INFO:teuthology.orchestra.run.smithi052.stdout: "want_state": "up:active", 2024-06-10T02:50:36.175 INFO:teuthology.orchestra.run.smithi052.stdout: "state": "up:active", 2024-06-10T02:50:36.175 INFO:teuthology.orchestra.run.smithi052.stdout: "fs_name": "cephfs", 2024-06-10T02:50:36.175 INFO:teuthology.orchestra.run.smithi052.stdout: "rank_uptime": 31.976433896, 2024-06-10T02:50:36.175 INFO:teuthology.orchestra.run.smithi052.stdout: "mdsmap_epoch": 17, 2024-06-10T02:50:36.176 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_epoch": 96, 2024-06-10T02:50:36.176 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_epoch_barrier": 96, 2024-06-10T02:50:36.176 INFO:teuthology.orchestra.run.smithi052.stdout: "uptime": 59.989849759000002 2024-06-10T02:50:36.176 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-10T02:50:36.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.171+0000 7f1d37bed700 1 -- 172.21.15.52:0/1004805548 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f1d1c0a2c20 msgr2=0x7f1d1c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:50:36.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.171+0000 7f1d37bed700 1 --2- 172.21.15.52:0/1004805548 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f1d1c0a2c20 0x7f1d1c0a50e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f1d2000b5c0 tx=0x7f1d20005960 comp rx=0 tx=0).stop 2024-06-10T02:50:36.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.171+0000 7f1d37bed700 1 -- 172.21.15.52:0/1004805548 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1d3006ac00 msgr2=0x7f1d30198d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:50:36.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.171+0000 7f1d37bed700 1 --2- 172.21.15.52:0/1004805548 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1d3006ac00 0x7f1d30198d90 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f1d2c00ba50 tx=0x7f1d2c00bd60 comp rx=0 tx=0).stop 2024-06-10T02:50:36.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.171+0000 7f1d37bed700 1 -- 172.21.15.52:0/1004805548 shutdown_connections 2024-06-10T02:50:36.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.171+0000 7f1d37bed700 1 --2- 172.21.15.52:0/1004805548 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1d30108530 0x7f1d3019d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:50:36.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.171+0000 7f1d37bed700 1 --2- 172.21.15.52:0/1004805548 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f1d1c0a2c20 0x7f1d1c0a50e0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:50:36.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.171+0000 7f1d37bed700 1 --2- 172.21.15.52:0/1004805548 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1d3006ac00 0x7f1d30198d90 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:50:36.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.171+0000 7f1d37bed700 1 --2- 172.21.15.52:0/1004805548 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1d3006b5c0 0x7f1d301992d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:50:36.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.171+0000 7f1d37bed700 1 -- 172.21.15.52:0/1004805548 >> 172.21.15.52:0/1004805548 conn(0x7f1d30077400 msgr2=0x7f1d30100270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:50:36.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.171+0000 7f1d37bed700 1 -- 172.21.15.52:0/1004805548 shutdown_connections 2024-06-10T02:50:36.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:36.171+0000 7f1d37bed700 1 -- 172.21.15.52:0/1004805548 wait complete. 2024-06-10T02:50:36.869 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:50:36 smithi052 ceph-mon[32362]: pgmap v322: 129 pgs: 129 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T02:50:36.900 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-06-10T02:50:36.900 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-06-10T02:50:36.900 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph fs ls 2024-06-10T02:50:37.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:50:36 smithi120 ceph-mon[36634]: pgmap v322: 129 pgs: 129 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T02:50:37.154 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:50:37.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:50:36 smithi188 ceph-mon[36599]: pgmap v322: 129 pgs: 129 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T02:50:38.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:38.834+0000 7fd1da7d4700 1 -- 172.21.15.52:0/3795062906 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd1d4107830 msgr2=0x7fd1d410f3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:50:38.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:38.834+0000 7fd1da7d4700 1 --2- 172.21.15.52:0/3795062906 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd1d4107830 0x7fd1d410f3b0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fd1cc00b0d0 tx=0x7fd1cc00b3e0 comp rx=0 tx=0).stop 2024-06-10T02:50:38.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:38.834+0000 7fd1da7d4700 1 -- 172.21.15.52:0/3795062906 shutdown_connections 2024-06-10T02:50:38.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:38.834+0000 7fd1da7d4700 1 --2- 172.21.15.52:0/3795062906 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd1d4107830 0x7fd1d410f3b0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:50:38.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:38.834+0000 7fd1da7d4700 1 --2- 172.21.15.52:0/3795062906 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd1d4102740 0x7fd1d4102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:50:38.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:38.834+0000 7fd1da7d4700 1 --2- 172.21.15.52:0/3795062906 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd1d41030f0 0x7fd1d4107100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:50:38.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:38.834+0000 7fd1da7d4700 1 -- 172.21.15.52:0/3795062906 >> 172.21.15.52:0/3795062906 conn(0x7fd1d40795b0 msgr2=0x7fd1d40799c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:50:38.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:38.834+0000 7fd1da7d4700 1 -- 172.21.15.52:0/3795062906 shutdown_connections 2024-06-10T02:50:38.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:38.834+0000 7fd1da7d4700 1 -- 172.21.15.52:0/3795062906 wait complete. 2024-06-10T02:50:38.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:38.835+0000 7fd1da7d4700 1 Processor -- start 2024-06-10T02:50:38.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:38.835+0000 7fd1da7d4700 1 -- start start 2024-06-10T02:50:38.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:38.835+0000 7fd1da7d4700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd1d4102740 0x7fd1d4074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:50:38.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:38.835+0000 7fd1da7d4700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd1d41030f0 0x7fd1d4074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:50:38.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:38.835+0000 7fd1da7d4700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd1d4107830 0x7fd1d4075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:50:38.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:38.835+0000 7fd1da7d4700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fd1d4112750 con 0x7fd1d4102740 2024-06-10T02:50:38.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:38.835+0000 7fd1da7d4700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd1d41125d0 con 0x7fd1d41030f0 2024-06-10T02:50:38.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:38.835+0000 7fd1da7d4700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fd1d41128d0 con 0x7fd1d4107830 2024-06-10T02:50:38.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:38.835+0000 7fd1d3fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd1d4102740 0x7fd1d4074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:50:38.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:38.835+0000 7fd1d3fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd1d4102740 0x7fd1d4074790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:41176/0 (socket says 172.21.15.52:41176) 2024-06-10T02:50:38.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:38.836+0000 7fd1d3fff700 1 -- 172.21.15.52:0/3803806072 learned_addr learned my addr 172.21.15.52:0/3803806072 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:50:38.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:38.836+0000 7fd1d8d71700 1 --2- 172.21.15.52:0/3803806072 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd1d4107830 0x7fd1d4075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:50:38.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:38.836+0000 7fd1d37fe700 1 --2- 172.21.15.52:0/3803806072 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd1d41030f0 0x7fd1d4074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:50:38.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:38.836+0000 7fd1d3fff700 1 -- 172.21.15.52:0/3803806072 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd1d4107830 msgr2=0x7fd1d4075210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:50:38.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:38.836+0000 7fd1d3fff700 1 --2- 172.21.15.52:0/3803806072 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd1d4107830 0x7fd1d4075210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:50:38.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:38.836+0000 7fd1d3fff700 1 -- 172.21.15.52:0/3803806072 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd1d41030f0 msgr2=0x7fd1d4074cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:50:38.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:38.836+0000 7fd1d3fff700 1 --2- 172.21.15.52:0/3803806072 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd1d41030f0 0x7fd1d4074cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:50:38.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:38.836+0000 7fd1d3fff700 1 -- 172.21.15.52:0/3803806072 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1d41a6f10 con 0x7fd1d4102740 2024-06-10T02:50:38.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:38.836+0000 7fd1d3fff700 1 --2- 172.21.15.52:0/3803806072 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd1d4102740 0x7fd1d4074790 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fd1c400c9b0 tx=0x7fd1c400ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:50:38.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:38.836+0000 7fd1d17fa700 1 -- 172.21.15.52:0/3803806072 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1c4007b00 con 0x7fd1d4102740 2024-06-10T02:50:38.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:38.836+0000 7fd1d17fa700 1 -- 172.21.15.52:0/3803806072 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1c4007ca0 con 0x7fd1d4102740 2024-06-10T02:50:38.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:38.836+0000 7fd1da7d4700 1 -- 172.21.15.52:0/3803806072 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1d41a7200 con 0x7fd1d4102740 2024-06-10T02:50:38.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:38.836+0000 7fd1d17fa700 1 -- 172.21.15.52:0/3803806072 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1c4018730 con 0x7fd1d4102740 2024-06-10T02:50:38.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:38.836+0000 7fd1da7d4700 1 -- 172.21.15.52:0/3803806072 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1d41a7740 con 0x7fd1d4102740 2024-06-10T02:50:38.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:38.837+0000 7fd1da7d4700 1 -- 172.21.15.52:0/3803806072 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1d4069180 con 0x7fd1d4102740 2024-06-10T02:50:38.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:38.839+0000 7fd1d17fa700 1 -- 172.21.15.52:0/3803806072 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fd1c40188d0 con 0x7fd1d4102740 2024-06-10T02:50:38.838 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:38.840+0000 7fd1d17fa700 1 --2- 172.21.15.52:0/3803806072 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fd1c00a2c20 0x7fd1c00a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:50:38.838 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:38.840+0000 7fd1d37fe700 1 --2- 172.21.15.52:0/3803806072 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fd1c00a2c20 0x7fd1c00a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:50:38.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:38.840+0000 7fd1d17fa700 1 -- 172.21.15.52:0/3803806072 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8802+0+0 (secure 0 0 0) 0x7fd1c40cf590 con 0x7fd1d4102740 2024-06-10T02:50:38.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:38.840+0000 7fd1d37fe700 1 --2- 172.21.15.52:0/3803806072 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fd1c00a2c20 0x7fd1c00a50e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fd1bc005fd0 tx=0x7fd1bc00af20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:50:38.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:38.844+0000 7fd1d17fa700 1 -- 172.21.15.52:0/3803806072 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fd1c4014030 con 0x7fd1d4102740 2024-06-10T02:50:38.843 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:50:38 smithi052 ceph-mon[32362]: pgmap v323: 129 pgs: 129 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T02:50:39.095 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:39.096+0000 7fd1da7d4700 1 -- 172.21.15.52:0/3803806072 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fd1d41a7b60 con 0x7fd1d4102740 2024-06-10T02:50:39.095 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:39.096+0000 7fd1d17fa700 1 -- 172.21.15.52:0/3803806072 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) ==== 53+0+88 (secure 0 0 0) 0x7fd1c4017360 con 0x7fd1d4102740 2024-06-10T02:50:39.095 INFO:teuthology.orchestra.run.smithi052.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-06-10T02:50:39.097 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:39.098+0000 7fd1da7d4700 1 -- 172.21.15.52:0/3803806072 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fd1c00a2c20 msgr2=0x7fd1c00a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:50:39.097 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:39.098+0000 7fd1da7d4700 1 --2- 172.21.15.52:0/3803806072 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fd1c00a2c20 0x7fd1c00a50e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fd1bc005fd0 tx=0x7fd1bc00af20 comp rx=0 tx=0).stop 2024-06-10T02:50:39.097 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:39.098+0000 7fd1da7d4700 1 -- 172.21.15.52:0/3803806072 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd1d4102740 msgr2=0x7fd1d4074790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:50:39.097 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:39.098+0000 7fd1da7d4700 1 --2- 172.21.15.52:0/3803806072 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd1d4102740 0x7fd1d4074790 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fd1c400c9b0 tx=0x7fd1c400ccc0 comp rx=0 tx=0).stop 2024-06-10T02:50:39.097 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:39.099+0000 7fd1da7d4700 1 -- 172.21.15.52:0/3803806072 shutdown_connections 2024-06-10T02:50:39.097 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:39.099+0000 7fd1da7d4700 1 --2- 172.21.15.52:0/3803806072 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd1d4107830 0x7fd1d4075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:50:39.097 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:39.099+0000 7fd1da7d4700 1 --2- 172.21.15.52:0/3803806072 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fd1c00a2c20 0x7fd1c00a50e0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:50:39.098 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:39.099+0000 7fd1da7d4700 1 --2- 172.21.15.52:0/3803806072 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd1d4102740 0x7fd1d4074790 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:50:39.098 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:39.099+0000 7fd1da7d4700 1 --2- 172.21.15.52:0/3803806072 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd1d41030f0 0x7fd1d4074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:50:39.098 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:39.099+0000 7fd1da7d4700 1 -- 172.21.15.52:0/3803806072 >> 172.21.15.52:0/3803806072 conn(0x7fd1d40795b0 msgr2=0x7fd1d4100350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:50:39.098 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:39.099+0000 7fd1da7d4700 1 -- 172.21.15.52:0/3803806072 shutdown_connections 2024-06-10T02:50:39.098 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:39.099+0000 7fd1da7d4700 1 -- 172.21.15.52:0/3803806072 wait complete. 2024-06-10T02:50:39.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:50:38 smithi120 ceph-mon[36634]: pgmap v323: 129 pgs: 129 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T02:50:39.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:50:38 smithi188 ceph-mon[36599]: pgmap v323: 129 pgs: 129 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T02:50:39.750 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-06-10T02:50:39.750 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-06-10T02:50:39.750 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi052.front.sepia.ceph.com 2024-06-10T02:50:39.750 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-06-10T02:50:39.750 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-06-10T02:50:39.750 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-06-10T02:50:39.751 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-06-10T02:50:39.751 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-06-10T02:50:39.751 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-06-10T02:50:39.751 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-10T02:50:39.751 DEBUG:teuthology.orchestra.run.smithi052:> ip addr 2024-06-10T02:50:39.768 INFO:teuthology.orchestra.run.smithi052.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-06-10T02:50:39.768 INFO:teuthology.orchestra.run.smithi052.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-06-10T02:50:39.768 INFO:teuthology.orchestra.run.smithi052.stdout: inet 127.0.0.1/8 scope host lo 2024-06-10T02:50:39.768 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-06-10T02:50:39.768 INFO:teuthology.orchestra.run.smithi052.stdout: inet6 ::1/128 scope host 2024-06-10T02:50:39.769 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-06-10T02:50:39.769 INFO:teuthology.orchestra.run.smithi052.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-06-10T02:50:39.769 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 0c:c4:7a:6b:fd:d2 brd ff:ff:ff:ff:ff:ff 2024-06-10T02:50:39.769 INFO:teuthology.orchestra.run.smithi052.stdout: altname enp7s0f0 2024-06-10T02:50:39.769 INFO:teuthology.orchestra.run.smithi052.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-06-10T02:50:39.769 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 0c:c4:7a:6b:fd:d3 brd ff:ff:ff:ff:ff:ff 2024-06-10T02:50:39.769 INFO:teuthology.orchestra.run.smithi052.stdout: altname enp7s0f1 2024-06-10T02:50:39.769 INFO:teuthology.orchestra.run.smithi052.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-06-10T02:50:39.769 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 0c:c4:7a:bd:15:de brd ff:ff:ff:ff:ff:ff 2024-06-10T02:50:39.769 INFO:teuthology.orchestra.run.smithi052.stdout: altname enp1s0f0 2024-06-10T02:50:39.769 INFO:teuthology.orchestra.run.smithi052.stdout: inet 172.21.15.52/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-06-10T02:50:39.769 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft 41903sec preferred_lft 41903sec 2024-06-10T02:50:39.769 INFO:teuthology.orchestra.run.smithi052.stdout: inet6 fe80::ec4:7aff:febd:15de/64 scope link noprefixroute 2024-06-10T02:50:39.770 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-06-10T02:50:39.770 INFO:teuthology.orchestra.run.smithi052.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-06-10T02:50:39.770 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 0c:c4:7a:bd:15:df brd ff:ff:ff:ff:ff:ff 2024-06-10T02:50:39.770 INFO:teuthology.orchestra.run.smithi052.stdout: altname enp1s0f1 2024-06-10T02:50:39.770 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-06-10T02:50:39.771 DEBUG:teuthology.orchestra.run.smithi052:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-06-10T02:50:39.771 DEBUG:teuthology.orchestra.run.smithi052:> set -e 2024-06-10T02:50:39.771 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip link add name ceph-brx type bridge 2024-06-10T02:50:39.771 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip addr flush dev ceph-brx 2024-06-10T02:50:39.771 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip link set ceph-brx up 2024-06-10T02:50:39.771 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-06-10T02:50:39.771 DEBUG:teuthology.orchestra.run.smithi052:> ') 2024-06-10T02:50:39.859 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:39 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-06-10T02:50:39.975 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:50:39 smithi052 ceph-mon[32362]: from='client.? 172.21.15.52:0/3803806072' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-06-10T02:50:39.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-10T02:50:39.986 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-10T02:50:39.986 DEBUG:teuthology.orchestra.run.smithi052:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-06-10T02:50:40.013 INFO:teuthology.orchestra.run.smithi052.stdout:1 2024-06-10T02:50:40.014 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-10T02:50:40.014 DEBUG:teuthology.orchestra.run.smithi052:> route 2024-06-10T02:50:40.089 INFO:teuthology.orchestra.run.smithi052.stdout:Kernel IP routing table 2024-06-10T02:50:40.089 INFO:teuthology.orchestra.run.smithi052.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-06-10T02:50:40.089 INFO:teuthology.orchestra.run.smithi052.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-06-10T02:50:40.089 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-06-10T02:50:40.089 INFO:teuthology.orchestra.run.smithi052.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-06-10T02:50:40.090 DEBUG:teuthology.orchestra.run.smithi052:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-06-10T02:50:40.090 DEBUG:teuthology.orchestra.run.smithi052:> set -e 2024-06-10T02:50:40.090 DEBUG:teuthology.orchestra.run.smithi052:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-06-10T02:50:40.090 DEBUG:teuthology.orchestra.run.smithi052:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-06-10T02:50:40.090 DEBUG:teuthology.orchestra.run.smithi052:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-06-10T02:50:40.090 DEBUG:teuthology.orchestra.run.smithi052:> ') 2024-06-10T02:50:40.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:50:39 smithi120 ceph-mon[36634]: from='client.? 172.21.15.52:0/3803806072' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-06-10T02:50:40.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:40 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-06-10T02:50:40.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:50:39 smithi188 ceph-mon[36599]: from='client.? 172.21.15.52:0/3803806072' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-06-10T02:50:40.421 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-10T02:50:40.427 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-10T02:50:40.427 DEBUG:teuthology.orchestra.run.smithi052:> ip netns list 2024-06-10T02:50:40.444 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-10T02:50:40.444 DEBUG:teuthology.orchestra.run.smithi052:> ip netns list-id 2024-06-10T02:50:40.499 DEBUG:teuthology.orchestra.run.smithi052:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-06-10T02:50:40.499 DEBUG:teuthology.orchestra.run.smithi052:> set -e 2024-06-10T02:50:40.500 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-06-10T02:50:40.500 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-06-10T02:50:40.500 DEBUG:teuthology.orchestra.run.smithi052:> ') 2024-06-10T02:50:40.589 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:40 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-06-10T02:50:40.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-10T02:50:40.621 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-06-10T02:50:40.621 DEBUG:teuthology.orchestra.run.smithi052:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-06-10T02:50:40.621 DEBUG:teuthology.orchestra.run.smithi052:> set -e 2024-06-10T02:50:40.621 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-06-10T02:50:40.621 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-06-10T02:50:40.621 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-06-10T02:50:40.621 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-06-10T02:50:40.622 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-06-10T02:50:40.622 DEBUG:teuthology.orchestra.run.smithi052:> ') 2024-06-10T02:50:40.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:40 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-06-10T02:50:40.749 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-10T02:50:40.755 DEBUG:teuthology.orchestra.run.smithi052:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-06-10T02:50:40.756 DEBUG:teuthology.orchestra.run.smithi052:> set -e 2024-06-10T02:50:40.756 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip link set brx.0 up 2024-06-10T02:50:40.756 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip link set dev brx.0 master ceph-brx 2024-06-10T02:50:40.756 DEBUG:teuthology.orchestra.run.smithi052:> ') 2024-06-10T02:50:40.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:40 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-06-10T02:50:40.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-10T02:50:40.836 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-06-10T02:50:40.837 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-06-10T02:50:40.837 DEBUG:teuthology.orchestra.run.smithi052:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-06-10T02:50:40.872 INFO:teuthology.orchestra.run.smithi052.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-06-10T02:50:40.872 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-06-10T02:50:40.872 DEBUG:teuthology.orchestra.run.smithi052:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-06-10T02:50:40.928 DEBUG:teuthology.orchestra.run.smithi052:> sudo modprobe fuse 2024-06-10T02:50:40.975 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:50:40 smithi052 ceph-mon[32362]: pgmap v324: 129 pgs: 129 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T02:50:40.995 DEBUG:teuthology.orchestra.run.smithi052:> cat /proc/self/mounts | awk '{print $2}' 2024-06-10T02:50:41.052 INFO:teuthology.orchestra.run.smithi052.stdout:/proc 2024-06-10T02:50:41.052 INFO:teuthology.orchestra.run.smithi052.stdout:/sys 2024-06-10T02:50:41.052 INFO:teuthology.orchestra.run.smithi052.stdout:/dev 2024-06-10T02:50:41.053 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/kernel/security 2024-06-10T02:50:41.053 INFO:teuthology.orchestra.run.smithi052.stdout:/dev/shm 2024-06-10T02:50:41.053 INFO:teuthology.orchestra.run.smithi052.stdout:/dev/pts 2024-06-10T02:50:41.053 INFO:teuthology.orchestra.run.smithi052.stdout:/run 2024-06-10T02:50:41.053 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/fs/cgroup 2024-06-10T02:50:41.053 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/fs/pstore 2024-06-10T02:50:41.053 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/fs/bpf 2024-06-10T02:50:41.053 INFO:teuthology.orchestra.run.smithi052.stdout:/ 2024-06-10T02:50:41.053 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/fs/selinux 2024-06-10T02:50:41.053 INFO:teuthology.orchestra.run.smithi052.stdout:/proc/sys/fs/binfmt_misc 2024-06-10T02:50:41.053 INFO:teuthology.orchestra.run.smithi052.stdout:/dev/mqueue 2024-06-10T02:50:41.054 INFO:teuthology.orchestra.run.smithi052.stdout:/dev/hugepages 2024-06-10T02:50:41.054 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/kernel/debug 2024-06-10T02:50:41.054 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/kernel/tracing 2024-06-10T02:50:41.054 INFO:teuthology.orchestra.run.smithi052.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-06-10T02:50:41.054 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/fs/fuse/connections 2024-06-10T02:50:41.054 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/kernel/config 2024-06-10T02:50:41.054 INFO:teuthology.orchestra.run.smithi052.stdout:/run/credentials/systemd-sysctl.service 2024-06-10T02:50:41.054 INFO:teuthology.orchestra.run.smithi052.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-06-10T02:50:41.054 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/nfs/rpc_pipefs 2024-06-10T02:50:41.054 INFO:teuthology.orchestra.run.smithi052.stdout:/run/user/1000 2024-06-10T02:50:41.054 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/ceph 2024-06-10T02:50:41.055 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/containers/storage/overlay 2024-06-10T02:50:41.055 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/containers/storage/overlay/41fe9f93e7f1169c7e6f8e42736722911cb4842b89a4809d887b92197f4c3515/merged 2024-06-10T02:50:41.055 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/containers/storage/overlay/d8fb14a84688ca8f78aa1c666d2c7e2c49c8bb3d7523a67dfbdf9cf1d6a6116c/merged 2024-06-10T02:50:41.055 INFO:teuthology.orchestra.run.smithi052.stdout:/run/user/0 2024-06-10T02:50:41.055 INFO:teuthology.orchestra.run.smithi052.stdout:/proc/sys/fs/binfmt_misc 2024-06-10T02:50:41.055 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/containers/storage/overlay/a3a7f4ebf3b9d902f81d865d4471b7922d17ccb96d7523a1116d7eb53898d3d4/merged 2024-06-10T02:50:41.055 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/containers/storage/overlay/9219000bd93396de76fe5bfb65e5ed55d243a736e3a779dfd2af17226da482be/merged 2024-06-10T02:50:41.055 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/containers/storage/overlay/ea3074f280ac3905d0d084c7fff892981d5c4962647e61e1affb6565ee257103/merged 2024-06-10T02:50:41.055 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/containers/storage/overlay/b4537c1c43b5b6119af83535e010fe28df60e256dc36f798b8e071649be5432a/merged 2024-06-10T02:50:41.055 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/containers/storage/overlay/63c7f9bf743691dbb6520a39cdadd5a77f6ca8cdd52c8dbd92b4aabdd84c2e8a/merged 2024-06-10T02:50:41.055 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/containers/storage/overlay/5857c3fbd1028d06a9f3533df68ef16c5824b5b0a723c4d19ebed3620f220a2f/merged 2024-06-10T02:50:41.056 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/containers/storage/overlay/b3679ea55cf13c8b4017bec9162e2f4a547a6434a24f97633b5253ce8d37065c/merged 2024-06-10T02:50:41.056 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/containers/storage/overlay/77bcfdc28bc1145354214314f8e58a2cf38bb33ffa2c2e2ec474ab47e968405a/merged 2024-06-10T02:50:41.056 INFO:teuthology.orchestra.run.smithi052.stdout:/run/netns 2024-06-10T02:50:41.056 INFO:teuthology.orchestra.run.smithi052.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-06-10T02:50:41.056 INFO:teuthology.orchestra.run.smithi052.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-06-10T02:50:41.056 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-10T02:50:41.056 DEBUG:teuthology.orchestra.run.smithi052:> ls /sys/fs/fuse/connections 2024-06-10T02:50:41.108 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-06-10T02:50:41.108 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-06-10T02:50:41.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:50:40 smithi120 ceph-mon[36634]: pgmap v324: 129 pgs: 129 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T02:50:41.151 DEBUG:teuthology.orchestra.run.smithi052:> sudo modprobe fuse 2024-06-10T02:50:41.176 DEBUG:teuthology.orchestra.run.smithi052:> cat /proc/self/mounts | awk '{print $2}' 2024-06-10T02:50:41.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:50:40 smithi188 ceph-mon[36599]: pgmap v324: 129 pgs: 129 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T02:50:41.234 INFO:teuthology.orchestra.run.smithi052.stdout:/proc 2024-06-10T02:50:41.234 INFO:teuthology.orchestra.run.smithi052.stdout:/sys 2024-06-10T02:50:41.234 INFO:teuthology.orchestra.run.smithi052.stdout:/dev 2024-06-10T02:50:41.235 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/kernel/security 2024-06-10T02:50:41.235 INFO:teuthology.orchestra.run.smithi052.stdout:/dev/shm 2024-06-10T02:50:41.235 INFO:teuthology.orchestra.run.smithi052.stdout:/dev/pts 2024-06-10T02:50:41.235 INFO:teuthology.orchestra.run.smithi052.stdout:/run 2024-06-10T02:50:41.235 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/fs/cgroup 2024-06-10T02:50:41.235 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/fs/pstore 2024-06-10T02:50:41.235 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/fs/bpf 2024-06-10T02:50:41.235 INFO:teuthology.orchestra.run.smithi052.stdout:/ 2024-06-10T02:50:41.235 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/fs/selinux 2024-06-10T02:50:41.235 INFO:teuthology.orchestra.run.smithi052.stdout:/proc/sys/fs/binfmt_misc 2024-06-10T02:50:41.235 INFO:teuthology.orchestra.run.smithi052.stdout:/dev/mqueue 2024-06-10T02:50:41.235 INFO:teuthology.orchestra.run.smithi052.stdout:/dev/hugepages 2024-06-10T02:50:41.235 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/kernel/debug 2024-06-10T02:50:41.236 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/kernel/tracing 2024-06-10T02:50:41.236 INFO:teuthology.orchestra.run.smithi052.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-06-10T02:50:41.236 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/fs/fuse/connections 2024-06-10T02:50:41.236 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/kernel/config 2024-06-10T02:50:41.236 INFO:teuthology.orchestra.run.smithi052.stdout:/run/credentials/systemd-sysctl.service 2024-06-10T02:50:41.236 INFO:teuthology.orchestra.run.smithi052.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-06-10T02:50:41.236 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/nfs/rpc_pipefs 2024-06-10T02:50:41.236 INFO:teuthology.orchestra.run.smithi052.stdout:/run/user/1000 2024-06-10T02:50:41.236 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/ceph 2024-06-10T02:50:41.236 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/containers/storage/overlay 2024-06-10T02:50:41.236 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/containers/storage/overlay/41fe9f93e7f1169c7e6f8e42736722911cb4842b89a4809d887b92197f4c3515/merged 2024-06-10T02:50:41.236 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/containers/storage/overlay/d8fb14a84688ca8f78aa1c666d2c7e2c49c8bb3d7523a67dfbdf9cf1d6a6116c/merged 2024-06-10T02:50:41.236 INFO:teuthology.orchestra.run.smithi052.stdout:/run/user/0 2024-06-10T02:50:41.236 INFO:teuthology.orchestra.run.smithi052.stdout:/proc/sys/fs/binfmt_misc 2024-06-10T02:50:41.237 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/containers/storage/overlay/a3a7f4ebf3b9d902f81d865d4471b7922d17ccb96d7523a1116d7eb53898d3d4/merged 2024-06-10T02:50:41.237 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/containers/storage/overlay/9219000bd93396de76fe5bfb65e5ed55d243a736e3a779dfd2af17226da482be/merged 2024-06-10T02:50:41.237 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/containers/storage/overlay/ea3074f280ac3905d0d084c7fff892981d5c4962647e61e1affb6565ee257103/merged 2024-06-10T02:50:41.237 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/containers/storage/overlay/b4537c1c43b5b6119af83535e010fe28df60e256dc36f798b8e071649be5432a/merged 2024-06-10T02:50:41.237 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/containers/storage/overlay/63c7f9bf743691dbb6520a39cdadd5a77f6ca8cdd52c8dbd92b4aabdd84c2e8a/merged 2024-06-10T02:50:41.237 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/containers/storage/overlay/5857c3fbd1028d06a9f3533df68ef16c5824b5b0a723c4d19ebed3620f220a2f/merged 2024-06-10T02:50:41.237 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/containers/storage/overlay/b3679ea55cf13c8b4017bec9162e2f4a547a6434a24f97633b5253ce8d37065c/merged 2024-06-10T02:50:41.237 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/containers/storage/overlay/77bcfdc28bc1145354214314f8e58a2cf38bb33ffa2c2e2ec474ab47e968405a/merged 2024-06-10T02:50:41.237 INFO:teuthology.orchestra.run.smithi052.stdout:/run/netns 2024-06-10T02:50:41.237 INFO:teuthology.orchestra.run.smithi052.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-06-10T02:50:41.237 INFO:teuthology.orchestra.run.smithi052.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-06-10T02:50:41.238 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-10T02:50:41.238 DEBUG:teuthology.orchestra.run.smithi052:> ls /sys/fs/fuse/connections 2024-06-10T02:50:41.268 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi052.stderr:2024-06-10T02:50:41.268+0000 7f305c509480 -1 init, newargv = 0x55ea8b9e2950 newargc=15 2024-06-10T02:50:41.268 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi052.stderr:ceph-fuse[78631]: starting ceph client 2024-06-10T02:50:41.274 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi052.stderr:ceph-fuse[78631]: starting fuse 2024-06-10T02:50:41.284 INFO:teuthology.orchestra.run.smithi052.stdout:119 2024-06-10T02:50:41.285 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [119] 2024-06-10T02:50:41.285 DEBUG:teuthology.orchestra.run.smithi052:> sudo stdin-killer -- python3 -c ' 2024-06-10T02:50:41.285 DEBUG:teuthology.orchestra.run.smithi052:> import glob 2024-06-10T02:50:41.285 DEBUG:teuthology.orchestra.run.smithi052:> import re 2024-06-10T02:50:41.285 DEBUG:teuthology.orchestra.run.smithi052:> import os 2024-06-10T02:50:41.285 DEBUG:teuthology.orchestra.run.smithi052:> import subprocess 2024-06-10T02:50:41.285 DEBUG:teuthology.orchestra.run.smithi052:> 2024-06-10T02:50:41.285 DEBUG:teuthology.orchestra.run.smithi052:> def _find_admin_socket(client_name): 2024-06-10T02:50:41.285 DEBUG:teuthology.orchestra.run.smithi052:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-06-10T02:50:41.286 DEBUG:teuthology.orchestra.run.smithi052:> files = glob.glob(asok_path) 2024-06-10T02:50:41.286 DEBUG:teuthology.orchestra.run.smithi052:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-06-10T02:50:41.286 DEBUG:teuthology.orchestra.run.smithi052:> 2024-06-10T02:50:41.286 DEBUG:teuthology.orchestra.run.smithi052:> # Given a non-glob path, it better be there 2024-06-10T02:50:41.286 DEBUG:teuthology.orchestra.run.smithi052:> if "*" not in asok_path: 2024-06-10T02:50:41.286 DEBUG:teuthology.orchestra.run.smithi052:> assert(len(files) == 1) 2024-06-10T02:50:41.286 DEBUG:teuthology.orchestra.run.smithi052:> return files[0] 2024-06-10T02:50:41.286 DEBUG:teuthology.orchestra.run.smithi052:> 2024-06-10T02:50:41.286 DEBUG:teuthology.orchestra.run.smithi052:> for f in files: 2024-06-10T02:50:41.286 DEBUG:teuthology.orchestra.run.smithi052:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-06-10T02:50:41.286 DEBUG:teuthology.orchestra.run.smithi052:> if os.path.exists("/proc/{0}".format(pid)): 2024-06-10T02:50:41.286 DEBUG:teuthology.orchestra.run.smithi052:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-06-10T02:50:41.286 DEBUG:teuthology.orchestra.run.smithi052:> contents = proc_f.read() 2024-06-10T02:50:41.287 DEBUG:teuthology.orchestra.run.smithi052:> if mountpoint in contents: 2024-06-10T02:50:41.287 DEBUG:teuthology.orchestra.run.smithi052:> return f 2024-06-10T02:50:41.287 DEBUG:teuthology.orchestra.run.smithi052:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-06-10T02:50:41.287 DEBUG:teuthology.orchestra.run.smithi052:> 2024-06-10T02:50:41.287 DEBUG:teuthology.orchestra.run.smithi052:> print(_find_admin_socket("client.admin")) 2024-06-10T02:50:41.287 DEBUG:teuthology.orchestra.run.smithi052:> ' 2024-06-10T02:50:41.407 INFO:teuthology.orchestra.run.smithi052.stdout:/var/run/ceph/ceph-client.admin.78631.asok 2024-06-10T02:50:41.410 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-10T02:50:41.417 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.78631.asok 2024-06-10T02:50:41.417 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-10T02:50:41.417 DEBUG:teuthology.orchestra.run.smithi052:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.78631.asok status 2024-06-10T02:50:41.556 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-10T02:50:41.556 INFO:teuthology.orchestra.run.smithi052.stdout: "metadata": { 2024-06-10T02:50:41.557 INFO:teuthology.orchestra.run.smithi052.stdout: "ceph_sha1": "3d89accfe3201286f000e065a1faea2a88c7a6ba", 2024-06-10T02:50:41.557 INFO:teuthology.orchestra.run.smithi052.stdout: "ceph_version": "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)", 2024-06-10T02:50:41.557 INFO:teuthology.orchestra.run.smithi052.stdout: "entity_id": "admin", 2024-06-10T02:50:41.557 INFO:teuthology.orchestra.run.smithi052.stdout: "hostname": "smithi052", 2024-06-10T02:50:41.557 INFO:teuthology.orchestra.run.smithi052.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-06-10T02:50:41.557 INFO:teuthology.orchestra.run.smithi052.stdout: "pid": "78631", 2024-06-10T02:50:41.557 INFO:teuthology.orchestra.run.smithi052.stdout: "root": "/" 2024-06-10T02:50:41.557 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T02:50:41.557 INFO:teuthology.orchestra.run.smithi052.stdout: "dentry_count": 0, 2024-06-10T02:50:41.557 INFO:teuthology.orchestra.run.smithi052.stdout: "dentry_pinned_count": 0, 2024-06-10T02:50:41.557 INFO:teuthology.orchestra.run.smithi052.stdout: "id": 14709, 2024-06-10T02:50:41.557 INFO:teuthology.orchestra.run.smithi052.stdout: "inst": { 2024-06-10T02:50:41.557 INFO:teuthology.orchestra.run.smithi052.stdout: "name": { 2024-06-10T02:50:41.557 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "client", 2024-06-10T02:50:41.558 INFO:teuthology.orchestra.run.smithi052.stdout: "num": 14709 2024-06-10T02:50:41.558 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T02:50:41.558 INFO:teuthology.orchestra.run.smithi052.stdout: "addr": { 2024-06-10T02:50:41.558 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "v1", 2024-06-10T02:50:41.558 INFO:teuthology.orchestra.run.smithi052.stdout: "addr": "192.168.0.1:0", 2024-06-10T02:50:41.558 INFO:teuthology.orchestra.run.smithi052.stdout: "nonce": 2163933450 2024-06-10T02:50:41.558 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T02:50:41.558 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T02:50:41.558 INFO:teuthology.orchestra.run.smithi052.stdout: "addr": { 2024-06-10T02:50:41.558 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "v1", 2024-06-10T02:50:41.558 INFO:teuthology.orchestra.run.smithi052.stdout: "addr": "192.168.0.1:0", 2024-06-10T02:50:41.558 INFO:teuthology.orchestra.run.smithi052.stdout: "nonce": 2163933450 2024-06-10T02:50:41.558 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T02:50:41.559 INFO:teuthology.orchestra.run.smithi052.stdout: "inst_str": "client.14709 192.168.0.1:0/2163933450", 2024-06-10T02:50:41.559 INFO:teuthology.orchestra.run.smithi052.stdout: "addr_str": "192.168.0.1:0/2163933450", 2024-06-10T02:50:41.559 INFO:teuthology.orchestra.run.smithi052.stdout: "inode_count": 1, 2024-06-10T02:50:41.559 INFO:teuthology.orchestra.run.smithi052.stdout: "mds_epoch": 17, 2024-06-10T02:50:41.559 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_epoch": 96, 2024-06-10T02:50:41.559 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_epoch_barrier": 0, 2024-06-10T02:50:41.559 INFO:teuthology.orchestra.run.smithi052.stdout: "blocklisted": false, 2024-06-10T02:50:41.559 INFO:teuthology.orchestra.run.smithi052.stdout: "fs_name": "cephfs" 2024-06-10T02:50:41.559 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-10T02:50:41.565 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-10T02:50:41.565 DEBUG:teuthology.orchestra.run.smithi052:> stat --file-system '--printf=%T 2024-06-10T02:50:41.565 DEBUG:teuthology.orchestra.run.smithi052:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-06-10T02:50:41.581 INFO:teuthology.orchestra.run.smithi052.stdout:fuseblk 2024-06-10T02:50:41.581 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-06-10T02:50:41.581 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-10T02:50:41.581 DEBUG:teuthology.orchestra.run.smithi052:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-06-10T02:50:41.649 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.admin && exec stdin-killer --timeout=900 -- bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-06-10T02:50:41.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:41 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-06-10T02:50:41.762 INFO:teuthology.orchestra.run.smithi052.stdout:# file: . 2024-06-10T02:50:41.762 INFO:teuthology.orchestra.run.smithi052.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-06-10T02:50:41.762 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-10T02:50:41.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-10T02:50:41.768 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-06-10T02:50:41.768 DEBUG:teuthology.orchestra.run.smithi052:> dd if=/proc/self/mounts of=/dev/stdout 2024-06-10T02:50:41.786 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph osd blocklist ls 2024-06-10T02:50:42.076 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:50:42.975 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:50:42 smithi052 ceph-mon[32362]: pgmap v325: 129 pgs: 129 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T02:50:43.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:50:42 smithi120 ceph-mon[36634]: pgmap v325: 129 pgs: 129 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T02:50:43.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:50:42 smithi188 ceph-mon[36599]: pgmap v325: 129 pgs: 129 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T02:50:43.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:43.776+0000 7f122aa8f700 1 -- 172.21.15.52:0/3580462977 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1224108500 msgr2=0x7f12241088e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:50:43.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:43.776+0000 7f122aa8f700 1 --2- 172.21.15.52:0/3580462977 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1224108500 0x7f12241088e0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f1218009990 tx=0x7f1218009ca0 comp rx=0 tx=0).stop 2024-06-10T02:50:43.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:43.776+0000 7f122aa8f700 1 -- 172.21.15.52:0/3580462977 shutdown_connections 2024-06-10T02:50:43.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:43.776+0000 7f122aa8f700 1 --2- 172.21.15.52:0/3580462977 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f122406b580 0x7f122410f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:50:43.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:43.776+0000 7f122aa8f700 1 --2- 172.21.15.52:0/3580462977 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1224108500 0x7f12241088e0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:50:43.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:43.776+0000 7f122aa8f700 1 --2- 172.21.15.52:0/3580462977 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f122406abc0 0x7f122406b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:50:43.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:43.776+0000 7f122aa8f700 1 -- 172.21.15.52:0/3580462977 >> 172.21.15.52:0/3580462977 conn(0x7f1224078340 msgr2=0x7f1224078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:50:43.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:43.776+0000 7f122aa8f700 1 -- 172.21.15.52:0/3580462977 shutdown_connections 2024-06-10T02:50:43.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:43.776+0000 7f122aa8f700 1 -- 172.21.15.52:0/3580462977 wait complete. 2024-06-10T02:50:43.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:43.777+0000 7f122aa8f700 1 Processor -- start 2024-06-10T02:50:43.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:43.777+0000 7f122aa8f700 1 -- start start 2024-06-10T02:50:43.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:43.777+0000 7f122aa8f700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f122406abc0 0x7f1224101490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:50:43.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:43.777+0000 7f122aa8f700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f122406b580 0x7f12241019d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:50:43.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:43.777+0000 7f122aa8f700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1224108500 0x7f1224101f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:50:43.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:43.777+0000 7f122aa8f700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f1224112780 con 0x7f122406abc0 2024-06-10T02:50:43.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:43.777+0000 7f122aa8f700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1224112600 con 0x7f1224108500 2024-06-10T02:50:43.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:43.777+0000 7f122aa8f700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f1224112900 con 0x7f122406b580 2024-06-10T02:50:43.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:43.777+0000 7f122882b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f122406abc0 0x7f1224101490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:50:43.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:43.777+0000 7f122882b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f122406abc0 0x7f1224101490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:45260/0 (socket says 172.21.15.52:45260) 2024-06-10T02:50:43.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:43.777+0000 7f122882b700 1 -- 172.21.15.52:0/1787269472 learned_addr learned my addr 172.21.15.52:0/1787269472 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:50:43.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:43.777+0000 7f1223fff700 1 --2- 172.21.15.52:0/1787269472 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f122406b580 0x7f12241019d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:50:43.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:43.777+0000 7f122902c700 1 --2- 172.21.15.52:0/1787269472 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1224108500 0x7f1224101f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:50:43.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:43.777+0000 7f122882b700 1 -- 172.21.15.52:0/1787269472 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f122406b580 msgr2=0x7f12241019d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:50:43.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:43.777+0000 7f122882b700 1 --2- 172.21.15.52:0/1787269472 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f122406b580 0x7f12241019d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:50:43.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:43.777+0000 7f122882b700 1 -- 172.21.15.52:0/1787269472 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1224108500 msgr2=0x7f1224101f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:50:43.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:43.777+0000 7f122882b700 1 --2- 172.21.15.52:0/1787269472 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1224108500 0x7f1224101f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:50:43.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:43.777+0000 7f122882b700 1 -- 172.21.15.52:0/1787269472 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12241a71a0 con 0x7f122406abc0 2024-06-10T02:50:43.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:43.778+0000 7f122882b700 1 --2- 172.21.15.52:0/1787269472 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f122406abc0 0x7f1224101490 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f1218000c00 tx=0x7f12180047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:50:43.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:43.778+0000 7f1221ffb700 1 -- 172.21.15.52:0/1787269472 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1218027070 con 0x7f122406abc0 2024-06-10T02:50:43.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:43.778+0000 7f122aa8f700 1 -- 172.21.15.52:0/1787269472 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12241a7430 con 0x7f122406abc0 2024-06-10T02:50:43.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:43.778+0000 7f1221ffb700 1 -- 172.21.15.52:0/1787269472 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1218003e40 con 0x7f122406abc0 2024-06-10T02:50:43.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:43.778+0000 7f1221ffb700 1 -- 172.21.15.52:0/1787269472 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1218022d10 con 0x7f122406abc0 2024-06-10T02:50:43.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:43.778+0000 7f122aa8f700 1 -- 172.21.15.52:0/1787269472 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12241a7890 con 0x7f122406abc0 2024-06-10T02:50:43.781 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:43.779+0000 7f122aa8f700 1 -- 172.21.15.52:0/1787269472 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f122410a9e0 con 0x7f122406abc0 2024-06-10T02:50:43.781 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:43.780+0000 7f1221ffb700 1 -- 172.21.15.52:0/1787269472 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f121802a070 con 0x7f122406abc0 2024-06-10T02:50:43.781 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:43.781+0000 7f1221ffb700 1 --2- 172.21.15.52:0/1787269472 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f12140a2c20 0x7f12140a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:50:43.781 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:43.781+0000 7f1223fff700 1 --2- 172.21.15.52:0/1787269472 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f12140a2c20 0x7f12140a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:50:43.781 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:43.781+0000 7f1221ffb700 1 -- 172.21.15.52:0/1787269472 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9092+0+0 (secure 0 0 0) 0x7f12180d01d0 con 0x7f122406abc0 2024-06-10T02:50:43.782 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:43.781+0000 7f1223fff700 1 --2- 172.21.15.52:0/1787269472 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f12140a2c20 0x7f12140a50e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f12240789b0 tx=0x7f121000b3c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:50:43.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:43.786+0000 7f1221ffb700 1 -- 172.21.15.52:0/1787269472 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f121805e850 con 0x7f122406abc0 2024-06-10T02:50:43.975 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:50:43 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]: dispatch 2024-06-10T02:50:43.976 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:50:43 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]: dispatch 2024-06-10T02:50:43.976 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:50:43 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 4]}]: dispatch 2024-06-10T02:50:43.976 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:50:43 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [3, 7]}]: dispatch 2024-06-10T02:50:43.976 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:50:43 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]: dispatch 2024-06-10T02:50:43.976 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:50:43 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]: dispatch 2024-06-10T02:50:43.976 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:50:43 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]: dispatch 2024-06-10T02:50:43.976 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:50:43 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]}]: dispatch 2024-06-10T02:50:43.976 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:50:43 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]: dispatch 2024-06-10T02:50:43.976 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:50:43 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [9, 11]}]: dispatch 2024-06-10T02:50:44.011 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:44.011+0000 7f122aa8f700 1 -- 172.21.15.52:0/1787269472 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f12241029c0 con 0x7f122406abc0 2024-06-10T02:50:44.011 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:44.011+0000 7f1221ffb700 1 -- 172.21.15.52:0/1787269472 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v97) ==== 81+0+590 (secure 0 0 0) 0x7f1218013070 con 0x7f122406abc0 2024-06-10T02:50:44.011 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.15.52:0/3535271671 2024-06-11T02:42:42.872312+0000 2024-06-10T02:50:44.011 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.15.52:0/1092934380 2024-06-11T02:42:42.872312+0000 2024-06-10T02:50:44.011 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.15.52:6800/896317057 2024-06-11T02:42:42.872312+0000 2024-06-10T02:50:44.011 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.15.52:6801/3934479642 2024-06-11T02:42:15.498118+0000 2024-06-10T02:50:44.011 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.15.52:6800/3934479642 2024-06-11T02:42:15.498118+0000 2024-06-10T02:50:44.011 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.15.52:0/2759694132 2024-06-11T02:42:15.498118+0000 2024-06-10T02:50:44.012 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.15.52:0/2593665631 2024-06-11T02:42:15.498118+0000 2024-06-10T02:50:44.012 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.15.52:0/3151504634 2024-06-11T02:42:15.498118+0000 2024-06-10T02:50:44.012 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.15.52:6801/896317057 2024-06-11T02:42:42.872312+0000 2024-06-10T02:50:44.012 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.15.52:0/1254480982 2024-06-11T02:42:42.872312+0000 2024-06-10T02:50:44.013 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:44.013+0000 7f122aa8f700 1 -- 172.21.15.52:0/1787269472 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f12140a2c20 msgr2=0x7f12140a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:50:44.013 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:44.013+0000 7f122aa8f700 1 --2- 172.21.15.52:0/1787269472 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f12140a2c20 0x7f12140a50e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f12240789b0 tx=0x7f121000b3c0 comp rx=0 tx=0).stop 2024-06-10T02:50:44.013 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:44.013+0000 7f122aa8f700 1 -- 172.21.15.52:0/1787269472 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f122406abc0 msgr2=0x7f1224101490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:50:44.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:44.013+0000 7f122aa8f700 1 --2- 172.21.15.52:0/1787269472 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f122406abc0 0x7f1224101490 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f1218000c00 tx=0x7f12180047b0 comp rx=0 tx=0).stop 2024-06-10T02:50:44.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:44.013+0000 7f122aa8f700 1 -- 172.21.15.52:0/1787269472 shutdown_connections 2024-06-10T02:50:44.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:44.013+0000 7f122aa8f700 1 --2- 172.21.15.52:0/1787269472 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1224108500 0x7f1224101f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:50:44.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:44.013+0000 7f122aa8f700 1 --2- 172.21.15.52:0/1787269472 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f12140a2c20 0x7f12140a50e0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:50:44.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:44.013+0000 7f122aa8f700 1 --2- 172.21.15.52:0/1787269472 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f122406abc0 0x7f1224101490 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:50:44.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:44.013+0000 7f122aa8f700 1 --2- 172.21.15.52:0/1787269472 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f122406b580 0x7f12241019d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:50:44.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:44.013+0000 7f122aa8f700 1 -- 172.21.15.52:0/1787269472 >> 172.21.15.52:0/1787269472 conn(0x7f1224078340 msgr2=0x7f1224100090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:50:44.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:44.013+0000 7f122aa8f700 1 -- 172.21.15.52:0/1787269472 shutdown_connections 2024-06-10T02:50:44.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:44.013+0000 7f122aa8f700 1 -- 172.21.15.52:0/1787269472 wait complete. 2024-06-10T02:50:44.015 INFO:teuthology.orchestra.run.smithi052.stderr:listed 10 entries 2024-06-10T02:50:44.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:50:43 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]: dispatch 2024-06-10T02:50:44.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:50:43 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]: dispatch 2024-06-10T02:50:44.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:50:43 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 4]}]: dispatch 2024-06-10T02:50:44.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:50:43 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [3, 7]}]: dispatch 2024-06-10T02:50:44.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:50:43 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]: dispatch 2024-06-10T02:50:44.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:50:43 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]: dispatch 2024-06-10T02:50:44.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:50:43 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]: dispatch 2024-06-10T02:50:44.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:50:43 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]}]: dispatch 2024-06-10T02:50:44.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:50:43 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]: dispatch 2024-06-10T02:50:44.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:50:43 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [9, 11]}]: dispatch 2024-06-10T02:50:44.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:50:43 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]: dispatch 2024-06-10T02:50:44.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:50:43 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]: dispatch 2024-06-10T02:50:44.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:50:43 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 4]}]: dispatch 2024-06-10T02:50:44.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:50:43 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [3, 7]}]: dispatch 2024-06-10T02:50:44.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:50:43 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]: dispatch 2024-06-10T02:50:44.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:50:43 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]: dispatch 2024-06-10T02:50:44.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:50:43 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]: dispatch 2024-06-10T02:50:44.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:50:43 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]}]: dispatch 2024-06-10T02:50:44.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:50:43 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]: dispatch 2024-06-10T02:50:44.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:50:43 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [9, 11]}]: dispatch 2024-06-10T02:50:44.682 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-06-10T02:50:44.682 DEBUG:teuthology.orchestra.run.smithi052:> dd if=/proc/self/mounts of=/dev/stdout 2024-06-10T02:50:44.702 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph osd blocklist ls 2024-06-10T02:50:44.975 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:50:44 smithi052 ceph-mon[32362]: pgmap v326: 129 pgs: 129 active+clean; 581 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-06-10T02:50:44.975 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:50:44 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]': finished 2024-06-10T02:50:44.976 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:50:44 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]': finished 2024-06-10T02:50:44.976 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:50:44 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 4]}]': finished 2024-06-10T02:50:44.976 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:50:44 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [3, 7]}]': finished 2024-06-10T02:50:44.976 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:50:44 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]': finished 2024-06-10T02:50:44.976 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:50:44 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]': finished 2024-06-10T02:50:44.976 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:50:44 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]': finished 2024-06-10T02:50:44.976 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:50:44 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]}]': finished 2024-06-10T02:50:44.976 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:50:44 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]': finished 2024-06-10T02:50:44.976 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:50:44 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [9, 11]}]': finished 2024-06-10T02:50:44.976 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:50:44 smithi052 ceph-mon[32362]: osdmap e97: 12 total, 12 up, 12 in 2024-06-10T02:50:44.976 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:50:44 smithi052 ceph-mon[32362]: from='client.? 172.21.15.52:0/1787269472' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-06-10T02:50:45.000 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:50:45.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:50:44 smithi120 ceph-mon[36634]: pgmap v326: 129 pgs: 129 active+clean; 581 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-06-10T02:50:45.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:50:44 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]': finished 2024-06-10T02:50:45.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:50:44 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]': finished 2024-06-10T02:50:45.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:50:44 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 4]}]': finished 2024-06-10T02:50:45.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:50:44 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [3, 7]}]': finished 2024-06-10T02:50:45.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:50:44 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]': finished 2024-06-10T02:50:45.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:50:44 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]': finished 2024-06-10T02:50:45.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:50:44 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]': finished 2024-06-10T02:50:45.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:50:44 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]}]': finished 2024-06-10T02:50:45.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:50:44 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]': finished 2024-06-10T02:50:45.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:50:44 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [9, 11]}]': finished 2024-06-10T02:50:45.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:50:44 smithi120 ceph-mon[36634]: osdmap e97: 12 total, 12 up, 12 in 2024-06-10T02:50:45.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:50:44 smithi120 ceph-mon[36634]: from='client.? 172.21.15.52:0/1787269472' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-06-10T02:50:45.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:50:44 smithi188 ceph-mon[36599]: pgmap v326: 129 pgs: 129 active+clean; 581 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-06-10T02:50:45.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:50:44 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]': finished 2024-06-10T02:50:45.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:50:44 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]': finished 2024-06-10T02:50:45.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:50:44 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 4]}]': finished 2024-06-10T02:50:45.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:50:44 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [3, 7]}]': finished 2024-06-10T02:50:45.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:50:44 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]': finished 2024-06-10T02:50:45.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:50:44 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]': finished 2024-06-10T02:50:45.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:50:44 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]': finished 2024-06-10T02:50:45.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:50:44 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]}]': finished 2024-06-10T02:50:45.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:50:44 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]': finished 2024-06-10T02:50:45.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:50:44 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [9, 11]}]': finished 2024-06-10T02:50:45.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:50:44 smithi188 ceph-mon[36599]: osdmap e97: 12 total, 12 up, 12 in 2024-06-10T02:50:45.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:50:44 smithi188 ceph-mon[36599]: from='client.? 172.21.15.52:0/1787269472' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-06-10T02:50:46.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:50:45 smithi120 ceph-mon[36634]: osdmap e98: 12 total, 12 up, 12 in 2024-06-10T02:50:46.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:50:45 smithi188 ceph-mon[36599]: osdmap e98: 12 total, 12 up, 12 in 2024-06-10T02:50:46.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:50:45 smithi052 ceph-mon[32362]: osdmap e98: 12 total, 12 up, 12 in 2024-06-10T02:50:46.763 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:46.763+0000 7fea1d0cb700 1 -- 172.21.15.52:0/7116050 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fea18101d40 msgr2=0x7fea1810e230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:50:46.763 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:46.763+0000 7fea1d0cb700 1 --2- 172.21.15.52:0/7116050 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fea18101d40 0x7fea1810e230 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fea10009990 tx=0x7fea10009ca0 comp rx=0 tx=0).stop 2024-06-10T02:50:46.763 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:46.763+0000 7fea1d0cb700 1 -- 172.21.15.52:0/7116050 shutdown_connections 2024-06-10T02:50:46.763 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:46.763+0000 7fea1d0cb700 1 --2- 172.21.15.52:0/7116050 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fea18101d40 0x7fea1810e230 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:50:46.764 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:46.763+0000 7fea1d0cb700 1 --2- 172.21.15.52:0/7116050 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fea181073f0 0x7fea181077d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:50:46.764 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:46.763+0000 7fea1d0cb700 1 --2- 172.21.15.52:0/7116050 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fea18101380 0x7fea18101800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:50:46.764 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:46.763+0000 7fea1d0cb700 1 -- 172.21.15.52:0/7116050 >> 172.21.15.52:0/7116050 conn(0x7fea18078300 msgr2=0x7fea18078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:50:46.764 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:46.763+0000 7fea1d0cb700 1 -- 172.21.15.52:0/7116050 shutdown_connections 2024-06-10T02:50:46.764 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:46.763+0000 7fea1d0cb700 1 -- 172.21.15.52:0/7116050 wait complete. 2024-06-10T02:50:46.766 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:46.764+0000 7fea1d0cb700 1 Processor -- start 2024-06-10T02:50:46.766 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:46.764+0000 7fea1d0cb700 1 -- start start 2024-06-10T02:50:46.766 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:46.764+0000 7fea1d0cb700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fea18101380 0x7fea1819e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:50:46.766 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:46.764+0000 7fea1d0cb700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fea18101d40 0x7fea1819ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:50:46.766 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:46.764+0000 7fea1d0cb700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fea181073f0 0x7fea18198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:50:46.766 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:46.765+0000 7fea1d0cb700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fea181127e0 con 0x7fea181073f0 2024-06-10T02:50:46.766 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:46.765+0000 7fea1d0cb700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fea18112660 con 0x7fea18101380 2024-06-10T02:50:46.766 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:46.765+0000 7fea1d0cb700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fea18112960 con 0x7fea18101d40 2024-06-10T02:50:46.766 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:46.765+0000 7fea1759e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fea181073f0 0x7fea18198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:50:46.766 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:46.765+0000 7fea16d9d700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fea18101380 0x7fea1819e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:50:46.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:46.765+0000 7fea1759e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fea181073f0 0x7fea18198a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:45292/0 (socket says 172.21.15.52:45292) 2024-06-10T02:50:46.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:46.765+0000 7fea1759e700 1 -- 172.21.15.52:0/1705463834 learned_addr learned my addr 172.21.15.52:0/1705463834 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:50:46.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:46.765+0000 7fea1659c700 1 --2- 172.21.15.52:0/1705463834 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fea18101d40 0x7fea1819ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:50:46.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:46.765+0000 7fea1759e700 1 -- 172.21.15.52:0/1705463834 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fea18101d40 msgr2=0x7fea1819ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:50:46.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:46.765+0000 7fea1759e700 1 --2- 172.21.15.52:0/1705463834 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fea18101d40 0x7fea1819ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:50:46.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:46.765+0000 7fea1759e700 1 -- 172.21.15.52:0/1705463834 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fea18101380 msgr2=0x7fea1819e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:50:46.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:46.765+0000 7fea1759e700 1 --2- 172.21.15.52:0/1705463834 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fea18101380 0x7fea1819e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:50:46.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:46.765+0000 7fea1759e700 1 -- 172.21.15.52:0/1705463834 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea18199160 con 0x7fea181073f0 2024-06-10T02:50:46.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:46.765+0000 7fea16d9d700 1 --2- 172.21.15.52:0/1705463834 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fea18101380 0x7fea1819e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T02:50:46.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:46.765+0000 7fea1759e700 1 --2- 172.21.15.52:0/1705463834 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fea181073f0 0x7fea18198a00 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fea100047b0 tx=0x7fea10005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:50:46.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:46.765+0000 7fe9fffff700 1 -- 172.21.15.52:0/1705463834 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea10026070 con 0x7fea181073f0 2024-06-10T02:50:46.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:46.766+0000 7fea1d0cb700 1 -- 172.21.15.52:0/1705463834 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea181993f0 con 0x7fea181073f0 2024-06-10T02:50:46.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:46.766+0000 7fe9fffff700 1 -- 172.21.15.52:0/1705463834 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea10004cc0 con 0x7fea181073f0 2024-06-10T02:50:46.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:46.766+0000 7fe9fffff700 1 -- 172.21.15.52:0/1705463834 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea10003ba0 con 0x7fea181073f0 2024-06-10T02:50:46.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:46.766+0000 7fea1d0cb700 1 -- 172.21.15.52:0/1705463834 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea181a56c0 con 0x7fea181073f0 2024-06-10T02:50:46.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:46.767+0000 7fea1d0cb700 1 -- 172.21.15.52:0/1705463834 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea18069180 con 0x7fea181073f0 2024-06-10T02:50:46.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:46.769+0000 7fe9fffff700 1 -- 172.21.15.52:0/1705463834 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fea10029070 con 0x7fea181073f0 2024-06-10T02:50:46.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:46.769+0000 7fe9fffff700 1 --2- 172.21.15.52:0/1705463834 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fea000a2ca0 0x7fea000a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:50:46.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:46.769+0000 7fe9fffff700 1 -- 172.21.15.52:0/1705463834 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9092+0+0 (secure 0 0 0) 0x7fea100cff20 con 0x7fea181073f0 2024-06-10T02:50:46.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:46.769+0000 7fea16d9d700 1 --2- 172.21.15.52:0/1705463834 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fea000a2ca0 0x7fea000a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:50:46.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:46.770+0000 7fea16d9d700 1 --2- 172.21.15.52:0/1705463834 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fea000a2ca0 0x7fea000a5160 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fea08009550 tx=0x7fea080091b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:50:46.772 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:46.773+0000 7fe9fffff700 1 -- 172.21.15.52:0/1705463834 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fea10030020 con 0x7fea181073f0 2024-06-10T02:50:46.975 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:50:46 smithi052 ceph-mon[32362]: pgmap v329: 129 pgs: 1 peering, 128 active+clean; 581 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 0 op/s 2024-06-10T02:50:46.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:46.994+0000 7fea1d0cb700 1 -- 172.21.15.52:0/1705463834 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fea18199ee0 con 0x7fea181073f0 2024-06-10T02:50:46.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:46.994+0000 7fe9fffff700 1 -- 172.21.15.52:0/1705463834 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v98) ==== 81+0+590 (secure 0 0 0) 0x7fea10013070 con 0x7fea181073f0 2024-06-10T02:50:46.994 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.15.52:0/3535271671 2024-06-11T02:42:42.872312+0000 2024-06-10T02:50:46.994 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.15.52:0/1092934380 2024-06-11T02:42:42.872312+0000 2024-06-10T02:50:46.994 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.15.52:6800/896317057 2024-06-11T02:42:42.872312+0000 2024-06-10T02:50:46.994 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.15.52:6801/3934479642 2024-06-11T02:42:15.498118+0000 2024-06-10T02:50:46.994 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.15.52:6800/3934479642 2024-06-11T02:42:15.498118+0000 2024-06-10T02:50:46.994 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.15.52:0/2759694132 2024-06-11T02:42:15.498118+0000 2024-06-10T02:50:46.994 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.15.52:0/2593665631 2024-06-11T02:42:15.498118+0000 2024-06-10T02:50:46.994 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.15.52:0/3151504634 2024-06-11T02:42:15.498118+0000 2024-06-10T02:50:46.994 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.15.52:6801/896317057 2024-06-11T02:42:42.872312+0000 2024-06-10T02:50:46.994 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.15.52:0/1254480982 2024-06-11T02:42:42.872312+0000 2024-06-10T02:50:46.997 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:46.997+0000 7fea1d0cb700 1 -- 172.21.15.52:0/1705463834 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fea000a2ca0 msgr2=0x7fea000a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:50:46.997 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:46.997+0000 7fea1d0cb700 1 --2- 172.21.15.52:0/1705463834 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fea000a2ca0 0x7fea000a5160 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fea08009550 tx=0x7fea080091b0 comp rx=0 tx=0).stop 2024-06-10T02:50:46.997 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:46.997+0000 7fea1d0cb700 1 -- 172.21.15.52:0/1705463834 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fea181073f0 msgr2=0x7fea18198a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:50:46.997 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:46.997+0000 7fea1d0cb700 1 --2- 172.21.15.52:0/1705463834 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fea181073f0 0x7fea18198a00 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fea100047b0 tx=0x7fea10005e80 comp rx=0 tx=0).stop 2024-06-10T02:50:46.997 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:46.997+0000 7fea1d0cb700 1 -- 172.21.15.52:0/1705463834 shutdown_connections 2024-06-10T02:50:46.997 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:46.997+0000 7fea1d0cb700 1 --2- 172.21.15.52:0/1705463834 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fea000a2ca0 0x7fea000a5160 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:50:46.997 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:46.997+0000 7fea1d0cb700 1 --2- 172.21.15.52:0/1705463834 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fea181073f0 0x7fea18198a00 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:50:46.997 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:46.997+0000 7fea1d0cb700 1 --2- 172.21.15.52:0/1705463834 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fea18101380 0x7fea1819e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:50:46.997 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:46.997+0000 7fea1d0cb700 1 --2- 172.21.15.52:0/1705463834 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fea18101d40 0x7fea1819ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:50:46.997 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:46.997+0000 7fea1d0cb700 1 -- 172.21.15.52:0/1705463834 >> 172.21.15.52:0/1705463834 conn(0x7fea18078300 msgr2=0x7fea18100070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:50:46.997 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:46.997+0000 7fea1d0cb700 1 -- 172.21.15.52:0/1705463834 shutdown_connections 2024-06-10T02:50:46.998 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:46.997+0000 7fea1d0cb700 1 -- 172.21.15.52:0/1705463834 wait complete. 2024-06-10T02:50:46.998 INFO:teuthology.orchestra.run.smithi052.stderr:listed 10 entries 2024-06-10T02:50:47.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:50:46 smithi120 ceph-mon[36634]: pgmap v329: 129 pgs: 1 peering, 128 active+clean; 581 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 0 op/s 2024-06-10T02:50:47.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:50:46 smithi188 ceph-mon[36599]: pgmap v329: 129 pgs: 1 peering, 128 active+clean; 581 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 0 op/s 2024-06-10T02:50:47.640 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi052.front.sepia.ceph.com... 2024-06-10T02:50:47.640 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-10T02:50:47.641 DEBUG:teuthology.orchestra.run.smithi052:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-06-10T02:50:47.668 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi052.stderr:ceph-fuse[78631]: fuse finished with error 0 and tester_r 0 2024-06-10T02:50:47.670 INFO:teuthology.orchestra.run:waiting for 300 2024-06-10T02:50:48.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:50:47 smithi120 ceph-mon[36634]: from='client.? 172.21.15.52:0/1705463834' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-06-10T02:50:48.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:50:47 smithi188 ceph-mon[36599]: from='client.? 172.21.15.52:0/1705463834' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-06-10T02:50:48.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:50:47 smithi052 ceph-mon[32362]: from='client.? 172.21.15.52:0/1705463834' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-06-10T02:50:49.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:50:48 smithi120 ceph-mon[36634]: pgmap v330: 129 pgs: 1 peering, 128 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 0 op/s; 72 B/s, 0 objects/s recovering 2024-06-10T02:50:49.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:50:48 smithi188 ceph-mon[36599]: pgmap v330: 129 pgs: 1 peering, 128 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 0 op/s; 72 B/s, 0 objects/s recovering 2024-06-10T02:50:49.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:50:48 smithi052 ceph-mon[32362]: pgmap v330: 129 pgs: 1 peering, 128 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 0 op/s; 72 B/s, 0 objects/s recovering 2024-06-10T02:50:51.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:50:50 smithi120 ceph-mon[36634]: pgmap v331: 129 pgs: 1 peering, 128 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 0 op/s; 72 B/s, 0 objects/s recovering 2024-06-10T02:50:51.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:50:50 smithi188 ceph-mon[36599]: pgmap v331: 129 pgs: 1 peering, 128 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 0 op/s; 72 B/s, 0 objects/s recovering 2024-06-10T02:50:51.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:50:50 smithi052 ceph-mon[32362]: pgmap v331: 129 pgs: 1 peering, 128 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 0 op/s; 72 B/s, 0 objects/s recovering 2024-06-10T02:50:53.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:50:52 smithi120 ceph-mon[36634]: pgmap v332: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 72 B/s, 0 objects/s recovering 2024-06-10T02:50:53.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:50:52 smithi188 ceph-mon[36599]: pgmap v332: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 72 B/s, 0 objects/s recovering 2024-06-10T02:50:53.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:50:52 smithi052 ceph-mon[32362]: pgmap v332: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 72 B/s, 0 objects/s recovering 2024-06-10T02:50:53.772 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi052.front.sepia.ceph.com 2024-06-10T02:50:53.772 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-10T02:50:53.772 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-06-10T02:50:53.802 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-10T02:50:54.121 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:50:55.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:50:54 smithi120 ceph-mon[36634]: pgmap v333: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 110 B/s wr, 0 op/s; 62 B/s, 0 objects/s recovering 2024-06-10T02:50:55.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:50:54 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T02:50:55.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:50:54 smithi188 ceph-mon[36599]: pgmap v333: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 110 B/s wr, 0 op/s; 62 B/s, 0 objects/s recovering 2024-06-10T02:50:55.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:50:54 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T02:50:55.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:50:54 smithi052 ceph-mon[32362]: pgmap v333: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 110 B/s wr, 0 op/s; 62 B/s, 0 objects/s recovering 2024-06-10T02:50:55.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:50:54 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T02:50:56.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:56.183+0000 7efc2be71700 1 -- 172.21.15.52:0/3231630817 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efc241024e0 msgr2=0x7efc24102960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:50:56.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:56.183+0000 7efc2be71700 1 --2- 172.21.15.52:0/3231630817 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efc241024e0 0x7efc24102960 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7efc14009990 tx=0x7efc14009ca0 comp rx=0 tx=0).stop 2024-06-10T02:50:56.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:56.183+0000 7efc2be71700 1 -- 172.21.15.52:0/3231630817 shutdown_connections 2024-06-10T02:50:56.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:56.183+0000 7efc2be71700 1 --2- 172.21.15.52:0/3231630817 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efc24102ea0 0x7efc2410f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:50:56.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:56.183+0000 7efc2be71700 1 --2- 172.21.15.52:0/3231630817 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efc241085f0 0x7efc241089d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:50:56.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:56.183+0000 7efc2be71700 1 --2- 172.21.15.52:0/3231630817 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efc241024e0 0x7efc24102960 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:50:56.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:56.183+0000 7efc2be71700 1 -- 172.21.15.52:0/3231630817 >> 172.21.15.52:0/3231630817 conn(0x7efc24079640 msgr2=0x7efc24079a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:50:56.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:56.183+0000 7efc2be71700 1 -- 172.21.15.52:0/3231630817 shutdown_connections 2024-06-10T02:50:56.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:56.183+0000 7efc2be71700 1 -- 172.21.15.52:0/3231630817 wait complete. 2024-06-10T02:50:56.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:56.184+0000 7efc2be71700 1 Processor -- start 2024-06-10T02:50:56.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:56.184+0000 7efc2be71700 1 -- start start 2024-06-10T02:50:56.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:56.184+0000 7efc2be71700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efc241024e0 0x7efc2419e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:50:56.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:56.184+0000 7efc2be71700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efc24102ea0 0x7efc2419ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:50:56.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:56.184+0000 7efc2be71700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efc241085f0 0x7efc24198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:50:56.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:56.185+0000 7efc2be71700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7efc24112790 con 0x7efc24102ea0 2024-06-10T02:50:56.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:56.185+0000 7efc2be71700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7efc24112610 con 0x7efc241085f0 2024-06-10T02:50:56.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:56.185+0000 7efc2be71700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7efc24112910 con 0x7efc241024e0 2024-06-10T02:50:56.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:56.185+0000 7efc2940c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efc24102ea0 0x7efc2419ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:50:56.505 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-10T02:50:56.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:56.185+0000 7efc2940c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efc24102ea0 0x7efc2419ed10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49334/0 (socket says 172.21.15.52:49334) 2024-06-10T02:50:56.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:56.185+0000 7efc2940c700 1 -- 172.21.15.52:0/297706186 learned_addr learned my addr 172.21.15.52:0/297706186 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:50:56.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:56.185+0000 7efc29c0d700 1 --2- 172.21.15.52:0/297706186 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efc241024e0 0x7efc2419e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:50:56.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:56.185+0000 7efc2a40e700 1 --2- 172.21.15.52:0/297706186 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efc241085f0 0x7efc24198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:50:56.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:56.185+0000 7efc2940c700 1 -- 172.21.15.52:0/297706186 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efc241024e0 msgr2=0x7efc2419e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:50:56.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:56.185+0000 7efc2940c700 1 --2- 172.21.15.52:0/297706186 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efc241024e0 0x7efc2419e7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:50:56.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:56.185+0000 7efc2940c700 1 -- 172.21.15.52:0/297706186 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efc241085f0 msgr2=0x7efc24198a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:50:56.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:56.185+0000 7efc2940c700 1 --2- 172.21.15.52:0/297706186 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efc241085f0 0x7efc24198a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:50:56.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:56.185+0000 7efc2940c700 1 -- 172.21.15.52:0/297706186 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc24199190 con 0x7efc24102ea0 2024-06-10T02:50:56.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:56.185+0000 7efc2940c700 1 --2- 172.21.15.52:0/297706186 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efc24102ea0 0x7efc2419ed10 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7efc14000c00 tx=0x7efc1400bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:50:56.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:56.185+0000 7efc1affd700 1 -- 172.21.15.52:0/297706186 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc14027070 con 0x7efc24102ea0 2024-06-10T02:50:56.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:56.185+0000 7efc1affd700 1 -- 172.21.15.52:0/297706186 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc14003d70 con 0x7efc24102ea0 2024-06-10T02:50:56.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:56.185+0000 7efc2be71700 1 -- 172.21.15.52:0/297706186 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc24199420 con 0x7efc24102ea0 2024-06-10T02:50:56.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:56.185+0000 7efc1affd700 1 -- 172.21.15.52:0/297706186 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc1400f460 con 0x7efc24102ea0 2024-06-10T02:50:56.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:56.186+0000 7efc2be71700 1 -- 172.21.15.52:0/297706186 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc241a56a0 con 0x7efc24102ea0 2024-06-10T02:50:56.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:56.187+0000 7efc2be71700 1 -- 172.21.15.52:0/297706186 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc24069180 con 0x7efc24102ea0 2024-06-10T02:50:56.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:56.188+0000 7efc1affd700 1 -- 172.21.15.52:0/297706186 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7efc1402a070 con 0x7efc24102ea0 2024-06-10T02:50:56.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:56.188+0000 7efc1affd700 1 --2- 172.21.15.52:0/297706186 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7efc100a2ca0 0x7efc100a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:50:56.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:56.189+0000 7efc1affd700 1 -- 172.21.15.52:0/297706186 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9092+0+0 (secure 0 0 0) 0x7efc140d00f0 con 0x7efc24102ea0 2024-06-10T02:50:56.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:56.189+0000 7efc29c0d700 1 --2- 172.21.15.52:0/297706186 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7efc100a2ca0 0x7efc100a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:50:56.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:56.189+0000 7efc29c0d700 1 --2- 172.21.15.52:0/297706186 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7efc100a2ca0 0x7efc100a5160 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7efc24100b10 tx=0x7efc200073d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:50:56.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:56.193+0000 7efc1affd700 1 -- 172.21.15.52:0/297706186 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7efc14013070 con 0x7efc24102ea0 2024-06-10T02:50:56.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:56.404+0000 7efc2be71700 1 -- 172.21.15.52:0/297706186 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7efc2402dca0 con 0x7efc24102ea0 2024-06-10T02:50:56.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:56.406+0000 7efc1affd700 1 -- 172.21.15.52:0/297706186 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v98) ==== 74+0+24598 (secure 0 0 0) 0x7efc1409cd80 con 0x7efc24102ea0 2024-06-10T02:50:56.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:56.408+0000 7efc2be71700 1 -- 172.21.15.52:0/297706186 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7efc100a2ca0 msgr2=0x7efc100a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:50:56.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:56.408+0000 7efc2be71700 1 --2- 172.21.15.52:0/297706186 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7efc100a2ca0 0x7efc100a5160 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7efc24100b10 tx=0x7efc200073d0 comp rx=0 tx=0).stop 2024-06-10T02:50:56.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:56.408+0000 7efc2be71700 1 -- 172.21.15.52:0/297706186 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efc24102ea0 msgr2=0x7efc2419ed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:50:56.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:56.408+0000 7efc2be71700 1 --2- 172.21.15.52:0/297706186 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efc24102ea0 0x7efc2419ed10 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7efc14000c00 tx=0x7efc1400bab0 comp rx=0 tx=0).stop 2024-06-10T02:50:56.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:56.409+0000 7efc2be71700 1 -- 172.21.15.52:0/297706186 shutdown_connections 2024-06-10T02:50:56.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:56.409+0000 7efc2be71700 1 --2- 172.21.15.52:0/297706186 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efc241085f0 0x7efc24198a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:50:56.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:56.409+0000 7efc2be71700 1 --2- 172.21.15.52:0/297706186 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efc241024e0 0x7efc2419e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:50:56.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:56.409+0000 7efc2be71700 1 --2- 172.21.15.52:0/297706186 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7efc100a2ca0 0x7efc100a5160 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:50:56.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:56.409+0000 7efc2be71700 1 --2- 172.21.15.52:0/297706186 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efc24102ea0 0x7efc2419ed10 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:50:56.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:56.409+0000 7efc2be71700 1 -- 172.21.15.52:0/297706186 >> 172.21.15.52:0/297706186 conn(0x7efc24079640 msgr2=0x7efc24100fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:50:56.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:56.409+0000 7efc2be71700 1 -- 172.21.15.52:0/297706186 shutdown_connections 2024-06-10T02:50:56.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:56.409+0000 7efc2be71700 1 -- 172.21.15.52:0/297706186 wait complete. 2024-06-10T02:50:56.508 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":98,"fsid":"d8676e9e-26d2-11ef-bc9b-c7b262605968","created":"2024-06-10T02:41:43.076837+0000","modified":"2024-06-10T02:50:44.731014+0000","last_up_change":"2024-06-10T02:48:57.297829+0000","last_in_change":"2024-06-10T02:48:36.865802+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-10T02:45:07.394690+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-10T02:49:57.926838+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-10T02:50:01.130393+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-10T02:50:10.655588+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e03c0691-cc5a-497a-bbc2-ce18c337f074","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":4255833408},{"type":"v1","addr":"172.21.15.52:6803","nonce":4255833408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":4255833408},{"type":"v1","addr":"172.21.15.52:6805","nonce":4255833408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":4255833408},{"type":"v1","addr":"172.21.15.52:6809","nonce":4255833408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":4255833408},{"type":"v1","addr":"172.21.15.52:6807","nonce":4255833408}]},"public_addr":"172.21.15.52:6803/4255833408","cluster_addr":"172.21.15.52:6805/4255833408","heartbeat_back_addr":"172.21.15.52:6809/4255833408","heartbeat_front_addr":"172.21.15.52:6807/4255833408","state":["exists","up"]},{"osd":1,"uuid":"a24d8da8-c5ec-44d7-ab30-56f6ce3efda7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":4116540053},{"type":"v1","addr":"172.21.15.120:6801","nonce":4116540053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":4116540053},{"type":"v1","addr":"172.21.15.120:6803","nonce":4116540053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":4116540053},{"type":"v1","addr":"172.21.15.120:6807","nonce":4116540053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":4116540053},{"type":"v1","addr":"172.21.15.120:6805","nonce":4116540053}]},"public_addr":"172.21.15.120:6801/4116540053","cluster_addr":"172.21.15.120:6803/4116540053","heartbeat_back_addr":"172.21.15.120:6807/4116540053","heartbeat_front_addr":"172.21.15.120:6805/4116540053","state":["exists","up"]},{"osd":2,"uuid":"a7162106-ccbf-45bd-bc0e-051ee43f4214","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":2624326999},{"type":"v1","addr":"172.21.15.188:6801","nonce":2624326999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":2624326999},{"type":"v1","addr":"172.21.15.188:6803","nonce":2624326999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":2624326999},{"type":"v1","addr":"172.21.15.188:6807","nonce":2624326999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":2624326999},{"type":"v1","addr":"172.21.15.188:6805","nonce":2624326999}]},"public_addr":"172.21.15.188:6801/2624326999","cluster_addr":"172.21.15.188:6803/2624326999","heartbeat_back_addr":"172.21.15.188:6807/2624326999","heartbeat_front_addr":"172.21.15.188:6805/2624326999","state":["exists","up"]},{"osd":3,"uuid":"53680244-b49a-4120-b24a-9ad7a6341dc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":116897011},{"type":"v1","addr":"172.21.15.52:6811","nonce":116897011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":116897011},{"type":"v1","addr":"172.21.15.52:6813","nonce":116897011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":116897011},{"type":"v1","addr":"172.21.15.52:6817","nonce":116897011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":116897011},{"type":"v1","addr":"172.21.15.52:6815","nonce":116897011}]},"public_addr":"172.21.15.52:6811/116897011","cluster_addr":"172.21.15.52:6813/116897011","heartbeat_back_addr":"172.21.15.52:6817/116897011","heartbeat_front_addr":"172.21.15.52:6815/116897011","state":["exists","up"]},{"osd":4,"uuid":"9f72f1f4-2ef3-49c5-8410-c108767d2ef5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":1320690621},{"type":"v1","addr":"172.21.15.120:6809","nonce":1320690621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":1320690621},{"type":"v1","addr":"172.21.15.120:6811","nonce":1320690621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":1320690621},{"type":"v1","addr":"172.21.15.120:6815","nonce":1320690621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":1320690621},{"type":"v1","addr":"172.21.15.120:6813","nonce":1320690621}]},"public_addr":"172.21.15.120:6809/1320690621","cluster_addr":"172.21.15.120:6811/1320690621","heartbeat_back_addr":"172.21.15.120:6815/1320690621","heartbeat_front_addr":"172.21.15.120:6813/1320690621","state":["exists","up"]},{"osd":5,"uuid":"001b5706-7f27-4fa4-9b69-969e6d765abc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3951947483},{"type":"v1","addr":"172.21.15.188:6809","nonce":3951947483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3951947483},{"type":"v1","addr":"172.21.15.188:6811","nonce":3951947483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3951947483},{"type":"v1","addr":"172.21.15.188:6815","nonce":3951947483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3951947483},{"type":"v1","addr":"172.21.15.188:6813","nonce":3951947483}]},"public_addr":"172.21.15.188:6809/3951947483","cluster_addr":"172.21.15.188:6811/3951947483","heartbeat_back_addr":"172.21.15.188:6815/3951947483","heartbeat_front_addr":"172.21.15.188:6813/3951947483","state":["exists","up"]},{"osd":6,"uuid":"140c903e-9b31-4a81-9ccd-2edc0a2decf9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":359084919},{"type":"v1","addr":"172.21.15.52:6819","nonce":359084919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":359084919},{"type":"v1","addr":"172.21.15.52:6821","nonce":359084919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":359084919},{"type":"v1","addr":"172.21.15.52:6825","nonce":359084919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":359084919},{"type":"v1","addr":"172.21.15.52:6823","nonce":359084919}]},"public_addr":"172.21.15.52:6819/359084919","cluster_addr":"172.21.15.52:6821/359084919","heartbeat_back_addr":"172.21.15.52:6825/359084919","heartbeat_front_addr":"172.21.15.52:6823/359084919","state":["exists","up"]},{"osd":7,"uuid":"57b735d4-5072-4ca2-a3ff-3763ad84f15a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":3737722011},{"type":"v1","addr":"172.21.15.120:6817","nonce":3737722011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6818","nonce":3737722011},{"type":"v1","addr":"172.21.15.120:6819","nonce":3737722011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":3737722011},{"type":"v1","addr":"172.21.15.120:6823","nonce":3737722011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":3737722011},{"type":"v1","addr":"172.21.15.120:6821","nonce":3737722011}]},"public_addr":"172.21.15.120:6817/3737722011","cluster_addr":"172.21.15.120:6819/3737722011","heartbeat_back_addr":"172.21.15.120:6823/3737722011","heartbeat_front_addr":"172.21.15.120:6821/3737722011","state":["exists","up"]},{"osd":8,"uuid":"4ffbb43b-854a-4eac-99cf-7af89bc65409","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1444828441},{"type":"v1","addr":"172.21.15.188:6817","nonce":1444828441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1444828441},{"type":"v1","addr":"172.21.15.188:6819","nonce":1444828441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1444828441},{"type":"v1","addr":"172.21.15.188:6823","nonce":1444828441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1444828441},{"type":"v1","addr":"172.21.15.188:6821","nonce":1444828441}]},"public_addr":"172.21.15.188:6817/1444828441","cluster_addr":"172.21.15.188:6819/1444828441","heartbeat_back_addr":"172.21.15.188:6823/1444828441","heartbeat_front_addr":"172.21.15.188:6821/1444828441","state":["exists","up"]},{"osd":9,"uuid":"5bfd985c-8acc-4d00-8561-40bbd541c07b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":924440658},{"type":"v1","addr":"172.21.15.52:6827","nonce":924440658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":924440658},{"type":"v1","addr":"172.21.15.52:6829","nonce":924440658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":924440658},{"type":"v1","addr":"172.21.15.52:6833","nonce":924440658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":924440658},{"type":"v1","addr":"172.21.15.52:6831","nonce":924440658}]},"public_addr":"172.21.15.52:6827/924440658","cluster_addr":"172.21.15.52:6829/924440658","heartbeat_back_addr":"172.21.15.52:6833/924440658","heartbeat_front_addr":"172.21.15.52:6831/924440658","state":["exists","up"]},{"osd":10,"uuid":"0180cddf-e109-42cc-829a-185604276f02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6824","nonce":1531769083},{"type":"v1","addr":"172.21.15.120:6825","nonce":1531769083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":1531769083},{"type":"v1","addr":"172.21.15.120:6827","nonce":1531769083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":1531769083},{"type":"v1","addr":"172.21.15.120:6831","nonce":1531769083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6828","nonce":1531769083},{"type":"v1","addr":"172.21.15.120:6829","nonce":1531769083}]},"public_addr":"172.21.15.120:6825/1531769083","cluster_addr":"172.21.15.120:6827/1531769083","heartbeat_back_addr":"172.21.15.120:6831/1531769083","heartbeat_front_addr":"172.21.15.120:6829/1531769083","state":["exists","up"]},{"osd":11,"uuid":"73aec750-82d1-4800-b6a0-4e305912f9ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":3769050170},{"type":"v1","addr":"172.21.15.188:6825","nonce":3769050170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":3769050170},{"type":"v1","addr":"172.21.15.188:6827","nonce":3769050170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":3769050170},{"type":"v1","addr":"172.21.15.188:6831","nonce":3769050170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":3769050170},{"type":"v1","addr":"172.21.15.188:6829","nonce":3769050170}]},"public_addr":"172.21.15.188:6825/3769050170","cluster_addr":"172.21.15.188:6827/3769050170","heartbeat_back_addr":"172.21.15.188:6831/3769050170","heartbeat_front_addr":"172.21.15.188:6829/3769050170","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:44:16.455752+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:44:40.219735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:45:03.558899+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:45:29.155004+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:45:53.981098+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:46:19.412020+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:46:45.462640+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:47:11.484578+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:47:36.879054+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:48:03.392446+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:48:29.422173+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:48:55.523423+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":4}]},{"pgid":"3.6","mappings":[{"from":3,"to":7}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.52:0/3535271671":"2024-06-11T02:42:42.872312+0000","172.21.15.52:0/1092934380":"2024-06-11T02:42:42.872312+0000","172.21.15.52:6800/896317057":"2024-06-11T02:42:42.872312+0000","172.21.15.52:6801/3934479642":"2024-06-11T02:42:15.498118+0000","172.21.15.52:6800/3934479642":"2024-06-11T02:42:15.498118+0000","172.21.15.52:0/2759694132":"2024-06-11T02:42:15.498118+0000","172.21.15.52:0/2593665631":"2024-06-11T02:42:15.498118+0000","172.21.15.52:0/3151504634":"2024-06-11T02:42:15.498118+0000","172.21.15.52:6801/896317057":"2024-06-11T02:42:42.872312+0000","172.21.15.52:0/1254480982":"2024-06-11T02:42:42.872312+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-10T02:50:56.726 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:50:56 smithi052 ceph-mon[32362]: pgmap v334: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 99 B/s wr, 0 op/s; 56 B/s, 0 objects/s recovering 2024-06-10T02:50:56.726 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:50:56 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:50:56.726 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:50:56 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:50:56.726 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:50:56 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:50:56.726 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:50:56 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:50:56.726 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:50:56 smithi052 ceph-mon[32362]: from='client.? 172.21.15.52:0/297706186' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T02:50:56.887 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:50:56 smithi120 ceph-mon[36634]: pgmap v334: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 99 B/s wr, 0 op/s; 56 B/s, 0 objects/s recovering 2024-06-10T02:50:56.887 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:50:56 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:50:56.887 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:50:56 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:50:56.887 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:50:56 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:50:56.887 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:50:56 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:50:56.888 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:50:56 smithi120 ceph-mon[36634]: from='client.? 172.21.15.52:0/297706186' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T02:50:56.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:50:56 smithi188 ceph-mon[36599]: pgmap v334: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 99 B/s wr, 0 op/s; 56 B/s, 0 objects/s recovering 2024-06-10T02:50:56.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:50:56 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:50:56.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:50:56 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:50:56.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:50:56 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:50:56.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:50:56 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:50:56.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:50:56 smithi188 ceph-mon[36599]: from='client.? 172.21.15.52:0/297706186' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T02:50:57.228 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-10T02:50:57.512 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:50:58.887 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:50:58 smithi120 ceph-mon[36634]: pgmap v335: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s; 48 B/s, 0 objects/s recovering 2024-06-10T02:50:58.887 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:50:58 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:50:58.887 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:50:58 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:50:58.887 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:50:58 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:50:58.887 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:50:58 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T02:50:58.887 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:50:58 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:50:58.910 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:50:58 smithi052 ceph-mon[32362]: pgmap v335: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s; 48 B/s, 0 objects/s recovering 2024-06-10T02:50:58.910 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:50:58 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:50:58.911 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:50:58 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:50:58.911 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:50:58 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:50:58.911 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:50:58 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T02:50:58.911 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:50:58 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:50:58.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:50:58 smithi188 ceph-mon[36599]: pgmap v335: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s; 48 B/s, 0 objects/s recovering 2024-06-10T02:50:58.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:50:58 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:50:58.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:50:58 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:50:58.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:50:58 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:50:58.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:50:58 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T02:50:58.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:50:58 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:50:59.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:59.160+0000 7f86af761700 1 -- 172.21.15.52:0/2347642050 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f86a8108580 msgr2=0x7f86a8108960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:50:59.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:59.160+0000 7f86af761700 1 --2- 172.21.15.52:0/2347642050 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f86a8108580 0x7f86a8108960 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f86a40099c0 tx=0x7f86a4009cd0 comp rx=0 tx=0).stop 2024-06-10T02:50:59.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:59.160+0000 7f86af761700 1 -- 172.21.15.52:0/2347642050 shutdown_connections 2024-06-10T02:50:59.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:59.160+0000 7f86af761700 1 --2- 172.21.15.52:0/2347642050 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f86a806b620 0x7f86a810f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:50:59.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:59.160+0000 7f86af761700 1 --2- 172.21.15.52:0/2347642050 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f86a8108580 0x7f86a8108960 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:50:59.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:59.160+0000 7f86af761700 1 --2- 172.21.15.52:0/2347642050 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f86a806ac60 0x7f86a806b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:50:59.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:59.160+0000 7f86af761700 1 -- 172.21.15.52:0/2347642050 >> 172.21.15.52:0/2347642050 conn(0x7f86a8078430 msgr2=0x7f86a8078840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:50:59.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:59.161+0000 7f86af761700 1 -- 172.21.15.52:0/2347642050 shutdown_connections 2024-06-10T02:50:59.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:59.161+0000 7f86af761700 1 -- 172.21.15.52:0/2347642050 wait complete. 2024-06-10T02:50:59.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:59.161+0000 7f86af761700 1 Processor -- start 2024-06-10T02:50:59.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:59.161+0000 7f86af761700 1 -- start start 2024-06-10T02:50:59.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:59.162+0000 7f86af761700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f86a806ac60 0x7f86a8101400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:50:59.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:59.162+0000 7f86af761700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f86a806b620 0x7f86a8101940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:50:59.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:59.162+0000 7f86af761700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f86a8108580 0x7f86a8101e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:50:59.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:59.162+0000 7f86af761700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f86a8112840 con 0x7f86a806ac60 2024-06-10T02:50:59.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:59.162+0000 7f86af761700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f86a81126c0 con 0x7f86a8108580 2024-06-10T02:50:59.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:59.162+0000 7f86af761700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f86a81129c0 con 0x7f86a806b620 2024-06-10T02:50:59.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:59.162+0000 7f86ad4fd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f86a806ac60 0x7f86a8101400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:50:59.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:59.162+0000 7f86ad4fd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f86a806ac60 0x7f86a8101400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49352/0 (socket says 172.21.15.52:49352) 2024-06-10T02:50:59.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:59.162+0000 7f86ad4fd700 1 -- 172.21.15.52:0/547132617 learned_addr learned my addr 172.21.15.52:0/547132617 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:50:59.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:59.162+0000 7f86accfc700 1 --2- 172.21.15.52:0/547132617 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f86a806b620 0x7f86a8101940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:50:59.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:59.162+0000 7f86adcfe700 1 --2- 172.21.15.52:0/547132617 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f86a8108580 0x7f86a8101e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:50:59.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:59.162+0000 7f86ad4fd700 1 -- 172.21.15.52:0/547132617 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f86a806b620 msgr2=0x7f86a8101940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:50:59.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:59.162+0000 7f86ad4fd700 1 --2- 172.21.15.52:0/547132617 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f86a806b620 0x7f86a8101940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:50:59.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:59.162+0000 7f86ad4fd700 1 -- 172.21.15.52:0/547132617 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f86a8108580 msgr2=0x7f86a8101e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:50:59.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:59.162+0000 7f86ad4fd700 1 --2- 172.21.15.52:0/547132617 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f86a8108580 0x7f86a8101e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:50:59.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:59.162+0000 7f86ad4fd700 1 -- 172.21.15.52:0/547132617 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86a81a7160 con 0x7f86a806ac60 2024-06-10T02:50:59.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:59.162+0000 7f86accfc700 1 --2- 172.21.15.52:0/547132617 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f86a806b620 0x7f86a8101940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T02:50:59.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:59.162+0000 7f86ad4fd700 1 --2- 172.21.15.52:0/547132617 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f86a806ac60 0x7f86a8101400 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f86a4006010 tx=0x7f86a4009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:50:59.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:59.163+0000 7f869e7fc700 1 -- 172.21.15.52:0/547132617 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86a4027070 con 0x7f86a806ac60 2024-06-10T02:50:59.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:59.163+0000 7f869e7fc700 1 -- 172.21.15.52:0/547132617 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86a4003780 con 0x7f86a806ac60 2024-06-10T02:50:59.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:59.163+0000 7f86af761700 1 -- 172.21.15.52:0/547132617 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86a81a73f0 con 0x7f86a806ac60 2024-06-10T02:50:59.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:59.163+0000 7f869e7fc700 1 -- 172.21.15.52:0/547132617 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86a4022da0 con 0x7f86a806ac60 2024-06-10T02:50:59.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:59.163+0000 7f86af761700 1 -- 172.21.15.52:0/547132617 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86a81a78a0 con 0x7f86a806ac60 2024-06-10T02:50:59.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:59.164+0000 7f86af761700 1 -- 172.21.15.52:0/547132617 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86a810aa10 con 0x7f86a806ac60 2024-06-10T02:50:59.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:59.165+0000 7f869e7fc700 1 -- 172.21.15.52:0/547132617 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f86a402a070 con 0x7f86a806ac60 2024-06-10T02:50:59.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:59.165+0000 7f869e7fc700 1 --2- 172.21.15.52:0/547132617 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f86940a2ca0 0x7f86940a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:50:59.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:59.166+0000 7f86accfc700 1 --2- 172.21.15.52:0/547132617 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f86940a2ca0 0x7f86940a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:50:59.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:59.166+0000 7f869e7fc700 1 -- 172.21.15.52:0/547132617 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9092+0+0 (secure 0 0 0) 0x7f86a40d0130 con 0x7f86a806ac60 2024-06-10T02:50:59.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:59.166+0000 7f86accfc700 1 --2- 172.21.15.52:0/547132617 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f86940a2ca0 0x7f86940a5160 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f8698005950 tx=0x7f86980058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:50:59.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:59.170+0000 7f869e7fc700 1 -- 172.21.15.52:0/547132617 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f86a405e850 con 0x7f86a806ac60 2024-06-10T02:50:59.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:59.378+0000 7f86af761700 1 -- 172.21.15.52:0/547132617 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f86a8102930 con 0x7f86a806ac60 2024-06-10T02:50:59.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:59.380+0000 7f869e7fc700 1 -- 172.21.15.52:0/547132617 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v98) ==== 74+0+24598 (secure 0 0 0) 0x7f86a4013070 con 0x7f86a806ac60 2024-06-10T02:50:59.379 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-10T02:50:59.379 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":98,"fsid":"d8676e9e-26d2-11ef-bc9b-c7b262605968","created":"2024-06-10T02:41:43.076837+0000","modified":"2024-06-10T02:50:44.731014+0000","last_up_change":"2024-06-10T02:48:57.297829+0000","last_in_change":"2024-06-10T02:48:36.865802+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-10T02:45:07.394690+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-10T02:49:57.926838+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-10T02:50:01.130393+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-10T02:50:10.655588+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e03c0691-cc5a-497a-bbc2-ce18c337f074","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":4255833408},{"type":"v1","addr":"172.21.15.52:6803","nonce":4255833408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":4255833408},{"type":"v1","addr":"172.21.15.52:6805","nonce":4255833408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":4255833408},{"type":"v1","addr":"172.21.15.52:6809","nonce":4255833408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":4255833408},{"type":"v1","addr":"172.21.15.52:6807","nonce":4255833408}]},"public_addr":"172.21.15.52:6803/4255833408","cluster_addr":"172.21.15.52:6805/4255833408","heartbeat_back_addr":"172.21.15.52:6809/4255833408","heartbeat_front_addr":"172.21.15.52:6807/4255833408","state":["exists","up"]},{"osd":1,"uuid":"a24d8da8-c5ec-44d7-ab30-56f6ce3efda7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":4116540053},{"type":"v1","addr":"172.21.15.120:6801","nonce":4116540053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":4116540053},{"type":"v1","addr":"172.21.15.120:6803","nonce":4116540053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":4116540053},{"type":"v1","addr":"172.21.15.120:6807","nonce":4116540053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":4116540053},{"type":"v1","addr":"172.21.15.120:6805","nonce":4116540053}]},"public_addr":"172.21.15.120:6801/4116540053","cluster_addr":"172.21.15.120:6803/4116540053","heartbeat_back_addr":"172.21.15.120:6807/4116540053","heartbeat_front_addr":"172.21.15.120:6805/4116540053","state":["exists","up"]},{"osd":2,"uuid":"a7162106-ccbf-45bd-bc0e-051ee43f4214","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":2624326999},{"type":"v1","addr":"172.21.15.188:6801","nonce":2624326999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":2624326999},{"type":"v1","addr":"172.21.15.188:6803","nonce":2624326999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":2624326999},{"type":"v1","addr":"172.21.15.188:6807","nonce":2624326999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":2624326999},{"type":"v1","addr":"172.21.15.188:6805","nonce":2624326999}]},"public_addr":"172.21.15.188:6801/2624326999","cluster_addr":"172.21.15.188:6803/2624326999","heartbeat_back_addr":"172.21.15.188:6807/2624326999","heartbeat_front_addr":"172.21.15.188:6805/2624326999","state":["exists","up"]},{"osd":3,"uuid":"53680244-b49a-4120-b24a-9ad7a6341dc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":116897011},{"type":"v1","addr":"172.21.15.52:6811","nonce":116897011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":116897011},{"type":"v1","addr":"172.21.15.52:6813","nonce":116897011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":116897011},{"type":"v1","addr":"172.21.15.52:6817","nonce":116897011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":116897011},{"type":"v1","addr":"172.21.15.52:6815","nonce":116897011}]},"public_addr":"172.21.15.52:6811/116897011","cluster_addr":"172.21.15.52:6813/116897011","heartbeat_back_addr":"172.21.15.52:6817/116897011","heartbeat_front_addr":"172.21.15.52:6815/116897011","state":["exists","up"]},{"osd":4,"uuid":"9f72f1f4-2ef3-49c5-8410-c108767d2ef5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":1320690621},{"type":"v1","addr":"172.21.15.120:6809","nonce":1320690621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":1320690621},{"type":"v1","addr":"172.21.15.120:6811","nonce":1320690621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":1320690621},{"type":"v1","addr":"172.21.15.120:6815","nonce":1320690621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":1320690621},{"type":"v1","addr":"172.21.15.120:6813","nonce":1320690621}]},"public_addr":"172.21.15.120:6809/1320690621","cluster_addr":"172.21.15.120:6811/1320690621","heartbeat_back_addr":"172.21.15.120:6815/1320690621","heartbeat_front_addr":"172.21.15.120:6813/1320690621","state":["exists","up"]},{"osd":5,"uuid":"001b5706-7f27-4fa4-9b69-969e6d765abc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3951947483},{"type":"v1","addr":"172.21.15.188:6809","nonce":3951947483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3951947483},{"type":"v1","addr":"172.21.15.188:6811","nonce":3951947483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3951947483},{"type":"v1","addr":"172.21.15.188:6815","nonce":3951947483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3951947483},{"type":"v1","addr":"172.21.15.188:6813","nonce":3951947483}]},"public_addr":"172.21.15.188:6809/3951947483","cluster_addr":"172.21.15.188:6811/3951947483","heartbeat_back_addr":"172.21.15.188:6815/3951947483","heartbeat_front_addr":"172.21.15.188:6813/3951947483","state":["exists","up"]},{"osd":6,"uuid":"140c903e-9b31-4a81-9ccd-2edc0a2decf9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":359084919},{"type":"v1","addr":"172.21.15.52:6819","nonce":359084919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":359084919},{"type":"v1","addr":"172.21.15.52:6821","nonce":359084919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":359084919},{"type":"v1","addr":"172.21.15.52:6825","nonce":359084919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":359084919},{"type":"v1","addr":"172.21.15.52:6823","nonce":359084919}]},"public_addr":"172.21.15.52:6819/359084919","cluster_addr":"172.21.15.52:6821/359084919","heartbeat_back_addr":"172.21.15.52:6825/359084919","heartbeat_front_addr":"172.21.15.52:6823/359084919","state":["exists","up"]},{"osd":7,"uuid":"57b735d4-5072-4ca2-a3ff-3763ad84f15a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":3737722011},{"type":"v1","addr":"172.21.15.120:6817","nonce":3737722011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6818","nonce":3737722011},{"type":"v1","addr":"172.21.15.120:6819","nonce":3737722011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":3737722011},{"type":"v1","addr":"172.21.15.120:6823","nonce":3737722011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":3737722011},{"type":"v1","addr":"172.21.15.120:6821","nonce":3737722011}]},"public_addr":"172.21.15.120:6817/3737722011","cluster_addr":"172.21.15.120:6819/3737722011","heartbeat_back_addr":"172.21.15.120:6823/3737722011","heartbeat_front_addr":"172.21.15.120:6821/3737722011","state":["exists","up"]},{"osd":8,"uuid":"4ffbb43b-854a-4eac-99cf-7af89bc65409","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1444828441},{"type":"v1","addr":"172.21.15.188:6817","nonce":1444828441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1444828441},{"type":"v1","addr":"172.21.15.188:6819","nonce":1444828441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1444828441},{"type":"v1","addr":"172.21.15.188:6823","nonce":1444828441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1444828441},{"type":"v1","addr":"172.21.15.188:6821","nonce":1444828441}]},"public_addr":"172.21.15.188:6817/1444828441","cluster_addr":"172.21.15.188:6819/1444828441","heartbeat_back_addr":"172.21.15.188:6823/1444828441","heartbeat_front_addr":"172.21.15.188:6821/1444828441","state":["exists","up"]},{"osd":9,"uuid":"5bfd985c-8acc-4d00-8561-40bbd541c07b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":924440658},{"type":"v1","addr":"172.21.15.52:6827","nonce":924440658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":924440658},{"type":"v1","addr":"172.21.15.52:6829","nonce":924440658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":924440658},{"type":"v1","addr":"172.21.15.52:6833","nonce":924440658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":924440658},{"type":"v1","addr":"172.21.15.52:6831","nonce":924440658}]},"public_addr":"172.21.15.52:6827/924440658","cluster_addr":"172.21.15.52:6829/924440658","heartbeat_back_addr":"172.21.15.52:6833/924440658","heartbeat_front_addr":"172.21.15.52:6831/924440658","state":["exists","up"]},{"osd":10,"uuid":"0180cddf-e109-42cc-829a-185604276f02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6824","nonce":1531769083},{"type":"v1","addr":"172.21.15.120:6825","nonce":1531769083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":1531769083},{"type":"v1","addr":"172.21.15.120:6827","nonce":1531769083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":1531769083},{"type":"v1","addr":"172.21.15.120:6831","nonce":1531769083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6828","nonce":1531769083},{"type":"v1","addr":"172.21.15.120:6829","nonce":1531769083}]},"public_addr":"172.21.15.120:6825/1531769083","cluster_addr":"172.21.15.120:6827/1531769083","heartbeat_back_addr":"172.21.15.120:6831/1531769083","heartbeat_front_addr":"172.21.15.120:6829/1531769083","state":["exists","up"]},{"osd":11,"uuid":"73aec750-82d1-4800-b6a0-4e305912f9ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":3769050170},{"type":"v1","addr":"172.21.15.188:6825","nonce":3769050170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":3769050170},{"type":"v1","addr":"172.21.15.188:6827","nonce":3769050170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":3769050170},{"type":"v1","addr":"172.21.15.188:6831","nonce":3769050170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":3769050170},{"type":"v1","addr":"172.21.15.188:6829","nonce":3769050170}]},"public_addr":"172.21.15.188:6825/3769050170","cluster_addr":"172.21.15.188:6827/3769050170","heartbeat_back_addr":"172.21.15.188:6831/3769050170","heartbeat_front_addr":"172.21.15.188:6829/3769050170","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:44:16.455752+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:44:40.219735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:45:03.558899+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:45:29.155004+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:45:53.981098+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:46:19.412020+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:46:45.462640+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:47:11.484578+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:47:36.879054+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:48:03.392446+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:48:29.422173+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:48:55.523423+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":4}]},{"pgid":"3.6","mappings":[{"from":3,"to":7}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.52:0/3535271671":"2024-06-11T02:42:42.872312+0000","172.21.15.52:0/1092934380":"2024-06-11T02:42:42.872312+0000","172.21.15.52:6800/896317057":"2024-06-11T02:42:42.872312+0000","172.21.15.52:6801/3934479642":"2024-06-11T02:42:15.498118+0000","172.21.15.52:6800/3934479642":"2024-06-11T02:42:15.498118+0000","172.21.15.52:0/2759694132":"2024-06-11T02:42:15.498118+0000","172.21.15.52:0/2593665631":"2024-06-11T02:42:15.498118+0000","172.21.15.52:0/3151504634":"2024-06-11T02:42:15.498118+0000","172.21.15.52:6801/896317057":"2024-06-11T02:42:42.872312+0000","172.21.15.52:0/1254480982":"2024-06-11T02:42:42.872312+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-10T02:50:59.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:59.382+0000 7f86af761700 1 -- 172.21.15.52:0/547132617 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f86940a2ca0 msgr2=0x7f86940a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:50:59.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:59.382+0000 7f86af761700 1 --2- 172.21.15.52:0/547132617 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f86940a2ca0 0x7f86940a5160 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f8698005950 tx=0x7f86980058e0 comp rx=0 tx=0).stop 2024-06-10T02:50:59.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:59.382+0000 7f86af761700 1 -- 172.21.15.52:0/547132617 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f86a806ac60 msgr2=0x7f86a8101400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:50:59.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:59.382+0000 7f86af761700 1 --2- 172.21.15.52:0/547132617 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f86a806ac60 0x7f86a8101400 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f86a4006010 tx=0x7f86a4009f80 comp rx=0 tx=0).stop 2024-06-10T02:50:59.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:59.383+0000 7f86af761700 1 -- 172.21.15.52:0/547132617 shutdown_connections 2024-06-10T02:50:59.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:59.383+0000 7f86af761700 1 --2- 172.21.15.52:0/547132617 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f86a8108580 0x7f86a8101e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:50:59.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:59.383+0000 7f86af761700 1 --2- 172.21.15.52:0/547132617 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f86940a2ca0 0x7f86940a5160 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:50:59.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:59.383+0000 7f86af761700 1 --2- 172.21.15.52:0/547132617 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f86a806ac60 0x7f86a8101400 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:50:59.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:59.383+0000 7f86af761700 1 --2- 172.21.15.52:0/547132617 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f86a806b620 0x7f86a8101940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:50:59.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:59.383+0000 7f86af761700 1 -- 172.21.15.52:0/547132617 >> 172.21.15.52:0/547132617 conn(0x7f86a8078430 msgr2=0x7f86a80ffdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:50:59.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:59.383+0000 7f86af761700 1 -- 172.21.15.52:0/547132617 shutdown_connections 2024-06-10T02:50:59.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:50:59.383+0000 7f86af761700 1 -- 172.21.15.52:0/547132617 wait complete. 2024-06-10T02:50:59.887 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:50:59 smithi120 ceph-mon[36634]: from='client.? 172.21.15.52:0/547132617' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T02:50:59.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:50:59 smithi188 ceph-mon[36599]: from='client.? 172.21.15.52:0/547132617' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T02:50:59.975 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:50:59 smithi052 ceph-mon[32362]: from='client.? 172.21.15.52:0/547132617' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T02:51:00.040 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph fs set cephfs standby_count_wanted 0 2024-06-10T02:51:00.297 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:51:00.726 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:51:00 smithi052 ceph-mon[32362]: pgmap v336: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-06-10T02:51:00.887 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:51:00 smithi120 ceph-mon[36634]: pgmap v336: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-06-10T02:51:00.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:51:00 smithi188 ceph-mon[36599]: pgmap v336: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-06-10T02:51:01.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:01.951+0000 7fb9cfaf2700 1 -- 172.21.15.52:0/262918968 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb9c80746c0 msgr2=0x7fb9c8074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:51:01.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:01.951+0000 7fb9cfaf2700 1 --2- 172.21.15.52:0/262918968 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb9c80746c0 0x7fb9c8074aa0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fb9c4009990 tx=0x7fb9c4009ca0 comp rx=0 tx=0).stop 2024-06-10T02:51:01.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:01.951+0000 7fb9cfaf2700 1 -- 172.21.15.52:0/262918968 shutdown_connections 2024-06-10T02:51:01.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:01.951+0000 7fb9cfaf2700 1 --2- 172.21.15.52:0/262918968 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb9c810ee90 0x7fb9c8111280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:51:01.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:01.951+0000 7fb9cfaf2700 1 --2- 172.21.15.52:0/262918968 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb9c80746c0 0x7fb9c8074aa0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:51:01.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:01.951+0000 7fb9cfaf2700 1 --2- 172.21.15.52:0/262918968 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb9c8074fe0 0x7fb9c810e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:51:01.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:01.951+0000 7fb9cfaf2700 1 -- 172.21.15.52:0/262918968 >> 172.21.15.52:0/262918968 conn(0x7fb9c80795f0 msgr2=0x7fb9c8079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:51:01.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:01.951+0000 7fb9cfaf2700 1 -- 172.21.15.52:0/262918968 shutdown_connections 2024-06-10T02:51:01.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:01.951+0000 7fb9cfaf2700 1 -- 172.21.15.52:0/262918968 wait complete. 2024-06-10T02:51:01.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:01.952+0000 7fb9cfaf2700 1 Processor -- start 2024-06-10T02:51:01.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:01.952+0000 7fb9cfaf2700 1 -- start start 2024-06-10T02:51:01.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:01.952+0000 7fb9cfaf2700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb9c80746c0 0x7fb9c819aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:51:01.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:01.952+0000 7fb9cfaf2700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb9c8074fe0 0x7fb9c819b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:51:01.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:01.952+0000 7fb9cfaf2700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb9c810ee90 0x7fb9c819f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:51:01.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:01.952+0000 7fb9cfaf2700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb9c8114960 con 0x7fb9c8074fe0 2024-06-10T02:51:01.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:01.952+0000 7fb9cfaf2700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fb9c81147e0 con 0x7fb9c80746c0 2024-06-10T02:51:01.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:01.952+0000 7fb9cfaf2700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fb9c8114ae0 con 0x7fb9c810ee90 2024-06-10T02:51:01.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:01.952+0000 7fb9cd08d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb9c8074fe0 0x7fb9c819b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:51:01.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:01.952+0000 7fb9cd08d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb9c8074fe0 0x7fb9c819b410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40058/0 (socket says 172.21.15.52:40058) 2024-06-10T02:51:01.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:01.952+0000 7fb9cd08d700 1 -- 172.21.15.52:0/1656183006 learned_addr learned my addr 172.21.15.52:0/1656183006 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:51:01.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:01.952+0000 7fb9cd88e700 1 --2- 172.21.15.52:0/1656183006 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb9c80746c0 0x7fb9c819aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:51:01.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:01.952+0000 7fb9ce08f700 1 --2- 172.21.15.52:0/1656183006 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb9c810ee90 0x7fb9c819f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:51:01.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:01.952+0000 7fb9cd08d700 1 -- 172.21.15.52:0/1656183006 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb9c810ee90 msgr2=0x7fb9c819f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:51:01.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:01.952+0000 7fb9cd08d700 1 --2- 172.21.15.52:0/1656183006 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb9c810ee90 0x7fb9c819f750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:51:01.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:01.952+0000 7fb9cd08d700 1 -- 172.21.15.52:0/1656183006 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb9c80746c0 msgr2=0x7fb9c819aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:51:01.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:01.952+0000 7fb9cd08d700 1 --2- 172.21.15.52:0/1656183006 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb9c80746c0 0x7fb9c819aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:51:01.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:01.952+0000 7fb9cd08d700 1 -- 172.21.15.52:0/1656183006 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9c819fe90 con 0x7fb9c8074fe0 2024-06-10T02:51:01.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:01.953+0000 7fb9cd08d700 1 --2- 172.21.15.52:0/1656183006 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb9c8074fe0 0x7fb9c819b410 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fb9b800d950 tx=0x7fb9b800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:51:01.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:01.953+0000 7fb9beffd700 1 -- 172.21.15.52:0/1656183006 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9b8016070 con 0x7fb9c8074fe0 2024-06-10T02:51:01.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:01.953+0000 7fb9cfaf2700 1 -- 172.21.15.52:0/1656183006 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9c81a0180 con 0x7fb9c8074fe0 2024-06-10T02:51:01.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:01.953+0000 7fb9beffd700 1 -- 172.21.15.52:0/1656183006 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9b8009e20 con 0x7fb9c8074fe0 2024-06-10T02:51:01.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:01.953+0000 7fb9beffd700 1 -- 172.21.15.52:0/1656183006 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9b800f620 con 0x7fb9c8074fe0 2024-06-10T02:51:01.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:01.953+0000 7fb9cfaf2700 1 -- 172.21.15.52:0/1656183006 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9c81a7a00 con 0x7fb9c8074fe0 2024-06-10T02:51:01.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:01.954+0000 7fb9cfaf2700 1 -- 172.21.15.52:0/1656183006 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9c8069180 con 0x7fb9c8074fe0 2024-06-10T02:51:01.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:01.956+0000 7fb9beffd700 1 -- 172.21.15.52:0/1656183006 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fb9b8019070 con 0x7fb9c8074fe0 2024-06-10T02:51:01.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:01.956+0000 7fb9beffd700 1 --2- 172.21.15.52:0/1656183006 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fb9b40a2ca0 0x7fb9b40a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:51:01.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:01.957+0000 7fb9cd88e700 1 --2- 172.21.15.52:0/1656183006 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fb9b40a2ca0 0x7fb9b40a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:51:01.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:01.957+0000 7fb9beffd700 1 -- 172.21.15.52:0/1656183006 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9092+0+0 (secure 0 0 0) 0x7fb9b80cf2a0 con 0x7fb9c8074fe0 2024-06-10T02:51:01.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:01.957+0000 7fb9cd88e700 1 --2- 172.21.15.52:0/1656183006 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fb9b40a2ca0 0x7fb9b40a5160 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb9c400b5c0 tx=0x7fb9c4005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:51:01.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:01.960+0000 7fb9beffd700 1 -- 172.21.15.52:0/1656183006 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fb9b809bea0 con 0x7fb9c8074fe0 2024-06-10T02:51:02.212 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:02.214+0000 7fb9cfaf2700 1 -- 172.21.15.52:0/1656183006 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fb9c819c210 con 0x7fb9c8074fe0 2024-06-10T02:51:02.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:02.572+0000 7fb9beffd700 1 -- 172.21.15.52:0/1656183006 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v18) ==== 118+0+0 (secure 0 0 0) 0x7fb9c819c210 con 0x7fb9c8074fe0 2024-06-10T02:51:02.573 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:02.574+0000 7fb9cfaf2700 1 -- 172.21.15.52:0/1656183006 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fb9b40a2ca0 msgr2=0x7fb9b40a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:51:02.573 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:02.574+0000 7fb9cfaf2700 1 --2- 172.21.15.52:0/1656183006 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fb9b40a2ca0 0x7fb9b40a5160 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb9c400b5c0 tx=0x7fb9c4005960 comp rx=0 tx=0).stop 2024-06-10T02:51:02.573 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:02.574+0000 7fb9cfaf2700 1 -- 172.21.15.52:0/1656183006 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb9c8074fe0 msgr2=0x7fb9c819b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:51:02.573 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:02.574+0000 7fb9cfaf2700 1 --2- 172.21.15.52:0/1656183006 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb9c8074fe0 0x7fb9c819b410 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fb9b800d950 tx=0x7fb9b800dc60 comp rx=0 tx=0).stop 2024-06-10T02:51:02.573 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:02.574+0000 7fb9cfaf2700 1 -- 172.21.15.52:0/1656183006 shutdown_connections 2024-06-10T02:51:02.573 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:02.574+0000 7fb9cfaf2700 1 --2- 172.21.15.52:0/1656183006 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb9c810ee90 0x7fb9c819f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:51:02.573 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:02.574+0000 7fb9cfaf2700 1 --2- 172.21.15.52:0/1656183006 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb9c80746c0 0x7fb9c819aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:51:02.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:02.574+0000 7fb9cfaf2700 1 --2- 172.21.15.52:0/1656183006 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fb9b40a2ca0 0x7fb9b40a5160 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:51:02.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:02.574+0000 7fb9cfaf2700 1 --2- 172.21.15.52:0/1656183006 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb9c8074fe0 0x7fb9c819b410 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:51:02.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:02.574+0000 7fb9cfaf2700 1 -- 172.21.15.52:0/1656183006 >> 172.21.15.52:0/1656183006 conn(0x7fb9c80795f0 msgr2=0x7fb9c8100980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:51:02.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:02.574+0000 7fb9cfaf2700 1 -- 172.21.15.52:0/1656183006 shutdown_connections 2024-06-10T02:51:02.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:02.574+0000 7fb9cfaf2700 1 -- 172.21.15.52:0/1656183006 wait complete. 2024-06-10T02:51:02.887 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:51:02 smithi120 ceph-mon[36634]: pgmap v337: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-06-10T02:51:02.887 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:51:02 smithi120 ceph-mon[36634]: from='client.? 172.21.15.52:0/1656183006' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-06-10T02:51:02.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:51:02 smithi188 ceph-mon[36599]: pgmap v337: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-06-10T02:51:02.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:51:02 smithi188 ceph-mon[36599]: from='client.? 172.21.15.52:0/1656183006' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-06-10T02:51:02.959 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:51:02 smithi052 ceph-mon[32362]: pgmap v337: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-06-10T02:51:02.959 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:51:02 smithi052 ceph-mon[32362]: from='client.? 172.21.15.52:0/1656183006' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-06-10T02:51:03.276 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2024-06-10T02:51:03.277 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph fs set cephfs max_mds 5 2024-06-10T02:51:03.531 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:51:03.887 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:51:03 smithi120 ceph-mon[36634]: from='client.? 172.21.15.52:0/1656183006' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-06-10T02:51:03.887 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:51:03 smithi120 ceph-mon[36634]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-06-10T02:51:03.888 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:51:03 smithi052 ceph-mon[32362]: from='client.? 172.21.15.52:0/1656183006' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-06-10T02:51:03.888 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:51:03 smithi052 ceph-mon[32362]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-06-10T02:51:03.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:51:03 smithi188 ceph-mon[36599]: from='client.? 172.21.15.52:0/1656183006' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-06-10T02:51:03.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:51:03 smithi188 ceph-mon[36599]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-06-10T02:51:04.693 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:51:04 smithi052 ceph-mon[32362]: pgmap v338: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-06-10T02:51:04.887 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:51:04 smithi120 ceph-mon[36634]: pgmap v338: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-06-10T02:51:04.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:51:04 smithi188 ceph-mon[36599]: pgmap v338: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-06-10T02:51:05.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:05.234+0000 7f55e2e70700 1 -- 172.21.15.52:0/2837837532 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f55dc101090 msgr2=0x7f55dc101470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:51:05.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:05.234+0000 7f55e2e70700 1 --2- 172.21.15.52:0/2837837532 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f55dc101090 0x7f55dc101470 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f55d000aaa0 tx=0x7f55d000adb0 comp rx=0 tx=0).stop 2024-06-10T02:51:05.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:05.234+0000 7f55e2e70700 1 -- 172.21.15.52:0/2837837532 shutdown_connections 2024-06-10T02:51:05.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:05.234+0000 7f55e2e70700 1 --2- 172.21.15.52:0/2837837532 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f55dc105f30 0x7f55dc10da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:51:05.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:05.234+0000 7f55e2e70700 1 --2- 172.21.15.52:0/2837837532 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f55dc101090 0x7f55dc101470 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:51:05.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:05.234+0000 7f55e2e70700 1 --2- 172.21.15.52:0/2837837532 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f55dc101a40 0x7f55dc105800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:51:05.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:05.234+0000 7f55e2e70700 1 -- 172.21.15.52:0/2837837532 >> 172.21.15.52:0/2837837532 conn(0x7f55dc0766b0 msgr2=0x7f55dc076ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:51:05.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:05.234+0000 7f55e2e70700 1 -- 172.21.15.52:0/2837837532 shutdown_connections 2024-06-10T02:51:05.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:05.234+0000 7f55e2e70700 1 -- 172.21.15.52:0/2837837532 wait complete. 2024-06-10T02:51:05.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:05.234+0000 7f55e2e70700 1 Processor -- start 2024-06-10T02:51:05.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:05.234+0000 7f55e2e70700 1 -- start start 2024-06-10T02:51:05.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:05.235+0000 7f55e2e70700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f55dc101090 0x7f55dc102830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:51:05.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:05.235+0000 7f55e2e70700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f55dc101a40 0x7f55dc102d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:51:05.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:05.235+0000 7f55e2e70700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f55dc105f30 0x7f55dc1032b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:51:05.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:05.235+0000 7f55e2e70700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f55dc110640 con 0x7f55dc105f30 2024-06-10T02:51:05.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:05.235+0000 7f55e2e70700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f55dc1104c0 con 0x7f55dc101090 2024-06-10T02:51:05.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:05.235+0000 7f55e2e70700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f55dc1107c0 con 0x7f55dc101a40 2024-06-10T02:51:05.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:05.235+0000 7f55e140d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f55dc105f30 0x7f55dc1032b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:51:05.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:05.235+0000 7f55e140d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f55dc105f30 0x7f55dc1032b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40074/0 (socket says 172.21.15.52:40074) 2024-06-10T02:51:05.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:05.235+0000 7f55e140d700 1 -- 172.21.15.52:0/794435998 learned_addr learned my addr 172.21.15.52:0/794435998 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:51:05.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:05.235+0000 7f55e0c0c700 1 --2- 172.21.15.52:0/794435998 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f55dc101090 0x7f55dc102830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:51:05.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:05.235+0000 7f55dbfff700 1 --2- 172.21.15.52:0/794435998 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f55dc101a40 0x7f55dc102d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:51:05.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:05.235+0000 7f55e140d700 1 -- 172.21.15.52:0/794435998 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f55dc101a40 msgr2=0x7f55dc102d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:51:05.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:05.235+0000 7f55e140d700 1 --2- 172.21.15.52:0/794435998 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f55dc101a40 0x7f55dc102d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:51:05.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:05.235+0000 7f55e140d700 1 -- 172.21.15.52:0/794435998 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f55dc101090 msgr2=0x7f55dc102830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:51:05.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:05.235+0000 7f55e140d700 1 --2- 172.21.15.52:0/794435998 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f55dc101090 0x7f55dc102830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:51:05.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:05.235+0000 7f55e140d700 1 -- 172.21.15.52:0/794435998 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55dc1a7140 con 0x7f55dc105f30 2024-06-10T02:51:05.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:05.235+0000 7f55e0c0c700 1 --2- 172.21.15.52:0/794435998 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f55dc101090 0x7f55dc102830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T02:51:05.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:05.235+0000 7f55e140d700 1 --2- 172.21.15.52:0/794435998 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f55dc105f30 0x7f55dc1032b0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f55d400ef30 tx=0x7f55d400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:51:05.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:05.236+0000 7f55d9ffb700 1 -- 172.21.15.52:0/794435998 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55d40092e0 con 0x7f55dc105f30 2024-06-10T02:51:05.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:05.236+0000 7f55d9ffb700 1 -- 172.21.15.52:0/794435998 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55d4004bb0 con 0x7f55dc105f30 2024-06-10T02:51:05.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:05.236+0000 7f55e2e70700 1 -- 172.21.15.52:0/794435998 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55dc1a73d0 con 0x7f55dc105f30 2024-06-10T02:51:05.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:05.236+0000 7f55d9ffb700 1 -- 172.21.15.52:0/794435998 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55d4004280 con 0x7f55dc105f30 2024-06-10T02:51:05.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:05.236+0000 7f55e2e70700 1 -- 172.21.15.52:0/794435998 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55dc1a7910 con 0x7f55dc105f30 2024-06-10T02:51:05.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:05.237+0000 7f55e2e70700 1 -- 172.21.15.52:0/794435998 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55dc069180 con 0x7f55dc105f30 2024-06-10T02:51:05.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:05.238+0000 7f55d9ffb700 1 -- 172.21.15.52:0/794435998 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f55d401b030 con 0x7f55dc105f30 2024-06-10T02:51:05.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:05.239+0000 7f55d9ffb700 1 --2- 172.21.15.52:0/794435998 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f55cc0a2c20 0x7f55cc0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:51:05.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:05.239+0000 7f55e0c0c700 1 --2- 172.21.15.52:0/794435998 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f55cc0a2c20 0x7f55cc0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:51:05.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:05.239+0000 7f55d9ffb700 1 -- 172.21.15.52:0/794435998 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9092+0+0 (secure 0 0 0) 0x7f55d40cee50 con 0x7f55dc105f30 2024-06-10T02:51:05.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:05.239+0000 7f55e0c0c700 1 --2- 172.21.15.52:0/794435998 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f55cc0a2c20 0x7f55cc0a50e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f55d000a9f0 tx=0x7f55d0005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:51:05.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:05.243+0000 7f55d9ffb700 1 -- 172.21.15.52:0/794435998 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f55d4022020 con 0x7f55dc105f30 2024-06-10T02:51:05.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:05.498+0000 7f55e2e70700 1 -- 172.21.15.52:0/794435998 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) -- 0x7f55dc02dd40 con 0x7f55dc105f30 2024-06-10T02:51:05.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:05.589+0000 7f55d9ffb700 1 -- 172.21.15.52:0/794435998 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v19) ==== 105+0+0 (secure 0 0 0) 0x7f55d4010040 con 0x7f55dc105f30 2024-06-10T02:51:05.591 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:05.592+0000 7f55e2e70700 1 -- 172.21.15.52:0/794435998 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f55cc0a2c20 msgr2=0x7f55cc0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:51:05.591 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:05.592+0000 7f55e2e70700 1 --2- 172.21.15.52:0/794435998 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f55cc0a2c20 0x7f55cc0a50e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f55d000a9f0 tx=0x7f55d0005960 comp rx=0 tx=0).stop 2024-06-10T02:51:05.591 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:05.592+0000 7f55e2e70700 1 -- 172.21.15.52:0/794435998 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f55dc105f30 msgr2=0x7f55dc1032b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:51:05.591 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:05.592+0000 7f55e2e70700 1 --2- 172.21.15.52:0/794435998 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f55dc105f30 0x7f55dc1032b0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f55d400ef30 tx=0x7f55d400c3a0 comp rx=0 tx=0).stop 2024-06-10T02:51:05.591 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:05.592+0000 7f55e2e70700 1 -- 172.21.15.52:0/794435998 shutdown_connections 2024-06-10T02:51:05.591 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:05.592+0000 7f55e2e70700 1 --2- 172.21.15.52:0/794435998 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f55cc0a2c20 0x7f55cc0a50e0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:51:05.591 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:05.592+0000 7f55e2e70700 1 --2- 172.21.15.52:0/794435998 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f55dc105f30 0x7f55dc1032b0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:51:05.591 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:05.592+0000 7f55e2e70700 1 --2- 172.21.15.52:0/794435998 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f55dc101090 0x7f55dc102830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:51:05.592 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:05.592+0000 7f55e2e70700 1 --2- 172.21.15.52:0/794435998 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f55dc101a40 0x7f55dc102d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:51:05.592 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:05.592+0000 7f55e2e70700 1 -- 172.21.15.52:0/794435998 >> 172.21.15.52:0/794435998 conn(0x7f55dc0766b0 msgr2=0x7f55dc0ff5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:51:05.592 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:05.592+0000 7f55e2e70700 1 -- 172.21.15.52:0/794435998 shutdown_connections 2024-06-10T02:51:05.592 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:05.592+0000 7f55e2e70700 1 -- 172.21.15.52:0/794435998 wait complete. 2024-06-10T02:51:05.766 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:51:05 smithi052 ceph-mon[32362]: from='client.? 172.21.15.52:0/794435998' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-06-10T02:51:05.887 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:51:05 smithi120 ceph-mon[36634]: from='client.? 172.21.15.52:0/794435998' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-06-10T02:51:05.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:51:05 smithi188 ceph-mon[36599]: from='client.? 172.21.15.52:0/794435998' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-06-10T02:51:06.226 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph fs set cephfs allow_standby_replay true 2024-06-10T02:51:06.482 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:51:06.887 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:51:06 smithi120 ceph-mon[36634]: pgmap v339: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T02:51:06.887 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:51:06 smithi120 ceph-mon[36634]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-06-10T02:51:06.887 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:51:06 smithi120 ceph-mon[36634]: from='client.? 172.21.15.52:0/794435998' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-06-10T02:51:06.888 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:51:06 smithi120 ceph-mon[36634]: daemon mds.c assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-06-10T02:51:06.888 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:51:06 smithi120 ceph-mon[36634]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-06-10T02:51:06.888 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:51:06 smithi120 ceph-mon[36634]: fsmap cephfs:2 {0=i=up:active,1=c=up:creating} 10 up:standby 2024-06-10T02:51:06.888 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:51:06 smithi120 ceph-mon[36634]: daemon mds.c is now active in filesystem cephfs as rank 1 2024-06-10T02:51:06.898 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:51:06 smithi052 ceph-mon[32362]: pgmap v339: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T02:51:06.898 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:51:06 smithi052 ceph-mon[32362]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-06-10T02:51:06.898 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:51:06 smithi052 ceph-mon[32362]: from='client.? 172.21.15.52:0/794435998' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-06-10T02:51:06.898 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:51:06 smithi052 ceph-mon[32362]: daemon mds.c assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-06-10T02:51:06.898 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:51:06 smithi052 ceph-mon[32362]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-06-10T02:51:06.898 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:51:06 smithi052 ceph-mon[32362]: fsmap cephfs:2 {0=i=up:active,1=c=up:creating} 10 up:standby 2024-06-10T02:51:06.898 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:51:06 smithi052 ceph-mon[32362]: daemon mds.c is now active in filesystem cephfs as rank 1 2024-06-10T02:51:06.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:51:06 smithi188 ceph-mon[36599]: pgmap v339: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T02:51:06.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:51:06 smithi188 ceph-mon[36599]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-06-10T02:51:06.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:51:06 smithi188 ceph-mon[36599]: from='client.? 172.21.15.52:0/794435998' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-06-10T02:51:06.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:51:06 smithi188 ceph-mon[36599]: daemon mds.c assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-06-10T02:51:06.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:51:06 smithi188 ceph-mon[36599]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-06-10T02:51:06.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:51:06 smithi188 ceph-mon[36599]: fsmap cephfs:2 {0=i=up:active,1=c=up:creating} 10 up:standby 2024-06-10T02:51:06.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:51:06 smithi188 ceph-mon[36599]: daemon mds.c is now active in filesystem cephfs as rank 1 2024-06-10T02:51:07.887 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:51:07 smithi120 ceph-mon[36634]: mds.? [v2:172.21.15.188:6836/819317726,v1:172.21.15.188:6837/819317726] up:active 2024-06-10T02:51:07.887 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:51:07 smithi120 ceph-mon[36634]: daemon mds.f assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-06-10T02:51:07.887 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:51:07 smithi120 ceph-mon[36634]: fsmap cephfs:2 {0=i=up:active,1=c=up:active} 10 up:standby 2024-06-10T02:51:07.887 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:51:07 smithi120 ceph-mon[36634]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=f=up:creating} 9 up:standby 2024-06-10T02:51:07.888 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:51:07 smithi120 ceph-mon[36634]: daemon mds.f is now active in filesystem cephfs as rank 2 2024-06-10T02:51:07.906 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:51:07 smithi052 ceph-mon[32362]: mds.? [v2:172.21.15.188:6836/819317726,v1:172.21.15.188:6837/819317726] up:active 2024-06-10T02:51:07.906 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:51:07 smithi052 ceph-mon[32362]: daemon mds.f assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-06-10T02:51:07.907 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:51:07 smithi052 ceph-mon[32362]: fsmap cephfs:2 {0=i=up:active,1=c=up:active} 10 up:standby 2024-06-10T02:51:07.907 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:51:07 smithi052 ceph-mon[32362]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=f=up:creating} 9 up:standby 2024-06-10T02:51:07.907 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:51:07 smithi052 ceph-mon[32362]: daemon mds.f is now active in filesystem cephfs as rank 2 2024-06-10T02:51:07.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:51:07 smithi188 ceph-mon[36599]: mds.? [v2:172.21.15.188:6836/819317726,v1:172.21.15.188:6837/819317726] up:active 2024-06-10T02:51:07.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:51:07 smithi188 ceph-mon[36599]: daemon mds.f assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-06-10T02:51:07.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:51:07 smithi188 ceph-mon[36599]: fsmap cephfs:2 {0=i=up:active,1=c=up:active} 10 up:standby 2024-06-10T02:51:07.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:51:07 smithi188 ceph-mon[36599]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=f=up:creating} 9 up:standby 2024-06-10T02:51:07.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:51:07 smithi188 ceph-mon[36599]: daemon mds.f is now active in filesystem cephfs as rank 2 2024-06-10T02:51:08.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:08.176+0000 7f78c3fc1700 1 -- 172.21.15.52:0/2351974017 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f78bc103120 msgr2=0x7f78bc107130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:51:08.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:08.176+0000 7f78c3fc1700 1 --2- 172.21.15.52:0/2351974017 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f78bc103120 0x7f78bc107130 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f78ac0099c0 tx=0x7f78ac009cd0 comp rx=0 tx=0).stop 2024-06-10T02:51:08.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:08.177+0000 7f78c3fc1700 1 -- 172.21.15.52:0/2351974017 shutdown_connections 2024-06-10T02:51:08.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:08.177+0000 7f78c3fc1700 1 --2- 172.21.15.52:0/2351974017 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f78bc107860 0x7f78bc10f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:51:08.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:08.177+0000 7f78c3fc1700 1 --2- 172.21.15.52:0/2351974017 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f78bc102770 0x7f78bc102b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:51:08.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:08.177+0000 7f78c3fc1700 1 --2- 172.21.15.52:0/2351974017 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f78bc103120 0x7f78bc107130 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:51:08.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:08.177+0000 7f78c3fc1700 1 -- 172.21.15.52:0/2351974017 >> 172.21.15.52:0/2351974017 conn(0x7f78bc079600 msgr2=0x7f78bc079a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:51:08.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:08.177+0000 7f78c3fc1700 1 -- 172.21.15.52:0/2351974017 shutdown_connections 2024-06-10T02:51:08.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:08.177+0000 7f78c3fc1700 1 -- 172.21.15.52:0/2351974017 wait complete. 2024-06-10T02:51:08.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:08.177+0000 7f78c3fc1700 1 Processor -- start 2024-06-10T02:51:08.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:08.177+0000 7f78c3fc1700 1 -- start start 2024-06-10T02:51:08.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:08.177+0000 7f78c3fc1700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f78bc102770 0x7f78bc19ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:51:08.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:08.178+0000 7f78c3fc1700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f78bc103120 0x7f78bc19efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:51:08.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:08.178+0000 7f78c3fc1700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f78bc107860 0x7f78bc198b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:51:08.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:08.178+0000 7f78c3fc1700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f78bc112870 con 0x7f78bc102770 2024-06-10T02:51:08.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:08.178+0000 7f78c3fc1700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f78bc1126f0 con 0x7f78bc107860 2024-06-10T02:51:08.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:08.178+0000 7f78c3fc1700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f78bc1129f0 con 0x7f78bc103120 2024-06-10T02:51:08.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:08.178+0000 7f78c1d5d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f78bc102770 0x7f78bc19ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:51:08.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:08.178+0000 7f78c1d5d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f78bc102770 0x7f78bc19ea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40092/0 (socket says 172.21.15.52:40092) 2024-06-10T02:51:08.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:08.178+0000 7f78c1d5d700 1 -- 172.21.15.52:0/656461920 learned_addr learned my addr 172.21.15.52:0/656461920 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:51:08.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:08.178+0000 7f78c155c700 1 --2- 172.21.15.52:0/656461920 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f78bc103120 0x7f78bc19efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:51:08.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:08.178+0000 7f78c255e700 1 --2- 172.21.15.52:0/656461920 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f78bc107860 0x7f78bc198b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:51:08.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:08.178+0000 7f78c1d5d700 1 -- 172.21.15.52:0/656461920 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f78bc103120 msgr2=0x7f78bc19efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:51:08.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:08.178+0000 7f78c1d5d700 1 --2- 172.21.15.52:0/656461920 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f78bc103120 0x7f78bc19efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:51:08.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:08.178+0000 7f78c1d5d700 1 -- 172.21.15.52:0/656461920 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f78bc107860 msgr2=0x7f78bc198b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:51:08.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:08.178+0000 7f78c1d5d700 1 --2- 172.21.15.52:0/656461920 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f78bc107860 0x7f78bc198b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:51:08.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:08.178+0000 7f78c1d5d700 1 -- 172.21.15.52:0/656461920 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78bc199410 con 0x7f78bc102770 2024-06-10T02:51:08.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:08.178+0000 7f78c155c700 1 --2- 172.21.15.52:0/656461920 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f78bc103120 0x7f78bc19efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T02:51:08.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:08.178+0000 7f78c1d5d700 1 --2- 172.21.15.52:0/656461920 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f78bc102770 0x7f78bc19ea60 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f78b800b530 tx=0x7f78b800b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:51:08.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:08.178+0000 7f78b2ffd700 1 -- 172.21.15.52:0/656461920 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78b8015020 con 0x7f78bc102770 2024-06-10T02:51:08.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:08.178+0000 7f78b2ffd700 1 -- 172.21.15.52:0/656461920 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f78b8011d10 con 0x7f78bc102770 2024-06-10T02:51:08.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:08.178+0000 7f78c3fc1700 1 -- 172.21.15.52:0/656461920 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78bc199700 con 0x7f78bc102770 2024-06-10T02:51:08.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:08.178+0000 7f78b2ffd700 1 -- 172.21.15.52:0/656461920 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78b8010e50 con 0x7f78bc102770 2024-06-10T02:51:08.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:08.178+0000 7f78c3fc1700 1 -- 172.21.15.52:0/656461920 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78bc1a59e0 con 0x7f78bc102770 2024-06-10T02:51:08.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:08.179+0000 7f78c3fc1700 1 -- 172.21.15.52:0/656461920 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78bc10aaa0 con 0x7f78bc102770 2024-06-10T02:51:08.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:08.181+0000 7f78b2ffd700 1 -- 172.21.15.52:0/656461920 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f78b8018020 con 0x7f78bc102770 2024-06-10T02:51:08.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:08.182+0000 7f78b2ffd700 1 --2- 172.21.15.52:0/656461920 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f78a80a2dc0 0x7f78a80a5280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:51:08.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:08.182+0000 7f78c155c700 1 --2- 172.21.15.52:0/656461920 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f78a80a2dc0 0x7f78a80a5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:51:08.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:08.182+0000 7f78b2ffd700 1 -- 172.21.15.52:0/656461920 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9092+0+0 (secure 0 0 0) 0x7f78b80cec00 con 0x7f78bc102770 2024-06-10T02:51:08.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:08.182+0000 7f78c155c700 1 --2- 172.21.15.52:0/656461920 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f78a80a2dc0 0x7f78a80a5280 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f78ac009f50 tx=0x7f78ac005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:51:08.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:08.186+0000 7f78b2ffd700 1 -- 172.21.15.52:0/656461920 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f78b80d5080 con 0x7f78bc102770 2024-06-10T02:51:08.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:08.441+0000 7f78c3fc1700 1 -- 172.21.15.52:0/656461920 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7f78bc199ff0 con 0x7f78bc102770 2024-06-10T02:51:08.607 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:08.609+0000 7f78b2ffd700 1 -- 172.21.15.52:0/656461920 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v25) ==== 121+0+0 (secure 0 0 0) 0x7f78b801f020 con 0x7f78bc102770 2024-06-10T02:51:08.611 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:08.612+0000 7f78c3fc1700 1 -- 172.21.15.52:0/656461920 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f78a80a2dc0 msgr2=0x7f78a80a5280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:51:08.611 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:08.612+0000 7f78c3fc1700 1 --2- 172.21.15.52:0/656461920 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f78a80a2dc0 0x7f78a80a5280 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f78ac009f50 tx=0x7f78ac005fd0 comp rx=0 tx=0).stop 2024-06-10T02:51:08.611 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:08.612+0000 7f78c3fc1700 1 -- 172.21.15.52:0/656461920 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f78bc102770 msgr2=0x7f78bc19ea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:51:08.611 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:08.612+0000 7f78c3fc1700 1 --2- 172.21.15.52:0/656461920 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f78bc102770 0x7f78bc19ea60 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f78b800b530 tx=0x7f78b800b8f0 comp rx=0 tx=0).stop 2024-06-10T02:51:08.611 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:08.612+0000 7f78c3fc1700 1 -- 172.21.15.52:0/656461920 shutdown_connections 2024-06-10T02:51:08.611 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:08.612+0000 7f78c3fc1700 1 --2- 172.21.15.52:0/656461920 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f78bc107860 0x7f78bc198b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:51:08.612 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:08.612+0000 7f78c3fc1700 1 --2- 172.21.15.52:0/656461920 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f78a80a2dc0 0x7f78a80a5280 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:51:08.612 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:08.612+0000 7f78c3fc1700 1 --2- 172.21.15.52:0/656461920 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f78bc102770 0x7f78bc19ea60 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:51:08.612 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:08.612+0000 7f78c3fc1700 1 --2- 172.21.15.52:0/656461920 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f78bc103120 0x7f78bc19efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:51:08.612 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:08.612+0000 7f78c3fc1700 1 -- 172.21.15.52:0/656461920 >> 172.21.15.52:0/656461920 conn(0x7f78bc079600 msgr2=0x7f78bc1089b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:51:08.612 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:08.613+0000 7f78c3fc1700 1 -- 172.21.15.52:0/656461920 shutdown_connections 2024-06-10T02:51:08.612 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:08.613+0000 7f78c3fc1700 1 -- 172.21.15.52:0/656461920 wait complete. 2024-06-10T02:51:08.725 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:51:08 smithi052 ceph-mon[32362]: pgmap v340: 129 pgs: 129 active+clean; 582 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s wr, 2 op/s 2024-06-10T02:51:08.725 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:51:08 smithi052 ceph-mon[32362]: mds.? [v2:172.21.15.188:6834/3904544259,v1:172.21.15.188:6835/3904544259] up:active 2024-06-10T02:51:08.726 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:51:08 smithi052 ceph-mon[32362]: daemon mds.b assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-06-10T02:51:08.726 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:51:08 smithi052 ceph-mon[32362]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=f=up:active} 9 up:standby 2024-06-10T02:51:08.726 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:51:08 smithi052 ceph-mon[32362]: fsmap cephfs:4 {0=i=up:active,1=c=up:active,2=f=up:active,3=b=up:creating} 8 up:standby 2024-06-10T02:51:08.726 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:51:08 smithi052 ceph-mon[32362]: daemon mds.b is now active in filesystem cephfs as rank 3 2024-06-10T02:51:08.726 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:51:08 smithi052 ceph-mon[32362]: from='client.? 172.21.15.52:0/656461920' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-06-10T02:51:08.887 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:51:08 smithi120 ceph-mon[36634]: pgmap v340: 129 pgs: 129 active+clean; 582 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s wr, 2 op/s 2024-06-10T02:51:08.887 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:51:08 smithi120 ceph-mon[36634]: mds.? [v2:172.21.15.188:6834/3904544259,v1:172.21.15.188:6835/3904544259] up:active 2024-06-10T02:51:08.887 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:51:08 smithi120 ceph-mon[36634]: daemon mds.b assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-06-10T02:51:08.887 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:51:08 smithi120 ceph-mon[36634]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=f=up:active} 9 up:standby 2024-06-10T02:51:08.887 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:51:08 smithi120 ceph-mon[36634]: fsmap cephfs:4 {0=i=up:active,1=c=up:active,2=f=up:active,3=b=up:creating} 8 up:standby 2024-06-10T02:51:08.887 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:51:08 smithi120 ceph-mon[36634]: daemon mds.b is now active in filesystem cephfs as rank 3 2024-06-10T02:51:08.887 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:51:08 smithi120 ceph-mon[36634]: from='client.? 172.21.15.52:0/656461920' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-06-10T02:51:08.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:51:08 smithi188 ceph-mon[36599]: pgmap v340: 129 pgs: 129 active+clean; 582 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s wr, 2 op/s 2024-06-10T02:51:08.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:51:08 smithi188 ceph-mon[36599]: mds.? [v2:172.21.15.188:6834/3904544259,v1:172.21.15.188:6835/3904544259] up:active 2024-06-10T02:51:08.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:51:08 smithi188 ceph-mon[36599]: daemon mds.b assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-06-10T02:51:08.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:51:08 smithi188 ceph-mon[36599]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=f=up:active} 9 up:standby 2024-06-10T02:51:08.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:51:08 smithi188 ceph-mon[36599]: fsmap cephfs:4 {0=i=up:active,1=c=up:active,2=f=up:active,3=b=up:creating} 8 up:standby 2024-06-10T02:51:08.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:51:08 smithi188 ceph-mon[36599]: daemon mds.b is now active in filesystem cephfs as rank 3 2024-06-10T02:51:08.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:51:08 smithi188 ceph-mon[36599]: from='client.? 172.21.15.52:0/656461920' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-06-10T02:51:09.256 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph fs set cephfs session_timeout 300 2024-06-10T02:51:09.508 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:51:09.860 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:51:09 smithi052 ceph-mon[32362]: from='client.? 172.21.15.52:0/656461920' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-06-10T02:51:09.860 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:51:09 smithi052 ceph-mon[32362]: mds.? [v2:172.21.15.120:6838/1672436487,v1:172.21.15.120:6839/1672436487] up:active 2024-06-10T02:51:09.860 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:51:09 smithi052 ceph-mon[32362]: daemon mds.h assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-06-10T02:51:09.860 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:51:09 smithi052 ceph-mon[32362]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-06-10T02:51:09.860 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:51:09 smithi052 ceph-mon[32362]: Cluster is now healthy 2024-06-10T02:51:09.861 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:51:09 smithi052 ceph-mon[32362]: fsmap cephfs:4 {0=i=up:active,1=c=up:active,2=f=up:active,3=b=up:active} 8 up:standby 2024-06-10T02:51:09.861 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:51:09 smithi052 ceph-mon[32362]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-06-10T02:51:09.861 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:51:09 smithi052 ceph-mon[32362]: daemon mds.h is now active in filesystem cephfs as rank 4 2024-06-10T02:51:09.887 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:51:09 smithi120 ceph-mon[36634]: from='client.? 172.21.15.52:0/656461920' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-06-10T02:51:09.887 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:51:09 smithi120 ceph-mon[36634]: mds.? [v2:172.21.15.120:6838/1672436487,v1:172.21.15.120:6839/1672436487] up:active 2024-06-10T02:51:09.887 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:51:09 smithi120 ceph-mon[36634]: daemon mds.h assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-06-10T02:51:09.887 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:51:09 smithi120 ceph-mon[36634]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-06-10T02:51:09.887 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:51:09 smithi120 ceph-mon[36634]: Cluster is now healthy 2024-06-10T02:51:09.887 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:51:09 smithi120 ceph-mon[36634]: fsmap cephfs:4 {0=i=up:active,1=c=up:active,2=f=up:active,3=b=up:active} 8 up:standby 2024-06-10T02:51:09.887 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:51:09 smithi120 ceph-mon[36634]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-06-10T02:51:09.887 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:51:09 smithi120 ceph-mon[36634]: daemon mds.h is now active in filesystem cephfs as rank 4 2024-06-10T02:51:09.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:51:09 smithi188 ceph-mon[36599]: from='client.? 172.21.15.52:0/656461920' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-06-10T02:51:09.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:51:09 smithi188 ceph-mon[36599]: mds.? [v2:172.21.15.120:6838/1672436487,v1:172.21.15.120:6839/1672436487] up:active 2024-06-10T02:51:09.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:51:09 smithi188 ceph-mon[36599]: daemon mds.h assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-06-10T02:51:09.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:51:09 smithi188 ceph-mon[36599]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-06-10T02:51:09.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:51:09 smithi188 ceph-mon[36599]: Cluster is now healthy 2024-06-10T02:51:09.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:51:09 smithi188 ceph-mon[36599]: fsmap cephfs:4 {0=i=up:active,1=c=up:active,2=f=up:active,3=b=up:active} 8 up:standby 2024-06-10T02:51:09.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:51:09 smithi188 ceph-mon[36599]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-06-10T02:51:09.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:51:09 smithi188 ceph-mon[36599]: daemon mds.h is now active in filesystem cephfs as rank 4 2024-06-10T02:51:10.887 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:51:10 smithi120 ceph-mon[36634]: pgmap v341: 129 pgs: 129 active+clean; 583 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s wr, 6 op/s 2024-06-10T02:51:10.887 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:51:10 smithi120 ceph-mon[36634]: mds.? [v2:172.21.15.120:6836/4132440952,v1:172.21.15.120:6837/4132440952] up:active 2024-06-10T02:51:10.887 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:51:10 smithi120 ceph-mon[36634]: fsmap cephfs:5 7 up:standby 5 up:active 2024-06-10T02:51:10.887 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:51:10 smithi120 ceph-mon[36634]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-10T02:51:10.917 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:51:10 smithi052 ceph-mon[32362]: pgmap v341: 129 pgs: 129 active+clean; 583 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s wr, 6 op/s 2024-06-10T02:51:10.918 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:51:10 smithi052 ceph-mon[32362]: mds.? [v2:172.21.15.120:6836/4132440952,v1:172.21.15.120:6837/4132440952] up:active 2024-06-10T02:51:10.918 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:51:10 smithi052 ceph-mon[32362]: fsmap cephfs:5 7 up:standby 5 up:active 2024-06-10T02:51:10.918 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:51:10 smithi052 ceph-mon[32362]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-10T02:51:10.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:51:10 smithi188 ceph-mon[36599]: pgmap v341: 129 pgs: 129 active+clean; 583 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s wr, 6 op/s 2024-06-10T02:51:10.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:51:10 smithi188 ceph-mon[36599]: mds.? [v2:172.21.15.120:6836/4132440952,v1:172.21.15.120:6837/4132440952] up:active 2024-06-10T02:51:10.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:51:10 smithi188 ceph-mon[36599]: fsmap cephfs:5 7 up:standby 5 up:active 2024-06-10T02:51:10.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:51:10 smithi188 ceph-mon[36599]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-10T02:51:11.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:11.159+0000 7f2bf0654700 1 -- 172.21.15.52:0/557415821 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2be8101390 msgr2=0x7f2be8101810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:51:11.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:11.159+0000 7f2bf0654700 1 --2- 172.21.15.52:0/557415821 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2be8101390 0x7f2be8101810 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f2bdc009990 tx=0x7f2bdc009ca0 comp rx=0 tx=0).stop 2024-06-10T02:51:11.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:11.159+0000 7f2bf0654700 1 -- 172.21.15.52:0/557415821 shutdown_connections 2024-06-10T02:51:11.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:11.159+0000 7f2bf0654700 1 --2- 172.21.15.52:0/557415821 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2be8101d50 0x7f2be810e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:51:11.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:11.159+0000 7f2bf0654700 1 --2- 172.21.15.52:0/557415821 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2be8107400 0x7f2be81077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:51:11.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:11.159+0000 7f2bf0654700 1 --2- 172.21.15.52:0/557415821 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2be8101390 0x7f2be8101810 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:51:11.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:11.159+0000 7f2bf0654700 1 -- 172.21.15.52:0/557415821 >> 172.21.15.52:0/557415821 conn(0x7f2be8078340 msgr2=0x7f2be8078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:51:11.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:11.159+0000 7f2bf0654700 1 -- 172.21.15.52:0/557415821 shutdown_connections 2024-06-10T02:51:11.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:11.159+0000 7f2bf0654700 1 -- 172.21.15.52:0/557415821 wait complete. 2024-06-10T02:51:11.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:11.160+0000 7f2bf0654700 1 Processor -- start 2024-06-10T02:51:11.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:11.160+0000 7f2bf0654700 1 -- start start 2024-06-10T02:51:11.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:11.160+0000 7f2bf0654700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2be8101390 0x7f2be819e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:51:11.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:11.160+0000 7f2bf0654700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2be8101d50 0x7f2be819edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:51:11.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:11.160+0000 7f2bf0654700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2be8107400 0x7f2be8198a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:51:11.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:11.160+0000 7f2bf0654700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f2be81116f0 con 0x7f2be8101390 2024-06-10T02:51:11.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:11.160+0000 7f2bf0654700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f2be8111570 con 0x7f2be8107400 2024-06-10T02:51:11.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:11.160+0000 7f2bf0654700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f2be8111870 con 0x7f2be8101d50 2024-06-10T02:51:11.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:11.160+0000 7f2bee3f0700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2be8101390 0x7f2be819e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:51:11.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:11.160+0000 7f2bee3f0700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2be8101390 0x7f2be819e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49534/0 (socket says 172.21.15.52:49534) 2024-06-10T02:51:11.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:11.160+0000 7f2bee3f0700 1 -- 172.21.15.52:0/3247217168 learned_addr learned my addr 172.21.15.52:0/3247217168 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:51:11.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:11.160+0000 7f2beebf1700 1 --2- 172.21.15.52:0/3247217168 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2be8107400 0x7f2be8198a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:51:11.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:11.160+0000 7f2bedbef700 1 --2- 172.21.15.52:0/3247217168 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2be8101d50 0x7f2be819edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:51:11.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:11.160+0000 7f2bee3f0700 1 -- 172.21.15.52:0/3247217168 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2be8101d50 msgr2=0x7f2be819edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:51:11.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:11.160+0000 7f2bee3f0700 1 --2- 172.21.15.52:0/3247217168 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2be8101d50 0x7f2be819edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:51:11.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:11.160+0000 7f2bee3f0700 1 -- 172.21.15.52:0/3247217168 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2be8107400 msgr2=0x7f2be8198a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:51:11.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:11.160+0000 7f2bee3f0700 1 --2- 172.21.15.52:0/3247217168 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2be8107400 0x7f2be8198a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:51:11.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:11.160+0000 7f2bee3f0700 1 -- 172.21.15.52:0/3247217168 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2be81991d0 con 0x7f2be8101390 2024-06-10T02:51:11.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:11.161+0000 7f2bedbef700 1 --2- 172.21.15.52:0/3247217168 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2be8101d50 0x7f2be819edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T02:51:11.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:11.161+0000 7f2bee3f0700 1 --2- 172.21.15.52:0/3247217168 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2be8101390 0x7f2be819e8b0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f2be000ada0 tx=0x7f2be000c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:51:11.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:11.161+0000 7f2be77fe700 1 -- 172.21.15.52:0/3247217168 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2be0017020 con 0x7f2be8101390 2024-06-10T02:51:11.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:11.161+0000 7f2be77fe700 1 -- 172.21.15.52:0/3247217168 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2be0004bb0 con 0x7f2be8101390 2024-06-10T02:51:11.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:11.161+0000 7f2bf0654700 1 -- 172.21.15.52:0/3247217168 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2be81994c0 con 0x7f2be8101390 2024-06-10T02:51:11.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:11.161+0000 7f2be77fe700 1 -- 172.21.15.52:0/3247217168 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2be0005460 con 0x7f2be8101390 2024-06-10T02:51:11.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:11.161+0000 7f2bf0654700 1 -- 172.21.15.52:0/3247217168 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2be81a5780 con 0x7f2be8101390 2024-06-10T02:51:11.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:11.162+0000 7f2bf0654700 1 -- 172.21.15.52:0/3247217168 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2be8069180 con 0x7f2be8101390 2024-06-10T02:51:11.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:11.163+0000 7f2be77fe700 1 -- 172.21.15.52:0/3247217168 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f2be000c980 con 0x7f2be8101390 2024-06-10T02:51:11.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:11.164+0000 7f2be77fe700 1 --2- 172.21.15.52:0/3247217168 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f2bd00a2c20 0x7f2bd00a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:51:11.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:11.164+0000 7f2bedbef700 1 --2- 172.21.15.52:0/3247217168 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f2bd00a2c20 0x7f2bd00a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:51:11.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:11.164+0000 7f2be77fe700 1 -- 172.21.15.52:0/3247217168 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9092+0+0 (secure 0 0 0) 0x7f2be00cf150 con 0x7f2be8101390 2024-06-10T02:51:11.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:11.164+0000 7f2bedbef700 1 --2- 172.21.15.52:0/3247217168 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f2bd00a2c20 0x7f2bd00a50e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f2bdc00b5c0 tx=0x7f2bdc005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:51:11.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:11.167+0000 7f2be77fe700 1 -- 172.21.15.52:0/3247217168 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f2be0021020 con 0x7f2be8101390 2024-06-10T02:51:11.418 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:11.419+0000 7f2bf0654700 1 -- 172.21.15.52:0/3247217168 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7f2be81a59b0 con 0x7f2be8101390 2024-06-10T02:51:11.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:11.631+0000 7f2be77fe700 1 -- 172.21.15.52:0/3247217168 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v29) ==== 115+0+0 (secure 0 0 0) 0x7f2be009bde0 con 0x7f2be8101390 2024-06-10T02:51:11.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:11.634+0000 7f2bf0654700 1 -- 172.21.15.52:0/3247217168 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f2bd00a2c20 msgr2=0x7f2bd00a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:51:11.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:11.634+0000 7f2bf0654700 1 --2- 172.21.15.52:0/3247217168 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f2bd00a2c20 0x7f2bd00a50e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f2bdc00b5c0 tx=0x7f2bdc005fb0 comp rx=0 tx=0).stop 2024-06-10T02:51:11.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:11.634+0000 7f2bf0654700 1 -- 172.21.15.52:0/3247217168 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2be8101390 msgr2=0x7f2be819e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:51:11.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:11.634+0000 7f2bf0654700 1 --2- 172.21.15.52:0/3247217168 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2be8101390 0x7f2be819e8b0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f2be000ada0 tx=0x7f2be000c330 comp rx=0 tx=0).stop 2024-06-10T02:51:11.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:11.634+0000 7f2bf0654700 1 -- 172.21.15.52:0/3247217168 shutdown_connections 2024-06-10T02:51:11.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:11.634+0000 7f2bf0654700 1 --2- 172.21.15.52:0/3247217168 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2be8107400 0x7f2be8198a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:51:11.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:11.634+0000 7f2bf0654700 1 --2- 172.21.15.52:0/3247217168 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f2bd00a2c20 0x7f2bd00a50e0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:51:11.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:11.634+0000 7f2bf0654700 1 --2- 172.21.15.52:0/3247217168 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2be8101390 0x7f2be819e8b0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:51:11.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:11.634+0000 7f2bf0654700 1 --2- 172.21.15.52:0/3247217168 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2be8101d50 0x7f2be819edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:51:11.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:11.634+0000 7f2bf0654700 1 -- 172.21.15.52:0/3247217168 >> 172.21.15.52:0/3247217168 conn(0x7f2be8078340 msgr2=0x7f2be8100580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:51:11.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:11.634+0000 7f2bf0654700 1 -- 172.21.15.52:0/3247217168 shutdown_connections 2024-06-10T02:51:11.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:11.634+0000 7f2bf0654700 1 -- 172.21.15.52:0/3247217168 wait complete. 2024-06-10T02:51:11.725 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:51:11 smithi052 ceph-mon[32362]: from='client.? 172.21.15.52:0/3247217168' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-06-10T02:51:11.887 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:51:11 smithi120 ceph-mon[36634]: from='client.? 172.21.15.52:0/3247217168' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-06-10T02:51:11.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:51:11 smithi188 ceph-mon[36599]: from='client.? 172.21.15.52:0/3247217168' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-06-10T02:51:12.337 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--size 25000000000'} subvols for filesystem 'cephfs' 2024-06-10T02:51:12.338 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph fs subvolumegroup create cephfs qa 2024-06-10T02:51:12.596 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:51:12.887 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:51:12 smithi120 ceph-mon[36634]: pgmap v342: 129 pgs: 129 active+clean; 583 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s rd, 2.0 KiB/s wr, 8 op/s 2024-06-10T02:51:12.887 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:51:12 smithi120 ceph-mon[36634]: from='client.? 172.21.15.52:0/3247217168' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-06-10T02:51:12.887 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:51:12 smithi120 ceph-mon[36634]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-10T02:51:12.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:51:12 smithi188 ceph-mon[36599]: pgmap v342: 129 pgs: 129 active+clean; 583 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s rd, 2.0 KiB/s wr, 8 op/s 2024-06-10T02:51:12.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:51:12 smithi188 ceph-mon[36599]: from='client.? 172.21.15.52:0/3247217168' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-06-10T02:51:12.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:51:12 smithi188 ceph-mon[36599]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-10T02:51:12.966 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:51:12 smithi052 ceph-mon[32362]: pgmap v342: 129 pgs: 129 active+clean; 583 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s rd, 2.0 KiB/s wr, 8 op/s 2024-06-10T02:51:12.967 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:51:12 smithi052 ceph-mon[32362]: from='client.? 172.21.15.52:0/3247217168' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-06-10T02:51:12.967 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:51:12 smithi052 ceph-mon[32362]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-10T02:51:14.296 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:14.296+0000 7f831e04e700 1 -- 172.21.15.52:0/1213187152 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8318101d50 msgr2=0x7f831810e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:51:14.296 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:14.296+0000 7f831e04e700 1 --2- 172.21.15.52:0/1213187152 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8318101d50 0x7f831810e240 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f8310009990 tx=0x7f8310009ca0 comp rx=0 tx=0).stop 2024-06-10T02:51:14.296 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:14.296+0000 7f831e04e700 1 -- 172.21.15.52:0/1213187152 shutdown_connections 2024-06-10T02:51:14.296 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:14.296+0000 7f831e04e700 1 --2- 172.21.15.52:0/1213187152 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8318101d50 0x7f831810e240 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:51:14.296 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:14.296+0000 7f831e04e700 1 --2- 172.21.15.52:0/1213187152 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8318107400 0x7f83181077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:51:14.296 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:14.296+0000 7f831e04e700 1 --2- 172.21.15.52:0/1213187152 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8318101390 0x7f8318101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:51:14.296 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:14.296+0000 7f831e04e700 1 -- 172.21.15.52:0/1213187152 >> 172.21.15.52:0/1213187152 conn(0x7f8318078340 msgr2=0x7f8318078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:51:14.297 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:14.296+0000 7f831e04e700 1 -- 172.21.15.52:0/1213187152 shutdown_connections 2024-06-10T02:51:14.297 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:14.296+0000 7f831e04e700 1 -- 172.21.15.52:0/1213187152 wait complete. 2024-06-10T02:51:14.297 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:14.297+0000 7f831e04e700 1 Processor -- start 2024-06-10T02:51:14.297 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:14.297+0000 7f831e04e700 1 -- start start 2024-06-10T02:51:14.298 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:14.298+0000 7f831e04e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8318101390 0x7f8318199950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:51:14.298 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:14.298+0000 7f831e04e700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8318101d50 0x7f8318199e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:51:14.298 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:14.298+0000 7f831e04e700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8318107400 0x7f83181a49d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:51:14.299 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:14.298+0000 7f831e04e700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f8318111670 con 0x7f8318101390 2024-06-10T02:51:14.299 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:14.298+0000 7f831e04e700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f83181114f0 con 0x7f8318107400 2024-06-10T02:51:14.299 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:14.298+0000 7f831e04e700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f83181117f0 con 0x7f8318101d50 2024-06-10T02:51:14.299 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:14.298+0000 7f83177fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8318101390 0x7f8318199950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:51:14.299 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:14.298+0000 7f83177fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8318101390 0x7f8318199950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49538/0 (socket says 172.21.15.52:49538) 2024-06-10T02:51:14.299 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:14.298+0000 7f83177fe700 1 -- 172.21.15.52:0/3346157935 learned_addr learned my addr 172.21.15.52:0/3346157935 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:51:14.299 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:14.298+0000 7f8317fff700 1 --2- 172.21.15.52:0/3346157935 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8318107400 0x7f83181a49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:51:14.299 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:14.298+0000 7f8316ffd700 1 --2- 172.21.15.52:0/3346157935 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8318101d50 0x7f8318199e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:51:14.299 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:14.298+0000 7f83177fe700 1 -- 172.21.15.52:0/3346157935 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8318101d50 msgr2=0x7f8318199e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:51:14.299 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:14.298+0000 7f83177fe700 1 --2- 172.21.15.52:0/3346157935 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8318101d50 0x7f8318199e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:51:14.299 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:14.298+0000 7f83177fe700 1 -- 172.21.15.52:0/3346157935 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8318107400 msgr2=0x7f83181a49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:51:14.300 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:14.298+0000 7f83177fe700 1 --2- 172.21.15.52:0/3346157935 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8318107400 0x7f83181a49d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:51:14.300 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:14.298+0000 7f83177fe700 1 -- 172.21.15.52:0/3346157935 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83181a4f10 con 0x7f8318101390 2024-06-10T02:51:14.300 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:14.299+0000 7f83177fe700 1 --2- 172.21.15.52:0/3346157935 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8318101390 0x7f8318199950 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f8308007e20 tx=0x7f830800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:51:14.300 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:14.299+0000 7f8314ff9700 1 -- 172.21.15.52:0/3346157935 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8308016020 con 0x7f8318101390 2024-06-10T02:51:14.300 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:14.299+0000 7f831e04e700 1 -- 172.21.15.52:0/3346157935 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83181a51a0 con 0x7f8318101390 2024-06-10T02:51:14.300 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:14.299+0000 7f8314ff9700 1 -- 172.21.15.52:0/3346157935 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f830800ad40 con 0x7f8318101390 2024-06-10T02:51:14.300 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:14.299+0000 7f8314ff9700 1 -- 172.21.15.52:0/3346157935 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8308011700 con 0x7f8318101390 2024-06-10T02:51:14.300 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:14.299+0000 7f831e04e700 1 -- 172.21.15.52:0/3346157935 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83181a5700 con 0x7f8318101390 2024-06-10T02:51:14.301 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:14.300+0000 7f831e04e700 1 -- 172.21.15.52:0/3346157935 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8318069180 con 0x7f8318101390 2024-06-10T02:51:14.301 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:14.301+0000 7f8314ff9700 1 -- 172.21.15.52:0/3346157935 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f8308019020 con 0x7f8318101390 2024-06-10T02:51:14.301 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:14.302+0000 7f8314ff9700 1 --2- 172.21.15.52:0/3346157935 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f83000a2c20 0x7f83000a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:51:14.301 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:14.302+0000 7f8316ffd700 1 --2- 172.21.15.52:0/3346157935 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f83000a2c20 0x7f83000a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:51:14.302 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:14.302+0000 7f8314ff9700 1 -- 172.21.15.52:0/3346157935 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9092+0+0 (secure 0 0 0) 0x7f83080cf0b0 con 0x7f8318101390 2024-06-10T02:51:14.302 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:14.302+0000 7f8316ffd700 1 --2- 172.21.15.52:0/3346157935 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f83000a2c20 0x7f83000a50e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f830c005950 tx=0x7f830c0058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:51:14.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:14.306+0000 7f8314ff9700 1 -- 172.21.15.52:0/3346157935 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f8308019310 con 0x7f8318101390 2024-06-10T02:51:14.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:14.540+0000 7f831e04e700 1 -- 172.21.15.52:0/3346157935 --> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f8318100390 con 0x7f83000a2c20 2024-06-10T02:51:14.545 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 10 02:51:14 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mgr-x[32595]: 2024-06-10T02:51:14.545+0000 7fe28db65700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-10T02:51:14.546 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 10 02:51:14 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mgr-x[32595]: 2024-06-10T02:51:14.545+0000 7fe28db65700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-10T02:51:14.546 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 10 02:51:14 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mgr-x[32595]: 2024-06-10T02:51:14.545+0000 7fe28db65700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-10T02:51:14.546 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 10 02:51:14 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mgr-x[32595]: 2024-06-10T02:51:14.545+0000 7fe28db65700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-10T02:51:14.546 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 10 02:51:14 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mgr-x[32595]: 2024-06-10T02:51:14.545+0000 7fe28db65700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-10T02:51:14.573 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:14.574+0000 7f8314ff9700 1 -- 172.21.15.52:0/3346157935 <== mgr.14150 v2:172.21.15.52:6800/3214763969 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f8318100390 con 0x7f83000a2c20 2024-06-10T02:51:14.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:14.576+0000 7f831e04e700 1 -- 172.21.15.52:0/3346157935 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f83000a2c20 msgr2=0x7f83000a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:51:14.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:14.577+0000 7f831e04e700 1 --2- 172.21.15.52:0/3346157935 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f83000a2c20 0x7f83000a50e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f830c005950 tx=0x7f830c0058e0 comp rx=0 tx=0).stop 2024-06-10T02:51:14.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:14.577+0000 7f831e04e700 1 -- 172.21.15.52:0/3346157935 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8318101390 msgr2=0x7f8318199950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:51:14.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:14.577+0000 7f831e04e700 1 --2- 172.21.15.52:0/3346157935 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8318101390 0x7f8318199950 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f8308007e20 tx=0x7f830800a340 comp rx=0 tx=0).stop 2024-06-10T02:51:14.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:14.577+0000 7f831e04e700 1 -- 172.21.15.52:0/3346157935 shutdown_connections 2024-06-10T02:51:14.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:14.577+0000 7f831e04e700 1 --2- 172.21.15.52:0/3346157935 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8318107400 0x7f83181a49d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:51:14.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:14.577+0000 7f831e04e700 1 --2- 172.21.15.52:0/3346157935 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f83000a2c20 0x7f83000a50e0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:51:14.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:14.577+0000 7f831e04e700 1 --2- 172.21.15.52:0/3346157935 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8318101390 0x7f8318199950 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:51:14.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:14.577+0000 7f831e04e700 1 --2- 172.21.15.52:0/3346157935 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8318101d50 0x7f8318199e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:51:14.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:14.577+0000 7f831e04e700 1 -- 172.21.15.52:0/3346157935 >> 172.21.15.52:0/3346157935 conn(0x7f8318078340 msgr2=0x7f83180ffc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:51:14.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:14.577+0000 7f831e04e700 1 -- 172.21.15.52:0/3346157935 shutdown_connections 2024-06-10T02:51:14.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:14.577+0000 7f831e04e700 1 -- 172.21.15.52:0/3346157935 wait complete. 2024-06-10T02:51:14.890 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:51:14 smithi120 ceph-mon[36634]: pgmap v343: 129 pgs: 129 active+clean; 587 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 4.0 KiB/s wr, 17 op/s 2024-06-10T02:51:14.974 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:51:14 smithi052 ceph-mon[32362]: pgmap v343: 129 pgs: 129 active+clean; 587 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 4.0 KiB/s wr, 17 op/s 2024-06-10T02:51:14.975 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:51:14 smithi188 ceph-mon[36599]: pgmap v343: 129 pgs: 129 active+clean; 587 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 4.0 KiB/s wr, 17 op/s 2024-06-10T02:51:15.220 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph fs subvolume create cephfs sv_0 --group_name qa 2024-06-10T02:51:15.473 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:51:15.901 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:51:15 smithi052 ceph-mon[32362]: from='client.14763 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T02:51:15.954 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:51:15 smithi188 ceph-mon[36599]: from='client.14763 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T02:51:16.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:51:15 smithi120 ceph-mon[36634]: from='client.14763 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T02:51:16.842 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:51:16 smithi052 ceph-mon[32362]: pgmap v344: 129 pgs: 129 active+clean; 587 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 4.0 KiB/s wr, 18 op/s 2024-06-10T02:51:16.842 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:51:16 smithi052 ceph-mon[32362]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-06-10T02:51:16.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:51:16 smithi188 ceph-mon[36599]: pgmap v344: 129 pgs: 129 active+clean; 587 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 4.0 KiB/s wr, 18 op/s 2024-06-10T02:51:16.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:51:16 smithi188 ceph-mon[36599]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-06-10T02:51:17.096 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:17.097+0000 7fe65c310700 1 -- 172.21.15.52:0/4079362127 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe654101d30 msgr2=0x7fe65410e210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:51:17.097 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:17.097+0000 7fe65c310700 1 --2- 172.21.15.52:0/4079362127 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe654101d30 0x7fe65410e210 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fe64c00b0a0 tx=0x7fe64c00b3b0 comp rx=0 tx=0).stop 2024-06-10T02:51:17.097 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:17.097+0000 7fe65c310700 1 -- 172.21.15.52:0/4079362127 shutdown_connections 2024-06-10T02:51:17.097 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:17.097+0000 7fe65c310700 1 --2- 172.21.15.52:0/4079362127 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe654101d30 0x7fe65410e210 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:51:17.097 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:17.097+0000 7fe65c310700 1 --2- 172.21.15.52:0/4079362127 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe6541073d0 0x7fe6541077b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:51:17.097 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:17.097+0000 7fe65c310700 1 --2- 172.21.15.52:0/4079362127 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe654101390 0x7fe6541017f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:51:17.097 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:17.097+0000 7fe65c310700 1 -- 172.21.15.52:0/4079362127 >> 172.21.15.52:0/4079362127 conn(0x7fe6540782c0 msgr2=0x7fe6540786d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:51:17.097 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:17.097+0000 7fe65c310700 1 -- 172.21.15.52:0/4079362127 shutdown_connections 2024-06-10T02:51:17.097 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:17.097+0000 7fe65c310700 1 -- 172.21.15.52:0/4079362127 wait complete. 2024-06-10T02:51:17.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:17.098+0000 7fe65c310700 1 Processor -- start 2024-06-10T02:51:17.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:17.098+0000 7fe65c310700 1 -- start start 2024-06-10T02:51:17.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:17.099+0000 7fe65c310700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe654101390 0x7fe6541978f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:51:17.100 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:17.099+0000 7fe65c310700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe654101d30 0x7fe654197e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:51:17.100 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:17.099+0000 7fe65c310700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe6541073d0 0x7fe654198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:51:17.100 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:17.099+0000 7fe65c310700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fe6541116e0 con 0x7fe6541073d0 2024-06-10T02:51:17.100 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:17.099+0000 7fe65c310700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe654111560 con 0x7fe654101390 2024-06-10T02:51:17.100 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:17.099+0000 7fe65c310700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fe654111860 con 0x7fe654101d30 2024-06-10T02:51:17.100 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:17.099+0000 7fe65a8ad700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe6541073d0 0x7fe654198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:51:17.100 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:17.099+0000 7fe65a8ad700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe6541073d0 0x7fe654198370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49568/0 (socket says 172.21.15.52:49568) 2024-06-10T02:51:17.100 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:17.099+0000 7fe65a8ad700 1 -- 172.21.15.52:0/885651804 learned_addr learned my addr 172.21.15.52:0/885651804 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:51:17.100 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:17.099+0000 7fe6598ab700 1 --2- 172.21.15.52:0/885651804 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe654101d30 0x7fe654197e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:51:17.100 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:17.099+0000 7fe65a0ac700 1 --2- 172.21.15.52:0/885651804 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe654101390 0x7fe6541978f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:51:17.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:17.099+0000 7fe65a8ad700 1 -- 172.21.15.52:0/885651804 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe654101d30 msgr2=0x7fe654197e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:51:17.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:17.099+0000 7fe65a8ad700 1 --2- 172.21.15.52:0/885651804 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe654101d30 0x7fe654197e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:51:17.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:17.099+0000 7fe65a8ad700 1 -- 172.21.15.52:0/885651804 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe654101390 msgr2=0x7fe6541978f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:51:17.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:17.099+0000 7fe65a8ad700 1 --2- 172.21.15.52:0/885651804 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe654101390 0x7fe6541978f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:51:17.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:17.099+0000 7fe65a8ad700 1 -- 172.21.15.52:0/885651804 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6541a7160 con 0x7fe6541073d0 2024-06-10T02:51:17.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:17.099+0000 7fe6598ab700 1 --2- 172.21.15.52:0/885651804 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe654101d30 0x7fe654197e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T02:51:17.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:17.099+0000 7fe65a0ac700 1 --2- 172.21.15.52:0/885651804 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe654101390 0x7fe6541978f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T02:51:17.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:17.099+0000 7fe65a8ad700 1 --2- 172.21.15.52:0/885651804 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe6541073d0 0x7fe654198370 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fe64c00d010 tx=0x7fe64c00be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:51:17.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:17.100+0000 7fe64b7fe700 1 -- 172.21.15.52:0/885651804 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe64c021070 con 0x7fe6541073d0 2024-06-10T02:51:17.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:17.100+0000 7fe65c310700 1 -- 172.21.15.52:0/885651804 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6541a73f0 con 0x7fe6541073d0 2024-06-10T02:51:17.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:17.100+0000 7fe64b7fe700 1 -- 172.21.15.52:0/885651804 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe64c007d30 con 0x7fe6541073d0 2024-06-10T02:51:17.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:17.100+0000 7fe64b7fe700 1 -- 172.21.15.52:0/885651804 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe64c01d450 con 0x7fe6541073d0 2024-06-10T02:51:17.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:17.100+0000 7fe65c310700 1 -- 172.21.15.52:0/885651804 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6541a78a0 con 0x7fe6541073d0 2024-06-10T02:51:17.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:17.101+0000 7fe65c310700 1 -- 172.21.15.52:0/885651804 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe654069180 con 0x7fe6541073d0 2024-06-10T02:51:17.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:17.102+0000 7fe64b7fe700 1 -- 172.21.15.52:0/885651804 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fe64c004120 con 0x7fe6541073d0 2024-06-10T02:51:17.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:17.103+0000 7fe64b7fe700 1 --2- 172.21.15.52:0/885651804 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fe6400a2cb0 0x7fe6400a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:51:17.103 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:17.103+0000 7fe65a0ac700 1 --2- 172.21.15.52:0/885651804 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fe6400a2cb0 0x7fe6400a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:51:17.103 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:17.103+0000 7fe64b7fe700 1 -- 172.21.15.52:0/885651804 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9092+0+0 (secure 0 0 0) 0x7fe64c0d0780 con 0x7fe6541073d0 2024-06-10T02:51:17.103 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:17.103+0000 7fe65a0ac700 1 --2- 172.21.15.52:0/885651804 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fe6400a2cb0 0x7fe6400a5170 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fe6500099d0 tx=0x7fe650008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:51:17.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:17.107+0000 7fe64b7fe700 1 -- 172.21.15.52:0/885651804 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fe64c09d2b0 con 0x7fe6541073d0 2024-06-10T02:51:17.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:51:16 smithi120 ceph-mon[36634]: pgmap v344: 129 pgs: 129 active+clean; 587 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 4.0 KiB/s wr, 18 op/s 2024-06-10T02:51:17.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:51:16 smithi120 ceph-mon[36634]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-06-10T02:51:17.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:17.347+0000 7fe65c310700 1 -- 172.21.15.52:0/885651804 --> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fe6540628d0 con 0x7fe6400a2cb0 2024-06-10T02:51:17.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:17.376+0000 7fe64b7fe700 1 -- 172.21.15.52:0/885651804 <== mgr.14150 v2:172.21.15.52:6800/3214763969 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fe6540628d0 con 0x7fe6400a2cb0 2024-06-10T02:51:17.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:17.379+0000 7fe65c310700 1 -- 172.21.15.52:0/885651804 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fe6400a2cb0 msgr2=0x7fe6400a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:51:17.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:17.379+0000 7fe65c310700 1 --2- 172.21.15.52:0/885651804 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fe6400a2cb0 0x7fe6400a5170 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fe6500099d0 tx=0x7fe650008040 comp rx=0 tx=0).stop 2024-06-10T02:51:17.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:17.379+0000 7fe65c310700 1 -- 172.21.15.52:0/885651804 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe6541073d0 msgr2=0x7fe654198370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:51:17.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:17.379+0000 7fe65c310700 1 --2- 172.21.15.52:0/885651804 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe6541073d0 0x7fe654198370 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fe64c00d010 tx=0x7fe64c00be90 comp rx=0 tx=0).stop 2024-06-10T02:51:17.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:17.379+0000 7fe65c310700 1 -- 172.21.15.52:0/885651804 shutdown_connections 2024-06-10T02:51:17.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:17.379+0000 7fe65c310700 1 --2- 172.21.15.52:0/885651804 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fe6400a2cb0 0x7fe6400a5170 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:51:17.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:17.379+0000 7fe65c310700 1 --2- 172.21.15.52:0/885651804 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe6541073d0 0x7fe654198370 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:51:17.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:17.379+0000 7fe65c310700 1 --2- 172.21.15.52:0/885651804 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe654101390 0x7fe6541978f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:51:17.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:17.379+0000 7fe65c310700 1 --2- 172.21.15.52:0/885651804 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe654101d30 0x7fe654197e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:51:17.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:17.379+0000 7fe65c310700 1 -- 172.21.15.52:0/885651804 >> 172.21.15.52:0/885651804 conn(0x7fe6540782c0 msgr2=0x7fe6540fff10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:51:17.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:17.379+0000 7fe65c310700 1 -- 172.21.15.52:0/885651804 shutdown_connections 2024-06-10T02:51:17.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:17.379+0000 7fe65c310700 1 -- 172.21.15.52:0/885651804 wait complete. 2024-06-10T02:51:18.010 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-06-10T02:51:18.260 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:51:18.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:51:18 smithi188 ceph-mon[36599]: pgmap v345: 129 pgs: 129 active+clean; 594 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 5.1 KiB/s wr, 22 op/s 2024-06-10T02:51:18.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:51:18 smithi188 ceph-mon[36599]: from='client.14775 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T02:51:18.975 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:51:18 smithi052 ceph-mon[32362]: pgmap v345: 129 pgs: 129 active+clean; 594 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 5.1 KiB/s wr, 22 op/s 2024-06-10T02:51:18.976 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:51:18 smithi052 ceph-mon[32362]: from='client.14775 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T02:51:19.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:51:18 smithi120 ceph-mon[36634]: pgmap v345: 129 pgs: 129 active+clean; 594 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 5.1 KiB/s wr, 22 op/s 2024-06-10T02:51:19.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:51:18 smithi120 ceph-mon[36634]: from='client.14775 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T02:51:20.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:20.078+0000 7fd592cb3700 1 -- 172.21.15.52:0/3166522952 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd58c108630 msgr2=0x7fd58c108a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:51:20.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:20.078+0000 7fd592cb3700 1 --2- 172.21.15.52:0/3166522952 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd58c108630 0x7fd58c108a10 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fd580009990 tx=0x7fd580009ca0 comp rx=0 tx=0).stop 2024-06-10T02:51:20.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:20.078+0000 7fd592cb3700 1 -- 172.21.15.52:0/3166522952 shutdown_connections 2024-06-10T02:51:20.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:20.078+0000 7fd592cb3700 1 --2- 172.21.15.52:0/3166522952 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd58c06b590 0x7fd58c10f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:51:20.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:20.078+0000 7fd592cb3700 1 --2- 172.21.15.52:0/3166522952 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd58c108630 0x7fd58c108a10 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:51:20.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:20.078+0000 7fd592cb3700 1 --2- 172.21.15.52:0/3166522952 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd58c06abd0 0x7fd58c06b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:51:20.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:20.078+0000 7fd592cb3700 1 -- 172.21.15.52:0/3166522952 >> 172.21.15.52:0/3166522952 conn(0x7fd58c078190 msgr2=0x7fd58c0785a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:51:20.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:20.078+0000 7fd592cb3700 1 -- 172.21.15.52:0/3166522952 shutdown_connections 2024-06-10T02:51:20.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:20.078+0000 7fd592cb3700 1 -- 172.21.15.52:0/3166522952 wait complete. 2024-06-10T02:51:20.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:20.079+0000 7fd592cb3700 1 Processor -- start 2024-06-10T02:51:20.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:20.079+0000 7fd592cb3700 1 -- start start 2024-06-10T02:51:20.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:20.079+0000 7fd592cb3700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd58c06abd0 0x7fd58c19ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:51:20.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:20.079+0000 7fd592cb3700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd58c06b590 0x7fd58c19b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:51:20.080 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:20.079+0000 7fd592cb3700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd58c108630 0x7fd58c19f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:51:20.080 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:20.079+0000 7fd592cb3700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fd58c112810 con 0x7fd58c108630 2024-06-10T02:51:20.080 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:20.079+0000 7fd592cb3700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd58c112690 con 0x7fd58c06b590 2024-06-10T02:51:20.080 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:20.079+0000 7fd592cb3700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fd58c112990 con 0x7fd58c06abd0 2024-06-10T02:51:20.080 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:20.079+0000 7fd591250700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd58c108630 0x7fd58c19f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:51:20.080 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:20.079+0000 7fd591250700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd58c108630 0x7fd58c19f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49586/0 (socket says 172.21.15.52:49586) 2024-06-10T02:51:20.080 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:20.079+0000 7fd591250700 1 -- 172.21.15.52:0/728179835 learned_addr learned my addr 172.21.15.52:0/728179835 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:51:20.080 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:20.079+0000 7fd58bfff700 1 --2- 172.21.15.52:0/728179835 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd58c06b590 0x7fd58c19b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:51:20.080 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:20.079+0000 7fd590a4f700 1 --2- 172.21.15.52:0/728179835 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd58c06abd0 0x7fd58c19ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:51:20.080 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:20.080+0000 7fd591250700 1 -- 172.21.15.52:0/728179835 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd58c06abd0 msgr2=0x7fd58c19ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:51:20.080 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:20.080+0000 7fd591250700 1 --2- 172.21.15.52:0/728179835 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd58c06abd0 0x7fd58c19ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:51:20.080 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:20.080+0000 7fd591250700 1 -- 172.21.15.52:0/728179835 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd58c06b590 msgr2=0x7fd58c19b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:51:20.081 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:20.080+0000 7fd591250700 1 --2- 172.21.15.52:0/728179835 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd58c06b590 0x7fd58c19b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:51:20.081 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:20.080+0000 7fd591250700 1 -- 172.21.15.52:0/728179835 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd58c19fdf0 con 0x7fd58c108630 2024-06-10T02:51:20.081 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:20.080+0000 7fd591250700 1 --2- 172.21.15.52:0/728179835 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd58c108630 0x7fd58c19f6b0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fd584007f10 tx=0x7fd58400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:51:20.081 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:20.080+0000 7fd589ffb700 1 -- 172.21.15.52:0/728179835 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd584009e20 con 0x7fd58c108630 2024-06-10T02:51:20.081 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:20.080+0000 7fd592cb3700 1 -- 172.21.15.52:0/728179835 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd58c1a00e0 con 0x7fd58c108630 2024-06-10T02:51:20.081 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:20.080+0000 7fd589ffb700 1 -- 172.21.15.52:0/728179835 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd584004bb0 con 0x7fd58c108630 2024-06-10T02:51:20.081 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:20.080+0000 7fd589ffb700 1 -- 172.21.15.52:0/728179835 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd584005960 con 0x7fd58c108630 2024-06-10T02:51:20.081 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:20.080+0000 7fd592cb3700 1 -- 172.21.15.52:0/728179835 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd58c1a7960 con 0x7fd58c108630 2024-06-10T02:51:20.082 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:20.082+0000 7fd592cb3700 1 -- 172.21.15.52:0/728179835 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd58c069180 con 0x7fd58c108630 2024-06-10T02:51:20.082 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:20.083+0000 7fd589ffb700 1 -- 172.21.15.52:0/728179835 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fd58401a020 con 0x7fd58c108630 2024-06-10T02:51:20.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:20.083+0000 7fd589ffb700 1 --2- 172.21.15.52:0/728179835 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fd57c0a2cb0 0x7fd57c0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:51:20.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:20.083+0000 7fd589ffb700 1 -- 172.21.15.52:0/728179835 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9092+0+0 (secure 0 0 0) 0x7fd5840ceb40 con 0x7fd58c108630 2024-06-10T02:51:20.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:20.083+0000 7fd590a4f700 1 --2- 172.21.15.52:0/728179835 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fd57c0a2cb0 0x7fd57c0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:51:20.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:20.084+0000 7fd590a4f700 1 --2- 172.21.15.52:0/728179835 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fd57c0a2cb0 0x7fd57c0a5170 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fd58000b5c0 tx=0x7fd580005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:51:20.087 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:20.088+0000 7fd589ffb700 1 -- 172.21.15.52:0/728179835 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fd584010070 con 0x7fd58c108630 2024-06-10T02:51:20.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:20.328+0000 7fd592cb3700 1 -- 172.21.15.52:0/728179835 --> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7fd58c10de70 con 0x7fd57c0a2cb0 2024-06-10T02:51:20.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:20.334+0000 7fd589ffb700 1 -- 172.21.15.52:0/728179835 <== mgr.14150 v2:172.21.15.52:6800/3214763969 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7fd58c10de70 con 0x7fd57c0a2cb0 2024-06-10T02:51:20.333 INFO:teuthology.orchestra.run.smithi052.stdout:/volumes/qa/sv_0/f209ec22-be45-4c7b-b0aa-beb5a2fd8f30 2024-06-10T02:51:20.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:20.336+0000 7fd592cb3700 1 -- 172.21.15.52:0/728179835 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fd57c0a2cb0 msgr2=0x7fd57c0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:51:20.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:20.336+0000 7fd592cb3700 1 --2- 172.21.15.52:0/728179835 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fd57c0a2cb0 0x7fd57c0a5170 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fd58000b5c0 tx=0x7fd580005960 comp rx=0 tx=0).stop 2024-06-10T02:51:20.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:20.336+0000 7fd592cb3700 1 -- 172.21.15.52:0/728179835 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd58c108630 msgr2=0x7fd58c19f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:51:20.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:20.336+0000 7fd592cb3700 1 --2- 172.21.15.52:0/728179835 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd58c108630 0x7fd58c19f6b0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fd584007f10 tx=0x7fd58400d3b0 comp rx=0 tx=0).stop 2024-06-10T02:51:20.336 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:20.336+0000 7fd592cb3700 1 -- 172.21.15.52:0/728179835 shutdown_connections 2024-06-10T02:51:20.336 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:20.336+0000 7fd592cb3700 1 --2- 172.21.15.52:0/728179835 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fd57c0a2cb0 0x7fd57c0a5170 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:51:20.336 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:20.336+0000 7fd592cb3700 1 --2- 172.21.15.52:0/728179835 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd58c108630 0x7fd58c19f6b0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:51:20.336 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:20.336+0000 7fd592cb3700 1 --2- 172.21.15.52:0/728179835 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd58c06abd0 0x7fd58c19ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:51:20.336 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:20.336+0000 7fd592cb3700 1 --2- 172.21.15.52:0/728179835 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd58c06b590 0x7fd58c19b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:51:20.336 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:20.336+0000 7fd592cb3700 1 -- 172.21.15.52:0/728179835 >> 172.21.15.52:0/728179835 conn(0x7fd58c078190 msgr2=0x7fd58c10d760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:51:20.336 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:20.337+0000 7fd592cb3700 1 -- 172.21.15.52:0/728179835 shutdown_connections 2024-06-10T02:51:20.336 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:20.337+0000 7fd592cb3700 1 -- 172.21.15.52:0/728179835 wait complete. 2024-06-10T02:51:20.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:51:20 smithi188 ceph-mon[36599]: pgmap v346: 129 pgs: 129 active+clean; 595 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 4.4 KiB/s wr, 21 op/s 2024-06-10T02:51:20.974 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:51:20 smithi052 ceph-mon[32362]: pgmap v346: 129 pgs: 129 active+clean; 595 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 4.4 KiB/s wr, 21 op/s 2024-06-10T02:51:21.007 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph fs subvolume create cephfs sv_1 --group_name qa 2024-06-10T02:51:21.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:51:20 smithi120 ceph-mon[36634]: pgmap v346: 129 pgs: 129 active+clean; 595 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 4.4 KiB/s wr, 21 op/s 2024-06-10T02:51:21.262 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:51:21.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:51:21 smithi188 ceph-mon[36599]: from='client.14781 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T02:51:21.975 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:51:21 smithi052 ceph-mon[32362]: from='client.14781 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T02:51:22.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:51:21 smithi120 ceph-mon[36634]: from='client.14781 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T02:51:22.930 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:51:22 smithi052 ceph-mon[32362]: pgmap v347: 129 pgs: 129 active+clean; 595 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 3.4 KiB/s wr, 17 op/s 2024-06-10T02:51:22.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:22.944+0000 7f981b1b6700 1 -- 172.21.15.52:0/671186744 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9814101d30 msgr2=0x7f981410e210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:51:22.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:22.944+0000 7f981b1b6700 1 --2- 172.21.15.52:0/671186744 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9814101d30 0x7f981410e210 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f980c00b0a0 tx=0x7f980c00b3b0 comp rx=0 tx=0).stop 2024-06-10T02:51:22.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:22.944+0000 7f981b1b6700 1 -- 172.21.15.52:0/671186744 shutdown_connections 2024-06-10T02:51:22.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:22.944+0000 7f981b1b6700 1 --2- 172.21.15.52:0/671186744 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9814101d30 0x7f981410e210 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:51:22.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:22.944+0000 7f981b1b6700 1 --2- 172.21.15.52:0/671186744 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f98141073d0 0x7f98141077b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:51:22.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:22.944+0000 7f981b1b6700 1 --2- 172.21.15.52:0/671186744 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9814101390 0x7f98141017f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:51:22.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:22.944+0000 7f981b1b6700 1 -- 172.21.15.52:0/671186744 >> 172.21.15.52:0/671186744 conn(0x7f98140782c0 msgr2=0x7f98140786d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:51:22.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:22.944+0000 7f981b1b6700 1 -- 172.21.15.52:0/671186744 shutdown_connections 2024-06-10T02:51:22.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:22.944+0000 7f981b1b6700 1 -- 172.21.15.52:0/671186744 wait complete. 2024-06-10T02:51:22.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:22.945+0000 7f981b1b6700 1 Processor -- start 2024-06-10T02:51:22.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:22.945+0000 7f981b1b6700 1 -- start start 2024-06-10T02:51:22.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:22.945+0000 7f981b1b6700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9814101390 0x7f9814103340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:51:22.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:22.945+0000 7f981b1b6700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9814101d30 0x7f9814103880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:51:22.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:22.945+0000 7f981b1b6700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f98141073d0 0x7f9814103dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:51:22.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:22.945+0000 7f981b1b6700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f9814111620 con 0x7f9814101d30 2024-06-10T02:51:22.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:22.945+0000 7f981b1b6700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f98141114a0 con 0x7f9814101390 2024-06-10T02:51:22.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:22.945+0000 7f981b1b6700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f98141117a0 con 0x7f98141073d0 2024-06-10T02:51:22.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:22.945+0000 7f9813fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9814101d30 0x7f9814103880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:51:22.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:22.945+0000 7f9813fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9814101d30 0x7f9814103880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:42734/0 (socket says 172.21.15.52:42734) 2024-06-10T02:51:22.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:22.945+0000 7f9813fff700 1 -- 172.21.15.52:0/2571718975 learned_addr learned my addr 172.21.15.52:0/2571718975 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:51:22.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:22.945+0000 7f9819753700 1 --2- 172.21.15.52:0/2571718975 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f98141073d0 0x7f9814103dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:51:22.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:22.945+0000 7f9818f52700 1 --2- 172.21.15.52:0/2571718975 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9814101390 0x7f9814103340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:51:22.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:22.946+0000 7f9813fff700 1 -- 172.21.15.52:0/2571718975 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f98141073d0 msgr2=0x7f9814103dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:51:22.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:22.946+0000 7f9813fff700 1 --2- 172.21.15.52:0/2571718975 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f98141073d0 0x7f9814103dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:51:22.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:22.946+0000 7f9813fff700 1 -- 172.21.15.52:0/2571718975 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9814101390 msgr2=0x7f9814103340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:51:22.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:22.946+0000 7f9813fff700 1 --2- 172.21.15.52:0/2571718975 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9814101390 0x7f9814103340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:51:22.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:22.946+0000 7f9813fff700 1 -- 172.21.15.52:0/2571718975 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98141a7190 con 0x7f9814101d30 2024-06-10T02:51:22.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:22.946+0000 7f9813fff700 1 --2- 172.21.15.52:0/2571718975 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9814101d30 0x7f9814103880 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f980800c920 tx=0x7f980800cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:51:22.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:22.946+0000 7f9811ffb700 1 -- 172.21.15.52:0/2571718975 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9808004a30 con 0x7f9814101d30 2024-06-10T02:51:22.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:22.946+0000 7f9811ffb700 1 -- 172.21.15.52:0/2571718975 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9808005500 con 0x7f9814101d30 2024-06-10T02:51:22.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:22.946+0000 7f981b1b6700 1 -- 172.21.15.52:0/2571718975 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98141a7480 con 0x7f9814101d30 2024-06-10T02:51:22.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:22.946+0000 7f9811ffb700 1 -- 172.21.15.52:0/2571718975 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9808007d50 con 0x7f9814101d30 2024-06-10T02:51:22.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:22.946+0000 7f981b1b6700 1 -- 172.21.15.52:0/2571718975 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98141a79c0 con 0x7f9814101d30 2024-06-10T02:51:22.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:22.947+0000 7f981b1b6700 1 -- 172.21.15.52:0/2571718975 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9814069180 con 0x7f9814101d30 2024-06-10T02:51:22.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:22.948+0000 7f9811ffb700 1 -- 172.21.15.52:0/2571718975 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f9808004d60 con 0x7f9814101d30 2024-06-10T02:51:22.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:22.949+0000 7f9811ffb700 1 --2- 172.21.15.52:0/2571718975 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f97fc0a2cb0 0x7f97fc0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:51:22.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:22.949+0000 7f9811ffb700 1 -- 172.21.15.52:0/2571718975 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9092+0+0 (secure 0 0 0) 0x7f98080cf320 con 0x7f9814101d30 2024-06-10T02:51:22.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:22.949+0000 7f9818f52700 1 --2- 172.21.15.52:0/2571718975 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f97fc0a2cb0 0x7f97fc0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:51:22.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:22.949+0000 7f9818f52700 1 --2- 172.21.15.52:0/2571718975 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f97fc0a2cb0 0x7f97fc0a5170 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f9804005950 tx=0x7f980400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:51:22.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:22.953+0000 7f9811ffb700 1 -- 172.21.15.52:0/2571718975 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f9808014030 con 0x7f9814101d30 2024-06-10T02:51:22.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:51:22 smithi188 ceph-mon[36599]: pgmap v347: 129 pgs: 129 active+clean; 595 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 3.4 KiB/s wr, 17 op/s 2024-06-10T02:51:23.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:51:22 smithi120 ceph-mon[36634]: pgmap v347: 129 pgs: 129 active+clean; 595 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 3.4 KiB/s wr, 17 op/s 2024-06-10T02:51:23.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:23.192+0000 7f981b1b6700 1 -- 172.21.15.52:0/2571718975 --> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f98140ffdf0 con 0x7f97fc0a2cb0 2024-06-10T02:51:23.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:23.219+0000 7f9811ffb700 1 -- 172.21.15.52:0/2571718975 <== mgr.14150 v2:172.21.15.52:6800/3214763969 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f98140ffdf0 con 0x7f97fc0a2cb0 2024-06-10T02:51:23.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:23.222+0000 7f981b1b6700 1 -- 172.21.15.52:0/2571718975 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f97fc0a2cb0 msgr2=0x7f97fc0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:51:23.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:23.222+0000 7f981b1b6700 1 --2- 172.21.15.52:0/2571718975 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f97fc0a2cb0 0x7f97fc0a5170 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f9804005950 tx=0x7f980400b410 comp rx=0 tx=0).stop 2024-06-10T02:51:23.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:23.222+0000 7f981b1b6700 1 -- 172.21.15.52:0/2571718975 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9814101d30 msgr2=0x7f9814103880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:51:23.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:23.222+0000 7f981b1b6700 1 --2- 172.21.15.52:0/2571718975 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9814101d30 0x7f9814103880 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f980800c920 tx=0x7f980800cc30 comp rx=0 tx=0).stop 2024-06-10T02:51:23.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:23.222+0000 7f981b1b6700 1 -- 172.21.15.52:0/2571718975 shutdown_connections 2024-06-10T02:51:23.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:23.222+0000 7f981b1b6700 1 --2- 172.21.15.52:0/2571718975 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f98141073d0 0x7f9814103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:51:23.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:23.222+0000 7f981b1b6700 1 --2- 172.21.15.52:0/2571718975 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9814101390 0x7f9814103340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:51:23.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:23.222+0000 7f981b1b6700 1 --2- 172.21.15.52:0/2571718975 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f97fc0a2cb0 0x7f97fc0a5170 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:51:23.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:23.222+0000 7f981b1b6700 1 --2- 172.21.15.52:0/2571718975 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9814101d30 0x7f9814103880 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:51:23.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:23.222+0000 7f981b1b6700 1 -- 172.21.15.52:0/2571718975 >> 172.21.15.52:0/2571718975 conn(0x7f98140782c0 msgr2=0x7f9814105620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:51:23.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:23.222+0000 7f981b1b6700 1 -- 172.21.15.52:0/2571718975 shutdown_connections 2024-06-10T02:51:23.222 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:23.222+0000 7f981b1b6700 1 -- 172.21.15.52:0/2571718975 wait complete. 2024-06-10T02:51:23.832 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-06-10T02:51:24.085 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:51:24.947 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:51:24 smithi052 ceph-mon[32362]: pgmap v348: 129 pgs: 129 active+clean; 614 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.2 KiB/s wr, 20 op/s 2024-06-10T02:51:24.947 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:51:24 smithi052 ceph-mon[32362]: from='client.14787 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T02:51:24.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:51:24 smithi188 ceph-mon[36599]: pgmap v348: 129 pgs: 129 active+clean; 614 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.2 KiB/s wr, 20 op/s 2024-06-10T02:51:24.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:51:24 smithi188 ceph-mon[36599]: from='client.14787 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T02:51:25.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:51:24 smithi120 ceph-mon[36634]: pgmap v348: 129 pgs: 129 active+clean; 614 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.2 KiB/s wr, 20 op/s 2024-06-10T02:51:25.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:51:24 smithi120 ceph-mon[36634]: from='client.14787 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T02:51:25.900 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:25.900+0000 7ff416885700 1 -- 172.21.15.52:0/2500189813 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff410101fc0 msgr2=0x7ff410106010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:51:25.900 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:25.900+0000 7ff416885700 1 --2- 172.21.15.52:0/2500189813 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff410101fc0 0x7ff410106010 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ff4040099c0 tx=0x7ff404009cd0 comp rx=0 tx=0).stop 2024-06-10T02:51:25.900 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:25.900+0000 7ff416885700 1 -- 172.21.15.52:0/2500189813 shutdown_connections 2024-06-10T02:51:25.900 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:25.901+0000 7ff416885700 1 --2- 172.21.15.52:0/2500189813 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff4101066f0 0x7ff41010e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:51:25.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:25.901+0000 7ff416885700 1 --2- 172.21.15.52:0/2500189813 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff410101610 0x7ff4101019f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:51:25.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:25.901+0000 7ff416885700 1 --2- 172.21.15.52:0/2500189813 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff410101fc0 0x7ff410106010 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:51:25.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:25.901+0000 7ff416885700 1 -- 172.21.15.52:0/2500189813 >> 172.21.15.52:0/2500189813 conn(0x7ff410078430 msgr2=0x7ff410078840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:51:25.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:25.901+0000 7ff416885700 1 -- 172.21.15.52:0/2500189813 shutdown_connections 2024-06-10T02:51:25.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:25.901+0000 7ff416885700 1 -- 172.21.15.52:0/2500189813 wait complete. 2024-06-10T02:51:25.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:25.901+0000 7ff416885700 1 Processor -- start 2024-06-10T02:51:25.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:25.901+0000 7ff416885700 1 -- start start 2024-06-10T02:51:25.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:25.902+0000 7ff416885700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff410101610 0x7ff41019e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:51:25.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:25.902+0000 7ff416885700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff410101fc0 0x7ff41019ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:51:25.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:25.902+0000 7ff416885700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff4101066f0 0x7ff4101989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:51:25.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:25.902+0000 7ff416885700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ff410111700 con 0x7ff410101fc0 2024-06-10T02:51:25.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:25.902+0000 7ff416885700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff410111580 con 0x7ff4101066f0 2024-06-10T02:51:25.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:25.902+0000 7ff416885700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ff410111880 con 0x7ff410101610 2024-06-10T02:51:25.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:25.902+0000 7ff40f7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff410101fc0 0x7ff41019ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:51:25.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:25.902+0000 7ff40f7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff410101fc0 0x7ff41019ed50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:42752/0 (socket says 172.21.15.52:42752) 2024-06-10T02:51:25.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:25.902+0000 7ff40f7fe700 1 -- 172.21.15.52:0/3627946488 learned_addr learned my addr 172.21.15.52:0/3627946488 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:51:25.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:25.902+0000 7ff414e22700 1 --2- 172.21.15.52:0/3627946488 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff4101066f0 0x7ff4101989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:51:25.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:25.902+0000 7ff40ffff700 1 --2- 172.21.15.52:0/3627946488 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff410101610 0x7ff41019e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:51:25.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:25.902+0000 7ff40f7fe700 1 -- 172.21.15.52:0/3627946488 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff410101610 msgr2=0x7ff41019e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:51:25.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:25.902+0000 7ff40f7fe700 1 --2- 172.21.15.52:0/3627946488 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff410101610 0x7ff41019e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:51:25.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:25.902+0000 7ff40f7fe700 1 -- 172.21.15.52:0/3627946488 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff4101066f0 msgr2=0x7ff4101989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:51:25.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:25.902+0000 7ff40f7fe700 1 --2- 172.21.15.52:0/3627946488 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff4101066f0 0x7ff4101989d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:51:25.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:25.902+0000 7ff40f7fe700 1 -- 172.21.15.52:0/3627946488 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff410199130 con 0x7ff410101fc0 2024-06-10T02:51:25.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:25.902+0000 7ff414e22700 1 --2- 172.21.15.52:0/3627946488 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff4101066f0 0x7ff4101989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T02:51:25.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:25.902+0000 7ff40f7fe700 1 --2- 172.21.15.52:0/3627946488 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff410101fc0 0x7ff41019ed50 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7ff40400c010 tx=0x7ff404003890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:51:25.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:25.903+0000 7ff40d7fa700 1 -- 172.21.15.52:0/3627946488 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff404027070 con 0x7ff410101fc0 2024-06-10T02:51:25.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:25.903+0000 7ff40d7fa700 1 -- 172.21.15.52:0/3627946488 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff404004050 con 0x7ff410101fc0 2024-06-10T02:51:25.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:25.903+0000 7ff416885700 1 -- 172.21.15.52:0/3627946488 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4101993c0 con 0x7ff410101fc0 2024-06-10T02:51:25.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:25.903+0000 7ff40d7fa700 1 -- 172.21.15.52:0/3627946488 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff404022e00 con 0x7ff410101fc0 2024-06-10T02:51:25.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:25.903+0000 7ff416885700 1 -- 172.21.15.52:0/3627946488 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4101a56e0 con 0x7ff410101fc0 2024-06-10T02:51:25.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:25.904+0000 7ff416885700 1 -- 172.21.15.52:0/3627946488 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff410069180 con 0x7ff410101fc0 2024-06-10T02:51:25.906 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:25.905+0000 7ff40d7fa700 1 -- 172.21.15.52:0/3627946488 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7ff40402a070 con 0x7ff410101fc0 2024-06-10T02:51:25.906 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:25.906+0000 7ff40d7fa700 1 --2- 172.21.15.52:0/3627946488 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7ff3f80a2cb0 0x7ff3f80a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:51:25.906 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:25.906+0000 7ff40ffff700 1 --2- 172.21.15.52:0/3627946488 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7ff3f80a2cb0 0x7ff3f80a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:51:25.906 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:25.906+0000 7ff40d7fa700 1 -- 172.21.15.52:0/3627946488 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9092+0+0 (secure 0 0 0) 0x7ff4040d1070 con 0x7ff410101fc0 2024-06-10T02:51:25.906 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:25.907+0000 7ff40ffff700 1 --2- 172.21.15.52:0/3627946488 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7ff3f80a2cb0 0x7ff3f80a5170 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7ff400005ea0 tx=0x7ff400005e30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:51:25.909 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:25.910+0000 7ff40d7fa700 1 -- 172.21.15.52:0/3627946488 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7ff404031070 con 0x7ff410101fc0 2024-06-10T02:51:26.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:26.152+0000 7ff416885700 1 -- 172.21.15.52:0/3627946488 --> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7ff410199f00 con 0x7ff3f80a2cb0 2024-06-10T02:51:26.157 INFO:teuthology.orchestra.run.smithi052.stdout:/volumes/qa/sv_1/80a54c16-16b3-4ac2-8c1a-868b99f8d35c 2024-06-10T02:51:26.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:26.158+0000 7ff40d7fa700 1 -- 172.21.15.52:0/3627946488 <== mgr.14150 v2:172.21.15.52:6800/3214763969 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7ff410199f00 con 0x7ff3f80a2cb0 2024-06-10T02:51:26.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:26.161+0000 7ff416885700 1 -- 172.21.15.52:0/3627946488 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7ff3f80a2cb0 msgr2=0x7ff3f80a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:51:26.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:26.161+0000 7ff416885700 1 --2- 172.21.15.52:0/3627946488 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7ff3f80a2cb0 0x7ff3f80a5170 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7ff400005ea0 tx=0x7ff400005e30 comp rx=0 tx=0).stop 2024-06-10T02:51:26.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:26.161+0000 7ff416885700 1 -- 172.21.15.52:0/3627946488 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff410101fc0 msgr2=0x7ff41019ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:51:26.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:26.161+0000 7ff416885700 1 --2- 172.21.15.52:0/3627946488 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff410101fc0 0x7ff41019ed50 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7ff40400c010 tx=0x7ff404003890 comp rx=0 tx=0).stop 2024-06-10T02:51:26.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:26.161+0000 7ff416885700 1 -- 172.21.15.52:0/3627946488 shutdown_connections 2024-06-10T02:51:26.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:26.161+0000 7ff416885700 1 --2- 172.21.15.52:0/3627946488 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff4101066f0 0x7ff4101989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:51:26.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:26.161+0000 7ff416885700 1 --2- 172.21.15.52:0/3627946488 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff410101610 0x7ff41019e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:51:26.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:26.161+0000 7ff416885700 1 --2- 172.21.15.52:0/3627946488 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7ff3f80a2cb0 0x7ff3f80a5170 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:51:26.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:26.161+0000 7ff416885700 1 --2- 172.21.15.52:0/3627946488 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff410101fc0 0x7ff41019ed50 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:51:26.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:26.161+0000 7ff416885700 1 -- 172.21.15.52:0/3627946488 >> 172.21.15.52:0/3627946488 conn(0x7ff410078430 msgr2=0x7ff41006ad60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:51:26.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:26.161+0000 7ff416885700 1 -- 172.21.15.52:0/3627946488 shutdown_connections 2024-06-10T02:51:26.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:26.161+0000 7ff416885700 1 -- 172.21.15.52:0/3627946488 wait complete. 2024-06-10T02:51:26.833 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:51:26 smithi052 ceph-mon[32362]: pgmap v349: 129 pgs: 129 active+clean; 614 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 3.2 KiB/s wr, 11 op/s 2024-06-10T02:51:26.862 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T02:51:26.939 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:51:26 smithi120 ceph-mon[36634]: pgmap v349: 129 pgs: 129 active+clean; 614 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 3.2 KiB/s wr, 11 op/s 2024-06-10T02:51:26.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:51:26 smithi188 ceph-mon[36599]: pgmap v349: 129 pgs: 129 active+clean; 614 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 3.2 KiB/s wr, 11 op/s 2024-06-10T02:51:27.115 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:51:27.941 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:51:27 smithi052 ceph-mon[32362]: from='client.14793 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T02:51:27.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:51:27 smithi188 ceph-mon[36599]: from='client.14793 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T02:51:28.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:51:27 smithi120 ceph-mon[36634]: from='client.14793 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T02:51:28.770 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:28.771+0000 7f421e7a6700 1 -- 172.21.15.52:0/1267858054 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4218103120 msgr2=0x7f4218107130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:51:28.770 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:28.771+0000 7f421e7a6700 1 --2- 172.21.15.52:0/1267858054 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4218103120 0x7f4218107130 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f42000099c0 tx=0x7f4200009cd0 comp rx=0 tx=0).stop 2024-06-10T02:51:28.770 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:28.771+0000 7f421e7a6700 1 -- 172.21.15.52:0/1267858054 shutdown_connections 2024-06-10T02:51:28.770 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:28.771+0000 7f421e7a6700 1 --2- 172.21.15.52:0/1267858054 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4218107860 0x7f421810f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:51:28.771 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:28.771+0000 7f421e7a6700 1 --2- 172.21.15.52:0/1267858054 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4218102770 0x7f4218102b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:51:28.771 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:28.771+0000 7f421e7a6700 1 --2- 172.21.15.52:0/1267858054 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4218103120 0x7f4218107130 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:51:28.771 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:28.771+0000 7f421e7a6700 1 -- 172.21.15.52:0/1267858054 >> 172.21.15.52:0/1267858054 conn(0x7f4218079600 msgr2=0x7f4218079a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:51:28.771 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:28.771+0000 7f421e7a6700 1 -- 172.21.15.52:0/1267858054 shutdown_connections 2024-06-10T02:51:28.771 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:28.771+0000 7f421e7a6700 1 -- 172.21.15.52:0/1267858054 wait complete. 2024-06-10T02:51:28.771 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:28.771+0000 7f421e7a6700 1 Processor -- start 2024-06-10T02:51:28.771 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:28.771+0000 7f421e7a6700 1 -- start start 2024-06-10T02:51:28.772 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:28.772+0000 7f421e7a6700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4218102770 0x7f421819e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:51:28.773 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:28.772+0000 7f421e7a6700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4218103120 0x7f421819edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:51:28.773 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:28.772+0000 7f421e7a6700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4218107860 0x7f4218198a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:51:28.773 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:28.772+0000 7f421e7a6700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f42181127e0 con 0x7f4218107860 2024-06-10T02:51:28.773 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:28.772+0000 7f421e7a6700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f4218112660 con 0x7f4218102770 2024-06-10T02:51:28.773 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:28.772+0000 7f421e7a6700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f4218112960 con 0x7f4218103120 2024-06-10T02:51:28.773 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:28.772+0000 7f421cd43700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4218107860 0x7f4218198a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:51:28.773 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:28.772+0000 7f421cd43700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4218107860 0x7f4218198a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:42762/0 (socket says 172.21.15.52:42762) 2024-06-10T02:51:28.773 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:28.772+0000 7f4217fff700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4218102770 0x7f421819e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:51:28.773 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:28.772+0000 7f421cd43700 1 -- 172.21.15.52:0/3987030818 learned_addr learned my addr 172.21.15.52:0/3987030818 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:51:28.773 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:28.772+0000 7f42177fe700 1 --2- 172.21.15.52:0/3987030818 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4218103120 0x7f421819edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:51:28.773 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:28.772+0000 7f421cd43700 1 -- 172.21.15.52:0/3987030818 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4218103120 msgr2=0x7f421819edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:51:28.773 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:28.772+0000 7f421cd43700 1 --2- 172.21.15.52:0/3987030818 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4218103120 0x7f421819edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:51:28.774 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:28.772+0000 7f421cd43700 1 -- 172.21.15.52:0/3987030818 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4218102770 msgr2=0x7f421819e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:51:28.774 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:28.772+0000 7f421cd43700 1 --2- 172.21.15.52:0/3987030818 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4218102770 0x7f421819e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:51:28.774 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:28.772+0000 7f421cd43700 1 -- 172.21.15.52:0/3987030818 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42181991e0 con 0x7f4218107860 2024-06-10T02:51:28.774 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:28.772+0000 7f4217fff700 1 --2- 172.21.15.52:0/3987030818 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4218102770 0x7f421819e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T02:51:28.774 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:28.773+0000 7f421cd43700 1 --2- 172.21.15.52:0/3987030818 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4218107860 0x7f4218198a80 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f4210007ee0 tx=0x7f421000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:51:28.774 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:28.773+0000 7f42157fa700 1 -- 172.21.15.52:0/3987030818 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4210004bb0 con 0x7f4218107860 2024-06-10T02:51:28.774 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:28.773+0000 7f421e7a6700 1 -- 172.21.15.52:0/3987030818 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42181994d0 con 0x7f4218107860 2024-06-10T02:51:28.774 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:28.773+0000 7f42157fa700 1 -- 172.21.15.52:0/3987030818 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4210004d50 con 0x7f4218107860 2024-06-10T02:51:28.774 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:28.773+0000 7f42157fa700 1 -- 172.21.15.52:0/3987030818 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4210005960 con 0x7f4218107860 2024-06-10T02:51:28.774 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:28.773+0000 7f421e7a6700 1 -- 172.21.15.52:0/3987030818 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42181a5740 con 0x7f4218107860 2024-06-10T02:51:28.774 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:28.774+0000 7f421e7a6700 1 -- 172.21.15.52:0/3987030818 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f421810aa30 con 0x7f4218107860 2024-06-10T02:51:28.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:28.775+0000 7f42157fa700 1 -- 172.21.15.52:0/3987030818 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f421001a020 con 0x7f4218107860 2024-06-10T02:51:28.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:28.776+0000 7f42157fa700 1 --2- 172.21.15.52:0/3987030818 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f42040a2cb0 0x7f42040a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:51:28.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:28.776+0000 7f4217fff700 1 --2- 172.21.15.52:0/3987030818 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f42040a2cb0 0x7f42040a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:51:28.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:28.776+0000 7f42157fa700 1 -- 172.21.15.52:0/3987030818 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9092+0+0 (secure 0 0 0) 0x7f42100cec70 con 0x7f4218107860 2024-06-10T02:51:28.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:28.776+0000 7f4217fff700 1 --2- 172.21.15.52:0/3987030818 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f42040a2cb0 0x7f42040a5170 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f4218107310 tx=0x7f420800b3c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:51:28.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:28.780+0000 7f42157fa700 1 -- 172.21.15.52:0/3987030818 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f4210010070 con 0x7f4218107860 2024-06-10T02:51:28.779 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:51:28 smithi052 ceph-mon[32362]: pgmap v350: 129 pgs: 129 active+clean; 640 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.9 KiB/s wr, 15 op/s 2024-06-10T02:51:28.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:51:28 smithi188 ceph-mon[36599]: pgmap v350: 129 pgs: 129 active+clean; 640 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.9 KiB/s wr, 15 op/s 2024-06-10T02:51:29.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:29.032+0000 7f421e7a6700 1 -- 172.21.15.52:0/3987030818 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4218199f60 con 0x7f4218107860 2024-06-10T02:51:29.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:29.033+0000 7f42157fa700 1 -- 172.21.15.52:0/3987030818 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11938 (secure 0 0 0) 0x7f421009b710 con 0x7f4218107860 2024-06-10T02:51:29.032 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-10T02:51:29.032 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/889673247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":889673247},{"type":"v1","addr":"172.21.15.52:6835","nonce":889673247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/3615663511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":3615663511},{"type":"v1","addr":"172.21.15.52:6837","nonce":3615663511}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T02:50:03.889766+0000","modified":"2024-06-10T02:51:11.420770+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24484,"mds_1":24469,"mds_2":24463,"mds_3":24449,"mds_4":24428},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14571":{"gid":14571,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6839/3059599042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":3059599042},{"type":"v1","addr":"172.21.15.52:6839","nonce":3059599042}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14583":{"gid":14583,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6841/3039275521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":3039275521},{"type":"v1","addr":"172.21.15.52:6841","nonce":3039275521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24383":{"gid":24383,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6833/2609002896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":2609002896},{"type":"v1","addr":"172.21.15.120:6833","nonce":2609002896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6835/1996370288","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":1996370288},{"type":"v1","addr":"172.21.15.120:6835","nonce":1996370288}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6833/529564979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":529564979},{"type":"v1","addr":"172.21.15.188:6833","nonce":529564979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":25,"addr":"172.21.15.120:6837/4132440952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":4132440952},{"type":"v1","addr":"172.21.15.120:6837","nonce":4132440952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"b","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.120:6839/1672436487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1672436487},{"type":"v1","addr":"172.21.15.120:6839","nonce":1672436487}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":2,"incarnation":22,"state":"up:active","state_seq":26,"addr":"172.21.15.188:6835/3904544259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":3904544259},{"type":"v1","addr":"172.21.15.188:6835","nonce":3904544259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.188:6837/819317726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":819317726},{"type":"v1","addr":"172.21.15.188:6837","nonce":819317726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.188:6839/310444836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":310444836},{"type":"v1","addr":"172.21.15.188:6839","nonce":310444836}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24484,"qdb_cluster":[24469,24449,24428,24463,24484]},"id":1}]} 2024-06-10T02:51:29.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:29.035+0000 7f421e7a6700 1 -- 172.21.15.52:0/3987030818 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f42040a2cb0 msgr2=0x7f42040a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:51:29.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:29.035+0000 7f421e7a6700 1 --2- 172.21.15.52:0/3987030818 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f42040a2cb0 0x7f42040a5170 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f4218107310 tx=0x7f420800b3c0 comp rx=0 tx=0).stop 2024-06-10T02:51:29.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:29.035+0000 7f421e7a6700 1 -- 172.21.15.52:0/3987030818 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4218107860 msgr2=0x7f4218198a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:51:29.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:29.035+0000 7f421e7a6700 1 --2- 172.21.15.52:0/3987030818 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4218107860 0x7f4218198a80 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f4210007ee0 tx=0x7f421000d3b0 comp rx=0 tx=0).stop 2024-06-10T02:51:29.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:29.035+0000 7f421e7a6700 1 -- 172.21.15.52:0/3987030818 shutdown_connections 2024-06-10T02:51:29.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:29.035+0000 7f421e7a6700 1 --2- 172.21.15.52:0/3987030818 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f42040a2cb0 0x7f42040a5170 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:51:29.035 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:29.035+0000 7f421e7a6700 1 --2- 172.21.15.52:0/3987030818 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4218107860 0x7f4218198a80 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:51:29.035 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:29.035+0000 7f421e7a6700 1 --2- 172.21.15.52:0/3987030818 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4218102770 0x7f421819e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:51:29.035 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:29.035+0000 7f421e7a6700 1 --2- 172.21.15.52:0/3987030818 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4218103120 0x7f421819edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:51:29.035 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:29.035+0000 7f421e7a6700 1 -- 172.21.15.52:0/3987030818 >> 172.21.15.52:0/3987030818 conn(0x7f4218079600 msgr2=0x7f42181007e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:51:29.035 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:29.035+0000 7f421e7a6700 1 -- 172.21.15.52:0/3987030818 shutdown_connections 2024-06-10T02:51:29.035 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:29.035+0000 7f421e7a6700 1 -- 172.21.15.52:0/3987030818 wait complete. 2024-06-10T02:51:29.035 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 29 2024-06-10T02:51:29.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:51:28 smithi120 ceph-mon[36634]: pgmap v350: 129 pgs: 129 active+clean; 640 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.9 KiB/s wr, 15 op/s 2024-06-10T02:51:29.759 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-10T02:51:29.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:51:29 smithi188 ceph-mon[36599]: from='client.? 172.21.15.52:0/3987030818' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T02:51:29.975 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:51:29 smithi052 ceph-mon[32362]: from='client.? 172.21.15.52:0/3987030818' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T02:51:30.014 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:51:30.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:51:29 smithi120 ceph-mon[36634]: from='client.? 172.21.15.52:0/3987030818' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T02:51:30.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:51:30 smithi188 ceph-mon[36599]: pgmap v351: 129 pgs: 129 active+clean; 640 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 4.8 KiB/s wr, 11 op/s 2024-06-10T02:51:30.976 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:51:30 smithi052 ceph-mon[32362]: pgmap v351: 129 pgs: 129 active+clean; 640 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 4.8 KiB/s wr, 11 op/s 2024-06-10T02:51:31.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:51:30 smithi120 ceph-mon[36634]: pgmap v351: 129 pgs: 129 active+clean; 640 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 4.8 KiB/s wr, 11 op/s 2024-06-10T02:51:31.757 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:31.757+0000 7fc5d9a5c700 1 -- 172.21.15.52:0/1902547305 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc5d41084f0 msgr2=0x7fc5d41088d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:51:31.757 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:31.757+0000 7fc5d9a5c700 1 --2- 172.21.15.52:0/1902547305 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc5d41084f0 0x7fc5d41088d0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fc5c4009990 tx=0x7fc5c4009ca0 comp rx=0 tx=0).stop 2024-06-10T02:51:31.757 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:31.757+0000 7fc5d9a5c700 1 -- 172.21.15.52:0/1902547305 shutdown_connections 2024-06-10T02:51:31.757 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:31.757+0000 7fc5d9a5c700 1 --2- 172.21.15.52:0/1902547305 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc5d4102e80 0x7fc5d410f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:51:31.757 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:31.757+0000 7fc5d9a5c700 1 --2- 172.21.15.52:0/1902547305 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc5d41084f0 0x7fc5d41088d0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:51:31.757 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:31.757+0000 7fc5d9a5c700 1 --2- 172.21.15.52:0/1902547305 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc5d41024e0 0x7fc5d4102940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:51:31.757 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:31.757+0000 7fc5d9a5c700 1 -- 172.21.15.52:0/1902547305 >> 172.21.15.52:0/1902547305 conn(0x7fc5d4079530 msgr2=0x7fc5d4079940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:51:31.758 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:31.758+0000 7fc5d9a5c700 1 -- 172.21.15.52:0/1902547305 shutdown_connections 2024-06-10T02:51:31.758 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:31.758+0000 7fc5d9a5c700 1 -- 172.21.15.52:0/1902547305 wait complete. 2024-06-10T02:51:31.758 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:31.758+0000 7fc5d9a5c700 1 Processor -- start 2024-06-10T02:51:31.758 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:31.758+0000 7fc5d9a5c700 1 -- start start 2024-06-10T02:51:31.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:31.759+0000 7fc5d9a5c700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc5d41024e0 0x7fc5d419e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:51:31.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:31.759+0000 7fc5d9a5c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc5d4102e80 0x7fc5d419ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:51:31.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:31.759+0000 7fc5d9a5c700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc5d41084f0 0x7fc5d41989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:51:31.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:31.759+0000 7fc5d9a5c700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fc5d4112730 con 0x7fc5d4102e80 2024-06-10T02:51:31.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:31.759+0000 7fc5d9a5c700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc5d41125b0 con 0x7fc5d41024e0 2024-06-10T02:51:31.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:31.759+0000 7fc5d9a5c700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc5d41128b0 con 0x7fc5d41084f0 2024-06-10T02:51:31.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:31.759+0000 7fc5ca7fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc5d4102e80 0x7fc5d419ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:51:31.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:31.759+0000 7fc5ca7fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc5d4102e80 0x7fc5d419ed50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37890/0 (socket says 172.21.15.52:37890) 2024-06-10T02:51:31.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:31.759+0000 7fc5ca7fc700 1 -- 172.21.15.52:0/2258588706 learned_addr learned my addr 172.21.15.52:0/2258588706 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:51:31.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:31.759+0000 7fc5d2ffd700 1 --2- 172.21.15.52:0/2258588706 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc5d41024e0 0x7fc5d419e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:51:31.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:31.759+0000 7fc5d37fe700 1 --2- 172.21.15.52:0/2258588706 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc5d41084f0 0x7fc5d41989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:51:31.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:31.759+0000 7fc5ca7fc700 1 -- 172.21.15.52:0/2258588706 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc5d41084f0 msgr2=0x7fc5d41989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:51:31.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:31.759+0000 7fc5ca7fc700 1 --2- 172.21.15.52:0/2258588706 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc5d41084f0 0x7fc5d41989d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:51:31.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:31.759+0000 7fc5ca7fc700 1 -- 172.21.15.52:0/2258588706 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc5d41024e0 msgr2=0x7fc5d419e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:51:31.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:31.759+0000 7fc5ca7fc700 1 --2- 172.21.15.52:0/2258588706 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc5d41024e0 0x7fc5d419e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:51:31.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:31.759+0000 7fc5ca7fc700 1 -- 172.21.15.52:0/2258588706 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5d4199130 con 0x7fc5d4102e80 2024-06-10T02:51:31.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:31.759+0000 7fc5ca7fc700 1 --2- 172.21.15.52:0/2258588706 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc5d4102e80 0x7fc5d419ed50 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fc5bc00d980 tx=0x7fc5bc00dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:51:31.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:31.759+0000 7fc5d8a5a700 1 -- 172.21.15.52:0/2258588706 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5bc016070 con 0x7fc5d4102e80 2024-06-10T02:51:31.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:31.759+0000 7fc5d8a5a700 1 -- 172.21.15.52:0/2258588706 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5bc004500 con 0x7fc5d4102e80 2024-06-10T02:51:31.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:31.759+0000 7fc5d9a5c700 1 -- 172.21.15.52:0/2258588706 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5d4199420 con 0x7fc5d4102e80 2024-06-10T02:51:31.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:31.760+0000 7fc5d8a5a700 1 -- 172.21.15.52:0/2258588706 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5bc004d00 con 0x7fc5d4102e80 2024-06-10T02:51:31.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:31.760+0000 7fc5d9a5c700 1 -- 172.21.15.52:0/2258588706 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5d41a56e0 con 0x7fc5d4102e80 2024-06-10T02:51:31.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:31.761+0000 7fc5d9a5c700 1 -- 172.21.15.52:0/2258588706 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5d4069180 con 0x7fc5d4102e80 2024-06-10T02:51:31.763 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:31.762+0000 7fc5d8a5a700 1 -- 172.21.15.52:0/2258588706 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fc5bc00bbf0 con 0x7fc5d4102e80 2024-06-10T02:51:31.763 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:31.763+0000 7fc5d8a5a700 1 --2- 172.21.15.52:0/2258588706 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fc5c00a2cb0 0x7fc5c00a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:51:31.763 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:31.763+0000 7fc5d2ffd700 1 --2- 172.21.15.52:0/2258588706 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fc5c00a2cb0 0x7fc5c00a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:51:31.763 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:31.763+0000 7fc5d8a5a700 1 -- 172.21.15.52:0/2258588706 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9092+0+0 (secure 0 0 0) 0x7fc5bc0cf370 con 0x7fc5d4102e80 2024-06-10T02:51:31.764 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:31.763+0000 7fc5d2ffd700 1 --2- 172.21.15.52:0/2258588706 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fc5c00a2cb0 0x7fc5c00a5170 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fc5c4006010 tx=0x7fc5c400b540 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:51:31.766 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:31.767+0000 7fc5d8a5a700 1 -- 172.21.15.52:0/2258588706 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fc5bc01f020 con 0x7fc5d4102e80 2024-06-10T02:51:31.973 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:31.974+0000 7fc5d9a5c700 1 -- 172.21.15.52:0/2258588706 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc5d402dca0 con 0x7fc5d4102e80 2024-06-10T02:51:31.975 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:31.976+0000 7fc5d8a5a700 1 -- 172.21.15.52:0/2258588706 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v98) ==== 74+0+24598 (secure 0 0 0) 0x7fc5bc09be10 con 0x7fc5d4102e80 2024-06-10T02:51:31.976 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-10T02:51:31.976 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":98,"fsid":"d8676e9e-26d2-11ef-bc9b-c7b262605968","created":"2024-06-10T02:41:43.076837+0000","modified":"2024-06-10T02:50:44.731014+0000","last_up_change":"2024-06-10T02:48:57.297829+0000","last_in_change":"2024-06-10T02:48:36.865802+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-10T02:45:07.394690+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-10T02:49:57.926838+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-10T02:50:01.130393+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-10T02:50:10.655588+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e03c0691-cc5a-497a-bbc2-ce18c337f074","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":4255833408},{"type":"v1","addr":"172.21.15.52:6803","nonce":4255833408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":4255833408},{"type":"v1","addr":"172.21.15.52:6805","nonce":4255833408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":4255833408},{"type":"v1","addr":"172.21.15.52:6809","nonce":4255833408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":4255833408},{"type":"v1","addr":"172.21.15.52:6807","nonce":4255833408}]},"public_addr":"172.21.15.52:6803/4255833408","cluster_addr":"172.21.15.52:6805/4255833408","heartbeat_back_addr":"172.21.15.52:6809/4255833408","heartbeat_front_addr":"172.21.15.52:6807/4255833408","state":["exists","up"]},{"osd":1,"uuid":"a24d8da8-c5ec-44d7-ab30-56f6ce3efda7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":4116540053},{"type":"v1","addr":"172.21.15.120:6801","nonce":4116540053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":4116540053},{"type":"v1","addr":"172.21.15.120:6803","nonce":4116540053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":4116540053},{"type":"v1","addr":"172.21.15.120:6807","nonce":4116540053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":4116540053},{"type":"v1","addr":"172.21.15.120:6805","nonce":4116540053}]},"public_addr":"172.21.15.120:6801/4116540053","cluster_addr":"172.21.15.120:6803/4116540053","heartbeat_back_addr":"172.21.15.120:6807/4116540053","heartbeat_front_addr":"172.21.15.120:6805/4116540053","state":["exists","up"]},{"osd":2,"uuid":"a7162106-ccbf-45bd-bc0e-051ee43f4214","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":2624326999},{"type":"v1","addr":"172.21.15.188:6801","nonce":2624326999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":2624326999},{"type":"v1","addr":"172.21.15.188:6803","nonce":2624326999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":2624326999},{"type":"v1","addr":"172.21.15.188:6807","nonce":2624326999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":2624326999},{"type":"v1","addr":"172.21.15.188:6805","nonce":2624326999}]},"public_addr":"172.21.15.188:6801/2624326999","cluster_addr":"172.21.15.188:6803/2624326999","heartbeat_back_addr":"172.21.15.188:6807/2624326999","heartbeat_front_addr":"172.21.15.188:6805/2624326999","state":["exists","up"]},{"osd":3,"uuid":"53680244-b49a-4120-b24a-9ad7a6341dc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":116897011},{"type":"v1","addr":"172.21.15.52:6811","nonce":116897011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":116897011},{"type":"v1","addr":"172.21.15.52:6813","nonce":116897011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":116897011},{"type":"v1","addr":"172.21.15.52:6817","nonce":116897011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":116897011},{"type":"v1","addr":"172.21.15.52:6815","nonce":116897011}]},"public_addr":"172.21.15.52:6811/116897011","cluster_addr":"172.21.15.52:6813/116897011","heartbeat_back_addr":"172.21.15.52:6817/116897011","heartbeat_front_addr":"172.21.15.52:6815/116897011","state":["exists","up"]},{"osd":4,"uuid":"9f72f1f4-2ef3-49c5-8410-c108767d2ef5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":1320690621},{"type":"v1","addr":"172.21.15.120:6809","nonce":1320690621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":1320690621},{"type":"v1","addr":"172.21.15.120:6811","nonce":1320690621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":1320690621},{"type":"v1","addr":"172.21.15.120:6815","nonce":1320690621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":1320690621},{"type":"v1","addr":"172.21.15.120:6813","nonce":1320690621}]},"public_addr":"172.21.15.120:6809/1320690621","cluster_addr":"172.21.15.120:6811/1320690621","heartbeat_back_addr":"172.21.15.120:6815/1320690621","heartbeat_front_addr":"172.21.15.120:6813/1320690621","state":["exists","up"]},{"osd":5,"uuid":"001b5706-7f27-4fa4-9b69-969e6d765abc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3951947483},{"type":"v1","addr":"172.21.15.188:6809","nonce":3951947483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3951947483},{"type":"v1","addr":"172.21.15.188:6811","nonce":3951947483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3951947483},{"type":"v1","addr":"172.21.15.188:6815","nonce":3951947483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3951947483},{"type":"v1","addr":"172.21.15.188:6813","nonce":3951947483}]},"public_addr":"172.21.15.188:6809/3951947483","cluster_addr":"172.21.15.188:6811/3951947483","heartbeat_back_addr":"172.21.15.188:6815/3951947483","heartbeat_front_addr":"172.21.15.188:6813/3951947483","state":["exists","up"]},{"osd":6,"uuid":"140c903e-9b31-4a81-9ccd-2edc0a2decf9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":359084919},{"type":"v1","addr":"172.21.15.52:6819","nonce":359084919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":359084919},{"type":"v1","addr":"172.21.15.52:6821","nonce":359084919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":359084919},{"type":"v1","addr":"172.21.15.52:6825","nonce":359084919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":359084919},{"type":"v1","addr":"172.21.15.52:6823","nonce":359084919}]},"public_addr":"172.21.15.52:6819/359084919","cluster_addr":"172.21.15.52:6821/359084919","heartbeat_back_addr":"172.21.15.52:6825/359084919","heartbeat_front_addr":"172.21.15.52:6823/359084919","state":["exists","up"]},{"osd":7,"uuid":"57b735d4-5072-4ca2-a3ff-3763ad84f15a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":3737722011},{"type":"v1","addr":"172.21.15.120:6817","nonce":3737722011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6818","nonce":3737722011},{"type":"v1","addr":"172.21.15.120:6819","nonce":3737722011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":3737722011},{"type":"v1","addr":"172.21.15.120:6823","nonce":3737722011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":3737722011},{"type":"v1","addr":"172.21.15.120:6821","nonce":3737722011}]},"public_addr":"172.21.15.120:6817/3737722011","cluster_addr":"172.21.15.120:6819/3737722011","heartbeat_back_addr":"172.21.15.120:6823/3737722011","heartbeat_front_addr":"172.21.15.120:6821/3737722011","state":["exists","up"]},{"osd":8,"uuid":"4ffbb43b-854a-4eac-99cf-7af89bc65409","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1444828441},{"type":"v1","addr":"172.21.15.188:6817","nonce":1444828441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1444828441},{"type":"v1","addr":"172.21.15.188:6819","nonce":1444828441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1444828441},{"type":"v1","addr":"172.21.15.188:6823","nonce":1444828441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1444828441},{"type":"v1","addr":"172.21.15.188:6821","nonce":1444828441}]},"public_addr":"172.21.15.188:6817/1444828441","cluster_addr":"172.21.15.188:6819/1444828441","heartbeat_back_addr":"172.21.15.188:6823/1444828441","heartbeat_front_addr":"172.21.15.188:6821/1444828441","state":["exists","up"]},{"osd":9,"uuid":"5bfd985c-8acc-4d00-8561-40bbd541c07b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":924440658},{"type":"v1","addr":"172.21.15.52:6827","nonce":924440658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":924440658},{"type":"v1","addr":"172.21.15.52:6829","nonce":924440658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":924440658},{"type":"v1","addr":"172.21.15.52:6833","nonce":924440658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":924440658},{"type":"v1","addr":"172.21.15.52:6831","nonce":924440658}]},"public_addr":"172.21.15.52:6827/924440658","cluster_addr":"172.21.15.52:6829/924440658","heartbeat_back_addr":"172.21.15.52:6833/924440658","heartbeat_front_addr":"172.21.15.52:6831/924440658","state":["exists","up"]},{"osd":10,"uuid":"0180cddf-e109-42cc-829a-185604276f02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6824","nonce":1531769083},{"type":"v1","addr":"172.21.15.120:6825","nonce":1531769083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":1531769083},{"type":"v1","addr":"172.21.15.120:6827","nonce":1531769083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":1531769083},{"type":"v1","addr":"172.21.15.120:6831","nonce":1531769083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6828","nonce":1531769083},{"type":"v1","addr":"172.21.15.120:6829","nonce":1531769083}]},"public_addr":"172.21.15.120:6825/1531769083","cluster_addr":"172.21.15.120:6827/1531769083","heartbeat_back_addr":"172.21.15.120:6831/1531769083","heartbeat_front_addr":"172.21.15.120:6829/1531769083","state":["exists","up"]},{"osd":11,"uuid":"73aec750-82d1-4800-b6a0-4e305912f9ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":3769050170},{"type":"v1","addr":"172.21.15.188:6825","nonce":3769050170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":3769050170},{"type":"v1","addr":"172.21.15.188:6827","nonce":3769050170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":3769050170},{"type":"v1","addr":"172.21.15.188:6831","nonce":3769050170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":3769050170},{"type":"v1","addr":"172.21.15.188:6829","nonce":3769050170}]},"public_addr":"172.21.15.188:6825/3769050170","cluster_addr":"172.21.15.188:6827/3769050170","heartbeat_back_addr":"172.21.15.188:6831/3769050170","heartbeat_front_addr":"172.21.15.188:6829/3769050170","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:44:16.455752+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:44:40.219735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:45:03.558899+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:45:29.155004+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:45:53.981098+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:46:19.412020+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:46:45.462640+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:47:11.484578+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:47:36.879054+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:48:03.392446+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:48:29.422173+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:48:55.523423+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":4}]},{"pgid":"3.6","mappings":[{"from":3,"to":7}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.52:0/3535271671":"2024-06-11T02:42:42.872312+0000","172.21.15.52:0/1092934380":"2024-06-11T02:42:42.872312+0000","172.21.15.52:6800/896317057":"2024-06-11T02:42:42.872312+0000","172.21.15.52:6801/3934479642":"2024-06-11T02:42:15.498118+0000","172.21.15.52:6800/3934479642":"2024-06-11T02:42:15.498118+0000","172.21.15.52:0/2759694132":"2024-06-11T02:42:15.498118+0000","172.21.15.52:0/2593665631":"2024-06-11T02:42:15.498118+0000","172.21.15.52:0/3151504634":"2024-06-11T02:42:15.498118+0000","172.21.15.52:6801/896317057":"2024-06-11T02:42:42.872312+0000","172.21.15.52:0/1254480982":"2024-06-11T02:42:42.872312+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-10T02:51:31.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:31.978+0000 7fc5d9a5c700 1 -- 172.21.15.52:0/2258588706 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fc5c00a2cb0 msgr2=0x7fc5c00a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:51:31.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:31.978+0000 7fc5d9a5c700 1 --2- 172.21.15.52:0/2258588706 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fc5c00a2cb0 0x7fc5c00a5170 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fc5c4006010 tx=0x7fc5c400b540 comp rx=0 tx=0).stop 2024-06-10T02:51:31.978 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:31.978+0000 7fc5d9a5c700 1 -- 172.21.15.52:0/2258588706 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc5d4102e80 msgr2=0x7fc5d419ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:51:31.978 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:31.978+0000 7fc5d9a5c700 1 --2- 172.21.15.52:0/2258588706 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc5d4102e80 0x7fc5d419ed50 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fc5bc00d980 tx=0x7fc5bc00dd40 comp rx=0 tx=0).stop 2024-06-10T02:51:31.978 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:31.979+0000 7fc5d9a5c700 1 -- 172.21.15.52:0/2258588706 shutdown_connections 2024-06-10T02:51:31.978 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:31.979+0000 7fc5d9a5c700 1 --2- 172.21.15.52:0/2258588706 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc5d41084f0 0x7fc5d41989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:51:31.978 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:31.979+0000 7fc5d9a5c700 1 --2- 172.21.15.52:0/2258588706 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc5d41024e0 0x7fc5d419e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:51:31.978 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:31.979+0000 7fc5d9a5c700 1 --2- 172.21.15.52:0/2258588706 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fc5c00a2cb0 0x7fc5c00a5170 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:51:31.978 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:31.979+0000 7fc5d9a5c700 1 --2- 172.21.15.52:0/2258588706 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc5d4102e80 0x7fc5d419ed50 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:51:31.978 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:31.979+0000 7fc5d9a5c700 1 -- 172.21.15.52:0/2258588706 >> 172.21.15.52:0/2258588706 conn(0x7fc5d4079530 msgr2=0x7fc5d41004b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:51:31.978 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:31.979+0000 7fc5d9a5c700 1 -- 172.21.15.52:0/2258588706 shutdown_connections 2024-06-10T02:51:31.978 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:31.979+0000 7fc5d9a5c700 1 -- 172.21.15.52:0/2258588706 wait complete. 2024-06-10T02:51:32.704 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-06-10T02:51:32.704 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph pg dump --format=json 2024-06-10T02:51:32.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:51:32 smithi188 ceph-mon[36599]: pgmap v352: 129 pgs: 129 active+clean; 640 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 4.7 KiB/s wr, 11 op/s 2024-06-10T02:51:32.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:51:32 smithi188 ceph-mon[36599]: from='client.? 172.21.15.52:0/2258588706' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T02:51:32.959 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:51:32.975 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:51:32 smithi052 ceph-mon[32362]: pgmap v352: 129 pgs: 129 active+clean; 640 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 4.7 KiB/s wr, 11 op/s 2024-06-10T02:51:32.975 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:51:32 smithi052 ceph-mon[32362]: from='client.? 172.21.15.52:0/2258588706' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T02:51:33.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:51:32 smithi120 ceph-mon[36634]: pgmap v352: 129 pgs: 129 active+clean; 640 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 4.7 KiB/s wr, 11 op/s 2024-06-10T02:51:33.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:51:32 smithi120 ceph-mon[36634]: from='client.? 172.21.15.52:0/2258588706' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T02:51:34.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:34.744+0000 7fdf2ba96700 1 -- 172.21.15.52:0/2199337399 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdf24107400 msgr2=0x7fdf241077e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:51:34.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:34.744+0000 7fdf2ba96700 1 --2- 172.21.15.52:0/2199337399 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdf24107400 0x7fdf241077e0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fdf20009990 tx=0x7fdf20009ca0 comp rx=0 tx=0).stop 2024-06-10T02:51:34.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:34.744+0000 7fdf2ba96700 1 -- 172.21.15.52:0/2199337399 shutdown_connections 2024-06-10T02:51:34.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:34.744+0000 7fdf2ba96700 1 --2- 172.21.15.52:0/2199337399 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdf24101d50 0x7fdf2410e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:51:34.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:34.744+0000 7fdf2ba96700 1 --2- 172.21.15.52:0/2199337399 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdf24107400 0x7fdf241077e0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:51:34.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:34.744+0000 7fdf2ba96700 1 --2- 172.21.15.52:0/2199337399 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdf24101390 0x7fdf24101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:51:34.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:34.744+0000 7fdf2ba96700 1 -- 172.21.15.52:0/2199337399 >> 172.21.15.52:0/2199337399 conn(0x7fdf24078340 msgr2=0x7fdf24078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:51:34.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:34.744+0000 7fdf2ba96700 1 -- 172.21.15.52:0/2199337399 shutdown_connections 2024-06-10T02:51:34.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:34.744+0000 7fdf2ba96700 1 -- 172.21.15.52:0/2199337399 wait complete. 2024-06-10T02:51:34.745 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:34.744+0000 7fdf2ba96700 1 Processor -- start 2024-06-10T02:51:34.745 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:34.745+0000 7fdf2ba96700 1 -- start start 2024-06-10T02:51:34.745 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:34.745+0000 7fdf2ba96700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdf24101390 0x7fdf24103070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:51:34.746 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:34.745+0000 7fdf2ba96700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdf24101d50 0x7fdf241035b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:51:34.746 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:34.745+0000 7fdf2ba96700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdf24107400 0x7fdf24103af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:51:34.746 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:34.745+0000 7fdf2ba96700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fdf24111670 con 0x7fdf24107400 2024-06-10T02:51:34.746 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:34.745+0000 7fdf2ba96700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fdf241114f0 con 0x7fdf24101390 2024-06-10T02:51:34.746 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:34.745+0000 7fdf2ba96700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fdf241117f0 con 0x7fdf24101d50 2024-06-10T02:51:34.746 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:34.745+0000 7fdf2a033700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdf24107400 0x7fdf24103af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:51:34.746 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:34.745+0000 7fdf2a033700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdf24107400 0x7fdf24103af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37910/0 (socket says 172.21.15.52:37910) 2024-06-10T02:51:34.746 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:34.745+0000 7fdf29031700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdf24101d50 0x7fdf241035b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:51:34.746 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:34.745+0000 7fdf2a033700 1 -- 172.21.15.52:0/3072894766 learned_addr learned my addr 172.21.15.52:0/3072894766 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:51:34.746 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:34.745+0000 7fdf29832700 1 --2- 172.21.15.52:0/3072894766 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdf24101390 0x7fdf24103070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:51:34.746 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:34.745+0000 7fdf2a033700 1 -- 172.21.15.52:0/3072894766 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdf24101d50 msgr2=0x7fdf241035b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:51:34.747 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:34.745+0000 7fdf2a033700 1 --2- 172.21.15.52:0/3072894766 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdf24101d50 0x7fdf241035b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:51:34.747 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:34.745+0000 7fdf2a033700 1 -- 172.21.15.52:0/3072894766 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdf24101390 msgr2=0x7fdf24103070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:51:34.747 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:34.745+0000 7fdf2a033700 1 --2- 172.21.15.52:0/3072894766 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdf24101390 0x7fdf24103070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:51:34.747 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:34.745+0000 7fdf2a033700 1 -- 172.21.15.52:0/3072894766 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf241a70e0 con 0x7fdf24107400 2024-06-10T02:51:34.747 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:34.745+0000 7fdf29832700 1 --2- 172.21.15.52:0/3072894766 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdf24101390 0x7fdf24103070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T02:51:34.747 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:34.745+0000 7fdf2a033700 1 --2- 172.21.15.52:0/3072894766 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdf24107400 0x7fdf24103af0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fdf1c007ee0 tx=0x7fdf1c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:51:34.747 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:34.746+0000 7fdf1affd700 1 -- 172.21.15.52:0/3072894766 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf1c00dbd0 con 0x7fdf24107400 2024-06-10T02:51:34.747 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:34.746+0000 7fdf1affd700 1 -- 172.21.15.52:0/3072894766 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf1c00dd70 con 0x7fdf24107400 2024-06-10T02:51:34.747 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:34.746+0000 7fdf2ba96700 1 -- 172.21.15.52:0/3072894766 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf241a73d0 con 0x7fdf24107400 2024-06-10T02:51:34.748 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:34.746+0000 7fdf1affd700 1 -- 172.21.15.52:0/3072894766 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf1c005a60 con 0x7fdf24107400 2024-06-10T02:51:34.748 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:34.746+0000 7fdf2ba96700 1 -- 172.21.15.52:0/3072894766 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf241a78c0 con 0x7fdf24107400 2024-06-10T02:51:34.748 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:34.747+0000 7fdf2ba96700 1 -- 172.21.15.52:0/3072894766 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf24069180 con 0x7fdf24107400 2024-06-10T02:51:34.748 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:34.748+0000 7fdf1affd700 1 -- 172.21.15.52:0/3072894766 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fdf1c01a020 con 0x7fdf24107400 2024-06-10T02:51:34.748 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:34.749+0000 7fdf1affd700 1 --2- 172.21.15.52:0/3072894766 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fdf100a2cb0 0x7fdf100a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:51:34.749 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:34.749+0000 7fdf29832700 1 --2- 172.21.15.52:0/3072894766 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fdf100a2cb0 0x7fdf100a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:51:34.749 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:34.749+0000 7fdf1affd700 1 -- 172.21.15.52:0/3072894766 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9092+0+0 (secure 0 0 0) 0x7fdf1c0cec60 con 0x7fdf24107400 2024-06-10T02:51:34.749 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:34.749+0000 7fdf29832700 1 --2- 172.21.15.52:0/3072894766 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fdf100a2cb0 0x7fdf100a5170 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fdf20006010 tx=0x7fdf2000b540 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:51:34.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:34.753+0000 7fdf1affd700 1 -- 172.21.15.52:0/3072894766 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fdf1c010070 con 0x7fdf24107400 2024-06-10T02:51:34.752 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:51:34 smithi052 ceph-mon[32362]: pgmap v353: 129 pgs: 129 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 4.9 KiB/s wr, 14 op/s 2024-06-10T02:51:34.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:51:34 smithi188 ceph-mon[36599]: pgmap v353: 129 pgs: 129 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 4.9 KiB/s wr, 14 op/s 2024-06-10T02:51:34.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:34.958+0000 7fdf2ba96700 1 -- 172.21.15.52:0/3072894766 --> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fdf241051b0 con 0x7fdf100a2cb0 2024-06-10T02:51:34.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:34.968+0000 7fdf1affd700 1 -- 172.21.15.52:0/3072894766 <== mgr.14150 v2:172.21.15.52:6800/3214763969 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340685 (secure 0 0 0) 0x7fdf241051b0 con 0x7fdf100a2cb0 2024-06-10T02:51:34.975 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-10T02:51:34.978 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:34.972+0000 7fdf2ba96700 1 -- 172.21.15.52:0/3072894766 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fdf100a2cb0 msgr2=0x7fdf100a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:51:34.978 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:34.972+0000 7fdf2ba96700 1 --2- 172.21.15.52:0/3072894766 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fdf100a2cb0 0x7fdf100a5170 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fdf20006010 tx=0x7fdf2000b540 comp rx=0 tx=0).stop 2024-06-10T02:51:34.978 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:34.972+0000 7fdf2ba96700 1 -- 172.21.15.52:0/3072894766 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdf24107400 msgr2=0x7fdf24103af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:51:34.978 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:34.972+0000 7fdf2ba96700 1 --2- 172.21.15.52:0/3072894766 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdf24107400 0x7fdf24103af0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fdf1c007ee0 tx=0x7fdf1c00d3b0 comp rx=0 tx=0).stop 2024-06-10T02:51:34.978 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:34.972+0000 7fdf2ba96700 1 -- 172.21.15.52:0/3072894766 shutdown_connections 2024-06-10T02:51:34.978 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:34.972+0000 7fdf2ba96700 1 --2- 172.21.15.52:0/3072894766 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fdf100a2cb0 0x7fdf100a5170 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:51:34.978 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:34.972+0000 7fdf2ba96700 1 --2- 172.21.15.52:0/3072894766 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdf24107400 0x7fdf24103af0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:51:34.978 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:34.972+0000 7fdf2ba96700 1 --2- 172.21.15.52:0/3072894766 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdf24101390 0x7fdf24103070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:51:34.978 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:34.972+0000 7fdf2ba96700 1 --2- 172.21.15.52:0/3072894766 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdf24101d50 0x7fdf241035b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:51:34.979 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:34.972+0000 7fdf2ba96700 1 -- 172.21.15.52:0/3072894766 >> 172.21.15.52:0/3072894766 conn(0x7fdf24078340 msgr2=0x7fdf24100630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:51:34.979 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:34.972+0000 7fdf2ba96700 1 -- 172.21.15.52:0/3072894766 shutdown_connections 2024-06-10T02:51:34.979 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:34.972+0000 7fdf2ba96700 1 -- 172.21.15.52:0/3072894766 wait complete. 2024-06-10T02:51:34.979 INFO:teuthology.orchestra.run.smithi052.stderr:dumped all 2024-06-10T02:51:35.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:51:34 smithi120 ceph-mon[36634]: pgmap v353: 129 pgs: 129 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 4.9 KiB/s wr, 14 op/s 2024-06-10T02:51:35.745 INFO:teuthology.orchestra.run.smithi052.stdout:{"pg_ready":true,"pg_map":{"version":353,"stamp":"2024-06-10T02:51:32.997932+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":656974,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":395,"num_read_kb":318,"num_write":431,"num_write_kb":3080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1181318,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":352,"ondisk_log_size":352,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":379044,"kb_used_data":13252,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124562780,"statfs":{"total":1151940427776,"available":1151552286720,"internally_reserved":0,"allocated":13570048,"data_stored":8594218,"data_compressed":426435,"data_compressed_allocated":577536,"data_compressed_original":2654208,"omap_allocated":19141,"internal_metadata":374453563},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":48067,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":150,"num_read_kb":119,"num_write":28,"num_write_kb":59,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003415"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.780589+0000","last_change":"2024-06-10T02:50:02.209146+0000","last_active":"2024-06-10T02:50:44.780589+0000","last_peered":"2024-06-10T02:50:44.780589+0000","last_clean":"2024-06-10T02:50:44.780589+0000","last_became_active":"2024-06-10T02:50:02.208027+0000","last_became_peered":"2024-06-10T02:50:02.208027+0000","last_unstale":"2024-06-10T02:50:44.780589+0000","last_undegraded":"2024-06-10T02:50:44.780589+0000","last_fullsized":"2024-06-10T02:50:44.780589+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:35:34.765849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.736487+0000","last_change":"2024-06-10T02:50:02.198767+0000","last_active":"2024-06-10T02:50:44.736487+0000","last_peered":"2024-06-10T02:50:44.736487+0000","last_clean":"2024-06-10T02:50:44.736487+0000","last_became_active":"2024-06-10T02:50:02.198450+0000","last_became_peered":"2024-06-10T02:50:02.198450+0000","last_unstale":"2024-06-10T02:50:44.736487+0000","last_undegraded":"2024-06-10T02:50:44.736487+0000","last_fullsized":"2024-06-10T02:50:44.736487+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:49:56.666083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.736775+0000","last_change":"2024-06-10T02:50:02.208885+0000","last_active":"2024-06-10T02:50:44.736775+0000","last_peered":"2024-06-10T02:50:44.736775+0000","last_clean":"2024-06-10T02:50:44.736775+0000","last_became_active":"2024-06-10T02:50:02.206953+0000","last_became_peered":"2024-06-10T02:50:02.206953+0000","last_unstale":"2024-06-10T02:50:44.736775+0000","last_undegraded":"2024-06-10T02:50:44.736775+0000","last_fullsized":"2024-06-10T02:50:44.736775+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:41:23.928939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.780539+0000","last_change":"2024-06-10T02:50:02.208632+0000","last_active":"2024-06-10T02:50:44.780539+0000","last_peered":"2024-06-10T02:50:44.780539+0000","last_clean":"2024-06-10T02:50:44.780539+0000","last_became_active":"2024-06-10T02:50:02.207587+0000","last_became_peered":"2024-06-10T02:50:02.207587+0000","last_unstale":"2024-06-10T02:50:44.780539+0000","last_undegraded":"2024-06-10T02:50:44.780539+0000","last_fullsized":"2024-06-10T02:50:44.780539+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T14:34:36.720170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.739593+0000","last_change":"2024-06-10T02:50:02.197966+0000","last_active":"2024-06-10T02:50:44.739593+0000","last_peered":"2024-06-10T02:50:44.739593+0000","last_clean":"2024-06-10T02:50:44.739593+0000","last_became_active":"2024-06-10T02:50:02.197612+0000","last_became_peered":"2024-06-10T02:50:02.197612+0000","last_unstale":"2024-06-10T02:50:44.739593+0000","last_undegraded":"2024-06-10T02:50:44.739593+0000","last_fullsized":"2024-06-10T02:50:44.739593+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:24:53.804613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.739586+0000","last_change":"2024-06-10T02:50:02.201952+0000","last_active":"2024-06-10T02:50:44.739586+0000","last_peered":"2024-06-10T02:50:44.739586+0000","last_clean":"2024-06-10T02:50:44.739586+0000","last_became_active":"2024-06-10T02:50:02.200828+0000","last_became_peered":"2024-06-10T02:50:02.200828+0000","last_unstale":"2024-06-10T02:50:44.739586+0000","last_undegraded":"2024-06-10T02:50:44.739586+0000","last_fullsized":"2024-06-10T02:50:44.739586+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:46:28.518846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.735333+0000","last_change":"2024-06-10T02:50:02.203483+0000","last_active":"2024-06-10T02:50:44.735333+0000","last_peered":"2024-06-10T02:50:44.735333+0000","last_clean":"2024-06-10T02:50:44.735333+0000","last_became_active":"2024-06-10T02:50:02.203153+0000","last_became_peered":"2024-06-10T02:50:02.203153+0000","last_unstale":"2024-06-10T02:50:44.735333+0000","last_undegraded":"2024-06-10T02:50:44.735333+0000","last_fullsized":"2024-06-10T02:50:44.735333+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:29:30.422449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.733706+0000","last_change":"2024-06-10T02:50:02.201686+0000","last_active":"2024-06-10T02:50:44.733706+0000","last_peered":"2024-06-10T02:50:44.733706+0000","last_clean":"2024-06-10T02:50:44.733706+0000","last_became_active":"2024-06-10T02:50:02.200973+0000","last_became_peered":"2024-06-10T02:50:02.200973+0000","last_unstale":"2024-06-10T02:50:44.733706+0000","last_undegraded":"2024-06-10T02:50:44.733706+0000","last_fullsized":"2024-06-10T02:50:44.733706+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:41:13.712675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.740098+0000","last_change":"2024-06-10T02:50:02.210497+0000","last_active":"2024-06-10T02:50:44.740098+0000","last_peered":"2024-06-10T02:50:44.740098+0000","last_clean":"2024-06-10T02:50:44.740098+0000","last_became_active":"2024-06-10T02:50:02.209675+0000","last_became_peered":"2024-06-10T02:50:02.209675+0000","last_unstale":"2024-06-10T02:50:44.740098+0000","last_undegraded":"2024-06-10T02:50:44.740098+0000","last_fullsized":"2024-06-10T02:50:44.740098+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:50:29.879669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":40,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.741139+0000","last_change":"2024-06-10T02:50:44.741139+0000","last_active":"2024-06-10T02:50:44.741139+0000","last_peered":"2024-06-10T02:50:44.741139+0000","last_clean":"2024-06-10T02:50:44.741139+0000","last_became_active":"2024-06-10T02:50:44.740685+0000","last_became_peered":"2024-06-10T02:50:44.740685+0000","last_unstale":"2024-06-10T02:50:44.741139+0000","last_undegraded":"2024-06-10T02:50:44.741139+0000","last_fullsized":"2024-06-10T02:50:44.741139+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:58:59.872732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.736093+0000","last_change":"2024-06-10T02:50:02.206993+0000","last_active":"2024-06-10T02:50:44.736093+0000","last_peered":"2024-06-10T02:50:44.736093+0000","last_clean":"2024-06-10T02:50:44.736093+0000","last_became_active":"2024-06-10T02:50:02.206650+0000","last_became_peered":"2024-06-10T02:50:02.206650+0000","last_unstale":"2024-06-10T02:50:44.736093+0000","last_undegraded":"2024-06-10T02:50:44.736093+0000","last_fullsized":"2024-06-10T02:50:44.736093+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T14:27:49.555350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.780395+0000","last_change":"2024-06-10T02:50:02.200816+0000","last_active":"2024-06-10T02:50:44.780395+0000","last_peered":"2024-06-10T02:50:44.780395+0000","last_clean":"2024-06-10T02:50:44.780395+0000","last_became_active":"2024-06-10T02:50:02.200503+0000","last_became_peered":"2024-06-10T02:50:02.200503+0000","last_unstale":"2024-06-10T02:50:44.780395+0000","last_undegraded":"2024-06-10T02:50:44.780395+0000","last_fullsized":"2024-06-10T02:50:44.780395+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:01:08.326728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.780143+0000","last_change":"2024-06-10T02:50:02.203644+0000","last_active":"2024-06-10T02:50:44.780143+0000","last_peered":"2024-06-10T02:50:44.780143+0000","last_clean":"2024-06-10T02:50:44.780143+0000","last_became_active":"2024-06-10T02:50:02.203100+0000","last_became_peered":"2024-06-10T02:50:02.203100+0000","last_unstale":"2024-06-10T02:50:44.780143+0000","last_undegraded":"2024-06-10T02:50:44.780143+0000","last_fullsized":"2024-06-10T02:50:44.780143+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:01:42.730089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.735964+0000","last_change":"2024-06-10T02:50:02.203330+0000","last_active":"2024-06-10T02:50:44.735964+0000","last_peered":"2024-06-10T02:50:44.735964+0000","last_clean":"2024-06-10T02:50:44.735964+0000","last_became_active":"2024-06-10T02:50:02.202995+0000","last_became_peered":"2024-06-10T02:50:02.202995+0000","last_unstale":"2024-06-10T02:50:44.735964+0000","last_undegraded":"2024-06-10T02:50:44.735964+0000","last_fullsized":"2024-06-10T02:50:44.735964+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:17:23.170591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.736189+0000","last_change":"2024-06-10T02:50:02.204875+0000","last_active":"2024-06-10T02:50:44.736189+0000","last_peered":"2024-06-10T02:50:44.736189+0000","last_clean":"2024-06-10T02:50:44.736189+0000","last_became_active":"2024-06-10T02:50:02.200706+0000","last_became_peered":"2024-06-10T02:50:02.200706+0000","last_unstale":"2024-06-10T02:50:44.736189+0000","last_undegraded":"2024-06-10T02:50:44.736189+0000","last_fullsized":"2024-06-10T02:50:44.736189+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:27:12.282772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.738872+0000","last_change":"2024-06-10T02:50:02.203187+0000","last_active":"2024-06-10T02:50:44.738872+0000","last_peered":"2024-06-10T02:50:44.738872+0000","last_clean":"2024-06-10T02:50:44.738872+0000","last_became_active":"2024-06-10T02:50:02.202034+0000","last_became_peered":"2024-06-10T02:50:02.202034+0000","last_unstale":"2024-06-10T02:50:44.738872+0000","last_undegraded":"2024-06-10T02:50:44.738872+0000","last_fullsized":"2024-06-10T02:50:44.738872+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:59:44.783084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.780351+0000","last_change":"2024-06-10T02:50:02.209102+0000","last_active":"2024-06-10T02:50:44.780351+0000","last_peered":"2024-06-10T02:50:44.780351+0000","last_clean":"2024-06-10T02:50:44.780351+0000","last_became_active":"2024-06-10T02:50:02.208730+0000","last_became_peered":"2024-06-10T02:50:02.208730+0000","last_unstale":"2024-06-10T02:50:44.780351+0000","last_undegraded":"2024-06-10T02:50:44.780351+0000","last_fullsized":"2024-06-10T02:50:44.780351+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:08:38.838415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.738035+0000","last_change":"2024-06-10T02:50:02.204008+0000","last_active":"2024-06-10T02:50:44.738035+0000","last_peered":"2024-06-10T02:50:44.738035+0000","last_clean":"2024-06-10T02:50:44.738035+0000","last_became_active":"2024-06-10T02:50:02.203655+0000","last_became_peered":"2024-06-10T02:50:02.203655+0000","last_unstale":"2024-06-10T02:50:44.738035+0000","last_undegraded":"2024-06-10T02:50:44.738035+0000","last_fullsized":"2024-06-10T02:50:44.738035+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:32:15.972662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.737751+0000","last_change":"2024-06-10T02:50:02.204917+0000","last_active":"2024-06-10T02:50:44.737751+0000","last_peered":"2024-06-10T02:50:44.737751+0000","last_clean":"2024-06-10T02:50:44.737751+0000","last_became_active":"2024-06-10T02:50:02.204568+0000","last_became_peered":"2024-06-10T02:50:02.204568+0000","last_unstale":"2024-06-10T02:50:44.737751+0000","last_undegraded":"2024-06-10T02:50:44.737751+0000","last_fullsized":"2024-06-10T02:50:44.737751+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:49:46.802665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.737040+0000","last_change":"2024-06-10T02:50:12.240697+0000","last_active":"2024-06-10T02:50:44.737040+0000","last_peered":"2024-06-10T02:50:44.737040+0000","last_clean":"2024-06-10T02:50:44.737040+0000","last_became_active":"2024-06-10T02:50:12.240229+0000","last_became_peered":"2024-06-10T02:50:12.240229+0000","last_unstale":"2024-06-10T02:50:44.737040+0000","last_undegraded":"2024-06-10T02:50:44.737040+0000","last_fullsized":"2024-06-10T02:50:44.737040+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:45:55.071960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"98'13","reported_seq":87,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:51:28.662094+0000","last_change":"2024-06-10T02:50:44.786783+0000","last_active":"2024-06-10T02:51:28.662094+0000","last_peered":"2024-06-10T02:51:28.662094+0000","last_clean":"2024-06-10T02:51:28.662094+0000","last_became_active":"2024-06-10T02:50:44.781773+0000","last_became_peered":"2024-06-10T02:50:44.781773+0000","last_unstale":"2024-06-10T02:51:28.662094+0000","last_undegraded":"2024-06-10T02:51:28.662094+0000","last_fullsized":"2024-06-10T02:51:28.662094+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:31:06.025854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":22,"num_read_kb":22,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,4],"acting":[9,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.734284+0000","last_change":"2024-06-10T02:50:02.202573+0000","last_active":"2024-06-10T02:50:44.734284+0000","last_peered":"2024-06-10T02:50:44.734284+0000","last_clean":"2024-06-10T02:50:44.734284+0000","last_became_active":"2024-06-10T02:50:02.202241+0000","last_became_peered":"2024-06-10T02:50:02.202241+0000","last_unstale":"2024-06-10T02:50:44.734284+0000","last_undegraded":"2024-06-10T02:50:44.734284+0000","last_fullsized":"2024-06-10T02:50:44.734284+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:31:40.230851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.779776+0000","last_change":"2024-06-10T02:50:12.238372+0000","last_active":"2024-06-10T02:50:44.779776+0000","last_peered":"2024-06-10T02:50:44.779776+0000","last_clean":"2024-06-10T02:50:44.779776+0000","last_became_active":"2024-06-10T02:50:12.237884+0000","last_became_peered":"2024-06-10T02:50:12.237884+0000","last_unstale":"2024-06-10T02:50:44.779776+0000","last_undegraded":"2024-06-10T02:50:44.779776+0000","last_fullsized":"2024-06-10T02:50:44.779776+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:55:31.432655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"98'2","reported_seq":42,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.734660+0000","last_change":"2024-06-10T02:49:59.031528+0000","last_active":"2024-06-10T02:50:44.734660+0000","last_peered":"2024-06-10T02:50:44.734660+0000","last_clean":"2024-06-10T02:50:44.734660+0000","last_became_active":"2024-06-10T02:49:59.031147+0000","last_became_peered":"2024-06-10T02:49:59.031147+0000","last_unstale":"2024-06-10T02:50:44.734660+0000","last_undegraded":"2024-06-10T02:50:44.734660+0000","last_fullsized":"2024-06-10T02:50:44.734660+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:17:08.534396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.736728+0000","last_change":"2024-06-10T02:50:02.202858+0000","last_active":"2024-06-10T02:50:44.736728+0000","last_peered":"2024-06-10T02:50:44.736728+0000","last_clean":"2024-06-10T02:50:44.736728+0000","last_became_active":"2024-06-10T02:50:02.202263+0000","last_became_peered":"2024-06-10T02:50:02.202263+0000","last_unstale":"2024-06-10T02:50:44.736728+0000","last_undegraded":"2024-06-10T02:50:44.736728+0000","last_fullsized":"2024-06-10T02:50:44.736728+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:23:15.382762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.780045+0000","last_change":"2024-06-10T02:50:12.237370+0000","last_active":"2024-06-10T02:50:44.780045+0000","last_peered":"2024-06-10T02:50:44.780045+0000","last_clean":"2024-06-10T02:50:44.780045+0000","last_became_active":"2024-06-10T02:50:12.236803+0000","last_became_peered":"2024-06-10T02:50:12.236803+0000","last_unstale":"2024-06-10T02:50:44.780045+0000","last_undegraded":"2024-06-10T02:50:44.780045+0000","last_fullsized":"2024-06-10T02:50:44.780045+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:06:40.695710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"98'12","reported_seq":53,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:51:09.633101+0000","last_change":"2024-06-10T02:49:59.000911+0000","last_active":"2024-06-10T02:51:09.633101+0000","last_peered":"2024-06-10T02:51:09.633101+0000","last_clean":"2024-06-10T02:51:09.633101+0000","last_became_active":"2024-06-10T02:49:59.000571+0000","last_became_peered":"2024-06-10T02:49:59.000571+0000","last_unstale":"2024-06-10T02:51:09.633101+0000","last_undegraded":"2024-06-10T02:51:09.633101+0000","last_fullsized":"2024-06-10T02:51:09.633101+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:49:24.635793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.738682+0000","last_change":"2024-06-10T02:50:02.209614+0000","last_active":"2024-06-10T02:50:44.738682+0000","last_peered":"2024-06-10T02:50:44.738682+0000","last_clean":"2024-06-10T02:50:44.738682+0000","last_became_active":"2024-06-10T02:50:02.209251+0000","last_became_peered":"2024-06-10T02:50:02.209251+0000","last_unstale":"2024-06-10T02:50:44.738682+0000","last_undegraded":"2024-06-10T02:50:44.738682+0000","last_fullsized":"2024-06-10T02:50:44.738682+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:35:32.256081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.779720+0000","last_change":"2024-06-10T02:50:12.237811+0000","last_active":"2024-06-10T02:50:44.779720+0000","last_peered":"2024-06-10T02:50:44.779720+0000","last_clean":"2024-06-10T02:50:44.779720+0000","last_became_active":"2024-06-10T02:50:12.237017+0000","last_became_peered":"2024-06-10T02:50:12.237017+0000","last_unstale":"2024-06-10T02:50:44.779720+0000","last_undegraded":"2024-06-10T02:50:44.779720+0000","last_fullsized":"2024-06-10T02:50:44.779720+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:14:29.818885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"98'5","reported_seq":46,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:51:08.627882+0000","last_change":"2024-06-10T02:49:58.996971+0000","last_active":"2024-06-10T02:51:08.627882+0000","last_peered":"2024-06-10T02:51:08.627882+0000","last_clean":"2024-06-10T02:51:08.627882+0000","last_became_active":"2024-06-10T02:49:58.996601+0000","last_became_peered":"2024-06-10T02:49:58.996601+0000","last_unstale":"2024-06-10T02:51:08.627882+0000","last_undegraded":"2024-06-10T02:51:08.627882+0000","last_fullsized":"2024-06-10T02:51:08.627882+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:26:45.963351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":40,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.745816+0000","last_change":"2024-06-10T02:50:44.745816+0000","last_active":"2024-06-10T02:50:44.745816+0000","last_peered":"2024-06-10T02:50:44.745816+0000","last_clean":"2024-06-10T02:50:44.745816+0000","last_became_active":"2024-06-10T02:50:44.745386+0000","last_became_peered":"2024-06-10T02:50:44.745386+0000","last_unstale":"2024-06-10T02:50:44.745816+0000","last_undegraded":"2024-06-10T02:50:44.745816+0000","last_fullsized":"2024-06-10T02:50:44.745816+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:08:53.977133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.737434+0000","last_change":"2024-06-10T02:50:12.236835+0000","last_active":"2024-06-10T02:50:44.737434+0000","last_peered":"2024-06-10T02:50:44.737434+0000","last_clean":"2024-06-10T02:50:44.737434+0000","last_became_active":"2024-06-10T02:50:12.236309+0000","last_became_peered":"2024-06-10T02:50:12.236309+0000","last_unstale":"2024-06-10T02:50:44.737434+0000","last_undegraded":"2024-06-10T02:50:44.737434+0000","last_fullsized":"2024-06-10T02:50:44.737434+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:09:02.789286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"98'6","reported_seq":71,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:51:29.664884+0000","last_change":"2024-06-10T02:49:58.997725+0000","last_active":"2024-06-10T02:51:29.664884+0000","last_peered":"2024-06-10T02:51:29.664884+0000","last_clean":"2024-06-10T02:51:29.664884+0000","last_became_active":"2024-06-10T02:49:58.997375+0000","last_became_peered":"2024-06-10T02:49:58.997375+0000","last_unstale":"2024-06-10T02:51:29.664884+0000","last_undegraded":"2024-06-10T02:51:29.664884+0000","last_fullsized":"2024-06-10T02:51:29.664884+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:28:49.714356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1012,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":25,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.739670+0000","last_change":"2024-06-10T02:50:02.201247+0000","last_active":"2024-06-10T02:50:44.739670+0000","last_peered":"2024-06-10T02:50:44.739670+0000","last_clean":"2024-06-10T02:50:44.739670+0000","last_became_active":"2024-06-10T02:50:02.200897+0000","last_became_peered":"2024-06-10T02:50:02.200897+0000","last_unstale":"2024-06-10T02:50:44.739670+0000","last_undegraded":"2024-06-10T02:50:44.739670+0000","last_fullsized":"2024-06-10T02:50:44.739670+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:01:58.166644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.737778+0000","last_change":"2024-06-10T02:50:12.243097+0000","last_active":"2024-06-10T02:50:44.737778+0000","last_peered":"2024-06-10T02:50:44.737778+0000","last_clean":"2024-06-10T02:50:44.737778+0000","last_became_active":"2024-06-10T02:50:12.242501+0000","last_became_peered":"2024-06-10T02:50:12.242501+0000","last_unstale":"2024-06-10T02:50:44.737778+0000","last_undegraded":"2024-06-10T02:50:44.737778+0000","last_fullsized":"2024-06-10T02:50:44.737778+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T14:42:44.220340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"98'7","reported_seq":47,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:51:07.617849+0000","last_change":"2024-06-10T02:49:58.996956+0000","last_active":"2024-06-10T02:51:07.617849+0000","last_peered":"2024-06-10T02:51:07.617849+0000","last_clean":"2024-06-10T02:51:07.617849+0000","last_became_active":"2024-06-10T02:49:58.996634+0000","last_became_peered":"2024-06-10T02:49:58.996634+0000","last_unstale":"2024-06-10T02:51:07.617849+0000","last_undegraded":"2024-06-10T02:51:07.617849+0000","last_fullsized":"2024-06-10T02:51:07.617849+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:25:02.364472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.741747+0000","last_change":"2024-06-10T02:50:02.205808+0000","last_active":"2024-06-10T02:50:44.741747+0000","last_peered":"2024-06-10T02:50:44.741747+0000","last_clean":"2024-06-10T02:50:44.741747+0000","last_became_active":"2024-06-10T02:50:02.203558+0000","last_became_peered":"2024-06-10T02:50:02.203558+0000","last_unstale":"2024-06-10T02:50:44.741747+0000","last_undegraded":"2024-06-10T02:50:44.741747+0000","last_fullsized":"2024-06-10T02:50:44.741747+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T13:43:23.315375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.780298+0000","last_change":"2024-06-10T02:50:12.242005+0000","last_active":"2024-06-10T02:50:44.780298+0000","last_peered":"2024-06-10T02:50:44.780298+0000","last_clean":"2024-06-10T02:50:44.780298+0000","last_became_active":"2024-06-10T02:50:12.241295+0000","last_became_peered":"2024-06-10T02:50:12.241295+0000","last_unstale":"2024-06-10T02:50:44.780298+0000","last_undegraded":"2024-06-10T02:50:44.780298+0000","last_fullsized":"2024-06-10T02:50:44.780298+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:32:52.702756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"98'3","reported_seq":43,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:51:06.613737+0000","last_change":"2024-06-10T02:49:58.995579+0000","last_active":"2024-06-10T02:51:06.613737+0000","last_peered":"2024-06-10T02:51:06.613737+0000","last_clean":"2024-06-10T02:51:06.613737+0000","last_became_active":"2024-06-10T02:49:58.995208+0000","last_became_peered":"2024-06-10T02:49:58.995208+0000","last_unstale":"2024-06-10T02:51:06.613737+0000","last_undegraded":"2024-06-10T02:51:06.613737+0000","last_fullsized":"2024-06-10T02:51:06.613737+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:26:03.513482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.733787+0000","last_change":"2024-06-10T02:50:02.197090+0000","last_active":"2024-06-10T02:50:44.733787+0000","last_peered":"2024-06-10T02:50:44.733787+0000","last_clean":"2024-06-10T02:50:44.733787+0000","last_became_active":"2024-06-10T02:50:02.196746+0000","last_became_peered":"2024-06-10T02:50:02.196746+0000","last_unstale":"2024-06-10T02:50:44.733787+0000","last_undegraded":"2024-06-10T02:50:44.733787+0000","last_fullsized":"2024-06-10T02:50:44.733787+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:52:09.987978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.736335+0000","last_change":"2024-06-10T02:50:12.234632+0000","last_active":"2024-06-10T02:50:44.736335+0000","last_peered":"2024-06-10T02:50:44.736335+0000","last_clean":"2024-06-10T02:50:44.736335+0000","last_became_active":"2024-06-10T02:50:12.234266+0000","last_became_peered":"2024-06-10T02:50:12.234266+0000","last_unstale":"2024-06-10T02:50:44.736335+0000","last_undegraded":"2024-06-10T02:50:44.736335+0000","last_fullsized":"2024-06-10T02:50:44.736335+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:21:27.348441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"98'3","reported_seq":44,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:51:08.628643+0000","last_change":"2024-06-10T02:49:58.999086+0000","last_active":"2024-06-10T02:51:08.628643+0000","last_peered":"2024-06-10T02:51:08.628643+0000","last_clean":"2024-06-10T02:51:08.628643+0000","last_became_active":"2024-06-10T02:49:58.998767+0000","last_became_peered":"2024-06-10T02:49:58.998767+0000","last_unstale":"2024-06-10T02:51:08.628643+0000","last_undegraded":"2024-06-10T02:51:08.628643+0000","last_fullsized":"2024-06-10T02:51:08.628643+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:28:12.922391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.738078+0000","last_change":"2024-06-10T02:50:02.196641+0000","last_active":"2024-06-10T02:50:44.738078+0000","last_peered":"2024-06-10T02:50:44.738078+0000","last_clean":"2024-06-10T02:50:44.738078+0000","last_became_active":"2024-06-10T02:50:02.196285+0000","last_became_peered":"2024-06-10T02:50:02.196285+0000","last_unstale":"2024-06-10T02:50:44.738078+0000","last_undegraded":"2024-06-10T02:50:44.738078+0000","last_fullsized":"2024-06-10T02:50:44.738078+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T13:55:31.284028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.733867+0000","last_change":"2024-06-10T02:50:12.240988+0000","last_active":"2024-06-10T02:50:44.733867+0000","last_peered":"2024-06-10T02:50:44.733867+0000","last_clean":"2024-06-10T02:50:44.733867+0000","last_became_active":"2024-06-10T02:50:12.240490+0000","last_became_peered":"2024-06-10T02:50:12.240490+0000","last_unstale":"2024-06-10T02:50:44.733867+0000","last_undegraded":"2024-06-10T02:50:44.733867+0000","last_fullsized":"2024-06-10T02:50:44.733867+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:54:36.015221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"98'4","reported_seq":90,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:51:31.666668+0000","last_change":"2024-06-10T02:49:58.989557+0000","last_active":"2024-06-10T02:51:31.666668+0000","last_peered":"2024-06-10T02:51:31.666668+0000","last_clean":"2024-06-10T02:51:31.666668+0000","last_became_active":"2024-06-10T02:49:58.989101+0000","last_became_peered":"2024-06-10T02:49:58.989101+0000","last_unstale":"2024-06-10T02:51:31.666668+0000","last_undegraded":"2024-06-10T02:51:31.666668+0000","last_fullsized":"2024-06-10T02:51:31.666668+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:27:42.182012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":46,"num_read_kb":46,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.739177+0000","last_change":"2024-06-10T02:50:02.208752+0000","last_active":"2024-06-10T02:50:44.739177+0000","last_peered":"2024-06-10T02:50:44.739177+0000","last_clean":"2024-06-10T02:50:44.739177+0000","last_became_active":"2024-06-10T02:50:02.208420+0000","last_became_peered":"2024-06-10T02:50:02.208420+0000","last_unstale":"2024-06-10T02:50:44.739177+0000","last_undegraded":"2024-06-10T02:50:44.739177+0000","last_fullsized":"2024-06-10T02:50:44.739177+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:28:09.412317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.737800+0000","last_change":"2024-06-10T02:50:12.236397+0000","last_active":"2024-06-10T02:50:44.737800+0000","last_peered":"2024-06-10T02:50:44.737800+0000","last_clean":"2024-06-10T02:50:44.737800+0000","last_became_active":"2024-06-10T02:50:12.235972+0000","last_became_peered":"2024-06-10T02:50:12.235972+0000","last_unstale":"2024-06-10T02:50:44.737800+0000","last_undegraded":"2024-06-10T02:50:44.737800+0000","last_fullsized":"2024-06-10T02:50:44.737800+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:38:41.723895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"98'4","reported_seq":44,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:51:08.626900+0000","last_change":"2024-06-10T02:49:58.996616+0000","last_active":"2024-06-10T02:51:08.626900+0000","last_peered":"2024-06-10T02:51:08.626900+0000","last_clean":"2024-06-10T02:51:08.626900+0000","last_became_active":"2024-06-10T02:49:58.996289+0000","last_became_peered":"2024-06-10T02:49:58.996289+0000","last_unstale":"2024-06-10T02:51:08.626900+0000","last_undegraded":"2024-06-10T02:51:08.626900+0000","last_fullsized":"2024-06-10T02:51:08.626900+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:22:12.993608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.736962+0000","last_change":"2024-06-10T02:50:02.205230+0000","last_active":"2024-06-10T02:50:44.736962+0000","last_peered":"2024-06-10T02:50:44.736962+0000","last_clean":"2024-06-10T02:50:44.736962+0000","last_became_active":"2024-06-10T02:50:02.204835+0000","last_became_peered":"2024-06-10T02:50:02.204835+0000","last_unstale":"2024-06-10T02:50:44.736962+0000","last_undegraded":"2024-06-10T02:50:44.736962+0000","last_fullsized":"2024-06-10T02:50:44.736962+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:17:35.084921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.780286+0000","last_change":"2024-06-10T02:50:12.231684+0000","last_active":"2024-06-10T02:50:44.780286+0000","last_peered":"2024-06-10T02:50:44.780286+0000","last_clean":"2024-06-10T02:50:44.780286+0000","last_became_active":"2024-06-10T02:50:12.231256+0000","last_became_peered":"2024-06-10T02:50:12.231256+0000","last_unstale":"2024-06-10T02:50:44.780286+0000","last_undegraded":"2024-06-10T02:50:44.780286+0000","last_fullsized":"2024-06-10T02:50:44.780286+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:17:31.878289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"98'5","reported_seq":46,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:51:07.616305+0000","last_change":"2024-06-10T02:49:59.033950+0000","last_active":"2024-06-10T02:51:07.616305+0000","last_peered":"2024-06-10T02:51:07.616305+0000","last_clean":"2024-06-10T02:51:07.616305+0000","last_became_active":"2024-06-10T02:49:59.033688+0000","last_became_peered":"2024-06-10T02:49:59.033688+0000","last_unstale":"2024-06-10T02:51:07.616305+0000","last_undegraded":"2024-06-10T02:51:07.616305+0000","last_fullsized":"2024-06-10T02:51:07.616305+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:32:03.892091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":40,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.781378+0000","last_change":"2024-06-10T02:50:44.781378+0000","last_active":"2024-06-10T02:50:44.781378+0000","last_peered":"2024-06-10T02:50:44.781378+0000","last_clean":"2024-06-10T02:50:44.781378+0000","last_became_active":"2024-06-10T02:50:44.780955+0000","last_became_peered":"2024-06-10T02:50:44.780955+0000","last_unstale":"2024-06-10T02:50:44.781378+0000","last_undegraded":"2024-06-10T02:50:44.781378+0000","last_fullsized":"2024-06-10T02:50:44.781378+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:49:13.760381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.738451+0000","last_change":"2024-06-10T02:50:12.239790+0000","last_active":"2024-06-10T02:50:44.738451+0000","last_peered":"2024-06-10T02:50:44.738451+0000","last_clean":"2024-06-10T02:50:44.738451+0000","last_became_active":"2024-06-10T02:50:12.238176+0000","last_became_peered":"2024-06-10T02:50:12.238176+0000","last_unstale":"2024-06-10T02:50:44.738451+0000","last_undegraded":"2024-06-10T02:50:44.738451+0000","last_fullsized":"2024-06-10T02:50:44.738451+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:03:47.705367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"98'33","reported_seq":127,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:51:31.678426+0000","last_change":"2024-06-10T02:49:58.997890+0000","last_active":"2024-06-10T02:51:31.678426+0000","last_peered":"2024-06-10T02:51:31.678426+0000","last_clean":"2024-06-10T02:51:31.678426+0000","last_became_active":"2024-06-10T02:49:58.997549+0000","last_became_peered":"2024-06-10T02:49:58.997549+0000","last_unstale":"2024-06-10T02:51:31.678426+0000","last_undegraded":"2024-06-10T02:51:31.678426+0000","last_fullsized":"2024-06-10T02:51:31.678426+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T14:40:50.907029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58390,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":54,"num_read_kb":84,"num_write":35,"num_write_kb":71,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.740133+0000","last_change":"2024-06-10T02:50:02.199707+0000","last_active":"2024-06-10T02:50:44.740133+0000","last_peered":"2024-06-10T02:50:44.740133+0000","last_clean":"2024-06-10T02:50:44.740133+0000","last_became_active":"2024-06-10T02:50:02.198987+0000","last_became_peered":"2024-06-10T02:50:02.198987+0000","last_unstale":"2024-06-10T02:50:44.740133+0000","last_undegraded":"2024-06-10T02:50:44.740133+0000","last_fullsized":"2024-06-10T02:50:44.740133+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:48:03.704590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.742929+0000","last_change":"2024-06-10T02:50:12.239608+0000","last_active":"2024-06-10T02:50:44.742929+0000","last_peered":"2024-06-10T02:50:44.742929+0000","last_clean":"2024-06-10T02:50:44.742929+0000","last_became_active":"2024-06-10T02:50:12.238588+0000","last_became_peered":"2024-06-10T02:50:12.238588+0000","last_unstale":"2024-06-10T02:50:44.742929+0000","last_undegraded":"2024-06-10T02:50:44.742929+0000","last_fullsized":"2024-06-10T02:50:44.742929+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:26:12.718921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"98'9","reported_seq":51,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:51:09.633206+0000","last_change":"2024-06-10T02:49:58.997193+0000","last_active":"2024-06-10T02:51:09.633206+0000","last_peered":"2024-06-10T02:51:09.633206+0000","last_clean":"2024-06-10T02:51:09.633206+0000","last_became_active":"2024-06-10T02:49:58.996712+0000","last_became_peered":"2024-06-10T02:49:58.996712+0000","last_unstale":"2024-06-10T02:51:09.633206+0000","last_undegraded":"2024-06-10T02:51:09.633206+0000","last_fullsized":"2024-06-10T02:51:09.633206+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:02:46.894489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.779923+0000","last_change":"2024-06-10T02:50:02.200544+0000","last_active":"2024-06-10T02:50:44.779923+0000","last_peered":"2024-06-10T02:50:44.779923+0000","last_clean":"2024-06-10T02:50:44.779923+0000","last_became_active":"2024-06-10T02:50:02.200201+0000","last_became_peered":"2024-06-10T02:50:02.200201+0000","last_unstale":"2024-06-10T02:50:44.779923+0000","last_undegraded":"2024-06-10T02:50:44.779923+0000","last_fullsized":"2024-06-10T02:50:44.779923+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T13:12:40.925448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.780172+0000","last_change":"2024-06-10T02:50:12.236317+0000","last_active":"2024-06-10T02:50:44.780172+0000","last_peered":"2024-06-10T02:50:44.780172+0000","last_clean":"2024-06-10T02:50:44.780172+0000","last_became_active":"2024-06-10T02:50:12.235806+0000","last_became_peered":"2024-06-10T02:50:12.235806+0000","last_unstale":"2024-06-10T02:50:44.780172+0000","last_undegraded":"2024-06-10T02:50:44.780172+0000","last_fullsized":"2024-06-10T02:50:44.780172+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:29:21.425987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"98'5","reported_seq":46,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:51:08.628686+0000","last_change":"2024-06-10T02:49:58.994149+0000","last_active":"2024-06-10T02:51:08.628686+0000","last_peered":"2024-06-10T02:51:08.628686+0000","last_clean":"2024-06-10T02:51:08.628686+0000","last_became_active":"2024-06-10T02:49:58.993775+0000","last_became_peered":"2024-06-10T02:49:58.993775+0000","last_unstale":"2024-06-10T02:51:08.628686+0000","last_undegraded":"2024-06-10T02:51:08.628686+0000","last_fullsized":"2024-06-10T02:51:08.628686+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:23:55.553943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.735604+0000","last_change":"2024-06-10T02:50:02.199979+0000","last_active":"2024-06-10T02:50:44.735604+0000","last_peered":"2024-06-10T02:50:44.735604+0000","last_clean":"2024-06-10T02:50:44.735604+0000","last_became_active":"2024-06-10T02:50:02.199596+0000","last_became_peered":"2024-06-10T02:50:02.199596+0000","last_unstale":"2024-06-10T02:50:44.735604+0000","last_undegraded":"2024-06-10T02:50:44.735604+0000","last_fullsized":"2024-06-10T02:50:44.735604+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:13:58.647460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.738672+0000","last_change":"2024-06-10T02:50:12.235253+0000","last_active":"2024-06-10T02:50:44.738672+0000","last_peered":"2024-06-10T02:50:44.738672+0000","last_clean":"2024-06-10T02:50:44.738672+0000","last_became_active":"2024-06-10T02:50:12.234773+0000","last_became_peered":"2024-06-10T02:50:12.234773+0000","last_unstale":"2024-06-10T02:50:44.738672+0000","last_undegraded":"2024-06-10T02:50:44.738672+0000","last_fullsized":"2024-06-10T02:50:44.738672+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:38:10.971411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"98'6","reported_seq":71,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:51:29.664242+0000","last_change":"2024-06-10T02:49:58.996912+0000","last_active":"2024-06-10T02:51:29.664242+0000","last_peered":"2024-06-10T02:51:29.664242+0000","last_clean":"2024-06-10T02:51:29.664242+0000","last_became_active":"2024-06-10T02:49:58.996549+0000","last_became_peered":"2024-06-10T02:49:58.996549+0000","last_unstale":"2024-06-10T02:51:29.664242+0000","last_undegraded":"2024-06-10T02:51:29.664242+0000","last_fullsized":"2024-06-10T02:51:29.664242+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:07:52.757676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":978,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":26,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":40,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.741186+0000","last_change":"2024-06-10T02:50:44.741186+0000","last_active":"2024-06-10T02:50:44.741186+0000","last_peered":"2024-06-10T02:50:44.741186+0000","last_clean":"2024-06-10T02:50:44.741186+0000","last_became_active":"2024-06-10T02:50:44.740782+0000","last_became_peered":"2024-06-10T02:50:44.740782+0000","last_unstale":"2024-06-10T02:50:44.741186+0000","last_undegraded":"2024-06-10T02:50:44.741186+0000","last_fullsized":"2024-06-10T02:50:44.741186+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:46:03.863243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.780292+0000","last_change":"2024-06-10T02:50:12.236505+0000","last_active":"2024-06-10T02:50:44.780292+0000","last_peered":"2024-06-10T02:50:44.780292+0000","last_clean":"2024-06-10T02:50:44.780292+0000","last_became_active":"2024-06-10T02:50:12.236112+0000","last_became_peered":"2024-06-10T02:50:12.236112+0000","last_unstale":"2024-06-10T02:50:44.780292+0000","last_undegraded":"2024-06-10T02:50:44.780292+0000","last_fullsized":"2024-06-10T02:50:44.780292+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:13:38.586736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"98'17","reported_seq":58,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:51:08.627154+0000","last_change":"2024-06-10T02:49:59.031965+0000","last_active":"2024-06-10T02:51:08.627154+0000","last_peered":"2024-06-10T02:51:08.627154+0000","last_clean":"2024-06-10T02:51:08.627154+0000","last_became_active":"2024-06-10T02:49:59.031495+0000","last_became_peered":"2024-06-10T02:49:59.031495+0000","last_unstale":"2024-06-10T02:51:08.627154+0000","last_undegraded":"2024-06-10T02:51:08.627154+0000","last_fullsized":"2024-06-10T02:51:08.627154+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T14:21:06.610677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.743175+0000","last_change":"2024-06-10T02:50:02.209926+0000","last_active":"2024-06-10T02:50:44.743175+0000","last_peered":"2024-06-10T02:50:44.743175+0000","last_clean":"2024-06-10T02:50:44.743175+0000","last_became_active":"2024-06-10T02:50:02.209540+0000","last_became_peered":"2024-06-10T02:50:02.209540+0000","last_unstale":"2024-06-10T02:50:44.743175+0000","last_undegraded":"2024-06-10T02:50:44.743175+0000","last_fullsized":"2024-06-10T02:50:44.743175+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:49:40.497692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.743114+0000","last_change":"2024-06-10T02:50:12.239236+0000","last_active":"2024-06-10T02:50:44.743114+0000","last_peered":"2024-06-10T02:50:44.743114+0000","last_clean":"2024-06-10T02:50:44.743114+0000","last_became_active":"2024-06-10T02:50:12.238623+0000","last_became_peered":"2024-06-10T02:50:12.238623+0000","last_unstale":"2024-06-10T02:50:44.743114+0000","last_undegraded":"2024-06-10T02:50:44.743114+0000","last_fullsized":"2024-06-10T02:50:44.743114+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:10:13.651629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"98'2","reported_seq":43,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:51:07.610072+0000","last_change":"2024-06-10T02:49:58.998372+0000","last_active":"2024-06-10T02:51:07.610072+0000","last_peered":"2024-06-10T02:51:07.610072+0000","last_clean":"2024-06-10T02:51:07.610072+0000","last_became_active":"2024-06-10T02:49:58.998058+0000","last_became_peered":"2024-06-10T02:49:58.998058+0000","last_unstale":"2024-06-10T02:51:07.610072+0000","last_undegraded":"2024-06-10T02:51:07.610072+0000","last_fullsized":"2024-06-10T02:51:07.610072+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:42:50.509851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.780381+0000","last_change":"2024-06-10T02:50:02.209256+0000","last_active":"2024-06-10T02:50:44.780381+0000","last_peered":"2024-06-10T02:50:44.780381+0000","last_clean":"2024-06-10T02:50:44.780381+0000","last_became_active":"2024-06-10T02:50:02.207727+0000","last_became_peered":"2024-06-10T02:50:02.207727+0000","last_unstale":"2024-06-10T02:50:44.780381+0000","last_undegraded":"2024-06-10T02:50:44.780381+0000","last_fullsized":"2024-06-10T02:50:44.780381+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:23:00.175943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.739489+0000","last_change":"2024-06-10T02:50:12.729438+0000","last_active":"2024-06-10T02:50:44.739489+0000","last_peered":"2024-06-10T02:50:44.739489+0000","last_clean":"2024-06-10T02:50:44.739489+0000","last_became_active":"2024-06-10T02:50:12.728918+0000","last_became_peered":"2024-06-10T02:50:12.728918+0000","last_unstale":"2024-06-10T02:50:44.739489+0000","last_undegraded":"2024-06-10T02:50:44.739489+0000","last_fullsized":"2024-06-10T02:50:44.739489+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:04:48.861763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"98'1","reported_seq":47,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.782798+0000","last_change":"2024-06-10T02:50:44.782241+0000","last_active":"2024-06-10T02:50:44.782798+0000","last_peered":"2024-06-10T02:50:44.782798+0000","last_clean":"2024-06-10T02:50:44.782798+0000","last_became_active":"2024-06-10T02:50:44.781799+0000","last_became_peered":"2024-06-10T02:50:44.781799+0000","last_unstale":"2024-06-10T02:50:44.782798+0000","last_undegraded":"2024-06-10T02:50:44.782798+0000","last_fullsized":"2024-06-10T02:50:44.782798+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:38:35.726258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.739478+0000","last_change":"2024-06-10T02:50:02.198888+0000","last_active":"2024-06-10T02:50:44.739478+0000","last_peered":"2024-06-10T02:50:44.739478+0000","last_clean":"2024-06-10T02:50:44.739478+0000","last_became_active":"2024-06-10T02:50:02.198346+0000","last_became_peered":"2024-06-10T02:50:02.198346+0000","last_unstale":"2024-06-10T02:50:44.739478+0000","last_undegraded":"2024-06-10T02:50:44.739478+0000","last_fullsized":"2024-06-10T02:50:44.739478+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:15:42.422898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.780257+0000","last_change":"2024-06-10T02:50:12.232763+0000","last_active":"2024-06-10T02:50:44.780257+0000","last_peered":"2024-06-10T02:50:44.780257+0000","last_clean":"2024-06-10T02:50:44.780257+0000","last_became_active":"2024-06-10T02:50:12.232390+0000","last_became_peered":"2024-06-10T02:50:12.232390+0000","last_unstale":"2024-06-10T02:50:44.780257+0000","last_undegraded":"2024-06-10T02:50:44.780257+0000","last_fullsized":"2024-06-10T02:50:44.780257+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:31:47.973500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"98'6","reported_seq":72,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:51:31.666279+0000","last_change":"2024-06-10T02:49:58.994308+0000","last_active":"2024-06-10T02:51:31.666279+0000","last_peered":"2024-06-10T02:51:31.666279+0000","last_clean":"2024-06-10T02:51:31.666279+0000","last_became_active":"2024-06-10T02:49:58.993494+0000","last_became_peered":"2024-06-10T02:49:58.993494+0000","last_unstale":"2024-06-10T02:51:31.666279+0000","last_undegraded":"2024-06-10T02:51:31.666279+0000","last_fullsized":"2024-06-10T02:51:31.666279+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:07:28.452276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":978,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":26,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.739405+0000","last_change":"2024-06-10T02:50:02.206872+0000","last_active":"2024-06-10T02:50:44.739405+0000","last_peered":"2024-06-10T02:50:44.739405+0000","last_clean":"2024-06-10T02:50:44.739405+0000","last_became_active":"2024-06-10T02:50:02.206307+0000","last_became_peered":"2024-06-10T02:50:02.206307+0000","last_unstale":"2024-06-10T02:50:44.739405+0000","last_undegraded":"2024-06-10T02:50:44.739405+0000","last_fullsized":"2024-06-10T02:50:44.739405+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:01:04.645674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.780303+0000","last_change":"2024-06-10T02:50:12.729392+0000","last_active":"2024-06-10T02:50:44.780303+0000","last_peered":"2024-06-10T02:50:44.780303+0000","last_clean":"2024-06-10T02:50:44.780303+0000","last_became_active":"2024-06-10T02:50:12.728866+0000","last_became_peered":"2024-06-10T02:50:12.728866+0000","last_unstale":"2024-06-10T02:50:44.780303+0000","last_undegraded":"2024-06-10T02:50:44.780303+0000","last_fullsized":"2024-06-10T02:50:44.780303+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:01:35.845184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.741921+0000","last_change":"2024-06-10T02:50:02.202303+0000","last_active":"2024-06-10T02:50:44.741921+0000","last_peered":"2024-06-10T02:50:44.741921+0000","last_clean":"2024-06-10T02:50:44.741921+0000","last_became_active":"2024-06-10T02:50:02.201998+0000","last_became_peered":"2024-06-10T02:50:02.201998+0000","last_unstale":"2024-06-10T02:50:44.741921+0000","last_undegraded":"2024-06-10T02:50:44.741921+0000","last_fullsized":"2024-06-10T02:50:44.741921+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:15:00.005693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"98'5","reported_seq":70,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:51:31.666116+0000","last_change":"2024-06-10T02:49:59.031513+0000","last_active":"2024-06-10T02:51:31.666116+0000","last_peered":"2024-06-10T02:51:31.666116+0000","last_clean":"2024-06-10T02:51:31.666116+0000","last_became_active":"2024-06-10T02:49:59.030965+0000","last_became_peered":"2024-06-10T02:49:59.030965+0000","last_unstale":"2024-06-10T02:51:31.666116+0000","last_undegraded":"2024-06-10T02:51:31.666116+0000","last_fullsized":"2024-06-10T02:51:31.666116+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:49:02.171578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":25,"num_read_kb":25,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"98'1","reported_seq":25,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.735962+0000","last_change":"2024-06-10T02:50:12.236334+0000","last_active":"2024-06-10T02:50:44.735962+0000","last_peered":"2024-06-10T02:50:44.735962+0000","last_clean":"2024-06-10T02:50:44.735962+0000","last_became_active":"2024-06-10T02:50:12.235820+0000","last_became_peered":"2024-06-10T02:50:12.235820+0000","last_unstale":"2024-06-10T02:50:44.735962+0000","last_undegraded":"2024-06-10T02:50:44.735962+0000","last_fullsized":"2024-06-10T02:50:44.735962+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:16:12.476881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.733920+0000","last_change":"2024-06-10T02:50:02.205146+0000","last_active":"2024-06-10T02:50:44.733920+0000","last_peered":"2024-06-10T02:50:44.733920+0000","last_clean":"2024-06-10T02:50:44.733920+0000","last_became_active":"2024-06-10T02:50:02.204785+0000","last_became_peered":"2024-06-10T02:50:02.204785+0000","last_unstale":"2024-06-10T02:50:44.733920+0000","last_undegraded":"2024-06-10T02:50:44.733920+0000","last_fullsized":"2024-06-10T02:50:44.733920+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T13:01:02.068989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"98'6","reported_seq":49,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:51:09.633782+0000","last_change":"2024-06-10T02:49:58.997654+0000","last_active":"2024-06-10T02:51:09.633782+0000","last_peered":"2024-06-10T02:51:09.633782+0000","last_clean":"2024-06-10T02:51:09.633782+0000","last_became_active":"2024-06-10T02:49:58.997294+0000","last_became_peered":"2024-06-10T02:49:58.997294+0000","last_unstale":"2024-06-10T02:51:09.633782+0000","last_undegraded":"2024-06-10T02:51:09.633782+0000","last_fullsized":"2024-06-10T02:51:09.633782+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:53:14.763198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":362,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.735090+0000","last_change":"2024-06-10T02:48:34.008201+0000","last_active":"2024-06-10T02:50:44.735090+0000","last_peered":"2024-06-10T02:50:44.735090+0000","last_clean":"2024-06-10T02:50:44.735090+0000","last_became_active":"2024-06-10T02:48:34.007633+0000","last_became_peered":"2024-06-10T02:48:34.007633+0000","last_unstale":"2024-06-10T02:50:44.735090+0000","last_undegraded":"2024-06-10T02:50:44.735090+0000","last_fullsized":"2024-06-10T02:50:44.735090+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:45:07.444823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:45:07.444823+0000","last_clean_scrub_stamp":"2024-06-10T02:45:07.444823+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:56:58.159559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":31,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.782130+0000","last_change":"2024-06-10T02:50:44.782130+0000","last_active":"2024-06-10T02:50:44.782130+0000","last_peered":"2024-06-10T02:50:44.782130+0000","last_clean":"2024-06-10T02:50:44.782130+0000","last_became_active":"2024-06-10T02:50:44.781530+0000","last_became_peered":"2024-06-10T02:50:44.781530+0000","last_unstale":"2024-06-10T02:50:44.782130+0000","last_undegraded":"2024-06-10T02:50:44.782130+0000","last_fullsized":"2024-06-10T02:50:44.782130+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:43:23.826207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,11,6,10],"acting":[5,11,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.779624+0000","last_change":"2024-06-10T02:50:02.213666+0000","last_active":"2024-06-10T02:50:44.779624+0000","last_peered":"2024-06-10T02:50:44.779624+0000","last_clean":"2024-06-10T02:50:44.779624+0000","last_became_active":"2024-06-10T02:50:02.213194+0000","last_became_peered":"2024-06-10T02:50:02.213194+0000","last_unstale":"2024-06-10T02:50:44.779624+0000","last_undegraded":"2024-06-10T02:50:44.779624+0000","last_fullsized":"2024-06-10T02:50:44.779624+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:22:55.905412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"98'4","reported_seq":44,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:51:06.614416+0000","last_change":"2024-06-10T02:49:58.997618+0000","last_active":"2024-06-10T02:51:06.614416+0000","last_peered":"2024-06-10T02:51:06.614416+0000","last_clean":"2024-06-10T02:51:06.614416+0000","last_became_active":"2024-06-10T02:49:58.997304+0000","last_became_peered":"2024-06-10T02:49:58.997304+0000","last_unstale":"2024-06-10T02:51:06.614416+0000","last_undegraded":"2024-06-10T02:51:06.614416+0000","last_fullsized":"2024-06-10T02:51:06.614416+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:48:59.520991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.780121+0000","last_change":"2024-06-10T02:50:12.236357+0000","last_active":"2024-06-10T02:50:44.780121+0000","last_peered":"2024-06-10T02:50:44.780121+0000","last_clean":"2024-06-10T02:50:44.780121+0000","last_became_active":"2024-06-10T02:50:12.235915+0000","last_became_peered":"2024-06-10T02:50:12.235915+0000","last_unstale":"2024-06-10T02:50:44.780121+0000","last_undegraded":"2024-06-10T02:50:44.780121+0000","last_fullsized":"2024-06-10T02:50:44.780121+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:20:06.287907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"98'4","reported_seq":45,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:51:08.627507+0000","last_change":"2024-06-10T02:49:59.031725+0000","last_active":"2024-06-10T02:51:08.627507+0000","last_peered":"2024-06-10T02:51:08.627507+0000","last_clean":"2024-06-10T02:51:08.627507+0000","last_became_active":"2024-06-10T02:49:59.031296+0000","last_became_peered":"2024-06-10T02:49:59.031296+0000","last_unstale":"2024-06-10T02:51:08.627507+0000","last_undegraded":"2024-06-10T02:51:08.627507+0000","last_fullsized":"2024-06-10T02:51:08.627507+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T13:17:58.571696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.779944+0000","last_change":"2024-06-10T02:50:02.201820+0000","last_active":"2024-06-10T02:50:44.779944+0000","last_peered":"2024-06-10T02:50:44.779944+0000","last_clean":"2024-06-10T02:50:44.779944+0000","last_became_active":"2024-06-10T02:50:02.201478+0000","last_became_peered":"2024-06-10T02:50:02.201478+0000","last_unstale":"2024-06-10T02:50:44.779944+0000","last_undegraded":"2024-06-10T02:50:44.779944+0000","last_fullsized":"2024-06-10T02:50:44.779944+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:57:25.963926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.733729+0000","last_change":"2024-06-10T02:50:12.729418+0000","last_active":"2024-06-10T02:50:44.733729+0000","last_peered":"2024-06-10T02:50:44.733729+0000","last_clean":"2024-06-10T02:50:44.733729+0000","last_became_active":"2024-06-10T02:50:12.728692+0000","last_became_peered":"2024-06-10T02:50:12.728692+0000","last_unstale":"2024-06-10T02:50:44.733729+0000","last_undegraded":"2024-06-10T02:50:44.733729+0000","last_fullsized":"2024-06-10T02:50:44.733729+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:25:00.472396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"98'14","reported_seq":54,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:51:26.212094+0000","last_change":"2024-06-10T02:49:58.996391+0000","last_active":"2024-06-10T02:51:26.212094+0000","last_peered":"2024-06-10T02:51:26.212094+0000","last_clean":"2024-06-10T02:51:26.212094+0000","last_became_active":"2024-06-10T02:49:58.996033+0000","last_became_peered":"2024-06-10T02:49:58.996033+0000","last_unstale":"2024-06-10T02:51:26.212094+0000","last_undegraded":"2024-06-10T02:51:26.212094+0000","last_fullsized":"2024-06-10T02:51:26.212094+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:03:34.568969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.734506+0000","last_change":"2024-06-10T02:50:02.197974+0000","last_active":"2024-06-10T02:50:44.734506+0000","last_peered":"2024-06-10T02:50:44.734506+0000","last_clean":"2024-06-10T02:50:44.734506+0000","last_became_active":"2024-06-10T02:50:02.197614+0000","last_became_peered":"2024-06-10T02:50:02.197614+0000","last_unstale":"2024-06-10T02:50:44.734506+0000","last_undegraded":"2024-06-10T02:50:44.734506+0000","last_fullsized":"2024-06-10T02:50:44.734506+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:36:55.575808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.737015+0000","last_change":"2024-06-10T02:50:12.234239+0000","last_active":"2024-06-10T02:50:44.737015+0000","last_peered":"2024-06-10T02:50:44.737015+0000","last_clean":"2024-06-10T02:50:44.737015+0000","last_became_active":"2024-06-10T02:50:12.233768+0000","last_became_peered":"2024-06-10T02:50:12.233768+0000","last_unstale":"2024-06-10T02:50:44.737015+0000","last_undegraded":"2024-06-10T02:50:44.737015+0000","last_fullsized":"2024-06-10T02:50:44.737015+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:51:44.230031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"98'7","reported_seq":48,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:51:08.628668+0000","last_change":"2024-06-10T02:49:59.192765+0000","last_active":"2024-06-10T02:51:08.628668+0000","last_peered":"2024-06-10T02:51:08.628668+0000","last_clean":"2024-06-10T02:51:08.628668+0000","last_became_active":"2024-06-10T02:49:59.192446+0000","last_became_peered":"2024-06-10T02:49:59.192446+0000","last_unstale":"2024-06-10T02:51:08.628668+0000","last_undegraded":"2024-06-10T02:51:08.628668+0000","last_fullsized":"2024-06-10T02:51:08.628668+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:53:51.862019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":40,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.781209+0000","last_change":"2024-06-10T02:50:44.781209+0000","last_active":"2024-06-10T02:50:44.781209+0000","last_peered":"2024-06-10T02:50:44.781209+0000","last_clean":"2024-06-10T02:50:44.781209+0000","last_became_active":"2024-06-10T02:50:44.780811+0000","last_became_peered":"2024-06-10T02:50:44.780811+0000","last_unstale":"2024-06-10T02:50:44.781209+0000","last_undegraded":"2024-06-10T02:50:44.781209+0000","last_fullsized":"2024-06-10T02:50:44.781209+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:12:40.663565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.780246+0000","last_change":"2024-06-10T02:50:12.730269+0000","last_active":"2024-06-10T02:50:44.780246+0000","last_peered":"2024-06-10T02:50:44.780246+0000","last_clean":"2024-06-10T02:50:44.780246+0000","last_became_active":"2024-06-10T02:50:12.729574+0000","last_became_peered":"2024-06-10T02:50:12.729574+0000","last_unstale":"2024-06-10T02:50:44.780246+0000","last_undegraded":"2024-06-10T02:50:44.780246+0000","last_fullsized":"2024-06-10T02:50:44.780246+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:51:16.235284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"98'2","reported_seq":48,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:51:05.606777+0000","last_change":"2024-06-10T02:50:44.782421+0000","last_active":"2024-06-10T02:51:05.606777+0000","last_peered":"2024-06-10T02:51:05.606777+0000","last_clean":"2024-06-10T02:51:05.606777+0000","last_became_active":"2024-06-10T02:50:44.781881+0000","last_became_peered":"2024-06-10T02:50:44.781881+0000","last_unstale":"2024-06-10T02:51:05.606777+0000","last_undegraded":"2024-06-10T02:51:05.606777+0000","last_fullsized":"2024-06-10T02:51:05.606777+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T14:41:58.771256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.739369+0000","last_change":"2024-06-10T02:50:02.197761+0000","last_active":"2024-06-10T02:50:44.739369+0000","last_peered":"2024-06-10T02:50:44.739369+0000","last_clean":"2024-06-10T02:50:44.739369+0000","last_became_active":"2024-06-10T02:50:02.197395+0000","last_became_peered":"2024-06-10T02:50:02.197395+0000","last_unstale":"2024-06-10T02:50:44.739369+0000","last_undegraded":"2024-06-10T02:50:44.739369+0000","last_fullsized":"2024-06-10T02:50:44.739369+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:46:37.537280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.738803+0000","last_change":"2024-06-10T02:50:12.241506+0000","last_active":"2024-06-10T02:50:44.738803+0000","last_peered":"2024-06-10T02:50:44.738803+0000","last_clean":"2024-06-10T02:50:44.738803+0000","last_became_active":"2024-06-10T02:50:12.240325+0000","last_became_peered":"2024-06-10T02:50:12.240325+0000","last_unstale":"2024-06-10T02:50:44.738803+0000","last_undegraded":"2024-06-10T02:50:44.738803+0000","last_fullsized":"2024-06-10T02:50:44.738803+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T13:45:41.869418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"98'7","reported_seq":47,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:51:08.627173+0000","last_change":"2024-06-10T02:49:58.987863+0000","last_active":"2024-06-10T02:51:08.627173+0000","last_peered":"2024-06-10T02:51:08.627173+0000","last_clean":"2024-06-10T02:51:08.627173+0000","last_became_active":"2024-06-10T02:49:58.987388+0000","last_became_peered":"2024-06-10T02:49:58.987388+0000","last_unstale":"2024-06-10T02:51:08.627173+0000","last_undegraded":"2024-06-10T02:51:08.627173+0000","last_fullsized":"2024-06-10T02:51:08.627173+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:29:20.340301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.780386+0000","last_change":"2024-06-10T02:50:02.200699+0000","last_active":"2024-06-10T02:50:44.780386+0000","last_peered":"2024-06-10T02:50:44.780386+0000","last_clean":"2024-06-10T02:50:44.780386+0000","last_became_active":"2024-06-10T02:50:02.200278+0000","last_became_peered":"2024-06-10T02:50:02.200278+0000","last_unstale":"2024-06-10T02:50:44.780386+0000","last_undegraded":"2024-06-10T02:50:44.780386+0000","last_fullsized":"2024-06-10T02:50:44.780386+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:26:36.236312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"98'1","reported_seq":25,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.780107+0000","last_change":"2024-06-10T02:50:12.236027+0000","last_active":"2024-06-10T02:50:44.780107+0000","last_peered":"2024-06-10T02:50:44.780107+0000","last_clean":"2024-06-10T02:50:44.780107+0000","last_became_active":"2024-06-10T02:50:12.235629+0000","last_became_peered":"2024-06-10T02:50:12.235629+0000","last_unstale":"2024-06-10T02:50:44.780107+0000","last_undegraded":"2024-06-10T02:50:44.780107+0000","last_fullsized":"2024-06-10T02:50:44.780107+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:58:48.218806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"98'3","reported_seq":43,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:51:07.620025+0000","last_change":"2024-06-10T02:49:58.998496+0000","last_active":"2024-06-10T02:51:07.620025+0000","last_peered":"2024-06-10T02:51:07.620025+0000","last_clean":"2024-06-10T02:51:07.620025+0000","last_became_active":"2024-06-10T02:49:58.997260+0000","last_became_peered":"2024-06-10T02:49:58.997260+0000","last_unstale":"2024-06-10T02:51:07.620025+0000","last_undegraded":"2024-06-10T02:51:07.620025+0000","last_fullsized":"2024-06-10T02:51:07.620025+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T14:22:04.658943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.734285+0000","last_change":"2024-06-10T02:50:02.201300+0000","last_active":"2024-06-10T02:50:44.734285+0000","last_peered":"2024-06-10T02:50:44.734285+0000","last_clean":"2024-06-10T02:50:44.734285+0000","last_became_active":"2024-06-10T02:50:02.200988+0000","last_became_peered":"2024-06-10T02:50:02.200988+0000","last_unstale":"2024-06-10T02:50:44.734285+0000","last_undegraded":"2024-06-10T02:50:44.734285+0000","last_fullsized":"2024-06-10T02:50:44.734285+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:11:12.962048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.780080+0000","last_change":"2024-06-10T02:50:12.728869+0000","last_active":"2024-06-10T02:50:44.780080+0000","last_peered":"2024-06-10T02:50:44.780080+0000","last_clean":"2024-06-10T02:50:44.780080+0000","last_became_active":"2024-06-10T02:50:12.728189+0000","last_became_peered":"2024-06-10T02:50:12.728189+0000","last_unstale":"2024-06-10T02:50:44.780080+0000","last_undegraded":"2024-06-10T02:50:44.780080+0000","last_fullsized":"2024-06-10T02:50:44.780080+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:08:36.624814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"98'4","reported_seq":45,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:51:05.605975+0000","last_change":"2024-06-10T02:49:59.000460+0000","last_active":"2024-06-10T02:51:05.605975+0000","last_peered":"2024-06-10T02:51:05.605975+0000","last_clean":"2024-06-10T02:51:05.605975+0000","last_became_active":"2024-06-10T02:49:59.000075+0000","last_became_peered":"2024-06-10T02:49:59.000075+0000","last_unstale":"2024-06-10T02:51:05.605975+0000","last_undegraded":"2024-06-10T02:51:05.605975+0000","last_fullsized":"2024-06-10T02:51:05.605975+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:06:27.935745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.735511+0000","last_change":"2024-06-10T02:50:02.210131+0000","last_active":"2024-06-10T02:50:44.735511+0000","last_peered":"2024-06-10T02:50:44.735511+0000","last_clean":"2024-06-10T02:50:44.735511+0000","last_became_active":"2024-06-10T02:50:02.209781+0000","last_became_peered":"2024-06-10T02:50:02.209781+0000","last_unstale":"2024-06-10T02:50:44.735511+0000","last_undegraded":"2024-06-10T02:50:44.735511+0000","last_fullsized":"2024-06-10T02:50:44.735511+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:27:39.966620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.738433+0000","last_change":"2024-06-10T02:50:12.236017+0000","last_active":"2024-06-10T02:50:44.738433+0000","last_peered":"2024-06-10T02:50:44.738433+0000","last_clean":"2024-06-10T02:50:44.738433+0000","last_became_active":"2024-06-10T02:50:12.235553+0000","last_became_peered":"2024-06-10T02:50:12.235553+0000","last_unstale":"2024-06-10T02:50:44.738433+0000","last_undegraded":"2024-06-10T02:50:44.738433+0000","last_fullsized":"2024-06-10T02:50:44.738433+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:23:55.437994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"98'6","reported_seq":71,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:51:29.664943+0000","last_change":"2024-06-10T02:49:59.192864+0000","last_active":"2024-06-10T02:51:29.664943+0000","last_peered":"2024-06-10T02:51:29.664943+0000","last_clean":"2024-06-10T02:51:29.664943+0000","last_became_active":"2024-06-10T02:49:59.192440+0000","last_became_peered":"2024-06-10T02:49:59.192440+0000","last_unstale":"2024-06-10T02:51:29.664943+0000","last_undegraded":"2024-06-10T02:51:29.664943+0000","last_fullsized":"2024-06-10T02:51:29.664943+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:36:05.868538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1560,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":25,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.736321+0000","last_change":"2024-06-10T02:50:02.205791+0000","last_active":"2024-06-10T02:50:44.736321+0000","last_peered":"2024-06-10T02:50:44.736321+0000","last_clean":"2024-06-10T02:50:44.736321+0000","last_became_active":"2024-06-10T02:50:02.205483+0000","last_became_peered":"2024-06-10T02:50:02.205483+0000","last_unstale":"2024-06-10T02:50:44.736321+0000","last_undegraded":"2024-06-10T02:50:44.736321+0000","last_fullsized":"2024-06-10T02:50:44.736321+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:33:25.875423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.736342+0000","last_change":"2024-06-10T02:50:12.239294+0000","last_active":"2024-06-10T02:50:44.736342+0000","last_peered":"2024-06-10T02:50:44.736342+0000","last_clean":"2024-06-10T02:50:44.736342+0000","last_became_active":"2024-06-10T02:50:12.238778+0000","last_became_peered":"2024-06-10T02:50:12.238778+0000","last_unstale":"2024-06-10T02:50:44.736342+0000","last_undegraded":"2024-06-10T02:50:44.736342+0000","last_fullsized":"2024-06-10T02:50:44.736342+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:26:09.203623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"98'3","reported_seq":44,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:51:08.627523+0000","last_change":"2024-06-10T02:49:58.989680+0000","last_active":"2024-06-10T02:51:08.627523+0000","last_peered":"2024-06-10T02:51:08.627523+0000","last_clean":"2024-06-10T02:51:08.627523+0000","last_became_active":"2024-06-10T02:49:58.989265+0000","last_became_peered":"2024-06-10T02:49:58.989265+0000","last_unstale":"2024-06-10T02:51:08.627523+0000","last_undegraded":"2024-06-10T02:51:08.627523+0000","last_fullsized":"2024-06-10T02:51:08.627523+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:36:34.370177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":40,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.743369+0000","last_change":"2024-06-10T02:50:44.743369+0000","last_active":"2024-06-10T02:50:44.743369+0000","last_peered":"2024-06-10T02:50:44.743369+0000","last_clean":"2024-06-10T02:50:44.743369+0000","last_became_active":"2024-06-10T02:50:44.743013+0000","last_became_peered":"2024-06-10T02:50:44.743013+0000","last_unstale":"2024-06-10T02:50:44.743369+0000","last_undegraded":"2024-06-10T02:50:44.743369+0000","last_fullsized":"2024-06-10T02:50:44.743369+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:54:43.728943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.740774+0000","last_change":"2024-06-10T02:50:12.235197+0000","last_active":"2024-06-10T02:50:44.740774+0000","last_peered":"2024-06-10T02:50:44.740774+0000","last_clean":"2024-06-10T02:50:44.740774+0000","last_became_active":"2024-06-10T02:50:12.234791+0000","last_became_peered":"2024-06-10T02:50:12.234791+0000","last_unstale":"2024-06-10T02:50:44.740774+0000","last_undegraded":"2024-06-10T02:50:44.740774+0000","last_fullsized":"2024-06-10T02:50:44.740774+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:09:11.164236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"98'5","reported_seq":45,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:51:08.625605+0000","last_change":"2024-06-10T02:49:58.996759+0000","last_active":"2024-06-10T02:51:08.625605+0000","last_peered":"2024-06-10T02:51:08.625605+0000","last_clean":"2024-06-10T02:51:08.625605+0000","last_became_active":"2024-06-10T02:49:58.996434+0000","last_became_peered":"2024-06-10T02:49:58.996434+0000","last_unstale":"2024-06-10T02:51:08.625605+0000","last_undegraded":"2024-06-10T02:51:08.625605+0000","last_fullsized":"2024-06-10T02:51:08.625605+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:09:35.327797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.736138+0000","last_change":"2024-06-10T02:50:02.201676+0000","last_active":"2024-06-10T02:50:44.736138+0000","last_peered":"2024-06-10T02:50:44.736138+0000","last_clean":"2024-06-10T02:50:44.736138+0000","last_became_active":"2024-06-10T02:50:02.201335+0000","last_became_peered":"2024-06-10T02:50:02.201335+0000","last_unstale":"2024-06-10T02:50:44.736138+0000","last_undegraded":"2024-06-10T02:50:44.736138+0000","last_fullsized":"2024-06-10T02:50:44.736138+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:49:47.038946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.780397+0000","last_change":"2024-06-10T02:50:02.203462+0000","last_active":"2024-06-10T02:50:44.780397+0000","last_peered":"2024-06-10T02:50:44.780397+0000","last_clean":"2024-06-10T02:50:44.780397+0000","last_became_active":"2024-06-10T02:50:02.203115+0000","last_became_peered":"2024-06-10T02:50:02.203115+0000","last_unstale":"2024-06-10T02:50:44.780397+0000","last_undegraded":"2024-06-10T02:50:44.780397+0000","last_fullsized":"2024-06-10T02:50:44.780397+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T13:18:12.400596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.737470+0000","last_change":"2024-06-10T02:50:02.205648+0000","last_active":"2024-06-10T02:50:44.737470+0000","last_peered":"2024-06-10T02:50:44.737470+0000","last_clean":"2024-06-10T02:50:44.737470+0000","last_became_active":"2024-06-10T02:50:02.205306+0000","last_became_peered":"2024-06-10T02:50:02.205306+0000","last_unstale":"2024-06-10T02:50:44.737470+0000","last_undegraded":"2024-06-10T02:50:44.737470+0000","last_fullsized":"2024-06-10T02:50:44.737470+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:22:55.020644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.737092+0000","last_change":"2024-06-10T02:50:02.198601+0000","last_active":"2024-06-10T02:50:44.737092+0000","last_peered":"2024-06-10T02:50:44.737092+0000","last_clean":"2024-06-10T02:50:44.737092+0000","last_became_active":"2024-06-10T02:50:02.198103+0000","last_became_peered":"2024-06-10T02:50:02.198103+0000","last_unstale":"2024-06-10T02:50:44.737092+0000","last_undegraded":"2024-06-10T02:50:44.737092+0000","last_fullsized":"2024-06-10T02:50:44.737092+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:56:28.033482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.736597+0000","last_change":"2024-06-10T02:50:02.202638+0000","last_active":"2024-06-10T02:50:44.736597+0000","last_peered":"2024-06-10T02:50:44.736597+0000","last_clean":"2024-06-10T02:50:44.736597+0000","last_became_active":"2024-06-10T02:50:02.202249+0000","last_became_peered":"2024-06-10T02:50:02.202249+0000","last_unstale":"2024-06-10T02:50:44.736597+0000","last_undegraded":"2024-06-10T02:50:44.736597+0000","last_fullsized":"2024-06-10T02:50:44.736597+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:46:29.424164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.779919+0000","last_change":"2024-06-10T02:50:02.213526+0000","last_active":"2024-06-10T02:50:44.779919+0000","last_peered":"2024-06-10T02:50:44.779919+0000","last_clean":"2024-06-10T02:50:44.779919+0000","last_became_active":"2024-06-10T02:50:02.213226+0000","last_became_peered":"2024-06-10T02:50:02.213226+0000","last_unstale":"2024-06-10T02:50:44.779919+0000","last_undegraded":"2024-06-10T02:50:44.779919+0000","last_fullsized":"2024-06-10T02:50:44.779919+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:01:56.846340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.780333+0000","last_change":"2024-06-10T02:50:02.206583+0000","last_active":"2024-06-10T02:50:44.780333+0000","last_peered":"2024-06-10T02:50:44.780333+0000","last_clean":"2024-06-10T02:50:44.780333+0000","last_became_active":"2024-06-10T02:50:02.206227+0000","last_became_peered":"2024-06-10T02:50:02.206227+0000","last_unstale":"2024-06-10T02:50:44.780333+0000","last_undegraded":"2024-06-10T02:50:44.780333+0000","last_fullsized":"2024-06-10T02:50:44.780333+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:25:07.246288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.737668+0000","last_change":"2024-06-10T02:50:02.206348+0000","last_active":"2024-06-10T02:50:44.737668+0000","last_peered":"2024-06-10T02:50:44.737668+0000","last_clean":"2024-06-10T02:50:44.737668+0000","last_became_active":"2024-06-10T02:50:02.206020+0000","last_became_peered":"2024-06-10T02:50:02.206020+0000","last_unstale":"2024-06-10T02:50:44.737668+0000","last_undegraded":"2024-06-10T02:50:44.737668+0000","last_fullsized":"2024-06-10T02:50:44.737668+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:02:26.619113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.734577+0000","last_change":"2024-06-10T02:50:02.199405+0000","last_active":"2024-06-10T02:50:44.734577+0000","last_peered":"2024-06-10T02:50:44.734577+0000","last_clean":"2024-06-10T02:50:44.734577+0000","last_became_active":"2024-06-10T02:50:02.199049+0000","last_became_peered":"2024-06-10T02:50:02.199049+0000","last_unstale":"2024-06-10T02:50:44.734577+0000","last_undegraded":"2024-06-10T02:50:44.734577+0000","last_fullsized":"2024-06-10T02:50:44.734577+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:33:04.091160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.741248+0000","last_change":"2024-06-10T02:50:02.202094+0000","last_active":"2024-06-10T02:50:44.741248+0000","last_peered":"2024-06-10T02:50:44.741248+0000","last_clean":"2024-06-10T02:50:44.741248+0000","last_became_active":"2024-06-10T02:50:02.201751+0000","last_became_peered":"2024-06-10T02:50:02.201751+0000","last_unstale":"2024-06-10T02:50:44.741248+0000","last_undegraded":"2024-06-10T02:50:44.741248+0000","last_fullsized":"2024-06-10T02:50:44.741248+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:52:53.946326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.734003+0000","last_change":"2024-06-10T02:50:02.199630+0000","last_active":"2024-06-10T02:50:44.734003+0000","last_peered":"2024-06-10T02:50:44.734003+0000","last_clean":"2024-06-10T02:50:44.734003+0000","last_became_active":"2024-06-10T02:50:02.199320+0000","last_became_peered":"2024-06-10T02:50:02.199320+0000","last_unstale":"2024-06-10T02:50:44.734003+0000","last_undegraded":"2024-06-10T02:50:44.734003+0000","last_fullsized":"2024-06-10T02:50:44.734003+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:17:28.761862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.738713+0000","last_change":"2024-06-10T02:50:02.201190+0000","last_active":"2024-06-10T02:50:44.738713+0000","last_peered":"2024-06-10T02:50:44.738713+0000","last_clean":"2024-06-10T02:50:44.738713+0000","last_became_active":"2024-06-10T02:50:02.200574+0000","last_became_peered":"2024-06-10T02:50:02.200574+0000","last_unstale":"2024-06-10T02:50:44.738713+0000","last_undegraded":"2024-06-10T02:50:44.738713+0000","last_fullsized":"2024-06-10T02:50:44.738713+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:02:07.636218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.780030+0000","last_change":"2024-06-10T02:50:02.202374+0000","last_active":"2024-06-10T02:50:44.780030+0000","last_peered":"2024-06-10T02:50:44.780030+0000","last_clean":"2024-06-10T02:50:44.780030+0000","last_became_active":"2024-06-10T02:50:02.201974+0000","last_became_peered":"2024-06-10T02:50:02.201974+0000","last_unstale":"2024-06-10T02:50:44.780030+0000","last_undegraded":"2024-06-10T02:50:44.780030+0000","last_fullsized":"2024-06-10T02:50:44.780030+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:45:34.977926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.780218+0000","last_change":"2024-06-10T02:50:02.209332+0000","last_active":"2024-06-10T02:50:44.780218+0000","last_peered":"2024-06-10T02:50:44.780218+0000","last_clean":"2024-06-10T02:50:44.780218+0000","last_became_active":"2024-06-10T02:50:02.208175+0000","last_became_peered":"2024-06-10T02:50:02.208175+0000","last_unstale":"2024-06-10T02:50:44.780218+0000","last_undegraded":"2024-06-10T02:50:44.780218+0000","last_fullsized":"2024-06-10T02:50:44.780218+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:44:40.857918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":284028,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66372,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":269,"num_read_kb":209,"num_write":248,"num_write_kb":140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":199116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":213,"ondisk_log_size":213,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351010,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32236,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712916,"statfs":{"total":95995035648,"available":95962025984,"internally_reserved":0,"allocated":1085440,"data_stored":555042,"data_compressed":26507,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612647,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32380,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712772,"statfs":{"total":95995035648,"available":95961878528,"internally_reserved":0,"allocated":1232896,"data_stored":1200248,"data_compressed":62624,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1599,"internal_metadata":31914433},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776172,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36424,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708728,"statfs":{"total":95995035648,"available":95957737472,"internally_reserved":0,"allocated":1114112,"data_stored":558678,"data_compressed":26507,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037808,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28088,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717064,"statfs":{"total":95995035648,"available":95966273536,"internally_reserved":0,"allocated":1097728,"data_stored":554288,"data_compressed":26507,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299446,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28148,"kb_used_data":1132,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717004,"statfs":{"total":95995035648,"available":95966212096,"internally_reserved":0,"allocated":1159168,"data_stored":1152236,"data_compressed":62624,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462972,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36396,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708756,"statfs":{"total":95995035648,"available":95957766144,"internally_reserved":0,"allocated":1085440,"data_stored":550476,"data_compressed":26507,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574933,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36460,"kb_used_data":1124,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708692,"statfs":{"total":95995035648,"available":95957700608,"internally_reserved":0,"allocated":1150976,"data_stored":609896,"data_compressed":26507,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738457,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28100,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717052,"statfs":{"total":95995035648,"available":95966261248,"internally_reserved":0,"allocated":1110016,"data_stored":551932,"data_compressed":26507,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411408,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28156,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716996,"statfs":{"total":95995035648,"available":95966203904,"internally_reserved":0,"allocated":1167360,"data_stored":609368,"data_compressed":26507,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149770,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36408,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708744,"statfs":{"total":95995035648,"available":95957753856,"internally_reserved":0,"allocated":1097728,"data_stored":549794,"data_compressed":26507,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888133,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28092,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717060,"statfs":{"total":95995035648,"available":95966269440,"internally_reserved":0,"allocated":1101824,"data_stored":555222,"data_compressed":26507,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724608,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28156,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716996,"statfs":{"total":95995035648,"available":95966203904,"internally_reserved":0,"allocated":1167360,"data_stored":1147038,"data_compressed":62624,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34481,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34481,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34481,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36117,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36117,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36117,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36117,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36117,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":60532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":61142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":60516,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-06-10T02:51:35.751 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph pg dump --format=json 2024-06-10T02:51:36.003 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:51:36.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:51:36 smithi188 ceph-mon[36599]: from='client.14811 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-10T02:51:36.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:51:36 smithi188 ceph-mon[36599]: pgmap v354: 129 pgs: 129 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 3.1 KiB/s wr, 10 op/s 2024-06-10T02:51:36.976 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:51:36 smithi052 ceph-mon[32362]: from='client.14811 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-10T02:51:36.976 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:51:36 smithi052 ceph-mon[32362]: pgmap v354: 129 pgs: 129 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 3.1 KiB/s wr, 10 op/s 2024-06-10T02:51:37.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:51:36 smithi120 ceph-mon[36634]: from='client.14811 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-10T02:51:37.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:51:36 smithi120 ceph-mon[36634]: pgmap v354: 129 pgs: 129 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 3.1 KiB/s wr, 10 op/s 2024-06-10T02:51:37.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:37.811+0000 7fc7a035f700 1 -- 172.21.15.52:0/2728182745 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc798101390 msgr2=0x7fc798101810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:51:37.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:37.811+0000 7fc7a035f700 1 --2- 172.21.15.52:0/2728182745 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc798101390 0x7fc798101810 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fc788009990 tx=0x7fc788009ca0 comp rx=0 tx=0).stop 2024-06-10T02:51:37.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:37.811+0000 7fc7a035f700 1 -- 172.21.15.52:0/2728182745 shutdown_connections 2024-06-10T02:51:37.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:37.811+0000 7fc7a035f700 1 --2- 172.21.15.52:0/2728182745 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc798101d50 0x7fc79810e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:51:37.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:37.811+0000 7fc7a035f700 1 --2- 172.21.15.52:0/2728182745 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc798107400 0x7fc7981077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:51:37.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:37.811+0000 7fc7a035f700 1 --2- 172.21.15.52:0/2728182745 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc798101390 0x7fc798101810 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:51:37.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:37.811+0000 7fc7a035f700 1 -- 172.21.15.52:0/2728182745 >> 172.21.15.52:0/2728182745 conn(0x7fc798078340 msgr2=0x7fc798078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:51:37.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:37.811+0000 7fc7a035f700 1 -- 172.21.15.52:0/2728182745 shutdown_connections 2024-06-10T02:51:37.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:37.811+0000 7fc7a035f700 1 -- 172.21.15.52:0/2728182745 wait complete. 2024-06-10T02:51:37.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:37.812+0000 7fc7a035f700 1 Processor -- start 2024-06-10T02:51:37.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:37.812+0000 7fc7a035f700 1 -- start start 2024-06-10T02:51:37.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:37.812+0000 7fc7a035f700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc798101390 0x7fc798103070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:51:37.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:37.812+0000 7fc7a035f700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc798101d50 0x7fc7981035b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:51:37.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:37.812+0000 7fc7a035f700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc798107400 0x7fc798103af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:51:37.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:37.812+0000 7fc7a035f700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fc7981116f0 con 0x7fc798101d50 2024-06-10T02:51:37.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:37.812+0000 7fc7a035f700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc798111570 con 0x7fc798107400 2024-06-10T02:51:37.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:37.812+0000 7fc7a035f700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc798111870 con 0x7fc798101390 2024-06-10T02:51:37.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:37.813+0000 7fc79d8fa700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc798101d50 0x7fc7981035b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:51:37.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:37.813+0000 7fc79d8fa700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc798101d50 0x7fc7981035b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37914/0 (socket says 172.21.15.52:37914) 2024-06-10T02:51:37.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:37.813+0000 7fc79d8fa700 1 -- 172.21.15.52:0/1521653962 learned_addr learned my addr 172.21.15.52:0/1521653962 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:51:37.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:37.813+0000 7fc79e8fc700 1 --2- 172.21.15.52:0/1521653962 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc798107400 0x7fc798103af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:51:37.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:37.813+0000 7fc79e0fb700 1 --2- 172.21.15.52:0/1521653962 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc798101390 0x7fc798103070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:51:37.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:37.813+0000 7fc79d8fa700 1 -- 172.21.15.52:0/1521653962 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc798101390 msgr2=0x7fc798103070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:51:37.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:37.813+0000 7fc79d8fa700 1 --2- 172.21.15.52:0/1521653962 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc798101390 0x7fc798103070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:51:37.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:37.813+0000 7fc79d8fa700 1 -- 172.21.15.52:0/1521653962 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc798107400 msgr2=0x7fc798103af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:51:37.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:37.813+0000 7fc79d8fa700 1 --2- 172.21.15.52:0/1521653962 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc798107400 0x7fc798103af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:51:37.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:37.813+0000 7fc79d8fa700 1 -- 172.21.15.52:0/1521653962 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7981a71a0 con 0x7fc798101d50 2024-06-10T02:51:37.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:37.813+0000 7fc79e8fc700 1 --2- 172.21.15.52:0/1521653962 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc798107400 0x7fc798103af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T02:51:37.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:37.813+0000 7fc79d8fa700 1 --2- 172.21.15.52:0/1521653962 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc798101d50 0x7fc7981035b0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fc7880038b0 tx=0x7fc788005590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:51:37.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:37.813+0000 7fc78f7fe700 1 -- 172.21.15.52:0/1521653962 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc788027070 con 0x7fc798101d50 2024-06-10T02:51:37.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:37.813+0000 7fc78f7fe700 1 -- 172.21.15.52:0/1521653962 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc7880040d0 con 0x7fc798101d50 2024-06-10T02:51:37.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:37.813+0000 7fc7a035f700 1 -- 172.21.15.52:0/1521653962 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7981a7430 con 0x7fc798101d50 2024-06-10T02:51:37.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:37.813+0000 7fc78f7fe700 1 -- 172.21.15.52:0/1521653962 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc788022dd0 con 0x7fc798101d50 2024-06-10T02:51:37.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:37.813+0000 7fc7a035f700 1 -- 172.21.15.52:0/1521653962 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7981a7960 con 0x7fc798101d50 2024-06-10T02:51:37.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:37.815+0000 7fc7a035f700 1 -- 172.21.15.52:0/1521653962 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc798069180 con 0x7fc798101d50 2024-06-10T02:51:37.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:37.816+0000 7fc78f7fe700 1 -- 172.21.15.52:0/1521653962 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fc78802a070 con 0x7fc798101d50 2024-06-10T02:51:37.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:37.816+0000 7fc78f7fe700 1 --2- 172.21.15.52:0/1521653962 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fc7840a2d30 0x7fc7840a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:51:37.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:37.817+0000 7fc78f7fe700 1 -- 172.21.15.52:0/1521653962 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9092+0+0 (secure 0 0 0) 0x7fc7880d0460 con 0x7fc798101d50 2024-06-10T02:51:37.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:37.817+0000 7fc79e0fb700 1 --2- 172.21.15.52:0/1521653962 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fc7840a2d30 0x7fc7840a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:51:37.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:37.817+0000 7fc79e0fb700 1 --2- 172.21.15.52:0/1521653962 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fc7840a2d30 0x7fc7840a51f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fc794009f20 tx=0x7fc7940094f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:51:37.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:37.821+0000 7fc78f7fe700 1 -- 172.21.15.52:0/1521653962 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fc788013070 con 0x7fc798101d50 2024-06-10T02:51:38.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:38.023+0000 7fc7a035f700 1 -- 172.21.15.52:0/1521653962 --> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc79806a190 con 0x7fc7840a2d30 2024-06-10T02:51:38.030 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:38.032+0000 7fc78f7fe700 1 -- 172.21.15.52:0/1521653962 <== mgr.14150 v2:172.21.15.52:6800/3214763969 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340686 (secure 0 0 0) 0x7fc79806a190 con 0x7fc7840a2d30 2024-06-10T02:51:38.035 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-10T02:51:38.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:38.036+0000 7fc7a035f700 1 -- 172.21.15.52:0/1521653962 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fc7840a2d30 msgr2=0x7fc7840a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:51:38.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:38.036+0000 7fc7a035f700 1 --2- 172.21.15.52:0/1521653962 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fc7840a2d30 0x7fc7840a51f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fc794009f20 tx=0x7fc7940094f0 comp rx=0 tx=0).stop 2024-06-10T02:51:38.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:38.036+0000 7fc7a035f700 1 -- 172.21.15.52:0/1521653962 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc798101d50 msgr2=0x7fc7981035b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:51:38.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:38.036+0000 7fc7a035f700 1 --2- 172.21.15.52:0/1521653962 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc798101d50 0x7fc7981035b0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fc7880038b0 tx=0x7fc788005590 comp rx=0 tx=0).stop 2024-06-10T02:51:38.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:38.036+0000 7fc7a035f700 1 -- 172.21.15.52:0/1521653962 shutdown_connections 2024-06-10T02:51:38.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:38.036+0000 7fc7a035f700 1 --2- 172.21.15.52:0/1521653962 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc798107400 0x7fc798103af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:51:38.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:38.036+0000 7fc7a035f700 1 --2- 172.21.15.52:0/1521653962 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc798101390 0x7fc798103070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:51:38.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:38.036+0000 7fc7a035f700 1 --2- 172.21.15.52:0/1521653962 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fc7840a2d30 0x7fc7840a51f0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:51:38.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:38.036+0000 7fc7a035f700 1 --2- 172.21.15.52:0/1521653962 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc798101d50 0x7fc7981035b0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:51:38.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:38.036+0000 7fc7a035f700 1 -- 172.21.15.52:0/1521653962 >> 172.21.15.52:0/1521653962 conn(0x7fc798078340 msgr2=0x7fc7981005a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:51:38.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:38.036+0000 7fc7a035f700 1 -- 172.21.15.52:0/1521653962 shutdown_connections 2024-06-10T02:51:38.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:38.036+0000 7fc7a035f700 1 -- 172.21.15.52:0/1521653962 wait complete. 2024-06-10T02:51:38.038 INFO:teuthology.orchestra.run.smithi052.stderr:dumped all 2024-06-10T02:51:38.708 INFO:teuthology.orchestra.run.smithi052.stdout:{"pg_ready":true,"pg_map":{"version":355,"stamp":"2024-06-10T02:51:36.998914+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":661239,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":436,"num_read_kb":343,"num_write":434,"num_write_kb":3085,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1181318,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":354,"ondisk_log_size":354,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":379052,"kb_used_data":13260,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124562772,"statfs":{"total":1151940427776,"available":1151552278528,"internally_reserved":0,"allocated":13578240,"data_stored":8599621,"data_compressed":426435,"data_compressed_allocated":577536,"data_compressed_original":2654208,"omap_allocated":19141,"internal_metadata":374453563},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":32429,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":144,"num_read_kb":104,"num_write":25,"num_write_kb":42,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003142"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.780589+0000","last_change":"2024-06-10T02:50:02.209146+0000","last_active":"2024-06-10T02:50:44.780589+0000","last_peered":"2024-06-10T02:50:44.780589+0000","last_clean":"2024-06-10T02:50:44.780589+0000","last_became_active":"2024-06-10T02:50:02.208027+0000","last_became_peered":"2024-06-10T02:50:02.208027+0000","last_unstale":"2024-06-10T02:50:44.780589+0000","last_undegraded":"2024-06-10T02:50:44.780589+0000","last_fullsized":"2024-06-10T02:50:44.780589+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:35:34.765849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.736487+0000","last_change":"2024-06-10T02:50:02.198767+0000","last_active":"2024-06-10T02:50:44.736487+0000","last_peered":"2024-06-10T02:50:44.736487+0000","last_clean":"2024-06-10T02:50:44.736487+0000","last_became_active":"2024-06-10T02:50:02.198450+0000","last_became_peered":"2024-06-10T02:50:02.198450+0000","last_unstale":"2024-06-10T02:50:44.736487+0000","last_undegraded":"2024-06-10T02:50:44.736487+0000","last_fullsized":"2024-06-10T02:50:44.736487+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:49:56.666083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.736775+0000","last_change":"2024-06-10T02:50:02.208885+0000","last_active":"2024-06-10T02:50:44.736775+0000","last_peered":"2024-06-10T02:50:44.736775+0000","last_clean":"2024-06-10T02:50:44.736775+0000","last_became_active":"2024-06-10T02:50:02.206953+0000","last_became_peered":"2024-06-10T02:50:02.206953+0000","last_unstale":"2024-06-10T02:50:44.736775+0000","last_undegraded":"2024-06-10T02:50:44.736775+0000","last_fullsized":"2024-06-10T02:50:44.736775+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:41:23.928939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.780539+0000","last_change":"2024-06-10T02:50:02.208632+0000","last_active":"2024-06-10T02:50:44.780539+0000","last_peered":"2024-06-10T02:50:44.780539+0000","last_clean":"2024-06-10T02:50:44.780539+0000","last_became_active":"2024-06-10T02:50:02.207587+0000","last_became_peered":"2024-06-10T02:50:02.207587+0000","last_unstale":"2024-06-10T02:50:44.780539+0000","last_undegraded":"2024-06-10T02:50:44.780539+0000","last_fullsized":"2024-06-10T02:50:44.780539+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T14:34:36.720170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.739593+0000","last_change":"2024-06-10T02:50:02.197966+0000","last_active":"2024-06-10T02:50:44.739593+0000","last_peered":"2024-06-10T02:50:44.739593+0000","last_clean":"2024-06-10T02:50:44.739593+0000","last_became_active":"2024-06-10T02:50:02.197612+0000","last_became_peered":"2024-06-10T02:50:02.197612+0000","last_unstale":"2024-06-10T02:50:44.739593+0000","last_undegraded":"2024-06-10T02:50:44.739593+0000","last_fullsized":"2024-06-10T02:50:44.739593+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:24:53.804613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.739586+0000","last_change":"2024-06-10T02:50:02.201952+0000","last_active":"2024-06-10T02:50:44.739586+0000","last_peered":"2024-06-10T02:50:44.739586+0000","last_clean":"2024-06-10T02:50:44.739586+0000","last_became_active":"2024-06-10T02:50:02.200828+0000","last_became_peered":"2024-06-10T02:50:02.200828+0000","last_unstale":"2024-06-10T02:50:44.739586+0000","last_undegraded":"2024-06-10T02:50:44.739586+0000","last_fullsized":"2024-06-10T02:50:44.739586+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:46:28.518846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.735333+0000","last_change":"2024-06-10T02:50:02.203483+0000","last_active":"2024-06-10T02:50:44.735333+0000","last_peered":"2024-06-10T02:50:44.735333+0000","last_clean":"2024-06-10T02:50:44.735333+0000","last_became_active":"2024-06-10T02:50:02.203153+0000","last_became_peered":"2024-06-10T02:50:02.203153+0000","last_unstale":"2024-06-10T02:50:44.735333+0000","last_undegraded":"2024-06-10T02:50:44.735333+0000","last_fullsized":"2024-06-10T02:50:44.735333+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:29:30.422449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.733706+0000","last_change":"2024-06-10T02:50:02.201686+0000","last_active":"2024-06-10T02:50:44.733706+0000","last_peered":"2024-06-10T02:50:44.733706+0000","last_clean":"2024-06-10T02:50:44.733706+0000","last_became_active":"2024-06-10T02:50:02.200973+0000","last_became_peered":"2024-06-10T02:50:02.200973+0000","last_unstale":"2024-06-10T02:50:44.733706+0000","last_undegraded":"2024-06-10T02:50:44.733706+0000","last_fullsized":"2024-06-10T02:50:44.733706+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:41:13.712675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.740098+0000","last_change":"2024-06-10T02:50:02.210497+0000","last_active":"2024-06-10T02:50:44.740098+0000","last_peered":"2024-06-10T02:50:44.740098+0000","last_clean":"2024-06-10T02:50:44.740098+0000","last_became_active":"2024-06-10T02:50:02.209675+0000","last_became_peered":"2024-06-10T02:50:02.209675+0000","last_unstale":"2024-06-10T02:50:44.740098+0000","last_undegraded":"2024-06-10T02:50:44.740098+0000","last_fullsized":"2024-06-10T02:50:44.740098+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:50:29.879669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":40,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.741139+0000","last_change":"2024-06-10T02:50:44.741139+0000","last_active":"2024-06-10T02:50:44.741139+0000","last_peered":"2024-06-10T02:50:44.741139+0000","last_clean":"2024-06-10T02:50:44.741139+0000","last_became_active":"2024-06-10T02:50:44.740685+0000","last_became_peered":"2024-06-10T02:50:44.740685+0000","last_unstale":"2024-06-10T02:50:44.741139+0000","last_undegraded":"2024-06-10T02:50:44.741139+0000","last_fullsized":"2024-06-10T02:50:44.741139+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:58:59.872732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.736093+0000","last_change":"2024-06-10T02:50:02.206993+0000","last_active":"2024-06-10T02:50:44.736093+0000","last_peered":"2024-06-10T02:50:44.736093+0000","last_clean":"2024-06-10T02:50:44.736093+0000","last_became_active":"2024-06-10T02:50:02.206650+0000","last_became_peered":"2024-06-10T02:50:02.206650+0000","last_unstale":"2024-06-10T02:50:44.736093+0000","last_undegraded":"2024-06-10T02:50:44.736093+0000","last_fullsized":"2024-06-10T02:50:44.736093+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T14:27:49.555350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.780395+0000","last_change":"2024-06-10T02:50:02.200816+0000","last_active":"2024-06-10T02:50:44.780395+0000","last_peered":"2024-06-10T02:50:44.780395+0000","last_clean":"2024-06-10T02:50:44.780395+0000","last_became_active":"2024-06-10T02:50:02.200503+0000","last_became_peered":"2024-06-10T02:50:02.200503+0000","last_unstale":"2024-06-10T02:50:44.780395+0000","last_undegraded":"2024-06-10T02:50:44.780395+0000","last_fullsized":"2024-06-10T02:50:44.780395+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:01:08.326728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.780143+0000","last_change":"2024-06-10T02:50:02.203644+0000","last_active":"2024-06-10T02:50:44.780143+0000","last_peered":"2024-06-10T02:50:44.780143+0000","last_clean":"2024-06-10T02:50:44.780143+0000","last_became_active":"2024-06-10T02:50:02.203100+0000","last_became_peered":"2024-06-10T02:50:02.203100+0000","last_unstale":"2024-06-10T02:50:44.780143+0000","last_undegraded":"2024-06-10T02:50:44.780143+0000","last_fullsized":"2024-06-10T02:50:44.780143+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:01:42.730089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.735964+0000","last_change":"2024-06-10T02:50:02.203330+0000","last_active":"2024-06-10T02:50:44.735964+0000","last_peered":"2024-06-10T02:50:44.735964+0000","last_clean":"2024-06-10T02:50:44.735964+0000","last_became_active":"2024-06-10T02:50:02.202995+0000","last_became_peered":"2024-06-10T02:50:02.202995+0000","last_unstale":"2024-06-10T02:50:44.735964+0000","last_undegraded":"2024-06-10T02:50:44.735964+0000","last_fullsized":"2024-06-10T02:50:44.735964+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:17:23.170591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.736189+0000","last_change":"2024-06-10T02:50:02.204875+0000","last_active":"2024-06-10T02:50:44.736189+0000","last_peered":"2024-06-10T02:50:44.736189+0000","last_clean":"2024-06-10T02:50:44.736189+0000","last_became_active":"2024-06-10T02:50:02.200706+0000","last_became_peered":"2024-06-10T02:50:02.200706+0000","last_unstale":"2024-06-10T02:50:44.736189+0000","last_undegraded":"2024-06-10T02:50:44.736189+0000","last_fullsized":"2024-06-10T02:50:44.736189+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:27:12.282772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.738872+0000","last_change":"2024-06-10T02:50:02.203187+0000","last_active":"2024-06-10T02:50:44.738872+0000","last_peered":"2024-06-10T02:50:44.738872+0000","last_clean":"2024-06-10T02:50:44.738872+0000","last_became_active":"2024-06-10T02:50:02.202034+0000","last_became_peered":"2024-06-10T02:50:02.202034+0000","last_unstale":"2024-06-10T02:50:44.738872+0000","last_undegraded":"2024-06-10T02:50:44.738872+0000","last_fullsized":"2024-06-10T02:50:44.738872+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:59:44.783084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.780351+0000","last_change":"2024-06-10T02:50:02.209102+0000","last_active":"2024-06-10T02:50:44.780351+0000","last_peered":"2024-06-10T02:50:44.780351+0000","last_clean":"2024-06-10T02:50:44.780351+0000","last_became_active":"2024-06-10T02:50:02.208730+0000","last_became_peered":"2024-06-10T02:50:02.208730+0000","last_unstale":"2024-06-10T02:50:44.780351+0000","last_undegraded":"2024-06-10T02:50:44.780351+0000","last_fullsized":"2024-06-10T02:50:44.780351+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:08:38.838415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.738035+0000","last_change":"2024-06-10T02:50:02.204008+0000","last_active":"2024-06-10T02:50:44.738035+0000","last_peered":"2024-06-10T02:50:44.738035+0000","last_clean":"2024-06-10T02:50:44.738035+0000","last_became_active":"2024-06-10T02:50:02.203655+0000","last_became_peered":"2024-06-10T02:50:02.203655+0000","last_unstale":"2024-06-10T02:50:44.738035+0000","last_undegraded":"2024-06-10T02:50:44.738035+0000","last_fullsized":"2024-06-10T02:50:44.738035+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:32:15.972662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.737751+0000","last_change":"2024-06-10T02:50:02.204917+0000","last_active":"2024-06-10T02:50:44.737751+0000","last_peered":"2024-06-10T02:50:44.737751+0000","last_clean":"2024-06-10T02:50:44.737751+0000","last_became_active":"2024-06-10T02:50:02.204568+0000","last_became_peered":"2024-06-10T02:50:02.204568+0000","last_unstale":"2024-06-10T02:50:44.737751+0000","last_undegraded":"2024-06-10T02:50:44.737751+0000","last_fullsized":"2024-06-10T02:50:44.737751+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:49:46.802665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.737040+0000","last_change":"2024-06-10T02:50:12.240697+0000","last_active":"2024-06-10T02:50:44.737040+0000","last_peered":"2024-06-10T02:50:44.737040+0000","last_clean":"2024-06-10T02:50:44.737040+0000","last_became_active":"2024-06-10T02:50:12.240229+0000","last_became_peered":"2024-06-10T02:50:12.240229+0000","last_unstale":"2024-06-10T02:50:44.737040+0000","last_undegraded":"2024-06-10T02:50:44.737040+0000","last_fullsized":"2024-06-10T02:50:44.737040+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:45:55.071960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"98'13","reported_seq":92,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:51:33.668648+0000","last_change":"2024-06-10T02:50:44.786783+0000","last_active":"2024-06-10T02:51:33.668648+0000","last_peered":"2024-06-10T02:51:33.668648+0000","last_clean":"2024-06-10T02:51:33.668648+0000","last_became_active":"2024-06-10T02:50:44.781773+0000","last_became_peered":"2024-06-10T02:50:44.781773+0000","last_unstale":"2024-06-10T02:51:33.668648+0000","last_undegraded":"2024-06-10T02:51:33.668648+0000","last_fullsized":"2024-06-10T02:51:33.668648+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:31:06.025854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":27,"num_read_kb":27,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,4],"acting":[9,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.734284+0000","last_change":"2024-06-10T02:50:02.202573+0000","last_active":"2024-06-10T02:50:44.734284+0000","last_peered":"2024-06-10T02:50:44.734284+0000","last_clean":"2024-06-10T02:50:44.734284+0000","last_became_active":"2024-06-10T02:50:02.202241+0000","last_became_peered":"2024-06-10T02:50:02.202241+0000","last_unstale":"2024-06-10T02:50:44.734284+0000","last_undegraded":"2024-06-10T02:50:44.734284+0000","last_fullsized":"2024-06-10T02:50:44.734284+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:31:40.230851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.779776+0000","last_change":"2024-06-10T02:50:12.238372+0000","last_active":"2024-06-10T02:50:44.779776+0000","last_peered":"2024-06-10T02:50:44.779776+0000","last_clean":"2024-06-10T02:50:44.779776+0000","last_became_active":"2024-06-10T02:50:12.237884+0000","last_became_peered":"2024-06-10T02:50:12.237884+0000","last_unstale":"2024-06-10T02:50:44.779776+0000","last_undegraded":"2024-06-10T02:50:44.779776+0000","last_fullsized":"2024-06-10T02:50:44.779776+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:55:31.432655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"98'2","reported_seq":42,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.734660+0000","last_change":"2024-06-10T02:49:59.031528+0000","last_active":"2024-06-10T02:50:44.734660+0000","last_peered":"2024-06-10T02:50:44.734660+0000","last_clean":"2024-06-10T02:50:44.734660+0000","last_became_active":"2024-06-10T02:49:59.031147+0000","last_became_peered":"2024-06-10T02:49:59.031147+0000","last_unstale":"2024-06-10T02:50:44.734660+0000","last_undegraded":"2024-06-10T02:50:44.734660+0000","last_fullsized":"2024-06-10T02:50:44.734660+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:17:08.534396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.736728+0000","last_change":"2024-06-10T02:50:02.202858+0000","last_active":"2024-06-10T02:50:44.736728+0000","last_peered":"2024-06-10T02:50:44.736728+0000","last_clean":"2024-06-10T02:50:44.736728+0000","last_became_active":"2024-06-10T02:50:02.202263+0000","last_became_peered":"2024-06-10T02:50:02.202263+0000","last_unstale":"2024-06-10T02:50:44.736728+0000","last_undegraded":"2024-06-10T02:50:44.736728+0000","last_fullsized":"2024-06-10T02:50:44.736728+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:23:15.382762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.780045+0000","last_change":"2024-06-10T02:50:12.237370+0000","last_active":"2024-06-10T02:50:44.780045+0000","last_peered":"2024-06-10T02:50:44.780045+0000","last_clean":"2024-06-10T02:50:44.780045+0000","last_became_active":"2024-06-10T02:50:12.236803+0000","last_became_peered":"2024-06-10T02:50:12.236803+0000","last_unstale":"2024-06-10T02:50:44.780045+0000","last_undegraded":"2024-06-10T02:50:44.780045+0000","last_fullsized":"2024-06-10T02:50:44.780045+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:06:40.695710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"98'12","reported_seq":53,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:51:09.633101+0000","last_change":"2024-06-10T02:49:59.000911+0000","last_active":"2024-06-10T02:51:09.633101+0000","last_peered":"2024-06-10T02:51:09.633101+0000","last_clean":"2024-06-10T02:51:09.633101+0000","last_became_active":"2024-06-10T02:49:59.000571+0000","last_became_peered":"2024-06-10T02:49:59.000571+0000","last_unstale":"2024-06-10T02:51:09.633101+0000","last_undegraded":"2024-06-10T02:51:09.633101+0000","last_fullsized":"2024-06-10T02:51:09.633101+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:49:24.635793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.738682+0000","last_change":"2024-06-10T02:50:02.209614+0000","last_active":"2024-06-10T02:50:44.738682+0000","last_peered":"2024-06-10T02:50:44.738682+0000","last_clean":"2024-06-10T02:50:44.738682+0000","last_became_active":"2024-06-10T02:50:02.209251+0000","last_became_peered":"2024-06-10T02:50:02.209251+0000","last_unstale":"2024-06-10T02:50:44.738682+0000","last_undegraded":"2024-06-10T02:50:44.738682+0000","last_fullsized":"2024-06-10T02:50:44.738682+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:35:32.256081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.779720+0000","last_change":"2024-06-10T02:50:12.237811+0000","last_active":"2024-06-10T02:50:44.779720+0000","last_peered":"2024-06-10T02:50:44.779720+0000","last_clean":"2024-06-10T02:50:44.779720+0000","last_became_active":"2024-06-10T02:50:12.237017+0000","last_became_peered":"2024-06-10T02:50:12.237017+0000","last_unstale":"2024-06-10T02:50:44.779720+0000","last_undegraded":"2024-06-10T02:50:44.779720+0000","last_fullsized":"2024-06-10T02:50:44.779720+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:14:29.818885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"98'5","reported_seq":46,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:51:08.627882+0000","last_change":"2024-06-10T02:49:58.996971+0000","last_active":"2024-06-10T02:51:08.627882+0000","last_peered":"2024-06-10T02:51:08.627882+0000","last_clean":"2024-06-10T02:51:08.627882+0000","last_became_active":"2024-06-10T02:49:58.996601+0000","last_became_peered":"2024-06-10T02:49:58.996601+0000","last_unstale":"2024-06-10T02:51:08.627882+0000","last_undegraded":"2024-06-10T02:51:08.627882+0000","last_fullsized":"2024-06-10T02:51:08.627882+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:26:45.963351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":40,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.745816+0000","last_change":"2024-06-10T02:50:44.745816+0000","last_active":"2024-06-10T02:50:44.745816+0000","last_peered":"2024-06-10T02:50:44.745816+0000","last_clean":"2024-06-10T02:50:44.745816+0000","last_became_active":"2024-06-10T02:50:44.745386+0000","last_became_peered":"2024-06-10T02:50:44.745386+0000","last_unstale":"2024-06-10T02:50:44.745816+0000","last_undegraded":"2024-06-10T02:50:44.745816+0000","last_fullsized":"2024-06-10T02:50:44.745816+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:08:53.977133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.737434+0000","last_change":"2024-06-10T02:50:12.236835+0000","last_active":"2024-06-10T02:50:44.737434+0000","last_peered":"2024-06-10T02:50:44.737434+0000","last_clean":"2024-06-10T02:50:44.737434+0000","last_became_active":"2024-06-10T02:50:12.236309+0000","last_became_peered":"2024-06-10T02:50:12.236309+0000","last_unstale":"2024-06-10T02:50:44.737434+0000","last_undegraded":"2024-06-10T02:50:44.737434+0000","last_fullsized":"2024-06-10T02:50:44.737434+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:09:02.789286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"98'6","reported_seq":76,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:51:34.671507+0000","last_change":"2024-06-10T02:49:58.997725+0000","last_active":"2024-06-10T02:51:34.671507+0000","last_peered":"2024-06-10T02:51:34.671507+0000","last_clean":"2024-06-10T02:51:34.671507+0000","last_became_active":"2024-06-10T02:49:58.997375+0000","last_became_peered":"2024-06-10T02:49:58.997375+0000","last_unstale":"2024-06-10T02:51:34.671507+0000","last_undegraded":"2024-06-10T02:51:34.671507+0000","last_fullsized":"2024-06-10T02:51:34.671507+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:28:49.714356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1012,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":30,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.739670+0000","last_change":"2024-06-10T02:50:02.201247+0000","last_active":"2024-06-10T02:50:44.739670+0000","last_peered":"2024-06-10T02:50:44.739670+0000","last_clean":"2024-06-10T02:50:44.739670+0000","last_became_active":"2024-06-10T02:50:02.200897+0000","last_became_peered":"2024-06-10T02:50:02.200897+0000","last_unstale":"2024-06-10T02:50:44.739670+0000","last_undegraded":"2024-06-10T02:50:44.739670+0000","last_fullsized":"2024-06-10T02:50:44.739670+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:01:58.166644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.737778+0000","last_change":"2024-06-10T02:50:12.243097+0000","last_active":"2024-06-10T02:50:44.737778+0000","last_peered":"2024-06-10T02:50:44.737778+0000","last_clean":"2024-06-10T02:50:44.737778+0000","last_became_active":"2024-06-10T02:50:12.242501+0000","last_became_peered":"2024-06-10T02:50:12.242501+0000","last_unstale":"2024-06-10T02:50:44.737778+0000","last_undegraded":"2024-06-10T02:50:44.737778+0000","last_fullsized":"2024-06-10T02:50:44.737778+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T14:42:44.220340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"98'7","reported_seq":47,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:51:07.617849+0000","last_change":"2024-06-10T02:49:58.996956+0000","last_active":"2024-06-10T02:51:07.617849+0000","last_peered":"2024-06-10T02:51:07.617849+0000","last_clean":"2024-06-10T02:51:07.617849+0000","last_became_active":"2024-06-10T02:49:58.996634+0000","last_became_peered":"2024-06-10T02:49:58.996634+0000","last_unstale":"2024-06-10T02:51:07.617849+0000","last_undegraded":"2024-06-10T02:51:07.617849+0000","last_fullsized":"2024-06-10T02:51:07.617849+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:25:02.364472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.741747+0000","last_change":"2024-06-10T02:50:02.205808+0000","last_active":"2024-06-10T02:50:44.741747+0000","last_peered":"2024-06-10T02:50:44.741747+0000","last_clean":"2024-06-10T02:50:44.741747+0000","last_became_active":"2024-06-10T02:50:02.203558+0000","last_became_peered":"2024-06-10T02:50:02.203558+0000","last_unstale":"2024-06-10T02:50:44.741747+0000","last_undegraded":"2024-06-10T02:50:44.741747+0000","last_fullsized":"2024-06-10T02:50:44.741747+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T13:43:23.315375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.780298+0000","last_change":"2024-06-10T02:50:12.242005+0000","last_active":"2024-06-10T02:50:44.780298+0000","last_peered":"2024-06-10T02:50:44.780298+0000","last_clean":"2024-06-10T02:50:44.780298+0000","last_became_active":"2024-06-10T02:50:12.241295+0000","last_became_peered":"2024-06-10T02:50:12.241295+0000","last_unstale":"2024-06-10T02:50:44.780298+0000","last_undegraded":"2024-06-10T02:50:44.780298+0000","last_fullsized":"2024-06-10T02:50:44.780298+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:32:52.702756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"98'3","reported_seq":43,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:51:06.613737+0000","last_change":"2024-06-10T02:49:58.995579+0000","last_active":"2024-06-10T02:51:06.613737+0000","last_peered":"2024-06-10T02:51:06.613737+0000","last_clean":"2024-06-10T02:51:06.613737+0000","last_became_active":"2024-06-10T02:49:58.995208+0000","last_became_peered":"2024-06-10T02:49:58.995208+0000","last_unstale":"2024-06-10T02:51:06.613737+0000","last_undegraded":"2024-06-10T02:51:06.613737+0000","last_fullsized":"2024-06-10T02:51:06.613737+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:26:03.513482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.733787+0000","last_change":"2024-06-10T02:50:02.197090+0000","last_active":"2024-06-10T02:50:44.733787+0000","last_peered":"2024-06-10T02:50:44.733787+0000","last_clean":"2024-06-10T02:50:44.733787+0000","last_became_active":"2024-06-10T02:50:02.196746+0000","last_became_peered":"2024-06-10T02:50:02.196746+0000","last_unstale":"2024-06-10T02:50:44.733787+0000","last_undegraded":"2024-06-10T02:50:44.733787+0000","last_fullsized":"2024-06-10T02:50:44.733787+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:52:09.987978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.736335+0000","last_change":"2024-06-10T02:50:12.234632+0000","last_active":"2024-06-10T02:50:44.736335+0000","last_peered":"2024-06-10T02:50:44.736335+0000","last_clean":"2024-06-10T02:50:44.736335+0000","last_became_active":"2024-06-10T02:50:12.234266+0000","last_became_peered":"2024-06-10T02:50:12.234266+0000","last_unstale":"2024-06-10T02:50:44.736335+0000","last_undegraded":"2024-06-10T02:50:44.736335+0000","last_fullsized":"2024-06-10T02:50:44.736335+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:21:27.348441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"98'3","reported_seq":44,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:51:08.628643+0000","last_change":"2024-06-10T02:49:58.999086+0000","last_active":"2024-06-10T02:51:08.628643+0000","last_peered":"2024-06-10T02:51:08.628643+0000","last_clean":"2024-06-10T02:51:08.628643+0000","last_became_active":"2024-06-10T02:49:58.998767+0000","last_became_peered":"2024-06-10T02:49:58.998767+0000","last_unstale":"2024-06-10T02:51:08.628643+0000","last_undegraded":"2024-06-10T02:51:08.628643+0000","last_fullsized":"2024-06-10T02:51:08.628643+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:28:12.922391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.738078+0000","last_change":"2024-06-10T02:50:02.196641+0000","last_active":"2024-06-10T02:50:44.738078+0000","last_peered":"2024-06-10T02:50:44.738078+0000","last_clean":"2024-06-10T02:50:44.738078+0000","last_became_active":"2024-06-10T02:50:02.196285+0000","last_became_peered":"2024-06-10T02:50:02.196285+0000","last_unstale":"2024-06-10T02:50:44.738078+0000","last_undegraded":"2024-06-10T02:50:44.738078+0000","last_fullsized":"2024-06-10T02:50:44.738078+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T13:55:31.284028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.733867+0000","last_change":"2024-06-10T02:50:12.240988+0000","last_active":"2024-06-10T02:50:44.733867+0000","last_peered":"2024-06-10T02:50:44.733867+0000","last_clean":"2024-06-10T02:50:44.733867+0000","last_became_active":"2024-06-10T02:50:12.240490+0000","last_became_peered":"2024-06-10T02:50:12.240490+0000","last_unstale":"2024-06-10T02:50:44.733867+0000","last_undegraded":"2024-06-10T02:50:44.733867+0000","last_fullsized":"2024-06-10T02:50:44.733867+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:54:36.015221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"98'4","reported_seq":100,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:51:36.673386+0000","last_change":"2024-06-10T02:49:58.989557+0000","last_active":"2024-06-10T02:51:36.673386+0000","last_peered":"2024-06-10T02:51:36.673386+0000","last_clean":"2024-06-10T02:51:36.673386+0000","last_became_active":"2024-06-10T02:49:58.989101+0000","last_became_peered":"2024-06-10T02:49:58.989101+0000","last_unstale":"2024-06-10T02:51:36.673386+0000","last_undegraded":"2024-06-10T02:51:36.673386+0000","last_fullsized":"2024-06-10T02:51:36.673386+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:27:42.182012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":56,"num_read_kb":56,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.739177+0000","last_change":"2024-06-10T02:50:02.208752+0000","last_active":"2024-06-10T02:50:44.739177+0000","last_peered":"2024-06-10T02:50:44.739177+0000","last_clean":"2024-06-10T02:50:44.739177+0000","last_became_active":"2024-06-10T02:50:02.208420+0000","last_became_peered":"2024-06-10T02:50:02.208420+0000","last_unstale":"2024-06-10T02:50:44.739177+0000","last_undegraded":"2024-06-10T02:50:44.739177+0000","last_fullsized":"2024-06-10T02:50:44.739177+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:28:09.412317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.737800+0000","last_change":"2024-06-10T02:50:12.236397+0000","last_active":"2024-06-10T02:50:44.737800+0000","last_peered":"2024-06-10T02:50:44.737800+0000","last_clean":"2024-06-10T02:50:44.737800+0000","last_became_active":"2024-06-10T02:50:12.235972+0000","last_became_peered":"2024-06-10T02:50:12.235972+0000","last_unstale":"2024-06-10T02:50:44.737800+0000","last_undegraded":"2024-06-10T02:50:44.737800+0000","last_fullsized":"2024-06-10T02:50:44.737800+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:38:41.723895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"98'4","reported_seq":44,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:51:08.626900+0000","last_change":"2024-06-10T02:49:58.996616+0000","last_active":"2024-06-10T02:51:08.626900+0000","last_peered":"2024-06-10T02:51:08.626900+0000","last_clean":"2024-06-10T02:51:08.626900+0000","last_became_active":"2024-06-10T02:49:58.996289+0000","last_became_peered":"2024-06-10T02:49:58.996289+0000","last_unstale":"2024-06-10T02:51:08.626900+0000","last_undegraded":"2024-06-10T02:51:08.626900+0000","last_fullsized":"2024-06-10T02:51:08.626900+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:22:12.993608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.736962+0000","last_change":"2024-06-10T02:50:02.205230+0000","last_active":"2024-06-10T02:50:44.736962+0000","last_peered":"2024-06-10T02:50:44.736962+0000","last_clean":"2024-06-10T02:50:44.736962+0000","last_became_active":"2024-06-10T02:50:02.204835+0000","last_became_peered":"2024-06-10T02:50:02.204835+0000","last_unstale":"2024-06-10T02:50:44.736962+0000","last_undegraded":"2024-06-10T02:50:44.736962+0000","last_fullsized":"2024-06-10T02:50:44.736962+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:17:35.084921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.780286+0000","last_change":"2024-06-10T02:50:12.231684+0000","last_active":"2024-06-10T02:50:44.780286+0000","last_peered":"2024-06-10T02:50:44.780286+0000","last_clean":"2024-06-10T02:50:44.780286+0000","last_became_active":"2024-06-10T02:50:12.231256+0000","last_became_peered":"2024-06-10T02:50:12.231256+0000","last_unstale":"2024-06-10T02:50:44.780286+0000","last_undegraded":"2024-06-10T02:50:44.780286+0000","last_fullsized":"2024-06-10T02:50:44.780286+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:17:31.878289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"98'5","reported_seq":46,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:51:07.616305+0000","last_change":"2024-06-10T02:49:59.033950+0000","last_active":"2024-06-10T02:51:07.616305+0000","last_peered":"2024-06-10T02:51:07.616305+0000","last_clean":"2024-06-10T02:51:07.616305+0000","last_became_active":"2024-06-10T02:49:59.033688+0000","last_became_peered":"2024-06-10T02:49:59.033688+0000","last_unstale":"2024-06-10T02:51:07.616305+0000","last_undegraded":"2024-06-10T02:51:07.616305+0000","last_fullsized":"2024-06-10T02:51:07.616305+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:32:03.892091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":40,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.781378+0000","last_change":"2024-06-10T02:50:44.781378+0000","last_active":"2024-06-10T02:50:44.781378+0000","last_peered":"2024-06-10T02:50:44.781378+0000","last_clean":"2024-06-10T02:50:44.781378+0000","last_became_active":"2024-06-10T02:50:44.780955+0000","last_became_peered":"2024-06-10T02:50:44.780955+0000","last_unstale":"2024-06-10T02:50:44.781378+0000","last_undegraded":"2024-06-10T02:50:44.781378+0000","last_fullsized":"2024-06-10T02:50:44.781378+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:49:13.760381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.738451+0000","last_change":"2024-06-10T02:50:12.239790+0000","last_active":"2024-06-10T02:50:44.738451+0000","last_peered":"2024-06-10T02:50:44.738451+0000","last_clean":"2024-06-10T02:50:44.738451+0000","last_became_active":"2024-06-10T02:50:12.238176+0000","last_became_peered":"2024-06-10T02:50:12.238176+0000","last_unstale":"2024-06-10T02:50:44.738451+0000","last_undegraded":"2024-06-10T02:50:44.738451+0000","last_fullsized":"2024-06-10T02:50:44.738451+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:03:47.705367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"98'34","reported_seq":139,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:51:36.685112+0000","last_change":"2024-06-10T02:49:58.997890+0000","last_active":"2024-06-10T02:51:36.685112+0000","last_peered":"2024-06-10T02:51:36.685112+0000","last_clean":"2024-06-10T02:51:36.685112+0000","last_became_active":"2024-06-10T02:49:58.997549+0000","last_became_peered":"2024-06-10T02:49:58.997549+0000","last_unstale":"2024-06-10T02:51:36.685112+0000","last_undegraded":"2024-06-10T02:51:36.685112+0000","last_fullsized":"2024-06-10T02:51:36.685112+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T14:40:50.907029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62655,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":65,"num_read_kb":94,"num_write":36,"num_write_kb":76,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.740133+0000","last_change":"2024-06-10T02:50:02.199707+0000","last_active":"2024-06-10T02:50:44.740133+0000","last_peered":"2024-06-10T02:50:44.740133+0000","last_clean":"2024-06-10T02:50:44.740133+0000","last_became_active":"2024-06-10T02:50:02.198987+0000","last_became_peered":"2024-06-10T02:50:02.198987+0000","last_unstale":"2024-06-10T02:50:44.740133+0000","last_undegraded":"2024-06-10T02:50:44.740133+0000","last_fullsized":"2024-06-10T02:50:44.740133+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:48:03.704590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.742929+0000","last_change":"2024-06-10T02:50:12.239608+0000","last_active":"2024-06-10T02:50:44.742929+0000","last_peered":"2024-06-10T02:50:44.742929+0000","last_clean":"2024-06-10T02:50:44.742929+0000","last_became_active":"2024-06-10T02:50:12.238588+0000","last_became_peered":"2024-06-10T02:50:12.238588+0000","last_unstale":"2024-06-10T02:50:44.742929+0000","last_undegraded":"2024-06-10T02:50:44.742929+0000","last_fullsized":"2024-06-10T02:50:44.742929+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:26:12.718921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"98'9","reported_seq":51,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:51:09.633206+0000","last_change":"2024-06-10T02:49:58.997193+0000","last_active":"2024-06-10T02:51:09.633206+0000","last_peered":"2024-06-10T02:51:09.633206+0000","last_clean":"2024-06-10T02:51:09.633206+0000","last_became_active":"2024-06-10T02:49:58.996712+0000","last_became_peered":"2024-06-10T02:49:58.996712+0000","last_unstale":"2024-06-10T02:51:09.633206+0000","last_undegraded":"2024-06-10T02:51:09.633206+0000","last_fullsized":"2024-06-10T02:51:09.633206+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:02:46.894489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.779923+0000","last_change":"2024-06-10T02:50:02.200544+0000","last_active":"2024-06-10T02:50:44.779923+0000","last_peered":"2024-06-10T02:50:44.779923+0000","last_clean":"2024-06-10T02:50:44.779923+0000","last_became_active":"2024-06-10T02:50:02.200201+0000","last_became_peered":"2024-06-10T02:50:02.200201+0000","last_unstale":"2024-06-10T02:50:44.779923+0000","last_undegraded":"2024-06-10T02:50:44.779923+0000","last_fullsized":"2024-06-10T02:50:44.779923+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T13:12:40.925448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.780172+0000","last_change":"2024-06-10T02:50:12.236317+0000","last_active":"2024-06-10T02:50:44.780172+0000","last_peered":"2024-06-10T02:50:44.780172+0000","last_clean":"2024-06-10T02:50:44.780172+0000","last_became_active":"2024-06-10T02:50:12.235806+0000","last_became_peered":"2024-06-10T02:50:12.235806+0000","last_unstale":"2024-06-10T02:50:44.780172+0000","last_undegraded":"2024-06-10T02:50:44.780172+0000","last_fullsized":"2024-06-10T02:50:44.780172+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:29:21.425987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"98'5","reported_seq":46,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:51:08.628686+0000","last_change":"2024-06-10T02:49:58.994149+0000","last_active":"2024-06-10T02:51:08.628686+0000","last_peered":"2024-06-10T02:51:08.628686+0000","last_clean":"2024-06-10T02:51:08.628686+0000","last_became_active":"2024-06-10T02:49:58.993775+0000","last_became_peered":"2024-06-10T02:49:58.993775+0000","last_unstale":"2024-06-10T02:51:08.628686+0000","last_undegraded":"2024-06-10T02:51:08.628686+0000","last_fullsized":"2024-06-10T02:51:08.628686+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:23:55.553943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.735604+0000","last_change":"2024-06-10T02:50:02.199979+0000","last_active":"2024-06-10T02:50:44.735604+0000","last_peered":"2024-06-10T02:50:44.735604+0000","last_clean":"2024-06-10T02:50:44.735604+0000","last_became_active":"2024-06-10T02:50:02.199596+0000","last_became_peered":"2024-06-10T02:50:02.199596+0000","last_unstale":"2024-06-10T02:50:44.735604+0000","last_undegraded":"2024-06-10T02:50:44.735604+0000","last_fullsized":"2024-06-10T02:50:44.735604+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:13:58.647460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.738672+0000","last_change":"2024-06-10T02:50:12.235253+0000","last_active":"2024-06-10T02:50:44.738672+0000","last_peered":"2024-06-10T02:50:44.738672+0000","last_clean":"2024-06-10T02:50:44.738672+0000","last_became_active":"2024-06-10T02:50:12.234773+0000","last_became_peered":"2024-06-10T02:50:12.234773+0000","last_unstale":"2024-06-10T02:50:44.738672+0000","last_undegraded":"2024-06-10T02:50:44.738672+0000","last_fullsized":"2024-06-10T02:50:44.738672+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:38:10.971411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"98'6","reported_seq":76,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:51:34.670726+0000","last_change":"2024-06-10T02:49:58.996912+0000","last_active":"2024-06-10T02:51:34.670726+0000","last_peered":"2024-06-10T02:51:34.670726+0000","last_clean":"2024-06-10T02:51:34.670726+0000","last_became_active":"2024-06-10T02:49:58.996549+0000","last_became_peered":"2024-06-10T02:49:58.996549+0000","last_unstale":"2024-06-10T02:51:34.670726+0000","last_undegraded":"2024-06-10T02:51:34.670726+0000","last_fullsized":"2024-06-10T02:51:34.670726+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:07:52.757676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":978,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":31,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":40,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.741186+0000","last_change":"2024-06-10T02:50:44.741186+0000","last_active":"2024-06-10T02:50:44.741186+0000","last_peered":"2024-06-10T02:50:44.741186+0000","last_clean":"2024-06-10T02:50:44.741186+0000","last_became_active":"2024-06-10T02:50:44.740782+0000","last_became_peered":"2024-06-10T02:50:44.740782+0000","last_unstale":"2024-06-10T02:50:44.741186+0000","last_undegraded":"2024-06-10T02:50:44.741186+0000","last_fullsized":"2024-06-10T02:50:44.741186+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:46:03.863243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.780292+0000","last_change":"2024-06-10T02:50:12.236505+0000","last_active":"2024-06-10T02:50:44.780292+0000","last_peered":"2024-06-10T02:50:44.780292+0000","last_clean":"2024-06-10T02:50:44.780292+0000","last_became_active":"2024-06-10T02:50:12.236112+0000","last_became_peered":"2024-06-10T02:50:12.236112+0000","last_unstale":"2024-06-10T02:50:44.780292+0000","last_undegraded":"2024-06-10T02:50:44.780292+0000","last_fullsized":"2024-06-10T02:50:44.780292+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:13:38.586736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"98'17","reported_seq":58,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:51:08.627154+0000","last_change":"2024-06-10T02:49:59.031965+0000","last_active":"2024-06-10T02:51:08.627154+0000","last_peered":"2024-06-10T02:51:08.627154+0000","last_clean":"2024-06-10T02:51:08.627154+0000","last_became_active":"2024-06-10T02:49:59.031495+0000","last_became_peered":"2024-06-10T02:49:59.031495+0000","last_unstale":"2024-06-10T02:51:08.627154+0000","last_undegraded":"2024-06-10T02:51:08.627154+0000","last_fullsized":"2024-06-10T02:51:08.627154+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T14:21:06.610677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.743175+0000","last_change":"2024-06-10T02:50:02.209926+0000","last_active":"2024-06-10T02:50:44.743175+0000","last_peered":"2024-06-10T02:50:44.743175+0000","last_clean":"2024-06-10T02:50:44.743175+0000","last_became_active":"2024-06-10T02:50:02.209540+0000","last_became_peered":"2024-06-10T02:50:02.209540+0000","last_unstale":"2024-06-10T02:50:44.743175+0000","last_undegraded":"2024-06-10T02:50:44.743175+0000","last_fullsized":"2024-06-10T02:50:44.743175+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:49:40.497692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.743114+0000","last_change":"2024-06-10T02:50:12.239236+0000","last_active":"2024-06-10T02:50:44.743114+0000","last_peered":"2024-06-10T02:50:44.743114+0000","last_clean":"2024-06-10T02:50:44.743114+0000","last_became_active":"2024-06-10T02:50:12.238623+0000","last_became_peered":"2024-06-10T02:50:12.238623+0000","last_unstale":"2024-06-10T02:50:44.743114+0000","last_undegraded":"2024-06-10T02:50:44.743114+0000","last_fullsized":"2024-06-10T02:50:44.743114+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:10:13.651629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"98'2","reported_seq":43,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:51:07.610072+0000","last_change":"2024-06-10T02:49:58.998372+0000","last_active":"2024-06-10T02:51:07.610072+0000","last_peered":"2024-06-10T02:51:07.610072+0000","last_clean":"2024-06-10T02:51:07.610072+0000","last_became_active":"2024-06-10T02:49:58.998058+0000","last_became_peered":"2024-06-10T02:49:58.998058+0000","last_unstale":"2024-06-10T02:51:07.610072+0000","last_undegraded":"2024-06-10T02:51:07.610072+0000","last_fullsized":"2024-06-10T02:51:07.610072+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:42:50.509851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.780381+0000","last_change":"2024-06-10T02:50:02.209256+0000","last_active":"2024-06-10T02:50:44.780381+0000","last_peered":"2024-06-10T02:50:44.780381+0000","last_clean":"2024-06-10T02:50:44.780381+0000","last_became_active":"2024-06-10T02:50:02.207727+0000","last_became_peered":"2024-06-10T02:50:02.207727+0000","last_unstale":"2024-06-10T02:50:44.780381+0000","last_undegraded":"2024-06-10T02:50:44.780381+0000","last_fullsized":"2024-06-10T02:50:44.780381+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:23:00.175943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.739489+0000","last_change":"2024-06-10T02:50:12.729438+0000","last_active":"2024-06-10T02:50:44.739489+0000","last_peered":"2024-06-10T02:50:44.739489+0000","last_clean":"2024-06-10T02:50:44.739489+0000","last_became_active":"2024-06-10T02:50:12.728918+0000","last_became_peered":"2024-06-10T02:50:12.728918+0000","last_unstale":"2024-06-10T02:50:44.739489+0000","last_undegraded":"2024-06-10T02:50:44.739489+0000","last_fullsized":"2024-06-10T02:50:44.739489+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:04:48.861763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"98'1","reported_seq":47,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.782798+0000","last_change":"2024-06-10T02:50:44.782241+0000","last_active":"2024-06-10T02:50:44.782798+0000","last_peered":"2024-06-10T02:50:44.782798+0000","last_clean":"2024-06-10T02:50:44.782798+0000","last_became_active":"2024-06-10T02:50:44.781799+0000","last_became_peered":"2024-06-10T02:50:44.781799+0000","last_unstale":"2024-06-10T02:50:44.782798+0000","last_undegraded":"2024-06-10T02:50:44.782798+0000","last_fullsized":"2024-06-10T02:50:44.782798+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:38:35.726258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.739478+0000","last_change":"2024-06-10T02:50:02.198888+0000","last_active":"2024-06-10T02:50:44.739478+0000","last_peered":"2024-06-10T02:50:44.739478+0000","last_clean":"2024-06-10T02:50:44.739478+0000","last_became_active":"2024-06-10T02:50:02.198346+0000","last_became_peered":"2024-06-10T02:50:02.198346+0000","last_unstale":"2024-06-10T02:50:44.739478+0000","last_undegraded":"2024-06-10T02:50:44.739478+0000","last_fullsized":"2024-06-10T02:50:44.739478+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:15:42.422898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.780257+0000","last_change":"2024-06-10T02:50:12.232763+0000","last_active":"2024-06-10T02:50:44.780257+0000","last_peered":"2024-06-10T02:50:44.780257+0000","last_clean":"2024-06-10T02:50:44.780257+0000","last_became_active":"2024-06-10T02:50:12.232390+0000","last_became_peered":"2024-06-10T02:50:12.232390+0000","last_unstale":"2024-06-10T02:50:44.780257+0000","last_undegraded":"2024-06-10T02:50:44.780257+0000","last_fullsized":"2024-06-10T02:50:44.780257+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:31:47.973500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"98'6","reported_seq":72,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:51:31.666279+0000","last_change":"2024-06-10T02:49:58.994308+0000","last_active":"2024-06-10T02:51:31.666279+0000","last_peered":"2024-06-10T02:51:31.666279+0000","last_clean":"2024-06-10T02:51:31.666279+0000","last_became_active":"2024-06-10T02:49:58.993494+0000","last_became_peered":"2024-06-10T02:49:58.993494+0000","last_unstale":"2024-06-10T02:51:31.666279+0000","last_undegraded":"2024-06-10T02:51:31.666279+0000","last_fullsized":"2024-06-10T02:51:31.666279+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:07:28.452276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":978,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":26,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.739405+0000","last_change":"2024-06-10T02:50:02.206872+0000","last_active":"2024-06-10T02:50:44.739405+0000","last_peered":"2024-06-10T02:50:44.739405+0000","last_clean":"2024-06-10T02:50:44.739405+0000","last_became_active":"2024-06-10T02:50:02.206307+0000","last_became_peered":"2024-06-10T02:50:02.206307+0000","last_unstale":"2024-06-10T02:50:44.739405+0000","last_undegraded":"2024-06-10T02:50:44.739405+0000","last_fullsized":"2024-06-10T02:50:44.739405+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:01:04.645674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.780303+0000","last_change":"2024-06-10T02:50:12.729392+0000","last_active":"2024-06-10T02:50:44.780303+0000","last_peered":"2024-06-10T02:50:44.780303+0000","last_clean":"2024-06-10T02:50:44.780303+0000","last_became_active":"2024-06-10T02:50:12.728866+0000","last_became_peered":"2024-06-10T02:50:12.728866+0000","last_unstale":"2024-06-10T02:50:44.780303+0000","last_undegraded":"2024-06-10T02:50:44.780303+0000","last_fullsized":"2024-06-10T02:50:44.780303+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:01:35.845184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.741921+0000","last_change":"2024-06-10T02:50:02.202303+0000","last_active":"2024-06-10T02:50:44.741921+0000","last_peered":"2024-06-10T02:50:44.741921+0000","last_clean":"2024-06-10T02:50:44.741921+0000","last_became_active":"2024-06-10T02:50:02.201998+0000","last_became_peered":"2024-06-10T02:50:02.201998+0000","last_unstale":"2024-06-10T02:50:44.741921+0000","last_undegraded":"2024-06-10T02:50:44.741921+0000","last_fullsized":"2024-06-10T02:50:44.741921+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:15:00.005693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"98'5","reported_seq":70,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:51:31.666116+0000","last_change":"2024-06-10T02:49:59.031513+0000","last_active":"2024-06-10T02:51:31.666116+0000","last_peered":"2024-06-10T02:51:31.666116+0000","last_clean":"2024-06-10T02:51:31.666116+0000","last_became_active":"2024-06-10T02:49:59.030965+0000","last_became_peered":"2024-06-10T02:49:59.030965+0000","last_unstale":"2024-06-10T02:51:31.666116+0000","last_undegraded":"2024-06-10T02:51:31.666116+0000","last_fullsized":"2024-06-10T02:51:31.666116+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:49:02.171578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":25,"num_read_kb":25,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"98'1","reported_seq":25,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.735962+0000","last_change":"2024-06-10T02:50:12.236334+0000","last_active":"2024-06-10T02:50:44.735962+0000","last_peered":"2024-06-10T02:50:44.735962+0000","last_clean":"2024-06-10T02:50:44.735962+0000","last_became_active":"2024-06-10T02:50:12.235820+0000","last_became_peered":"2024-06-10T02:50:12.235820+0000","last_unstale":"2024-06-10T02:50:44.735962+0000","last_undegraded":"2024-06-10T02:50:44.735962+0000","last_fullsized":"2024-06-10T02:50:44.735962+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:16:12.476881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.733920+0000","last_change":"2024-06-10T02:50:02.205146+0000","last_active":"2024-06-10T02:50:44.733920+0000","last_peered":"2024-06-10T02:50:44.733920+0000","last_clean":"2024-06-10T02:50:44.733920+0000","last_became_active":"2024-06-10T02:50:02.204785+0000","last_became_peered":"2024-06-10T02:50:02.204785+0000","last_unstale":"2024-06-10T02:50:44.733920+0000","last_undegraded":"2024-06-10T02:50:44.733920+0000","last_fullsized":"2024-06-10T02:50:44.733920+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T13:01:02.068989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"98'6","reported_seq":49,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:51:09.633782+0000","last_change":"2024-06-10T02:49:58.997654+0000","last_active":"2024-06-10T02:51:09.633782+0000","last_peered":"2024-06-10T02:51:09.633782+0000","last_clean":"2024-06-10T02:51:09.633782+0000","last_became_active":"2024-06-10T02:49:58.997294+0000","last_became_peered":"2024-06-10T02:49:58.997294+0000","last_unstale":"2024-06-10T02:51:09.633782+0000","last_undegraded":"2024-06-10T02:51:09.633782+0000","last_fullsized":"2024-06-10T02:51:09.633782+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:53:14.763198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":362,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.735090+0000","last_change":"2024-06-10T02:48:34.008201+0000","last_active":"2024-06-10T02:50:44.735090+0000","last_peered":"2024-06-10T02:50:44.735090+0000","last_clean":"2024-06-10T02:50:44.735090+0000","last_became_active":"2024-06-10T02:48:34.007633+0000","last_became_peered":"2024-06-10T02:48:34.007633+0000","last_unstale":"2024-06-10T02:50:44.735090+0000","last_undegraded":"2024-06-10T02:50:44.735090+0000","last_fullsized":"2024-06-10T02:50:44.735090+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:45:07.444823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:45:07.444823+0000","last_clean_scrub_stamp":"2024-06-10T02:45:07.444823+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:56:58.159559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":31,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.782130+0000","last_change":"2024-06-10T02:50:44.782130+0000","last_active":"2024-06-10T02:50:44.782130+0000","last_peered":"2024-06-10T02:50:44.782130+0000","last_clean":"2024-06-10T02:50:44.782130+0000","last_became_active":"2024-06-10T02:50:44.781530+0000","last_became_peered":"2024-06-10T02:50:44.781530+0000","last_unstale":"2024-06-10T02:50:44.782130+0000","last_undegraded":"2024-06-10T02:50:44.782130+0000","last_fullsized":"2024-06-10T02:50:44.782130+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:43:23.826207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,11,6,10],"acting":[5,11,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.779624+0000","last_change":"2024-06-10T02:50:02.213666+0000","last_active":"2024-06-10T02:50:44.779624+0000","last_peered":"2024-06-10T02:50:44.779624+0000","last_clean":"2024-06-10T02:50:44.779624+0000","last_became_active":"2024-06-10T02:50:02.213194+0000","last_became_peered":"2024-06-10T02:50:02.213194+0000","last_unstale":"2024-06-10T02:50:44.779624+0000","last_undegraded":"2024-06-10T02:50:44.779624+0000","last_fullsized":"2024-06-10T02:50:44.779624+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:22:55.905412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"98'4","reported_seq":44,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:51:06.614416+0000","last_change":"2024-06-10T02:49:58.997618+0000","last_active":"2024-06-10T02:51:06.614416+0000","last_peered":"2024-06-10T02:51:06.614416+0000","last_clean":"2024-06-10T02:51:06.614416+0000","last_became_active":"2024-06-10T02:49:58.997304+0000","last_became_peered":"2024-06-10T02:49:58.997304+0000","last_unstale":"2024-06-10T02:51:06.614416+0000","last_undegraded":"2024-06-10T02:51:06.614416+0000","last_fullsized":"2024-06-10T02:51:06.614416+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:48:59.520991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.780121+0000","last_change":"2024-06-10T02:50:12.236357+0000","last_active":"2024-06-10T02:50:44.780121+0000","last_peered":"2024-06-10T02:50:44.780121+0000","last_clean":"2024-06-10T02:50:44.780121+0000","last_became_active":"2024-06-10T02:50:12.235915+0000","last_became_peered":"2024-06-10T02:50:12.235915+0000","last_unstale":"2024-06-10T02:50:44.780121+0000","last_undegraded":"2024-06-10T02:50:44.780121+0000","last_fullsized":"2024-06-10T02:50:44.780121+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:20:06.287907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"98'4","reported_seq":45,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:51:08.627507+0000","last_change":"2024-06-10T02:49:59.031725+0000","last_active":"2024-06-10T02:51:08.627507+0000","last_peered":"2024-06-10T02:51:08.627507+0000","last_clean":"2024-06-10T02:51:08.627507+0000","last_became_active":"2024-06-10T02:49:59.031296+0000","last_became_peered":"2024-06-10T02:49:59.031296+0000","last_unstale":"2024-06-10T02:51:08.627507+0000","last_undegraded":"2024-06-10T02:51:08.627507+0000","last_fullsized":"2024-06-10T02:51:08.627507+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T13:17:58.571696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.779944+0000","last_change":"2024-06-10T02:50:02.201820+0000","last_active":"2024-06-10T02:50:44.779944+0000","last_peered":"2024-06-10T02:50:44.779944+0000","last_clean":"2024-06-10T02:50:44.779944+0000","last_became_active":"2024-06-10T02:50:02.201478+0000","last_became_peered":"2024-06-10T02:50:02.201478+0000","last_unstale":"2024-06-10T02:50:44.779944+0000","last_undegraded":"2024-06-10T02:50:44.779944+0000","last_fullsized":"2024-06-10T02:50:44.779944+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:57:25.963926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.733729+0000","last_change":"2024-06-10T02:50:12.729418+0000","last_active":"2024-06-10T02:50:44.733729+0000","last_peered":"2024-06-10T02:50:44.733729+0000","last_clean":"2024-06-10T02:50:44.733729+0000","last_became_active":"2024-06-10T02:50:12.728692+0000","last_became_peered":"2024-06-10T02:50:12.728692+0000","last_unstale":"2024-06-10T02:50:44.733729+0000","last_undegraded":"2024-06-10T02:50:44.733729+0000","last_fullsized":"2024-06-10T02:50:44.733729+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:25:00.472396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"98'15","reported_seq":55,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:51:29.212545+0000","last_change":"2024-06-10T02:49:58.996391+0000","last_active":"2024-06-10T02:51:29.212545+0000","last_peered":"2024-06-10T02:51:29.212545+0000","last_clean":"2024-06-10T02:51:29.212545+0000","last_became_active":"2024-06-10T02:49:58.996033+0000","last_became_peered":"2024-06-10T02:49:58.996033+0000","last_unstale":"2024-06-10T02:51:29.212545+0000","last_undegraded":"2024-06-10T02:51:29.212545+0000","last_fullsized":"2024-06-10T02:51:29.212545+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:03:34.568969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.734506+0000","last_change":"2024-06-10T02:50:02.197974+0000","last_active":"2024-06-10T02:50:44.734506+0000","last_peered":"2024-06-10T02:50:44.734506+0000","last_clean":"2024-06-10T02:50:44.734506+0000","last_became_active":"2024-06-10T02:50:02.197614+0000","last_became_peered":"2024-06-10T02:50:02.197614+0000","last_unstale":"2024-06-10T02:50:44.734506+0000","last_undegraded":"2024-06-10T02:50:44.734506+0000","last_fullsized":"2024-06-10T02:50:44.734506+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:36:55.575808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.737015+0000","last_change":"2024-06-10T02:50:12.234239+0000","last_active":"2024-06-10T02:50:44.737015+0000","last_peered":"2024-06-10T02:50:44.737015+0000","last_clean":"2024-06-10T02:50:44.737015+0000","last_became_active":"2024-06-10T02:50:12.233768+0000","last_became_peered":"2024-06-10T02:50:12.233768+0000","last_unstale":"2024-06-10T02:50:44.737015+0000","last_undegraded":"2024-06-10T02:50:44.737015+0000","last_fullsized":"2024-06-10T02:50:44.737015+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:51:44.230031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"98'7","reported_seq":48,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:51:08.628668+0000","last_change":"2024-06-10T02:49:59.192765+0000","last_active":"2024-06-10T02:51:08.628668+0000","last_peered":"2024-06-10T02:51:08.628668+0000","last_clean":"2024-06-10T02:51:08.628668+0000","last_became_active":"2024-06-10T02:49:59.192446+0000","last_became_peered":"2024-06-10T02:49:59.192446+0000","last_unstale":"2024-06-10T02:51:08.628668+0000","last_undegraded":"2024-06-10T02:51:08.628668+0000","last_fullsized":"2024-06-10T02:51:08.628668+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:53:51.862019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":40,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.781209+0000","last_change":"2024-06-10T02:50:44.781209+0000","last_active":"2024-06-10T02:50:44.781209+0000","last_peered":"2024-06-10T02:50:44.781209+0000","last_clean":"2024-06-10T02:50:44.781209+0000","last_became_active":"2024-06-10T02:50:44.780811+0000","last_became_peered":"2024-06-10T02:50:44.780811+0000","last_unstale":"2024-06-10T02:50:44.781209+0000","last_undegraded":"2024-06-10T02:50:44.781209+0000","last_fullsized":"2024-06-10T02:50:44.781209+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:12:40.663565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.780246+0000","last_change":"2024-06-10T02:50:12.730269+0000","last_active":"2024-06-10T02:50:44.780246+0000","last_peered":"2024-06-10T02:50:44.780246+0000","last_clean":"2024-06-10T02:50:44.780246+0000","last_became_active":"2024-06-10T02:50:12.729574+0000","last_became_peered":"2024-06-10T02:50:12.729574+0000","last_unstale":"2024-06-10T02:50:44.780246+0000","last_undegraded":"2024-06-10T02:50:44.780246+0000","last_fullsized":"2024-06-10T02:50:44.780246+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:51:16.235284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"98'2","reported_seq":48,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:51:05.606777+0000","last_change":"2024-06-10T02:50:44.782421+0000","last_active":"2024-06-10T02:51:05.606777+0000","last_peered":"2024-06-10T02:51:05.606777+0000","last_clean":"2024-06-10T02:51:05.606777+0000","last_became_active":"2024-06-10T02:50:44.781881+0000","last_became_peered":"2024-06-10T02:50:44.781881+0000","last_unstale":"2024-06-10T02:51:05.606777+0000","last_undegraded":"2024-06-10T02:51:05.606777+0000","last_fullsized":"2024-06-10T02:51:05.606777+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T14:41:58.771256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.739369+0000","last_change":"2024-06-10T02:50:02.197761+0000","last_active":"2024-06-10T02:50:44.739369+0000","last_peered":"2024-06-10T02:50:44.739369+0000","last_clean":"2024-06-10T02:50:44.739369+0000","last_became_active":"2024-06-10T02:50:02.197395+0000","last_became_peered":"2024-06-10T02:50:02.197395+0000","last_unstale":"2024-06-10T02:50:44.739369+0000","last_undegraded":"2024-06-10T02:50:44.739369+0000","last_fullsized":"2024-06-10T02:50:44.739369+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:46:37.537280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.738803+0000","last_change":"2024-06-10T02:50:12.241506+0000","last_active":"2024-06-10T02:50:44.738803+0000","last_peered":"2024-06-10T02:50:44.738803+0000","last_clean":"2024-06-10T02:50:44.738803+0000","last_became_active":"2024-06-10T02:50:12.240325+0000","last_became_peered":"2024-06-10T02:50:12.240325+0000","last_unstale":"2024-06-10T02:50:44.738803+0000","last_undegraded":"2024-06-10T02:50:44.738803+0000","last_fullsized":"2024-06-10T02:50:44.738803+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T13:45:41.869418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"98'7","reported_seq":47,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:51:08.627173+0000","last_change":"2024-06-10T02:49:58.987863+0000","last_active":"2024-06-10T02:51:08.627173+0000","last_peered":"2024-06-10T02:51:08.627173+0000","last_clean":"2024-06-10T02:51:08.627173+0000","last_became_active":"2024-06-10T02:49:58.987388+0000","last_became_peered":"2024-06-10T02:49:58.987388+0000","last_unstale":"2024-06-10T02:51:08.627173+0000","last_undegraded":"2024-06-10T02:51:08.627173+0000","last_fullsized":"2024-06-10T02:51:08.627173+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:29:20.340301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.780386+0000","last_change":"2024-06-10T02:50:02.200699+0000","last_active":"2024-06-10T02:50:44.780386+0000","last_peered":"2024-06-10T02:50:44.780386+0000","last_clean":"2024-06-10T02:50:44.780386+0000","last_became_active":"2024-06-10T02:50:02.200278+0000","last_became_peered":"2024-06-10T02:50:02.200278+0000","last_unstale":"2024-06-10T02:50:44.780386+0000","last_undegraded":"2024-06-10T02:50:44.780386+0000","last_fullsized":"2024-06-10T02:50:44.780386+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:26:36.236312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"98'1","reported_seq":25,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.780107+0000","last_change":"2024-06-10T02:50:12.236027+0000","last_active":"2024-06-10T02:50:44.780107+0000","last_peered":"2024-06-10T02:50:44.780107+0000","last_clean":"2024-06-10T02:50:44.780107+0000","last_became_active":"2024-06-10T02:50:12.235629+0000","last_became_peered":"2024-06-10T02:50:12.235629+0000","last_unstale":"2024-06-10T02:50:44.780107+0000","last_undegraded":"2024-06-10T02:50:44.780107+0000","last_fullsized":"2024-06-10T02:50:44.780107+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:58:48.218806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"98'3","reported_seq":43,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:51:07.620025+0000","last_change":"2024-06-10T02:49:58.998496+0000","last_active":"2024-06-10T02:51:07.620025+0000","last_peered":"2024-06-10T02:51:07.620025+0000","last_clean":"2024-06-10T02:51:07.620025+0000","last_became_active":"2024-06-10T02:49:58.997260+0000","last_became_peered":"2024-06-10T02:49:58.997260+0000","last_unstale":"2024-06-10T02:51:07.620025+0000","last_undegraded":"2024-06-10T02:51:07.620025+0000","last_fullsized":"2024-06-10T02:51:07.620025+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T14:22:04.658943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.734285+0000","last_change":"2024-06-10T02:50:02.201300+0000","last_active":"2024-06-10T02:50:44.734285+0000","last_peered":"2024-06-10T02:50:44.734285+0000","last_clean":"2024-06-10T02:50:44.734285+0000","last_became_active":"2024-06-10T02:50:02.200988+0000","last_became_peered":"2024-06-10T02:50:02.200988+0000","last_unstale":"2024-06-10T02:50:44.734285+0000","last_undegraded":"2024-06-10T02:50:44.734285+0000","last_fullsized":"2024-06-10T02:50:44.734285+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:11:12.962048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.780080+0000","last_change":"2024-06-10T02:50:12.728869+0000","last_active":"2024-06-10T02:50:44.780080+0000","last_peered":"2024-06-10T02:50:44.780080+0000","last_clean":"2024-06-10T02:50:44.780080+0000","last_became_active":"2024-06-10T02:50:12.728189+0000","last_became_peered":"2024-06-10T02:50:12.728189+0000","last_unstale":"2024-06-10T02:50:44.780080+0000","last_undegraded":"2024-06-10T02:50:44.780080+0000","last_fullsized":"2024-06-10T02:50:44.780080+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:08:36.624814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"98'4","reported_seq":45,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:51:05.605975+0000","last_change":"2024-06-10T02:49:59.000460+0000","last_active":"2024-06-10T02:51:05.605975+0000","last_peered":"2024-06-10T02:51:05.605975+0000","last_clean":"2024-06-10T02:51:05.605975+0000","last_became_active":"2024-06-10T02:49:59.000075+0000","last_became_peered":"2024-06-10T02:49:59.000075+0000","last_unstale":"2024-06-10T02:51:05.605975+0000","last_undegraded":"2024-06-10T02:51:05.605975+0000","last_fullsized":"2024-06-10T02:51:05.605975+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:06:27.935745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.735511+0000","last_change":"2024-06-10T02:50:02.210131+0000","last_active":"2024-06-10T02:50:44.735511+0000","last_peered":"2024-06-10T02:50:44.735511+0000","last_clean":"2024-06-10T02:50:44.735511+0000","last_became_active":"2024-06-10T02:50:02.209781+0000","last_became_peered":"2024-06-10T02:50:02.209781+0000","last_unstale":"2024-06-10T02:50:44.735511+0000","last_undegraded":"2024-06-10T02:50:44.735511+0000","last_fullsized":"2024-06-10T02:50:44.735511+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:27:39.966620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.738433+0000","last_change":"2024-06-10T02:50:12.236017+0000","last_active":"2024-06-10T02:50:44.738433+0000","last_peered":"2024-06-10T02:50:44.738433+0000","last_clean":"2024-06-10T02:50:44.738433+0000","last_became_active":"2024-06-10T02:50:12.235553+0000","last_became_peered":"2024-06-10T02:50:12.235553+0000","last_unstale":"2024-06-10T02:50:44.738433+0000","last_undegraded":"2024-06-10T02:50:44.738433+0000","last_fullsized":"2024-06-10T02:50:44.738433+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:23:55.437994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"98'6","reported_seq":76,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:51:34.671507+0000","last_change":"2024-06-10T02:49:59.192864+0000","last_active":"2024-06-10T02:51:34.671507+0000","last_peered":"2024-06-10T02:51:34.671507+0000","last_clean":"2024-06-10T02:51:34.671507+0000","last_became_active":"2024-06-10T02:49:59.192440+0000","last_became_peered":"2024-06-10T02:49:59.192440+0000","last_unstale":"2024-06-10T02:51:34.671507+0000","last_undegraded":"2024-06-10T02:51:34.671507+0000","last_fullsized":"2024-06-10T02:51:34.671507+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:36:05.868538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1560,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":30,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.736321+0000","last_change":"2024-06-10T02:50:02.205791+0000","last_active":"2024-06-10T02:50:44.736321+0000","last_peered":"2024-06-10T02:50:44.736321+0000","last_clean":"2024-06-10T02:50:44.736321+0000","last_became_active":"2024-06-10T02:50:02.205483+0000","last_became_peered":"2024-06-10T02:50:02.205483+0000","last_unstale":"2024-06-10T02:50:44.736321+0000","last_undegraded":"2024-06-10T02:50:44.736321+0000","last_fullsized":"2024-06-10T02:50:44.736321+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:33:25.875423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.736342+0000","last_change":"2024-06-10T02:50:12.239294+0000","last_active":"2024-06-10T02:50:44.736342+0000","last_peered":"2024-06-10T02:50:44.736342+0000","last_clean":"2024-06-10T02:50:44.736342+0000","last_became_active":"2024-06-10T02:50:12.238778+0000","last_became_peered":"2024-06-10T02:50:12.238778+0000","last_unstale":"2024-06-10T02:50:44.736342+0000","last_undegraded":"2024-06-10T02:50:44.736342+0000","last_fullsized":"2024-06-10T02:50:44.736342+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:26:09.203623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"98'3","reported_seq":44,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:51:08.627523+0000","last_change":"2024-06-10T02:49:58.989680+0000","last_active":"2024-06-10T02:51:08.627523+0000","last_peered":"2024-06-10T02:51:08.627523+0000","last_clean":"2024-06-10T02:51:08.627523+0000","last_became_active":"2024-06-10T02:49:58.989265+0000","last_became_peered":"2024-06-10T02:49:58.989265+0000","last_unstale":"2024-06-10T02:51:08.627523+0000","last_undegraded":"2024-06-10T02:51:08.627523+0000","last_fullsized":"2024-06-10T02:51:08.627523+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:36:34.370177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":40,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.743369+0000","last_change":"2024-06-10T02:50:44.743369+0000","last_active":"2024-06-10T02:50:44.743369+0000","last_peered":"2024-06-10T02:50:44.743369+0000","last_clean":"2024-06-10T02:50:44.743369+0000","last_became_active":"2024-06-10T02:50:44.743013+0000","last_became_peered":"2024-06-10T02:50:44.743013+0000","last_unstale":"2024-06-10T02:50:44.743369+0000","last_undegraded":"2024-06-10T02:50:44.743369+0000","last_fullsized":"2024-06-10T02:50:44.743369+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:54:43.728943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.740774+0000","last_change":"2024-06-10T02:50:12.235197+0000","last_active":"2024-06-10T02:50:44.740774+0000","last_peered":"2024-06-10T02:50:44.740774+0000","last_clean":"2024-06-10T02:50:44.740774+0000","last_became_active":"2024-06-10T02:50:12.234791+0000","last_became_peered":"2024-06-10T02:50:12.234791+0000","last_unstale":"2024-06-10T02:50:44.740774+0000","last_undegraded":"2024-06-10T02:50:44.740774+0000","last_fullsized":"2024-06-10T02:50:44.740774+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:09:11.164236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"98'5","reported_seq":45,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:51:08.625605+0000","last_change":"2024-06-10T02:49:58.996759+0000","last_active":"2024-06-10T02:51:08.625605+0000","last_peered":"2024-06-10T02:51:08.625605+0000","last_clean":"2024-06-10T02:51:08.625605+0000","last_became_active":"2024-06-10T02:49:58.996434+0000","last_became_peered":"2024-06-10T02:49:58.996434+0000","last_unstale":"2024-06-10T02:51:08.625605+0000","last_undegraded":"2024-06-10T02:51:08.625605+0000","last_fullsized":"2024-06-10T02:51:08.625605+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:09:35.327797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.736138+0000","last_change":"2024-06-10T02:50:02.201676+0000","last_active":"2024-06-10T02:50:44.736138+0000","last_peered":"2024-06-10T02:50:44.736138+0000","last_clean":"2024-06-10T02:50:44.736138+0000","last_became_active":"2024-06-10T02:50:02.201335+0000","last_became_peered":"2024-06-10T02:50:02.201335+0000","last_unstale":"2024-06-10T02:50:44.736138+0000","last_undegraded":"2024-06-10T02:50:44.736138+0000","last_fullsized":"2024-06-10T02:50:44.736138+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:49:47.038946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.780397+0000","last_change":"2024-06-10T02:50:02.203462+0000","last_active":"2024-06-10T02:50:44.780397+0000","last_peered":"2024-06-10T02:50:44.780397+0000","last_clean":"2024-06-10T02:50:44.780397+0000","last_became_active":"2024-06-10T02:50:02.203115+0000","last_became_peered":"2024-06-10T02:50:02.203115+0000","last_unstale":"2024-06-10T02:50:44.780397+0000","last_undegraded":"2024-06-10T02:50:44.780397+0000","last_fullsized":"2024-06-10T02:50:44.780397+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T13:18:12.400596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.737470+0000","last_change":"2024-06-10T02:50:02.205648+0000","last_active":"2024-06-10T02:50:44.737470+0000","last_peered":"2024-06-10T02:50:44.737470+0000","last_clean":"2024-06-10T02:50:44.737470+0000","last_became_active":"2024-06-10T02:50:02.205306+0000","last_became_peered":"2024-06-10T02:50:02.205306+0000","last_unstale":"2024-06-10T02:50:44.737470+0000","last_undegraded":"2024-06-10T02:50:44.737470+0000","last_fullsized":"2024-06-10T02:50:44.737470+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:22:55.020644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.737092+0000","last_change":"2024-06-10T02:50:02.198601+0000","last_active":"2024-06-10T02:50:44.737092+0000","last_peered":"2024-06-10T02:50:44.737092+0000","last_clean":"2024-06-10T02:50:44.737092+0000","last_became_active":"2024-06-10T02:50:02.198103+0000","last_became_peered":"2024-06-10T02:50:02.198103+0000","last_unstale":"2024-06-10T02:50:44.737092+0000","last_undegraded":"2024-06-10T02:50:44.737092+0000","last_fullsized":"2024-06-10T02:50:44.737092+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:56:28.033482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.736597+0000","last_change":"2024-06-10T02:50:02.202638+0000","last_active":"2024-06-10T02:50:44.736597+0000","last_peered":"2024-06-10T02:50:44.736597+0000","last_clean":"2024-06-10T02:50:44.736597+0000","last_became_active":"2024-06-10T02:50:02.202249+0000","last_became_peered":"2024-06-10T02:50:02.202249+0000","last_unstale":"2024-06-10T02:50:44.736597+0000","last_undegraded":"2024-06-10T02:50:44.736597+0000","last_fullsized":"2024-06-10T02:50:44.736597+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:46:29.424164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.779919+0000","last_change":"2024-06-10T02:50:02.213526+0000","last_active":"2024-06-10T02:50:44.779919+0000","last_peered":"2024-06-10T02:50:44.779919+0000","last_clean":"2024-06-10T02:50:44.779919+0000","last_became_active":"2024-06-10T02:50:02.213226+0000","last_became_peered":"2024-06-10T02:50:02.213226+0000","last_unstale":"2024-06-10T02:50:44.779919+0000","last_undegraded":"2024-06-10T02:50:44.779919+0000","last_fullsized":"2024-06-10T02:50:44.779919+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:01:56.846340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.780333+0000","last_change":"2024-06-10T02:50:02.206583+0000","last_active":"2024-06-10T02:50:44.780333+0000","last_peered":"2024-06-10T02:50:44.780333+0000","last_clean":"2024-06-10T02:50:44.780333+0000","last_became_active":"2024-06-10T02:50:02.206227+0000","last_became_peered":"2024-06-10T02:50:02.206227+0000","last_unstale":"2024-06-10T02:50:44.780333+0000","last_undegraded":"2024-06-10T02:50:44.780333+0000","last_fullsized":"2024-06-10T02:50:44.780333+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:25:07.246288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.737668+0000","last_change":"2024-06-10T02:50:02.206348+0000","last_active":"2024-06-10T02:50:44.737668+0000","last_peered":"2024-06-10T02:50:44.737668+0000","last_clean":"2024-06-10T02:50:44.737668+0000","last_became_active":"2024-06-10T02:50:02.206020+0000","last_became_peered":"2024-06-10T02:50:02.206020+0000","last_unstale":"2024-06-10T02:50:44.737668+0000","last_undegraded":"2024-06-10T02:50:44.737668+0000","last_fullsized":"2024-06-10T02:50:44.737668+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:02:26.619113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.734577+0000","last_change":"2024-06-10T02:50:02.199405+0000","last_active":"2024-06-10T02:50:44.734577+0000","last_peered":"2024-06-10T02:50:44.734577+0000","last_clean":"2024-06-10T02:50:44.734577+0000","last_became_active":"2024-06-10T02:50:02.199049+0000","last_became_peered":"2024-06-10T02:50:02.199049+0000","last_unstale":"2024-06-10T02:50:44.734577+0000","last_undegraded":"2024-06-10T02:50:44.734577+0000","last_fullsized":"2024-06-10T02:50:44.734577+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:33:04.091160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.741248+0000","last_change":"2024-06-10T02:50:02.202094+0000","last_active":"2024-06-10T02:50:44.741248+0000","last_peered":"2024-06-10T02:50:44.741248+0000","last_clean":"2024-06-10T02:50:44.741248+0000","last_became_active":"2024-06-10T02:50:02.201751+0000","last_became_peered":"2024-06-10T02:50:02.201751+0000","last_unstale":"2024-06-10T02:50:44.741248+0000","last_undegraded":"2024-06-10T02:50:44.741248+0000","last_fullsized":"2024-06-10T02:50:44.741248+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:52:53.946326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.734003+0000","last_change":"2024-06-10T02:50:02.199630+0000","last_active":"2024-06-10T02:50:44.734003+0000","last_peered":"2024-06-10T02:50:44.734003+0000","last_clean":"2024-06-10T02:50:44.734003+0000","last_became_active":"2024-06-10T02:50:02.199320+0000","last_became_peered":"2024-06-10T02:50:02.199320+0000","last_unstale":"2024-06-10T02:50:44.734003+0000","last_undegraded":"2024-06-10T02:50:44.734003+0000","last_fullsized":"2024-06-10T02:50:44.734003+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:17:28.761862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.738713+0000","last_change":"2024-06-10T02:50:02.201190+0000","last_active":"2024-06-10T02:50:44.738713+0000","last_peered":"2024-06-10T02:50:44.738713+0000","last_clean":"2024-06-10T02:50:44.738713+0000","last_became_active":"2024-06-10T02:50:02.200574+0000","last_became_peered":"2024-06-10T02:50:02.200574+0000","last_unstale":"2024-06-10T02:50:44.738713+0000","last_undegraded":"2024-06-10T02:50:44.738713+0000","last_fullsized":"2024-06-10T02:50:44.738713+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:02:07.636218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.780030+0000","last_change":"2024-06-10T02:50:02.202374+0000","last_active":"2024-06-10T02:50:44.780030+0000","last_peered":"2024-06-10T02:50:44.780030+0000","last_clean":"2024-06-10T02:50:44.780030+0000","last_became_active":"2024-06-10T02:50:02.201974+0000","last_became_peered":"2024-06-10T02:50:02.201974+0000","last_unstale":"2024-06-10T02:50:44.780030+0000","last_undegraded":"2024-06-10T02:50:44.780030+0000","last_fullsized":"2024-06-10T02:50:44.780030+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:45:34.977926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T02:50:44.780218+0000","last_change":"2024-06-10T02:50:02.209332+0000","last_active":"2024-06-10T02:50:44.780218+0000","last_peered":"2024-06-10T02:50:44.780218+0000","last_clean":"2024-06-10T02:50:44.780218+0000","last_became_active":"2024-06-10T02:50:02.208175+0000","last_became_peered":"2024-06-10T02:50:02.208175+0000","last_unstale":"2024-06-10T02:50:44.780218+0000","last_undegraded":"2024-06-10T02:50:44.780218+0000","last_fullsized":"2024-06-10T02:50:44.780218+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:44:40.857918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":284028,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":70637,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":310,"num_read_kb":234,"num_write":251,"num_write_kb":145,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":573440,"data_stored":207646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":215,"ondisk_log_size":215,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351010,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32236,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712916,"statfs":{"total":95995035648,"available":95962025984,"internally_reserved":0,"allocated":1085440,"data_stored":555042,"data_compressed":26507,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612648,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32384,"kb_used_data":1208,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712768,"statfs":{"total":95995035648,"available":95961874432,"internally_reserved":0,"allocated":1236992,"data_stored":1204513,"data_compressed":62624,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1599,"internal_metadata":31914433},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776173,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36424,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708728,"statfs":{"total":95995035648,"available":95957737472,"internally_reserved":0,"allocated":1114112,"data_stored":558678,"data_compressed":26507,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037809,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28088,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717064,"statfs":{"total":95995035648,"available":95966273536,"internally_reserved":0,"allocated":1097728,"data_stored":554288,"data_compressed":26507,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299446,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28148,"kb_used_data":1132,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717004,"statfs":{"total":95995035648,"available":95966212096,"internally_reserved":0,"allocated":1159168,"data_stored":1152236,"data_compressed":62624,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462972,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36396,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708756,"statfs":{"total":95995035648,"available":95957766144,"internally_reserved":0,"allocated":1085440,"data_stored":550476,"data_compressed":26507,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574933,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36460,"kb_used_data":1124,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708692,"statfs":{"total":95995035648,"available":95957700608,"internally_reserved":0,"allocated":1150976,"data_stored":609896,"data_compressed":26507,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738458,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28100,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717052,"statfs":{"total":95995035648,"available":95966261248,"internally_reserved":0,"allocated":1110016,"data_stored":551932,"data_compressed":26507,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411409,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28160,"kb_used_data":1144,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716992,"statfs":{"total":95995035648,"available":95966199808,"internally_reserved":0,"allocated":1171456,"data_stored":610506,"data_compressed":26507,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149770,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36408,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708744,"statfs":{"total":95995035648,"available":95957753856,"internally_reserved":0,"allocated":1097728,"data_stored":549794,"data_compressed":26507,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888134,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28092,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717060,"statfs":{"total":95995035648,"available":95966269440,"internally_reserved":0,"allocated":1101824,"data_stored":555222,"data_compressed":26507,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724609,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28156,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716996,"statfs":{"total":95995035648,"available":95966203904,"internally_reserved":0,"allocated":1167360,"data_stored":1147038,"data_compressed":62624,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34481,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34481,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34481,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36117,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36117,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36117,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36117,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36117,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":60532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":65407,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":64781,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-06-10T02:51:38.714 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-06-10T02:51:38.715 INFO:tasks.cephadm:Setting up client nodes... 2024-06-10T02:51:38.715 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-06-10T02:51:38.725 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:51:38 smithi052 ceph-mon[32362]: pgmap v355: 129 pgs: 129 active+clean; 646 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 3.5 KiB/s wr, 14 op/s 2024-06-10T02:51:38.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:51:38 smithi188 ceph-mon[36599]: pgmap v355: 129 pgs: 129 active+clean; 646 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 3.5 KiB/s wr, 14 op/s 2024-06-10T02:51:38.969 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:51:39.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:51:38 smithi120 ceph-mon[36634]: pgmap v355: 129 pgs: 129 active+clean; 646 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 3.5 KiB/s wr, 14 op/s 2024-06-10T02:51:39.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:51:39 smithi188 ceph-mon[36599]: from='client.14817 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-10T02:51:39.975 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:51:39 smithi052 ceph-mon[32362]: from='client.14817 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-10T02:51:40.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:51:39 smithi120 ceph-mon[36634]: from='client.14817 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-10T02:51:40.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:40.708+0000 7f1bb3608700 1 -- 172.21.15.52:0/1199982540 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1bac1040e0 msgr2=0x7f1bac10a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:51:40.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:40.708+0000 7f1bb3608700 1 --2- 172.21.15.52:0/1199982540 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1bac1040e0 0x7f1bac10a990 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f1ba400b0d0 tx=0x7f1ba400b3e0 comp rx=0 tx=0).stop 2024-06-10T02:51:40.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:40.708+0000 7f1bb3608700 1 -- 172.21.15.52:0/1199982540 shutdown_connections 2024-06-10T02:51:40.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:40.708+0000 7f1bb3608700 1 --2- 172.21.15.52:0/1199982540 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1bac1040e0 0x7f1bac10a990 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:51:40.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:40.708+0000 7f1bb3608700 1 --2- 172.21.15.52:0/1199982540 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1bac102540 0x7f1bac102960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:51:40.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:40.708+0000 7f1bb3608700 1 --2- 172.21.15.52:0/1199982540 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1bac103740 0x7f1bac103ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:51:40.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:40.708+0000 7f1bb3608700 1 -- 172.21.15.52:0/1199982540 >> 172.21.15.52:0/1199982540 conn(0x7f1bac0fdae0 msgr2=0x7f1bac0fff20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:51:40.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:40.708+0000 7f1bb3608700 1 -- 172.21.15.52:0/1199982540 shutdown_connections 2024-06-10T02:51:40.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:40.708+0000 7f1bb3608700 1 -- 172.21.15.52:0/1199982540 wait complete. 2024-06-10T02:51:40.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:40.708+0000 7f1bb3608700 1 Processor -- start 2024-06-10T02:51:40.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:40.708+0000 7f1bb3608700 1 -- start start 2024-06-10T02:51:40.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:40.709+0000 7f1bb3608700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1bac102540 0x7f1bac198580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:51:40.710 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:40.709+0000 7f1bb3608700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1bac103740 0x7f1bac198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:51:40.710 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:40.709+0000 7f1bb3608700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1bac1040e0 0x7f1bac19fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:51:40.710 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:40.709+0000 7f1bb3608700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f1bac10e180 con 0x7f1bac1040e0 2024-06-10T02:51:40.710 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:40.709+0000 7f1bb3608700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1bac10e000 con 0x7f1bac102540 2024-06-10T02:51:40.710 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:40.709+0000 7f1bb3608700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f1bac10e300 con 0x7f1bac103740 2024-06-10T02:51:40.710 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:40.709+0000 7f1bb1ba5700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1bac1040e0 0x7f1bac19fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:51:40.710 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:40.709+0000 7f1bb1ba5700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1bac1040e0 0x7f1bac19fb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37934/0 (socket says 172.21.15.52:37934) 2024-06-10T02:51:40.710 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:40.709+0000 7f1bb1ba5700 1 -- 172.21.15.52:0/400806011 learned_addr learned my addr 172.21.15.52:0/400806011 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:51:40.710 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:40.709+0000 7f1bb13a4700 1 --2- 172.21.15.52:0/400806011 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1bac102540 0x7f1bac198580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:51:40.710 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:40.709+0000 7f1bb0ba3700 1 --2- 172.21.15.52:0/400806011 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1bac103740 0x7f1bac198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:51:40.710 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:40.709+0000 7f1bb1ba5700 1 -- 172.21.15.52:0/400806011 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1bac103740 msgr2=0x7f1bac198ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:51:40.710 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:40.709+0000 7f1bb1ba5700 1 --2- 172.21.15.52:0/400806011 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1bac103740 0x7f1bac198ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:51:40.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:40.709+0000 7f1bb1ba5700 1 -- 172.21.15.52:0/400806011 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1bac102540 msgr2=0x7f1bac198580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:51:40.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:40.709+0000 7f1bb1ba5700 1 --2- 172.21.15.52:0/400806011 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1bac102540 0x7f1bac198580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:51:40.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:40.709+0000 7f1bb1ba5700 1 -- 172.21.15.52:0/400806011 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bac1a01e0 con 0x7f1bac1040e0 2024-06-10T02:51:40.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:40.709+0000 7f1bb0ba3700 1 --2- 172.21.15.52:0/400806011 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1bac103740 0x7f1bac198ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T02:51:40.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:40.709+0000 7f1bb1ba5700 1 --2- 172.21.15.52:0/400806011 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1bac1040e0 0x7f1bac19fb40 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f1ba4006070 tx=0x7f1ba4007d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:51:40.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:40.710+0000 7f1ba27fc700 1 -- 172.21.15.52:0/400806011 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ba4003c00 con 0x7f1bac1040e0 2024-06-10T02:51:40.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:40.710+0000 7f1bb3608700 1 -- 172.21.15.52:0/400806011 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1bac1a0470 con 0x7f1bac1040e0 2024-06-10T02:51:40.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:40.710+0000 7f1ba27fc700 1 -- 172.21.15.52:0/400806011 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ba4003da0 con 0x7f1bac1040e0 2024-06-10T02:51:40.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:40.710+0000 7f1ba27fc700 1 -- 172.21.15.52:0/400806011 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ba40054b0 con 0x7f1bac1040e0 2024-06-10T02:51:40.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:40.710+0000 7f1bb3608700 1 -- 172.21.15.52:0/400806011 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1bac1a0a20 con 0x7f1bac1040e0 2024-06-10T02:51:40.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:40.711+0000 7f1bb3608700 1 -- 172.21.15.52:0/400806011 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1bac050a00 con 0x7f1bac1040e0 2024-06-10T02:51:40.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:40.712+0000 7f1ba27fc700 1 -- 172.21.15.52:0/400806011 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f1ba4005650 con 0x7f1bac1040e0 2024-06-10T02:51:40.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:40.712+0000 7f1ba27fc700 1 --2- 172.21.15.52:0/400806011 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f1b980a2d60 0x7f1b980a5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:51:40.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:40.713+0000 7f1bb13a4700 1 --2- 172.21.15.52:0/400806011 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f1b980a2d60 0x7f1b980a5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:51:40.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:40.713+0000 7f1ba27fc700 1 -- 172.21.15.52:0/400806011 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9092+0+0 (secure 0 0 0) 0x7f1ba400f020 con 0x7f1bac1040e0 2024-06-10T02:51:40.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:40.713+0000 7f1bb13a4700 1 --2- 172.21.15.52:0/400806011 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f1b980a2d60 0x7f1b980a5220 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f1bac1035a0 tx=0x7f1ba800a3b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:51:40.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:40.716+0000 7f1ba27fc700 1 -- 172.21.15.52:0/400806011 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f1ba402b020 con 0x7f1bac1040e0 2024-06-10T02:51:40.725 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:51:40 smithi052 ceph-mon[32362]: pgmap v356: 129 pgs: 129 active+clean; 646 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 767 B/s wr, 10 op/s 2024-06-10T02:51:40.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:51:40 smithi188 ceph-mon[36599]: pgmap v356: 129 pgs: 129 active+clean; 646 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 767 B/s wr, 10 op/s 2024-06-10T02:51:41.010 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:41.011+0000 7f1bb3608700 1 -- 172.21.15.52:0/400806011 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f1bac02dca0 con 0x7f1bac1040e0 2024-06-10T02:51:41.012 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:41.013+0000 7f1ba27fc700 1 -- 172.21.15.52:0/400806011 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7f1ba409d7c0 con 0x7f1bac1040e0 2024-06-10T02:51:41.012 INFO:teuthology.orchestra.run.smithi052.stdout:[client.0] 2024-06-10T02:51:41.012 INFO:teuthology.orchestra.run.smithi052.stdout: key = AQA9amZmNcnFABAA8s6dO7/O+4rqjCKnFwi6YQ== 2024-06-10T02:51:41.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:41.017+0000 7f1bb3608700 1 -- 172.21.15.52:0/400806011 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f1b980a2d60 msgr2=0x7f1b980a5220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:51:41.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:41.017+0000 7f1bb3608700 1 --2- 172.21.15.52:0/400806011 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f1b980a2d60 0x7f1b980a5220 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f1bac1035a0 tx=0x7f1ba800a3b0 comp rx=0 tx=0).stop 2024-06-10T02:51:41.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:41.017+0000 7f1bb3608700 1 -- 172.21.15.52:0/400806011 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1bac1040e0 msgr2=0x7f1bac19fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:51:41.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:41.017+0000 7f1bb3608700 1 --2- 172.21.15.52:0/400806011 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1bac1040e0 0x7f1bac19fb40 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f1ba4006070 tx=0x7f1ba4007d90 comp rx=0 tx=0).stop 2024-06-10T02:51:41.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:41.017+0000 7f1bb3608700 1 -- 172.21.15.52:0/400806011 shutdown_connections 2024-06-10T02:51:41.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:41.017+0000 7f1bb3608700 1 --2- 172.21.15.52:0/400806011 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f1b980a2d60 0x7f1b980a5220 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:51:41.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:41.017+0000 7f1bb3608700 1 --2- 172.21.15.52:0/400806011 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1bac1040e0 0x7f1bac19fb40 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:51:41.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:41.017+0000 7f1bb3608700 1 --2- 172.21.15.52:0/400806011 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1bac102540 0x7f1bac198580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:51:41.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:41.017+0000 7f1bb3608700 1 --2- 172.21.15.52:0/400806011 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1bac103740 0x7f1bac198ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:51:41.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:41.017+0000 7f1bb3608700 1 -- 172.21.15.52:0/400806011 >> 172.21.15.52:0/400806011 conn(0x7f1bac0fdae0 msgr2=0x7f1bac106970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:51:41.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:41.017+0000 7f1bb3608700 1 -- 172.21.15.52:0/400806011 shutdown_connections 2024-06-10T02:51:41.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:41.017+0000 7f1bb3608700 1 -- 172.21.15.52:0/400806011 wait complete. 2024-06-10T02:51:41.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:51:40 smithi120 ceph-mon[36634]: pgmap v356: 129 pgs: 129 active+clean; 646 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 767 B/s wr, 10 op/s 2024-06-10T02:51:41.753 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-06-10T02:51:41.753 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-06-10T02:51:41.753 DEBUG:teuthology.orchestra.run.smithi052:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-06-10T02:51:41.836 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-06-10T02:51:41.836 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-06-10T02:51:41.837 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph mgr dump --format=json 2024-06-10T02:51:41.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:51:41 smithi188 ceph-mon[36599]: from='client.? 172.21.15.52:0/400806011' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-06-10T02:51:41.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:51:41 smithi188 ceph-mon[36599]: from='client.? 172.21.15.52:0/400806011' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-06-10T02:51:41.975 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:51:41 smithi052 ceph-mon[32362]: from='client.? 172.21.15.52:0/400806011' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-06-10T02:51:41.976 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:51:41 smithi052 ceph-mon[32362]: from='client.? 172.21.15.52:0/400806011' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-06-10T02:51:42.130 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:51:42.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:51:41 smithi120 ceph-mon[36634]: from='client.? 172.21.15.52:0/400806011' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-06-10T02:51:42.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:51:41 smithi120 ceph-mon[36634]: from='client.? 172.21.15.52:0/400806011' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-06-10T02:51:42.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:51:42 smithi188 ceph-mon[36599]: pgmap v357: 129 pgs: 129 active+clean; 646 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 767 B/s wr, 9 op/s 2024-06-10T02:51:42.975 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:51:42 smithi052 ceph-mon[32362]: pgmap v357: 129 pgs: 129 active+clean; 646 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 767 B/s wr, 9 op/s 2024-06-10T02:51:43.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:51:42 smithi120 ceph-mon[36634]: pgmap v357: 129 pgs: 129 active+clean; 646 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 767 B/s wr, 9 op/s 2024-06-10T02:51:43.868 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:43.868+0000 7f7ea4677700 1 -- 172.21.15.52:0/2384294997 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7e9c108580 msgr2=0x7f7e9c108960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:51:43.868 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:43.868+0000 7f7ea4677700 1 --2- 172.21.15.52:0/2384294997 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7e9c108580 0x7f7e9c108960 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f7e980099c0 tx=0x7f7e98009cd0 comp rx=0 tx=0).stop 2024-06-10T02:51:43.868 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:43.868+0000 7f7ea4677700 1 -- 172.21.15.52:0/2384294997 shutdown_connections 2024-06-10T02:51:43.868 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:43.868+0000 7f7ea4677700 1 --2- 172.21.15.52:0/2384294997 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7e9c06b620 0x7f7e9c10f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:51:43.868 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:43.868+0000 7f7ea4677700 1 --2- 172.21.15.52:0/2384294997 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7e9c108580 0x7f7e9c108960 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:51:43.868 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:43.868+0000 7f7ea4677700 1 --2- 172.21.15.52:0/2384294997 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7e9c06ac60 0x7f7e9c06b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:51:43.868 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:43.868+0000 7f7ea4677700 1 -- 172.21.15.52:0/2384294997 >> 172.21.15.52:0/2384294997 conn(0x7f7e9c078430 msgr2=0x7f7e9c078840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:51:43.869 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:43.868+0000 7f7ea4677700 1 -- 172.21.15.52:0/2384294997 shutdown_connections 2024-06-10T02:51:43.869 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:43.868+0000 7f7ea4677700 1 -- 172.21.15.52:0/2384294997 wait complete. 2024-06-10T02:51:43.869 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:43.868+0000 7f7ea4677700 1 Processor -- start 2024-06-10T02:51:43.869 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:43.869+0000 7f7ea4677700 1 -- start start 2024-06-10T02:51:43.869 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:43.869+0000 7f7ea4677700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7e9c06ac60 0x7f7e9c101400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:51:43.869 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:43.869+0000 7f7ea4677700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7e9c06b620 0x7f7e9c101940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:51:43.869 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:43.869+0000 7f7ea4677700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7e9c108580 0x7f7e9c101e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:51:43.869 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:43.869+0000 7f7ea4677700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f7e9c1127c0 con 0x7f7e9c06ac60 2024-06-10T02:51:43.869 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:43.869+0000 7f7ea4677700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f7e9c112640 con 0x7f7e9c108580 2024-06-10T02:51:43.869 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:43.869+0000 7f7ea4677700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f7e9c112940 con 0x7f7e9c06b620 2024-06-10T02:51:43.869 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:43.869+0000 7f7ea2413700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7e9c06ac60 0x7f7e9c101400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:51:43.869 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:43.869+0000 7f7ea2413700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7e9c06ac60 0x7f7e9c101400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52374/0 (socket says 172.21.15.52:52374) 2024-06-10T02:51:43.869 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:43.869+0000 7f7ea2413700 1 -- 172.21.15.52:0/1798982007 learned_addr learned my addr 172.21.15.52:0/1798982007 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:51:43.870 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:43.869+0000 7f7ea2c14700 1 --2- 172.21.15.52:0/1798982007 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7e9c108580 0x7f7e9c101e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:51:43.870 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:43.869+0000 7f7ea1c12700 1 --2- 172.21.15.52:0/1798982007 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7e9c06b620 0x7f7e9c101940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:51:43.870 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:43.869+0000 7f7ea2413700 1 -- 172.21.15.52:0/1798982007 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7e9c06b620 msgr2=0x7f7e9c101940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:51:43.870 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:43.869+0000 7f7ea2413700 1 --2- 172.21.15.52:0/1798982007 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7e9c06b620 0x7f7e9c101940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:51:43.870 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:43.869+0000 7f7ea2413700 1 -- 172.21.15.52:0/1798982007 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7e9c108580 msgr2=0x7f7e9c101e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:51:43.870 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:43.869+0000 7f7ea2413700 1 --2- 172.21.15.52:0/1798982007 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7e9c108580 0x7f7e9c101e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:51:43.870 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:43.869+0000 7f7ea2413700 1 -- 172.21.15.52:0/1798982007 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e9c1a7120 con 0x7f7e9c06ac60 2024-06-10T02:51:43.870 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:43.869+0000 7f7ea1c12700 1 --2- 172.21.15.52:0/1798982007 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7e9c06b620 0x7f7e9c101940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T02:51:43.870 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:43.869+0000 7f7ea2413700 1 --2- 172.21.15.52:0/1798982007 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7e9c06ac60 0x7f7e9c101400 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f7e98006010 tx=0x7f7e98009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:51:43.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:43.870+0000 7f7e937fe700 1 -- 172.21.15.52:0/1798982007 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e98027070 con 0x7f7e9c06ac60 2024-06-10T02:51:43.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:43.870+0000 7f7e937fe700 1 -- 172.21.15.52:0/1798982007 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e98004050 con 0x7f7e9c06ac60 2024-06-10T02:51:43.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:43.870+0000 7f7e937fe700 1 -- 172.21.15.52:0/1798982007 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e98022e10 con 0x7f7e9c06ac60 2024-06-10T02:51:43.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:43.870+0000 7f7ea4677700 1 -- 172.21.15.52:0/1798982007 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e9c1a73b0 con 0x7f7e9c06ac60 2024-06-10T02:51:43.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:43.870+0000 7f7ea4677700 1 -- 172.21.15.52:0/1798982007 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e9c1a7800 con 0x7f7e9c06ac60 2024-06-10T02:51:43.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:43.871+0000 7f7ea4677700 1 -- 172.21.15.52:0/1798982007 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e9c10aa10 con 0x7f7e9c06ac60 2024-06-10T02:51:43.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:43.872+0000 7f7e937fe700 1 -- 172.21.15.52:0/1798982007 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f7e9802a070 con 0x7f7e9c06ac60 2024-06-10T02:51:43.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:43.872+0000 7f7e937fe700 1 --2- 172.21.15.52:0/1798982007 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f7e880a2cb0 0x7f7e880a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:51:43.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:43.873+0000 7f7ea1c12700 1 --2- 172.21.15.52:0/1798982007 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f7e880a2cb0 0x7f7e880a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:51:43.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:43.873+0000 7f7e937fe700 1 -- 172.21.15.52:0/1798982007 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9092+0+0 (secure 0 0 0) 0x7f7e980d0470 con 0x7f7e9c06ac60 2024-06-10T02:51:43.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:43.873+0000 7f7ea1c12700 1 --2- 172.21.15.52:0/1798982007 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f7e880a2cb0 0x7f7e880a5170 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f7e8c005fd0 tx=0x7f7e8c005ee0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:51:43.875 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:43.876+0000 7f7e937fe700 1 -- 172.21.15.52:0/1798982007 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f7e98013070 con 0x7f7e9c06ac60 2024-06-10T02:51:44.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:44.141+0000 7f7ea4677700 1 -- 172.21.15.52:0/1798982007 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f7e9c10a3f0 con 0x7f7e9c06ac60 2024-06-10T02:51:44.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:44.147+0000 7f7e937fe700 1 -- 172.21.15.52:0/1798982007 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+263201 (secure 0 0 0) 0x7f7e9809cfa0 con 0x7f7e9c06ac60 2024-06-10T02:51:44.149 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-10T02:51:44.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:44.150+0000 7f7ea4677700 1 -- 172.21.15.52:0/1798982007 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f7e880a2cb0 msgr2=0x7f7e880a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:51:44.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:44.150+0000 7f7ea4677700 1 --2- 172.21.15.52:0/1798982007 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f7e880a2cb0 0x7f7e880a5170 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f7e8c005fd0 tx=0x7f7e8c005ee0 comp rx=0 tx=0).stop 2024-06-10T02:51:44.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:44.150+0000 7f7ea4677700 1 -- 172.21.15.52:0/1798982007 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7e9c06ac60 msgr2=0x7f7e9c101400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:51:44.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:44.150+0000 7f7ea4677700 1 --2- 172.21.15.52:0/1798982007 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7e9c06ac60 0x7f7e9c101400 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f7e98006010 tx=0x7f7e98009f80 comp rx=0 tx=0).stop 2024-06-10T02:51:44.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:44.150+0000 7f7ea4677700 1 -- 172.21.15.52:0/1798982007 shutdown_connections 2024-06-10T02:51:44.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:44.150+0000 7f7ea4677700 1 --2- 172.21.15.52:0/1798982007 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7e9c108580 0x7f7e9c101e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:51:44.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:44.150+0000 7f7ea4677700 1 --2- 172.21.15.52:0/1798982007 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f7e880a2cb0 0x7f7e880a5170 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:51:44.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:44.150+0000 7f7ea4677700 1 --2- 172.21.15.52:0/1798982007 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7e9c06ac60 0x7f7e9c101400 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:51:44.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:44.150+0000 7f7ea4677700 1 --2- 172.21.15.52:0/1798982007 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7e9c06b620 0x7f7e9c101940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:51:44.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:44.150+0000 7f7ea4677700 1 -- 172.21.15.52:0/1798982007 >> 172.21.15.52:0/1798982007 conn(0x7f7e9c078430 msgr2=0x7f7e9c0fff20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:51:44.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:44.150+0000 7f7ea4677700 1 -- 172.21.15.52:0/1798982007 shutdown_connections 2024-06-10T02:51:44.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:44.150+0000 7f7ea4677700 1 -- 172.21.15.52:0/1798982007 wait complete. 2024-06-10T02:51:44.951 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:51:44 smithi052 ceph-mon[32362]: pgmap v358: 129 pgs: 129 active+clean; 646 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 682 B/s wr, 12 op/s 2024-06-10T02:51:44.951 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:51:44 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.9", "id": [5, 8]}]: dispatch 2024-06-10T02:51:44.952 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:51:44 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]: dispatch 2024-06-10T02:51:44.952 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:51:44 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]: dispatch 2024-06-10T02:51:44.952 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:51:44 smithi052 ceph-mon[32362]: from='client.? 172.21.15.52:0/1798982007' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-06-10T02:51:44.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:51:44 smithi188 ceph-mon[36599]: pgmap v358: 129 pgs: 129 active+clean; 646 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 682 B/s wr, 12 op/s 2024-06-10T02:51:44.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:51:44 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.9", "id": [5, 8]}]: dispatch 2024-06-10T02:51:44.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:51:44 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]: dispatch 2024-06-10T02:51:44.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:51:44 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]: dispatch 2024-06-10T02:51:44.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:51:44 smithi188 ceph-mon[36599]: from='client.? 172.21.15.52:0/1798982007' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-06-10T02:51:44.982 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":17,"flags":0,"active_gid":14150,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":3214763969},{"type":"v1","addr":"172.21.15.52:6801","nonce":3214763969}]},"active_addr":"172.21.15.52:6801/3214763969","active_change":"2024-06-10T02:42:42.872425+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24119,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.52:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.52:0","nonce":2021062140}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.52:0","nonce":2646182408}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.52:0","nonce":2736987751}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.52:0","nonce":1363984630}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.52:0","nonce":1019387507}]}]} 2024-06-10T02:51:44.987 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-06-10T02:51:44.987 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-06-10T02:51:44.987 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-10T02:51:45.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:51:44 smithi120 ceph-mon[36634]: pgmap v358: 129 pgs: 129 active+clean; 646 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 682 B/s wr, 12 op/s 2024-06-10T02:51:45.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:51:44 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.9", "id": [5, 8]}]: dispatch 2024-06-10T02:51:45.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:51:44 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]: dispatch 2024-06-10T02:51:45.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:51:44 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]: dispatch 2024-06-10T02:51:45.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:51:44 smithi120 ceph-mon[36634]: from='client.? 172.21.15.52:0/1798982007' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-06-10T02:51:45.248 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:51:45.976 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:51:45 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.9", "id": [5, 8]}]': finished 2024-06-10T02:51:45.976 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:51:45 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]': finished 2024-06-10T02:51:45.976 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:51:45 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]': finished 2024-06-10T02:51:45.976 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:51:45 smithi052 ceph-mon[32362]: osdmap e99: 12 total, 12 up, 12 in 2024-06-10T02:51:46.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:51:45 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.9", "id": [5, 8]}]': finished 2024-06-10T02:51:46.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:51:45 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]': finished 2024-06-10T02:51:46.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:51:45 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]': finished 2024-06-10T02:51:46.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:51:45 smithi120 ceph-mon[36634]: osdmap e99: 12 total, 12 up, 12 in 2024-06-10T02:51:46.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:51:45 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.9", "id": [5, 8]}]': finished 2024-06-10T02:51:46.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:51:45 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]': finished 2024-06-10T02:51:46.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:51:45 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]': finished 2024-06-10T02:51:46.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:51:45 smithi188 ceph-mon[36599]: osdmap e99: 12 total, 12 up, 12 in 2024-06-10T02:51:46.726 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:51:46 smithi052 ceph-mon[32362]: pgmap v360: 129 pgs: 129 active+clean; 646 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 511 B/s wr, 10 op/s 2024-06-10T02:51:46.726 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:51:46 smithi052 ceph-mon[32362]: osdmap e100: 12 total, 12 up, 12 in 2024-06-10T02:51:47.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:47.024+0000 7f4a4acc7700 1 -- 172.21.15.52:0/3773658538 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4a441013d0 msgr2=0x7f4a44101830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:51:47.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:47.024+0000 7f4a4acc7700 1 --2- 172.21.15.52:0/3773658538 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4a441013d0 0x7f4a44101830 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f4a38009990 tx=0x7f4a38009ca0 comp rx=0 tx=0).stop 2024-06-10T02:51:47.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:47.024+0000 7f4a4acc7700 1 -- 172.21.15.52:0/3773658538 shutdown_connections 2024-06-10T02:51:47.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:47.024+0000 7f4a4acc7700 1 --2- 172.21.15.52:0/3773658538 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4a44101d70 0x7f4a4410e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:51:47.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:47.024+0000 7f4a4acc7700 1 --2- 172.21.15.52:0/3773658538 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4a44107410 0x7f4a441077f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:51:47.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:47.024+0000 7f4a4acc7700 1 --2- 172.21.15.52:0/3773658538 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4a441013d0 0x7f4a44101830 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:51:47.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:47.024+0000 7f4a4acc7700 1 -- 172.21.15.52:0/3773658538 >> 172.21.15.52:0/3773658538 conn(0x7f4a440782c0 msgr2=0x7f4a440786d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:51:47.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:47.024+0000 7f4a4acc7700 1 -- 172.21.15.52:0/3773658538 shutdown_connections 2024-06-10T02:51:47.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:47.024+0000 7f4a4acc7700 1 -- 172.21.15.52:0/3773658538 wait complete. 2024-06-10T02:51:47.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:47.025+0000 7f4a4acc7700 1 Processor -- start 2024-06-10T02:51:47.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:47.025+0000 7f4a4acc7700 1 -- start start 2024-06-10T02:51:47.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:47.025+0000 7f4a4acc7700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4a441013d0 0x7f4a44199b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:51:47.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:47.025+0000 7f4a4acc7700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4a44101d70 0x7f4a4419a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:51:47.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:47.025+0000 7f4a4acc7700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4a44107410 0x7f4a441a4b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:51:47.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:47.025+0000 7f4a4acc7700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f4a44111670 con 0x7f4a441013d0 2024-06-10T02:51:47.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:47.025+0000 7f4a4acc7700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f4a441114f0 con 0x7f4a44101d70 2024-06-10T02:51:47.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:47.025+0000 7f4a4acc7700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f4a441117f0 con 0x7f4a44107410 2024-06-10T02:51:47.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:47.025+0000 7f4a48a63700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4a441013d0 0x7f4a44199b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:51:47.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:47.025+0000 7f4a48a63700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4a441013d0 0x7f4a44199b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52388/0 (socket says 172.21.15.52:52388) 2024-06-10T02:51:47.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:47.025+0000 7f4a48a63700 1 -- 172.21.15.52:0/2465030976 learned_addr learned my addr 172.21.15.52:0/2465030976 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:51:47.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:47.026+0000 7f4a43fff700 1 --2- 172.21.15.52:0/2465030976 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4a44101d70 0x7f4a4419a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:51:47.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:47.026+0000 7f4a49264700 1 --2- 172.21.15.52:0/2465030976 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4a44107410 0x7f4a441a4b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:51:47.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:47.026+0000 7f4a48a63700 1 -- 172.21.15.52:0/2465030976 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4a44107410 msgr2=0x7f4a441a4b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:51:47.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:47.026+0000 7f4a48a63700 1 --2- 172.21.15.52:0/2465030976 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4a44107410 0x7f4a441a4b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:51:47.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:47.026+0000 7f4a48a63700 1 -- 172.21.15.52:0/2465030976 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4a44101d70 msgr2=0x7f4a4419a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:51:47.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:47.026+0000 7f4a48a63700 1 --2- 172.21.15.52:0/2465030976 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4a44101d70 0x7f4a4419a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:51:47.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:47.026+0000 7f4a48a63700 1 -- 172.21.15.52:0/2465030976 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a441a50d0 con 0x7f4a441013d0 2024-06-10T02:51:47.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:47.026+0000 7f4a48a63700 1 --2- 172.21.15.52:0/2465030976 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4a441013d0 0x7f4a44199b10 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f4a3400d950 tx=0x7f4a3400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:51:47.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:47.026+0000 7f4a41ffb700 1 -- 172.21.15.52:0/2465030976 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a34016070 con 0x7f4a441013d0 2024-06-10T02:51:47.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:47.026+0000 7f4a41ffb700 1 -- 172.21.15.52:0/2465030976 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a34009e20 con 0x7f4a441013d0 2024-06-10T02:51:47.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:47.026+0000 7f4a41ffb700 1 -- 172.21.15.52:0/2465030976 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a3400f620 con 0x7f4a441013d0 2024-06-10T02:51:47.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:47.026+0000 7f4a4acc7700 1 -- 172.21.15.52:0/2465030976 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a441a5300 con 0x7f4a441013d0 2024-06-10T02:51:47.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:47.027+0000 7f4a4acc7700 1 -- 172.21.15.52:0/2465030976 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a441a5810 con 0x7f4a441013d0 2024-06-10T02:51:47.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:47.028+0000 7f4a4acc7700 1 -- 172.21.15.52:0/2465030976 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a44102ac0 con 0x7f4a441013d0 2024-06-10T02:51:47.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:47.029+0000 7f4a41ffb700 1 -- 172.21.15.52:0/2465030976 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f4a34019070 con 0x7f4a441013d0 2024-06-10T02:51:47.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:47.029+0000 7f4a41ffb700 1 --2- 172.21.15.52:0/2465030976 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f4a2c0a2cb0 0x7f4a2c0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:51:47.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:47.029+0000 7f4a43fff700 1 --2- 172.21.15.52:0/2465030976 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f4a2c0a2cb0 0x7f4a2c0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:51:47.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:47.029+0000 7f4a41ffb700 1 -- 172.21.15.52:0/2465030976 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f4a340cf0b0 con 0x7f4a441013d0 2024-06-10T02:51:47.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:47.030+0000 7f4a43fff700 1 --2- 172.21.15.52:0/2465030976 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f4a2c0a2cb0 0x7f4a2c0a5170 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f4a3800b5c0 tx=0x7f4a38005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:51:47.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:47.034+0000 7f4a41ffb700 1 -- 172.21.15.52:0/2465030976 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f4a3409bb00 con 0x7f4a441013d0 2024-06-10T02:51:47.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:51:46 smithi120 ceph-mon[36634]: pgmap v360: 129 pgs: 129 active+clean; 646 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 511 B/s wr, 10 op/s 2024-06-10T02:51:47.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:51:46 smithi120 ceph-mon[36634]: osdmap e100: 12 total, 12 up, 12 in 2024-06-10T02:51:47.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:51:46 smithi188 ceph-mon[36599]: pgmap v360: 129 pgs: 129 active+clean; 646 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 511 B/s wr, 10 op/s 2024-06-10T02:51:47.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:51:46 smithi188 ceph-mon[36599]: osdmap e100: 12 total, 12 up, 12 in 2024-06-10T02:51:47.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:47.241+0000 7f4a4acc7700 1 -- 172.21.15.52:0/2465030976 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4a4406b210 con 0x7f4a441013d0 2024-06-10T02:51:47.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:47.243+0000 7f4a41ffb700 1 -- 172.21.15.52:0/2465030976 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v100) ==== 74+0+24739 (secure 0 0 0) 0x7f4a3409b400 con 0x7f4a441013d0 2024-06-10T02:51:47.242 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-10T02:51:47.242 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":100,"fsid":"d8676e9e-26d2-11ef-bc9b-c7b262605968","created":"2024-06-10T02:41:43.076837+0000","modified":"2024-06-10T02:51:45.708105+0000","last_up_change":"2024-06-10T02:48:57.297829+0000","last_in_change":"2024-06-10T02:48:36.865802+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-10T02:45:07.394690+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-10T02:49:57.926838+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-10T02:50:01.130393+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-10T02:50:10.655588+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e03c0691-cc5a-497a-bbc2-ce18c337f074","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":4255833408},{"type":"v1","addr":"172.21.15.52:6803","nonce":4255833408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":4255833408},{"type":"v1","addr":"172.21.15.52:6805","nonce":4255833408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":4255833408},{"type":"v1","addr":"172.21.15.52:6809","nonce":4255833408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":4255833408},{"type":"v1","addr":"172.21.15.52:6807","nonce":4255833408}]},"public_addr":"172.21.15.52:6803/4255833408","cluster_addr":"172.21.15.52:6805/4255833408","heartbeat_back_addr":"172.21.15.52:6809/4255833408","heartbeat_front_addr":"172.21.15.52:6807/4255833408","state":["exists","up"]},{"osd":1,"uuid":"a24d8da8-c5ec-44d7-ab30-56f6ce3efda7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":4116540053},{"type":"v1","addr":"172.21.15.120:6801","nonce":4116540053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":4116540053},{"type":"v1","addr":"172.21.15.120:6803","nonce":4116540053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":4116540053},{"type":"v1","addr":"172.21.15.120:6807","nonce":4116540053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":4116540053},{"type":"v1","addr":"172.21.15.120:6805","nonce":4116540053}]},"public_addr":"172.21.15.120:6801/4116540053","cluster_addr":"172.21.15.120:6803/4116540053","heartbeat_back_addr":"172.21.15.120:6807/4116540053","heartbeat_front_addr":"172.21.15.120:6805/4116540053","state":["exists","up"]},{"osd":2,"uuid":"a7162106-ccbf-45bd-bc0e-051ee43f4214","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":2624326999},{"type":"v1","addr":"172.21.15.188:6801","nonce":2624326999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":2624326999},{"type":"v1","addr":"172.21.15.188:6803","nonce":2624326999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":2624326999},{"type":"v1","addr":"172.21.15.188:6807","nonce":2624326999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":2624326999},{"type":"v1","addr":"172.21.15.188:6805","nonce":2624326999}]},"public_addr":"172.21.15.188:6801/2624326999","cluster_addr":"172.21.15.188:6803/2624326999","heartbeat_back_addr":"172.21.15.188:6807/2624326999","heartbeat_front_addr":"172.21.15.188:6805/2624326999","state":["exists","up"]},{"osd":3,"uuid":"53680244-b49a-4120-b24a-9ad7a6341dc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":116897011},{"type":"v1","addr":"172.21.15.52:6811","nonce":116897011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":116897011},{"type":"v1","addr":"172.21.15.52:6813","nonce":116897011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":116897011},{"type":"v1","addr":"172.21.15.52:6817","nonce":116897011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":116897011},{"type":"v1","addr":"172.21.15.52:6815","nonce":116897011}]},"public_addr":"172.21.15.52:6811/116897011","cluster_addr":"172.21.15.52:6813/116897011","heartbeat_back_addr":"172.21.15.52:6817/116897011","heartbeat_front_addr":"172.21.15.52:6815/116897011","state":["exists","up"]},{"osd":4,"uuid":"9f72f1f4-2ef3-49c5-8410-c108767d2ef5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":1320690621},{"type":"v1","addr":"172.21.15.120:6809","nonce":1320690621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":1320690621},{"type":"v1","addr":"172.21.15.120:6811","nonce":1320690621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":1320690621},{"type":"v1","addr":"172.21.15.120:6815","nonce":1320690621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":1320690621},{"type":"v1","addr":"172.21.15.120:6813","nonce":1320690621}]},"public_addr":"172.21.15.120:6809/1320690621","cluster_addr":"172.21.15.120:6811/1320690621","heartbeat_back_addr":"172.21.15.120:6815/1320690621","heartbeat_front_addr":"172.21.15.120:6813/1320690621","state":["exists","up"]},{"osd":5,"uuid":"001b5706-7f27-4fa4-9b69-969e6d765abc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3951947483},{"type":"v1","addr":"172.21.15.188:6809","nonce":3951947483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3951947483},{"type":"v1","addr":"172.21.15.188:6811","nonce":3951947483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3951947483},{"type":"v1","addr":"172.21.15.188:6815","nonce":3951947483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3951947483},{"type":"v1","addr":"172.21.15.188:6813","nonce":3951947483}]},"public_addr":"172.21.15.188:6809/3951947483","cluster_addr":"172.21.15.188:6811/3951947483","heartbeat_back_addr":"172.21.15.188:6815/3951947483","heartbeat_front_addr":"172.21.15.188:6813/3951947483","state":["exists","up"]},{"osd":6,"uuid":"140c903e-9b31-4a81-9ccd-2edc0a2decf9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":359084919},{"type":"v1","addr":"172.21.15.52:6819","nonce":359084919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":359084919},{"type":"v1","addr":"172.21.15.52:6821","nonce":359084919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":359084919},{"type":"v1","addr":"172.21.15.52:6825","nonce":359084919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":359084919},{"type":"v1","addr":"172.21.15.52:6823","nonce":359084919}]},"public_addr":"172.21.15.52:6819/359084919","cluster_addr":"172.21.15.52:6821/359084919","heartbeat_back_addr":"172.21.15.52:6825/359084919","heartbeat_front_addr":"172.21.15.52:6823/359084919","state":["exists","up"]},{"osd":7,"uuid":"57b735d4-5072-4ca2-a3ff-3763ad84f15a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":3737722011},{"type":"v1","addr":"172.21.15.120:6817","nonce":3737722011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6818","nonce":3737722011},{"type":"v1","addr":"172.21.15.120:6819","nonce":3737722011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":3737722011},{"type":"v1","addr":"172.21.15.120:6823","nonce":3737722011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":3737722011},{"type":"v1","addr":"172.21.15.120:6821","nonce":3737722011}]},"public_addr":"172.21.15.120:6817/3737722011","cluster_addr":"172.21.15.120:6819/3737722011","heartbeat_back_addr":"172.21.15.120:6823/3737722011","heartbeat_front_addr":"172.21.15.120:6821/3737722011","state":["exists","up"]},{"osd":8,"uuid":"4ffbb43b-854a-4eac-99cf-7af89bc65409","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1444828441},{"type":"v1","addr":"172.21.15.188:6817","nonce":1444828441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1444828441},{"type":"v1","addr":"172.21.15.188:6819","nonce":1444828441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1444828441},{"type":"v1","addr":"172.21.15.188:6823","nonce":1444828441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1444828441},{"type":"v1","addr":"172.21.15.188:6821","nonce":1444828441}]},"public_addr":"172.21.15.188:6817/1444828441","cluster_addr":"172.21.15.188:6819/1444828441","heartbeat_back_addr":"172.21.15.188:6823/1444828441","heartbeat_front_addr":"172.21.15.188:6821/1444828441","state":["exists","up"]},{"osd":9,"uuid":"5bfd985c-8acc-4d00-8561-40bbd541c07b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":924440658},{"type":"v1","addr":"172.21.15.52:6827","nonce":924440658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":924440658},{"type":"v1","addr":"172.21.15.52:6829","nonce":924440658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":924440658},{"type":"v1","addr":"172.21.15.52:6833","nonce":924440658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":924440658},{"type":"v1","addr":"172.21.15.52:6831","nonce":924440658}]},"public_addr":"172.21.15.52:6827/924440658","cluster_addr":"172.21.15.52:6829/924440658","heartbeat_back_addr":"172.21.15.52:6833/924440658","heartbeat_front_addr":"172.21.15.52:6831/924440658","state":["exists","up"]},{"osd":10,"uuid":"0180cddf-e109-42cc-829a-185604276f02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6824","nonce":1531769083},{"type":"v1","addr":"172.21.15.120:6825","nonce":1531769083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":1531769083},{"type":"v1","addr":"172.21.15.120:6827","nonce":1531769083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":1531769083},{"type":"v1","addr":"172.21.15.120:6831","nonce":1531769083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6828","nonce":1531769083},{"type":"v1","addr":"172.21.15.120:6829","nonce":1531769083}]},"public_addr":"172.21.15.120:6825/1531769083","cluster_addr":"172.21.15.120:6827/1531769083","heartbeat_back_addr":"172.21.15.120:6831/1531769083","heartbeat_front_addr":"172.21.15.120:6829/1531769083","state":["exists","up"]},{"osd":11,"uuid":"73aec750-82d1-4800-b6a0-4e305912f9ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":3769050170},{"type":"v1","addr":"172.21.15.188:6825","nonce":3769050170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":3769050170},{"type":"v1","addr":"172.21.15.188:6827","nonce":3769050170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":3769050170},{"type":"v1","addr":"172.21.15.188:6831","nonce":3769050170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":3769050170},{"type":"v1","addr":"172.21.15.188:6829","nonce":3769050170}]},"public_addr":"172.21.15.188:6825/3769050170","cluster_addr":"172.21.15.188:6827/3769050170","heartbeat_back_addr":"172.21.15.188:6831/3769050170","heartbeat_front_addr":"172.21.15.188:6829/3769050170","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:44:16.455752+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:44:40.219735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:45:03.558899+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:45:29.155004+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:45:53.981098+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:46:19.412020+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:46:45.462640+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:47:11.484578+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:47:36.879054+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:48:03.392446+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:48:29.422173+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:48:55.523423+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":4}]},{"pgid":"3.6","mappings":[{"from":3,"to":7}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.9","mappings":[{"from":5,"to":8}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.52:0/3535271671":"2024-06-11T02:42:42.872312+0000","172.21.15.52:0/1092934380":"2024-06-11T02:42:42.872312+0000","172.21.15.52:6800/896317057":"2024-06-11T02:42:42.872312+0000","172.21.15.52:6801/3934479642":"2024-06-11T02:42:15.498118+0000","172.21.15.52:6800/3934479642":"2024-06-11T02:42:15.498118+0000","172.21.15.52:0/2759694132":"2024-06-11T02:42:15.498118+0000","172.21.15.52:0/2593665631":"2024-06-11T02:42:15.498118+0000","172.21.15.52:0/3151504634":"2024-06-11T02:42:15.498118+0000","172.21.15.52:6801/896317057":"2024-06-11T02:42:42.872312+0000","172.21.15.52:0/1254480982":"2024-06-11T02:42:42.872312+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-10T02:51:47.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:47.245+0000 7f4a4acc7700 1 -- 172.21.15.52:0/2465030976 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f4a2c0a2cb0 msgr2=0x7f4a2c0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:51:47.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:47.245+0000 7f4a4acc7700 1 --2- 172.21.15.52:0/2465030976 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f4a2c0a2cb0 0x7f4a2c0a5170 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f4a3800b5c0 tx=0x7f4a38005960 comp rx=0 tx=0).stop 2024-06-10T02:51:47.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:47.245+0000 7f4a4acc7700 1 -- 172.21.15.52:0/2465030976 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4a441013d0 msgr2=0x7f4a44199b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:51:47.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:47.245+0000 7f4a4acc7700 1 --2- 172.21.15.52:0/2465030976 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4a441013d0 0x7f4a44199b10 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f4a3400d950 tx=0x7f4a3400dc60 comp rx=0 tx=0).stop 2024-06-10T02:51:47.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:47.245+0000 7f4a4acc7700 1 -- 172.21.15.52:0/2465030976 shutdown_connections 2024-06-10T02:51:47.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:47.245+0000 7f4a4acc7700 1 --2- 172.21.15.52:0/2465030976 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4a44107410 0x7f4a441a4b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:51:47.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:47.245+0000 7f4a4acc7700 1 --2- 172.21.15.52:0/2465030976 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f4a2c0a2cb0 0x7f4a2c0a5170 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:51:47.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:47.245+0000 7f4a4acc7700 1 --2- 172.21.15.52:0/2465030976 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4a441013d0 0x7f4a44199b10 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:51:47.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:47.245+0000 7f4a4acc7700 1 --2- 172.21.15.52:0/2465030976 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4a44101d70 0x7f4a4419a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:51:47.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:47.245+0000 7f4a4acc7700 1 -- 172.21.15.52:0/2465030976 >> 172.21.15.52:0/2465030976 conn(0x7f4a440782c0 msgr2=0x7f4a44100100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:51:47.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:47.246+0000 7f4a4acc7700 1 -- 172.21.15.52:0/2465030976 shutdown_connections 2024-06-10T02:51:47.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:47.246+0000 7f4a4acc7700 1 -- 172.21.15.52:0/2465030976 wait complete. 2024-06-10T02:51:47.976 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:51:47 smithi052 ceph-mon[32362]: from='client.? 172.21.15.52:0/2465030976' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T02:51:48.024 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-06-10T02:51:48.024 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-10T02:51:48.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:51:47 smithi120 ceph-mon[36634]: from='client.? 172.21.15.52:0/2465030976' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T02:51:48.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:51:47 smithi188 ceph-mon[36599]: from='client.? 172.21.15.52:0/2465030976' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T02:51:48.284 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:51:48.975 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:51:48 smithi052 ceph-mon[32362]: pgmap v362: 129 pgs: 129 active+clean; 646 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s 2024-06-10T02:51:49.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:51:48 smithi120 ceph-mon[36634]: pgmap v362: 129 pgs: 129 active+clean; 646 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s 2024-06-10T02:51:49.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:51:48 smithi188 ceph-mon[36599]: pgmap v362: 129 pgs: 129 active+clean; 646 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s 2024-06-10T02:51:50.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:50.024+0000 7f3cb6d5b700 1 -- 172.21.15.52:0/4124845613 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3cb0107400 msgr2=0x7f3cb01077e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:51:50.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:50.024+0000 7f3cb6d5b700 1 --2- 172.21.15.52:0/4124845613 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3cb0107400 0x7f3cb01077e0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f3cac009990 tx=0x7f3cac009ca0 comp rx=0 tx=0).stop 2024-06-10T02:51:50.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:50.024+0000 7f3cb6d5b700 1 -- 172.21.15.52:0/4124845613 shutdown_connections 2024-06-10T02:51:50.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:50.024+0000 7f3cb6d5b700 1 --2- 172.21.15.52:0/4124845613 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3cb0101d50 0x7f3cb010e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:51:50.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:50.024+0000 7f3cb6d5b700 1 --2- 172.21.15.52:0/4124845613 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3cb0107400 0x7f3cb01077e0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:51:50.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:50.024+0000 7f3cb6d5b700 1 --2- 172.21.15.52:0/4124845613 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3cb0101390 0x7f3cb0101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:51:50.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:50.024+0000 7f3cb6d5b700 1 -- 172.21.15.52:0/4124845613 >> 172.21.15.52:0/4124845613 conn(0x7f3cb0078340 msgr2=0x7f3cb0078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:51:50.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:50.025+0000 7f3cb6d5b700 1 -- 172.21.15.52:0/4124845613 shutdown_connections 2024-06-10T02:51:50.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:50.025+0000 7f3cb6d5b700 1 -- 172.21.15.52:0/4124845613 wait complete. 2024-06-10T02:51:50.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:50.025+0000 7f3cb6d5b700 1 Processor -- start 2024-06-10T02:51:50.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:50.025+0000 7f3cb6d5b700 1 -- start start 2024-06-10T02:51:50.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:50.025+0000 7f3cb6d5b700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3cb0101390 0x7f3cb019ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:51:50.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:50.025+0000 7f3cb6d5b700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3cb0101d50 0x7f3cb019b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:51:50.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:50.025+0000 7f3cb6d5b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3cb0107400 0x7f3cb019f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:51:50.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:50.025+0000 7f3cb6d5b700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f3cb0111740 con 0x7f3cb0107400 2024-06-10T02:51:50.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:50.025+0000 7f3cb6d5b700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f3cb01115c0 con 0x7f3cb0101d50 2024-06-10T02:51:50.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:50.025+0000 7f3cb6d5b700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f3cb01118c0 con 0x7f3cb0101390 2024-06-10T02:51:50.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:50.026+0000 7f3cb52f8700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3cb0107400 0x7f3cb019f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:51:50.030 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:50.026+0000 7f3cb52f8700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3cb0107400 0x7f3cb019f710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52394/0 (socket says 172.21.15.52:52394) 2024-06-10T02:51:50.030 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:50.026+0000 7f3cb52f8700 1 -- 172.21.15.52:0/4291117128 learned_addr learned my addr 172.21.15.52:0/4291117128 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:51:50.030 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:50.026+0000 7f3ca7fff700 1 --2- 172.21.15.52:0/4291117128 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3cb0101d50 0x7f3cb019b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:51:50.030 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:50.026+0000 7f3cb4af7700 1 --2- 172.21.15.52:0/4291117128 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3cb0101390 0x7f3cb019ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:51:50.030 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:50.026+0000 7f3cb52f8700 1 -- 172.21.15.52:0/4291117128 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3cb0101390 msgr2=0x7f3cb019ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:51:50.030 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:50.026+0000 7f3cb52f8700 1 --2- 172.21.15.52:0/4291117128 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3cb0101390 0x7f3cb019ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:51:50.030 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:50.026+0000 7f3cb52f8700 1 -- 172.21.15.52:0/4291117128 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3cb0101d50 msgr2=0x7f3cb019b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:51:50.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:50.026+0000 7f3cb52f8700 1 --2- 172.21.15.52:0/4291117128 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3cb0101d50 0x7f3cb019b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:51:50.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:50.026+0000 7f3cb52f8700 1 -- 172.21.15.52:0/4291117128 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cb019fe50 con 0x7f3cb0107400 2024-06-10T02:51:50.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:50.026+0000 7f3cb4af7700 1 --2- 172.21.15.52:0/4291117128 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3cb0101390 0x7f3cb019ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T02:51:50.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:50.026+0000 7f3cb52f8700 1 --2- 172.21.15.52:0/4291117128 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3cb0107400 0x7f3cb019f710 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f3ca800b970 tx=0x7f3ca800bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:51:50.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:50.026+0000 7f3ca5ffb700 1 -- 172.21.15.52:0/4291117128 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ca8019070 con 0x7f3cb0107400 2024-06-10T02:51:50.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:50.026+0000 7f3ca5ffb700 1 -- 172.21.15.52:0/4291117128 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ca800ce10 con 0x7f3cb0107400 2024-06-10T02:51:50.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:50.026+0000 7f3cb6d5b700 1 -- 172.21.15.52:0/4291117128 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3cb01a0140 con 0x7f3cb0107400 2024-06-10T02:51:50.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:50.026+0000 7f3ca5ffb700 1 -- 172.21.15.52:0/4291117128 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ca8007370 con 0x7f3cb0107400 2024-06-10T02:51:50.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:50.027+0000 7f3cb6d5b700 1 -- 172.21.15.52:0/4291117128 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3cb01a79c0 con 0x7f3cb0107400 2024-06-10T02:51:50.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:50.028+0000 7f3cb6d5b700 1 -- 172.21.15.52:0/4291117128 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3cb0069180 con 0x7f3cb0107400 2024-06-10T02:51:50.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:50.029+0000 7f3ca5ffb700 1 -- 172.21.15.52:0/4291117128 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f3ca801c070 con 0x7f3cb0107400 2024-06-10T02:51:50.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:50.029+0000 7f3ca5ffb700 1 --2- 172.21.15.52:0/4291117128 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f3c980a2cb0 0x7f3c980a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:51:50.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:50.029+0000 7f3cb4af7700 1 --2- 172.21.15.52:0/4291117128 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f3c980a2cb0 0x7f3c980a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:51:50.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:50.030+0000 7f3ca5ffb700 1 -- 172.21.15.52:0/4291117128 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f3ca80ce7e0 con 0x7f3cb0107400 2024-06-10T02:51:50.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:50.030+0000 7f3cb4af7700 1 --2- 172.21.15.52:0/4291117128 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f3c980a2cb0 0x7f3c980a5170 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f3cac009750 tx=0x7f3cac00b540 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:51:50.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:50.034+0000 7f3ca5ffb700 1 -- 172.21.15.52:0/4291117128 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f3ca809b340 con 0x7f3cb0107400 2024-06-10T02:51:50.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:50.241+0000 7f3cb6d5b700 1 -- 172.21.15.52:0/4291117128 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3cb01a7bf0 con 0x7f3cb0107400 2024-06-10T02:51:50.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:50.243+0000 7f3ca5ffb700 1 -- 172.21.15.52:0/4291117128 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v100) ==== 74+0+24739 (secure 0 0 0) 0x7f3ca809ac40 con 0x7f3cb0107400 2024-06-10T02:51:50.242 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-10T02:51:50.243 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":100,"fsid":"d8676e9e-26d2-11ef-bc9b-c7b262605968","created":"2024-06-10T02:41:43.076837+0000","modified":"2024-06-10T02:51:45.708105+0000","last_up_change":"2024-06-10T02:48:57.297829+0000","last_in_change":"2024-06-10T02:48:36.865802+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-10T02:45:07.394690+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-10T02:49:57.926838+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-10T02:50:01.130393+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-10T02:50:10.655588+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e03c0691-cc5a-497a-bbc2-ce18c337f074","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":4255833408},{"type":"v1","addr":"172.21.15.52:6803","nonce":4255833408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":4255833408},{"type":"v1","addr":"172.21.15.52:6805","nonce":4255833408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":4255833408},{"type":"v1","addr":"172.21.15.52:6809","nonce":4255833408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":4255833408},{"type":"v1","addr":"172.21.15.52:6807","nonce":4255833408}]},"public_addr":"172.21.15.52:6803/4255833408","cluster_addr":"172.21.15.52:6805/4255833408","heartbeat_back_addr":"172.21.15.52:6809/4255833408","heartbeat_front_addr":"172.21.15.52:6807/4255833408","state":["exists","up"]},{"osd":1,"uuid":"a24d8da8-c5ec-44d7-ab30-56f6ce3efda7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":4116540053},{"type":"v1","addr":"172.21.15.120:6801","nonce":4116540053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":4116540053},{"type":"v1","addr":"172.21.15.120:6803","nonce":4116540053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":4116540053},{"type":"v1","addr":"172.21.15.120:6807","nonce":4116540053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":4116540053},{"type":"v1","addr":"172.21.15.120:6805","nonce":4116540053}]},"public_addr":"172.21.15.120:6801/4116540053","cluster_addr":"172.21.15.120:6803/4116540053","heartbeat_back_addr":"172.21.15.120:6807/4116540053","heartbeat_front_addr":"172.21.15.120:6805/4116540053","state":["exists","up"]},{"osd":2,"uuid":"a7162106-ccbf-45bd-bc0e-051ee43f4214","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":2624326999},{"type":"v1","addr":"172.21.15.188:6801","nonce":2624326999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":2624326999},{"type":"v1","addr":"172.21.15.188:6803","nonce":2624326999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":2624326999},{"type":"v1","addr":"172.21.15.188:6807","nonce":2624326999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":2624326999},{"type":"v1","addr":"172.21.15.188:6805","nonce":2624326999}]},"public_addr":"172.21.15.188:6801/2624326999","cluster_addr":"172.21.15.188:6803/2624326999","heartbeat_back_addr":"172.21.15.188:6807/2624326999","heartbeat_front_addr":"172.21.15.188:6805/2624326999","state":["exists","up"]},{"osd":3,"uuid":"53680244-b49a-4120-b24a-9ad7a6341dc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":116897011},{"type":"v1","addr":"172.21.15.52:6811","nonce":116897011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":116897011},{"type":"v1","addr":"172.21.15.52:6813","nonce":116897011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":116897011},{"type":"v1","addr":"172.21.15.52:6817","nonce":116897011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":116897011},{"type":"v1","addr":"172.21.15.52:6815","nonce":116897011}]},"public_addr":"172.21.15.52:6811/116897011","cluster_addr":"172.21.15.52:6813/116897011","heartbeat_back_addr":"172.21.15.52:6817/116897011","heartbeat_front_addr":"172.21.15.52:6815/116897011","state":["exists","up"]},{"osd":4,"uuid":"9f72f1f4-2ef3-49c5-8410-c108767d2ef5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":1320690621},{"type":"v1","addr":"172.21.15.120:6809","nonce":1320690621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":1320690621},{"type":"v1","addr":"172.21.15.120:6811","nonce":1320690621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":1320690621},{"type":"v1","addr":"172.21.15.120:6815","nonce":1320690621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":1320690621},{"type":"v1","addr":"172.21.15.120:6813","nonce":1320690621}]},"public_addr":"172.21.15.120:6809/1320690621","cluster_addr":"172.21.15.120:6811/1320690621","heartbeat_back_addr":"172.21.15.120:6815/1320690621","heartbeat_front_addr":"172.21.15.120:6813/1320690621","state":["exists","up"]},{"osd":5,"uuid":"001b5706-7f27-4fa4-9b69-969e6d765abc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3951947483},{"type":"v1","addr":"172.21.15.188:6809","nonce":3951947483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3951947483},{"type":"v1","addr":"172.21.15.188:6811","nonce":3951947483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3951947483},{"type":"v1","addr":"172.21.15.188:6815","nonce":3951947483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3951947483},{"type":"v1","addr":"172.21.15.188:6813","nonce":3951947483}]},"public_addr":"172.21.15.188:6809/3951947483","cluster_addr":"172.21.15.188:6811/3951947483","heartbeat_back_addr":"172.21.15.188:6815/3951947483","heartbeat_front_addr":"172.21.15.188:6813/3951947483","state":["exists","up"]},{"osd":6,"uuid":"140c903e-9b31-4a81-9ccd-2edc0a2decf9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":359084919},{"type":"v1","addr":"172.21.15.52:6819","nonce":359084919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":359084919},{"type":"v1","addr":"172.21.15.52:6821","nonce":359084919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":359084919},{"type":"v1","addr":"172.21.15.52:6825","nonce":359084919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":359084919},{"type":"v1","addr":"172.21.15.52:6823","nonce":359084919}]},"public_addr":"172.21.15.52:6819/359084919","cluster_addr":"172.21.15.52:6821/359084919","heartbeat_back_addr":"172.21.15.52:6825/359084919","heartbeat_front_addr":"172.21.15.52:6823/359084919","state":["exists","up"]},{"osd":7,"uuid":"57b735d4-5072-4ca2-a3ff-3763ad84f15a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":3737722011},{"type":"v1","addr":"172.21.15.120:6817","nonce":3737722011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6818","nonce":3737722011},{"type":"v1","addr":"172.21.15.120:6819","nonce":3737722011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":3737722011},{"type":"v1","addr":"172.21.15.120:6823","nonce":3737722011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":3737722011},{"type":"v1","addr":"172.21.15.120:6821","nonce":3737722011}]},"public_addr":"172.21.15.120:6817/3737722011","cluster_addr":"172.21.15.120:6819/3737722011","heartbeat_back_addr":"172.21.15.120:6823/3737722011","heartbeat_front_addr":"172.21.15.120:6821/3737722011","state":["exists","up"]},{"osd":8,"uuid":"4ffbb43b-854a-4eac-99cf-7af89bc65409","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1444828441},{"type":"v1","addr":"172.21.15.188:6817","nonce":1444828441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1444828441},{"type":"v1","addr":"172.21.15.188:6819","nonce":1444828441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1444828441},{"type":"v1","addr":"172.21.15.188:6823","nonce":1444828441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1444828441},{"type":"v1","addr":"172.21.15.188:6821","nonce":1444828441}]},"public_addr":"172.21.15.188:6817/1444828441","cluster_addr":"172.21.15.188:6819/1444828441","heartbeat_back_addr":"172.21.15.188:6823/1444828441","heartbeat_front_addr":"172.21.15.188:6821/1444828441","state":["exists","up"]},{"osd":9,"uuid":"5bfd985c-8acc-4d00-8561-40bbd541c07b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":924440658},{"type":"v1","addr":"172.21.15.52:6827","nonce":924440658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":924440658},{"type":"v1","addr":"172.21.15.52:6829","nonce":924440658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":924440658},{"type":"v1","addr":"172.21.15.52:6833","nonce":924440658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":924440658},{"type":"v1","addr":"172.21.15.52:6831","nonce":924440658}]},"public_addr":"172.21.15.52:6827/924440658","cluster_addr":"172.21.15.52:6829/924440658","heartbeat_back_addr":"172.21.15.52:6833/924440658","heartbeat_front_addr":"172.21.15.52:6831/924440658","state":["exists","up"]},{"osd":10,"uuid":"0180cddf-e109-42cc-829a-185604276f02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6824","nonce":1531769083},{"type":"v1","addr":"172.21.15.120:6825","nonce":1531769083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":1531769083},{"type":"v1","addr":"172.21.15.120:6827","nonce":1531769083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":1531769083},{"type":"v1","addr":"172.21.15.120:6831","nonce":1531769083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6828","nonce":1531769083},{"type":"v1","addr":"172.21.15.120:6829","nonce":1531769083}]},"public_addr":"172.21.15.120:6825/1531769083","cluster_addr":"172.21.15.120:6827/1531769083","heartbeat_back_addr":"172.21.15.120:6831/1531769083","heartbeat_front_addr":"172.21.15.120:6829/1531769083","state":["exists","up"]},{"osd":11,"uuid":"73aec750-82d1-4800-b6a0-4e305912f9ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":3769050170},{"type":"v1","addr":"172.21.15.188:6825","nonce":3769050170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":3769050170},{"type":"v1","addr":"172.21.15.188:6827","nonce":3769050170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":3769050170},{"type":"v1","addr":"172.21.15.188:6831","nonce":3769050170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":3769050170},{"type":"v1","addr":"172.21.15.188:6829","nonce":3769050170}]},"public_addr":"172.21.15.188:6825/3769050170","cluster_addr":"172.21.15.188:6827/3769050170","heartbeat_back_addr":"172.21.15.188:6831/3769050170","heartbeat_front_addr":"172.21.15.188:6829/3769050170","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:44:16.455752+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:44:40.219735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:45:03.558899+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:45:29.155004+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:45:53.981098+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:46:19.412020+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:46:45.462640+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:47:11.484578+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:47:36.879054+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:48:03.392446+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:48:29.422173+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:48:55.523423+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":4}]},{"pgid":"3.6","mappings":[{"from":3,"to":7}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.9","mappings":[{"from":5,"to":8}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.52:0/3535271671":"2024-06-11T02:42:42.872312+0000","172.21.15.52:0/1092934380":"2024-06-11T02:42:42.872312+0000","172.21.15.52:6800/896317057":"2024-06-11T02:42:42.872312+0000","172.21.15.52:6801/3934479642":"2024-06-11T02:42:15.498118+0000","172.21.15.52:6800/3934479642":"2024-06-11T02:42:15.498118+0000","172.21.15.52:0/2759694132":"2024-06-11T02:42:15.498118+0000","172.21.15.52:0/2593665631":"2024-06-11T02:42:15.498118+0000","172.21.15.52:0/3151504634":"2024-06-11T02:42:15.498118+0000","172.21.15.52:6801/896317057":"2024-06-11T02:42:42.872312+0000","172.21.15.52:0/1254480982":"2024-06-11T02:42:42.872312+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-10T02:51:50.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:50.245+0000 7f3cb6d5b700 1 -- 172.21.15.52:0/4291117128 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f3c980a2cb0 msgr2=0x7f3c980a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:51:50.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:50.245+0000 7f3cb6d5b700 1 --2- 172.21.15.52:0/4291117128 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f3c980a2cb0 0x7f3c980a5170 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f3cac009750 tx=0x7f3cac00b540 comp rx=0 tx=0).stop 2024-06-10T02:51:50.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:50.245+0000 7f3cb6d5b700 1 -- 172.21.15.52:0/4291117128 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3cb0107400 msgr2=0x7f3cb019f710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:51:50.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:50.245+0000 7f3cb6d5b700 1 --2- 172.21.15.52:0/4291117128 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3cb0107400 0x7f3cb019f710 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f3ca800b970 tx=0x7f3ca800bc80 comp rx=0 tx=0).stop 2024-06-10T02:51:50.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:50.245+0000 7f3cb6d5b700 1 -- 172.21.15.52:0/4291117128 shutdown_connections 2024-06-10T02:51:50.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:50.245+0000 7f3cb6d5b700 1 --2- 172.21.15.52:0/4291117128 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f3c980a2cb0 0x7f3c980a5170 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:51:50.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:50.245+0000 7f3cb6d5b700 1 --2- 172.21.15.52:0/4291117128 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3cb0107400 0x7f3cb019f710 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:51:50.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:50.245+0000 7f3cb6d5b700 1 --2- 172.21.15.52:0/4291117128 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3cb0101390 0x7f3cb019ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:51:50.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:50.245+0000 7f3cb6d5b700 1 --2- 172.21.15.52:0/4291117128 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3cb0101d50 0x7f3cb019b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:51:50.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:50.245+0000 7f3cb6d5b700 1 -- 172.21.15.52:0/4291117128 >> 172.21.15.52:0/4291117128 conn(0x7f3cb0078340 msgr2=0x7f3cb006a270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:51:50.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:50.245+0000 7f3cb6d5b700 1 -- 172.21.15.52:0/4291117128 shutdown_connections 2024-06-10T02:51:50.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:50.246+0000 7f3cb6d5b700 1 -- 172.21.15.52:0/4291117128 wait complete. 2024-06-10T02:51:50.932 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:51:50 smithi052 ceph-mon[32362]: pgmap v363: 129 pgs: 129 active+clean; 646 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 11 op/s 2024-06-10T02:51:50.932 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:51:50 smithi052 ceph-mon[32362]: from='client.? 172.21.15.52:0/4291117128' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T02:51:50.964 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-06-10T02:51:50.965 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-06-10T02:51:50.965 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-06-10T02:51:50.966 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-06-10T02:51:50.966 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-06-10T02:51:50.966 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-06-10T02:51:50.967 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-06-10T02:51:50.967 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-06-10T02:51:50.968 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-06-10T02:51:50.968 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-06-10T02:51:50.968 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-06-10T02:51:50.969 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-06-10T02:51:51.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:51:50 smithi120 ceph-mon[36634]: pgmap v363: 129 pgs: 129 active+clean; 646 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 11 op/s 2024-06-10T02:51:51.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:51:50 smithi120 ceph-mon[36634]: from='client.? 172.21.15.52:0/4291117128' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T02:51:51.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:51:50 smithi188 ceph-mon[36599]: pgmap v363: 129 pgs: 129 active+clean; 646 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 11 op/s 2024-06-10T02:51:51.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:51:50 smithi188 ceph-mon[36599]: from='client.? 172.21.15.52:0/4291117128' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T02:51:51.641 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:51:51.648 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:51:51.700 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:51:52.979 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:51:52 smithi052 ceph-mon[32362]: pgmap v364: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2024-06-10T02:51:53.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:51:52 smithi120 ceph-mon[36634]: pgmap v364: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2024-06-10T02:51:53.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:51:52 smithi188 ceph-mon[36599]: pgmap v364: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2024-06-10T02:51:53.383 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:51:53.386 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:51:53.387 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:51:53.387 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:51:53.404 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:51:53.405 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:51:53.408 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:51:53.840 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:51:53.841 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:51:54.960 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:51:54 smithi052 ceph-mon[32362]: pgmap v365: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 11 op/s 2024-06-10T02:51:55.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:51:54 smithi120 ceph-mon[36634]: pgmap v365: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 11 op/s 2024-06-10T02:51:55.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:51:54 smithi188 ceph-mon[36599]: pgmap v365: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 11 op/s 2024-06-10T02:51:56.845 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:51:56 smithi052 ceph-mon[32362]: pgmap v366: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-06-10T02:51:57.004 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:51:56 smithi188 ceph-mon[36599]: pgmap v366: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-06-10T02:51:57.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:51:56 smithi120 ceph-mon[36634]: pgmap v366: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-06-10T02:51:59.100 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:51:58 smithi052 ceph-mon[32362]: pgmap v367: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-06-10T02:51:59.100 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:51:58 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T02:51:59.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:51:58 smithi120 ceph-mon[36634]: pgmap v367: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-06-10T02:51:59.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:51:58 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T02:51:59.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:51:58 smithi188 ceph-mon[36599]: pgmap v367: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-06-10T02:51:59.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:51:58 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T02:51:59.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:59.209+0000 7f3ec0216700 1 -- 172.21.15.52:0/560743649 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3eb8103120 msgr2=0x7f3eb8107130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:51:59.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:59.209+0000 7f3ec0216700 1 --2- 172.21.15.52:0/560743649 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3eb8103120 0x7f3eb8107130 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f3ea80099c0 tx=0x7f3ea8009cd0 comp rx=0 tx=0).stop 2024-06-10T02:51:59.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:59.209+0000 7f3ec0216700 1 -- 172.21.15.52:0/560743649 shutdown_connections 2024-06-10T02:51:59.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:59.209+0000 7f3ec0216700 1 --2- 172.21.15.52:0/560743649 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3eb8107860 0x7f3eb810f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:51:59.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:59.209+0000 7f3ec0216700 1 --2- 172.21.15.52:0/560743649 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3eb8102770 0x7f3eb8102b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:51:59.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:59.209+0000 7f3ec0216700 1 --2- 172.21.15.52:0/560743649 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3eb8103120 0x7f3eb8107130 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:51:59.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:59.209+0000 7f3ec0216700 1 -- 172.21.15.52:0/560743649 >> 172.21.15.52:0/560743649 conn(0x7f3eb8079600 msgr2=0x7f3eb8079a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:51:59.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:59.209+0000 7f3ec0216700 1 -- 172.21.15.52:0/560743649 shutdown_connections 2024-06-10T02:51:59.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:59.209+0000 7f3ec0216700 1 -- 172.21.15.52:0/560743649 wait complete. 2024-06-10T02:51:59.212 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:59.210+0000 7f3ec0216700 1 Processor -- start 2024-06-10T02:51:59.212 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:59.210+0000 7f3ec0216700 1 -- start start 2024-06-10T02:51:59.212 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:59.210+0000 7f3ec0216700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3eb8102770 0x7f3eb8074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:51:59.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:59.210+0000 7f3ec0216700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3eb8103120 0x7f3eb8074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:51:59.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:59.210+0000 7f3ec0216700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3eb8107860 0x7f3eb8075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:51:59.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:59.210+0000 7f3ec0216700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f3eb8112770 con 0x7f3eb8103120 2024-06-10T02:51:59.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:59.210+0000 7f3ec0216700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f3eb81125f0 con 0x7f3eb8107860 2024-06-10T02:51:59.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:59.210+0000 7f3ec0216700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f3eb81128f0 con 0x7f3eb8102770 2024-06-10T02:51:59.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:59.211+0000 7f3ebd7b1700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3eb8103120 0x7f3eb8074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:51:59.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:59.211+0000 7f3ebdfb2700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3eb8102770 0x7f3eb8074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:51:59.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:59.211+0000 7f3ebd7b1700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3eb8103120 0x7f3eb8074cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54828/0 (socket says 172.21.15.52:54828) 2024-06-10T02:51:59.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:59.211+0000 7f3ebd7b1700 1 -- 172.21.15.52:0/129392289 learned_addr learned my addr 172.21.15.52:0/129392289 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:51:59.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:59.211+0000 7f3ebe7b3700 1 --2- 172.21.15.52:0/129392289 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3eb8107860 0x7f3eb8075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:51:59.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:59.211+0000 7f3ebd7b1700 1 -- 172.21.15.52:0/129392289 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3eb8102770 msgr2=0x7f3eb8074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:51:59.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:59.211+0000 7f3ebd7b1700 1 --2- 172.21.15.52:0/129392289 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3eb8102770 0x7f3eb8074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:51:59.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:59.211+0000 7f3ebd7b1700 1 -- 172.21.15.52:0/129392289 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3eb8107860 msgr2=0x7f3eb8075210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:51:59.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:59.211+0000 7f3ebd7b1700 1 --2- 172.21.15.52:0/129392289 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3eb8107860 0x7f3eb8075210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:51:59.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:59.211+0000 7f3ebd7b1700 1 -- 172.21.15.52:0/129392289 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3eb81ab550 con 0x7f3eb8103120 2024-06-10T02:51:59.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:59.211+0000 7f3ebe7b3700 1 --2- 172.21.15.52:0/129392289 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3eb8107860 0x7f3eb8075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T02:51:59.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:59.211+0000 7f3ebdfb2700 1 --2- 172.21.15.52:0/129392289 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3eb8102770 0x7f3eb8074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T02:51:59.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:59.211+0000 7f3ebd7b1700 1 --2- 172.21.15.52:0/129392289 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3eb8103120 0x7f3eb8074cd0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f3ea80059d0 tx=0x7f3ea80038b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:51:59.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:59.211+0000 7f3eaeffd700 1 -- 172.21.15.52:0/129392289 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ea8027070 con 0x7f3eb8103120 2024-06-10T02:51:59.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:59.211+0000 7f3ec0216700 1 -- 172.21.15.52:0/129392289 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3eb81ab780 con 0x7f3eb8103120 2024-06-10T02:51:59.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:59.211+0000 7f3eaeffd700 1 -- 172.21.15.52:0/129392289 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ea8004050 con 0x7f3eb8103120 2024-06-10T02:51:59.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:59.211+0000 7f3eaeffd700 1 -- 172.21.15.52:0/129392289 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ea8022db0 con 0x7f3eb8103120 2024-06-10T02:51:59.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:59.211+0000 7f3ec0216700 1 -- 172.21.15.52:0/129392289 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3eb81abc30 con 0x7f3eb8103120 2024-06-10T02:51:59.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:59.212+0000 7f3ec0216700 1 -- 172.21.15.52:0/129392289 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3eb8062c20 con 0x7f3eb8103120 2024-06-10T02:51:59.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:59.214+0000 7f3eaeffd700 1 -- 172.21.15.52:0/129392289 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f3ea802a070 con 0x7f3eb8103120 2024-06-10T02:51:59.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:59.214+0000 7f3eaeffd700 1 --2- 172.21.15.52:0/129392289 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f3ea40a2cb0 0x7f3ea40a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:51:59.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:59.214+0000 7f3ebdfb2700 1 --2- 172.21.15.52:0/129392289 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f3ea40a2cb0 0x7f3ea40a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:51:59.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:59.215+0000 7f3eaeffd700 1 -- 172.21.15.52:0/129392289 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f3ea80d0c50 con 0x7f3eb8103120 2024-06-10T02:51:59.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:59.215+0000 7f3eaeffd700 1 --2- 172.21.15.52:0/129392289 >> [v2:172.21.15.52:6802/4255833408,v1:172.21.15.52:6803/4255833408] conn(0x7f3ea40adf90 0x7f3ea40b0430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:51:59.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:59.215+0000 7f3eaeffd700 1 -- 172.21.15.52:0/129392289 --> [v2:172.21.15.52:6802/4255833408,v1:172.21.15.52:6803/4255833408] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3ea80095d0 con 0x7f3ea40adf90 2024-06-10T02:51:59.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:59.215+0000 7f3eaeffd700 1 -- 172.21.15.52:0/129392289 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_get_version_reply(handle=1 version=100) ==== 24+0+0 (secure 0 0 0) 0x7f3ea80d10a0 con 0x7f3eb8103120 2024-06-10T02:51:59.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:59.215+0000 7f3ebe7b3700 1 --2- 172.21.15.52:0/129392289 >> [v2:172.21.15.52:6802/4255833408,v1:172.21.15.52:6803/4255833408] conn(0x7f3ea40adf90 0x7f3ea40b0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:51:59.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:59.215+0000 7f3ebdfb2700 1 --2- 172.21.15.52:0/129392289 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f3ea40a2cb0 0x7f3ea40a5170 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f3eb4009fd0 tx=0x7f3eb40091b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:51:59.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:59.215+0000 7f3ebe7b3700 1 --2- 172.21.15.52:0/129392289 >> [v2:172.21.15.52:6802/4255833408,v1:172.21.15.52:6803/4255833408] conn(0x7f3ea40adf90 0x7f3ea40b0430 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:51:59.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:59.216+0000 7f3eaeffd700 1 -- 172.21.15.52:0/129392289 <== osd.0 v2:172.21.15.52:6802/4255833408 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3ea80095d0 con 0x7f3ea40adf90 2024-06-10T02:51:59.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:59.232+0000 7f3ec0216700 1 -- 172.21.15.52:0/129392289 --> [v2:172.21.15.52:6802/4255833408,v1:172.21.15.52:6803/4255833408] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f3eb80630a0 con 0x7f3ea40adf90 2024-06-10T02:51:59.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:59.233+0000 7f3eaeffd700 1 -- 172.21.15.52:0/129392289 <== osd.0 v2:172.21.15.52:6802/4255833408 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f3eb80630a0 con 0x7f3ea40adf90 2024-06-10T02:51:59.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:59.233+0000 7f3ec0216700 1 -- 172.21.15.52:0/129392289 >> [v2:172.21.15.52:6802/4255833408,v1:172.21.15.52:6803/4255833408] conn(0x7f3ea40adf90 msgr2=0x7f3ea40b0430 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:51:59.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:59.233+0000 7f3ec0216700 1 --2- 172.21.15.52:0/129392289 >> [v2:172.21.15.52:6802/4255833408,v1:172.21.15.52:6803/4255833408] conn(0x7f3ea40adf90 0x7f3ea40b0430 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:51:59.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:59.233+0000 7f3ec0216700 1 -- 172.21.15.52:0/129392289 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f3ea40a2cb0 msgr2=0x7f3ea40a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:51:59.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:59.233+0000 7f3ec0216700 1 --2- 172.21.15.52:0/129392289 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f3ea40a2cb0 0x7f3ea40a5170 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f3eb4009fd0 tx=0x7f3eb40091b0 comp rx=0 tx=0).stop 2024-06-10T02:51:59.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:59.233+0000 7f3ec0216700 1 -- 172.21.15.52:0/129392289 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3eb8103120 msgr2=0x7f3eb8074cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:51:59.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:59.233+0000 7f3ec0216700 1 --2- 172.21.15.52:0/129392289 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3eb8103120 0x7f3eb8074cd0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f3ea80059d0 tx=0x7f3ea80038b0 comp rx=0 tx=0).stop 2024-06-10T02:51:59.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:59.233+0000 7f3ebe7b3700 1 -- 172.21.15.52:0/129392289 reap_dead start 2024-06-10T02:51:59.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:59.233+0000 7f3ec0216700 1 -- 172.21.15.52:0/129392289 shutdown_connections 2024-06-10T02:51:59.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:59.233+0000 7f3ec0216700 1 -- 172.21.15.52:0/129392289 >> 172.21.15.52:0/129392289 conn(0x7f3eb8079600 msgr2=0x7f3eb8101cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:51:59.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:59.233+0000 7f3ec0216700 1 -- 172.21.15.52:0/129392289 shutdown_connections 2024-06-10T02:51:59.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:59.233+0000 7f3ec0216700 1 -- 172.21.15.52:0/129392289 wait complete. 2024-06-10T02:51:59.301 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:58.384+0000 7fa8e1e79700 1 -- 172.21.15.52:0/3823751068 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa8dc1084f0 msgr2=0x7fa8dc1088d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:51:59.301 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:58.384+0000 7fa8e1e79700 1 --2- 172.21.15.52:0/3823751068 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa8dc1084f0 0x7fa8dc1088d0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fa8cc009990 tx=0x7fa8cc009ca0 comp rx=0 tx=0).stop 2024-06-10T02:51:59.302 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:58.384+0000 7fa8e1e79700 1 -- 172.21.15.52:0/3823751068 shutdown_connections 2024-06-10T02:51:59.302 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:58.384+0000 7fa8e1e79700 1 --2- 172.21.15.52:0/3823751068 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa8dc06b3c0 0x7fa8dc10f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:51:59.302 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:58.384+0000 7fa8e1e79700 1 --2- 172.21.15.52:0/3823751068 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa8dc1084f0 0x7fa8dc1088d0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:51:59.302 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:58.384+0000 7fa8e1e79700 1 --2- 172.21.15.52:0/3823751068 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa8dc06aa00 0x7fa8dc06ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:51:59.302 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:58.384+0000 7fa8e1e79700 1 -- 172.21.15.52:0/3823751068 >> 172.21.15.52:0/3823751068 conn(0x7fa8dc078300 msgr2=0x7fa8dc078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:51:59.302 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:58.384+0000 7fa8e1e79700 1 -- 172.21.15.52:0/3823751068 shutdown_connections 2024-06-10T02:51:59.302 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:58.384+0000 7fa8e1e79700 1 -- 172.21.15.52:0/3823751068 wait complete. 2024-06-10T02:51:59.302 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:58.385+0000 7fa8e1e79700 1 Processor -- start 2024-06-10T02:51:59.302 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:58.385+0000 7fa8e1e79700 1 -- start start 2024-06-10T02:51:59.302 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:58.385+0000 7fa8e1e79700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa8dc06aa00 0x7fa8dc19e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:51:59.302 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:58.385+0000 7fa8e1e79700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa8dc06b3c0 0x7fa8dc19edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:51:59.302 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:58.385+0000 7fa8e1e79700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa8dc1084f0 0x7fa8dc198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:51:59.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:58.385+0000 7fa8e1e79700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fa8dc112750 con 0x7fa8dc06aa00 2024-06-10T02:51:59.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:58.385+0000 7fa8e1e79700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa8dc1125d0 con 0x7fa8dc1084f0 2024-06-10T02:51:59.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:58.385+0000 7fa8e1e79700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa8dc1128d0 con 0x7fa8dc06b3c0 2024-06-10T02:51:59.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:58.386+0000 7fa8db7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa8dc06aa00 0x7fa8dc19e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:51:59.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:58.386+0000 7fa8db7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa8dc06aa00 0x7fa8dc19e870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54822/0 (socket says 172.21.15.52:54822) 2024-06-10T02:51:59.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:58.386+0000 7fa8db7fe700 1 -- 172.21.15.52:0/2997162561 learned_addr learned my addr 172.21.15.52:0/2997162561 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:51:59.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:58.386+0000 7fa8d2ffd700 1 --2- 172.21.15.52:0/2997162561 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa8dc06b3c0 0x7fa8dc19edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:51:59.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:58.386+0000 7fa8dbfff700 1 --2- 172.21.15.52:0/2997162561 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa8dc1084f0 0x7fa8dc198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:51:59.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:58.386+0000 7fa8db7fe700 1 -- 172.21.15.52:0/2997162561 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa8dc06b3c0 msgr2=0x7fa8dc19edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:51:59.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:58.386+0000 7fa8db7fe700 1 --2- 172.21.15.52:0/2997162561 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa8dc06b3c0 0x7fa8dc19edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:51:59.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:58.386+0000 7fa8db7fe700 1 -- 172.21.15.52:0/2997162561 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa8dc1084f0 msgr2=0x7fa8dc198a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:51:59.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:58.386+0000 7fa8db7fe700 1 --2- 172.21.15.52:0/2997162561 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa8dc1084f0 0x7fa8dc198a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:51:59.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:58.386+0000 7fa8db7fe700 1 -- 172.21.15.52:0/2997162561 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8dc199190 con 0x7fa8dc06aa00 2024-06-10T02:51:59.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:58.386+0000 7fa8db7fe700 1 --2- 172.21.15.52:0/2997162561 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa8dc06aa00 0x7fa8dc19e870 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fa8cc000c00 tx=0x7fa8cc0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:51:59.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:58.386+0000 7fa8d97fa700 1 -- 172.21.15.52:0/2997162561 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8cc027070 con 0x7fa8dc06aa00 2024-06-10T02:51:59.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:58.386+0000 7fa8d97fa700 1 -- 172.21.15.52:0/2997162561 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa8cc003780 con 0x7fa8dc06aa00 2024-06-10T02:51:59.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:58.386+0000 7fa8e1e79700 1 -- 172.21.15.52:0/2997162561 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8dc199420 con 0x7fa8dc06aa00 2024-06-10T02:51:59.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:58.386+0000 7fa8d97fa700 1 -- 172.21.15.52:0/2997162561 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8cc022dd0 con 0x7fa8dc06aa00 2024-06-10T02:51:59.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:58.386+0000 7fa8e1e79700 1 -- 172.21.15.52:0/2997162561 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8dc1a5740 con 0x7fa8dc06aa00 2024-06-10T02:51:59.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:58.387+0000 7fa8e1e79700 1 -- 172.21.15.52:0/2997162561 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa8dc062ba0 con 0x7fa8dc06aa00 2024-06-10T02:51:59.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:58.388+0000 7fa8d97fa700 1 -- 172.21.15.52:0/2997162561 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fa8cc02a070 con 0x7fa8dc06aa00 2024-06-10T02:51:59.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:58.389+0000 7fa8d97fa700 1 --2- 172.21.15.52:0/2997162561 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fa8c80a2cb0 0x7fa8c80a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:51:59.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:58.389+0000 7fa8d2ffd700 1 --2- 172.21.15.52:0/2997162561 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fa8c80a2cb0 0x7fa8c80a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:51:59.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:58.389+0000 7fa8d97fa700 1 -- 172.21.15.52:0/2997162561 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7fa8cc0d08a0 con 0x7fa8dc06aa00 2024-06-10T02:51:59.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:58.389+0000 7fa8d97fa700 1 --2- 172.21.15.52:0/2997162561 >> [v2:172.21.15.120:6800/4116540053,v1:172.21.15.120:6801/4116540053] conn(0x7fa8c80adf90 0x7fa8c80b0430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:51:59.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:58.389+0000 7fa8d97fa700 1 -- 172.21.15.52:0/2997162561 --> [v2:172.21.15.120:6800/4116540053,v1:172.21.15.120:6801/4116540053] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa8cc001d30 con 0x7fa8c80adf90 2024-06-10T02:51:59.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:58.389+0000 7fa8d97fa700 1 -- 172.21.15.52:0/2997162561 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_get_version_reply(handle=1 version=100) ==== 24+0+0 (secure 0 0 0) 0x7fa8cc0d0cf0 con 0x7fa8dc06aa00 2024-06-10T02:51:59.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:58.390+0000 7fa8d2ffd700 1 --2- 172.21.15.52:0/2997162561 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fa8c80a2cb0 0x7fa8c80a5170 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa8dc0789e0 tx=0x7fa8c40073d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:51:59.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:58.390+0000 7fa8dbfff700 1 --2- 172.21.15.52:0/2997162561 >> [v2:172.21.15.120:6800/4116540053,v1:172.21.15.120:6801/4116540053] conn(0x7fa8c80adf90 0x7fa8c80b0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:51:59.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:58.390+0000 7fa8dbfff700 1 --2- 172.21.15.52:0/2997162561 >> [v2:172.21.15.120:6800/4116540053,v1:172.21.15.120:6801/4116540053] conn(0x7fa8c80adf90 0x7fa8c80b0430 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:51:59.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:58.391+0000 7fa8d97fa700 1 -- 172.21.15.52:0/2997162561 <== osd.1 v2:172.21.15.120:6800/4116540053 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa8cc001d30 con 0x7fa8c80adf90 2024-06-10T02:51:59.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:58.407+0000 7fa8e1e79700 1 -- 172.21.15.52:0/2997162561 --> [v2:172.21.15.120:6800/4116540053,v1:172.21.15.120:6801/4116540053] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa8dc063020 con 0x7fa8c80adf90 2024-06-10T02:51:59.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:58.408+0000 7fa8d97fa700 1 -- 172.21.15.52:0/2997162561 <== osd.1 v2:172.21.15.120:6800/4116540053 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fa8dc063020 con 0x7fa8c80adf90 2024-06-10T02:51:59.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:58.408+0000 7fa8e1e79700 1 -- 172.21.15.52:0/2997162561 >> [v2:172.21.15.120:6800/4116540053,v1:172.21.15.120:6801/4116540053] conn(0x7fa8c80adf90 msgr2=0x7fa8c80b0430 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:51:59.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:58.408+0000 7fa8e1e79700 1 --2- 172.21.15.52:0/2997162561 >> [v2:172.21.15.120:6800/4116540053,v1:172.21.15.120:6801/4116540053] conn(0x7fa8c80adf90 0x7fa8c80b0430 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:51:59.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:58.408+0000 7fa8e1e79700 1 -- 172.21.15.52:0/2997162561 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fa8c80a2cb0 msgr2=0x7fa8c80a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:51:59.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:58.408+0000 7fa8e1e79700 1 --2- 172.21.15.52:0/2997162561 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fa8c80a2cb0 0x7fa8c80a5170 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa8dc0789e0 tx=0x7fa8c40073d0 comp rx=0 tx=0).stop 2024-06-10T02:51:59.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:58.408+0000 7fa8e1e79700 1 -- 172.21.15.52:0/2997162561 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa8dc06aa00 msgr2=0x7fa8dc19e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:51:59.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:58.408+0000 7fa8e1e79700 1 --2- 172.21.15.52:0/2997162561 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa8dc06aa00 0x7fa8dc19e870 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fa8cc000c00 tx=0x7fa8cc0047b0 comp rx=0 tx=0).stop 2024-06-10T02:51:59.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:58.408+0000 7fa8dbfff700 1 -- 172.21.15.52:0/2997162561 reap_dead start 2024-06-10T02:51:59.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:58.408+0000 7fa8e1e79700 1 -- 172.21.15.52:0/2997162561 shutdown_connections 2024-06-10T02:51:59.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:58.408+0000 7fa8e1e79700 1 -- 172.21.15.52:0/2997162561 >> 172.21.15.52:0/2997162561 conn(0x7fa8dc078300 msgr2=0x7fa8dc100150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:51:59.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:58.408+0000 7fa8e1e79700 1 -- 172.21.15.52:0/2997162561 shutdown_connections 2024-06-10T02:51:59.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:51:58.408+0000 7fa8e1e79700 1 -- 172.21.15.52:0/2997162561 wait complete. 2024-06-10T02:52:00.546 INFO:teuthology.orchestra.run.smithi052.stdout:34359738464 2024-06-10T02:52:00.546 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-06-10T02:52:00.805 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:52:00 smithi052 ceph-mon[32362]: pgmap v368: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-06-10T02:52:00.805 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:52:00 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:52:00.805 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:52:00 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:52:00.806 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:52:00 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:52:00.806 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:52:00 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:52:01.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:52:00 smithi120 ceph-mon[36634]: pgmap v368: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-06-10T02:52:01.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:52:00 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:52:01.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:52:00 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:52:01.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:52:00 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:52:01.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:52:00 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:52:01.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:52:00 smithi188 ceph-mon[36599]: pgmap v368: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-06-10T02:52:01.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:52:00 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:52:01.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:52:00 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:52:01.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:52:00 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:52:01.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:52:00 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:52:03.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:52:02 smithi120 ceph-mon[36634]: pgmap v369: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-06-10T02:52:03.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:52:02 smithi188 ceph-mon[36599]: pgmap v369: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-06-10T02:52:03.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:52:02 smithi052 ceph-mon[32362]: pgmap v369: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-06-10T02:52:03.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:03.345+0000 7f10e54b6700 1 -- 172.21.15.52:0/4145212036 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f10e0108500 msgr2=0x7f10e01088e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:52:03.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:03.345+0000 7f10e54b6700 1 --2- 172.21.15.52:0/4145212036 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f10e0108500 0x7f10e01088e0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f10d0009990 tx=0x7f10d0009ca0 comp rx=0 tx=0).stop 2024-06-10T02:52:03.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:03.345+0000 7f10e54b6700 1 -- 172.21.15.52:0/4145212036 shutdown_connections 2024-06-10T02:52:03.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:03.345+0000 7f10e54b6700 1 --2- 172.21.15.52:0/4145212036 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f10e006b580 0x7f10e010f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:03.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:03.345+0000 7f10e54b6700 1 --2- 172.21.15.52:0/4145212036 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f10e0108500 0x7f10e01088e0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:03.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:03.345+0000 7f10e54b6700 1 --2- 172.21.15.52:0/4145212036 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f10e006abc0 0x7f10e006b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:03.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:03.345+0000 7f10e54b6700 1 -- 172.21.15.52:0/4145212036 >> 172.21.15.52:0/4145212036 conn(0x7f10e0078340 msgr2=0x7f10e0078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:52:03.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:03.345+0000 7f10e54b6700 1 -- 172.21.15.52:0/4145212036 shutdown_connections 2024-06-10T02:52:03.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:03.345+0000 7f10e54b6700 1 -- 172.21.15.52:0/4145212036 wait complete. 2024-06-10T02:52:03.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:03.346+0000 7f10e54b6700 1 Processor -- start 2024-06-10T02:52:03.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:03.346+0000 7f10e54b6700 1 -- start start 2024-06-10T02:52:03.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:03.346+0000 7f10e54b6700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f10e006abc0 0x7f10e0198d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:52:03.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:03.346+0000 7f10e54b6700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f10e006b580 0x7f10e01992b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:52:03.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:03.346+0000 7f10e54b6700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f10e0108500 0x7f10e019d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:52:03.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:03.346+0000 7f10e54b6700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f10e01127e0 con 0x7f10e006abc0 2024-06-10T02:52:03.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:03.346+0000 7f10e54b6700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f10e0112660 con 0x7f10e006b580 2024-06-10T02:52:03.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:03.346+0000 7f10e54b6700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f10e0112960 con 0x7f10e0108500 2024-06-10T02:52:03.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:03.346+0000 7f10deffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f10e006abc0 0x7f10e0198d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:52:03.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:03.347+0000 7f10deffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f10e006abc0 0x7f10e0198d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60074/0 (socket says 172.21.15.52:60074) 2024-06-10T02:52:03.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:03.347+0000 7f10deffd700 1 -- 172.21.15.52:0/4163593957 learned_addr learned my addr 172.21.15.52:0/4163593957 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:52:03.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:03.347+0000 7f10de7fc700 1 --2- 172.21.15.52:0/4163593957 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f10e006b580 0x7f10e01992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:52:03.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:03.347+0000 7f10df7fe700 1 --2- 172.21.15.52:0/4163593957 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f10e0108500 0x7f10e019d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:52:03.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:03.347+0000 7f10deffd700 1 -- 172.21.15.52:0/4163593957 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f10e0108500 msgr2=0x7f10e019d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:52:03.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:03.347+0000 7f10deffd700 1 --2- 172.21.15.52:0/4163593957 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f10e0108500 0x7f10e019d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:03.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:03.347+0000 7f10deffd700 1 -- 172.21.15.52:0/4163593957 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f10e006b580 msgr2=0x7f10e01992b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:52:03.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:03.347+0000 7f10deffd700 1 --2- 172.21.15.52:0/4163593957 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f10e006b580 0x7f10e01992b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:03.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:03.347+0000 7f10deffd700 1 -- 172.21.15.52:0/4163593957 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10e019dd80 con 0x7f10e006abc0 2024-06-10T02:52:03.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:03.347+0000 7f10deffd700 1 --2- 172.21.15.52:0/4163593957 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f10e006abc0 0x7f10e0198d70 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f10d0000c00 tx=0x7f10d00047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:52:03.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:03.347+0000 7f10c7fff700 1 -- 172.21.15.52:0/4163593957 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10d0027070 con 0x7f10e006abc0 2024-06-10T02:52:03.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:03.347+0000 7f10e54b6700 1 -- 172.21.15.52:0/4163593957 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10e019e010 con 0x7f10e006abc0 2024-06-10T02:52:03.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:03.347+0000 7f10c7fff700 1 -- 172.21.15.52:0/4163593957 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f10d0003e40 con 0x7f10e006abc0 2024-06-10T02:52:03.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:03.347+0000 7f10c7fff700 1 -- 172.21.15.52:0/4163593957 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10d0022d10 con 0x7f10e006abc0 2024-06-10T02:52:03.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:03.347+0000 7f10e54b6700 1 -- 172.21.15.52:0/4163593957 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10e01a58f0 con 0x7f10e006abc0 2024-06-10T02:52:03.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:03.348+0000 7f10e54b6700 1 -- 172.21.15.52:0/4163593957 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f10e0062ba0 con 0x7f10e006abc0 2024-06-10T02:52:03.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:03.350+0000 7f10c7fff700 1 -- 172.21.15.52:0/4163593957 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f10d002a070 con 0x7f10e006abc0 2024-06-10T02:52:03.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:03.350+0000 7f10c7fff700 1 --2- 172.21.15.52:0/4163593957 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f10c80a2cb0 0x7f10c80a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:52:03.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:03.350+0000 7f10c7fff700 1 -- 172.21.15.52:0/4163593957 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f10d00d0d00 con 0x7f10e006abc0 2024-06-10T02:52:03.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:03.350+0000 7f10de7fc700 1 --2- 172.21.15.52:0/4163593957 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f10c80a2cb0 0x7f10c80a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:52:03.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:03.351+0000 7f10c7fff700 1 --2- 172.21.15.52:0/4163593957 >> [v2:172.21.15.188:6808/3951947483,v1:172.21.15.188:6809/3951947483] conn(0x7f10c80adf90 0x7f10c80b0430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:52:03.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:03.351+0000 7f10c7fff700 1 -- 172.21.15.52:0/4163593957 --> [v2:172.21.15.188:6808/3951947483,v1:172.21.15.188:6809/3951947483] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f10d0010ec0 con 0x7f10c80adf90 2024-06-10T02:52:03.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:03.351+0000 7f10c7fff700 1 -- 172.21.15.52:0/4163593957 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_get_version_reply(handle=1 version=100) ==== 24+0+0 (secure 0 0 0) 0x7f10d00d1150 con 0x7f10e006abc0 2024-06-10T02:52:03.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:03.351+0000 7f10df7fe700 1 --2- 172.21.15.52:0/4163593957 >> [v2:172.21.15.188:6808/3951947483,v1:172.21.15.188:6809/3951947483] conn(0x7f10c80adf90 0x7f10c80b0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:52:03.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:03.351+0000 7f10de7fc700 1 --2- 172.21.15.52:0/4163593957 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f10c80a2cb0 0x7f10c80a5170 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f10e0078a60 tx=0x7f10d4009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:52:03.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:03.351+0000 7f10df7fe700 1 --2- 172.21.15.52:0/4163593957 >> [v2:172.21.15.188:6808/3951947483,v1:172.21.15.188:6809/3951947483] conn(0x7f10c80adf90 0x7f10c80b0430 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:52:03.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:03.352+0000 7f10c7fff700 1 -- 172.21.15.52:0/4163593957 <== osd.5 v2:172.21.15.188:6808/3951947483 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f10d0010ec0 con 0x7f10c80adf90 2024-06-10T02:52:03.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:03.369+0000 7f10e54b6700 1 -- 172.21.15.52:0/4163593957 --> [v2:172.21.15.188:6808/3951947483,v1:172.21.15.188:6809/3951947483] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f10e0063020 con 0x7f10c80adf90 2024-06-10T02:52:03.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:03.369+0000 7f10c7fff700 1 -- 172.21.15.52:0/4163593957 <== osd.5 v2:172.21.15.188:6808/3951947483 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f10e0063020 con 0x7f10c80adf90 2024-06-10T02:52:03.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:03.370+0000 7f10e54b6700 1 -- 172.21.15.52:0/4163593957 >> [v2:172.21.15.188:6808/3951947483,v1:172.21.15.188:6809/3951947483] conn(0x7f10c80adf90 msgr2=0x7f10c80b0430 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:52:03.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:03.370+0000 7f10e54b6700 1 --2- 172.21.15.52:0/4163593957 >> [v2:172.21.15.188:6808/3951947483,v1:172.21.15.188:6809/3951947483] conn(0x7f10c80adf90 0x7f10c80b0430 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:03.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:03.370+0000 7f10e54b6700 1 -- 172.21.15.52:0/4163593957 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f10c80a2cb0 msgr2=0x7f10c80a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:52:03.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:03.370+0000 7f10e54b6700 1 --2- 172.21.15.52:0/4163593957 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f10c80a2cb0 0x7f10c80a5170 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f10e0078a60 tx=0x7f10d4009190 comp rx=0 tx=0).stop 2024-06-10T02:52:03.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:03.370+0000 7f10e54b6700 1 -- 172.21.15.52:0/4163593957 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f10e006abc0 msgr2=0x7f10e0198d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:52:03.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:03.370+0000 7f10e54b6700 1 --2- 172.21.15.52:0/4163593957 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f10e006abc0 0x7f10e0198d70 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f10d0000c00 tx=0x7f10d00047b0 comp rx=0 tx=0).stop 2024-06-10T02:52:03.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:03.370+0000 7f10df7fe700 1 -- 172.21.15.52:0/4163593957 reap_dead start 2024-06-10T02:52:03.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:03.370+0000 7f10e54b6700 1 -- 172.21.15.52:0/4163593957 shutdown_connections 2024-06-10T02:52:03.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:03.370+0000 7f10e54b6700 1 -- 172.21.15.52:0/4163593957 >> 172.21.15.52:0/4163593957 conn(0x7f10e0078340 msgr2=0x7f10e0100150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:52:03.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:03.370+0000 7f10e54b6700 1 -- 172.21.15.52:0/4163593957 shutdown_connections 2024-06-10T02:52:03.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:03.370+0000 7f10e54b6700 1 -- 172.21.15.52:0/4163593957 wait complete. 2024-06-10T02:52:04.924 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:52:04 smithi052 ceph-mon[32362]: pgmap v370: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-06-10T02:52:05.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:52:04 smithi120 ceph-mon[36634]: pgmap v370: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-06-10T02:52:05.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:52:04 smithi188 ceph-mon[36599]: pgmap v370: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-06-10T02:52:06.206 INFO:teuthology.orchestra.run.smithi052.stdout:55834574939 2024-06-10T02:52:06.207 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-06-10T02:52:07.068 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:52:06 smithi052 ceph-mon[32362]: pgmap v371: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-10T02:52:07.080 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:07.079+0000 7fe877594700 1 -- 172.21.15.52:0/2064480801 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe870108500 msgr2=0x7fe8701088e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:52:07.080 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:07.079+0000 7fe877594700 1 --2- 172.21.15.52:0/2064480801 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe870108500 0x7fe8701088e0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fe86c009990 tx=0x7fe86c009ca0 comp rx=0 tx=0).stop 2024-06-10T02:52:07.080 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:07.079+0000 7fe877594700 1 -- 172.21.15.52:0/2064480801 shutdown_connections 2024-06-10T02:52:07.080 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:07.079+0000 7fe877594700 1 --2- 172.21.15.52:0/2064480801 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe87006b5c0 0x7fe87010f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:07.080 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:07.079+0000 7fe877594700 1 --2- 172.21.15.52:0/2064480801 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe870108500 0x7fe8701088e0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:07.080 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:07.079+0000 7fe877594700 1 --2- 172.21.15.52:0/2064480801 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe87006ac00 0x7fe87006b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:07.080 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:07.079+0000 7fe877594700 1 -- 172.21.15.52:0/2064480801 >> 172.21.15.52:0/2064480801 conn(0x7fe870078340 msgr2=0x7fe870078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:52:07.080 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:07.079+0000 7fe877594700 1 -- 172.21.15.52:0/2064480801 shutdown_connections 2024-06-10T02:52:07.080 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:07.079+0000 7fe877594700 1 -- 172.21.15.52:0/2064480801 wait complete. 2024-06-10T02:52:07.080 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:07.080+0000 7fe877594700 1 Processor -- start 2024-06-10T02:52:07.080 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:07.080+0000 7fe877594700 1 -- start start 2024-06-10T02:52:07.081 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:07.080+0000 7fe877594700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe87006ac00 0x7fe87019e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:52:07.081 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:07.080+0000 7fe877594700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe87006b5c0 0x7fe87019eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:52:07.081 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:07.080+0000 7fe877594700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe870108500 0x7fe870198a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:52:07.081 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:07.080+0000 7fe877594700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fe870112780 con 0x7fe87006ac00 2024-06-10T02:52:07.081 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:07.080+0000 7fe877594700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe870112600 con 0x7fe87006b5c0 2024-06-10T02:52:07.081 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:07.080+0000 7fe877594700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fe870112900 con 0x7fe870108500 2024-06-10T02:52:07.081 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:07.080+0000 7fe875330700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe87006ac00 0x7fe87019e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:52:07.081 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:07.080+0000 7fe875330700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe87006ac00 0x7fe87019e860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60086/0 (socket says 172.21.15.52:60086) 2024-06-10T02:52:07.081 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:07.080+0000 7fe875330700 1 -- 172.21.15.52:0/946891185 learned_addr learned my addr 172.21.15.52:0/946891185 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:52:07.081 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:07.080+0000 7fe875b31700 1 --2- 172.21.15.52:0/946891185 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe870108500 0x7fe870198a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:52:07.081 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:07.080+0000 7fe874b2f700 1 --2- 172.21.15.52:0/946891185 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe87006b5c0 0x7fe87019eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:52:07.081 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:07.080+0000 7fe875330700 1 -- 172.21.15.52:0/946891185 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe870108500 msgr2=0x7fe870198a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:52:07.081 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:07.080+0000 7fe875330700 1 --2- 172.21.15.52:0/946891185 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe870108500 0x7fe870198a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:07.082 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:07.080+0000 7fe875330700 1 -- 172.21.15.52:0/946891185 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe87006b5c0 msgr2=0x7fe87019eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:52:07.082 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:07.080+0000 7fe875330700 1 --2- 172.21.15.52:0/946891185 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe87006b5c0 0x7fe87019eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:07.082 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:07.081+0000 7fe875330700 1 -- 172.21.15.52:0/946891185 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe870199180 con 0x7fe87006ac00 2024-06-10T02:52:07.082 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:07.081+0000 7fe874b2f700 1 --2- 172.21.15.52:0/946891185 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe87006b5c0 0x7fe87019eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T02:52:07.082 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:07.081+0000 7fe875330700 1 --2- 172.21.15.52:0/946891185 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe87006ac00 0x7fe87019e860 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fe86c000c00 tx=0x7fe86c0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:52:07.082 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:07.081+0000 7fe8667fc700 1 -- 172.21.15.52:0/946891185 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe86c027070 con 0x7fe87006ac00 2024-06-10T02:52:07.082 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:07.081+0000 7fe8667fc700 1 -- 172.21.15.52:0/946891185 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe86c003780 con 0x7fe87006ac00 2024-06-10T02:52:07.082 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:07.081+0000 7fe877594700 1 -- 172.21.15.52:0/946891185 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe870199410 con 0x7fe87006ac00 2024-06-10T02:52:07.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:07.082+0000 7fe8667fc700 1 -- 172.21.15.52:0/946891185 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe86c022dd0 con 0x7fe87006ac00 2024-06-10T02:52:07.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:07.082+0000 7fe877594700 1 -- 172.21.15.52:0/946891185 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8701a5730 con 0x7fe87006ac00 2024-06-10T02:52:07.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:07.082+0000 7fe877594700 1 -- 172.21.15.52:0/946891185 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe870062c20 con 0x7fe87006ac00 2024-06-10T02:52:07.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:07.083+0000 7fe8667fc700 1 -- 172.21.15.52:0/946891185 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fe86c02a070 con 0x7fe87006ac00 2024-06-10T02:52:07.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:07.084+0000 7fe8667fc700 1 --2- 172.21.15.52:0/946891185 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fe85c0a2d30 0x7fe85c0a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:52:07.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:07.084+0000 7fe874b2f700 1 --2- 172.21.15.52:0/946891185 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fe85c0a2d30 0x7fe85c0a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:52:07.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:07.084+0000 7fe8667fc700 1 -- 172.21.15.52:0/946891185 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7fe86c0d01d0 con 0x7fe87006ac00 2024-06-10T02:52:07.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:07.084+0000 7fe8667fc700 1 --2- 172.21.15.52:0/946891185 >> [v2:172.21.15.52:6826/924440658,v1:172.21.15.52:6827/924440658] conn(0x7fe85c0ae010 0x7fe85c0b04b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:52:07.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:07.084+0000 7fe8667fc700 1 -- 172.21.15.52:0/946891185 --> [v2:172.21.15.52:6826/924440658,v1:172.21.15.52:6827/924440658] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe86c0043d0 con 0x7fe85c0ae010 2024-06-10T02:52:07.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:07.084+0000 7fe8667fc700 1 -- 172.21.15.52:0/946891185 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_get_version_reply(handle=1 version=100) ==== 24+0+0 (secure 0 0 0) 0x7fe86c0d7050 con 0x7fe87006ac00 2024-06-10T02:52:07.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:07.085+0000 7fe875b31700 1 --2- 172.21.15.52:0/946891185 >> [v2:172.21.15.52:6826/924440658,v1:172.21.15.52:6827/924440658] conn(0x7fe85c0ae010 0x7fe85c0b04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:52:07.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:07.085+0000 7fe874b2f700 1 --2- 172.21.15.52:0/946891185 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fe85c0a2d30 0x7fe85c0a51f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fe870078940 tx=0x7fe860005f50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:52:07.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:07.085+0000 7fe875b31700 1 --2- 172.21.15.52:0/946891185 >> [v2:172.21.15.52:6826/924440658,v1:172.21.15.52:6827/924440658] conn(0x7fe85c0ae010 0x7fe85c0b04b0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:52:07.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:07.086+0000 7fe8667fc700 1 -- 172.21.15.52:0/946891185 <== osd.9 v2:172.21.15.52:6826/924440658 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe86c0043d0 con 0x7fe85c0ae010 2024-06-10T02:52:07.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:07.118+0000 7fe877594700 1 -- 172.21.15.52:0/946891185 --> [v2:172.21.15.52:6826/924440658,v1:172.21.15.52:6827/924440658] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe870063050 con 0x7fe85c0ae010 2024-06-10T02:52:07.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:07.118+0000 7fe8667fc700 1 -- 172.21.15.52:0/946891185 <== osd.9 v2:172.21.15.52:6826/924440658 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fe870063050 con 0x7fe85c0ae010 2024-06-10T02:52:07.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:07.118+0000 7fe877594700 1 -- 172.21.15.52:0/946891185 >> [v2:172.21.15.52:6826/924440658,v1:172.21.15.52:6827/924440658] conn(0x7fe85c0ae010 msgr2=0x7fe85c0b04b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:52:07.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:07.118+0000 7fe877594700 1 --2- 172.21.15.52:0/946891185 >> [v2:172.21.15.52:6826/924440658,v1:172.21.15.52:6827/924440658] conn(0x7fe85c0ae010 0x7fe85c0b04b0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:07.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:07.119+0000 7fe877594700 1 -- 172.21.15.52:0/946891185 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fe85c0a2d30 msgr2=0x7fe85c0a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:52:07.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:07.119+0000 7fe877594700 1 --2- 172.21.15.52:0/946891185 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fe85c0a2d30 0x7fe85c0a51f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fe870078940 tx=0x7fe860005f50 comp rx=0 tx=0).stop 2024-06-10T02:52:07.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:07.119+0000 7fe877594700 1 -- 172.21.15.52:0/946891185 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe87006ac00 msgr2=0x7fe87019e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:52:07.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:07.119+0000 7fe877594700 1 --2- 172.21.15.52:0/946891185 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe87006ac00 0x7fe87019e860 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fe86c000c00 tx=0x7fe86c0047b0 comp rx=0 tx=0).stop 2024-06-10T02:52:07.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:07.119+0000 7fe875b31700 1 -- 172.21.15.52:0/946891185 reap_dead start 2024-06-10T02:52:07.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:07.119+0000 7fe877594700 1 -- 172.21.15.52:0/946891185 shutdown_connections 2024-06-10T02:52:07.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:07.119+0000 7fe877594700 1 -- 172.21.15.52:0/946891185 >> 172.21.15.52:0/946891185 conn(0x7fe870078340 msgr2=0x7fe870100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:52:07.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:07.119+0000 7fe877594700 1 -- 172.21.15.52:0/946891185 shutdown_connections 2024-06-10T02:52:07.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:07.119+0000 7fe877594700 1 -- 172.21.15.52:0/946891185 wait complete. 2024-06-10T02:52:07.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:52:06 smithi120 ceph-mon[36634]: pgmap v371: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-10T02:52:07.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:52:06 smithi188 ceph-mon[36599]: pgmap v371: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-10T02:52:07.301 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:07.302+0000 7f9fc66ee700 1 -- 172.21.15.52:0/2835060915 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9fc0101390 msgr2=0x7f9fc0101810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:52:07.301 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:07.302+0000 7f9fc66ee700 1 --2- 172.21.15.52:0/2835060915 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9fc0101390 0x7f9fc0101810 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f9fb4009990 tx=0x7f9fb4009ca0 comp rx=0 tx=0).stop 2024-06-10T02:52:07.301 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:07.302+0000 7f9fc66ee700 1 -- 172.21.15.52:0/2835060915 shutdown_connections 2024-06-10T02:52:07.301 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:07.302+0000 7f9fc66ee700 1 --2- 172.21.15.52:0/2835060915 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9fc0101d50 0x7f9fc010e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:07.301 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:07.302+0000 7f9fc66ee700 1 --2- 172.21.15.52:0/2835060915 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9fc0107400 0x7f9fc01077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:07.301 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:07.302+0000 7f9fc66ee700 1 --2- 172.21.15.52:0/2835060915 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9fc0101390 0x7f9fc0101810 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:07.301 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:07.302+0000 7f9fc66ee700 1 -- 172.21.15.52:0/2835060915 >> 172.21.15.52:0/2835060915 conn(0x7f9fc0078340 msgr2=0x7f9fc0078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:52:07.302 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:07.302+0000 7f9fc66ee700 1 -- 172.21.15.52:0/2835060915 shutdown_connections 2024-06-10T02:52:07.302 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:07.302+0000 7f9fc66ee700 1 -- 172.21.15.52:0/2835060915 wait complete. 2024-06-10T02:52:07.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:07.302+0000 7f9fc66ee700 1 Processor -- start 2024-06-10T02:52:07.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:07.303+0000 7f9fc66ee700 1 -- start start 2024-06-10T02:52:07.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:07.303+0000 7f9fc66ee700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9fc0101390 0x7f9fc0103070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:52:07.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:07.303+0000 7f9fc66ee700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9fc0101d50 0x7f9fc01035b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:52:07.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:07.303+0000 7f9fc66ee700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9fc0107400 0x7f9fc0103af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:52:07.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:07.303+0000 7f9fc66ee700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f9fc01116a0 con 0x7f9fc0107400 2024-06-10T02:52:07.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:07.303+0000 7f9fc66ee700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f9fc0111520 con 0x7f9fc0101d50 2024-06-10T02:52:07.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:07.303+0000 7f9fc66ee700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f9fc0111820 con 0x7f9fc0101390 2024-06-10T02:52:07.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:07.303+0000 7f9fc4c8b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9fc0107400 0x7f9fc0103af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:52:07.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:07.303+0000 7f9fc4c8b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9fc0107400 0x7f9fc0103af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60106/0 (socket says 172.21.15.52:60106) 2024-06-10T02:52:07.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:07.303+0000 7f9fc4c8b700 1 -- 172.21.15.52:0/1213473199 learned_addr learned my addr 172.21.15.52:0/1213473199 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:52:07.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:07.303+0000 7f9fbf7fe700 1 --2- 172.21.15.52:0/1213473199 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9fc0101d50 0x7f9fc01035b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:52:07.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:07.303+0000 7f9fbffff700 1 --2- 172.21.15.52:0/1213473199 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9fc0101390 0x7f9fc0103070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:52:07.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:07.303+0000 7f9fc4c8b700 1 -- 172.21.15.52:0/1213473199 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9fc0101390 msgr2=0x7f9fc0103070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:52:07.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:07.303+0000 7f9fc4c8b700 1 --2- 172.21.15.52:0/1213473199 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9fc0101390 0x7f9fc0103070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:07.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:07.303+0000 7f9fc4c8b700 1 -- 172.21.15.52:0/1213473199 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9fc0101d50 msgr2=0x7f9fc01035b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:52:07.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:07.303+0000 7f9fc4c8b700 1 --2- 172.21.15.52:0/1213473199 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9fc0101d50 0x7f9fc01035b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:07.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:07.303+0000 7f9fc4c8b700 1 -- 172.21.15.52:0/1213473199 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fc01a7170 con 0x7f9fc0107400 2024-06-10T02:52:07.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:07.304+0000 7f9fc4c8b700 1 --2- 172.21.15.52:0/1213473199 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9fc0107400 0x7f9fc0103af0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f9fb800ada0 tx=0x7f9fb800c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:52:07.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:07.304+0000 7f9fbd7fa700 1 -- 172.21.15.52:0/1213473199 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fb8017020 con 0x7f9fc0107400 2024-06-10T02:52:07.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:07.304+0000 7f9fc66ee700 1 -- 172.21.15.52:0/1213473199 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fc01a7460 con 0x7f9fc0107400 2024-06-10T02:52:07.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:07.304+0000 7f9fbd7fa700 1 -- 172.21.15.52:0/1213473199 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9fb800a0d0 con 0x7f9fc0107400 2024-06-10T02:52:07.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:07.304+0000 7f9fbd7fa700 1 -- 172.21.15.52:0/1213473199 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fb8005480 con 0x7f9fc0107400 2024-06-10T02:52:07.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:07.304+0000 7f9fc66ee700 1 -- 172.21.15.52:0/1213473199 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fc01a79c0 con 0x7f9fc0107400 2024-06-10T02:52:07.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:07.305+0000 7f9fc66ee700 1 -- 172.21.15.52:0/1213473199 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9fc0105420 con 0x7f9fc0107400 2024-06-10T02:52:07.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:07.306+0000 7f9fbd7fa700 1 -- 172.21.15.52:0/1213473199 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f9fb8005620 con 0x7f9fc0107400 2024-06-10T02:52:07.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:07.307+0000 7f9fbd7fa700 1 --2- 172.21.15.52:0/1213473199 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f9fa80a2d30 0x7f9fa80a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:52:07.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:07.307+0000 7f9fbffff700 1 --2- 172.21.15.52:0/1213473199 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f9fa80a2d30 0x7f9fa80a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:52:07.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:07.307+0000 7f9fbd7fa700 1 -- 172.21.15.52:0/1213473199 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f9fb80cec90 con 0x7f9fc0107400 2024-06-10T02:52:07.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:07.308+0000 7f9fbd7fa700 1 --2- 172.21.15.52:0/1213473199 >> [v2:172.21.15.52:6810/116897011,v1:172.21.15.52:6811/116897011] conn(0x7f9fa80ae010 0x7f9fa80b04b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:52:07.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:07.308+0000 7f9fbd7fa700 1 -- 172.21.15.52:0/1213473199 --> [v2:172.21.15.52:6810/116897011,v1:172.21.15.52:6811/116897011] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9fb801de70 con 0x7f9fa80ae010 2024-06-10T02:52:07.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:07.308+0000 7f9fbd7fa700 1 -- 172.21.15.52:0/1213473199 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_get_version_reply(handle=1 version=100) ==== 24+0+0 (secure 0 0 0) 0x7f9fb80d42a0 con 0x7f9fc0107400 2024-06-10T02:52:07.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:07.308+0000 7f9fbf7fe700 1 --2- 172.21.15.52:0/1213473199 >> [v2:172.21.15.52:6810/116897011,v1:172.21.15.52:6811/116897011] conn(0x7f9fa80ae010 0x7f9fa80b04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:52:07.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:07.308+0000 7f9fbffff700 1 --2- 172.21.15.52:0/1213473199 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f9fa80a2d30 0x7f9fa80a51f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f9fb0009bd0 tx=0x7f9fb0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:52:07.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:07.308+0000 7f9fbf7fe700 1 --2- 172.21.15.52:0/1213473199 >> [v2:172.21.15.52:6810/116897011,v1:172.21.15.52:6811/116897011] conn(0x7f9fa80ae010 0x7f9fa80b04b0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:52:07.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:07.309+0000 7f9fbd7fa700 1 -- 172.21.15.52:0/1213473199 <== osd.3 v2:172.21.15.52:6810/116897011 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9fb801de70 con 0x7f9fa80ae010 2024-06-10T02:52:07.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:07.325+0000 7f9fc66ee700 1 -- 172.21.15.52:0/1213473199 --> [v2:172.21.15.52:6810/116897011,v1:172.21.15.52:6811/116897011] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9fc0102ab0 con 0x7f9fa80ae010 2024-06-10T02:52:07.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:07.326+0000 7f9fbd7fa700 1 -- 172.21.15.52:0/1213473199 <== osd.3 v2:172.21.15.52:6810/116897011 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f9fc0102ab0 con 0x7f9fa80ae010 2024-06-10T02:52:07.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:07.326+0000 7f9fc66ee700 1 -- 172.21.15.52:0/1213473199 >> [v2:172.21.15.52:6810/116897011,v1:172.21.15.52:6811/116897011] conn(0x7f9fa80ae010 msgr2=0x7f9fa80b04b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:52:07.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:07.326+0000 7f9fc66ee700 1 --2- 172.21.15.52:0/1213473199 >> [v2:172.21.15.52:6810/116897011,v1:172.21.15.52:6811/116897011] conn(0x7f9fa80ae010 0x7f9fa80b04b0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:07.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:07.326+0000 7f9fc66ee700 1 -- 172.21.15.52:0/1213473199 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f9fa80a2d30 msgr2=0x7f9fa80a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:52:07.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:07.326+0000 7f9fc66ee700 1 --2- 172.21.15.52:0/1213473199 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f9fa80a2d30 0x7f9fa80a51f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f9fb0009bd0 tx=0x7f9fb0009260 comp rx=0 tx=0).stop 2024-06-10T02:52:07.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:07.326+0000 7f9fc66ee700 1 -- 172.21.15.52:0/1213473199 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9fc0107400 msgr2=0x7f9fc0103af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:52:07.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:07.326+0000 7f9fc66ee700 1 --2- 172.21.15.52:0/1213473199 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9fc0107400 0x7f9fc0103af0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f9fb800ada0 tx=0x7f9fb800c330 comp rx=0 tx=0).stop 2024-06-10T02:52:07.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:07.326+0000 7f9fc4c8b700 1 -- 172.21.15.52:0/1213473199 reap_dead start 2024-06-10T02:52:07.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:07.327+0000 7f9fc66ee700 1 -- 172.21.15.52:0/1213473199 shutdown_connections 2024-06-10T02:52:07.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:07.327+0000 7f9fc66ee700 1 -- 172.21.15.52:0/1213473199 >> 172.21.15.52:0/1213473199 conn(0x7f9fc0078340 msgr2=0x7f9fc0100670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:52:07.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:07.327+0000 7f9fc66ee700 1 -- 172.21.15.52:0/1213473199 shutdown_connections 2024-06-10T02:52:07.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:07.327+0000 7f9fc66ee700 1 -- 172.21.15.52:0/1213473199 wait complete. 2024-06-10T02:52:08.770 INFO:teuthology.orchestra.run.smithi052.stdout:111669149778 2024-06-10T02:52:08.771 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-06-10T02:52:09.006 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:52:08 smithi052 ceph-mon[32362]: pgmap v372: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-10T02:52:09.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:52:08 smithi120 ceph-mon[36634]: pgmap v372: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-10T02:52:09.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:52:08 smithi188 ceph-mon[36599]: pgmap v372: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-10T02:52:10.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:10.532+0000 7f07c8d29700 1 -- 172.21.15.52:0/2486263921 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f07c4101610 msgr2=0x7f07c41019f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:52:10.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:10.532+0000 7f07c8d29700 1 --2- 172.21.15.52:0/2486263921 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f07c4101610 0x7f07c41019f0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f07b40099c0 tx=0x7f07b4009cd0 comp rx=0 tx=0).stop 2024-06-10T02:52:10.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:10.532+0000 7f07c8d29700 1 -- 172.21.15.52:0/2486263921 shutdown_connections 2024-06-10T02:52:10.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:10.532+0000 7f07c8d29700 1 --2- 172.21.15.52:0/2486263921 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f07c41066f0 0x7f07c410e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:10.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:10.532+0000 7f07c8d29700 1 --2- 172.21.15.52:0/2486263921 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f07c4101610 0x7f07c41019f0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:10.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:10.532+0000 7f07c8d29700 1 --2- 172.21.15.52:0/2486263921 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f07c4101fc0 0x7f07c4106010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:10.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:10.532+0000 7f07c8d29700 1 -- 172.21.15.52:0/2486263921 >> 172.21.15.52:0/2486263921 conn(0x7f07c4078430 msgr2=0x7f07c4078840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:52:10.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:10.532+0000 7f07c8d29700 1 -- 172.21.15.52:0/2486263921 shutdown_connections 2024-06-10T02:52:10.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:10.532+0000 7f07c8d29700 1 -- 172.21.15.52:0/2486263921 wait complete. 2024-06-10T02:52:10.534 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:10.533+0000 7f07c8d29700 1 Processor -- start 2024-06-10T02:52:10.534 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:10.533+0000 7f07c8d29700 1 -- start start 2024-06-10T02:52:10.534 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:10.534+0000 7f07c8d29700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f07c4101610 0x7f07c4198de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:52:10.534 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:10.534+0000 7f07c8d29700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f07c4101fc0 0x7f07c4199320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:52:10.535 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:10.534+0000 7f07c8d29700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f07c41066f0 0x7f07c419d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:52:10.535 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:10.534+0000 7f07c8d29700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f07c41117d0 con 0x7f07c41066f0 2024-06-10T02:52:10.535 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:10.534+0000 7f07c8d29700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f07c4111650 con 0x7f07c4101fc0 2024-06-10T02:52:10.535 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:10.534+0000 7f07c8d29700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f07c4111950 con 0x7f07c4101610 2024-06-10T02:52:10.535 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:10.534+0000 7f07c2d9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f07c41066f0 0x7f07c419d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:52:10.535 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:10.534+0000 7f07c2d9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f07c41066f0 0x7f07c419d6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60120/0 (socket says 172.21.15.52:60120) 2024-06-10T02:52:10.535 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:10.534+0000 7f07c2d9d700 1 -- 172.21.15.52:0/151217273 learned_addr learned my addr 172.21.15.52:0/151217273 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:52:10.535 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:10.534+0000 7f07c259c700 1 --2- 172.21.15.52:0/151217273 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f07c4101610 0x7f07c4198de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:52:10.535 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:10.534+0000 7f07b9d9b700 1 --2- 172.21.15.52:0/151217273 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f07c4101fc0 0x7f07c4199320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:52:10.535 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:10.534+0000 7f07c2d9d700 1 -- 172.21.15.52:0/151217273 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f07c4101610 msgr2=0x7f07c4198de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:52:10.535 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:10.534+0000 7f07c2d9d700 1 --2- 172.21.15.52:0/151217273 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f07c4101610 0x7f07c4198de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:10.535 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:10.534+0000 7f07c2d9d700 1 -- 172.21.15.52:0/151217273 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f07c4101fc0 msgr2=0x7f07c4199320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:52:10.536 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:10.534+0000 7f07c2d9d700 1 --2- 172.21.15.52:0/151217273 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f07c4101fc0 0x7f07c4199320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:10.536 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:10.534+0000 7f07c2d9d700 1 -- 172.21.15.52:0/151217273 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07c419ddf0 con 0x7f07c41066f0 2024-06-10T02:52:10.536 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:10.534+0000 7f07c2d9d700 1 --2- 172.21.15.52:0/151217273 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f07c41066f0 0x7f07c419d6b0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f07bc00b970 tx=0x7f07bc00bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:52:10.536 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:10.535+0000 7f07bbfff700 1 -- 172.21.15.52:0/151217273 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07bc00d860 con 0x7f07c41066f0 2024-06-10T02:52:10.536 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:10.535+0000 7f07bbfff700 1 -- 172.21.15.52:0/151217273 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f07bc00e430 con 0x7f07c41066f0 2024-06-10T02:52:10.536 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:10.535+0000 7f07c8d29700 1 -- 172.21.15.52:0/151217273 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07c419e0e0 con 0x7f07c41066f0 2024-06-10T02:52:10.536 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:10.535+0000 7f07bbfff700 1 -- 172.21.15.52:0/151217273 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07bc017660 con 0x7f07c41066f0 2024-06-10T02:52:10.536 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:10.535+0000 7f07c8d29700 1 -- 172.21.15.52:0/151217273 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07c41a5960 con 0x7f07c41066f0 2024-06-10T02:52:10.536 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:10.535+0000 7f07c8d29700 1 -- 172.21.15.52:0/151217273 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f07c4062c20 con 0x7f07c41066f0 2024-06-10T02:52:10.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:10.537+0000 7f07bbfff700 1 -- 172.21.15.52:0/151217273 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f07bc00e5e0 con 0x7f07c41066f0 2024-06-10T02:52:10.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:10.538+0000 7f07bbfff700 1 --2- 172.21.15.52:0/151217273 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f07b00a2d30 0x7f07b00a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:52:10.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:10.538+0000 7f07c259c700 1 --2- 172.21.15.52:0/151217273 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f07b00a2d30 0x7f07b00a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:52:10.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:10.538+0000 7f07bbfff700 1 -- 172.21.15.52:0/151217273 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f07bc0cf8e0 con 0x7f07c41066f0 2024-06-10T02:52:10.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:10.538+0000 7f07bbfff700 1 --2- 172.21.15.52:0/151217273 >> [v2:172.21.15.120:6816/3737722011,v1:172.21.15.120:6817/3737722011] conn(0x7f07b00ae010 0x7f07b00b04b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:52:10.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:10.538+0000 7f07bbfff700 1 -- 172.21.15.52:0/151217273 --> [v2:172.21.15.120:6816/3737722011,v1:172.21.15.120:6817/3737722011] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f07b00b0b80 con 0x7f07b00ae010 2024-06-10T02:52:10.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:10.538+0000 7f07bbfff700 1 -- 172.21.15.52:0/151217273 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_get_version_reply(handle=1 version=100) ==== 24+0+0 (secure 0 0 0) 0x7f07bc0cfd30 con 0x7f07c41066f0 2024-06-10T02:52:10.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:10.538+0000 7f07c259c700 1 --2- 172.21.15.52:0/151217273 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f07b00a2d30 0x7f07b00a51f0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f07b4000c00 tx=0x7f07b4005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:52:10.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:10.538+0000 7f07b9d9b700 1 --2- 172.21.15.52:0/151217273 >> [v2:172.21.15.120:6816/3737722011,v1:172.21.15.120:6817/3737722011] conn(0x7f07b00ae010 0x7f07b00b04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:52:10.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:10.539+0000 7f07b9d9b700 1 --2- 172.21.15.52:0/151217273 >> [v2:172.21.15.120:6816/3737722011,v1:172.21.15.120:6817/3737722011] conn(0x7f07b00ae010 0x7f07b00b04b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:52:10.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:10.540+0000 7f07bbfff700 1 -- 172.21.15.52:0/151217273 <== osd.7 v2:172.21.15.120:6816/3737722011 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f07b00b0b80 con 0x7f07b00ae010 2024-06-10T02:52:10.555 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:10.556+0000 7f07c8d29700 1 -- 172.21.15.52:0/151217273 --> [v2:172.21.15.120:6816/3737722011,v1:172.21.15.120:6817/3737722011] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f07c4063020 con 0x7f07b00ae010 2024-06-10T02:52:10.555 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:10.557+0000 7f07bbfff700 1 -- 172.21.15.52:0/151217273 <== osd.7 v2:172.21.15.120:6816/3737722011 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f07c4063020 con 0x7f07b00ae010 2024-06-10T02:52:10.556 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:10.557+0000 7f07c8d29700 1 -- 172.21.15.52:0/151217273 >> [v2:172.21.15.120:6816/3737722011,v1:172.21.15.120:6817/3737722011] conn(0x7f07b00ae010 msgr2=0x7f07b00b04b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:52:10.556 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:10.557+0000 7f07c8d29700 1 --2- 172.21.15.52:0/151217273 >> [v2:172.21.15.120:6816/3737722011,v1:172.21.15.120:6817/3737722011] conn(0x7f07b00ae010 0x7f07b00b04b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:10.557 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:10.557+0000 7f07c8d29700 1 -- 172.21.15.52:0/151217273 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f07b00a2d30 msgr2=0x7f07b00a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:52:10.557 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:10.557+0000 7f07c8d29700 1 --2- 172.21.15.52:0/151217273 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f07b00a2d30 0x7f07b00a51f0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f07b4000c00 tx=0x7f07b4005fb0 comp rx=0 tx=0).stop 2024-06-10T02:52:10.557 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:10.557+0000 7f07c8d29700 1 -- 172.21.15.52:0/151217273 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f07c41066f0 msgr2=0x7f07c419d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:52:10.557 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:10.557+0000 7f07c8d29700 1 --2- 172.21.15.52:0/151217273 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f07c41066f0 0x7f07c419d6b0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f07bc00b970 tx=0x7f07bc00bc80 comp rx=0 tx=0).stop 2024-06-10T02:52:10.557 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:10.557+0000 7f07c2d9d700 1 -- 172.21.15.52:0/151217273 reap_dead start 2024-06-10T02:52:10.557 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:10.557+0000 7f07c8d29700 1 -- 172.21.15.52:0/151217273 shutdown_connections 2024-06-10T02:52:10.557 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:10.557+0000 7f07c8d29700 1 -- 172.21.15.52:0/151217273 >> 172.21.15.52:0/151217273 conn(0x7f07c4078430 msgr2=0x7f07c406af30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:52:10.557 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:10.558+0000 7f07c8d29700 1 -- 172.21.15.52:0/151217273 shutdown_connections 2024-06-10T02:52:10.557 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:10.558+0000 7f07c8d29700 1 -- 172.21.15.52:0/151217273 wait complete. 2024-06-10T02:52:10.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:52:10 smithi052 ceph-mon[32362]: pgmap v373: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-06-10T02:52:11.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:52:10 smithi120 ceph-mon[36634]: pgmap v373: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-06-10T02:52:11.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:52:10 smithi188 ceph-mon[36599]: pgmap v373: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-06-10T02:52:11.376 INFO:teuthology.orchestra.run.smithi052.stdout:167503724615 2024-06-10T02:52:11.377 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-06-10T02:52:11.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:11.545+0000 7f6730f5e700 1 -- 172.21.15.52:0/1203262030 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f672c06ab30 msgr2=0x7f672c10f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:52:11.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:11.545+0000 7f6730f5e700 1 --2- 172.21.15.52:0/1203262030 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f672c06ab30 0x7f672c10f320 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f672400b0a0 tx=0x7f672400b3b0 comp rx=0 tx=0).stop 2024-06-10T02:52:11.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:11.545+0000 7f6730f5e700 1 -- 172.21.15.52:0/1203262030 shutdown_connections 2024-06-10T02:52:11.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:11.545+0000 7f6730f5e700 1 --2- 172.21.15.52:0/1203262030 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f672c06ab30 0x7f672c10f320 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:11.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:11.545+0000 7f6730f5e700 1 --2- 172.21.15.52:0/1203262030 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f672c108500 0x7f672c1088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:11.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:11.545+0000 7f6730f5e700 1 --2- 172.21.15.52:0/1203262030 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f672c06a170 0x7f672c06a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:11.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:11.545+0000 7f6730f5e700 1 -- 172.21.15.52:0/1203262030 >> 172.21.15.52:0/1203262030 conn(0x7f672c078340 msgr2=0x7f672c078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:52:11.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:11.545+0000 7f6730f5e700 1 -- 172.21.15.52:0/1203262030 shutdown_connections 2024-06-10T02:52:11.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:11.545+0000 7f6730f5e700 1 -- 172.21.15.52:0/1203262030 wait complete. 2024-06-10T02:52:11.546 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:11.546+0000 7f6730f5e700 1 Processor -- start 2024-06-10T02:52:11.546 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:11.547+0000 7f6730f5e700 1 -- start start 2024-06-10T02:52:11.546 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:11.547+0000 7f6730f5e700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f672c06a170 0x7f672c19e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:52:11.546 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:11.547+0000 7f6730f5e700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f672c06ab30 0x7f672c19edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:52:11.546 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:11.547+0000 7f672a59c700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f672c06a170 0x7f672c19e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:52:11.547 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:11.547+0000 7f672a59c700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f672c06a170 0x7f672c19e880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.52:40714/0 (socket says 172.21.15.52:40714) 2024-06-10T02:52:11.547 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:11.547+0000 7f6730f5e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f672c108500 0x7f672c198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:52:11.547 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:11.547+0000 7f672a59c700 1 -- 172.21.15.52:0/1184425702 learned_addr learned my addr 172.21.15.52:0/1184425702 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:52:11.547 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:11.547+0000 7f6721d9b700 1 --2- 172.21.15.52:0/1184425702 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f672c06ab30 0x7f672c19edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:52:11.547 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:11.547+0000 7f672ad9d700 1 --2- 172.21.15.52:0/1184425702 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f672c108500 0x7f672c198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:52:11.547 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:11.547+0000 7f6730f5e700 1 -- 172.21.15.52:0/1184425702 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f672c112780 con 0x7f672c108500 2024-06-10T02:52:11.547 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:11.547+0000 7f6730f5e700 1 -- 172.21.15.52:0/1184425702 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f672c112600 con 0x7f672c06a170 2024-06-10T02:52:11.547 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:11.547+0000 7f6730f5e700 1 -- 172.21.15.52:0/1184425702 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f672c112900 con 0x7f672c06ab30 2024-06-10T02:52:11.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:11.548+0000 7f6721d9b700 1 -- 172.21.15.52:0/1184425702 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f672c06a170 msgr2=0x7f672c19e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:52:11.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:11.548+0000 7f6721d9b700 1 --2- 172.21.15.52:0/1184425702 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f672c06a170 0x7f672c19e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:11.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:11.548+0000 7f6721d9b700 1 -- 172.21.15.52:0/1184425702 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f672c108500 msgr2=0x7f672c198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:52:11.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:11.548+0000 7f6721d9b700 1 --2- 172.21.15.52:0/1184425702 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f672c108500 0x7f672c198a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:11.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:11.548+0000 7f6721d9b700 1 -- 172.21.15.52:0/1184425702 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f672c1991a0 con 0x7f672c06ab30 2024-06-10T02:52:11.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:11.548+0000 7f672ad9d700 1 --2- 172.21.15.52:0/1184425702 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f672c108500 0x7f672c198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T02:52:11.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:11.548+0000 7f672a59c700 1 --2- 172.21.15.52:0/1184425702 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f672c06a170 0x7f672c19e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T02:52:11.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:11.548+0000 7f6721d9b700 1 --2- 172.21.15.52:0/1184425702 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f672c06ab30 0x7f672c19edc0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f671400d750 tx=0x7f671400da60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:52:11.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:11.548+0000 7f6723fff700 1 -- 172.21.15.52:0/1184425702 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6714016070 con 0x7f672c06ab30 2024-06-10T02:52:11.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:11.549+0000 7f6723fff700 1 -- 172.21.15.52:0/1184425702 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6714005020 con 0x7f672c06ab30 2024-06-10T02:52:11.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:11.549+0000 7f6723fff700 1 -- 172.21.15.52:0/1184425702 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f671400fb30 con 0x7f672c06ab30 2024-06-10T02:52:11.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:11.549+0000 7f6730f5e700 1 -- 172.21.15.52:0/1184425702 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f672c199490 con 0x7f672c06ab30 2024-06-10T02:52:11.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:11.549+0000 7f6730f5e700 1 -- 172.21.15.52:0/1184425702 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f672c1a5750 con 0x7f672c06ab30 2024-06-10T02:52:11.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:11.549+0000 7f6730f5e700 1 -- 172.21.15.52:0/1184425702 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f672c062b60 con 0x7f672c06ab30 2024-06-10T02:52:11.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:11.552+0000 7f6723fff700 1 -- 172.21.15.52:0/1184425702 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f67140051d0 con 0x7f672c06ab30 2024-06-10T02:52:11.552 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:11.553+0000 7f6723fff700 1 --2- 172.21.15.52:0/1184425702 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f67180a2d30 0x7f67180a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:52:11.552 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:11.553+0000 7f6723fff700 1 -- 172.21.15.52:0/1184425702 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f67140cf2e0 con 0x7f672c06ab30 2024-06-10T02:52:11.552 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:11.553+0000 7f672a59c700 1 --2- 172.21.15.52:0/1184425702 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f67180a2d30 0x7f67180a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:52:11.552 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:11.553+0000 7f6723fff700 1 --2- 172.21.15.52:0/1184425702 >> [v2:172.21.15.120:6808/1320690621,v1:172.21.15.120:6809/1320690621] conn(0x7f67180ae010 0x7f67180b04b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:52:11.552 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:11.553+0000 7f6723fff700 1 -- 172.21.15.52:0/1184425702 --> [v2:172.21.15.120:6808/1320690621,v1:172.21.15.120:6809/1320690621] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6714014e80 con 0x7f67180ae010 2024-06-10T02:52:11.552 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:11.553+0000 7f6723fff700 1 -- 172.21.15.52:0/1184425702 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_get_version_reply(handle=1 version=100) ==== 24+0+0 (secure 0 0 0) 0x7f67140cf690 con 0x7f672c06ab30 2024-06-10T02:52:11.553 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:11.553+0000 7f672a59c700 1 --2- 172.21.15.52:0/1184425702 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f67180a2d30 0x7f67180a51f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f672c199b80 tx=0x7f671c00b500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:52:11.553 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:11.553+0000 7f672ad9d700 1 --2- 172.21.15.52:0/1184425702 >> [v2:172.21.15.120:6808/1320690621,v1:172.21.15.120:6809/1320690621] conn(0x7f67180ae010 0x7f67180b04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:52:11.553 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:11.554+0000 7f672ad9d700 1 --2- 172.21.15.52:0/1184425702 >> [v2:172.21.15.120:6808/1320690621,v1:172.21.15.120:6809/1320690621] conn(0x7f67180ae010 0x7f67180b04b0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:52:11.553 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:11.555+0000 7f6723fff700 1 -- 172.21.15.52:0/1184425702 <== osd.4 v2:172.21.15.120:6808/1320690621 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6714014e80 con 0x7f67180ae010 2024-06-10T02:52:11.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:11.573+0000 7f6730f5e700 1 -- 172.21.15.52:0/1184425702 --> [v2:172.21.15.120:6808/1320690621,v1:172.21.15.120:6809/1320690621] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f672c062fe0 con 0x7f67180ae010 2024-06-10T02:52:11.572 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:11.573+0000 7f6723fff700 1 -- 172.21.15.52:0/1184425702 <== osd.4 v2:172.21.15.120:6808/1320690621 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f672c062fe0 con 0x7f67180ae010 2024-06-10T02:52:11.573 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:11.574+0000 7f6730f5e700 1 -- 172.21.15.52:0/1184425702 >> [v2:172.21.15.120:6808/1320690621,v1:172.21.15.120:6809/1320690621] conn(0x7f67180ae010 msgr2=0x7f67180b04b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:52:11.573 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:11.574+0000 7f6730f5e700 1 --2- 172.21.15.52:0/1184425702 >> [v2:172.21.15.120:6808/1320690621,v1:172.21.15.120:6809/1320690621] conn(0x7f67180ae010 0x7f67180b04b0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:11.573 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:11.574+0000 7f6730f5e700 1 -- 172.21.15.52:0/1184425702 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f67180a2d30 msgr2=0x7f67180a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:52:11.573 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:11.574+0000 7f6730f5e700 1 --2- 172.21.15.52:0/1184425702 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f67180a2d30 0x7f67180a51f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f672c199b80 tx=0x7f671c00b500 comp rx=0 tx=0).stop 2024-06-10T02:52:11.573 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:11.574+0000 7f6730f5e700 1 -- 172.21.15.52:0/1184425702 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f672c06ab30 msgr2=0x7f672c19edc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:52:11.573 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:11.574+0000 7f6730f5e700 1 --2- 172.21.15.52:0/1184425702 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f672c06ab30 0x7f672c19edc0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f671400d750 tx=0x7f671400da60 comp rx=0 tx=0).stop 2024-06-10T02:52:11.573 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:11.574+0000 7f672ad9d700 1 -- 172.21.15.52:0/1184425702 reap_dead start 2024-06-10T02:52:11.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:11.574+0000 7f6730f5e700 1 -- 172.21.15.52:0/1184425702 shutdown_connections 2024-06-10T02:52:11.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:11.574+0000 7f6730f5e700 1 -- 172.21.15.52:0/1184425702 >> 172.21.15.52:0/1184425702 conn(0x7f672c078340 msgr2=0x7f672c0fff10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:52:11.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:11.574+0000 7f6730f5e700 1 -- 172.21.15.52:0/1184425702 shutdown_connections 2024-06-10T02:52:11.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:11.574+0000 7f6730f5e700 1 -- 172.21.15.52:0/1184425702 wait complete. 2024-06-10T02:52:11.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:11.045+0000 7f6348671700 1 -- 172.21.15.52:0/3201763513 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6340101d50 msgr2=0x7f634010e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:52:11.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:11.045+0000 7f6348671700 1 --2- 172.21.15.52:0/3201763513 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6340101d50 0x7f634010e240 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f633800b0a0 tx=0x7f633800b3b0 comp rx=0 tx=0).stop 2024-06-10T02:52:11.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:11.045+0000 7f6348671700 1 -- 172.21.15.52:0/3201763513 shutdown_connections 2024-06-10T02:52:11.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:11.045+0000 7f6348671700 1 --2- 172.21.15.52:0/3201763513 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6340101d50 0x7f634010e240 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:11.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:11.045+0000 7f6348671700 1 --2- 172.21.15.52:0/3201763513 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6340107400 0x7f63401077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:11.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:11.046+0000 7f6348671700 1 --2- 172.21.15.52:0/3201763513 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6340101390 0x7f6340101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:11.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:11.046+0000 7f6348671700 1 -- 172.21.15.52:0/3201763513 >> 172.21.15.52:0/3201763513 conn(0x7f6340078340 msgr2=0x7f6340078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:52:11.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:11.046+0000 7f6348671700 1 -- 172.21.15.52:0/3201763513 shutdown_connections 2024-06-10T02:52:11.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:11.046+0000 7f6348671700 1 -- 172.21.15.52:0/3201763513 wait complete. 2024-06-10T02:52:11.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:11.046+0000 7f6348671700 1 Processor -- start 2024-06-10T02:52:11.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:11.046+0000 7f6348671700 1 -- start start 2024-06-10T02:52:11.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:11.047+0000 7f6348671700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6340101390 0x7f634019d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:52:11.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:11.047+0000 7f6348671700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6340101d50 0x7f634019dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:52:11.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:11.047+0000 7f6348671700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6340107400 0x7f6340197950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:52:11.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:11.047+0000 7f6348671700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f6340111690 con 0x7f6340101d50 2024-06-10T02:52:11.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:11.047+0000 7f6348671700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f6340111510 con 0x7f6340101390 2024-06-10T02:52:11.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:11.047+0000 7f6348671700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f6340111810 con 0x7f6340107400 2024-06-10T02:52:11.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:11.047+0000 7f6345c0c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6340101d50 0x7f634019dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:52:11.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:11.047+0000 7f6345c0c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6340101d50 0x7f634019dc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48304/0 (socket says 172.21.15.52:48304) 2024-06-10T02:52:11.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:11.047+0000 7f6345c0c700 1 -- 172.21.15.52:0/2789655410 learned_addr learned my addr 172.21.15.52:0/2789655410 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:52:11.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:11.047+0000 7f634640d700 1 --2- 172.21.15.52:0/2789655410 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6340101390 0x7f634019d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:52:11.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:11.047+0000 7f6346c0e700 1 --2- 172.21.15.52:0/2789655410 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6340107400 0x7f6340197950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:52:11.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:11.047+0000 7f6345c0c700 1 -- 172.21.15.52:0/2789655410 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6340107400 msgr2=0x7f6340197950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:52:11.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:11.047+0000 7f6345c0c700 1 --2- 172.21.15.52:0/2789655410 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6340107400 0x7f6340197950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:11.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:11.047+0000 7f6345c0c700 1 -- 172.21.15.52:0/2789655410 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6340101390 msgr2=0x7f634019d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:52:11.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:11.047+0000 7f6345c0c700 1 --2- 172.21.15.52:0/2789655410 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6340101390 0x7f634019d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:11.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:11.047+0000 7f6345c0c700 1 -- 172.21.15.52:0/2789655410 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63401980b0 con 0x7f6340101d50 2024-06-10T02:52:11.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:11.047+0000 7f634640d700 1 --2- 172.21.15.52:0/2789655410 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6340101390 0x7f634019d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T02:52:11.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:11.047+0000 7f6345c0c700 1 --2- 172.21.15.52:0/2789655410 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6340101d50 0x7f634019dc80 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f6330009970 tx=0x7f6330009c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:52:11.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:11.048+0000 7f63377fe700 1 -- 172.21.15.52:0/2789655410 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f633001c070 con 0x7f6340101d50 2024-06-10T02:52:11.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:11.048+0000 7f6348671700 1 -- 172.21.15.52:0/2789655410 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63401983a0 con 0x7f6340101d50 2024-06-10T02:52:11.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:11.048+0000 7f63377fe700 1 -- 172.21.15.52:0/2789655410 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f633000d260 con 0x7f6340101d50 2024-06-10T02:52:11.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:11.048+0000 7f63377fe700 1 -- 172.21.15.52:0/2789655410 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6330017a60 con 0x7f6340101d50 2024-06-10T02:52:11.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:11.048+0000 7f6348671700 1 -- 172.21.15.52:0/2789655410 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f634006a170 con 0x7f6340101d50 2024-06-10T02:52:11.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:11.048+0000 7f6348671700 1 -- 172.21.15.52:0/2789655410 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f634006a760 con 0x7f6340101d50 2024-06-10T02:52:11.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:11.050+0000 7f63377fe700 1 -- 172.21.15.52:0/2789655410 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f633001f070 con 0x7f6340101d50 2024-06-10T02:52:11.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:11.050+0000 7f63377fe700 1 --2- 172.21.15.52:0/2789655410 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f632c0a2cb0 0x7f632c0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:52:11.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:11.051+0000 7f634640d700 1 --2- 172.21.15.52:0/2789655410 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f632c0a2cb0 0x7f632c0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:52:11.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:11.051+0000 7f63377fe700 1 -- 172.21.15.52:0/2789655410 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f63300d0c10 con 0x7f6340101d50 2024-06-10T02:52:11.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:11.051+0000 7f63377fe700 1 --2- 172.21.15.52:0/2789655410 >> [v2:172.21.15.188:6800/2624326999,v1:172.21.15.188:6801/2624326999] conn(0x7f632c0adf90 0x7f632c0b0430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:52:11.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:11.051+0000 7f63377fe700 1 -- 172.21.15.52:0/2789655410 --> [v2:172.21.15.188:6800/2624326999,v1:172.21.15.188:6801/2624326999] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6330023e50 con 0x7f632c0adf90 2024-06-10T02:52:11.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:11.051+0000 7f63377fe700 1 -- 172.21.15.52:0/2789655410 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_get_version_reply(handle=1 version=100) ==== 24+0+0 (secure 0 0 0) 0x7f63300d1060 con 0x7f6340101d50 2024-06-10T02:52:11.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:11.051+0000 7f634640d700 1 --2- 172.21.15.52:0/2789655410 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f632c0a2cb0 0x7f632c0a5170 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f63400ffb00 tx=0x7f633c00b3c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:52:11.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:11.051+0000 7f6346c0e700 1 --2- 172.21.15.52:0/2789655410 >> [v2:172.21.15.188:6800/2624326999,v1:172.21.15.188:6801/2624326999] conn(0x7f632c0adf90 0x7f632c0b0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:52:11.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:11.052+0000 7f6346c0e700 1 --2- 172.21.15.52:0/2789655410 >> [v2:172.21.15.188:6800/2624326999,v1:172.21.15.188:6801/2624326999] conn(0x7f632c0adf90 0x7f632c0b0430 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:52:11.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:11.052+0000 7f63377fe700 1 -- 172.21.15.52:0/2789655410 <== osd.2 v2:172.21.15.188:6800/2624326999 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6330023e50 con 0x7f632c0adf90 2024-06-10T02:52:11.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:11.069+0000 7f6348671700 1 -- 172.21.15.52:0/2789655410 --> [v2:172.21.15.188:6800/2624326999,v1:172.21.15.188:6801/2624326999] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f634006ab60 con 0x7f632c0adf90 2024-06-10T02:52:11.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:11.070+0000 7f63377fe700 1 -- 172.21.15.52:0/2789655410 <== osd.2 v2:172.21.15.188:6800/2624326999 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f634006ab60 con 0x7f632c0adf90 2024-06-10T02:52:11.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:11.070+0000 7f6348671700 1 -- 172.21.15.52:0/2789655410 >> [v2:172.21.15.188:6800/2624326999,v1:172.21.15.188:6801/2624326999] conn(0x7f632c0adf90 msgr2=0x7f632c0b0430 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:52:11.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:11.070+0000 7f6348671700 1 --2- 172.21.15.52:0/2789655410 >> [v2:172.21.15.188:6800/2624326999,v1:172.21.15.188:6801/2624326999] conn(0x7f632c0adf90 0x7f632c0b0430 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:11.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:11.070+0000 7f6348671700 1 -- 172.21.15.52:0/2789655410 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f632c0a2cb0 msgr2=0x7f632c0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:52:11.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:11.070+0000 7f6348671700 1 --2- 172.21.15.52:0/2789655410 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f632c0a2cb0 0x7f632c0a5170 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f63400ffb00 tx=0x7f633c00b3c0 comp rx=0 tx=0).stop 2024-06-10T02:52:11.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:11.070+0000 7f6348671700 1 -- 172.21.15.52:0/2789655410 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6340101d50 msgr2=0x7f634019dc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:52:11.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:11.070+0000 7f6348671700 1 --2- 172.21.15.52:0/2789655410 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6340101d50 0x7f634019dc80 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f6330009970 tx=0x7f6330009c80 comp rx=0 tx=0).stop 2024-06-10T02:52:11.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:11.070+0000 7f6346c0e700 1 -- 172.21.15.52:0/2789655410 reap_dead start 2024-06-10T02:52:11.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:11.070+0000 7f6348671700 1 -- 172.21.15.52:0/2789655410 shutdown_connections 2024-06-10T02:52:11.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:11.070+0000 7f6348671700 1 -- 172.21.15.52:0/2789655410 >> 172.21.15.52:0/2789655410 conn(0x7f6340078340 msgr2=0x7f63400ffe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:52:11.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:11.070+0000 7f6348671700 1 -- 172.21.15.52:0/2789655410 shutdown_connections 2024-06-10T02:52:11.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:11.070+0000 7f6348671700 1 -- 172.21.15.52:0/2789655410 wait complete. 2024-06-10T02:52:11.794 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:10.942+0000 7ff0cef88700 1 -- 172.21.15.52:0/2892185386 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff0c8101510 msgr2=0x7ff0c810d9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:52:11.794 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:10.942+0000 7ff0cef88700 1 --2- 172.21.15.52:0/2892185386 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff0c8101510 0x7ff0c810d9d0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7ff0c000c210 tx=0x7ff0c000c520 comp rx=0 tx=0).stop 2024-06-10T02:52:11.794 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:10.942+0000 7ff0cef88700 1 -- 172.21.15.52:0/2892185386 shutdown_connections 2024-06-10T02:52:11.794 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:10.942+0000 7ff0cef88700 1 --2- 172.21.15.52:0/2892185386 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff0c8101510 0x7ff0c810d9d0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:11.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:10.942+0000 7ff0cef88700 1 --2- 172.21.15.52:0/2892185386 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff0c8106b90 0x7ff0c8106f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:11.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:10.942+0000 7ff0cef88700 1 --2- 172.21.15.52:0/2892185386 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff0c8100b50 0x7ff0c8100fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:11.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:10.942+0000 7ff0cef88700 1 -- 172.21.15.52:0/2892185386 >> 172.21.15.52:0/2892185386 conn(0x7ff0c8077a00 msgr2=0x7ff0c80fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:52:11.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:10.942+0000 7ff0cef88700 1 -- 172.21.15.52:0/2892185386 shutdown_connections 2024-06-10T02:52:11.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:10.942+0000 7ff0cef88700 1 -- 172.21.15.52:0/2892185386 wait complete. 2024-06-10T02:52:11.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:10.943+0000 7ff0cef88700 1 Processor -- start 2024-06-10T02:52:11.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:10.943+0000 7ff0cef88700 1 -- start start 2024-06-10T02:52:11.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:10.943+0000 7ff0cef88700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff0c8100b50 0x7ff0c819e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:52:11.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:10.943+0000 7ff0cef88700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff0c8101510 0x7ff0c819ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:52:11.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:10.943+0000 7ff0cef88700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff0c8106b90 0x7ff0c8198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:52:11.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:10.943+0000 7ff0cef88700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ff0c81105c0 con 0x7ff0c8106b90 2024-06-10T02:52:11.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:10.943+0000 7ff0cef88700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff0c8110440 con 0x7ff0c8101510 2024-06-10T02:52:11.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:10.943+0000 7ff0cef88700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ff0c8110740 con 0x7ff0c8100b50 2024-06-10T02:52:11.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:10.943+0000 7ff0cd525700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff0c8106b90 0x7ff0c8198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:52:11.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:10.943+0000 7ff0cd525700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff0c8106b90 0x7ff0c8198a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48294/0 (socket says 172.21.15.52:48294) 2024-06-10T02:52:11.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:10.943+0000 7ff0cd525700 1 -- 172.21.15.52:0/315807414 learned_addr learned my addr 172.21.15.52:0/315807414 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:52:11.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:10.943+0000 7ff0ccd24700 1 --2- 172.21.15.52:0/315807414 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff0c8100b50 0x7ff0c819e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:52:11.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:10.944+0000 7ff0bffff700 1 --2- 172.21.15.52:0/315807414 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff0c8101510 0x7ff0c819ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:52:11.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:10.944+0000 7ff0cd525700 1 -- 172.21.15.52:0/315807414 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff0c8100b50 msgr2=0x7ff0c819e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:52:11.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:10.944+0000 7ff0cd525700 1 --2- 172.21.15.52:0/315807414 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff0c8100b50 0x7ff0c819e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:11.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:10.944+0000 7ff0cd525700 1 -- 172.21.15.52:0/315807414 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff0c8101510 msgr2=0x7ff0c819ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:52:11.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:10.944+0000 7ff0cd525700 1 --2- 172.21.15.52:0/315807414 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff0c8101510 0x7ff0c819ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:11.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:10.944+0000 7ff0cd525700 1 -- 172.21.15.52:0/315807414 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0c8199190 con 0x7ff0c8106b90 2024-06-10T02:52:11.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:10.944+0000 7ff0bffff700 1 --2- 172.21.15.52:0/315807414 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff0c8101510 0x7ff0c819ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T02:52:11.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:10.944+0000 7ff0cd525700 1 --2- 172.21.15.52:0/315807414 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff0c8106b90 0x7ff0c8198a30 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7ff0c000a6f0 tx=0x7ff0c000a890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:52:11.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:10.944+0000 7ff0bdffb700 1 -- 172.21.15.52:0/315807414 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0c0022020 con 0x7ff0c8106b90 2024-06-10T02:52:11.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:10.944+0000 7ff0bdffb700 1 -- 172.21.15.52:0/315807414 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0c0008d10 con 0x7ff0c8106b90 2024-06-10T02:52:11.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:10.944+0000 7ff0bdffb700 1 -- 172.21.15.52:0/315807414 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0c001e860 con 0x7ff0c8106b90 2024-06-10T02:52:11.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:10.944+0000 7ff0cef88700 1 -- 172.21.15.52:0/315807414 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0c8199420 con 0x7ff0c8106b90 2024-06-10T02:52:11.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:10.944+0000 7ff0cef88700 1 -- 172.21.15.52:0/315807414 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0c81a56f0 con 0x7ff0c8106b90 2024-06-10T02:52:11.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:10.945+0000 7ff0cef88700 1 -- 172.21.15.52:0/315807414 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff0c8062c20 con 0x7ff0c8106b90 2024-06-10T02:52:11.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:10.947+0000 7ff0bdffb700 1 -- 172.21.15.52:0/315807414 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7ff0c0025020 con 0x7ff0c8106b90 2024-06-10T02:52:11.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:10.947+0000 7ff0bdffb700 1 --2- 172.21.15.52:0/315807414 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7ff0b00a2cb0 0x7ff0b00a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:52:11.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:10.947+0000 7ff0bdffb700 1 -- 172.21.15.52:0/315807414 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7ff0c0010020 con 0x7ff0c8106b90 2024-06-10T02:52:11.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:10.947+0000 7ff0ccd24700 1 --2- 172.21.15.52:0/315807414 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7ff0b00a2cb0 0x7ff0b00a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:52:11.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:10.948+0000 7ff0bdffb700 1 --2- 172.21.15.52:0/315807414 >> [v2:172.21.15.188:6816/1444828441,v1:172.21.15.188:6817/1444828441] conn(0x7ff0b00adf90 0x7ff0b00b0430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:52:11.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:10.948+0000 7ff0bdffb700 1 -- 172.21.15.52:0/315807414 --> [v2:172.21.15.188:6816/1444828441,v1:172.21.15.188:6817/1444828441] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff0b00b0b00 con 0x7ff0b00adf90 2024-06-10T02:52:11.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:10.948+0000 7ff0bdffb700 1 -- 172.21.15.52:0/315807414 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_get_version_reply(handle=1 version=100) ==== 24+0+0 (secure 0 0 0) 0x7ff0c00d1300 con 0x7ff0c8106b90 2024-06-10T02:52:11.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:10.948+0000 7ff0bffff700 1 --2- 172.21.15.52:0/315807414 >> [v2:172.21.15.188:6816/1444828441,v1:172.21.15.188:6817/1444828441] conn(0x7ff0b00adf90 0x7ff0b00b0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:52:11.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:10.948+0000 7ff0ccd24700 1 --2- 172.21.15.52:0/315807414 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7ff0b00a2cb0 0x7ff0b00a5170 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7ff0c8104dc0 tx=0x7ff0c4009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:52:11.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:10.948+0000 7ff0bffff700 1 --2- 172.21.15.52:0/315807414 >> [v2:172.21.15.188:6816/1444828441,v1:172.21.15.188:6817/1444828441] conn(0x7ff0b00adf90 0x7ff0b00b0430 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:52:11.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:10.949+0000 7ff0bdffb700 1 -- 172.21.15.52:0/315807414 <== osd.8 v2:172.21.15.188:6816/1444828441 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff0b00b0b00 con 0x7ff0b00adf90 2024-06-10T02:52:11.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:10.966+0000 7ff0cef88700 1 -- 172.21.15.52:0/315807414 --> [v2:172.21.15.188:6816/1444828441,v1:172.21.15.188:6817/1444828441] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff0c80630a0 con 0x7ff0b00adf90 2024-06-10T02:52:11.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:10.966+0000 7ff0bdffb700 1 -- 172.21.15.52:0/315807414 <== osd.8 v2:172.21.15.188:6816/1444828441 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7ff0c80630a0 con 0x7ff0b00adf90 2024-06-10T02:52:11.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:10.966+0000 7ff0cef88700 1 -- 172.21.15.52:0/315807414 >> [v2:172.21.15.188:6816/1444828441,v1:172.21.15.188:6817/1444828441] conn(0x7ff0b00adf90 msgr2=0x7ff0b00b0430 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:52:11.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:10.966+0000 7ff0cef88700 1 --2- 172.21.15.52:0/315807414 >> [v2:172.21.15.188:6816/1444828441,v1:172.21.15.188:6817/1444828441] conn(0x7ff0b00adf90 0x7ff0b00b0430 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:11.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:10.967+0000 7ff0cef88700 1 -- 172.21.15.52:0/315807414 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7ff0b00a2cb0 msgr2=0x7ff0b00a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:52:11.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:10.967+0000 7ff0cef88700 1 --2- 172.21.15.52:0/315807414 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7ff0b00a2cb0 0x7ff0b00a5170 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7ff0c8104dc0 tx=0x7ff0c4009260 comp rx=0 tx=0).stop 2024-06-10T02:52:11.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:10.967+0000 7ff0cef88700 1 -- 172.21.15.52:0/315807414 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff0c8106b90 msgr2=0x7ff0c8198a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:52:11.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:10.967+0000 7ff0cef88700 1 --2- 172.21.15.52:0/315807414 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff0c8106b90 0x7ff0c8198a30 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7ff0c000a6f0 tx=0x7ff0c000a890 comp rx=0 tx=0).stop 2024-06-10T02:52:11.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:10.967+0000 7ff0cd525700 1 -- 172.21.15.52:0/315807414 reap_dead start 2024-06-10T02:52:11.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:10.967+0000 7ff0cef88700 1 -- 172.21.15.52:0/315807414 shutdown_connections 2024-06-10T02:52:11.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:10.967+0000 7ff0cef88700 1 -- 172.21.15.52:0/315807414 >> 172.21.15.52:0/315807414 conn(0x7ff0c8077a00 msgr2=0x7ff0c80ff390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:52:11.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:10.967+0000 7ff0cef88700 1 -- 172.21.15.52:0/315807414 shutdown_connections 2024-06-10T02:52:11.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:10.967+0000 7ff0cef88700 1 -- 172.21.15.52:0/315807414 wait complete. 2024-06-10T02:52:12.478 INFO:teuthology.orchestra.run.smithi052.stdout:257698037817 2024-06-10T02:52:12.478 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-06-10T02:52:12.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:12.489+0000 7f28139b0700 1 -- 172.21.15.52:0/1402537535 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f280c101d50 msgr2=0x7f280c10e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:52:12.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:12.489+0000 7f28139b0700 1 --2- 172.21.15.52:0/1402537535 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f280c101d50 0x7f280c10e240 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f280400b0a0 tx=0x7f280400b3b0 comp rx=0 tx=0).stop 2024-06-10T02:52:12.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:12.489+0000 7f28139b0700 1 -- 172.21.15.52:0/1402537535 shutdown_connections 2024-06-10T02:52:12.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:12.489+0000 7f28139b0700 1 --2- 172.21.15.52:0/1402537535 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f280c101d50 0x7f280c10e240 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:12.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:12.489+0000 7f28139b0700 1 --2- 172.21.15.52:0/1402537535 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f280c107400 0x7f280c1077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:12.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:12.489+0000 7f28139b0700 1 --2- 172.21.15.52:0/1402537535 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f280c101390 0x7f280c101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:12.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:12.489+0000 7f28139b0700 1 -- 172.21.15.52:0/1402537535 >> 172.21.15.52:0/1402537535 conn(0x7f280c078340 msgr2=0x7f280c078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:52:12.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:12.489+0000 7f28139b0700 1 -- 172.21.15.52:0/1402537535 shutdown_connections 2024-06-10T02:52:12.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:12.489+0000 7f28139b0700 1 -- 172.21.15.52:0/1402537535 wait complete. 2024-06-10T02:52:12.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:12.490+0000 7f28139b0700 1 Processor -- start 2024-06-10T02:52:12.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:12.490+0000 7f28139b0700 1 -- start start 2024-06-10T02:52:12.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:12.490+0000 7f28139b0700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f280c101390 0x7f280c103070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:52:12.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:12.490+0000 7f28139b0700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f280c101d50 0x7f280c1035b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:52:12.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:12.490+0000 7f28139b0700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f280c107400 0x7f280c103af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:52:12.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:12.490+0000 7f28139b0700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f280c111670 con 0x7f280c107400 2024-06-10T02:52:12.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:12.490+0000 7f28139b0700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f280c1114f0 con 0x7f280c101d50 2024-06-10T02:52:12.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:12.490+0000 7f28139b0700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f280c1117f0 con 0x7f280c101390 2024-06-10T02:52:12.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:12.491+0000 7f2811f4d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f280c107400 0x7f280c103af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:52:12.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:12.491+0000 7f2811f4d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f280c107400 0x7f280c103af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48352/0 (socket says 172.21.15.52:48352) 2024-06-10T02:52:12.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:12.491+0000 7f2811f4d700 1 -- 172.21.15.52:0/480724511 learned_addr learned my addr 172.21.15.52:0/480724511 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:52:12.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:12.491+0000 7f2810f4b700 1 --2- 172.21.15.52:0/480724511 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f280c101d50 0x7f280c1035b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:52:12.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:12.491+0000 7f281174c700 1 --2- 172.21.15.52:0/480724511 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f280c101390 0x7f280c103070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:52:12.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:12.491+0000 7f2811f4d700 1 -- 172.21.15.52:0/480724511 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f280c101390 msgr2=0x7f280c103070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:52:12.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:12.491+0000 7f2811f4d700 1 --2- 172.21.15.52:0/480724511 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f280c101390 0x7f280c103070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:12.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:12.491+0000 7f2811f4d700 1 -- 172.21.15.52:0/480724511 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f280c101d50 msgr2=0x7f280c1035b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:52:12.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:12.491+0000 7f2811f4d700 1 --2- 172.21.15.52:0/480724511 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f280c101d50 0x7f280c1035b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:12.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:12.491+0000 7f2811f4d700 1 -- 172.21.15.52:0/480724511 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f280c1a7060 con 0x7f280c107400 2024-06-10T02:52:12.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:12.491+0000 7f2811f4d700 1 --2- 172.21.15.52:0/480724511 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f280c107400 0x7f280c103af0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f28040062a0 tx=0x7f2804004d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:52:12.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:12.491+0000 7f28027fc700 1 -- 172.21.15.52:0/480724511 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2804021070 con 0x7f280c107400 2024-06-10T02:52:12.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:12.491+0000 7f28027fc700 1 -- 172.21.15.52:0/480724511 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2804003c00 con 0x7f280c107400 2024-06-10T02:52:12.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:12.491+0000 7f28139b0700 1 -- 172.21.15.52:0/480724511 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f280c1a72f0 con 0x7f280c107400 2024-06-10T02:52:12.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:12.491+0000 7f28027fc700 1 -- 172.21.15.52:0/480724511 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2804004880 con 0x7f280c107400 2024-06-10T02:52:12.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:12.491+0000 7f28139b0700 1 -- 172.21.15.52:0/480724511 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f280c1a77f0 con 0x7f280c107400 2024-06-10T02:52:12.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:12.492+0000 7f28139b0700 1 -- 172.21.15.52:0/480724511 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f280c06a440 con 0x7f280c107400 2024-06-10T02:52:12.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:12.494+0000 7f28027fc700 1 -- 172.21.15.52:0/480724511 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f2804003db0 con 0x7f280c107400 2024-06-10T02:52:12.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:12.494+0000 7f28027fc700 1 --2- 172.21.15.52:0/480724511 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f27f80a2d30 0x7f27f80a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:52:12.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:12.495+0000 7f281174c700 1 --2- 172.21.15.52:0/480724511 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f27f80a2d30 0x7f27f80a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:52:12.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:12.495+0000 7f28027fc700 1 -- 172.21.15.52:0/480724511 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f280400f020 con 0x7f280c107400 2024-06-10T02:52:12.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:12.495+0000 7f28027fc700 1 --2- 172.21.15.52:0/480724511 >> [v2:172.21.15.52:6818/359084919,v1:172.21.15.52:6819/359084919] conn(0x7f27f80ae010 0x7f27f80b04b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:52:12.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:12.495+0000 7f28027fc700 1 -- 172.21.15.52:0/480724511 --> [v2:172.21.15.52:6818/359084919,v1:172.21.15.52:6819/359084919] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f27f80b0b80 con 0x7f27f80ae010 2024-06-10T02:52:12.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:12.495+0000 7f28027fc700 1 -- 172.21.15.52:0/480724511 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_get_version_reply(handle=1 version=100) ==== 24+0+0 (secure 0 0 0) 0x7f28040d1050 con 0x7f280c107400 2024-06-10T02:52:12.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:12.495+0000 7f2810f4b700 1 --2- 172.21.15.52:0/480724511 >> [v2:172.21.15.52:6818/359084919,v1:172.21.15.52:6819/359084919] conn(0x7f27f80ae010 0x7f27f80b04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:52:12.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:12.495+0000 7f281174c700 1 --2- 172.21.15.52:0/480724511 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f27f80a2d30 0x7f27f80a51f0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f280800abd0 tx=0x7f280800a380 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:52:12.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:12.495+0000 7f2810f4b700 1 --2- 172.21.15.52:0/480724511 >> [v2:172.21.15.52:6818/359084919,v1:172.21.15.52:6819/359084919] conn(0x7f27f80ae010 0x7f27f80b04b0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:52:12.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:12.496+0000 7f28027fc700 1 -- 172.21.15.52:0/480724511 <== osd.6 v2:172.21.15.52:6818/359084919 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f27f80b0b80 con 0x7f27f80ae010 2024-06-10T02:52:12.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:12.513+0000 7f28139b0700 1 -- 172.21.15.52:0/480724511 --> [v2:172.21.15.52:6818/359084919,v1:172.21.15.52:6819/359084919] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f280c102ab0 con 0x7f27f80ae010 2024-06-10T02:52:12.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:12.513+0000 7f28027fc700 1 -- 172.21.15.52:0/480724511 <== osd.6 v2:172.21.15.52:6818/359084919 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f280c102ab0 con 0x7f27f80ae010 2024-06-10T02:52:12.572 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:12.514+0000 7f28139b0700 1 -- 172.21.15.52:0/480724511 >> [v2:172.21.15.52:6818/359084919,v1:172.21.15.52:6819/359084919] conn(0x7f27f80ae010 msgr2=0x7f27f80b04b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:52:12.572 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:12.514+0000 7f28139b0700 1 --2- 172.21.15.52:0/480724511 >> [v2:172.21.15.52:6818/359084919,v1:172.21.15.52:6819/359084919] conn(0x7f27f80ae010 0x7f27f80b04b0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:12.572 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:12.514+0000 7f28139b0700 1 -- 172.21.15.52:0/480724511 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f27f80a2d30 msgr2=0x7f27f80a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:52:12.572 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:12.514+0000 7f28139b0700 1 --2- 172.21.15.52:0/480724511 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f27f80a2d30 0x7f27f80a51f0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f280800abd0 tx=0x7f280800a380 comp rx=0 tx=0).stop 2024-06-10T02:52:12.572 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:12.514+0000 7f28139b0700 1 -- 172.21.15.52:0/480724511 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f280c107400 msgr2=0x7f280c103af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:52:12.572 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:12.514+0000 7f28139b0700 1 --2- 172.21.15.52:0/480724511 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f280c107400 0x7f280c103af0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f28040062a0 tx=0x7f2804004d70 comp rx=0 tx=0).stop 2024-06-10T02:52:12.572 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:12.514+0000 7f2811f4d700 1 -- 172.21.15.52:0/480724511 reap_dead start 2024-06-10T02:52:12.572 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:12.514+0000 7f28139b0700 1 -- 172.21.15.52:0/480724511 shutdown_connections 2024-06-10T02:52:12.572 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:12.514+0000 7f28139b0700 1 -- 172.21.15.52:0/480724511 >> 172.21.15.52:0/480724511 conn(0x7f280c078340 msgr2=0x7f280c105350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:52:12.572 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:12.514+0000 7f28139b0700 1 -- 172.21.15.52:0/480724511 shutdown_connections 2024-06-10T02:52:12.572 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:12.514+0000 7f28139b0700 1 -- 172.21.15.52:0/480724511 wait complete. 2024-06-10T02:52:13.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:52:12 smithi120 ceph-mon[36634]: pgmap v374: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-06-10T02:52:13.190 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:52:12 smithi052 ceph-mon[32362]: pgmap v374: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-06-10T02:52:13.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:52:12 smithi188 ceph-mon[36599]: pgmap v374: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-06-10T02:52:13.507 INFO:teuthology.orchestra.run.smithi052.stdout:77309411416 2024-06-10T02:52:13.507 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-06-10T02:52:14.337 INFO:teuthology.orchestra.run.smithi052.stdout:223338299454 2024-06-10T02:52:14.337 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-06-10T02:52:14.897 INFO:teuthology.orchestra.run.smithi052.stdout:292057776180 2024-06-10T02:52:14.897 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-06-10T02:52:15.072 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:52:14 smithi052 ceph-mon[32362]: pgmap v375: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-06-10T02:52:15.091 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:15.092+0000 7f22de458700 1 -- 172.21.15.52:0/3405504612 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f22d81030f0 msgr2=0x7f22d8107110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:52:15.091 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:15.092+0000 7f22de458700 1 --2- 172.21.15.52:0/3405504612 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f22d81030f0 0x7f22d8107110 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f22c0009990 tx=0x7f22c0009ca0 comp rx=0 tx=0).stop 2024-06-10T02:52:15.091 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:15.092+0000 7f22de458700 1 -- 172.21.15.52:0/3405504612 shutdown_connections 2024-06-10T02:52:15.091 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:15.092+0000 7f22de458700 1 --2- 172.21.15.52:0/3405504612 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f22d8107840 0x7f22d810f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:15.091 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:15.092+0000 7f22de458700 1 --2- 172.21.15.52:0/3405504612 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f22d8102740 0x7f22d8102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:15.091 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:15.092+0000 7f22de458700 1 --2- 172.21.15.52:0/3405504612 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f22d81030f0 0x7f22d8107110 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:15.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:15.092+0000 7f22de458700 1 -- 172.21.15.52:0/3405504612 >> 172.21.15.52:0/3405504612 conn(0x7f22d80795f0 msgr2=0x7f22d8079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:52:15.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:15.092+0000 7f22de458700 1 -- 172.21.15.52:0/3405504612 shutdown_connections 2024-06-10T02:52:15.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:15.092+0000 7f22de458700 1 -- 172.21.15.52:0/3405504612 wait complete. 2024-06-10T02:52:15.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:15.092+0000 7f22de458700 1 Processor -- start 2024-06-10T02:52:15.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:15.092+0000 7f22de458700 1 -- start start 2024-06-10T02:52:15.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:15.093+0000 7f22de458700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f22d8102740 0x7f22d819ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:52:15.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:15.093+0000 7f22de458700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f22d81030f0 0x7f22d819b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:52:15.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:15.093+0000 7f22de458700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f22d8107840 0x7f22d819f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:52:15.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:15.093+0000 7f22de458700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f22d81149c0 con 0x7f22d8102740 2024-06-10T02:52:15.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:15.093+0000 7f22de458700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f22d8114840 con 0x7f22d81030f0 2024-06-10T02:52:15.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:15.093+0000 7f22de458700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f22d8114b40 con 0x7f22d8107840 2024-06-10T02:52:15.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:15.093+0000 7f22d7fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f22d8102740 0x7f22d819ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:52:15.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:15.093+0000 7f22d7fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f22d8102740 0x7f22d819ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48384/0 (socket says 172.21.15.52:48384) 2024-06-10T02:52:15.094 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:15.093+0000 7f22d7fff700 1 -- 172.21.15.52:0/1212276708 learned_addr learned my addr 172.21.15.52:0/1212276708 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:52:15.094 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:15.093+0000 7f22dc9f5700 1 --2- 172.21.15.52:0/1212276708 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f22d8107840 0x7f22d819f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:52:15.094 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:15.093+0000 7f22d77fe700 1 --2- 172.21.15.52:0/1212276708 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f22d81030f0 0x7f22d819b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:52:15.094 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:15.093+0000 7f22d7fff700 1 -- 172.21.15.52:0/1212276708 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f22d8107840 msgr2=0x7f22d819f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:52:15.094 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:15.093+0000 7f22d7fff700 1 --2- 172.21.15.52:0/1212276708 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f22d8107840 0x7f22d819f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:15.094 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:15.093+0000 7f22d7fff700 1 -- 172.21.15.52:0/1212276708 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f22d81030f0 msgr2=0x7f22d819b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:52:15.094 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:15.093+0000 7f22d7fff700 1 --2- 172.21.15.52:0/1212276708 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f22d81030f0 0x7f22d819b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:15.094 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:15.094+0000 7f22d7fff700 1 -- 172.21.15.52:0/1212276708 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22d819fe20 con 0x7f22d8102740 2024-06-10T02:52:15.094 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:15.094+0000 7f22d77fe700 1 --2- 172.21.15.52:0/1212276708 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f22d81030f0 0x7f22d819b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T02:52:15.094 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:15.094+0000 7f22d7fff700 1 --2- 172.21.15.52:0/1212276708 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f22d8102740 0x7f22d819ae10 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f22c800d950 tx=0x7f22c800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:52:15.094 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:15.094+0000 7f22d57fa700 1 -- 172.21.15.52:0/1212276708 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22c8016070 con 0x7f22d8102740 2024-06-10T02:52:15.094 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:15.094+0000 7f22de458700 1 -- 172.21.15.52:0/1212276708 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22d81a0110 con 0x7f22d8102740 2024-06-10T02:52:15.094 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:15.094+0000 7f22d57fa700 1 -- 172.21.15.52:0/1212276708 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f22c8009e20 con 0x7f22d8102740 2024-06-10T02:52:15.095 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:15.094+0000 7f22d57fa700 1 -- 172.21.15.52:0/1212276708 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22c800f620 con 0x7f22d8102740 2024-06-10T02:52:15.095 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:15.094+0000 7f22de458700 1 -- 172.21.15.52:0/1212276708 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22d81a7990 con 0x7f22d8102740 2024-06-10T02:52:15.095 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:15.095+0000 7f22de458700 1 -- 172.21.15.52:0/1212276708 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f22d8062c20 con 0x7f22d8102740 2024-06-10T02:52:15.095 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:15.096+0000 7f22d57fa700 1 -- 172.21.15.52:0/1212276708 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f22c8019070 con 0x7f22d8102740 2024-06-10T02:52:15.096 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:15.097+0000 7f22d57fa700 1 --2- 172.21.15.52:0/1212276708 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f22c40a2cb0 0x7f22c40a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:52:15.096 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:15.097+0000 7f22d57fa700 1 -- 172.21.15.52:0/1212276708 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f22c80cf570 con 0x7f22d8102740 2024-06-10T02:52:15.096 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:15.097+0000 7f22d77fe700 1 --2- 172.21.15.52:0/1212276708 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f22c40a2cb0 0x7f22c40a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:52:15.097 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:15.097+0000 7f22d57fa700 1 --2- 172.21.15.52:0/1212276708 >> [v2:172.21.15.120:6824/1531769083,v1:172.21.15.120:6825/1531769083] conn(0x7f22c40adf90 0x7f22c40b0430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:52:15.097 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:15.097+0000 7f22d57fa700 1 -- 172.21.15.52:0/1212276708 --> [v2:172.21.15.120:6824/1531769083,v1:172.21.15.120:6825/1531769083] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f22c8015ec0 con 0x7f22c40adf90 2024-06-10T02:52:15.097 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:15.097+0000 7f22d57fa700 1 -- 172.21.15.52:0/1212276708 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_get_version_reply(handle=1 version=100) ==== 24+0+0 (secure 0 0 0) 0x7f22c80cf9c0 con 0x7f22d8102740 2024-06-10T02:52:15.097 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:15.098+0000 7f22d77fe700 1 --2- 172.21.15.52:0/1212276708 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f22c40a2cb0 0x7f22c40a5170 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f22c0006010 tx=0x7f22c000b540 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:52:15.097 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:15.098+0000 7f22dc9f5700 1 --2- 172.21.15.52:0/1212276708 >> [v2:172.21.15.120:6824/1531769083,v1:172.21.15.120:6825/1531769083] conn(0x7f22c40adf90 0x7f22c40b0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:52:15.097 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:15.098+0000 7f22dc9f5700 1 --2- 172.21.15.52:0/1212276708 >> [v2:172.21.15.120:6824/1531769083,v1:172.21.15.120:6825/1531769083] conn(0x7f22c40adf90 0x7f22c40b0430 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:52:15.098 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:15.099+0000 7f22d57fa700 1 -- 172.21.15.52:0/1212276708 <== osd.10 v2:172.21.15.120:6824/1531769083 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f22c8015ec0 con 0x7f22c40adf90 2024-06-10T02:52:15.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:15.122+0000 7f22de458700 1 -- 172.21.15.52:0/1212276708 --> [v2:172.21.15.120:6824/1531769083,v1:172.21.15.120:6825/1531769083] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f22d8063020 con 0x7f22c40adf90 2024-06-10T02:52:15.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:15.123+0000 7f22d57fa700 1 -- 172.21.15.52:0/1212276708 <== osd.10 v2:172.21.15.120:6824/1531769083 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f22d8063020 con 0x7f22c40adf90 2024-06-10T02:52:15.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:15.123+0000 7f22de458700 1 -- 172.21.15.52:0/1212276708 >> [v2:172.21.15.120:6824/1531769083,v1:172.21.15.120:6825/1531769083] conn(0x7f22c40adf90 msgr2=0x7f22c40b0430 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:52:15.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:15.123+0000 7f22de458700 1 --2- 172.21.15.52:0/1212276708 >> [v2:172.21.15.120:6824/1531769083,v1:172.21.15.120:6825/1531769083] conn(0x7f22c40adf90 0x7f22c40b0430 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:15.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:15.123+0000 7f22de458700 1 -- 172.21.15.52:0/1212276708 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f22c40a2cb0 msgr2=0x7f22c40a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:52:15.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:15.123+0000 7f22de458700 1 --2- 172.21.15.52:0/1212276708 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f22c40a2cb0 0x7f22c40a5170 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f22c0006010 tx=0x7f22c000b540 comp rx=0 tx=0).stop 2024-06-10T02:52:15.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:15.123+0000 7f22de458700 1 -- 172.21.15.52:0/1212276708 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f22d8102740 msgr2=0x7f22d819ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:52:15.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:15.123+0000 7f22de458700 1 --2- 172.21.15.52:0/1212276708 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f22d8102740 0x7f22d819ae10 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f22c800d950 tx=0x7f22c800dc60 comp rx=0 tx=0).stop 2024-06-10T02:52:15.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:15.123+0000 7f22dc9f5700 1 -- 172.21.15.52:0/1212276708 reap_dead start 2024-06-10T02:52:15.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:15.123+0000 7f22de458700 1 -- 172.21.15.52:0/1212276708 shutdown_connections 2024-06-10T02:52:15.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:15.123+0000 7f22de458700 1 -- 172.21.15.52:0/1212276708 >> 172.21.15.52:0/1212276708 conn(0x7f22d80795f0 msgr2=0x7f22d8101ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:52:15.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:15.123+0000 7f22de458700 1 -- 172.21.15.52:0/1212276708 shutdown_connections 2024-06-10T02:52:15.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:15.124+0000 7f22de458700 1 -- 172.21.15.52:0/1212276708 wait complete. 2024-06-10T02:52:15.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:52:14 smithi120 ceph-mon[36634]: pgmap v375: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-06-10T02:52:15.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:52:14 smithi188 ceph-mon[36599]: pgmap v375: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-06-10T02:52:15.798 INFO:teuthology.orchestra.run.smithi052.stdout:146028888142 2024-06-10T02:52:15.799 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-06-10T02:52:16.190 INFO:teuthology.orchestra.run.smithi052.stdout:201863462980 2024-06-10T02:52:16.190 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-06-10T02:52:16.936 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:52:16.936 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:52:16.982 INFO:teuthology.orchestra.run.smithi052.stdout:313532612656 2024-06-10T02:52:16.983 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-06-10T02:52:17.015 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:52:16 smithi052 ceph-mon[32362]: pgmap v376: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-10T02:52:17.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:52:16 smithi120 ceph-mon[36634]: pgmap v376: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-10T02:52:17.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:52:16 smithi188 ceph-mon[36599]: pgmap v376: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-10T02:52:18.619 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:18.619+0000 7f4826760700 1 -- 172.21.15.52:0/4193024586 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f482006aac0 msgr2=0x7f482010ea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:52:18.619 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:18.619+0000 7f4826760700 1 --2- 172.21.15.52:0/4193024586 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f482006aac0 0x7f482010ea00 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f4814009990 tx=0x7f4814009ca0 comp rx=0 tx=0).stop 2024-06-10T02:52:18.619 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:18.620+0000 7f4826760700 1 -- 172.21.15.52:0/4193024586 shutdown_connections 2024-06-10T02:52:18.619 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:18.620+0000 7f4826760700 1 --2- 172.21.15.52:0/4193024586 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f482010ef40 0x7f4820111330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:18.619 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:18.620+0000 7f4826760700 1 --2- 172.21.15.52:0/4193024586 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f482006a1a0 0x7f482006a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:18.619 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:18.620+0000 7f4826760700 1 --2- 172.21.15.52:0/4193024586 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f482006aac0 0x7f482010ea00 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:18.619 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:18.620+0000 7f4826760700 1 -- 172.21.15.52:0/4193024586 >> 172.21.15.52:0/4193024586 conn(0x7f4820078340 msgr2=0x7f4820078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:52:18.620 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:18.620+0000 7f4826760700 1 -- 172.21.15.52:0/4193024586 shutdown_connections 2024-06-10T02:52:18.620 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:18.620+0000 7f4826760700 1 -- 172.21.15.52:0/4193024586 wait complete. 2024-06-10T02:52:18.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:18.621+0000 7f4826760700 1 Processor -- start 2024-06-10T02:52:18.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:18.621+0000 7f4826760700 1 -- start start 2024-06-10T02:52:18.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:18.622+0000 7f4826760700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f482006a1a0 0x7f482019aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:52:18.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:18.622+0000 7f4826760700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f482006aac0 0x7f482019b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:52:18.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:18.622+0000 7f4826760700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f482010ef40 0x7f482019f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:52:18.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:18.622+0000 7f4826760700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f48201149c0 con 0x7f482010ef40 2024-06-10T02:52:18.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:18.622+0000 7f4826760700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f4820114840 con 0x7f482006aac0 2024-06-10T02:52:18.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:18.622+0000 7f4826760700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f4820114b40 con 0x7f482006a1a0 2024-06-10T02:52:18.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:18.622+0000 7f4824cfd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f482010ef40 0x7f482019f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:52:18.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:18.622+0000 7f4824cfd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f482010ef40 0x7f482019f7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48398/0 (socket says 172.21.15.52:48398) 2024-06-10T02:52:18.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:18.622+0000 7f4824cfd700 1 -- 172.21.15.52:0/612335049 learned_addr learned my addr 172.21.15.52:0/612335049 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:52:18.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:18.622+0000 7f481f7fe700 1 --2- 172.21.15.52:0/612335049 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f482006aac0 0x7f482019b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:52:18.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:18.623+0000 7f4824cfd700 1 -- 172.21.15.52:0/612335049 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f482006a1a0 msgr2=0x7f482019aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:52:18.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:18.623+0000 7f4824cfd700 1 --2- 172.21.15.52:0/612335049 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f482006a1a0 0x7f482019aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:18.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:18.623+0000 7f4824cfd700 1 -- 172.21.15.52:0/612335049 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f482006aac0 msgr2=0x7f482019b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:52:18.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:18.623+0000 7f4824cfd700 1 --2- 172.21.15.52:0/612335049 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f482006aac0 0x7f482019b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:18.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:18.623+0000 7f4824cfd700 1 -- 172.21.15.52:0/612335049 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f482019fee0 con 0x7f482010ef40 2024-06-10T02:52:18.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:18.623+0000 7f481f7fe700 1 --2- 172.21.15.52:0/612335049 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f482006aac0 0x7f482019b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T02:52:18.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:18.623+0000 7f4824cfd700 1 --2- 172.21.15.52:0/612335049 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f482010ef40 0x7f482019f7a0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f481800ef90 tx=0x7f481800c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:52:18.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:18.623+0000 7f481d7fa700 1 -- 172.21.15.52:0/612335049 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48180092e0 con 0x7f482010ef40 2024-06-10T02:52:18.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:18.623+0000 7f481d7fa700 1 -- 172.21.15.52:0/612335049 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4818004bb0 con 0x7f482010ef40 2024-06-10T02:52:18.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:18.623+0000 7f4826760700 1 -- 172.21.15.52:0/612335049 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48201a0170 con 0x7f482010ef40 2024-06-10T02:52:18.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:18.623+0000 7f481d7fa700 1 -- 172.21.15.52:0/612335049 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4818004280 con 0x7f482010ef40 2024-06-10T02:52:18.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:18.623+0000 7f4826760700 1 -- 172.21.15.52:0/612335049 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48201a7a50 con 0x7f482010ef40 2024-06-10T02:52:18.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:18.624+0000 7f4806ffd700 1 -- 172.21.15.52:0/612335049 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4800001030 con 0x7f482010ef40 2024-06-10T02:52:18.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:18.627+0000 7f481d7fa700 1 -- 172.21.15.52:0/612335049 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f481801b030 con 0x7f482010ef40 2024-06-10T02:52:18.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:18.628+0000 7f481d7fa700 1 --2- 172.21.15.52:0/612335049 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f48080a2d30 0x7f48080a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:52:18.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:18.628+0000 7f481d7fa700 1 -- 172.21.15.52:0/612335049 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f48180cfc00 con 0x7f482010ef40 2024-06-10T02:52:18.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:18.628+0000 7f481d7fa700 1 --2- 172.21.15.52:0/612335049 >> [v2:172.21.15.188:6824/3769050170,v1:172.21.15.188:6825/3769050170] conn(0x7f48080ae010 0x7f48080b04b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:52:18.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:18.629+0000 7f481d7fa700 1 -- 172.21.15.52:0/612335049 --> [v2:172.21.15.188:6824/3769050170,v1:172.21.15.188:6825/3769050170] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f481801fe50 con 0x7f48080ae010 2024-06-10T02:52:18.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:18.629+0000 7f481d7fa700 1 -- 172.21.15.52:0/612335049 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_get_version_reply(handle=1 version=100) ==== 24+0+0 (secure 0 0 0) 0x7f48180cfff0 con 0x7f482010ef40 2024-06-10T02:52:18.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:18.629+0000 7f481f7fe700 1 --2- 172.21.15.52:0/612335049 >> [v2:172.21.15.188:6824/3769050170,v1:172.21.15.188:6825/3769050170] conn(0x7f48080ae010 0x7f48080b04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:52:18.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:18.635+0000 7f481ffff700 1 --2- 172.21.15.52:0/612335049 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f48080a2d30 0x7f48080a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:52:18.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:18.635+0000 7f481f7fe700 1 --2- 172.21.15.52:0/612335049 >> [v2:172.21.15.188:6824/3769050170,v1:172.21.15.188:6825/3769050170] conn(0x7f48080ae010 0x7f48080b04b0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:52:18.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:18.635+0000 7f481ffff700 1 --2- 172.21.15.52:0/612335049 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f48080a2d30 0x7f48080a51f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f4810005950 tx=0x7f48100058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:52:18.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:18.637+0000 7f481d7fa700 1 -- 172.21.15.52:0/612335049 <== osd.11 v2:172.21.15.188:6824/3769050170 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f481801fe50 con 0x7f48080ae010 2024-06-10T02:52:18.645 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:52:18.646 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:52:18.649 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:52:18.652 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:52:18.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:18.674+0000 7f4806ffd700 1 -- 172.21.15.52:0/612335049 --> [v2:172.21.15.188:6824/3769050170,v1:172.21.15.188:6825/3769050170] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4800002de0 con 0x7f48080ae010 2024-06-10T02:52:18.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:18.674+0000 7f481d7fa700 1 -- 172.21.15.52:0/612335049 <== osd.11 v2:172.21.15.188:6824/3769050170 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f4800002de0 con 0x7f48080ae010 2024-06-10T02:52:18.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:18.675+0000 7f4806ffd700 1 -- 172.21.15.52:0/612335049 >> [v2:172.21.15.188:6824/3769050170,v1:172.21.15.188:6825/3769050170] conn(0x7f48080ae010 msgr2=0x7f48080b04b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:52:18.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:18.675+0000 7f4806ffd700 1 --2- 172.21.15.52:0/612335049 >> [v2:172.21.15.188:6824/3769050170,v1:172.21.15.188:6825/3769050170] conn(0x7f48080ae010 0x7f48080b04b0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:18.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:18.675+0000 7f4806ffd700 1 -- 172.21.15.52:0/612335049 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f48080a2d30 msgr2=0x7f48080a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:52:18.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:18.676+0000 7f4806ffd700 1 --2- 172.21.15.52:0/612335049 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f48080a2d30 0x7f48080a51f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f4810005950 tx=0x7f48100058e0 comp rx=0 tx=0).stop 2024-06-10T02:52:18.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:18.676+0000 7f4806ffd700 1 -- 172.21.15.52:0/612335049 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f482010ef40 msgr2=0x7f482019f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:52:18.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:18.676+0000 7f4806ffd700 1 --2- 172.21.15.52:0/612335049 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f482010ef40 0x7f482019f7a0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f481800ef90 tx=0x7f481800c450 comp rx=0 tx=0).stop 2024-06-10T02:52:18.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:18.676+0000 7f4824cfd700 1 -- 172.21.15.52:0/612335049 reap_dead start 2024-06-10T02:52:18.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:18.677+0000 7f4806ffd700 1 -- 172.21.15.52:0/612335049 shutdown_connections 2024-06-10T02:52:18.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:18.677+0000 7f4806ffd700 1 -- 172.21.15.52:0/612335049 >> 172.21.15.52:0/612335049 conn(0x7f4820078340 msgr2=0x7f48201102f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:52:18.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:18.678+0000 7f4806ffd700 1 -- 172.21.15.52:0/612335049 shutdown_connections 2024-06-10T02:52:18.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:18.678+0000 7f4806ffd700 1 -- 172.21.15.52:0/612335049 wait complete. 2024-06-10T02:52:19.038 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:52:18 smithi052 ceph-mon[32362]: pgmap v377: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-10T02:52:19.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:52:18 smithi120 ceph-mon[36634]: pgmap v377: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-10T02:52:19.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:52:18 smithi188 ceph-mon[36599]: pgmap v377: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-10T02:52:20.864 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:52:20 smithi052 ceph-mon[32362]: pgmap v378: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-10T02:52:21.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:52:20 smithi120 ceph-mon[36634]: pgmap v378: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-10T02:52:21.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:52:20 smithi188 ceph-mon[36599]: pgmap v378: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-10T02:52:21.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:21.258+0000 7fc0d883b700 1 -- 172.21.15.52:0/140662855 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc0d4102e80 msgr2=0x7fc0d410f330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:52:21.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:21.258+0000 7fc0d883b700 1 --2- 172.21.15.52:0/140662855 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc0d4102e80 0x7fc0d410f330 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fc0cc00b0a0 tx=0x7fc0cc00b3b0 comp rx=0 tx=0).stop 2024-06-10T02:52:21.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:21.258+0000 7fc0d883b700 1 -- 172.21.15.52:0/140662855 shutdown_connections 2024-06-10T02:52:21.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:21.258+0000 7fc0d883b700 1 --2- 172.21.15.52:0/140662855 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc0d4102e80 0x7fc0d410f330 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:21.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:21.258+0000 7fc0d883b700 1 --2- 172.21.15.52:0/140662855 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc0d41084f0 0x7fc0d41088d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:21.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:21.258+0000 7fc0d883b700 1 --2- 172.21.15.52:0/140662855 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc0d41024e0 0x7fc0d4102940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:21.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:21.258+0000 7fc0d883b700 1 -- 172.21.15.52:0/140662855 >> 172.21.15.52:0/140662855 conn(0x7fc0d4079530 msgr2=0x7fc0d4079940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:52:21.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:21.258+0000 7fc0d883b700 1 -- 172.21.15.52:0/140662855 shutdown_connections 2024-06-10T02:52:21.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:21.258+0000 7fc0d883b700 1 -- 172.21.15.52:0/140662855 wait complete. 2024-06-10T02:52:21.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:21.259+0000 7fc0d883b700 1 Processor -- start 2024-06-10T02:52:21.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:21.259+0000 7fc0d883b700 1 -- start start 2024-06-10T02:52:21.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:21.259+0000 7fc0d883b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc0d41024e0 0x7fc0d419e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:52:21.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:21.259+0000 7fc0d883b700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc0d4102e80 0x7fc0d419ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:52:21.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:21.259+0000 7fc0d883b700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc0d41084f0 0x7fc0d4198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:52:21.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:21.259+0000 7fc0d883b700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fc0d41127f0 con 0x7fc0d41024e0 2024-06-10T02:52:21.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:21.259+0000 7fc0d883b700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc0d4112670 con 0x7fc0d4102e80 2024-06-10T02:52:21.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:21.259+0000 7fc0d883b700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc0d4112970 con 0x7fc0d41084f0 2024-06-10T02:52:21.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:21.259+0000 7fc0d259c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc0d41024e0 0x7fc0d419e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:52:21.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:21.259+0000 7fc0d259c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc0d41024e0 0x7fc0d419e930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:34528/0 (socket says 172.21.15.52:34528) 2024-06-10T02:52:21.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:21.259+0000 7fc0d259c700 1 -- 172.21.15.52:0/2924738719 learned_addr learned my addr 172.21.15.52:0/2924738719 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:52:21.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:21.259+0000 7fc0d1d9b700 1 --2- 172.21.15.52:0/2924738719 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc0d4102e80 0x7fc0d419ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:52:21.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:21.259+0000 7fc0d2d9d700 1 --2- 172.21.15.52:0/2924738719 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc0d41084f0 0x7fc0d4198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:52:21.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:21.259+0000 7fc0d259c700 1 -- 172.21.15.52:0/2924738719 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc0d41084f0 msgr2=0x7fc0d4198af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:52:21.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:21.259+0000 7fc0d259c700 1 --2- 172.21.15.52:0/2924738719 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc0d41084f0 0x7fc0d4198af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:21.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:21.259+0000 7fc0d259c700 1 -- 172.21.15.52:0/2924738719 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc0d4102e80 msgr2=0x7fc0d419ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:52:21.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:21.259+0000 7fc0d259c700 1 --2- 172.21.15.52:0/2924738719 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc0d4102e80 0x7fc0d419ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:21.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:21.259+0000 7fc0d259c700 1 -- 172.21.15.52:0/2924738719 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0d4199250 con 0x7fc0d41024e0 2024-06-10T02:52:21.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:21.259+0000 7fc0d2d9d700 1 --2- 172.21.15.52:0/2924738719 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc0d41084f0 0x7fc0d4198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T02:52:21.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:21.260+0000 7fc0d259c700 1 --2- 172.21.15.52:0/2924738719 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc0d41024e0 0x7fc0d419e930 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fc0c400d950 tx=0x7fc0c400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:52:21.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:21.260+0000 7fc0c37fe700 1 -- 172.21.15.52:0/2924738719 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0c4016070 con 0x7fc0d41024e0 2024-06-10T02:52:21.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:21.260+0000 7fc0c37fe700 1 -- 172.21.15.52:0/2924738719 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc0c4009e20 con 0x7fc0d41024e0 2024-06-10T02:52:21.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:21.260+0000 7fc0d883b700 1 -- 172.21.15.52:0/2924738719 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0d4199540 con 0x7fc0d41024e0 2024-06-10T02:52:21.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:21.260+0000 7fc0c37fe700 1 -- 172.21.15.52:0/2924738719 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0c400f620 con 0x7fc0d41024e0 2024-06-10T02:52:21.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:21.260+0000 7fc0d883b700 1 -- 172.21.15.52:0/2924738719 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0d41a5800 con 0x7fc0d41024e0 2024-06-10T02:52:21.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:21.261+0000 7fc0d883b700 1 -- 172.21.15.52:0/2924738719 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0d4069180 con 0x7fc0d41024e0 2024-06-10T02:52:21.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:21.262+0000 7fc0c37fe700 1 -- 172.21.15.52:0/2924738719 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fc0c4019070 con 0x7fc0d41024e0 2024-06-10T02:52:21.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:21.263+0000 7fc0c37fe700 1 --2- 172.21.15.52:0/2924738719 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fc0bc0a2cb0 0x7fc0bc0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:52:21.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:21.263+0000 7fc0c37fe700 1 -- 172.21.15.52:0/2924738719 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7fc0c40cf1a0 con 0x7fc0d41024e0 2024-06-10T02:52:21.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:21.263+0000 7fc0d1d9b700 1 --2- 172.21.15.52:0/2924738719 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fc0bc0a2cb0 0x7fc0bc0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:52:21.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:21.263+0000 7fc0d1d9b700 1 --2- 172.21.15.52:0/2924738719 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fc0bc0a2cb0 0x7fc0bc0a5170 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc0d4079b80 tx=0x7fc0c8009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:52:21.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:21.267+0000 7fc0c37fe700 1 -- 172.21.15.52:0/2924738719 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fc0c409bb90 con 0x7fc0d41024e0 2024-06-10T02:52:21.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:21.481+0000 7fc0d883b700 1 -- 172.21.15.52:0/2924738719 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fc0d402dca0 con 0x7fc0d41024e0 2024-06-10T02:52:21.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:21.482+0000 7fc0c37fe700 1 -- 172.21.15.52:0/2924738719 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fc0c409b490 con 0x7fc0d41024e0 2024-06-10T02:52:21.481 INFO:teuthology.orchestra.run.smithi052.stdout:55834574943 2024-06-10T02:52:21.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:21.484+0000 7fc0d883b700 1 -- 172.21.15.52:0/2924738719 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fc0bc0a2cb0 msgr2=0x7fc0bc0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:52:21.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:21.484+0000 7fc0d883b700 1 --2- 172.21.15.52:0/2924738719 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fc0bc0a2cb0 0x7fc0bc0a5170 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc0d4079b80 tx=0x7fc0c8009190 comp rx=0 tx=0).stop 2024-06-10T02:52:21.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:21.484+0000 7fc0d883b700 1 -- 172.21.15.52:0/2924738719 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc0d41024e0 msgr2=0x7fc0d419e930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:52:21.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:21.484+0000 7fc0d883b700 1 --2- 172.21.15.52:0/2924738719 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc0d41024e0 0x7fc0d419e930 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fc0c400d950 tx=0x7fc0c400dc60 comp rx=0 tx=0).stop 2024-06-10T02:52:21.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:21.484+0000 7fc0d883b700 1 -- 172.21.15.52:0/2924738719 shutdown_connections 2024-06-10T02:52:21.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:21.484+0000 7fc0d883b700 1 --2- 172.21.15.52:0/2924738719 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc0d41084f0 0x7fc0d4198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:21.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:21.484+0000 7fc0d883b700 1 --2- 172.21.15.52:0/2924738719 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fc0bc0a2cb0 0x7fc0bc0a5170 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:21.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:21.484+0000 7fc0d883b700 1 --2- 172.21.15.52:0/2924738719 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc0d41024e0 0x7fc0d419e930 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:21.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:21.484+0000 7fc0d883b700 1 --2- 172.21.15.52:0/2924738719 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc0d4102e80 0x7fc0d419ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:21.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:21.484+0000 7fc0d883b700 1 -- 172.21.15.52:0/2924738719 >> 172.21.15.52:0/2924738719 conn(0x7fc0d4079530 msgr2=0x7fc0d4100630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:52:21.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:21.484+0000 7fc0d883b700 1 -- 172.21.15.52:0/2924738719 shutdown_connections 2024-06-10T02:52:21.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:21.484+0000 7fc0d883b700 1 -- 172.21.15.52:0/2924738719 wait complete. 2024-06-10T02:52:22.082 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:52:21 smithi052 ceph-mon[32362]: from='client.? 172.21.15.52:0/2924738719' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-06-10T02:52:22.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:52:21 smithi120 ceph-mon[36634]: from='client.? 172.21.15.52:0/2924738719' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-06-10T02:52:22.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:52:21 smithi188 ceph-mon[36599]: from='client.? 172.21.15.52:0/2924738719' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-06-10T02:52:23.024 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:52:22 smithi052 ceph-mon[32362]: pgmap v379: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s 2024-06-10T02:52:23.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:52:22 smithi120 ceph-mon[36634]: pgmap v379: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s 2024-06-10T02:52:23.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:52:22 smithi188 ceph-mon[36599]: pgmap v379: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s 2024-06-10T02:52:23.525 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:52:23.525 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:52:23.528 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:52:23.529 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:52:23.948 INFO:teuthology.orchestra.run.smithi052.stdout:347892351020 2024-06-10T02:52:23.949 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-06-10T02:52:25.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:52:24 smithi120 ceph-mon[36634]: pgmap v380: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-10T02:52:25.165 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:52:24 smithi052 ceph-mon[32362]: pgmap v380: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-10T02:52:25.199 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574939 got 55834574943 for osd.1 2024-06-10T02:52:25.199 DEBUG:teuthology.parallel:result is None 2024-06-10T02:52:25.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:52:24 smithi188 ceph-mon[36599]: pgmap v380: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-10T02:52:25.880 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:25.880+0000 7f2fefc8e700 1 -- 172.21.15.52:0/3594698169 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2fe806a170 msgr2=0x7f2fe806a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:52:25.880 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:25.880+0000 7f2fefc8e700 1 --2- 172.21.15.52:0/3594698169 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2fe806a170 0x7f2fe806a5f0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f2fd8009990 tx=0x7f2fd8009ca0 comp rx=0 tx=0).stop 2024-06-10T02:52:25.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:25.880+0000 7f2fefc8e700 1 -- 172.21.15.52:0/3594698169 shutdown_connections 2024-06-10T02:52:25.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:25.880+0000 7f2fefc8e700 1 --2- 172.21.15.52:0/3594698169 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2fe806ab30 0x7f2fe810f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:25.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:25.880+0000 7f2fefc8e700 1 --2- 172.21.15.52:0/3594698169 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2fe8108500 0x7f2fe81088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:25.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:25.880+0000 7f2fefc8e700 1 --2- 172.21.15.52:0/3594698169 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2fe806a170 0x7f2fe806a5f0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:25.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:25.880+0000 7f2fefc8e700 1 -- 172.21.15.52:0/3594698169 >> 172.21.15.52:0/3594698169 conn(0x7f2fe8078340 msgr2=0x7f2fe8078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:52:25.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:25.880+0000 7f2fefc8e700 1 -- 172.21.15.52:0/3594698169 shutdown_connections 2024-06-10T02:52:25.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:25.880+0000 7f2fefc8e700 1 -- 172.21.15.52:0/3594698169 wait complete. 2024-06-10T02:52:25.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:25.881+0000 7f2fefc8e700 1 Processor -- start 2024-06-10T02:52:25.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:25.881+0000 7f2fefc8e700 1 -- start start 2024-06-10T02:52:25.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:25.881+0000 7f2fefc8e700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2fe806a170 0x7f2fe8101460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:52:25.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:25.881+0000 7f2fefc8e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2fe806ab30 0x7f2fe81019a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:52:25.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:25.881+0000 7f2fefc8e700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2fe8108500 0x7f2fe8101ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:52:25.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:25.881+0000 7f2fefc8e700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f2fe8112860 con 0x7f2fe806ab30 2024-06-10T02:52:25.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:25.881+0000 7f2fefc8e700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f2fe81126e0 con 0x7f2fe806a170 2024-06-10T02:52:25.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:25.881+0000 7f2fefc8e700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f2fe81129e0 con 0x7f2fe8108500 2024-06-10T02:52:25.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:25.881+0000 7f2fed229700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2fe806ab30 0x7f2fe81019a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:52:25.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:25.881+0000 7f2fed229700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2fe806ab30 0x7f2fe81019a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:34544/0 (socket says 172.21.15.52:34544) 2024-06-10T02:52:25.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:25.881+0000 7f2fed229700 1 -- 172.21.15.52:0/244549298 learned_addr learned my addr 172.21.15.52:0/244549298 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:52:25.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:25.881+0000 7f2feda2a700 1 --2- 172.21.15.52:0/244549298 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2fe806a170 0x7f2fe8101460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:52:25.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:25.881+0000 7f2fee22b700 1 --2- 172.21.15.52:0/244549298 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2fe8108500 0x7f2fe8101ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:52:25.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:25.882+0000 7f2fed229700 1 -- 172.21.15.52:0/244549298 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2fe8108500 msgr2=0x7f2fe8101ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:52:25.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:25.882+0000 7f2fed229700 1 --2- 172.21.15.52:0/244549298 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2fe8108500 0x7f2fe8101ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:25.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:25.882+0000 7f2fed229700 1 -- 172.21.15.52:0/244549298 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2fe806a170 msgr2=0x7f2fe8101460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:52:25.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:25.882+0000 7f2fed229700 1 --2- 172.21.15.52:0/244549298 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2fe806a170 0x7f2fe8101460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:25.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:25.882+0000 7f2fed229700 1 -- 172.21.15.52:0/244549298 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fe81a72c0 con 0x7f2fe806ab30 2024-06-10T02:52:25.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:25.882+0000 7f2fed229700 1 --2- 172.21.15.52:0/244549298 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2fe806ab30 0x7f2fe81019a0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f2fd8000c00 tx=0x7f2fd8003680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:52:25.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:25.882+0000 7f2fdeffd700 1 -- 172.21.15.52:0/244549298 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fd8027070 con 0x7f2fe806ab30 2024-06-10T02:52:25.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:25.882+0000 7f2fefc8e700 1 -- 172.21.15.52:0/244549298 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fe81a7550 con 0x7f2fe806ab30 2024-06-10T02:52:25.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:25.882+0000 7f2fdeffd700 1 -- 172.21.15.52:0/244549298 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2fd8004050 con 0x7f2fe806ab30 2024-06-10T02:52:25.884 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:25.882+0000 7f2fdeffd700 1 -- 172.21.15.52:0/244549298 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fd800f460 con 0x7f2fe806ab30 2024-06-10T02:52:25.884 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:25.882+0000 7f2fefc8e700 1 -- 172.21.15.52:0/244549298 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fe81a79a0 con 0x7f2fe806ab30 2024-06-10T02:52:25.884 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:25.883+0000 7f2fefc8e700 1 -- 172.21.15.52:0/244549298 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2fe810a9e0 con 0x7f2fe806ab30 2024-06-10T02:52:25.884 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:25.884+0000 7f2fdeffd700 1 -- 172.21.15.52:0/244549298 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f2fd802a070 con 0x7f2fe806ab30 2024-06-10T02:52:25.884 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:25.885+0000 7f2fdeffd700 1 --2- 172.21.15.52:0/244549298 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f2fd40a2cb0 0x7f2fd40a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:52:25.885 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:25.885+0000 7f2feda2a700 1 --2- 172.21.15.52:0/244549298 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f2fd40a2cb0 0x7f2fd40a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:52:25.885 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:25.885+0000 7f2fdeffd700 1 -- 172.21.15.52:0/244549298 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f2fd80d0a90 con 0x7f2fe806ab30 2024-06-10T02:52:25.885 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:25.886+0000 7f2feda2a700 1 --2- 172.21.15.52:0/244549298 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f2fd40a2cb0 0x7f2fd40a5170 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f2fe4009a10 tx=0x7f2fe4009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:52:25.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:25.889+0000 7f2fdeffd700 1 -- 172.21.15.52:0/244549298 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f2fd8031070 con 0x7f2fe806ab30 2024-06-10T02:52:26.100 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:26.101+0000 7f2fefc8e700 1 -- 172.21.15.52:0/244549298 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7f2fe810a3c0 con 0x7f2fe806ab30 2024-06-10T02:52:26.100 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:26.101+0000 7f2fdeffd700 1 -- 172.21.15.52:0/244549298 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f2fd8013070 con 0x7f2fe806ab30 2024-06-10T02:52:26.100 INFO:teuthology.orchestra.run.smithi052.stdout:292057776183 2024-06-10T02:52:26.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:26.103+0000 7f2fefc8e700 1 -- 172.21.15.52:0/244549298 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f2fd40a2cb0 msgr2=0x7f2fd40a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:52:26.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:26.103+0000 7f2fefc8e700 1 --2- 172.21.15.52:0/244549298 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f2fd40a2cb0 0x7f2fd40a5170 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f2fe4009a10 tx=0x7f2fe4009190 comp rx=0 tx=0).stop 2024-06-10T02:52:26.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:26.103+0000 7f2fefc8e700 1 -- 172.21.15.52:0/244549298 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2fe806ab30 msgr2=0x7f2fe81019a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:52:26.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:26.103+0000 7f2fefc8e700 1 --2- 172.21.15.52:0/244549298 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2fe806ab30 0x7f2fe81019a0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f2fd8000c00 tx=0x7f2fd8003680 comp rx=0 tx=0).stop 2024-06-10T02:52:26.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:26.103+0000 7f2fefc8e700 1 -- 172.21.15.52:0/244549298 shutdown_connections 2024-06-10T02:52:26.103 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:26.103+0000 7f2fefc8e700 1 --2- 172.21.15.52:0/244549298 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2fe8108500 0x7f2fe8101ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:26.103 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:26.103+0000 7f2fefc8e700 1 --2- 172.21.15.52:0/244549298 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2fe806a170 0x7f2fe8101460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:26.103 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:26.103+0000 7f2fefc8e700 1 --2- 172.21.15.52:0/244549298 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f2fd40a2cb0 0x7f2fd40a5170 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:26.103 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:26.103+0000 7f2fefc8e700 1 --2- 172.21.15.52:0/244549298 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2fe806ab30 0x7f2fe81019a0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:26.103 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:26.103+0000 7f2fefc8e700 1 -- 172.21.15.52:0/244549298 >> 172.21.15.52:0/244549298 conn(0x7f2fe8078340 msgr2=0x7f2fe81001c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:52:26.103 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:26.103+0000 7f2fefc8e700 1 -- 172.21.15.52:0/244549298 shutdown_connections 2024-06-10T02:52:26.103 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:26.103+0000 7f2fefc8e700 1 -- 172.21.15.52:0/244549298 wait complete. 2024-06-10T02:52:26.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:26.193+0000 7fa18ba0a700 1 -- 172.21.15.52:0/2749178109 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa184102530 msgr2=0x7fa1841029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:52:26.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:26.193+0000 7fa18ba0a700 1 --2- 172.21.15.52:0/2749178109 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa184102530 0x7fa1841029b0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fa1740099c0 tx=0x7fa174009cd0 comp rx=0 tx=0).stop 2024-06-10T02:52:26.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:26.193+0000 7fa18ba0a700 1 -- 172.21.15.52:0/2749178109 shutdown_connections 2024-06-10T02:52:26.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:26.193+0000 7fa18ba0a700 1 --2- 172.21.15.52:0/2749178109 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa184102ef0 0x7fa18410f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:26.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:26.193+0000 7fa18ba0a700 1 --2- 172.21.15.52:0/2749178109 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa1841085d0 0x7fa1841089b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:26.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:26.193+0000 7fa18ba0a700 1 --2- 172.21.15.52:0/2749178109 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa184102530 0x7fa1841029b0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:26.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:26.193+0000 7fa18ba0a700 1 -- 172.21.15.52:0/2749178109 >> 172.21.15.52:0/2749178109 conn(0x7fa184079640 msgr2=0x7fa184079a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:52:26.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:26.193+0000 7fa18ba0a700 1 -- 172.21.15.52:0/2749178109 shutdown_connections 2024-06-10T02:52:26.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:26.193+0000 7fa18ba0a700 1 -- 172.21.15.52:0/2749178109 wait complete. 2024-06-10T02:52:26.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:26.194+0000 7fa18ba0a700 1 Processor -- start 2024-06-10T02:52:26.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:26.194+0000 7fa18ba0a700 1 -- start start 2024-06-10T02:52:26.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:26.194+0000 7fa18ba0a700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa184102530 0x7fa184074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:52:26.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:26.194+0000 7fa18ba0a700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa184102ef0 0x7fa184074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:52:26.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:26.194+0000 7fa18ba0a700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa1841085d0 0x7fa184075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:52:26.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:26.195+0000 7fa18ba0a700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fa184112830 con 0x7fa1841085d0 2024-06-10T02:52:26.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:26.195+0000 7fa18ba0a700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa1841126b0 con 0x7fa184102ef0 2024-06-10T02:52:26.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:26.195+0000 7fa18ba0a700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa1841129b0 con 0x7fa184102530 2024-06-10T02:52:26.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:26.195+0000 7fa189fa7700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa1841085d0 0x7fa184075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:52:26.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:26.195+0000 7fa189fa7700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa1841085d0 0x7fa184075210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:34550/0 (socket says 172.21.15.52:34550) 2024-06-10T02:52:26.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:26.195+0000 7fa189fa7700 1 -- 172.21.15.52:0/3817441244 learned_addr learned my addr 172.21.15.52:0/3817441244 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:52:26.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:26.195+0000 7fa1897a6700 1 --2- 172.21.15.52:0/3817441244 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa184102530 0x7fa184074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:52:26.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:26.195+0000 7fa188fa5700 1 --2- 172.21.15.52:0/3817441244 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa184102ef0 0x7fa184074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:52:26.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:26.195+0000 7fa189fa7700 1 -- 172.21.15.52:0/3817441244 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa184102530 msgr2=0x7fa184074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:52:26.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:26.195+0000 7fa189fa7700 1 --2- 172.21.15.52:0/3817441244 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa184102530 0x7fa184074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:26.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:26.195+0000 7fa189fa7700 1 -- 172.21.15.52:0/3817441244 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa184102ef0 msgr2=0x7fa184074cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:52:26.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:26.195+0000 7fa189fa7700 1 --2- 172.21.15.52:0/3817441244 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa184102ef0 0x7fa184074cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:26.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:26.195+0000 7fa189fa7700 1 -- 172.21.15.52:0/3817441244 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1841a71e0 con 0x7fa1841085d0 2024-06-10T02:52:26.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:26.195+0000 7fa189fa7700 1 --2- 172.21.15.52:0/3817441244 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa1841085d0 0x7fa184075210 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fa17c00ef30 tx=0x7fa17c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:52:26.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:26.195+0000 7fa17a7fc700 1 -- 172.21.15.52:0/3817441244 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa17c0092e0 con 0x7fa1841085d0 2024-06-10T02:52:26.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:26.195+0000 7fa17a7fc700 1 -- 172.21.15.52:0/3817441244 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa17c004bb0 con 0x7fa1841085d0 2024-06-10T02:52:26.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:26.195+0000 7fa18ba0a700 1 -- 172.21.15.52:0/3817441244 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1841a7470 con 0x7fa1841085d0 2024-06-10T02:52:26.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:26.195+0000 7fa17a7fc700 1 -- 172.21.15.52:0/3817441244 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa17c004280 con 0x7fa1841085d0 2024-06-10T02:52:26.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:26.196+0000 7fa18ba0a700 1 -- 172.21.15.52:0/3817441244 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1841a7930 con 0x7fa1841085d0 2024-06-10T02:52:26.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:26.196+0000 7fa18ba0a700 1 -- 172.21.15.52:0/3817441244 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa18410aa60 con 0x7fa1841085d0 2024-06-10T02:52:26.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:26.198+0000 7fa17a7fc700 1 -- 172.21.15.52:0/3817441244 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fa17c01b030 con 0x7fa1841085d0 2024-06-10T02:52:26.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:26.199+0000 7fa17a7fc700 1 --2- 172.21.15.52:0/3817441244 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fa1700a2cb0 0x7fa1700a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:52:26.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:26.199+0000 7fa1897a6700 1 --2- 172.21.15.52:0/3817441244 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fa1700a2cb0 0x7fa1700a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:52:26.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:26.199+0000 7fa17a7fc700 1 -- 172.21.15.52:0/3817441244 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7fa17c0cfa10 con 0x7fa1841085d0 2024-06-10T02:52:26.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:26.200+0000 7fa1897a6700 1 --2- 172.21.15.52:0/3817441244 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fa1700a2cb0 0x7fa1700a5170 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fa180009be0 tx=0x7fa180009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:52:26.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:26.204+0000 7fa17a7fc700 1 -- 172.21.15.52:0/3817441244 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fa17c022020 con 0x7fa1841085d0 2024-06-10T02:52:26.413 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:26.414+0000 7fa18ba0a700 1 -- 172.21.15.52:0/3817441244 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fa1840691b0 con 0x7fa1841085d0 2024-06-10T02:52:26.413 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:26.414+0000 7fa17a7fc700 1 -- 172.21.15.52:0/3817441244 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fa1840691b0 con 0x7fa1841085d0 2024-06-10T02:52:26.414 INFO:teuthology.orchestra.run.smithi052.stdout:77309411419 2024-06-10T02:52:26.416 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:26.417+0000 7fa18ba0a700 1 -- 172.21.15.52:0/3817441244 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fa1700a2cb0 msgr2=0x7fa1700a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:52:26.416 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:26.417+0000 7fa18ba0a700 1 --2- 172.21.15.52:0/3817441244 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fa1700a2cb0 0x7fa1700a5170 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fa180009be0 tx=0x7fa180009260 comp rx=0 tx=0).stop 2024-06-10T02:52:26.416 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:26.417+0000 7fa18ba0a700 1 -- 172.21.15.52:0/3817441244 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa1841085d0 msgr2=0x7fa184075210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:52:26.416 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:26.417+0000 7fa18ba0a700 1 --2- 172.21.15.52:0/3817441244 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa1841085d0 0x7fa184075210 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fa17c00ef30 tx=0x7fa17c00c3a0 comp rx=0 tx=0).stop 2024-06-10T02:52:26.416 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:26.417+0000 7fa18ba0a700 1 -- 172.21.15.52:0/3817441244 shutdown_connections 2024-06-10T02:52:26.416 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:26.417+0000 7fa18ba0a700 1 --2- 172.21.15.52:0/3817441244 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fa1700a2cb0 0x7fa1700a5170 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:26.416 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:26.417+0000 7fa18ba0a700 1 --2- 172.21.15.52:0/3817441244 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa1841085d0 0x7fa184075210 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:26.416 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:26.417+0000 7fa18ba0a700 1 --2- 172.21.15.52:0/3817441244 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa184102530 0x7fa184074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:26.416 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:26.417+0000 7fa18ba0a700 1 --2- 172.21.15.52:0/3817441244 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa184102ef0 0x7fa184074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:26.416 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:26.417+0000 7fa18ba0a700 1 -- 172.21.15.52:0/3817441244 >> 172.21.15.52:0/3817441244 conn(0x7fa184079640 msgr2=0x7fa1841010d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:52:26.417 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:26.417+0000 7fa18ba0a700 1 -- 172.21.15.52:0/3817441244 shutdown_connections 2024-06-10T02:52:26.417 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:26.417+0000 7fa18ba0a700 1 -- 172.21.15.52:0/3817441244 wait complete. 2024-06-10T02:52:26.878 INFO:tasks.cephadm.ceph_manager.ceph:need seq 292057776180 got 292057776183 for osd.9 2024-06-10T02:52:26.878 DEBUG:teuthology.parallel:result is None 2024-06-10T02:52:27.109 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:52:26 smithi052 ceph-mon[32362]: pgmap v381: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-10T02:52:27.109 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:52:26 smithi052 ceph-mon[32362]: from='client.? 172.21.15.52:0/244549298' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-06-10T02:52:27.109 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:52:26 smithi052 ceph-mon[32362]: from='client.? 172.21.15.52:0/3817441244' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-06-10T02:52:27.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:52:26 smithi120 ceph-mon[36634]: pgmap v381: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-10T02:52:27.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:52:26 smithi120 ceph-mon[36634]: from='client.? 172.21.15.52:0/244549298' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-06-10T02:52:27.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:52:26 smithi120 ceph-mon[36634]: from='client.? 172.21.15.52:0/3817441244' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-06-10T02:52:27.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:52:26 smithi188 ceph-mon[36599]: pgmap v381: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-10T02:52:27.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:52:26 smithi188 ceph-mon[36599]: from='client.? 172.21.15.52:0/244549298' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-06-10T02:52:27.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:52:26 smithi188 ceph-mon[36599]: from='client.? 172.21.15.52:0/3817441244' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-06-10T02:52:28.940 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411416 got 77309411419 for osd.2 2024-06-10T02:52:28.940 DEBUG:teuthology.parallel:result is None 2024-06-10T02:52:29.101 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:52:28 smithi052 ceph-mon[32362]: pgmap v382: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-10T02:52:29.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:52:28 smithi120 ceph-mon[36634]: pgmap v382: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-10T02:52:29.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:52:28 smithi188 ceph-mon[36599]: pgmap v382: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-10T02:52:29.411 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:29.412+0000 7f2f1de05700 1 -- 172.21.15.52:0/655843364 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2f18102770 msgr2=0x7f2f18102b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:52:29.411 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:29.412+0000 7f2f1de05700 1 --2- 172.21.15.52:0/655843364 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2f18102770 0x7f2f18102b50 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f2f080099c0 tx=0x7f2f08009cd0 comp rx=0 tx=0).stop 2024-06-10T02:52:29.411 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:29.412+0000 7f2f1de05700 1 -- 172.21.15.52:0/655843364 shutdown_connections 2024-06-10T02:52:29.411 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:29.412+0000 7f2f1de05700 1 --2- 172.21.15.52:0/655843364 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2f18107860 0x7f2f1810f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:29.411 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:29.412+0000 7f2f1de05700 1 --2- 172.21.15.52:0/655843364 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2f18102770 0x7f2f18102b50 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:29.411 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:29.412+0000 7f2f1de05700 1 --2- 172.21.15.52:0/655843364 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2f18103120 0x7f2f18107130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:29.411 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:29.412+0000 7f2f1de05700 1 -- 172.21.15.52:0/655843364 >> 172.21.15.52:0/655843364 conn(0x7f2f18079600 msgr2=0x7f2f18079a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:52:29.411 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:29.412+0000 7f2f1de05700 1 -- 172.21.15.52:0/655843364 shutdown_connections 2024-06-10T02:52:29.411 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:29.412+0000 7f2f1de05700 1 -- 172.21.15.52:0/655843364 wait complete. 2024-06-10T02:52:29.413 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:29.412+0000 7f2f1de05700 1 Processor -- start 2024-06-10T02:52:29.413 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:29.413+0000 7f2f1de05700 1 -- start start 2024-06-10T02:52:29.413 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:29.413+0000 7f2f1de05700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2f18102770 0x7f2f1819e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:52:29.413 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:29.413+0000 7f2f1de05700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2f18103120 0x7f2f1819ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:52:29.413 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:29.413+0000 7f2f1de05700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2f18107860 0x7f2f181989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:52:29.413 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:29.413+0000 7f2f1de05700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f2f181127a0 con 0x7f2f18107860 2024-06-10T02:52:29.414 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:29.413+0000 7f2f1de05700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f2f18112620 con 0x7f2f18103120 2024-06-10T02:52:29.414 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:29.413+0000 7f2f1de05700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f2f18112920 con 0x7f2f18102770 2024-06-10T02:52:29.414 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:29.413+0000 7f2f17fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2f18107860 0x7f2f181989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:52:29.414 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:29.413+0000 7f2f17fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2f18107860 0x7f2f181989c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:34570/0 (socket says 172.21.15.52:34570) 2024-06-10T02:52:29.414 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:29.413+0000 7f2f17fff700 1 -- 172.21.15.52:0/3884027481 learned_addr learned my addr 172.21.15.52:0/3884027481 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:52:29.414 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:29.413+0000 7f2f16ffd700 1 --2- 172.21.15.52:0/3884027481 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2f18103120 0x7f2f1819ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:52:29.414 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:29.413+0000 7f2f177fe700 1 --2- 172.21.15.52:0/3884027481 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2f18102770 0x7f2f1819e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:52:29.414 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:29.414+0000 7f2f17fff700 1 -- 172.21.15.52:0/3884027481 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2f18102770 msgr2=0x7f2f1819e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:52:29.414 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:29.414+0000 7f2f17fff700 1 --2- 172.21.15.52:0/3884027481 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2f18102770 0x7f2f1819e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:29.414 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:29.414+0000 7f2f17fff700 1 -- 172.21.15.52:0/3884027481 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2f18103120 msgr2=0x7f2f1819ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:52:29.414 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:29.414+0000 7f2f17fff700 1 --2- 172.21.15.52:0/3884027481 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2f18103120 0x7f2f1819ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:29.414 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:29.414+0000 7f2f17fff700 1 -- 172.21.15.52:0/3884027481 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f18199120 con 0x7f2f18107860 2024-06-10T02:52:29.415 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:29.414+0000 7f2f17fff700 1 --2- 172.21.15.52:0/3884027481 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2f18107860 0x7f2f181989c0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f2f10007ee0 tx=0x7f2f1000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:52:29.415 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:29.414+0000 7f2f14ff9700 1 -- 172.21.15.52:0/3884027481 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f10004bb0 con 0x7f2f18107860 2024-06-10T02:52:29.415 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:29.414+0000 7f2f14ff9700 1 -- 172.21.15.52:0/3884027481 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2f10004d50 con 0x7f2f18107860 2024-06-10T02:52:29.415 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:29.414+0000 7f2f1de05700 1 -- 172.21.15.52:0/3884027481 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f18199410 con 0x7f2f18107860 2024-06-10T02:52:29.415 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:29.414+0000 7f2f14ff9700 1 -- 172.21.15.52:0/3884027481 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f10005960 con 0x7f2f18107860 2024-06-10T02:52:29.415 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:29.414+0000 7f2f1de05700 1 -- 172.21.15.52:0/3884027481 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f181a56d0 con 0x7f2f18107860 2024-06-10T02:52:29.416 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:29.415+0000 7f2f1de05700 1 -- 172.21.15.52:0/3884027481 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f1810aaa0 con 0x7f2f18107860 2024-06-10T02:52:29.416 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:29.416+0000 7f2f14ff9700 1 -- 172.21.15.52:0/3884027481 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f2f1001a020 con 0x7f2f18107860 2024-06-10T02:52:29.416 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:29.417+0000 7f2f14ff9700 1 --2- 172.21.15.52:0/3884027481 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f2f000a2d30 0x7f2f000a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:52:29.416 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:29.417+0000 7f2f177fe700 1 --2- 172.21.15.52:0/3884027481 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f2f000a2d30 0x7f2f000a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:52:29.416 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:29.417+0000 7f2f14ff9700 1 -- 172.21.15.52:0/3884027481 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f2f100ced00 con 0x7f2f18107860 2024-06-10T02:52:29.416 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:29.418+0000 7f2f177fe700 1 --2- 172.21.15.52:0/3884027481 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f2f000a2d30 0x7f2f000a51f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f2f08000c00 tx=0x7f2f08005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:52:29.420 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:29.422+0000 7f2f14ff9700 1 -- 172.21.15.52:0/3884027481 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f2f10010070 con 0x7f2f18107860 2024-06-10T02:52:29.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:29.636+0000 7f2f1de05700 1 -- 172.21.15.52:0/3884027481 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f2f1810a480 con 0x7f2f18107860 2024-06-10T02:52:29.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:29.636+0000 7f2f14ff9700 1 -- 172.21.15.52:0/3884027481 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f2f1009b6f0 con 0x7f2f18107860 2024-06-10T02:52:29.635 INFO:teuthology.orchestra.run.smithi052.stdout:167503724620 2024-06-10T02:52:29.638 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:29.638+0000 7f2f1de05700 1 -- 172.21.15.52:0/3884027481 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f2f000a2d30 msgr2=0x7f2f000a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:52:29.638 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:29.638+0000 7f2f1de05700 1 --2- 172.21.15.52:0/3884027481 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f2f000a2d30 0x7f2f000a51f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f2f08000c00 tx=0x7f2f08005fb0 comp rx=0 tx=0).stop 2024-06-10T02:52:29.638 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:29.638+0000 7f2f1de05700 1 -- 172.21.15.52:0/3884027481 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2f18107860 msgr2=0x7f2f181989c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:52:29.638 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:29.638+0000 7f2f1de05700 1 --2- 172.21.15.52:0/3884027481 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2f18107860 0x7f2f181989c0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f2f10007ee0 tx=0x7f2f1000d3b0 comp rx=0 tx=0).stop 2024-06-10T02:52:29.638 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:29.639+0000 7f2f1de05700 1 -- 172.21.15.52:0/3884027481 shutdown_connections 2024-06-10T02:52:29.638 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:29.639+0000 7f2f1de05700 1 --2- 172.21.15.52:0/3884027481 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f2f000a2d30 0x7f2f000a51f0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:29.638 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:29.639+0000 7f2f1de05700 1 --2- 172.21.15.52:0/3884027481 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2f18107860 0x7f2f181989c0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:29.638 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:29.639+0000 7f2f1de05700 1 --2- 172.21.15.52:0/3884027481 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2f18102770 0x7f2f1819e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:29.638 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:29.639+0000 7f2f1de05700 1 --2- 172.21.15.52:0/3884027481 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2f18103120 0x7f2f1819ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:29.638 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:29.639+0000 7f2f1de05700 1 -- 172.21.15.52:0/3884027481 >> 172.21.15.52:0/3884027481 conn(0x7f2f18079600 msgr2=0x7f2f18100520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:52:29.638 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:29.639+0000 7f2f1de05700 1 -- 172.21.15.52:0/3884027481 shutdown_connections 2024-06-10T02:52:29.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:29.639+0000 7f2f1de05700 1 -- 172.21.15.52:0/3884027481 wait complete. 2024-06-10T02:52:29.976 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:52:29 smithi052 ceph-mon[32362]: from='client.? 172.21.15.52:0/3884027481' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-06-10T02:52:30.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:52:29 smithi120 ceph-mon[36634]: from='client.? 172.21.15.52:0/3884027481' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-06-10T02:52:30.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:52:29 smithi188 ceph-mon[36599]: from='client.? 172.21.15.52:0/3884027481' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-06-10T02:52:30.964 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:52:30 smithi052 ceph-mon[32362]: pgmap v383: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-10T02:52:31.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:52:30 smithi120 ceph-mon[36634]: pgmap v383: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-10T02:52:31.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:52:30 smithi188 ceph-mon[36599]: pgmap v383: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-10T02:52:31.374 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:52:32.644 INFO:tasks.cephadm.ceph_manager.ceph:need seq 167503724615 got 167503724620 for osd.5 2024-06-10T02:52:32.644 DEBUG:teuthology.parallel:result is None 2024-06-10T02:52:32.976 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:52:32 smithi052 ceph-mon[32362]: pgmap v384: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-06-10T02:52:33.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:52:32 smithi120 ceph-mon[36634]: pgmap v384: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-06-10T02:52:33.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:52:32 smithi188 ceph-mon[36599]: pgmap v384: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-06-10T02:52:33.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:33.711+0000 7f9dd9b50700 1 -- 172.21.15.52:0/2429475105 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9dd4101d50 msgr2=0x7f9dd4102130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:52:33.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:33.711+0000 7f9dd9b50700 1 --2- 172.21.15.52:0/2429475105 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9dd4101d50 0x7f9dd4102130 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f9dc40099c0 tx=0x7f9dc4009cd0 comp rx=0 tx=0).stop 2024-06-10T02:52:33.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:33.711+0000 7f9dd9b50700 1 -- 172.21.15.52:0/2429475105 shutdown_connections 2024-06-10T02:52:33.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:33.711+0000 7f9dd9b50700 1 --2- 172.21.15.52:0/2429475105 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9dd4106700 0x7f9dd410e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:33.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:33.711+0000 7f9dd9b50700 1 --2- 172.21.15.52:0/2429475105 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9dd4101d50 0x7f9dd4102130 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:33.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:33.711+0000 7f9dd9b50700 1 --2- 172.21.15.52:0/2429475105 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9dd4102700 0x7f9dd4106020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:33.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:33.711+0000 7f9dd9b50700 1 -- 172.21.15.52:0/2429475105 >> 172.21.15.52:0/2429475105 conn(0x7f9dd4077960 msgr2=0x7f9dd40fed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:52:33.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:33.711+0000 7f9dd9b50700 1 -- 172.21.15.52:0/2429475105 shutdown_connections 2024-06-10T02:52:33.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:33.711+0000 7f9dd9b50700 1 -- 172.21.15.52:0/2429475105 wait complete. 2024-06-10T02:52:33.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:33.712+0000 7f9dd9b50700 1 Processor -- start 2024-06-10T02:52:33.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:33.712+0000 7f9dd9b50700 1 -- start start 2024-06-10T02:52:33.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:33.712+0000 7f9dd9b50700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9dd4101d50 0x7f9dd419d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:52:33.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:33.712+0000 7f9dd9b50700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9dd4102700 0x7f9dd419ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:52:33.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:33.712+0000 7f9dd9b50700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9dd4106700 0x7f9dd4197a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:52:33.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:33.712+0000 7f9dd9b50700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f9dd4111770 con 0x7f9dd4106700 2024-06-10T02:52:33.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:33.712+0000 7f9dd9b50700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f9dd41115f0 con 0x7f9dd4102700 2024-06-10T02:52:33.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:33.712+0000 7f9dd9b50700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f9dd41118f0 con 0x7f9dd4101d50 2024-06-10T02:52:33.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:33.713+0000 7f9dd3fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9dd4106700 0x7f9dd4197a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:52:33.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:33.713+0000 7f9dd3fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9dd4106700 0x7f9dd4197a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:35352/0 (socket says 172.21.15.52:35352) 2024-06-10T02:52:33.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:33.713+0000 7f9dd3fff700 1 -- 172.21.15.52:0/1980589728 learned_addr learned my addr 172.21.15.52:0/1980589728 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:52:33.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:33.713+0000 7f9dd37fe700 1 --2- 172.21.15.52:0/1980589728 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9dd4101d50 0x7f9dd419d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:52:33.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:33.713+0000 7f9dd2ffd700 1 --2- 172.21.15.52:0/1980589728 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9dd4102700 0x7f9dd419ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:52:33.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:33.713+0000 7f9dd3fff700 1 -- 172.21.15.52:0/1980589728 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9dd4101d50 msgr2=0x7f9dd419d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:52:33.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:33.713+0000 7f9dd3fff700 1 --2- 172.21.15.52:0/1980589728 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9dd4101d50 0x7f9dd419d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:33.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:33.713+0000 7f9dd3fff700 1 -- 172.21.15.52:0/1980589728 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9dd4102700 msgr2=0x7f9dd419ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:52:33.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:33.713+0000 7f9dd3fff700 1 --2- 172.21.15.52:0/1980589728 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9dd4102700 0x7f9dd419ddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:33.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:33.713+0000 7f9dd3fff700 1 -- 172.21.15.52:0/1980589728 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9dd41981d0 con 0x7f9dd4106700 2024-06-10T02:52:33.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:33.713+0000 7f9dd2ffd700 1 --2- 172.21.15.52:0/1980589728 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9dd4102700 0x7f9dd419ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T02:52:33.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:33.713+0000 7f9dd3fff700 1 --2- 172.21.15.52:0/1980589728 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9dd4106700 0x7f9dd4197a70 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f9dcc007ee0 tx=0x7f9dcc00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:52:33.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:33.713+0000 7f9dd0ff9700 1 -- 172.21.15.52:0/1980589728 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dcc00dbd0 con 0x7f9dd4106700 2024-06-10T02:52:33.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:33.713+0000 7f9dd0ff9700 1 -- 172.21.15.52:0/1980589728 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9dcc00dd70 con 0x7f9dd4106700 2024-06-10T02:52:33.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:33.713+0000 7f9dd0ff9700 1 -- 172.21.15.52:0/1980589728 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dcc005a60 con 0x7f9dd4106700 2024-06-10T02:52:33.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:33.713+0000 7f9dd9b50700 1 -- 172.21.15.52:0/1980589728 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9dd41984c0 con 0x7f9dd4106700 2024-06-10T02:52:33.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:33.713+0000 7f9dd9b50700 1 -- 172.21.15.52:0/1980589728 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9dd41a4780 con 0x7f9dd4106700 2024-06-10T02:52:33.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:33.714+0000 7f9dd9b50700 1 -- 172.21.15.52:0/1980589728 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9dd4109990 con 0x7f9dd4106700 2024-06-10T02:52:33.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:33.716+0000 7f9dd0ff9700 1 -- 172.21.15.52:0/1980589728 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f9dcc01a020 con 0x7f9dd4106700 2024-06-10T02:52:33.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:33.716+0000 7f9dd0ff9700 1 --2- 172.21.15.52:0/1980589728 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f9dbc0a2d30 0x7f9dbc0a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:52:33.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:33.716+0000 7f9dd37fe700 1 --2- 172.21.15.52:0/1980589728 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f9dbc0a2d30 0x7f9dbc0a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:52:33.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:33.716+0000 7f9dd0ff9700 1 -- 172.21.15.52:0/1980589728 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f9dcc0cebf0 con 0x7f9dd4106700 2024-06-10T02:52:33.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:33.717+0000 7f9dd37fe700 1 --2- 172.21.15.52:0/1980589728 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f9dbc0a2d30 0x7f9dbc0a51f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f9dc4000c00 tx=0x7f9dc4005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:52:33.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:33.721+0000 7f9dd0ff9700 1 -- 172.21.15.52:0/1980589728 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f9dcc010070 con 0x7f9dd4106700 2024-06-10T02:52:33.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:33.953+0000 7f9dd9b50700 1 -- 172.21.15.52:0/1980589728 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f9dd4198fa0 con 0x7f9dd4106700 2024-06-10T02:52:33.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:33.953+0000 7f9dd0ff9700 1 -- 172.21.15.52:0/1980589728 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f9dcc09b640 con 0x7f9dd4106700 2024-06-10T02:52:33.952 INFO:teuthology.orchestra.run.smithi052.stdout:34359738470 2024-06-10T02:52:33.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:33.955+0000 7f9dd9b50700 1 -- 172.21.15.52:0/1980589728 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f9dbc0a2d30 msgr2=0x7f9dbc0a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:52:33.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:33.955+0000 7f9dd9b50700 1 --2- 172.21.15.52:0/1980589728 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f9dbc0a2d30 0x7f9dbc0a51f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f9dc4000c00 tx=0x7f9dc4005fb0 comp rx=0 tx=0).stop 2024-06-10T02:52:33.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:33.955+0000 7f9dd9b50700 1 -- 172.21.15.52:0/1980589728 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9dd4106700 msgr2=0x7f9dd4197a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:52:33.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:33.955+0000 7f9dd9b50700 1 --2- 172.21.15.52:0/1980589728 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9dd4106700 0x7f9dd4197a70 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f9dcc007ee0 tx=0x7f9dcc00d3b0 comp rx=0 tx=0).stop 2024-06-10T02:52:33.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:33.955+0000 7f9dd9b50700 1 -- 172.21.15.52:0/1980589728 shutdown_connections 2024-06-10T02:52:33.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:33.956+0000 7f9dd9b50700 1 --2- 172.21.15.52:0/1980589728 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f9dbc0a2d30 0x7f9dbc0a51f0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:33.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:33.956+0000 7f9dd9b50700 1 --2- 172.21.15.52:0/1980589728 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9dd4106700 0x7f9dd4197a70 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:33.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:33.956+0000 7f9dd9b50700 1 --2- 172.21.15.52:0/1980589728 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9dd4101d50 0x7f9dd419d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:33.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:33.956+0000 7f9dd9b50700 1 --2- 172.21.15.52:0/1980589728 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9dd4102700 0x7f9dd419ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:33.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:33.956+0000 7f9dd9b50700 1 -- 172.21.15.52:0/1980589728 >> 172.21.15.52:0/1980589728 conn(0x7f9dd4077960 msgr2=0x7f9dd40fed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:52:33.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:33.956+0000 7f9dd9b50700 1 -- 172.21.15.52:0/1980589728 shutdown_connections 2024-06-10T02:52:33.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:33.956+0000 7f9dd9b50700 1 -- 172.21.15.52:0/1980589728 wait complete. 2024-06-10T02:52:35.006 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:35.006+0000 7f271c518700 1 -- 172.21.15.52:0/379694886 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2714101d30 msgr2=0x7f271410e210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:52:35.006 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:35.006+0000 7f271c518700 1 --2- 172.21.15.52:0/379694886 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2714101d30 0x7f271410e210 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f270c00b0a0 tx=0x7f270c00b3b0 comp rx=0 tx=0).stop 2024-06-10T02:52:35.006 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:35.006+0000 7f271c518700 1 -- 172.21.15.52:0/379694886 shutdown_connections 2024-06-10T02:52:35.006 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:35.006+0000 7f271c518700 1 --2- 172.21.15.52:0/379694886 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2714101d30 0x7f271410e210 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:35.006 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:35.006+0000 7f271c518700 1 --2- 172.21.15.52:0/379694886 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f27141073d0 0x7f27141077b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:35.006 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:35.006+0000 7f271c518700 1 --2- 172.21.15.52:0/379694886 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2714101390 0x7f27141017f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:35.006 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:35.007+0000 7f271c518700 1 -- 172.21.15.52:0/379694886 >> 172.21.15.52:0/379694886 conn(0x7f27140782c0 msgr2=0x7f27140786d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:52:35.006 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:35.007+0000 7f271c518700 1 -- 172.21.15.52:0/379694886 shutdown_connections 2024-06-10T02:52:35.006 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:35.007+0000 7f271c518700 1 -- 172.21.15.52:0/379694886 wait complete. 2024-06-10T02:52:35.008 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:35.007+0000 7f271c518700 1 Processor -- start 2024-06-10T02:52:35.008 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:35.007+0000 7f271c518700 1 -- start start 2024-06-10T02:52:35.008 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:35.007+0000 7f271c518700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2714101390 0x7f271419e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:52:35.008 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:35.007+0000 7f271c518700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2714101d30 0x7f271419ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:52:35.008 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:35.007+0000 7f271c518700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f27141073d0 0x7f2714198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:52:35.008 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:35.007+0000 7f271c518700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f27141127b0 con 0x7f2714101d30 2024-06-10T02:52:35.008 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:35.007+0000 7f271c518700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f2714112630 con 0x7f27141073d0 2024-06-10T02:52:35.009 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:35.007+0000 7f271c518700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f2714112930 con 0x7f2714101390 2024-06-10T02:52:35.009 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:35.008+0000 7f2719ab3700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2714101d30 0x7f271419ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:52:35.009 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:35.008+0000 7f2719ab3700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2714101d30 0x7f271419ecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:35364/0 (socket says 172.21.15.52:35364) 2024-06-10T02:52:35.009 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:35.008+0000 7f2719ab3700 1 -- 172.21.15.52:0/1967591282 learned_addr learned my addr 172.21.15.52:0/1967591282 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:52:35.009 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:35.008+0000 7f271aab5700 1 --2- 172.21.15.52:0/1967591282 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f27141073d0 0x7f2714198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:52:35.009 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:35.008+0000 7f271a2b4700 1 --2- 172.21.15.52:0/1967591282 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2714101390 0x7f271419e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:52:35.009 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:35.008+0000 7f2719ab3700 1 -- 172.21.15.52:0/1967591282 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2714101390 msgr2=0x7f271419e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:52:35.009 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:35.008+0000 7f2719ab3700 1 --2- 172.21.15.52:0/1967591282 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2714101390 0x7f271419e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:35.009 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:35.008+0000 7f2719ab3700 1 -- 172.21.15.52:0/1967591282 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f27141073d0 msgr2=0x7f2714198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:52:35.009 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:35.008+0000 7f2719ab3700 1 --2- 172.21.15.52:0/1967591282 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f27141073d0 0x7f2714198950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:35.009 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:35.008+0000 7f2719ab3700 1 -- 172.21.15.52:0/1967591282 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27141990b0 con 0x7f2714101d30 2024-06-10T02:52:35.009 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:35.008+0000 7f271a2b4700 1 --2- 172.21.15.52:0/1967591282 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2714101390 0x7f271419e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T02:52:35.009 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:35.008+0000 7f2719ab3700 1 --2- 172.21.15.52:0/1967591282 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2714101d30 0x7f271419ecd0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f270400d950 tx=0x7f270400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:52:35.010 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:35.008+0000 7f270b7fe700 1 -- 172.21.15.52:0/1967591282 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2704016070 con 0x7f2714101d30 2024-06-10T02:52:35.010 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:35.008+0000 7f271c518700 1 -- 172.21.15.52:0/1967591282 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27141993a0 con 0x7f2714101d30 2024-06-10T02:52:35.010 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:35.008+0000 7f270b7fe700 1 -- 172.21.15.52:0/1967591282 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2704009e20 con 0x7f2714101d30 2024-06-10T02:52:35.010 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:35.008+0000 7f270b7fe700 1 -- 172.21.15.52:0/1967591282 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f270400f620 con 0x7f2714101d30 2024-06-10T02:52:35.010 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:35.008+0000 7f271c518700 1 -- 172.21.15.52:0/1967591282 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27141a5660 con 0x7f2714101d30 2024-06-10T02:52:35.010 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:35.009+0000 7f271c518700 1 -- 172.21.15.52:0/1967591282 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2714069180 con 0x7f2714101d30 2024-06-10T02:52:35.011 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:35.011+0000 7f270b7fe700 1 -- 172.21.15.52:0/1967591282 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f2704019070 con 0x7f2714101d30 2024-06-10T02:52:35.011 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:35.011+0000 7f270b7fe700 1 --2- 172.21.15.52:0/1967591282 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f27000a2cb0 0x7f27000a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:52:35.011 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:35.011+0000 7f270b7fe700 1 -- 172.21.15.52:0/1967591282 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f27040cf140 con 0x7f2714101d30 2024-06-10T02:52:35.011 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:35.011+0000 7f271a2b4700 1 --2- 172.21.15.52:0/1967591282 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f27000a2cb0 0x7f27000a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:52:35.011 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:35.012+0000 7f271a2b4700 1 --2- 172.21.15.52:0/1967591282 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f27000a2cb0 0x7f27000a5170 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f2710005e10 tx=0x7f2710005d60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:52:35.013 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:35.015+0000 7f270b7fe700 1 -- 172.21.15.52:0/1967591282 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f270409bb90 con 0x7f2714101d30 2024-06-10T02:52:35.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:52:34 smithi120 ceph-mon[36634]: pgmap v385: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-06-10T02:52:35.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:52:34 smithi120 ceph-mon[36634]: from='client.? 172.21.15.52:0/1980589728' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-06-10T02:52:35.199 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:52:35.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:52:34 smithi188 ceph-mon[36599]: pgmap v385: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-06-10T02:52:35.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:52:34 smithi188 ceph-mon[36599]: from='client.? 172.21.15.52:0/1980589728' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-06-10T02:52:35.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:52:34 smithi052 ceph-mon[32362]: pgmap v385: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-06-10T02:52:35.226 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:52:34 smithi052 ceph-mon[32362]: from='client.? 172.21.15.52:0/1980589728' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-06-10T02:52:35.226 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:35.227+0000 7f271c518700 1 -- 172.21.15.52:0/1967591282 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7f271402dca0 con 0x7f2714101d30 2024-06-10T02:52:35.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:35.228+0000 7f270b7fe700 1 -- 172.21.15.52:0/1967591282 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f270409b490 con 0x7f2714101d30 2024-06-10T02:52:35.227 INFO:teuthology.orchestra.run.smithi052.stdout:257698037821 2024-06-10T02:52:35.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:35.230+0000 7f271c518700 1 -- 172.21.15.52:0/1967591282 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f27000a2cb0 msgr2=0x7f27000a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:52:35.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:35.230+0000 7f271c518700 1 --2- 172.21.15.52:0/1967591282 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f27000a2cb0 0x7f27000a5170 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f2710005e10 tx=0x7f2710005d60 comp rx=0 tx=0).stop 2024-06-10T02:52:35.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:35.230+0000 7f271c518700 1 -- 172.21.15.52:0/1967591282 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2714101d30 msgr2=0x7f271419ecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:52:35.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:35.230+0000 7f271c518700 1 --2- 172.21.15.52:0/1967591282 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2714101d30 0x7f271419ecd0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f270400d950 tx=0x7f270400dc60 comp rx=0 tx=0).stop 2024-06-10T02:52:35.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:35.230+0000 7f271c518700 1 -- 172.21.15.52:0/1967591282 shutdown_connections 2024-06-10T02:52:35.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:35.230+0000 7f271c518700 1 --2- 172.21.15.52:0/1967591282 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f27141073d0 0x7f2714198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:35.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:35.230+0000 7f271c518700 1 --2- 172.21.15.52:0/1967591282 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2714101390 0x7f271419e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:35.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:35.230+0000 7f271c518700 1 --2- 172.21.15.52:0/1967591282 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f27000a2cb0 0x7f27000a5170 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:35.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:35.230+0000 7f271c518700 1 --2- 172.21.15.52:0/1967591282 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2714101d30 0x7f271419ecd0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:35.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:35.230+0000 7f271c518700 1 -- 172.21.15.52:0/1967591282 >> 172.21.15.52:0/1967591282 conn(0x7f27140782c0 msgr2=0x7f27141001d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:52:35.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:35.230+0000 7f271c518700 1 -- 172.21.15.52:0/1967591282 shutdown_connections 2024-06-10T02:52:35.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:35.230+0000 7f271c518700 1 -- 172.21.15.52:0/1967591282 wait complete. 2024-06-10T02:52:35.742 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738464 got 34359738470 for osd.0 2024-06-10T02:52:35.743 DEBUG:teuthology.parallel:result is None 2024-06-10T02:52:35.975 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:52:35 smithi052 ceph-mon[32362]: from='client.? 172.21.15.52:0/1967591282' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-06-10T02:52:36.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:52:35 smithi120 ceph-mon[36634]: from='client.? 172.21.15.52:0/1967591282' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-06-10T02:52:36.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:52:35 smithi188 ceph-mon[36599]: from='client.? 172.21.15.52:0/1967591282' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-06-10T02:52:36.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:36.512+0000 7fad1c775700 1 -- 172.21.15.52:0/465638367 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fad1406aa90 msgr2=0x7fad1410ea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:52:36.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:36.512+0000 7fad1c775700 1 --2- 172.21.15.52:0/465638367 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fad1406aa90 0x7fad1410ea00 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fad04009990 tx=0x7fad04009ca0 comp rx=0 tx=0).stop 2024-06-10T02:52:36.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:36.512+0000 7fad1c775700 1 -- 172.21.15.52:0/465638367 shutdown_connections 2024-06-10T02:52:36.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:36.512+0000 7fad1c775700 1 --2- 172.21.15.52:0/465638367 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fad1410ef40 0x7fad14111330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:36.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:36.512+0000 7fad1c775700 1 --2- 172.21.15.52:0/465638367 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fad1406a170 0x7fad1406a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:36.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:36.512+0000 7fad1c775700 1 --2- 172.21.15.52:0/465638367 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fad1406aa90 0x7fad1410ea00 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:36.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:36.512+0000 7fad1c775700 1 -- 172.21.15.52:0/465638367 >> 172.21.15.52:0/465638367 conn(0x7fad140782c0 msgr2=0x7fad140786d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:52:36.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:36.512+0000 7fad1c775700 1 -- 172.21.15.52:0/465638367 shutdown_connections 2024-06-10T02:52:36.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:36.512+0000 7fad1c775700 1 -- 172.21.15.52:0/465638367 wait complete. 2024-06-10T02:52:36.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:36.513+0000 7fad1c775700 1 Processor -- start 2024-06-10T02:52:36.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:36.513+0000 7fad1c775700 1 -- start start 2024-06-10T02:52:36.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:36.513+0000 7fad1c775700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fad1406a170 0x7fad1419aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:52:36.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:36.513+0000 7fad1c775700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fad1406aa90 0x7fad1419b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:52:36.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:36.513+0000 7fad1c775700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fad1410ef40 0x7fad1419f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:52:36.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:36.513+0000 7fad1c775700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fad14114960 con 0x7fad1410ef40 2024-06-10T02:52:36.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:36.513+0000 7fad1c775700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fad141147e0 con 0x7fad1406a170 2024-06-10T02:52:36.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:36.513+0000 7fad1c775700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fad14114ae0 con 0x7fad1406aa90 2024-06-10T02:52:36.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:36.513+0000 7fad1ad12700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fad1410ef40 0x7fad1419f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:52:36.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:36.513+0000 7fad1a511700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fad1406a170 0x7fad1419aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:52:36.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:36.513+0000 7fad1ad12700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fad1410ef40 0x7fad1419f730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:35372/0 (socket says 172.21.15.52:35372) 2024-06-10T02:52:36.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:36.513+0000 7fad1ad12700 1 -- 172.21.15.52:0/3207760105 learned_addr learned my addr 172.21.15.52:0/3207760105 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:52:36.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:36.513+0000 7fad19d10700 1 --2- 172.21.15.52:0/3207760105 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fad1406aa90 0x7fad1419b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:52:36.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:36.514+0000 7fad1a511700 1 -- 172.21.15.52:0/3207760105 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fad1406aa90 msgr2=0x7fad1419b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:52:36.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:36.514+0000 7fad1a511700 1 --2- 172.21.15.52:0/3207760105 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fad1406aa90 0x7fad1419b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:36.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:36.514+0000 7fad1a511700 1 -- 172.21.15.52:0/3207760105 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fad1410ef40 msgr2=0x7fad1419f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:52:36.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:36.514+0000 7fad1a511700 1 --2- 172.21.15.52:0/3207760105 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fad1410ef40 0x7fad1419f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:36.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:36.514+0000 7fad1a511700 1 -- 172.21.15.52:0/3207760105 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad1419fe70 con 0x7fad1406a170 2024-06-10T02:52:36.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:36.514+0000 7fad1ad12700 1 --2- 172.21.15.52:0/3207760105 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fad1410ef40 0x7fad1419f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T02:52:36.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:36.514+0000 7fad1a511700 1 --2- 172.21.15.52:0/3207760105 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fad1406a170 0x7fad1419aeb0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fad1000d950 tx=0x7fad1000dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:52:36.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:36.514+0000 7fad0b7fe700 1 -- 172.21.15.52:0/3207760105 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad10016070 con 0x7fad1406a170 2024-06-10T02:52:36.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:36.514+0000 7fad1c775700 1 -- 172.21.15.52:0/3207760105 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad141a0160 con 0x7fad1406a170 2024-06-10T02:52:36.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:36.514+0000 7fad0b7fe700 1 -- 172.21.15.52:0/3207760105 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad1000fd70 con 0x7fad1406a170 2024-06-10T02:52:36.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:36.514+0000 7fad1c775700 1 -- 172.21.15.52:0/3207760105 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad141a79e0 con 0x7fad1406a170 2024-06-10T02:52:36.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:36.514+0000 7fad0b7fe700 1 -- 172.21.15.52:0/3207760105 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad1000e3b0 con 0x7fad1406a170 2024-06-10T02:52:36.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:36.515+0000 7fad1c775700 1 -- 172.21.15.52:0/3207760105 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad14069180 con 0x7fad1406a170 2024-06-10T02:52:36.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:36.517+0000 7fad0b7fe700 1 -- 172.21.15.52:0/3207760105 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fad10019070 con 0x7fad1406a170 2024-06-10T02:52:36.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:36.517+0000 7fad0b7fe700 1 --2- 172.21.15.52:0/3207760105 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fad000a2cb0 0x7fad000a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:52:36.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:36.517+0000 7fad19d10700 1 --2- 172.21.15.52:0/3207760105 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fad000a2cb0 0x7fad000a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:52:36.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:36.517+0000 7fad0b7fe700 1 -- 172.21.15.52:0/3207760105 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7fad100cf490 con 0x7fad1406a170 2024-06-10T02:52:36.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:36.518+0000 7fad19d10700 1 --2- 172.21.15.52:0/3207760105 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fad000a2cb0 0x7fad000a5170 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fad0400b5c0 tx=0x7fad04005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:52:36.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:36.521+0000 7fad0b7fe700 1 -- 172.21.15.52:0/3207760105 <== mon.2 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fad10010460 con 0x7fad1406a170 2024-06-10T02:52:36.745 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:36.744+0000 7fad1c775700 1 -- 172.21.15.52:0/3207760105 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7fad1419c1c0 con 0x7fad1406a170 2024-06-10T02:52:36.746 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:36.744+0000 7fad0b7fe700 1 -- 172.21.15.52:0/3207760105 <== mon.2 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fad1009bee0 con 0x7fad1406a170 2024-06-10T02:52:36.746 INFO:teuthology.orchestra.run.smithi052.stdout:111669149783 2024-06-10T02:52:36.747 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:36.746+0000 7fad1c775700 1 -- 172.21.15.52:0/3207760105 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fad000a2cb0 msgr2=0x7fad000a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:52:36.747 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:36.746+0000 7fad1c775700 1 --2- 172.21.15.52:0/3207760105 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fad000a2cb0 0x7fad000a5170 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fad0400b5c0 tx=0x7fad04005960 comp rx=0 tx=0).stop 2024-06-10T02:52:36.747 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:36.746+0000 7fad1c775700 1 -- 172.21.15.52:0/3207760105 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fad1406a170 msgr2=0x7fad1419aeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:52:36.747 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:36.746+0000 7fad1c775700 1 --2- 172.21.15.52:0/3207760105 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fad1406a170 0x7fad1419aeb0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fad1000d950 tx=0x7fad1000dc60 comp rx=0 tx=0).stop 2024-06-10T02:52:36.747 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:36.746+0000 7fad1c775700 1 -- 172.21.15.52:0/3207760105 shutdown_connections 2024-06-10T02:52:36.747 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:36.746+0000 7fad1c775700 1 --2- 172.21.15.52:0/3207760105 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fad000a2cb0 0x7fad000a5170 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:36.747 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:36.746+0000 7fad1c775700 1 --2- 172.21.15.52:0/3207760105 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fad1410ef40 0x7fad1419f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:36.747 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:36.746+0000 7fad1c775700 1 --2- 172.21.15.52:0/3207760105 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fad1406a170 0x7fad1419aeb0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:36.747 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:36.746+0000 7fad1c775700 1 --2- 172.21.15.52:0/3207760105 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fad1406aa90 0x7fad1419b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:36.747 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:36.747+0000 7fad1c775700 1 -- 172.21.15.52:0/3207760105 >> 172.21.15.52:0/3207760105 conn(0x7fad140782c0 msgr2=0x7fad14110430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:52:36.747 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:36.747+0000 7fad1c775700 1 -- 172.21.15.52:0/3207760105 shutdown_connections 2024-06-10T02:52:36.747 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:36.747+0000 7fad1c775700 1 -- 172.21.15.52:0/3207760105 wait complete. 2024-06-10T02:52:36.976 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:52:36 smithi052 ceph-mon[32362]: pgmap v386: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-10T02:52:36.976 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:52:36 smithi052 ceph-mon[32362]: from='client.? 172.21.15.52:0/3207760105' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-06-10T02:52:37.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:52:36 smithi120 ceph-mon[36634]: pgmap v386: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-10T02:52:37.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:52:36 smithi120 ceph-mon[36634]: from='client.? 172.21.15.52:0/3207760105' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-06-10T02:52:37.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:52:36 smithi188 ceph-mon[36599]: pgmap v386: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-10T02:52:37.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:52:36 smithi188 ceph-mon[36599]: from='client.? 172.21.15.52:0/3207760105' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-06-10T02:52:37.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:37.598+0000 7fc4c8453700 1 -- 172.21.15.52:0/911251177 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc4c0108500 msgr2=0x7fc4c01088e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:52:37.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:37.598+0000 7fc4c8453700 1 --2- 172.21.15.52:0/911251177 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc4c0108500 0x7fc4c01088e0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fc4bc009990 tx=0x7fc4bc009ca0 comp rx=0 tx=0).stop 2024-06-10T02:52:37.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:37.598+0000 7fc4c8453700 1 -- 172.21.15.52:0/911251177 shutdown_connections 2024-06-10T02:52:37.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:37.598+0000 7fc4c8453700 1 --2- 172.21.15.52:0/911251177 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc4c006ab30 0x7fc4c010f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:37.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:37.598+0000 7fc4c8453700 1 --2- 172.21.15.52:0/911251177 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc4c0108500 0x7fc4c01088e0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:37.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:37.598+0000 7fc4c8453700 1 --2- 172.21.15.52:0/911251177 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc4c006a170 0x7fc4c006a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:37.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:37.598+0000 7fc4c8453700 1 -- 172.21.15.52:0/911251177 >> 172.21.15.52:0/911251177 conn(0x7fc4c0078340 msgr2=0x7fc4c0078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:52:37.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:37.598+0000 7fc4c8453700 1 -- 172.21.15.52:0/911251177 shutdown_connections 2024-06-10T02:52:37.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:37.598+0000 7fc4c8453700 1 -- 172.21.15.52:0/911251177 wait complete. 2024-06-10T02:52:37.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:37.599+0000 7fc4c8453700 1 Processor -- start 2024-06-10T02:52:37.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:37.599+0000 7fc4c8453700 1 -- start start 2024-06-10T02:52:37.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:37.599+0000 7fc4c8453700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc4c006a170 0x7fc4c019e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:52:37.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:37.599+0000 7fc4c8453700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc4c006ab30 0x7fc4c019ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:52:37.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:37.599+0000 7fc4c8453700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc4c0108500 0x7fc4c0198ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:52:37.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:37.599+0000 7fc4c8453700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fc4c0112780 con 0x7fc4c006a170 2024-06-10T02:52:37.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:37.599+0000 7fc4c8453700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc4c0112600 con 0x7fc4c006ab30 2024-06-10T02:52:37.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:37.599+0000 7fc4c8453700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc4c0112900 con 0x7fc4c0108500 2024-06-10T02:52:37.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:37.600+0000 7fc4c61ef700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc4c006a170 0x7fc4c019e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:52:37.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:37.600+0000 7fc4c61ef700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc4c006a170 0x7fc4c019e8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:35394/0 (socket says 172.21.15.52:35394) 2024-06-10T02:52:37.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:37.600+0000 7fc4c61ef700 1 -- 172.21.15.52:0/3000550810 learned_addr learned my addr 172.21.15.52:0/3000550810 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:52:37.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:37.600+0000 7fc4c69f0700 1 --2- 172.21.15.52:0/3000550810 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc4c0108500 0x7fc4c0198ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:52:37.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:37.600+0000 7fc4c59ee700 1 --2- 172.21.15.52:0/3000550810 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc4c006ab30 0x7fc4c019ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:52:37.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:37.600+0000 7fc4c61ef700 1 -- 172.21.15.52:0/3000550810 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc4c0108500 msgr2=0x7fc4c0198ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:52:37.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:37.600+0000 7fc4c61ef700 1 --2- 172.21.15.52:0/3000550810 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc4c0108500 0x7fc4c0198ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:37.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:37.600+0000 7fc4c61ef700 1 -- 172.21.15.52:0/3000550810 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc4c006ab30 msgr2=0x7fc4c019ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:52:37.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:37.600+0000 7fc4c61ef700 1 --2- 172.21.15.52:0/3000550810 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc4c006ab30 0x7fc4c019ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:37.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:37.600+0000 7fc4c61ef700 1 -- 172.21.15.52:0/3000550810 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4c0199240 con 0x7fc4c006a170 2024-06-10T02:52:37.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:37.600+0000 7fc4c69f0700 1 --2- 172.21.15.52:0/3000550810 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc4c0108500 0x7fc4c0198ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T02:52:37.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:37.600+0000 7fc4c59ee700 1 --2- 172.21.15.52:0/3000550810 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc4c006ab30 0x7fc4c019ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T02:52:37.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:37.600+0000 7fc4c61ef700 1 --2- 172.21.15.52:0/3000550810 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc4c006a170 0x7fc4c019e8d0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fc4bc000c00 tx=0x7fc4bc00bb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:52:37.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:37.600+0000 7fc4b77fe700 1 -- 172.21.15.52:0/3000550810 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4bc027070 con 0x7fc4c006a170 2024-06-10T02:52:37.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:37.600+0000 7fc4c8453700 1 -- 172.21.15.52:0/3000550810 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4c01994d0 con 0x7fc4c006a170 2024-06-10T02:52:37.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:37.600+0000 7fc4b77fe700 1 -- 172.21.15.52:0/3000550810 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4bc022470 con 0x7fc4c006a170 2024-06-10T02:52:37.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:37.600+0000 7fc4b77fe700 1 -- 172.21.15.52:0/3000550810 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4bc00f6a0 con 0x7fc4c006a170 2024-06-10T02:52:37.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:37.600+0000 7fc4c8453700 1 -- 172.21.15.52:0/3000550810 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4c01a57a0 con 0x7fc4c006a170 2024-06-10T02:52:37.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:37.601+0000 7fc4c8453700 1 -- 172.21.15.52:0/3000550810 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4c010a9e0 con 0x7fc4c006a170 2024-06-10T02:52:37.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:37.603+0000 7fc4b77fe700 1 -- 172.21.15.52:0/3000550810 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fc4bc02a070 con 0x7fc4c006a170 2024-06-10T02:52:37.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:37.603+0000 7fc4b77fe700 1 --2- 172.21.15.52:0/3000550810 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fc4ac0a2cb0 0x7fc4ac0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:52:37.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:37.603+0000 7fc4c59ee700 1 --2- 172.21.15.52:0/3000550810 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fc4ac0a2cb0 0x7fc4ac0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:52:37.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:37.603+0000 7fc4b77fe700 1 -- 172.21.15.52:0/3000550810 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7fc4bc0d03c0 con 0x7fc4c006a170 2024-06-10T02:52:37.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:37.604+0000 7fc4c59ee700 1 --2- 172.21.15.52:0/3000550810 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fc4ac0a2cb0 0x7fc4ac0a5170 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fc4c00789b0 tx=0x7fc4b0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:52:37.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:37.608+0000 7fc4b77fe700 1 -- 172.21.15.52:0/3000550810 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fc4bc013070 con 0x7fc4c006a170 2024-06-10T02:52:37.846 INFO:teuthology.orchestra.run.smithi052.stdout:146028888147 2024-06-10T02:52:37.847 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:37.846+0000 7fc4c8453700 1 -- 172.21.15.52:0/3000550810 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7fc4c0199fc0 con 0x7fc4c006a170 2024-06-10T02:52:37.847 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:37.847+0000 7fc4b77fe700 1 -- 172.21.15.52:0/3000550810 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fc4bc09cea0 con 0x7fc4c006a170 2024-06-10T02:52:37.848 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:37.849+0000 7fc4c8453700 1 -- 172.21.15.52:0/3000550810 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fc4ac0a2cb0 msgr2=0x7fc4ac0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:52:37.849 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:37.849+0000 7fc4c8453700 1 --2- 172.21.15.52:0/3000550810 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fc4ac0a2cb0 0x7fc4ac0a5170 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fc4c00789b0 tx=0x7fc4b0009260 comp rx=0 tx=0).stop 2024-06-10T02:52:37.849 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:37.849+0000 7fc4c8453700 1 -- 172.21.15.52:0/3000550810 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc4c006a170 msgr2=0x7fc4c019e8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:52:37.849 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:37.849+0000 7fc4c8453700 1 --2- 172.21.15.52:0/3000550810 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc4c006a170 0x7fc4c019e8d0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fc4bc000c00 tx=0x7fc4bc00bb60 comp rx=0 tx=0).stop 2024-06-10T02:52:37.849 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:37.849+0000 7fc4c8453700 1 -- 172.21.15.52:0/3000550810 shutdown_connections 2024-06-10T02:52:37.849 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:37.849+0000 7fc4c8453700 1 --2- 172.21.15.52:0/3000550810 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc4c0108500 0x7fc4c0198ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:37.849 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:37.849+0000 7fc4c8453700 1 --2- 172.21.15.52:0/3000550810 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fc4ac0a2cb0 0x7fc4ac0a5170 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:37.849 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:37.849+0000 7fc4c8453700 1 --2- 172.21.15.52:0/3000550810 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc4c006a170 0x7fc4c019e8d0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:37.849 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:37.849+0000 7fc4c8453700 1 --2- 172.21.15.52:0/3000550810 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc4c006ab30 0x7fc4c019ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:37.849 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:37.849+0000 7fc4c8453700 1 -- 172.21.15.52:0/3000550810 >> 172.21.15.52:0/3000550810 conn(0x7fc4c0078340 msgr2=0x7fc4c0100020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:52:37.849 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:37.849+0000 7fc4c8453700 1 -- 172.21.15.52:0/3000550810 shutdown_connections 2024-06-10T02:52:37.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:37.849+0000 7fc4c8453700 1 -- 172.21.15.52:0/3000550810 wait complete. 2024-06-10T02:52:38.974 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:52:38 smithi052 ceph-mon[32362]: pgmap v387: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-10T02:52:38.975 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:52:38 smithi052 ceph-mon[32362]: from='client.? 172.21.15.52:0/3000550810' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-06-10T02:52:39.004 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149778 got 111669149783 for osd.3 2024-06-10T02:52:39.004 DEBUG:teuthology.parallel:result is None 2024-06-10T02:52:39.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:52:38 smithi120 ceph-mon[36634]: pgmap v387: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-10T02:52:39.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:52:38 smithi120 ceph-mon[36634]: from='client.? 172.21.15.52:0/3000550810' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-06-10T02:52:39.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:39.186+0000 7fbe9eb76700 1 -- 172.21.15.52:0/179695807 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbe98102770 msgr2=0x7fbe98102b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:52:39.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:39.186+0000 7fbe9eb76700 1 --2- 172.21.15.52:0/179695807 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbe98102770 0x7fbe98102b50 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fbe8c0099c0 tx=0x7fbe8c009cd0 comp rx=0 tx=0).stop 2024-06-10T02:52:39.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:39.186+0000 7fbe9eb76700 1 -- 172.21.15.52:0/179695807 shutdown_connections 2024-06-10T02:52:39.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:39.186+0000 7fbe9eb76700 1 --2- 172.21.15.52:0/179695807 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbe98107860 0x7fbe9810f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:39.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:39.186+0000 7fbe9eb76700 1 --2- 172.21.15.52:0/179695807 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbe98102770 0x7fbe98102b50 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:39.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:39.186+0000 7fbe9eb76700 1 --2- 172.21.15.52:0/179695807 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbe98103120 0x7fbe98107130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:39.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:39.186+0000 7fbe9eb76700 1 -- 172.21.15.52:0/179695807 >> 172.21.15.52:0/179695807 conn(0x7fbe98079600 msgr2=0x7fbe98079a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:52:39.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:39.186+0000 7fbe9eb76700 1 -- 172.21.15.52:0/179695807 shutdown_connections 2024-06-10T02:52:39.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:39.186+0000 7fbe9eb76700 1 -- 172.21.15.52:0/179695807 wait complete. 2024-06-10T02:52:39.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:39.186+0000 7fbe9eb76700 1 Processor -- start 2024-06-10T02:52:39.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:39.187+0000 7fbe9eb76700 1 -- start start 2024-06-10T02:52:39.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:39.187+0000 7fbe9eb76700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbe98102770 0x7fbe9819e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:52:39.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:39.187+0000 7fbe9eb76700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbe98103120 0x7fbe9819ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:52:39.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:39.187+0000 7fbe9eb76700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbe98107860 0x7fbe98198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:52:39.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:39.187+0000 7fbe9eb76700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fbe98112820 con 0x7fbe98103120 2024-06-10T02:52:39.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:39.187+0000 7fbe9eb76700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fbe981126a0 con 0x7fbe98107860 2024-06-10T02:52:39.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:39.187+0000 7fbe9eb76700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fbe981129a0 con 0x7fbe98102770 2024-06-10T02:52:39.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:39.187+0000 7fbe97fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbe98103120 0x7fbe9819ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:52:39.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:39.187+0000 7fbe97fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbe98103120 0x7fbe9819ee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:35410/0 (socket says 172.21.15.52:35410) 2024-06-10T02:52:39.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:39.187+0000 7fbe97fff700 1 -- 172.21.15.52:0/735014873 learned_addr learned my addr 172.21.15.52:0/735014873 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:52:39.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:39.187+0000 7fbe9c912700 1 --2- 172.21.15.52:0/735014873 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbe98102770 0x7fbe9819e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:52:39.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:39.187+0000 7fbe9d113700 1 --2- 172.21.15.52:0/735014873 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbe98107860 0x7fbe98198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:52:39.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:39.187+0000 7fbe97fff700 1 -- 172.21.15.52:0/735014873 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbe98102770 msgr2=0x7fbe9819e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:52:39.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:39.187+0000 7fbe97fff700 1 --2- 172.21.15.52:0/735014873 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbe98102770 0x7fbe9819e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:39.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:39.187+0000 7fbe97fff700 1 -- 172.21.15.52:0/735014873 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbe98107860 msgr2=0x7fbe98198b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:52:39.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:39.187+0000 7fbe97fff700 1 --2- 172.21.15.52:0/735014873 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbe98107860 0x7fbe98198b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:39.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:39.187+0000 7fbe97fff700 1 -- 172.21.15.52:0/735014873 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe98199260 con 0x7fbe98103120 2024-06-10T02:52:39.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:39.187+0000 7fbe9c912700 1 --2- 172.21.15.52:0/735014873 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbe98102770 0x7fbe9819e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T02:52:39.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:39.188+0000 7fbe97fff700 1 --2- 172.21.15.52:0/735014873 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbe98103120 0x7fbe9819ee30 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fbe8400d980 tx=0x7fbe8400dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:52:39.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:39.188+0000 7fbe95ffb700 1 -- 172.21.15.52:0/735014873 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe84017070 con 0x7fbe98103120 2024-06-10T02:52:39.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:39.188+0000 7fbe9eb76700 1 -- 172.21.15.52:0/735014873 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe98199550 con 0x7fbe98103120 2024-06-10T02:52:39.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:39.188+0000 7fbe95ffb700 1 -- 172.21.15.52:0/735014873 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe84004d10 con 0x7fbe98103120 2024-06-10T02:52:39.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:39.188+0000 7fbe95ffb700 1 -- 172.21.15.52:0/735014873 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe840055c0 con 0x7fbe98103120 2024-06-10T02:52:39.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:39.188+0000 7fbe9eb76700 1 -- 172.21.15.52:0/735014873 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe981a57c0 con 0x7fbe98103120 2024-06-10T02:52:39.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:39.189+0000 7fbe9eb76700 1 -- 172.21.15.52:0/735014873 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe9810aaa0 con 0x7fbe98103120 2024-06-10T02:52:39.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:39.190+0000 7fbe95ffb700 1 -- 172.21.15.52:0/735014873 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fbe84004750 con 0x7fbe98103120 2024-06-10T02:52:39.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:39.191+0000 7fbe95ffb700 1 --2- 172.21.15.52:0/735014873 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fbe880a2cb0 0x7fbe880a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:52:39.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:39.191+0000 7fbe9c912700 1 --2- 172.21.15.52:0/735014873 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fbe880a2cb0 0x7fbe880a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:52:39.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:39.191+0000 7fbe95ffb700 1 -- 172.21.15.52:0/735014873 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7fbe840cf540 con 0x7fbe98103120 2024-06-10T02:52:39.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:39.191+0000 7fbe9c912700 1 --2- 172.21.15.52:0/735014873 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fbe880a2cb0 0x7fbe880a5170 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fbe8c000c00 tx=0x7fbe8c005940 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:52:39.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:39.196+0000 7fbe95ffb700 1 -- 172.21.15.52:0/735014873 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fbe84020020 con 0x7fbe98103120 2024-06-10T02:52:39.205 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:52:38 smithi188 ceph-mon[36599]: pgmap v387: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-10T02:52:39.205 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:52:38 smithi188 ceph-mon[36599]: from='client.? 172.21.15.52:0/3000550810' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-06-10T02:52:39.408 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:39.408+0000 7fbe9eb76700 1 -- 172.21.15.52:0/735014873 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7fbe9810a480 con 0x7fbe98103120 2024-06-10T02:52:39.408 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:39.409+0000 7fbe95ffb700 1 -- 172.21.15.52:0/735014873 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7fbe8409bf30 con 0x7fbe98103120 2024-06-10T02:52:39.408 INFO:teuthology.orchestra.run.smithi052.stdout:313532612661 2024-06-10T02:52:39.410 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:39.411+0000 7fbe9eb76700 1 -- 172.21.15.52:0/735014873 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fbe880a2cb0 msgr2=0x7fbe880a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:52:39.410 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:39.411+0000 7fbe9eb76700 1 --2- 172.21.15.52:0/735014873 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fbe880a2cb0 0x7fbe880a5170 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fbe8c000c00 tx=0x7fbe8c005940 comp rx=0 tx=0).stop 2024-06-10T02:52:39.410 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:39.411+0000 7fbe9eb76700 1 -- 172.21.15.52:0/735014873 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbe98103120 msgr2=0x7fbe9819ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:52:39.410 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:39.411+0000 7fbe9eb76700 1 --2- 172.21.15.52:0/735014873 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbe98103120 0x7fbe9819ee30 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fbe8400d980 tx=0x7fbe8400dd40 comp rx=0 tx=0).stop 2024-06-10T02:52:39.410 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:39.411+0000 7fbe9eb76700 1 -- 172.21.15.52:0/735014873 shutdown_connections 2024-06-10T02:52:39.410 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:39.411+0000 7fbe9eb76700 1 --2- 172.21.15.52:0/735014873 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbe98107860 0x7fbe98198b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:39.410 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:39.411+0000 7fbe9eb76700 1 --2- 172.21.15.52:0/735014873 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbe98102770 0x7fbe9819e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:39.410 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:39.411+0000 7fbe9eb76700 1 --2- 172.21.15.52:0/735014873 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fbe880a2cb0 0x7fbe880a5170 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:39.411 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:39.411+0000 7fbe9eb76700 1 --2- 172.21.15.52:0/735014873 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbe98103120 0x7fbe9819ee30 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:39.411 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:39.411+0000 7fbe9eb76700 1 -- 172.21.15.52:0/735014873 >> 172.21.15.52:0/735014873 conn(0x7fbe98079600 msgr2=0x7fbe981004c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:52:39.411 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:39.411+0000 7fbe9eb76700 1 -- 172.21.15.52:0/735014873 shutdown_connections 2024-06-10T02:52:39.411 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:39.411+0000 7fbe9eb76700 1 -- 172.21.15.52:0/735014873 wait complete. 2024-06-10T02:52:39.909 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:52:39 smithi052 ceph-mon[32362]: from='client.? 172.21.15.52:0/735014873' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-06-10T02:52:40.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:52:39 smithi120 ceph-mon[36634]: from='client.? 172.21.15.52:0/735014873' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-06-10T02:52:40.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:52:39 smithi188 ceph-mon[36599]: from='client.? 172.21.15.52:0/735014873' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-06-10T02:52:40.323 INFO:tasks.cephadm.ceph_manager.ceph:need seq 257698037817 got 257698037821 for osd.8 2024-06-10T02:52:40.323 DEBUG:teuthology.parallel:result is None 2024-06-10T02:52:41.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:52:40 smithi120 ceph-mon[36634]: pgmap v388: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-10T02:52:41.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:52:40 smithi188 ceph-mon[36599]: pgmap v388: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-10T02:52:41.217 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:52:40 smithi052 ceph-mon[32362]: pgmap v388: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-10T02:52:41.250 INFO:tasks.cephadm.ceph_manager.ceph:need seq 146028888142 got 146028888147 for osd.4 2024-06-10T02:52:41.250 DEBUG:teuthology.parallel:result is None 2024-06-10T02:52:41.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:41.965+0000 7ffa2bf6a700 1 -- 172.21.15.52:0/2571619335 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffa2406aa00 msgr2=0x7ffa2406ae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:52:41.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:41.965+0000 7ffa2bf6a700 1 --2- 172.21.15.52:0/2571619335 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffa2406aa00 0x7ffa2406ae80 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7ffa18009990 tx=0x7ffa18009ca0 comp rx=0 tx=0).stop 2024-06-10T02:52:41.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:41.965+0000 7ffa2bf6a700 1 -- 172.21.15.52:0/2571619335 shutdown_connections 2024-06-10T02:52:41.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:41.966+0000 7ffa2bf6a700 1 --2- 172.21.15.52:0/2571619335 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffa2406b3c0 0x7ffa2410f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:41.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:41.966+0000 7ffa2bf6a700 1 --2- 172.21.15.52:0/2571619335 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffa241084f0 0x7ffa241088d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:41.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:41.966+0000 7ffa2bf6a700 1 --2- 172.21.15.52:0/2571619335 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffa2406aa00 0x7ffa2406ae80 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:41.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:41.966+0000 7ffa2bf6a700 1 -- 172.21.15.52:0/2571619335 >> 172.21.15.52:0/2571619335 conn(0x7ffa24078300 msgr2=0x7ffa24078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:52:41.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:41.966+0000 7ffa2bf6a700 1 -- 172.21.15.52:0/2571619335 shutdown_connections 2024-06-10T02:52:41.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:41.966+0000 7ffa2bf6a700 1 -- 172.21.15.52:0/2571619335 wait complete. 2024-06-10T02:52:41.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:41.966+0000 7ffa2bf6a700 1 Processor -- start 2024-06-10T02:52:41.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:41.966+0000 7ffa2bf6a700 1 -- start start 2024-06-10T02:52:41.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:41.967+0000 7ffa2bf6a700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffa2406aa00 0x7ffa24101450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:52:41.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:41.967+0000 7ffa2bf6a700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffa2406b3c0 0x7ffa24101990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:52:41.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:41.967+0000 7ffa2bf6a700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffa241084f0 0x7ffa24101ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:52:41.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:41.967+0000 7ffa2bf6a700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ffa24112770 con 0x7ffa241084f0 2024-06-10T02:52:41.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:41.967+0000 7ffa2bf6a700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ffa241125f0 con 0x7ffa2406b3c0 2024-06-10T02:52:41.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:41.967+0000 7ffa2bf6a700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ffa241128f0 con 0x7ffa2406aa00 2024-06-10T02:52:41.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:41.967+0000 7ffa2a507700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffa241084f0 0x7ffa24101ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:52:41.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:41.967+0000 7ffa2a507700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffa241084f0 0x7ffa24101ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:39860/0 (socket says 172.21.15.52:39860) 2024-06-10T02:52:41.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:41.967+0000 7ffa29505700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffa2406b3c0 0x7ffa24101990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:52:41.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:41.967+0000 7ffa2a507700 1 -- 172.21.15.52:0/1973185556 learned_addr learned my addr 172.21.15.52:0/1973185556 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:52:41.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:41.967+0000 7ffa29d06700 1 --2- 172.21.15.52:0/1973185556 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffa2406aa00 0x7ffa24101450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:52:41.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:41.967+0000 7ffa2a507700 1 -- 172.21.15.52:0/1973185556 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffa2406aa00 msgr2=0x7ffa24101450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:52:41.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:41.967+0000 7ffa2a507700 1 --2- 172.21.15.52:0/1973185556 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffa2406aa00 0x7ffa24101450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:41.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:41.967+0000 7ffa2a507700 1 -- 172.21.15.52:0/1973185556 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffa2406b3c0 msgr2=0x7ffa24101990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:52:41.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:41.967+0000 7ffa2a507700 1 --2- 172.21.15.52:0/1973185556 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffa2406b3c0 0x7ffa24101990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:41.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:41.967+0000 7ffa2a507700 1 -- 172.21.15.52:0/1973185556 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa241a70c0 con 0x7ffa241084f0 2024-06-10T02:52:41.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:41.967+0000 7ffa2a507700 1 --2- 172.21.15.52:0/1973185556 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffa241084f0 0x7ffa24101ed0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7ffa1c00c900 tx=0x7ffa1c00cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:52:41.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:41.968+0000 7ffa22ffd700 1 -- 172.21.15.52:0/1973185556 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa1c007b00 con 0x7ffa241084f0 2024-06-10T02:52:41.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:41.968+0000 7ffa2bf6a700 1 -- 172.21.15.52:0/1973185556 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa241a73b0 con 0x7ffa241084f0 2024-06-10T02:52:41.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:41.968+0000 7ffa2bf6a700 1 -- 172.21.15.52:0/1973185556 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa241a78c0 con 0x7ffa241084f0 2024-06-10T02:52:41.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:41.968+0000 7ffa22ffd700 1 -- 172.21.15.52:0/1973185556 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa1c00ce50 con 0x7ffa241084f0 2024-06-10T02:52:41.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:41.968+0000 7ffa22ffd700 1 -- 172.21.15.52:0/1973185556 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa1c018660 con 0x7ffa241084f0 2024-06-10T02:52:41.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:41.969+0000 7ffa2bf6a700 1 -- 172.21.15.52:0/1973185556 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa10005330 con 0x7ffa241084f0 2024-06-10T02:52:41.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:41.970+0000 7ffa22ffd700 1 -- 172.21.15.52:0/1973185556 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7ffa1c007ca0 con 0x7ffa241084f0 2024-06-10T02:52:41.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:41.971+0000 7ffa22ffd700 1 --2- 172.21.15.52:0/1973185556 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7ffa0c0a2d30 0x7ffa0c0a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:52:41.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:41.971+0000 7ffa29d06700 1 --2- 172.21.15.52:0/1973185556 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7ffa0c0a2d30 0x7ffa0c0a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:52:41.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:41.971+0000 7ffa22ffd700 1 -- 172.21.15.52:0/1973185556 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7ffa1c0cf2c0 con 0x7ffa241084f0 2024-06-10T02:52:41.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:41.971+0000 7ffa29d06700 1 --2- 172.21.15.52:0/1973185556 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7ffa0c0a2d30 0x7ffa0c0a51f0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7ffa140076e0 tx=0x7ffa14007670 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:52:41.973 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:41.974+0000 7ffa22ffd700 1 -- 172.21.15.52:0/1973185556 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7ffa1c014030 con 0x7ffa241084f0 2024-06-10T02:52:42.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:42.175+0000 7efdc5efc700 1 -- 172.21.15.52:0/173774657 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efdc01085f0 msgr2=0x7efdc01089d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:52:42.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:42.175+0000 7efdc5efc700 1 --2- 172.21.15.52:0/173774657 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efdc01085f0 0x7efdc01089d0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7efdb0009990 tx=0x7efdb0009ca0 comp rx=0 tx=0).stop 2024-06-10T02:52:42.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:42.175+0000 7efdc5efc700 1 -- 172.21.15.52:0/173774657 shutdown_connections 2024-06-10T02:52:42.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:42.175+0000 7efdc5efc700 1 --2- 172.21.15.52:0/173774657 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efdc0102ea0 0x7efdc010f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:42.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:42.175+0000 7efdc5efc700 1 --2- 172.21.15.52:0/173774657 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efdc01085f0 0x7efdc01089d0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:42.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:42.175+0000 7efdc5efc700 1 --2- 172.21.15.52:0/173774657 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efdc01024e0 0x7efdc0102960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:42.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:42.175+0000 7efdc5efc700 1 -- 172.21.15.52:0/173774657 >> 172.21.15.52:0/173774657 conn(0x7efdc0079640 msgr2=0x7efdc0079a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:52:42.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:42.175+0000 7efdc5efc700 1 -- 172.21.15.52:0/173774657 shutdown_connections 2024-06-10T02:52:42.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:42.175+0000 7efdc5efc700 1 -- 172.21.15.52:0/173774657 wait complete. 2024-06-10T02:52:42.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:42.176+0000 7efdc5efc700 1 Processor -- start 2024-06-10T02:52:42.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:42.176+0000 7efdc5efc700 1 -- start start 2024-06-10T02:52:42.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:42.176+0000 7efdc5efc700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efdc01024e0 0x7efdc0074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:52:42.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:42.176+0000 7efdc5efc700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efdc0102ea0 0x7efdc0074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:52:42.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:42.176+0000 7efdc5efc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efdc01085f0 0x7efdc0075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:52:42.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:42.176+0000 7efdc5efc700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7efdc01127e0 con 0x7efdc01085f0 2024-06-10T02:52:42.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:42.176+0000 7efdc5efc700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7efdc0112660 con 0x7efdc01024e0 2024-06-10T02:52:42.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:42.176+0000 7efdc5efc700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7efdc0112960 con 0x7efdc0102ea0 2024-06-10T02:52:42.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:42.176+0000 7efdbffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efdc01085f0 0x7efdc0075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:52:42.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:42.176+0000 7efdbffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efdc01085f0 0x7efdc0075210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:39884/0 (socket says 172.21.15.52:39884) 2024-06-10T02:52:42.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:42.176+0000 7efdbffff700 1 -- 172.21.15.52:0/291549188 learned_addr learned my addr 172.21.15.52:0/291549188 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:52:42.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:42.177+0000 7efdbeffd700 1 --2- 172.21.15.52:0/291549188 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efdc0102ea0 0x7efdc0074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:52:42.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:42.177+0000 7efdbf7fe700 1 --2- 172.21.15.52:0/291549188 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efdc01024e0 0x7efdc0074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:52:42.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:42.177+0000 7efdbffff700 1 -- 172.21.15.52:0/291549188 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efdc0102ea0 msgr2=0x7efdc0074cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:52:42.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:42.177+0000 7efdbffff700 1 --2- 172.21.15.52:0/291549188 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efdc0102ea0 0x7efdc0074cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:42.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:42.177+0000 7efdbffff700 1 -- 172.21.15.52:0/291549188 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efdc01024e0 msgr2=0x7efdc0074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:52:42.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:42.177+0000 7efdbffff700 1 --2- 172.21.15.52:0/291549188 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efdc01024e0 0x7efdc0074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:42.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:42.177+0000 7efdbffff700 1 -- 172.21.15.52:0/291549188 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdc01a7140 con 0x7efdc01085f0 2024-06-10T02:52:42.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:42.177+0000 7efdbeffd700 1 --2- 172.21.15.52:0/291549188 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efdc0102ea0 0x7efdc0074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T02:52:42.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:42.177+0000 7efdbffff700 1 --2- 172.21.15.52:0/291549188 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efdc01085f0 0x7efdc0075210 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7efdb8007f10 tx=0x7efdb800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:52:42.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:42.177+0000 7efdbcff9700 1 -- 172.21.15.52:0/291549188 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdb800ddd0 con 0x7efdc01085f0 2024-06-10T02:52:42.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:42.177+0000 7efdc5efc700 1 -- 172.21.15.52:0/291549188 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efdc01a7430 con 0x7efdc01085f0 2024-06-10T02:52:42.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:42.177+0000 7efdbcff9700 1 -- 172.21.15.52:0/291549188 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efdb8004020 con 0x7efdc01085f0 2024-06-10T02:52:42.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:42.178+0000 7efdc5efc700 1 -- 172.21.15.52:0/291549188 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efdc01a7970 con 0x7efdc01085f0 2024-06-10T02:52:42.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:42.178+0000 7efdbcff9700 1 -- 172.21.15.52:0/291549188 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdb8015660 con 0x7efdc01085f0 2024-06-10T02:52:42.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:42.179+0000 7efdc5efc700 1 -- 172.21.15.52:0/291549188 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efdc0103c00 con 0x7efdc01085f0 2024-06-10T02:52:42.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:42.180+0000 7efdbcff9700 1 -- 172.21.15.52:0/291549188 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7efdb80041d0 con 0x7efdc01085f0 2024-06-10T02:52:42.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:42.181+0000 7efdbcff9700 1 --2- 172.21.15.52:0/291549188 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7efda80a2d30 0x7efda80a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:52:42.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:42.181+0000 7efdbf7fe700 1 --2- 172.21.15.52:0/291549188 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7efda80a2d30 0x7efda80a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:52:42.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:42.181+0000 7efdbcff9700 1 -- 172.21.15.52:0/291549188 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7efdb80cee60 con 0x7efdc01085f0 2024-06-10T02:52:42.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:42.182+0000 7efdbf7fe700 1 --2- 172.21.15.52:0/291549188 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7efda80a2d30 0x7efda80a51f0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7efdb000b5c0 tx=0x7efdb0005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:52:42.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:42.185+0000 7efdbcff9700 1 -- 172.21.15.52:0/291549188 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7efdb8010070 con 0x7efdc01085f0 2024-06-10T02:52:42.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:42.190+0000 7ffa2bf6a700 1 -- 172.21.15.52:0/1973185556 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7ffa100058f0 con 0x7ffa241084f0 2024-06-10T02:52:42.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:42.191+0000 7ffa22ffd700 1 -- 172.21.15.52:0/1973185556 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7ffa1c09be20 con 0x7ffa241084f0 2024-06-10T02:52:42.190 INFO:teuthology.orchestra.run.smithi052.stdout:201863462986 2024-06-10T02:52:42.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:42.194+0000 7ffa2bf6a700 1 -- 172.21.15.52:0/1973185556 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7ffa0c0a2d30 msgr2=0x7ffa0c0a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:52:42.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:42.194+0000 7ffa2bf6a700 1 --2- 172.21.15.52:0/1973185556 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7ffa0c0a2d30 0x7ffa0c0a51f0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7ffa140076e0 tx=0x7ffa14007670 comp rx=0 tx=0).stop 2024-06-10T02:52:42.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:42.194+0000 7ffa2bf6a700 1 -- 172.21.15.52:0/1973185556 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffa241084f0 msgr2=0x7ffa24101ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:52:42.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:42.194+0000 7ffa2bf6a700 1 --2- 172.21.15.52:0/1973185556 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffa241084f0 0x7ffa24101ed0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7ffa1c00c900 tx=0x7ffa1c00cc10 comp rx=0 tx=0).stop 2024-06-10T02:52:42.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:42.194+0000 7ffa2bf6a700 1 -- 172.21.15.52:0/1973185556 shutdown_connections 2024-06-10T02:52:42.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:42.194+0000 7ffa2bf6a700 1 --2- 172.21.15.52:0/1973185556 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7ffa0c0a2d30 0x7ffa0c0a51f0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:42.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:42.194+0000 7ffa2bf6a700 1 --2- 172.21.15.52:0/1973185556 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffa241084f0 0x7ffa24101ed0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:42.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:42.194+0000 7ffa2bf6a700 1 --2- 172.21.15.52:0/1973185556 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffa2406aa00 0x7ffa24101450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:42.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:42.194+0000 7ffa2bf6a700 1 --2- 172.21.15.52:0/1973185556 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffa2406b3c0 0x7ffa24101990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:42.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:42.194+0000 7ffa2bf6a700 1 -- 172.21.15.52:0/1973185556 >> 172.21.15.52:0/1973185556 conn(0x7ffa24078300 msgr2=0x7ffa240fff10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:52:42.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:42.194+0000 7ffa2bf6a700 1 -- 172.21.15.52:0/1973185556 shutdown_connections 2024-06-10T02:52:42.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:42.194+0000 7ffa2bf6a700 1 -- 172.21.15.52:0/1973185556 wait complete. 2024-06-10T02:52:42.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:42.407+0000 7efdc5efc700 1 -- 172.21.15.52:0/291549188 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7efdc002dca0 con 0x7efdc01085f0 2024-06-10T02:52:42.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:42.407+0000 7efdbcff9700 1 -- 172.21.15.52:0/291549188 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7efdb809b8b0 con 0x7efdc01085f0 2024-06-10T02:52:42.407 INFO:teuthology.orchestra.run.smithi052.stdout:223338299460 2024-06-10T02:52:42.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:42.410+0000 7efdc5efc700 1 -- 172.21.15.52:0/291549188 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7efda80a2d30 msgr2=0x7efda80a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:52:42.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:42.410+0000 7efdc5efc700 1 --2- 172.21.15.52:0/291549188 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7efda80a2d30 0x7efda80a51f0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7efdb000b5c0 tx=0x7efdb0005fd0 comp rx=0 tx=0).stop 2024-06-10T02:52:42.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:42.410+0000 7efdc5efc700 1 -- 172.21.15.52:0/291549188 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efdc01085f0 msgr2=0x7efdc0075210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:52:42.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:42.410+0000 7efdc5efc700 1 --2- 172.21.15.52:0/291549188 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efdc01085f0 0x7efdc0075210 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7efdb8007f10 tx=0x7efdb800d3b0 comp rx=0 tx=0).stop 2024-06-10T02:52:42.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:42.410+0000 7efdc5efc700 1 -- 172.21.15.52:0/291549188 shutdown_connections 2024-06-10T02:52:42.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:42.410+0000 7efdc5efc700 1 --2- 172.21.15.52:0/291549188 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7efda80a2d30 0x7efda80a51f0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:42.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:42.410+0000 7efdc5efc700 1 --2- 172.21.15.52:0/291549188 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efdc01085f0 0x7efdc0075210 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:42.410 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:42.410+0000 7efdc5efc700 1 --2- 172.21.15.52:0/291549188 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efdc01024e0 0x7efdc0074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:42.410 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:42.410+0000 7efdc5efc700 1 --2- 172.21.15.52:0/291549188 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efdc0102ea0 0x7efdc0074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:42.410 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:42.410+0000 7efdc5efc700 1 -- 172.21.15.52:0/291549188 >> 172.21.15.52:0/291549188 conn(0x7efdc0079640 msgr2=0x7efdc0101030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:52:42.410 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:42.410+0000 7efdc5efc700 1 -- 172.21.15.52:0/291549188 shutdown_connections 2024-06-10T02:52:42.410 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:42.410+0000 7efdc5efc700 1 -- 172.21.15.52:0/291549188 wait complete. 2024-06-10T02:52:42.717 INFO:tasks.cephadm.ceph_manager.ceph:need seq 313532612656 got 313532612661 for osd.10 2024-06-10T02:52:42.717 DEBUG:teuthology.parallel:result is None 2024-06-10T02:52:42.950 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:52:42 smithi052 ceph-mon[32362]: pgmap v389: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-06-10T02:52:42.950 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:52:42 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:52:42.950 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:52:42 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:52:42.951 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:52:42 smithi052 ceph-mon[32362]: from='client.? 172.21.15.52:0/1973185556' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-06-10T02:52:42.951 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:52:42 smithi052 ceph-mon[32362]: from='client.? 172.21.15.52:0/291549188' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-06-10T02:52:43.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:52:42 smithi120 ceph-mon[36634]: pgmap v389: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-06-10T02:52:43.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:52:42 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:52:43.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:52:42 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:52:43.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:52:42 smithi120 ceph-mon[36634]: from='client.? 172.21.15.52:0/1973185556' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-06-10T02:52:43.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:52:42 smithi120 ceph-mon[36634]: from='client.? 172.21.15.52:0/291549188' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-06-10T02:52:43.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:52:42 smithi188 ceph-mon[36599]: pgmap v389: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-06-10T02:52:43.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:52:42 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:52:43.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:52:42 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:52:43.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:52:42 smithi188 ceph-mon[36599]: from='client.? 172.21.15.52:0/1973185556' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-06-10T02:52:43.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:52:42 smithi188 ceph-mon[36599]: from='client.? 172.21.15.52:0/291549188' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-06-10T02:52:43.606 INFO:tasks.cephadm.ceph_manager.ceph:need seq 201863462980 got 201863462986 for osd.6 2024-06-10T02:52:43.606 DEBUG:teuthology.parallel:result is None 2024-06-10T02:52:43.828 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:52:43 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:52:43.828 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:52:43 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T02:52:43.828 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:52:43 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:52:44.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:52:43 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:52:44.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:52:43 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T02:52:44.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:52:43 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:52:44.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:52:43 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:52:44.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:52:43 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T02:52:44.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:52:43 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:52:44.255 INFO:tasks.cephadm.ceph_manager.ceph:need seq 223338299454 got 223338299460 for osd.7 2024-06-10T02:52:44.255 DEBUG:teuthology.parallel:result is None 2024-06-10T02:52:44.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:44.480+0000 7f2a8c854700 1 -- 172.21.15.52:0/4156856869 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2a8806ab30 msgr2=0x7f2a8810f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:52:44.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:44.480+0000 7f2a8c854700 1 --2- 172.21.15.52:0/4156856869 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2a8806ab30 0x7f2a8810f320 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f2a8000b0a0 tx=0x7f2a8000b3b0 comp rx=0 tx=0).stop 2024-06-10T02:52:44.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:44.480+0000 7f2a8c854700 1 -- 172.21.15.52:0/4156856869 shutdown_connections 2024-06-10T02:52:44.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:44.480+0000 7f2a8c854700 1 --2- 172.21.15.52:0/4156856869 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2a8806ab30 0x7f2a8810f320 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:44.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:44.480+0000 7f2a8c854700 1 --2- 172.21.15.52:0/4156856869 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2a88108500 0x7f2a881088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:44.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:44.480+0000 7f2a8c854700 1 --2- 172.21.15.52:0/4156856869 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2a8806a170 0x7f2a8806a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:44.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:44.480+0000 7f2a8c854700 1 -- 172.21.15.52:0/4156856869 >> 172.21.15.52:0/4156856869 conn(0x7f2a88078340 msgr2=0x7f2a88078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:52:44.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:44.480+0000 7f2a8c854700 1 -- 172.21.15.52:0/4156856869 shutdown_connections 2024-06-10T02:52:44.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:44.480+0000 7f2a8c854700 1 -- 172.21.15.52:0/4156856869 wait complete. 2024-06-10T02:52:44.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:44.481+0000 7f2a8c854700 1 Processor -- start 2024-06-10T02:52:44.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:44.481+0000 7f2a8c854700 1 -- start start 2024-06-10T02:52:44.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:44.481+0000 7f2a8c854700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2a8806a170 0x7f2a8819e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:52:44.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:44.481+0000 7f2a8c854700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2a8806ab30 0x7f2a8819eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:52:44.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:44.481+0000 7f2a8c854700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2a88108500 0x7f2a88198a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:52:44.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:44.481+0000 7f2a8c854700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f2a881127e0 con 0x7f2a8806ab30 2024-06-10T02:52:44.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:44.481+0000 7f2a8c854700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f2a88112660 con 0x7f2a88108500 2024-06-10T02:52:44.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:44.481+0000 7f2a8c854700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f2a88112960 con 0x7f2a8806a170 2024-06-10T02:52:44.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:44.481+0000 7f2a7dd9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2a8806ab30 0x7f2a8819eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:52:44.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:44.481+0000 7f2a7dd9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2a8806ab30 0x7f2a8819eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:39896/0 (socket says 172.21.15.52:39896) 2024-06-10T02:52:44.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:44.482+0000 7f2a7dd9b700 1 -- 172.21.15.52:0/46648852 learned_addr learned my addr 172.21.15.52:0/46648852 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:52:44.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:44.482+0000 7f2a8659c700 1 --2- 172.21.15.52:0/46648852 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2a8806a170 0x7f2a8819e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:52:44.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:44.482+0000 7f2a86d9d700 1 --2- 172.21.15.52:0/46648852 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2a88108500 0x7f2a88198a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:52:44.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:44.482+0000 7f2a7dd9b700 1 -- 172.21.15.52:0/46648852 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2a8806a170 msgr2=0x7f2a8819e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:52:44.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:44.482+0000 7f2a7dd9b700 1 --2- 172.21.15.52:0/46648852 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2a8806a170 0x7f2a8819e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:44.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:44.482+0000 7f2a7dd9b700 1 -- 172.21.15.52:0/46648852 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2a88108500 msgr2=0x7f2a88198a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:52:44.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:44.482+0000 7f2a7dd9b700 1 --2- 172.21.15.52:0/46648852 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2a88108500 0x7f2a88198a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:44.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:44.482+0000 7f2a7dd9b700 1 -- 172.21.15.52:0/46648852 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a88199180 con 0x7f2a8806ab30 2024-06-10T02:52:44.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:44.482+0000 7f2a8659c700 1 --2- 172.21.15.52:0/46648852 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2a8806a170 0x7f2a8819e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T02:52:44.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:44.482+0000 7f2a7dd9b700 1 --2- 172.21.15.52:0/46648852 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2a8806ab30 0x7f2a8819eda0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f2a7000b530 tx=0x7f2a7000b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:52:44.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:44.482+0000 7f2a7ffff700 1 -- 172.21.15.52:0/46648852 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a70015020 con 0x7f2a8806ab30 2024-06-10T02:52:44.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:44.482+0000 7f2a7ffff700 1 -- 172.21.15.52:0/46648852 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a70011d10 con 0x7f2a8806ab30 2024-06-10T02:52:44.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:44.482+0000 7f2a8c854700 1 -- 172.21.15.52:0/46648852 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a88199470 con 0x7f2a8806ab30 2024-06-10T02:52:44.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:44.482+0000 7f2a7ffff700 1 -- 172.21.15.52:0/46648852 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a70010e50 con 0x7f2a8806ab30 2024-06-10T02:52:44.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:44.482+0000 7f2a8c854700 1 -- 172.21.15.52:0/46648852 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a881a5730 con 0x7f2a8806ab30 2024-06-10T02:52:44.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:44.483+0000 7f2a8c854700 1 -- 172.21.15.52:0/46648852 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a88069180 con 0x7f2a8806ab30 2024-06-10T02:52:44.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:44.485+0000 7f2a7ffff700 1 -- 172.21.15.52:0/46648852 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f2a70018020 con 0x7f2a8806ab30 2024-06-10T02:52:44.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:44.485+0000 7f2a7ffff700 1 --2- 172.21.15.52:0/46648852 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f2a740a2dc0 0x7f2a740a5280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:52:44.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:44.485+0000 7f2a7ffff700 1 -- 172.21.15.52:0/46648852 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f2a700cec70 con 0x7f2a8806ab30 2024-06-10T02:52:44.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:44.485+0000 7f2a8659c700 1 --2- 172.21.15.52:0/46648852 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f2a740a2dc0 0x7f2a740a5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:52:44.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:44.486+0000 7f2a8659c700 1 --2- 172.21.15.52:0/46648852 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f2a740a2dc0 0x7f2a740a5280 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f2a88199b60 tx=0x7f2a7800a680 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:52:44.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:44.490+0000 7f2a7ffff700 1 -- 172.21.15.52:0/46648852 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f2a7001f020 con 0x7f2a8806ab30 2024-06-10T02:52:44.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:44.696+0000 7f2a8c854700 1 -- 172.21.15.52:0/46648852 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f2a88199f50 con 0x7f2a8806ab30 2024-06-10T02:52:44.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:44.697+0000 7f2a7ffff700 1 -- 172.21.15.52:0/46648852 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f2a7009b820 con 0x7f2a8806ab30 2024-06-10T02:52:44.696 INFO:teuthology.orchestra.run.smithi052.stdout:347892351025 2024-06-10T02:52:44.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:44.699+0000 7f2a8c854700 1 -- 172.21.15.52:0/46648852 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f2a740a2dc0 msgr2=0x7f2a740a5280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:52:44.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:44.699+0000 7f2a8c854700 1 --2- 172.21.15.52:0/46648852 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f2a740a2dc0 0x7f2a740a5280 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f2a88199b60 tx=0x7f2a7800a680 comp rx=0 tx=0).stop 2024-06-10T02:52:44.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:44.699+0000 7f2a8c854700 1 -- 172.21.15.52:0/46648852 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2a8806ab30 msgr2=0x7f2a8819eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:52:44.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:44.699+0000 7f2a8c854700 1 --2- 172.21.15.52:0/46648852 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2a8806ab30 0x7f2a8819eda0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f2a7000b530 tx=0x7f2a7000b8f0 comp rx=0 tx=0).stop 2024-06-10T02:52:44.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:44.699+0000 7f2a8c854700 1 -- 172.21.15.52:0/46648852 shutdown_connections 2024-06-10T02:52:44.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:44.699+0000 7f2a8c854700 1 --2- 172.21.15.52:0/46648852 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2a88108500 0x7f2a88198a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:44.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:44.699+0000 7f2a8c854700 1 --2- 172.21.15.52:0/46648852 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2a8806a170 0x7f2a8819e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:44.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:44.699+0000 7f2a8c854700 1 --2- 172.21.15.52:0/46648852 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f2a740a2dc0 0x7f2a740a5280 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:44.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:44.699+0000 7f2a8c854700 1 --2- 172.21.15.52:0/46648852 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2a8806ab30 0x7f2a8819eda0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:44.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:44.699+0000 7f2a8c854700 1 -- 172.21.15.52:0/46648852 >> 172.21.15.52:0/46648852 conn(0x7f2a88078340 msgr2=0x7f2a880ffe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:52:44.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:44.699+0000 7f2a8c854700 1 -- 172.21.15.52:0/46648852 shutdown_connections 2024-06-10T02:52:44.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:44.699+0000 7f2a8c854700 1 -- 172.21.15.52:0/46648852 wait complete. 2024-06-10T02:52:45.081 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:52:44 smithi052 ceph-mon[32362]: pgmap v390: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-06-10T02:52:45.081 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:52:44 smithi052 ceph-mon[32362]: mgrmap e18: x(active, since 10m), standbys: z, y 2024-06-10T02:52:45.081 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:52:44 smithi052 ceph-mon[32362]: from='client.? 172.21.15.52:0/46648852' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-06-10T02:52:45.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:52:44 smithi120 ceph-mon[36634]: pgmap v390: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-06-10T02:52:45.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:52:44 smithi120 ceph-mon[36634]: mgrmap e18: x(active, since 10m), standbys: z, y 2024-06-10T02:52:45.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:52:44 smithi120 ceph-mon[36634]: from='client.? 172.21.15.52:0/46648852' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-06-10T02:52:45.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:52:44 smithi188 ceph-mon[36599]: pgmap v390: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-06-10T02:52:45.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:52:44 smithi188 ceph-mon[36599]: mgrmap e18: x(active, since 10m), standbys: z, y 2024-06-10T02:52:45.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:52:44 smithi188 ceph-mon[36599]: from='client.? 172.21.15.52:0/46648852' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-06-10T02:52:45.394 INFO:tasks.cephadm.ceph_manager.ceph:need seq 347892351020 got 347892351025 for osd.11 2024-06-10T02:52:45.394 DEBUG:teuthology.parallel:result is None 2024-06-10T02:52:45.395 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-06-10T02:52:45.395 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph pg dump --format=json 2024-06-10T02:52:45.649 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:52:47.073 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:52:46 smithi052 ceph-mon[32362]: pgmap v391: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-10T02:52:47.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:52:46 smithi120 ceph-mon[36634]: pgmap v391: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-10T02:52:47.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:52:46 smithi188 ceph-mon[36599]: pgmap v391: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-10T02:52:47.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:47.334+0000 7fe8b88ff700 1 -- 172.21.15.52:0/860202677 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe8b41024b0 msgr2=0x7fe8b410e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:52:47.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:47.334+0000 7fe8b88ff700 1 --2- 172.21.15.52:0/860202677 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe8b41024b0 0x7fe8b410e260 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fe8ac00b0a0 tx=0x7fe8ac00b3b0 comp rx=0 tx=0).stop 2024-06-10T02:52:47.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:47.335+0000 7fe8b88ff700 1 -- 172.21.15.52:0/860202677 shutdown_connections 2024-06-10T02:52:47.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:47.335+0000 7fe8b88ff700 1 --2- 172.21.15.52:0/860202677 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe8b41024b0 0x7fe8b410e260 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:47.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:47.335+0000 7fe8b88ff700 1 --2- 172.21.15.52:0/860202677 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe8b41063c0 0x7fe8b41067a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:47.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:47.335+0000 7fe8b88ff700 1 --2- 172.21.15.52:0/860202677 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe8b4101af0 0x7fe8b4101f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:47.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:47.335+0000 7fe8b88ff700 1 -- 172.21.15.52:0/860202677 >> 172.21.15.52:0/860202677 conn(0x7fe8b4077990 msgr2=0x7fe8b40fed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:52:47.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:47.335+0000 7fe8b88ff700 1 -- 172.21.15.52:0/860202677 shutdown_connections 2024-06-10T02:52:47.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:47.335+0000 7fe8b88ff700 1 -- 172.21.15.52:0/860202677 wait complete. 2024-06-10T02:52:47.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:47.335+0000 7fe8b88ff700 1 Processor -- start 2024-06-10T02:52:47.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:47.336+0000 7fe8b88ff700 1 -- start start 2024-06-10T02:52:47.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:47.336+0000 7fe8b88ff700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe8b4101af0 0x7fe8b41999f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:52:47.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:47.336+0000 7fe8b88ff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe8b41024b0 0x7fe8b4199f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:52:47.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:47.336+0000 7fe8b88ff700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe8b41063c0 0x7fe8b41a4a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:52:47.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:47.336+0000 7fe8b88ff700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fe8b4111710 con 0x7fe8b41024b0 2024-06-10T02:52:47.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:47.336+0000 7fe8b88ff700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe8b4111590 con 0x7fe8b4101af0 2024-06-10T02:52:47.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:47.336+0000 7fe8b88ff700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fe8b4111890 con 0x7fe8b41063c0 2024-06-10T02:52:47.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:47.336+0000 7fe8a9d9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe8b41024b0 0x7fe8b4199f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:52:47.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:47.336+0000 7fe8a9d9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe8b41024b0 0x7fe8b4199f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:39922/0 (socket says 172.21.15.52:39922) 2024-06-10T02:52:47.338 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:47.336+0000 7fe8a9d9b700 1 -- 172.21.15.52:0/851272091 learned_addr learned my addr 172.21.15.52:0/851272091 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:52:47.338 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:47.336+0000 7fe8b259c700 1 --2- 172.21.15.52:0/851272091 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe8b4101af0 0x7fe8b41999f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:52:47.338 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:47.336+0000 7fe8b2d9d700 1 --2- 172.21.15.52:0/851272091 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe8b41063c0 0x7fe8b41a4a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:52:47.338 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:47.336+0000 7fe8a9d9b700 1 -- 172.21.15.52:0/851272091 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe8b41063c0 msgr2=0x7fe8b41a4a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:52:47.338 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:47.336+0000 7fe8a9d9b700 1 --2- 172.21.15.52:0/851272091 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe8b41063c0 0x7fe8b41a4a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:47.338 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:47.336+0000 7fe8a9d9b700 1 -- 172.21.15.52:0/851272091 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe8b4101af0 msgr2=0x7fe8b41999f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:52:47.338 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:47.336+0000 7fe8a9d9b700 1 --2- 172.21.15.52:0/851272091 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe8b4101af0 0x7fe8b41999f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:47.338 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:47.337+0000 7fe8a9d9b700 1 -- 172.21.15.52:0/851272091 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8b41a4fb0 con 0x7fe8b41024b0 2024-06-10T02:52:47.338 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:47.337+0000 7fe8a9d9b700 1 --2- 172.21.15.52:0/851272091 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe8b41024b0 0x7fe8b4199f30 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fe89c00c920 tx=0x7fe89c00cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:52:47.338 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:47.337+0000 7fe8abfff700 1 -- 172.21.15.52:0/851272091 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe89c004a30 con 0x7fe8b41024b0 2024-06-10T02:52:47.338 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:47.337+0000 7fe8abfff700 1 -- 172.21.15.52:0/851272091 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe89c005500 con 0x7fe8b41024b0 2024-06-10T02:52:47.338 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:47.337+0000 7fe8abfff700 1 -- 172.21.15.52:0/851272091 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe89c007d00 con 0x7fe8b41024b0 2024-06-10T02:52:47.339 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:47.337+0000 7fe8b88ff700 1 -- 172.21.15.52:0/851272091 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8b41a5180 con 0x7fe8b41024b0 2024-06-10T02:52:47.339 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:47.337+0000 7fe8b88ff700 1 -- 172.21.15.52:0/851272091 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8b41a5670 con 0x7fe8b41024b0 2024-06-10T02:52:47.339 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:47.338+0000 7fe8b88ff700 1 -- 172.21.15.52:0/851272091 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8b4069180 con 0x7fe8b41024b0 2024-06-10T02:52:47.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:47.339+0000 7fe8abfff700 1 -- 172.21.15.52:0/851272091 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fe89c005000 con 0x7fe8b41024b0 2024-06-10T02:52:47.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:47.340+0000 7fe8abfff700 1 --2- 172.21.15.52:0/851272091 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fe8a00a2ca0 0x7fe8a00a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:52:47.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:47.340+0000 7fe8b259c700 1 --2- 172.21.15.52:0/851272091 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fe8a00a2ca0 0x7fe8a00a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:52:47.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:47.340+0000 7fe8abfff700 1 -- 172.21.15.52:0/851272091 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7fe89c0ceef0 con 0x7fe8b41024b0 2024-06-10T02:52:47.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:47.340+0000 7fe8b259c700 1 --2- 172.21.15.52:0/851272091 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fe8a00a2ca0 0x7fe8a00a5160 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fe8a4009a10 tx=0x7fe8a4009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:52:47.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:47.344+0000 7fe8abfff700 1 -- 172.21.15.52:0/851272091 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fe89c014030 con 0x7fe8b41024b0 2024-06-10T02:52:47.546 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:47.547+0000 7fe8b88ff700 1 -- 172.21.15.52:0/851272091 --> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe8b40628d0 con 0x7fe8a00a2ca0 2024-06-10T02:52:47.556 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:47.557+0000 7fe8abfff700 1 -- 172.21.15.52:0/851272091 <== mgr.14150 v2:172.21.15.52:6800/3214763969 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+341019 (secure 0 0 0) 0x7fe8b40628d0 con 0x7fe8a00a2ca0 2024-06-10T02:52:47.560 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-10T02:52:47.563 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:47.560+0000 7fe8b88ff700 1 -- 172.21.15.52:0/851272091 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fe8a00a2ca0 msgr2=0x7fe8a00a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:52:47.563 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:47.560+0000 7fe8b88ff700 1 --2- 172.21.15.52:0/851272091 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fe8a00a2ca0 0x7fe8a00a5160 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fe8a4009a10 tx=0x7fe8a4009190 comp rx=0 tx=0).stop 2024-06-10T02:52:47.563 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:47.560+0000 7fe8b88ff700 1 -- 172.21.15.52:0/851272091 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe8b41024b0 msgr2=0x7fe8b4199f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:52:47.563 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:47.560+0000 7fe8b88ff700 1 --2- 172.21.15.52:0/851272091 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe8b41024b0 0x7fe8b4199f30 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fe89c00c920 tx=0x7fe89c00cc30 comp rx=0 tx=0).stop 2024-06-10T02:52:47.563 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:47.560+0000 7fe8b88ff700 1 -- 172.21.15.52:0/851272091 shutdown_connections 2024-06-10T02:52:47.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:47.560+0000 7fe8b88ff700 1 --2- 172.21.15.52:0/851272091 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe8b41063c0 0x7fe8b41a4a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:47.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:47.560+0000 7fe8b88ff700 1 --2- 172.21.15.52:0/851272091 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe8b4101af0 0x7fe8b41999f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:47.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:47.560+0000 7fe8b88ff700 1 --2- 172.21.15.52:0/851272091 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fe8a00a2ca0 0x7fe8a00a5160 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:47.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:47.560+0000 7fe8b88ff700 1 --2- 172.21.15.52:0/851272091 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe8b41024b0 0x7fe8b4199f30 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:47.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:47.560+0000 7fe8b88ff700 1 -- 172.21.15.52:0/851272091 >> 172.21.15.52:0/851272091 conn(0x7fe8b4077990 msgr2=0x7fe8b4104270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:52:47.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:47.561+0000 7fe8b88ff700 1 -- 172.21.15.52:0/851272091 shutdown_connections 2024-06-10T02:52:47.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:47.561+0000 7fe8b88ff700 1 -- 172.21.15.52:0/851272091 wait complete. 2024-06-10T02:52:47.564 INFO:teuthology.orchestra.run.smithi052.stderr:dumped all 2024-06-10T02:52:48.330 INFO:teuthology.orchestra.run.smithi052.stdout:{"pg_ready":true,"pg_map":{"version":392,"stamp":"2024-06-10T02:52:47.019309+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":661911,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1139,"num_read_kb":696,"num_write":445,"num_write_kb":3094,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1181318,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":591,"ondisk_log_size":591,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":379376,"kb_used_data":13556,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124562448,"statfs":{"total":1151940427776,"available":1151551946752,"internally_reserved":0,"allocated":13881344,"data_stored":8835827,"data_compressed":497715,"data_compressed_allocated":675840,"data_compressed_original":2850816,"omap_allocated":19141,"internal_metadata":374453563},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":672,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":144,"num_read_kb":74,"num_write":11,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003606"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":39,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.365141+0000","last_change":"2024-06-10T02:50:02.209146+0000","last_active":"2024-06-10T02:51:46.365141+0000","last_peered":"2024-06-10T02:51:46.365141+0000","last_clean":"2024-06-10T02:51:46.365141+0000","last_became_active":"2024-06-10T02:50:02.208027+0000","last_became_peered":"2024-06-10T02:50:02.208027+0000","last_unstale":"2024-06-10T02:51:46.365141+0000","last_undegraded":"2024-06-10T02:51:46.365141+0000","last_fullsized":"2024-06-10T02:51:46.365141+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:35:34.765849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"100'4","reported_seq":43,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.989740+0000","last_change":"2024-06-10T02:50:02.198767+0000","last_active":"2024-06-10T02:52:42.989740+0000","last_peered":"2024-06-10T02:52:42.989740+0000","last_clean":"2024-06-10T02:52:42.989740+0000","last_became_active":"2024-06-10T02:50:02.198450+0000","last_became_peered":"2024-06-10T02:50:02.198450+0000","last_unstale":"2024-06-10T02:52:42.989740+0000","last_undegraded":"2024-06-10T02:52:42.989740+0000","last_fullsized":"2024-06-10T02:52:42.989740+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:49:56.666083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"100'7","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:43.000326+0000","last_change":"2024-06-10T02:50:02.208885+0000","last_active":"2024-06-10T02:52:43.000326+0000","last_peered":"2024-06-10T02:52:43.000326+0000","last_clean":"2024-06-10T02:52:43.000326+0000","last_became_active":"2024-06-10T02:50:02.206953+0000","last_became_peered":"2024-06-10T02:50:02.206953+0000","last_unstale":"2024-06-10T02:52:43.000326+0000","last_undegraded":"2024-06-10T02:52:43.000326+0000","last_fullsized":"2024-06-10T02:52:43.000326+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:41:23.928939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"100'8","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.993994+0000","last_change":"2024-06-10T02:50:02.208632+0000","last_active":"2024-06-10T02:52:42.993994+0000","last_peered":"2024-06-10T02:52:42.993994+0000","last_clean":"2024-06-10T02:52:42.993994+0000","last_became_active":"2024-06-10T02:50:02.207587+0000","last_became_peered":"2024-06-10T02:50:02.207587+0000","last_unstale":"2024-06-10T02:52:42.993994+0000","last_undegraded":"2024-06-10T02:52:42.993994+0000","last_fullsized":"2024-06-10T02:52:42.993994+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T14:34:36.720170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"100'4","reported_seq":43,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.973606+0000","last_change":"2024-06-10T02:50:02.197966+0000","last_active":"2024-06-10T02:52:42.973606+0000","last_peered":"2024-06-10T02:52:42.973606+0000","last_clean":"2024-06-10T02:52:42.973606+0000","last_became_active":"2024-06-10T02:50:02.197612+0000","last_became_peered":"2024-06-10T02:50:02.197612+0000","last_unstale":"2024-06-10T02:52:42.973606+0000","last_undegraded":"2024-06-10T02:52:42.973606+0000","last_fullsized":"2024-06-10T02:52:42.973606+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:24:53.804613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"100'6","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.987354+0000","last_change":"2024-06-10T02:50:02.201952+0000","last_active":"2024-06-10T02:52:42.987354+0000","last_peered":"2024-06-10T02:52:42.987354+0000","last_clean":"2024-06-10T02:52:42.987354+0000","last_became_active":"2024-06-10T02:50:02.200828+0000","last_became_peered":"2024-06-10T02:50:02.200828+0000","last_unstale":"2024-06-10T02:52:42.987354+0000","last_undegraded":"2024-06-10T02:52:42.987354+0000","last_fullsized":"2024-06-10T02:52:42.987354+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:46:28.518846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":39,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.295316+0000","last_change":"2024-06-10T02:50:02.203483+0000","last_active":"2024-06-10T02:51:46.295316+0000","last_peered":"2024-06-10T02:51:46.295316+0000","last_clean":"2024-06-10T02:51:46.295316+0000","last_became_active":"2024-06-10T02:50:02.203153+0000","last_became_peered":"2024-06-10T02:50:02.203153+0000","last_unstale":"2024-06-10T02:51:46.295316+0000","last_undegraded":"2024-06-10T02:51:46.295316+0000","last_fullsized":"2024-06-10T02:51:46.295316+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:29:30.422449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":39,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.831352+0000","last_change":"2024-06-10T02:50:02.201686+0000","last_active":"2024-06-10T02:51:45.831352+0000","last_peered":"2024-06-10T02:51:45.831352+0000","last_clean":"2024-06-10T02:51:45.831352+0000","last_became_active":"2024-06-10T02:50:02.200973+0000","last_became_peered":"2024-06-10T02:50:02.200973+0000","last_unstale":"2024-06-10T02:51:45.831352+0000","last_undegraded":"2024-06-10T02:51:45.831352+0000","last_fullsized":"2024-06-10T02:51:45.831352+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:41:13.712675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":39,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.854799+0000","last_change":"2024-06-10T02:50:02.210497+0000","last_active":"2024-06-10T02:51:45.854799+0000","last_peered":"2024-06-10T02:51:45.854799+0000","last_clean":"2024-06-10T02:51:45.854799+0000","last_became_active":"2024-06-10T02:50:02.209675+0000","last_became_peered":"2024-06-10T02:50:02.209675+0000","last_unstale":"2024-06-10T02:51:45.854799+0000","last_undegraded":"2024-06-10T02:51:45.854799+0000","last_fullsized":"2024-06-10T02:51:45.854799+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:50:29.879669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"100'16","reported_seq":61,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.994975+0000","last_change":"2024-06-10T02:50:44.741139+0000","last_active":"2024-06-10T02:52:42.994975+0000","last_peered":"2024-06-10T02:52:42.994975+0000","last_clean":"2024-06-10T02:52:42.994975+0000","last_became_active":"2024-06-10T02:50:44.740685+0000","last_became_peered":"2024-06-10T02:50:44.740685+0000","last_unstale":"2024-06-10T02:52:42.994975+0000","last_undegraded":"2024-06-10T02:52:42.994975+0000","last_fullsized":"2024-06-10T02:52:42.994975+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:58:59.872732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":39,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.294800+0000","last_change":"2024-06-10T02:50:02.206993+0000","last_active":"2024-06-10T02:51:46.294800+0000","last_peered":"2024-06-10T02:51:46.294800+0000","last_clean":"2024-06-10T02:51:46.294800+0000","last_became_active":"2024-06-10T02:50:02.206650+0000","last_became_peered":"2024-06-10T02:50:02.206650+0000","last_unstale":"2024-06-10T02:51:46.294800+0000","last_undegraded":"2024-06-10T02:51:46.294800+0000","last_fullsized":"2024-06-10T02:51:46.294800+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T14:27:49.555350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"100'7","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.979413+0000","last_change":"2024-06-10T02:50:02.200816+0000","last_active":"2024-06-10T02:52:42.979413+0000","last_peered":"2024-06-10T02:52:42.979413+0000","last_clean":"2024-06-10T02:52:42.979413+0000","last_became_active":"2024-06-10T02:50:02.200503+0000","last_became_peered":"2024-06-10T02:50:02.200503+0000","last_unstale":"2024-06-10T02:52:42.979413+0000","last_undegraded":"2024-06-10T02:52:42.979413+0000","last_fullsized":"2024-06-10T02:52:42.979413+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:01:08.326728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":39,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.715035+0000","last_change":"2024-06-10T02:50:02.203644+0000","last_active":"2024-06-10T02:51:45.715035+0000","last_peered":"2024-06-10T02:51:45.715035+0000","last_clean":"2024-06-10T02:51:45.715035+0000","last_became_active":"2024-06-10T02:50:02.203100+0000","last_became_peered":"2024-06-10T02:50:02.203100+0000","last_unstale":"2024-06-10T02:51:45.715035+0000","last_undegraded":"2024-06-10T02:51:45.715035+0000","last_fullsized":"2024-06-10T02:51:45.715035+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:01:42.730089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"100'7","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.995068+0000","last_change":"2024-06-10T02:50:02.203330+0000","last_active":"2024-06-10T02:52:42.995068+0000","last_peered":"2024-06-10T02:52:42.995068+0000","last_clean":"2024-06-10T02:52:42.995068+0000","last_became_active":"2024-06-10T02:50:02.202995+0000","last_became_peered":"2024-06-10T02:50:02.202995+0000","last_unstale":"2024-06-10T02:52:42.995068+0000","last_undegraded":"2024-06-10T02:52:42.995068+0000","last_fullsized":"2024-06-10T02:52:42.995068+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:17:23.170591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"100'7","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.986439+0000","last_change":"2024-06-10T02:50:02.204875+0000","last_active":"2024-06-10T02:52:42.986439+0000","last_peered":"2024-06-10T02:52:42.986439+0000","last_clean":"2024-06-10T02:52:42.986439+0000","last_became_active":"2024-06-10T02:50:02.200706+0000","last_became_peered":"2024-06-10T02:50:02.200706+0000","last_unstale":"2024-06-10T02:52:42.986439+0000","last_undegraded":"2024-06-10T02:52:42.986439+0000","last_fullsized":"2024-06-10T02:52:42.986439+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:27:12.282772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":39,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.854133+0000","last_change":"2024-06-10T02:50:02.203187+0000","last_active":"2024-06-10T02:51:45.854133+0000","last_peered":"2024-06-10T02:51:45.854133+0000","last_clean":"2024-06-10T02:51:45.854133+0000","last_became_active":"2024-06-10T02:50:02.202034+0000","last_became_peered":"2024-06-10T02:50:02.202034+0000","last_unstale":"2024-06-10T02:51:45.854133+0000","last_undegraded":"2024-06-10T02:51:45.854133+0000","last_fullsized":"2024-06-10T02:51:45.854133+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:59:44.783084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":39,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.365442+0000","last_change":"2024-06-10T02:50:02.209102+0000","last_active":"2024-06-10T02:51:46.365442+0000","last_peered":"2024-06-10T02:51:46.365442+0000","last_clean":"2024-06-10T02:51:46.365442+0000","last_became_active":"2024-06-10T02:50:02.208730+0000","last_became_peered":"2024-06-10T02:50:02.208730+0000","last_unstale":"2024-06-10T02:51:46.365442+0000","last_undegraded":"2024-06-10T02:51:46.365442+0000","last_fullsized":"2024-06-10T02:51:46.365442+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:08:38.838415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"100'10","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:43.000459+0000","last_change":"2024-06-10T02:50:02.204008+0000","last_active":"2024-06-10T02:52:43.000459+0000","last_peered":"2024-06-10T02:52:43.000459+0000","last_clean":"2024-06-10T02:52:43.000459+0000","last_became_active":"2024-06-10T02:50:02.203655+0000","last_became_peered":"2024-06-10T02:50:02.203655+0000","last_unstale":"2024-06-10T02:52:43.000459+0000","last_undegraded":"2024-06-10T02:52:43.000459+0000","last_fullsized":"2024-06-10T02:52:43.000459+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:32:15.972662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"100'7","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.991723+0000","last_change":"2024-06-10T02:50:02.204917+0000","last_active":"2024-06-10T02:52:42.991723+0000","last_peered":"2024-06-10T02:52:42.991723+0000","last_clean":"2024-06-10T02:52:42.991723+0000","last_became_active":"2024-06-10T02:50:02.204568+0000","last_became_peered":"2024-06-10T02:50:02.204568+0000","last_unstale":"2024-06-10T02:52:42.991723+0000","last_undegraded":"2024-06-10T02:52:42.991723+0000","last_fullsized":"2024-06-10T02:52:42.991723+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:49:46.802665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.721297+0000","last_change":"2024-06-10T02:50:12.240697+0000","last_active":"2024-06-10T02:51:45.721297+0000","last_peered":"2024-06-10T02:51:45.721297+0000","last_clean":"2024-06-10T02:51:45.721297+0000","last_became_active":"2024-06-10T02:50:12.240229+0000","last_became_peered":"2024-06-10T02:50:12.240229+0000","last_unstale":"2024-06-10T02:51:45.721297+0000","last_undegraded":"2024-06-10T02:51:45.721297+0000","last_fullsized":"2024-06-10T02:51:45.721297+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:45:55.071960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"100'14","reported_seq":168,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:43.776213+0000","last_change":"2024-06-10T02:50:44.786783+0000","last_active":"2024-06-10T02:52:43.776213+0000","last_peered":"2024-06-10T02:52:43.776213+0000","last_clean":"2024-06-10T02:52:43.776213+0000","last_became_active":"2024-06-10T02:50:44.781773+0000","last_became_peered":"2024-06-10T02:50:44.781773+0000","last_unstale":"2024-06-10T02:52:43.776213+0000","last_undegraded":"2024-06-10T02:52:43.776213+0000","last_fullsized":"2024-06-10T02:52:43.776213+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:31:06.025854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":97,"num_read_kb":97,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,4],"acting":[9,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":39,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.831964+0000","last_change":"2024-06-10T02:50:02.202573+0000","last_active":"2024-06-10T02:51:45.831964+0000","last_peered":"2024-06-10T02:51:45.831964+0000","last_clean":"2024-06-10T02:51:45.831964+0000","last_became_active":"2024-06-10T02:50:02.202241+0000","last_became_peered":"2024-06-10T02:50:02.202241+0000","last_unstale":"2024-06-10T02:51:45.831964+0000","last_undegraded":"2024-06-10T02:51:45.831964+0000","last_fullsized":"2024-06-10T02:51:45.831964+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:31:40.230851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.383047+0000","last_change":"2024-06-10T02:50:12.238372+0000","last_active":"2024-06-10T02:51:46.383047+0000","last_peered":"2024-06-10T02:51:46.383047+0000","last_clean":"2024-06-10T02:51:46.383047+0000","last_became_active":"2024-06-10T02:50:12.237884+0000","last_became_peered":"2024-06-10T02:50:12.237884+0000","last_unstale":"2024-06-10T02:51:46.383047+0000","last_undegraded":"2024-06-10T02:51:46.383047+0000","last_fullsized":"2024-06-10T02:51:46.383047+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:55:31.432655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"98'2","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.832183+0000","last_change":"2024-06-10T02:49:59.031528+0000","last_active":"2024-06-10T02:51:45.832183+0000","last_peered":"2024-06-10T02:51:45.832183+0000","last_clean":"2024-06-10T02:51:45.832183+0000","last_became_active":"2024-06-10T02:49:59.031147+0000","last_became_peered":"2024-06-10T02:49:59.031147+0000","last_unstale":"2024-06-10T02:51:45.832183+0000","last_undegraded":"2024-06-10T02:51:45.832183+0000","last_fullsized":"2024-06-10T02:51:45.832183+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:17:08.534396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"100'6","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:43.000760+0000","last_change":"2024-06-10T02:50:02.202858+0000","last_active":"2024-06-10T02:52:43.000760+0000","last_peered":"2024-06-10T02:52:43.000760+0000","last_clean":"2024-06-10T02:52:43.000760+0000","last_became_active":"2024-06-10T02:50:02.202263+0000","last_became_peered":"2024-06-10T02:50:02.202263+0000","last_unstale":"2024-06-10T02:52:43.000760+0000","last_undegraded":"2024-06-10T02:52:43.000760+0000","last_fullsized":"2024-06-10T02:52:43.000760+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:23:15.382762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.383227+0000","last_change":"2024-06-10T02:50:12.237370+0000","last_active":"2024-06-10T02:51:46.383227+0000","last_peered":"2024-06-10T02:51:46.383227+0000","last_clean":"2024-06-10T02:51:46.383227+0000","last_became_active":"2024-06-10T02:50:12.236803+0000","last_became_peered":"2024-06-10T02:50:12.236803+0000","last_unstale":"2024-06-10T02:51:46.383227+0000","last_undegraded":"2024-06-10T02:51:46.383227+0000","last_fullsized":"2024-06-10T02:51:46.383227+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:06:40.695710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"98'12","reported_seq":58,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.724307+0000","last_change":"2024-06-10T02:49:59.000911+0000","last_active":"2024-06-10T02:51:45.724307+0000","last_peered":"2024-06-10T02:51:45.724307+0000","last_clean":"2024-06-10T02:51:45.724307+0000","last_became_active":"2024-06-10T02:49:59.000571+0000","last_became_peered":"2024-06-10T02:49:59.000571+0000","last_unstale":"2024-06-10T02:51:45.724307+0000","last_undegraded":"2024-06-10T02:51:45.724307+0000","last_fullsized":"2024-06-10T02:51:45.724307+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:49:24.635793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":39,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.854338+0000","last_change":"2024-06-10T02:50:02.209614+0000","last_active":"2024-06-10T02:51:45.854338+0000","last_peered":"2024-06-10T02:51:45.854338+0000","last_clean":"2024-06-10T02:51:45.854338+0000","last_became_active":"2024-06-10T02:50:02.209251+0000","last_became_peered":"2024-06-10T02:50:02.209251+0000","last_unstale":"2024-06-10T02:51:45.854338+0000","last_undegraded":"2024-06-10T02:51:45.854338+0000","last_fullsized":"2024-06-10T02:51:45.854338+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:35:32.256081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.383229+0000","last_change":"2024-06-10T02:50:12.237811+0000","last_active":"2024-06-10T02:51:46.383229+0000","last_peered":"2024-06-10T02:51:46.383229+0000","last_clean":"2024-06-10T02:51:46.383229+0000","last_became_active":"2024-06-10T02:50:12.237017+0000","last_became_peered":"2024-06-10T02:50:12.237017+0000","last_unstale":"2024-06-10T02:51:46.383229+0000","last_undegraded":"2024-06-10T02:51:46.383229+0000","last_fullsized":"2024-06-10T02:51:46.383229+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:14:29.818885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"98'5","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.723967+0000","last_change":"2024-06-10T02:49:58.996971+0000","last_active":"2024-06-10T02:51:45.723967+0000","last_peered":"2024-06-10T02:51:45.723967+0000","last_clean":"2024-06-10T02:51:45.723967+0000","last_became_active":"2024-06-10T02:49:58.996601+0000","last_became_peered":"2024-06-10T02:49:58.996601+0000","last_unstale":"2024-06-10T02:51:45.723967+0000","last_undegraded":"2024-06-10T02:51:45.723967+0000","last_fullsized":"2024-06-10T02:51:45.723967+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:26:45.963351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"100'15","reported_seq":60,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:43.002596+0000","last_change":"2024-06-10T02:50:44.745816+0000","last_active":"2024-06-10T02:52:43.002596+0000","last_peered":"2024-06-10T02:52:43.002596+0000","last_clean":"2024-06-10T02:52:43.002596+0000","last_became_active":"2024-06-10T02:50:44.745386+0000","last_became_peered":"2024-06-10T02:50:44.745386+0000","last_unstale":"2024-06-10T02:52:43.002596+0000","last_undegraded":"2024-06-10T02:52:43.002596+0000","last_fullsized":"2024-06-10T02:52:43.002596+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:08:53.977133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.721700+0000","last_change":"2024-06-10T02:50:12.236835+0000","last_active":"2024-06-10T02:51:45.721700+0000","last_peered":"2024-06-10T02:51:45.721700+0000","last_clean":"2024-06-10T02:51:45.721700+0000","last_became_active":"2024-06-10T02:50:12.236309+0000","last_became_peered":"2024-06-10T02:50:12.236309+0000","last_unstale":"2024-06-10T02:51:45.721700+0000","last_undegraded":"2024-06-10T02:51:45.721700+0000","last_fullsized":"2024-06-10T02:51:45.721700+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:09:02.789286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"100'7","reported_seq":153,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:44.769598+0000","last_change":"2024-06-10T02:49:58.997725+0000","last_active":"2024-06-10T02:52:44.769598+0000","last_peered":"2024-06-10T02:52:44.769598+0000","last_clean":"2024-06-10T02:52:44.769598+0000","last_became_active":"2024-06-10T02:49:58.997375+0000","last_became_peered":"2024-06-10T02:49:58.997375+0000","last_unstale":"2024-06-10T02:52:44.769598+0000","last_undegraded":"2024-06-10T02:52:44.769598+0000","last_fullsized":"2024-06-10T02:52:44.769598+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:28:49.714356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1153,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":101,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"100'7","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.995883+0000","last_change":"2024-06-10T02:50:02.201247+0000","last_active":"2024-06-10T02:52:42.995883+0000","last_peered":"2024-06-10T02:52:42.995883+0000","last_clean":"2024-06-10T02:52:42.995883+0000","last_became_active":"2024-06-10T02:50:02.200897+0000","last_became_peered":"2024-06-10T02:50:02.200897+0000","last_unstale":"2024-06-10T02:52:42.995883+0000","last_undegraded":"2024-06-10T02:52:42.995883+0000","last_fullsized":"2024-06-10T02:52:42.995883+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:01:58.166644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.721648+0000","last_change":"2024-06-10T02:50:12.243097+0000","last_active":"2024-06-10T02:51:45.721648+0000","last_peered":"2024-06-10T02:51:45.721648+0000","last_clean":"2024-06-10T02:51:45.721648+0000","last_became_active":"2024-06-10T02:50:12.242501+0000","last_became_peered":"2024-06-10T02:50:12.242501+0000","last_unstale":"2024-06-10T02:51:45.721648+0000","last_undegraded":"2024-06-10T02:51:45.721648+0000","last_fullsized":"2024-06-10T02:51:45.721648+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T14:42:44.220340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"98'7","reported_seq":52,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.720539+0000","last_change":"2024-06-10T02:49:58.996956+0000","last_active":"2024-06-10T02:51:45.720539+0000","last_peered":"2024-06-10T02:51:45.720539+0000","last_clean":"2024-06-10T02:51:45.720539+0000","last_became_active":"2024-06-10T02:49:58.996634+0000","last_became_peered":"2024-06-10T02:49:58.996634+0000","last_unstale":"2024-06-10T02:51:45.720539+0000","last_undegraded":"2024-06-10T02:51:45.720539+0000","last_fullsized":"2024-06-10T02:51:45.720539+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:25:02.364472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"100'8","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.995286+0000","last_change":"2024-06-10T02:50:02.205808+0000","last_active":"2024-06-10T02:52:42.995286+0000","last_peered":"2024-06-10T02:52:42.995286+0000","last_clean":"2024-06-10T02:52:42.995286+0000","last_became_active":"2024-06-10T02:50:02.203558+0000","last_became_peered":"2024-06-10T02:50:02.203558+0000","last_unstale":"2024-06-10T02:52:42.995286+0000","last_undegraded":"2024-06-10T02:52:42.995286+0000","last_fullsized":"2024-06-10T02:52:42.995286+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T13:43:23.315375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.365641+0000","last_change":"2024-06-10T02:50:12.242005+0000","last_active":"2024-06-10T02:51:46.365641+0000","last_peered":"2024-06-10T02:51:46.365641+0000","last_clean":"2024-06-10T02:51:46.365641+0000","last_became_active":"2024-06-10T02:50:12.241295+0000","last_became_peered":"2024-06-10T02:50:12.241295+0000","last_unstale":"2024-06-10T02:51:46.365641+0000","last_undegraded":"2024-06-10T02:51:46.365641+0000","last_fullsized":"2024-06-10T02:51:46.365641+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:32:52.702756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"98'3","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.714696+0000","last_change":"2024-06-10T02:49:58.995579+0000","last_active":"2024-06-10T02:51:45.714696+0000","last_peered":"2024-06-10T02:51:45.714696+0000","last_clean":"2024-06-10T02:51:45.714696+0000","last_became_active":"2024-06-10T02:49:58.995208+0000","last_became_peered":"2024-06-10T02:49:58.995208+0000","last_unstale":"2024-06-10T02:51:45.714696+0000","last_undegraded":"2024-06-10T02:51:45.714696+0000","last_fullsized":"2024-06-10T02:51:45.714696+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:26:03.513482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":39,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.831456+0000","last_change":"2024-06-10T02:50:02.197090+0000","last_active":"2024-06-10T02:51:45.831456+0000","last_peered":"2024-06-10T02:51:45.831456+0000","last_clean":"2024-06-10T02:51:45.831456+0000","last_became_active":"2024-06-10T02:50:02.196746+0000","last_became_peered":"2024-06-10T02:50:02.196746+0000","last_unstale":"2024-06-10T02:51:45.831456+0000","last_undegraded":"2024-06-10T02:51:45.831456+0000","last_fullsized":"2024-06-10T02:51:45.831456+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:52:09.987978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.012041+0000","last_change":"2024-06-10T02:50:12.234632+0000","last_active":"2024-06-10T02:51:46.012041+0000","last_peered":"2024-06-10T02:51:46.012041+0000","last_clean":"2024-06-10T02:51:46.012041+0000","last_became_active":"2024-06-10T02:50:12.234266+0000","last_became_peered":"2024-06-10T02:50:12.234266+0000","last_unstale":"2024-06-10T02:51:46.012041+0000","last_undegraded":"2024-06-10T02:51:46.012041+0000","last_fullsized":"2024-06-10T02:51:46.012041+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:21:27.348441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"98'3","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.383558+0000","last_change":"2024-06-10T02:49:58.999086+0000","last_active":"2024-06-10T02:51:46.383558+0000","last_peered":"2024-06-10T02:51:46.383558+0000","last_clean":"2024-06-10T02:51:46.383558+0000","last_became_active":"2024-06-10T02:49:58.998767+0000","last_became_peered":"2024-06-10T02:49:58.998767+0000","last_unstale":"2024-06-10T02:51:46.383558+0000","last_undegraded":"2024-06-10T02:51:46.383558+0000","last_fullsized":"2024-06-10T02:51:46.383558+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:28:12.922391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"100'11","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.990813+0000","last_change":"2024-06-10T02:50:02.196641+0000","last_active":"2024-06-10T02:52:42.990813+0000","last_peered":"2024-06-10T02:52:42.990813+0000","last_clean":"2024-06-10T02:52:42.990813+0000","last_became_active":"2024-06-10T02:50:02.196285+0000","last_became_peered":"2024-06-10T02:50:02.196285+0000","last_unstale":"2024-06-10T02:52:42.990813+0000","last_undegraded":"2024-06-10T02:52:42.990813+0000","last_fullsized":"2024-06-10T02:52:42.990813+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T13:55:31.284028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.831864+0000","last_change":"2024-06-10T02:50:12.240988+0000","last_active":"2024-06-10T02:51:45.831864+0000","last_peered":"2024-06-10T02:51:45.831864+0000","last_clean":"2024-06-10T02:51:45.831864+0000","last_became_active":"2024-06-10T02:50:12.240490+0000","last_became_peered":"2024-06-10T02:50:12.240490+0000","last_unstale":"2024-06-10T02:51:45.831864+0000","last_undegraded":"2024-06-10T02:51:45.831864+0000","last_fullsized":"2024-06-10T02:51:45.831864+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:54:36.015221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"100'6","reported_seq":247,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:46.772088+0000","last_change":"2024-06-10T02:49:58.989557+0000","last_active":"2024-06-10T02:52:46.772088+0000","last_peered":"2024-06-10T02:52:46.772088+0000","last_clean":"2024-06-10T02:52:46.772088+0000","last_became_active":"2024-06-10T02:49:58.989101+0000","last_became_peered":"2024-06-10T02:49:58.989101+0000","last_unstale":"2024-06-10T02:52:46.772088+0000","last_undegraded":"2024-06-10T02:52:46.772088+0000","last_fullsized":"2024-06-10T02:52:46.772088+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:27:42.182012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":196,"num_read_kb":196,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":39,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.854882+0000","last_change":"2024-06-10T02:50:02.208752+0000","last_active":"2024-06-10T02:51:45.854882+0000","last_peered":"2024-06-10T02:51:45.854882+0000","last_clean":"2024-06-10T02:51:45.854882+0000","last_became_active":"2024-06-10T02:50:02.208420+0000","last_became_peered":"2024-06-10T02:50:02.208420+0000","last_unstale":"2024-06-10T02:51:45.854882+0000","last_undegraded":"2024-06-10T02:51:45.854882+0000","last_fullsized":"2024-06-10T02:51:45.854882+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:28:09.412317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.382833+0000","last_change":"2024-06-10T02:50:12.236397+0000","last_active":"2024-06-10T02:51:46.382833+0000","last_peered":"2024-06-10T02:51:46.382833+0000","last_clean":"2024-06-10T02:51:46.382833+0000","last_became_active":"2024-06-10T02:50:12.235972+0000","last_became_peered":"2024-06-10T02:50:12.235972+0000","last_unstale":"2024-06-10T02:51:46.382833+0000","last_undegraded":"2024-06-10T02:51:46.382833+0000","last_fullsized":"2024-06-10T02:51:46.382833+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:38:41.723895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"98'4","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.295527+0000","last_change":"2024-06-10T02:49:58.996616+0000","last_active":"2024-06-10T02:51:46.295527+0000","last_peered":"2024-06-10T02:51:46.295527+0000","last_clean":"2024-06-10T02:51:46.295527+0000","last_became_active":"2024-06-10T02:49:58.996289+0000","last_became_peered":"2024-06-10T02:49:58.996289+0000","last_unstale":"2024-06-10T02:51:46.295527+0000","last_undegraded":"2024-06-10T02:51:46.295527+0000","last_fullsized":"2024-06-10T02:51:46.295527+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:22:12.993608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"100'9","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.983255+0000","last_change":"2024-06-10T02:50:02.205230+0000","last_active":"2024-06-10T02:52:42.983255+0000","last_peered":"2024-06-10T02:52:42.983255+0000","last_clean":"2024-06-10T02:52:42.983255+0000","last_became_active":"2024-06-10T02:50:02.204835+0000","last_became_peered":"2024-06-10T02:50:02.204835+0000","last_unstale":"2024-06-10T02:52:42.983255+0000","last_undegraded":"2024-06-10T02:52:42.983255+0000","last_fullsized":"2024-06-10T02:52:42.983255+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:17:35.084921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.383415+0000","last_change":"2024-06-10T02:50:12.231684+0000","last_active":"2024-06-10T02:51:46.383415+0000","last_peered":"2024-06-10T02:51:46.383415+0000","last_clean":"2024-06-10T02:51:46.383415+0000","last_became_active":"2024-06-10T02:50:12.231256+0000","last_became_peered":"2024-06-10T02:50:12.231256+0000","last_unstale":"2024-06-10T02:51:46.383415+0000","last_undegraded":"2024-06-10T02:51:46.383415+0000","last_fullsized":"2024-06-10T02:51:46.383415+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:17:31.878289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"98'5","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.383170+0000","last_change":"2024-06-10T02:49:59.033950+0000","last_active":"2024-06-10T02:51:46.383170+0000","last_peered":"2024-06-10T02:51:46.383170+0000","last_clean":"2024-06-10T02:51:46.383170+0000","last_became_active":"2024-06-10T02:49:59.033688+0000","last_became_peered":"2024-06-10T02:49:59.033688+0000","last_unstale":"2024-06-10T02:51:46.383170+0000","last_undegraded":"2024-06-10T02:51:46.383170+0000","last_fullsized":"2024-06-10T02:51:46.383170+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:32:03.892091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.854465+0000","last_change":"2024-06-10T02:50:44.781378+0000","last_active":"2024-06-10T02:51:45.854465+0000","last_peered":"2024-06-10T02:51:45.854465+0000","last_clean":"2024-06-10T02:51:45.854465+0000","last_became_active":"2024-06-10T02:50:44.780955+0000","last_became_peered":"2024-06-10T02:50:44.780955+0000","last_unstale":"2024-06-10T02:51:45.854465+0000","last_undegraded":"2024-06-10T02:51:45.854465+0000","last_fullsized":"2024-06-10T02:51:45.854465+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:49:13.760381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.385274+0000","last_change":"2024-06-10T02:51:46.384748+0000","last_active":"2024-06-10T02:51:46.385274+0000","last_peered":"2024-06-10T02:51:46.385274+0000","last_clean":"2024-06-10T02:51:46.385274+0000","last_became_active":"2024-06-10T02:51:46.384402+0000","last_became_peered":"2024-06-10T02:51:46.384402+0000","last_unstale":"2024-06-10T02:51:46.385274+0000","last_undegraded":"2024-06-10T02:51:46.385274+0000","last_fullsized":"2024-06-10T02:51:46.385274+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:06:48.056342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.14","version":"100'37","reported_seq":288,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:46.788464+0000","last_change":"2024-06-10T02:49:58.997890+0000","last_active":"2024-06-10T02:52:46.788464+0000","last_peered":"2024-06-10T02:52:46.788464+0000","last_clean":"2024-06-10T02:52:46.788464+0000","last_became_active":"2024-06-10T02:49:58.997549+0000","last_became_peered":"2024-06-10T02:49:58.997549+0000","last_unstale":"2024-06-10T02:52:46.788464+0000","last_undegraded":"2024-06-10T02:52:46.788464+0000","last_fullsized":"2024-06-10T02:52:46.788464+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T14:40:50.907029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62904,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":206,"num_read_kb":165,"num_write":39,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"100'6","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.991053+0000","last_change":"2024-06-10T02:50:02.199707+0000","last_active":"2024-06-10T02:52:42.991053+0000","last_peered":"2024-06-10T02:52:42.991053+0000","last_clean":"2024-06-10T02:52:42.991053+0000","last_became_active":"2024-06-10T02:50:02.198987+0000","last_became_peered":"2024-06-10T02:50:02.198987+0000","last_unstale":"2024-06-10T02:52:42.991053+0000","last_undegraded":"2024-06-10T02:52:42.991053+0000","last_fullsized":"2024-06-10T02:52:42.991053+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:48:03.704590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.383847+0000","last_change":"2024-06-10T02:50:12.239608+0000","last_active":"2024-06-10T02:51:46.383847+0000","last_peered":"2024-06-10T02:51:46.383847+0000","last_clean":"2024-06-10T02:51:46.383847+0000","last_became_active":"2024-06-10T02:50:12.238588+0000","last_became_peered":"2024-06-10T02:50:12.238588+0000","last_unstale":"2024-06-10T02:51:46.383847+0000","last_undegraded":"2024-06-10T02:51:46.383847+0000","last_fullsized":"2024-06-10T02:51:46.383847+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:26:12.718921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"98'9","reported_seq":56,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.721583+0000","last_change":"2024-06-10T02:49:58.997193+0000","last_active":"2024-06-10T02:51:45.721583+0000","last_peered":"2024-06-10T02:51:45.721583+0000","last_clean":"2024-06-10T02:51:45.721583+0000","last_became_active":"2024-06-10T02:49:58.996712+0000","last_became_peered":"2024-06-10T02:49:58.996712+0000","last_unstale":"2024-06-10T02:51:45.721583+0000","last_undegraded":"2024-06-10T02:51:45.721583+0000","last_fullsized":"2024-06-10T02:51:45.721583+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:02:46.894489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":39,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.714811+0000","last_change":"2024-06-10T02:50:02.200544+0000","last_active":"2024-06-10T02:51:45.714811+0000","last_peered":"2024-06-10T02:51:45.714811+0000","last_clean":"2024-06-10T02:51:45.714811+0000","last_became_active":"2024-06-10T02:50:02.200201+0000","last_became_peered":"2024-06-10T02:50:02.200201+0000","last_unstale":"2024-06-10T02:51:45.714811+0000","last_undegraded":"2024-06-10T02:51:45.714811+0000","last_fullsized":"2024-06-10T02:51:45.714811+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T13:12:40.925448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.365243+0000","last_change":"2024-06-10T02:50:12.236317+0000","last_active":"2024-06-10T02:51:46.365243+0000","last_peered":"2024-06-10T02:51:46.365243+0000","last_clean":"2024-06-10T02:51:46.365243+0000","last_became_active":"2024-06-10T02:50:12.235806+0000","last_became_peered":"2024-06-10T02:50:12.235806+0000","last_unstale":"2024-06-10T02:51:46.365243+0000","last_undegraded":"2024-06-10T02:51:46.365243+0000","last_fullsized":"2024-06-10T02:51:46.365243+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:29:21.425987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"98'5","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.383944+0000","last_change":"2024-06-10T02:49:58.994149+0000","last_active":"2024-06-10T02:51:46.383944+0000","last_peered":"2024-06-10T02:51:46.383944+0000","last_clean":"2024-06-10T02:51:46.383944+0000","last_became_active":"2024-06-10T02:49:58.993775+0000","last_became_peered":"2024-06-10T02:49:58.993775+0000","last_unstale":"2024-06-10T02:51:46.383944+0000","last_undegraded":"2024-06-10T02:51:46.383944+0000","last_fullsized":"2024-06-10T02:51:46.383944+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:23:55.553943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":39,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.294608+0000","last_change":"2024-06-10T02:50:02.199979+0000","last_active":"2024-06-10T02:51:46.294608+0000","last_peered":"2024-06-10T02:51:46.294608+0000","last_clean":"2024-06-10T02:51:46.294608+0000","last_became_active":"2024-06-10T02:50:02.199596+0000","last_became_peered":"2024-06-10T02:50:02.199596+0000","last_unstale":"2024-06-10T02:51:46.294608+0000","last_undegraded":"2024-06-10T02:51:46.294608+0000","last_fullsized":"2024-06-10T02:51:46.294608+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:13:58.647460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.383103+0000","last_change":"2024-06-10T02:50:12.235253+0000","last_active":"2024-06-10T02:51:46.383103+0000","last_peered":"2024-06-10T02:51:46.383103+0000","last_clean":"2024-06-10T02:51:46.383103+0000","last_became_active":"2024-06-10T02:50:12.234773+0000","last_became_peered":"2024-06-10T02:50:12.234773+0000","last_unstale":"2024-06-10T02:51:46.383103+0000","last_undegraded":"2024-06-10T02:51:46.383103+0000","last_fullsized":"2024-06-10T02:51:46.383103+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:38:10.971411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"100'7","reported_seq":153,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:44.777792+0000","last_change":"2024-06-10T02:49:58.996912+0000","last_active":"2024-06-10T02:52:44.777792+0000","last_peered":"2024-06-10T02:52:44.777792+0000","last_clean":"2024-06-10T02:52:44.777792+0000","last_became_active":"2024-06-10T02:49:58.996549+0000","last_became_peered":"2024-06-10T02:49:58.996549+0000","last_unstale":"2024-06-10T02:52:44.777792+0000","last_undegraded":"2024-06-10T02:52:44.777792+0000","last_fullsized":"2024-06-10T02:52:44.777792+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:07:52.757676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1119,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":102,"num_read_kb":5,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.295096+0000","last_change":"2024-06-10T02:50:44.741186+0000","last_active":"2024-06-10T02:51:46.295096+0000","last_peered":"2024-06-10T02:51:46.295096+0000","last_clean":"2024-06-10T02:51:46.295096+0000","last_became_active":"2024-06-10T02:50:44.740782+0000","last_became_peered":"2024-06-10T02:50:44.740782+0000","last_unstale":"2024-06-10T02:51:46.295096+0000","last_undegraded":"2024-06-10T02:51:46.295096+0000","last_fullsized":"2024-06-10T02:51:46.295096+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:46:03.863243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.715228+0000","last_change":"2024-06-10T02:50:12.236505+0000","last_active":"2024-06-10T02:51:45.715228+0000","last_peered":"2024-06-10T02:51:45.715228+0000","last_clean":"2024-06-10T02:51:45.715228+0000","last_became_active":"2024-06-10T02:50:12.236112+0000","last_became_peered":"2024-06-10T02:50:12.236112+0000","last_unstale":"2024-06-10T02:51:45.715228+0000","last_undegraded":"2024-06-10T02:51:45.715228+0000","last_fullsized":"2024-06-10T02:51:45.715228+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:13:38.586736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"98'17","reported_seq":63,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.723449+0000","last_change":"2024-06-10T02:49:59.031965+0000","last_active":"2024-06-10T02:51:45.723449+0000","last_peered":"2024-06-10T02:51:45.723449+0000","last_clean":"2024-06-10T02:51:45.723449+0000","last_became_active":"2024-06-10T02:49:59.031495+0000","last_became_peered":"2024-06-10T02:49:59.031495+0000","last_unstale":"2024-06-10T02:51:45.723449+0000","last_undegraded":"2024-06-10T02:51:45.723449+0000","last_fullsized":"2024-06-10T02:51:45.723449+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T14:21:06.610677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"100'6","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:43.002009+0000","last_change":"2024-06-10T02:50:02.209926+0000","last_active":"2024-06-10T02:52:43.002009+0000","last_peered":"2024-06-10T02:52:43.002009+0000","last_clean":"2024-06-10T02:52:43.002009+0000","last_became_active":"2024-06-10T02:50:02.209540+0000","last_became_peered":"2024-06-10T02:50:02.209540+0000","last_unstale":"2024-06-10T02:52:43.002009+0000","last_undegraded":"2024-06-10T02:52:43.002009+0000","last_fullsized":"2024-06-10T02:52:43.002009+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:49:40.497692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.013686+0000","last_change":"2024-06-10T02:51:46.013156+0000","last_active":"2024-06-10T02:51:46.013686+0000","last_peered":"2024-06-10T02:51:46.013686+0000","last_clean":"2024-06-10T02:51:46.013686+0000","last_became_active":"2024-06-10T02:51:46.012769+0000","last_became_peered":"2024-06-10T02:51:46.012769+0000","last_unstale":"2024-06-10T02:51:46.013686+0000","last_undegraded":"2024-06-10T02:51:46.013686+0000","last_fullsized":"2024-06-10T02:51:46.013686+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:44:49.545676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,3,1],"acting":[8,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.f","version":"98'2","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.724214+0000","last_change":"2024-06-10T02:49:58.998372+0000","last_active":"2024-06-10T02:51:45.724214+0000","last_peered":"2024-06-10T02:51:45.724214+0000","last_clean":"2024-06-10T02:51:45.724214+0000","last_became_active":"2024-06-10T02:49:58.998058+0000","last_became_peered":"2024-06-10T02:49:58.998058+0000","last_unstale":"2024-06-10T02:51:45.724214+0000","last_undegraded":"2024-06-10T02:51:45.724214+0000","last_fullsized":"2024-06-10T02:51:45.724214+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:42:50.509851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":39,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.365560+0000","last_change":"2024-06-10T02:50:02.209256+0000","last_active":"2024-06-10T02:51:46.365560+0000","last_peered":"2024-06-10T02:51:46.365560+0000","last_clean":"2024-06-10T02:51:46.365560+0000","last_became_active":"2024-06-10T02:50:02.207727+0000","last_became_peered":"2024-06-10T02:50:02.207727+0000","last_unstale":"2024-06-10T02:51:46.365560+0000","last_undegraded":"2024-06-10T02:51:46.365560+0000","last_fullsized":"2024-06-10T02:51:46.365560+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:23:00.175943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.723666+0000","last_change":"2024-06-10T02:50:12.729438+0000","last_active":"2024-06-10T02:51:45.723666+0000","last_peered":"2024-06-10T02:51:45.723666+0000","last_clean":"2024-06-10T02:51:45.723666+0000","last_became_active":"2024-06-10T02:50:12.728918+0000","last_became_peered":"2024-06-10T02:50:12.728918+0000","last_unstale":"2024-06-10T02:51:45.723666+0000","last_undegraded":"2024-06-10T02:51:45.723666+0000","last_fullsized":"2024-06-10T02:51:45.723666+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:04:48.861763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"98'1","reported_seq":52,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.383319+0000","last_change":"2024-06-10T02:50:44.782241+0000","last_active":"2024-06-10T02:51:46.383319+0000","last_peered":"2024-06-10T02:51:46.383319+0000","last_clean":"2024-06-10T02:51:46.383319+0000","last_became_active":"2024-06-10T02:50:44.781799+0000","last_became_peered":"2024-06-10T02:50:44.781799+0000","last_unstale":"2024-06-10T02:51:46.383319+0000","last_undegraded":"2024-06-10T02:51:46.383319+0000","last_fullsized":"2024-06-10T02:51:46.383319+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:38:35.726258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"100'4","reported_seq":43,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.992975+0000","last_change":"2024-06-10T02:50:02.198888+0000","last_active":"2024-06-10T02:52:42.992975+0000","last_peered":"2024-06-10T02:52:42.992975+0000","last_clean":"2024-06-10T02:52:42.992975+0000","last_became_active":"2024-06-10T02:50:02.198346+0000","last_became_peered":"2024-06-10T02:50:02.198346+0000","last_unstale":"2024-06-10T02:52:42.992975+0000","last_undegraded":"2024-06-10T02:52:42.992975+0000","last_fullsized":"2024-06-10T02:52:42.992975+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:15:42.422898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.715171+0000","last_change":"2024-06-10T02:50:12.232763+0000","last_active":"2024-06-10T02:51:45.715171+0000","last_peered":"2024-06-10T02:51:45.715171+0000","last_clean":"2024-06-10T02:51:45.715171+0000","last_became_active":"2024-06-10T02:50:12.232390+0000","last_became_peered":"2024-06-10T02:50:12.232390+0000","last_unstale":"2024-06-10T02:51:45.715171+0000","last_undegraded":"2024-06-10T02:51:45.715171+0000","last_fullsized":"2024-06-10T02:51:45.715171+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:31:47.973500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"98'6","reported_seq":147,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:41.774327+0000","last_change":"2024-06-10T02:49:58.994308+0000","last_active":"2024-06-10T02:52:41.774327+0000","last_peered":"2024-06-10T02:52:41.774327+0000","last_clean":"2024-06-10T02:52:41.774327+0000","last_became_active":"2024-06-10T02:49:58.993494+0000","last_became_peered":"2024-06-10T02:49:58.993494+0000","last_unstale":"2024-06-10T02:52:41.774327+0000","last_undegraded":"2024-06-10T02:52:41.774327+0000","last_fullsized":"2024-06-10T02:52:41.774327+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:07:28.452276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":978,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"100'7","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.988576+0000","last_change":"2024-06-10T02:50:02.206872+0000","last_active":"2024-06-10T02:52:42.988576+0000","last_peered":"2024-06-10T02:52:42.988576+0000","last_clean":"2024-06-10T02:52:42.988576+0000","last_became_active":"2024-06-10T02:50:02.206307+0000","last_became_peered":"2024-06-10T02:50:02.206307+0000","last_unstale":"2024-06-10T02:52:42.988576+0000","last_undegraded":"2024-06-10T02:52:42.988576+0000","last_fullsized":"2024-06-10T02:52:42.988576+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:01:04.645674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.715107+0000","last_change":"2024-06-10T02:50:12.729392+0000","last_active":"2024-06-10T02:51:45.715107+0000","last_peered":"2024-06-10T02:51:45.715107+0000","last_clean":"2024-06-10T02:51:45.715107+0000","last_became_active":"2024-06-10T02:50:12.728866+0000","last_became_peered":"2024-06-10T02:50:12.728866+0000","last_unstale":"2024-06-10T02:51:45.715107+0000","last_undegraded":"2024-06-10T02:51:45.715107+0000","last_fullsized":"2024-06-10T02:51:45.715107+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:01:35.845184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"100'8","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.994980+0000","last_change":"2024-06-10T02:50:02.202303+0000","last_active":"2024-06-10T02:52:42.994980+0000","last_peered":"2024-06-10T02:52:42.994980+0000","last_clean":"2024-06-10T02:52:42.994980+0000","last_became_active":"2024-06-10T02:50:02.201998+0000","last_became_peered":"2024-06-10T02:50:02.201998+0000","last_unstale":"2024-06-10T02:52:42.994980+0000","last_undegraded":"2024-06-10T02:52:42.994980+0000","last_fullsized":"2024-06-10T02:52:42.994980+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:15:00.005693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"98'5","reported_seq":144,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:40.770814+0000","last_change":"2024-06-10T02:49:59.031513+0000","last_active":"2024-06-10T02:52:40.770814+0000","last_peered":"2024-06-10T02:52:40.770814+0000","last_clean":"2024-06-10T02:52:40.770814+0000","last_became_active":"2024-06-10T02:49:59.030965+0000","last_became_peered":"2024-06-10T02:49:59.030965+0000","last_unstale":"2024-06-10T02:52:40.770814+0000","last_undegraded":"2024-06-10T02:52:40.770814+0000","last_fullsized":"2024-06-10T02:52:40.770814+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:49:02.171578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":94,"num_read_kb":94,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"98'1","reported_seq":30,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.295434+0000","last_change":"2024-06-10T02:50:12.236334+0000","last_active":"2024-06-10T02:51:46.295434+0000","last_peered":"2024-06-10T02:51:46.295434+0000","last_clean":"2024-06-10T02:51:46.295434+0000","last_became_active":"2024-06-10T02:50:12.235820+0000","last_became_peered":"2024-06-10T02:50:12.235820+0000","last_unstale":"2024-06-10T02:51:46.295434+0000","last_undegraded":"2024-06-10T02:51:46.295434+0000","last_fullsized":"2024-06-10T02:51:46.295434+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:16:12.476881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":39,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.831548+0000","last_change":"2024-06-10T02:50:02.205146+0000","last_active":"2024-06-10T02:51:45.831548+0000","last_peered":"2024-06-10T02:51:45.831548+0000","last_clean":"2024-06-10T02:51:45.831548+0000","last_became_active":"2024-06-10T02:50:02.204785+0000","last_became_peered":"2024-06-10T02:50:02.204785+0000","last_unstale":"2024-06-10T02:51:45.831548+0000","last_undegraded":"2024-06-10T02:51:45.831548+0000","last_fullsized":"2024-06-10T02:51:45.831548+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T13:01:02.068989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"98'6","reported_seq":54,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.383508+0000","last_change":"2024-06-10T02:49:58.997654+0000","last_active":"2024-06-10T02:51:46.383508+0000","last_peered":"2024-06-10T02:51:46.383508+0000","last_clean":"2024-06-10T02:51:46.383508+0000","last_became_active":"2024-06-10T02:49:58.997294+0000","last_became_peered":"2024-06-10T02:49:58.997294+0000","last_unstale":"2024-06-10T02:51:46.383508+0000","last_undegraded":"2024-06-10T02:51:46.383508+0000","last_fullsized":"2024-06-10T02:51:46.383508+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:53:14.763198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":366,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.294398+0000","last_change":"2024-06-10T02:48:34.008201+0000","last_active":"2024-06-10T02:51:46.294398+0000","last_peered":"2024-06-10T02:51:46.294398+0000","last_clean":"2024-06-10T02:51:46.294398+0000","last_became_active":"2024-06-10T02:48:34.007633+0000","last_became_peered":"2024-06-10T02:48:34.007633+0000","last_unstale":"2024-06-10T02:51:46.294398+0000","last_undegraded":"2024-06-10T02:51:46.294398+0000","last_fullsized":"2024-06-10T02:51:46.294398+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:45:07.444823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:45:07.444823+0000","last_clean_scrub_stamp":"2024-06-10T02:45:07.444823+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:56:58.159559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":36,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.383275+0000","last_change":"2024-06-10T02:50:44.782130+0000","last_active":"2024-06-10T02:51:46.383275+0000","last_peered":"2024-06-10T02:51:46.383275+0000","last_clean":"2024-06-10T02:51:46.383275+0000","last_became_active":"2024-06-10T02:50:44.781530+0000","last_became_peered":"2024-06-10T02:50:44.781530+0000","last_unstale":"2024-06-10T02:51:46.383275+0000","last_undegraded":"2024-06-10T02:51:46.383275+0000","last_fullsized":"2024-06-10T02:51:46.383275+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:43:23.826207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,11,6,10],"acting":[5,11,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":39,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.714584+0000","last_change":"2024-06-10T02:50:02.213666+0000","last_active":"2024-06-10T02:51:45.714584+0000","last_peered":"2024-06-10T02:51:45.714584+0000","last_clean":"2024-06-10T02:51:45.714584+0000","last_became_active":"2024-06-10T02:50:02.213194+0000","last_became_peered":"2024-06-10T02:50:02.213194+0000","last_unstale":"2024-06-10T02:51:45.714584+0000","last_undegraded":"2024-06-10T02:51:45.714584+0000","last_fullsized":"2024-06-10T02:51:45.714584+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:22:55.905412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"98'4","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.294927+0000","last_change":"2024-06-10T02:49:58.997618+0000","last_active":"2024-06-10T02:51:46.294927+0000","last_peered":"2024-06-10T02:51:46.294927+0000","last_clean":"2024-06-10T02:51:46.294927+0000","last_became_active":"2024-06-10T02:49:58.997304+0000","last_became_peered":"2024-06-10T02:49:58.997304+0000","last_unstale":"2024-06-10T02:51:46.294927+0000","last_undegraded":"2024-06-10T02:51:46.294927+0000","last_fullsized":"2024-06-10T02:51:46.294927+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:48:59.520991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.364962+0000","last_change":"2024-06-10T02:50:12.236357+0000","last_active":"2024-06-10T02:51:46.364962+0000","last_peered":"2024-06-10T02:51:46.364962+0000","last_clean":"2024-06-10T02:51:46.364962+0000","last_became_active":"2024-06-10T02:50:12.235915+0000","last_became_peered":"2024-06-10T02:50:12.235915+0000","last_unstale":"2024-06-10T02:51:46.364962+0000","last_undegraded":"2024-06-10T02:51:46.364962+0000","last_fullsized":"2024-06-10T02:51:46.364962+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:20:06.287907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"98'4","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.724192+0000","last_change":"2024-06-10T02:49:59.031725+0000","last_active":"2024-06-10T02:51:45.724192+0000","last_peered":"2024-06-10T02:51:45.724192+0000","last_clean":"2024-06-10T02:51:45.724192+0000","last_became_active":"2024-06-10T02:49:59.031296+0000","last_became_peered":"2024-06-10T02:49:59.031296+0000","last_unstale":"2024-06-10T02:51:45.724192+0000","last_undegraded":"2024-06-10T02:51:45.724192+0000","last_fullsized":"2024-06-10T02:51:45.724192+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T13:17:58.571696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":39,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.715131+0000","last_change":"2024-06-10T02:50:02.201820+0000","last_active":"2024-06-10T02:51:45.715131+0000","last_peered":"2024-06-10T02:51:45.715131+0000","last_clean":"2024-06-10T02:51:45.715131+0000","last_became_active":"2024-06-10T02:50:02.201478+0000","last_became_peered":"2024-06-10T02:50:02.201478+0000","last_unstale":"2024-06-10T02:51:45.715131+0000","last_undegraded":"2024-06-10T02:51:45.715131+0000","last_fullsized":"2024-06-10T02:51:45.715131+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:57:25.963926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.831431+0000","last_change":"2024-06-10T02:50:12.729418+0000","last_active":"2024-06-10T02:51:45.831431+0000","last_peered":"2024-06-10T02:51:45.831431+0000","last_clean":"2024-06-10T02:51:45.831431+0000","last_became_active":"2024-06-10T02:50:12.728692+0000","last_became_peered":"2024-06-10T02:50:12.728692+0000","last_unstale":"2024-06-10T02:51:45.831431+0000","last_undegraded":"2024-06-10T02:51:45.831431+0000","last_fullsized":"2024-06-10T02:51:45.831431+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:25:00.472396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"100'16","reported_seq":61,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:07.119317+0000","last_change":"2024-06-10T02:49:58.996391+0000","last_active":"2024-06-10T02:52:07.119317+0000","last_peered":"2024-06-10T02:52:07.119317+0000","last_clean":"2024-06-10T02:52:07.119317+0000","last_became_active":"2024-06-10T02:49:58.996033+0000","last_became_peered":"2024-06-10T02:49:58.996033+0000","last_unstale":"2024-06-10T02:52:07.119317+0000","last_undegraded":"2024-06-10T02:52:07.119317+0000","last_fullsized":"2024-06-10T02:52:07.119317+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:03:34.568969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":39,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.832169+0000","last_change":"2024-06-10T02:50:02.197974+0000","last_active":"2024-06-10T02:51:45.832169+0000","last_peered":"2024-06-10T02:51:45.832169+0000","last_clean":"2024-06-10T02:51:45.832169+0000","last_became_active":"2024-06-10T02:50:02.197614+0000","last_became_peered":"2024-06-10T02:50:02.197614+0000","last_unstale":"2024-06-10T02:51:45.832169+0000","last_undegraded":"2024-06-10T02:51:45.832169+0000","last_fullsized":"2024-06-10T02:51:45.832169+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:36:55.575808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.721202+0000","last_change":"2024-06-10T02:50:12.234239+0000","last_active":"2024-06-10T02:51:45.721202+0000","last_peered":"2024-06-10T02:51:45.721202+0000","last_clean":"2024-06-10T02:51:45.721202+0000","last_became_active":"2024-06-10T02:50:12.233768+0000","last_became_peered":"2024-06-10T02:50:12.233768+0000","last_unstale":"2024-06-10T02:51:45.721202+0000","last_undegraded":"2024-06-10T02:51:45.721202+0000","last_fullsized":"2024-06-10T02:51:45.721202+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:51:44.230031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"98'7","reported_seq":53,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.383587+0000","last_change":"2024-06-10T02:49:59.192765+0000","last_active":"2024-06-10T02:51:46.383587+0000","last_peered":"2024-06-10T02:51:46.383587+0000","last_clean":"2024-06-10T02:51:46.383587+0000","last_became_active":"2024-06-10T02:49:59.192446+0000","last_became_peered":"2024-06-10T02:49:59.192446+0000","last_unstale":"2024-06-10T02:51:46.383587+0000","last_undegraded":"2024-06-10T02:51:46.383587+0000","last_fullsized":"2024-06-10T02:51:46.383587+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:53:51.862019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"100'7","reported_seq":52,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.989395+0000","last_change":"2024-06-10T02:50:44.781209+0000","last_active":"2024-06-10T02:52:42.989395+0000","last_peered":"2024-06-10T02:52:42.989395+0000","last_clean":"2024-06-10T02:52:42.989395+0000","last_became_active":"2024-06-10T02:50:44.780811+0000","last_became_peered":"2024-06-10T02:50:44.780811+0000","last_unstale":"2024-06-10T02:52:42.989395+0000","last_undegraded":"2024-06-10T02:52:42.989395+0000","last_fullsized":"2024-06-10T02:52:42.989395+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:12:40.663565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.383517+0000","last_change":"2024-06-10T02:50:12.730269+0000","last_active":"2024-06-10T02:51:46.383517+0000","last_peered":"2024-06-10T02:51:46.383517+0000","last_clean":"2024-06-10T02:51:46.383517+0000","last_became_active":"2024-06-10T02:50:12.729574+0000","last_became_peered":"2024-06-10T02:50:12.729574+0000","last_unstale":"2024-06-10T02:51:46.383517+0000","last_undegraded":"2024-06-10T02:51:46.383517+0000","last_fullsized":"2024-06-10T02:51:46.383517+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:51:16.235284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"98'2","reported_seq":53,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.854226+0000","last_change":"2024-06-10T02:50:44.782421+0000","last_active":"2024-06-10T02:51:45.854226+0000","last_peered":"2024-06-10T02:51:45.854226+0000","last_clean":"2024-06-10T02:51:45.854226+0000","last_became_active":"2024-06-10T02:50:44.781881+0000","last_became_peered":"2024-06-10T02:50:44.781881+0000","last_unstale":"2024-06-10T02:51:45.854226+0000","last_undegraded":"2024-06-10T02:51:45.854226+0000","last_fullsized":"2024-06-10T02:51:45.854226+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T14:41:58.771256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"100'2","reported_seq":41,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.952928+0000","last_change":"2024-06-10T02:50:02.197761+0000","last_active":"2024-06-10T02:52:42.952928+0000","last_peered":"2024-06-10T02:52:42.952928+0000","last_clean":"2024-06-10T02:52:42.952928+0000","last_became_active":"2024-06-10T02:50:02.197395+0000","last_became_peered":"2024-06-10T02:50:02.197395+0000","last_unstale":"2024-06-10T02:52:42.952928+0000","last_undegraded":"2024-06-10T02:52:42.952928+0000","last_fullsized":"2024-06-10T02:52:42.952928+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:46:37.537280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.385820+0000","last_change":"2024-06-10T02:51:46.385129+0000","last_active":"2024-06-10T02:51:46.385820+0000","last_peered":"2024-06-10T02:51:46.385820+0000","last_clean":"2024-06-10T02:51:46.385820+0000","last_became_active":"2024-06-10T02:51:46.384553+0000","last_became_peered":"2024-06-10T02:51:46.384553+0000","last_unstale":"2024-06-10T02:51:46.385820+0000","last_undegraded":"2024-06-10T02:51:46.385820+0000","last_fullsized":"2024-06-10T02:51:46.385820+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:08:11.183679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"98'7","reported_seq":52,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.831750+0000","last_change":"2024-06-10T02:49:58.987863+0000","last_active":"2024-06-10T02:51:45.831750+0000","last_peered":"2024-06-10T02:51:45.831750+0000","last_clean":"2024-06-10T02:51:45.831750+0000","last_became_active":"2024-06-10T02:49:58.987388+0000","last_became_peered":"2024-06-10T02:49:58.987388+0000","last_unstale":"2024-06-10T02:51:45.831750+0000","last_undegraded":"2024-06-10T02:51:45.831750+0000","last_fullsized":"2024-06-10T02:51:45.831750+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:29:20.340301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":39,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.364776+0000","last_change":"2024-06-10T02:50:02.200699+0000","last_active":"2024-06-10T02:51:46.364776+0000","last_peered":"2024-06-10T02:51:46.364776+0000","last_clean":"2024-06-10T02:51:46.364776+0000","last_became_active":"2024-06-10T02:50:02.200278+0000","last_became_peered":"2024-06-10T02:50:02.200278+0000","last_unstale":"2024-06-10T02:51:46.364776+0000","last_undegraded":"2024-06-10T02:51:46.364776+0000","last_fullsized":"2024-06-10T02:51:46.364776+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:26:36.236312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"98'1","reported_seq":30,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.383543+0000","last_change":"2024-06-10T02:50:12.236027+0000","last_active":"2024-06-10T02:51:46.383543+0000","last_peered":"2024-06-10T02:51:46.383543+0000","last_clean":"2024-06-10T02:51:46.383543+0000","last_became_active":"2024-06-10T02:50:12.235629+0000","last_became_peered":"2024-06-10T02:50:12.235629+0000","last_unstale":"2024-06-10T02:51:46.383543+0000","last_undegraded":"2024-06-10T02:51:46.383543+0000","last_fullsized":"2024-06-10T02:51:46.383543+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:58:48.218806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"98'3","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.721449+0000","last_change":"2024-06-10T02:49:58.998496+0000","last_active":"2024-06-10T02:51:45.721449+0000","last_peered":"2024-06-10T02:51:45.721449+0000","last_clean":"2024-06-10T02:51:45.721449+0000","last_became_active":"2024-06-10T02:49:58.997260+0000","last_became_peered":"2024-06-10T02:49:58.997260+0000","last_unstale":"2024-06-10T02:51:45.721449+0000","last_undegraded":"2024-06-10T02:51:45.721449+0000","last_fullsized":"2024-06-10T02:51:45.721449+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T14:22:04.658943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":39,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.831703+0000","last_change":"2024-06-10T02:50:02.201300+0000","last_active":"2024-06-10T02:51:45.831703+0000","last_peered":"2024-06-10T02:51:45.831703+0000","last_clean":"2024-06-10T02:51:45.831703+0000","last_became_active":"2024-06-10T02:50:02.200988+0000","last_became_peered":"2024-06-10T02:50:02.200988+0000","last_unstale":"2024-06-10T02:51:45.831703+0000","last_undegraded":"2024-06-10T02:51:45.831703+0000","last_fullsized":"2024-06-10T02:51:45.831703+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:11:12.962048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.383484+0000","last_change":"2024-06-10T02:50:12.728869+0000","last_active":"2024-06-10T02:51:46.383484+0000","last_peered":"2024-06-10T02:51:46.383484+0000","last_clean":"2024-06-10T02:51:46.383484+0000","last_became_active":"2024-06-10T02:50:12.728189+0000","last_became_peered":"2024-06-10T02:50:12.728189+0000","last_unstale":"2024-06-10T02:51:46.383484+0000","last_undegraded":"2024-06-10T02:51:46.383484+0000","last_fullsized":"2024-06-10T02:51:46.383484+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:08:36.624814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"98'4","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.723488+0000","last_change":"2024-06-10T02:49:59.000460+0000","last_active":"2024-06-10T02:51:45.723488+0000","last_peered":"2024-06-10T02:51:45.723488+0000","last_clean":"2024-06-10T02:51:45.723488+0000","last_became_active":"2024-06-10T02:49:59.000075+0000","last_became_peered":"2024-06-10T02:49:59.000075+0000","last_unstale":"2024-06-10T02:51:45.723488+0000","last_undegraded":"2024-06-10T02:51:45.723488+0000","last_fullsized":"2024-06-10T02:51:45.723488+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:06:27.935745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":39,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.294492+0000","last_change":"2024-06-10T02:50:02.210131+0000","last_active":"2024-06-10T02:51:46.294492+0000","last_peered":"2024-06-10T02:51:46.294492+0000","last_clean":"2024-06-10T02:51:46.294492+0000","last_became_active":"2024-06-10T02:50:02.209781+0000","last_became_peered":"2024-06-10T02:50:02.209781+0000","last_unstale":"2024-06-10T02:51:46.294492+0000","last_undegraded":"2024-06-10T02:51:46.294492+0000","last_fullsized":"2024-06-10T02:51:46.294492+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:27:39.966620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.854407+0000","last_change":"2024-06-10T02:50:12.236017+0000","last_active":"2024-06-10T02:51:45.854407+0000","last_peered":"2024-06-10T02:51:45.854407+0000","last_clean":"2024-06-10T02:51:45.854407+0000","last_became_active":"2024-06-10T02:50:12.235553+0000","last_became_peered":"2024-06-10T02:50:12.235553+0000","last_unstale":"2024-06-10T02:51:45.854407+0000","last_undegraded":"2024-06-10T02:51:45.854407+0000","last_fullsized":"2024-06-10T02:51:45.854407+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:23:55.437994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"100'7","reported_seq":153,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:44.778801+0000","last_change":"2024-06-10T02:49:59.192864+0000","last_active":"2024-06-10T02:52:44.778801+0000","last_peered":"2024-06-10T02:52:44.778801+0000","last_clean":"2024-06-10T02:52:44.778801+0000","last_became_active":"2024-06-10T02:49:59.192440+0000","last_became_peered":"2024-06-10T02:49:59.192440+0000","last_unstale":"2024-06-10T02:52:44.778801+0000","last_undegraded":"2024-06-10T02:52:44.778801+0000","last_fullsized":"2024-06-10T02:52:44.778801+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:36:05.868538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1701,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":101,"num_read_kb":4,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":39,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.294538+0000","last_change":"2024-06-10T02:50:02.205791+0000","last_active":"2024-06-10T02:51:46.294538+0000","last_peered":"2024-06-10T02:51:46.294538+0000","last_clean":"2024-06-10T02:51:46.294538+0000","last_became_active":"2024-06-10T02:50:02.205483+0000","last_became_peered":"2024-06-10T02:50:02.205483+0000","last_unstale":"2024-06-10T02:51:46.294538+0000","last_undegraded":"2024-06-10T02:51:46.294538+0000","last_fullsized":"2024-06-10T02:51:46.294538+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:33:25.875423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.011597+0000","last_change":"2024-06-10T02:50:12.239294+0000","last_active":"2024-06-10T02:51:46.011597+0000","last_peered":"2024-06-10T02:51:46.011597+0000","last_clean":"2024-06-10T02:51:46.011597+0000","last_became_active":"2024-06-10T02:50:12.238778+0000","last_became_peered":"2024-06-10T02:50:12.238778+0000","last_unstale":"2024-06-10T02:51:46.011597+0000","last_undegraded":"2024-06-10T02:51:46.011597+0000","last_fullsized":"2024-06-10T02:51:46.011597+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:26:09.203623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"98'3","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.383589+0000","last_change":"2024-06-10T02:49:58.989680+0000","last_active":"2024-06-10T02:51:46.383589+0000","last_peered":"2024-06-10T02:51:46.383589+0000","last_clean":"2024-06-10T02:51:46.383589+0000","last_became_active":"2024-06-10T02:49:58.989265+0000","last_became_peered":"2024-06-10T02:49:58.989265+0000","last_unstale":"2024-06-10T02:51:46.383589+0000","last_undegraded":"2024-06-10T02:51:46.383589+0000","last_fullsized":"2024-06-10T02:51:46.383589+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:36:34.370177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.295393+0000","last_change":"2024-06-10T02:50:44.743369+0000","last_active":"2024-06-10T02:51:46.295393+0000","last_peered":"2024-06-10T02:51:46.295393+0000","last_clean":"2024-06-10T02:51:46.295393+0000","last_became_active":"2024-06-10T02:50:44.743013+0000","last_became_peered":"2024-06-10T02:50:44.743013+0000","last_unstale":"2024-06-10T02:51:46.295393+0000","last_undegraded":"2024-06-10T02:51:46.295393+0000","last_fullsized":"2024-06-10T02:51:46.295393+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:54:43.728943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.382656+0000","last_change":"2024-06-10T02:50:12.235197+0000","last_active":"2024-06-10T02:51:46.382656+0000","last_peered":"2024-06-10T02:51:46.382656+0000","last_clean":"2024-06-10T02:51:46.382656+0000","last_became_active":"2024-06-10T02:50:12.234791+0000","last_became_peered":"2024-06-10T02:50:12.234791+0000","last_unstale":"2024-06-10T02:51:46.382656+0000","last_undegraded":"2024-06-10T02:51:46.382656+0000","last_fullsized":"2024-06-10T02:51:46.382656+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:09:11.164236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"98'5","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.715164+0000","last_change":"2024-06-10T02:49:58.996759+0000","last_active":"2024-06-10T02:51:45.715164+0000","last_peered":"2024-06-10T02:51:45.715164+0000","last_clean":"2024-06-10T02:51:45.715164+0000","last_became_active":"2024-06-10T02:49:58.996434+0000","last_became_peered":"2024-06-10T02:49:58.996434+0000","last_unstale":"2024-06-10T02:51:45.715164+0000","last_undegraded":"2024-06-10T02:51:45.715164+0000","last_fullsized":"2024-06-10T02:51:45.715164+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:09:35.327797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":39,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.294620+0000","last_change":"2024-06-10T02:50:02.201676+0000","last_active":"2024-06-10T02:51:46.294620+0000","last_peered":"2024-06-10T02:51:46.294620+0000","last_clean":"2024-06-10T02:51:46.294620+0000","last_became_active":"2024-06-10T02:50:02.201335+0000","last_became_peered":"2024-06-10T02:50:02.201335+0000","last_unstale":"2024-06-10T02:51:46.294620+0000","last_undegraded":"2024-06-10T02:51:46.294620+0000","last_fullsized":"2024-06-10T02:51:46.294620+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:49:47.038946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":39,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.364688+0000","last_change":"2024-06-10T02:50:02.203462+0000","last_active":"2024-06-10T02:51:46.364688+0000","last_peered":"2024-06-10T02:51:46.364688+0000","last_clean":"2024-06-10T02:51:46.364688+0000","last_became_active":"2024-06-10T02:50:02.203115+0000","last_became_peered":"2024-06-10T02:50:02.203115+0000","last_unstale":"2024-06-10T02:51:46.364688+0000","last_undegraded":"2024-06-10T02:51:46.364688+0000","last_fullsized":"2024-06-10T02:51:46.364688+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T13:18:12.400596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"100'12","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.997935+0000","last_change":"2024-06-10T02:50:02.205648+0000","last_active":"2024-06-10T02:52:42.997935+0000","last_peered":"2024-06-10T02:52:42.997935+0000","last_clean":"2024-06-10T02:52:42.997935+0000","last_became_active":"2024-06-10T02:50:02.205306+0000","last_became_peered":"2024-06-10T02:50:02.205306+0000","last_unstale":"2024-06-10T02:52:42.997935+0000","last_undegraded":"2024-06-10T02:52:42.997935+0000","last_fullsized":"2024-06-10T02:52:42.997935+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:22:55.020644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"100'8","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.991481+0000","last_change":"2024-06-10T02:50:02.198601+0000","last_active":"2024-06-10T02:52:42.991481+0000","last_peered":"2024-06-10T02:52:42.991481+0000","last_clean":"2024-06-10T02:52:42.991481+0000","last_became_active":"2024-06-10T02:50:02.198103+0000","last_became_peered":"2024-06-10T02:50:02.198103+0000","last_unstale":"2024-06-10T02:52:42.991481+0000","last_undegraded":"2024-06-10T02:52:42.991481+0000","last_fullsized":"2024-06-10T02:52:42.991481+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:56:28.033482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"100'7","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.985332+0000","last_change":"2024-06-10T02:50:02.202638+0000","last_active":"2024-06-10T02:52:42.985332+0000","last_peered":"2024-06-10T02:52:42.985332+0000","last_clean":"2024-06-10T02:52:42.985332+0000","last_became_active":"2024-06-10T02:50:02.202249+0000","last_became_peered":"2024-06-10T02:50:02.202249+0000","last_unstale":"2024-06-10T02:52:42.985332+0000","last_undegraded":"2024-06-10T02:52:42.985332+0000","last_fullsized":"2024-06-10T02:52:42.985332+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:46:29.424164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":39,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.714974+0000","last_change":"2024-06-10T02:50:02.213526+0000","last_active":"2024-06-10T02:51:45.714974+0000","last_peered":"2024-06-10T02:51:45.714974+0000","last_clean":"2024-06-10T02:51:45.714974+0000","last_became_active":"2024-06-10T02:50:02.213226+0000","last_became_peered":"2024-06-10T02:50:02.213226+0000","last_unstale":"2024-06-10T02:51:45.714974+0000","last_undegraded":"2024-06-10T02:51:45.714974+0000","last_fullsized":"2024-06-10T02:51:45.714974+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:01:56.846340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":39,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.364861+0000","last_change":"2024-06-10T02:50:02.206583+0000","last_active":"2024-06-10T02:51:46.364861+0000","last_peered":"2024-06-10T02:51:46.364861+0000","last_clean":"2024-06-10T02:51:46.364861+0000","last_became_active":"2024-06-10T02:50:02.206227+0000","last_became_peered":"2024-06-10T02:50:02.206227+0000","last_unstale":"2024-06-10T02:51:46.364861+0000","last_undegraded":"2024-06-10T02:51:46.364861+0000","last_fullsized":"2024-06-10T02:51:46.364861+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:25:07.246288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"100'15","reported_seq":54,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:43.002970+0000","last_change":"2024-06-10T02:50:02.206348+0000","last_active":"2024-06-10T02:52:43.002970+0000","last_peered":"2024-06-10T02:52:43.002970+0000","last_clean":"2024-06-10T02:52:43.002970+0000","last_became_active":"2024-06-10T02:50:02.206020+0000","last_became_peered":"2024-06-10T02:50:02.206020+0000","last_unstale":"2024-06-10T02:52:43.002970+0000","last_undegraded":"2024-06-10T02:52:43.002970+0000","last_fullsized":"2024-06-10T02:52:43.002970+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:02:26.619113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":39,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.832234+0000","last_change":"2024-06-10T02:50:02.199405+0000","last_active":"2024-06-10T02:51:45.832234+0000","last_peered":"2024-06-10T02:51:45.832234+0000","last_clean":"2024-06-10T02:51:45.832234+0000","last_became_active":"2024-06-10T02:50:02.199049+0000","last_became_peered":"2024-06-10T02:50:02.199049+0000","last_unstale":"2024-06-10T02:51:45.832234+0000","last_undegraded":"2024-06-10T02:51:45.832234+0000","last_fullsized":"2024-06-10T02:51:45.832234+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:33:04.091160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"100'6","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.992703+0000","last_change":"2024-06-10T02:50:02.202094+0000","last_active":"2024-06-10T02:52:42.992703+0000","last_peered":"2024-06-10T02:52:42.992703+0000","last_clean":"2024-06-10T02:52:42.992703+0000","last_became_active":"2024-06-10T02:50:02.201751+0000","last_became_peered":"2024-06-10T02:50:02.201751+0000","last_unstale":"2024-06-10T02:52:42.992703+0000","last_undegraded":"2024-06-10T02:52:42.992703+0000","last_fullsized":"2024-06-10T02:52:42.992703+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:52:53.946326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":39,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.831865+0000","last_change":"2024-06-10T02:50:02.199630+0000","last_active":"2024-06-10T02:51:45.831865+0000","last_peered":"2024-06-10T02:51:45.831865+0000","last_clean":"2024-06-10T02:51:45.831865+0000","last_became_active":"2024-06-10T02:50:02.199320+0000","last_became_peered":"2024-06-10T02:50:02.199320+0000","last_unstale":"2024-06-10T02:51:45.831865+0000","last_undegraded":"2024-06-10T02:51:45.831865+0000","last_fullsized":"2024-06-10T02:51:45.831865+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:17:28.761862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":39,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.854671+0000","last_change":"2024-06-10T02:50:02.201190+0000","last_active":"2024-06-10T02:51:45.854671+0000","last_peered":"2024-06-10T02:51:45.854671+0000","last_clean":"2024-06-10T02:51:45.854671+0000","last_became_active":"2024-06-10T02:50:02.200574+0000","last_became_peered":"2024-06-10T02:50:02.200574+0000","last_unstale":"2024-06-10T02:51:45.854671+0000","last_undegraded":"2024-06-10T02:51:45.854671+0000","last_fullsized":"2024-06-10T02:51:45.854671+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:02:07.636218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":39,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.714912+0000","last_change":"2024-06-10T02:50:02.202374+0000","last_active":"2024-06-10T02:51:45.714912+0000","last_peered":"2024-06-10T02:51:45.714912+0000","last_clean":"2024-06-10T02:51:45.714912+0000","last_became_active":"2024-06-10T02:50:02.201974+0000","last_became_peered":"2024-06-10T02:50:02.201974+0000","last_unstale":"2024-06-10T02:51:45.714912+0000","last_undegraded":"2024-06-10T02:51:45.714912+0000","last_fullsized":"2024-06-10T02:51:45.714912+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:45:34.977926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":39,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.364803+0000","last_change":"2024-06-10T02:50:02.209332+0000","last_active":"2024-06-10T02:51:46.364803+0000","last_peered":"2024-06-10T02:51:46.364803+0000","last_clean":"2024-06-10T02:51:46.364803+0000","last_became_active":"2024-06-10T02:50:02.208175+0000","last_became_peered":"2024-06-10T02:50:02.208175+0000","last_unstale":"2024-06-10T02:51:46.364803+0000","last_undegraded":"2024-06-10T02:51:46.364803+0000","last_fullsized":"2024-06-10T02:51:46.364803+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:44:40.857918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":227,"ondisk_log_size":227,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":284028,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":71309,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1013,"num_read_kb":587,"num_write":262,"num_write_kb":154,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":577536,"data_stored":213255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":225,"ondisk_log_size":225,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351025,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32264,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712888,"statfs":{"total":95995035648,"available":95961997312,"internally_reserved":0,"allocated":1110016,"data_stored":573903,"data_compressed":32447,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612663,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32408,"kb_used_data":1232,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712744,"statfs":{"total":95995035648,"available":95961849856,"internally_reserved":0,"allocated":1261568,"data_stored":1223764,"data_compressed":68564,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1599,"internal_metadata":31914433},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776188,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36448,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708704,"statfs":{"total":95995035648,"available":95957712896,"internally_reserved":0,"allocated":1138688,"data_stored":577539,"data_compressed":32447,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037824,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28112,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717040,"statfs":{"total":95995035648,"available":95966248960,"internally_reserved":0,"allocated":1122304,"data_stored":573149,"data_compressed":32447,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299461,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28180,"kb_used_data":1156,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716972,"statfs":{"total":95995035648,"available":95966179328,"internally_reserved":0,"allocated":1183744,"data_stored":1171097,"data_compressed":68564,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462987,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36428,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708724,"statfs":{"total":95995035648,"available":95957733376,"internally_reserved":0,"allocated":1110016,"data_stored":569337,"data_compressed":32447,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574948,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36496,"kb_used_data":1152,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708656,"statfs":{"total":95995035648,"available":95957663744,"internally_reserved":0,"allocated":1179648,"data_stored":633022,"data_compressed":32447,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738473,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28124,"kb_used_data":1108,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717028,"statfs":{"total":95995035648,"available":95966236672,"internally_reserved":0,"allocated":1134592,"data_stored":570934,"data_compressed":32447,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411424,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28188,"kb_used_data":1172,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716964,"statfs":{"total":95995035648,"available":95966171136,"internally_reserved":0,"allocated":1200128,"data_stored":634163,"data_compressed":32447,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149785,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36432,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708720,"statfs":{"total":95995035648,"available":95957729280,"internally_reserved":0,"allocated":1122304,"data_stored":568655,"data_compressed":32447,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888149,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28116,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717036,"statfs":{"total":95995035648,"available":95966244864,"internally_reserved":0,"allocated":1126400,"data_stored":574224,"data_compressed":32447,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724624,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28180,"kb_used_data":1164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716972,"statfs":{"total":95995035648,"available":95966179328,"internally_reserved":0,"allocated":1191936,"data_stored":1166040,"data_compressed":68564,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34481,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34481,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34481,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36117,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36117,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36117,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36117,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36117,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":64797,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":65938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1903,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3351,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":65171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-06-10T02:52:48.336 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph pg dump --format=json 2024-06-10T02:52:48.591 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:52:48.976 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:52:48 smithi052 ceph-mon[32362]: pgmap v392: 129 pgs: 129 active+clean; 646 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 767 B/s wr, 12 op/s 2024-06-10T02:52:48.976 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:52:48 smithi052 ceph-mon[32362]: from='client.14991 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-10T02:52:49.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:52:48 smithi120 ceph-mon[36634]: pgmap v392: 129 pgs: 129 active+clean; 646 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 767 B/s wr, 12 op/s 2024-06-10T02:52:49.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:52:48 smithi120 ceph-mon[36634]: from='client.14991 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-10T02:52:49.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:52:48 smithi188 ceph-mon[36599]: pgmap v392: 129 pgs: 129 active+clean; 646 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 767 B/s wr, 12 op/s 2024-06-10T02:52:49.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:52:48 smithi188 ceph-mon[36599]: from='client.14991 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-10T02:52:50.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:50.447+0000 7f60b6f12700 1 -- 172.21.15.52:0/2602782134 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f60b0101390 msgr2=0x7f60b0101810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:52:50.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:50.447+0000 7f60b6f12700 1 --2- 172.21.15.52:0/2602782134 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f60b0101390 0x7f60b0101810 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f60a4009990 tx=0x7f60a4009ca0 comp rx=0 tx=0).stop 2024-06-10T02:52:50.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:50.447+0000 7f60b6f12700 1 -- 172.21.15.52:0/2602782134 shutdown_connections 2024-06-10T02:52:50.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:50.447+0000 7f60b6f12700 1 --2- 172.21.15.52:0/2602782134 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f60b0101d50 0x7f60b010e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:50.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:50.447+0000 7f60b6f12700 1 --2- 172.21.15.52:0/2602782134 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f60b0107400 0x7f60b01077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:50.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:50.447+0000 7f60b6f12700 1 --2- 172.21.15.52:0/2602782134 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f60b0101390 0x7f60b0101810 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:50.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:50.447+0000 7f60b6f12700 1 -- 172.21.15.52:0/2602782134 >> 172.21.15.52:0/2602782134 conn(0x7f60b0078340 msgr2=0x7f60b0078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:52:50.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:50.447+0000 7f60b6f12700 1 -- 172.21.15.52:0/2602782134 shutdown_connections 2024-06-10T02:52:50.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:50.447+0000 7f60b6f12700 1 -- 172.21.15.52:0/2602782134 wait complete. 2024-06-10T02:52:50.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:50.448+0000 7f60b6f12700 1 Processor -- start 2024-06-10T02:52:50.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:50.448+0000 7f60b6f12700 1 -- start start 2024-06-10T02:52:50.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:50.448+0000 7f60b6f12700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f60b0101390 0x7f60b0103070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:52:50.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:50.448+0000 7f60b6f12700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f60b0101d50 0x7f60b01035b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:52:50.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:50.448+0000 7f60b6f12700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f60b0107400 0x7f60b0103af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:52:50.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:50.448+0000 7f60b6f12700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f60b01116c0 con 0x7f60b0101d50 2024-06-10T02:52:50.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:50.448+0000 7f60b6f12700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f60b0111540 con 0x7f60b0107400 2024-06-10T02:52:50.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:50.448+0000 7f60b6f12700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f60b0111840 con 0x7f60b0101390 2024-06-10T02:52:50.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:50.448+0000 7f60affff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f60b0101d50 0x7f60b01035b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:52:50.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:50.448+0000 7f60affff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f60b0101d50 0x7f60b01035b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:39942/0 (socket says 172.21.15.52:39942) 2024-06-10T02:52:50.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:50.448+0000 7f60affff700 1 -- 172.21.15.52:0/3501399762 learned_addr learned my addr 172.21.15.52:0/3501399762 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:52:50.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:50.448+0000 7f60b4cae700 1 --2- 172.21.15.52:0/3501399762 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f60b0101390 0x7f60b0103070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:52:50.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:50.448+0000 7f60b54af700 1 --2- 172.21.15.52:0/3501399762 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f60b0107400 0x7f60b0103af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:52:50.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:50.449+0000 7f60affff700 1 -- 172.21.15.52:0/3501399762 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f60b0101390 msgr2=0x7f60b0103070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:52:50.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:50.449+0000 7f60affff700 1 --2- 172.21.15.52:0/3501399762 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f60b0101390 0x7f60b0103070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:50.454 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:50.449+0000 7f60affff700 1 -- 172.21.15.52:0/3501399762 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f60b0107400 msgr2=0x7f60b0103af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:52:50.454 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:50.449+0000 7f60affff700 1 --2- 172.21.15.52:0/3501399762 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f60b0107400 0x7f60b0103af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:50.454 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:50.449+0000 7f60affff700 1 -- 172.21.15.52:0/3501399762 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60b01a7110 con 0x7f60b0101d50 2024-06-10T02:52:50.454 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:50.449+0000 7f60affff700 1 --2- 172.21.15.52:0/3501399762 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f60b0101d50 0x7f60b01035b0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f60a4000c00 tx=0x7f60a4003890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:52:50.454 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:50.449+0000 7f60adffb700 1 -- 172.21.15.52:0/3501399762 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60a4027070 con 0x7f60b0101d50 2024-06-10T02:52:50.454 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:50.449+0000 7f60adffb700 1 -- 172.21.15.52:0/3501399762 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f60a4003e40 con 0x7f60b0101d50 2024-06-10T02:52:50.454 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:50.449+0000 7f60b6f12700 1 -- 172.21.15.52:0/3501399762 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60b01a73a0 con 0x7f60b0101d50 2024-06-10T02:52:50.454 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:50.449+0000 7f60adffb700 1 -- 172.21.15.52:0/3501399762 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60a4022c90 con 0x7f60b0101d50 2024-06-10T02:52:50.454 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:50.449+0000 7f60b6f12700 1 -- 172.21.15.52:0/3501399762 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60b01a7850 con 0x7f60b0101d50 2024-06-10T02:52:50.454 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:50.451+0000 7f60b6f12700 1 -- 172.21.15.52:0/3501399762 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60b0069180 con 0x7f60b0101d50 2024-06-10T02:52:50.454 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:50.452+0000 7f60adffb700 1 -- 172.21.15.52:0/3501399762 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f60a402a070 con 0x7f60b0101d50 2024-06-10T02:52:50.454 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:50.452+0000 7f60adffb700 1 --2- 172.21.15.52:0/3501399762 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f60980a2ca0 0x7f60980a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:52:50.454 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:50.452+0000 7f60b4cae700 1 --2- 172.21.15.52:0/3501399762 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f60980a2ca0 0x7f60980a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:52:50.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:50.452+0000 7f60adffb700 1 -- 172.21.15.52:0/3501399762 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f60a40d0130 con 0x7f60b0101d50 2024-06-10T02:52:50.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:50.453+0000 7f60b4cae700 1 --2- 172.21.15.52:0/3501399762 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f60980a2ca0 0x7f60980a5160 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f60a0007fd0 tx=0x7f60a00079e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:52:50.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:50.457+0000 7f60adffb700 1 -- 172.21.15.52:0/3501399762 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f60a405e850 con 0x7f60b0101d50 2024-06-10T02:52:50.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:50.658+0000 7f60b6f12700 1 -- 172.21.15.52:0/3501399762 --> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f60b01051b0 con 0x7f60980a2ca0 2024-06-10T02:52:50.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:50.668+0000 7f60adffb700 1 -- 172.21.15.52:0/3501399762 <== mgr.14150 v2:172.21.15.52:6800/3214763969 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+341163 (secure 0 0 0) 0x7f60b01051b0 con 0x7f60980a2ca0 2024-06-10T02:52:50.671 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-10T02:52:50.673 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:50.671+0000 7f60b6f12700 1 -- 172.21.15.52:0/3501399762 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f60980a2ca0 msgr2=0x7f60980a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:52:50.673 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:50.671+0000 7f60b6f12700 1 --2- 172.21.15.52:0/3501399762 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f60980a2ca0 0x7f60980a5160 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f60a0007fd0 tx=0x7f60a00079e0 comp rx=0 tx=0).stop 2024-06-10T02:52:50.673 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:50.671+0000 7f60b6f12700 1 -- 172.21.15.52:0/3501399762 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f60b0101d50 msgr2=0x7f60b01035b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:52:50.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:50.671+0000 7f60b6f12700 1 --2- 172.21.15.52:0/3501399762 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f60b0101d50 0x7f60b01035b0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f60a4000c00 tx=0x7f60a4003890 comp rx=0 tx=0).stop 2024-06-10T02:52:50.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:50.671+0000 7f60b6f12700 1 -- 172.21.15.52:0/3501399762 shutdown_connections 2024-06-10T02:52:50.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:50.671+0000 7f60b6f12700 1 --2- 172.21.15.52:0/3501399762 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f60b0107400 0x7f60b0103af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:50.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:50.671+0000 7f60b6f12700 1 --2- 172.21.15.52:0/3501399762 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f60b0101390 0x7f60b0103070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:50.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:50.671+0000 7f60b6f12700 1 --2- 172.21.15.52:0/3501399762 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f60980a2ca0 0x7f60980a5160 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:50.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:50.671+0000 7f60b6f12700 1 --2- 172.21.15.52:0/3501399762 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f60b0101d50 0x7f60b01035b0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:50.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:50.671+0000 7f60b6f12700 1 -- 172.21.15.52:0/3501399762 >> 172.21.15.52:0/3501399762 conn(0x7f60b0078340 msgr2=0x7f60b0100590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:52:50.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:50.672+0000 7f60b6f12700 1 -- 172.21.15.52:0/3501399762 shutdown_connections 2024-06-10T02:52:50.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:50.672+0000 7f60b6f12700 1 -- 172.21.15.52:0/3501399762 wait complete. 2024-06-10T02:52:50.674 INFO:teuthology.orchestra.run.smithi052.stderr:dumped all 2024-06-10T02:52:51.033 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:52:50 smithi052 ceph-mon[32362]: pgmap v393: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 938 B/s wr, 10 op/s 2024-06-10T02:52:51.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:52:50 smithi120 ceph-mon[36634]: pgmap v393: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 938 B/s wr, 10 op/s 2024-06-10T02:52:51.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:52:50 smithi188 ceph-mon[36599]: pgmap v393: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 938 B/s wr, 10 op/s 2024-06-10T02:52:51.453 INFO:teuthology.orchestra.run.smithi052.stdout:{"pg_ready":true,"pg_map":{"version":393,"stamp":"2024-06-10T02:52:49.019902+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":662052,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1150,"num_read_kb":702,"num_write":447,"num_write_kb":3096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1181318,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":866,"ondisk_log_size":866,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":379348,"kb_used_data":13556,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124562476,"statfs":{"total":1151940427776,"available":1151551975424,"internally_reserved":0,"allocated":13881344,"data_stored":8836922,"data_compressed":497715,"data_compressed_allocated":675840,"data_compressed_original":2850816,"omap_allocated":19141,"internal_metadata":374453563},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":22,"apply_latency_ms":22,"commit_latency_ns":22000000,"apply_latency_ns":22000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":813,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":115,"num_read_kb":60,"num_write":13,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003726"},"pg_stats":[{"pgid":"3.3f","version":"100'8","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:43.001859+0000","last_change":"2024-06-10T02:50:02.209146+0000","last_active":"2024-06-10T02:52:43.001859+0000","last_peered":"2024-06-10T02:52:43.001859+0000","last_clean":"2024-06-10T02:52:43.001859+0000","last_became_active":"2024-06-10T02:50:02.208027+0000","last_became_peered":"2024-06-10T02:50:02.208027+0000","last_unstale":"2024-06-10T02:52:43.001859+0000","last_undegraded":"2024-06-10T02:52:43.001859+0000","last_fullsized":"2024-06-10T02:52:43.001859+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:35:34.765849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"100'4","reported_seq":43,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.989740+0000","last_change":"2024-06-10T02:50:02.198767+0000","last_active":"2024-06-10T02:52:42.989740+0000","last_peered":"2024-06-10T02:52:42.989740+0000","last_clean":"2024-06-10T02:52:42.989740+0000","last_became_active":"2024-06-10T02:50:02.198450+0000","last_became_peered":"2024-06-10T02:50:02.198450+0000","last_unstale":"2024-06-10T02:52:42.989740+0000","last_undegraded":"2024-06-10T02:52:42.989740+0000","last_fullsized":"2024-06-10T02:52:42.989740+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:49:56.666083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"100'7","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:43.000326+0000","last_change":"2024-06-10T02:50:02.208885+0000","last_active":"2024-06-10T02:52:43.000326+0000","last_peered":"2024-06-10T02:52:43.000326+0000","last_clean":"2024-06-10T02:52:43.000326+0000","last_became_active":"2024-06-10T02:50:02.206953+0000","last_became_peered":"2024-06-10T02:50:02.206953+0000","last_unstale":"2024-06-10T02:52:43.000326+0000","last_undegraded":"2024-06-10T02:52:43.000326+0000","last_fullsized":"2024-06-10T02:52:43.000326+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:41:23.928939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"100'8","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.993994+0000","last_change":"2024-06-10T02:50:02.208632+0000","last_active":"2024-06-10T02:52:42.993994+0000","last_peered":"2024-06-10T02:52:42.993994+0000","last_clean":"2024-06-10T02:52:42.993994+0000","last_became_active":"2024-06-10T02:50:02.207587+0000","last_became_peered":"2024-06-10T02:50:02.207587+0000","last_unstale":"2024-06-10T02:52:42.993994+0000","last_undegraded":"2024-06-10T02:52:42.993994+0000","last_fullsized":"2024-06-10T02:52:42.993994+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T14:34:36.720170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"100'4","reported_seq":43,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.973606+0000","last_change":"2024-06-10T02:50:02.197966+0000","last_active":"2024-06-10T02:52:42.973606+0000","last_peered":"2024-06-10T02:52:42.973606+0000","last_clean":"2024-06-10T02:52:42.973606+0000","last_became_active":"2024-06-10T02:50:02.197612+0000","last_became_peered":"2024-06-10T02:50:02.197612+0000","last_unstale":"2024-06-10T02:52:42.973606+0000","last_undegraded":"2024-06-10T02:52:42.973606+0000","last_fullsized":"2024-06-10T02:52:42.973606+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:24:53.804613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"100'6","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.987354+0000","last_change":"2024-06-10T02:50:02.201952+0000","last_active":"2024-06-10T02:52:42.987354+0000","last_peered":"2024-06-10T02:52:42.987354+0000","last_clean":"2024-06-10T02:52:42.987354+0000","last_became_active":"2024-06-10T02:50:02.200828+0000","last_became_peered":"2024-06-10T02:50:02.200828+0000","last_unstale":"2024-06-10T02:52:42.987354+0000","last_undegraded":"2024-06-10T02:52:42.987354+0000","last_fullsized":"2024-06-10T02:52:42.987354+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:46:28.518846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"100'5","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.990208+0000","last_change":"2024-06-10T02:50:02.203483+0000","last_active":"2024-06-10T02:52:42.990208+0000","last_peered":"2024-06-10T02:52:42.990208+0000","last_clean":"2024-06-10T02:52:42.990208+0000","last_became_active":"2024-06-10T02:50:02.203153+0000","last_became_peered":"2024-06-10T02:50:02.203153+0000","last_unstale":"2024-06-10T02:52:42.990208+0000","last_undegraded":"2024-06-10T02:52:42.990208+0000","last_fullsized":"2024-06-10T02:52:42.990208+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:29:30.422449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"100'9","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.995630+0000","last_change":"2024-06-10T02:50:02.201686+0000","last_active":"2024-06-10T02:52:42.995630+0000","last_peered":"2024-06-10T02:52:42.995630+0000","last_clean":"2024-06-10T02:52:42.995630+0000","last_became_active":"2024-06-10T02:50:02.200973+0000","last_became_peered":"2024-06-10T02:50:02.200973+0000","last_unstale":"2024-06-10T02:52:42.995630+0000","last_undegraded":"2024-06-10T02:52:42.995630+0000","last_fullsized":"2024-06-10T02:52:42.995630+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:41:13.712675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"100'11","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:43.001422+0000","last_change":"2024-06-10T02:50:02.210497+0000","last_active":"2024-06-10T02:52:43.001422+0000","last_peered":"2024-06-10T02:52:43.001422+0000","last_clean":"2024-06-10T02:52:43.001422+0000","last_became_active":"2024-06-10T02:50:02.209675+0000","last_became_peered":"2024-06-10T02:50:02.209675+0000","last_unstale":"2024-06-10T02:52:43.001422+0000","last_undegraded":"2024-06-10T02:52:43.001422+0000","last_fullsized":"2024-06-10T02:52:43.001422+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:50:29.879669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"100'16","reported_seq":61,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.994975+0000","last_change":"2024-06-10T02:50:44.741139+0000","last_active":"2024-06-10T02:52:42.994975+0000","last_peered":"2024-06-10T02:52:42.994975+0000","last_clean":"2024-06-10T02:52:42.994975+0000","last_became_active":"2024-06-10T02:50:44.740685+0000","last_became_peered":"2024-06-10T02:50:44.740685+0000","last_unstale":"2024-06-10T02:52:42.994975+0000","last_undegraded":"2024-06-10T02:52:42.994975+0000","last_fullsized":"2024-06-10T02:52:42.994975+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:58:59.872732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"100'6","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.999574+0000","last_change":"2024-06-10T02:50:02.206993+0000","last_active":"2024-06-10T02:52:42.999574+0000","last_peered":"2024-06-10T02:52:42.999574+0000","last_clean":"2024-06-10T02:52:42.999574+0000","last_became_active":"2024-06-10T02:50:02.206650+0000","last_became_peered":"2024-06-10T02:50:02.206650+0000","last_unstale":"2024-06-10T02:52:42.999574+0000","last_undegraded":"2024-06-10T02:52:42.999574+0000","last_fullsized":"2024-06-10T02:52:42.999574+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T14:27:49.555350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"100'7","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.979413+0000","last_change":"2024-06-10T02:50:02.200816+0000","last_active":"2024-06-10T02:52:42.979413+0000","last_peered":"2024-06-10T02:52:42.979413+0000","last_clean":"2024-06-10T02:52:42.979413+0000","last_became_active":"2024-06-10T02:50:02.200503+0000","last_became_peered":"2024-06-10T02:50:02.200503+0000","last_unstale":"2024-06-10T02:52:42.979413+0000","last_undegraded":"2024-06-10T02:52:42.979413+0000","last_fullsized":"2024-06-10T02:52:42.979413+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:01:08.326728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"100'10","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.987233+0000","last_change":"2024-06-10T02:50:02.203644+0000","last_active":"2024-06-10T02:52:42.987233+0000","last_peered":"2024-06-10T02:52:42.987233+0000","last_clean":"2024-06-10T02:52:42.987233+0000","last_became_active":"2024-06-10T02:50:02.203100+0000","last_became_peered":"2024-06-10T02:50:02.203100+0000","last_unstale":"2024-06-10T02:52:42.987233+0000","last_undegraded":"2024-06-10T02:52:42.987233+0000","last_fullsized":"2024-06-10T02:52:42.987233+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:01:42.730089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"100'7","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.995068+0000","last_change":"2024-06-10T02:50:02.203330+0000","last_active":"2024-06-10T02:52:42.995068+0000","last_peered":"2024-06-10T02:52:42.995068+0000","last_clean":"2024-06-10T02:52:42.995068+0000","last_became_active":"2024-06-10T02:50:02.202995+0000","last_became_peered":"2024-06-10T02:50:02.202995+0000","last_unstale":"2024-06-10T02:52:42.995068+0000","last_undegraded":"2024-06-10T02:52:42.995068+0000","last_fullsized":"2024-06-10T02:52:42.995068+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:17:23.170591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"100'7","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.986439+0000","last_change":"2024-06-10T02:50:02.204875+0000","last_active":"2024-06-10T02:52:42.986439+0000","last_peered":"2024-06-10T02:52:42.986439+0000","last_clean":"2024-06-10T02:52:42.986439+0000","last_became_active":"2024-06-10T02:50:02.200706+0000","last_became_peered":"2024-06-10T02:50:02.200706+0000","last_unstale":"2024-06-10T02:52:42.986439+0000","last_undegraded":"2024-06-10T02:52:42.986439+0000","last_fullsized":"2024-06-10T02:52:42.986439+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:27:12.282772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"100'9","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.999375+0000","last_change":"2024-06-10T02:50:02.203187+0000","last_active":"2024-06-10T02:52:42.999375+0000","last_peered":"2024-06-10T02:52:42.999375+0000","last_clean":"2024-06-10T02:52:42.999375+0000","last_became_active":"2024-06-10T02:50:02.202034+0000","last_became_peered":"2024-06-10T02:50:02.202034+0000","last_unstale":"2024-06-10T02:52:42.999375+0000","last_undegraded":"2024-06-10T02:52:42.999375+0000","last_fullsized":"2024-06-10T02:52:42.999375+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:59:44.783084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"100'8","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:43.002372+0000","last_change":"2024-06-10T02:50:02.209102+0000","last_active":"2024-06-10T02:52:43.002372+0000","last_peered":"2024-06-10T02:52:43.002372+0000","last_clean":"2024-06-10T02:52:43.002372+0000","last_became_active":"2024-06-10T02:50:02.208730+0000","last_became_peered":"2024-06-10T02:50:02.208730+0000","last_unstale":"2024-06-10T02:52:43.002372+0000","last_undegraded":"2024-06-10T02:52:43.002372+0000","last_fullsized":"2024-06-10T02:52:43.002372+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:08:38.838415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"100'10","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:43.000459+0000","last_change":"2024-06-10T02:50:02.204008+0000","last_active":"2024-06-10T02:52:43.000459+0000","last_peered":"2024-06-10T02:52:43.000459+0000","last_clean":"2024-06-10T02:52:43.000459+0000","last_became_active":"2024-06-10T02:50:02.203655+0000","last_became_peered":"2024-06-10T02:50:02.203655+0000","last_unstale":"2024-06-10T02:52:43.000459+0000","last_undegraded":"2024-06-10T02:52:43.000459+0000","last_fullsized":"2024-06-10T02:52:43.000459+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:32:15.972662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"100'7","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.991723+0000","last_change":"2024-06-10T02:50:02.204917+0000","last_active":"2024-06-10T02:52:42.991723+0000","last_peered":"2024-06-10T02:52:42.991723+0000","last_clean":"2024-06-10T02:52:42.991723+0000","last_became_active":"2024-06-10T02:50:02.204568+0000","last_became_peered":"2024-06-10T02:50:02.204568+0000","last_unstale":"2024-06-10T02:52:42.991723+0000","last_undegraded":"2024-06-10T02:52:42.991723+0000","last_fullsized":"2024-06-10T02:52:42.991723+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:49:46.802665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.721297+0000","last_change":"2024-06-10T02:50:12.240697+0000","last_active":"2024-06-10T02:51:45.721297+0000","last_peered":"2024-06-10T02:51:45.721297+0000","last_clean":"2024-06-10T02:51:45.721297+0000","last_became_active":"2024-06-10T02:50:12.240229+0000","last_became_peered":"2024-06-10T02:50:12.240229+0000","last_unstale":"2024-06-10T02:51:45.721297+0000","last_undegraded":"2024-06-10T02:51:45.721297+0000","last_fullsized":"2024-06-10T02:51:45.721297+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:45:55.071960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"100'14","reported_seq":168,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:43.776213+0000","last_change":"2024-06-10T02:50:44.786783+0000","last_active":"2024-06-10T02:52:43.776213+0000","last_peered":"2024-06-10T02:52:43.776213+0000","last_clean":"2024-06-10T02:52:43.776213+0000","last_became_active":"2024-06-10T02:50:44.781773+0000","last_became_peered":"2024-06-10T02:50:44.781773+0000","last_unstale":"2024-06-10T02:52:43.776213+0000","last_undegraded":"2024-06-10T02:52:43.776213+0000","last_fullsized":"2024-06-10T02:52:43.776213+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:31:06.025854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":97,"num_read_kb":97,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,4],"acting":[9,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"100'5","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.991706+0000","last_change":"2024-06-10T02:50:02.202573+0000","last_active":"2024-06-10T02:52:42.991706+0000","last_peered":"2024-06-10T02:52:42.991706+0000","last_clean":"2024-06-10T02:52:42.991706+0000","last_became_active":"2024-06-10T02:50:02.202241+0000","last_became_peered":"2024-06-10T02:50:02.202241+0000","last_unstale":"2024-06-10T02:52:42.991706+0000","last_undegraded":"2024-06-10T02:52:42.991706+0000","last_fullsized":"2024-06-10T02:52:42.991706+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:31:40.230851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.383047+0000","last_change":"2024-06-10T02:50:12.238372+0000","last_active":"2024-06-10T02:51:46.383047+0000","last_peered":"2024-06-10T02:51:46.383047+0000","last_clean":"2024-06-10T02:51:46.383047+0000","last_became_active":"2024-06-10T02:50:12.237884+0000","last_became_peered":"2024-06-10T02:50:12.237884+0000","last_unstale":"2024-06-10T02:51:46.383047+0000","last_undegraded":"2024-06-10T02:51:46.383047+0000","last_fullsized":"2024-06-10T02:51:46.383047+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:55:31.432655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"98'2","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.832183+0000","last_change":"2024-06-10T02:49:59.031528+0000","last_active":"2024-06-10T02:51:45.832183+0000","last_peered":"2024-06-10T02:51:45.832183+0000","last_clean":"2024-06-10T02:51:45.832183+0000","last_became_active":"2024-06-10T02:49:59.031147+0000","last_became_peered":"2024-06-10T02:49:59.031147+0000","last_unstale":"2024-06-10T02:51:45.832183+0000","last_undegraded":"2024-06-10T02:51:45.832183+0000","last_fullsized":"2024-06-10T02:51:45.832183+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:17:08.534396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"100'6","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:43.000760+0000","last_change":"2024-06-10T02:50:02.202858+0000","last_active":"2024-06-10T02:52:43.000760+0000","last_peered":"2024-06-10T02:52:43.000760+0000","last_clean":"2024-06-10T02:52:43.000760+0000","last_became_active":"2024-06-10T02:50:02.202263+0000","last_became_peered":"2024-06-10T02:50:02.202263+0000","last_unstale":"2024-06-10T02:52:43.000760+0000","last_undegraded":"2024-06-10T02:52:43.000760+0000","last_fullsized":"2024-06-10T02:52:43.000760+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:23:15.382762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.383227+0000","last_change":"2024-06-10T02:50:12.237370+0000","last_active":"2024-06-10T02:51:46.383227+0000","last_peered":"2024-06-10T02:51:46.383227+0000","last_clean":"2024-06-10T02:51:46.383227+0000","last_became_active":"2024-06-10T02:50:12.236803+0000","last_became_peered":"2024-06-10T02:50:12.236803+0000","last_unstale":"2024-06-10T02:51:46.383227+0000","last_undegraded":"2024-06-10T02:51:46.383227+0000","last_fullsized":"2024-06-10T02:51:46.383227+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:06:40.695710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"98'12","reported_seq":58,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.724307+0000","last_change":"2024-06-10T02:49:59.000911+0000","last_active":"2024-06-10T02:51:45.724307+0000","last_peered":"2024-06-10T02:51:45.724307+0000","last_clean":"2024-06-10T02:51:45.724307+0000","last_became_active":"2024-06-10T02:49:59.000571+0000","last_became_peered":"2024-06-10T02:49:59.000571+0000","last_unstale":"2024-06-10T02:51:45.724307+0000","last_undegraded":"2024-06-10T02:51:45.724307+0000","last_fullsized":"2024-06-10T02:51:45.724307+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:49:24.635793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"100'9","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.997165+0000","last_change":"2024-06-10T02:50:02.209614+0000","last_active":"2024-06-10T02:52:42.997165+0000","last_peered":"2024-06-10T02:52:42.997165+0000","last_clean":"2024-06-10T02:52:42.997165+0000","last_became_active":"2024-06-10T02:50:02.209251+0000","last_became_peered":"2024-06-10T02:50:02.209251+0000","last_unstale":"2024-06-10T02:52:42.997165+0000","last_undegraded":"2024-06-10T02:52:42.997165+0000","last_fullsized":"2024-06-10T02:52:42.997165+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:35:32.256081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.383229+0000","last_change":"2024-06-10T02:50:12.237811+0000","last_active":"2024-06-10T02:51:46.383229+0000","last_peered":"2024-06-10T02:51:46.383229+0000","last_clean":"2024-06-10T02:51:46.383229+0000","last_became_active":"2024-06-10T02:50:12.237017+0000","last_became_peered":"2024-06-10T02:50:12.237017+0000","last_unstale":"2024-06-10T02:51:46.383229+0000","last_undegraded":"2024-06-10T02:51:46.383229+0000","last_fullsized":"2024-06-10T02:51:46.383229+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:14:29.818885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"98'5","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.723967+0000","last_change":"2024-06-10T02:49:58.996971+0000","last_active":"2024-06-10T02:51:45.723967+0000","last_peered":"2024-06-10T02:51:45.723967+0000","last_clean":"2024-06-10T02:51:45.723967+0000","last_became_active":"2024-06-10T02:49:58.996601+0000","last_became_peered":"2024-06-10T02:49:58.996601+0000","last_unstale":"2024-06-10T02:51:45.723967+0000","last_undegraded":"2024-06-10T02:51:45.723967+0000","last_fullsized":"2024-06-10T02:51:45.723967+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:26:45.963351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"100'15","reported_seq":60,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:43.002596+0000","last_change":"2024-06-10T02:50:44.745816+0000","last_active":"2024-06-10T02:52:43.002596+0000","last_peered":"2024-06-10T02:52:43.002596+0000","last_clean":"2024-06-10T02:52:43.002596+0000","last_became_active":"2024-06-10T02:50:44.745386+0000","last_became_peered":"2024-06-10T02:50:44.745386+0000","last_unstale":"2024-06-10T02:52:43.002596+0000","last_undegraded":"2024-06-10T02:52:43.002596+0000","last_fullsized":"2024-06-10T02:52:43.002596+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:08:53.977133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.721700+0000","last_change":"2024-06-10T02:50:12.236835+0000","last_active":"2024-06-10T02:51:45.721700+0000","last_peered":"2024-06-10T02:51:45.721700+0000","last_clean":"2024-06-10T02:51:45.721700+0000","last_became_active":"2024-06-10T02:50:12.236309+0000","last_became_peered":"2024-06-10T02:50:12.236309+0000","last_unstale":"2024-06-10T02:51:45.721700+0000","last_undegraded":"2024-06-10T02:51:45.721700+0000","last_fullsized":"2024-06-10T02:51:45.721700+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:09:02.789286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"100'7","reported_seq":153,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:44.769598+0000","last_change":"2024-06-10T02:49:58.997725+0000","last_active":"2024-06-10T02:52:44.769598+0000","last_peered":"2024-06-10T02:52:44.769598+0000","last_clean":"2024-06-10T02:52:44.769598+0000","last_became_active":"2024-06-10T02:49:58.997375+0000","last_became_peered":"2024-06-10T02:49:58.997375+0000","last_unstale":"2024-06-10T02:52:44.769598+0000","last_undegraded":"2024-06-10T02:52:44.769598+0000","last_fullsized":"2024-06-10T02:52:44.769598+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:28:49.714356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1153,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":101,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"100'7","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.995883+0000","last_change":"2024-06-10T02:50:02.201247+0000","last_active":"2024-06-10T02:52:42.995883+0000","last_peered":"2024-06-10T02:52:42.995883+0000","last_clean":"2024-06-10T02:52:42.995883+0000","last_became_active":"2024-06-10T02:50:02.200897+0000","last_became_peered":"2024-06-10T02:50:02.200897+0000","last_unstale":"2024-06-10T02:52:42.995883+0000","last_undegraded":"2024-06-10T02:52:42.995883+0000","last_fullsized":"2024-06-10T02:52:42.995883+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:01:58.166644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.721648+0000","last_change":"2024-06-10T02:50:12.243097+0000","last_active":"2024-06-10T02:51:45.721648+0000","last_peered":"2024-06-10T02:51:45.721648+0000","last_clean":"2024-06-10T02:51:45.721648+0000","last_became_active":"2024-06-10T02:50:12.242501+0000","last_became_peered":"2024-06-10T02:50:12.242501+0000","last_unstale":"2024-06-10T02:51:45.721648+0000","last_undegraded":"2024-06-10T02:51:45.721648+0000","last_fullsized":"2024-06-10T02:51:45.721648+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T14:42:44.220340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"98'7","reported_seq":52,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.720539+0000","last_change":"2024-06-10T02:49:58.996956+0000","last_active":"2024-06-10T02:51:45.720539+0000","last_peered":"2024-06-10T02:51:45.720539+0000","last_clean":"2024-06-10T02:51:45.720539+0000","last_became_active":"2024-06-10T02:49:58.996634+0000","last_became_peered":"2024-06-10T02:49:58.996634+0000","last_unstale":"2024-06-10T02:51:45.720539+0000","last_undegraded":"2024-06-10T02:51:45.720539+0000","last_fullsized":"2024-06-10T02:51:45.720539+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:25:02.364472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"100'8","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.995286+0000","last_change":"2024-06-10T02:50:02.205808+0000","last_active":"2024-06-10T02:52:42.995286+0000","last_peered":"2024-06-10T02:52:42.995286+0000","last_clean":"2024-06-10T02:52:42.995286+0000","last_became_active":"2024-06-10T02:50:02.203558+0000","last_became_peered":"2024-06-10T02:50:02.203558+0000","last_unstale":"2024-06-10T02:52:42.995286+0000","last_undegraded":"2024-06-10T02:52:42.995286+0000","last_fullsized":"2024-06-10T02:52:42.995286+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T13:43:23.315375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.365641+0000","last_change":"2024-06-10T02:50:12.242005+0000","last_active":"2024-06-10T02:51:46.365641+0000","last_peered":"2024-06-10T02:51:46.365641+0000","last_clean":"2024-06-10T02:51:46.365641+0000","last_became_active":"2024-06-10T02:50:12.241295+0000","last_became_peered":"2024-06-10T02:50:12.241295+0000","last_unstale":"2024-06-10T02:51:46.365641+0000","last_undegraded":"2024-06-10T02:51:46.365641+0000","last_fullsized":"2024-06-10T02:51:46.365641+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:32:52.702756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"98'3","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.714696+0000","last_change":"2024-06-10T02:49:58.995579+0000","last_active":"2024-06-10T02:51:45.714696+0000","last_peered":"2024-06-10T02:51:45.714696+0000","last_clean":"2024-06-10T02:51:45.714696+0000","last_became_active":"2024-06-10T02:49:58.995208+0000","last_became_peered":"2024-06-10T02:49:58.995208+0000","last_unstale":"2024-06-10T02:51:45.714696+0000","last_undegraded":"2024-06-10T02:51:45.714696+0000","last_fullsized":"2024-06-10T02:51:45.714696+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:26:03.513482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"100'8","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.993655+0000","last_change":"2024-06-10T02:50:02.197090+0000","last_active":"2024-06-10T02:52:42.993655+0000","last_peered":"2024-06-10T02:52:42.993655+0000","last_clean":"2024-06-10T02:52:42.993655+0000","last_became_active":"2024-06-10T02:50:02.196746+0000","last_became_peered":"2024-06-10T02:50:02.196746+0000","last_unstale":"2024-06-10T02:52:42.993655+0000","last_undegraded":"2024-06-10T02:52:42.993655+0000","last_fullsized":"2024-06-10T02:52:42.993655+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:52:09.987978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.012041+0000","last_change":"2024-06-10T02:50:12.234632+0000","last_active":"2024-06-10T02:51:46.012041+0000","last_peered":"2024-06-10T02:51:46.012041+0000","last_clean":"2024-06-10T02:51:46.012041+0000","last_became_active":"2024-06-10T02:50:12.234266+0000","last_became_peered":"2024-06-10T02:50:12.234266+0000","last_unstale":"2024-06-10T02:51:46.012041+0000","last_undegraded":"2024-06-10T02:51:46.012041+0000","last_fullsized":"2024-06-10T02:51:46.012041+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:21:27.348441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"98'3","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.383558+0000","last_change":"2024-06-10T02:49:58.999086+0000","last_active":"2024-06-10T02:51:46.383558+0000","last_peered":"2024-06-10T02:51:46.383558+0000","last_clean":"2024-06-10T02:51:46.383558+0000","last_became_active":"2024-06-10T02:49:58.998767+0000","last_became_peered":"2024-06-10T02:49:58.998767+0000","last_unstale":"2024-06-10T02:51:46.383558+0000","last_undegraded":"2024-06-10T02:51:46.383558+0000","last_fullsized":"2024-06-10T02:51:46.383558+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:28:12.922391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"100'11","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.990813+0000","last_change":"2024-06-10T02:50:02.196641+0000","last_active":"2024-06-10T02:52:42.990813+0000","last_peered":"2024-06-10T02:52:42.990813+0000","last_clean":"2024-06-10T02:52:42.990813+0000","last_became_active":"2024-06-10T02:50:02.196285+0000","last_became_peered":"2024-06-10T02:50:02.196285+0000","last_unstale":"2024-06-10T02:52:42.990813+0000","last_undegraded":"2024-06-10T02:52:42.990813+0000","last_fullsized":"2024-06-10T02:52:42.990813+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T13:55:31.284028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.831864+0000","last_change":"2024-06-10T02:50:12.240988+0000","last_active":"2024-06-10T02:51:45.831864+0000","last_peered":"2024-06-10T02:51:45.831864+0000","last_clean":"2024-06-10T02:51:45.831864+0000","last_became_active":"2024-06-10T02:50:12.240490+0000","last_became_peered":"2024-06-10T02:50:12.240490+0000","last_unstale":"2024-06-10T02:51:45.831864+0000","last_undegraded":"2024-06-10T02:51:45.831864+0000","last_fullsized":"2024-06-10T02:51:45.831864+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:54:36.015221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"100'6","reported_seq":247,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:46.772088+0000","last_change":"2024-06-10T02:49:58.989557+0000","last_active":"2024-06-10T02:52:46.772088+0000","last_peered":"2024-06-10T02:52:46.772088+0000","last_clean":"2024-06-10T02:52:46.772088+0000","last_became_active":"2024-06-10T02:49:58.989101+0000","last_became_peered":"2024-06-10T02:49:58.989101+0000","last_unstale":"2024-06-10T02:52:46.772088+0000","last_undegraded":"2024-06-10T02:52:46.772088+0000","last_fullsized":"2024-06-10T02:52:46.772088+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:27:42.182012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":196,"num_read_kb":196,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"100'10","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.998692+0000","last_change":"2024-06-10T02:50:02.208752+0000","last_active":"2024-06-10T02:52:42.998692+0000","last_peered":"2024-06-10T02:52:42.998692+0000","last_clean":"2024-06-10T02:52:42.998692+0000","last_became_active":"2024-06-10T02:50:02.208420+0000","last_became_peered":"2024-06-10T02:50:02.208420+0000","last_unstale":"2024-06-10T02:52:42.998692+0000","last_undegraded":"2024-06-10T02:52:42.998692+0000","last_fullsized":"2024-06-10T02:52:42.998692+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:28:09.412317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.382833+0000","last_change":"2024-06-10T02:50:12.236397+0000","last_active":"2024-06-10T02:51:46.382833+0000","last_peered":"2024-06-10T02:51:46.382833+0000","last_clean":"2024-06-10T02:51:46.382833+0000","last_became_active":"2024-06-10T02:50:12.235972+0000","last_became_peered":"2024-06-10T02:50:12.235972+0000","last_unstale":"2024-06-10T02:51:46.382833+0000","last_undegraded":"2024-06-10T02:51:46.382833+0000","last_fullsized":"2024-06-10T02:51:46.382833+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:38:41.723895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"98'4","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.295527+0000","last_change":"2024-06-10T02:49:58.996616+0000","last_active":"2024-06-10T02:51:46.295527+0000","last_peered":"2024-06-10T02:51:46.295527+0000","last_clean":"2024-06-10T02:51:46.295527+0000","last_became_active":"2024-06-10T02:49:58.996289+0000","last_became_peered":"2024-06-10T02:49:58.996289+0000","last_unstale":"2024-06-10T02:51:46.295527+0000","last_undegraded":"2024-06-10T02:51:46.295527+0000","last_fullsized":"2024-06-10T02:51:46.295527+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:22:12.993608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"100'9","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.983255+0000","last_change":"2024-06-10T02:50:02.205230+0000","last_active":"2024-06-10T02:52:42.983255+0000","last_peered":"2024-06-10T02:52:42.983255+0000","last_clean":"2024-06-10T02:52:42.983255+0000","last_became_active":"2024-06-10T02:50:02.204835+0000","last_became_peered":"2024-06-10T02:50:02.204835+0000","last_unstale":"2024-06-10T02:52:42.983255+0000","last_undegraded":"2024-06-10T02:52:42.983255+0000","last_fullsized":"2024-06-10T02:52:42.983255+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:17:35.084921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.383415+0000","last_change":"2024-06-10T02:50:12.231684+0000","last_active":"2024-06-10T02:51:46.383415+0000","last_peered":"2024-06-10T02:51:46.383415+0000","last_clean":"2024-06-10T02:51:46.383415+0000","last_became_active":"2024-06-10T02:50:12.231256+0000","last_became_peered":"2024-06-10T02:50:12.231256+0000","last_unstale":"2024-06-10T02:51:46.383415+0000","last_undegraded":"2024-06-10T02:51:46.383415+0000","last_fullsized":"2024-06-10T02:51:46.383415+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:17:31.878289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"98'5","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.383170+0000","last_change":"2024-06-10T02:49:59.033950+0000","last_active":"2024-06-10T02:51:46.383170+0000","last_peered":"2024-06-10T02:51:46.383170+0000","last_clean":"2024-06-10T02:51:46.383170+0000","last_became_active":"2024-06-10T02:49:59.033688+0000","last_became_peered":"2024-06-10T02:49:59.033688+0000","last_unstale":"2024-06-10T02:51:46.383170+0000","last_undegraded":"2024-06-10T02:51:46.383170+0000","last_fullsized":"2024-06-10T02:51:46.383170+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:32:03.892091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"100'11","reported_seq":56,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.999378+0000","last_change":"2024-06-10T02:50:44.781378+0000","last_active":"2024-06-10T02:52:42.999378+0000","last_peered":"2024-06-10T02:52:42.999378+0000","last_clean":"2024-06-10T02:52:42.999378+0000","last_became_active":"2024-06-10T02:50:44.780955+0000","last_became_peered":"2024-06-10T02:50:44.780955+0000","last_unstale":"2024-06-10T02:52:42.999378+0000","last_undegraded":"2024-06-10T02:52:42.999378+0000","last_fullsized":"2024-06-10T02:52:42.999378+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:49:13.760381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.385274+0000","last_change":"2024-06-10T02:51:46.384748+0000","last_active":"2024-06-10T02:51:46.385274+0000","last_peered":"2024-06-10T02:51:46.385274+0000","last_clean":"2024-06-10T02:51:46.385274+0000","last_became_active":"2024-06-10T02:51:46.384402+0000","last_became_peered":"2024-06-10T02:51:46.384402+0000","last_unstale":"2024-06-10T02:51:46.385274+0000","last_undegraded":"2024-06-10T02:51:46.385274+0000","last_fullsized":"2024-06-10T02:51:46.385274+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:06:48.056342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.14","version":"100'37","reported_seq":288,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:46.788464+0000","last_change":"2024-06-10T02:49:58.997890+0000","last_active":"2024-06-10T02:52:46.788464+0000","last_peered":"2024-06-10T02:52:46.788464+0000","last_clean":"2024-06-10T02:52:46.788464+0000","last_became_active":"2024-06-10T02:49:58.997549+0000","last_became_peered":"2024-06-10T02:49:58.997549+0000","last_unstale":"2024-06-10T02:52:46.788464+0000","last_undegraded":"2024-06-10T02:52:46.788464+0000","last_fullsized":"2024-06-10T02:52:46.788464+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T14:40:50.907029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62904,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":206,"num_read_kb":165,"num_write":39,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"100'6","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.991053+0000","last_change":"2024-06-10T02:50:02.199707+0000","last_active":"2024-06-10T02:52:42.991053+0000","last_peered":"2024-06-10T02:52:42.991053+0000","last_clean":"2024-06-10T02:52:42.991053+0000","last_became_active":"2024-06-10T02:50:02.198987+0000","last_became_peered":"2024-06-10T02:50:02.198987+0000","last_unstale":"2024-06-10T02:52:42.991053+0000","last_undegraded":"2024-06-10T02:52:42.991053+0000","last_fullsized":"2024-06-10T02:52:42.991053+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:48:03.704590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.383847+0000","last_change":"2024-06-10T02:50:12.239608+0000","last_active":"2024-06-10T02:51:46.383847+0000","last_peered":"2024-06-10T02:51:46.383847+0000","last_clean":"2024-06-10T02:51:46.383847+0000","last_became_active":"2024-06-10T02:50:12.238588+0000","last_became_peered":"2024-06-10T02:50:12.238588+0000","last_unstale":"2024-06-10T02:51:46.383847+0000","last_undegraded":"2024-06-10T02:51:46.383847+0000","last_fullsized":"2024-06-10T02:51:46.383847+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:26:12.718921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"98'9","reported_seq":56,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.721583+0000","last_change":"2024-06-10T02:49:58.997193+0000","last_active":"2024-06-10T02:51:45.721583+0000","last_peered":"2024-06-10T02:51:45.721583+0000","last_clean":"2024-06-10T02:51:45.721583+0000","last_became_active":"2024-06-10T02:49:58.996712+0000","last_became_peered":"2024-06-10T02:49:58.996712+0000","last_unstale":"2024-06-10T02:51:45.721583+0000","last_undegraded":"2024-06-10T02:51:45.721583+0000","last_fullsized":"2024-06-10T02:51:45.721583+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:02:46.894489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"100'9","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.994408+0000","last_change":"2024-06-10T02:50:02.200544+0000","last_active":"2024-06-10T02:52:42.994408+0000","last_peered":"2024-06-10T02:52:42.994408+0000","last_clean":"2024-06-10T02:52:42.994408+0000","last_became_active":"2024-06-10T02:50:02.200201+0000","last_became_peered":"2024-06-10T02:50:02.200201+0000","last_unstale":"2024-06-10T02:52:42.994408+0000","last_undegraded":"2024-06-10T02:52:42.994408+0000","last_fullsized":"2024-06-10T02:52:42.994408+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T13:12:40.925448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.365243+0000","last_change":"2024-06-10T02:50:12.236317+0000","last_active":"2024-06-10T02:51:46.365243+0000","last_peered":"2024-06-10T02:51:46.365243+0000","last_clean":"2024-06-10T02:51:46.365243+0000","last_became_active":"2024-06-10T02:50:12.235806+0000","last_became_peered":"2024-06-10T02:50:12.235806+0000","last_unstale":"2024-06-10T02:51:46.365243+0000","last_undegraded":"2024-06-10T02:51:46.365243+0000","last_fullsized":"2024-06-10T02:51:46.365243+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:29:21.425987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"98'5","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.383944+0000","last_change":"2024-06-10T02:49:58.994149+0000","last_active":"2024-06-10T02:51:46.383944+0000","last_peered":"2024-06-10T02:51:46.383944+0000","last_clean":"2024-06-10T02:51:46.383944+0000","last_became_active":"2024-06-10T02:49:58.993775+0000","last_became_peered":"2024-06-10T02:49:58.993775+0000","last_unstale":"2024-06-10T02:51:46.383944+0000","last_undegraded":"2024-06-10T02:51:46.383944+0000","last_fullsized":"2024-06-10T02:51:46.383944+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:23:55.553943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"100'15","reported_seq":54,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.995273+0000","last_change":"2024-06-10T02:50:02.199979+0000","last_active":"2024-06-10T02:52:42.995273+0000","last_peered":"2024-06-10T02:52:42.995273+0000","last_clean":"2024-06-10T02:52:42.995273+0000","last_became_active":"2024-06-10T02:50:02.199596+0000","last_became_peered":"2024-06-10T02:50:02.199596+0000","last_unstale":"2024-06-10T02:52:42.995273+0000","last_undegraded":"2024-06-10T02:52:42.995273+0000","last_fullsized":"2024-06-10T02:52:42.995273+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:13:58.647460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.383103+0000","last_change":"2024-06-10T02:50:12.235253+0000","last_active":"2024-06-10T02:51:46.383103+0000","last_peered":"2024-06-10T02:51:46.383103+0000","last_clean":"2024-06-10T02:51:46.383103+0000","last_became_active":"2024-06-10T02:50:12.234773+0000","last_became_peered":"2024-06-10T02:50:12.234773+0000","last_unstale":"2024-06-10T02:51:46.383103+0000","last_undegraded":"2024-06-10T02:51:46.383103+0000","last_fullsized":"2024-06-10T02:51:46.383103+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:38:10.971411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"100'7","reported_seq":153,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:44.777792+0000","last_change":"2024-06-10T02:49:58.996912+0000","last_active":"2024-06-10T02:52:44.777792+0000","last_peered":"2024-06-10T02:52:44.777792+0000","last_clean":"2024-06-10T02:52:44.777792+0000","last_became_active":"2024-06-10T02:49:58.996549+0000","last_became_peered":"2024-06-10T02:49:58.996549+0000","last_unstale":"2024-06-10T02:52:44.777792+0000","last_undegraded":"2024-06-10T02:52:44.777792+0000","last_fullsized":"2024-06-10T02:52:44.777792+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:07:52.757676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1119,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":102,"num_read_kb":5,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"100'6","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.995546+0000","last_change":"2024-06-10T02:50:44.741186+0000","last_active":"2024-06-10T02:52:42.995546+0000","last_peered":"2024-06-10T02:52:42.995546+0000","last_clean":"2024-06-10T02:52:42.995546+0000","last_became_active":"2024-06-10T02:50:44.740782+0000","last_became_peered":"2024-06-10T02:50:44.740782+0000","last_unstale":"2024-06-10T02:52:42.995546+0000","last_undegraded":"2024-06-10T02:52:42.995546+0000","last_fullsized":"2024-06-10T02:52:42.995546+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:46:03.863243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.715228+0000","last_change":"2024-06-10T02:50:12.236505+0000","last_active":"2024-06-10T02:51:45.715228+0000","last_peered":"2024-06-10T02:51:45.715228+0000","last_clean":"2024-06-10T02:51:45.715228+0000","last_became_active":"2024-06-10T02:50:12.236112+0000","last_became_peered":"2024-06-10T02:50:12.236112+0000","last_unstale":"2024-06-10T02:51:45.715228+0000","last_undegraded":"2024-06-10T02:51:45.715228+0000","last_fullsized":"2024-06-10T02:51:45.715228+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:13:38.586736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"98'17","reported_seq":63,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.723449+0000","last_change":"2024-06-10T02:49:59.031965+0000","last_active":"2024-06-10T02:51:45.723449+0000","last_peered":"2024-06-10T02:51:45.723449+0000","last_clean":"2024-06-10T02:51:45.723449+0000","last_became_active":"2024-06-10T02:49:59.031495+0000","last_became_peered":"2024-06-10T02:49:59.031495+0000","last_unstale":"2024-06-10T02:51:45.723449+0000","last_undegraded":"2024-06-10T02:51:45.723449+0000","last_fullsized":"2024-06-10T02:51:45.723449+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T14:21:06.610677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"100'6","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:43.002009+0000","last_change":"2024-06-10T02:50:02.209926+0000","last_active":"2024-06-10T02:52:43.002009+0000","last_peered":"2024-06-10T02:52:43.002009+0000","last_clean":"2024-06-10T02:52:43.002009+0000","last_became_active":"2024-06-10T02:50:02.209540+0000","last_became_peered":"2024-06-10T02:50:02.209540+0000","last_unstale":"2024-06-10T02:52:43.002009+0000","last_undegraded":"2024-06-10T02:52:43.002009+0000","last_fullsized":"2024-06-10T02:52:43.002009+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:49:40.497692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.013686+0000","last_change":"2024-06-10T02:51:46.013156+0000","last_active":"2024-06-10T02:51:46.013686+0000","last_peered":"2024-06-10T02:51:46.013686+0000","last_clean":"2024-06-10T02:51:46.013686+0000","last_became_active":"2024-06-10T02:51:46.012769+0000","last_became_peered":"2024-06-10T02:51:46.012769+0000","last_unstale":"2024-06-10T02:51:46.013686+0000","last_undegraded":"2024-06-10T02:51:46.013686+0000","last_fullsized":"2024-06-10T02:51:46.013686+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:44:49.545676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,3,1],"acting":[8,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.f","version":"98'2","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.724214+0000","last_change":"2024-06-10T02:49:58.998372+0000","last_active":"2024-06-10T02:51:45.724214+0000","last_peered":"2024-06-10T02:51:45.724214+0000","last_clean":"2024-06-10T02:51:45.724214+0000","last_became_active":"2024-06-10T02:49:58.998058+0000","last_became_peered":"2024-06-10T02:49:58.998058+0000","last_unstale":"2024-06-10T02:51:45.724214+0000","last_undegraded":"2024-06-10T02:51:45.724214+0000","last_fullsized":"2024-06-10T02:51:45.724214+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:42:50.509851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"100'5","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.993794+0000","last_change":"2024-06-10T02:50:02.209256+0000","last_active":"2024-06-10T02:52:42.993794+0000","last_peered":"2024-06-10T02:52:42.993794+0000","last_clean":"2024-06-10T02:52:42.993794+0000","last_became_active":"2024-06-10T02:50:02.207727+0000","last_became_peered":"2024-06-10T02:50:02.207727+0000","last_unstale":"2024-06-10T02:52:42.993794+0000","last_undegraded":"2024-06-10T02:52:42.993794+0000","last_fullsized":"2024-06-10T02:52:42.993794+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:23:00.175943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.723666+0000","last_change":"2024-06-10T02:50:12.729438+0000","last_active":"2024-06-10T02:51:45.723666+0000","last_peered":"2024-06-10T02:51:45.723666+0000","last_clean":"2024-06-10T02:51:45.723666+0000","last_became_active":"2024-06-10T02:50:12.728918+0000","last_became_peered":"2024-06-10T02:50:12.728918+0000","last_unstale":"2024-06-10T02:51:45.723666+0000","last_undegraded":"2024-06-10T02:51:45.723666+0000","last_fullsized":"2024-06-10T02:51:45.723666+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:04:48.861763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"98'1","reported_seq":52,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.383319+0000","last_change":"2024-06-10T02:50:44.782241+0000","last_active":"2024-06-10T02:51:46.383319+0000","last_peered":"2024-06-10T02:51:46.383319+0000","last_clean":"2024-06-10T02:51:46.383319+0000","last_became_active":"2024-06-10T02:50:44.781799+0000","last_became_peered":"2024-06-10T02:50:44.781799+0000","last_unstale":"2024-06-10T02:51:46.383319+0000","last_undegraded":"2024-06-10T02:51:46.383319+0000","last_fullsized":"2024-06-10T02:51:46.383319+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:38:35.726258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"100'4","reported_seq":43,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.992975+0000","last_change":"2024-06-10T02:50:02.198888+0000","last_active":"2024-06-10T02:52:42.992975+0000","last_peered":"2024-06-10T02:52:42.992975+0000","last_clean":"2024-06-10T02:52:42.992975+0000","last_became_active":"2024-06-10T02:50:02.198346+0000","last_became_peered":"2024-06-10T02:50:02.198346+0000","last_unstale":"2024-06-10T02:52:42.992975+0000","last_undegraded":"2024-06-10T02:52:42.992975+0000","last_fullsized":"2024-06-10T02:52:42.992975+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:15:42.422898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.715171+0000","last_change":"2024-06-10T02:50:12.232763+0000","last_active":"2024-06-10T02:51:45.715171+0000","last_peered":"2024-06-10T02:51:45.715171+0000","last_clean":"2024-06-10T02:51:45.715171+0000","last_became_active":"2024-06-10T02:50:12.232390+0000","last_became_peered":"2024-06-10T02:50:12.232390+0000","last_unstale":"2024-06-10T02:51:45.715171+0000","last_undegraded":"2024-06-10T02:51:45.715171+0000","last_fullsized":"2024-06-10T02:51:45.715171+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:31:47.973500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"100'7","reported_seq":154,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:46.780890+0000","last_change":"2024-06-10T02:49:58.994308+0000","last_active":"2024-06-10T02:52:46.780890+0000","last_peered":"2024-06-10T02:52:46.780890+0000","last_clean":"2024-06-10T02:52:46.780890+0000","last_became_active":"2024-06-10T02:49:58.993494+0000","last_became_peered":"2024-06-10T02:49:58.993494+0000","last_unstale":"2024-06-10T02:52:46.780890+0000","last_undegraded":"2024-06-10T02:52:46.780890+0000","last_fullsized":"2024-06-10T02:52:46.780890+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:07:28.452276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1119,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":102,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"100'7","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.988576+0000","last_change":"2024-06-10T02:50:02.206872+0000","last_active":"2024-06-10T02:52:42.988576+0000","last_peered":"2024-06-10T02:52:42.988576+0000","last_clean":"2024-06-10T02:52:42.988576+0000","last_became_active":"2024-06-10T02:50:02.206307+0000","last_became_peered":"2024-06-10T02:50:02.206307+0000","last_unstale":"2024-06-10T02:52:42.988576+0000","last_undegraded":"2024-06-10T02:52:42.988576+0000","last_fullsized":"2024-06-10T02:52:42.988576+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:01:04.645674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.715107+0000","last_change":"2024-06-10T02:50:12.729392+0000","last_active":"2024-06-10T02:51:45.715107+0000","last_peered":"2024-06-10T02:51:45.715107+0000","last_clean":"2024-06-10T02:51:45.715107+0000","last_became_active":"2024-06-10T02:50:12.728866+0000","last_became_peered":"2024-06-10T02:50:12.728866+0000","last_unstale":"2024-06-10T02:51:45.715107+0000","last_undegraded":"2024-06-10T02:51:45.715107+0000","last_fullsized":"2024-06-10T02:51:45.715107+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:01:35.845184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"100'8","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.994980+0000","last_change":"2024-06-10T02:50:02.202303+0000","last_active":"2024-06-10T02:52:42.994980+0000","last_peered":"2024-06-10T02:52:42.994980+0000","last_clean":"2024-06-10T02:52:42.994980+0000","last_became_active":"2024-06-10T02:50:02.201998+0000","last_became_peered":"2024-06-10T02:50:02.201998+0000","last_unstale":"2024-06-10T02:52:42.994980+0000","last_undegraded":"2024-06-10T02:52:42.994980+0000","last_fullsized":"2024-06-10T02:52:42.994980+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:15:00.005693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"100'6","reported_seq":150,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:45.778945+0000","last_change":"2024-06-10T02:49:59.031513+0000","last_active":"2024-06-10T02:52:45.778945+0000","last_peered":"2024-06-10T02:52:45.778945+0000","last_clean":"2024-06-10T02:52:45.778945+0000","last_became_active":"2024-06-10T02:49:59.030965+0000","last_became_peered":"2024-06-10T02:49:59.030965+0000","last_unstale":"2024-06-10T02:52:45.778945+0000","last_undegraded":"2024-06-10T02:52:45.778945+0000","last_fullsized":"2024-06-10T02:52:45.778945+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:49:02.171578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":99,"num_read_kb":99,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"98'1","reported_seq":30,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.295434+0000","last_change":"2024-06-10T02:50:12.236334+0000","last_active":"2024-06-10T02:51:46.295434+0000","last_peered":"2024-06-10T02:51:46.295434+0000","last_clean":"2024-06-10T02:51:46.295434+0000","last_became_active":"2024-06-10T02:50:12.235820+0000","last_became_peered":"2024-06-10T02:50:12.235820+0000","last_unstale":"2024-06-10T02:51:46.295434+0000","last_undegraded":"2024-06-10T02:51:46.295434+0000","last_fullsized":"2024-06-10T02:51:46.295434+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:16:12.476881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"100'8","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:43.001236+0000","last_change":"2024-06-10T02:50:02.205146+0000","last_active":"2024-06-10T02:52:43.001236+0000","last_peered":"2024-06-10T02:52:43.001236+0000","last_clean":"2024-06-10T02:52:43.001236+0000","last_became_active":"2024-06-10T02:50:02.204785+0000","last_became_peered":"2024-06-10T02:50:02.204785+0000","last_unstale":"2024-06-10T02:52:43.001236+0000","last_undegraded":"2024-06-10T02:52:43.001236+0000","last_fullsized":"2024-06-10T02:52:43.001236+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T13:01:02.068989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"98'6","reported_seq":54,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.383508+0000","last_change":"2024-06-10T02:49:58.997654+0000","last_active":"2024-06-10T02:51:46.383508+0000","last_peered":"2024-06-10T02:51:46.383508+0000","last_clean":"2024-06-10T02:51:46.383508+0000","last_became_active":"2024-06-10T02:49:58.997294+0000","last_became_peered":"2024-06-10T02:49:58.997294+0000","last_unstale":"2024-06-10T02:51:46.383508+0000","last_undegraded":"2024-06-10T02:51:46.383508+0000","last_fullsized":"2024-06-10T02:51:46.383508+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:53:14.763198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":366,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.294398+0000","last_change":"2024-06-10T02:48:34.008201+0000","last_active":"2024-06-10T02:51:46.294398+0000","last_peered":"2024-06-10T02:51:46.294398+0000","last_clean":"2024-06-10T02:51:46.294398+0000","last_became_active":"2024-06-10T02:48:34.007633+0000","last_became_peered":"2024-06-10T02:48:34.007633+0000","last_unstale":"2024-06-10T02:51:46.294398+0000","last_undegraded":"2024-06-10T02:51:46.294398+0000","last_fullsized":"2024-06-10T02:51:46.294398+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:45:07.444823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:45:07.444823+0000","last_clean_scrub_stamp":"2024-06-10T02:45:07.444823+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:56:58.159559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":36,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.383275+0000","last_change":"2024-06-10T02:50:44.782130+0000","last_active":"2024-06-10T02:51:46.383275+0000","last_peered":"2024-06-10T02:51:46.383275+0000","last_clean":"2024-06-10T02:51:46.383275+0000","last_became_active":"2024-06-10T02:50:44.781530+0000","last_became_peered":"2024-06-10T02:50:44.781530+0000","last_unstale":"2024-06-10T02:51:46.383275+0000","last_undegraded":"2024-06-10T02:51:46.383275+0000","last_fullsized":"2024-06-10T02:51:46.383275+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:43:23.826207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,11,6,10],"acting":[5,11,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"100'5","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.994022+0000","last_change":"2024-06-10T02:50:02.213666+0000","last_active":"2024-06-10T02:52:42.994022+0000","last_peered":"2024-06-10T02:52:42.994022+0000","last_clean":"2024-06-10T02:52:42.994022+0000","last_became_active":"2024-06-10T02:50:02.213194+0000","last_became_peered":"2024-06-10T02:50:02.213194+0000","last_unstale":"2024-06-10T02:52:42.994022+0000","last_undegraded":"2024-06-10T02:52:42.994022+0000","last_fullsized":"2024-06-10T02:52:42.994022+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:22:55.905412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"98'4","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.294927+0000","last_change":"2024-06-10T02:49:58.997618+0000","last_active":"2024-06-10T02:51:46.294927+0000","last_peered":"2024-06-10T02:51:46.294927+0000","last_clean":"2024-06-10T02:51:46.294927+0000","last_became_active":"2024-06-10T02:49:58.997304+0000","last_became_peered":"2024-06-10T02:49:58.997304+0000","last_unstale":"2024-06-10T02:51:46.294927+0000","last_undegraded":"2024-06-10T02:51:46.294927+0000","last_fullsized":"2024-06-10T02:51:46.294927+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:48:59.520991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.364962+0000","last_change":"2024-06-10T02:50:12.236357+0000","last_active":"2024-06-10T02:51:46.364962+0000","last_peered":"2024-06-10T02:51:46.364962+0000","last_clean":"2024-06-10T02:51:46.364962+0000","last_became_active":"2024-06-10T02:50:12.235915+0000","last_became_peered":"2024-06-10T02:50:12.235915+0000","last_unstale":"2024-06-10T02:51:46.364962+0000","last_undegraded":"2024-06-10T02:51:46.364962+0000","last_fullsized":"2024-06-10T02:51:46.364962+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:20:06.287907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"98'4","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.724192+0000","last_change":"2024-06-10T02:49:59.031725+0000","last_active":"2024-06-10T02:51:45.724192+0000","last_peered":"2024-06-10T02:51:45.724192+0000","last_clean":"2024-06-10T02:51:45.724192+0000","last_became_active":"2024-06-10T02:49:59.031296+0000","last_became_peered":"2024-06-10T02:49:59.031296+0000","last_unstale":"2024-06-10T02:51:45.724192+0000","last_undegraded":"2024-06-10T02:51:45.724192+0000","last_fullsized":"2024-06-10T02:51:45.724192+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T13:17:58.571696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"100'9","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.978128+0000","last_change":"2024-06-10T02:50:02.201820+0000","last_active":"2024-06-10T02:52:42.978128+0000","last_peered":"2024-06-10T02:52:42.978128+0000","last_clean":"2024-06-10T02:52:42.978128+0000","last_became_active":"2024-06-10T02:50:02.201478+0000","last_became_peered":"2024-06-10T02:50:02.201478+0000","last_unstale":"2024-06-10T02:52:42.978128+0000","last_undegraded":"2024-06-10T02:52:42.978128+0000","last_fullsized":"2024-06-10T02:52:42.978128+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:57:25.963926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.831431+0000","last_change":"2024-06-10T02:50:12.729418+0000","last_active":"2024-06-10T02:51:45.831431+0000","last_peered":"2024-06-10T02:51:45.831431+0000","last_clean":"2024-06-10T02:51:45.831431+0000","last_became_active":"2024-06-10T02:50:12.728692+0000","last_became_peered":"2024-06-10T02:50:12.728692+0000","last_unstale":"2024-06-10T02:51:45.831431+0000","last_undegraded":"2024-06-10T02:51:45.831431+0000","last_fullsized":"2024-06-10T02:51:45.831431+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:25:00.472396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"100'16","reported_seq":61,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:07.119317+0000","last_change":"2024-06-10T02:49:58.996391+0000","last_active":"2024-06-10T02:52:07.119317+0000","last_peered":"2024-06-10T02:52:07.119317+0000","last_clean":"2024-06-10T02:52:07.119317+0000","last_became_active":"2024-06-10T02:49:58.996033+0000","last_became_peered":"2024-06-10T02:49:58.996033+0000","last_unstale":"2024-06-10T02:52:07.119317+0000","last_undegraded":"2024-06-10T02:52:07.119317+0000","last_fullsized":"2024-06-10T02:52:07.119317+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:03:34.568969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"100'5","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.993669+0000","last_change":"2024-06-10T02:50:02.197974+0000","last_active":"2024-06-10T02:52:42.993669+0000","last_peered":"2024-06-10T02:52:42.993669+0000","last_clean":"2024-06-10T02:52:42.993669+0000","last_became_active":"2024-06-10T02:50:02.197614+0000","last_became_peered":"2024-06-10T02:50:02.197614+0000","last_unstale":"2024-06-10T02:52:42.993669+0000","last_undegraded":"2024-06-10T02:52:42.993669+0000","last_fullsized":"2024-06-10T02:52:42.993669+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:36:55.575808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.721202+0000","last_change":"2024-06-10T02:50:12.234239+0000","last_active":"2024-06-10T02:51:45.721202+0000","last_peered":"2024-06-10T02:51:45.721202+0000","last_clean":"2024-06-10T02:51:45.721202+0000","last_became_active":"2024-06-10T02:50:12.233768+0000","last_became_peered":"2024-06-10T02:50:12.233768+0000","last_unstale":"2024-06-10T02:51:45.721202+0000","last_undegraded":"2024-06-10T02:51:45.721202+0000","last_fullsized":"2024-06-10T02:51:45.721202+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:51:44.230031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"98'7","reported_seq":53,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.383587+0000","last_change":"2024-06-10T02:49:59.192765+0000","last_active":"2024-06-10T02:51:46.383587+0000","last_peered":"2024-06-10T02:51:46.383587+0000","last_clean":"2024-06-10T02:51:46.383587+0000","last_became_active":"2024-06-10T02:49:59.192446+0000","last_became_peered":"2024-06-10T02:49:59.192446+0000","last_unstale":"2024-06-10T02:51:46.383587+0000","last_undegraded":"2024-06-10T02:51:46.383587+0000","last_fullsized":"2024-06-10T02:51:46.383587+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:53:51.862019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"100'7","reported_seq":52,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.989395+0000","last_change":"2024-06-10T02:50:44.781209+0000","last_active":"2024-06-10T02:52:42.989395+0000","last_peered":"2024-06-10T02:52:42.989395+0000","last_clean":"2024-06-10T02:52:42.989395+0000","last_became_active":"2024-06-10T02:50:44.780811+0000","last_became_peered":"2024-06-10T02:50:44.780811+0000","last_unstale":"2024-06-10T02:52:42.989395+0000","last_undegraded":"2024-06-10T02:52:42.989395+0000","last_fullsized":"2024-06-10T02:52:42.989395+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:12:40.663565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.383517+0000","last_change":"2024-06-10T02:50:12.730269+0000","last_active":"2024-06-10T02:51:46.383517+0000","last_peered":"2024-06-10T02:51:46.383517+0000","last_clean":"2024-06-10T02:51:46.383517+0000","last_became_active":"2024-06-10T02:50:12.729574+0000","last_became_peered":"2024-06-10T02:50:12.729574+0000","last_unstale":"2024-06-10T02:51:46.383517+0000","last_undegraded":"2024-06-10T02:51:46.383517+0000","last_fullsized":"2024-06-10T02:51:46.383517+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:51:16.235284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"98'2","reported_seq":53,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.854226+0000","last_change":"2024-06-10T02:50:44.782421+0000","last_active":"2024-06-10T02:51:45.854226+0000","last_peered":"2024-06-10T02:51:45.854226+0000","last_clean":"2024-06-10T02:51:45.854226+0000","last_became_active":"2024-06-10T02:50:44.781881+0000","last_became_peered":"2024-06-10T02:50:44.781881+0000","last_unstale":"2024-06-10T02:51:45.854226+0000","last_undegraded":"2024-06-10T02:51:45.854226+0000","last_fullsized":"2024-06-10T02:51:45.854226+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T14:41:58.771256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"100'2","reported_seq":41,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.952928+0000","last_change":"2024-06-10T02:50:02.197761+0000","last_active":"2024-06-10T02:52:42.952928+0000","last_peered":"2024-06-10T02:52:42.952928+0000","last_clean":"2024-06-10T02:52:42.952928+0000","last_became_active":"2024-06-10T02:50:02.197395+0000","last_became_peered":"2024-06-10T02:50:02.197395+0000","last_unstale":"2024-06-10T02:52:42.952928+0000","last_undegraded":"2024-06-10T02:52:42.952928+0000","last_fullsized":"2024-06-10T02:52:42.952928+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:46:37.537280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.385820+0000","last_change":"2024-06-10T02:51:46.385129+0000","last_active":"2024-06-10T02:51:46.385820+0000","last_peered":"2024-06-10T02:51:46.385820+0000","last_clean":"2024-06-10T02:51:46.385820+0000","last_became_active":"2024-06-10T02:51:46.384553+0000","last_became_peered":"2024-06-10T02:51:46.384553+0000","last_unstale":"2024-06-10T02:51:46.385820+0000","last_undegraded":"2024-06-10T02:51:46.385820+0000","last_fullsized":"2024-06-10T02:51:46.385820+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:08:11.183679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"98'7","reported_seq":52,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.831750+0000","last_change":"2024-06-10T02:49:58.987863+0000","last_active":"2024-06-10T02:51:45.831750+0000","last_peered":"2024-06-10T02:51:45.831750+0000","last_clean":"2024-06-10T02:51:45.831750+0000","last_became_active":"2024-06-10T02:49:58.987388+0000","last_became_peered":"2024-06-10T02:49:58.987388+0000","last_unstale":"2024-06-10T02:51:45.831750+0000","last_undegraded":"2024-06-10T02:51:45.831750+0000","last_fullsized":"2024-06-10T02:51:45.831750+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:29:20.340301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"100'9","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.988655+0000","last_change":"2024-06-10T02:50:02.200699+0000","last_active":"2024-06-10T02:52:42.988655+0000","last_peered":"2024-06-10T02:52:42.988655+0000","last_clean":"2024-06-10T02:52:42.988655+0000","last_became_active":"2024-06-10T02:50:02.200278+0000","last_became_peered":"2024-06-10T02:50:02.200278+0000","last_unstale":"2024-06-10T02:52:42.988655+0000","last_undegraded":"2024-06-10T02:52:42.988655+0000","last_fullsized":"2024-06-10T02:52:42.988655+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:26:36.236312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"98'1","reported_seq":30,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.383543+0000","last_change":"2024-06-10T02:50:12.236027+0000","last_active":"2024-06-10T02:51:46.383543+0000","last_peered":"2024-06-10T02:51:46.383543+0000","last_clean":"2024-06-10T02:51:46.383543+0000","last_became_active":"2024-06-10T02:50:12.235629+0000","last_became_peered":"2024-06-10T02:50:12.235629+0000","last_unstale":"2024-06-10T02:51:46.383543+0000","last_undegraded":"2024-06-10T02:51:46.383543+0000","last_fullsized":"2024-06-10T02:51:46.383543+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:58:48.218806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"98'3","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.721449+0000","last_change":"2024-06-10T02:49:58.998496+0000","last_active":"2024-06-10T02:51:45.721449+0000","last_peered":"2024-06-10T02:51:45.721449+0000","last_clean":"2024-06-10T02:51:45.721449+0000","last_became_active":"2024-06-10T02:49:58.997260+0000","last_became_peered":"2024-06-10T02:49:58.997260+0000","last_unstale":"2024-06-10T02:51:45.721449+0000","last_undegraded":"2024-06-10T02:51:45.721449+0000","last_fullsized":"2024-06-10T02:51:45.721449+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T14:22:04.658943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"100'8","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.990375+0000","last_change":"2024-06-10T02:50:02.201300+0000","last_active":"2024-06-10T02:52:42.990375+0000","last_peered":"2024-06-10T02:52:42.990375+0000","last_clean":"2024-06-10T02:52:42.990375+0000","last_became_active":"2024-06-10T02:50:02.200988+0000","last_became_peered":"2024-06-10T02:50:02.200988+0000","last_unstale":"2024-06-10T02:52:42.990375+0000","last_undegraded":"2024-06-10T02:52:42.990375+0000","last_fullsized":"2024-06-10T02:52:42.990375+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:11:12.962048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.383484+0000","last_change":"2024-06-10T02:50:12.728869+0000","last_active":"2024-06-10T02:51:46.383484+0000","last_peered":"2024-06-10T02:51:46.383484+0000","last_clean":"2024-06-10T02:51:46.383484+0000","last_became_active":"2024-06-10T02:50:12.728189+0000","last_became_peered":"2024-06-10T02:50:12.728189+0000","last_unstale":"2024-06-10T02:51:46.383484+0000","last_undegraded":"2024-06-10T02:51:46.383484+0000","last_fullsized":"2024-06-10T02:51:46.383484+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:08:36.624814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"98'4","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.723488+0000","last_change":"2024-06-10T02:49:59.000460+0000","last_active":"2024-06-10T02:51:45.723488+0000","last_peered":"2024-06-10T02:51:45.723488+0000","last_clean":"2024-06-10T02:51:45.723488+0000","last_became_active":"2024-06-10T02:49:59.000075+0000","last_became_peered":"2024-06-10T02:49:59.000075+0000","last_unstale":"2024-06-10T02:51:45.723488+0000","last_undegraded":"2024-06-10T02:51:45.723488+0000","last_fullsized":"2024-06-10T02:51:45.723488+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:06:27.935745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"100'5","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:43.001075+0000","last_change":"2024-06-10T02:50:02.210131+0000","last_active":"2024-06-10T02:52:43.001075+0000","last_peered":"2024-06-10T02:52:43.001075+0000","last_clean":"2024-06-10T02:52:43.001075+0000","last_became_active":"2024-06-10T02:50:02.209781+0000","last_became_peered":"2024-06-10T02:50:02.209781+0000","last_unstale":"2024-06-10T02:52:43.001075+0000","last_undegraded":"2024-06-10T02:52:43.001075+0000","last_fullsized":"2024-06-10T02:52:43.001075+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:27:39.966620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.854407+0000","last_change":"2024-06-10T02:50:12.236017+0000","last_active":"2024-06-10T02:51:45.854407+0000","last_peered":"2024-06-10T02:51:45.854407+0000","last_clean":"2024-06-10T02:51:45.854407+0000","last_became_active":"2024-06-10T02:50:12.235553+0000","last_became_peered":"2024-06-10T02:50:12.235553+0000","last_unstale":"2024-06-10T02:51:45.854407+0000","last_undegraded":"2024-06-10T02:51:45.854407+0000","last_fullsized":"2024-06-10T02:51:45.854407+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:23:55.437994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"100'7","reported_seq":153,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:44.778801+0000","last_change":"2024-06-10T02:49:59.192864+0000","last_active":"2024-06-10T02:52:44.778801+0000","last_peered":"2024-06-10T02:52:44.778801+0000","last_clean":"2024-06-10T02:52:44.778801+0000","last_became_active":"2024-06-10T02:49:59.192440+0000","last_became_peered":"2024-06-10T02:49:59.192440+0000","last_unstale":"2024-06-10T02:52:44.778801+0000","last_undegraded":"2024-06-10T02:52:44.778801+0000","last_fullsized":"2024-06-10T02:52:44.778801+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:36:05.868538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1701,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":101,"num_read_kb":4,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"100'7","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:43.000525+0000","last_change":"2024-06-10T02:50:02.205791+0000","last_active":"2024-06-10T02:52:43.000525+0000","last_peered":"2024-06-10T02:52:43.000525+0000","last_clean":"2024-06-10T02:52:43.000525+0000","last_became_active":"2024-06-10T02:50:02.205483+0000","last_became_peered":"2024-06-10T02:50:02.205483+0000","last_unstale":"2024-06-10T02:52:43.000525+0000","last_undegraded":"2024-06-10T02:52:43.000525+0000","last_fullsized":"2024-06-10T02:52:43.000525+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:33:25.875423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.011597+0000","last_change":"2024-06-10T02:50:12.239294+0000","last_active":"2024-06-10T02:51:46.011597+0000","last_peered":"2024-06-10T02:51:46.011597+0000","last_clean":"2024-06-10T02:51:46.011597+0000","last_became_active":"2024-06-10T02:50:12.238778+0000","last_became_peered":"2024-06-10T02:50:12.238778+0000","last_unstale":"2024-06-10T02:51:46.011597+0000","last_undegraded":"2024-06-10T02:51:46.011597+0000","last_fullsized":"2024-06-10T02:51:46.011597+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:26:09.203623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"98'3","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.383589+0000","last_change":"2024-06-10T02:49:58.989680+0000","last_active":"2024-06-10T02:51:46.383589+0000","last_peered":"2024-06-10T02:51:46.383589+0000","last_clean":"2024-06-10T02:51:46.383589+0000","last_became_active":"2024-06-10T02:49:58.989265+0000","last_became_peered":"2024-06-10T02:49:58.989265+0000","last_unstale":"2024-06-10T02:51:46.383589+0000","last_undegraded":"2024-06-10T02:51:46.383589+0000","last_fullsized":"2024-06-10T02:51:46.383589+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:36:34.370177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"100'6","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.990330+0000","last_change":"2024-06-10T02:50:44.743369+0000","last_active":"2024-06-10T02:52:42.990330+0000","last_peered":"2024-06-10T02:52:42.990330+0000","last_clean":"2024-06-10T02:52:42.990330+0000","last_became_active":"2024-06-10T02:50:44.743013+0000","last_became_peered":"2024-06-10T02:50:44.743013+0000","last_unstale":"2024-06-10T02:52:42.990330+0000","last_undegraded":"2024-06-10T02:52:42.990330+0000","last_fullsized":"2024-06-10T02:52:42.990330+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:54:43.728943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.382656+0000","last_change":"2024-06-10T02:50:12.235197+0000","last_active":"2024-06-10T02:51:46.382656+0000","last_peered":"2024-06-10T02:51:46.382656+0000","last_clean":"2024-06-10T02:51:46.382656+0000","last_became_active":"2024-06-10T02:50:12.234791+0000","last_became_peered":"2024-06-10T02:50:12.234791+0000","last_unstale":"2024-06-10T02:51:46.382656+0000","last_undegraded":"2024-06-10T02:51:46.382656+0000","last_fullsized":"2024-06-10T02:51:46.382656+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:09:11.164236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"98'5","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.715164+0000","last_change":"2024-06-10T02:49:58.996759+0000","last_active":"2024-06-10T02:51:45.715164+0000","last_peered":"2024-06-10T02:51:45.715164+0000","last_clean":"2024-06-10T02:51:45.715164+0000","last_became_active":"2024-06-10T02:49:58.996434+0000","last_became_peered":"2024-06-10T02:49:58.996434+0000","last_unstale":"2024-06-10T02:51:45.715164+0000","last_undegraded":"2024-06-10T02:51:45.715164+0000","last_fullsized":"2024-06-10T02:51:45.715164+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:09:35.327797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"100'9","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.996813+0000","last_change":"2024-06-10T02:50:02.201676+0000","last_active":"2024-06-10T02:52:42.996813+0000","last_peered":"2024-06-10T02:52:42.996813+0000","last_clean":"2024-06-10T02:52:42.996813+0000","last_became_active":"2024-06-10T02:50:02.201335+0000","last_became_peered":"2024-06-10T02:50:02.201335+0000","last_unstale":"2024-06-10T02:52:42.996813+0000","last_undegraded":"2024-06-10T02:52:42.996813+0000","last_fullsized":"2024-06-10T02:52:42.996813+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:49:47.038946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"100'9","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.992754+0000","last_change":"2024-06-10T02:50:02.203462+0000","last_active":"2024-06-10T02:52:42.992754+0000","last_peered":"2024-06-10T02:52:42.992754+0000","last_clean":"2024-06-10T02:52:42.992754+0000","last_became_active":"2024-06-10T02:50:02.203115+0000","last_became_peered":"2024-06-10T02:50:02.203115+0000","last_unstale":"2024-06-10T02:52:42.992754+0000","last_undegraded":"2024-06-10T02:52:42.992754+0000","last_fullsized":"2024-06-10T02:52:42.992754+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T13:18:12.400596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"100'12","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.997935+0000","last_change":"2024-06-10T02:50:02.205648+0000","last_active":"2024-06-10T02:52:42.997935+0000","last_peered":"2024-06-10T02:52:42.997935+0000","last_clean":"2024-06-10T02:52:42.997935+0000","last_became_active":"2024-06-10T02:50:02.205306+0000","last_became_peered":"2024-06-10T02:50:02.205306+0000","last_unstale":"2024-06-10T02:52:42.997935+0000","last_undegraded":"2024-06-10T02:52:42.997935+0000","last_fullsized":"2024-06-10T02:52:42.997935+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:22:55.020644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"100'8","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.991481+0000","last_change":"2024-06-10T02:50:02.198601+0000","last_active":"2024-06-10T02:52:42.991481+0000","last_peered":"2024-06-10T02:52:42.991481+0000","last_clean":"2024-06-10T02:52:42.991481+0000","last_became_active":"2024-06-10T02:50:02.198103+0000","last_became_peered":"2024-06-10T02:50:02.198103+0000","last_unstale":"2024-06-10T02:52:42.991481+0000","last_undegraded":"2024-06-10T02:52:42.991481+0000","last_fullsized":"2024-06-10T02:52:42.991481+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:56:28.033482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"100'7","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.985332+0000","last_change":"2024-06-10T02:50:02.202638+0000","last_active":"2024-06-10T02:52:42.985332+0000","last_peered":"2024-06-10T02:52:42.985332+0000","last_clean":"2024-06-10T02:52:42.985332+0000","last_became_active":"2024-06-10T02:50:02.202249+0000","last_became_peered":"2024-06-10T02:50:02.202249+0000","last_unstale":"2024-06-10T02:52:42.985332+0000","last_undegraded":"2024-06-10T02:52:42.985332+0000","last_fullsized":"2024-06-10T02:52:42.985332+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:46:29.424164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"100'4","reported_seq":43,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.992649+0000","last_change":"2024-06-10T02:50:02.213526+0000","last_active":"2024-06-10T02:52:42.992649+0000","last_peered":"2024-06-10T02:52:42.992649+0000","last_clean":"2024-06-10T02:52:42.992649+0000","last_became_active":"2024-06-10T02:50:02.213226+0000","last_became_peered":"2024-06-10T02:50:02.213226+0000","last_unstale":"2024-06-10T02:52:42.992649+0000","last_undegraded":"2024-06-10T02:52:42.992649+0000","last_fullsized":"2024-06-10T02:52:42.992649+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:01:56.846340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"100'6","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.993061+0000","last_change":"2024-06-10T02:50:02.206583+0000","last_active":"2024-06-10T02:52:42.993061+0000","last_peered":"2024-06-10T02:52:42.993061+0000","last_clean":"2024-06-10T02:52:42.993061+0000","last_became_active":"2024-06-10T02:50:02.206227+0000","last_became_peered":"2024-06-10T02:50:02.206227+0000","last_unstale":"2024-06-10T02:52:42.993061+0000","last_undegraded":"2024-06-10T02:52:42.993061+0000","last_fullsized":"2024-06-10T02:52:42.993061+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:25:07.246288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"100'15","reported_seq":54,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:43.002970+0000","last_change":"2024-06-10T02:50:02.206348+0000","last_active":"2024-06-10T02:52:43.002970+0000","last_peered":"2024-06-10T02:52:43.002970+0000","last_clean":"2024-06-10T02:52:43.002970+0000","last_became_active":"2024-06-10T02:50:02.206020+0000","last_became_peered":"2024-06-10T02:50:02.206020+0000","last_unstale":"2024-06-10T02:52:43.002970+0000","last_undegraded":"2024-06-10T02:52:43.002970+0000","last_fullsized":"2024-06-10T02:52:43.002970+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:02:26.619113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"100'2","reported_seq":41,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.993521+0000","last_change":"2024-06-10T02:50:02.199405+0000","last_active":"2024-06-10T02:52:42.993521+0000","last_peered":"2024-06-10T02:52:42.993521+0000","last_clean":"2024-06-10T02:52:42.993521+0000","last_became_active":"2024-06-10T02:50:02.199049+0000","last_became_peered":"2024-06-10T02:50:02.199049+0000","last_unstale":"2024-06-10T02:52:42.993521+0000","last_undegraded":"2024-06-10T02:52:42.993521+0000","last_fullsized":"2024-06-10T02:52:42.993521+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:33:04.091160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"100'6","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.992703+0000","last_change":"2024-06-10T02:50:02.202094+0000","last_active":"2024-06-10T02:52:42.992703+0000","last_peered":"2024-06-10T02:52:42.992703+0000","last_clean":"2024-06-10T02:52:42.992703+0000","last_became_active":"2024-06-10T02:50:02.201751+0000","last_became_peered":"2024-06-10T02:50:02.201751+0000","last_unstale":"2024-06-10T02:52:42.992703+0000","last_undegraded":"2024-06-10T02:52:42.992703+0000","last_fullsized":"2024-06-10T02:52:42.992703+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:52:53.946326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"100'8","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.993442+0000","last_change":"2024-06-10T02:50:02.199630+0000","last_active":"2024-06-10T02:52:42.993442+0000","last_peered":"2024-06-10T02:52:42.993442+0000","last_clean":"2024-06-10T02:52:42.993442+0000","last_became_active":"2024-06-10T02:50:02.199320+0000","last_became_peered":"2024-06-10T02:50:02.199320+0000","last_unstale":"2024-06-10T02:52:42.993442+0000","last_undegraded":"2024-06-10T02:52:42.993442+0000","last_fullsized":"2024-06-10T02:52:42.993442+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:17:28.761862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"100'10","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:43.000631+0000","last_change":"2024-06-10T02:50:02.201190+0000","last_active":"2024-06-10T02:52:43.000631+0000","last_peered":"2024-06-10T02:52:43.000631+0000","last_clean":"2024-06-10T02:52:43.000631+0000","last_became_active":"2024-06-10T02:50:02.200574+0000","last_became_peered":"2024-06-10T02:50:02.200574+0000","last_unstale":"2024-06-10T02:52:43.000631+0000","last_undegraded":"2024-06-10T02:52:43.000631+0000","last_fullsized":"2024-06-10T02:52:43.000631+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:02:07.636218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"100'11","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.992436+0000","last_change":"2024-06-10T02:50:02.202374+0000","last_active":"2024-06-10T02:52:42.992436+0000","last_peered":"2024-06-10T02:52:42.992436+0000","last_clean":"2024-06-10T02:52:42.992436+0000","last_became_active":"2024-06-10T02:50:02.201974+0000","last_became_peered":"2024-06-10T02:50:02.201974+0000","last_unstale":"2024-06-10T02:52:42.992436+0000","last_undegraded":"2024-06-10T02:52:42.992436+0000","last_fullsized":"2024-06-10T02:52:42.992436+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:45:34.977926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"100'8","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.995177+0000","last_change":"2024-06-10T02:50:02.209332+0000","last_active":"2024-06-10T02:52:42.995177+0000","last_peered":"2024-06-10T02:52:42.995177+0000","last_clean":"2024-06-10T02:52:42.995177+0000","last_became_active":"2024-06-10T02:50:02.208175+0000","last_became_peered":"2024-06-10T02:50:02.208175+0000","last_unstale":"2024-06-10T02:52:42.995177+0000","last_undegraded":"2024-06-10T02:52:42.995177+0000","last_fullsized":"2024-06-10T02:52:42.995177+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:44:40.857918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":284028,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":71450,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1024,"num_read_kb":593,"num_write":264,"num_write_kb":156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":577536,"data_stored":214350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":227,"ondisk_log_size":227,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351026,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32260,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712892,"statfs":{"total":95995035648,"available":95962001408,"internally_reserved":0,"allocated":1110016,"data_stored":574044,"data_compressed":32447,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612663,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32408,"kb_used_data":1232,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712744,"statfs":{"total":95995035648,"available":95961849856,"internally_reserved":0,"allocated":1261568,"data_stored":1223764,"data_compressed":68564,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1599,"internal_metadata":31914433},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776188,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36448,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708704,"statfs":{"total":95995035648,"available":95957712896,"internally_reserved":0,"allocated":1138688,"data_stored":577539,"data_compressed":32447,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037825,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28112,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717040,"statfs":{"total":95995035648,"available":95966248960,"internally_reserved":0,"allocated":1122304,"data_stored":573149,"data_compressed":32447,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299462,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28172,"kb_used_data":1156,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716980,"statfs":{"total":95995035648,"available":95966187520,"internally_reserved":0,"allocated":1183744,"data_stored":1171520,"data_compressed":68564,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462988,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36420,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708732,"statfs":{"total":95995035648,"available":95957741568,"internally_reserved":0,"allocated":1110016,"data_stored":569478,"data_compressed":32447,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574949,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36488,"kb_used_data":1152,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708664,"statfs":{"total":95995035648,"available":95957671936,"internally_reserved":0,"allocated":1179648,"data_stored":633412,"data_compressed":32447,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738473,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28124,"kb_used_data":1108,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717028,"statfs":{"total":95995035648,"available":95966236672,"internally_reserved":0,"allocated":1134592,"data_stored":570934,"data_compressed":32447,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411424,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28188,"kb_used_data":1172,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716964,"statfs":{"total":95995035648,"available":95966171136,"internally_reserved":0,"allocated":1200128,"data_stored":634163,"data_compressed":32447,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149786,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36432,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708720,"statfs":{"total":95995035648,"available":95957729280,"internally_reserved":0,"allocated":1122304,"data_stored":568655,"data_compressed":32447,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888149,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28116,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717036,"statfs":{"total":95995035648,"available":95966244864,"internally_reserved":0,"allocated":1126400,"data_stored":574224,"data_compressed":32447,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724624,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28180,"kb_used_data":1164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716972,"statfs":{"total":95995035648,"available":95966179328,"internally_reserved":0,"allocated":1191936,"data_stored":1166040,"data_compressed":68564,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34481,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34481,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34481,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36117,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36117,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36117,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36117,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36117,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":65187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":65938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1903,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3351,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1253,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":65171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-06-10T02:52:51.458 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-06-10T02:52:51.459 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-06-10T02:52:51.459 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-06-10T02:52:51.459 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph health --format=json 2024-06-10T02:52:51.718 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:52:52.079 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:52:51 smithi052 ceph-mon[32362]: from='client.14997 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-10T02:52:52.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:52:51 smithi120 ceph-mon[36634]: from='client.14997 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-10T02:52:52.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:52:51 smithi188 ceph-mon[36599]: from='client.14997 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-10T02:52:53.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:52:52 smithi052 ceph-mon[32362]: pgmap v394: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 938 B/s wr, 10 op/s 2024-06-10T02:52:53.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:52:52 smithi120 ceph-mon[36634]: pgmap v394: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 938 B/s wr, 10 op/s 2024-06-10T02:52:53.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:52:52 smithi188 ceph-mon[36599]: pgmap v394: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 938 B/s wr, 10 op/s 2024-06-10T02:52:53.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:53.472+0000 7f14fb645700 1 -- 172.21.15.52:0/1669763386 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f14f406a170 msgr2=0x7f14f406a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:52:53.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:53.472+0000 7f14fb645700 1 --2- 172.21.15.52:0/1669763386 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f14f406a170 0x7f14f406a550 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f14f0009990 tx=0x7f14f0009ca0 comp rx=0 tx=0).stop 2024-06-10T02:52:53.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:53.473+0000 7f14fb645700 1 -- 172.21.15.52:0/1669763386 shutdown_connections 2024-06-10T02:52:53.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:53.473+0000 7f14fb645700 1 --2- 172.21.15.52:0/1669763386 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f14f410ef40 0x7f14f4111330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:53.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:53.473+0000 7f14fb645700 1 --2- 172.21.15.52:0/1669763386 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f14f406a170 0x7f14f406a550 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:53.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:53.473+0000 7f14fb645700 1 --2- 172.21.15.52:0/1669763386 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f14f406aa90 0x7f14f410ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:53.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:53.473+0000 7f14fb645700 1 -- 172.21.15.52:0/1669763386 >> 172.21.15.52:0/1669763386 conn(0x7f14f40782c0 msgr2=0x7f14f40786d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:52:53.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:53.473+0000 7f14fb645700 1 -- 172.21.15.52:0/1669763386 shutdown_connections 2024-06-10T02:52:53.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:53.473+0000 7f14fb645700 1 -- 172.21.15.52:0/1669763386 wait complete. 2024-06-10T02:52:53.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:53.473+0000 7f14fb645700 1 Processor -- start 2024-06-10T02:52:53.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:53.473+0000 7f14fb645700 1 -- start start 2024-06-10T02:52:53.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:53.474+0000 7f14fb645700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f14f406a170 0x7f14f419ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:52:53.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:53.474+0000 7f14fb645700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f14f406aa90 0x7f14f419b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:52:53.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:53.474+0000 7f14fb645700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f14f410ef40 0x7f14f419f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:52:53.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:53.474+0000 7f14fb645700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f14f4114970 con 0x7f14f410ef40 2024-06-10T02:52:53.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:53.474+0000 7f14fb645700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f14f41147f0 con 0x7f14f406aa90 2024-06-10T02:52:53.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:53.474+0000 7f14fb645700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f14f4114af0 con 0x7f14f406a170 2024-06-10T02:52:53.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:53.474+0000 7f14f9be2700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f14f410ef40 0x7f14f419f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:52:53.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:53.474+0000 7f14f9be2700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f14f410ef40 0x7f14f419f6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:39074/0 (socket says 172.21.15.52:39074) 2024-06-10T02:52:53.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:53.474+0000 7f14f93e1700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f14f406a170 0x7f14f419ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:52:53.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:53.474+0000 7f14f9be2700 1 -- 172.21.15.52:0/2005577506 learned_addr learned my addr 172.21.15.52:0/2005577506 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:52:53.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:53.474+0000 7f14f8be0700 1 --2- 172.21.15.52:0/2005577506 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f14f406aa90 0x7f14f419b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:52:53.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:53.474+0000 7f14f9be2700 1 -- 172.21.15.52:0/2005577506 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f14f406a170 msgr2=0x7f14f419ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:52:53.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:53.474+0000 7f14f9be2700 1 --2- 172.21.15.52:0/2005577506 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f14f406a170 0x7f14f419ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:53.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:53.474+0000 7f14f9be2700 1 -- 172.21.15.52:0/2005577506 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f14f406aa90 msgr2=0x7f14f419b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:52:53.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:53.474+0000 7f14f9be2700 1 --2- 172.21.15.52:0/2005577506 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f14f406aa90 0x7f14f419b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:53.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:53.474+0000 7f14f9be2700 1 -- 172.21.15.52:0/2005577506 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14f419fde0 con 0x7f14f410ef40 2024-06-10T02:52:53.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:53.474+0000 7f14f9be2700 1 --2- 172.21.15.52:0/2005577506 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f14f410ef40 0x7f14f419f6a0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f14ec00ada0 tx=0x7f14ec00c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:52:53.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:53.475+0000 7f14ea7fc700 1 -- 172.21.15.52:0/2005577506 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14ec017020 con 0x7f14f410ef40 2024-06-10T02:52:53.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:53.475+0000 7f14fb645700 1 -- 172.21.15.52:0/2005577506 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14f41a00d0 con 0x7f14f410ef40 2024-06-10T02:52:53.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:53.475+0000 7f14ea7fc700 1 -- 172.21.15.52:0/2005577506 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14ec00cdb0 con 0x7f14f410ef40 2024-06-10T02:52:53.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:53.475+0000 7f14ea7fc700 1 -- 172.21.15.52:0/2005577506 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14ec0053a0 con 0x7f14f410ef40 2024-06-10T02:52:53.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:53.475+0000 7f14fb645700 1 -- 172.21.15.52:0/2005577506 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14f41a7950 con 0x7f14f410ef40 2024-06-10T02:52:53.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:53.476+0000 7f14fb645700 1 -- 172.21.15.52:0/2005577506 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14f4069180 con 0x7f14f410ef40 2024-06-10T02:52:53.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:53.477+0000 7f14ea7fc700 1 -- 172.21.15.52:0/2005577506 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f14ec00c8d0 con 0x7f14f410ef40 2024-06-10T02:52:53.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:53.478+0000 7f14ea7fc700 1 --2- 172.21.15.52:0/2005577506 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f14e00a2c20 0x7f14e00a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:52:53.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:53.478+0000 7f14f93e1700 1 --2- 172.21.15.52:0/2005577506 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f14e00a2c20 0x7f14e00a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:52:53.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:53.478+0000 7f14ea7fc700 1 -- 172.21.15.52:0/2005577506 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f14ec0cefe0 con 0x7f14f410ef40 2024-06-10T02:52:53.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:53.479+0000 7f14f93e1700 1 --2- 172.21.15.52:0/2005577506 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f14e00a2c20 0x7f14e00a50e0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f14f00098e0 tx=0x7f14f0005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:52:53.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:53.482+0000 7f14ea7fc700 1 -- 172.21.15.52:0/2005577506 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f14ec021020 con 0x7f14f410ef40 2024-06-10T02:52:53.735 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:53.736+0000 7f14fb645700 1 -- 172.21.15.52:0/2005577506 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f14f402dca0 con 0x7f14f410ef40 2024-06-10T02:52:53.735 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:53.736+0000 7f14ea7fc700 1 -- 172.21.15.52:0/2005577506 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f14ec09bbe0 con 0x7f14f410ef40 2024-06-10T02:52:53.735 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-10T02:52:53.735 INFO:teuthology.orchestra.run.smithi052.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-06-10T02:52:53.737 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:53.738+0000 7f14fb645700 1 -- 172.21.15.52:0/2005577506 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f14e00a2c20 msgr2=0x7f14e00a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:52:53.737 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:53.738+0000 7f14fb645700 1 --2- 172.21.15.52:0/2005577506 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f14e00a2c20 0x7f14e00a50e0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f14f00098e0 tx=0x7f14f0005960 comp rx=0 tx=0).stop 2024-06-10T02:52:53.737 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:53.738+0000 7f14fb645700 1 -- 172.21.15.52:0/2005577506 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f14f410ef40 msgr2=0x7f14f419f6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:52:53.737 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:53.738+0000 7f14fb645700 1 --2- 172.21.15.52:0/2005577506 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f14f410ef40 0x7f14f419f6a0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f14ec00ada0 tx=0x7f14ec00c330 comp rx=0 tx=0).stop 2024-06-10T02:52:53.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:53.738+0000 7f14fb645700 1 -- 172.21.15.52:0/2005577506 shutdown_connections 2024-06-10T02:52:53.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:53.738+0000 7f14fb645700 1 --2- 172.21.15.52:0/2005577506 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f14e00a2c20 0x7f14e00a50e0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:53.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:53.738+0000 7f14fb645700 1 --2- 172.21.15.52:0/2005577506 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f14f410ef40 0x7f14f419f6a0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:53.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:53.738+0000 7f14fb645700 1 --2- 172.21.15.52:0/2005577506 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f14f406a170 0x7f14f419ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:53.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:53.738+0000 7f14fb645700 1 --2- 172.21.15.52:0/2005577506 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f14f406aa90 0x7f14f419b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:53.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:53.738+0000 7f14fb645700 1 -- 172.21.15.52:0/2005577506 >> 172.21.15.52:0/2005577506 conn(0x7f14f40782c0 msgr2=0x7f14f41104f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:52:53.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:53.738+0000 7f14fb645700 1 -- 172.21.15.52:0/2005577506 shutdown_connections 2024-06-10T02:52:53.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:53.739+0000 7f14fb645700 1 -- 172.21.15.52:0/2005577506 wait complete. 2024-06-10T02:52:54.051 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:52:53 smithi052 ceph-mon[32362]: from='client.? 172.21.15.52:0/2005577506' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-06-10T02:52:54.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:52:53 smithi120 ceph-mon[36634]: from='client.? 172.21.15.52:0/2005577506' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-06-10T02:52:54.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:52:53 smithi188 ceph-mon[36599]: from='client.? 172.21.15.52:0/2005577506' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-06-10T02:52:54.487 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-06-10T02:52:54.488 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T02:52:54.746 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:52:55.106 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:52:54 smithi052 ceph-mon[32362]: pgmap v395: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 938 B/s wr, 13 op/s 2024-06-10T02:52:55.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:52:54 smithi120 ceph-mon[36634]: pgmap v395: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 938 B/s wr, 13 op/s 2024-06-10T02:52:55.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:52:54 smithi188 ceph-mon[36599]: pgmap v395: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 938 B/s wr, 13 op/s 2024-06-10T02:52:56.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:56.452+0000 7fa1f60a3700 1 -- 172.21.15.52:0/300000828 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa1f0101610 msgr2=0x7fa1f01019f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:52:56.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:56.452+0000 7fa1f60a3700 1 --2- 172.21.15.52:0/300000828 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa1f0101610 0x7fa1f01019f0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fa1e00099c0 tx=0x7fa1e0009cd0 comp rx=0 tx=0).stop 2024-06-10T02:52:56.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:56.452+0000 7fa1f60a3700 1 -- 172.21.15.52:0/300000828 shutdown_connections 2024-06-10T02:52:56.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:56.452+0000 7fa1f60a3700 1 --2- 172.21.15.52:0/300000828 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa1f01066f0 0x7fa1f010e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:56.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:56.452+0000 7fa1f60a3700 1 --2- 172.21.15.52:0/300000828 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa1f0101610 0x7fa1f01019f0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:56.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:56.452+0000 7fa1f60a3700 1 --2- 172.21.15.52:0/300000828 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa1f0101fc0 0x7fa1f0106010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:56.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:56.452+0000 7fa1f60a3700 1 -- 172.21.15.52:0/300000828 >> 172.21.15.52:0/300000828 conn(0x7fa1f0078430 msgr2=0x7fa1f0078840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:52:56.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:56.452+0000 7fa1f60a3700 1 -- 172.21.15.52:0/300000828 shutdown_connections 2024-06-10T02:52:56.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:56.452+0000 7fa1f60a3700 1 -- 172.21.15.52:0/300000828 wait complete. 2024-06-10T02:52:56.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:56.453+0000 7fa1f60a3700 1 Processor -- start 2024-06-10T02:52:56.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:56.453+0000 7fa1f60a3700 1 -- start start 2024-06-10T02:52:56.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:56.453+0000 7fa1f60a3700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa1f0101610 0x7fa1f01030b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:52:56.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:56.454+0000 7fa1f60a3700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa1f0101fc0 0x7fa1f01035f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:52:56.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:56.454+0000 7fa1f60a3700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa1f01066f0 0x7fa1f0103b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:52:56.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:56.454+0000 7fa1f60a3700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fa1f01116d0 con 0x7fa1f0101fc0 2024-06-10T02:52:56.454 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:56.454+0000 7fa1f60a3700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa1f0111550 con 0x7fa1f0101610 2024-06-10T02:52:56.454 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:56.454+0000 7fa1f60a3700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa1f0111850 con 0x7fa1f01066f0 2024-06-10T02:52:56.454 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:56.454+0000 7fa1eeffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa1f0101fc0 0x7fa1f01035f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:52:56.454 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:56.454+0000 7fa1eeffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa1f0101fc0 0x7fa1f01035f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:39086/0 (socket says 172.21.15.52:39086) 2024-06-10T02:52:56.454 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:56.454+0000 7fa1eeffd700 1 -- 172.21.15.52:0/1597005262 learned_addr learned my addr 172.21.15.52:0/1597005262 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:52:56.454 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:56.454+0000 7fa1ef7fe700 1 --2- 172.21.15.52:0/1597005262 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa1f0101610 0x7fa1f01030b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:52:56.454 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:56.454+0000 7fa1effff700 1 --2- 172.21.15.52:0/1597005262 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa1f01066f0 0x7fa1f0103b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:52:56.454 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:56.454+0000 7fa1eeffd700 1 -- 172.21.15.52:0/1597005262 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa1f01066f0 msgr2=0x7fa1f0103b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:52:56.454 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:56.454+0000 7fa1eeffd700 1 --2- 172.21.15.52:0/1597005262 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa1f01066f0 0x7fa1f0103b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:56.454 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:56.454+0000 7fa1eeffd700 1 -- 172.21.15.52:0/1597005262 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa1f0101610 msgr2=0x7fa1f01030b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:52:56.454 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:56.454+0000 7fa1eeffd700 1 --2- 172.21.15.52:0/1597005262 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa1f0101610 0x7fa1f01030b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:56.454 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:56.454+0000 7fa1eeffd700 1 -- 172.21.15.52:0/1597005262 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1f01a71a0 con 0x7fa1f0101fc0 2024-06-10T02:52:56.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:56.454+0000 7fa1eeffd700 1 --2- 172.21.15.52:0/1597005262 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa1f0101fc0 0x7fa1f01035f0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fa1e400d980 tx=0x7fa1e400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:52:56.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:56.455+0000 7fa1ecff9700 1 -- 172.21.15.52:0/1597005262 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1e4017070 con 0x7fa1f0101fc0 2024-06-10T02:52:56.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:56.455+0000 7fa1f60a3700 1 -- 172.21.15.52:0/1597005262 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1f01a7490 con 0x7fa1f0101fc0 2024-06-10T02:52:56.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:56.455+0000 7fa1ecff9700 1 -- 172.21.15.52:0/1597005262 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1e4009e10 con 0x7fa1f0101fc0 2024-06-10T02:52:56.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:56.455+0000 7fa1ecff9700 1 -- 172.21.15.52:0/1597005262 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1e400f620 con 0x7fa1f0101fc0 2024-06-10T02:52:56.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:56.455+0000 7fa1f60a3700 1 -- 172.21.15.52:0/1597005262 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1f01a79d0 con 0x7fa1f0101fc0 2024-06-10T02:52:56.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:56.456+0000 7fa1f60a3700 1 -- 172.21.15.52:0/1597005262 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1f0069180 con 0x7fa1f0101fc0 2024-06-10T02:52:56.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:56.457+0000 7fa1ecff9700 1 -- 172.21.15.52:0/1597005262 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fa1e4010460 con 0x7fa1f0101fc0 2024-06-10T02:52:56.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:56.458+0000 7fa1ecff9700 1 --2- 172.21.15.52:0/1597005262 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fa1d80a2c20 0x7fa1d80a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:52:56.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:56.458+0000 7fa1ef7fe700 1 --2- 172.21.15.52:0/1597005262 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fa1d80a2c20 0x7fa1d80a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:52:56.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:56.458+0000 7fa1ecff9700 1 -- 172.21.15.52:0/1597005262 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7fa1e40ceff0 con 0x7fa1f0101fc0 2024-06-10T02:52:56.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:56.458+0000 7fa1ef7fe700 1 --2- 172.21.15.52:0/1597005262 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fa1d80a2c20 0x7fa1d80a50e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fa1e0000c00 tx=0x7fa1e0005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:52:56.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:56.462+0000 7fa1ecff9700 1 -- 172.21.15.52:0/1597005262 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fa1e409bb40 con 0x7fa1f0101fc0 2024-06-10T02:52:56.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:56.715+0000 7fa1f60a3700 1 -- 172.21.15.52:0/1597005262 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa1f002dca0 con 0x7fa1f0101fc0 2024-06-10T02:52:56.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:56.715+0000 7fa1ecff9700 1 -- 172.21.15.52:0/1597005262 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11938 (secure 0 0 0) 0x7fa1e409b440 con 0x7fa1f0101fc0 2024-06-10T02:52:56.715 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-10T02:52:56.715 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/889673247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":889673247},{"type":"v1","addr":"172.21.15.52:6835","nonce":889673247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/3615663511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":3615663511},{"type":"v1","addr":"172.21.15.52:6837","nonce":3615663511}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T02:50:03.889766+0000","modified":"2024-06-10T02:51:11.420770+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24484,"mds_1":24469,"mds_2":24463,"mds_3":24449,"mds_4":24428},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14571":{"gid":14571,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6839/3059599042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":3059599042},{"type":"v1","addr":"172.21.15.52:6839","nonce":3059599042}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14583":{"gid":14583,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6841/3039275521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":3039275521},{"type":"v1","addr":"172.21.15.52:6841","nonce":3039275521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24383":{"gid":24383,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6833/2609002896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":2609002896},{"type":"v1","addr":"172.21.15.120:6833","nonce":2609002896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6835/1996370288","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":1996370288},{"type":"v1","addr":"172.21.15.120:6835","nonce":1996370288}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6833/529564979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":529564979},{"type":"v1","addr":"172.21.15.188:6833","nonce":529564979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":25,"addr":"172.21.15.120:6837/4132440952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":4132440952},{"type":"v1","addr":"172.21.15.120:6837","nonce":4132440952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"b","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.120:6839/1672436487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1672436487},{"type":"v1","addr":"172.21.15.120:6839","nonce":1672436487}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":2,"incarnation":22,"state":"up:active","state_seq":26,"addr":"172.21.15.188:6835/3904544259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":3904544259},{"type":"v1","addr":"172.21.15.188:6835","nonce":3904544259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.188:6837/819317726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":819317726},{"type":"v1","addr":"172.21.15.188:6837","nonce":819317726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.188:6839/310444836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":310444836},{"type":"v1","addr":"172.21.15.188:6839","nonce":310444836}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24484,"qdb_cluster":[24469,24449,24428,24463,24484]},"id":1}]} 2024-06-10T02:52:56.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:56.718+0000 7fa1f60a3700 1 -- 172.21.15.52:0/1597005262 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fa1d80a2c20 msgr2=0x7fa1d80a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:52:56.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:56.718+0000 7fa1f60a3700 1 --2- 172.21.15.52:0/1597005262 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fa1d80a2c20 0x7fa1d80a50e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fa1e0000c00 tx=0x7fa1e0005fb0 comp rx=0 tx=0).stop 2024-06-10T02:52:56.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:56.718+0000 7fa1f60a3700 1 -- 172.21.15.52:0/1597005262 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa1f0101fc0 msgr2=0x7fa1f01035f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:52:56.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:56.718+0000 7fa1f60a3700 1 --2- 172.21.15.52:0/1597005262 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa1f0101fc0 0x7fa1f01035f0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fa1e400d980 tx=0x7fa1e400dc90 comp rx=0 tx=0).stop 2024-06-10T02:52:56.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:56.718+0000 7fa1f60a3700 1 -- 172.21.15.52:0/1597005262 shutdown_connections 2024-06-10T02:52:56.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:56.718+0000 7fa1f60a3700 1 --2- 172.21.15.52:0/1597005262 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa1f01066f0 0x7fa1f0103b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:56.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:56.718+0000 7fa1f60a3700 1 --2- 172.21.15.52:0/1597005262 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa1f0101610 0x7fa1f01030b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:56.718 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:56.718+0000 7fa1f60a3700 1 --2- 172.21.15.52:0/1597005262 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fa1d80a2c20 0x7fa1d80a50e0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:56.718 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:56.718+0000 7fa1f60a3700 1 --2- 172.21.15.52:0/1597005262 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa1f0101fc0 0x7fa1f01035f0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:56.718 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:56.718+0000 7fa1f60a3700 1 -- 172.21.15.52:0/1597005262 >> 172.21.15.52:0/1597005262 conn(0x7fa1f0078430 msgr2=0x7fa1f01005c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:52:56.718 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:56.718+0000 7fa1f60a3700 1 -- 172.21.15.52:0/1597005262 shutdown_connections 2024-06-10T02:52:56.718 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:56.718+0000 7fa1f60a3700 1 -- 172.21.15.52:0/1597005262 wait complete. 2024-06-10T02:52:56.718 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 29 2024-06-10T02:52:56.976 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:52:56 smithi052 ceph-mon[32362]: pgmap v396: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 938 B/s wr, 9 op/s 2024-06-10T02:52:56.976 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:52:56 smithi052 ceph-mon[32362]: from='client.? 172.21.15.52:0/1597005262' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T02:52:57.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:52:56 smithi120 ceph-mon[36634]: pgmap v396: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 938 B/s wr, 9 op/s 2024-06-10T02:52:57.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:52:56 smithi120 ceph-mon[36634]: from='client.? 172.21.15.52:0/1597005262' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T02:52:57.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:52:56 smithi188 ceph-mon[36599]: pgmap v396: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 938 B/s wr, 9 op/s 2024-06-10T02:52:57.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:52:56 smithi188 ceph-mon[36599]: from='client.? 172.21.15.52:0/1597005262' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T02:52:57.437 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T02:52:57.692 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:52:59.132 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:52:58 smithi052 ceph-mon[32362]: pgmap v397: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 938 B/s wr, 13 op/s 2024-06-10T02:52:59.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:52:58 smithi120 ceph-mon[36634]: pgmap v397: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 938 B/s wr, 13 op/s 2024-06-10T02:52:59.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:52:58 smithi188 ceph-mon[36599]: pgmap v397: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 938 B/s wr, 13 op/s 2024-06-10T02:52:59.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:59.404+0000 7f3631f22700 1 -- 172.21.15.52:0/1408866639 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f362c1024e0 msgr2=0x7f362c102960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:52:59.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:59.404+0000 7f3631f22700 1 --2- 172.21.15.52:0/1408866639 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f362c1024e0 0x7f362c102960 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f3614009990 tx=0x7f3614009ca0 comp rx=0 tx=0).stop 2024-06-10T02:52:59.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:59.404+0000 7f3631f22700 1 -- 172.21.15.52:0/1408866639 shutdown_connections 2024-06-10T02:52:59.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:59.404+0000 7f3631f22700 1 --2- 172.21.15.52:0/1408866639 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f362c102ea0 0x7f362c10f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:59.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:59.404+0000 7f3631f22700 1 --2- 172.21.15.52:0/1408866639 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f362c1085f0 0x7f362c1089d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:59.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:59.404+0000 7f3631f22700 1 --2- 172.21.15.52:0/1408866639 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f362c1024e0 0x7f362c102960 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:59.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:59.404+0000 7f3631f22700 1 -- 172.21.15.52:0/1408866639 >> 172.21.15.52:0/1408866639 conn(0x7f362c079640 msgr2=0x7f362c079a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:52:59.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:59.404+0000 7f3631f22700 1 -- 172.21.15.52:0/1408866639 shutdown_connections 2024-06-10T02:52:59.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:59.405+0000 7f3631f22700 1 -- 172.21.15.52:0/1408866639 wait complete. 2024-06-10T02:52:59.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:59.405+0000 7f3631f22700 1 Processor -- start 2024-06-10T02:52:59.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:59.406+0000 7f3631f22700 1 -- start start 2024-06-10T02:52:59.407 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:59.406+0000 7f3631f22700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f362c1024e0 0x7f362c19e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:52:59.407 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:59.406+0000 7f3631f22700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f362c102ea0 0x7f362c19edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:52:59.407 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:59.406+0000 7f3631f22700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f362c1085f0 0x7f362c198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:52:59.407 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:59.406+0000 7f3631f22700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f362c1127b0 con 0x7f362c1085f0 2024-06-10T02:52:59.407 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:59.406+0000 7f3631f22700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f362c112630 con 0x7f362c102ea0 2024-06-10T02:52:59.407 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:59.406+0000 7f3631f22700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f362c112930 con 0x7f362c1024e0 2024-06-10T02:52:59.407 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:59.406+0000 7f362bfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f362c1085f0 0x7f362c198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:52:59.407 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:59.406+0000 7f362bfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f362c1085f0 0x7f362c198a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:39100/0 (socket says 172.21.15.52:39100) 2024-06-10T02:52:59.407 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:59.406+0000 7f362bfff700 1 -- 172.21.15.52:0/3059332266 learned_addr learned my addr 172.21.15.52:0/3059332266 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:52:59.407 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:59.406+0000 7f362affd700 1 --2- 172.21.15.52:0/3059332266 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f362c102ea0 0x7f362c19edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:52:59.407 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:59.406+0000 7f362b7fe700 1 --2- 172.21.15.52:0/3059332266 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f362c1024e0 0x7f362c19e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:52:59.407 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:59.406+0000 7f362bfff700 1 -- 172.21.15.52:0/3059332266 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f362c1024e0 msgr2=0x7f362c19e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:52:59.408 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:59.406+0000 7f362bfff700 1 --2- 172.21.15.52:0/3059332266 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f362c1024e0 0x7f362c19e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:59.408 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:59.406+0000 7f362bfff700 1 -- 172.21.15.52:0/3059332266 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f362c102ea0 msgr2=0x7f362c19edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:52:59.408 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:59.406+0000 7f362bfff700 1 --2- 172.21.15.52:0/3059332266 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f362c102ea0 0x7f362c19edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:59.408 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:59.406+0000 7f362bfff700 1 -- 172.21.15.52:0/3059332266 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f362c199190 con 0x7f362c1085f0 2024-06-10T02:52:59.408 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:59.407+0000 7f362bfff700 1 --2- 172.21.15.52:0/3059332266 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f362c1085f0 0x7f362c198a30 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f362400d6d0 tx=0x7f362400d9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:52:59.408 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:59.407+0000 7f3628ff9700 1 -- 172.21.15.52:0/3059332266 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3624016070 con 0x7f362c1085f0 2024-06-10T02:52:59.408 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:59.407+0000 7f3628ff9700 1 -- 172.21.15.52:0/3059332266 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3624005020 con 0x7f362c1085f0 2024-06-10T02:52:59.408 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:59.407+0000 7f3631f22700 1 -- 172.21.15.52:0/3059332266 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f362c199480 con 0x7f362c1085f0 2024-06-10T02:52:59.408 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:59.407+0000 7f3628ff9700 1 -- 172.21.15.52:0/3059332266 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f362400fb20 con 0x7f362c1085f0 2024-06-10T02:52:59.408 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:59.407+0000 7f3631f22700 1 -- 172.21.15.52:0/3059332266 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f362c1a5740 con 0x7f362c1085f0 2024-06-10T02:52:59.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:59.408+0000 7f3631f22700 1 -- 172.21.15.52:0/3059332266 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f362c069180 con 0x7f362c1085f0 2024-06-10T02:52:59.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:59.409+0000 7f3628ff9700 1 -- 172.21.15.52:0/3059332266 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f36240051d0 con 0x7f362c1085f0 2024-06-10T02:52:59.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:59.410+0000 7f3628ff9700 1 --2- 172.21.15.52:0/3059332266 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f36180a2c20 0x7f36180a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:52:59.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:59.410+0000 7f362b7fe700 1 --2- 172.21.15.52:0/3059332266 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f36180a2c20 0x7f36180a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:52:59.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:59.410+0000 7f3628ff9700 1 -- 172.21.15.52:0/3059332266 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f36240cf3f0 con 0x7f362c1085f0 2024-06-10T02:52:59.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:59.410+0000 7f362b7fe700 1 --2- 172.21.15.52:0/3059332266 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f36180a2c20 0x7f36180a50e0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f361c009550 tx=0x7f361c0091b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:52:59.413 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:59.414+0000 7f3628ff9700 1 -- 172.21.15.52:0/3059332266 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f362401f020 con 0x7f362c1085f0 2024-06-10T02:52:59.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:59.666+0000 7f3631f22700 1 -- 172.21.15.52:0/3059332266 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f362c1a5910 con 0x7f362c1085f0 2024-06-10T02:52:59.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:59.666+0000 7f3628ff9700 1 -- 172.21.15.52:0/3059332266 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11938 (secure 0 0 0) 0x7f362409bf40 con 0x7f362c1085f0 2024-06-10T02:52:59.665 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-10T02:52:59.666 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/889673247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":889673247},{"type":"v1","addr":"172.21.15.52:6835","nonce":889673247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/3615663511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":3615663511},{"type":"v1","addr":"172.21.15.52:6837","nonce":3615663511}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T02:50:03.889766+0000","modified":"2024-06-10T02:51:11.420770+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24484,"mds_1":24469,"mds_2":24463,"mds_3":24449,"mds_4":24428},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14571":{"gid":14571,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6839/3059599042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":3059599042},{"type":"v1","addr":"172.21.15.52:6839","nonce":3059599042}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14583":{"gid":14583,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6841/3039275521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":3039275521},{"type":"v1","addr":"172.21.15.52:6841","nonce":3039275521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24383":{"gid":24383,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6833/2609002896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":2609002896},{"type":"v1","addr":"172.21.15.120:6833","nonce":2609002896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6835/1996370288","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":1996370288},{"type":"v1","addr":"172.21.15.120:6835","nonce":1996370288}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6833/529564979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":529564979},{"type":"v1","addr":"172.21.15.188:6833","nonce":529564979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":25,"addr":"172.21.15.120:6837/4132440952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":4132440952},{"type":"v1","addr":"172.21.15.120:6837","nonce":4132440952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"b","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.120:6839/1672436487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1672436487},{"type":"v1","addr":"172.21.15.120:6839","nonce":1672436487}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":2,"incarnation":22,"state":"up:active","state_seq":26,"addr":"172.21.15.188:6835/3904544259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":3904544259},{"type":"v1","addr":"172.21.15.188:6835","nonce":3904544259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.188:6837/819317726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":819317726},{"type":"v1","addr":"172.21.15.188:6837","nonce":819317726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.188:6839/310444836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":310444836},{"type":"v1","addr":"172.21.15.188:6839","nonce":310444836}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24484,"qdb_cluster":[24469,24449,24428,24463,24484]},"id":1}]} 2024-06-10T02:52:59.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:59.669+0000 7f3631f22700 1 -- 172.21.15.52:0/3059332266 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f36180a2c20 msgr2=0x7f36180a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:52:59.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:59.669+0000 7f3631f22700 1 --2- 172.21.15.52:0/3059332266 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f36180a2c20 0x7f36180a50e0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f361c009550 tx=0x7f361c0091b0 comp rx=0 tx=0).stop 2024-06-10T02:52:59.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:59.669+0000 7f3631f22700 1 -- 172.21.15.52:0/3059332266 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f362c1085f0 msgr2=0x7f362c198a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:52:59.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:59.669+0000 7f3631f22700 1 --2- 172.21.15.52:0/3059332266 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f362c1085f0 0x7f362c198a30 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f362400d6d0 tx=0x7f362400d9e0 comp rx=0 tx=0).stop 2024-06-10T02:52:59.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:59.669+0000 7f3631f22700 1 -- 172.21.15.52:0/3059332266 shutdown_connections 2024-06-10T02:52:59.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:59.669+0000 7f3631f22700 1 --2- 172.21.15.52:0/3059332266 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f36180a2c20 0x7f36180a50e0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:59.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:59.669+0000 7f3631f22700 1 --2- 172.21.15.52:0/3059332266 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f362c1085f0 0x7f362c198a30 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:59.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:59.669+0000 7f3631f22700 1 --2- 172.21.15.52:0/3059332266 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f362c1024e0 0x7f362c19e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:59.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:59.669+0000 7f3631f22700 1 --2- 172.21.15.52:0/3059332266 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f362c102ea0 0x7f362c19edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:52:59.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:59.669+0000 7f3631f22700 1 -- 172.21.15.52:0/3059332266 >> 172.21.15.52:0/3059332266 conn(0x7f362c079640 msgr2=0x7f362c100ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:52:59.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:59.669+0000 7f3631f22700 1 -- 172.21.15.52:0/3059332266 shutdown_connections 2024-06-10T02:52:59.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:52:59.669+0000 7f3631f22700 1 -- 172.21.15.52:0/3059332266 wait complete. 2024-06-10T02:52:59.669 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 29 2024-06-10T02:52:59.976 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:52:59 smithi052 ceph-mon[32362]: from='client.? 172.21.15.52:0/3059332266' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T02:53:00.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:52:59 smithi120 ceph-mon[36634]: from='client.? 172.21.15.52:0/3059332266' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T02:53:00.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:52:59 smithi188 ceph-mon[36599]: from='client.? 172.21.15.52:0/3059332266' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T02:53:00.392 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-10T02:53:00.648 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:53:01.103 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:53:00 smithi052 ceph-mon[32362]: pgmap v398: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 9 op/s 2024-06-10T02:53:01.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:53:00 smithi120 ceph-mon[36634]: pgmap v398: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 9 op/s 2024-06-10T02:53:01.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:53:00 smithi188 ceph-mon[36599]: pgmap v398: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 9 op/s 2024-06-10T02:53:02.513 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:02.512+0000 7f2f4fb0f700 1 -- 172.21.15.52:0/3299599268 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2f48102740 msgr2=0x7f2f48102b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:53:02.513 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:02.512+0000 7f2f4fb0f700 1 --2- 172.21.15.52:0/3299599268 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2f48102740 0x7f2f48102b20 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f2f440099c0 tx=0x7f2f44009cd0 comp rx=0 tx=0).stop 2024-06-10T02:53:02.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:02.512+0000 7f2f4fb0f700 1 -- 172.21.15.52:0/3299599268 shutdown_connections 2024-06-10T02:53:02.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:02.512+0000 7f2f4fb0f700 1 --2- 172.21.15.52:0/3299599268 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2f48107840 0x7f2f4810f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:02.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:02.512+0000 7f2f4fb0f700 1 --2- 172.21.15.52:0/3299599268 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2f48102740 0x7f2f48102b20 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:02.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:02.512+0000 7f2f4fb0f700 1 --2- 172.21.15.52:0/3299599268 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2f481030f0 0x7f2f48107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:02.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:02.512+0000 7f2f4fb0f700 1 -- 172.21.15.52:0/3299599268 >> 172.21.15.52:0/3299599268 conn(0x7f2f480795f0 msgr2=0x7f2f48079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:53:02.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:02.513+0000 7f2f4fb0f700 1 -- 172.21.15.52:0/3299599268 shutdown_connections 2024-06-10T02:53:02.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:02.513+0000 7f2f4fb0f700 1 -- 172.21.15.52:0/3299599268 wait complete. 2024-06-10T02:53:02.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:02.513+0000 7f2f4fb0f700 1 Processor -- start 2024-06-10T02:53:02.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:02.513+0000 7f2f4fb0f700 1 -- start start 2024-06-10T02:53:02.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:02.513+0000 7f2f4fb0f700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2f48102740 0x7f2f48074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:02.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:02.513+0000 7f2f4fb0f700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2f481030f0 0x7f2f48074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:02.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:02.513+0000 7f2f4fb0f700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2f48107840 0x7f2f48075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:02.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:02.513+0000 7f2f4fb0f700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f2f481127d0 con 0x7f2f48102740 2024-06-10T02:53:02.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:02.513+0000 7f2f4fb0f700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f2f48112650 con 0x7f2f48107840 2024-06-10T02:53:02.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:02.513+0000 7f2f4fb0f700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f2f48112950 con 0x7f2f481030f0 2024-06-10T02:53:02.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:02.514+0000 7f2f4d8ab700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2f48102740 0x7f2f48074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:02.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:02.514+0000 7f2f4d8ab700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2f48102740 0x7f2f48074790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:35676/0 (socket says 172.21.15.52:35676) 2024-06-10T02:53:02.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:02.514+0000 7f2f4d8ab700 1 -- 172.21.15.52:0/1979873921 learned_addr learned my addr 172.21.15.52:0/1979873921 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:53:02.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:02.514+0000 7f2f4d0aa700 1 --2- 172.21.15.52:0/1979873921 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2f481030f0 0x7f2f48074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:02.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:02.514+0000 7f2f4e0ac700 1 --2- 172.21.15.52:0/1979873921 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2f48107840 0x7f2f48075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:02.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:02.514+0000 7f2f4d8ab700 1 -- 172.21.15.52:0/1979873921 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2f481030f0 msgr2=0x7f2f48074cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:53:02.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:02.514+0000 7f2f4d8ab700 1 --2- 172.21.15.52:0/1979873921 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2f481030f0 0x7f2f48074cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:02.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:02.514+0000 7f2f4d8ab700 1 -- 172.21.15.52:0/1979873921 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2f48107840 msgr2=0x7f2f48075210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:53:02.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:02.514+0000 7f2f4d8ab700 1 --2- 172.21.15.52:0/1979873921 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2f48107840 0x7f2f48075210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:02.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:02.514+0000 7f2f4d8ab700 1 -- 172.21.15.52:0/1979873921 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f481a7130 con 0x7f2f48102740 2024-06-10T02:53:02.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:02.514+0000 7f2f4e0ac700 1 --2- 172.21.15.52:0/1979873921 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2f48107840 0x7f2f48075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T02:53:02.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:02.514+0000 7f2f4d0aa700 1 --2- 172.21.15.52:0/1979873921 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2f481030f0 0x7f2f48074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T02:53:02.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:02.514+0000 7f2f4d8ab700 1 --2- 172.21.15.52:0/1979873921 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2f48102740 0x7f2f48074790 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f2f44005910 tx=0x7f2f4400fab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:53:02.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:02.514+0000 7f2f3effd700 1 -- 172.21.15.52:0/1979873921 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f44027070 con 0x7f2f48102740 2024-06-10T02:53:02.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:02.514+0000 7f2f3effd700 1 -- 172.21.15.52:0/1979873921 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2f44003680 con 0x7f2f48102740 2024-06-10T02:53:02.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:02.514+0000 7f2f4fb0f700 1 -- 172.21.15.52:0/1979873921 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f481a73c0 con 0x7f2f48102740 2024-06-10T02:53:02.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:02.514+0000 7f2f3effd700 1 -- 172.21.15.52:0/1979873921 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f44017650 con 0x7f2f48102740 2024-06-10T02:53:02.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:02.514+0000 7f2f4fb0f700 1 -- 172.21.15.52:0/1979873921 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f481a7870 con 0x7f2f48102740 2024-06-10T02:53:02.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:02.515+0000 7f2f4fb0f700 1 -- 172.21.15.52:0/1979873921 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f4810aa80 con 0x7f2f48102740 2024-06-10T02:53:02.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:02.516+0000 7f2f3effd700 1 -- 172.21.15.52:0/1979873921 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f2f44022470 con 0x7f2f48102740 2024-06-10T02:53:02.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:02.517+0000 7f2f3effd700 1 --2- 172.21.15.52:0/1979873921 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f2f340a2ca0 0x7f2f340a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:02.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:02.517+0000 7f2f3effd700 1 -- 172.21.15.52:0/1979873921 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f2f440d01a0 con 0x7f2f48102740 2024-06-10T02:53:02.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:02.517+0000 7f2f4d0aa700 1 --2- 172.21.15.52:0/1979873921 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f2f340a2ca0 0x7f2f340a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:02.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:02.518+0000 7f2f4d0aa700 1 --2- 172.21.15.52:0/1979873921 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f2f340a2ca0 0x7f2f340a5160 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f2f48101f60 tx=0x7f2f3800b2c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:53:02.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:02.521+0000 7f2f3effd700 1 -- 172.21.15.52:0/1979873921 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f2f4405d850 con 0x7f2f48102740 2024-06-10T02:53:02.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:02.730+0000 7f2f4fb0f700 1 -- 172.21.15.52:0/1979873921 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2f48075cc0 con 0x7f2f48102740 2024-06-10T02:53:02.730 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:02.731+0000 7f2f3effd700 1 -- 172.21.15.52:0/1979873921 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v100) ==== 74+0+24739 (secure 0 0 0) 0x7f2f44013070 con 0x7f2f48102740 2024-06-10T02:53:02.731 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-10T02:53:02.731 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":100,"fsid":"d8676e9e-26d2-11ef-bc9b-c7b262605968","created":"2024-06-10T02:41:43.076837+0000","modified":"2024-06-10T02:51:45.708105+0000","last_up_change":"2024-06-10T02:48:57.297829+0000","last_in_change":"2024-06-10T02:48:36.865802+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-10T02:45:07.394690+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-10T02:49:57.926838+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-10T02:50:01.130393+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-10T02:50:10.655588+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e03c0691-cc5a-497a-bbc2-ce18c337f074","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":4255833408},{"type":"v1","addr":"172.21.15.52:6803","nonce":4255833408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":4255833408},{"type":"v1","addr":"172.21.15.52:6805","nonce":4255833408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":4255833408},{"type":"v1","addr":"172.21.15.52:6809","nonce":4255833408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":4255833408},{"type":"v1","addr":"172.21.15.52:6807","nonce":4255833408}]},"public_addr":"172.21.15.52:6803/4255833408","cluster_addr":"172.21.15.52:6805/4255833408","heartbeat_back_addr":"172.21.15.52:6809/4255833408","heartbeat_front_addr":"172.21.15.52:6807/4255833408","state":["exists","up"]},{"osd":1,"uuid":"a24d8da8-c5ec-44d7-ab30-56f6ce3efda7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":4116540053},{"type":"v1","addr":"172.21.15.120:6801","nonce":4116540053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":4116540053},{"type":"v1","addr":"172.21.15.120:6803","nonce":4116540053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":4116540053},{"type":"v1","addr":"172.21.15.120:6807","nonce":4116540053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":4116540053},{"type":"v1","addr":"172.21.15.120:6805","nonce":4116540053}]},"public_addr":"172.21.15.120:6801/4116540053","cluster_addr":"172.21.15.120:6803/4116540053","heartbeat_back_addr":"172.21.15.120:6807/4116540053","heartbeat_front_addr":"172.21.15.120:6805/4116540053","state":["exists","up"]},{"osd":2,"uuid":"a7162106-ccbf-45bd-bc0e-051ee43f4214","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":2624326999},{"type":"v1","addr":"172.21.15.188:6801","nonce":2624326999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":2624326999},{"type":"v1","addr":"172.21.15.188:6803","nonce":2624326999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":2624326999},{"type":"v1","addr":"172.21.15.188:6807","nonce":2624326999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":2624326999},{"type":"v1","addr":"172.21.15.188:6805","nonce":2624326999}]},"public_addr":"172.21.15.188:6801/2624326999","cluster_addr":"172.21.15.188:6803/2624326999","heartbeat_back_addr":"172.21.15.188:6807/2624326999","heartbeat_front_addr":"172.21.15.188:6805/2624326999","state":["exists","up"]},{"osd":3,"uuid":"53680244-b49a-4120-b24a-9ad7a6341dc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":116897011},{"type":"v1","addr":"172.21.15.52:6811","nonce":116897011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":116897011},{"type":"v1","addr":"172.21.15.52:6813","nonce":116897011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":116897011},{"type":"v1","addr":"172.21.15.52:6817","nonce":116897011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":116897011},{"type":"v1","addr":"172.21.15.52:6815","nonce":116897011}]},"public_addr":"172.21.15.52:6811/116897011","cluster_addr":"172.21.15.52:6813/116897011","heartbeat_back_addr":"172.21.15.52:6817/116897011","heartbeat_front_addr":"172.21.15.52:6815/116897011","state":["exists","up"]},{"osd":4,"uuid":"9f72f1f4-2ef3-49c5-8410-c108767d2ef5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":1320690621},{"type":"v1","addr":"172.21.15.120:6809","nonce":1320690621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":1320690621},{"type":"v1","addr":"172.21.15.120:6811","nonce":1320690621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":1320690621},{"type":"v1","addr":"172.21.15.120:6815","nonce":1320690621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":1320690621},{"type":"v1","addr":"172.21.15.120:6813","nonce":1320690621}]},"public_addr":"172.21.15.120:6809/1320690621","cluster_addr":"172.21.15.120:6811/1320690621","heartbeat_back_addr":"172.21.15.120:6815/1320690621","heartbeat_front_addr":"172.21.15.120:6813/1320690621","state":["exists","up"]},{"osd":5,"uuid":"001b5706-7f27-4fa4-9b69-969e6d765abc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3951947483},{"type":"v1","addr":"172.21.15.188:6809","nonce":3951947483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3951947483},{"type":"v1","addr":"172.21.15.188:6811","nonce":3951947483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3951947483},{"type":"v1","addr":"172.21.15.188:6815","nonce":3951947483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3951947483},{"type":"v1","addr":"172.21.15.188:6813","nonce":3951947483}]},"public_addr":"172.21.15.188:6809/3951947483","cluster_addr":"172.21.15.188:6811/3951947483","heartbeat_back_addr":"172.21.15.188:6815/3951947483","heartbeat_front_addr":"172.21.15.188:6813/3951947483","state":["exists","up"]},{"osd":6,"uuid":"140c903e-9b31-4a81-9ccd-2edc0a2decf9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":359084919},{"type":"v1","addr":"172.21.15.52:6819","nonce":359084919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":359084919},{"type":"v1","addr":"172.21.15.52:6821","nonce":359084919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":359084919},{"type":"v1","addr":"172.21.15.52:6825","nonce":359084919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":359084919},{"type":"v1","addr":"172.21.15.52:6823","nonce":359084919}]},"public_addr":"172.21.15.52:6819/359084919","cluster_addr":"172.21.15.52:6821/359084919","heartbeat_back_addr":"172.21.15.52:6825/359084919","heartbeat_front_addr":"172.21.15.52:6823/359084919","state":["exists","up"]},{"osd":7,"uuid":"57b735d4-5072-4ca2-a3ff-3763ad84f15a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":3737722011},{"type":"v1","addr":"172.21.15.120:6817","nonce":3737722011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6818","nonce":3737722011},{"type":"v1","addr":"172.21.15.120:6819","nonce":3737722011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":3737722011},{"type":"v1","addr":"172.21.15.120:6823","nonce":3737722011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":3737722011},{"type":"v1","addr":"172.21.15.120:6821","nonce":3737722011}]},"public_addr":"172.21.15.120:6817/3737722011","cluster_addr":"172.21.15.120:6819/3737722011","heartbeat_back_addr":"172.21.15.120:6823/3737722011","heartbeat_front_addr":"172.21.15.120:6821/3737722011","state":["exists","up"]},{"osd":8,"uuid":"4ffbb43b-854a-4eac-99cf-7af89bc65409","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1444828441},{"type":"v1","addr":"172.21.15.188:6817","nonce":1444828441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1444828441},{"type":"v1","addr":"172.21.15.188:6819","nonce":1444828441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1444828441},{"type":"v1","addr":"172.21.15.188:6823","nonce":1444828441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1444828441},{"type":"v1","addr":"172.21.15.188:6821","nonce":1444828441}]},"public_addr":"172.21.15.188:6817/1444828441","cluster_addr":"172.21.15.188:6819/1444828441","heartbeat_back_addr":"172.21.15.188:6823/1444828441","heartbeat_front_addr":"172.21.15.188:6821/1444828441","state":["exists","up"]},{"osd":9,"uuid":"5bfd985c-8acc-4d00-8561-40bbd541c07b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":924440658},{"type":"v1","addr":"172.21.15.52:6827","nonce":924440658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":924440658},{"type":"v1","addr":"172.21.15.52:6829","nonce":924440658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":924440658},{"type":"v1","addr":"172.21.15.52:6833","nonce":924440658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":924440658},{"type":"v1","addr":"172.21.15.52:6831","nonce":924440658}]},"public_addr":"172.21.15.52:6827/924440658","cluster_addr":"172.21.15.52:6829/924440658","heartbeat_back_addr":"172.21.15.52:6833/924440658","heartbeat_front_addr":"172.21.15.52:6831/924440658","state":["exists","up"]},{"osd":10,"uuid":"0180cddf-e109-42cc-829a-185604276f02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6824","nonce":1531769083},{"type":"v1","addr":"172.21.15.120:6825","nonce":1531769083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":1531769083},{"type":"v1","addr":"172.21.15.120:6827","nonce":1531769083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":1531769083},{"type":"v1","addr":"172.21.15.120:6831","nonce":1531769083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6828","nonce":1531769083},{"type":"v1","addr":"172.21.15.120:6829","nonce":1531769083}]},"public_addr":"172.21.15.120:6825/1531769083","cluster_addr":"172.21.15.120:6827/1531769083","heartbeat_back_addr":"172.21.15.120:6831/1531769083","heartbeat_front_addr":"172.21.15.120:6829/1531769083","state":["exists","up"]},{"osd":11,"uuid":"73aec750-82d1-4800-b6a0-4e305912f9ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":3769050170},{"type":"v1","addr":"172.21.15.188:6825","nonce":3769050170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":3769050170},{"type":"v1","addr":"172.21.15.188:6827","nonce":3769050170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":3769050170},{"type":"v1","addr":"172.21.15.188:6831","nonce":3769050170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":3769050170},{"type":"v1","addr":"172.21.15.188:6829","nonce":3769050170}]},"public_addr":"172.21.15.188:6825/3769050170","cluster_addr":"172.21.15.188:6827/3769050170","heartbeat_back_addr":"172.21.15.188:6831/3769050170","heartbeat_front_addr":"172.21.15.188:6829/3769050170","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:44:16.455752+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:44:40.219735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:45:03.558899+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:45:29.155004+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:45:53.981098+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:46:19.412020+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:46:45.462640+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:47:11.484578+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:47:36.879054+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:48:03.392446+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:48:29.422173+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:48:55.523423+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":4}]},{"pgid":"3.6","mappings":[{"from":3,"to":7}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.9","mappings":[{"from":5,"to":8}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.52:0/3535271671":"2024-06-11T02:42:42.872312+0000","172.21.15.52:0/1092934380":"2024-06-11T02:42:42.872312+0000","172.21.15.52:6800/896317057":"2024-06-11T02:42:42.872312+0000","172.21.15.52:6801/3934479642":"2024-06-11T02:42:15.498118+0000","172.21.15.52:6800/3934479642":"2024-06-11T02:42:15.498118+0000","172.21.15.52:0/2759694132":"2024-06-11T02:42:15.498118+0000","172.21.15.52:0/2593665631":"2024-06-11T02:42:15.498118+0000","172.21.15.52:0/3151504634":"2024-06-11T02:42:15.498118+0000","172.21.15.52:6801/896317057":"2024-06-11T02:42:42.872312+0000","172.21.15.52:0/1254480982":"2024-06-11T02:42:42.872312+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-10T02:53:02.733 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:02.734+0000 7f2f4fb0f700 1 -- 172.21.15.52:0/1979873921 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f2f340a2ca0 msgr2=0x7f2f340a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:53:02.733 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:02.734+0000 7f2f4fb0f700 1 --2- 172.21.15.52:0/1979873921 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f2f340a2ca0 0x7f2f340a5160 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f2f48101f60 tx=0x7f2f3800b2c0 comp rx=0 tx=0).stop 2024-06-10T02:53:02.733 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:02.734+0000 7f2f4fb0f700 1 -- 172.21.15.52:0/1979873921 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2f48102740 msgr2=0x7f2f48074790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:53:02.733 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:02.734+0000 7f2f4fb0f700 1 --2- 172.21.15.52:0/1979873921 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2f48102740 0x7f2f48074790 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f2f44005910 tx=0x7f2f4400fab0 comp rx=0 tx=0).stop 2024-06-10T02:53:02.733 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:02.734+0000 7f2f4fb0f700 1 -- 172.21.15.52:0/1979873921 shutdown_connections 2024-06-10T02:53:02.733 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:02.734+0000 7f2f4fb0f700 1 --2- 172.21.15.52:0/1979873921 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2f48107840 0x7f2f48075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:02.733 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:02.734+0000 7f2f4fb0f700 1 --2- 172.21.15.52:0/1979873921 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f2f340a2ca0 0x7f2f340a5160 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:02.733 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:02.734+0000 7f2f4fb0f700 1 --2- 172.21.15.52:0/1979873921 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2f48102740 0x7f2f48074790 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:02.734 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:02.734+0000 7f2f4fb0f700 1 --2- 172.21.15.52:0/1979873921 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2f481030f0 0x7f2f48074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:02.734 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:02.734+0000 7f2f4fb0f700 1 -- 172.21.15.52:0/1979873921 >> 172.21.15.52:0/1979873921 conn(0x7f2f480795f0 msgr2=0x7f2f48108990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:53:02.734 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:02.734+0000 7f2f4fb0f700 1 -- 172.21.15.52:0/1979873921 shutdown_connections 2024-06-10T02:53:02.734 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:02.734+0000 7f2f4fb0f700 1 -- 172.21.15.52:0/1979873921 wait complete. 2024-06-10T02:53:03.024 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:53:02 smithi052 ceph-mon[32362]: pgmap v399: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-10T02:53:03.024 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:53:02 smithi052 ceph-mon[32362]: from='client.? 172.21.15.52:0/1979873921' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T02:53:03.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:53:02 smithi120 ceph-mon[36634]: pgmap v399: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-10T02:53:03.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:53:02 smithi120 ceph-mon[36634]: from='client.? 172.21.15.52:0/1979873921' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T02:53:03.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:53:02 smithi188 ceph-mon[36599]: pgmap v399: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-10T02:53:03.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:53:02 smithi188 ceph-mon[36599]: from='client.? 172.21.15.52:0/1979873921' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T02:53:03.447 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T02:53:03.705 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:53:05.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:53:04 smithi052 ceph-mon[32362]: pgmap v400: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-06-10T02:53:05.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:53:04 smithi120 ceph-mon[36634]: pgmap v400: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-06-10T02:53:05.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:53:04 smithi188 ceph-mon[36599]: pgmap v400: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-06-10T02:53:05.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:05.347+0000 7f659502a700 1 -- 172.21.15.52:0/1878322466 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6590108500 msgr2=0x7f65901088e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:53:05.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:05.347+0000 7f659502a700 1 --2- 172.21.15.52:0/1878322466 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6590108500 0x7f65901088e0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f6580009990 tx=0x7f6580009ca0 comp rx=0 tx=0).stop 2024-06-10T02:53:05.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:05.347+0000 7f659502a700 1 -- 172.21.15.52:0/1878322466 shutdown_connections 2024-06-10T02:53:05.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:05.347+0000 7f659502a700 1 --2- 172.21.15.52:0/1878322466 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f659006b5a0 0x7f659010f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:05.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:05.347+0000 7f659502a700 1 --2- 172.21.15.52:0/1878322466 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6590108500 0x7f65901088e0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:05.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:05.347+0000 7f659502a700 1 --2- 172.21.15.52:0/1878322466 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f659006abe0 0x7f659006b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:05.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:05.347+0000 7f659502a700 1 -- 172.21.15.52:0/1878322466 >> 172.21.15.52:0/1878322466 conn(0x7f6590078360 msgr2=0x7f6590078770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:53:05.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:05.347+0000 7f659502a700 1 -- 172.21.15.52:0/1878322466 shutdown_connections 2024-06-10T02:53:05.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:05.347+0000 7f659502a700 1 -- 172.21.15.52:0/1878322466 wait complete. 2024-06-10T02:53:05.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:05.348+0000 7f659502a700 1 Processor -- start 2024-06-10T02:53:05.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:05.348+0000 7f659502a700 1 -- start start 2024-06-10T02:53:05.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:05.348+0000 7f659502a700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f659006abe0 0x7f659019e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:05.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:05.348+0000 7f659502a700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f659006b5a0 0x7f659019ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:05.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:05.348+0000 7f659502a700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6590108500 0x7f6590198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:05.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:05.348+0000 7f659502a700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f65901127e0 con 0x7f6590108500 2024-06-10T02:53:05.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:05.348+0000 7f659502a700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f6590112660 con 0x7f659006b5a0 2024-06-10T02:53:05.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:05.348+0000 7f659502a700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f6590112960 con 0x7f659006abe0 2024-06-10T02:53:05.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:05.348+0000 7f658f59e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6590108500 0x7f6590198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:05.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:05.348+0000 7f658f59e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6590108500 0x7f6590198b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:35694/0 (socket says 172.21.15.52:35694) 2024-06-10T02:53:05.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:05.348+0000 7f658f59e700 1 -- 172.21.15.52:0/967790331 learned_addr learned my addr 172.21.15.52:0/967790331 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:53:05.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:05.348+0000 7f658659c700 1 --2- 172.21.15.52:0/967790331 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f659006b5a0 0x7f659019ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:05.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:05.348+0000 7f658ed9d700 1 --2- 172.21.15.52:0/967790331 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f659006abe0 0x7f659019e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:05.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:05.349+0000 7f658f59e700 1 -- 172.21.15.52:0/967790331 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f659006abe0 msgr2=0x7f659019e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:53:05.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:05.349+0000 7f658f59e700 1 --2- 172.21.15.52:0/967790331 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f659006abe0 0x7f659019e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:05.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:05.349+0000 7f658f59e700 1 -- 172.21.15.52:0/967790331 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f659006b5a0 msgr2=0x7f659019ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:53:05.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:05.349+0000 7f658f59e700 1 --2- 172.21.15.52:0/967790331 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f659006b5a0 0x7f659019ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:05.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:05.349+0000 7f658f59e700 1 -- 172.21.15.52:0/967790331 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6590199260 con 0x7f6590108500 2024-06-10T02:53:05.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:05.349+0000 7f658f59e700 1 --2- 172.21.15.52:0/967790331 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6590108500 0x7f6590198b00 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f658800b970 tx=0x7f658800bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:53:05.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:05.349+0000 7f658cd99700 1 -- 172.21.15.52:0/967790331 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f658800d860 con 0x7f6590108500 2024-06-10T02:53:05.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:05.349+0000 7f659502a700 1 -- 172.21.15.52:0/967790331 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6590199550 con 0x7f6590108500 2024-06-10T02:53:05.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:05.349+0000 7f658cd99700 1 -- 172.21.15.52:0/967790331 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f658800e430 con 0x7f6590108500 2024-06-10T02:53:05.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:05.349+0000 7f658cd99700 1 -- 172.21.15.52:0/967790331 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6588017660 con 0x7f6590108500 2024-06-10T02:53:05.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:05.349+0000 7f659502a700 1 -- 172.21.15.52:0/967790331 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65901a5810 con 0x7f6590108500 2024-06-10T02:53:05.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:05.351+0000 7f659502a700 1 -- 172.21.15.52:0/967790331 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f659010a9e0 con 0x7f6590108500 2024-06-10T02:53:05.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:05.352+0000 7f658cd99700 1 -- 172.21.15.52:0/967790331 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f658800e5e0 con 0x7f6590108500 2024-06-10T02:53:05.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:05.352+0000 7f658cd99700 1 --2- 172.21.15.52:0/967790331 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f657c0a2c20 0x7f657c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:05.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:05.352+0000 7f658ed9d700 1 --2- 172.21.15.52:0/967790331 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f657c0a2c20 0x7f657c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:05.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:05.352+0000 7f658cd99700 1 -- 172.21.15.52:0/967790331 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f65880cef80 con 0x7f6590108500 2024-06-10T02:53:05.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:05.353+0000 7f658ed9d700 1 --2- 172.21.15.52:0/967790331 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f657c0a2c20 0x7f657c0a50e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f658000b5c0 tx=0x7f6580005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:53:05.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:05.357+0000 7f658cd99700 1 -- 172.21.15.52:0/967790331 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f6588013030 con 0x7f6590108500 2024-06-10T02:53:05.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:05.609+0000 7f659502a700 1 -- 172.21.15.52:0/967790331 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6590199ff0 con 0x7f6590108500 2024-06-10T02:53:05.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:05.610+0000 7f658cd99700 1 -- 172.21.15.52:0/967790331 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11938 (secure 0 0 0) 0x7f658809bb30 con 0x7f6590108500 2024-06-10T02:53:05.609 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-10T02:53:05.609 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/889673247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":889673247},{"type":"v1","addr":"172.21.15.52:6835","nonce":889673247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/3615663511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":3615663511},{"type":"v1","addr":"172.21.15.52:6837","nonce":3615663511}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T02:50:03.889766+0000","modified":"2024-06-10T02:51:11.420770+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24484,"mds_1":24469,"mds_2":24463,"mds_3":24449,"mds_4":24428},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14571":{"gid":14571,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6839/3059599042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":3059599042},{"type":"v1","addr":"172.21.15.52:6839","nonce":3059599042}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14583":{"gid":14583,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6841/3039275521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":3039275521},{"type":"v1","addr":"172.21.15.52:6841","nonce":3039275521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24383":{"gid":24383,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6833/2609002896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":2609002896},{"type":"v1","addr":"172.21.15.120:6833","nonce":2609002896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6835/1996370288","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":1996370288},{"type":"v1","addr":"172.21.15.120:6835","nonce":1996370288}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6833/529564979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":529564979},{"type":"v1","addr":"172.21.15.188:6833","nonce":529564979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":25,"addr":"172.21.15.120:6837/4132440952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":4132440952},{"type":"v1","addr":"172.21.15.120:6837","nonce":4132440952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"b","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.120:6839/1672436487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1672436487},{"type":"v1","addr":"172.21.15.120:6839","nonce":1672436487}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":2,"incarnation":22,"state":"up:active","state_seq":26,"addr":"172.21.15.188:6835/3904544259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":3904544259},{"type":"v1","addr":"172.21.15.188:6835","nonce":3904544259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.188:6837/819317726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":819317726},{"type":"v1","addr":"172.21.15.188:6837","nonce":819317726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.188:6839/310444836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":310444836},{"type":"v1","addr":"172.21.15.188:6839","nonce":310444836}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24484,"qdb_cluster":[24469,24449,24428,24463,24484]},"id":1}]} 2024-06-10T02:53:05.611 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:05.612+0000 7f659502a700 1 -- 172.21.15.52:0/967790331 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f657c0a2c20 msgr2=0x7f657c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:53:05.611 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:05.612+0000 7f659502a700 1 --2- 172.21.15.52:0/967790331 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f657c0a2c20 0x7f657c0a50e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f658000b5c0 tx=0x7f6580005960 comp rx=0 tx=0).stop 2024-06-10T02:53:05.611 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:05.612+0000 7f659502a700 1 -- 172.21.15.52:0/967790331 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6590108500 msgr2=0x7f6590198b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:53:05.611 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:05.612+0000 7f659502a700 1 --2- 172.21.15.52:0/967790331 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6590108500 0x7f6590198b00 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f658800b970 tx=0x7f658800bc80 comp rx=0 tx=0).stop 2024-06-10T02:53:05.611 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:05.612+0000 7f659502a700 1 -- 172.21.15.52:0/967790331 shutdown_connections 2024-06-10T02:53:05.612 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:05.612+0000 7f659502a700 1 --2- 172.21.15.52:0/967790331 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f657c0a2c20 0x7f657c0a50e0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:05.612 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:05.612+0000 7f659502a700 1 --2- 172.21.15.52:0/967790331 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6590108500 0x7f6590198b00 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:05.612 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:05.612+0000 7f659502a700 1 --2- 172.21.15.52:0/967790331 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f659006abe0 0x7f659019e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:05.612 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:05.612+0000 7f659502a700 1 --2- 172.21.15.52:0/967790331 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f659006b5a0 0x7f659019ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:05.612 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:05.612+0000 7f659502a700 1 -- 172.21.15.52:0/967790331 >> 172.21.15.52:0/967790331 conn(0x7f6590078360 msgr2=0x7f65901000c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:53:05.612 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:05.612+0000 7f659502a700 1 -- 172.21.15.52:0/967790331 shutdown_connections 2024-06-10T02:53:05.612 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:05.612+0000 7f659502a700 1 -- 172.21.15.52:0/967790331 wait complete. 2024-06-10T02:53:05.612 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 29 2024-06-10T02:53:05.976 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:53:05 smithi052 ceph-mon[32362]: from='client.? 172.21.15.52:0/967790331' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T02:53:06.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:53:05 smithi120 ceph-mon[36634]: from='client.? 172.21.15.52:0/967790331' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T02:53:06.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:53:05 smithi188 ceph-mon[36599]: from='client.? 172.21.15.52:0/967790331' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T02:53:06.429 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 29, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-06-10T02:50:03.889766+0000', 'modified': '2024-06-10T02:51:11.420770+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24484, 'mds_1': 24469, 'mds_2': 24463, 'mds_3': 24449, 'mds_4': 24428}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14571': {'gid': 14571, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.52:6839/3059599042', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.52:6838', 'nonce': 3059599042}, {'type': 'v1', 'addr': '172.21.15.52:6839', 'nonce': 3059599042}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14583': {'gid': 14583, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.52:6841/3039275521', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.52:6840', 'nonce': 3039275521}, {'type': 'v1', 'addr': '172.21.15.52:6841', 'nonce': 3039275521}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24383': {'gid': 24383, 'name': 'k', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.120:6833/2609002896', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6832', 'nonce': 2609002896}, {'type': 'v1', 'addr': '172.21.15.120:6833', 'nonce': 2609002896}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24401': {'gid': 24401, 'name': 'e', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.120:6835/1996370288', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6834', 'nonce': 1996370288}, {'type': 'v1', 'addr': '172.21.15.120:6835', 'nonce': 1996370288}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24427': {'gid': 24427, 'name': 'l', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.188:6833/529564979', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6832', 'nonce': 529564979}, {'type': 'v1', 'addr': '172.21.15.188:6833', 'nonce': 529564979}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24428': {'gid': 24428, 'name': 'h', 'rank': 4, 'incarnation': 26, 'state': 'up:active', 'state_seq': 25, 'addr': '172.21.15.120:6837/4132440952', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6836', 'nonce': 4132440952}, {'type': 'v1', 'addr': '172.21.15.120:6837', 'nonce': 4132440952}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24449': {'gid': 24449, 'name': 'b', 'rank': 3, 'incarnation': 24, 'state': 'up:active', 'state_seq': 22, 'addr': '172.21.15.120:6839/1672436487', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6838', 'nonce': 1672436487}, {'type': 'v1', 'addr': '172.21.15.120:6839', 'nonce': 1672436487}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24463': {'gid': 24463, 'name': 'f', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 26, 'addr': '172.21.15.188:6835/3904544259', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6834', 'nonce': 3904544259}, {'type': 'v1', 'addr': '172.21.15.188:6835', 'nonce': 3904544259}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24469': {'gid': 24469, 'name': 'c', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 25, 'addr': '172.21.15.188:6837/819317726', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6836', 'nonce': 819317726}, {'type': 'v1', 'addr': '172.21.15.188:6837', 'nonce': 819317726}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24484': {'gid': 24484, 'name': 'i', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.188:6839/310444836', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6838', 'nonce': 310444836}, {'type': 'v1', 'addr': '172.21.15.188:6839', 'nonce': 310444836}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24484, 'qdb_cluster': [24469, 24449, 24428, 24463, 24484]} 2024-06-10T02:53:06.429 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-06-10T02:53:06.429 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph tell mds.h status 2024-06-10T02:53:06.682 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:53:07.056 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:53:06 smithi052 ceph-mon[32362]: pgmap v401: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-10T02:53:07.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:53:06 smithi120 ceph-mon[36634]: pgmap v401: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-10T02:53:07.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:53:06 smithi188 ceph-mon[36599]: pgmap v401: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-10T02:53:08.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.353+0000 7fde5dc9a700 1 -- 172.21.15.52:0/2873071810 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fde5810ee90 msgr2=0x7fde58111280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:53:08.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.353+0000 7fde5dc9a700 1 --2- 172.21.15.52:0/2873071810 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fde5810ee90 0x7fde58111280 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fde5000b0a0 tx=0x7fde5000b3b0 comp rx=0 tx=0).stop 2024-06-10T02:53:08.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.353+0000 7fde5dc9a700 1 -- 172.21.15.52:0/2873071810 shutdown_connections 2024-06-10T02:53:08.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.353+0000 7fde5dc9a700 1 --2- 172.21.15.52:0/2873071810 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fde5810ee90 0x7fde58111280 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:08.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.353+0000 7fde5dc9a700 1 --2- 172.21.15.52:0/2873071810 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fde580746c0 0x7fde58074aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:08.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.353+0000 7fde5dc9a700 1 --2- 172.21.15.52:0/2873071810 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fde58074fe0 0x7fde5810e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:08.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.353+0000 7fde5dc9a700 1 -- 172.21.15.52:0/2873071810 >> 172.21.15.52:0/2873071810 conn(0x7fde580795f0 msgr2=0x7fde58079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:53:08.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.353+0000 7fde5dc9a700 1 -- 172.21.15.52:0/2873071810 shutdown_connections 2024-06-10T02:53:08.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.353+0000 7fde5dc9a700 1 -- 172.21.15.52:0/2873071810 wait complete. 2024-06-10T02:53:08.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.354+0000 7fde5dc9a700 1 Processor -- start 2024-06-10T02:53:08.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.354+0000 7fde5dc9a700 1 -- start start 2024-06-10T02:53:08.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.354+0000 7fde5dc9a700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fde580746c0 0x7fde5819ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:08.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.354+0000 7fde5dc9a700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fde58074fe0 0x7fde5819b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:08.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.354+0000 7fde5dc9a700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fde5810ee90 0x7fde5819f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:08.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.354+0000 7fde5dc9a700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fde581149c0 con 0x7fde58074fe0 2024-06-10T02:53:08.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.354+0000 7fde5dc9a700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fde58114840 con 0x7fde5810ee90 2024-06-10T02:53:08.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.354+0000 7fde5dc9a700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fde58114b40 con 0x7fde580746c0 2024-06-10T02:53:08.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.355+0000 7fde56ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fde58074fe0 0x7fde5819b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:08.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.355+0000 7fde56ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fde58074fe0 0x7fde5819b2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:35712/0 (socket says 172.21.15.52:35712) 2024-06-10T02:53:08.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.355+0000 7fde56ffd700 1 -- 172.21.15.52:0/2996341058 learned_addr learned my addr 172.21.15.52:0/2996341058 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:53:08.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.355+0000 7fde577fe700 1 --2- 172.21.15.52:0/2996341058 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fde580746c0 0x7fde5819ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:08.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.355+0000 7fde57fff700 1 --2- 172.21.15.52:0/2996341058 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fde5810ee90 0x7fde5819f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:08.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.355+0000 7fde56ffd700 1 -- 172.21.15.52:0/2996341058 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fde580746c0 msgr2=0x7fde5819ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:53:08.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.355+0000 7fde56ffd700 1 --2- 172.21.15.52:0/2996341058 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fde580746c0 0x7fde5819ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:08.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.355+0000 7fde56ffd700 1 -- 172.21.15.52:0/2996341058 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fde5810ee90 msgr2=0x7fde5819f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:53:08.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.355+0000 7fde56ffd700 1 --2- 172.21.15.52:0/2996341058 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fde5810ee90 0x7fde5819f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:08.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.355+0000 7fde56ffd700 1 -- 172.21.15.52:0/2996341058 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde5819fda0 con 0x7fde58074fe0 2024-06-10T02:53:08.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.355+0000 7fde56ffd700 1 --2- 172.21.15.52:0/2996341058 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fde58074fe0 0x7fde5819b2d0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fde4c00d980 tx=0x7fde4c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:53:08.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.355+0000 7fde54ff9700 1 -- 172.21.15.52:0/2996341058 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde4c016070 con 0x7fde58074fe0 2024-06-10T02:53:08.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.355+0000 7fde54ff9700 1 -- 172.21.15.52:0/2996341058 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde4c004d10 con 0x7fde58074fe0 2024-06-10T02:53:08.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.355+0000 7fde5dc9a700 1 -- 172.21.15.52:0/2996341058 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde581a0090 con 0x7fde58074fe0 2024-06-10T02:53:08.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.355+0000 7fde54ff9700 1 -- 172.21.15.52:0/2996341058 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde4c0055f0 con 0x7fde58074fe0 2024-06-10T02:53:08.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.355+0000 7fde5dc9a700 1 -- 172.21.15.52:0/2996341058 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde581a7910 con 0x7fde58074fe0 2024-06-10T02:53:08.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.358+0000 7fde54ff9700 1 -- 172.21.15.52:0/2996341058 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fde4c004750 con 0x7fde58074fe0 2024-06-10T02:53:08.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.358+0000 7fde54ff9700 1 --2- 172.21.15.52:0/2996341058 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fde400a2ca0 0x7fde400a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:08.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.358+0000 7fde577fe700 1 --2- 172.21.15.52:0/2996341058 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fde400a2ca0 0x7fde400a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:08.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.358+0000 7fde54ff9700 1 -- 172.21.15.52:0/2996341058 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7fde4c0cf120 con 0x7fde58074fe0 2024-06-10T02:53:08.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.359+0000 7fde577fe700 1 --2- 172.21.15.52:0/2996341058 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fde400a2ca0 0x7fde400a5160 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fde48005950 tx=0x7fde4800a300 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:53:08.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.359+0000 7fde6851db80 1 Processor -- start 2024-06-10T02:53:08.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.359+0000 7fde6851db80 1 -- start start 2024-06-10T02:53:08.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.359+0000 7fde6851db80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5623cb9a6f10 0x5623cb9a72f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:08.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.359+0000 7fde6851db80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5623cb9a7830 0x5623cb9a1210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:08.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.359+0000 7fde6851db80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5623cb9a1750 0x5623cb9a1b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:08.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.359+0000 7fde6851db80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5623cb893f10 con 0x5623cb9a7830 2024-06-10T02:53:08.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.359+0000 7fde6851db80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5623cb8f2800 con 0x5623cb9a6f10 2024-06-10T02:53:08.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.359+0000 7fde6851db80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5623cb85a410 con 0x5623cb9a1750 2024-06-10T02:53:08.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.359+0000 7fde56ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5623cb9a7830 0x5623cb9a1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:08.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.359+0000 7fde56ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5623cb9a7830 0x5623cb9a1210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:35720/0 (socket says 172.21.15.52:35720) 2024-06-10T02:53:08.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.359+0000 7fde56ffd700 1 -- 172.21.15.52:0/4165302477 learned_addr learned my addr 172.21.15.52:0/4165302477 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:53:08.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.359+0000 7fde57fff700 1 --2- 172.21.15.52:0/4165302477 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5623cb9a1750 0x5623cb9a1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:08.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.359+0000 7fde577fe700 1 --2- 172.21.15.52:0/4165302477 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5623cb9a6f10 0x5623cb9a72f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:08.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.360+0000 7fde56ffd700 1 -- 172.21.15.52:0/4165302477 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5623cb9a1750 msgr2=0x5623cb9a1b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:53:08.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.360+0000 7fde56ffd700 1 --2- 172.21.15.52:0/4165302477 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5623cb9a1750 0x5623cb9a1b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:08.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.360+0000 7fde56ffd700 1 -- 172.21.15.52:0/4165302477 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5623cb9a6f10 msgr2=0x5623cb9a72f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:53:08.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.360+0000 7fde56ffd700 1 --2- 172.21.15.52:0/4165302477 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5623cb9a6f10 0x5623cb9a72f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:08.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.360+0000 7fde56ffd700 1 -- 172.21.15.52:0/4165302477 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5623cb841880 con 0x5623cb9a7830 2024-06-10T02:53:08.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.360+0000 7fde56ffd700 1 --2- 172.21.15.52:0/4165302477 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5623cb9a7830 0x5623cb9a1210 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fde4c0a3d10 tx=0x7fde4c09b190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=113509871d352953 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:53:08.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.360+0000 7fde3dffb700 1 -- 172.21.15.52:0/4165302477 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde4c09b5d0 con 0x5623cb9a7830 2024-06-10T02:53:08.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.360+0000 7fde3dffb700 1 -- 172.21.15.52:0/4165302477 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde4c09f980 con 0x5623cb9a7830 2024-06-10T02:53:08.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.360+0000 7fde3dffb700 1 -- 172.21.15.52:0/4165302477 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde4c09b830 con 0x5623cb9a7830 2024-06-10T02:53:08.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.361+0000 7fde6851db80 1 -- 172.21.15.52:0/4165302477 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5623cb9a7830 msgr2=0x5623cb9a1210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:53:08.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.361+0000 7fde6851db80 1 --2- 172.21.15.52:0/4165302477 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5623cb9a7830 0x5623cb9a1210 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fde4c0a3d10 tx=0x7fde4c09b190 comp rx=0 tx=0).stop 2024-06-10T02:53:08.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.361+0000 7fde6851db80 1 -- 172.21.15.52:0/4165302477 shutdown_connections 2024-06-10T02:53:08.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.361+0000 7fde6851db80 1 --2- 172.21.15.52:0/4165302477 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5623cb9a1750 0x5623cb9a1b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:08.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.361+0000 7fde6851db80 1 --2- 172.21.15.52:0/4165302477 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5623cb9a6f10 0x5623cb9a72f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:08.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.361+0000 7fde6851db80 1 --2- 172.21.15.52:0/4165302477 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5623cb9a7830 0x5623cb9a1210 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:08.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.361+0000 7fde6851db80 1 -- 172.21.15.52:0/4165302477 >> 172.21.15.52:0/4165302477 conn(0x5623cb8ce030 msgr2=0x5623cb99c1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:53:08.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.361+0000 7fde6851db80 1 -- 172.21.15.52:0/4165302477 shutdown_connections 2024-06-10T02:53:08.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.361+0000 7fde6851db80 1 -- 172.21.15.52:0/4165302477 wait complete. 2024-06-10T02:53:08.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.361+0000 7fde6851db80 1 -- start start 2024-06-10T02:53:08.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.361+0000 7fde6851db80 1 Processor -- start 2024-06-10T02:53:08.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.361+0000 7fde6851db80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5623cb9a1750 0x5623cb9ad5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:08.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.361+0000 7fde6851db80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5623cb9a6f10 0x5623cb9aa900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:08.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.361+0000 7fde6851db80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5623cb9a7830 0x5623cb9aae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:08.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.361+0000 7fde6851db80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5623cb892f70 con 0x5623cb9a1750 2024-06-10T02:53:08.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.361+0000 7fde6851db80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5623cb8ee7c0 con 0x5623cb9a7830 2024-06-10T02:53:08.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.362+0000 7fde6851db80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5623cb9ae1b0 con 0x5623cb9a6f10 2024-06-10T02:53:08.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.362+0000 7fde6851db80 10 client.0 _ll_register_callbacks cb 0x5623cb888560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T02:53:08.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.362+0000 7fde577fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5623cb9a1750 0x5623cb9ad5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:08.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.362+0000 7fde577fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5623cb9a1750 0x5623cb9ad5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:35730/0 (socket says 172.21.15.52:35730) 2024-06-10T02:53:08.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.362+0000 7fde577fe700 1 -- 172.21.15.52:0/2327163226 learned_addr learned my addr 172.21.15.52:0/2327163226 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:53:08.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.362+0000 7fde57fff700 1 --2- 172.21.15.52:0/2327163226 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5623cb9a7830 0x5623cb9aae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:08.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.362+0000 7fde56ffd700 1 --2- 172.21.15.52:0/2327163226 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5623cb9a6f10 0x5623cb9aa900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:08.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.362+0000 7fde577fe700 1 -- 172.21.15.52:0/2327163226 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5623cb9a6f10 msgr2=0x5623cb9aa900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:53:08.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.362+0000 7fde577fe700 1 --2- 172.21.15.52:0/2327163226 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5623cb9a6f10 0x5623cb9aa900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:08.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.362+0000 7fde577fe700 1 -- 172.21.15.52:0/2327163226 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5623cb9a7830 msgr2=0x5623cb9aae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:53:08.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.362+0000 7fde577fe700 1 --2- 172.21.15.52:0/2327163226 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5623cb9a7830 0x5623cb9aae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:08.386 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.362+0000 7fde577fe700 1 -- 172.21.15.52:0/2327163226 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5623cb8b8700 con 0x5623cb9a1750 2024-06-10T02:53:08.386 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.362+0000 7fde577fe700 1 --2- 172.21.15.52:0/2327163226 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5623cb9a1750 0x5623cb9ad5b0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fde4801da60 tx=0x7fde480047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0246fe2efcbf06d server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:53:08.386 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.362+0000 7fde36ffd700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-10T02:53:08.386 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.362+0000 7fde36ffd700 1 -- 172.21.15.52:0/2327163226 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde48016070 con 0x5623cb9a1750 2024-06-10T02:53:08.386 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.362+0000 7fde36ffd700 1 -- 172.21.15.52:0/2327163226 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7fde48004a30 con 0x5623cb9a1750 2024-06-10T02:53:08.386 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.362+0000 7fde6851db80 1 -- 172.21.15.52:0/2327163226 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fde4c09f980 con 0x5623cb9a1750 2024-06-10T02:53:08.386 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.362+0000 7fde36ffd700 1 -- 172.21.15.52:0/2327163226 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fde48030050 con 0x5623cb9a1750 2024-06-10T02:53:08.386 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.363+0000 7fde6851db80 10 client.15039 fetch_fsmap learned FSMap version 29 2024-06-10T02:53:08.386 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.363+0000 7fde6851db80 1 -- 172.21.15.52:0/2327163226 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5623cb9b89c0 con 0x5623cb9a1750 2024-06-10T02:53:08.387 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.363+0000 7fde36ffd700 1 -- 172.21.15.52:0/2327163226 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde4c09f980 con 0x5623cb9a1750 2024-06-10T02:53:08.387 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.363+0000 7fde36ffd700 1 -- 172.21.15.52:0/2327163226 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fde48024030 con 0x5623cb9a1750 2024-06-10T02:53:08.387 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.363+0000 7fde6851db80 10 client.15039 fetch_fsmap finished waiting for FSMap version 29 2024-06-10T02:53:08.387 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.363+0000 7fde6851db80 10 client.15039 resolve_mds: resolved name 'h' to daemon mds.h 2024-06-10T02:53:08.387 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.363+0000 7fde6851db80 20 client.15039 populate_metadata read hostname 'smithi052' 2024-06-10T02:53:08.387 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.363+0000 7fde6851db80 1 --2- 172.21.15.52:0/2327163226 >> [v2:172.21.15.120:6836/4132440952,v1:172.21.15.120:6837/4132440952] conn(0x5623cb9baff0 0x5623cb9bc460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:08.387 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.363+0000 7fde6851db80 4 client.15039 mds_command: new command op to 24428 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T02:53:08.387 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.363+0000 7fde6851db80 1 -- 172.21.15.52:0/2327163226 --> [v2:172.21.15.120:6836/4132440952,v1:172.21.15.120:6837/4132440952] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5623cb8ffd00 con 0x5623cb9baff0 2024-06-10T02:53:08.387 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.363+0000 7fde56ffd700 1 --2- 172.21.15.52:0/2327163226 >> [v2:172.21.15.120:6836/4132440952,v1:172.21.15.120:6837/4132440952] conn(0x5623cb9baff0 0x5623cb9bc460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:08.387 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.364+0000 7fde56ffd700 1 --2- 172.21.15.52:0/2327163226 >> [v2:172.21.15.120:6836/4132440952,v1:172.21.15.120:6837/4132440952] conn(0x5623cb9baff0 0x5623cb9bc460 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=2a7e5ad5f10272bf server_cookie=fd11f93959be4cd8 in_seq=0 out_seq=0 2024-06-10T02:53:08.387 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.364+0000 7fde36ffd700 10 client.15039 ms_handle_connect on v2:172.21.15.120:6836/4132440952 2024-06-10T02:53:08.387 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.364+0000 7fde36ffd700 1 -- 172.21.15.52:0/2327163226 <== mds.4 v2:172.21.15.120:6836/4132440952 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5623cb8ffd00 con 0x5623cb9baff0 2024-06-10T02:53:08.388 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.364+0000 7fde36ffd700 10 client.15039 handle_command_reply: tid=0 2024-06-10T02:53:08.388 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.365+0000 7fde6851db80 1 client.15039 shutdown 2024-06-10T02:53:08.388 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.365+0000 7fde6851db80 1 -- 172.21.15.52:0/2327163226 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5623cb9a1750 msgr2=0x5623cb9ad5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:53:08.388 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.365+0000 7fde6851db80 1 --2- 172.21.15.52:0/2327163226 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5623cb9a1750 0x5623cb9ad5b0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fde4801da60 tx=0x7fde480047c0 comp rx=0 tx=0).stop 2024-06-10T02:53:08.388 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.365+0000 7fde6851db80 1 -- 172.21.15.52:0/2327163226 shutdown_connections 2024-06-10T02:53:08.388 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.365+0000 7fde6851db80 1 --2- 172.21.15.52:0/2327163226 >> [v2:172.21.15.120:6836/4132440952,v1:172.21.15.120:6837/4132440952] conn(0x5623cb9baff0 0x5623cb9bc460 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:08.388 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.365+0000 7fde6851db80 1 --2- 172.21.15.52:0/2327163226 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5623cb9a7830 0x5623cb9aae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:08.388 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.365+0000 7fde6851db80 1 --2- 172.21.15.52:0/2327163226 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5623cb9a1750 0x5623cb9ad5b0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:08.388 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.365+0000 7fde36ffd700 0 client.15039 ms_handle_reset on v2:172.21.15.120:6836/4132440952 2024-06-10T02:53:08.388 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.365+0000 7fde6851db80 1 --2- 172.21.15.52:0/2327163226 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5623cb9a6f10 0x5623cb9aa900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:08.388 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.365+0000 7fde6851db80 1 -- 172.21.15.52:0/2327163226 >> 172.21.15.52:0/2327163226 conn(0x5623cb8ce030 msgr2=0x5623cb99b6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:53:08.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.365+0000 7fde6851db80 1 -- 172.21.15.52:0/2327163226 shutdown_connections 2024-06-10T02:53:08.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.365+0000 7fde6851db80 1 -- 172.21.15.52:0/2327163226 wait complete. 2024-06-10T02:53:08.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.365+0000 7fde6851db80 20 client.15039 trim_cache size 0 max 16384 2024-06-10T02:53:08.393 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.387+0000 7fde6851db80 1 Processor -- start 2024-06-10T02:53:08.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.387+0000 7fde6851db80 1 -- start start 2024-06-10T02:53:08.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.387+0000 7fde6851db80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5623cb9a1750 0x5623cb9a1210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:08.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.387+0000 7fde6851db80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5623cb9a6f10 0x5623cb9a8360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:08.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.387+0000 7fde6851db80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5623cb9a7830 0x5623cb9ad0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:08.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.387+0000 7fde6851db80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5623cb9ba7a0 con 0x5623cb9a7830 2024-06-10T02:53:08.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.387+0000 7fde6851db80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5623cb9bae70 con 0x5623cb9a6f10 2024-06-10T02:53:08.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.387+0000 7fde6851db80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5623cb9bb540 con 0x5623cb9a1750 2024-06-10T02:53:08.395 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.387+0000 7fde57fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5623cb9a7830 0x5623cb9ad0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:08.395 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.387+0000 7fde57fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5623cb9a7830 0x5623cb9ad0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:35742/0 (socket says 172.21.15.52:35742) 2024-06-10T02:53:08.395 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.387+0000 7fde57fff700 1 -- 172.21.15.52:0/3588342710 learned_addr learned my addr 172.21.15.52:0/3588342710 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:53:08.395 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.387+0000 7fde577fe700 1 --2- 172.21.15.52:0/3588342710 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5623cb9a1750 0x5623cb9a1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:08.395 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.387+0000 7fde56ffd700 1 --2- 172.21.15.52:0/3588342710 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5623cb9a6f10 0x5623cb9a8360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:08.395 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.387+0000 7fde57fff700 1 -- 172.21.15.52:0/3588342710 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5623cb9a1750 msgr2=0x5623cb9a1210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:53:08.395 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.387+0000 7fde57fff700 1 --2- 172.21.15.52:0/3588342710 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5623cb9a1750 0x5623cb9a1210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:08.395 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.387+0000 7fde57fff700 1 -- 172.21.15.52:0/3588342710 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5623cb9a6f10 msgr2=0x5623cb9a8360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:53:08.395 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.387+0000 7fde57fff700 1 --2- 172.21.15.52:0/3588342710 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5623cb9a6f10 0x5623cb9a8360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:08.395 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.387+0000 7fde57fff700 1 -- 172.21.15.52:0/3588342710 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5623cb9a5370 con 0x5623cb9a7830 2024-06-10T02:53:08.395 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.387+0000 7fde56ffd700 1 --2- 172.21.15.52:0/3588342710 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5623cb9a6f10 0x5623cb9a8360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T02:53:08.395 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.388+0000 7fde57fff700 1 --2- 172.21.15.52:0/3588342710 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5623cb9a7830 0x5623cb9ad0f0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fde50015000 tx=0x7fde50009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d41339a0a40f33eb server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:53:08.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.388+0000 7fde3d7fa700 1 -- 172.21.15.52:0/3588342710 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde50004a50 con 0x5623cb9a7830 2024-06-10T02:53:08.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.388+0000 7fde3d7fa700 1 -- 172.21.15.52:0/3588342710 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde50004bf0 con 0x5623cb9a7830 2024-06-10T02:53:08.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.388+0000 7fde3d7fa700 1 -- 172.21.15.52:0/3588342710 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde500053a0 con 0x5623cb9a7830 2024-06-10T02:53:08.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.389+0000 7fde6851db80 1 -- 172.21.15.52:0/3588342710 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5623cb9a7830 msgr2=0x5623cb9ad0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:53:08.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.389+0000 7fde6851db80 1 --2- 172.21.15.52:0/3588342710 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5623cb9a7830 0x5623cb9ad0f0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fde50015000 tx=0x7fde50009310 comp rx=0 tx=0).stop 2024-06-10T02:53:08.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.389+0000 7fde6851db80 1 -- 172.21.15.52:0/3588342710 shutdown_connections 2024-06-10T02:53:08.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.389+0000 7fde6851db80 1 --2- 172.21.15.52:0/3588342710 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5623cb9a7830 0x5623cb9ad0f0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:08.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.389+0000 7fde6851db80 1 --2- 172.21.15.52:0/3588342710 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5623cb9a1750 0x5623cb9a1210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:08.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.389+0000 7fde6851db80 1 --2- 172.21.15.52:0/3588342710 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5623cb9a6f10 0x5623cb9a8360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:08.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.389+0000 7fde6851db80 1 -- 172.21.15.52:0/3588342710 >> 172.21.15.52:0/3588342710 conn(0x5623cb8ce030 msgr2=0x5623cb9a7c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:53:08.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.389+0000 7fde6851db80 1 -- 172.21.15.52:0/3588342710 shutdown_connections 2024-06-10T02:53:08.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.389+0000 7fde6851db80 1 -- 172.21.15.52:0/3588342710 wait complete. 2024-06-10T02:53:08.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.389+0000 7fde6851db80 1 -- start start 2024-06-10T02:53:08.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.389+0000 7fde6851db80 1 Processor -- start 2024-06-10T02:53:08.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.389+0000 7fde6851db80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5623cb9a1750 0x5623cb99b650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:08.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.389+0000 7fde6851db80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5623cb9a6f10 0x5623cb99bb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:08.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.389+0000 7fde6851db80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5623cb9a7830 0x5623cb99c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:08.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.389+0000 7fde6851db80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5623cb9bc2e0 con 0x5623cb9a6f10 2024-06-10T02:53:08.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.389+0000 7fde6851db80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5623cb9bbc10 con 0x5623cb9a7830 2024-06-10T02:53:08.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.389+0000 7fde6851db80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5623cb9bc9b0 con 0x5623cb9a1750 2024-06-10T02:53:08.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.390+0000 7fde6851db80 10 client.0 _ll_register_callbacks cb 0x5623cb888560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T02:53:08.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.390+0000 7fde56ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5623cb9a6f10 0x5623cb99bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:08.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.390+0000 7fde56ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5623cb9a6f10 0x5623cb99bb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:35752/0 (socket says 172.21.15.52:35752) 2024-06-10T02:53:08.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.390+0000 7fde56ffd700 1 -- 172.21.15.52:0/4251363632 learned_addr learned my addr 172.21.15.52:0/4251363632 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:53:08.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.390+0000 7fde577fe700 1 --2- 172.21.15.52:0/4251363632 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5623cb9a1750 0x5623cb99b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:08.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.390+0000 7fde57fff700 1 --2- 172.21.15.52:0/4251363632 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5623cb9a7830 0x5623cb99c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:08.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.390+0000 7fde56ffd700 1 -- 172.21.15.52:0/4251363632 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5623cb9a1750 msgr2=0x5623cb99b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:53:08.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.390+0000 7fde56ffd700 1 --2- 172.21.15.52:0/4251363632 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5623cb9a1750 0x5623cb99b650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:08.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.390+0000 7fde56ffd700 1 -- 172.21.15.52:0/4251363632 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5623cb9a7830 msgr2=0x5623cb99c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:53:08.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.390+0000 7fde56ffd700 1 --2- 172.21.15.52:0/4251363632 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5623cb9a7830 0x5623cb99c0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:08.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.390+0000 7fde56ffd700 1 -- 172.21.15.52:0/4251363632 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5623cb9c3d70 con 0x5623cb9a6f10 2024-06-10T02:53:08.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.390+0000 7fde56ffd700 1 --2- 172.21.15.52:0/4251363632 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5623cb9a6f10 0x5623cb99bb90 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fde4c09b2d0 tx=0x7fde4c086040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33bb9eb8f67544d7 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:53:08.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.390+0000 7fde36ffd700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-10T02:53:08.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.390+0000 7fde36ffd700 1 -- 172.21.15.52:0/4251363632 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde4c07e070 con 0x5623cb9a6f10 2024-06-10T02:53:08.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.390+0000 7fde36ffd700 1 -- 172.21.15.52:0/4251363632 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7fde4c004340 con 0x5623cb9a6f10 2024-06-10T02:53:08.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.390+0000 7fde6851db80 1 -- 172.21.15.52:0/4251363632 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fde50004bf0 con 0x5623cb9a6f10 2024-06-10T02:53:08.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.391+0000 7fde36ffd700 1 -- 172.21.15.52:0/4251363632 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fde4c09b5d0 con 0x5623cb9a6f10 2024-06-10T02:53:08.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.391+0000 7fde6851db80 10 client.15045 fetch_fsmap learned FSMap version 29 2024-06-10T02:53:08.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.391+0000 7fde6851db80 1 -- 172.21.15.52:0/4251363632 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5623cba26b30 con 0x5623cb9a6f10 2024-06-10T02:53:08.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.391+0000 7fde36ffd700 1 -- 172.21.15.52:0/4251363632 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde50004bf0 con 0x5623cb9a6f10 2024-06-10T02:53:08.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.391+0000 7fde36ffd700 1 -- 172.21.15.52:0/4251363632 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fde4c09b830 con 0x5623cb9a6f10 2024-06-10T02:53:08.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.391+0000 7fde6851db80 10 client.15045 fetch_fsmap finished waiting for FSMap version 29 2024-06-10T02:53:08.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.391+0000 7fde6851db80 10 client.15045 resolve_mds: resolved name 'h' to daemon mds.h 2024-06-10T02:53:08.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.391+0000 7fde6851db80 20 client.15045 populate_metadata read hostname 'smithi052' 2024-06-10T02:53:08.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.391+0000 7fde6851db80 1 --2- 172.21.15.52:0/4251363632 >> [v2:172.21.15.120:6836/4132440952,v1:172.21.15.120:6837/4132440952] conn(0x5623cba2b100 0x5623cba29e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:08.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.391+0000 7fde6851db80 4 client.15045 mds_command: new command op to 24428 tid=0 multi_id=0 [{"prefix": "status"}] 2024-06-10T02:53:08.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.391+0000 7fde6851db80 1 -- 172.21.15.52:0/4251363632 --> [v2:172.21.15.120:6836/4132440952,v1:172.21.15.120:6837/4132440952] -- command(tid 0: {"prefix": "status"}) -- 0x5623cb8fa280 con 0x5623cba2b100 2024-06-10T02:53:08.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.391+0000 7fde577fe700 1 --2- 172.21.15.52:0/4251363632 >> [v2:172.21.15.120:6836/4132440952,v1:172.21.15.120:6837/4132440952] conn(0x5623cba2b100 0x5623cba29e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:08.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.392+0000 7fde577fe700 1 --2- 172.21.15.52:0/4251363632 >> [v2:172.21.15.120:6836/4132440952,v1:172.21.15.120:6837/4132440952] conn(0x5623cba2b100 0x5623cba29e10 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ada8d64b987953a5 server_cookie=542f73181d9de9e3 in_seq=0 out_seq=0 2024-06-10T02:53:08.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.392+0000 7fde36ffd700 10 client.15045 ms_handle_connect on v2:172.21.15.120:6836/4132440952 2024-06-10T02:53:08.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.392+0000 7fde36ffd700 1 -- 172.21.15.52:0/4251363632 <== mds.4 v2:172.21.15.120:6836/4132440952 1 ==== command_reply(tid 0: 0 ) ==== 8+0+333 (crc 0 0 0) 0x5623cb8fa280 con 0x5623cba2b100 2024-06-10T02:53:08.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.392+0000 7fde36ffd700 10 client.15045 handle_command_reply: tid=0 2024-06-10T02:53:08.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.392+0000 7fde6851db80 1 client.15045 shutdown 2024-06-10T02:53:08.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.392+0000 7fde6851db80 1 -- 172.21.15.52:0/4251363632 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5623cb9a6f10 msgr2=0x5623cb99bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:53:08.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.392+0000 7fde6851db80 1 --2- 172.21.15.52:0/4251363632 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5623cb9a6f10 0x5623cb99bb90 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fde4c09b2d0 tx=0x7fde4c086040 comp rx=0 tx=0).stop 2024-06-10T02:53:08.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.392+0000 7fde6851db80 1 -- 172.21.15.52:0/4251363632 shutdown_connections 2024-06-10T02:53:08.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.392+0000 7fde6851db80 1 --2- 172.21.15.52:0/4251363632 >> [v2:172.21.15.120:6836/4132440952,v1:172.21.15.120:6837/4132440952] conn(0x5623cba2b100 0x5623cba29e10 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:08.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.392+0000 7fde6851db80 1 --2- 172.21.15.52:0/4251363632 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5623cb9a7830 0x5623cb99c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:08.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.392+0000 7fde6851db80 1 --2- 172.21.15.52:0/4251363632 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5623cb9a1750 0x5623cb99b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:08.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.392+0000 7fde6851db80 1 --2- 172.21.15.52:0/4251363632 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5623cb9a6f10 0x5623cb99bb90 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:08.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.392+0000 7fde6851db80 1 -- 172.21.15.52:0/4251363632 >> 172.21.15.52:0/4251363632 conn(0x5623cb8ce030 msgr2=0x5623cb9a59c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:53:08.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.392+0000 7fde36ffd700 0 client.15045 ms_handle_reset on v2:172.21.15.120:6836/4132440952 2024-06-10T02:53:08.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.392+0000 7fde6851db80 1 -- 172.21.15.52:0/4251363632 shutdown_connections 2024-06-10T02:53:08.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.392+0000 7fde6851db80 1 -- 172.21.15.52:0/4251363632 wait complete. 2024-06-10T02:53:08.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.392+0000 7fde6851db80 20 client.15045 trim_cache size 0 max 16384 2024-06-10T02:53:08.401 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-10T02:53:08.401 INFO:teuthology.orchestra.run.smithi052.stdout: "cluster_fsid": "d8676e9e-26d2-11ef-bc9b-c7b262605968", 2024-06-10T02:53:08.401 INFO:teuthology.orchestra.run.smithi052.stdout: "whoami": 4, 2024-06-10T02:53:08.401 INFO:teuthology.orchestra.run.smithi052.stdout: "id": 24428, 2024-06-10T02:53:08.401 INFO:teuthology.orchestra.run.smithi052.stdout: "want_state": "up:active", 2024-06-10T02:53:08.401 INFO:teuthology.orchestra.run.smithi052.stdout: "state": "up:active", 2024-06-10T02:53:08.401 INFO:teuthology.orchestra.run.smithi052.stdout: "fs_name": "cephfs", 2024-06-10T02:53:08.401 INFO:teuthology.orchestra.run.smithi052.stdout: "rank_uptime": 119.77420098499999, 2024-06-10T02:53:08.401 INFO:teuthology.orchestra.run.smithi052.stdout: "mdsmap_epoch": 29, 2024-06-10T02:53:08.401 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_epoch": 100, 2024-06-10T02:53:08.402 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_epoch_barrier": 98, 2024-06-10T02:53:08.402 INFO:teuthology.orchestra.run.smithi052.stdout: "uptime": 215.04029398399999 2024-06-10T02:53:08.402 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-10T02:53:08.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.393+0000 7fde5dc9a700 1 -- 172.21.15.52:0/2996341058 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fde400a2ca0 msgr2=0x7fde400a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:53:08.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.393+0000 7fde5dc9a700 1 --2- 172.21.15.52:0/2996341058 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fde400a2ca0 0x7fde400a5160 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fde48005950 tx=0x7fde4800a300 comp rx=0 tx=0).stop 2024-06-10T02:53:08.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.393+0000 7fde5dc9a700 1 -- 172.21.15.52:0/2996341058 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fde58074fe0 msgr2=0x7fde5819b2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:53:08.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.393+0000 7fde5dc9a700 1 --2- 172.21.15.52:0/2996341058 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fde58074fe0 0x7fde5819b2d0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fde4c00d980 tx=0x7fde4c00dc90 comp rx=0 tx=0).stop 2024-06-10T02:53:08.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.393+0000 7fde5dc9a700 1 -- 172.21.15.52:0/2996341058 shutdown_connections 2024-06-10T02:53:08.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.393+0000 7fde5dc9a700 1 --2- 172.21.15.52:0/2996341058 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fde5810ee90 0x7fde5819f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:08.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.393+0000 7fde5dc9a700 1 --2- 172.21.15.52:0/2996341058 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fde580746c0 0x7fde5819ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:08.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.393+0000 7fde5dc9a700 1 --2- 172.21.15.52:0/2996341058 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fde400a2ca0 0x7fde400a5160 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:08.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.393+0000 7fde5dc9a700 1 --2- 172.21.15.52:0/2996341058 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fde58074fe0 0x7fde5819b2d0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:08.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.393+0000 7fde5dc9a700 1 -- 172.21.15.52:0/2996341058 >> 172.21.15.52:0/2996341058 conn(0x7fde580795f0 msgr2=0x7fde581005b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:53:08.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.393+0000 7fde5dc9a700 1 -- 172.21.15.52:0/2996341058 shutdown_connections 2024-06-10T02:53:08.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:08.393+0000 7fde5dc9a700 1 -- 172.21.15.52:0/2996341058 wait complete. 2024-06-10T02:53:09.092 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:53:08 smithi052 ceph-mon[32362]: pgmap v402: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-10T02:53:09.120 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph tell mds.b status 2024-06-10T02:53:09.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:53:08 smithi120 ceph-mon[36634]: pgmap v402: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-10T02:53:09.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:53:08 smithi188 ceph-mon[36599]: pgmap v402: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-10T02:53:09.372 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:53:11.098 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:53:10 smithi052 ceph-mon[32362]: pgmap v403: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-10T02:53:11.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.098+0000 7f2ccb4a1700 1 -- 172.21.15.52:0/1596965656 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2cc406b4e0 msgr2=0x7f2cc410ea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:53:11.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.098+0000 7f2ccb4a1700 1 --2- 172.21.15.52:0/1596965656 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2cc406b4e0 0x7f2cc410ea30 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f2cb4009990 tx=0x7f2cb4009ca0 comp rx=0 tx=0).stop 2024-06-10T02:53:11.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.098+0000 7f2ccb4a1700 1 -- 172.21.15.52:0/1596965656 shutdown_connections 2024-06-10T02:53:11.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.098+0000 7f2ccb4a1700 1 --2- 172.21.15.52:0/1596965656 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2cc410ef70 0x7f2cc4111360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:11.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.098+0000 7f2ccb4a1700 1 --2- 172.21.15.52:0/1596965656 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2cc406abc0 0x7f2cc406afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:11.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.098+0000 7f2ccb4a1700 1 --2- 172.21.15.52:0/1596965656 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2cc406b4e0 0x7f2cc410ea30 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:11.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.098+0000 7f2ccb4a1700 1 -- 172.21.15.52:0/1596965656 >> 172.21.15.52:0/1596965656 conn(0x7f2cc4078340 msgr2=0x7f2cc4078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:53:11.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.098+0000 7f2ccb4a1700 1 -- 172.21.15.52:0/1596965656 shutdown_connections 2024-06-10T02:53:11.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.098+0000 7f2ccb4a1700 1 -- 172.21.15.52:0/1596965656 wait complete. 2024-06-10T02:53:11.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.099+0000 7f2ccb4a1700 1 Processor -- start 2024-06-10T02:53:11.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.099+0000 7f2ccb4a1700 1 -- start start 2024-06-10T02:53:11.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.099+0000 7f2ccb4a1700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2cc406abc0 0x7f2cc419af10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:11.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.099+0000 7f2ccb4a1700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2cc406b4e0 0x7f2cc419b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:11.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.099+0000 7f2ccb4a1700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2cc410ef70 0x7f2cc419f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:11.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.099+0000 7f2ccb4a1700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f2cc4114a20 con 0x7f2cc406abc0 2024-06-10T02:53:11.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.100+0000 7f2ccb4a1700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f2cc41148a0 con 0x7f2cc406b4e0 2024-06-10T02:53:11.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.100+0000 7f2ccb4a1700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f2cc4114ba0 con 0x7f2cc410ef70 2024-06-10T02:53:11.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.100+0000 7f2cc923d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2cc406abc0 0x7f2cc419af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:11.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.100+0000 7f2cc923d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2cc406abc0 0x7f2cc419af10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53054/0 (socket says 172.21.15.52:53054) 2024-06-10T02:53:11.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.100+0000 7f2cc923d700 1 -- 172.21.15.52:0/2605785456 learned_addr learned my addr 172.21.15.52:0/2605785456 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:53:11.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.100+0000 7f2cc8a3c700 1 --2- 172.21.15.52:0/2605785456 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2cc406b4e0 0x7f2cc419b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:11.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.100+0000 7f2cc9a3e700 1 --2- 172.21.15.52:0/2605785456 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2cc410ef70 0x7f2cc419f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:11.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.100+0000 7f2cc923d700 1 -- 172.21.15.52:0/2605785456 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2cc410ef70 msgr2=0x7f2cc419f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:53:11.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.100+0000 7f2cc923d700 1 --2- 172.21.15.52:0/2605785456 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2cc410ef70 0x7f2cc419f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:11.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.100+0000 7f2cc923d700 1 -- 172.21.15.52:0/2605785456 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2cc406b4e0 msgr2=0x7f2cc419b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:53:11.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.100+0000 7f2cc923d700 1 --2- 172.21.15.52:0/2605785456 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2cc406b4e0 0x7f2cc419b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:11.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.100+0000 7f2cc923d700 1 -- 172.21.15.52:0/2605785456 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cc419ff20 con 0x7f2cc406abc0 2024-06-10T02:53:11.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.100+0000 7f2cc923d700 1 --2- 172.21.15.52:0/2605785456 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2cc406abc0 0x7f2cc419af10 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f2cc000d980 tx=0x7f2cc000dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:53:11.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.100+0000 7f2cba7fc700 1 -- 172.21.15.52:0/2605785456 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cc0017070 con 0x7f2cc406abc0 2024-06-10T02:53:11.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.100+0000 7f2ccb4a1700 1 -- 172.21.15.52:0/2605785456 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2cc41a0210 con 0x7f2cc406abc0 2024-06-10T02:53:11.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.100+0000 7f2cba7fc700 1 -- 172.21.15.52:0/2605785456 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2cc0004d10 con 0x7f2cc406abc0 2024-06-10T02:53:11.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.100+0000 7f2cba7fc700 1 -- 172.21.15.52:0/2605785456 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cc00055c0 con 0x7f2cc406abc0 2024-06-10T02:53:11.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.100+0000 7f2ccb4a1700 1 -- 172.21.15.52:0/2605785456 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2cc41a7a90 con 0x7f2cc406abc0 2024-06-10T02:53:11.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.103+0000 7f2cba7fc700 1 -- 172.21.15.52:0/2605785456 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f2cc0004750 con 0x7f2cc406abc0 2024-06-10T02:53:11.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.103+0000 7f2cba7fc700 1 --2- 172.21.15.52:0/2605785456 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f2cb00a2c20 0x7f2cb00a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:11.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.103+0000 7f2cba7fc700 1 -- 172.21.15.52:0/2605785456 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f2cc00cf3a0 con 0x7f2cc406abc0 2024-06-10T02:53:11.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.104+0000 7f2cc8a3c700 1 --2- 172.21.15.52:0/2605785456 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f2cb00a2c20 0x7f2cb00a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:11.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.104+0000 7f2cc8a3c700 1 --2- 172.21.15.52:0/2605785456 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f2cb00a2c20 0x7f2cb00a50e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f2cb400b2e0 tx=0x7f2cb401a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:53:11.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.104+0000 7f2cd5d24b80 1 Processor -- start 2024-06-10T02:53:11.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.104+0000 7f2cd5d24b80 1 -- start start 2024-06-10T02:53:11.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.104+0000 7f2cd5d24b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e3b91275f0 0x55e3b91279d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:11.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.104+0000 7f2cd5d24b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e3b9127f10 0x55e3b9121990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:11.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.104+0000 7f2cd5d24b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e3b9121ed0 0x55e3b91222f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:11.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.104+0000 7f2cd5d24b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55e3b90737c0 con 0x55e3b91275f0 2024-06-10T02:53:11.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.104+0000 7f2cd5d24b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55e3b901b6d0 con 0x55e3b9127f10 2024-06-10T02:53:11.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.104+0000 7f2cd5d24b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55e3b906f780 con 0x55e3b9121ed0 2024-06-10T02:53:11.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.105+0000 7f2cc923d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e3b91275f0 0x55e3b91279d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:11.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.105+0000 7f2cc923d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e3b91275f0 0x55e3b91279d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53068/0 (socket says 172.21.15.52:53068) 2024-06-10T02:53:11.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.105+0000 7f2cc923d700 1 -- 172.21.15.52:0/968205723 learned_addr learned my addr 172.21.15.52:0/968205723 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:53:11.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.105+0000 7f2cc8a3c700 1 --2- 172.21.15.52:0/968205723 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e3b9127f10 0x55e3b9121990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:11.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.105+0000 7f2cc9a3e700 1 --2- 172.21.15.52:0/968205723 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e3b9121ed0 0x55e3b91222f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:11.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.105+0000 7f2cc923d700 1 -- 172.21.15.52:0/968205723 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e3b9121ed0 msgr2=0x55e3b91222f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:53:11.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.105+0000 7f2cc923d700 1 --2- 172.21.15.52:0/968205723 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e3b9121ed0 0x55e3b91222f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:11.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.105+0000 7f2cc923d700 1 -- 172.21.15.52:0/968205723 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e3b9127f10 msgr2=0x55e3b9121990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:53:11.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.105+0000 7f2cc923d700 1 --2- 172.21.15.52:0/968205723 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e3b9127f10 0x55e3b9121990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:11.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.105+0000 7f2cc923d700 1 -- 172.21.15.52:0/968205723 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e3b9011f00 con 0x55e3b91275f0 2024-06-10T02:53:11.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.105+0000 7f2cc9a3e700 1 --2- 172.21.15.52:0/968205723 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e3b9121ed0 0x55e3b91222f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T02:53:11.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.105+0000 7f2cc8a3c700 1 --2- 172.21.15.52:0/968205723 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e3b9127f10 0x55e3b9121990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T02:53:11.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.105+0000 7f2cc923d700 1 --2- 172.21.15.52:0/968205723 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e3b91275f0 0x55e3b91279d0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f2cc009b7e0 tx=0x7f2cc009b2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b97f4817c0bb25c server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:53:11.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.105+0000 7f2caf7fe700 1 -- 172.21.15.52:0/968205723 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cc000ee10 con 0x55e3b91275f0 2024-06-10T02:53:11.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.105+0000 7f2caf7fe700 1 -- 172.21.15.52:0/968205723 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2cc009b850 con 0x55e3b91275f0 2024-06-10T02:53:11.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.105+0000 7f2caf7fe700 1 -- 172.21.15.52:0/968205723 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cc0004a40 con 0x55e3b91275f0 2024-06-10T02:53:11.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.106+0000 7f2cd5d24b80 1 -- 172.21.15.52:0/968205723 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e3b91275f0 msgr2=0x55e3b91279d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:53:11.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.106+0000 7f2cd5d24b80 1 --2- 172.21.15.52:0/968205723 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e3b91275f0 0x55e3b91279d0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f2cc009b7e0 tx=0x7f2cc009b2d0 comp rx=0 tx=0).stop 2024-06-10T02:53:11.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.106+0000 7f2cd5d24b80 1 -- 172.21.15.52:0/968205723 shutdown_connections 2024-06-10T02:53:11.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.106+0000 7f2cd5d24b80 1 --2- 172.21.15.52:0/968205723 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e3b9121ed0 0x55e3b91222f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:11.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.106+0000 7f2cd5d24b80 1 --2- 172.21.15.52:0/968205723 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e3b91275f0 0x55e3b91279d0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:11.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.106+0000 7f2cd5d24b80 1 --2- 172.21.15.52:0/968205723 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e3b9127f10 0x55e3b9121990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:11.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.106+0000 7f2cd5d24b80 1 -- 172.21.15.52:0/968205723 >> 172.21.15.52:0/968205723 conn(0x55e3b904cc70 msgr2=0x55e3b911c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:53:11.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.106+0000 7f2cd5d24b80 1 -- 172.21.15.52:0/968205723 shutdown_connections 2024-06-10T02:53:11.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.106+0000 7f2cd5d24b80 1 -- 172.21.15.52:0/968205723 wait complete. 2024-06-10T02:53:11.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.106+0000 7f2cd5d24b80 1 -- start start 2024-06-10T02:53:11.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.107+0000 7f2cd5d24b80 1 Processor -- start 2024-06-10T02:53:11.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.107+0000 7f2cd5d24b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e3b9121ed0 0x55e3b912dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:11.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.107+0000 7f2cd5d24b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e3b91275f0 0x55e3b912aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:11.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.107+0000 7f2cd5d24b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e3b9127f10 0x55e3b912b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:11.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.107+0000 7f2cd5d24b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55e3b912e980 con 0x55e3b91275f0 2024-06-10T02:53:11.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.107+0000 7f2cd5d24b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55e3b9013ba0 con 0x55e3b9127f10 2024-06-10T02:53:11.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.107+0000 7f2cd5d24b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55e3b912eb00 con 0x55e3b9121ed0 2024-06-10T02:53:11.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.107+0000 7f2cd5d24b80 10 client.0 _ll_register_callbacks cb 0x55e3b900d8d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T02:53:11.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.107+0000 7f2cc8a3c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e3b91275f0 0x55e3b912aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:11.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.107+0000 7f2cc8a3c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e3b91275f0 0x55e3b912aff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53080/0 (socket says 172.21.15.52:53080) 2024-06-10T02:53:11.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.107+0000 7f2cc8a3c700 1 -- 172.21.15.52:0/1599570720 learned_addr learned my addr 172.21.15.52:0/1599570720 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:53:11.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.107+0000 7f2cc9a3e700 1 --2- 172.21.15.52:0/1599570720 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e3b9127f10 0x55e3b912b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:11.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.107+0000 7f2cc923d700 1 --2- 172.21.15.52:0/1599570720 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e3b9121ed0 0x55e3b912dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:11.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.107+0000 7f2cc8a3c700 1 -- 172.21.15.52:0/1599570720 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e3b9121ed0 msgr2=0x55e3b912dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:53:11.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.107+0000 7f2cc8a3c700 1 --2- 172.21.15.52:0/1599570720 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e3b9121ed0 0x55e3b912dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:11.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.108+0000 7f2cc8a3c700 1 -- 172.21.15.52:0/1599570720 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e3b9127f10 msgr2=0x55e3b912b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:53:11.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.108+0000 7f2cc8a3c700 1 --2- 172.21.15.52:0/1599570720 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e3b9127f10 0x55e3b912b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:11.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.108+0000 7f2cc8a3c700 1 -- 172.21.15.52:0/1599570720 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e3b8fc2880 con 0x55e3b91275f0 2024-06-10T02:53:11.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.108+0000 7f2cc8a3c700 1 --2- 172.21.15.52:0/1599570720 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e3b91275f0 0x55e3b912aff0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f2cb4015000 tx=0x7f2cb4009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21f612333c7201e0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:53:11.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.108+0000 7f2cacff9700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-10T02:53:11.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.108+0000 7f2cacff9700 1 -- 172.21.15.52:0/1599570720 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cb4034060 con 0x55e3b91275f0 2024-06-10T02:53:11.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.108+0000 7f2cacff9700 1 -- 172.21.15.52:0/1599570720 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f2cb4023610 con 0x55e3b91275f0 2024-06-10T02:53:11.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.108+0000 7f2cd5d24b80 1 -- 172.21.15.52:0/1599570720 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2cc009b850 con 0x55e3b91275f0 2024-06-10T02:53:11.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.108+0000 7f2cacff9700 1 -- 172.21.15.52:0/1599570720 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2cb403b050 con 0x55e3b91275f0 2024-06-10T02:53:11.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.108+0000 7f2cd5d24b80 10 client.15057 fetch_fsmap learned FSMap version 29 2024-06-10T02:53:11.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.108+0000 7f2cd5d24b80 1 -- 172.21.15.52:0/1599570720 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e3b9139280 con 0x55e3b91275f0 2024-06-10T02:53:11.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.108+0000 7f2cacff9700 1 -- 172.21.15.52:0/1599570720 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2cc009b850 con 0x55e3b91275f0 2024-06-10T02:53:11.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.108+0000 7f2cacff9700 1 -- 172.21.15.52:0/1599570720 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f2cb402f970 con 0x55e3b91275f0 2024-06-10T02:53:11.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.109+0000 7f2cd5d24b80 10 client.15057 fetch_fsmap finished waiting for FSMap version 29 2024-06-10T02:53:11.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.109+0000 7f2cd5d24b80 10 client.15057 resolve_mds: resolved name 'b' to daemon mds.b 2024-06-10T02:53:11.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.109+0000 7f2cd5d24b80 20 client.15057 populate_metadata read hostname 'smithi052' 2024-06-10T02:53:11.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.109+0000 7f2cd5d24b80 1 --2- 172.21.15.52:0/1599570720 >> [v2:172.21.15.120:6838/1672436487,v1:172.21.15.120:6839/1672436487] conn(0x55e3b913b890 0x55e3b913cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:11.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.109+0000 7f2cd5d24b80 4 client.15057 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T02:53:11.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.109+0000 7f2cd5d24b80 1 -- 172.21.15.52:0/1599570720 --> [v2:172.21.15.120:6838/1672436487,v1:172.21.15.120:6839/1672436487] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e3b9080cc0 con 0x55e3b913b890 2024-06-10T02:53:11.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.109+0000 7f2cc923d700 1 --2- 172.21.15.52:0/1599570720 >> [v2:172.21.15.120:6838/1672436487,v1:172.21.15.120:6839/1672436487] conn(0x55e3b913b890 0x55e3b913cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:11.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.109+0000 7f2cc923d700 1 --2- 172.21.15.52:0/1599570720 >> [v2:172.21.15.120:6838/1672436487,v1:172.21.15.120:6839/1672436487] conn(0x55e3b913b890 0x55e3b913cd00 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=8266ebb55c60129f server_cookie=fe582f2825a3323d in_seq=0 out_seq=0 2024-06-10T02:53:11.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.109+0000 7f2cacff9700 10 client.15057 ms_handle_connect on v2:172.21.15.120:6838/1672436487 2024-06-10T02:53:11.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.110+0000 7f2cacff9700 1 -- 172.21.15.52:0/1599570720 <== mds.3 v2:172.21.15.120:6838/1672436487 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55e3b9080cc0 con 0x55e3b913b890 2024-06-10T02:53:11.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.110+0000 7f2cacff9700 10 client.15057 handle_command_reply: tid=0 2024-06-10T02:53:11.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.110+0000 7f2cd5d24b80 1 client.15057 shutdown 2024-06-10T02:53:11.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.110+0000 7f2cd5d24b80 1 -- 172.21.15.52:0/1599570720 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e3b91275f0 msgr2=0x55e3b912aff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:53:11.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.110+0000 7f2cd5d24b80 1 --2- 172.21.15.52:0/1599570720 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e3b91275f0 0x55e3b912aff0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f2cb4015000 tx=0x7f2cb4009ca0 comp rx=0 tx=0).stop 2024-06-10T02:53:11.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.110+0000 7f2cd5d24b80 1 -- 172.21.15.52:0/1599570720 shutdown_connections 2024-06-10T02:53:11.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.110+0000 7f2cd5d24b80 1 --2- 172.21.15.52:0/1599570720 >> [v2:172.21.15.120:6838/1672436487,v1:172.21.15.120:6839/1672436487] conn(0x55e3b913b890 0x55e3b913cd00 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:11.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.110+0000 7f2cd5d24b80 1 --2- 172.21.15.52:0/1599570720 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e3b9127f10 0x55e3b912b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:11.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.110+0000 7f2cd5d24b80 1 --2- 172.21.15.52:0/1599570720 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e3b9121ed0 0x55e3b912dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:11.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.110+0000 7f2cacff9700 0 client.15057 ms_handle_reset on v2:172.21.15.120:6838/1672436487 2024-06-10T02:53:11.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.110+0000 7f2cd5d24b80 1 --2- 172.21.15.52:0/1599570720 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e3b91275f0 0x55e3b912aff0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:11.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.110+0000 7f2cd5d24b80 1 -- 172.21.15.52:0/1599570720 >> 172.21.15.52:0/1599570720 conn(0x55e3b904cc70 msgr2=0x55e3b9120980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:53:11.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.110+0000 7f2cd5d24b80 1 -- 172.21.15.52:0/1599570720 shutdown_connections 2024-06-10T02:53:11.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.110+0000 7f2cd5d24b80 1 -- 172.21.15.52:0/1599570720 wait complete. 2024-06-10T02:53:11.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.111+0000 7f2cd5d24b80 20 client.15057 trim_cache size 0 max 16384 2024-06-10T02:53:11.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.132+0000 7f2cd5d24b80 1 Processor -- start 2024-06-10T02:53:11.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.132+0000 7f2cd5d24b80 1 -- start start 2024-06-10T02:53:11.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.133+0000 7f2cd5d24b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e3b9121ed0 0x55e3b9121990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:11.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.133+0000 7f2cd5d24b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e3b91275f0 0x55e3b9129f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:11.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.133+0000 7f2cd5d24b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e3b9127f10 0x55e3b9128a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:11.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.133+0000 7f2cd5d24b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55e3b913b040 con 0x55e3b9127f10 2024-06-10T02:53:11.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.133+0000 7f2cd5d24b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55e3b913b710 con 0x55e3b9121ed0 2024-06-10T02:53:11.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.133+0000 7f2cd5d24b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55e3b913bde0 con 0x55e3b91275f0 2024-06-10T02:53:11.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.133+0000 7f2cc9a3e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e3b9127f10 0x55e3b9128a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:11.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.133+0000 7f2cc9a3e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e3b9127f10 0x55e3b9128a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53094/0 (socket says 172.21.15.52:53094) 2024-06-10T02:53:11.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.133+0000 7f2cc9a3e700 1 -- 172.21.15.52:0/2131431889 learned_addr learned my addr 172.21.15.52:0/2131431889 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:53:11.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.133+0000 7f2cc8a3c700 1 --2- 172.21.15.52:0/2131431889 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e3b91275f0 0x55e3b9129f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:11.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.133+0000 7f2cc923d700 1 --2- 172.21.15.52:0/2131431889 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e3b9121ed0 0x55e3b9121990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:11.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.133+0000 7f2cc9a3e700 1 -- 172.21.15.52:0/2131431889 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e3b91275f0 msgr2=0x55e3b9129f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:53:11.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.133+0000 7f2cc9a3e700 1 --2- 172.21.15.52:0/2131431889 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e3b91275f0 0x55e3b9129f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:11.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.133+0000 7f2cc9a3e700 1 -- 172.21.15.52:0/2131431889 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e3b9121ed0 msgr2=0x55e3b9121990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:53:11.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.133+0000 7f2cc9a3e700 1 --2- 172.21.15.52:0/2131431889 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e3b9121ed0 0x55e3b9121990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:11.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.133+0000 7f2cc9a3e700 1 -- 172.21.15.52:0/2131431889 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e3b911e030 con 0x55e3b9127f10 2024-06-10T02:53:11.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.133+0000 7f2cc8a3c700 1 --2- 172.21.15.52:0/2131431889 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e3b91275f0 0x55e3b9129f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T02:53:11.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.133+0000 7f2cc923d700 1 --2- 172.21.15.52:0/2131431889 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e3b9121ed0 0x55e3b9121990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T02:53:11.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.133+0000 7f2cc9a3e700 1 --2- 172.21.15.52:0/2131431889 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e3b9127f10 0x55e3b9128a00 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f2cbc008580 tx=0x7f2cbc00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5c7f53dbe4e1384 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:53:11.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.134+0000 7f2caeffd700 1 -- 172.21.15.52:0/2131431889 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cbc01b020 con 0x55e3b9127f10 2024-06-10T02:53:11.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.134+0000 7f2caeffd700 1 -- 172.21.15.52:0/2131431889 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2cbc017b60 con 0x55e3b9127f10 2024-06-10T02:53:11.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.134+0000 7f2caeffd700 1 -- 172.21.15.52:0/2131431889 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cbc007bf0 con 0x55e3b9127f10 2024-06-10T02:53:11.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.134+0000 7f2cd5d24b80 1 -- 172.21.15.52:0/2131431889 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e3b9127f10 msgr2=0x55e3b9128a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:53:11.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.134+0000 7f2cd5d24b80 1 --2- 172.21.15.52:0/2131431889 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e3b9127f10 0x55e3b9128a00 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f2cbc008580 tx=0x7f2cbc00b040 comp rx=0 tx=0).stop 2024-06-10T02:53:11.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.134+0000 7f2cd5d24b80 1 -- 172.21.15.52:0/2131431889 shutdown_connections 2024-06-10T02:53:11.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.134+0000 7f2cd5d24b80 1 --2- 172.21.15.52:0/2131431889 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e3b9127f10 0x55e3b9128a00 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:11.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.134+0000 7f2cd5d24b80 1 --2- 172.21.15.52:0/2131431889 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e3b9121ed0 0x55e3b9121990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:11.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.134+0000 7f2cd5d24b80 1 --2- 172.21.15.52:0/2131431889 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e3b91275f0 0x55e3b9129f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:11.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.134+0000 7f2cd5d24b80 1 -- 172.21.15.52:0/2131431889 >> 172.21.15.52:0/2131431889 conn(0x55e3b904cc70 msgr2=0x55e3b91282f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:53:11.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.134+0000 7f2cd5d24b80 1 -- 172.21.15.52:0/2131431889 shutdown_connections 2024-06-10T02:53:11.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.134+0000 7f2cd5d24b80 1 -- 172.21.15.52:0/2131431889 wait complete. 2024-06-10T02:53:11.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.135+0000 7f2cd5d24b80 1 -- start start 2024-06-10T02:53:11.141 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.135+0000 7f2cd5d24b80 1 Processor -- start 2024-06-10T02:53:11.141 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.135+0000 7f2cd5d24b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e3b9121ed0 0x55e3b9146180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:11.141 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.135+0000 7f2cd5d24b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e3b91275f0 0x55e3b91466c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:11.141 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.135+0000 7f2cd5d24b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e3b9127f10 0x55e3b9141130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:11.141 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.135+0000 7f2cd5d24b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55e3b913cb80 con 0x55e3b91275f0 2024-06-10T02:53:11.141 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.135+0000 7f2cd5d24b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55e3b913c4b0 con 0x55e3b9127f10 2024-06-10T02:53:11.141 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.135+0000 7f2cd5d24b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55e3b913d250 con 0x55e3b9121ed0 2024-06-10T02:53:11.141 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.135+0000 7f2cd5d24b80 10 client.0 _ll_register_callbacks cb 0x55e3b900d8d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T02:53:11.141 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.135+0000 7f2cc8a3c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e3b91275f0 0x55e3b91466c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:11.141 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.135+0000 7f2cc8a3c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e3b91275f0 0x55e3b91466c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53102/0 (socket says 172.21.15.52:53102) 2024-06-10T02:53:11.141 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.135+0000 7f2cc8a3c700 1 -- 172.21.15.52:0/3583205489 learned_addr learned my addr 172.21.15.52:0/3583205489 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:53:11.141 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.135+0000 7f2cc923d700 1 --2- 172.21.15.52:0/3583205489 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e3b9121ed0 0x55e3b9146180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:11.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.136+0000 7f2cc9a3e700 1 --2- 172.21.15.52:0/3583205489 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e3b9127f10 0x55e3b9141130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:11.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.136+0000 7f2cc8a3c700 1 -- 172.21.15.52:0/3583205489 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e3b9121ed0 msgr2=0x55e3b9146180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:53:11.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.136+0000 7f2cc8a3c700 1 --2- 172.21.15.52:0/3583205489 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e3b9121ed0 0x55e3b9146180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:11.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.136+0000 7f2cc8a3c700 1 -- 172.21.15.52:0/3583205489 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e3b9127f10 msgr2=0x55e3b9141130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:53:11.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.136+0000 7f2cc8a3c700 1 --2- 172.21.15.52:0/3583205489 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e3b9127f10 0x55e3b9141130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:11.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.136+0000 7f2cc8a3c700 1 -- 172.21.15.52:0/3583205489 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e3b9141670 con 0x55e3b91275f0 2024-06-10T02:53:11.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.136+0000 7f2cc8a3c700 1 --2- 172.21.15.52:0/3583205489 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e3b91275f0 0x55e3b91466c0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f2cb402fdc0 tx=0x7f2cb40176a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c45a92f59b65755d server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:53:11.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.136+0000 7f2cacff9700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-10T02:53:11.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.136+0000 7f2cacff9700 1 -- 172.21.15.52:0/3583205489 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cb4044060 con 0x55e3b91275f0 2024-06-10T02:53:11.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.136+0000 7f2cacff9700 1 -- 172.21.15.52:0/3583205489 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f2cb4017b80 con 0x55e3b91275f0 2024-06-10T02:53:11.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.136+0000 7f2cd5d24b80 1 -- 172.21.15.52:0/3583205489 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2cbc017b60 con 0x55e3b91275f0 2024-06-10T02:53:11.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.136+0000 7f2cacff9700 1 -- 172.21.15.52:0/3583205489 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2cb4034050 con 0x55e3b91275f0 2024-06-10T02:53:11.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.136+0000 7f2cd5d24b80 10 client.15063 fetch_fsmap learned FSMap version 29 2024-06-10T02:53:11.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.136+0000 7f2cd5d24b80 1 -- 172.21.15.52:0/3583205489 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e3b91a7830 con 0x55e3b91275f0 2024-06-10T02:53:11.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.136+0000 7f2cacff9700 1 -- 172.21.15.52:0/3583205489 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2cbc017b60 con 0x55e3b91275f0 2024-06-10T02:53:11.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.137+0000 7f2cacff9700 1 -- 172.21.15.52:0/3583205489 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f2cb403f5c0 con 0x55e3b91275f0 2024-06-10T02:53:11.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.137+0000 7f2cd5d24b80 10 client.15063 fetch_fsmap finished waiting for FSMap version 29 2024-06-10T02:53:11.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.137+0000 7f2cd5d24b80 10 client.15063 resolve_mds: resolved name 'b' to daemon mds.b 2024-06-10T02:53:11.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.137+0000 7f2cd5d24b80 20 client.15063 populate_metadata read hostname 'smithi052' 2024-06-10T02:53:11.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.137+0000 7f2cd5d24b80 1 --2- 172.21.15.52:0/3583205489 >> [v2:172.21.15.120:6838/1672436487,v1:172.21.15.120:6839/1672436487] conn(0x55e3b91ac3e0 0x55e3b91ab0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:11.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.137+0000 7f2cd5d24b80 4 client.15063 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "status"}] 2024-06-10T02:53:11.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.137+0000 7f2cd5d24b80 1 -- 172.21.15.52:0/3583205489 --> [v2:172.21.15.120:6838/1672436487,v1:172.21.15.120:6839/1672436487] -- command(tid 0: {"prefix": "status"}) -- 0x55e3b907b240 con 0x55e3b91ac3e0 2024-06-10T02:53:11.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.137+0000 7f2cc923d700 1 --2- 172.21.15.52:0/3583205489 >> [v2:172.21.15.120:6838/1672436487,v1:172.21.15.120:6839/1672436487] conn(0x55e3b91ac3e0 0x55e3b91ab0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:11.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.138+0000 7f2cc923d700 1 --2- 172.21.15.52:0/3583205489 >> [v2:172.21.15.120:6838/1672436487,v1:172.21.15.120:6839/1672436487] conn(0x55e3b91ac3e0 0x55e3b91ab0f0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=6ff85a3bf3f88471 server_cookie=928402e667a72bf6 in_seq=0 out_seq=0 2024-06-10T02:53:11.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.138+0000 7f2cacff9700 10 client.15063 ms_handle_connect on v2:172.21.15.120:6838/1672436487 2024-06-10T02:53:11.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.138+0000 7f2cacff9700 1 -- 172.21.15.52:0/3583205489 <== mds.3 v2:172.21.15.120:6838/1672436487 1 ==== command_reply(tid 0: 0 ) ==== 8+0+328 (crc 0 0 0) 0x55e3b907b240 con 0x55e3b91ac3e0 2024-06-10T02:53:11.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.138+0000 7f2cacff9700 10 client.15063 handle_command_reply: tid=0 2024-06-10T02:53:11.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.138+0000 7f2cd5d24b80 1 client.15063 shutdown 2024-06-10T02:53:11.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.138+0000 7f2cd5d24b80 1 -- 172.21.15.52:0/3583205489 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e3b91275f0 msgr2=0x55e3b91466c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:53:11.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.138+0000 7f2cd5d24b80 1 --2- 172.21.15.52:0/3583205489 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e3b91275f0 0x55e3b91466c0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f2cb402fdc0 tx=0x7f2cb40176a0 comp rx=0 tx=0).stop 2024-06-10T02:53:11.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.138+0000 7f2cd5d24b80 1 -- 172.21.15.52:0/3583205489 shutdown_connections 2024-06-10T02:53:11.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.138+0000 7f2cd5d24b80 1 --2- 172.21.15.52:0/3583205489 >> [v2:172.21.15.120:6838/1672436487,v1:172.21.15.120:6839/1672436487] conn(0x55e3b91ac3e0 0x55e3b91ab0f0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:11.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.138+0000 7f2cd5d24b80 1 --2- 172.21.15.52:0/3583205489 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e3b9127f10 0x55e3b9141130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:11.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.138+0000 7f2cd5d24b80 1 --2- 172.21.15.52:0/3583205489 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e3b9121ed0 0x55e3b9146180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:11.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.138+0000 7f2cd5d24b80 1 --2- 172.21.15.52:0/3583205489 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e3b91275f0 0x55e3b91466c0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:11.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.138+0000 7f2cd5d24b80 1 -- 172.21.15.52:0/3583205489 >> 172.21.15.52:0/3583205489 conn(0x55e3b904cc70 msgr2=0x55e3b91282f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:53:11.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.138+0000 7f2cacff9700 0 client.15063 ms_handle_reset on v2:172.21.15.120:6838/1672436487 2024-06-10T02:53:11.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.138+0000 7f2cd5d24b80 1 -- 172.21.15.52:0/3583205489 shutdown_connections 2024-06-10T02:53:11.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.138+0000 7f2cd5d24b80 1 -- 172.21.15.52:0/3583205489 wait complete. 2024-06-10T02:53:11.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.138+0000 7f2cd5d24b80 20 client.15063 trim_cache size 0 max 16384 2024-06-10T02:53:11.145 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-10T02:53:11.145 INFO:teuthology.orchestra.run.smithi052.stdout: "cluster_fsid": "d8676e9e-26d2-11ef-bc9b-c7b262605968", 2024-06-10T02:53:11.146 INFO:teuthology.orchestra.run.smithi052.stdout: "whoami": 3, 2024-06-10T02:53:11.146 INFO:teuthology.orchestra.run.smithi052.stdout: "id": 24449, 2024-06-10T02:53:11.146 INFO:teuthology.orchestra.run.smithi052.stdout: "want_state": "up:active", 2024-06-10T02:53:11.146 INFO:teuthology.orchestra.run.smithi052.stdout: "state": "up:active", 2024-06-10T02:53:11.146 INFO:teuthology.orchestra.run.smithi052.stdout: "fs_name": "cephfs", 2024-06-10T02:53:11.146 INFO:teuthology.orchestra.run.smithi052.stdout: "rank_uptime": 123.530416734, 2024-06-10T02:53:11.146 INFO:teuthology.orchestra.run.smithi052.stdout: "mdsmap_epoch": 29, 2024-06-10T02:53:11.146 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_epoch": 100, 2024-06-10T02:53:11.146 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_epoch_barrier": 98, 2024-06-10T02:53:11.146 INFO:teuthology.orchestra.run.smithi052.stdout: "uptime": 204.45406302000001 2024-06-10T02:53:11.146 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-10T02:53:11.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.139+0000 7f2ccb4a1700 1 -- 172.21.15.52:0/2605785456 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f2cb00a2c20 msgr2=0x7f2cb00a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:53:11.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.139+0000 7f2ccb4a1700 1 --2- 172.21.15.52:0/2605785456 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f2cb00a2c20 0x7f2cb00a50e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f2cb400b2e0 tx=0x7f2cb401a040 comp rx=0 tx=0).stop 2024-06-10T02:53:11.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.139+0000 7f2ccb4a1700 1 -- 172.21.15.52:0/2605785456 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2cc406abc0 msgr2=0x7f2cc419af10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:53:11.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.139+0000 7f2ccb4a1700 1 --2- 172.21.15.52:0/2605785456 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2cc406abc0 0x7f2cc419af10 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f2cc000d980 tx=0x7f2cc000dd40 comp rx=0 tx=0).stop 2024-06-10T02:53:11.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.139+0000 7f2ccb4a1700 1 -- 172.21.15.52:0/2605785456 shutdown_connections 2024-06-10T02:53:11.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.139+0000 7f2ccb4a1700 1 --2- 172.21.15.52:0/2605785456 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2cc410ef70 0x7f2cc419f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:11.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.139+0000 7f2ccb4a1700 1 --2- 172.21.15.52:0/2605785456 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f2cb00a2c20 0x7f2cb00a50e0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:11.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.139+0000 7f2ccb4a1700 1 --2- 172.21.15.52:0/2605785456 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2cc406abc0 0x7f2cc419af10 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:11.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.139+0000 7f2ccb4a1700 1 --2- 172.21.15.52:0/2605785456 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2cc406b4e0 0x7f2cc419b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:11.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.139+0000 7f2ccb4a1700 1 -- 172.21.15.52:0/2605785456 >> 172.21.15.52:0/2605785456 conn(0x7f2cc4078340 msgr2=0x7f2cc4110420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:53:11.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.139+0000 7f2ccb4a1700 1 -- 172.21.15.52:0/2605785456 shutdown_connections 2024-06-10T02:53:11.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:11.139+0000 7f2ccb4a1700 1 -- 172.21.15.52:0/2605785456 wait complete. 2024-06-10T02:53:11.149 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:53:10 smithi120 ceph-mon[36634]: pgmap v403: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-10T02:53:11.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:53:10 smithi188 ceph-mon[36599]: pgmap v403: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-10T02:53:11.833 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph tell mds.f status 2024-06-10T02:53:12.083 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:53:13.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:53:12 smithi120 ceph-mon[36634]: pgmap v404: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-10T02:53:13.202 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:53:12 smithi052 ceph-mon[32362]: pgmap v404: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-10T02:53:13.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:53:12 smithi188 ceph-mon[36599]: pgmap v404: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-10T02:53:13.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.783+0000 7f917bf2b700 1 -- 172.21.15.52:0/4008560068 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f917406b3a0 msgr2=0x7f917410f310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:53:13.786 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.783+0000 7f917bf2b700 1 --2- 172.21.15.52:0/4008560068 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f917406b3a0 0x7f917410f310 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f916c009990 tx=0x7f916c009ca0 comp rx=0 tx=0).stop 2024-06-10T02:53:13.786 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.783+0000 7f917bf2b700 1 -- 172.21.15.52:0/4008560068 shutdown_connections 2024-06-10T02:53:13.786 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.783+0000 7f917bf2b700 1 --2- 172.21.15.52:0/4008560068 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f917406b3a0 0x7f917410f310 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:13.786 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.783+0000 7f917bf2b700 1 --2- 172.21.15.52:0/4008560068 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f91741084d0 0x7f91741088b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:13.786 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.783+0000 7f917bf2b700 1 --2- 172.21.15.52:0/4008560068 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f917406aa00 0x7f917406ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:13.786 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.783+0000 7f917bf2b700 1 -- 172.21.15.52:0/4008560068 >> 172.21.15.52:0/4008560068 conn(0x7f9174078330 msgr2=0x7f9174078740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:53:13.787 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.783+0000 7f917bf2b700 1 -- 172.21.15.52:0/4008560068 shutdown_connections 2024-06-10T02:53:13.787 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.783+0000 7f917bf2b700 1 -- 172.21.15.52:0/4008560068 wait complete. 2024-06-10T02:53:13.787 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.784+0000 7f917bf2b700 1 Processor -- start 2024-06-10T02:53:13.787 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.784+0000 7f917bf2b700 1 -- start start 2024-06-10T02:53:13.787 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.784+0000 7f917bf2b700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f917406aa00 0x7f9174101450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:13.787 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.784+0000 7f917bf2b700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f917406b3a0 0x7f9174101990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:13.787 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.784+0000 7f917bf2b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f91741084d0 0x7f9174101ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:13.787 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.784+0000 7f917bf2b700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f9174112750 con 0x7f91741084d0 2024-06-10T02:53:13.787 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.784+0000 7f917bf2b700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f91741125d0 con 0x7f917406b3a0 2024-06-10T02:53:13.787 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.784+0000 7f917bf2b700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f91741128d0 con 0x7f917406aa00 2024-06-10T02:53:13.787 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.785+0000 7f917a4c8700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f91741084d0 0x7f9174101ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:13.787 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.785+0000 7f917a4c8700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f91741084d0 0x7f9174101ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53120/0 (socket says 172.21.15.52:53120) 2024-06-10T02:53:13.788 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.785+0000 7f917a4c8700 1 -- 172.21.15.52:0/1496501928 learned_addr learned my addr 172.21.15.52:0/1496501928 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:53:13.788 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.785+0000 7f9179cc7700 1 --2- 172.21.15.52:0/1496501928 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f917406aa00 0x7f9174101450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:13.788 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.785+0000 7f91794c6700 1 --2- 172.21.15.52:0/1496501928 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f917406b3a0 0x7f9174101990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:13.788 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.785+0000 7f917a4c8700 1 -- 172.21.15.52:0/1496501928 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f917406aa00 msgr2=0x7f9174101450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:53:13.788 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.785+0000 7f917a4c8700 1 --2- 172.21.15.52:0/1496501928 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f917406aa00 0x7f9174101450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:13.788 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.785+0000 7f917a4c8700 1 -- 172.21.15.52:0/1496501928 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f917406b3a0 msgr2=0x7f9174101990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:53:13.788 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.785+0000 7f917a4c8700 1 --2- 172.21.15.52:0/1496501928 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f917406b3a0 0x7f9174101990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:13.788 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.785+0000 7f917a4c8700 1 -- 172.21.15.52:0/1496501928 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91741a7150 con 0x7f91741084d0 2024-06-10T02:53:13.788 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.785+0000 7f917a4c8700 1 --2- 172.21.15.52:0/1496501928 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f91741084d0 0x7f9174101ed0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f916c00b820 tx=0x7f916c00bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:53:13.788 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.785+0000 7f9172ffd700 1 -- 172.21.15.52:0/1496501928 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f916c027070 con 0x7f91741084d0 2024-06-10T02:53:13.789 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.785+0000 7f9172ffd700 1 -- 172.21.15.52:0/1496501928 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f916c003d70 con 0x7f91741084d0 2024-06-10T02:53:13.789 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.785+0000 7f917bf2b700 1 -- 172.21.15.52:0/1496501928 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91741a73e0 con 0x7f91741084d0 2024-06-10T02:53:13.789 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.785+0000 7f9172ffd700 1 -- 172.21.15.52:0/1496501928 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f916c00f460 con 0x7f91741084d0 2024-06-10T02:53:13.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.785+0000 7f917bf2b700 1 -- 172.21.15.52:0/1496501928 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91741a7890 con 0x7f91741084d0 2024-06-10T02:53:13.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.788+0000 7f9172ffd700 1 -- 172.21.15.52:0/1496501928 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f916c02a070 con 0x7f91741084d0 2024-06-10T02:53:13.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.788+0000 7f9172ffd700 1 --2- 172.21.15.52:0/1496501928 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f915c0a2ca0 0x7f915c0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:13.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.788+0000 7f9172ffd700 1 -- 172.21.15.52:0/1496501928 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f916c0d0230 con 0x7f91741084d0 2024-06-10T02:53:13.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.788+0000 7f9179cc7700 1 --2- 172.21.15.52:0/1496501928 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f915c0a2ca0 0x7f915c0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:13.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.789+0000 7f91867aeb80 1 Processor -- start 2024-06-10T02:53:13.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.789+0000 7f9179cc7700 1 --2- 172.21.15.52:0/1496501928 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f915c0a2ca0 0x7f915c0a5160 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f916400f540 tx=0x7f916400b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:53:13.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.789+0000 7f91867aeb80 1 -- start start 2024-06-10T02:53:13.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.789+0000 7f91867aeb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558b31f3d5f0 0x558b31f3d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:13.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.789+0000 7f91867aeb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558b31f3df10 0x558b31f37990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:13.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.789+0000 7f91867aeb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558b31f37ed0 0x558b31f382f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:13.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.789+0000 7f91867aeb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x558b31e897c0 con 0x558b31f3df10 2024-06-10T02:53:13.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.789+0000 7f91867aeb80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x558b31e316d0 con 0x558b31f3d5f0 2024-06-10T02:53:13.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.789+0000 7f91867aeb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x558b31e85780 con 0x558b31f37ed0 2024-06-10T02:53:13.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.789+0000 7f91794c6700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558b31f3df10 0x558b31f37990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:13.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.789+0000 7f91794c6700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558b31f3df10 0x558b31f37990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53124/0 (socket says 172.21.15.52:53124) 2024-06-10T02:53:13.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.789+0000 7f91794c6700 1 -- 172.21.15.52:0/3413923078 learned_addr learned my addr 172.21.15.52:0/3413923078 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:53:13.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.790+0000 7f9179cc7700 1 --2- 172.21.15.52:0/3413923078 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558b31f3d5f0 0x558b31f3d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:13.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.790+0000 7f917a4c8700 1 --2- 172.21.15.52:0/3413923078 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558b31f37ed0 0x558b31f382f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:13.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.790+0000 7f91794c6700 1 -- 172.21.15.52:0/3413923078 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558b31f37ed0 msgr2=0x558b31f382f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:53:13.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.790+0000 7f91794c6700 1 --2- 172.21.15.52:0/3413923078 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558b31f37ed0 0x558b31f382f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:13.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.790+0000 7f91794c6700 1 -- 172.21.15.52:0/3413923078 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558b31f3d5f0 msgr2=0x558b31f3d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:53:13.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.790+0000 7f91794c6700 1 --2- 172.21.15.52:0/3413923078 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558b31f3d5f0 0x558b31f3d9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:13.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.790+0000 7f91794c6700 1 -- 172.21.15.52:0/3413923078 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b31e27f00 con 0x558b31f3df10 2024-06-10T02:53:13.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.790+0000 7f917a4c8700 1 --2- 172.21.15.52:0/3413923078 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558b31f37ed0 0x558b31f382f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T02:53:13.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.790+0000 7f91794c6700 1 --2- 172.21.15.52:0/3413923078 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558b31f3df10 0x558b31f37990 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f91680046c0 tx=0x7f916800b9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44ed4a0dd97d9db3 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:53:13.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.790+0000 7f915bfff700 1 -- 172.21.15.52:0/3413923078 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9168004a30 con 0x558b31f3df10 2024-06-10T02:53:13.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.790+0000 7f915bfff700 1 -- 172.21.15.52:0/3413923078 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9168015070 con 0x558b31f3df10 2024-06-10T02:53:13.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.790+0000 7f915bfff700 1 -- 172.21.15.52:0/3413923078 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9168003c20 con 0x558b31f3df10 2024-06-10T02:53:13.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.791+0000 7f91867aeb80 1 -- 172.21.15.52:0/3413923078 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558b31f3df10 msgr2=0x558b31f37990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:53:13.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.791+0000 7f91867aeb80 1 --2- 172.21.15.52:0/3413923078 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558b31f3df10 0x558b31f37990 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f91680046c0 tx=0x7f916800b9e0 comp rx=0 tx=0).stop 2024-06-10T02:53:13.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.791+0000 7f91867aeb80 1 -- 172.21.15.52:0/3413923078 shutdown_connections 2024-06-10T02:53:13.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.791+0000 7f91867aeb80 1 --2- 172.21.15.52:0/3413923078 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558b31f37ed0 0x558b31f382f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:13.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.791+0000 7f91867aeb80 1 --2- 172.21.15.52:0/3413923078 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558b31f3d5f0 0x558b31f3d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:13.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.791+0000 7f91867aeb80 1 --2- 172.21.15.52:0/3413923078 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558b31f3df10 0x558b31f37990 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:13.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.791+0000 7f91867aeb80 1 -- 172.21.15.52:0/3413923078 >> 172.21.15.52:0/3413923078 conn(0x558b31e62c70 msgr2=0x558b31f32800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:53:13.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.791+0000 7f91867aeb80 1 -- 172.21.15.52:0/3413923078 shutdown_connections 2024-06-10T02:53:13.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.791+0000 7f91867aeb80 1 -- 172.21.15.52:0/3413923078 wait complete. 2024-06-10T02:53:13.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.792+0000 7f91867aeb80 1 -- start start 2024-06-10T02:53:13.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.792+0000 7f91867aeb80 1 Processor -- start 2024-06-10T02:53:13.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.792+0000 7f91867aeb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558b31f37ed0 0x558b31f43ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:13.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.792+0000 7f91867aeb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558b31f3d5f0 0x558b31f40fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:13.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.792+0000 7f91867aeb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558b31f3df10 0x558b31f414e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:13.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.792+0000 7f91867aeb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x558b31f44980 con 0x558b31f37ed0 2024-06-10T02:53:13.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.793+0000 7f9179cc7700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558b31f37ed0 0x558b31f43ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:13.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.793+0000 7f91867aeb80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x558b31e29ba0 con 0x558b31f3df10 2024-06-10T02:53:13.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.793+0000 7f91794c6700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558b31f3d5f0 0x558b31f40fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:13.801 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.793+0000 7f9179cc7700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558b31f37ed0 0x558b31f43ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53126/0 (socket says 172.21.15.52:53126) 2024-06-10T02:53:13.801 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.793+0000 7f9179cc7700 1 -- 172.21.15.52:0/502480446 learned_addr learned my addr 172.21.15.52:0/502480446 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:53:13.801 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.793+0000 7f917a4c8700 1 --2- 172.21.15.52:0/502480446 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558b31f3df10 0x558b31f414e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:13.801 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.793+0000 7f91867aeb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x558b31f44b00 con 0x558b31f3d5f0 2024-06-10T02:53:13.801 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.793+0000 7f91867aeb80 10 client.0 _ll_register_callbacks cb 0x558b31e238d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T02:53:13.801 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.793+0000 7f91794c6700 1 -- 172.21.15.52:0/502480446 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558b31f3df10 msgr2=0x558b31f414e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:53:13.801 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.793+0000 7f91794c6700 1 --2- 172.21.15.52:0/502480446 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558b31f3df10 0x558b31f414e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:13.801 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.793+0000 7f91794c6700 1 -- 172.21.15.52:0/502480446 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558b31f37ed0 msgr2=0x558b31f43ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:53:13.801 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.793+0000 7f91794c6700 1 --2- 172.21.15.52:0/502480446 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558b31f37ed0 0x558b31f43ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:13.801 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.793+0000 7f91794c6700 1 -- 172.21.15.52:0/502480446 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b31dd8880 con 0x558b31f3d5f0 2024-06-10T02:53:13.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.793+0000 7f917a4c8700 1 --2- 172.21.15.52:0/502480446 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558b31f3df10 0x558b31f414e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T02:53:13.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.793+0000 7f9179cc7700 1 --2- 172.21.15.52:0/502480446 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558b31f37ed0 0x558b31f43ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T02:53:13.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.793+0000 7f91794c6700 1 --2- 172.21.15.52:0/502480446 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558b31f3d5f0 0x558b31f40fa0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9168003780 tx=0x7f91680049f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=67e7a83b940ec686 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:53:13.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.793+0000 7f91597fa700 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-06-10T02:53:13.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.794+0000 7f91597fa700 1 -- 172.21.15.52:0/502480446 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9168028060 con 0x558b31f3d5f0 2024-06-10T02:53:13.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.794+0000 7f91867aeb80 1 -- 172.21.15.52:0/502480446 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9168015070 con 0x558b31f3d5f0 2024-06-10T02:53:13.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.794+0000 7f91597fa700 1 -- 172.21.15.52:0/502480446 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f9168003880 con 0x558b31f3d5f0 2024-06-10T02:53:13.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.794+0000 7f91597fa700 1 -- 172.21.15.52:0/502480446 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f9168003a80 con 0x558b31f3d5f0 2024-06-10T02:53:13.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.794+0000 7f91867aeb80 10 client.24835 fetch_fsmap learned FSMap version 29 2024-06-10T02:53:13.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.794+0000 7f91867aeb80 1 -- 172.21.15.52:0/502480446 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558b31f4f1e0 con 0x558b31f3d5f0 2024-06-10T02:53:13.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.794+0000 7f91597fa700 1 -- 172.21.15.52:0/502480446 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9168015070 con 0x558b31f3d5f0 2024-06-10T02:53:13.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.794+0000 7f91597fa700 1 -- 172.21.15.52:0/502480446 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f9168011440 con 0x558b31f3d5f0 2024-06-10T02:53:13.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.794+0000 7f91867aeb80 10 client.24835 fetch_fsmap finished waiting for FSMap version 29 2024-06-10T02:53:13.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.794+0000 7f91867aeb80 10 client.24835 resolve_mds: resolved name 'f' to daemon mds.f 2024-06-10T02:53:13.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.794+0000 7f91867aeb80 20 client.24835 populate_metadata read hostname 'smithi052' 2024-06-10T02:53:13.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.794+0000 7f91867aeb80 1 --2- 172.21.15.52:0/502480446 >> [v2:172.21.15.188:6834/3904544259,v1:172.21.15.188:6835/3904544259] conn(0x558b31f51400 0x558b31f52870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:13.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.794+0000 7f91867aeb80 4 client.24835 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T02:53:13.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.794+0000 7f91867aeb80 1 -- 172.21.15.52:0/502480446 --> [v2:172.21.15.188:6834/3904544259,v1:172.21.15.188:6835/3904544259] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558b31e96cc0 con 0x558b31f51400 2024-06-10T02:53:13.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.795+0000 7f9179cc7700 1 --2- 172.21.15.52:0/502480446 >> [v2:172.21.15.188:6834/3904544259,v1:172.21.15.188:6835/3904544259] conn(0x558b31f51400 0x558b31f52870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:13.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.795+0000 7f9179cc7700 1 --2- 172.21.15.52:0/502480446 >> [v2:172.21.15.188:6834/3904544259,v1:172.21.15.188:6835/3904544259] conn(0x558b31f51400 0x558b31f52870 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d8ad275b3d34a1b2 server_cookie=fb76d430fef23d3d in_seq=0 out_seq=0 2024-06-10T02:53:13.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.795+0000 7f91597fa700 10 client.24835 ms_handle_connect on v2:172.21.15.188:6834/3904544259 2024-06-10T02:53:13.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.796+0000 7f91597fa700 1 -- 172.21.15.52:0/502480446 <== mds.2 v2:172.21.15.188:6834/3904544259 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x558b31e96cc0 con 0x558b31f51400 2024-06-10T02:53:13.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.796+0000 7f91597fa700 10 client.24835 handle_command_reply: tid=0 2024-06-10T02:53:13.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.796+0000 7f91867aeb80 1 client.24835 shutdown 2024-06-10T02:53:13.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.796+0000 7f91867aeb80 1 -- 172.21.15.52:0/502480446 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558b31f3d5f0 msgr2=0x558b31f40fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:53:13.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.796+0000 7f91867aeb80 1 --2- 172.21.15.52:0/502480446 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558b31f3d5f0 0x558b31f40fa0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9168003780 tx=0x7f91680049f0 comp rx=0 tx=0).stop 2024-06-10T02:53:13.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.796+0000 7f91867aeb80 1 -- 172.21.15.52:0/502480446 shutdown_connections 2024-06-10T02:53:13.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.796+0000 7f91867aeb80 1 --2- 172.21.15.52:0/502480446 >> [v2:172.21.15.188:6834/3904544259,v1:172.21.15.188:6835/3904544259] conn(0x558b31f51400 0x558b31f52870 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:13.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.796+0000 7f91867aeb80 1 --2- 172.21.15.52:0/502480446 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558b31f3df10 0x558b31f414e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:13.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.796+0000 7f91867aeb80 1 --2- 172.21.15.52:0/502480446 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558b31f37ed0 0x558b31f43ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:13.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.796+0000 7f91867aeb80 1 --2- 172.21.15.52:0/502480446 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558b31f3d5f0 0x558b31f40fa0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:13.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.796+0000 7f91867aeb80 1 -- 172.21.15.52:0/502480446 >> 172.21.15.52:0/502480446 conn(0x558b31e62c70 msgr2=0x558b31f36980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:53:13.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.796+0000 7f91597fa700 0 client.24835 ms_handle_reset on v2:172.21.15.188:6834/3904544259 2024-06-10T02:53:13.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.796+0000 7f91867aeb80 1 -- 172.21.15.52:0/502480446 shutdown_connections 2024-06-10T02:53:13.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.796+0000 7f91867aeb80 1 -- 172.21.15.52:0/502480446 wait complete. 2024-06-10T02:53:13.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.796+0000 7f91867aeb80 20 client.24835 trim_cache size 0 max 16384 2024-06-10T02:53:13.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.819+0000 7f91867aeb80 1 Processor -- start 2024-06-10T02:53:13.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.819+0000 7f91867aeb80 1 -- start start 2024-06-10T02:53:13.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.819+0000 7f91867aeb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558b31f37ed0 0x558b31f37990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:13.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.819+0000 7f91867aeb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558b31f3d5f0 0x558b31f43ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:13.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.819+0000 7f91867aeb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558b31f3df10 0x558b31f2bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:13.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.819+0000 7f91867aeb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x558b31f50bb0 con 0x558b31f3df10 2024-06-10T02:53:13.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.819+0000 7f91867aeb80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x558b31f51280 con 0x558b31f3d5f0 2024-06-10T02:53:13.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.819+0000 7f91867aeb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x558b31f51950 con 0x558b31f37ed0 2024-06-10T02:53:13.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.820+0000 7f917a4c8700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558b31f3df10 0x558b31f2bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:13.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.820+0000 7f917a4c8700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558b31f3df10 0x558b31f2bdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53128/0 (socket says 172.21.15.52:53128) 2024-06-10T02:53:13.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.820+0000 7f917a4c8700 1 -- 172.21.15.52:0/3458121601 learned_addr learned my addr 172.21.15.52:0/3458121601 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:53:13.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.820+0000 7f9179cc7700 1 --2- 172.21.15.52:0/3458121601 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558b31f37ed0 0x558b31f37990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:13.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.820+0000 7f91794c6700 1 --2- 172.21.15.52:0/3458121601 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558b31f3d5f0 0x558b31f43ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:13.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.820+0000 7f917a4c8700 1 -- 172.21.15.52:0/3458121601 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558b31f37ed0 msgr2=0x558b31f37990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:53:13.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.820+0000 7f917a4c8700 1 --2- 172.21.15.52:0/3458121601 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558b31f37ed0 0x558b31f37990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:13.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.820+0000 7f917a4c8700 1 -- 172.21.15.52:0/3458121601 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558b31f3d5f0 msgr2=0x558b31f43ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:53:13.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.820+0000 7f917a4c8700 1 --2- 172.21.15.52:0/3458121601 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558b31f3d5f0 0x558b31f43ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:13.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.820+0000 7f917a4c8700 1 -- 172.21.15.52:0/3458121601 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b31f3f510 con 0x558b31f3df10 2024-06-10T02:53:13.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.820+0000 7f91794c6700 1 --2- 172.21.15.52:0/3458121601 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558b31f3d5f0 0x558b31f43ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T02:53:13.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.820+0000 7f9179cc7700 1 --2- 172.21.15.52:0/3458121601 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558b31f37ed0 0x558b31f37990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T02:53:13.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.820+0000 7f917a4c8700 1 --2- 172.21.15.52:0/3458121601 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558b31f3df10 0x558b31f2bdb0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f916c09c470 tx=0x7f916c0a3040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5629867a2806fc9 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:53:13.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.820+0000 7f915b7fe700 1 -- 172.21.15.52:0/3458121601 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f916c0b4020 con 0x558b31f3df10 2024-06-10T02:53:13.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.820+0000 7f915b7fe700 1 -- 172.21.15.52:0/3458121601 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f916c0cee50 con 0x558b31f3df10 2024-06-10T02:53:13.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.820+0000 7f915b7fe700 1 -- 172.21.15.52:0/3458121601 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f916c09c770 con 0x558b31f3df10 2024-06-10T02:53:13.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.821+0000 7f91867aeb80 1 -- 172.21.15.52:0/3458121601 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558b31f3df10 msgr2=0x558b31f2bdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:53:13.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.821+0000 7f91867aeb80 1 --2- 172.21.15.52:0/3458121601 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558b31f3df10 0x558b31f2bdb0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f916c09c470 tx=0x7f916c0a3040 comp rx=0 tx=0).stop 2024-06-10T02:53:13.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.821+0000 7f91867aeb80 1 -- 172.21.15.52:0/3458121601 shutdown_connections 2024-06-10T02:53:13.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.821+0000 7f91867aeb80 1 --2- 172.21.15.52:0/3458121601 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558b31f3df10 0x558b31f2bdb0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:13.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.821+0000 7f91867aeb80 1 --2- 172.21.15.52:0/3458121601 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558b31f37ed0 0x558b31f37990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:13.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.821+0000 7f91867aeb80 1 --2- 172.21.15.52:0/3458121601 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558b31f3d5f0 0x558b31f43ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:13.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.821+0000 7f91867aeb80 1 -- 172.21.15.52:0/3458121601 >> 172.21.15.52:0/3458121601 conn(0x558b31e62c70 msgr2=0x558b31f3f890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:53:13.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.821+0000 7f91867aeb80 1 -- 172.21.15.52:0/3458121601 shutdown_connections 2024-06-10T02:53:13.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.821+0000 7f91867aeb80 1 -- 172.21.15.52:0/3458121601 wait complete. 2024-06-10T02:53:13.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.822+0000 7f91867aeb80 1 -- start start 2024-06-10T02:53:13.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.822+0000 7f91867aeb80 1 Processor -- start 2024-06-10T02:53:13.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.822+0000 7f91867aeb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558b31f37ed0 0x558b31f4cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:13.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.822+0000 7f91867aeb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558b31f3d5f0 0x558b31f4d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:13.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.822+0000 7f91867aeb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558b31f3df10 0x558b31f5ace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:13.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.822+0000 7f91867aeb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x558b31f526f0 con 0x558b31f3d5f0 2024-06-10T02:53:13.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.822+0000 7f91867aeb80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x558b31f52020 con 0x558b31f3df10 2024-06-10T02:53:13.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.822+0000 7f91867aeb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x558b31f52dc0 con 0x558b31f37ed0 2024-06-10T02:53:13.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.822+0000 7f91794c6700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558b31f3d5f0 0x558b31f4d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:13.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.822+0000 7f91794c6700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558b31f3d5f0 0x558b31f4d4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53134/0 (socket says 172.21.15.52:53134) 2024-06-10T02:53:13.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.822+0000 7f91867aeb80 10 client.0 _ll_register_callbacks cb 0x558b31e238d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T02:53:13.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.822+0000 7f91794c6700 1 -- 172.21.15.52:0/432826618 learned_addr learned my addr 172.21.15.52:0/432826618 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:53:13.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.822+0000 7f9179cc7700 1 --2- 172.21.15.52:0/432826618 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558b31f37ed0 0x558b31f4cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:13.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.822+0000 7f917a4c8700 1 --2- 172.21.15.52:0/432826618 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558b31f3df10 0x558b31f5ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:13.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.823+0000 7f91794c6700 1 -- 172.21.15.52:0/432826618 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558b31f37ed0 msgr2=0x558b31f4cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:53:13.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.823+0000 7f91794c6700 1 --2- 172.21.15.52:0/432826618 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558b31f37ed0 0x558b31f4cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:13.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.823+0000 7f91794c6700 1 -- 172.21.15.52:0/432826618 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558b31f3df10 msgr2=0x558b31f5ace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:53:13.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.823+0000 7f91794c6700 1 --2- 172.21.15.52:0/432826618 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558b31f3df10 0x558b31f5ace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:13.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.823+0000 7f91794c6700 1 -- 172.21.15.52:0/432826618 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b31f5b2a0 con 0x558b31f3d5f0 2024-06-10T02:53:13.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.823+0000 7f9179cc7700 1 --2- 172.21.15.52:0/432826618 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558b31f37ed0 0x558b31f4cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T02:53:13.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.823+0000 7f91794c6700 1 --2- 172.21.15.52:0/432826618 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558b31f3d5f0 0x558b31f4d4e0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x558b31f34260 tx=0x7f916800b440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27479b24ff1879f server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:53:13.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.823+0000 7f91597fa700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-10T02:53:13.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.823+0000 7f91597fa700 1 -- 172.21.15.52:0/432826618 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9168028040 con 0x558b31f3d5f0 2024-06-10T02:53:13.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.823+0000 7f91597fa700 1 -- 172.21.15.52:0/432826618 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f9168011a80 con 0x558b31f3d5f0 2024-06-10T02:53:13.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.823+0000 7f91867aeb80 1 -- 172.21.15.52:0/432826618 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f916c0cee50 con 0x558b31f3d5f0 2024-06-10T02:53:13.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.823+0000 7f91597fa700 1 -- 172.21.15.52:0/432826618 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f9168011c80 con 0x558b31f3d5f0 2024-06-10T02:53:13.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.823+0000 7f91867aeb80 10 client.15081 fetch_fsmap learned FSMap version 29 2024-06-10T02:53:13.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.823+0000 7f91867aeb80 1 -- 172.21.15.52:0/432826618 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558b31f5c560 con 0x558b31f3d5f0 2024-06-10T02:53:13.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.823+0000 7f91597fa700 1 -- 172.21.15.52:0/432826618 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f916c0cee50 con 0x558b31f3d5f0 2024-06-10T02:53:13.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.824+0000 7f91597fa700 1 -- 172.21.15.52:0/432826618 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f9168024480 con 0x558b31f3d5f0 2024-06-10T02:53:13.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.824+0000 7f91867aeb80 10 client.15081 fetch_fsmap finished waiting for FSMap version 29 2024-06-10T02:53:13.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.824+0000 7f91867aeb80 10 client.15081 resolve_mds: resolved name 'f' to daemon mds.f 2024-06-10T02:53:13.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.824+0000 7f91867aeb80 20 client.15081 populate_metadata read hostname 'smithi052' 2024-06-10T02:53:13.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.824+0000 7f91867aeb80 1 --2- 172.21.15.52:0/432826618 >> [v2:172.21.15.188:6834/3904544259,v1:172.21.15.188:6835/3904544259] conn(0x558b31fc1080 0x558b31fbfd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:13.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.824+0000 7f91867aeb80 4 client.15081 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "status"}] 2024-06-10T02:53:13.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.824+0000 7f91867aeb80 1 -- 172.21.15.52:0/432826618 --> [v2:172.21.15.188:6834/3904544259,v1:172.21.15.188:6835/3904544259] -- command(tid 0: {"prefix": "status"}) -- 0x558b31e91240 con 0x558b31fc1080 2024-06-10T02:53:13.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.824+0000 7f9179cc7700 1 --2- 172.21.15.52:0/432826618 >> [v2:172.21.15.188:6834/3904544259,v1:172.21.15.188:6835/3904544259] conn(0x558b31fc1080 0x558b31fbfd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:13.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.824+0000 7f9179cc7700 1 --2- 172.21.15.52:0/432826618 >> [v2:172.21.15.188:6834/3904544259,v1:172.21.15.188:6835/3904544259] conn(0x558b31fc1080 0x558b31fbfd90 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e4836b71cdfbdfd7 server_cookie=5f45f6cf5786eece in_seq=0 out_seq=0 2024-06-10T02:53:13.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.824+0000 7f91597fa700 10 client.15081 ms_handle_connect on v2:172.21.15.188:6834/3904544259 2024-06-10T02:53:13.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.824+0000 7f91597fa700 1 -- 172.21.15.52:0/432826618 <== mds.2 v2:172.21.15.188:6834/3904544259 1 ==== command_reply(tid 0: 0 ) ==== 8+0+328 (crc 0 0 0) 0x558b31e91240 con 0x558b31fc1080 2024-06-10T02:53:13.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.824+0000 7f91597fa700 10 client.15081 handle_command_reply: tid=0 2024-06-10T02:53:13.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.825+0000 7f91867aeb80 1 client.15081 shutdown 2024-06-10T02:53:13.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.825+0000 7f91867aeb80 1 -- 172.21.15.52:0/432826618 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558b31f3d5f0 msgr2=0x558b31f4d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:53:13.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.825+0000 7f91867aeb80 1 --2- 172.21.15.52:0/432826618 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558b31f3d5f0 0x558b31f4d4e0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x558b31f34260 tx=0x7f916800b440 comp rx=0 tx=0).stop 2024-06-10T02:53:13.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.825+0000 7f91867aeb80 1 -- 172.21.15.52:0/432826618 shutdown_connections 2024-06-10T02:53:13.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.825+0000 7f91867aeb80 1 --2- 172.21.15.52:0/432826618 >> [v2:172.21.15.188:6834/3904544259,v1:172.21.15.188:6835/3904544259] conn(0x558b31fc1080 0x558b31fbfd90 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:13.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.825+0000 7f91867aeb80 1 --2- 172.21.15.52:0/432826618 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558b31f3df10 0x558b31f5ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:13.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.825+0000 7f91867aeb80 1 --2- 172.21.15.52:0/432826618 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558b31f37ed0 0x558b31f4cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:13.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.825+0000 7f91597fa700 0 client.15081 ms_handle_reset on v2:172.21.15.188:6834/3904544259 2024-06-10T02:53:13.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.825+0000 7f91867aeb80 1 --2- 172.21.15.52:0/432826618 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558b31f3d5f0 0x558b31f4d4e0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:13.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.825+0000 7f91867aeb80 1 -- 172.21.15.52:0/432826618 >> 172.21.15.52:0/432826618 conn(0x558b31e62c70 msgr2=0x558b31f2bdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:53:13.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.825+0000 7f91867aeb80 1 -- 172.21.15.52:0/432826618 shutdown_connections 2024-06-10T02:53:13.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.825+0000 7f91867aeb80 1 -- 172.21.15.52:0/432826618 wait complete. 2024-06-10T02:53:13.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.825+0000 7f91867aeb80 20 client.15081 trim_cache size 0 max 16384 2024-06-10T02:53:13.833 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-10T02:53:13.833 INFO:teuthology.orchestra.run.smithi052.stdout: "cluster_fsid": "d8676e9e-26d2-11ef-bc9b-c7b262605968", 2024-06-10T02:53:13.833 INFO:teuthology.orchestra.run.smithi052.stdout: "whoami": 2, 2024-06-10T02:53:13.833 INFO:teuthology.orchestra.run.smithi052.stdout: "id": 24463, 2024-06-10T02:53:13.833 INFO:teuthology.orchestra.run.smithi052.stdout: "want_state": "up:active", 2024-06-10T02:53:13.833 INFO:teuthology.orchestra.run.smithi052.stdout: "state": "up:active", 2024-06-10T02:53:13.834 INFO:teuthology.orchestra.run.smithi052.stdout: "fs_name": "cephfs", 2024-06-10T02:53:13.834 INFO:teuthology.orchestra.run.smithi052.stdout: "rank_uptime": 127.220959838, 2024-06-10T02:53:13.834 INFO:teuthology.orchestra.run.smithi052.stdout: "mdsmap_epoch": 29, 2024-06-10T02:53:13.834 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_epoch": 100, 2024-06-10T02:53:13.834 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_epoch_barrier": 98, 2024-06-10T02:53:13.834 INFO:teuthology.orchestra.run.smithi052.stdout: "uptime": 226.20057431000001 2024-06-10T02:53:13.834 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-10T02:53:13.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.825+0000 7f917bf2b700 1 -- 172.21.15.52:0/1496501928 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f915c0a2ca0 msgr2=0x7f915c0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:53:13.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.825+0000 7f917bf2b700 1 --2- 172.21.15.52:0/1496501928 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f915c0a2ca0 0x7f915c0a5160 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f916400f540 tx=0x7f916400b040 comp rx=0 tx=0).stop 2024-06-10T02:53:13.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.825+0000 7f917bf2b700 1 -- 172.21.15.52:0/1496501928 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f91741084d0 msgr2=0x7f9174101ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:53:13.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.825+0000 7f917bf2b700 1 --2- 172.21.15.52:0/1496501928 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f91741084d0 0x7f9174101ed0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f916c00b820 tx=0x7f916c00bab0 comp rx=0 tx=0).stop 2024-06-10T02:53:13.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.826+0000 7f917bf2b700 1 -- 172.21.15.52:0/1496501928 shutdown_connections 2024-06-10T02:53:13.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.826+0000 7f917bf2b700 1 --2- 172.21.15.52:0/1496501928 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f915c0a2ca0 0x7f915c0a5160 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:13.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.826+0000 7f917bf2b700 1 --2- 172.21.15.52:0/1496501928 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f91741084d0 0x7f9174101ed0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:13.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.826+0000 7f917bf2b700 1 --2- 172.21.15.52:0/1496501928 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f917406aa00 0x7f9174101450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:13.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.826+0000 7f917bf2b700 1 --2- 172.21.15.52:0/1496501928 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f917406b3a0 0x7f9174101990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:13.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.826+0000 7f917bf2b700 1 -- 172.21.15.52:0/1496501928 >> 172.21.15.52:0/1496501928 conn(0x7f9174078330 msgr2=0x7f91740ff3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:53:13.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.826+0000 7f917bf2b700 1 -- 172.21.15.52:0/1496501928 shutdown_connections 2024-06-10T02:53:13.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:13.826+0000 7f917bf2b700 1 -- 172.21.15.52:0/1496501928 wait complete. 2024-06-10T02:53:14.474 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph tell mds.c status 2024-06-10T02:53:14.726 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:53:15.099 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:53:14 smithi052 ceph-mon[32362]: pgmap v405: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-10T02:53:15.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:53:14 smithi120 ceph-mon[36634]: pgmap v405: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-10T02:53:15.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:53:14 smithi188 ceph-mon[36599]: pgmap v405: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-10T02:53:16.392 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.389+0000 7fb805bbe700 1 -- 172.21.15.52:0/827607817 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb800107400 msgr2=0x7fb8001077e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:53:16.392 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.389+0000 7fb805bbe700 1 --2- 172.21.15.52:0/827607817 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb800107400 0x7fb8001077e0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fb7f0009990 tx=0x7fb7f0009ca0 comp rx=0 tx=0).stop 2024-06-10T02:53:16.392 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.389+0000 7fb805bbe700 1 -- 172.21.15.52:0/827607817 shutdown_connections 2024-06-10T02:53:16.392 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.389+0000 7fb805bbe700 1 --2- 172.21.15.52:0/827607817 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb800101d50 0x7fb80010e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:16.392 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.389+0000 7fb805bbe700 1 --2- 172.21.15.52:0/827607817 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb800107400 0x7fb8001077e0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:16.392 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.389+0000 7fb805bbe700 1 --2- 172.21.15.52:0/827607817 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb800101390 0x7fb800101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:16.392 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.389+0000 7fb805bbe700 1 -- 172.21.15.52:0/827607817 >> 172.21.15.52:0/827607817 conn(0x7fb800078340 msgr2=0x7fb800078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:53:16.393 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.390+0000 7fb805bbe700 1 -- 172.21.15.52:0/827607817 shutdown_connections 2024-06-10T02:53:16.393 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.390+0000 7fb805bbe700 1 -- 172.21.15.52:0/827607817 wait complete. 2024-06-10T02:53:16.393 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.390+0000 7fb805bbe700 1 Processor -- start 2024-06-10T02:53:16.393 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.390+0000 7fb805bbe700 1 -- start start 2024-06-10T02:53:16.393 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.390+0000 7fb805bbe700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb800101390 0x7fb800103070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:16.393 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.390+0000 7fb805bbe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb800101d50 0x7fb8001035b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:16.393 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.390+0000 7fb805bbe700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb800107400 0x7fb800103af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:16.393 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.390+0000 7fb805bbe700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb800111740 con 0x7fb800101d50 2024-06-10T02:53:16.393 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.390+0000 7fb805bbe700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fb8001115c0 con 0x7fb800101390 2024-06-10T02:53:16.393 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.390+0000 7fb805bbe700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fb8001118c0 con 0x7fb800107400 2024-06-10T02:53:16.393 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.391+0000 7fb7f7fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb800101d50 0x7fb8001035b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:16.393 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.391+0000 7fb7f7fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb800101d50 0x7fb8001035b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53156/0 (socket says 172.21.15.52:53156) 2024-06-10T02:53:16.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.391+0000 7fb7ff7fe700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb800101390 0x7fb800103070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:16.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.391+0000 7fb7f7fff700 1 -- 172.21.15.52:0/842981638 learned_addr learned my addr 172.21.15.52:0/842981638 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:53:16.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.391+0000 7fb7fffff700 1 --2- 172.21.15.52:0/842981638 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb800107400 0x7fb800103af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:16.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.391+0000 7fb7f7fff700 1 -- 172.21.15.52:0/842981638 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb800107400 msgr2=0x7fb800103af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:53:16.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.391+0000 7fb7f7fff700 1 --2- 172.21.15.52:0/842981638 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb800107400 0x7fb800103af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:16.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.391+0000 7fb7f7fff700 1 -- 172.21.15.52:0/842981638 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb800101390 msgr2=0x7fb800103070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:53:16.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.391+0000 7fb7f7fff700 1 --2- 172.21.15.52:0/842981638 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb800101390 0x7fb800103070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:16.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.391+0000 7fb7f7fff700 1 -- 172.21.15.52:0/842981638 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8001a7200 con 0x7fb800101d50 2024-06-10T02:53:16.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.391+0000 7fb7ff7fe700 1 --2- 172.21.15.52:0/842981638 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb800101390 0x7fb800103070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T02:53:16.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.391+0000 7fb7f7fff700 1 --2- 172.21.15.52:0/842981638 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb800101d50 0x7fb8001035b0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fb7e800d950 tx=0x7fb7e800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:53:16.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.391+0000 7fb7fd7fa700 1 -- 172.21.15.52:0/842981638 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7e8016070 con 0x7fb800101d50 2024-06-10T02:53:16.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.391+0000 7fb7fd7fa700 1 -- 172.21.15.52:0/842981638 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb7e800fd70 con 0x7fb800101d50 2024-06-10T02:53:16.395 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.391+0000 7fb805bbe700 1 -- 172.21.15.52:0/842981638 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8001a74f0 con 0x7fb800101d50 2024-06-10T02:53:16.395 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.391+0000 7fb7fd7fa700 1 -- 172.21.15.52:0/842981638 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7e800e610 con 0x7fb800101d50 2024-06-10T02:53:16.395 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.392+0000 7fb805bbe700 1 -- 172.21.15.52:0/842981638 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8001a7a80 con 0x7fb800101d50 2024-06-10T02:53:16.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.394+0000 7fb7fd7fa700 1 -- 172.21.15.52:0/842981638 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fb7e8019070 con 0x7fb800101d50 2024-06-10T02:53:16.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.395+0000 7fb7fd7fa700 1 --2- 172.21.15.52:0/842981638 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fb7ec0a2ca0 0x7fb7ec0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:16.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.395+0000 7fb7ff7fe700 1 --2- 172.21.15.52:0/842981638 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fb7ec0a2ca0 0x7fb7ec0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:16.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.395+0000 7fb7fd7fa700 1 -- 172.21.15.52:0/842981638 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7fb7e80cee80 con 0x7fb800101d50 2024-06-10T02:53:16.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.396+0000 7fb7ff7fe700 1 --2- 172.21.15.52:0/842981638 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fb7ec0a2ca0 0x7fb7ec0a5160 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fb7f000b5c0 tx=0x7fb7f0005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:53:16.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.396+0000 7fb810441b80 1 Processor -- start 2024-06-10T02:53:16.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.396+0000 7fb810441b80 1 -- start start 2024-06-10T02:53:16.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.396+0000 7fb810441b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e4a3297f10 0x55e4a32982f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:16.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.396+0000 7fb810441b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e4a3298830 0x55e4a3292210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:16.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.396+0000 7fb810441b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e4a3292750 0x55e4a3292b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:16.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.396+0000 7fb810441b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55e4a3184f10 con 0x55e4a3292750 2024-06-10T02:53:16.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.396+0000 7fb810441b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55e4a31e3800 con 0x55e4a3297f10 2024-06-10T02:53:16.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.396+0000 7fb810441b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55e4a314b410 con 0x55e4a3298830 2024-06-10T02:53:16.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.396+0000 7fb7fffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e4a3292750 0x55e4a3292b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:16.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.396+0000 7fb7fffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e4a3292750 0x55e4a3292b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53170/0 (socket says 172.21.15.52:53170) 2024-06-10T02:53:16.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.396+0000 7fb7fffff700 1 -- 172.21.15.52:0/3850341891 learned_addr learned my addr 172.21.15.52:0/3850341891 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:53:16.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.396+0000 7fb7f7fff700 1 --2- 172.21.15.52:0/3850341891 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e4a3298830 0x55e4a3292210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:16.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.396+0000 7fb7ff7fe700 1 --2- 172.21.15.52:0/3850341891 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e4a3297f10 0x55e4a32982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:16.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.397+0000 7fb7fffff700 1 -- 172.21.15.52:0/3850341891 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e4a3298830 msgr2=0x55e4a3292210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:53:16.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.397+0000 7fb7fffff700 1 --2- 172.21.15.52:0/3850341891 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e4a3298830 0x55e4a3292210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:16.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.397+0000 7fb7fffff700 1 -- 172.21.15.52:0/3850341891 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e4a3297f10 msgr2=0x55e4a32982f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:53:16.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.397+0000 7fb7fffff700 1 --2- 172.21.15.52:0/3850341891 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e4a3297f10 0x55e4a32982f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:16.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.397+0000 7fb7fffff700 1 -- 172.21.15.52:0/3850341891 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e4a3132880 con 0x55e4a3292750 2024-06-10T02:53:16.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.397+0000 7fb7ff7fe700 1 --2- 172.21.15.52:0/3850341891 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e4a3297f10 0x55e4a32982f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T02:53:16.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.397+0000 7fb7fffff700 1 --2- 172.21.15.52:0/3850341891 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e4a3292750 0x55e4a3292b70 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fb7f800a430 tx=0x7fb7f8009650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73dd4b6ebb935a2e server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:53:16.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.397+0000 7fb7f5ffb700 1 -- 172.21.15.52:0/3850341891 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7f8014620 con 0x55e4a3292750 2024-06-10T02:53:16.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.397+0000 7fb7f5ffb700 1 -- 172.21.15.52:0/3850341891 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb7f8019070 con 0x55e4a3292750 2024-06-10T02:53:16.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.397+0000 7fb7f5ffb700 1 -- 172.21.15.52:0/3850341891 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7f8013b30 con 0x55e4a3292750 2024-06-10T02:53:16.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.398+0000 7fb810441b80 1 -- 172.21.15.52:0/3850341891 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e4a3292750 msgr2=0x55e4a3292b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:53:16.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.398+0000 7fb810441b80 1 --2- 172.21.15.52:0/3850341891 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e4a3292750 0x55e4a3292b70 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fb7f800a430 tx=0x7fb7f8009650 comp rx=0 tx=0).stop 2024-06-10T02:53:16.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.398+0000 7fb810441b80 1 -- 172.21.15.52:0/3850341891 shutdown_connections 2024-06-10T02:53:16.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.398+0000 7fb810441b80 1 --2- 172.21.15.52:0/3850341891 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e4a3292750 0x55e4a3292b70 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:16.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.398+0000 7fb810441b80 1 --2- 172.21.15.52:0/3850341891 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e4a3297f10 0x55e4a32982f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:16.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.398+0000 7fb810441b80 1 --2- 172.21.15.52:0/3850341891 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e4a3298830 0x55e4a3292210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:16.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.398+0000 7fb810441b80 1 -- 172.21.15.52:0/3850341891 >> 172.21.15.52:0/3850341891 conn(0x55e4a31bf030 msgr2=0x55e4a328d1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:53:16.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.398+0000 7fb810441b80 1 -- 172.21.15.52:0/3850341891 shutdown_connections 2024-06-10T02:53:16.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.398+0000 7fb810441b80 1 -- 172.21.15.52:0/3850341891 wait complete. 2024-06-10T02:53:16.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.398+0000 7fb810441b80 1 -- start start 2024-06-10T02:53:16.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.399+0000 7fb810441b80 1 Processor -- start 2024-06-10T02:53:16.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.399+0000 7fb810441b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e4a3292750 0x55e4a329e570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:16.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.399+0000 7fb810441b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e4a3297f10 0x55e4a329b870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:16.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.399+0000 7fb810441b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e4a3298830 0x55e4a329bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:16.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.399+0000 7fb810441b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55e4a3183f70 con 0x55e4a3298830 2024-06-10T02:53:16.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.399+0000 7fb810441b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55e4a31df7c0 con 0x55e4a3292750 2024-06-10T02:53:16.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.399+0000 7fb810441b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55e4a329f210 con 0x55e4a3297f10 2024-06-10T02:53:16.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.399+0000 7fb810441b80 10 client.0 _ll_register_callbacks cb 0x55e4a3179560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T02:53:16.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.399+0000 7fb7fffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e4a3298830 0x55e4a329bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:16.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.399+0000 7fb7fffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e4a3298830 0x55e4a329bdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53172/0 (socket says 172.21.15.52:53172) 2024-06-10T02:53:16.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.399+0000 7fb7fffff700 1 -- 172.21.15.52:0/792226102 learned_addr learned my addr 172.21.15.52:0/792226102 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:53:16.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.399+0000 7fb7f7fff700 1 --2- 172.21.15.52:0/792226102 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e4a3297f10 0x55e4a329b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:16.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.399+0000 7fb7ff7fe700 1 --2- 172.21.15.52:0/792226102 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e4a3292750 0x55e4a329e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:16.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.400+0000 7fb7fffff700 1 -- 172.21.15.52:0/792226102 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e4a3297f10 msgr2=0x55e4a329b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:53:16.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.400+0000 7fb7fffff700 1 --2- 172.21.15.52:0/792226102 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e4a3297f10 0x55e4a329b870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:16.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.400+0000 7fb7fffff700 1 -- 172.21.15.52:0/792226102 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e4a3292750 msgr2=0x55e4a329e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:53:16.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.400+0000 7fb7fffff700 1 --2- 172.21.15.52:0/792226102 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e4a3292750 0x55e4a329e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:16.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.400+0000 7fb7fffff700 1 -- 172.21.15.52:0/792226102 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e4a31a9700 con 0x55e4a3298830 2024-06-10T02:53:16.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.400+0000 7fb7fffff700 1 --2- 172.21.15.52:0/792226102 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e4a3298830 0x55e4a329bdb0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fb7f8013710 tx=0x7fb7f80139a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf293490d7d23630 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:53:16.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.400+0000 7fb7deffd700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-10T02:53:16.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.400+0000 7fb7deffd700 1 -- 172.21.15.52:0/792226102 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7f8014420 con 0x55e4a3298830 2024-06-10T02:53:16.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.400+0000 7fb810441b80 1 -- 172.21.15.52:0/792226102 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb7f8019070 con 0x55e4a3298830 2024-06-10T02:53:16.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.400+0000 7fb7deffd700 1 -- 172.21.15.52:0/792226102 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7fb7f80145c0 con 0x55e4a3298830 2024-06-10T02:53:16.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.400+0000 7fb7deffd700 1 -- 172.21.15.52:0/792226102 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb7f8031050 con 0x55e4a3298830 2024-06-10T02:53:16.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.400+0000 7fb810441b80 10 client.15093 fetch_fsmap learned FSMap version 29 2024-06-10T02:53:16.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.400+0000 7fb810441b80 1 -- 172.21.15.52:0/792226102 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e4a32a99a0 con 0x55e4a3298830 2024-06-10T02:53:16.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.400+0000 7fb7deffd700 1 -- 172.21.15.52:0/792226102 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb7f8019070 con 0x55e4a3298830 2024-06-10T02:53:16.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.401+0000 7fb7deffd700 1 -- 172.21.15.52:0/792226102 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fb7f801e7d0 con 0x55e4a3298830 2024-06-10T02:53:16.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.401+0000 7fb810441b80 10 client.15093 fetch_fsmap finished waiting for FSMap version 29 2024-06-10T02:53:16.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.401+0000 7fb810441b80 10 client.15093 resolve_mds: resolved name 'c' to daemon mds.c 2024-06-10T02:53:16.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.401+0000 7fb810441b80 20 client.15093 populate_metadata read hostname 'smithi052' 2024-06-10T02:53:16.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.401+0000 7fb810441b80 1 --2- 172.21.15.52:0/792226102 >> [v2:172.21.15.188:6836/819317726,v1:172.21.15.188:6837/819317726] conn(0x55e4a32abba0 0x55e4a32aae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:16.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.401+0000 7fb810441b80 4 client.15093 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T02:53:16.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.401+0000 7fb810441b80 1 -- 172.21.15.52:0/792226102 --> [v2:172.21.15.188:6836/819317726,v1:172.21.15.188:6837/819317726] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e4a31f0d00 con 0x55e4a32abba0 2024-06-10T02:53:16.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.401+0000 7fb7ff7fe700 1 --2- 172.21.15.52:0/792226102 >> [v2:172.21.15.188:6836/819317726,v1:172.21.15.188:6837/819317726] conn(0x55e4a32abba0 0x55e4a32aae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:16.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.402+0000 7fb7ff7fe700 1 --2- 172.21.15.52:0/792226102 >> [v2:172.21.15.188:6836/819317726,v1:172.21.15.188:6837/819317726] conn(0x55e4a32abba0 0x55e4a32aae00 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e2d0ed1c79bdda88 server_cookie=4ad192cedeb5b02e in_seq=0 out_seq=0 2024-06-10T02:53:16.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.402+0000 7fb7deffd700 10 client.15093 ms_handle_connect on v2:172.21.15.188:6836/819317726 2024-06-10T02:53:16.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.402+0000 7fb7deffd700 1 -- 172.21.15.52:0/792226102 <== mds.1 v2:172.21.15.188:6836/819317726 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55e4a31f0d00 con 0x55e4a32abba0 2024-06-10T02:53:16.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.402+0000 7fb7deffd700 10 client.15093 handle_command_reply: tid=0 2024-06-10T02:53:16.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.402+0000 7fb810441b80 1 client.15093 shutdown 2024-06-10T02:53:16.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.403+0000 7fb810441b80 1 -- 172.21.15.52:0/792226102 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e4a3298830 msgr2=0x55e4a329bdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:53:16.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.403+0000 7fb810441b80 1 --2- 172.21.15.52:0/792226102 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e4a3298830 0x55e4a329bdb0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fb7f8013710 tx=0x7fb7f80139a0 comp rx=0 tx=0).stop 2024-06-10T02:53:16.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.403+0000 7fb810441b80 1 -- 172.21.15.52:0/792226102 shutdown_connections 2024-06-10T02:53:16.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.403+0000 7fb810441b80 1 --2- 172.21.15.52:0/792226102 >> [v2:172.21.15.188:6836/819317726,v1:172.21.15.188:6837/819317726] conn(0x55e4a32abba0 0x55e4a32aae00 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:16.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.403+0000 7fb810441b80 1 --2- 172.21.15.52:0/792226102 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e4a3298830 0x55e4a329bdb0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:16.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.403+0000 7fb810441b80 1 --2- 172.21.15.52:0/792226102 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e4a3292750 0x55e4a329e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:16.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.403+0000 7fb810441b80 1 --2- 172.21.15.52:0/792226102 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e4a3297f10 0x55e4a329b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:16.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.403+0000 7fb810441b80 1 -- 172.21.15.52:0/792226102 >> 172.21.15.52:0/792226102 conn(0x55e4a31bf030 msgr2=0x55e4a328c6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:53:16.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.403+0000 7fb7deffd700 0 client.15093 ms_handle_reset on v2:172.21.15.188:6836/819317726 2024-06-10T02:53:16.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.403+0000 7fb810441b80 1 -- 172.21.15.52:0/792226102 shutdown_connections 2024-06-10T02:53:16.407 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.403+0000 7fb810441b80 1 -- 172.21.15.52:0/792226102 wait complete. 2024-06-10T02:53:16.407 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.403+0000 7fb810441b80 20 client.15093 trim_cache size 0 max 16384 2024-06-10T02:53:16.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.425+0000 7fb810441b80 1 Processor -- start 2024-06-10T02:53:16.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.425+0000 7fb810441b80 1 -- start start 2024-06-10T02:53:16.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.425+0000 7fb810441b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e4a3292750 0x55e4a3292210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:16.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.425+0000 7fb810441b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e4a3297f10 0x55e4a329c500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:16.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.425+0000 7fb810441b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e4a3298830 0x55e4a328ce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:16.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.425+0000 7fb810441b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55e4a32ab350 con 0x55e4a3298830 2024-06-10T02:53:16.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.425+0000 7fb810441b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55e4a32aba20 con 0x55e4a3297f10 2024-06-10T02:53:16.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.425+0000 7fb810441b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55e4a32ac0f0 con 0x55e4a3292750 2024-06-10T02:53:16.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.425+0000 7fb7fffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e4a3298830 0x55e4a328ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:16.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.425+0000 7fb7fffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e4a3298830 0x55e4a328ce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53184/0 (socket says 172.21.15.52:53184) 2024-06-10T02:53:16.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.425+0000 7fb7fffff700 1 -- 172.21.15.52:0/3214722968 learned_addr learned my addr 172.21.15.52:0/3214722968 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:53:16.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.426+0000 7fb7f7fff700 1 --2- 172.21.15.52:0/3214722968 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e4a3297f10 0x55e4a329c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:16.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.426+0000 7fb7fffff700 1 -- 172.21.15.52:0/3214722968 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e4a3292750 msgr2=0x55e4a3292210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:53:16.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.426+0000 7fb7fffff700 1 --2- 172.21.15.52:0/3214722968 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e4a3292750 0x55e4a3292210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:16.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.426+0000 7fb7fffff700 1 -- 172.21.15.52:0/3214722968 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e4a3297f10 msgr2=0x55e4a329c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:53:16.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.426+0000 7fb7fffff700 1 --2- 172.21.15.52:0/3214722968 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e4a3297f10 0x55e4a329c500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:16.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.426+0000 7fb7fffff700 1 -- 172.21.15.52:0/3214722968 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e4a32ad380 con 0x55e4a3298830 2024-06-10T02:53:16.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.426+0000 7fb7f7fff700 1 --2- 172.21.15.52:0/3214722968 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e4a3297f10 0x55e4a329c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T02:53:16.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.426+0000 7fb7fffff700 1 --2- 172.21.15.52:0/3214722968 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e4a3298830 0x55e4a328ce30 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fb7f8009bb0 tx=0x7fb7f8009870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=605136916667a3f4 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:53:16.428 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.426+0000 7fb7f57fa700 1 -- 172.21.15.52:0/3214722968 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7f8013e10 con 0x55e4a3298830 2024-06-10T02:53:16.428 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.426+0000 7fb7f57fa700 1 -- 172.21.15.52:0/3214722968 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb7f8019070 con 0x55e4a3298830 2024-06-10T02:53:16.428 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.426+0000 7fb7f57fa700 1 -- 172.21.15.52:0/3214722968 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7f801c740 con 0x55e4a3298830 2024-06-10T02:53:16.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.427+0000 7fb810441b80 1 -- 172.21.15.52:0/3214722968 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e4a3298830 msgr2=0x55e4a328ce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:53:16.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.427+0000 7fb810441b80 1 --2- 172.21.15.52:0/3214722968 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e4a3298830 0x55e4a328ce30 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fb7f8009bb0 tx=0x7fb7f8009870 comp rx=0 tx=0).stop 2024-06-10T02:53:16.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.427+0000 7fb810441b80 1 -- 172.21.15.52:0/3214722968 shutdown_connections 2024-06-10T02:53:16.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.427+0000 7fb810441b80 1 --2- 172.21.15.52:0/3214722968 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e4a3298830 0x55e4a328ce30 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:16.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.427+0000 7fb810441b80 1 --2- 172.21.15.52:0/3214722968 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e4a3292750 0x55e4a3292210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:16.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.427+0000 7fb810441b80 1 --2- 172.21.15.52:0/3214722968 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e4a3297f10 0x55e4a329c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:16.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.427+0000 7fb810441b80 1 -- 172.21.15.52:0/3214722968 >> 172.21.15.52:0/3214722968 conn(0x55e4a31bf030 msgr2=0x55e4a3292b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:53:16.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.427+0000 7fb810441b80 1 -- 172.21.15.52:0/3214722968 shutdown_connections 2024-06-10T02:53:16.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.427+0000 7fb810441b80 1 -- 172.21.15.52:0/3214722968 wait complete. 2024-06-10T02:53:16.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.427+0000 7fb810441b80 1 -- start start 2024-06-10T02:53:16.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.427+0000 7fb810441b80 1 Processor -- start 2024-06-10T02:53:16.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.428+0000 7fb810441b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e4a3292750 0x55e4a328be20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:16.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.428+0000 7fb810441b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e4a3297f10 0x55e4a328c360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:16.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.428+0000 7fb810441b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e4a3298830 0x55e4a32b5480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:16.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.428+0000 7fb810441b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55e4a32ace90 con 0x55e4a3298830 2024-06-10T02:53:16.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.428+0000 7fb810441b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55e4a32ac7c0 con 0x55e4a3297f10 2024-06-10T02:53:16.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.428+0000 7fb810441b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55e4a32ad560 con 0x55e4a3292750 2024-06-10T02:53:16.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.428+0000 7fb810441b80 10 client.0 _ll_register_callbacks cb 0x55e4a3179560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T02:53:16.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.428+0000 7fb7fffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e4a3298830 0x55e4a32b5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:16.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.428+0000 7fb7f7fff700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e4a3297f10 0x55e4a328c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:16.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.428+0000 7fb7fffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e4a3298830 0x55e4a32b5480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53196/0 (socket says 172.21.15.52:53196) 2024-06-10T02:53:16.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.428+0000 7fb7fffff700 1 -- 172.21.15.52:0/2515486182 learned_addr learned my addr 172.21.15.52:0/2515486182 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:53:16.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.428+0000 7fb7ff7fe700 1 --2- 172.21.15.52:0/2515486182 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e4a3292750 0x55e4a328be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:16.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.428+0000 7fb7fffff700 1 -- 172.21.15.52:0/2515486182 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e4a3292750 msgr2=0x55e4a328be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:53:16.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.428+0000 7fb7fffff700 1 --2- 172.21.15.52:0/2515486182 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e4a3292750 0x55e4a328be20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:16.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.428+0000 7fb7fffff700 1 -- 172.21.15.52:0/2515486182 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e4a3297f10 msgr2=0x55e4a328c360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:53:16.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.428+0000 7fb7fffff700 1 --2- 172.21.15.52:0/2515486182 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e4a3297f10 0x55e4a328c360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:16.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.428+0000 7fb7fffff700 1 -- 172.21.15.52:0/2515486182 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e4a328cc30 con 0x55e4a3298830 2024-06-10T02:53:16.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.428+0000 7fb7fffff700 1 --2- 172.21.15.52:0/2515486182 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e4a3298830 0x55e4a32b5480 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fb7f801ef60 tx=0x7fb7f8013980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bd3bf50c61dd9bd server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:53:16.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.428+0000 7fb7deffd700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-10T02:53:16.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.429+0000 7fb7deffd700 1 -- 172.21.15.52:0/2515486182 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7f803b070 con 0x55e4a3298830 2024-06-10T02:53:16.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.429+0000 7fb7deffd700 1 -- 172.21.15.52:0/2515486182 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7fb7f801e600 con 0x55e4a3298830 2024-06-10T02:53:16.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.429+0000 7fb810441b80 1 -- 172.21.15.52:0/2515486182 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb7f8019070 con 0x55e4a3298830 2024-06-10T02:53:16.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.429+0000 7fb7deffd700 1 -- 172.21.15.52:0/2515486182 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb7f8048050 con 0x55e4a3298830 2024-06-10T02:53:16.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.429+0000 7fb810441b80 10 client.15099 fetch_fsmap learned FSMap version 29 2024-06-10T02:53:16.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.429+0000 7fb810441b80 1 -- 172.21.15.52:0/2515486182 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e4a32b6a30 con 0x55e4a3298830 2024-06-10T02:53:16.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.429+0000 7fb7deffd700 1 -- 172.21.15.52:0/2515486182 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb7f8019070 con 0x55e4a3298830 2024-06-10T02:53:16.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.429+0000 7fb7deffd700 1 -- 172.21.15.52:0/2515486182 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fb7f8014420 con 0x55e4a3298830 2024-06-10T02:53:16.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.429+0000 7fb810441b80 10 client.15099 fetch_fsmap finished waiting for FSMap version 29 2024-06-10T02:53:16.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.429+0000 7fb810441b80 10 client.15099 resolve_mds: resolved name 'c' to daemon mds.c 2024-06-10T02:53:16.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.429+0000 7fb810441b80 20 client.15099 populate_metadata read hostname 'smithi052' 2024-06-10T02:53:16.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.429+0000 7fb810441b80 1 --2- 172.21.15.52:0/2515486182 >> [v2:172.21.15.188:6836/819317726,v1:172.21.15.188:6837/819317726] conn(0x55e4a331b4b0 0x55e4a331a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:16.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.429+0000 7fb810441b80 4 client.15099 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "status"}] 2024-06-10T02:53:16.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.429+0000 7fb810441b80 1 -- 172.21.15.52:0/2515486182 --> [v2:172.21.15.188:6836/819317726,v1:172.21.15.188:6837/819317726] -- command(tid 0: {"prefix": "status"}) -- 0x55e4a31eb280 con 0x55e4a331b4b0 2024-06-10T02:53:16.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.429+0000 7fb7ff7fe700 1 --2- 172.21.15.52:0/2515486182 >> [v2:172.21.15.188:6836/819317726,v1:172.21.15.188:6837/819317726] conn(0x55e4a331b4b0 0x55e4a331a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:16.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.430+0000 7fb7ff7fe700 1 --2- 172.21.15.52:0/2515486182 >> [v2:172.21.15.188:6836/819317726,v1:172.21.15.188:6837/819317726] conn(0x55e4a331b4b0 0x55e4a331a1c0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d59e0fe2a1d4ea94 server_cookie=943d5a9a3f1ba3c3 in_seq=0 out_seq=0 2024-06-10T02:53:16.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.430+0000 7fb7deffd700 10 client.15099 ms_handle_connect on v2:172.21.15.188:6836/819317726 2024-06-10T02:53:16.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.430+0000 7fb7deffd700 1 -- 172.21.15.52:0/2515486182 <== mds.1 v2:172.21.15.188:6836/819317726 1 ==== command_reply(tid 0: 0 ) ==== 8+0+333 (crc 0 0 0) 0x55e4a31eb280 con 0x55e4a331b4b0 2024-06-10T02:53:16.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.430+0000 7fb7deffd700 10 client.15099 handle_command_reply: tid=0 2024-06-10T02:53:16.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.430+0000 7fb810441b80 1 client.15099 shutdown 2024-06-10T02:53:16.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.430+0000 7fb810441b80 1 -- 172.21.15.52:0/2515486182 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e4a3298830 msgr2=0x55e4a32b5480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:53:16.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.430+0000 7fb810441b80 1 --2- 172.21.15.52:0/2515486182 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e4a3298830 0x55e4a32b5480 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fb7f801ef60 tx=0x7fb7f8013980 comp rx=0 tx=0).stop 2024-06-10T02:53:16.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.430+0000 7fb810441b80 1 -- 172.21.15.52:0/2515486182 shutdown_connections 2024-06-10T02:53:16.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.430+0000 7fb810441b80 1 --2- 172.21.15.52:0/2515486182 >> [v2:172.21.15.188:6836/819317726,v1:172.21.15.188:6837/819317726] conn(0x55e4a331b4b0 0x55e4a331a1c0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:16.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.430+0000 7fb810441b80 1 --2- 172.21.15.52:0/2515486182 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e4a3298830 0x55e4a32b5480 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:16.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.430+0000 7fb810441b80 1 --2- 172.21.15.52:0/2515486182 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e4a3292750 0x55e4a328be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:16.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.430+0000 7fb810441b80 1 --2- 172.21.15.52:0/2515486182 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e4a3297f10 0x55e4a328c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:16.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.430+0000 7fb810441b80 1 -- 172.21.15.52:0/2515486182 >> 172.21.15.52:0/2515486182 conn(0x55e4a31bf030 msgr2=0x55e4a329a0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:53:16.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.430+0000 7fb7deffd700 0 client.15099 ms_handle_reset on v2:172.21.15.188:6836/819317726 2024-06-10T02:53:16.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.431+0000 7fb810441b80 1 -- 172.21.15.52:0/2515486182 shutdown_connections 2024-06-10T02:53:16.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.431+0000 7fb810441b80 1 -- 172.21.15.52:0/2515486182 wait complete. 2024-06-10T02:53:16.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.431+0000 7fb810441b80 20 client.15099 trim_cache size 0 max 16384 2024-06-10T02:53:16.436 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-10T02:53:16.436 INFO:teuthology.orchestra.run.smithi052.stdout: "cluster_fsid": "d8676e9e-26d2-11ef-bc9b-c7b262605968", 2024-06-10T02:53:16.436 INFO:teuthology.orchestra.run.smithi052.stdout: "whoami": 1, 2024-06-10T02:53:16.436 INFO:teuthology.orchestra.run.smithi052.stdout: "id": 24469, 2024-06-10T02:53:16.436 INFO:teuthology.orchestra.run.smithi052.stdout: "want_state": "up:active", 2024-06-10T02:53:16.436 INFO:teuthology.orchestra.run.smithi052.stdout: "state": "up:active", 2024-06-10T02:53:16.436 INFO:teuthology.orchestra.run.smithi052.stdout: "fs_name": "cephfs", 2024-06-10T02:53:16.436 INFO:teuthology.orchestra.run.smithi052.stdout: "rank_uptime": 130.83614899299999, 2024-06-10T02:53:16.437 INFO:teuthology.orchestra.run.smithi052.stdout: "mdsmap_epoch": 29, 2024-06-10T02:53:16.437 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_epoch": 100, 2024-06-10T02:53:16.437 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_epoch_barrier": 98, 2024-06-10T02:53:16.437 INFO:teuthology.orchestra.run.smithi052.stdout: "uptime": 226.01633875799999 2024-06-10T02:53:16.437 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-10T02:53:16.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.431+0000 7fb805bbe700 1 -- 172.21.15.52:0/842981638 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fb7ec0a2ca0 msgr2=0x7fb7ec0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:53:16.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.431+0000 7fb805bbe700 1 --2- 172.21.15.52:0/842981638 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fb7ec0a2ca0 0x7fb7ec0a5160 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fb7f000b5c0 tx=0x7fb7f0005fd0 comp rx=0 tx=0).stop 2024-06-10T02:53:16.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.431+0000 7fb805bbe700 1 -- 172.21.15.52:0/842981638 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb800101d50 msgr2=0x7fb8001035b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:53:16.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.431+0000 7fb805bbe700 1 --2- 172.21.15.52:0/842981638 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb800101d50 0x7fb8001035b0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fb7e800d950 tx=0x7fb7e800dc60 comp rx=0 tx=0).stop 2024-06-10T02:53:16.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.431+0000 7fb805bbe700 1 -- 172.21.15.52:0/842981638 shutdown_connections 2024-06-10T02:53:16.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.431+0000 7fb805bbe700 1 --2- 172.21.15.52:0/842981638 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb800107400 0x7fb800103af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:16.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.431+0000 7fb805bbe700 1 --2- 172.21.15.52:0/842981638 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb800101390 0x7fb800103070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:16.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.431+0000 7fb805bbe700 1 --2- 172.21.15.52:0/842981638 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fb7ec0a2ca0 0x7fb7ec0a5160 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:16.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.431+0000 7fb805bbe700 1 --2- 172.21.15.52:0/842981638 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb800101d50 0x7fb8001035b0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:16.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.431+0000 7fb805bbe700 1 -- 172.21.15.52:0/842981638 >> 172.21.15.52:0/842981638 conn(0x7fb800078340 msgr2=0x7fb800100660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:53:16.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.431+0000 7fb805bbe700 1 -- 172.21.15.52:0/842981638 shutdown_connections 2024-06-10T02:53:16.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:16.431+0000 7fb805bbe700 1 -- 172.21.15.52:0/842981638 wait complete. 2024-06-10T02:53:17.123 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:53:16 smithi052 ceph-mon[32362]: pgmap v406: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-10T02:53:17.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:53:16 smithi120 ceph-mon[36634]: pgmap v406: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-10T02:53:17.153 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph tell mds.i status 2024-06-10T02:53:17.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:53:16 smithi188 ceph-mon[36599]: pgmap v406: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-10T02:53:17.406 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:53:19.119 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:53:18 smithi052 ceph-mon[32362]: pgmap v407: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-10T02:53:19.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.135+0000 7fa714c8f700 1 -- 172.21.15.52:0/1543991957 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa71006b5c0 msgr2=0x7fa71010f370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:53:19.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.135+0000 7fa714c8f700 1 --2- 172.21.15.52:0/1543991957 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa71006b5c0 0x7fa71010f370 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fa70800b0a0 tx=0x7fa70800b3b0 comp rx=0 tx=0).stop 2024-06-10T02:53:19.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.135+0000 7fa714c8f700 1 -- 172.21.15.52:0/1543991957 shutdown_connections 2024-06-10T02:53:19.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.135+0000 7fa714c8f700 1 --2- 172.21.15.52:0/1543991957 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa71006b5c0 0x7fa71010f370 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:19.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.135+0000 7fa714c8f700 1 --2- 172.21.15.52:0/1543991957 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa710108530 0x7fa710108910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:19.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.135+0000 7fa714c8f700 1 --2- 172.21.15.52:0/1543991957 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa71006ac00 0x7fa71006b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:19.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.135+0000 7fa714c8f700 1 -- 172.21.15.52:0/1543991957 >> 172.21.15.52:0/1543991957 conn(0x7fa710077400 msgr2=0x7fa710077810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:53:19.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.135+0000 7fa714c8f700 1 -- 172.21.15.52:0/1543991957 shutdown_connections 2024-06-10T02:53:19.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.135+0000 7fa714c8f700 1 -- 172.21.15.52:0/1543991957 wait complete. 2024-06-10T02:53:19.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:53:18 smithi120 ceph-mon[36634]: pgmap v407: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-10T02:53:19.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.136+0000 7fa714c8f700 1 Processor -- start 2024-06-10T02:53:19.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.136+0000 7fa714c8f700 1 -- start start 2024-06-10T02:53:19.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.137+0000 7fa714c8f700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa71006ac00 0x7fa71019e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:19.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.137+0000 7fa714c8f700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa71006b5c0 0x7fa71019ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:19.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.137+0000 7fa714c8f700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa710108530 0x7fa7101989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:19.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.137+0000 7fa714c8f700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fa710112780 con 0x7fa710108530 2024-06-10T02:53:19.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.137+0000 7fa714c8f700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa710112600 con 0x7fa71006ac00 2024-06-10T02:53:19.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.137+0000 7fa714c8f700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa710112900 con 0x7fa71006b5c0 2024-06-10T02:53:19.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.137+0000 7fa70ed9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa710108530 0x7fa7101989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:19.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.137+0000 7fa70ed9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa710108530 0x7fa7101989e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53206/0 (socket says 172.21.15.52:53206) 2024-06-10T02:53:19.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.137+0000 7fa70ed9d700 1 -- 172.21.15.52:0/3277039032 learned_addr learned my addr 172.21.15.52:0/3277039032 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:53:19.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.137+0000 7fa707fff700 1 --2- 172.21.15.52:0/3277039032 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa71006b5c0 0x7fa71019ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:19.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.137+0000 7fa70e59c700 1 --2- 172.21.15.52:0/3277039032 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa71006ac00 0x7fa71019e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:19.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.138+0000 7fa70ed9d700 1 -- 172.21.15.52:0/3277039032 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa71006b5c0 msgr2=0x7fa71019ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:53:19.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.138+0000 7fa70ed9d700 1 --2- 172.21.15.52:0/3277039032 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa71006b5c0 0x7fa71019ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:19.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.138+0000 7fa70ed9d700 1 -- 172.21.15.52:0/3277039032 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa71006ac00 msgr2=0x7fa71019e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:53:19.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.138+0000 7fa70ed9d700 1 --2- 172.21.15.52:0/3277039032 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa71006ac00 0x7fa71019e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:19.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.138+0000 7fa70ed9d700 1 -- 172.21.15.52:0/3277039032 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa710199140 con 0x7fa710108530 2024-06-10T02:53:19.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.138+0000 7fa70ed9d700 1 --2- 172.21.15.52:0/3277039032 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa710108530 0x7fa7101989e0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fa7080062a0 tx=0x7fa708004d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:53:19.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.138+0000 7fa7077fe700 1 -- 172.21.15.52:0/3277039032 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa708021070 con 0x7fa710108530 2024-06-10T02:53:19.141 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.138+0000 7fa714c8f700 1 -- 172.21.15.52:0/3277039032 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7101993d0 con 0x7fa710108530 2024-06-10T02:53:19.141 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.138+0000 7fa714c8f700 1 -- 172.21.15.52:0/3277039032 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7101013c0 con 0x7fa710108530 2024-06-10T02:53:19.141 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.138+0000 7fa7077fe700 1 -- 172.21.15.52:0/3277039032 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa708003c00 con 0x7fa710108530 2024-06-10T02:53:19.141 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.138+0000 7fa7077fe700 1 -- 172.21.15.52:0/3277039032 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa708004880 con 0x7fa710108530 2024-06-10T02:53:19.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.141+0000 7fa7077fe700 1 -- 172.21.15.52:0/3277039032 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fa708004020 con 0x7fa710108530 2024-06-10T02:53:19.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.141+0000 7fa7077fe700 1 --2- 172.21.15.52:0/3277039032 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fa6fc0a2ca0 0x7fa6fc0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:19.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.142+0000 7fa7077fe700 1 -- 172.21.15.52:0/3277039032 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7fa70800f020 con 0x7fa710108530 2024-06-10T02:53:19.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.142+0000 7fa70e59c700 1 --2- 172.21.15.52:0/3277039032 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fa6fc0a2ca0 0x7fa6fc0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:19.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.142+0000 7fa70e59c700 1 --2- 172.21.15.52:0/3277039032 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fa6fc0a2ca0 0x7fa6fc0a5160 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fa710199b20 tx=0x7fa70000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:53:19.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.143+0000 7fa71f512b80 1 Processor -- start 2024-06-10T02:53:19.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.144+0000 7fa71f512b80 1 -- start start 2024-06-10T02:53:19.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.144+0000 7fa71f512b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d5a53fa5f0 0x55d5a53fa9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:19.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.144+0000 7fa71f512b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d5a53faf10 0x55d5a53f4990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:19.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.144+0000 7fa71f512b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d5a53f4ed0 0x55d5a53f52f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:19.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.144+0000 7fa71f512b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55d5a53467c0 con 0x55d5a53f4ed0 2024-06-10T02:53:19.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.144+0000 7fa71f512b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55d5a52ee6d0 con 0x55d5a53faf10 2024-06-10T02:53:19.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.144+0000 7fa71f512b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55d5a5342780 con 0x55d5a53fa5f0 2024-06-10T02:53:19.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.144+0000 7fa70ed9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d5a53f4ed0 0x55d5a53f52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:19.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.144+0000 7fa70ed9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d5a53f4ed0 0x55d5a53f52f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53212/0 (socket says 172.21.15.52:53212) 2024-06-10T02:53:19.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.144+0000 7fa70ed9d700 1 -- 172.21.15.52:0/2401544385 learned_addr learned my addr 172.21.15.52:0/2401544385 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:53:19.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.144+0000 7fa707fff700 1 --2- 172.21.15.52:0/2401544385 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d5a53faf10 0x55d5a53f4990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:19.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.145+0000 7fa70e59c700 1 --2- 172.21.15.52:0/2401544385 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d5a53fa5f0 0x55d5a53fa9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:19.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.145+0000 7fa70ed9d700 1 -- 172.21.15.52:0/2401544385 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d5a53fa5f0 msgr2=0x55d5a53fa9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:53:19.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.145+0000 7fa70ed9d700 1 --2- 172.21.15.52:0/2401544385 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d5a53fa5f0 0x55d5a53fa9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:19.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.145+0000 7fa70ed9d700 1 -- 172.21.15.52:0/2401544385 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d5a53faf10 msgr2=0x55d5a53f4990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:53:19.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.145+0000 7fa70ed9d700 1 --2- 172.21.15.52:0/2401544385 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d5a53faf10 0x55d5a53f4990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:19.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.145+0000 7fa70ed9d700 1 -- 172.21.15.52:0/2401544385 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d5a52e4f00 con 0x55d5a53f4ed0 2024-06-10T02:53:19.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.145+0000 7fa70ed9d700 1 --2- 172.21.15.52:0/2401544385 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d5a53f4ed0 0x55d5a53f52f0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fa7080043f0 tx=0x7fa7080099e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63c3c5c104ca9b5e server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:53:19.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.145+0000 7fa704ff9700 1 -- 172.21.15.52:0/2401544385 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa70809d370 con 0x55d5a53f4ed0 2024-06-10T02:53:19.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.145+0000 7fa704ff9700 1 -- 172.21.15.52:0/2401544385 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa70809cc70 con 0x55d5a53f4ed0 2024-06-10T02:53:19.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.145+0000 7fa704ff9700 1 -- 172.21.15.52:0/2401544385 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa70809d130 con 0x55d5a53f4ed0 2024-06-10T02:53:19.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.146+0000 7fa71f512b80 1 -- 172.21.15.52:0/2401544385 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d5a53f4ed0 msgr2=0x55d5a53f52f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:53:19.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.146+0000 7fa71f512b80 1 --2- 172.21.15.52:0/2401544385 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d5a53f4ed0 0x55d5a53f52f0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fa7080043f0 tx=0x7fa7080099e0 comp rx=0 tx=0).stop 2024-06-10T02:53:19.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.146+0000 7fa71f512b80 1 -- 172.21.15.52:0/2401544385 shutdown_connections 2024-06-10T02:53:19.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.146+0000 7fa71f512b80 1 --2- 172.21.15.52:0/2401544385 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d5a53f4ed0 0x55d5a53f52f0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:19.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.146+0000 7fa71f512b80 1 --2- 172.21.15.52:0/2401544385 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d5a53fa5f0 0x55d5a53fa9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:19.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.146+0000 7fa71f512b80 1 --2- 172.21.15.52:0/2401544385 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d5a53faf10 0x55d5a53f4990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:19.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.146+0000 7fa71f512b80 1 -- 172.21.15.52:0/2401544385 >> 172.21.15.52:0/2401544385 conn(0x55d5a531fc70 msgr2=0x55d5a53ef800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:53:19.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.147+0000 7fa71f512b80 1 -- 172.21.15.52:0/2401544385 shutdown_connections 2024-06-10T02:53:19.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.147+0000 7fa71f512b80 1 -- 172.21.15.52:0/2401544385 wait complete. 2024-06-10T02:53:19.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.147+0000 7fa71f512b80 1 -- start start 2024-06-10T02:53:19.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.148+0000 7fa71f512b80 1 Processor -- start 2024-06-10T02:53:19.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.148+0000 7fa71f512b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d5a53f4ed0 0x55d5a5400ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:19.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.148+0000 7fa71f512b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d5a53fa5f0 0x55d5a53fdfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:19.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.148+0000 7fa71f512b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d5a53faf10 0x55d5a53fe4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:19.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.148+0000 7fa71f512b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55d5a54016a0 con 0x55d5a53f4ed0 2024-06-10T02:53:19.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.148+0000 7fa71f512b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55d5a52e6ba0 con 0x55d5a53fa5f0 2024-06-10T02:53:19.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.148+0000 7fa71f512b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55d5a5401820 con 0x55d5a53faf10 2024-06-10T02:53:19.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.148+0000 7fa71f512b80 10 client.0 _ll_register_callbacks cb 0x55d5a52e08d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T02:53:19.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.148+0000 7fa70e59c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d5a53f4ed0 0x55d5a5400ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:19.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.148+0000 7fa70e59c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d5a53f4ed0 0x55d5a5400ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53214/0 (socket says 172.21.15.52:53214) 2024-06-10T02:53:19.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.148+0000 7fa70e59c700 1 -- 172.21.15.52:0/558272714 learned_addr learned my addr 172.21.15.52:0/558272714 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:53:19.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.148+0000 7fa707fff700 1 --2- 172.21.15.52:0/558272714 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d5a53fa5f0 0x55d5a53fdfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:19.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.148+0000 7fa70ed9d700 1 --2- 172.21.15.52:0/558272714 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d5a53faf10 0x55d5a53fe4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:19.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.149+0000 7fa70e59c700 1 -- 172.21.15.52:0/558272714 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d5a53faf10 msgr2=0x55d5a53fe4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:53:19.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.149+0000 7fa70e59c700 1 --2- 172.21.15.52:0/558272714 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d5a53faf10 0x55d5a53fe4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:19.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.149+0000 7fa70e59c700 1 -- 172.21.15.52:0/558272714 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d5a53fa5f0 msgr2=0x55d5a53fdfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:53:19.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.149+0000 7fa70e59c700 1 --2- 172.21.15.52:0/558272714 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d5a53fa5f0 0x55d5a53fdfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:19.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.149+0000 7fa70e59c700 1 -- 172.21.15.52:0/558272714 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d5a5295880 con 0x55d5a53f4ed0 2024-06-10T02:53:19.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.149+0000 7fa70ed9d700 1 --2- 172.21.15.52:0/558272714 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d5a53faf10 0x55d5a53fe4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T02:53:19.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.149+0000 7fa707fff700 1 --2- 172.21.15.52:0/558272714 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d5a53fa5f0 0x55d5a53fdfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T02:53:19.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.149+0000 7fa70e59c700 1 --2- 172.21.15.52:0/558272714 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d5a53f4ed0 0x55d5a5400ca0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fa7000116c0 tx=0x7fa7000047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21e21329fb28ae32 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:53:19.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.149+0000 7fa6f5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-10T02:53:19.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.149+0000 7fa6f5ffb700 1 -- 172.21.15.52:0/558272714 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa700028670 con 0x55d5a53f4ed0 2024-06-10T02:53:19.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.149+0000 7fa6f5ffb700 1 -- 172.21.15.52:0/558272714 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7fa700028810 con 0x55d5a53f4ed0 2024-06-10T02:53:19.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.149+0000 7fa71f512b80 1 -- 172.21.15.52:0/558272714 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa70809cc70 con 0x55d5a53f4ed0 2024-06-10T02:53:19.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.149+0000 7fa6f5ffb700 1 -- 172.21.15.52:0/558272714 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa700028a50 con 0x55d5a53f4ed0 2024-06-10T02:53:19.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.149+0000 7fa71f512b80 10 client.15111 fetch_fsmap learned FSMap version 29 2024-06-10T02:53:19.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.149+0000 7fa71f512b80 1 -- 172.21.15.52:0/558272714 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d5a540c0c0 con 0x55d5a53f4ed0 2024-06-10T02:53:19.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.150+0000 7fa6f5ffb700 1 -- 172.21.15.52:0/558272714 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa70809cc70 con 0x55d5a53f4ed0 2024-06-10T02:53:19.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.150+0000 7fa6f5ffb700 1 -- 172.21.15.52:0/558272714 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fa70001fc10 con 0x55d5a53f4ed0 2024-06-10T02:53:19.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.150+0000 7fa71f512b80 10 client.15111 fetch_fsmap finished waiting for FSMap version 29 2024-06-10T02:53:19.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.150+0000 7fa71f512b80 10 client.15111 resolve_mds: resolved name 'i' to daemon mds.i 2024-06-10T02:53:19.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.150+0000 7fa71f512b80 20 client.15111 populate_metadata read hostname 'smithi052' 2024-06-10T02:53:19.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.150+0000 7fa71f512b80 1 --2- 172.21.15.52:0/558272714 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55d5a540e7b0 0x55d5a540fc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:19.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.150+0000 7fa71f512b80 4 client.15111 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T02:53:19.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.150+0000 7fa71f512b80 1 -- 172.21.15.52:0/558272714 --> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d5a5353cc0 con 0x55d5a540e7b0 2024-06-10T02:53:19.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.150+0000 7fa707fff700 1 --2- 172.21.15.52:0/558272714 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55d5a540e7b0 0x55d5a540fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:19.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.151+0000 7fa707fff700 1 --2- 172.21.15.52:0/558272714 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55d5a540e7b0 0x55d5a540fc20 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a28e32c1c11dd43 server_cookie=3f06decfbc019c13 in_seq=0 out_seq=0 2024-06-10T02:53:19.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.151+0000 7fa6f5ffb700 10 client.15111 ms_handle_connect on v2:172.21.15.188:6838/310444836 2024-06-10T02:53:19.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.152+0000 7fa6f5ffb700 1 -- 172.21.15.52:0/558272714 <== mds.0 v2:172.21.15.188:6838/310444836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55d5a5353cc0 con 0x55d5a540e7b0 2024-06-10T02:53:19.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.152+0000 7fa6f5ffb700 10 client.15111 handle_command_reply: tid=0 2024-06-10T02:53:19.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.152+0000 7fa71f512b80 1 client.15111 shutdown 2024-06-10T02:53:19.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.152+0000 7fa71f512b80 1 -- 172.21.15.52:0/558272714 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d5a53f4ed0 msgr2=0x55d5a5400ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:53:19.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.152+0000 7fa71f512b80 1 --2- 172.21.15.52:0/558272714 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d5a53f4ed0 0x55d5a5400ca0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fa7000116c0 tx=0x7fa7000047c0 comp rx=0 tx=0).stop 2024-06-10T02:53:19.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.152+0000 7fa71f512b80 1 -- 172.21.15.52:0/558272714 shutdown_connections 2024-06-10T02:53:19.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.152+0000 7fa71f512b80 1 --2- 172.21.15.52:0/558272714 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55d5a540e7b0 0x55d5a540fc20 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:19.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.152+0000 7fa71f512b80 1 --2- 172.21.15.52:0/558272714 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d5a53faf10 0x55d5a53fe4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:19.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.152+0000 7fa71f512b80 1 --2- 172.21.15.52:0/558272714 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d5a53f4ed0 0x55d5a5400ca0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:19.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.152+0000 7fa71f512b80 1 --2- 172.21.15.52:0/558272714 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d5a53fa5f0 0x55d5a53fdfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:19.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.152+0000 7fa6f5ffb700 0 client.15111 ms_handle_reset on v2:172.21.15.188:6838/310444836 2024-06-10T02:53:19.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.152+0000 7fa71f512b80 1 -- 172.21.15.52:0/558272714 >> 172.21.15.52:0/558272714 conn(0x55d5a531fc70 msgr2=0x55d5a53f3980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:53:19.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.152+0000 7fa71f512b80 1 -- 172.21.15.52:0/558272714 shutdown_connections 2024-06-10T02:53:19.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.152+0000 7fa71f512b80 1 -- 172.21.15.52:0/558272714 wait complete. 2024-06-10T02:53:19.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.152+0000 7fa71f512b80 20 client.15111 trim_cache size 0 max 16384 2024-06-10T02:53:19.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.174+0000 7fa71f512b80 1 Processor -- start 2024-06-10T02:53:19.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.174+0000 7fa71f512b80 1 -- start start 2024-06-10T02:53:19.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.175+0000 7fa71f512b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d5a53f4ed0 0x55d5a53f4990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:19.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.175+0000 7fa71f512b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d5a53fa5f0 0x55d5a53fe510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:19.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.175+0000 7fa71f512b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d5a53faf10 0x55d5a5419080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:19.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.175+0000 7fa71f512b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55d5a540df60 con 0x55d5a53fa5f0 2024-06-10T02:53:19.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.175+0000 7fa71f512b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55d5a540e630 con 0x55d5a53faf10 2024-06-10T02:53:19.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.175+0000 7fa71f512b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55d5a540ed00 con 0x55d5a53f4ed0 2024-06-10T02:53:19.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.175+0000 7fa707fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d5a53fa5f0 0x55d5a53fe510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:19.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.175+0000 7fa707fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d5a53fa5f0 0x55d5a53fe510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53222/0 (socket says 172.21.15.52:53222) 2024-06-10T02:53:19.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.175+0000 7fa707fff700 1 -- 172.21.15.52:0/1925300397 learned_addr learned my addr 172.21.15.52:0/1925300397 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:53:19.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.175+0000 7fa70e59c700 1 --2- 172.21.15.52:0/1925300397 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d5a53f4ed0 0x55d5a53f4990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:19.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.175+0000 7fa70ed9d700 1 --2- 172.21.15.52:0/1925300397 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d5a53faf10 0x55d5a5419080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:19.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.175+0000 7fa707fff700 1 -- 172.21.15.52:0/1925300397 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d5a53f4ed0 msgr2=0x55d5a53f4990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:53:19.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.175+0000 7fa707fff700 1 --2- 172.21.15.52:0/1925300397 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d5a53f4ed0 0x55d5a53f4990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:19.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.175+0000 7fa707fff700 1 -- 172.21.15.52:0/1925300397 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d5a53faf10 msgr2=0x55d5a5419080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:53:19.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.175+0000 7fa707fff700 1 --2- 172.21.15.52:0/1925300397 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d5a53faf10 0x55d5a5419080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:19.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.175+0000 7fa707fff700 1 -- 172.21.15.52:0/1925300397 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d5a53f06d0 con 0x55d5a53fa5f0 2024-06-10T02:53:19.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.176+0000 7fa707fff700 1 --2- 172.21.15.52:0/1925300397 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d5a53fa5f0 0x55d5a53fe510 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fa6f8005d90 tx=0x7fa6f800b3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9af00cd5a02c345 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:53:19.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.176+0000 7fa6f7fff700 1 -- 172.21.15.52:0/1925300397 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6f8003be0 con 0x55d5a53fa5f0 2024-06-10T02:53:19.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.176+0000 7fa6f7fff700 1 -- 172.21.15.52:0/1925300397 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6f8003d80 con 0x55d5a53fa5f0 2024-06-10T02:53:19.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.176+0000 7fa6f7fff700 1 -- 172.21.15.52:0/1925300397 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6f8005690 con 0x55d5a53fa5f0 2024-06-10T02:53:19.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.177+0000 7fa71f512b80 1 -- 172.21.15.52:0/1925300397 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d5a53fa5f0 msgr2=0x55d5a53fe510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:53:19.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.177+0000 7fa71f512b80 1 --2- 172.21.15.52:0/1925300397 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d5a53fa5f0 0x55d5a53fe510 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fa6f8005d90 tx=0x7fa6f800b3c0 comp rx=0 tx=0).stop 2024-06-10T02:53:19.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.177+0000 7fa71f512b80 1 -- 172.21.15.52:0/1925300397 shutdown_connections 2024-06-10T02:53:19.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.177+0000 7fa71f512b80 1 --2- 172.21.15.52:0/1925300397 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d5a53faf10 0x55d5a5419080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:19.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.177+0000 7fa71f512b80 1 --2- 172.21.15.52:0/1925300397 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d5a53f4ed0 0x55d5a53f4990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:19.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.177+0000 7fa71f512b80 1 --2- 172.21.15.52:0/1925300397 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d5a53fa5f0 0x55d5a53fe510 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:19.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.177+0000 7fa71f512b80 1 -- 172.21.15.52:0/1925300397 >> 172.21.15.52:0/1925300397 conn(0x55d5a531fc70 msgr2=0x55d5a53fc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:53:19.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.177+0000 7fa71f512b80 1 -- 172.21.15.52:0/1925300397 shutdown_connections 2024-06-10T02:53:19.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.177+0000 7fa71f512b80 1 -- 172.21.15.52:0/1925300397 wait complete. 2024-06-10T02:53:19.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.177+0000 7fa71f512b80 1 -- start start 2024-06-10T02:53:19.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.177+0000 7fa71f512b80 1 Processor -- start 2024-06-10T02:53:19.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.177+0000 7fa71f512b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d5a53f4ed0 0x55d5a53fd920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:19.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.177+0000 7fa71f512b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d5a53fa5f0 0x55d5a53fde60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:19.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.177+0000 7fa71f512b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d5a53faf10 0x55d5a53fe3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:19.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.177+0000 7fa71f512b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55d5a540faa0 con 0x55d5a53faf10 2024-06-10T02:53:19.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.178+0000 7fa71f512b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55d5a540f3d0 con 0x55d5a53fa5f0 2024-06-10T02:53:19.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.178+0000 7fa71f512b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55d5a5410170 con 0x55d5a53f4ed0 2024-06-10T02:53:19.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.178+0000 7fa71f512b80 10 client.0 _ll_register_callbacks cb 0x55d5a52e08d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T02:53:19.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.178+0000 7fa70ed9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d5a53faf10 0x55d5a53fe3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:19.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.178+0000 7fa70e59c700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d5a53f4ed0 0x55d5a53fd920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:19.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.178+0000 7fa70ed9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d5a53faf10 0x55d5a53fe3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53234/0 (socket says 172.21.15.52:53234) 2024-06-10T02:53:19.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.178+0000 7fa70ed9d700 1 -- 172.21.15.52:0/1139442848 learned_addr learned my addr 172.21.15.52:0/1139442848 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:53:19.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.178+0000 7fa707fff700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d5a53fa5f0 0x55d5a53fde60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:19.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.178+0000 7fa70e59c700 1 -- 172.21.15.52:0/1139442848 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d5a53fa5f0 msgr2=0x55d5a53fde60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:53:19.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.178+0000 7fa70e59c700 1 --2- 172.21.15.52:0/1139442848 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d5a53fa5f0 0x55d5a53fde60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:19.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.178+0000 7fa70e59c700 1 -- 172.21.15.52:0/1139442848 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d5a53faf10 msgr2=0x55d5a53fe3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:53:19.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.178+0000 7fa70e59c700 1 --2- 172.21.15.52:0/1139442848 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d5a53faf10 0x55d5a53fe3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:19.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.178+0000 7fa70e59c700 1 -- 172.21.15.52:0/1139442848 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d5a5417700 con 0x55d5a53f4ed0 2024-06-10T02:53:19.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.178+0000 7fa70e59c700 1 --2- 172.21.15.52:0/1139442848 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d5a53f4ed0 0x55d5a53fd920 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa70000b660 tx=0x7fa700028720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=874d701a3cc20bd8 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:53:19.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.178+0000 7fa6f5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-06-10T02:53:19.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.179+0000 7fa6f5ffb700 1 -- 172.21.15.52:0/1139442848 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa700011b30 con 0x55d5a53f4ed0 2024-06-10T02:53:19.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.179+0000 7fa71f512b80 1 -- 172.21.15.52:0/1139442848 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa6f8003d80 con 0x55d5a53f4ed0 2024-06-10T02:53:19.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.179+0000 7fa6f5ffb700 1 -- 172.21.15.52:0/1139442848 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7fa70001ec60 con 0x55d5a53f4ed0 2024-06-10T02:53:19.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.179+0000 7fa6f5ffb700 1 -- 172.21.15.52:0/1139442848 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa70001ee60 con 0x55d5a53f4ed0 2024-06-10T02:53:19.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.179+0000 7fa71f512b80 10 client.24868 fetch_fsmap learned FSMap version 29 2024-06-10T02:53:19.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.179+0000 7fa71f512b80 1 -- 172.21.15.52:0/1139442848 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d5a547a3c0 con 0x55d5a53f4ed0 2024-06-10T02:53:19.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.179+0000 7fa6f5ffb700 1 -- 172.21.15.52:0/1139442848 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6f8003d80 con 0x55d5a53f4ed0 2024-06-10T02:53:19.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.179+0000 7fa6f5ffb700 1 -- 172.21.15.52:0/1139442848 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fa7000404e0 con 0x55d5a53f4ed0 2024-06-10T02:53:19.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.179+0000 7fa71f512b80 10 client.24868 fetch_fsmap finished waiting for FSMap version 29 2024-06-10T02:53:19.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.179+0000 7fa71f512b80 10 client.24868 resolve_mds: resolved name 'i' to daemon mds.i 2024-06-10T02:53:19.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.179+0000 7fa71f512b80 20 client.24868 populate_metadata read hostname 'smithi052' 2024-06-10T02:53:19.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.179+0000 7fa71f512b80 1 --2- 172.21.15.52:0/1139442848 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55d5a547f010 0x55d5a547dd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:19.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.179+0000 7fa71f512b80 4 client.24868 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "status"}] 2024-06-10T02:53:19.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.179+0000 7fa71f512b80 1 -- 172.21.15.52:0/1139442848 --> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] -- command(tid 0: {"prefix": "status"}) -- 0x55d5a534e240 con 0x55d5a547f010 2024-06-10T02:53:19.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.180+0000 7fa707fff700 1 --2- 172.21.15.52:0/1139442848 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55d5a547f010 0x55d5a547dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:19.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.180+0000 7fa6f5ffb700 10 client.24868 ms_handle_connect on v2:172.21.15.188:6838/310444836 2024-06-10T02:53:19.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.180+0000 7fa707fff700 1 --2- 172.21.15.52:0/1139442848 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55d5a547f010 0x55d5a547dd20 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c283ab0f1a9b7759 server_cookie=f659ae771a37223a in_seq=0 out_seq=0 2024-06-10T02:53:19.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.180+0000 7fa6f5ffb700 1 -- 172.21.15.52:0/1139442848 <== mds.0 v2:172.21.15.188:6838/310444836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+328 (crc 0 0 0) 0x55d5a534e240 con 0x55d5a547f010 2024-06-10T02:53:19.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.180+0000 7fa6f5ffb700 10 client.24868 handle_command_reply: tid=0 2024-06-10T02:53:19.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.180+0000 7fa71f512b80 1 client.24868 shutdown 2024-06-10T02:53:19.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.180+0000 7fa71f512b80 1 -- 172.21.15.52:0/1139442848 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d5a53f4ed0 msgr2=0x55d5a53fd920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:53:19.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.180+0000 7fa71f512b80 1 --2- 172.21.15.52:0/1139442848 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d5a53f4ed0 0x55d5a53fd920 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa70000b660 tx=0x7fa700028720 comp rx=0 tx=0).stop 2024-06-10T02:53:19.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.180+0000 7fa71f512b80 1 -- 172.21.15.52:0/1139442848 shutdown_connections 2024-06-10T02:53:19.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.180+0000 7fa71f512b80 1 --2- 172.21.15.52:0/1139442848 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55d5a547f010 0x55d5a547dd20 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:19.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.180+0000 7fa71f512b80 1 --2- 172.21.15.52:0/1139442848 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d5a53faf10 0x55d5a53fe3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:19.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.180+0000 7fa71f512b80 1 --2- 172.21.15.52:0/1139442848 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d5a53f4ed0 0x55d5a53fd920 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:19.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.180+0000 7fa6f5ffb700 0 client.24868 ms_handle_reset on v2:172.21.15.188:6838/310444836 2024-06-10T02:53:19.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.180+0000 7fa71f512b80 1 --2- 172.21.15.52:0/1139442848 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d5a53fa5f0 0x55d5a53fde60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:19.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.180+0000 7fa71f512b80 1 -- 172.21.15.52:0/1139442848 >> 172.21.15.52:0/1139442848 conn(0x55d5a531fc70 msgr2=0x55d5a53e8db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:53:19.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.181+0000 7fa71f512b80 1 -- 172.21.15.52:0/1139442848 shutdown_connections 2024-06-10T02:53:19.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.181+0000 7fa71f512b80 1 -- 172.21.15.52:0/1139442848 wait complete. 2024-06-10T02:53:19.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.181+0000 7fa71f512b80 20 client.24868 trim_cache size 0 max 16384 2024-06-10T02:53:19.187 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-10T02:53:19.187 INFO:teuthology.orchestra.run.smithi052.stdout: "cluster_fsid": "d8676e9e-26d2-11ef-bc9b-c7b262605968", 2024-06-10T02:53:19.187 INFO:teuthology.orchestra.run.smithi052.stdout: "whoami": 0, 2024-06-10T02:53:19.187 INFO:teuthology.orchestra.run.smithi052.stdout: "id": 24484, 2024-06-10T02:53:19.188 INFO:teuthology.orchestra.run.smithi052.stdout: "want_state": "up:active", 2024-06-10T02:53:19.188 INFO:teuthology.orchestra.run.smithi052.stdout: "state": "up:active", 2024-06-10T02:53:19.188 INFO:teuthology.orchestra.run.smithi052.stdout: "fs_name": "cephfs", 2024-06-10T02:53:19.188 INFO:teuthology.orchestra.run.smithi052.stdout: "rank_uptime": 194.98746945900001, 2024-06-10T02:53:19.188 INFO:teuthology.orchestra.run.smithi052.stdout: "mdsmap_epoch": 29, 2024-06-10T02:53:19.188 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_epoch": 100, 2024-06-10T02:53:19.188 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_epoch_barrier": 98, 2024-06-10T02:53:19.188 INFO:teuthology.orchestra.run.smithi052.stdout: "uptime": 223.000889957 2024-06-10T02:53:19.188 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-10T02:53:19.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.181+0000 7fa714c8f700 1 -- 172.21.15.52:0/3277039032 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fa6fc0a2ca0 msgr2=0x7fa6fc0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:53:19.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.181+0000 7fa714c8f700 1 --2- 172.21.15.52:0/3277039032 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fa6fc0a2ca0 0x7fa6fc0a5160 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fa710199b20 tx=0x7fa70000b410 comp rx=0 tx=0).stop 2024-06-10T02:53:19.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.181+0000 7fa714c8f700 1 -- 172.21.15.52:0/3277039032 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa710108530 msgr2=0x7fa7101989e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:53:19.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.181+0000 7fa714c8f700 1 --2- 172.21.15.52:0/3277039032 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa710108530 0x7fa7101989e0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fa7080062a0 tx=0x7fa708004d70 comp rx=0 tx=0).stop 2024-06-10T02:53:19.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.181+0000 7fa714c8f700 1 -- 172.21.15.52:0/3277039032 shutdown_connections 2024-06-10T02:53:19.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.181+0000 7fa714c8f700 1 --2- 172.21.15.52:0/3277039032 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fa6fc0a2ca0 0x7fa6fc0a5160 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:19.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.181+0000 7fa714c8f700 1 --2- 172.21.15.52:0/3277039032 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa710108530 0x7fa7101989e0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:19.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.181+0000 7fa714c8f700 1 --2- 172.21.15.52:0/3277039032 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa71006ac00 0x7fa71019e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:19.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.181+0000 7fa714c8f700 1 --2- 172.21.15.52:0/3277039032 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa71006b5c0 0x7fa71019ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:19.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.181+0000 7fa714c8f700 1 -- 172.21.15.52:0/3277039032 >> 172.21.15.52:0/3277039032 conn(0x7fa710077400 msgr2=0x7fa710100040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:53:19.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.182+0000 7fa714c8f700 1 -- 172.21.15.52:0/3277039032 shutdown_connections 2024-06-10T02:53:19.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:19.182+0000 7fa714c8f700 1 -- 172.21.15.52:0/3277039032 wait complete. 2024-06-10T02:53:19.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:53:18 smithi188 ceph-mon[36599]: pgmap v407: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-10T02:53:19.887 INFO:tasks.cephadm:Setup complete, yielding 2024-06-10T02:53:19.888 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-06-10T02:53:19.898 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi052.front.sepia.ceph.com 2024-06-10T02:53:19.898 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- bash -c 'ceph orch status' 2024-06-10T02:53:20.148 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:53:21.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:53:20 smithi120 ceph-mon[36634]: pgmap v408: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-10T02:53:21.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:53:20 smithi188 ceph-mon[36599]: pgmap v408: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-10T02:53:21.226 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:53:20 smithi052 ceph-mon[32362]: pgmap v408: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-10T02:53:21.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:21.785+0000 7f13b132f700 1 -- 172.21.15.52:0/2895832013 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f13ac105910 msgr2=0x7f13ac107d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:53:21.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:21.785+0000 7f13b132f700 1 --2- 172.21.15.52:0/2895832013 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f13ac105910 0x7f13ac107d00 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f1394009990 tx=0x7f1394009ca0 comp rx=0 tx=0).stop 2024-06-10T02:53:21.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:21.785+0000 7f13b132f700 1 -- 172.21.15.52:0/2895832013 shutdown_connections 2024-06-10T02:53:21.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:21.785+0000 7f13b132f700 1 --2- 172.21.15.52:0/2895832013 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f13ac108240 0x7f13ac10a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:21.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:21.785+0000 7f13b132f700 1 --2- 172.21.15.52:0/2895832013 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f13ac102fe0 0x7f13ac1053d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:21.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:21.785+0000 7f13b132f700 1 --2- 172.21.15.52:0/2895832013 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f13ac105910 0x7f13ac107d00 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:21.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:21.785+0000 7f13b132f700 1 -- 172.21.15.52:0/2895832013 >> 172.21.15.52:0/2895832013 conn(0x7f13ac0fc990 msgr2=0x7f13ac0fedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:53:21.786 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:21.785+0000 7f13b132f700 1 -- 172.21.15.52:0/2895832013 shutdown_connections 2024-06-10T02:53:21.786 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:21.785+0000 7f13b132f700 1 -- 172.21.15.52:0/2895832013 wait complete. 2024-06-10T02:53:21.786 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:21.786+0000 7f13b132f700 1 Processor -- start 2024-06-10T02:53:21.786 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:21.786+0000 7f13b132f700 1 -- start start 2024-06-10T02:53:21.787 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:21.786+0000 7f13b132f700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f13ac102fe0 0x7f13ac102940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:21.787 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:21.786+0000 7f13b132f700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f13ac105910 0x7f13ac100f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:21.787 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:21.786+0000 7f13b132f700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f13ac108240 0x7f13ac1014d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:21.787 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:21.786+0000 7f13b132f700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f13ac10e100 con 0x7f13ac105910 2024-06-10T02:53:21.788 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:21.786+0000 7f13b132f700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f13ac10df80 con 0x7f13ac108240 2024-06-10T02:53:21.788 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:21.786+0000 7f13b132f700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f13ac10e280 con 0x7f13ac102fe0 2024-06-10T02:53:21.788 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:21.787+0000 7f13a27fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f13ac105910 0x7f13ac100f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:21.788 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:21.787+0000 7f13a27fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f13ac105910 0x7f13ac100f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:46122/0 (socket says 172.21.15.52:46122) 2024-06-10T02:53:21.788 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:21.787+0000 7f13a27fc700 1 -- 172.21.15.52:0/4018965870 learned_addr learned my addr 172.21.15.52:0/4018965870 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:53:21.788 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:21.787+0000 7f13ab7fe700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f13ac108240 0x7f13ac1014d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:21.788 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:21.787+0000 7f13aaffd700 1 --2- 172.21.15.52:0/4018965870 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f13ac102fe0 0x7f13ac102940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:21.788 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:21.787+0000 7f13a27fc700 1 -- 172.21.15.52:0/4018965870 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f13ac102fe0 msgr2=0x7f13ac102940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:53:21.788 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:21.787+0000 7f13a27fc700 1 --2- 172.21.15.52:0/4018965870 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f13ac102fe0 0x7f13ac102940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:21.788 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:21.787+0000 7f13a27fc700 1 -- 172.21.15.52:0/4018965870 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f13ac108240 msgr2=0x7f13ac1014d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:53:21.788 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:21.787+0000 7f13a27fc700 1 --2- 172.21.15.52:0/4018965870 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f13ac108240 0x7f13ac1014d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:21.788 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:21.787+0000 7f13a27fc700 1 -- 172.21.15.52:0/4018965870 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13ac101d80 con 0x7f13ac105910 2024-06-10T02:53:21.788 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:21.787+0000 7f13ab7fe700 1 --2- 172.21.15.52:0/4018965870 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f13ac108240 0x7f13ac1014d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T02:53:21.789 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:21.787+0000 7f13a27fc700 1 --2- 172.21.15.52:0/4018965870 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f13ac105910 0x7f13ac100f90 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f13940098e0 tx=0x7f1394005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:53:21.789 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:21.787+0000 7f13a8ff9700 1 -- 172.21.15.52:0/4018965870 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1394027070 con 0x7f13ac105910 2024-06-10T02:53:21.789 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:21.788+0000 7f13b132f700 1 -- 172.21.15.52:0/4018965870 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13ac1a28b0 con 0x7f13ac105910 2024-06-10T02:53:21.789 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:21.788+0000 7f13a8ff9700 1 -- 172.21.15.52:0/4018965870 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1394003f40 con 0x7f13ac105910 2024-06-10T02:53:21.789 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:21.788+0000 7f13a8ff9700 1 -- 172.21.15.52:0/4018965870 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1394022e30 con 0x7f13ac105910 2024-06-10T02:53:21.789 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:21.788+0000 7f13b132f700 1 -- 172.21.15.52:0/4018965870 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13ac1a2d00 con 0x7f13ac105910 2024-06-10T02:53:21.789 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:21.789+0000 7f13b132f700 1 -- 172.21.15.52:0/4018965870 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13ac069180 con 0x7f13ac105910 2024-06-10T02:53:21.790 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:21.790+0000 7f13a8ff9700 1 -- 172.21.15.52:0/4018965870 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f139402a070 con 0x7f13ac105910 2024-06-10T02:53:21.790 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:21.791+0000 7f13a8ff9700 1 --2- 172.21.15.52:0/4018965870 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f13980a2c20 0x7f13980a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:21.790 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:21.791+0000 7f13aaffd700 1 --2- 172.21.15.52:0/4018965870 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f13980a2c20 0x7f13980a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:21.790 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:21.791+0000 7f13a8ff9700 1 -- 172.21.15.52:0/4018965870 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f13940d0250 con 0x7f13ac105910 2024-06-10T02:53:21.790 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:21.791+0000 7f13aaffd700 1 --2- 172.21.15.52:0/4018965870 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f13980a2c20 0x7f13980a50e0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f139c0060b0 tx=0x7f139c00c040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:53:21.794 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:21.795+0000 7f13a8ff9700 1 -- 172.21.15.52:0/4018965870 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f1394013070 con 0x7f13ac105910 2024-06-10T02:53:22.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:22.021+0000 7f13b132f700 1 -- 172.21.15.52:0/4018965870 --> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f13ac0628d0 con 0x7f13980a2c20 2024-06-10T02:53:22.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:22.022+0000 7f13a8ff9700 1 -- 172.21.15.52:0/4018965870 <== mgr.14150 v2:172.21.15.52:6800/3214763969 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f13ac0628d0 con 0x7f13980a2c20 2024-06-10T02:53:22.021 INFO:teuthology.orchestra.run.smithi052.stdout:Backend: cephadm 2024-06-10T02:53:22.021 INFO:teuthology.orchestra.run.smithi052.stdout:Available: Yes 2024-06-10T02:53:22.021 INFO:teuthology.orchestra.run.smithi052.stdout:Paused: No 2024-06-10T02:53:22.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:22.024+0000 7f13b132f700 1 -- 172.21.15.52:0/4018965870 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f13980a2c20 msgr2=0x7f13980a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:53:22.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:22.024+0000 7f13b132f700 1 --2- 172.21.15.52:0/4018965870 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f13980a2c20 0x7f13980a50e0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f139c0060b0 tx=0x7f139c00c040 comp rx=0 tx=0).stop 2024-06-10T02:53:22.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:22.024+0000 7f13b132f700 1 -- 172.21.15.52:0/4018965870 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f13ac105910 msgr2=0x7f13ac100f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:53:22.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:22.024+0000 7f13b132f700 1 --2- 172.21.15.52:0/4018965870 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f13ac105910 0x7f13ac100f90 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f13940098e0 tx=0x7f1394005e80 comp rx=0 tx=0).stop 2024-06-10T02:53:22.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:22.024+0000 7f13b132f700 1 -- 172.21.15.52:0/4018965870 shutdown_connections 2024-06-10T02:53:22.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:22.024+0000 7f13b132f700 1 --2- 172.21.15.52:0/4018965870 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f13ac108240 0x7f13ac1014d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:22.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:22.024+0000 7f13b132f700 1 --2- 172.21.15.52:0/4018965870 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f13ac102fe0 0x7f13ac102940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:22.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:22.024+0000 7f13b132f700 1 --2- 172.21.15.52:0/4018965870 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f13980a2c20 0x7f13980a50e0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:22.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:22.024+0000 7f13b132f700 1 --2- 172.21.15.52:0/4018965870 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f13ac105910 0x7f13ac100f90 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:22.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:22.025+0000 7f13b132f700 1 -- 172.21.15.52:0/4018965870 >> 172.21.15.52:0/4018965870 conn(0x7f13ac0fc990 msgr2=0x7f13ac103aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:53:22.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:22.025+0000 7f13b132f700 1 -- 172.21.15.52:0/4018965870 shutdown_connections 2024-06-10T02:53:22.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:22.025+0000 7f13b132f700 1 -- 172.21.15.52:0/4018965870 wait complete. 2024-06-10T02:53:22.708 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- bash -c 'ceph orch ps' 2024-06-10T02:53:22.962 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:53:22.975 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:53:22 smithi052 ceph-mon[32362]: pgmap v409: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-10T02:53:23.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:53:22 smithi120 ceph-mon[36634]: pgmap v409: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-10T02:53:23.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:53:22 smithi188 ceph-mon[36599]: pgmap v409: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-10T02:53:24.043 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:53:23 smithi052 ceph-mon[32362]: from='client.15123 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T02:53:24.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:53:23 smithi120 ceph-mon[36634]: from='client.15123 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T02:53:24.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:53:23 smithi188 ceph-mon[36599]: from='client.15123 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T02:53:24.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:24.566+0000 7fbf53d3f700 1 -- 172.21.15.52:0/2695402476 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbf4c1025f0 msgr2=0x7fbf4c102a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:53:24.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:24.566+0000 7fbf53d3f700 1 --2- 172.21.15.52:0/2695402476 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbf4c1025f0 0x7fbf4c102a50 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fbf3c0099c0 tx=0x7fbf3c009cd0 comp rx=0 tx=0).stop 2024-06-10T02:53:24.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:24.566+0000 7fbf53d3f700 1 -- 172.21.15.52:0/2695402476 shutdown_connections 2024-06-10T02:53:24.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:24.566+0000 7fbf53d3f700 1 --2- 172.21.15.52:0/2695402476 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbf4c102f90 0x7fbf4c109840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:24.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:24.566+0000 7fbf53d3f700 1 --2- 172.21.15.52:0/2695402476 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbf4c1013f0 0x7fbf4c101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:24.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:24.566+0000 7fbf53d3f700 1 --2- 172.21.15.52:0/2695402476 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbf4c1025f0 0x7fbf4c102a50 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:24.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:24.566+0000 7fbf53d3f700 1 -- 172.21.15.52:0/2695402476 >> 172.21.15.52:0/2695402476 conn(0x7fbf4c0fc990 msgr2=0x7fbf4c0fedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:53:24.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:24.566+0000 7fbf53d3f700 1 -- 172.21.15.52:0/2695402476 shutdown_connections 2024-06-10T02:53:24.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:24.566+0000 7fbf53d3f700 1 -- 172.21.15.52:0/2695402476 wait complete. 2024-06-10T02:53:24.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:24.567+0000 7fbf53d3f700 1 Processor -- start 2024-06-10T02:53:24.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:24.567+0000 7fbf53d3f700 1 -- start start 2024-06-10T02:53:24.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:24.567+0000 7fbf53d3f700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbf4c1013f0 0x7fbf4c105380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:24.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:24.567+0000 7fbf53d3f700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbf4c1025f0 0x7fbf4c1039d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:24.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:24.567+0000 7fbf53d3f700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbf4c102f90 0x7fbf4c103f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:24.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:24.567+0000 7fbf53d3f700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fbf4c10d030 con 0x7fbf4c102f90 2024-06-10T02:53:24.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:24.567+0000 7fbf53d3f700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fbf4c10ceb0 con 0x7fbf4c1025f0 2024-06-10T02:53:24.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:24.567+0000 7fbf53d3f700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fbf4c10d1b0 con 0x7fbf4c1013f0 2024-06-10T02:53:24.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:24.567+0000 7fbf522dc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbf4c102f90 0x7fbf4c103f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:24.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:24.567+0000 7fbf522dc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbf4c102f90 0x7fbf4c103f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:46134/0 (socket says 172.21.15.52:46134) 2024-06-10T02:53:24.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:24.567+0000 7fbf522dc700 1 -- 172.21.15.52:0/121234796 learned_addr learned my addr 172.21.15.52:0/121234796 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:53:24.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:24.567+0000 7fbf512da700 1 --2- 172.21.15.52:0/121234796 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbf4c1025f0 0x7fbf4c1039d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:24.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:24.567+0000 7fbf51adb700 1 --2- 172.21.15.52:0/121234796 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbf4c1013f0 0x7fbf4c105380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:24.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:24.567+0000 7fbf522dc700 1 -- 172.21.15.52:0/121234796 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbf4c1013f0 msgr2=0x7fbf4c105380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:53:24.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:24.567+0000 7fbf522dc700 1 --2- 172.21.15.52:0/121234796 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbf4c1013f0 0x7fbf4c105380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:24.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:24.567+0000 7fbf522dc700 1 -- 172.21.15.52:0/121234796 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbf4c1025f0 msgr2=0x7fbf4c1039d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:53:24.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:24.568+0000 7fbf522dc700 1 --2- 172.21.15.52:0/121234796 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbf4c1025f0 0x7fbf4c1039d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:24.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:24.568+0000 7fbf522dc700 1 -- 172.21.15.52:0/121234796 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf4c1047c0 con 0x7fbf4c102f90 2024-06-10T02:53:24.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:24.568+0000 7fbf512da700 1 --2- 172.21.15.52:0/121234796 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbf4c1025f0 0x7fbf4c1039d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T02:53:24.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:24.568+0000 7fbf51adb700 1 --2- 172.21.15.52:0/121234796 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbf4c1013f0 0x7fbf4c105380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T02:53:24.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:24.568+0000 7fbf522dc700 1 --2- 172.21.15.52:0/121234796 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbf4c102f90 0x7fbf4c103f10 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fbf4400b970 tx=0x7fbf4400bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:53:24.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:24.568+0000 7fbf42ffd700 1 -- 172.21.15.52:0/121234796 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf44019070 con 0x7fbf4c102f90 2024-06-10T02:53:24.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:24.568+0000 7fbf42ffd700 1 -- 172.21.15.52:0/121234796 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf44014440 con 0x7fbf4c102f90 2024-06-10T02:53:24.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:24.568+0000 7fbf53d3f700 1 -- 172.21.15.52:0/121234796 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf4c1a2970 con 0x7fbf4c102f90 2024-06-10T02:53:24.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:24.568+0000 7fbf42ffd700 1 -- 172.21.15.52:0/121234796 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf440075d0 con 0x7fbf4c102f90 2024-06-10T02:53:24.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:24.568+0000 7fbf53d3f700 1 -- 172.21.15.52:0/121234796 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf4c1a2f00 con 0x7fbf4c102f90 2024-06-10T02:53:24.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:24.569+0000 7fbf53d3f700 1 -- 172.21.15.52:0/121234796 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf4c069180 con 0x7fbf4c102f90 2024-06-10T02:53:24.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:24.571+0000 7fbf42ffd700 1 -- 172.21.15.52:0/121234796 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fbf4401c070 con 0x7fbf4c102f90 2024-06-10T02:53:24.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:24.572+0000 7fbf42ffd700 1 --2- 172.21.15.52:0/121234796 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fbf380a2c20 0x7fbf380a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:24.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:24.572+0000 7fbf51adb700 1 --2- 172.21.15.52:0/121234796 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fbf380a2c20 0x7fbf380a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:24.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:24.572+0000 7fbf42ffd700 1 -- 172.21.15.52:0/121234796 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7fbf440cf400 con 0x7fbf4c102f90 2024-06-10T02:53:24.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:24.572+0000 7fbf51adb700 1 --2- 172.21.15.52:0/121234796 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fbf380a2c20 0x7fbf380a50e0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fbf48005fd0 tx=0x7fbf48005f00 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:53:24.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:24.575+0000 7fbf42ffd700 1 -- 172.21.15.52:0/121234796 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fbf44023030 con 0x7fbf4c102f90 2024-06-10T02:53:24.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:24.799+0000 7fbf53d3f700 1 -- 172.21.15.52:0/121234796 --> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7fbf4c0febb0 con 0x7fbf380a2c20 2024-06-10T02:53:24.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:24.812+0000 7fbf42ffd700 1 -- 172.21.15.52:0/121234796 <== mgr.14150 v2:172.21.15.52:6800/3214763969 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4123 (secure 0 0 0) 0x7fbf4c0febb0 con 0x7fbf380a2c20 2024-06-10T02:53:24.811 INFO:teuthology.orchestra.run.smithi052.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-06-10T02:53:24.812 INFO:teuthology.orchestra.run.smithi052.stdout:mds.a smithi052 running (3m) 43s ago 3m 20.3M - 19.0.0-2946-g3d89accf b045ac545f82 20535a539d25 2024-06-10T02:53:24.812 INFO:teuthology.orchestra.run.smithi052.stdout:mds.b smithi120 running (3m) 84s ago 3m 18.0M - 19.0.0-2946-g3d89accf b045ac545f82 920e913c3ac7 2024-06-10T02:53:24.812 INFO:teuthology.orchestra.run.smithi052.stdout:mds.c smithi188 running (3m) 84s ago 3m 18.5M - 19.0.0-2946-g3d89accf b045ac545f82 714d5a7132bb 2024-06-10T02:53:24.812 INFO:teuthology.orchestra.run.smithi052.stdout:mds.d smithi052 running (4m) 43s ago 4m 16.0M - 19.0.0-2946-g3d89accf b045ac545f82 51f2513273d2 2024-06-10T02:53:24.812 INFO:teuthology.orchestra.run.smithi052.stdout:mds.e smithi120 running (4m) 84s ago 3m 18.6M - 19.0.0-2946-g3d89accf b045ac545f82 e38b88a6eb05 2024-06-10T02:53:24.812 INFO:teuthology.orchestra.run.smithi052.stdout:mds.f smithi188 running (3m) 84s ago 3m 20.5M - 19.0.0-2946-g3d89accf b045ac545f82 d833706531e8 2024-06-10T02:53:24.812 INFO:teuthology.orchestra.run.smithi052.stdout:mds.g smithi052 running (3m) 43s ago 3m 17.9M - 19.0.0-2946-g3d89accf b045ac545f82 76f195dccc98 2024-06-10T02:53:24.812 INFO:teuthology.orchestra.run.smithi052.stdout:mds.h smithi120 running (3m) 84s ago 3m 18.3M - 19.0.0-2946-g3d89accf b045ac545f82 a7d21c767cd4 2024-06-10T02:53:24.812 INFO:teuthology.orchestra.run.smithi052.stdout:mds.i smithi188 running (3m) 84s ago 3m 26.4M - 19.0.0-2946-g3d89accf b045ac545f82 843470528b5b 2024-06-10T02:53:24.812 INFO:teuthology.orchestra.run.smithi052.stdout:mds.j smithi052 running (4m) 43s ago 4m 13.6M - 19.0.0-2946-g3d89accf b045ac545f82 6ba576331e94 2024-06-10T02:53:24.812 INFO:teuthology.orchestra.run.smithi052.stdout:mds.k smithi120 running (4m) 84s ago 4m 16.4M - 19.0.0-2946-g3d89accf b045ac545f82 87d4e55dc881 2024-06-10T02:53:24.812 INFO:teuthology.orchestra.run.smithi052.stdout:mds.l smithi188 running (4m) 84s ago 4m 20.6M - 19.0.0-2946-g3d89accf b045ac545f82 14557ac164f9 2024-06-10T02:53:24.813 INFO:teuthology.orchestra.run.smithi052.stdout:mgr.x smithi052 *:9283,8765 running (11m) 43s ago 11m 517M - 19.0.0-2946-g3d89accf b045ac545f82 c194861d5c8b 2024-06-10T02:53:24.813 INFO:teuthology.orchestra.run.smithi052.stdout:mgr.y smithi120 *:8443,8765 running (9m) 84s ago 9m 454M - 19.0.0-2946-g3d89accf b045ac545f82 cb5986bc3039 2024-06-10T02:53:24.813 INFO:teuthology.orchestra.run.smithi052.stdout:mgr.z smithi188 *:8443,8765 running (9m) 84s ago 9m 455M - 19.0.0-2946-g3d89accf b045ac545f82 2ede476ce097 2024-06-10T02:53:24.813 INFO:teuthology.orchestra.run.smithi052.stdout:mon.a smithi052 running (11m) 43s ago 11m 50.4M 2048M 19.0.0-2946-g3d89accf b045ac545f82 cb4e54e38ee7 2024-06-10T02:53:24.813 INFO:teuthology.orchestra.run.smithi052.stdout:mon.b smithi120 running (9m) 84s ago 9m 39.6M 2048M 19.0.0-2946-g3d89accf b045ac545f82 c34e60e12cac 2024-06-10T02:53:24.813 INFO:teuthology.orchestra.run.smithi052.stdout:mon.c smithi188 running (9m) 84s ago 9m 37.7M 2048M 19.0.0-2946-g3d89accf b045ac545f82 2e0de8d1db4b 2024-06-10T02:53:24.813 INFO:teuthology.orchestra.run.smithi052.stdout:osd.0 smithi052 running (9m) 43s ago 9m 46.1M 4273M 19.0.0-2946-g3d89accf b045ac545f82 c48f3cd30f88 2024-06-10T02:53:24.813 INFO:teuthology.orchestra.run.smithi052.stdout:osd.1 smithi120 running (8m) 84s ago 8m 57.0M 4272M 19.0.0-2946-g3d89accf b045ac545f82 69c8735fce9e 2024-06-10T02:53:24.813 INFO:teuthology.orchestra.run.smithi052.stdout:osd.2 smithi188 running (8m) 84s ago 8m 49.6M 4272M 19.0.0-2946-g3d89accf b045ac545f82 6126f8078a4a 2024-06-10T02:53:24.813 INFO:teuthology.orchestra.run.smithi052.stdout:osd.3 smithi052 running (8m) 43s ago 8m 53.9M 4273M 19.0.0-2946-g3d89accf b045ac545f82 2d725109637a 2024-06-10T02:53:24.813 INFO:teuthology.orchestra.run.smithi052.stdout:osd.4 smithi120 running (7m) 84s ago 7m 43.8M 4272M 19.0.0-2946-g3d89accf b045ac545f82 843d778f56e5 2024-06-10T02:53:24.813 INFO:teuthology.orchestra.run.smithi052.stdout:osd.5 smithi188 running (7m) 84s ago 7m 45.3M 4272M 19.0.0-2946-g3d89accf b045ac545f82 3f6cf57d2a0f 2024-06-10T02:53:24.813 INFO:teuthology.orchestra.run.smithi052.stdout:osd.6 smithi052 running (6m) 43s ago 6m 54.2M 4273M 19.0.0-2946-g3d89accf b045ac545f82 ba23282b7e32 2024-06-10T02:53:24.813 INFO:teuthology.orchestra.run.smithi052.stdout:osd.7 smithi120 running (6m) 84s ago 6m 45.3M 4272M 19.0.0-2946-g3d89accf b045ac545f82 9e6d20ad28f8 2024-06-10T02:53:24.814 INFO:teuthology.orchestra.run.smithi052.stdout:osd.8 smithi188 running (5m) 84s ago 5m 45.1M 4272M 19.0.0-2946-g3d89accf b045ac545f82 4eb83d6e0842 2024-06-10T02:53:24.814 INFO:teuthology.orchestra.run.smithi052.stdout:osd.9 smithi052 running (5m) 43s ago 5m 54.7M 4273M 19.0.0-2946-g3d89accf b045ac545f82 bacb807c288d 2024-06-10T02:53:24.814 INFO:teuthology.orchestra.run.smithi052.stdout:osd.10 smithi120 running (5m) 84s ago 5m 54.7M 4272M 19.0.0-2946-g3d89accf b045ac545f82 b2c5b5c905b9 2024-06-10T02:53:24.814 INFO:teuthology.orchestra.run.smithi052.stdout:osd.11 smithi188 running (4m) 84s ago 4m 53.0M 4272M 19.0.0-2946-g3d89accf b045ac545f82 9d67c43c665b 2024-06-10T02:53:24.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:24.815+0000 7fbf53d3f700 1 -- 172.21.15.52:0/121234796 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fbf380a2c20 msgr2=0x7fbf380a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:53:24.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:24.815+0000 7fbf53d3f700 1 --2- 172.21.15.52:0/121234796 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fbf380a2c20 0x7fbf380a50e0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fbf48005fd0 tx=0x7fbf48005f00 comp rx=0 tx=0).stop 2024-06-10T02:53:24.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:24.815+0000 7fbf53d3f700 1 -- 172.21.15.52:0/121234796 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbf4c102f90 msgr2=0x7fbf4c103f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:53:24.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:24.815+0000 7fbf53d3f700 1 --2- 172.21.15.52:0/121234796 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbf4c102f90 0x7fbf4c103f10 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fbf4400b970 tx=0x7fbf4400bd30 comp rx=0 tx=0).stop 2024-06-10T02:53:24.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:24.815+0000 7fbf53d3f700 1 -- 172.21.15.52:0/121234796 shutdown_connections 2024-06-10T02:53:24.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:24.815+0000 7fbf53d3f700 1 --2- 172.21.15.52:0/121234796 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fbf380a2c20 0x7fbf380a50e0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:24.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:24.815+0000 7fbf53d3f700 1 --2- 172.21.15.52:0/121234796 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbf4c102f90 0x7fbf4c103f10 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:24.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:24.815+0000 7fbf53d3f700 1 --2- 172.21.15.52:0/121234796 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbf4c1013f0 0x7fbf4c105380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:24.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:24.815+0000 7fbf53d3f700 1 --2- 172.21.15.52:0/121234796 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbf4c1025f0 0x7fbf4c1039d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:24.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:24.815+0000 7fbf53d3f700 1 -- 172.21.15.52:0/121234796 >> 172.21.15.52:0/121234796 conn(0x7fbf4c0fc990 msgr2=0x7fbf4c0fe4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:53:24.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:24.815+0000 7fbf53d3f700 1 -- 172.21.15.52:0/121234796 shutdown_connections 2024-06-10T02:53:24.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:24.815+0000 7fbf53d3f700 1 -- 172.21.15.52:0/121234796 wait complete. 2024-06-10T02:53:25.135 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:53:24 smithi052 ceph-mon[32362]: pgmap v410: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-10T02:53:25.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:53:24 smithi120 ceph-mon[36634]: pgmap v410: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-10T02:53:25.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:53:24 smithi188 ceph-mon[36599]: pgmap v410: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-10T02:53:25.480 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- bash -c 'ceph orch ls' 2024-06-10T02:53:25.730 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:53:26.106 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:53:25 smithi052 ceph-mon[32362]: from='client.15129 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T02:53:26.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:53:25 smithi120 ceph-mon[36634]: from='client.15129 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T02:53:26.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:53:25 smithi188 ceph-mon[36599]: from='client.15129 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T02:53:27.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:53:26 smithi120 ceph-mon[36634]: pgmap v411: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-10T02:53:27.155 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:53:26 smithi052 ceph-mon[32362]: pgmap v411: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-10T02:53:27.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:53:26 smithi188 ceph-mon[36599]: pgmap v411: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-10T02:53:27.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:27.389+0000 7f5dfdd12700 1 -- 172.21.15.52:0/1471285433 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5df8108280 msgr2=0x7f5df810a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:53:27.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:27.389+0000 7f5dfdd12700 1 --2- 172.21.15.52:0/1471285433 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5df8108280 0x7f5df810a670 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f5df0009990 tx=0x7f5df0009ca0 comp rx=0 tx=0).stop 2024-06-10T02:53:27.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:27.389+0000 7f5dfdd12700 1 -- 172.21.15.52:0/1471285433 shutdown_connections 2024-06-10T02:53:27.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:27.389+0000 7f5dfdd12700 1 --2- 172.21.15.52:0/1471285433 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5df8108280 0x7f5df810a670 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:27.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:27.389+0000 7f5dfdd12700 1 --2- 172.21.15.52:0/1471285433 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5df8103020 0x7f5df8105410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:27.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:27.389+0000 7f5dfdd12700 1 --2- 172.21.15.52:0/1471285433 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5df8105950 0x7f5df8107d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:27.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:27.389+0000 7f5dfdd12700 1 -- 172.21.15.52:0/1471285433 >> 172.21.15.52:0/1471285433 conn(0x7f5df80fc990 msgr2=0x7f5df80fedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:53:27.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:27.389+0000 7f5dfdd12700 1 -- 172.21.15.52:0/1471285433 shutdown_connections 2024-06-10T02:53:27.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:27.389+0000 7f5dfdd12700 1 -- 172.21.15.52:0/1471285433 wait complete. 2024-06-10T02:53:27.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:27.390+0000 7f5dfdd12700 1 Processor -- start 2024-06-10T02:53:27.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:27.390+0000 7f5dfdd12700 1 -- start start 2024-06-10T02:53:27.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:27.390+0000 7f5dfdd12700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5df8103020 0x7f5df8198690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:27.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:27.390+0000 7f5dfdd12700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5df8105950 0x7f5df8198bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:27.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:27.390+0000 7f5dfdd12700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5df8108280 0x7f5df819fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:27.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:27.390+0000 7f5dfdd12700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f5df810e0a0 con 0x7f5df8103020 2024-06-10T02:53:27.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:27.390+0000 7f5dfdd12700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f5df810df20 con 0x7f5df8108280 2024-06-10T02:53:27.392 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:27.390+0000 7f5dfdd12700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f5df810e220 con 0x7f5df8105950 2024-06-10T02:53:27.392 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:27.390+0000 7f5df77fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5df8103020 0x7f5df8198690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:27.392 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:27.390+0000 7f5df77fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5df8103020 0x7f5df8198690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:46152/0 (socket says 172.21.15.52:46152) 2024-06-10T02:53:27.392 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:27.390+0000 7f5df77fe700 1 -- 172.21.15.52:0/2858655990 learned_addr learned my addr 172.21.15.52:0/2858655990 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:53:27.392 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:27.391+0000 7f5df6ffd700 1 --2- 172.21.15.52:0/2858655990 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5df8105950 0x7f5df8198bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:27.392 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:27.391+0000 7f5df7fff700 1 --2- 172.21.15.52:0/2858655990 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5df8108280 0x7f5df819fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:27.392 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:27.391+0000 7f5df77fe700 1 -- 172.21.15.52:0/2858655990 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5df8105950 msgr2=0x7f5df8198bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:53:27.392 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:27.391+0000 7f5df77fe700 1 --2- 172.21.15.52:0/2858655990 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5df8105950 0x7f5df8198bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:27.392 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:27.391+0000 7f5df77fe700 1 -- 172.21.15.52:0/2858655990 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5df8108280 msgr2=0x7f5df819fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:53:27.392 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:27.391+0000 7f5df77fe700 1 --2- 172.21.15.52:0/2858655990 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5df8108280 0x7f5df819fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:27.392 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:27.391+0000 7f5df77fe700 1 -- 172.21.15.52:0/2858655990 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5df81a02f0 con 0x7f5df8103020 2024-06-10T02:53:27.392 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:27.391+0000 7f5df77fe700 1 --2- 172.21.15.52:0/2858655990 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5df8103020 0x7f5df8198690 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f5de8007ee0 tx=0x7f5de800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:53:27.392 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:27.391+0000 7f5df4ff9700 1 -- 172.21.15.52:0/2858655990 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5de801a070 con 0x7f5df8103020 2024-06-10T02:53:27.393 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:27.391+0000 7f5dfdd12700 1 -- 172.21.15.52:0/2858655990 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5df81a05e0 con 0x7f5df8103020 2024-06-10T02:53:27.393 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:27.391+0000 7f5df4ff9700 1 -- 172.21.15.52:0/2858655990 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5de800de90 con 0x7f5df8103020 2024-06-10T02:53:27.393 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:27.391+0000 7f5df4ff9700 1 -- 172.21.15.52:0/2858655990 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5de8014860 con 0x7f5df8103020 2024-06-10T02:53:27.393 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:27.391+0000 7f5dfdd12700 1 -- 172.21.15.52:0/2858655990 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5df81a0b70 con 0x7f5df8103020 2024-06-10T02:53:27.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:27.392+0000 7f5dfdd12700 1 -- 172.21.15.52:0/2858655990 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5df8069180 con 0x7f5df8103020 2024-06-10T02:53:27.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:27.393+0000 7f5df4ff9700 1 -- 172.21.15.52:0/2858655990 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f5de801d070 con 0x7f5df8103020 2024-06-10T02:53:27.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:27.394+0000 7f5df4ff9700 1 --2- 172.21.15.52:0/2858655990 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f5de00a2c20 0x7f5de00a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:27.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:27.394+0000 7f5df6ffd700 1 --2- 172.21.15.52:0/2858655990 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f5de00a2c20 0x7f5de00a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:27.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:27.394+0000 7f5df4ff9700 1 -- 172.21.15.52:0/2858655990 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f5de80ced30 con 0x7f5df8103020 2024-06-10T02:53:27.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:27.394+0000 7f5df6ffd700 1 --2- 172.21.15.52:0/2858655990 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f5de00a2c20 0x7f5de00a50e0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f5dec005fd0 tx=0x7f5dec00a560 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:53:27.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:27.399+0000 7f5df4ff9700 1 -- 172.21.15.52:0/2858655990 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f5de8010070 con 0x7f5df8103020 2024-06-10T02:53:27.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:27.615+0000 7f5dfdd12700 1 -- 172.21.15.52:0/2858655990 --> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7f5df80628d0 con 0x7f5de00a2c20 2024-06-10T02:53:27.620 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:27.621+0000 7f5df4ff9700 1 -- 172.21.15.52:0/2858655990 <== mgr.14150 v2:172.21.15.52:6800/3214763969 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7f5df80628d0 con 0x7f5de00a2c20 2024-06-10T02:53:27.620 INFO:teuthology.orchestra.run.smithi052.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-06-10T02:53:27.620 INFO:teuthology.orchestra.run.smithi052.stdout:mds.all 12/12 87s ago 4m smithi052=a;smithi052=d;smithi052=g;smithi052=j;smithi120=b;smithi120=e;smithi120=h;smithi120=k;smithi188=c;smithi188=f;smithi188=i;smithi188=l;count:12 2024-06-10T02:53:27.620 INFO:teuthology.orchestra.run.smithi052.stdout:mgr 3/3 87s ago 9m smithi052=x;smithi120=y;smithi188=z;count:3 2024-06-10T02:53:27.620 INFO:teuthology.orchestra.run.smithi052.stdout:mon 3/3 87s ago 10m smithi052:172.21.15.52=a;smithi120:172.21.15.120=b;smithi188:172.21.15.188=c;count:3 2024-06-10T02:53:27.620 INFO:teuthology.orchestra.run.smithi052.stdout:osd 12 87s ago - 2024-06-10T02:53:27.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:27.623+0000 7f5dfdd12700 1 -- 172.21.15.52:0/2858655990 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f5de00a2c20 msgr2=0x7f5de00a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:53:27.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:27.623+0000 7f5dfdd12700 1 --2- 172.21.15.52:0/2858655990 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f5de00a2c20 0x7f5de00a50e0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f5dec005fd0 tx=0x7f5dec00a560 comp rx=0 tx=0).stop 2024-06-10T02:53:27.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:27.623+0000 7f5dfdd12700 1 -- 172.21.15.52:0/2858655990 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5df8103020 msgr2=0x7f5df8198690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:53:27.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:27.623+0000 7f5dfdd12700 1 --2- 172.21.15.52:0/2858655990 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5df8103020 0x7f5df8198690 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f5de8007ee0 tx=0x7f5de800d3b0 comp rx=0 tx=0).stop 2024-06-10T02:53:27.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:27.623+0000 7f5dfdd12700 1 -- 172.21.15.52:0/2858655990 shutdown_connections 2024-06-10T02:53:27.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:27.623+0000 7f5dfdd12700 1 --2- 172.21.15.52:0/2858655990 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5df8108280 0x7f5df819fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:27.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:27.623+0000 7f5dfdd12700 1 --2- 172.21.15.52:0/2858655990 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f5de00a2c20 0x7f5de00a50e0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:27.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:27.623+0000 7f5dfdd12700 1 --2- 172.21.15.52:0/2858655990 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5df8103020 0x7f5df8198690 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:27.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:27.623+0000 7f5dfdd12700 1 --2- 172.21.15.52:0/2858655990 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5df8105950 0x7f5df8198bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:27.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:27.623+0000 7f5dfdd12700 1 -- 172.21.15.52:0/2858655990 >> 172.21.15.52:0/2858655990 conn(0x7f5df80fc990 msgr2=0x7f5df81038c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:53:27.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:27.623+0000 7f5dfdd12700 1 -- 172.21.15.52:0/2858655990 shutdown_connections 2024-06-10T02:53:27.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:27.623+0000 7f5dfdd12700 1 -- 172.21.15.52:0/2858655990 wait complete. 2024-06-10T02:53:28.313 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- bash -c 'ceph orch host ls' 2024-06-10T02:53:28.571 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:53:28.950 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:53:28 smithi052 ceph-mon[32362]: pgmap v412: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-06-10T02:53:28.950 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:53:28 smithi052 ceph-mon[32362]: from='client.15135 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T02:53:29.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:53:28 smithi120 ceph-mon[36634]: pgmap v412: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-06-10T02:53:29.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:53:28 smithi120 ceph-mon[36634]: from='client.15135 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T02:53:29.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:53:28 smithi188 ceph-mon[36599]: pgmap v412: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-06-10T02:53:29.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:53:28 smithi188 ceph-mon[36599]: from='client.15135 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T02:53:30.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:30.240+0000 7eff8d550700 1 -- 172.21.15.52:0/701396519 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7eff88103020 msgr2=0x7eff88105410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:53:30.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:30.240+0000 7eff8d550700 1 --2- 172.21.15.52:0/701396519 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7eff88103020 0x7eff88105410 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7eff78009990 tx=0x7eff78009ca0 comp rx=0 tx=0).stop 2024-06-10T02:53:30.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:30.240+0000 7eff8d550700 1 -- 172.21.15.52:0/701396519 shutdown_connections 2024-06-10T02:53:30.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:30.240+0000 7eff8d550700 1 --2- 172.21.15.52:0/701396519 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7eff88108280 0x7eff8810a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:30.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:30.240+0000 7eff8d550700 1 --2- 172.21.15.52:0/701396519 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7eff88103020 0x7eff88105410 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:30.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:30.240+0000 7eff8d550700 1 --2- 172.21.15.52:0/701396519 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7eff88105950 0x7eff88107d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:30.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:30.240+0000 7eff8d550700 1 -- 172.21.15.52:0/701396519 >> 172.21.15.52:0/701396519 conn(0x7eff880fc9f0 msgr2=0x7eff880fee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:53:30.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:30.240+0000 7eff8d550700 1 -- 172.21.15.52:0/701396519 shutdown_connections 2024-06-10T02:53:30.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:30.240+0000 7eff8d550700 1 -- 172.21.15.52:0/701396519 wait complete. 2024-06-10T02:53:30.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:30.241+0000 7eff8d550700 1 Processor -- start 2024-06-10T02:53:30.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:30.241+0000 7eff8d550700 1 -- start start 2024-06-10T02:53:30.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:30.241+0000 7eff8d550700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7eff88103020 0x7eff88198690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:30.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:30.241+0000 7eff8d550700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7eff88105950 0x7eff88198bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:30.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:30.241+0000 7eff8d550700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7eff88108280 0x7eff8819fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:30.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:30.241+0000 7eff8d550700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7eff8810e100 con 0x7eff88108280 2024-06-10T02:53:30.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:30.241+0000 7eff8d550700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7eff8810df80 con 0x7eff88103020 2024-06-10T02:53:30.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:30.241+0000 7eff8d550700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7eff8810e280 con 0x7eff88105950 2024-06-10T02:53:30.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:30.241+0000 7eff877fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7eff88108280 0x7eff8819fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:30.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:30.241+0000 7eff877fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7eff88108280 0x7eff8819fc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:46180/0 (socket says 172.21.15.52:46180) 2024-06-10T02:53:30.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:30.241+0000 7eff877fe700 1 -- 172.21.15.52:0/411968577 learned_addr learned my addr 172.21.15.52:0/411968577 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:53:30.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:30.241+0000 7eff867fc700 1 --2- 172.21.15.52:0/411968577 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7eff88105950 0x7eff88198bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:30.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:30.241+0000 7eff86ffd700 1 --2- 172.21.15.52:0/411968577 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7eff88103020 0x7eff88198690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:30.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:30.242+0000 7eff877fe700 1 -- 172.21.15.52:0/411968577 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7eff88105950 msgr2=0x7eff88198bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:53:30.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:30.242+0000 7eff877fe700 1 --2- 172.21.15.52:0/411968577 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7eff88105950 0x7eff88198bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:30.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:30.242+0000 7eff877fe700 1 -- 172.21.15.52:0/411968577 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7eff88103020 msgr2=0x7eff88198690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:53:30.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:30.242+0000 7eff877fe700 1 --2- 172.21.15.52:0/411968577 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7eff88103020 0x7eff88198690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:30.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:30.242+0000 7eff877fe700 1 -- 172.21.15.52:0/411968577 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff881a02f0 con 0x7eff88108280 2024-06-10T02:53:30.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:30.242+0000 7eff86ffd700 1 --2- 172.21.15.52:0/411968577 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7eff88103020 0x7eff88198690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T02:53:30.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:30.242+0000 7eff877fe700 1 --2- 172.21.15.52:0/411968577 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7eff88108280 0x7eff8819fc50 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7eff80007d70 tx=0x7eff8000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:53:30.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:30.242+0000 7eff6ffff700 1 -- 172.21.15.52:0/411968577 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff80016020 con 0x7eff88108280 2024-06-10T02:53:30.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:30.242+0000 7eff8d550700 1 -- 172.21.15.52:0/411968577 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff881a05e0 con 0x7eff88108280 2024-06-10T02:53:30.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:30.242+0000 7eff6ffff700 1 -- 172.21.15.52:0/411968577 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff8000ad00 con 0x7eff88108280 2024-06-10T02:53:30.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:30.242+0000 7eff6ffff700 1 -- 172.21.15.52:0/411968577 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff80011600 con 0x7eff88108280 2024-06-10T02:53:30.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:30.242+0000 7eff8d550700 1 -- 172.21.15.52:0/411968577 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff881a0b20 con 0x7eff88108280 2024-06-10T02:53:30.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:30.243+0000 7eff8d550700 1 -- 172.21.15.52:0/411968577 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff88069180 con 0x7eff88108280 2024-06-10T02:53:30.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:30.245+0000 7eff6ffff700 1 -- 172.21.15.52:0/411968577 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7eff80019020 con 0x7eff88108280 2024-06-10T02:53:30.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:30.246+0000 7eff6ffff700 1 --2- 172.21.15.52:0/411968577 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7eff700a2c20 0x7eff700a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:30.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:30.246+0000 7eff6ffff700 1 -- 172.21.15.52:0/411968577 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7eff800cf7b0 con 0x7eff88108280 2024-06-10T02:53:30.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:30.246+0000 7eff86ffd700 1 --2- 172.21.15.52:0/411968577 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7eff700a2c20 0x7eff700a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:30.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:30.246+0000 7eff86ffd700 1 --2- 172.21.15.52:0/411968577 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7eff700a2c20 0x7eff700a50e0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7eff7800b5c0 tx=0x7eff78005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:53:30.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:30.250+0000 7eff6ffff700 1 -- 172.21.15.52:0/411968577 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7eff80020020 con 0x7eff88108280 2024-06-10T02:53:30.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:30.462+0000 7eff8d550700 1 -- 172.21.15.52:0/411968577 --> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7eff880628d0 con 0x7eff700a2c20 2024-06-10T02:53:30.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:30.464+0000 7eff6ffff700 1 -- 172.21.15.52:0/411968577 <== mgr.14150 v2:172.21.15.52:6800/3214763969 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7eff880628d0 con 0x7eff700a2c20 2024-06-10T02:53:30.463 INFO:teuthology.orchestra.run.smithi052.stdout:HOST ADDR LABELS STATUS 2024-06-10T02:53:30.463 INFO:teuthology.orchestra.run.smithi052.stdout:smithi052 172.21.15.52 2024-06-10T02:53:30.463 INFO:teuthology.orchestra.run.smithi052.stdout:smithi120 172.21.15.120 2024-06-10T02:53:30.463 INFO:teuthology.orchestra.run.smithi052.stdout:smithi188 172.21.15.188 2024-06-10T02:53:30.463 INFO:teuthology.orchestra.run.smithi052.stdout:3 hosts in cluster 2024-06-10T02:53:30.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:30.466+0000 7eff8d550700 1 -- 172.21.15.52:0/411968577 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7eff700a2c20 msgr2=0x7eff700a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:53:30.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:30.466+0000 7eff8d550700 1 --2- 172.21.15.52:0/411968577 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7eff700a2c20 0x7eff700a50e0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7eff7800b5c0 tx=0x7eff78005fb0 comp rx=0 tx=0).stop 2024-06-10T02:53:30.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:30.466+0000 7eff8d550700 1 -- 172.21.15.52:0/411968577 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7eff88108280 msgr2=0x7eff8819fc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:53:30.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:30.466+0000 7eff8d550700 1 --2- 172.21.15.52:0/411968577 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7eff88108280 0x7eff8819fc50 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7eff80007d70 tx=0x7eff8000a340 comp rx=0 tx=0).stop 2024-06-10T02:53:30.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:30.466+0000 7eff8d550700 1 -- 172.21.15.52:0/411968577 shutdown_connections 2024-06-10T02:53:30.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:30.466+0000 7eff8d550700 1 --2- 172.21.15.52:0/411968577 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7eff700a2c20 0x7eff700a50e0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:30.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:30.466+0000 7eff8d550700 1 --2- 172.21.15.52:0/411968577 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7eff88108280 0x7eff8819fc50 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:30.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:30.466+0000 7eff8d550700 1 --2- 172.21.15.52:0/411968577 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7eff88103020 0x7eff88198690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:30.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:30.466+0000 7eff8d550700 1 --2- 172.21.15.52:0/411968577 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7eff88105950 0x7eff88198bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:30.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:30.466+0000 7eff8d550700 1 -- 172.21.15.52:0/411968577 >> 172.21.15.52:0/411968577 conn(0x7eff880fc9f0 msgr2=0x7eff88103960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:53:30.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:30.466+0000 7eff8d550700 1 -- 172.21.15.52:0/411968577 shutdown_connections 2024-06-10T02:53:30.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:30.466+0000 7eff8d550700 1 -- 172.21.15.52:0/411968577 wait complete. 2024-06-10T02:53:31.081 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:53:30 smithi052 ceph-mon[32362]: pgmap v413: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-06-10T02:53:31.110 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- bash -c 'ceph orch device ls' 2024-06-10T02:53:31.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:53:30 smithi120 ceph-mon[36634]: pgmap v413: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-06-10T02:53:31.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:53:30 smithi188 ceph-mon[36599]: pgmap v413: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-06-10T02:53:31.361 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:53:32.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:53:31 smithi120 ceph-mon[36634]: from='client.15141 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T02:53:32.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:53:31 smithi120 ceph-mon[36634]: pgmap v414: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-06-10T02:53:32.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:53:31 smithi188 ceph-mon[36599]: from='client.15141 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T02:53:32.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:53:31 smithi188 ceph-mon[36599]: pgmap v414: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-06-10T02:53:32.226 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:53:31 smithi052 ceph-mon[32362]: from='client.15141 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T02:53:32.226 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:53:31 smithi052 ceph-mon[32362]: pgmap v414: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-06-10T02:53:33.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:33.155+0000 7f6a4fc72700 1 -- 172.21.15.52:0/4277559787 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6a48102fa0 msgr2=0x7f6a48109850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:53:33.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:33.155+0000 7f6a4fc72700 1 --2- 172.21.15.52:0/4277559787 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6a48102fa0 0x7f6a48109850 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f6a4000b0d0 tx=0x7f6a4000b3e0 comp rx=0 tx=0).stop 2024-06-10T02:53:33.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:33.155+0000 7f6a4fc72700 1 -- 172.21.15.52:0/4277559787 shutdown_connections 2024-06-10T02:53:33.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:33.155+0000 7f6a4fc72700 1 --2- 172.21.15.52:0/4277559787 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6a48102fa0 0x7f6a48109850 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:33.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:33.155+0000 7f6a4fc72700 1 --2- 172.21.15.52:0/4277559787 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6a48101400 0x7f6a48101820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:33.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:33.155+0000 7f6a4fc72700 1 --2- 172.21.15.52:0/4277559787 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6a48102600 0x7f6a48102a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:33.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:33.155+0000 7f6a4fc72700 1 -- 172.21.15.52:0/4277559787 >> 172.21.15.52:0/4277559787 conn(0x7f6a480fc9c0 msgr2=0x7f6a480fede0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:53:33.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:33.155+0000 7f6a4fc72700 1 -- 172.21.15.52:0/4277559787 shutdown_connections 2024-06-10T02:53:33.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:33.155+0000 7f6a4fc72700 1 -- 172.21.15.52:0/4277559787 wait complete. 2024-06-10T02:53:33.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:33.155+0000 7f6a4fc72700 1 Processor -- start 2024-06-10T02:53:33.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:33.155+0000 7f6a4fc72700 1 -- start start 2024-06-10T02:53:33.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:33.156+0000 7f6a4fc72700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6a48101400 0x7f6a48105390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:33.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:33.156+0000 7f6a4fc72700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6a48102600 0x7f6a481039e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:33.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:33.156+0000 7f6a4fc72700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6a48102fa0 0x7f6a48103f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:33.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:33.156+0000 7f6a4fc72700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f6a4810d060 con 0x7f6a48102600 2024-06-10T02:53:33.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:33.156+0000 7f6a4fc72700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f6a4810cee0 con 0x7f6a48102fa0 2024-06-10T02:53:33.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:33.156+0000 7f6a4fc72700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f6a4810d1e0 con 0x7f6a48101400 2024-06-10T02:53:33.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:33.156+0000 7f6a4d20d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6a48102600 0x7f6a481039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:33.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:33.156+0000 7f6a4d20d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6a48102600 0x7f6a481039e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54088/0 (socket says 172.21.15.52:54088) 2024-06-10T02:53:33.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:33.156+0000 7f6a4d20d700 1 -- 172.21.15.52:0/2841634342 learned_addr learned my addr 172.21.15.52:0/2841634342 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:53:33.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:33.156+0000 7f6a4da0e700 1 --2- 172.21.15.52:0/2841634342 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6a48101400 0x7f6a48105390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:33.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:33.156+0000 7f6a4e20f700 1 --2- 172.21.15.52:0/2841634342 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6a48102fa0 0x7f6a48103f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:33.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:33.156+0000 7f6a4d20d700 1 -- 172.21.15.52:0/2841634342 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6a48101400 msgr2=0x7f6a48105390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:53:33.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:33.156+0000 7f6a4d20d700 1 --2- 172.21.15.52:0/2841634342 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6a48101400 0x7f6a48105390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:33.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:33.156+0000 7f6a4d20d700 1 -- 172.21.15.52:0/2841634342 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6a48102fa0 msgr2=0x7f6a48103f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:53:33.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:33.156+0000 7f6a4d20d700 1 --2- 172.21.15.52:0/2841634342 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6a48102fa0 0x7f6a48103f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:33.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:33.156+0000 7f6a4d20d700 1 -- 172.21.15.52:0/2841634342 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a481047d0 con 0x7f6a48102600 2024-06-10T02:53:33.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:33.156+0000 7f6a4da0e700 1 --2- 172.21.15.52:0/2841634342 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6a48101400 0x7f6a48105390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T02:53:33.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:33.156+0000 7f6a4e20f700 1 --2- 172.21.15.52:0/2841634342 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6a48102fa0 0x7f6a48103f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T02:53:33.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:33.156+0000 7f6a4d20d700 1 --2- 172.21.15.52:0/2841634342 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6a48102600 0x7f6a481039e0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f6a3800d950 tx=0x7f6a3800dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:53:33.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:33.156+0000 7f6a3effd700 1 -- 172.21.15.52:0/2841634342 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a38016070 con 0x7f6a48102600 2024-06-10T02:53:33.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:33.156+0000 7f6a3effd700 1 -- 172.21.15.52:0/2841634342 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a38009b40 con 0x7f6a48102600 2024-06-10T02:53:33.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:33.156+0000 7f6a4fc72700 1 -- 172.21.15.52:0/2841634342 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a481a2920 con 0x7f6a48102600 2024-06-10T02:53:33.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:33.156+0000 7f6a3effd700 1 -- 172.21.15.52:0/2841634342 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a3800f620 con 0x7f6a48102600 2024-06-10T02:53:33.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:33.157+0000 7f6a4fc72700 1 -- 172.21.15.52:0/2841634342 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a481a2eb0 con 0x7f6a48102600 2024-06-10T02:53:33.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:33.158+0000 7f6a4fc72700 1 -- 172.21.15.52:0/2841634342 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a48069180 con 0x7f6a48102600 2024-06-10T02:53:33.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:33.159+0000 7f6a3effd700 1 -- 172.21.15.52:0/2841634342 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f6a38019070 con 0x7f6a48102600 2024-06-10T02:53:33.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:33.159+0000 7f6a3effd700 1 --2- 172.21.15.52:0/2841634342 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f6a340a2c20 0x7f6a340a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:33.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:33.159+0000 7f6a4da0e700 1 --2- 172.21.15.52:0/2841634342 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f6a340a2c20 0x7f6a340a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:33.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:33.159+0000 7f6a3effd700 1 -- 172.21.15.52:0/2841634342 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f6a380cf3e0 con 0x7f6a48102600 2024-06-10T02:53:33.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:33.160+0000 7f6a4da0e700 1 --2- 172.21.15.52:0/2841634342 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f6a340a2c20 0x7f6a340a50e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f6a48102460 tx=0x7f6a44008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:53:33.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:33.164+0000 7f6a3effd700 1 -- 172.21.15.52:0/2841634342 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f6a3809bf90 con 0x7f6a48102600 2024-06-10T02:53:33.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:33.385+0000 7f6a4fc72700 1 -- 172.21.15.52:0/2841634342 --> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7f6a480feb30 con 0x7f6a340a2c20 2024-06-10T02:53:33.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:33.386+0000 7f6a3effd700 1 -- 172.21.15.52:0/2841634342 <== mgr.14150 v2:172.21.15.52:6800/3214763969 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7f6a480feb30 con 0x7f6a340a2c20 2024-06-10T02:53:33.386 INFO:teuthology.orchestra.run.smithi052.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-06-10T02:53:33.386 INFO:teuthology.orchestra.run.smithi052.stdout:smithi052 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB7513010N480DGN 447G No 5m ago Has a FileSystem, LVM detected 2024-06-10T02:53:33.386 INFO:teuthology.orchestra.run.smithi052.stdout:smithi120 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400XV400BGN 372G No 5m ago Has a FileSystem, LVM detected 2024-06-10T02:53:33.386 INFO:teuthology.orchestra.run.smithi052.stdout:smithi188 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620500MP400BGN 372G No 4m ago Has a FileSystem, LVM detected 2024-06-10T02:53:33.388 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:33.389+0000 7f6a4fc72700 1 -- 172.21.15.52:0/2841634342 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f6a340a2c20 msgr2=0x7f6a340a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:53:33.388 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:33.389+0000 7f6a4fc72700 1 --2- 172.21.15.52:0/2841634342 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f6a340a2c20 0x7f6a340a50e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f6a48102460 tx=0x7f6a44008040 comp rx=0 tx=0).stop 2024-06-10T02:53:33.388 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:33.389+0000 7f6a4fc72700 1 -- 172.21.15.52:0/2841634342 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6a48102600 msgr2=0x7f6a481039e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:53:33.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:33.389+0000 7f6a4fc72700 1 --2- 172.21.15.52:0/2841634342 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6a48102600 0x7f6a481039e0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f6a3800d950 tx=0x7f6a3800dd10 comp rx=0 tx=0).stop 2024-06-10T02:53:33.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:33.389+0000 7f6a4fc72700 1 -- 172.21.15.52:0/2841634342 shutdown_connections 2024-06-10T02:53:33.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:33.389+0000 7f6a4fc72700 1 --2- 172.21.15.52:0/2841634342 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6a48102fa0 0x7f6a48103f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:33.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:33.389+0000 7f6a4fc72700 1 --2- 172.21.15.52:0/2841634342 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6a48101400 0x7f6a48105390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:33.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:33.389+0000 7f6a4fc72700 1 --2- 172.21.15.52:0/2841634342 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f6a340a2c20 0x7f6a340a50e0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:33.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:33.389+0000 7f6a4fc72700 1 --2- 172.21.15.52:0/2841634342 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6a48102600 0x7f6a481039e0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:33.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:33.389+0000 7f6a4fc72700 1 -- 172.21.15.52:0/2841634342 >> 172.21.15.52:0/2841634342 conn(0x7f6a480fc9c0 msgr2=0x7f6a480fe420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:53:33.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:33.389+0000 7f6a4fc72700 1 -- 172.21.15.52:0/2841634342 shutdown_connections 2024-06-10T02:53:33.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:33.389+0000 7f6a4fc72700 1 -- 172.21.15.52:0/2841634342 wait complete. 2024-06-10T02:53:34.029 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-06-10T02:53:34.038 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi052.front.sepia.ceph.com 2024-06-10T02:53:34.038 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- bash -c 'ceph fs dump' 2024-06-10T02:53:34.291 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:53:34.388 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:53:34 smithi120 ceph-mon[36634]: pgmap v415: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s 2024-06-10T02:53:34.388 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:53:34 smithi120 ceph-mon[36634]: from='client.15147 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T02:53:34.422 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:53:34 smithi052 ceph-mon[32362]: pgmap v415: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s 2024-06-10T02:53:34.422 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:53:34 smithi052 ceph-mon[32362]: from='client.15147 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T02:53:34.455 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:53:34 smithi188 ceph-mon[36599]: pgmap v415: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s 2024-06-10T02:53:34.455 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:53:34 smithi188 ceph-mon[36599]: from='client.15147 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T02:53:35.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:35.940+0000 7f6eb722e700 1 -- 172.21.15.52:0/1649609558 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6eb0102f80 msgr2=0x7f6eb0105370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:53:35.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:35.940+0000 7f6eb722e700 1 --2- 172.21.15.52:0/1649609558 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6eb0102f80 0x7f6eb0105370 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f6eac009990 tx=0x7f6eac009ca0 comp rx=0 tx=0).stop 2024-06-10T02:53:35.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:35.940+0000 7f6eb722e700 1 -- 172.21.15.52:0/1649609558 shutdown_connections 2024-06-10T02:53:35.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:35.940+0000 7f6eb722e700 1 --2- 172.21.15.52:0/1649609558 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6eb01081e0 0x7f6eb010a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:35.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:35.940+0000 7f6eb722e700 1 --2- 172.21.15.52:0/1649609558 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6eb0102f80 0x7f6eb0105370 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:35.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:35.940+0000 7f6eb722e700 1 --2- 172.21.15.52:0/1649609558 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6eb01058b0 0x7f6eb0107ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:35.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:35.940+0000 7f6eb722e700 1 -- 172.21.15.52:0/1649609558 >> 172.21.15.52:0/1649609558 conn(0x7f6eb00fc990 msgr2=0x7f6eb00fedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:53:35.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:35.940+0000 7f6eb722e700 1 -- 172.21.15.52:0/1649609558 shutdown_connections 2024-06-10T02:53:35.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:35.940+0000 7f6eb722e700 1 -- 172.21.15.52:0/1649609558 wait complete. 2024-06-10T02:53:35.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:35.941+0000 7f6eb722e700 1 Processor -- start 2024-06-10T02:53:35.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:35.941+0000 7f6eb722e700 1 -- start start 2024-06-10T02:53:35.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:35.941+0000 7f6eb722e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6eb0102f80 0x7f6eb0198820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:35.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:35.941+0000 7f6eb722e700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6eb01058b0 0x7f6eb0198d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:35.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:35.941+0000 7f6eb722e700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6eb01081e0 0x7f6eb019fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:35.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:35.941+0000 7f6eb722e700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f6eb010e140 con 0x7f6eb0102f80 2024-06-10T02:53:35.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:35.941+0000 7f6eb722e700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f6eb010dfc0 con 0x7f6eb01081e0 2024-06-10T02:53:35.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:35.941+0000 7f6eb722e700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f6eb010e2c0 con 0x7f6eb01058b0 2024-06-10T02:53:35.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:35.941+0000 7f6eb4fca700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6eb0102f80 0x7f6eb0198820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:35.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:35.941+0000 7f6eb4fca700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6eb0102f80 0x7f6eb0198820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54110/0 (socket says 172.21.15.52:54110) 2024-06-10T02:53:35.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:35.941+0000 7f6eb4fca700 1 -- 172.21.15.52:0/26936167 learned_addr learned my addr 172.21.15.52:0/26936167 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:53:35.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:35.941+0000 7f6eb57cb700 1 --2- 172.21.15.52:0/26936167 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6eb01081e0 0x7f6eb019fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:35.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:35.941+0000 7f6ea7fff700 1 --2- 172.21.15.52:0/26936167 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6eb01058b0 0x7f6eb0198d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:35.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:35.942+0000 7f6eb4fca700 1 -- 172.21.15.52:0/26936167 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6eb01058b0 msgr2=0x7f6eb0198d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:53:35.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:35.942+0000 7f6eb4fca700 1 --2- 172.21.15.52:0/26936167 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6eb01058b0 0x7f6eb0198d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:35.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:35.942+0000 7f6eb4fca700 1 -- 172.21.15.52:0/26936167 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6eb01081e0 msgr2=0x7f6eb019fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:53:35.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:35.942+0000 7f6eb4fca700 1 --2- 172.21.15.52:0/26936167 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6eb01081e0 0x7f6eb019fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:35.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:35.942+0000 7f6eb4fca700 1 -- 172.21.15.52:0/26936167 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6eb01a03e0 con 0x7f6eb0102f80 2024-06-10T02:53:35.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:35.942+0000 7f6eb4fca700 1 --2- 172.21.15.52:0/26936167 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6eb0102f80 0x7f6eb0198820 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f6eac000c00 tx=0x7f6eac005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:53:35.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:35.942+0000 7f6ea5ffb700 1 -- 172.21.15.52:0/26936167 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eac027070 con 0x7f6eb0102f80 2024-06-10T02:53:35.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:35.942+0000 7f6ea5ffb700 1 -- 172.21.15.52:0/26936167 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6eac003f40 con 0x7f6eb0102f80 2024-06-10T02:53:35.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:35.942+0000 7f6eb722e700 1 -- 172.21.15.52:0/26936167 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6eb01a0670 con 0x7f6eb0102f80 2024-06-10T02:53:35.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:35.942+0000 7f6ea5ffb700 1 -- 172.21.15.52:0/26936167 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eac022e30 con 0x7f6eb0102f80 2024-06-10T02:53:35.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:35.942+0000 7f6eb722e700 1 -- 172.21.15.52:0/26936167 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6eb01a0b40 con 0x7f6eb0102f80 2024-06-10T02:53:35.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:35.943+0000 7f6eb722e700 1 -- 172.21.15.52:0/26936167 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6eb0069180 con 0x7f6eb0102f80 2024-06-10T02:53:35.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:35.945+0000 7f6ea5ffb700 1 -- 172.21.15.52:0/26936167 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f6eac02a070 con 0x7f6eb0102f80 2024-06-10T02:53:35.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:35.945+0000 7f6ea5ffb700 1 --2- 172.21.15.52:0/26936167 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f6e980a2c20 0x7f6e980a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:35.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:35.945+0000 7f6ea7fff700 1 --2- 172.21.15.52:0/26936167 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f6e980a2c20 0x7f6e980a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:35.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:35.945+0000 7f6ea5ffb700 1 -- 172.21.15.52:0/26936167 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f6eac0cffe0 con 0x7f6eb0102f80 2024-06-10T02:53:35.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:35.946+0000 7f6ea7fff700 1 --2- 172.21.15.52:0/26936167 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f6e980a2c20 0x7f6e980a50e0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f6ea0005950 tx=0x7f6ea000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:53:35.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:35.950+0000 7f6ea5ffb700 1 -- 172.21.15.52:0/26936167 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f6eac013070 con 0x7f6eb0102f80 2024-06-10T02:53:36.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:36.202+0000 7f6eb722e700 1 -- 172.21.15.52:0/26936167 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f6eb00630a0 con 0x7f6eb0102f80 2024-06-10T02:53:36.201 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:53:36 smithi052 ceph-mon[32362]: pgmap v416: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 8 op/s 2024-06-10T02:53:36.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:36.202+0000 7f6ea5ffb700 1 -- 172.21.15.52:0/26936167 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 29 v29) ==== 76+0+3096 (secure 0 0 0) 0x7f6eac0095a0 con 0x7f6eb0102f80 2024-06-10T02:53:36.202 INFO:teuthology.orchestra.run.smithi052.stdout:e29 2024-06-10T02:53:36.202 INFO:teuthology.orchestra.run.smithi052.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-06-10T02:53:36.202 INFO:teuthology.orchestra.run.smithi052.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-06-10T02:53:36.202 INFO:teuthology.orchestra.run.smithi052.stdout:legacy client fscid: 1 2024-06-10T02:53:36.202 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-10T02:53:36.202 INFO:teuthology.orchestra.run.smithi052.stdout:Filesystem 'cephfs' (1) 2024-06-10T02:53:36.202 INFO:teuthology.orchestra.run.smithi052.stdout:fs_name cephfs 2024-06-10T02:53:36.203 INFO:teuthology.orchestra.run.smithi052.stdout:epoch 29 2024-06-10T02:53:36.203 INFO:teuthology.orchestra.run.smithi052.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-06-10T02:53:36.203 INFO:teuthology.orchestra.run.smithi052.stdout:created 2024-06-10T02:50:03.889766+0000 2024-06-10T02:53:36.203 INFO:teuthology.orchestra.run.smithi052.stdout:modified 2024-06-10T02:51:11.420770+0000 2024-06-10T02:53:36.203 INFO:teuthology.orchestra.run.smithi052.stdout:tableserver 0 2024-06-10T02:53:36.203 INFO:teuthology.orchestra.run.smithi052.stdout:root 0 2024-06-10T02:53:36.203 INFO:teuthology.orchestra.run.smithi052.stdout:session_timeout 300 2024-06-10T02:53:36.203 INFO:teuthology.orchestra.run.smithi052.stdout:session_autoclose 300 2024-06-10T02:53:36.203 INFO:teuthology.orchestra.run.smithi052.stdout:max_file_size 1099511627776 2024-06-10T02:53:36.203 INFO:teuthology.orchestra.run.smithi052.stdout:max_xattr_size 65536 2024-06-10T02:53:36.203 INFO:teuthology.orchestra.run.smithi052.stdout:required_client_features {} 2024-06-10T02:53:36.203 INFO:teuthology.orchestra.run.smithi052.stdout:last_failure 0 2024-06-10T02:53:36.204 INFO:teuthology.orchestra.run.smithi052.stdout:last_failure_osd_epoch 0 2024-06-10T02:53:36.204 INFO:teuthology.orchestra.run.smithi052.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-06-10T02:53:36.204 INFO:teuthology.orchestra.run.smithi052.stdout:max_mds 5 2024-06-10T02:53:36.204 INFO:teuthology.orchestra.run.smithi052.stdout:in 0,1,2,3,4 2024-06-10T02:53:36.204 INFO:teuthology.orchestra.run.smithi052.stdout:up {0=24484,1=24469,2=24463,3=24449,4=24428} 2024-06-10T02:53:36.204 INFO:teuthology.orchestra.run.smithi052.stdout:failed 2024-06-10T02:53:36.204 INFO:teuthology.orchestra.run.smithi052.stdout:damaged 2024-06-10T02:53:36.204 INFO:teuthology.orchestra.run.smithi052.stdout:stopped 2024-06-10T02:53:36.204 INFO:teuthology.orchestra.run.smithi052.stdout:data_pools [3,4] 2024-06-10T02:53:36.204 INFO:teuthology.orchestra.run.smithi052.stdout:metadata_pool 2 2024-06-10T02:53:36.204 INFO:teuthology.orchestra.run.smithi052.stdout:inline_data disabled 2024-06-10T02:53:36.204 INFO:teuthology.orchestra.run.smithi052.stdout:balancer 2024-06-10T02:53:36.205 INFO:teuthology.orchestra.run.smithi052.stdout:bal_rank_mask -1 2024-06-10T02:53:36.205 INFO:teuthology.orchestra.run.smithi052.stdout:standby_count_wanted 0 2024-06-10T02:53:36.205 INFO:teuthology.orchestra.run.smithi052.stdout:qdb_cluster leader: 24484 members: 24469,24449,24428,24463,24484 2024-06-10T02:53:36.205 INFO:teuthology.orchestra.run.smithi052.stdout:[mds.i{0:24484} state up:active seq 9 addr [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] compat {c=[1],r=[1],i=[1fff]}] 2024-06-10T02:53:36.205 INFO:teuthology.orchestra.run.smithi052.stdout:[mds.l{0:24427} state up:standby-replay seq 1 addr [v2:172.21.15.188:6832/529564979,v1:172.21.15.188:6833/529564979] compat {c=[1],r=[1],i=[1fff]}] 2024-06-10T02:53:36.205 INFO:teuthology.orchestra.run.smithi052.stdout:[mds.c{1:24469} state up:active seq 25 addr [v2:172.21.15.188:6836/819317726,v1:172.21.15.188:6837/819317726] compat {c=[1],r=[1],i=[1fff]}] 2024-06-10T02:53:36.205 INFO:teuthology.orchestra.run.smithi052.stdout:[mds.e{1:24401} state up:standby-replay seq 1 addr [v2:172.21.15.120:6834/1996370288,v1:172.21.15.120:6835/1996370288] compat {c=[1],r=[1],i=[1fff]}] 2024-06-10T02:53:36.205 INFO:teuthology.orchestra.run.smithi052.stdout:[mds.f{2:24463} state up:active seq 26 addr [v2:172.21.15.188:6834/3904544259,v1:172.21.15.188:6835/3904544259] compat {c=[1],r=[1],i=[1fff]}] 2024-06-10T02:53:36.205 INFO:teuthology.orchestra.run.smithi052.stdout:[mds.k{2:24383} state up:standby-replay seq 1 addr [v2:172.21.15.120:6832/2609002896,v1:172.21.15.120:6833/2609002896] compat {c=[1],r=[1],i=[1fff]}] 2024-06-10T02:53:36.205 INFO:teuthology.orchestra.run.smithi052.stdout:[mds.b{3:24449} state up:active seq 22 addr [v2:172.21.15.120:6838/1672436487,v1:172.21.15.120:6839/1672436487] compat {c=[1],r=[1],i=[1fff]}] 2024-06-10T02:53:36.205 INFO:teuthology.orchestra.run.smithi052.stdout:[mds.a{3:14583} state up:standby-replay seq 1 addr [v2:172.21.15.52:6840/3039275521,v1:172.21.15.52:6841/3039275521] compat {c=[1],r=[1],i=[1fff]}] 2024-06-10T02:53:36.205 INFO:teuthology.orchestra.run.smithi052.stdout:[mds.h{4:24428} state up:active seq 25 addr [v2:172.21.15.120:6836/4132440952,v1:172.21.15.120:6837/4132440952] compat {c=[1],r=[1],i=[1fff]}] 2024-06-10T02:53:36.206 INFO:teuthology.orchestra.run.smithi052.stdout:[mds.g{4:14571} state up:standby-replay seq 1 addr [v2:172.21.15.52:6838/3059599042,v1:172.21.15.52:6839/3059599042] compat {c=[1],r=[1],i=[1fff]}] 2024-06-10T02:53:36.206 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-10T02:53:36.206 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-10T02:53:36.206 INFO:teuthology.orchestra.run.smithi052.stdout:Standby daemons: 2024-06-10T02:53:36.206 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-10T02:53:36.206 INFO:teuthology.orchestra.run.smithi052.stdout:[mds.j{-1:14511} state up:standby seq 1 addr [v2:172.21.15.52:6834/889673247,v1:172.21.15.52:6835/889673247] compat {c=[1],r=[1],i=[1fff]}] 2024-06-10T02:53:36.206 INFO:teuthology.orchestra.run.smithi052.stdout:[mds.d{-1:14523} state up:standby seq 1 addr [v2:172.21.15.52:6836/3615663511,v1:172.21.15.52:6837/3615663511] compat {c=[1],r=[1],i=[1fff]}] 2024-06-10T02:53:36.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:36.206+0000 7f6eb722e700 1 -- 172.21.15.52:0/26936167 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f6e980a2c20 msgr2=0x7f6e980a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:53:36.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:36.206+0000 7f6eb722e700 1 --2- 172.21.15.52:0/26936167 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f6e980a2c20 0x7f6e980a50e0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f6ea0005950 tx=0x7f6ea000b410 comp rx=0 tx=0).stop 2024-06-10T02:53:36.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:36.206+0000 7f6eb722e700 1 -- 172.21.15.52:0/26936167 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6eb0102f80 msgr2=0x7f6eb0198820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:53:36.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:36.206+0000 7f6eb722e700 1 --2- 172.21.15.52:0/26936167 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6eb0102f80 0x7f6eb0198820 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f6eac000c00 tx=0x7f6eac005e80 comp rx=0 tx=0).stop 2024-06-10T02:53:36.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:36.206+0000 7f6eb722e700 1 -- 172.21.15.52:0/26936167 shutdown_connections 2024-06-10T02:53:36.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:36.206+0000 7f6eb722e700 1 --2- 172.21.15.52:0/26936167 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6eb01081e0 0x7f6eb019fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:36.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:36.206+0000 7f6eb722e700 1 --2- 172.21.15.52:0/26936167 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f6e980a2c20 0x7f6e980a50e0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:36.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:36.206+0000 7f6eb722e700 1 --2- 172.21.15.52:0/26936167 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6eb0102f80 0x7f6eb0198820 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:36.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:36.206+0000 7f6eb722e700 1 --2- 172.21.15.52:0/26936167 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6eb01058b0 0x7f6eb0198d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:36.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:36.206+0000 7f6eb722e700 1 -- 172.21.15.52:0/26936167 >> 172.21.15.52:0/26936167 conn(0x7f6eb00fc990 msgr2=0x7f6eb0103940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:53:36.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:36.206+0000 7f6eb722e700 1 -- 172.21.15.52:0/26936167 shutdown_connections 2024-06-10T02:53:36.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:36.206+0000 7f6eb722e700 1 -- 172.21.15.52:0/26936167 wait complete. 2024-06-10T02:53:36.209 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 29 2024-06-10T02:53:36.387 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:53:36 smithi120 ceph-mon[36634]: pgmap v416: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 8 op/s 2024-06-10T02:53:36.453 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:53:36 smithi188 ceph-mon[36599]: pgmap v416: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 8 op/s 2024-06-10T02:53:36.872 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- bash -c 'ceph osd dump' 2024-06-10T02:53:37.126 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:53:37.386 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:53:37 smithi052 ceph-mon[32362]: from='client.? 172.21.15.52:0/26936167' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-06-10T02:53:37.387 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:53:37 smithi120 ceph-mon[36634]: from='client.? 172.21.15.52:0/26936167' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-06-10T02:53:37.452 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:53:37 smithi188 ceph-mon[36599]: from='client.? 172.21.15.52:0/26936167' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-06-10T02:53:38.226 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:53:38 smithi052 ceph-mon[32362]: pgmap v417: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s 2024-06-10T02:53:38.387 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:53:38 smithi120 ceph-mon[36634]: pgmap v417: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s 2024-06-10T02:53:38.453 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:53:38 smithi188 ceph-mon[36599]: pgmap v417: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s 2024-06-10T02:53:38.784 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:38.783+0000 7fb11ac9b700 1 -- 172.21.15.52:0/1548973623 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb1141036f0 msgr2=0x7fb114103b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:53:38.784 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:38.783+0000 7fb11ac9b700 1 --2- 172.21.15.52:0/1548973623 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb1141036f0 0x7fb114103b50 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fb1000099c0 tx=0x7fb100009cd0 comp rx=0 tx=0).stop 2024-06-10T02:53:38.784 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:38.783+0000 7fb11ac9b700 1 -- 172.21.15.52:0/1548973623 shutdown_connections 2024-06-10T02:53:38.784 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:38.783+0000 7fb11ac9b700 1 --2- 172.21.15.52:0/1548973623 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb114104090 0x7fb11410a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:38.784 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:38.783+0000 7fb11ac9b700 1 --2- 172.21.15.52:0/1548973623 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb1141024f0 0x7fb114102910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:38.784 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:38.783+0000 7fb11ac9b700 1 --2- 172.21.15.52:0/1548973623 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb1141036f0 0x7fb114103b50 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:38.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:38.783+0000 7fb11ac9b700 1 -- 172.21.15.52:0/1548973623 >> 172.21.15.52:0/1548973623 conn(0x7fb1140fda90 msgr2=0x7fb1140ffed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:53:38.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:38.783+0000 7fb11ac9b700 1 -- 172.21.15.52:0/1548973623 shutdown_connections 2024-06-10T02:53:38.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:38.783+0000 7fb11ac9b700 1 -- 172.21.15.52:0/1548973623 wait complete. 2024-06-10T02:53:38.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:38.783+0000 7fb11ac9b700 1 Processor -- start 2024-06-10T02:53:38.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:38.783+0000 7fb11ac9b700 1 -- start start 2024-06-10T02:53:38.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:38.784+0000 7fb11ac9b700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb1141024f0 0x7fb114198670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:38.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:38.784+0000 7fb11ac9b700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb1141036f0 0x7fb114198bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:38.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:38.784+0000 7fb11ac9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb114104090 0x7fb11419fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:38.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:38.784+0000 7fb11ac9b700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb11410e0c0 con 0x7fb114104090 2024-06-10T02:53:38.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:38.784+0000 7fb11ac9b700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fb11410df40 con 0x7fb1141024f0 2024-06-10T02:53:38.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:38.784+0000 7fb11ac9b700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fb11410e240 con 0x7fb1141036f0 2024-06-10T02:53:38.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:38.784+0000 7fb119238700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb114104090 0x7fb11419fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:38.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:38.784+0000 7fb119238700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb114104090 0x7fb11419fc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54120/0 (socket says 172.21.15.52:54120) 2024-06-10T02:53:38.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:38.784+0000 7fb119238700 1 -- 172.21.15.52:0/4224770970 learned_addr learned my addr 172.21.15.52:0/4224770970 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:53:38.786 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:38.784+0000 7fb113fff700 1 --2- 172.21.15.52:0/4224770970 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb1141036f0 0x7fb114198bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:38.786 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:38.784+0000 7fb118a37700 1 --2- 172.21.15.52:0/4224770970 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb1141024f0 0x7fb114198670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:38.786 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:38.784+0000 7fb119238700 1 -- 172.21.15.52:0/4224770970 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb1141036f0 msgr2=0x7fb114198bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:53:38.786 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:38.784+0000 7fb119238700 1 --2- 172.21.15.52:0/4224770970 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb1141036f0 0x7fb114198bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:38.786 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:38.784+0000 7fb119238700 1 -- 172.21.15.52:0/4224770970 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb1141024f0 msgr2=0x7fb114198670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:53:38.786 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:38.784+0000 7fb119238700 1 --2- 172.21.15.52:0/4224770970 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb1141024f0 0x7fb114198670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:38.786 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:38.784+0000 7fb119238700 1 -- 172.21.15.52:0/4224770970 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1141a02d0 con 0x7fb114104090 2024-06-10T02:53:38.786 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:38.784+0000 7fb118a37700 1 --2- 172.21.15.52:0/4224770970 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb1141024f0 0x7fb114198670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T02:53:38.786 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:38.784+0000 7fb119238700 1 --2- 172.21.15.52:0/4224770970 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb114104090 0x7fb11419fc30 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fb10c007f10 tx=0x7fb10c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:53:38.786 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:38.784+0000 7fb111ffb700 1 -- 172.21.15.52:0/4224770970 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb10c004bb0 con 0x7fb114104090 2024-06-10T02:53:38.786 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:38.784+0000 7fb111ffb700 1 -- 172.21.15.52:0/4224770970 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb10c004d50 con 0x7fb114104090 2024-06-10T02:53:38.786 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:38.785+0000 7fb11ac9b700 1 -- 172.21.15.52:0/4224770970 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb114074720 con 0x7fb114104090 2024-06-10T02:53:38.786 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:38.785+0000 7fb111ffb700 1 -- 172.21.15.52:0/4224770970 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb10c005a60 con 0x7fb114104090 2024-06-10T02:53:38.787 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:38.785+0000 7fb11ac9b700 1 -- 172.21.15.52:0/4224770970 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb114074c60 con 0x7fb114104090 2024-06-10T02:53:38.787 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:38.786+0000 7fb11ac9b700 1 -- 172.21.15.52:0/4224770970 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb114069180 con 0x7fb114104090 2024-06-10T02:53:38.787 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:38.787+0000 7fb111ffb700 1 -- 172.21.15.52:0/4224770970 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fb10c01a020 con 0x7fb114104090 2024-06-10T02:53:38.788 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:38.787+0000 7fb111ffb700 1 --2- 172.21.15.52:0/4224770970 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fb1040a2ca0 0x7fb1040a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:38.788 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:38.787+0000 7fb111ffb700 1 -- 172.21.15.52:0/4224770970 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7fb10c0ceb70 con 0x7fb114104090 2024-06-10T02:53:38.788 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:38.788+0000 7fb118a37700 1 --2- 172.21.15.52:0/4224770970 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fb1040a2ca0 0x7fb1040a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:38.788 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:38.788+0000 7fb118a37700 1 --2- 172.21.15.52:0/4224770970 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fb1040a2ca0 0x7fb1040a5160 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fb114103550 tx=0x7fb10800a680 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:53:38.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:38.792+0000 7fb111ffb700 1 -- 172.21.15.52:0/4224770970 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fb10c010070 con 0x7fb114104090 2024-06-10T02:53:38.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:39.000+0000 7fb11ac9b700 1 -- 172.21.15.52:0/4224770970 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7fb114076040 con 0x7fb114104090 2024-06-10T02:53:39.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:39.001+0000 7fb111ffb700 1 -- 172.21.15.52:0/4224770970 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v100) ==== 56+0+5679 (secure 0 0 0) 0x7fb10c09b720 con 0x7fb114104090 2024-06-10T02:53:39.000 INFO:teuthology.orchestra.run.smithi052.stdout:epoch 100 2024-06-10T02:53:39.000 INFO:teuthology.orchestra.run.smithi052.stdout:fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 2024-06-10T02:53:39.000 INFO:teuthology.orchestra.run.smithi052.stdout:created 2024-06-10T02:41:43.076837+0000 2024-06-10T02:53:39.000 INFO:teuthology.orchestra.run.smithi052.stdout:modified 2024-06-10T02:51:45.708105+0000 2024-06-10T02:53:39.000 INFO:teuthology.orchestra.run.smithi052.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-06-10T02:53:39.001 INFO:teuthology.orchestra.run.smithi052.stdout:crush_version 27 2024-06-10T02:53:39.001 INFO:teuthology.orchestra.run.smithi052.stdout:full_ratio 0.95 2024-06-10T02:53:39.001 INFO:teuthology.orchestra.run.smithi052.stdout:backfillfull_ratio 0.9 2024-06-10T02:53:39.001 INFO:teuthology.orchestra.run.smithi052.stdout:nearfull_ratio 0.85 2024-06-10T02:53:39.001 INFO:teuthology.orchestra.run.smithi052.stdout:require_min_compat_client luminous 2024-06-10T02:53:39.001 INFO:teuthology.orchestra.run.smithi052.stdout:min_compat_client luminous 2024-06-10T02:53:39.001 INFO:teuthology.orchestra.run.smithi052.stdout:require_osd_release squid 2024-06-10T02:53:39.001 INFO:teuthology.orchestra.run.smithi052.stdout:stretch_mode_enabled false 2024-06-10T02:53:39.001 INFO:teuthology.orchestra.run.smithi052.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 22 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-06-10T02:53:39.001 INFO:teuthology.orchestra.run.smithi052.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 89 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 2.25 2024-06-10T02:53:39.001 INFO:teuthology.orchestra.run.smithi052.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 89 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.05 application cephfs read_balance_score 1.50 2024-06-10T02:53:39.001 INFO:teuthology.orchestra.run.smithi052.stdout:pool 4 'cephfs_data_ec' erasure profile cephfs_data_ec size 4 min_size 3 crush_rule 1 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 96 flags hashpspool,ec_overwrites stripe_width 8192 pg_num_min 64 target_size_ratio 0.9 application cephfs 2024-06-10T02:53:39.001 INFO:teuthology.orchestra.run.smithi052.stdout:max_osd 12 2024-06-10T02:53:39.001 INFO:teuthology.orchestra.run.smithi052.stdout:osd.0 up in weight 1 up_from 8 up_thru 97 down_at 0 last_clean_interval [0,0) [v2:172.21.15.52:6802/4255833408,v1:172.21.15.52:6803/4255833408] [v2:172.21.15.52:6804/4255833408,v1:172.21.15.52:6805/4255833408] exists,up e03c0691-cc5a-497a-bbc2-ce18c337f074 2024-06-10T02:53:39.002 INFO:teuthology.orchestra.run.smithi052.stdout:osd.1 up in weight 1 up_from 13 up_thru 97 down_at 0 last_clean_interval [0,0) [v2:172.21.15.120:6800/4116540053,v1:172.21.15.120:6801/4116540053] [v2:172.21.15.120:6802/4116540053,v1:172.21.15.120:6803/4116540053] exists,up a24d8da8-c5ec-44d7-ab30-56f6ce3efda7 2024-06-10T02:53:39.002 INFO:teuthology.orchestra.run.smithi052.stdout:osd.2 up in weight 1 up_from 18 up_thru 92 down_at 0 last_clean_interval [0,0) [v2:172.21.15.188:6800/2624326999,v1:172.21.15.188:6801/2624326999] [v2:172.21.15.188:6802/2624326999,v1:172.21.15.188:6803/2624326999] exists,up a7162106-ccbf-45bd-bc0e-051ee43f4214 2024-06-10T02:53:39.002 INFO:teuthology.orchestra.run.smithi052.stdout:osd.3 up in weight 1 up_from 26 up_thru 92 down_at 0 last_clean_interval [0,0) [v2:172.21.15.52:6810/116897011,v1:172.21.15.52:6811/116897011] [v2:172.21.15.52:6812/116897011,v1:172.21.15.52:6813/116897011] exists,up 53680244-b49a-4120-b24a-9ad7a6341dc5 2024-06-10T02:53:39.002 INFO:teuthology.orchestra.run.smithi052.stdout:osd.4 up in weight 1 up_from 34 up_thru 92 down_at 0 last_clean_interval [0,0) [v2:172.21.15.120:6808/1320690621,v1:172.21.15.120:6809/1320690621] [v2:172.21.15.120:6810/1320690621,v1:172.21.15.120:6811/1320690621] exists,up 9f72f1f4-2ef3-49c5-8410-c108767d2ef5 2024-06-10T02:53:39.002 INFO:teuthology.orchestra.run.smithi052.stdout:osd.5 up in weight 1 up_from 39 up_thru 97 down_at 0 last_clean_interval [0,0) [v2:172.21.15.188:6808/3951947483,v1:172.21.15.188:6809/3951947483] [v2:172.21.15.188:6810/3951947483,v1:172.21.15.188:6811/3951947483] exists,up 001b5706-7f27-4fa4-9b69-969e6d765abc 2024-06-10T02:53:39.002 INFO:teuthology.orchestra.run.smithi052.stdout:osd.6 up in weight 1 up_from 47 up_thru 92 down_at 0 last_clean_interval [0,0) [v2:172.21.15.52:6818/359084919,v1:172.21.15.52:6819/359084919] [v2:172.21.15.52:6820/359084919,v1:172.21.15.52:6821/359084919] exists,up 140c903e-9b31-4a81-9ccd-2edc0a2decf9 2024-06-10T02:53:39.002 INFO:teuthology.orchestra.run.smithi052.stdout:osd.7 up in weight 1 up_from 52 up_thru 97 down_at 0 last_clean_interval [0,0) [v2:172.21.15.120:6816/3737722011,v1:172.21.15.120:6817/3737722011] [v2:172.21.15.120:6818/3737722011,v1:172.21.15.120:6819/3737722011] exists,up 57b735d4-5072-4ca2-a3ff-3763ad84f15a 2024-06-10T02:53:39.002 INFO:teuthology.orchestra.run.smithi052.stdout:osd.8 up in weight 1 up_from 60 up_thru 99 down_at 0 last_clean_interval [0,0) [v2:172.21.15.188:6816/1444828441,v1:172.21.15.188:6817/1444828441] [v2:172.21.15.188:6818/1444828441,v1:172.21.15.188:6819/1444828441] exists,up 4ffbb43b-854a-4eac-99cf-7af89bc65409 2024-06-10T02:53:39.002 INFO:teuthology.orchestra.run.smithi052.stdout:osd.9 up in weight 1 up_from 68 up_thru 97 down_at 0 last_clean_interval [0,0) [v2:172.21.15.52:6826/924440658,v1:172.21.15.52:6827/924440658] [v2:172.21.15.52:6828/924440658,v1:172.21.15.52:6829/924440658] exists,up 5bfd985c-8acc-4d00-8561-40bbd541c07b 2024-06-10T02:53:39.002 INFO:teuthology.orchestra.run.smithi052.stdout:osd.10 up in weight 1 up_from 73 up_thru 99 down_at 0 last_clean_interval [0,0) [v2:172.21.15.120:6824/1531769083,v1:172.21.15.120:6825/1531769083] [v2:172.21.15.120:6826/1531769083,v1:172.21.15.120:6827/1531769083] exists,up 0180cddf-e109-42cc-829a-185604276f02 2024-06-10T02:53:39.002 INFO:teuthology.orchestra.run.smithi052.stdout:osd.11 up in weight 1 up_from 81 up_thru 92 down_at 0 last_clean_interval [0,0) [v2:172.21.15.188:6824/3769050170,v1:172.21.15.188:6825/3769050170] [v2:172.21.15.188:6826/3769050170,v1:172.21.15.188:6827/3769050170] exists,up 73aec750-82d1-4800-b6a0-4e305912f9ee 2024-06-10T02:53:39.002 INFO:teuthology.orchestra.run.smithi052.stdout:pg_upmap_items 2.4 [2,11] 2024-06-10T02:53:39.003 INFO:teuthology.orchestra.run.smithi052.stdout:pg_upmap_items 2.8 [2,11] 2024-06-10T02:53:39.003 INFO:teuthology.orchestra.run.smithi052.stdout:pg_upmap_items 2.1f [8,4] 2024-06-10T02:53:39.003 INFO:teuthology.orchestra.run.smithi052.stdout:pg_upmap_items 3.6 [3,7] 2024-06-10T02:53:39.003 INFO:teuthology.orchestra.run.smithi052.stdout:pg_upmap_items 3.c [3,9] 2024-06-10T02:53:39.003 INFO:teuthology.orchestra.run.smithi052.stdout:pg_upmap_items 3.10 [5,9] 2024-06-10T02:53:39.003 INFO:teuthology.orchestra.run.smithi052.stdout:pg_upmap_items 3.14 [4,6] 2024-06-10T02:53:39.003 INFO:teuthology.orchestra.run.smithi052.stdout:pg_upmap_items 3.1d [4,8] 2024-06-10T02:53:39.003 INFO:teuthology.orchestra.run.smithi052.stdout:pg_upmap_items 3.36 [4,9] 2024-06-10T02:53:39.003 INFO:teuthology.orchestra.run.smithi052.stdout:pg_upmap_items 4.6 [9,11] 2024-06-10T02:53:39.003 INFO:teuthology.orchestra.run.smithi052.stdout:pg_upmap_items 4.9 [5,8] 2024-06-10T02:53:39.003 INFO:teuthology.orchestra.run.smithi052.stdout:pg_upmap_items 4.f [9,8] 2024-06-10T02:53:39.003 INFO:teuthology.orchestra.run.smithi052.stdout:pg_upmap_items 4.12 [9,10] 2024-06-10T02:53:39.004 INFO:teuthology.orchestra.run.smithi052.stdout:blocklist 172.21.15.52:0/3535271671 expires 2024-06-11T02:42:42.872312+0000 2024-06-10T02:53:39.004 INFO:teuthology.orchestra.run.smithi052.stdout:blocklist 172.21.15.52:0/1092934380 expires 2024-06-11T02:42:42.872312+0000 2024-06-10T02:53:39.004 INFO:teuthology.orchestra.run.smithi052.stdout:blocklist 172.21.15.52:6800/896317057 expires 2024-06-11T02:42:42.872312+0000 2024-06-10T02:53:39.004 INFO:teuthology.orchestra.run.smithi052.stdout:blocklist 172.21.15.52:6801/3934479642 expires 2024-06-11T02:42:15.498118+0000 2024-06-10T02:53:39.004 INFO:teuthology.orchestra.run.smithi052.stdout:blocklist 172.21.15.52:6800/3934479642 expires 2024-06-11T02:42:15.498118+0000 2024-06-10T02:53:39.004 INFO:teuthology.orchestra.run.smithi052.stdout:blocklist 172.21.15.52:0/2759694132 expires 2024-06-11T02:42:15.498118+0000 2024-06-10T02:53:39.004 INFO:teuthology.orchestra.run.smithi052.stdout:blocklist 172.21.15.52:0/2593665631 expires 2024-06-11T02:42:15.498118+0000 2024-06-10T02:53:39.004 INFO:teuthology.orchestra.run.smithi052.stdout:blocklist 172.21.15.52:0/3151504634 expires 2024-06-11T02:42:15.498118+0000 2024-06-10T02:53:39.004 INFO:teuthology.orchestra.run.smithi052.stdout:blocklist 172.21.15.52:6801/896317057 expires 2024-06-11T02:42:42.872312+0000 2024-06-10T02:53:39.004 INFO:teuthology.orchestra.run.smithi052.stdout:blocklist 172.21.15.52:0/1254480982 expires 2024-06-11T02:42:42.872312+0000 2024-06-10T02:53:39.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:39.003+0000 7fb11ac9b700 1 -- 172.21.15.52:0/4224770970 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fb1040a2ca0 msgr2=0x7fb1040a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:53:39.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:39.003+0000 7fb11ac9b700 1 --2- 172.21.15.52:0/4224770970 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fb1040a2ca0 0x7fb1040a5160 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fb114103550 tx=0x7fb10800a680 comp rx=0 tx=0).stop 2024-06-10T02:53:39.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:39.003+0000 7fb11ac9b700 1 -- 172.21.15.52:0/4224770970 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb114104090 msgr2=0x7fb11419fc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:53:39.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:39.003+0000 7fb11ac9b700 1 --2- 172.21.15.52:0/4224770970 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb114104090 0x7fb11419fc30 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fb10c007f10 tx=0x7fb10c00d3b0 comp rx=0 tx=0).stop 2024-06-10T02:53:39.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:39.003+0000 7fb11ac9b700 1 -- 172.21.15.52:0/4224770970 shutdown_connections 2024-06-10T02:53:39.006 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:39.003+0000 7fb11ac9b700 1 --2- 172.21.15.52:0/4224770970 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fb1040a2ca0 0x7fb1040a5160 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:39.006 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:39.003+0000 7fb11ac9b700 1 --2- 172.21.15.52:0/4224770970 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb114104090 0x7fb11419fc30 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:39.006 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:39.003+0000 7fb11ac9b700 1 --2- 172.21.15.52:0/4224770970 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb1141024f0 0x7fb114198670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:39.006 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:39.003+0000 7fb11ac9b700 1 --2- 172.21.15.52:0/4224770970 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb1141036f0 0x7fb114198bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:39.006 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:39.003+0000 7fb11ac9b700 1 -- 172.21.15.52:0/4224770970 >> 172.21.15.52:0/4224770970 conn(0x7fb1140fda90 msgr2=0x7fb1141069f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:53:39.006 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:39.003+0000 7fb11ac9b700 1 -- 172.21.15.52:0/4224770970 shutdown_connections 2024-06-10T02:53:39.006 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:39.003+0000 7fb11ac9b700 1 -- 172.21.15.52:0/4224770970 wait complete. 2024-06-10T02:53:39.368 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:53:39 smithi052 ceph-mon[32362]: from='client.? 172.21.15.52:0/4224770970' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-06-10T02:53:39.387 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:53:39 smithi120 ceph-mon[36634]: from='client.? 172.21.15.52:0/4224770970' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-06-10T02:53:39.453 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:53:39 smithi188 ceph-mon[36599]: from='client.? 172.21.15.52:0/4224770970' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-06-10T02:53:39.689 INFO:teuthology.run_tasks:Running task fs.ready... 2024-06-10T02:53:39.700 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T02:53:39.953 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:53:40.387 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:53:40 smithi120 ceph-mon[36634]: pgmap v418: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s 2024-06-10T02:53:40.407 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:53:40 smithi052 ceph-mon[32362]: pgmap v418: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s 2024-06-10T02:53:40.453 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:53:40 smithi188 ceph-mon[36599]: pgmap v418: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s 2024-06-10T02:53:41.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:41.635+0000 7fbadf754700 1 -- 172.21.15.52:0/472986859 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbad8101df0 msgr2=0x7fbad81021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:53:41.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:41.635+0000 7fbadf754700 1 --2- 172.21.15.52:0/472986859 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbad8101df0 0x7fbad81021d0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fbad400b0d0 tx=0x7fbad400b3e0 comp rx=0 tx=0).stop 2024-06-10T02:53:41.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:41.635+0000 7fbadf754700 1 -- 172.21.15.52:0/472986859 shutdown_connections 2024-06-10T02:53:41.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:41.635+0000 7fbadf754700 1 --2- 172.21.15.52:0/472986859 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbad8106750 0x7fbad810e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:41.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:41.635+0000 7fbadf754700 1 --2- 172.21.15.52:0/472986859 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbad8101df0 0x7fbad81021d0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:41.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:41.635+0000 7fbadf754700 1 --2- 172.21.15.52:0/472986859 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbad81027a0 0x7fbad8106020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:41.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:41.635+0000 7fbadf754700 1 -- 172.21.15.52:0/472986859 >> 172.21.15.52:0/472986859 conn(0x7fbad8077a00 msgr2=0x7fbad80fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:53:41.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:41.635+0000 7fbadf754700 1 -- 172.21.15.52:0/472986859 shutdown_connections 2024-06-10T02:53:41.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:41.635+0000 7fbadf754700 1 -- 172.21.15.52:0/472986859 wait complete. 2024-06-10T02:53:41.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:41.636+0000 7fbadf754700 1 Processor -- start 2024-06-10T02:53:41.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:41.636+0000 7fbadf754700 1 -- start start 2024-06-10T02:53:41.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:41.636+0000 7fbadf754700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbad8101df0 0x7fbad819d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:41.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:41.636+0000 7fbadf754700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbad81027a0 0x7fbad819dd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:41.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:41.636+0000 7fbadf754700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbad8106750 0x7fbad8197990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:41.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:41.636+0000 7fbadf754700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fbad81116f0 con 0x7fbad81027a0 2024-06-10T02:53:41.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:41.636+0000 7fbadf754700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fbad8111570 con 0x7fbad8101df0 2024-06-10T02:53:41.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:41.636+0000 7fbadf754700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fbad8111870 con 0x7fbad8106750 2024-06-10T02:53:41.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:41.636+0000 7fbadccef700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbad81027a0 0x7fbad819dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:41.638 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:41.636+0000 7fbadccef700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbad81027a0 0x7fbad819dd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36998/0 (socket says 172.21.15.52:36998) 2024-06-10T02:53:41.638 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:41.636+0000 7fbadccef700 1 -- 172.21.15.52:0/2648075899 learned_addr learned my addr 172.21.15.52:0/2648075899 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:53:41.638 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:41.636+0000 7fbaddcf1700 1 --2- 172.21.15.52:0/2648075899 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbad8106750 0x7fbad8197990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:41.638 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:41.636+0000 7fbadd4f0700 1 --2- 172.21.15.52:0/2648075899 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbad8101df0 0x7fbad819d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:41.638 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:41.637+0000 7fbadccef700 1 -- 172.21.15.52:0/2648075899 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbad8106750 msgr2=0x7fbad8197990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:53:41.638 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:41.637+0000 7fbadccef700 1 --2- 172.21.15.52:0/2648075899 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbad8106750 0x7fbad8197990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:41.638 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:41.637+0000 7fbadccef700 1 -- 172.21.15.52:0/2648075899 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbad8101df0 msgr2=0x7fbad819d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:53:41.638 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:41.637+0000 7fbadccef700 1 --2- 172.21.15.52:0/2648075899 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbad8101df0 0x7fbad819d7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:41.638 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:41.637+0000 7fbadccef700 1 -- 172.21.15.52:0/2648075899 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbad81980f0 con 0x7fbad81027a0 2024-06-10T02:53:41.638 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:41.637+0000 7fbadd4f0700 1 --2- 172.21.15.52:0/2648075899 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbad8101df0 0x7fbad819d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T02:53:41.638 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:41.637+0000 7fbadccef700 1 --2- 172.21.15.52:0/2648075899 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbad81027a0 0x7fbad819dd10 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fbac8007b50 tx=0x7fbac8007e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:53:41.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:41.637+0000 7fbace7fc700 1 -- 172.21.15.52:0/2648075899 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbac800e500 con 0x7fbad81027a0 2024-06-10T02:53:41.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:41.637+0000 7fbadf754700 1 -- 172.21.15.52:0/2648075899 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbad81983e0 con 0x7fbad81027a0 2024-06-10T02:53:41.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:41.637+0000 7fbace7fc700 1 -- 172.21.15.52:0/2648075899 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbac800eb80 con 0x7fbad81027a0 2024-06-10T02:53:41.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:41.637+0000 7fbace7fc700 1 -- 172.21.15.52:0/2648075899 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbac800ab90 con 0x7fbad81027a0 2024-06-10T02:53:41.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:41.637+0000 7fbadf754700 1 -- 172.21.15.52:0/2648075899 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbad8100a80 con 0x7fbad81027a0 2024-06-10T02:53:41.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:41.638+0000 7fbadf754700 1 -- 172.21.15.52:0/2648075899 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbad8069180 con 0x7fbad81027a0 2024-06-10T02:53:41.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:41.639+0000 7fbace7fc700 1 -- 172.21.15.52:0/2648075899 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fbac800e6a0 con 0x7fbad81027a0 2024-06-10T02:53:41.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:41.640+0000 7fbace7fc700 1 --2- 172.21.15.52:0/2648075899 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fbac40a2ca0 0x7fbac40a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:41.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:41.640+0000 7fbace7fc700 1 -- 172.21.15.52:0/2648075899 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7fbac80d0840 con 0x7fbad81027a0 2024-06-10T02:53:41.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:41.640+0000 7fbadd4f0700 1 --2- 172.21.15.52:0/2648075899 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fbac40a2ca0 0x7fbac40a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:41.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:41.640+0000 7fbadd4f0700 1 --2- 172.21.15.52:0/2648075899 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fbac40a2ca0 0x7fbac40a5160 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fbad4006130 tx=0x7fbad4006000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:53:41.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:41.644+0000 7fbace7fc700 1 -- 172.21.15.52:0/2648075899 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fbac8023060 con 0x7fbad81027a0 2024-06-10T02:53:41.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:41.897+0000 7fbadf754700 1 -- 172.21.15.52:0/2648075899 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbad81014d0 con 0x7fbad81027a0 2024-06-10T02:53:41.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:41.897+0000 7fbace7fc700 1 -- 172.21.15.52:0/2648075899 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11938 (secure 0 0 0) 0x7fbac80b4ec0 con 0x7fbad81027a0 2024-06-10T02:53:41.897 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-10T02:53:41.897 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/889673247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":889673247},{"type":"v1","addr":"172.21.15.52:6835","nonce":889673247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/3615663511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":3615663511},{"type":"v1","addr":"172.21.15.52:6837","nonce":3615663511}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T02:50:03.889766+0000","modified":"2024-06-10T02:51:11.420770+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24484,"mds_1":24469,"mds_2":24463,"mds_3":24449,"mds_4":24428},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14571":{"gid":14571,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6839/3059599042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":3059599042},{"type":"v1","addr":"172.21.15.52:6839","nonce":3059599042}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14583":{"gid":14583,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6841/3039275521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":3039275521},{"type":"v1","addr":"172.21.15.52:6841","nonce":3039275521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24383":{"gid":24383,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6833/2609002896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":2609002896},{"type":"v1","addr":"172.21.15.120:6833","nonce":2609002896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6835/1996370288","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":1996370288},{"type":"v1","addr":"172.21.15.120:6835","nonce":1996370288}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6833/529564979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":529564979},{"type":"v1","addr":"172.21.15.188:6833","nonce":529564979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":25,"addr":"172.21.15.120:6837/4132440952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":4132440952},{"type":"v1","addr":"172.21.15.120:6837","nonce":4132440952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"b","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.120:6839/1672436487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1672436487},{"type":"v1","addr":"172.21.15.120:6839","nonce":1672436487}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":2,"incarnation":22,"state":"up:active","state_seq":26,"addr":"172.21.15.188:6835/3904544259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":3904544259},{"type":"v1","addr":"172.21.15.188:6835","nonce":3904544259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.188:6837/819317726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":819317726},{"type":"v1","addr":"172.21.15.188:6837","nonce":819317726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.188:6839/310444836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":310444836},{"type":"v1","addr":"172.21.15.188:6839","nonce":310444836}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24484,"qdb_cluster":[24469,24449,24428,24463,24484]},"id":1}]} 2024-06-10T02:53:41.899 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:41.899+0000 7fbadf754700 1 -- 172.21.15.52:0/2648075899 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fbac40a2ca0 msgr2=0x7fbac40a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:53:41.899 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:41.899+0000 7fbadf754700 1 --2- 172.21.15.52:0/2648075899 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fbac40a2ca0 0x7fbac40a5160 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fbad4006130 tx=0x7fbad4006000 comp rx=0 tx=0).stop 2024-06-10T02:53:41.899 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:41.899+0000 7fbadf754700 1 -- 172.21.15.52:0/2648075899 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbad81027a0 msgr2=0x7fbad819dd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:53:41.899 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:41.899+0000 7fbadf754700 1 --2- 172.21.15.52:0/2648075899 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbad81027a0 0x7fbad819dd10 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fbac8007b50 tx=0x7fbac8007e60 comp rx=0 tx=0).stop 2024-06-10T02:53:41.899 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:41.900+0000 7fbadf754700 1 -- 172.21.15.52:0/2648075899 shutdown_connections 2024-06-10T02:53:41.899 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:41.900+0000 7fbadf754700 1 --2- 172.21.15.52:0/2648075899 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbad8106750 0x7fbad8197990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:41.899 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:41.900+0000 7fbadf754700 1 --2- 172.21.15.52:0/2648075899 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbad8101df0 0x7fbad819d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:41.899 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:41.900+0000 7fbadf754700 1 --2- 172.21.15.52:0/2648075899 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fbac40a2ca0 0x7fbac40a5160 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:41.899 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:41.900+0000 7fbadf754700 1 --2- 172.21.15.52:0/2648075899 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbad81027a0 0x7fbad819dd10 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:41.899 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:41.900+0000 7fbadf754700 1 -- 172.21.15.52:0/2648075899 >> 172.21.15.52:0/2648075899 conn(0x7fbad8077a00 msgr2=0x7fbad8104ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:53:41.899 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:41.900+0000 7fbadf754700 1 -- 172.21.15.52:0/2648075899 shutdown_connections 2024-06-10T02:53:41.900 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:41.900+0000 7fbadf754700 1 -- 172.21.15.52:0/2648075899 wait complete. 2024-06-10T02:53:41.900 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 29 2024-06-10T02:53:42.226 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:53:42 smithi052 ceph-mon[32362]: pgmap v419: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s 2024-06-10T02:53:42.226 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:53:42 smithi052 ceph-mon[32362]: from='client.? 172.21.15.52:0/2648075899' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T02:53:42.387 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:53:42 smithi120 ceph-mon[36634]: pgmap v419: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s 2024-06-10T02:53:42.388 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:53:42 smithi120 ceph-mon[36634]: from='client.? 172.21.15.52:0/2648075899' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T02:53:42.453 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:53:42 smithi188 ceph-mon[36599]: pgmap v419: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s 2024-06-10T02:53:42.453 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:53:42 smithi188 ceph-mon[36599]: from='client.? 172.21.15.52:0/2648075899' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T02:53:42.561 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T02:53:42.812 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:53:44.226 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:53:44 smithi052 ceph-mon[32362]: pgmap v420: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-06-10T02:53:44.226 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:53:44 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T02:53:44.226 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:53:44 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:53:44.227 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:53:44 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:53:44.227 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:53:44 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:53:44.227 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:53:44 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:53:44.227 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:53:44 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:53:44.227 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:53:44 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:53:44.388 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:53:44 smithi120 ceph-mon[36634]: pgmap v420: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-06-10T02:53:44.388 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:53:44 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T02:53:44.388 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:53:44 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:53:44.388 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:53:44 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:53:44.388 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:53:44 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:53:44.388 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:53:44 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:53:44.388 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:53:44 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:53:44.388 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:53:44 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:53:44.453 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:53:44 smithi188 ceph-mon[36599]: pgmap v420: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-06-10T02:53:44.453 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:53:44 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T02:53:44.453 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:53:44 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:53:44.453 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:53:44 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:53:44.453 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:53:44 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:53:44.453 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:53:44 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:53:44.453 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:53:44 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:53:44.453 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:53:44 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:53:44.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:44.486+0000 7fb10ea53700 1 -- 172.21.15.52:0/2982251056 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb10806ab30 msgr2=0x7fb10810f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:53:44.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:44.486+0000 7fb10ea53700 1 --2- 172.21.15.52:0/2982251056 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb10806ab30 0x7fb10810f320 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fb10000b0a0 tx=0x7fb10000b3b0 comp rx=0 tx=0).stop 2024-06-10T02:53:44.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:44.486+0000 7fb10ea53700 1 -- 172.21.15.52:0/2982251056 shutdown_connections 2024-06-10T02:53:44.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:44.486+0000 7fb10ea53700 1 --2- 172.21.15.52:0/2982251056 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb10806ab30 0x7fb10810f320 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:44.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:44.486+0000 7fb10ea53700 1 --2- 172.21.15.52:0/2982251056 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb108108500 0x7fb1081088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:44.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:44.486+0000 7fb10ea53700 1 --2- 172.21.15.52:0/2982251056 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb10806a170 0x7fb10806a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:44.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:44.486+0000 7fb10ea53700 1 -- 172.21.15.52:0/2982251056 >> 172.21.15.52:0/2982251056 conn(0x7fb108078340 msgr2=0x7fb108078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:53:44.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:44.486+0000 7fb10ea53700 1 -- 172.21.15.52:0/2982251056 shutdown_connections 2024-06-10T02:53:44.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:44.486+0000 7fb10ea53700 1 -- 172.21.15.52:0/2982251056 wait complete. 2024-06-10T02:53:44.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:44.487+0000 7fb10ea53700 1 Processor -- start 2024-06-10T02:53:44.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:44.487+0000 7fb10ea53700 1 -- start start 2024-06-10T02:53:44.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:44.488+0000 7fb10ea53700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb10806a170 0x7fb10819e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:44.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:44.488+0000 7fb10ea53700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb10806ab30 0x7fb10819ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:44.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:44.488+0000 7fb10ea53700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb108108500 0x7fb1081989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:44.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:44.488+0000 7fb10ea53700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb108112760 con 0x7fb10806ab30 2024-06-10T02:53:44.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:44.488+0000 7fb10ea53700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fb1081125e0 con 0x7fb10806a170 2024-06-10T02:53:44.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:44.488+0000 7fb10ea53700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fb1081128e0 con 0x7fb108108500 2024-06-10T02:53:44.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:44.488+0000 7fb1077fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb10806ab30 0x7fb10819ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:44.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:44.488+0000 7fb1077fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb10806ab30 0x7fb10819ed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37016/0 (socket says 172.21.15.52:37016) 2024-06-10T02:53:44.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:44.488+0000 7fb1077fe700 1 -- 172.21.15.52:0/100871593 learned_addr learned my addr 172.21.15.52:0/100871593 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:53:44.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:44.488+0000 7fb107fff700 1 --2- 172.21.15.52:0/100871593 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb10806a170 0x7fb10819e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:44.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:44.488+0000 7fb10cff0700 1 --2- 172.21.15.52:0/100871593 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb108108500 0x7fb1081989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:44.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:44.488+0000 7fb1077fe700 1 -- 172.21.15.52:0/100871593 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb108108500 msgr2=0x7fb1081989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:53:44.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:44.488+0000 7fb1077fe700 1 --2- 172.21.15.52:0/100871593 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb108108500 0x7fb1081989a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:44.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:44.488+0000 7fb1077fe700 1 -- 172.21.15.52:0/100871593 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb10806a170 msgr2=0x7fb10819e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:53:44.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:44.488+0000 7fb1077fe700 1 --2- 172.21.15.52:0/100871593 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb10806a170 0x7fb10819e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:44.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:44.488+0000 7fb1077fe700 1 -- 172.21.15.52:0/100871593 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb108199100 con 0x7fb10806ab30 2024-06-10T02:53:44.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:44.488+0000 7fb1077fe700 1 --2- 172.21.15.52:0/100871593 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb10806ab30 0x7fb10819ed20 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fb0fc00d980 tx=0x7fb0fc00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:53:44.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:44.489+0000 7fb1057fa700 1 -- 172.21.15.52:0/100871593 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0fc016070 con 0x7fb10806ab30 2024-06-10T02:53:44.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:44.489+0000 7fb1057fa700 1 -- 172.21.15.52:0/100871593 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0fc004d10 con 0x7fb10806ab30 2024-06-10T02:53:44.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:44.489+0000 7fb1057fa700 1 -- 172.21.15.52:0/100871593 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0fc005670 con 0x7fb10806ab30 2024-06-10T02:53:44.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:44.489+0000 7fb10ea53700 1 -- 172.21.15.52:0/100871593 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1081993f0 con 0x7fb10806ab30 2024-06-10T02:53:44.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:44.489+0000 7fb10ea53700 1 -- 172.21.15.52:0/100871593 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1081a56b0 con 0x7fb10806ab30 2024-06-10T02:53:44.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:44.490+0000 7fb10ea53700 1 -- 172.21.15.52:0/100871593 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb108069180 con 0x7fb10806ab30 2024-06-10T02:53:44.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:44.491+0000 7fb1057fa700 1 -- 172.21.15.52:0/100871593 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fb0fc004750 con 0x7fb10806ab30 2024-06-10T02:53:44.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:44.492+0000 7fb1057fa700 1 --2- 172.21.15.52:0/100871593 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fb0f00a2ca0 0x7fb0f00a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:44.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:44.492+0000 7fb1057fa700 1 -- 172.21.15.52:0/100871593 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7fb0fc0cf0f0 con 0x7fb10806ab30 2024-06-10T02:53:44.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:44.492+0000 7fb107fff700 1 --2- 172.21.15.52:0/100871593 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fb0f00a2ca0 0x7fb0f00a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:44.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:44.492+0000 7fb107fff700 1 --2- 172.21.15.52:0/100871593 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fb0f00a2ca0 0x7fb0f00a5160 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fb108199ae0 tx=0x7fb0f8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:53:44.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:44.497+0000 7fb1057fa700 1 -- 172.21.15.52:0/100871593 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fb0fc01f020 con 0x7fb10806ab30 2024-06-10T02:53:44.770 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:44.771+0000 7fb10ea53700 1 -- 172.21.15.52:0/100871593 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb10802dca0 con 0x7fb10806ab30 2024-06-10T02:53:44.771 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:44.772+0000 7fb1057fa700 1 -- 172.21.15.52:0/100871593 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11938 (secure 0 0 0) 0x7fb0fc022350 con 0x7fb10806ab30 2024-06-10T02:53:44.771 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-10T02:53:44.771 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/889673247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":889673247},{"type":"v1","addr":"172.21.15.52:6835","nonce":889673247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/3615663511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":3615663511},{"type":"v1","addr":"172.21.15.52:6837","nonce":3615663511}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T02:50:03.889766+0000","modified":"2024-06-10T02:51:11.420770+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24484,"mds_1":24469,"mds_2":24463,"mds_3":24449,"mds_4":24428},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14571":{"gid":14571,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6839/3059599042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":3059599042},{"type":"v1","addr":"172.21.15.52:6839","nonce":3059599042}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14583":{"gid":14583,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6841/3039275521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":3039275521},{"type":"v1","addr":"172.21.15.52:6841","nonce":3039275521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24383":{"gid":24383,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6833/2609002896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":2609002896},{"type":"v1","addr":"172.21.15.120:6833","nonce":2609002896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6835/1996370288","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":1996370288},{"type":"v1","addr":"172.21.15.120:6835","nonce":1996370288}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6833/529564979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":529564979},{"type":"v1","addr":"172.21.15.188:6833","nonce":529564979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":25,"addr":"172.21.15.120:6837/4132440952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":4132440952},{"type":"v1","addr":"172.21.15.120:6837","nonce":4132440952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"b","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.120:6839/1672436487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1672436487},{"type":"v1","addr":"172.21.15.120:6839","nonce":1672436487}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":2,"incarnation":22,"state":"up:active","state_seq":26,"addr":"172.21.15.188:6835/3904544259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":3904544259},{"type":"v1","addr":"172.21.15.188:6835","nonce":3904544259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.188:6837/819317726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":819317726},{"type":"v1","addr":"172.21.15.188:6837","nonce":819317726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.188:6839/310444836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":310444836},{"type":"v1","addr":"172.21.15.188:6839","nonce":310444836}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24484,"qdb_cluster":[24469,24449,24428,24463,24484]},"id":1}]} 2024-06-10T02:53:44.774 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:44.774+0000 7fb10ea53700 1 -- 172.21.15.52:0/100871593 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fb0f00a2ca0 msgr2=0x7fb0f00a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:53:44.774 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:44.774+0000 7fb10ea53700 1 --2- 172.21.15.52:0/100871593 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fb0f00a2ca0 0x7fb0f00a5160 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fb108199ae0 tx=0x7fb0f8009260 comp rx=0 tx=0).stop 2024-06-10T02:53:44.774 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:44.774+0000 7fb10ea53700 1 -- 172.21.15.52:0/100871593 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb10806ab30 msgr2=0x7fb10819ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:53:44.774 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:44.774+0000 7fb10ea53700 1 --2- 172.21.15.52:0/100871593 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb10806ab30 0x7fb10819ed20 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fb0fc00d980 tx=0x7fb0fc00dc90 comp rx=0 tx=0).stop 2024-06-10T02:53:44.774 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:44.775+0000 7fb10ea53700 1 -- 172.21.15.52:0/100871593 shutdown_connections 2024-06-10T02:53:44.774 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:44.775+0000 7fb10ea53700 1 --2- 172.21.15.52:0/100871593 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb108108500 0x7fb1081989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:44.774 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:44.775+0000 7fb10ea53700 1 --2- 172.21.15.52:0/100871593 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb10806a170 0x7fb10819e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:44.774 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:44.775+0000 7fb10ea53700 1 --2- 172.21.15.52:0/100871593 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fb0f00a2ca0 0x7fb0f00a5160 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:44.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:44.775+0000 7fb10ea53700 1 --2- 172.21.15.52:0/100871593 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb10806ab30 0x7fb10819ed20 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:44.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:44.775+0000 7fb10ea53700 1 -- 172.21.15.52:0/100871593 >> 172.21.15.52:0/100871593 conn(0x7fb108078340 msgr2=0x7fb1080ffee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:53:44.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:44.775+0000 7fb10ea53700 1 -- 172.21.15.52:0/100871593 shutdown_connections 2024-06-10T02:53:44.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:44.775+0000 7fb10ea53700 1 -- 172.21.15.52:0/100871593 wait complete. 2024-06-10T02:53:44.775 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 29 2024-06-10T02:53:45.387 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:53:45 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:53:45.388 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:53:45 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T02:53:45.388 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:53:45 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:53:45.388 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:53:45 smithi120 ceph-mon[36634]: from='client.? 172.21.15.52:0/100871593' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T02:53:45.433 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:53:45 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:53:45.433 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:53:45 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T02:53:45.433 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:53:45 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:53:45.433 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:53:45 smithi052 ceph-mon[32362]: from='client.? 172.21.15.52:0/100871593' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T02:53:45.452 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:53:45 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:53:45.453 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:53:45 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T02:53:45.453 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:53:45 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:53:45.453 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:53:45 smithi188 ceph-mon[36599]: from='client.? 172.21.15.52:0/100871593' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T02:53:45.463 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-10T02:53:45.719 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:53:46.859 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:53:46 smithi052 ceph-mon[32362]: pgmap v421: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-10T02:53:47.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:53:46 smithi120 ceph-mon[36634]: pgmap v421: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-10T02:53:47.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:53:46 smithi188 ceph-mon[36599]: pgmap v421: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-10T02:53:47.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:47.402+0000 7f0a44f04700 1 -- 172.21.15.52:0/477172430 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0a40103150 msgr2=0x7f0a40107140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:53:47.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:47.402+0000 7f0a44f04700 1 --2- 172.21.15.52:0/477172430 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0a40103150 0x7f0a40107140 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f0a340099c0 tx=0x7f0a34009cd0 comp rx=0 tx=0).stop 2024-06-10T02:53:47.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:47.402+0000 7f0a44f04700 1 -- 172.21.15.52:0/477172430 shutdown_connections 2024-06-10T02:53:47.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:47.402+0000 7f0a44f04700 1 --2- 172.21.15.52:0/477172430 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0a40107870 0x7f0a4010f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:47.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:47.402+0000 7f0a44f04700 1 --2- 172.21.15.52:0/477172430 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0a401027a0 0x7f0a40102b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:47.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:47.402+0000 7f0a44f04700 1 --2- 172.21.15.52:0/477172430 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0a40103150 0x7f0a40107140 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:47.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:47.402+0000 7f0a44f04700 1 -- 172.21.15.52:0/477172430 >> 172.21.15.52:0/477172430 conn(0x7f0a40079520 msgr2=0x7f0a40079930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:53:47.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:47.402+0000 7f0a44f04700 1 -- 172.21.15.52:0/477172430 shutdown_connections 2024-06-10T02:53:47.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:47.402+0000 7f0a44f04700 1 -- 172.21.15.52:0/477172430 wait complete. 2024-06-10T02:53:47.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:47.403+0000 7f0a44f04700 1 Processor -- start 2024-06-10T02:53:47.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:47.403+0000 7f0a44f04700 1 -- start start 2024-06-10T02:53:47.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:47.403+0000 7f0a44f04700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0a401027a0 0x7f0a4019e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:47.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:47.403+0000 7f0a44f04700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0a40103150 0x7f0a4019ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:47.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:47.403+0000 7f0a44f04700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0a40107870 0x7f0a40198ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:47.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:47.404+0000 7f0a44f04700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f0a40112880 con 0x7f0a40107870 2024-06-10T02:53:47.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:47.404+0000 7f0a44f04700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f0a40112700 con 0x7f0a40103150 2024-06-10T02:53:47.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:47.404+0000 7f0a44f04700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f0a40112a00 con 0x7f0a401027a0 2024-06-10T02:53:47.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:47.404+0000 7f0a3ed9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0a40107870 0x7f0a40198ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:47.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:47.404+0000 7f0a3ed9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0a40107870 0x7f0a40198ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37036/0 (socket says 172.21.15.52:37036) 2024-06-10T02:53:47.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:47.404+0000 7f0a3ed9d700 1 -- 172.21.15.52:0/1910976456 learned_addr learned my addr 172.21.15.52:0/1910976456 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:53:47.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:47.404+0000 7f0a3e59c700 1 --2- 172.21.15.52:0/1910976456 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0a401027a0 0x7f0a4019e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:47.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:47.404+0000 7f0a3dd9b700 1 --2- 172.21.15.52:0/1910976456 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0a40103150 0x7f0a4019ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:47.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:47.404+0000 7f0a3ed9d700 1 -- 172.21.15.52:0/1910976456 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0a401027a0 msgr2=0x7f0a4019e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:53:47.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:47.404+0000 7f0a3ed9d700 1 --2- 172.21.15.52:0/1910976456 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0a401027a0 0x7f0a4019e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:47.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:47.404+0000 7f0a3ed9d700 1 -- 172.21.15.52:0/1910976456 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0a40103150 msgr2=0x7f0a4019ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:53:47.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:47.404+0000 7f0a3ed9d700 1 --2- 172.21.15.52:0/1910976456 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0a40103150 0x7f0a4019ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:47.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:47.404+0000 7f0a3ed9d700 1 -- 172.21.15.52:0/1910976456 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a40199240 con 0x7f0a40107870 2024-06-10T02:53:47.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:47.404+0000 7f0a3ed9d700 1 --2- 172.21.15.52:0/1910976456 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0a40107870 0x7f0a40198ae0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f0a3800b550 tx=0x7f0a3800b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:53:47.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:47.404+0000 7f0a2f7fe700 1 -- 172.21.15.52:0/1910976456 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a38015020 con 0x7f0a40107870 2024-06-10T02:53:47.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:47.404+0000 7f0a2f7fe700 1 -- 172.21.15.52:0/1910976456 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a38011d10 con 0x7f0a40107870 2024-06-10T02:53:47.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:47.404+0000 7f0a44f04700 1 -- 172.21.15.52:0/1910976456 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a40199530 con 0x7f0a40107870 2024-06-10T02:53:47.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:47.404+0000 7f0a2f7fe700 1 -- 172.21.15.52:0/1910976456 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a38010e00 con 0x7f0a40107870 2024-06-10T02:53:47.407 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:47.405+0000 7f0a44f04700 1 -- 172.21.15.52:0/1910976456 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a400746c0 con 0x7f0a40107870 2024-06-10T02:53:47.407 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:47.406+0000 7f0a44f04700 1 -- 172.21.15.52:0/1910976456 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a40069180 con 0x7f0a40107870 2024-06-10T02:53:47.407 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:47.407+0000 7f0a2f7fe700 1 -- 172.21.15.52:0/1910976456 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f0a38018020 con 0x7f0a40107870 2024-06-10T02:53:47.408 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:47.407+0000 7f0a2f7fe700 1 --2- 172.21.15.52:0/1910976456 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f0a280a2c20 0x7f0a280a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:47.408 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:47.407+0000 7f0a3e59c700 1 --2- 172.21.15.52:0/1910976456 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f0a280a2c20 0x7f0a280a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:47.408 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:47.407+0000 7f0a2f7fe700 1 -- 172.21.15.52:0/1910976456 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f0a380ceab0 con 0x7f0a40107870 2024-06-10T02:53:47.408 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:47.408+0000 7f0a3e59c700 1 --2- 172.21.15.52:0/1910976456 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f0a280a2c20 0x7f0a280a50e0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f0a3000a400 tx=0x7f0a3000d040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:53:47.411 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:47.412+0000 7f0a2f7fe700 1 -- 172.21.15.52:0/1910976456 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f0a3801f020 con 0x7f0a40107870 2024-06-10T02:53:47.619 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:47.620+0000 7f0a44f04700 1 -- 172.21.15.52:0/1910976456 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0a40199fd0 con 0x7f0a40107870 2024-06-10T02:53:47.620 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:47.621+0000 7f0a2f7fe700 1 -- 172.21.15.52:0/1910976456 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v100) ==== 74+0+24739 (secure 0 0 0) 0x7f0a3809b660 con 0x7f0a40107870 2024-06-10T02:53:47.621 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-10T02:53:47.621 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":100,"fsid":"d8676e9e-26d2-11ef-bc9b-c7b262605968","created":"2024-06-10T02:41:43.076837+0000","modified":"2024-06-10T02:51:45.708105+0000","last_up_change":"2024-06-10T02:48:57.297829+0000","last_in_change":"2024-06-10T02:48:36.865802+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-10T02:45:07.394690+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-10T02:49:57.926838+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-10T02:50:01.130393+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-10T02:50:10.655588+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e03c0691-cc5a-497a-bbc2-ce18c337f074","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":4255833408},{"type":"v1","addr":"172.21.15.52:6803","nonce":4255833408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":4255833408},{"type":"v1","addr":"172.21.15.52:6805","nonce":4255833408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":4255833408},{"type":"v1","addr":"172.21.15.52:6809","nonce":4255833408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":4255833408},{"type":"v1","addr":"172.21.15.52:6807","nonce":4255833408}]},"public_addr":"172.21.15.52:6803/4255833408","cluster_addr":"172.21.15.52:6805/4255833408","heartbeat_back_addr":"172.21.15.52:6809/4255833408","heartbeat_front_addr":"172.21.15.52:6807/4255833408","state":["exists","up"]},{"osd":1,"uuid":"a24d8da8-c5ec-44d7-ab30-56f6ce3efda7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":4116540053},{"type":"v1","addr":"172.21.15.120:6801","nonce":4116540053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":4116540053},{"type":"v1","addr":"172.21.15.120:6803","nonce":4116540053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":4116540053},{"type":"v1","addr":"172.21.15.120:6807","nonce":4116540053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":4116540053},{"type":"v1","addr":"172.21.15.120:6805","nonce":4116540053}]},"public_addr":"172.21.15.120:6801/4116540053","cluster_addr":"172.21.15.120:6803/4116540053","heartbeat_back_addr":"172.21.15.120:6807/4116540053","heartbeat_front_addr":"172.21.15.120:6805/4116540053","state":["exists","up"]},{"osd":2,"uuid":"a7162106-ccbf-45bd-bc0e-051ee43f4214","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":2624326999},{"type":"v1","addr":"172.21.15.188:6801","nonce":2624326999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":2624326999},{"type":"v1","addr":"172.21.15.188:6803","nonce":2624326999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":2624326999},{"type":"v1","addr":"172.21.15.188:6807","nonce":2624326999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":2624326999},{"type":"v1","addr":"172.21.15.188:6805","nonce":2624326999}]},"public_addr":"172.21.15.188:6801/2624326999","cluster_addr":"172.21.15.188:6803/2624326999","heartbeat_back_addr":"172.21.15.188:6807/2624326999","heartbeat_front_addr":"172.21.15.188:6805/2624326999","state":["exists","up"]},{"osd":3,"uuid":"53680244-b49a-4120-b24a-9ad7a6341dc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":116897011},{"type":"v1","addr":"172.21.15.52:6811","nonce":116897011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":116897011},{"type":"v1","addr":"172.21.15.52:6813","nonce":116897011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":116897011},{"type":"v1","addr":"172.21.15.52:6817","nonce":116897011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":116897011},{"type":"v1","addr":"172.21.15.52:6815","nonce":116897011}]},"public_addr":"172.21.15.52:6811/116897011","cluster_addr":"172.21.15.52:6813/116897011","heartbeat_back_addr":"172.21.15.52:6817/116897011","heartbeat_front_addr":"172.21.15.52:6815/116897011","state":["exists","up"]},{"osd":4,"uuid":"9f72f1f4-2ef3-49c5-8410-c108767d2ef5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":1320690621},{"type":"v1","addr":"172.21.15.120:6809","nonce":1320690621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":1320690621},{"type":"v1","addr":"172.21.15.120:6811","nonce":1320690621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":1320690621},{"type":"v1","addr":"172.21.15.120:6815","nonce":1320690621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":1320690621},{"type":"v1","addr":"172.21.15.120:6813","nonce":1320690621}]},"public_addr":"172.21.15.120:6809/1320690621","cluster_addr":"172.21.15.120:6811/1320690621","heartbeat_back_addr":"172.21.15.120:6815/1320690621","heartbeat_front_addr":"172.21.15.120:6813/1320690621","state":["exists","up"]},{"osd":5,"uuid":"001b5706-7f27-4fa4-9b69-969e6d765abc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3951947483},{"type":"v1","addr":"172.21.15.188:6809","nonce":3951947483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3951947483},{"type":"v1","addr":"172.21.15.188:6811","nonce":3951947483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3951947483},{"type":"v1","addr":"172.21.15.188:6815","nonce":3951947483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3951947483},{"type":"v1","addr":"172.21.15.188:6813","nonce":3951947483}]},"public_addr":"172.21.15.188:6809/3951947483","cluster_addr":"172.21.15.188:6811/3951947483","heartbeat_back_addr":"172.21.15.188:6815/3951947483","heartbeat_front_addr":"172.21.15.188:6813/3951947483","state":["exists","up"]},{"osd":6,"uuid":"140c903e-9b31-4a81-9ccd-2edc0a2decf9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":359084919},{"type":"v1","addr":"172.21.15.52:6819","nonce":359084919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":359084919},{"type":"v1","addr":"172.21.15.52:6821","nonce":359084919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":359084919},{"type":"v1","addr":"172.21.15.52:6825","nonce":359084919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":359084919},{"type":"v1","addr":"172.21.15.52:6823","nonce":359084919}]},"public_addr":"172.21.15.52:6819/359084919","cluster_addr":"172.21.15.52:6821/359084919","heartbeat_back_addr":"172.21.15.52:6825/359084919","heartbeat_front_addr":"172.21.15.52:6823/359084919","state":["exists","up"]},{"osd":7,"uuid":"57b735d4-5072-4ca2-a3ff-3763ad84f15a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":3737722011},{"type":"v1","addr":"172.21.15.120:6817","nonce":3737722011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6818","nonce":3737722011},{"type":"v1","addr":"172.21.15.120:6819","nonce":3737722011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":3737722011},{"type":"v1","addr":"172.21.15.120:6823","nonce":3737722011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":3737722011},{"type":"v1","addr":"172.21.15.120:6821","nonce":3737722011}]},"public_addr":"172.21.15.120:6817/3737722011","cluster_addr":"172.21.15.120:6819/3737722011","heartbeat_back_addr":"172.21.15.120:6823/3737722011","heartbeat_front_addr":"172.21.15.120:6821/3737722011","state":["exists","up"]},{"osd":8,"uuid":"4ffbb43b-854a-4eac-99cf-7af89bc65409","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1444828441},{"type":"v1","addr":"172.21.15.188:6817","nonce":1444828441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1444828441},{"type":"v1","addr":"172.21.15.188:6819","nonce":1444828441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1444828441},{"type":"v1","addr":"172.21.15.188:6823","nonce":1444828441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1444828441},{"type":"v1","addr":"172.21.15.188:6821","nonce":1444828441}]},"public_addr":"172.21.15.188:6817/1444828441","cluster_addr":"172.21.15.188:6819/1444828441","heartbeat_back_addr":"172.21.15.188:6823/1444828441","heartbeat_front_addr":"172.21.15.188:6821/1444828441","state":["exists","up"]},{"osd":9,"uuid":"5bfd985c-8acc-4d00-8561-40bbd541c07b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":924440658},{"type":"v1","addr":"172.21.15.52:6827","nonce":924440658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":924440658},{"type":"v1","addr":"172.21.15.52:6829","nonce":924440658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":924440658},{"type":"v1","addr":"172.21.15.52:6833","nonce":924440658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":924440658},{"type":"v1","addr":"172.21.15.52:6831","nonce":924440658}]},"public_addr":"172.21.15.52:6827/924440658","cluster_addr":"172.21.15.52:6829/924440658","heartbeat_back_addr":"172.21.15.52:6833/924440658","heartbeat_front_addr":"172.21.15.52:6831/924440658","state":["exists","up"]},{"osd":10,"uuid":"0180cddf-e109-42cc-829a-185604276f02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6824","nonce":1531769083},{"type":"v1","addr":"172.21.15.120:6825","nonce":1531769083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":1531769083},{"type":"v1","addr":"172.21.15.120:6827","nonce":1531769083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":1531769083},{"type":"v1","addr":"172.21.15.120:6831","nonce":1531769083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6828","nonce":1531769083},{"type":"v1","addr":"172.21.15.120:6829","nonce":1531769083}]},"public_addr":"172.21.15.120:6825/1531769083","cluster_addr":"172.21.15.120:6827/1531769083","heartbeat_back_addr":"172.21.15.120:6831/1531769083","heartbeat_front_addr":"172.21.15.120:6829/1531769083","state":["exists","up"]},{"osd":11,"uuid":"73aec750-82d1-4800-b6a0-4e305912f9ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":3769050170},{"type":"v1","addr":"172.21.15.188:6825","nonce":3769050170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":3769050170},{"type":"v1","addr":"172.21.15.188:6827","nonce":3769050170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":3769050170},{"type":"v1","addr":"172.21.15.188:6831","nonce":3769050170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":3769050170},{"type":"v1","addr":"172.21.15.188:6829","nonce":3769050170}]},"public_addr":"172.21.15.188:6825/3769050170","cluster_addr":"172.21.15.188:6827/3769050170","heartbeat_back_addr":"172.21.15.188:6831/3769050170","heartbeat_front_addr":"172.21.15.188:6829/3769050170","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:44:16.455752+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:44:40.219735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:45:03.558899+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:45:29.155004+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:45:53.981098+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:46:19.412020+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:46:45.462640+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:47:11.484578+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:47:36.879054+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:48:03.392446+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:48:29.422173+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:48:55.523423+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":4}]},{"pgid":"3.6","mappings":[{"from":3,"to":7}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.9","mappings":[{"from":5,"to":8}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.52:0/3535271671":"2024-06-11T02:42:42.872312+0000","172.21.15.52:0/1092934380":"2024-06-11T02:42:42.872312+0000","172.21.15.52:6800/896317057":"2024-06-11T02:42:42.872312+0000","172.21.15.52:6801/3934479642":"2024-06-11T02:42:15.498118+0000","172.21.15.52:6800/3934479642":"2024-06-11T02:42:15.498118+0000","172.21.15.52:0/2759694132":"2024-06-11T02:42:15.498118+0000","172.21.15.52:0/2593665631":"2024-06-11T02:42:15.498118+0000","172.21.15.52:0/3151504634":"2024-06-11T02:42:15.498118+0000","172.21.15.52:6801/896317057":"2024-06-11T02:42:42.872312+0000","172.21.15.52:0/1254480982":"2024-06-11T02:42:42.872312+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-10T02:53:47.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:47.624+0000 7f0a44f04700 1 -- 172.21.15.52:0/1910976456 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f0a280a2c20 msgr2=0x7f0a280a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:53:47.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:47.624+0000 7f0a44f04700 1 --2- 172.21.15.52:0/1910976456 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f0a280a2c20 0x7f0a280a50e0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f0a3000a400 tx=0x7f0a3000d040 comp rx=0 tx=0).stop 2024-06-10T02:53:47.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:47.624+0000 7f0a44f04700 1 -- 172.21.15.52:0/1910976456 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0a40107870 msgr2=0x7f0a40198ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:53:47.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:47.624+0000 7f0a44f04700 1 --2- 172.21.15.52:0/1910976456 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0a40107870 0x7f0a40198ae0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f0a3800b550 tx=0x7f0a3800b860 comp rx=0 tx=0).stop 2024-06-10T02:53:47.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:47.624+0000 7f0a44f04700 1 -- 172.21.15.52:0/1910976456 shutdown_connections 2024-06-10T02:53:47.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:47.624+0000 7f0a44f04700 1 --2- 172.21.15.52:0/1910976456 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f0a280a2c20 0x7f0a280a50e0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:47.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:47.624+0000 7f0a44f04700 1 --2- 172.21.15.52:0/1910976456 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0a40107870 0x7f0a40198ae0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:47.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:47.624+0000 7f0a44f04700 1 --2- 172.21.15.52:0/1910976456 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0a401027a0 0x7f0a4019e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:47.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:47.624+0000 7f0a44f04700 1 --2- 172.21.15.52:0/1910976456 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0a40103150 0x7f0a4019ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:47.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:47.624+0000 7f0a44f04700 1 -- 172.21.15.52:0/1910976456 >> 172.21.15.52:0/1910976456 conn(0x7f0a40079520 msgr2=0x7f0a401089c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:53:47.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:47.624+0000 7f0a44f04700 1 -- 172.21.15.52:0/1910976456 shutdown_connections 2024-06-10T02:53:47.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:47.624+0000 7f0a44f04700 1 -- 172.21.15.52:0/1910976456 wait complete. 2024-06-10T02:53:47.976 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:53:47 smithi052 ceph-mon[32362]: from='client.? 172.21.15.52:0/1910976456' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T02:53:48.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:53:47 smithi120 ceph-mon[36634]: from='client.? 172.21.15.52:0/1910976456' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T02:53:48.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:53:47 smithi188 ceph-mon[36599]: from='client.? 172.21.15.52:0/1910976456' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T02:53:48.283 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 29, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-06-10T02:50:03.889766+0000', 'modified': '2024-06-10T02:51:11.420770+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24484, 'mds_1': 24469, 'mds_2': 24463, 'mds_3': 24449, 'mds_4': 24428}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14571': {'gid': 14571, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.52:6839/3059599042', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.52:6838', 'nonce': 3059599042}, {'type': 'v1', 'addr': '172.21.15.52:6839', 'nonce': 3059599042}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14583': {'gid': 14583, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.52:6841/3039275521', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.52:6840', 'nonce': 3039275521}, {'type': 'v1', 'addr': '172.21.15.52:6841', 'nonce': 3039275521}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24383': {'gid': 24383, 'name': 'k', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.120:6833/2609002896', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6832', 'nonce': 2609002896}, {'type': 'v1', 'addr': '172.21.15.120:6833', 'nonce': 2609002896}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24401': {'gid': 24401, 'name': 'e', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.120:6835/1996370288', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6834', 'nonce': 1996370288}, {'type': 'v1', 'addr': '172.21.15.120:6835', 'nonce': 1996370288}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24427': {'gid': 24427, 'name': 'l', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.188:6833/529564979', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6832', 'nonce': 529564979}, {'type': 'v1', 'addr': '172.21.15.188:6833', 'nonce': 529564979}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24428': {'gid': 24428, 'name': 'h', 'rank': 4, 'incarnation': 26, 'state': 'up:active', 'state_seq': 25, 'addr': '172.21.15.120:6837/4132440952', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6836', 'nonce': 4132440952}, {'type': 'v1', 'addr': '172.21.15.120:6837', 'nonce': 4132440952}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24449': {'gid': 24449, 'name': 'b', 'rank': 3, 'incarnation': 24, 'state': 'up:active', 'state_seq': 22, 'addr': '172.21.15.120:6839/1672436487', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6838', 'nonce': 1672436487}, {'type': 'v1', 'addr': '172.21.15.120:6839', 'nonce': 1672436487}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24463': {'gid': 24463, 'name': 'f', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 26, 'addr': '172.21.15.188:6835/3904544259', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6834', 'nonce': 3904544259}, {'type': 'v1', 'addr': '172.21.15.188:6835', 'nonce': 3904544259}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24469': {'gid': 24469, 'name': 'c', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 25, 'addr': '172.21.15.188:6837/819317726', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6836', 'nonce': 819317726}, {'type': 'v1', 'addr': '172.21.15.188:6837', 'nonce': 819317726}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24484': {'gid': 24484, 'name': 'i', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.188:6839/310444836', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6838', 'nonce': 310444836}, {'type': 'v1', 'addr': '172.21.15.188:6839', 'nonce': 310444836}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24484, 'qdb_cluster': [24469, 24449, 24428, 24463, 24484]} 2024-06-10T02:53:48.283 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-06-10T02:53:48.283 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph tell mds.h status 2024-06-10T02:53:48.536 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:53:48.976 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:53:48 smithi052 ceph-mon[32362]: pgmap v422: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-10T02:53:49.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:53:48 smithi120 ceph-mon[36634]: pgmap v422: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-10T02:53:49.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:53:48 smithi188 ceph-mon[36599]: pgmap v422: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-10T02:53:50.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.163+0000 7f8562b48700 1 -- 172.21.15.52:0/2270368959 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f855c102740 msgr2=0x7f855c102b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:53:50.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.163+0000 7f8562b48700 1 --2- 172.21.15.52:0/2270368959 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f855c102740 0x7f855c102b20 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f854c0099c0 tx=0x7f854c009cd0 comp rx=0 tx=0).stop 2024-06-10T02:53:50.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.163+0000 7f8562b48700 1 -- 172.21.15.52:0/2270368959 shutdown_connections 2024-06-10T02:53:50.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.163+0000 7f8562b48700 1 --2- 172.21.15.52:0/2270368959 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f855c107840 0x7f855c10f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:50.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.163+0000 7f8562b48700 1 --2- 172.21.15.52:0/2270368959 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f855c102740 0x7f855c102b20 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:50.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.163+0000 7f8562b48700 1 --2- 172.21.15.52:0/2270368959 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f855c1030f0 0x7f855c107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:50.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.163+0000 7f8562b48700 1 -- 172.21.15.52:0/2270368959 >> 172.21.15.52:0/2270368959 conn(0x7f855c0795f0 msgr2=0x7f855c079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:53:50.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.163+0000 7f8562b48700 1 -- 172.21.15.52:0/2270368959 shutdown_connections 2024-06-10T02:53:50.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.163+0000 7f8562b48700 1 -- 172.21.15.52:0/2270368959 wait complete. 2024-06-10T02:53:50.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.164+0000 7f8562b48700 1 Processor -- start 2024-06-10T02:53:50.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.164+0000 7f8562b48700 1 -- start start 2024-06-10T02:53:50.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.165+0000 7f8562b48700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f855c102740 0x7f855c19e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:50.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.165+0000 7f8562b48700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f855c1030f0 0x7f855c19ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:50.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.165+0000 7f8562b48700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f855c107840 0x7f855c1989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:50.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.165+0000 7f8562b48700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f855c112780 con 0x7f855c1030f0 2024-06-10T02:53:50.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.165+0000 7f8562b48700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f855c112600 con 0x7f855c107840 2024-06-10T02:53:50.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.165+0000 7f8562b48700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f855c112900 con 0x7f855c102740 2024-06-10T02:53:50.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.165+0000 7f855bfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f855c1030f0 0x7f855c19ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:50.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.165+0000 7f855bfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f855c1030f0 0x7f855c19ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37054/0 (socket says 172.21.15.52:37054) 2024-06-10T02:53:50.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.165+0000 7f855bfff700 1 -- 172.21.15.52:0/404419868 learned_addr learned my addr 172.21.15.52:0/404419868 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:53:50.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.165+0000 7f85608e4700 1 --2- 172.21.15.52:0/404419868 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f855c102740 0x7f855c19e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:50.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.165+0000 7f85610e5700 1 --2- 172.21.15.52:0/404419868 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f855c107840 0x7f855c1989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:50.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.165+0000 7f855bfff700 1 -- 172.21.15.52:0/404419868 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f855c102740 msgr2=0x7f855c19e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:53:50.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.165+0000 7f855bfff700 1 --2- 172.21.15.52:0/404419868 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f855c102740 0x7f855c19e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:50.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.165+0000 7f855bfff700 1 -- 172.21.15.52:0/404419868 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f855c107840 msgr2=0x7f855c1989e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:53:50.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.165+0000 7f855bfff700 1 --2- 172.21.15.52:0/404419868 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f855c107840 0x7f855c1989e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:50.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.165+0000 7f855bfff700 1 -- 172.21.15.52:0/404419868 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f855c199140 con 0x7f855c1030f0 2024-06-10T02:53:50.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.165+0000 7f85608e4700 1 --2- 172.21.15.52:0/404419868 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f855c102740 0x7f855c19e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T02:53:50.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.165+0000 7f855bfff700 1 --2- 172.21.15.52:0/404419868 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f855c1030f0 0x7f855c19ed60 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f855000d950 tx=0x7f855000dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:53:50.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.165+0000 7f8559ffb700 1 -- 172.21.15.52:0/404419868 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8550016070 con 0x7f855c1030f0 2024-06-10T02:53:50.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.166+0000 7f8559ffb700 1 -- 172.21.15.52:0/404419868 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8550009b40 con 0x7f855c1030f0 2024-06-10T02:53:50.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.166+0000 7f8562b48700 1 -- 172.21.15.52:0/404419868 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f855c199430 con 0x7f855c1030f0 2024-06-10T02:53:50.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.166+0000 7f8559ffb700 1 -- 172.21.15.52:0/404419868 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f855000f620 con 0x7f855c1030f0 2024-06-10T02:53:50.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.166+0000 7f8562b48700 1 -- 172.21.15.52:0/404419868 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f855c0746c0 con 0x7f855c1030f0 2024-06-10T02:53:50.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.168+0000 7f8559ffb700 1 -- 172.21.15.52:0/404419868 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f8550019070 con 0x7f855c1030f0 2024-06-10T02:53:50.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.169+0000 7f8559ffb700 1 --2- 172.21.15.52:0/404419868 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f85440a2ca0 0x7f85440a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:50.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.169+0000 7f8559ffb700 1 -- 172.21.15.52:0/404419868 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f85500cf3b0 con 0x7f855c1030f0 2024-06-10T02:53:50.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.169+0000 7f85608e4700 1 --2- 172.21.15.52:0/404419868 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f85440a2ca0 0x7f85440a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:50.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.169+0000 7f85608e4700 1 --2- 172.21.15.52:0/404419868 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f85440a2ca0 0x7f85440a5160 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f854c000c00 tx=0x7f854c005940 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:53:50.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.169+0000 7f856d3cbb80 1 Processor -- start 2024-06-10T02:53:50.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.169+0000 7f856d3cbb80 1 -- start start 2024-06-10T02:53:50.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.170+0000 7f856d3cbb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558fb58285f0 0x558fb58289d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:50.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.170+0000 7f856d3cbb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558fb5828f10 0x558fb5822990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:50.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.170+0000 7f856d3cbb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558fb5822ed0 0x558fb58232f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:50.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.170+0000 7f856d3cbb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x558fb57747c0 con 0x558fb5828f10 2024-06-10T02:53:50.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.170+0000 7f856d3cbb80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x558fb571c6d0 con 0x558fb5822ed0 2024-06-10T02:53:50.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.170+0000 7f856d3cbb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x558fb5770780 con 0x558fb58285f0 2024-06-10T02:53:50.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.170+0000 7f855bfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558fb5828f10 0x558fb5822990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:50.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.170+0000 7f855bfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558fb5828f10 0x558fb5822990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37064/0 (socket says 172.21.15.52:37064) 2024-06-10T02:53:50.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.170+0000 7f855bfff700 1 -- 172.21.15.52:0/1219486461 learned_addr learned my addr 172.21.15.52:0/1219486461 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:53:50.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.170+0000 7f85610e5700 1 --2- 172.21.15.52:0/1219486461 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558fb5822ed0 0x558fb58232f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:50.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.170+0000 7f85608e4700 1 --2- 172.21.15.52:0/1219486461 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558fb58285f0 0x558fb58289d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:50.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.170+0000 7f855bfff700 1 -- 172.21.15.52:0/1219486461 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558fb58285f0 msgr2=0x558fb58289d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:53:50.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.170+0000 7f855bfff700 1 --2- 172.21.15.52:0/1219486461 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558fb58285f0 0x558fb58289d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:50.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.170+0000 7f855bfff700 1 -- 172.21.15.52:0/1219486461 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558fb5822ed0 msgr2=0x558fb58232f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:53:50.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.170+0000 7f855bfff700 1 --2- 172.21.15.52:0/1219486461 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558fb5822ed0 0x558fb58232f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:50.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.170+0000 7f855bfff700 1 -- 172.21.15.52:0/1219486461 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558fb5712f00 con 0x558fb5828f10 2024-06-10T02:53:50.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.170+0000 7f855bfff700 1 --2- 172.21.15.52:0/1219486461 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558fb5828f10 0x558fb5822990 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f8550097cc0 tx=0x7f855009b290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=719fa0d38772746e server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:53:50.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.170+0000 7f8542ffd700 1 -- 172.21.15.52:0/1219486461 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8550020070 con 0x558fb5828f10 2024-06-10T02:53:50.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.171+0000 7f8542ffd700 1 -- 172.21.15.52:0/1219486461 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f855009fc10 con 0x558fb5828f10 2024-06-10T02:53:50.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.171+0000 7f8542ffd700 1 -- 172.21.15.52:0/1219486461 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85500860b0 con 0x558fb5828f10 2024-06-10T02:53:50.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.171+0000 7f856d3cbb80 1 -- 172.21.15.52:0/1219486461 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558fb5828f10 msgr2=0x558fb5822990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:53:50.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.171+0000 7f856d3cbb80 1 --2- 172.21.15.52:0/1219486461 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558fb5828f10 0x558fb5822990 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f8550097cc0 tx=0x7f855009b290 comp rx=0 tx=0).stop 2024-06-10T02:53:50.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.171+0000 7f856d3cbb80 1 -- 172.21.15.52:0/1219486461 shutdown_connections 2024-06-10T02:53:50.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.171+0000 7f856d3cbb80 1 --2- 172.21.15.52:0/1219486461 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558fb5822ed0 0x558fb58232f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:50.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.171+0000 7f856d3cbb80 1 --2- 172.21.15.52:0/1219486461 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558fb58285f0 0x558fb58289d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:50.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.171+0000 7f856d3cbb80 1 --2- 172.21.15.52:0/1219486461 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558fb5828f10 0x558fb5822990 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:50.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.171+0000 7f856d3cbb80 1 -- 172.21.15.52:0/1219486461 >> 172.21.15.52:0/1219486461 conn(0x558fb574dc70 msgr2=0x558fb581d800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:53:50.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.171+0000 7f856d3cbb80 1 -- 172.21.15.52:0/1219486461 shutdown_connections 2024-06-10T02:53:50.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.171+0000 7f856d3cbb80 1 -- 172.21.15.52:0/1219486461 wait complete. 2024-06-10T02:53:50.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.172+0000 7f856d3cbb80 1 -- start start 2024-06-10T02:53:50.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.172+0000 7f856d3cbb80 1 Processor -- start 2024-06-10T02:53:50.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.172+0000 7f856d3cbb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558fb5822ed0 0x558fb582ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:50.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.172+0000 7f856d3cbb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558fb58285f0 0x558fb582c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:50.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.172+0000 7f856d3cbb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558fb5828f10 0x558fb582c540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:50.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.172+0000 7f856d3cbb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x558fb582f920 con 0x558fb5822ed0 2024-06-10T02:53:50.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.172+0000 7f856d3cbb80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x558fb5714ba0 con 0x558fb5828f10 2024-06-10T02:53:50.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.172+0000 7f856d3cbb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x558fb582faa0 con 0x558fb58285f0 2024-06-10T02:53:50.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.172+0000 7f856d3cbb80 10 client.0 _ll_register_callbacks cb 0x558fb570e8d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T02:53:50.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.172+0000 7f85608e4700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558fb5822ed0 0x558fb582ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:50.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.172+0000 7f85608e4700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558fb5822ed0 0x558fb582ecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37066/0 (socket says 172.21.15.52:37066) 2024-06-10T02:53:50.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.172+0000 7f85608e4700 1 -- 172.21.15.52:0/3790684839 learned_addr learned my addr 172.21.15.52:0/3790684839 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:53:50.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.172+0000 7f85610e5700 1 --2- 172.21.15.52:0/3790684839 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558fb5828f10 0x558fb582c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:50.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.172+0000 7f855bfff700 1 --2- 172.21.15.52:0/3790684839 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558fb58285f0 0x558fb582c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:50.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.173+0000 7f85608e4700 1 -- 172.21.15.52:0/3790684839 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558fb58285f0 msgr2=0x558fb582c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:53:50.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.173+0000 7f85608e4700 1 --2- 172.21.15.52:0/3790684839 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558fb58285f0 0x558fb582c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:50.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.173+0000 7f85608e4700 1 -- 172.21.15.52:0/3790684839 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558fb5828f10 msgr2=0x558fb582c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:53:50.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.173+0000 7f85608e4700 1 --2- 172.21.15.52:0/3790684839 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558fb5828f10 0x558fb582c540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:50.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.173+0000 7f85608e4700 1 -- 172.21.15.52:0/3790684839 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558fb56c3880 con 0x558fb5822ed0 2024-06-10T02:53:50.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.173+0000 7f855bfff700 1 --2- 172.21.15.52:0/3790684839 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558fb58285f0 0x558fb582c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T02:53:50.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.173+0000 7f85608e4700 1 --2- 172.21.15.52:0/3790684839 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558fb5822ed0 0x558fb582ecb0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f854c0099c0 tx=0x7f854c009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=376f1a39e089feed server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:53:50.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.173+0000 7f853bfff700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-10T02:53:50.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.173+0000 7f853bfff700 1 -- 172.21.15.52:0/3790684839 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f854c003d70 con 0x558fb5822ed0 2024-06-10T02:53:50.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.173+0000 7f856d3cbb80 1 -- 172.21.15.52:0/3790684839 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f855009fc10 con 0x558fb5822ed0 2024-06-10T02:53:50.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.173+0000 7f853bfff700 1 -- 172.21.15.52:0/3790684839 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f854c00f8c0 con 0x558fb5822ed0 2024-06-10T02:53:50.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.173+0000 7f853bfff700 1 -- 172.21.15.52:0/3790684839 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f854c00fac0 con 0x558fb5822ed0 2024-06-10T02:53:50.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.173+0000 7f856d3cbb80 10 client.15189 fetch_fsmap learned FSMap version 29 2024-06-10T02:53:50.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.173+0000 7f856d3cbb80 1 -- 172.21.15.52:0/3790684839 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558fb583a2d0 con 0x558fb5822ed0 2024-06-10T02:53:50.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.173+0000 7f853bfff700 1 -- 172.21.15.52:0/3790684839 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f855009fc10 con 0x558fb5822ed0 2024-06-10T02:53:50.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.174+0000 7f853bfff700 1 -- 172.21.15.52:0/3790684839 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f854c030450 con 0x558fb5822ed0 2024-06-10T02:53:50.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.174+0000 7f856d3cbb80 10 client.15189 fetch_fsmap finished waiting for FSMap version 29 2024-06-10T02:53:50.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.174+0000 7f856d3cbb80 10 client.15189 resolve_mds: resolved name 'h' to daemon mds.h 2024-06-10T02:53:50.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.174+0000 7f856d3cbb80 20 client.15189 populate_metadata read hostname 'smithi052' 2024-06-10T02:53:50.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.174+0000 7f856d3cbb80 1 --2- 172.21.15.52:0/3790684839 >> [v2:172.21.15.120:6836/4132440952,v1:172.21.15.120:6837/4132440952] conn(0x558fb583c900 0x558fb583dd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:50.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.174+0000 7f856d3cbb80 4 client.15189 mds_command: new command op to 24428 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T02:53:50.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.174+0000 7f856d3cbb80 1 -- 172.21.15.52:0/3790684839 --> [v2:172.21.15.120:6836/4132440952,v1:172.21.15.120:6837/4132440952] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558fb5781cc0 con 0x558fb583c900 2024-06-10T02:53:50.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.174+0000 7f855bfff700 1 --2- 172.21.15.52:0/3790684839 >> [v2:172.21.15.120:6836/4132440952,v1:172.21.15.120:6837/4132440952] conn(0x558fb583c900 0x558fb583dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:50.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.174+0000 7f853bfff700 10 client.15189 ms_handle_connect on v2:172.21.15.120:6836/4132440952 2024-06-10T02:53:50.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.174+0000 7f855bfff700 1 --2- 172.21.15.52:0/3790684839 >> [v2:172.21.15.120:6836/4132440952,v1:172.21.15.120:6837/4132440952] conn(0x558fb583c900 0x558fb583dd70 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=1f33e1ad117b6650 server_cookie=84c06cded757a787 in_seq=0 out_seq=0 2024-06-10T02:53:50.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.175+0000 7f853bfff700 1 -- 172.21.15.52:0/3790684839 <== mds.4 v2:172.21.15.120:6836/4132440952 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x558fb5781cc0 con 0x558fb583c900 2024-06-10T02:53:50.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.175+0000 7f853bfff700 10 client.15189 handle_command_reply: tid=0 2024-06-10T02:53:50.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.175+0000 7f856d3cbb80 1 client.15189 shutdown 2024-06-10T02:53:50.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.175+0000 7f856d3cbb80 1 -- 172.21.15.52:0/3790684839 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558fb5822ed0 msgr2=0x558fb582ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:53:50.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.175+0000 7f856d3cbb80 1 --2- 172.21.15.52:0/3790684839 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558fb5822ed0 0x558fb582ecb0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f854c0099c0 tx=0x7f854c009cd0 comp rx=0 tx=0).stop 2024-06-10T02:53:50.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.175+0000 7f856d3cbb80 1 -- 172.21.15.52:0/3790684839 shutdown_connections 2024-06-10T02:53:50.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.175+0000 7f856d3cbb80 1 --2- 172.21.15.52:0/3790684839 >> [v2:172.21.15.120:6836/4132440952,v1:172.21.15.120:6837/4132440952] conn(0x558fb583c900 0x558fb583dd70 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:50.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.175+0000 7f856d3cbb80 1 --2- 172.21.15.52:0/3790684839 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558fb5828f10 0x558fb582c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:50.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.175+0000 7f856d3cbb80 1 --2- 172.21.15.52:0/3790684839 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558fb5822ed0 0x558fb582ecb0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:50.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.175+0000 7f856d3cbb80 1 --2- 172.21.15.52:0/3790684839 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558fb58285f0 0x558fb582c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:50.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.175+0000 7f856d3cbb80 1 -- 172.21.15.52:0/3790684839 >> 172.21.15.52:0/3790684839 conn(0x558fb574dc70 msgr2=0x558fb581d5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:53:50.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.176+0000 7f853bfff700 0 client.15189 ms_handle_reset on v2:172.21.15.120:6836/4132440952 2024-06-10T02:53:50.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.176+0000 7f856d3cbb80 1 -- 172.21.15.52:0/3790684839 shutdown_connections 2024-06-10T02:53:50.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.176+0000 7f856d3cbb80 1 -- 172.21.15.52:0/3790684839 wait complete. 2024-06-10T02:53:50.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.176+0000 7f856d3cbb80 20 client.15189 trim_cache size 0 max 16384 2024-06-10T02:53:50.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.197+0000 7f856d3cbb80 1 Processor -- start 2024-06-10T02:53:50.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.197+0000 7f856d3cbb80 1 -- start start 2024-06-10T02:53:50.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.197+0000 7f856d3cbb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558fb5822ed0 0x558fb5822990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:50.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.197+0000 7f856d3cbb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558fb58285f0 0x558fb58471d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:50.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.197+0000 7f856d3cbb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558fb5828f10 0x558fb5847710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:50.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.197+0000 7f856d3cbb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x558fb583c0b0 con 0x558fb58285f0 2024-06-10T02:53:50.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.197+0000 7f856d3cbb80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x558fb583c780 con 0x558fb5822ed0 2024-06-10T02:53:50.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.197+0000 7f856d3cbb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x558fb583ce50 con 0x558fb5828f10 2024-06-10T02:53:50.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.197+0000 7f855bfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558fb58285f0 0x558fb58471d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:50.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.198+0000 7f855bfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558fb58285f0 0x558fb58471d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37072/0 (socket says 172.21.15.52:37072) 2024-06-10T02:53:50.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.198+0000 7f855bfff700 1 -- 172.21.15.52:0/1719627840 learned_addr learned my addr 172.21.15.52:0/1719627840 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:53:50.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.198+0000 7f85608e4700 1 --2- 172.21.15.52:0/1719627840 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558fb5822ed0 0x558fb5822990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:50.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.198+0000 7f85610e5700 1 --2- 172.21.15.52:0/1719627840 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558fb5828f10 0x558fb5847710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:50.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.198+0000 7f855bfff700 1 -- 172.21.15.52:0/1719627840 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558fb5828f10 msgr2=0x558fb5847710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:53:50.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.198+0000 7f855bfff700 1 --2- 172.21.15.52:0/1719627840 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558fb5828f10 0x558fb5847710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:50.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.198+0000 7f855bfff700 1 -- 172.21.15.52:0/1719627840 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558fb5822ed0 msgr2=0x558fb5822990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:53:50.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.198+0000 7f855bfff700 1 --2- 172.21.15.52:0/1719627840 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558fb5822ed0 0x558fb5822990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:50.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.198+0000 7f855bfff700 1 -- 172.21.15.52:0/1719627840 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558fb582a570 con 0x558fb58285f0 2024-06-10T02:53:50.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.198+0000 7f85610e5700 1 --2- 172.21.15.52:0/1719627840 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558fb5828f10 0x558fb5847710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T02:53:50.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.198+0000 7f85608e4700 1 --2- 172.21.15.52:0/1719627840 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558fb5822ed0 0x558fb5822990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T02:53:50.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.198+0000 7f855bfff700 1 --2- 172.21.15.52:0/1719627840 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558fb58285f0 0x558fb58471d0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f855009b7f0 tx=0x7f85500ce040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e79f810a93276c42 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:53:50.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.198+0000 7f85427fc700 1 -- 172.21.15.52:0/1719627840 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8550020040 con 0x558fb58285f0 2024-06-10T02:53:50.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.198+0000 7f85427fc700 1 -- 172.21.15.52:0/1719627840 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8550099b60 con 0x558fb58285f0 2024-06-10T02:53:50.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.198+0000 7f85427fc700 1 -- 172.21.15.52:0/1719627840 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f855009bac0 con 0x558fb58285f0 2024-06-10T02:53:50.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.199+0000 7f856d3cbb80 1 -- 172.21.15.52:0/1719627840 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558fb58285f0 msgr2=0x558fb58471d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:53:50.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.199+0000 7f856d3cbb80 1 --2- 172.21.15.52:0/1719627840 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558fb58285f0 0x558fb58471d0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f855009b7f0 tx=0x7f85500ce040 comp rx=0 tx=0).stop 2024-06-10T02:53:50.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.199+0000 7f856d3cbb80 1 -- 172.21.15.52:0/1719627840 shutdown_connections 2024-06-10T02:53:50.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.199+0000 7f856d3cbb80 1 --2- 172.21.15.52:0/1719627840 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558fb5828f10 0x558fb5847710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:50.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.199+0000 7f856d3cbb80 1 --2- 172.21.15.52:0/1719627840 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558fb5822ed0 0x558fb5822990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:50.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.199+0000 7f856d3cbb80 1 --2- 172.21.15.52:0/1719627840 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558fb58285f0 0x558fb58471d0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:50.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.199+0000 7f856d3cbb80 1 -- 172.21.15.52:0/1719627840 >> 172.21.15.52:0/1719627840 conn(0x558fb574dc70 msgr2=0x558fb582a8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:53:50.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.199+0000 7f856d3cbb80 1 -- 172.21.15.52:0/1719627840 shutdown_connections 2024-06-10T02:53:50.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.199+0000 7f856d3cbb80 1 -- 172.21.15.52:0/1719627840 wait complete. 2024-06-10T02:53:50.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.199+0000 7f856d3cbb80 1 -- start start 2024-06-10T02:53:50.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.199+0000 7f856d3cbb80 1 Processor -- start 2024-06-10T02:53:50.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.200+0000 7f856d3cbb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558fb5822ed0 0x558fb5821d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:50.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.200+0000 7f856d3cbb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558fb58285f0 0x558fb58222c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:50.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.200+0000 7f856d3cbb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558fb5828f10 0x558fb5822800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:50.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.200+0000 7f856d3cbb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x558fb583dbf0 con 0x558fb5828f10 2024-06-10T02:53:50.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.200+0000 7f856d3cbb80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x558fb583d520 con 0x558fb5822ed0 2024-06-10T02:53:50.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.200+0000 7f856d3cbb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x558fb583e2c0 con 0x558fb58285f0 2024-06-10T02:53:50.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.200+0000 7f856d3cbb80 10 client.0 _ll_register_callbacks cb 0x558fb570e8d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T02:53:50.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.200+0000 7f85610e5700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558fb5828f10 0x558fb5822800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:50.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.200+0000 7f85610e5700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558fb5828f10 0x558fb5822800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37074/0 (socket says 172.21.15.52:37074) 2024-06-10T02:53:50.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.200+0000 7f85610e5700 1 -- 172.21.15.52:0/1067841244 learned_addr learned my addr 172.21.15.52:0/1067841244 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:53:50.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.200+0000 7f85608e4700 1 --2- 172.21.15.52:0/1067841244 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558fb5822ed0 0x558fb5821d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:50.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.200+0000 7f855bfff700 1 --2- 172.21.15.52:0/1067841244 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558fb58285f0 0x558fb58222c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:50.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.200+0000 7f85610e5700 1 -- 172.21.15.52:0/1067841244 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558fb58285f0 msgr2=0x558fb58222c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:53:50.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.200+0000 7f85610e5700 1 --2- 172.21.15.52:0/1067841244 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558fb58285f0 0x558fb58222c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:50.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.200+0000 7f85610e5700 1 -- 172.21.15.52:0/1067841244 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558fb5822ed0 msgr2=0x558fb5821d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:53:50.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.200+0000 7f85610e5700 1 --2- 172.21.15.52:0/1067841244 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558fb5822ed0 0x558fb5821d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:50.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.200+0000 7f85610e5700 1 -- 172.21.15.52:0/1067841244 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558fb5845680 con 0x558fb5828f10 2024-06-10T02:53:50.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.201+0000 7f85610e5700 1 --2- 172.21.15.52:0/1067841244 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558fb5828f10 0x558fb5822800 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f8554007a00 tx=0x7f8554007dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4730d36a904fb78 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:53:50.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.201+0000 7f853bfff700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-10T02:53:50.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.201+0000 7f853bfff700 1 -- 172.21.15.52:0/1067841244 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f855401c030 con 0x558fb5828f10 2024-06-10T02:53:50.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.201+0000 7f853bfff700 1 -- 172.21.15.52:0/1067841244 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f8554012040 con 0x558fb5828f10 2024-06-10T02:53:50.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.201+0000 7f856d3cbb80 1 -- 172.21.15.52:0/1067841244 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8550099b60 con 0x558fb5828f10 2024-06-10T02:53:50.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.201+0000 7f853bfff700 1 -- 172.21.15.52:0/1067841244 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8554029050 con 0x558fb5828f10 2024-06-10T02:53:50.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.201+0000 7f856d3cbb80 10 client.15195 fetch_fsmap learned FSMap version 29 2024-06-10T02:53:50.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.201+0000 7f856d3cbb80 1 -- 172.21.15.52:0/1067841244 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558fb58a83d0 con 0x558fb5828f10 2024-06-10T02:53:50.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.201+0000 7f853bfff700 1 -- 172.21.15.52:0/1067841244 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8554015e30 con 0x558fb5828f10 2024-06-10T02:53:50.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.202+0000 7f853bfff700 1 -- 172.21.15.52:0/1067841244 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f8554018450 con 0x558fb5828f10 2024-06-10T02:53:50.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.202+0000 7f856d3cbb80 10 client.15195 fetch_fsmap finished waiting for FSMap version 29 2024-06-10T02:53:50.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.202+0000 7f856d3cbb80 10 client.15195 resolve_mds: resolved name 'h' to daemon mds.h 2024-06-10T02:53:50.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.202+0000 7f856d3cbb80 20 client.15195 populate_metadata read hostname 'smithi052' 2024-06-10T02:53:50.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.202+0000 7f856d3cbb80 1 --2- 172.21.15.52:0/1067841244 >> [v2:172.21.15.120:6836/4132440952,v1:172.21.15.120:6837/4132440952] conn(0x558fb58acfe0 0x558fb58abcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:50.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.202+0000 7f856d3cbb80 4 client.15195 mds_command: new command op to 24428 tid=0 multi_id=0 [{"prefix": "status"}] 2024-06-10T02:53:50.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.202+0000 7f856d3cbb80 1 -- 172.21.15.52:0/1067841244 --> [v2:172.21.15.120:6836/4132440952,v1:172.21.15.120:6837/4132440952] -- command(tid 0: {"prefix": "status"}) -- 0x7f8550099b60 con 0x558fb58acfe0 2024-06-10T02:53:50.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.202+0000 7f85608e4700 1 --2- 172.21.15.52:0/1067841244 >> [v2:172.21.15.120:6836/4132440952,v1:172.21.15.120:6837/4132440952] conn(0x558fb58acfe0 0x558fb58abcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:50.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.203+0000 7f85608e4700 1 --2- 172.21.15.52:0/1067841244 >> [v2:172.21.15.120:6836/4132440952,v1:172.21.15.120:6837/4132440952] conn(0x558fb58acfe0 0x558fb58abcf0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=2cdcbf60edbea65b server_cookie=a20f39b4a2f07d8 in_seq=0 out_seq=0 2024-06-10T02:53:50.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.203+0000 7f853bfff700 10 client.15195 ms_handle_connect on v2:172.21.15.120:6836/4132440952 2024-06-10T02:53:50.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.203+0000 7f853bfff700 1 -- 172.21.15.52:0/1067841244 <== mds.4 v2:172.21.15.120:6836/4132440952 1 ==== command_reply(tid 0: 0 ) ==== 8+0+333 (crc 0 0 0) 0x7f8550099b60 con 0x558fb58acfe0 2024-06-10T02:53:50.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.203+0000 7f853bfff700 10 client.15195 handle_command_reply: tid=0 2024-06-10T02:53:50.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.203+0000 7f856d3cbb80 1 client.15195 shutdown 2024-06-10T02:53:50.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.203+0000 7f856d3cbb80 1 -- 172.21.15.52:0/1067841244 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558fb5828f10 msgr2=0x558fb5822800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:53:50.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.203+0000 7f856d3cbb80 1 --2- 172.21.15.52:0/1067841244 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558fb5828f10 0x558fb5822800 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f8554007a00 tx=0x7f8554007dc0 comp rx=0 tx=0).stop 2024-06-10T02:53:50.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.203+0000 7f856d3cbb80 1 -- 172.21.15.52:0/1067841244 shutdown_connections 2024-06-10T02:53:50.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.203+0000 7f856d3cbb80 1 --2- 172.21.15.52:0/1067841244 >> [v2:172.21.15.120:6836/4132440952,v1:172.21.15.120:6837/4132440952] conn(0x558fb58acfe0 0x558fb58abcf0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:50.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.203+0000 7f856d3cbb80 1 --2- 172.21.15.52:0/1067841244 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558fb5828f10 0x558fb5822800 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:50.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.203+0000 7f856d3cbb80 1 --2- 172.21.15.52:0/1067841244 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558fb5822ed0 0x558fb5821d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:50.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.203+0000 7f853bfff700 0 client.15195 ms_handle_reset on v2:172.21.15.120:6836/4132440952 2024-06-10T02:53:50.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.203+0000 7f856d3cbb80 1 --2- 172.21.15.52:0/1067841244 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558fb58285f0 0x558fb58222c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:50.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.203+0000 7f856d3cbb80 1 -- 172.21.15.52:0/1067841244 >> 172.21.15.52:0/1067841244 conn(0x558fb574dc70 msgr2=0x558fb5816db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:53:50.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.204+0000 7f856d3cbb80 1 -- 172.21.15.52:0/1067841244 shutdown_connections 2024-06-10T02:53:50.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.204+0000 7f856d3cbb80 1 -- 172.21.15.52:0/1067841244 wait complete. 2024-06-10T02:53:50.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.204+0000 7f856d3cbb80 20 client.15195 trim_cache size 0 max 16384 2024-06-10T02:53:50.208 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-10T02:53:50.208 INFO:teuthology.orchestra.run.smithi052.stdout: "cluster_fsid": "d8676e9e-26d2-11ef-bc9b-c7b262605968", 2024-06-10T02:53:50.208 INFO:teuthology.orchestra.run.smithi052.stdout: "whoami": 4, 2024-06-10T02:53:50.208 INFO:teuthology.orchestra.run.smithi052.stdout: "id": 24428, 2024-06-10T02:53:50.209 INFO:teuthology.orchestra.run.smithi052.stdout: "want_state": "up:active", 2024-06-10T02:53:50.209 INFO:teuthology.orchestra.run.smithi052.stdout: "state": "up:active", 2024-06-10T02:53:50.209 INFO:teuthology.orchestra.run.smithi052.stdout: "fs_name": "cephfs", 2024-06-10T02:53:50.209 INFO:teuthology.orchestra.run.smithi052.stdout: "rank_uptime": 161.58541273599999, 2024-06-10T02:53:50.209 INFO:teuthology.orchestra.run.smithi052.stdout: "mdsmap_epoch": 29, 2024-06-10T02:53:50.209 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_epoch": 100, 2024-06-10T02:53:50.209 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_epoch_barrier": 98, 2024-06-10T02:53:50.209 INFO:teuthology.orchestra.run.smithi052.stdout: "uptime": 256.85150262600001 2024-06-10T02:53:50.209 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-10T02:53:50.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.204+0000 7f8562b48700 1 -- 172.21.15.52:0/404419868 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f85440a2ca0 msgr2=0x7f85440a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:53:50.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.204+0000 7f8562b48700 1 --2- 172.21.15.52:0/404419868 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f85440a2ca0 0x7f85440a5160 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f854c000c00 tx=0x7f854c005940 comp rx=0 tx=0).stop 2024-06-10T02:53:50.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.204+0000 7f8562b48700 1 -- 172.21.15.52:0/404419868 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f855c1030f0 msgr2=0x7f855c19ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:53:50.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.204+0000 7f8562b48700 1 --2- 172.21.15.52:0/404419868 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f855c1030f0 0x7f855c19ed60 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f855000d950 tx=0x7f855000dd10 comp rx=0 tx=0).stop 2024-06-10T02:53:50.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.204+0000 7f8562b48700 1 -- 172.21.15.52:0/404419868 shutdown_connections 2024-06-10T02:53:50.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.205+0000 7f8562b48700 1 --2- 172.21.15.52:0/404419868 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f855c107840 0x7f855c1989e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:50.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.205+0000 7f8562b48700 1 --2- 172.21.15.52:0/404419868 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f855c102740 0x7f855c19e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:50.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.205+0000 7f8562b48700 1 --2- 172.21.15.52:0/404419868 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f85440a2ca0 0x7f85440a5160 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:50.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.205+0000 7f8562b48700 1 --2- 172.21.15.52:0/404419868 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f855c1030f0 0x7f855c19ed60 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:50.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.205+0000 7f8562b48700 1 -- 172.21.15.52:0/404419868 >> 172.21.15.52:0/404419868 conn(0x7f855c0795f0 msgr2=0x7f855c1006d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:53:50.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.205+0000 7f8562b48700 1 -- 172.21.15.52:0/404419868 shutdown_connections 2024-06-10T02:53:50.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:50.205+0000 7f8562b48700 1 -- 172.21.15.52:0/404419868 wait complete. 2024-06-10T02:53:50.844 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:53:50 smithi052 ceph-mon[32362]: pgmap v423: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-10T02:53:50.875 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph tell mds.b status 2024-06-10T02:53:51.131 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:53:51.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:53:50 smithi120 ceph-mon[36634]: pgmap v423: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-10T02:53:51.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:53:50 smithi188 ceph-mon[36599]: pgmap v423: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-10T02:53:52.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.752+0000 7f0221cea700 1 -- 172.21.15.52:0/678085523 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f021c1030f0 msgr2=0x7f021c107110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:53:52.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.752+0000 7f0221cea700 1 --2- 172.21.15.52:0/678085523 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f021c1030f0 0x7f021c107110 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f02040099c0 tx=0x7f0204009cd0 comp rx=0 tx=0).stop 2024-06-10T02:53:52.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.752+0000 7f0221cea700 1 -- 172.21.15.52:0/678085523 shutdown_connections 2024-06-10T02:53:52.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.752+0000 7f0221cea700 1 --2- 172.21.15.52:0/678085523 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f021c107840 0x7f021c10f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:52.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.752+0000 7f0221cea700 1 --2- 172.21.15.52:0/678085523 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f021c102740 0x7f021c102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:52.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.752+0000 7f0221cea700 1 --2- 172.21.15.52:0/678085523 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f021c1030f0 0x7f021c107110 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:52.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.752+0000 7f0221cea700 1 -- 172.21.15.52:0/678085523 >> 172.21.15.52:0/678085523 conn(0x7f021c0795f0 msgr2=0x7f021c079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:53:52.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.753+0000 7f0221cea700 1 -- 172.21.15.52:0/678085523 shutdown_connections 2024-06-10T02:53:52.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.753+0000 7f0221cea700 1 -- 172.21.15.52:0/678085523 wait complete. 2024-06-10T02:53:52.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.753+0000 7f0221cea700 1 Processor -- start 2024-06-10T02:53:52.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.753+0000 7f0221cea700 1 -- start start 2024-06-10T02:53:52.755 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.754+0000 7f0221cea700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f021c102740 0x7f021c19e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:52.755 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.754+0000 7f0221cea700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f021c1030f0 0x7f021c19ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:52.755 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.754+0000 7f0221cea700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f021c107840 0x7f021c1989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:52.755 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.754+0000 7f0221cea700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f021c112750 con 0x7f021c102740 2024-06-10T02:53:52.755 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.754+0000 7f0221cea700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f021c1125d0 con 0x7f021c107840 2024-06-10T02:53:52.755 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.754+0000 7f0221cea700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f021c1128d0 con 0x7f021c1030f0 2024-06-10T02:53:52.755 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.754+0000 7f021b7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f021c102740 0x7f021c19e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:52.755 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.754+0000 7f021b7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f021c102740 0x7f021c19e810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:41942/0 (socket says 172.21.15.52:41942) 2024-06-10T02:53:52.755 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.754+0000 7f021b7fe700 1 -- 172.21.15.52:0/3229231518 learned_addr learned my addr 172.21.15.52:0/3229231518 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:53:52.755 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.754+0000 7f021bfff700 1 --2- 172.21.15.52:0/3229231518 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f021c107840 0x7f021c1989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:52.756 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.754+0000 7f0212ffd700 1 --2- 172.21.15.52:0/3229231518 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f021c1030f0 0x7f021c19ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:52.756 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.754+0000 7f021b7fe700 1 -- 172.21.15.52:0/3229231518 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f021c1030f0 msgr2=0x7f021c19ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:53:52.756 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.754+0000 7f021b7fe700 1 --2- 172.21.15.52:0/3229231518 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f021c1030f0 0x7f021c19ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:52.756 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.754+0000 7f021b7fe700 1 -- 172.21.15.52:0/3229231518 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f021c107840 msgr2=0x7f021c1989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:53:52.756 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.754+0000 7f021b7fe700 1 --2- 172.21.15.52:0/3229231518 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f021c107840 0x7f021c1989d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:52.756 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.754+0000 7f021b7fe700 1 -- 172.21.15.52:0/3229231518 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f021c199130 con 0x7f021c102740 2024-06-10T02:53:52.756 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.754+0000 7f0212ffd700 1 --2- 172.21.15.52:0/3229231518 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f021c1030f0 0x7f021c19ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T02:53:52.756 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.754+0000 7f021b7fe700 1 --2- 172.21.15.52:0/3229231518 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f021c102740 0x7f021c19e810 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f020c00d980 tx=0x7f020c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:53:52.756 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.754+0000 7f02197fa700 1 -- 172.21.15.52:0/3229231518 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f020c016070 con 0x7f021c102740 2024-06-10T02:53:52.756 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.755+0000 7f0221cea700 1 -- 172.21.15.52:0/3229231518 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f021c199420 con 0x7f021c102740 2024-06-10T02:53:52.756 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.755+0000 7f02197fa700 1 -- 172.21.15.52:0/3229231518 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f020c004d10 con 0x7f021c102740 2024-06-10T02:53:52.756 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.755+0000 7f02197fa700 1 -- 172.21.15.52:0/3229231518 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f020c0055f0 con 0x7f021c102740 2024-06-10T02:53:52.756 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.755+0000 7f0221cea700 1 -- 172.21.15.52:0/3229231518 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f021c1a56e0 con 0x7f021c102740 2024-06-10T02:53:52.757 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.757+0000 7f02197fa700 1 -- 172.21.15.52:0/3229231518 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f020c004750 con 0x7f021c102740 2024-06-10T02:53:52.757 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.758+0000 7f02197fa700 1 --2- 172.21.15.52:0/3229231518 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f02080a2c20 0x7f02080a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:52.758 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.758+0000 7f0212ffd700 1 --2- 172.21.15.52:0/3229231518 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f02080a2c20 0x7f02080a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:52.758 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.758+0000 7f02197fa700 1 -- 172.21.15.52:0/3229231518 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f020c0cf300 con 0x7f021c102740 2024-06-10T02:53:52.758 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.758+0000 7f0212ffd700 1 --2- 172.21.15.52:0/3229231518 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f02080a2c20 0x7f02080a50e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f020400b580 tx=0x7f0204009f30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:53:52.758 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.759+0000 7f022c56db80 1 Processor -- start 2024-06-10T02:53:52.758 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.759+0000 7f022c56db80 1 -- start start 2024-06-10T02:53:52.759 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.759+0000 7f022c56db80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562219cf9320 0x562219cf9700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:52.759 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.759+0000 7f022c56db80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562219cf9c40 0x562219cf3620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:52.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.759+0000 7f022c56db80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562219cf3b60 0x562219cf3fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:52.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.759+0000 7f022c56db80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x562219be1f10 con 0x562219cf9320 2024-06-10T02:53:52.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.759+0000 7f022c56db80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x562219c40800 con 0x562219cf3b60 2024-06-10T02:53:52.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.759+0000 7f022c56db80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x562219ba8410 con 0x562219cf9c40 2024-06-10T02:53:52.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.759+0000 7f021b7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562219cf9320 0x562219cf9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:52.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.759+0000 7f021b7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562219cf9320 0x562219cf9700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:41956/0 (socket says 172.21.15.52:41956) 2024-06-10T02:53:52.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.759+0000 7f021b7fe700 1 -- 172.21.15.52:0/1220796810 learned_addr learned my addr 172.21.15.52:0/1220796810 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:53:52.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.759+0000 7f021bfff700 1 --2- 172.21.15.52:0/1220796810 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562219cf3b60 0x562219cf3fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:52.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.759+0000 7f0212ffd700 1 --2- 172.21.15.52:0/1220796810 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562219cf9c40 0x562219cf3620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:52.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.759+0000 7f021b7fe700 1 -- 172.21.15.52:0/1220796810 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562219cf9c40 msgr2=0x562219cf3620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:53:52.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.759+0000 7f021b7fe700 1 --2- 172.21.15.52:0/1220796810 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562219cf9c40 0x562219cf3620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:52.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.759+0000 7f021b7fe700 1 -- 172.21.15.52:0/1220796810 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562219cf3b60 msgr2=0x562219cf3fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:53:52.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.759+0000 7f021b7fe700 1 --2- 172.21.15.52:0/1220796810 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562219cf3b60 0x562219cf3fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:52.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.759+0000 7f021b7fe700 1 -- 172.21.15.52:0/1220796810 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562219b8f880 con 0x562219cf9320 2024-06-10T02:53:52.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.759+0000 7f0212ffd700 1 --2- 172.21.15.52:0/1220796810 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562219cf9c40 0x562219cf3620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T02:53:52.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.759+0000 7f021bfff700 1 --2- 172.21.15.52:0/1220796810 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562219cf3b60 0x562219cf3fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T02:53:52.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.760+0000 7f021b7fe700 1 --2- 172.21.15.52:0/1220796810 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562219cf9320 0x562219cf9700 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f020c09b740 tx=0x7f020c09b210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5ec3331147d415 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:53:52.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.760+0000 7f0211ffb700 1 -- 172.21.15.52:0/1220796810 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f020c09fb60 con 0x562219cf9320 2024-06-10T02:53:52.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.760+0000 7f0211ffb700 1 -- 172.21.15.52:0/1220796810 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f020c0a4420 con 0x562219cf9320 2024-06-10T02:53:52.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.760+0000 7f0211ffb700 1 -- 172.21.15.52:0/1220796810 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f020c00fc70 con 0x562219cf9320 2024-06-10T02:53:52.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.760+0000 7f022c56db80 1 -- 172.21.15.52:0/1220796810 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562219cf9320 msgr2=0x562219cf9700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:53:52.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.760+0000 7f022c56db80 1 --2- 172.21.15.52:0/1220796810 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562219cf9320 0x562219cf9700 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f020c09b740 tx=0x7f020c09b210 comp rx=0 tx=0).stop 2024-06-10T02:53:52.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.760+0000 7f022c56db80 1 -- 172.21.15.52:0/1220796810 shutdown_connections 2024-06-10T02:53:52.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.760+0000 7f022c56db80 1 --2- 172.21.15.52:0/1220796810 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562219cf3b60 0x562219cf3fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:52.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.760+0000 7f022c56db80 1 --2- 172.21.15.52:0/1220796810 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562219cf9320 0x562219cf9700 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:52.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.760+0000 7f022c56db80 1 --2- 172.21.15.52:0/1220796810 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562219cf9c40 0x562219cf3620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:52.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.760+0000 7f022c56db80 1 -- 172.21.15.52:0/1220796810 >> 172.21.15.52:0/1220796810 conn(0x562219c1c030 msgr2=0x562219cea1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:53:52.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.761+0000 7f022c56db80 1 -- 172.21.15.52:0/1220796810 shutdown_connections 2024-06-10T02:53:52.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.761+0000 7f022c56db80 1 -- 172.21.15.52:0/1220796810 wait complete. 2024-06-10T02:53:52.763 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.761+0000 7f022c56db80 1 -- start start 2024-06-10T02:53:52.763 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.761+0000 7f022c56db80 1 Processor -- start 2024-06-10T02:53:52.763 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.761+0000 7f022c56db80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562219cf3b60 0x562219cff9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:52.763 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.761+0000 7f022c56db80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562219cf9320 0x562219cfccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:52.763 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.761+0000 7f022c56db80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562219cf9c40 0x562219cfd230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:52.763 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.761+0000 7f022c56db80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x562219be0f70 con 0x562219cf9c40 2024-06-10T02:53:52.763 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.761+0000 7f022c56db80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x562219c3c7c0 con 0x562219cf3b60 2024-06-10T02:53:52.764 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.761+0000 7f022c56db80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x562219d005d0 con 0x562219cf9320 2024-06-10T02:53:52.764 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.762+0000 7f022c56db80 10 client.0 _ll_register_callbacks cb 0x562219bd6560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T02:53:52.764 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.762+0000 7f021bfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562219cf9c40 0x562219cfd230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:52.764 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.762+0000 7f021bfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562219cf9c40 0x562219cfd230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:41970/0 (socket says 172.21.15.52:41970) 2024-06-10T02:53:52.764 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.762+0000 7f021bfff700 1 -- 172.21.15.52:0/856127728 learned_addr learned my addr 172.21.15.52:0/856127728 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:53:52.764 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.762+0000 7f021b7fe700 1 --2- 172.21.15.52:0/856127728 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562219cf3b60 0x562219cff9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:52.764 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.762+0000 7f0212ffd700 1 --2- 172.21.15.52:0/856127728 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562219cf9320 0x562219cfccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:52.764 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.762+0000 7f021bfff700 1 -- 172.21.15.52:0/856127728 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562219cf9320 msgr2=0x562219cfccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:53:52.764 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.762+0000 7f021bfff700 1 --2- 172.21.15.52:0/856127728 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562219cf9320 0x562219cfccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:52.764 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.762+0000 7f021bfff700 1 -- 172.21.15.52:0/856127728 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562219cf3b60 msgr2=0x562219cff9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:53:52.765 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.762+0000 7f021bfff700 1 --2- 172.21.15.52:0/856127728 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562219cf3b60 0x562219cff9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:52.765 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.762+0000 7f021bfff700 1 -- 172.21.15.52:0/856127728 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562219c06700 con 0x562219cf9c40 2024-06-10T02:53:52.765 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:53:52 smithi052 ceph-mon[32362]: pgmap v424: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-10T02:53:52.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.762+0000 7f021bfff700 1 --2- 172.21.15.52:0/856127728 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562219cf9c40 0x562219cfd230 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f0214009fc0 tx=0x7f0214003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a4a9eed28e790e7 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:53:52.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.762+0000 7f01faffd700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-10T02:53:52.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.762+0000 7f01faffd700 1 -- 172.21.15.52:0/856127728 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f021401a020 con 0x562219cf9c40 2024-06-10T02:53:52.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.762+0000 7f01faffd700 1 -- 172.21.15.52:0/856127728 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f0214007910 con 0x562219cf9c40 2024-06-10T02:53:52.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.762+0000 7f022c56db80 1 -- 172.21.15.52:0/856127728 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f020c0a4420 con 0x562219cf9c40 2024-06-10T02:53:52.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.763+0000 7f01faffd700 1 -- 172.21.15.52:0/856127728 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f0214004bb0 con 0x562219cf9c40 2024-06-10T02:53:52.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.763+0000 7f022c56db80 10 client.15207 fetch_fsmap learned FSMap version 29 2024-06-10T02:53:52.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.763+0000 7f022c56db80 1 -- 172.21.15.52:0/856127728 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x562219d0adf0 con 0x562219cf9c40 2024-06-10T02:53:52.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.763+0000 7f01faffd700 1 -- 172.21.15.52:0/856127728 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f020c0a4420 con 0x562219cf9c40 2024-06-10T02:53:52.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.763+0000 7f01faffd700 1 -- 172.21.15.52:0/856127728 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f0214016b40 con 0x562219cf9c40 2024-06-10T02:53:52.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.763+0000 7f022c56db80 10 client.15207 fetch_fsmap finished waiting for FSMap version 29 2024-06-10T02:53:52.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.763+0000 7f022c56db80 10 client.15207 resolve_mds: resolved name 'b' to daemon mds.b 2024-06-10T02:53:52.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.763+0000 7f022c56db80 20 client.15207 populate_metadata read hostname 'smithi052' 2024-06-10T02:53:52.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.763+0000 7f022c56db80 1 --2- 172.21.15.52:0/856127728 >> [v2:172.21.15.120:6838/1672436487,v1:172.21.15.120:6839/1672436487] conn(0x562219d0d460 0x562219d0e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:52.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.763+0000 7f022c56db80 4 client.15207 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T02:53:52.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.763+0000 7f022c56db80 1 -- 172.21.15.52:0/856127728 --> [v2:172.21.15.120:6838/1672436487,v1:172.21.15.120:6839/1672436487] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562219c4dd00 con 0x562219d0d460 2024-06-10T02:53:52.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.764+0000 7f021b7fe700 1 --2- 172.21.15.52:0/856127728 >> [v2:172.21.15.120:6838/1672436487,v1:172.21.15.120:6839/1672436487] conn(0x562219d0d460 0x562219d0e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:52.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.764+0000 7f01faffd700 10 client.15207 ms_handle_connect on v2:172.21.15.120:6838/1672436487 2024-06-10T02:53:52.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.764+0000 7f021b7fe700 1 --2- 172.21.15.52:0/856127728 >> [v2:172.21.15.120:6838/1672436487,v1:172.21.15.120:6839/1672436487] conn(0x562219d0d460 0x562219d0e8d0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=542cbf823af0802a server_cookie=49f7caf52e7d5c4a in_seq=0 out_seq=0 2024-06-10T02:53:52.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.765+0000 7f01faffd700 1 -- 172.21.15.52:0/856127728 <== mds.3 v2:172.21.15.120:6838/1672436487 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x562219c4dd00 con 0x562219d0d460 2024-06-10T02:53:52.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.765+0000 7f01faffd700 10 client.15207 handle_command_reply: tid=0 2024-06-10T02:53:52.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.765+0000 7f022c56db80 1 client.15207 shutdown 2024-06-10T02:53:52.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.765+0000 7f022c56db80 1 -- 172.21.15.52:0/856127728 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562219cf9c40 msgr2=0x562219cfd230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:53:52.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.765+0000 7f022c56db80 1 --2- 172.21.15.52:0/856127728 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562219cf9c40 0x562219cfd230 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f0214009fc0 tx=0x7f0214003f90 comp rx=0 tx=0).stop 2024-06-10T02:53:52.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.765+0000 7f022c56db80 1 -- 172.21.15.52:0/856127728 shutdown_connections 2024-06-10T02:53:52.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.765+0000 7f022c56db80 1 --2- 172.21.15.52:0/856127728 >> [v2:172.21.15.120:6838/1672436487,v1:172.21.15.120:6839/1672436487] conn(0x562219d0d460 0x562219d0e8d0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:52.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.765+0000 7f022c56db80 1 --2- 172.21.15.52:0/856127728 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562219cf9c40 0x562219cfd230 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:52.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.765+0000 7f022c56db80 1 --2- 172.21.15.52:0/856127728 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562219cf3b60 0x562219cff9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:52.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.765+0000 7f01faffd700 0 client.15207 ms_handle_reset on v2:172.21.15.120:6838/1672436487 2024-06-10T02:53:52.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.765+0000 7f022c56db80 1 --2- 172.21.15.52:0/856127728 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562219cf9320 0x562219cfccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:52.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.765+0000 7f022c56db80 1 -- 172.21.15.52:0/856127728 >> 172.21.15.52:0/856127728 conn(0x562219c1c030 msgr2=0x562219ce96b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:53:52.770 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.765+0000 7f022c56db80 1 -- 172.21.15.52:0/856127728 shutdown_connections 2024-06-10T02:53:52.770 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.765+0000 7f022c56db80 1 -- 172.21.15.52:0/856127728 wait complete. 2024-06-10T02:53:52.770 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.765+0000 7f022c56db80 20 client.15207 trim_cache size 0 max 16384 2024-06-10T02:53:52.794 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.787+0000 7f022c56db80 1 Processor -- start 2024-06-10T02:53:52.794 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.787+0000 7f022c56db80 1 -- start start 2024-06-10T02:53:52.794 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.787+0000 7f022c56db80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562219cf3b60 0x562219cfd3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:52.794 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.787+0000 7f022c56db80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562219cf9320 0x562219cff530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:52.794 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.787+0000 7f022c56db80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562219cf9c40 0x562219ce9e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:52.794 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.787+0000 7f022c56db80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x562219d0cc10 con 0x562219cf9c40 2024-06-10T02:53:52.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.787+0000 7f022c56db80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x562219d0d2e0 con 0x562219cf9320 2024-06-10T02:53:52.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.787+0000 7f022c56db80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x562219d0d9b0 con 0x562219cf3b60 2024-06-10T02:53:52.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.787+0000 7f021bfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562219cf9c40 0x562219ce9e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:52.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.787+0000 7f021bfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562219cf9c40 0x562219ce9e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:41978/0 (socket says 172.21.15.52:41978) 2024-06-10T02:53:52.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.787+0000 7f021bfff700 1 -- 172.21.15.52:0/4294830820 learned_addr learned my addr 172.21.15.52:0/4294830820 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:53:52.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.787+0000 7f0212ffd700 1 --2- 172.21.15.52:0/4294830820 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562219cf9320 0x562219cff530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:52.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.787+0000 7f021b7fe700 1 --2- 172.21.15.52:0/4294830820 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562219cf3b60 0x562219cfd3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:52.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.787+0000 7f021bfff700 1 -- 172.21.15.52:0/4294830820 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562219cf3b60 msgr2=0x562219cfd3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:53:52.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.787+0000 7f021bfff700 1 --2- 172.21.15.52:0/4294830820 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562219cf3b60 0x562219cfd3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:52.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.787+0000 7f021bfff700 1 -- 172.21.15.52:0/4294830820 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562219cf9320 msgr2=0x562219cff530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:53:52.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.787+0000 7f021bfff700 1 --2- 172.21.15.52:0/4294830820 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562219cf9320 0x562219cff530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:52.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.787+0000 7f021bfff700 1 -- 172.21.15.52:0/4294830820 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562219d0f830 con 0x562219cf9c40 2024-06-10T02:53:52.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.787+0000 7f0212ffd700 1 --2- 172.21.15.52:0/4294830820 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562219cf9320 0x562219cff530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T02:53:52.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.787+0000 7f021b7fe700 1 --2- 172.21.15.52:0/4294830820 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562219cf3b60 0x562219cfd3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T02:53:52.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.788+0000 7f021bfff700 1 --2- 172.21.15.52:0/4294830820 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562219cf9c40 0x562219ce9e30 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f02140095c0 tx=0x7f02140095f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb8fb37fc068372d server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:53:52.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.788+0000 7f02117fa700 1 -- 172.21.15.52:0/4294830820 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f021402d020 con 0x562219cf9c40 2024-06-10T02:53:52.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.788+0000 7f02117fa700 1 -- 172.21.15.52:0/4294830820 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0214005e60 con 0x562219cf9c40 2024-06-10T02:53:52.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.788+0000 7f02117fa700 1 -- 172.21.15.52:0/4294830820 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f021401a070 con 0x562219cf9c40 2024-06-10T02:53:52.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.788+0000 7f022c56db80 1 -- 172.21.15.52:0/4294830820 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562219cf9c40 msgr2=0x562219ce9e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:53:52.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.788+0000 7f022c56db80 1 --2- 172.21.15.52:0/4294830820 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562219cf9c40 0x562219ce9e30 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f02140095c0 tx=0x7f02140095f0 comp rx=0 tx=0).stop 2024-06-10T02:53:52.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.789+0000 7f022c56db80 1 -- 172.21.15.52:0/4294830820 shutdown_connections 2024-06-10T02:53:52.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.789+0000 7f022c56db80 1 --2- 172.21.15.52:0/4294830820 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562219cf9c40 0x562219ce9e30 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:52.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.789+0000 7f022c56db80 1 --2- 172.21.15.52:0/4294830820 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562219cf3b60 0x562219cfd3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:52.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.789+0000 7f022c56db80 1 --2- 172.21.15.52:0/4294830820 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562219cf9320 0x562219cff530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:52.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.789+0000 7f022c56db80 1 -- 172.21.15.52:0/4294830820 >> 172.21.15.52:0/4294830820 conn(0x562219c1c030 msgr2=0x562219cfccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:53:52.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.789+0000 7f022c56db80 1 -- 172.21.15.52:0/4294830820 shutdown_connections 2024-06-10T02:53:52.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.789+0000 7f022c56db80 1 -- 172.21.15.52:0/4294830820 wait complete. 2024-06-10T02:53:52.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.789+0000 7f022c56db80 1 -- start start 2024-06-10T02:53:52.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.789+0000 7f022c56db80 1 Processor -- start 2024-06-10T02:53:52.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.789+0000 7f022c56db80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562219cf3b60 0x562219d17d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:52.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.789+0000 7f022c56db80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562219cf9320 0x562219d18290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:52.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.789+0000 7f022c56db80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562219cf9c40 0x562219d12d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:52.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.789+0000 7f022c56db80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x562219d0e750 con 0x562219cf9c40 2024-06-10T02:53:52.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.789+0000 7f022c56db80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x562219d0e080 con 0x562219cf9320 2024-06-10T02:53:52.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.789+0000 7f022c56db80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x562219d0ee20 con 0x562219cf3b60 2024-06-10T02:53:52.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.790+0000 7f022c56db80 10 client.0 _ll_register_callbacks cb 0x562219bd6560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T02:53:52.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.790+0000 7f021bfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562219cf9c40 0x562219d12d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:52.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.790+0000 7f021bfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562219cf9c40 0x562219d12d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:41984/0 (socket says 172.21.15.52:41984) 2024-06-10T02:53:52.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.790+0000 7f021bfff700 1 -- 172.21.15.52:0/4279875779 learned_addr learned my addr 172.21.15.52:0/4279875779 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:53:52.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.790+0000 7f0212ffd700 1 --2- 172.21.15.52:0/4279875779 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562219cf9320 0x562219d18290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:52.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.790+0000 7f021b7fe700 1 --2- 172.21.15.52:0/4279875779 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562219cf3b60 0x562219d17d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:52.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.790+0000 7f021bfff700 1 -- 172.21.15.52:0/4279875779 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562219cf3b60 msgr2=0x562219d17d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:53:52.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.790+0000 7f021bfff700 1 --2- 172.21.15.52:0/4279875779 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562219cf3b60 0x562219d17d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:52.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.790+0000 7f021bfff700 1 -- 172.21.15.52:0/4279875779 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562219cf9320 msgr2=0x562219d18290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:53:52.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.790+0000 7f021bfff700 1 --2- 172.21.15.52:0/4279875779 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562219cf9320 0x562219d18290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:52.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.790+0000 7f021bfff700 1 -- 172.21.15.52:0/4279875779 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562219d13240 con 0x562219cf9c40 2024-06-10T02:53:52.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.790+0000 7f0212ffd700 1 --2- 172.21.15.52:0/4279875779 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562219cf9320 0x562219d18290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T02:53:52.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.790+0000 7f021bfff700 1 --2- 172.21.15.52:0/4279875779 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562219cf9c40 0x562219d12d00 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f0214000f80 tx=0x7f0214007ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d59d7449521ffb6 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:53:52.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.790+0000 7f01faffd700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-10T02:53:52.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.790+0000 7f01faffd700 1 -- 172.21.15.52:0/4279875779 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f021401a070 con 0x562219cf9c40 2024-06-10T02:53:52.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.790+0000 7f01faffd700 1 -- 172.21.15.52:0/4279875779 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f0214005500 con 0x562219cf9c40 2024-06-10T02:53:52.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.790+0000 7f022c56db80 1 -- 172.21.15.52:0/4279875779 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0214005e60 con 0x562219cf9c40 2024-06-10T02:53:52.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.790+0000 7f01faffd700 1 -- 172.21.15.52:0/4279875779 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f021402d030 con 0x562219cf9c40 2024-06-10T02:53:52.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.790+0000 7f022c56db80 10 client.15213 fetch_fsmap learned FSMap version 29 2024-06-10T02:53:52.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.791+0000 7f022c56db80 1 -- 172.21.15.52:0/4279875779 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x562219d79460 con 0x562219cf9c40 2024-06-10T02:53:52.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.791+0000 7f01faffd700 1 -- 172.21.15.52:0/4279875779 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0214005e60 con 0x562219cf9c40 2024-06-10T02:53:52.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.791+0000 7f01faffd700 1 -- 172.21.15.52:0/4279875779 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f0214029480 con 0x562219cf9c40 2024-06-10T02:53:52.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.791+0000 7f022c56db80 10 client.15213 fetch_fsmap finished waiting for FSMap version 29 2024-06-10T02:53:52.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.791+0000 7f022c56db80 10 client.15213 resolve_mds: resolved name 'b' to daemon mds.b 2024-06-10T02:53:52.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.791+0000 7f022c56db80 20 client.15213 populate_metadata read hostname 'smithi052' 2024-06-10T02:53:52.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.791+0000 7f022c56db80 1 --2- 172.21.15.52:0/4279875779 >> [v2:172.21.15.120:6838/1672436487,v1:172.21.15.120:6839/1672436487] conn(0x562219d7dcb0 0x562219d7c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:52.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.791+0000 7f022c56db80 4 client.15213 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "status"}] 2024-06-10T02:53:52.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.791+0000 7f022c56db80 1 -- 172.21.15.52:0/4279875779 --> [v2:172.21.15.120:6838/1672436487,v1:172.21.15.120:6839/1672436487] -- command(tid 0: {"prefix": "status"}) -- 0x562219c48280 con 0x562219d7dcb0 2024-06-10T02:53:52.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.791+0000 7f021b7fe700 1 --2- 172.21.15.52:0/4279875779 >> [v2:172.21.15.120:6838/1672436487,v1:172.21.15.120:6839/1672436487] conn(0x562219d7dcb0 0x562219d7c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:52.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.792+0000 7f021b7fe700 1 --2- 172.21.15.52:0/4279875779 >> [v2:172.21.15.120:6838/1672436487,v1:172.21.15.120:6839/1672436487] conn(0x562219d7dcb0 0x562219d7c9c0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=41ed3696f1821539 server_cookie=ddddfbb839fda0e7 in_seq=0 out_seq=0 2024-06-10T02:53:52.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.792+0000 7f01faffd700 10 client.15213 ms_handle_connect on v2:172.21.15.120:6838/1672436487 2024-06-10T02:53:52.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.792+0000 7f01faffd700 1 -- 172.21.15.52:0/4279875779 <== mds.3 v2:172.21.15.120:6838/1672436487 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x562219c48280 con 0x562219d7dcb0 2024-06-10T02:53:52.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.792+0000 7f01faffd700 10 client.15213 handle_command_reply: tid=0 2024-06-10T02:53:52.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.792+0000 7f022c56db80 1 client.15213 shutdown 2024-06-10T02:53:52.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.792+0000 7f022c56db80 1 -- 172.21.15.52:0/4279875779 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562219cf9c40 msgr2=0x562219d12d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:53:52.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.792+0000 7f022c56db80 1 --2- 172.21.15.52:0/4279875779 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562219cf9c40 0x562219d12d00 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f0214000f80 tx=0x7f0214007ee0 comp rx=0 tx=0).stop 2024-06-10T02:53:52.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.792+0000 7f022c56db80 1 -- 172.21.15.52:0/4279875779 shutdown_connections 2024-06-10T02:53:52.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.792+0000 7f022c56db80 1 --2- 172.21.15.52:0/4279875779 >> [v2:172.21.15.120:6838/1672436487,v1:172.21.15.120:6839/1672436487] conn(0x562219d7dcb0 0x562219d7c9c0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:52.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.792+0000 7f022c56db80 1 --2- 172.21.15.52:0/4279875779 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562219cf9c40 0x562219d12d00 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:52.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.792+0000 7f022c56db80 1 --2- 172.21.15.52:0/4279875779 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562219cf3b60 0x562219d17d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:52.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.792+0000 7f022c56db80 1 --2- 172.21.15.52:0/4279875779 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562219cf9320 0x562219d18290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:52.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.792+0000 7f01faffd700 0 client.15213 ms_handle_reset on v2:172.21.15.120:6838/1672436487 2024-06-10T02:53:52.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.792+0000 7f022c56db80 1 -- 172.21.15.52:0/4279875779 >> 172.21.15.52:0/4279875779 conn(0x562219c1c030 msgr2=0x562219cfa020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:53:52.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.792+0000 7f022c56db80 1 -- 172.21.15.52:0/4279875779 shutdown_connections 2024-06-10T02:53:52.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.792+0000 7f022c56db80 1 -- 172.21.15.52:0/4279875779 wait complete. 2024-06-10T02:53:52.801 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.792+0000 7f022c56db80 20 client.15213 trim_cache size 0 max 16384 2024-06-10T02:53:52.801 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-10T02:53:52.801 INFO:teuthology.orchestra.run.smithi052.stdout: "cluster_fsid": "d8676e9e-26d2-11ef-bc9b-c7b262605968", 2024-06-10T02:53:52.801 INFO:teuthology.orchestra.run.smithi052.stdout: "whoami": 3, 2024-06-10T02:53:52.801 INFO:teuthology.orchestra.run.smithi052.stdout: "id": 24449, 2024-06-10T02:53:52.801 INFO:teuthology.orchestra.run.smithi052.stdout: "want_state": "up:active", 2024-06-10T02:53:52.802 INFO:teuthology.orchestra.run.smithi052.stdout: "state": "up:active", 2024-06-10T02:53:52.802 INFO:teuthology.orchestra.run.smithi052.stdout: "fs_name": "cephfs", 2024-06-10T02:53:52.802 INFO:teuthology.orchestra.run.smithi052.stdout: "rank_uptime": 165.18340171, 2024-06-10T02:53:52.802 INFO:teuthology.orchestra.run.smithi052.stdout: "mdsmap_epoch": 29, 2024-06-10T02:53:52.802 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_epoch": 100, 2024-06-10T02:53:52.802 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_epoch_barrier": 98, 2024-06-10T02:53:52.802 INFO:teuthology.orchestra.run.smithi052.stdout: "uptime": 246.107047483 2024-06-10T02:53:52.802 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-10T02:53:52.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.793+0000 7f0221cea700 1 -- 172.21.15.52:0/3229231518 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f02080a2c20 msgr2=0x7f02080a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:53:52.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.793+0000 7f0221cea700 1 --2- 172.21.15.52:0/3229231518 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f02080a2c20 0x7f02080a50e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f020400b580 tx=0x7f0204009f30 comp rx=0 tx=0).stop 2024-06-10T02:53:52.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.793+0000 7f0221cea700 1 -- 172.21.15.52:0/3229231518 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f021c102740 msgr2=0x7f021c19e810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:53:52.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.793+0000 7f0221cea700 1 --2- 172.21.15.52:0/3229231518 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f021c102740 0x7f021c19e810 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f020c00d980 tx=0x7f020c00dc90 comp rx=0 tx=0).stop 2024-06-10T02:53:52.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.793+0000 7f0221cea700 1 -- 172.21.15.52:0/3229231518 shutdown_connections 2024-06-10T02:53:52.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.793+0000 7f0221cea700 1 --2- 172.21.15.52:0/3229231518 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f021c107840 0x7f021c1989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:52.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.793+0000 7f0221cea700 1 --2- 172.21.15.52:0/3229231518 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f02080a2c20 0x7f02080a50e0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:52.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.793+0000 7f0221cea700 1 --2- 172.21.15.52:0/3229231518 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f021c102740 0x7f021c19e810 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:52.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.793+0000 7f0221cea700 1 --2- 172.21.15.52:0/3229231518 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f021c1030f0 0x7f021c19ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:52.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.793+0000 7f0221cea700 1 -- 172.21.15.52:0/3229231518 >> 172.21.15.52:0/3229231518 conn(0x7f021c0795f0 msgr2=0x7f021c101ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:53:52.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.793+0000 7f0221cea700 1 -- 172.21.15.52:0/3229231518 shutdown_connections 2024-06-10T02:53:52.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:52.793+0000 7f0221cea700 1 -- 172.21.15.52:0/3229231518 wait complete. 2024-06-10T02:53:53.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:53:52 smithi120 ceph-mon[36634]: pgmap v424: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-10T02:53:53.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:53:52 smithi188 ceph-mon[36599]: pgmap v424: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-10T02:53:53.491 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph tell mds.f status 2024-06-10T02:53:53.742 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:53:54.976 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:53:54 smithi052 ceph-mon[32362]: pgmap v425: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-10T02:53:55.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:53:54 smithi120 ceph-mon[36634]: pgmap v425: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-10T02:53:55.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:53:54 smithi188 ceph-mon[36599]: pgmap v425: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-10T02:53:55.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.519+0000 7fd34b985700 1 -- 172.21.15.52:0/2896141373 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd34406a170 msgr2=0x7fd34406a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:53:55.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.519+0000 7fd34b985700 1 --2- 172.21.15.52:0/2896141373 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd34406a170 0x7fd34406a5f0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fd338009990 tx=0x7fd338009ca0 comp rx=0 tx=0).stop 2024-06-10T02:53:55.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.519+0000 7fd34b985700 1 -- 172.21.15.52:0/2896141373 shutdown_connections 2024-06-10T02:53:55.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.519+0000 7fd34b985700 1 --2- 172.21.15.52:0/2896141373 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd34406ab30 0x7fd34410f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:55.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.519+0000 7fd34b985700 1 --2- 172.21.15.52:0/2896141373 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd344108500 0x7fd3441088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:55.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.519+0000 7fd34b985700 1 --2- 172.21.15.52:0/2896141373 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd34406a170 0x7fd34406a5f0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:55.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.519+0000 7fd34b985700 1 -- 172.21.15.52:0/2896141373 >> 172.21.15.52:0/2896141373 conn(0x7fd344078340 msgr2=0x7fd344078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:53:55.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.519+0000 7fd34b985700 1 -- 172.21.15.52:0/2896141373 shutdown_connections 2024-06-10T02:53:55.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.519+0000 7fd34b985700 1 -- 172.21.15.52:0/2896141373 wait complete. 2024-06-10T02:53:55.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.520+0000 7fd34b985700 1 Processor -- start 2024-06-10T02:53:55.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.520+0000 7fd34b985700 1 -- start start 2024-06-10T02:53:55.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.520+0000 7fd34b985700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd34406a170 0x7fd344198d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:55.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.520+0000 7fd34b985700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd34406ab30 0x7fd344199250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:55.523 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.520+0000 7fd34b985700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd344108500 0x7fd34419d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:55.523 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.520+0000 7fd34b985700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fd3441127e0 con 0x7fd34406a170 2024-06-10T02:53:55.523 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.520+0000 7fd34b985700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd344112660 con 0x7fd34406ab30 2024-06-10T02:53:55.523 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.520+0000 7fd34b985700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fd344112960 con 0x7fd344108500 2024-06-10T02:53:55.523 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.520+0000 7fd349721700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd34406a170 0x7fd344198d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:55.523 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.520+0000 7fd349721700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd34406a170 0x7fd344198d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:41990/0 (socket says 172.21.15.52:41990) 2024-06-10T02:53:55.523 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.520+0000 7fd349721700 1 -- 172.21.15.52:0/4201277654 learned_addr learned my addr 172.21.15.52:0/4201277654 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:53:55.523 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.520+0000 7fd348f20700 1 --2- 172.21.15.52:0/4201277654 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd34406ab30 0x7fd344199250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:55.524 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.521+0000 7fd349f22700 1 --2- 172.21.15.52:0/4201277654 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd344108500 0x7fd34419d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:55.524 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.521+0000 7fd349721700 1 -- 172.21.15.52:0/4201277654 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd344108500 msgr2=0x7fd34419d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:53:55.524 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.521+0000 7fd349721700 1 --2- 172.21.15.52:0/4201277654 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd344108500 0x7fd34419d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:55.524 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.521+0000 7fd349721700 1 -- 172.21.15.52:0/4201277654 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd34406ab30 msgr2=0x7fd344199250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:53:55.524 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.521+0000 7fd349721700 1 --2- 172.21.15.52:0/4201277654 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd34406ab30 0x7fd344199250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:55.524 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.521+0000 7fd349721700 1 -- 172.21.15.52:0/4201277654 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd34419dd20 con 0x7fd34406a170 2024-06-10T02:53:55.524 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.521+0000 7fd349721700 1 --2- 172.21.15.52:0/4201277654 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd34406a170 0x7fd344198d10 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fd33400ba50 tx=0x7fd33400bd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:53:55.524 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.521+0000 7fd3427fc700 1 -- 172.21.15.52:0/4201277654 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd334019070 con 0x7fd34406a170 2024-06-10T02:53:55.524 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.521+0000 7fd3427fc700 1 -- 172.21.15.52:0/4201277654 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd334014440 con 0x7fd34406a170 2024-06-10T02:53:55.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.521+0000 7fd34b985700 1 -- 172.21.15.52:0/4201277654 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd34419e010 con 0x7fd34406a170 2024-06-10T02:53:55.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.521+0000 7fd3427fc700 1 -- 172.21.15.52:0/4201277654 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3340075d0 con 0x7fd34406a170 2024-06-10T02:53:55.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.521+0000 7fd34b985700 1 -- 172.21.15.52:0/4201277654 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3441a5890 con 0x7fd34406a170 2024-06-10T02:53:55.533 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.523+0000 7fd3427fc700 1 -- 172.21.15.52:0/4201277654 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fd33401c070 con 0x7fd34406a170 2024-06-10T02:53:55.533 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.524+0000 7fd3427fc700 1 --2- 172.21.15.52:0/4201277654 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fd32c0a2ca0 0x7fd32c0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:55.533 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.524+0000 7fd348f20700 1 --2- 172.21.15.52:0/4201277654 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fd32c0a2ca0 0x7fd32c0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:55.533 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.524+0000 7fd3427fc700 1 -- 172.21.15.52:0/4201277654 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7fd3340ce810 con 0x7fd34406a170 2024-06-10T02:53:55.533 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.525+0000 7fd348f20700 1 --2- 172.21.15.52:0/4201277654 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fd32c0a2ca0 0x7fd32c0a5160 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fd33800b5c0 tx=0x7fd338005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:53:55.533 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.525+0000 7fd356208b80 1 Processor -- start 2024-06-10T02:53:55.533 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.525+0000 7fd356208b80 1 -- start start 2024-06-10T02:53:55.533 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.525+0000 7fd356208b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5565cfe145f0 0x5565cfe149d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:55.533 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.525+0000 7fd356208b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5565cfe14f10 0x5565cfe0e990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:55.534 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.525+0000 7fd356208b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5565cfe0eed0 0x5565cfe0f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:55.534 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.525+0000 7fd356208b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5565cfd607c0 con 0x5565cfe145f0 2024-06-10T02:53:55.534 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.525+0000 7fd356208b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5565cfd086d0 con 0x5565cfe0eed0 2024-06-10T02:53:55.534 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.525+0000 7fd356208b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5565cfd5c780 con 0x5565cfe14f10 2024-06-10T02:53:55.534 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.525+0000 7fd349721700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5565cfe145f0 0x5565cfe149d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:55.534 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.525+0000 7fd349721700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5565cfe145f0 0x5565cfe149d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:41996/0 (socket says 172.21.15.52:41996) 2024-06-10T02:53:55.534 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.526+0000 7fd349721700 1 -- 172.21.15.52:0/331316478 learned_addr learned my addr 172.21.15.52:0/331316478 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:53:55.534 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.526+0000 7fd348f20700 1 --2- 172.21.15.52:0/331316478 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5565cfe14f10 0x5565cfe0e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:55.534 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.526+0000 7fd349f22700 1 --2- 172.21.15.52:0/331316478 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5565cfe0eed0 0x5565cfe0f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:55.534 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.526+0000 7fd349721700 1 -- 172.21.15.52:0/331316478 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5565cfe14f10 msgr2=0x5565cfe0e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:53:55.535 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.526+0000 7fd349721700 1 --2- 172.21.15.52:0/331316478 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5565cfe14f10 0x5565cfe0e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:55.535 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.526+0000 7fd349721700 1 -- 172.21.15.52:0/331316478 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5565cfe0eed0 msgr2=0x5565cfe0f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:53:55.535 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.526+0000 7fd349721700 1 --2- 172.21.15.52:0/331316478 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5565cfe0eed0 0x5565cfe0f2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:55.535 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.526+0000 7fd349721700 1 -- 172.21.15.52:0/331316478 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565cfcfef00 con 0x5565cfe145f0 2024-06-10T02:53:55.535 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.526+0000 7fd349721700 1 --2- 172.21.15.52:0/331316478 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5565cfe145f0 0x5565cfe149d0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fd33409a960 tx=0x7fd334004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a65c5e97db1d9620 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:53:55.535 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.526+0000 7fd32b7fe700 1 -- 172.21.15.52:0/331316478 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3340a38d0 con 0x5565cfe145f0 2024-06-10T02:53:55.535 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.526+0000 7fd32b7fe700 1 -- 172.21.15.52:0/331316478 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd3340b0070 con 0x5565cfe145f0 2024-06-10T02:53:55.535 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.526+0000 7fd32b7fe700 1 -- 172.21.15.52:0/331316478 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3340a3440 con 0x5565cfe145f0 2024-06-10T02:53:55.535 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.527+0000 7fd356208b80 1 -- 172.21.15.52:0/331316478 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5565cfe145f0 msgr2=0x5565cfe149d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:53:55.535 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.527+0000 7fd356208b80 1 --2- 172.21.15.52:0/331316478 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5565cfe145f0 0x5565cfe149d0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fd33409a960 tx=0x7fd334004ad0 comp rx=0 tx=0).stop 2024-06-10T02:53:55.535 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.527+0000 7fd356208b80 1 -- 172.21.15.52:0/331316478 shutdown_connections 2024-06-10T02:53:55.535 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.527+0000 7fd356208b80 1 --2- 172.21.15.52:0/331316478 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5565cfe0eed0 0x5565cfe0f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:55.536 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.527+0000 7fd356208b80 1 --2- 172.21.15.52:0/331316478 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5565cfe145f0 0x5565cfe149d0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:55.536 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.527+0000 7fd356208b80 1 --2- 172.21.15.52:0/331316478 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5565cfe14f10 0x5565cfe0e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:55.536 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.527+0000 7fd356208b80 1 -- 172.21.15.52:0/331316478 >> 172.21.15.52:0/331316478 conn(0x5565cfd39c70 msgr2=0x5565cfe09800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:53:55.536 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.527+0000 7fd356208b80 1 -- 172.21.15.52:0/331316478 shutdown_connections 2024-06-10T02:53:55.536 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.527+0000 7fd356208b80 1 -- 172.21.15.52:0/331316478 wait complete. 2024-06-10T02:53:55.536 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.527+0000 7fd356208b80 1 -- start start 2024-06-10T02:53:55.536 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.528+0000 7fd356208b80 1 Processor -- start 2024-06-10T02:53:55.536 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.528+0000 7fd356208b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5565cfe0eed0 0x5565cfe1ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:55.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.528+0000 7fd356208b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5565cfe145f0 0x5565cfe17f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:55.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.528+0000 7fd356208b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5565cfe14f10 0x5565cfe18470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:55.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.528+0000 7fd356208b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5565cfe1b920 con 0x5565cfe0eed0 2024-06-10T02:53:55.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.528+0000 7fd356208b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5565cfd00ba0 con 0x5565cfe145f0 2024-06-10T02:53:55.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.528+0000 7fd356208b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5565cfe1baa0 con 0x5565cfe14f10 2024-06-10T02:53:55.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.528+0000 7fd356208b80 10 client.0 _ll_register_callbacks cb 0x5565cfcfa8d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T02:53:55.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.528+0000 7fd349721700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5565cfe0eed0 0x5565cfe1ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:55.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.528+0000 7fd349721700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5565cfe0eed0 0x5565cfe1ac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:42012/0 (socket says 172.21.15.52:42012) 2024-06-10T02:53:55.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.528+0000 7fd349721700 1 -- 172.21.15.52:0/2807331810 learned_addr learned my addr 172.21.15.52:0/2807331810 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:53:55.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.528+0000 7fd348f20700 1 --2- 172.21.15.52:0/2807331810 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5565cfe145f0 0x5565cfe17f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:55.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.528+0000 7fd349f22700 1 --2- 172.21.15.52:0/2807331810 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5565cfe14f10 0x5565cfe18470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:55.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.528+0000 7fd349721700 1 -- 172.21.15.52:0/2807331810 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5565cfe14f10 msgr2=0x5565cfe18470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:53:55.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.528+0000 7fd349721700 1 --2- 172.21.15.52:0/2807331810 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5565cfe14f10 0x5565cfe18470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:55.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.528+0000 7fd349721700 1 -- 172.21.15.52:0/2807331810 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5565cfe145f0 msgr2=0x5565cfe17f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:53:55.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.528+0000 7fd349721700 1 --2- 172.21.15.52:0/2807331810 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5565cfe145f0 0x5565cfe17f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:55.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.528+0000 7fd349721700 1 -- 172.21.15.52:0/2807331810 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5565cfcaf880 con 0x5565cfe0eed0 2024-06-10T02:53:55.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.529+0000 7fd349721700 1 --2- 172.21.15.52:0/2807331810 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5565cfe0eed0 0x5565cfe1ac30 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fd334097070 tx=0x7fd33409f0d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ecc71da455c7c60 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:53:55.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.529+0000 7fd328ff9700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-10T02:53:55.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.529+0000 7fd328ff9700 1 -- 172.21.15.52:0/2807331810 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3340b8070 con 0x5565cfe0eed0 2024-06-10T02:53:55.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.529+0000 7fd328ff9700 1 -- 172.21.15.52:0/2807331810 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7fd3340a3440 con 0x5565cfe0eed0 2024-06-10T02:53:55.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.529+0000 7fd356208b80 1 -- 172.21.15.52:0/2807331810 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd3340b0070 con 0x5565cfe0eed0 2024-06-10T02:53:55.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.529+0000 7fd328ff9700 1 -- 172.21.15.52:0/2807331810 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd3340b3e60 con 0x5565cfe0eed0 2024-06-10T02:53:55.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.529+0000 7fd356208b80 10 client.15225 fetch_fsmap learned FSMap version 29 2024-06-10T02:53:55.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.529+0000 7fd356208b80 1 -- 172.21.15.52:0/2807331810 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5565cfe26200 con 0x5565cfe0eed0 2024-06-10T02:53:55.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.529+0000 7fd328ff9700 1 -- 172.21.15.52:0/2807331810 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd3340b0070 con 0x5565cfe0eed0 2024-06-10T02:53:55.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.529+0000 7fd328ff9700 1 -- 172.21.15.52:0/2807331810 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fd334099b20 con 0x5565cfe0eed0 2024-06-10T02:53:55.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.529+0000 7fd356208b80 10 client.15225 fetch_fsmap finished waiting for FSMap version 29 2024-06-10T02:53:55.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.530+0000 7fd356208b80 10 client.15225 resolve_mds: resolved name 'f' to daemon mds.f 2024-06-10T02:53:55.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.530+0000 7fd356208b80 20 client.15225 populate_metadata read hostname 'smithi052' 2024-06-10T02:53:55.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.530+0000 7fd356208b80 1 --2- 172.21.15.52:0/2807331810 >> [v2:172.21.15.188:6834/3904544259,v1:172.21.15.188:6835/3904544259] conn(0x5565cfe27550 0x5565cfe29720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:55.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.530+0000 7fd356208b80 4 client.15225 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T02:53:55.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.530+0000 7fd356208b80 1 -- 172.21.15.52:0/2807331810 --> [v2:172.21.15.188:6834/3904544259,v1:172.21.15.188:6835/3904544259] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5565cfd6dcc0 con 0x5565cfe27550 2024-06-10T02:53:55.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.530+0000 7fd348f20700 1 --2- 172.21.15.52:0/2807331810 >> [v2:172.21.15.188:6834/3904544259,v1:172.21.15.188:6835/3904544259] conn(0x5565cfe27550 0x5565cfe29720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:55.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.530+0000 7fd328ff9700 10 client.15225 ms_handle_connect on v2:172.21.15.188:6834/3904544259 2024-06-10T02:53:55.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.530+0000 7fd348f20700 1 --2- 172.21.15.52:0/2807331810 >> [v2:172.21.15.188:6834/3904544259,v1:172.21.15.188:6835/3904544259] conn(0x5565cfe27550 0x5565cfe29720 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8f80f7acfa077dc0 server_cookie=be2a960ec31e6e2 in_seq=0 out_seq=0 2024-06-10T02:53:55.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.531+0000 7fd328ff9700 1 -- 172.21.15.52:0/2807331810 <== mds.2 v2:172.21.15.188:6834/3904544259 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5565cfd6dcc0 con 0x5565cfe27550 2024-06-10T02:53:55.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.531+0000 7fd328ff9700 10 client.15225 handle_command_reply: tid=0 2024-06-10T02:53:55.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.531+0000 7fd356208b80 1 client.15225 shutdown 2024-06-10T02:53:55.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.531+0000 7fd356208b80 1 -- 172.21.15.52:0/2807331810 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5565cfe0eed0 msgr2=0x5565cfe1ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:53:55.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.531+0000 7fd356208b80 1 --2- 172.21.15.52:0/2807331810 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5565cfe0eed0 0x5565cfe1ac30 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fd334097070 tx=0x7fd33409f0d0 comp rx=0 tx=0).stop 2024-06-10T02:53:55.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.531+0000 7fd356208b80 1 -- 172.21.15.52:0/2807331810 shutdown_connections 2024-06-10T02:53:55.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.531+0000 7fd356208b80 1 --2- 172.21.15.52:0/2807331810 >> [v2:172.21.15.188:6834/3904544259,v1:172.21.15.188:6835/3904544259] conn(0x5565cfe27550 0x5565cfe29720 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:55.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.531+0000 7fd356208b80 1 --2- 172.21.15.52:0/2807331810 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5565cfe14f10 0x5565cfe18470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:55.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.531+0000 7fd356208b80 1 --2- 172.21.15.52:0/2807331810 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5565cfe0eed0 0x5565cfe1ac30 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:55.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.531+0000 7fd356208b80 1 --2- 172.21.15.52:0/2807331810 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5565cfe145f0 0x5565cfe17f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:55.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.531+0000 7fd356208b80 1 -- 172.21.15.52:0/2807331810 >> 172.21.15.52:0/2807331810 conn(0x5565cfd39c70 msgr2=0x5565cfe095a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:53:55.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.531+0000 7fd328ff9700 0 client.15225 ms_handle_reset on v2:172.21.15.188:6834/3904544259 2024-06-10T02:53:55.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.531+0000 7fd356208b80 1 -- 172.21.15.52:0/2807331810 shutdown_connections 2024-06-10T02:53:55.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.531+0000 7fd356208b80 1 -- 172.21.15.52:0/2807331810 wait complete. 2024-06-10T02:53:55.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.531+0000 7fd356208b80 20 client.15225 trim_cache size 0 max 16384 2024-06-10T02:53:55.558 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.553+0000 7fd356208b80 1 Processor -- start 2024-06-10T02:53:55.558 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.553+0000 7fd356208b80 1 -- start start 2024-06-10T02:53:55.558 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.553+0000 7fd356208b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5565cfe0eed0 0x5565cfe0e990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:55.558 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.553+0000 7fd356208b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5565cfe145f0 0x5565cfe27be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:55.558 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.553+0000 7fd356208b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5565cfe14f10 0x5565cfe16860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:55.558 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.553+0000 7fd356208b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5565cfe27a60 con 0x5565cfe14f10 2024-06-10T02:53:55.558 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.553+0000 7fd356208b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5565cfe28130 con 0x5565cfe145f0 2024-06-10T02:53:55.558 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.553+0000 7fd356208b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5565cfe28800 con 0x5565cfe0eed0 2024-06-10T02:53:55.559 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.553+0000 7fd349f22700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5565cfe14f10 0x5565cfe16860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:55.559 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.553+0000 7fd349f22700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5565cfe14f10 0x5565cfe16860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:42028/0 (socket says 172.21.15.52:42028) 2024-06-10T02:53:55.559 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.554+0000 7fd349f22700 1 -- 172.21.15.52:0/1007891333 learned_addr learned my addr 172.21.15.52:0/1007891333 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:53:55.559 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.554+0000 7fd348f20700 1 --2- 172.21.15.52:0/1007891333 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5565cfe145f0 0x5565cfe27be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:55.559 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.554+0000 7fd349721700 1 --2- 172.21.15.52:0/1007891333 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5565cfe0eed0 0x5565cfe0e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:55.559 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.554+0000 7fd349f22700 1 -- 172.21.15.52:0/1007891333 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5565cfe0eed0 msgr2=0x5565cfe0e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:53:55.559 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.554+0000 7fd349f22700 1 --2- 172.21.15.52:0/1007891333 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5565cfe0eed0 0x5565cfe0e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:55.559 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.554+0000 7fd349f22700 1 -- 172.21.15.52:0/1007891333 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5565cfe145f0 msgr2=0x5565cfe27be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:53:55.559 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.554+0000 7fd349f22700 1 --2- 172.21.15.52:0/1007891333 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5565cfe145f0 0x5565cfe27be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:55.559 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.554+0000 7fd349f22700 1 -- 172.21.15.52:0/1007891333 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565cfe28cf0 con 0x5565cfe14f10 2024-06-10T02:53:55.559 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.554+0000 7fd349f22700 1 --2- 172.21.15.52:0/1007891333 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5565cfe14f10 0x5565cfe16860 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fd33c00ec80 tx=0x7fd33c00ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5fed9ed6531fcf7 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:53:55.559 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.554+0000 7fd32affd700 1 -- 172.21.15.52:0/1007891333 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd33c01f070 con 0x5565cfe14f10 2024-06-10T02:53:55.560 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.554+0000 7fd32affd700 1 -- 172.21.15.52:0/1007891333 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd33c011c90 con 0x5565cfe14f10 2024-06-10T02:53:55.560 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.554+0000 7fd32affd700 1 -- 172.21.15.52:0/1007891333 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd33c0107f0 con 0x5565cfe14f10 2024-06-10T02:53:55.560 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.555+0000 7fd356208b80 1 -- 172.21.15.52:0/1007891333 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5565cfe14f10 msgr2=0x5565cfe16860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:53:55.560 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.555+0000 7fd356208b80 1 --2- 172.21.15.52:0/1007891333 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5565cfe14f10 0x5565cfe16860 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fd33c00ec80 tx=0x7fd33c00ef90 comp rx=0 tx=0).stop 2024-06-10T02:53:55.560 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.555+0000 7fd356208b80 1 -- 172.21.15.52:0/1007891333 shutdown_connections 2024-06-10T02:53:55.560 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.555+0000 7fd356208b80 1 --2- 172.21.15.52:0/1007891333 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5565cfe14f10 0x5565cfe16860 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:55.560 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.555+0000 7fd356208b80 1 --2- 172.21.15.52:0/1007891333 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5565cfe0eed0 0x5565cfe0e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:55.560 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.555+0000 7fd356208b80 1 --2- 172.21.15.52:0/1007891333 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5565cfe145f0 0x5565cfe27be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:55.560 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.555+0000 7fd356208b80 1 -- 172.21.15.52:0/1007891333 >> 172.21.15.52:0/1007891333 conn(0x5565cfd39c70 msgr2=0x5565cfe0f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:53:55.560 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.555+0000 7fd356208b80 1 -- 172.21.15.52:0/1007891333 shutdown_connections 2024-06-10T02:53:55.560 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.555+0000 7fd356208b80 1 -- 172.21.15.52:0/1007891333 wait complete. 2024-06-10T02:53:55.560 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.555+0000 7fd356208b80 1 -- start start 2024-06-10T02:53:55.561 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.556+0000 7fd356208b80 1 Processor -- start 2024-06-10T02:53:55.561 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.556+0000 7fd356208b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5565cfe0eed0 0x5565cfe33480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:55.561 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.556+0000 7fd356208b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5565cfe145f0 0x5565cfe0db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:55.561 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.556+0000 7fd356208b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5565cfe14f10 0x5565cfe0e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:55.561 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.556+0000 7fd356208b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5565cfe295a0 con 0x5565cfe14f10 2024-06-10T02:53:55.561 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.556+0000 7fd356208b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5565cfe28ed0 con 0x5565cfe0eed0 2024-06-10T02:53:55.561 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.556+0000 7fd356208b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5565cfe29c70 con 0x5565cfe145f0 2024-06-10T02:53:55.561 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.556+0000 7fd349f22700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5565cfe14f10 0x5565cfe0e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:55.561 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.556+0000 7fd356208b80 10 client.0 _ll_register_callbacks cb 0x5565cfcfa8d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T02:53:55.561 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.556+0000 7fd349f22700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5565cfe14f10 0x5565cfe0e2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:42036/0 (socket says 172.21.15.52:42036) 2024-06-10T02:53:55.561 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.556+0000 7fd349f22700 1 -- 172.21.15.52:0/3406289896 learned_addr learned my addr 172.21.15.52:0/3406289896 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:53:55.561 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.556+0000 7fd349721700 1 --2- 172.21.15.52:0/3406289896 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5565cfe0eed0 0x5565cfe33480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:55.562 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.556+0000 7fd348f20700 1 --2- 172.21.15.52:0/3406289896 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5565cfe145f0 0x5565cfe0db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:55.562 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.557+0000 7fd349f22700 1 -- 172.21.15.52:0/3406289896 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5565cfe145f0 msgr2=0x5565cfe0db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:53:55.562 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.557+0000 7fd349f22700 1 --2- 172.21.15.52:0/3406289896 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5565cfe145f0 0x5565cfe0db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:55.562 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.557+0000 7fd349f22700 1 -- 172.21.15.52:0/3406289896 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5565cfe0eed0 msgr2=0x5565cfe33480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:53:55.562 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.557+0000 7fd349f22700 1 --2- 172.21.15.52:0/3406289896 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5565cfe0eed0 0x5565cfe33480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:55.562 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.557+0000 7fd349f22700 1 -- 172.21.15.52:0/3406289896 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5565cfe30f70 con 0x5565cfe14f10 2024-06-10T02:53:55.562 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.557+0000 7fd349721700 1 --2- 172.21.15.52:0/3406289896 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5565cfe0eed0 0x5565cfe33480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T02:53:55.562 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.557+0000 7fd348f20700 1 --2- 172.21.15.52:0/3406289896 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5565cfe145f0 0x5565cfe0db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T02:53:55.562 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.557+0000 7fd349f22700 1 --2- 172.21.15.52:0/3406289896 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5565cfe14f10 0x5565cfe0e2c0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fd33c00ec50 tx=0x7fd33c0057d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10b0219ff41eeb05 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:53:55.562 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.557+0000 7fd328ff9700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-10T02:53:55.562 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.557+0000 7fd328ff9700 1 -- 172.21.15.52:0/3406289896 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd33c01f040 con 0x5565cfe14f10 2024-06-10T02:53:55.562 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.557+0000 7fd328ff9700 1 -- 172.21.15.52:0/3406289896 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7fd33c01a410 con 0x5565cfe14f10 2024-06-10T02:53:55.562 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.557+0000 7fd356208b80 1 -- 172.21.15.52:0/3406289896 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd33c011c90 con 0x5565cfe14f10 2024-06-10T02:53:55.563 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.557+0000 7fd328ff9700 1 -- 172.21.15.52:0/3406289896 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd33c033050 con 0x5565cfe14f10 2024-06-10T02:53:55.563 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.557+0000 7fd356208b80 10 client.15231 fetch_fsmap learned FSMap version 29 2024-06-10T02:53:55.563 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.557+0000 7fd356208b80 1 -- 172.21.15.52:0/3406289896 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5565cfe93d20 con 0x5565cfe14f10 2024-06-10T02:53:55.563 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.558+0000 7fd328ff9700 1 -- 172.21.15.52:0/3406289896 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd33c02a070 con 0x5565cfe14f10 2024-06-10T02:53:55.563 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.558+0000 7fd328ff9700 1 -- 172.21.15.52:0/3406289896 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fd33c022030 con 0x5565cfe14f10 2024-06-10T02:53:55.563 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.558+0000 7fd356208b80 10 client.15231 fetch_fsmap finished waiting for FSMap version 29 2024-06-10T02:53:55.563 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.558+0000 7fd356208b80 10 client.15231 resolve_mds: resolved name 'f' to daemon mds.f 2024-06-10T02:53:55.563 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.558+0000 7fd356208b80 20 client.15231 populate_metadata read hostname 'smithi052' 2024-06-10T02:53:55.563 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.558+0000 7fd356208b80 1 --2- 172.21.15.52:0/3406289896 >> [v2:172.21.15.188:6834/3904544259,v1:172.21.15.188:6835/3904544259] conn(0x5565cfe984c0 0x5565cfe971d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:55.563 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.558+0000 7fd356208b80 4 client.15231 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "status"}] 2024-06-10T02:53:55.563 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.558+0000 7fd356208b80 1 -- 172.21.15.52:0/3406289896 --> [v2:172.21.15.188:6834/3904544259,v1:172.21.15.188:6835/3904544259] -- command(tid 0: {"prefix": "status"}) -- 0x7fd33c011c90 con 0x5565cfe984c0 2024-06-10T02:53:55.563 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.558+0000 7fd349721700 1 --2- 172.21.15.52:0/3406289896 >> [v2:172.21.15.188:6834/3904544259,v1:172.21.15.188:6835/3904544259] conn(0x5565cfe984c0 0x5565cfe971d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:55.563 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.559+0000 7fd328ff9700 10 client.15231 ms_handle_connect on v2:172.21.15.188:6834/3904544259 2024-06-10T02:53:55.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.559+0000 7fd349721700 1 --2- 172.21.15.52:0/3406289896 >> [v2:172.21.15.188:6834/3904544259,v1:172.21.15.188:6835/3904544259] conn(0x5565cfe984c0 0x5565cfe971d0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=bb27ef8709201f30 server_cookie=e82d3dc6c3b42f2b in_seq=0 out_seq=0 2024-06-10T02:53:55.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.559+0000 7fd328ff9700 1 -- 172.21.15.52:0/3406289896 <== mds.2 v2:172.21.15.188:6834/3904544259 1 ==== command_reply(tid 0: 0 ) ==== 8+0+328 (crc 0 0 0) 0x7fd33c011c90 con 0x5565cfe984c0 2024-06-10T02:53:55.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.559+0000 7fd328ff9700 10 client.15231 handle_command_reply: tid=0 2024-06-10T02:53:55.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.559+0000 7fd356208b80 1 client.15231 shutdown 2024-06-10T02:53:55.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.559+0000 7fd356208b80 1 -- 172.21.15.52:0/3406289896 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5565cfe14f10 msgr2=0x5565cfe0e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:53:55.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.559+0000 7fd356208b80 1 --2- 172.21.15.52:0/3406289896 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5565cfe14f10 0x5565cfe0e2c0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fd33c00ec50 tx=0x7fd33c0057d0 comp rx=0 tx=0).stop 2024-06-10T02:53:55.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.559+0000 7fd356208b80 1 -- 172.21.15.52:0/3406289896 shutdown_connections 2024-06-10T02:53:55.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.559+0000 7fd356208b80 1 --2- 172.21.15.52:0/3406289896 >> [v2:172.21.15.188:6834/3904544259,v1:172.21.15.188:6835/3904544259] conn(0x5565cfe984c0 0x5565cfe971d0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:55.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.559+0000 7fd356208b80 1 --2- 172.21.15.52:0/3406289896 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5565cfe14f10 0x5565cfe0e2c0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:55.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.559+0000 7fd356208b80 1 --2- 172.21.15.52:0/3406289896 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5565cfe0eed0 0x5565cfe33480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:55.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.559+0000 7fd356208b80 1 --2- 172.21.15.52:0/3406289896 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5565cfe145f0 0x5565cfe0db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:55.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.559+0000 7fd328ff9700 0 client.15231 ms_handle_reset on v2:172.21.15.188:6834/3904544259 2024-06-10T02:53:55.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.559+0000 7fd356208b80 1 -- 172.21.15.52:0/3406289896 >> 172.21.15.52:0/3406289896 conn(0x5565cfd39c70 msgr2=0x5565cfe02db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:53:55.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.559+0000 7fd356208b80 1 -- 172.21.15.52:0/3406289896 shutdown_connections 2024-06-10T02:53:55.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.559+0000 7fd356208b80 1 -- 172.21.15.52:0/3406289896 wait complete. 2024-06-10T02:53:55.566 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.560+0000 7fd356208b80 20 client.15231 trim_cache size 0 max 16384 2024-06-10T02:53:55.566 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-10T02:53:55.566 INFO:teuthology.orchestra.run.smithi052.stdout: "cluster_fsid": "d8676e9e-26d2-11ef-bc9b-c7b262605968", 2024-06-10T02:53:55.566 INFO:teuthology.orchestra.run.smithi052.stdout: "whoami": 2, 2024-06-10T02:53:55.566 INFO:teuthology.orchestra.run.smithi052.stdout: "id": 24463, 2024-06-10T02:53:55.566 INFO:teuthology.orchestra.run.smithi052.stdout: "want_state": "up:active", 2024-06-10T02:53:55.566 INFO:teuthology.orchestra.run.smithi052.stdout: "state": "up:active", 2024-06-10T02:53:55.566 INFO:teuthology.orchestra.run.smithi052.stdout: "fs_name": "cephfs", 2024-06-10T02:53:55.566 INFO:teuthology.orchestra.run.smithi052.stdout: "rank_uptime": 168.954426751, 2024-06-10T02:53:55.567 INFO:teuthology.orchestra.run.smithi052.stdout: "mdsmap_epoch": 29, 2024-06-10T02:53:55.567 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_epoch": 100, 2024-06-10T02:53:55.567 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_epoch_barrier": 98, 2024-06-10T02:53:55.567 INFO:teuthology.orchestra.run.smithi052.stdout: "uptime": 267.93404114800001 2024-06-10T02:53:55.567 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-10T02:53:55.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.560+0000 7fd34b985700 1 -- 172.21.15.52:0/4201277654 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fd32c0a2ca0 msgr2=0x7fd32c0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:53:55.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.560+0000 7fd34b985700 1 --2- 172.21.15.52:0/4201277654 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fd32c0a2ca0 0x7fd32c0a5160 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fd33800b5c0 tx=0x7fd338005960 comp rx=0 tx=0).stop 2024-06-10T02:53:55.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.560+0000 7fd34b985700 1 -- 172.21.15.52:0/4201277654 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd34406a170 msgr2=0x7fd344198d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:53:55.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.560+0000 7fd34b985700 1 --2- 172.21.15.52:0/4201277654 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd34406a170 0x7fd344198d10 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fd33400ba50 tx=0x7fd33400bd60 comp rx=0 tx=0).stop 2024-06-10T02:53:55.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.560+0000 7fd34b985700 1 -- 172.21.15.52:0/4201277654 shutdown_connections 2024-06-10T02:53:55.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.560+0000 7fd34b985700 1 --2- 172.21.15.52:0/4201277654 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd344108500 0x7fd34419d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:55.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.560+0000 7fd34b985700 1 --2- 172.21.15.52:0/4201277654 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fd32c0a2ca0 0x7fd32c0a5160 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:55.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.560+0000 7fd34b985700 1 --2- 172.21.15.52:0/4201277654 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd34406a170 0x7fd344198d10 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:55.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.560+0000 7fd34b985700 1 --2- 172.21.15.52:0/4201277654 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd34406ab30 0x7fd344199250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:55.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.560+0000 7fd34b985700 1 -- 172.21.15.52:0/4201277654 >> 172.21.15.52:0/4201277654 conn(0x7fd344078340 msgr2=0x7fd3441000e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:53:55.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.560+0000 7fd34b985700 1 -- 172.21.15.52:0/4201277654 shutdown_connections 2024-06-10T02:53:55.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:55.560+0000 7fd34b985700 1 -- 172.21.15.52:0/4201277654 wait complete. 2024-06-10T02:53:56.347 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph tell mds.c status 2024-06-10T02:53:56.601 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:53:56.976 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:53:56 smithi052 ceph-mon[32362]: pgmap v426: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-10T02:53:57.005 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:53:56 smithi188 ceph-mon[36599]: pgmap v426: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-10T02:53:57.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:53:56 smithi120 ceph-mon[36634]: pgmap v426: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-10T02:53:58.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.390+0000 7f393efaf700 1 -- 172.21.15.52:0/1464409647 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3938101fc0 msgr2=0x7f3938106010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:53:58.392 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.390+0000 7f393efaf700 1 --2- 172.21.15.52:0/1464409647 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3938101fc0 0x7f3938106010 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f39280099c0 tx=0x7f3928009cd0 comp rx=0 tx=0).stop 2024-06-10T02:53:58.392 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.390+0000 7f393efaf700 1 -- 172.21.15.52:0/1464409647 shutdown_connections 2024-06-10T02:53:58.392 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.390+0000 7f393efaf700 1 --2- 172.21.15.52:0/1464409647 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f39381066f0 0x7f393810e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:58.392 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.390+0000 7f393efaf700 1 --2- 172.21.15.52:0/1464409647 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3938101610 0x7f39381019f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:58.392 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.390+0000 7f393efaf700 1 --2- 172.21.15.52:0/1464409647 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3938101fc0 0x7f3938106010 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:58.392 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.390+0000 7f393efaf700 1 -- 172.21.15.52:0/1464409647 >> 172.21.15.52:0/1464409647 conn(0x7f3938078430 msgr2=0x7f3938078840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:53:58.392 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.390+0000 7f393efaf700 1 -- 172.21.15.52:0/1464409647 shutdown_connections 2024-06-10T02:53:58.392 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.390+0000 7f393efaf700 1 -- 172.21.15.52:0/1464409647 wait complete. 2024-06-10T02:53:58.392 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.391+0000 7f393efaf700 1 Processor -- start 2024-06-10T02:53:58.392 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.391+0000 7f393efaf700 1 -- start start 2024-06-10T02:53:58.392 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.391+0000 7f393efaf700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3938101610 0x7f3938199a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:58.392 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.391+0000 7f393efaf700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3938101fc0 0x7f3938199fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:58.392 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.391+0000 7f393efaf700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f39381066f0 0x7f39381a4b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:58.393 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.391+0000 7f393efaf700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f3938111750 con 0x7f3938101fc0 2024-06-10T02:53:58.393 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.391+0000 7f393efaf700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f39381115d0 con 0x7f39381066f0 2024-06-10T02:53:58.393 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.391+0000 7f393efaf700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f39381118d0 con 0x7f3938101610 2024-06-10T02:53:58.393 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.391+0000 7f392ffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3938101fc0 0x7f3938199fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:58.393 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.391+0000 7f392ffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3938101fc0 0x7f3938199fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:42048/0 (socket says 172.21.15.52:42048) 2024-06-10T02:53:58.393 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.392+0000 7f392ffff700 1 -- 172.21.15.52:0/1365239612 learned_addr learned my addr 172.21.15.52:0/1365239612 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:53:58.393 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.392+0000 7f393cd4b700 1 --2- 172.21.15.52:0/1365239612 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3938101610 0x7f3938199a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:58.393 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.392+0000 7f393d54c700 1 --2- 172.21.15.52:0/1365239612 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f39381066f0 0x7f39381a4b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:58.393 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.392+0000 7f392ffff700 1 -- 172.21.15.52:0/1365239612 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3938101610 msgr2=0x7f3938199a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:53:58.393 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.392+0000 7f392ffff700 1 --2- 172.21.15.52:0/1365239612 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3938101610 0x7f3938199a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:58.393 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.392+0000 7f392ffff700 1 -- 172.21.15.52:0/1365239612 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f39381066f0 msgr2=0x7f39381a4b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:53:58.393 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.392+0000 7f392ffff700 1 --2- 172.21.15.52:0/1365239612 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f39381066f0 0x7f39381a4b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:58.393 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.392+0000 7f392ffff700 1 -- 172.21.15.52:0/1365239612 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39381a5050 con 0x7f3938101fc0 2024-06-10T02:53:58.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.392+0000 7f393d54c700 1 --2- 172.21.15.52:0/1365239612 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f39381066f0 0x7f39381a4b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T02:53:58.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.392+0000 7f393cd4b700 1 --2- 172.21.15.52:0/1365239612 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3938101610 0x7f3938199a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T02:53:58.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.392+0000 7f392ffff700 1 --2- 172.21.15.52:0/1365239612 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3938101fc0 0x7f3938199fd0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f3928000c00 tx=0x7f392800bfb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:53:58.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.392+0000 7f392dffb700 1 -- 172.21.15.52:0/1365239612 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3928027070 con 0x7f3938101fc0 2024-06-10T02:53:58.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.392+0000 7f392dffb700 1 -- 172.21.15.52:0/1365239612 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3928004130 con 0x7f3938101fc0 2024-06-10T02:53:58.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.392+0000 7f393efaf700 1 -- 172.21.15.52:0/1365239612 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39381a5220 con 0x7f3938101fc0 2024-06-10T02:53:58.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.392+0000 7f392dffb700 1 -- 172.21.15.52:0/1365239612 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f392800f460 con 0x7f3938101fc0 2024-06-10T02:53:58.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.392+0000 7f393efaf700 1 -- 172.21.15.52:0/1365239612 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39381a56d0 con 0x7f3938101fc0 2024-06-10T02:53:58.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.395+0000 7f392dffb700 1 -- 172.21.15.52:0/1365239612 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f392802a070 con 0x7f3938101fc0 2024-06-10T02:53:58.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.395+0000 7f392dffb700 1 --2- 172.21.15.52:0/1365239612 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f39200a2ca0 0x7f39200a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:58.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.395+0000 7f393cd4b700 1 --2- 172.21.15.52:0/1365239612 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f39200a2ca0 0x7f39200a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:58.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.395+0000 7f392dffb700 1 -- 172.21.15.52:0/1365239612 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f39280d0380 con 0x7f3938101fc0 2024-06-10T02:53:58.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.396+0000 7f393cd4b700 1 --2- 172.21.15.52:0/1365239612 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f39200a2ca0 0x7f39200a5160 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f3934005fd0 tx=0x7f3934005e30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:53:58.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.396+0000 7f3949832b80 1 Processor -- start 2024-06-10T02:53:58.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.396+0000 7f3949832b80 1 -- start start 2024-06-10T02:53:58.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.396+0000 7f3949832b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5589a58e55f0 0x5589a58e59d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:58.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.396+0000 7f3949832b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5589a58e5f10 0x5589a58df990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:58.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.396+0000 7f3949832b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5589a58dfed0 0x5589a58e02f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:58.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.396+0000 7f3949832b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5589a58317c0 con 0x5589a58e55f0 2024-06-10T02:53:58.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.396+0000 7f3949832b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5589a57d96d0 con 0x5589a58dfed0 2024-06-10T02:53:58.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.396+0000 7f3949832b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5589a582d780 con 0x5589a58e5f10 2024-06-10T02:53:58.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.397+0000 7f393cd4b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5589a58e55f0 0x5589a58e59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:58.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.397+0000 7f393cd4b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5589a58e55f0 0x5589a58e59d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:42052/0 (socket says 172.21.15.52:42052) 2024-06-10T02:53:58.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.397+0000 7f393cd4b700 1 -- 172.21.15.52:0/1746194351 learned_addr learned my addr 172.21.15.52:0/1746194351 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:53:58.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.397+0000 7f392ffff700 1 --2- 172.21.15.52:0/1746194351 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5589a58e5f10 0x5589a58df990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:58.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.397+0000 7f393d54c700 1 --2- 172.21.15.52:0/1746194351 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5589a58dfed0 0x5589a58e02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:58.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.397+0000 7f393cd4b700 1 -- 172.21.15.52:0/1746194351 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5589a58e5f10 msgr2=0x5589a58df990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:53:58.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.397+0000 7f393cd4b700 1 --2- 172.21.15.52:0/1746194351 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5589a58e5f10 0x5589a58df990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:58.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.397+0000 7f393cd4b700 1 -- 172.21.15.52:0/1746194351 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5589a58dfed0 msgr2=0x5589a58e02f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:53:58.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.397+0000 7f393cd4b700 1 --2- 172.21.15.52:0/1746194351 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5589a58dfed0 0x5589a58e02f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:58.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.397+0000 7f393cd4b700 1 -- 172.21.15.52:0/1746194351 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5589a57cff00 con 0x5589a58e55f0 2024-06-10T02:53:58.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.397+0000 7f393d54c700 1 --2- 172.21.15.52:0/1746194351 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5589a58dfed0 0x5589a58e02f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T02:53:58.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.397+0000 7f393cd4b700 1 --2- 172.21.15.52:0/1746194351 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5589a58e55f0 0x5589a58e59d0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f393400a5c0 tx=0x7f3934011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45b97f9313e8679c server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:53:58.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.397+0000 7f391effd700 1 -- 172.21.15.52:0/1746194351 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3934021070 con 0x5589a58e55f0 2024-06-10T02:53:58.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.397+0000 7f391effd700 1 -- 172.21.15.52:0/1746194351 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3934010e50 con 0x5589a58e55f0 2024-06-10T02:53:58.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.397+0000 7f391effd700 1 -- 172.21.15.52:0/1746194351 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f393400f7a0 con 0x5589a58e55f0 2024-06-10T02:53:58.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.398+0000 7f3949832b80 1 -- 172.21.15.52:0/1746194351 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5589a58e55f0 msgr2=0x5589a58e59d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:53:58.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.398+0000 7f3949832b80 1 --2- 172.21.15.52:0/1746194351 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5589a58e55f0 0x5589a58e59d0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f393400a5c0 tx=0x7f3934011000 comp rx=0 tx=0).stop 2024-06-10T02:53:58.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.398+0000 7f3949832b80 1 -- 172.21.15.52:0/1746194351 shutdown_connections 2024-06-10T02:53:58.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.398+0000 7f3949832b80 1 --2- 172.21.15.52:0/1746194351 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5589a58dfed0 0x5589a58e02f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:58.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.398+0000 7f3949832b80 1 --2- 172.21.15.52:0/1746194351 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5589a58e55f0 0x5589a58e59d0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:58.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.398+0000 7f3949832b80 1 --2- 172.21.15.52:0/1746194351 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5589a58e5f10 0x5589a58df990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:58.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.398+0000 7f3949832b80 1 -- 172.21.15.52:0/1746194351 >> 172.21.15.52:0/1746194351 conn(0x5589a580ac70 msgr2=0x5589a58da800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:53:58.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.398+0000 7f3949832b80 1 -- 172.21.15.52:0/1746194351 shutdown_connections 2024-06-10T02:53:58.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.398+0000 7f3949832b80 1 -- 172.21.15.52:0/1746194351 wait complete. 2024-06-10T02:53:58.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.398+0000 7f3949832b80 1 -- start start 2024-06-10T02:53:58.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.399+0000 7f3949832b80 1 Processor -- start 2024-06-10T02:53:58.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.399+0000 7f3949832b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5589a58dfed0 0x5589a58ebc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:58.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.399+0000 7f3949832b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5589a58e55f0 0x5589a58e8f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:58.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.399+0000 7f3949832b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5589a58e5f10 0x5589a58e94b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:58.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.399+0000 7f3949832b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5589a58ec980 con 0x5589a58e5f10 2024-06-10T02:53:58.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.399+0000 7f3949832b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5589a57d1ba0 con 0x5589a58e55f0 2024-06-10T02:53:58.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.399+0000 7f3949832b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5589a58ecb00 con 0x5589a58dfed0 2024-06-10T02:53:58.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.399+0000 7f3949832b80 10 client.0 _ll_register_callbacks cb 0x5589a57cb8d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T02:53:58.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.399+0000 7f393d54c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5589a58e5f10 0x5589a58e94b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:58.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.399+0000 7f393d54c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5589a58e5f10 0x5589a58e94b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:42060/0 (socket says 172.21.15.52:42060) 2024-06-10T02:53:58.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.399+0000 7f393d54c700 1 -- 172.21.15.52:0/4108704642 learned_addr learned my addr 172.21.15.52:0/4108704642 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:53:58.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.399+0000 7f392ffff700 1 --2- 172.21.15.52:0/4108704642 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5589a58e55f0 0x5589a58e8f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:58.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.399+0000 7f393cd4b700 1 --2- 172.21.15.52:0/4108704642 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5589a58dfed0 0x5589a58ebc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:58.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.400+0000 7f393d54c700 1 -- 172.21.15.52:0/4108704642 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5589a58dfed0 msgr2=0x5589a58ebc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:53:58.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.400+0000 7f393d54c700 1 --2- 172.21.15.52:0/4108704642 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5589a58dfed0 0x5589a58ebc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:58.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.400+0000 7f393d54c700 1 -- 172.21.15.52:0/4108704642 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5589a58e55f0 msgr2=0x5589a58e8f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:53:58.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.400+0000 7f393d54c700 1 --2- 172.21.15.52:0/4108704642 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5589a58e55f0 0x5589a58e8f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:58.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.400+0000 7f393d54c700 1 -- 172.21.15.52:0/4108704642 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5589a5780880 con 0x5589a58e5f10 2024-06-10T02:53:58.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.400+0000 7f392ffff700 1 --2- 172.21.15.52:0/4108704642 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5589a58e55f0 0x5589a58e8f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T02:53:58.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.400+0000 7f393d54c700 1 --2- 172.21.15.52:0/4108704642 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5589a58e5f10 0x5589a58e94b0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f393000e930 tx=0x7f393000ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cf1368dcca6db93 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:53:58.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.400+0000 7f3917fff700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-10T02:53:58.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.400+0000 7f3917fff700 1 -- 172.21.15.52:0/4108704642 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f393000c9d0 con 0x5589a58e5f10 2024-06-10T02:53:58.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.400+0000 7f3917fff700 1 -- 172.21.15.52:0/4108704642 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f393000cb70 con 0x5589a58e5f10 2024-06-10T02:53:58.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.400+0000 7f3949832b80 1 -- 172.21.15.52:0/4108704642 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3934010e50 con 0x5589a58e5f10 2024-06-10T02:53:58.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.400+0000 7f3917fff700 1 -- 172.21.15.52:0/4108704642 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f3930023050 con 0x5589a58e5f10 2024-06-10T02:53:58.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.400+0000 7f3949832b80 10 client.15243 fetch_fsmap learned FSMap version 29 2024-06-10T02:53:58.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.400+0000 7f3949832b80 1 -- 172.21.15.52:0/4108704642 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5589a58f72e0 con 0x5589a58e5f10 2024-06-10T02:53:58.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.400+0000 7f3917fff700 1 -- 172.21.15.52:0/4108704642 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3934010e50 con 0x5589a58e5f10 2024-06-10T02:53:58.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.401+0000 7f3917fff700 1 -- 172.21.15.52:0/4108704642 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f393001a410 con 0x5589a58e5f10 2024-06-10T02:53:58.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.401+0000 7f3949832b80 10 client.15243 fetch_fsmap finished waiting for FSMap version 29 2024-06-10T02:53:58.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.401+0000 7f3949832b80 10 client.15243 resolve_mds: resolved name 'c' to daemon mds.c 2024-06-10T02:53:58.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.401+0000 7f3949832b80 20 client.15243 populate_metadata read hostname 'smithi052' 2024-06-10T02:53:58.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.401+0000 7f3949832b80 1 --2- 172.21.15.52:0/4108704642 >> [v2:172.21.15.188:6836/819317726,v1:172.21.15.188:6837/819317726] conn(0x5589a58f98d0 0x5589a58fad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:58.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.401+0000 7f3949832b80 4 client.15243 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T02:53:58.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.401+0000 7f3949832b80 1 -- 172.21.15.52:0/4108704642 --> [v2:172.21.15.188:6836/819317726,v1:172.21.15.188:6837/819317726] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5589a583ecc0 con 0x5589a58f98d0 2024-06-10T02:53:58.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.401+0000 7f393cd4b700 1 --2- 172.21.15.52:0/4108704642 >> [v2:172.21.15.188:6836/819317726,v1:172.21.15.188:6837/819317726] conn(0x5589a58f98d0 0x5589a58fad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:58.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.401+0000 7f393cd4b700 1 --2- 172.21.15.52:0/4108704642 >> [v2:172.21.15.188:6836/819317726,v1:172.21.15.188:6837/819317726] conn(0x5589a58f98d0 0x5589a58fad40 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c852de294fa690d2 server_cookie=d7d8b3c3c8a4b08d in_seq=0 out_seq=0 2024-06-10T02:53:58.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.401+0000 7f3917fff700 10 client.15243 ms_handle_connect on v2:172.21.15.188:6836/819317726 2024-06-10T02:53:58.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.402+0000 7f3917fff700 1 -- 172.21.15.52:0/4108704642 <== mds.1 v2:172.21.15.188:6836/819317726 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5589a583ecc0 con 0x5589a58f98d0 2024-06-10T02:53:58.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.402+0000 7f3917fff700 10 client.15243 handle_command_reply: tid=0 2024-06-10T02:53:58.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.402+0000 7f3949832b80 1 client.15243 shutdown 2024-06-10T02:53:58.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.402+0000 7f3949832b80 1 -- 172.21.15.52:0/4108704642 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5589a58e5f10 msgr2=0x5589a58e94b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:53:58.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.402+0000 7f3949832b80 1 --2- 172.21.15.52:0/4108704642 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5589a58e5f10 0x5589a58e94b0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f393000e930 tx=0x7f393000ec40 comp rx=0 tx=0).stop 2024-06-10T02:53:58.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.402+0000 7f3949832b80 1 -- 172.21.15.52:0/4108704642 shutdown_connections 2024-06-10T02:53:58.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.402+0000 7f3949832b80 1 --2- 172.21.15.52:0/4108704642 >> [v2:172.21.15.188:6836/819317726,v1:172.21.15.188:6837/819317726] conn(0x5589a58f98d0 0x5589a58fad40 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:58.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.402+0000 7f3949832b80 1 --2- 172.21.15.52:0/4108704642 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5589a58e5f10 0x5589a58e94b0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:58.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.402+0000 7f3949832b80 1 --2- 172.21.15.52:0/4108704642 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5589a58dfed0 0x5589a58ebc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:58.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.402+0000 7f3917fff700 0 client.15243 ms_handle_reset on v2:172.21.15.188:6836/819317726 2024-06-10T02:53:58.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.402+0000 7f3949832b80 1 --2- 172.21.15.52:0/4108704642 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5589a58e55f0 0x5589a58e8f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:58.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.402+0000 7f3949832b80 1 -- 172.21.15.52:0/4108704642 >> 172.21.15.52:0/4108704642 conn(0x5589a580ac70 msgr2=0x5589a58de980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:53:58.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.403+0000 7f3949832b80 1 -- 172.21.15.52:0/4108704642 shutdown_connections 2024-06-10T02:53:58.407 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.403+0000 7f3949832b80 1 -- 172.21.15.52:0/4108704642 wait complete. 2024-06-10T02:53:58.407 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.403+0000 7f3949832b80 20 client.15243 trim_cache size 0 max 16384 2024-06-10T02:53:58.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.424+0000 7f3949832b80 1 Processor -- start 2024-06-10T02:53:58.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.424+0000 7f3949832b80 1 -- start start 2024-06-10T02:53:58.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.424+0000 7f3949832b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5589a58dfed0 0x5589a58df990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:58.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.424+0000 7f3949832b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5589a58e55f0 0x5589a58e36e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:58.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.424+0000 7f3949832b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5589a58e5f10 0x5589a58e8870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:58.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.424+0000 7f3949832b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5589a58f9080 con 0x5589a58e5f10 2024-06-10T02:53:58.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.424+0000 7f3949832b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5589a58f9750 con 0x5589a58dfed0 2024-06-10T02:53:58.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.424+0000 7f3949832b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5589a58f9e20 con 0x5589a58e55f0 2024-06-10T02:53:58.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.424+0000 7f393d54c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5589a58e5f10 0x5589a58e8870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:58.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.425+0000 7f393d54c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5589a58e5f10 0x5589a58e8870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:42064/0 (socket says 172.21.15.52:42064) 2024-06-10T02:53:58.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.425+0000 7f393d54c700 1 -- 172.21.15.52:0/1574005658 learned_addr learned my addr 172.21.15.52:0/1574005658 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:53:58.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.425+0000 7f393cd4b700 1 --2- 172.21.15.52:0/1574005658 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5589a58dfed0 0x5589a58df990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:58.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.425+0000 7f392ffff700 1 --2- 172.21.15.52:0/1574005658 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5589a58e55f0 0x5589a58e36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:58.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.425+0000 7f393d54c700 1 -- 172.21.15.52:0/1574005658 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5589a58e55f0 msgr2=0x5589a58e36e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:53:58.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.425+0000 7f393d54c700 1 --2- 172.21.15.52:0/1574005658 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5589a58e55f0 0x5589a58e36e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:58.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.425+0000 7f393d54c700 1 -- 172.21.15.52:0/1574005658 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5589a58dfed0 msgr2=0x5589a58df990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:53:58.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.425+0000 7f393d54c700 1 --2- 172.21.15.52:0/1574005658 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5589a58dfed0 0x5589a58df990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:58.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.425+0000 7f393d54c700 1 -- 172.21.15.52:0/1574005658 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5589a58e74e0 con 0x5589a58e5f10 2024-06-10T02:53:58.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.425+0000 7f393d54c700 1 --2- 172.21.15.52:0/1574005658 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5589a58e5f10 0x5589a58e8870 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f393001a7e0 tx=0x7f393000c380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18b0b42a0f45ed1e server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:53:58.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.425+0000 7f391e7fc700 1 -- 172.21.15.52:0/1574005658 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f393002d030 con 0x5589a58e5f10 2024-06-10T02:53:58.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.425+0000 7f391e7fc700 1 -- 172.21.15.52:0/1574005658 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3930007a70 con 0x5589a58e5f10 2024-06-10T02:53:58.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.425+0000 7f391e7fc700 1 -- 172.21.15.52:0/1574005658 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f393000c9d0 con 0x5589a58e5f10 2024-06-10T02:53:58.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.426+0000 7f3949832b80 1 -- 172.21.15.52:0/1574005658 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5589a58e5f10 msgr2=0x5589a58e8870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:53:58.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.426+0000 7f3949832b80 1 --2- 172.21.15.52:0/1574005658 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5589a58e5f10 0x5589a58e8870 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f393001a7e0 tx=0x7f393000c380 comp rx=0 tx=0).stop 2024-06-10T02:53:58.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.426+0000 7f3949832b80 1 -- 172.21.15.52:0/1574005658 shutdown_connections 2024-06-10T02:53:58.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.426+0000 7f3949832b80 1 --2- 172.21.15.52:0/1574005658 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5589a58e5f10 0x5589a58e8870 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:58.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.426+0000 7f3949832b80 1 --2- 172.21.15.52:0/1574005658 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5589a58dfed0 0x5589a58df990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:58.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.426+0000 7f3949832b80 1 --2- 172.21.15.52:0/1574005658 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5589a58e55f0 0x5589a58e36e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:58.428 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.426+0000 7f3949832b80 1 -- 172.21.15.52:0/1574005658 >> 172.21.15.52:0/1574005658 conn(0x5589a580ac70 msgr2=0x5589a58e62f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:53:58.428 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.426+0000 7f3949832b80 1 -- 172.21.15.52:0/1574005658 shutdown_connections 2024-06-10T02:53:58.428 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.426+0000 7f3949832b80 1 -- 172.21.15.52:0/1574005658 wait complete. 2024-06-10T02:53:58.428 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.426+0000 7f3949832b80 1 -- start start 2024-06-10T02:53:58.428 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.426+0000 7f3949832b80 1 Processor -- start 2024-06-10T02:53:58.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.427+0000 7f3949832b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5589a58dfed0 0x5589a58e8220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:58.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.427+0000 7f3949832b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5589a58e55f0 0x5589a58e8760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:58.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.427+0000 7f3949832b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5589a58e5f10 0x5589a59035e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:58.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.427+0000 7f3949832b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5589a58fabc0 con 0x5589a58dfed0 2024-06-10T02:53:58.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.427+0000 7f3949832b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5589a58fa4f0 con 0x5589a58e55f0 2024-06-10T02:53:58.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.427+0000 7f3949832b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5589a58fb290 con 0x5589a58e5f10 2024-06-10T02:53:58.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.427+0000 7f3949832b80 10 client.0 _ll_register_callbacks cb 0x5589a57cb8d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T02:53:58.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.427+0000 7f393cd4b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5589a58dfed0 0x5589a58e8220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:58.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.427+0000 7f393cd4b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5589a58dfed0 0x5589a58e8220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:42070/0 (socket says 172.21.15.52:42070) 2024-06-10T02:53:58.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.427+0000 7f393cd4b700 1 -- 172.21.15.52:0/1656613486 learned_addr learned my addr 172.21.15.52:0/1656613486 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:53:58.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.427+0000 7f392ffff700 1 --2- 172.21.15.52:0/1656613486 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5589a58e55f0 0x5589a58e8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:58.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.427+0000 7f393d54c700 1 --2- 172.21.15.52:0/1656613486 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5589a58e5f10 0x5589a59035e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:58.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.427+0000 7f393cd4b700 1 -- 172.21.15.52:0/1656613486 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5589a58e5f10 msgr2=0x5589a59035e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:53:58.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.427+0000 7f393cd4b700 1 --2- 172.21.15.52:0/1656613486 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5589a58e5f10 0x5589a59035e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:58.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.427+0000 7f393cd4b700 1 -- 172.21.15.52:0/1656613486 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5589a58e55f0 msgr2=0x5589a58e8760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:53:58.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.427+0000 7f393cd4b700 1 --2- 172.21.15.52:0/1656613486 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5589a58e55f0 0x5589a58e8760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:58.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.427+0000 7f393cd4b700 1 -- 172.21.15.52:0/1656613486 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5589a5903e20 con 0x5589a58dfed0 2024-06-10T02:53:58.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.427+0000 7f393d54c700 1 --2- 172.21.15.52:0/1656613486 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5589a58e5f10 0x5589a59035e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T02:53:58.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.427+0000 7f393cd4b700 1 --2- 172.21.15.52:0/1656613486 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5589a58dfed0 0x5589a58e8220 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f393400a5c0 tx=0x7f3934010e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c638dc94e5531c3d server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:53:58.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.427+0000 7f3917fff700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-10T02:53:58.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.428+0000 7f3917fff700 1 -- 172.21.15.52:0/1656613486 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3934021040 con 0x5589a58dfed0 2024-06-10T02:53:58.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.428+0000 7f3949832b80 1 -- 172.21.15.52:0/1656613486 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3930007a70 con 0x5589a58dfed0 2024-06-10T02:53:58.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.428+0000 7f3917fff700 1 -- 172.21.15.52:0/1656613486 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f3934004d20 con 0x5589a58dfed0 2024-06-10T02:53:58.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.428+0000 7f3917fff700 1 -- 172.21.15.52:0/1656613486 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f3934046050 con 0x5589a58dfed0 2024-06-10T02:53:58.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.428+0000 7f3949832b80 10 client.15249 fetch_fsmap learned FSMap version 29 2024-06-10T02:53:58.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.428+0000 7f3949832b80 1 -- 172.21.15.52:0/1656613486 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5589a5969d40 con 0x5589a58dfed0 2024-06-10T02:53:58.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.428+0000 7f3917fff700 1 -- 172.21.15.52:0/1656613486 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3930007a70 con 0x5589a58dfed0 2024-06-10T02:53:58.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.428+0000 7f3917fff700 1 -- 172.21.15.52:0/1656613486 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f393401d820 con 0x5589a58dfed0 2024-06-10T02:53:58.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.428+0000 7f3949832b80 10 client.15249 fetch_fsmap finished waiting for FSMap version 29 2024-06-10T02:53:58.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.428+0000 7f3949832b80 10 client.15249 resolve_mds: resolved name 'c' to daemon mds.c 2024-06-10T02:53:58.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.428+0000 7f3949832b80 20 client.15249 populate_metadata read hostname 'smithi052' 2024-06-10T02:53:58.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.428+0000 7f3949832b80 1 --2- 172.21.15.52:0/1656613486 >> [v2:172.21.15.188:6836/819317726,v1:172.21.15.188:6837/819317726] conn(0x5589a596e600 0x5589a596d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:53:58.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.428+0000 7f3949832b80 4 client.15249 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "status"}] 2024-06-10T02:53:58.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.428+0000 7f3949832b80 1 -- 172.21.15.52:0/1656613486 --> [v2:172.21.15.188:6836/819317726,v1:172.21.15.188:6837/819317726] -- command(tid 0: {"prefix": "status"}) -- 0x5589a5839240 con 0x5589a596e600 2024-06-10T02:53:58.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.430+0000 7f392ffff700 1 --2- 172.21.15.52:0/1656613486 >> [v2:172.21.15.188:6836/819317726,v1:172.21.15.188:6837/819317726] conn(0x5589a596e600 0x5589a596d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:53:58.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.431+0000 7f392ffff700 1 --2- 172.21.15.52:0/1656613486 >> [v2:172.21.15.188:6836/819317726,v1:172.21.15.188:6837/819317726] conn(0x5589a596e600 0x5589a596d310 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ea2c0599b79aed35 server_cookie=1742b3825b2404bc in_seq=0 out_seq=0 2024-06-10T02:53:58.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.431+0000 7f3917fff700 10 client.15249 ms_handle_connect on v2:172.21.15.188:6836/819317726 2024-06-10T02:53:58.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.432+0000 7f3917fff700 1 -- 172.21.15.52:0/1656613486 <== mds.1 v2:172.21.15.188:6836/819317726 1 ==== command_reply(tid 0: 0 ) ==== 8+0+333 (crc 0 0 0) 0x5589a5839240 con 0x5589a596e600 2024-06-10T02:53:58.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.432+0000 7f3917fff700 10 client.15249 handle_command_reply: tid=0 2024-06-10T02:53:58.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.432+0000 7f3949832b80 1 client.15249 shutdown 2024-06-10T02:53:58.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.432+0000 7f3949832b80 1 -- 172.21.15.52:0/1656613486 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5589a58dfed0 msgr2=0x5589a58e8220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:53:58.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.432+0000 7f3949832b80 1 --2- 172.21.15.52:0/1656613486 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5589a58dfed0 0x5589a58e8220 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f393400a5c0 tx=0x7f3934010e50 comp rx=0 tx=0).stop 2024-06-10T02:53:58.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.432+0000 7f3949832b80 1 -- 172.21.15.52:0/1656613486 shutdown_connections 2024-06-10T02:53:58.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.432+0000 7f3949832b80 1 --2- 172.21.15.52:0/1656613486 >> [v2:172.21.15.188:6836/819317726,v1:172.21.15.188:6837/819317726] conn(0x5589a596e600 0x5589a596d310 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:58.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.432+0000 7f3917fff700 0 client.15249 ms_handle_reset on v2:172.21.15.188:6836/819317726 2024-06-10T02:53:58.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.432+0000 7f3949832b80 1 --2- 172.21.15.52:0/1656613486 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5589a58e5f10 0x5589a59035e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:58.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.432+0000 7f3949832b80 1 --2- 172.21.15.52:0/1656613486 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5589a58dfed0 0x5589a58e8220 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:58.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.432+0000 7f3949832b80 1 --2- 172.21.15.52:0/1656613486 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5589a58e55f0 0x5589a58e8760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:58.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.432+0000 7f3949832b80 1 -- 172.21.15.52:0/1656613486 >> 172.21.15.52:0/1656613486 conn(0x5589a580ac70 msgr2=0x5589a58e62f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:53:58.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.432+0000 7f3949832b80 1 -- 172.21.15.52:0/1656613486 shutdown_connections 2024-06-10T02:53:58.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.432+0000 7f3949832b80 1 -- 172.21.15.52:0/1656613486 wait complete. 2024-06-10T02:53:58.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.432+0000 7f3949832b80 20 client.15249 trim_cache size 0 max 16384 2024-06-10T02:53:58.435 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-10T02:53:58.435 INFO:teuthology.orchestra.run.smithi052.stdout: "cluster_fsid": "d8676e9e-26d2-11ef-bc9b-c7b262605968", 2024-06-10T02:53:58.435 INFO:teuthology.orchestra.run.smithi052.stdout: "whoami": 1, 2024-06-10T02:53:58.435 INFO:teuthology.orchestra.run.smithi052.stdout: "id": 24469, 2024-06-10T02:53:58.435 INFO:teuthology.orchestra.run.smithi052.stdout: "want_state": "up:active", 2024-06-10T02:53:58.435 INFO:teuthology.orchestra.run.smithi052.stdout: "state": "up:active", 2024-06-10T02:53:58.436 INFO:teuthology.orchestra.run.smithi052.stdout: "fs_name": "cephfs", 2024-06-10T02:53:58.436 INFO:teuthology.orchestra.run.smithi052.stdout: "rank_uptime": 172.83685343400001, 2024-06-10T02:53:58.436 INFO:teuthology.orchestra.run.smithi052.stdout: "mdsmap_epoch": 29, 2024-06-10T02:53:58.436 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_epoch": 100, 2024-06-10T02:53:58.436 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_epoch_barrier": 98, 2024-06-10T02:53:58.436 INFO:teuthology.orchestra.run.smithi052.stdout: "uptime": 268.01704355800001 2024-06-10T02:53:58.436 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-10T02:53:58.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.433+0000 7f393efaf700 1 -- 172.21.15.52:0/1365239612 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f39200a2ca0 msgr2=0x7f39200a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:53:58.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.433+0000 7f393efaf700 1 --2- 172.21.15.52:0/1365239612 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f39200a2ca0 0x7f39200a5160 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f3934005fd0 tx=0x7f3934005e30 comp rx=0 tx=0).stop 2024-06-10T02:53:58.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.433+0000 7f393efaf700 1 -- 172.21.15.52:0/1365239612 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3938101fc0 msgr2=0x7f3938199fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:53:58.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.433+0000 7f393efaf700 1 --2- 172.21.15.52:0/1365239612 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3938101fc0 0x7f3938199fd0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f3928000c00 tx=0x7f392800bfb0 comp rx=0 tx=0).stop 2024-06-10T02:53:58.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.433+0000 7f393efaf700 1 -- 172.21.15.52:0/1365239612 shutdown_connections 2024-06-10T02:53:58.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.433+0000 7f393efaf700 1 --2- 172.21.15.52:0/1365239612 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f39381066f0 0x7f39381a4b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:58.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.433+0000 7f393efaf700 1 --2- 172.21.15.52:0/1365239612 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3938101610 0x7f3938199a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:58.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.433+0000 7f393efaf700 1 --2- 172.21.15.52:0/1365239612 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f39200a2ca0 0x7f39200a5160 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:58.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.433+0000 7f393efaf700 1 --2- 172.21.15.52:0/1365239612 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3938101fc0 0x7f3938199fd0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:53:58.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.433+0000 7f393efaf700 1 -- 172.21.15.52:0/1365239612 >> 172.21.15.52:0/1365239612 conn(0x7f3938078430 msgr2=0x7f39380ff2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:53:58.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.433+0000 7f393efaf700 1 -- 172.21.15.52:0/1365239612 shutdown_connections 2024-06-10T02:53:58.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:53:58.433+0000 7f393efaf700 1 -- 172.21.15.52:0/1365239612 wait complete. 2024-06-10T02:53:59.127 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:53:58 smithi052 ceph-mon[32362]: pgmap v427: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-10T02:53:59.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:53:58 smithi120 ceph-mon[36634]: pgmap v427: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-10T02:53:59.160 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph tell mds.i status 2024-06-10T02:53:59.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:53:58 smithi188 ceph-mon[36599]: pgmap v427: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-10T02:53:59.415 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:54:00.875 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:54:00 smithi052 ceph-mon[32362]: pgmap v428: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-10T02:54:01.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.123+0000 7ffa25b8c700 1 -- 172.21.15.52:0/3033794456 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffa2006b580 msgr2=0x7ffa2010f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:54:01.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.123+0000 7ffa25b8c700 1 --2- 172.21.15.52:0/3033794456 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffa2006b580 0x7ffa2010f320 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7ffa1800b0a0 tx=0x7ffa1800b3b0 comp rx=0 tx=0).stop 2024-06-10T02:54:01.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.123+0000 7ffa25b8c700 1 -- 172.21.15.52:0/3033794456 shutdown_connections 2024-06-10T02:54:01.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.123+0000 7ffa25b8c700 1 --2- 172.21.15.52:0/3033794456 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffa2006b580 0x7ffa2010f320 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:01.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.123+0000 7ffa25b8c700 1 --2- 172.21.15.52:0/3033794456 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffa20108500 0x7ffa201088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:01.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.123+0000 7ffa25b8c700 1 --2- 172.21.15.52:0/3033794456 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffa2006abc0 0x7ffa2006b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:01.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.123+0000 7ffa25b8c700 1 -- 172.21.15.52:0/3033794456 >> 172.21.15.52:0/3033794456 conn(0x7ffa20078340 msgr2=0x7ffa20078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:54:01.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.123+0000 7ffa25b8c700 1 -- 172.21.15.52:0/3033794456 shutdown_connections 2024-06-10T02:54:01.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.123+0000 7ffa25b8c700 1 -- 172.21.15.52:0/3033794456 wait complete. 2024-06-10T02:54:01.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.124+0000 7ffa25b8c700 1 Processor -- start 2024-06-10T02:54:01.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.124+0000 7ffa25b8c700 1 -- start start 2024-06-10T02:54:01.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.125+0000 7ffa25b8c700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffa2006abc0 0x7ffa2019e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:01.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.125+0000 7ffa25b8c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffa2006b580 0x7ffa2019ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:01.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.125+0000 7ffa25b8c700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffa20108500 0x7ffa20198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:01.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.125+0000 7ffa25b8c700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ffa20112760 con 0x7ffa2006b580 2024-06-10T02:54:01.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.125+0000 7ffa25b8c700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ffa201125e0 con 0x7ffa2006abc0 2024-06-10T02:54:01.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.125+0000 7ffa25b8c700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ffa201128e0 con 0x7ffa20108500 2024-06-10T02:54:01.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.125+0000 7ffa16ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffa2006b580 0x7ffa2019ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:01.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.125+0000 7ffa16ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffa2006b580 0x7ffa2019ede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:38866/0 (socket says 172.21.15.52:38866) 2024-06-10T02:54:01.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.125+0000 7ffa16ffd700 1 -- 172.21.15.52:0/2335875971 learned_addr learned my addr 172.21.15.52:0/2335875971 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:54:01.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.125+0000 7ffa1f7fe700 1 --2- 172.21.15.52:0/2335875971 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffa2006abc0 0x7ffa2019e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:01.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.125+0000 7ffa1ffff700 1 --2- 172.21.15.52:0/2335875971 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffa20108500 0x7ffa20198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:01.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.125+0000 7ffa16ffd700 1 -- 172.21.15.52:0/2335875971 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffa20108500 msgr2=0x7ffa20198a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:54:01.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.125+0000 7ffa16ffd700 1 --2- 172.21.15.52:0/2335875971 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffa20108500 0x7ffa20198a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:01.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.125+0000 7ffa16ffd700 1 -- 172.21.15.52:0/2335875971 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffa2006abc0 msgr2=0x7ffa2019e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:54:01.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.125+0000 7ffa16ffd700 1 --2- 172.21.15.52:0/2335875971 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffa2006abc0 0x7ffa2019e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:01.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.125+0000 7ffa16ffd700 1 -- 172.21.15.52:0/2335875971 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa201991c0 con 0x7ffa2006b580 2024-06-10T02:54:01.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.125+0000 7ffa1ffff700 1 --2- 172.21.15.52:0/2335875971 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffa20108500 0x7ffa20198a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T02:54:01.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.125+0000 7ffa16ffd700 1 --2- 172.21.15.52:0/2335875971 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffa2006b580 0x7ffa2019ede0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7ffa0800d950 tx=0x7ffa0800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:54:01.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.126+0000 7ffa1d7fa700 1 -- 172.21.15.52:0/2335875971 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa08016070 con 0x7ffa2006b580 2024-06-10T02:54:01.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.126+0000 7ffa1d7fa700 1 -- 172.21.15.52:0/2335875971 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa08009e20 con 0x7ffa2006b580 2024-06-10T02:54:01.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.126+0000 7ffa1d7fa700 1 -- 172.21.15.52:0/2335875971 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa0800f620 con 0x7ffa2006b580 2024-06-10T02:54:01.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.126+0000 7ffa25b8c700 1 -- 172.21.15.52:0/2335875971 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa201994b0 con 0x7ffa2006b580 2024-06-10T02:54:01.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.126+0000 7ffa25b8c700 1 -- 172.21.15.52:0/2335875971 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa201a5770 con 0x7ffa2006b580 2024-06-10T02:54:01.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.128+0000 7ffa1d7fa700 1 -- 172.21.15.52:0/2335875971 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ffa08019070 con 0x7ffa2006b580 2024-06-10T02:54:01.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.129+0000 7ffa1d7fa700 1 --2- 172.21.15.52:0/2335875971 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7ffa0c0a2c20 0x7ffa0c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:01.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.129+0000 7ffa1f7fe700 1 --2- 172.21.15.52:0/2335875971 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7ffa0c0a2c20 0x7ffa0c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:01.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.129+0000 7ffa1d7fa700 1 -- 172.21.15.52:0/2335875971 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7ffa080cefe0 con 0x7ffa2006b580 2024-06-10T02:54:01.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.130+0000 7ffa1f7fe700 1 --2- 172.21.15.52:0/2335875971 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7ffa0c0a2c20 0x7ffa0c0a50e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7ffa20199ba0 tx=0x7ffa1000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:54:01.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.130+0000 7ffa3040fb80 1 Processor -- start 2024-06-10T02:54:01.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.130+0000 7ffa3040fb80 1 -- start start 2024-06-10T02:54:01.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.130+0000 7ffa3040fb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5654f40cbf80 0x5654f40cc360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:01.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.130+0000 7ffa3040fb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5654f40cc8a0 0x5654f40d0670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:01.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.130+0000 7ffa3040fb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5654f40d0bb0 0x5654f40d1010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:01.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.130+0000 7ffa3040fb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5654f40197c0 con 0x5654f40cc8a0 2024-06-10T02:54:01.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.130+0000 7ffa3040fb80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5654f3fc16d0 con 0x5654f40cbf80 2024-06-10T02:54:01.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.130+0000 7ffa3040fb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5654f4015780 con 0x5654f40d0bb0 2024-06-10T02:54:01.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.130+0000 7ffa16ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5654f40cc8a0 0x5654f40d0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:01.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.130+0000 7ffa16ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5654f40cc8a0 0x5654f40d0670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:38870/0 (socket says 172.21.15.52:38870) 2024-06-10T02:54:01.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.130+0000 7ffa16ffd700 1 -- 172.21.15.52:0/305580080 learned_addr learned my addr 172.21.15.52:0/305580080 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:54:01.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.131+0000 7ffa1f7fe700 1 --2- 172.21.15.52:0/305580080 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5654f40cbf80 0x5654f40cc360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:01.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.131+0000 7ffa1ffff700 1 --2- 172.21.15.52:0/305580080 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5654f40d0bb0 0x5654f40d1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:01.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.131+0000 7ffa16ffd700 1 -- 172.21.15.52:0/305580080 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5654f40d0bb0 msgr2=0x5654f40d1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:54:01.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.131+0000 7ffa16ffd700 1 --2- 172.21.15.52:0/305580080 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5654f40d0bb0 0x5654f40d1010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:01.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.131+0000 7ffa16ffd700 1 -- 172.21.15.52:0/305580080 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5654f40cbf80 msgr2=0x5654f40cc360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:54:01.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.131+0000 7ffa16ffd700 1 --2- 172.21.15.52:0/305580080 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5654f40cbf80 0x5654f40cc360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:01.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.131+0000 7ffa16ffd700 1 -- 172.21.15.52:0/305580080 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654f3fb7f00 con 0x5654f40cc8a0 2024-06-10T02:54:01.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.131+0000 7ffa16ffd700 1 --2- 172.21.15.52:0/305580080 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5654f40cc8a0 0x5654f40d0670 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7ffa080978d0 tx=0x7ffa08005910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bea0129003c7de59 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:54:01.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.131+0000 7ffa15ffb700 1 -- 172.21.15.52:0/305580080 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa08020070 con 0x5654f40cc8a0 2024-06-10T02:54:01.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.131+0000 7ffa15ffb700 1 -- 172.21.15.52:0/305580080 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa080a40e0 con 0x5654f40cc8a0 2024-06-10T02:54:01.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.131+0000 7ffa15ffb700 1 -- 172.21.15.52:0/305580080 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa080ccdd0 con 0x5654f40cc8a0 2024-06-10T02:54:01.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.132+0000 7ffa3040fb80 1 -- 172.21.15.52:0/305580080 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5654f40cc8a0 msgr2=0x5654f40d0670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:54:01.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.132+0000 7ffa3040fb80 1 --2- 172.21.15.52:0/305580080 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5654f40cc8a0 0x5654f40d0670 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7ffa080978d0 tx=0x7ffa08005910 comp rx=0 tx=0).stop 2024-06-10T02:54:01.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.132+0000 7ffa3040fb80 1 -- 172.21.15.52:0/305580080 shutdown_connections 2024-06-10T02:54:01.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.132+0000 7ffa3040fb80 1 --2- 172.21.15.52:0/305580080 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5654f40d0bb0 0x5654f40d1010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:01.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.132+0000 7ffa3040fb80 1 --2- 172.21.15.52:0/305580080 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5654f40cbf80 0x5654f40cc360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:01.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.132+0000 7ffa3040fb80 1 --2- 172.21.15.52:0/305580080 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5654f40cc8a0 0x5654f40d0670 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:01.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.132+0000 7ffa3040fb80 1 -- 172.21.15.52:0/305580080 >> 172.21.15.52:0/305580080 conn(0x5654f3ff2c70 msgr2=0x5654f40c2800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:54:01.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.132+0000 7ffa3040fb80 1 -- 172.21.15.52:0/305580080 shutdown_connections 2024-06-10T02:54:01.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.132+0000 7ffa3040fb80 1 -- 172.21.15.52:0/305580080 wait complete. 2024-06-10T02:54:01.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.132+0000 7ffa3040fb80 1 -- start start 2024-06-10T02:54:01.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.133+0000 7ffa3040fb80 1 Processor -- start 2024-06-10T02:54:01.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.133+0000 7ffa3040fb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5654f40cbf80 0x5654f40d80c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:01.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.133+0000 7ffa3040fb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5654f40cc8a0 0x5654f40d5430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:01.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.133+0000 7ffa3040fb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5654f40d0bb0 0x5654f40d5970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:01.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.133+0000 7ffa3040fb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5654f40d8d10 con 0x5654f40cc8a0 2024-06-10T02:54:01.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.133+0000 7ffa3040fb80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5654f3fb9ba0 con 0x5654f40d0bb0 2024-06-10T02:54:01.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.133+0000 7ffa3040fb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5654f40d8e90 con 0x5654f40cbf80 2024-06-10T02:54:01.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.133+0000 7ffa3040fb80 10 client.0 _ll_register_callbacks cb 0x5654f3fb38d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T02:54:01.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.133+0000 7ffa16ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5654f40cc8a0 0x5654f40d5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:01.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.133+0000 7ffa16ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5654f40cc8a0 0x5654f40d5430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:38878/0 (socket says 172.21.15.52:38878) 2024-06-10T02:54:01.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.133+0000 7ffa16ffd700 1 -- 172.21.15.52:0/1155277632 learned_addr learned my addr 172.21.15.52:0/1155277632 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:54:01.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.133+0000 7ffa1f7fe700 1 --2- 172.21.15.52:0/1155277632 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5654f40cbf80 0x5654f40d80c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:01.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.133+0000 7ffa1ffff700 1 --2- 172.21.15.52:0/1155277632 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5654f40d0bb0 0x5654f40d5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:01.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.133+0000 7ffa16ffd700 1 -- 172.21.15.52:0/1155277632 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5654f40cbf80 msgr2=0x5654f40d80c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:54:01.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.133+0000 7ffa16ffd700 1 --2- 172.21.15.52:0/1155277632 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5654f40cbf80 0x5654f40d80c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:01.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.133+0000 7ffa16ffd700 1 -- 172.21.15.52:0/1155277632 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5654f40d0bb0 msgr2=0x5654f40d5970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:54:01.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.133+0000 7ffa16ffd700 1 --2- 172.21.15.52:0/1155277632 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5654f40d0bb0 0x5654f40d5970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:01.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.133+0000 7ffa16ffd700 1 -- 172.21.15.52:0/1155277632 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654f3f68880 con 0x5654f40cc8a0 2024-06-10T02:54:01.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.134+0000 7ffa16ffd700 1 --2- 172.21.15.52:0/1155277632 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5654f40cc8a0 0x5654f40d5430 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7ffa0800b2f0 tx=0x7ffa08097790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb6cfb44f217be63 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:54:01.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.134+0000 7ff9feffd700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-10T02:54:01.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.134+0000 7ff9feffd700 1 -- 172.21.15.52:0/1155277632 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa08020070 con 0x5654f40cc8a0 2024-06-10T02:54:01.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.134+0000 7ff9feffd700 1 -- 172.21.15.52:0/1155277632 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7ffa0809b6d0 con 0x5654f40cc8a0 2024-06-10T02:54:01.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.134+0000 7ffa3040fb80 1 -- 172.21.15.52:0/1155277632 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa080a40e0 con 0x5654f40cc8a0 2024-06-10T02:54:01.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.134+0000 7ff9feffd700 1 -- 172.21.15.52:0/1155277632 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ffa0809b8d0 con 0x5654f40cc8a0 2024-06-10T02:54:01.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.134+0000 7ffa3040fb80 10 client.15261 fetch_fsmap learned FSMap version 29 2024-06-10T02:54:01.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.134+0000 7ffa3040fb80 1 -- 172.21.15.52:0/1155277632 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5654f40e36a0 con 0x5654f40cc8a0 2024-06-10T02:54:01.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.134+0000 7ff9feffd700 1 -- 172.21.15.52:0/1155277632 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa080a40e0 con 0x5654f40cc8a0 2024-06-10T02:54:01.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.134+0000 7ff9feffd700 1 -- 172.21.15.52:0/1155277632 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7ffa0809a8c0 con 0x5654f40cc8a0 2024-06-10T02:54:01.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.134+0000 7ffa3040fb80 10 client.15261 fetch_fsmap finished waiting for FSMap version 29 2024-06-10T02:54:01.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.134+0000 7ffa3040fb80 10 client.15261 resolve_mds: resolved name 'i' to daemon mds.i 2024-06-10T02:54:01.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.134+0000 7ffa3040fb80 20 client.15261 populate_metadata read hostname 'smithi052' 2024-06-10T02:54:01.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.134+0000 7ffa3040fb80 1 --2- 172.21.15.52:0/1155277632 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x5654f40e5860 0x5654f40e6cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:01.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.134+0000 7ffa3040fb80 4 client.15261 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T02:54:01.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.134+0000 7ffa3040fb80 1 -- 172.21.15.52:0/1155277632 --> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5654f4026cc0 con 0x5654f40e5860 2024-06-10T02:54:01.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.135+0000 7ffa1f7fe700 1 --2- 172.21.15.52:0/1155277632 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x5654f40e5860 0x5654f40e6cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:01.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.135+0000 7ffa1f7fe700 1 --2- 172.21.15.52:0/1155277632 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x5654f40e5860 0x5654f40e6cd0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b1912e959ed0bb0b server_cookie=3864d147643cd250 in_seq=0 out_seq=0 2024-06-10T02:54:01.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.135+0000 7ff9feffd700 10 client.15261 ms_handle_connect on v2:172.21.15.188:6838/310444836 2024-06-10T02:54:01.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.136+0000 7ff9feffd700 1 -- 172.21.15.52:0/1155277632 <== mds.0 v2:172.21.15.188:6838/310444836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5654f4026cc0 con 0x5654f40e5860 2024-06-10T02:54:01.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.136+0000 7ff9feffd700 10 client.15261 handle_command_reply: tid=0 2024-06-10T02:54:01.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.136+0000 7ffa3040fb80 1 client.15261 shutdown 2024-06-10T02:54:01.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.136+0000 7ffa3040fb80 1 -- 172.21.15.52:0/1155277632 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5654f40cc8a0 msgr2=0x5654f40d5430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:54:01.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.136+0000 7ffa3040fb80 1 --2- 172.21.15.52:0/1155277632 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5654f40cc8a0 0x5654f40d5430 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7ffa0800b2f0 tx=0x7ffa08097790 comp rx=0 tx=0).stop 2024-06-10T02:54:01.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.136+0000 7ffa3040fb80 1 -- 172.21.15.52:0/1155277632 shutdown_connections 2024-06-10T02:54:01.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.136+0000 7ffa3040fb80 1 --2- 172.21.15.52:0/1155277632 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x5654f40e5860 0x5654f40e6cd0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:01.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.136+0000 7ffa3040fb80 1 --2- 172.21.15.52:0/1155277632 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5654f40d0bb0 0x5654f40d5970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:01.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.136+0000 7ffa3040fb80 1 --2- 172.21.15.52:0/1155277632 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5654f40cbf80 0x5654f40d80c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:01.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.136+0000 7ff9feffd700 0 client.15261 ms_handle_reset on v2:172.21.15.188:6838/310444836 2024-06-10T02:54:01.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.136+0000 7ffa3040fb80 1 --2- 172.21.15.52:0/1155277632 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5654f40cc8a0 0x5654f40d5430 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:01.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.136+0000 7ffa3040fb80 1 -- 172.21.15.52:0/1155277632 >> 172.21.15.52:0/1155277632 conn(0x5654f3ff2c70 msgr2=0x5654f40c6980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:54:01.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.136+0000 7ffa3040fb80 1 -- 172.21.15.52:0/1155277632 shutdown_connections 2024-06-10T02:54:01.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.136+0000 7ffa3040fb80 1 -- 172.21.15.52:0/1155277632 wait complete. 2024-06-10T02:54:01.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.136+0000 7ffa3040fb80 20 client.15261 trim_cache size 0 max 16384 2024-06-10T02:54:01.140 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:54:00 smithi120 ceph-mon[36634]: pgmap v428: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-10T02:54:01.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.158+0000 7ffa3040fb80 1 Processor -- start 2024-06-10T02:54:01.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.158+0000 7ffa3040fb80 1 -- start start 2024-06-10T02:54:01.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.158+0000 7ffa3040fb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5654f40cbf80 0x5654f40bc640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:01.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.158+0000 7ffa3040fb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5654f40cc8a0 0x5654f40c6980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:01.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.158+0000 7ffa3040fb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5654f40d0bb0 0x5654f40ce570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:01.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.158+0000 7ffa3040fb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5654f40e5010 con 0x5654f40cc8a0 2024-06-10T02:54:01.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.158+0000 7ffa3040fb80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5654f40e56e0 con 0x5654f40d0bb0 2024-06-10T02:54:01.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.158+0000 7ffa3040fb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5654f40e5db0 con 0x5654f40cbf80 2024-06-10T02:54:01.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.158+0000 7ffa16ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5654f40cc8a0 0x5654f40c6980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:01.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.158+0000 7ffa16ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5654f40cc8a0 0x5654f40c6980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:38886/0 (socket says 172.21.15.52:38886) 2024-06-10T02:54:01.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.158+0000 7ffa16ffd700 1 -- 172.21.15.52:0/38644403 learned_addr learned my addr 172.21.15.52:0/38644403 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:54:01.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.158+0000 7ffa1f7fe700 1 --2- 172.21.15.52:0/38644403 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5654f40cbf80 0x5654f40bc640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:01.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.159+0000 7ffa16ffd700 1 -- 172.21.15.52:0/38644403 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5654f40cbf80 msgr2=0x5654f40bc640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:54:01.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.159+0000 7ffa16ffd700 1 --2- 172.21.15.52:0/38644403 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5654f40cbf80 0x5654f40bc640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:01.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.159+0000 7ffa1ffff700 1 --2- 172.21.15.52:0/38644403 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5654f40d0bb0 0x5654f40ce570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:01.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.159+0000 7ffa16ffd700 1 -- 172.21.15.52:0/38644403 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5654f40d0bb0 msgr2=0x5654f40ce570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:54:01.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.159+0000 7ffa16ffd700 1 --2- 172.21.15.52:0/38644403 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5654f40d0bb0 0x5654f40ce570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:01.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.159+0000 7ffa16ffd700 1 -- 172.21.15.52:0/38644403 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654f40e5500 con 0x5654f40cc8a0 2024-06-10T02:54:01.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.159+0000 7ffa1f7fe700 1 --2- 172.21.15.52:0/38644403 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5654f40cbf80 0x5654f40bc640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T02:54:01.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.159+0000 7ffa16ffd700 1 --2- 172.21.15.52:0/38644403 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5654f40cc8a0 0x5654f40c6980 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7ffa080978d0 tx=0x7ffa08005910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aca8347cfb7de16d server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:54:01.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.159+0000 7ffa157fa700 1 -- 172.21.15.52:0/38644403 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa08020040 con 0x5654f40cc8a0 2024-06-10T02:54:01.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.159+0000 7ffa157fa700 1 -- 172.21.15.52:0/38644403 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa08099770 con 0x5654f40cc8a0 2024-06-10T02:54:01.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.159+0000 7ffa157fa700 1 -- 172.21.15.52:0/38644403 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa080b4290 con 0x5654f40cc8a0 2024-06-10T02:54:01.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.160+0000 7ffa3040fb80 1 -- 172.21.15.52:0/38644403 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5654f40cc8a0 msgr2=0x5654f40c6980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:54:01.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.160+0000 7ffa3040fb80 1 --2- 172.21.15.52:0/38644403 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5654f40cc8a0 0x5654f40c6980 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7ffa080978d0 tx=0x7ffa08005910 comp rx=0 tx=0).stop 2024-06-10T02:54:01.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.160+0000 7ffa3040fb80 1 -- 172.21.15.52:0/38644403 shutdown_connections 2024-06-10T02:54:01.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.160+0000 7ffa3040fb80 1 --2- 172.21.15.52:0/38644403 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5654f40d0bb0 0x5654f40ce570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:01.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.160+0000 7ffa3040fb80 1 --2- 172.21.15.52:0/38644403 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5654f40cbf80 0x5654f40bc640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:01.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.160+0000 7ffa3040fb80 1 --2- 172.21.15.52:0/38644403 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5654f40cc8a0 0x5654f40c6980 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:01.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.160+0000 7ffa3040fb80 1 -- 172.21.15.52:0/38644403 >> 172.21.15.52:0/38644403 conn(0x5654f3ff2c70 msgr2=0x5654f40c7230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:54:01.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.160+0000 7ffa3040fb80 1 -- 172.21.15.52:0/38644403 shutdown_connections 2024-06-10T02:54:01.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.160+0000 7ffa3040fb80 1 -- 172.21.15.52:0/38644403 wait complete. 2024-06-10T02:54:01.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.160+0000 7ffa3040fb80 1 -- start start 2024-06-10T02:54:01.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.160+0000 7ffa3040fb80 1 Processor -- start 2024-06-10T02:54:01.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.161+0000 7ffa3040fb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5654f40cbf80 0x5654f40f07e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:01.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.161+0000 7ffa3040fb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5654f40cc8a0 0x5654f40ebb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:01.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.161+0000 7ffa3040fb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5654f40d0bb0 0x5654f40ec0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:01.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.161+0000 7ffa3040fb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5654f40e6b50 con 0x5654f40d0bb0 2024-06-10T02:54:01.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.161+0000 7ffa3040fb80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5654f40e6480 con 0x5654f40cbf80 2024-06-10T02:54:01.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.161+0000 7ffa3040fb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5654f40e7220 con 0x5654f40cc8a0 2024-06-10T02:54:01.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.161+0000 7ffa3040fb80 10 client.0 _ll_register_callbacks cb 0x5654f3fb38d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T02:54:01.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.161+0000 7ffa1ffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5654f40d0bb0 0x5654f40ec0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:01.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.161+0000 7ffa1ffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5654f40d0bb0 0x5654f40ec0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:38894/0 (socket says 172.21.15.52:38894) 2024-06-10T02:54:01.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.161+0000 7ffa1ffff700 1 -- 172.21.15.52:0/2953224 learned_addr learned my addr 172.21.15.52:0/2953224 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:54:01.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.161+0000 7ffa16ffd700 1 --2- 172.21.15.52:0/2953224 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5654f40cc8a0 0x5654f40ebb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:01.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.161+0000 7ffa1f7fe700 1 --2- 172.21.15.52:0/2953224 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5654f40cbf80 0x5654f40f07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:01.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.161+0000 7ffa1ffff700 1 -- 172.21.15.52:0/2953224 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5654f40cc8a0 msgr2=0x5654f40ebb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:54:01.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.161+0000 7ffa1ffff700 1 --2- 172.21.15.52:0/2953224 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5654f40cc8a0 0x5654f40ebb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:01.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.161+0000 7ffa1ffff700 1 -- 172.21.15.52:0/2953224 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5654f40cbf80 msgr2=0x5654f40f07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:54:01.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.161+0000 7ffa1ffff700 1 --2- 172.21.15.52:0/2953224 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5654f40cbf80 0x5654f40f07e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:01.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.161+0000 7ffa1ffff700 1 -- 172.21.15.52:0/2953224 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654f40ec8e0 con 0x5654f40d0bb0 2024-06-10T02:54:01.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.161+0000 7ffa1ffff700 1 --2- 172.21.15.52:0/2953224 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5654f40d0bb0 0x5654f40ec0a0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7ffa1800b0a0 tx=0x7ffa18004c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afd0e352b6b7aee1 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:54:01.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.161+0000 7ff9feffd700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-10T02:54:01.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.161+0000 7ff9feffd700 1 -- 172.21.15.52:0/2953224 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa1802a060 con 0x5654f40d0bb0 2024-06-10T02:54:01.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.162+0000 7ff9feffd700 1 -- 172.21.15.52:0/2953224 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7ffa180076f0 con 0x5654f40d0bb0 2024-06-10T02:54:01.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.162+0000 7ffa3040fb80 1 -- 172.21.15.52:0/2953224 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa08099770 con 0x5654f40d0bb0 2024-06-10T02:54:01.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.162+0000 7ff9feffd700 1 -- 172.21.15.52:0/2953224 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ffa18031050 con 0x5654f40d0bb0 2024-06-10T02:54:01.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.162+0000 7ffa3040fb80 10 client.15267 fetch_fsmap learned FSMap version 29 2024-06-10T02:54:01.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.162+0000 7ffa3040fb80 1 -- 172.21.15.52:0/2953224 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5654f4151010 con 0x5654f40d0bb0 2024-06-10T02:54:01.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.162+0000 7ff9feffd700 1 -- 172.21.15.52:0/2953224 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa08099770 con 0x5654f40d0bb0 2024-06-10T02:54:01.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.162+0000 7ff9feffd700 1 -- 172.21.15.52:0/2953224 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7ffa18011070 con 0x5654f40d0bb0 2024-06-10T02:54:01.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.162+0000 7ffa3040fb80 10 client.15267 fetch_fsmap finished waiting for FSMap version 29 2024-06-10T02:54:01.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.162+0000 7ffa3040fb80 10 client.15267 resolve_mds: resolved name 'i' to daemon mds.i 2024-06-10T02:54:01.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.162+0000 7ffa3040fb80 20 client.15267 populate_metadata read hostname 'smithi052' 2024-06-10T02:54:01.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.162+0000 7ffa3040fb80 1 --2- 172.21.15.52:0/2953224 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x5654f4155c30 0x5654f4154940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:01.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.162+0000 7ffa3040fb80 4 client.15267 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "status"}] 2024-06-10T02:54:01.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.162+0000 7ffa3040fb80 1 -- 172.21.15.52:0/2953224 --> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] -- command(tid 0: {"prefix": "status"}) -- 0x5654f4021240 con 0x5654f4155c30 2024-06-10T02:54:01.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.162+0000 7ffa1f7fe700 1 --2- 172.21.15.52:0/2953224 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x5654f4155c30 0x5654f4154940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:01.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.163+0000 7ffa1f7fe700 1 --2- 172.21.15.52:0/2953224 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x5654f4155c30 0x5654f4154940 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f545d26320db4757 server_cookie=ff8225adf4eac685 in_seq=0 out_seq=0 2024-06-10T02:54:01.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.163+0000 7ff9feffd700 10 client.15267 ms_handle_connect on v2:172.21.15.188:6838/310444836 2024-06-10T02:54:01.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.163+0000 7ff9feffd700 1 -- 172.21.15.52:0/2953224 <== mds.0 v2:172.21.15.188:6838/310444836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+333 (crc 0 0 0) 0x5654f4021240 con 0x5654f4155c30 2024-06-10T02:54:01.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.163+0000 7ff9feffd700 10 client.15267 handle_command_reply: tid=0 2024-06-10T02:54:01.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.163+0000 7ffa3040fb80 1 client.15267 shutdown 2024-06-10T02:54:01.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.163+0000 7ffa3040fb80 1 -- 172.21.15.52:0/2953224 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5654f40d0bb0 msgr2=0x5654f40ec0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:54:01.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.163+0000 7ffa3040fb80 1 --2- 172.21.15.52:0/2953224 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5654f40d0bb0 0x5654f40ec0a0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7ffa1800b0a0 tx=0x7ffa18004c80 comp rx=0 tx=0).stop 2024-06-10T02:54:01.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.163+0000 7ffa3040fb80 1 -- 172.21.15.52:0/2953224 shutdown_connections 2024-06-10T02:54:01.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.163+0000 7ffa3040fb80 1 --2- 172.21.15.52:0/2953224 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x5654f4155c30 0x5654f4154940 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:01.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.163+0000 7ffa3040fb80 1 --2- 172.21.15.52:0/2953224 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5654f40d0bb0 0x5654f40ec0a0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:01.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.163+0000 7ffa3040fb80 1 --2- 172.21.15.52:0/2953224 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5654f40cbf80 0x5654f40f07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:01.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.163+0000 7ff9feffd700 0 client.15267 ms_handle_reset on v2:172.21.15.188:6838/310444836 2024-06-10T02:54:01.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.163+0000 7ffa3040fb80 1 --2- 172.21.15.52:0/2953224 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5654f40cc8a0 0x5654f40ebb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:01.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.163+0000 7ffa3040fb80 1 -- 172.21.15.52:0/2953224 >> 172.21.15.52:0/2953224 conn(0x5654f3ff2c70 msgr2=0x5654f40bc4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:54:01.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.163+0000 7ffa3040fb80 1 -- 172.21.15.52:0/2953224 shutdown_connections 2024-06-10T02:54:01.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.163+0000 7ffa3040fb80 1 -- 172.21.15.52:0/2953224 wait complete. 2024-06-10T02:54:01.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.163+0000 7ffa3040fb80 20 client.15267 trim_cache size 0 max 16384 2024-06-10T02:54:01.168 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-10T02:54:01.168 INFO:teuthology.orchestra.run.smithi052.stdout: "cluster_fsid": "d8676e9e-26d2-11ef-bc9b-c7b262605968", 2024-06-10T02:54:01.168 INFO:teuthology.orchestra.run.smithi052.stdout: "whoami": 0, 2024-06-10T02:54:01.168 INFO:teuthology.orchestra.run.smithi052.stdout: "id": 24484, 2024-06-10T02:54:01.168 INFO:teuthology.orchestra.run.smithi052.stdout: "want_state": "up:active", 2024-06-10T02:54:01.168 INFO:teuthology.orchestra.run.smithi052.stdout: "state": "up:active", 2024-06-10T02:54:01.169 INFO:teuthology.orchestra.run.smithi052.stdout: "fs_name": "cephfs", 2024-06-10T02:54:01.169 INFO:teuthology.orchestra.run.smithi052.stdout: "rank_uptime": 236.96921910399999, 2024-06-10T02:54:01.169 INFO:teuthology.orchestra.run.smithi052.stdout: "mdsmap_epoch": 29, 2024-06-10T02:54:01.169 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_epoch": 100, 2024-06-10T02:54:01.169 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_epoch_barrier": 98, 2024-06-10T02:54:01.169 INFO:teuthology.orchestra.run.smithi052.stdout: "uptime": 264.98263637100001 2024-06-10T02:54:01.169 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-10T02:54:01.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.164+0000 7ffa25b8c700 1 -- 172.21.15.52:0/2335875971 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7ffa0c0a2c20 msgr2=0x7ffa0c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:54:01.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.164+0000 7ffa25b8c700 1 --2- 172.21.15.52:0/2335875971 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7ffa0c0a2c20 0x7ffa0c0a50e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7ffa20199ba0 tx=0x7ffa1000b410 comp rx=0 tx=0).stop 2024-06-10T02:54:01.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.164+0000 7ffa25b8c700 1 -- 172.21.15.52:0/2335875971 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffa2006b580 msgr2=0x7ffa2019ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:54:01.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.164+0000 7ffa25b8c700 1 --2- 172.21.15.52:0/2335875971 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffa2006b580 0x7ffa2019ede0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7ffa0800d950 tx=0x7ffa0800dc60 comp rx=0 tx=0).stop 2024-06-10T02:54:01.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.164+0000 7ffa25b8c700 1 -- 172.21.15.52:0/2335875971 shutdown_connections 2024-06-10T02:54:01.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.164+0000 7ffa25b8c700 1 --2- 172.21.15.52:0/2335875971 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffa20108500 0x7ffa20198a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:01.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.164+0000 7ffa25b8c700 1 --2- 172.21.15.52:0/2335875971 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffa2006abc0 0x7ffa2019e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:01.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.164+0000 7ffa25b8c700 1 --2- 172.21.15.52:0/2335875971 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7ffa0c0a2c20 0x7ffa0c0a50e0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:01.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.164+0000 7ffa25b8c700 1 --2- 172.21.15.52:0/2335875971 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffa2006b580 0x7ffa2019ede0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:01.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.164+0000 7ffa25b8c700 1 -- 172.21.15.52:0/2335875971 >> 172.21.15.52:0/2335875971 conn(0x7ffa20078340 msgr2=0x7ffa20100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:54:01.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.164+0000 7ffa25b8c700 1 -- 172.21.15.52:0/2335875971 shutdown_connections 2024-06-10T02:54:01.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:01.164+0000 7ffa25b8c700 1 -- 172.21.15.52:0/2335875971 wait complete. 2024-06-10T02:54:01.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:54:00 smithi188 ceph-mon[36599]: pgmap v428: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-10T02:54:01.880 INFO:teuthology.run_tasks:Running task sequential... 2024-06-10T02:54:01.889 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-06-10T02:54:01.889 INFO:teuthology.task.sequential:In sequential, running task print... 2024-06-10T02:54:01.889 INFO:teuthology.task.print:Enabling mgr modules 2024-06-10T02:54:01.889 INFO:teuthology.run_tasks:Running task kclient... 2024-06-10T02:54:01.909 INFO:tasks.kclient:Mounting kernel clients... 2024-06-10T02:54:01.910 INFO:tasks.kclient:config is {'client.0': {}} 2024-06-10T02:54:01.910 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-10T02:54:01.910 DEBUG:teuthology.orchestra.run.smithi052:> ip netns list 2024-06-10T02:54:01.929 INFO:teuthology.orchestra.run.smithi052.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-06-10T02:54:01.929 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-10T02:54:01.929 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-06-10T02:54:01.999 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-10T02:54:02.000 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip link delete ceph-brx 2024-06-10T02:54:02.091 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=secure', 'nowsync'], 'syntax': 'v2', 'mount_subvol_num': 1} 2024-06-10T02:54:02.091 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-06-10T02:54:02.091 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-06-10T02:54:02.092 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph fs ls 2024-06-10T02:54:02.346 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:54:03.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:54:02 smithi120 ceph-mon[36634]: pgmap v429: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-10T02:54:03.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:54:02 smithi188 ceph-mon[36599]: pgmap v429: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-10T02:54:03.226 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:54:02 smithi052 ceph-mon[32362]: pgmap v429: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-10T02:54:04.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:04.237+0000 7fcaf34b9700 1 -- 172.21.15.52:0/1357456163 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcaec101390 msgr2=0x7fcaec101810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:54:04.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:04.237+0000 7fcaf34b9700 1 --2- 172.21.15.52:0/1357456163 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcaec101390 0x7fcaec101810 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fcadc009990 tx=0x7fcadc009ca0 comp rx=0 tx=0).stop 2024-06-10T02:54:04.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:04.237+0000 7fcaf34b9700 1 -- 172.21.15.52:0/1357456163 shutdown_connections 2024-06-10T02:54:04.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:04.237+0000 7fcaf34b9700 1 --2- 172.21.15.52:0/1357456163 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcaec101d50 0x7fcaec10e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:04.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:04.237+0000 7fcaf34b9700 1 --2- 172.21.15.52:0/1357456163 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcaec107400 0x7fcaec1077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:04.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:04.237+0000 7fcaf34b9700 1 --2- 172.21.15.52:0/1357456163 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcaec101390 0x7fcaec101810 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:04.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:04.237+0000 7fcaf34b9700 1 -- 172.21.15.52:0/1357456163 >> 172.21.15.52:0/1357456163 conn(0x7fcaec078340 msgr2=0x7fcaec078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:54:04.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:04.237+0000 7fcaf34b9700 1 -- 172.21.15.52:0/1357456163 shutdown_connections 2024-06-10T02:54:04.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:04.237+0000 7fcaf34b9700 1 -- 172.21.15.52:0/1357456163 wait complete. 2024-06-10T02:54:04.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:04.238+0000 7fcaf34b9700 1 Processor -- start 2024-06-10T02:54:04.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:04.238+0000 7fcaf34b9700 1 -- start start 2024-06-10T02:54:04.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:04.238+0000 7fcaf34b9700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcaec101390 0x7fcaec19d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:04.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:04.238+0000 7fcaf34b9700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcaec101d50 0x7fcaec19dc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:04.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:04.238+0000 7fcaf34b9700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcaec107400 0x7fcaec1978f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:04.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:04.238+0000 7fcaf34b9700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fcaec111670 con 0x7fcaec101390 2024-06-10T02:54:04.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:04.238+0000 7fcaf34b9700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fcaec1114f0 con 0x7fcaec101d50 2024-06-10T02:54:04.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:04.238+0000 7fcaf34b9700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fcaec1117f0 con 0x7fcaec107400 2024-06-10T02:54:04.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:04.238+0000 7fcaf1255700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcaec101390 0x7fcaec19d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:04.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:04.239+0000 7fcaf1a56700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcaec107400 0x7fcaec1978f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:04.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:04.239+0000 7fcaf1255700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcaec101390 0x7fcaec19d730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:38910/0 (socket says 172.21.15.52:38910) 2024-06-10T02:54:04.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:04.239+0000 7fcaf1255700 1 -- 172.21.15.52:0/305871628 learned_addr learned my addr 172.21.15.52:0/305871628 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:54:04.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:04.239+0000 7fcaf0a54700 1 --2- 172.21.15.52:0/305871628 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcaec101d50 0x7fcaec19dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:04.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:04.239+0000 7fcaf1255700 1 -- 172.21.15.52:0/305871628 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcaec107400 msgr2=0x7fcaec1978f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:54:04.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:04.239+0000 7fcaf1255700 1 --2- 172.21.15.52:0/305871628 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcaec107400 0x7fcaec1978f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:04.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:04.239+0000 7fcaf1255700 1 -- 172.21.15.52:0/305871628 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcaec101d50 msgr2=0x7fcaec19dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:54:04.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:04.239+0000 7fcaf1255700 1 --2- 172.21.15.52:0/305871628 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcaec101d50 0x7fcaec19dc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:04.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:04.239+0000 7fcaf1255700 1 -- 172.21.15.52:0/305871628 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcaec198050 con 0x7fcaec101390 2024-06-10T02:54:04.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:04.239+0000 7fcaf1255700 1 --2- 172.21.15.52:0/305871628 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcaec101390 0x7fcaec19d730 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fcae800c920 tx=0x7fcae800cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:54:04.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:04.239+0000 7fcae27fc700 1 -- 172.21.15.52:0/305871628 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcae8004a30 con 0x7fcaec101390 2024-06-10T02:54:04.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:04.239+0000 7fcaf34b9700 1 -- 172.21.15.52:0/305871628 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcaec198340 con 0x7fcaec101390 2024-06-10T02:54:04.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:04.239+0000 7fcae27fc700 1 -- 172.21.15.52:0/305871628 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcae8005500 con 0x7fcaec101390 2024-06-10T02:54:04.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:04.239+0000 7fcae27fc700 1 -- 172.21.15.52:0/305871628 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcae8007d00 con 0x7fcaec101390 2024-06-10T02:54:04.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:04.240+0000 7fcaf34b9700 1 -- 172.21.15.52:0/305871628 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcaec06a170 con 0x7fcaec101390 2024-06-10T02:54:04.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:04.241+0000 7fcaf34b9700 1 -- 172.21.15.52:0/305871628 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcaec069180 con 0x7fcaec101390 2024-06-10T02:54:04.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:04.242+0000 7fcae27fc700 1 -- 172.21.15.52:0/305871628 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fcae8005000 con 0x7fcaec101390 2024-06-10T02:54:04.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:04.243+0000 7fcae27fc700 1 --2- 172.21.15.52:0/305871628 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fcad80a2c20 0x7fcad80a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:04.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:04.244+0000 7fcaf0a54700 1 --2- 172.21.15.52:0/305871628 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fcad80a2c20 0x7fcad80a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:04.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:04.244+0000 7fcae27fc700 1 -- 172.21.15.52:0/305871628 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7fcae80cee70 con 0x7fcaec101390 2024-06-10T02:54:04.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:04.244+0000 7fcaf0a54700 1 --2- 172.21.15.52:0/305871628 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fcad80a2c20 0x7fcad80a50e0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fcadc00b5c0 tx=0x7fcadc020040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:54:04.248 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:04.249+0000 7fcae27fc700 1 -- 172.21.15.52:0/305871628 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fcae8014030 con 0x7fcaec101390 2024-06-10T02:54:04.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:04.503+0000 7fcaf34b9700 1 -- 172.21.15.52:0/305871628 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fcaec06aba0 con 0x7fcaec101390 2024-06-10T02:54:04.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:04.504+0000 7fcae27fc700 1 -- 172.21.15.52:0/305871628 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v29) ==== 53+0+88 (secure 0 0 0) 0x7fcae809ba20 con 0x7fcaec101390 2024-06-10T02:54:04.503 INFO:teuthology.orchestra.run.smithi052.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-06-10T02:54:04.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:04.506+0000 7fcaf34b9700 1 -- 172.21.15.52:0/305871628 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fcad80a2c20 msgr2=0x7fcad80a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:54:04.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:04.506+0000 7fcaf34b9700 1 --2- 172.21.15.52:0/305871628 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fcad80a2c20 0x7fcad80a50e0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fcadc00b5c0 tx=0x7fcadc020040 comp rx=0 tx=0).stop 2024-06-10T02:54:04.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:04.506+0000 7fcaf34b9700 1 -- 172.21.15.52:0/305871628 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcaec101390 msgr2=0x7fcaec19d730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:54:04.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:04.506+0000 7fcaf34b9700 1 --2- 172.21.15.52:0/305871628 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcaec101390 0x7fcaec19d730 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fcae800c920 tx=0x7fcae800cc30 comp rx=0 tx=0).stop 2024-06-10T02:54:04.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:04.506+0000 7fcaf34b9700 1 -- 172.21.15.52:0/305871628 shutdown_connections 2024-06-10T02:54:04.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:04.506+0000 7fcaf34b9700 1 --2- 172.21.15.52:0/305871628 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcaec107400 0x7fcaec1978f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:04.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:04.506+0000 7fcaf34b9700 1 --2- 172.21.15.52:0/305871628 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fcad80a2c20 0x7fcad80a50e0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:04.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:04.506+0000 7fcaf34b9700 1 --2- 172.21.15.52:0/305871628 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcaec101390 0x7fcaec19d730 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:04.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:04.506+0000 7fcaf34b9700 1 --2- 172.21.15.52:0/305871628 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcaec101d50 0x7fcaec19dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:04.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:04.506+0000 7fcaf34b9700 1 -- 172.21.15.52:0/305871628 >> 172.21.15.52:0/305871628 conn(0x7fcaec078340 msgr2=0x7fcaec0ffe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:54:04.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:04.506+0000 7fcaf34b9700 1 -- 172.21.15.52:0/305871628 shutdown_connections 2024-06-10T02:54:04.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:04.506+0000 7fcaf34b9700 1 -- 172.21.15.52:0/305871628 wait complete. 2024-06-10T02:54:04.865 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:54:04 smithi052 ceph-mon[32362]: pgmap v430: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-10T02:54:04.865 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:54:04 smithi052 ceph-mon[32362]: from='client.? 172.21.15.52:0/305871628' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-06-10T02:54:05.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:54:04 smithi120 ceph-mon[36634]: pgmap v430: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-10T02:54:05.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:54:04 smithi120 ceph-mon[36634]: from='client.? 172.21.15.52:0/305871628' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-06-10T02:54:05.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:54:04 smithi188 ceph-mon[36599]: pgmap v430: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-10T02:54:05.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:54:04 smithi188 ceph-mon[36599]: from='client.? 172.21.15.52:0/305871628' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-06-10T02:54:05.205 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-06-10T02:54:05.205 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-06-10T02:54:05.206 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi052.front.sepia.ceph.com 2024-06-10T02:54:05.206 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-06-10T02:54:05.206 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-06-10T02:54:05.206 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-06-10T02:54:05.206 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-06-10T02:54:05.206 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-06-10T02:54:05.206 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-06-10T02:54:05.206 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-10T02:54:05.206 DEBUG:teuthology.orchestra.run.smithi052:> ip addr 2024-06-10T02:54:05.224 INFO:teuthology.orchestra.run.smithi052.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-06-10T02:54:05.224 INFO:teuthology.orchestra.run.smithi052.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-06-10T02:54:05.225 INFO:teuthology.orchestra.run.smithi052.stdout: inet 127.0.0.1/8 scope host lo 2024-06-10T02:54:05.225 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-06-10T02:54:05.225 INFO:teuthology.orchestra.run.smithi052.stdout: inet6 ::1/128 scope host 2024-06-10T02:54:05.225 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-06-10T02:54:05.225 INFO:teuthology.orchestra.run.smithi052.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-06-10T02:54:05.225 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 0c:c4:7a:6b:fd:d2 brd ff:ff:ff:ff:ff:ff 2024-06-10T02:54:05.225 INFO:teuthology.orchestra.run.smithi052.stdout: altname enp7s0f0 2024-06-10T02:54:05.225 INFO:teuthology.orchestra.run.smithi052.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-06-10T02:54:05.225 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 0c:c4:7a:6b:fd:d3 brd ff:ff:ff:ff:ff:ff 2024-06-10T02:54:05.226 INFO:teuthology.orchestra.run.smithi052.stdout: altname enp7s0f1 2024-06-10T02:54:05.226 INFO:teuthology.orchestra.run.smithi052.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-06-10T02:54:05.226 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 0c:c4:7a:bd:15:de brd ff:ff:ff:ff:ff:ff 2024-06-10T02:54:05.226 INFO:teuthology.orchestra.run.smithi052.stdout: altname enp1s0f0 2024-06-10T02:54:05.226 INFO:teuthology.orchestra.run.smithi052.stdout: inet 172.21.15.52/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-06-10T02:54:05.226 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft 41698sec preferred_lft 41698sec 2024-06-10T02:54:05.226 INFO:teuthology.orchestra.run.smithi052.stdout: inet6 fe80::ec4:7aff:febd:15de/64 scope link noprefixroute 2024-06-10T02:54:05.226 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-06-10T02:54:05.226 INFO:teuthology.orchestra.run.smithi052.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-06-10T02:54:05.226 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 0c:c4:7a:bd:15:df brd ff:ff:ff:ff:ff:ff 2024-06-10T02:54:05.226 INFO:teuthology.orchestra.run.smithi052.stdout: altname enp1s0f1 2024-06-10T02:54:05.227 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-06-10T02:54:05.227 DEBUG:teuthology.orchestra.run.smithi052:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-06-10T02:54:05.227 DEBUG:teuthology.orchestra.run.smithi052:> set -e 2024-06-10T02:54:05.227 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip link add name ceph-brx type bridge 2024-06-10T02:54:05.227 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip addr flush dev ceph-brx 2024-06-10T02:54:05.227 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip link set ceph-brx up 2024-06-10T02:54:05.227 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-06-10T02:54:05.227 DEBUG:teuthology.orchestra.run.smithi052:> ') 2024-06-10T02:54:05.322 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:05 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-06-10T02:54:05.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-10T02:54:05.375 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-10T02:54:05.375 DEBUG:teuthology.orchestra.run.smithi052:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-06-10T02:54:05.400 INFO:teuthology.orchestra.run.smithi052.stdout:1 2024-06-10T02:54:05.402 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-10T02:54:05.402 DEBUG:teuthology.orchestra.run.smithi052:> route 2024-06-10T02:54:05.459 INFO:teuthology.orchestra.run.smithi052.stdout:Kernel IP routing table 2024-06-10T02:54:05.459 INFO:teuthology.orchestra.run.smithi052.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-06-10T02:54:05.459 INFO:teuthology.orchestra.run.smithi052.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-06-10T02:54:05.459 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-06-10T02:54:05.459 INFO:teuthology.orchestra.run.smithi052.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-06-10T02:54:05.460 DEBUG:teuthology.orchestra.run.smithi052:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-06-10T02:54:05.460 DEBUG:teuthology.orchestra.run.smithi052:> set -e 2024-06-10T02:54:05.460 DEBUG:teuthology.orchestra.run.smithi052:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-06-10T02:54:05.460 DEBUG:teuthology.orchestra.run.smithi052:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-06-10T02:54:05.460 DEBUG:teuthology.orchestra.run.smithi052:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-06-10T02:54:05.460 DEBUG:teuthology.orchestra.run.smithi052:> ') 2024-06-10T02:54:05.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:05 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-06-10T02:54:05.583 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-10T02:54:05.589 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-10T02:54:05.590 DEBUG:teuthology.orchestra.run.smithi052:> ip netns list 2024-06-10T02:54:05.606 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-10T02:54:05.606 DEBUG:teuthology.orchestra.run.smithi052:> ip netns list-id 2024-06-10T02:54:05.667 DEBUG:teuthology.orchestra.run.smithi052:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-06-10T02:54:05.667 DEBUG:teuthology.orchestra.run.smithi052:> set -e 2024-06-10T02:54:05.667 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-06-10T02:54:05.667 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-06-10T02:54:05.667 DEBUG:teuthology.orchestra.run.smithi052:> ') 2024-06-10T02:54:05.765 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:05 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-06-10T02:54:05.789 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-10T02:54:05.795 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-06-10T02:54:05.796 DEBUG:teuthology.orchestra.run.smithi052:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-06-10T02:54:05.796 DEBUG:teuthology.orchestra.run.smithi052:> set -e 2024-06-10T02:54:05.796 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-06-10T02:54:05.796 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-06-10T02:54:05.797 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-06-10T02:54:05.797 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-06-10T02:54:05.797 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-06-10T02:54:05.797 DEBUG:teuthology.orchestra.run.smithi052:> ') 2024-06-10T02:54:05.848 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:05 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-06-10T02:54:05.911 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-10T02:54:05.920 DEBUG:teuthology.orchestra.run.smithi052:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-06-10T02:54:05.920 DEBUG:teuthology.orchestra.run.smithi052:> set -e 2024-06-10T02:54:05.920 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip link set brx.0 up 2024-06-10T02:54:05.920 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip link set dev brx.0 master ceph-brx 2024-06-10T02:54:05.920 DEBUG:teuthology.orchestra.run.smithi052:> ') 2024-06-10T02:54:05.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:05 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-06-10T02:54:05.995 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-10T02:54:06.001 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-06-10T02:54:06.001 DEBUG:teuthology.orchestra.run.smithi052:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-06-10T02:54:06.019 INFO:teuthology.orchestra.run.smithi052.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-06-10T02:54:06.019 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-06-10T02:54:06.019 DEBUG:teuthology.orchestra.run.smithi052:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-06-10T02:54:06.081 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-06-10T02:54:06.081 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-06-10T02:54:06.081 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/volumes/qa/sv_1/80a54c16-16b3-4ac2-8c1a-868b99f8d35c 2024-06-10T02:54:06.081 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-10T02:54:06.082 DEBUG:teuthology.orchestra.run.smithi052:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/volumes/qa/sv_1/80a54c16-16b3-4ac2-8c1a-868b99f8d35c /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=secure,nowsync,nofallback 2024-06-10T02:54:06.170 INFO:teuthology.orchestra.run.smithi052.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=secure,nowsync,nofallback 2024-06-10T02:54:06.170 INFO:teuthology.orchestra.run.smithi052.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=secure,nowsync". 2024-06-10T02:54:06.325 INFO:teuthology.orchestra.run.smithi052.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=secure,nowsync,nofallback 2024-06-10T02:54:06.326 INFO:teuthology.orchestra.run.smithi052.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=secure,nowsync". 2024-06-10T02:54:06.326 INFO:teuthology.orchestra.run.smithi052.stdout:mount.ceph: resolved to: "172.21.15.52:3300,172.21.15.120:3300,172.21.15.188:3300" 2024-06-10T02:54:06.326 INFO:teuthology.orchestra.run.smithi052.stdout:mount.ceph: trying mount with new device syntax: 0@d8676e9e-26d2-11ef-bc9b-c7b262605968.cephfs=/volumes/qa/sv_1/80a54c16-16b3-4ac2-8c1a-868b99f8d35c 2024-06-10T02:54:06.326 INFO:teuthology.orchestra.run.smithi052.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=secure,nowsync,name=0,key=0,mon_addr=172.21.15.52:3300/172.21.15.120:3300/172.21.15.188:3300" will pass to kernel 2024-06-10T02:54:06.327 INFO:teuthology.orchestra.run.smithi052.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-06-10T02:54:06.327 INFO:teuthology.orchestra.run.smithi052.stdout: You just mounted a file system that supports labels which does not 2024-06-10T02:54:06.327 INFO:teuthology.orchestra.run.smithi052.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-06-10T02:54:06.327 INFO:teuthology.orchestra.run.smithi052.stdout: applications will generate AVC messages and not be allowed access to 2024-06-10T02:54:06.328 INFO:teuthology.orchestra.run.smithi052.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-06-10T02:54:06.329 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-06-10T02:54:06.329 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-10T02:54:06.329 DEBUG:teuthology.orchestra.run.smithi052:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-06-10T02:54:06.358 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-06-10T02:54:06.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:06 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-06-10T02:54:06.454 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-10T02:54:06.460 INFO:teuthology.orchestra.run.smithi052.stdout:client24967 2024-06-10T02:54:06.460 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-06-10T02:54:06.461 DEBUG:teuthology.orchestra.run.smithi052:> dd if=/proc/self/mounts of=/dev/stdout 2024-06-10T02:54:06.481 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-06-10T02:54:06.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:06 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-06-10T02:54:06.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-10T02:54:06.593 INFO:teuthology.orchestra.run.smithi052.stdout:client24967 2024-06-10T02:54:06.594 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-06-10T02:54:06.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:06 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-06-10T02:54:06.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-10T02:54:06.654 INFO:teuthology.orchestra.run.smithi052.stdout:client24967 2024-06-10T02:54:06.654 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968.client24967/status') 2024-06-10T02:54:06.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:06 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-06-10T02:54:06.716 INFO:teuthology.orchestra.run.smithi052.stdout:instance: client.24967 (3)172.21.15.52:0/3218352606 2024-06-10T02:54:06.716 INFO:teuthology.orchestra.run.smithi052.stdout:blocklisted: false 2024-06-10T02:54:06.717 INFO:teuthology.orchestra.run.smithi052.stderr:0+1 records in 2024-06-10T02:54:06.717 INFO:teuthology.orchestra.run.smithi052.stderr:0+1 records out 2024-06-10T02:54:06.717 INFO:teuthology.orchestra.run.smithi052.stderr:71 bytes copied, 2.7006e-05 s, 2.6 MB/s 2024-06-10T02:54:06.718 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-10T02:54:06.724 INFO:teuthology.run_tasks:Running task exec... 2024-06-10T02:54:06.739 INFO:teuthology.task.exec:Executing custom commands... 2024-06-10T02:54:06.739 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi052.front.sepia.ceph.com 2024-06-10T02:54:06.739 DEBUG:teuthology.orchestra.run.smithi052:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate false' 2024-06-10T02:54:06.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:06.848+0000 7f2ad035c640 1 -- 172.21.15.52:0/3166742842 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2ac8107700 msgr2=0x7f2ac8109b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:54:06.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:06.848+0000 7f2ad035c640 1 --2- 172.21.15.52:0/3166742842 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2ac8107700 0x7f2ac8109b50 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f2ac0009ca0 tx=0x7f2ac002a8c0 comp rx=0 tx=0).stop 2024-06-10T02:54:06.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:06.848+0000 7f2ad035c640 1 -- 172.21.15.52:0/3166742842 shutdown_connections 2024-06-10T02:54:06.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:06.848+0000 7f2ad035c640 1 --2- 172.21.15.52:0/3166742842 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2ac8107700 0x7f2ac8109b50 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:06.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:06.848+0000 7f2ad035c640 1 --2- 172.21.15.52:0/3166742842 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2ac8104dd0 0x7f2ac81071c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:06.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:06.848+0000 7f2ad035c640 1 --2- 172.21.15.52:0/3166742842 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2ac81024a0 0x7f2ac8104890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:06.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:06.848+0000 7f2ad035c640 1 -- 172.21.15.52:0/3166742842 >> 172.21.15.52:0/3166742842 conn(0x7f2ac80fbeb0 msgr2=0x7f2ac80fe2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:54:06.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:06.848+0000 7f2ad035c640 1 -- 172.21.15.52:0/3166742842 shutdown_connections 2024-06-10T02:54:06.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:06.848+0000 7f2ad035c640 1 -- 172.21.15.52:0/3166742842 wait complete. 2024-06-10T02:54:06.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:06.849+0000 7f2ad035c640 1 Processor -- start 2024-06-10T02:54:06.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:06.849+0000 7f2ad035c640 1 -- start start 2024-06-10T02:54:06.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:06.849+0000 7f2ad035c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2ac81024a0 0x7f2ac8197ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:06.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:06.849+0000 7f2ad035c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2ac8104dd0 0x7f2ac81981e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:06.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:06.849+0000 7f2ad035c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2ac8107700 0x7f2ac819f240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:06.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:06.849+0000 7f2ad035c640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f2ac810d710 con 0x7f2ac8104dd0 2024-06-10T02:54:06.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:06.849+0000 7f2ad035c640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f2ac810d590 con 0x7f2ac81024a0 2024-06-10T02:54:06.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:06.849+0000 7f2ad035c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f2ac810d890 con 0x7f2ac8107700 2024-06-10T02:54:06.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:06.849+0000 7f2acd8f7640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2ac8104dd0 0x7f2ac81981e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:06.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:06.849+0000 7f2acd8f7640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2ac8104dd0 0x7f2ac81981e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:38930/0 (socket says 172.21.15.52:38930) 2024-06-10T02:54:06.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:06.849+0000 7f2acd8f7640 1 -- 172.21.15.52:0/2371134135 learned_addr learned my addr 172.21.15.52:0/2371134135 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:54:06.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:06.850+0000 7f2ace8f9640 1 --2- 172.21.15.52:0/2371134135 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2ac8107700 0x7f2ac819f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:06.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:06.850+0000 7f2ace0f8640 1 --2- 172.21.15.52:0/2371134135 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2ac81024a0 0x7f2ac8197ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:06.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:06.850+0000 7f2acd8f7640 1 -- 172.21.15.52:0/2371134135 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2ac8107700 msgr2=0x7f2ac819f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:54:06.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:06.850+0000 7f2acd8f7640 1 --2- 172.21.15.52:0/2371134135 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2ac8107700 0x7f2ac819f240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:06.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:06.850+0000 7f2acd8f7640 1 -- 172.21.15.52:0/2371134135 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2ac81024a0 msgr2=0x7f2ac8197ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:54:06.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:06.850+0000 7f2acd8f7640 1 --2- 172.21.15.52:0/2371134135 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2ac81024a0 0x7f2ac8197ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:06.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:06.850+0000 7f2acd8f7640 1 -- 172.21.15.52:0/2371134135 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ac819f8e0 con 0x7f2ac8104dd0 2024-06-10T02:54:06.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:06.850+0000 7f2ace0f8640 1 --2- 172.21.15.52:0/2371134135 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2ac81024a0 0x7f2ac8197ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T02:54:06.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:06.850+0000 7f2acd8f7640 1 --2- 172.21.15.52:0/2371134135 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2ac8104dd0 0x7f2ac81981e0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f2ab800c970 tx=0x7f2ab800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:54:06.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:06.850+0000 7f2abf7fe640 1 -- 172.21.15.52:0/2371134135 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ab8007bb0 con 0x7f2ac8104dd0 2024-06-10T02:54:06.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:06.850+0000 7f2ad035c640 1 -- 172.21.15.52:0/2371134135 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ac819fbd0 con 0x7f2ac8104dd0 2024-06-10T02:54:06.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:06.850+0000 7f2abf7fe640 1 -- 172.21.15.52:0/2371134135 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ab8007d50 con 0x7f2ac8104dd0 2024-06-10T02:54:06.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:06.850+0000 7f2abf7fe640 1 -- 172.21.15.52:0/2371134135 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ab800f450 con 0x7f2ac8104dd0 2024-06-10T02:54:06.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:06.850+0000 7f2ad035c640 1 -- 172.21.15.52:0/2371134135 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ac81a0170 con 0x7f2ac8104dd0 2024-06-10T02:54:06.854 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:06.851+0000 7f2ad035c640 1 -- 172.21.15.52:0/2371134135 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ac80fd9a0 con 0x7f2ac8104dd0 2024-06-10T02:54:06.854 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:06.852+0000 7f2abf7fe640 1 -- 172.21.15.52:0/2371134135 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f2ab8004f90 con 0x7f2ac8104dd0 2024-06-10T02:54:06.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:06.853+0000 7f2abf7fe640 1 --2- 172.21.15.52:0/2371134135 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f2ab40a2c30 0x7f2ab40a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:06.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:06.853+0000 7f2abf7fe640 1 -- 172.21.15.52:0/2371134135 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f2ab80cf210 con 0x7f2ac8104dd0 2024-06-10T02:54:06.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:06.853+0000 7f2ace0f8640 1 --2- 172.21.15.52:0/2371134135 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f2ab40a2c30 0x7f2ab40a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:06.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:06.854+0000 7f2ace0f8640 1 --2- 172.21.15.52:0/2371134135 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f2ab40a2c30 0x7f2ab40a50f0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f2ac4006fd0 tx=0x7f2ac4008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:54:06.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:06.858+0000 7f2abf7fe640 1 -- 172.21.15.52:0/2371134135 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f2ab8014030 con 0x7f2ac8104dd0 2024-06-10T02:54:06.976 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:54:06 smithi052 ceph-mon[32362]: pgmap v431: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-10T02:54:07.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:07.085+0000 7f2ad035c640 1 -- 172.21.15.52:0/2371134135 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} v 0) -- 0x7f2ac80625a0 con 0x7f2ac8104dd0 2024-06-10T02:54:07.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:54:06 smithi120 ceph-mon[36634]: pgmap v431: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-10T02:54:07.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:54:06 smithi188 ceph-mon[36599]: pgmap v431: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-10T02:54:07.757 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:07.757+0000 7f2abf7fe640 1 -- 172.21.15.52:0/2371134135 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]=0 v30) ==== 118+0+0 (secure 0 0 0) 0x7f2ab809bdc0 con 0x7f2ac8104dd0 2024-06-10T02:54:07.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:07.761+0000 7f2ad035c640 1 -- 172.21.15.52:0/2371134135 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f2ab40a2c30 msgr2=0x7f2ab40a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:54:07.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:07.761+0000 7f2ad035c640 1 --2- 172.21.15.52:0/2371134135 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f2ab40a2c30 0x7f2ab40a50f0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f2ac4006fd0 tx=0x7f2ac4008040 comp rx=0 tx=0).stop 2024-06-10T02:54:07.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:07.761+0000 7f2ad035c640 1 -- 172.21.15.52:0/2371134135 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2ac8104dd0 msgr2=0x7f2ac81981e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:54:07.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:07.761+0000 7f2ad035c640 1 --2- 172.21.15.52:0/2371134135 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2ac8104dd0 0x7f2ac81981e0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f2ab800c970 tx=0x7f2ab800ce30 comp rx=0 tx=0).stop 2024-06-10T02:54:07.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:07.761+0000 7f2ad035c640 1 -- 172.21.15.52:0/2371134135 shutdown_connections 2024-06-10T02:54:07.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:07.761+0000 7f2ad035c640 1 --2- 172.21.15.52:0/2371134135 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f2ab40a2c30 0x7f2ab40a50f0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:07.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:07.761+0000 7f2ad035c640 1 --2- 172.21.15.52:0/2371134135 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2ac8107700 0x7f2ac819f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:07.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:07.761+0000 7f2ad035c640 1 --2- 172.21.15.52:0/2371134135 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2ac8104dd0 0x7f2ac81981e0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:07.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:07.761+0000 7f2ad035c640 1 --2- 172.21.15.52:0/2371134135 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2ac81024a0 0x7f2ac8197ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:07.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:07.761+0000 7f2ad035c640 1 -- 172.21.15.52:0/2371134135 >> 172.21.15.52:0/2371134135 conn(0x7f2ac80fbeb0 msgr2=0x7f2ac8102d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:54:07.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:07.761+0000 7f2ad035c640 1 -- 172.21.15.52:0/2371134135 shutdown_connections 2024-06-10T02:54:07.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:07.761+0000 7f2ad035c640 1 -- 172.21.15.52:0/2371134135 wait complete. 2024-06-10T02:54:07.775 DEBUG:teuthology.orchestra.run.smithi052:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs subvolumegroup pin cephfs qa random 0.10' 2024-06-10T02:54:07.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:07.887+0000 7f5b4e876640 1 -- 172.21.15.52:0/2725468435 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5b481025d0 msgr2=0x7f5b481049c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:54:07.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:07.887+0000 7f5b4e876640 1 --2- 172.21.15.52:0/2725468435 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5b481025d0 0x7f5b481049c0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f5b38009900 tx=0x7f5b3802a410 comp rx=0 tx=0).stop 2024-06-10T02:54:07.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:07.887+0000 7f5b4e876640 1 -- 172.21.15.52:0/2725468435 shutdown_connections 2024-06-10T02:54:07.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:07.887+0000 7f5b4e876640 1 --2- 172.21.15.52:0/2725468435 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5b48107830 0x7f5b48109c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:07.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:07.887+0000 7f5b4e876640 1 --2- 172.21.15.52:0/2725468435 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5b48104f00 0x7f5b481072f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:07.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:07.887+0000 7f5b4e876640 1 --2- 172.21.15.52:0/2725468435 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5b481025d0 0x7f5b481049c0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:07.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:07.887+0000 7f5b4e876640 1 -- 172.21.15.52:0/2725468435 >> 172.21.15.52:0/2725468435 conn(0x7f5b480fbf20 msgr2=0x7f5b480fe360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:54:07.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:07.887+0000 7f5b4e876640 1 -- 172.21.15.52:0/2725468435 shutdown_connections 2024-06-10T02:54:07.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:07.887+0000 7f5b4e876640 1 -- 172.21.15.52:0/2725468435 wait complete. 2024-06-10T02:54:07.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:07.888+0000 7f5b4e876640 1 Processor -- start 2024-06-10T02:54:07.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:07.888+0000 7f5b4e876640 1 -- start start 2024-06-10T02:54:07.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:07.888+0000 7f5b4e876640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5b481025d0 0x7f5b48101f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:07.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:07.888+0000 7f5b4e876640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5b48104f00 0x7f5b48100580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:07.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:07.888+0000 7f5b4e876640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5b48107830 0x7f5b48100ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:07.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:07.888+0000 7f5b4e876640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f5b4810d730 con 0x7f5b48104f00 2024-06-10T02:54:07.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:07.888+0000 7f5b4e876640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f5b4810d5b0 con 0x7f5b481025d0 2024-06-10T02:54:07.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:07.888+0000 7f5b4e876640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f5b4810d8b0 con 0x7f5b48107830 2024-06-10T02:54:07.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:07.888+0000 7f5b477fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5b48104f00 0x7f5b48100580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:07.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:07.888+0000 7f5b477fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5b48104f00 0x7f5b48100580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:38948/0 (socket says 172.21.15.52:38948) 2024-06-10T02:54:07.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:07.888+0000 7f5b477fe640 1 -- 172.21.15.52:0/1345373042 learned_addr learned my addr 172.21.15.52:0/1345373042 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:54:07.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:07.888+0000 7f5b47fff640 1 --2- 172.21.15.52:0/1345373042 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5b481025d0 0x7f5b48101f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:07.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:07.888+0000 7f5b4ce13640 1 --2- 172.21.15.52:0/1345373042 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5b48107830 0x7f5b48100ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:07.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:07.888+0000 7f5b477fe640 1 -- 172.21.15.52:0/1345373042 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5b48107830 msgr2=0x7f5b48100ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:54:07.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:07.889+0000 7f5b477fe640 1 --2- 172.21.15.52:0/1345373042 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5b48107830 0x7f5b48100ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:07.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:07.889+0000 7f5b477fe640 1 -- 172.21.15.52:0/1345373042 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5b481025d0 msgr2=0x7f5b48101f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:54:07.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:07.889+0000 7f5b477fe640 1 --2- 172.21.15.52:0/1345373042 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5b481025d0 0x7f5b48101f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:07.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:07.889+0000 7f5b477fe640 1 -- 172.21.15.52:0/1345373042 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b48101350 con 0x7f5b48104f00 2024-06-10T02:54:07.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:07.889+0000 7f5b47fff640 1 --2- 172.21.15.52:0/1345373042 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5b481025d0 0x7f5b48101f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T02:54:07.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:07.889+0000 7f5b4ce13640 1 --2- 172.21.15.52:0/1345373042 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5b48107830 0x7f5b48100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T02:54:07.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:07.889+0000 7f5b477fe640 1 --2- 172.21.15.52:0/1345373042 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5b48104f00 0x7f5b48100580 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f5b3c00b450 tx=0x7f5b3c00b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:54:07.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:07.889+0000 7f5b457fa640 1 -- 172.21.15.52:0/1345373042 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b3c015020 con 0x7f5b48104f00 2024-06-10T02:54:07.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:07.889+0000 7f5b4e876640 1 -- 172.21.15.52:0/1345373042 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b481a1da0 con 0x7f5b48104f00 2024-06-10T02:54:07.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:07.889+0000 7f5b457fa640 1 -- 172.21.15.52:0/1345373042 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b3c011d70 con 0x7f5b48104f00 2024-06-10T02:54:07.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:07.889+0000 7f5b457fa640 1 -- 172.21.15.52:0/1345373042 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b3c00f340 con 0x7f5b48104f00 2024-06-10T02:54:07.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:07.889+0000 7f5b4e876640 1 -- 172.21.15.52:0/1345373042 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b481a22e0 con 0x7f5b48104f00 2024-06-10T02:54:07.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:07.890+0000 7f5b4e876640 1 -- 172.21.15.52:0/1345373042 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b480fda10 con 0x7f5b48104f00 2024-06-10T02:54:07.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:07.891+0000 7f5b457fa640 1 -- 172.21.15.52:0/1345373042 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f5b3c018020 con 0x7f5b48104f00 2024-06-10T02:54:07.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:07.892+0000 7f5b457fa640 1 --2- 172.21.15.52:0/1345373042 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f5b300a2cb0 0x7f5b300a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:07.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:07.892+0000 7f5b47fff640 1 --2- 172.21.15.52:0/1345373042 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f5b300a2cb0 0x7f5b300a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:07.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:07.893+0000 7f5b457fa640 1 -- 172.21.15.52:0/1345373042 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f5b3c0cef80 con 0x7f5b48104f00 2024-06-10T02:54:07.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:07.893+0000 7f5b47fff640 1 --2- 172.21.15.52:0/1345373042 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f5b300a2cb0 0x7f5b300a5170 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f5b38000c00 tx=0x7f5b380315d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:54:07.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:07.897+0000 7f5b457fa640 1 -- 172.21.15.52:0/1345373042 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f5b3c01f020 con 0x7f5b48104f00 2024-06-10T02:54:08.110 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:08.110+0000 7f5b4e876640 1 -- 172.21.15.52:0/1345373042 --> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) -- 0x7f5b48061e80 con 0x7f5b300a2cb0 2024-06-10T02:54:08.115 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:54:07 smithi052 ceph-mon[32362]: from='client.? 172.21.15.52:0/2371134135' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-06-10T02:54:08.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:08.128+0000 7f5b457fa640 1 -- 172.21.15.52:0/1345373042 <== mgr.14150 v2:172.21.15.52:6800/3214763969 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3 (secure 0 0 0) 0x7f5b48061e80 con 0x7f5b300a2cb0 2024-06-10T02:54:08.127 INFO:teuthology.orchestra.run.smithi052.stdout:{} 2024-06-10T02:54:08.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:08.130+0000 7f5b4e876640 1 -- 172.21.15.52:0/1345373042 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f5b300a2cb0 msgr2=0x7f5b300a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:54:08.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:08.130+0000 7f5b4e876640 1 --2- 172.21.15.52:0/1345373042 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f5b300a2cb0 0x7f5b300a5170 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f5b38000c00 tx=0x7f5b380315d0 comp rx=0 tx=0).stop 2024-06-10T02:54:08.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:08.130+0000 7f5b4e876640 1 -- 172.21.15.52:0/1345373042 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5b48104f00 msgr2=0x7f5b48100580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:54:08.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:08.130+0000 7f5b4e876640 1 --2- 172.21.15.52:0/1345373042 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5b48104f00 0x7f5b48100580 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f5b3c00b450 tx=0x7f5b3c00b910 comp rx=0 tx=0).stop 2024-06-10T02:54:08.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:08.130+0000 7f5b4e876640 1 -- 172.21.15.52:0/1345373042 shutdown_connections 2024-06-10T02:54:08.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:08.130+0000 7f5b4e876640 1 --2- 172.21.15.52:0/1345373042 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f5b300a2cb0 0x7f5b300a5170 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:08.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:08.130+0000 7f5b4e876640 1 --2- 172.21.15.52:0/1345373042 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5b48107830 0x7f5b48100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:08.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:08.130+0000 7f5b4e876640 1 --2- 172.21.15.52:0/1345373042 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5b48104f00 0x7f5b48100580 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:08.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:08.130+0000 7f5b4e876640 1 --2- 172.21.15.52:0/1345373042 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5b481025d0 0x7f5b48101f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:08.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:08.130+0000 7f5b4e876640 1 -- 172.21.15.52:0/1345373042 >> 172.21.15.52:0/1345373042 conn(0x7f5b480fbf20 msgr2=0x7f5b48102ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:54:08.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:08.130+0000 7f5b4e876640 1 -- 172.21.15.52:0/1345373042 shutdown_connections 2024-06-10T02:54:08.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:08.130+0000 7f5b4e876640 1 -- 172.21.15.52:0/1345373042 wait complete. 2024-06-10T02:54:08.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:54:07 smithi120 ceph-mon[36634]: from='client.? 172.21.15.52:0/2371134135' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-06-10T02:54:08.140 INFO:teuthology.run_tasks:Running task check-counter... 2024-06-10T02:54:08.152 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-06-10T02:54:08.167 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-06-10T02:54:08.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:54:07 smithi188 ceph-mon[36599]: from='client.? 172.21.15.52:0/2371134135' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-06-10T02:54:08.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:08.273+0000 7f5c54d7e640 1 -- 172.21.15.52:0/2883632093 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5c501035a0 msgr2=0x7f5c50109e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:54:08.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:08.273+0000 7f5c54d7e640 1 --2- 172.21.15.52:0/2883632093 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5c501035a0 0x7f5c50109e30 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f5c48009ca0 tx=0x7f5c4802a8c0 comp rx=0 tx=0).stop 2024-06-10T02:54:08.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:08.273+0000 7f5c54d7e640 1 -- 172.21.15.52:0/2883632093 shutdown_connections 2024-06-10T02:54:08.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:08.273+0000 7f5c54d7e640 1 --2- 172.21.15.52:0/2883632093 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5c501035a0 0x7f5c50109e30 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:08.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:08.273+0000 7f5c54d7e640 1 --2- 172.21.15.52:0/2883632093 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5c50102be0 0x7f5c50103060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:08.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:08.273+0000 7f5c54d7e640 1 --2- 172.21.15.52:0/2883632093 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5c501019f0 0x7f5c50101e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:08.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:08.273+0000 7f5c54d7e640 1 -- 172.21.15.52:0/2883632093 >> 172.21.15.52:0/2883632093 conn(0x7f5c500fcfa0 msgr2=0x7f5c500ff3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:54:08.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:08.274+0000 7f5c54d7e640 1 -- 172.21.15.52:0/2883632093 shutdown_connections 2024-06-10T02:54:08.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:08.274+0000 7f5c54d7e640 1 -- 172.21.15.52:0/2883632093 wait complete. 2024-06-10T02:54:08.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:08.274+0000 7f5c54d7e640 1 Processor -- start 2024-06-10T02:54:08.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:08.274+0000 7f5c54d7e640 1 -- start start 2024-06-10T02:54:08.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:08.274+0000 7f5c54d7e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5c501019f0 0x7f5c50197d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:08.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:08.275+0000 7f5c54d7e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5c50102be0 0x7f5c50198240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:08.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:08.275+0000 7f5c54d7e640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5c501035a0 0x7f5c5019f270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:08.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:08.275+0000 7f5c54d7e640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f5c5010d670 con 0x7f5c501035a0 2024-06-10T02:54:08.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:08.275+0000 7f5c54d7e640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f5c5010d4f0 con 0x7f5c50102be0 2024-06-10T02:54:08.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:08.275+0000 7f5c54d7e640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f5c5010d7f0 con 0x7f5c501019f0 2024-06-10T02:54:08.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:08.275+0000 7f5c4ed9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5c501035a0 0x7f5c5019f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:08.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:08.275+0000 7f5c4ed9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5c501035a0 0x7f5c5019f270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:38986/0 (socket says 172.21.15.52:38986) 2024-06-10T02:54:08.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:08.275+0000 7f5c4ed9d640 1 -- 172.21.15.52:0/2458285763 learned_addr learned my addr 172.21.15.52:0/2458285763 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:54:08.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:08.275+0000 7f5c4e59c640 1 --2- 172.21.15.52:0/2458285763 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5c501019f0 0x7f5c50197d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:08.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:08.275+0000 7f5c4dd9b640 1 --2- 172.21.15.52:0/2458285763 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5c50102be0 0x7f5c50198240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:08.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:08.275+0000 7f5c4ed9d640 1 -- 172.21.15.52:0/2458285763 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5c501019f0 msgr2=0x7f5c50197d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:54:08.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:08.275+0000 7f5c4ed9d640 1 --2- 172.21.15.52:0/2458285763 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5c501019f0 0x7f5c50197d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:08.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:08.275+0000 7f5c4ed9d640 1 -- 172.21.15.52:0/2458285763 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5c50102be0 msgr2=0x7f5c50198240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:54:08.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:08.275+0000 7f5c4ed9d640 1 --2- 172.21.15.52:0/2458285763 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5c50102be0 0x7f5c50198240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:08.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:08.275+0000 7f5c4ed9d640 1 -- 172.21.15.52:0/2458285763 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c5019f910 con 0x7f5c501035a0 2024-06-10T02:54:08.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:08.275+0000 7f5c4ed9d640 1 --2- 172.21.15.52:0/2458285763 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5c501035a0 0x7f5c5019f270 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f5c48005e30 tx=0x7f5c48031ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:54:08.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:08.275+0000 7f5c3f7fe640 1 -- 172.21.15.52:0/2458285763 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c48003c30 con 0x7f5c501035a0 2024-06-10T02:54:08.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:08.275+0000 7f5c3f7fe640 1 -- 172.21.15.52:0/2458285763 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c48003dd0 con 0x7f5c501035a0 2024-06-10T02:54:08.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:08.275+0000 7f5c54d7e640 1 -- 172.21.15.52:0/2458285763 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c5019fba0 con 0x7f5c501035a0 2024-06-10T02:54:08.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:08.275+0000 7f5c3f7fe640 1 -- 172.21.15.52:0/2458285763 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c4802fde0 con 0x7f5c501035a0 2024-06-10T02:54:08.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:08.275+0000 7f5c54d7e640 1 -- 172.21.15.52:0/2458285763 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c501a0140 con 0x7f5c501035a0 2024-06-10T02:54:08.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:08.276+0000 7f5c54d7e640 1 -- 172.21.15.52:0/2458285763 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c50108d10 con 0x7f5c501035a0 2024-06-10T02:54:08.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:08.278+0000 7f5c3f7fe640 1 -- 172.21.15.52:0/2458285763 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f5c48004050 con 0x7f5c501035a0 2024-06-10T02:54:08.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:08.278+0000 7f5c3f7fe640 1 --2- 172.21.15.52:0/2458285763 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f5c380a2c10 0x7f5c380a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:08.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:08.278+0000 7f5c4e59c640 1 --2- 172.21.15.52:0/2458285763 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f5c380a2c10 0x7f5c380a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:08.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:08.278+0000 7f5c3f7fe640 1 -- 172.21.15.52:0/2458285763 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f5c480f0970 con 0x7f5c501035a0 2024-06-10T02:54:08.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:08.279+0000 7f5c4e59c640 1 --2- 172.21.15.52:0/2458285763 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f5c380a2c10 0x7f5c380a50d0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f5c501991d0 tx=0x7f5c4000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:54:08.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:08.282+0000 7f5c3f7fe640 1 -- 172.21.15.52:0/2458285763 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f5c480bd640 con 0x7f5c501035a0 2024-06-10T02:54:08.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:08.467+0000 7f5c54d7e640 1 -- 172.21.15.52:0/2458285763 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5c5002d4b0 con 0x7f5c501035a0 2024-06-10T02:54:08.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:08.469+0000 7f5c3f7fe640 1 -- 172.21.15.52:0/2458285763 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v100) ==== 74+0+24739 (secure 0 0 0) 0x7f5c480bcf40 con 0x7f5c501035a0 2024-06-10T02:54:08.468 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-10T02:54:08.468 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":100,"fsid":"d8676e9e-26d2-11ef-bc9b-c7b262605968","created":"2024-06-10T02:41:43.076837+0000","modified":"2024-06-10T02:51:45.708105+0000","last_up_change":"2024-06-10T02:48:57.297829+0000","last_in_change":"2024-06-10T02:48:36.865802+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-10T02:45:07.394690+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-10T02:49:57.926838+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-10T02:50:01.130393+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-10T02:50:10.655588+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e03c0691-cc5a-497a-bbc2-ce18c337f074","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":4255833408},{"type":"v1","addr":"172.21.15.52:6803","nonce":4255833408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":4255833408},{"type":"v1","addr":"172.21.15.52:6805","nonce":4255833408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":4255833408},{"type":"v1","addr":"172.21.15.52:6809","nonce":4255833408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":4255833408},{"type":"v1","addr":"172.21.15.52:6807","nonce":4255833408}]},"public_addr":"172.21.15.52:6803/4255833408","cluster_addr":"172.21.15.52:6805/4255833408","heartbeat_back_addr":"172.21.15.52:6809/4255833408","heartbeat_front_addr":"172.21.15.52:6807/4255833408","state":["exists","up"]},{"osd":1,"uuid":"a24d8da8-c5ec-44d7-ab30-56f6ce3efda7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":4116540053},{"type":"v1","addr":"172.21.15.120:6801","nonce":4116540053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":4116540053},{"type":"v1","addr":"172.21.15.120:6803","nonce":4116540053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":4116540053},{"type":"v1","addr":"172.21.15.120:6807","nonce":4116540053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":4116540053},{"type":"v1","addr":"172.21.15.120:6805","nonce":4116540053}]},"public_addr":"172.21.15.120:6801/4116540053","cluster_addr":"172.21.15.120:6803/4116540053","heartbeat_back_addr":"172.21.15.120:6807/4116540053","heartbeat_front_addr":"172.21.15.120:6805/4116540053","state":["exists","up"]},{"osd":2,"uuid":"a7162106-ccbf-45bd-bc0e-051ee43f4214","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":2624326999},{"type":"v1","addr":"172.21.15.188:6801","nonce":2624326999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":2624326999},{"type":"v1","addr":"172.21.15.188:6803","nonce":2624326999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":2624326999},{"type":"v1","addr":"172.21.15.188:6807","nonce":2624326999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":2624326999},{"type":"v1","addr":"172.21.15.188:6805","nonce":2624326999}]},"public_addr":"172.21.15.188:6801/2624326999","cluster_addr":"172.21.15.188:6803/2624326999","heartbeat_back_addr":"172.21.15.188:6807/2624326999","heartbeat_front_addr":"172.21.15.188:6805/2624326999","state":["exists","up"]},{"osd":3,"uuid":"53680244-b49a-4120-b24a-9ad7a6341dc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":116897011},{"type":"v1","addr":"172.21.15.52:6811","nonce":116897011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":116897011},{"type":"v1","addr":"172.21.15.52:6813","nonce":116897011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":116897011},{"type":"v1","addr":"172.21.15.52:6817","nonce":116897011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":116897011},{"type":"v1","addr":"172.21.15.52:6815","nonce":116897011}]},"public_addr":"172.21.15.52:6811/116897011","cluster_addr":"172.21.15.52:6813/116897011","heartbeat_back_addr":"172.21.15.52:6817/116897011","heartbeat_front_addr":"172.21.15.52:6815/116897011","state":["exists","up"]},{"osd":4,"uuid":"9f72f1f4-2ef3-49c5-8410-c108767d2ef5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":1320690621},{"type":"v1","addr":"172.21.15.120:6809","nonce":1320690621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":1320690621},{"type":"v1","addr":"172.21.15.120:6811","nonce":1320690621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":1320690621},{"type":"v1","addr":"172.21.15.120:6815","nonce":1320690621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":1320690621},{"type":"v1","addr":"172.21.15.120:6813","nonce":1320690621}]},"public_addr":"172.21.15.120:6809/1320690621","cluster_addr":"172.21.15.120:6811/1320690621","heartbeat_back_addr":"172.21.15.120:6815/1320690621","heartbeat_front_addr":"172.21.15.120:6813/1320690621","state":["exists","up"]},{"osd":5,"uuid":"001b5706-7f27-4fa4-9b69-969e6d765abc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3951947483},{"type":"v1","addr":"172.21.15.188:6809","nonce":3951947483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3951947483},{"type":"v1","addr":"172.21.15.188:6811","nonce":3951947483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3951947483},{"type":"v1","addr":"172.21.15.188:6815","nonce":3951947483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3951947483},{"type":"v1","addr":"172.21.15.188:6813","nonce":3951947483}]},"public_addr":"172.21.15.188:6809/3951947483","cluster_addr":"172.21.15.188:6811/3951947483","heartbeat_back_addr":"172.21.15.188:6815/3951947483","heartbeat_front_addr":"172.21.15.188:6813/3951947483","state":["exists","up"]},{"osd":6,"uuid":"140c903e-9b31-4a81-9ccd-2edc0a2decf9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":359084919},{"type":"v1","addr":"172.21.15.52:6819","nonce":359084919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":359084919},{"type":"v1","addr":"172.21.15.52:6821","nonce":359084919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":359084919},{"type":"v1","addr":"172.21.15.52:6825","nonce":359084919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":359084919},{"type":"v1","addr":"172.21.15.52:6823","nonce":359084919}]},"public_addr":"172.21.15.52:6819/359084919","cluster_addr":"172.21.15.52:6821/359084919","heartbeat_back_addr":"172.21.15.52:6825/359084919","heartbeat_front_addr":"172.21.15.52:6823/359084919","state":["exists","up"]},{"osd":7,"uuid":"57b735d4-5072-4ca2-a3ff-3763ad84f15a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":3737722011},{"type":"v1","addr":"172.21.15.120:6817","nonce":3737722011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6818","nonce":3737722011},{"type":"v1","addr":"172.21.15.120:6819","nonce":3737722011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":3737722011},{"type":"v1","addr":"172.21.15.120:6823","nonce":3737722011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":3737722011},{"type":"v1","addr":"172.21.15.120:6821","nonce":3737722011}]},"public_addr":"172.21.15.120:6817/3737722011","cluster_addr":"172.21.15.120:6819/3737722011","heartbeat_back_addr":"172.21.15.120:6823/3737722011","heartbeat_front_addr":"172.21.15.120:6821/3737722011","state":["exists","up"]},{"osd":8,"uuid":"4ffbb43b-854a-4eac-99cf-7af89bc65409","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1444828441},{"type":"v1","addr":"172.21.15.188:6817","nonce":1444828441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1444828441},{"type":"v1","addr":"172.21.15.188:6819","nonce":1444828441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1444828441},{"type":"v1","addr":"172.21.15.188:6823","nonce":1444828441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1444828441},{"type":"v1","addr":"172.21.15.188:6821","nonce":1444828441}]},"public_addr":"172.21.15.188:6817/1444828441","cluster_addr":"172.21.15.188:6819/1444828441","heartbeat_back_addr":"172.21.15.188:6823/1444828441","heartbeat_front_addr":"172.21.15.188:6821/1444828441","state":["exists","up"]},{"osd":9,"uuid":"5bfd985c-8acc-4d00-8561-40bbd541c07b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":924440658},{"type":"v1","addr":"172.21.15.52:6827","nonce":924440658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":924440658},{"type":"v1","addr":"172.21.15.52:6829","nonce":924440658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":924440658},{"type":"v1","addr":"172.21.15.52:6833","nonce":924440658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":924440658},{"type":"v1","addr":"172.21.15.52:6831","nonce":924440658}]},"public_addr":"172.21.15.52:6827/924440658","cluster_addr":"172.21.15.52:6829/924440658","heartbeat_back_addr":"172.21.15.52:6833/924440658","heartbeat_front_addr":"172.21.15.52:6831/924440658","state":["exists","up"]},{"osd":10,"uuid":"0180cddf-e109-42cc-829a-185604276f02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6824","nonce":1531769083},{"type":"v1","addr":"172.21.15.120:6825","nonce":1531769083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":1531769083},{"type":"v1","addr":"172.21.15.120:6827","nonce":1531769083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":1531769083},{"type":"v1","addr":"172.21.15.120:6831","nonce":1531769083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6828","nonce":1531769083},{"type":"v1","addr":"172.21.15.120:6829","nonce":1531769083}]},"public_addr":"172.21.15.120:6825/1531769083","cluster_addr":"172.21.15.120:6827/1531769083","heartbeat_back_addr":"172.21.15.120:6831/1531769083","heartbeat_front_addr":"172.21.15.120:6829/1531769083","state":["exists","up"]},{"osd":11,"uuid":"73aec750-82d1-4800-b6a0-4e305912f9ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":3769050170},{"type":"v1","addr":"172.21.15.188:6825","nonce":3769050170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":3769050170},{"type":"v1","addr":"172.21.15.188:6827","nonce":3769050170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":3769050170},{"type":"v1","addr":"172.21.15.188:6831","nonce":3769050170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":3769050170},{"type":"v1","addr":"172.21.15.188:6829","nonce":3769050170}]},"public_addr":"172.21.15.188:6825/3769050170","cluster_addr":"172.21.15.188:6827/3769050170","heartbeat_back_addr":"172.21.15.188:6831/3769050170","heartbeat_front_addr":"172.21.15.188:6829/3769050170","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:44:16.455752+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:44:40.219735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:45:03.558899+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:45:29.155004+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:45:53.981098+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:46:19.412020+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:46:45.462640+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:47:11.484578+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:47:36.879054+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:48:03.392446+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:48:29.422173+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:48:55.523423+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":4}]},{"pgid":"3.6","mappings":[{"from":3,"to":7}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.9","mappings":[{"from":5,"to":8}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.52:0/3535271671":"2024-06-11T02:42:42.872312+0000","172.21.15.52:0/1092934380":"2024-06-11T02:42:42.872312+0000","172.21.15.52:6800/896317057":"2024-06-11T02:42:42.872312+0000","172.21.15.52:6801/3934479642":"2024-06-11T02:42:15.498118+0000","172.21.15.52:6800/3934479642":"2024-06-11T02:42:15.498118+0000","172.21.15.52:0/2759694132":"2024-06-11T02:42:15.498118+0000","172.21.15.52:0/2593665631":"2024-06-11T02:42:15.498118+0000","172.21.15.52:0/3151504634":"2024-06-11T02:42:15.498118+0000","172.21.15.52:6801/896317057":"2024-06-11T02:42:42.872312+0000","172.21.15.52:0/1254480982":"2024-06-11T02:42:42.872312+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-10T02:54:08.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:08.471+0000 7f5c54d7e640 1 -- 172.21.15.52:0/2458285763 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f5c380a2c10 msgr2=0x7f5c380a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:54:08.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:08.471+0000 7f5c54d7e640 1 --2- 172.21.15.52:0/2458285763 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f5c380a2c10 0x7f5c380a50d0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f5c501991d0 tx=0x7f5c4000b390 comp rx=0 tx=0).stop 2024-06-10T02:54:08.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:08.471+0000 7f5c54d7e640 1 -- 172.21.15.52:0/2458285763 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5c501035a0 msgr2=0x7f5c5019f270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:54:08.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:08.471+0000 7f5c54d7e640 1 --2- 172.21.15.52:0/2458285763 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5c501035a0 0x7f5c5019f270 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f5c48005e30 tx=0x7f5c48031ee0 comp rx=0 tx=0).stop 2024-06-10T02:54:08.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:08.471+0000 7f5c54d7e640 1 -- 172.21.15.52:0/2458285763 shutdown_connections 2024-06-10T02:54:08.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:08.471+0000 7f5c54d7e640 1 --2- 172.21.15.52:0/2458285763 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5c501035a0 0x7f5c5019f270 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:08.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:08.471+0000 7f5c54d7e640 1 --2- 172.21.15.52:0/2458285763 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5c50102be0 0x7f5c50198240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:08.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:08.471+0000 7f5c54d7e640 1 --2- 172.21.15.52:0/2458285763 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f5c380a2c10 0x7f5c380a50d0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:08.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:08.471+0000 7f5c54d7e640 1 --2- 172.21.15.52:0/2458285763 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5c501019f0 0x7f5c50197d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:08.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:08.471+0000 7f5c54d7e640 1 -- 172.21.15.52:0/2458285763 >> 172.21.15.52:0/2458285763 conn(0x7f5c500fcfa0 msgr2=0x7f5c50105ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:54:08.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:08.471+0000 7f5c54d7e640 1 -- 172.21.15.52:0/2458285763 shutdown_connections 2024-06-10T02:54:08.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:08.471+0000 7f5c54d7e640 1 -- 172.21.15.52:0/2458285763 wait complete. 2024-06-10T02:54:08.476 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 10 02:54:08 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mgr-x[32595]: 2024-06-10T02:54:08.115+0000 7fe28db65700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-10T02:54:08.476 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 10 02:54:08 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mgr-x[32595]: 2024-06-10T02:54:08.115+0000 7fe28db65700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-10T02:54:08.476 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 10 02:54:08 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mgr-x[32595]: 2024-06-10T02:54:08.115+0000 7fe28db65700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-10T02:54:08.476 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 10 02:54:08 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mgr-x[32595]: 2024-06-10T02:54:08.115+0000 7fe28db65700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-10T02:54:08.476 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 10 02:54:08 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mgr-x[32595]: 2024-06-10T02:54:08.115+0000 7fe28db65700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-10T02:54:08.481 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-06-10T02:45:07.394690+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-06-10T02:49:57.926838+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-06-10T02:50:01.130393+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-06-10T02:50:10.655588+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '96', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-06-10T02:54:08.482 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-06-10T02:54:08.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:08.590+0000 7f65e7c95640 1 -- 172.21.15.52:0/2192139369 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f65e0104e50 msgr2=0x7f65e0107240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:54:08.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:08.590+0000 7f65e7c95640 1 --2- 172.21.15.52:0/2192139369 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f65e0104e50 0x7f65e0107240 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f65d0009900 tx=0x7f65d002a410 comp rx=0 tx=0).stop 2024-06-10T02:54:08.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:08.590+0000 7f65e7c95640 1 -- 172.21.15.52:0/2192139369 shutdown_connections 2024-06-10T02:54:08.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:08.590+0000 7f65e7c95640 1 --2- 172.21.15.52:0/2192139369 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f65e0107780 0x7f65e0109b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:08.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:08.590+0000 7f65e7c95640 1 --2- 172.21.15.52:0/2192139369 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f65e0104e50 0x7f65e0107240 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:08.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:08.590+0000 7f65e7c95640 1 --2- 172.21.15.52:0/2192139369 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f65e0102520 0x7f65e0104910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:08.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:08.590+0000 7f65e7c95640 1 -- 172.21.15.52:0/2192139369 >> 172.21.15.52:0/2192139369 conn(0x7f65e00fbe90 msgr2=0x7f65e00fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:54:08.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:08.591+0000 7f65e7c95640 1 -- 172.21.15.52:0/2192139369 shutdown_connections 2024-06-10T02:54:08.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:08.591+0000 7f65e7c95640 1 -- 172.21.15.52:0/2192139369 wait complete. 2024-06-10T02:54:08.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:08.591+0000 7f65e7c95640 1 Processor -- start 2024-06-10T02:54:08.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:08.591+0000 7f65e7c95640 1 -- start start 2024-06-10T02:54:08.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:08.591+0000 7f65e7c95640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f65e0102520 0x7f65e0197d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:08.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:08.591+0000 7f65e7c95640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f65e0104e50 0x7f65e0198280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:08.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:08.591+0000 7f65e7c95640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f65e0107780 0x7f65e019f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:08.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:08.591+0000 7f65e7c95640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f65e010d680 con 0x7f65e0102520 2024-06-10T02:54:08.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:08.591+0000 7f65e7c95640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f65e010d500 con 0x7f65e0104e50 2024-06-10T02:54:08.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:08.591+0000 7f65e7c95640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f65e010d800 con 0x7f65e0107780 2024-06-10T02:54:08.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:08.591+0000 7f65e5a31640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f65e0102520 0x7f65e0197d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:08.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:08.591+0000 7f65e5a31640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f65e0102520 0x7f65e0197d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:39006/0 (socket says 172.21.15.52:39006) 2024-06-10T02:54:08.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:08.591+0000 7f65e5a31640 1 -- 172.21.15.52:0/4027781687 learned_addr learned my addr 172.21.15.52:0/4027781687 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:54:08.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:08.592+0000 7f65e6232640 1 --2- 172.21.15.52:0/4027781687 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f65e0107780 0x7f65e019f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:08.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:08.592+0000 7f65e5230640 1 --2- 172.21.15.52:0/4027781687 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f65e0104e50 0x7f65e0198280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:08.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:08.592+0000 7f65e5a31640 1 -- 172.21.15.52:0/4027781687 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f65e0107780 msgr2=0x7f65e019f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:54:08.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:08.592+0000 7f65e5a31640 1 --2- 172.21.15.52:0/4027781687 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f65e0107780 0x7f65e019f2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:08.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:08.592+0000 7f65e5a31640 1 -- 172.21.15.52:0/4027781687 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f65e0104e50 msgr2=0x7f65e0198280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:54:08.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:08.592+0000 7f65e5a31640 1 --2- 172.21.15.52:0/4027781687 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f65e0104e50 0x7f65e0198280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:08.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:08.592+0000 7f65e5a31640 1 -- 172.21.15.52:0/4027781687 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65e019f950 con 0x7f65e0102520 2024-06-10T02:54:08.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:08.592+0000 7f65e5230640 1 --2- 172.21.15.52:0/4027781687 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f65e0104e50 0x7f65e0198280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T02:54:08.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:08.592+0000 7f65e5a31640 1 --2- 172.21.15.52:0/4027781687 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f65e0102520 0x7f65e0197d40 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f65dc00d950 tx=0x7f65dc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:54:08.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:08.592+0000 7f65d6ffd640 1 -- 172.21.15.52:0/4027781687 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65dc017070 con 0x7f65e0102520 2024-06-10T02:54:08.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:08.592+0000 7f65d6ffd640 1 -- 172.21.15.52:0/4027781687 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f65dc009c30 con 0x7f65e0102520 2024-06-10T02:54:08.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:08.592+0000 7f65e7c95640 1 -- 172.21.15.52:0/4027781687 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65e019fc40 con 0x7f65e0102520 2024-06-10T02:54:08.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:08.592+0000 7f65d6ffd640 1 -- 172.21.15.52:0/4027781687 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65dc00f6b0 con 0x7f65e0102520 2024-06-10T02:54:08.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:08.592+0000 7f65e7c95640 1 -- 172.21.15.52:0/4027781687 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65e01a0190 con 0x7f65e0102520 2024-06-10T02:54:08.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:08.593+0000 7f65e7c95640 1 -- 172.21.15.52:0/4027781687 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65e00fd980 con 0x7f65e0102520 2024-06-10T02:54:08.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:08.594+0000 7f65d6ffd640 1 -- 172.21.15.52:0/4027781687 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f65dc0109d0 con 0x7f65e0102520 2024-06-10T02:54:08.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:08.595+0000 7f65d6ffd640 1 --2- 172.21.15.52:0/4027781687 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f65cc0a2c10 0x7f65cc0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:08.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:08.595+0000 7f65e5230640 1 --2- 172.21.15.52:0/4027781687 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f65cc0a2c10 0x7f65cc0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:08.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:08.595+0000 7f65d6ffd640 1 -- 172.21.15.52:0/4027781687 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f65dc0cf0d0 con 0x7f65e0102520 2024-06-10T02:54:08.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:08.596+0000 7f65e5230640 1 --2- 172.21.15.52:0/4027781687 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f65cc0a2c10 0x7f65cc0a50d0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f65d0000c00 tx=0x7f65d0031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:54:08.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:08.600+0000 7f65d6ffd640 1 -- 172.21.15.52:0/4027781687 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f65dc009de0 con 0x7f65e0102520 2024-06-10T02:54:08.788 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:08.788+0000 7f65e7c95640 1 -- 172.21.15.52:0/4027781687 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f65e002d2e0 con 0x7f65e0102520 2024-06-10T02:54:08.788 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:08.788+0000 7f65d6ffd640 1 -- 172.21.15.52:0/4027781687 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v100) ==== 93+0+10 (secure 0 0 0) 0x7f65dc09bc20 con 0x7f65e0102520 2024-06-10T02:54:08.788 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 1 2024-06-10T02:54:08.790 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:08.790+0000 7f65e7c95640 1 -- 172.21.15.52:0/4027781687 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f65cc0a2c10 msgr2=0x7f65cc0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:54:08.790 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:08.790+0000 7f65e7c95640 1 --2- 172.21.15.52:0/4027781687 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f65cc0a2c10 0x7f65cc0a50d0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f65d0000c00 tx=0x7f65d0031400 comp rx=0 tx=0).stop 2024-06-10T02:54:08.790 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:08.790+0000 7f65e7c95640 1 -- 172.21.15.52:0/4027781687 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f65e0102520 msgr2=0x7f65e0197d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:54:08.790 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:08.790+0000 7f65e7c95640 1 --2- 172.21.15.52:0/4027781687 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f65e0102520 0x7f65e0197d40 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f65dc00d950 tx=0x7f65dc00de10 comp rx=0 tx=0).stop 2024-06-10T02:54:08.790 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:08.790+0000 7f65e7c95640 1 -- 172.21.15.52:0/4027781687 shutdown_connections 2024-06-10T02:54:08.790 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:08.790+0000 7f65e7c95640 1 --2- 172.21.15.52:0/4027781687 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f65cc0a2c10 0x7f65cc0a50d0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:08.790 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:08.790+0000 7f65e7c95640 1 --2- 172.21.15.52:0/4027781687 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f65e0107780 0x7f65e019f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:08.790 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:08.790+0000 7f65e7c95640 1 --2- 172.21.15.52:0/4027781687 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f65e0104e50 0x7f65e0198280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:08.790 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:08.790+0000 7f65e7c95640 1 --2- 172.21.15.52:0/4027781687 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f65e0102520 0x7f65e0197d40 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:08.790 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:08.790+0000 7f65e7c95640 1 -- 172.21.15.52:0/4027781687 >> 172.21.15.52:0/4027781687 conn(0x7f65e00fbe90 msgr2=0x7f65e0102dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:54:08.790 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:08.790+0000 7f65e7c95640 1 -- 172.21.15.52:0/4027781687 shutdown_connections 2024-06-10T02:54:08.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:08.790+0000 7f65e7c95640 1 -- 172.21.15.52:0/4027781687 wait complete. 2024-06-10T02:54:08.801 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-06-10T02:54:08.911 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:08.910+0000 7fd6072b3640 1 -- 172.21.15.52:0/3329676334 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd600101af0 msgr2=0x7fd600101f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:54:08.912 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:08.910+0000 7fd6072b3640 1 --2- 172.21.15.52:0/3329676334 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd600101af0 0x7fd600101f70 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fd5f4009900 tx=0x7fd5f402a410 comp rx=0 tx=0).stop 2024-06-10T02:54:08.912 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:08.910+0000 7fd6072b3640 1 -- 172.21.15.52:0/3329676334 shutdown_connections 2024-06-10T02:54:08.912 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:08.910+0000 7fd6072b3640 1 --2- 172.21.15.52:0/3329676334 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd6001024b0 0x7fd600108d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:08.912 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:08.910+0000 7fd6072b3640 1 --2- 172.21.15.52:0/3329676334 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd600101af0 0x7fd600101f70 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:08.912 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:08.910+0000 7fd6072b3640 1 --2- 172.21.15.52:0/3329676334 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd600100900 0x7fd600100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:08.912 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:08.910+0000 7fd6072b3640 1 -- 172.21.15.52:0/3329676334 >> 172.21.15.52:0/3329676334 conn(0x7fd6000fbe90 msgr2=0x7fd6000fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:54:08.912 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:08.910+0000 7fd6072b3640 1 -- 172.21.15.52:0/3329676334 shutdown_connections 2024-06-10T02:54:08.912 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:08.910+0000 7fd6072b3640 1 -- 172.21.15.52:0/3329676334 wait complete. 2024-06-10T02:54:08.912 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:08.911+0000 7fd6072b3640 1 Processor -- start 2024-06-10T02:54:08.912 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:08.911+0000 7fd6072b3640 1 -- start start 2024-06-10T02:54:08.912 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:08.911+0000 7fd6072b3640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd600100900 0x7fd600197db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:08.913 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:08.911+0000 7fd6072b3640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd600101af0 0x7fd6001982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:08.913 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:08.911+0000 7fd6072b3640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd6001024b0 0x7fd60019f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:08.913 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:08.911+0000 7fd6072b3640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fd60010c620 con 0x7fd600101af0 2024-06-10T02:54:08.913 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:08.911+0000 7fd6072b3640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd60010c4a0 con 0x7fd600100900 2024-06-10T02:54:08.913 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:08.911+0000 7fd6072b3640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fd60010c7a0 con 0x7fd6001024b0 2024-06-10T02:54:08.913 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:08.911+0000 7fd60484e640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd600101af0 0x7fd6001982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:08.913 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:08.911+0000 7fd60484e640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd600101af0 0x7fd6001982f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:39016/0 (socket says 172.21.15.52:39016) 2024-06-10T02:54:08.913 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:08.911+0000 7fd60484e640 1 -- 172.21.15.52:0/737784212 learned_addr learned my addr 172.21.15.52:0/737784212 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:54:08.913 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:08.911+0000 7fd605850640 1 --2- 172.21.15.52:0/737784212 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd6001024b0 0x7fd60019f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:08.913 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:08.911+0000 7fd60504f640 1 --2- 172.21.15.52:0/737784212 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd600100900 0x7fd600197db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:08.913 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:08.911+0000 7fd60484e640 1 -- 172.21.15.52:0/737784212 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd6001024b0 msgr2=0x7fd60019f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:54:08.913 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:08.911+0000 7fd60484e640 1 --2- 172.21.15.52:0/737784212 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd6001024b0 0x7fd60019f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:08.913 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:08.911+0000 7fd60484e640 1 -- 172.21.15.52:0/737784212 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd600100900 msgr2=0x7fd600197db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:54:08.914 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:08.911+0000 7fd60484e640 1 --2- 172.21.15.52:0/737784212 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd600100900 0x7fd600197db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:08.914 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:08.911+0000 7fd60484e640 1 -- 172.21.15.52:0/737784212 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd60019f9d0 con 0x7fd600101af0 2024-06-10T02:54:08.914 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:08.911+0000 7fd60504f640 1 --2- 172.21.15.52:0/737784212 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd600100900 0x7fd600197db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T02:54:08.914 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:08.912+0000 7fd60484e640 1 --2- 172.21.15.52:0/737784212 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd600101af0 0x7fd6001982f0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fd5f4009230 tx=0x7fd5f4004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:54:08.914 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:08.912+0000 7fd5f27fc640 1 -- 172.21.15.52:0/737784212 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5f403f030 con 0x7fd600101af0 2024-06-10T02:54:08.914 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:08.912+0000 7fd6072b3640 1 -- 172.21.15.52:0/737784212 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd60019fc60 con 0x7fd600101af0 2024-06-10T02:54:08.914 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:08.912+0000 7fd5f27fc640 1 -- 172.21.15.52:0/737784212 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd5f4003ce0 con 0x7fd600101af0 2024-06-10T02:54:08.914 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:08.912+0000 7fd5f27fc640 1 -- 172.21.15.52:0/737784212 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5f4035070 con 0x7fd600101af0 2024-06-10T02:54:08.914 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:08.912+0000 7fd6072b3640 1 -- 172.21.15.52:0/737784212 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6001a0150 con 0x7fd600101af0 2024-06-10T02:54:08.915 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:08.913+0000 7fd6072b3640 1 -- 172.21.15.52:0/737784212 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6001a0380 con 0x7fd600101af0 2024-06-10T02:54:08.915 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:08.914+0000 7fd5f27fc640 1 -- 172.21.15.52:0/737784212 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fd5f4042030 con 0x7fd600101af0 2024-06-10T02:54:08.915 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:08.915+0000 7fd5f27fc640 1 --2- 172.21.15.52:0/737784212 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fd5ec0a2b90 0x7fd5ec0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:08.915 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:08.915+0000 7fd60504f640 1 --2- 172.21.15.52:0/737784212 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fd5ec0a2b90 0x7fd5ec0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:08.915 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:08.915+0000 7fd5f27fc640 1 -- 172.21.15.52:0/737784212 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7fd5f40efbc0 con 0x7fd600101af0 2024-06-10T02:54:08.915 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:08.915+0000 7fd60504f640 1 --2- 172.21.15.52:0/737784212 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fd5ec0a2b90 0x7fd5ec0a5050 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fd600101950 tx=0x7fd5fc008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:54:08.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:08.920+0000 7fd5f27fc640 1 -- 172.21.15.52:0/737784212 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fd5f40bc710 con 0x7fd600101af0 2024-06-10T02:54:09.104 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:09.104+0000 7fd6072b3640 1 -- 172.21.15.52:0/737784212 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fd6001a0710 con 0x7fd600101af0 2024-06-10T02:54:09.104 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:09.105+0000 7fd5f27fc640 1 -- 172.21.15.52:0/737784212 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v100) ==== 104+0+11 (secure 0 0 0) 0x7fd5f40bc010 con 0x7fd600101af0 2024-06-10T02:54:09.104 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 32 2024-06-10T02:54:09.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:09.107+0000 7fd6072b3640 1 -- 172.21.15.52:0/737784212 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fd5ec0a2b90 msgr2=0x7fd5ec0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:54:09.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:09.107+0000 7fd6072b3640 1 --2- 172.21.15.52:0/737784212 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fd5ec0a2b90 0x7fd5ec0a5050 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fd600101950 tx=0x7fd5fc008040 comp rx=0 tx=0).stop 2024-06-10T02:54:09.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:09.107+0000 7fd6072b3640 1 -- 172.21.15.52:0/737784212 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd600101af0 msgr2=0x7fd6001982f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:54:09.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:09.107+0000 7fd6072b3640 1 --2- 172.21.15.52:0/737784212 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd600101af0 0x7fd6001982f0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fd5f4009230 tx=0x7fd5f4004430 comp rx=0 tx=0).stop 2024-06-10T02:54:09.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:09.107+0000 7fd6072b3640 1 -- 172.21.15.52:0/737784212 shutdown_connections 2024-06-10T02:54:09.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:09.107+0000 7fd6072b3640 1 --2- 172.21.15.52:0/737784212 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fd5ec0a2b90 0x7fd5ec0a5050 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:09.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:09.107+0000 7fd6072b3640 1 --2- 172.21.15.52:0/737784212 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd6001024b0 0x7fd60019f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:09.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:09.107+0000 7fd6072b3640 1 --2- 172.21.15.52:0/737784212 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd600101af0 0x7fd6001982f0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:09.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:09.107+0000 7fd6072b3640 1 --2- 172.21.15.52:0/737784212 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd600100900 0x7fd600197db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:09.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:09.107+0000 7fd6072b3640 1 -- 172.21.15.52:0/737784212 >> 172.21.15.52:0/737784212 conn(0x7fd6000fbe90 msgr2=0x7fd6000fd950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:54:09.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:09.107+0000 7fd6072b3640 1 -- 172.21.15.52:0/737784212 shutdown_connections 2024-06-10T02:54:09.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:09.107+0000 7fd6072b3640 1 -- 172.21.15.52:0/737784212 wait complete. 2024-06-10T02:54:09.116 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:54:08 smithi052 ceph-mon[32362]: pgmap v432: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 341 B/s wr, 11 op/s 2024-06-10T02:54:09.116 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:54:08 smithi052 ceph-mon[32362]: from='client.? 172.21.15.52:0/2371134135' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-06-10T02:54:09.116 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:54:08 smithi052 ceph-mon[32362]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-10T02:54:09.116 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:54:08 smithi052 ceph-mon[32362]: from='client.? 172.21.15.52:0/2458285763' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T02:54:09.117 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-06-10T02:54:09.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:54:08 smithi120 ceph-mon[36634]: pgmap v432: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 341 B/s wr, 11 op/s 2024-06-10T02:54:09.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:54:08 smithi120 ceph-mon[36634]: from='client.? 172.21.15.52:0/2371134135' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-06-10T02:54:09.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:54:08 smithi120 ceph-mon[36634]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-10T02:54:09.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:54:08 smithi120 ceph-mon[36634]: from='client.? 172.21.15.52:0/2458285763' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T02:54:09.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:54:08 smithi188 ceph-mon[36599]: pgmap v432: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 341 B/s wr, 11 op/s 2024-06-10T02:54:09.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:54:08 smithi188 ceph-mon[36599]: from='client.? 172.21.15.52:0/2371134135' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-06-10T02:54:09.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:54:08 smithi188 ceph-mon[36599]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-10T02:54:09.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:54:08 smithi188 ceph-mon[36599]: from='client.? 172.21.15.52:0/2458285763' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T02:54:09.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:09.225+0000 7f7ae9542640 1 -- 172.21.15.52:0/1353968731 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7ae41035a0 msgr2=0x7f7ae4109e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:54:09.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:09.225+0000 7f7ae9542640 1 --2- 172.21.15.52:0/1353968731 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7ae41035a0 0x7f7ae4109e30 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f7adc009900 tx=0x7f7adc02a410 comp rx=0 tx=0).stop 2024-06-10T02:54:09.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:09.226+0000 7f7ae9542640 1 -- 172.21.15.52:0/1353968731 shutdown_connections 2024-06-10T02:54:09.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:09.226+0000 7f7ae9542640 1 --2- 172.21.15.52:0/1353968731 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7ae41035a0 0x7f7ae4109e30 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:09.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:09.226+0000 7f7ae9542640 1 --2- 172.21.15.52:0/1353968731 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7ae4102be0 0x7f7ae4103060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:09.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:09.226+0000 7f7ae9542640 1 --2- 172.21.15.52:0/1353968731 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7ae41019f0 0x7f7ae4101e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:09.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:09.226+0000 7f7ae9542640 1 -- 172.21.15.52:0/1353968731 >> 172.21.15.52:0/1353968731 conn(0x7f7ae40fcf80 msgr2=0x7f7ae40ff3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:54:09.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:09.226+0000 7f7ae9542640 1 -- 172.21.15.52:0/1353968731 shutdown_connections 2024-06-10T02:54:09.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:09.226+0000 7f7ae9542640 1 -- 172.21.15.52:0/1353968731 wait complete. 2024-06-10T02:54:09.226 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:09.226+0000 7f7ae9542640 1 Processor -- start 2024-06-10T02:54:09.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:09.226+0000 7f7ae9542640 1 -- start start 2024-06-10T02:54:09.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:09.226+0000 7f7ae9542640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7ae41019f0 0x7f7ae4197be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:09.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:09.226+0000 7f7ae9542640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7ae4102be0 0x7f7ae4198120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:09.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:09.227+0000 7f7ae9542640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7ae41035a0 0x7f7ae419f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:09.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:09.227+0000 7f7ae9542640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f7ae410d650 con 0x7f7ae4102be0 2024-06-10T02:54:09.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:09.227+0000 7f7ae9542640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f7ae410d4d0 con 0x7f7ae41019f0 2024-06-10T02:54:09.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:09.227+0000 7f7ae9542640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f7ae410d7d0 con 0x7f7ae41035a0 2024-06-10T02:54:09.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:09.227+0000 7f7ae27fc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7ae4102be0 0x7f7ae4198120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:09.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:09.227+0000 7f7ae27fc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7ae4102be0 0x7f7ae4198120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:39034/0 (socket says 172.21.15.52:39034) 2024-06-10T02:54:09.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:09.227+0000 7f7ae27fc640 1 -- 172.21.15.52:0/73104869 learned_addr learned my addr 172.21.15.52:0/73104869 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:54:09.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:09.227+0000 7f7ae37fe640 1 --2- 172.21.15.52:0/73104869 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7ae41035a0 0x7f7ae419f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:09.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:09.227+0000 7f7ae2ffd640 1 --2- 172.21.15.52:0/73104869 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7ae41019f0 0x7f7ae4197be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:09.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:09.227+0000 7f7ae27fc640 1 -- 172.21.15.52:0/73104869 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7ae41035a0 msgr2=0x7f7ae419f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:54:09.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:09.227+0000 7f7ae27fc640 1 --2- 172.21.15.52:0/73104869 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7ae41035a0 0x7f7ae419f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:09.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:09.227+0000 7f7ae27fc640 1 -- 172.21.15.52:0/73104869 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7ae41019f0 msgr2=0x7f7ae4197be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:54:09.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:09.227+0000 7f7ae27fc640 1 --2- 172.21.15.52:0/73104869 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7ae41019f0 0x7f7ae4197be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:09.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:09.227+0000 7f7ae27fc640 1 -- 172.21.15.52:0/73104869 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ae419f800 con 0x7f7ae4102be0 2024-06-10T02:54:09.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:09.227+0000 7f7ae27fc640 1 --2- 172.21.15.52:0/73104869 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7ae4102be0 0x7f7ae4198120 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f7ad800c920 tx=0x7f7ad800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:54:09.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:09.227+0000 7f7acbfff640 1 -- 172.21.15.52:0/73104869 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ad8007c70 con 0x7f7ae4102be0 2024-06-10T02:54:09.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:09.227+0000 7f7ae9542640 1 -- 172.21.15.52:0/73104869 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ae419faf0 con 0x7f7ae4102be0 2024-06-10T02:54:09.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:09.227+0000 7f7acbfff640 1 -- 172.21.15.52:0/73104869 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ad8007e10 con 0x7f7ae4102be0 2024-06-10T02:54:09.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:09.227+0000 7f7acbfff640 1 -- 172.21.15.52:0/73104869 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ad8018890 con 0x7f7ae4102be0 2024-06-10T02:54:09.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:09.227+0000 7f7ae9542640 1 -- 172.21.15.52:0/73104869 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ae41a0090 con 0x7f7ae4102be0 2024-06-10T02:54:09.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:09.229+0000 7f7ae9542640 1 -- 172.21.15.52:0/73104869 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ae4108d10 con 0x7f7ae4102be0 2024-06-10T02:54:09.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:09.230+0000 7f7acbfff640 1 -- 172.21.15.52:0/73104869 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f7ad8018a50 con 0x7f7ae4102be0 2024-06-10T02:54:09.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:09.231+0000 7f7acbfff640 1 --2- 172.21.15.52:0/73104869 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f7acc0a2b90 0x7f7acc0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:09.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:09.231+0000 7f7ae2ffd640 1 --2- 172.21.15.52:0/73104869 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f7acc0a2b90 0x7f7acc0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:09.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:09.231+0000 7f7acbfff640 1 -- 172.21.15.52:0/73104869 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f7ad80cf880 con 0x7f7ae4102be0 2024-06-10T02:54:09.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:09.232+0000 7f7ae2ffd640 1 --2- 172.21.15.52:0/73104869 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f7acc0a2b90 0x7f7acc0a5050 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f7ae4102a40 tx=0x7f7ad4009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:54:09.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:09.235+0000 7f7acbfff640 1 -- 172.21.15.52:0/73104869 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f7ad8014030 con 0x7f7ae4102be0 2024-06-10T02:54:09.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:09.420+0000 7f7ae9542640 1 -- 172.21.15.52:0/73104869 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f7ae402d2e0 con 0x7f7ae4102be0 2024-06-10T02:54:09.420 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:09.421+0000 7f7acbfff640 1 -- 172.21.15.52:0/73104869 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v100) ==== 100+0+11 (secure 0 0 0) 0x7f7ad809c460 con 0x7f7ae4102be0 2024-06-10T02:54:09.420 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 64 2024-06-10T02:54:09.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:09.423+0000 7f7ae9542640 1 -- 172.21.15.52:0/73104869 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f7acc0a2b90 msgr2=0x7f7acc0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:54:09.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:09.423+0000 7f7ae9542640 1 --2- 172.21.15.52:0/73104869 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f7acc0a2b90 0x7f7acc0a5050 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f7ae4102a40 tx=0x7f7ad4009500 comp rx=0 tx=0).stop 2024-06-10T02:54:09.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:09.423+0000 7f7ae9542640 1 -- 172.21.15.52:0/73104869 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7ae4102be0 msgr2=0x7f7ae4198120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:54:09.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:09.423+0000 7f7ae9542640 1 --2- 172.21.15.52:0/73104869 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7ae4102be0 0x7f7ae4198120 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f7ad800c920 tx=0x7f7ad800cde0 comp rx=0 tx=0).stop 2024-06-10T02:54:09.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:09.423+0000 7f7ae9542640 1 -- 172.21.15.52:0/73104869 shutdown_connections 2024-06-10T02:54:09.423 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:09.423+0000 7f7ae9542640 1 --2- 172.21.15.52:0/73104869 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f7acc0a2b90 0x7f7acc0a5050 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:09.423 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:09.423+0000 7f7ae9542640 1 --2- 172.21.15.52:0/73104869 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7ae41035a0 0x7f7ae419f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:09.423 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:09.423+0000 7f7ae9542640 1 --2- 172.21.15.52:0/73104869 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7ae4102be0 0x7f7ae4198120 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:09.423 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:09.423+0000 7f7ae9542640 1 --2- 172.21.15.52:0/73104869 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7ae41019f0 0x7f7ae4197be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:09.423 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:09.423+0000 7f7ae9542640 1 -- 172.21.15.52:0/73104869 >> 172.21.15.52:0/73104869 conn(0x7f7ae40fcf80 msgr2=0x7f7ae4105ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:54:09.423 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:09.423+0000 7f7ae9542640 1 -- 172.21.15.52:0/73104869 shutdown_connections 2024-06-10T02:54:09.423 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:09.423+0000 7f7ae9542640 1 -- 172.21.15.52:0/73104869 wait complete. 2024-06-10T02:54:09.433 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-06-10T02:54:09.543 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:09.541+0000 7f11c4fa2640 1 -- 172.21.15.52:0/3844889721 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f11c0102490 msgr2=0x7f11c0108d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:54:09.543 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:09.542+0000 7f11c4fa2640 1 --2- 172.21.15.52:0/3844889721 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f11c0102490 0x7f11c0108d20 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f11b8009900 tx=0x7f11b802a410 comp rx=0 tx=0).stop 2024-06-10T02:54:09.543 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:09.542+0000 7f11c4fa2640 1 -- 172.21.15.52:0/3844889721 shutdown_connections 2024-06-10T02:54:09.543 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:09.542+0000 7f11c4fa2640 1 --2- 172.21.15.52:0/3844889721 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f11c0102490 0x7f11c0108d20 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:09.543 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:09.542+0000 7f11c4fa2640 1 --2- 172.21.15.52:0/3844889721 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f11c0101ad0 0x7f11c0101f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:09.543 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:09.542+0000 7f11c4fa2640 1 --2- 172.21.15.52:0/3844889721 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f11c01008e0 0x7f11c0100d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:09.543 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:09.542+0000 7f11c4fa2640 1 -- 172.21.15.52:0/3844889721 >> 172.21.15.52:0/3844889721 conn(0x7f11c00fbe50 msgr2=0x7f11c00fe290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:54:09.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:09.542+0000 7f11c4fa2640 1 -- 172.21.15.52:0/3844889721 shutdown_connections 2024-06-10T02:54:09.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:09.542+0000 7f11c4fa2640 1 -- 172.21.15.52:0/3844889721 wait complete. 2024-06-10T02:54:09.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:09.542+0000 7f11c4fa2640 1 Processor -- start 2024-06-10T02:54:09.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:09.542+0000 7f11c4fa2640 1 -- start start 2024-06-10T02:54:09.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:09.543+0000 7f11c4fa2640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f11c01008e0 0x7f11c0197c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:09.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:09.543+0000 7f11c4fa2640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f11c0101ad0 0x7f11c01981b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:09.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:09.543+0000 7f11c4fa2640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f11c0102490 0x7f11c019f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:09.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:09.543+0000 7f11c4fa2640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f11c010c540 con 0x7f11c01008e0 2024-06-10T02:54:09.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:09.543+0000 7f11c4fa2640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f11c010c3c0 con 0x7f11c0101ad0 2024-06-10T02:54:09.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:09.543+0000 7f11c4fa2640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f11c010c6c0 con 0x7f11c0102490 2024-06-10T02:54:09.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:09.543+0000 7f11be59c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f11c01008e0 0x7f11c0197c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:09.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:09.543+0000 7f11be59c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f11c01008e0 0x7f11c0197c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:39042/0 (socket says 172.21.15.52:39042) 2024-06-10T02:54:09.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:09.543+0000 7f11be59c640 1 -- 172.21.15.52:0/3045464952 learned_addr learned my addr 172.21.15.52:0/3045464952 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:54:09.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:09.543+0000 7f11bdd9b640 1 --2- 172.21.15.52:0/3045464952 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f11c0101ad0 0x7f11c01981b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:09.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:09.543+0000 7f11bed9d640 1 --2- 172.21.15.52:0/3045464952 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f11c0102490 0x7f11c019f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:09.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:09.543+0000 7f11be59c640 1 -- 172.21.15.52:0/3045464952 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f11c0102490 msgr2=0x7f11c019f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:54:09.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:09.543+0000 7f11be59c640 1 --2- 172.21.15.52:0/3045464952 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f11c0102490 0x7f11c019f1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:09.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:09.543+0000 7f11be59c640 1 -- 172.21.15.52:0/3045464952 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f11c0101ad0 msgr2=0x7f11c01981b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:54:09.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:09.543+0000 7f11be59c640 1 --2- 172.21.15.52:0/3045464952 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f11c0101ad0 0x7f11c01981b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:09.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:09.543+0000 7f11be59c640 1 -- 172.21.15.52:0/3045464952 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11c019f880 con 0x7f11c01008e0 2024-06-10T02:54:09.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:09.543+0000 7f11bed9d640 1 --2- 172.21.15.52:0/3045464952 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f11c0102490 0x7f11c019f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T02:54:09.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:09.543+0000 7f11bdd9b640 1 --2- 172.21.15.52:0/3045464952 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f11c0101ad0 0x7f11c01981b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T02:54:09.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:09.543+0000 7f11be59c640 1 --2- 172.21.15.52:0/3045464952 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f11c01008e0 0x7f11c0197c70 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f11b000ccb0 tx=0x7f11b000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:54:09.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:09.543+0000 7f11af7fe640 1 -- 172.21.15.52:0/3045464952 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11b0003990 con 0x7f11c01008e0 2024-06-10T02:54:09.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:09.543+0000 7f11af7fe640 1 -- 172.21.15.52:0/3045464952 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f11b0003b30 con 0x7f11c01008e0 2024-06-10T02:54:09.546 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:09.543+0000 7f11c4fa2640 1 -- 172.21.15.52:0/3045464952 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11c019fb70 con 0x7f11c01008e0 2024-06-10T02:54:09.546 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:09.543+0000 7f11af7fe640 1 -- 172.21.15.52:0/3045464952 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11b000f7b0 con 0x7f11c01008e0 2024-06-10T02:54:09.546 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:09.543+0000 7f11c4fa2640 1 -- 172.21.15.52:0/3045464952 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11c01a00c0 con 0x7f11c01008e0 2024-06-10T02:54:09.547 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:09.544+0000 7f11c4fa2640 1 -- 172.21.15.52:0/3045464952 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11c01a0290 con 0x7f11c01008e0 2024-06-10T02:54:09.547 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:09.546+0000 7f11af7fe640 1 -- 172.21.15.52:0/3045464952 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f11b000f950 con 0x7f11c01008e0 2024-06-10T02:54:09.547 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:09.546+0000 7f11af7fe640 1 --2- 172.21.15.52:0/3045464952 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f11a80a2b90 0x7f11a80a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:09.547 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:09.546+0000 7f11bdd9b640 1 --2- 172.21.15.52:0/3045464952 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f11a80a2b90 0x7f11a80a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:09.547 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:09.546+0000 7f11af7fe640 1 -- 172.21.15.52:0/3045464952 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f11b00cf9d0 con 0x7f11c01008e0 2024-06-10T02:54:09.547 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:09.547+0000 7f11bdd9b640 1 --2- 172.21.15.52:0/3045464952 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f11a80a2b90 0x7f11a80a5050 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f11b400baa0 tx=0x7f11b4005d90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:54:09.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:09.550+0000 7f11af7fe640 1 -- 172.21.15.52:0/3045464952 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f11b0014030 con 0x7f11c01008e0 2024-06-10T02:54:09.737 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:09.737+0000 7f11c4fa2640 1 -- 172.21.15.52:0/3045464952 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f11c01a0460 con 0x7f11c01008e0 2024-06-10T02:54:09.737 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:09.738+0000 7f11af7fe640 1 -- 172.21.15.52:0/3045464952 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v100) ==== 103+0+11 (secure 0 0 0) 0x7f11b009c580 con 0x7f11c01008e0 2024-06-10T02:54:09.737 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 32 2024-06-10T02:54:09.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:09.740+0000 7f11c4fa2640 1 -- 172.21.15.52:0/3045464952 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f11a80a2b90 msgr2=0x7f11a80a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:54:09.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:09.740+0000 7f11c4fa2640 1 --2- 172.21.15.52:0/3045464952 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f11a80a2b90 0x7f11a80a5050 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f11b400baa0 tx=0x7f11b4005d90 comp rx=0 tx=0).stop 2024-06-10T02:54:09.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:09.740+0000 7f11c4fa2640 1 -- 172.21.15.52:0/3045464952 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f11c01008e0 msgr2=0x7f11c0197c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:54:09.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:09.740+0000 7f11c4fa2640 1 --2- 172.21.15.52:0/3045464952 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f11c01008e0 0x7f11c0197c70 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f11b000ccb0 tx=0x7f11b000a620 comp rx=0 tx=0).stop 2024-06-10T02:54:09.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:09.740+0000 7f11c4fa2640 1 -- 172.21.15.52:0/3045464952 shutdown_connections 2024-06-10T02:54:09.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:09.740+0000 7f11c4fa2640 1 --2- 172.21.15.52:0/3045464952 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f11a80a2b90 0x7f11a80a5050 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:09.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:09.740+0000 7f11c4fa2640 1 --2- 172.21.15.52:0/3045464952 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f11c0102490 0x7f11c019f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:09.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:09.740+0000 7f11c4fa2640 1 --2- 172.21.15.52:0/3045464952 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f11c0101ad0 0x7f11c01981b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:09.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:09.740+0000 7f11c4fa2640 1 --2- 172.21.15.52:0/3045464952 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f11c01008e0 0x7f11c0197c70 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:09.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:09.740+0000 7f11c4fa2640 1 -- 172.21.15.52:0/3045464952 >> 172.21.15.52:0/3045464952 conn(0x7f11c00fbe50 msgr2=0x7f11c00fd8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:54:09.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:09.740+0000 7f11c4fa2640 1 -- 172.21.15.52:0/3045464952 shutdown_connections 2024-06-10T02:54:09.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:09.740+0000 7f11c4fa2640 1 -- 172.21.15.52:0/3045464952 wait complete. 2024-06-10T02:54:09.750 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-06-10T02:54:09.751 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T02:54:10.006 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:54:10.129 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:54:09 smithi052 ceph-mon[32362]: from='client.15285 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T02:54:10.129 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:54:09 smithi052 ceph-mon[32362]: from='client.? 172.21.15.52:0/4027781687' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-06-10T02:54:10.129 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:54:09 smithi052 ceph-mon[32362]: from='client.? 172.21.15.52:0/737784212' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-06-10T02:54:10.129 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:54:09 smithi052 ceph-mon[32362]: from='client.? 172.21.15.52:0/73104869' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-06-10T02:54:10.129 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:54:09 smithi052 ceph-mon[32362]: from='client.? 172.21.15.52:0/3045464952' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-06-10T02:54:10.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:54:09 smithi120 ceph-mon[36634]: from='client.15285 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T02:54:10.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:54:09 smithi120 ceph-mon[36634]: from='client.? 172.21.15.52:0/4027781687' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-06-10T02:54:10.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:54:09 smithi120 ceph-mon[36634]: from='client.? 172.21.15.52:0/737784212' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-06-10T02:54:10.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:54:09 smithi120 ceph-mon[36634]: from='client.? 172.21.15.52:0/73104869' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-06-10T02:54:10.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:54:09 smithi120 ceph-mon[36634]: from='client.? 172.21.15.52:0/3045464952' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-06-10T02:54:10.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:54:09 smithi188 ceph-mon[36599]: from='client.15285 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T02:54:10.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:54:09 smithi188 ceph-mon[36599]: from='client.? 172.21.15.52:0/4027781687' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-06-10T02:54:10.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:54:09 smithi188 ceph-mon[36599]: from='client.? 172.21.15.52:0/737784212' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-06-10T02:54:10.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:54:09 smithi188 ceph-mon[36599]: from='client.? 172.21.15.52:0/73104869' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-06-10T02:54:10.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:54:09 smithi188 ceph-mon[36599]: from='client.? 172.21.15.52:0/3045464952' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-06-10T02:54:11.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:54:10 smithi120 ceph-mon[36634]: pgmap v433: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 341 B/s wr, 9 op/s 2024-06-10T02:54:11.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:54:10 smithi120 ceph-mon[36634]: mgrmap e19: x(active, since 11m), standbys: z, y 2024-06-10T02:54:11.141 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:54:10 smithi052 ceph-mon[32362]: pgmap v433: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 341 B/s wr, 9 op/s 2024-06-10T02:54:11.141 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:54:10 smithi052 ceph-mon[32362]: mgrmap e19: x(active, since 11m), standbys: z, y 2024-06-10T02:54:11.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:54:10 smithi188 ceph-mon[36599]: pgmap v433: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 341 B/s wr, 9 op/s 2024-06-10T02:54:11.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:54:10 smithi188 ceph-mon[36599]: mgrmap e19: x(active, since 11m), standbys: z, y 2024-06-10T02:54:11.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:11.775+0000 7fde081b6700 1 -- 172.21.15.52:0/2226943085 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fde00108530 msgr2=0x7fde00108910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:54:11.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:11.775+0000 7fde081b6700 1 --2- 172.21.15.52:0/2226943085 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fde00108530 0x7fde00108910 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fddfc009990 tx=0x7fddfc009ca0 comp rx=0 tx=0).stop 2024-06-10T02:54:11.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:11.775+0000 7fde081b6700 1 -- 172.21.15.52:0/2226943085 shutdown_connections 2024-06-10T02:54:11.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:11.776+0000 7fde081b6700 1 --2- 172.21.15.52:0/2226943085 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fde0006b5c0 0x7fde0010f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:11.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:11.776+0000 7fde081b6700 1 --2- 172.21.15.52:0/2226943085 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fde00108530 0x7fde00108910 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:11.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:11.776+0000 7fde081b6700 1 --2- 172.21.15.52:0/2226943085 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fde0006ac00 0x7fde0006b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:11.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:11.776+0000 7fde081b6700 1 -- 172.21.15.52:0/2226943085 >> 172.21.15.52:0/2226943085 conn(0x7fde00077400 msgr2=0x7fde00077810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:54:11.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:11.776+0000 7fde081b6700 1 -- 172.21.15.52:0/2226943085 shutdown_connections 2024-06-10T02:54:11.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:11.776+0000 7fde081b6700 1 -- 172.21.15.52:0/2226943085 wait complete. 2024-06-10T02:54:11.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:11.776+0000 7fde081b6700 1 Processor -- start 2024-06-10T02:54:11.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:11.777+0000 7fde081b6700 1 -- start start 2024-06-10T02:54:11.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:11.777+0000 7fde081b6700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fde0006ac00 0x7fde00198d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:11.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:11.777+0000 7fde081b6700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fde0006b5c0 0x7fde001992d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:11.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:11.777+0000 7fde081b6700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fde00108530 0x7fde0019d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:11.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:11.777+0000 7fde081b6700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fde00112860 con 0x7fde00108530 2024-06-10T02:54:11.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:11.777+0000 7fde081b6700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fde001126e0 con 0x7fde0006ac00 2024-06-10T02:54:11.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:11.777+0000 7fde081b6700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fde001129e0 con 0x7fde0006b5c0 2024-06-10T02:54:11.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:11.777+0000 7fde06753700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fde00108530 0x7fde0019d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:11.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:11.777+0000 7fde06753700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fde00108530 0x7fde0019d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60026/0 (socket says 172.21.15.52:60026) 2024-06-10T02:54:11.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:11.777+0000 7fde05f52700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fde0006ac00 0x7fde00198d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:11.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:11.777+0000 7fde06753700 1 -- 172.21.15.52:0/4165619478 learned_addr learned my addr 172.21.15.52:0/4165619478 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:54:11.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:11.777+0000 7fde05751700 1 --2- 172.21.15.52:0/4165619478 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fde0006b5c0 0x7fde001992d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:11.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:11.777+0000 7fde06753700 1 -- 172.21.15.52:0/4165619478 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fde0006b5c0 msgr2=0x7fde001992d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:54:11.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:11.777+0000 7fde06753700 1 --2- 172.21.15.52:0/4165619478 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fde0006b5c0 0x7fde001992d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:11.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:11.777+0000 7fde06753700 1 -- 172.21.15.52:0/4165619478 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fde0006ac00 msgr2=0x7fde00198d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:54:11.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:11.777+0000 7fde06753700 1 --2- 172.21.15.52:0/4165619478 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fde0006ac00 0x7fde00198d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:11.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:11.777+0000 7fde06753700 1 -- 172.21.15.52:0/4165619478 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde0019dda0 con 0x7fde00108530 2024-06-10T02:54:11.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:11.778+0000 7fde06753700 1 --2- 172.21.15.52:0/4165619478 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fde00108530 0x7fde0019d660 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fddf800b970 tx=0x7fddf800bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:54:11.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:11.778+0000 7fddf2ffd700 1 -- 172.21.15.52:0/4165619478 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddf800d860 con 0x7fde00108530 2024-06-10T02:54:11.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:11.778+0000 7fddf2ffd700 1 -- 172.21.15.52:0/4165619478 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fddf800e430 con 0x7fde00108530 2024-06-10T02:54:11.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:11.778+0000 7fde081b6700 1 -- 172.21.15.52:0/4165619478 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde0019e090 con 0x7fde00108530 2024-06-10T02:54:11.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:11.778+0000 7fddf2ffd700 1 -- 172.21.15.52:0/4165619478 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddf8017660 con 0x7fde00108530 2024-06-10T02:54:11.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:11.778+0000 7fde081b6700 1 -- 172.21.15.52:0/4165619478 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde001a5910 con 0x7fde00108530 2024-06-10T02:54:11.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:11.779+0000 7fde081b6700 1 -- 172.21.15.52:0/4165619478 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde0010aa30 con 0x7fde00108530 2024-06-10T02:54:11.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:11.780+0000 7fddf2ffd700 1 -- 172.21.15.52:0/4165619478 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7fddf800e5e0 con 0x7fde00108530 2024-06-10T02:54:11.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:11.781+0000 7fddf2ffd700 1 --2- 172.21.15.52:0/4165619478 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fddec0a2cb0 0x7fddec0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:11.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:11.781+0000 7fde05f52700 1 --2- 172.21.15.52:0/4165619478 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fddec0a2cb0 0x7fddec0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:11.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:11.781+0000 7fddf2ffd700 1 -- 172.21.15.52:0/4165619478 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7fddf80cf110 con 0x7fde00108530 2024-06-10T02:54:11.781 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:11.781+0000 7fde05f52700 1 --2- 172.21.15.52:0/4165619478 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fddec0a2cb0 0x7fddec0a5170 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fddfc00b5c0 tx=0x7fddfc005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:54:11.784 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:11.785+0000 7fddf2ffd700 1 -- 172.21.15.52:0/4165619478 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fddf8013030 con 0x7fde00108530 2024-06-10T02:54:12.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:12.041+0000 7fde081b6700 1 -- 172.21.15.52:0/4165619478 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fde0010a410 con 0x7fde00108530 2024-06-10T02:54:12.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:12.041+0000 7fddf2ffd700 1 -- 172.21.15.52:0/4165619478 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11938 (secure 0 0 0) 0x7fddf809bb60 con 0x7fde00108530 2024-06-10T02:54:12.041 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-10T02:54:12.041 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/889673247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":889673247},{"type":"v1","addr":"172.21.15.52:6835","nonce":889673247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/3615663511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":3615663511},{"type":"v1","addr":"172.21.15.52:6837","nonce":3615663511}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T02:50:03.889766+0000","modified":"2024-06-10T02:54:07.086405+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24484,"mds_1":24469,"mds_2":24463,"mds_3":24449,"mds_4":24428},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14571":{"gid":14571,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6839/3059599042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":3059599042},{"type":"v1","addr":"172.21.15.52:6839","nonce":3059599042}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14583":{"gid":14583,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6841/3039275521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":3039275521},{"type":"v1","addr":"172.21.15.52:6841","nonce":3039275521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24383":{"gid":24383,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6833/2609002896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":2609002896},{"type":"v1","addr":"172.21.15.120:6833","nonce":2609002896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6835/1996370288","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":1996370288},{"type":"v1","addr":"172.21.15.120:6835","nonce":1996370288}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6833/529564979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":529564979},{"type":"v1","addr":"172.21.15.188:6833","nonce":529564979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":25,"addr":"172.21.15.120:6837/4132440952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":4132440952},{"type":"v1","addr":"172.21.15.120:6837","nonce":4132440952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"b","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.120:6839/1672436487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1672436487},{"type":"v1","addr":"172.21.15.120:6839","nonce":1672436487}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":2,"incarnation":22,"state":"up:active","state_seq":26,"addr":"172.21.15.188:6835/3904544259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":3904544259},{"type":"v1","addr":"172.21.15.188:6835","nonce":3904544259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.188:6837/819317726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":819317726},{"type":"v1","addr":"172.21.15.188:6837","nonce":819317726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.188:6839/310444836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":310444836},{"type":"v1","addr":"172.21.15.188:6839","nonce":310444836}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24484,"qdb_cluster":[24469,24484,24463,24428,24449]},"id":1}]} 2024-06-10T02:54:12.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:12.044+0000 7fde081b6700 1 -- 172.21.15.52:0/4165619478 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fddec0a2cb0 msgr2=0x7fddec0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:54:12.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:12.044+0000 7fde081b6700 1 --2- 172.21.15.52:0/4165619478 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fddec0a2cb0 0x7fddec0a5170 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fddfc00b5c0 tx=0x7fddfc005960 comp rx=0 tx=0).stop 2024-06-10T02:54:12.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:12.044+0000 7fde081b6700 1 -- 172.21.15.52:0/4165619478 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fde00108530 msgr2=0x7fde0019d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:54:12.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:12.044+0000 7fde081b6700 1 --2- 172.21.15.52:0/4165619478 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fde00108530 0x7fde0019d660 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fddf800b970 tx=0x7fddf800bc80 comp rx=0 tx=0).stop 2024-06-10T02:54:12.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:12.044+0000 7fde081b6700 1 -- 172.21.15.52:0/4165619478 shutdown_connections 2024-06-10T02:54:12.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:12.044+0000 7fde081b6700 1 --2- 172.21.15.52:0/4165619478 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fddec0a2cb0 0x7fddec0a5170 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:12.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:12.044+0000 7fde081b6700 1 --2- 172.21.15.52:0/4165619478 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fde00108530 0x7fde0019d660 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:12.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:12.044+0000 7fde081b6700 1 --2- 172.21.15.52:0/4165619478 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fde0006ac00 0x7fde00198d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:12.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:12.044+0000 7fde081b6700 1 --2- 172.21.15.52:0/4165619478 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fde0006b5c0 0x7fde001992d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:12.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:12.044+0000 7fde081b6700 1 -- 172.21.15.52:0/4165619478 >> 172.21.15.52:0/4165619478 conn(0x7fde00077400 msgr2=0x7fde00100270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:54:12.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:12.044+0000 7fde081b6700 1 -- 172.21.15.52:0/4165619478 shutdown_connections 2024-06-10T02:54:12.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:12.044+0000 7fde081b6700 1 -- 172.21.15.52:0/4165619478 wait complete. 2024-06-10T02:54:12.044 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 30 2024-06-10T02:54:12.617 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-06-10T02:54:12.617 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-06-10T02:54:12.618 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-06-10T02:54:12.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:12.727+0000 7f67c6153640 1 -- 172.21.15.52:0/3618643942 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f67c0104db0 msgr2=0x7f67c01071a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:54:12.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:12.727+0000 7f67c6153640 1 --2- 172.21.15.52:0/3618643942 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f67c0104db0 0x7f67c01071a0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f67b4009900 tx=0x7f67b402a410 comp rx=0 tx=0).stop 2024-06-10T02:54:12.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:12.728+0000 7f67c6153640 1 -- 172.21.15.52:0/3618643942 shutdown_connections 2024-06-10T02:54:12.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:12.728+0000 7f67c6153640 1 --2- 172.21.15.52:0/3618643942 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f67c01076e0 0x7f67c0109b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:12.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:12.728+0000 7f67c6153640 1 --2- 172.21.15.52:0/3618643942 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f67c0104db0 0x7f67c01071a0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:12.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:12.728+0000 7f67c6153640 1 --2- 172.21.15.52:0/3618643942 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f67c0102480 0x7f67c0104870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:12.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:12.728+0000 7f67c6153640 1 -- 172.21.15.52:0/3618643942 >> 172.21.15.52:0/3618643942 conn(0x7f67c00fbe90 msgr2=0x7f67c00fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:54:12.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:12.728+0000 7f67c6153640 1 -- 172.21.15.52:0/3618643942 shutdown_connections 2024-06-10T02:54:12.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:12.728+0000 7f67c6153640 1 -- 172.21.15.52:0/3618643942 wait complete. 2024-06-10T02:54:12.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:12.728+0000 7f67c6153640 1 Processor -- start 2024-06-10T02:54:12.730 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:12.728+0000 7f67c6153640 1 -- start start 2024-06-10T02:54:12.730 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:12.728+0000 7f67c6153640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f67c0102480 0x7f67c0197d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:12.730 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:12.728+0000 7f67c6153640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f67c0104db0 0x7f67c0198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:12.730 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:12.729+0000 7f67c6153640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f67c01076e0 0x7f67c019f300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:12.730 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:12.729+0000 7f67c6153640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f67c010d680 con 0x7f67c01076e0 2024-06-10T02:54:12.730 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:12.729+0000 7f67c6153640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f67c010d500 con 0x7f67c0104db0 2024-06-10T02:54:12.730 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:12.729+0000 7f67c6153640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f67c010d800 con 0x7f67c0102480 2024-06-10T02:54:12.730 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:12.729+0000 7f67bffff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f67c01076e0 0x7f67c019f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:12.730 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:12.729+0000 7f67bffff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f67c01076e0 0x7f67c019f300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60040/0 (socket says 172.21.15.52:60040) 2024-06-10T02:54:12.730 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:12.729+0000 7f67bffff640 1 -- 172.21.15.52:0/2417080089 learned_addr learned my addr 172.21.15.52:0/2417080089 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:54:12.730 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:12.729+0000 7f67bf7fe640 1 --2- 172.21.15.52:0/2417080089 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f67c0102480 0x7f67c0197d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:12.730 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:12.729+0000 7f67beffd640 1 --2- 172.21.15.52:0/2417080089 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f67c0104db0 0x7f67c0198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:12.731 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:12.729+0000 7f67bffff640 1 -- 172.21.15.52:0/2417080089 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f67c0102480 msgr2=0x7f67c0197d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:54:12.731 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:12.729+0000 7f67bffff640 1 --2- 172.21.15.52:0/2417080089 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f67c0102480 0x7f67c0197d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:12.731 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:12.729+0000 7f67bffff640 1 -- 172.21.15.52:0/2417080089 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f67c0104db0 msgr2=0x7f67c0198290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:54:12.731 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:12.729+0000 7f67bffff640 1 --2- 172.21.15.52:0/2417080089 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f67c0104db0 0x7f67c0198290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:12.731 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:12.729+0000 7f67bffff640 1 -- 172.21.15.52:0/2417080089 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67c019f970 con 0x7f67c01076e0 2024-06-10T02:54:12.731 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:12.729+0000 7f67bf7fe640 1 --2- 172.21.15.52:0/2417080089 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f67c0102480 0x7f67c0197d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T02:54:12.731 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:12.729+0000 7f67bffff640 1 --2- 172.21.15.52:0/2417080089 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f67c01076e0 0x7f67c019f300 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f67b800eaa0 tx=0x7f67b800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:54:12.731 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:12.729+0000 7f67bcff9640 1 -- 172.21.15.52:0/2417080089 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67b800cdb0 con 0x7f67c01076e0 2024-06-10T02:54:12.731 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:12.729+0000 7f67c6153640 1 -- 172.21.15.52:0/2417080089 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67c019fc60 con 0x7f67c01076e0 2024-06-10T02:54:12.731 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:12.729+0000 7f67bcff9640 1 -- 172.21.15.52:0/2417080089 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67b8005020 con 0x7f67c01076e0 2024-06-10T02:54:12.731 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:12.729+0000 7f67bcff9640 1 -- 172.21.15.52:0/2417080089 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67b80116d0 con 0x7f67c01076e0 2024-06-10T02:54:12.731 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:12.729+0000 7f67c6153640 1 -- 172.21.15.52:0/2417080089 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67c01a01b0 con 0x7f67c01076e0 2024-06-10T02:54:12.732 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:12.730+0000 7f67c6153640 1 -- 172.21.15.52:0/2417080089 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67c00fd980 con 0x7f67c01076e0 2024-06-10T02:54:12.732 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:12.732+0000 7f67bcff9640 1 -- 172.21.15.52:0/2417080089 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7f67b80054d0 con 0x7f67c01076e0 2024-06-10T02:54:12.732 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:12.732+0000 7f67bcff9640 1 --2- 172.21.15.52:0/2417080089 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f67a80a2c20 0x7f67a80a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:12.732 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:12.732+0000 7f67bcff9640 1 -- 172.21.15.52:0/2417080089 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f67b80cf850 con 0x7f67c01076e0 2024-06-10T02:54:12.732 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:12.732+0000 7f67bf7fe640 1 --2- 172.21.15.52:0/2417080089 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f67a80a2c20 0x7f67a80a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:12.733 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:12.733+0000 7f67bf7fe640 1 --2- 172.21.15.52:0/2417080089 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f67a80a2c20 0x7f67a80a50e0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f67b0009b40 tx=0x7f67b0005e50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:54:12.736 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:12.737+0000 7f67bcff9640 1 -- 172.21.15.52:0/2417080089 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f67b8016030 con 0x7f67c01076e0 2024-06-10T02:54:12.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:12.920+0000 7f67c6153640 1 -- 172.21.15.52:0/2417080089 --> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f67c0061de0 con 0x7f67a80a2c20 2024-06-10T02:54:12.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:12.930+0000 7f67bcff9640 1 -- 172.21.15.52:0/2417080089 <== mgr.14150 v2:172.21.15.52:6800/3214763969 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+341018 (secure 0 0 0) 0x7f67c0061de0 con 0x7f67a80a2c20 2024-06-10T02:54:12.934 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-10T02:54:12.936 INFO:teuthology.orchestra.run.smithi052.stderr:dumped all 2024-06-10T02:54:12.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:12.933+0000 7f67c6153640 1 -- 172.21.15.52:0/2417080089 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f67a80a2c20 msgr2=0x7f67a80a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:54:12.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:12.933+0000 7f67c6153640 1 --2- 172.21.15.52:0/2417080089 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f67a80a2c20 0x7f67a80a50e0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f67b0009b40 tx=0x7f67b0005e50 comp rx=0 tx=0).stop 2024-06-10T02:54:12.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:12.933+0000 7f67c6153640 1 -- 172.21.15.52:0/2417080089 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f67c01076e0 msgr2=0x7f67c019f300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:54:12.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:12.933+0000 7f67c6153640 1 --2- 172.21.15.52:0/2417080089 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f67c01076e0 0x7f67c019f300 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f67b800eaa0 tx=0x7f67b800ef60 comp rx=0 tx=0).stop 2024-06-10T02:54:12.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:12.933+0000 7f67c6153640 1 -- 172.21.15.52:0/2417080089 shutdown_connections 2024-06-10T02:54:12.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:12.933+0000 7f67c6153640 1 --2- 172.21.15.52:0/2417080089 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f67c01076e0 0x7f67c019f300 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:12.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:12.933+0000 7f67c6153640 1 --2- 172.21.15.52:0/2417080089 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f67c0104db0 0x7f67c0198290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:12.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:12.933+0000 7f67c6153640 1 --2- 172.21.15.52:0/2417080089 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f67a80a2c20 0x7f67a80a50e0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:12.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:12.933+0000 7f67c6153640 1 --2- 172.21.15.52:0/2417080089 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f67c0102480 0x7f67c0197d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:12.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:12.933+0000 7f67c6153640 1 -- 172.21.15.52:0/2417080089 >> 172.21.15.52:0/2417080089 conn(0x7f67c00fbe90 msgr2=0x7f67c0102d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:54:12.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:12.934+0000 7f67c6153640 1 -- 172.21.15.52:0/2417080089 shutdown_connections 2024-06-10T02:54:12.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:12.934+0000 7f67c6153640 1 -- 172.21.15.52:0/2417080089 wait complete. 2024-06-10T02:54:12.943 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:54:12 smithi052 ceph-mon[32362]: pgmap v434: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 597 B/s wr, 9 op/s 2024-06-10T02:54:12.943 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:54:12 smithi052 ceph-mon[32362]: from='client.? 172.21.15.52:0/4165619478' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T02:54:12.944 INFO:teuthology.orchestra.run.smithi052.stdout:{"pg_ready":true,"pg_map":{"version":434,"stamp":"2024-06-10T02:54:11.043435+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":662668,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1955,"num_read_kb":1104,"num_write":456,"num_write_kb":3103,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1181318,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":873,"ondisk_log_size":873,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":379348,"kb_used_data":13556,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124562476,"statfs":{"total":1151940427776,"available":1151551975424,"internally_reserved":0,"allocated":13881344,"data_stored":8837538,"data_compressed":497715,"data_compressed_allocated":675840,"data_compressed_original":2850816,"omap_allocated":19141,"internal_metadata":374453563},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":616,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":110,"num_read_kb":55,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003443"},"pg_stats":[{"pgid":"3.3f","version":"100'8","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:43.001859+0000","last_change":"2024-06-10T02:50:02.209146+0000","last_active":"2024-06-10T02:52:43.001859+0000","last_peered":"2024-06-10T02:52:43.001859+0000","last_clean":"2024-06-10T02:52:43.001859+0000","last_became_active":"2024-06-10T02:50:02.208027+0000","last_became_peered":"2024-06-10T02:50:02.208027+0000","last_unstale":"2024-06-10T02:52:43.001859+0000","last_undegraded":"2024-06-10T02:52:43.001859+0000","last_fullsized":"2024-06-10T02:52:43.001859+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:35:34.765849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"100'4","reported_seq":43,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.989740+0000","last_change":"2024-06-10T02:50:02.198767+0000","last_active":"2024-06-10T02:52:42.989740+0000","last_peered":"2024-06-10T02:52:42.989740+0000","last_clean":"2024-06-10T02:52:42.989740+0000","last_became_active":"2024-06-10T02:50:02.198450+0000","last_became_peered":"2024-06-10T02:50:02.198450+0000","last_unstale":"2024-06-10T02:52:42.989740+0000","last_undegraded":"2024-06-10T02:52:42.989740+0000","last_fullsized":"2024-06-10T02:52:42.989740+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:49:56.666083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"100'7","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:43.000326+0000","last_change":"2024-06-10T02:50:02.208885+0000","last_active":"2024-06-10T02:52:43.000326+0000","last_peered":"2024-06-10T02:52:43.000326+0000","last_clean":"2024-06-10T02:52:43.000326+0000","last_became_active":"2024-06-10T02:50:02.206953+0000","last_became_peered":"2024-06-10T02:50:02.206953+0000","last_unstale":"2024-06-10T02:52:43.000326+0000","last_undegraded":"2024-06-10T02:52:43.000326+0000","last_fullsized":"2024-06-10T02:52:43.000326+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:41:23.928939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"100'8","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.994111+0000","last_change":"2024-06-10T02:50:02.208632+0000","last_active":"2024-06-10T02:52:42.994111+0000","last_peered":"2024-06-10T02:52:42.994111+0000","last_clean":"2024-06-10T02:52:42.994111+0000","last_became_active":"2024-06-10T02:50:02.207587+0000","last_became_peered":"2024-06-10T02:50:02.207587+0000","last_unstale":"2024-06-10T02:52:42.994111+0000","last_undegraded":"2024-06-10T02:52:42.994111+0000","last_fullsized":"2024-06-10T02:52:42.994111+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T14:34:36.720170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"100'4","reported_seq":43,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.973606+0000","last_change":"2024-06-10T02:50:02.197966+0000","last_active":"2024-06-10T02:52:42.973606+0000","last_peered":"2024-06-10T02:52:42.973606+0000","last_clean":"2024-06-10T02:52:42.973606+0000","last_became_active":"2024-06-10T02:50:02.197612+0000","last_became_peered":"2024-06-10T02:50:02.197612+0000","last_unstale":"2024-06-10T02:52:42.973606+0000","last_undegraded":"2024-06-10T02:52:42.973606+0000","last_fullsized":"2024-06-10T02:52:42.973606+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:24:53.804613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"100'6","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.987354+0000","last_change":"2024-06-10T02:50:02.201952+0000","last_active":"2024-06-10T02:52:42.987354+0000","last_peered":"2024-06-10T02:52:42.987354+0000","last_clean":"2024-06-10T02:52:42.987354+0000","last_became_active":"2024-06-10T02:50:02.200828+0000","last_became_peered":"2024-06-10T02:50:02.200828+0000","last_unstale":"2024-06-10T02:52:42.987354+0000","last_undegraded":"2024-06-10T02:52:42.987354+0000","last_fullsized":"2024-06-10T02:52:42.987354+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:46:28.518846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"100'5","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.990208+0000","last_change":"2024-06-10T02:50:02.203483+0000","last_active":"2024-06-10T02:52:42.990208+0000","last_peered":"2024-06-10T02:52:42.990208+0000","last_clean":"2024-06-10T02:52:42.990208+0000","last_became_active":"2024-06-10T02:50:02.203153+0000","last_became_peered":"2024-06-10T02:50:02.203153+0000","last_unstale":"2024-06-10T02:52:42.990208+0000","last_undegraded":"2024-06-10T02:52:42.990208+0000","last_fullsized":"2024-06-10T02:52:42.990208+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:29:30.422449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"100'9","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.995630+0000","last_change":"2024-06-10T02:50:02.201686+0000","last_active":"2024-06-10T02:52:42.995630+0000","last_peered":"2024-06-10T02:52:42.995630+0000","last_clean":"2024-06-10T02:52:42.995630+0000","last_became_active":"2024-06-10T02:50:02.200973+0000","last_became_peered":"2024-06-10T02:50:02.200973+0000","last_unstale":"2024-06-10T02:52:42.995630+0000","last_undegraded":"2024-06-10T02:52:42.995630+0000","last_fullsized":"2024-06-10T02:52:42.995630+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:41:13.712675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"100'11","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:43.001422+0000","last_change":"2024-06-10T02:50:02.210497+0000","last_active":"2024-06-10T02:52:43.001422+0000","last_peered":"2024-06-10T02:52:43.001422+0000","last_clean":"2024-06-10T02:52:43.001422+0000","last_became_active":"2024-06-10T02:50:02.209675+0000","last_became_peered":"2024-06-10T02:50:02.209675+0000","last_unstale":"2024-06-10T02:52:43.001422+0000","last_undegraded":"2024-06-10T02:52:43.001422+0000","last_fullsized":"2024-06-10T02:52:43.001422+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:50:29.879669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"100'16","reported_seq":61,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.994975+0000","last_change":"2024-06-10T02:50:44.741139+0000","last_active":"2024-06-10T02:52:42.994975+0000","last_peered":"2024-06-10T02:52:42.994975+0000","last_clean":"2024-06-10T02:52:42.994975+0000","last_became_active":"2024-06-10T02:50:44.740685+0000","last_became_peered":"2024-06-10T02:50:44.740685+0000","last_unstale":"2024-06-10T02:52:42.994975+0000","last_undegraded":"2024-06-10T02:52:42.994975+0000","last_fullsized":"2024-06-10T02:52:42.994975+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:58:59.872732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"100'6","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.999574+0000","last_change":"2024-06-10T02:50:02.206993+0000","last_active":"2024-06-10T02:52:42.999574+0000","last_peered":"2024-06-10T02:52:42.999574+0000","last_clean":"2024-06-10T02:52:42.999574+0000","last_became_active":"2024-06-10T02:50:02.206650+0000","last_became_peered":"2024-06-10T02:50:02.206650+0000","last_unstale":"2024-06-10T02:52:42.999574+0000","last_undegraded":"2024-06-10T02:52:42.999574+0000","last_fullsized":"2024-06-10T02:52:42.999574+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T14:27:49.555350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"100'7","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.979413+0000","last_change":"2024-06-10T02:50:02.200816+0000","last_active":"2024-06-10T02:52:42.979413+0000","last_peered":"2024-06-10T02:52:42.979413+0000","last_clean":"2024-06-10T02:52:42.979413+0000","last_became_active":"2024-06-10T02:50:02.200503+0000","last_became_peered":"2024-06-10T02:50:02.200503+0000","last_unstale":"2024-06-10T02:52:42.979413+0000","last_undegraded":"2024-06-10T02:52:42.979413+0000","last_fullsized":"2024-06-10T02:52:42.979413+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:01:08.326728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"100'10","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.987233+0000","last_change":"2024-06-10T02:50:02.203644+0000","last_active":"2024-06-10T02:52:42.987233+0000","last_peered":"2024-06-10T02:52:42.987233+0000","last_clean":"2024-06-10T02:52:42.987233+0000","last_became_active":"2024-06-10T02:50:02.203100+0000","last_became_peered":"2024-06-10T02:50:02.203100+0000","last_unstale":"2024-06-10T02:52:42.987233+0000","last_undegraded":"2024-06-10T02:52:42.987233+0000","last_fullsized":"2024-06-10T02:52:42.987233+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:01:42.730089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"100'7","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.995068+0000","last_change":"2024-06-10T02:50:02.203330+0000","last_active":"2024-06-10T02:52:42.995068+0000","last_peered":"2024-06-10T02:52:42.995068+0000","last_clean":"2024-06-10T02:52:42.995068+0000","last_became_active":"2024-06-10T02:50:02.202995+0000","last_became_peered":"2024-06-10T02:50:02.202995+0000","last_unstale":"2024-06-10T02:52:42.995068+0000","last_undegraded":"2024-06-10T02:52:42.995068+0000","last_fullsized":"2024-06-10T02:52:42.995068+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:17:23.170591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"100'7","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.986439+0000","last_change":"2024-06-10T02:50:02.204875+0000","last_active":"2024-06-10T02:52:42.986439+0000","last_peered":"2024-06-10T02:52:42.986439+0000","last_clean":"2024-06-10T02:52:42.986439+0000","last_became_active":"2024-06-10T02:50:02.200706+0000","last_became_peered":"2024-06-10T02:50:02.200706+0000","last_unstale":"2024-06-10T02:52:42.986439+0000","last_undegraded":"2024-06-10T02:52:42.986439+0000","last_fullsized":"2024-06-10T02:52:42.986439+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:27:12.282772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"100'9","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.999375+0000","last_change":"2024-06-10T02:50:02.203187+0000","last_active":"2024-06-10T02:52:42.999375+0000","last_peered":"2024-06-10T02:52:42.999375+0000","last_clean":"2024-06-10T02:52:42.999375+0000","last_became_active":"2024-06-10T02:50:02.202034+0000","last_became_peered":"2024-06-10T02:50:02.202034+0000","last_unstale":"2024-06-10T02:52:42.999375+0000","last_undegraded":"2024-06-10T02:52:42.999375+0000","last_fullsized":"2024-06-10T02:52:42.999375+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:59:44.783084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"100'8","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:43.002372+0000","last_change":"2024-06-10T02:50:02.209102+0000","last_active":"2024-06-10T02:52:43.002372+0000","last_peered":"2024-06-10T02:52:43.002372+0000","last_clean":"2024-06-10T02:52:43.002372+0000","last_became_active":"2024-06-10T02:50:02.208730+0000","last_became_peered":"2024-06-10T02:50:02.208730+0000","last_unstale":"2024-06-10T02:52:43.002372+0000","last_undegraded":"2024-06-10T02:52:43.002372+0000","last_fullsized":"2024-06-10T02:52:43.002372+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:08:38.838415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"100'10","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:43.000459+0000","last_change":"2024-06-10T02:50:02.204008+0000","last_active":"2024-06-10T02:52:43.000459+0000","last_peered":"2024-06-10T02:52:43.000459+0000","last_clean":"2024-06-10T02:52:43.000459+0000","last_became_active":"2024-06-10T02:50:02.203655+0000","last_became_peered":"2024-06-10T02:50:02.203655+0000","last_unstale":"2024-06-10T02:52:43.000459+0000","last_undegraded":"2024-06-10T02:52:43.000459+0000","last_fullsized":"2024-06-10T02:52:43.000459+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:32:15.972662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"100'7","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.991723+0000","last_change":"2024-06-10T02:50:02.204917+0000","last_active":"2024-06-10T02:52:42.991723+0000","last_peered":"2024-06-10T02:52:42.991723+0000","last_clean":"2024-06-10T02:52:42.991723+0000","last_became_active":"2024-06-10T02:50:02.204568+0000","last_became_peered":"2024-06-10T02:50:02.204568+0000","last_unstale":"2024-06-10T02:52:42.991723+0000","last_undegraded":"2024-06-10T02:52:42.991723+0000","last_fullsized":"2024-06-10T02:52:42.991723+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:49:46.802665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.721297+0000","last_change":"2024-06-10T02:50:12.240697+0000","last_active":"2024-06-10T02:51:45.721297+0000","last_peered":"2024-06-10T02:51:45.721297+0000","last_clean":"2024-06-10T02:51:45.721297+0000","last_became_active":"2024-06-10T02:50:12.240229+0000","last_became_peered":"2024-06-10T02:50:12.240229+0000","last_unstale":"2024-06-10T02:51:45.721297+0000","last_undegraded":"2024-06-10T02:51:45.721297+0000","last_fullsized":"2024-06-10T02:51:45.721297+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:45:55.071960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"100'15","reported_seq":255,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:54:09.893400+0000","last_change":"2024-06-10T02:50:44.786783+0000","last_active":"2024-06-10T02:54:09.893400+0000","last_peered":"2024-06-10T02:54:09.893400+0000","last_clean":"2024-06-10T02:54:09.893400+0000","last_became_active":"2024-06-10T02:50:44.781773+0000","last_became_peered":"2024-06-10T02:50:44.781773+0000","last_unstale":"2024-06-10T02:54:09.893400+0000","last_undegraded":"2024-06-10T02:54:09.893400+0000","last_fullsized":"2024-06-10T02:54:09.893400+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:31:06.025854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":182,"num_read_kb":182,"num_write":16,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,4],"acting":[9,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"100'5","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.994824+0000","last_change":"2024-06-10T02:50:02.202573+0000","last_active":"2024-06-10T02:52:42.994824+0000","last_peered":"2024-06-10T02:52:42.994824+0000","last_clean":"2024-06-10T02:52:42.994824+0000","last_became_active":"2024-06-10T02:50:02.202241+0000","last_became_peered":"2024-06-10T02:50:02.202241+0000","last_unstale":"2024-06-10T02:52:42.994824+0000","last_undegraded":"2024-06-10T02:52:42.994824+0000","last_fullsized":"2024-06-10T02:52:42.994824+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:31:40.230851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.383047+0000","last_change":"2024-06-10T02:50:12.238372+0000","last_active":"2024-06-10T02:51:46.383047+0000","last_peered":"2024-06-10T02:51:46.383047+0000","last_clean":"2024-06-10T02:51:46.383047+0000","last_became_active":"2024-06-10T02:50:12.237884+0000","last_became_peered":"2024-06-10T02:50:12.237884+0000","last_unstale":"2024-06-10T02:51:46.383047+0000","last_undegraded":"2024-06-10T02:51:46.383047+0000","last_fullsized":"2024-06-10T02:51:46.383047+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:55:31.432655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"98'2","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:02.117661+0000","last_change":"2024-06-10T02:49:59.031528+0000","last_active":"2024-06-10T02:52:02.117661+0000","last_peered":"2024-06-10T02:52:02.117661+0000","last_clean":"2024-06-10T02:52:02.117661+0000","last_became_active":"2024-06-10T02:49:59.031147+0000","last_became_peered":"2024-06-10T02:49:59.031147+0000","last_unstale":"2024-06-10T02:52:02.117661+0000","last_undegraded":"2024-06-10T02:52:02.117661+0000","last_fullsized":"2024-06-10T02:52:02.117661+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:17:08.534396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"100'6","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:43.000760+0000","last_change":"2024-06-10T02:50:02.202858+0000","last_active":"2024-06-10T02:52:43.000760+0000","last_peered":"2024-06-10T02:52:43.000760+0000","last_clean":"2024-06-10T02:52:43.000760+0000","last_became_active":"2024-06-10T02:50:02.202263+0000","last_became_peered":"2024-06-10T02:50:02.202263+0000","last_unstale":"2024-06-10T02:52:43.000760+0000","last_undegraded":"2024-06-10T02:52:43.000760+0000","last_fullsized":"2024-06-10T02:52:43.000760+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:23:15.382762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.383227+0000","last_change":"2024-06-10T02:50:12.237370+0000","last_active":"2024-06-10T02:51:46.383227+0000","last_peered":"2024-06-10T02:51:46.383227+0000","last_clean":"2024-06-10T02:51:46.383227+0000","last_became_active":"2024-06-10T02:50:12.236803+0000","last_became_peered":"2024-06-10T02:50:12.236803+0000","last_unstale":"2024-06-10T02:51:46.383227+0000","last_undegraded":"2024-06-10T02:51:46.383227+0000","last_fullsized":"2024-06-10T02:51:46.383227+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:06:40.695710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"98'12","reported_seq":58,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.724307+0000","last_change":"2024-06-10T02:49:59.000911+0000","last_active":"2024-06-10T02:51:45.724307+0000","last_peered":"2024-06-10T02:51:45.724307+0000","last_clean":"2024-06-10T02:51:45.724307+0000","last_became_active":"2024-06-10T02:49:59.000571+0000","last_became_peered":"2024-06-10T02:49:59.000571+0000","last_unstale":"2024-06-10T02:51:45.724307+0000","last_undegraded":"2024-06-10T02:51:45.724307+0000","last_fullsized":"2024-06-10T02:51:45.724307+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:49:24.635793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"100'9","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.997165+0000","last_change":"2024-06-10T02:50:02.209614+0000","last_active":"2024-06-10T02:52:42.997165+0000","last_peered":"2024-06-10T02:52:42.997165+0000","last_clean":"2024-06-10T02:52:42.997165+0000","last_became_active":"2024-06-10T02:50:02.209251+0000","last_became_peered":"2024-06-10T02:50:02.209251+0000","last_unstale":"2024-06-10T02:52:42.997165+0000","last_undegraded":"2024-06-10T02:52:42.997165+0000","last_fullsized":"2024-06-10T02:52:42.997165+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:35:32.256081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.383229+0000","last_change":"2024-06-10T02:50:12.237811+0000","last_active":"2024-06-10T02:51:46.383229+0000","last_peered":"2024-06-10T02:51:46.383229+0000","last_clean":"2024-06-10T02:51:46.383229+0000","last_became_active":"2024-06-10T02:50:12.237017+0000","last_became_peered":"2024-06-10T02:50:12.237017+0000","last_unstale":"2024-06-10T02:51:46.383229+0000","last_undegraded":"2024-06-10T02:51:46.383229+0000","last_fullsized":"2024-06-10T02:51:46.383229+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:14:29.818885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"98'5","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.723967+0000","last_change":"2024-06-10T02:49:58.996971+0000","last_active":"2024-06-10T02:51:45.723967+0000","last_peered":"2024-06-10T02:51:45.723967+0000","last_clean":"2024-06-10T02:51:45.723967+0000","last_became_active":"2024-06-10T02:49:58.996601+0000","last_became_peered":"2024-06-10T02:49:58.996601+0000","last_unstale":"2024-06-10T02:51:45.723967+0000","last_undegraded":"2024-06-10T02:51:45.723967+0000","last_fullsized":"2024-06-10T02:51:45.723967+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:26:45.963351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"100'15","reported_seq":60,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:43.002596+0000","last_change":"2024-06-10T02:50:44.745816+0000","last_active":"2024-06-10T02:52:43.002596+0000","last_peered":"2024-06-10T02:52:43.002596+0000","last_clean":"2024-06-10T02:52:43.002596+0000","last_became_active":"2024-06-10T02:50:44.745386+0000","last_became_peered":"2024-06-10T02:50:44.745386+0000","last_unstale":"2024-06-10T02:52:43.002596+0000","last_undegraded":"2024-06-10T02:52:43.002596+0000","last_fullsized":"2024-06-10T02:52:43.002596+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:08:53.977133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.721700+0000","last_change":"2024-06-10T02:50:12.236835+0000","last_active":"2024-06-10T02:51:45.721700+0000","last_peered":"2024-06-10T02:51:45.721700+0000","last_clean":"2024-06-10T02:51:45.721700+0000","last_became_active":"2024-06-10T02:50:12.236309+0000","last_became_peered":"2024-06-10T02:50:12.236309+0000","last_unstale":"2024-06-10T02:51:45.721700+0000","last_undegraded":"2024-06-10T02:51:45.721700+0000","last_fullsized":"2024-06-10T02:51:45.721700+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:09:02.789286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"100'8","reported_seq":234,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:54:05.888742+0000","last_change":"2024-06-10T02:49:58.997725+0000","last_active":"2024-06-10T02:54:05.888742+0000","last_peered":"2024-06-10T02:54:05.888742+0000","last_clean":"2024-06-10T02:54:05.888742+0000","last_became_active":"2024-06-10T02:49:58.997375+0000","last_became_peered":"2024-06-10T02:49:58.997375+0000","last_unstale":"2024-06-10T02:54:05.888742+0000","last_undegraded":"2024-06-10T02:54:05.888742+0000","last_fullsized":"2024-06-10T02:54:05.888742+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:28:49.714356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1461,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":181,"num_read_kb":4,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"100'7","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.995883+0000","last_change":"2024-06-10T02:50:02.201247+0000","last_active":"2024-06-10T02:52:42.995883+0000","last_peered":"2024-06-10T02:52:42.995883+0000","last_clean":"2024-06-10T02:52:42.995883+0000","last_became_active":"2024-06-10T02:50:02.200897+0000","last_became_peered":"2024-06-10T02:50:02.200897+0000","last_unstale":"2024-06-10T02:52:42.995883+0000","last_undegraded":"2024-06-10T02:52:42.995883+0000","last_fullsized":"2024-06-10T02:52:42.995883+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:01:58.166644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.721648+0000","last_change":"2024-06-10T02:50:12.243097+0000","last_active":"2024-06-10T02:51:45.721648+0000","last_peered":"2024-06-10T02:51:45.721648+0000","last_clean":"2024-06-10T02:51:45.721648+0000","last_became_active":"2024-06-10T02:50:12.242501+0000","last_became_peered":"2024-06-10T02:50:12.242501+0000","last_unstale":"2024-06-10T02:51:45.721648+0000","last_undegraded":"2024-06-10T02:51:45.721648+0000","last_fullsized":"2024-06-10T02:51:45.721648+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T14:42:44.220340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"98'7","reported_seq":52,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.720539+0000","last_change":"2024-06-10T02:49:58.996956+0000","last_active":"2024-06-10T02:51:45.720539+0000","last_peered":"2024-06-10T02:51:45.720539+0000","last_clean":"2024-06-10T02:51:45.720539+0000","last_became_active":"2024-06-10T02:49:58.996634+0000","last_became_peered":"2024-06-10T02:49:58.996634+0000","last_unstale":"2024-06-10T02:51:45.720539+0000","last_undegraded":"2024-06-10T02:51:45.720539+0000","last_fullsized":"2024-06-10T02:51:45.720539+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:25:02.364472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"100'8","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.995286+0000","last_change":"2024-06-10T02:50:02.205808+0000","last_active":"2024-06-10T02:52:42.995286+0000","last_peered":"2024-06-10T02:52:42.995286+0000","last_clean":"2024-06-10T02:52:42.995286+0000","last_became_active":"2024-06-10T02:50:02.203558+0000","last_became_peered":"2024-06-10T02:50:02.203558+0000","last_unstale":"2024-06-10T02:52:42.995286+0000","last_undegraded":"2024-06-10T02:52:42.995286+0000","last_fullsized":"2024-06-10T02:52:42.995286+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T13:43:23.315375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.365641+0000","last_change":"2024-06-10T02:50:12.242005+0000","last_active":"2024-06-10T02:51:46.365641+0000","last_peered":"2024-06-10T02:51:46.365641+0000","last_clean":"2024-06-10T02:51:46.365641+0000","last_became_active":"2024-06-10T02:50:12.241295+0000","last_became_peered":"2024-06-10T02:50:12.241295+0000","last_unstale":"2024-06-10T02:51:46.365641+0000","last_undegraded":"2024-06-10T02:51:46.365641+0000","last_fullsized":"2024-06-10T02:51:46.365641+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:32:52.702756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"98'3","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.714696+0000","last_change":"2024-06-10T02:49:58.995579+0000","last_active":"2024-06-10T02:51:45.714696+0000","last_peered":"2024-06-10T02:51:45.714696+0000","last_clean":"2024-06-10T02:51:45.714696+0000","last_became_active":"2024-06-10T02:49:58.995208+0000","last_became_peered":"2024-06-10T02:49:58.995208+0000","last_unstale":"2024-06-10T02:51:45.714696+0000","last_undegraded":"2024-06-10T02:51:45.714696+0000","last_fullsized":"2024-06-10T02:51:45.714696+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:26:03.513482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"100'8","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.993655+0000","last_change":"2024-06-10T02:50:02.197090+0000","last_active":"2024-06-10T02:52:42.993655+0000","last_peered":"2024-06-10T02:52:42.993655+0000","last_clean":"2024-06-10T02:52:42.993655+0000","last_became_active":"2024-06-10T02:50:02.196746+0000","last_became_peered":"2024-06-10T02:50:02.196746+0000","last_unstale":"2024-06-10T02:52:42.993655+0000","last_undegraded":"2024-06-10T02:52:42.993655+0000","last_fullsized":"2024-06-10T02:52:42.993655+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:52:09.987978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.012041+0000","last_change":"2024-06-10T02:50:12.234632+0000","last_active":"2024-06-10T02:51:46.012041+0000","last_peered":"2024-06-10T02:51:46.012041+0000","last_clean":"2024-06-10T02:51:46.012041+0000","last_became_active":"2024-06-10T02:50:12.234266+0000","last_became_peered":"2024-06-10T02:50:12.234266+0000","last_unstale":"2024-06-10T02:51:46.012041+0000","last_undegraded":"2024-06-10T02:51:46.012041+0000","last_fullsized":"2024-06-10T02:51:46.012041+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:21:27.348441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"98'3","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.383558+0000","last_change":"2024-06-10T02:49:58.999086+0000","last_active":"2024-06-10T02:51:46.383558+0000","last_peered":"2024-06-10T02:51:46.383558+0000","last_clean":"2024-06-10T02:51:46.383558+0000","last_became_active":"2024-06-10T02:49:58.998767+0000","last_became_peered":"2024-06-10T02:49:58.998767+0000","last_unstale":"2024-06-10T02:51:46.383558+0000","last_undegraded":"2024-06-10T02:51:46.383558+0000","last_fullsized":"2024-06-10T02:51:46.383558+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:28:12.922391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"100'11","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.990813+0000","last_change":"2024-06-10T02:50:02.196641+0000","last_active":"2024-06-10T02:52:42.990813+0000","last_peered":"2024-06-10T02:52:42.990813+0000","last_clean":"2024-06-10T02:52:42.990813+0000","last_became_active":"2024-06-10T02:50:02.196285+0000","last_became_peered":"2024-06-10T02:50:02.196285+0000","last_unstale":"2024-06-10T02:52:42.990813+0000","last_undegraded":"2024-06-10T02:52:42.990813+0000","last_fullsized":"2024-06-10T02:52:42.990813+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T13:55:31.284028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.831864+0000","last_change":"2024-06-10T02:50:12.240988+0000","last_active":"2024-06-10T02:51:45.831864+0000","last_peered":"2024-06-10T02:51:45.831864+0000","last_clean":"2024-06-10T02:51:45.831864+0000","last_became_active":"2024-06-10T02:50:12.240490+0000","last_became_peered":"2024-06-10T02:50:12.240490+0000","last_unstale":"2024-06-10T02:51:45.831864+0000","last_undegraded":"2024-06-10T02:51:45.831864+0000","last_fullsized":"2024-06-10T02:51:45.831864+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:54:36.015221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"100'7","reported_seq":406,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:54:05.888220+0000","last_change":"2024-06-10T02:49:58.989557+0000","last_active":"2024-06-10T02:54:05.888220+0000","last_peered":"2024-06-10T02:54:05.888220+0000","last_clean":"2024-06-10T02:54:05.888220+0000","last_became_active":"2024-06-10T02:49:58.989101+0000","last_became_peered":"2024-06-10T02:49:58.989101+0000","last_unstale":"2024-06-10T02:54:05.888220+0000","last_undegraded":"2024-06-10T02:54:05.888220+0000","last_fullsized":"2024-06-10T02:54:05.888220+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:27:42.182012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":354,"num_read_kb":354,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"100'10","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.998692+0000","last_change":"2024-06-10T02:50:02.208752+0000","last_active":"2024-06-10T02:52:42.998692+0000","last_peered":"2024-06-10T02:52:42.998692+0000","last_clean":"2024-06-10T02:52:42.998692+0000","last_became_active":"2024-06-10T02:50:02.208420+0000","last_became_peered":"2024-06-10T02:50:02.208420+0000","last_unstale":"2024-06-10T02:52:42.998692+0000","last_undegraded":"2024-06-10T02:52:42.998692+0000","last_fullsized":"2024-06-10T02:52:42.998692+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:28:09.412317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.382833+0000","last_change":"2024-06-10T02:50:12.236397+0000","last_active":"2024-06-10T02:51:46.382833+0000","last_peered":"2024-06-10T02:51:46.382833+0000","last_clean":"2024-06-10T02:51:46.382833+0000","last_became_active":"2024-06-10T02:50:12.235972+0000","last_became_peered":"2024-06-10T02:50:12.235972+0000","last_unstale":"2024-06-10T02:51:46.382833+0000","last_undegraded":"2024-06-10T02:51:46.382833+0000","last_fullsized":"2024-06-10T02:51:46.382833+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:38:41.723895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"98'4","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.295527+0000","last_change":"2024-06-10T02:49:58.996616+0000","last_active":"2024-06-10T02:51:46.295527+0000","last_peered":"2024-06-10T02:51:46.295527+0000","last_clean":"2024-06-10T02:51:46.295527+0000","last_became_active":"2024-06-10T02:49:58.996289+0000","last_became_peered":"2024-06-10T02:49:58.996289+0000","last_unstale":"2024-06-10T02:51:46.295527+0000","last_undegraded":"2024-06-10T02:51:46.295527+0000","last_fullsized":"2024-06-10T02:51:46.295527+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:22:12.993608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"100'9","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.983255+0000","last_change":"2024-06-10T02:50:02.205230+0000","last_active":"2024-06-10T02:52:42.983255+0000","last_peered":"2024-06-10T02:52:42.983255+0000","last_clean":"2024-06-10T02:52:42.983255+0000","last_became_active":"2024-06-10T02:50:02.204835+0000","last_became_peered":"2024-06-10T02:50:02.204835+0000","last_unstale":"2024-06-10T02:52:42.983255+0000","last_undegraded":"2024-06-10T02:52:42.983255+0000","last_fullsized":"2024-06-10T02:52:42.983255+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:17:35.084921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.383415+0000","last_change":"2024-06-10T02:50:12.231684+0000","last_active":"2024-06-10T02:51:46.383415+0000","last_peered":"2024-06-10T02:51:46.383415+0000","last_clean":"2024-06-10T02:51:46.383415+0000","last_became_active":"2024-06-10T02:50:12.231256+0000","last_became_peered":"2024-06-10T02:50:12.231256+0000","last_unstale":"2024-06-10T02:51:46.383415+0000","last_undegraded":"2024-06-10T02:51:46.383415+0000","last_fullsized":"2024-06-10T02:51:46.383415+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:17:31.878289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"98'5","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.383170+0000","last_change":"2024-06-10T02:49:59.033950+0000","last_active":"2024-06-10T02:51:46.383170+0000","last_peered":"2024-06-10T02:51:46.383170+0000","last_clean":"2024-06-10T02:51:46.383170+0000","last_became_active":"2024-06-10T02:49:59.033688+0000","last_became_peered":"2024-06-10T02:49:59.033688+0000","last_unstale":"2024-06-10T02:51:46.383170+0000","last_undegraded":"2024-06-10T02:51:46.383170+0000","last_fullsized":"2024-06-10T02:51:46.383170+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:32:03.892091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"100'11","reported_seq":56,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.999378+0000","last_change":"2024-06-10T02:50:44.781378+0000","last_active":"2024-06-10T02:52:42.999378+0000","last_peered":"2024-06-10T02:52:42.999378+0000","last_clean":"2024-06-10T02:52:42.999378+0000","last_became_active":"2024-06-10T02:50:44.780955+0000","last_became_peered":"2024-06-10T02:50:44.780955+0000","last_unstale":"2024-06-10T02:52:42.999378+0000","last_undegraded":"2024-06-10T02:52:42.999378+0000","last_fullsized":"2024-06-10T02:52:42.999378+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:49:13.760381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.827482+0000","last_change":"2024-06-10T02:51:46.384748+0000","last_active":"2024-06-10T02:51:46.827482+0000","last_peered":"2024-06-10T02:51:46.827482+0000","last_clean":"2024-06-10T02:51:46.827482+0000","last_became_active":"2024-06-10T02:51:46.384402+0000","last_became_peered":"2024-06-10T02:51:46.384402+0000","last_unstale":"2024-06-10T02:51:46.827482+0000","last_undegraded":"2024-06-10T02:51:46.827482+0000","last_fullsized":"2024-06-10T02:51:46.827482+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:06:48.056342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.14","version":"100'39","reported_seq":448,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:54:06.325760+0000","last_change":"2024-06-10T02:49:58.997890+0000","last_active":"2024-06-10T02:54:06.325760+0000","last_peered":"2024-06-10T02:54:06.325760+0000","last_clean":"2024-06-10T02:54:06.325760+0000","last_became_active":"2024-06-10T02:49:58.997549+0000","last_became_peered":"2024-06-10T02:49:58.997549+0000","last_unstale":"2024-06-10T02:54:06.325760+0000","last_undegraded":"2024-06-10T02:54:06.325760+0000","last_fullsized":"2024-06-10T02:54:06.325760+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T14:40:50.907029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":63212,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":364,"num_read_kb":244,"num_write":41,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"100'6","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.991053+0000","last_change":"2024-06-10T02:50:02.199707+0000","last_active":"2024-06-10T02:52:42.991053+0000","last_peered":"2024-06-10T02:52:42.991053+0000","last_clean":"2024-06-10T02:52:42.991053+0000","last_became_active":"2024-06-10T02:50:02.198987+0000","last_became_peered":"2024-06-10T02:50:02.198987+0000","last_unstale":"2024-06-10T02:52:42.991053+0000","last_undegraded":"2024-06-10T02:52:42.991053+0000","last_fullsized":"2024-06-10T02:52:42.991053+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:48:03.704590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.383847+0000","last_change":"2024-06-10T02:50:12.239608+0000","last_active":"2024-06-10T02:51:46.383847+0000","last_peered":"2024-06-10T02:51:46.383847+0000","last_clean":"2024-06-10T02:51:46.383847+0000","last_became_active":"2024-06-10T02:50:12.238588+0000","last_became_peered":"2024-06-10T02:50:12.238588+0000","last_unstale":"2024-06-10T02:51:46.383847+0000","last_undegraded":"2024-06-10T02:51:46.383847+0000","last_fullsized":"2024-06-10T02:51:46.383847+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:26:12.718921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"98'9","reported_seq":56,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.721583+0000","last_change":"2024-06-10T02:49:58.997193+0000","last_active":"2024-06-10T02:51:45.721583+0000","last_peered":"2024-06-10T02:51:45.721583+0000","last_clean":"2024-06-10T02:51:45.721583+0000","last_became_active":"2024-06-10T02:49:58.996712+0000","last_became_peered":"2024-06-10T02:49:58.996712+0000","last_unstale":"2024-06-10T02:51:45.721583+0000","last_undegraded":"2024-06-10T02:51:45.721583+0000","last_fullsized":"2024-06-10T02:51:45.721583+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:02:46.894489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"100'9","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.994408+0000","last_change":"2024-06-10T02:50:02.200544+0000","last_active":"2024-06-10T02:52:42.994408+0000","last_peered":"2024-06-10T02:52:42.994408+0000","last_clean":"2024-06-10T02:52:42.994408+0000","last_became_active":"2024-06-10T02:50:02.200201+0000","last_became_peered":"2024-06-10T02:50:02.200201+0000","last_unstale":"2024-06-10T02:52:42.994408+0000","last_undegraded":"2024-06-10T02:52:42.994408+0000","last_fullsized":"2024-06-10T02:52:42.994408+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T13:12:40.925448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.365243+0000","last_change":"2024-06-10T02:50:12.236317+0000","last_active":"2024-06-10T02:51:46.365243+0000","last_peered":"2024-06-10T02:51:46.365243+0000","last_clean":"2024-06-10T02:51:46.365243+0000","last_became_active":"2024-06-10T02:50:12.235806+0000","last_became_peered":"2024-06-10T02:50:12.235806+0000","last_unstale":"2024-06-10T02:51:46.365243+0000","last_undegraded":"2024-06-10T02:51:46.365243+0000","last_fullsized":"2024-06-10T02:51:46.365243+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:29:21.425987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"98'5","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.383944+0000","last_change":"2024-06-10T02:49:58.994149+0000","last_active":"2024-06-10T02:51:46.383944+0000","last_peered":"2024-06-10T02:51:46.383944+0000","last_clean":"2024-06-10T02:51:46.383944+0000","last_became_active":"2024-06-10T02:49:58.993775+0000","last_became_peered":"2024-06-10T02:49:58.993775+0000","last_unstale":"2024-06-10T02:51:46.383944+0000","last_undegraded":"2024-06-10T02:51:46.383944+0000","last_fullsized":"2024-06-10T02:51:46.383944+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:23:55.553943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"100'15","reported_seq":54,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.995273+0000","last_change":"2024-06-10T02:50:02.199979+0000","last_active":"2024-06-10T02:52:42.995273+0000","last_peered":"2024-06-10T02:52:42.995273+0000","last_clean":"2024-06-10T02:52:42.995273+0000","last_became_active":"2024-06-10T02:50:02.199596+0000","last_became_peered":"2024-06-10T02:50:02.199596+0000","last_unstale":"2024-06-10T02:52:42.995273+0000","last_undegraded":"2024-06-10T02:52:42.995273+0000","last_fullsized":"2024-06-10T02:52:42.995273+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:13:58.647460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.383103+0000","last_change":"2024-06-10T02:50:12.235253+0000","last_active":"2024-06-10T02:51:46.383103+0000","last_peered":"2024-06-10T02:51:46.383103+0000","last_clean":"2024-06-10T02:51:46.383103+0000","last_became_active":"2024-06-10T02:50:12.234773+0000","last_became_peered":"2024-06-10T02:50:12.234773+0000","last_unstale":"2024-06-10T02:51:46.383103+0000","last_undegraded":"2024-06-10T02:51:46.383103+0000","last_fullsized":"2024-06-10T02:51:46.383103+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:38:10.971411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"100'7","reported_seq":238,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:54:09.893481+0000","last_change":"2024-06-10T02:49:58.996912+0000","last_active":"2024-06-10T02:54:09.893481+0000","last_peered":"2024-06-10T02:54:09.893481+0000","last_clean":"2024-06-10T02:54:09.893481+0000","last_became_active":"2024-06-10T02:49:58.996549+0000","last_became_peered":"2024-06-10T02:49:58.996549+0000","last_unstale":"2024-06-10T02:54:09.893481+0000","last_undegraded":"2024-06-10T02:54:09.893481+0000","last_fullsized":"2024-06-10T02:54:09.893481+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:07:52.757676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1119,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":187,"num_read_kb":5,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"100'6","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.995546+0000","last_change":"2024-06-10T02:50:44.741186+0000","last_active":"2024-06-10T02:52:42.995546+0000","last_peered":"2024-06-10T02:52:42.995546+0000","last_clean":"2024-06-10T02:52:42.995546+0000","last_became_active":"2024-06-10T02:50:44.740782+0000","last_became_peered":"2024-06-10T02:50:44.740782+0000","last_unstale":"2024-06-10T02:52:42.995546+0000","last_undegraded":"2024-06-10T02:52:42.995546+0000","last_fullsized":"2024-06-10T02:52:42.995546+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:46:03.863243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.715228+0000","last_change":"2024-06-10T02:50:12.236505+0000","last_active":"2024-06-10T02:51:45.715228+0000","last_peered":"2024-06-10T02:51:45.715228+0000","last_clean":"2024-06-10T02:51:45.715228+0000","last_became_active":"2024-06-10T02:50:12.236112+0000","last_became_peered":"2024-06-10T02:50:12.236112+0000","last_unstale":"2024-06-10T02:51:45.715228+0000","last_undegraded":"2024-06-10T02:51:45.715228+0000","last_fullsized":"2024-06-10T02:51:45.715228+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:13:38.586736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"98'17","reported_seq":63,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.723449+0000","last_change":"2024-06-10T02:49:59.031965+0000","last_active":"2024-06-10T02:51:45.723449+0000","last_peered":"2024-06-10T02:51:45.723449+0000","last_clean":"2024-06-10T02:51:45.723449+0000","last_became_active":"2024-06-10T02:49:59.031495+0000","last_became_peered":"2024-06-10T02:49:59.031495+0000","last_unstale":"2024-06-10T02:51:45.723449+0000","last_undegraded":"2024-06-10T02:51:45.723449+0000","last_fullsized":"2024-06-10T02:51:45.723449+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T14:21:06.610677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"100'6","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:43.002009+0000","last_change":"2024-06-10T02:50:02.209926+0000","last_active":"2024-06-10T02:52:43.002009+0000","last_peered":"2024-06-10T02:52:43.002009+0000","last_clean":"2024-06-10T02:52:43.002009+0000","last_became_active":"2024-06-10T02:50:02.209540+0000","last_became_peered":"2024-06-10T02:50:02.209540+0000","last_unstale":"2024-06-10T02:52:43.002009+0000","last_undegraded":"2024-06-10T02:52:43.002009+0000","last_fullsized":"2024-06-10T02:52:43.002009+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:49:40.497692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:48.927934+0000","last_change":"2024-06-10T02:51:46.013156+0000","last_active":"2024-06-10T02:51:48.927934+0000","last_peered":"2024-06-10T02:51:48.927934+0000","last_clean":"2024-06-10T02:51:48.927934+0000","last_became_active":"2024-06-10T02:51:46.012769+0000","last_became_peered":"2024-06-10T02:51:46.012769+0000","last_unstale":"2024-06-10T02:51:48.927934+0000","last_undegraded":"2024-06-10T02:51:48.927934+0000","last_fullsized":"2024-06-10T02:51:48.927934+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:44:49.545676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,3,1],"acting":[8,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.f","version":"98'2","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.724214+0000","last_change":"2024-06-10T02:49:58.998372+0000","last_active":"2024-06-10T02:51:45.724214+0000","last_peered":"2024-06-10T02:51:45.724214+0000","last_clean":"2024-06-10T02:51:45.724214+0000","last_became_active":"2024-06-10T02:49:58.998058+0000","last_became_peered":"2024-06-10T02:49:58.998058+0000","last_unstale":"2024-06-10T02:51:45.724214+0000","last_undegraded":"2024-06-10T02:51:45.724214+0000","last_fullsized":"2024-06-10T02:51:45.724214+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:42:50.509851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"100'5","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.993794+0000","last_change":"2024-06-10T02:50:02.209256+0000","last_active":"2024-06-10T02:52:42.993794+0000","last_peered":"2024-06-10T02:52:42.993794+0000","last_clean":"2024-06-10T02:52:42.993794+0000","last_became_active":"2024-06-10T02:50:02.207727+0000","last_became_peered":"2024-06-10T02:50:02.207727+0000","last_unstale":"2024-06-10T02:52:42.993794+0000","last_undegraded":"2024-06-10T02:52:42.993794+0000","last_fullsized":"2024-06-10T02:52:42.993794+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:23:00.175943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.723666+0000","last_change":"2024-06-10T02:50:12.729438+0000","last_active":"2024-06-10T02:51:45.723666+0000","last_peered":"2024-06-10T02:51:45.723666+0000","last_clean":"2024-06-10T02:51:45.723666+0000","last_became_active":"2024-06-10T02:50:12.728918+0000","last_became_peered":"2024-06-10T02:50:12.728918+0000","last_unstale":"2024-06-10T02:51:45.723666+0000","last_undegraded":"2024-06-10T02:51:45.723666+0000","last_fullsized":"2024-06-10T02:51:45.723666+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:04:48.861763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"98'1","reported_seq":53,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:05.628603+0000","last_change":"2024-06-10T02:50:44.782241+0000","last_active":"2024-06-10T02:52:05.628603+0000","last_peered":"2024-06-10T02:52:05.628603+0000","last_clean":"2024-06-10T02:52:05.628603+0000","last_became_active":"2024-06-10T02:50:44.781799+0000","last_became_peered":"2024-06-10T02:50:44.781799+0000","last_unstale":"2024-06-10T02:52:05.628603+0000","last_undegraded":"2024-06-10T02:52:05.628603+0000","last_fullsized":"2024-06-10T02:52:05.628603+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:38:35.726258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"100'4","reported_seq":43,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.992975+0000","last_change":"2024-06-10T02:50:02.198888+0000","last_active":"2024-06-10T02:52:42.992975+0000","last_peered":"2024-06-10T02:52:42.992975+0000","last_clean":"2024-06-10T02:52:42.992975+0000","last_became_active":"2024-06-10T02:50:02.198346+0000","last_became_peered":"2024-06-10T02:50:02.198346+0000","last_unstale":"2024-06-10T02:52:42.992975+0000","last_undegraded":"2024-06-10T02:52:42.992975+0000","last_fullsized":"2024-06-10T02:52:42.992975+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:15:42.422898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.715171+0000","last_change":"2024-06-10T02:50:12.232763+0000","last_active":"2024-06-10T02:51:45.715171+0000","last_peered":"2024-06-10T02:51:45.715171+0000","last_clean":"2024-06-10T02:51:45.715171+0000","last_became_active":"2024-06-10T02:50:12.232390+0000","last_became_peered":"2024-06-10T02:50:12.232390+0000","last_unstale":"2024-06-10T02:51:45.715171+0000","last_undegraded":"2024-06-10T02:51:45.715171+0000","last_fullsized":"2024-06-10T02:51:45.715171+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:31:47.973500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"100'7","reported_seq":233,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:54:05.888111+0000","last_change":"2024-06-10T02:49:58.994308+0000","last_active":"2024-06-10T02:54:05.888111+0000","last_peered":"2024-06-10T02:54:05.888111+0000","last_clean":"2024-06-10T02:54:05.888111+0000","last_became_active":"2024-06-10T02:49:58.993494+0000","last_became_peered":"2024-06-10T02:49:58.993494+0000","last_unstale":"2024-06-10T02:54:05.888111+0000","last_undegraded":"2024-06-10T02:54:05.888111+0000","last_fullsized":"2024-06-10T02:54:05.888111+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:07:28.452276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1119,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":181,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"100'7","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.988576+0000","last_change":"2024-06-10T02:50:02.206872+0000","last_active":"2024-06-10T02:52:42.988576+0000","last_peered":"2024-06-10T02:52:42.988576+0000","last_clean":"2024-06-10T02:52:42.988576+0000","last_became_active":"2024-06-10T02:50:02.206307+0000","last_became_peered":"2024-06-10T02:50:02.206307+0000","last_unstale":"2024-06-10T02:52:42.988576+0000","last_undegraded":"2024-06-10T02:52:42.988576+0000","last_fullsized":"2024-06-10T02:52:42.988576+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:01:04.645674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.715107+0000","last_change":"2024-06-10T02:50:12.729392+0000","last_active":"2024-06-10T02:51:45.715107+0000","last_peered":"2024-06-10T02:51:45.715107+0000","last_clean":"2024-06-10T02:51:45.715107+0000","last_became_active":"2024-06-10T02:50:12.728866+0000","last_became_peered":"2024-06-10T02:50:12.728866+0000","last_unstale":"2024-06-10T02:51:45.715107+0000","last_undegraded":"2024-06-10T02:51:45.715107+0000","last_fullsized":"2024-06-10T02:51:45.715107+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:01:35.845184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"100'8","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.995032+0000","last_change":"2024-06-10T02:50:02.202303+0000","last_active":"2024-06-10T02:52:42.995032+0000","last_peered":"2024-06-10T02:52:42.995032+0000","last_clean":"2024-06-10T02:52:42.995032+0000","last_became_active":"2024-06-10T02:50:02.201998+0000","last_became_peered":"2024-06-10T02:50:02.201998+0000","last_unstale":"2024-06-10T02:52:42.995032+0000","last_undegraded":"2024-06-10T02:52:42.995032+0000","last_fullsized":"2024-06-10T02:52:42.995032+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:15:00.005693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"100'6","reported_seq":230,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:54:05.887639+0000","last_change":"2024-06-10T02:49:59.031513+0000","last_active":"2024-06-10T02:54:05.887639+0000","last_peered":"2024-06-10T02:54:05.887639+0000","last_clean":"2024-06-10T02:54:05.887639+0000","last_became_active":"2024-06-10T02:49:59.030965+0000","last_became_peered":"2024-06-10T02:49:59.030965+0000","last_unstale":"2024-06-10T02:54:05.887639+0000","last_undegraded":"2024-06-10T02:54:05.887639+0000","last_fullsized":"2024-06-10T02:54:05.887639+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:49:02.171578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":179,"num_read_kb":179,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"98'1","reported_seq":30,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.295434+0000","last_change":"2024-06-10T02:50:12.236334+0000","last_active":"2024-06-10T02:51:46.295434+0000","last_peered":"2024-06-10T02:51:46.295434+0000","last_clean":"2024-06-10T02:51:46.295434+0000","last_became_active":"2024-06-10T02:50:12.235820+0000","last_became_peered":"2024-06-10T02:50:12.235820+0000","last_unstale":"2024-06-10T02:51:46.295434+0000","last_undegraded":"2024-06-10T02:51:46.295434+0000","last_fullsized":"2024-06-10T02:51:46.295434+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:16:12.476881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"100'8","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:43.001236+0000","last_change":"2024-06-10T02:50:02.205146+0000","last_active":"2024-06-10T02:52:43.001236+0000","last_peered":"2024-06-10T02:52:43.001236+0000","last_clean":"2024-06-10T02:52:43.001236+0000","last_became_active":"2024-06-10T02:50:02.204785+0000","last_became_peered":"2024-06-10T02:50:02.204785+0000","last_unstale":"2024-06-10T02:52:43.001236+0000","last_undegraded":"2024-06-10T02:52:43.001236+0000","last_fullsized":"2024-06-10T02:52:43.001236+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T13:01:02.068989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"98'6","reported_seq":55,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:06.016314+0000","last_change":"2024-06-10T02:49:58.997654+0000","last_active":"2024-06-10T02:52:06.016314+0000","last_peered":"2024-06-10T02:52:06.016314+0000","last_clean":"2024-06-10T02:52:06.016314+0000","last_became_active":"2024-06-10T02:49:58.997294+0000","last_became_peered":"2024-06-10T02:49:58.997294+0000","last_unstale":"2024-06-10T02:52:06.016314+0000","last_undegraded":"2024-06-10T02:52:06.016314+0000","last_fullsized":"2024-06-10T02:52:06.016314+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:53:14.763198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":367,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.294476+0000","last_change":"2024-06-10T02:48:34.008201+0000","last_active":"2024-06-10T02:51:46.294476+0000","last_peered":"2024-06-10T02:51:46.294476+0000","last_clean":"2024-06-10T02:51:46.294476+0000","last_became_active":"2024-06-10T02:48:34.007633+0000","last_became_peered":"2024-06-10T02:48:34.007633+0000","last_unstale":"2024-06-10T02:51:46.294476+0000","last_undegraded":"2024-06-10T02:51:46.294476+0000","last_fullsized":"2024-06-10T02:51:46.294476+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:45:07.444823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:45:07.444823+0000","last_clean_scrub_stamp":"2024-06-10T02:45:07.444823+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:56:58.159559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":36,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.383275+0000","last_change":"2024-06-10T02:50:44.782130+0000","last_active":"2024-06-10T02:51:46.383275+0000","last_peered":"2024-06-10T02:51:46.383275+0000","last_clean":"2024-06-10T02:51:46.383275+0000","last_became_active":"2024-06-10T02:50:44.781530+0000","last_became_peered":"2024-06-10T02:50:44.781530+0000","last_unstale":"2024-06-10T02:51:46.383275+0000","last_undegraded":"2024-06-10T02:51:46.383275+0000","last_fullsized":"2024-06-10T02:51:46.383275+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:43:23.826207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,11,6,10],"acting":[5,11,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"100'5","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.994022+0000","last_change":"2024-06-10T02:50:02.213666+0000","last_active":"2024-06-10T02:52:42.994022+0000","last_peered":"2024-06-10T02:52:42.994022+0000","last_clean":"2024-06-10T02:52:42.994022+0000","last_became_active":"2024-06-10T02:50:02.213194+0000","last_became_peered":"2024-06-10T02:50:02.213194+0000","last_unstale":"2024-06-10T02:52:42.994022+0000","last_undegraded":"2024-06-10T02:52:42.994022+0000","last_fullsized":"2024-06-10T02:52:42.994022+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:22:55.905412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"98'4","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:02.798809+0000","last_change":"2024-06-10T02:49:58.997618+0000","last_active":"2024-06-10T02:52:02.798809+0000","last_peered":"2024-06-10T02:52:02.798809+0000","last_clean":"2024-06-10T02:52:02.798809+0000","last_became_active":"2024-06-10T02:49:58.997304+0000","last_became_peered":"2024-06-10T02:49:58.997304+0000","last_unstale":"2024-06-10T02:52:02.798809+0000","last_undegraded":"2024-06-10T02:52:02.798809+0000","last_fullsized":"2024-06-10T02:52:02.798809+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:48:59.520991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.364962+0000","last_change":"2024-06-10T02:50:12.236357+0000","last_active":"2024-06-10T02:51:46.364962+0000","last_peered":"2024-06-10T02:51:46.364962+0000","last_clean":"2024-06-10T02:51:46.364962+0000","last_became_active":"2024-06-10T02:50:12.235915+0000","last_became_peered":"2024-06-10T02:50:12.235915+0000","last_unstale":"2024-06-10T02:51:46.364962+0000","last_undegraded":"2024-06-10T02:51:46.364962+0000","last_fullsized":"2024-06-10T02:51:46.364962+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:20:06.287907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"98'4","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:03.929566+0000","last_change":"2024-06-10T02:49:59.031725+0000","last_active":"2024-06-10T02:52:03.929566+0000","last_peered":"2024-06-10T02:52:03.929566+0000","last_clean":"2024-06-10T02:52:03.929566+0000","last_became_active":"2024-06-10T02:49:59.031296+0000","last_became_peered":"2024-06-10T02:49:59.031296+0000","last_unstale":"2024-06-10T02:52:03.929566+0000","last_undegraded":"2024-06-10T02:52:03.929566+0000","last_fullsized":"2024-06-10T02:52:03.929566+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T13:17:58.571696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"100'9","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.978128+0000","last_change":"2024-06-10T02:50:02.201820+0000","last_active":"2024-06-10T02:52:42.978128+0000","last_peered":"2024-06-10T02:52:42.978128+0000","last_clean":"2024-06-10T02:52:42.978128+0000","last_became_active":"2024-06-10T02:50:02.201478+0000","last_became_peered":"2024-06-10T02:50:02.201478+0000","last_unstale":"2024-06-10T02:52:42.978128+0000","last_undegraded":"2024-06-10T02:52:42.978128+0000","last_fullsized":"2024-06-10T02:52:42.978128+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:57:25.963926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.831431+0000","last_change":"2024-06-10T02:50:12.729418+0000","last_active":"2024-06-10T02:51:45.831431+0000","last_peered":"2024-06-10T02:51:45.831431+0000","last_clean":"2024-06-10T02:51:45.831431+0000","last_became_active":"2024-06-10T02:50:12.728692+0000","last_became_peered":"2024-06-10T02:50:12.728692+0000","last_unstale":"2024-06-10T02:51:45.831431+0000","last_undegraded":"2024-06-10T02:51:45.831431+0000","last_fullsized":"2024-06-10T02:51:45.831431+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:25:00.472396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"100'18","reported_seq":63,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:54:07.234777+0000","last_change":"2024-06-10T02:49:58.996391+0000","last_active":"2024-06-10T02:54:07.234777+0000","last_peered":"2024-06-10T02:54:07.234777+0000","last_clean":"2024-06-10T02:54:07.234777+0000","last_became_active":"2024-06-10T02:49:58.996033+0000","last_became_peered":"2024-06-10T02:49:58.996033+0000","last_unstale":"2024-06-10T02:54:07.234777+0000","last_undegraded":"2024-06-10T02:54:07.234777+0000","last_fullsized":"2024-06-10T02:54:07.234777+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:03:34.568969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":34,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"100'5","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.993669+0000","last_change":"2024-06-10T02:50:02.197974+0000","last_active":"2024-06-10T02:52:42.993669+0000","last_peered":"2024-06-10T02:52:42.993669+0000","last_clean":"2024-06-10T02:52:42.993669+0000","last_became_active":"2024-06-10T02:50:02.197614+0000","last_became_peered":"2024-06-10T02:50:02.197614+0000","last_unstale":"2024-06-10T02:52:42.993669+0000","last_undegraded":"2024-06-10T02:52:42.993669+0000","last_fullsized":"2024-06-10T02:52:42.993669+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:36:55.575808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.721202+0000","last_change":"2024-06-10T02:50:12.234239+0000","last_active":"2024-06-10T02:51:45.721202+0000","last_peered":"2024-06-10T02:51:45.721202+0000","last_clean":"2024-06-10T02:51:45.721202+0000","last_became_active":"2024-06-10T02:50:12.233768+0000","last_became_peered":"2024-06-10T02:50:12.233768+0000","last_unstale":"2024-06-10T02:51:45.721202+0000","last_undegraded":"2024-06-10T02:51:45.721202+0000","last_fullsized":"2024-06-10T02:51:45.721202+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:51:44.230031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"98'7","reported_seq":53,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.383587+0000","last_change":"2024-06-10T02:49:59.192765+0000","last_active":"2024-06-10T02:51:46.383587+0000","last_peered":"2024-06-10T02:51:46.383587+0000","last_clean":"2024-06-10T02:51:46.383587+0000","last_became_active":"2024-06-10T02:49:59.192446+0000","last_became_peered":"2024-06-10T02:49:59.192446+0000","last_unstale":"2024-06-10T02:51:46.383587+0000","last_undegraded":"2024-06-10T02:51:46.383587+0000","last_fullsized":"2024-06-10T02:51:46.383587+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:53:51.862019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"100'7","reported_seq":52,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.989395+0000","last_change":"2024-06-10T02:50:44.781209+0000","last_active":"2024-06-10T02:52:42.989395+0000","last_peered":"2024-06-10T02:52:42.989395+0000","last_clean":"2024-06-10T02:52:42.989395+0000","last_became_active":"2024-06-10T02:50:44.780811+0000","last_became_peered":"2024-06-10T02:50:44.780811+0000","last_unstale":"2024-06-10T02:52:42.989395+0000","last_undegraded":"2024-06-10T02:52:42.989395+0000","last_fullsized":"2024-06-10T02:52:42.989395+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:12:40.663565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.383517+0000","last_change":"2024-06-10T02:50:12.730269+0000","last_active":"2024-06-10T02:51:46.383517+0000","last_peered":"2024-06-10T02:51:46.383517+0000","last_clean":"2024-06-10T02:51:46.383517+0000","last_became_active":"2024-06-10T02:50:12.729574+0000","last_became_peered":"2024-06-10T02:50:12.729574+0000","last_unstale":"2024-06-10T02:51:46.383517+0000","last_undegraded":"2024-06-10T02:51:46.383517+0000","last_fullsized":"2024-06-10T02:51:46.383517+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:51:16.235284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"98'2","reported_seq":53,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.854226+0000","last_change":"2024-06-10T02:50:44.782421+0000","last_active":"2024-06-10T02:51:45.854226+0000","last_peered":"2024-06-10T02:51:45.854226+0000","last_clean":"2024-06-10T02:51:45.854226+0000","last_became_active":"2024-06-10T02:50:44.781881+0000","last_became_peered":"2024-06-10T02:50:44.781881+0000","last_unstale":"2024-06-10T02:51:45.854226+0000","last_undegraded":"2024-06-10T02:51:45.854226+0000","last_fullsized":"2024-06-10T02:51:45.854226+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T14:41:58.771256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"100'2","reported_seq":41,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.952928+0000","last_change":"2024-06-10T02:50:02.197761+0000","last_active":"2024-06-10T02:52:42.952928+0000","last_peered":"2024-06-10T02:52:42.952928+0000","last_clean":"2024-06-10T02:52:42.952928+0000","last_became_active":"2024-06-10T02:50:02.197395+0000","last_became_peered":"2024-06-10T02:50:02.197395+0000","last_unstale":"2024-06-10T02:52:42.952928+0000","last_undegraded":"2024-06-10T02:52:42.952928+0000","last_fullsized":"2024-06-10T02:52:42.952928+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:46:37.537280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:48.927972+0000","last_change":"2024-06-10T02:51:46.385129+0000","last_active":"2024-06-10T02:51:48.927972+0000","last_peered":"2024-06-10T02:51:48.927972+0000","last_clean":"2024-06-10T02:51:48.927972+0000","last_became_active":"2024-06-10T02:51:46.384553+0000","last_became_peered":"2024-06-10T02:51:46.384553+0000","last_unstale":"2024-06-10T02:51:48.927972+0000","last_undegraded":"2024-06-10T02:51:48.927972+0000","last_fullsized":"2024-06-10T02:51:48.927972+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:08:11.183679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"98'7","reported_seq":52,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.831750+0000","last_change":"2024-06-10T02:49:58.987863+0000","last_active":"2024-06-10T02:51:45.831750+0000","last_peered":"2024-06-10T02:51:45.831750+0000","last_clean":"2024-06-10T02:51:45.831750+0000","last_became_active":"2024-06-10T02:49:58.987388+0000","last_became_peered":"2024-06-10T02:49:58.987388+0000","last_unstale":"2024-06-10T02:51:45.831750+0000","last_undegraded":"2024-06-10T02:51:45.831750+0000","last_fullsized":"2024-06-10T02:51:45.831750+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:29:20.340301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"100'9","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.988655+0000","last_change":"2024-06-10T02:50:02.200699+0000","last_active":"2024-06-10T02:52:42.988655+0000","last_peered":"2024-06-10T02:52:42.988655+0000","last_clean":"2024-06-10T02:52:42.988655+0000","last_became_active":"2024-06-10T02:50:02.200278+0000","last_became_peered":"2024-06-10T02:50:02.200278+0000","last_unstale":"2024-06-10T02:52:42.988655+0000","last_undegraded":"2024-06-10T02:52:42.988655+0000","last_fullsized":"2024-06-10T02:52:42.988655+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:26:36.236312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"98'1","reported_seq":30,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.383543+0000","last_change":"2024-06-10T02:50:12.236027+0000","last_active":"2024-06-10T02:51:46.383543+0000","last_peered":"2024-06-10T02:51:46.383543+0000","last_clean":"2024-06-10T02:51:46.383543+0000","last_became_active":"2024-06-10T02:50:12.235629+0000","last_became_peered":"2024-06-10T02:50:12.235629+0000","last_unstale":"2024-06-10T02:51:46.383543+0000","last_undegraded":"2024-06-10T02:51:46.383543+0000","last_fullsized":"2024-06-10T02:51:46.383543+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:58:48.218806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"98'3","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.721449+0000","last_change":"2024-06-10T02:49:58.998496+0000","last_active":"2024-06-10T02:51:45.721449+0000","last_peered":"2024-06-10T02:51:45.721449+0000","last_clean":"2024-06-10T02:51:45.721449+0000","last_became_active":"2024-06-10T02:49:58.997260+0000","last_became_peered":"2024-06-10T02:49:58.997260+0000","last_unstale":"2024-06-10T02:51:45.721449+0000","last_undegraded":"2024-06-10T02:51:45.721449+0000","last_fullsized":"2024-06-10T02:51:45.721449+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T14:22:04.658943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"100'8","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.990375+0000","last_change":"2024-06-10T02:50:02.201300+0000","last_active":"2024-06-10T02:52:42.990375+0000","last_peered":"2024-06-10T02:52:42.990375+0000","last_clean":"2024-06-10T02:52:42.990375+0000","last_became_active":"2024-06-10T02:50:02.200988+0000","last_became_peered":"2024-06-10T02:50:02.200988+0000","last_unstale":"2024-06-10T02:52:42.990375+0000","last_undegraded":"2024-06-10T02:52:42.990375+0000","last_fullsized":"2024-06-10T02:52:42.990375+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:11:12.962048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.383484+0000","last_change":"2024-06-10T02:50:12.728869+0000","last_active":"2024-06-10T02:51:46.383484+0000","last_peered":"2024-06-10T02:51:46.383484+0000","last_clean":"2024-06-10T02:51:46.383484+0000","last_became_active":"2024-06-10T02:50:12.728189+0000","last_became_peered":"2024-06-10T02:50:12.728189+0000","last_unstale":"2024-06-10T02:51:46.383484+0000","last_undegraded":"2024-06-10T02:51:46.383484+0000","last_fullsized":"2024-06-10T02:51:46.383484+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:08:36.624814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"98'4","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.723488+0000","last_change":"2024-06-10T02:49:59.000460+0000","last_active":"2024-06-10T02:51:45.723488+0000","last_peered":"2024-06-10T02:51:45.723488+0000","last_clean":"2024-06-10T02:51:45.723488+0000","last_became_active":"2024-06-10T02:49:59.000075+0000","last_became_peered":"2024-06-10T02:49:59.000075+0000","last_unstale":"2024-06-10T02:51:45.723488+0000","last_undegraded":"2024-06-10T02:51:45.723488+0000","last_fullsized":"2024-06-10T02:51:45.723488+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:06:27.935745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"100'5","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:43.001075+0000","last_change":"2024-06-10T02:50:02.210131+0000","last_active":"2024-06-10T02:52:43.001075+0000","last_peered":"2024-06-10T02:52:43.001075+0000","last_clean":"2024-06-10T02:52:43.001075+0000","last_became_active":"2024-06-10T02:50:02.209781+0000","last_became_peered":"2024-06-10T02:50:02.209781+0000","last_unstale":"2024-06-10T02:52:43.001075+0000","last_undegraded":"2024-06-10T02:52:43.001075+0000","last_fullsized":"2024-06-10T02:52:43.001075+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:27:39.966620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.854407+0000","last_change":"2024-06-10T02:50:12.236017+0000","last_active":"2024-06-10T02:51:45.854407+0000","last_peered":"2024-06-10T02:51:45.854407+0000","last_clean":"2024-06-10T02:51:45.854407+0000","last_became_active":"2024-06-10T02:50:12.235553+0000","last_became_peered":"2024-06-10T02:50:12.235553+0000","last_unstale":"2024-06-10T02:51:45.854407+0000","last_undegraded":"2024-06-10T02:51:45.854407+0000","last_fullsized":"2024-06-10T02:51:45.854407+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:23:55.437994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"100'7","reported_seq":233,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:54:04.887243+0000","last_change":"2024-06-10T02:49:59.192864+0000","last_active":"2024-06-10T02:54:04.887243+0000","last_peered":"2024-06-10T02:54:04.887243+0000","last_clean":"2024-06-10T02:54:04.887243+0000","last_became_active":"2024-06-10T02:49:59.192440+0000","last_became_peered":"2024-06-10T02:49:59.192440+0000","last_unstale":"2024-06-10T02:54:04.887243+0000","last_undegraded":"2024-06-10T02:54:04.887243+0000","last_fullsized":"2024-06-10T02:54:04.887243+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:36:05.868538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1701,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":181,"num_read_kb":4,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"100'7","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:43.000525+0000","last_change":"2024-06-10T02:50:02.205791+0000","last_active":"2024-06-10T02:52:43.000525+0000","last_peered":"2024-06-10T02:52:43.000525+0000","last_clean":"2024-06-10T02:52:43.000525+0000","last_became_active":"2024-06-10T02:50:02.205483+0000","last_became_peered":"2024-06-10T02:50:02.205483+0000","last_unstale":"2024-06-10T02:52:43.000525+0000","last_undegraded":"2024-06-10T02:52:43.000525+0000","last_fullsized":"2024-06-10T02:52:43.000525+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:33:25.875423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.011597+0000","last_change":"2024-06-10T02:50:12.239294+0000","last_active":"2024-06-10T02:51:46.011597+0000","last_peered":"2024-06-10T02:51:46.011597+0000","last_clean":"2024-06-10T02:51:46.011597+0000","last_became_active":"2024-06-10T02:50:12.238778+0000","last_became_peered":"2024-06-10T02:50:12.238778+0000","last_unstale":"2024-06-10T02:51:46.011597+0000","last_undegraded":"2024-06-10T02:51:46.011597+0000","last_fullsized":"2024-06-10T02:51:46.011597+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:26:09.203623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"98'3","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.383589+0000","last_change":"2024-06-10T02:49:58.989680+0000","last_active":"2024-06-10T02:51:46.383589+0000","last_peered":"2024-06-10T02:51:46.383589+0000","last_clean":"2024-06-10T02:51:46.383589+0000","last_became_active":"2024-06-10T02:49:58.989265+0000","last_became_peered":"2024-06-10T02:49:58.989265+0000","last_unstale":"2024-06-10T02:51:46.383589+0000","last_undegraded":"2024-06-10T02:51:46.383589+0000","last_fullsized":"2024-06-10T02:51:46.383589+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:36:34.370177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"100'6","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.990330+0000","last_change":"2024-06-10T02:50:44.743369+0000","last_active":"2024-06-10T02:52:42.990330+0000","last_peered":"2024-06-10T02:52:42.990330+0000","last_clean":"2024-06-10T02:52:42.990330+0000","last_became_active":"2024-06-10T02:50:44.743013+0000","last_became_peered":"2024-06-10T02:50:44.743013+0000","last_unstale":"2024-06-10T02:52:42.990330+0000","last_undegraded":"2024-06-10T02:52:42.990330+0000","last_fullsized":"2024-06-10T02:52:42.990330+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:54:43.728943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.382656+0000","last_change":"2024-06-10T02:50:12.235197+0000","last_active":"2024-06-10T02:51:46.382656+0000","last_peered":"2024-06-10T02:51:46.382656+0000","last_clean":"2024-06-10T02:51:46.382656+0000","last_became_active":"2024-06-10T02:50:12.234791+0000","last_became_peered":"2024-06-10T02:50:12.234791+0000","last_unstale":"2024-06-10T02:51:46.382656+0000","last_undegraded":"2024-06-10T02:51:46.382656+0000","last_fullsized":"2024-06-10T02:51:46.382656+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:09:11.164236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"98'5","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.715164+0000","last_change":"2024-06-10T02:49:58.996759+0000","last_active":"2024-06-10T02:51:45.715164+0000","last_peered":"2024-06-10T02:51:45.715164+0000","last_clean":"2024-06-10T02:51:45.715164+0000","last_became_active":"2024-06-10T02:49:58.996434+0000","last_became_peered":"2024-06-10T02:49:58.996434+0000","last_unstale":"2024-06-10T02:51:45.715164+0000","last_undegraded":"2024-06-10T02:51:45.715164+0000","last_fullsized":"2024-06-10T02:51:45.715164+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:09:35.327797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"100'9","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.996813+0000","last_change":"2024-06-10T02:50:02.201676+0000","last_active":"2024-06-10T02:52:42.996813+0000","last_peered":"2024-06-10T02:52:42.996813+0000","last_clean":"2024-06-10T02:52:42.996813+0000","last_became_active":"2024-06-10T02:50:02.201335+0000","last_became_peered":"2024-06-10T02:50:02.201335+0000","last_unstale":"2024-06-10T02:52:42.996813+0000","last_undegraded":"2024-06-10T02:52:42.996813+0000","last_fullsized":"2024-06-10T02:52:42.996813+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:49:47.038946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"100'9","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.992754+0000","last_change":"2024-06-10T02:50:02.203462+0000","last_active":"2024-06-10T02:52:42.992754+0000","last_peered":"2024-06-10T02:52:42.992754+0000","last_clean":"2024-06-10T02:52:42.992754+0000","last_became_active":"2024-06-10T02:50:02.203115+0000","last_became_peered":"2024-06-10T02:50:02.203115+0000","last_unstale":"2024-06-10T02:52:42.992754+0000","last_undegraded":"2024-06-10T02:52:42.992754+0000","last_fullsized":"2024-06-10T02:52:42.992754+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T13:18:12.400596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"100'12","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.997935+0000","last_change":"2024-06-10T02:50:02.205648+0000","last_active":"2024-06-10T02:52:42.997935+0000","last_peered":"2024-06-10T02:52:42.997935+0000","last_clean":"2024-06-10T02:52:42.997935+0000","last_became_active":"2024-06-10T02:50:02.205306+0000","last_became_peered":"2024-06-10T02:50:02.205306+0000","last_unstale":"2024-06-10T02:52:42.997935+0000","last_undegraded":"2024-06-10T02:52:42.997935+0000","last_fullsized":"2024-06-10T02:52:42.997935+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:22:55.020644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"100'8","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.991481+0000","last_change":"2024-06-10T02:50:02.198601+0000","last_active":"2024-06-10T02:52:42.991481+0000","last_peered":"2024-06-10T02:52:42.991481+0000","last_clean":"2024-06-10T02:52:42.991481+0000","last_became_active":"2024-06-10T02:50:02.198103+0000","last_became_peered":"2024-06-10T02:50:02.198103+0000","last_unstale":"2024-06-10T02:52:42.991481+0000","last_undegraded":"2024-06-10T02:52:42.991481+0000","last_fullsized":"2024-06-10T02:52:42.991481+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:56:28.033482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"100'7","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.985332+0000","last_change":"2024-06-10T02:50:02.202638+0000","last_active":"2024-06-10T02:52:42.985332+0000","last_peered":"2024-06-10T02:52:42.985332+0000","last_clean":"2024-06-10T02:52:42.985332+0000","last_became_active":"2024-06-10T02:50:02.202249+0000","last_became_peered":"2024-06-10T02:50:02.202249+0000","last_unstale":"2024-06-10T02:52:42.985332+0000","last_undegraded":"2024-06-10T02:52:42.985332+0000","last_fullsized":"2024-06-10T02:52:42.985332+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:46:29.424164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"100'4","reported_seq":43,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.992649+0000","last_change":"2024-06-10T02:50:02.213526+0000","last_active":"2024-06-10T02:52:42.992649+0000","last_peered":"2024-06-10T02:52:42.992649+0000","last_clean":"2024-06-10T02:52:42.992649+0000","last_became_active":"2024-06-10T02:50:02.213226+0000","last_became_peered":"2024-06-10T02:50:02.213226+0000","last_unstale":"2024-06-10T02:52:42.992649+0000","last_undegraded":"2024-06-10T02:52:42.992649+0000","last_fullsized":"2024-06-10T02:52:42.992649+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:01:56.846340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"100'6","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.993061+0000","last_change":"2024-06-10T02:50:02.206583+0000","last_active":"2024-06-10T02:52:42.993061+0000","last_peered":"2024-06-10T02:52:42.993061+0000","last_clean":"2024-06-10T02:52:42.993061+0000","last_became_active":"2024-06-10T02:50:02.206227+0000","last_became_peered":"2024-06-10T02:50:02.206227+0000","last_unstale":"2024-06-10T02:52:42.993061+0000","last_undegraded":"2024-06-10T02:52:42.993061+0000","last_fullsized":"2024-06-10T02:52:42.993061+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:25:07.246288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"100'15","reported_seq":54,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:43.002970+0000","last_change":"2024-06-10T02:50:02.206348+0000","last_active":"2024-06-10T02:52:43.002970+0000","last_peered":"2024-06-10T02:52:43.002970+0000","last_clean":"2024-06-10T02:52:43.002970+0000","last_became_active":"2024-06-10T02:50:02.206020+0000","last_became_peered":"2024-06-10T02:50:02.206020+0000","last_unstale":"2024-06-10T02:52:43.002970+0000","last_undegraded":"2024-06-10T02:52:43.002970+0000","last_fullsized":"2024-06-10T02:52:43.002970+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:02:26.619113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"100'2","reported_seq":41,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.993521+0000","last_change":"2024-06-10T02:50:02.199405+0000","last_active":"2024-06-10T02:52:42.993521+0000","last_peered":"2024-06-10T02:52:42.993521+0000","last_clean":"2024-06-10T02:52:42.993521+0000","last_became_active":"2024-06-10T02:50:02.199049+0000","last_became_peered":"2024-06-10T02:50:02.199049+0000","last_unstale":"2024-06-10T02:52:42.993521+0000","last_undegraded":"2024-06-10T02:52:42.993521+0000","last_fullsized":"2024-06-10T02:52:42.993521+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:33:04.091160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"100'6","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.992703+0000","last_change":"2024-06-10T02:50:02.202094+0000","last_active":"2024-06-10T02:52:42.992703+0000","last_peered":"2024-06-10T02:52:42.992703+0000","last_clean":"2024-06-10T02:52:42.992703+0000","last_became_active":"2024-06-10T02:50:02.201751+0000","last_became_peered":"2024-06-10T02:50:02.201751+0000","last_unstale":"2024-06-10T02:52:42.992703+0000","last_undegraded":"2024-06-10T02:52:42.992703+0000","last_fullsized":"2024-06-10T02:52:42.992703+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:52:53.946326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"100'8","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.993442+0000","last_change":"2024-06-10T02:50:02.199630+0000","last_active":"2024-06-10T02:52:42.993442+0000","last_peered":"2024-06-10T02:52:42.993442+0000","last_clean":"2024-06-10T02:52:42.993442+0000","last_became_active":"2024-06-10T02:50:02.199320+0000","last_became_peered":"2024-06-10T02:50:02.199320+0000","last_unstale":"2024-06-10T02:52:42.993442+0000","last_undegraded":"2024-06-10T02:52:42.993442+0000","last_fullsized":"2024-06-10T02:52:42.993442+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:17:28.761862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"100'10","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:43.000631+0000","last_change":"2024-06-10T02:50:02.201190+0000","last_active":"2024-06-10T02:52:43.000631+0000","last_peered":"2024-06-10T02:52:43.000631+0000","last_clean":"2024-06-10T02:52:43.000631+0000","last_became_active":"2024-06-10T02:50:02.200574+0000","last_became_peered":"2024-06-10T02:50:02.200574+0000","last_unstale":"2024-06-10T02:52:43.000631+0000","last_undegraded":"2024-06-10T02:52:43.000631+0000","last_fullsized":"2024-06-10T02:52:43.000631+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:02:07.636218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"100'11","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.992436+0000","last_change":"2024-06-10T02:50:02.202374+0000","last_active":"2024-06-10T02:52:42.992436+0000","last_peered":"2024-06-10T02:52:42.992436+0000","last_clean":"2024-06-10T02:52:42.992436+0000","last_became_active":"2024-06-10T02:50:02.201974+0000","last_became_peered":"2024-06-10T02:50:02.201974+0000","last_unstale":"2024-06-10T02:52:42.992436+0000","last_undegraded":"2024-06-10T02:52:42.992436+0000","last_fullsized":"2024-06-10T02:52:42.992436+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:45:34.977926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"100'8","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.995177+0000","last_change":"2024-06-10T02:50:02.209332+0000","last_active":"2024-06-10T02:52:42.995177+0000","last_peered":"2024-06-10T02:52:42.995177+0000","last_clean":"2024-06-10T02:52:42.995177+0000","last_became_active":"2024-06-10T02:50:02.208175+0000","last_became_peered":"2024-06-10T02:50:02.208175+0000","last_unstale":"2024-06-10T02:52:42.995177+0000","last_undegraded":"2024-06-10T02:52:42.995177+0000","last_fullsized":"2024-06-10T02:52:42.995177+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:44:40.857918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":284028,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":72066,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1829,"num_read_kb":995,"num_write":273,"num_write_kb":163,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":577536,"data_stored":215582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":234,"ondisk_log_size":234,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351042,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32260,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712892,"statfs":{"total":95995035648,"available":95962001408,"internally_reserved":0,"allocated":1110016,"data_stored":574044,"data_compressed":32447,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612679,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32408,"kb_used_data":1232,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712744,"statfs":{"total":95995035648,"available":95961849856,"internally_reserved":0,"allocated":1261568,"data_stored":1223764,"data_compressed":68564,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1599,"internal_metadata":31914433},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776205,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36448,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708704,"statfs":{"total":95995035648,"available":95957712896,"internally_reserved":0,"allocated":1138688,"data_stored":577539,"data_compressed":32447,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037841,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28112,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717040,"statfs":{"total":95995035648,"available":95966248960,"internally_reserved":0,"allocated":1122304,"data_stored":573149,"data_compressed":32447,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299478,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28172,"kb_used_data":1156,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716980,"statfs":{"total":95995035648,"available":95966187520,"internally_reserved":0,"allocated":1183744,"data_stored":1171828,"data_compressed":68564,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863463004,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36420,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708732,"statfs":{"total":95995035648,"available":95957741568,"internally_reserved":0,"allocated":1110016,"data_stored":569478,"data_compressed":32447,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574965,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36488,"kb_used_data":1152,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708664,"statfs":{"total":95995035648,"available":95957671936,"internally_reserved":0,"allocated":1179648,"data_stored":633720,"data_compressed":32447,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738490,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28124,"kb_used_data":1108,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717028,"statfs":{"total":95995035648,"available":95966236672,"internally_reserved":0,"allocated":1134592,"data_stored":570934,"data_compressed":32447,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411440,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28188,"kb_used_data":1172,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716964,"statfs":{"total":95995035648,"available":95966171136,"internally_reserved":0,"allocated":1200128,"data_stored":634163,"data_compressed":32447,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149802,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36432,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708720,"statfs":{"total":95995035648,"available":95957729280,"internally_reserved":0,"allocated":1122304,"data_stored":568655,"data_compressed":32447,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888166,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28116,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717036,"statfs":{"total":95995035648,"available":95966244864,"internally_reserved":0,"allocated":1126400,"data_stored":574224,"data_compressed":32447,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724640,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28180,"kb_used_data":1164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716972,"statfs":{"total":95995035648,"available":95966179328,"internally_reserved":0,"allocated":1191936,"data_stored":1166040,"data_compressed":68564,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34481,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34481,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34481,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36117,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36117,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36117,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36117,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36117,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":65495,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":65938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1903,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1253,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":65479,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-06-10T02:54:12.950 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-06-10T02:54:13.094 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:13.090+0000 7f029056a640 1 -- 172.21.15.52:0/2728818625 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0288100900 msgr2=0x7f0288100d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:54:13.094 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:13.090+0000 7f029056a640 1 --2- 172.21.15.52:0/2728818625 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0288100900 0x7f0288100d20 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f0284009900 tx=0x7f028402a410 comp rx=0 tx=0).stop 2024-06-10T02:54:13.094 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:13.090+0000 7f029056a640 1 -- 172.21.15.52:0/2728818625 shutdown_connections 2024-06-10T02:54:13.094 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:13.090+0000 7f029056a640 1 --2- 172.21.15.52:0/2728818625 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f02881024b0 0x7f0288108d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:13.094 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:13.090+0000 7f029056a640 1 --2- 172.21.15.52:0/2728818625 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0288101af0 0x7f0288101f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:13.094 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:13.090+0000 7f029056a640 1 --2- 172.21.15.52:0/2728818625 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0288100900 0x7f0288100d20 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:13.094 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:13.090+0000 7f029056a640 1 -- 172.21.15.52:0/2728818625 >> 172.21.15.52:0/2728818625 conn(0x7f02880fbe90 msgr2=0x7f02880fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:54:13.094 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:13.090+0000 7f029056a640 1 -- 172.21.15.52:0/2728818625 shutdown_connections 2024-06-10T02:54:13.095 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:13.090+0000 7f029056a640 1 -- 172.21.15.52:0/2728818625 wait complete. 2024-06-10T02:54:13.095 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:13.091+0000 7f029056a640 1 Processor -- start 2024-06-10T02:54:13.095 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:13.091+0000 7f029056a640 1 -- start start 2024-06-10T02:54:13.095 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:13.091+0000 7f029056a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0288100900 0x7f0288196bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:13.095 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:13.091+0000 7f029056a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0288101af0 0x7f0288197130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:13.095 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:13.091+0000 7f029056a640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f02881024b0 0x7f028819e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:13.095 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:13.091+0000 7f029056a640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f028810c550 con 0x7f02881024b0 2024-06-10T02:54:13.095 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:13.091+0000 7f029056a640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f028810c3d0 con 0x7f0288101af0 2024-06-10T02:54:13.095 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:13.091+0000 7f029056a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f028810c6d0 con 0x7f0288100900 2024-06-10T02:54:13.095 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:13.091+0000 7f028eb07640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f02881024b0 0x7f028819e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:13.095 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:13.091+0000 7f028eb07640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f02881024b0 0x7f028819e1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60054/0 (socket says 172.21.15.52:60054) 2024-06-10T02:54:13.095 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:13.091+0000 7f028eb07640 1 -- 172.21.15.52:0/1667430958 learned_addr learned my addr 172.21.15.52:0/1667430958 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:54:13.095 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:13.091+0000 7f028db05640 1 --2- 172.21.15.52:0/1667430958 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0288101af0 0x7f0288197130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:13.096 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:13.091+0000 7f028eb07640 1 -- 172.21.15.52:0/1667430958 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0288100900 msgr2=0x7f0288196bf0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-06-10T02:54:13.096 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:13.091+0000 7f028eb07640 1 --2- 172.21.15.52:0/1667430958 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0288100900 0x7f0288196bf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:13.096 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:13.091+0000 7f028eb07640 1 -- 172.21.15.52:0/1667430958 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0288101af0 msgr2=0x7f0288197130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:54:13.096 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:13.091+0000 7f028eb07640 1 --2- 172.21.15.52:0/1667430958 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0288101af0 0x7f0288197130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:13.096 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:13.091+0000 7f028eb07640 1 -- 172.21.15.52:0/1667430958 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f028819e810 con 0x7f02881024b0 2024-06-10T02:54:13.096 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:13.091+0000 7f028eb07640 1 --2- 172.21.15.52:0/1667430958 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f02881024b0 0x7f028819e1a0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f028000eaa0 tx=0x7f028000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:54:13.096 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:13.092+0000 7f027b7fe640 1 -- 172.21.15.52:0/1667430958 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f028000cdb0 con 0x7f02881024b0 2024-06-10T02:54:13.096 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:13.092+0000 7f027b7fe640 1 -- 172.21.15.52:0/1667430958 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0280005020 con 0x7f02881024b0 2024-06-10T02:54:13.096 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:13.092+0000 7f029056a640 1 -- 172.21.15.52:0/1667430958 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f028819eb00 con 0x7f02881024b0 2024-06-10T02:54:13.096 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:13.092+0000 7f027b7fe640 1 -- 172.21.15.52:0/1667430958 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02800116b0 con 0x7f02881024b0 2024-06-10T02:54:13.096 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:13.092+0000 7f029056a640 1 -- 172.21.15.52:0/1667430958 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f028819f050 con 0x7f02881024b0 2024-06-10T02:54:13.096 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:13.093+0000 7f029056a640 1 -- 172.21.15.52:0/1667430958 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0288107ca0 con 0x7f02881024b0 2024-06-10T02:54:13.096 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:13.094+0000 7f027b7fe640 1 -- 172.21.15.52:0/1667430958 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7f0280005560 con 0x7f02881024b0 2024-06-10T02:54:13.096 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:13.095+0000 7f027b7fe640 1 --2- 172.21.15.52:0/1667430958 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f02740a2ca0 0x7f02740a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:13.097 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:13.095+0000 7f028e306640 1 --2- 172.21.15.52:0/1667430958 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f02740a2ca0 0x7f02740a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:13.097 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:13.095+0000 7f027b7fe640 1 -- 172.21.15.52:0/1667430958 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f02800cfbe0 con 0x7f02881024b0 2024-06-10T02:54:13.097 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:13.095+0000 7f028e306640 1 --2- 172.21.15.52:0/1667430958 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f02740a2ca0 0x7f02740a5160 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f0284005c20 tx=0x7f0284000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:54:13.097 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:13.099+0000 7f027b7fe640 1 -- 172.21.15.52:0/1667430958 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f0280016030 con 0x7f02881024b0 2024-06-10T02:54:13.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:54:12 smithi120 ceph-mon[36634]: pgmap v434: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 597 B/s wr, 9 op/s 2024-06-10T02:54:13.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:54:12 smithi120 ceph-mon[36634]: from='client.? 172.21.15.52:0/4165619478' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T02:54:13.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:54:12 smithi188 ceph-mon[36599]: pgmap v434: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 597 B/s wr, 9 op/s 2024-06-10T02:54:13.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:54:12 smithi188 ceph-mon[36599]: from='client.? 172.21.15.52:0/4165619478' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T02:54:13.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:13.279+0000 7f029056a640 1 -- 172.21.15.52:0/1667430958 --> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0288061de0 con 0x7f02740a2ca0 2024-06-10T02:54:13.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:13.289+0000 7f027b7fe640 1 -- 172.21.15.52:0/1667430958 <== mgr.14150 v2:172.21.15.52:6800/3214763969 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+341022 (secure 0 0 0) 0x7f0288061de0 con 0x7f02740a2ca0 2024-06-10T02:54:13.292 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-10T02:54:13.293 INFO:teuthology.orchestra.run.smithi052.stderr:dumped all 2024-06-10T02:54:13.295 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:13.292+0000 7f029056a640 1 -- 172.21.15.52:0/1667430958 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f02740a2ca0 msgr2=0x7f02740a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:54:13.295 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:13.292+0000 7f029056a640 1 --2- 172.21.15.52:0/1667430958 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f02740a2ca0 0x7f02740a5160 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f0284005c20 tx=0x7f0284000bc0 comp rx=0 tx=0).stop 2024-06-10T02:54:13.295 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:13.292+0000 7f029056a640 1 -- 172.21.15.52:0/1667430958 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f02881024b0 msgr2=0x7f028819e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:54:13.295 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:13.292+0000 7f029056a640 1 --2- 172.21.15.52:0/1667430958 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f02881024b0 0x7f028819e1a0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f028000eaa0 tx=0x7f028000ef60 comp rx=0 tx=0).stop 2024-06-10T02:54:13.296 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:13.292+0000 7f029056a640 1 -- 172.21.15.52:0/1667430958 shutdown_connections 2024-06-10T02:54:13.296 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:13.292+0000 7f029056a640 1 --2- 172.21.15.52:0/1667430958 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f02881024b0 0x7f028819e1a0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:13.296 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:13.292+0000 7f029056a640 1 --2- 172.21.15.52:0/1667430958 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0288101af0 0x7f0288197130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:13.296 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:13.292+0000 7f029056a640 1 --2- 172.21.15.52:0/1667430958 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f02740a2ca0 0x7f02740a5160 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:13.296 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:13.292+0000 7f029056a640 1 --2- 172.21.15.52:0/1667430958 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0288100900 0x7f0288196bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:13.296 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:13.292+0000 7f029056a640 1 -- 172.21.15.52:0/1667430958 >> 172.21.15.52:0/1667430958 conn(0x7f02880fbe90 msgr2=0x7f0288104e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:54:13.296 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:13.293+0000 7f029056a640 1 -- 172.21.15.52:0/1667430958 shutdown_connections 2024-06-10T02:54:13.296 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:13.293+0000 7f029056a640 1 -- 172.21.15.52:0/1667430958 wait complete. 2024-06-10T02:54:13.303 INFO:teuthology.orchestra.run.smithi052.stdout:{"pg_ready":true,"pg_map":{"version":435,"stamp":"2024-06-10T02:54:13.043895+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":671341,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1999,"num_read_kb":1133,"num_write":459,"num_write_kb":3113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1181318,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":876,"ondisk_log_size":876,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":379364,"kb_used_data":13572,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124562460,"statfs":{"total":1151940427776,"available":1151551959040,"internally_reserved":0,"allocated":13897728,"data_stored":8859749,"data_compressed":497715,"data_compressed_allocated":675840,"data_compressed_original":2850816,"omap_allocated":19141,"internal_metadata":374453563},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":9289,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":144,"num_read_kb":79,"num_write":12,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003351"},"pg_stats":[{"pgid":"3.3f","version":"100'8","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:43.001975+0000","last_change":"2024-06-10T02:50:02.209146+0000","last_active":"2024-06-10T02:52:43.001975+0000","last_peered":"2024-06-10T02:52:43.001975+0000","last_clean":"2024-06-10T02:52:43.001975+0000","last_became_active":"2024-06-10T02:50:02.208027+0000","last_became_peered":"2024-06-10T02:50:02.208027+0000","last_unstale":"2024-06-10T02:52:43.001975+0000","last_undegraded":"2024-06-10T02:52:43.001975+0000","last_fullsized":"2024-06-10T02:52:43.001975+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:35:34.765849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"100'4","reported_seq":43,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.989740+0000","last_change":"2024-06-10T02:50:02.198767+0000","last_active":"2024-06-10T02:52:42.989740+0000","last_peered":"2024-06-10T02:52:42.989740+0000","last_clean":"2024-06-10T02:52:42.989740+0000","last_became_active":"2024-06-10T02:50:02.198450+0000","last_became_peered":"2024-06-10T02:50:02.198450+0000","last_unstale":"2024-06-10T02:52:42.989740+0000","last_undegraded":"2024-06-10T02:52:42.989740+0000","last_fullsized":"2024-06-10T02:52:42.989740+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:49:56.666083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"100'7","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:43.000326+0000","last_change":"2024-06-10T02:50:02.208885+0000","last_active":"2024-06-10T02:52:43.000326+0000","last_peered":"2024-06-10T02:52:43.000326+0000","last_clean":"2024-06-10T02:52:43.000326+0000","last_became_active":"2024-06-10T02:50:02.206953+0000","last_became_peered":"2024-06-10T02:50:02.206953+0000","last_unstale":"2024-06-10T02:52:43.000326+0000","last_undegraded":"2024-06-10T02:52:43.000326+0000","last_fullsized":"2024-06-10T02:52:43.000326+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:41:23.928939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"100'8","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.994111+0000","last_change":"2024-06-10T02:50:02.208632+0000","last_active":"2024-06-10T02:52:42.994111+0000","last_peered":"2024-06-10T02:52:42.994111+0000","last_clean":"2024-06-10T02:52:42.994111+0000","last_became_active":"2024-06-10T02:50:02.207587+0000","last_became_peered":"2024-06-10T02:50:02.207587+0000","last_unstale":"2024-06-10T02:52:42.994111+0000","last_undegraded":"2024-06-10T02:52:42.994111+0000","last_fullsized":"2024-06-10T02:52:42.994111+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T14:34:36.720170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"100'4","reported_seq":43,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.973606+0000","last_change":"2024-06-10T02:50:02.197966+0000","last_active":"2024-06-10T02:52:42.973606+0000","last_peered":"2024-06-10T02:52:42.973606+0000","last_clean":"2024-06-10T02:52:42.973606+0000","last_became_active":"2024-06-10T02:50:02.197612+0000","last_became_peered":"2024-06-10T02:50:02.197612+0000","last_unstale":"2024-06-10T02:52:42.973606+0000","last_undegraded":"2024-06-10T02:52:42.973606+0000","last_fullsized":"2024-06-10T02:52:42.973606+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:24:53.804613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"100'6","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.987354+0000","last_change":"2024-06-10T02:50:02.201952+0000","last_active":"2024-06-10T02:52:42.987354+0000","last_peered":"2024-06-10T02:52:42.987354+0000","last_clean":"2024-06-10T02:52:42.987354+0000","last_became_active":"2024-06-10T02:50:02.200828+0000","last_became_peered":"2024-06-10T02:50:02.200828+0000","last_unstale":"2024-06-10T02:52:42.987354+0000","last_undegraded":"2024-06-10T02:52:42.987354+0000","last_fullsized":"2024-06-10T02:52:42.987354+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:46:28.518846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"100'5","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.990208+0000","last_change":"2024-06-10T02:50:02.203483+0000","last_active":"2024-06-10T02:52:42.990208+0000","last_peered":"2024-06-10T02:52:42.990208+0000","last_clean":"2024-06-10T02:52:42.990208+0000","last_became_active":"2024-06-10T02:50:02.203153+0000","last_became_peered":"2024-06-10T02:50:02.203153+0000","last_unstale":"2024-06-10T02:52:42.990208+0000","last_undegraded":"2024-06-10T02:52:42.990208+0000","last_fullsized":"2024-06-10T02:52:42.990208+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:29:30.422449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"100'9","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.995630+0000","last_change":"2024-06-10T02:50:02.201686+0000","last_active":"2024-06-10T02:52:42.995630+0000","last_peered":"2024-06-10T02:52:42.995630+0000","last_clean":"2024-06-10T02:52:42.995630+0000","last_became_active":"2024-06-10T02:50:02.200973+0000","last_became_peered":"2024-06-10T02:50:02.200973+0000","last_unstale":"2024-06-10T02:52:42.995630+0000","last_undegraded":"2024-06-10T02:52:42.995630+0000","last_fullsized":"2024-06-10T02:52:42.995630+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:41:13.712675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"100'11","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:43.001422+0000","last_change":"2024-06-10T02:50:02.210497+0000","last_active":"2024-06-10T02:52:43.001422+0000","last_peered":"2024-06-10T02:52:43.001422+0000","last_clean":"2024-06-10T02:52:43.001422+0000","last_became_active":"2024-06-10T02:50:02.209675+0000","last_became_peered":"2024-06-10T02:50:02.209675+0000","last_unstale":"2024-06-10T02:52:43.001422+0000","last_undegraded":"2024-06-10T02:52:43.001422+0000","last_fullsized":"2024-06-10T02:52:43.001422+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:50:29.879669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"100'16","reported_seq":61,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.994975+0000","last_change":"2024-06-10T02:50:44.741139+0000","last_active":"2024-06-10T02:52:42.994975+0000","last_peered":"2024-06-10T02:52:42.994975+0000","last_clean":"2024-06-10T02:52:42.994975+0000","last_became_active":"2024-06-10T02:50:44.740685+0000","last_became_peered":"2024-06-10T02:50:44.740685+0000","last_unstale":"2024-06-10T02:52:42.994975+0000","last_undegraded":"2024-06-10T02:52:42.994975+0000","last_fullsized":"2024-06-10T02:52:42.994975+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:58:59.872732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"100'6","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.999574+0000","last_change":"2024-06-10T02:50:02.206993+0000","last_active":"2024-06-10T02:52:42.999574+0000","last_peered":"2024-06-10T02:52:42.999574+0000","last_clean":"2024-06-10T02:52:42.999574+0000","last_became_active":"2024-06-10T02:50:02.206650+0000","last_became_peered":"2024-06-10T02:50:02.206650+0000","last_unstale":"2024-06-10T02:52:42.999574+0000","last_undegraded":"2024-06-10T02:52:42.999574+0000","last_fullsized":"2024-06-10T02:52:42.999574+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T14:27:49.555350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"100'7","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.979413+0000","last_change":"2024-06-10T02:50:02.200816+0000","last_active":"2024-06-10T02:52:42.979413+0000","last_peered":"2024-06-10T02:52:42.979413+0000","last_clean":"2024-06-10T02:52:42.979413+0000","last_became_active":"2024-06-10T02:50:02.200503+0000","last_became_peered":"2024-06-10T02:50:02.200503+0000","last_unstale":"2024-06-10T02:52:42.979413+0000","last_undegraded":"2024-06-10T02:52:42.979413+0000","last_fullsized":"2024-06-10T02:52:42.979413+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:01:08.326728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"100'10","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.987233+0000","last_change":"2024-06-10T02:50:02.203644+0000","last_active":"2024-06-10T02:52:42.987233+0000","last_peered":"2024-06-10T02:52:42.987233+0000","last_clean":"2024-06-10T02:52:42.987233+0000","last_became_active":"2024-06-10T02:50:02.203100+0000","last_became_peered":"2024-06-10T02:50:02.203100+0000","last_unstale":"2024-06-10T02:52:42.987233+0000","last_undegraded":"2024-06-10T02:52:42.987233+0000","last_fullsized":"2024-06-10T02:52:42.987233+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:01:42.730089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"100'7","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.995068+0000","last_change":"2024-06-10T02:50:02.203330+0000","last_active":"2024-06-10T02:52:42.995068+0000","last_peered":"2024-06-10T02:52:42.995068+0000","last_clean":"2024-06-10T02:52:42.995068+0000","last_became_active":"2024-06-10T02:50:02.202995+0000","last_became_peered":"2024-06-10T02:50:02.202995+0000","last_unstale":"2024-06-10T02:52:42.995068+0000","last_undegraded":"2024-06-10T02:52:42.995068+0000","last_fullsized":"2024-06-10T02:52:42.995068+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:17:23.170591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"100'7","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.986439+0000","last_change":"2024-06-10T02:50:02.204875+0000","last_active":"2024-06-10T02:52:42.986439+0000","last_peered":"2024-06-10T02:52:42.986439+0000","last_clean":"2024-06-10T02:52:42.986439+0000","last_became_active":"2024-06-10T02:50:02.200706+0000","last_became_peered":"2024-06-10T02:50:02.200706+0000","last_unstale":"2024-06-10T02:52:42.986439+0000","last_undegraded":"2024-06-10T02:52:42.986439+0000","last_fullsized":"2024-06-10T02:52:42.986439+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:27:12.282772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"100'9","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.999375+0000","last_change":"2024-06-10T02:50:02.203187+0000","last_active":"2024-06-10T02:52:42.999375+0000","last_peered":"2024-06-10T02:52:42.999375+0000","last_clean":"2024-06-10T02:52:42.999375+0000","last_became_active":"2024-06-10T02:50:02.202034+0000","last_became_peered":"2024-06-10T02:50:02.202034+0000","last_unstale":"2024-06-10T02:52:42.999375+0000","last_undegraded":"2024-06-10T02:52:42.999375+0000","last_fullsized":"2024-06-10T02:52:42.999375+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:59:44.783084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"100'8","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:43.002372+0000","last_change":"2024-06-10T02:50:02.209102+0000","last_active":"2024-06-10T02:52:43.002372+0000","last_peered":"2024-06-10T02:52:43.002372+0000","last_clean":"2024-06-10T02:52:43.002372+0000","last_became_active":"2024-06-10T02:50:02.208730+0000","last_became_peered":"2024-06-10T02:50:02.208730+0000","last_unstale":"2024-06-10T02:52:43.002372+0000","last_undegraded":"2024-06-10T02:52:43.002372+0000","last_fullsized":"2024-06-10T02:52:43.002372+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:08:38.838415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"100'10","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:43.000459+0000","last_change":"2024-06-10T02:50:02.204008+0000","last_active":"2024-06-10T02:52:43.000459+0000","last_peered":"2024-06-10T02:52:43.000459+0000","last_clean":"2024-06-10T02:52:43.000459+0000","last_became_active":"2024-06-10T02:50:02.203655+0000","last_became_peered":"2024-06-10T02:50:02.203655+0000","last_unstale":"2024-06-10T02:52:43.000459+0000","last_undegraded":"2024-06-10T02:52:43.000459+0000","last_fullsized":"2024-06-10T02:52:43.000459+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:32:15.972662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"100'7","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.991723+0000","last_change":"2024-06-10T02:50:02.204917+0000","last_active":"2024-06-10T02:52:42.991723+0000","last_peered":"2024-06-10T02:52:42.991723+0000","last_clean":"2024-06-10T02:52:42.991723+0000","last_became_active":"2024-06-10T02:50:02.204568+0000","last_became_peered":"2024-06-10T02:50:02.204568+0000","last_unstale":"2024-06-10T02:52:42.991723+0000","last_undegraded":"2024-06-10T02:52:42.991723+0000","last_fullsized":"2024-06-10T02:52:42.991723+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:49:46.802665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.721297+0000","last_change":"2024-06-10T02:50:12.240697+0000","last_active":"2024-06-10T02:51:45.721297+0000","last_peered":"2024-06-10T02:51:45.721297+0000","last_clean":"2024-06-10T02:51:45.721297+0000","last_became_active":"2024-06-10T02:50:12.240229+0000","last_became_peered":"2024-06-10T02:50:12.240229+0000","last_unstale":"2024-06-10T02:51:45.721297+0000","last_undegraded":"2024-06-10T02:51:45.721297+0000","last_fullsized":"2024-06-10T02:51:45.721297+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:45:55.071960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"100'15","reported_seq":255,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:54:09.893400+0000","last_change":"2024-06-10T02:50:44.786783+0000","last_active":"2024-06-10T02:54:09.893400+0000","last_peered":"2024-06-10T02:54:09.893400+0000","last_clean":"2024-06-10T02:54:09.893400+0000","last_became_active":"2024-06-10T02:50:44.781773+0000","last_became_peered":"2024-06-10T02:50:44.781773+0000","last_unstale":"2024-06-10T02:54:09.893400+0000","last_undegraded":"2024-06-10T02:54:09.893400+0000","last_fullsized":"2024-06-10T02:54:09.893400+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:31:06.025854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":182,"num_read_kb":182,"num_write":16,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,4],"acting":[9,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"100'5","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.994824+0000","last_change":"2024-06-10T02:50:02.202573+0000","last_active":"2024-06-10T02:52:42.994824+0000","last_peered":"2024-06-10T02:52:42.994824+0000","last_clean":"2024-06-10T02:52:42.994824+0000","last_became_active":"2024-06-10T02:50:02.202241+0000","last_became_peered":"2024-06-10T02:50:02.202241+0000","last_unstale":"2024-06-10T02:52:42.994824+0000","last_undegraded":"2024-06-10T02:52:42.994824+0000","last_fullsized":"2024-06-10T02:52:42.994824+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:31:40.230851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.383047+0000","last_change":"2024-06-10T02:50:12.238372+0000","last_active":"2024-06-10T02:51:46.383047+0000","last_peered":"2024-06-10T02:51:46.383047+0000","last_clean":"2024-06-10T02:51:46.383047+0000","last_became_active":"2024-06-10T02:50:12.237884+0000","last_became_peered":"2024-06-10T02:50:12.237884+0000","last_unstale":"2024-06-10T02:51:46.383047+0000","last_undegraded":"2024-06-10T02:51:46.383047+0000","last_fullsized":"2024-06-10T02:51:46.383047+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:55:31.432655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"98'2","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:02.117661+0000","last_change":"2024-06-10T02:49:59.031528+0000","last_active":"2024-06-10T02:52:02.117661+0000","last_peered":"2024-06-10T02:52:02.117661+0000","last_clean":"2024-06-10T02:52:02.117661+0000","last_became_active":"2024-06-10T02:49:59.031147+0000","last_became_peered":"2024-06-10T02:49:59.031147+0000","last_unstale":"2024-06-10T02:52:02.117661+0000","last_undegraded":"2024-06-10T02:52:02.117661+0000","last_fullsized":"2024-06-10T02:52:02.117661+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:17:08.534396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"100'6","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:43.000760+0000","last_change":"2024-06-10T02:50:02.202858+0000","last_active":"2024-06-10T02:52:43.000760+0000","last_peered":"2024-06-10T02:52:43.000760+0000","last_clean":"2024-06-10T02:52:43.000760+0000","last_became_active":"2024-06-10T02:50:02.202263+0000","last_became_peered":"2024-06-10T02:50:02.202263+0000","last_unstale":"2024-06-10T02:52:43.000760+0000","last_undegraded":"2024-06-10T02:52:43.000760+0000","last_fullsized":"2024-06-10T02:52:43.000760+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:23:15.382762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.383227+0000","last_change":"2024-06-10T02:50:12.237370+0000","last_active":"2024-06-10T02:51:46.383227+0000","last_peered":"2024-06-10T02:51:46.383227+0000","last_clean":"2024-06-10T02:51:46.383227+0000","last_became_active":"2024-06-10T02:50:12.236803+0000","last_became_peered":"2024-06-10T02:50:12.236803+0000","last_unstale":"2024-06-10T02:51:46.383227+0000","last_undegraded":"2024-06-10T02:51:46.383227+0000","last_fullsized":"2024-06-10T02:51:46.383227+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:06:40.695710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"98'12","reported_seq":58,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.724307+0000","last_change":"2024-06-10T02:49:59.000911+0000","last_active":"2024-06-10T02:51:45.724307+0000","last_peered":"2024-06-10T02:51:45.724307+0000","last_clean":"2024-06-10T02:51:45.724307+0000","last_became_active":"2024-06-10T02:49:59.000571+0000","last_became_peered":"2024-06-10T02:49:59.000571+0000","last_unstale":"2024-06-10T02:51:45.724307+0000","last_undegraded":"2024-06-10T02:51:45.724307+0000","last_fullsized":"2024-06-10T02:51:45.724307+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:49:24.635793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"100'9","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.997165+0000","last_change":"2024-06-10T02:50:02.209614+0000","last_active":"2024-06-10T02:52:42.997165+0000","last_peered":"2024-06-10T02:52:42.997165+0000","last_clean":"2024-06-10T02:52:42.997165+0000","last_became_active":"2024-06-10T02:50:02.209251+0000","last_became_peered":"2024-06-10T02:50:02.209251+0000","last_unstale":"2024-06-10T02:52:42.997165+0000","last_undegraded":"2024-06-10T02:52:42.997165+0000","last_fullsized":"2024-06-10T02:52:42.997165+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:35:32.256081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.383229+0000","last_change":"2024-06-10T02:50:12.237811+0000","last_active":"2024-06-10T02:51:46.383229+0000","last_peered":"2024-06-10T02:51:46.383229+0000","last_clean":"2024-06-10T02:51:46.383229+0000","last_became_active":"2024-06-10T02:50:12.237017+0000","last_became_peered":"2024-06-10T02:50:12.237017+0000","last_unstale":"2024-06-10T02:51:46.383229+0000","last_undegraded":"2024-06-10T02:51:46.383229+0000","last_fullsized":"2024-06-10T02:51:46.383229+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:14:29.818885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"98'5","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.723967+0000","last_change":"2024-06-10T02:49:58.996971+0000","last_active":"2024-06-10T02:51:45.723967+0000","last_peered":"2024-06-10T02:51:45.723967+0000","last_clean":"2024-06-10T02:51:45.723967+0000","last_became_active":"2024-06-10T02:49:58.996601+0000","last_became_peered":"2024-06-10T02:49:58.996601+0000","last_unstale":"2024-06-10T02:51:45.723967+0000","last_undegraded":"2024-06-10T02:51:45.723967+0000","last_fullsized":"2024-06-10T02:51:45.723967+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:26:45.963351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"100'15","reported_seq":60,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:43.002596+0000","last_change":"2024-06-10T02:50:44.745816+0000","last_active":"2024-06-10T02:52:43.002596+0000","last_peered":"2024-06-10T02:52:43.002596+0000","last_clean":"2024-06-10T02:52:43.002596+0000","last_became_active":"2024-06-10T02:50:44.745386+0000","last_became_peered":"2024-06-10T02:50:44.745386+0000","last_unstale":"2024-06-10T02:52:43.002596+0000","last_undegraded":"2024-06-10T02:52:43.002596+0000","last_fullsized":"2024-06-10T02:52:43.002596+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:08:53.977133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.721700+0000","last_change":"2024-06-10T02:50:12.236835+0000","last_active":"2024-06-10T02:51:45.721700+0000","last_peered":"2024-06-10T02:51:45.721700+0000","last_clean":"2024-06-10T02:51:45.721700+0000","last_became_active":"2024-06-10T02:50:12.236309+0000","last_became_peered":"2024-06-10T02:50:12.236309+0000","last_unstale":"2024-06-10T02:51:45.721700+0000","last_undegraded":"2024-06-10T02:51:45.721700+0000","last_fullsized":"2024-06-10T02:51:45.721700+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:09:02.789286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"100'8","reported_seq":240,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:54:09.894381+0000","last_change":"2024-06-10T02:49:58.997725+0000","last_active":"2024-06-10T02:54:09.894381+0000","last_peered":"2024-06-10T02:54:09.894381+0000","last_clean":"2024-06-10T02:54:09.894381+0000","last_became_active":"2024-06-10T02:49:58.997375+0000","last_became_peered":"2024-06-10T02:49:58.997375+0000","last_unstale":"2024-06-10T02:54:09.894381+0000","last_undegraded":"2024-06-10T02:54:09.894381+0000","last_fullsized":"2024-06-10T02:54:09.894381+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:28:49.714356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1461,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":187,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"100'7","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.995883+0000","last_change":"2024-06-10T02:50:02.201247+0000","last_active":"2024-06-10T02:52:42.995883+0000","last_peered":"2024-06-10T02:52:42.995883+0000","last_clean":"2024-06-10T02:52:42.995883+0000","last_became_active":"2024-06-10T02:50:02.200897+0000","last_became_peered":"2024-06-10T02:50:02.200897+0000","last_unstale":"2024-06-10T02:52:42.995883+0000","last_undegraded":"2024-06-10T02:52:42.995883+0000","last_fullsized":"2024-06-10T02:52:42.995883+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:01:58.166644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.721648+0000","last_change":"2024-06-10T02:50:12.243097+0000","last_active":"2024-06-10T02:51:45.721648+0000","last_peered":"2024-06-10T02:51:45.721648+0000","last_clean":"2024-06-10T02:51:45.721648+0000","last_became_active":"2024-06-10T02:50:12.242501+0000","last_became_peered":"2024-06-10T02:50:12.242501+0000","last_unstale":"2024-06-10T02:51:45.721648+0000","last_undegraded":"2024-06-10T02:51:45.721648+0000","last_fullsized":"2024-06-10T02:51:45.721648+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T14:42:44.220340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"98'7","reported_seq":52,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.720539+0000","last_change":"2024-06-10T02:49:58.996956+0000","last_active":"2024-06-10T02:51:45.720539+0000","last_peered":"2024-06-10T02:51:45.720539+0000","last_clean":"2024-06-10T02:51:45.720539+0000","last_became_active":"2024-06-10T02:49:58.996634+0000","last_became_peered":"2024-06-10T02:49:58.996634+0000","last_unstale":"2024-06-10T02:51:45.720539+0000","last_undegraded":"2024-06-10T02:51:45.720539+0000","last_fullsized":"2024-06-10T02:51:45.720539+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:25:02.364472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"100'8","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.995286+0000","last_change":"2024-06-10T02:50:02.205808+0000","last_active":"2024-06-10T02:52:42.995286+0000","last_peered":"2024-06-10T02:52:42.995286+0000","last_clean":"2024-06-10T02:52:42.995286+0000","last_became_active":"2024-06-10T02:50:02.203558+0000","last_became_peered":"2024-06-10T02:50:02.203558+0000","last_unstale":"2024-06-10T02:52:42.995286+0000","last_undegraded":"2024-06-10T02:52:42.995286+0000","last_fullsized":"2024-06-10T02:52:42.995286+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T13:43:23.315375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.365641+0000","last_change":"2024-06-10T02:50:12.242005+0000","last_active":"2024-06-10T02:51:46.365641+0000","last_peered":"2024-06-10T02:51:46.365641+0000","last_clean":"2024-06-10T02:51:46.365641+0000","last_became_active":"2024-06-10T02:50:12.241295+0000","last_became_peered":"2024-06-10T02:50:12.241295+0000","last_unstale":"2024-06-10T02:51:46.365641+0000","last_undegraded":"2024-06-10T02:51:46.365641+0000","last_fullsized":"2024-06-10T02:51:46.365641+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:32:52.702756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"98'3","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.714696+0000","last_change":"2024-06-10T02:49:58.995579+0000","last_active":"2024-06-10T02:51:45.714696+0000","last_peered":"2024-06-10T02:51:45.714696+0000","last_clean":"2024-06-10T02:51:45.714696+0000","last_became_active":"2024-06-10T02:49:58.995208+0000","last_became_peered":"2024-06-10T02:49:58.995208+0000","last_unstale":"2024-06-10T02:51:45.714696+0000","last_undegraded":"2024-06-10T02:51:45.714696+0000","last_fullsized":"2024-06-10T02:51:45.714696+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:26:03.513482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"100'8","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.993655+0000","last_change":"2024-06-10T02:50:02.197090+0000","last_active":"2024-06-10T02:52:42.993655+0000","last_peered":"2024-06-10T02:52:42.993655+0000","last_clean":"2024-06-10T02:52:42.993655+0000","last_became_active":"2024-06-10T02:50:02.196746+0000","last_became_peered":"2024-06-10T02:50:02.196746+0000","last_unstale":"2024-06-10T02:52:42.993655+0000","last_undegraded":"2024-06-10T02:52:42.993655+0000","last_fullsized":"2024-06-10T02:52:42.993655+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:52:09.987978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.012041+0000","last_change":"2024-06-10T02:50:12.234632+0000","last_active":"2024-06-10T02:51:46.012041+0000","last_peered":"2024-06-10T02:51:46.012041+0000","last_clean":"2024-06-10T02:51:46.012041+0000","last_became_active":"2024-06-10T02:50:12.234266+0000","last_became_peered":"2024-06-10T02:50:12.234266+0000","last_unstale":"2024-06-10T02:51:46.012041+0000","last_undegraded":"2024-06-10T02:51:46.012041+0000","last_fullsized":"2024-06-10T02:51:46.012041+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:21:27.348441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"98'3","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.383558+0000","last_change":"2024-06-10T02:49:58.999086+0000","last_active":"2024-06-10T02:51:46.383558+0000","last_peered":"2024-06-10T02:51:46.383558+0000","last_clean":"2024-06-10T02:51:46.383558+0000","last_became_active":"2024-06-10T02:49:58.998767+0000","last_became_peered":"2024-06-10T02:49:58.998767+0000","last_unstale":"2024-06-10T02:51:46.383558+0000","last_undegraded":"2024-06-10T02:51:46.383558+0000","last_fullsized":"2024-06-10T02:51:46.383558+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:28:12.922391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"100'11","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.990813+0000","last_change":"2024-06-10T02:50:02.196641+0000","last_active":"2024-06-10T02:52:42.990813+0000","last_peered":"2024-06-10T02:52:42.990813+0000","last_clean":"2024-06-10T02:52:42.990813+0000","last_became_active":"2024-06-10T02:50:02.196285+0000","last_became_peered":"2024-06-10T02:50:02.196285+0000","last_unstale":"2024-06-10T02:52:42.990813+0000","last_undegraded":"2024-06-10T02:52:42.990813+0000","last_fullsized":"2024-06-10T02:52:42.990813+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T13:55:31.284028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.831864+0000","last_change":"2024-06-10T02:50:12.240988+0000","last_active":"2024-06-10T02:51:45.831864+0000","last_peered":"2024-06-10T02:51:45.831864+0000","last_clean":"2024-06-10T02:51:45.831864+0000","last_became_active":"2024-06-10T02:50:12.240490+0000","last_became_peered":"2024-06-10T02:50:12.240490+0000","last_unstale":"2024-06-10T02:51:45.831864+0000","last_undegraded":"2024-06-10T02:51:45.831864+0000","last_fullsized":"2024-06-10T02:51:45.831864+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:54:36.015221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"100'7","reported_seq":416,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:54:10.894993+0000","last_change":"2024-06-10T02:49:58.989557+0000","last_active":"2024-06-10T02:54:10.894993+0000","last_peered":"2024-06-10T02:54:10.894993+0000","last_clean":"2024-06-10T02:54:10.894993+0000","last_became_active":"2024-06-10T02:49:58.989101+0000","last_became_peered":"2024-06-10T02:49:58.989101+0000","last_unstale":"2024-06-10T02:54:10.894993+0000","last_undegraded":"2024-06-10T02:54:10.894993+0000","last_fullsized":"2024-06-10T02:54:10.894993+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:27:42.182012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":364,"num_read_kb":364,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"100'10","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.998692+0000","last_change":"2024-06-10T02:50:02.208752+0000","last_active":"2024-06-10T02:52:42.998692+0000","last_peered":"2024-06-10T02:52:42.998692+0000","last_clean":"2024-06-10T02:52:42.998692+0000","last_became_active":"2024-06-10T02:50:02.208420+0000","last_became_peered":"2024-06-10T02:50:02.208420+0000","last_unstale":"2024-06-10T02:52:42.998692+0000","last_undegraded":"2024-06-10T02:52:42.998692+0000","last_fullsized":"2024-06-10T02:52:42.998692+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:28:09.412317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.382833+0000","last_change":"2024-06-10T02:50:12.236397+0000","last_active":"2024-06-10T02:51:46.382833+0000","last_peered":"2024-06-10T02:51:46.382833+0000","last_clean":"2024-06-10T02:51:46.382833+0000","last_became_active":"2024-06-10T02:50:12.235972+0000","last_became_peered":"2024-06-10T02:50:12.235972+0000","last_unstale":"2024-06-10T02:51:46.382833+0000","last_undegraded":"2024-06-10T02:51:46.382833+0000","last_fullsized":"2024-06-10T02:51:46.382833+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:38:41.723895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"98'4","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.295527+0000","last_change":"2024-06-10T02:49:58.996616+0000","last_active":"2024-06-10T02:51:46.295527+0000","last_peered":"2024-06-10T02:51:46.295527+0000","last_clean":"2024-06-10T02:51:46.295527+0000","last_became_active":"2024-06-10T02:49:58.996289+0000","last_became_peered":"2024-06-10T02:49:58.996289+0000","last_unstale":"2024-06-10T02:51:46.295527+0000","last_undegraded":"2024-06-10T02:51:46.295527+0000","last_fullsized":"2024-06-10T02:51:46.295527+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:22:12.993608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"100'9","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.983255+0000","last_change":"2024-06-10T02:50:02.205230+0000","last_active":"2024-06-10T02:52:42.983255+0000","last_peered":"2024-06-10T02:52:42.983255+0000","last_clean":"2024-06-10T02:52:42.983255+0000","last_became_active":"2024-06-10T02:50:02.204835+0000","last_became_peered":"2024-06-10T02:50:02.204835+0000","last_unstale":"2024-06-10T02:52:42.983255+0000","last_undegraded":"2024-06-10T02:52:42.983255+0000","last_fullsized":"2024-06-10T02:52:42.983255+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:17:35.084921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.383415+0000","last_change":"2024-06-10T02:50:12.231684+0000","last_active":"2024-06-10T02:51:46.383415+0000","last_peered":"2024-06-10T02:51:46.383415+0000","last_clean":"2024-06-10T02:51:46.383415+0000","last_became_active":"2024-06-10T02:50:12.231256+0000","last_became_peered":"2024-06-10T02:50:12.231256+0000","last_unstale":"2024-06-10T02:51:46.383415+0000","last_undegraded":"2024-06-10T02:51:46.383415+0000","last_fullsized":"2024-06-10T02:51:46.383415+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:17:31.878289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"98'5","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.383170+0000","last_change":"2024-06-10T02:49:59.033950+0000","last_active":"2024-06-10T02:51:46.383170+0000","last_peered":"2024-06-10T02:51:46.383170+0000","last_clean":"2024-06-10T02:51:46.383170+0000","last_became_active":"2024-06-10T02:49:59.033688+0000","last_became_peered":"2024-06-10T02:49:59.033688+0000","last_unstale":"2024-06-10T02:51:46.383170+0000","last_undegraded":"2024-06-10T02:51:46.383170+0000","last_fullsized":"2024-06-10T02:51:46.383170+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:32:03.892091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"100'11","reported_seq":56,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.999378+0000","last_change":"2024-06-10T02:50:44.781378+0000","last_active":"2024-06-10T02:52:42.999378+0000","last_peered":"2024-06-10T02:52:42.999378+0000","last_clean":"2024-06-10T02:52:42.999378+0000","last_became_active":"2024-06-10T02:50:44.780955+0000","last_became_peered":"2024-06-10T02:50:44.780955+0000","last_unstale":"2024-06-10T02:52:42.999378+0000","last_undegraded":"2024-06-10T02:52:42.999378+0000","last_fullsized":"2024-06-10T02:52:42.999378+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:49:13.760381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.827482+0000","last_change":"2024-06-10T02:51:46.384748+0000","last_active":"2024-06-10T02:51:46.827482+0000","last_peered":"2024-06-10T02:51:46.827482+0000","last_clean":"2024-06-10T02:51:46.827482+0000","last_became_active":"2024-06-10T02:51:46.384402+0000","last_became_peered":"2024-06-10T02:51:46.384402+0000","last_unstale":"2024-06-10T02:51:46.827482+0000","last_undegraded":"2024-06-10T02:51:46.827482+0000","last_fullsized":"2024-06-10T02:51:46.827482+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:06:48.056342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.14","version":"100'41","reported_seq":462,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:54:10.914777+0000","last_change":"2024-06-10T02:49:58.997890+0000","last_active":"2024-06-10T02:54:10.914777+0000","last_peered":"2024-06-10T02:54:10.914777+0000","last_clean":"2024-06-10T02:54:10.914777+0000","last_became_active":"2024-06-10T02:49:58.997549+0000","last_became_peered":"2024-06-10T02:49:58.997549+0000","last_unstale":"2024-06-10T02:54:10.914777+0000","last_undegraded":"2024-06-10T02:54:10.914777+0000","last_fullsized":"2024-06-10T02:54:10.914777+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T14:40:50.907029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71495,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":376,"num_read_kb":256,"num_write":43,"num_write_kb":90,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"100'6","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.991053+0000","last_change":"2024-06-10T02:50:02.199707+0000","last_active":"2024-06-10T02:52:42.991053+0000","last_peered":"2024-06-10T02:52:42.991053+0000","last_clean":"2024-06-10T02:52:42.991053+0000","last_became_active":"2024-06-10T02:50:02.198987+0000","last_became_peered":"2024-06-10T02:50:02.198987+0000","last_unstale":"2024-06-10T02:52:42.991053+0000","last_undegraded":"2024-06-10T02:52:42.991053+0000","last_fullsized":"2024-06-10T02:52:42.991053+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:48:03.704590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.383847+0000","last_change":"2024-06-10T02:50:12.239608+0000","last_active":"2024-06-10T02:51:46.383847+0000","last_peered":"2024-06-10T02:51:46.383847+0000","last_clean":"2024-06-10T02:51:46.383847+0000","last_became_active":"2024-06-10T02:50:12.238588+0000","last_became_peered":"2024-06-10T02:50:12.238588+0000","last_unstale":"2024-06-10T02:51:46.383847+0000","last_undegraded":"2024-06-10T02:51:46.383847+0000","last_fullsized":"2024-06-10T02:51:46.383847+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:26:12.718921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"98'9","reported_seq":56,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.721583+0000","last_change":"2024-06-10T02:49:58.997193+0000","last_active":"2024-06-10T02:51:45.721583+0000","last_peered":"2024-06-10T02:51:45.721583+0000","last_clean":"2024-06-10T02:51:45.721583+0000","last_became_active":"2024-06-10T02:49:58.996712+0000","last_became_peered":"2024-06-10T02:49:58.996712+0000","last_unstale":"2024-06-10T02:51:45.721583+0000","last_undegraded":"2024-06-10T02:51:45.721583+0000","last_fullsized":"2024-06-10T02:51:45.721583+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:02:46.894489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"100'9","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.994408+0000","last_change":"2024-06-10T02:50:02.200544+0000","last_active":"2024-06-10T02:52:42.994408+0000","last_peered":"2024-06-10T02:52:42.994408+0000","last_clean":"2024-06-10T02:52:42.994408+0000","last_became_active":"2024-06-10T02:50:02.200201+0000","last_became_peered":"2024-06-10T02:50:02.200201+0000","last_unstale":"2024-06-10T02:52:42.994408+0000","last_undegraded":"2024-06-10T02:52:42.994408+0000","last_fullsized":"2024-06-10T02:52:42.994408+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T13:12:40.925448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.365243+0000","last_change":"2024-06-10T02:50:12.236317+0000","last_active":"2024-06-10T02:51:46.365243+0000","last_peered":"2024-06-10T02:51:46.365243+0000","last_clean":"2024-06-10T02:51:46.365243+0000","last_became_active":"2024-06-10T02:50:12.235806+0000","last_became_peered":"2024-06-10T02:50:12.235806+0000","last_unstale":"2024-06-10T02:51:46.365243+0000","last_undegraded":"2024-06-10T02:51:46.365243+0000","last_fullsized":"2024-06-10T02:51:46.365243+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:29:21.425987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"98'5","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.383944+0000","last_change":"2024-06-10T02:49:58.994149+0000","last_active":"2024-06-10T02:51:46.383944+0000","last_peered":"2024-06-10T02:51:46.383944+0000","last_clean":"2024-06-10T02:51:46.383944+0000","last_became_active":"2024-06-10T02:49:58.993775+0000","last_became_peered":"2024-06-10T02:49:58.993775+0000","last_unstale":"2024-06-10T02:51:46.383944+0000","last_undegraded":"2024-06-10T02:51:46.383944+0000","last_fullsized":"2024-06-10T02:51:46.383944+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:23:55.553943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"100'15","reported_seq":54,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.995273+0000","last_change":"2024-06-10T02:50:02.199979+0000","last_active":"2024-06-10T02:52:42.995273+0000","last_peered":"2024-06-10T02:52:42.995273+0000","last_clean":"2024-06-10T02:52:42.995273+0000","last_became_active":"2024-06-10T02:50:02.199596+0000","last_became_peered":"2024-06-10T02:50:02.199596+0000","last_unstale":"2024-06-10T02:52:42.995273+0000","last_undegraded":"2024-06-10T02:52:42.995273+0000","last_fullsized":"2024-06-10T02:52:42.995273+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:13:58.647460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.383103+0000","last_change":"2024-06-10T02:50:12.235253+0000","last_active":"2024-06-10T02:51:46.383103+0000","last_peered":"2024-06-10T02:51:46.383103+0000","last_clean":"2024-06-10T02:51:46.383103+0000","last_became_active":"2024-06-10T02:50:12.234773+0000","last_became_peered":"2024-06-10T02:50:12.234773+0000","last_unstale":"2024-06-10T02:51:46.383103+0000","last_undegraded":"2024-06-10T02:51:46.383103+0000","last_fullsized":"2024-06-10T02:51:46.383103+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:38:10.971411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"100'7","reported_seq":238,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:54:09.893481+0000","last_change":"2024-06-10T02:49:58.996912+0000","last_active":"2024-06-10T02:54:09.893481+0000","last_peered":"2024-06-10T02:54:09.893481+0000","last_clean":"2024-06-10T02:54:09.893481+0000","last_became_active":"2024-06-10T02:49:58.996549+0000","last_became_peered":"2024-06-10T02:49:58.996549+0000","last_unstale":"2024-06-10T02:54:09.893481+0000","last_undegraded":"2024-06-10T02:54:09.893481+0000","last_fullsized":"2024-06-10T02:54:09.893481+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:07:52.757676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1119,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":187,"num_read_kb":5,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"100'6","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.995546+0000","last_change":"2024-06-10T02:50:44.741186+0000","last_active":"2024-06-10T02:52:42.995546+0000","last_peered":"2024-06-10T02:52:42.995546+0000","last_clean":"2024-06-10T02:52:42.995546+0000","last_became_active":"2024-06-10T02:50:44.740782+0000","last_became_peered":"2024-06-10T02:50:44.740782+0000","last_unstale":"2024-06-10T02:52:42.995546+0000","last_undegraded":"2024-06-10T02:52:42.995546+0000","last_fullsized":"2024-06-10T02:52:42.995546+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:46:03.863243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.715228+0000","last_change":"2024-06-10T02:50:12.236505+0000","last_active":"2024-06-10T02:51:45.715228+0000","last_peered":"2024-06-10T02:51:45.715228+0000","last_clean":"2024-06-10T02:51:45.715228+0000","last_became_active":"2024-06-10T02:50:12.236112+0000","last_became_peered":"2024-06-10T02:50:12.236112+0000","last_unstale":"2024-06-10T02:51:45.715228+0000","last_undegraded":"2024-06-10T02:51:45.715228+0000","last_fullsized":"2024-06-10T02:51:45.715228+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:13:38.586736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"98'17","reported_seq":63,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.723449+0000","last_change":"2024-06-10T02:49:59.031965+0000","last_active":"2024-06-10T02:51:45.723449+0000","last_peered":"2024-06-10T02:51:45.723449+0000","last_clean":"2024-06-10T02:51:45.723449+0000","last_became_active":"2024-06-10T02:49:59.031495+0000","last_became_peered":"2024-06-10T02:49:59.031495+0000","last_unstale":"2024-06-10T02:51:45.723449+0000","last_undegraded":"2024-06-10T02:51:45.723449+0000","last_fullsized":"2024-06-10T02:51:45.723449+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T14:21:06.610677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"100'6","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:43.002009+0000","last_change":"2024-06-10T02:50:02.209926+0000","last_active":"2024-06-10T02:52:43.002009+0000","last_peered":"2024-06-10T02:52:43.002009+0000","last_clean":"2024-06-10T02:52:43.002009+0000","last_became_active":"2024-06-10T02:50:02.209540+0000","last_became_peered":"2024-06-10T02:50:02.209540+0000","last_unstale":"2024-06-10T02:52:43.002009+0000","last_undegraded":"2024-06-10T02:52:43.002009+0000","last_fullsized":"2024-06-10T02:52:43.002009+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:49:40.497692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:48.927934+0000","last_change":"2024-06-10T02:51:46.013156+0000","last_active":"2024-06-10T02:51:48.927934+0000","last_peered":"2024-06-10T02:51:48.927934+0000","last_clean":"2024-06-10T02:51:48.927934+0000","last_became_active":"2024-06-10T02:51:46.012769+0000","last_became_peered":"2024-06-10T02:51:46.012769+0000","last_unstale":"2024-06-10T02:51:48.927934+0000","last_undegraded":"2024-06-10T02:51:48.927934+0000","last_fullsized":"2024-06-10T02:51:48.927934+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:44:49.545676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,3,1],"acting":[8,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.f","version":"98'2","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.724214+0000","last_change":"2024-06-10T02:49:58.998372+0000","last_active":"2024-06-10T02:51:45.724214+0000","last_peered":"2024-06-10T02:51:45.724214+0000","last_clean":"2024-06-10T02:51:45.724214+0000","last_became_active":"2024-06-10T02:49:58.998058+0000","last_became_peered":"2024-06-10T02:49:58.998058+0000","last_unstale":"2024-06-10T02:51:45.724214+0000","last_undegraded":"2024-06-10T02:51:45.724214+0000","last_fullsized":"2024-06-10T02:51:45.724214+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:42:50.509851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"100'5","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.993794+0000","last_change":"2024-06-10T02:50:02.209256+0000","last_active":"2024-06-10T02:52:42.993794+0000","last_peered":"2024-06-10T02:52:42.993794+0000","last_clean":"2024-06-10T02:52:42.993794+0000","last_became_active":"2024-06-10T02:50:02.207727+0000","last_became_peered":"2024-06-10T02:50:02.207727+0000","last_unstale":"2024-06-10T02:52:42.993794+0000","last_undegraded":"2024-06-10T02:52:42.993794+0000","last_fullsized":"2024-06-10T02:52:42.993794+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:23:00.175943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.723666+0000","last_change":"2024-06-10T02:50:12.729438+0000","last_active":"2024-06-10T02:51:45.723666+0000","last_peered":"2024-06-10T02:51:45.723666+0000","last_clean":"2024-06-10T02:51:45.723666+0000","last_became_active":"2024-06-10T02:50:12.728918+0000","last_became_peered":"2024-06-10T02:50:12.728918+0000","last_unstale":"2024-06-10T02:51:45.723666+0000","last_undegraded":"2024-06-10T02:51:45.723666+0000","last_fullsized":"2024-06-10T02:51:45.723666+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:04:48.861763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"98'1","reported_seq":53,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:05.628603+0000","last_change":"2024-06-10T02:50:44.782241+0000","last_active":"2024-06-10T02:52:05.628603+0000","last_peered":"2024-06-10T02:52:05.628603+0000","last_clean":"2024-06-10T02:52:05.628603+0000","last_became_active":"2024-06-10T02:50:44.781799+0000","last_became_peered":"2024-06-10T02:50:44.781799+0000","last_unstale":"2024-06-10T02:52:05.628603+0000","last_undegraded":"2024-06-10T02:52:05.628603+0000","last_fullsized":"2024-06-10T02:52:05.628603+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:38:35.726258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"100'4","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.993135+0000","last_change":"2024-06-10T02:50:02.198888+0000","last_active":"2024-06-10T02:52:42.993135+0000","last_peered":"2024-06-10T02:52:42.993135+0000","last_clean":"2024-06-10T02:52:42.993135+0000","last_became_active":"2024-06-10T02:50:02.198346+0000","last_became_peered":"2024-06-10T02:50:02.198346+0000","last_unstale":"2024-06-10T02:52:42.993135+0000","last_undegraded":"2024-06-10T02:52:42.993135+0000","last_fullsized":"2024-06-10T02:52:42.993135+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:15:42.422898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.715171+0000","last_change":"2024-06-10T02:50:12.232763+0000","last_active":"2024-06-10T02:51:45.715171+0000","last_peered":"2024-06-10T02:51:45.715171+0000","last_clean":"2024-06-10T02:51:45.715171+0000","last_became_active":"2024-06-10T02:50:12.232390+0000","last_became_peered":"2024-06-10T02:50:12.232390+0000","last_unstale":"2024-06-10T02:51:45.715171+0000","last_undegraded":"2024-06-10T02:51:45.715171+0000","last_fullsized":"2024-06-10T02:51:45.715171+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:31:47.973500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"100'8","reported_seq":241,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:54:11.896674+0000","last_change":"2024-06-10T02:49:58.994308+0000","last_active":"2024-06-10T02:54:11.896674+0000","last_peered":"2024-06-10T02:54:11.896674+0000","last_clean":"2024-06-10T02:54:11.896674+0000","last_became_active":"2024-06-10T02:49:58.993494+0000","last_became_peered":"2024-06-10T02:49:58.993494+0000","last_unstale":"2024-06-10T02:54:11.896674+0000","last_undegraded":"2024-06-10T02:54:11.896674+0000","last_fullsized":"2024-06-10T02:54:11.896674+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:07:28.452276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1509,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":187,"num_read_kb":4,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"100'7","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.988698+0000","last_change":"2024-06-10T02:50:02.206872+0000","last_active":"2024-06-10T02:52:42.988698+0000","last_peered":"2024-06-10T02:52:42.988698+0000","last_clean":"2024-06-10T02:52:42.988698+0000","last_became_active":"2024-06-10T02:50:02.206307+0000","last_became_peered":"2024-06-10T02:50:02.206307+0000","last_unstale":"2024-06-10T02:52:42.988698+0000","last_undegraded":"2024-06-10T02:52:42.988698+0000","last_fullsized":"2024-06-10T02:52:42.988698+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:01:04.645674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.715107+0000","last_change":"2024-06-10T02:50:12.729392+0000","last_active":"2024-06-10T02:51:45.715107+0000","last_peered":"2024-06-10T02:51:45.715107+0000","last_clean":"2024-06-10T02:51:45.715107+0000","last_became_active":"2024-06-10T02:50:12.728866+0000","last_became_peered":"2024-06-10T02:50:12.728866+0000","last_unstale":"2024-06-10T02:51:45.715107+0000","last_undegraded":"2024-06-10T02:51:45.715107+0000","last_fullsized":"2024-06-10T02:51:45.715107+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:01:35.845184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"100'8","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.995032+0000","last_change":"2024-06-10T02:50:02.202303+0000","last_active":"2024-06-10T02:52:42.995032+0000","last_peered":"2024-06-10T02:52:42.995032+0000","last_clean":"2024-06-10T02:52:42.995032+0000","last_became_active":"2024-06-10T02:50:02.201998+0000","last_became_peered":"2024-06-10T02:50:02.201998+0000","last_unstale":"2024-06-10T02:52:42.995032+0000","last_undegraded":"2024-06-10T02:52:42.995032+0000","last_fullsized":"2024-06-10T02:52:42.995032+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:15:00.005693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"100'6","reported_seq":236,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:54:11.895869+0000","last_change":"2024-06-10T02:49:59.031513+0000","last_active":"2024-06-10T02:54:11.895869+0000","last_peered":"2024-06-10T02:54:11.895869+0000","last_clean":"2024-06-10T02:54:11.895869+0000","last_became_active":"2024-06-10T02:49:59.030965+0000","last_became_peered":"2024-06-10T02:49:59.030965+0000","last_unstale":"2024-06-10T02:54:11.895869+0000","last_undegraded":"2024-06-10T02:54:11.895869+0000","last_fullsized":"2024-06-10T02:54:11.895869+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:49:02.171578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":184,"num_read_kb":184,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"98'1","reported_seq":30,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.295434+0000","last_change":"2024-06-10T02:50:12.236334+0000","last_active":"2024-06-10T02:51:46.295434+0000","last_peered":"2024-06-10T02:51:46.295434+0000","last_clean":"2024-06-10T02:51:46.295434+0000","last_became_active":"2024-06-10T02:50:12.235820+0000","last_became_peered":"2024-06-10T02:50:12.235820+0000","last_unstale":"2024-06-10T02:51:46.295434+0000","last_undegraded":"2024-06-10T02:51:46.295434+0000","last_fullsized":"2024-06-10T02:51:46.295434+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:16:12.476881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"100'8","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:43.001311+0000","last_change":"2024-06-10T02:50:02.205146+0000","last_active":"2024-06-10T02:52:43.001311+0000","last_peered":"2024-06-10T02:52:43.001311+0000","last_clean":"2024-06-10T02:52:43.001311+0000","last_became_active":"2024-06-10T02:50:02.204785+0000","last_became_peered":"2024-06-10T02:50:02.204785+0000","last_unstale":"2024-06-10T02:52:43.001311+0000","last_undegraded":"2024-06-10T02:52:43.001311+0000","last_fullsized":"2024-06-10T02:52:43.001311+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T13:01:02.068989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"98'6","reported_seq":55,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:06.016314+0000","last_change":"2024-06-10T02:49:58.997654+0000","last_active":"2024-06-10T02:52:06.016314+0000","last_peered":"2024-06-10T02:52:06.016314+0000","last_clean":"2024-06-10T02:52:06.016314+0000","last_became_active":"2024-06-10T02:49:58.997294+0000","last_became_peered":"2024-06-10T02:49:58.997294+0000","last_unstale":"2024-06-10T02:52:06.016314+0000","last_undegraded":"2024-06-10T02:52:06.016314+0000","last_fullsized":"2024-06-10T02:52:06.016314+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:53:14.763198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":367,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.294476+0000","last_change":"2024-06-10T02:48:34.008201+0000","last_active":"2024-06-10T02:51:46.294476+0000","last_peered":"2024-06-10T02:51:46.294476+0000","last_clean":"2024-06-10T02:51:46.294476+0000","last_became_active":"2024-06-10T02:48:34.007633+0000","last_became_peered":"2024-06-10T02:48:34.007633+0000","last_unstale":"2024-06-10T02:51:46.294476+0000","last_undegraded":"2024-06-10T02:51:46.294476+0000","last_fullsized":"2024-06-10T02:51:46.294476+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:45:07.444823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:45:07.444823+0000","last_clean_scrub_stamp":"2024-06-10T02:45:07.444823+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:56:58.159559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":36,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.383275+0000","last_change":"2024-06-10T02:50:44.782130+0000","last_active":"2024-06-10T02:51:46.383275+0000","last_peered":"2024-06-10T02:51:46.383275+0000","last_clean":"2024-06-10T02:51:46.383275+0000","last_became_active":"2024-06-10T02:50:44.781530+0000","last_became_peered":"2024-06-10T02:50:44.781530+0000","last_unstale":"2024-06-10T02:51:46.383275+0000","last_undegraded":"2024-06-10T02:51:46.383275+0000","last_fullsized":"2024-06-10T02:51:46.383275+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:43:23.826207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,11,6,10],"acting":[5,11,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"100'5","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.994100+0000","last_change":"2024-06-10T02:50:02.213666+0000","last_active":"2024-06-10T02:52:42.994100+0000","last_peered":"2024-06-10T02:52:42.994100+0000","last_clean":"2024-06-10T02:52:42.994100+0000","last_became_active":"2024-06-10T02:50:02.213194+0000","last_became_peered":"2024-06-10T02:50:02.213194+0000","last_unstale":"2024-06-10T02:52:42.994100+0000","last_undegraded":"2024-06-10T02:52:42.994100+0000","last_fullsized":"2024-06-10T02:52:42.994100+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:22:55.905412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"98'4","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:02.798809+0000","last_change":"2024-06-10T02:49:58.997618+0000","last_active":"2024-06-10T02:52:02.798809+0000","last_peered":"2024-06-10T02:52:02.798809+0000","last_clean":"2024-06-10T02:52:02.798809+0000","last_became_active":"2024-06-10T02:49:58.997304+0000","last_became_peered":"2024-06-10T02:49:58.997304+0000","last_unstale":"2024-06-10T02:52:02.798809+0000","last_undegraded":"2024-06-10T02:52:02.798809+0000","last_fullsized":"2024-06-10T02:52:02.798809+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:48:59.520991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.364962+0000","last_change":"2024-06-10T02:50:12.236357+0000","last_active":"2024-06-10T02:51:46.364962+0000","last_peered":"2024-06-10T02:51:46.364962+0000","last_clean":"2024-06-10T02:51:46.364962+0000","last_became_active":"2024-06-10T02:50:12.235915+0000","last_became_peered":"2024-06-10T02:50:12.235915+0000","last_unstale":"2024-06-10T02:51:46.364962+0000","last_undegraded":"2024-06-10T02:51:46.364962+0000","last_fullsized":"2024-06-10T02:51:46.364962+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:20:06.287907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"98'4","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:03.929566+0000","last_change":"2024-06-10T02:49:59.031725+0000","last_active":"2024-06-10T02:52:03.929566+0000","last_peered":"2024-06-10T02:52:03.929566+0000","last_clean":"2024-06-10T02:52:03.929566+0000","last_became_active":"2024-06-10T02:49:59.031296+0000","last_became_peered":"2024-06-10T02:49:59.031296+0000","last_unstale":"2024-06-10T02:52:03.929566+0000","last_undegraded":"2024-06-10T02:52:03.929566+0000","last_fullsized":"2024-06-10T02:52:03.929566+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T13:17:58.571696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"100'9","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.986516+0000","last_change":"2024-06-10T02:50:02.201820+0000","last_active":"2024-06-10T02:52:42.986516+0000","last_peered":"2024-06-10T02:52:42.986516+0000","last_clean":"2024-06-10T02:52:42.986516+0000","last_became_active":"2024-06-10T02:50:02.201478+0000","last_became_peered":"2024-06-10T02:50:02.201478+0000","last_unstale":"2024-06-10T02:52:42.986516+0000","last_undegraded":"2024-06-10T02:52:42.986516+0000","last_fullsized":"2024-06-10T02:52:42.986516+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:57:25.963926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":30,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:12.118458+0000","last_change":"2024-06-10T02:50:12.729418+0000","last_active":"2024-06-10T02:52:12.118458+0000","last_peered":"2024-06-10T02:52:12.118458+0000","last_clean":"2024-06-10T02:52:12.118458+0000","last_became_active":"2024-06-10T02:50:12.728692+0000","last_became_peered":"2024-06-10T02:50:12.728692+0000","last_unstale":"2024-06-10T02:52:12.118458+0000","last_undegraded":"2024-06-10T02:52:12.118458+0000","last_fullsized":"2024-06-10T02:52:12.118458+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:25:00.472396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"100'18","reported_seq":63,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:54:07.234777+0000","last_change":"2024-06-10T02:49:58.996391+0000","last_active":"2024-06-10T02:54:07.234777+0000","last_peered":"2024-06-10T02:54:07.234777+0000","last_clean":"2024-06-10T02:54:07.234777+0000","last_became_active":"2024-06-10T02:49:58.996033+0000","last_became_peered":"2024-06-10T02:49:58.996033+0000","last_unstale":"2024-06-10T02:54:07.234777+0000","last_undegraded":"2024-06-10T02:54:07.234777+0000","last_fullsized":"2024-06-10T02:54:07.234777+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:03:34.568969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":34,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"100'5","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.993669+0000","last_change":"2024-06-10T02:50:02.197974+0000","last_active":"2024-06-10T02:52:42.993669+0000","last_peered":"2024-06-10T02:52:42.993669+0000","last_clean":"2024-06-10T02:52:42.993669+0000","last_became_active":"2024-06-10T02:50:02.197614+0000","last_became_peered":"2024-06-10T02:50:02.197614+0000","last_unstale":"2024-06-10T02:52:42.993669+0000","last_undegraded":"2024-06-10T02:52:42.993669+0000","last_fullsized":"2024-06-10T02:52:42.993669+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:36:55.575808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.721202+0000","last_change":"2024-06-10T02:50:12.234239+0000","last_active":"2024-06-10T02:51:45.721202+0000","last_peered":"2024-06-10T02:51:45.721202+0000","last_clean":"2024-06-10T02:51:45.721202+0000","last_became_active":"2024-06-10T02:50:12.233768+0000","last_became_peered":"2024-06-10T02:50:12.233768+0000","last_unstale":"2024-06-10T02:51:45.721202+0000","last_undegraded":"2024-06-10T02:51:45.721202+0000","last_fullsized":"2024-06-10T02:51:45.721202+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:51:44.230031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"98'7","reported_seq":53,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.383587+0000","last_change":"2024-06-10T02:49:59.192765+0000","last_active":"2024-06-10T02:51:46.383587+0000","last_peered":"2024-06-10T02:51:46.383587+0000","last_clean":"2024-06-10T02:51:46.383587+0000","last_became_active":"2024-06-10T02:49:59.192446+0000","last_became_peered":"2024-06-10T02:49:59.192446+0000","last_unstale":"2024-06-10T02:51:46.383587+0000","last_undegraded":"2024-06-10T02:51:46.383587+0000","last_fullsized":"2024-06-10T02:51:46.383587+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:53:51.862019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"100'7","reported_seq":52,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.989395+0000","last_change":"2024-06-10T02:50:44.781209+0000","last_active":"2024-06-10T02:52:42.989395+0000","last_peered":"2024-06-10T02:52:42.989395+0000","last_clean":"2024-06-10T02:52:42.989395+0000","last_became_active":"2024-06-10T02:50:44.780811+0000","last_became_peered":"2024-06-10T02:50:44.780811+0000","last_unstale":"2024-06-10T02:52:42.989395+0000","last_undegraded":"2024-06-10T02:52:42.989395+0000","last_fullsized":"2024-06-10T02:52:42.989395+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:12:40.663565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.383517+0000","last_change":"2024-06-10T02:50:12.730269+0000","last_active":"2024-06-10T02:51:46.383517+0000","last_peered":"2024-06-10T02:51:46.383517+0000","last_clean":"2024-06-10T02:51:46.383517+0000","last_became_active":"2024-06-10T02:50:12.729574+0000","last_became_peered":"2024-06-10T02:50:12.729574+0000","last_unstale":"2024-06-10T02:51:46.383517+0000","last_undegraded":"2024-06-10T02:51:46.383517+0000","last_fullsized":"2024-06-10T02:51:46.383517+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:51:16.235284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"98'2","reported_seq":53,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.854226+0000","last_change":"2024-06-10T02:50:44.782421+0000","last_active":"2024-06-10T02:51:45.854226+0000","last_peered":"2024-06-10T02:51:45.854226+0000","last_clean":"2024-06-10T02:51:45.854226+0000","last_became_active":"2024-06-10T02:50:44.781881+0000","last_became_peered":"2024-06-10T02:50:44.781881+0000","last_unstale":"2024-06-10T02:51:45.854226+0000","last_undegraded":"2024-06-10T02:51:45.854226+0000","last_fullsized":"2024-06-10T02:51:45.854226+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T14:41:58.771256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"100'2","reported_seq":41,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.952928+0000","last_change":"2024-06-10T02:50:02.197761+0000","last_active":"2024-06-10T02:52:42.952928+0000","last_peered":"2024-06-10T02:52:42.952928+0000","last_clean":"2024-06-10T02:52:42.952928+0000","last_became_active":"2024-06-10T02:50:02.197395+0000","last_became_peered":"2024-06-10T02:50:02.197395+0000","last_unstale":"2024-06-10T02:52:42.952928+0000","last_undegraded":"2024-06-10T02:52:42.952928+0000","last_fullsized":"2024-06-10T02:52:42.952928+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:46:37.537280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:48.927972+0000","last_change":"2024-06-10T02:51:46.385129+0000","last_active":"2024-06-10T02:51:48.927972+0000","last_peered":"2024-06-10T02:51:48.927972+0000","last_clean":"2024-06-10T02:51:48.927972+0000","last_became_active":"2024-06-10T02:51:46.384553+0000","last_became_peered":"2024-06-10T02:51:46.384553+0000","last_unstale":"2024-06-10T02:51:48.927972+0000","last_undegraded":"2024-06-10T02:51:48.927972+0000","last_fullsized":"2024-06-10T02:51:48.927972+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:08:11.183679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"98'7","reported_seq":52,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.831750+0000","last_change":"2024-06-10T02:49:58.987863+0000","last_active":"2024-06-10T02:51:45.831750+0000","last_peered":"2024-06-10T02:51:45.831750+0000","last_clean":"2024-06-10T02:51:45.831750+0000","last_became_active":"2024-06-10T02:49:58.987388+0000","last_became_peered":"2024-06-10T02:49:58.987388+0000","last_unstale":"2024-06-10T02:51:45.831750+0000","last_undegraded":"2024-06-10T02:51:45.831750+0000","last_fullsized":"2024-06-10T02:51:45.831750+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:29:20.340301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"100'9","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.988655+0000","last_change":"2024-06-10T02:50:02.200699+0000","last_active":"2024-06-10T02:52:42.988655+0000","last_peered":"2024-06-10T02:52:42.988655+0000","last_clean":"2024-06-10T02:52:42.988655+0000","last_became_active":"2024-06-10T02:50:02.200278+0000","last_became_peered":"2024-06-10T02:50:02.200278+0000","last_unstale":"2024-06-10T02:52:42.988655+0000","last_undegraded":"2024-06-10T02:52:42.988655+0000","last_fullsized":"2024-06-10T02:52:42.988655+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:26:36.236312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"98'1","reported_seq":30,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.383543+0000","last_change":"2024-06-10T02:50:12.236027+0000","last_active":"2024-06-10T02:51:46.383543+0000","last_peered":"2024-06-10T02:51:46.383543+0000","last_clean":"2024-06-10T02:51:46.383543+0000","last_became_active":"2024-06-10T02:50:12.235629+0000","last_became_peered":"2024-06-10T02:50:12.235629+0000","last_unstale":"2024-06-10T02:51:46.383543+0000","last_undegraded":"2024-06-10T02:51:46.383543+0000","last_fullsized":"2024-06-10T02:51:46.383543+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:58:48.218806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"98'3","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.721449+0000","last_change":"2024-06-10T02:49:58.998496+0000","last_active":"2024-06-10T02:51:45.721449+0000","last_peered":"2024-06-10T02:51:45.721449+0000","last_clean":"2024-06-10T02:51:45.721449+0000","last_became_active":"2024-06-10T02:49:58.997260+0000","last_became_peered":"2024-06-10T02:49:58.997260+0000","last_unstale":"2024-06-10T02:51:45.721449+0000","last_undegraded":"2024-06-10T02:51:45.721449+0000","last_fullsized":"2024-06-10T02:51:45.721449+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T14:22:04.658943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"100'8","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.990375+0000","last_change":"2024-06-10T02:50:02.201300+0000","last_active":"2024-06-10T02:52:42.990375+0000","last_peered":"2024-06-10T02:52:42.990375+0000","last_clean":"2024-06-10T02:52:42.990375+0000","last_became_active":"2024-06-10T02:50:02.200988+0000","last_became_peered":"2024-06-10T02:50:02.200988+0000","last_unstale":"2024-06-10T02:52:42.990375+0000","last_undegraded":"2024-06-10T02:52:42.990375+0000","last_fullsized":"2024-06-10T02:52:42.990375+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:11:12.962048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.383484+0000","last_change":"2024-06-10T02:50:12.728869+0000","last_active":"2024-06-10T02:51:46.383484+0000","last_peered":"2024-06-10T02:51:46.383484+0000","last_clean":"2024-06-10T02:51:46.383484+0000","last_became_active":"2024-06-10T02:50:12.728189+0000","last_became_peered":"2024-06-10T02:50:12.728189+0000","last_unstale":"2024-06-10T02:51:46.383484+0000","last_undegraded":"2024-06-10T02:51:46.383484+0000","last_fullsized":"2024-06-10T02:51:46.383484+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:08:36.624814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"98'4","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.723488+0000","last_change":"2024-06-10T02:49:59.000460+0000","last_active":"2024-06-10T02:51:45.723488+0000","last_peered":"2024-06-10T02:51:45.723488+0000","last_clean":"2024-06-10T02:51:45.723488+0000","last_became_active":"2024-06-10T02:49:59.000075+0000","last_became_peered":"2024-06-10T02:49:59.000075+0000","last_unstale":"2024-06-10T02:51:45.723488+0000","last_undegraded":"2024-06-10T02:51:45.723488+0000","last_fullsized":"2024-06-10T02:51:45.723488+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:06:27.935745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"100'5","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:43.001075+0000","last_change":"2024-06-10T02:50:02.210131+0000","last_active":"2024-06-10T02:52:43.001075+0000","last_peered":"2024-06-10T02:52:43.001075+0000","last_clean":"2024-06-10T02:52:43.001075+0000","last_became_active":"2024-06-10T02:50:02.209781+0000","last_became_peered":"2024-06-10T02:50:02.209781+0000","last_unstale":"2024-06-10T02:52:43.001075+0000","last_undegraded":"2024-06-10T02:52:43.001075+0000","last_fullsized":"2024-06-10T02:52:43.001075+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:27:39.966620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.854407+0000","last_change":"2024-06-10T02:50:12.236017+0000","last_active":"2024-06-10T02:51:45.854407+0000","last_peered":"2024-06-10T02:51:45.854407+0000","last_clean":"2024-06-10T02:51:45.854407+0000","last_became_active":"2024-06-10T02:50:12.235553+0000","last_became_peered":"2024-06-10T02:50:12.235553+0000","last_unstale":"2024-06-10T02:51:45.854407+0000","last_undegraded":"2024-06-10T02:51:45.854407+0000","last_fullsized":"2024-06-10T02:51:45.854407+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:23:55.437994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"100'7","reported_seq":238,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:54:09.894120+0000","last_change":"2024-06-10T02:49:59.192864+0000","last_active":"2024-06-10T02:54:09.894120+0000","last_peered":"2024-06-10T02:54:09.894120+0000","last_clean":"2024-06-10T02:54:09.894120+0000","last_became_active":"2024-06-10T02:49:59.192440+0000","last_became_peered":"2024-06-10T02:49:59.192440+0000","last_unstale":"2024-06-10T02:54:09.894120+0000","last_undegraded":"2024-06-10T02:54:09.894120+0000","last_fullsized":"2024-06-10T02:54:09.894120+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:36:05.868538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1701,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":186,"num_read_kb":4,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"100'7","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:43.000525+0000","last_change":"2024-06-10T02:50:02.205791+0000","last_active":"2024-06-10T02:52:43.000525+0000","last_peered":"2024-06-10T02:52:43.000525+0000","last_clean":"2024-06-10T02:52:43.000525+0000","last_became_active":"2024-06-10T02:50:02.205483+0000","last_became_peered":"2024-06-10T02:50:02.205483+0000","last_unstale":"2024-06-10T02:52:43.000525+0000","last_undegraded":"2024-06-10T02:52:43.000525+0000","last_fullsized":"2024-06-10T02:52:43.000525+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:33:25.875423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.011597+0000","last_change":"2024-06-10T02:50:12.239294+0000","last_active":"2024-06-10T02:51:46.011597+0000","last_peered":"2024-06-10T02:51:46.011597+0000","last_clean":"2024-06-10T02:51:46.011597+0000","last_became_active":"2024-06-10T02:50:12.238778+0000","last_became_peered":"2024-06-10T02:50:12.238778+0000","last_unstale":"2024-06-10T02:51:46.011597+0000","last_undegraded":"2024-06-10T02:51:46.011597+0000","last_fullsized":"2024-06-10T02:51:46.011597+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:26:09.203623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"98'3","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.383589+0000","last_change":"2024-06-10T02:49:58.989680+0000","last_active":"2024-06-10T02:51:46.383589+0000","last_peered":"2024-06-10T02:51:46.383589+0000","last_clean":"2024-06-10T02:51:46.383589+0000","last_became_active":"2024-06-10T02:49:58.989265+0000","last_became_peered":"2024-06-10T02:49:58.989265+0000","last_unstale":"2024-06-10T02:51:46.383589+0000","last_undegraded":"2024-06-10T02:51:46.383589+0000","last_fullsized":"2024-06-10T02:51:46.383589+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:36:34.370177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"100'6","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.990330+0000","last_change":"2024-06-10T02:50:44.743369+0000","last_active":"2024-06-10T02:52:42.990330+0000","last_peered":"2024-06-10T02:52:42.990330+0000","last_clean":"2024-06-10T02:52:42.990330+0000","last_became_active":"2024-06-10T02:50:44.743013+0000","last_became_peered":"2024-06-10T02:50:44.743013+0000","last_unstale":"2024-06-10T02:52:42.990330+0000","last_undegraded":"2024-06-10T02:52:42.990330+0000","last_fullsized":"2024-06-10T02:52:42.990330+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:54:43.728943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.382656+0000","last_change":"2024-06-10T02:50:12.235197+0000","last_active":"2024-06-10T02:51:46.382656+0000","last_peered":"2024-06-10T02:51:46.382656+0000","last_clean":"2024-06-10T02:51:46.382656+0000","last_became_active":"2024-06-10T02:50:12.234791+0000","last_became_peered":"2024-06-10T02:50:12.234791+0000","last_unstale":"2024-06-10T02:51:46.382656+0000","last_undegraded":"2024-06-10T02:51:46.382656+0000","last_fullsized":"2024-06-10T02:51:46.382656+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:09:11.164236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"98'5","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.715164+0000","last_change":"2024-06-10T02:49:58.996759+0000","last_active":"2024-06-10T02:51:45.715164+0000","last_peered":"2024-06-10T02:51:45.715164+0000","last_clean":"2024-06-10T02:51:45.715164+0000","last_became_active":"2024-06-10T02:49:58.996434+0000","last_became_peered":"2024-06-10T02:49:58.996434+0000","last_unstale":"2024-06-10T02:51:45.715164+0000","last_undegraded":"2024-06-10T02:51:45.715164+0000","last_fullsized":"2024-06-10T02:51:45.715164+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:09:35.327797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"100'9","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.996813+0000","last_change":"2024-06-10T02:50:02.201676+0000","last_active":"2024-06-10T02:52:42.996813+0000","last_peered":"2024-06-10T02:52:42.996813+0000","last_clean":"2024-06-10T02:52:42.996813+0000","last_became_active":"2024-06-10T02:50:02.201335+0000","last_became_peered":"2024-06-10T02:50:02.201335+0000","last_unstale":"2024-06-10T02:52:42.996813+0000","last_undegraded":"2024-06-10T02:52:42.996813+0000","last_fullsized":"2024-06-10T02:52:42.996813+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:49:47.038946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"100'9","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.992923+0000","last_change":"2024-06-10T02:50:02.203462+0000","last_active":"2024-06-10T02:52:42.992923+0000","last_peered":"2024-06-10T02:52:42.992923+0000","last_clean":"2024-06-10T02:52:42.992923+0000","last_became_active":"2024-06-10T02:50:02.203115+0000","last_became_peered":"2024-06-10T02:50:02.203115+0000","last_unstale":"2024-06-10T02:52:42.992923+0000","last_undegraded":"2024-06-10T02:52:42.992923+0000","last_fullsized":"2024-06-10T02:52:42.992923+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T13:18:12.400596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"100'12","reported_seq":52,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.998015+0000","last_change":"2024-06-10T02:50:02.205648+0000","last_active":"2024-06-10T02:52:42.998015+0000","last_peered":"2024-06-10T02:52:42.998015+0000","last_clean":"2024-06-10T02:52:42.998015+0000","last_became_active":"2024-06-10T02:50:02.205306+0000","last_became_peered":"2024-06-10T02:50:02.205306+0000","last_unstale":"2024-06-10T02:52:42.998015+0000","last_undegraded":"2024-06-10T02:52:42.998015+0000","last_fullsized":"2024-06-10T02:52:42.998015+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:22:55.020644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"100'8","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.991617+0000","last_change":"2024-06-10T02:50:02.198601+0000","last_active":"2024-06-10T02:52:42.991617+0000","last_peered":"2024-06-10T02:52:42.991617+0000","last_clean":"2024-06-10T02:52:42.991617+0000","last_became_active":"2024-06-10T02:50:02.198103+0000","last_became_peered":"2024-06-10T02:50:02.198103+0000","last_unstale":"2024-06-10T02:52:42.991617+0000","last_undegraded":"2024-06-10T02:52:42.991617+0000","last_fullsized":"2024-06-10T02:52:42.991617+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:56:28.033482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"100'7","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.985332+0000","last_change":"2024-06-10T02:50:02.202638+0000","last_active":"2024-06-10T02:52:42.985332+0000","last_peered":"2024-06-10T02:52:42.985332+0000","last_clean":"2024-06-10T02:52:42.985332+0000","last_became_active":"2024-06-10T02:50:02.202249+0000","last_became_peered":"2024-06-10T02:50:02.202249+0000","last_unstale":"2024-06-10T02:52:42.985332+0000","last_undegraded":"2024-06-10T02:52:42.985332+0000","last_fullsized":"2024-06-10T02:52:42.985332+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:46:29.424164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"100'4","reported_seq":43,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.992649+0000","last_change":"2024-06-10T02:50:02.213526+0000","last_active":"2024-06-10T02:52:42.992649+0000","last_peered":"2024-06-10T02:52:42.992649+0000","last_clean":"2024-06-10T02:52:42.992649+0000","last_became_active":"2024-06-10T02:50:02.213226+0000","last_became_peered":"2024-06-10T02:50:02.213226+0000","last_unstale":"2024-06-10T02:52:42.992649+0000","last_undegraded":"2024-06-10T02:52:42.992649+0000","last_fullsized":"2024-06-10T02:52:42.992649+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:01:56.846340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"100'6","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.993061+0000","last_change":"2024-06-10T02:50:02.206583+0000","last_active":"2024-06-10T02:52:42.993061+0000","last_peered":"2024-06-10T02:52:42.993061+0000","last_clean":"2024-06-10T02:52:42.993061+0000","last_became_active":"2024-06-10T02:50:02.206227+0000","last_became_peered":"2024-06-10T02:50:02.206227+0000","last_unstale":"2024-06-10T02:52:42.993061+0000","last_undegraded":"2024-06-10T02:52:42.993061+0000","last_fullsized":"2024-06-10T02:52:42.993061+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:25:07.246288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"100'15","reported_seq":54,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:43.002970+0000","last_change":"2024-06-10T02:50:02.206348+0000","last_active":"2024-06-10T02:52:43.002970+0000","last_peered":"2024-06-10T02:52:43.002970+0000","last_clean":"2024-06-10T02:52:43.002970+0000","last_became_active":"2024-06-10T02:50:02.206020+0000","last_became_peered":"2024-06-10T02:50:02.206020+0000","last_unstale":"2024-06-10T02:52:43.002970+0000","last_undegraded":"2024-06-10T02:52:43.002970+0000","last_fullsized":"2024-06-10T02:52:43.002970+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:02:26.619113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"100'2","reported_seq":41,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.993521+0000","last_change":"2024-06-10T02:50:02.199405+0000","last_active":"2024-06-10T02:52:42.993521+0000","last_peered":"2024-06-10T02:52:42.993521+0000","last_clean":"2024-06-10T02:52:42.993521+0000","last_became_active":"2024-06-10T02:50:02.199049+0000","last_became_peered":"2024-06-10T02:50:02.199049+0000","last_unstale":"2024-06-10T02:52:42.993521+0000","last_undegraded":"2024-06-10T02:52:42.993521+0000","last_fullsized":"2024-06-10T02:52:42.993521+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:33:04.091160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"100'6","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.992703+0000","last_change":"2024-06-10T02:50:02.202094+0000","last_active":"2024-06-10T02:52:42.992703+0000","last_peered":"2024-06-10T02:52:42.992703+0000","last_clean":"2024-06-10T02:52:42.992703+0000","last_became_active":"2024-06-10T02:50:02.201751+0000","last_became_peered":"2024-06-10T02:50:02.201751+0000","last_unstale":"2024-06-10T02:52:42.992703+0000","last_undegraded":"2024-06-10T02:52:42.992703+0000","last_fullsized":"2024-06-10T02:52:42.992703+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:52:53.946326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"100'8","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.993442+0000","last_change":"2024-06-10T02:50:02.199630+0000","last_active":"2024-06-10T02:52:42.993442+0000","last_peered":"2024-06-10T02:52:42.993442+0000","last_clean":"2024-06-10T02:52:42.993442+0000","last_became_active":"2024-06-10T02:50:02.199320+0000","last_became_peered":"2024-06-10T02:50:02.199320+0000","last_unstale":"2024-06-10T02:52:42.993442+0000","last_undegraded":"2024-06-10T02:52:42.993442+0000","last_fullsized":"2024-06-10T02:52:42.993442+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:17:28.761862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"100'10","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:43.000631+0000","last_change":"2024-06-10T02:50:02.201190+0000","last_active":"2024-06-10T02:52:43.000631+0000","last_peered":"2024-06-10T02:52:43.000631+0000","last_clean":"2024-06-10T02:52:43.000631+0000","last_became_active":"2024-06-10T02:50:02.200574+0000","last_became_peered":"2024-06-10T02:50:02.200574+0000","last_unstale":"2024-06-10T02:52:43.000631+0000","last_undegraded":"2024-06-10T02:52:43.000631+0000","last_fullsized":"2024-06-10T02:52:43.000631+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:02:07.636218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"100'11","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.992436+0000","last_change":"2024-06-10T02:50:02.202374+0000","last_active":"2024-06-10T02:52:42.992436+0000","last_peered":"2024-06-10T02:52:42.992436+0000","last_clean":"2024-06-10T02:52:42.992436+0000","last_became_active":"2024-06-10T02:50:02.201974+0000","last_became_peered":"2024-06-10T02:50:02.201974+0000","last_unstale":"2024-06-10T02:52:42.992436+0000","last_undegraded":"2024-06-10T02:52:42.992436+0000","last_fullsized":"2024-06-10T02:52:42.992436+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:45:34.977926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"100'8","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.995177+0000","last_change":"2024-06-10T02:50:02.209332+0000","last_active":"2024-06-10T02:52:42.995177+0000","last_peered":"2024-06-10T02:52:42.995177+0000","last_clean":"2024-06-10T02:52:42.995177+0000","last_became_active":"2024-06-10T02:50:02.208175+0000","last_became_peered":"2024-06-10T02:50:02.208175+0000","last_unstale":"2024-06-10T02:52:42.995177+0000","last_undegraded":"2024-06-10T02:52:42.995177+0000","last_fullsized":"2024-06-10T02:52:42.995177+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:44:40.857918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":284028,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":80739,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1873,"num_read_kb":1024,"num_write":276,"num_write_kb":173,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":602112,"data_stored":242217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":237,"ondisk_log_size":237,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351043,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32260,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712892,"statfs":{"total":95995035648,"available":95962001408,"internally_reserved":0,"allocated":1110016,"data_stored":574044,"data_compressed":32447,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612680,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32412,"kb_used_data":1236,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712740,"statfs":{"total":95995035648,"available":95961845760,"internally_reserved":0,"allocated":1265664,"data_stored":1229835,"data_compressed":68564,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1599,"internal_metadata":31914433},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776205,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36448,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708704,"statfs":{"total":95995035648,"available":95957712896,"internally_reserved":0,"allocated":1138688,"data_stored":577539,"data_compressed":32447,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037841,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28112,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717040,"statfs":{"total":95995035648,"available":95966248960,"internally_reserved":0,"allocated":1122304,"data_stored":573149,"data_compressed":32447,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299479,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28172,"kb_used_data":1156,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716980,"statfs":{"total":95995035648,"available":95966187520,"internally_reserved":0,"allocated":1183744,"data_stored":1172218,"data_compressed":68564,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863463005,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36420,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708732,"statfs":{"total":95995035648,"available":95957741568,"internally_reserved":0,"allocated":1110016,"data_stored":569868,"data_compressed":32447,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574966,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36496,"kb_used_data":1160,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708656,"statfs":{"total":95995035648,"available":95957663744,"internally_reserved":0,"allocated":1187840,"data_stored":642393,"data_compressed":32447,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738490,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28124,"kb_used_data":1108,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717028,"statfs":{"total":95995035648,"available":95966236672,"internally_reserved":0,"allocated":1134592,"data_stored":570934,"data_compressed":32447,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411441,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28192,"kb_used_data":1176,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716960,"statfs":{"total":95995035648,"available":95966167040,"internally_reserved":0,"allocated":1204224,"data_stored":640542,"data_compressed":32447,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149803,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36432,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708720,"statfs":{"total":95995035648,"available":95957729280,"internally_reserved":0,"allocated":1122304,"data_stored":568655,"data_compressed":32447,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888166,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28116,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717036,"statfs":{"total":95995035648,"available":95966244864,"internally_reserved":0,"allocated":1126400,"data_stored":574224,"data_compressed":32447,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724641,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28180,"kb_used_data":1164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716972,"statfs":{"total":95995035648,"available":95966179328,"internally_reserved":0,"allocated":1191936,"data_stored":1166348,"data_compressed":68564,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34481,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34481,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34481,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36117,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36117,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36117,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36117,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36117,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":74168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":74837,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1903,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":126976,"data_stored":73762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-06-10T02:54:13.308 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-06-10T02:54:13.308 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-06-10T02:54:13.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:13.448+0000 7fe380dee640 1 -- 172.21.15.52:0/3747563887 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe37c107740 msgr2=0x7fe37c109b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:54:13.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:13.448+0000 7fe380dee640 1 --2- 172.21.15.52:0/3747563887 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe37c107740 0x7fe37c109b30 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fe36c009ca0 tx=0x7fe36c02a8c0 comp rx=0 tx=0).stop 2024-06-10T02:54:13.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:13.449+0000 7fe380dee640 1 -- 172.21.15.52:0/3747563887 shutdown_connections 2024-06-10T02:54:13.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:13.449+0000 7fe380dee640 1 --2- 172.21.15.52:0/3747563887 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe37c107740 0x7fe37c109b30 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:13.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:13.449+0000 7fe380dee640 1 --2- 172.21.15.52:0/3747563887 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe37c104e10 0x7fe37c107200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:13.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:13.449+0000 7fe380dee640 1 --2- 172.21.15.52:0/3747563887 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe37c1024e0 0x7fe37c1048d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:13.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:13.449+0000 7fe380dee640 1 -- 172.21.15.52:0/3747563887 >> 172.21.15.52:0/3747563887 conn(0x7fe37c0fbe50 msgr2=0x7fe37c0fe290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:54:13.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:13.449+0000 7fe380dee640 1 -- 172.21.15.52:0/3747563887 shutdown_connections 2024-06-10T02:54:13.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:13.449+0000 7fe380dee640 1 -- 172.21.15.52:0/3747563887 wait complete. 2024-06-10T02:54:13.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:13.449+0000 7fe380dee640 1 Processor -- start 2024-06-10T02:54:13.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:13.449+0000 7fe380dee640 1 -- start start 2024-06-10T02:54:13.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:13.450+0000 7fe380dee640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe37c1024e0 0x7fe37c197c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:13.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:13.450+0000 7fe380dee640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe37c104e10 0x7fe37c198180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:13.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:13.450+0000 7fe380dee640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe37c107740 0x7fe37c19f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:13.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:13.450+0000 7fe380dee640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fe37c10d660 con 0x7fe37c1024e0 2024-06-10T02:54:13.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:13.450+0000 7fe380dee640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe37c10d4e0 con 0x7fe37c107740 2024-06-10T02:54:13.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:13.450+0000 7fe380dee640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fe37c10d7e0 con 0x7fe37c104e10 2024-06-10T02:54:13.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:13.450+0000 7fe37a59c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe37c1024e0 0x7fe37c197c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:13.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:13.450+0000 7fe37a59c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe37c1024e0 0x7fe37c197c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60078/0 (socket says 172.21.15.52:60078) 2024-06-10T02:54:13.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:13.450+0000 7fe37a59c640 1 -- 172.21.15.52:0/415368314 learned_addr learned my addr 172.21.15.52:0/415368314 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:54:13.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:13.450+0000 7fe37a59c640 1 -- 172.21.15.52:0/415368314 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe37c104e10 msgr2=0x7fe37c198180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:54:13.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:13.450+0000 7fe37ad9d640 1 --2- 172.21.15.52:0/415368314 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe37c107740 0x7fe37c19f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:13.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:13.450+0000 7fe379d9b640 1 --2- 172.21.15.52:0/415368314 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe37c104e10 0x7fe37c198180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:13.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:13.450+0000 7fe37a59c640 1 --2- 172.21.15.52:0/415368314 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe37c104e10 0x7fe37c198180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:13.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:13.450+0000 7fe37a59c640 1 -- 172.21.15.52:0/415368314 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe37c107740 msgr2=0x7fe37c19f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:54:13.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:13.450+0000 7fe37a59c640 1 --2- 172.21.15.52:0/415368314 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe37c107740 0x7fe37c19f1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:13.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:13.450+0000 7fe37a59c640 1 -- 172.21.15.52:0/415368314 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe37c19f850 con 0x7fe37c1024e0 2024-06-10T02:54:13.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:13.450+0000 7fe379d9b640 1 --2- 172.21.15.52:0/415368314 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe37c104e10 0x7fe37c198180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T02:54:13.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:13.450+0000 7fe37ad9d640 1 --2- 172.21.15.52:0/415368314 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe37c107740 0x7fe37c19f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T02:54:13.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:13.450+0000 7fe37a59c640 1 --2- 172.21.15.52:0/415368314 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe37c1024e0 0x7fe37c197c40 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fe37400c890 tx=0x7fe37400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:54:13.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:13.450+0000 7fe3737fe640 1 -- 172.21.15.52:0/415368314 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe374004220 con 0x7fe37c1024e0 2024-06-10T02:54:13.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:13.450+0000 7fe380dee640 1 -- 172.21.15.52:0/415368314 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe37c19fb40 con 0x7fe37c1024e0 2024-06-10T02:54:13.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:13.450+0000 7fe3737fe640 1 -- 172.21.15.52:0/415368314 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe3740043c0 con 0x7fe37c1024e0 2024-06-10T02:54:13.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:13.450+0000 7fe3737fe640 1 -- 172.21.15.52:0/415368314 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe374004a00 con 0x7fe37c1024e0 2024-06-10T02:54:13.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:13.451+0000 7fe380dee640 1 -- 172.21.15.52:0/415368314 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe37c1a0090 con 0x7fe37c1024e0 2024-06-10T02:54:13.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:13.451+0000 7fe380dee640 1 -- 172.21.15.52:0/415368314 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe37c0fd940 con 0x7fe37c1024e0 2024-06-10T02:54:13.454 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:13.453+0000 7fe3737fe640 1 -- 172.21.15.52:0/415368314 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7fe3740079b0 con 0x7fe37c1024e0 2024-06-10T02:54:13.454 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:13.453+0000 7fe3737fe640 1 --2- 172.21.15.52:0/415368314 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fe3680a2c20 0x7fe3680a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:13.454 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:13.453+0000 7fe3737fe640 1 -- 172.21.15.52:0/415368314 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7fe3740cf3c0 con 0x7fe37c1024e0 2024-06-10T02:54:13.454 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:13.453+0000 7fe379d9b640 1 --2- 172.21.15.52:0/415368314 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fe3680a2c20 0x7fe3680a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:13.454 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:13.454+0000 7fe379d9b640 1 --2- 172.21.15.52:0/415368314 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fe3680a2c20 0x7fe3680a50e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fe37c069610 tx=0x7fe364005d90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:54:13.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:13.458+0000 7fe3737fe640 1 -- 172.21.15.52:0/415368314 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fe374014030 con 0x7fe37c1024e0 2024-06-10T02:54:13.638 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:13.639+0000 7fe380dee640 1 -- 172.21.15.52:0/415368314 --> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe37c061de0 con 0x7fe3680a2c20 2024-06-10T02:54:13.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:13.649+0000 7fe3737fe640 1 -- 172.21.15.52:0/415368314 <== mgr.14150 v2:172.21.15.52:6800/3214763969 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+341022 (secure 0 0 0) 0x7fe37c061de0 con 0x7fe3680a2c20 2024-06-10T02:54:13.651 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-10T02:54:13.653 INFO:teuthology.orchestra.run.smithi052.stderr:dumped all 2024-06-10T02:54:13.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:13.652+0000 7fe380dee640 1 -- 172.21.15.52:0/415368314 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fe3680a2c20 msgr2=0x7fe3680a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:54:13.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:13.652+0000 7fe380dee640 1 --2- 172.21.15.52:0/415368314 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fe3680a2c20 0x7fe3680a50e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fe37c069610 tx=0x7fe364005d90 comp rx=0 tx=0).stop 2024-06-10T02:54:13.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:13.652+0000 7fe380dee640 1 -- 172.21.15.52:0/415368314 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe37c1024e0 msgr2=0x7fe37c197c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:54:13.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:13.652+0000 7fe380dee640 1 --2- 172.21.15.52:0/415368314 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe37c1024e0 0x7fe37c197c40 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fe37400c890 tx=0x7fe37400cd50 comp rx=0 tx=0).stop 2024-06-10T02:54:13.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:13.652+0000 7fe380dee640 1 -- 172.21.15.52:0/415368314 shutdown_connections 2024-06-10T02:54:13.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:13.652+0000 7fe380dee640 1 --2- 172.21.15.52:0/415368314 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe37c107740 0x7fe37c19f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:13.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:13.652+0000 7fe380dee640 1 --2- 172.21.15.52:0/415368314 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fe3680a2c20 0x7fe3680a50e0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:13.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:13.652+0000 7fe380dee640 1 --2- 172.21.15.52:0/415368314 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe37c104e10 0x7fe37c198180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:13.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:13.652+0000 7fe380dee640 1 --2- 172.21.15.52:0/415368314 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe37c1024e0 0x7fe37c197c40 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:13.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:13.652+0000 7fe380dee640 1 -- 172.21.15.52:0/415368314 >> 172.21.15.52:0/415368314 conn(0x7fe37c0fbe50 msgr2=0x7fe37c102e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:54:13.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:13.652+0000 7fe380dee640 1 -- 172.21.15.52:0/415368314 shutdown_connections 2024-06-10T02:54:13.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:13.652+0000 7fe380dee640 1 -- 172.21.15.52:0/415368314 wait complete. 2024-06-10T02:54:13.662 INFO:teuthology.orchestra.run.smithi052.stdout:{"pg_ready":true,"pg_map":{"version":435,"stamp":"2024-06-10T02:54:13.043895+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":671341,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1999,"num_read_kb":1133,"num_write":459,"num_write_kb":3113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1181318,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":876,"ondisk_log_size":876,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":379364,"kb_used_data":13572,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124562460,"statfs":{"total":1151940427776,"available":1151551959040,"internally_reserved":0,"allocated":13897728,"data_stored":8859749,"data_compressed":497715,"data_compressed_allocated":675840,"data_compressed_original":2850816,"omap_allocated":19141,"internal_metadata":374453563},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":9289,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":144,"num_read_kb":79,"num_write":12,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003351"},"pg_stats":[{"pgid":"3.3f","version":"100'8","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:43.001975+0000","last_change":"2024-06-10T02:50:02.209146+0000","last_active":"2024-06-10T02:52:43.001975+0000","last_peered":"2024-06-10T02:52:43.001975+0000","last_clean":"2024-06-10T02:52:43.001975+0000","last_became_active":"2024-06-10T02:50:02.208027+0000","last_became_peered":"2024-06-10T02:50:02.208027+0000","last_unstale":"2024-06-10T02:52:43.001975+0000","last_undegraded":"2024-06-10T02:52:43.001975+0000","last_fullsized":"2024-06-10T02:52:43.001975+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:35:34.765849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"100'4","reported_seq":43,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.989740+0000","last_change":"2024-06-10T02:50:02.198767+0000","last_active":"2024-06-10T02:52:42.989740+0000","last_peered":"2024-06-10T02:52:42.989740+0000","last_clean":"2024-06-10T02:52:42.989740+0000","last_became_active":"2024-06-10T02:50:02.198450+0000","last_became_peered":"2024-06-10T02:50:02.198450+0000","last_unstale":"2024-06-10T02:52:42.989740+0000","last_undegraded":"2024-06-10T02:52:42.989740+0000","last_fullsized":"2024-06-10T02:52:42.989740+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:49:56.666083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"100'7","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:43.000326+0000","last_change":"2024-06-10T02:50:02.208885+0000","last_active":"2024-06-10T02:52:43.000326+0000","last_peered":"2024-06-10T02:52:43.000326+0000","last_clean":"2024-06-10T02:52:43.000326+0000","last_became_active":"2024-06-10T02:50:02.206953+0000","last_became_peered":"2024-06-10T02:50:02.206953+0000","last_unstale":"2024-06-10T02:52:43.000326+0000","last_undegraded":"2024-06-10T02:52:43.000326+0000","last_fullsized":"2024-06-10T02:52:43.000326+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:41:23.928939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"100'8","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.994111+0000","last_change":"2024-06-10T02:50:02.208632+0000","last_active":"2024-06-10T02:52:42.994111+0000","last_peered":"2024-06-10T02:52:42.994111+0000","last_clean":"2024-06-10T02:52:42.994111+0000","last_became_active":"2024-06-10T02:50:02.207587+0000","last_became_peered":"2024-06-10T02:50:02.207587+0000","last_unstale":"2024-06-10T02:52:42.994111+0000","last_undegraded":"2024-06-10T02:52:42.994111+0000","last_fullsized":"2024-06-10T02:52:42.994111+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T14:34:36.720170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"100'4","reported_seq":43,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.973606+0000","last_change":"2024-06-10T02:50:02.197966+0000","last_active":"2024-06-10T02:52:42.973606+0000","last_peered":"2024-06-10T02:52:42.973606+0000","last_clean":"2024-06-10T02:52:42.973606+0000","last_became_active":"2024-06-10T02:50:02.197612+0000","last_became_peered":"2024-06-10T02:50:02.197612+0000","last_unstale":"2024-06-10T02:52:42.973606+0000","last_undegraded":"2024-06-10T02:52:42.973606+0000","last_fullsized":"2024-06-10T02:52:42.973606+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:24:53.804613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"100'6","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.987354+0000","last_change":"2024-06-10T02:50:02.201952+0000","last_active":"2024-06-10T02:52:42.987354+0000","last_peered":"2024-06-10T02:52:42.987354+0000","last_clean":"2024-06-10T02:52:42.987354+0000","last_became_active":"2024-06-10T02:50:02.200828+0000","last_became_peered":"2024-06-10T02:50:02.200828+0000","last_unstale":"2024-06-10T02:52:42.987354+0000","last_undegraded":"2024-06-10T02:52:42.987354+0000","last_fullsized":"2024-06-10T02:52:42.987354+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:46:28.518846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"100'5","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.990208+0000","last_change":"2024-06-10T02:50:02.203483+0000","last_active":"2024-06-10T02:52:42.990208+0000","last_peered":"2024-06-10T02:52:42.990208+0000","last_clean":"2024-06-10T02:52:42.990208+0000","last_became_active":"2024-06-10T02:50:02.203153+0000","last_became_peered":"2024-06-10T02:50:02.203153+0000","last_unstale":"2024-06-10T02:52:42.990208+0000","last_undegraded":"2024-06-10T02:52:42.990208+0000","last_fullsized":"2024-06-10T02:52:42.990208+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:29:30.422449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"100'9","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.995630+0000","last_change":"2024-06-10T02:50:02.201686+0000","last_active":"2024-06-10T02:52:42.995630+0000","last_peered":"2024-06-10T02:52:42.995630+0000","last_clean":"2024-06-10T02:52:42.995630+0000","last_became_active":"2024-06-10T02:50:02.200973+0000","last_became_peered":"2024-06-10T02:50:02.200973+0000","last_unstale":"2024-06-10T02:52:42.995630+0000","last_undegraded":"2024-06-10T02:52:42.995630+0000","last_fullsized":"2024-06-10T02:52:42.995630+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:41:13.712675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"100'11","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:43.001422+0000","last_change":"2024-06-10T02:50:02.210497+0000","last_active":"2024-06-10T02:52:43.001422+0000","last_peered":"2024-06-10T02:52:43.001422+0000","last_clean":"2024-06-10T02:52:43.001422+0000","last_became_active":"2024-06-10T02:50:02.209675+0000","last_became_peered":"2024-06-10T02:50:02.209675+0000","last_unstale":"2024-06-10T02:52:43.001422+0000","last_undegraded":"2024-06-10T02:52:43.001422+0000","last_fullsized":"2024-06-10T02:52:43.001422+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:50:29.879669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"100'16","reported_seq":61,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.994975+0000","last_change":"2024-06-10T02:50:44.741139+0000","last_active":"2024-06-10T02:52:42.994975+0000","last_peered":"2024-06-10T02:52:42.994975+0000","last_clean":"2024-06-10T02:52:42.994975+0000","last_became_active":"2024-06-10T02:50:44.740685+0000","last_became_peered":"2024-06-10T02:50:44.740685+0000","last_unstale":"2024-06-10T02:52:42.994975+0000","last_undegraded":"2024-06-10T02:52:42.994975+0000","last_fullsized":"2024-06-10T02:52:42.994975+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:58:59.872732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"100'6","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.999574+0000","last_change":"2024-06-10T02:50:02.206993+0000","last_active":"2024-06-10T02:52:42.999574+0000","last_peered":"2024-06-10T02:52:42.999574+0000","last_clean":"2024-06-10T02:52:42.999574+0000","last_became_active":"2024-06-10T02:50:02.206650+0000","last_became_peered":"2024-06-10T02:50:02.206650+0000","last_unstale":"2024-06-10T02:52:42.999574+0000","last_undegraded":"2024-06-10T02:52:42.999574+0000","last_fullsized":"2024-06-10T02:52:42.999574+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T14:27:49.555350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"100'7","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.979413+0000","last_change":"2024-06-10T02:50:02.200816+0000","last_active":"2024-06-10T02:52:42.979413+0000","last_peered":"2024-06-10T02:52:42.979413+0000","last_clean":"2024-06-10T02:52:42.979413+0000","last_became_active":"2024-06-10T02:50:02.200503+0000","last_became_peered":"2024-06-10T02:50:02.200503+0000","last_unstale":"2024-06-10T02:52:42.979413+0000","last_undegraded":"2024-06-10T02:52:42.979413+0000","last_fullsized":"2024-06-10T02:52:42.979413+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:01:08.326728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"100'10","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.987233+0000","last_change":"2024-06-10T02:50:02.203644+0000","last_active":"2024-06-10T02:52:42.987233+0000","last_peered":"2024-06-10T02:52:42.987233+0000","last_clean":"2024-06-10T02:52:42.987233+0000","last_became_active":"2024-06-10T02:50:02.203100+0000","last_became_peered":"2024-06-10T02:50:02.203100+0000","last_unstale":"2024-06-10T02:52:42.987233+0000","last_undegraded":"2024-06-10T02:52:42.987233+0000","last_fullsized":"2024-06-10T02:52:42.987233+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:01:42.730089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"100'7","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.995068+0000","last_change":"2024-06-10T02:50:02.203330+0000","last_active":"2024-06-10T02:52:42.995068+0000","last_peered":"2024-06-10T02:52:42.995068+0000","last_clean":"2024-06-10T02:52:42.995068+0000","last_became_active":"2024-06-10T02:50:02.202995+0000","last_became_peered":"2024-06-10T02:50:02.202995+0000","last_unstale":"2024-06-10T02:52:42.995068+0000","last_undegraded":"2024-06-10T02:52:42.995068+0000","last_fullsized":"2024-06-10T02:52:42.995068+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:17:23.170591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"100'7","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.986439+0000","last_change":"2024-06-10T02:50:02.204875+0000","last_active":"2024-06-10T02:52:42.986439+0000","last_peered":"2024-06-10T02:52:42.986439+0000","last_clean":"2024-06-10T02:52:42.986439+0000","last_became_active":"2024-06-10T02:50:02.200706+0000","last_became_peered":"2024-06-10T02:50:02.200706+0000","last_unstale":"2024-06-10T02:52:42.986439+0000","last_undegraded":"2024-06-10T02:52:42.986439+0000","last_fullsized":"2024-06-10T02:52:42.986439+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:27:12.282772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"100'9","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.999375+0000","last_change":"2024-06-10T02:50:02.203187+0000","last_active":"2024-06-10T02:52:42.999375+0000","last_peered":"2024-06-10T02:52:42.999375+0000","last_clean":"2024-06-10T02:52:42.999375+0000","last_became_active":"2024-06-10T02:50:02.202034+0000","last_became_peered":"2024-06-10T02:50:02.202034+0000","last_unstale":"2024-06-10T02:52:42.999375+0000","last_undegraded":"2024-06-10T02:52:42.999375+0000","last_fullsized":"2024-06-10T02:52:42.999375+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:59:44.783084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"100'8","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:43.002372+0000","last_change":"2024-06-10T02:50:02.209102+0000","last_active":"2024-06-10T02:52:43.002372+0000","last_peered":"2024-06-10T02:52:43.002372+0000","last_clean":"2024-06-10T02:52:43.002372+0000","last_became_active":"2024-06-10T02:50:02.208730+0000","last_became_peered":"2024-06-10T02:50:02.208730+0000","last_unstale":"2024-06-10T02:52:43.002372+0000","last_undegraded":"2024-06-10T02:52:43.002372+0000","last_fullsized":"2024-06-10T02:52:43.002372+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:08:38.838415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"100'10","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:43.000459+0000","last_change":"2024-06-10T02:50:02.204008+0000","last_active":"2024-06-10T02:52:43.000459+0000","last_peered":"2024-06-10T02:52:43.000459+0000","last_clean":"2024-06-10T02:52:43.000459+0000","last_became_active":"2024-06-10T02:50:02.203655+0000","last_became_peered":"2024-06-10T02:50:02.203655+0000","last_unstale":"2024-06-10T02:52:43.000459+0000","last_undegraded":"2024-06-10T02:52:43.000459+0000","last_fullsized":"2024-06-10T02:52:43.000459+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:32:15.972662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"100'7","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.991723+0000","last_change":"2024-06-10T02:50:02.204917+0000","last_active":"2024-06-10T02:52:42.991723+0000","last_peered":"2024-06-10T02:52:42.991723+0000","last_clean":"2024-06-10T02:52:42.991723+0000","last_became_active":"2024-06-10T02:50:02.204568+0000","last_became_peered":"2024-06-10T02:50:02.204568+0000","last_unstale":"2024-06-10T02:52:42.991723+0000","last_undegraded":"2024-06-10T02:52:42.991723+0000","last_fullsized":"2024-06-10T02:52:42.991723+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:49:46.802665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.721297+0000","last_change":"2024-06-10T02:50:12.240697+0000","last_active":"2024-06-10T02:51:45.721297+0000","last_peered":"2024-06-10T02:51:45.721297+0000","last_clean":"2024-06-10T02:51:45.721297+0000","last_became_active":"2024-06-10T02:50:12.240229+0000","last_became_peered":"2024-06-10T02:50:12.240229+0000","last_unstale":"2024-06-10T02:51:45.721297+0000","last_undegraded":"2024-06-10T02:51:45.721297+0000","last_fullsized":"2024-06-10T02:51:45.721297+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:45:55.071960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"100'15","reported_seq":255,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:54:09.893400+0000","last_change":"2024-06-10T02:50:44.786783+0000","last_active":"2024-06-10T02:54:09.893400+0000","last_peered":"2024-06-10T02:54:09.893400+0000","last_clean":"2024-06-10T02:54:09.893400+0000","last_became_active":"2024-06-10T02:50:44.781773+0000","last_became_peered":"2024-06-10T02:50:44.781773+0000","last_unstale":"2024-06-10T02:54:09.893400+0000","last_undegraded":"2024-06-10T02:54:09.893400+0000","last_fullsized":"2024-06-10T02:54:09.893400+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:31:06.025854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":182,"num_read_kb":182,"num_write":16,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,4],"acting":[9,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"100'5","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.994824+0000","last_change":"2024-06-10T02:50:02.202573+0000","last_active":"2024-06-10T02:52:42.994824+0000","last_peered":"2024-06-10T02:52:42.994824+0000","last_clean":"2024-06-10T02:52:42.994824+0000","last_became_active":"2024-06-10T02:50:02.202241+0000","last_became_peered":"2024-06-10T02:50:02.202241+0000","last_unstale":"2024-06-10T02:52:42.994824+0000","last_undegraded":"2024-06-10T02:52:42.994824+0000","last_fullsized":"2024-06-10T02:52:42.994824+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:31:40.230851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.383047+0000","last_change":"2024-06-10T02:50:12.238372+0000","last_active":"2024-06-10T02:51:46.383047+0000","last_peered":"2024-06-10T02:51:46.383047+0000","last_clean":"2024-06-10T02:51:46.383047+0000","last_became_active":"2024-06-10T02:50:12.237884+0000","last_became_peered":"2024-06-10T02:50:12.237884+0000","last_unstale":"2024-06-10T02:51:46.383047+0000","last_undegraded":"2024-06-10T02:51:46.383047+0000","last_fullsized":"2024-06-10T02:51:46.383047+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:55:31.432655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"98'2","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:02.117661+0000","last_change":"2024-06-10T02:49:59.031528+0000","last_active":"2024-06-10T02:52:02.117661+0000","last_peered":"2024-06-10T02:52:02.117661+0000","last_clean":"2024-06-10T02:52:02.117661+0000","last_became_active":"2024-06-10T02:49:59.031147+0000","last_became_peered":"2024-06-10T02:49:59.031147+0000","last_unstale":"2024-06-10T02:52:02.117661+0000","last_undegraded":"2024-06-10T02:52:02.117661+0000","last_fullsized":"2024-06-10T02:52:02.117661+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:17:08.534396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"100'6","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:43.000760+0000","last_change":"2024-06-10T02:50:02.202858+0000","last_active":"2024-06-10T02:52:43.000760+0000","last_peered":"2024-06-10T02:52:43.000760+0000","last_clean":"2024-06-10T02:52:43.000760+0000","last_became_active":"2024-06-10T02:50:02.202263+0000","last_became_peered":"2024-06-10T02:50:02.202263+0000","last_unstale":"2024-06-10T02:52:43.000760+0000","last_undegraded":"2024-06-10T02:52:43.000760+0000","last_fullsized":"2024-06-10T02:52:43.000760+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:23:15.382762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.383227+0000","last_change":"2024-06-10T02:50:12.237370+0000","last_active":"2024-06-10T02:51:46.383227+0000","last_peered":"2024-06-10T02:51:46.383227+0000","last_clean":"2024-06-10T02:51:46.383227+0000","last_became_active":"2024-06-10T02:50:12.236803+0000","last_became_peered":"2024-06-10T02:50:12.236803+0000","last_unstale":"2024-06-10T02:51:46.383227+0000","last_undegraded":"2024-06-10T02:51:46.383227+0000","last_fullsized":"2024-06-10T02:51:46.383227+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:06:40.695710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"98'12","reported_seq":58,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.724307+0000","last_change":"2024-06-10T02:49:59.000911+0000","last_active":"2024-06-10T02:51:45.724307+0000","last_peered":"2024-06-10T02:51:45.724307+0000","last_clean":"2024-06-10T02:51:45.724307+0000","last_became_active":"2024-06-10T02:49:59.000571+0000","last_became_peered":"2024-06-10T02:49:59.000571+0000","last_unstale":"2024-06-10T02:51:45.724307+0000","last_undegraded":"2024-06-10T02:51:45.724307+0000","last_fullsized":"2024-06-10T02:51:45.724307+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:49:24.635793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"100'9","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.997165+0000","last_change":"2024-06-10T02:50:02.209614+0000","last_active":"2024-06-10T02:52:42.997165+0000","last_peered":"2024-06-10T02:52:42.997165+0000","last_clean":"2024-06-10T02:52:42.997165+0000","last_became_active":"2024-06-10T02:50:02.209251+0000","last_became_peered":"2024-06-10T02:50:02.209251+0000","last_unstale":"2024-06-10T02:52:42.997165+0000","last_undegraded":"2024-06-10T02:52:42.997165+0000","last_fullsized":"2024-06-10T02:52:42.997165+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:35:32.256081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.383229+0000","last_change":"2024-06-10T02:50:12.237811+0000","last_active":"2024-06-10T02:51:46.383229+0000","last_peered":"2024-06-10T02:51:46.383229+0000","last_clean":"2024-06-10T02:51:46.383229+0000","last_became_active":"2024-06-10T02:50:12.237017+0000","last_became_peered":"2024-06-10T02:50:12.237017+0000","last_unstale":"2024-06-10T02:51:46.383229+0000","last_undegraded":"2024-06-10T02:51:46.383229+0000","last_fullsized":"2024-06-10T02:51:46.383229+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:14:29.818885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"98'5","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.723967+0000","last_change":"2024-06-10T02:49:58.996971+0000","last_active":"2024-06-10T02:51:45.723967+0000","last_peered":"2024-06-10T02:51:45.723967+0000","last_clean":"2024-06-10T02:51:45.723967+0000","last_became_active":"2024-06-10T02:49:58.996601+0000","last_became_peered":"2024-06-10T02:49:58.996601+0000","last_unstale":"2024-06-10T02:51:45.723967+0000","last_undegraded":"2024-06-10T02:51:45.723967+0000","last_fullsized":"2024-06-10T02:51:45.723967+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:26:45.963351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"100'15","reported_seq":60,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:43.002596+0000","last_change":"2024-06-10T02:50:44.745816+0000","last_active":"2024-06-10T02:52:43.002596+0000","last_peered":"2024-06-10T02:52:43.002596+0000","last_clean":"2024-06-10T02:52:43.002596+0000","last_became_active":"2024-06-10T02:50:44.745386+0000","last_became_peered":"2024-06-10T02:50:44.745386+0000","last_unstale":"2024-06-10T02:52:43.002596+0000","last_undegraded":"2024-06-10T02:52:43.002596+0000","last_fullsized":"2024-06-10T02:52:43.002596+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:08:53.977133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.721700+0000","last_change":"2024-06-10T02:50:12.236835+0000","last_active":"2024-06-10T02:51:45.721700+0000","last_peered":"2024-06-10T02:51:45.721700+0000","last_clean":"2024-06-10T02:51:45.721700+0000","last_became_active":"2024-06-10T02:50:12.236309+0000","last_became_peered":"2024-06-10T02:50:12.236309+0000","last_unstale":"2024-06-10T02:51:45.721700+0000","last_undegraded":"2024-06-10T02:51:45.721700+0000","last_fullsized":"2024-06-10T02:51:45.721700+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:09:02.789286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"100'8","reported_seq":240,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:54:09.894381+0000","last_change":"2024-06-10T02:49:58.997725+0000","last_active":"2024-06-10T02:54:09.894381+0000","last_peered":"2024-06-10T02:54:09.894381+0000","last_clean":"2024-06-10T02:54:09.894381+0000","last_became_active":"2024-06-10T02:49:58.997375+0000","last_became_peered":"2024-06-10T02:49:58.997375+0000","last_unstale":"2024-06-10T02:54:09.894381+0000","last_undegraded":"2024-06-10T02:54:09.894381+0000","last_fullsized":"2024-06-10T02:54:09.894381+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:28:49.714356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1461,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":187,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"100'7","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.995883+0000","last_change":"2024-06-10T02:50:02.201247+0000","last_active":"2024-06-10T02:52:42.995883+0000","last_peered":"2024-06-10T02:52:42.995883+0000","last_clean":"2024-06-10T02:52:42.995883+0000","last_became_active":"2024-06-10T02:50:02.200897+0000","last_became_peered":"2024-06-10T02:50:02.200897+0000","last_unstale":"2024-06-10T02:52:42.995883+0000","last_undegraded":"2024-06-10T02:52:42.995883+0000","last_fullsized":"2024-06-10T02:52:42.995883+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:01:58.166644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.721648+0000","last_change":"2024-06-10T02:50:12.243097+0000","last_active":"2024-06-10T02:51:45.721648+0000","last_peered":"2024-06-10T02:51:45.721648+0000","last_clean":"2024-06-10T02:51:45.721648+0000","last_became_active":"2024-06-10T02:50:12.242501+0000","last_became_peered":"2024-06-10T02:50:12.242501+0000","last_unstale":"2024-06-10T02:51:45.721648+0000","last_undegraded":"2024-06-10T02:51:45.721648+0000","last_fullsized":"2024-06-10T02:51:45.721648+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T14:42:44.220340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"98'7","reported_seq":52,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.720539+0000","last_change":"2024-06-10T02:49:58.996956+0000","last_active":"2024-06-10T02:51:45.720539+0000","last_peered":"2024-06-10T02:51:45.720539+0000","last_clean":"2024-06-10T02:51:45.720539+0000","last_became_active":"2024-06-10T02:49:58.996634+0000","last_became_peered":"2024-06-10T02:49:58.996634+0000","last_unstale":"2024-06-10T02:51:45.720539+0000","last_undegraded":"2024-06-10T02:51:45.720539+0000","last_fullsized":"2024-06-10T02:51:45.720539+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:25:02.364472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"100'8","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.995286+0000","last_change":"2024-06-10T02:50:02.205808+0000","last_active":"2024-06-10T02:52:42.995286+0000","last_peered":"2024-06-10T02:52:42.995286+0000","last_clean":"2024-06-10T02:52:42.995286+0000","last_became_active":"2024-06-10T02:50:02.203558+0000","last_became_peered":"2024-06-10T02:50:02.203558+0000","last_unstale":"2024-06-10T02:52:42.995286+0000","last_undegraded":"2024-06-10T02:52:42.995286+0000","last_fullsized":"2024-06-10T02:52:42.995286+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T13:43:23.315375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.365641+0000","last_change":"2024-06-10T02:50:12.242005+0000","last_active":"2024-06-10T02:51:46.365641+0000","last_peered":"2024-06-10T02:51:46.365641+0000","last_clean":"2024-06-10T02:51:46.365641+0000","last_became_active":"2024-06-10T02:50:12.241295+0000","last_became_peered":"2024-06-10T02:50:12.241295+0000","last_unstale":"2024-06-10T02:51:46.365641+0000","last_undegraded":"2024-06-10T02:51:46.365641+0000","last_fullsized":"2024-06-10T02:51:46.365641+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:32:52.702756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"98'3","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.714696+0000","last_change":"2024-06-10T02:49:58.995579+0000","last_active":"2024-06-10T02:51:45.714696+0000","last_peered":"2024-06-10T02:51:45.714696+0000","last_clean":"2024-06-10T02:51:45.714696+0000","last_became_active":"2024-06-10T02:49:58.995208+0000","last_became_peered":"2024-06-10T02:49:58.995208+0000","last_unstale":"2024-06-10T02:51:45.714696+0000","last_undegraded":"2024-06-10T02:51:45.714696+0000","last_fullsized":"2024-06-10T02:51:45.714696+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:26:03.513482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"100'8","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.993655+0000","last_change":"2024-06-10T02:50:02.197090+0000","last_active":"2024-06-10T02:52:42.993655+0000","last_peered":"2024-06-10T02:52:42.993655+0000","last_clean":"2024-06-10T02:52:42.993655+0000","last_became_active":"2024-06-10T02:50:02.196746+0000","last_became_peered":"2024-06-10T02:50:02.196746+0000","last_unstale":"2024-06-10T02:52:42.993655+0000","last_undegraded":"2024-06-10T02:52:42.993655+0000","last_fullsized":"2024-06-10T02:52:42.993655+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:52:09.987978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.012041+0000","last_change":"2024-06-10T02:50:12.234632+0000","last_active":"2024-06-10T02:51:46.012041+0000","last_peered":"2024-06-10T02:51:46.012041+0000","last_clean":"2024-06-10T02:51:46.012041+0000","last_became_active":"2024-06-10T02:50:12.234266+0000","last_became_peered":"2024-06-10T02:50:12.234266+0000","last_unstale":"2024-06-10T02:51:46.012041+0000","last_undegraded":"2024-06-10T02:51:46.012041+0000","last_fullsized":"2024-06-10T02:51:46.012041+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:21:27.348441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"98'3","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.383558+0000","last_change":"2024-06-10T02:49:58.999086+0000","last_active":"2024-06-10T02:51:46.383558+0000","last_peered":"2024-06-10T02:51:46.383558+0000","last_clean":"2024-06-10T02:51:46.383558+0000","last_became_active":"2024-06-10T02:49:58.998767+0000","last_became_peered":"2024-06-10T02:49:58.998767+0000","last_unstale":"2024-06-10T02:51:46.383558+0000","last_undegraded":"2024-06-10T02:51:46.383558+0000","last_fullsized":"2024-06-10T02:51:46.383558+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:28:12.922391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"100'11","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.990813+0000","last_change":"2024-06-10T02:50:02.196641+0000","last_active":"2024-06-10T02:52:42.990813+0000","last_peered":"2024-06-10T02:52:42.990813+0000","last_clean":"2024-06-10T02:52:42.990813+0000","last_became_active":"2024-06-10T02:50:02.196285+0000","last_became_peered":"2024-06-10T02:50:02.196285+0000","last_unstale":"2024-06-10T02:52:42.990813+0000","last_undegraded":"2024-06-10T02:52:42.990813+0000","last_fullsized":"2024-06-10T02:52:42.990813+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T13:55:31.284028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.831864+0000","last_change":"2024-06-10T02:50:12.240988+0000","last_active":"2024-06-10T02:51:45.831864+0000","last_peered":"2024-06-10T02:51:45.831864+0000","last_clean":"2024-06-10T02:51:45.831864+0000","last_became_active":"2024-06-10T02:50:12.240490+0000","last_became_peered":"2024-06-10T02:50:12.240490+0000","last_unstale":"2024-06-10T02:51:45.831864+0000","last_undegraded":"2024-06-10T02:51:45.831864+0000","last_fullsized":"2024-06-10T02:51:45.831864+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:54:36.015221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"100'7","reported_seq":416,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:54:10.894993+0000","last_change":"2024-06-10T02:49:58.989557+0000","last_active":"2024-06-10T02:54:10.894993+0000","last_peered":"2024-06-10T02:54:10.894993+0000","last_clean":"2024-06-10T02:54:10.894993+0000","last_became_active":"2024-06-10T02:49:58.989101+0000","last_became_peered":"2024-06-10T02:49:58.989101+0000","last_unstale":"2024-06-10T02:54:10.894993+0000","last_undegraded":"2024-06-10T02:54:10.894993+0000","last_fullsized":"2024-06-10T02:54:10.894993+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:27:42.182012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":364,"num_read_kb":364,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"100'10","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.998692+0000","last_change":"2024-06-10T02:50:02.208752+0000","last_active":"2024-06-10T02:52:42.998692+0000","last_peered":"2024-06-10T02:52:42.998692+0000","last_clean":"2024-06-10T02:52:42.998692+0000","last_became_active":"2024-06-10T02:50:02.208420+0000","last_became_peered":"2024-06-10T02:50:02.208420+0000","last_unstale":"2024-06-10T02:52:42.998692+0000","last_undegraded":"2024-06-10T02:52:42.998692+0000","last_fullsized":"2024-06-10T02:52:42.998692+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:28:09.412317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.382833+0000","last_change":"2024-06-10T02:50:12.236397+0000","last_active":"2024-06-10T02:51:46.382833+0000","last_peered":"2024-06-10T02:51:46.382833+0000","last_clean":"2024-06-10T02:51:46.382833+0000","last_became_active":"2024-06-10T02:50:12.235972+0000","last_became_peered":"2024-06-10T02:50:12.235972+0000","last_unstale":"2024-06-10T02:51:46.382833+0000","last_undegraded":"2024-06-10T02:51:46.382833+0000","last_fullsized":"2024-06-10T02:51:46.382833+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:38:41.723895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"98'4","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.295527+0000","last_change":"2024-06-10T02:49:58.996616+0000","last_active":"2024-06-10T02:51:46.295527+0000","last_peered":"2024-06-10T02:51:46.295527+0000","last_clean":"2024-06-10T02:51:46.295527+0000","last_became_active":"2024-06-10T02:49:58.996289+0000","last_became_peered":"2024-06-10T02:49:58.996289+0000","last_unstale":"2024-06-10T02:51:46.295527+0000","last_undegraded":"2024-06-10T02:51:46.295527+0000","last_fullsized":"2024-06-10T02:51:46.295527+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:22:12.993608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"100'9","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.983255+0000","last_change":"2024-06-10T02:50:02.205230+0000","last_active":"2024-06-10T02:52:42.983255+0000","last_peered":"2024-06-10T02:52:42.983255+0000","last_clean":"2024-06-10T02:52:42.983255+0000","last_became_active":"2024-06-10T02:50:02.204835+0000","last_became_peered":"2024-06-10T02:50:02.204835+0000","last_unstale":"2024-06-10T02:52:42.983255+0000","last_undegraded":"2024-06-10T02:52:42.983255+0000","last_fullsized":"2024-06-10T02:52:42.983255+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:17:35.084921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.383415+0000","last_change":"2024-06-10T02:50:12.231684+0000","last_active":"2024-06-10T02:51:46.383415+0000","last_peered":"2024-06-10T02:51:46.383415+0000","last_clean":"2024-06-10T02:51:46.383415+0000","last_became_active":"2024-06-10T02:50:12.231256+0000","last_became_peered":"2024-06-10T02:50:12.231256+0000","last_unstale":"2024-06-10T02:51:46.383415+0000","last_undegraded":"2024-06-10T02:51:46.383415+0000","last_fullsized":"2024-06-10T02:51:46.383415+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:17:31.878289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"98'5","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.383170+0000","last_change":"2024-06-10T02:49:59.033950+0000","last_active":"2024-06-10T02:51:46.383170+0000","last_peered":"2024-06-10T02:51:46.383170+0000","last_clean":"2024-06-10T02:51:46.383170+0000","last_became_active":"2024-06-10T02:49:59.033688+0000","last_became_peered":"2024-06-10T02:49:59.033688+0000","last_unstale":"2024-06-10T02:51:46.383170+0000","last_undegraded":"2024-06-10T02:51:46.383170+0000","last_fullsized":"2024-06-10T02:51:46.383170+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:32:03.892091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"100'11","reported_seq":56,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.999378+0000","last_change":"2024-06-10T02:50:44.781378+0000","last_active":"2024-06-10T02:52:42.999378+0000","last_peered":"2024-06-10T02:52:42.999378+0000","last_clean":"2024-06-10T02:52:42.999378+0000","last_became_active":"2024-06-10T02:50:44.780955+0000","last_became_peered":"2024-06-10T02:50:44.780955+0000","last_unstale":"2024-06-10T02:52:42.999378+0000","last_undegraded":"2024-06-10T02:52:42.999378+0000","last_fullsized":"2024-06-10T02:52:42.999378+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:49:13.760381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.827482+0000","last_change":"2024-06-10T02:51:46.384748+0000","last_active":"2024-06-10T02:51:46.827482+0000","last_peered":"2024-06-10T02:51:46.827482+0000","last_clean":"2024-06-10T02:51:46.827482+0000","last_became_active":"2024-06-10T02:51:46.384402+0000","last_became_peered":"2024-06-10T02:51:46.384402+0000","last_unstale":"2024-06-10T02:51:46.827482+0000","last_undegraded":"2024-06-10T02:51:46.827482+0000","last_fullsized":"2024-06-10T02:51:46.827482+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:06:48.056342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.14","version":"100'41","reported_seq":462,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:54:10.914777+0000","last_change":"2024-06-10T02:49:58.997890+0000","last_active":"2024-06-10T02:54:10.914777+0000","last_peered":"2024-06-10T02:54:10.914777+0000","last_clean":"2024-06-10T02:54:10.914777+0000","last_became_active":"2024-06-10T02:49:58.997549+0000","last_became_peered":"2024-06-10T02:49:58.997549+0000","last_unstale":"2024-06-10T02:54:10.914777+0000","last_undegraded":"2024-06-10T02:54:10.914777+0000","last_fullsized":"2024-06-10T02:54:10.914777+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T14:40:50.907029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71495,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":376,"num_read_kb":256,"num_write":43,"num_write_kb":90,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"100'6","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.991053+0000","last_change":"2024-06-10T02:50:02.199707+0000","last_active":"2024-06-10T02:52:42.991053+0000","last_peered":"2024-06-10T02:52:42.991053+0000","last_clean":"2024-06-10T02:52:42.991053+0000","last_became_active":"2024-06-10T02:50:02.198987+0000","last_became_peered":"2024-06-10T02:50:02.198987+0000","last_unstale":"2024-06-10T02:52:42.991053+0000","last_undegraded":"2024-06-10T02:52:42.991053+0000","last_fullsized":"2024-06-10T02:52:42.991053+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:48:03.704590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.383847+0000","last_change":"2024-06-10T02:50:12.239608+0000","last_active":"2024-06-10T02:51:46.383847+0000","last_peered":"2024-06-10T02:51:46.383847+0000","last_clean":"2024-06-10T02:51:46.383847+0000","last_became_active":"2024-06-10T02:50:12.238588+0000","last_became_peered":"2024-06-10T02:50:12.238588+0000","last_unstale":"2024-06-10T02:51:46.383847+0000","last_undegraded":"2024-06-10T02:51:46.383847+0000","last_fullsized":"2024-06-10T02:51:46.383847+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:26:12.718921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"98'9","reported_seq":56,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.721583+0000","last_change":"2024-06-10T02:49:58.997193+0000","last_active":"2024-06-10T02:51:45.721583+0000","last_peered":"2024-06-10T02:51:45.721583+0000","last_clean":"2024-06-10T02:51:45.721583+0000","last_became_active":"2024-06-10T02:49:58.996712+0000","last_became_peered":"2024-06-10T02:49:58.996712+0000","last_unstale":"2024-06-10T02:51:45.721583+0000","last_undegraded":"2024-06-10T02:51:45.721583+0000","last_fullsized":"2024-06-10T02:51:45.721583+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:02:46.894489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"100'9","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.994408+0000","last_change":"2024-06-10T02:50:02.200544+0000","last_active":"2024-06-10T02:52:42.994408+0000","last_peered":"2024-06-10T02:52:42.994408+0000","last_clean":"2024-06-10T02:52:42.994408+0000","last_became_active":"2024-06-10T02:50:02.200201+0000","last_became_peered":"2024-06-10T02:50:02.200201+0000","last_unstale":"2024-06-10T02:52:42.994408+0000","last_undegraded":"2024-06-10T02:52:42.994408+0000","last_fullsized":"2024-06-10T02:52:42.994408+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T13:12:40.925448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.365243+0000","last_change":"2024-06-10T02:50:12.236317+0000","last_active":"2024-06-10T02:51:46.365243+0000","last_peered":"2024-06-10T02:51:46.365243+0000","last_clean":"2024-06-10T02:51:46.365243+0000","last_became_active":"2024-06-10T02:50:12.235806+0000","last_became_peered":"2024-06-10T02:50:12.235806+0000","last_unstale":"2024-06-10T02:51:46.365243+0000","last_undegraded":"2024-06-10T02:51:46.365243+0000","last_fullsized":"2024-06-10T02:51:46.365243+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:29:21.425987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"98'5","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.383944+0000","last_change":"2024-06-10T02:49:58.994149+0000","last_active":"2024-06-10T02:51:46.383944+0000","last_peered":"2024-06-10T02:51:46.383944+0000","last_clean":"2024-06-10T02:51:46.383944+0000","last_became_active":"2024-06-10T02:49:58.993775+0000","last_became_peered":"2024-06-10T02:49:58.993775+0000","last_unstale":"2024-06-10T02:51:46.383944+0000","last_undegraded":"2024-06-10T02:51:46.383944+0000","last_fullsized":"2024-06-10T02:51:46.383944+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:23:55.553943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"100'15","reported_seq":54,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.995273+0000","last_change":"2024-06-10T02:50:02.199979+0000","last_active":"2024-06-10T02:52:42.995273+0000","last_peered":"2024-06-10T02:52:42.995273+0000","last_clean":"2024-06-10T02:52:42.995273+0000","last_became_active":"2024-06-10T02:50:02.199596+0000","last_became_peered":"2024-06-10T02:50:02.199596+0000","last_unstale":"2024-06-10T02:52:42.995273+0000","last_undegraded":"2024-06-10T02:52:42.995273+0000","last_fullsized":"2024-06-10T02:52:42.995273+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:13:58.647460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.383103+0000","last_change":"2024-06-10T02:50:12.235253+0000","last_active":"2024-06-10T02:51:46.383103+0000","last_peered":"2024-06-10T02:51:46.383103+0000","last_clean":"2024-06-10T02:51:46.383103+0000","last_became_active":"2024-06-10T02:50:12.234773+0000","last_became_peered":"2024-06-10T02:50:12.234773+0000","last_unstale":"2024-06-10T02:51:46.383103+0000","last_undegraded":"2024-06-10T02:51:46.383103+0000","last_fullsized":"2024-06-10T02:51:46.383103+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:38:10.971411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"100'7","reported_seq":238,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:54:09.893481+0000","last_change":"2024-06-10T02:49:58.996912+0000","last_active":"2024-06-10T02:54:09.893481+0000","last_peered":"2024-06-10T02:54:09.893481+0000","last_clean":"2024-06-10T02:54:09.893481+0000","last_became_active":"2024-06-10T02:49:58.996549+0000","last_became_peered":"2024-06-10T02:49:58.996549+0000","last_unstale":"2024-06-10T02:54:09.893481+0000","last_undegraded":"2024-06-10T02:54:09.893481+0000","last_fullsized":"2024-06-10T02:54:09.893481+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:07:52.757676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1119,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":187,"num_read_kb":5,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"100'6","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.995546+0000","last_change":"2024-06-10T02:50:44.741186+0000","last_active":"2024-06-10T02:52:42.995546+0000","last_peered":"2024-06-10T02:52:42.995546+0000","last_clean":"2024-06-10T02:52:42.995546+0000","last_became_active":"2024-06-10T02:50:44.740782+0000","last_became_peered":"2024-06-10T02:50:44.740782+0000","last_unstale":"2024-06-10T02:52:42.995546+0000","last_undegraded":"2024-06-10T02:52:42.995546+0000","last_fullsized":"2024-06-10T02:52:42.995546+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:46:03.863243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.715228+0000","last_change":"2024-06-10T02:50:12.236505+0000","last_active":"2024-06-10T02:51:45.715228+0000","last_peered":"2024-06-10T02:51:45.715228+0000","last_clean":"2024-06-10T02:51:45.715228+0000","last_became_active":"2024-06-10T02:50:12.236112+0000","last_became_peered":"2024-06-10T02:50:12.236112+0000","last_unstale":"2024-06-10T02:51:45.715228+0000","last_undegraded":"2024-06-10T02:51:45.715228+0000","last_fullsized":"2024-06-10T02:51:45.715228+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:13:38.586736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"98'17","reported_seq":63,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.723449+0000","last_change":"2024-06-10T02:49:59.031965+0000","last_active":"2024-06-10T02:51:45.723449+0000","last_peered":"2024-06-10T02:51:45.723449+0000","last_clean":"2024-06-10T02:51:45.723449+0000","last_became_active":"2024-06-10T02:49:59.031495+0000","last_became_peered":"2024-06-10T02:49:59.031495+0000","last_unstale":"2024-06-10T02:51:45.723449+0000","last_undegraded":"2024-06-10T02:51:45.723449+0000","last_fullsized":"2024-06-10T02:51:45.723449+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T14:21:06.610677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"100'6","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:43.002009+0000","last_change":"2024-06-10T02:50:02.209926+0000","last_active":"2024-06-10T02:52:43.002009+0000","last_peered":"2024-06-10T02:52:43.002009+0000","last_clean":"2024-06-10T02:52:43.002009+0000","last_became_active":"2024-06-10T02:50:02.209540+0000","last_became_peered":"2024-06-10T02:50:02.209540+0000","last_unstale":"2024-06-10T02:52:43.002009+0000","last_undegraded":"2024-06-10T02:52:43.002009+0000","last_fullsized":"2024-06-10T02:52:43.002009+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:49:40.497692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:48.927934+0000","last_change":"2024-06-10T02:51:46.013156+0000","last_active":"2024-06-10T02:51:48.927934+0000","last_peered":"2024-06-10T02:51:48.927934+0000","last_clean":"2024-06-10T02:51:48.927934+0000","last_became_active":"2024-06-10T02:51:46.012769+0000","last_became_peered":"2024-06-10T02:51:46.012769+0000","last_unstale":"2024-06-10T02:51:48.927934+0000","last_undegraded":"2024-06-10T02:51:48.927934+0000","last_fullsized":"2024-06-10T02:51:48.927934+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:44:49.545676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,3,1],"acting":[8,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.f","version":"98'2","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.724214+0000","last_change":"2024-06-10T02:49:58.998372+0000","last_active":"2024-06-10T02:51:45.724214+0000","last_peered":"2024-06-10T02:51:45.724214+0000","last_clean":"2024-06-10T02:51:45.724214+0000","last_became_active":"2024-06-10T02:49:58.998058+0000","last_became_peered":"2024-06-10T02:49:58.998058+0000","last_unstale":"2024-06-10T02:51:45.724214+0000","last_undegraded":"2024-06-10T02:51:45.724214+0000","last_fullsized":"2024-06-10T02:51:45.724214+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:42:50.509851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"100'5","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.993794+0000","last_change":"2024-06-10T02:50:02.209256+0000","last_active":"2024-06-10T02:52:42.993794+0000","last_peered":"2024-06-10T02:52:42.993794+0000","last_clean":"2024-06-10T02:52:42.993794+0000","last_became_active":"2024-06-10T02:50:02.207727+0000","last_became_peered":"2024-06-10T02:50:02.207727+0000","last_unstale":"2024-06-10T02:52:42.993794+0000","last_undegraded":"2024-06-10T02:52:42.993794+0000","last_fullsized":"2024-06-10T02:52:42.993794+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:23:00.175943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.723666+0000","last_change":"2024-06-10T02:50:12.729438+0000","last_active":"2024-06-10T02:51:45.723666+0000","last_peered":"2024-06-10T02:51:45.723666+0000","last_clean":"2024-06-10T02:51:45.723666+0000","last_became_active":"2024-06-10T02:50:12.728918+0000","last_became_peered":"2024-06-10T02:50:12.728918+0000","last_unstale":"2024-06-10T02:51:45.723666+0000","last_undegraded":"2024-06-10T02:51:45.723666+0000","last_fullsized":"2024-06-10T02:51:45.723666+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:04:48.861763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"98'1","reported_seq":53,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:05.628603+0000","last_change":"2024-06-10T02:50:44.782241+0000","last_active":"2024-06-10T02:52:05.628603+0000","last_peered":"2024-06-10T02:52:05.628603+0000","last_clean":"2024-06-10T02:52:05.628603+0000","last_became_active":"2024-06-10T02:50:44.781799+0000","last_became_peered":"2024-06-10T02:50:44.781799+0000","last_unstale":"2024-06-10T02:52:05.628603+0000","last_undegraded":"2024-06-10T02:52:05.628603+0000","last_fullsized":"2024-06-10T02:52:05.628603+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:38:35.726258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"100'4","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.993135+0000","last_change":"2024-06-10T02:50:02.198888+0000","last_active":"2024-06-10T02:52:42.993135+0000","last_peered":"2024-06-10T02:52:42.993135+0000","last_clean":"2024-06-10T02:52:42.993135+0000","last_became_active":"2024-06-10T02:50:02.198346+0000","last_became_peered":"2024-06-10T02:50:02.198346+0000","last_unstale":"2024-06-10T02:52:42.993135+0000","last_undegraded":"2024-06-10T02:52:42.993135+0000","last_fullsized":"2024-06-10T02:52:42.993135+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:15:42.422898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.715171+0000","last_change":"2024-06-10T02:50:12.232763+0000","last_active":"2024-06-10T02:51:45.715171+0000","last_peered":"2024-06-10T02:51:45.715171+0000","last_clean":"2024-06-10T02:51:45.715171+0000","last_became_active":"2024-06-10T02:50:12.232390+0000","last_became_peered":"2024-06-10T02:50:12.232390+0000","last_unstale":"2024-06-10T02:51:45.715171+0000","last_undegraded":"2024-06-10T02:51:45.715171+0000","last_fullsized":"2024-06-10T02:51:45.715171+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:31:47.973500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"100'8","reported_seq":241,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:54:11.896674+0000","last_change":"2024-06-10T02:49:58.994308+0000","last_active":"2024-06-10T02:54:11.896674+0000","last_peered":"2024-06-10T02:54:11.896674+0000","last_clean":"2024-06-10T02:54:11.896674+0000","last_became_active":"2024-06-10T02:49:58.993494+0000","last_became_peered":"2024-06-10T02:49:58.993494+0000","last_unstale":"2024-06-10T02:54:11.896674+0000","last_undegraded":"2024-06-10T02:54:11.896674+0000","last_fullsized":"2024-06-10T02:54:11.896674+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:07:28.452276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1509,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":187,"num_read_kb":4,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"100'7","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.988698+0000","last_change":"2024-06-10T02:50:02.206872+0000","last_active":"2024-06-10T02:52:42.988698+0000","last_peered":"2024-06-10T02:52:42.988698+0000","last_clean":"2024-06-10T02:52:42.988698+0000","last_became_active":"2024-06-10T02:50:02.206307+0000","last_became_peered":"2024-06-10T02:50:02.206307+0000","last_unstale":"2024-06-10T02:52:42.988698+0000","last_undegraded":"2024-06-10T02:52:42.988698+0000","last_fullsized":"2024-06-10T02:52:42.988698+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:01:04.645674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.715107+0000","last_change":"2024-06-10T02:50:12.729392+0000","last_active":"2024-06-10T02:51:45.715107+0000","last_peered":"2024-06-10T02:51:45.715107+0000","last_clean":"2024-06-10T02:51:45.715107+0000","last_became_active":"2024-06-10T02:50:12.728866+0000","last_became_peered":"2024-06-10T02:50:12.728866+0000","last_unstale":"2024-06-10T02:51:45.715107+0000","last_undegraded":"2024-06-10T02:51:45.715107+0000","last_fullsized":"2024-06-10T02:51:45.715107+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:01:35.845184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"100'8","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.995032+0000","last_change":"2024-06-10T02:50:02.202303+0000","last_active":"2024-06-10T02:52:42.995032+0000","last_peered":"2024-06-10T02:52:42.995032+0000","last_clean":"2024-06-10T02:52:42.995032+0000","last_became_active":"2024-06-10T02:50:02.201998+0000","last_became_peered":"2024-06-10T02:50:02.201998+0000","last_unstale":"2024-06-10T02:52:42.995032+0000","last_undegraded":"2024-06-10T02:52:42.995032+0000","last_fullsized":"2024-06-10T02:52:42.995032+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:15:00.005693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"100'6","reported_seq":236,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:54:11.895869+0000","last_change":"2024-06-10T02:49:59.031513+0000","last_active":"2024-06-10T02:54:11.895869+0000","last_peered":"2024-06-10T02:54:11.895869+0000","last_clean":"2024-06-10T02:54:11.895869+0000","last_became_active":"2024-06-10T02:49:59.030965+0000","last_became_peered":"2024-06-10T02:49:59.030965+0000","last_unstale":"2024-06-10T02:54:11.895869+0000","last_undegraded":"2024-06-10T02:54:11.895869+0000","last_fullsized":"2024-06-10T02:54:11.895869+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:49:02.171578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":184,"num_read_kb":184,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"98'1","reported_seq":30,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.295434+0000","last_change":"2024-06-10T02:50:12.236334+0000","last_active":"2024-06-10T02:51:46.295434+0000","last_peered":"2024-06-10T02:51:46.295434+0000","last_clean":"2024-06-10T02:51:46.295434+0000","last_became_active":"2024-06-10T02:50:12.235820+0000","last_became_peered":"2024-06-10T02:50:12.235820+0000","last_unstale":"2024-06-10T02:51:46.295434+0000","last_undegraded":"2024-06-10T02:51:46.295434+0000","last_fullsized":"2024-06-10T02:51:46.295434+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:16:12.476881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"100'8","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:43.001311+0000","last_change":"2024-06-10T02:50:02.205146+0000","last_active":"2024-06-10T02:52:43.001311+0000","last_peered":"2024-06-10T02:52:43.001311+0000","last_clean":"2024-06-10T02:52:43.001311+0000","last_became_active":"2024-06-10T02:50:02.204785+0000","last_became_peered":"2024-06-10T02:50:02.204785+0000","last_unstale":"2024-06-10T02:52:43.001311+0000","last_undegraded":"2024-06-10T02:52:43.001311+0000","last_fullsized":"2024-06-10T02:52:43.001311+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T13:01:02.068989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"98'6","reported_seq":55,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:06.016314+0000","last_change":"2024-06-10T02:49:58.997654+0000","last_active":"2024-06-10T02:52:06.016314+0000","last_peered":"2024-06-10T02:52:06.016314+0000","last_clean":"2024-06-10T02:52:06.016314+0000","last_became_active":"2024-06-10T02:49:58.997294+0000","last_became_peered":"2024-06-10T02:49:58.997294+0000","last_unstale":"2024-06-10T02:52:06.016314+0000","last_undegraded":"2024-06-10T02:52:06.016314+0000","last_fullsized":"2024-06-10T02:52:06.016314+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:53:14.763198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":367,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.294476+0000","last_change":"2024-06-10T02:48:34.008201+0000","last_active":"2024-06-10T02:51:46.294476+0000","last_peered":"2024-06-10T02:51:46.294476+0000","last_clean":"2024-06-10T02:51:46.294476+0000","last_became_active":"2024-06-10T02:48:34.007633+0000","last_became_peered":"2024-06-10T02:48:34.007633+0000","last_unstale":"2024-06-10T02:51:46.294476+0000","last_undegraded":"2024-06-10T02:51:46.294476+0000","last_fullsized":"2024-06-10T02:51:46.294476+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:45:07.444823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:45:07.444823+0000","last_clean_scrub_stamp":"2024-06-10T02:45:07.444823+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:56:58.159559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":36,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.383275+0000","last_change":"2024-06-10T02:50:44.782130+0000","last_active":"2024-06-10T02:51:46.383275+0000","last_peered":"2024-06-10T02:51:46.383275+0000","last_clean":"2024-06-10T02:51:46.383275+0000","last_became_active":"2024-06-10T02:50:44.781530+0000","last_became_peered":"2024-06-10T02:50:44.781530+0000","last_unstale":"2024-06-10T02:51:46.383275+0000","last_undegraded":"2024-06-10T02:51:46.383275+0000","last_fullsized":"2024-06-10T02:51:46.383275+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:43:23.826207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,11,6,10],"acting":[5,11,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"100'5","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.994100+0000","last_change":"2024-06-10T02:50:02.213666+0000","last_active":"2024-06-10T02:52:42.994100+0000","last_peered":"2024-06-10T02:52:42.994100+0000","last_clean":"2024-06-10T02:52:42.994100+0000","last_became_active":"2024-06-10T02:50:02.213194+0000","last_became_peered":"2024-06-10T02:50:02.213194+0000","last_unstale":"2024-06-10T02:52:42.994100+0000","last_undegraded":"2024-06-10T02:52:42.994100+0000","last_fullsized":"2024-06-10T02:52:42.994100+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:22:55.905412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"98'4","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:02.798809+0000","last_change":"2024-06-10T02:49:58.997618+0000","last_active":"2024-06-10T02:52:02.798809+0000","last_peered":"2024-06-10T02:52:02.798809+0000","last_clean":"2024-06-10T02:52:02.798809+0000","last_became_active":"2024-06-10T02:49:58.997304+0000","last_became_peered":"2024-06-10T02:49:58.997304+0000","last_unstale":"2024-06-10T02:52:02.798809+0000","last_undegraded":"2024-06-10T02:52:02.798809+0000","last_fullsized":"2024-06-10T02:52:02.798809+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:48:59.520991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.364962+0000","last_change":"2024-06-10T02:50:12.236357+0000","last_active":"2024-06-10T02:51:46.364962+0000","last_peered":"2024-06-10T02:51:46.364962+0000","last_clean":"2024-06-10T02:51:46.364962+0000","last_became_active":"2024-06-10T02:50:12.235915+0000","last_became_peered":"2024-06-10T02:50:12.235915+0000","last_unstale":"2024-06-10T02:51:46.364962+0000","last_undegraded":"2024-06-10T02:51:46.364962+0000","last_fullsized":"2024-06-10T02:51:46.364962+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:20:06.287907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"98'4","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:03.929566+0000","last_change":"2024-06-10T02:49:59.031725+0000","last_active":"2024-06-10T02:52:03.929566+0000","last_peered":"2024-06-10T02:52:03.929566+0000","last_clean":"2024-06-10T02:52:03.929566+0000","last_became_active":"2024-06-10T02:49:59.031296+0000","last_became_peered":"2024-06-10T02:49:59.031296+0000","last_unstale":"2024-06-10T02:52:03.929566+0000","last_undegraded":"2024-06-10T02:52:03.929566+0000","last_fullsized":"2024-06-10T02:52:03.929566+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T13:17:58.571696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"100'9","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.986516+0000","last_change":"2024-06-10T02:50:02.201820+0000","last_active":"2024-06-10T02:52:42.986516+0000","last_peered":"2024-06-10T02:52:42.986516+0000","last_clean":"2024-06-10T02:52:42.986516+0000","last_became_active":"2024-06-10T02:50:02.201478+0000","last_became_peered":"2024-06-10T02:50:02.201478+0000","last_unstale":"2024-06-10T02:52:42.986516+0000","last_undegraded":"2024-06-10T02:52:42.986516+0000","last_fullsized":"2024-06-10T02:52:42.986516+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:57:25.963926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":30,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:12.118458+0000","last_change":"2024-06-10T02:50:12.729418+0000","last_active":"2024-06-10T02:52:12.118458+0000","last_peered":"2024-06-10T02:52:12.118458+0000","last_clean":"2024-06-10T02:52:12.118458+0000","last_became_active":"2024-06-10T02:50:12.728692+0000","last_became_peered":"2024-06-10T02:50:12.728692+0000","last_unstale":"2024-06-10T02:52:12.118458+0000","last_undegraded":"2024-06-10T02:52:12.118458+0000","last_fullsized":"2024-06-10T02:52:12.118458+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:25:00.472396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"100'18","reported_seq":63,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:54:07.234777+0000","last_change":"2024-06-10T02:49:58.996391+0000","last_active":"2024-06-10T02:54:07.234777+0000","last_peered":"2024-06-10T02:54:07.234777+0000","last_clean":"2024-06-10T02:54:07.234777+0000","last_became_active":"2024-06-10T02:49:58.996033+0000","last_became_peered":"2024-06-10T02:49:58.996033+0000","last_unstale":"2024-06-10T02:54:07.234777+0000","last_undegraded":"2024-06-10T02:54:07.234777+0000","last_fullsized":"2024-06-10T02:54:07.234777+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:03:34.568969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":34,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"100'5","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.993669+0000","last_change":"2024-06-10T02:50:02.197974+0000","last_active":"2024-06-10T02:52:42.993669+0000","last_peered":"2024-06-10T02:52:42.993669+0000","last_clean":"2024-06-10T02:52:42.993669+0000","last_became_active":"2024-06-10T02:50:02.197614+0000","last_became_peered":"2024-06-10T02:50:02.197614+0000","last_unstale":"2024-06-10T02:52:42.993669+0000","last_undegraded":"2024-06-10T02:52:42.993669+0000","last_fullsized":"2024-06-10T02:52:42.993669+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:36:55.575808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.721202+0000","last_change":"2024-06-10T02:50:12.234239+0000","last_active":"2024-06-10T02:51:45.721202+0000","last_peered":"2024-06-10T02:51:45.721202+0000","last_clean":"2024-06-10T02:51:45.721202+0000","last_became_active":"2024-06-10T02:50:12.233768+0000","last_became_peered":"2024-06-10T02:50:12.233768+0000","last_unstale":"2024-06-10T02:51:45.721202+0000","last_undegraded":"2024-06-10T02:51:45.721202+0000","last_fullsized":"2024-06-10T02:51:45.721202+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:51:44.230031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"98'7","reported_seq":53,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.383587+0000","last_change":"2024-06-10T02:49:59.192765+0000","last_active":"2024-06-10T02:51:46.383587+0000","last_peered":"2024-06-10T02:51:46.383587+0000","last_clean":"2024-06-10T02:51:46.383587+0000","last_became_active":"2024-06-10T02:49:59.192446+0000","last_became_peered":"2024-06-10T02:49:59.192446+0000","last_unstale":"2024-06-10T02:51:46.383587+0000","last_undegraded":"2024-06-10T02:51:46.383587+0000","last_fullsized":"2024-06-10T02:51:46.383587+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:53:51.862019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"100'7","reported_seq":52,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.989395+0000","last_change":"2024-06-10T02:50:44.781209+0000","last_active":"2024-06-10T02:52:42.989395+0000","last_peered":"2024-06-10T02:52:42.989395+0000","last_clean":"2024-06-10T02:52:42.989395+0000","last_became_active":"2024-06-10T02:50:44.780811+0000","last_became_peered":"2024-06-10T02:50:44.780811+0000","last_unstale":"2024-06-10T02:52:42.989395+0000","last_undegraded":"2024-06-10T02:52:42.989395+0000","last_fullsized":"2024-06-10T02:52:42.989395+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:12:40.663565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.383517+0000","last_change":"2024-06-10T02:50:12.730269+0000","last_active":"2024-06-10T02:51:46.383517+0000","last_peered":"2024-06-10T02:51:46.383517+0000","last_clean":"2024-06-10T02:51:46.383517+0000","last_became_active":"2024-06-10T02:50:12.729574+0000","last_became_peered":"2024-06-10T02:50:12.729574+0000","last_unstale":"2024-06-10T02:51:46.383517+0000","last_undegraded":"2024-06-10T02:51:46.383517+0000","last_fullsized":"2024-06-10T02:51:46.383517+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:51:16.235284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"98'2","reported_seq":53,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.854226+0000","last_change":"2024-06-10T02:50:44.782421+0000","last_active":"2024-06-10T02:51:45.854226+0000","last_peered":"2024-06-10T02:51:45.854226+0000","last_clean":"2024-06-10T02:51:45.854226+0000","last_became_active":"2024-06-10T02:50:44.781881+0000","last_became_peered":"2024-06-10T02:50:44.781881+0000","last_unstale":"2024-06-10T02:51:45.854226+0000","last_undegraded":"2024-06-10T02:51:45.854226+0000","last_fullsized":"2024-06-10T02:51:45.854226+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T14:41:58.771256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"100'2","reported_seq":41,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.952928+0000","last_change":"2024-06-10T02:50:02.197761+0000","last_active":"2024-06-10T02:52:42.952928+0000","last_peered":"2024-06-10T02:52:42.952928+0000","last_clean":"2024-06-10T02:52:42.952928+0000","last_became_active":"2024-06-10T02:50:02.197395+0000","last_became_peered":"2024-06-10T02:50:02.197395+0000","last_unstale":"2024-06-10T02:52:42.952928+0000","last_undegraded":"2024-06-10T02:52:42.952928+0000","last_fullsized":"2024-06-10T02:52:42.952928+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:46:37.537280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:48.927972+0000","last_change":"2024-06-10T02:51:46.385129+0000","last_active":"2024-06-10T02:51:48.927972+0000","last_peered":"2024-06-10T02:51:48.927972+0000","last_clean":"2024-06-10T02:51:48.927972+0000","last_became_active":"2024-06-10T02:51:46.384553+0000","last_became_peered":"2024-06-10T02:51:46.384553+0000","last_unstale":"2024-06-10T02:51:48.927972+0000","last_undegraded":"2024-06-10T02:51:48.927972+0000","last_fullsized":"2024-06-10T02:51:48.927972+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:08:11.183679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"98'7","reported_seq":52,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.831750+0000","last_change":"2024-06-10T02:49:58.987863+0000","last_active":"2024-06-10T02:51:45.831750+0000","last_peered":"2024-06-10T02:51:45.831750+0000","last_clean":"2024-06-10T02:51:45.831750+0000","last_became_active":"2024-06-10T02:49:58.987388+0000","last_became_peered":"2024-06-10T02:49:58.987388+0000","last_unstale":"2024-06-10T02:51:45.831750+0000","last_undegraded":"2024-06-10T02:51:45.831750+0000","last_fullsized":"2024-06-10T02:51:45.831750+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:29:20.340301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"100'9","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.988655+0000","last_change":"2024-06-10T02:50:02.200699+0000","last_active":"2024-06-10T02:52:42.988655+0000","last_peered":"2024-06-10T02:52:42.988655+0000","last_clean":"2024-06-10T02:52:42.988655+0000","last_became_active":"2024-06-10T02:50:02.200278+0000","last_became_peered":"2024-06-10T02:50:02.200278+0000","last_unstale":"2024-06-10T02:52:42.988655+0000","last_undegraded":"2024-06-10T02:52:42.988655+0000","last_fullsized":"2024-06-10T02:52:42.988655+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:26:36.236312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"98'1","reported_seq":30,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.383543+0000","last_change":"2024-06-10T02:50:12.236027+0000","last_active":"2024-06-10T02:51:46.383543+0000","last_peered":"2024-06-10T02:51:46.383543+0000","last_clean":"2024-06-10T02:51:46.383543+0000","last_became_active":"2024-06-10T02:50:12.235629+0000","last_became_peered":"2024-06-10T02:50:12.235629+0000","last_unstale":"2024-06-10T02:51:46.383543+0000","last_undegraded":"2024-06-10T02:51:46.383543+0000","last_fullsized":"2024-06-10T02:51:46.383543+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:58:48.218806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"98'3","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.721449+0000","last_change":"2024-06-10T02:49:58.998496+0000","last_active":"2024-06-10T02:51:45.721449+0000","last_peered":"2024-06-10T02:51:45.721449+0000","last_clean":"2024-06-10T02:51:45.721449+0000","last_became_active":"2024-06-10T02:49:58.997260+0000","last_became_peered":"2024-06-10T02:49:58.997260+0000","last_unstale":"2024-06-10T02:51:45.721449+0000","last_undegraded":"2024-06-10T02:51:45.721449+0000","last_fullsized":"2024-06-10T02:51:45.721449+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T14:22:04.658943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"100'8","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.990375+0000","last_change":"2024-06-10T02:50:02.201300+0000","last_active":"2024-06-10T02:52:42.990375+0000","last_peered":"2024-06-10T02:52:42.990375+0000","last_clean":"2024-06-10T02:52:42.990375+0000","last_became_active":"2024-06-10T02:50:02.200988+0000","last_became_peered":"2024-06-10T02:50:02.200988+0000","last_unstale":"2024-06-10T02:52:42.990375+0000","last_undegraded":"2024-06-10T02:52:42.990375+0000","last_fullsized":"2024-06-10T02:52:42.990375+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:11:12.962048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.383484+0000","last_change":"2024-06-10T02:50:12.728869+0000","last_active":"2024-06-10T02:51:46.383484+0000","last_peered":"2024-06-10T02:51:46.383484+0000","last_clean":"2024-06-10T02:51:46.383484+0000","last_became_active":"2024-06-10T02:50:12.728189+0000","last_became_peered":"2024-06-10T02:50:12.728189+0000","last_unstale":"2024-06-10T02:51:46.383484+0000","last_undegraded":"2024-06-10T02:51:46.383484+0000","last_fullsized":"2024-06-10T02:51:46.383484+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:08:36.624814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"98'4","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.723488+0000","last_change":"2024-06-10T02:49:59.000460+0000","last_active":"2024-06-10T02:51:45.723488+0000","last_peered":"2024-06-10T02:51:45.723488+0000","last_clean":"2024-06-10T02:51:45.723488+0000","last_became_active":"2024-06-10T02:49:59.000075+0000","last_became_peered":"2024-06-10T02:49:59.000075+0000","last_unstale":"2024-06-10T02:51:45.723488+0000","last_undegraded":"2024-06-10T02:51:45.723488+0000","last_fullsized":"2024-06-10T02:51:45.723488+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:06:27.935745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"100'5","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:43.001075+0000","last_change":"2024-06-10T02:50:02.210131+0000","last_active":"2024-06-10T02:52:43.001075+0000","last_peered":"2024-06-10T02:52:43.001075+0000","last_clean":"2024-06-10T02:52:43.001075+0000","last_became_active":"2024-06-10T02:50:02.209781+0000","last_became_peered":"2024-06-10T02:50:02.209781+0000","last_unstale":"2024-06-10T02:52:43.001075+0000","last_undegraded":"2024-06-10T02:52:43.001075+0000","last_fullsized":"2024-06-10T02:52:43.001075+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:27:39.966620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.854407+0000","last_change":"2024-06-10T02:50:12.236017+0000","last_active":"2024-06-10T02:51:45.854407+0000","last_peered":"2024-06-10T02:51:45.854407+0000","last_clean":"2024-06-10T02:51:45.854407+0000","last_became_active":"2024-06-10T02:50:12.235553+0000","last_became_peered":"2024-06-10T02:50:12.235553+0000","last_unstale":"2024-06-10T02:51:45.854407+0000","last_undegraded":"2024-06-10T02:51:45.854407+0000","last_fullsized":"2024-06-10T02:51:45.854407+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:23:55.437994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"100'7","reported_seq":238,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:54:09.894120+0000","last_change":"2024-06-10T02:49:59.192864+0000","last_active":"2024-06-10T02:54:09.894120+0000","last_peered":"2024-06-10T02:54:09.894120+0000","last_clean":"2024-06-10T02:54:09.894120+0000","last_became_active":"2024-06-10T02:49:59.192440+0000","last_became_peered":"2024-06-10T02:49:59.192440+0000","last_unstale":"2024-06-10T02:54:09.894120+0000","last_undegraded":"2024-06-10T02:54:09.894120+0000","last_fullsized":"2024-06-10T02:54:09.894120+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:36:05.868538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1701,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":186,"num_read_kb":4,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"100'7","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:43.000525+0000","last_change":"2024-06-10T02:50:02.205791+0000","last_active":"2024-06-10T02:52:43.000525+0000","last_peered":"2024-06-10T02:52:43.000525+0000","last_clean":"2024-06-10T02:52:43.000525+0000","last_became_active":"2024-06-10T02:50:02.205483+0000","last_became_peered":"2024-06-10T02:50:02.205483+0000","last_unstale":"2024-06-10T02:52:43.000525+0000","last_undegraded":"2024-06-10T02:52:43.000525+0000","last_fullsized":"2024-06-10T02:52:43.000525+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:33:25.875423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.011597+0000","last_change":"2024-06-10T02:50:12.239294+0000","last_active":"2024-06-10T02:51:46.011597+0000","last_peered":"2024-06-10T02:51:46.011597+0000","last_clean":"2024-06-10T02:51:46.011597+0000","last_became_active":"2024-06-10T02:50:12.238778+0000","last_became_peered":"2024-06-10T02:50:12.238778+0000","last_unstale":"2024-06-10T02:51:46.011597+0000","last_undegraded":"2024-06-10T02:51:46.011597+0000","last_fullsized":"2024-06-10T02:51:46.011597+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:26:09.203623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"98'3","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.383589+0000","last_change":"2024-06-10T02:49:58.989680+0000","last_active":"2024-06-10T02:51:46.383589+0000","last_peered":"2024-06-10T02:51:46.383589+0000","last_clean":"2024-06-10T02:51:46.383589+0000","last_became_active":"2024-06-10T02:49:58.989265+0000","last_became_peered":"2024-06-10T02:49:58.989265+0000","last_unstale":"2024-06-10T02:51:46.383589+0000","last_undegraded":"2024-06-10T02:51:46.383589+0000","last_fullsized":"2024-06-10T02:51:46.383589+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:36:34.370177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"100'6","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.990330+0000","last_change":"2024-06-10T02:50:44.743369+0000","last_active":"2024-06-10T02:52:42.990330+0000","last_peered":"2024-06-10T02:52:42.990330+0000","last_clean":"2024-06-10T02:52:42.990330+0000","last_became_active":"2024-06-10T02:50:44.743013+0000","last_became_peered":"2024-06-10T02:50:44.743013+0000","last_unstale":"2024-06-10T02:52:42.990330+0000","last_undegraded":"2024-06-10T02:52:42.990330+0000","last_fullsized":"2024-06-10T02:52:42.990330+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:54:43.728943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.382656+0000","last_change":"2024-06-10T02:50:12.235197+0000","last_active":"2024-06-10T02:51:46.382656+0000","last_peered":"2024-06-10T02:51:46.382656+0000","last_clean":"2024-06-10T02:51:46.382656+0000","last_became_active":"2024-06-10T02:50:12.234791+0000","last_became_peered":"2024-06-10T02:50:12.234791+0000","last_unstale":"2024-06-10T02:51:46.382656+0000","last_undegraded":"2024-06-10T02:51:46.382656+0000","last_fullsized":"2024-06-10T02:51:46.382656+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:09:11.164236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"98'5","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.715164+0000","last_change":"2024-06-10T02:49:58.996759+0000","last_active":"2024-06-10T02:51:45.715164+0000","last_peered":"2024-06-10T02:51:45.715164+0000","last_clean":"2024-06-10T02:51:45.715164+0000","last_became_active":"2024-06-10T02:49:58.996434+0000","last_became_peered":"2024-06-10T02:49:58.996434+0000","last_unstale":"2024-06-10T02:51:45.715164+0000","last_undegraded":"2024-06-10T02:51:45.715164+0000","last_fullsized":"2024-06-10T02:51:45.715164+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:09:35.327797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"100'9","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.996813+0000","last_change":"2024-06-10T02:50:02.201676+0000","last_active":"2024-06-10T02:52:42.996813+0000","last_peered":"2024-06-10T02:52:42.996813+0000","last_clean":"2024-06-10T02:52:42.996813+0000","last_became_active":"2024-06-10T02:50:02.201335+0000","last_became_peered":"2024-06-10T02:50:02.201335+0000","last_unstale":"2024-06-10T02:52:42.996813+0000","last_undegraded":"2024-06-10T02:52:42.996813+0000","last_fullsized":"2024-06-10T02:52:42.996813+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:49:47.038946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"100'9","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.992923+0000","last_change":"2024-06-10T02:50:02.203462+0000","last_active":"2024-06-10T02:52:42.992923+0000","last_peered":"2024-06-10T02:52:42.992923+0000","last_clean":"2024-06-10T02:52:42.992923+0000","last_became_active":"2024-06-10T02:50:02.203115+0000","last_became_peered":"2024-06-10T02:50:02.203115+0000","last_unstale":"2024-06-10T02:52:42.992923+0000","last_undegraded":"2024-06-10T02:52:42.992923+0000","last_fullsized":"2024-06-10T02:52:42.992923+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T13:18:12.400596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"100'12","reported_seq":52,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.998015+0000","last_change":"2024-06-10T02:50:02.205648+0000","last_active":"2024-06-10T02:52:42.998015+0000","last_peered":"2024-06-10T02:52:42.998015+0000","last_clean":"2024-06-10T02:52:42.998015+0000","last_became_active":"2024-06-10T02:50:02.205306+0000","last_became_peered":"2024-06-10T02:50:02.205306+0000","last_unstale":"2024-06-10T02:52:42.998015+0000","last_undegraded":"2024-06-10T02:52:42.998015+0000","last_fullsized":"2024-06-10T02:52:42.998015+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:22:55.020644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"100'8","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.991617+0000","last_change":"2024-06-10T02:50:02.198601+0000","last_active":"2024-06-10T02:52:42.991617+0000","last_peered":"2024-06-10T02:52:42.991617+0000","last_clean":"2024-06-10T02:52:42.991617+0000","last_became_active":"2024-06-10T02:50:02.198103+0000","last_became_peered":"2024-06-10T02:50:02.198103+0000","last_unstale":"2024-06-10T02:52:42.991617+0000","last_undegraded":"2024-06-10T02:52:42.991617+0000","last_fullsized":"2024-06-10T02:52:42.991617+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:56:28.033482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"100'7","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.985332+0000","last_change":"2024-06-10T02:50:02.202638+0000","last_active":"2024-06-10T02:52:42.985332+0000","last_peered":"2024-06-10T02:52:42.985332+0000","last_clean":"2024-06-10T02:52:42.985332+0000","last_became_active":"2024-06-10T02:50:02.202249+0000","last_became_peered":"2024-06-10T02:50:02.202249+0000","last_unstale":"2024-06-10T02:52:42.985332+0000","last_undegraded":"2024-06-10T02:52:42.985332+0000","last_fullsized":"2024-06-10T02:52:42.985332+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:46:29.424164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"100'4","reported_seq":43,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.992649+0000","last_change":"2024-06-10T02:50:02.213526+0000","last_active":"2024-06-10T02:52:42.992649+0000","last_peered":"2024-06-10T02:52:42.992649+0000","last_clean":"2024-06-10T02:52:42.992649+0000","last_became_active":"2024-06-10T02:50:02.213226+0000","last_became_peered":"2024-06-10T02:50:02.213226+0000","last_unstale":"2024-06-10T02:52:42.992649+0000","last_undegraded":"2024-06-10T02:52:42.992649+0000","last_fullsized":"2024-06-10T02:52:42.992649+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:01:56.846340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"100'6","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.993061+0000","last_change":"2024-06-10T02:50:02.206583+0000","last_active":"2024-06-10T02:52:42.993061+0000","last_peered":"2024-06-10T02:52:42.993061+0000","last_clean":"2024-06-10T02:52:42.993061+0000","last_became_active":"2024-06-10T02:50:02.206227+0000","last_became_peered":"2024-06-10T02:50:02.206227+0000","last_unstale":"2024-06-10T02:52:42.993061+0000","last_undegraded":"2024-06-10T02:52:42.993061+0000","last_fullsized":"2024-06-10T02:52:42.993061+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:25:07.246288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"100'15","reported_seq":54,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:43.002970+0000","last_change":"2024-06-10T02:50:02.206348+0000","last_active":"2024-06-10T02:52:43.002970+0000","last_peered":"2024-06-10T02:52:43.002970+0000","last_clean":"2024-06-10T02:52:43.002970+0000","last_became_active":"2024-06-10T02:50:02.206020+0000","last_became_peered":"2024-06-10T02:50:02.206020+0000","last_unstale":"2024-06-10T02:52:43.002970+0000","last_undegraded":"2024-06-10T02:52:43.002970+0000","last_fullsized":"2024-06-10T02:52:43.002970+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:02:26.619113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"100'2","reported_seq":41,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.993521+0000","last_change":"2024-06-10T02:50:02.199405+0000","last_active":"2024-06-10T02:52:42.993521+0000","last_peered":"2024-06-10T02:52:42.993521+0000","last_clean":"2024-06-10T02:52:42.993521+0000","last_became_active":"2024-06-10T02:50:02.199049+0000","last_became_peered":"2024-06-10T02:50:02.199049+0000","last_unstale":"2024-06-10T02:52:42.993521+0000","last_undegraded":"2024-06-10T02:52:42.993521+0000","last_fullsized":"2024-06-10T02:52:42.993521+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:33:04.091160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"100'6","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.992703+0000","last_change":"2024-06-10T02:50:02.202094+0000","last_active":"2024-06-10T02:52:42.992703+0000","last_peered":"2024-06-10T02:52:42.992703+0000","last_clean":"2024-06-10T02:52:42.992703+0000","last_became_active":"2024-06-10T02:50:02.201751+0000","last_became_peered":"2024-06-10T02:50:02.201751+0000","last_unstale":"2024-06-10T02:52:42.992703+0000","last_undegraded":"2024-06-10T02:52:42.992703+0000","last_fullsized":"2024-06-10T02:52:42.992703+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:52:53.946326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"100'8","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.993442+0000","last_change":"2024-06-10T02:50:02.199630+0000","last_active":"2024-06-10T02:52:42.993442+0000","last_peered":"2024-06-10T02:52:42.993442+0000","last_clean":"2024-06-10T02:52:42.993442+0000","last_became_active":"2024-06-10T02:50:02.199320+0000","last_became_peered":"2024-06-10T02:50:02.199320+0000","last_unstale":"2024-06-10T02:52:42.993442+0000","last_undegraded":"2024-06-10T02:52:42.993442+0000","last_fullsized":"2024-06-10T02:52:42.993442+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:17:28.761862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"100'10","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:43.000631+0000","last_change":"2024-06-10T02:50:02.201190+0000","last_active":"2024-06-10T02:52:43.000631+0000","last_peered":"2024-06-10T02:52:43.000631+0000","last_clean":"2024-06-10T02:52:43.000631+0000","last_became_active":"2024-06-10T02:50:02.200574+0000","last_became_peered":"2024-06-10T02:50:02.200574+0000","last_unstale":"2024-06-10T02:52:43.000631+0000","last_undegraded":"2024-06-10T02:52:43.000631+0000","last_fullsized":"2024-06-10T02:52:43.000631+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:02:07.636218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"100'11","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.992436+0000","last_change":"2024-06-10T02:50:02.202374+0000","last_active":"2024-06-10T02:52:42.992436+0000","last_peered":"2024-06-10T02:52:42.992436+0000","last_clean":"2024-06-10T02:52:42.992436+0000","last_became_active":"2024-06-10T02:50:02.201974+0000","last_became_peered":"2024-06-10T02:50:02.201974+0000","last_unstale":"2024-06-10T02:52:42.992436+0000","last_undegraded":"2024-06-10T02:52:42.992436+0000","last_fullsized":"2024-06-10T02:52:42.992436+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:45:34.977926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"100'8","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.995177+0000","last_change":"2024-06-10T02:50:02.209332+0000","last_active":"2024-06-10T02:52:42.995177+0000","last_peered":"2024-06-10T02:52:42.995177+0000","last_clean":"2024-06-10T02:52:42.995177+0000","last_became_active":"2024-06-10T02:50:02.208175+0000","last_became_peered":"2024-06-10T02:50:02.208175+0000","last_unstale":"2024-06-10T02:52:42.995177+0000","last_undegraded":"2024-06-10T02:52:42.995177+0000","last_fullsized":"2024-06-10T02:52:42.995177+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:44:40.857918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":284028,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":80739,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1873,"num_read_kb":1024,"num_write":276,"num_write_kb":173,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":602112,"data_stored":242217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":237,"ondisk_log_size":237,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351043,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32260,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712892,"statfs":{"total":95995035648,"available":95962001408,"internally_reserved":0,"allocated":1110016,"data_stored":574044,"data_compressed":32447,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612680,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32412,"kb_used_data":1236,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712740,"statfs":{"total":95995035648,"available":95961845760,"internally_reserved":0,"allocated":1265664,"data_stored":1229835,"data_compressed":68564,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1599,"internal_metadata":31914433},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776205,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36448,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708704,"statfs":{"total":95995035648,"available":95957712896,"internally_reserved":0,"allocated":1138688,"data_stored":577539,"data_compressed":32447,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037841,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28112,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717040,"statfs":{"total":95995035648,"available":95966248960,"internally_reserved":0,"allocated":1122304,"data_stored":573149,"data_compressed":32447,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299479,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28172,"kb_used_data":1156,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716980,"statfs":{"total":95995035648,"available":95966187520,"internally_reserved":0,"allocated":1183744,"data_stored":1172218,"data_compressed":68564,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863463005,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36420,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708732,"statfs":{"total":95995035648,"available":95957741568,"internally_reserved":0,"allocated":1110016,"data_stored":569868,"data_compressed":32447,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574966,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36496,"kb_used_data":1160,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708656,"statfs":{"total":95995035648,"available":95957663744,"internally_reserved":0,"allocated":1187840,"data_stored":642393,"data_compressed":32447,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738490,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28124,"kb_used_data":1108,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717028,"statfs":{"total":95995035648,"available":95966236672,"internally_reserved":0,"allocated":1134592,"data_stored":570934,"data_compressed":32447,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411441,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28192,"kb_used_data":1176,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716960,"statfs":{"total":95995035648,"available":95966167040,"internally_reserved":0,"allocated":1204224,"data_stored":640542,"data_compressed":32447,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149803,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36432,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708720,"statfs":{"total":95995035648,"available":95957729280,"internally_reserved":0,"allocated":1122304,"data_stored":568655,"data_compressed":32447,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888166,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28116,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717036,"statfs":{"total":95995035648,"available":95966244864,"internally_reserved":0,"allocated":1126400,"data_stored":574224,"data_compressed":32447,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724641,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28180,"kb_used_data":1164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716972,"statfs":{"total":95995035648,"available":95966179328,"internally_reserved":0,"allocated":1191936,"data_stored":1166348,"data_compressed":68564,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34481,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34481,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34481,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36117,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36117,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36117,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36117,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36117,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":74168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":74837,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1903,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":126976,"data_stored":73762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-06-10T02:54:13.668 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T02:54:13.957 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:54:15.094 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:54:14 smithi052 ceph-mon[32362]: from='client.15333 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-10T02:54:15.094 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:54:14 smithi052 ceph-mon[32362]: pgmap v435: 129 pgs: 129 active+clean; 656 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 1.4 KiB/s wr, 12 op/s 2024-06-10T02:54:15.094 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:54:14 smithi052 ceph-mon[32362]: from='client.15339 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-10T02:54:15.094 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:54:14 smithi052 ceph-mon[32362]: from='client.15345 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-10T02:54:15.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:54:14 smithi120 ceph-mon[36634]: from='client.15333 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-10T02:54:15.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:54:14 smithi120 ceph-mon[36634]: pgmap v435: 129 pgs: 129 active+clean; 656 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 1.4 KiB/s wr, 12 op/s 2024-06-10T02:54:15.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:54:14 smithi120 ceph-mon[36634]: from='client.15339 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-10T02:54:15.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:54:14 smithi120 ceph-mon[36634]: from='client.15345 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-10T02:54:15.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:54:14 smithi188 ceph-mon[36599]: from='client.15333 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-10T02:54:15.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:54:14 smithi188 ceph-mon[36599]: pgmap v435: 129 pgs: 129 active+clean; 656 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 1.4 KiB/s wr, 12 op/s 2024-06-10T02:54:15.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:54:14 smithi188 ceph-mon[36599]: from='client.15339 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-10T02:54:15.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:54:14 smithi188 ceph-mon[36599]: from='client.15345 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-10T02:54:15.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:15.737+0000 7f5800fac700 1 -- 172.21.15.52:0/2606938236 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f57fc107840 msgr2=0x7f57fc10f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:54:15.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:15.737+0000 7f5800fac700 1 --2- 172.21.15.52:0/2606938236 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f57fc107840 0x7f57fc10f3c0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f57f400b0d0 tx=0x7f57f400b3e0 comp rx=0 tx=0).stop 2024-06-10T02:54:15.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:15.737+0000 7f5800fac700 1 -- 172.21.15.52:0/2606938236 shutdown_connections 2024-06-10T02:54:15.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:15.737+0000 7f5800fac700 1 --2- 172.21.15.52:0/2606938236 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f57fc107840 0x7f57fc10f3c0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:15.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:15.737+0000 7f5800fac700 1 --2- 172.21.15.52:0/2606938236 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f57fc102740 0x7f57fc102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:15.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:15.737+0000 7f5800fac700 1 --2- 172.21.15.52:0/2606938236 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f57fc1030f0 0x7f57fc107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:15.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:15.737+0000 7f5800fac700 1 -- 172.21.15.52:0/2606938236 >> 172.21.15.52:0/2606938236 conn(0x7f57fc0795f0 msgr2=0x7f57fc079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:54:15.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:15.737+0000 7f5800fac700 1 -- 172.21.15.52:0/2606938236 shutdown_connections 2024-06-10T02:54:15.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:15.737+0000 7f5800fac700 1 -- 172.21.15.52:0/2606938236 wait complete. 2024-06-10T02:54:15.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:15.738+0000 7f5800fac700 1 Processor -- start 2024-06-10T02:54:15.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:15.738+0000 7f5800fac700 1 -- start start 2024-06-10T02:54:15.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:15.738+0000 7f5800fac700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f57fc102740 0x7f57fc19e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:15.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:15.738+0000 7f5800fac700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f57fc1030f0 0x7f57fc19ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:15.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:15.738+0000 7f5800fac700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f57fc107840 0x7f57fc198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:15.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:15.738+0000 7f5800fac700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f57fc112760 con 0x7f57fc102740 2024-06-10T02:54:15.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:15.738+0000 7f5800fac700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f57fc1125e0 con 0x7f57fc1030f0 2024-06-10T02:54:15.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:15.738+0000 7f5800fac700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f57fc1128e0 con 0x7f57fc107840 2024-06-10T02:54:15.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:15.738+0000 7f57fa59c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f57fc102740 0x7f57fc19e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:15.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:15.738+0000 7f57fa59c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f57fc102740 0x7f57fc19e6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60100/0 (socket says 172.21.15.52:60100) 2024-06-10T02:54:15.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:15.738+0000 7f57fa59c700 1 -- 172.21.15.52:0/1049079281 learned_addr learned my addr 172.21.15.52:0/1049079281 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:54:15.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:15.738+0000 7f57f9d9b700 1 --2- 172.21.15.52:0/1049079281 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f57fc1030f0 0x7f57fc19ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:15.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:15.738+0000 7f57fad9d700 1 --2- 172.21.15.52:0/1049079281 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f57fc107840 0x7f57fc198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:15.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:15.738+0000 7f57fa59c700 1 -- 172.21.15.52:0/1049079281 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f57fc107840 msgr2=0x7f57fc198880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:54:15.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:15.738+0000 7f57fa59c700 1 --2- 172.21.15.52:0/1049079281 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f57fc107840 0x7f57fc198880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:15.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:15.738+0000 7f57fa59c700 1 -- 172.21.15.52:0/1049079281 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f57fc1030f0 msgr2=0x7f57fc19ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:54:15.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:15.738+0000 7f57fa59c700 1 --2- 172.21.15.52:0/1049079281 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f57fc1030f0 0x7f57fc19ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:15.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:15.738+0000 7f57fa59c700 1 -- 172.21.15.52:0/1049079281 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57fc199060 con 0x7f57fc102740 2024-06-10T02:54:15.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:15.738+0000 7f57fad9d700 1 --2- 172.21.15.52:0/1049079281 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f57fc107840 0x7f57fc198880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T02:54:15.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:15.739+0000 7f57fa59c700 1 --2- 172.21.15.52:0/1049079281 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f57fc102740 0x7f57fc19e6b0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f57ec00d950 tx=0x7f57ec00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:54:15.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:15.739+0000 7f57eb7fe700 1 -- 172.21.15.52:0/1049079281 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57ec016070 con 0x7f57fc102740 2024-06-10T02:54:15.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:15.739+0000 7f57eb7fe700 1 -- 172.21.15.52:0/1049079281 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f57ec009e20 con 0x7f57fc102740 2024-06-10T02:54:15.741 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:15.739+0000 7f5800fac700 1 -- 172.21.15.52:0/1049079281 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57fc199350 con 0x7f57fc102740 2024-06-10T02:54:15.741 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:15.739+0000 7f57eb7fe700 1 -- 172.21.15.52:0/1049079281 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57ec00f620 con 0x7f57fc102740 2024-06-10T02:54:15.741 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:15.739+0000 7f5800fac700 1 -- 172.21.15.52:0/1049079281 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57fc1a5510 con 0x7f57fc102740 2024-06-10T02:54:15.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:15.740+0000 7f5800fac700 1 -- 172.21.15.52:0/1049079281 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57fc103bd0 con 0x7f57fc102740 2024-06-10T02:54:15.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:15.741+0000 7f57eb7fe700 1 -- 172.21.15.52:0/1049079281 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7f57ec019070 con 0x7f57fc102740 2024-06-10T02:54:15.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:15.742+0000 7f57eb7fe700 1 --2- 172.21.15.52:0/1049079281 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f57e40a2cb0 0x7f57e40a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:15.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:15.742+0000 7f57f9d9b700 1 --2- 172.21.15.52:0/1049079281 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f57e40a2cb0 0x7f57e40a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:15.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:15.742+0000 7f57eb7fe700 1 -- 172.21.15.52:0/1049079281 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f57ec0cf150 con 0x7f57fc102740 2024-06-10T02:54:15.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:15.743+0000 7f57f9d9b700 1 --2- 172.21.15.52:0/1049079281 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f57e40a2cb0 0x7f57e40a5170 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f57fc108b40 tx=0x7f57f000a380 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:54:15.746 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:15.747+0000 7f57eb7fe700 1 -- 172.21.15.52:0/1049079281 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f57ec09bba0 con 0x7f57fc102740 2024-06-10T02:54:15.997 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:15.998+0000 7f5800fac700 1 -- 172.21.15.52:0/1049079281 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f57fc1a5780 con 0x7f57fc102740 2024-06-10T02:54:15.997 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:15.998+0000 7f57eb7fe700 1 -- 172.21.15.52:0/1049079281 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11938 (secure 0 0 0) 0x7f57ec09b4a0 con 0x7f57fc102740 2024-06-10T02:54:15.998 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-10T02:54:15.998 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/889673247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":889673247},{"type":"v1","addr":"172.21.15.52:6835","nonce":889673247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/3615663511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":3615663511},{"type":"v1","addr":"172.21.15.52:6837","nonce":3615663511}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T02:50:03.889766+0000","modified":"2024-06-10T02:54:07.086405+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24484,"mds_1":24469,"mds_2":24463,"mds_3":24449,"mds_4":24428},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14571":{"gid":14571,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6839/3059599042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":3059599042},{"type":"v1","addr":"172.21.15.52:6839","nonce":3059599042}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14583":{"gid":14583,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6841/3039275521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":3039275521},{"type":"v1","addr":"172.21.15.52:6841","nonce":3039275521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24383":{"gid":24383,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6833/2609002896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":2609002896},{"type":"v1","addr":"172.21.15.120:6833","nonce":2609002896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6835/1996370288","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":1996370288},{"type":"v1","addr":"172.21.15.120:6835","nonce":1996370288}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6833/529564979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":529564979},{"type":"v1","addr":"172.21.15.188:6833","nonce":529564979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":25,"addr":"172.21.15.120:6837/4132440952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":4132440952},{"type":"v1","addr":"172.21.15.120:6837","nonce":4132440952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"b","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.120:6839/1672436487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1672436487},{"type":"v1","addr":"172.21.15.120:6839","nonce":1672436487}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":2,"incarnation":22,"state":"up:active","state_seq":26,"addr":"172.21.15.188:6835/3904544259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":3904544259},{"type":"v1","addr":"172.21.15.188:6835","nonce":3904544259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.188:6837/819317726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":819317726},{"type":"v1","addr":"172.21.15.188:6837","nonce":819317726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.188:6839/310444836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":310444836},{"type":"v1","addr":"172.21.15.188:6839","nonce":310444836}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24484,"qdb_cluster":[24469,24484,24463,24428,24449]},"id":1}]} 2024-06-10T02:54:16.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:16.001+0000 7f5800fac700 1 -- 172.21.15.52:0/1049079281 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f57e40a2cb0 msgr2=0x7f57e40a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:54:16.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:16.001+0000 7f5800fac700 1 --2- 172.21.15.52:0/1049079281 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f57e40a2cb0 0x7f57e40a5170 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f57fc108b40 tx=0x7f57f000a380 comp rx=0 tx=0).stop 2024-06-10T02:54:16.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:16.001+0000 7f5800fac700 1 -- 172.21.15.52:0/1049079281 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f57fc102740 msgr2=0x7f57fc19e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:54:16.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:16.001+0000 7f5800fac700 1 --2- 172.21.15.52:0/1049079281 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f57fc102740 0x7f57fc19e6b0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f57ec00d950 tx=0x7f57ec00dc60 comp rx=0 tx=0).stop 2024-06-10T02:54:16.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:16.001+0000 7f5800fac700 1 -- 172.21.15.52:0/1049079281 shutdown_connections 2024-06-10T02:54:16.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:16.001+0000 7f5800fac700 1 --2- 172.21.15.52:0/1049079281 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f57fc107840 0x7f57fc198880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:16.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:16.001+0000 7f5800fac700 1 --2- 172.21.15.52:0/1049079281 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f57e40a2cb0 0x7f57e40a5170 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:16.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:16.001+0000 7f5800fac700 1 --2- 172.21.15.52:0/1049079281 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f57fc102740 0x7f57fc19e6b0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:16.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:16.001+0000 7f5800fac700 1 --2- 172.21.15.52:0/1049079281 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f57fc1030f0 0x7f57fc19ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:16.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:16.001+0000 7f5800fac700 1 -- 172.21.15.52:0/1049079281 >> 172.21.15.52:0/1049079281 conn(0x7f57fc0795f0 msgr2=0x7f57fc1003f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:54:16.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:16.001+0000 7f5800fac700 1 -- 172.21.15.52:0/1049079281 shutdown_connections 2024-06-10T02:54:16.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:16.001+0000 7f5800fac700 1 -- 172.21.15.52:0/1049079281 wait complete. 2024-06-10T02:54:16.001 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 30 2024-06-10T02:54:16.752 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-10T02:54:16.976 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:54:16 smithi052 ceph-mon[32362]: pgmap v436: 129 pgs: 129 active+clean; 656 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1.4 KiB/s wr, 9 op/s 2024-06-10T02:54:16.976 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:54:16 smithi052 ceph-mon[32362]: from='client.? 172.21.15.52:0/1049079281' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T02:54:17.014 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:54:17.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:54:16 smithi120 ceph-mon[36634]: pgmap v436: 129 pgs: 129 active+clean; 656 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1.4 KiB/s wr, 9 op/s 2024-06-10T02:54:17.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:54:16 smithi120 ceph-mon[36634]: from='client.? 172.21.15.52:0/1049079281' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T02:54:17.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:54:16 smithi188 ceph-mon[36599]: pgmap v436: 129 pgs: 129 active+clean; 656 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1.4 KiB/s wr, 9 op/s 2024-06-10T02:54:17.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:54:16 smithi188 ceph-mon[36599]: from='client.? 172.21.15.52:0/1049079281' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T02:54:18.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:18.689+0000 7f3aa4162700 1 -- 172.21.15.52:0/1722999059 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3a9c06ae60 msgr2=0x7f3a9c06b240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:54:18.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:18.689+0000 7f3aa4162700 1 --2- 172.21.15.52:0/1722999059 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3a9c06ae60 0x7f3a9c06b240 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f3a980099c0 tx=0x7f3a98009cd0 comp rx=0 tx=0).stop 2024-06-10T02:54:18.689 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:18.689+0000 7f3aa4162700 1 -- 172.21.15.52:0/1722999059 shutdown_connections 2024-06-10T02:54:18.689 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:18.689+0000 7f3aa4162700 1 --2- 172.21.15.52:0/1722999059 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3a9c1077b0 0x7f3a9c10f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:18.689 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:18.689+0000 7f3aa4162700 1 --2- 172.21.15.52:0/1722999059 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3a9c06ae60 0x7f3a9c06b240 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:18.689 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:18.689+0000 7f3aa4162700 1 --2- 172.21.15.52:0/1722999059 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3a9c06b810 0x7f3a9c1070d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:18.689 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:18.689+0000 7f3aa4162700 1 -- 172.21.15.52:0/1722999059 >> 172.21.15.52:0/1722999059 conn(0x7f3a9c0773e0 msgr2=0x7f3a9c0777f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:54:18.689 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:18.689+0000 7f3aa4162700 1 -- 172.21.15.52:0/1722999059 shutdown_connections 2024-06-10T02:54:18.689 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:18.689+0000 7f3aa4162700 1 -- 172.21.15.52:0/1722999059 wait complete. 2024-06-10T02:54:18.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:18.690+0000 7f3aa4162700 1 Processor -- start 2024-06-10T02:54:18.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:18.690+0000 7f3aa4162700 1 -- start start 2024-06-10T02:54:18.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:18.690+0000 7f3aa4162700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3a9c06ae60 0x7f3a9c1014f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:18.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:18.690+0000 7f3aa4162700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3a9c06b810 0x7f3a9c101a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:18.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:18.690+0000 7f3aa4162700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3a9c1077b0 0x7f3a9c101f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:18.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:18.690+0000 7f3aa4162700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f3a9c112800 con 0x7f3a9c1077b0 2024-06-10T02:54:18.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:18.690+0000 7f3aa4162700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f3a9c112680 con 0x7f3a9c06ae60 2024-06-10T02:54:18.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:18.690+0000 7f3aa4162700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f3a9c112980 con 0x7f3a9c06b810 2024-06-10T02:54:18.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:18.690+0000 7f3aa26ff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3a9c1077b0 0x7f3a9c101f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:18.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:18.690+0000 7f3aa26ff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3a9c1077b0 0x7f3a9c101f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60108/0 (socket says 172.21.15.52:60108) 2024-06-10T02:54:18.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:18.690+0000 7f3aa26ff700 1 -- 172.21.15.52:0/1157406834 learned_addr learned my addr 172.21.15.52:0/1157406834 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:54:18.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:18.690+0000 7f3aa16fd700 1 --2- 172.21.15.52:0/1157406834 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3a9c06b810 0x7f3a9c101a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:18.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:18.690+0000 7f3aa1efe700 1 --2- 172.21.15.52:0/1157406834 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3a9c06ae60 0x7f3a9c1014f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:18.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:18.690+0000 7f3aa26ff700 1 -- 172.21.15.52:0/1157406834 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3a9c06b810 msgr2=0x7f3a9c101a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:54:18.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:18.690+0000 7f3aa26ff700 1 --2- 172.21.15.52:0/1157406834 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3a9c06b810 0x7f3a9c101a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:18.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:18.690+0000 7f3aa26ff700 1 -- 172.21.15.52:0/1157406834 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3a9c06ae60 msgr2=0x7f3a9c1014f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:54:18.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:18.690+0000 7f3aa26ff700 1 --2- 172.21.15.52:0/1157406834 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3a9c06ae60 0x7f3a9c1014f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:18.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:18.690+0000 7f3aa26ff700 1 -- 172.21.15.52:0/1157406834 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a9c1a71a0 con 0x7f3a9c1077b0 2024-06-10T02:54:18.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:18.691+0000 7f3aa26ff700 1 --2- 172.21.15.52:0/1157406834 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3a9c1077b0 0x7f3a9c101f70 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f3a94007ee0 tx=0x7f3a9400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:54:18.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:18.691+0000 7f3a92ffd700 1 -- 172.21.15.52:0/1157406834 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a94004bb0 con 0x7f3a9c1077b0 2024-06-10T02:54:18.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:18.691+0000 7f3a92ffd700 1 -- 172.21.15.52:0/1157406834 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3a94004d50 con 0x7f3a9c1077b0 2024-06-10T02:54:18.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:18.691+0000 7f3a92ffd700 1 -- 172.21.15.52:0/1157406834 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a94005960 con 0x7f3a9c1077b0 2024-06-10T02:54:18.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:18.691+0000 7f3aa4162700 1 -- 172.21.15.52:0/1157406834 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a9c1a7490 con 0x7f3a9c1077b0 2024-06-10T02:54:18.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:18.691+0000 7f3aa4162700 1 -- 172.21.15.52:0/1157406834 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a9c1a7980 con 0x7f3a9c1077b0 2024-06-10T02:54:18.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:18.692+0000 7f3aa4162700 1 -- 172.21.15.52:0/1157406834 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a9c069180 con 0x7f3a9c1077b0 2024-06-10T02:54:18.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:18.694+0000 7f3a92ffd700 1 -- 172.21.15.52:0/1157406834 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7f3a9401a020 con 0x7f3a9c1077b0 2024-06-10T02:54:18.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:18.695+0000 7f3a92ffd700 1 --2- 172.21.15.52:0/1157406834 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f3a880a2cb0 0x7f3a880a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:18.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:18.695+0000 7f3aa1efe700 1 --2- 172.21.15.52:0/1157406834 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f3a880a2cb0 0x7f3a880a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:18.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:18.695+0000 7f3a92ffd700 1 -- 172.21.15.52:0/1157406834 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f3a940cf300 con 0x7f3a9c1077b0 2024-06-10T02:54:18.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:18.695+0000 7f3aa1efe700 1 --2- 172.21.15.52:0/1157406834 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f3a880a2cb0 0x7f3a880a5170 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f3a98009f50 tx=0x7f3a98005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:54:18.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:18.698+0000 7f3a92ffd700 1 -- 172.21.15.52:0/1157406834 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f3a94010070 con 0x7f3a9c1077b0 2024-06-10T02:54:18.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:18.906+0000 7f3aa4162700 1 -- 172.21.15.52:0/1157406834 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3a9c02de60 con 0x7f3a9c1077b0 2024-06-10T02:54:18.906 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:18.907+0000 7f3a92ffd700 1 -- 172.21.15.52:0/1157406834 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v100) ==== 74+0+24739 (secure 0 0 0) 0x7f3a9409bcf0 con 0x7f3a9c1077b0 2024-06-10T02:54:18.907 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-10T02:54:18.907 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":100,"fsid":"d8676e9e-26d2-11ef-bc9b-c7b262605968","created":"2024-06-10T02:41:43.076837+0000","modified":"2024-06-10T02:51:45.708105+0000","last_up_change":"2024-06-10T02:48:57.297829+0000","last_in_change":"2024-06-10T02:48:36.865802+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-10T02:45:07.394690+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-10T02:49:57.926838+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-10T02:50:01.130393+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-10T02:50:10.655588+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e03c0691-cc5a-497a-bbc2-ce18c337f074","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":4255833408},{"type":"v1","addr":"172.21.15.52:6803","nonce":4255833408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":4255833408},{"type":"v1","addr":"172.21.15.52:6805","nonce":4255833408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":4255833408},{"type":"v1","addr":"172.21.15.52:6809","nonce":4255833408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":4255833408},{"type":"v1","addr":"172.21.15.52:6807","nonce":4255833408}]},"public_addr":"172.21.15.52:6803/4255833408","cluster_addr":"172.21.15.52:6805/4255833408","heartbeat_back_addr":"172.21.15.52:6809/4255833408","heartbeat_front_addr":"172.21.15.52:6807/4255833408","state":["exists","up"]},{"osd":1,"uuid":"a24d8da8-c5ec-44d7-ab30-56f6ce3efda7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":4116540053},{"type":"v1","addr":"172.21.15.120:6801","nonce":4116540053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":4116540053},{"type":"v1","addr":"172.21.15.120:6803","nonce":4116540053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":4116540053},{"type":"v1","addr":"172.21.15.120:6807","nonce":4116540053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":4116540053},{"type":"v1","addr":"172.21.15.120:6805","nonce":4116540053}]},"public_addr":"172.21.15.120:6801/4116540053","cluster_addr":"172.21.15.120:6803/4116540053","heartbeat_back_addr":"172.21.15.120:6807/4116540053","heartbeat_front_addr":"172.21.15.120:6805/4116540053","state":["exists","up"]},{"osd":2,"uuid":"a7162106-ccbf-45bd-bc0e-051ee43f4214","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":2624326999},{"type":"v1","addr":"172.21.15.188:6801","nonce":2624326999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":2624326999},{"type":"v1","addr":"172.21.15.188:6803","nonce":2624326999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":2624326999},{"type":"v1","addr":"172.21.15.188:6807","nonce":2624326999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":2624326999},{"type":"v1","addr":"172.21.15.188:6805","nonce":2624326999}]},"public_addr":"172.21.15.188:6801/2624326999","cluster_addr":"172.21.15.188:6803/2624326999","heartbeat_back_addr":"172.21.15.188:6807/2624326999","heartbeat_front_addr":"172.21.15.188:6805/2624326999","state":["exists","up"]},{"osd":3,"uuid":"53680244-b49a-4120-b24a-9ad7a6341dc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":116897011},{"type":"v1","addr":"172.21.15.52:6811","nonce":116897011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":116897011},{"type":"v1","addr":"172.21.15.52:6813","nonce":116897011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":116897011},{"type":"v1","addr":"172.21.15.52:6817","nonce":116897011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":116897011},{"type":"v1","addr":"172.21.15.52:6815","nonce":116897011}]},"public_addr":"172.21.15.52:6811/116897011","cluster_addr":"172.21.15.52:6813/116897011","heartbeat_back_addr":"172.21.15.52:6817/116897011","heartbeat_front_addr":"172.21.15.52:6815/116897011","state":["exists","up"]},{"osd":4,"uuid":"9f72f1f4-2ef3-49c5-8410-c108767d2ef5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":1320690621},{"type":"v1","addr":"172.21.15.120:6809","nonce":1320690621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":1320690621},{"type":"v1","addr":"172.21.15.120:6811","nonce":1320690621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":1320690621},{"type":"v1","addr":"172.21.15.120:6815","nonce":1320690621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":1320690621},{"type":"v1","addr":"172.21.15.120:6813","nonce":1320690621}]},"public_addr":"172.21.15.120:6809/1320690621","cluster_addr":"172.21.15.120:6811/1320690621","heartbeat_back_addr":"172.21.15.120:6815/1320690621","heartbeat_front_addr":"172.21.15.120:6813/1320690621","state":["exists","up"]},{"osd":5,"uuid":"001b5706-7f27-4fa4-9b69-969e6d765abc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3951947483},{"type":"v1","addr":"172.21.15.188:6809","nonce":3951947483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3951947483},{"type":"v1","addr":"172.21.15.188:6811","nonce":3951947483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3951947483},{"type":"v1","addr":"172.21.15.188:6815","nonce":3951947483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3951947483},{"type":"v1","addr":"172.21.15.188:6813","nonce":3951947483}]},"public_addr":"172.21.15.188:6809/3951947483","cluster_addr":"172.21.15.188:6811/3951947483","heartbeat_back_addr":"172.21.15.188:6815/3951947483","heartbeat_front_addr":"172.21.15.188:6813/3951947483","state":["exists","up"]},{"osd":6,"uuid":"140c903e-9b31-4a81-9ccd-2edc0a2decf9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":359084919},{"type":"v1","addr":"172.21.15.52:6819","nonce":359084919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":359084919},{"type":"v1","addr":"172.21.15.52:6821","nonce":359084919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":359084919},{"type":"v1","addr":"172.21.15.52:6825","nonce":359084919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":359084919},{"type":"v1","addr":"172.21.15.52:6823","nonce":359084919}]},"public_addr":"172.21.15.52:6819/359084919","cluster_addr":"172.21.15.52:6821/359084919","heartbeat_back_addr":"172.21.15.52:6825/359084919","heartbeat_front_addr":"172.21.15.52:6823/359084919","state":["exists","up"]},{"osd":7,"uuid":"57b735d4-5072-4ca2-a3ff-3763ad84f15a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":3737722011},{"type":"v1","addr":"172.21.15.120:6817","nonce":3737722011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6818","nonce":3737722011},{"type":"v1","addr":"172.21.15.120:6819","nonce":3737722011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":3737722011},{"type":"v1","addr":"172.21.15.120:6823","nonce":3737722011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":3737722011},{"type":"v1","addr":"172.21.15.120:6821","nonce":3737722011}]},"public_addr":"172.21.15.120:6817/3737722011","cluster_addr":"172.21.15.120:6819/3737722011","heartbeat_back_addr":"172.21.15.120:6823/3737722011","heartbeat_front_addr":"172.21.15.120:6821/3737722011","state":["exists","up"]},{"osd":8,"uuid":"4ffbb43b-854a-4eac-99cf-7af89bc65409","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1444828441},{"type":"v1","addr":"172.21.15.188:6817","nonce":1444828441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1444828441},{"type":"v1","addr":"172.21.15.188:6819","nonce":1444828441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1444828441},{"type":"v1","addr":"172.21.15.188:6823","nonce":1444828441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1444828441},{"type":"v1","addr":"172.21.15.188:6821","nonce":1444828441}]},"public_addr":"172.21.15.188:6817/1444828441","cluster_addr":"172.21.15.188:6819/1444828441","heartbeat_back_addr":"172.21.15.188:6823/1444828441","heartbeat_front_addr":"172.21.15.188:6821/1444828441","state":["exists","up"]},{"osd":9,"uuid":"5bfd985c-8acc-4d00-8561-40bbd541c07b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":924440658},{"type":"v1","addr":"172.21.15.52:6827","nonce":924440658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":924440658},{"type":"v1","addr":"172.21.15.52:6829","nonce":924440658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":924440658},{"type":"v1","addr":"172.21.15.52:6833","nonce":924440658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":924440658},{"type":"v1","addr":"172.21.15.52:6831","nonce":924440658}]},"public_addr":"172.21.15.52:6827/924440658","cluster_addr":"172.21.15.52:6829/924440658","heartbeat_back_addr":"172.21.15.52:6833/924440658","heartbeat_front_addr":"172.21.15.52:6831/924440658","state":["exists","up"]},{"osd":10,"uuid":"0180cddf-e109-42cc-829a-185604276f02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6824","nonce":1531769083},{"type":"v1","addr":"172.21.15.120:6825","nonce":1531769083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":1531769083},{"type":"v1","addr":"172.21.15.120:6827","nonce":1531769083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":1531769083},{"type":"v1","addr":"172.21.15.120:6831","nonce":1531769083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6828","nonce":1531769083},{"type":"v1","addr":"172.21.15.120:6829","nonce":1531769083}]},"public_addr":"172.21.15.120:6825/1531769083","cluster_addr":"172.21.15.120:6827/1531769083","heartbeat_back_addr":"172.21.15.120:6831/1531769083","heartbeat_front_addr":"172.21.15.120:6829/1531769083","state":["exists","up"]},{"osd":11,"uuid":"73aec750-82d1-4800-b6a0-4e305912f9ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":3769050170},{"type":"v1","addr":"172.21.15.188:6825","nonce":3769050170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":3769050170},{"type":"v1","addr":"172.21.15.188:6827","nonce":3769050170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":3769050170},{"type":"v1","addr":"172.21.15.188:6831","nonce":3769050170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":3769050170},{"type":"v1","addr":"172.21.15.188:6829","nonce":3769050170}]},"public_addr":"172.21.15.188:6825/3769050170","cluster_addr":"172.21.15.188:6827/3769050170","heartbeat_back_addr":"172.21.15.188:6831/3769050170","heartbeat_front_addr":"172.21.15.188:6829/3769050170","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:44:16.455752+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:44:40.219735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:45:03.558899+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:45:29.155004+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:45:53.981098+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:46:19.412020+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:46:45.462640+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:47:11.484578+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:47:36.879054+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:48:03.392446+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:48:29.422173+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:48:55.523423+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":4}]},{"pgid":"3.6","mappings":[{"from":3,"to":7}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.9","mappings":[{"from":5,"to":8}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.52:0/3535271671":"2024-06-11T02:42:42.872312+0000","172.21.15.52:0/1092934380":"2024-06-11T02:42:42.872312+0000","172.21.15.52:6800/896317057":"2024-06-11T02:42:42.872312+0000","172.21.15.52:6801/3934479642":"2024-06-11T02:42:15.498118+0000","172.21.15.52:6800/3934479642":"2024-06-11T02:42:15.498118+0000","172.21.15.52:0/2759694132":"2024-06-11T02:42:15.498118+0000","172.21.15.52:0/2593665631":"2024-06-11T02:42:15.498118+0000","172.21.15.52:0/3151504634":"2024-06-11T02:42:15.498118+0000","172.21.15.52:6801/896317057":"2024-06-11T02:42:42.872312+0000","172.21.15.52:0/1254480982":"2024-06-11T02:42:42.872312+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-10T02:54:18.909 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:18.909+0000 7f3aa4162700 1 -- 172.21.15.52:0/1157406834 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f3a880a2cb0 msgr2=0x7f3a880a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:54:18.909 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:18.909+0000 7f3aa4162700 1 --2- 172.21.15.52:0/1157406834 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f3a880a2cb0 0x7f3a880a5170 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f3a98009f50 tx=0x7f3a98005fb0 comp rx=0 tx=0).stop 2024-06-10T02:54:18.909 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:18.909+0000 7f3aa4162700 1 -- 172.21.15.52:0/1157406834 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3a9c1077b0 msgr2=0x7f3a9c101f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:54:18.909 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:18.909+0000 7f3aa4162700 1 --2- 172.21.15.52:0/1157406834 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3a9c1077b0 0x7f3a9c101f70 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f3a94007ee0 tx=0x7f3a9400d3b0 comp rx=0 tx=0).stop 2024-06-10T02:54:18.909 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:18.909+0000 7f3aa4162700 1 -- 172.21.15.52:0/1157406834 shutdown_connections 2024-06-10T02:54:18.909 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:18.909+0000 7f3aa4162700 1 --2- 172.21.15.52:0/1157406834 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f3a880a2cb0 0x7f3a880a5170 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:18.909 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:18.909+0000 7f3aa4162700 1 --2- 172.21.15.52:0/1157406834 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3a9c1077b0 0x7f3a9c101f70 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:18.909 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:18.909+0000 7f3aa4162700 1 --2- 172.21.15.52:0/1157406834 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3a9c06ae60 0x7f3a9c1014f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:18.909 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:18.909+0000 7f3aa4162700 1 --2- 172.21.15.52:0/1157406834 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3a9c06b810 0x7f3a9c101a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:18.909 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:18.909+0000 7f3aa4162700 1 -- 172.21.15.52:0/1157406834 >> 172.21.15.52:0/1157406834 conn(0x7f3a9c0773e0 msgr2=0x7f3a9c0ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:54:18.909 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:18.909+0000 7f3aa4162700 1 -- 172.21.15.52:0/1157406834 shutdown_connections 2024-06-10T02:54:18.909 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:18.910+0000 7f3aa4162700 1 -- 172.21.15.52:0/1157406834 wait complete. 2024-06-10T02:54:19.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:54:18 smithi120 ceph-mon[36634]: pgmap v437: 129 pgs: 129 active+clean; 656 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 1.4 KiB/s wr, 13 op/s 2024-06-10T02:54:19.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:54:18 smithi188 ceph-mon[36599]: pgmap v437: 129 pgs: 129 active+clean; 656 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 1.4 KiB/s wr, 13 op/s 2024-06-10T02:54:19.226 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:54:18 smithi052 ceph-mon[32362]: pgmap v437: 129 pgs: 129 active+clean; 656 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 1.4 KiB/s wr, 13 op/s 2024-06-10T02:54:19.690 DEBUG:tasks.fwd_scrub:Yielding 2024-06-10T02:54:19.691 INFO:teuthology.run_tasks:Running task quiescer... 2024-06-10T02:54:19.704 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-06-10T02:54:19.705 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph pg dump --format=json 2024-06-10T02:54:19.705 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-06-10T02:54:19.705 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-06-10T02:54:19.705 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-06-10T02:54:19.974 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:54:19.976 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:54:19 smithi052 ceph-mon[32362]: from='client.? 172.21.15.52:0/1157406834' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T02:54:19.978 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:54:20.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:54:19 smithi120 ceph-mon[36634]: from='client.? 172.21.15.52:0/1157406834' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T02:54:20.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:54:19 smithi188 ceph-mon[36599]: from='client.? 172.21.15.52:0/1157406834' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T02:54:20.976 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:54:20 smithi052 ceph-mon[32362]: pgmap v438: 129 pgs: 129 active+clean; 656 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2024-06-10T02:54:21.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:54:20 smithi120 ceph-mon[36634]: pgmap v438: 129 pgs: 129 active+clean; 656 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2024-06-10T02:54:21.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:54:20 smithi188 ceph-mon[36599]: pgmap v438: 129 pgs: 129 active+clean; 656 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2024-06-10T02:54:22.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:22.825+0000 7f4bb9a4d700 1 -- 172.21.15.52:0/3824731262 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4bb41084f0 msgr2=0x7f4bb41088d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:54:22.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:22.825+0000 7f4bb9a4d700 1 --2- 172.21.15.52:0/3824731262 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4bb41084f0 0x7f4bb41088d0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f4ba4009990 tx=0x7f4ba4009ca0 comp rx=0 tx=0).stop 2024-06-10T02:54:22.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:22.825+0000 7f4bb9a4d700 1 -- 172.21.15.52:0/3824731262 shutdown_connections 2024-06-10T02:54:22.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:22.825+0000 7f4bb9a4d700 1 --2- 172.21.15.52:0/3824731262 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4bb4102e80 0x7f4bb410f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:22.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:22.825+0000 7f4bb9a4d700 1 --2- 172.21.15.52:0/3824731262 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4bb41084f0 0x7f4bb41088d0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:22.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:22.825+0000 7f4bb9a4d700 1 --2- 172.21.15.52:0/3824731262 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4bb41024e0 0x7f4bb4102940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:22.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:22.825+0000 7f4bb9a4d700 1 -- 172.21.15.52:0/3824731262 >> 172.21.15.52:0/3824731262 conn(0x7f4bb4079530 msgr2=0x7f4bb4079940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:54:22.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:22.825+0000 7f4bb9a4d700 1 -- 172.21.15.52:0/3824731262 shutdown_connections 2024-06-10T02:54:22.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:22.825+0000 7f4bb9a4d700 1 -- 172.21.15.52:0/3824731262 wait complete. 2024-06-10T02:54:22.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:22.825+0000 7f4bb9a4d700 1 Processor -- start 2024-06-10T02:54:22.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:22.826+0000 7f4bb9a4d700 1 -- start start 2024-06-10T02:54:22.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:22.826+0000 7f4bb9a4d700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4bb41024e0 0x7f4bb419e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:22.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:22.826+0000 7f4bb9a4d700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4bb4102e80 0x7f4bb419ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:22.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:22.826+0000 7f4bb9a4d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4bb41084f0 0x7f4bb4198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:22.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:22.826+0000 7f4bb9a4d700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f4bb41127b0 con 0x7f4bb41084f0 2024-06-10T02:54:22.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:22.826+0000 7f4bb9a4d700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f4bb4112630 con 0x7f4bb41024e0 2024-06-10T02:54:22.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:22.826+0000 7f4bb9a4d700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f4bb4112930 con 0x7f4bb4102e80 2024-06-10T02:54:22.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:22.826+0000 7f4bb37fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4bb41084f0 0x7f4bb4198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:22.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:22.826+0000 7f4bb37fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4bb41084f0 0x7f4bb4198af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37468/0 (socket says 172.21.15.52:37468) 2024-06-10T02:54:22.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:22.826+0000 7f4bb37fe700 1 -- 172.21.15.52:0/3243784278 learned_addr learned my addr 172.21.15.52:0/3243784278 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:54:22.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:22.826+0000 7f4bb27fc700 1 --2- 172.21.15.52:0/3243784278 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4bb4102e80 0x7f4bb419ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:22.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:22.826+0000 7f4bb2ffd700 1 --2- 172.21.15.52:0/3243784278 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4bb41024e0 0x7f4bb419e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:22.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:22.826+0000 7f4bb37fe700 1 -- 172.21.15.52:0/3243784278 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4bb4102e80 msgr2=0x7f4bb419ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:54:22.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:22.826+0000 7f4bb37fe700 1 --2- 172.21.15.52:0/3243784278 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4bb4102e80 0x7f4bb419ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:22.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:22.826+0000 7f4bb37fe700 1 -- 172.21.15.52:0/3243784278 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4bb41024e0 msgr2=0x7f4bb419e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:54:22.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:22.826+0000 7f4bb37fe700 1 --2- 172.21.15.52:0/3243784278 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4bb41024e0 0x7f4bb419e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:22.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:22.826+0000 7f4bb37fe700 1 -- 172.21.15.52:0/3243784278 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bb4199250 con 0x7f4bb41084f0 2024-06-10T02:54:22.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:22.826+0000 7f4bb2ffd700 1 --2- 172.21.15.52:0/3243784278 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4bb41024e0 0x7f4bb419e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T02:54:22.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:22.827+0000 7f4bb37fe700 1 --2- 172.21.15.52:0/3243784278 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4bb41084f0 0x7f4bb4198af0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f4bac00b970 tx=0x7f4bac00bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:54:22.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:22.827+0000 7f4bb8a4b700 1 -- 172.21.15.52:0/3243784278 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bac019070 con 0x7f4bb41084f0 2024-06-10T02:54:22.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:22.827+0000 7f4bb8a4b700 1 -- 172.21.15.52:0/3243784278 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4bac00ce10 con 0x7f4bb41084f0 2024-06-10T02:54:22.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:22.827+0000 7f4bb8a4b700 1 -- 172.21.15.52:0/3243784278 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bac007370 con 0x7f4bb41084f0 2024-06-10T02:54:22.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:22.827+0000 7f4bb9a4d700 1 -- 172.21.15.52:0/3243784278 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4bb4199540 con 0x7f4bb41084f0 2024-06-10T02:54:22.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:22.827+0000 7f4bb9a4d700 1 -- 172.21.15.52:0/3243784278 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4bb41a5800 con 0x7f4bb41084f0 2024-06-10T02:54:22.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:22.828+0000 7f4bb9a4d700 1 -- 172.21.15.52:0/3243784278 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4bb4069180 con 0x7f4bb41084f0 2024-06-10T02:54:22.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:22.829+0000 7f4bb8a4b700 1 -- 172.21.15.52:0/3243784278 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7f4bac01c070 con 0x7f4bb41084f0 2024-06-10T02:54:22.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:22.830+0000 7f4bb8a4b700 1 --2- 172.21.15.52:0/3243784278 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f4ba00a2cb0 0x7f4ba00a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:22.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:22.830+0000 7f4bb2ffd700 1 --2- 172.21.15.52:0/3243784278 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f4ba00a2cb0 0x7f4ba00a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:22.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:22.830+0000 7f4bb8a4b700 1 -- 172.21.15.52:0/3243784278 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f4bac0cecf0 con 0x7f4bb41084f0 2024-06-10T02:54:22.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:22.831+0000 7f4bb2ffd700 1 --2- 172.21.15.52:0/3243784278 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f4ba00a2cb0 0x7f4ba00a5170 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f4ba4006010 tx=0x7f4ba400b540 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:54:22.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:22.834+0000 7f4bb8a4b700 1 -- 172.21.15.52:0/3243784278 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f4bac09b850 con 0x7f4bb41084f0 2024-06-10T02:54:22.976 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:54:22 smithi052 ceph-mon[32362]: pgmap v439: 129 pgs: 129 active+clean; 656 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2024-06-10T02:54:23.036 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.037+0000 7f4bb9a4d700 1 -- 172.21.15.52:0/3243784278 --> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4bb40628d0 con 0x7f4ba00a2cb0 2024-06-10T02:54:23.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.040+0000 7f0e97477700 1 -- 172.21.15.52:0/1600542384 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0e9006b3c0 msgr2=0x7f0e9010f310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:54:23.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.040+0000 7f0e97477700 1 --2- 172.21.15.52:0/1600542384 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0e9006b3c0 0x7f0e9010f310 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f0e8800b0a0 tx=0x7f0e8800b3b0 comp rx=0 tx=0).stop 2024-06-10T02:54:23.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.040+0000 7f0e97477700 1 -- 172.21.15.52:0/1600542384 shutdown_connections 2024-06-10T02:54:23.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.040+0000 7f0e97477700 1 --2- 172.21.15.52:0/1600542384 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0e9006b3c0 0x7f0e9010f310 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:23.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.040+0000 7f0e97477700 1 --2- 172.21.15.52:0/1600542384 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0e901084f0 0x7f0e901088d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:23.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.040+0000 7f0e97477700 1 --2- 172.21.15.52:0/1600542384 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0e9006aa00 0x7f0e9006ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:23.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.040+0000 7f0e97477700 1 -- 172.21.15.52:0/1600542384 >> 172.21.15.52:0/1600542384 conn(0x7f0e90078300 msgr2=0x7f0e90078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:54:23.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.041+0000 7f0e97477700 1 -- 172.21.15.52:0/1600542384 shutdown_connections 2024-06-10T02:54:23.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.041+0000 7f0e97477700 1 -- 172.21.15.52:0/1600542384 wait complete. 2024-06-10T02:54:23.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.041+0000 7f0e97477700 1 Processor -- start 2024-06-10T02:54:23.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.041+0000 7f0e97477700 1 -- start start 2024-06-10T02:54:23.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.042+0000 7f0e97477700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0e9006aa00 0x7f0e9019e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:23.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.042+0000 7f0e97477700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0e9006b3c0 0x7f0e9019ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:23.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.042+0000 7f0e97477700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0e901084f0 0x7f0e90198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:23.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.042+0000 7f0e97477700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f0e901127d0 con 0x7f0e901084f0 2024-06-10T02:54:23.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.042+0000 7f0e97477700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f0e90112650 con 0x7f0e9006aa00 2024-06-10T02:54:23.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.042+0000 7f0e97477700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f0e90112950 con 0x7f0e9006b3c0 2024-06-10T02:54:23.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.042+0000 7f0e95a14700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0e901084f0 0x7f0e90198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:23.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.042+0000 7f0e94a12700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0e9006b3c0 0x7f0e9019ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:23.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.042+0000 7f0e94a12700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0e9006b3c0 0x7f0e9019ee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.52:50302/0 (socket says 172.21.15.52:50302) 2024-06-10T02:54:23.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.042+0000 7f0e95a14700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0e901084f0 0x7f0e90198af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37498/0 (socket says 172.21.15.52:37498) 2024-06-10T02:54:23.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.042+0000 7f0e94a12700 1 -- 172.21.15.52:0/939455489 learned_addr learned my addr 172.21.15.52:0/939455489 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:54:23.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.042+0000 7f0e95213700 1 --2- 172.21.15.52:0/939455489 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0e9006aa00 0x7f0e9019e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:23.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.042+0000 7f0e94a12700 1 -- 172.21.15.52:0/939455489 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0e9006aa00 msgr2=0x7f0e9019e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:54:23.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.042+0000 7f0e94a12700 1 --2- 172.21.15.52:0/939455489 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0e9006aa00 0x7f0e9019e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:23.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.042+0000 7f0e94a12700 1 -- 172.21.15.52:0/939455489 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0e901084f0 msgr2=0x7f0e90198af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:54:23.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.042+0000 7f0e94a12700 1 --2- 172.21.15.52:0/939455489 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0e901084f0 0x7f0e90198af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:23.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.042+0000 7f0e94a12700 1 -- 172.21.15.52:0/939455489 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e90199250 con 0x7f0e9006b3c0 2024-06-10T02:54:23.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.042+0000 7f0e95213700 1 --2- 172.21.15.52:0/939455489 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0e9006aa00 0x7f0e9019e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T02:54:23.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.043+0000 7f0e94a12700 1 --2- 172.21.15.52:0/939455489 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0e9006b3c0 0x7f0e9019ee20 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f0e8000b530 tx=0x7f0e8000b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:54:23.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.043+0000 7f0e867fc700 1 -- 172.21.15.52:0/939455489 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e80015020 con 0x7f0e9006b3c0 2024-06-10T02:54:23.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.043+0000 7f0e97477700 1 -- 172.21.15.52:0/939455489 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e90199540 con 0x7f0e9006b3c0 2024-06-10T02:54:23.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.043+0000 7f0e867fc700 1 -- 172.21.15.52:0/939455489 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e80011d10 con 0x7f0e9006b3c0 2024-06-10T02:54:23.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.043+0000 7f0e867fc700 1 -- 172.21.15.52:0/939455489 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e8000f2e0 con 0x7f0e9006b3c0 2024-06-10T02:54:23.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.043+0000 7f0e97477700 1 -- 172.21.15.52:0/939455489 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e901a57b0 con 0x7f0e9006b3c0 2024-06-10T02:54:23.046 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.045+0000 7f0e867fc700 1 -- 172.21.15.52:0/939455489 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7f0e80018020 con 0x7f0e9006b3c0 2024-06-10T02:54:23.046 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.046+0000 7f0e867fc700 1 --2- 172.21.15.52:0/939455489 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f0e7c0a2cb0 0x7f0e7c0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:23.046 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.046+0000 7f0e95213700 1 --2- 172.21.15.52:0/939455489 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f0e7c0a2cb0 0x7f0e7c0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:23.047 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.046+0000 7f0e867fc700 1 -- 172.21.15.52:0/939455489 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f0e800cece0 con 0x7f0e9006b3c0 2024-06-10T02:54:23.047 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.047+0000 7f0e95213700 1 --2- 172.21.15.52:0/939455489 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f0e7c0a2cb0 0x7f0e7c0a5170 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f0e900ffb40 tx=0x7f0e8c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:54:23.047 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.048+0000 7f0ea1cfab80 1 Processor -- start 2024-06-10T02:54:23.047 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.048+0000 7f0ea1cfab80 1 -- start start 2024-06-10T02:54:23.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.049+0000 7f0ea1cfab80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dd6a38c6c0 0x55dd6a38caa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:23.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.049+0000 7f0ea1cfab80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dd6a38cfe0 0x55dd6a386a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:23.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.049+0000 7f0ea1cfab80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dd6a386fa0 0x55dd6a3873c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:23.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.049+0000 7f0ea1cfab80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55dd6a2d8890 con 0x55dd6a38c6c0 2024-06-10T02:54:23.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.049+0000 7f0ea1cfab80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55dd6a2807a0 con 0x55dd6a386fa0 2024-06-10T02:54:23.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.049+0000 7f0ea1cfab80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55dd6a2d4850 con 0x55dd6a38cfe0 2024-06-10T02:54:23.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.049+0000 7f0e95213700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dd6a38c6c0 0x55dd6a38caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:23.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.049+0000 7f0e95213700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dd6a38c6c0 0x55dd6a38caa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37508/0 (socket says 172.21.15.52:37508) 2024-06-10T02:54:23.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.049+0000 7f0e95213700 1 -- 172.21.15.52:0/3764243906 learned_addr learned my addr 172.21.15.52:0/3764243906 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:54:23.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.049+0000 7f0e95a14700 1 --2- 172.21.15.52:0/3764243906 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dd6a386fa0 0x55dd6a3873c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:23.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.049+0000 7f0e94a12700 1 --2- 172.21.15.52:0/3764243906 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dd6a38cfe0 0x55dd6a386a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:23.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.049+0000 7f0e95213700 1 -- 172.21.15.52:0/3764243906 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dd6a38cfe0 msgr2=0x55dd6a386a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:54:23.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.049+0000 7f0e95213700 1 --2- 172.21.15.52:0/3764243906 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dd6a38cfe0 0x55dd6a386a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:23.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.049+0000 7f0e95213700 1 -- 172.21.15.52:0/3764243906 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dd6a386fa0 msgr2=0x55dd6a3873c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:54:23.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.049+0000 7f0e95213700 1 --2- 172.21.15.52:0/3764243906 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dd6a386fa0 0x55dd6a3873c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:23.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.049+0000 7f0e95213700 1 -- 172.21.15.52:0/3764243906 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dd6a276fd0 con 0x55dd6a38c6c0 2024-06-10T02:54:23.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.049+0000 7f0e95a14700 1 --2- 172.21.15.52:0/3764243906 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dd6a386fa0 0x55dd6a3873c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T02:54:23.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.049+0000 7f0e95213700 1 --2- 172.21.15.52:0/3764243906 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dd6a38c6c0 0x55dd6a38caa0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f0e8c011ed0 tx=0x7f0e8c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ba515b177d37e69 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:54:23.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.050+0000 7f0e7b7fe700 1 -- 172.21.15.52:0/3764243906 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e8c01f620 con 0x55dd6a38c6c0 2024-06-10T02:54:23.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.050+0000 7f0e7b7fe700 1 -- 172.21.15.52:0/3764243906 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e8c01fca0 con 0x55dd6a38c6c0 2024-06-10T02:54:23.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.050+0000 7f0e7b7fe700 1 -- 172.21.15.52:0/3764243906 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e8c01dd50 con 0x55dd6a38c6c0 2024-06-10T02:54:23.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.051+0000 7f0ea1cfab80 1 -- 172.21.15.52:0/3764243906 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dd6a38c6c0 msgr2=0x55dd6a38caa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:54:23.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.051+0000 7f0ea1cfab80 1 --2- 172.21.15.52:0/3764243906 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dd6a38c6c0 0x55dd6a38caa0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f0e8c011ed0 tx=0x7f0e8c014000 comp rx=0 tx=0).stop 2024-06-10T02:54:23.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.051+0000 7f0ea1cfab80 1 -- 172.21.15.52:0/3764243906 shutdown_connections 2024-06-10T02:54:23.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.051+0000 7f0ea1cfab80 1 --2- 172.21.15.52:0/3764243906 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dd6a386fa0 0x55dd6a3873c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:23.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.051+0000 7f0ea1cfab80 1 --2- 172.21.15.52:0/3764243906 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dd6a38c6c0 0x55dd6a38caa0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:23.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.051+0000 7f0ea1cfab80 1 --2- 172.21.15.52:0/3764243906 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dd6a38cfe0 0x55dd6a386a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:23.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.051+0000 7f0ea1cfab80 1 -- 172.21.15.52:0/3764243906 >> 172.21.15.52:0/3764243906 conn(0x55dd6a2b1d40 msgr2=0x55dd6a3818d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:54:23.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.051+0000 7f0ea1cfab80 1 -- 172.21.15.52:0/3764243906 shutdown_connections 2024-06-10T02:54:23.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.051+0000 7f0ea1cfab80 1 -- 172.21.15.52:0/3764243906 wait complete. 2024-06-10T02:54:23.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.051+0000 7f0ea1cfab80 1 -- start start 2024-06-10T02:54:23.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.051+0000 7f0ea1cfab80 1 Processor -- start 2024-06-10T02:54:23.054 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.052+0000 7f0ea1cfab80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dd6a386fa0 0x55dd6a392d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:23.054 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.052+0000 7f0ea1cfab80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dd6a38c6c0 0x55dd6a390040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:23.054 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.052+0000 7f0ea1cfab80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dd6a38cfe0 0x55dd6a390580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:23.054 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.052+0000 7f0ea1cfab80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55dd6a393a50 con 0x55dd6a386fa0 2024-06-10T02:54:23.054 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.052+0000 7f0ea1cfab80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55dd6a278c70 con 0x55dd6a38cfe0 2024-06-10T02:54:23.054 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.052+0000 7f0ea1cfab80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55dd6a393bd0 con 0x55dd6a38c6c0 2024-06-10T02:54:23.054 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.052+0000 7f0ea1cfab80 10 client.0 _ll_register_callbacks cb 0x55dd6a2729a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T02:54:23.054 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.052+0000 7f0e95a14700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dd6a38cfe0 0x55dd6a390580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:23.054 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.052+0000 7f0e95a14700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dd6a38cfe0 0x55dd6a390580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.52:58258/0 (socket says 172.21.15.52:58258) 2024-06-10T02:54:23.054 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.052+0000 7f0e95a14700 1 -- 172.21.15.52:0/171241503 learned_addr learned my addr 172.21.15.52:0/171241503 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:54:23.054 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.052+0000 7f0e94a12700 1 --2- 172.21.15.52:0/171241503 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dd6a38c6c0 0x55dd6a390040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:23.054 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.052+0000 7f0e95213700 1 --2- 172.21.15.52:0/171241503 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dd6a386fa0 0x55dd6a392d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:23.054 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.052+0000 7f0e95a14700 1 -- 172.21.15.52:0/171241503 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dd6a38c6c0 msgr2=0x55dd6a390040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:54:23.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.052+0000 7f0e95a14700 1 --2- 172.21.15.52:0/171241503 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dd6a38c6c0 0x55dd6a390040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:23.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.052+0000 7f0e95a14700 1 -- 172.21.15.52:0/171241503 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dd6a386fa0 msgr2=0x55dd6a392d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:54:23.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.052+0000 7f0e95a14700 1 --2- 172.21.15.52:0/171241503 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dd6a386fa0 0x55dd6a392d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:23.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.052+0000 7f0e95a14700 1 -- 172.21.15.52:0/171241503 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dd6a227950 con 0x55dd6a38cfe0 2024-06-10T02:54:23.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.052+0000 7f0e95213700 1 --2- 172.21.15.52:0/171241503 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dd6a386fa0 0x55dd6a392d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T02:54:23.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.052+0000 7f4bb8a4b700 1 -- 172.21.15.52:0/3243784278 <== mgr.14150 v2:172.21.15.52:6800/3214763969 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+341021 (secure 0 0 0) 0x7f4bb40628d0 con 0x7f4ba00a2cb0 2024-06-10T02:54:23.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.053+0000 7f0e95a14700 1 --2- 172.21.15.52:0/171241503 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dd6a38cfe0 0x55dd6a390580 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f0e8800be50 tx=0x7f0e8800be80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c725ca4d67757fab server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:54:23.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.053+0000 7f0e78ff9700 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-06-10T02:54:23.056 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-10T02:54:23.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.053+0000 7f0e78ff9700 1 -- 172.21.15.52:0/171241503 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e880074e0 con 0x55dd6a38cfe0 2024-06-10T02:54:23.058 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.053+0000 7f0ea1cfab80 1 -- 172.21.15.52:0/171241503 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0e8c01fca0 con 0x55dd6a38cfe0 2024-06-10T02:54:23.058 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.053+0000 7f0e78ff9700 1 -- 172.21.15.52:0/171241503 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f0e88007680 con 0x55dd6a38cfe0 2024-06-10T02:54:23.058 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.053+0000 7f0e78ff9700 1 -- 172.21.15.52:0/171241503 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f0e880078c0 con 0x55dd6a38cfe0 2024-06-10T02:54:23.058 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.053+0000 7f0ea1cfab80 10 client.25055 fetch_fsmap learned FSMap version 30 2024-06-10T02:54:23.058 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.053+0000 7f0ea1cfab80 1 -- 172.21.15.52:0/171241503 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55dd6a39e310 con 0x55dd6a38cfe0 2024-06-10T02:54:23.059 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.053+0000 7f0e78ff9700 1 -- 172.21.15.52:0/171241503 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e8c01fca0 con 0x55dd6a38cfe0 2024-06-10T02:54:23.059 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.054+0000 7f0e78ff9700 1 -- 172.21.15.52:0/171241503 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f0e88011070 con 0x55dd6a38cfe0 2024-06-10T02:54:23.059 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.054+0000 7f0ea1cfab80 10 client.25055 fetch_fsmap finished waiting for FSMap version 30 2024-06-10T02:54:23.059 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.054+0000 7f0ea1cfab80 10 client.25055 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T02:54:23.059 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.054+0000 7f0ea1cfab80 20 client.25055 populate_metadata read hostname 'smithi052' 2024-06-10T02:54:23.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.054+0000 7f0ea1cfab80 1 --2- 172.21.15.52:0/171241503 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55dd6a3a08a0 0x55dd6a3a2c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:23.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.054+0000 7f0ea1cfab80 4 client.25055 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T02:54:23.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.054+0000 7f0ea1cfab80 1 -- 172.21.15.52:0/171241503 --> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dd6a2e5d90 con 0x55dd6a3a08a0 2024-06-10T02:54:23.061 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.054+0000 7f0e95213700 1 --2- 172.21.15.52:0/171241503 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55dd6a3a08a0 0x55dd6a3a2c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:23.063 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.055+0000 7f0e95213700 1 --2- 172.21.15.52:0/171241503 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55dd6a3a08a0 0x55dd6a3a2c90 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=65b9d4597f62def6 server_cookie=b7b74f8868b2a098 in_seq=0 out_seq=0 2024-06-10T02:54:23.063 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.055+0000 7f0e78ff9700 10 client.25055 ms_handle_connect on v2:172.21.15.188:6838/310444836 2024-06-10T02:54:23.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.055+0000 7f0e78ff9700 1 -- 172.21.15.52:0/171241503 <== mds.0 v2:172.21.15.188:6838/310444836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55dd6a2e5d90 con 0x55dd6a3a08a0 2024-06-10T02:54:23.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.055+0000 7f0e78ff9700 10 client.25055 handle_command_reply: tid=0 2024-06-10T02:54:23.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.055+0000 7f0ea1cfab80 1 client.25055 shutdown 2024-06-10T02:54:23.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.056+0000 7f0ea1cfab80 1 -- 172.21.15.52:0/171241503 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dd6a38cfe0 msgr2=0x55dd6a390580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:54:23.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.056+0000 7f0ea1cfab80 1 --2- 172.21.15.52:0/171241503 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dd6a38cfe0 0x55dd6a390580 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f0e8800be50 tx=0x7f0e8800be80 comp rx=0 tx=0).stop 2024-06-10T02:54:23.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.056+0000 7f0ea1cfab80 1 -- 172.21.15.52:0/171241503 shutdown_connections 2024-06-10T02:54:23.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.056+0000 7f0ea1cfab80 1 --2- 172.21.15.52:0/171241503 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55dd6a3a08a0 0x55dd6a3a2c90 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:23.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.056+0000 7f0ea1cfab80 1 --2- 172.21.15.52:0/171241503 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dd6a38cfe0 0x55dd6a390580 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:23.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.056+0000 7f0ea1cfab80 1 --2- 172.21.15.52:0/171241503 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dd6a386fa0 0x55dd6a392d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:23.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.056+0000 7f0e78ff9700 0 client.25055 ms_handle_reset on v2:172.21.15.188:6838/310444836 2024-06-10T02:54:23.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.056+0000 7f0ea1cfab80 1 --2- 172.21.15.52:0/171241503 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dd6a38c6c0 0x55dd6a390040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:23.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.056+0000 7f0ea1cfab80 1 -- 172.21.15.52:0/171241503 >> 172.21.15.52:0/171241503 conn(0x55dd6a2b1d40 msgr2=0x55dd6a385a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:54:23.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.056+0000 7f0ea1cfab80 1 -- 172.21.15.52:0/171241503 shutdown_connections 2024-06-10T02:54:23.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.056+0000 7f0ea1cfab80 1 -- 172.21.15.52:0/171241503 wait complete. 2024-06-10T02:54:23.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.056+0000 7f0ea1cfab80 20 client.25055 trim_cache size 0 max 16384 2024-06-10T02:54:23.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.056+0000 7f4bb9a4d700 1 -- 172.21.15.52:0/3243784278 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f4ba00a2cb0 msgr2=0x7f4ba00a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:54:23.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.056+0000 7f4bb9a4d700 1 --2- 172.21.15.52:0/3243784278 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f4ba00a2cb0 0x7f4ba00a5170 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f4ba4006010 tx=0x7f4ba400b540 comp rx=0 tx=0).stop 2024-06-10T02:54:23.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.056+0000 7f4bb9a4d700 1 -- 172.21.15.52:0/3243784278 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4bb41084f0 msgr2=0x7f4bb4198af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:54:23.068 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.057+0000 7f4bb9a4d700 1 --2- 172.21.15.52:0/3243784278 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4bb41084f0 0x7f4bb4198af0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f4bac00b970 tx=0x7f4bac00bc80 comp rx=0 tx=0).stop 2024-06-10T02:54:23.068 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.057+0000 7f4bb9a4d700 1 -- 172.21.15.52:0/3243784278 shutdown_connections 2024-06-10T02:54:23.068 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.057+0000 7f4bb9a4d700 1 --2- 172.21.15.52:0/3243784278 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f4ba00a2cb0 0x7f4ba00a5170 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:23.068 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.057+0000 7f4bb9a4d700 1 --2- 172.21.15.52:0/3243784278 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4bb41084f0 0x7f4bb4198af0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:23.068 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.057+0000 7f4bb9a4d700 1 --2- 172.21.15.52:0/3243784278 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4bb41024e0 0x7f4bb419e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:23.068 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.057+0000 7f4bb9a4d700 1 --2- 172.21.15.52:0/3243784278 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4bb4102e80 0x7f4bb419ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:23.068 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.057+0000 7f4bb9a4d700 1 -- 172.21.15.52:0/3243784278 >> 172.21.15.52:0/3243784278 conn(0x7f4bb4079530 msgr2=0x7f4bb4100690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:54:23.068 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.057+0000 7f4bb9a4d700 1 -- 172.21.15.52:0/3243784278 shutdown_connections 2024-06-10T02:54:23.068 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.057+0000 7f4bb9a4d700 1 -- 172.21.15.52:0/3243784278 wait complete. 2024-06-10T02:54:23.068 INFO:teuthology.orchestra.run.smithi052.stderr:dumped all 2024-06-10T02:54:23.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.078+0000 7f0ea1cfab80 1 Processor -- start 2024-06-10T02:54:23.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.078+0000 7f0ea1cfab80 1 -- start start 2024-06-10T02:54:23.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.078+0000 7f0ea1cfab80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dd6a386fa0 0x55dd6a386a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:23.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.078+0000 7f0ea1cfab80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dd6a38c6c0 0x55dd6a37b710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:23.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.078+0000 7f0ea1cfab80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dd6a38cfe0 0x55dd6a38dad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:23.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.078+0000 7f0ea1cfab80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f0e901ac1b0 con 0x55dd6a386fa0 2024-06-10T02:54:23.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.078+0000 7f0ea1cfab80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f0e901adcf0 con 0x55dd6a38cfe0 2024-06-10T02:54:23.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.078+0000 7f0ea1cfab80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f0e901ac880 con 0x55dd6a38c6c0 2024-06-10T02:54:23.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.079+0000 7f0e95213700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dd6a386fa0 0x55dd6a386a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:23.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.079+0000 7f0e95213700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dd6a386fa0 0x55dd6a386a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37524/0 (socket says 172.21.15.52:37524) 2024-06-10T02:54:23.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.079+0000 7f0e95213700 1 -- 172.21.15.52:0/3250956990 learned_addr learned my addr 172.21.15.52:0/3250956990 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:54:23.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.079+0000 7f0e94a12700 1 --2- 172.21.15.52:0/3250956990 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dd6a38c6c0 0x55dd6a37b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:23.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.079+0000 7f0e95a14700 1 --2- 172.21.15.52:0/3250956990 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dd6a38cfe0 0x55dd6a38dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:23.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.079+0000 7f0e95213700 1 -- 172.21.15.52:0/3250956990 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dd6a38c6c0 msgr2=0x55dd6a37b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:54:23.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.079+0000 7f0e95213700 1 --2- 172.21.15.52:0/3250956990 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dd6a38c6c0 0x55dd6a37b710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:23.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.079+0000 7f0e95213700 1 -- 172.21.15.52:0/3250956990 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dd6a38cfe0 msgr2=0x55dd6a38dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:54:23.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.079+0000 7f0e95213700 1 --2- 172.21.15.52:0/3250956990 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dd6a38cfe0 0x55dd6a38dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:23.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.079+0000 7f0e95213700 1 -- 172.21.15.52:0/3250956990 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dd6a38e010 con 0x55dd6a386fa0 2024-06-10T02:54:23.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.079+0000 7f0e94a12700 1 --2- 172.21.15.52:0/3250956990 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dd6a38c6c0 0x55dd6a37b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T02:54:23.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.079+0000 7f0e95213700 1 --2- 172.21.15.52:0/3250956990 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dd6a386fa0 0x55dd6a386a60 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f0e8c011ed0 tx=0x7f0e8c009830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=187258e9148d4914 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:54:23.082 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.079+0000 7f0e7affd700 1 -- 172.21.15.52:0/3250956990 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e8c01dea0 con 0x55dd6a386fa0 2024-06-10T02:54:23.082 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.079+0000 7f0e7affd700 1 -- 172.21.15.52:0/3250956990 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e8c01fe50 con 0x55dd6a386fa0 2024-06-10T02:54:23.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.079+0000 7f0e7affd700 1 -- 172.21.15.52:0/3250956990 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e8c0055b0 con 0x55dd6a386fa0 2024-06-10T02:54:23.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.080+0000 7f0ea1cfab80 1 -- 172.21.15.52:0/3250956990 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dd6a386fa0 msgr2=0x55dd6a386a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:54:23.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.080+0000 7f0ea1cfab80 1 --2- 172.21.15.52:0/3250956990 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dd6a386fa0 0x55dd6a386a60 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f0e8c011ed0 tx=0x7f0e8c009830 comp rx=0 tx=0).stop 2024-06-10T02:54:23.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.080+0000 7f0ea1cfab80 1 -- 172.21.15.52:0/3250956990 shutdown_connections 2024-06-10T02:54:23.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.080+0000 7f0ea1cfab80 1 --2- 172.21.15.52:0/3250956990 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dd6a38cfe0 0x55dd6a38dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:23.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.080+0000 7f0ea1cfab80 1 --2- 172.21.15.52:0/3250956990 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dd6a386fa0 0x55dd6a386a60 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:23.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.080+0000 7f0ea1cfab80 1 --2- 172.21.15.52:0/3250956990 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dd6a38c6c0 0x55dd6a37b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:23.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.080+0000 7f0ea1cfab80 1 -- 172.21.15.52:0/3250956990 >> 172.21.15.52:0/3250956990 conn(0x55dd6a2b1d40 msgr2=0x55dd6a38d3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:54:23.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.080+0000 7f0ea1cfab80 1 -- 172.21.15.52:0/3250956990 shutdown_connections 2024-06-10T02:54:23.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.080+0000 7f0ea1cfab80 1 -- 172.21.15.52:0/3250956990 wait complete. 2024-06-10T02:54:23.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.081+0000 7f0ea1cfab80 1 -- start start 2024-06-10T02:54:23.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.081+0000 7f0ea1cfab80 1 Processor -- start 2024-06-10T02:54:23.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.081+0000 7f0ea1cfab80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dd6a386fa0 0x55dd6a398e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:23.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.081+0000 7f0ea1cfab80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dd6a38c6c0 0x55dd6a3993d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:23.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.081+0000 7f0ea1cfab80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dd6a38cfe0 0x55dd6a395e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:23.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.081+0000 7f0ea1cfab80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f0e901ad620 con 0x55dd6a38c6c0 2024-06-10T02:54:23.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.081+0000 7f0ea1cfab80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f0e901acf50 con 0x55dd6a386fa0 2024-06-10T02:54:23.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.081+0000 7f0ea1cfab80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f0e901aa150 con 0x55dd6a38cfe0 2024-06-10T02:54:23.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.081+0000 7f0ea1cfab80 10 client.0 _ll_register_callbacks cb 0x55dd6a2729a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T02:54:23.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.081+0000 7f0e94a12700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dd6a38c6c0 0x55dd6a3993d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:23.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.081+0000 7f0e94a12700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dd6a38c6c0 0x55dd6a3993d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37536/0 (socket says 172.21.15.52:37536) 2024-06-10T02:54:23.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.081+0000 7f0e94a12700 1 -- 172.21.15.52:0/1822621188 learned_addr learned my addr 172.21.15.52:0/1822621188 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:54:23.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.081+0000 7f0e95a14700 1 --2- 172.21.15.52:0/1822621188 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dd6a38cfe0 0x55dd6a395e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:23.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.082+0000 7f0e95213700 1 --2- 172.21.15.52:0/1822621188 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dd6a386fa0 0x55dd6a398e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:23.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.082+0000 7f0e94a12700 1 -- 172.21.15.52:0/1822621188 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dd6a38cfe0 msgr2=0x55dd6a395e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:54:23.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.082+0000 7f0e94a12700 1 --2- 172.21.15.52:0/1822621188 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dd6a38cfe0 0x55dd6a395e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:23.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.082+0000 7f0e94a12700 1 -- 172.21.15.52:0/1822621188 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dd6a386fa0 msgr2=0x55dd6a398e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:54:23.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.082+0000 7f0e94a12700 1 --2- 172.21.15.52:0/1822621188 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dd6a386fa0 0x55dd6a398e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:23.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.082+0000 7f0e94a12700 1 -- 172.21.15.52:0/1822621188 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dd6a399a50 con 0x55dd6a38c6c0 2024-06-10T02:54:23.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.082+0000 7f0e94a12700 1 --2- 172.21.15.52:0/1822621188 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dd6a38c6c0 0x55dd6a3993d0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f0e8009b0d0 tx=0x7f0e80004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4aa932aca9d83e8b server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:54:23.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.082+0000 7f0e78ff9700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-10T02:54:23.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.082+0000 7f0e78ff9700 1 -- 172.21.15.52:0/1822621188 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e8009f4f0 con 0x55dd6a38c6c0 2024-06-10T02:54:23.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.082+0000 7f0e78ff9700 1 -- 172.21.15.52:0/1822621188 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f0e800cc530 con 0x55dd6a38c6c0 2024-06-10T02:54:23.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.082+0000 7f0ea1cfab80 1 -- 172.21.15.52:0/1822621188 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0e8c01fe50 con 0x55dd6a38c6c0 2024-06-10T02:54:23.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.082+0000 7f0e78ff9700 1 -- 172.21.15.52:0/1822621188 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f0e800a3db0 con 0x55dd6a38c6c0 2024-06-10T02:54:23.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.082+0000 7f0ea1cfab80 10 client.15381 fetch_fsmap learned FSMap version 30 2024-06-10T02:54:23.086 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.082+0000 7f0ea1cfab80 1 -- 172.21.15.52:0/1822621188 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55dd6a40bb30 con 0x55dd6a38c6c0 2024-06-10T02:54:23.086 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.083+0000 7f0e78ff9700 1 -- 172.21.15.52:0/1822621188 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e8c01fe50 con 0x55dd6a38c6c0 2024-06-10T02:54:23.086 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.083+0000 7f0e78ff9700 1 -- 172.21.15.52:0/1822621188 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f0e800a3920 con 0x55dd6a38c6c0 2024-06-10T02:54:23.086 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.083+0000 7f0ea1cfab80 10 client.15381 fetch_fsmap finished waiting for FSMap version 30 2024-06-10T02:54:23.086 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.083+0000 7f0ea1cfab80 10 client.15381 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T02:54:23.086 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.083+0000 7f0ea1cfab80 20 client.15381 populate_metadata read hostname 'smithi052' 2024-06-10T02:54:23.086 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.083+0000 7f0ea1cfab80 1 --2- 172.21.15.52:0/1822621188 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55dd6a40c240 0x55dd6a40e660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:23.086 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.083+0000 7f0ea1cfab80 4 client.15381 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-06-10T02:54:23.086 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.083+0000 7f0ea1cfab80 1 -- 172.21.15.52:0/1822621188 --> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55dd6a2e0310 con 0x55dd6a40c240 2024-06-10T02:54:23.086 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.083+0000 7f0e95213700 1 --2- 172.21.15.52:0/1822621188 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55dd6a40c240 0x55dd6a40e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:23.088 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.084+0000 7f0e78ff9700 10 client.15381 ms_handle_connect on v2:172.21.15.188:6838/310444836 2024-06-10T02:54:23.088 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.084+0000 7f0e95213700 1 --2- 172.21.15.52:0/1822621188 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55dd6a40c240 0x55dd6a40e660 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=110c7f64d2cdd556 server_cookie=bc156fe6d6785be4 in_seq=0 out_seq=0 2024-06-10T02:54:23.088 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.086+0000 7f0e78ff9700 1 -- 172.21.15.52:0/1822621188 <== mds.0 v2:172.21.15.188:6838/310444836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55dd6a2e0310 con 0x55dd6a40c240 2024-06-10T02:54:23.088 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.086+0000 7f0e78ff9700 10 client.15381 handle_command_reply: tid=0 2024-06-10T02:54:23.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.086+0000 7f0ea1cfab80 1 client.15381 shutdown 2024-06-10T02:54:23.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.086+0000 7f0ea1cfab80 1 -- 172.21.15.52:0/1822621188 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dd6a38c6c0 msgr2=0x55dd6a3993d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:54:23.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.086+0000 7f0ea1cfab80 1 --2- 172.21.15.52:0/1822621188 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dd6a38c6c0 0x55dd6a3993d0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f0e8009b0d0 tx=0x7f0e80004750 comp rx=0 tx=0).stop 2024-06-10T02:54:23.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.086+0000 7f0ea1cfab80 1 -- 172.21.15.52:0/1822621188 shutdown_connections 2024-06-10T02:54:23.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.086+0000 7f0ea1cfab80 1 --2- 172.21.15.52:0/1822621188 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55dd6a40c240 0x55dd6a40e660 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:23.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.086+0000 7f0ea1cfab80 1 --2- 172.21.15.52:0/1822621188 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dd6a38cfe0 0x55dd6a395e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:23.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.086+0000 7f0ea1cfab80 1 --2- 172.21.15.52:0/1822621188 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dd6a386fa0 0x55dd6a398e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:23.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.086+0000 7f0ea1cfab80 1 --2- 172.21.15.52:0/1822621188 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dd6a38c6c0 0x55dd6a3993d0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:23.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.086+0000 7f0ea1cfab80 1 -- 172.21.15.52:0/1822621188 >> 172.21.15.52:0/1822621188 conn(0x55dd6a2b1d40 msgr2=0x55dd6a38e800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:54:23.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.086+0000 7f0e78ff9700 0 client.15381 ms_handle_reset on v2:172.21.15.188:6838/310444836 2024-06-10T02:54:23.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.087+0000 7f0ea1cfab80 1 -- 172.21.15.52:0/1822621188 shutdown_connections 2024-06-10T02:54:23.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.087+0000 7f0ea1cfab80 1 -- 172.21.15.52:0/1822621188 wait complete. 2024-06-10T02:54:23.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.087+0000 7f0ea1cfab80 20 client.15381 trim_cache size 0 max 16384 2024-06-10T02:54:23.090 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-10T02:54:23.090 INFO:teuthology.orchestra.run.smithi052.stdout: "return_code": 0, 2024-06-10T02:54:23.090 INFO:teuthology.orchestra.run.smithi052.stdout: "scrub_tag": "56a3f2c4-a8fe-4165-bef8-424700e25c1f", 2024-06-10T02:54:23.090 INFO:teuthology.orchestra.run.smithi052.stdout: "mode": "asynchronous" 2024-06-10T02:54:23.090 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-10T02:54:23.091 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.087+0000 7f0e97477700 1 -- 172.21.15.52:0/939455489 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f0e7c0a2cb0 msgr2=0x7f0e7c0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:54:23.091 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.087+0000 7f0e97477700 1 --2- 172.21.15.52:0/939455489 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f0e7c0a2cb0 0x7f0e7c0a5170 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f0e900ffb40 tx=0x7f0e8c009260 comp rx=0 tx=0).stop 2024-06-10T02:54:23.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.087+0000 7f0e97477700 1 -- 172.21.15.52:0/939455489 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0e9006b3c0 msgr2=0x7f0e9019ee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:54:23.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.087+0000 7f0e97477700 1 --2- 172.21.15.52:0/939455489 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0e9006b3c0 0x7f0e9019ee20 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f0e8000b530 tx=0x7f0e8000b840 comp rx=0 tx=0).stop 2024-06-10T02:54:23.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.088+0000 7f0e97477700 1 -- 172.21.15.52:0/939455489 shutdown_connections 2024-06-10T02:54:23.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.088+0000 7f0e97477700 1 --2- 172.21.15.52:0/939455489 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f0e7c0a2cb0 0x7f0e7c0a5170 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:23.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.088+0000 7f0e97477700 1 --2- 172.21.15.52:0/939455489 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0e901084f0 0x7f0e90198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:23.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.088+0000 7f0e97477700 1 --2- 172.21.15.52:0/939455489 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0e9006aa00 0x7f0e9019e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:23.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.088+0000 7f0e97477700 1 --2- 172.21.15.52:0/939455489 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0e9006b3c0 0x7f0e9019ee20 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:23.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.088+0000 7f0e97477700 1 -- 172.21.15.52:0/939455489 >> 172.21.15.52:0/939455489 conn(0x7f0e90078300 msgr2=0x7f0e900fff90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:54:23.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.088+0000 7f0e97477700 1 -- 172.21.15.52:0/939455489 shutdown_connections 2024-06-10T02:54:23.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:23.088+0000 7f0e97477700 1 -- 172.21.15.52:0/939455489 wait complete. 2024-06-10T02:54:23.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:54:22 smithi120 ceph-mon[36634]: pgmap v439: 129 pgs: 129 active+clean; 656 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2024-06-10T02:54:23.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:54:22 smithi188 ceph-mon[36599]: pgmap v439: 129 pgs: 129 active+clean; 656 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2024-06-10T02:54:23.976 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:54:23 smithi052 ceph-mon[32362]: scrub queued for path: / 2024-06-10T02:54:23.976 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:54:23 smithi052 ceph-mon[32362]: scrub summary: idle+waiting paths [/] 2024-06-10T02:54:23.976 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:54:23 smithi052 ceph-mon[32362]: scrub summary: active paths [/] 2024-06-10T02:54:23.976 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:54:23 smithi052 ceph-mon[32362]: scrub summary: idle+waiting paths [/] 2024-06-10T02:54:24.114 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-10T02:54:24.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:54:23 smithi120 ceph-mon[36634]: scrub queued for path: / 2024-06-10T02:54:24.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:54:23 smithi120 ceph-mon[36634]: scrub summary: idle+waiting paths [/] 2024-06-10T02:54:24.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:54:23 smithi120 ceph-mon[36634]: scrub summary: active paths [/] 2024-06-10T02:54:24.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:54:23 smithi120 ceph-mon[36634]: scrub summary: idle+waiting paths [/] 2024-06-10T02:54:24.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:54:23 smithi188 ceph-mon[36599]: scrub queued for path: / 2024-06-10T02:54:24.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:54:23 smithi188 ceph-mon[36599]: scrub summary: idle+waiting paths [/] 2024-06-10T02:54:24.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:54:23 smithi188 ceph-mon[36599]: scrub summary: active paths [/] 2024-06-10T02:54:24.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:54:23 smithi188 ceph-mon[36599]: scrub summary: idle+waiting paths [/] 2024-06-10T02:54:24.516 INFO:teuthology.orchestra.run.smithi052.stdout:{"pg_ready":true,"pg_map":{"version":439,"stamp":"2024-06-10T02:54:21.046207+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":671341,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":2060,"num_read_kb":1166,"num_write":459,"num_write_kb":3113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1181318,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":876,"ondisk_log_size":876,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":379372,"kb_used_data":13580,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124562452,"statfs":{"total":1151940427776,"available":1151551950848,"internally_reserved":0,"allocated":13905920,"data_stored":8864789,"data_compressed":497715,"data_compressed_allocated":675840,"data_compressed_original":2850816,"omap_allocated":19141,"internal_metadata":374453563},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":8673,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":115,"num_read_kb":67,"num_write":8,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003311"},"pg_stats":[{"pgid":"3.3f","version":"100'8","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:43.001975+0000","last_change":"2024-06-10T02:50:02.209146+0000","last_active":"2024-06-10T02:52:43.001975+0000","last_peered":"2024-06-10T02:52:43.001975+0000","last_clean":"2024-06-10T02:52:43.001975+0000","last_became_active":"2024-06-10T02:50:02.208027+0000","last_became_peered":"2024-06-10T02:50:02.208027+0000","last_unstale":"2024-06-10T02:52:43.001975+0000","last_undegraded":"2024-06-10T02:52:43.001975+0000","last_fullsized":"2024-06-10T02:52:43.001975+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:35:34.765849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"100'4","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.995310+0000","last_change":"2024-06-10T02:50:02.198767+0000","last_active":"2024-06-10T02:52:42.995310+0000","last_peered":"2024-06-10T02:52:42.995310+0000","last_clean":"2024-06-10T02:52:42.995310+0000","last_became_active":"2024-06-10T02:50:02.198450+0000","last_became_peered":"2024-06-10T02:50:02.198450+0000","last_unstale":"2024-06-10T02:52:42.995310+0000","last_undegraded":"2024-06-10T02:52:42.995310+0000","last_fullsized":"2024-06-10T02:52:42.995310+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:49:56.666083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"100'7","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:43.000415+0000","last_change":"2024-06-10T02:50:02.208885+0000","last_active":"2024-06-10T02:52:43.000415+0000","last_peered":"2024-06-10T02:52:43.000415+0000","last_clean":"2024-06-10T02:52:43.000415+0000","last_became_active":"2024-06-10T02:50:02.206953+0000","last_became_peered":"2024-06-10T02:50:02.206953+0000","last_unstale":"2024-06-10T02:52:43.000415+0000","last_undegraded":"2024-06-10T02:52:43.000415+0000","last_fullsized":"2024-06-10T02:52:43.000415+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:41:23.928939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"100'8","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.994111+0000","last_change":"2024-06-10T02:50:02.208632+0000","last_active":"2024-06-10T02:52:42.994111+0000","last_peered":"2024-06-10T02:52:42.994111+0000","last_clean":"2024-06-10T02:52:42.994111+0000","last_became_active":"2024-06-10T02:50:02.207587+0000","last_became_peered":"2024-06-10T02:50:02.207587+0000","last_unstale":"2024-06-10T02:52:42.994111+0000","last_undegraded":"2024-06-10T02:52:42.994111+0000","last_fullsized":"2024-06-10T02:52:42.994111+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T14:34:36.720170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"100'4","reported_seq":43,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.973606+0000","last_change":"2024-06-10T02:50:02.197966+0000","last_active":"2024-06-10T02:52:42.973606+0000","last_peered":"2024-06-10T02:52:42.973606+0000","last_clean":"2024-06-10T02:52:42.973606+0000","last_became_active":"2024-06-10T02:50:02.197612+0000","last_became_peered":"2024-06-10T02:50:02.197612+0000","last_unstale":"2024-06-10T02:52:42.973606+0000","last_undegraded":"2024-06-10T02:52:42.973606+0000","last_fullsized":"2024-06-10T02:52:42.973606+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:24:53.804613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"100'6","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.987354+0000","last_change":"2024-06-10T02:50:02.201952+0000","last_active":"2024-06-10T02:52:42.987354+0000","last_peered":"2024-06-10T02:52:42.987354+0000","last_clean":"2024-06-10T02:52:42.987354+0000","last_became_active":"2024-06-10T02:50:02.200828+0000","last_became_peered":"2024-06-10T02:50:02.200828+0000","last_unstale":"2024-06-10T02:52:42.987354+0000","last_undegraded":"2024-06-10T02:52:42.987354+0000","last_fullsized":"2024-06-10T02:52:42.987354+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:46:28.518846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"100'5","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.990208+0000","last_change":"2024-06-10T02:50:02.203483+0000","last_active":"2024-06-10T02:52:42.990208+0000","last_peered":"2024-06-10T02:52:42.990208+0000","last_clean":"2024-06-10T02:52:42.990208+0000","last_became_active":"2024-06-10T02:50:02.203153+0000","last_became_peered":"2024-06-10T02:50:02.203153+0000","last_unstale":"2024-06-10T02:52:42.990208+0000","last_undegraded":"2024-06-10T02:52:42.990208+0000","last_fullsized":"2024-06-10T02:52:42.990208+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:29:30.422449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"100'9","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.995630+0000","last_change":"2024-06-10T02:50:02.201686+0000","last_active":"2024-06-10T02:52:42.995630+0000","last_peered":"2024-06-10T02:52:42.995630+0000","last_clean":"2024-06-10T02:52:42.995630+0000","last_became_active":"2024-06-10T02:50:02.200973+0000","last_became_peered":"2024-06-10T02:50:02.200973+0000","last_unstale":"2024-06-10T02:52:42.995630+0000","last_undegraded":"2024-06-10T02:52:42.995630+0000","last_fullsized":"2024-06-10T02:52:42.995630+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:41:13.712675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"100'11","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:43.001422+0000","last_change":"2024-06-10T02:50:02.210497+0000","last_active":"2024-06-10T02:52:43.001422+0000","last_peered":"2024-06-10T02:52:43.001422+0000","last_clean":"2024-06-10T02:52:43.001422+0000","last_became_active":"2024-06-10T02:50:02.209675+0000","last_became_peered":"2024-06-10T02:50:02.209675+0000","last_unstale":"2024-06-10T02:52:43.001422+0000","last_undegraded":"2024-06-10T02:52:43.001422+0000","last_fullsized":"2024-06-10T02:52:43.001422+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:50:29.879669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"100'16","reported_seq":61,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.994975+0000","last_change":"2024-06-10T02:50:44.741139+0000","last_active":"2024-06-10T02:52:42.994975+0000","last_peered":"2024-06-10T02:52:42.994975+0000","last_clean":"2024-06-10T02:52:42.994975+0000","last_became_active":"2024-06-10T02:50:44.740685+0000","last_became_peered":"2024-06-10T02:50:44.740685+0000","last_unstale":"2024-06-10T02:52:42.994975+0000","last_undegraded":"2024-06-10T02:52:42.994975+0000","last_fullsized":"2024-06-10T02:52:42.994975+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:58:59.872732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"100'6","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.999574+0000","last_change":"2024-06-10T02:50:02.206993+0000","last_active":"2024-06-10T02:52:42.999574+0000","last_peered":"2024-06-10T02:52:42.999574+0000","last_clean":"2024-06-10T02:52:42.999574+0000","last_became_active":"2024-06-10T02:50:02.206650+0000","last_became_peered":"2024-06-10T02:50:02.206650+0000","last_unstale":"2024-06-10T02:52:42.999574+0000","last_undegraded":"2024-06-10T02:52:42.999574+0000","last_fullsized":"2024-06-10T02:52:42.999574+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T14:27:49.555350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"100'7","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.979413+0000","last_change":"2024-06-10T02:50:02.200816+0000","last_active":"2024-06-10T02:52:42.979413+0000","last_peered":"2024-06-10T02:52:42.979413+0000","last_clean":"2024-06-10T02:52:42.979413+0000","last_became_active":"2024-06-10T02:50:02.200503+0000","last_became_peered":"2024-06-10T02:50:02.200503+0000","last_unstale":"2024-06-10T02:52:42.979413+0000","last_undegraded":"2024-06-10T02:52:42.979413+0000","last_fullsized":"2024-06-10T02:52:42.979413+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:01:08.326728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"100'10","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.987233+0000","last_change":"2024-06-10T02:50:02.203644+0000","last_active":"2024-06-10T02:52:42.987233+0000","last_peered":"2024-06-10T02:52:42.987233+0000","last_clean":"2024-06-10T02:52:42.987233+0000","last_became_active":"2024-06-10T02:50:02.203100+0000","last_became_peered":"2024-06-10T02:50:02.203100+0000","last_unstale":"2024-06-10T02:52:42.987233+0000","last_undegraded":"2024-06-10T02:52:42.987233+0000","last_fullsized":"2024-06-10T02:52:42.987233+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:01:42.730089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"100'7","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.995068+0000","last_change":"2024-06-10T02:50:02.203330+0000","last_active":"2024-06-10T02:52:42.995068+0000","last_peered":"2024-06-10T02:52:42.995068+0000","last_clean":"2024-06-10T02:52:42.995068+0000","last_became_active":"2024-06-10T02:50:02.202995+0000","last_became_peered":"2024-06-10T02:50:02.202995+0000","last_unstale":"2024-06-10T02:52:42.995068+0000","last_undegraded":"2024-06-10T02:52:42.995068+0000","last_fullsized":"2024-06-10T02:52:42.995068+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:17:23.170591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"100'7","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.986439+0000","last_change":"2024-06-10T02:50:02.204875+0000","last_active":"2024-06-10T02:52:42.986439+0000","last_peered":"2024-06-10T02:52:42.986439+0000","last_clean":"2024-06-10T02:52:42.986439+0000","last_became_active":"2024-06-10T02:50:02.200706+0000","last_became_peered":"2024-06-10T02:50:02.200706+0000","last_unstale":"2024-06-10T02:52:42.986439+0000","last_undegraded":"2024-06-10T02:52:42.986439+0000","last_fullsized":"2024-06-10T02:52:42.986439+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:27:12.282772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"100'9","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.999375+0000","last_change":"2024-06-10T02:50:02.203187+0000","last_active":"2024-06-10T02:52:42.999375+0000","last_peered":"2024-06-10T02:52:42.999375+0000","last_clean":"2024-06-10T02:52:42.999375+0000","last_became_active":"2024-06-10T02:50:02.202034+0000","last_became_peered":"2024-06-10T02:50:02.202034+0000","last_unstale":"2024-06-10T02:52:42.999375+0000","last_undegraded":"2024-06-10T02:52:42.999375+0000","last_fullsized":"2024-06-10T02:52:42.999375+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:59:44.783084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"100'8","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:43.002372+0000","last_change":"2024-06-10T02:50:02.209102+0000","last_active":"2024-06-10T02:52:43.002372+0000","last_peered":"2024-06-10T02:52:43.002372+0000","last_clean":"2024-06-10T02:52:43.002372+0000","last_became_active":"2024-06-10T02:50:02.208730+0000","last_became_peered":"2024-06-10T02:50:02.208730+0000","last_unstale":"2024-06-10T02:52:43.002372+0000","last_undegraded":"2024-06-10T02:52:43.002372+0000","last_fullsized":"2024-06-10T02:52:43.002372+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:08:38.838415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"100'10","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:43.000459+0000","last_change":"2024-06-10T02:50:02.204008+0000","last_active":"2024-06-10T02:52:43.000459+0000","last_peered":"2024-06-10T02:52:43.000459+0000","last_clean":"2024-06-10T02:52:43.000459+0000","last_became_active":"2024-06-10T02:50:02.203655+0000","last_became_peered":"2024-06-10T02:50:02.203655+0000","last_unstale":"2024-06-10T02:52:43.000459+0000","last_undegraded":"2024-06-10T02:52:43.000459+0000","last_fullsized":"2024-06-10T02:52:43.000459+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:32:15.972662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"100'7","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.991723+0000","last_change":"2024-06-10T02:50:02.204917+0000","last_active":"2024-06-10T02:52:42.991723+0000","last_peered":"2024-06-10T02:52:42.991723+0000","last_clean":"2024-06-10T02:52:42.991723+0000","last_became_active":"2024-06-10T02:50:02.204568+0000","last_became_peered":"2024-06-10T02:50:02.204568+0000","last_unstale":"2024-06-10T02:52:42.991723+0000","last_undegraded":"2024-06-10T02:52:42.991723+0000","last_fullsized":"2024-06-10T02:52:42.991723+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:49:46.802665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.721297+0000","last_change":"2024-06-10T02:50:12.240697+0000","last_active":"2024-06-10T02:51:45.721297+0000","last_peered":"2024-06-10T02:51:45.721297+0000","last_clean":"2024-06-10T02:51:45.721297+0000","last_became_active":"2024-06-10T02:50:12.240229+0000","last_became_peered":"2024-06-10T02:50:12.240229+0000","last_unstale":"2024-06-10T02:51:45.721297+0000","last_undegraded":"2024-06-10T02:51:45.721297+0000","last_fullsized":"2024-06-10T02:51:45.721297+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:45:55.071960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"100'15","reported_seq":265,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:54:18.905596+0000","last_change":"2024-06-10T02:50:44.786783+0000","last_active":"2024-06-10T02:54:18.905596+0000","last_peered":"2024-06-10T02:54:18.905596+0000","last_clean":"2024-06-10T02:54:18.905596+0000","last_became_active":"2024-06-10T02:50:44.781773+0000","last_became_peered":"2024-06-10T02:50:44.781773+0000","last_unstale":"2024-06-10T02:54:18.905596+0000","last_undegraded":"2024-06-10T02:54:18.905596+0000","last_fullsized":"2024-06-10T02:54:18.905596+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:31:06.025854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":192,"num_read_kb":192,"num_write":16,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,4],"acting":[9,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"100'5","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.994824+0000","last_change":"2024-06-10T02:50:02.202573+0000","last_active":"2024-06-10T02:52:42.994824+0000","last_peered":"2024-06-10T02:52:42.994824+0000","last_clean":"2024-06-10T02:52:42.994824+0000","last_became_active":"2024-06-10T02:50:02.202241+0000","last_became_peered":"2024-06-10T02:50:02.202241+0000","last_unstale":"2024-06-10T02:52:42.994824+0000","last_undegraded":"2024-06-10T02:52:42.994824+0000","last_fullsized":"2024-06-10T02:52:42.994824+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:31:40.230851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.383047+0000","last_change":"2024-06-10T02:50:12.238372+0000","last_active":"2024-06-10T02:51:46.383047+0000","last_peered":"2024-06-10T02:51:46.383047+0000","last_clean":"2024-06-10T02:51:46.383047+0000","last_became_active":"2024-06-10T02:50:12.237884+0000","last_became_peered":"2024-06-10T02:50:12.237884+0000","last_unstale":"2024-06-10T02:51:46.383047+0000","last_undegraded":"2024-06-10T02:51:46.383047+0000","last_fullsized":"2024-06-10T02:51:46.383047+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:55:31.432655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"98'2","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:02.117661+0000","last_change":"2024-06-10T02:49:59.031528+0000","last_active":"2024-06-10T02:52:02.117661+0000","last_peered":"2024-06-10T02:52:02.117661+0000","last_clean":"2024-06-10T02:52:02.117661+0000","last_became_active":"2024-06-10T02:49:59.031147+0000","last_became_peered":"2024-06-10T02:49:59.031147+0000","last_unstale":"2024-06-10T02:52:02.117661+0000","last_undegraded":"2024-06-10T02:52:02.117661+0000","last_fullsized":"2024-06-10T02:52:02.117661+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:17:08.534396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"100'6","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:43.000839+0000","last_change":"2024-06-10T02:50:02.202858+0000","last_active":"2024-06-10T02:52:43.000839+0000","last_peered":"2024-06-10T02:52:43.000839+0000","last_clean":"2024-06-10T02:52:43.000839+0000","last_became_active":"2024-06-10T02:50:02.202263+0000","last_became_peered":"2024-06-10T02:50:02.202263+0000","last_unstale":"2024-06-10T02:52:43.000839+0000","last_undegraded":"2024-06-10T02:52:43.000839+0000","last_fullsized":"2024-06-10T02:52:43.000839+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:23:15.382762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.383227+0000","last_change":"2024-06-10T02:50:12.237370+0000","last_active":"2024-06-10T02:51:46.383227+0000","last_peered":"2024-06-10T02:51:46.383227+0000","last_clean":"2024-06-10T02:51:46.383227+0000","last_became_active":"2024-06-10T02:50:12.236803+0000","last_became_peered":"2024-06-10T02:50:12.236803+0000","last_unstale":"2024-06-10T02:51:46.383227+0000","last_undegraded":"2024-06-10T02:51:46.383227+0000","last_fullsized":"2024-06-10T02:51:46.383227+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:06:40.695710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"98'12","reported_seq":58,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.724307+0000","last_change":"2024-06-10T02:49:59.000911+0000","last_active":"2024-06-10T02:51:45.724307+0000","last_peered":"2024-06-10T02:51:45.724307+0000","last_clean":"2024-06-10T02:51:45.724307+0000","last_became_active":"2024-06-10T02:49:59.000571+0000","last_became_peered":"2024-06-10T02:49:59.000571+0000","last_unstale":"2024-06-10T02:51:45.724307+0000","last_undegraded":"2024-06-10T02:51:45.724307+0000","last_fullsized":"2024-06-10T02:51:45.724307+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:49:24.635793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"100'9","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.997165+0000","last_change":"2024-06-10T02:50:02.209614+0000","last_active":"2024-06-10T02:52:42.997165+0000","last_peered":"2024-06-10T02:52:42.997165+0000","last_clean":"2024-06-10T02:52:42.997165+0000","last_became_active":"2024-06-10T02:50:02.209251+0000","last_became_peered":"2024-06-10T02:50:02.209251+0000","last_unstale":"2024-06-10T02:52:42.997165+0000","last_undegraded":"2024-06-10T02:52:42.997165+0000","last_fullsized":"2024-06-10T02:52:42.997165+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:35:32.256081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.383229+0000","last_change":"2024-06-10T02:50:12.237811+0000","last_active":"2024-06-10T02:51:46.383229+0000","last_peered":"2024-06-10T02:51:46.383229+0000","last_clean":"2024-06-10T02:51:46.383229+0000","last_became_active":"2024-06-10T02:50:12.237017+0000","last_became_peered":"2024-06-10T02:50:12.237017+0000","last_unstale":"2024-06-10T02:51:46.383229+0000","last_undegraded":"2024-06-10T02:51:46.383229+0000","last_fullsized":"2024-06-10T02:51:46.383229+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:14:29.818885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"98'5","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.723967+0000","last_change":"2024-06-10T02:49:58.996971+0000","last_active":"2024-06-10T02:51:45.723967+0000","last_peered":"2024-06-10T02:51:45.723967+0000","last_clean":"2024-06-10T02:51:45.723967+0000","last_became_active":"2024-06-10T02:49:58.996601+0000","last_became_peered":"2024-06-10T02:49:58.996601+0000","last_unstale":"2024-06-10T02:51:45.723967+0000","last_undegraded":"2024-06-10T02:51:45.723967+0000","last_fullsized":"2024-06-10T02:51:45.723967+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:26:45.963351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"100'15","reported_seq":60,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:43.002596+0000","last_change":"2024-06-10T02:50:44.745816+0000","last_active":"2024-06-10T02:52:43.002596+0000","last_peered":"2024-06-10T02:52:43.002596+0000","last_clean":"2024-06-10T02:52:43.002596+0000","last_became_active":"2024-06-10T02:50:44.745386+0000","last_became_peered":"2024-06-10T02:50:44.745386+0000","last_unstale":"2024-06-10T02:52:43.002596+0000","last_undegraded":"2024-06-10T02:52:43.002596+0000","last_fullsized":"2024-06-10T02:52:43.002596+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:08:53.977133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.721700+0000","last_change":"2024-06-10T02:50:12.236835+0000","last_active":"2024-06-10T02:51:45.721700+0000","last_peered":"2024-06-10T02:51:45.721700+0000","last_clean":"2024-06-10T02:51:45.721700+0000","last_became_active":"2024-06-10T02:50:12.236309+0000","last_became_peered":"2024-06-10T02:50:12.236309+0000","last_unstale":"2024-06-10T02:51:45.721700+0000","last_undegraded":"2024-06-10T02:51:45.721700+0000","last_fullsized":"2024-06-10T02:51:45.721700+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:09:02.789286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"100'8","reported_seq":245,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:54:14.901154+0000","last_change":"2024-06-10T02:49:58.997725+0000","last_active":"2024-06-10T02:54:14.901154+0000","last_peered":"2024-06-10T02:54:14.901154+0000","last_clean":"2024-06-10T02:54:14.901154+0000","last_became_active":"2024-06-10T02:49:58.997375+0000","last_became_peered":"2024-06-10T02:49:58.997375+0000","last_unstale":"2024-06-10T02:54:14.901154+0000","last_undegraded":"2024-06-10T02:54:14.901154+0000","last_fullsized":"2024-06-10T02:54:14.901154+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:28:49.714356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1461,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":192,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"100'7","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.995883+0000","last_change":"2024-06-10T02:50:02.201247+0000","last_active":"2024-06-10T02:52:42.995883+0000","last_peered":"2024-06-10T02:52:42.995883+0000","last_clean":"2024-06-10T02:52:42.995883+0000","last_became_active":"2024-06-10T02:50:02.200897+0000","last_became_peered":"2024-06-10T02:50:02.200897+0000","last_unstale":"2024-06-10T02:52:42.995883+0000","last_undegraded":"2024-06-10T02:52:42.995883+0000","last_fullsized":"2024-06-10T02:52:42.995883+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:01:58.166644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.721648+0000","last_change":"2024-06-10T02:50:12.243097+0000","last_active":"2024-06-10T02:51:45.721648+0000","last_peered":"2024-06-10T02:51:45.721648+0000","last_clean":"2024-06-10T02:51:45.721648+0000","last_became_active":"2024-06-10T02:50:12.242501+0000","last_became_peered":"2024-06-10T02:50:12.242501+0000","last_unstale":"2024-06-10T02:51:45.721648+0000","last_undegraded":"2024-06-10T02:51:45.721648+0000","last_fullsized":"2024-06-10T02:51:45.721648+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T14:42:44.220340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"98'7","reported_seq":52,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.720539+0000","last_change":"2024-06-10T02:49:58.996956+0000","last_active":"2024-06-10T02:51:45.720539+0000","last_peered":"2024-06-10T02:51:45.720539+0000","last_clean":"2024-06-10T02:51:45.720539+0000","last_became_active":"2024-06-10T02:49:58.996634+0000","last_became_peered":"2024-06-10T02:49:58.996634+0000","last_unstale":"2024-06-10T02:51:45.720539+0000","last_undegraded":"2024-06-10T02:51:45.720539+0000","last_fullsized":"2024-06-10T02:51:45.720539+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:25:02.364472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"100'8","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.995286+0000","last_change":"2024-06-10T02:50:02.205808+0000","last_active":"2024-06-10T02:52:42.995286+0000","last_peered":"2024-06-10T02:52:42.995286+0000","last_clean":"2024-06-10T02:52:42.995286+0000","last_became_active":"2024-06-10T02:50:02.203558+0000","last_became_peered":"2024-06-10T02:50:02.203558+0000","last_unstale":"2024-06-10T02:52:42.995286+0000","last_undegraded":"2024-06-10T02:52:42.995286+0000","last_fullsized":"2024-06-10T02:52:42.995286+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T13:43:23.315375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.365641+0000","last_change":"2024-06-10T02:50:12.242005+0000","last_active":"2024-06-10T02:51:46.365641+0000","last_peered":"2024-06-10T02:51:46.365641+0000","last_clean":"2024-06-10T02:51:46.365641+0000","last_became_active":"2024-06-10T02:50:12.241295+0000","last_became_peered":"2024-06-10T02:50:12.241295+0000","last_unstale":"2024-06-10T02:51:46.365641+0000","last_undegraded":"2024-06-10T02:51:46.365641+0000","last_fullsized":"2024-06-10T02:51:46.365641+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:32:52.702756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"98'3","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.714696+0000","last_change":"2024-06-10T02:49:58.995579+0000","last_active":"2024-06-10T02:51:45.714696+0000","last_peered":"2024-06-10T02:51:45.714696+0000","last_clean":"2024-06-10T02:51:45.714696+0000","last_became_active":"2024-06-10T02:49:58.995208+0000","last_became_peered":"2024-06-10T02:49:58.995208+0000","last_unstale":"2024-06-10T02:51:45.714696+0000","last_undegraded":"2024-06-10T02:51:45.714696+0000","last_fullsized":"2024-06-10T02:51:45.714696+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:26:03.513482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"100'8","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.993655+0000","last_change":"2024-06-10T02:50:02.197090+0000","last_active":"2024-06-10T02:52:42.993655+0000","last_peered":"2024-06-10T02:52:42.993655+0000","last_clean":"2024-06-10T02:52:42.993655+0000","last_became_active":"2024-06-10T02:50:02.196746+0000","last_became_peered":"2024-06-10T02:50:02.196746+0000","last_unstale":"2024-06-10T02:52:42.993655+0000","last_undegraded":"2024-06-10T02:52:42.993655+0000","last_fullsized":"2024-06-10T02:52:42.993655+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:52:09.987978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":30,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:14.133630+0000","last_change":"2024-06-10T02:50:12.234632+0000","last_active":"2024-06-10T02:52:14.133630+0000","last_peered":"2024-06-10T02:52:14.133630+0000","last_clean":"2024-06-10T02:52:14.133630+0000","last_became_active":"2024-06-10T02:50:12.234266+0000","last_became_peered":"2024-06-10T02:50:12.234266+0000","last_unstale":"2024-06-10T02:52:14.133630+0000","last_undegraded":"2024-06-10T02:52:14.133630+0000","last_fullsized":"2024-06-10T02:52:14.133630+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:21:27.348441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"98'3","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.383558+0000","last_change":"2024-06-10T02:49:58.999086+0000","last_active":"2024-06-10T02:51:46.383558+0000","last_peered":"2024-06-10T02:51:46.383558+0000","last_clean":"2024-06-10T02:51:46.383558+0000","last_became_active":"2024-06-10T02:49:58.998767+0000","last_became_peered":"2024-06-10T02:49:58.998767+0000","last_unstale":"2024-06-10T02:51:46.383558+0000","last_undegraded":"2024-06-10T02:51:46.383558+0000","last_fullsized":"2024-06-10T02:51:46.383558+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:28:12.922391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"100'11","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.990813+0000","last_change":"2024-06-10T02:50:02.196641+0000","last_active":"2024-06-10T02:52:42.990813+0000","last_peered":"2024-06-10T02:52:42.990813+0000","last_clean":"2024-06-10T02:52:42.990813+0000","last_became_active":"2024-06-10T02:50:02.196285+0000","last_became_peered":"2024-06-10T02:50:02.196285+0000","last_unstale":"2024-06-10T02:52:42.990813+0000","last_undegraded":"2024-06-10T02:52:42.990813+0000","last_fullsized":"2024-06-10T02:52:42.990813+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T13:55:31.284028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.831864+0000","last_change":"2024-06-10T02:50:12.240988+0000","last_active":"2024-06-10T02:51:45.831864+0000","last_peered":"2024-06-10T02:51:45.831864+0000","last_clean":"2024-06-10T02:51:45.831864+0000","last_became_active":"2024-06-10T02:50:12.240490+0000","last_became_peered":"2024-06-10T02:50:12.240490+0000","last_unstale":"2024-06-10T02:51:45.831864+0000","last_undegraded":"2024-06-10T02:51:45.831864+0000","last_fullsized":"2024-06-10T02:51:45.831864+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:54:36.015221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"100'7","reported_seq":426,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:54:15.901857+0000","last_change":"2024-06-10T02:49:58.989557+0000","last_active":"2024-06-10T02:54:15.901857+0000","last_peered":"2024-06-10T02:54:15.901857+0000","last_clean":"2024-06-10T02:54:15.901857+0000","last_became_active":"2024-06-10T02:49:58.989101+0000","last_became_peered":"2024-06-10T02:49:58.989101+0000","last_unstale":"2024-06-10T02:54:15.901857+0000","last_undegraded":"2024-06-10T02:54:15.901857+0000","last_fullsized":"2024-06-10T02:54:15.901857+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:27:42.182012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":374,"num_read_kb":374,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"100'10","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.998692+0000","last_change":"2024-06-10T02:50:02.208752+0000","last_active":"2024-06-10T02:52:42.998692+0000","last_peered":"2024-06-10T02:52:42.998692+0000","last_clean":"2024-06-10T02:52:42.998692+0000","last_became_active":"2024-06-10T02:50:02.208420+0000","last_became_peered":"2024-06-10T02:50:02.208420+0000","last_unstale":"2024-06-10T02:52:42.998692+0000","last_undegraded":"2024-06-10T02:52:42.998692+0000","last_fullsized":"2024-06-10T02:52:42.998692+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:28:09.412317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.382833+0000","last_change":"2024-06-10T02:50:12.236397+0000","last_active":"2024-06-10T02:51:46.382833+0000","last_peered":"2024-06-10T02:51:46.382833+0000","last_clean":"2024-06-10T02:51:46.382833+0000","last_became_active":"2024-06-10T02:50:12.235972+0000","last_became_peered":"2024-06-10T02:50:12.235972+0000","last_unstale":"2024-06-10T02:51:46.382833+0000","last_undegraded":"2024-06-10T02:51:46.382833+0000","last_fullsized":"2024-06-10T02:51:46.382833+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:38:41.723895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"98'4","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.295527+0000","last_change":"2024-06-10T02:49:58.996616+0000","last_active":"2024-06-10T02:51:46.295527+0000","last_peered":"2024-06-10T02:51:46.295527+0000","last_clean":"2024-06-10T02:51:46.295527+0000","last_became_active":"2024-06-10T02:49:58.996289+0000","last_became_peered":"2024-06-10T02:49:58.996289+0000","last_unstale":"2024-06-10T02:51:46.295527+0000","last_undegraded":"2024-06-10T02:51:46.295527+0000","last_fullsized":"2024-06-10T02:51:46.295527+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:22:12.993608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"100'9","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.983255+0000","last_change":"2024-06-10T02:50:02.205230+0000","last_active":"2024-06-10T02:52:42.983255+0000","last_peered":"2024-06-10T02:52:42.983255+0000","last_clean":"2024-06-10T02:52:42.983255+0000","last_became_active":"2024-06-10T02:50:02.204835+0000","last_became_peered":"2024-06-10T02:50:02.204835+0000","last_unstale":"2024-06-10T02:52:42.983255+0000","last_undegraded":"2024-06-10T02:52:42.983255+0000","last_fullsized":"2024-06-10T02:52:42.983255+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:17:35.084921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.383415+0000","last_change":"2024-06-10T02:50:12.231684+0000","last_active":"2024-06-10T02:51:46.383415+0000","last_peered":"2024-06-10T02:51:46.383415+0000","last_clean":"2024-06-10T02:51:46.383415+0000","last_became_active":"2024-06-10T02:50:12.231256+0000","last_became_peered":"2024-06-10T02:50:12.231256+0000","last_unstale":"2024-06-10T02:51:46.383415+0000","last_undegraded":"2024-06-10T02:51:46.383415+0000","last_fullsized":"2024-06-10T02:51:46.383415+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:17:31.878289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"98'5","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.383170+0000","last_change":"2024-06-10T02:49:59.033950+0000","last_active":"2024-06-10T02:51:46.383170+0000","last_peered":"2024-06-10T02:51:46.383170+0000","last_clean":"2024-06-10T02:51:46.383170+0000","last_became_active":"2024-06-10T02:49:59.033688+0000","last_became_peered":"2024-06-10T02:49:59.033688+0000","last_unstale":"2024-06-10T02:51:46.383170+0000","last_undegraded":"2024-06-10T02:51:46.383170+0000","last_fullsized":"2024-06-10T02:51:46.383170+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:32:03.892091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"100'11","reported_seq":56,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.999378+0000","last_change":"2024-06-10T02:50:44.781378+0000","last_active":"2024-06-10T02:52:42.999378+0000","last_peered":"2024-06-10T02:52:42.999378+0000","last_clean":"2024-06-10T02:52:42.999378+0000","last_became_active":"2024-06-10T02:50:44.780955+0000","last_became_peered":"2024-06-10T02:50:44.780955+0000","last_unstale":"2024-06-10T02:52:42.999378+0000","last_undegraded":"2024-06-10T02:52:42.999378+0000","last_fullsized":"2024-06-10T02:52:42.999378+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:49:13.760381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.827482+0000","last_change":"2024-06-10T02:51:46.384748+0000","last_active":"2024-06-10T02:51:46.827482+0000","last_peered":"2024-06-10T02:51:46.827482+0000","last_clean":"2024-06-10T02:51:46.827482+0000","last_became_active":"2024-06-10T02:51:46.384402+0000","last_became_peered":"2024-06-10T02:51:46.384402+0000","last_unstale":"2024-06-10T02:51:46.827482+0000","last_undegraded":"2024-06-10T02:51:46.827482+0000","last_fullsized":"2024-06-10T02:51:46.827482+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:06:48.056342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.14","version":"100'41","reported_seq":473,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:54:15.922453+0000","last_change":"2024-06-10T02:49:58.997890+0000","last_active":"2024-06-10T02:54:15.922453+0000","last_peered":"2024-06-10T02:54:15.922453+0000","last_clean":"2024-06-10T02:54:15.922453+0000","last_became_active":"2024-06-10T02:49:58.997549+0000","last_became_peered":"2024-06-10T02:49:58.997549+0000","last_unstale":"2024-06-10T02:54:15.922453+0000","last_undegraded":"2024-06-10T02:54:15.922453+0000","last_fullsized":"2024-06-10T02:54:15.922453+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T14:40:50.907029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71495,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":387,"num_read_kb":264,"num_write":43,"num_write_kb":90,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"100'6","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.991053+0000","last_change":"2024-06-10T02:50:02.199707+0000","last_active":"2024-06-10T02:52:42.991053+0000","last_peered":"2024-06-10T02:52:42.991053+0000","last_clean":"2024-06-10T02:52:42.991053+0000","last_became_active":"2024-06-10T02:50:02.198987+0000","last_became_peered":"2024-06-10T02:50:02.198987+0000","last_unstale":"2024-06-10T02:52:42.991053+0000","last_undegraded":"2024-06-10T02:52:42.991053+0000","last_fullsized":"2024-06-10T02:52:42.991053+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:48:03.704590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.383847+0000","last_change":"2024-06-10T02:50:12.239608+0000","last_active":"2024-06-10T02:51:46.383847+0000","last_peered":"2024-06-10T02:51:46.383847+0000","last_clean":"2024-06-10T02:51:46.383847+0000","last_became_active":"2024-06-10T02:50:12.238588+0000","last_became_peered":"2024-06-10T02:50:12.238588+0000","last_unstale":"2024-06-10T02:51:46.383847+0000","last_undegraded":"2024-06-10T02:51:46.383847+0000","last_fullsized":"2024-06-10T02:51:46.383847+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:26:12.718921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"98'9","reported_seq":56,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.721583+0000","last_change":"2024-06-10T02:49:58.997193+0000","last_active":"2024-06-10T02:51:45.721583+0000","last_peered":"2024-06-10T02:51:45.721583+0000","last_clean":"2024-06-10T02:51:45.721583+0000","last_became_active":"2024-06-10T02:49:58.996712+0000","last_became_peered":"2024-06-10T02:49:58.996712+0000","last_unstale":"2024-06-10T02:51:45.721583+0000","last_undegraded":"2024-06-10T02:51:45.721583+0000","last_fullsized":"2024-06-10T02:51:45.721583+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:02:46.894489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"100'9","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.994408+0000","last_change":"2024-06-10T02:50:02.200544+0000","last_active":"2024-06-10T02:52:42.994408+0000","last_peered":"2024-06-10T02:52:42.994408+0000","last_clean":"2024-06-10T02:52:42.994408+0000","last_became_active":"2024-06-10T02:50:02.200201+0000","last_became_peered":"2024-06-10T02:50:02.200201+0000","last_unstale":"2024-06-10T02:52:42.994408+0000","last_undegraded":"2024-06-10T02:52:42.994408+0000","last_fullsized":"2024-06-10T02:52:42.994408+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T13:12:40.925448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.365243+0000","last_change":"2024-06-10T02:50:12.236317+0000","last_active":"2024-06-10T02:51:46.365243+0000","last_peered":"2024-06-10T02:51:46.365243+0000","last_clean":"2024-06-10T02:51:46.365243+0000","last_became_active":"2024-06-10T02:50:12.235806+0000","last_became_peered":"2024-06-10T02:50:12.235806+0000","last_unstale":"2024-06-10T02:51:46.365243+0000","last_undegraded":"2024-06-10T02:51:46.365243+0000","last_fullsized":"2024-06-10T02:51:46.365243+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:29:21.425987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"98'5","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.383944+0000","last_change":"2024-06-10T02:49:58.994149+0000","last_active":"2024-06-10T02:51:46.383944+0000","last_peered":"2024-06-10T02:51:46.383944+0000","last_clean":"2024-06-10T02:51:46.383944+0000","last_became_active":"2024-06-10T02:49:58.993775+0000","last_became_peered":"2024-06-10T02:49:58.993775+0000","last_unstale":"2024-06-10T02:51:46.383944+0000","last_undegraded":"2024-06-10T02:51:46.383944+0000","last_fullsized":"2024-06-10T02:51:46.383944+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:23:55.553943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"100'15","reported_seq":54,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.995273+0000","last_change":"2024-06-10T02:50:02.199979+0000","last_active":"2024-06-10T02:52:42.995273+0000","last_peered":"2024-06-10T02:52:42.995273+0000","last_clean":"2024-06-10T02:52:42.995273+0000","last_became_active":"2024-06-10T02:50:02.199596+0000","last_became_peered":"2024-06-10T02:50:02.199596+0000","last_unstale":"2024-06-10T02:52:42.995273+0000","last_undegraded":"2024-06-10T02:52:42.995273+0000","last_fullsized":"2024-06-10T02:52:42.995273+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:13:58.647460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.383103+0000","last_change":"2024-06-10T02:50:12.235253+0000","last_active":"2024-06-10T02:51:46.383103+0000","last_peered":"2024-06-10T02:51:46.383103+0000","last_clean":"2024-06-10T02:51:46.383103+0000","last_became_active":"2024-06-10T02:50:12.234773+0000","last_became_peered":"2024-06-10T02:50:12.234773+0000","last_unstale":"2024-06-10T02:51:46.383103+0000","last_undegraded":"2024-06-10T02:51:46.383103+0000","last_fullsized":"2024-06-10T02:51:46.383103+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:38:10.971411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"100'7","reported_seq":248,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:54:19.907563+0000","last_change":"2024-06-10T02:49:58.996912+0000","last_active":"2024-06-10T02:54:19.907563+0000","last_peered":"2024-06-10T02:54:19.907563+0000","last_clean":"2024-06-10T02:54:19.907563+0000","last_became_active":"2024-06-10T02:49:58.996549+0000","last_became_peered":"2024-06-10T02:49:58.996549+0000","last_unstale":"2024-06-10T02:54:19.907563+0000","last_undegraded":"2024-06-10T02:54:19.907563+0000","last_fullsized":"2024-06-10T02:54:19.907563+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:07:52.757676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1119,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":197,"num_read_kb":5,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"100'6","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.995546+0000","last_change":"2024-06-10T02:50:44.741186+0000","last_active":"2024-06-10T02:52:42.995546+0000","last_peered":"2024-06-10T02:52:42.995546+0000","last_clean":"2024-06-10T02:52:42.995546+0000","last_became_active":"2024-06-10T02:50:44.740782+0000","last_became_peered":"2024-06-10T02:50:44.740782+0000","last_unstale":"2024-06-10T02:52:42.995546+0000","last_undegraded":"2024-06-10T02:52:42.995546+0000","last_fullsized":"2024-06-10T02:52:42.995546+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:46:03.863243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.715228+0000","last_change":"2024-06-10T02:50:12.236505+0000","last_active":"2024-06-10T02:51:45.715228+0000","last_peered":"2024-06-10T02:51:45.715228+0000","last_clean":"2024-06-10T02:51:45.715228+0000","last_became_active":"2024-06-10T02:50:12.236112+0000","last_became_peered":"2024-06-10T02:50:12.236112+0000","last_unstale":"2024-06-10T02:51:45.715228+0000","last_undegraded":"2024-06-10T02:51:45.715228+0000","last_fullsized":"2024-06-10T02:51:45.715228+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:13:38.586736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"98'17","reported_seq":63,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.723449+0000","last_change":"2024-06-10T02:49:59.031965+0000","last_active":"2024-06-10T02:51:45.723449+0000","last_peered":"2024-06-10T02:51:45.723449+0000","last_clean":"2024-06-10T02:51:45.723449+0000","last_became_active":"2024-06-10T02:49:59.031495+0000","last_became_peered":"2024-06-10T02:49:59.031495+0000","last_unstale":"2024-06-10T02:51:45.723449+0000","last_undegraded":"2024-06-10T02:51:45.723449+0000","last_fullsized":"2024-06-10T02:51:45.723449+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T14:21:06.610677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"100'6","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:43.002009+0000","last_change":"2024-06-10T02:50:02.209926+0000","last_active":"2024-06-10T02:52:43.002009+0000","last_peered":"2024-06-10T02:52:43.002009+0000","last_clean":"2024-06-10T02:52:43.002009+0000","last_became_active":"2024-06-10T02:50:02.209540+0000","last_became_peered":"2024-06-10T02:50:02.209540+0000","last_unstale":"2024-06-10T02:52:43.002009+0000","last_undegraded":"2024-06-10T02:52:43.002009+0000","last_fullsized":"2024-06-10T02:52:43.002009+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:49:40.497692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:48.927934+0000","last_change":"2024-06-10T02:51:46.013156+0000","last_active":"2024-06-10T02:51:48.927934+0000","last_peered":"2024-06-10T02:51:48.927934+0000","last_clean":"2024-06-10T02:51:48.927934+0000","last_became_active":"2024-06-10T02:51:46.012769+0000","last_became_peered":"2024-06-10T02:51:46.012769+0000","last_unstale":"2024-06-10T02:51:48.927934+0000","last_undegraded":"2024-06-10T02:51:48.927934+0000","last_fullsized":"2024-06-10T02:51:48.927934+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:44:49.545676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,3,1],"acting":[8,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.f","version":"98'2","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.724214+0000","last_change":"2024-06-10T02:49:58.998372+0000","last_active":"2024-06-10T02:51:45.724214+0000","last_peered":"2024-06-10T02:51:45.724214+0000","last_clean":"2024-06-10T02:51:45.724214+0000","last_became_active":"2024-06-10T02:49:58.998058+0000","last_became_peered":"2024-06-10T02:49:58.998058+0000","last_unstale":"2024-06-10T02:51:45.724214+0000","last_undegraded":"2024-06-10T02:51:45.724214+0000","last_fullsized":"2024-06-10T02:51:45.724214+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:42:50.509851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"100'5","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.993794+0000","last_change":"2024-06-10T02:50:02.209256+0000","last_active":"2024-06-10T02:52:42.993794+0000","last_peered":"2024-06-10T02:52:42.993794+0000","last_clean":"2024-06-10T02:52:42.993794+0000","last_became_active":"2024-06-10T02:50:02.207727+0000","last_became_peered":"2024-06-10T02:50:02.207727+0000","last_unstale":"2024-06-10T02:52:42.993794+0000","last_undegraded":"2024-06-10T02:52:42.993794+0000","last_fullsized":"2024-06-10T02:52:42.993794+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:23:00.175943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.723666+0000","last_change":"2024-06-10T02:50:12.729438+0000","last_active":"2024-06-10T02:51:45.723666+0000","last_peered":"2024-06-10T02:51:45.723666+0000","last_clean":"2024-06-10T02:51:45.723666+0000","last_became_active":"2024-06-10T02:50:12.728918+0000","last_became_peered":"2024-06-10T02:50:12.728918+0000","last_unstale":"2024-06-10T02:51:45.723666+0000","last_undegraded":"2024-06-10T02:51:45.723666+0000","last_fullsized":"2024-06-10T02:51:45.723666+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:04:48.861763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"98'1","reported_seq":53,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:05.628603+0000","last_change":"2024-06-10T02:50:44.782241+0000","last_active":"2024-06-10T02:52:05.628603+0000","last_peered":"2024-06-10T02:52:05.628603+0000","last_clean":"2024-06-10T02:52:05.628603+0000","last_became_active":"2024-06-10T02:50:44.781799+0000","last_became_peered":"2024-06-10T02:50:44.781799+0000","last_unstale":"2024-06-10T02:52:05.628603+0000","last_undegraded":"2024-06-10T02:52:05.628603+0000","last_fullsized":"2024-06-10T02:52:05.628603+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:38:35.726258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"100'4","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.993135+0000","last_change":"2024-06-10T02:50:02.198888+0000","last_active":"2024-06-10T02:52:42.993135+0000","last_peered":"2024-06-10T02:52:42.993135+0000","last_clean":"2024-06-10T02:52:42.993135+0000","last_became_active":"2024-06-10T02:50:02.198346+0000","last_became_peered":"2024-06-10T02:50:02.198346+0000","last_unstale":"2024-06-10T02:52:42.993135+0000","last_undegraded":"2024-06-10T02:52:42.993135+0000","last_fullsized":"2024-06-10T02:52:42.993135+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:15:42.422898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.715171+0000","last_change":"2024-06-10T02:50:12.232763+0000","last_active":"2024-06-10T02:51:45.715171+0000","last_peered":"2024-06-10T02:51:45.715171+0000","last_clean":"2024-06-10T02:51:45.715171+0000","last_became_active":"2024-06-10T02:50:12.232390+0000","last_became_peered":"2024-06-10T02:50:12.232390+0000","last_unstale":"2024-06-10T02:51:45.715171+0000","last_undegraded":"2024-06-10T02:51:45.715171+0000","last_fullsized":"2024-06-10T02:51:45.715171+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:31:47.973500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"100'8","reported_seq":246,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:54:15.902189+0000","last_change":"2024-06-10T02:49:58.994308+0000","last_active":"2024-06-10T02:54:15.902189+0000","last_peered":"2024-06-10T02:54:15.902189+0000","last_clean":"2024-06-10T02:54:15.902189+0000","last_became_active":"2024-06-10T02:49:58.993494+0000","last_became_peered":"2024-06-10T02:49:58.993494+0000","last_unstale":"2024-06-10T02:54:15.902189+0000","last_undegraded":"2024-06-10T02:54:15.902189+0000","last_fullsized":"2024-06-10T02:54:15.902189+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:07:28.452276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1509,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":192,"num_read_kb":4,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"100'7","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.988698+0000","last_change":"2024-06-10T02:50:02.206872+0000","last_active":"2024-06-10T02:52:42.988698+0000","last_peered":"2024-06-10T02:52:42.988698+0000","last_clean":"2024-06-10T02:52:42.988698+0000","last_became_active":"2024-06-10T02:50:02.206307+0000","last_became_peered":"2024-06-10T02:50:02.206307+0000","last_unstale":"2024-06-10T02:52:42.988698+0000","last_undegraded":"2024-06-10T02:52:42.988698+0000","last_fullsized":"2024-06-10T02:52:42.988698+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:01:04.645674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.715107+0000","last_change":"2024-06-10T02:50:12.729392+0000","last_active":"2024-06-10T02:51:45.715107+0000","last_peered":"2024-06-10T02:51:45.715107+0000","last_clean":"2024-06-10T02:51:45.715107+0000","last_became_active":"2024-06-10T02:50:12.728866+0000","last_became_peered":"2024-06-10T02:50:12.728866+0000","last_unstale":"2024-06-10T02:51:45.715107+0000","last_undegraded":"2024-06-10T02:51:45.715107+0000","last_fullsized":"2024-06-10T02:51:45.715107+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:01:35.845184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"100'8","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.995032+0000","last_change":"2024-06-10T02:50:02.202303+0000","last_active":"2024-06-10T02:52:42.995032+0000","last_peered":"2024-06-10T02:52:42.995032+0000","last_clean":"2024-06-10T02:52:42.995032+0000","last_became_active":"2024-06-10T02:50:02.201998+0000","last_became_peered":"2024-06-10T02:50:02.201998+0000","last_unstale":"2024-06-10T02:52:42.995032+0000","last_undegraded":"2024-06-10T02:52:42.995032+0000","last_fullsized":"2024-06-10T02:52:42.995032+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:15:00.005693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"100'6","reported_seq":241,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:54:15.901655+0000","last_change":"2024-06-10T02:49:59.031513+0000","last_active":"2024-06-10T02:54:15.901655+0000","last_peered":"2024-06-10T02:54:15.901655+0000","last_clean":"2024-06-10T02:54:15.901655+0000","last_became_active":"2024-06-10T02:49:59.030965+0000","last_became_peered":"2024-06-10T02:49:59.030965+0000","last_unstale":"2024-06-10T02:54:15.901655+0000","last_undegraded":"2024-06-10T02:54:15.901655+0000","last_fullsized":"2024-06-10T02:54:15.901655+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:49:02.171578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":189,"num_read_kb":189,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"98'1","reported_seq":30,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.295434+0000","last_change":"2024-06-10T02:50:12.236334+0000","last_active":"2024-06-10T02:51:46.295434+0000","last_peered":"2024-06-10T02:51:46.295434+0000","last_clean":"2024-06-10T02:51:46.295434+0000","last_became_active":"2024-06-10T02:50:12.235820+0000","last_became_peered":"2024-06-10T02:50:12.235820+0000","last_unstale":"2024-06-10T02:51:46.295434+0000","last_undegraded":"2024-06-10T02:51:46.295434+0000","last_fullsized":"2024-06-10T02:51:46.295434+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:16:12.476881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"100'8","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:43.001311+0000","last_change":"2024-06-10T02:50:02.205146+0000","last_active":"2024-06-10T02:52:43.001311+0000","last_peered":"2024-06-10T02:52:43.001311+0000","last_clean":"2024-06-10T02:52:43.001311+0000","last_became_active":"2024-06-10T02:50:02.204785+0000","last_became_peered":"2024-06-10T02:50:02.204785+0000","last_unstale":"2024-06-10T02:52:43.001311+0000","last_undegraded":"2024-06-10T02:52:43.001311+0000","last_fullsized":"2024-06-10T02:52:43.001311+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T13:01:02.068989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"98'6","reported_seq":55,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:06.016314+0000","last_change":"2024-06-10T02:49:58.997654+0000","last_active":"2024-06-10T02:52:06.016314+0000","last_peered":"2024-06-10T02:52:06.016314+0000","last_clean":"2024-06-10T02:52:06.016314+0000","last_became_active":"2024-06-10T02:49:58.997294+0000","last_became_peered":"2024-06-10T02:49:58.997294+0000","last_unstale":"2024-06-10T02:52:06.016314+0000","last_undegraded":"2024-06-10T02:52:06.016314+0000","last_fullsized":"2024-06-10T02:52:06.016314+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:53:14.763198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":367,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.294476+0000","last_change":"2024-06-10T02:48:34.008201+0000","last_active":"2024-06-10T02:51:46.294476+0000","last_peered":"2024-06-10T02:51:46.294476+0000","last_clean":"2024-06-10T02:51:46.294476+0000","last_became_active":"2024-06-10T02:48:34.007633+0000","last_became_peered":"2024-06-10T02:48:34.007633+0000","last_unstale":"2024-06-10T02:51:46.294476+0000","last_undegraded":"2024-06-10T02:51:46.294476+0000","last_fullsized":"2024-06-10T02:51:46.294476+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:45:07.444823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:45:07.444823+0000","last_clean_scrub_stamp":"2024-06-10T02:45:07.444823+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:56:58.159559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":36,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.383275+0000","last_change":"2024-06-10T02:50:44.782130+0000","last_active":"2024-06-10T02:51:46.383275+0000","last_peered":"2024-06-10T02:51:46.383275+0000","last_clean":"2024-06-10T02:51:46.383275+0000","last_became_active":"2024-06-10T02:50:44.781530+0000","last_became_peered":"2024-06-10T02:50:44.781530+0000","last_unstale":"2024-06-10T02:51:46.383275+0000","last_undegraded":"2024-06-10T02:51:46.383275+0000","last_fullsized":"2024-06-10T02:51:46.383275+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:43:23.826207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,11,6,10],"acting":[5,11,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"100'5","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.994100+0000","last_change":"2024-06-10T02:50:02.213666+0000","last_active":"2024-06-10T02:52:42.994100+0000","last_peered":"2024-06-10T02:52:42.994100+0000","last_clean":"2024-06-10T02:52:42.994100+0000","last_became_active":"2024-06-10T02:50:02.213194+0000","last_became_peered":"2024-06-10T02:50:02.213194+0000","last_unstale":"2024-06-10T02:52:42.994100+0000","last_undegraded":"2024-06-10T02:52:42.994100+0000","last_fullsized":"2024-06-10T02:52:42.994100+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:22:55.905412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"98'4","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:02.798809+0000","last_change":"2024-06-10T02:49:58.997618+0000","last_active":"2024-06-10T02:52:02.798809+0000","last_peered":"2024-06-10T02:52:02.798809+0000","last_clean":"2024-06-10T02:52:02.798809+0000","last_became_active":"2024-06-10T02:49:58.997304+0000","last_became_peered":"2024-06-10T02:49:58.997304+0000","last_unstale":"2024-06-10T02:52:02.798809+0000","last_undegraded":"2024-06-10T02:52:02.798809+0000","last_fullsized":"2024-06-10T02:52:02.798809+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:48:59.520991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.364962+0000","last_change":"2024-06-10T02:50:12.236357+0000","last_active":"2024-06-10T02:51:46.364962+0000","last_peered":"2024-06-10T02:51:46.364962+0000","last_clean":"2024-06-10T02:51:46.364962+0000","last_became_active":"2024-06-10T02:50:12.235915+0000","last_became_peered":"2024-06-10T02:50:12.235915+0000","last_unstale":"2024-06-10T02:51:46.364962+0000","last_undegraded":"2024-06-10T02:51:46.364962+0000","last_fullsized":"2024-06-10T02:51:46.364962+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:20:06.287907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"98'4","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:03.929566+0000","last_change":"2024-06-10T02:49:59.031725+0000","last_active":"2024-06-10T02:52:03.929566+0000","last_peered":"2024-06-10T02:52:03.929566+0000","last_clean":"2024-06-10T02:52:03.929566+0000","last_became_active":"2024-06-10T02:49:59.031296+0000","last_became_peered":"2024-06-10T02:49:59.031296+0000","last_unstale":"2024-06-10T02:52:03.929566+0000","last_undegraded":"2024-06-10T02:52:03.929566+0000","last_fullsized":"2024-06-10T02:52:03.929566+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T13:17:58.571696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"100'9","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.986516+0000","last_change":"2024-06-10T02:50:02.201820+0000","last_active":"2024-06-10T02:52:42.986516+0000","last_peered":"2024-06-10T02:52:42.986516+0000","last_clean":"2024-06-10T02:52:42.986516+0000","last_became_active":"2024-06-10T02:50:02.201478+0000","last_became_peered":"2024-06-10T02:50:02.201478+0000","last_unstale":"2024-06-10T02:52:42.986516+0000","last_undegraded":"2024-06-10T02:52:42.986516+0000","last_fullsized":"2024-06-10T02:52:42.986516+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:57:25.963926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":30,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:12.118458+0000","last_change":"2024-06-10T02:50:12.729418+0000","last_active":"2024-06-10T02:52:12.118458+0000","last_peered":"2024-06-10T02:52:12.118458+0000","last_clean":"2024-06-10T02:52:12.118458+0000","last_became_active":"2024-06-10T02:50:12.728692+0000","last_became_peered":"2024-06-10T02:50:12.728692+0000","last_unstale":"2024-06-10T02:52:12.118458+0000","last_undegraded":"2024-06-10T02:52:12.118458+0000","last_fullsized":"2024-06-10T02:52:12.118458+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:25:00.472396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"100'18","reported_seq":64,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:54:08.234903+0000","last_change":"2024-06-10T02:49:58.996391+0000","last_active":"2024-06-10T02:54:08.234903+0000","last_peered":"2024-06-10T02:54:08.234903+0000","last_clean":"2024-06-10T02:54:08.234903+0000","last_became_active":"2024-06-10T02:49:58.996033+0000","last_became_peered":"2024-06-10T02:49:58.996033+0000","last_unstale":"2024-06-10T02:54:08.234903+0000","last_undegraded":"2024-06-10T02:54:08.234903+0000","last_fullsized":"2024-06-10T02:54:08.234903+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:03:34.568969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":34,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"100'5","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.993669+0000","last_change":"2024-06-10T02:50:02.197974+0000","last_active":"2024-06-10T02:52:42.993669+0000","last_peered":"2024-06-10T02:52:42.993669+0000","last_clean":"2024-06-10T02:52:42.993669+0000","last_became_active":"2024-06-10T02:50:02.197614+0000","last_became_peered":"2024-06-10T02:50:02.197614+0000","last_unstale":"2024-06-10T02:52:42.993669+0000","last_undegraded":"2024-06-10T02:52:42.993669+0000","last_fullsized":"2024-06-10T02:52:42.993669+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:36:55.575808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.721202+0000","last_change":"2024-06-10T02:50:12.234239+0000","last_active":"2024-06-10T02:51:45.721202+0000","last_peered":"2024-06-10T02:51:45.721202+0000","last_clean":"2024-06-10T02:51:45.721202+0000","last_became_active":"2024-06-10T02:50:12.233768+0000","last_became_peered":"2024-06-10T02:50:12.233768+0000","last_unstale":"2024-06-10T02:51:45.721202+0000","last_undegraded":"2024-06-10T02:51:45.721202+0000","last_fullsized":"2024-06-10T02:51:45.721202+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:51:44.230031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"98'7","reported_seq":54,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:11.016968+0000","last_change":"2024-06-10T02:49:59.192765+0000","last_active":"2024-06-10T02:52:11.016968+0000","last_peered":"2024-06-10T02:52:11.016968+0000","last_clean":"2024-06-10T02:52:11.016968+0000","last_became_active":"2024-06-10T02:49:59.192446+0000","last_became_peered":"2024-06-10T02:49:59.192446+0000","last_unstale":"2024-06-10T02:52:11.016968+0000","last_undegraded":"2024-06-10T02:52:11.016968+0000","last_fullsized":"2024-06-10T02:52:11.016968+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:53:51.862019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"100'7","reported_seq":52,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.989395+0000","last_change":"2024-06-10T02:50:44.781209+0000","last_active":"2024-06-10T02:52:42.989395+0000","last_peered":"2024-06-10T02:52:42.989395+0000","last_clean":"2024-06-10T02:52:42.989395+0000","last_became_active":"2024-06-10T02:50:44.780811+0000","last_became_peered":"2024-06-10T02:50:44.780811+0000","last_unstale":"2024-06-10T02:52:42.989395+0000","last_undegraded":"2024-06-10T02:52:42.989395+0000","last_fullsized":"2024-06-10T02:52:42.989395+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:12:40.663565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.383517+0000","last_change":"2024-06-10T02:50:12.730269+0000","last_active":"2024-06-10T02:51:46.383517+0000","last_peered":"2024-06-10T02:51:46.383517+0000","last_clean":"2024-06-10T02:51:46.383517+0000","last_became_active":"2024-06-10T02:50:12.729574+0000","last_became_peered":"2024-06-10T02:50:12.729574+0000","last_unstale":"2024-06-10T02:51:46.383517+0000","last_undegraded":"2024-06-10T02:51:46.383517+0000","last_fullsized":"2024-06-10T02:51:46.383517+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:51:16.235284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"98'2","reported_seq":53,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.854226+0000","last_change":"2024-06-10T02:50:44.782421+0000","last_active":"2024-06-10T02:51:45.854226+0000","last_peered":"2024-06-10T02:51:45.854226+0000","last_clean":"2024-06-10T02:51:45.854226+0000","last_became_active":"2024-06-10T02:50:44.781881+0000","last_became_peered":"2024-06-10T02:50:44.781881+0000","last_unstale":"2024-06-10T02:51:45.854226+0000","last_undegraded":"2024-06-10T02:51:45.854226+0000","last_fullsized":"2024-06-10T02:51:45.854226+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T14:41:58.771256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"100'2","reported_seq":41,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.952928+0000","last_change":"2024-06-10T02:50:02.197761+0000","last_active":"2024-06-10T02:52:42.952928+0000","last_peered":"2024-06-10T02:52:42.952928+0000","last_clean":"2024-06-10T02:52:42.952928+0000","last_became_active":"2024-06-10T02:50:02.197395+0000","last_became_peered":"2024-06-10T02:50:02.197395+0000","last_unstale":"2024-06-10T02:52:42.952928+0000","last_undegraded":"2024-06-10T02:52:42.952928+0000","last_fullsized":"2024-06-10T02:52:42.952928+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:46:37.537280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:48.927972+0000","last_change":"2024-06-10T02:51:46.385129+0000","last_active":"2024-06-10T02:51:48.927972+0000","last_peered":"2024-06-10T02:51:48.927972+0000","last_clean":"2024-06-10T02:51:48.927972+0000","last_became_active":"2024-06-10T02:51:46.384553+0000","last_became_peered":"2024-06-10T02:51:46.384553+0000","last_unstale":"2024-06-10T02:51:48.927972+0000","last_undegraded":"2024-06-10T02:51:48.927972+0000","last_fullsized":"2024-06-10T02:51:48.927972+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:08:11.183679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"98'7","reported_seq":52,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.831750+0000","last_change":"2024-06-10T02:49:58.987863+0000","last_active":"2024-06-10T02:51:45.831750+0000","last_peered":"2024-06-10T02:51:45.831750+0000","last_clean":"2024-06-10T02:51:45.831750+0000","last_became_active":"2024-06-10T02:49:58.987388+0000","last_became_peered":"2024-06-10T02:49:58.987388+0000","last_unstale":"2024-06-10T02:51:45.831750+0000","last_undegraded":"2024-06-10T02:51:45.831750+0000","last_fullsized":"2024-06-10T02:51:45.831750+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:29:20.340301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"100'9","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.988655+0000","last_change":"2024-06-10T02:50:02.200699+0000","last_active":"2024-06-10T02:52:42.988655+0000","last_peered":"2024-06-10T02:52:42.988655+0000","last_clean":"2024-06-10T02:52:42.988655+0000","last_became_active":"2024-06-10T02:50:02.200278+0000","last_became_peered":"2024-06-10T02:50:02.200278+0000","last_unstale":"2024-06-10T02:52:42.988655+0000","last_undegraded":"2024-06-10T02:52:42.988655+0000","last_fullsized":"2024-06-10T02:52:42.988655+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:26:36.236312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"98'1","reported_seq":30,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.383543+0000","last_change":"2024-06-10T02:50:12.236027+0000","last_active":"2024-06-10T02:51:46.383543+0000","last_peered":"2024-06-10T02:51:46.383543+0000","last_clean":"2024-06-10T02:51:46.383543+0000","last_became_active":"2024-06-10T02:50:12.235629+0000","last_became_peered":"2024-06-10T02:50:12.235629+0000","last_unstale":"2024-06-10T02:51:46.383543+0000","last_undegraded":"2024-06-10T02:51:46.383543+0000","last_fullsized":"2024-06-10T02:51:46.383543+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:58:48.218806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"98'3","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.721449+0000","last_change":"2024-06-10T02:49:58.998496+0000","last_active":"2024-06-10T02:51:45.721449+0000","last_peered":"2024-06-10T02:51:45.721449+0000","last_clean":"2024-06-10T02:51:45.721449+0000","last_became_active":"2024-06-10T02:49:58.997260+0000","last_became_peered":"2024-06-10T02:49:58.997260+0000","last_unstale":"2024-06-10T02:51:45.721449+0000","last_undegraded":"2024-06-10T02:51:45.721449+0000","last_fullsized":"2024-06-10T02:51:45.721449+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T14:22:04.658943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"100'8","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.990375+0000","last_change":"2024-06-10T02:50:02.201300+0000","last_active":"2024-06-10T02:52:42.990375+0000","last_peered":"2024-06-10T02:52:42.990375+0000","last_clean":"2024-06-10T02:52:42.990375+0000","last_became_active":"2024-06-10T02:50:02.200988+0000","last_became_peered":"2024-06-10T02:50:02.200988+0000","last_unstale":"2024-06-10T02:52:42.990375+0000","last_undegraded":"2024-06-10T02:52:42.990375+0000","last_fullsized":"2024-06-10T02:52:42.990375+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:11:12.962048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.383484+0000","last_change":"2024-06-10T02:50:12.728869+0000","last_active":"2024-06-10T02:51:46.383484+0000","last_peered":"2024-06-10T02:51:46.383484+0000","last_clean":"2024-06-10T02:51:46.383484+0000","last_became_active":"2024-06-10T02:50:12.728189+0000","last_became_peered":"2024-06-10T02:50:12.728189+0000","last_unstale":"2024-06-10T02:51:46.383484+0000","last_undegraded":"2024-06-10T02:51:46.383484+0000","last_fullsized":"2024-06-10T02:51:46.383484+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:08:36.624814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"98'4","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.723488+0000","last_change":"2024-06-10T02:49:59.000460+0000","last_active":"2024-06-10T02:51:45.723488+0000","last_peered":"2024-06-10T02:51:45.723488+0000","last_clean":"2024-06-10T02:51:45.723488+0000","last_became_active":"2024-06-10T02:49:59.000075+0000","last_became_peered":"2024-06-10T02:49:59.000075+0000","last_unstale":"2024-06-10T02:51:45.723488+0000","last_undegraded":"2024-06-10T02:51:45.723488+0000","last_fullsized":"2024-06-10T02:51:45.723488+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:06:27.935745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"100'5","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:43.001075+0000","last_change":"2024-06-10T02:50:02.210131+0000","last_active":"2024-06-10T02:52:43.001075+0000","last_peered":"2024-06-10T02:52:43.001075+0000","last_clean":"2024-06-10T02:52:43.001075+0000","last_became_active":"2024-06-10T02:50:02.209781+0000","last_became_peered":"2024-06-10T02:50:02.209781+0000","last_unstale":"2024-06-10T02:52:43.001075+0000","last_undegraded":"2024-06-10T02:52:43.001075+0000","last_fullsized":"2024-06-10T02:52:43.001075+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:27:39.966620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.854407+0000","last_change":"2024-06-10T02:50:12.236017+0000","last_active":"2024-06-10T02:51:45.854407+0000","last_peered":"2024-06-10T02:51:45.854407+0000","last_clean":"2024-06-10T02:51:45.854407+0000","last_became_active":"2024-06-10T02:50:12.235553+0000","last_became_peered":"2024-06-10T02:50:12.235553+0000","last_unstale":"2024-06-10T02:51:45.854407+0000","last_undegraded":"2024-06-10T02:51:45.854407+0000","last_fullsized":"2024-06-10T02:51:45.854407+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:23:55.437994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"100'7","reported_seq":243,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:54:14.901132+0000","last_change":"2024-06-10T02:49:59.192864+0000","last_active":"2024-06-10T02:54:14.901132+0000","last_peered":"2024-06-10T02:54:14.901132+0000","last_clean":"2024-06-10T02:54:14.901132+0000","last_became_active":"2024-06-10T02:49:59.192440+0000","last_became_peered":"2024-06-10T02:49:59.192440+0000","last_unstale":"2024-06-10T02:54:14.901132+0000","last_undegraded":"2024-06-10T02:54:14.901132+0000","last_fullsized":"2024-06-10T02:54:14.901132+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:36:05.868538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1701,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":191,"num_read_kb":4,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"100'7","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:43.000525+0000","last_change":"2024-06-10T02:50:02.205791+0000","last_active":"2024-06-10T02:52:43.000525+0000","last_peered":"2024-06-10T02:52:43.000525+0000","last_clean":"2024-06-10T02:52:43.000525+0000","last_became_active":"2024-06-10T02:50:02.205483+0000","last_became_peered":"2024-06-10T02:50:02.205483+0000","last_unstale":"2024-06-10T02:52:43.000525+0000","last_undegraded":"2024-06-10T02:52:43.000525+0000","last_fullsized":"2024-06-10T02:52:43.000525+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:33:25.875423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.011597+0000","last_change":"2024-06-10T02:50:12.239294+0000","last_active":"2024-06-10T02:51:46.011597+0000","last_peered":"2024-06-10T02:51:46.011597+0000","last_clean":"2024-06-10T02:51:46.011597+0000","last_became_active":"2024-06-10T02:50:12.238778+0000","last_became_peered":"2024-06-10T02:50:12.238778+0000","last_unstale":"2024-06-10T02:51:46.011597+0000","last_undegraded":"2024-06-10T02:51:46.011597+0000","last_fullsized":"2024-06-10T02:51:46.011597+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:26:09.203623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"98'3","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.383589+0000","last_change":"2024-06-10T02:49:58.989680+0000","last_active":"2024-06-10T02:51:46.383589+0000","last_peered":"2024-06-10T02:51:46.383589+0000","last_clean":"2024-06-10T02:51:46.383589+0000","last_became_active":"2024-06-10T02:49:58.989265+0000","last_became_peered":"2024-06-10T02:49:58.989265+0000","last_unstale":"2024-06-10T02:51:46.383589+0000","last_undegraded":"2024-06-10T02:51:46.383589+0000","last_fullsized":"2024-06-10T02:51:46.383589+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:36:34.370177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"100'6","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.990330+0000","last_change":"2024-06-10T02:50:44.743369+0000","last_active":"2024-06-10T02:52:42.990330+0000","last_peered":"2024-06-10T02:52:42.990330+0000","last_clean":"2024-06-10T02:52:42.990330+0000","last_became_active":"2024-06-10T02:50:44.743013+0000","last_became_peered":"2024-06-10T02:50:44.743013+0000","last_unstale":"2024-06-10T02:52:42.990330+0000","last_undegraded":"2024-06-10T02:52:42.990330+0000","last_fullsized":"2024-06-10T02:52:42.990330+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:54:43.728943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.382656+0000","last_change":"2024-06-10T02:50:12.235197+0000","last_active":"2024-06-10T02:51:46.382656+0000","last_peered":"2024-06-10T02:51:46.382656+0000","last_clean":"2024-06-10T02:51:46.382656+0000","last_became_active":"2024-06-10T02:50:12.234791+0000","last_became_peered":"2024-06-10T02:50:12.234791+0000","last_unstale":"2024-06-10T02:51:46.382656+0000","last_undegraded":"2024-06-10T02:51:46.382656+0000","last_fullsized":"2024-06-10T02:51:46.382656+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:09:11.164236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"98'5","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.715164+0000","last_change":"2024-06-10T02:49:58.996759+0000","last_active":"2024-06-10T02:51:45.715164+0000","last_peered":"2024-06-10T02:51:45.715164+0000","last_clean":"2024-06-10T02:51:45.715164+0000","last_became_active":"2024-06-10T02:49:58.996434+0000","last_became_peered":"2024-06-10T02:49:58.996434+0000","last_unstale":"2024-06-10T02:51:45.715164+0000","last_undegraded":"2024-06-10T02:51:45.715164+0000","last_fullsized":"2024-06-10T02:51:45.715164+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:09:35.327797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"100'9","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.996813+0000","last_change":"2024-06-10T02:50:02.201676+0000","last_active":"2024-06-10T02:52:42.996813+0000","last_peered":"2024-06-10T02:52:42.996813+0000","last_clean":"2024-06-10T02:52:42.996813+0000","last_became_active":"2024-06-10T02:50:02.201335+0000","last_became_peered":"2024-06-10T02:50:02.201335+0000","last_unstale":"2024-06-10T02:52:42.996813+0000","last_undegraded":"2024-06-10T02:52:42.996813+0000","last_fullsized":"2024-06-10T02:52:42.996813+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:49:47.038946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"100'9","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.992923+0000","last_change":"2024-06-10T02:50:02.203462+0000","last_active":"2024-06-10T02:52:42.992923+0000","last_peered":"2024-06-10T02:52:42.992923+0000","last_clean":"2024-06-10T02:52:42.992923+0000","last_became_active":"2024-06-10T02:50:02.203115+0000","last_became_peered":"2024-06-10T02:50:02.203115+0000","last_unstale":"2024-06-10T02:52:42.992923+0000","last_undegraded":"2024-06-10T02:52:42.992923+0000","last_fullsized":"2024-06-10T02:52:42.992923+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T13:18:12.400596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"100'12","reported_seq":52,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.998015+0000","last_change":"2024-06-10T02:50:02.205648+0000","last_active":"2024-06-10T02:52:42.998015+0000","last_peered":"2024-06-10T02:52:42.998015+0000","last_clean":"2024-06-10T02:52:42.998015+0000","last_became_active":"2024-06-10T02:50:02.205306+0000","last_became_peered":"2024-06-10T02:50:02.205306+0000","last_unstale":"2024-06-10T02:52:42.998015+0000","last_undegraded":"2024-06-10T02:52:42.998015+0000","last_fullsized":"2024-06-10T02:52:42.998015+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:22:55.020644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"100'8","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.991617+0000","last_change":"2024-06-10T02:50:02.198601+0000","last_active":"2024-06-10T02:52:42.991617+0000","last_peered":"2024-06-10T02:52:42.991617+0000","last_clean":"2024-06-10T02:52:42.991617+0000","last_became_active":"2024-06-10T02:50:02.198103+0000","last_became_peered":"2024-06-10T02:50:02.198103+0000","last_unstale":"2024-06-10T02:52:42.991617+0000","last_undegraded":"2024-06-10T02:52:42.991617+0000","last_fullsized":"2024-06-10T02:52:42.991617+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:56:28.033482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"100'7","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.985450+0000","last_change":"2024-06-10T02:50:02.202638+0000","last_active":"2024-06-10T02:52:42.985450+0000","last_peered":"2024-06-10T02:52:42.985450+0000","last_clean":"2024-06-10T02:52:42.985450+0000","last_became_active":"2024-06-10T02:50:02.202249+0000","last_became_peered":"2024-06-10T02:50:02.202249+0000","last_unstale":"2024-06-10T02:52:42.985450+0000","last_undegraded":"2024-06-10T02:52:42.985450+0000","last_fullsized":"2024-06-10T02:52:42.985450+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:46:29.424164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"100'4","reported_seq":43,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.992649+0000","last_change":"2024-06-10T02:50:02.213526+0000","last_active":"2024-06-10T02:52:42.992649+0000","last_peered":"2024-06-10T02:52:42.992649+0000","last_clean":"2024-06-10T02:52:42.992649+0000","last_became_active":"2024-06-10T02:50:02.213226+0000","last_became_peered":"2024-06-10T02:50:02.213226+0000","last_unstale":"2024-06-10T02:52:42.992649+0000","last_undegraded":"2024-06-10T02:52:42.992649+0000","last_fullsized":"2024-06-10T02:52:42.992649+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:01:56.846340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"100'6","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.993061+0000","last_change":"2024-06-10T02:50:02.206583+0000","last_active":"2024-06-10T02:52:42.993061+0000","last_peered":"2024-06-10T02:52:42.993061+0000","last_clean":"2024-06-10T02:52:42.993061+0000","last_became_active":"2024-06-10T02:50:02.206227+0000","last_became_peered":"2024-06-10T02:50:02.206227+0000","last_unstale":"2024-06-10T02:52:42.993061+0000","last_undegraded":"2024-06-10T02:52:42.993061+0000","last_fullsized":"2024-06-10T02:52:42.993061+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:25:07.246288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"100'15","reported_seq":54,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:43.002970+0000","last_change":"2024-06-10T02:50:02.206348+0000","last_active":"2024-06-10T02:52:43.002970+0000","last_peered":"2024-06-10T02:52:43.002970+0000","last_clean":"2024-06-10T02:52:43.002970+0000","last_became_active":"2024-06-10T02:50:02.206020+0000","last_became_peered":"2024-06-10T02:50:02.206020+0000","last_unstale":"2024-06-10T02:52:43.002970+0000","last_undegraded":"2024-06-10T02:52:43.002970+0000","last_fullsized":"2024-06-10T02:52:43.002970+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:02:26.619113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"100'2","reported_seq":41,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.993521+0000","last_change":"2024-06-10T02:50:02.199405+0000","last_active":"2024-06-10T02:52:42.993521+0000","last_peered":"2024-06-10T02:52:42.993521+0000","last_clean":"2024-06-10T02:52:42.993521+0000","last_became_active":"2024-06-10T02:50:02.199049+0000","last_became_peered":"2024-06-10T02:50:02.199049+0000","last_unstale":"2024-06-10T02:52:42.993521+0000","last_undegraded":"2024-06-10T02:52:42.993521+0000","last_fullsized":"2024-06-10T02:52:42.993521+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:33:04.091160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"100'6","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.992703+0000","last_change":"2024-06-10T02:50:02.202094+0000","last_active":"2024-06-10T02:52:42.992703+0000","last_peered":"2024-06-10T02:52:42.992703+0000","last_clean":"2024-06-10T02:52:42.992703+0000","last_became_active":"2024-06-10T02:50:02.201751+0000","last_became_peered":"2024-06-10T02:50:02.201751+0000","last_unstale":"2024-06-10T02:52:42.992703+0000","last_undegraded":"2024-06-10T02:52:42.992703+0000","last_fullsized":"2024-06-10T02:52:42.992703+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:52:53.946326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"100'8","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.993442+0000","last_change":"2024-06-10T02:50:02.199630+0000","last_active":"2024-06-10T02:52:42.993442+0000","last_peered":"2024-06-10T02:52:42.993442+0000","last_clean":"2024-06-10T02:52:42.993442+0000","last_became_active":"2024-06-10T02:50:02.199320+0000","last_became_peered":"2024-06-10T02:50:02.199320+0000","last_unstale":"2024-06-10T02:52:42.993442+0000","last_undegraded":"2024-06-10T02:52:42.993442+0000","last_fullsized":"2024-06-10T02:52:42.993442+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:17:28.761862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"100'10","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:43.000631+0000","last_change":"2024-06-10T02:50:02.201190+0000","last_active":"2024-06-10T02:52:43.000631+0000","last_peered":"2024-06-10T02:52:43.000631+0000","last_clean":"2024-06-10T02:52:43.000631+0000","last_became_active":"2024-06-10T02:50:02.200574+0000","last_became_peered":"2024-06-10T02:50:02.200574+0000","last_unstale":"2024-06-10T02:52:43.000631+0000","last_undegraded":"2024-06-10T02:52:43.000631+0000","last_fullsized":"2024-06-10T02:52:43.000631+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:02:07.636218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"100'11","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.992436+0000","last_change":"2024-06-10T02:50:02.202374+0000","last_active":"2024-06-10T02:52:42.992436+0000","last_peered":"2024-06-10T02:52:42.992436+0000","last_clean":"2024-06-10T02:52:42.992436+0000","last_became_active":"2024-06-10T02:50:02.201974+0000","last_became_peered":"2024-06-10T02:50:02.201974+0000","last_unstale":"2024-06-10T02:52:42.992436+0000","last_undegraded":"2024-06-10T02:52:42.992436+0000","last_fullsized":"2024-06-10T02:52:42.992436+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:45:34.977926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"100'8","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.995177+0000","last_change":"2024-06-10T02:50:02.209332+0000","last_active":"2024-06-10T02:52:42.995177+0000","last_peered":"2024-06-10T02:52:42.995177+0000","last_clean":"2024-06-10T02:52:42.995177+0000","last_became_active":"2024-06-10T02:50:02.208175+0000","last_became_peered":"2024-06-10T02:50:02.208175+0000","last_unstale":"2024-06-10T02:52:42.995177+0000","last_undegraded":"2024-06-10T02:52:42.995177+0000","last_fullsized":"2024-06-10T02:52:42.995177+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:44:40.857918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":284028,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":80739,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1934,"num_read_kb":1057,"num_write":276,"num_write_kb":173,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":602112,"data_stored":242217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":237,"ondisk_log_size":237,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351044,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32260,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712892,"statfs":{"total":95995035648,"available":95962001408,"internally_reserved":0,"allocated":1110016,"data_stored":574044,"data_compressed":32447,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612681,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32416,"kb_used_data":1240,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712736,"statfs":{"total":95995035648,"available":95961841664,"internally_reserved":0,"allocated":1269760,"data_stored":1232355,"data_compressed":68564,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1599,"internal_metadata":31914433},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776207,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36448,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708704,"statfs":{"total":95995035648,"available":95957712896,"internally_reserved":0,"allocated":1138688,"data_stored":577539,"data_compressed":32447,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037843,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28112,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717040,"statfs":{"total":95995035648,"available":95966248960,"internally_reserved":0,"allocated":1122304,"data_stored":573149,"data_compressed":32447,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299480,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28172,"kb_used_data":1156,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716980,"statfs":{"total":95995035648,"available":95966187520,"internally_reserved":0,"allocated":1183744,"data_stored":1172218,"data_compressed":68564,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863463006,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36420,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708732,"statfs":{"total":95995035648,"available":95957741568,"internally_reserved":0,"allocated":1110016,"data_stored":569868,"data_compressed":32447,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574967,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36496,"kb_used_data":1160,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708656,"statfs":{"total":95995035648,"available":95957663744,"internally_reserved":0,"allocated":1187840,"data_stored":642393,"data_compressed":32447,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738492,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28124,"kb_used_data":1108,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717028,"statfs":{"total":95995035648,"available":95966236672,"internally_reserved":0,"allocated":1134592,"data_stored":570934,"data_compressed":32447,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411442,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28196,"kb_used_data":1180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716956,"statfs":{"total":95995035648,"available":95966162944,"internally_reserved":0,"allocated":1208320,"data_stored":643062,"data_compressed":32447,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149804,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36432,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708720,"statfs":{"total":95995035648,"available":95957729280,"internally_reserved":0,"allocated":1122304,"data_stored":568655,"data_compressed":32447,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888168,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28116,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717036,"statfs":{"total":95995035648,"available":95966244864,"internally_reserved":0,"allocated":1126400,"data_stored":574224,"data_compressed":32447,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724642,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28180,"kb_used_data":1164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716972,"statfs":{"total":95995035648,"available":95966179328,"internally_reserved":0,"allocated":1191936,"data_stored":1166348,"data_compressed":68564,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34481,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34481,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34481,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36117,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36117,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36117,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36117,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36117,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":74168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":74837,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1903,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":126976,"data_stored":73762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-06-10T02:54:24.522 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph pg dump --format=json 2024-06-10T02:54:24.605 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:54:25.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:54:24 smithi120 ceph-mon[36634]: from='client.15363 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-10T02:54:25.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:54:24 smithi120 ceph-mon[36634]: pgmap v440: 129 pgs: 129 active+clean; 656 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 852 B/s wr, 12 op/s 2024-06-10T02:54:25.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:54:24 smithi188 ceph-mon[36599]: from='client.15363 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-10T02:54:25.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:54:24 smithi188 ceph-mon[36599]: pgmap v440: 129 pgs: 129 active+clean; 656 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 852 B/s wr, 12 op/s 2024-06-10T02:54:25.226 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:54:24 smithi052 ceph-mon[32362]: from='client.15363 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-10T02:54:25.226 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:54:24 smithi052 ceph-mon[32362]: pgmap v440: 129 pgs: 129 active+clean; 656 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 852 B/s wr, 12 op/s 2024-06-10T02:54:25.253 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:54:27.065 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:54:26 smithi052 ceph-mon[32362]: pgmap v441: 129 pgs: 129 active+clean; 656 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s 2024-06-10T02:54:27.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:54:26 smithi120 ceph-mon[36634]: pgmap v441: 129 pgs: 129 active+clean; 656 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s 2024-06-10T02:54:27.206 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:54:26 smithi188 ceph-mon[36599]: pgmap v441: 129 pgs: 129 active+clean; 656 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s 2024-06-10T02:54:27.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.510+0000 7f3d6512e700 1 -- 172.21.15.52:0/617869854 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3d60108500 msgr2=0x7f3d601088e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:54:27.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.510+0000 7f3d6512e700 1 --2- 172.21.15.52:0/617869854 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3d60108500 0x7f3d601088e0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f3d50009990 tx=0x7f3d50009ca0 comp rx=0 tx=0).stop 2024-06-10T02:54:27.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.511+0000 7f3d6512e700 1 -- 172.21.15.52:0/617869854 shutdown_connections 2024-06-10T02:54:27.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.511+0000 7f3d6512e700 1 --2- 172.21.15.52:0/617869854 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3d6006b580 0x7f3d6010f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:27.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.511+0000 7f3d6512e700 1 --2- 172.21.15.52:0/617869854 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3d60108500 0x7f3d601088e0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:27.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.511+0000 7f3d6512e700 1 --2- 172.21.15.52:0/617869854 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3d6006abc0 0x7f3d6006b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:27.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.511+0000 7f3d6512e700 1 -- 172.21.15.52:0/617869854 >> 172.21.15.52:0/617869854 conn(0x7f3d60078340 msgr2=0x7f3d60078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:54:27.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.511+0000 7f3d6512e700 1 -- 172.21.15.52:0/617869854 shutdown_connections 2024-06-10T02:54:27.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.511+0000 7f3d6512e700 1 -- 172.21.15.52:0/617869854 wait complete. 2024-06-10T02:54:27.513 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.511+0000 7f3d6512e700 1 Processor -- start 2024-06-10T02:54:27.513 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.511+0000 7f3d6512e700 1 -- start start 2024-06-10T02:54:27.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.512+0000 7f3d6512e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3d6006abc0 0x7f3d6019e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:27.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.512+0000 7f3d6512e700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3d6006b580 0x7f3d6019ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:27.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.512+0000 7f3d6512e700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3d60108500 0x7f3d60198a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:27.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.512+0000 7f3d6512e700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f3d60112760 con 0x7f3d6006abc0 2024-06-10T02:54:27.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.512+0000 7f3d6512e700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f3d601125e0 con 0x7f3d6006b580 2024-06-10T02:54:27.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.512+0000 7f3d6512e700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f3d601128e0 con 0x7f3d60108500 2024-06-10T02:54:27.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.512+0000 7f3d5ed9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3d6006abc0 0x7f3d6019e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:27.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.512+0000 7f3d5ed9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3d6006abc0 0x7f3d6019e8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37588/0 (socket says 172.21.15.52:37588) 2024-06-10T02:54:27.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.512+0000 7f3d5ed9d700 1 -- 172.21.15.52:0/4293328586 learned_addr learned my addr 172.21.15.52:0/4293328586 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:54:27.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.512+0000 7f3d5f59e700 1 --2- 172.21.15.52:0/4293328586 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3d60108500 0x7f3d60198a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:27.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.512+0000 7f3d57fff700 1 --2- 172.21.15.52:0/4293328586 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3d6006b580 0x7f3d6019ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:27.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.512+0000 7f3d5ed9d700 1 -- 172.21.15.52:0/4293328586 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3d60108500 msgr2=0x7f3d60198a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:54:27.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.512+0000 7f3d5ed9d700 1 --2- 172.21.15.52:0/4293328586 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3d60108500 0x7f3d60198a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:27.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.512+0000 7f3d5ed9d700 1 -- 172.21.15.52:0/4293328586 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3d6006b580 msgr2=0x7f3d6019ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:54:27.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.512+0000 7f3d5ed9d700 1 --2- 172.21.15.52:0/4293328586 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3d6006b580 0x7f3d6019ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:27.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.512+0000 7f3d5ed9d700 1 -- 172.21.15.52:0/4293328586 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d601991e0 con 0x7f3d6006abc0 2024-06-10T02:54:27.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.512+0000 7f3d5ed9d700 1 --2- 172.21.15.52:0/4293328586 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3d6006abc0 0x7f3d6019e8c0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f3d50000c00 tx=0x7f3d5000bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:54:27.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.513+0000 7f3d5cd99700 1 -- 172.21.15.52:0/4293328586 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d50027070 con 0x7f3d6006abc0 2024-06-10T02:54:27.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.513+0000 7f3d5cd99700 1 -- 172.21.15.52:0/4293328586 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d50003d70 con 0x7f3d6006abc0 2024-06-10T02:54:27.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.513+0000 7f3d6512e700 1 -- 172.21.15.52:0/4293328586 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d60199470 con 0x7f3d6006abc0 2024-06-10T02:54:27.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.513+0000 7f3d5cd99700 1 -- 172.21.15.52:0/4293328586 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d5000f460 con 0x7f3d6006abc0 2024-06-10T02:54:27.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.513+0000 7f3d6512e700 1 -- 172.21.15.52:0/4293328586 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d601a5790 con 0x7f3d6006abc0 2024-06-10T02:54:27.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.514+0000 7f3d6512e700 1 -- 172.21.15.52:0/4293328586 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d6010a9e0 con 0x7f3d6006abc0 2024-06-10T02:54:27.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.515+0000 7f3d5cd99700 1 -- 172.21.15.52:0/4293328586 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7f3d5002a070 con 0x7f3d6006abc0 2024-06-10T02:54:27.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.516+0000 7f3d5cd99700 1 --2- 172.21.15.52:0/4293328586 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f3d4c0a2d30 0x7f3d4c0a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:27.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.516+0000 7f3d57fff700 1 --2- 172.21.15.52:0/4293328586 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f3d4c0a2d30 0x7f3d4c0a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:27.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.516+0000 7f3d5cd99700 1 -- 172.21.15.52:0/4293328586 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f3d500d0350 con 0x7f3d6006abc0 2024-06-10T02:54:27.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.517+0000 7f3d57fff700 1 --2- 172.21.15.52:0/4293328586 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f3d4c0a2d30 0x7f3d4c0a51f0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f3d600789d0 tx=0x7f3d480073d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:54:27.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.521+0000 7f3d5cd99700 1 -- 172.21.15.52:0/4293328586 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f3d50013070 con 0x7f3d6006abc0 2024-06-10T02:54:27.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.408+0000 7f3becf66700 1 -- 172.21.15.52:0/928256978 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3be8101380 msgr2=0x7f3be8101800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:54:27.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.408+0000 7f3becf66700 1 --2- 172.21.15.52:0/928256978 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3be8101380 0x7f3be8101800 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f3bd0009990 tx=0x7f3bd0009ca0 comp rx=0 tx=0).stop 2024-06-10T02:54:27.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.408+0000 7f3becf66700 1 -- 172.21.15.52:0/928256978 shutdown_connections 2024-06-10T02:54:27.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.408+0000 7f3becf66700 1 --2- 172.21.15.52:0/928256978 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3be8101d40 0x7f3be810e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:27.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.408+0000 7f3becf66700 1 --2- 172.21.15.52:0/928256978 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3be81073f0 0x7f3be81077d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:27.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.408+0000 7f3becf66700 1 --2- 172.21.15.52:0/928256978 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3be8101380 0x7f3be8101800 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:27.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.408+0000 7f3becf66700 1 -- 172.21.15.52:0/928256978 >> 172.21.15.52:0/928256978 conn(0x7f3be8078300 msgr2=0x7f3be8078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:54:27.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.408+0000 7f3becf66700 1 -- 172.21.15.52:0/928256978 shutdown_connections 2024-06-10T02:54:27.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.408+0000 7f3becf66700 1 -- 172.21.15.52:0/928256978 wait complete. 2024-06-10T02:54:27.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.409+0000 7f3becf66700 1 Processor -- start 2024-06-10T02:54:27.638 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.409+0000 7f3becf66700 1 -- start start 2024-06-10T02:54:27.638 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.409+0000 7f3becf66700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3be8101380 0x7f3be8103060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:27.638 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.409+0000 7f3becf66700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3be8101d40 0x7f3be81035a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:27.638 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.409+0000 7f3becf66700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3be81073f0 0x7f3be8103ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:27.638 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.409+0000 7f3becf66700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f3be8111660 con 0x7f3be8101380 2024-06-10T02:54:27.638 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.409+0000 7f3becf66700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f3be81114e0 con 0x7f3be8101d40 2024-06-10T02:54:27.638 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.409+0000 7f3becf66700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f3be81117e0 con 0x7f3be81073f0 2024-06-10T02:54:27.638 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.409+0000 7f3be659c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3be8101380 0x7f3be8103060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:27.638 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.409+0000 7f3be659c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3be8101380 0x7f3be8103060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37552/0 (socket says 172.21.15.52:37552) 2024-06-10T02:54:27.638 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.409+0000 7f3be659c700 1 -- 172.21.15.52:0/2927027437 learned_addr learned my addr 172.21.15.52:0/2927027437 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:54:27.638 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.409+0000 7f3be5d9b700 1 --2- 172.21.15.52:0/2927027437 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3be8101d40 0x7f3be81035a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:27.638 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.409+0000 7f3be6d9d700 1 --2- 172.21.15.52:0/2927027437 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3be81073f0 0x7f3be8103ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:27.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.409+0000 7f3be659c700 1 -- 172.21.15.52:0/2927027437 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3be81073f0 msgr2=0x7f3be8103ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:54:27.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.409+0000 7f3be659c700 1 --2- 172.21.15.52:0/2927027437 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3be81073f0 0x7f3be8103ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:27.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.409+0000 7f3be659c700 1 -- 172.21.15.52:0/2927027437 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3be8101d40 msgr2=0x7f3be81035a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:54:27.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.409+0000 7f3be659c700 1 --2- 172.21.15.52:0/2927027437 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3be8101d40 0x7f3be81035a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:27.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.410+0000 7f3be659c700 1 -- 172.21.15.52:0/2927027437 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3be81a7150 con 0x7f3be8101380 2024-06-10T02:54:27.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.410+0000 7f3be659c700 1 --2- 172.21.15.52:0/2927027437 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3be8101380 0x7f3be8103060 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f3bd800c920 tx=0x7f3bd800cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:54:27.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.410+0000 7f3bdf7fe700 1 -- 172.21.15.52:0/2927027437 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bd8007b00 con 0x7f3be8101380 2024-06-10T02:54:27.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.410+0000 7f3bdf7fe700 1 -- 172.21.15.52:0/2927027437 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3bd8004d10 con 0x7f3be8101380 2024-06-10T02:54:27.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.410+0000 7f3becf66700 1 -- 172.21.15.52:0/2927027437 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3be81a7440 con 0x7f3be8101380 2024-06-10T02:54:27.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.410+0000 7f3bdf7fe700 1 -- 172.21.15.52:0/2927027437 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bd8005680 con 0x7f3be8101380 2024-06-10T02:54:27.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.410+0000 7f3becf66700 1 -- 172.21.15.52:0/2927027437 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3be81a7980 con 0x7f3be8101380 2024-06-10T02:54:27.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.412+0000 7f3bdf7fe700 1 -- 172.21.15.52:0/2927027437 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7f3bd8004750 con 0x7f3be8101380 2024-06-10T02:54:27.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.414+0000 7f3bdf7fe700 1 --2- 172.21.15.52:0/2927027437 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f3bd40a2cb0 0x7f3bd40a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:27.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.414+0000 7f3be5d9b700 1 --2- 172.21.15.52:0/2927027437 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f3bd40a2cb0 0x7f3bd40a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:27.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.414+0000 7f3bdf7fe700 1 -- 172.21.15.52:0/2927027437 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f3bd80cf0d0 con 0x7f3be8101380 2024-06-10T02:54:27.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.414+0000 7f3be5d9b700 1 --2- 172.21.15.52:0/2927027437 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f3bd40a2cb0 0x7f3bd40a5170 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f3be8078860 tx=0x7f3bd000b560 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:54:27.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.415+0000 7f3bf77e9b80 1 Processor -- start 2024-06-10T02:54:27.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.415+0000 7f3bf77e9b80 1 -- start start 2024-06-10T02:54:27.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.415+0000 7f3bf77e9b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e3e8c29f60 0x55e3e8c2a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:27.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.415+0000 7f3bf77e9b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e3e8c2a880 0x55e3e8c24260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:27.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.415+0000 7f3bf77e9b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e3e8c247a0 0x55e3e8c24bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:27.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.415+0000 7f3bf77e9b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55e3e8b16f60 con 0x55e3e8c247a0 2024-06-10T02:54:27.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.415+0000 7f3bf77e9b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55e3e8b75850 con 0x55e3e8c29f60 2024-06-10T02:54:27.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.415+0000 7f3bf77e9b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55e3e8add460 con 0x55e3e8c2a880 2024-06-10T02:54:27.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.415+0000 7f3be6d9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e3e8c247a0 0x55e3e8c24bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:27.641 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.415+0000 7f3be6d9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e3e8c247a0 0x55e3e8c24bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37558/0 (socket says 172.21.15.52:37558) 2024-06-10T02:54:27.641 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.415+0000 7f3be6d9d700 1 -- 172.21.15.52:0/2328329698 learned_addr learned my addr 172.21.15.52:0/2328329698 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:54:27.641 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.415+0000 7f3be659c700 1 --2- 172.21.15.52:0/2328329698 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e3e8c29f60 0x55e3e8c2a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:27.641 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.415+0000 7f3be5d9b700 1 --2- 172.21.15.52:0/2328329698 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e3e8c2a880 0x55e3e8c24260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:27.641 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.416+0000 7f3be6d9d700 1 -- 172.21.15.52:0/2328329698 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e3e8c2a880 msgr2=0x55e3e8c24260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:54:27.641 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.416+0000 7f3be6d9d700 1 --2- 172.21.15.52:0/2328329698 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e3e8c2a880 0x55e3e8c24260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:27.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.416+0000 7f3be6d9d700 1 -- 172.21.15.52:0/2328329698 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e3e8c29f60 msgr2=0x55e3e8c2a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:54:27.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.416+0000 7f3be6d9d700 1 --2- 172.21.15.52:0/2328329698 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e3e8c29f60 0x55e3e8c2a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:27.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.416+0000 7f3be6d9d700 1 -- 172.21.15.52:0/2328329698 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e3e8ac48d0 con 0x55e3e8c247a0 2024-06-10T02:54:27.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.416+0000 7f3be6d9d700 1 --2- 172.21.15.52:0/2328329698 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e3e8c247a0 0x55e3e8c24bc0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f3be0009dc0 tx=0x7f3be00077e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13a4f332962179e6 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:54:27.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.416+0000 7f3bdcff9700 1 -- 172.21.15.52:0/2328329698 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3be0014610 con 0x55e3e8c247a0 2024-06-10T02:54:27.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.416+0000 7f3bdcff9700 1 -- 172.21.15.52:0/2328329698 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3be0019070 con 0x55e3e8c247a0 2024-06-10T02:54:27.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.416+0000 7f3bdcff9700 1 -- 172.21.15.52:0/2328329698 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3be0013a30 con 0x55e3e8c247a0 2024-06-10T02:54:27.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.417+0000 7f3bf77e9b80 1 -- 172.21.15.52:0/2328329698 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e3e8c247a0 msgr2=0x55e3e8c24bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:54:27.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.417+0000 7f3bf77e9b80 1 --2- 172.21.15.52:0/2328329698 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e3e8c247a0 0x55e3e8c24bc0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f3be0009dc0 tx=0x7f3be00077e0 comp rx=0 tx=0).stop 2024-06-10T02:54:27.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.417+0000 7f3bf77e9b80 1 -- 172.21.15.52:0/2328329698 shutdown_connections 2024-06-10T02:54:27.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.417+0000 7f3bf77e9b80 1 --2- 172.21.15.52:0/2328329698 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e3e8c247a0 0x55e3e8c24bc0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:27.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.417+0000 7f3bf77e9b80 1 --2- 172.21.15.52:0/2328329698 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e3e8c29f60 0x55e3e8c2a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:27.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.417+0000 7f3bf77e9b80 1 --2- 172.21.15.52:0/2328329698 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e3e8c2a880 0x55e3e8c24260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:27.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.417+0000 7f3bf77e9b80 1 -- 172.21.15.52:0/2328329698 >> 172.21.15.52:0/2328329698 conn(0x55e3e8b51080 msgr2=0x55e3e8c1f220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:54:27.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.417+0000 7f3bf77e9b80 1 -- 172.21.15.52:0/2328329698 shutdown_connections 2024-06-10T02:54:27.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.417+0000 7f3bf77e9b80 1 -- 172.21.15.52:0/2328329698 wait complete. 2024-06-10T02:54:27.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.417+0000 7f3bf77e9b80 1 -- start start 2024-06-10T02:54:27.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.417+0000 7f3bf77e9b80 1 Processor -- start 2024-06-10T02:54:27.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.418+0000 7f3bf77e9b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e3e8c247a0 0x55e3e8c30580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:27.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.418+0000 7f3bf77e9b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e3e8c29f60 0x55e3e8c2d880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:27.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.418+0000 7f3bf77e9b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e3e8c2a880 0x55e3e8c2ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:27.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.418+0000 7f3bf77e9b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55e3e8b15fc0 con 0x55e3e8c29f60 2024-06-10T02:54:27.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.418+0000 7f3bf77e9b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55e3e8b71810 con 0x55e3e8c247a0 2024-06-10T02:54:27.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.418+0000 7f3bf77e9b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55e3e8c31260 con 0x55e3e8c2a880 2024-06-10T02:54:27.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.418+0000 7f3bf77e9b80 10 client.0 _ll_register_callbacks cb 0x55e3e8b0b5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T02:54:27.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.418+0000 7f3be5d9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e3e8c29f60 0x55e3e8c2d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:27.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.418+0000 7f3be5d9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e3e8c29f60 0x55e3e8c2d880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37562/0 (socket says 172.21.15.52:37562) 2024-06-10T02:54:27.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.418+0000 7f3be5d9b700 1 -- 172.21.15.52:0/1167950338 learned_addr learned my addr 172.21.15.52:0/1167950338 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:54:27.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.418+0000 7f3be6d9d700 1 --2- 172.21.15.52:0/1167950338 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e3e8c2a880 0x55e3e8c2ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:27.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.418+0000 7f3be659c700 1 --2- 172.21.15.52:0/1167950338 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e3e8c247a0 0x55e3e8c30580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:27.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.418+0000 7f3be5d9b700 1 -- 172.21.15.52:0/1167950338 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e3e8c2a880 msgr2=0x55e3e8c2ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:54:27.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.418+0000 7f3be5d9b700 1 --2- 172.21.15.52:0/1167950338 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e3e8c2a880 0x55e3e8c2ddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:27.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.418+0000 7f3be5d9b700 1 -- 172.21.15.52:0/1167950338 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e3e8c247a0 msgr2=0x55e3e8c30580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:54:27.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.418+0000 7f3be5d9b700 1 --2- 172.21.15.52:0/1167950338 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e3e8c247a0 0x55e3e8c30580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:27.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.418+0000 7f3be5d9b700 1 -- 172.21.15.52:0/1167950338 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e3e8b3b750 con 0x55e3e8c29f60 2024-06-10T02:54:27.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.418+0000 7f3be5d9b700 1 --2- 172.21.15.52:0/1167950338 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e3e8c29f60 0x55e3e8c2d880 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f3bd0015000 tx=0x7f3bd0009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39129f0186c078c6 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:54:27.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.418+0000 7f3bcdffb700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-10T02:54:27.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.418+0000 7f3bcdffb700 1 -- 172.21.15.52:0/1167950338 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bd0035060 con 0x55e3e8c29f60 2024-06-10T02:54:27.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.419+0000 7f3bcdffb700 1 -- 172.21.15.52:0/1167950338 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f3bd0024410 con 0x55e3e8c29f60 2024-06-10T02:54:27.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.419+0000 7f3bf77e9b80 1 -- 172.21.15.52:0/1167950338 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3be0019070 con 0x55e3e8c29f60 2024-06-10T02:54:27.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.419+0000 7f3bcdffb700 1 -- 172.21.15.52:0/1167950338 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f3bd003c050 con 0x55e3e8c29f60 2024-06-10T02:54:27.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.419+0000 7f3bf77e9b80 10 client.15393 fetch_fsmap learned FSMap version 30 2024-06-10T02:54:27.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.419+0000 7f3bf77e9b80 1 -- 172.21.15.52:0/1167950338 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e3e8c3b970 con 0x55e3e8c29f60 2024-06-10T02:54:27.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.419+0000 7f3bcdffb700 1 -- 172.21.15.52:0/1167950338 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3be0019070 con 0x55e3e8c29f60 2024-06-10T02:54:27.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.419+0000 7f3bcdffb700 1 -- 172.21.15.52:0/1167950338 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f3bd002f070 con 0x55e3e8c29f60 2024-06-10T02:54:27.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.419+0000 7f3bf77e9b80 10 client.15393 fetch_fsmap finished waiting for FSMap version 30 2024-06-10T02:54:27.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.419+0000 7f3bf77e9b80 10 client.15393 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T02:54:27.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.419+0000 7f3bf77e9b80 20 client.15393 populate_metadata read hostname 'smithi052' 2024-06-10T02:54:27.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.419+0000 7f3bf77e9b80 1 --2- 172.21.15.52:0/1167950338 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55e3e8c3e000 0x55e3e8c40420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:27.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.419+0000 7f3bf77e9b80 4 client.15393 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T02:54:27.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.419+0000 7f3bf77e9b80 1 -- 172.21.15.52:0/1167950338 --> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e3e8b82d50 con 0x55e3e8c3e000 2024-06-10T02:54:27.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.420+0000 7f3be659c700 1 --2- 172.21.15.52:0/1167950338 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55e3e8c3e000 0x55e3e8c40420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:27.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.420+0000 7f3be659c700 1 --2- 172.21.15.52:0/1167950338 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55e3e8c3e000 0x55e3e8c40420 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad709ebf43cc5f4e server_cookie=478bac5927953d4 in_seq=0 out_seq=0 2024-06-10T02:54:27.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.420+0000 7f3bcdffb700 10 client.15393 ms_handle_connect on v2:172.21.15.188:6838/310444836 2024-06-10T02:54:27.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.421+0000 7f3bcdffb700 1 -- 172.21.15.52:0/1167950338 <== mds.0 v2:172.21.15.188:6838/310444836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55e3e8b82d50 con 0x55e3e8c3e000 2024-06-10T02:54:27.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.421+0000 7f3bcdffb700 10 client.15393 handle_command_reply: tid=0 2024-06-10T02:54:27.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.421+0000 7f3bf77e9b80 1 client.15393 shutdown 2024-06-10T02:54:27.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.421+0000 7f3bf77e9b80 1 -- 172.21.15.52:0/1167950338 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e3e8c29f60 msgr2=0x55e3e8c2d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:54:27.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.421+0000 7f3bf77e9b80 1 --2- 172.21.15.52:0/1167950338 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e3e8c29f60 0x55e3e8c2d880 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f3bd0015000 tx=0x7f3bd0009ca0 comp rx=0 tx=0).stop 2024-06-10T02:54:27.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.421+0000 7f3bf77e9b80 1 -- 172.21.15.52:0/1167950338 shutdown_connections 2024-06-10T02:54:27.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.421+0000 7f3bf77e9b80 1 --2- 172.21.15.52:0/1167950338 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55e3e8c3e000 0x55e3e8c40420 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:27.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.421+0000 7f3bcdffb700 0 client.15393 ms_handle_reset on v2:172.21.15.188:6838/310444836 2024-06-10T02:54:27.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.421+0000 7f3bf77e9b80 1 --2- 172.21.15.52:0/1167950338 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e3e8c2a880 0x55e3e8c2ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:27.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.421+0000 7f3bf77e9b80 1 --2- 172.21.15.52:0/1167950338 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e3e8c247a0 0x55e3e8c30580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:27.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.421+0000 7f3bf77e9b80 1 --2- 172.21.15.52:0/1167950338 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e3e8c29f60 0x55e3e8c2d880 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:27.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.421+0000 7f3bf77e9b80 1 -- 172.21.15.52:0/1167950338 >> 172.21.15.52:0/1167950338 conn(0x55e3e8b51080 msgr2=0x55e3e8c1e700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:54:27.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.421+0000 7f3bf77e9b80 1 -- 172.21.15.52:0/1167950338 shutdown_connections 2024-06-10T02:54:27.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.421+0000 7f3bf77e9b80 1 -- 172.21.15.52:0/1167950338 wait complete. 2024-06-10T02:54:27.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.421+0000 7f3bf77e9b80 20 client.15393 trim_cache size 0 max 16384 2024-06-10T02:54:27.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.442+0000 7f3bf77e9b80 1 Processor -- start 2024-06-10T02:54:27.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.442+0000 7f3bf77e9b80 1 -- start start 2024-06-10T02:54:27.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.442+0000 7f3bf77e9b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e3e8c247a0 0x55e3e8c24260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:27.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.442+0000 7f3bf77e9b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e3e8c29f60 0x55e3e8c2b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:27.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.442+0000 7f3bf77e9b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e3e8c2a880 0x55e3e8c2f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:27.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.442+0000 7f3bf77e9b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f3be81b0520 con 0x55e3e8c247a0 2024-06-10T02:54:27.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.442+0000 7f3bf77e9b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f3be81b2060 con 0x55e3e8c2a880 2024-06-10T02:54:27.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.442+0000 7f3bf77e9b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f3be81b0bf0 con 0x55e3e8c29f60 2024-06-10T02:54:27.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.442+0000 7f3be659c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e3e8c247a0 0x55e3e8c24260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:27.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.442+0000 7f3be659c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e3e8c247a0 0x55e3e8c24260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37566/0 (socket says 172.21.15.52:37566) 2024-06-10T02:54:27.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.442+0000 7f3be659c700 1 -- 172.21.15.52:0/1447965235 learned_addr learned my addr 172.21.15.52:0/1447965235 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:54:27.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.442+0000 7f3be5d9b700 1 --2- 172.21.15.52:0/1447965235 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e3e8c29f60 0x55e3e8c2b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:27.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.442+0000 7f3be6d9d700 1 --2- 172.21.15.52:0/1447965235 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e3e8c2a880 0x55e3e8c2f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:27.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.442+0000 7f3be659c700 1 -- 172.21.15.52:0/1447965235 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e3e8c29f60 msgr2=0x55e3e8c2b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:54:27.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.442+0000 7f3be659c700 1 --2- 172.21.15.52:0/1447965235 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e3e8c29f60 0x55e3e8c2b4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:27.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.442+0000 7f3be659c700 1 -- 172.21.15.52:0/1447965235 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e3e8c2a880 msgr2=0x55e3e8c2f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:54:27.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.442+0000 7f3be659c700 1 --2- 172.21.15.52:0/1447965235 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e3e8c2a880 0x55e3e8c2f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:27.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.442+0000 7f3be659c700 1 -- 172.21.15.52:0/1447965235 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e3e8c2bdf0 con 0x55e3e8c247a0 2024-06-10T02:54:27.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.442+0000 7f3be6d9d700 1 --2- 172.21.15.52:0/1447965235 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e3e8c2a880 0x55e3e8c2f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T02:54:27.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.443+0000 7f3be659c700 1 --2- 172.21.15.52:0/1447965235 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e3e8c247a0 0x55e3e8c24260 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f3bd8097820 tx=0x7f3bd8000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99ba02c81f33d0b0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:54:27.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.443+0000 7f3bcffff700 1 -- 172.21.15.52:0/1447965235 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bd801d040 con 0x55e3e8c247a0 2024-06-10T02:54:27.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.443+0000 7f3bcffff700 1 -- 172.21.15.52:0/1447965235 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3bd8083020 con 0x55e3e8c247a0 2024-06-10T02:54:27.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.443+0000 7f3bcffff700 1 -- 172.21.15.52:0/1447965235 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bd80cc8c0 con 0x55e3e8c247a0 2024-06-10T02:54:27.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.444+0000 7f3bf77e9b80 1 -- 172.21.15.52:0/1447965235 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e3e8c247a0 msgr2=0x55e3e8c24260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:54:27.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.444+0000 7f3bf77e9b80 1 --2- 172.21.15.52:0/1447965235 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e3e8c247a0 0x55e3e8c24260 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f3bd8097820 tx=0x7f3bd8000bc0 comp rx=0 tx=0).stop 2024-06-10T02:54:27.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.444+0000 7f3bf77e9b80 1 -- 172.21.15.52:0/1447965235 shutdown_connections 2024-06-10T02:54:27.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.444+0000 7f3bf77e9b80 1 --2- 172.21.15.52:0/1447965235 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e3e8c2a880 0x55e3e8c2f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:27.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.444+0000 7f3bf77e9b80 1 --2- 172.21.15.52:0/1447965235 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e3e8c247a0 0x55e3e8c24260 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:27.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.444+0000 7f3bf77e9b80 1 --2- 172.21.15.52:0/1447965235 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e3e8c29f60 0x55e3e8c2b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:27.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.444+0000 7f3bf77e9b80 1 -- 172.21.15.52:0/1447965235 >> 172.21.15.52:0/1447965235 conn(0x55e3e8b51080 msgr2=0x55e3e8c2f0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:54:27.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.444+0000 7f3bf77e9b80 1 -- 172.21.15.52:0/1447965235 shutdown_connections 2024-06-10T02:54:27.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.444+0000 7f3bf77e9b80 1 -- 172.21.15.52:0/1447965235 wait complete. 2024-06-10T02:54:27.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.444+0000 7f3bf77e9b80 1 -- start start 2024-06-10T02:54:27.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.444+0000 7f3bf77e9b80 1 Processor -- start 2024-06-10T02:54:27.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.444+0000 7f3bf77e9b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e3e8c247a0 0x55e3e8c36520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:27.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.444+0000 7f3bf77e9b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e3e8c29f60 0x55e3e8c36a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:27.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.444+0000 7f3bf77e9b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e3e8c2a880 0x55e3e8c334f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:27.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.444+0000 7f3bf77e9b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f3be81b1990 con 0x55e3e8c247a0 2024-06-10T02:54:27.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.444+0000 7f3bf77e9b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f3be81b12c0 con 0x55e3e8c29f60 2024-06-10T02:54:27.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.444+0000 7f3bf77e9b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f3be81ae4c0 con 0x55e3e8c2a880 2024-06-10T02:54:27.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.444+0000 7f3bf77e9b80 10 client.0 _ll_register_callbacks cb 0x55e3e8b0b5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T02:54:27.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.445+0000 7f3be659c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e3e8c247a0 0x55e3e8c36520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:27.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.445+0000 7f3be659c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e3e8c247a0 0x55e3e8c36520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37570/0 (socket says 172.21.15.52:37570) 2024-06-10T02:54:27.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.445+0000 7f3be659c700 1 -- 172.21.15.52:0/3404410496 learned_addr learned my addr 172.21.15.52:0/3404410496 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:54:27.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.445+0000 7f3be6d9d700 1 --2- 172.21.15.52:0/3404410496 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e3e8c2a880 0x55e3e8c334f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:27.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.445+0000 7f3be5d9b700 1 --2- 172.21.15.52:0/3404410496 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e3e8c29f60 0x55e3e8c36a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:27.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.445+0000 7f3be659c700 1 -- 172.21.15.52:0/3404410496 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e3e8c2a880 msgr2=0x55e3e8c334f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:54:27.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.445+0000 7f3be659c700 1 --2- 172.21.15.52:0/3404410496 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e3e8c2a880 0x55e3e8c334f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:27.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.445+0000 7f3be659c700 1 -- 172.21.15.52:0/3404410496 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e3e8c29f60 msgr2=0x55e3e8c36a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:54:27.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.445+0000 7f3be659c700 1 --2- 172.21.15.52:0/3404410496 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e3e8c29f60 0x55e3e8c36a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:27.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.445+0000 7f3be659c700 1 -- 172.21.15.52:0/3404410496 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e3e8c33a30 con 0x55e3e8c247a0 2024-06-10T02:54:27.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.445+0000 7f3be6d9d700 1 --2- 172.21.15.52:0/3404410496 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e3e8c2a880 0x55e3e8c334f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T02:54:27.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.445+0000 7f3be659c700 1 --2- 172.21.15.52:0/3404410496 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e3e8c247a0 0x55e3e8c36520 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f3bd80074a0 tx=0x7f3bd80998d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4224d373f6da8454 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:54:27.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.445+0000 7f3bcdffb700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-10T02:54:27.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.445+0000 7f3bcdffb700 1 -- 172.21.15.52:0/3404410496 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bd801d040 con 0x55e3e8c247a0 2024-06-10T02:54:27.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.445+0000 7f3bcdffb700 1 -- 172.21.15.52:0/3404410496 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f3bd8004a40 con 0x55e3e8c247a0 2024-06-10T02:54:27.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.445+0000 7f3bf77e9b80 1 -- 172.21.15.52:0/3404410496 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3bd8083020 con 0x55e3e8c247a0 2024-06-10T02:54:27.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.445+0000 7f3bcdffb700 1 -- 172.21.15.52:0/3404410496 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f3bd80cc8c0 con 0x55e3e8c247a0 2024-06-10T02:54:27.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.445+0000 7f3bf77e9b80 10 client.15399 fetch_fsmap learned FSMap version 30 2024-06-10T02:54:27.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.445+0000 7f3bf77e9b80 1 -- 172.21.15.52:0/3404410496 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e3e8ca9480 con 0x55e3e8c247a0 2024-06-10T02:54:27.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.446+0000 7f3bcdffb700 1 -- 172.21.15.52:0/3404410496 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3bd8083020 con 0x55e3e8c247a0 2024-06-10T02:54:27.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.446+0000 7f3bcdffb700 1 -- 172.21.15.52:0/3404410496 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f3bd80a3ba0 con 0x55e3e8c247a0 2024-06-10T02:54:27.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.446+0000 7f3bf77e9b80 10 client.15399 fetch_fsmap finished waiting for FSMap version 30 2024-06-10T02:54:27.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.446+0000 7f3bf77e9b80 10 client.15399 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T02:54:27.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.446+0000 7f3bf77e9b80 20 client.15399 populate_metadata read hostname 'smithi052' 2024-06-10T02:54:27.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.446+0000 7f3bf77e9b80 1 --2- 172.21.15.52:0/3404410496 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55e3e8c3e000 0x55e3e8caba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:27.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.446+0000 7f3bf77e9b80 4 client.15399 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-10T02:54:27.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.446+0000 7f3bf77e9b80 1 -- 172.21.15.52:0/3404410496 --> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e3e8b7d2d0 con 0x55e3e8c3e000 2024-06-10T02:54:27.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.446+0000 7f3be5d9b700 1 --2- 172.21.15.52:0/3404410496 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55e3e8c3e000 0x55e3e8caba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:27.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.447+0000 7f3be5d9b700 1 --2- 172.21.15.52:0/3404410496 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55e3e8c3e000 0x55e3e8caba30 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e827c275740dd56 server_cookie=6ebe508be63ca64f in_seq=0 out_seq=0 2024-06-10T02:54:27.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.447+0000 7f3bcdffb700 10 client.15399 ms_handle_connect on v2:172.21.15.188:6838/310444836 2024-06-10T02:54:27.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.447+0000 7f3bcdffb700 1 -- 172.21.15.52:0/3404410496 <== mds.0 v2:172.21.15.188:6838/310444836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55e3e8b7d2d0 con 0x55e3e8c3e000 2024-06-10T02:54:27.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.447+0000 7f3bcdffb700 10 client.15399 handle_command_reply: tid=0 2024-06-10T02:54:27.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.447+0000 7f3bf77e9b80 1 client.15399 shutdown 2024-06-10T02:54:27.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.447+0000 7f3bf77e9b80 1 -- 172.21.15.52:0/3404410496 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e3e8c247a0 msgr2=0x55e3e8c36520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:54:27.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.447+0000 7f3bf77e9b80 1 --2- 172.21.15.52:0/3404410496 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e3e8c247a0 0x55e3e8c36520 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f3bd80074a0 tx=0x7f3bd80998d0 comp rx=0 tx=0).stop 2024-06-10T02:54:27.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.447+0000 7f3bf77e9b80 1 -- 172.21.15.52:0/3404410496 shutdown_connections 2024-06-10T02:54:27.653 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-10T02:54:27.653 INFO:teuthology.orchestra.run.smithi052.stdout: "status": "IDLE (waiting for more scrubs)", 2024-06-10T02:54:27.653 INFO:teuthology.orchestra.run.smithi052.stdout: "scrubs": { 2024-06-10T02:54:27.653 INFO:teuthology.orchestra.run.smithi052.stdout: "56a3f2c4-a8fe-4165-bef8-424700e25c1f": { 2024-06-10T02:54:27.653 INFO:teuthology.orchestra.run.smithi052.stdout: "path": "/", 2024-06-10T02:54:27.653 INFO:teuthology.orchestra.run.smithi052.stdout: "tag": "56a3f2c4-a8fe-4165-bef8-424700e25c1f", 2024-06-10T02:54:27.653 INFO:teuthology.orchestra.run.smithi052.stdout: "options": "recursive,force" 2024-06-10T02:54:27.654 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T02:54:27.654 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T02:54:27.654 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-10T02:54:27.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.447+0000 7f3bf77e9b80 1 --2- 172.21.15.52:0/3404410496 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55e3e8c3e000 0x55e3e8caba30 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:27.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.447+0000 7f3bf77e9b80 1 --2- 172.21.15.52:0/3404410496 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e3e8c2a880 0x55e3e8c334f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:27.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.447+0000 7f3bf77e9b80 1 --2- 172.21.15.52:0/3404410496 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e3e8c247a0 0x55e3e8c36520 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:27.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.447+0000 7f3bf77e9b80 1 --2- 172.21.15.52:0/3404410496 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e3e8c29f60 0x55e3e8c36a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:27.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.447+0000 7f3bf77e9b80 1 -- 172.21.15.52:0/3404410496 >> 172.21.15.52:0/3404410496 conn(0x55e3e8b51080 msgr2=0x55e3e8c2b370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:54:27.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.447+0000 7f3bcdffb700 0 client.15399 ms_handle_reset on v2:172.21.15.188:6838/310444836 2024-06-10T02:54:27.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.447+0000 7f3bf77e9b80 1 -- 172.21.15.52:0/3404410496 shutdown_connections 2024-06-10T02:54:27.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.447+0000 7f3bf77e9b80 1 -- 172.21.15.52:0/3404410496 wait complete. 2024-06-10T02:54:27.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.447+0000 7f3bf77e9b80 20 client.15399 trim_cache size 0 max 16384 2024-06-10T02:54:27.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.448+0000 7f3becf66700 1 -- 172.21.15.52:0/2927027437 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f3bd40a2cb0 msgr2=0x7f3bd40a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:54:27.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.448+0000 7f3becf66700 1 --2- 172.21.15.52:0/2927027437 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f3bd40a2cb0 0x7f3bd40a5170 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f3be8078860 tx=0x7f3bd000b560 comp rx=0 tx=0).stop 2024-06-10T02:54:27.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.448+0000 7f3becf66700 1 -- 172.21.15.52:0/2927027437 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3be8101380 msgr2=0x7f3be8103060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:54:27.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.448+0000 7f3becf66700 1 --2- 172.21.15.52:0/2927027437 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3be8101380 0x7f3be8103060 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f3bd800c920 tx=0x7f3bd800cc30 comp rx=0 tx=0).stop 2024-06-10T02:54:27.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.448+0000 7f3becf66700 1 -- 172.21.15.52:0/2927027437 shutdown_connections 2024-06-10T02:54:27.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.448+0000 7f3becf66700 1 --2- 172.21.15.52:0/2927027437 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3be81073f0 0x7f3be8103ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:27.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.448+0000 7f3becf66700 1 --2- 172.21.15.52:0/2927027437 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f3bd40a2cb0 0x7f3bd40a5170 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:27.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.448+0000 7f3becf66700 1 --2- 172.21.15.52:0/2927027437 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3be8101380 0x7f3be8103060 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:27.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.448+0000 7f3becf66700 1 --2- 172.21.15.52:0/2927027437 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3be8101d40 0x7f3be81035a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:27.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.448+0000 7f3becf66700 1 -- 172.21.15.52:0/2927027437 >> 172.21.15.52:0/2927027437 conn(0x7f3be8078300 msgr2=0x7f3be806ad00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:54:27.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.448+0000 7f3becf66700 1 -- 172.21.15.52:0/2927027437 shutdown_connections 2024-06-10T02:54:27.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.448+0000 7f3becf66700 1 -- 172.21.15.52:0/2927027437 wait complete. 2024-06-10T02:54:27.726 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.726+0000 7f3d6512e700 1 -- 172.21.15.52:0/4293328586 --> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3d60199fb0 con 0x7f3d4c0a2d30 2024-06-10T02:54:27.735 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.736+0000 7f3d5cd99700 1 -- 172.21.15.52:0/4293328586 <== mgr.14150 v2:172.21.15.52:6800/3214763969 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+341017 (secure 0 0 0) 0x7f3d60199fb0 con 0x7f3d4c0a2d30 2024-06-10T02:54:27.740 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-10T02:54:27.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.739+0000 7f3d6512e700 1 -- 172.21.15.52:0/4293328586 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f3d4c0a2d30 msgr2=0x7f3d4c0a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:54:27.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.739+0000 7f3d6512e700 1 --2- 172.21.15.52:0/4293328586 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f3d4c0a2d30 0x7f3d4c0a51f0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f3d600789d0 tx=0x7f3d480073d0 comp rx=0 tx=0).stop 2024-06-10T02:54:27.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.739+0000 7f3d6512e700 1 -- 172.21.15.52:0/4293328586 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3d6006abc0 msgr2=0x7f3d6019e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:54:27.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.739+0000 7f3d6512e700 1 --2- 172.21.15.52:0/4293328586 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3d6006abc0 0x7f3d6019e8c0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f3d50000c00 tx=0x7f3d5000bab0 comp rx=0 tx=0).stop 2024-06-10T02:54:27.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.739+0000 7f3d6512e700 1 -- 172.21.15.52:0/4293328586 shutdown_connections 2024-06-10T02:54:27.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.739+0000 7f3d6512e700 1 --2- 172.21.15.52:0/4293328586 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3d60108500 0x7f3d60198a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:27.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.739+0000 7f3d6512e700 1 --2- 172.21.15.52:0/4293328586 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f3d4c0a2d30 0x7f3d4c0a51f0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:27.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.739+0000 7f3d6512e700 1 --2- 172.21.15.52:0/4293328586 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3d6006abc0 0x7f3d6019e8c0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:27.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.739+0000 7f3d6512e700 1 --2- 172.21.15.52:0/4293328586 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3d6006b580 0x7f3d6019ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:27.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.739+0000 7f3d6512e700 1 -- 172.21.15.52:0/4293328586 >> 172.21.15.52:0/4293328586 conn(0x7f3d60078340 msgr2=0x7f3d601000c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:54:27.744 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.739+0000 7f3d6512e700 1 -- 172.21.15.52:0/4293328586 shutdown_connections 2024-06-10T02:54:27.744 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:27.739+0000 7f3d6512e700 1 -- 172.21.15.52:0/4293328586 wait complete. 2024-06-10T02:54:27.744 INFO:teuthology.orchestra.run.smithi052.stderr:dumped all 2024-06-10T02:54:28.337 INFO:tasks.cephfs.filesystem:scrub status for tag:56a3f2c4-a8fe-4165-bef8-424700e25c1f - {'path': '/', 'tag': '56a3f2c4-a8fe-4165-bef8-424700e25c1f', 'options': 'recursive,force'} 2024-06-10T02:54:28.976 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:54:28 smithi052 ceph-mon[32362]: pgmap v442: 129 pgs: 129 active+clean; 656 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-06-10T02:54:28.976 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:54:28 smithi052 ceph-mon[32362]: from='client.15405 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-10T02:54:29.015 INFO:teuthology.orchestra.run.smithi052.stdout:{"pg_ready":true,"pg_map":{"version":442,"stamp":"2024-06-10T02:54:27.059757+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":671341,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":2142,"num_read_kb":1208,"num_write":459,"num_write_kb":3113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1181318,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":876,"ondisk_log_size":876,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":379372,"kb_used_data":13580,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124562452,"statfs":{"total":1151940427776,"available":1151551950848,"internally_reserved":0,"allocated":13905920,"data_stored":8864789,"data_compressed":497715,"data_compressed_allocated":675840,"data_compressed_original":2850816,"omap_allocated":19141,"internal_metadata":374453563},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":143,"num_read_kb":75,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.015248"},"pg_stats":[{"pgid":"3.3f","version":"100'8","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:43.001975+0000","last_change":"2024-06-10T02:50:02.209146+0000","last_active":"2024-06-10T02:52:43.001975+0000","last_peered":"2024-06-10T02:52:43.001975+0000","last_clean":"2024-06-10T02:52:43.001975+0000","last_became_active":"2024-06-10T02:50:02.208027+0000","last_became_peered":"2024-06-10T02:50:02.208027+0000","last_unstale":"2024-06-10T02:52:43.001975+0000","last_undegraded":"2024-06-10T02:52:43.001975+0000","last_fullsized":"2024-06-10T02:52:43.001975+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:35:34.765849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"100'4","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.995310+0000","last_change":"2024-06-10T02:50:02.198767+0000","last_active":"2024-06-10T02:52:42.995310+0000","last_peered":"2024-06-10T02:52:42.995310+0000","last_clean":"2024-06-10T02:52:42.995310+0000","last_became_active":"2024-06-10T02:50:02.198450+0000","last_became_peered":"2024-06-10T02:50:02.198450+0000","last_unstale":"2024-06-10T02:52:42.995310+0000","last_undegraded":"2024-06-10T02:52:42.995310+0000","last_fullsized":"2024-06-10T02:52:42.995310+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:49:56.666083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"100'7","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:43.000415+0000","last_change":"2024-06-10T02:50:02.208885+0000","last_active":"2024-06-10T02:52:43.000415+0000","last_peered":"2024-06-10T02:52:43.000415+0000","last_clean":"2024-06-10T02:52:43.000415+0000","last_became_active":"2024-06-10T02:50:02.206953+0000","last_became_peered":"2024-06-10T02:50:02.206953+0000","last_unstale":"2024-06-10T02:52:43.000415+0000","last_undegraded":"2024-06-10T02:52:43.000415+0000","last_fullsized":"2024-06-10T02:52:43.000415+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:41:23.928939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"100'8","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.994111+0000","last_change":"2024-06-10T02:50:02.208632+0000","last_active":"2024-06-10T02:52:42.994111+0000","last_peered":"2024-06-10T02:52:42.994111+0000","last_clean":"2024-06-10T02:52:42.994111+0000","last_became_active":"2024-06-10T02:50:02.207587+0000","last_became_peered":"2024-06-10T02:50:02.207587+0000","last_unstale":"2024-06-10T02:52:42.994111+0000","last_undegraded":"2024-06-10T02:52:42.994111+0000","last_fullsized":"2024-06-10T02:52:42.994111+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T14:34:36.720170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"100'4","reported_seq":43,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.973606+0000","last_change":"2024-06-10T02:50:02.197966+0000","last_active":"2024-06-10T02:52:42.973606+0000","last_peered":"2024-06-10T02:52:42.973606+0000","last_clean":"2024-06-10T02:52:42.973606+0000","last_became_active":"2024-06-10T02:50:02.197612+0000","last_became_peered":"2024-06-10T02:50:02.197612+0000","last_unstale":"2024-06-10T02:52:42.973606+0000","last_undegraded":"2024-06-10T02:52:42.973606+0000","last_fullsized":"2024-06-10T02:52:42.973606+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:24:53.804613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"100'6","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.987354+0000","last_change":"2024-06-10T02:50:02.201952+0000","last_active":"2024-06-10T02:52:42.987354+0000","last_peered":"2024-06-10T02:52:42.987354+0000","last_clean":"2024-06-10T02:52:42.987354+0000","last_became_active":"2024-06-10T02:50:02.200828+0000","last_became_peered":"2024-06-10T02:50:02.200828+0000","last_unstale":"2024-06-10T02:52:42.987354+0000","last_undegraded":"2024-06-10T02:52:42.987354+0000","last_fullsized":"2024-06-10T02:52:42.987354+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:46:28.518846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"100'5","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.990208+0000","last_change":"2024-06-10T02:50:02.203483+0000","last_active":"2024-06-10T02:52:42.990208+0000","last_peered":"2024-06-10T02:52:42.990208+0000","last_clean":"2024-06-10T02:52:42.990208+0000","last_became_active":"2024-06-10T02:50:02.203153+0000","last_became_peered":"2024-06-10T02:50:02.203153+0000","last_unstale":"2024-06-10T02:52:42.990208+0000","last_undegraded":"2024-06-10T02:52:42.990208+0000","last_fullsized":"2024-06-10T02:52:42.990208+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:29:30.422449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"100'9","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.995630+0000","last_change":"2024-06-10T02:50:02.201686+0000","last_active":"2024-06-10T02:52:42.995630+0000","last_peered":"2024-06-10T02:52:42.995630+0000","last_clean":"2024-06-10T02:52:42.995630+0000","last_became_active":"2024-06-10T02:50:02.200973+0000","last_became_peered":"2024-06-10T02:50:02.200973+0000","last_unstale":"2024-06-10T02:52:42.995630+0000","last_undegraded":"2024-06-10T02:52:42.995630+0000","last_fullsized":"2024-06-10T02:52:42.995630+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:41:13.712675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"100'11","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:43.001422+0000","last_change":"2024-06-10T02:50:02.210497+0000","last_active":"2024-06-10T02:52:43.001422+0000","last_peered":"2024-06-10T02:52:43.001422+0000","last_clean":"2024-06-10T02:52:43.001422+0000","last_became_active":"2024-06-10T02:50:02.209675+0000","last_became_peered":"2024-06-10T02:50:02.209675+0000","last_unstale":"2024-06-10T02:52:43.001422+0000","last_undegraded":"2024-06-10T02:52:43.001422+0000","last_fullsized":"2024-06-10T02:52:43.001422+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:50:29.879669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"100'16","reported_seq":61,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.994975+0000","last_change":"2024-06-10T02:50:44.741139+0000","last_active":"2024-06-10T02:52:42.994975+0000","last_peered":"2024-06-10T02:52:42.994975+0000","last_clean":"2024-06-10T02:52:42.994975+0000","last_became_active":"2024-06-10T02:50:44.740685+0000","last_became_peered":"2024-06-10T02:50:44.740685+0000","last_unstale":"2024-06-10T02:52:42.994975+0000","last_undegraded":"2024-06-10T02:52:42.994975+0000","last_fullsized":"2024-06-10T02:52:42.994975+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:58:59.872732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"100'6","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.999574+0000","last_change":"2024-06-10T02:50:02.206993+0000","last_active":"2024-06-10T02:52:42.999574+0000","last_peered":"2024-06-10T02:52:42.999574+0000","last_clean":"2024-06-10T02:52:42.999574+0000","last_became_active":"2024-06-10T02:50:02.206650+0000","last_became_peered":"2024-06-10T02:50:02.206650+0000","last_unstale":"2024-06-10T02:52:42.999574+0000","last_undegraded":"2024-06-10T02:52:42.999574+0000","last_fullsized":"2024-06-10T02:52:42.999574+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T14:27:49.555350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"100'7","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.979413+0000","last_change":"2024-06-10T02:50:02.200816+0000","last_active":"2024-06-10T02:52:42.979413+0000","last_peered":"2024-06-10T02:52:42.979413+0000","last_clean":"2024-06-10T02:52:42.979413+0000","last_became_active":"2024-06-10T02:50:02.200503+0000","last_became_peered":"2024-06-10T02:50:02.200503+0000","last_unstale":"2024-06-10T02:52:42.979413+0000","last_undegraded":"2024-06-10T02:52:42.979413+0000","last_fullsized":"2024-06-10T02:52:42.979413+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:01:08.326728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"100'10","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.987233+0000","last_change":"2024-06-10T02:50:02.203644+0000","last_active":"2024-06-10T02:52:42.987233+0000","last_peered":"2024-06-10T02:52:42.987233+0000","last_clean":"2024-06-10T02:52:42.987233+0000","last_became_active":"2024-06-10T02:50:02.203100+0000","last_became_peered":"2024-06-10T02:50:02.203100+0000","last_unstale":"2024-06-10T02:52:42.987233+0000","last_undegraded":"2024-06-10T02:52:42.987233+0000","last_fullsized":"2024-06-10T02:52:42.987233+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:01:42.730089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"100'7","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.995068+0000","last_change":"2024-06-10T02:50:02.203330+0000","last_active":"2024-06-10T02:52:42.995068+0000","last_peered":"2024-06-10T02:52:42.995068+0000","last_clean":"2024-06-10T02:52:42.995068+0000","last_became_active":"2024-06-10T02:50:02.202995+0000","last_became_peered":"2024-06-10T02:50:02.202995+0000","last_unstale":"2024-06-10T02:52:42.995068+0000","last_undegraded":"2024-06-10T02:52:42.995068+0000","last_fullsized":"2024-06-10T02:52:42.995068+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:17:23.170591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"100'7","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.986439+0000","last_change":"2024-06-10T02:50:02.204875+0000","last_active":"2024-06-10T02:52:42.986439+0000","last_peered":"2024-06-10T02:52:42.986439+0000","last_clean":"2024-06-10T02:52:42.986439+0000","last_became_active":"2024-06-10T02:50:02.200706+0000","last_became_peered":"2024-06-10T02:50:02.200706+0000","last_unstale":"2024-06-10T02:52:42.986439+0000","last_undegraded":"2024-06-10T02:52:42.986439+0000","last_fullsized":"2024-06-10T02:52:42.986439+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:27:12.282772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"100'9","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.999375+0000","last_change":"2024-06-10T02:50:02.203187+0000","last_active":"2024-06-10T02:52:42.999375+0000","last_peered":"2024-06-10T02:52:42.999375+0000","last_clean":"2024-06-10T02:52:42.999375+0000","last_became_active":"2024-06-10T02:50:02.202034+0000","last_became_peered":"2024-06-10T02:50:02.202034+0000","last_unstale":"2024-06-10T02:52:42.999375+0000","last_undegraded":"2024-06-10T02:52:42.999375+0000","last_fullsized":"2024-06-10T02:52:42.999375+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:59:44.783084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"100'8","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:43.002372+0000","last_change":"2024-06-10T02:50:02.209102+0000","last_active":"2024-06-10T02:52:43.002372+0000","last_peered":"2024-06-10T02:52:43.002372+0000","last_clean":"2024-06-10T02:52:43.002372+0000","last_became_active":"2024-06-10T02:50:02.208730+0000","last_became_peered":"2024-06-10T02:50:02.208730+0000","last_unstale":"2024-06-10T02:52:43.002372+0000","last_undegraded":"2024-06-10T02:52:43.002372+0000","last_fullsized":"2024-06-10T02:52:43.002372+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:08:38.838415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"100'10","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:43.000459+0000","last_change":"2024-06-10T02:50:02.204008+0000","last_active":"2024-06-10T02:52:43.000459+0000","last_peered":"2024-06-10T02:52:43.000459+0000","last_clean":"2024-06-10T02:52:43.000459+0000","last_became_active":"2024-06-10T02:50:02.203655+0000","last_became_peered":"2024-06-10T02:50:02.203655+0000","last_unstale":"2024-06-10T02:52:43.000459+0000","last_undegraded":"2024-06-10T02:52:43.000459+0000","last_fullsized":"2024-06-10T02:52:43.000459+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:32:15.972662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"100'7","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.991723+0000","last_change":"2024-06-10T02:50:02.204917+0000","last_active":"2024-06-10T02:52:42.991723+0000","last_peered":"2024-06-10T02:52:42.991723+0000","last_clean":"2024-06-10T02:52:42.991723+0000","last_became_active":"2024-06-10T02:50:02.204568+0000","last_became_peered":"2024-06-10T02:50:02.204568+0000","last_unstale":"2024-06-10T02:52:42.991723+0000","last_undegraded":"2024-06-10T02:52:42.991723+0000","last_fullsized":"2024-06-10T02:52:42.991723+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:49:46.802665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.721297+0000","last_change":"2024-06-10T02:50:12.240697+0000","last_active":"2024-06-10T02:51:45.721297+0000","last_peered":"2024-06-10T02:51:45.721297+0000","last_clean":"2024-06-10T02:51:45.721297+0000","last_became_active":"2024-06-10T02:50:12.240229+0000","last_became_peered":"2024-06-10T02:50:12.240229+0000","last_unstale":"2024-06-10T02:51:45.721297+0000","last_undegraded":"2024-06-10T02:51:45.721297+0000","last_fullsized":"2024-06-10T02:51:45.721297+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:45:55.071960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"100'15","reported_seq":271,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:54:23.912326+0000","last_change":"2024-06-10T02:50:44.786783+0000","last_active":"2024-06-10T02:54:23.912326+0000","last_peered":"2024-06-10T02:54:23.912326+0000","last_clean":"2024-06-10T02:54:23.912326+0000","last_became_active":"2024-06-10T02:50:44.781773+0000","last_became_peered":"2024-06-10T02:50:44.781773+0000","last_unstale":"2024-06-10T02:54:23.912326+0000","last_undegraded":"2024-06-10T02:54:23.912326+0000","last_fullsized":"2024-06-10T02:54:23.912326+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:31:06.025854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":198,"num_read_kb":198,"num_write":16,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,4],"acting":[9,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"100'5","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.994824+0000","last_change":"2024-06-10T02:50:02.202573+0000","last_active":"2024-06-10T02:52:42.994824+0000","last_peered":"2024-06-10T02:52:42.994824+0000","last_clean":"2024-06-10T02:52:42.994824+0000","last_became_active":"2024-06-10T02:50:02.202241+0000","last_became_peered":"2024-06-10T02:50:02.202241+0000","last_unstale":"2024-06-10T02:52:42.994824+0000","last_undegraded":"2024-06-10T02:52:42.994824+0000","last_fullsized":"2024-06-10T02:52:42.994824+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:31:40.230851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.383047+0000","last_change":"2024-06-10T02:50:12.238372+0000","last_active":"2024-06-10T02:51:46.383047+0000","last_peered":"2024-06-10T02:51:46.383047+0000","last_clean":"2024-06-10T02:51:46.383047+0000","last_became_active":"2024-06-10T02:50:12.237884+0000","last_became_peered":"2024-06-10T02:50:12.237884+0000","last_unstale":"2024-06-10T02:51:46.383047+0000","last_undegraded":"2024-06-10T02:51:46.383047+0000","last_fullsized":"2024-06-10T02:51:46.383047+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:55:31.432655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"98'2","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:02.117661+0000","last_change":"2024-06-10T02:49:59.031528+0000","last_active":"2024-06-10T02:52:02.117661+0000","last_peered":"2024-06-10T02:52:02.117661+0000","last_clean":"2024-06-10T02:52:02.117661+0000","last_became_active":"2024-06-10T02:49:59.031147+0000","last_became_peered":"2024-06-10T02:49:59.031147+0000","last_unstale":"2024-06-10T02:52:02.117661+0000","last_undegraded":"2024-06-10T02:52:02.117661+0000","last_fullsized":"2024-06-10T02:52:02.117661+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:17:08.534396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"100'6","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:43.000839+0000","last_change":"2024-06-10T02:50:02.202858+0000","last_active":"2024-06-10T02:52:43.000839+0000","last_peered":"2024-06-10T02:52:43.000839+0000","last_clean":"2024-06-10T02:52:43.000839+0000","last_became_active":"2024-06-10T02:50:02.202263+0000","last_became_peered":"2024-06-10T02:50:02.202263+0000","last_unstale":"2024-06-10T02:52:43.000839+0000","last_undegraded":"2024-06-10T02:52:43.000839+0000","last_fullsized":"2024-06-10T02:52:43.000839+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:23:15.382762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.383227+0000","last_change":"2024-06-10T02:50:12.237370+0000","last_active":"2024-06-10T02:51:46.383227+0000","last_peered":"2024-06-10T02:51:46.383227+0000","last_clean":"2024-06-10T02:51:46.383227+0000","last_became_active":"2024-06-10T02:50:12.236803+0000","last_became_peered":"2024-06-10T02:50:12.236803+0000","last_unstale":"2024-06-10T02:51:46.383227+0000","last_undegraded":"2024-06-10T02:51:46.383227+0000","last_fullsized":"2024-06-10T02:51:46.383227+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:06:40.695710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"98'12","reported_seq":58,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.724307+0000","last_change":"2024-06-10T02:49:59.000911+0000","last_active":"2024-06-10T02:51:45.724307+0000","last_peered":"2024-06-10T02:51:45.724307+0000","last_clean":"2024-06-10T02:51:45.724307+0000","last_became_active":"2024-06-10T02:49:59.000571+0000","last_became_peered":"2024-06-10T02:49:59.000571+0000","last_unstale":"2024-06-10T02:51:45.724307+0000","last_undegraded":"2024-06-10T02:51:45.724307+0000","last_fullsized":"2024-06-10T02:51:45.724307+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:49:24.635793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"100'9","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.997165+0000","last_change":"2024-06-10T02:50:02.209614+0000","last_active":"2024-06-10T02:52:42.997165+0000","last_peered":"2024-06-10T02:52:42.997165+0000","last_clean":"2024-06-10T02:52:42.997165+0000","last_became_active":"2024-06-10T02:50:02.209251+0000","last_became_peered":"2024-06-10T02:50:02.209251+0000","last_unstale":"2024-06-10T02:52:42.997165+0000","last_undegraded":"2024-06-10T02:52:42.997165+0000","last_fullsized":"2024-06-10T02:52:42.997165+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:35:32.256081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.383229+0000","last_change":"2024-06-10T02:50:12.237811+0000","last_active":"2024-06-10T02:51:46.383229+0000","last_peered":"2024-06-10T02:51:46.383229+0000","last_clean":"2024-06-10T02:51:46.383229+0000","last_became_active":"2024-06-10T02:50:12.237017+0000","last_became_peered":"2024-06-10T02:50:12.237017+0000","last_unstale":"2024-06-10T02:51:46.383229+0000","last_undegraded":"2024-06-10T02:51:46.383229+0000","last_fullsized":"2024-06-10T02:51:46.383229+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:14:29.818885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"98'5","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.723967+0000","last_change":"2024-06-10T02:49:58.996971+0000","last_active":"2024-06-10T02:51:45.723967+0000","last_peered":"2024-06-10T02:51:45.723967+0000","last_clean":"2024-06-10T02:51:45.723967+0000","last_became_active":"2024-06-10T02:49:58.996601+0000","last_became_peered":"2024-06-10T02:49:58.996601+0000","last_unstale":"2024-06-10T02:51:45.723967+0000","last_undegraded":"2024-06-10T02:51:45.723967+0000","last_fullsized":"2024-06-10T02:51:45.723967+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:26:45.963351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"100'15","reported_seq":60,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:43.002596+0000","last_change":"2024-06-10T02:50:44.745816+0000","last_active":"2024-06-10T02:52:43.002596+0000","last_peered":"2024-06-10T02:52:43.002596+0000","last_clean":"2024-06-10T02:52:43.002596+0000","last_became_active":"2024-06-10T02:50:44.745386+0000","last_became_peered":"2024-06-10T02:50:44.745386+0000","last_unstale":"2024-06-10T02:52:43.002596+0000","last_undegraded":"2024-06-10T02:52:43.002596+0000","last_fullsized":"2024-06-10T02:52:43.002596+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:08:53.977133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.721700+0000","last_change":"2024-06-10T02:50:12.236835+0000","last_active":"2024-06-10T02:51:45.721700+0000","last_peered":"2024-06-10T02:51:45.721700+0000","last_clean":"2024-06-10T02:51:45.721700+0000","last_became_active":"2024-06-10T02:50:12.236309+0000","last_became_peered":"2024-06-10T02:50:12.236309+0000","last_unstale":"2024-06-10T02:51:45.721700+0000","last_undegraded":"2024-06-10T02:51:45.721700+0000","last_fullsized":"2024-06-10T02:51:45.721700+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:09:02.789286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"100'8","reported_seq":255,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:54:24.914850+0000","last_change":"2024-06-10T02:49:58.997725+0000","last_active":"2024-06-10T02:54:24.914850+0000","last_peered":"2024-06-10T02:54:24.914850+0000","last_clean":"2024-06-10T02:54:24.914850+0000","last_became_active":"2024-06-10T02:49:58.997375+0000","last_became_peered":"2024-06-10T02:49:58.997375+0000","last_unstale":"2024-06-10T02:54:24.914850+0000","last_undegraded":"2024-06-10T02:54:24.914850+0000","last_fullsized":"2024-06-10T02:54:24.914850+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:28:49.714356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1461,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":202,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"100'7","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.995883+0000","last_change":"2024-06-10T02:50:02.201247+0000","last_active":"2024-06-10T02:52:42.995883+0000","last_peered":"2024-06-10T02:52:42.995883+0000","last_clean":"2024-06-10T02:52:42.995883+0000","last_became_active":"2024-06-10T02:50:02.200897+0000","last_became_peered":"2024-06-10T02:50:02.200897+0000","last_unstale":"2024-06-10T02:52:42.995883+0000","last_undegraded":"2024-06-10T02:52:42.995883+0000","last_fullsized":"2024-06-10T02:52:42.995883+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:01:58.166644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.721648+0000","last_change":"2024-06-10T02:50:12.243097+0000","last_active":"2024-06-10T02:51:45.721648+0000","last_peered":"2024-06-10T02:51:45.721648+0000","last_clean":"2024-06-10T02:51:45.721648+0000","last_became_active":"2024-06-10T02:50:12.242501+0000","last_became_peered":"2024-06-10T02:50:12.242501+0000","last_unstale":"2024-06-10T02:51:45.721648+0000","last_undegraded":"2024-06-10T02:51:45.721648+0000","last_fullsized":"2024-06-10T02:51:45.721648+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T14:42:44.220340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"98'7","reported_seq":52,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.720539+0000","last_change":"2024-06-10T02:49:58.996956+0000","last_active":"2024-06-10T02:51:45.720539+0000","last_peered":"2024-06-10T02:51:45.720539+0000","last_clean":"2024-06-10T02:51:45.720539+0000","last_became_active":"2024-06-10T02:49:58.996634+0000","last_became_peered":"2024-06-10T02:49:58.996634+0000","last_unstale":"2024-06-10T02:51:45.720539+0000","last_undegraded":"2024-06-10T02:51:45.720539+0000","last_fullsized":"2024-06-10T02:51:45.720539+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:25:02.364472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"100'8","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.995286+0000","last_change":"2024-06-10T02:50:02.205808+0000","last_active":"2024-06-10T02:52:42.995286+0000","last_peered":"2024-06-10T02:52:42.995286+0000","last_clean":"2024-06-10T02:52:42.995286+0000","last_became_active":"2024-06-10T02:50:02.203558+0000","last_became_peered":"2024-06-10T02:50:02.203558+0000","last_unstale":"2024-06-10T02:52:42.995286+0000","last_undegraded":"2024-06-10T02:52:42.995286+0000","last_fullsized":"2024-06-10T02:52:42.995286+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T13:43:23.315375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":30,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:17.877607+0000","last_change":"2024-06-10T02:50:12.242005+0000","last_active":"2024-06-10T02:52:17.877607+0000","last_peered":"2024-06-10T02:52:17.877607+0000","last_clean":"2024-06-10T02:52:17.877607+0000","last_became_active":"2024-06-10T02:50:12.241295+0000","last_became_peered":"2024-06-10T02:50:12.241295+0000","last_unstale":"2024-06-10T02:52:17.877607+0000","last_undegraded":"2024-06-10T02:52:17.877607+0000","last_fullsized":"2024-06-10T02:52:17.877607+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:32:52.702756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"98'3","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.714696+0000","last_change":"2024-06-10T02:49:58.995579+0000","last_active":"2024-06-10T02:51:45.714696+0000","last_peered":"2024-06-10T02:51:45.714696+0000","last_clean":"2024-06-10T02:51:45.714696+0000","last_became_active":"2024-06-10T02:49:58.995208+0000","last_became_peered":"2024-06-10T02:49:58.995208+0000","last_unstale":"2024-06-10T02:51:45.714696+0000","last_undegraded":"2024-06-10T02:51:45.714696+0000","last_fullsized":"2024-06-10T02:51:45.714696+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:26:03.513482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"100'8","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.993655+0000","last_change":"2024-06-10T02:50:02.197090+0000","last_active":"2024-06-10T02:52:42.993655+0000","last_peered":"2024-06-10T02:52:42.993655+0000","last_clean":"2024-06-10T02:52:42.993655+0000","last_became_active":"2024-06-10T02:50:02.196746+0000","last_became_peered":"2024-06-10T02:50:02.196746+0000","last_unstale":"2024-06-10T02:52:42.993655+0000","last_undegraded":"2024-06-10T02:52:42.993655+0000","last_fullsized":"2024-06-10T02:52:42.993655+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:52:09.987978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":30,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:14.133630+0000","last_change":"2024-06-10T02:50:12.234632+0000","last_active":"2024-06-10T02:52:14.133630+0000","last_peered":"2024-06-10T02:52:14.133630+0000","last_clean":"2024-06-10T02:52:14.133630+0000","last_became_active":"2024-06-10T02:50:12.234266+0000","last_became_peered":"2024-06-10T02:50:12.234266+0000","last_unstale":"2024-06-10T02:52:14.133630+0000","last_undegraded":"2024-06-10T02:52:14.133630+0000","last_fullsized":"2024-06-10T02:52:14.133630+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:21:27.348441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"98'3","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.383558+0000","last_change":"2024-06-10T02:49:58.999086+0000","last_active":"2024-06-10T02:51:46.383558+0000","last_peered":"2024-06-10T02:51:46.383558+0000","last_clean":"2024-06-10T02:51:46.383558+0000","last_became_active":"2024-06-10T02:49:58.998767+0000","last_became_peered":"2024-06-10T02:49:58.998767+0000","last_unstale":"2024-06-10T02:51:46.383558+0000","last_undegraded":"2024-06-10T02:51:46.383558+0000","last_fullsized":"2024-06-10T02:51:46.383558+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:28:12.922391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"100'11","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.990813+0000","last_change":"2024-06-10T02:50:02.196641+0000","last_active":"2024-06-10T02:52:42.990813+0000","last_peered":"2024-06-10T02:52:42.990813+0000","last_clean":"2024-06-10T02:52:42.990813+0000","last_became_active":"2024-06-10T02:50:02.196285+0000","last_became_peered":"2024-06-10T02:50:02.196285+0000","last_unstale":"2024-06-10T02:52:42.990813+0000","last_undegraded":"2024-06-10T02:52:42.990813+0000","last_fullsized":"2024-06-10T02:52:42.990813+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T13:55:31.284028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.831864+0000","last_change":"2024-06-10T02:50:12.240988+0000","last_active":"2024-06-10T02:51:45.831864+0000","last_peered":"2024-06-10T02:51:45.831864+0000","last_clean":"2024-06-10T02:51:45.831864+0000","last_became_active":"2024-06-10T02:50:12.240490+0000","last_became_peered":"2024-06-10T02:50:12.240490+0000","last_unstale":"2024-06-10T02:51:45.831864+0000","last_undegraded":"2024-06-10T02:51:45.831864+0000","last_fullsized":"2024-06-10T02:51:45.831864+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:54:36.015221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"100'7","reported_seq":446,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:54:25.915515+0000","last_change":"2024-06-10T02:49:58.989557+0000","last_active":"2024-06-10T02:54:25.915515+0000","last_peered":"2024-06-10T02:54:25.915515+0000","last_clean":"2024-06-10T02:54:25.915515+0000","last_became_active":"2024-06-10T02:49:58.989101+0000","last_became_peered":"2024-06-10T02:49:58.989101+0000","last_unstale":"2024-06-10T02:54:25.915515+0000","last_undegraded":"2024-06-10T02:54:25.915515+0000","last_fullsized":"2024-06-10T02:54:25.915515+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:27:42.182012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":394,"num_read_kb":394,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"100'10","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.998692+0000","last_change":"2024-06-10T02:50:02.208752+0000","last_active":"2024-06-10T02:52:42.998692+0000","last_peered":"2024-06-10T02:52:42.998692+0000","last_clean":"2024-06-10T02:52:42.998692+0000","last_became_active":"2024-06-10T02:50:02.208420+0000","last_became_peered":"2024-06-10T02:50:02.208420+0000","last_unstale":"2024-06-10T02:52:42.998692+0000","last_undegraded":"2024-06-10T02:52:42.998692+0000","last_fullsized":"2024-06-10T02:52:42.998692+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:28:09.412317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.382833+0000","last_change":"2024-06-10T02:50:12.236397+0000","last_active":"2024-06-10T02:51:46.382833+0000","last_peered":"2024-06-10T02:51:46.382833+0000","last_clean":"2024-06-10T02:51:46.382833+0000","last_became_active":"2024-06-10T02:50:12.235972+0000","last_became_peered":"2024-06-10T02:50:12.235972+0000","last_unstale":"2024-06-10T02:51:46.382833+0000","last_undegraded":"2024-06-10T02:51:46.382833+0000","last_fullsized":"2024-06-10T02:51:46.382833+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:38:41.723895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"98'4","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.295527+0000","last_change":"2024-06-10T02:49:58.996616+0000","last_active":"2024-06-10T02:51:46.295527+0000","last_peered":"2024-06-10T02:51:46.295527+0000","last_clean":"2024-06-10T02:51:46.295527+0000","last_became_active":"2024-06-10T02:49:58.996289+0000","last_became_peered":"2024-06-10T02:49:58.996289+0000","last_unstale":"2024-06-10T02:51:46.295527+0000","last_undegraded":"2024-06-10T02:51:46.295527+0000","last_fullsized":"2024-06-10T02:51:46.295527+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:22:12.993608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"100'9","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.983255+0000","last_change":"2024-06-10T02:50:02.205230+0000","last_active":"2024-06-10T02:52:42.983255+0000","last_peered":"2024-06-10T02:52:42.983255+0000","last_clean":"2024-06-10T02:52:42.983255+0000","last_became_active":"2024-06-10T02:50:02.204835+0000","last_became_peered":"2024-06-10T02:50:02.204835+0000","last_unstale":"2024-06-10T02:52:42.983255+0000","last_undegraded":"2024-06-10T02:52:42.983255+0000","last_fullsized":"2024-06-10T02:52:42.983255+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:17:35.084921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.383415+0000","last_change":"2024-06-10T02:50:12.231684+0000","last_active":"2024-06-10T02:51:46.383415+0000","last_peered":"2024-06-10T02:51:46.383415+0000","last_clean":"2024-06-10T02:51:46.383415+0000","last_became_active":"2024-06-10T02:50:12.231256+0000","last_became_peered":"2024-06-10T02:50:12.231256+0000","last_unstale":"2024-06-10T02:51:46.383415+0000","last_undegraded":"2024-06-10T02:51:46.383415+0000","last_fullsized":"2024-06-10T02:51:46.383415+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:17:31.878289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"98'5","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.383170+0000","last_change":"2024-06-10T02:49:59.033950+0000","last_active":"2024-06-10T02:51:46.383170+0000","last_peered":"2024-06-10T02:51:46.383170+0000","last_clean":"2024-06-10T02:51:46.383170+0000","last_became_active":"2024-06-10T02:49:59.033688+0000","last_became_peered":"2024-06-10T02:49:59.033688+0000","last_unstale":"2024-06-10T02:51:46.383170+0000","last_undegraded":"2024-06-10T02:51:46.383170+0000","last_fullsized":"2024-06-10T02:51:46.383170+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:32:03.892091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"100'11","reported_seq":56,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.999378+0000","last_change":"2024-06-10T02:50:44.781378+0000","last_active":"2024-06-10T02:52:42.999378+0000","last_peered":"2024-06-10T02:52:42.999378+0000","last_clean":"2024-06-10T02:52:42.999378+0000","last_became_active":"2024-06-10T02:50:44.780955+0000","last_became_peered":"2024-06-10T02:50:44.780955+0000","last_unstale":"2024-06-10T02:52:42.999378+0000","last_undegraded":"2024-06-10T02:52:42.999378+0000","last_fullsized":"2024-06-10T02:52:42.999378+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:49:13.760381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.827482+0000","last_change":"2024-06-10T02:51:46.384748+0000","last_active":"2024-06-10T02:51:46.827482+0000","last_peered":"2024-06-10T02:51:46.827482+0000","last_clean":"2024-06-10T02:51:46.827482+0000","last_became_active":"2024-06-10T02:51:46.384402+0000","last_became_peered":"2024-06-10T02:51:46.384402+0000","last_unstale":"2024-06-10T02:51:46.827482+0000","last_undegraded":"2024-06-10T02:51:46.827482+0000","last_fullsized":"2024-06-10T02:51:46.827482+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:06:48.056342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.14","version":"100'41","reported_seq":494,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:54:25.937884+0000","last_change":"2024-06-10T02:49:58.997890+0000","last_active":"2024-06-10T02:54:25.937884+0000","last_peered":"2024-06-10T02:54:25.937884+0000","last_clean":"2024-06-10T02:54:25.937884+0000","last_became_active":"2024-06-10T02:49:58.997549+0000","last_became_peered":"2024-06-10T02:49:58.997549+0000","last_unstale":"2024-06-10T02:54:25.937884+0000","last_undegraded":"2024-06-10T02:54:25.937884+0000","last_fullsized":"2024-06-10T02:54:25.937884+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T14:40:50.907029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71495,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":408,"num_read_kb":275,"num_write":43,"num_write_kb":90,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"100'6","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.991053+0000","last_change":"2024-06-10T02:50:02.199707+0000","last_active":"2024-06-10T02:52:42.991053+0000","last_peered":"2024-06-10T02:52:42.991053+0000","last_clean":"2024-06-10T02:52:42.991053+0000","last_became_active":"2024-06-10T02:50:02.198987+0000","last_became_peered":"2024-06-10T02:50:02.198987+0000","last_unstale":"2024-06-10T02:52:42.991053+0000","last_undegraded":"2024-06-10T02:52:42.991053+0000","last_fullsized":"2024-06-10T02:52:42.991053+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:48:03.704590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.383847+0000","last_change":"2024-06-10T02:50:12.239608+0000","last_active":"2024-06-10T02:51:46.383847+0000","last_peered":"2024-06-10T02:51:46.383847+0000","last_clean":"2024-06-10T02:51:46.383847+0000","last_became_active":"2024-06-10T02:50:12.238588+0000","last_became_peered":"2024-06-10T02:50:12.238588+0000","last_unstale":"2024-06-10T02:51:46.383847+0000","last_undegraded":"2024-06-10T02:51:46.383847+0000","last_fullsized":"2024-06-10T02:51:46.383847+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:26:12.718921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"98'9","reported_seq":56,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.721583+0000","last_change":"2024-06-10T02:49:58.997193+0000","last_active":"2024-06-10T02:51:45.721583+0000","last_peered":"2024-06-10T02:51:45.721583+0000","last_clean":"2024-06-10T02:51:45.721583+0000","last_became_active":"2024-06-10T02:49:58.996712+0000","last_became_peered":"2024-06-10T02:49:58.996712+0000","last_unstale":"2024-06-10T02:51:45.721583+0000","last_undegraded":"2024-06-10T02:51:45.721583+0000","last_fullsized":"2024-06-10T02:51:45.721583+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:02:46.894489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"100'9","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.994408+0000","last_change":"2024-06-10T02:50:02.200544+0000","last_active":"2024-06-10T02:52:42.994408+0000","last_peered":"2024-06-10T02:52:42.994408+0000","last_clean":"2024-06-10T02:52:42.994408+0000","last_became_active":"2024-06-10T02:50:02.200201+0000","last_became_peered":"2024-06-10T02:50:02.200201+0000","last_unstale":"2024-06-10T02:52:42.994408+0000","last_undegraded":"2024-06-10T02:52:42.994408+0000","last_fullsized":"2024-06-10T02:52:42.994408+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T13:12:40.925448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.365243+0000","last_change":"2024-06-10T02:50:12.236317+0000","last_active":"2024-06-10T02:51:46.365243+0000","last_peered":"2024-06-10T02:51:46.365243+0000","last_clean":"2024-06-10T02:51:46.365243+0000","last_became_active":"2024-06-10T02:50:12.235806+0000","last_became_peered":"2024-06-10T02:50:12.235806+0000","last_unstale":"2024-06-10T02:51:46.365243+0000","last_undegraded":"2024-06-10T02:51:46.365243+0000","last_fullsized":"2024-06-10T02:51:46.365243+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:29:21.425987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"98'5","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.383944+0000","last_change":"2024-06-10T02:49:58.994149+0000","last_active":"2024-06-10T02:51:46.383944+0000","last_peered":"2024-06-10T02:51:46.383944+0000","last_clean":"2024-06-10T02:51:46.383944+0000","last_became_active":"2024-06-10T02:49:58.993775+0000","last_became_peered":"2024-06-10T02:49:58.993775+0000","last_unstale":"2024-06-10T02:51:46.383944+0000","last_undegraded":"2024-06-10T02:51:46.383944+0000","last_fullsized":"2024-06-10T02:51:46.383944+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:23:55.553943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"100'15","reported_seq":54,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.995273+0000","last_change":"2024-06-10T02:50:02.199979+0000","last_active":"2024-06-10T02:52:42.995273+0000","last_peered":"2024-06-10T02:52:42.995273+0000","last_clean":"2024-06-10T02:52:42.995273+0000","last_became_active":"2024-06-10T02:50:02.199596+0000","last_became_peered":"2024-06-10T02:50:02.199596+0000","last_unstale":"2024-06-10T02:52:42.995273+0000","last_undegraded":"2024-06-10T02:52:42.995273+0000","last_fullsized":"2024-06-10T02:52:42.995273+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:13:58.647460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.383103+0000","last_change":"2024-06-10T02:50:12.235253+0000","last_active":"2024-06-10T02:51:46.383103+0000","last_peered":"2024-06-10T02:51:46.383103+0000","last_clean":"2024-06-10T02:51:46.383103+0000","last_became_active":"2024-06-10T02:50:12.234773+0000","last_became_peered":"2024-06-10T02:50:12.234773+0000","last_unstale":"2024-06-10T02:51:46.383103+0000","last_undegraded":"2024-06-10T02:51:46.383103+0000","last_fullsized":"2024-06-10T02:51:46.383103+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:38:10.971411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"100'7","reported_seq":253,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:54:24.914341+0000","last_change":"2024-06-10T02:49:58.996912+0000","last_active":"2024-06-10T02:54:24.914341+0000","last_peered":"2024-06-10T02:54:24.914341+0000","last_clean":"2024-06-10T02:54:24.914341+0000","last_became_active":"2024-06-10T02:49:58.996549+0000","last_became_peered":"2024-06-10T02:49:58.996549+0000","last_unstale":"2024-06-10T02:54:24.914341+0000","last_undegraded":"2024-06-10T02:54:24.914341+0000","last_fullsized":"2024-06-10T02:54:24.914341+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:07:52.757676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1119,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":202,"num_read_kb":5,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"100'6","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.995546+0000","last_change":"2024-06-10T02:50:44.741186+0000","last_active":"2024-06-10T02:52:42.995546+0000","last_peered":"2024-06-10T02:52:42.995546+0000","last_clean":"2024-06-10T02:52:42.995546+0000","last_became_active":"2024-06-10T02:50:44.740782+0000","last_became_peered":"2024-06-10T02:50:44.740782+0000","last_unstale":"2024-06-10T02:52:42.995546+0000","last_undegraded":"2024-06-10T02:52:42.995546+0000","last_fullsized":"2024-06-10T02:52:42.995546+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:46:03.863243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.715228+0000","last_change":"2024-06-10T02:50:12.236505+0000","last_active":"2024-06-10T02:51:45.715228+0000","last_peered":"2024-06-10T02:51:45.715228+0000","last_clean":"2024-06-10T02:51:45.715228+0000","last_became_active":"2024-06-10T02:50:12.236112+0000","last_became_peered":"2024-06-10T02:50:12.236112+0000","last_unstale":"2024-06-10T02:51:45.715228+0000","last_undegraded":"2024-06-10T02:51:45.715228+0000","last_fullsized":"2024-06-10T02:51:45.715228+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:13:38.586736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"98'17","reported_seq":63,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.723449+0000","last_change":"2024-06-10T02:49:59.031965+0000","last_active":"2024-06-10T02:51:45.723449+0000","last_peered":"2024-06-10T02:51:45.723449+0000","last_clean":"2024-06-10T02:51:45.723449+0000","last_became_active":"2024-06-10T02:49:59.031495+0000","last_became_peered":"2024-06-10T02:49:59.031495+0000","last_unstale":"2024-06-10T02:51:45.723449+0000","last_undegraded":"2024-06-10T02:51:45.723449+0000","last_fullsized":"2024-06-10T02:51:45.723449+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T14:21:06.610677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"100'6","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:43.002009+0000","last_change":"2024-06-10T02:50:02.209926+0000","last_active":"2024-06-10T02:52:43.002009+0000","last_peered":"2024-06-10T02:52:43.002009+0000","last_clean":"2024-06-10T02:52:43.002009+0000","last_became_active":"2024-06-10T02:50:02.209540+0000","last_became_peered":"2024-06-10T02:50:02.209540+0000","last_unstale":"2024-06-10T02:52:43.002009+0000","last_undegraded":"2024-06-10T02:52:43.002009+0000","last_fullsized":"2024-06-10T02:52:43.002009+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:49:40.497692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:48.927934+0000","last_change":"2024-06-10T02:51:46.013156+0000","last_active":"2024-06-10T02:51:48.927934+0000","last_peered":"2024-06-10T02:51:48.927934+0000","last_clean":"2024-06-10T02:51:48.927934+0000","last_became_active":"2024-06-10T02:51:46.012769+0000","last_became_peered":"2024-06-10T02:51:46.012769+0000","last_unstale":"2024-06-10T02:51:48.927934+0000","last_undegraded":"2024-06-10T02:51:48.927934+0000","last_fullsized":"2024-06-10T02:51:48.927934+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:44:49.545676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,3,1],"acting":[8,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.f","version":"98'2","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.724214+0000","last_change":"2024-06-10T02:49:58.998372+0000","last_active":"2024-06-10T02:51:45.724214+0000","last_peered":"2024-06-10T02:51:45.724214+0000","last_clean":"2024-06-10T02:51:45.724214+0000","last_became_active":"2024-06-10T02:49:58.998058+0000","last_became_peered":"2024-06-10T02:49:58.998058+0000","last_unstale":"2024-06-10T02:51:45.724214+0000","last_undegraded":"2024-06-10T02:51:45.724214+0000","last_fullsized":"2024-06-10T02:51:45.724214+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:42:50.509851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"100'5","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.993794+0000","last_change":"2024-06-10T02:50:02.209256+0000","last_active":"2024-06-10T02:52:42.993794+0000","last_peered":"2024-06-10T02:52:42.993794+0000","last_clean":"2024-06-10T02:52:42.993794+0000","last_became_active":"2024-06-10T02:50:02.207727+0000","last_became_peered":"2024-06-10T02:50:02.207727+0000","last_unstale":"2024-06-10T02:52:42.993794+0000","last_undegraded":"2024-06-10T02:52:42.993794+0000","last_fullsized":"2024-06-10T02:52:42.993794+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:23:00.175943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":30,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:18.931244+0000","last_change":"2024-06-10T02:50:12.729438+0000","last_active":"2024-06-10T02:52:18.931244+0000","last_peered":"2024-06-10T02:52:18.931244+0000","last_clean":"2024-06-10T02:52:18.931244+0000","last_became_active":"2024-06-10T02:50:12.728918+0000","last_became_peered":"2024-06-10T02:50:12.728918+0000","last_unstale":"2024-06-10T02:52:18.931244+0000","last_undegraded":"2024-06-10T02:52:18.931244+0000","last_fullsized":"2024-06-10T02:52:18.931244+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:04:48.861763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"98'1","reported_seq":53,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:05.628603+0000","last_change":"2024-06-10T02:50:44.782241+0000","last_active":"2024-06-10T02:52:05.628603+0000","last_peered":"2024-06-10T02:52:05.628603+0000","last_clean":"2024-06-10T02:52:05.628603+0000","last_became_active":"2024-06-10T02:50:44.781799+0000","last_became_peered":"2024-06-10T02:50:44.781799+0000","last_unstale":"2024-06-10T02:52:05.628603+0000","last_undegraded":"2024-06-10T02:52:05.628603+0000","last_fullsized":"2024-06-10T02:52:05.628603+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:38:35.726258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"100'4","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.993135+0000","last_change":"2024-06-10T02:50:02.198888+0000","last_active":"2024-06-10T02:52:42.993135+0000","last_peered":"2024-06-10T02:52:42.993135+0000","last_clean":"2024-06-10T02:52:42.993135+0000","last_became_active":"2024-06-10T02:50:02.198346+0000","last_became_peered":"2024-06-10T02:50:02.198346+0000","last_unstale":"2024-06-10T02:52:42.993135+0000","last_undegraded":"2024-06-10T02:52:42.993135+0000","last_fullsized":"2024-06-10T02:52:42.993135+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:15:42.422898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":30,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:17.689328+0000","last_change":"2024-06-10T02:50:12.232763+0000","last_active":"2024-06-10T02:52:17.689328+0000","last_peered":"2024-06-10T02:52:17.689328+0000","last_clean":"2024-06-10T02:52:17.689328+0000","last_became_active":"2024-06-10T02:50:12.232390+0000","last_became_peered":"2024-06-10T02:50:12.232390+0000","last_unstale":"2024-06-10T02:52:17.689328+0000","last_undegraded":"2024-06-10T02:52:17.689328+0000","last_fullsized":"2024-06-10T02:52:17.689328+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:31:47.973500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"100'8","reported_seq":251,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:54:20.908803+0000","last_change":"2024-06-10T02:49:58.994308+0000","last_active":"2024-06-10T02:54:20.908803+0000","last_peered":"2024-06-10T02:54:20.908803+0000","last_clean":"2024-06-10T02:54:20.908803+0000","last_became_active":"2024-06-10T02:49:58.993494+0000","last_became_peered":"2024-06-10T02:49:58.993494+0000","last_unstale":"2024-06-10T02:54:20.908803+0000","last_undegraded":"2024-06-10T02:54:20.908803+0000","last_fullsized":"2024-06-10T02:54:20.908803+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:07:28.452276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1509,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":197,"num_read_kb":4,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"100'7","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.988698+0000","last_change":"2024-06-10T02:50:02.206872+0000","last_active":"2024-06-10T02:52:42.988698+0000","last_peered":"2024-06-10T02:52:42.988698+0000","last_clean":"2024-06-10T02:52:42.988698+0000","last_became_active":"2024-06-10T02:50:02.206307+0000","last_became_peered":"2024-06-10T02:50:02.206307+0000","last_unstale":"2024-06-10T02:52:42.988698+0000","last_undegraded":"2024-06-10T02:52:42.988698+0000","last_fullsized":"2024-06-10T02:52:42.988698+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:01:04.645674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.715107+0000","last_change":"2024-06-10T02:50:12.729392+0000","last_active":"2024-06-10T02:51:45.715107+0000","last_peered":"2024-06-10T02:51:45.715107+0000","last_clean":"2024-06-10T02:51:45.715107+0000","last_became_active":"2024-06-10T02:50:12.728866+0000","last_became_peered":"2024-06-10T02:50:12.728866+0000","last_unstale":"2024-06-10T02:51:45.715107+0000","last_undegraded":"2024-06-10T02:51:45.715107+0000","last_fullsized":"2024-06-10T02:51:45.715107+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:01:35.845184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"100'8","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.995032+0000","last_change":"2024-06-10T02:50:02.202303+0000","last_active":"2024-06-10T02:52:42.995032+0000","last_peered":"2024-06-10T02:52:42.995032+0000","last_clean":"2024-06-10T02:52:42.995032+0000","last_became_active":"2024-06-10T02:50:02.201998+0000","last_became_peered":"2024-06-10T02:50:02.201998+0000","last_unstale":"2024-06-10T02:52:42.995032+0000","last_undegraded":"2024-06-10T02:52:42.995032+0000","last_fullsized":"2024-06-10T02:52:42.995032+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:15:00.005693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"100'6","reported_seq":246,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:54:20.908359+0000","last_change":"2024-06-10T02:49:59.031513+0000","last_active":"2024-06-10T02:54:20.908359+0000","last_peered":"2024-06-10T02:54:20.908359+0000","last_clean":"2024-06-10T02:54:20.908359+0000","last_became_active":"2024-06-10T02:49:59.030965+0000","last_became_peered":"2024-06-10T02:49:59.030965+0000","last_unstale":"2024-06-10T02:54:20.908359+0000","last_undegraded":"2024-06-10T02:54:20.908359+0000","last_fullsized":"2024-06-10T02:54:20.908359+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:49:02.171578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":194,"num_read_kb":194,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"98'1","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:17.800716+0000","last_change":"2024-06-10T02:50:12.236334+0000","last_active":"2024-06-10T02:52:17.800716+0000","last_peered":"2024-06-10T02:52:17.800716+0000","last_clean":"2024-06-10T02:52:17.800716+0000","last_became_active":"2024-06-10T02:50:12.235820+0000","last_became_peered":"2024-06-10T02:50:12.235820+0000","last_unstale":"2024-06-10T02:52:17.800716+0000","last_undegraded":"2024-06-10T02:52:17.800716+0000","last_fullsized":"2024-06-10T02:52:17.800716+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:16:12.476881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"100'8","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:43.001311+0000","last_change":"2024-06-10T02:50:02.205146+0000","last_active":"2024-06-10T02:52:43.001311+0000","last_peered":"2024-06-10T02:52:43.001311+0000","last_clean":"2024-06-10T02:52:43.001311+0000","last_became_active":"2024-06-10T02:50:02.204785+0000","last_became_peered":"2024-06-10T02:50:02.204785+0000","last_unstale":"2024-06-10T02:52:43.001311+0000","last_undegraded":"2024-06-10T02:52:43.001311+0000","last_fullsized":"2024-06-10T02:52:43.001311+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T13:01:02.068989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"98'6","reported_seq":55,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:06.016314+0000","last_change":"2024-06-10T02:49:58.997654+0000","last_active":"2024-06-10T02:52:06.016314+0000","last_peered":"2024-06-10T02:52:06.016314+0000","last_clean":"2024-06-10T02:52:06.016314+0000","last_became_active":"2024-06-10T02:49:58.997294+0000","last_became_peered":"2024-06-10T02:49:58.997294+0000","last_unstale":"2024-06-10T02:52:06.016314+0000","last_undegraded":"2024-06-10T02:52:06.016314+0000","last_fullsized":"2024-06-10T02:52:06.016314+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:53:14.763198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":367,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.294476+0000","last_change":"2024-06-10T02:48:34.008201+0000","last_active":"2024-06-10T02:51:46.294476+0000","last_peered":"2024-06-10T02:51:46.294476+0000","last_clean":"2024-06-10T02:51:46.294476+0000","last_became_active":"2024-06-10T02:48:34.007633+0000","last_became_peered":"2024-06-10T02:48:34.007633+0000","last_unstale":"2024-06-10T02:51:46.294476+0000","last_undegraded":"2024-06-10T02:51:46.294476+0000","last_fullsized":"2024-06-10T02:51:46.294476+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:45:07.444823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:45:07.444823+0000","last_clean_scrub_stamp":"2024-06-10T02:45:07.444823+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:56:58.159559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":36,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.383275+0000","last_change":"2024-06-10T02:50:44.782130+0000","last_active":"2024-06-10T02:51:46.383275+0000","last_peered":"2024-06-10T02:51:46.383275+0000","last_clean":"2024-06-10T02:51:46.383275+0000","last_became_active":"2024-06-10T02:50:44.781530+0000","last_became_peered":"2024-06-10T02:50:44.781530+0000","last_unstale":"2024-06-10T02:51:46.383275+0000","last_undegraded":"2024-06-10T02:51:46.383275+0000","last_fullsized":"2024-06-10T02:51:46.383275+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:43:23.826207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,11,6,10],"acting":[5,11,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"100'5","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.994100+0000","last_change":"2024-06-10T02:50:02.213666+0000","last_active":"2024-06-10T02:52:42.994100+0000","last_peered":"2024-06-10T02:52:42.994100+0000","last_clean":"2024-06-10T02:52:42.994100+0000","last_became_active":"2024-06-10T02:50:02.213194+0000","last_became_peered":"2024-06-10T02:50:02.213194+0000","last_unstale":"2024-06-10T02:52:42.994100+0000","last_undegraded":"2024-06-10T02:52:42.994100+0000","last_fullsized":"2024-06-10T02:52:42.994100+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:22:55.905412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"98'4","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:02.798809+0000","last_change":"2024-06-10T02:49:58.997618+0000","last_active":"2024-06-10T02:52:02.798809+0000","last_peered":"2024-06-10T02:52:02.798809+0000","last_clean":"2024-06-10T02:52:02.798809+0000","last_became_active":"2024-06-10T02:49:58.997304+0000","last_became_peered":"2024-06-10T02:49:58.997304+0000","last_unstale":"2024-06-10T02:52:02.798809+0000","last_undegraded":"2024-06-10T02:52:02.798809+0000","last_fullsized":"2024-06-10T02:52:02.798809+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:48:59.520991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":30,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:17.877545+0000","last_change":"2024-06-10T02:50:12.236357+0000","last_active":"2024-06-10T02:52:17.877545+0000","last_peered":"2024-06-10T02:52:17.877545+0000","last_clean":"2024-06-10T02:52:17.877545+0000","last_became_active":"2024-06-10T02:50:12.235915+0000","last_became_peered":"2024-06-10T02:50:12.235915+0000","last_unstale":"2024-06-10T02:52:17.877545+0000","last_undegraded":"2024-06-10T02:52:17.877545+0000","last_fullsized":"2024-06-10T02:52:17.877545+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:20:06.287907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"98'4","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:03.929566+0000","last_change":"2024-06-10T02:49:59.031725+0000","last_active":"2024-06-10T02:52:03.929566+0000","last_peered":"2024-06-10T02:52:03.929566+0000","last_clean":"2024-06-10T02:52:03.929566+0000","last_became_active":"2024-06-10T02:49:59.031296+0000","last_became_peered":"2024-06-10T02:49:59.031296+0000","last_unstale":"2024-06-10T02:52:03.929566+0000","last_undegraded":"2024-06-10T02:52:03.929566+0000","last_fullsized":"2024-06-10T02:52:03.929566+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T13:17:58.571696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"100'9","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.986516+0000","last_change":"2024-06-10T02:50:02.201820+0000","last_active":"2024-06-10T02:52:42.986516+0000","last_peered":"2024-06-10T02:52:42.986516+0000","last_clean":"2024-06-10T02:52:42.986516+0000","last_became_active":"2024-06-10T02:50:02.201478+0000","last_became_peered":"2024-06-10T02:50:02.201478+0000","last_unstale":"2024-06-10T02:52:42.986516+0000","last_undegraded":"2024-06-10T02:52:42.986516+0000","last_fullsized":"2024-06-10T02:52:42.986516+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:57:25.963926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":30,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:12.118458+0000","last_change":"2024-06-10T02:50:12.729418+0000","last_active":"2024-06-10T02:52:12.118458+0000","last_peered":"2024-06-10T02:52:12.118458+0000","last_clean":"2024-06-10T02:52:12.118458+0000","last_became_active":"2024-06-10T02:50:12.728692+0000","last_became_peered":"2024-06-10T02:50:12.728692+0000","last_unstale":"2024-06-10T02:52:12.118458+0000","last_undegraded":"2024-06-10T02:52:12.118458+0000","last_fullsized":"2024-06-10T02:52:12.118458+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:25:00.472396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"100'18","reported_seq":64,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:54:08.234903+0000","last_change":"2024-06-10T02:49:58.996391+0000","last_active":"2024-06-10T02:54:08.234903+0000","last_peered":"2024-06-10T02:54:08.234903+0000","last_clean":"2024-06-10T02:54:08.234903+0000","last_became_active":"2024-06-10T02:49:58.996033+0000","last_became_peered":"2024-06-10T02:49:58.996033+0000","last_unstale":"2024-06-10T02:54:08.234903+0000","last_undegraded":"2024-06-10T02:54:08.234903+0000","last_fullsized":"2024-06-10T02:54:08.234903+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:03:34.568969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":34,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"100'5","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.995465+0000","last_change":"2024-06-10T02:50:02.197974+0000","last_active":"2024-06-10T02:52:42.995465+0000","last_peered":"2024-06-10T02:52:42.995465+0000","last_clean":"2024-06-10T02:52:42.995465+0000","last_became_active":"2024-06-10T02:50:02.197614+0000","last_became_peered":"2024-06-10T02:50:02.197614+0000","last_unstale":"2024-06-10T02:52:42.995465+0000","last_undegraded":"2024-06-10T02:52:42.995465+0000","last_fullsized":"2024-06-10T02:52:42.995465+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:36:55.575808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":30,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:16.242935+0000","last_change":"2024-06-10T02:50:12.234239+0000","last_active":"2024-06-10T02:52:16.242935+0000","last_peered":"2024-06-10T02:52:16.242935+0000","last_clean":"2024-06-10T02:52:16.242935+0000","last_became_active":"2024-06-10T02:50:12.233768+0000","last_became_peered":"2024-06-10T02:50:12.233768+0000","last_unstale":"2024-06-10T02:52:16.242935+0000","last_undegraded":"2024-06-10T02:52:16.242935+0000","last_fullsized":"2024-06-10T02:52:16.242935+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:51:44.230031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"98'7","reported_seq":54,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:11.016968+0000","last_change":"2024-06-10T02:49:59.192765+0000","last_active":"2024-06-10T02:52:11.016968+0000","last_peered":"2024-06-10T02:52:11.016968+0000","last_clean":"2024-06-10T02:52:11.016968+0000","last_became_active":"2024-06-10T02:49:59.192446+0000","last_became_peered":"2024-06-10T02:49:59.192446+0000","last_unstale":"2024-06-10T02:52:11.016968+0000","last_undegraded":"2024-06-10T02:52:11.016968+0000","last_fullsized":"2024-06-10T02:52:11.016968+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:53:51.862019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"100'7","reported_seq":53,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.989869+0000","last_change":"2024-06-10T02:50:44.781209+0000","last_active":"2024-06-10T02:52:42.989869+0000","last_peered":"2024-06-10T02:52:42.989869+0000","last_clean":"2024-06-10T02:52:42.989869+0000","last_became_active":"2024-06-10T02:50:44.780811+0000","last_became_peered":"2024-06-10T02:50:44.780811+0000","last_unstale":"2024-06-10T02:52:42.989869+0000","last_undegraded":"2024-06-10T02:52:42.989869+0000","last_fullsized":"2024-06-10T02:52:42.989869+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:12:40.663565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.383517+0000","last_change":"2024-06-10T02:50:12.730269+0000","last_active":"2024-06-10T02:51:46.383517+0000","last_peered":"2024-06-10T02:51:46.383517+0000","last_clean":"2024-06-10T02:51:46.383517+0000","last_became_active":"2024-06-10T02:50:12.729574+0000","last_became_peered":"2024-06-10T02:50:12.729574+0000","last_unstale":"2024-06-10T02:51:46.383517+0000","last_undegraded":"2024-06-10T02:51:46.383517+0000","last_fullsized":"2024-06-10T02:51:46.383517+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:51:16.235284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"98'2","reported_seq":54,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:12.774945+0000","last_change":"2024-06-10T02:50:44.782421+0000","last_active":"2024-06-10T02:52:12.774945+0000","last_peered":"2024-06-10T02:52:12.774945+0000","last_clean":"2024-06-10T02:52:12.774945+0000","last_became_active":"2024-06-10T02:50:44.781881+0000","last_became_peered":"2024-06-10T02:50:44.781881+0000","last_unstale":"2024-06-10T02:52:12.774945+0000","last_undegraded":"2024-06-10T02:52:12.774945+0000","last_fullsized":"2024-06-10T02:52:12.774945+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T14:41:58.771256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"100'2","reported_seq":42,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.984615+0000","last_change":"2024-06-10T02:50:02.197761+0000","last_active":"2024-06-10T02:52:42.984615+0000","last_peered":"2024-06-10T02:52:42.984615+0000","last_clean":"2024-06-10T02:52:42.984615+0000","last_became_active":"2024-06-10T02:50:02.197395+0000","last_became_peered":"2024-06-10T02:50:02.197395+0000","last_unstale":"2024-06-10T02:52:42.984615+0000","last_undegraded":"2024-06-10T02:52:42.984615+0000","last_fullsized":"2024-06-10T02:52:42.984615+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:46:37.537280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:48.927972+0000","last_change":"2024-06-10T02:51:46.385129+0000","last_active":"2024-06-10T02:51:48.927972+0000","last_peered":"2024-06-10T02:51:48.927972+0000","last_clean":"2024-06-10T02:51:48.927972+0000","last_became_active":"2024-06-10T02:51:46.384553+0000","last_became_peered":"2024-06-10T02:51:46.384553+0000","last_unstale":"2024-06-10T02:51:48.927972+0000","last_undegraded":"2024-06-10T02:51:48.927972+0000","last_fullsized":"2024-06-10T02:51:48.927972+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:08:11.183679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"98'7","reported_seq":53,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:12.118484+0000","last_change":"2024-06-10T02:49:58.987863+0000","last_active":"2024-06-10T02:52:12.118484+0000","last_peered":"2024-06-10T02:52:12.118484+0000","last_clean":"2024-06-10T02:52:12.118484+0000","last_became_active":"2024-06-10T02:49:58.987388+0000","last_became_peered":"2024-06-10T02:49:58.987388+0000","last_unstale":"2024-06-10T02:52:12.118484+0000","last_undegraded":"2024-06-10T02:52:12.118484+0000","last_fullsized":"2024-06-10T02:52:12.118484+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:29:20.340301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"100'9","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.989752+0000","last_change":"2024-06-10T02:50:02.200699+0000","last_active":"2024-06-10T02:52:42.989752+0000","last_peered":"2024-06-10T02:52:42.989752+0000","last_clean":"2024-06-10T02:52:42.989752+0000","last_became_active":"2024-06-10T02:50:02.200278+0000","last_became_peered":"2024-06-10T02:50:02.200278+0000","last_unstale":"2024-06-10T02:52:42.989752+0000","last_undegraded":"2024-06-10T02:52:42.989752+0000","last_fullsized":"2024-06-10T02:52:42.989752+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:26:36.236312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"98'1","reported_seq":30,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.383543+0000","last_change":"2024-06-10T02:50:12.236027+0000","last_active":"2024-06-10T02:51:46.383543+0000","last_peered":"2024-06-10T02:51:46.383543+0000","last_clean":"2024-06-10T02:51:46.383543+0000","last_became_active":"2024-06-10T02:50:12.235629+0000","last_became_peered":"2024-06-10T02:50:12.235629+0000","last_unstale":"2024-06-10T02:51:46.383543+0000","last_undegraded":"2024-06-10T02:51:46.383543+0000","last_fullsized":"2024-06-10T02:51:46.383543+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:58:48.218806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"98'3","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:11.242480+0000","last_change":"2024-06-10T02:49:58.998496+0000","last_active":"2024-06-10T02:52:11.242480+0000","last_peered":"2024-06-10T02:52:11.242480+0000","last_clean":"2024-06-10T02:52:11.242480+0000","last_became_active":"2024-06-10T02:49:58.997260+0000","last_became_peered":"2024-06-10T02:49:58.997260+0000","last_unstale":"2024-06-10T02:52:11.242480+0000","last_undegraded":"2024-06-10T02:52:11.242480+0000","last_fullsized":"2024-06-10T02:52:11.242480+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T14:22:04.658943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"100'8","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.990375+0000","last_change":"2024-06-10T02:50:02.201300+0000","last_active":"2024-06-10T02:52:42.990375+0000","last_peered":"2024-06-10T02:52:42.990375+0000","last_clean":"2024-06-10T02:52:42.990375+0000","last_became_active":"2024-06-10T02:50:02.200988+0000","last_became_peered":"2024-06-10T02:50:02.200988+0000","last_unstale":"2024-06-10T02:52:42.990375+0000","last_undegraded":"2024-06-10T02:52:42.990375+0000","last_fullsized":"2024-06-10T02:52:42.990375+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:11:12.962048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.383484+0000","last_change":"2024-06-10T02:50:12.728869+0000","last_active":"2024-06-10T02:51:46.383484+0000","last_peered":"2024-06-10T02:51:46.383484+0000","last_clean":"2024-06-10T02:51:46.383484+0000","last_became_active":"2024-06-10T02:50:12.728189+0000","last_became_peered":"2024-06-10T02:50:12.728189+0000","last_unstale":"2024-06-10T02:51:46.383484+0000","last_undegraded":"2024-06-10T02:51:46.383484+0000","last_fullsized":"2024-06-10T02:51:46.383484+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:08:36.624814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"98'4","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:10.967286+0000","last_change":"2024-06-10T02:49:59.000460+0000","last_active":"2024-06-10T02:52:10.967286+0000","last_peered":"2024-06-10T02:52:10.967286+0000","last_clean":"2024-06-10T02:52:10.967286+0000","last_became_active":"2024-06-10T02:49:59.000075+0000","last_became_peered":"2024-06-10T02:49:59.000075+0000","last_unstale":"2024-06-10T02:52:10.967286+0000","last_undegraded":"2024-06-10T02:52:10.967286+0000","last_fullsized":"2024-06-10T02:52:10.967286+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:06:27.935745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"100'5","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:43.001259+0000","last_change":"2024-06-10T02:50:02.210131+0000","last_active":"2024-06-10T02:52:43.001259+0000","last_peered":"2024-06-10T02:52:43.001259+0000","last_clean":"2024-06-10T02:52:43.001259+0000","last_became_active":"2024-06-10T02:50:02.209781+0000","last_became_peered":"2024-06-10T02:50:02.209781+0000","last_unstale":"2024-06-10T02:52:43.001259+0000","last_undegraded":"2024-06-10T02:52:43.001259+0000","last_fullsized":"2024-06-10T02:52:43.001259+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:27:39.966620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.854407+0000","last_change":"2024-06-10T02:50:12.236017+0000","last_active":"2024-06-10T02:51:45.854407+0000","last_peered":"2024-06-10T02:51:45.854407+0000","last_clean":"2024-06-10T02:51:45.854407+0000","last_became_active":"2024-06-10T02:50:12.235553+0000","last_became_peered":"2024-06-10T02:50:12.235553+0000","last_unstale":"2024-06-10T02:51:45.854407+0000","last_undegraded":"2024-06-10T02:51:45.854407+0000","last_fullsized":"2024-06-10T02:51:45.854407+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:23:55.437994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"100'7","reported_seq":253,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:54:24.914730+0000","last_change":"2024-06-10T02:49:59.192864+0000","last_active":"2024-06-10T02:54:24.914730+0000","last_peered":"2024-06-10T02:54:24.914730+0000","last_clean":"2024-06-10T02:54:24.914730+0000","last_became_active":"2024-06-10T02:49:59.192440+0000","last_became_peered":"2024-06-10T02:49:59.192440+0000","last_unstale":"2024-06-10T02:54:24.914730+0000","last_undegraded":"2024-06-10T02:54:24.914730+0000","last_fullsized":"2024-06-10T02:54:24.914730+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:36:05.868538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1701,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":201,"num_read_kb":4,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"100'7","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:43.000525+0000","last_change":"2024-06-10T02:50:02.205791+0000","last_active":"2024-06-10T02:52:43.000525+0000","last_peered":"2024-06-10T02:52:43.000525+0000","last_clean":"2024-06-10T02:52:43.000525+0000","last_became_active":"2024-06-10T02:50:02.205483+0000","last_became_peered":"2024-06-10T02:50:02.205483+0000","last_unstale":"2024-06-10T02:52:43.000525+0000","last_undegraded":"2024-06-10T02:52:43.000525+0000","last_fullsized":"2024-06-10T02:52:43.000525+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:33:25.875423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.011597+0000","last_change":"2024-06-10T02:50:12.239294+0000","last_active":"2024-06-10T02:51:46.011597+0000","last_peered":"2024-06-10T02:51:46.011597+0000","last_clean":"2024-06-10T02:51:46.011597+0000","last_became_active":"2024-06-10T02:50:12.238778+0000","last_became_peered":"2024-06-10T02:50:12.238778+0000","last_unstale":"2024-06-10T02:51:46.011597+0000","last_undegraded":"2024-06-10T02:51:46.011597+0000","last_fullsized":"2024-06-10T02:51:46.011597+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:26:09.203623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"98'3","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.383589+0000","last_change":"2024-06-10T02:49:58.989680+0000","last_active":"2024-06-10T02:51:46.383589+0000","last_peered":"2024-06-10T02:51:46.383589+0000","last_clean":"2024-06-10T02:51:46.383589+0000","last_became_active":"2024-06-10T02:49:58.989265+0000","last_became_peered":"2024-06-10T02:49:58.989265+0000","last_unstale":"2024-06-10T02:51:46.383589+0000","last_undegraded":"2024-06-10T02:51:46.383589+0000","last_fullsized":"2024-06-10T02:51:46.383589+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:36:34.370177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"100'6","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.990330+0000","last_change":"2024-06-10T02:50:44.743369+0000","last_active":"2024-06-10T02:52:42.990330+0000","last_peered":"2024-06-10T02:52:42.990330+0000","last_clean":"2024-06-10T02:52:42.990330+0000","last_became_active":"2024-06-10T02:50:44.743013+0000","last_became_peered":"2024-06-10T02:50:44.743013+0000","last_unstale":"2024-06-10T02:52:42.990330+0000","last_undegraded":"2024-06-10T02:52:42.990330+0000","last_fullsized":"2024-06-10T02:52:42.990330+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:54:43.728943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.382656+0000","last_change":"2024-06-10T02:50:12.235197+0000","last_active":"2024-06-10T02:51:46.382656+0000","last_peered":"2024-06-10T02:51:46.382656+0000","last_clean":"2024-06-10T02:51:46.382656+0000","last_became_active":"2024-06-10T02:50:12.234791+0000","last_became_peered":"2024-06-10T02:50:12.234791+0000","last_unstale":"2024-06-10T02:51:46.382656+0000","last_undegraded":"2024-06-10T02:51:46.382656+0000","last_fullsized":"2024-06-10T02:51:46.382656+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:09:11.164236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"98'5","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.715164+0000","last_change":"2024-06-10T02:49:58.996759+0000","last_active":"2024-06-10T02:51:45.715164+0000","last_peered":"2024-06-10T02:51:45.715164+0000","last_clean":"2024-06-10T02:51:45.715164+0000","last_became_active":"2024-06-10T02:49:58.996434+0000","last_became_peered":"2024-06-10T02:49:58.996434+0000","last_unstale":"2024-06-10T02:51:45.715164+0000","last_undegraded":"2024-06-10T02:51:45.715164+0000","last_fullsized":"2024-06-10T02:51:45.715164+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:09:35.327797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"100'9","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.996813+0000","last_change":"2024-06-10T02:50:02.201676+0000","last_active":"2024-06-10T02:52:42.996813+0000","last_peered":"2024-06-10T02:52:42.996813+0000","last_clean":"2024-06-10T02:52:42.996813+0000","last_became_active":"2024-06-10T02:50:02.201335+0000","last_became_peered":"2024-06-10T02:50:02.201335+0000","last_unstale":"2024-06-10T02:52:42.996813+0000","last_undegraded":"2024-06-10T02:52:42.996813+0000","last_fullsized":"2024-06-10T02:52:42.996813+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:49:47.038946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"100'9","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.992923+0000","last_change":"2024-06-10T02:50:02.203462+0000","last_active":"2024-06-10T02:52:42.992923+0000","last_peered":"2024-06-10T02:52:42.992923+0000","last_clean":"2024-06-10T02:52:42.992923+0000","last_became_active":"2024-06-10T02:50:02.203115+0000","last_became_peered":"2024-06-10T02:50:02.203115+0000","last_unstale":"2024-06-10T02:52:42.992923+0000","last_undegraded":"2024-06-10T02:52:42.992923+0000","last_fullsized":"2024-06-10T02:52:42.992923+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T13:18:12.400596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"100'12","reported_seq":52,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.998015+0000","last_change":"2024-06-10T02:50:02.205648+0000","last_active":"2024-06-10T02:52:42.998015+0000","last_peered":"2024-06-10T02:52:42.998015+0000","last_clean":"2024-06-10T02:52:42.998015+0000","last_became_active":"2024-06-10T02:50:02.205306+0000","last_became_peered":"2024-06-10T02:50:02.205306+0000","last_unstale":"2024-06-10T02:52:42.998015+0000","last_undegraded":"2024-06-10T02:52:42.998015+0000","last_fullsized":"2024-06-10T02:52:42.998015+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:22:55.020644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"100'8","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.991617+0000","last_change":"2024-06-10T02:50:02.198601+0000","last_active":"2024-06-10T02:52:42.991617+0000","last_peered":"2024-06-10T02:52:42.991617+0000","last_clean":"2024-06-10T02:52:42.991617+0000","last_became_active":"2024-06-10T02:50:02.198103+0000","last_became_peered":"2024-06-10T02:50:02.198103+0000","last_unstale":"2024-06-10T02:52:42.991617+0000","last_undegraded":"2024-06-10T02:52:42.991617+0000","last_fullsized":"2024-06-10T02:52:42.991617+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:56:28.033482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"100'7","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.985450+0000","last_change":"2024-06-10T02:50:02.202638+0000","last_active":"2024-06-10T02:52:42.985450+0000","last_peered":"2024-06-10T02:52:42.985450+0000","last_clean":"2024-06-10T02:52:42.985450+0000","last_became_active":"2024-06-10T02:50:02.202249+0000","last_became_peered":"2024-06-10T02:50:02.202249+0000","last_unstale":"2024-06-10T02:52:42.985450+0000","last_undegraded":"2024-06-10T02:52:42.985450+0000","last_fullsized":"2024-06-10T02:52:42.985450+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:46:29.424164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"100'4","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.992759+0000","last_change":"2024-06-10T02:50:02.213526+0000","last_active":"2024-06-10T02:52:42.992759+0000","last_peered":"2024-06-10T02:52:42.992759+0000","last_clean":"2024-06-10T02:52:42.992759+0000","last_became_active":"2024-06-10T02:50:02.213226+0000","last_became_peered":"2024-06-10T02:50:02.213226+0000","last_unstale":"2024-06-10T02:52:42.992759+0000","last_undegraded":"2024-06-10T02:52:42.992759+0000","last_fullsized":"2024-06-10T02:52:42.992759+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:01:56.846340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"100'6","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.993163+0000","last_change":"2024-06-10T02:50:02.206583+0000","last_active":"2024-06-10T02:52:42.993163+0000","last_peered":"2024-06-10T02:52:42.993163+0000","last_clean":"2024-06-10T02:52:42.993163+0000","last_became_active":"2024-06-10T02:50:02.206227+0000","last_became_peered":"2024-06-10T02:50:02.206227+0000","last_unstale":"2024-06-10T02:52:42.993163+0000","last_undegraded":"2024-06-10T02:52:42.993163+0000","last_fullsized":"2024-06-10T02:52:42.993163+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:25:07.246288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"100'15","reported_seq":55,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:43.003046+0000","last_change":"2024-06-10T02:50:02.206348+0000","last_active":"2024-06-10T02:52:43.003046+0000","last_peered":"2024-06-10T02:52:43.003046+0000","last_clean":"2024-06-10T02:52:43.003046+0000","last_became_active":"2024-06-10T02:50:02.206020+0000","last_became_peered":"2024-06-10T02:50:02.206020+0000","last_unstale":"2024-06-10T02:52:43.003046+0000","last_undegraded":"2024-06-10T02:52:43.003046+0000","last_fullsized":"2024-06-10T02:52:43.003046+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:02:26.619113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"100'2","reported_seq":42,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.993595+0000","last_change":"2024-06-10T02:50:02.199405+0000","last_active":"2024-06-10T02:52:42.993595+0000","last_peered":"2024-06-10T02:52:42.993595+0000","last_clean":"2024-06-10T02:52:42.993595+0000","last_became_active":"2024-06-10T02:50:02.199049+0000","last_became_peered":"2024-06-10T02:50:02.199049+0000","last_unstale":"2024-06-10T02:52:42.993595+0000","last_undegraded":"2024-06-10T02:52:42.993595+0000","last_fullsized":"2024-06-10T02:52:42.993595+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:33:04.091160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"100'6","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.992849+0000","last_change":"2024-06-10T02:50:02.202094+0000","last_active":"2024-06-10T02:52:42.992849+0000","last_peered":"2024-06-10T02:52:42.992849+0000","last_clean":"2024-06-10T02:52:42.992849+0000","last_became_active":"2024-06-10T02:50:02.201751+0000","last_became_peered":"2024-06-10T02:50:02.201751+0000","last_unstale":"2024-06-10T02:52:42.992849+0000","last_undegraded":"2024-06-10T02:52:42.992849+0000","last_fullsized":"2024-06-10T02:52:42.992849+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:52:53.946326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"100'8","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.993442+0000","last_change":"2024-06-10T02:50:02.199630+0000","last_active":"2024-06-10T02:52:42.993442+0000","last_peered":"2024-06-10T02:52:42.993442+0000","last_clean":"2024-06-10T02:52:42.993442+0000","last_became_active":"2024-06-10T02:50:02.199320+0000","last_became_peered":"2024-06-10T02:50:02.199320+0000","last_unstale":"2024-06-10T02:52:42.993442+0000","last_undegraded":"2024-06-10T02:52:42.993442+0000","last_fullsized":"2024-06-10T02:52:42.993442+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:17:28.761862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"100'10","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:43.000631+0000","last_change":"2024-06-10T02:50:02.201190+0000","last_active":"2024-06-10T02:52:43.000631+0000","last_peered":"2024-06-10T02:52:43.000631+0000","last_clean":"2024-06-10T02:52:43.000631+0000","last_became_active":"2024-06-10T02:50:02.200574+0000","last_became_peered":"2024-06-10T02:50:02.200574+0000","last_unstale":"2024-06-10T02:52:43.000631+0000","last_undegraded":"2024-06-10T02:52:43.000631+0000","last_fullsized":"2024-06-10T02:52:43.000631+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:02:07.636218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"100'11","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.992436+0000","last_change":"2024-06-10T02:50:02.202374+0000","last_active":"2024-06-10T02:52:42.992436+0000","last_peered":"2024-06-10T02:52:42.992436+0000","last_clean":"2024-06-10T02:52:42.992436+0000","last_became_active":"2024-06-10T02:50:02.201974+0000","last_became_peered":"2024-06-10T02:50:02.201974+0000","last_unstale":"2024-06-10T02:52:42.992436+0000","last_undegraded":"2024-06-10T02:52:42.992436+0000","last_fullsized":"2024-06-10T02:52:42.992436+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:45:34.977926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"100'8","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.995177+0000","last_change":"2024-06-10T02:50:02.209332+0000","last_active":"2024-06-10T02:52:42.995177+0000","last_peered":"2024-06-10T02:52:42.995177+0000","last_clean":"2024-06-10T02:52:42.995177+0000","last_became_active":"2024-06-10T02:50:02.208175+0000","last_became_peered":"2024-06-10T02:50:02.208175+0000","last_unstale":"2024-06-10T02:52:42.995177+0000","last_undegraded":"2024-06-10T02:52:42.995177+0000","last_fullsized":"2024-06-10T02:52:42.995177+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:44:40.857918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":284028,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":80739,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":2016,"num_read_kb":1099,"num_write":276,"num_write_kb":173,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":602112,"data_stored":242217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":237,"ondisk_log_size":237,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351045,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32260,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712892,"statfs":{"total":95995035648,"available":95962001408,"internally_reserved":0,"allocated":1110016,"data_stored":574044,"data_compressed":32447,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612683,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32416,"kb_used_data":1240,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712736,"statfs":{"total":95995035648,"available":95961841664,"internally_reserved":0,"allocated":1269760,"data_stored":1232355,"data_compressed":68564,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1599,"internal_metadata":31914433},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776208,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36448,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708704,"statfs":{"total":95995035648,"available":95957712896,"internally_reserved":0,"allocated":1138688,"data_stored":577539,"data_compressed":32447,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037844,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28112,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717040,"statfs":{"total":95995035648,"available":95966248960,"internally_reserved":0,"allocated":1122304,"data_stored":573149,"data_compressed":32447,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299481,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28172,"kb_used_data":1156,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716980,"statfs":{"total":95995035648,"available":95966187520,"internally_reserved":0,"allocated":1183744,"data_stored":1172218,"data_compressed":68564,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863463007,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36420,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708732,"statfs":{"total":95995035648,"available":95957741568,"internally_reserved":0,"allocated":1110016,"data_stored":569868,"data_compressed":32447,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574968,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36496,"kb_used_data":1160,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708656,"statfs":{"total":95995035648,"available":95957663744,"internally_reserved":0,"allocated":1187840,"data_stored":642393,"data_compressed":32447,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738493,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28124,"kb_used_data":1108,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717028,"statfs":{"total":95995035648,"available":95966236672,"internally_reserved":0,"allocated":1134592,"data_stored":570934,"data_compressed":32447,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411444,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28196,"kb_used_data":1180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716956,"statfs":{"total":95995035648,"available":95966162944,"internally_reserved":0,"allocated":1208320,"data_stored":643062,"data_compressed":32447,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149805,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36432,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708720,"statfs":{"total":95995035648,"available":95957729280,"internally_reserved":0,"allocated":1122304,"data_stored":568655,"data_compressed":32447,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888169,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28116,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717036,"statfs":{"total":95995035648,"available":95966244864,"internally_reserved":0,"allocated":1126400,"data_stored":574224,"data_compressed":32447,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724644,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28180,"kb_used_data":1164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716972,"statfs":{"total":95995035648,"available":95966179328,"internally_reserved":0,"allocated":1191936,"data_stored":1166348,"data_compressed":68564,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34481,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34481,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34481,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36117,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36117,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36117,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36117,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36117,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":74168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":74837,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1903,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":126976,"data_stored":73762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-06-10T02:54:29.021 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-06-10T02:54:29.021 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph pg dump --format=json 2024-06-10T02:54:29.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:54:28 smithi120 ceph-mon[36634]: pgmap v442: 129 pgs: 129 active+clean; 656 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-06-10T02:54:29.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:54:28 smithi120 ceph-mon[36634]: from='client.15405 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-10T02:54:29.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:54:28 smithi188 ceph-mon[36599]: pgmap v442: 129 pgs: 129 active+clean; 656 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-06-10T02:54:29.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:54:28 smithi188 ceph-mon[36599]: from='client.15405 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-10T02:54:29.282 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:54:30.856 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:54:30 smithi052 ceph-mon[32362]: pgmap v443: 129 pgs: 129 active+clean; 656 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-06-10T02:54:31.110 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:31.109+0000 7f16ca58d700 1 -- 172.21.15.52:0/2188188288 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f16c4103120 msgr2=0x7f16c4107130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:54:31.110 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:31.109+0000 7f16ca58d700 1 --2- 172.21.15.52:0/2188188288 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f16c4103120 0x7f16c4107130 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f16b80099c0 tx=0x7f16b8009cd0 comp rx=0 tx=0).stop 2024-06-10T02:54:31.110 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:31.109+0000 7f16ca58d700 1 -- 172.21.15.52:0/2188188288 shutdown_connections 2024-06-10T02:54:31.110 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:31.109+0000 7f16ca58d700 1 --2- 172.21.15.52:0/2188188288 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f16c4107860 0x7f16c410f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:31.110 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:31.109+0000 7f16ca58d700 1 --2- 172.21.15.52:0/2188188288 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f16c4102770 0x7f16c4102b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:31.111 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:31.109+0000 7f16ca58d700 1 --2- 172.21.15.52:0/2188188288 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f16c4103120 0x7f16c4107130 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:31.111 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:31.109+0000 7f16ca58d700 1 -- 172.21.15.52:0/2188188288 >> 172.21.15.52:0/2188188288 conn(0x7f16c4079600 msgr2=0x7f16c4079a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:54:31.111 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:31.110+0000 7f16ca58d700 1 -- 172.21.15.52:0/2188188288 shutdown_connections 2024-06-10T02:54:31.111 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:31.110+0000 7f16ca58d700 1 -- 172.21.15.52:0/2188188288 wait complete. 2024-06-10T02:54:31.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:31.110+0000 7f16ca58d700 1 Processor -- start 2024-06-10T02:54:31.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:31.110+0000 7f16ca58d700 1 -- start start 2024-06-10T02:54:31.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:31.111+0000 7f16ca58d700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f16c4102770 0x7f16c419afb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:31.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:31.111+0000 7f16ca58d700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f16c4103120 0x7f16c419b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:31.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:31.111+0000 7f16ca58d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f16c4107860 0x7f16c419f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:31.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:31.111+0000 7f16ca58d700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f16c41128c0 con 0x7f16c4107860 2024-06-10T02:54:31.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:31.111+0000 7f16ca58d700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f16c4112740 con 0x7f16c4103120 2024-06-10T02:54:31.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:31.111+0000 7f16ca58d700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f16c4112a40 con 0x7f16c4102770 2024-06-10T02:54:31.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:31.111+0000 7f16c8b2a700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f16c4107860 0x7f16c419f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:31.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:31.111+0000 7f16c8b2a700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f16c4107860 0x7f16c419f830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51828/0 (socket says 172.21.15.52:51828) 2024-06-10T02:54:31.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:31.111+0000 7f16c8b2a700 1 -- 172.21.15.52:0/2241024456 learned_addr learned my addr 172.21.15.52:0/2241024456 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:54:31.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:31.111+0000 7f16c3fff700 1 --2- 172.21.15.52:0/2241024456 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f16c4102770 0x7f16c419afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:31.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:31.111+0000 7f16c37fe700 1 --2- 172.21.15.52:0/2241024456 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f16c4103120 0x7f16c419b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:31.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:31.111+0000 7f16c8b2a700 1 -- 172.21.15.52:0/2241024456 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f16c4102770 msgr2=0x7f16c419afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:54:31.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:31.111+0000 7f16c8b2a700 1 --2- 172.21.15.52:0/2241024456 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f16c4102770 0x7f16c419afb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:31.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:31.111+0000 7f16c8b2a700 1 -- 172.21.15.52:0/2241024456 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f16c4103120 msgr2=0x7f16c419b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:54:31.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:31.111+0000 7f16c8b2a700 1 --2- 172.21.15.52:0/2241024456 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f16c4103120 0x7f16c419b4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:31.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:31.111+0000 7f16c8b2a700 1 -- 172.21.15.52:0/2241024456 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16c419ff70 con 0x7f16c4107860 2024-06-10T02:54:31.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:31.111+0000 7f16c37fe700 1 --2- 172.21.15.52:0/2241024456 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f16c4103120 0x7f16c419b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T02:54:31.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:31.112+0000 7f16c8b2a700 1 --2- 172.21.15.52:0/2241024456 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f16c4107860 0x7f16c419f830 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f16bc00b970 tx=0x7f16bc00bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:54:31.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:31.112+0000 7f16c17fa700 1 -- 172.21.15.52:0/2241024456 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16bc019070 con 0x7f16c4107860 2024-06-10T02:54:31.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:31.112+0000 7f16c17fa700 1 -- 172.21.15.52:0/2241024456 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16bc00ce10 con 0x7f16c4107860 2024-06-10T02:54:31.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:31.112+0000 7f16ca58d700 1 -- 172.21.15.52:0/2241024456 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16c41a0260 con 0x7f16c4107860 2024-06-10T02:54:31.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:31.112+0000 7f16c17fa700 1 -- 172.21.15.52:0/2241024456 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16bc007370 con 0x7f16c4107860 2024-06-10T02:54:31.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:31.112+0000 7f16ca58d700 1 -- 172.21.15.52:0/2241024456 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16c41a7ae0 con 0x7f16c4107860 2024-06-10T02:54:31.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:31.113+0000 7f16ca58d700 1 -- 172.21.15.52:0/2241024456 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16c4069180 con 0x7f16c4107860 2024-06-10T02:54:31.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:31.114+0000 7f16c17fa700 1 -- 172.21.15.52:0/2241024456 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7f16bc01c070 con 0x7f16c4107860 2024-06-10T02:54:31.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:31.115+0000 7f16c17fa700 1 --2- 172.21.15.52:0/2241024456 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f16ac0a2cb0 0x7f16ac0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:31.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:31.115+0000 7f16c17fa700 1 -- 172.21.15.52:0/2241024456 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f16bc0ce830 con 0x7f16c4107860 2024-06-10T02:54:31.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:31.115+0000 7f16c3fff700 1 --2- 172.21.15.52:0/2241024456 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f16ac0a2cb0 0x7f16ac0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:31.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:31.115+0000 7f16c3fff700 1 --2- 172.21.15.52:0/2241024456 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f16ac0a2cb0 0x7f16ac0a5170 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f16b4009a10 tx=0x7f16b4009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:54:31.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:31.120+0000 7f16c17fa700 1 -- 172.21.15.52:0/2241024456 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f16bc09b390 con 0x7f16c4107860 2024-06-10T02:54:31.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:54:30 smithi120 ceph-mon[36634]: pgmap v443: 129 pgs: 129 active+clean; 656 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-06-10T02:54:31.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:54:30 smithi188 ceph-mon[36599]: pgmap v443: 129 pgs: 129 active+clean; 656 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-06-10T02:54:31.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:31.321+0000 7f16ca58d700 1 -- 172.21.15.52:0/2241024456 --> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f16c41089b0 con 0x7f16ac0a2cb0 2024-06-10T02:54:31.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:31.330+0000 7f16c17fa700 1 -- 172.21.15.52:0/2241024456 <== mgr.14150 v2:172.21.15.52:6800/3214763969 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+341017 (secure 0 0 0) 0x7f16c41089b0 con 0x7f16ac0a2cb0 2024-06-10T02:54:31.334 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-10T02:54:31.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:31.333+0000 7f16ca58d700 1 -- 172.21.15.52:0/2241024456 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f16ac0a2cb0 msgr2=0x7f16ac0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:54:31.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:31.333+0000 7f16ca58d700 1 --2- 172.21.15.52:0/2241024456 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f16ac0a2cb0 0x7f16ac0a5170 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f16b4009a10 tx=0x7f16b4009190 comp rx=0 tx=0).stop 2024-06-10T02:54:31.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:31.333+0000 7f16ca58d700 1 -- 172.21.15.52:0/2241024456 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f16c4107860 msgr2=0x7f16c419f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:54:31.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:31.333+0000 7f16ca58d700 1 --2- 172.21.15.52:0/2241024456 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f16c4107860 0x7f16c419f830 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f16bc00b970 tx=0x7f16bc00bc80 comp rx=0 tx=0).stop 2024-06-10T02:54:31.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:31.334+0000 7f16ca58d700 1 -- 172.21.15.52:0/2241024456 shutdown_connections 2024-06-10T02:54:31.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:31.334+0000 7f16ca58d700 1 --2- 172.21.15.52:0/2241024456 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f16ac0a2cb0 0x7f16ac0a5170 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:31.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:31.334+0000 7f16ca58d700 1 --2- 172.21.15.52:0/2241024456 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f16c4107860 0x7f16c419f830 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:31.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:31.334+0000 7f16ca58d700 1 --2- 172.21.15.52:0/2241024456 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f16c4102770 0x7f16c419afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:31.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:31.334+0000 7f16ca58d700 1 --2- 172.21.15.52:0/2241024456 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f16c4103120 0x7f16c419b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:31.338 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:31.334+0000 7f16ca58d700 1 -- 172.21.15.52:0/2241024456 >> 172.21.15.52:0/2241024456 conn(0x7f16c4079600 msgr2=0x7f16c4101d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:54:31.338 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:31.334+0000 7f16ca58d700 1 -- 172.21.15.52:0/2241024456 shutdown_connections 2024-06-10T02:54:31.338 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:31.334+0000 7f16ca58d700 1 -- 172.21.15.52:0/2241024456 wait complete. 2024-06-10T02:54:31.338 INFO:teuthology.orchestra.run.smithi052.stderr:dumped all 2024-06-10T02:54:32.099 INFO:teuthology.orchestra.run.smithi052.stdout:{"pg_ready":true,"pg_map":{"version":444,"stamp":"2024-06-10T02:54:31.060814+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":671341,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":2162,"num_read_kb":1218,"num_write":459,"num_write_kb":3113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1181318,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":876,"ondisk_log_size":876,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":379372,"kb_used_data":13580,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124562452,"statfs":{"total":1151940427776,"available":1151551950848,"internally_reserved":0,"allocated":13905920,"data_stored":8864789,"data_compressed":497715,"data_compressed_allocated":675840,"data_compressed_original":2850816,"omap_allocated":19141,"internal_metadata":374453563},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":112,"num_read_kb":57,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.015161"},"pg_stats":[{"pgid":"3.3f","version":"100'8","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:43.001975+0000","last_change":"2024-06-10T02:50:02.209146+0000","last_active":"2024-06-10T02:52:43.001975+0000","last_peered":"2024-06-10T02:52:43.001975+0000","last_clean":"2024-06-10T02:52:43.001975+0000","last_became_active":"2024-06-10T02:50:02.208027+0000","last_became_peered":"2024-06-10T02:50:02.208027+0000","last_unstale":"2024-06-10T02:52:43.001975+0000","last_undegraded":"2024-06-10T02:52:43.001975+0000","last_fullsized":"2024-06-10T02:52:43.001975+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:35:34.765849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"100'4","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.995310+0000","last_change":"2024-06-10T02:50:02.198767+0000","last_active":"2024-06-10T02:52:42.995310+0000","last_peered":"2024-06-10T02:52:42.995310+0000","last_clean":"2024-06-10T02:52:42.995310+0000","last_became_active":"2024-06-10T02:50:02.198450+0000","last_became_peered":"2024-06-10T02:50:02.198450+0000","last_unstale":"2024-06-10T02:52:42.995310+0000","last_undegraded":"2024-06-10T02:52:42.995310+0000","last_fullsized":"2024-06-10T02:52:42.995310+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:49:56.666083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"100'7","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:43.000415+0000","last_change":"2024-06-10T02:50:02.208885+0000","last_active":"2024-06-10T02:52:43.000415+0000","last_peered":"2024-06-10T02:52:43.000415+0000","last_clean":"2024-06-10T02:52:43.000415+0000","last_became_active":"2024-06-10T02:50:02.206953+0000","last_became_peered":"2024-06-10T02:50:02.206953+0000","last_unstale":"2024-06-10T02:52:43.000415+0000","last_undegraded":"2024-06-10T02:52:43.000415+0000","last_fullsized":"2024-06-10T02:52:43.000415+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:41:23.928939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"100'8","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.994111+0000","last_change":"2024-06-10T02:50:02.208632+0000","last_active":"2024-06-10T02:52:42.994111+0000","last_peered":"2024-06-10T02:52:42.994111+0000","last_clean":"2024-06-10T02:52:42.994111+0000","last_became_active":"2024-06-10T02:50:02.207587+0000","last_became_peered":"2024-06-10T02:50:02.207587+0000","last_unstale":"2024-06-10T02:52:42.994111+0000","last_undegraded":"2024-06-10T02:52:42.994111+0000","last_fullsized":"2024-06-10T02:52:42.994111+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T14:34:36.720170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"100'4","reported_seq":43,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.973606+0000","last_change":"2024-06-10T02:50:02.197966+0000","last_active":"2024-06-10T02:52:42.973606+0000","last_peered":"2024-06-10T02:52:42.973606+0000","last_clean":"2024-06-10T02:52:42.973606+0000","last_became_active":"2024-06-10T02:50:02.197612+0000","last_became_peered":"2024-06-10T02:50:02.197612+0000","last_unstale":"2024-06-10T02:52:42.973606+0000","last_undegraded":"2024-06-10T02:52:42.973606+0000","last_fullsized":"2024-06-10T02:52:42.973606+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:24:53.804613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"100'6","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.987354+0000","last_change":"2024-06-10T02:50:02.201952+0000","last_active":"2024-06-10T02:52:42.987354+0000","last_peered":"2024-06-10T02:52:42.987354+0000","last_clean":"2024-06-10T02:52:42.987354+0000","last_became_active":"2024-06-10T02:50:02.200828+0000","last_became_peered":"2024-06-10T02:50:02.200828+0000","last_unstale":"2024-06-10T02:52:42.987354+0000","last_undegraded":"2024-06-10T02:52:42.987354+0000","last_fullsized":"2024-06-10T02:52:42.987354+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:46:28.518846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"100'5","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.990208+0000","last_change":"2024-06-10T02:50:02.203483+0000","last_active":"2024-06-10T02:52:42.990208+0000","last_peered":"2024-06-10T02:52:42.990208+0000","last_clean":"2024-06-10T02:52:42.990208+0000","last_became_active":"2024-06-10T02:50:02.203153+0000","last_became_peered":"2024-06-10T02:50:02.203153+0000","last_unstale":"2024-06-10T02:52:42.990208+0000","last_undegraded":"2024-06-10T02:52:42.990208+0000","last_fullsized":"2024-06-10T02:52:42.990208+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:29:30.422449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"100'9","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.995630+0000","last_change":"2024-06-10T02:50:02.201686+0000","last_active":"2024-06-10T02:52:42.995630+0000","last_peered":"2024-06-10T02:52:42.995630+0000","last_clean":"2024-06-10T02:52:42.995630+0000","last_became_active":"2024-06-10T02:50:02.200973+0000","last_became_peered":"2024-06-10T02:50:02.200973+0000","last_unstale":"2024-06-10T02:52:42.995630+0000","last_undegraded":"2024-06-10T02:52:42.995630+0000","last_fullsized":"2024-06-10T02:52:42.995630+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:41:13.712675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"100'11","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:43.001422+0000","last_change":"2024-06-10T02:50:02.210497+0000","last_active":"2024-06-10T02:52:43.001422+0000","last_peered":"2024-06-10T02:52:43.001422+0000","last_clean":"2024-06-10T02:52:43.001422+0000","last_became_active":"2024-06-10T02:50:02.209675+0000","last_became_peered":"2024-06-10T02:50:02.209675+0000","last_unstale":"2024-06-10T02:52:43.001422+0000","last_undegraded":"2024-06-10T02:52:43.001422+0000","last_fullsized":"2024-06-10T02:52:43.001422+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:50:29.879669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"100'16","reported_seq":61,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.994975+0000","last_change":"2024-06-10T02:50:44.741139+0000","last_active":"2024-06-10T02:52:42.994975+0000","last_peered":"2024-06-10T02:52:42.994975+0000","last_clean":"2024-06-10T02:52:42.994975+0000","last_became_active":"2024-06-10T02:50:44.740685+0000","last_became_peered":"2024-06-10T02:50:44.740685+0000","last_unstale":"2024-06-10T02:52:42.994975+0000","last_undegraded":"2024-06-10T02:52:42.994975+0000","last_fullsized":"2024-06-10T02:52:42.994975+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:58:59.872732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"100'6","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.999574+0000","last_change":"2024-06-10T02:50:02.206993+0000","last_active":"2024-06-10T02:52:42.999574+0000","last_peered":"2024-06-10T02:52:42.999574+0000","last_clean":"2024-06-10T02:52:42.999574+0000","last_became_active":"2024-06-10T02:50:02.206650+0000","last_became_peered":"2024-06-10T02:50:02.206650+0000","last_unstale":"2024-06-10T02:52:42.999574+0000","last_undegraded":"2024-06-10T02:52:42.999574+0000","last_fullsized":"2024-06-10T02:52:42.999574+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T14:27:49.555350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"100'7","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.979413+0000","last_change":"2024-06-10T02:50:02.200816+0000","last_active":"2024-06-10T02:52:42.979413+0000","last_peered":"2024-06-10T02:52:42.979413+0000","last_clean":"2024-06-10T02:52:42.979413+0000","last_became_active":"2024-06-10T02:50:02.200503+0000","last_became_peered":"2024-06-10T02:50:02.200503+0000","last_unstale":"2024-06-10T02:52:42.979413+0000","last_undegraded":"2024-06-10T02:52:42.979413+0000","last_fullsized":"2024-06-10T02:52:42.979413+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:01:08.326728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"100'10","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.987233+0000","last_change":"2024-06-10T02:50:02.203644+0000","last_active":"2024-06-10T02:52:42.987233+0000","last_peered":"2024-06-10T02:52:42.987233+0000","last_clean":"2024-06-10T02:52:42.987233+0000","last_became_active":"2024-06-10T02:50:02.203100+0000","last_became_peered":"2024-06-10T02:50:02.203100+0000","last_unstale":"2024-06-10T02:52:42.987233+0000","last_undegraded":"2024-06-10T02:52:42.987233+0000","last_fullsized":"2024-06-10T02:52:42.987233+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:01:42.730089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"100'7","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.995068+0000","last_change":"2024-06-10T02:50:02.203330+0000","last_active":"2024-06-10T02:52:42.995068+0000","last_peered":"2024-06-10T02:52:42.995068+0000","last_clean":"2024-06-10T02:52:42.995068+0000","last_became_active":"2024-06-10T02:50:02.202995+0000","last_became_peered":"2024-06-10T02:50:02.202995+0000","last_unstale":"2024-06-10T02:52:42.995068+0000","last_undegraded":"2024-06-10T02:52:42.995068+0000","last_fullsized":"2024-06-10T02:52:42.995068+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:17:23.170591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"100'7","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.986439+0000","last_change":"2024-06-10T02:50:02.204875+0000","last_active":"2024-06-10T02:52:42.986439+0000","last_peered":"2024-06-10T02:52:42.986439+0000","last_clean":"2024-06-10T02:52:42.986439+0000","last_became_active":"2024-06-10T02:50:02.200706+0000","last_became_peered":"2024-06-10T02:50:02.200706+0000","last_unstale":"2024-06-10T02:52:42.986439+0000","last_undegraded":"2024-06-10T02:52:42.986439+0000","last_fullsized":"2024-06-10T02:52:42.986439+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:27:12.282772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"100'9","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.999375+0000","last_change":"2024-06-10T02:50:02.203187+0000","last_active":"2024-06-10T02:52:42.999375+0000","last_peered":"2024-06-10T02:52:42.999375+0000","last_clean":"2024-06-10T02:52:42.999375+0000","last_became_active":"2024-06-10T02:50:02.202034+0000","last_became_peered":"2024-06-10T02:50:02.202034+0000","last_unstale":"2024-06-10T02:52:42.999375+0000","last_undegraded":"2024-06-10T02:52:42.999375+0000","last_fullsized":"2024-06-10T02:52:42.999375+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:59:44.783084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"100'8","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:43.002372+0000","last_change":"2024-06-10T02:50:02.209102+0000","last_active":"2024-06-10T02:52:43.002372+0000","last_peered":"2024-06-10T02:52:43.002372+0000","last_clean":"2024-06-10T02:52:43.002372+0000","last_became_active":"2024-06-10T02:50:02.208730+0000","last_became_peered":"2024-06-10T02:50:02.208730+0000","last_unstale":"2024-06-10T02:52:43.002372+0000","last_undegraded":"2024-06-10T02:52:43.002372+0000","last_fullsized":"2024-06-10T02:52:43.002372+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:08:38.838415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"100'10","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:43.000459+0000","last_change":"2024-06-10T02:50:02.204008+0000","last_active":"2024-06-10T02:52:43.000459+0000","last_peered":"2024-06-10T02:52:43.000459+0000","last_clean":"2024-06-10T02:52:43.000459+0000","last_became_active":"2024-06-10T02:50:02.203655+0000","last_became_peered":"2024-06-10T02:50:02.203655+0000","last_unstale":"2024-06-10T02:52:43.000459+0000","last_undegraded":"2024-06-10T02:52:43.000459+0000","last_fullsized":"2024-06-10T02:52:43.000459+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:32:15.972662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"100'7","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.991723+0000","last_change":"2024-06-10T02:50:02.204917+0000","last_active":"2024-06-10T02:52:42.991723+0000","last_peered":"2024-06-10T02:52:42.991723+0000","last_clean":"2024-06-10T02:52:42.991723+0000","last_became_active":"2024-06-10T02:50:02.204568+0000","last_became_peered":"2024-06-10T02:50:02.204568+0000","last_unstale":"2024-06-10T02:52:42.991723+0000","last_undegraded":"2024-06-10T02:52:42.991723+0000","last_fullsized":"2024-06-10T02:52:42.991723+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:49:46.802665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.721297+0000","last_change":"2024-06-10T02:50:12.240697+0000","last_active":"2024-06-10T02:51:45.721297+0000","last_peered":"2024-06-10T02:51:45.721297+0000","last_clean":"2024-06-10T02:51:45.721297+0000","last_became_active":"2024-06-10T02:50:12.240229+0000","last_became_peered":"2024-06-10T02:50:12.240229+0000","last_unstale":"2024-06-10T02:51:45.721297+0000","last_undegraded":"2024-06-10T02:51:45.721297+0000","last_fullsized":"2024-06-10T02:51:45.721297+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:45:55.071960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"100'15","reported_seq":276,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:54:28.919524+0000","last_change":"2024-06-10T02:50:44.786783+0000","last_active":"2024-06-10T02:54:28.919524+0000","last_peered":"2024-06-10T02:54:28.919524+0000","last_clean":"2024-06-10T02:54:28.919524+0000","last_became_active":"2024-06-10T02:50:44.781773+0000","last_became_peered":"2024-06-10T02:50:44.781773+0000","last_unstale":"2024-06-10T02:54:28.919524+0000","last_undegraded":"2024-06-10T02:54:28.919524+0000","last_fullsized":"2024-06-10T02:54:28.919524+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:31:06.025854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":203,"num_read_kb":203,"num_write":16,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,4],"acting":[9,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"100'5","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.994824+0000","last_change":"2024-06-10T02:50:02.202573+0000","last_active":"2024-06-10T02:52:42.994824+0000","last_peered":"2024-06-10T02:52:42.994824+0000","last_clean":"2024-06-10T02:52:42.994824+0000","last_became_active":"2024-06-10T02:50:02.202241+0000","last_became_peered":"2024-06-10T02:50:02.202241+0000","last_unstale":"2024-06-10T02:52:42.994824+0000","last_undegraded":"2024-06-10T02:52:42.994824+0000","last_fullsized":"2024-06-10T02:52:42.994824+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:31:40.230851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.383047+0000","last_change":"2024-06-10T02:50:12.238372+0000","last_active":"2024-06-10T02:51:46.383047+0000","last_peered":"2024-06-10T02:51:46.383047+0000","last_clean":"2024-06-10T02:51:46.383047+0000","last_became_active":"2024-06-10T02:50:12.237884+0000","last_became_peered":"2024-06-10T02:50:12.237884+0000","last_unstale":"2024-06-10T02:51:46.383047+0000","last_undegraded":"2024-06-10T02:51:46.383047+0000","last_fullsized":"2024-06-10T02:51:46.383047+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:55:31.432655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"98'2","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:02.117661+0000","last_change":"2024-06-10T02:49:59.031528+0000","last_active":"2024-06-10T02:52:02.117661+0000","last_peered":"2024-06-10T02:52:02.117661+0000","last_clean":"2024-06-10T02:52:02.117661+0000","last_became_active":"2024-06-10T02:49:59.031147+0000","last_became_peered":"2024-06-10T02:49:59.031147+0000","last_unstale":"2024-06-10T02:52:02.117661+0000","last_undegraded":"2024-06-10T02:52:02.117661+0000","last_fullsized":"2024-06-10T02:52:02.117661+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:17:08.534396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"100'6","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:43.000839+0000","last_change":"2024-06-10T02:50:02.202858+0000","last_active":"2024-06-10T02:52:43.000839+0000","last_peered":"2024-06-10T02:52:43.000839+0000","last_clean":"2024-06-10T02:52:43.000839+0000","last_became_active":"2024-06-10T02:50:02.202263+0000","last_became_peered":"2024-06-10T02:50:02.202263+0000","last_unstale":"2024-06-10T02:52:43.000839+0000","last_undegraded":"2024-06-10T02:52:43.000839+0000","last_fullsized":"2024-06-10T02:52:43.000839+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:23:15.382762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.383227+0000","last_change":"2024-06-10T02:50:12.237370+0000","last_active":"2024-06-10T02:51:46.383227+0000","last_peered":"2024-06-10T02:51:46.383227+0000","last_clean":"2024-06-10T02:51:46.383227+0000","last_became_active":"2024-06-10T02:50:12.236803+0000","last_became_peered":"2024-06-10T02:50:12.236803+0000","last_unstale":"2024-06-10T02:51:46.383227+0000","last_undegraded":"2024-06-10T02:51:46.383227+0000","last_fullsized":"2024-06-10T02:51:46.383227+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:06:40.695710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"98'12","reported_seq":58,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.724307+0000","last_change":"2024-06-10T02:49:59.000911+0000","last_active":"2024-06-10T02:51:45.724307+0000","last_peered":"2024-06-10T02:51:45.724307+0000","last_clean":"2024-06-10T02:51:45.724307+0000","last_became_active":"2024-06-10T02:49:59.000571+0000","last_became_peered":"2024-06-10T02:49:59.000571+0000","last_unstale":"2024-06-10T02:51:45.724307+0000","last_undegraded":"2024-06-10T02:51:45.724307+0000","last_fullsized":"2024-06-10T02:51:45.724307+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:49:24.635793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"100'9","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.997165+0000","last_change":"2024-06-10T02:50:02.209614+0000","last_active":"2024-06-10T02:52:42.997165+0000","last_peered":"2024-06-10T02:52:42.997165+0000","last_clean":"2024-06-10T02:52:42.997165+0000","last_became_active":"2024-06-10T02:50:02.209251+0000","last_became_peered":"2024-06-10T02:50:02.209251+0000","last_unstale":"2024-06-10T02:52:42.997165+0000","last_undegraded":"2024-06-10T02:52:42.997165+0000","last_fullsized":"2024-06-10T02:52:42.997165+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:35:32.256081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.383229+0000","last_change":"2024-06-10T02:50:12.237811+0000","last_active":"2024-06-10T02:51:46.383229+0000","last_peered":"2024-06-10T02:51:46.383229+0000","last_clean":"2024-06-10T02:51:46.383229+0000","last_became_active":"2024-06-10T02:50:12.237017+0000","last_became_peered":"2024-06-10T02:50:12.237017+0000","last_unstale":"2024-06-10T02:51:46.383229+0000","last_undegraded":"2024-06-10T02:51:46.383229+0000","last_fullsized":"2024-06-10T02:51:46.383229+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:14:29.818885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"98'5","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.723967+0000","last_change":"2024-06-10T02:49:58.996971+0000","last_active":"2024-06-10T02:51:45.723967+0000","last_peered":"2024-06-10T02:51:45.723967+0000","last_clean":"2024-06-10T02:51:45.723967+0000","last_became_active":"2024-06-10T02:49:58.996601+0000","last_became_peered":"2024-06-10T02:49:58.996601+0000","last_unstale":"2024-06-10T02:51:45.723967+0000","last_undegraded":"2024-06-10T02:51:45.723967+0000","last_fullsized":"2024-06-10T02:51:45.723967+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:26:45.963351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"100'15","reported_seq":60,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:43.002596+0000","last_change":"2024-06-10T02:50:44.745816+0000","last_active":"2024-06-10T02:52:43.002596+0000","last_peered":"2024-06-10T02:52:43.002596+0000","last_clean":"2024-06-10T02:52:43.002596+0000","last_became_active":"2024-06-10T02:50:44.745386+0000","last_became_peered":"2024-06-10T02:50:44.745386+0000","last_unstale":"2024-06-10T02:52:43.002596+0000","last_undegraded":"2024-06-10T02:52:43.002596+0000","last_fullsized":"2024-06-10T02:52:43.002596+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:08:53.977133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.721700+0000","last_change":"2024-06-10T02:50:12.236835+0000","last_active":"2024-06-10T02:51:45.721700+0000","last_peered":"2024-06-10T02:51:45.721700+0000","last_clean":"2024-06-10T02:51:45.721700+0000","last_became_active":"2024-06-10T02:50:12.236309+0000","last_became_peered":"2024-06-10T02:50:12.236309+0000","last_unstale":"2024-06-10T02:51:45.721700+0000","last_undegraded":"2024-06-10T02:51:45.721700+0000","last_fullsized":"2024-06-10T02:51:45.721700+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:09:02.789286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"100'8","reported_seq":255,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:54:24.914850+0000","last_change":"2024-06-10T02:49:58.997725+0000","last_active":"2024-06-10T02:54:24.914850+0000","last_peered":"2024-06-10T02:54:24.914850+0000","last_clean":"2024-06-10T02:54:24.914850+0000","last_became_active":"2024-06-10T02:49:58.997375+0000","last_became_peered":"2024-06-10T02:49:58.997375+0000","last_unstale":"2024-06-10T02:54:24.914850+0000","last_undegraded":"2024-06-10T02:54:24.914850+0000","last_fullsized":"2024-06-10T02:54:24.914850+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:28:49.714356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1461,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":202,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"100'7","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.995883+0000","last_change":"2024-06-10T02:50:02.201247+0000","last_active":"2024-06-10T02:52:42.995883+0000","last_peered":"2024-06-10T02:52:42.995883+0000","last_clean":"2024-06-10T02:52:42.995883+0000","last_became_active":"2024-06-10T02:50:02.200897+0000","last_became_peered":"2024-06-10T02:50:02.200897+0000","last_unstale":"2024-06-10T02:52:42.995883+0000","last_undegraded":"2024-06-10T02:52:42.995883+0000","last_fullsized":"2024-06-10T02:52:42.995883+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:01:58.166644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.721648+0000","last_change":"2024-06-10T02:50:12.243097+0000","last_active":"2024-06-10T02:51:45.721648+0000","last_peered":"2024-06-10T02:51:45.721648+0000","last_clean":"2024-06-10T02:51:45.721648+0000","last_became_active":"2024-06-10T02:50:12.242501+0000","last_became_peered":"2024-06-10T02:50:12.242501+0000","last_unstale":"2024-06-10T02:51:45.721648+0000","last_undegraded":"2024-06-10T02:51:45.721648+0000","last_fullsized":"2024-06-10T02:51:45.721648+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T14:42:44.220340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"98'7","reported_seq":52,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.720539+0000","last_change":"2024-06-10T02:49:58.996956+0000","last_active":"2024-06-10T02:51:45.720539+0000","last_peered":"2024-06-10T02:51:45.720539+0000","last_clean":"2024-06-10T02:51:45.720539+0000","last_became_active":"2024-06-10T02:49:58.996634+0000","last_became_peered":"2024-06-10T02:49:58.996634+0000","last_unstale":"2024-06-10T02:51:45.720539+0000","last_undegraded":"2024-06-10T02:51:45.720539+0000","last_fullsized":"2024-06-10T02:51:45.720539+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:25:02.364472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"100'8","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.995286+0000","last_change":"2024-06-10T02:50:02.205808+0000","last_active":"2024-06-10T02:52:42.995286+0000","last_peered":"2024-06-10T02:52:42.995286+0000","last_clean":"2024-06-10T02:52:42.995286+0000","last_became_active":"2024-06-10T02:50:02.203558+0000","last_became_peered":"2024-06-10T02:50:02.203558+0000","last_unstale":"2024-06-10T02:52:42.995286+0000","last_undegraded":"2024-06-10T02:52:42.995286+0000","last_fullsized":"2024-06-10T02:52:42.995286+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T13:43:23.315375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":30,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:17.877607+0000","last_change":"2024-06-10T02:50:12.242005+0000","last_active":"2024-06-10T02:52:17.877607+0000","last_peered":"2024-06-10T02:52:17.877607+0000","last_clean":"2024-06-10T02:52:17.877607+0000","last_became_active":"2024-06-10T02:50:12.241295+0000","last_became_peered":"2024-06-10T02:50:12.241295+0000","last_unstale":"2024-06-10T02:52:17.877607+0000","last_undegraded":"2024-06-10T02:52:17.877607+0000","last_fullsized":"2024-06-10T02:52:17.877607+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:32:52.702756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"98'3","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.714696+0000","last_change":"2024-06-10T02:49:58.995579+0000","last_active":"2024-06-10T02:51:45.714696+0000","last_peered":"2024-06-10T02:51:45.714696+0000","last_clean":"2024-06-10T02:51:45.714696+0000","last_became_active":"2024-06-10T02:49:58.995208+0000","last_became_peered":"2024-06-10T02:49:58.995208+0000","last_unstale":"2024-06-10T02:51:45.714696+0000","last_undegraded":"2024-06-10T02:51:45.714696+0000","last_fullsized":"2024-06-10T02:51:45.714696+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:26:03.513482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"100'8","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.993655+0000","last_change":"2024-06-10T02:50:02.197090+0000","last_active":"2024-06-10T02:52:42.993655+0000","last_peered":"2024-06-10T02:52:42.993655+0000","last_clean":"2024-06-10T02:52:42.993655+0000","last_became_active":"2024-06-10T02:50:02.196746+0000","last_became_peered":"2024-06-10T02:50:02.196746+0000","last_unstale":"2024-06-10T02:52:42.993655+0000","last_undegraded":"2024-06-10T02:52:42.993655+0000","last_fullsized":"2024-06-10T02:52:42.993655+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:52:09.987978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":30,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:14.133630+0000","last_change":"2024-06-10T02:50:12.234632+0000","last_active":"2024-06-10T02:52:14.133630+0000","last_peered":"2024-06-10T02:52:14.133630+0000","last_clean":"2024-06-10T02:52:14.133630+0000","last_became_active":"2024-06-10T02:50:12.234266+0000","last_became_peered":"2024-06-10T02:50:12.234266+0000","last_unstale":"2024-06-10T02:52:14.133630+0000","last_undegraded":"2024-06-10T02:52:14.133630+0000","last_fullsized":"2024-06-10T02:52:14.133630+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:21:27.348441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"98'3","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.383558+0000","last_change":"2024-06-10T02:49:58.999086+0000","last_active":"2024-06-10T02:51:46.383558+0000","last_peered":"2024-06-10T02:51:46.383558+0000","last_clean":"2024-06-10T02:51:46.383558+0000","last_became_active":"2024-06-10T02:49:58.998767+0000","last_became_peered":"2024-06-10T02:49:58.998767+0000","last_unstale":"2024-06-10T02:51:46.383558+0000","last_undegraded":"2024-06-10T02:51:46.383558+0000","last_fullsized":"2024-06-10T02:51:46.383558+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:28:12.922391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"100'11","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.990813+0000","last_change":"2024-06-10T02:50:02.196641+0000","last_active":"2024-06-10T02:52:42.990813+0000","last_peered":"2024-06-10T02:52:42.990813+0000","last_clean":"2024-06-10T02:52:42.990813+0000","last_became_active":"2024-06-10T02:50:02.196285+0000","last_became_peered":"2024-06-10T02:50:02.196285+0000","last_unstale":"2024-06-10T02:52:42.990813+0000","last_undegraded":"2024-06-10T02:52:42.990813+0000","last_fullsized":"2024-06-10T02:52:42.990813+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T13:55:31.284028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.831864+0000","last_change":"2024-06-10T02:50:12.240988+0000","last_active":"2024-06-10T02:51:45.831864+0000","last_peered":"2024-06-10T02:51:45.831864+0000","last_clean":"2024-06-10T02:51:45.831864+0000","last_became_active":"2024-06-10T02:50:12.240490+0000","last_became_peered":"2024-06-10T02:50:12.240490+0000","last_unstale":"2024-06-10T02:51:45.831864+0000","last_undegraded":"2024-06-10T02:51:45.831864+0000","last_fullsized":"2024-06-10T02:51:45.831864+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:54:36.015221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"100'7","reported_seq":446,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:54:25.915515+0000","last_change":"2024-06-10T02:49:58.989557+0000","last_active":"2024-06-10T02:54:25.915515+0000","last_peered":"2024-06-10T02:54:25.915515+0000","last_clean":"2024-06-10T02:54:25.915515+0000","last_became_active":"2024-06-10T02:49:58.989101+0000","last_became_peered":"2024-06-10T02:49:58.989101+0000","last_unstale":"2024-06-10T02:54:25.915515+0000","last_undegraded":"2024-06-10T02:54:25.915515+0000","last_fullsized":"2024-06-10T02:54:25.915515+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:27:42.182012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":394,"num_read_kb":394,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"100'10","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.998692+0000","last_change":"2024-06-10T02:50:02.208752+0000","last_active":"2024-06-10T02:52:42.998692+0000","last_peered":"2024-06-10T02:52:42.998692+0000","last_clean":"2024-06-10T02:52:42.998692+0000","last_became_active":"2024-06-10T02:50:02.208420+0000","last_became_peered":"2024-06-10T02:50:02.208420+0000","last_unstale":"2024-06-10T02:52:42.998692+0000","last_undegraded":"2024-06-10T02:52:42.998692+0000","last_fullsized":"2024-06-10T02:52:42.998692+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:28:09.412317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.382833+0000","last_change":"2024-06-10T02:50:12.236397+0000","last_active":"2024-06-10T02:51:46.382833+0000","last_peered":"2024-06-10T02:51:46.382833+0000","last_clean":"2024-06-10T02:51:46.382833+0000","last_became_active":"2024-06-10T02:50:12.235972+0000","last_became_peered":"2024-06-10T02:50:12.235972+0000","last_unstale":"2024-06-10T02:51:46.382833+0000","last_undegraded":"2024-06-10T02:51:46.382833+0000","last_fullsized":"2024-06-10T02:51:46.382833+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:38:41.723895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"98'4","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.295527+0000","last_change":"2024-06-10T02:49:58.996616+0000","last_active":"2024-06-10T02:51:46.295527+0000","last_peered":"2024-06-10T02:51:46.295527+0000","last_clean":"2024-06-10T02:51:46.295527+0000","last_became_active":"2024-06-10T02:49:58.996289+0000","last_became_peered":"2024-06-10T02:49:58.996289+0000","last_unstale":"2024-06-10T02:51:46.295527+0000","last_undegraded":"2024-06-10T02:51:46.295527+0000","last_fullsized":"2024-06-10T02:51:46.295527+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:22:12.993608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"100'9","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.983255+0000","last_change":"2024-06-10T02:50:02.205230+0000","last_active":"2024-06-10T02:52:42.983255+0000","last_peered":"2024-06-10T02:52:42.983255+0000","last_clean":"2024-06-10T02:52:42.983255+0000","last_became_active":"2024-06-10T02:50:02.204835+0000","last_became_peered":"2024-06-10T02:50:02.204835+0000","last_unstale":"2024-06-10T02:52:42.983255+0000","last_undegraded":"2024-06-10T02:52:42.983255+0000","last_fullsized":"2024-06-10T02:52:42.983255+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:17:35.084921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.383415+0000","last_change":"2024-06-10T02:50:12.231684+0000","last_active":"2024-06-10T02:51:46.383415+0000","last_peered":"2024-06-10T02:51:46.383415+0000","last_clean":"2024-06-10T02:51:46.383415+0000","last_became_active":"2024-06-10T02:50:12.231256+0000","last_became_peered":"2024-06-10T02:50:12.231256+0000","last_unstale":"2024-06-10T02:51:46.383415+0000","last_undegraded":"2024-06-10T02:51:46.383415+0000","last_fullsized":"2024-06-10T02:51:46.383415+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:17:31.878289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"98'5","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.383170+0000","last_change":"2024-06-10T02:49:59.033950+0000","last_active":"2024-06-10T02:51:46.383170+0000","last_peered":"2024-06-10T02:51:46.383170+0000","last_clean":"2024-06-10T02:51:46.383170+0000","last_became_active":"2024-06-10T02:49:59.033688+0000","last_became_peered":"2024-06-10T02:49:59.033688+0000","last_unstale":"2024-06-10T02:51:46.383170+0000","last_undegraded":"2024-06-10T02:51:46.383170+0000","last_fullsized":"2024-06-10T02:51:46.383170+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:32:03.892091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"100'11","reported_seq":56,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.999378+0000","last_change":"2024-06-10T02:50:44.781378+0000","last_active":"2024-06-10T02:52:42.999378+0000","last_peered":"2024-06-10T02:52:42.999378+0000","last_clean":"2024-06-10T02:52:42.999378+0000","last_became_active":"2024-06-10T02:50:44.780955+0000","last_became_peered":"2024-06-10T02:50:44.780955+0000","last_unstale":"2024-06-10T02:52:42.999378+0000","last_undegraded":"2024-06-10T02:52:42.999378+0000","last_fullsized":"2024-06-10T02:52:42.999378+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:49:13.760381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.827482+0000","last_change":"2024-06-10T02:51:46.384748+0000","last_active":"2024-06-10T02:51:46.827482+0000","last_peered":"2024-06-10T02:51:46.827482+0000","last_clean":"2024-06-10T02:51:46.827482+0000","last_became_active":"2024-06-10T02:51:46.384402+0000","last_became_peered":"2024-06-10T02:51:46.384402+0000","last_unstale":"2024-06-10T02:51:46.827482+0000","last_undegraded":"2024-06-10T02:51:46.827482+0000","last_fullsized":"2024-06-10T02:51:46.827482+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:06:48.056342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.14","version":"100'41","reported_seq":494,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:54:25.937884+0000","last_change":"2024-06-10T02:49:58.997890+0000","last_active":"2024-06-10T02:54:25.937884+0000","last_peered":"2024-06-10T02:54:25.937884+0000","last_clean":"2024-06-10T02:54:25.937884+0000","last_became_active":"2024-06-10T02:49:58.997549+0000","last_became_peered":"2024-06-10T02:49:58.997549+0000","last_unstale":"2024-06-10T02:54:25.937884+0000","last_undegraded":"2024-06-10T02:54:25.937884+0000","last_fullsized":"2024-06-10T02:54:25.937884+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T14:40:50.907029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71495,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":408,"num_read_kb":275,"num_write":43,"num_write_kb":90,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"100'6","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.991053+0000","last_change":"2024-06-10T02:50:02.199707+0000","last_active":"2024-06-10T02:52:42.991053+0000","last_peered":"2024-06-10T02:52:42.991053+0000","last_clean":"2024-06-10T02:52:42.991053+0000","last_became_active":"2024-06-10T02:50:02.198987+0000","last_became_peered":"2024-06-10T02:50:02.198987+0000","last_unstale":"2024-06-10T02:52:42.991053+0000","last_undegraded":"2024-06-10T02:52:42.991053+0000","last_fullsized":"2024-06-10T02:52:42.991053+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:48:03.704590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.383847+0000","last_change":"2024-06-10T02:50:12.239608+0000","last_active":"2024-06-10T02:51:46.383847+0000","last_peered":"2024-06-10T02:51:46.383847+0000","last_clean":"2024-06-10T02:51:46.383847+0000","last_became_active":"2024-06-10T02:50:12.238588+0000","last_became_peered":"2024-06-10T02:50:12.238588+0000","last_unstale":"2024-06-10T02:51:46.383847+0000","last_undegraded":"2024-06-10T02:51:46.383847+0000","last_fullsized":"2024-06-10T02:51:46.383847+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:26:12.718921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"98'9","reported_seq":56,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.721583+0000","last_change":"2024-06-10T02:49:58.997193+0000","last_active":"2024-06-10T02:51:45.721583+0000","last_peered":"2024-06-10T02:51:45.721583+0000","last_clean":"2024-06-10T02:51:45.721583+0000","last_became_active":"2024-06-10T02:49:58.996712+0000","last_became_peered":"2024-06-10T02:49:58.996712+0000","last_unstale":"2024-06-10T02:51:45.721583+0000","last_undegraded":"2024-06-10T02:51:45.721583+0000","last_fullsized":"2024-06-10T02:51:45.721583+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:02:46.894489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"100'9","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.994408+0000","last_change":"2024-06-10T02:50:02.200544+0000","last_active":"2024-06-10T02:52:42.994408+0000","last_peered":"2024-06-10T02:52:42.994408+0000","last_clean":"2024-06-10T02:52:42.994408+0000","last_became_active":"2024-06-10T02:50:02.200201+0000","last_became_peered":"2024-06-10T02:50:02.200201+0000","last_unstale":"2024-06-10T02:52:42.994408+0000","last_undegraded":"2024-06-10T02:52:42.994408+0000","last_fullsized":"2024-06-10T02:52:42.994408+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T13:12:40.925448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.365243+0000","last_change":"2024-06-10T02:50:12.236317+0000","last_active":"2024-06-10T02:51:46.365243+0000","last_peered":"2024-06-10T02:51:46.365243+0000","last_clean":"2024-06-10T02:51:46.365243+0000","last_became_active":"2024-06-10T02:50:12.235806+0000","last_became_peered":"2024-06-10T02:50:12.235806+0000","last_unstale":"2024-06-10T02:51:46.365243+0000","last_undegraded":"2024-06-10T02:51:46.365243+0000","last_fullsized":"2024-06-10T02:51:46.365243+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:29:21.425987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"98'5","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.383944+0000","last_change":"2024-06-10T02:49:58.994149+0000","last_active":"2024-06-10T02:51:46.383944+0000","last_peered":"2024-06-10T02:51:46.383944+0000","last_clean":"2024-06-10T02:51:46.383944+0000","last_became_active":"2024-06-10T02:49:58.993775+0000","last_became_peered":"2024-06-10T02:49:58.993775+0000","last_unstale":"2024-06-10T02:51:46.383944+0000","last_undegraded":"2024-06-10T02:51:46.383944+0000","last_fullsized":"2024-06-10T02:51:46.383944+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:23:55.553943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"100'15","reported_seq":54,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.995273+0000","last_change":"2024-06-10T02:50:02.199979+0000","last_active":"2024-06-10T02:52:42.995273+0000","last_peered":"2024-06-10T02:52:42.995273+0000","last_clean":"2024-06-10T02:52:42.995273+0000","last_became_active":"2024-06-10T02:50:02.199596+0000","last_became_peered":"2024-06-10T02:50:02.199596+0000","last_unstale":"2024-06-10T02:52:42.995273+0000","last_undegraded":"2024-06-10T02:52:42.995273+0000","last_fullsized":"2024-06-10T02:52:42.995273+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:13:58.647460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.383103+0000","last_change":"2024-06-10T02:50:12.235253+0000","last_active":"2024-06-10T02:51:46.383103+0000","last_peered":"2024-06-10T02:51:46.383103+0000","last_clean":"2024-06-10T02:51:46.383103+0000","last_became_active":"2024-06-10T02:50:12.234773+0000","last_became_peered":"2024-06-10T02:50:12.234773+0000","last_unstale":"2024-06-10T02:51:46.383103+0000","last_undegraded":"2024-06-10T02:51:46.383103+0000","last_fullsized":"2024-06-10T02:51:46.383103+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:38:10.971411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"100'7","reported_seq":258,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:54:29.921712+0000","last_change":"2024-06-10T02:49:58.996912+0000","last_active":"2024-06-10T02:54:29.921712+0000","last_peered":"2024-06-10T02:54:29.921712+0000","last_clean":"2024-06-10T02:54:29.921712+0000","last_became_active":"2024-06-10T02:49:58.996549+0000","last_became_peered":"2024-06-10T02:49:58.996549+0000","last_unstale":"2024-06-10T02:54:29.921712+0000","last_undegraded":"2024-06-10T02:54:29.921712+0000","last_fullsized":"2024-06-10T02:54:29.921712+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:07:52.757676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1119,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":207,"num_read_kb":5,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"100'6","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.995546+0000","last_change":"2024-06-10T02:50:44.741186+0000","last_active":"2024-06-10T02:52:42.995546+0000","last_peered":"2024-06-10T02:52:42.995546+0000","last_clean":"2024-06-10T02:52:42.995546+0000","last_became_active":"2024-06-10T02:50:44.740782+0000","last_became_peered":"2024-06-10T02:50:44.740782+0000","last_unstale":"2024-06-10T02:52:42.995546+0000","last_undegraded":"2024-06-10T02:52:42.995546+0000","last_fullsized":"2024-06-10T02:52:42.995546+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:46:03.863243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.715228+0000","last_change":"2024-06-10T02:50:12.236505+0000","last_active":"2024-06-10T02:51:45.715228+0000","last_peered":"2024-06-10T02:51:45.715228+0000","last_clean":"2024-06-10T02:51:45.715228+0000","last_became_active":"2024-06-10T02:50:12.236112+0000","last_became_peered":"2024-06-10T02:50:12.236112+0000","last_unstale":"2024-06-10T02:51:45.715228+0000","last_undegraded":"2024-06-10T02:51:45.715228+0000","last_fullsized":"2024-06-10T02:51:45.715228+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:13:38.586736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"98'17","reported_seq":63,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.723449+0000","last_change":"2024-06-10T02:49:59.031965+0000","last_active":"2024-06-10T02:51:45.723449+0000","last_peered":"2024-06-10T02:51:45.723449+0000","last_clean":"2024-06-10T02:51:45.723449+0000","last_became_active":"2024-06-10T02:49:59.031495+0000","last_became_peered":"2024-06-10T02:49:59.031495+0000","last_unstale":"2024-06-10T02:51:45.723449+0000","last_undegraded":"2024-06-10T02:51:45.723449+0000","last_fullsized":"2024-06-10T02:51:45.723449+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T14:21:06.610677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"100'6","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:43.002009+0000","last_change":"2024-06-10T02:50:02.209926+0000","last_active":"2024-06-10T02:52:43.002009+0000","last_peered":"2024-06-10T02:52:43.002009+0000","last_clean":"2024-06-10T02:52:43.002009+0000","last_became_active":"2024-06-10T02:50:02.209540+0000","last_became_peered":"2024-06-10T02:50:02.209540+0000","last_unstale":"2024-06-10T02:52:43.002009+0000","last_undegraded":"2024-06-10T02:52:43.002009+0000","last_fullsized":"2024-06-10T02:52:43.002009+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:49:40.497692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:48.927934+0000","last_change":"2024-06-10T02:51:46.013156+0000","last_active":"2024-06-10T02:51:48.927934+0000","last_peered":"2024-06-10T02:51:48.927934+0000","last_clean":"2024-06-10T02:51:48.927934+0000","last_became_active":"2024-06-10T02:51:46.012769+0000","last_became_peered":"2024-06-10T02:51:46.012769+0000","last_unstale":"2024-06-10T02:51:48.927934+0000","last_undegraded":"2024-06-10T02:51:48.927934+0000","last_fullsized":"2024-06-10T02:51:48.927934+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:44:49.545676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,3,1],"acting":[8,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.f","version":"98'2","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:13.930803+0000","last_change":"2024-06-10T02:49:58.998372+0000","last_active":"2024-06-10T02:52:13.930803+0000","last_peered":"2024-06-10T02:52:13.930803+0000","last_clean":"2024-06-10T02:52:13.930803+0000","last_became_active":"2024-06-10T02:49:58.998058+0000","last_became_peered":"2024-06-10T02:49:58.998058+0000","last_unstale":"2024-06-10T02:52:13.930803+0000","last_undegraded":"2024-06-10T02:52:13.930803+0000","last_fullsized":"2024-06-10T02:52:13.930803+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:42:50.509851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"100'5","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.993794+0000","last_change":"2024-06-10T02:50:02.209256+0000","last_active":"2024-06-10T02:52:42.993794+0000","last_peered":"2024-06-10T02:52:42.993794+0000","last_clean":"2024-06-10T02:52:42.993794+0000","last_became_active":"2024-06-10T02:50:02.207727+0000","last_became_peered":"2024-06-10T02:50:02.207727+0000","last_unstale":"2024-06-10T02:52:42.993794+0000","last_undegraded":"2024-06-10T02:52:42.993794+0000","last_fullsized":"2024-06-10T02:52:42.993794+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:23:00.175943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":30,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:18.931244+0000","last_change":"2024-06-10T02:50:12.729438+0000","last_active":"2024-06-10T02:52:18.931244+0000","last_peered":"2024-06-10T02:52:18.931244+0000","last_clean":"2024-06-10T02:52:18.931244+0000","last_became_active":"2024-06-10T02:50:12.728918+0000","last_became_peered":"2024-06-10T02:50:12.728918+0000","last_unstale":"2024-06-10T02:52:18.931244+0000","last_undegraded":"2024-06-10T02:52:18.931244+0000","last_fullsized":"2024-06-10T02:52:18.931244+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:04:48.861763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"98'1","reported_seq":53,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:05.628603+0000","last_change":"2024-06-10T02:50:44.782241+0000","last_active":"2024-06-10T02:52:05.628603+0000","last_peered":"2024-06-10T02:52:05.628603+0000","last_clean":"2024-06-10T02:52:05.628603+0000","last_became_active":"2024-06-10T02:50:44.781799+0000","last_became_peered":"2024-06-10T02:50:44.781799+0000","last_unstale":"2024-06-10T02:52:05.628603+0000","last_undegraded":"2024-06-10T02:52:05.628603+0000","last_fullsized":"2024-06-10T02:52:05.628603+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:38:35.726258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"100'4","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.993135+0000","last_change":"2024-06-10T02:50:02.198888+0000","last_active":"2024-06-10T02:52:42.993135+0000","last_peered":"2024-06-10T02:52:42.993135+0000","last_clean":"2024-06-10T02:52:42.993135+0000","last_became_active":"2024-06-10T02:50:02.198346+0000","last_became_peered":"2024-06-10T02:50:02.198346+0000","last_unstale":"2024-06-10T02:52:42.993135+0000","last_undegraded":"2024-06-10T02:52:42.993135+0000","last_fullsized":"2024-06-10T02:52:42.993135+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:15:42.422898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":30,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:17.689328+0000","last_change":"2024-06-10T02:50:12.232763+0000","last_active":"2024-06-10T02:52:17.689328+0000","last_peered":"2024-06-10T02:52:17.689328+0000","last_clean":"2024-06-10T02:52:17.689328+0000","last_became_active":"2024-06-10T02:50:12.232390+0000","last_became_peered":"2024-06-10T02:50:12.232390+0000","last_unstale":"2024-06-10T02:52:17.689328+0000","last_undegraded":"2024-06-10T02:52:17.689328+0000","last_fullsized":"2024-06-10T02:52:17.689328+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:31:47.973500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"100'8","reported_seq":256,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:54:25.915687+0000","last_change":"2024-06-10T02:49:58.994308+0000","last_active":"2024-06-10T02:54:25.915687+0000","last_peered":"2024-06-10T02:54:25.915687+0000","last_clean":"2024-06-10T02:54:25.915687+0000","last_became_active":"2024-06-10T02:49:58.993494+0000","last_became_peered":"2024-06-10T02:49:58.993494+0000","last_unstale":"2024-06-10T02:54:25.915687+0000","last_undegraded":"2024-06-10T02:54:25.915687+0000","last_fullsized":"2024-06-10T02:54:25.915687+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:07:28.452276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1509,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":202,"num_read_kb":4,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"100'7","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.988698+0000","last_change":"2024-06-10T02:50:02.206872+0000","last_active":"2024-06-10T02:52:42.988698+0000","last_peered":"2024-06-10T02:52:42.988698+0000","last_clean":"2024-06-10T02:52:42.988698+0000","last_became_active":"2024-06-10T02:50:02.206307+0000","last_became_peered":"2024-06-10T02:50:02.206307+0000","last_unstale":"2024-06-10T02:52:42.988698+0000","last_undegraded":"2024-06-10T02:52:42.988698+0000","last_fullsized":"2024-06-10T02:52:42.988698+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:01:04.645674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.715107+0000","last_change":"2024-06-10T02:50:12.729392+0000","last_active":"2024-06-10T02:51:45.715107+0000","last_peered":"2024-06-10T02:51:45.715107+0000","last_clean":"2024-06-10T02:51:45.715107+0000","last_became_active":"2024-06-10T02:50:12.728866+0000","last_became_peered":"2024-06-10T02:50:12.728866+0000","last_unstale":"2024-06-10T02:51:45.715107+0000","last_undegraded":"2024-06-10T02:51:45.715107+0000","last_fullsized":"2024-06-10T02:51:45.715107+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:01:35.845184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"100'8","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.995032+0000","last_change":"2024-06-10T02:50:02.202303+0000","last_active":"2024-06-10T02:52:42.995032+0000","last_peered":"2024-06-10T02:52:42.995032+0000","last_clean":"2024-06-10T02:52:42.995032+0000","last_became_active":"2024-06-10T02:50:02.201998+0000","last_became_peered":"2024-06-10T02:50:02.201998+0000","last_unstale":"2024-06-10T02:52:42.995032+0000","last_undegraded":"2024-06-10T02:52:42.995032+0000","last_fullsized":"2024-06-10T02:52:42.995032+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:15:00.005693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"100'6","reported_seq":251,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:54:25.915220+0000","last_change":"2024-06-10T02:49:59.031513+0000","last_active":"2024-06-10T02:54:25.915220+0000","last_peered":"2024-06-10T02:54:25.915220+0000","last_clean":"2024-06-10T02:54:25.915220+0000","last_became_active":"2024-06-10T02:49:59.030965+0000","last_became_peered":"2024-06-10T02:49:59.030965+0000","last_unstale":"2024-06-10T02:54:25.915220+0000","last_undegraded":"2024-06-10T02:54:25.915220+0000","last_fullsized":"2024-06-10T02:54:25.915220+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:49:02.171578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":199,"num_read_kb":199,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"98'1","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:17.800716+0000","last_change":"2024-06-10T02:50:12.236334+0000","last_active":"2024-06-10T02:52:17.800716+0000","last_peered":"2024-06-10T02:52:17.800716+0000","last_clean":"2024-06-10T02:52:17.800716+0000","last_became_active":"2024-06-10T02:50:12.235820+0000","last_became_peered":"2024-06-10T02:50:12.235820+0000","last_unstale":"2024-06-10T02:52:17.800716+0000","last_undegraded":"2024-06-10T02:52:17.800716+0000","last_fullsized":"2024-06-10T02:52:17.800716+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:16:12.476881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"100'8","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:43.001311+0000","last_change":"2024-06-10T02:50:02.205146+0000","last_active":"2024-06-10T02:52:43.001311+0000","last_peered":"2024-06-10T02:52:43.001311+0000","last_clean":"2024-06-10T02:52:43.001311+0000","last_became_active":"2024-06-10T02:50:02.204785+0000","last_became_peered":"2024-06-10T02:50:02.204785+0000","last_unstale":"2024-06-10T02:52:43.001311+0000","last_undegraded":"2024-06-10T02:52:43.001311+0000","last_fullsized":"2024-06-10T02:52:43.001311+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T13:01:02.068989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"98'6","reported_seq":55,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:06.016314+0000","last_change":"2024-06-10T02:49:58.997654+0000","last_active":"2024-06-10T02:52:06.016314+0000","last_peered":"2024-06-10T02:52:06.016314+0000","last_clean":"2024-06-10T02:52:06.016314+0000","last_became_active":"2024-06-10T02:49:58.997294+0000","last_became_peered":"2024-06-10T02:49:58.997294+0000","last_unstale":"2024-06-10T02:52:06.016314+0000","last_undegraded":"2024-06-10T02:52:06.016314+0000","last_fullsized":"2024-06-10T02:52:06.016314+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:53:14.763198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":367,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.294476+0000","last_change":"2024-06-10T02:48:34.008201+0000","last_active":"2024-06-10T02:51:46.294476+0000","last_peered":"2024-06-10T02:51:46.294476+0000","last_clean":"2024-06-10T02:51:46.294476+0000","last_became_active":"2024-06-10T02:48:34.007633+0000","last_became_peered":"2024-06-10T02:48:34.007633+0000","last_unstale":"2024-06-10T02:51:46.294476+0000","last_undegraded":"2024-06-10T02:51:46.294476+0000","last_fullsized":"2024-06-10T02:51:46.294476+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:45:07.444823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:45:07.444823+0000","last_clean_scrub_stamp":"2024-06-10T02:45:07.444823+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:56:58.159559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":36,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.383275+0000","last_change":"2024-06-10T02:50:44.782130+0000","last_active":"2024-06-10T02:51:46.383275+0000","last_peered":"2024-06-10T02:51:46.383275+0000","last_clean":"2024-06-10T02:51:46.383275+0000","last_became_active":"2024-06-10T02:50:44.781530+0000","last_became_peered":"2024-06-10T02:50:44.781530+0000","last_unstale":"2024-06-10T02:51:46.383275+0000","last_undegraded":"2024-06-10T02:51:46.383275+0000","last_fullsized":"2024-06-10T02:51:46.383275+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:43:23.826207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,11,6,10],"acting":[5,11,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"100'5","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.994100+0000","last_change":"2024-06-10T02:50:02.213666+0000","last_active":"2024-06-10T02:52:42.994100+0000","last_peered":"2024-06-10T02:52:42.994100+0000","last_clean":"2024-06-10T02:52:42.994100+0000","last_became_active":"2024-06-10T02:50:02.213194+0000","last_became_peered":"2024-06-10T02:50:02.213194+0000","last_unstale":"2024-06-10T02:52:42.994100+0000","last_undegraded":"2024-06-10T02:52:42.994100+0000","last_fullsized":"2024-06-10T02:52:42.994100+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:22:55.905412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"98'4","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:02.798809+0000","last_change":"2024-06-10T02:49:58.997618+0000","last_active":"2024-06-10T02:52:02.798809+0000","last_peered":"2024-06-10T02:52:02.798809+0000","last_clean":"2024-06-10T02:52:02.798809+0000","last_became_active":"2024-06-10T02:49:58.997304+0000","last_became_peered":"2024-06-10T02:49:58.997304+0000","last_unstale":"2024-06-10T02:52:02.798809+0000","last_undegraded":"2024-06-10T02:52:02.798809+0000","last_fullsized":"2024-06-10T02:52:02.798809+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:48:59.520991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":30,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:17.877545+0000","last_change":"2024-06-10T02:50:12.236357+0000","last_active":"2024-06-10T02:52:17.877545+0000","last_peered":"2024-06-10T02:52:17.877545+0000","last_clean":"2024-06-10T02:52:17.877545+0000","last_became_active":"2024-06-10T02:50:12.235915+0000","last_became_peered":"2024-06-10T02:50:12.235915+0000","last_unstale":"2024-06-10T02:52:17.877545+0000","last_undegraded":"2024-06-10T02:52:17.877545+0000","last_fullsized":"2024-06-10T02:52:17.877545+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:20:06.287907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"98'4","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:03.929566+0000","last_change":"2024-06-10T02:49:59.031725+0000","last_active":"2024-06-10T02:52:03.929566+0000","last_peered":"2024-06-10T02:52:03.929566+0000","last_clean":"2024-06-10T02:52:03.929566+0000","last_became_active":"2024-06-10T02:49:59.031296+0000","last_became_peered":"2024-06-10T02:49:59.031296+0000","last_unstale":"2024-06-10T02:52:03.929566+0000","last_undegraded":"2024-06-10T02:52:03.929566+0000","last_fullsized":"2024-06-10T02:52:03.929566+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T13:17:58.571696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"100'9","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.986516+0000","last_change":"2024-06-10T02:50:02.201820+0000","last_active":"2024-06-10T02:52:42.986516+0000","last_peered":"2024-06-10T02:52:42.986516+0000","last_clean":"2024-06-10T02:52:42.986516+0000","last_became_active":"2024-06-10T02:50:02.201478+0000","last_became_peered":"2024-06-10T02:50:02.201478+0000","last_unstale":"2024-06-10T02:52:42.986516+0000","last_undegraded":"2024-06-10T02:52:42.986516+0000","last_fullsized":"2024-06-10T02:52:42.986516+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:57:25.963926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":30,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:12.118458+0000","last_change":"2024-06-10T02:50:12.729418+0000","last_active":"2024-06-10T02:52:12.118458+0000","last_peered":"2024-06-10T02:52:12.118458+0000","last_clean":"2024-06-10T02:52:12.118458+0000","last_became_active":"2024-06-10T02:50:12.728692+0000","last_became_peered":"2024-06-10T02:50:12.728692+0000","last_unstale":"2024-06-10T02:52:12.118458+0000","last_undegraded":"2024-06-10T02:52:12.118458+0000","last_fullsized":"2024-06-10T02:52:12.118458+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:25:00.472396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"100'18","reported_seq":64,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:54:08.234903+0000","last_change":"2024-06-10T02:49:58.996391+0000","last_active":"2024-06-10T02:54:08.234903+0000","last_peered":"2024-06-10T02:54:08.234903+0000","last_clean":"2024-06-10T02:54:08.234903+0000","last_became_active":"2024-06-10T02:49:58.996033+0000","last_became_peered":"2024-06-10T02:49:58.996033+0000","last_unstale":"2024-06-10T02:54:08.234903+0000","last_undegraded":"2024-06-10T02:54:08.234903+0000","last_fullsized":"2024-06-10T02:54:08.234903+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:03:34.568969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":34,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"100'5","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.995465+0000","last_change":"2024-06-10T02:50:02.197974+0000","last_active":"2024-06-10T02:52:42.995465+0000","last_peered":"2024-06-10T02:52:42.995465+0000","last_clean":"2024-06-10T02:52:42.995465+0000","last_became_active":"2024-06-10T02:50:02.197614+0000","last_became_peered":"2024-06-10T02:50:02.197614+0000","last_unstale":"2024-06-10T02:52:42.995465+0000","last_undegraded":"2024-06-10T02:52:42.995465+0000","last_fullsized":"2024-06-10T02:52:42.995465+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:36:55.575808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":30,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:16.242935+0000","last_change":"2024-06-10T02:50:12.234239+0000","last_active":"2024-06-10T02:52:16.242935+0000","last_peered":"2024-06-10T02:52:16.242935+0000","last_clean":"2024-06-10T02:52:16.242935+0000","last_became_active":"2024-06-10T02:50:12.233768+0000","last_became_peered":"2024-06-10T02:50:12.233768+0000","last_unstale":"2024-06-10T02:52:16.242935+0000","last_undegraded":"2024-06-10T02:52:16.242935+0000","last_fullsized":"2024-06-10T02:52:16.242935+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:51:44.230031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"98'7","reported_seq":54,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:11.016968+0000","last_change":"2024-06-10T02:49:59.192765+0000","last_active":"2024-06-10T02:52:11.016968+0000","last_peered":"2024-06-10T02:52:11.016968+0000","last_clean":"2024-06-10T02:52:11.016968+0000","last_became_active":"2024-06-10T02:49:59.192446+0000","last_became_peered":"2024-06-10T02:49:59.192446+0000","last_unstale":"2024-06-10T02:52:11.016968+0000","last_undegraded":"2024-06-10T02:52:11.016968+0000","last_fullsized":"2024-06-10T02:52:11.016968+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:53:51.862019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"100'7","reported_seq":53,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.989869+0000","last_change":"2024-06-10T02:50:44.781209+0000","last_active":"2024-06-10T02:52:42.989869+0000","last_peered":"2024-06-10T02:52:42.989869+0000","last_clean":"2024-06-10T02:52:42.989869+0000","last_became_active":"2024-06-10T02:50:44.780811+0000","last_became_peered":"2024-06-10T02:50:44.780811+0000","last_unstale":"2024-06-10T02:52:42.989869+0000","last_undegraded":"2024-06-10T02:52:42.989869+0000","last_fullsized":"2024-06-10T02:52:42.989869+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:12:40.663565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.383517+0000","last_change":"2024-06-10T02:50:12.730269+0000","last_active":"2024-06-10T02:51:46.383517+0000","last_peered":"2024-06-10T02:51:46.383517+0000","last_clean":"2024-06-10T02:51:46.383517+0000","last_became_active":"2024-06-10T02:50:12.729574+0000","last_became_peered":"2024-06-10T02:50:12.729574+0000","last_unstale":"2024-06-10T02:51:46.383517+0000","last_undegraded":"2024-06-10T02:51:46.383517+0000","last_fullsized":"2024-06-10T02:51:46.383517+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:51:16.235284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"98'2","reported_seq":54,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:12.774945+0000","last_change":"2024-06-10T02:50:44.782421+0000","last_active":"2024-06-10T02:52:12.774945+0000","last_peered":"2024-06-10T02:52:12.774945+0000","last_clean":"2024-06-10T02:52:12.774945+0000","last_became_active":"2024-06-10T02:50:44.781881+0000","last_became_peered":"2024-06-10T02:50:44.781881+0000","last_unstale":"2024-06-10T02:52:12.774945+0000","last_undegraded":"2024-06-10T02:52:12.774945+0000","last_fullsized":"2024-06-10T02:52:12.774945+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T14:41:58.771256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"100'2","reported_seq":42,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.984615+0000","last_change":"2024-06-10T02:50:02.197761+0000","last_active":"2024-06-10T02:52:42.984615+0000","last_peered":"2024-06-10T02:52:42.984615+0000","last_clean":"2024-06-10T02:52:42.984615+0000","last_became_active":"2024-06-10T02:50:02.197395+0000","last_became_peered":"2024-06-10T02:50:02.197395+0000","last_unstale":"2024-06-10T02:52:42.984615+0000","last_undegraded":"2024-06-10T02:52:42.984615+0000","last_fullsized":"2024-06-10T02:52:42.984615+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:46:37.537280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:48.927972+0000","last_change":"2024-06-10T02:51:46.385129+0000","last_active":"2024-06-10T02:51:48.927972+0000","last_peered":"2024-06-10T02:51:48.927972+0000","last_clean":"2024-06-10T02:51:48.927972+0000","last_became_active":"2024-06-10T02:51:46.384553+0000","last_became_peered":"2024-06-10T02:51:46.384553+0000","last_unstale":"2024-06-10T02:51:48.927972+0000","last_undegraded":"2024-06-10T02:51:48.927972+0000","last_fullsized":"2024-06-10T02:51:48.927972+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:08:11.183679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"98'7","reported_seq":53,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:12.118484+0000","last_change":"2024-06-10T02:49:58.987863+0000","last_active":"2024-06-10T02:52:12.118484+0000","last_peered":"2024-06-10T02:52:12.118484+0000","last_clean":"2024-06-10T02:52:12.118484+0000","last_became_active":"2024-06-10T02:49:58.987388+0000","last_became_peered":"2024-06-10T02:49:58.987388+0000","last_unstale":"2024-06-10T02:52:12.118484+0000","last_undegraded":"2024-06-10T02:52:12.118484+0000","last_fullsized":"2024-06-10T02:52:12.118484+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:29:20.340301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"100'9","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.989752+0000","last_change":"2024-06-10T02:50:02.200699+0000","last_active":"2024-06-10T02:52:42.989752+0000","last_peered":"2024-06-10T02:52:42.989752+0000","last_clean":"2024-06-10T02:52:42.989752+0000","last_became_active":"2024-06-10T02:50:02.200278+0000","last_became_peered":"2024-06-10T02:50:02.200278+0000","last_unstale":"2024-06-10T02:52:42.989752+0000","last_undegraded":"2024-06-10T02:52:42.989752+0000","last_fullsized":"2024-06-10T02:52:42.989752+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:26:36.236312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"98'1","reported_seq":30,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.383543+0000","last_change":"2024-06-10T02:50:12.236027+0000","last_active":"2024-06-10T02:51:46.383543+0000","last_peered":"2024-06-10T02:51:46.383543+0000","last_clean":"2024-06-10T02:51:46.383543+0000","last_became_active":"2024-06-10T02:50:12.235629+0000","last_became_peered":"2024-06-10T02:50:12.235629+0000","last_unstale":"2024-06-10T02:51:46.383543+0000","last_undegraded":"2024-06-10T02:51:46.383543+0000","last_fullsized":"2024-06-10T02:51:46.383543+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:58:48.218806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"98'3","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:11.242480+0000","last_change":"2024-06-10T02:49:58.998496+0000","last_active":"2024-06-10T02:52:11.242480+0000","last_peered":"2024-06-10T02:52:11.242480+0000","last_clean":"2024-06-10T02:52:11.242480+0000","last_became_active":"2024-06-10T02:49:58.997260+0000","last_became_peered":"2024-06-10T02:49:58.997260+0000","last_unstale":"2024-06-10T02:52:11.242480+0000","last_undegraded":"2024-06-10T02:52:11.242480+0000","last_fullsized":"2024-06-10T02:52:11.242480+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T14:22:04.658943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"100'8","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.990375+0000","last_change":"2024-06-10T02:50:02.201300+0000","last_active":"2024-06-10T02:52:42.990375+0000","last_peered":"2024-06-10T02:52:42.990375+0000","last_clean":"2024-06-10T02:52:42.990375+0000","last_became_active":"2024-06-10T02:50:02.200988+0000","last_became_peered":"2024-06-10T02:50:02.200988+0000","last_unstale":"2024-06-10T02:52:42.990375+0000","last_undegraded":"2024-06-10T02:52:42.990375+0000","last_fullsized":"2024-06-10T02:52:42.990375+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:11:12.962048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.383484+0000","last_change":"2024-06-10T02:50:12.728869+0000","last_active":"2024-06-10T02:51:46.383484+0000","last_peered":"2024-06-10T02:51:46.383484+0000","last_clean":"2024-06-10T02:51:46.383484+0000","last_became_active":"2024-06-10T02:50:12.728189+0000","last_became_peered":"2024-06-10T02:50:12.728189+0000","last_unstale":"2024-06-10T02:51:46.383484+0000","last_undegraded":"2024-06-10T02:51:46.383484+0000","last_fullsized":"2024-06-10T02:51:46.383484+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:08:36.624814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"98'4","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:10.967286+0000","last_change":"2024-06-10T02:49:59.000460+0000","last_active":"2024-06-10T02:52:10.967286+0000","last_peered":"2024-06-10T02:52:10.967286+0000","last_clean":"2024-06-10T02:52:10.967286+0000","last_became_active":"2024-06-10T02:49:59.000075+0000","last_became_peered":"2024-06-10T02:49:59.000075+0000","last_unstale":"2024-06-10T02:52:10.967286+0000","last_undegraded":"2024-06-10T02:52:10.967286+0000","last_fullsized":"2024-06-10T02:52:10.967286+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:06:27.935745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"100'5","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:43.001259+0000","last_change":"2024-06-10T02:50:02.210131+0000","last_active":"2024-06-10T02:52:43.001259+0000","last_peered":"2024-06-10T02:52:43.001259+0000","last_clean":"2024-06-10T02:52:43.001259+0000","last_became_active":"2024-06-10T02:50:02.209781+0000","last_became_peered":"2024-06-10T02:50:02.209781+0000","last_unstale":"2024-06-10T02:52:43.001259+0000","last_undegraded":"2024-06-10T02:52:43.001259+0000","last_fullsized":"2024-06-10T02:52:43.001259+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:27:39.966620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.854407+0000","last_change":"2024-06-10T02:50:12.236017+0000","last_active":"2024-06-10T02:51:45.854407+0000","last_peered":"2024-06-10T02:51:45.854407+0000","last_clean":"2024-06-10T02:51:45.854407+0000","last_became_active":"2024-06-10T02:50:12.235553+0000","last_became_peered":"2024-06-10T02:50:12.235553+0000","last_unstale":"2024-06-10T02:51:45.854407+0000","last_undegraded":"2024-06-10T02:51:45.854407+0000","last_fullsized":"2024-06-10T02:51:45.854407+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:23:55.437994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"100'7","reported_seq":253,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:54:24.914730+0000","last_change":"2024-06-10T02:49:59.192864+0000","last_active":"2024-06-10T02:54:24.914730+0000","last_peered":"2024-06-10T02:54:24.914730+0000","last_clean":"2024-06-10T02:54:24.914730+0000","last_became_active":"2024-06-10T02:49:59.192440+0000","last_became_peered":"2024-06-10T02:49:59.192440+0000","last_unstale":"2024-06-10T02:54:24.914730+0000","last_undegraded":"2024-06-10T02:54:24.914730+0000","last_fullsized":"2024-06-10T02:54:24.914730+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:36:05.868538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1701,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":201,"num_read_kb":4,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"100'7","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:43.000525+0000","last_change":"2024-06-10T02:50:02.205791+0000","last_active":"2024-06-10T02:52:43.000525+0000","last_peered":"2024-06-10T02:52:43.000525+0000","last_clean":"2024-06-10T02:52:43.000525+0000","last_became_active":"2024-06-10T02:50:02.205483+0000","last_became_peered":"2024-06-10T02:50:02.205483+0000","last_unstale":"2024-06-10T02:52:43.000525+0000","last_undegraded":"2024-06-10T02:52:43.000525+0000","last_fullsized":"2024-06-10T02:52:43.000525+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:33:25.875423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.011597+0000","last_change":"2024-06-10T02:50:12.239294+0000","last_active":"2024-06-10T02:51:46.011597+0000","last_peered":"2024-06-10T02:51:46.011597+0000","last_clean":"2024-06-10T02:51:46.011597+0000","last_became_active":"2024-06-10T02:50:12.238778+0000","last_became_peered":"2024-06-10T02:50:12.238778+0000","last_unstale":"2024-06-10T02:51:46.011597+0000","last_undegraded":"2024-06-10T02:51:46.011597+0000","last_fullsized":"2024-06-10T02:51:46.011597+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:26:09.203623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"98'3","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.383589+0000","last_change":"2024-06-10T02:49:58.989680+0000","last_active":"2024-06-10T02:51:46.383589+0000","last_peered":"2024-06-10T02:51:46.383589+0000","last_clean":"2024-06-10T02:51:46.383589+0000","last_became_active":"2024-06-10T02:49:58.989265+0000","last_became_peered":"2024-06-10T02:49:58.989265+0000","last_unstale":"2024-06-10T02:51:46.383589+0000","last_undegraded":"2024-06-10T02:51:46.383589+0000","last_fullsized":"2024-06-10T02:51:46.383589+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:36:34.370177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"100'6","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.990330+0000","last_change":"2024-06-10T02:50:44.743369+0000","last_active":"2024-06-10T02:52:42.990330+0000","last_peered":"2024-06-10T02:52:42.990330+0000","last_clean":"2024-06-10T02:52:42.990330+0000","last_became_active":"2024-06-10T02:50:44.743013+0000","last_became_peered":"2024-06-10T02:50:44.743013+0000","last_unstale":"2024-06-10T02:52:42.990330+0000","last_undegraded":"2024-06-10T02:52:42.990330+0000","last_fullsized":"2024-06-10T02:52:42.990330+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:54:43.728943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:46.382656+0000","last_change":"2024-06-10T02:50:12.235197+0000","last_active":"2024-06-10T02:51:46.382656+0000","last_peered":"2024-06-10T02:51:46.382656+0000","last_clean":"2024-06-10T02:51:46.382656+0000","last_became_active":"2024-06-10T02:50:12.234791+0000","last_became_peered":"2024-06-10T02:50:12.234791+0000","last_unstale":"2024-06-10T02:51:46.382656+0000","last_undegraded":"2024-06-10T02:51:46.382656+0000","last_fullsized":"2024-06-10T02:51:46.382656+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:11.216025+0000","last_clean_scrub_stamp":"2024-06-10T02:50:11.216025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:09:11.164236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"98'5","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:51:45.715164+0000","last_change":"2024-06-10T02:49:58.996759+0000","last_active":"2024-06-10T02:51:45.715164+0000","last_peered":"2024-06-10T02:51:45.715164+0000","last_clean":"2024-06-10T02:51:45.715164+0000","last_became_active":"2024-06-10T02:49:58.996434+0000","last_became_peered":"2024-06-10T02:49:58.996434+0000","last_unstale":"2024-06-10T02:51:45.715164+0000","last_undegraded":"2024-06-10T02:51:45.715164+0000","last_fullsized":"2024-06-10T02:51:45.715164+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:49:57.977046+0000","last_clean_scrub_stamp":"2024-06-10T02:49:57.977046+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:09:35.327797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"100'9","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.996813+0000","last_change":"2024-06-10T02:50:02.201676+0000","last_active":"2024-06-10T02:52:42.996813+0000","last_peered":"2024-06-10T02:52:42.996813+0000","last_clean":"2024-06-10T02:52:42.996813+0000","last_became_active":"2024-06-10T02:50:02.201335+0000","last_became_peered":"2024-06-10T02:50:02.201335+0000","last_unstale":"2024-06-10T02:52:42.996813+0000","last_undegraded":"2024-06-10T02:52:42.996813+0000","last_fullsized":"2024-06-10T02:52:42.996813+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:49:47.038946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"100'9","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.992923+0000","last_change":"2024-06-10T02:50:02.203462+0000","last_active":"2024-06-10T02:52:42.992923+0000","last_peered":"2024-06-10T02:52:42.992923+0000","last_clean":"2024-06-10T02:52:42.992923+0000","last_became_active":"2024-06-10T02:50:02.203115+0000","last_became_peered":"2024-06-10T02:50:02.203115+0000","last_unstale":"2024-06-10T02:52:42.992923+0000","last_undegraded":"2024-06-10T02:52:42.992923+0000","last_fullsized":"2024-06-10T02:52:42.992923+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T13:18:12.400596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"100'12","reported_seq":52,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.998015+0000","last_change":"2024-06-10T02:50:02.205648+0000","last_active":"2024-06-10T02:52:42.998015+0000","last_peered":"2024-06-10T02:52:42.998015+0000","last_clean":"2024-06-10T02:52:42.998015+0000","last_became_active":"2024-06-10T02:50:02.205306+0000","last_became_peered":"2024-06-10T02:50:02.205306+0000","last_unstale":"2024-06-10T02:52:42.998015+0000","last_undegraded":"2024-06-10T02:52:42.998015+0000","last_fullsized":"2024-06-10T02:52:42.998015+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:22:55.020644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"100'8","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.991617+0000","last_change":"2024-06-10T02:50:02.198601+0000","last_active":"2024-06-10T02:52:42.991617+0000","last_peered":"2024-06-10T02:52:42.991617+0000","last_clean":"2024-06-10T02:52:42.991617+0000","last_became_active":"2024-06-10T02:50:02.198103+0000","last_became_peered":"2024-06-10T02:50:02.198103+0000","last_unstale":"2024-06-10T02:52:42.991617+0000","last_undegraded":"2024-06-10T02:52:42.991617+0000","last_fullsized":"2024-06-10T02:52:42.991617+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:56:28.033482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"100'7","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.985450+0000","last_change":"2024-06-10T02:50:02.202638+0000","last_active":"2024-06-10T02:52:42.985450+0000","last_peered":"2024-06-10T02:52:42.985450+0000","last_clean":"2024-06-10T02:52:42.985450+0000","last_became_active":"2024-06-10T02:50:02.202249+0000","last_became_peered":"2024-06-10T02:50:02.202249+0000","last_unstale":"2024-06-10T02:52:42.985450+0000","last_undegraded":"2024-06-10T02:52:42.985450+0000","last_fullsized":"2024-06-10T02:52:42.985450+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:46:29.424164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"100'4","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.992759+0000","last_change":"2024-06-10T02:50:02.213526+0000","last_active":"2024-06-10T02:52:42.992759+0000","last_peered":"2024-06-10T02:52:42.992759+0000","last_clean":"2024-06-10T02:52:42.992759+0000","last_became_active":"2024-06-10T02:50:02.213226+0000","last_became_peered":"2024-06-10T02:50:02.213226+0000","last_unstale":"2024-06-10T02:52:42.992759+0000","last_undegraded":"2024-06-10T02:52:42.992759+0000","last_fullsized":"2024-06-10T02:52:42.992759+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:01:56.846340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"100'6","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.993163+0000","last_change":"2024-06-10T02:50:02.206583+0000","last_active":"2024-06-10T02:52:42.993163+0000","last_peered":"2024-06-10T02:52:42.993163+0000","last_clean":"2024-06-10T02:52:42.993163+0000","last_became_active":"2024-06-10T02:50:02.206227+0000","last_became_peered":"2024-06-10T02:50:02.206227+0000","last_unstale":"2024-06-10T02:52:42.993163+0000","last_undegraded":"2024-06-10T02:52:42.993163+0000","last_fullsized":"2024-06-10T02:52:42.993163+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:25:07.246288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"100'15","reported_seq":55,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:43.003046+0000","last_change":"2024-06-10T02:50:02.206348+0000","last_active":"2024-06-10T02:52:43.003046+0000","last_peered":"2024-06-10T02:52:43.003046+0000","last_clean":"2024-06-10T02:52:43.003046+0000","last_became_active":"2024-06-10T02:50:02.206020+0000","last_became_peered":"2024-06-10T02:50:02.206020+0000","last_unstale":"2024-06-10T02:52:43.003046+0000","last_undegraded":"2024-06-10T02:52:43.003046+0000","last_fullsized":"2024-06-10T02:52:43.003046+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:02:26.619113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"100'2","reported_seq":42,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.993595+0000","last_change":"2024-06-10T02:50:02.199405+0000","last_active":"2024-06-10T02:52:42.993595+0000","last_peered":"2024-06-10T02:52:42.993595+0000","last_clean":"2024-06-10T02:52:42.993595+0000","last_became_active":"2024-06-10T02:50:02.199049+0000","last_became_peered":"2024-06-10T02:50:02.199049+0000","last_unstale":"2024-06-10T02:52:42.993595+0000","last_undegraded":"2024-06-10T02:52:42.993595+0000","last_fullsized":"2024-06-10T02:52:42.993595+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:33:04.091160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"100'6","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.992849+0000","last_change":"2024-06-10T02:50:02.202094+0000","last_active":"2024-06-10T02:52:42.992849+0000","last_peered":"2024-06-10T02:52:42.992849+0000","last_clean":"2024-06-10T02:52:42.992849+0000","last_became_active":"2024-06-10T02:50:02.201751+0000","last_became_peered":"2024-06-10T02:50:02.201751+0000","last_unstale":"2024-06-10T02:52:42.992849+0000","last_undegraded":"2024-06-10T02:52:42.992849+0000","last_fullsized":"2024-06-10T02:52:42.992849+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:52:53.946326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"100'8","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.993442+0000","last_change":"2024-06-10T02:50:02.199630+0000","last_active":"2024-06-10T02:52:42.993442+0000","last_peered":"2024-06-10T02:52:42.993442+0000","last_clean":"2024-06-10T02:52:42.993442+0000","last_became_active":"2024-06-10T02:50:02.199320+0000","last_became_peered":"2024-06-10T02:50:02.199320+0000","last_unstale":"2024-06-10T02:52:42.993442+0000","last_undegraded":"2024-06-10T02:52:42.993442+0000","last_fullsized":"2024-06-10T02:52:42.993442+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:17:28.761862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"100'10","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:43.000631+0000","last_change":"2024-06-10T02:50:02.201190+0000","last_active":"2024-06-10T02:52:43.000631+0000","last_peered":"2024-06-10T02:52:43.000631+0000","last_clean":"2024-06-10T02:52:43.000631+0000","last_became_active":"2024-06-10T02:50:02.200574+0000","last_became_peered":"2024-06-10T02:50:02.200574+0000","last_unstale":"2024-06-10T02:52:43.000631+0000","last_undegraded":"2024-06-10T02:52:43.000631+0000","last_fullsized":"2024-06-10T02:52:43.000631+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:02:07.636218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"100'11","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.992436+0000","last_change":"2024-06-10T02:50:02.202374+0000","last_active":"2024-06-10T02:52:42.992436+0000","last_peered":"2024-06-10T02:52:42.992436+0000","last_clean":"2024-06-10T02:52:42.992436+0000","last_became_active":"2024-06-10T02:50:02.201974+0000","last_became_peered":"2024-06-10T02:50:02.201974+0000","last_unstale":"2024-06-10T02:52:42.992436+0000","last_undegraded":"2024-06-10T02:52:42.992436+0000","last_fullsized":"2024-06-10T02:52:42.992436+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:45:34.977926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"100'8","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T02:52:42.995177+0000","last_change":"2024-06-10T02:50:02.209332+0000","last_active":"2024-06-10T02:52:42.995177+0000","last_peered":"2024-06-10T02:52:42.995177+0000","last_clean":"2024-06-10T02:52:42.995177+0000","last_became_active":"2024-06-10T02:50:02.208175+0000","last_became_peered":"2024-06-10T02:50:02.208175+0000","last_unstale":"2024-06-10T02:52:42.995177+0000","last_undegraded":"2024-06-10T02:52:42.995177+0000","last_fullsized":"2024-06-10T02:52:42.995177+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T02:50:01.180576+0000","last_clean_scrub_stamp":"2024-06-10T02:50:01.180576+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:44:40.857918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":284028,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":80739,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":2036,"num_read_kb":1109,"num_write":276,"num_write_kb":173,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":602112,"data_stored":242217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":237,"ondisk_log_size":237,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351046,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32260,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712892,"statfs":{"total":95995035648,"available":95962001408,"internally_reserved":0,"allocated":1110016,"data_stored":574044,"data_compressed":32447,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612683,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32416,"kb_used_data":1240,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712736,"statfs":{"total":95995035648,"available":95961841664,"internally_reserved":0,"allocated":1269760,"data_stored":1232355,"data_compressed":68564,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1599,"internal_metadata":31914433},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776209,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36448,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708704,"statfs":{"total":95995035648,"available":95957712896,"internally_reserved":0,"allocated":1138688,"data_stored":577539,"data_compressed":32447,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037845,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28112,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717040,"statfs":{"total":95995035648,"available":95966248960,"internally_reserved":0,"allocated":1122304,"data_stored":573149,"data_compressed":32447,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299482,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28172,"kb_used_data":1156,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716980,"statfs":{"total":95995035648,"available":95966187520,"internally_reserved":0,"allocated":1183744,"data_stored":1172218,"data_compressed":68564,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863463008,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36420,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708732,"statfs":{"total":95995035648,"available":95957741568,"internally_reserved":0,"allocated":1110016,"data_stored":569868,"data_compressed":32447,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574969,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36496,"kb_used_data":1160,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708656,"statfs":{"total":95995035648,"available":95957663744,"internally_reserved":0,"allocated":1187840,"data_stored":642393,"data_compressed":32447,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738494,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28124,"kb_used_data":1108,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717028,"statfs":{"total":95995035648,"available":95966236672,"internally_reserved":0,"allocated":1134592,"data_stored":570934,"data_compressed":32447,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411444,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28196,"kb_used_data":1180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716956,"statfs":{"total":95995035648,"available":95966162944,"internally_reserved":0,"allocated":1208320,"data_stored":643062,"data_compressed":32447,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149806,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36432,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708720,"statfs":{"total":95995035648,"available":95957729280,"internally_reserved":0,"allocated":1122304,"data_stored":568655,"data_compressed":32447,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888170,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28116,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717036,"statfs":{"total":95995035648,"available":95966244864,"internally_reserved":0,"allocated":1126400,"data_stored":574224,"data_compressed":32447,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724644,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28180,"kb_used_data":1164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716972,"statfs":{"total":95995035648,"available":95966179328,"internally_reserved":0,"allocated":1191936,"data_stored":1166348,"data_compressed":68564,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34481,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34481,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34481,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36117,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36117,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36117,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36117,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36117,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":74168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":74837,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1903,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":126976,"data_stored":73762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-06-10T02:54:32.104 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T02:54:32.360 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:54:33.116 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:54:32 smithi052 ceph-mon[32362]: pgmap v444: 129 pgs: 129 active+clean; 656 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-06-10T02:54:33.116 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:54:32 smithi052 ceph-mon[32362]: from='client.15411 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-10T02:54:33.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:54:32 smithi120 ceph-mon[36634]: pgmap v444: 129 pgs: 129 active+clean; 656 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-06-10T02:54:33.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:54:32 smithi120 ceph-mon[36634]: from='client.15411 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-10T02:54:33.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:54:32 smithi188 ceph-mon[36599]: pgmap v444: 129 pgs: 129 active+clean; 656 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-06-10T02:54:33.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:54:32 smithi188 ceph-mon[36599]: from='client.15411 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-10T02:54:34.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:34.070+0000 7f5220b6c700 1 -- 172.21.15.52:0/2577563292 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f521c107840 msgr2=0x7f521c10f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:54:34.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:34.070+0000 7f5220b6c700 1 --2- 172.21.15.52:0/2577563292 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f521c107840 0x7f521c10f3c0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f521400b0d0 tx=0x7f521400b3e0 comp rx=0 tx=0).stop 2024-06-10T02:54:34.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:34.071+0000 7f5220b6c700 1 -- 172.21.15.52:0/2577563292 shutdown_connections 2024-06-10T02:54:34.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:34.071+0000 7f5220b6c700 1 --2- 172.21.15.52:0/2577563292 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f521c107840 0x7f521c10f3c0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:34.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:34.071+0000 7f5220b6c700 1 --2- 172.21.15.52:0/2577563292 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f521c102740 0x7f521c102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:34.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:34.071+0000 7f5220b6c700 1 --2- 172.21.15.52:0/2577563292 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f521c1030f0 0x7f521c107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:34.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:34.071+0000 7f5220b6c700 1 -- 172.21.15.52:0/2577563292 >> 172.21.15.52:0/2577563292 conn(0x7f521c0795f0 msgr2=0x7f521c079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:54:34.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:34.071+0000 7f5220b6c700 1 -- 172.21.15.52:0/2577563292 shutdown_connections 2024-06-10T02:54:34.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:34.071+0000 7f5220b6c700 1 -- 172.21.15.52:0/2577563292 wait complete. 2024-06-10T02:54:34.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:34.071+0000 7f5220b6c700 1 Processor -- start 2024-06-10T02:54:34.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:34.072+0000 7f5220b6c700 1 -- start start 2024-06-10T02:54:34.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:34.072+0000 7f5220b6c700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f521c102740 0x7f521c1a0a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:34.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:34.072+0000 7f5220b6c700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f521c1030f0 0x7f521c1a0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:34.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:34.072+0000 7f5220b6c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f521c107840 0x7f521c19ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:34.074 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:34.072+0000 7f5220b6c700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f521c112760 con 0x7f521c107840 2024-06-10T02:54:34.074 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:34.072+0000 7f5220b6c700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f521c1125e0 con 0x7f521c102740 2024-06-10T02:54:34.074 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:34.072+0000 7f5220b6c700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f521c1128e0 con 0x7f521c1030f0 2024-06-10T02:54:34.074 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:34.072+0000 7f521ad9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f521c107840 0x7f521c19ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:34.074 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:34.072+0000 7f521ad9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f521c107840 0x7f521c19ab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51850/0 (socket says 172.21.15.52:51850) 2024-06-10T02:54:34.074 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:34.072+0000 7f521ad9d700 1 -- 172.21.15.52:0/3093478671 learned_addr learned my addr 172.21.15.52:0/3093478671 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:54:34.074 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:34.072+0000 7f521a59c700 1 --2- 172.21.15.52:0/3093478671 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f521c102740 0x7f521c1a0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:34.074 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:34.072+0000 7f521ad9d700 1 -- 172.21.15.52:0/3093478671 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f521c1030f0 msgr2=0x7f521c1a0fa0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-06-10T02:54:34.074 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:34.072+0000 7f521ad9d700 1 --2- 172.21.15.52:0/3093478671 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f521c1030f0 0x7f521c1a0fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:34.074 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:34.072+0000 7f521ad9d700 1 -- 172.21.15.52:0/3093478671 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f521c102740 msgr2=0x7f521c1a0a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:54:34.074 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:34.072+0000 7f521ad9d700 1 --2- 172.21.15.52:0/3093478671 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f521c102740 0x7f521c1a0a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:34.074 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:34.072+0000 7f521ad9d700 1 -- 172.21.15.52:0/3093478671 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f521c19b410 con 0x7f521c107840 2024-06-10T02:54:34.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:34.072+0000 7f521ad9d700 1 --2- 172.21.15.52:0/3093478671 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f521c107840 0x7f521c19ab30 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f5214006070 tx=0x7f5214004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:54:34.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:34.073+0000 7f520b7fe700 1 -- 172.21.15.52:0/3093478671 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5214021070 con 0x7f521c107840 2024-06-10T02:54:34.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:34.073+0000 7f520b7fe700 1 -- 172.21.15.52:0/3093478671 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5214007b80 con 0x7f521c107840 2024-06-10T02:54:34.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:34.073+0000 7f5220b6c700 1 -- 172.21.15.52:0/3093478671 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f521c19b6a0 con 0x7f521c107840 2024-06-10T02:54:34.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:34.073+0000 7f520b7fe700 1 -- 172.21.15.52:0/3093478671 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5214005330 con 0x7f521c107840 2024-06-10T02:54:34.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:34.073+0000 7f5220b6c700 1 -- 172.21.15.52:0/3093478671 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f521c1a79e0 con 0x7f521c107840 2024-06-10T02:54:34.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:34.074+0000 7f5220b6c700 1 -- 172.21.15.52:0/3093478671 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f521c19ba10 con 0x7f521c107840 2024-06-10T02:54:34.076 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:34.075+0000 7f520b7fe700 1 -- 172.21.15.52:0/3093478671 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7f5214024070 con 0x7f521c107840 2024-06-10T02:54:34.076 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:34.076+0000 7f520b7fe700 1 --2- 172.21.15.52:0/3093478671 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f52040a2cb0 0x7f52040a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:34.076 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:34.076+0000 7f521a59c700 1 --2- 172.21.15.52:0/3093478671 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f52040a2cb0 0x7f52040a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:34.076 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:34.076+0000 7f520b7fe700 1 -- 172.21.15.52:0/3093478671 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f521400f020 con 0x7f521c107840 2024-06-10T02:54:34.076 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:34.076+0000 7f521a59c700 1 --2- 172.21.15.52:0/3093478671 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f52040a2cb0 0x7f52040a5170 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f520c005950 tx=0x7f520c00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:54:34.080 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:34.080+0000 7f520b7fe700 1 -- 172.21.15.52:0/3093478671 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f521c19ba10 con 0x7f521c107840 2024-06-10T02:54:34.338 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:34.338+0000 7f5220b6c700 1 -- 172.21.15.52:0/3093478671 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f521c103bd0 con 0x7f521c107840 2024-06-10T02:54:34.338 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:34.338+0000 7f520b7fe700 1 -- 172.21.15.52:0/3093478671 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11938 (secure 0 0 0) 0x7f521402b020 con 0x7f521c107840 2024-06-10T02:54:34.339 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-10T02:54:34.339 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/889673247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":889673247},{"type":"v1","addr":"172.21.15.52:6835","nonce":889673247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/3615663511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":3615663511},{"type":"v1","addr":"172.21.15.52:6837","nonce":3615663511}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T02:50:03.889766+0000","modified":"2024-06-10T02:54:07.086405+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24484,"mds_1":24469,"mds_2":24463,"mds_3":24449,"mds_4":24428},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14571":{"gid":14571,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6839/3059599042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":3059599042},{"type":"v1","addr":"172.21.15.52:6839","nonce":3059599042}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14583":{"gid":14583,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6841/3039275521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":3039275521},{"type":"v1","addr":"172.21.15.52:6841","nonce":3039275521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24383":{"gid":24383,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6833/2609002896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":2609002896},{"type":"v1","addr":"172.21.15.120:6833","nonce":2609002896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6835/1996370288","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":1996370288},{"type":"v1","addr":"172.21.15.120:6835","nonce":1996370288}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6833/529564979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":529564979},{"type":"v1","addr":"172.21.15.188:6833","nonce":529564979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":25,"addr":"172.21.15.120:6837/4132440952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":4132440952},{"type":"v1","addr":"172.21.15.120:6837","nonce":4132440952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"b","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.120:6839/1672436487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1672436487},{"type":"v1","addr":"172.21.15.120:6839","nonce":1672436487}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":2,"incarnation":22,"state":"up:active","state_seq":26,"addr":"172.21.15.188:6835/3904544259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":3904544259},{"type":"v1","addr":"172.21.15.188:6835","nonce":3904544259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.188:6837/819317726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":819317726},{"type":"v1","addr":"172.21.15.188:6837","nonce":819317726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.188:6839/310444836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":310444836},{"type":"v1","addr":"172.21.15.188:6839","nonce":310444836}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24484,"qdb_cluster":[24469,24484,24463,24428,24449]},"id":1}]} 2024-06-10T02:54:34.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:34.341+0000 7f5220b6c700 1 -- 172.21.15.52:0/3093478671 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f52040a2cb0 msgr2=0x7f52040a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:54:34.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:34.341+0000 7f5220b6c700 1 --2- 172.21.15.52:0/3093478671 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f52040a2cb0 0x7f52040a5170 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f520c005950 tx=0x7f520c00b410 comp rx=0 tx=0).stop 2024-06-10T02:54:34.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:34.341+0000 7f5220b6c700 1 -- 172.21.15.52:0/3093478671 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f521c107840 msgr2=0x7f521c19ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:54:34.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:34.341+0000 7f5220b6c700 1 --2- 172.21.15.52:0/3093478671 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f521c107840 0x7f521c19ab30 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f5214006070 tx=0x7f5214004d00 comp rx=0 tx=0).stop 2024-06-10T02:54:34.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:34.341+0000 7f5220b6c700 1 -- 172.21.15.52:0/3093478671 shutdown_connections 2024-06-10T02:54:34.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:34.341+0000 7f5220b6c700 1 --2- 172.21.15.52:0/3093478671 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f52040a2cb0 0x7f52040a5170 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:34.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:34.341+0000 7f5220b6c700 1 --2- 172.21.15.52:0/3093478671 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f521c107840 0x7f521c19ab30 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:34.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:34.341+0000 7f5220b6c700 1 --2- 172.21.15.52:0/3093478671 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f521c102740 0x7f521c1a0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:34.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:34.341+0000 7f5220b6c700 1 --2- 172.21.15.52:0/3093478671 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f521c1030f0 0x7f521c1a0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:34.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:34.341+0000 7f5220b6c700 1 -- 172.21.15.52:0/3093478671 >> 172.21.15.52:0/3093478671 conn(0x7f521c0795f0 msgr2=0x7f521c100650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:54:34.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:34.341+0000 7f5220b6c700 1 -- 172.21.15.52:0/3093478671 shutdown_connections 2024-06-10T02:54:34.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:34.341+0000 7f5220b6c700 1 -- 172.21.15.52:0/3093478671 wait complete. 2024-06-10T02:54:34.342 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 30 2024-06-10T02:54:34.985 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:54:34 smithi052 ceph-mon[32362]: pgmap v445: 129 pgs: 129 active+clean; 656 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 11 op/s 2024-06-10T02:54:34.985 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:54:34 smithi052 ceph-mon[32362]: from='client.? 172.21.15.52:0/3093478671' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T02:54:35.014 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T02:54:35.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:54:34 smithi120 ceph-mon[36634]: pgmap v445: 129 pgs: 129 active+clean; 656 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 11 op/s 2024-06-10T02:54:35.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:54:34 smithi120 ceph-mon[36634]: from='client.? 172.21.15.52:0/3093478671' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T02:54:35.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:54:34 smithi188 ceph-mon[36599]: pgmap v445: 129 pgs: 129 active+clean; 656 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 11 op/s 2024-06-10T02:54:35.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:54:34 smithi188 ceph-mon[36599]: from='client.? 172.21.15.52:0/3093478671' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T02:54:35.266 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:54:37.063 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:37.063+0000 7fe6b03ee700 1 -- 172.21.15.52:0/4132654603 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe6a8108500 msgr2=0x7fe6a81088e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:54:37.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:37.063+0000 7fe6b03ee700 1 --2- 172.21.15.52:0/4132654603 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe6a8108500 0x7fe6a81088e0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fe6a4009990 tx=0x7fe6a4009ca0 comp rx=0 tx=0).stop 2024-06-10T02:54:37.064 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:54:36 smithi052 ceph-mon[32362]: pgmap v446: 129 pgs: 129 active+clean; 656 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-06-10T02:54:37.064 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:54:36 smithi052 ceph-mon[32362]: scrub summary: idle 2024-06-10T02:54:37.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:37.063+0000 7fe6b03ee700 1 -- 172.21.15.52:0/4132654603 shutdown_connections 2024-06-10T02:54:37.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:37.063+0000 7fe6b03ee700 1 --2- 172.21.15.52:0/4132654603 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe6a806b580 0x7fe6a810f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:37.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:37.063+0000 7fe6b03ee700 1 --2- 172.21.15.52:0/4132654603 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe6a8108500 0x7fe6a81088e0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:37.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:37.063+0000 7fe6b03ee700 1 --2- 172.21.15.52:0/4132654603 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe6a806abc0 0x7fe6a806b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:37.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:37.063+0000 7fe6b03ee700 1 -- 172.21.15.52:0/4132654603 >> 172.21.15.52:0/4132654603 conn(0x7fe6a8078340 msgr2=0x7fe6a8078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:54:37.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:37.064+0000 7fe6b03ee700 1 -- 172.21.15.52:0/4132654603 shutdown_connections 2024-06-10T02:54:37.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:37.064+0000 7fe6b03ee700 1 -- 172.21.15.52:0/4132654603 wait complete. 2024-06-10T02:54:37.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:37.064+0000 7fe6b03ee700 1 Processor -- start 2024-06-10T02:54:37.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:37.064+0000 7fe6b03ee700 1 -- start start 2024-06-10T02:54:37.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:37.065+0000 7fe6b03ee700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe6a806abc0 0x7fe6a819e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:37.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:37.065+0000 7fe6b03ee700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe6a806b580 0x7fe6a819ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:37.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:37.065+0000 7fe6b03ee700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe6a8108500 0x7fe6a81988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:37.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:37.065+0000 7fe6b03ee700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fe6a8112760 con 0x7fe6a806b580 2024-06-10T02:54:37.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:37.065+0000 7fe6b03ee700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe6a81125e0 con 0x7fe6a8108500 2024-06-10T02:54:37.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:37.065+0000 7fe6b03ee700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fe6a81128e0 con 0x7fe6a806abc0 2024-06-10T02:54:37.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:37.065+0000 7fe6ad989700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe6a806b580 0x7fe6a819ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:37.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:37.065+0000 7fe6ad989700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe6a806b580 0x7fe6a819ec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51860/0 (socket says 172.21.15.52:51860) 2024-06-10T02:54:37.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:37.065+0000 7fe6ad989700 1 -- 172.21.15.52:0/3209884911 learned_addr learned my addr 172.21.15.52:0/3209884911 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:54:37.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:37.065+0000 7fe6ae98b700 1 --2- 172.21.15.52:0/3209884911 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe6a8108500 0x7fe6a81988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:37.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:37.065+0000 7fe6ae18a700 1 --2- 172.21.15.52:0/3209884911 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe6a806abc0 0x7fe6a819e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:37.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:37.065+0000 7fe6ad989700 1 -- 172.21.15.52:0/3209884911 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe6a806abc0 msgr2=0x7fe6a819e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:54:37.068 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:37.065+0000 7fe6ad989700 1 --2- 172.21.15.52:0/3209884911 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe6a806abc0 0x7fe6a819e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:37.068 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:37.065+0000 7fe6ad989700 1 -- 172.21.15.52:0/3209884911 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe6a8108500 msgr2=0x7fe6a81988e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:54:37.068 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:37.065+0000 7fe6ad989700 1 --2- 172.21.15.52:0/3209884911 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe6a8108500 0x7fe6a81988e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:37.068 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:37.065+0000 7fe6ad989700 1 -- 172.21.15.52:0/3209884911 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6a8199040 con 0x7fe6a806b580 2024-06-10T02:54:37.068 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:37.065+0000 7fe6ad989700 1 --2- 172.21.15.52:0/3209884911 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe6a806b580 0x7fe6a819ec60 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fe69800c9b0 tx=0x7fe69800ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:54:37.068 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:37.066+0000 7fe69f7fe700 1 -- 172.21.15.52:0/3209884911 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe698007b00 con 0x7fe6a806b580 2024-06-10T02:54:37.068 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:37.066+0000 7fe6b03ee700 1 -- 172.21.15.52:0/3209884911 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6a8199330 con 0x7fe6a806b580 2024-06-10T02:54:37.068 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:37.066+0000 7fe69f7fe700 1 -- 172.21.15.52:0/3209884911 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe69800ce50 con 0x7fe6a806b580 2024-06-10T02:54:37.068 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:37.066+0000 7fe69f7fe700 1 -- 172.21.15.52:0/3209884911 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe698018660 con 0x7fe6a806b580 2024-06-10T02:54:37.068 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:37.066+0000 7fe6b03ee700 1 -- 172.21.15.52:0/3209884911 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6a81a55f0 con 0x7fe6a806b580 2024-06-10T02:54:37.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:37.067+0000 7fe6b03ee700 1 -- 172.21.15.52:0/3209884911 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6a8069180 con 0x7fe6a806b580 2024-06-10T02:54:37.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:37.068+0000 7fe69f7fe700 1 -- 172.21.15.52:0/3209884911 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7fe69800f450 con 0x7fe6a806b580 2024-06-10T02:54:37.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:37.069+0000 7fe69f7fe700 1 --2- 172.21.15.52:0/3209884911 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fe6940a2cb0 0x7fe6940a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:37.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:37.069+0000 7fe6ae18a700 1 --2- 172.21.15.52:0/3209884911 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fe6940a2cb0 0x7fe6940a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:37.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:37.069+0000 7fe69f7fe700 1 -- 172.21.15.52:0/3209884911 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7fe6980cf720 con 0x7fe6a806b580 2024-06-10T02:54:37.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:37.069+0000 7fe6ae18a700 1 --2- 172.21.15.52:0/3209884911 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fe6940a2cb0 0x7fe6940a5170 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fe6a400b5c0 tx=0x7fe6a4005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:54:37.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:37.072+0000 7fe69f7fe700 1 -- 172.21.15.52:0/3209884911 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fe698014030 con 0x7fe6a806b580 2024-06-10T02:54:37.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:54:36 smithi120 ceph-mon[36634]: pgmap v446: 129 pgs: 129 active+clean; 656 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-06-10T02:54:37.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:54:36 smithi120 ceph-mon[36634]: scrub summary: idle 2024-06-10T02:54:37.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:54:36 smithi188 ceph-mon[36599]: pgmap v446: 129 pgs: 129 active+clean; 656 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-06-10T02:54:37.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:54:36 smithi188 ceph-mon[36599]: scrub summary: idle 2024-06-10T02:54:37.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:37.326+0000 7fe6b03ee700 1 -- 172.21.15.52:0/3209884911 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe6a81a5880 con 0x7fe6a806b580 2024-06-10T02:54:37.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:37.327+0000 7fe69f7fe700 1 -- 172.21.15.52:0/3209884911 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11938 (secure 0 0 0) 0x7fe69809c200 con 0x7fe6a806b580 2024-06-10T02:54:37.327 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-10T02:54:37.327 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/889673247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":889673247},{"type":"v1","addr":"172.21.15.52:6835","nonce":889673247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/3615663511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":3615663511},{"type":"v1","addr":"172.21.15.52:6837","nonce":3615663511}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T02:50:03.889766+0000","modified":"2024-06-10T02:54:07.086405+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24484,"mds_1":24469,"mds_2":24463,"mds_3":24449,"mds_4":24428},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14571":{"gid":14571,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6839/3059599042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":3059599042},{"type":"v1","addr":"172.21.15.52:6839","nonce":3059599042}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14583":{"gid":14583,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6841/3039275521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":3039275521},{"type":"v1","addr":"172.21.15.52:6841","nonce":3039275521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24383":{"gid":24383,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6833/2609002896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":2609002896},{"type":"v1","addr":"172.21.15.120:6833","nonce":2609002896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6835/1996370288","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":1996370288},{"type":"v1","addr":"172.21.15.120:6835","nonce":1996370288}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6833/529564979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":529564979},{"type":"v1","addr":"172.21.15.188:6833","nonce":529564979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":25,"addr":"172.21.15.120:6837/4132440952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":4132440952},{"type":"v1","addr":"172.21.15.120:6837","nonce":4132440952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"b","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.120:6839/1672436487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1672436487},{"type":"v1","addr":"172.21.15.120:6839","nonce":1672436487}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":2,"incarnation":22,"state":"up:active","state_seq":26,"addr":"172.21.15.188:6835/3904544259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":3904544259},{"type":"v1","addr":"172.21.15.188:6835","nonce":3904544259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.188:6837/819317726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":819317726},{"type":"v1","addr":"172.21.15.188:6837","nonce":819317726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.188:6839/310444836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":310444836},{"type":"v1","addr":"172.21.15.188:6839","nonce":310444836}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24484,"qdb_cluster":[24469,24484,24463,24428,24449]},"id":1}]} 2024-06-10T02:54:37.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:37.330+0000 7fe6b03ee700 1 -- 172.21.15.52:0/3209884911 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fe6940a2cb0 msgr2=0x7fe6940a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:54:37.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:37.330+0000 7fe6b03ee700 1 --2- 172.21.15.52:0/3209884911 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fe6940a2cb0 0x7fe6940a5170 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fe6a400b5c0 tx=0x7fe6a4005960 comp rx=0 tx=0).stop 2024-06-10T02:54:37.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:37.330+0000 7fe6b03ee700 1 -- 172.21.15.52:0/3209884911 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe6a806b580 msgr2=0x7fe6a819ec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:54:37.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:37.330+0000 7fe6b03ee700 1 --2- 172.21.15.52:0/3209884911 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe6a806b580 0x7fe6a819ec60 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fe69800c9b0 tx=0x7fe69800ccc0 comp rx=0 tx=0).stop 2024-06-10T02:54:37.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:37.330+0000 7fe6b03ee700 1 -- 172.21.15.52:0/3209884911 shutdown_connections 2024-06-10T02:54:37.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:37.330+0000 7fe6b03ee700 1 --2- 172.21.15.52:0/3209884911 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe6a8108500 0x7fe6a81988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:37.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:37.330+0000 7fe6b03ee700 1 --2- 172.21.15.52:0/3209884911 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe6a806abc0 0x7fe6a819e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:37.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:37.330+0000 7fe6b03ee700 1 --2- 172.21.15.52:0/3209884911 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fe6940a2cb0 0x7fe6940a5170 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:37.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:37.330+0000 7fe6b03ee700 1 --2- 172.21.15.52:0/3209884911 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe6a806b580 0x7fe6a819ec60 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:37.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:37.330+0000 7fe6b03ee700 1 -- 172.21.15.52:0/3209884911 >> 172.21.15.52:0/3209884911 conn(0x7fe6a8078340 msgr2=0x7fe6a80ffd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:54:37.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:37.330+0000 7fe6b03ee700 1 -- 172.21.15.52:0/3209884911 shutdown_connections 2024-06-10T02:54:37.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:37.330+0000 7fe6b03ee700 1 -- 172.21.15.52:0/3209884911 wait complete. 2024-06-10T02:54:37.330 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 30 2024-06-10T02:54:37.938 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:54:37 smithi052 ceph-mon[32362]: from='client.? 172.21.15.52:0/3209884911' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T02:54:37.967 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-10T02:54:38.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:54:37 smithi120 ceph-mon[36634]: from='client.? 172.21.15.52:0/3209884911' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T02:54:38.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:54:37 smithi188 ceph-mon[36599]: from='client.? 172.21.15.52:0/3209884911' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T02:54:38.227 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:54:38.976 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:54:38 smithi052 ceph-mon[32362]: pgmap v447: 129 pgs: 129 active+clean; 656 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 11 op/s 2024-06-10T02:54:39.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:54:38 smithi120 ceph-mon[36634]: pgmap v447: 129 pgs: 129 active+clean; 656 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 11 op/s 2024-06-10T02:54:39.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:54:38 smithi188 ceph-mon[36599]: pgmap v447: 129 pgs: 129 active+clean; 656 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 11 op/s 2024-06-10T02:54:39.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:39.959+0000 7f03efc2e700 1 -- 172.21.15.52:0/1133621614 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f03e8102ed0 msgr2=0x7f03e810f360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:54:39.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:39.959+0000 7f03efc2e700 1 --2- 172.21.15.52:0/1133621614 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f03e8102ed0 0x7f03e810f360 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f03e000b0a0 tx=0x7f03e000b3b0 comp rx=0 tx=0).stop 2024-06-10T02:54:39.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:39.959+0000 7f03efc2e700 1 -- 172.21.15.52:0/1133621614 shutdown_connections 2024-06-10T02:54:39.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:39.959+0000 7f03efc2e700 1 --2- 172.21.15.52:0/1133621614 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f03e8102ed0 0x7f03e810f360 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:39.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:39.959+0000 7f03efc2e700 1 --2- 172.21.15.52:0/1133621614 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f03e8108520 0x7f03e8108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:39.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:39.959+0000 7f03efc2e700 1 --2- 172.21.15.52:0/1133621614 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f03e8102510 0x7f03e8102990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:39.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:39.959+0000 7f03efc2e700 1 -- 172.21.15.52:0/1133621614 >> 172.21.15.52:0/1133621614 conn(0x7f03e8079640 msgr2=0x7f03e8079a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:54:39.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:39.959+0000 7f03efc2e700 1 -- 172.21.15.52:0/1133621614 shutdown_connections 2024-06-10T02:54:39.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:39.959+0000 7f03efc2e700 1 -- 172.21.15.52:0/1133621614 wait complete. 2024-06-10T02:54:39.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:39.960+0000 7f03efc2e700 1 Processor -- start 2024-06-10T02:54:39.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:39.960+0000 7f03efc2e700 1 -- start start 2024-06-10T02:54:39.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:39.960+0000 7f03efc2e700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f03e8102510 0x7f03e819e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:39.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:39.960+0000 7f03efc2e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f03e8102ed0 0x7f03e819ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:39.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:39.960+0000 7f03efc2e700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f03e8108520 0x7f03e8198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:39.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:39.960+0000 7f03efc2e700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f03e81127c0 con 0x7f03e8102ed0 2024-06-10T02:54:39.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:39.960+0000 7f03efc2e700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f03e8112640 con 0x7f03e8108520 2024-06-10T02:54:39.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:39.960+0000 7f03efc2e700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f03e8112940 con 0x7f03e8102510 2024-06-10T02:54:39.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:39.960+0000 7f03ed1c9700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f03e8102ed0 0x7f03e819ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:39.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:39.960+0000 7f03ed1c9700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f03e8102ed0 0x7f03e819ec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51876/0 (socket says 172.21.15.52:51876) 2024-06-10T02:54:39.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:39.960+0000 7f03ed1c9700 1 -- 172.21.15.52:0/1298124524 learned_addr learned my addr 172.21.15.52:0/1298124524 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:54:39.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:39.960+0000 7f03ed9ca700 1 --2- 172.21.15.52:0/1298124524 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f03e8102510 0x7f03e819e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:39.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:39.960+0000 7f03ee1cb700 1 --2- 172.21.15.52:0/1298124524 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f03e8108520 0x7f03e8198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:39.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:39.960+0000 7f03ed1c9700 1 -- 172.21.15.52:0/1298124524 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f03e8102510 msgr2=0x7f03e819e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:54:39.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:39.960+0000 7f03ed1c9700 1 --2- 172.21.15.52:0/1298124524 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f03e8102510 0x7f03e819e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:39.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:39.961+0000 7f03ed1c9700 1 -- 172.21.15.52:0/1298124524 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f03e8108520 msgr2=0x7f03e8198920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:54:39.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:39.961+0000 7f03ed1c9700 1 --2- 172.21.15.52:0/1298124524 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f03e8108520 0x7f03e8198920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:39.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:39.961+0000 7f03ed1c9700 1 -- 172.21.15.52:0/1298124524 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03e8199080 con 0x7f03e8102ed0 2024-06-10T02:54:39.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:39.961+0000 7f03ed1c9700 1 --2- 172.21.15.52:0/1298124524 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f03e8102ed0 0x7f03e819ec50 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f03d800c9b0 tx=0x7f03d800ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:54:39.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:39.961+0000 7f03deffd700 1 -- 172.21.15.52:0/1298124524 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03d8007b00 con 0x7f03e8102ed0 2024-06-10T02:54:39.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:39.961+0000 7f03efc2e700 1 -- 172.21.15.52:0/1298124524 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03e8199370 con 0x7f03e8102ed0 2024-06-10T02:54:39.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:39.961+0000 7f03deffd700 1 -- 172.21.15.52:0/1298124524 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f03d800ce50 con 0x7f03e8102ed0 2024-06-10T02:54:39.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:39.961+0000 7f03deffd700 1 -- 172.21.15.52:0/1298124524 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03d8018660 con 0x7f03e8102ed0 2024-06-10T02:54:39.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:39.961+0000 7f03efc2e700 1 -- 172.21.15.52:0/1298124524 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03e81a55e0 con 0x7f03e8102ed0 2024-06-10T02:54:39.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:39.962+0000 7f03efc2e700 1 -- 172.21.15.52:0/1298124524 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03e8069180 con 0x7f03e8102ed0 2024-06-10T02:54:39.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:39.963+0000 7f03deffd700 1 -- 172.21.15.52:0/1298124524 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7f03d800f450 con 0x7f03e8102ed0 2024-06-10T02:54:39.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:39.964+0000 7f03deffd700 1 --2- 172.21.15.52:0/1298124524 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f03d40a2cb0 0x7f03d40a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:39.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:39.964+0000 7f03deffd700 1 -- 172.21.15.52:0/1298124524 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f03d80cf4b0 con 0x7f03e8102ed0 2024-06-10T02:54:39.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:39.964+0000 7f03ed9ca700 1 --2- 172.21.15.52:0/1298124524 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f03d40a2cb0 0x7f03d40a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:39.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:39.964+0000 7f03ed9ca700 1 --2- 172.21.15.52:0/1298124524 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f03d40a2cb0 0x7f03d40a5170 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f03e8100be0 tx=0x7f03e400b3c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:54:39.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:39.969+0000 7f03deffd700 1 -- 172.21.15.52:0/1298124524 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f03d8014030 con 0x7f03e8102ed0 2024-06-10T02:54:40.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:40.176+0000 7f03efc2e700 1 -- 172.21.15.52:0/1298124524 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f03e802dca0 con 0x7f03e8102ed0 2024-06-10T02:54:40.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:40.178+0000 7f03deffd700 1 -- 172.21.15.52:0/1298124524 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v100) ==== 74+0+24739 (secure 0 0 0) 0x7f03d809bf00 con 0x7f03e8102ed0 2024-06-10T02:54:40.177 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-10T02:54:40.178 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":100,"fsid":"d8676e9e-26d2-11ef-bc9b-c7b262605968","created":"2024-06-10T02:41:43.076837+0000","modified":"2024-06-10T02:51:45.708105+0000","last_up_change":"2024-06-10T02:48:57.297829+0000","last_in_change":"2024-06-10T02:48:36.865802+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-10T02:45:07.394690+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-10T02:49:57.926838+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-10T02:50:01.130393+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-10T02:50:10.655588+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e03c0691-cc5a-497a-bbc2-ce18c337f074","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":4255833408},{"type":"v1","addr":"172.21.15.52:6803","nonce":4255833408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":4255833408},{"type":"v1","addr":"172.21.15.52:6805","nonce":4255833408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":4255833408},{"type":"v1","addr":"172.21.15.52:6809","nonce":4255833408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":4255833408},{"type":"v1","addr":"172.21.15.52:6807","nonce":4255833408}]},"public_addr":"172.21.15.52:6803/4255833408","cluster_addr":"172.21.15.52:6805/4255833408","heartbeat_back_addr":"172.21.15.52:6809/4255833408","heartbeat_front_addr":"172.21.15.52:6807/4255833408","state":["exists","up"]},{"osd":1,"uuid":"a24d8da8-c5ec-44d7-ab30-56f6ce3efda7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":4116540053},{"type":"v1","addr":"172.21.15.120:6801","nonce":4116540053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":4116540053},{"type":"v1","addr":"172.21.15.120:6803","nonce":4116540053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":4116540053},{"type":"v1","addr":"172.21.15.120:6807","nonce":4116540053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":4116540053},{"type":"v1","addr":"172.21.15.120:6805","nonce":4116540053}]},"public_addr":"172.21.15.120:6801/4116540053","cluster_addr":"172.21.15.120:6803/4116540053","heartbeat_back_addr":"172.21.15.120:6807/4116540053","heartbeat_front_addr":"172.21.15.120:6805/4116540053","state":["exists","up"]},{"osd":2,"uuid":"a7162106-ccbf-45bd-bc0e-051ee43f4214","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":2624326999},{"type":"v1","addr":"172.21.15.188:6801","nonce":2624326999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":2624326999},{"type":"v1","addr":"172.21.15.188:6803","nonce":2624326999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":2624326999},{"type":"v1","addr":"172.21.15.188:6807","nonce":2624326999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":2624326999},{"type":"v1","addr":"172.21.15.188:6805","nonce":2624326999}]},"public_addr":"172.21.15.188:6801/2624326999","cluster_addr":"172.21.15.188:6803/2624326999","heartbeat_back_addr":"172.21.15.188:6807/2624326999","heartbeat_front_addr":"172.21.15.188:6805/2624326999","state":["exists","up"]},{"osd":3,"uuid":"53680244-b49a-4120-b24a-9ad7a6341dc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":116897011},{"type":"v1","addr":"172.21.15.52:6811","nonce":116897011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":116897011},{"type":"v1","addr":"172.21.15.52:6813","nonce":116897011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":116897011},{"type":"v1","addr":"172.21.15.52:6817","nonce":116897011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":116897011},{"type":"v1","addr":"172.21.15.52:6815","nonce":116897011}]},"public_addr":"172.21.15.52:6811/116897011","cluster_addr":"172.21.15.52:6813/116897011","heartbeat_back_addr":"172.21.15.52:6817/116897011","heartbeat_front_addr":"172.21.15.52:6815/116897011","state":["exists","up"]},{"osd":4,"uuid":"9f72f1f4-2ef3-49c5-8410-c108767d2ef5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":1320690621},{"type":"v1","addr":"172.21.15.120:6809","nonce":1320690621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":1320690621},{"type":"v1","addr":"172.21.15.120:6811","nonce":1320690621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":1320690621},{"type":"v1","addr":"172.21.15.120:6815","nonce":1320690621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":1320690621},{"type":"v1","addr":"172.21.15.120:6813","nonce":1320690621}]},"public_addr":"172.21.15.120:6809/1320690621","cluster_addr":"172.21.15.120:6811/1320690621","heartbeat_back_addr":"172.21.15.120:6815/1320690621","heartbeat_front_addr":"172.21.15.120:6813/1320690621","state":["exists","up"]},{"osd":5,"uuid":"001b5706-7f27-4fa4-9b69-969e6d765abc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3951947483},{"type":"v1","addr":"172.21.15.188:6809","nonce":3951947483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3951947483},{"type":"v1","addr":"172.21.15.188:6811","nonce":3951947483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3951947483},{"type":"v1","addr":"172.21.15.188:6815","nonce":3951947483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3951947483},{"type":"v1","addr":"172.21.15.188:6813","nonce":3951947483}]},"public_addr":"172.21.15.188:6809/3951947483","cluster_addr":"172.21.15.188:6811/3951947483","heartbeat_back_addr":"172.21.15.188:6815/3951947483","heartbeat_front_addr":"172.21.15.188:6813/3951947483","state":["exists","up"]},{"osd":6,"uuid":"140c903e-9b31-4a81-9ccd-2edc0a2decf9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":359084919},{"type":"v1","addr":"172.21.15.52:6819","nonce":359084919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":359084919},{"type":"v1","addr":"172.21.15.52:6821","nonce":359084919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":359084919},{"type":"v1","addr":"172.21.15.52:6825","nonce":359084919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":359084919},{"type":"v1","addr":"172.21.15.52:6823","nonce":359084919}]},"public_addr":"172.21.15.52:6819/359084919","cluster_addr":"172.21.15.52:6821/359084919","heartbeat_back_addr":"172.21.15.52:6825/359084919","heartbeat_front_addr":"172.21.15.52:6823/359084919","state":["exists","up"]},{"osd":7,"uuid":"57b735d4-5072-4ca2-a3ff-3763ad84f15a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":3737722011},{"type":"v1","addr":"172.21.15.120:6817","nonce":3737722011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6818","nonce":3737722011},{"type":"v1","addr":"172.21.15.120:6819","nonce":3737722011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":3737722011},{"type":"v1","addr":"172.21.15.120:6823","nonce":3737722011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":3737722011},{"type":"v1","addr":"172.21.15.120:6821","nonce":3737722011}]},"public_addr":"172.21.15.120:6817/3737722011","cluster_addr":"172.21.15.120:6819/3737722011","heartbeat_back_addr":"172.21.15.120:6823/3737722011","heartbeat_front_addr":"172.21.15.120:6821/3737722011","state":["exists","up"]},{"osd":8,"uuid":"4ffbb43b-854a-4eac-99cf-7af89bc65409","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1444828441},{"type":"v1","addr":"172.21.15.188:6817","nonce":1444828441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1444828441},{"type":"v1","addr":"172.21.15.188:6819","nonce":1444828441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1444828441},{"type":"v1","addr":"172.21.15.188:6823","nonce":1444828441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1444828441},{"type":"v1","addr":"172.21.15.188:6821","nonce":1444828441}]},"public_addr":"172.21.15.188:6817/1444828441","cluster_addr":"172.21.15.188:6819/1444828441","heartbeat_back_addr":"172.21.15.188:6823/1444828441","heartbeat_front_addr":"172.21.15.188:6821/1444828441","state":["exists","up"]},{"osd":9,"uuid":"5bfd985c-8acc-4d00-8561-40bbd541c07b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":924440658},{"type":"v1","addr":"172.21.15.52:6827","nonce":924440658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":924440658},{"type":"v1","addr":"172.21.15.52:6829","nonce":924440658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":924440658},{"type":"v1","addr":"172.21.15.52:6833","nonce":924440658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":924440658},{"type":"v1","addr":"172.21.15.52:6831","nonce":924440658}]},"public_addr":"172.21.15.52:6827/924440658","cluster_addr":"172.21.15.52:6829/924440658","heartbeat_back_addr":"172.21.15.52:6833/924440658","heartbeat_front_addr":"172.21.15.52:6831/924440658","state":["exists","up"]},{"osd":10,"uuid":"0180cddf-e109-42cc-829a-185604276f02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6824","nonce":1531769083},{"type":"v1","addr":"172.21.15.120:6825","nonce":1531769083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":1531769083},{"type":"v1","addr":"172.21.15.120:6827","nonce":1531769083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":1531769083},{"type":"v1","addr":"172.21.15.120:6831","nonce":1531769083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6828","nonce":1531769083},{"type":"v1","addr":"172.21.15.120:6829","nonce":1531769083}]},"public_addr":"172.21.15.120:6825/1531769083","cluster_addr":"172.21.15.120:6827/1531769083","heartbeat_back_addr":"172.21.15.120:6831/1531769083","heartbeat_front_addr":"172.21.15.120:6829/1531769083","state":["exists","up"]},{"osd":11,"uuid":"73aec750-82d1-4800-b6a0-4e305912f9ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":3769050170},{"type":"v1","addr":"172.21.15.188:6825","nonce":3769050170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":3769050170},{"type":"v1","addr":"172.21.15.188:6827","nonce":3769050170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":3769050170},{"type":"v1","addr":"172.21.15.188:6831","nonce":3769050170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":3769050170},{"type":"v1","addr":"172.21.15.188:6829","nonce":3769050170}]},"public_addr":"172.21.15.188:6825/3769050170","cluster_addr":"172.21.15.188:6827/3769050170","heartbeat_back_addr":"172.21.15.188:6831/3769050170","heartbeat_front_addr":"172.21.15.188:6829/3769050170","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:44:16.455752+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:44:40.219735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:45:03.558899+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:45:29.155004+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:45:53.981098+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:46:19.412020+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:46:45.462640+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:47:11.484578+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:47:36.879054+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:48:03.392446+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:48:29.422173+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:48:55.523423+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":4}]},{"pgid":"3.6","mappings":[{"from":3,"to":7}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.9","mappings":[{"from":5,"to":8}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.52:0/3535271671":"2024-06-11T02:42:42.872312+0000","172.21.15.52:0/1092934380":"2024-06-11T02:42:42.872312+0000","172.21.15.52:6800/896317057":"2024-06-11T02:42:42.872312+0000","172.21.15.52:6801/3934479642":"2024-06-11T02:42:15.498118+0000","172.21.15.52:6800/3934479642":"2024-06-11T02:42:15.498118+0000","172.21.15.52:0/2759694132":"2024-06-11T02:42:15.498118+0000","172.21.15.52:0/2593665631":"2024-06-11T02:42:15.498118+0000","172.21.15.52:0/3151504634":"2024-06-11T02:42:15.498118+0000","172.21.15.52:6801/896317057":"2024-06-11T02:42:42.872312+0000","172.21.15.52:0/1254480982":"2024-06-11T02:42:42.872312+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-10T02:54:40.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:40.180+0000 7f03efc2e700 1 -- 172.21.15.52:0/1298124524 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f03d40a2cb0 msgr2=0x7f03d40a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:54:40.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:40.180+0000 7f03efc2e700 1 --2- 172.21.15.52:0/1298124524 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f03d40a2cb0 0x7f03d40a5170 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f03e8100be0 tx=0x7f03e400b3c0 comp rx=0 tx=0).stop 2024-06-10T02:54:40.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:40.180+0000 7f03efc2e700 1 -- 172.21.15.52:0/1298124524 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f03e8102ed0 msgr2=0x7f03e819ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:54:40.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:40.180+0000 7f03efc2e700 1 --2- 172.21.15.52:0/1298124524 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f03e8102ed0 0x7f03e819ec50 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f03d800c9b0 tx=0x7f03d800ccc0 comp rx=0 tx=0).stop 2024-06-10T02:54:40.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:40.180+0000 7f03efc2e700 1 -- 172.21.15.52:0/1298124524 shutdown_connections 2024-06-10T02:54:40.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:40.180+0000 7f03efc2e700 1 --2- 172.21.15.52:0/1298124524 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f03e8108520 0x7f03e8198920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:40.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:40.180+0000 7f03efc2e700 1 --2- 172.21.15.52:0/1298124524 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f03e8102510 0x7f03e819e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:40.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:40.180+0000 7f03efc2e700 1 --2- 172.21.15.52:0/1298124524 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f03d40a2cb0 0x7f03d40a5170 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:40.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:40.180+0000 7f03efc2e700 1 --2- 172.21.15.52:0/1298124524 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f03e8102ed0 0x7f03e819ec50 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:40.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:40.180+0000 7f03efc2e700 1 -- 172.21.15.52:0/1298124524 >> 172.21.15.52:0/1298124524 conn(0x7f03e8079640 msgr2=0x7f03e8100f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:54:40.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:40.180+0000 7f03efc2e700 1 -- 172.21.15.52:0/1298124524 shutdown_connections 2024-06-10T02:54:40.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:40.180+0000 7f03efc2e700 1 -- 172.21.15.52:0/1298124524 wait complete. 2024-06-10T02:54:40.805 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph daemon mon.a --format=json config get run_dir 2024-06-10T02:54:41.059 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:54:41.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:54:40 smithi120 ceph-mon[36634]: pgmap v448: 129 pgs: 129 active+clean; 656 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-10T02:54:41.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:54:40 smithi120 ceph-mon[36634]: from='client.? 172.21.15.52:0/1298124524' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T02:54:41.182 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:54:40 smithi052 ceph-mon[32362]: pgmap v448: 129 pgs: 129 active+clean; 656 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-10T02:54:41.182 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:54:40 smithi052 ceph-mon[32362]: from='client.? 172.21.15.52:0/1298124524' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T02:54:41.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:54:40 smithi188 ceph-mon[36599]: pgmap v448: 129 pgs: 129 active+clean; 656 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-10T02:54:41.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:54:40 smithi188 ceph-mon[36599]: from='client.? 172.21.15.52:0/1298124524' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T02:54:43.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:54:42 smithi120 ceph-mon[36634]: pgmap v449: 129 pgs: 129 active+clean; 656 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-10T02:54:43.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:54:42 smithi188 ceph-mon[36599]: pgmap v449: 129 pgs: 129 active+clean; 656 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-10T02:54:43.226 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:54:42 smithi052 ceph-mon[32362]: pgmap v449: 129 pgs: 129 active+clean; 656 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-10T02:54:43.479 INFO:teuthology.orchestra.run.smithi052.stdout:{"run_dir":"/var/run/ceph"} 2024-06-10T02:54:43.479 DEBUG:tasks.cephfs.filesystem:_json_asok output { "run_dir": "/var/run/ceph" } 2024-06-10T02:54:43.479 INFO:tasks.quiescer.fs.[cephfs]:Initializing Quiescer with seed 178 2024-06-10T02:54:43.479 DEBUG:tasks.quiescer:Yielding 2024-06-10T02:54:43.480 INFO:teuthology.run_tasks:Running task workunit... 2024-06-10T02:54:43.494 INFO:tasks.workunit:Pulling workunits from ref 7b7a5d375cf4e4afab96b9d4d229b699ae8304bb 2024-06-10T02:54:43.494 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-06-10T02:54:43.494 INFO:tasks.workunit:timeout=3h 2024-06-10T02:54:43.495 INFO:tasks.workunit:cleanup=True 2024-06-10T02:54:43.495 DEBUG:teuthology.orchestra.run.smithi052:> stat -- /home/ubuntu/cephtest/mnt.0 2024-06-10T02:54:43.495 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T02:54:43.513 INFO:teuthology.orchestra.run.smithi052.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-06-10T02:54:43.513 INFO:teuthology.orchestra.run.smithi052.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-06-10T02:54:43.513 INFO:teuthology.orchestra.run.smithi052.stdout:Device: 77h/119d Inode: 1099511628283 Links: 2 2024-06-10T02:54:43.513 INFO:teuthology.orchestra.run.smithi052.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-10T02:54:43.513 INFO:teuthology.orchestra.run.smithi052.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-06-10T02:54:43.513 INFO:teuthology.orchestra.run.smithi052.stdout:Access: 2024-06-10 02:51:23.198023121 +0000 2024-06-10T02:54:43.513 INFO:teuthology.orchestra.run.smithi052.stdout:Modify: 2024-06-10 02:51:23.198023121 +0000 2024-06-10T02:54:43.513 INFO:teuthology.orchestra.run.smithi052.stdout:Change: 2024-06-10 02:54:06.356250577 +0000 2024-06-10T02:54:43.514 INFO:teuthology.orchestra.run.smithi052.stdout: Birth: 2024-06-10 02:51:23.198023121 +0000 2024-06-10T02:54:43.514 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-06-10T02:54:43.514 DEBUG:teuthology.orchestra.run.smithi052:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-06-10T02:54:43.590 DEBUG:teuthology.orchestra.run.smithi052:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 7b7a5d375cf4e4afab96b9d4d229b699ae8304bb 2024-06-10T02:54:43.647 INFO:tasks.workunit.client.0.smithi052.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-06-10T02:54:43.765 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:54:44.877 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:54:44 smithi052 ceph-mon[32362]: pgmap v450: 129 pgs: 129 active+clean; 656 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-10T02:54:45.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:54:44 smithi120 ceph-mon[36634]: pgmap v450: 129 pgs: 129 active+clean; 656 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-10T02:54:45.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:54:44 smithi188 ceph-mon[36599]: pgmap v450: 129 pgs: 129 active+clean; 656 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-10T02:54:45.415 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:45.415+0000 7f0d8142d700 1 -- 172.21.15.52:0/565929426 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0d7c1046a0 msgr2=0x7f0d7c104b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:54:45.415 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:45.415+0000 7f0d8142d700 1 --2- 172.21.15.52:0/565929426 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0d7c1046a0 0x7f0d7c104b20 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f0d70009990 tx=0x7f0d70009ca0 comp rx=0 tx=0).stop 2024-06-10T02:54:45.415 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:45.415+0000 7f0d8142d700 1 -- 172.21.15.52:0/565929426 shutdown_connections 2024-06-10T02:54:45.416 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:45.415+0000 7f0d8142d700 1 --2- 172.21.15.52:0/565929426 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0d7c105060 0x7f0d7c111570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:45.416 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:45.416+0000 7f0d8142d700 1 --2- 172.21.15.52:0/565929426 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0d7c10a6a0 0x7f0d7c10aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:45.416 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:45.416+0000 7f0d8142d700 1 --2- 172.21.15.52:0/565929426 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0d7c1046a0 0x7f0d7c104b20 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:45.416 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:45.416+0000 7f0d8142d700 1 -- 172.21.15.52:0/565929426 >> 172.21.15.52:0/565929426 conn(0x7f0d7c100170 msgr2=0x7f0d7c102590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:54:45.416 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:45.416+0000 7f0d8142d700 1 -- 172.21.15.52:0/565929426 shutdown_connections 2024-06-10T02:54:45.416 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:45.416+0000 7f0d8142d700 1 -- 172.21.15.52:0/565929426 wait complete. 2024-06-10T02:54:45.416 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:45.416+0000 7f0d8142d700 1 Processor -- start 2024-06-10T02:54:45.418 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:45.417+0000 7f0d8142d700 1 -- start start 2024-06-10T02:54:45.418 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:45.417+0000 7f0d8142d700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0d7c1046a0 0x7f0d7c19ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:45.418 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:45.417+0000 7f0d8142d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0d7c105060 0x7f0d7c19b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:45.418 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:45.417+0000 7f0d8142d700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0d7c10a6a0 0x7f0d7c19f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:45.418 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:45.417+0000 7f0d8142d700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f0d7c114a30 con 0x7f0d7c105060 2024-06-10T02:54:45.418 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:45.417+0000 7f0d8142d700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f0d7c1148b0 con 0x7f0d7c10a6a0 2024-06-10T02:54:45.418 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:45.417+0000 7f0d8142d700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f0d7c114bb0 con 0x7f0d7c1046a0 2024-06-10T02:54:45.418 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:45.417+0000 7f0d7a7fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0d7c105060 0x7f0d7c19b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:45.418 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:45.417+0000 7f0d7a7fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0d7c105060 0x7f0d7c19b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51744/0 (socket says 172.21.15.52:51744) 2024-06-10T02:54:45.418 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:45.417+0000 7f0d7a7fc700 1 -- 172.21.15.52:0/3733933515 learned_addr learned my addr 172.21.15.52:0/3733933515 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:54:45.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:45.417+0000 7f0d7b7fe700 1 --2- 172.21.15.52:0/3733933515 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0d7c10a6a0 0x7f0d7c19f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:45.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:45.417+0000 7f0d7affd700 1 --2- 172.21.15.52:0/3733933515 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0d7c1046a0 0x7f0d7c19ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:45.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:45.417+0000 7f0d7a7fc700 1 -- 172.21.15.52:0/3733933515 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0d7c1046a0 msgr2=0x7f0d7c19ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:54:45.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:45.417+0000 7f0d7a7fc700 1 --2- 172.21.15.52:0/3733933515 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0d7c1046a0 0x7f0d7c19ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:45.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:45.417+0000 7f0d7a7fc700 1 -- 172.21.15.52:0/3733933515 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0d7c10a6a0 msgr2=0x7f0d7c19f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:54:45.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:45.417+0000 7f0d7a7fc700 1 --2- 172.21.15.52:0/3733933515 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0d7c10a6a0 0x7f0d7c19f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:45.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:45.417+0000 7f0d7a7fc700 1 -- 172.21.15.52:0/3733933515 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d7c19fe20 con 0x7f0d7c105060 2024-06-10T02:54:45.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:45.417+0000 7f0d7b7fe700 1 --2- 172.21.15.52:0/3733933515 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0d7c10a6a0 0x7f0d7c19f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T02:54:45.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:45.418+0000 7f0d7a7fc700 1 --2- 172.21.15.52:0/3733933515 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0d7c105060 0x7f0d7c19b3a0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f0d70000c00 tx=0x7f0d7000bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:54:45.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:45.418+0000 7f0d63fff700 1 -- 172.21.15.52:0/3733933515 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d70027070 con 0x7f0d7c105060 2024-06-10T02:54:45.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:45.418+0000 7f0d8142d700 1 -- 172.21.15.52:0/3733933515 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d7c1a00b0 con 0x7f0d7c105060 2024-06-10T02:54:45.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:45.418+0000 7f0d63fff700 1 -- 172.21.15.52:0/3733933515 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d70003d70 con 0x7f0d7c105060 2024-06-10T02:54:45.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:45.418+0000 7f0d63fff700 1 -- 172.21.15.52:0/3733933515 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d7000f460 con 0x7f0d7c105060 2024-06-10T02:54:45.420 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:45.418+0000 7f0d8142d700 1 -- 172.21.15.52:0/3733933515 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d7c1a7990 con 0x7f0d7c105060 2024-06-10T02:54:45.420 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:45.419+0000 7f0d8142d700 1 -- 172.21.15.52:0/3733933515 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d7c10cc30 con 0x7f0d7c105060 2024-06-10T02:54:45.420 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:45.420+0000 7f0d63fff700 1 -- 172.21.15.52:0/3733933515 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7f0d7002a070 con 0x7f0d7c105060 2024-06-10T02:54:45.421 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:45.421+0000 7f0d63fff700 1 --2- 172.21.15.52:0/3733933515 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f0d640a2cb0 0x7f0d640a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:45.421 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:45.421+0000 7f0d63fff700 1 -- 172.21.15.52:0/3733933515 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f0d700d0710 con 0x7f0d7c105060 2024-06-10T02:54:45.421 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:45.421+0000 7f0d7affd700 1 --2- 172.21.15.52:0/3733933515 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f0d640a2cb0 0x7f0d640a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:45.421 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:45.422+0000 7f0d7affd700 1 --2- 172.21.15.52:0/3733933515 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f0d640a2cb0 0x7f0d640a5170 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f0d6c009be0 tx=0x7f0d6c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:54:45.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:45.425+0000 7f0d63fff700 1 -- 172.21.15.52:0/3733933515 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f0d70013070 con 0x7f0d7c105060 2024-06-10T02:54:45.684 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:45.685+0000 7f0d8142d700 1 -- 172.21.15.52:0/3733933515 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0d7c069180 con 0x7f0d7c105060 2024-06-10T02:54:45.685 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:45.686+0000 7f0d63fff700 1 -- 172.21.15.52:0/3733933515 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11938 (secure 0 0 0) 0x7f0d7009d1f0 con 0x7f0d7c105060 2024-06-10T02:54:45.685 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-10T02:54:45.686 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/889673247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":889673247},{"type":"v1","addr":"172.21.15.52:6835","nonce":889673247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/3615663511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":3615663511},{"type":"v1","addr":"172.21.15.52:6837","nonce":3615663511}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T02:50:03.889766+0000","modified":"2024-06-10T02:54:07.086405+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24484,"mds_1":24469,"mds_2":24463,"mds_3":24449,"mds_4":24428},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14571":{"gid":14571,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6839/3059599042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":3059599042},{"type":"v1","addr":"172.21.15.52:6839","nonce":3059599042}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14583":{"gid":14583,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6841/3039275521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":3039275521},{"type":"v1","addr":"172.21.15.52:6841","nonce":3039275521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24383":{"gid":24383,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6833/2609002896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":2609002896},{"type":"v1","addr":"172.21.15.120:6833","nonce":2609002896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6835/1996370288","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":1996370288},{"type":"v1","addr":"172.21.15.120:6835","nonce":1996370288}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6833/529564979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":529564979},{"type":"v1","addr":"172.21.15.188:6833","nonce":529564979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":25,"addr":"172.21.15.120:6837/4132440952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":4132440952},{"type":"v1","addr":"172.21.15.120:6837","nonce":4132440952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"b","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.120:6839/1672436487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1672436487},{"type":"v1","addr":"172.21.15.120:6839","nonce":1672436487}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":2,"incarnation":22,"state":"up:active","state_seq":26,"addr":"172.21.15.188:6835/3904544259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":3904544259},{"type":"v1","addr":"172.21.15.188:6835","nonce":3904544259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.188:6837/819317726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":819317726},{"type":"v1","addr":"172.21.15.188:6837","nonce":819317726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.188:6839/310444836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":310444836},{"type":"v1","addr":"172.21.15.188:6839","nonce":310444836}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24484,"qdb_cluster":[24469,24484,24463,24428,24449]},"id":1}]} 2024-06-10T02:54:45.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:45.688+0000 7f0d8142d700 1 -- 172.21.15.52:0/3733933515 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f0d640a2cb0 msgr2=0x7f0d640a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:54:45.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:45.688+0000 7f0d8142d700 1 --2- 172.21.15.52:0/3733933515 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f0d640a2cb0 0x7f0d640a5170 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f0d6c009be0 tx=0x7f0d6c009260 comp rx=0 tx=0).stop 2024-06-10T02:54:45.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:45.688+0000 7f0d8142d700 1 -- 172.21.15.52:0/3733933515 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0d7c105060 msgr2=0x7f0d7c19b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:54:45.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:45.688+0000 7f0d8142d700 1 --2- 172.21.15.52:0/3733933515 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0d7c105060 0x7f0d7c19b3a0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f0d70000c00 tx=0x7f0d7000bab0 comp rx=0 tx=0).stop 2024-06-10T02:54:45.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:45.688+0000 7f0d8142d700 1 -- 172.21.15.52:0/3733933515 shutdown_connections 2024-06-10T02:54:45.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:45.688+0000 7f0d8142d700 1 --2- 172.21.15.52:0/3733933515 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0d7c10a6a0 0x7f0d7c19f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:45.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:45.688+0000 7f0d8142d700 1 --2- 172.21.15.52:0/3733933515 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0d7c1046a0 0x7f0d7c19ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:45.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:45.688+0000 7f0d8142d700 1 --2- 172.21.15.52:0/3733933515 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f0d640a2cb0 0x7f0d640a5170 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:45.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:45.688+0000 7f0d8142d700 1 --2- 172.21.15.52:0/3733933515 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0d7c105060 0x7f0d7c19b3a0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:45.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:45.688+0000 7f0d8142d700 1 -- 172.21.15.52:0/3733933515 >> 172.21.15.52:0/3733933515 conn(0x7f0d7c100170 msgr2=0x7f0d7c100770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:54:45.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:45.688+0000 7f0d8142d700 1 -- 172.21.15.52:0/3733933515 shutdown_connections 2024-06-10T02:54:45.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:45.689+0000 7f0d8142d700 1 -- 172.21.15.52:0/3733933515 wait complete. 2024-06-10T02:54:45.689 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 30 2024-06-10T02:54:45.976 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:54:45 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T02:54:45.976 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:54:45 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:54:45.976 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:54:45 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T02:54:45.976 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:54:45 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:54:45.977 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:54:45 smithi052 ceph-mon[32362]: from='client.? 172.21.15.52:0/3733933515' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T02:54:46.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:54:45 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T02:54:46.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:54:45 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:54:46.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:54:45 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T02:54:46.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:54:45 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:54:46.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:54:45 smithi120 ceph-mon[36634]: from='client.? 172.21.15.52:0/3733933515' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T02:54:46.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:54:45 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T02:54:46.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:54:45 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:54:46.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:54:45 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T02:54:46.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:54:45 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:54:46.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:54:45 smithi188 ceph-mon[36599]: from='client.? 172.21.15.52:0/3733933515' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T02:54:46.427 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 30, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-06-10T02:50:03.889766+0000', 'modified': '2024-06-10T02:54:07.086405+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24484, 'mds_1': 24469, 'mds_2': 24463, 'mds_3': 24449, 'mds_4': 24428}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14571': {'gid': 14571, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.52:6839/3059599042', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.52:6838', 'nonce': 3059599042}, {'type': 'v1', 'addr': '172.21.15.52:6839', 'nonce': 3059599042}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14583': {'gid': 14583, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.52:6841/3039275521', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.52:6840', 'nonce': 3039275521}, {'type': 'v1', 'addr': '172.21.15.52:6841', 'nonce': 3039275521}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24383': {'gid': 24383, 'name': 'k', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.120:6833/2609002896', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6832', 'nonce': 2609002896}, {'type': 'v1', 'addr': '172.21.15.120:6833', 'nonce': 2609002896}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24401': {'gid': 24401, 'name': 'e', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.120:6835/1996370288', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6834', 'nonce': 1996370288}, {'type': 'v1', 'addr': '172.21.15.120:6835', 'nonce': 1996370288}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24427': {'gid': 24427, 'name': 'l', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.188:6833/529564979', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6832', 'nonce': 529564979}, {'type': 'v1', 'addr': '172.21.15.188:6833', 'nonce': 529564979}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24428': {'gid': 24428, 'name': 'h', 'rank': 4, 'incarnation': 26, 'state': 'up:active', 'state_seq': 25, 'addr': '172.21.15.120:6837/4132440952', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6836', 'nonce': 4132440952}, {'type': 'v1', 'addr': '172.21.15.120:6837', 'nonce': 4132440952}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24449': {'gid': 24449, 'name': 'b', 'rank': 3, 'incarnation': 24, 'state': 'up:active', 'state_seq': 22, 'addr': '172.21.15.120:6839/1672436487', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6838', 'nonce': 1672436487}, {'type': 'v1', 'addr': '172.21.15.120:6839', 'nonce': 1672436487}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24463': {'gid': 24463, 'name': 'f', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 26, 'addr': '172.21.15.188:6835/3904544259', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6834', 'nonce': 3904544259}, {'type': 'v1', 'addr': '172.21.15.188:6835', 'nonce': 3904544259}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24469': {'gid': 24469, 'name': 'c', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 25, 'addr': '172.21.15.188:6837/819317726', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6836', 'nonce': 819317726}, {'type': 'v1', 'addr': '172.21.15.188:6837', 'nonce': 819317726}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24484': {'gid': 24484, 'name': 'i', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.188:6839/310444836', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6838', 'nonce': 310444836}, {'type': 'v1', 'addr': '172.21.15.188:6839', 'nonce': 310444836}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24484, 'qdb_cluster': [24469, 24484, 24463, 24428, 24449]} 2024-06-10T02:54:46.428 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-06-10T02:54:46.428 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph tell mds.h status 2024-06-10T02:54:46.720 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:54:47.083 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:54:46 smithi052 ceph-mon[32362]: pgmap v451: 129 pgs: 129 active+clean; 656 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-10T02:54:47.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:54:46 smithi120 ceph-mon[36634]: pgmap v451: 129 pgs: 129 active+clean; 656 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-10T02:54:47.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:54:46 smithi188 ceph-mon[36599]: pgmap v451: 129 pgs: 129 active+clean; 656 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-10T02:54:48.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.376+0000 7ff726947700 1 -- 172.21.15.52:0/1362029263 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff72010f240 msgr2=0x7ff720111630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:54:48.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.376+0000 7ff726947700 1 --2- 172.21.15.52:0/1362029263 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff72010f240 0x7ff720111630 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7ff71800b0a0 tx=0x7ff71800b3b0 comp rx=0 tx=0).stop 2024-06-10T02:54:48.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.376+0000 7ff726947700 1 -- 172.21.15.52:0/1362029263 shutdown_connections 2024-06-10T02:54:48.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.376+0000 7ff726947700 1 --2- 172.21.15.52:0/1362029263 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff72010f240 0x7ff720111630 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:48.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.376+0000 7ff726947700 1 --2- 172.21.15.52:0/1362029263 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff72006ac40 0x7ff72006b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:48.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.376+0000 7ff726947700 1 --2- 172.21.15.52:0/1362029263 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff72006b560 0x7ff72010ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:48.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.376+0000 7ff726947700 1 -- 172.21.15.52:0/1362029263 >> 172.21.15.52:0/1362029263 conn(0x7ff720078190 msgr2=0x7ff7200785a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:54:48.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.376+0000 7ff726947700 1 -- 172.21.15.52:0/1362029263 shutdown_connections 2024-06-10T02:54:48.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.376+0000 7ff726947700 1 -- 172.21.15.52:0/1362029263 wait complete. 2024-06-10T02:54:48.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.377+0000 7ff726947700 1 Processor -- start 2024-06-10T02:54:48.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.377+0000 7ff726947700 1 -- start start 2024-06-10T02:54:48.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.377+0000 7ff726947700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff72006ac40 0x7ff72019af70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:48.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.377+0000 7ff726947700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff72006b560 0x7ff72019b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:48.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.377+0000 7ff726947700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff72010f240 0x7ff72019f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:48.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.377+0000 7ff726947700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ff720114a80 con 0x7ff72006ac40 2024-06-10T02:54:48.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.377+0000 7ff726947700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff720114900 con 0x7ff72010f240 2024-06-10T02:54:48.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.377+0000 7ff726947700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ff720114c00 con 0x7ff72006b560 2024-06-10T02:54:48.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.377+0000 7ff71ffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff72006ac40 0x7ff72019af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:48.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.377+0000 7ff71ffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff72006ac40 0x7ff72019af70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51766/0 (socket says 172.21.15.52:51766) 2024-06-10T02:54:48.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.377+0000 7ff71ffff700 1 -- 172.21.15.52:0/3381313327 learned_addr learned my addr 172.21.15.52:0/3381313327 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:54:48.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.377+0000 7ff724ee4700 1 --2- 172.21.15.52:0/3381313327 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff72010f240 0x7ff72019f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:48.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.377+0000 7ff717fff700 1 --2- 172.21.15.52:0/3381313327 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff72006b560 0x7ff72019b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:48.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.378+0000 7ff71ffff700 1 -- 172.21.15.52:0/3381313327 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff72006b560 msgr2=0x7ff72019b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:54:48.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.378+0000 7ff71ffff700 1 --2- 172.21.15.52:0/3381313327 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff72006b560 0x7ff72019b4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:48.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.378+0000 7ff71ffff700 1 -- 172.21.15.52:0/3381313327 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff72010f240 msgr2=0x7ff72019f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:54:48.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.378+0000 7ff71ffff700 1 --2- 172.21.15.52:0/3381313327 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff72010f240 0x7ff72019f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:48.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.378+0000 7ff71ffff700 1 -- 172.21.15.52:0/3381313327 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff72019ff80 con 0x7ff72006ac40 2024-06-10T02:54:48.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.378+0000 7ff717fff700 1 --2- 172.21.15.52:0/3381313327 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff72006b560 0x7ff72019b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T02:54:48.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.378+0000 7ff71ffff700 1 --2- 172.21.15.52:0/3381313327 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff72006ac40 0x7ff72019af70 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7ff71000d980 tx=0x7ff71000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:54:48.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.378+0000 7ff71dffb700 1 -- 172.21.15.52:0/3381313327 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff710017030 con 0x7ff72006ac40 2024-06-10T02:54:48.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.378+0000 7ff726947700 1 -- 172.21.15.52:0/3381313327 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7201a0270 con 0x7ff72006ac40 2024-06-10T02:54:48.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.378+0000 7ff71dffb700 1 -- 172.21.15.52:0/3381313327 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff710004500 con 0x7ff72006ac40 2024-06-10T02:54:48.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.378+0000 7ff71dffb700 1 -- 172.21.15.52:0/3381313327 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff710003aa0 con 0x7ff72006ac40 2024-06-10T02:54:48.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.378+0000 7ff726947700 1 -- 172.21.15.52:0/3381313327 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7201a7af0 con 0x7ff72006ac40 2024-06-10T02:54:48.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.380+0000 7ff71dffb700 1 -- 172.21.15.52:0/3381313327 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7ff71001a070 con 0x7ff72006ac40 2024-06-10T02:54:48.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.381+0000 7ff71dffb700 1 --2- 172.21.15.52:0/3381313327 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7ff70c0a2cb0 0x7ff70c0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:48.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.381+0000 7ff717fff700 1 --2- 172.21.15.52:0/3381313327 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7ff70c0a2cb0 0x7ff70c0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:48.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.381+0000 7ff71dffb700 1 -- 172.21.15.52:0/3381313327 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7ff7100cef50 con 0x7ff72006ac40 2024-06-10T02:54:48.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.382+0000 7ff717fff700 1 --2- 172.21.15.52:0/3381313327 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7ff70c0a2cb0 0x7ff70c0a5170 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7ff708005e10 tx=0x7ff708005d60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:54:48.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.382+0000 7ff7311cab80 1 Processor -- start 2024-06-10T02:54:48.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.382+0000 7ff7311cab80 1 -- start start 2024-06-10T02:54:48.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.382+0000 7ff7311cab80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e2311ec5f0 0x55e2311ec9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:48.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.382+0000 7ff7311cab80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e2311ecf10 0x55e2311e6990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:48.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.382+0000 7ff7311cab80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e2311e6ed0 0x55e2311e72f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:48.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.382+0000 7ff7311cab80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55e2311387c0 con 0x55e2311ecf10 2024-06-10T02:54:48.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.382+0000 7ff7311cab80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55e2310e06d0 con 0x55e2311e6ed0 2024-06-10T02:54:48.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.382+0000 7ff7311cab80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55e231134780 con 0x55e2311ec5f0 2024-06-10T02:54:48.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.382+0000 7ff717fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e2311ecf10 0x55e2311e6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:48.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.382+0000 7ff717fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e2311ecf10 0x55e2311e6990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51776/0 (socket says 172.21.15.52:51776) 2024-06-10T02:54:48.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.382+0000 7ff717fff700 1 -- 172.21.15.52:0/3005092451 learned_addr learned my addr 172.21.15.52:0/3005092451 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:54:48.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.382+0000 7ff71ffff700 1 --2- 172.21.15.52:0/3005092451 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e2311ec5f0 0x55e2311ec9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:48.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.382+0000 7ff724ee4700 1 --2- 172.21.15.52:0/3005092451 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e2311e6ed0 0x55e2311e72f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:48.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.382+0000 7ff717fff700 1 -- 172.21.15.52:0/3005092451 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e2311ec5f0 msgr2=0x55e2311ec9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:54:48.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.382+0000 7ff717fff700 1 --2- 172.21.15.52:0/3005092451 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e2311ec5f0 0x55e2311ec9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:48.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.382+0000 7ff717fff700 1 -- 172.21.15.52:0/3005092451 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e2311e6ed0 msgr2=0x55e2311e72f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:54:48.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.382+0000 7ff717fff700 1 --2- 172.21.15.52:0/3005092451 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e2311e6ed0 0x55e2311e72f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:48.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.382+0000 7ff717fff700 1 -- 172.21.15.52:0/3005092451 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e2310d6f00 con 0x55e2311ecf10 2024-06-10T02:54:48.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.382+0000 7ff71ffff700 1 --2- 172.21.15.52:0/3005092451 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e2311ec5f0 0x55e2311ec9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T02:54:48.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.383+0000 7ff717fff700 1 --2- 172.21.15.52:0/3005092451 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e2311ecf10 0x55e2311e6990 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7ff7080044c0 tx=0x7ff708012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4170334854adf488 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:54:48.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.383+0000 7ff7167fc700 1 -- 172.21.15.52:0/3005092451 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff708003880 con 0x55e2311ecf10 2024-06-10T02:54:48.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.383+0000 7ff7167fc700 1 -- 172.21.15.52:0/3005092451 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff708024070 con 0x55e2311ecf10 2024-06-10T02:54:48.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.383+0000 7ff7167fc700 1 -- 172.21.15.52:0/3005092451 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff70801fe40 con 0x55e2311ecf10 2024-06-10T02:54:48.388 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.383+0000 7ff7311cab80 1 -- 172.21.15.52:0/3005092451 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e2311ecf10 msgr2=0x55e2311e6990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:54:48.388 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.383+0000 7ff7311cab80 1 --2- 172.21.15.52:0/3005092451 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e2311ecf10 0x55e2311e6990 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7ff7080044c0 tx=0x7ff708012000 comp rx=0 tx=0).stop 2024-06-10T02:54:48.388 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.384+0000 7ff7311cab80 1 -- 172.21.15.52:0/3005092451 shutdown_connections 2024-06-10T02:54:48.388 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.384+0000 7ff7311cab80 1 --2- 172.21.15.52:0/3005092451 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e2311e6ed0 0x55e2311e72f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:48.388 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.384+0000 7ff7311cab80 1 --2- 172.21.15.52:0/3005092451 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e2311ec5f0 0x55e2311ec9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:48.388 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.384+0000 7ff7311cab80 1 --2- 172.21.15.52:0/3005092451 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e2311ecf10 0x55e2311e6990 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:48.388 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.384+0000 7ff7311cab80 1 -- 172.21.15.52:0/3005092451 >> 172.21.15.52:0/3005092451 conn(0x55e231111c70 msgr2=0x55e2311e1800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:54:48.388 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.384+0000 7ff7311cab80 1 -- 172.21.15.52:0/3005092451 shutdown_connections 2024-06-10T02:54:48.388 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.384+0000 7ff7311cab80 1 -- 172.21.15.52:0/3005092451 wait complete. 2024-06-10T02:54:48.388 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.384+0000 7ff7311cab80 1 -- start start 2024-06-10T02:54:48.388 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.384+0000 7ff7311cab80 1 Processor -- start 2024-06-10T02:54:48.388 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.384+0000 7ff7311cab80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e2311e6ed0 0x55e2311f2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:48.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.385+0000 7ff7311cab80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e2311ec5f0 0x55e2311eff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:48.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.385+0000 7ff7311cab80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e2311ecf10 0x55e2311f0450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:48.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.385+0000 7ff7311cab80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55e2311f3980 con 0x55e2311ec5f0 2024-06-10T02:54:48.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.385+0000 7ff7311cab80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55e2310d8ba0 con 0x55e2311e6ed0 2024-06-10T02:54:48.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.385+0000 7ff7311cab80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55e2311f3b00 con 0x55e2311ecf10 2024-06-10T02:54:48.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.385+0000 7ff7311cab80 10 client.0 _ll_register_callbacks cb 0x55e2310d28d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T02:54:48.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.385+0000 7ff717fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e2311ec5f0 0x55e2311eff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:48.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.385+0000 7ff717fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e2311ec5f0 0x55e2311eff10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51786/0 (socket says 172.21.15.52:51786) 2024-06-10T02:54:48.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.385+0000 7ff717fff700 1 -- 172.21.15.52:0/183057082 learned_addr learned my addr 172.21.15.52:0/183057082 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:54:48.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.385+0000 7ff724ee4700 1 --2- 172.21.15.52:0/183057082 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e2311ecf10 0x55e2311f0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:48.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.385+0000 7ff71ffff700 1 --2- 172.21.15.52:0/183057082 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e2311e6ed0 0x55e2311f2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:48.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.385+0000 7ff717fff700 1 -- 172.21.15.52:0/183057082 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e2311ecf10 msgr2=0x55e2311f0450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:54:48.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.385+0000 7ff717fff700 1 --2- 172.21.15.52:0/183057082 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e2311ecf10 0x55e2311f0450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:48.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.385+0000 7ff717fff700 1 -- 172.21.15.52:0/183057082 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e2311e6ed0 msgr2=0x55e2311f2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:54:48.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.385+0000 7ff717fff700 1 --2- 172.21.15.52:0/183057082 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e2311e6ed0 0x55e2311f2c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:48.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.385+0000 7ff717fff700 1 -- 172.21.15.52:0/183057082 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e231087880 con 0x55e2311ec5f0 2024-06-10T02:54:48.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.385+0000 7ff717fff700 1 --2- 172.21.15.52:0/183057082 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e2311ec5f0 0x55e2311eff10 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7ff708005950 tx=0x7ff708004130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5652223cfb44ff2 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:54:48.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.385+0000 7ff6ff7fe700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-10T02:54:48.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.385+0000 7ff6ff7fe700 1 -- 172.21.15.52:0/183057082 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff708003680 con 0x55e2311ec5f0 2024-06-10T02:54:48.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.385+0000 7ff6ff7fe700 1 -- 172.21.15.52:0/183057082 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7ff708003820 con 0x55e2311ec5f0 2024-06-10T02:54:48.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.385+0000 7ff7311cab80 1 -- 172.21.15.52:0/183057082 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff708024070 con 0x55e2311ec5f0 2024-06-10T02:54:48.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.386+0000 7ff6ff7fe700 1 -- 172.21.15.52:0/183057082 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff70803d050 con 0x55e2311ec5f0 2024-06-10T02:54:48.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.386+0000 7ff7311cab80 10 client.15447 fetch_fsmap learned FSMap version 30 2024-06-10T02:54:48.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.386+0000 7ff7311cab80 1 -- 172.21.15.52:0/183057082 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e2311fe1e0 con 0x55e2311ec5f0 2024-06-10T02:54:48.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.386+0000 7ff6ff7fe700 1 -- 172.21.15.52:0/183057082 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff708024070 con 0x55e2311ec5f0 2024-06-10T02:54:48.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.386+0000 7ff6ff7fe700 1 -- 172.21.15.52:0/183057082 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7ff708032020 con 0x55e2311ec5f0 2024-06-10T02:54:48.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.386+0000 7ff7311cab80 10 client.15447 fetch_fsmap finished waiting for FSMap version 30 2024-06-10T02:54:48.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.386+0000 7ff7311cab80 10 client.15447 resolve_mds: resolved name 'h' to daemon mds.h 2024-06-10T02:54:48.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.386+0000 7ff7311cab80 20 client.15447 populate_metadata read hostname 'smithi052' 2024-06-10T02:54:48.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.386+0000 7ff7311cab80 1 --2- 172.21.15.52:0/183057082 >> [v2:172.21.15.120:6836/4132440952,v1:172.21.15.120:6837/4132440952] conn(0x55e231200560 0x55e2312019d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:48.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.386+0000 7ff7311cab80 4 client.15447 mds_command: new command op to 24428 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T02:54:48.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.386+0000 7ff7311cab80 1 -- 172.21.15.52:0/183057082 --> [v2:172.21.15.120:6836/4132440952,v1:172.21.15.120:6837/4132440952] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e231145cc0 con 0x55e231200560 2024-06-10T02:54:48.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.386+0000 7ff71ffff700 1 --2- 172.21.15.52:0/183057082 >> [v2:172.21.15.120:6836/4132440952,v1:172.21.15.120:6837/4132440952] conn(0x55e231200560 0x55e2312019d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:48.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.387+0000 7ff71ffff700 1 --2- 172.21.15.52:0/183057082 >> [v2:172.21.15.120:6836/4132440952,v1:172.21.15.120:6837/4132440952] conn(0x55e231200560 0x55e2312019d0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=afb7118978e72fad server_cookie=7288d3679fbcd025 in_seq=0 out_seq=0 2024-06-10T02:54:48.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.387+0000 7ff6ff7fe700 10 client.15447 ms_handle_connect on v2:172.21.15.120:6836/4132440952 2024-06-10T02:54:48.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.388+0000 7ff6ff7fe700 1 -- 172.21.15.52:0/183057082 <== mds.4 v2:172.21.15.120:6836/4132440952 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55e231145cc0 con 0x55e231200560 2024-06-10T02:54:48.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.388+0000 7ff6ff7fe700 10 client.15447 handle_command_reply: tid=0 2024-06-10T02:54:48.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.388+0000 7ff7311cab80 1 client.15447 shutdown 2024-06-10T02:54:48.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.388+0000 7ff7311cab80 1 -- 172.21.15.52:0/183057082 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e2311ec5f0 msgr2=0x55e2311eff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:54:48.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.388+0000 7ff7311cab80 1 --2- 172.21.15.52:0/183057082 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e2311ec5f0 0x55e2311eff10 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7ff708005950 tx=0x7ff708004130 comp rx=0 tx=0).stop 2024-06-10T02:54:48.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.388+0000 7ff7311cab80 1 -- 172.21.15.52:0/183057082 shutdown_connections 2024-06-10T02:54:48.392 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.388+0000 7ff7311cab80 1 --2- 172.21.15.52:0/183057082 >> [v2:172.21.15.120:6836/4132440952,v1:172.21.15.120:6837/4132440952] conn(0x55e231200560 0x55e2312019d0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:48.392 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.388+0000 7ff7311cab80 1 --2- 172.21.15.52:0/183057082 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e2311ecf10 0x55e2311f0450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:48.392 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.388+0000 7ff7311cab80 1 --2- 172.21.15.52:0/183057082 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e2311e6ed0 0x55e2311f2c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:48.392 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.388+0000 7ff7311cab80 1 --2- 172.21.15.52:0/183057082 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e2311ec5f0 0x55e2311eff10 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:48.392 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.388+0000 7ff7311cab80 1 -- 172.21.15.52:0/183057082 >> 172.21.15.52:0/183057082 conn(0x55e231111c70 msgr2=0x55e2311e15a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:54:48.392 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.388+0000 7ff6ff7fe700 0 client.15447 ms_handle_reset on v2:172.21.15.120:6836/4132440952 2024-06-10T02:54:48.392 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.388+0000 7ff7311cab80 1 -- 172.21.15.52:0/183057082 shutdown_connections 2024-06-10T02:54:48.392 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.388+0000 7ff7311cab80 1 -- 172.21.15.52:0/183057082 wait complete. 2024-06-10T02:54:48.392 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.388+0000 7ff7311cab80 20 client.15447 trim_cache size 0 max 16384 2024-06-10T02:54:48.410 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.409+0000 7ff7311cab80 1 Processor -- start 2024-06-10T02:54:48.410 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.409+0000 7ff7311cab80 1 -- start start 2024-06-10T02:54:48.410 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.410+0000 7ff7311cab80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e2311e6ed0 0x55e2311e6990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:48.410 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.410+0000 7ff7311cab80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e2311ec5f0 0x55e2311f0480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:48.410 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.410+0000 7ff7311cab80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e2311ecf10 0x55e2311ed330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:48.410 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.410+0000 7ff7311cab80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55e2311ffd10 con 0x55e2311ecf10 2024-06-10T02:54:48.410 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.410+0000 7ff7311cab80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55e2312003e0 con 0x55e2311e6ed0 2024-06-10T02:54:48.410 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.410+0000 7ff7311cab80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55e231200ab0 con 0x55e2311ec5f0 2024-06-10T02:54:48.410 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.410+0000 7ff724ee4700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e2311ecf10 0x55e2311ed330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:48.410 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.410+0000 7ff724ee4700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e2311ecf10 0x55e2311ed330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51792/0 (socket says 172.21.15.52:51792) 2024-06-10T02:54:48.411 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.410+0000 7ff717fff700 1 --2- 172.21.15.52:0/1817239552 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e2311ec5f0 0x55e2311f0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:48.411 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.410+0000 7ff724ee4700 1 -- 172.21.15.52:0/1817239552 learned_addr learned my addr 172.21.15.52:0/1817239552 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:54:48.411 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.410+0000 7ff71ffff700 1 --2- 172.21.15.52:0/1817239552 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e2311e6ed0 0x55e2311e6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:48.411 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.410+0000 7ff724ee4700 1 -- 172.21.15.52:0/1817239552 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e2311ec5f0 msgr2=0x55e2311f0480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:54:48.411 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.410+0000 7ff724ee4700 1 --2- 172.21.15.52:0/1817239552 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e2311ec5f0 0x55e2311f0480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:48.411 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.410+0000 7ff724ee4700 1 -- 172.21.15.52:0/1817239552 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e2311e6ed0 msgr2=0x55e2311e6990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:54:48.411 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.410+0000 7ff724ee4700 1 --2- 172.21.15.52:0/1817239552 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e2311e6ed0 0x55e2311e6990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:48.411 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.410+0000 7ff724ee4700 1 -- 172.21.15.52:0/1817239552 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e2311ee480 con 0x55e2311ecf10 2024-06-10T02:54:48.411 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.410+0000 7ff717fff700 1 --2- 172.21.15.52:0/1817239552 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e2311ec5f0 0x55e2311f0480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T02:54:48.411 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.410+0000 7ff724ee4700 1 --2- 172.21.15.52:0/1817239552 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e2311ecf10 0x55e2311ed330 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7ff718015000 tx=0x7ff718009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbc7b63379f9d331 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:54:48.411 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.411+0000 7ff715ffb700 1 -- 172.21.15.52:0/1817239552 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff718009c40 con 0x55e2311ecf10 2024-06-10T02:54:48.411 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.411+0000 7ff715ffb700 1 -- 172.21.15.52:0/1817239552 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff7180074e0 con 0x55e2311ecf10 2024-06-10T02:54:48.411 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.411+0000 7ff715ffb700 1 -- 172.21.15.52:0/1817239552 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7180052a0 con 0x55e2311ecf10 2024-06-10T02:54:48.415 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.412+0000 7ff7311cab80 1 -- 172.21.15.52:0/1817239552 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e2311ecf10 msgr2=0x55e2311ed330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:54:48.415 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.412+0000 7ff7311cab80 1 --2- 172.21.15.52:0/1817239552 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e2311ecf10 0x55e2311ed330 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7ff718015000 tx=0x7ff718009310 comp rx=0 tx=0).stop 2024-06-10T02:54:48.415 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.412+0000 7ff7311cab80 1 -- 172.21.15.52:0/1817239552 shutdown_connections 2024-06-10T02:54:48.415 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.412+0000 7ff7311cab80 1 --2- 172.21.15.52:0/1817239552 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e2311ecf10 0x55e2311ed330 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:48.415 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.412+0000 7ff7311cab80 1 --2- 172.21.15.52:0/1817239552 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e2311e6ed0 0x55e2311e6990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:48.415 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.412+0000 7ff7311cab80 1 --2- 172.21.15.52:0/1817239552 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e2311ec5f0 0x55e2311f0480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:48.415 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.412+0000 7ff7311cab80 1 -- 172.21.15.52:0/1817239552 >> 172.21.15.52:0/1817239552 conn(0x55e231111c70 msgr2=0x55e2311ee800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:54:48.415 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.412+0000 7ff7311cab80 1 -- 172.21.15.52:0/1817239552 shutdown_connections 2024-06-10T02:54:48.416 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.412+0000 7ff7311cab80 1 -- 172.21.15.52:0/1817239552 wait complete. 2024-06-10T02:54:48.416 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.412+0000 7ff7311cab80 1 -- start start 2024-06-10T02:54:48.416 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.412+0000 7ff7311cab80 1 Processor -- start 2024-06-10T02:54:48.416 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.413+0000 7ff7311cab80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e2311e6ed0 0x55e23120b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:48.416 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.413+0000 7ff7311cab80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e2311ec5f0 0x55e2311ef530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:48.416 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.413+0000 7ff7311cab80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e2311ecf10 0x55e2311efa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:48.416 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.413+0000 7ff7311cab80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55e231201850 con 0x55e2311e6ed0 2024-06-10T02:54:48.416 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.413+0000 7ff7311cab80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55e231201180 con 0x55e2311ec5f0 2024-06-10T02:54:48.416 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.413+0000 7ff7311cab80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55e231201f20 con 0x55e2311ecf10 2024-06-10T02:54:48.416 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.413+0000 7ff71ffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e2311e6ed0 0x55e23120b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:48.416 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.413+0000 7ff7311cab80 10 client.0 _ll_register_callbacks cb 0x55e2310d28d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T02:54:48.416 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.413+0000 7ff71ffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e2311e6ed0 0x55e23120b5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51800/0 (socket says 172.21.15.52:51800) 2024-06-10T02:54:48.416 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.413+0000 7ff71ffff700 1 -- 172.21.15.52:0/3566725325 learned_addr learned my addr 172.21.15.52:0/3566725325 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:54:48.416 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.413+0000 7ff717fff700 1 --2- 172.21.15.52:0/3566725325 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e2311ec5f0 0x55e2311ef530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:48.417 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.413+0000 7ff724ee4700 1 --2- 172.21.15.52:0/3566725325 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e2311ecf10 0x55e2311efa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:48.417 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.413+0000 7ff71ffff700 1 -- 172.21.15.52:0/3566725325 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e2311ecf10 msgr2=0x55e2311efa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:54:48.417 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.413+0000 7ff71ffff700 1 --2- 172.21.15.52:0/3566725325 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e2311ecf10 0x55e2311efa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:48.417 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.413+0000 7ff71ffff700 1 -- 172.21.15.52:0/3566725325 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e2311ec5f0 msgr2=0x55e2311ef530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:54:48.417 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.413+0000 7ff71ffff700 1 --2- 172.21.15.52:0/3566725325 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e2311ec5f0 0x55e2311ef530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:48.417 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.413+0000 7ff71ffff700 1 -- 172.21.15.52:0/3566725325 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e2311f02d0 con 0x55e2311e6ed0 2024-06-10T02:54:48.417 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.413+0000 7ff724ee4700 1 --2- 172.21.15.52:0/3566725325 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e2311ecf10 0x55e2311efa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T02:54:48.417 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.413+0000 7ff71ffff700 1 --2- 172.21.15.52:0/3566725325 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e2311e6ed0 0x55e23120b5f0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7ff710097c10 tx=0x7ff7100bb040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5c2aa47276b5bbf server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:54:48.417 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.413+0000 7ff6ff7fe700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-10T02:54:48.417 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.414+0000 7ff6ff7fe700 1 -- 172.21.15.52:0/3566725325 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff71009f5f0 con 0x55e2311e6ed0 2024-06-10T02:54:48.417 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.414+0000 7ff6ff7fe700 1 -- 172.21.15.52:0/3566725325 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7ff710009a80 con 0x55e2311e6ed0 2024-06-10T02:54:48.417 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.414+0000 7ff7311cab80 1 -- 172.21.15.52:0/3566725325 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff7180074e0 con 0x55e2311e6ed0 2024-06-10T02:54:48.417 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.414+0000 7ff6ff7fe700 1 -- 172.21.15.52:0/3566725325 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff7100a3eb0 con 0x55e2311e6ed0 2024-06-10T02:54:48.418 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.414+0000 7ff7311cab80 10 client.15453 fetch_fsmap learned FSMap version 30 2024-06-10T02:54:48.418 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.414+0000 7ff7311cab80 1 -- 172.21.15.52:0/3566725325 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e23126be50 con 0x55e2311e6ed0 2024-06-10T02:54:48.418 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.414+0000 7ff6ff7fe700 1 -- 172.21.15.52:0/3566725325 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff7180074e0 con 0x55e2311e6ed0 2024-06-10T02:54:48.418 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.414+0000 7ff6ff7fe700 1 -- 172.21.15.52:0/3566725325 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7ff710024060 con 0x55e2311e6ed0 2024-06-10T02:54:48.418 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.414+0000 7ff7311cab80 10 client.15453 fetch_fsmap finished waiting for FSMap version 30 2024-06-10T02:54:48.418 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.414+0000 7ff7311cab80 10 client.15453 resolve_mds: resolved name 'h' to daemon mds.h 2024-06-10T02:54:48.418 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.414+0000 7ff7311cab80 20 client.15453 populate_metadata read hostname 'smithi052' 2024-06-10T02:54:48.418 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.414+0000 7ff7311cab80 1 --2- 172.21.15.52:0/3566725325 >> [v2:172.21.15.120:6836/4132440952,v1:172.21.15.120:6837/4132440952] conn(0x55e231270a10 0x55e23126f720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:48.418 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.414+0000 7ff7311cab80 4 client.15453 mds_command: new command op to 24428 tid=0 multi_id=0 [{"prefix": "status"}] 2024-06-10T02:54:48.418 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.414+0000 7ff7311cab80 1 -- 172.21.15.52:0/3566725325 --> [v2:172.21.15.120:6836/4132440952,v1:172.21.15.120:6837/4132440952] -- command(tid 0: {"prefix": "status"}) -- 0x55e231140240 con 0x55e231270a10 2024-06-10T02:54:48.418 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.415+0000 7ff717fff700 1 --2- 172.21.15.52:0/3566725325 >> [v2:172.21.15.120:6836/4132440952,v1:172.21.15.120:6837/4132440952] conn(0x55e231270a10 0x55e23126f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:48.418 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.415+0000 7ff6ff7fe700 10 client.15453 ms_handle_connect on v2:172.21.15.120:6836/4132440952 2024-06-10T02:54:48.418 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.415+0000 7ff717fff700 1 --2- 172.21.15.52:0/3566725325 >> [v2:172.21.15.120:6836/4132440952,v1:172.21.15.120:6837/4132440952] conn(0x55e231270a10 0x55e23126f720 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=7e92be552dd28f4b server_cookie=d4eb4eb2f54dc3a2 in_seq=0 out_seq=0 2024-06-10T02:54:48.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.415+0000 7ff6ff7fe700 1 -- 172.21.15.52:0/3566725325 <== mds.4 v2:172.21.15.120:6836/4132440952 1 ==== command_reply(tid 0: 0 ) ==== 8+0+334 (crc 0 0 0) 0x55e231140240 con 0x55e231270a10 2024-06-10T02:54:48.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.415+0000 7ff6ff7fe700 10 client.15453 handle_command_reply: tid=0 2024-06-10T02:54:48.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.415+0000 7ff7311cab80 1 client.15453 shutdown 2024-06-10T02:54:48.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.416+0000 7ff7311cab80 1 -- 172.21.15.52:0/3566725325 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e2311e6ed0 msgr2=0x55e23120b5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:54:48.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.416+0000 7ff7311cab80 1 --2- 172.21.15.52:0/3566725325 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e2311e6ed0 0x55e23120b5f0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7ff710097c10 tx=0x7ff7100bb040 comp rx=0 tx=0).stop 2024-06-10T02:54:48.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.416+0000 7ff7311cab80 1 -- 172.21.15.52:0/3566725325 shutdown_connections 2024-06-10T02:54:48.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.416+0000 7ff7311cab80 1 --2- 172.21.15.52:0/3566725325 >> [v2:172.21.15.120:6836/4132440952,v1:172.21.15.120:6837/4132440952] conn(0x55e231270a10 0x55e23126f720 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:48.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.416+0000 7ff6ff7fe700 0 client.15453 ms_handle_reset on v2:172.21.15.120:6836/4132440952 2024-06-10T02:54:48.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.416+0000 7ff7311cab80 1 --2- 172.21.15.52:0/3566725325 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e2311ecf10 0x55e2311efa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:48.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.416+0000 7ff7311cab80 1 --2- 172.21.15.52:0/3566725325 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e2311e6ed0 0x55e23120b5f0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:48.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.416+0000 7ff7311cab80 1 --2- 172.21.15.52:0/3566725325 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e2311ec5f0 0x55e2311ef530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:48.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.416+0000 7ff7311cab80 1 -- 172.21.15.52:0/3566725325 >> 172.21.15.52:0/3566725325 conn(0x55e231111c70 msgr2=0x55e2311ed330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:54:48.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.416+0000 7ff7311cab80 1 -- 172.21.15.52:0/3566725325 shutdown_connections 2024-06-10T02:54:48.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.416+0000 7ff7311cab80 1 -- 172.21.15.52:0/3566725325 wait complete. 2024-06-10T02:54:48.420 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.416+0000 7ff7311cab80 20 client.15453 trim_cache size 0 max 16384 2024-06-10T02:54:48.420 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-10T02:54:48.420 INFO:teuthology.orchestra.run.smithi052.stdout: "cluster_fsid": "d8676e9e-26d2-11ef-bc9b-c7b262605968", 2024-06-10T02:54:48.420 INFO:teuthology.orchestra.run.smithi052.stdout: "whoami": 4, 2024-06-10T02:54:48.420 INFO:teuthology.orchestra.run.smithi052.stdout: "id": 24428, 2024-06-10T02:54:48.420 INFO:teuthology.orchestra.run.smithi052.stdout: "want_state": "up:active", 2024-06-10T02:54:48.420 INFO:teuthology.orchestra.run.smithi052.stdout: "state": "up:active", 2024-06-10T02:54:48.420 INFO:teuthology.orchestra.run.smithi052.stdout: "fs_name": "cephfs", 2024-06-10T02:54:48.420 INFO:teuthology.orchestra.run.smithi052.stdout: "rank_uptime": 219.79766036300001, 2024-06-10T02:54:48.420 INFO:teuthology.orchestra.run.smithi052.stdout: "mdsmap_epoch": 30, 2024-06-10T02:54:48.421 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_epoch": 100, 2024-06-10T02:54:48.421 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_epoch_barrier": 100, 2024-06-10T02:54:48.421 INFO:teuthology.orchestra.run.smithi052.stdout: "uptime": 315.06374955299998 2024-06-10T02:54:48.421 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-10T02:54:48.421 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.416+0000 7ff726947700 1 -- 172.21.15.52:0/3381313327 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7ff70c0a2cb0 msgr2=0x7ff70c0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:54:48.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.416+0000 7ff726947700 1 --2- 172.21.15.52:0/3381313327 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7ff70c0a2cb0 0x7ff70c0a5170 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7ff708005e10 tx=0x7ff708005d60 comp rx=0 tx=0).stop 2024-06-10T02:54:48.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.416+0000 7ff726947700 1 -- 172.21.15.52:0/3381313327 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff72006ac40 msgr2=0x7ff72019af70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:54:48.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.416+0000 7ff726947700 1 --2- 172.21.15.52:0/3381313327 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff72006ac40 0x7ff72019af70 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7ff71000d980 tx=0x7ff71000dc90 comp rx=0 tx=0).stop 2024-06-10T02:54:48.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.417+0000 7ff726947700 1 -- 172.21.15.52:0/3381313327 shutdown_connections 2024-06-10T02:54:48.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.417+0000 7ff726947700 1 --2- 172.21.15.52:0/3381313327 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff72010f240 0x7ff72019f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:48.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.417+0000 7ff726947700 1 --2- 172.21.15.52:0/3381313327 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7ff70c0a2cb0 0x7ff70c0a5170 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:48.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.417+0000 7ff726947700 1 --2- 172.21.15.52:0/3381313327 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff72006ac40 0x7ff72019af70 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:48.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.417+0000 7ff726947700 1 --2- 172.21.15.52:0/3381313327 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff72006b560 0x7ff72019b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:48.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.417+0000 7ff726947700 1 -- 172.21.15.52:0/3381313327 >> 172.21.15.52:0/3381313327 conn(0x7ff720078190 msgr2=0x7ff7201106f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:54:48.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.417+0000 7ff726947700 1 -- 172.21.15.52:0/3381313327 shutdown_connections 2024-06-10T02:54:48.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:48.417+0000 7ff726947700 1 -- 172.21.15.52:0/3381313327 wait complete. 2024-06-10T02:54:49.107 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:54:48 smithi052 ceph-mon[32362]: pgmap v452: 129 pgs: 129 active+clean; 661 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 853 B/s wr, 12 op/s 2024-06-10T02:54:49.109 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph tell mds.b status 2024-06-10T02:54:49.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:54:48 smithi120 ceph-mon[36634]: pgmap v452: 129 pgs: 129 active+clean; 661 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 853 B/s wr, 12 op/s 2024-06-10T02:54:49.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:54:48 smithi188 ceph-mon[36599]: pgmap v452: 129 pgs: 129 active+clean; 661 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 853 B/s wr, 12 op/s 2024-06-10T02:54:49.360 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:54:51.110 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:54:50 smithi052 ceph-mon[32362]: pgmap v453: 129 pgs: 129 active+clean; 661 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 853 B/s wr, 9 op/s 2024-06-10T02:54:51.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.126+0000 7f4b88ca7700 1 -- 172.21.15.52:0/884260026 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4b841066b0 msgr2=0x7f4b8410e280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:54:51.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.126+0000 7f4b88ca7700 1 --2- 172.21.15.52:0/884260026 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4b841066b0 0x7f4b8410e280 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f4b7c00b0d0 tx=0x7f4b7c00b3e0 comp rx=0 tx=0).stop 2024-06-10T02:54:51.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.126+0000 7f4b88ca7700 1 -- 172.21.15.52:0/884260026 shutdown_connections 2024-06-10T02:54:51.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.126+0000 7f4b88ca7700 1 --2- 172.21.15.52:0/884260026 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4b841066b0 0x7f4b8410e280 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:51.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.126+0000 7f4b88ca7700 1 --2- 172.21.15.52:0/884260026 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4b84101620 0x7f4b84101a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:51.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.126+0000 7f4b88ca7700 1 --2- 172.21.15.52:0/884260026 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4b84101fd0 0x7f4b84105fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:51.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.126+0000 7f4b88ca7700 1 -- 172.21.15.52:0/884260026 >> 172.21.15.52:0/884260026 conn(0x7f4b84078340 msgr2=0x7f4b84078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:54:51.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.126+0000 7f4b88ca7700 1 -- 172.21.15.52:0/884260026 shutdown_connections 2024-06-10T02:54:51.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.126+0000 7f4b88ca7700 1 -- 172.21.15.52:0/884260026 wait complete. 2024-06-10T02:54:51.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.127+0000 7f4b88ca7700 1 Processor -- start 2024-06-10T02:54:51.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.127+0000 7f4b88ca7700 1 -- start start 2024-06-10T02:54:51.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.128+0000 7f4b88ca7700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4b84101620 0x7f4b84103070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:51.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.128+0000 7f4b88ca7700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4b84101fd0 0x7f4b841035b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:51.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.128+0000 7f4b88ca7700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4b841066b0 0x7f4b84103af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:51.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.128+0000 7f4b88ca7700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f4b841116f0 con 0x7f4b841066b0 2024-06-10T02:54:51.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.128+0000 7f4b88ca7700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f4b84111570 con 0x7f4b84101fd0 2024-06-10T02:54:51.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.128+0000 7f4b88ca7700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f4b84111870 con 0x7f4b84101620 2024-06-10T02:54:51.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.128+0000 7f4b82d9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4b841066b0 0x7f4b84103af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:51.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.128+0000 7f4b8259c700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4b84101620 0x7f4b84103070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:51.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.128+0000 7f4b82d9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4b841066b0 0x7f4b84103af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48394/0 (socket says 172.21.15.52:48394) 2024-06-10T02:54:51.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.128+0000 7f4b82d9d700 1 -- 172.21.15.52:0/631070805 learned_addr learned my addr 172.21.15.52:0/631070805 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:54:51.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.128+0000 7f4b81d9b700 1 --2- 172.21.15.52:0/631070805 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4b84101fd0 0x7f4b841035b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:51.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.128+0000 7f4b82d9d700 1 -- 172.21.15.52:0/631070805 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4b84101620 msgr2=0x7f4b84103070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:54:51.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.128+0000 7f4b82d9d700 1 --2- 172.21.15.52:0/631070805 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4b84101620 0x7f4b84103070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:51.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.128+0000 7f4b82d9d700 1 -- 172.21.15.52:0/631070805 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4b84101fd0 msgr2=0x7f4b841035b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:54:51.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.128+0000 7f4b82d9d700 1 --2- 172.21.15.52:0/631070805 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4b84101fd0 0x7f4b841035b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:51.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.128+0000 7f4b82d9d700 1 -- 172.21.15.52:0/631070805 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b841a70e0 con 0x7f4b841066b0 2024-06-10T02:54:51.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.128+0000 7f4b82d9d700 1 --2- 172.21.15.52:0/631070805 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4b841066b0 0x7f4b84103af0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f4b7c006070 tx=0x7f4b7c004d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:54:51.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.128+0000 7f4b737fe700 1 -- 172.21.15.52:0/631070805 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b7c021070 con 0x7f4b841066b0 2024-06-10T02:54:51.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.128+0000 7f4b88ca7700 1 -- 172.21.15.52:0/631070805 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b841a7370 con 0x7f4b841066b0 2024-06-10T02:54:51.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.128+0000 7f4b737fe700 1 -- 172.21.15.52:0/631070805 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b7c003c00 con 0x7f4b841066b0 2024-06-10T02:54:51.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.128+0000 7f4b737fe700 1 -- 172.21.15.52:0/631070805 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b7c01d450 con 0x7f4b841066b0 2024-06-10T02:54:51.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.129+0000 7f4b88ca7700 1 -- 172.21.15.52:0/631070805 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b841a7870 con 0x7f4b841066b0 2024-06-10T02:54:51.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.131+0000 7f4b737fe700 1 -- 172.21.15.52:0/631070805 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7f4b7c007980 con 0x7f4b841066b0 2024-06-10T02:54:51.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.131+0000 7f4b737fe700 1 --2- 172.21.15.52:0/631070805 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f4b6c0a2cb0 0x7f4b6c0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:51.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.132+0000 7f4b737fe700 1 -- 172.21.15.52:0/631070805 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f4b7c00f020 con 0x7f4b841066b0 2024-06-10T02:54:51.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.132+0000 7f4b8259c700 1 --2- 172.21.15.52:0/631070805 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f4b6c0a2cb0 0x7f4b6c0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:51.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.132+0000 7f4b8259c700 1 --2- 172.21.15.52:0/631070805 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f4b6c0a2cb0 0x7f4b6c0a5170 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f4b74005950 tx=0x7f4b7400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:54:51.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.132+0000 7f4b9352ab80 1 Processor -- start 2024-06-10T02:54:51.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.132+0000 7f4b9352ab80 1 -- start start 2024-06-10T02:54:51.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.133+0000 7f4b9352ab80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564f2b5695f0 0x564f2b5699d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:51.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.133+0000 7f4b9352ab80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564f2b569f10 0x564f2b563990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:51.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.133+0000 7f4b9352ab80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564f2b563ed0 0x564f2b5642f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:51.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.133+0000 7f4b9352ab80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x564f2b4b57c0 con 0x564f2b569f10 2024-06-10T02:54:51.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.133+0000 7f4b9352ab80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x564f2b45d6d0 con 0x564f2b563ed0 2024-06-10T02:54:51.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.133+0000 7f4b9352ab80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x564f2b4b1780 con 0x564f2b5695f0 2024-06-10T02:54:51.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.133+0000 7f4b81d9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564f2b569f10 0x564f2b563990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:51.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.133+0000 7f4b81d9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564f2b569f10 0x564f2b563990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48400/0 (socket says 172.21.15.52:48400) 2024-06-10T02:54:51.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.133+0000 7f4b81d9b700 1 -- 172.21.15.52:0/1053633383 learned_addr learned my addr 172.21.15.52:0/1053633383 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:54:51.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.133+0000 7f4b82d9d700 1 --2- 172.21.15.52:0/1053633383 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564f2b563ed0 0x564f2b5642f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:51.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.133+0000 7f4b8259c700 1 --2- 172.21.15.52:0/1053633383 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564f2b5695f0 0x564f2b5699d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:51.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.133+0000 7f4b81d9b700 1 -- 172.21.15.52:0/1053633383 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564f2b5695f0 msgr2=0x564f2b5699d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:54:51.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.133+0000 7f4b81d9b700 1 --2- 172.21.15.52:0/1053633383 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564f2b5695f0 0x564f2b5699d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:51.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.133+0000 7f4b81d9b700 1 -- 172.21.15.52:0/1053633383 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564f2b563ed0 msgr2=0x564f2b5642f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:54:51.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.133+0000 7f4b81d9b700 1 --2- 172.21.15.52:0/1053633383 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564f2b563ed0 0x564f2b5642f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:51.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.133+0000 7f4b81d9b700 1 -- 172.21.15.52:0/1053633383 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f2b453f00 con 0x564f2b569f10 2024-06-10T02:54:51.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.133+0000 7f4b81d9b700 1 --2- 172.21.15.52:0/1053633383 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564f2b569f10 0x564f2b563990 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f4b78008b40 tx=0x7f4b78008e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4e4d35a424edc58 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:54:51.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.134+0000 7f4b70ff9700 1 -- 172.21.15.52:0/1053633383 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b78014070 con 0x564f2b569f10 2024-06-10T02:54:51.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.134+0000 7f4b70ff9700 1 -- 172.21.15.52:0/1053633383 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b78004500 con 0x564f2b569f10 2024-06-10T02:54:51.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.134+0000 7f4b70ff9700 1 -- 172.21.15.52:0/1053633383 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b78010430 con 0x564f2b569f10 2024-06-10T02:54:51.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.134+0000 7f4b9352ab80 1 -- 172.21.15.52:0/1053633383 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564f2b569f10 msgr2=0x564f2b563990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:54:51.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.134+0000 7f4b9352ab80 1 --2- 172.21.15.52:0/1053633383 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564f2b569f10 0x564f2b563990 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f4b78008b40 tx=0x7f4b78008e50 comp rx=0 tx=0).stop 2024-06-10T02:54:51.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.134+0000 7f4b9352ab80 1 -- 172.21.15.52:0/1053633383 shutdown_connections 2024-06-10T02:54:51.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.134+0000 7f4b9352ab80 1 --2- 172.21.15.52:0/1053633383 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564f2b563ed0 0x564f2b5642f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:51.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.134+0000 7f4b9352ab80 1 --2- 172.21.15.52:0/1053633383 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564f2b5695f0 0x564f2b5699d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:51.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.134+0000 7f4b9352ab80 1 --2- 172.21.15.52:0/1053633383 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564f2b569f10 0x564f2b563990 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:51.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.134+0000 7f4b9352ab80 1 -- 172.21.15.52:0/1053633383 >> 172.21.15.52:0/1053633383 conn(0x564f2b48ec70 msgr2=0x564f2b55e800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:54:51.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.135+0000 7f4b9352ab80 1 -- 172.21.15.52:0/1053633383 shutdown_connections 2024-06-10T02:54:51.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.135+0000 7f4b9352ab80 1 -- 172.21.15.52:0/1053633383 wait complete. 2024-06-10T02:54:51.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.135+0000 7f4b9352ab80 1 -- start start 2024-06-10T02:54:51.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.135+0000 7f4b9352ab80 1 Processor -- start 2024-06-10T02:54:51.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.135+0000 7f4b9352ab80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564f2b563ed0 0x564f2b56fc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:51.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.135+0000 7f4b9352ab80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564f2b5695f0 0x564f2b56cf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:51.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.135+0000 7f4b9352ab80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564f2b569f10 0x564f2b56d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:51.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.135+0000 7f4b9352ab80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x564f2b570980 con 0x564f2b5695f0 2024-06-10T02:54:51.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.135+0000 7f4b9352ab80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x564f2b455ba0 con 0x564f2b563ed0 2024-06-10T02:54:51.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.135+0000 7f4b9352ab80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x564f2b570b00 con 0x564f2b569f10 2024-06-10T02:54:51.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.135+0000 7f4b9352ab80 10 client.0 _ll_register_callbacks cb 0x564f2b44f8d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T02:54:51.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.135+0000 7f4b81d9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564f2b5695f0 0x564f2b56cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:51.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.135+0000 7f4b81d9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564f2b5695f0 0x564f2b56cf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48402/0 (socket says 172.21.15.52:48402) 2024-06-10T02:54:51.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.135+0000 7f4b81d9b700 1 -- 172.21.15.52:0/2274467007 learned_addr learned my addr 172.21.15.52:0/2274467007 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:54:51.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.135+0000 7f4b8259c700 1 --2- 172.21.15.52:0/2274467007 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564f2b563ed0 0x564f2b56fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:51.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.136+0000 7f4b82d9d700 1 --2- 172.21.15.52:0/2274467007 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564f2b569f10 0x564f2b56d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:51.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.136+0000 7f4b81d9b700 1 -- 172.21.15.52:0/2274467007 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564f2b569f10 msgr2=0x564f2b56d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:54:51.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.136+0000 7f4b81d9b700 1 --2- 172.21.15.52:0/2274467007 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564f2b569f10 0x564f2b56d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:51.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.136+0000 7f4b81d9b700 1 -- 172.21.15.52:0/2274467007 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564f2b563ed0 msgr2=0x564f2b56fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:54:51.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.136+0000 7f4b81d9b700 1 --2- 172.21.15.52:0/2274467007 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564f2b563ed0 0x564f2b56fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:51.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.136+0000 7f4b81d9b700 1 -- 172.21.15.52:0/2274467007 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f2b404880 con 0x564f2b5695f0 2024-06-10T02:54:51.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.136+0000 7f4b81d9b700 1 --2- 172.21.15.52:0/2274467007 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564f2b5695f0 0x564f2b56cf60 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f4b78003c70 tx=0x7f4b78003ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd7ed181abe1ebbd server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:54:51.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.136+0000 7f4b69ffb700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-10T02:54:51.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.136+0000 7f4b69ffb700 1 -- 172.21.15.52:0/2274467007 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b78014030 con 0x564f2b5695f0 2024-06-10T02:54:51.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.136+0000 7f4b69ffb700 1 -- 172.21.15.52:0/2274467007 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f4b78004140 con 0x564f2b5695f0 2024-06-10T02:54:51.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.136+0000 7f4b9352ab80 1 -- 172.21.15.52:0/2274467007 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4b78004500 con 0x564f2b5695f0 2024-06-10T02:54:51.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.136+0000 7f4b69ffb700 1 -- 172.21.15.52:0/2274467007 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f4b78004340 con 0x564f2b5695f0 2024-06-10T02:54:51.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.136+0000 7f4b9352ab80 10 client.15465 fetch_fsmap learned FSMap version 30 2024-06-10T02:54:51.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.136+0000 7f4b9352ab80 1 -- 172.21.15.52:0/2274467007 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x564f2b57b260 con 0x564f2b5695f0 2024-06-10T02:54:51.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.137+0000 7f4b69ffb700 1 -- 172.21.15.52:0/2274467007 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b78004500 con 0x564f2b5695f0 2024-06-10T02:54:51.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.137+0000 7f4b69ffb700 1 -- 172.21.15.52:0/2274467007 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f4b780049b0 con 0x564f2b5695f0 2024-06-10T02:54:51.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.137+0000 7f4b9352ab80 10 client.15465 fetch_fsmap finished waiting for FSMap version 30 2024-06-10T02:54:51.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.137+0000 7f4b9352ab80 10 client.15465 resolve_mds: resolved name 'b' to daemon mds.b 2024-06-10T02:54:51.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.137+0000 7f4b9352ab80 20 client.15465 populate_metadata read hostname 'smithi052' 2024-06-10T02:54:51.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.137+0000 7f4b9352ab80 1 --2- 172.21.15.52:0/2274467007 >> [v2:172.21.15.120:6838/1672436487,v1:172.21.15.120:6839/1672436487] conn(0x564f2b57d660 0x564f2b57ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:51.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.137+0000 7f4b9352ab80 4 client.15465 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T02:54:51.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.137+0000 7f4b9352ab80 1 -- 172.21.15.52:0/2274467007 --> [v2:172.21.15.120:6838/1672436487,v1:172.21.15.120:6839/1672436487] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564f2b4c2cc0 con 0x564f2b57d660 2024-06-10T02:54:51.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.137+0000 7f4b8259c700 1 --2- 172.21.15.52:0/2274467007 >> [v2:172.21.15.120:6838/1672436487,v1:172.21.15.120:6839/1672436487] conn(0x564f2b57d660 0x564f2b57ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:51.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.138+0000 7f4b8259c700 1 --2- 172.21.15.52:0/2274467007 >> [v2:172.21.15.120:6838/1672436487,v1:172.21.15.120:6839/1672436487] conn(0x564f2b57d660 0x564f2b57ead0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d89e573f246e258b server_cookie=1dc059e5879acd02 in_seq=0 out_seq=0 2024-06-10T02:54:51.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.138+0000 7f4b69ffb700 10 client.15465 ms_handle_connect on v2:172.21.15.120:6838/1672436487 2024-06-10T02:54:51.141 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:54:50 smithi120 ceph-mon[36634]: pgmap v453: 129 pgs: 129 active+clean; 661 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 853 B/s wr, 9 op/s 2024-06-10T02:54:51.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.138+0000 7f4b69ffb700 1 -- 172.21.15.52:0/2274467007 <== mds.3 v2:172.21.15.120:6838/1672436487 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x564f2b4c2cc0 con 0x564f2b57d660 2024-06-10T02:54:51.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.138+0000 7f4b69ffb700 10 client.15465 handle_command_reply: tid=0 2024-06-10T02:54:51.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.138+0000 7f4b9352ab80 1 client.15465 shutdown 2024-06-10T02:54:51.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.138+0000 7f4b9352ab80 1 -- 172.21.15.52:0/2274467007 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564f2b5695f0 msgr2=0x564f2b56cf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:54:51.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.138+0000 7f4b9352ab80 1 --2- 172.21.15.52:0/2274467007 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564f2b5695f0 0x564f2b56cf60 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f4b78003c70 tx=0x7f4b78003ca0 comp rx=0 tx=0).stop 2024-06-10T02:54:51.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.138+0000 7f4b9352ab80 1 -- 172.21.15.52:0/2274467007 shutdown_connections 2024-06-10T02:54:51.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.138+0000 7f4b9352ab80 1 --2- 172.21.15.52:0/2274467007 >> [v2:172.21.15.120:6838/1672436487,v1:172.21.15.120:6839/1672436487] conn(0x564f2b57d660 0x564f2b57ead0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:51.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.138+0000 7f4b9352ab80 1 --2- 172.21.15.52:0/2274467007 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564f2b569f10 0x564f2b56d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:51.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.138+0000 7f4b9352ab80 1 --2- 172.21.15.52:0/2274467007 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564f2b563ed0 0x564f2b56fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:51.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.138+0000 7f4b69ffb700 0 client.15465 ms_handle_reset on v2:172.21.15.120:6838/1672436487 2024-06-10T02:54:51.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.138+0000 7f4b9352ab80 1 --2- 172.21.15.52:0/2274467007 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564f2b5695f0 0x564f2b56cf60 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:51.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.139+0000 7f4b9352ab80 1 -- 172.21.15.52:0/2274467007 >> 172.21.15.52:0/2274467007 conn(0x564f2b48ec70 msgr2=0x564f2b562980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:54:51.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.139+0000 7f4b9352ab80 1 -- 172.21.15.52:0/2274467007 shutdown_connections 2024-06-10T02:54:51.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.139+0000 7f4b9352ab80 1 -- 172.21.15.52:0/2274467007 wait complete. 2024-06-10T02:54:51.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.139+0000 7f4b9352ab80 20 client.15465 trim_cache size 0 max 16384 2024-06-10T02:54:51.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.160+0000 7f4b9352ab80 1 Processor -- start 2024-06-10T02:54:51.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.160+0000 7f4b9352ab80 1 -- start start 2024-06-10T02:54:51.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.160+0000 7f4b9352ab80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564f2b563ed0 0x564f2b563990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:51.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.160+0000 7f4b9352ab80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564f2b5695f0 0x564f2b56d510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:51.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.160+0000 7f4b9352ab80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564f2b569f10 0x564f2b587f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:51.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.160+0000 7f4b9352ab80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x564f2b57ce10 con 0x564f2b5695f0 2024-06-10T02:54:51.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.160+0000 7f4b9352ab80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x564f2b57d4e0 con 0x564f2b563ed0 2024-06-10T02:54:51.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.160+0000 7f4b9352ab80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x564f2b57dbb0 con 0x564f2b569f10 2024-06-10T02:54:51.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.160+0000 7f4b81d9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564f2b5695f0 0x564f2b56d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:51.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.160+0000 7f4b81d9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564f2b5695f0 0x564f2b56d510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48412/0 (socket says 172.21.15.52:48412) 2024-06-10T02:54:51.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.160+0000 7f4b81d9b700 1 -- 172.21.15.52:0/2437388036 learned_addr learned my addr 172.21.15.52:0/2437388036 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:54:51.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.161+0000 7f4b82d9d700 1 --2- 172.21.15.52:0/2437388036 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564f2b569f10 0x564f2b587f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:51.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.161+0000 7f4b81d9b700 1 -- 172.21.15.52:0/2437388036 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564f2b569f10 msgr2=0x564f2b587f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:54:51.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.161+0000 7f4b81d9b700 1 --2- 172.21.15.52:0/2437388036 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564f2b569f10 0x564f2b587f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:51.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.161+0000 7f4b81d9b700 1 -- 172.21.15.52:0/2437388036 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564f2b563ed0 msgr2=0x564f2b563990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:54:51.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.161+0000 7f4b81d9b700 1 --2- 172.21.15.52:0/2437388036 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564f2b563ed0 0x564f2b563990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:51.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.161+0000 7f4b81d9b700 1 -- 172.21.15.52:0/2437388036 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f2b56b510 con 0x564f2b5695f0 2024-06-10T02:54:51.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.161+0000 7f4b82d9d700 1 --2- 172.21.15.52:0/2437388036 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564f2b569f10 0x564f2b587f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T02:54:51.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.161+0000 7f4b81d9b700 1 --2- 172.21.15.52:0/2437388036 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564f2b5695f0 0x564f2b56d510 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f4b78004d20 tx=0x7f4b7800e630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c2f65ea26db3652 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:54:51.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.161+0000 7f4b6bfff700 1 -- 172.21.15.52:0/2437388036 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b78014040 con 0x564f2b5695f0 2024-06-10T02:54:51.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.161+0000 7f4b6bfff700 1 -- 172.21.15.52:0/2437388036 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b78004140 con 0x564f2b5695f0 2024-06-10T02:54:51.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.161+0000 7f4b6bfff700 1 -- 172.21.15.52:0/2437388036 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b780044e0 con 0x564f2b5695f0 2024-06-10T02:54:51.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.162+0000 7f4b9352ab80 1 -- 172.21.15.52:0/2437388036 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564f2b5695f0 msgr2=0x564f2b56d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:54:51.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.162+0000 7f4b9352ab80 1 --2- 172.21.15.52:0/2437388036 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564f2b5695f0 0x564f2b56d510 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f4b78004d20 tx=0x7f4b7800e630 comp rx=0 tx=0).stop 2024-06-10T02:54:51.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.162+0000 7f4b9352ab80 1 -- 172.21.15.52:0/2437388036 shutdown_connections 2024-06-10T02:54:51.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.162+0000 7f4b9352ab80 1 --2- 172.21.15.52:0/2437388036 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564f2b569f10 0x564f2b587f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:51.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.162+0000 7f4b9352ab80 1 --2- 172.21.15.52:0/2437388036 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564f2b563ed0 0x564f2b563990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:51.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.162+0000 7f4b9352ab80 1 --2- 172.21.15.52:0/2437388036 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564f2b5695f0 0x564f2b56d510 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:51.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.162+0000 7f4b9352ab80 1 -- 172.21.15.52:0/2437388036 >> 172.21.15.52:0/2437388036 conn(0x564f2b48ec70 msgr2=0x564f2b56b890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:54:51.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.162+0000 7f4b9352ab80 1 -- 172.21.15.52:0/2437388036 shutdown_connections 2024-06-10T02:54:51.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.162+0000 7f4b9352ab80 1 -- 172.21.15.52:0/2437388036 wait complete. 2024-06-10T02:54:51.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.162+0000 7f4b9352ab80 1 -- start start 2024-06-10T02:54:51.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.162+0000 7f4b9352ab80 1 Processor -- start 2024-06-10T02:54:51.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.162+0000 7f4b9352ab80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564f2b563ed0 0x564f2b56c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:51.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.162+0000 7f4b9352ab80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564f2b5695f0 0x564f2b56ce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:51.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.162+0000 7f4b9352ab80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564f2b569f10 0x564f2b56d360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:51.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.162+0000 7f4b9352ab80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x564f2b57e950 con 0x564f2b5695f0 2024-06-10T02:54:51.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.162+0000 7f4b9352ab80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x564f2b57e280 con 0x564f2b563ed0 2024-06-10T02:54:51.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.162+0000 7f4b9352ab80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x564f2b57f020 con 0x564f2b569f10 2024-06-10T02:54:51.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.162+0000 7f4b9352ab80 10 client.0 _ll_register_callbacks cb 0x564f2b44f8d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T02:54:51.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.163+0000 7f4b81d9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564f2b5695f0 0x564f2b56ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:51.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.163+0000 7f4b81d9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564f2b5695f0 0x564f2b56ce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48414/0 (socket says 172.21.15.52:48414) 2024-06-10T02:54:51.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.163+0000 7f4b81d9b700 1 -- 172.21.15.52:0/2954203890 learned_addr learned my addr 172.21.15.52:0/2954203890 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:54:51.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.163+0000 7f4b82d9d700 1 --2- 172.21.15.52:0/2954203890 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564f2b569f10 0x564f2b56d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:51.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.163+0000 7f4b8259c700 1 --2- 172.21.15.52:0/2954203890 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564f2b563ed0 0x564f2b56c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:51.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.163+0000 7f4b81d9b700 1 -- 172.21.15.52:0/2954203890 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564f2b569f10 msgr2=0x564f2b56d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:54:51.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.163+0000 7f4b81d9b700 1 --2- 172.21.15.52:0/2954203890 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564f2b569f10 0x564f2b56d360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:51.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.163+0000 7f4b81d9b700 1 -- 172.21.15.52:0/2954203890 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564f2b563ed0 msgr2=0x564f2b56c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:54:51.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.163+0000 7f4b81d9b700 1 --2- 172.21.15.52:0/2954203890 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564f2b563ed0 0x564f2b56c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:51.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.163+0000 7f4b81d9b700 1 -- 172.21.15.52:0/2954203890 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f2b5862e0 con 0x564f2b5695f0 2024-06-10T02:54:51.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.163+0000 7f4b82d9d700 1 --2- 172.21.15.52:0/2954203890 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564f2b569f10 0x564f2b56d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T02:54:51.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.163+0000 7f4b8259c700 1 --2- 172.21.15.52:0/2954203890 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564f2b563ed0 0x564f2b56c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T02:54:51.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.163+0000 7f4b81d9b700 1 --2- 172.21.15.52:0/2954203890 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564f2b5695f0 0x564f2b56ce20 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f4b78004310 tx=0x7f4b78004840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9484c25a5dd6e18c server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:54:51.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.163+0000 7f4b69ffb700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-10T02:54:51.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.163+0000 7f4b69ffb700 1 -- 172.21.15.52:0/2954203890 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b78004ae0 con 0x564f2b5695f0 2024-06-10T02:54:51.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.163+0000 7f4b69ffb700 1 -- 172.21.15.52:0/2954203890 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f4b7801f070 con 0x564f2b5695f0 2024-06-10T02:54:51.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.163+0000 7f4b9352ab80 1 -- 172.21.15.52:0/2954203890 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4b78004140 con 0x564f2b5695f0 2024-06-10T02:54:51.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.163+0000 7f4b69ffb700 1 -- 172.21.15.52:0/2954203890 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f4b78041050 con 0x564f2b5695f0 2024-06-10T02:54:51.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.163+0000 7f4b9352ab80 10 client.15471 fetch_fsmap learned FSMap version 30 2024-06-10T02:54:51.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.163+0000 7f4b9352ab80 1 -- 172.21.15.52:0/2954203890 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x564f2b5e90b0 con 0x564f2b5695f0 2024-06-10T02:54:51.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.164+0000 7f4b69ffb700 1 -- 172.21.15.52:0/2954203890 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b78004140 con 0x564f2b5695f0 2024-06-10T02:54:51.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.164+0000 7f4b69ffb700 1 -- 172.21.15.52:0/2954203890 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f4b78010a30 con 0x564f2b5695f0 2024-06-10T02:54:51.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.164+0000 7f4b9352ab80 10 client.15471 fetch_fsmap finished waiting for FSMap version 30 2024-06-10T02:54:51.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.164+0000 7f4b9352ab80 10 client.15471 resolve_mds: resolved name 'b' to daemon mds.b 2024-06-10T02:54:51.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.164+0000 7f4b9352ab80 20 client.15471 populate_metadata read hostname 'smithi052' 2024-06-10T02:54:51.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.164+0000 7f4b9352ab80 1 --2- 172.21.15.52:0/2954203890 >> [v2:172.21.15.120:6838/1672436487,v1:172.21.15.120:6839/1672436487] conn(0x564f2b5ed7a0 0x564f2b5ec4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:51.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.164+0000 7f4b9352ab80 4 client.15471 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "status"}] 2024-06-10T02:54:51.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.164+0000 7f4b9352ab80 1 -- 172.21.15.52:0/2954203890 --> [v2:172.21.15.120:6838/1672436487,v1:172.21.15.120:6839/1672436487] -- command(tid 0: {"prefix": "status"}) -- 0x564f2b4bd240 con 0x564f2b5ed7a0 2024-06-10T02:54:51.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.164+0000 7f4b8259c700 1 --2- 172.21.15.52:0/2954203890 >> [v2:172.21.15.120:6838/1672436487,v1:172.21.15.120:6839/1672436487] conn(0x564f2b5ed7a0 0x564f2b5ec4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:51.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.165+0000 7f4b8259c700 1 --2- 172.21.15.52:0/2954203890 >> [v2:172.21.15.120:6838/1672436487,v1:172.21.15.120:6839/1672436487] conn(0x564f2b5ed7a0 0x564f2b5ec4b0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b67f176449b7f8f0 server_cookie=8e57b08b84b2c860 in_seq=0 out_seq=0 2024-06-10T02:54:51.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.165+0000 7f4b69ffb700 10 client.15471 ms_handle_connect on v2:172.21.15.120:6838/1672436487 2024-06-10T02:54:51.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.165+0000 7f4b69ffb700 1 -- 172.21.15.52:0/2954203890 <== mds.3 v2:172.21.15.120:6838/1672436487 1 ==== command_reply(tid 0: 0 ) ==== 8+0+334 (crc 0 0 0) 0x564f2b4bd240 con 0x564f2b5ed7a0 2024-06-10T02:54:51.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.165+0000 7f4b69ffb700 10 client.15471 handle_command_reply: tid=0 2024-06-10T02:54:51.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.165+0000 7f4b9352ab80 1 client.15471 shutdown 2024-06-10T02:54:51.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.165+0000 7f4b9352ab80 1 -- 172.21.15.52:0/2954203890 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564f2b5695f0 msgr2=0x564f2b56ce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:54:51.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.165+0000 7f4b9352ab80 1 --2- 172.21.15.52:0/2954203890 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564f2b5695f0 0x564f2b56ce20 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f4b78004310 tx=0x7f4b78004840 comp rx=0 tx=0).stop 2024-06-10T02:54:51.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.165+0000 7f4b9352ab80 1 -- 172.21.15.52:0/2954203890 shutdown_connections 2024-06-10T02:54:51.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.165+0000 7f4b9352ab80 1 --2- 172.21.15.52:0/2954203890 >> [v2:172.21.15.120:6838/1672436487,v1:172.21.15.120:6839/1672436487] conn(0x564f2b5ed7a0 0x564f2b5ec4b0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:51.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.165+0000 7f4b9352ab80 1 --2- 172.21.15.52:0/2954203890 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564f2b569f10 0x564f2b56d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:51.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.165+0000 7f4b9352ab80 1 --2- 172.21.15.52:0/2954203890 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564f2b563ed0 0x564f2b56c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:51.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.165+0000 7f4b9352ab80 1 --2- 172.21.15.52:0/2954203890 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564f2b5695f0 0x564f2b56ce20 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:51.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.165+0000 7f4b9352ab80 1 -- 172.21.15.52:0/2954203890 >> 172.21.15.52:0/2954203890 conn(0x564f2b48ec70 msgr2=0x564f2b557db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:54:51.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.165+0000 7f4b69ffb700 0 client.15471 ms_handle_reset on v2:172.21.15.120:6838/1672436487 2024-06-10T02:54:51.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.165+0000 7f4b9352ab80 1 -- 172.21.15.52:0/2954203890 shutdown_connections 2024-06-10T02:54:51.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.165+0000 7f4b9352ab80 1 -- 172.21.15.52:0/2954203890 wait complete. 2024-06-10T02:54:51.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.165+0000 7f4b9352ab80 20 client.15471 trim_cache size 0 max 16384 2024-06-10T02:54:51.170 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-10T02:54:51.171 INFO:teuthology.orchestra.run.smithi052.stdout: "cluster_fsid": "d8676e9e-26d2-11ef-bc9b-c7b262605968", 2024-06-10T02:54:51.171 INFO:teuthology.orchestra.run.smithi052.stdout: "whoami": 3, 2024-06-10T02:54:51.171 INFO:teuthology.orchestra.run.smithi052.stdout: "id": 24449, 2024-06-10T02:54:51.171 INFO:teuthology.orchestra.run.smithi052.stdout: "want_state": "up:active", 2024-06-10T02:54:51.171 INFO:teuthology.orchestra.run.smithi052.stdout: "state": "up:active", 2024-06-10T02:54:51.171 INFO:teuthology.orchestra.run.smithi052.stdout: "fs_name": "cephfs", 2024-06-10T02:54:51.171 INFO:teuthology.orchestra.run.smithi052.stdout: "rank_uptime": 223.55649175100001, 2024-06-10T02:54:51.171 INFO:teuthology.orchestra.run.smithi052.stdout: "mdsmap_epoch": 30, 2024-06-10T02:54:51.171 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_epoch": 100, 2024-06-10T02:54:51.171 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_epoch_barrier": 100, 2024-06-10T02:54:51.171 INFO:teuthology.orchestra.run.smithi052.stdout: "uptime": 304.48013822399997 2024-06-10T02:54:51.171 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-10T02:54:51.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.166+0000 7f4b88ca7700 1 -- 172.21.15.52:0/631070805 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f4b6c0a2cb0 msgr2=0x7f4b6c0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:54:51.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.166+0000 7f4b88ca7700 1 --2- 172.21.15.52:0/631070805 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f4b6c0a2cb0 0x7f4b6c0a5170 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f4b74005950 tx=0x7f4b7400b410 comp rx=0 tx=0).stop 2024-06-10T02:54:51.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.166+0000 7f4b88ca7700 1 -- 172.21.15.52:0/631070805 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4b841066b0 msgr2=0x7f4b84103af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:54:51.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.166+0000 7f4b88ca7700 1 --2- 172.21.15.52:0/631070805 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4b841066b0 0x7f4b84103af0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f4b7c006070 tx=0x7f4b7c004d50 comp rx=0 tx=0).stop 2024-06-10T02:54:51.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.166+0000 7f4b88ca7700 1 -- 172.21.15.52:0/631070805 shutdown_connections 2024-06-10T02:54:51.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.166+0000 7f4b88ca7700 1 --2- 172.21.15.52:0/631070805 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f4b6c0a2cb0 0x7f4b6c0a5170 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:51.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.166+0000 7f4b88ca7700 1 --2- 172.21.15.52:0/631070805 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4b841066b0 0x7f4b84103af0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:51.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.166+0000 7f4b88ca7700 1 --2- 172.21.15.52:0/631070805 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4b84101620 0x7f4b84103070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:51.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.166+0000 7f4b88ca7700 1 --2- 172.21.15.52:0/631070805 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4b84101fd0 0x7f4b841035b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:51.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.166+0000 7f4b88ca7700 1 -- 172.21.15.52:0/631070805 >> 172.21.15.52:0/631070805 conn(0x7f4b84078340 msgr2=0x7f4b841051b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:54:51.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.166+0000 7f4b88ca7700 1 -- 172.21.15.52:0/631070805 shutdown_connections 2024-06-10T02:54:51.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:51.166+0000 7f4b88ca7700 1 -- 172.21.15.52:0/631070805 wait complete. 2024-06-10T02:54:51.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:54:50 smithi188 ceph-mon[36599]: pgmap v453: 129 pgs: 129 active+clean; 661 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 853 B/s wr, 9 op/s 2024-06-10T02:54:51.966 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph tell mds.f status 2024-06-10T02:54:52.219 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:54:52.976 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:54:52 smithi052 ceph-mon[32362]: pgmap v454: 129 pgs: 129 active+clean; 661 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 853 B/s wr, 9 op/s 2024-06-10T02:54:53.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:54:52 smithi120 ceph-mon[36634]: pgmap v454: 129 pgs: 129 active+clean; 661 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 853 B/s wr, 9 op/s 2024-06-10T02:54:53.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:54:52 smithi188 ceph-mon[36599]: pgmap v454: 129 pgs: 129 active+clean; 661 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 853 B/s wr, 9 op/s 2024-06-10T02:54:53.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.889+0000 7efd74374700 1 -- 172.21.15.52:0/1452773125 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efd6c107400 msgr2=0x7efd6c1077e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:54:53.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.889+0000 7efd74374700 1 --2- 172.21.15.52:0/1452773125 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efd6c107400 0x7efd6c1077e0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7efd68009990 tx=0x7efd68009ca0 comp rx=0 tx=0).stop 2024-06-10T02:54:53.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.889+0000 7efd74374700 1 -- 172.21.15.52:0/1452773125 shutdown_connections 2024-06-10T02:54:53.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.889+0000 7efd74374700 1 --2- 172.21.15.52:0/1452773125 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efd6c101d50 0x7efd6c10e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:53.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.889+0000 7efd74374700 1 --2- 172.21.15.52:0/1452773125 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efd6c107400 0x7efd6c1077e0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:53.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.889+0000 7efd74374700 1 --2- 172.21.15.52:0/1452773125 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efd6c101390 0x7efd6c101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:53.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.889+0000 7efd74374700 1 -- 172.21.15.52:0/1452773125 >> 172.21.15.52:0/1452773125 conn(0x7efd6c078340 msgr2=0x7efd6c078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:54:53.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.889+0000 7efd74374700 1 -- 172.21.15.52:0/1452773125 shutdown_connections 2024-06-10T02:54:53.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.889+0000 7efd74374700 1 -- 172.21.15.52:0/1452773125 wait complete. 2024-06-10T02:54:53.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.890+0000 7efd74374700 1 Processor -- start 2024-06-10T02:54:53.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.890+0000 7efd74374700 1 -- start start 2024-06-10T02:54:53.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.890+0000 7efd74374700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efd6c101390 0x7efd6c19e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:53.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.890+0000 7efd74374700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efd6c101d50 0x7efd6c19eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:53.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.890+0000 7efd74374700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efd6c107400 0x7efd6c198a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:53.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.890+0000 7efd74374700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7efd6c1116a0 con 0x7efd6c101390 2024-06-10T02:54:53.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.890+0000 7efd74374700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7efd6c111520 con 0x7efd6c107400 2024-06-10T02:54:53.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.890+0000 7efd74374700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7efd6c111820 con 0x7efd6c101d50 2024-06-10T02:54:53.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.890+0000 7efd72110700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efd6c101390 0x7efd6c19e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:53.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.890+0000 7efd72110700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efd6c101390 0x7efd6c19e860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48432/0 (socket says 172.21.15.52:48432) 2024-06-10T02:54:53.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.890+0000 7efd72110700 1 -- 172.21.15.52:0/1413641990 learned_addr learned my addr 172.21.15.52:0/1413641990 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:54:53.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.890+0000 7efd7190f700 1 --2- 172.21.15.52:0/1413641990 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efd6c101d50 0x7efd6c19eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:53.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.891+0000 7efd72911700 1 --2- 172.21.15.52:0/1413641990 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efd6c107400 0x7efd6c198a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:53.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.891+0000 7efd72110700 1 -- 172.21.15.52:0/1413641990 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efd6c101d50 msgr2=0x7efd6c19eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:54:53.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.891+0000 7efd72110700 1 --2- 172.21.15.52:0/1413641990 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efd6c101d50 0x7efd6c19eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:53.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.891+0000 7efd72110700 1 -- 172.21.15.52:0/1413641990 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efd6c107400 msgr2=0x7efd6c198a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:54:53.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.891+0000 7efd72110700 1 --2- 172.21.15.52:0/1413641990 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efd6c107400 0x7efd6c198a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:53.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.891+0000 7efd72110700 1 -- 172.21.15.52:0/1413641990 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd6c199180 con 0x7efd6c101390 2024-06-10T02:54:53.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.891+0000 7efd72911700 1 --2- 172.21.15.52:0/1413641990 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efd6c107400 0x7efd6c198a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T02:54:53.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.891+0000 7efd72110700 1 --2- 172.21.15.52:0/1413641990 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efd6c101390 0x7efd6c19e860 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7efd68000c00 tx=0x7efd680047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:54:53.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.891+0000 7efd637fe700 1 -- 172.21.15.52:0/1413641990 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd68027070 con 0x7efd6c101390 2024-06-10T02:54:53.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.891+0000 7efd637fe700 1 -- 172.21.15.52:0/1413641990 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd68003780 con 0x7efd6c101390 2024-06-10T02:54:53.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.891+0000 7efd637fe700 1 -- 172.21.15.52:0/1413641990 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd68022dd0 con 0x7efd6c101390 2024-06-10T02:54:53.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.891+0000 7efd74374700 1 -- 172.21.15.52:0/1413641990 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd6c199410 con 0x7efd6c101390 2024-06-10T02:54:53.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.891+0000 7efd74374700 1 -- 172.21.15.52:0/1413641990 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd6c1a5730 con 0x7efd6c101390 2024-06-10T02:54:53.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.894+0000 7efd637fe700 1 -- 172.21.15.52:0/1413641990 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7efd6802a070 con 0x7efd6c101390 2024-06-10T02:54:53.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.895+0000 7efd637fe700 1 --2- 172.21.15.52:0/1413641990 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7efd580a2d30 0x7efd580a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:53.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.895+0000 7efd637fe700 1 -- 172.21.15.52:0/1413641990 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7efd680d0230 con 0x7efd6c101390 2024-06-10T02:54:53.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.895+0000 7efd7190f700 1 --2- 172.21.15.52:0/1413641990 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7efd580a2d30 0x7efd580a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:53.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.895+0000 7efd7ebf7b80 1 Processor -- start 2024-06-10T02:54:53.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.895+0000 7efd7190f700 1 --2- 172.21.15.52:0/1413641990 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7efd580a2d30 0x7efd580a51f0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7efd6c0788b0 tx=0x7efd5c00a300 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:54:53.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.896+0000 7efd7ebf7b80 1 -- start start 2024-06-10T02:54:53.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.896+0000 7efd7ebf7b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c6ed508a20 0x55c6ed508e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:53.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.896+0000 7efd7ebf7b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c6ed509340 0x55c6ed502da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:53.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.896+0000 7efd7ebf7b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c6ed5032e0 0x55c6ed503700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:53.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.896+0000 7efd7ebf7b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55c6ed4507c0 con 0x55c6ed508a20 2024-06-10T02:54:53.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.896+0000 7efd7ebf7b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55c6ed3f86d0 con 0x55c6ed509340 2024-06-10T02:54:53.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.896+0000 7efd7ebf7b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55c6ed44c780 con 0x55c6ed5032e0 2024-06-10T02:54:53.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.896+0000 7efd72110700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c6ed508a20 0x55c6ed508e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:53.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.896+0000 7efd72110700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c6ed508a20 0x55c6ed508e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48434/0 (socket says 172.21.15.52:48434) 2024-06-10T02:54:53.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.896+0000 7efd72110700 1 -- 172.21.15.52:0/3163980425 learned_addr learned my addr 172.21.15.52:0/3163980425 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:54:53.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.896+0000 7efd7190f700 1 --2- 172.21.15.52:0/3163980425 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c6ed509340 0x55c6ed502da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:53.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.896+0000 7efd72911700 1 --2- 172.21.15.52:0/3163980425 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c6ed5032e0 0x55c6ed503700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:53.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.896+0000 7efd72110700 1 -- 172.21.15.52:0/3163980425 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c6ed5032e0 msgr2=0x55c6ed503700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:54:53.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.896+0000 7efd72110700 1 --2- 172.21.15.52:0/3163980425 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c6ed5032e0 0x55c6ed503700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:53.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.896+0000 7efd72110700 1 -- 172.21.15.52:0/3163980425 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c6ed509340 msgr2=0x55c6ed502da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:54:53.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.896+0000 7efd72110700 1 --2- 172.21.15.52:0/3163980425 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c6ed509340 0x55c6ed502da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:53.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.896+0000 7efd72110700 1 -- 172.21.15.52:0/3163980425 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c6ed3eef00 con 0x55c6ed508a20 2024-06-10T02:54:53.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.896+0000 7efd7190f700 1 --2- 172.21.15.52:0/3163980425 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c6ed509340 0x55c6ed502da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T02:54:53.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.897+0000 7efd72110700 1 --2- 172.21.15.52:0/3163980425 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c6ed508a20 0x55c6ed508e00 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7efd680a4cf0 tx=0x7efd6809c0d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=160ca290d6d7892f server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:54:53.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.897+0000 7efd60d70700 1 -- 172.21.15.52:0/3163980425 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd680a09f0 con 0x55c6ed508a20 2024-06-10T02:54:53.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.897+0000 7efd60d70700 1 -- 172.21.15.52:0/3163980425 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd680a5220 con 0x55c6ed508a20 2024-06-10T02:54:53.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.897+0000 7efd60d70700 1 -- 172.21.15.52:0/3163980425 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd680a4d90 con 0x55c6ed508a20 2024-06-10T02:54:53.900 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.897+0000 7efd7ebf7b80 1 -- 172.21.15.52:0/3163980425 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c6ed508a20 msgr2=0x55c6ed508e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:54:53.900 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.897+0000 7efd7ebf7b80 1 --2- 172.21.15.52:0/3163980425 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c6ed508a20 0x55c6ed508e00 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7efd680a4cf0 tx=0x7efd6809c0d0 comp rx=0 tx=0).stop 2024-06-10T02:54:53.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.897+0000 7efd7ebf7b80 1 -- 172.21.15.52:0/3163980425 shutdown_connections 2024-06-10T02:54:53.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.897+0000 7efd7ebf7b80 1 --2- 172.21.15.52:0/3163980425 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c6ed5032e0 0x55c6ed503700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:53.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.897+0000 7efd7ebf7b80 1 --2- 172.21.15.52:0/3163980425 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c6ed508a20 0x55c6ed508e00 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:53.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.898+0000 7efd7ebf7b80 1 --2- 172.21.15.52:0/3163980425 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c6ed509340 0x55c6ed502da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:53.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.898+0000 7efd7ebf7b80 1 -- 172.21.15.52:0/3163980425 >> 172.21.15.52:0/3163980425 conn(0x55c6ed429c70 msgr2=0x55c6ed4f9800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:54:53.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.898+0000 7efd7ebf7b80 1 -- 172.21.15.52:0/3163980425 shutdown_connections 2024-06-10T02:54:53.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.898+0000 7efd7ebf7b80 1 -- 172.21.15.52:0/3163980425 wait complete. 2024-06-10T02:54:53.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.898+0000 7efd7ebf7b80 1 -- start start 2024-06-10T02:54:53.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.898+0000 7efd7ebf7b80 1 Processor -- start 2024-06-10T02:54:53.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.898+0000 7efd7ebf7b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c6ed5032e0 0x55c6ed50f060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:53.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.898+0000 7efd7ebf7b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c6ed508a20 0x55c6ed50c360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:53.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.898+0000 7efd7ebf7b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c6ed509340 0x55c6ed50c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:53.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.898+0000 7efd7ebf7b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55c6ed50faa0 con 0x55c6ed508a20 2024-06-10T02:54:53.902 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.898+0000 7efd7ebf7b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55c6ed3f0ba0 con 0x55c6ed509340 2024-06-10T02:54:53.902 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.898+0000 7efd7ebf7b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55c6ed50fc20 con 0x55c6ed5032e0 2024-06-10T02:54:53.902 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.899+0000 7efd7ebf7b80 10 client.0 _ll_register_callbacks cb 0x55c6ed3ea8d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T02:54:53.902 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.899+0000 7efd7190f700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c6ed508a20 0x55c6ed50c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:53.902 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.899+0000 7efd7190f700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c6ed508a20 0x55c6ed50c360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48448/0 (socket says 172.21.15.52:48448) 2024-06-10T02:54:53.902 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.899+0000 7efd7190f700 1 -- 172.21.15.52:0/2017974323 learned_addr learned my addr 172.21.15.52:0/2017974323 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:54:53.902 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.899+0000 7efd72911700 1 --2- 172.21.15.52:0/2017974323 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c6ed509340 0x55c6ed50c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:53.902 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.899+0000 7efd72110700 1 --2- 172.21.15.52:0/2017974323 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c6ed5032e0 0x55c6ed50f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:53.902 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.899+0000 7efd7190f700 1 -- 172.21.15.52:0/2017974323 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c6ed5032e0 msgr2=0x55c6ed50f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:54:53.902 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.899+0000 7efd7190f700 1 --2- 172.21.15.52:0/2017974323 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c6ed5032e0 0x55c6ed50f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:53.902 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.899+0000 7efd7190f700 1 -- 172.21.15.52:0/2017974323 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c6ed509340 msgr2=0x55c6ed50c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:54:53.902 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.899+0000 7efd7190f700 1 --2- 172.21.15.52:0/2017974323 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c6ed509340 0x55c6ed50c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:53.902 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.899+0000 7efd7190f700 1 -- 172.21.15.52:0/2017974323 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c6ed39f880 con 0x55c6ed508a20 2024-06-10T02:54:53.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.899+0000 7efd72110700 1 --2- 172.21.15.52:0/2017974323 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c6ed5032e0 0x55c6ed50f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T02:54:53.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.899+0000 7efd7190f700 1 --2- 172.21.15.52:0/2017974323 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c6ed508a20 0x55c6ed50c360 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7efd5c00a870 tx=0x7efd5c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5717fd0a739fec9 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:54:53.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.899+0000 7efd55ffb700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-10T02:54:53.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.899+0000 7efd55ffb700 1 -- 172.21.15.52:0/2017974323 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd5c016070 con 0x55c6ed508a20 2024-06-10T02:54:53.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.899+0000 7efd55ffb700 1 -- 172.21.15.52:0/2017974323 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7efd5c004a30 con 0x55c6ed508a20 2024-06-10T02:54:53.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.899+0000 7efd7ebf7b80 1 -- 172.21.15.52:0/2017974323 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd680a5220 con 0x55c6ed508a20 2024-06-10T02:54:53.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.900+0000 7efd55ffb700 1 -- 172.21.15.52:0/2017974323 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7efd5c004c30 con 0x55c6ed508a20 2024-06-10T02:54:53.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.900+0000 7efd7ebf7b80 10 client.15483 fetch_fsmap learned FSMap version 30 2024-06-10T02:54:53.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.900+0000 7efd7ebf7b80 1 -- 172.21.15.52:0/2017974323 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c6ed51a330 con 0x55c6ed508a20 2024-06-10T02:54:53.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.900+0000 7efd55ffb700 1 -- 172.21.15.52:0/2017974323 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd680a5220 con 0x55c6ed508a20 2024-06-10T02:54:53.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.900+0000 7efd55ffb700 1 -- 172.21.15.52:0/2017974323 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7efd5c010b00 con 0x55c6ed508a20 2024-06-10T02:54:53.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.900+0000 7efd7ebf7b80 10 client.15483 fetch_fsmap finished waiting for FSMap version 30 2024-06-10T02:54:53.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.900+0000 7efd7ebf7b80 10 client.15483 resolve_mds: resolved name 'f' to daemon mds.f 2024-06-10T02:54:53.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.900+0000 7efd7ebf7b80 20 client.15483 populate_metadata read hostname 'smithi052' 2024-06-10T02:54:53.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.900+0000 7efd7ebf7b80 1 --2- 172.21.15.52:0/2017974323 >> [v2:172.21.15.188:6834/3904544259,v1:172.21.15.188:6835/3904544259] conn(0x55c6ed51c980 0x55c6ed51ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:53.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.900+0000 7efd7ebf7b80 4 client.15483 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T02:54:53.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.900+0000 7efd7ebf7b80 1 -- 172.21.15.52:0/2017974323 --> [v2:172.21.15.188:6834/3904544259,v1:172.21.15.188:6835/3904544259] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c6ed45dcc0 con 0x55c6ed51c980 2024-06-10T02:54:53.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.901+0000 7efd72110700 1 --2- 172.21.15.52:0/2017974323 >> [v2:172.21.15.188:6834/3904544259,v1:172.21.15.188:6835/3904544259] conn(0x55c6ed51c980 0x55c6ed51ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:53.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.901+0000 7efd72110700 1 --2- 172.21.15.52:0/2017974323 >> [v2:172.21.15.188:6834/3904544259,v1:172.21.15.188:6835/3904544259] conn(0x55c6ed51c980 0x55c6ed51ddf0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=849b2d153cef1aa2 server_cookie=90218a76538f0924 in_seq=0 out_seq=0 2024-06-10T02:54:53.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.901+0000 7efd55ffb700 10 client.15483 ms_handle_connect on v2:172.21.15.188:6834/3904544259 2024-06-10T02:54:53.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.902+0000 7efd55ffb700 1 -- 172.21.15.52:0/2017974323 <== mds.2 v2:172.21.15.188:6834/3904544259 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55c6ed45dcc0 con 0x55c6ed51c980 2024-06-10T02:54:53.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.902+0000 7efd55ffb700 10 client.15483 handle_command_reply: tid=0 2024-06-10T02:54:53.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.902+0000 7efd7ebf7b80 1 client.15483 shutdown 2024-06-10T02:54:53.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.902+0000 7efd7ebf7b80 1 -- 172.21.15.52:0/2017974323 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c6ed508a20 msgr2=0x55c6ed50c360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:54:53.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.902+0000 7efd7ebf7b80 1 --2- 172.21.15.52:0/2017974323 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c6ed508a20 0x55c6ed50c360 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7efd5c00a870 tx=0x7efd5c0047c0 comp rx=0 tx=0).stop 2024-06-10T02:54:53.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.902+0000 7efd7ebf7b80 1 -- 172.21.15.52:0/2017974323 shutdown_connections 2024-06-10T02:54:53.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.902+0000 7efd7ebf7b80 1 --2- 172.21.15.52:0/2017974323 >> [v2:172.21.15.188:6834/3904544259,v1:172.21.15.188:6835/3904544259] conn(0x55c6ed51c980 0x55c6ed51ddf0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:53.906 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.902+0000 7efd7ebf7b80 1 --2- 172.21.15.52:0/2017974323 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c6ed509340 0x55c6ed50c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:53.906 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.902+0000 7efd7ebf7b80 1 --2- 172.21.15.52:0/2017974323 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c6ed5032e0 0x55c6ed50f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:53.906 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.902+0000 7efd55ffb700 0 client.15483 ms_handle_reset on v2:172.21.15.188:6834/3904544259 2024-06-10T02:54:53.906 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.902+0000 7efd7ebf7b80 1 --2- 172.21.15.52:0/2017974323 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c6ed508a20 0x55c6ed50c360 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:53.906 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.902+0000 7efd7ebf7b80 1 -- 172.21.15.52:0/2017974323 >> 172.21.15.52:0/2017974323 conn(0x55c6ed429c70 msgr2=0x55c6ed4fd980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:54:53.906 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.902+0000 7efd7ebf7b80 1 -- 172.21.15.52:0/2017974323 shutdown_connections 2024-06-10T02:54:53.906 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.902+0000 7efd7ebf7b80 1 -- 172.21.15.52:0/2017974323 wait complete. 2024-06-10T02:54:53.906 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.902+0000 7efd7ebf7b80 20 client.15483 trim_cache size 0 max 16384 2024-06-10T02:54:53.924 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.924+0000 7efd7ebf7b80 1 Processor -- start 2024-06-10T02:54:53.924 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.924+0000 7efd7ebf7b80 1 -- start start 2024-06-10T02:54:53.924 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.924+0000 7efd7ebf7b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c6ed5032e0 0x55c6ed50ac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:53.925 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.924+0000 7efd7ebf7b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c6ed508a20 0x55c6ed50b190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:53.925 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.924+0000 7efd7ebf7b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c6ed509340 0x55c6ed509820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:53.925 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.924+0000 7efd7ebf7b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55c6ed51c130 con 0x55c6ed508a20 2024-06-10T02:54:53.925 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.924+0000 7efd7ebf7b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55c6ed51c800 con 0x55c6ed509340 2024-06-10T02:54:53.925 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.924+0000 7efd7ebf7b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55c6ed51ced0 con 0x55c6ed5032e0 2024-06-10T02:54:53.925 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.924+0000 7efd7190f700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c6ed508a20 0x55c6ed50b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:53.925 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.924+0000 7efd7190f700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c6ed508a20 0x55c6ed50b190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48456/0 (socket says 172.21.15.52:48456) 2024-06-10T02:54:53.925 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.924+0000 7efd7190f700 1 -- 172.21.15.52:0/4247909753 learned_addr learned my addr 172.21.15.52:0/4247909753 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:54:53.925 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.924+0000 7efd72110700 1 --2- 172.21.15.52:0/4247909753 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c6ed5032e0 0x55c6ed50ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:53.925 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.924+0000 7efd7190f700 1 -- 172.21.15.52:0/4247909753 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c6ed5032e0 msgr2=0x55c6ed50ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:54:53.925 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.924+0000 7efd72911700 1 --2- 172.21.15.52:0/4247909753 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c6ed509340 0x55c6ed509820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:53.925 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.924+0000 7efd7190f700 1 --2- 172.21.15.52:0/4247909753 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c6ed5032e0 0x55c6ed50ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:53.925 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.924+0000 7efd7190f700 1 -- 172.21.15.52:0/4247909753 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c6ed509340 msgr2=0x55c6ed509820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:54:53.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.924+0000 7efd7190f700 1 --2- 172.21.15.52:0/4247909753 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c6ed509340 0x55c6ed509820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:53.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.924+0000 7efd7190f700 1 -- 172.21.15.52:0/4247909753 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c6ed51ed50 con 0x55c6ed508a20 2024-06-10T02:54:53.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.924+0000 7efd72911700 1 --2- 172.21.15.52:0/4247909753 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c6ed509340 0x55c6ed509820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T02:54:53.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.924+0000 7efd72110700 1 --2- 172.21.15.52:0/4247909753 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c6ed5032e0 0x55c6ed50ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T02:54:53.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.925+0000 7efd7190f700 1 --2- 172.21.15.52:0/4247909753 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c6ed508a20 0x55c6ed50b190 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7efd5c010ef0 tx=0x7efd5c0046f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d2065ae3f677889 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:54:53.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.925+0000 7efd57fff700 1 -- 172.21.15.52:0/4247909753 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd5c004a30 con 0x55c6ed508a20 2024-06-10T02:54:53.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.925+0000 7efd57fff700 1 -- 172.21.15.52:0/4247909753 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd5c03c070 con 0x55c6ed508a20 2024-06-10T02:54:53.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.925+0000 7efd57fff700 1 -- 172.21.15.52:0/4247909753 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd5c004dd0 con 0x55c6ed508a20 2024-06-10T02:54:53.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.926+0000 7efd7ebf7b80 1 -- 172.21.15.52:0/4247909753 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c6ed508a20 msgr2=0x55c6ed50b190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:54:53.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.926+0000 7efd7ebf7b80 1 --2- 172.21.15.52:0/4247909753 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c6ed508a20 0x55c6ed50b190 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7efd5c010ef0 tx=0x7efd5c0046f0 comp rx=0 tx=0).stop 2024-06-10T02:54:53.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.926+0000 7efd7ebf7b80 1 -- 172.21.15.52:0/4247909753 shutdown_connections 2024-06-10T02:54:53.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.926+0000 7efd7ebf7b80 1 --2- 172.21.15.52:0/4247909753 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c6ed509340 0x55c6ed509820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:53.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.926+0000 7efd7ebf7b80 1 --2- 172.21.15.52:0/4247909753 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c6ed5032e0 0x55c6ed50ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:53.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.926+0000 7efd7ebf7b80 1 --2- 172.21.15.52:0/4247909753 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c6ed508a20 0x55c6ed50b190 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:53.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.926+0000 7efd7ebf7b80 1 -- 172.21.15.52:0/4247909753 >> 172.21.15.52:0/4247909753 conn(0x55c6ed429c70 msgr2=0x55c6ed4f9330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:54:53.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.926+0000 7efd7ebf7b80 1 -- 172.21.15.52:0/4247909753 shutdown_connections 2024-06-10T02:54:53.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.926+0000 7efd7ebf7b80 1 -- 172.21.15.52:0/4247909753 wait complete. 2024-06-10T02:54:53.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.926+0000 7efd7ebf7b80 1 -- start start 2024-06-10T02:54:53.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.926+0000 7efd7ebf7b80 1 Processor -- start 2024-06-10T02:54:53.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.926+0000 7efd7ebf7b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c6ed5032e0 0x55c6ed527270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:53.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.926+0000 7efd7ebf7b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c6ed508a20 0x55c6ed5277b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:53.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.926+0000 7efd7ebf7b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c6ed509340 0x55c6ed522220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:53.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.926+0000 7efd7ebf7b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55c6ed51dc70 con 0x55c6ed508a20 2024-06-10T02:54:53.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.926+0000 7efd7ebf7b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55c6ed51d5a0 con 0x55c6ed5032e0 2024-06-10T02:54:53.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.926+0000 7efd7ebf7b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55c6ed51e340 con 0x55c6ed509340 2024-06-10T02:54:53.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.926+0000 7efd7ebf7b80 10 client.0 _ll_register_callbacks cb 0x55c6ed3ea8d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T02:54:53.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.927+0000 7efd7190f700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c6ed508a20 0x55c6ed5277b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:53.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.927+0000 7efd7190f700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c6ed508a20 0x55c6ed5277b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48470/0 (socket says 172.21.15.52:48470) 2024-06-10T02:54:53.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.927+0000 7efd7190f700 1 -- 172.21.15.52:0/1033592172 learned_addr learned my addr 172.21.15.52:0/1033592172 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:54:53.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.927+0000 7efd72911700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c6ed509340 0x55c6ed522220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:53.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.927+0000 7efd72110700 1 --2- 172.21.15.52:0/1033592172 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c6ed5032e0 0x55c6ed527270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:53.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.927+0000 7efd7190f700 1 -- 172.21.15.52:0/1033592172 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c6ed509340 msgr2=0x55c6ed522220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:54:53.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.927+0000 7efd7190f700 1 --2- 172.21.15.52:0/1033592172 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c6ed509340 0x55c6ed522220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:53.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.927+0000 7efd7190f700 1 -- 172.21.15.52:0/1033592172 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c6ed5032e0 msgr2=0x55c6ed527270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:54:53.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.927+0000 7efd7190f700 1 --2- 172.21.15.52:0/1033592172 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c6ed5032e0 0x55c6ed527270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:53.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.927+0000 7efd7190f700 1 -- 172.21.15.52:0/1033592172 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c6ed527e10 con 0x55c6ed508a20 2024-06-10T02:54:53.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.927+0000 7efd7190f700 1 --2- 172.21.15.52:0/1033592172 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c6ed508a20 0x55c6ed5277b0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7efd5c027560 tx=0x7efd5c027e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f173f5e3225c066c server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:54:53.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.927+0000 7efd55ffb700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-10T02:54:53.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.927+0000 7efd55ffb700 1 -- 172.21.15.52:0/1033592172 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd5c046020 con 0x55c6ed508a20 2024-06-10T02:54:53.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.927+0000 7efd55ffb700 1 -- 172.21.15.52:0/1033592172 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7efd5c0106c0 con 0x55c6ed508a20 2024-06-10T02:54:53.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.927+0000 7efd7ebf7b80 1 -- 172.21.15.52:0/1033592172 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd5c03c070 con 0x55c6ed508a20 2024-06-10T02:54:53.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.927+0000 7efd55ffb700 1 -- 172.21.15.52:0/1033592172 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7efd5c016040 con 0x55c6ed508a20 2024-06-10T02:54:53.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.927+0000 7efd7ebf7b80 10 client.15489 fetch_fsmap learned FSMap version 30 2024-06-10T02:54:53.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.927+0000 7efd7ebf7b80 1 -- 172.21.15.52:0/1033592172 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c6ed588720 con 0x55c6ed508a20 2024-06-10T02:54:53.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.928+0000 7efd55ffb700 1 -- 172.21.15.52:0/1033592172 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd5c03c070 con 0x55c6ed508a20 2024-06-10T02:54:53.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.928+0000 7efd55ffb700 1 -- 172.21.15.52:0/1033592172 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7efd5c0055d0 con 0x55c6ed508a20 2024-06-10T02:54:53.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.928+0000 7efd7ebf7b80 10 client.15489 fetch_fsmap finished waiting for FSMap version 30 2024-06-10T02:54:53.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.928+0000 7efd7ebf7b80 10 client.15489 resolve_mds: resolved name 'f' to daemon mds.f 2024-06-10T02:54:53.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.928+0000 7efd7ebf7b80 20 client.15489 populate_metadata read hostname 'smithi052' 2024-06-10T02:54:53.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.928+0000 7efd7ebf7b80 1 --2- 172.21.15.52:0/1033592172 >> [v2:172.21.15.188:6834/3904544259,v1:172.21.15.188:6835/3904544259] conn(0x55c6ed58ccd0 0x55c6ed58b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:53.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.928+0000 7efd7ebf7b80 4 client.15489 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "status"}] 2024-06-10T02:54:53.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.928+0000 7efd7ebf7b80 1 -- 172.21.15.52:0/1033592172 --> [v2:172.21.15.188:6834/3904544259,v1:172.21.15.188:6835/3904544259] -- command(tid 0: {"prefix": "status"}) -- 0x55c6ed458240 con 0x55c6ed58ccd0 2024-06-10T02:54:53.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.928+0000 7efd72110700 1 --2- 172.21.15.52:0/1033592172 >> [v2:172.21.15.188:6834/3904544259,v1:172.21.15.188:6835/3904544259] conn(0x55c6ed58ccd0 0x55c6ed58b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:53.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.928+0000 7efd72110700 1 --2- 172.21.15.52:0/1033592172 >> [v2:172.21.15.188:6834/3904544259,v1:172.21.15.188:6835/3904544259] conn(0x55c6ed58ccd0 0x55c6ed58b9e0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d73e23e88459c7fb server_cookie=212954fdbc70ef17 in_seq=0 out_seq=0 2024-06-10T02:54:53.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.928+0000 7efd55ffb700 10 client.15489 ms_handle_connect on v2:172.21.15.188:6834/3904544259 2024-06-10T02:54:53.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.929+0000 7efd55ffb700 1 -- 172.21.15.52:0/1033592172 <== mds.2 v2:172.21.15.188:6834/3904544259 1 ==== command_reply(tid 0: 0 ) ==== 8+0+334 (crc 0 0 0) 0x55c6ed458240 con 0x55c6ed58ccd0 2024-06-10T02:54:53.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.929+0000 7efd55ffb700 10 client.15489 handle_command_reply: tid=0 2024-06-10T02:54:53.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.929+0000 7efd7ebf7b80 1 client.15489 shutdown 2024-06-10T02:54:53.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.929+0000 7efd7ebf7b80 1 -- 172.21.15.52:0/1033592172 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c6ed508a20 msgr2=0x55c6ed5277b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:54:53.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.929+0000 7efd7ebf7b80 1 --2- 172.21.15.52:0/1033592172 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c6ed508a20 0x55c6ed5277b0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7efd5c027560 tx=0x7efd5c027e00 comp rx=0 tx=0).stop 2024-06-10T02:54:53.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.929+0000 7efd7ebf7b80 1 -- 172.21.15.52:0/1033592172 shutdown_connections 2024-06-10T02:54:53.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.929+0000 7efd7ebf7b80 1 --2- 172.21.15.52:0/1033592172 >> [v2:172.21.15.188:6834/3904544259,v1:172.21.15.188:6835/3904544259] conn(0x55c6ed58ccd0 0x55c6ed58b9e0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:53.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.929+0000 7efd7ebf7b80 1 --2- 172.21.15.52:0/1033592172 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c6ed509340 0x55c6ed522220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:53.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.929+0000 7efd7ebf7b80 1 --2- 172.21.15.52:0/1033592172 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c6ed5032e0 0x55c6ed527270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:53.934 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.929+0000 7efd55ffb700 0 client.15489 ms_handle_reset on v2:172.21.15.188:6834/3904544259 2024-06-10T02:54:53.934 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.929+0000 7efd7ebf7b80 1 --2- 172.21.15.52:0/1033592172 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c6ed508a20 0x55c6ed5277b0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:53.934 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.929+0000 7efd7ebf7b80 1 -- 172.21.15.52:0/1033592172 >> 172.21.15.52:0/1033592172 conn(0x55c6ed429c70 msgr2=0x55c6ed509800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:54:53.934 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.929+0000 7efd7ebf7b80 1 -- 172.21.15.52:0/1033592172 shutdown_connections 2024-06-10T02:54:53.934 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.929+0000 7efd7ebf7b80 1 -- 172.21.15.52:0/1033592172 wait complete. 2024-06-10T02:54:53.934 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.929+0000 7efd7ebf7b80 20 client.15489 trim_cache size 0 max 16384 2024-06-10T02:54:53.934 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-10T02:54:53.934 INFO:teuthology.orchestra.run.smithi052.stdout: "cluster_fsid": "d8676e9e-26d2-11ef-bc9b-c7b262605968", 2024-06-10T02:54:53.934 INFO:teuthology.orchestra.run.smithi052.stdout: "whoami": 2, 2024-06-10T02:54:53.934 INFO:teuthology.orchestra.run.smithi052.stdout: "id": 24463, 2024-06-10T02:54:53.935 INFO:teuthology.orchestra.run.smithi052.stdout: "want_state": "up:active", 2024-06-10T02:54:53.935 INFO:teuthology.orchestra.run.smithi052.stdout: "state": "up:active", 2024-06-10T02:54:53.935 INFO:teuthology.orchestra.run.smithi052.stdout: "fs_name": "cephfs", 2024-06-10T02:54:53.935 INFO:teuthology.orchestra.run.smithi052.stdout: "rank_uptime": 227.32413681899999, 2024-06-10T02:54:53.935 INFO:teuthology.orchestra.run.smithi052.stdout: "mdsmap_epoch": 30, 2024-06-10T02:54:53.935 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_epoch": 100, 2024-06-10T02:54:53.935 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_epoch_barrier": 100, 2024-06-10T02:54:53.935 INFO:teuthology.orchestra.run.smithi052.stdout: "uptime": 326.30375134299999 2024-06-10T02:54:53.935 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-10T02:54:53.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.929+0000 7efd74374700 1 -- 172.21.15.52:0/1413641990 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7efd580a2d30 msgr2=0x7efd580a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:54:53.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.929+0000 7efd74374700 1 --2- 172.21.15.52:0/1413641990 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7efd580a2d30 0x7efd580a51f0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7efd6c0788b0 tx=0x7efd5c00a300 comp rx=0 tx=0).stop 2024-06-10T02:54:53.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.930+0000 7efd74374700 1 -- 172.21.15.52:0/1413641990 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efd6c101390 msgr2=0x7efd6c19e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:54:53.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.930+0000 7efd74374700 1 --2- 172.21.15.52:0/1413641990 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efd6c101390 0x7efd6c19e860 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7efd68000c00 tx=0x7efd680047b0 comp rx=0 tx=0).stop 2024-06-10T02:54:53.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.930+0000 7efd74374700 1 -- 172.21.15.52:0/1413641990 shutdown_connections 2024-06-10T02:54:53.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.930+0000 7efd74374700 1 --2- 172.21.15.52:0/1413641990 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efd6c107400 0x7efd6c198a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:53.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.930+0000 7efd74374700 1 --2- 172.21.15.52:0/1413641990 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7efd580a2d30 0x7efd580a51f0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:53.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.930+0000 7efd74374700 1 --2- 172.21.15.52:0/1413641990 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efd6c101390 0x7efd6c19e860 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:53.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.930+0000 7efd74374700 1 --2- 172.21.15.52:0/1413641990 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efd6c101d50 0x7efd6c19eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:53.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.930+0000 7efd74374700 1 -- 172.21.15.52:0/1413641990 >> 172.21.15.52:0/1413641990 conn(0x7efd6c078340 msgr2=0x7efd6c100650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:54:53.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.930+0000 7efd74374700 1 -- 172.21.15.52:0/1413641990 shutdown_connections 2024-06-10T02:54:53.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:53.930+0000 7efd74374700 1 -- 172.21.15.52:0/1413641990 wait complete. 2024-06-10T02:54:54.682 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph tell mds.c status 2024-06-10T02:54:54.933 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:54:55.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:54:54 smithi120 ceph-mon[36634]: pgmap v455: 129 pgs: 129 active+clean; 661 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 853 B/s wr, 12 op/s 2024-06-10T02:54:55.194 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:54:54 smithi052 ceph-mon[32362]: pgmap v455: 129 pgs: 129 active+clean; 661 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 853 B/s wr, 12 op/s 2024-06-10T02:54:55.202 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:54:54 smithi188 ceph-mon[36599]: pgmap v455: 129 pgs: 129 active+clean; 661 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 853 B/s wr, 12 op/s 2024-06-10T02:54:56.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.600+0000 7fcdd00aa700 1 -- 172.21.15.52:0/2693152703 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcdc806aa00 msgr2=0x7fcdc806ae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:54:56.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.600+0000 7fcdd00aa700 1 --2- 172.21.15.52:0/2693152703 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcdc806aa00 0x7fcdc806ae80 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fcdb8009990 tx=0x7fcdb8009ca0 comp rx=0 tx=0).stop 2024-06-10T02:54:56.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.600+0000 7fcdd00aa700 1 -- 172.21.15.52:0/2693152703 shutdown_connections 2024-06-10T02:54:56.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.600+0000 7fcdd00aa700 1 --2- 172.21.15.52:0/2693152703 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcdc806b3c0 0x7fcdc810f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:56.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.600+0000 7fcdd00aa700 1 --2- 172.21.15.52:0/2693152703 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcdc81084f0 0x7fcdc81088d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:56.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.600+0000 7fcdd00aa700 1 --2- 172.21.15.52:0/2693152703 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcdc806aa00 0x7fcdc806ae80 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:56.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.600+0000 7fcdd00aa700 1 -- 172.21.15.52:0/2693152703 >> 172.21.15.52:0/2693152703 conn(0x7fcdc8078300 msgr2=0x7fcdc8078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:54:56.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.600+0000 7fcdd00aa700 1 -- 172.21.15.52:0/2693152703 shutdown_connections 2024-06-10T02:54:56.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.600+0000 7fcdd00aa700 1 -- 172.21.15.52:0/2693152703 wait complete. 2024-06-10T02:54:56.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.601+0000 7fcdd00aa700 1 Processor -- start 2024-06-10T02:54:56.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.601+0000 7fcdd00aa700 1 -- start start 2024-06-10T02:54:56.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.601+0000 7fcdd00aa700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcdc806aa00 0x7fcdc819e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:56.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.601+0000 7fcdd00aa700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcdc806b3c0 0x7fcdc819edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:56.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.601+0000 7fcdd00aa700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcdc81084f0 0x7fcdc8198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:56.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.601+0000 7fcdd00aa700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fcdc8112770 con 0x7fcdc806b3c0 2024-06-10T02:54:56.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.601+0000 7fcdd00aa700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fcdc81125f0 con 0x7fcdc81084f0 2024-06-10T02:54:56.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.601+0000 7fcdd00aa700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fcdc81128f0 con 0x7fcdc806aa00 2024-06-10T02:54:56.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.601+0000 7fcdcd645700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcdc806b3c0 0x7fcdc819edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:56.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.601+0000 7fcdcd645700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcdc806b3c0 0x7fcdc819edd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48486/0 (socket says 172.21.15.52:48486) 2024-06-10T02:54:56.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.601+0000 7fcdcd645700 1 -- 172.21.15.52:0/1248766839 learned_addr learned my addr 172.21.15.52:0/1248766839 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:54:56.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.602+0000 7fcdcde46700 1 --2- 172.21.15.52:0/1248766839 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcdc806aa00 0x7fcdc819e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:56.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.602+0000 7fcdce647700 1 --2- 172.21.15.52:0/1248766839 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcdc81084f0 0x7fcdc8198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:56.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.602+0000 7fcdcd645700 1 -- 172.21.15.52:0/1248766839 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcdc806aa00 msgr2=0x7fcdc819e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:54:56.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.602+0000 7fcdcd645700 1 --2- 172.21.15.52:0/1248766839 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcdc806aa00 0x7fcdc819e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:56.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.602+0000 7fcdcd645700 1 -- 172.21.15.52:0/1248766839 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcdc81084f0 msgr2=0x7fcdc8198a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:54:56.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.602+0000 7fcdcd645700 1 --2- 172.21.15.52:0/1248766839 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcdc81084f0 0x7fcdc8198a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:56.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.602+0000 7fcdcd645700 1 -- 172.21.15.52:0/1248766839 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdc81991b0 con 0x7fcdc806b3c0 2024-06-10T02:54:56.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.602+0000 7fcdce647700 1 --2- 172.21.15.52:0/1248766839 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcdc81084f0 0x7fcdc8198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T02:54:56.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.602+0000 7fcdcd645700 1 --2- 172.21.15.52:0/1248766839 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcdc806b3c0 0x7fcdc819edd0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fcdb8000c00 tx=0x7fcdb800bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:54:56.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.602+0000 7fcdbeffd700 1 -- 172.21.15.52:0/1248766839 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdb8027070 con 0x7fcdc806b3c0 2024-06-10T02:54:56.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.602+0000 7fcdbeffd700 1 -- 172.21.15.52:0/1248766839 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcdb8003d70 con 0x7fcdc806b3c0 2024-06-10T02:54:56.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.602+0000 7fcdbeffd700 1 -- 172.21.15.52:0/1248766839 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdb800f460 con 0x7fcdc806b3c0 2024-06-10T02:54:56.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.602+0000 7fcdd00aa700 1 -- 172.21.15.52:0/1248766839 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcdc8199440 con 0x7fcdc806b3c0 2024-06-10T02:54:56.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.603+0000 7fcdd00aa700 1 -- 172.21.15.52:0/1248766839 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcdc81a5760 con 0x7fcdc806b3c0 2024-06-10T02:54:56.607 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.605+0000 7fcdbeffd700 1 -- 172.21.15.52:0/1248766839 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7fcdb802a070 con 0x7fcdc806b3c0 2024-06-10T02:54:56.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.606+0000 7fcdbeffd700 1 --2- 172.21.15.52:0/1248766839 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fcdb40a2d30 0x7fcdb40a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:56.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.606+0000 7fcdcde46700 1 --2- 172.21.15.52:0/1248766839 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fcdb40a2d30 0x7fcdb40a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:56.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.606+0000 7fcdbeffd700 1 -- 172.21.15.52:0/1248766839 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7fcdb80d01e0 con 0x7fcdc806b3c0 2024-06-10T02:54:56.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.606+0000 7fcdda92db80 1 Processor -- start 2024-06-10T02:54:56.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.606+0000 7fcdcde46700 1 --2- 172.21.15.52:0/1248766839 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fcdb40a2d30 0x7fcdb40a51f0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fcdc8199b90 tx=0x7fcdc4009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:54:56.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.607+0000 7fcdda92db80 1 -- start start 2024-06-10T02:54:56.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.607+0000 7fcdda92db80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f3c2a0c320 0x55f3c2a0c700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:56.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.607+0000 7fcdda92db80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f3c2a0cc40 0x55f3c2a06620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:56.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.607+0000 7fcdda92db80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f3c2a06b60 0x55f3c2a06fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:56.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.607+0000 7fcdda92db80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55f3c28f4f10 con 0x55f3c2a06b60 2024-06-10T02:54:56.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.607+0000 7fcdda92db80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55f3c2953800 con 0x55f3c2a0cc40 2024-06-10T02:54:56.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.607+0000 7fcdda92db80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55f3c28bb410 con 0x55f3c2a0c320 2024-06-10T02:54:56.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.607+0000 7fcdce647700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f3c2a06b60 0x55f3c2a06fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:56.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.607+0000 7fcdce647700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f3c2a06b60 0x55f3c2a06fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48488/0 (socket says 172.21.15.52:48488) 2024-06-10T02:54:56.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.607+0000 7fcdce647700 1 -- 172.21.15.52:0/714128292 learned_addr learned my addr 172.21.15.52:0/714128292 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:54:56.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.607+0000 7fcdcde46700 1 --2- 172.21.15.52:0/714128292 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f3c2a0c320 0x55f3c2a0c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:56.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.607+0000 7fcdcd645700 1 --2- 172.21.15.52:0/714128292 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f3c2a0cc40 0x55f3c2a06620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:56.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.608+0000 7fcdce647700 1 -- 172.21.15.52:0/714128292 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f3c2a0c320 msgr2=0x55f3c2a0c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:54:56.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.608+0000 7fcdce647700 1 --2- 172.21.15.52:0/714128292 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f3c2a0c320 0x55f3c2a0c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:56.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.608+0000 7fcdce647700 1 -- 172.21.15.52:0/714128292 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f3c2a0cc40 msgr2=0x55f3c2a06620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:54:56.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.608+0000 7fcdce647700 1 --2- 172.21.15.52:0/714128292 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f3c2a0cc40 0x55f3c2a06620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:56.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.608+0000 7fcdce647700 1 -- 172.21.15.52:0/714128292 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f3c28a2880 con 0x55f3c2a06b60 2024-06-10T02:54:56.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.608+0000 7fcdce647700 1 --2- 172.21.15.52:0/714128292 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f3c2a06b60 0x55f3c2a06fa0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fcdc0009400 tx=0x7fcdc0009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f7525f56a4bf829 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:54:56.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.608+0000 7fcdb3fff700 1 -- 172.21.15.52:0/714128292 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdc0007e60 con 0x55f3c2a06b60 2024-06-10T02:54:56.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.608+0000 7fcdb3fff700 1 -- 172.21.15.52:0/714128292 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcdc0019070 con 0x55f3c2a06b60 2024-06-10T02:54:56.610 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.608+0000 7fcdb3fff700 1 -- 172.21.15.52:0/714128292 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdc00139d0 con 0x55f3c2a06b60 2024-06-10T02:54:56.612 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.609+0000 7fcdda92db80 1 -- 172.21.15.52:0/714128292 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f3c2a06b60 msgr2=0x55f3c2a06fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:54:56.612 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.609+0000 7fcdda92db80 1 --2- 172.21.15.52:0/714128292 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f3c2a06b60 0x55f3c2a06fa0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fcdc0009400 tx=0x7fcdc0009920 comp rx=0 tx=0).stop 2024-06-10T02:54:56.612 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.609+0000 7fcdda92db80 1 -- 172.21.15.52:0/714128292 shutdown_connections 2024-06-10T02:54:56.612 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.609+0000 7fcdda92db80 1 --2- 172.21.15.52:0/714128292 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f3c2a06b60 0x55f3c2a06fa0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:56.612 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.609+0000 7fcdda92db80 1 --2- 172.21.15.52:0/714128292 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f3c2a0c320 0x55f3c2a0c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:56.612 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.609+0000 7fcdda92db80 1 --2- 172.21.15.52:0/714128292 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f3c2a0cc40 0x55f3c2a06620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:56.612 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.609+0000 7fcdda92db80 1 -- 172.21.15.52:0/714128292 >> 172.21.15.52:0/714128292 conn(0x55f3c292f030 msgr2=0x55f3c29fd1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:54:56.612 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.609+0000 7fcdda92db80 1 -- 172.21.15.52:0/714128292 shutdown_connections 2024-06-10T02:54:56.612 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.609+0000 7fcdda92db80 1 -- 172.21.15.52:0/714128292 wait complete. 2024-06-10T02:54:56.613 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.610+0000 7fcdda92db80 1 -- start start 2024-06-10T02:54:56.613 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.610+0000 7fcdda92db80 1 Processor -- start 2024-06-10T02:54:56.613 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.610+0000 7fcdda92db80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f3c2a06b60 0x55f3c2a128f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:56.613 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.610+0000 7fcdda92db80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f3c2a0c320 0x55f3c2a0fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:56.613 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.610+0000 7fcdda92db80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f3c2a0cc40 0x55f3c2a10130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:56.613 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.610+0000 7fcdda92db80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55f3c28f3f70 con 0x55f3c2a0cc40 2024-06-10T02:54:56.613 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.610+0000 7fcdda92db80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55f3c294f7c0 con 0x55f3c2a06b60 2024-06-10T02:54:56.613 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.610+0000 7fcdda92db80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55f3c2a13630 con 0x55f3c2a0c320 2024-06-10T02:54:56.613 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.610+0000 7fcdda92db80 10 client.0 _ll_register_callbacks cb 0x55f3c28e9560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T02:54:56.613 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.610+0000 7fcdce647700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f3c2a0cc40 0x55f3c2a10130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:56.613 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.610+0000 7fcdce647700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f3c2a0cc40 0x55f3c2a10130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48498/0 (socket says 172.21.15.52:48498) 2024-06-10T02:54:56.613 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.610+0000 7fcdce647700 1 -- 172.21.15.52:0/774676430 learned_addr learned my addr 172.21.15.52:0/774676430 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:54:56.613 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.611+0000 7fcdcd645700 1 --2- 172.21.15.52:0/774676430 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f3c2a0c320 0x55f3c2a0fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:56.613 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.611+0000 7fcdcde46700 1 --2- 172.21.15.52:0/774676430 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f3c2a06b60 0x55f3c2a128f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:56.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.611+0000 7fcdce647700 1 -- 172.21.15.52:0/774676430 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f3c2a0c320 msgr2=0x55f3c2a0fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:54:56.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.611+0000 7fcdce647700 1 --2- 172.21.15.52:0/774676430 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f3c2a0c320 0x55f3c2a0fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:56.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.611+0000 7fcdce647700 1 -- 172.21.15.52:0/774676430 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f3c2a06b60 msgr2=0x55f3c2a128f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:54:56.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.611+0000 7fcdce647700 1 --2- 172.21.15.52:0/774676430 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f3c2a06b60 0x55f3c2a128f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:56.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.611+0000 7fcdce647700 1 -- 172.21.15.52:0/774676430 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3c2919700 con 0x55f3c2a0cc40 2024-06-10T02:54:56.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.611+0000 7fcdce647700 1 --2- 172.21.15.52:0/774676430 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f3c2a0cc40 0x55f3c2a10130 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fcdc0013550 tx=0x7fcdc0013de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3b1d65572187aba server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:54:56.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.611+0000 7fcdb17fa700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-10T02:54:56.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.611+0000 7fcdb17fa700 1 -- 172.21.15.52:0/774676430 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdc0022070 con 0x55f3c2a0cc40 2024-06-10T02:54:56.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.611+0000 7fcdb17fa700 1 -- 172.21.15.52:0/774676430 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7fcdc0014410 con 0x55f3c2a0cc40 2024-06-10T02:54:56.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.611+0000 7fcdda92db80 1 -- 172.21.15.52:0/774676430 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcdc0019070 con 0x55f3c2a0cc40 2024-06-10T02:54:56.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.611+0000 7fcdb17fa700 1 -- 172.21.15.52:0/774676430 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fcdc0030050 con 0x55f3c2a0cc40 2024-06-10T02:54:56.615 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.611+0000 7fcdda92db80 10 client.15501 fetch_fsmap learned FSMap version 30 2024-06-10T02:54:56.615 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.611+0000 7fcdda92db80 1 -- 172.21.15.52:0/774676430 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f3c2a1dd40 con 0x55f3c2a0cc40 2024-06-10T02:54:56.615 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.612+0000 7fcdb17fa700 1 -- 172.21.15.52:0/774676430 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcdc00095e0 con 0x55f3c2a0cc40 2024-06-10T02:54:56.615 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.612+0000 7fcdb17fa700 1 -- 172.21.15.52:0/774676430 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fcdc00276c0 con 0x55f3c2a0cc40 2024-06-10T02:54:56.616 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.612+0000 7fcdda92db80 10 client.15501 fetch_fsmap finished waiting for FSMap version 30 2024-06-10T02:54:56.616 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.612+0000 7fcdda92db80 10 client.15501 resolve_mds: resolved name 'c' to daemon mds.c 2024-06-10T02:54:56.616 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.612+0000 7fcdda92db80 20 client.15501 populate_metadata read hostname 'smithi052' 2024-06-10T02:54:56.616 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.612+0000 7fcdda92db80 1 --2- 172.21.15.52:0/774676430 >> [v2:172.21.15.188:6836/819317726,v1:172.21.15.188:6837/819317726] conn(0x55f3c2a1ff90 0x55f3c2a21400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:56.616 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.612+0000 7fcdda92db80 4 client.15501 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T02:54:56.616 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.612+0000 7fcdda92db80 1 -- 172.21.15.52:0/774676430 --> [v2:172.21.15.188:6836/819317726,v1:172.21.15.188:6837/819317726] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fcdc0019070 con 0x55f3c2a1ff90 2024-06-10T02:54:56.616 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.612+0000 7fcdcde46700 1 --2- 172.21.15.52:0/774676430 >> [v2:172.21.15.188:6836/819317726,v1:172.21.15.188:6837/819317726] conn(0x55f3c2a1ff90 0x55f3c2a21400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:56.616 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.613+0000 7fcdcde46700 1 --2- 172.21.15.52:0/774676430 >> [v2:172.21.15.188:6836/819317726,v1:172.21.15.188:6837/819317726] conn(0x55f3c2a1ff90 0x55f3c2a21400 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=94ed6fc68cd7472f server_cookie=1de5a02ca557b454 in_seq=0 out_seq=0 2024-06-10T02:54:56.616 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.613+0000 7fcdb17fa700 10 client.15501 ms_handle_connect on v2:172.21.15.188:6836/819317726 2024-06-10T02:54:56.617 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.613+0000 7fcdb17fa700 1 -- 172.21.15.52:0/774676430 <== mds.1 v2:172.21.15.188:6836/819317726 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x7fcdc0019070 con 0x55f3c2a1ff90 2024-06-10T02:54:56.618 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.613+0000 7fcdb17fa700 10 client.15501 handle_command_reply: tid=0 2024-06-10T02:54:56.618 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.613+0000 7fcdda92db80 1 client.15501 shutdown 2024-06-10T02:54:56.618 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.614+0000 7fcdda92db80 1 -- 172.21.15.52:0/774676430 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f3c2a0cc40 msgr2=0x55f3c2a10130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:54:56.618 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.614+0000 7fcdda92db80 1 --2- 172.21.15.52:0/774676430 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f3c2a0cc40 0x55f3c2a10130 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fcdc0013550 tx=0x7fcdc0013de0 comp rx=0 tx=0).stop 2024-06-10T02:54:56.618 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.614+0000 7fcdda92db80 1 -- 172.21.15.52:0/774676430 shutdown_connections 2024-06-10T02:54:56.618 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.614+0000 7fcdda92db80 1 --2- 172.21.15.52:0/774676430 >> [v2:172.21.15.188:6836/819317726,v1:172.21.15.188:6837/819317726] conn(0x55f3c2a1ff90 0x55f3c2a21400 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:56.618 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.614+0000 7fcdda92db80 1 --2- 172.21.15.52:0/774676430 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f3c2a0cc40 0x55f3c2a10130 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:56.618 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.614+0000 7fcdda92db80 1 --2- 172.21.15.52:0/774676430 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f3c2a06b60 0x55f3c2a128f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:56.618 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.614+0000 7fcdda92db80 1 --2- 172.21.15.52:0/774676430 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f3c2a0c320 0x55f3c2a0fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:56.618 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.614+0000 7fcdb17fa700 0 client.15501 ms_handle_reset on v2:172.21.15.188:6836/819317726 2024-06-10T02:54:56.618 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.614+0000 7fcdda92db80 1 -- 172.21.15.52:0/774676430 >> 172.21.15.52:0/774676430 conn(0x55f3c292f030 msgr2=0x55f3c29fc6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:54:56.618 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.614+0000 7fcdda92db80 1 -- 172.21.15.52:0/774676430 shutdown_connections 2024-06-10T02:54:56.619 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.614+0000 7fcdda92db80 1 -- 172.21.15.52:0/774676430 wait complete. 2024-06-10T02:54:56.619 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.614+0000 7fcdda92db80 20 client.15501 trim_cache size 0 max 16384 2024-06-10T02:54:56.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.636+0000 7fcdda92db80 1 Processor -- start 2024-06-10T02:54:56.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.636+0000 7fcdda92db80 1 -- start start 2024-06-10T02:54:56.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.636+0000 7fcdda92db80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f3c2a06b60 0x55f3c2a0d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:56.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.636+0000 7fcdda92db80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f3c2a0c320 0x55f3c29fce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:56.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.636+0000 7fcdda92db80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f3c2a0cc40 0x55f3c29fd370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:56.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.636+0000 7fcdda92db80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55f3c2a1f740 con 0x55f3c2a0c320 2024-06-10T02:54:56.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.636+0000 7fcdda92db80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55f3c2a1fe10 con 0x55f3c2a06b60 2024-06-10T02:54:56.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.636+0000 7fcdda92db80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55f3c2a204e0 con 0x55f3c2a0cc40 2024-06-10T02:54:56.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.636+0000 7fcdcd645700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f3c2a0c320 0x55f3c29fce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:56.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.636+0000 7fcdcd645700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f3c2a0c320 0x55f3c29fce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48506/0 (socket says 172.21.15.52:48506) 2024-06-10T02:54:56.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.636+0000 7fcdcd645700 1 -- 172.21.15.52:0/4146779936 learned_addr learned my addr 172.21.15.52:0/4146779936 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:54:56.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.636+0000 7fcdcde46700 1 --2- 172.21.15.52:0/4146779936 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f3c2a06b60 0x55f3c2a0d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:56.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.636+0000 7fcdce647700 1 --2- 172.21.15.52:0/4146779936 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f3c2a0cc40 0x55f3c29fd370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:56.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.636+0000 7fcdcd645700 1 -- 172.21.15.52:0/4146779936 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f3c2a0cc40 msgr2=0x55f3c29fd370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:54:56.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.636+0000 7fcdcd645700 1 --2- 172.21.15.52:0/4146779936 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f3c2a0cc40 0x55f3c29fd370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:56.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.636+0000 7fcdcd645700 1 -- 172.21.15.52:0/4146779936 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f3c2a06b60 msgr2=0x55f3c2a0d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:54:56.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.636+0000 7fcdcd645700 1 --2- 172.21.15.52:0/4146779936 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f3c2a06b60 0x55f3c2a0d7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:56.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.636+0000 7fcdcd645700 1 -- 172.21.15.52:0/4146779936 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f3c2a13630 con 0x55f3c2a0c320 2024-06-10T02:54:56.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.637+0000 7fcdce647700 1 --2- 172.21.15.52:0/4146779936 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f3c2a0cc40 0x55f3c29fd370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T02:54:56.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.637+0000 7fcdcd645700 1 --2- 172.21.15.52:0/4146779936 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f3c2a0c320 0x55f3c29fce30 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fcdb8098a20 tx=0x7fcdb801a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37611d2233fbcf94 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:54:56.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.637+0000 7fcdb37fe700 1 -- 172.21.15.52:0/4146779936 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdb80bc040 con 0x55f3c2a0c320 2024-06-10T02:54:56.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.637+0000 7fcdb37fe700 1 -- 172.21.15.52:0/4146779936 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcdb809c5c0 con 0x55f3c2a0c320 2024-06-10T02:54:56.638 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.637+0000 7fcdb37fe700 1 -- 172.21.15.52:0/4146779936 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdb809c820 con 0x55f3c2a0c320 2024-06-10T02:54:56.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.638+0000 7fcdda92db80 1 -- 172.21.15.52:0/4146779936 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f3c2a0c320 msgr2=0x55f3c29fce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:54:56.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.638+0000 7fcdda92db80 1 --2- 172.21.15.52:0/4146779936 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f3c2a0c320 0x55f3c29fce30 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fcdb8098a20 tx=0x7fcdb801a040 comp rx=0 tx=0).stop 2024-06-10T02:54:56.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.638+0000 7fcdda92db80 1 -- 172.21.15.52:0/4146779936 shutdown_connections 2024-06-10T02:54:56.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.638+0000 7fcdda92db80 1 --2- 172.21.15.52:0/4146779936 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f3c2a0cc40 0x55f3c29fd370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:56.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.638+0000 7fcdda92db80 1 --2- 172.21.15.52:0/4146779936 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f3c2a06b60 0x55f3c2a0d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:56.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.638+0000 7fcdda92db80 1 --2- 172.21.15.52:0/4146779936 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f3c2a0c320 0x55f3c29fce30 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:56.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.638+0000 7fcdda92db80 1 -- 172.21.15.52:0/4146779936 >> 172.21.15.52:0/4146779936 conn(0x55f3c292f030 msgr2=0x55f3c2a0d0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:54:56.641 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.638+0000 7fcdda92db80 1 -- 172.21.15.52:0/4146779936 shutdown_connections 2024-06-10T02:54:56.641 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.638+0000 7fcdda92db80 1 -- 172.21.15.52:0/4146779936 wait complete. 2024-06-10T02:54:56.641 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.639+0000 7fcdda92db80 1 -- start start 2024-06-10T02:54:56.641 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.639+0000 7fcdda92db80 1 Processor -- start 2024-06-10T02:54:56.641 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.639+0000 7fcdda92db80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f3c2a06b60 0x55f3c2a037f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:56.641 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.639+0000 7fcdda92db80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f3c2a0c320 0x55f3c2a03d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:56.641 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.639+0000 7fcdda92db80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f3c2a0cc40 0x55f3c2a04270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:56.641 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.639+0000 7fcdda92db80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55f3c2a21280 con 0x55f3c2a0c320 2024-06-10T02:54:56.641 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.639+0000 7fcdda92db80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55f3c2a20bb0 con 0x55f3c2a0cc40 2024-06-10T02:54:56.641 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.639+0000 7fcdda92db80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55f3c2a21950 con 0x55f3c2a06b60 2024-06-10T02:54:56.641 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.639+0000 7fcdda92db80 10 client.0 _ll_register_callbacks cb 0x55f3c28e9560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T02:54:56.641 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.639+0000 7fcdcde46700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f3c2a06b60 0x55f3c2a037f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:56.641 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.640+0000 7fcdce647700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f3c2a0cc40 0x55f3c2a04270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:56.641 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.640+0000 7fcdcde46700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f3c2a06b60 0x55f3c2a037f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.52:56198/0 (socket says 172.21.15.52:56198) 2024-06-10T02:54:56.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.640+0000 7fcdcde46700 1 -- 172.21.15.52:0/1268643103 learned_addr learned my addr 172.21.15.52:0/1268643103 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:54:56.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.640+0000 7fcdcd645700 1 --2- 172.21.15.52:0/1268643103 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f3c2a0c320 0x55f3c2a03d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:56.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.640+0000 7fcdce647700 1 -- 172.21.15.52:0/1268643103 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f3c2a06b60 msgr2=0x55f3c2a037f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:54:56.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.640+0000 7fcdce647700 1 --2- 172.21.15.52:0/1268643103 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f3c2a06b60 0x55f3c2a037f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:56.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.640+0000 7fcdce647700 1 -- 172.21.15.52:0/1268643103 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f3c2a0c320 msgr2=0x55f3c2a03d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:54:56.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.640+0000 7fcdce647700 1 --2- 172.21.15.52:0/1268643103 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f3c2a0c320 0x55f3c2a03d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:56.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.640+0000 7fcdce647700 1 -- 172.21.15.52:0/1268643103 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3c2a2aa10 con 0x55f3c2a0cc40 2024-06-10T02:54:56.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.640+0000 7fcdce647700 1 --2- 172.21.15.52:0/1268643103 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f3c2a0cc40 0x55f3c2a04270 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fcdc0000f80 tx=0x7fcdc0007ac0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4ae93346fca77af6 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:54:56.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.640+0000 7fcdb17fa700 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-06-10T02:54:56.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.641+0000 7fcdb17fa700 1 -- 172.21.15.52:0/1268643103 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdc0022070 con 0x55f3c2a0cc40 2024-06-10T02:54:56.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.641+0000 7fcdb17fa700 1 -- 172.21.15.52:0/1268643103 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7fcdc0027670 con 0x55f3c2a0cc40 2024-06-10T02:54:56.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.641+0000 7fcdda92db80 1 -- 172.21.15.52:0/1268643103 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcdb809c5c0 con 0x55f3c2a0cc40 2024-06-10T02:54:56.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.641+0000 7fcdb17fa700 1 -- 172.21.15.52:0/1268643103 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fcdc0007e60 con 0x55f3c2a0cc40 2024-06-10T02:54:56.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.641+0000 7fcdda92db80 10 client.25154 fetch_fsmap learned FSMap version 30 2024-06-10T02:54:56.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.641+0000 7fcdda92db80 1 -- 172.21.15.52:0/1268643103 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f3c2a90580 con 0x55f3c2a0cc40 2024-06-10T02:54:56.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.641+0000 7fcdb17fa700 1 -- 172.21.15.52:0/1268643103 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcdb809c5c0 con 0x55f3c2a0cc40 2024-06-10T02:54:56.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.641+0000 7fcdb17fa700 1 -- 172.21.15.52:0/1268643103 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fcdc001ecb0 con 0x55f3c2a0cc40 2024-06-10T02:54:56.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.642+0000 7fcdda92db80 10 client.25154 fetch_fsmap finished waiting for FSMap version 30 2024-06-10T02:54:56.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.642+0000 7fcdda92db80 10 client.25154 resolve_mds: resolved name 'c' to daemon mds.c 2024-06-10T02:54:56.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.642+0000 7fcdda92db80 20 client.25154 populate_metadata read hostname 'smithi052' 2024-06-10T02:54:56.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.642+0000 7fcdda92db80 1 --2- 172.21.15.52:0/1268643103 >> [v2:172.21.15.188:6836/819317726,v1:172.21.15.188:6837/819317726] conn(0x55f3c2a95060 0x55f3c2a93d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:56.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.642+0000 7fcdda92db80 4 client.25154 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "status"}] 2024-06-10T02:54:56.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.642+0000 7fcdda92db80 1 -- 172.21.15.52:0/1268643103 --> [v2:172.21.15.188:6836/819317726,v1:172.21.15.188:6837/819317726] -- command(tid 0: {"prefix": "status"}) -- 0x55f3c2960d00 con 0x55f3c2a95060 2024-06-10T02:54:56.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.642+0000 7fcdcde46700 1 --2- 172.21.15.52:0/1268643103 >> [v2:172.21.15.188:6836/819317726,v1:172.21.15.188:6837/819317726] conn(0x55f3c2a95060 0x55f3c2a93d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:56.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.642+0000 7fcdcde46700 1 --2- 172.21.15.52:0/1268643103 >> [v2:172.21.15.188:6836/819317726,v1:172.21.15.188:6837/819317726] conn(0x55f3c2a95060 0x55f3c2a93d70 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=65d37f12a92ae38a server_cookie=c660f757769ca1c5 in_seq=0 out_seq=0 2024-06-10T02:54:56.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.642+0000 7fcdb17fa700 10 client.25154 ms_handle_connect on v2:172.21.15.188:6836/819317726 2024-06-10T02:54:56.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.643+0000 7fcdb17fa700 1 -- 172.21.15.52:0/1268643103 <== mds.1 v2:172.21.15.188:6836/819317726 1 ==== command_reply(tid 0: 0 ) ==== 8+0+334 (crc 0 0 0) 0x55f3c2960d00 con 0x55f3c2a95060 2024-06-10T02:54:56.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.643+0000 7fcdb17fa700 10 client.25154 handle_command_reply: tid=0 2024-06-10T02:54:56.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.643+0000 7fcdda92db80 1 client.25154 shutdown 2024-06-10T02:54:56.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.643+0000 7fcdda92db80 1 -- 172.21.15.52:0/1268643103 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f3c2a0cc40 msgr2=0x55f3c2a04270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:54:56.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.643+0000 7fcdda92db80 1 --2- 172.21.15.52:0/1268643103 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f3c2a0cc40 0x55f3c2a04270 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fcdc0000f80 tx=0x7fcdc0007ac0 comp rx=0 tx=0).stop 2024-06-10T02:54:56.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.643+0000 7fcdda92db80 1 -- 172.21.15.52:0/1268643103 shutdown_connections 2024-06-10T02:54:56.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.643+0000 7fcdda92db80 1 --2- 172.21.15.52:0/1268643103 >> [v2:172.21.15.188:6836/819317726,v1:172.21.15.188:6837/819317726] conn(0x55f3c2a95060 0x55f3c2a93d70 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:56.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.643+0000 7fcdda92db80 1 --2- 172.21.15.52:0/1268643103 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f3c2a0cc40 0x55f3c2a04270 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:56.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.643+0000 7fcdda92db80 1 --2- 172.21.15.52:0/1268643103 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f3c2a06b60 0x55f3c2a037f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:56.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.643+0000 7fcdda92db80 1 --2- 172.21.15.52:0/1268643103 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f3c2a0c320 0x55f3c2a03d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:56.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.643+0000 7fcdb17fa700 0 client.25154 ms_handle_reset on v2:172.21.15.188:6836/819317726 2024-06-10T02:54:56.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.643+0000 7fcdda92db80 1 -- 172.21.15.52:0/1268643103 >> 172.21.15.52:0/1268643103 conn(0x55f3c292f030 msgr2=0x55f3c2a11420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:54:56.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.643+0000 7fcdda92db80 1 -- 172.21.15.52:0/1268643103 shutdown_connections 2024-06-10T02:54:56.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.643+0000 7fcdda92db80 1 -- 172.21.15.52:0/1268643103 wait complete. 2024-06-10T02:54:56.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.643+0000 7fcdda92db80 20 client.25154 trim_cache size 0 max 16384 2024-06-10T02:54:56.647 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-10T02:54:56.647 INFO:teuthology.orchestra.run.smithi052.stdout: "cluster_fsid": "d8676e9e-26d2-11ef-bc9b-c7b262605968", 2024-06-10T02:54:56.647 INFO:teuthology.orchestra.run.smithi052.stdout: "whoami": 1, 2024-06-10T02:54:56.647 INFO:teuthology.orchestra.run.smithi052.stdout: "id": 24469, 2024-06-10T02:54:56.647 INFO:teuthology.orchestra.run.smithi052.stdout: "want_state": "up:active", 2024-06-10T02:54:56.647 INFO:teuthology.orchestra.run.smithi052.stdout: "state": "up:active", 2024-06-10T02:54:56.647 INFO:teuthology.orchestra.run.smithi052.stdout: "fs_name": "cephfs", 2024-06-10T02:54:56.647 INFO:teuthology.orchestra.run.smithi052.stdout: "rank_uptime": 231.04764511400001, 2024-06-10T02:54:56.647 INFO:teuthology.orchestra.run.smithi052.stdout: "mdsmap_epoch": 30, 2024-06-10T02:54:56.647 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_epoch": 100, 2024-06-10T02:54:56.647 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_epoch_barrier": 100, 2024-06-10T02:54:56.647 INFO:teuthology.orchestra.run.smithi052.stdout: "uptime": 326.22783492500002 2024-06-10T02:54:56.647 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-10T02:54:56.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.644+0000 7fcdd00aa700 1 -- 172.21.15.52:0/1248766839 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fcdb40a2d30 msgr2=0x7fcdb40a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:54:56.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.644+0000 7fcdd00aa700 1 --2- 172.21.15.52:0/1248766839 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fcdb40a2d30 0x7fcdb40a51f0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fcdc8199b90 tx=0x7fcdc4009260 comp rx=0 tx=0).stop 2024-06-10T02:54:56.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.644+0000 7fcdd00aa700 1 -- 172.21.15.52:0/1248766839 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcdc806b3c0 msgr2=0x7fcdc819edd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:54:56.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.644+0000 7fcdd00aa700 1 --2- 172.21.15.52:0/1248766839 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcdc806b3c0 0x7fcdc819edd0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fcdb8000c00 tx=0x7fcdb800bab0 comp rx=0 tx=0).stop 2024-06-10T02:54:56.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.644+0000 7fcdd00aa700 1 -- 172.21.15.52:0/1248766839 shutdown_connections 2024-06-10T02:54:56.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.644+0000 7fcdd00aa700 1 --2- 172.21.15.52:0/1248766839 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcdc81084f0 0x7fcdc8198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:56.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.644+0000 7fcdd00aa700 1 --2- 172.21.15.52:0/1248766839 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcdc806aa00 0x7fcdc819e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:56.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.644+0000 7fcdd00aa700 1 --2- 172.21.15.52:0/1248766839 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fcdb40a2d30 0x7fcdb40a51f0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:56.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.644+0000 7fcdd00aa700 1 --2- 172.21.15.52:0/1248766839 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcdc806b3c0 0x7fcdc819edd0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:56.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.644+0000 7fcdd00aa700 1 -- 172.21.15.52:0/1248766839 >> 172.21.15.52:0/1248766839 conn(0x7fcdc8078300 msgr2=0x7fcdc80fff70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:54:56.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.644+0000 7fcdd00aa700 1 -- 172.21.15.52:0/1248766839 shutdown_connections 2024-06-10T02:54:56.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:56.644+0000 7fcdd00aa700 1 -- 172.21.15.52:0/1248766839 wait complete. 2024-06-10T02:54:56.963 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:54:56 smithi052 ceph-mon[32362]: pgmap v456: 129 pgs: 129 active+clean; 661 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 853 B/s wr, 8 op/s 2024-06-10T02:54:57.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:54:56 smithi120 ceph-mon[36634]: pgmap v456: 129 pgs: 129 active+clean; 661 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 853 B/s wr, 8 op/s 2024-06-10T02:54:57.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:54:56 smithi188 ceph-mon[36599]: pgmap v456: 129 pgs: 129 active+clean; 661 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 853 B/s wr, 8 op/s 2024-06-10T02:54:57.352 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph tell mds.i status 2024-06-10T02:54:57.620 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:54:58.339 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-10T02:54:58.833 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:54:59.068 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:54:58 smithi052 ceph-mon[32362]: pgmap v457: 129 pgs: 129 active+clean; 661 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 853 B/s wr, 12 op/s 2024-06-10T02:54:59.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:54:58 smithi120 ceph-mon[36634]: pgmap v457: 129 pgs: 129 active+clean; 661 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 853 B/s wr, 12 op/s 2024-06-10T02:54:59.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:54:58 smithi188 ceph-mon[36599]: pgmap v457: 129 pgs: 129 active+clean; 661 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 853 B/s wr, 12 op/s 2024-06-10T02:54:59.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.327+0000 7f202ead3700 1 -- 172.21.15.52:0/1743793206 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2028108500 msgr2=0x7f20281088e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:54:59.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.327+0000 7f202ead3700 1 --2- 172.21.15.52:0/1743793206 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2028108500 0x7f20281088e0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f201c009990 tx=0x7f201c009ca0 comp rx=0 tx=0).stop 2024-06-10T02:54:59.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.327+0000 7f202ead3700 1 -- 172.21.15.52:0/1743793206 shutdown_connections 2024-06-10T02:54:59.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.327+0000 7f202ead3700 1 --2- 172.21.15.52:0/1743793206 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f202806ab30 0x7f202810f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:59.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.327+0000 7f202ead3700 1 --2- 172.21.15.52:0/1743793206 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2028108500 0x7f20281088e0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:59.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.327+0000 7f202ead3700 1 --2- 172.21.15.52:0/1743793206 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f202806a170 0x7f202806a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:59.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.327+0000 7f202ead3700 1 -- 172.21.15.52:0/1743793206 >> 172.21.15.52:0/1743793206 conn(0x7f2028078340 msgr2=0x7f2028078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:54:59.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.328+0000 7f202ead3700 1 -- 172.21.15.52:0/1743793206 shutdown_connections 2024-06-10T02:54:59.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.328+0000 7f202ead3700 1 -- 172.21.15.52:0/1743793206 wait complete. 2024-06-10T02:54:59.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.328+0000 7f202ead3700 1 Processor -- start 2024-06-10T02:54:59.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.328+0000 7f202ead3700 1 -- start start 2024-06-10T02:54:59.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.329+0000 7f202ead3700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f202806a170 0x7f202819e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:59.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.329+0000 7f202ead3700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f202806ab30 0x7f202819ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:59.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.329+0000 7f202ead3700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2028108500 0x7f2028198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:59.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.329+0000 7f202ead3700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f2028112780 con 0x7f202806ab30 2024-06-10T02:54:59.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.329+0000 7f202ead3700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f2028112600 con 0x7f2028108500 2024-06-10T02:54:59.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.329+0000 7f202ead3700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f2028112900 con 0x7f202806a170 2024-06-10T02:54:59.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.329+0000 7f2027fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f202806ab30 0x7f202819ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:59.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.329+0000 7f2027fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f202806ab30 0x7f202819ede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48544/0 (socket says 172.21.15.52:48544) 2024-06-10T02:54:59.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.329+0000 7f2027fff700 1 -- 172.21.15.52:0/3661645874 learned_addr learned my addr 172.21.15.52:0/3661645874 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:54:59.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.329+0000 7f202c86f700 1 --2- 172.21.15.52:0/3661645874 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f202806a170 0x7f202819e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:59.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.329+0000 7f2027fff700 1 -- 172.21.15.52:0/3661645874 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f202806a170 msgr2=0x7f202819e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:54:59.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.329+0000 7f202d070700 1 --2- 172.21.15.52:0/3661645874 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2028108500 0x7f2028198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:59.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.329+0000 7f2027fff700 1 --2- 172.21.15.52:0/3661645874 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f202806a170 0x7f202819e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:59.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.329+0000 7f2027fff700 1 -- 172.21.15.52:0/3661645874 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2028108500 msgr2=0x7f2028198a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:54:59.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.329+0000 7f2027fff700 1 --2- 172.21.15.52:0/3661645874 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2028108500 0x7f2028198a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:59.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.329+0000 7f2027fff700 1 -- 172.21.15.52:0/3661645874 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20281991c0 con 0x7f202806ab30 2024-06-10T02:54:59.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.329+0000 7f202d070700 1 --2- 172.21.15.52:0/3661645874 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2028108500 0x7f2028198a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T02:54:59.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.330+0000 7f2027fff700 1 --2- 172.21.15.52:0/3661645874 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f202806ab30 0x7f202819ede0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f201400e9c0 tx=0x7f201400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:54:59.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.330+0000 7f2025ffb700 1 -- 172.21.15.52:0/3661645874 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f201400cbd0 con 0x7f202806ab30 2024-06-10T02:54:59.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.330+0000 7f202ead3700 1 -- 172.21.15.52:0/3661645874 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20281994b0 con 0x7f202806ab30 2024-06-10T02:54:59.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.330+0000 7f2025ffb700 1 -- 172.21.15.52:0/3661645874 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f201400cd70 con 0x7f202806ab30 2024-06-10T02:54:59.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.330+0000 7f2025ffb700 1 -- 172.21.15.52:0/3661645874 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2014010430 con 0x7f202806ab30 2024-06-10T02:54:59.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.330+0000 7f202ead3700 1 -- 172.21.15.52:0/3661645874 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2028101390 con 0x7f202806ab30 2024-06-10T02:54:59.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.333+0000 7f2025ffb700 1 -- 172.21.15.52:0/3661645874 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7f2014004750 con 0x7f202806ab30 2024-06-10T02:54:59.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.334+0000 7f2039356b80 1 Processor -- start 2024-06-10T02:54:59.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.334+0000 7f2039356b80 1 -- start start 2024-06-10T02:54:59.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.334+0000 7f2039356b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563cecda25f0 0x563cecda29d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:59.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.334+0000 7f2039356b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563cecda2f10 0x563cecd9c990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:59.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.334+0000 7f2025ffb700 1 --2- 172.21.15.52:0/3661645874 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f20180a2d30 0x7f20180a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:59.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.334+0000 7f2039356b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563cecd9ced0 0x563cecd9d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:59.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.334+0000 7f2039356b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x563ceccee7c0 con 0x563cecda25f0 2024-06-10T02:54:59.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.334+0000 7f2039356b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x563cecc966d0 con 0x563cecda2f10 2024-06-10T02:54:59.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.334+0000 7f2039356b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x563ceccea780 con 0x563cecd9ced0 2024-06-10T02:54:59.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.334+0000 7f202d070700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563cecda25f0 0x563cecda29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:59.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.334+0000 7f202d070700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563cecda25f0 0x563cecda29d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48546/0 (socket says 172.21.15.52:48546) 2024-06-10T02:54:59.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.335+0000 7f202d070700 1 -- 172.21.15.52:0/281600249 learned_addr learned my addr 172.21.15.52:0/281600249 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:54:59.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.335+0000 7f202c86f700 1 --2- 172.21.15.52:0/3661645874 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f20180a2d30 0x7f20180a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:59.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.335+0000 7f2025ffb700 1 -- 172.21.15.52:0/3661645874 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f20140cf7c0 con 0x7f202806ab30 2024-06-10T02:54:59.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.335+0000 7f2027fff700 1 --2- 172.21.15.52:0/281600249 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563cecda2f10 0x563cecd9c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:59.336 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.335+0000 7f202d070700 1 --2- 172.21.15.52:0/281600249 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563cecd9ced0 0x563cecd9d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:59.336 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.335+0000 7f202d070700 1 -- 172.21.15.52:0/281600249 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563cecd9ced0 msgr2=0x563cecd9d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:54:59.336 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.335+0000 7f202d070700 1 --2- 172.21.15.52:0/281600249 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563cecd9ced0 0x563cecd9d2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:59.336 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.335+0000 7f202d070700 1 -- 172.21.15.52:0/281600249 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563cecda2f10 msgr2=0x563cecd9c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:54:59.336 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.335+0000 7f202d070700 1 --2- 172.21.15.52:0/281600249 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563cecda2f10 0x563cecd9c990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:59.336 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.335+0000 7f202d070700 1 -- 172.21.15.52:0/281600249 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563cecc8cf00 con 0x563cecda25f0 2024-06-10T02:54:59.336 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.335+0000 7f202c86f700 1 --2- 172.21.15.52:0/3661645874 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f20180a2d30 0x7f20180a51f0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f201c00b2e0 tx=0x7f201c01a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:54:59.336 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.335+0000 7f202d070700 1 --2- 172.21.15.52:0/281600249 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563cecda25f0 0x563cecda29d0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f2020009f80 tx=0x7f2020009470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72bbb40737639e9a server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:54:59.336 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.335+0000 7f2012ffd700 1 -- 172.21.15.52:0/281600249 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f202000f050 con 0x563cecda25f0 2024-06-10T02:54:59.336 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.335+0000 7f2012ffd700 1 -- 172.21.15.52:0/281600249 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2020013040 con 0x563cecda25f0 2024-06-10T02:54:59.336 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.335+0000 7f2012ffd700 1 -- 172.21.15.52:0/281600249 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2020012040 con 0x563cecda25f0 2024-06-10T02:54:59.339 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.336+0000 7f2039356b80 1 -- 172.21.15.52:0/281600249 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563cecda25f0 msgr2=0x563cecda29d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:54:59.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.336+0000 7f2039356b80 1 --2- 172.21.15.52:0/281600249 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563cecda25f0 0x563cecda29d0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f2020009f80 tx=0x7f2020009470 comp rx=0 tx=0).stop 2024-06-10T02:54:59.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.336+0000 7f2039356b80 1 -- 172.21.15.52:0/281600249 shutdown_connections 2024-06-10T02:54:59.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.336+0000 7f2039356b80 1 --2- 172.21.15.52:0/281600249 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563cecd9ced0 0x563cecd9d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:59.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.336+0000 7f2039356b80 1 --2- 172.21.15.52:0/281600249 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563cecda25f0 0x563cecda29d0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:59.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.336+0000 7f2039356b80 1 --2- 172.21.15.52:0/281600249 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563cecda2f10 0x563cecd9c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:59.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.336+0000 7f2039356b80 1 -- 172.21.15.52:0/281600249 >> 172.21.15.52:0/281600249 conn(0x563ceccc7c70 msgr2=0x563cecd97800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:54:59.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.336+0000 7f2039356b80 1 -- 172.21.15.52:0/281600249 shutdown_connections 2024-06-10T02:54:59.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.336+0000 7f2039356b80 1 -- 172.21.15.52:0/281600249 wait complete. 2024-06-10T02:54:59.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.337+0000 7f2039356b80 1 -- start start 2024-06-10T02:54:59.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.337+0000 7f2039356b80 1 Processor -- start 2024-06-10T02:54:59.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.337+0000 7f2039356b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563cecd9ced0 0x563cecda8c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:59.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.337+0000 7f2039356b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563cecda25f0 0x563cecda5fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:59.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.337+0000 7f2039356b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563cecda2f10 0x563cecda6520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:59.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.337+0000 7f2039356b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x563cecda9980 con 0x563cecd9ced0 2024-06-10T02:54:59.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.337+0000 7f2039356b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x563cecc8eba0 con 0x563cecda25f0 2024-06-10T02:54:59.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.337+0000 7f2039356b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x563cecda9b00 con 0x563cecda2f10 2024-06-10T02:54:59.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.338+0000 7f2039356b80 10 client.0 _ll_register_callbacks cb 0x563cecc888d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T02:54:59.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.338+0000 7f202c86f700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563cecd9ced0 0x563cecda8c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:59.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.338+0000 7f202c86f700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563cecd9ced0 0x563cecda8c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48550/0 (socket says 172.21.15.52:48550) 2024-06-10T02:54:59.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.338+0000 7f202c86f700 1 -- 172.21.15.52:0/783049731 learned_addr learned my addr 172.21.15.52:0/783049731 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:54:59.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.338+0000 7f202d070700 1 --2- 172.21.15.52:0/783049731 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563cecda2f10 0x563cecda6520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:59.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.338+0000 7f2027fff700 1 --2- 172.21.15.52:0/783049731 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563cecda25f0 0x563cecda5fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:59.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.338+0000 7f202c86f700 1 -- 172.21.15.52:0/783049731 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563cecda2f10 msgr2=0x563cecda6520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:54:59.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.338+0000 7f202c86f700 1 --2- 172.21.15.52:0/783049731 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563cecda2f10 0x563cecda6520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:59.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.338+0000 7f202c86f700 1 -- 172.21.15.52:0/783049731 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563cecda25f0 msgr2=0x563cecda5fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:54:59.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.338+0000 7f202c86f700 1 --2- 172.21.15.52:0/783049731 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563cecda25f0 0x563cecda5fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:59.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.338+0000 7f202c86f700 1 -- 172.21.15.52:0/783049731 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563cecc3d880 con 0x563cecd9ced0 2024-06-10T02:54:59.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.338+0000 7f202c86f700 1 --2- 172.21.15.52:0/783049731 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563cecd9ced0 0x563cecda8c20 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f201c015000 tx=0x7f201c009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7248c35fc5045739 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:54:59.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.338+0000 7f2007fff700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-10T02:54:59.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.338+0000 7f2007fff700 1 -- 172.21.15.52:0/783049731 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f201c003b50 con 0x563cecd9ced0 2024-06-10T02:54:59.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.339+0000 7f2039356b80 1 -- 172.21.15.52:0/783049731 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2020013040 con 0x563cecd9ced0 2024-06-10T02:54:59.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.338+0000 7f2007fff700 1 -- 172.21.15.52:0/783049731 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f201c003cf0 con 0x563cecd9ced0 2024-06-10T02:54:59.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.339+0000 7f2007fff700 1 -- 172.21.15.52:0/783049731 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f201c035540 con 0x563cecd9ced0 2024-06-10T02:54:59.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.339+0000 7f2039356b80 10 client.15519 fetch_fsmap learned FSMap version 30 2024-06-10T02:54:59.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.339+0000 7f2039356b80 1 -- 172.21.15.52:0/783049731 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x563cecdb42a0 con 0x563cecd9ced0 2024-06-10T02:54:59.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.339+0000 7f2007fff700 1 -- 172.21.15.52:0/783049731 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2020013040 con 0x563cecd9ced0 2024-06-10T02:54:59.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.339+0000 7f2007fff700 1 -- 172.21.15.52:0/783049731 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f201c02e070 con 0x563cecd9ced0 2024-06-10T02:54:59.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.339+0000 7f2039356b80 10 client.15519 fetch_fsmap finished waiting for FSMap version 30 2024-06-10T02:54:59.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.339+0000 7f2039356b80 10 client.15519 resolve_mds: resolved name 'i' to daemon mds.i 2024-06-10T02:54:59.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.339+0000 7f2039356b80 20 client.15519 populate_metadata read hostname 'smithi052' 2024-06-10T02:54:59.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.340+0000 7f2039356b80 1 --2- 172.21.15.52:0/783049731 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x563cecdb6880 0x563cecdb7cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:59.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.340+0000 7f2039356b80 4 client.15519 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T02:54:59.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.340+0000 7f2039356b80 1 -- 172.21.15.52:0/783049731 --> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563ceccfbcc0 con 0x563cecdb6880 2024-06-10T02:54:59.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.340+0000 7f2027fff700 1 --2- 172.21.15.52:0/783049731 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x563cecdb6880 0x563cecdb7cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:59.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.340+0000 7f2007fff700 10 client.15519 ms_handle_connect on v2:172.21.15.188:6838/310444836 2024-06-10T02:54:59.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.340+0000 7f2027fff700 1 --2- 172.21.15.52:0/783049731 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x563cecdb6880 0x563cecdb7cf0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50450729869fa76e server_cookie=d6fd2cd1b321d7b8 in_seq=0 out_seq=0 2024-06-10T02:54:59.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.341+0000 7f2007fff700 1 -- 172.21.15.52:0/783049731 <== mds.0 v2:172.21.15.188:6838/310444836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x563ceccfbcc0 con 0x563cecdb6880 2024-06-10T02:54:59.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.341+0000 7f2007fff700 10 client.15519 handle_command_reply: tid=0 2024-06-10T02:54:59.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.341+0000 7f2039356b80 1 client.15519 shutdown 2024-06-10T02:54:59.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.342+0000 7f2039356b80 1 -- 172.21.15.52:0/783049731 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563cecd9ced0 msgr2=0x563cecda8c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:54:59.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.342+0000 7f2039356b80 1 --2- 172.21.15.52:0/783049731 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563cecd9ced0 0x563cecda8c20 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f201c015000 tx=0x7f201c009310 comp rx=0 tx=0).stop 2024-06-10T02:54:59.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.342+0000 7f2039356b80 1 -- 172.21.15.52:0/783049731 shutdown_connections 2024-06-10T02:54:59.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.342+0000 7f2039356b80 1 --2- 172.21.15.52:0/783049731 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x563cecdb6880 0x563cecdb7cf0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:59.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.342+0000 7f2039356b80 1 --2- 172.21.15.52:0/783049731 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563cecda2f10 0x563cecda6520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:59.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.342+0000 7f2039356b80 1 --2- 172.21.15.52:0/783049731 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563cecd9ced0 0x563cecda8c20 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:59.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.342+0000 7f2039356b80 1 --2- 172.21.15.52:0/783049731 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563cecda25f0 0x563cecda5fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:59.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.342+0000 7f2039356b80 1 -- 172.21.15.52:0/783049731 >> 172.21.15.52:0/783049731 conn(0x563ceccc7c70 msgr2=0x563cecd9b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:54:59.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.342+0000 7f2007fff700 0 client.15519 ms_handle_reset on v2:172.21.15.188:6838/310444836 2024-06-10T02:54:59.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.342+0000 7f2039356b80 1 -- 172.21.15.52:0/783049731 shutdown_connections 2024-06-10T02:54:59.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.342+0000 7f2039356b80 1 -- 172.21.15.52:0/783049731 wait complete. 2024-06-10T02:54:59.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.342+0000 7f2039356b80 20 client.15519 trim_cache size 0 max 16384 2024-06-10T02:54:59.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.366+0000 7f2039356b80 1 Processor -- start 2024-06-10T02:54:59.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.366+0000 7f2039356b80 1 -- start start 2024-06-10T02:54:59.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.366+0000 7f2039356b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563cecd9ced0 0x563cecd9c990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:59.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.366+0000 7f2039356b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563cecda25f0 0x563cecda6590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:59.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.366+0000 7f2039356b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563cecda2f10 0x563cecdc1150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:59.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.366+0000 7f2039356b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x563cecdb6030 con 0x563cecda25f0 2024-06-10T02:54:59.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.366+0000 7f2039356b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x563cecdb6700 con 0x563cecda2f10 2024-06-10T02:54:59.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.366+0000 7f2039356b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x563cecdb6dd0 con 0x563cecd9ced0 2024-06-10T02:54:59.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.367+0000 7f2027fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563cecda25f0 0x563cecda6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:59.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.367+0000 7f2027fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563cecda25f0 0x563cecda6590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48554/0 (socket says 172.21.15.52:48554) 2024-06-10T02:54:59.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.367+0000 7f2027fff700 1 -- 172.21.15.52:0/2039473132 learned_addr learned my addr 172.21.15.52:0/2039473132 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:54:59.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.367+0000 7f202c86f700 1 --2- 172.21.15.52:0/2039473132 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563cecd9ced0 0x563cecd9c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:59.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.367+0000 7f202d070700 1 --2- 172.21.15.52:0/2039473132 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563cecda2f10 0x563cecdc1150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:59.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.367+0000 7f2027fff700 1 -- 172.21.15.52:0/2039473132 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563cecd9ced0 msgr2=0x563cecd9c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:54:59.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.367+0000 7f2027fff700 1 --2- 172.21.15.52:0/2039473132 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563cecd9ced0 0x563cecd9c990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:59.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.367+0000 7f2027fff700 1 -- 172.21.15.52:0/2039473132 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563cecda2f10 msgr2=0x563cecdc1150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:54:59.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.367+0000 7f2027fff700 1 --2- 172.21.15.52:0/2039473132 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563cecda2f10 0x563cecdc1150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:59.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.367+0000 7f2027fff700 1 -- 172.21.15.52:0/2039473132 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563cecda6f00 con 0x563cecda25f0 2024-06-10T02:54:59.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.367+0000 7f202c86f700 1 --2- 172.21.15.52:0/2039473132 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563cecd9ced0 0x563cecd9c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T02:54:59.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.367+0000 7f2027fff700 1 --2- 172.21.15.52:0/2039473132 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563cecda25f0 0x563cecda6590 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f2014097f70 tx=0x7f2014009880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a132f73d2d42612c server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:54:59.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.367+0000 7f20127fc700 1 -- 172.21.15.52:0/2039473132 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f201409c210 con 0x563cecda25f0 2024-06-10T02:54:59.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.368+0000 7f20127fc700 1 -- 172.21.15.52:0/2039473132 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f201409bb10 con 0x563cecda25f0 2024-06-10T02:54:59.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.368+0000 7f20127fc700 1 -- 172.21.15.52:0/2039473132 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2014004a40 con 0x563cecda25f0 2024-06-10T02:54:59.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.368+0000 7f2039356b80 1 -- 172.21.15.52:0/2039473132 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563cecda25f0 msgr2=0x563cecda6590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:54:59.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.368+0000 7f2039356b80 1 --2- 172.21.15.52:0/2039473132 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563cecda25f0 0x563cecda6590 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f2014097f70 tx=0x7f2014009880 comp rx=0 tx=0).stop 2024-06-10T02:54:59.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.368+0000 7f2039356b80 1 -- 172.21.15.52:0/2039473132 shutdown_connections 2024-06-10T02:54:59.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.368+0000 7f2039356b80 1 --2- 172.21.15.52:0/2039473132 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563cecda2f10 0x563cecdc1150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:59.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.368+0000 7f2039356b80 1 --2- 172.21.15.52:0/2039473132 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563cecd9ced0 0x563cecd9c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:59.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.368+0000 7f2039356b80 1 --2- 172.21.15.52:0/2039473132 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563cecda25f0 0x563cecda6590 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:59.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.368+0000 7f2039356b80 1 -- 172.21.15.52:0/2039473132 >> 172.21.15.52:0/2039473132 conn(0x563ceccc7c70 msgr2=0x563cecda4910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:54:59.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.368+0000 7f2039356b80 1 -- 172.21.15.52:0/2039473132 shutdown_connections 2024-06-10T02:54:59.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.368+0000 7f2039356b80 1 -- 172.21.15.52:0/2039473132 wait complete. 2024-06-10T02:54:59.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.369+0000 7f2039356b80 1 -- start start 2024-06-10T02:54:59.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.369+0000 7f2039356b80 1 Processor -- start 2024-06-10T02:54:59.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.369+0000 7f2039356b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563cecd9ced0 0x563cecda5b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:59.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.369+0000 7f2039356b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563cecda25f0 0x563cecda6060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:59.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.369+0000 7f2039356b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563cecda2f10 0x563cecda65a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:59.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.369+0000 7f2039356b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x563cecdb7b70 con 0x563cecda25f0 2024-06-10T02:54:59.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.369+0000 7f2039356b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x563cecdb74a0 con 0x563cecda2f10 2024-06-10T02:54:59.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.369+0000 7f2039356b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x563cecdb8240 con 0x563cecd9ced0 2024-06-10T02:54:59.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.369+0000 7f2039356b80 10 client.0 _ll_register_callbacks cb 0x563cecc888d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T02:54:59.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.369+0000 7f2027fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563cecda25f0 0x563cecda6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:59.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.369+0000 7f2027fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563cecda25f0 0x563cecda6060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48558/0 (socket says 172.21.15.52:48558) 2024-06-10T02:54:59.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.369+0000 7f2027fff700 1 -- 172.21.15.52:0/1602361174 learned_addr learned my addr 172.21.15.52:0/1602361174 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:54:59.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.369+0000 7f202c86f700 1 --2- 172.21.15.52:0/1602361174 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563cecd9ced0 0x563cecda5b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:59.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.370+0000 7f202d070700 1 --2- 172.21.15.52:0/1602361174 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563cecda2f10 0x563cecda65a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:59.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.370+0000 7f202c86f700 1 -- 172.21.15.52:0/1602361174 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563cecda2f10 msgr2=0x563cecda65a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:54:59.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.370+0000 7f202c86f700 1 --2- 172.21.15.52:0/1602361174 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563cecda2f10 0x563cecda65a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:59.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.370+0000 7f202c86f700 1 -- 172.21.15.52:0/1602361174 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563cecda25f0 msgr2=0x563cecda6060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:54:59.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.370+0000 7f202c86f700 1 --2- 172.21.15.52:0/1602361174 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563cecda25f0 0x563cecda6060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:59.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.370+0000 7f202c86f700 1 -- 172.21.15.52:0/1602361174 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563cecdbfad0 con 0x563cecd9ced0 2024-06-10T02:54:59.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.370+0000 7f2027fff700 1 --2- 172.21.15.52:0/1602361174 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563cecda25f0 0x563cecda6060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T02:54:59.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.370+0000 7f202c86f700 1 --2- 172.21.15.52:0/1602361174 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563cecd9ced0 0x563cecda5b20 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f201c00fe50 tx=0x7f201c035380 comp rx=0 tx=0).ready entity=mon.1 client_cookie=728f8f6e13e67d8d server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:54:59.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.370+0000 7f2007fff700 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-06-10T02:54:59.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.370+0000 7f2007fff700 1 -- 172.21.15.52:0/1602361174 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f201c042060 con 0x563cecd9ced0 2024-06-10T02:54:59.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.370+0000 7f2039356b80 1 -- 172.21.15.52:0/1602361174 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f201409bb10 con 0x563cecd9ced0 2024-06-10T02:54:59.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.370+0000 7f2007fff700 1 -- 172.21.15.52:0/1602361174 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f201c003d60 con 0x563cecd9ced0 2024-06-10T02:54:59.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.371+0000 7f2007fff700 1 -- 172.21.15.52:0/1602361174 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f201c049050 con 0x563cecd9ced0 2024-06-10T02:54:59.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.371+0000 7f2039356b80 10 client.25156 fetch_fsmap learned FSMap version 30 2024-06-10T02:54:59.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.371+0000 7f2039356b80 1 -- 172.21.15.52:0/1602361174 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x563cece225f0 con 0x563cecd9ced0 2024-06-10T02:54:59.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.371+0000 7f2007fff700 1 -- 172.21.15.52:0/1602361174 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f201409bb10 con 0x563cecd9ced0 2024-06-10T02:54:59.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.371+0000 7f2007fff700 1 -- 172.21.15.52:0/1602361174 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f201c03c030 con 0x563cecd9ced0 2024-06-10T02:54:59.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.371+0000 7f2039356b80 10 client.25156 fetch_fsmap finished waiting for FSMap version 30 2024-06-10T02:54:59.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.371+0000 7f2039356b80 10 client.25156 resolve_mds: resolved name 'i' to daemon mds.i 2024-06-10T02:54:59.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.371+0000 7f2039356b80 20 client.25156 populate_metadata read hostname 'smithi052' 2024-06-10T02:54:59.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.371+0000 7f2039356b80 1 --2- 172.21.15.52:0/1602361174 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x563cece27170 0x563cece25e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:54:59.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.371+0000 7f2039356b80 4 client.25156 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "status"}] 2024-06-10T02:54:59.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.371+0000 7f2039356b80 1 -- 172.21.15.52:0/1602361174 --> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] -- command(tid 0: {"prefix": "status"}) -- 0x563ceccf6240 con 0x563cece27170 2024-06-10T02:54:59.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.372+0000 7f2027fff700 1 --2- 172.21.15.52:0/1602361174 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x563cece27170 0x563cece25e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:54:59.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.372+0000 7f2027fff700 1 --2- 172.21.15.52:0/1602361174 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x563cece27170 0x563cece25e80 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ebd64b5ad4851d5a server_cookie=26ded82745071b71 in_seq=0 out_seq=0 2024-06-10T02:54:59.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.372+0000 7f2007fff700 10 client.25156 ms_handle_connect on v2:172.21.15.188:6838/310444836 2024-06-10T02:54:59.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.372+0000 7f2007fff700 1 -- 172.21.15.52:0/1602361174 <== mds.0 v2:172.21.15.188:6838/310444836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+334 (crc 0 0 0) 0x563ceccf6240 con 0x563cece27170 2024-06-10T02:54:59.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.372+0000 7f2007fff700 10 client.25156 handle_command_reply: tid=0 2024-06-10T02:54:59.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.372+0000 7f2039356b80 1 client.25156 shutdown 2024-06-10T02:54:59.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.373+0000 7f2039356b80 1 -- 172.21.15.52:0/1602361174 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563cecd9ced0 msgr2=0x563cecda5b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:54:59.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.373+0000 7f2039356b80 1 --2- 172.21.15.52:0/1602361174 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563cecd9ced0 0x563cecda5b20 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f201c00fe50 tx=0x7f201c035380 comp rx=0 tx=0).stop 2024-06-10T02:54:59.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.373+0000 7f2039356b80 1 -- 172.21.15.52:0/1602361174 shutdown_connections 2024-06-10T02:54:59.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.373+0000 7f2039356b80 1 --2- 172.21.15.52:0/1602361174 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x563cece27170 0x563cece25e80 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:59.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.373+0000 7f2039356b80 1 --2- 172.21.15.52:0/1602361174 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563cecda2f10 0x563cecda65a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:59.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.373+0000 7f2039356b80 1 --2- 172.21.15.52:0/1602361174 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563cecd9ced0 0x563cecda5b20 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:59.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.373+0000 7f2007fff700 0 client.25156 ms_handle_reset on v2:172.21.15.188:6838/310444836 2024-06-10T02:54:59.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.373+0000 7f2039356b80 1 --2- 172.21.15.52:0/1602361174 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563cecda25f0 0x563cecda6060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:59.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.373+0000 7f2039356b80 1 -- 172.21.15.52:0/1602361174 >> 172.21.15.52:0/1602361174 conn(0x563ceccc7c70 msgr2=0x563cecd90db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:54:59.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.373+0000 7f2039356b80 1 -- 172.21.15.52:0/1602361174 shutdown_connections 2024-06-10T02:54:59.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.373+0000 7f2039356b80 1 -- 172.21.15.52:0/1602361174 wait complete. 2024-06-10T02:54:59.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.373+0000 7f2039356b80 20 client.25156 trim_cache size 0 max 16384 2024-06-10T02:54:59.378 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-10T02:54:59.378 INFO:teuthology.orchestra.run.smithi052.stdout: "cluster_fsid": "d8676e9e-26d2-11ef-bc9b-c7b262605968", 2024-06-10T02:54:59.378 INFO:teuthology.orchestra.run.smithi052.stdout: "whoami": 0, 2024-06-10T02:54:59.378 INFO:teuthology.orchestra.run.smithi052.stdout: "id": 24484, 2024-06-10T02:54:59.378 INFO:teuthology.orchestra.run.smithi052.stdout: "want_state": "up:active", 2024-06-10T02:54:59.378 INFO:teuthology.orchestra.run.smithi052.stdout: "state": "up:active", 2024-06-10T02:54:59.378 INFO:teuthology.orchestra.run.smithi052.stdout: "fs_name": "cephfs", 2024-06-10T02:54:59.379 INFO:teuthology.orchestra.run.smithi052.stdout: "rank_uptime": 295.17868943299999, 2024-06-10T02:54:59.379 INFO:teuthology.orchestra.run.smithi052.stdout: "mdsmap_epoch": 30, 2024-06-10T02:54:59.379 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_epoch": 100, 2024-06-10T02:54:59.379 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_epoch_barrier": 100, 2024-06-10T02:54:59.379 INFO:teuthology.orchestra.run.smithi052.stdout: "uptime": 323.19210643999998 2024-06-10T02:54:59.379 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-10T02:54:59.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.374+0000 7f202ead3700 1 -- 172.21.15.52:0/3661645874 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f20180a2d30 msgr2=0x7f20180a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:54:59.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.374+0000 7f202ead3700 1 --2- 172.21.15.52:0/3661645874 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f20180a2d30 0x7f20180a51f0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f201c00b2e0 tx=0x7f201c01a040 comp rx=0 tx=0).stop 2024-06-10T02:54:59.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.374+0000 7f202ead3700 1 -- 172.21.15.52:0/3661645874 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f202806ab30 msgr2=0x7f202819ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:54:59.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.374+0000 7f202ead3700 1 --2- 172.21.15.52:0/3661645874 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f202806ab30 0x7f202819ede0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f201400e9c0 tx=0x7f201400ecd0 comp rx=0 tx=0).stop 2024-06-10T02:54:59.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.374+0000 7f202ead3700 1 -- 172.21.15.52:0/3661645874 shutdown_connections 2024-06-10T02:54:59.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.374+0000 7f202ead3700 1 --2- 172.21.15.52:0/3661645874 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2028108500 0x7f2028198a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:59.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.374+0000 7f202ead3700 1 --2- 172.21.15.52:0/3661645874 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f202806a170 0x7f202819e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:59.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.374+0000 7f202ead3700 1 --2- 172.21.15.52:0/3661645874 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f20180a2d30 0x7f20180a51f0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:59.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.374+0000 7f202ead3700 1 --2- 172.21.15.52:0/3661645874 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f202806ab30 0x7f202819ede0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:54:59.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.374+0000 7f202ead3700 1 -- 172.21.15.52:0/3661645874 >> 172.21.15.52:0/3661645874 conn(0x7f2028078340 msgr2=0x7f2028100150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:54:59.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.374+0000 7f202ead3700 1 -- 172.21.15.52:0/3661645874 shutdown_connections 2024-06-10T02:54:59.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:54:59.374+0000 7f202ead3700 1 -- 172.21.15.52:0/3661645874 wait complete. 2024-06-10T02:55:00.747 INFO:tasks.quiescer:Ready to start quiesce thrashing; initial delay: 15 sec 2024-06-10T02:55:00.976 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:55:00 smithi052 ceph-mon[32362]: pgmap v458: 129 pgs: 129 active+clean; 661 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-10T02:55:01.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:55:00 smithi120 ceph-mon[36634]: pgmap v458: 129 pgs: 129 active+clean; 661 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-10T02:55:01.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:55:00 smithi188 ceph-mon[36599]: pgmap v458: 129 pgs: 129 active+clean; 661 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-10T02:55:01.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.677+0000 7fc3ea8df700 1 -- 172.21.15.52:0/2047706823 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc3e4101390 msgr2=0x7fc3e4101810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:55:01.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.677+0000 7fc3ea8df700 1 --2- 172.21.15.52:0/2047706823 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc3e4101390 0x7fc3e4101810 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fc3d8009990 tx=0x7fc3d8009ca0 comp rx=0 tx=0).stop 2024-06-10T02:55:01.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.677+0000 7fc3ea8df700 1 -- 172.21.15.52:0/2047706823 shutdown_connections 2024-06-10T02:55:01.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.677+0000 7fc3ea8df700 1 --2- 172.21.15.52:0/2047706823 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc3e4101d50 0x7fc3e410e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:01.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.677+0000 7fc3ea8df700 1 --2- 172.21.15.52:0/2047706823 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc3e4107400 0x7fc3e41077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:01.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.677+0000 7fc3ea8df700 1 --2- 172.21.15.52:0/2047706823 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc3e4101390 0x7fc3e4101810 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:01.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.677+0000 7fc3ea8df700 1 -- 172.21.15.52:0/2047706823 >> 172.21.15.52:0/2047706823 conn(0x7fc3e4078340 msgr2=0x7fc3e4078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:55:01.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.677+0000 7fc3ea8df700 1 -- 172.21.15.52:0/2047706823 shutdown_connections 2024-06-10T02:55:01.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.677+0000 7fc3ea8df700 1 -- 172.21.15.52:0/2047706823 wait complete. 2024-06-10T02:55:01.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.678+0000 7fc3ea8df700 1 Processor -- start 2024-06-10T02:55:01.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.678+0000 7fc3ea8df700 1 -- start start 2024-06-10T02:55:01.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.678+0000 7fc3ea8df700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc3e4101390 0x7fc3e4199ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:01.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.678+0000 7fc3ea8df700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc3e4101d50 0x7fc3e419a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:01.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.678+0000 7fc3ea8df700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc3e4107400 0x7fc3e41a4b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:01.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.678+0000 7fc3ea8df700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fc3e41116c0 con 0x7fc3e4101d50 2024-06-10T02:55:01.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.678+0000 7fc3ea8df700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc3e4111540 con 0x7fc3e4101390 2024-06-10T02:55:01.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.678+0000 7fc3ea8df700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc3e4111840 con 0x7fc3e4107400 2024-06-10T02:55:01.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.678+0000 7fc3e37fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc3e4101d50 0x7fc3e419a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:01.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.678+0000 7fc3e37fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc3e4101d50 0x7fc3e419a000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:42658/0 (socket says 172.21.15.52:42658) 2024-06-10T02:55:01.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.678+0000 7fc3e37fe700 1 -- 172.21.15.52:0/2095635374 learned_addr learned my addr 172.21.15.52:0/2095635374 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:55:01.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.678+0000 7fc3e3fff700 1 --2- 172.21.15.52:0/2095635374 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc3e4101390 0x7fc3e4199ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:01.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.678+0000 7fc3e8e7c700 1 --2- 172.21.15.52:0/2095635374 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc3e4107400 0x7fc3e41a4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:01.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.679+0000 7fc3e37fe700 1 -- 172.21.15.52:0/2095635374 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc3e4107400 msgr2=0x7fc3e41a4b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:55:01.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.679+0000 7fc3e37fe700 1 --2- 172.21.15.52:0/2095635374 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc3e4107400 0x7fc3e41a4b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:01.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.679+0000 7fc3e37fe700 1 -- 172.21.15.52:0/2095635374 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc3e4101390 msgr2=0x7fc3e4199ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:55:01.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.679+0000 7fc3e37fe700 1 --2- 172.21.15.52:0/2095635374 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc3e4101390 0x7fc3e4199ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:01.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.679+0000 7fc3e37fe700 1 -- 172.21.15.52:0/2095635374 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3e41a5080 con 0x7fc3e4101d50 2024-06-10T02:55:01.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.679+0000 7fc3e8e7c700 1 --2- 172.21.15.52:0/2095635374 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc3e4107400 0x7fc3e41a4b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T02:55:01.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.679+0000 7fc3e37fe700 1 --2- 172.21.15.52:0/2095635374 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc3e4101d50 0x7fc3e419a000 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fc3d8000c00 tx=0x7fc3d8003890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:55:01.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.679+0000 7fc3e17fa700 1 -- 172.21.15.52:0/2095635374 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3d8027070 con 0x7fc3e4101d50 2024-06-10T02:55:01.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.679+0000 7fc3ea8df700 1 -- 172.21.15.52:0/2095635374 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3e41a5250 con 0x7fc3e4101d50 2024-06-10T02:55:01.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.679+0000 7fc3e17fa700 1 -- 172.21.15.52:0/2095635374 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc3d8005690 con 0x7fc3e4101d50 2024-06-10T02:55:01.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.679+0000 7fc3e17fa700 1 -- 172.21.15.52:0/2095635374 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3d8022c60 con 0x7fc3e4101d50 2024-06-10T02:55:01.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.679+0000 7fc3ea8df700 1 -- 172.21.15.52:0/2095635374 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3e41a5700 con 0x7fc3e4101d50 2024-06-10T02:55:01.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.682+0000 7fc3e17fa700 1 -- 172.21.15.52:0/2095635374 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7fc3d802a070 con 0x7fc3e4101d50 2024-06-10T02:55:01.684 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.683+0000 7fc3f5162b80 1 Processor -- start 2024-06-10T02:55:01.684 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.683+0000 7fc3f5162b80 1 -- start start 2024-06-10T02:55:01.684 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.683+0000 7fc3f5162b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563859d1f760 0x563859d1b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:01.684 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.683+0000 7fc3f5162b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563859d1bf20 0x563859d1c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:01.684 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.683+0000 7fc3f5162b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563859d2e640 0x563859d30a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:01.684 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.683+0000 7fc3f5162b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x563859c6d810 con 0x563859d2e640 2024-06-10T02:55:01.684 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.683+0000 7fc3f5162b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x563859c15720 con 0x563859d1bf20 2024-06-10T02:55:01.684 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.683+0000 7fc3f5162b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x563859c697d0 con 0x563859d1f760 2024-06-10T02:55:01.684 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.683+0000 7fc3e17fa700 1 --2- 172.21.15.52:0/2095635374 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fc3cc0a2d30 0x7fc3cc0a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:01.684 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.683+0000 7fc3e37fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563859d2e640 0x563859d30a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:01.684 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.683+0000 7fc3e37fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563859d2e640 0x563859d30a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:42660/0 (socket says 172.21.15.52:42660) 2024-06-10T02:55:01.684 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.683+0000 7fc3e37fe700 1 -- 172.21.15.52:0/1137880427 learned_addr learned my addr 172.21.15.52:0/1137880427 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:55:01.685 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.684+0000 7fc3e3fff700 1 --2- 172.21.15.52:0/1137880427 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563859d1bf20 0x563859d1c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:01.685 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.684+0000 7fc3e8e7c700 1 --2- 172.21.15.52:0/2095635374 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fc3cc0a2d30 0x7fc3cc0a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:01.685 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.684+0000 7fc3e8e7c700 1 --2- 172.21.15.52:0/1137880427 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563859d1f760 0x563859d1b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:01.685 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.684+0000 7fc3e37fe700 1 -- 172.21.15.52:0/1137880427 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563859d1f760 msgr2=0x563859d1b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:55:01.685 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.684+0000 7fc3e37fe700 1 --2- 172.21.15.52:0/1137880427 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563859d1f760 0x563859d1b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:01.685 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.684+0000 7fc3e37fe700 1 -- 172.21.15.52:0/1137880427 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563859d1bf20 msgr2=0x563859d1c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:55:01.685 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.684+0000 7fc3e37fe700 1 --2- 172.21.15.52:0/1137880427 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563859d1bf20 0x563859d1c3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:01.685 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.684+0000 7fc3e37fe700 1 -- 172.21.15.52:0/1137880427 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563859c0bf50 con 0x563859d2e640 2024-06-10T02:55:01.685 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.684+0000 7fc3e3fff700 1 --2- 172.21.15.52:0/1137880427 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563859d1bf20 0x563859d1c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T02:55:01.685 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.684+0000 7fc3e17fa700 1 -- 172.21.15.52:0/2095635374 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7fc3d8022e00 con 0x7fc3e4101d50 2024-06-10T02:55:01.685 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.684+0000 7fc3e37fe700 1 --2- 172.21.15.52:0/1137880427 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563859d2e640 0x563859d30a30 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fc3d80991a0 tx=0x7fc3d8020fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1e915df33943324 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:55:01.685 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.684+0000 7fc3e8e7c700 1 --2- 172.21.15.52:0/2095635374 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fc3cc0a2d30 0x7fc3cc0a51f0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fc3dc007fd0 tx=0x7fc3dc007a60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:55:01.685 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.684+0000 7fc3ca7fc700 1 -- 172.21.15.52:0/1137880427 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3d80a30c0 con 0x563859d2e640 2024-06-10T02:55:01.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.684+0000 7fc3ca7fc700 1 -- 172.21.15.52:0/1137880427 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc3d809ced0 con 0x563859d2e640 2024-06-10T02:55:01.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.685+0000 7fc3ca7fc700 1 -- 172.21.15.52:0/1137880427 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3d80a53b0 con 0x563859d2e640 2024-06-10T02:55:01.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.685+0000 7fc3f5162b80 1 -- 172.21.15.52:0/1137880427 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563859d2e640 msgr2=0x563859d30a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:55:01.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.685+0000 7fc3f5162b80 1 --2- 172.21.15.52:0/1137880427 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563859d2e640 0x563859d30a30 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fc3d80991a0 tx=0x7fc3d8020fc0 comp rx=0 tx=0).stop 2024-06-10T02:55:01.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.685+0000 7fc3f5162b80 1 -- 172.21.15.52:0/1137880427 shutdown_connections 2024-06-10T02:55:01.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.685+0000 7fc3f5162b80 1 --2- 172.21.15.52:0/1137880427 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563859d2e640 0x563859d30a30 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:01.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.685+0000 7fc3f5162b80 1 --2- 172.21.15.52:0/1137880427 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563859d1f760 0x563859d1b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:01.689 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.685+0000 7fc3f5162b80 1 --2- 172.21.15.52:0/1137880427 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563859d1bf20 0x563859d1c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:01.689 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.685+0000 7fc3f5162b80 1 -- 172.21.15.52:0/1137880427 >> 172.21.15.52:0/1137880427 conn(0x563859c46cc0 msgr2=0x563859d16850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:55:01.689 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.686+0000 7fc3f5162b80 1 -- 172.21.15.52:0/1137880427 shutdown_connections 2024-06-10T02:55:01.689 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.686+0000 7fc3f5162b80 1 -- 172.21.15.52:0/1137880427 wait complete. 2024-06-10T02:55:01.689 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.686+0000 7fc3f5162b80 1 -- start start 2024-06-10T02:55:01.689 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.686+0000 7fc3f5162b80 1 Processor -- start 2024-06-10T02:55:01.689 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.686+0000 7fc3f5162b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563859d1bf20 0x563859d2dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:01.689 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.686+0000 7fc3f5162b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563859d1f760 0x563859d2b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:01.689 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.686+0000 7fc3f5162b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563859d2e640 0x563859d2b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:01.689 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.687+0000 7fc3f5162b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x563859d30f70 con 0x563859d2e640 2024-06-10T02:55:01.689 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.687+0000 7fc3f5162b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x563859c0dbf0 con 0x563859d1f760 2024-06-10T02:55:01.689 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.687+0000 7fc3f5162b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x563859d310f0 con 0x563859d1bf20 2024-06-10T02:55:01.689 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.687+0000 7fc3f5162b80 10 client.0 _ll_register_callbacks cb 0x563859c07920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T02:55:01.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.687+0000 7fc3e8e7c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563859d2e640 0x563859d2b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:01.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.687+0000 7fc3e8e7c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563859d2e640 0x563859d2b660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:42672/0 (socket says 172.21.15.52:42672) 2024-06-10T02:55:01.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.687+0000 7fc3e8e7c700 1 -- 172.21.15.52:0/3951128964 learned_addr learned my addr 172.21.15.52:0/3951128964 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:55:01.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.687+0000 7fc3e3fff700 1 --2- 172.21.15.52:0/3951128964 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563859d1bf20 0x563859d2dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:01.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.687+0000 7fc3e37fe700 1 --2- 172.21.15.52:0/3951128964 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563859d1f760 0x563859d2b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:01.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.687+0000 7fc3e8e7c700 1 -- 172.21.15.52:0/3951128964 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563859d1bf20 msgr2=0x563859d2dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:55:01.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.687+0000 7fc3e8e7c700 1 --2- 172.21.15.52:0/3951128964 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563859d1bf20 0x563859d2dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:01.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.687+0000 7fc3e8e7c700 1 -- 172.21.15.52:0/3951128964 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563859d1f760 msgr2=0x563859d2b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:55:01.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.687+0000 7fc3e8e7c700 1 --2- 172.21.15.52:0/3951128964 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563859d1f760 0x563859d2b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:01.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.687+0000 7fc3e8e7c700 1 -- 172.21.15.52:0/3951128964 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563859bbc8d0 con 0x563859d2e640 2024-06-10T02:55:01.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.687+0000 7fc3e8e7c700 1 --2- 172.21.15.52:0/3951128964 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563859d2e640 0x563859d2b660 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fc3dc014b30 tx=0x7fc3dc009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=455f5a6cd8d6f69e server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:55:01.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.687+0000 7fc3c37fe700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-10T02:55:01.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.687+0000 7fc3c37fe700 1 -- 172.21.15.52:0/3951128964 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3dc027060 con 0x563859d2e640 2024-06-10T02:55:01.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.687+0000 7fc3c37fe700 1 -- 172.21.15.52:0/3951128964 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7fc3dc017840 con 0x563859d2e640 2024-06-10T02:55:01.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.688+0000 7fc3f5162b80 1 -- 172.21.15.52:0/3951128964 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc3d809ced0 con 0x563859d2e640 2024-06-10T02:55:01.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.688+0000 7fc3c37fe700 1 -- 172.21.15.52:0/3951128964 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fc3dc02e050 con 0x563859d2e640 2024-06-10T02:55:01.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.688+0000 7fc3f5162b80 10 client.15537 fetch_fsmap learned FSMap version 30 2024-06-10T02:55:01.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.688+0000 7fc3f5162b80 1 -- 172.21.15.52:0/3951128964 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x563859d3b800 con 0x563859d2e640 2024-06-10T02:55:01.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.688+0000 7fc3c37fe700 1 -- 172.21.15.52:0/3951128964 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc3d809ced0 con 0x563859d2e640 2024-06-10T02:55:01.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.688+0000 7fc3c37fe700 1 -- 172.21.15.52:0/3951128964 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fc3dc016070 con 0x563859d2e640 2024-06-10T02:55:01.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.688+0000 7fc3f5162b80 10 client.15537 fetch_fsmap finished waiting for FSMap version 30 2024-06-10T02:55:01.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.688+0000 7fc3f5162b80 10 client.15537 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T02:55:01.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.688+0000 7fc3f5162b80 20 client.15537 populate_metadata read hostname 'smithi052' 2024-06-10T02:55:01.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.688+0000 7fc3f5162b80 1 --2- 172.21.15.52:0/3951128964 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x563859d3de40 0x563859d40230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:01.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.688+0000 7fc3f5162b80 4 client.15537 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T02:55:01.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.688+0000 7fc3f5162b80 1 -- 172.21.15.52:0/3951128964 --> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563859c7ad10 con 0x563859d3de40 2024-06-10T02:55:01.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.689+0000 7fc3e3fff700 1 --2- 172.21.15.52:0/3951128964 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x563859d3de40 0x563859d40230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:01.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.689+0000 7fc3c37fe700 10 client.15537 ms_handle_connect on v2:172.21.15.188:6838/310444836 2024-06-10T02:55:01.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.689+0000 7fc3e3fff700 1 --2- 172.21.15.52:0/3951128964 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x563859d3de40 0x563859d40230 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=90aa0b4554ee3286 server_cookie=e17031bd93e8a33 in_seq=0 out_seq=0 2024-06-10T02:55:01.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.690+0000 7fc3c37fe700 1 -- 172.21.15.52:0/3951128964 <== mds.0 v2:172.21.15.188:6838/310444836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x563859c7ad10 con 0x563859d3de40 2024-06-10T02:55:01.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.690+0000 7fc3c37fe700 10 client.15537 handle_command_reply: tid=0 2024-06-10T02:55:01.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.690+0000 7fc3f5162b80 1 client.15537 shutdown 2024-06-10T02:55:01.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.690+0000 7fc3f5162b80 1 -- 172.21.15.52:0/3951128964 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563859d2e640 msgr2=0x563859d2b660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:55:01.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.690+0000 7fc3f5162b80 1 --2- 172.21.15.52:0/3951128964 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563859d2e640 0x563859d2b660 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fc3dc014b30 tx=0x7fc3dc009040 comp rx=0 tx=0).stop 2024-06-10T02:55:01.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.690+0000 7fc3f5162b80 1 -- 172.21.15.52:0/3951128964 shutdown_connections 2024-06-10T02:55:01.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.690+0000 7fc3f5162b80 1 --2- 172.21.15.52:0/3951128964 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x563859d3de40 0x563859d40230 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:01.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.690+0000 7fc3f5162b80 1 --2- 172.21.15.52:0/3951128964 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563859d2e640 0x563859d2b660 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:01.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.690+0000 7fc3f5162b80 1 --2- 172.21.15.52:0/3951128964 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563859d1bf20 0x563859d2dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:01.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.690+0000 7fc3c37fe700 0 client.15537 ms_handle_reset on v2:172.21.15.188:6838/310444836 2024-06-10T02:55:01.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.690+0000 7fc3f5162b80 1 --2- 172.21.15.52:0/3951128964 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563859d1f760 0x563859d2b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:01.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.690+0000 7fc3f5162b80 1 -- 172.21.15.52:0/3951128964 >> 172.21.15.52:0/3951128964 conn(0x563859c46cc0 msgr2=0x563859d2ea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:55:01.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.690+0000 7fc3f5162b80 1 -- 172.21.15.52:0/3951128964 shutdown_connections 2024-06-10T02:55:01.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.690+0000 7fc3f5162b80 1 -- 172.21.15.52:0/3951128964 wait complete. 2024-06-10T02:55:01.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.690+0000 7fc3f5162b80 20 client.15537 trim_cache size 0 max 16384 2024-06-10T02:55:01.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.712+0000 7fc3f5162b80 1 Processor -- start 2024-06-10T02:55:01.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.712+0000 7fc3f5162b80 1 -- start start 2024-06-10T02:55:01.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.712+0000 7fc3f5162b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563859d1bf20 0x563859d2b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:01.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.712+0000 7fc3f5162b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563859d1f760 0x563859d10510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:01.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.712+0000 7fc3f5162b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563859d2e640 0x563859d2fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:01.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.712+0000 7fc3f5162b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fc3e41ac0d0 con 0x563859d2e640 2024-06-10T02:55:01.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.712+0000 7fc3f5162b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc3e41adc10 con 0x563859d1f760 2024-06-10T02:55:01.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.712+0000 7fc3f5162b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc3e41ac7a0 con 0x563859d1bf20 2024-06-10T02:55:01.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.712+0000 7fc3e3fff700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563859d1bf20 0x563859d2b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:01.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.712+0000 7fc3e3fff700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563859d1bf20 0x563859d2b6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.52:37488/0 (socket says 172.21.15.52:37488) 2024-06-10T02:55:01.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.712+0000 7fc3e3fff700 1 -- 172.21.15.52:0/2319530487 learned_addr learned my addr 172.21.15.52:0/2319530487 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:55:01.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.712+0000 7fc3e8e7c700 1 --2- 172.21.15.52:0/2319530487 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563859d2e640 0x563859d2fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:01.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.712+0000 7fc3e37fe700 1 --2- 172.21.15.52:0/2319530487 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563859d1f760 0x563859d10510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:01.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.713+0000 7fc3e37fe700 1 -- 172.21.15.52:0/2319530487 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563859d1bf20 msgr2=0x563859d2b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:55:01.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.713+0000 7fc3e37fe700 1 --2- 172.21.15.52:0/2319530487 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563859d1bf20 0x563859d2b6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:01.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.713+0000 7fc3e37fe700 1 -- 172.21.15.52:0/2319530487 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563859d2e640 msgr2=0x563859d2fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:55:01.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.713+0000 7fc3e37fe700 1 --2- 172.21.15.52:0/2319530487 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563859d2e640 0x563859d2fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:01.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.713+0000 7fc3e37fe700 1 -- 172.21.15.52:0/2319530487 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563859d184b0 con 0x563859d1f760 2024-06-10T02:55:01.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.713+0000 7fc3e3fff700 1 --2- 172.21.15.52:0/2319530487 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563859d1bf20 0x563859d2b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T02:55:01.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.713+0000 7fc3e37fe700 1 --2- 172.21.15.52:0/2319530487 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563859d1f760 0x563859d10510 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fc3d809c4d0 tx=0x7fc3d801a040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c59044dbeef8747 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:55:01.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.713+0000 7fc3c9ffb700 1 -- 172.21.15.52:0/2319530487 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3d809c5a0 con 0x563859d1f760 2024-06-10T02:55:01.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.713+0000 7fc3c9ffb700 1 -- 172.21.15.52:0/2319530487 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc3d80a30b0 con 0x563859d1f760 2024-06-10T02:55:01.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.713+0000 7fc3c9ffb700 1 -- 172.21.15.52:0/2319530487 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3d80a53b0 con 0x563859d1f760 2024-06-10T02:55:01.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.714+0000 7fc3f5162b80 1 -- 172.21.15.52:0/2319530487 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563859d1f760 msgr2=0x563859d10510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:55:01.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.714+0000 7fc3f5162b80 1 --2- 172.21.15.52:0/2319530487 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563859d1f760 0x563859d10510 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fc3d809c4d0 tx=0x7fc3d801a040 comp rx=0 tx=0).stop 2024-06-10T02:55:01.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.714+0000 7fc3f5162b80 1 -- 172.21.15.52:0/2319530487 shutdown_connections 2024-06-10T02:55:01.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.714+0000 7fc3f5162b80 1 --2- 172.21.15.52:0/2319530487 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563859d2e640 0x563859d2fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:01.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.714+0000 7fc3f5162b80 1 --2- 172.21.15.52:0/2319530487 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563859d1bf20 0x563859d2b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:01.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.714+0000 7fc3f5162b80 1 --2- 172.21.15.52:0/2319530487 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563859d1f760 0x563859d10510 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:01.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.714+0000 7fc3f5162b80 1 -- 172.21.15.52:0/2319530487 >> 172.21.15.52:0/2319530487 conn(0x563859c46cc0 msgr2=0x563859d0fe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:55:01.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.714+0000 7fc3f5162b80 1 -- 172.21.15.52:0/2319530487 shutdown_connections 2024-06-10T02:55:01.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.714+0000 7fc3f5162b80 1 -- 172.21.15.52:0/2319530487 wait complete. 2024-06-10T02:55:01.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.715+0000 7fc3f5162b80 1 -- start start 2024-06-10T02:55:01.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.715+0000 7fc3f5162b80 1 Processor -- start 2024-06-10T02:55:01.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.715+0000 7fc3f5162b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563859d1bf20 0x563859d1a240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:01.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.715+0000 7fc3f5162b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563859d1f760 0x563859d36340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:01.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.715+0000 7fc3f5162b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563859d2e640 0x563859d36930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:01.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.715+0000 7fc3f5162b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fc3e41ad540 con 0x563859d1bf20 2024-06-10T02:55:01.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.715+0000 7fc3f5162b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc3e41ace70 con 0x563859d2e640 2024-06-10T02:55:01.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.715+0000 7fc3f5162b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc3e41aa090 con 0x563859d1f760 2024-06-10T02:55:01.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.715+0000 7fc3f5162b80 10 client.0 _ll_register_callbacks cb 0x563859c07920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T02:55:01.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.715+0000 7fc3e3fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563859d1bf20 0x563859d1a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:01.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.715+0000 7fc3e3fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563859d1bf20 0x563859d1a240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:42690/0 (socket says 172.21.15.52:42690) 2024-06-10T02:55:01.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.715+0000 7fc3e3fff700 1 -- 172.21.15.52:0/3659358019 learned_addr learned my addr 172.21.15.52:0/3659358019 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:55:01.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.716+0000 7fc3e3fff700 1 -- 172.21.15.52:0/3659358019 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563859d1f760 msgr2=0x563859d36340 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-10T02:55:01.718 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.716+0000 7fc3e8e7c700 1 --2- 172.21.15.52:0/3659358019 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563859d2e640 0x563859d36930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:01.718 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.716+0000 7fc3e3fff700 1 --2- 172.21.15.52:0/3659358019 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563859d1f760 0x563859d36340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:01.718 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.716+0000 7fc3e3fff700 1 -- 172.21.15.52:0/3659358019 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563859d2e640 msgr2=0x563859d36930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:55:01.718 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.716+0000 7fc3e3fff700 1 --2- 172.21.15.52:0/3659358019 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563859d2e640 0x563859d36930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:01.718 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.716+0000 7fc3e3fff700 1 -- 172.21.15.52:0/3659358019 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563859d45580 con 0x563859d1bf20 2024-06-10T02:55:01.718 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.716+0000 7fc3e8e7c700 1 --2- 172.21.15.52:0/3659358019 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563859d2e640 0x563859d36930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T02:55:01.718 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.716+0000 7fc3e3fff700 1 --2- 172.21.15.52:0/3659358019 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563859d1bf20 0x563859d1a240 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fc3d4004290 tx=0x7fc3d4004650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=332372eaec0e30af server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:55:01.718 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.716+0000 7fc3c37fe700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-10T02:55:01.718 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.716+0000 7fc3c37fe700 1 -- 172.21.15.52:0/3659358019 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3d4004a30 con 0x563859d1bf20 2024-06-10T02:55:01.718 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.716+0000 7fc3c37fe700 1 -- 172.21.15.52:0/3659358019 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7fc3d4004bd0 con 0x563859d1bf20 2024-06-10T02:55:01.718 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.716+0000 7fc3f5162b80 1 -- 172.21.15.52:0/3659358019 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc3d80a30b0 con 0x563859d1bf20 2024-06-10T02:55:01.720 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.717+0000 7fc3c37fe700 1 -- 172.21.15.52:0/3659358019 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fc3d402f050 con 0x563859d1bf20 2024-06-10T02:55:01.720 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.717+0000 7fc3f5162b80 10 client.15543 fetch_fsmap learned FSMap version 30 2024-06-10T02:55:01.720 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.717+0000 7fc3f5162b80 1 -- 172.21.15.52:0/3659358019 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x563859da8db0 con 0x563859d1bf20 2024-06-10T02:55:01.720 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.717+0000 7fc3c37fe700 1 -- 172.21.15.52:0/3659358019 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc3d80a30b0 con 0x563859d1bf20 2024-06-10T02:55:01.721 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.717+0000 7fc3c37fe700 1 -- 172.21.15.52:0/3659358019 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fc3d4026780 con 0x563859d1bf20 2024-06-10T02:55:01.721 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.717+0000 7fc3f5162b80 10 client.15543 fetch_fsmap finished waiting for FSMap version 30 2024-06-10T02:55:01.721 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.717+0000 7fc3f5162b80 10 client.15543 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T02:55:01.721 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.717+0000 7fc3f5162b80 20 client.15543 populate_metadata read hostname 'smithi052' 2024-06-10T02:55:01.721 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.717+0000 7fc3f5162b80 1 --2- 172.21.15.52:0/3659358019 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x563859da94f0 0x563859dab930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:01.721 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.717+0000 7fc3f5162b80 4 client.15543 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-10T02:55:01.721 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.717+0000 7fc3f5162b80 1 -- 172.21.15.52:0/3659358019 --> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563859c75290 con 0x563859da94f0 2024-06-10T02:55:01.721 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.718+0000 7fc3e37fe700 1 --2- 172.21.15.52:0/3659358019 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x563859da94f0 0x563859dab930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:01.721 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.718+0000 7fc3c37fe700 10 client.15543 ms_handle_connect on v2:172.21.15.188:6838/310444836 2024-06-10T02:55:01.721 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.718+0000 7fc3e37fe700 1 --2- 172.21.15.52:0/3659358019 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x563859da94f0 0x563859dab930 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=afe2554c29154d92 server_cookie=9b9dc366369a5a4c in_seq=0 out_seq=0 2024-06-10T02:55:01.721 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.718+0000 7fc3c37fe700 1 -- 172.21.15.52:0/3659358019 <== mds.0 v2:172.21.15.188:6838/310444836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x563859c75290 con 0x563859da94f0 2024-06-10T02:55:01.721 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.718+0000 7fc3c37fe700 10 client.15543 handle_command_reply: tid=0 2024-06-10T02:55:01.721 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.718+0000 7fc3f5162b80 1 client.15543 shutdown 2024-06-10T02:55:01.721 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.719+0000 7fc3f5162b80 1 -- 172.21.15.52:0/3659358019 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563859d1bf20 msgr2=0x563859d1a240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:55:01.722 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.719+0000 7fc3f5162b80 1 --2- 172.21.15.52:0/3659358019 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563859d1bf20 0x563859d1a240 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fc3d4004290 tx=0x7fc3d4004650 comp rx=0 tx=0).stop 2024-06-10T02:55:01.722 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.719+0000 7fc3f5162b80 1 -- 172.21.15.52:0/3659358019 shutdown_connections 2024-06-10T02:55:01.722 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.719+0000 7fc3f5162b80 1 --2- 172.21.15.52:0/3659358019 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x563859da94f0 0x563859dab930 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:01.722 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.719+0000 7fc3c37fe700 0 client.15543 ms_handle_reset on v2:172.21.15.188:6838/310444836 2024-06-10T02:55:01.722 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.719+0000 7fc3f5162b80 1 --2- 172.21.15.52:0/3659358019 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563859d2e640 0x563859d36930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:01.722 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.719+0000 7fc3f5162b80 1 --2- 172.21.15.52:0/3659358019 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563859d1bf20 0x563859d1a240 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:01.722 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.719+0000 7fc3f5162b80 1 --2- 172.21.15.52:0/3659358019 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563859d1f760 0x563859d36340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:01.722 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.719+0000 7fc3f5162b80 1 -- 172.21.15.52:0/3659358019 >> 172.21.15.52:0/3659358019 conn(0x563859c46cc0 msgr2=0x563859d1ace0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:55:01.722 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.719+0000 7fc3f5162b80 1 -- 172.21.15.52:0/3659358019 shutdown_connections 2024-06-10T02:55:01.722 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-10T02:55:01.723 INFO:teuthology.orchestra.run.smithi052.stdout: "status": "no active scrubs running", 2024-06-10T02:55:01.723 INFO:teuthology.orchestra.run.smithi052.stdout: "scrubs": {} 2024-06-10T02:55:01.723 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-10T02:55:01.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.719+0000 7fc3f5162b80 1 -- 172.21.15.52:0/3659358019 wait complete. 2024-06-10T02:55:01.724 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.719+0000 7fc3f5162b80 20 client.15543 trim_cache size 0 max 16384 2024-06-10T02:55:01.724 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.720+0000 7fc3ea8df700 1 -- 172.21.15.52:0/2095635374 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fc3cc0a2d30 msgr2=0x7fc3cc0a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:55:01.724 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.720+0000 7fc3ea8df700 1 --2- 172.21.15.52:0/2095635374 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fc3cc0a2d30 0x7fc3cc0a51f0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fc3dc007fd0 tx=0x7fc3dc007a60 comp rx=0 tx=0).stop 2024-06-10T02:55:01.724 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.720+0000 7fc3ea8df700 1 -- 172.21.15.52:0/2095635374 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc3e4101d50 msgr2=0x7fc3e419a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:55:01.724 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.720+0000 7fc3ea8df700 1 --2- 172.21.15.52:0/2095635374 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc3e4101d50 0x7fc3e419a000 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fc3d8000c00 tx=0x7fc3d8003890 comp rx=0 tx=0).stop 2024-06-10T02:55:01.724 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.720+0000 7fc3ea8df700 1 -- 172.21.15.52:0/2095635374 shutdown_connections 2024-06-10T02:55:01.724 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.720+0000 7fc3ea8df700 1 --2- 172.21.15.52:0/2095635374 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc3e4107400 0x7fc3e41a4b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:01.724 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.720+0000 7fc3ea8df700 1 --2- 172.21.15.52:0/2095635374 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc3e4101390 0x7fc3e4199ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:01.724 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.720+0000 7fc3ea8df700 1 --2- 172.21.15.52:0/2095635374 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fc3cc0a2d30 0x7fc3cc0a51f0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:01.724 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.720+0000 7fc3ea8df700 1 --2- 172.21.15.52:0/2095635374 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc3e4101d50 0x7fc3e419a000 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:01.724 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.720+0000 7fc3ea8df700 1 -- 172.21.15.52:0/2095635374 >> 172.21.15.52:0/2095635374 conn(0x7fc3e4078340 msgr2=0x7fc3e40ffe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:55:01.725 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.720+0000 7fc3ea8df700 1 -- 172.21.15.52:0/2095635374 shutdown_connections 2024-06-10T02:55:01.725 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:01.720+0000 7fc3ea8df700 1 -- 172.21.15.52:0/2095635374 wait complete. 2024-06-10T02:55:02.478 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-06-10T02:55:02.479 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T02:55:02.736 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:55:03.138 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:55:02 smithi052 ceph-mon[32362]: pgmap v459: 129 pgs: 129 active+clean; 661 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-10T02:55:03.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:55:02 smithi120 ceph-mon[36634]: pgmap v459: 129 pgs: 129 active+clean; 661 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-10T02:55:03.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:55:02 smithi188 ceph-mon[36599]: pgmap v459: 129 pgs: 129 active+clean; 661 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-10T02:55:04.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:04.466+0000 7f7707820700 1 -- 172.21.15.52:0/3288507978 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7700102ea0 msgr2=0x7f770010f380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:55:04.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:04.466+0000 7f7707820700 1 --2- 172.21.15.52:0/3288507978 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7700102ea0 0x7f770010f380 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f76f800b0a0 tx=0x7f76f800b3b0 comp rx=0 tx=0).stop 2024-06-10T02:55:04.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:04.466+0000 7f7707820700 1 -- 172.21.15.52:0/3288507978 shutdown_connections 2024-06-10T02:55:04.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:04.466+0000 7f7707820700 1 --2- 172.21.15.52:0/3288507978 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7700102ea0 0x7f770010f380 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:04.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:04.466+0000 7f7707820700 1 --2- 172.21.15.52:0/3288507978 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f77001085f0 0x7f77001089d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:04.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:04.466+0000 7f7707820700 1 --2- 172.21.15.52:0/3288507978 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f77001024e0 0x7f7700102960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:04.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:04.466+0000 7f7707820700 1 -- 172.21.15.52:0/3288507978 >> 172.21.15.52:0/3288507978 conn(0x7f7700079640 msgr2=0x7f7700079a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:55:04.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:04.466+0000 7f7707820700 1 -- 172.21.15.52:0/3288507978 shutdown_connections 2024-06-10T02:55:04.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:04.466+0000 7f7707820700 1 -- 172.21.15.52:0/3288507978 wait complete. 2024-06-10T02:55:04.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:04.467+0000 7f7707820700 1 Processor -- start 2024-06-10T02:55:04.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:04.467+0000 7f7707820700 1 -- start start 2024-06-10T02:55:04.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:04.467+0000 7f7707820700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f77001024e0 0x7f7700074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:04.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:04.467+0000 7f7707820700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7700102ea0 0x7f7700074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:04.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:04.467+0000 7f7707820700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f77001085f0 0x7f7700075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:04.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:04.467+0000 7f7707820700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f7700112790 con 0x7f77001024e0 2024-06-10T02:55:04.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:04.467+0000 7f7707820700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f7700112610 con 0x7f7700102ea0 2024-06-10T02:55:04.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:04.467+0000 7f7707820700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f7700112910 con 0x7f77001085f0 2024-06-10T02:55:04.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:04.467+0000 7f77055bc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f77001024e0 0x7f7700074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:04.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:04.467+0000 7f77055bc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f77001024e0 0x7f7700074790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:42708/0 (socket says 172.21.15.52:42708) 2024-06-10T02:55:04.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:04.467+0000 7f77055bc700 1 -- 172.21.15.52:0/2079601661 learned_addr learned my addr 172.21.15.52:0/2079601661 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:55:04.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:04.468+0000 7f7704dbb700 1 --2- 172.21.15.52:0/2079601661 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7700102ea0 0x7f7700074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:04.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:04.468+0000 7f7705dbd700 1 --2- 172.21.15.52:0/2079601661 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f77001085f0 0x7f7700075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:04.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:04.468+0000 7f77055bc700 1 -- 172.21.15.52:0/2079601661 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f77001085f0 msgr2=0x7f7700075210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:55:04.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:04.468+0000 7f77055bc700 1 --2- 172.21.15.52:0/2079601661 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f77001085f0 0x7f7700075210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:04.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:04.468+0000 7f77055bc700 1 -- 172.21.15.52:0/2079601661 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7700102ea0 msgr2=0x7f7700074cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:55:04.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:04.468+0000 7f77055bc700 1 --2- 172.21.15.52:0/2079601661 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7700102ea0 0x7f7700074cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:04.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:04.468+0000 7f77055bc700 1 -- 172.21.15.52:0/2079601661 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77001a7150 con 0x7f77001024e0 2024-06-10T02:55:04.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:04.468+0000 7f7704dbb700 1 --2- 172.21.15.52:0/2079601661 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7700102ea0 0x7f7700074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T02:55:04.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:04.468+0000 7f77055bc700 1 --2- 172.21.15.52:0/2079601661 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f77001024e0 0x7f7700074790 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f76fc00c920 tx=0x7f76fc00cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:55:04.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:04.468+0000 7f76f67fc700 1 -- 172.21.15.52:0/2079601661 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76fc004a30 con 0x7f77001024e0 2024-06-10T02:55:04.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:04.468+0000 7f7707820700 1 -- 172.21.15.52:0/2079601661 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77001a7440 con 0x7f77001024e0 2024-06-10T02:55:04.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:04.468+0000 7f76f67fc700 1 -- 172.21.15.52:0/2079601661 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76fc005500 con 0x7f77001024e0 2024-06-10T02:55:04.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:04.468+0000 7f76f67fc700 1 -- 172.21.15.52:0/2079601661 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76fc007d50 con 0x7f77001024e0 2024-06-10T02:55:04.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:04.469+0000 7f7707820700 1 -- 172.21.15.52:0/2079601661 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77001a7980 con 0x7f77001024e0 2024-06-10T02:55:04.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:04.470+0000 7f7707820700 1 -- 172.21.15.52:0/2079601661 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7700069180 con 0x7f77001024e0 2024-06-10T02:55:04.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:04.471+0000 7f76f67fc700 1 -- 172.21.15.52:0/2079601661 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7f76fc004d60 con 0x7f77001024e0 2024-06-10T02:55:04.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:04.472+0000 7f76f67fc700 1 --2- 172.21.15.52:0/2079601661 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f76ec0a2cb0 0x7f76ec0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:04.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:04.472+0000 7f7704dbb700 1 --2- 172.21.15.52:0/2079601661 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f76ec0a2cb0 0x7f76ec0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:04.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:04.472+0000 7f76f67fc700 1 -- 172.21.15.52:0/2079601661 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f76fc0cf380 con 0x7f77001024e0 2024-06-10T02:55:04.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:04.472+0000 7f7704dbb700 1 --2- 172.21.15.52:0/2079601661 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f76ec0a2cb0 0x7f76ec0a5170 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f7700108cb0 tx=0x7f76f000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:55:04.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:04.476+0000 7f76f67fc700 1 -- 172.21.15.52:0/2079601661 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f76fc014030 con 0x7f77001024e0 2024-06-10T02:55:04.744 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:04.745+0000 7f7707820700 1 -- 172.21.15.52:0/2079601661 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f770002dca0 con 0x7f77001024e0 2024-06-10T02:55:04.745 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:04.746+0000 7f76f67fc700 1 -- 172.21.15.52:0/2079601661 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11938 (secure 0 0 0) 0x7f76fc09bdd0 con 0x7f77001024e0 2024-06-10T02:55:04.745 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-10T02:55:04.745 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/889673247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":889673247},{"type":"v1","addr":"172.21.15.52:6835","nonce":889673247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/3615663511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":3615663511},{"type":"v1","addr":"172.21.15.52:6837","nonce":3615663511}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T02:50:03.889766+0000","modified":"2024-06-10T02:54:07.086405+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24484,"mds_1":24469,"mds_2":24463,"mds_3":24449,"mds_4":24428},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14571":{"gid":14571,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6839/3059599042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":3059599042},{"type":"v1","addr":"172.21.15.52:6839","nonce":3059599042}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14583":{"gid":14583,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6841/3039275521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":3039275521},{"type":"v1","addr":"172.21.15.52:6841","nonce":3039275521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24383":{"gid":24383,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6833/2609002896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":2609002896},{"type":"v1","addr":"172.21.15.120:6833","nonce":2609002896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6835/1996370288","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":1996370288},{"type":"v1","addr":"172.21.15.120:6835","nonce":1996370288}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6833/529564979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":529564979},{"type":"v1","addr":"172.21.15.188:6833","nonce":529564979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":25,"addr":"172.21.15.120:6837/4132440952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":4132440952},{"type":"v1","addr":"172.21.15.120:6837","nonce":4132440952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"b","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.120:6839/1672436487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1672436487},{"type":"v1","addr":"172.21.15.120:6839","nonce":1672436487}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":2,"incarnation":22,"state":"up:active","state_seq":26,"addr":"172.21.15.188:6835/3904544259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":3904544259},{"type":"v1","addr":"172.21.15.188:6835","nonce":3904544259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.188:6837/819317726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":819317726},{"type":"v1","addr":"172.21.15.188:6837","nonce":819317726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.188:6839/310444836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":310444836},{"type":"v1","addr":"172.21.15.188:6839","nonce":310444836}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24484,"qdb_cluster":[24469,24484,24463,24428,24449]},"id":1}]} 2024-06-10T02:55:04.748 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:04.748+0000 7f7707820700 1 -- 172.21.15.52:0/2079601661 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f76ec0a2cb0 msgr2=0x7f76ec0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:55:04.748 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:04.748+0000 7f7707820700 1 --2- 172.21.15.52:0/2079601661 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f76ec0a2cb0 0x7f76ec0a5170 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f7700108cb0 tx=0x7f76f000b410 comp rx=0 tx=0).stop 2024-06-10T02:55:04.748 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:04.748+0000 7f7707820700 1 -- 172.21.15.52:0/2079601661 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f77001024e0 msgr2=0x7f7700074790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:55:04.748 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:04.748+0000 7f7707820700 1 --2- 172.21.15.52:0/2079601661 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f77001024e0 0x7f7700074790 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f76fc00c920 tx=0x7f76fc00cc30 comp rx=0 tx=0).stop 2024-06-10T02:55:04.748 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:04.748+0000 7f7707820700 1 -- 172.21.15.52:0/2079601661 shutdown_connections 2024-06-10T02:55:04.748 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:04.748+0000 7f7707820700 1 --2- 172.21.15.52:0/2079601661 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f77001085f0 0x7f7700075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:04.748 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:04.748+0000 7f7707820700 1 --2- 172.21.15.52:0/2079601661 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f76ec0a2cb0 0x7f76ec0a5170 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:04.748 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:04.748+0000 7f7707820700 1 --2- 172.21.15.52:0/2079601661 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f77001024e0 0x7f7700074790 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:04.748 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:04.748+0000 7f7707820700 1 --2- 172.21.15.52:0/2079601661 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7700102ea0 0x7f7700074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:04.749 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:04.748+0000 7f7707820700 1 -- 172.21.15.52:0/2079601661 >> 172.21.15.52:0/2079601661 conn(0x7f7700079640 msgr2=0x7f7700100ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:55:04.749 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:04.748+0000 7f7707820700 1 -- 172.21.15.52:0/2079601661 shutdown_connections 2024-06-10T02:55:04.749 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:04.749+0000 7f7707820700 1 -- 172.21.15.52:0/2079601661 wait complete. 2024-06-10T02:55:04.749 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 30 2024-06-10T02:55:05.077 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:55:04 smithi052 ceph-mon[32362]: pgmap v460: 129 pgs: 129 active+clean; 661 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-10T02:55:05.077 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:55:04 smithi052 ceph-mon[32362]: from='client.? 172.21.15.52:0/2079601661' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T02:55:05.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:55:04 smithi120 ceph-mon[36634]: pgmap v460: 129 pgs: 129 active+clean; 661 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-10T02:55:05.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:55:04 smithi120 ceph-mon[36634]: from='client.? 172.21.15.52:0/2079601661' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T02:55:05.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:55:04 smithi188 ceph-mon[36599]: pgmap v460: 129 pgs: 129 active+clean; 661 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-10T02:55:05.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:55:04 smithi188 ceph-mon[36599]: from='client.? 172.21.15.52:0/2079601661' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T02:55:05.527 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-06-10T02:55:05.792 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:55:07.127 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:55:06 smithi052 ceph-mon[32362]: pgmap v461: 129 pgs: 129 active+clean; 661 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-10T02:55:07.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:55:06 smithi120 ceph-mon[36634]: pgmap v461: 129 pgs: 129 active+clean; 661 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-10T02:55:07.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:55:06 smithi188 ceph-mon[36599]: pgmap v461: 129 pgs: 129 active+clean; 661 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-10T02:55:07.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.506+0000 7f6d4621f700 1 -- 172.21.15.52:0/3699367821 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6d40108500 msgr2=0x7f6d401088e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:55:07.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.506+0000 7f6d4621f700 1 --2- 172.21.15.52:0/3699367821 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6d40108500 0x7f6d401088e0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f6d30009990 tx=0x7f6d30009ca0 comp rx=0 tx=0).stop 2024-06-10T02:55:07.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.506+0000 7f6d4621f700 1 -- 172.21.15.52:0/3699367821 shutdown_connections 2024-06-10T02:55:07.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.506+0000 7f6d4621f700 1 --2- 172.21.15.52:0/3699367821 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6d4006ab30 0x7f6d4010f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:07.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.506+0000 7f6d4621f700 1 --2- 172.21.15.52:0/3699367821 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6d40108500 0x7f6d401088e0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:07.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.506+0000 7f6d4621f700 1 --2- 172.21.15.52:0/3699367821 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6d4006a170 0x7f6d4006a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:07.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.506+0000 7f6d4621f700 1 -- 172.21.15.52:0/3699367821 >> 172.21.15.52:0/3699367821 conn(0x7f6d40078340 msgr2=0x7f6d40078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:55:07.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.506+0000 7f6d4621f700 1 -- 172.21.15.52:0/3699367821 shutdown_connections 2024-06-10T02:55:07.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.506+0000 7f6d4621f700 1 -- 172.21.15.52:0/3699367821 wait complete. 2024-06-10T02:55:07.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.506+0000 7f6d4621f700 1 Processor -- start 2024-06-10T02:55:07.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.507+0000 7f6d4621f700 1 -- start start 2024-06-10T02:55:07.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.507+0000 7f6d4621f700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6d4006a170 0x7f6d4019e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:07.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.507+0000 7f6d4621f700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6d4006ab30 0x7f6d4019eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:07.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.507+0000 7f6d4621f700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6d40108500 0x7f6d40198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:07.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.507+0000 7f6d4621f700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f6d40112760 con 0x7f6d4006a170 2024-06-10T02:55:07.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.507+0000 7f6d4621f700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f6d401125e0 con 0x7f6d40108500 2024-06-10T02:55:07.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.507+0000 7f6d4621f700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f6d401128e0 con 0x7f6d4006ab30 2024-06-10T02:55:07.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.507+0000 7f6d3f7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6d4006a170 0x7f6d4019e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:07.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.507+0000 7f6d3f7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6d4006a170 0x7f6d4019e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:42722/0 (socket says 172.21.15.52:42722) 2024-06-10T02:55:07.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.507+0000 7f6d3f7fe700 1 -- 172.21.15.52:0/3706918625 learned_addr learned my addr 172.21.15.52:0/3706918625 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:55:07.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.507+0000 7f6d3effd700 1 --2- 172.21.15.52:0/3706918625 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6d4006ab30 0x7f6d4019eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:07.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.508+0000 7f6d3ffff700 1 --2- 172.21.15.52:0/3706918625 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6d40108500 0x7f6d40198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:07.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.508+0000 7f6d3f7fe700 1 -- 172.21.15.52:0/3706918625 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6d4006ab30 msgr2=0x7f6d4019eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:55:07.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.508+0000 7f6d3f7fe700 1 --2- 172.21.15.52:0/3706918625 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6d4006ab30 0x7f6d4019eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:07.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.508+0000 7f6d3f7fe700 1 -- 172.21.15.52:0/3706918625 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6d40108500 msgr2=0x7f6d40198920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:55:07.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.508+0000 7f6d3f7fe700 1 --2- 172.21.15.52:0/3706918625 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6d40108500 0x7f6d40198920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:07.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.508+0000 7f6d3f7fe700 1 -- 172.21.15.52:0/3706918625 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d40199080 con 0x7f6d4006a170 2024-06-10T02:55:07.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.508+0000 7f6d3f7fe700 1 --2- 172.21.15.52:0/3706918625 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6d4006a170 0x7f6d4019e760 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f6d30000c00 tx=0x7f6d30003680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:55:07.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.508+0000 7f6d3cff9700 1 -- 172.21.15.52:0/3706918625 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d30027070 con 0x7f6d4006a170 2024-06-10T02:55:07.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.508+0000 7f6d3cff9700 1 -- 172.21.15.52:0/3706918625 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d30004110 con 0x7f6d4006a170 2024-06-10T02:55:07.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.508+0000 7f6d3cff9700 1 -- 172.21.15.52:0/3706918625 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d3000f460 con 0x7f6d4006a170 2024-06-10T02:55:07.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.508+0000 7f6d4621f700 1 -- 172.21.15.52:0/3706918625 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d40199310 con 0x7f6d4006a170 2024-06-10T02:55:07.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.508+0000 7f6d4621f700 1 -- 172.21.15.52:0/3706918625 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d401a5630 con 0x7f6d4006a170 2024-06-10T02:55:07.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.511+0000 7f6d3cff9700 1 -- 172.21.15.52:0/3706918625 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7f6d3002a070 con 0x7f6d4006a170 2024-06-10T02:55:07.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.512+0000 7f6d3cff9700 1 --2- 172.21.15.52:0/3706918625 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f6d280a2d30 0x7f6d280a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:07.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.512+0000 7f6d3effd700 1 --2- 172.21.15.52:0/3706918625 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f6d280a2d30 0x7f6d280a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:07.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.512+0000 7f6d3cff9700 1 -- 172.21.15.52:0/3706918625 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f6d300d0480 con 0x7f6d4006a170 2024-06-10T02:55:07.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.513+0000 7f6d3effd700 1 --2- 172.21.15.52:0/3706918625 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f6d280a2d30 0x7f6d280a51f0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f6d40078840 tx=0x7f6d3400a380 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:55:07.513 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.513+0000 7f6d50aa2b80 1 Processor -- start 2024-06-10T02:55:07.513 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.514+0000 7f6d50aa2b80 1 -- start start 2024-06-10T02:55:07.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.514+0000 7f6d50aa2b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564177431a70 0x564177431e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:07.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.514+0000 7f6d50aa2b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564177432390 0x56417742bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:07.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.514+0000 7f6d50aa2b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56417742c330 0x56417742c750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:07.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.514+0000 7f6d50aa2b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x564177379810 con 0x56417742c330 2024-06-10T02:55:07.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.514+0000 7f6d50aa2b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x564177321720 con 0x564177432390 2024-06-10T02:55:07.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.514+0000 7f6d50aa2b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5641773757d0 con 0x564177431a70 2024-06-10T02:55:07.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.514+0000 7f6d3ffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56417742c330 0x56417742c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:07.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.514+0000 7f6d3f7fe700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564177431a70 0x564177431e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:07.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.514+0000 7f6d3ffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56417742c330 0x56417742c750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:42728/0 (socket says 172.21.15.52:42728) 2024-06-10T02:55:07.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.514+0000 7f6d3ffff700 1 -- 172.21.15.52:0/1321813976 learned_addr learned my addr 172.21.15.52:0/1321813976 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:55:07.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.514+0000 7f6d3effd700 1 --2- 172.21.15.52:0/1321813976 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564177432390 0x56417742bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:07.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.514+0000 7f6d3effd700 1 -- 172.21.15.52:0/1321813976 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564177431a70 msgr2=0x564177431e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:55:07.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.514+0000 7f6d3effd700 1 --2- 172.21.15.52:0/1321813976 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564177431a70 0x564177431e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:07.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.514+0000 7f6d3effd700 1 -- 172.21.15.52:0/1321813976 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56417742c330 msgr2=0x56417742c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:55:07.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.514+0000 7f6d3effd700 1 --2- 172.21.15.52:0/1321813976 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56417742c330 0x56417742c750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:07.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.515+0000 7f6d3effd700 1 -- 172.21.15.52:0/1321813976 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564177317f50 con 0x564177432390 2024-06-10T02:55:07.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.515+0000 7f6d3ffff700 1 --2- 172.21.15.52:0/1321813976 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56417742c330 0x56417742c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T02:55:07.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.515+0000 7f6d3effd700 1 --2- 172.21.15.52:0/1321813976 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564177432390 0x56417742bdf0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f6d3400fde0 tx=0x7f6d34012000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3fad7ad7ae02cf0e server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:55:07.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.515+0000 7f6d25ffb700 1 -- 172.21.15.52:0/1321813976 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d34016070 con 0x564177432390 2024-06-10T02:55:07.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.515+0000 7f6d25ffb700 1 -- 172.21.15.52:0/1321813976 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d34014070 con 0x564177432390 2024-06-10T02:55:07.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.515+0000 7f6d25ffb700 1 -- 172.21.15.52:0/1321813976 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d34010da0 con 0x564177432390 2024-06-10T02:55:07.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.516+0000 7f6d50aa2b80 1 -- 172.21.15.52:0/1321813976 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564177432390 msgr2=0x56417742bdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:55:07.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.516+0000 7f6d50aa2b80 1 --2- 172.21.15.52:0/1321813976 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564177432390 0x56417742bdf0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f6d3400fde0 tx=0x7f6d34012000 comp rx=0 tx=0).stop 2024-06-10T02:55:07.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.516+0000 7f6d50aa2b80 1 -- 172.21.15.52:0/1321813976 shutdown_connections 2024-06-10T02:55:07.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.516+0000 7f6d50aa2b80 1 --2- 172.21.15.52:0/1321813976 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56417742c330 0x56417742c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:07.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.516+0000 7f6d50aa2b80 1 --2- 172.21.15.52:0/1321813976 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564177431a70 0x564177431e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:07.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.516+0000 7f6d50aa2b80 1 --2- 172.21.15.52:0/1321813976 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564177432390 0x56417742bdf0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:07.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.516+0000 7f6d50aa2b80 1 -- 172.21.15.52:0/1321813976 >> 172.21.15.52:0/1321813976 conn(0x564177352cc0 msgr2=0x564177422850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:55:07.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.516+0000 7f6d50aa2b80 1 -- 172.21.15.52:0/1321813976 shutdown_connections 2024-06-10T02:55:07.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.516+0000 7f6d50aa2b80 1 -- 172.21.15.52:0/1321813976 wait complete. 2024-06-10T02:55:07.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.517+0000 7f6d50aa2b80 1 -- start start 2024-06-10T02:55:07.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.517+0000 7f6d50aa2b80 1 Processor -- start 2024-06-10T02:55:07.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.517+0000 7f6d50aa2b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56417742c330 0x564177438190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:07.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.517+0000 7f6d50aa2b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564177431a70 0x564177435500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:07.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.517+0000 7f6d50aa2b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564177432390 0x564177435a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:07.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.517+0000 7f6d50aa2b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x564177438dd0 con 0x56417742c330 2024-06-10T02:55:07.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.517+0000 7f6d50aa2b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x564177319bf0 con 0x564177432390 2024-06-10T02:55:07.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.517+0000 7f6d50aa2b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x564177438f50 con 0x564177431a70 2024-06-10T02:55:07.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.517+0000 7f6d50aa2b80 10 client.0 _ll_register_callbacks cb 0x564177313920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T02:55:07.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.518+0000 7f6d3f7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56417742c330 0x564177438190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:07.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.518+0000 7f6d3f7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56417742c330 0x564177438190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:42740/0 (socket says 172.21.15.52:42740) 2024-06-10T02:55:07.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.518+0000 7f6d3f7fe700 1 -- 172.21.15.52:0/2041316248 learned_addr learned my addr 172.21.15.52:0/2041316248 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:55:07.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.518+0000 7f6d3ffff700 1 --2- 172.21.15.52:0/2041316248 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564177432390 0x564177435a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:07.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.518+0000 7f6d3f7fe700 1 -- 172.21.15.52:0/2041316248 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564177431a70 msgr2=0x564177435500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:55:07.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.518+0000 7f6d3f7fe700 1 --2- 172.21.15.52:0/2041316248 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564177431a70 0x564177435500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:07.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.518+0000 7f6d3f7fe700 1 -- 172.21.15.52:0/2041316248 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564177432390 msgr2=0x564177435a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:55:07.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.518+0000 7f6d3f7fe700 1 --2- 172.21.15.52:0/2041316248 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564177432390 0x564177435a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:07.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.518+0000 7f6d3f7fe700 1 -- 172.21.15.52:0/2041316248 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5641772c88d0 con 0x56417742c330 2024-06-10T02:55:07.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.518+0000 7f6d3f7fe700 1 --2- 172.21.15.52:0/2041316248 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56417742c330 0x564177438190 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f6d300098e0 tx=0x7f6d30022970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60df0a77a2d5d8a7 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:55:07.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.518+0000 7f6d1effd700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-10T02:55:07.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.518+0000 7f6d1effd700 1 -- 172.21.15.52:0/2041316248 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d3009cf60 con 0x56417742c330 2024-06-10T02:55:07.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.518+0000 7f6d1effd700 1 -- 172.21.15.52:0/2041316248 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f6d3009ab60 con 0x56417742c330 2024-06-10T02:55:07.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.518+0000 7f6d50aa2b80 1 -- 172.21.15.52:0/2041316248 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d34014070 con 0x56417742c330 2024-06-10T02:55:07.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.519+0000 7f6d1effd700 1 -- 172.21.15.52:0/2041316248 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f6d3009c860 con 0x56417742c330 2024-06-10T02:55:07.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.519+0000 7f6d50aa2b80 10 client.15561 fetch_fsmap learned FSMap version 30 2024-06-10T02:55:07.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.519+0000 7f6d50aa2b80 1 -- 172.21.15.52:0/2041316248 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5641774437e0 con 0x56417742c330 2024-06-10T02:55:07.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.519+0000 7f6d1effd700 1 -- 172.21.15.52:0/2041316248 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d300a0c10 con 0x56417742c330 2024-06-10T02:55:07.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.519+0000 7f6d1effd700 1 -- 172.21.15.52:0/2041316248 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f6d3009cac0 con 0x56417742c330 2024-06-10T02:55:07.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.519+0000 7f6d50aa2b80 10 client.15561 fetch_fsmap finished waiting for FSMap version 30 2024-06-10T02:55:07.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.519+0000 7f6d50aa2b80 10 client.15561 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-06-10T02:55:07.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.519+0000 7f6d50aa2b80 20 client.15561 populate_metadata read hostname 'smithi052' 2024-06-10T02:55:07.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.519+0000 7f6d50aa2b80 1 --2- 172.21.15.52:0/2041316248 >> [v2:172.21.15.120:6836/4132440952,v1:172.21.15.120:6837/4132440952] conn(0x564177445da0 0x5641774481e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:07.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.519+0000 7f6d50aa2b80 4 client.15561 mds_command: new command op to 24428 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T02:55:07.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.519+0000 7f6d50aa2b80 1 -- 172.21.15.52:0/2041316248 --> [v2:172.21.15.120:6836/4132440952,v1:172.21.15.120:6837/4132440952] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f6d34014070 con 0x564177445da0 2024-06-10T02:55:07.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.520+0000 7f6d3effd700 1 --2- 172.21.15.52:0/2041316248 >> [v2:172.21.15.120:6836/4132440952,v1:172.21.15.120:6837/4132440952] conn(0x564177445da0 0x5641774481e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:07.523 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.520+0000 7f6d3effd700 1 --2- 172.21.15.52:0/2041316248 >> [v2:172.21.15.120:6836/4132440952,v1:172.21.15.120:6837/4132440952] conn(0x564177445da0 0x5641774481e0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=29d36a15c66658ea server_cookie=fac35ceb750931ee in_seq=0 out_seq=0 2024-06-10T02:55:07.523 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.520+0000 7f6d1effd700 10 client.15561 ms_handle_connect on v2:172.21.15.120:6836/4132440952 2024-06-10T02:55:07.523 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.521+0000 7f6d1effd700 1 -- 172.21.15.52:0/2041316248 <== mds.4 v2:172.21.15.120:6836/4132440952 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x7f6d34014070 con 0x564177445da0 2024-06-10T02:55:07.523 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.521+0000 7f6d1effd700 10 client.15561 handle_command_reply: tid=0 2024-06-10T02:55:07.523 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.521+0000 7f6d50aa2b80 1 client.15561 shutdown 2024-06-10T02:55:07.523 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.521+0000 7f6d50aa2b80 1 -- 172.21.15.52:0/2041316248 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56417742c330 msgr2=0x564177438190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:55:07.523 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.521+0000 7f6d50aa2b80 1 --2- 172.21.15.52:0/2041316248 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56417742c330 0x564177438190 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f6d300098e0 tx=0x7f6d30022970 comp rx=0 tx=0).stop 2024-06-10T02:55:07.523 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.521+0000 7f6d50aa2b80 1 -- 172.21.15.52:0/2041316248 shutdown_connections 2024-06-10T02:55:07.523 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.521+0000 7f6d50aa2b80 1 --2- 172.21.15.52:0/2041316248 >> [v2:172.21.15.120:6836/4132440952,v1:172.21.15.120:6837/4132440952] conn(0x564177445da0 0x5641774481e0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:07.523 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.521+0000 7f6d50aa2b80 1 --2- 172.21.15.52:0/2041316248 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564177432390 0x564177435a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:07.523 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.521+0000 7f6d1effd700 0 client.15561 ms_handle_reset on v2:172.21.15.120:6836/4132440952 2024-06-10T02:55:07.523 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.521+0000 7f6d50aa2b80 1 --2- 172.21.15.52:0/2041316248 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56417742c330 0x564177438190 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:07.523 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.521+0000 7f6d50aa2b80 1 --2- 172.21.15.52:0/2041316248 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564177431a70 0x564177435500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:07.524 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.521+0000 7f6d50aa2b80 1 -- 172.21.15.52:0/2041316248 >> 172.21.15.52:0/2041316248 conn(0x564177352cc0 msgr2=0x5641774269d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:55:07.524 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.521+0000 7f6d50aa2b80 1 -- 172.21.15.52:0/2041316248 shutdown_connections 2024-06-10T02:55:07.524 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.521+0000 7f6d50aa2b80 1 -- 172.21.15.52:0/2041316248 wait complete. 2024-06-10T02:55:07.524 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.521+0000 7f6d50aa2b80 20 client.15561 trim_cache size 0 max 16384 2024-06-10T02:55:07.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.538+0000 7f6d50aa2b80 1 Processor -- start 2024-06-10T02:55:07.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.538+0000 7f6d50aa2b80 1 -- start start 2024-06-10T02:55:07.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.538+0000 7f6d50aa2b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564177431a70 0x564177442070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:07.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.538+0000 7f6d50aa2b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564177432390 0x5641774269d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:07.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.538+0000 7f6d50aa2b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564177445da0 0x564177426f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:07.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.538+0000 7f6d50aa2b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f6d401abf80 con 0x564177431a70 2024-06-10T02:55:07.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.538+0000 7f6d50aa2b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f6d401adac0 con 0x564177445da0 2024-06-10T02:55:07.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.538+0000 7f6d50aa2b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f6d401ac650 con 0x564177432390 2024-06-10T02:55:07.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.538+0000 7f6d3f7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564177431a70 0x564177442070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:07.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.538+0000 7f6d3f7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564177431a70 0x564177442070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:42742/0 (socket says 172.21.15.52:42742) 2024-06-10T02:55:07.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.538+0000 7f6d3f7fe700 1 -- 172.21.15.52:0/2158608450 learned_addr learned my addr 172.21.15.52:0/2158608450 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:55:07.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.539+0000 7f6d3effd700 1 --2- 172.21.15.52:0/2158608450 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564177432390 0x5641774269d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:07.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.539+0000 7f6d3ffff700 1 --2- 172.21.15.52:0/2158608450 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564177445da0 0x564177426f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:07.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.539+0000 7f6d3f7fe700 1 -- 172.21.15.52:0/2158608450 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564177432390 msgr2=0x5641774269d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:55:07.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.539+0000 7f6d3f7fe700 1 --2- 172.21.15.52:0/2158608450 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564177432390 0x5641774269d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:07.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.539+0000 7f6d3f7fe700 1 -- 172.21.15.52:0/2158608450 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564177445da0 msgr2=0x564177426f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:55:07.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.539+0000 7f6d3f7fe700 1 --2- 172.21.15.52:0/2158608450 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564177445da0 0x564177426f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:07.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.539+0000 7f6d3f7fe700 1 -- 172.21.15.52:0/2158608450 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564177433a90 con 0x564177431a70 2024-06-10T02:55:07.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.539+0000 7f6d3f7fe700 1 --2- 172.21.15.52:0/2158608450 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564177431a70 0x564177442070 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f6d300229e0 tx=0x7f6d30022a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=118fe0aeb390919b server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:55:07.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.539+0000 7f6d257fa700 1 -- 172.21.15.52:0/2158608450 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d3009cf60 con 0x564177431a70 2024-06-10T02:55:07.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.539+0000 7f6d257fa700 1 -- 172.21.15.52:0/2158608450 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d300a0c10 con 0x564177431a70 2024-06-10T02:55:07.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.539+0000 7f6d257fa700 1 -- 172.21.15.52:0/2158608450 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d3009ab60 con 0x564177431a70 2024-06-10T02:55:07.543 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.540+0000 7f6d50aa2b80 1 -- 172.21.15.52:0/2158608450 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564177431a70 msgr2=0x564177442070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:55:07.543 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.540+0000 7f6d50aa2b80 1 --2- 172.21.15.52:0/2158608450 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564177431a70 0x564177442070 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f6d300229e0 tx=0x7f6d30022a30 comp rx=0 tx=0).stop 2024-06-10T02:55:07.543 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.540+0000 7f6d50aa2b80 1 -- 172.21.15.52:0/2158608450 shutdown_connections 2024-06-10T02:55:07.543 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.540+0000 7f6d50aa2b80 1 --2- 172.21.15.52:0/2158608450 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564177445da0 0x564177426f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:07.543 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.540+0000 7f6d50aa2b80 1 --2- 172.21.15.52:0/2158608450 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564177431a70 0x564177442070 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:07.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.540+0000 7f6d50aa2b80 1 --2- 172.21.15.52:0/2158608450 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564177432390 0x5641774269d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:07.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.540+0000 7f6d50aa2b80 1 -- 172.21.15.52:0/2158608450 >> 172.21.15.52:0/2158608450 conn(0x564177352cc0 msgr2=0x564177433e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:55:07.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.540+0000 7f6d50aa2b80 1 -- 172.21.15.52:0/2158608450 shutdown_connections 2024-06-10T02:55:07.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.540+0000 7f6d50aa2b80 1 -- 172.21.15.52:0/2158608450 wait complete. 2024-06-10T02:55:07.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.541+0000 7f6d50aa2b80 1 -- start start 2024-06-10T02:55:07.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.541+0000 7f6d50aa2b80 1 Processor -- start 2024-06-10T02:55:07.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.541+0000 7f6d50aa2b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564177431a70 0x56417743e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:07.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.541+0000 7f6d50aa2b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564177432390 0x56417744d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:07.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.542+0000 7f6d3effd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564177432390 0x56417744d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:07.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.542+0000 7f6d3f7fe700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564177431a70 0x56417743e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:07.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.542+0000 7f6d3f7fe700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564177431a70 0x56417743e970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.52:37554/0 (socket says 172.21.15.52:37554) 2024-06-10T02:55:07.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.542+0000 7f6d3effd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564177432390 0x56417744d3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:42746/0 (socket says 172.21.15.52:42746) 2024-06-10T02:55:07.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.542+0000 7f6d3f7fe700 1 -- 172.21.15.52:0/93442877 learned_addr learned my addr 172.21.15.52:0/93442877 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:55:07.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.542+0000 7f6d50aa2b80 1 --2- 172.21.15.52:0/93442877 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564177445da0 0x56417744d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:07.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.542+0000 7f6d50aa2b80 1 -- 172.21.15.52:0/93442877 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f6d401ad3f0 con 0x564177432390 2024-06-10T02:55:07.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.542+0000 7f6d50aa2b80 1 -- 172.21.15.52:0/93442877 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f6d401acd20 con 0x564177445da0 2024-06-10T02:55:07.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.542+0000 7f6d50aa2b80 1 -- 172.21.15.52:0/93442877 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f6d401a9f20 con 0x564177431a70 2024-06-10T02:55:07.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.542+0000 7f6d50aa2b80 10 client.0 _ll_register_callbacks cb 0x564177313920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T02:55:07.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.542+0000 7f6d3ffff700 1 --2- 172.21.15.52:0/93442877 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564177445da0 0x56417744d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:07.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.542+0000 7f6d3f7fe700 1 -- 172.21.15.52:0/93442877 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564177445da0 msgr2=0x56417744d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:55:07.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.542+0000 7f6d3f7fe700 1 --2- 172.21.15.52:0/93442877 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564177445da0 0x56417744d8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:07.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.542+0000 7f6d3f7fe700 1 -- 172.21.15.52:0/93442877 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564177432390 msgr2=0x56417744d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:55:07.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.542+0000 7f6d3f7fe700 1 --2- 172.21.15.52:0/93442877 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564177432390 0x56417744d3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:07.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.542+0000 7f6d3f7fe700 1 -- 172.21.15.52:0/93442877 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56417744e120 con 0x564177431a70 2024-06-10T02:55:07.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.542+0000 7f6d3effd700 1 --2- 172.21.15.52:0/93442877 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564177432390 0x56417744d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T02:55:07.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.543+0000 7f6d3f7fe700 1 --2- 172.21.15.52:0/93442877 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564177431a70 0x56417743e970 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f6d30003900 tx=0x7f6d30022970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=29aef8a015ce6c4e server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:55:07.546 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.543+0000 7f6d1effd700 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-06-10T02:55:07.546 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.543+0000 7f6d1effd700 1 -- 172.21.15.52:0/93442877 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d300a5440 con 0x564177431a70 2024-06-10T02:55:07.546 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.543+0000 7f6d1effd700 1 -- 172.21.15.52:0/93442877 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f6d3009cac0 con 0x564177431a70 2024-06-10T02:55:07.546 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.543+0000 7f6d50aa2b80 1 -- 172.21.15.52:0/93442877 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d300a0c10 con 0x564177431a70 2024-06-10T02:55:07.546 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.543+0000 7f6d1effd700 1 -- 172.21.15.52:0/93442877 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f6d3009cf60 con 0x564177431a70 2024-06-10T02:55:07.546 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.543+0000 7f6d50aa2b80 10 client.25180 fetch_fsmap learned FSMap version 30 2024-06-10T02:55:07.546 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.543+0000 7f6d50aa2b80 1 -- 172.21.15.52:0/93442877 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56417744e120 con 0x564177431a70 2024-06-10T02:55:07.546 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.543+0000 7f6d1effd700 1 -- 172.21.15.52:0/93442877 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d3009bcb0 con 0x564177431a70 2024-06-10T02:55:07.546 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.544+0000 7f6d1effd700 1 -- 172.21.15.52:0/93442877 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f6d30098640 con 0x564177431a70 2024-06-10T02:55:07.546 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.544+0000 7f6d50aa2b80 10 client.25180 fetch_fsmap finished waiting for FSMap version 30 2024-06-10T02:55:07.546 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.544+0000 7f6d50aa2b80 10 client.25180 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-06-10T02:55:07.546 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.544+0000 7f6d50aa2b80 20 client.25180 populate_metadata read hostname 'smithi052' 2024-06-10T02:55:07.546 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.544+0000 7f6d50aa2b80 1 --2- 172.21.15.52:0/93442877 >> [v2:172.21.15.120:6836/4132440952,v1:172.21.15.120:6837/4132440952] conn(0x5641774b0c90 0x5641774b3080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:07.547 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.544+0000 7f6d50aa2b80 4 client.25180 mds_command: new command op to 24428 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T02:55:07.547 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.544+0000 7f6d50aa2b80 1 -- 172.21.15.52:0/93442877 --> [v2:172.21.15.120:6836/4132440952,v1:172.21.15.120:6837/4132440952] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f6d300a0c10 con 0x5641774b0c90 2024-06-10T02:55:07.547 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.544+0000 7f6d3effd700 1 --2- 172.21.15.52:0/93442877 >> [v2:172.21.15.120:6836/4132440952,v1:172.21.15.120:6837/4132440952] conn(0x5641774b0c90 0x5641774b3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:07.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.544+0000 7f6d1effd700 10 client.25180 ms_handle_connect on v2:172.21.15.120:6836/4132440952 2024-06-10T02:55:07.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.545+0000 7f6d3effd700 1 --2- 172.21.15.52:0/93442877 >> [v2:172.21.15.120:6836/4132440952,v1:172.21.15.120:6837/4132440952] conn(0x5641774b0c90 0x5641774b3080 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b4cfbf4c12f3f6a0 server_cookie=525fa90bc3295c90 in_seq=0 out_seq=0 2024-06-10T02:55:07.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.545+0000 7f6d1effd700 1 -- 172.21.15.52:0/93442877 <== mds.4 v2:172.21.15.120:6836/4132440952 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f6d300a0c10 con 0x5641774b0c90 2024-06-10T02:55:07.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.545+0000 7f6d1effd700 10 client.25180 handle_command_reply: tid=0 2024-06-10T02:55:07.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.545+0000 7f6d50aa2b80 1 client.25180 shutdown 2024-06-10T02:55:07.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.545+0000 7f6d50aa2b80 1 -- 172.21.15.52:0/93442877 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564177431a70 msgr2=0x56417743e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:55:07.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.545+0000 7f6d50aa2b80 1 --2- 172.21.15.52:0/93442877 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564177431a70 0x56417743e970 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f6d30003900 tx=0x7f6d30022970 comp rx=0 tx=0).stop 2024-06-10T02:55:07.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.545+0000 7f6d50aa2b80 1 -- 172.21.15.52:0/93442877 shutdown_connections 2024-06-10T02:55:07.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.545+0000 7f6d50aa2b80 1 --2- 172.21.15.52:0/93442877 >> [v2:172.21.15.120:6836/4132440952,v1:172.21.15.120:6837/4132440952] conn(0x5641774b0c90 0x5641774b3080 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:07.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.545+0000 7f6d50aa2b80 1 --2- 172.21.15.52:0/93442877 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564177445da0 0x56417744d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:07.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.545+0000 7f6d1effd700 0 client.25180 ms_handle_reset on v2:172.21.15.120:6836/4132440952 2024-06-10T02:55:07.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.545+0000 7f6d50aa2b80 1 --2- 172.21.15.52:0/93442877 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564177431a70 0x56417743e970 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:07.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.545+0000 7f6d50aa2b80 1 --2- 172.21.15.52:0/93442877 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564177432390 0x56417744d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:07.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.545+0000 7f6d50aa2b80 1 -- 172.21.15.52:0/93442877 >> 172.21.15.52:0/93442877 conn(0x564177352cc0 msgr2=0x5641774269d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:55:07.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.545+0000 7f6d50aa2b80 1 -- 172.21.15.52:0/93442877 shutdown_connections 2024-06-10T02:55:07.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.545+0000 7f6d50aa2b80 1 -- 172.21.15.52:0/93442877 wait complete. 2024-06-10T02:55:07.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.545+0000 7f6d50aa2b80 20 client.25180 trim_cache size 0 max 16384 2024-06-10T02:55:07.549 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-06-10T02:55:07.550 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.546+0000 7f6d4621f700 1 -- 172.21.15.52:0/3706918625 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f6d280a2d30 msgr2=0x7f6d280a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:55:07.550 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.546+0000 7f6d4621f700 1 --2- 172.21.15.52:0/3706918625 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f6d280a2d30 0x7f6d280a51f0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f6d40078840 tx=0x7f6d3400a380 comp rx=0 tx=0).stop 2024-06-10T02:55:07.550 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.546+0000 7f6d4621f700 1 -- 172.21.15.52:0/3706918625 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6d4006a170 msgr2=0x7f6d4019e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:55:07.550 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.546+0000 7f6d4621f700 1 --2- 172.21.15.52:0/3706918625 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6d4006a170 0x7f6d4019e760 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f6d30000c00 tx=0x7f6d30003680 comp rx=0 tx=0).stop 2024-06-10T02:55:07.550 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.546+0000 7f6d4621f700 1 -- 172.21.15.52:0/3706918625 shutdown_connections 2024-06-10T02:55:07.550 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.546+0000 7f6d4621f700 1 --2- 172.21.15.52:0/3706918625 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6d40108500 0x7f6d40198920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:07.550 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.546+0000 7f6d4621f700 1 --2- 172.21.15.52:0/3706918625 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f6d280a2d30 0x7f6d280a51f0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:07.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.546+0000 7f6d4621f700 1 --2- 172.21.15.52:0/3706918625 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6d4006a170 0x7f6d4019e760 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:07.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.546+0000 7f6d4621f700 1 --2- 172.21.15.52:0/3706918625 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6d4006ab30 0x7f6d4019eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:07.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.546+0000 7f6d4621f700 1 -- 172.21.15.52:0/3706918625 >> 172.21.15.52:0/3706918625 conn(0x7f6d40078340 msgr2=0x7f6d400ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:55:07.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.546+0000 7f6d4621f700 1 -- 172.21.15.52:0/3706918625 shutdown_connections 2024-06-10T02:55:07.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:07.547+0000 7f6d4621f700 1 -- 172.21.15.52:0/3706918625 wait complete. 2024-06-10T02:55:08.280 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-06-10T02:55:08.555 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:55:08.976 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:55:08 smithi052 ceph-mon[32362]: pgmap v462: 129 pgs: 129 active+clean; 661 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-10T02:55:09.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:55:08 smithi120 ceph-mon[36634]: pgmap v462: 129 pgs: 129 active+clean; 661 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-10T02:55:09.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:55:08 smithi188 ceph-mon[36599]: pgmap v462: 129 pgs: 129 active+clean; 661 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-10T02:55:10.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.327+0000 7fa8d1059700 1 -- 172.21.15.52:0/3751771774 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa8cc101390 msgr2=0x7fa8cc101810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:55:10.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.327+0000 7fa8d1059700 1 --2- 172.21.15.52:0/3751771774 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa8cc101390 0x7fa8cc101810 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fa8c0009990 tx=0x7fa8c0009ca0 comp rx=0 tx=0).stop 2024-06-10T02:55:10.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.327+0000 7fa8d1059700 1 -- 172.21.15.52:0/3751771774 shutdown_connections 2024-06-10T02:55:10.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.327+0000 7fa8d1059700 1 --2- 172.21.15.52:0/3751771774 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa8cc101d50 0x7fa8cc10e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:10.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.327+0000 7fa8d1059700 1 --2- 172.21.15.52:0/3751771774 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa8cc107400 0x7fa8cc1077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:10.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.327+0000 7fa8d1059700 1 --2- 172.21.15.52:0/3751771774 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa8cc101390 0x7fa8cc101810 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:10.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.327+0000 7fa8d1059700 1 -- 172.21.15.52:0/3751771774 >> 172.21.15.52:0/3751771774 conn(0x7fa8cc078340 msgr2=0x7fa8cc078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:55:10.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.328+0000 7fa8d1059700 1 -- 172.21.15.52:0/3751771774 shutdown_connections 2024-06-10T02:55:10.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.328+0000 7fa8d1059700 1 -- 172.21.15.52:0/3751771774 wait complete. 2024-06-10T02:55:10.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.329+0000 7fa8d1059700 1 Processor -- start 2024-06-10T02:55:10.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.329+0000 7fa8d1059700 1 -- start start 2024-06-10T02:55:10.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.329+0000 7fa8d1059700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa8cc101390 0x7fa8cc19e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:10.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.329+0000 7fa8d1059700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa8cc101d50 0x7fa8cc19ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:10.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.330+0000 7fa8d1059700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa8cc107400 0x7fa8cc198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:10.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.330+0000 7fa8d1059700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fa8cc111670 con 0x7fa8cc101d50 2024-06-10T02:55:10.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.330+0000 7fa8ca59c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa8cc101d50 0x7fa8cc19ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:10.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.330+0000 7fa8ca59c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa8cc101d50 0x7fa8cc19ed10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:42760/0 (socket says 172.21.15.52:42760) 2024-06-10T02:55:10.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.330+0000 7fa8ca59c700 1 -- 172.21.15.52:0/1779588390 learned_addr learned my addr 172.21.15.52:0/1779588390 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:55:10.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.330+0000 7fa8cad9d700 1 --2- 172.21.15.52:0/1779588390 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa8cc101390 0x7fa8cc19e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:10.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.330+0000 7fa8d1059700 1 -- 172.21.15.52:0/1779588390 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa8cc1114f0 con 0x7fa8cc101390 2024-06-10T02:55:10.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.330+0000 7fa8d1059700 1 -- 172.21.15.52:0/1779588390 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa8cc1117f0 con 0x7fa8cc107400 2024-06-10T02:55:10.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.330+0000 7fa8cb59e700 1 --2- 172.21.15.52:0/1779588390 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa8cc107400 0x7fa8cc198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:10.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.330+0000 7fa8ca59c700 1 -- 172.21.15.52:0/1779588390 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa8cc107400 msgr2=0x7fa8cc198990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:55:10.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.330+0000 7fa8ca59c700 1 --2- 172.21.15.52:0/1779588390 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa8cc107400 0x7fa8cc198990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:10.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.330+0000 7fa8ca59c700 1 -- 172.21.15.52:0/1779588390 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa8cc101390 msgr2=0x7fa8cc19e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:55:10.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.330+0000 7fa8ca59c700 1 --2- 172.21.15.52:0/1779588390 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa8cc101390 0x7fa8cc19e7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:10.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.330+0000 7fa8ca59c700 1 -- 172.21.15.52:0/1779588390 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8cc1990f0 con 0x7fa8cc101d50 2024-06-10T02:55:10.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.331+0000 7fa8ca59c700 1 --2- 172.21.15.52:0/1779588390 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa8cc101d50 0x7fa8cc19ed10 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fa8c00038b0 tx=0x7fa8c0009f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:55:10.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.331+0000 7fa8b3fff700 1 -- 172.21.15.52:0/1779588390 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8c0027070 con 0x7fa8cc101d50 2024-06-10T02:55:10.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.331+0000 7fa8b3fff700 1 -- 172.21.15.52:0/1779588390 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa8c0004050 con 0x7fa8cc101d50 2024-06-10T02:55:10.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.331+0000 7fa8b3fff700 1 -- 172.21.15.52:0/1779588390 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8c0022c00 con 0x7fa8cc101d50 2024-06-10T02:55:10.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.332+0000 7fa8d1059700 1 -- 172.21.15.52:0/1779588390 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8cc199380 con 0x7fa8cc101d50 2024-06-10T02:55:10.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.332+0000 7fa8d1059700 1 -- 172.21.15.52:0/1779588390 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8cc1a56a0 con 0x7fa8cc101d50 2024-06-10T02:55:10.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.334+0000 7fa8b3fff700 1 -- 172.21.15.52:0/1779588390 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7fa8c0003b70 con 0x7fa8cc101d50 2024-06-10T02:55:10.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.335+0000 7fa8b3fff700 1 --2- 172.21.15.52:0/1779588390 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fa8b40a2d30 0x7fa8b40a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:10.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.336+0000 7fa8cad9d700 1 --2- 172.21.15.52:0/1779588390 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fa8b40a2d30 0x7fa8b40a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:10.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.336+0000 7fa8b3fff700 1 -- 172.21.15.52:0/1779588390 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7fa8c00d03b0 con 0x7fa8cc101d50 2024-06-10T02:55:10.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.336+0000 7fa8cad9d700 1 --2- 172.21.15.52:0/1779588390 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fa8b40a2d30 0x7fa8b40a51f0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fa8cc199ad0 tx=0x7fa8bc00a380 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:55:10.336 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.337+0000 7fa8db8dcb80 1 Processor -- start 2024-06-10T02:55:10.336 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.337+0000 7fa8db8dcb80 1 -- start start 2024-06-10T02:55:10.336 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.337+0000 7fa8db8dcb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558b3143c640 0x558b3143ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:10.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.337+0000 7fa8db8dcb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558b3143cf60 0x558b314369e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:10.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.337+0000 7fa8db8dcb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558b31436f20 0x558b31437340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:10.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.337+0000 7fa8db8dcb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x558b31388810 con 0x558b3143cf60 2024-06-10T02:55:10.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.337+0000 7fa8db8dcb80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x558b31330720 con 0x558b3143c640 2024-06-10T02:55:10.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.337+0000 7fa8db8dcb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x558b313847d0 con 0x558b31436f20 2024-06-10T02:55:10.338 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.337+0000 7fa8ca59c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558b3143cf60 0x558b314369e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:10.338 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.337+0000 7fa8ca59c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558b3143cf60 0x558b314369e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:42772/0 (socket says 172.21.15.52:42772) 2024-06-10T02:55:10.338 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.337+0000 7fa8ca59c700 1 -- 172.21.15.52:0/2863732010 learned_addr learned my addr 172.21.15.52:0/2863732010 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:55:10.338 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.337+0000 7fa8cb59e700 1 --2- 172.21.15.52:0/2863732010 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558b31436f20 0x558b31437340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:10.338 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.338+0000 7fa8cad9d700 1 --2- 172.21.15.52:0/2863732010 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558b3143c640 0x558b3143ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:10.338 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.338+0000 7fa8ca59c700 1 -- 172.21.15.52:0/2863732010 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558b31436f20 msgr2=0x558b31437340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:55:10.338 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.338+0000 7fa8ca59c700 1 --2- 172.21.15.52:0/2863732010 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558b31436f20 0x558b31437340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:10.338 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.338+0000 7fa8ca59c700 1 -- 172.21.15.52:0/2863732010 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558b3143c640 msgr2=0x558b3143ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:55:10.338 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.338+0000 7fa8ca59c700 1 --2- 172.21.15.52:0/2863732010 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558b3143c640 0x558b3143ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:10.338 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.338+0000 7fa8ca59c700 1 -- 172.21.15.52:0/2863732010 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b31326f50 con 0x558b3143cf60 2024-06-10T02:55:10.339 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.338+0000 7fa8ca59c700 1 --2- 172.21.15.52:0/2863732010 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558b3143cf60 0x558b314369e0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fa8c0099100 tx=0x7fa8c009c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1294fda19cdb49e5 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:55:10.339 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.338+0000 7fa8b17fa700 1 -- 172.21.15.52:0/2863732010 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8c009c090 con 0x558b3143cf60 2024-06-10T02:55:10.339 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.338+0000 7fa8b17fa700 1 -- 172.21.15.52:0/2863732010 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa8c009ce30 con 0x558b3143cf60 2024-06-10T02:55:10.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.338+0000 7fa8b17fa700 1 -- 172.21.15.52:0/2863732010 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8c0003e60 con 0x558b3143cf60 2024-06-10T02:55:10.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.339+0000 7fa8db8dcb80 1 -- 172.21.15.52:0/2863732010 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558b3143cf60 msgr2=0x558b314369e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:55:10.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.339+0000 7fa8db8dcb80 1 --2- 172.21.15.52:0/2863732010 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558b3143cf60 0x558b314369e0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fa8c0099100 tx=0x7fa8c009c500 comp rx=0 tx=0).stop 2024-06-10T02:55:10.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.339+0000 7fa8db8dcb80 1 -- 172.21.15.52:0/2863732010 shutdown_connections 2024-06-10T02:55:10.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.339+0000 7fa8db8dcb80 1 --2- 172.21.15.52:0/2863732010 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558b31436f20 0x558b31437340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:10.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.339+0000 7fa8db8dcb80 1 --2- 172.21.15.52:0/2863732010 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558b3143c640 0x558b3143ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:10.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.339+0000 7fa8db8dcb80 1 --2- 172.21.15.52:0/2863732010 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558b3143cf60 0x558b314369e0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:10.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.339+0000 7fa8db8dcb80 1 -- 172.21.15.52:0/2863732010 >> 172.21.15.52:0/2863732010 conn(0x558b31361cc0 msgr2=0x558b31431850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:55:10.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.339+0000 7fa8db8dcb80 1 -- 172.21.15.52:0/2863732010 shutdown_connections 2024-06-10T02:55:10.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.339+0000 7fa8db8dcb80 1 -- 172.21.15.52:0/2863732010 wait complete. 2024-06-10T02:55:10.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.340+0000 7fa8db8dcb80 1 -- start start 2024-06-10T02:55:10.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.340+0000 7fa8db8dcb80 1 Processor -- start 2024-06-10T02:55:10.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.340+0000 7fa8db8dcb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558b31436f20 0x558b31442cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:10.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.340+0000 7fa8db8dcb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558b3143c640 0x558b3143fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:10.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.340+0000 7fa8db8dcb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558b3143cf60 0x558b31440530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:10.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.340+0000 7fa8db8dcb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x558b31447b00 con 0x558b31436f20 2024-06-10T02:55:10.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.340+0000 7fa8db8dcb80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x558b31328bf0 con 0x558b3143c640 2024-06-10T02:55:10.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.340+0000 7fa8db8dcb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x558b31447c80 con 0x558b3143cf60 2024-06-10T02:55:10.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.340+0000 7fa8db8dcb80 10 client.0 _ll_register_callbacks cb 0x558b31322920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T02:55:10.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.340+0000 7fa8cad9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558b31436f20 0x558b31442cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:10.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.340+0000 7fa8cb59e700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558b3143cf60 0x558b31440530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:10.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.340+0000 7fa8cad9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558b31436f20 0x558b31442cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:42774/0 (socket says 172.21.15.52:42774) 2024-06-10T02:55:10.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.340+0000 7fa8cad9d700 1 -- 172.21.15.52:0/2161646232 learned_addr learned my addr 172.21.15.52:0/2161646232 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:55:10.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.340+0000 7fa8ca59c700 1 --2- 172.21.15.52:0/2161646232 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558b3143c640 0x558b3143fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:10.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.341+0000 7fa8ca59c700 1 -- 172.21.15.52:0/2161646232 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558b3143cf60 msgr2=0x558b31440530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:55:10.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.341+0000 7fa8ca59c700 1 --2- 172.21.15.52:0/2161646232 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558b3143cf60 0x558b31440530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:10.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.341+0000 7fa8ca59c700 1 -- 172.21.15.52:0/2161646232 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558b31436f20 msgr2=0x558b31442cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:55:10.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.341+0000 7fa8ca59c700 1 --2- 172.21.15.52:0/2161646232 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558b31436f20 0x558b31442cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:10.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.341+0000 7fa8ca59c700 1 -- 172.21.15.52:0/2161646232 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b312d78d0 con 0x558b3143c640 2024-06-10T02:55:10.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.341+0000 7fa8cad9d700 1 --2- 172.21.15.52:0/2161646232 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558b31436f20 0x558b31442cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T02:55:10.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.341+0000 7fa8ca59c700 1 --2- 172.21.15.52:0/2161646232 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558b3143c640 0x558b3143fff0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fa8c009c6c0 tx=0x7fa8c00092f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=523fb0c75de08b22 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:55:10.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.341+0000 7fa8aeff5700 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-06-10T02:55:10.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.341+0000 7fa8aeff5700 1 -- 172.21.15.52:0/2161646232 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8c009c090 con 0x558b3143c640 2024-06-10T02:55:10.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.341+0000 7fa8aeff5700 1 -- 172.21.15.52:0/2161646232 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7fa8c0022470 con 0x558b3143c640 2024-06-10T02:55:10.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.341+0000 7fa8db8dcb80 1 -- 172.21.15.52:0/2161646232 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa8c009ce30 con 0x558b3143c640 2024-06-10T02:55:10.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.342+0000 7fa8aeff5700 1 -- 172.21.15.52:0/2161646232 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa8c009c730 con 0x558b3143c640 2024-06-10T02:55:10.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.342+0000 7fa8db8dcb80 10 client.25217 fetch_fsmap learned FSMap version 30 2024-06-10T02:55:10.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.342+0000 7fa8db8dcb80 1 -- 172.21.15.52:0/2161646232 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x558b31447b00 con 0x558b3143c640 2024-06-10T02:55:10.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.342+0000 7fa8aeff5700 1 -- 172.21.15.52:0/2161646232 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa8c00a0ae0 con 0x558b3143c640 2024-06-10T02:55:10.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.342+0000 7fa8aeff5700 1 -- 172.21.15.52:0/2161646232 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fa8c00a4f10 con 0x558b3143c640 2024-06-10T02:55:10.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.342+0000 7fa8db8dcb80 10 client.25217 fetch_fsmap finished waiting for FSMap version 30 2024-06-10T02:55:10.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.342+0000 7fa8db8dcb80 10 client.25217 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.b 2024-06-10T02:55:10.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.342+0000 7fa8db8dcb80 20 client.25217 populate_metadata read hostname 'smithi052' 2024-06-10T02:55:10.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.342+0000 7fa8db8dcb80 1 --2- 172.21.15.52:0/2161646232 >> [v2:172.21.15.120:6838/1672436487,v1:172.21.15.120:6839/1672436487] conn(0x558b314549a0 0x558b31456dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:10.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.342+0000 7fa8db8dcb80 4 client.25217 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T02:55:10.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.342+0000 7fa8db8dcb80 1 -- 172.21.15.52:0/2161646232 --> [v2:172.21.15.120:6838/1672436487,v1:172.21.15.120:6839/1672436487] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fa8c009ce30 con 0x558b314549a0 2024-06-10T02:55:10.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.342+0000 7fa8cad9d700 1 --2- 172.21.15.52:0/2161646232 >> [v2:172.21.15.120:6838/1672436487,v1:172.21.15.120:6839/1672436487] conn(0x558b314549a0 0x558b31456dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:10.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.343+0000 7fa8cad9d700 1 --2- 172.21.15.52:0/2161646232 >> [v2:172.21.15.120:6838/1672436487,v1:172.21.15.120:6839/1672436487] conn(0x558b314549a0 0x558b31456dc0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=12defee290e3e9fb server_cookie=caf2c10762f60355 in_seq=0 out_seq=0 2024-06-10T02:55:10.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.343+0000 7fa8aeff5700 10 client.25217 ms_handle_connect on v2:172.21.15.120:6838/1672436487 2024-06-10T02:55:10.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.344+0000 7fa8aeff5700 1 -- 172.21.15.52:0/2161646232 <== mds.3 v2:172.21.15.120:6838/1672436487 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x7fa8c009ce30 con 0x558b314549a0 2024-06-10T02:55:10.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.344+0000 7fa8aeff5700 10 client.25217 handle_command_reply: tid=0 2024-06-10T02:55:10.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.344+0000 7fa8db8dcb80 1 client.25217 shutdown 2024-06-10T02:55:10.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.344+0000 7fa8db8dcb80 1 -- 172.21.15.52:0/2161646232 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558b3143c640 msgr2=0x558b3143fff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:55:10.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.344+0000 7fa8db8dcb80 1 --2- 172.21.15.52:0/2161646232 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558b3143c640 0x558b3143fff0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fa8c009c6c0 tx=0x7fa8c00092f0 comp rx=0 tx=0).stop 2024-06-10T02:55:10.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.344+0000 7fa8db8dcb80 1 -- 172.21.15.52:0/2161646232 shutdown_connections 2024-06-10T02:55:10.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.344+0000 7fa8db8dcb80 1 --2- 172.21.15.52:0/2161646232 >> [v2:172.21.15.120:6838/1672436487,v1:172.21.15.120:6839/1672436487] conn(0x558b314549a0 0x558b31456dc0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:10.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.344+0000 7fa8db8dcb80 1 --2- 172.21.15.52:0/2161646232 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558b3143cf60 0x558b31440530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:10.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.344+0000 7fa8aeff5700 0 client.25217 ms_handle_reset on v2:172.21.15.120:6838/1672436487 2024-06-10T02:55:10.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.344+0000 7fa8db8dcb80 1 --2- 172.21.15.52:0/2161646232 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558b31436f20 0x558b31442cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:10.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.344+0000 7fa8db8dcb80 1 --2- 172.21.15.52:0/2161646232 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558b3143c640 0x558b3143fff0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:10.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.344+0000 7fa8db8dcb80 1 -- 172.21.15.52:0/2161646232 >> 172.21.15.52:0/2161646232 conn(0x558b31361cc0 msgr2=0x558b314315f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:55:10.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.344+0000 7fa8db8dcb80 1 -- 172.21.15.52:0/2161646232 shutdown_connections 2024-06-10T02:55:10.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.344+0000 7fa8db8dcb80 1 -- 172.21.15.52:0/2161646232 wait complete. 2024-06-10T02:55:10.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.345+0000 7fa8db8dcb80 20 client.25217 trim_cache size 0 max 16384 2024-06-10T02:55:10.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.360+0000 7fa8db8dcb80 1 Processor -- start 2024-06-10T02:55:10.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.360+0000 7fa8db8dcb80 1 -- start start 2024-06-10T02:55:10.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.360+0000 7fa8db8dcb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558b31436f20 0x558b314369e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:10.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.360+0000 7fa8db8dcb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558b3143c640 0x558b314406c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:10.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.360+0000 7fa8db8dcb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558b3143cf60 0x558b3143d400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:10.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.360+0000 7fa8db8dcb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fa8cc1ac0f0 con 0x558b3143c640 2024-06-10T02:55:10.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.360+0000 7fa8db8dcb80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa8cc1adc30 con 0x558b31436f20 2024-06-10T02:55:10.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.360+0000 7fa8db8dcb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa8cc1ac7c0 con 0x558b3143cf60 2024-06-10T02:55:10.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.360+0000 7fa8ca59c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558b3143c640 0x558b314406c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:10.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.360+0000 7fa8ca59c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558b3143c640 0x558b314406c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:42784/0 (socket says 172.21.15.52:42784) 2024-06-10T02:55:10.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.360+0000 7fa8ca59c700 1 -- 172.21.15.52:0/2722922756 learned_addr learned my addr 172.21.15.52:0/2722922756 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:55:10.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.361+0000 7fa8cad9d700 1 --2- 172.21.15.52:0/2722922756 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558b31436f20 0x558b314369e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:10.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.361+0000 7fa8cb59e700 1 --2- 172.21.15.52:0/2722922756 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558b3143cf60 0x558b3143d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:10.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.361+0000 7fa8ca59c700 1 -- 172.21.15.52:0/2722922756 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558b3143cf60 msgr2=0x558b3143d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:55:10.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.361+0000 7fa8ca59c700 1 --2- 172.21.15.52:0/2722922756 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558b3143cf60 0x558b3143d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:10.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.361+0000 7fa8ca59c700 1 -- 172.21.15.52:0/2722922756 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558b31436f20 msgr2=0x558b314369e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:55:10.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.361+0000 7fa8ca59c700 1 --2- 172.21.15.52:0/2722922756 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558b31436f20 0x558b314369e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:10.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.361+0000 7fa8ca59c700 1 -- 172.21.15.52:0/2722922756 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b3143e580 con 0x558b3143c640 2024-06-10T02:55:10.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.361+0000 7fa8ca59c700 1 --2- 172.21.15.52:0/2722922756 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558b3143c640 0x558b314406c0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fa8c009a9e0 tx=0x7fa8c00b0c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f090aa5d911d089 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:55:10.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.361+0000 7fa8b0ff9700 1 -- 172.21.15.52:0/2722922756 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8c009c090 con 0x558b3143c640 2024-06-10T02:55:10.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.361+0000 7fa8b0ff9700 1 -- 172.21.15.52:0/2722922756 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa8c009ce30 con 0x558b3143c640 2024-06-10T02:55:10.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.361+0000 7fa8b0ff9700 1 -- 172.21.15.52:0/2722922756 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8c009c500 con 0x558b3143c640 2024-06-10T02:55:10.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.362+0000 7fa8db8dcb80 1 -- 172.21.15.52:0/2722922756 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558b3143c640 msgr2=0x558b314406c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:55:10.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.362+0000 7fa8db8dcb80 1 --2- 172.21.15.52:0/2722922756 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558b3143c640 0x558b314406c0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fa8c009a9e0 tx=0x7fa8c00b0c10 comp rx=0 tx=0).stop 2024-06-10T02:55:10.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.362+0000 7fa8db8dcb80 1 -- 172.21.15.52:0/2722922756 shutdown_connections 2024-06-10T02:55:10.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.363+0000 7fa8db8dcb80 1 --2- 172.21.15.52:0/2722922756 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558b3143cf60 0x558b3143d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:10.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.363+0000 7fa8db8dcb80 1 --2- 172.21.15.52:0/2722922756 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558b31436f20 0x558b314369e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:10.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.363+0000 7fa8db8dcb80 1 --2- 172.21.15.52:0/2722922756 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558b3143c640 0x558b314406c0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:10.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.363+0000 7fa8db8dcb80 1 -- 172.21.15.52:0/2722922756 >> 172.21.15.52:0/2722922756 conn(0x558b31361cc0 msgr2=0x558b3143ffb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:55:10.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.363+0000 7fa8db8dcb80 1 -- 172.21.15.52:0/2722922756 shutdown_connections 2024-06-10T02:55:10.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.363+0000 7fa8db8dcb80 1 -- 172.21.15.52:0/2722922756 wait complete. 2024-06-10T02:55:10.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.363+0000 7fa8db8dcb80 1 -- start start 2024-06-10T02:55:10.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.363+0000 7fa8db8dcb80 1 Processor -- start 2024-06-10T02:55:10.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.364+0000 7fa8db8dcb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558b31436f20 0x558b3145b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:10.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.364+0000 7fa8db8dcb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558b3143c640 0x558b3145ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:10.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.364+0000 7fa8db8dcb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558b3143cf60 0x558b3145bf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:10.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.364+0000 7fa8db8dcb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fa8cc1ad560 con 0x558b3143c640 2024-06-10T02:55:10.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.364+0000 7fa8db8dcb80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa8cc1ace90 con 0x558b31436f20 2024-06-10T02:55:10.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.364+0000 7fa8db8dcb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa8cc1aa090 con 0x558b3143cf60 2024-06-10T02:55:10.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.364+0000 7fa8ca59c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558b3143c640 0x558b3145ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:10.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.364+0000 7fa8db8dcb80 10 client.0 _ll_register_callbacks cb 0x558b31322920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T02:55:10.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.364+0000 7fa8ca59c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558b3143c640 0x558b3145ba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:42798/0 (socket says 172.21.15.52:42798) 2024-06-10T02:55:10.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.364+0000 7fa8ca59c700 1 -- 172.21.15.52:0/1725534519 learned_addr learned my addr 172.21.15.52:0/1725534519 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:55:10.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.364+0000 7fa8cad9d700 1 --2- 172.21.15.52:0/1725534519 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558b31436f20 0x558b3145b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:10.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.364+0000 7fa8cb59e700 1 --2- 172.21.15.52:0/1725534519 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558b3143cf60 0x558b3145bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:10.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.364+0000 7fa8ca59c700 1 -- 172.21.15.52:0/1725534519 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558b3143cf60 msgr2=0x558b3145bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:55:10.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.364+0000 7fa8ca59c700 1 --2- 172.21.15.52:0/1725534519 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558b3143cf60 0x558b3145bf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:10.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.364+0000 7fa8ca59c700 1 -- 172.21.15.52:0/1725534519 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558b31436f20 msgr2=0x558b3145b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:55:10.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.364+0000 7fa8ca59c700 1 --2- 172.21.15.52:0/1725534519 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558b31436f20 0x558b3145b4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:10.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.364+0000 7fa8ca59c700 1 -- 172.21.15.52:0/1725534519 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b3144bf30 con 0x558b3143c640 2024-06-10T02:55:10.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.364+0000 7fa8ca59c700 1 --2- 172.21.15.52:0/1725534519 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558b3143c640 0x558b3145ba10 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fa8c009c6c0 tx=0x7fa8c009ac40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3be79885d3ff697b server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:55:10.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.364+0000 7fa8aeff5700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-10T02:55:10.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.365+0000 7fa8aeff5700 1 -- 172.21.15.52:0/1725534519 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8c009c500 con 0x558b3143c640 2024-06-10T02:55:10.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.365+0000 7fa8aeff5700 1 -- 172.21.15.52:0/1725534519 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7fa8c00a5080 con 0x558b3143c640 2024-06-10T02:55:10.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.365+0000 7fa8db8dcb80 1 -- 172.21.15.52:0/1725534519 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa8c009ce30 con 0x558b3143c640 2024-06-10T02:55:10.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.365+0000 7fa8aeff5700 1 -- 172.21.15.52:0/1725534519 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa8c009c090 con 0x558b3143c640 2024-06-10T02:55:10.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.365+0000 7fa8db8dcb80 10 client.15585 fetch_fsmap learned FSMap version 30 2024-06-10T02:55:10.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.365+0000 7fa8db8dcb80 1 -- 172.21.15.52:0/1725534519 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x558b3144bf30 con 0x558b3143c640 2024-06-10T02:55:10.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.365+0000 7fa8aeff5700 1 -- 172.21.15.52:0/1725534519 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa8c009ce30 con 0x558b3143c640 2024-06-10T02:55:10.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.365+0000 7fa8aeff5700 1 -- 172.21.15.52:0/1725534519 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fa8c009b5f0 con 0x558b3143c640 2024-06-10T02:55:10.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.365+0000 7fa8db8dcb80 10 client.15585 fetch_fsmap finished waiting for FSMap version 30 2024-06-10T02:55:10.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.365+0000 7fa8db8dcb80 10 client.15585 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.b 2024-06-10T02:55:10.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.365+0000 7fa8db8dcb80 20 client.15585 populate_metadata read hostname 'smithi052' 2024-06-10T02:55:10.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.366+0000 7fa8db8dcb80 1 --2- 172.21.15.52:0/1725534519 >> [v2:172.21.15.120:6838/1672436487,v1:172.21.15.120:6839/1672436487] conn(0x558b314549a0 0x558b3144d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:10.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.366+0000 7fa8db8dcb80 4 client.15585 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T02:55:10.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.366+0000 7fa8db8dcb80 1 -- 172.21.15.52:0/1725534519 --> [v2:172.21.15.120:6838/1672436487,v1:172.21.15.120:6839/1672436487] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558b31395d10 con 0x558b314549a0 2024-06-10T02:55:10.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.366+0000 7fa8cad9d700 1 --2- 172.21.15.52:0/1725534519 >> [v2:172.21.15.120:6838/1672436487,v1:172.21.15.120:6839/1672436487] conn(0x558b314549a0 0x558b3144d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:10.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.366+0000 7fa8cad9d700 1 --2- 172.21.15.52:0/1725534519 >> [v2:172.21.15.120:6838/1672436487,v1:172.21.15.120:6839/1672436487] conn(0x558b314549a0 0x558b3144d300 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b4b839c982d5974f server_cookie=6ac2d37e0e8c522d in_seq=0 out_seq=0 2024-06-10T02:55:10.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.366+0000 7fa8aeff5700 10 client.15585 ms_handle_connect on v2:172.21.15.120:6838/1672436487 2024-06-10T02:55:10.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.367+0000 7fa8aeff5700 1 -- 172.21.15.52:0/1725534519 <== mds.3 v2:172.21.15.120:6838/1672436487 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558b31395d10 con 0x558b314549a0 2024-06-10T02:55:10.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.367+0000 7fa8aeff5700 10 client.15585 handle_command_reply: tid=0 2024-06-10T02:55:10.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.367+0000 7fa8db8dcb80 1 client.15585 shutdown 2024-06-10T02:55:10.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.367+0000 7fa8db8dcb80 1 -- 172.21.15.52:0/1725534519 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558b3143c640 msgr2=0x558b3145ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:55:10.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.367+0000 7fa8db8dcb80 1 --2- 172.21.15.52:0/1725534519 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558b3143c640 0x558b3145ba10 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fa8c009c6c0 tx=0x7fa8c009ac40 comp rx=0 tx=0).stop 2024-06-10T02:55:10.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.367+0000 7fa8db8dcb80 1 -- 172.21.15.52:0/1725534519 shutdown_connections 2024-06-10T02:55:10.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.367+0000 7fa8db8dcb80 1 --2- 172.21.15.52:0/1725534519 >> [v2:172.21.15.120:6838/1672436487,v1:172.21.15.120:6839/1672436487] conn(0x558b314549a0 0x558b3144d300 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:10.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.367+0000 7fa8db8dcb80 1 --2- 172.21.15.52:0/1725534519 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558b3143cf60 0x558b3145bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:10.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.367+0000 7fa8db8dcb80 1 --2- 172.21.15.52:0/1725534519 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558b31436f20 0x558b3145b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:10.369 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-06-10T02:55:10.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.367+0000 7fa8db8dcb80 1 --2- 172.21.15.52:0/1725534519 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558b3143c640 0x558b3145ba10 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:10.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.367+0000 7fa8db8dcb80 1 -- 172.21.15.52:0/1725534519 >> 172.21.15.52:0/1725534519 conn(0x558b31361cc0 msgr2=0x558b3143d400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:55:10.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.367+0000 7fa8aeff5700 0 client.15585 ms_handle_reset on v2:172.21.15.120:6838/1672436487 2024-06-10T02:55:10.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.367+0000 7fa8db8dcb80 1 -- 172.21.15.52:0/1725534519 shutdown_connections 2024-06-10T02:55:10.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.367+0000 7fa8db8dcb80 1 -- 172.21.15.52:0/1725534519 wait complete. 2024-06-10T02:55:10.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.367+0000 7fa8db8dcb80 20 client.15585 trim_cache size 0 max 16384 2024-06-10T02:55:10.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.368+0000 7fa8d1059700 1 -- 172.21.15.52:0/1779588390 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fa8b40a2d30 msgr2=0x7fa8b40a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:55:10.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.368+0000 7fa8d1059700 1 --2- 172.21.15.52:0/1779588390 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fa8b40a2d30 0x7fa8b40a51f0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fa8cc199ad0 tx=0x7fa8bc00a380 comp rx=0 tx=0).stop 2024-06-10T02:55:10.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.368+0000 7fa8d1059700 1 -- 172.21.15.52:0/1779588390 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa8cc101d50 msgr2=0x7fa8cc19ed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:55:10.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.368+0000 7fa8d1059700 1 --2- 172.21.15.52:0/1779588390 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa8cc101d50 0x7fa8cc19ed10 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fa8c00038b0 tx=0x7fa8c0009f50 comp rx=0 tx=0).stop 2024-06-10T02:55:10.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.368+0000 7fa8d1059700 1 -- 172.21.15.52:0/1779588390 shutdown_connections 2024-06-10T02:55:10.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.368+0000 7fa8d1059700 1 --2- 172.21.15.52:0/1779588390 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa8cc107400 0x7fa8cc198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:10.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.368+0000 7fa8d1059700 1 --2- 172.21.15.52:0/1779588390 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa8cc101390 0x7fa8cc19e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:10.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.368+0000 7fa8d1059700 1 --2- 172.21.15.52:0/1779588390 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fa8b40a2d30 0x7fa8b40a51f0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:10.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.368+0000 7fa8d1059700 1 --2- 172.21.15.52:0/1779588390 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa8cc101d50 0x7fa8cc19ed10 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:10.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.368+0000 7fa8d1059700 1 -- 172.21.15.52:0/1779588390 >> 172.21.15.52:0/1779588390 conn(0x7fa8cc078340 msgr2=0x7fa8cc1005a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:55:10.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.368+0000 7fa8d1059700 1 -- 172.21.15.52:0/1779588390 shutdown_connections 2024-06-10T02:55:10.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:10.368+0000 7fa8d1059700 1 -- 172.21.15.52:0/1779588390 wait complete. 2024-06-10T02:55:11.091 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-06-10T02:55:11.114 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:55:10 smithi052 ceph-mon[32362]: pgmap v463: 129 pgs: 129 active+clean; 661 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-10T02:55:11.137 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:55:10 smithi120 ceph-mon[36634]: pgmap v463: 129 pgs: 129 active+clean; 661 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-10T02:55:11.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:55:10 smithi188 ceph-mon[36599]: pgmap v463: 129 pgs: 129 active+clean; 661 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-10T02:55:11.395 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:55:12.928 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:55:12 smithi052 ceph-mon[32362]: pgmap v464: 129 pgs: 129 active+clean; 661 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-10T02:55:13.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:55:12 smithi120 ceph-mon[36634]: pgmap v464: 129 pgs: 129 active+clean; 661 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-10T02:55:13.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:55:12 smithi188 ceph-mon[36599]: pgmap v464: 129 pgs: 129 active+clean; 661 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-10T02:55:13.217 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.218+0000 7f6d91c1c700 1 -- 172.21.15.52:0/103325222 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6d8c108530 msgr2=0x7f6d8c108910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:55:13.217 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.218+0000 7f6d91c1c700 1 --2- 172.21.15.52:0/103325222 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6d8c108530 0x7f6d8c108910 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f6d7c009990 tx=0x7f6d7c009ca0 comp rx=0 tx=0).stop 2024-06-10T02:55:13.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.218+0000 7f6d91c1c700 1 -- 172.21.15.52:0/103325222 shutdown_connections 2024-06-10T02:55:13.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.218+0000 7f6d91c1c700 1 --2- 172.21.15.52:0/103325222 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6d8c06b520 0x7f6d8c10f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:13.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.218+0000 7f6d91c1c700 1 --2- 172.21.15.52:0/103325222 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6d8c108530 0x7f6d8c108910 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:13.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.218+0000 7f6d91c1c700 1 --2- 172.21.15.52:0/103325222 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6d8c06ab60 0x7f6d8c06afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:13.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.218+0000 7f6d91c1c700 1 -- 172.21.15.52:0/103325222 >> 172.21.15.52:0/103325222 conn(0x7f6d8c0783d0 msgr2=0x7f6d8c0787e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:55:13.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.218+0000 7f6d91c1c700 1 -- 172.21.15.52:0/103325222 shutdown_connections 2024-06-10T02:55:13.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.218+0000 7f6d91c1c700 1 -- 172.21.15.52:0/103325222 wait complete. 2024-06-10T02:55:13.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.218+0000 7f6d91c1c700 1 Processor -- start 2024-06-10T02:55:13.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.219+0000 7f6d91c1c700 1 -- start start 2024-06-10T02:55:13.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.219+0000 7f6d91c1c700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6d8c06ab60 0x7f6d8c101490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:13.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.219+0000 7f6d91c1c700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6d8c06b520 0x7f6d8c1019d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:13.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.219+0000 7f6d91c1c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6d8c108530 0x7f6d8c101f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:13.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.219+0000 7f6d8bfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6d8c108530 0x7f6d8c101f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:13.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.219+0000 7f6d8bfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6d8c108530 0x7f6d8c101f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49618/0 (socket says 172.21.15.52:49618) 2024-06-10T02:55:13.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.219+0000 7f6d91c1c700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f6d8c112790 con 0x7f6d8c108530 2024-06-10T02:55:13.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.219+0000 7f6d8bfff700 1 -- 172.21.15.52:0/4090050146 learned_addr learned my addr 172.21.15.52:0/4090050146 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:55:13.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.219+0000 7f6d91c1c700 1 -- 172.21.15.52:0/4090050146 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f6d8c112610 con 0x7f6d8c06b520 2024-06-10T02:55:13.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.219+0000 7f6d91c1c700 1 -- 172.21.15.52:0/4090050146 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f6d8c112910 con 0x7f6d8c06ab60 2024-06-10T02:55:13.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.219+0000 7f6d8affd700 1 --2- 172.21.15.52:0/4090050146 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6d8c06b520 0x7f6d8c1019d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:13.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.220+0000 7f6d8bfff700 1 -- 172.21.15.52:0/4090050146 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6d8c06ab60 msgr2=0x7f6d8c101490 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-06-10T02:55:13.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.220+0000 7f6d8bfff700 1 --2- 172.21.15.52:0/4090050146 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6d8c06ab60 0x7f6d8c101490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:13.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.220+0000 7f6d8bfff700 1 -- 172.21.15.52:0/4090050146 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6d8c06b520 msgr2=0x7f6d8c1019d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:55:13.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.220+0000 7f6d8bfff700 1 --2- 172.21.15.52:0/4090050146 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6d8c06b520 0x7f6d8c1019d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:13.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.220+0000 7f6d8bfff700 1 -- 172.21.15.52:0/4090050146 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d8c1a7270 con 0x7f6d8c108530 2024-06-10T02:55:13.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.220+0000 7f6d8bfff700 1 --2- 172.21.15.52:0/4090050146 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6d8c108530 0x7f6d8c101f10 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f6d8400ef30 tx=0x7f6d8400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:55:13.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.220+0000 7f6d88ff9700 1 -- 172.21.15.52:0/4090050146 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d840092e0 con 0x7f6d8c108530 2024-06-10T02:55:13.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.220+0000 7f6d88ff9700 1 -- 172.21.15.52:0/4090050146 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d84004bb0 con 0x7f6d8c108530 2024-06-10T02:55:13.222 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.220+0000 7f6d91c1c700 1 -- 172.21.15.52:0/4090050146 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d8c1a7500 con 0x7f6d8c108530 2024-06-10T02:55:13.222 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.221+0000 7f6d88ff9700 1 -- 172.21.15.52:0/4090050146 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d84004280 con 0x7f6d8c108530 2024-06-10T02:55:13.222 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.221+0000 7f6d91c1c700 1 -- 172.21.15.52:0/4090050146 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d8c1a7a40 con 0x7f6d8c108530 2024-06-10T02:55:13.222 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.223+0000 7f6d88ff9700 1 -- 172.21.15.52:0/4090050146 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7f6d8401b030 con 0x7f6d8c108530 2024-06-10T02:55:13.223 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.224+0000 7f6d88ff9700 1 --2- 172.21.15.52:0/4090050146 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f6d740a2d30 0x7f6d740a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:13.223 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.224+0000 7f6d8b7fe700 1 --2- 172.21.15.52:0/4090050146 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f6d740a2d30 0x7f6d740a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:13.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.224+0000 7f6d88ff9700 1 -- 172.21.15.52:0/4090050146 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f6d840cf070 con 0x7f6d8c108530 2024-06-10T02:55:13.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.225+0000 7f6d8b7fe700 1 --2- 172.21.15.52:0/4090050146 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f6d740a2d30 0x7f6d740a51f0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f6d7c0098e0 tx=0x7f6d7c005ab0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:55:13.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.225+0000 7f6d9c49fb80 1 Processor -- start 2024-06-10T02:55:13.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.226+0000 7f6d9c49fb80 1 -- start start 2024-06-10T02:55:13.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.226+0000 7f6d9c49fb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564267fc9f60 0x564267fca340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:13.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.226+0000 7f6d9c49fb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564267fca880 0x564267fc4260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:13.226 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.226+0000 7f6d9c49fb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564267fc47a0 0x564267fc4bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:13.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.226+0000 7f6d8affd700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564267fca880 0x564267fc4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:13.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.227+0000 7f6d8affd700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564267fca880 0x564267fc4260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.52:39324/0 (socket says 172.21.15.52:39324) 2024-06-10T02:55:13.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.227+0000 7f6d8affd700 1 -- 172.21.15.52:0/1202195905 learned_addr learned my addr 172.21.15.52:0/1202195905 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:55:13.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.227+0000 7f6d8bfff700 1 --2- 172.21.15.52:0/1202195905 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564267fc47a0 0x564267fc4bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:13.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.227+0000 7f6d8b7fe700 1 --2- 172.21.15.52:0/1202195905 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564267fc9f60 0x564267fca340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:13.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.227+0000 7f6d9c49fb80 1 -- 172.21.15.52:0/1202195905 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x564267eb6f60 con 0x564267fc47a0 2024-06-10T02:55:13.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.227+0000 7f6d9c49fb80 1 -- 172.21.15.52:0/1202195905 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x564267f15850 con 0x564267fc9f60 2024-06-10T02:55:13.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.227+0000 7f6d9c49fb80 1 -- 172.21.15.52:0/1202195905 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x564267e7d460 con 0x564267fca880 2024-06-10T02:55:13.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.227+0000 7f6d8affd700 1 -- 172.21.15.52:0/1202195905 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564267fc9f60 msgr2=0x564267fca340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:55:13.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.227+0000 7f6d8affd700 1 --2- 172.21.15.52:0/1202195905 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564267fc9f60 0x564267fca340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:13.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.227+0000 7f6d8affd700 1 -- 172.21.15.52:0/1202195905 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564267fc47a0 msgr2=0x564267fc4bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:55:13.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.227+0000 7f6d8affd700 1 --2- 172.21.15.52:0/1202195905 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564267fc47a0 0x564267fc4bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:13.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.227+0000 7f6d8affd700 1 -- 172.21.15.52:0/1202195905 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564267e648d0 con 0x564267fca880 2024-06-10T02:55:13.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.228+0000 7f6d8affd700 1 --2- 172.21.15.52:0/1202195905 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564267fca880 0x564267fc4260 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f6d80009cf0 tx=0x7f6d80011440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=25513e98dc84cb28 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:55:13.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.228+0000 7f6d71ffb700 1 -- 172.21.15.52:0/1202195905 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d80011d80 con 0x564267fca880 2024-06-10T02:55:13.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.228+0000 7f6d71ffb700 1 -- 172.21.15.52:0/1202195905 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d80004530 con 0x564267fca880 2024-06-10T02:55:13.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.228+0000 7f6d71ffb700 1 -- 172.21.15.52:0/1202195905 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d80005310 con 0x564267fca880 2024-06-10T02:55:13.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.229+0000 7f6d9c49fb80 1 -- 172.21.15.52:0/1202195905 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564267fca880 msgr2=0x564267fc4260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:55:13.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.229+0000 7f6d9c49fb80 1 --2- 172.21.15.52:0/1202195905 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564267fca880 0x564267fc4260 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f6d80009cf0 tx=0x7f6d80011440 comp rx=0 tx=0).stop 2024-06-10T02:55:13.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.229+0000 7f6d9c49fb80 1 -- 172.21.15.52:0/1202195905 shutdown_connections 2024-06-10T02:55:13.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.229+0000 7f6d9c49fb80 1 --2- 172.21.15.52:0/1202195905 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564267fc47a0 0x564267fc4bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:13.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.229+0000 7f6d9c49fb80 1 --2- 172.21.15.52:0/1202195905 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564267fc9f60 0x564267fca340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:13.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.230+0000 7f6d9c49fb80 1 --2- 172.21.15.52:0/1202195905 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564267fca880 0x564267fc4260 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:13.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.230+0000 7f6d9c49fb80 1 -- 172.21.15.52:0/1202195905 >> 172.21.15.52:0/1202195905 conn(0x564267ef1080 msgr2=0x564267fbf220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:55:13.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.230+0000 7f6d9c49fb80 1 -- 172.21.15.52:0/1202195905 shutdown_connections 2024-06-10T02:55:13.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.230+0000 7f6d9c49fb80 1 -- 172.21.15.52:0/1202195905 wait complete. 2024-06-10T02:55:13.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.230+0000 7f6d9c49fb80 1 -- start start 2024-06-10T02:55:13.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.231+0000 7f6d9c49fb80 1 Processor -- start 2024-06-10T02:55:13.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.231+0000 7f6d9c49fb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564267fc47a0 0x564267fd0650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:13.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.232+0000 7f6d9c49fb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564267fc9f60 0x564267fcd950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:13.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.232+0000 7f6d9c49fb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564267fca880 0x564267fcde90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:13.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.232+0000 7f6d8b7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564267fc47a0 0x564267fd0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:13.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.232+0000 7f6d9c49fb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x564267eb5fc0 con 0x564267fc47a0 2024-06-10T02:55:13.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.232+0000 7f6d8bfff700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564267fca880 0x564267fcde90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:13.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.232+0000 7f6d8bfff700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564267fca880 0x564267fcde90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.52:41134/0 (socket says 172.21.15.52:41134) 2024-06-10T02:55:13.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.232+0000 7f6d8bfff700 1 -- 172.21.15.52:0/1203879442 learned_addr learned my addr 172.21.15.52:0/1203879442 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:55:13.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.232+0000 7f6d8affd700 1 --2- 172.21.15.52:0/1203879442 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564267fc9f60 0x564267fcd950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:13.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.232+0000 7f6d9c49fb80 1 -- 172.21.15.52:0/1203879442 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x564267f11810 con 0x564267fca880 2024-06-10T02:55:13.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.232+0000 7f6d9c49fb80 1 -- 172.21.15.52:0/1203879442 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x564267fd1260 con 0x564267fc9f60 2024-06-10T02:55:13.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.232+0000 7f6d9c49fb80 10 client.0 _ll_register_callbacks cb 0x564267eab5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T02:55:13.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.233+0000 7f6d8affd700 1 -- 172.21.15.52:0/1203879442 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564267fca880 msgr2=0x564267fcde90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:55:13.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.233+0000 7f6d8affd700 1 --2- 172.21.15.52:0/1203879442 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564267fca880 0x564267fcde90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:13.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.233+0000 7f6d8affd700 1 -- 172.21.15.52:0/1203879442 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564267fc47a0 msgr2=0x564267fd0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:55:13.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.233+0000 7f6d8affd700 1 --2- 172.21.15.52:0/1203879442 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564267fc47a0 0x564267fd0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:13.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.233+0000 7f6d8affd700 1 -- 172.21.15.52:0/1203879442 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564267edb750 con 0x564267fc9f60 2024-06-10T02:55:13.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.233+0000 7f6d8affd700 1 --2- 172.21.15.52:0/1203879442 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564267fc9f60 0x564267fcd950 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f6d80009cf0 tx=0x7f6d80004280 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8b745e1b5bcda10a server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:55:13.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.233+0000 7f6d6affd700 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-06-10T02:55:13.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.233+0000 7f6d6affd700 1 -- 172.21.15.52:0/1203879442 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d80004960 con 0x564267fc9f60 2024-06-10T02:55:13.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.233+0000 7f6d9c49fb80 1 -- 172.21.15.52:0/1203879442 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d80004530 con 0x564267fc9f60 2024-06-10T02:55:13.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.233+0000 7f6d6affd700 1 -- 172.21.15.52:0/1203879442 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f6d80004b00 con 0x564267fc9f60 2024-06-10T02:55:13.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.233+0000 7f6d6affd700 1 -- 172.21.15.52:0/1203879442 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f6d80030050 con 0x564267fc9f60 2024-06-10T02:55:13.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.234+0000 7f6d9c49fb80 10 client.25204 fetch_fsmap learned FSMap version 30 2024-06-10T02:55:13.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.234+0000 7f6d9c49fb80 1 -- 172.21.15.52:0/1203879442 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x564267fdb830 con 0x564267fc9f60 2024-06-10T02:55:13.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.234+0000 7f6d6affd700 1 -- 172.21.15.52:0/1203879442 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d80004530 con 0x564267fc9f60 2024-06-10T02:55:13.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.234+0000 7f6d6affd700 1 -- 172.21.15.52:0/1203879442 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f6d80018070 con 0x564267fc9f60 2024-06-10T02:55:13.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.234+0000 7f6d9c49fb80 10 client.25204 fetch_fsmap finished waiting for FSMap version 30 2024-06-10T02:55:13.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.234+0000 7f6d9c49fb80 10 client.25204 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-06-10T02:55:13.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.234+0000 7f6d9c49fb80 20 client.25204 populate_metadata read hostname 'smithi052' 2024-06-10T02:55:13.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.234+0000 7f6d9c49fb80 1 --2- 172.21.15.52:0/1203879442 >> [v2:172.21.15.188:6834/3904544259,v1:172.21.15.188:6835/3904544259] conn(0x564267fde100 0x564267fe0520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:13.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.235+0000 7f6d9c49fb80 4 client.25204 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T02:55:13.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.235+0000 7f6d9c49fb80 1 -- 172.21.15.52:0/1203879442 --> [v2:172.21.15.188:6834/3904544259,v1:172.21.15.188:6835/3904544259] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564267f22d50 con 0x564267fde100 2024-06-10T02:55:13.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.235+0000 7f6d8b7fe700 1 --2- 172.21.15.52:0/1203879442 >> [v2:172.21.15.188:6834/3904544259,v1:172.21.15.188:6835/3904544259] conn(0x564267fde100 0x564267fe0520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:13.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.235+0000 7f6d8b7fe700 1 --2- 172.21.15.52:0/1203879442 >> [v2:172.21.15.188:6834/3904544259,v1:172.21.15.188:6835/3904544259] conn(0x564267fde100 0x564267fe0520 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=bf41c1666efc2c80 server_cookie=6c201f55b153865e in_seq=0 out_seq=0 2024-06-10T02:55:13.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.235+0000 7f6d6affd700 10 client.25204 ms_handle_connect on v2:172.21.15.188:6834/3904544259 2024-06-10T02:55:13.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.236+0000 7f6d6affd700 1 -- 172.21.15.52:0/1203879442 <== mds.2 v2:172.21.15.188:6834/3904544259 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x564267f22d50 con 0x564267fde100 2024-06-10T02:55:13.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.236+0000 7f6d6affd700 10 client.25204 handle_command_reply: tid=0 2024-06-10T02:55:13.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.236+0000 7f6d9c49fb80 1 client.25204 shutdown 2024-06-10T02:55:13.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.236+0000 7f6d9c49fb80 1 -- 172.21.15.52:0/1203879442 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564267fc9f60 msgr2=0x564267fcd950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:55:13.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.237+0000 7f6d9c49fb80 1 --2- 172.21.15.52:0/1203879442 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564267fc9f60 0x564267fcd950 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f6d80009cf0 tx=0x7f6d80004280 comp rx=0 tx=0).stop 2024-06-10T02:55:13.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.237+0000 7f6d9c49fb80 1 -- 172.21.15.52:0/1203879442 shutdown_connections 2024-06-10T02:55:13.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.237+0000 7f6d9c49fb80 1 --2- 172.21.15.52:0/1203879442 >> [v2:172.21.15.188:6834/3904544259,v1:172.21.15.188:6835/3904544259] conn(0x564267fde100 0x564267fe0520 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:13.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.237+0000 7f6d9c49fb80 1 --2- 172.21.15.52:0/1203879442 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564267fca880 0x564267fcde90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:13.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.237+0000 7f6d6affd700 0 client.25204 ms_handle_reset on v2:172.21.15.188:6834/3904544259 2024-06-10T02:55:13.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.237+0000 7f6d9c49fb80 1 --2- 172.21.15.52:0/1203879442 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564267fc47a0 0x564267fd0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:13.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.237+0000 7f6d9c49fb80 1 --2- 172.21.15.52:0/1203879442 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564267fc9f60 0x564267fcd950 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:13.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.237+0000 7f6d9c49fb80 1 -- 172.21.15.52:0/1203879442 >> 172.21.15.52:0/1203879442 conn(0x564267ef1080 msgr2=0x564267fbe700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:55:13.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.237+0000 7f6d9c49fb80 1 -- 172.21.15.52:0/1203879442 shutdown_connections 2024-06-10T02:55:13.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.237+0000 7f6d9c49fb80 1 -- 172.21.15.52:0/1203879442 wait complete. 2024-06-10T02:55:13.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.237+0000 7f6d9c49fb80 20 client.25204 trim_cache size 0 max 16384 2024-06-10T02:55:13.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.253+0000 7f6d9c49fb80 1 Processor -- start 2024-06-10T02:55:13.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.253+0000 7f6d9c49fb80 1 -- start start 2024-06-10T02:55:13.253 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.253+0000 7f6d9c49fb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564267fc47a0 0x564267fc4260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:13.253 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.253+0000 7f6d9c49fb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564267fc9f60 0x564267fce020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:13.253 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.253+0000 7f6d9c49fb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564267fca880 0x564267fbee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:13.253 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.253+0000 7f6d9c49fb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f6d8c1b06a0 con 0x564267fc47a0 2024-06-10T02:55:13.253 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.253+0000 7f6d9c49fb80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f6d8c1b21e0 con 0x564267fca880 2024-06-10T02:55:13.253 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.253+0000 7f6d9c49fb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f6d8c1b0d70 con 0x564267fc9f60 2024-06-10T02:55:13.254 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.254+0000 7f6d8b7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564267fc47a0 0x564267fc4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:13.254 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.254+0000 7f6d8b7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564267fc47a0 0x564267fc4260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49640/0 (socket says 172.21.15.52:49640) 2024-06-10T02:55:13.254 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.254+0000 7f6d8b7fe700 1 -- 172.21.15.52:0/2362948403 learned_addr learned my addr 172.21.15.52:0/2362948403 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:55:13.254 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.254+0000 7f6d8bfff700 1 --2- 172.21.15.52:0/2362948403 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564267fca880 0x564267fbee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:13.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.254+0000 7f6d8affd700 1 --2- 172.21.15.52:0/2362948403 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564267fc9f60 0x564267fce020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:13.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.254+0000 7f6d8b7fe700 1 -- 172.21.15.52:0/2362948403 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564267fc9f60 msgr2=0x564267fce020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:55:13.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.254+0000 7f6d8b7fe700 1 --2- 172.21.15.52:0/2362948403 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564267fc9f60 0x564267fce020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:13.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.254+0000 7f6d8b7fe700 1 -- 172.21.15.52:0/2362948403 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564267fca880 msgr2=0x564267fbee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:55:13.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.254+0000 7f6d8b7fe700 1 --2- 172.21.15.52:0/2362948403 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564267fca880 0x564267fbee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:13.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.254+0000 7f6d8b7fe700 1 -- 172.21.15.52:0/2362948403 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564267fce890 con 0x564267fc47a0 2024-06-10T02:55:13.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.254+0000 7f6d8bfff700 1 --2- 172.21.15.52:0/2362948403 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564267fca880 0x564267fbee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T02:55:13.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.254+0000 7f6d8b7fe700 1 --2- 172.21.15.52:0/2362948403 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564267fc47a0 0x564267fc4260 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f6d7c009990 tx=0x7f6d7c018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42736c7d45e29de4 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:55:13.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.254+0000 7f6d717fa700 1 -- 172.21.15.52:0/2362948403 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d7c005520 con 0x564267fc47a0 2024-06-10T02:55:13.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.254+0000 7f6d717fa700 1 -- 172.21.15.52:0/2362948403 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d7c0056c0 con 0x564267fc47a0 2024-06-10T02:55:13.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.254+0000 7f6d717fa700 1 -- 172.21.15.52:0/2362948403 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d7c031b30 con 0x564267fc47a0 2024-06-10T02:55:13.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.255+0000 7f6d9c49fb80 1 -- 172.21.15.52:0/2362948403 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564267fc47a0 msgr2=0x564267fc4260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:55:13.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.255+0000 7f6d9c49fb80 1 --2- 172.21.15.52:0/2362948403 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564267fc47a0 0x564267fc4260 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f6d7c009990 tx=0x7f6d7c018000 comp rx=0 tx=0).stop 2024-06-10T02:55:13.257 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.255+0000 7f6d9c49fb80 1 -- 172.21.15.52:0/2362948403 shutdown_connections 2024-06-10T02:55:13.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.255+0000 7f6d9c49fb80 1 --2- 172.21.15.52:0/2362948403 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564267fca880 0x564267fbee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:13.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.255+0000 7f6d9c49fb80 1 --2- 172.21.15.52:0/2362948403 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564267fc47a0 0x564267fc4260 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:13.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.255+0000 7f6d9c49fb80 1 --2- 172.21.15.52:0/2362948403 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564267fc9f60 0x564267fce020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:13.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.255+0000 7f6d9c49fb80 1 -- 172.21.15.52:0/2362948403 >> 172.21.15.52:0/2362948403 conn(0x564267ef1080 msgr2=0x564267fcd910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:55:13.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.255+0000 7f6d9c49fb80 1 -- 172.21.15.52:0/2362948403 shutdown_connections 2024-06-10T02:55:13.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.256+0000 7f6d9c49fb80 1 -- 172.21.15.52:0/2362948403 wait complete. 2024-06-10T02:55:13.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.256+0000 7f6d9c49fb80 1 -- start start 2024-06-10T02:55:13.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.256+0000 7f6d9c49fb80 1 Processor -- start 2024-06-10T02:55:13.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.256+0000 7f6d9c49fb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564267fc47a0 0x564267fe2b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:13.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.256+0000 7f6d9c49fb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564267fc9f60 0x564267fe3050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:13.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.257+0000 7f6d9c49fb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564267fca880 0x564267fe3590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:13.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.257+0000 7f6d9c49fb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f6d8c1b1b10 con 0x564267fc47a0 2024-06-10T02:55:13.259 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.257+0000 7f6d9c49fb80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f6d8c1b1440 con 0x564267fca880 2024-06-10T02:55:13.259 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.257+0000 7f6d9c49fb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f6d8c1ae640 con 0x564267fc9f60 2024-06-10T02:55:13.259 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.257+0000 7f6d9c49fb80 10 client.0 _ll_register_callbacks cb 0x564267eab5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T02:55:13.259 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.257+0000 7f6d8b7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564267fc47a0 0x564267fe2b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:13.259 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.257+0000 7f6d8b7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564267fc47a0 0x564267fe2b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49656/0 (socket says 172.21.15.52:49656) 2024-06-10T02:55:13.259 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.257+0000 7f6d8b7fe700 1 -- 172.21.15.52:0/1863679894 learned_addr learned my addr 172.21.15.52:0/1863679894 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:55:13.259 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.257+0000 7f6d8affd700 1 --2- 172.21.15.52:0/1863679894 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564267fc9f60 0x564267fe3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:13.259 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.257+0000 7f6d8bfff700 1 --2- 172.21.15.52:0/1863679894 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564267fca880 0x564267fe3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:13.259 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.257+0000 7f6d8b7fe700 1 -- 172.21.15.52:0/1863679894 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564267fc9f60 msgr2=0x564267fe3050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:55:13.259 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.257+0000 7f6d8b7fe700 1 --2- 172.21.15.52:0/1863679894 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564267fc9f60 0x564267fe3050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:13.259 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.257+0000 7f6d8b7fe700 1 -- 172.21.15.52:0/1863679894 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564267fca880 msgr2=0x564267fe3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:55:13.259 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.257+0000 7f6d8b7fe700 1 --2- 172.21.15.52:0/1863679894 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564267fca880 0x564267fe3590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:13.259 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.257+0000 7f6d8b7fe700 1 -- 172.21.15.52:0/1863679894 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564268046ca0 con 0x564267fc47a0 2024-06-10T02:55:13.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.257+0000 7f6d8bfff700 1 --2- 172.21.15.52:0/1863679894 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564267fca880 0x564267fe3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T02:55:13.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.257+0000 7f6d8b7fe700 1 --2- 172.21.15.52:0/1863679894 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564267fc47a0 0x564267fe2b10 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f6d7c000c00 tx=0x7f6d7c031430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b217b7ec1dfbb216 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:55:13.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.257+0000 7f6d6affd700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-10T02:55:13.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.258+0000 7f6d6affd700 1 -- 172.21.15.52:0/1863679894 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d7c0319b0 con 0x564267fc47a0 2024-06-10T02:55:13.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.258+0000 7f6d9c49fb80 1 -- 172.21.15.52:0/1863679894 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x564267f1d2d0 con 0x564267fc47a0 2024-06-10T02:55:13.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.258+0000 7f6d6affd700 1 -- 172.21.15.52:0/1863679894 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f6d7c031b50 con 0x564267fc47a0 2024-06-10T02:55:13.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.258+0000 7f6d6affd700 1 -- 172.21.15.52:0/1863679894 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f6d7c04a050 con 0x564267fc47a0 2024-06-10T02:55:13.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.258+0000 7f6d9c49fb80 10 client.15603 fetch_fsmap learned FSMap version 30 2024-06-10T02:55:13.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.258+0000 7f6d9c49fb80 1 -- 172.21.15.52:0/1863679894 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x564268047f20 con 0x564267fc47a0 2024-06-10T02:55:13.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.258+0000 7f6d6affd700 1 -- 172.21.15.52:0/1863679894 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x564267f1d2d0 con 0x564267fc47a0 2024-06-10T02:55:13.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.259+0000 7f6d6affd700 1 -- 172.21.15.52:0/1863679894 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f6d7c039040 con 0x564267fc47a0 2024-06-10T02:55:13.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.259+0000 7f6d9c49fb80 10 client.15603 fetch_fsmap finished waiting for FSMap version 30 2024-06-10T02:55:13.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.259+0000 7f6d9c49fb80 10 client.15603 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-06-10T02:55:13.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.259+0000 7f6d9c49fb80 20 client.15603 populate_metadata read hostname 'smithi052' 2024-06-10T02:55:13.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.260+0000 7f6d9c49fb80 1 --2- 172.21.15.52:0/1863679894 >> [v2:172.21.15.188:6834/3904544259,v1:172.21.15.188:6835/3904544259] conn(0x564267fde100 0x56426804a530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:13.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.260+0000 7f6d9c49fb80 4 client.15603 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T02:55:13.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.260+0000 7f6d9c49fb80 1 -- 172.21.15.52:0/1863679894 --> [v2:172.21.15.188:6834/3904544259,v1:172.21.15.188:6835/3904544259] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564267ed0f50 con 0x564267fde100 2024-06-10T02:55:13.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.260+0000 7f6d8affd700 1 --2- 172.21.15.52:0/1863679894 >> [v2:172.21.15.188:6834/3904544259,v1:172.21.15.188:6835/3904544259] conn(0x564267fde100 0x56426804a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:13.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.260+0000 7f6d8affd700 1 --2- 172.21.15.52:0/1863679894 >> [v2:172.21.15.188:6834/3904544259,v1:172.21.15.188:6835/3904544259] conn(0x564267fde100 0x56426804a530 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=252da9f11d2d2ff server_cookie=75e4f1063b85c08e in_seq=0 out_seq=0 2024-06-10T02:55:13.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.260+0000 7f6d6affd700 10 client.15603 ms_handle_connect on v2:172.21.15.188:6834/3904544259 2024-06-10T02:55:13.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.260+0000 7f6d6affd700 1 -- 172.21.15.52:0/1863679894 <== mds.2 v2:172.21.15.188:6834/3904544259 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564267ed0f50 con 0x564267fde100 2024-06-10T02:55:13.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.260+0000 7f6d6affd700 10 client.15603 handle_command_reply: tid=0 2024-06-10T02:55:13.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.260+0000 7f6d9c49fb80 1 client.15603 shutdown 2024-06-10T02:55:13.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.261+0000 7f6d9c49fb80 1 -- 172.21.15.52:0/1863679894 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564267fc47a0 msgr2=0x564267fe2b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:55:13.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.261+0000 7f6d9c49fb80 1 --2- 172.21.15.52:0/1863679894 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564267fc47a0 0x564267fe2b10 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f6d7c000c00 tx=0x7f6d7c031430 comp rx=0 tx=0).stop 2024-06-10T02:55:13.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.261+0000 7f6d9c49fb80 1 -- 172.21.15.52:0/1863679894 shutdown_connections 2024-06-10T02:55:13.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.261+0000 7f6d9c49fb80 1 --2- 172.21.15.52:0/1863679894 >> [v2:172.21.15.188:6834/3904544259,v1:172.21.15.188:6835/3904544259] conn(0x564267fde100 0x56426804a530 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:13.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.261+0000 7f6d9c49fb80 1 --2- 172.21.15.52:0/1863679894 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564267fca880 0x564267fe3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:13.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.261+0000 7f6d9c49fb80 1 --2- 172.21.15.52:0/1863679894 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564267fc47a0 0x564267fe2b10 secure :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f6d7c000c00 tx=0x7f6d7c031430 comp rx=0 tx=0).stop 2024-06-10T02:55:13.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.261+0000 7f6d9c49fb80 1 --2- 172.21.15.52:0/1863679894 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564267fc9f60 0x564267fe3050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:13.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.261+0000 7f6d9c49fb80 1 -- 172.21.15.52:0/1863679894 >> 172.21.15.52:0/1863679894 conn(0x564267ef1080 msgr2=0x564267fcad20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:55:13.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.261+0000 7f6d6affd700 0 client.15603 ms_handle_reset on v2:172.21.15.188:6834/3904544259 2024-06-10T02:55:13.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.261+0000 7f6d9c49fb80 1 -- 172.21.15.52:0/1863679894 shutdown_connections 2024-06-10T02:55:13.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.261+0000 7f6d9c49fb80 1 -- 172.21.15.52:0/1863679894 wait complete. 2024-06-10T02:55:13.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.261+0000 7f6d9c49fb80 20 client.15603 trim_cache size 0 max 16384 2024-06-10T02:55:13.264 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-06-10T02:55:13.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.262+0000 7f6d91c1c700 1 -- 172.21.15.52:0/4090050146 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f6d740a2d30 msgr2=0x7f6d740a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:55:13.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.262+0000 7f6d91c1c700 1 --2- 172.21.15.52:0/4090050146 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f6d740a2d30 0x7f6d740a51f0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f6d7c0098e0 tx=0x7f6d7c005ab0 comp rx=0 tx=0).stop 2024-06-10T02:55:13.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.262+0000 7f6d91c1c700 1 -- 172.21.15.52:0/4090050146 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6d8c108530 msgr2=0x7f6d8c101f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:55:13.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.262+0000 7f6d91c1c700 1 --2- 172.21.15.52:0/4090050146 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6d8c108530 0x7f6d8c101f10 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f6d8400ef30 tx=0x7f6d8400c3a0 comp rx=0 tx=0).stop 2024-06-10T02:55:13.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.262+0000 7f6d91c1c700 1 -- 172.21.15.52:0/4090050146 shutdown_connections 2024-06-10T02:55:13.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.262+0000 7f6d91c1c700 1 --2- 172.21.15.52:0/4090050146 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f6d740a2d30 0x7f6d740a51f0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:13.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.262+0000 7f6d91c1c700 1 --2- 172.21.15.52:0/4090050146 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6d8c108530 0x7f6d8c101f10 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:13.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.262+0000 7f6d91c1c700 1 --2- 172.21.15.52:0/4090050146 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6d8c06ab60 0x7f6d8c101490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:13.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.262+0000 7f6d91c1c700 1 --2- 172.21.15.52:0/4090050146 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6d8c06b520 0x7f6d8c1019d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:13.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.263+0000 7f6d91c1c700 1 -- 172.21.15.52:0/4090050146 >> 172.21.15.52:0/4090050146 conn(0x7f6d8c0783d0 msgr2=0x7f6d8c100150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:55:13.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.263+0000 7f6d91c1c700 1 -- 172.21.15.52:0/4090050146 shutdown_connections 2024-06-10T02:55:13.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:13.263+0000 7f6d91c1c700 1 -- 172.21.15.52:0/4090050146 wait complete. 2024-06-10T02:55:13.974 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-06-10T02:55:14.375 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:55:15.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:55:14 smithi120 ceph-mon[36634]: pgmap v465: 129 pgs: 129 active+clean; 661 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-10T02:55:15.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:55:14 smithi120 ceph-mon[36634]: mgrmap e20: x(active, since 12m), standbys: z, y 2024-06-10T02:55:15.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:55:14 smithi188 ceph-mon[36599]: pgmap v465: 129 pgs: 129 active+clean; 661 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-10T02:55:15.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:55:14 smithi188 ceph-mon[36599]: mgrmap e20: x(active, since 12m), standbys: z, y 2024-06-10T02:55:15.226 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:55:14 smithi052 ceph-mon[32362]: pgmap v465: 129 pgs: 129 active+clean; 661 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-10T02:55:15.226 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:55:14 smithi052 ceph-mon[32362]: mgrmap e20: x(active, since 12m), standbys: z, y 2024-06-10T02:55:15.748 INFO:tasks.quiescer.fs.[cephfs]:Total duration (60) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-06-10T02:55:15.748 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 30.0 2024-06-10T02:55:15.748 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T02:55:16.062 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:55:15 smithi052 ceph-mon[32362]: pgmap v466: 129 pgs: 129 active+clean; 661 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-10T02:55:16.094 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.093+0000 7f6392618700 1 -- 172.21.15.52:0/4098265116 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f638c101380 msgr2=0x7f638c101800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:55:16.094 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.093+0000 7f6392618700 1 --2- 172.21.15.52:0/4098265116 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f638c101380 0x7f638c101800 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f6380009990 tx=0x7f6380009ca0 comp rx=0 tx=0).stop 2024-06-10T02:55:16.094 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.093+0000 7f6392618700 1 -- 172.21.15.52:0/4098265116 shutdown_connections 2024-06-10T02:55:16.094 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.093+0000 7f6392618700 1 --2- 172.21.15.52:0/4098265116 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f638c101d40 0x7f638c10e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:16.094 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.093+0000 7f6392618700 1 --2- 172.21.15.52:0/4098265116 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f638c1073f0 0x7f638c1077d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:16.095 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.093+0000 7f6392618700 1 --2- 172.21.15.52:0/4098265116 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f638c101380 0x7f638c101800 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:16.095 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.093+0000 7f6392618700 1 -- 172.21.15.52:0/4098265116 >> 172.21.15.52:0/4098265116 conn(0x7f638c078300 msgr2=0x7f638c078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:55:16.095 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.093+0000 7f6392618700 1 -- 172.21.15.52:0/4098265116 shutdown_connections 2024-06-10T02:55:16.095 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.093+0000 7f6392618700 1 -- 172.21.15.52:0/4098265116 wait complete. 2024-06-10T02:55:16.095 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.095+0000 7f6392618700 1 Processor -- start 2024-06-10T02:55:16.095 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.096+0000 7f6392618700 1 -- start start 2024-06-10T02:55:16.096 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.096+0000 7f6392618700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f638c101380 0x7f638c19e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:16.096 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.096+0000 7f6392618700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f638c101d40 0x7f638c19ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:16.096 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.096+0000 7f6392618700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f638c1073f0 0x7f638c198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:16.096 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.096+0000 7f6392618700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f638c1127e0 con 0x7f638c101380 2024-06-10T02:55:16.097 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.096+0000 7f6392618700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f638c112660 con 0x7f638c1073f0 2024-06-10T02:55:16.097 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.096+0000 7f6392618700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f638c112960 con 0x7f638c101d40 2024-06-10T02:55:16.097 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.096+0000 7f638bfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f638c101380 0x7f638c19e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:16.097 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.096+0000 7f638bfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f638c101380 0x7f638c19e940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49668/0 (socket says 172.21.15.52:49668) 2024-06-10T02:55:16.097 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.096+0000 7f638bfff700 1 -- 172.21.15.52:0/1846389273 learned_addr learned my addr 172.21.15.52:0/1846389273 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:55:16.097 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.096+0000 7f638b7fe700 1 --2- 172.21.15.52:0/1846389273 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f638c101d40 0x7f638c19ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:16.097 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.097+0000 7f6390bb5700 1 --2- 172.21.15.52:0/1846389273 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f638c1073f0 0x7f638c198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:16.097 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.097+0000 7f6390bb5700 1 -- 172.21.15.52:0/1846389273 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f638c101d40 msgr2=0x7f638c19ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:55:16.097 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.097+0000 7f6390bb5700 1 --2- 172.21.15.52:0/1846389273 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f638c101d40 0x7f638c19ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:16.097 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.097+0000 7f6390bb5700 1 -- 172.21.15.52:0/1846389273 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f638c101380 msgr2=0x7f638c19e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:55:16.097 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.097+0000 7f6390bb5700 1 --2- 172.21.15.52:0/1846389273 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f638c101380 0x7f638c19e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:16.097 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.097+0000 7f6390bb5700 1 -- 172.21.15.52:0/1846389273 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f638c199260 con 0x7f638c1073f0 2024-06-10T02:55:16.097 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.097+0000 7f6390bb5700 1 --2- 172.21.15.52:0/1846389273 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f638c1073f0 0x7f638c198b00 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f638400b330 tx=0x7f638400b640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:55:16.098 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.098+0000 7f63897fa700 1 -- 172.21.15.52:0/1846389273 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f638400bdd0 con 0x7f638c1073f0 2024-06-10T02:55:16.098 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.098+0000 7f6392618700 1 -- 172.21.15.52:0/1846389273 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f638c1994f0 con 0x7f638c1073f0 2024-06-10T02:55:16.098 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.098+0000 7f63897fa700 1 -- 172.21.15.52:0/1846389273 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6384004bb0 con 0x7f638c1073f0 2024-06-10T02:55:16.098 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.098+0000 7f63897fa700 1 -- 172.21.15.52:0/1846389273 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6384019410 con 0x7f638c1073f0 2024-06-10T02:55:16.098 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.098+0000 7f6392618700 1 -- 172.21.15.52:0/1846389273 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f638c10ebf0 con 0x7f638c1073f0 2024-06-10T02:55:16.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.100+0000 7f63897fa700 1 -- 172.21.15.52:0/1846389273 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f6384020020 con 0x7f638c1073f0 2024-06-10T02:55:16.100 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.101+0000 7f63897fa700 1 --2- 172.21.15.52:0/1846389273 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f63740a2c20 0x7f63740a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:16.100 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.101+0000 7f63897fa700 1 -- 172.21.15.52:0/1846389273 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f63840d2af0 con 0x7f638c1073f0 2024-06-10T02:55:16.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.102+0000 7f638bfff700 1 --2- 172.21.15.52:0/1846389273 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f63740a2c20 0x7f63740a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:16.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.102+0000 7f638bfff700 1 --2- 172.21.15.52:0/1846389273 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f63740a2c20 0x7f63740a50e0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f637c007c10 tx=0x7f637c007480 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:55:16.103 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.103+0000 7f639ce9bb80 1 Processor -- start 2024-06-10T02:55:16.103 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.103+0000 7f639ce9bb80 1 -- start start 2024-06-10T02:55:16.103 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.104+0000 7f639ce9bb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d62be2a640 0x55d62be2aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:16.104 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.104+0000 7f639ce9bb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d62be2af60 0x55d62be249e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:16.104 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.104+0000 7f639ce9bb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d62be24f20 0x55d62be25340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:16.104 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.104+0000 7f639ce9bb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55d62bd76810 con 0x55d62be2af60 2024-06-10T02:55:16.104 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.104+0000 7f639ce9bb80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55d62bd1e720 con 0x55d62be2a640 2024-06-10T02:55:16.104 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.104+0000 7f639ce9bb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55d62bd727d0 con 0x55d62be24f20 2024-06-10T02:55:16.104 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.104+0000 7f638bfff700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d62be2a640 0x55d62be2aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:16.104 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.104+0000 7f638bfff700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d62be2a640 0x55d62be2aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.52:41174/0 (socket says 172.21.15.52:41174) 2024-06-10T02:55:16.104 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.104+0000 7f638bfff700 1 -- 172.21.15.52:0/897794306 learned_addr learned my addr 172.21.15.52:0/897794306 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:55:16.105 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.105+0000 7f638bfff700 1 -- 172.21.15.52:0/897794306 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d62be24f20 msgr2=0x55d62be25340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:55:16.105 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.105+0000 7f638b7fe700 1 --2- 172.21.15.52:0/897794306 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d62be2af60 0x55d62be249e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:16.105 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.105+0000 7f6390bb5700 1 --2- 172.21.15.52:0/897794306 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d62be24f20 0x55d62be25340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:16.105 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.105+0000 7f638bfff700 1 --2- 172.21.15.52:0/897794306 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d62be24f20 0x55d62be25340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:16.105 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.105+0000 7f638bfff700 1 -- 172.21.15.52:0/897794306 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d62be2af60 msgr2=0x55d62be249e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:55:16.105 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.105+0000 7f638bfff700 1 --2- 172.21.15.52:0/897794306 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d62be2af60 0x55d62be249e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:16.105 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.105+0000 7f638bfff700 1 -- 172.21.15.52:0/897794306 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d62bd14f50 con 0x55d62be2a640 2024-06-10T02:55:16.105 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.105+0000 7f6390bb5700 1 --2- 172.21.15.52:0/897794306 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d62be24f20 0x55d62be25340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T02:55:16.105 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.105+0000 7f638b7fe700 1 --2- 172.21.15.52:0/897794306 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d62be2af60 0x55d62be249e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T02:55:16.105 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.105+0000 7f638bfff700 1 --2- 172.21.15.52:0/897794306 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d62be2a640 0x55d62be2aa20 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f637c010e60 tx=0x7f637c013000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5232b11fb34bac84 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:55:16.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.106+0000 7f63727fc700 1 -- 172.21.15.52:0/897794306 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f637c01e620 con 0x55d62be2a640 2024-06-10T02:55:16.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.106+0000 7f63727fc700 1 -- 172.21.15.52:0/897794306 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f637c010e90 con 0x55d62be2a640 2024-06-10T02:55:16.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.107+0000 7f63727fc700 1 -- 172.21.15.52:0/897794306 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f637c01ccf0 con 0x55d62be2a640 2024-06-10T02:55:16.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.107+0000 7f639ce9bb80 1 -- 172.21.15.52:0/897794306 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d62be2a640 msgr2=0x55d62be2aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:55:16.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.107+0000 7f639ce9bb80 1 --2- 172.21.15.52:0/897794306 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d62be2a640 0x55d62be2aa20 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f637c010e60 tx=0x7f637c013000 comp rx=0 tx=0).stop 2024-06-10T02:55:16.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.107+0000 7f639ce9bb80 1 -- 172.21.15.52:0/897794306 shutdown_connections 2024-06-10T02:55:16.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.107+0000 7f639ce9bb80 1 --2- 172.21.15.52:0/897794306 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d62be24f20 0x55d62be25340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:16.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.107+0000 7f639ce9bb80 1 --2- 172.21.15.52:0/897794306 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d62be2a640 0x55d62be2aa20 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:16.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.107+0000 7f639ce9bb80 1 --2- 172.21.15.52:0/897794306 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d62be2af60 0x55d62be249e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:16.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.108+0000 7f639ce9bb80 1 -- 172.21.15.52:0/897794306 >> 172.21.15.52:0/897794306 conn(0x55d62bd4fcc0 msgr2=0x55d62be1f850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:55:16.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.108+0000 7f639ce9bb80 1 -- 172.21.15.52:0/897794306 shutdown_connections 2024-06-10T02:55:16.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.108+0000 7f639ce9bb80 1 -- 172.21.15.52:0/897794306 wait complete. 2024-06-10T02:55:16.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.109+0000 7f639ce9bb80 1 -- start start 2024-06-10T02:55:16.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.109+0000 7f639ce9bb80 1 Processor -- start 2024-06-10T02:55:16.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.110+0000 7f639ce9bb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d62be24f20 0x55d62be30bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:16.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.110+0000 7f639ce9bb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d62be2a640 0x55d62be2dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:16.110 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.110+0000 7f638bfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d62be24f20 0x55d62be30bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:16.110 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.110+0000 7f638bfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d62be24f20 0x55d62be30bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49684/0 (socket says 172.21.15.52:49684) 2024-06-10T02:55:16.110 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.110+0000 7f639ce9bb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d62be2af60 0x55d62be2e400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:16.110 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.110+0000 7f639ce9bb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55d62be31a70 con 0x55d62be24f20 2024-06-10T02:55:16.110 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.110+0000 7f639ce9bb80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55d62bd16bf0 con 0x55d62be2a640 2024-06-10T02:55:16.110 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.110+0000 7f639ce9bb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55d62be31bf0 con 0x55d62be2af60 2024-06-10T02:55:16.110 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.110+0000 7f638bfff700 1 -- 172.21.15.52:0/3960654200 learned_addr learned my addr 172.21.15.52:0/3960654200 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:55:16.110 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.110+0000 7f639ce9bb80 10 client.0 _ll_register_callbacks cb 0x55d62bd10920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T02:55:16.110 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.110+0000 7f638b7fe700 1 --2- 172.21.15.52:0/3960654200 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d62be2a640 0x55d62be2dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:16.111 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.111+0000 7f638b7fe700 1 -- 172.21.15.52:0/3960654200 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d62be2af60 msgr2=0x55d62be2e400 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-10T02:55:16.111 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.111+0000 7f638b7fe700 1 --2- 172.21.15.52:0/3960654200 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d62be2af60 0x55d62be2e400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:16.111 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.111+0000 7f638b7fe700 1 -- 172.21.15.52:0/3960654200 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d62be24f20 msgr2=0x55d62be30bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:55:16.111 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.111+0000 7f638b7fe700 1 --2- 172.21.15.52:0/3960654200 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d62be24f20 0x55d62be30bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:16.111 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.111+0000 7f638b7fe700 1 -- 172.21.15.52:0/3960654200 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d62bcc58d0 con 0x55d62be2a640 2024-06-10T02:55:16.111 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.111+0000 7f638b7fe700 1 --2- 172.21.15.52:0/3960654200 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d62be2a640 0x55d62be2dec0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f638000b2e0 tx=0x7f638000bfd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5b59f7c971cf210c server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:55:16.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.111+0000 7f636b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-06-10T02:55:16.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.111+0000 7f636b7fe700 1 -- 172.21.15.52:0/3960654200 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f638000f840 con 0x55d62be2a640 2024-06-10T02:55:16.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.111+0000 7f636b7fe700 1 -- 172.21.15.52:0/3960654200 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f638000f9e0 con 0x55d62be2a640 2024-06-10T02:55:16.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.111+0000 7f639ce9bb80 1 -- 172.21.15.52:0/3960654200 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f637c010e90 con 0x55d62be2a640 2024-06-10T02:55:16.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.112+0000 7f636b7fe700 1 -- 172.21.15.52:0/3960654200 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f638002f050 con 0x55d62be2a640 2024-06-10T02:55:16.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.112+0000 7f639ce9bb80 10 client.25253 fetch_fsmap learned FSMap version 30 2024-06-10T02:55:16.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.112+0000 7f639ce9bb80 1 -- 172.21.15.52:0/3960654200 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d62be44a70 con 0x55d62be2a640 2024-06-10T02:55:16.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.112+0000 7f636b7fe700 1 -- 172.21.15.52:0/3960654200 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f637c010e90 con 0x55d62be2a640 2024-06-10T02:55:16.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.112+0000 7f636b7fe700 1 -- 172.21.15.52:0/3960654200 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f638002a030 con 0x55d62be2a640 2024-06-10T02:55:16.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.112+0000 7f639ce9bb80 10 client.25253 fetch_fsmap finished waiting for FSMap version 30 2024-06-10T02:55:16.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.113+0000 7f639ce9bb80 10 client.25253 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-06-10T02:55:16.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.113+0000 7f639ce9bb80 20 client.25253 populate_metadata read hostname 'smithi052' 2024-06-10T02:55:16.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.113+0000 7f639ce9bb80 1 --2- 172.21.15.52:0/3960654200 >> [v2:172.21.15.188:6836/819317726,v1:172.21.15.188:6837/819317726] conn(0x55d62be470b0 0x55d62be494c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:16.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.113+0000 7f639ce9bb80 4 client.25253 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T02:55:16.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.113+0000 7f639ce9bb80 1 -- 172.21.15.52:0/3960654200 --> [v2:172.21.15.188:6836/819317726,v1:172.21.15.188:6837/819317726] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d62bd83d10 con 0x55d62be470b0 2024-06-10T02:55:16.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.113+0000 7f638bfff700 1 --2- 172.21.15.52:0/3960654200 >> [v2:172.21.15.188:6836/819317726,v1:172.21.15.188:6837/819317726] conn(0x55d62be470b0 0x55d62be494c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:16.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.114+0000 7f636b7fe700 10 client.25253 ms_handle_connect on v2:172.21.15.188:6836/819317726 2024-06-10T02:55:16.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.114+0000 7f638bfff700 1 --2- 172.21.15.52:0/3960654200 >> [v2:172.21.15.188:6836/819317726,v1:172.21.15.188:6837/819317726] conn(0x55d62be470b0 0x55d62be494c0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=dcce41705f1eef00 server_cookie=a768b76dfa2508b5 in_seq=0 out_seq=0 2024-06-10T02:55:16.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.114+0000 7f636b7fe700 1 -- 172.21.15.52:0/3960654200 <== mds.1 v2:172.21.15.188:6836/819317726 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55d62bd83d10 con 0x55d62be470b0 2024-06-10T02:55:16.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.114+0000 7f636b7fe700 10 client.25253 handle_command_reply: tid=0 2024-06-10T02:55:16.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.114+0000 7f639ce9bb80 1 client.25253 shutdown 2024-06-10T02:55:16.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.116+0000 7f639ce9bb80 1 -- 172.21.15.52:0/3960654200 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d62be2a640 msgr2=0x55d62be2dec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:55:16.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.116+0000 7f639ce9bb80 1 --2- 172.21.15.52:0/3960654200 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d62be2a640 0x55d62be2dec0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f638000b2e0 tx=0x7f638000bfd0 comp rx=0 tx=0).stop 2024-06-10T02:55:16.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.116+0000 7f639ce9bb80 1 -- 172.21.15.52:0/3960654200 shutdown_connections 2024-06-10T02:55:16.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.116+0000 7f639ce9bb80 1 --2- 172.21.15.52:0/3960654200 >> [v2:172.21.15.188:6836/819317726,v1:172.21.15.188:6837/819317726] conn(0x55d62be470b0 0x55d62be494c0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:16.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.116+0000 7f639ce9bb80 1 --2- 172.21.15.52:0/3960654200 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d62be2af60 0x55d62be2e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:16.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.116+0000 7f639ce9bb80 1 --2- 172.21.15.52:0/3960654200 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d62be24f20 0x55d62be30bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:16.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.116+0000 7f636b7fe700 0 client.25253 ms_handle_reset on v2:172.21.15.188:6836/819317726 2024-06-10T02:55:16.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.117+0000 7f639ce9bb80 1 --2- 172.21.15.52:0/3960654200 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d62be2a640 0x55d62be2dec0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:16.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.117+0000 7f639ce9bb80 1 -- 172.21.15.52:0/3960654200 >> 172.21.15.52:0/3960654200 conn(0x55d62bd4fcc0 msgr2=0x55d62be239d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:55:16.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.117+0000 7f639ce9bb80 1 -- 172.21.15.52:0/3960654200 shutdown_connections 2024-06-10T02:55:16.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.117+0000 7f639ce9bb80 1 -- 172.21.15.52:0/3960654200 wait complete. 2024-06-10T02:55:16.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.117+0000 7f639ce9bb80 20 client.25253 trim_cache size 0 max 16384 2024-06-10T02:55:16.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.141+0000 7f639ce9bb80 1 Processor -- start 2024-06-10T02:55:16.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.141+0000 7f639ce9bb80 1 -- start start 2024-06-10T02:55:16.141 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.141+0000 7f639ce9bb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d62be24f20 0x55d62be249e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:16.141 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.142+0000 7f639ce9bb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d62be2a640 0x55d62be2bb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:16.141 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.142+0000 7f639ce9bb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d62be2af60 0x55d62be4ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:16.141 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.142+0000 7f639ce9bb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f638c1ac220 con 0x55d62be24f20 2024-06-10T02:55:16.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.142+0000 7f639ce9bb80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f638c1add60 con 0x55d62be2a640 2024-06-10T02:55:16.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.142+0000 7f639ce9bb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f638c1ac8f0 con 0x55d62be2af60 2024-06-10T02:55:16.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.142+0000 7f638bfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d62be24f20 0x55d62be249e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:16.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.142+0000 7f638b7fe700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d62be2a640 0x55d62be2bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:16.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.142+0000 7f638bfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d62be24f20 0x55d62be249e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49692/0 (socket says 172.21.15.52:49692) 2024-06-10T02:55:16.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.142+0000 7f638bfff700 1 -- 172.21.15.52:0/3350142653 learned_addr learned my addr 172.21.15.52:0/3350142653 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:55:16.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.142+0000 7f6390bb5700 1 --2- 172.21.15.52:0/3350142653 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d62be2af60 0x55d62be4ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:16.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.142+0000 7f6390bb5700 1 -- 172.21.15.52:0/3350142653 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d62be2a640 msgr2=0x55d62be2bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:55:16.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.142+0000 7f6390bb5700 1 --2- 172.21.15.52:0/3350142653 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d62be2a640 0x55d62be2bb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:16.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.142+0000 7f6390bb5700 1 -- 172.21.15.52:0/3350142653 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d62be24f20 msgr2=0x55d62be249e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:55:16.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.142+0000 7f6390bb5700 1 --2- 172.21.15.52:0/3350142653 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d62be24f20 0x55d62be249e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:16.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.142+0000 7f6390bb5700 1 -- 172.21.15.52:0/3350142653 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d62be20520 con 0x55d62be2af60 2024-06-10T02:55:16.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.143+0000 7f6390bb5700 1 --2- 172.21.15.52:0/3350142653 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d62be2af60 0x55d62be4ed90 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f63840d01b0 tx=0x7f638409eb80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d80cbb8677a7767b server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:55:16.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.143+0000 7f6371ffb700 1 -- 172.21.15.52:0/3350142653 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f638409f640 con 0x55d62be2af60 2024-06-10T02:55:16.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.143+0000 7f6371ffb700 1 -- 172.21.15.52:0/3350142653 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f638409ef40 con 0x55d62be2af60 2024-06-10T02:55:16.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.143+0000 7f6371ffb700 1 -- 172.21.15.52:0/3350142653 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63840a7980 con 0x55d62be2af60 2024-06-10T02:55:16.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.144+0000 7f639ce9bb80 1 -- 172.21.15.52:0/3350142653 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d62be2af60 msgr2=0x55d62be4ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:55:16.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.144+0000 7f639ce9bb80 1 --2- 172.21.15.52:0/3350142653 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d62be2af60 0x55d62be4ed90 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f63840d01b0 tx=0x7f638409eb80 comp rx=0 tx=0).stop 2024-06-10T02:55:16.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.144+0000 7f639ce9bb80 1 -- 172.21.15.52:0/3350142653 shutdown_connections 2024-06-10T02:55:16.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.144+0000 7f639ce9bb80 1 --2- 172.21.15.52:0/3350142653 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d62be2af60 0x55d62be4ed90 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:16.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.144+0000 7f639ce9bb80 1 --2- 172.21.15.52:0/3350142653 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d62be24f20 0x55d62be249e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:16.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.144+0000 7f639ce9bb80 1 --2- 172.21.15.52:0/3350142653 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d62be2a640 0x55d62be2bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:16.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.144+0000 7f639ce9bb80 1 -- 172.21.15.52:0/3350142653 >> 172.21.15.52:0/3350142653 conn(0x55d62bd4fcc0 msgr2=0x55d62be2b400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:55:16.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.144+0000 7f639ce9bb80 1 -- 172.21.15.52:0/3350142653 shutdown_connections 2024-06-10T02:55:16.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.144+0000 7f639ce9bb80 1 -- 172.21.15.52:0/3350142653 wait complete. 2024-06-10T02:55:16.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.145+0000 7f639ce9bb80 1 -- start start 2024-06-10T02:55:16.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.145+0000 7f639ce9bb80 1 Processor -- start 2024-06-10T02:55:16.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.146+0000 7f639ce9bb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d62be24f20 0x55d62be37780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:16.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.146+0000 7f639ce9bb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d62be2a640 0x55d62be4e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:16.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.146+0000 7f639ce9bb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d62be2af60 0x55d62be4edc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:16.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.146+0000 7f638bfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d62be24f20 0x55d62be37780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:16.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.146+0000 7f638bfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d62be24f20 0x55d62be37780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49706/0 (socket says 172.21.15.52:49706) 2024-06-10T02:55:16.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.146+0000 7f638b7fe700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d62be2a640 0x55d62be4e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:16.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.146+0000 7f638bfff700 1 -- 172.21.15.52:0/3596059779 learned_addr learned my addr 172.21.15.52:0/3596059779 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:55:16.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.146+0000 7f639ce9bb80 1 -- 172.21.15.52:0/3596059779 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f638c1ad690 con 0x55d62be24f20 2024-06-10T02:55:16.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.146+0000 7f639ce9bb80 1 -- 172.21.15.52:0/3596059779 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f638c1acfc0 con 0x55d62be2a640 2024-06-10T02:55:16.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.146+0000 7f6390bb5700 1 --2- 172.21.15.52:0/3596059779 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d62be2af60 0x55d62be4edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:16.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.146+0000 7f639ce9bb80 1 -- 172.21.15.52:0/3596059779 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f638c1aa1c0 con 0x55d62be2af60 2024-06-10T02:55:16.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.147+0000 7f639ce9bb80 10 client.0 _ll_register_callbacks cb 0x55d62bd10920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T02:55:16.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.147+0000 7f6390bb5700 1 -- 172.21.15.52:0/3596059779 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d62be2a640 msgr2=0x55d62be4e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:55:16.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.147+0000 7f6390bb5700 1 --2- 172.21.15.52:0/3596059779 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d62be2a640 0x55d62be4e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:16.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.147+0000 7f6390bb5700 1 -- 172.21.15.52:0/3596059779 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d62be24f20 msgr2=0x55d62be37780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:55:16.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.147+0000 7f6390bb5700 1 --2- 172.21.15.52:0/3596059779 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d62be24f20 0x55d62be37780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:16.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.147+0000 7f6390bb5700 1 -- 172.21.15.52:0/3596059779 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d62be4f600 con 0x55d62be2af60 2024-06-10T02:55:16.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.147+0000 7f638b7fe700 1 --2- 172.21.15.52:0/3596059779 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d62be2a640 0x55d62be4e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T02:55:16.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.147+0000 7f6390bb5700 1 --2- 172.21.15.52:0/3596059779 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d62be2af60 0x55d62be4edc0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6384009260 tx=0x7f638400ba00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=42aea18ea327e9b0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:55:16.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.147+0000 7f636b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-06-10T02:55:16.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.147+0000 7f636b7fe700 1 -- 172.21.15.52:0/3596059779 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63840b7070 con 0x55d62be2af60 2024-06-10T02:55:16.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.147+0000 7f636b7fe700 1 -- 172.21.15.52:0/3596059779 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f63840bdd80 con 0x55d62be2af60 2024-06-10T02:55:16.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.148+0000 7f639ce9bb80 1 -- 172.21.15.52:0/3596059779 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f638409ef40 con 0x55d62be2af60 2024-06-10T02:55:16.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.148+0000 7f636b7fe700 1 -- 172.21.15.52:0/3596059779 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f638409f640 con 0x55d62be2af60 2024-06-10T02:55:16.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.148+0000 7f639ce9bb80 10 client.25222 fetch_fsmap learned FSMap version 30 2024-06-10T02:55:16.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.148+0000 7f639ce9bb80 1 -- 172.21.15.52:0/3596059779 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d62beb1e40 con 0x55d62be2af60 2024-06-10T02:55:16.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.149+0000 7f636b7fe700 1 -- 172.21.15.52:0/3596059779 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63840a32f0 con 0x55d62be2af60 2024-06-10T02:55:16.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.149+0000 7f636b7fe700 1 -- 172.21.15.52:0/3596059779 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f6384099ab0 con 0x55d62be2af60 2024-06-10T02:55:16.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.149+0000 7f639ce9bb80 10 client.25222 fetch_fsmap finished waiting for FSMap version 30 2024-06-10T02:55:16.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.149+0000 7f639ce9bb80 10 client.25222 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-06-10T02:55:16.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.149+0000 7f639ce9bb80 20 client.25222 populate_metadata read hostname 'smithi052' 2024-06-10T02:55:16.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.149+0000 7f639ce9bb80 1 --2- 172.21.15.52:0/3596059779 >> [v2:172.21.15.188:6836/819317726,v1:172.21.15.188:6837/819317726] conn(0x55d62be470b0 0x55d62beb4270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:16.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.149+0000 7f639ce9bb80 4 client.25222 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T02:55:16.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.149+0000 7f639ce9bb80 1 -- 172.21.15.52:0/3596059779 --> [v2:172.21.15.188:6836/819317726,v1:172.21.15.188:6837/819317726] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f638409ef40 con 0x55d62be470b0 2024-06-10T02:55:16.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.149+0000 7f638bfff700 1 --2- 172.21.15.52:0/3596059779 >> [v2:172.21.15.188:6836/819317726,v1:172.21.15.188:6837/819317726] conn(0x55d62be470b0 0x55d62beb4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:16.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.150+0000 7f636b7fe700 10 client.25222 ms_handle_connect on v2:172.21.15.188:6836/819317726 2024-06-10T02:55:16.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.150+0000 7f638bfff700 1 --2- 172.21.15.52:0/3596059779 >> [v2:172.21.15.188:6836/819317726,v1:172.21.15.188:6837/819317726] conn(0x55d62be470b0 0x55d62beb4270 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d986494d98574884 server_cookie=a400dafa584fefae in_seq=0 out_seq=0 2024-06-10T02:55:16.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.150+0000 7f636b7fe700 1 -- 172.21.15.52:0/3596059779 <== mds.1 v2:172.21.15.188:6836/819317726 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f638409ef40 con 0x55d62be470b0 2024-06-10T02:55:16.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.150+0000 7f636b7fe700 10 client.25222 handle_command_reply: tid=0 2024-06-10T02:55:16.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.150+0000 7f639ce9bb80 1 client.25222 shutdown 2024-06-10T02:55:16.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.151+0000 7f639ce9bb80 1 -- 172.21.15.52:0/3596059779 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d62be2af60 msgr2=0x55d62be4edc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:55:16.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.151+0000 7f639ce9bb80 1 --2- 172.21.15.52:0/3596059779 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d62be2af60 0x55d62be4edc0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6384009260 tx=0x7f638400ba00 comp rx=0 tx=0).stop 2024-06-10T02:55:16.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.151+0000 7f639ce9bb80 1 -- 172.21.15.52:0/3596059779 shutdown_connections 2024-06-10T02:55:16.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.151+0000 7f639ce9bb80 1 --2- 172.21.15.52:0/3596059779 >> [v2:172.21.15.188:6836/819317726,v1:172.21.15.188:6837/819317726] conn(0x55d62be470b0 0x55d62beb4270 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:16.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.151+0000 7f639ce9bb80 1 --2- 172.21.15.52:0/3596059779 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d62be2af60 0x55d62be4edc0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:16.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.151+0000 7f636b7fe700 0 client.25222 ms_handle_reset on v2:172.21.15.188:6836/819317726 2024-06-10T02:55:16.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.151+0000 7f639ce9bb80 1 --2- 172.21.15.52:0/3596059779 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d62be24f20 0x55d62be37780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:16.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.151+0000 7f639ce9bb80 1 --2- 172.21.15.52:0/3596059779 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d62be2a640 0x55d62be4e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:16.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.151+0000 7f639ce9bb80 1 -- 172.21.15.52:0/3596059779 >> 172.21.15.52:0/3596059779 conn(0x55d62bd4fcc0 msgr2=0x55d62be18e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:55:16.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.151+0000 7f639ce9bb80 1 -- 172.21.15.52:0/3596059779 shutdown_connections 2024-06-10T02:55:16.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.152+0000 7f639ce9bb80 1 -- 172.21.15.52:0/3596059779 wait complete. 2024-06-10T02:55:16.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.152+0000 7f639ce9bb80 20 client.25222 trim_cache size 0 max 16384 2024-06-10T02:55:16.153 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-06-10T02:55:16.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.153+0000 7f6392618700 1 -- 172.21.15.52:0/1846389273 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f63740a2c20 msgr2=0x7f63740a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:55:16.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.153+0000 7f6392618700 1 --2- 172.21.15.52:0/1846389273 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f63740a2c20 0x7f63740a50e0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f637c007c10 tx=0x7f637c007480 comp rx=0 tx=0).stop 2024-06-10T02:55:16.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.153+0000 7f6392618700 1 -- 172.21.15.52:0/1846389273 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f638c1073f0 msgr2=0x7f638c198b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:55:16.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.153+0000 7f6392618700 1 --2- 172.21.15.52:0/1846389273 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f638c1073f0 0x7f638c198b00 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f638400b330 tx=0x7f638400b640 comp rx=0 tx=0).stop 2024-06-10T02:55:16.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.153+0000 7f6392618700 1 -- 172.21.15.52:0/1846389273 shutdown_connections 2024-06-10T02:55:16.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.153+0000 7f6392618700 1 --2- 172.21.15.52:0/1846389273 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f638c1073f0 0x7f638c198b00 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:16.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.153+0000 7f6392618700 1 --2- 172.21.15.52:0/1846389273 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f63740a2c20 0x7f63740a50e0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:16.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.153+0000 7f6392618700 1 --2- 172.21.15.52:0/1846389273 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f638c101380 0x7f638c19e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:16.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.153+0000 7f6392618700 1 --2- 172.21.15.52:0/1846389273 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f638c101d40 0x7f638c19ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:16.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.153+0000 7f6392618700 1 -- 172.21.15.52:0/1846389273 >> 172.21.15.52:0/1846389273 conn(0x7f638c078300 msgr2=0x7f638c0fffa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:55:16.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.153+0000 7f6392618700 1 -- 172.21.15.52:0/1846389273 shutdown_connections 2024-06-10T02:55:16.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:16.153+0000 7f6392618700 1 -- 172.21.15.52:0/1846389273 wait complete. 2024-06-10T02:55:16.166 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:55:16.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:55:15 smithi188 ceph-mon[36599]: pgmap v466: 129 pgs: 129 active+clean; 661 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-10T02:55:16.387 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:55:15 smithi120 ceph-mon[36634]: pgmap v466: 129 pgs: 129 active+clean; 661 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-10T02:55:16.993 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-06-10T02:55:18.226 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:55:18 smithi052 ceph-mon[32362]: pgmap v467: 129 pgs: 129 active+clean; 662 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 341 B/s wr, 12 op/s 2024-06-10T02:55:18.388 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:55:18 smithi120 ceph-mon[36634]: pgmap v467: 129 pgs: 129 active+clean; 662 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 341 B/s wr, 12 op/s 2024-06-10T02:55:18.453 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:55:18 smithi188 ceph-mon[36599]: pgmap v467: 129 pgs: 129 active+clean; 662 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 341 B/s wr, 12 op/s 2024-06-10T02:55:18.566 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:18.566+0000 7fa0b8c1c700 1 -- 172.21.15.52:0/3664742682 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa0b41084d0 msgr2=0x7fa0b41088b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:55:18.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:18.566+0000 7fa0b8c1c700 1 --2- 172.21.15.52:0/3664742682 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa0b41084d0 0x7fa0b41088b0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fa0a4009990 tx=0x7fa0a4009ca0 comp rx=0 tx=0).stop 2024-06-10T02:55:18.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:18.567+0000 7fa0b8c1c700 1 -- 172.21.15.52:0/3664742682 shutdown_connections 2024-06-10T02:55:18.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:18.567+0000 7fa0b8c1c700 1 --2- 172.21.15.52:0/3664742682 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa0b406b3a0 0x7fa0b410f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:18.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:18.567+0000 7fa0b8c1c700 1 --2- 172.21.15.52:0/3664742682 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa0b41084d0 0x7fa0b41088b0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:18.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:18.567+0000 7fa0b8c1c700 1 --2- 172.21.15.52:0/3664742682 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa0b406aa00 0x7fa0b406ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:18.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:18.567+0000 7fa0b8c1c700 1 -- 172.21.15.52:0/3664742682 >> 172.21.15.52:0/3664742682 conn(0x7fa0b4078330 msgr2=0x7fa0b4078740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:55:18.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:18.567+0000 7fa0b8c1c700 1 -- 172.21.15.52:0/3664742682 shutdown_connections 2024-06-10T02:55:18.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:18.567+0000 7fa0b8c1c700 1 -- 172.21.15.52:0/3664742682 wait complete. 2024-06-10T02:55:18.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:18.568+0000 7fa0b8c1c700 1 Processor -- start 2024-06-10T02:55:18.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:18.568+0000 7fa0b8c1c700 1 -- start start 2024-06-10T02:55:18.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:18.568+0000 7fa0b8c1c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa0b406aa00 0x7fa0b410e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:18.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:18.568+0000 7fa0b8c1c700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa0b406b3a0 0x7fa0b410ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:18.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:18.568+0000 7fa0b259c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa0b406aa00 0x7fa0b410e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:18.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:18.568+0000 7fa0b259c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa0b406aa00 0x7fa0b410e740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49736/0 (socket says 172.21.15.52:49736) 2024-06-10T02:55:18.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:18.568+0000 7fa0b259c700 1 -- 172.21.15.52:0/3662024286 learned_addr learned my addr 172.21.15.52:0/3662024286 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:55:18.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:18.568+0000 7fa0b1d9b700 1 --2- 172.21.15.52:0/3662024286 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa0b406b3a0 0x7fa0b410ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:18.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:18.568+0000 7fa0b8c1c700 1 --2- 172.21.15.52:0/3662024286 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa0b41084d0 0x7fa0b41a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:18.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:18.568+0000 7fa0b8c1c700 1 -- 172.21.15.52:0/3662024286 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fa0b4112730 con 0x7fa0b406aa00 2024-06-10T02:55:18.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:18.568+0000 7fa0b8c1c700 1 -- 172.21.15.52:0/3662024286 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa0b41125b0 con 0x7fa0b41084d0 2024-06-10T02:55:18.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:18.568+0000 7fa0b8c1c700 1 -- 172.21.15.52:0/3662024286 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa0b41128b0 con 0x7fa0b406b3a0 2024-06-10T02:55:18.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:18.569+0000 7fa0b2d9d700 1 --2- 172.21.15.52:0/3662024286 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa0b41084d0 0x7fa0b41a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:18.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:18.569+0000 7fa0b259c700 1 -- 172.21.15.52:0/3662024286 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa0b406b3a0 msgr2=0x7fa0b410ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:55:18.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:18.569+0000 7fa0b259c700 1 --2- 172.21.15.52:0/3662024286 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa0b406b3a0 0x7fa0b410ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:18.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:18.569+0000 7fa0b259c700 1 -- 172.21.15.52:0/3662024286 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa0b41084d0 msgr2=0x7fa0b41a3b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:55:18.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:18.569+0000 7fa0b259c700 1 --2- 172.21.15.52:0/3662024286 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa0b41084d0 0x7fa0b41a3b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:18.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:18.569+0000 7fa0b259c700 1 -- 172.21.15.52:0/3662024286 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0b41a4240 con 0x7fa0b406aa00 2024-06-10T02:55:18.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:18.569+0000 7fa0b259c700 1 --2- 172.21.15.52:0/3662024286 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa0b406aa00 0x7fa0b410e740 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fa0a400bae0 tx=0x7fa0a400bb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:55:18.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:18.569+0000 7fa0b2d9d700 1 --2- 172.21.15.52:0/3662024286 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa0b41084d0 0x7fa0b41a3b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T02:55:18.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:18.569+0000 7fa0a37fe700 1 -- 172.21.15.52:0/3662024286 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0a4026070 con 0x7fa0b406aa00 2024-06-10T02:55:18.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:18.569+0000 7fa0a37fe700 1 -- 172.21.15.52:0/3662024286 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa0a400f460 con 0x7fa0b406aa00 2024-06-10T02:55:18.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:18.569+0000 7fa0a37fe700 1 -- 172.21.15.52:0/3662024286 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0a4021670 con 0x7fa0b406aa00 2024-06-10T02:55:18.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:18.569+0000 7fa0b8c1c700 1 -- 172.21.15.52:0/3662024286 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0b41a44d0 con 0x7fa0b406aa00 2024-06-10T02:55:18.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:18.570+0000 7fa0b8c1c700 1 -- 172.21.15.52:0/3662024286 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0b41abde0 con 0x7fa0b406aa00 2024-06-10T02:55:18.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:18.571+0000 7fa0b8c1c700 1 -- 172.21.15.52:0/3662024286 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0b4103b80 con 0x7fa0b406aa00 2024-06-10T02:55:18.572 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:18.573+0000 7fa0a37fe700 1 -- 172.21.15.52:0/3662024286 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fa0a4029070 con 0x7fa0b406aa00 2024-06-10T02:55:18.573 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:18.574+0000 7fa0a37fe700 1 --2- 172.21.15.52:0/3662024286 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fa09c0a2d70 0x7fa09c0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:18.573 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:18.574+0000 7fa0a37fe700 1 -- 172.21.15.52:0/3662024286 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7fa0a40d0150 con 0x7fa0b406aa00 2024-06-10T02:55:18.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:18.575+0000 7fa0b1d9b700 1 --2- 172.21.15.52:0/3662024286 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fa09c0a2d70 0x7fa09c0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:18.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:18.576+0000 7fa0b1d9b700 1 --2- 172.21.15.52:0/3662024286 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fa09c0a2d70 0x7fa09c0a5230 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fa0b4078830 tx=0x7fa0a8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:55:18.580 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:18.581+0000 7fa0a37fe700 1 -- 172.21.15.52:0/3662024286 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fa0a405d850 con 0x7fa0b406aa00 2024-06-10T02:55:18.587 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:55:19.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:19.078+0000 7fa0b8c1c700 1 -- 172.21.15.52:0/3662024286 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa0b41a05f0 con 0x7fa0b406aa00 2024-06-10T02:55:19.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:19.079+0000 7fa0a37fe700 1 -- 172.21.15.52:0/3662024286 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11938 (secure 0 0 0) 0x7fa0b41a05f0 con 0x7fa0b406aa00 2024-06-10T02:55:19.078 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-10T02:55:19.079 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/889673247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":889673247},{"type":"v1","addr":"172.21.15.52:6835","nonce":889673247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/3615663511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":3615663511},{"type":"v1","addr":"172.21.15.52:6837","nonce":3615663511}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T02:50:03.889766+0000","modified":"2024-06-10T02:54:07.086405+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24484,"mds_1":24469,"mds_2":24463,"mds_3":24449,"mds_4":24428},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14571":{"gid":14571,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6839/3059599042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":3059599042},{"type":"v1","addr":"172.21.15.52:6839","nonce":3059599042}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14583":{"gid":14583,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6841/3039275521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":3039275521},{"type":"v1","addr":"172.21.15.52:6841","nonce":3039275521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24383":{"gid":24383,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6833/2609002896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":2609002896},{"type":"v1","addr":"172.21.15.120:6833","nonce":2609002896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6835/1996370288","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":1996370288},{"type":"v1","addr":"172.21.15.120:6835","nonce":1996370288}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6833/529564979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":529564979},{"type":"v1","addr":"172.21.15.188:6833","nonce":529564979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":25,"addr":"172.21.15.120:6837/4132440952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":4132440952},{"type":"v1","addr":"172.21.15.120:6837","nonce":4132440952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"b","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.120:6839/1672436487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1672436487},{"type":"v1","addr":"172.21.15.120:6839","nonce":1672436487}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":2,"incarnation":22,"state":"up:active","state_seq":26,"addr":"172.21.15.188:6835/3904544259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":3904544259},{"type":"v1","addr":"172.21.15.188:6835","nonce":3904544259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.188:6837/819317726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":819317726},{"type":"v1","addr":"172.21.15.188:6837","nonce":819317726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.188:6839/310444836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":310444836},{"type":"v1","addr":"172.21.15.188:6839","nonce":310444836}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24484,"qdb_cluster":[24469,24484,24463,24428,24449]},"id":1}]} 2024-06-10T02:55:19.081 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:19.082+0000 7fa0b8c1c700 1 -- 172.21.15.52:0/3662024286 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fa09c0a2d70 msgr2=0x7fa09c0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:55:19.081 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:19.082+0000 7fa0b8c1c700 1 --2- 172.21.15.52:0/3662024286 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fa09c0a2d70 0x7fa09c0a5230 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fa0b4078830 tx=0x7fa0a8009260 comp rx=0 tx=0).stop 2024-06-10T02:55:19.082 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:19.082+0000 7fa0b8c1c700 1 -- 172.21.15.52:0/3662024286 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa0b406aa00 msgr2=0x7fa0b410e740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:55:19.082 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:19.082+0000 7fa0b8c1c700 1 --2- 172.21.15.52:0/3662024286 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa0b406aa00 0x7fa0b410e740 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fa0a400bae0 tx=0x7fa0a400bb10 comp rx=0 tx=0).stop 2024-06-10T02:55:19.082 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:19.082+0000 7fa0b8c1c700 1 -- 172.21.15.52:0/3662024286 shutdown_connections 2024-06-10T02:55:19.082 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:19.082+0000 7fa0b8c1c700 1 --2- 172.21.15.52:0/3662024286 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa0b41084d0 0x7fa0b41a3b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:19.082 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:19.082+0000 7fa0b8c1c700 1 --2- 172.21.15.52:0/3662024286 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fa09c0a2d70 0x7fa09c0a5230 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:19.082 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:19.082+0000 7fa0b8c1c700 1 --2- 172.21.15.52:0/3662024286 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa0b406aa00 0x7fa0b410e740 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:19.082 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:19.082+0000 7fa0b8c1c700 1 --2- 172.21.15.52:0/3662024286 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa0b406b3a0 0x7fa0b410ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:19.082 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:19.082+0000 7fa0b8c1c700 1 -- 172.21.15.52:0/3662024286 >> 172.21.15.52:0/3662024286 conn(0x7fa0b4078330 msgr2=0x7fa0b4104070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:55:19.082 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:19.082+0000 7fa0b8c1c700 1 -- 172.21.15.52:0/3662024286 shutdown_connections 2024-06-10T02:55:19.082 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:19.082+0000 7fa0b8c1c700 1 -- 172.21.15.52:0/3662024286 wait complete. 2024-06-10T02:55:19.083 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 30 2024-06-10T02:55:19.131 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:55:19 smithi052 ceph-mon[32362]: from='client.? 172.21.15.52:0/3662024286' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T02:55:19.388 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:55:19 smithi120 ceph-mon[36634]: from='client.? 172.21.15.52:0/3662024286' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T02:55:19.453 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:55:19 smithi188 ceph-mon[36599]: from='client.? 172.21.15.52:0/3662024286' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T02:55:20.388 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:55:20 smithi120 ceph-mon[36634]: pgmap v468: 129 pgs: 129 active+clean; 662 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 9 op/s 2024-06-10T02:55:20.453 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:55:20 smithi188 ceph-mon[36599]: pgmap v468: 129 pgs: 129 active+clean; 662 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 9 op/s 2024-06-10T02:55:20.472 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:55:20 smithi052 ceph-mon[32362]: pgmap v468: 129 pgs: 129 active+clean; 662 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 9 op/s 2024-06-10T02:55:20.508 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24484 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0' 2024-06-10T02:55:20.508 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24484 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0 2024-06-10T02:55:21.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.123+0000 7f58b7fe7700 1 -- 172.21.15.52:0/2035305967 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f58b01073f0 msgr2=0x7f58b01077d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:55:21.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.123+0000 7f58b7fe7700 1 --2- 172.21.15.52:0/2035305967 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f58b01073f0 0x7f58b01077d0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f58ac009990 tx=0x7f58ac009ca0 comp rx=0 tx=0).stop 2024-06-10T02:55:21.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.123+0000 7f58b7fe7700 1 -- 172.21.15.52:0/2035305967 shutdown_connections 2024-06-10T02:55:21.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.123+0000 7f58b7fe7700 1 --2- 172.21.15.52:0/2035305967 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f58b0101d40 0x7f58b010e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:21.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.123+0000 7f58b7fe7700 1 --2- 172.21.15.52:0/2035305967 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f58b01073f0 0x7f58b01077d0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:21.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.123+0000 7f58b7fe7700 1 --2- 172.21.15.52:0/2035305967 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f58b0101380 0x7f58b0101800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:21.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.123+0000 7f58b7fe7700 1 -- 172.21.15.52:0/2035305967 >> 172.21.15.52:0/2035305967 conn(0x7f58b0078300 msgr2=0x7f58b0078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:55:21.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.123+0000 7f58b7fe7700 1 -- 172.21.15.52:0/2035305967 shutdown_connections 2024-06-10T02:55:21.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.123+0000 7f58b7fe7700 1 -- 172.21.15.52:0/2035305967 wait complete. 2024-06-10T02:55:21.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.124+0000 7f58b7fe7700 1 Processor -- start 2024-06-10T02:55:21.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.124+0000 7f58b7fe7700 1 -- start start 2024-06-10T02:55:21.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.124+0000 7f58b7fe7700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f58b0101380 0x7f58b010a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:21.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.124+0000 7f58b7fe7700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f58b0101d40 0x7f58b010a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:21.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.124+0000 7f58b7fe7700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f58b01073f0 0x7f58b010aee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:21.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.124+0000 7f58b7fe7700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f58b0110550 con 0x7f58b01073f0 2024-06-10T02:55:21.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.124+0000 7f58b7fe7700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f58b01103d0 con 0x7f58b0101d40 2024-06-10T02:55:21.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.124+0000 7f58b7fe7700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f58b01106d0 con 0x7f58b0101380 2024-06-10T02:55:21.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.124+0000 7f58b6584700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f58b01073f0 0x7f58b010aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:21.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.124+0000 7f58b6584700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f58b01073f0 0x7f58b010aee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48376/0 (socket says 172.21.15.52:48376) 2024-06-10T02:55:21.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.124+0000 7f58b6584700 1 -- 172.21.15.52:0/2967895761 learned_addr learned my addr 172.21.15.52:0/2967895761 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:55:21.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.125+0000 7f58b5d83700 1 --2- 172.21.15.52:0/2967895761 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f58b0101380 0x7f58b010a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:21.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.125+0000 7f58b6584700 1 -- 172.21.15.52:0/2967895761 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f58b0101380 msgr2=0x7f58b010a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:55:21.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.125+0000 7f58b6584700 1 --2- 172.21.15.52:0/2967895761 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f58b0101380 0x7f58b010a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:21.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.125+0000 7f58b6584700 1 -- 172.21.15.52:0/2967895761 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f58b0101d40 msgr2=0x7f58b010a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:55:21.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.125+0000 7f58b5582700 1 --2- 172.21.15.52:0/2967895761 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f58b0101d40 0x7f58b010a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:21.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.125+0000 7f58b6584700 1 --2- 172.21.15.52:0/2967895761 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f58b0101d40 0x7f58b010a9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:21.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.125+0000 7f58b6584700 1 -- 172.21.15.52:0/2967895761 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58b01ab490 con 0x7f58b01073f0 2024-06-10T02:55:21.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.125+0000 7f58b6584700 1 --2- 172.21.15.52:0/2967895761 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f58b01073f0 0x7f58b010aee0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f58a8007f10 tx=0x7f58a800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:55:21.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.125+0000 7f58a6ffd700 1 -- 172.21.15.52:0/2967895761 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58a8004bb0 con 0x7f58b01073f0 2024-06-10T02:55:21.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.126+0000 7f58a6ffd700 1 -- 172.21.15.52:0/2967895761 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f58a8004d50 con 0x7f58b01073f0 2024-06-10T02:55:21.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.126+0000 7f58a6ffd700 1 -- 172.21.15.52:0/2967895761 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58a8005a60 con 0x7f58b01073f0 2024-06-10T02:55:21.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.126+0000 7f58b7fe7700 1 -- 172.21.15.52:0/2967895761 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58b01ab780 con 0x7f58b01073f0 2024-06-10T02:55:21.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.126+0000 7f58b7fe7700 1 -- 172.21.15.52:0/2967895761 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58b01abbb0 con 0x7f58b01073f0 2024-06-10T02:55:21.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.129+0000 7f58a6ffd700 1 -- 172.21.15.52:0/2967895761 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f58a801a020 con 0x7f58b01073f0 2024-06-10T02:55:21.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.130+0000 7f58c286ab80 1 Processor -- start 2024-06-10T02:55:21.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.130+0000 7f58c286ab80 1 -- start start 2024-06-10T02:55:21.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.130+0000 7f58c286ab80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55650514af60 0x55650514b340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:21.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.130+0000 7f58c286ab80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55650514b880 0x556505145260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:21.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.131+0000 7f58c286ab80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5565051457a0 0x556505145bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:21.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.131+0000 7f58c286ab80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x556505037f60 con 0x5565051457a0 2024-06-10T02:55:21.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.131+0000 7f58c286ab80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x556505096850 con 0x55650514af60 2024-06-10T02:55:21.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.131+0000 7f58c286ab80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x556504ffe460 con 0x55650514b880 2024-06-10T02:55:21.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.131+0000 7f58a6ffd700 1 --2- 172.21.15.52:0/2967895761 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f589c0a2d20 0x7f589c0a51e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:21.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.131+0000 7f58b5582700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5565051457a0 0x556505145bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:21.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.131+0000 7f58b5582700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5565051457a0 0x556505145bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48388/0 (socket says 172.21.15.52:48388) 2024-06-10T02:55:21.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.131+0000 7f58b5582700 1 -- 172.21.15.52:0/647163391 learned_addr learned my addr 172.21.15.52:0/647163391 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:55:21.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.131+0000 7f58b5582700 1 --2- 172.21.15.52:0/647163391 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55650514af60 0x55650514b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:21.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.131+0000 7f58a6ffd700 1 -- 172.21.15.52:0/2967895761 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f58a80ceb40 con 0x7f58b01073f0 2024-06-10T02:55:21.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.131+0000 7f58b5d83700 1 --2- 172.21.15.52:0/647163391 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55650514b880 0x556505145260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:21.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.132+0000 7f58b6584700 1 --2- 172.21.15.52:0/2967895761 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f589c0a2d20 0x7f589c0a51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:21.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.132+0000 7f58b5582700 1 -- 172.21.15.52:0/647163391 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55650514b880 msgr2=0x556505145260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:55:21.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.132+0000 7f58b5582700 1 --2- 172.21.15.52:0/647163391 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55650514b880 0x556505145260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:21.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.132+0000 7f58b5582700 1 -- 172.21.15.52:0/647163391 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55650514af60 msgr2=0x55650514b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:55:21.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.132+0000 7f58b5582700 1 --2- 172.21.15.52:0/647163391 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55650514af60 0x55650514b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:21.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.132+0000 7f58b5582700 1 -- 172.21.15.52:0/647163391 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556504fe58d0 con 0x5565051457a0 2024-06-10T02:55:21.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.133+0000 7f58b5582700 1 --2- 172.21.15.52:0/647163391 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5565051457a0 0x556505145bc0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f58a0015900 tx=0x7f58a00093f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c688aca353f236c server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:55:21.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.133+0000 7f589bfff700 1 -- 172.21.15.52:0/647163391 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58a0004830 con 0x5565051457a0 2024-06-10T02:55:21.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.133+0000 7f589bfff700 1 -- 172.21.15.52:0/647163391 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f58a0005020 con 0x5565051457a0 2024-06-10T02:55:21.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.133+0000 7f589bfff700 1 -- 172.21.15.52:0/647163391 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58a00106e0 con 0x5565051457a0 2024-06-10T02:55:21.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.134+0000 7f58c286ab80 1 -- 172.21.15.52:0/647163391 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5565051457a0 msgr2=0x556505145bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:55:21.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.134+0000 7f58b6584700 1 --2- 172.21.15.52:0/2967895761 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f589c0a2d20 0x7f589c0a51e0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f58a8097960 tx=0x7f58a800df40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:55:21.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.134+0000 7f58c286ab80 1 --2- 172.21.15.52:0/647163391 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5565051457a0 0x556505145bc0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f58a0015900 tx=0x7f58a00093f0 comp rx=0 tx=0).stop 2024-06-10T02:55:21.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.135+0000 7f58c286ab80 1 -- 172.21.15.52:0/647163391 shutdown_connections 2024-06-10T02:55:21.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.135+0000 7f58c286ab80 1 --2- 172.21.15.52:0/647163391 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5565051457a0 0x556505145bc0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:21.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.135+0000 7f58c286ab80 1 --2- 172.21.15.52:0/647163391 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55650514af60 0x55650514b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:21.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.135+0000 7f58c286ab80 1 --2- 172.21.15.52:0/647163391 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55650514b880 0x556505145260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:21.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.135+0000 7f58c286ab80 1 -- 172.21.15.52:0/647163391 >> 172.21.15.52:0/647163391 conn(0x556505072080 msgr2=0x556505140220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:55:21.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.135+0000 7f58c286ab80 1 -- 172.21.15.52:0/647163391 shutdown_connections 2024-06-10T02:55:21.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.135+0000 7f58c286ab80 1 -- 172.21.15.52:0/647163391 wait complete. 2024-06-10T02:55:21.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.135+0000 7f58c286ab80 1 -- start start 2024-06-10T02:55:21.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.136+0000 7f58c286ab80 1 Processor -- start 2024-06-10T02:55:21.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.136+0000 7f58c286ab80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5565051457a0 0x556505151670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:21.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.136+0000 7f58c286ab80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55650514af60 0x55650514e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:21.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.136+0000 7f58c286ab80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55650514b880 0x55650514eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:21.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.136+0000 7f58c286ab80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x556505036fc0 con 0x55650514af60 2024-06-10T02:55:21.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.136+0000 7f58c286ab80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x556505092810 con 0x5565051457a0 2024-06-10T02:55:21.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.136+0000 7f58c286ab80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x556505152260 con 0x55650514b880 2024-06-10T02:55:21.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.136+0000 7f58b5d83700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55650514af60 0x55650514e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:21.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.136+0000 7f58b5d83700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55650514af60 0x55650514e970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48404/0 (socket says 172.21.15.52:48404) 2024-06-10T02:55:21.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.136+0000 7f58b5d83700 1 -- 172.21.15.52:0/2713498479 learned_addr learned my addr 172.21.15.52:0/2713498479 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:55:21.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.137+0000 7f58c286ab80 10 client.0 _ll_register_callbacks cb 0x55650502c5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T02:55:21.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.137+0000 7f58b5582700 1 --2- 172.21.15.52:0/2713498479 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5565051457a0 0x556505151670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:21.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.137+0000 7f58b5582700 1 --2- 172.21.15.52:0/2713498479 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55650514b880 0x55650514eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:21.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.137+0000 7f58b5d83700 1 -- 172.21.15.52:0/2713498479 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55650514b880 msgr2=0x55650514eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:55:21.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.137+0000 7f58b5d83700 1 --2- 172.21.15.52:0/2713498479 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55650514b880 0x55650514eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:21.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.137+0000 7f58b5d83700 1 -- 172.21.15.52:0/2713498479 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5565051457a0 msgr2=0x556505151670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:55:21.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.137+0000 7f58b5d83700 1 --2- 172.21.15.52:0/2713498479 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5565051457a0 0x556505151670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:21.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.137+0000 7f58b5d83700 1 -- 172.21.15.52:0/2713498479 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55650505c750 con 0x55650514af60 2024-06-10T02:55:21.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.137+0000 7f58b5582700 1 --2- 172.21.15.52:0/2713498479 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5565051457a0 0x556505151670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T02:55:21.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.137+0000 7f58b5d83700 1 --2- 172.21.15.52:0/2713498479 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55650514af60 0x55650514e970 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f58ac009990 tx=0x7f58ac0038b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f97f2d96331b0310 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:55:21.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.137+0000 7f58997fa700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-10T02:55:21.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.138+0000 7f58997fa700 1 -- 172.21.15.52:0/2713498479 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58ac027060 con 0x55650514af60 2024-06-10T02:55:21.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.138+0000 7f58997fa700 1 -- 172.21.15.52:0/2713498479 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f58ac0054d0 con 0x55650514af60 2024-06-10T02:55:21.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.138+0000 7f58c286ab80 1 -- 172.21.15.52:0/2713498479 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f58a0005020 con 0x55650514af60 2024-06-10T02:55:21.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.138+0000 7f58997fa700 1 -- 172.21.15.52:0/2713498479 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f58ac02e050 con 0x55650514af60 2024-06-10T02:55:21.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.138+0000 7f58c286ab80 10 client.15636 fetch_fsmap learned FSMap version 30 2024-06-10T02:55:21.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.138+0000 7f58c286ab80 1 -- 172.21.15.52:0/2713498479 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55650515ca60 con 0x55650514af60 2024-06-10T02:55:21.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.139+0000 7f58997fa700 1 -- 172.21.15.52:0/2713498479 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f58a0005020 con 0x55650514af60 2024-06-10T02:55:21.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.139+0000 7f58997fa700 1 -- 172.21.15.52:0/2713498479 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f58ac00f4f0 con 0x55650514af60 2024-06-10T02:55:21.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.139+0000 7f58c286ab80 10 client.15636 fetch_fsmap finished waiting for FSMap version 30 2024-06-10T02:55:21.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.139+0000 7f58c286ab80 10 client.15636 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T02:55:21.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.139+0000 7f58c286ab80 20 client.15636 populate_metadata read hostname 'smithi052' 2024-06-10T02:55:21.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.139+0000 7f58c286ab80 1 --2- 172.21.15.52:0/2713498479 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55650515f0a0 0x556505161490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:21.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.139+0000 7f58c286ab80 4 client.15636 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T02:55:21.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.139+0000 7f58c286ab80 1 -- 172.21.15.52:0/2713498479 --> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5565050a3d50 con 0x55650515f0a0 2024-06-10T02:55:21.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.139+0000 7f58b5582700 1 --2- 172.21.15.52:0/2713498479 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55650515f0a0 0x556505161490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:21.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.140+0000 7f58997fa700 10 client.15636 ms_handle_connect on v2:172.21.15.188:6838/310444836 2024-06-10T02:55:21.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.140+0000 7f58b5582700 1 --2- 172.21.15.52:0/2713498479 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55650515f0a0 0x556505161490 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e7e8b9652042700 server_cookie=f692e3f4711fd13c in_seq=0 out_seq=0 2024-06-10T02:55:21.141 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.141+0000 7f58997fa700 1 -- 172.21.15.52:0/2713498479 <== mds.0 v2:172.21.15.188:6838/310444836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5565050a3d50 con 0x55650515f0a0 2024-06-10T02:55:21.141 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.141+0000 7f58997fa700 10 client.15636 handle_command_reply: tid=0 2024-06-10T02:55:21.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.141+0000 7f58c286ab80 1 client.15636 shutdown 2024-06-10T02:55:21.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.141+0000 7f58c286ab80 1 -- 172.21.15.52:0/2713498479 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55650514af60 msgr2=0x55650514e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:55:21.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.141+0000 7f58c286ab80 1 --2- 172.21.15.52:0/2713498479 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55650514af60 0x55650514e970 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f58ac009990 tx=0x7f58ac0038b0 comp rx=0 tx=0).stop 2024-06-10T02:55:21.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.141+0000 7f58c286ab80 1 -- 172.21.15.52:0/2713498479 shutdown_connections 2024-06-10T02:55:21.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.141+0000 7f58c286ab80 1 --2- 172.21.15.52:0/2713498479 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55650515f0a0 0x556505161490 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:21.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.141+0000 7f58c286ab80 1 --2- 172.21.15.52:0/2713498479 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55650514b880 0x55650514eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:21.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.141+0000 7f58997fa700 0 client.15636 ms_handle_reset on v2:172.21.15.188:6838/310444836 2024-06-10T02:55:21.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.141+0000 7f58c286ab80 1 --2- 172.21.15.52:0/2713498479 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5565051457a0 0x556505151670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:21.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.141+0000 7f58c286ab80 1 --2- 172.21.15.52:0/2713498479 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55650514af60 0x55650514e970 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:21.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.141+0000 7f58c286ab80 1 -- 172.21.15.52:0/2713498479 >> 172.21.15.52:0/2713498479 conn(0x556505072080 msgr2=0x55650513f700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:55:21.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.142+0000 7f58c286ab80 1 -- 172.21.15.52:0/2713498479 shutdown_connections 2024-06-10T02:55:21.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.142+0000 7f58c286ab80 1 -- 172.21.15.52:0/2713498479 wait complete. 2024-06-10T02:55:21.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.142+0000 7f58c286ab80 20 client.15636 trim_cache size 0 max 16384 2024-06-10T02:55:21.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.172+0000 7f58c286ab80 1 Processor -- start 2024-06-10T02:55:21.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.172+0000 7f58c286ab80 1 -- start start 2024-06-10T02:55:21.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.173+0000 7f58c286ab80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5565051457a0 0x556505145260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:21.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.173+0000 7f58c286ab80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55650514af60 0x55650514f040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:21.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.173+0000 7f58c286ab80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55650514b880 0x55650513fe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:21.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.173+0000 7f58b5582700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5565051457a0 0x556505145260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:21.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.173+0000 7f58b5582700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5565051457a0 0x556505145260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.52:50422/0 (socket says 172.21.15.52:50422) 2024-06-10T02:55:21.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.174+0000 7f58b5d83700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55650514af60 0x55650514f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:21.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.174+0000 7f58b5d83700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55650514af60 0x55650514f040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.52:57946/0 (socket says 172.21.15.52:57946) 2024-06-10T02:55:21.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.174+0000 7f58b5582700 1 -- 172.21.15.52:0/2606689803 learned_addr learned my addr 172.21.15.52:0/2606689803 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:55:21.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.174+0000 7f58c286ab80 1 -- 172.21.15.52:0/2606689803 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f58b01b49a0 con 0x55650514b880 2024-06-10T02:55:21.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.174+0000 7f58c286ab80 1 -- 172.21.15.52:0/2606689803 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f58b01b64e0 con 0x5565051457a0 2024-06-10T02:55:21.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.174+0000 7f58c286ab80 1 -- 172.21.15.52:0/2606689803 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f58b01b5070 con 0x55650514af60 2024-06-10T02:55:21.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.174+0000 7f58b5582700 1 --2- 172.21.15.52:0/2606689803 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55650514b880 0x55650513fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:21.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.174+0000 7f58b5d83700 1 -- 172.21.15.52:0/2606689803 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5565051457a0 msgr2=0x556505145260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:55:21.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.174+0000 7f58b5d83700 1 --2- 172.21.15.52:0/2606689803 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5565051457a0 0x556505145260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:21.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.174+0000 7f58b5d83700 1 -- 172.21.15.52:0/2606689803 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55650514b880 msgr2=0x55650513fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:55:21.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.174+0000 7f58b5d83700 1 --2- 172.21.15.52:0/2606689803 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55650514b880 0x55650513fe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:21.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.174+0000 7f58b5d83700 1 -- 172.21.15.52:0/2606689803 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55650514cee0 con 0x55650514af60 2024-06-10T02:55:21.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.175+0000 7f58b5d83700 1 --2- 172.21.15.52:0/2606689803 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55650514af60 0x55650514f040 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f58ac00f910 tx=0x7f58ac009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=107ba8428ad8fd48 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:55:21.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.175+0000 7f589b7fe700 1 -- 172.21.15.52:0/2606689803 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58ac027040 con 0x55650514af60 2024-06-10T02:55:21.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.175+0000 7f589b7fe700 1 -- 172.21.15.52:0/2606689803 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f58ac00f660 con 0x55650514af60 2024-06-10T02:55:21.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.175+0000 7f589b7fe700 1 -- 172.21.15.52:0/2606689803 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58ac00b9d0 con 0x55650514af60 2024-06-10T02:55:21.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.176+0000 7f58c286ab80 1 -- 172.21.15.52:0/2606689803 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55650514af60 msgr2=0x55650514f040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:55:21.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.176+0000 7f58c286ab80 1 --2- 172.21.15.52:0/2606689803 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55650514af60 0x55650514f040 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f58ac00f910 tx=0x7f58ac009310 comp rx=0 tx=0).stop 2024-06-10T02:55:21.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.176+0000 7f58c286ab80 1 -- 172.21.15.52:0/2606689803 shutdown_connections 2024-06-10T02:55:21.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.176+0000 7f58c286ab80 1 --2- 172.21.15.52:0/2606689803 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55650514b880 0x55650513fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:21.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.176+0000 7f58c286ab80 1 --2- 172.21.15.52:0/2606689803 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5565051457a0 0x556505145260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:21.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.176+0000 7f58c286ab80 1 --2- 172.21.15.52:0/2606689803 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55650514af60 0x55650514f040 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:21.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.176+0000 7f58c286ab80 1 -- 172.21.15.52:0/2606689803 >> 172.21.15.52:0/2606689803 conn(0x556505072080 msgr2=0x55650514e930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:55:21.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.176+0000 7f58c286ab80 1 -- 172.21.15.52:0/2606689803 shutdown_connections 2024-06-10T02:55:21.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.176+0000 7f58c286ab80 1 -- 172.21.15.52:0/2606689803 wait complete. 2024-06-10T02:55:21.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.177+0000 7f58c286ab80 1 -- start start 2024-06-10T02:55:21.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.177+0000 7f58c286ab80 1 Processor -- start 2024-06-10T02:55:21.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.178+0000 7f58c286ab80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5565051457a0 0x556505157520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:21.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.178+0000 7f58c286ab80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55650514af60 0x556505157a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:21.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.178+0000 7f58c286ab80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55650514b880 0x556505166150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:21.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.178+0000 7f58c286ab80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f58b01b5e10 con 0x5565051457a0 2024-06-10T02:55:21.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.178+0000 7f58c286ab80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f58b01b5740 con 0x55650514af60 2024-06-10T02:55:21.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.178+0000 7f58c286ab80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f58b01b2940 con 0x55650514b880 2024-06-10T02:55:21.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.178+0000 7f58c286ab80 10 client.0 _ll_register_callbacks cb 0x55650502c5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T02:55:21.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.178+0000 7f58b5d83700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55650514af60 0x556505157a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:21.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.178+0000 7f58b5d83700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55650514af60 0x556505157a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.52:50428/0 (socket says 172.21.15.52:50428) 2024-06-10T02:55:21.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.178+0000 7f58b5d83700 1 -- 172.21.15.52:0/4133241930 learned_addr learned my addr 172.21.15.52:0/4133241930 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:55:21.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.178+0000 7f58b5582700 1 --2- 172.21.15.52:0/4133241930 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5565051457a0 0x556505157520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:21.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.178+0000 7f58b5d83700 1 -- 172.21.15.52:0/4133241930 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55650514b880 msgr2=0x556505166150 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-10T02:55:21.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.178+0000 7f58b5d83700 1 --2- 172.21.15.52:0/4133241930 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55650514b880 0x556505166150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:21.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.178+0000 7f58b5d83700 1 -- 172.21.15.52:0/4133241930 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5565051457a0 msgr2=0x556505157520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:55:21.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.178+0000 7f58b5d83700 1 --2- 172.21.15.52:0/4133241930 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5565051457a0 0x556505157520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:21.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.178+0000 7f58b5d83700 1 -- 172.21.15.52:0/4133241930 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556505166790 con 0x55650514af60 2024-06-10T02:55:21.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.179+0000 7f58b5d83700 1 --2- 172.21.15.52:0/4133241930 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55650514af60 0x556505157a60 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f58ac0059a0 tx=0x7f58ac00fd90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7712711b5b3786d5 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:55:21.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.179+0000 7f58997fa700 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-06-10T02:55:21.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.179+0000 7f58997fa700 1 -- 172.21.15.52:0/4133241930 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58ac009590 con 0x55650514af60 2024-06-10T02:55:21.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.179+0000 7f58997fa700 1 -- 172.21.15.52:0/4133241930 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f58ac0054e0 con 0x55650514af60 2024-06-10T02:55:21.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.179+0000 7f58c286ab80 1 -- 172.21.15.52:0/4133241930 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f58ac00f660 con 0x55650514af60 2024-06-10T02:55:21.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.180+0000 7f58997fa700 1 -- 172.21.15.52:0/4133241930 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f58ac049050 con 0x55650514af60 2024-06-10T02:55:21.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.180+0000 7f58c286ab80 10 client.25280 fetch_fsmap learned FSMap version 30 2024-06-10T02:55:21.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.180+0000 7f58c286ab80 1 -- 172.21.15.52:0/4133241930 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5565051c9650 con 0x55650514af60 2024-06-10T02:55:21.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.180+0000 7f58997fa700 1 -- 172.21.15.52:0/4133241930 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f58ac005bc0 con 0x55650514af60 2024-06-10T02:55:21.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.181+0000 7f58997fa700 1 -- 172.21.15.52:0/4133241930 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f58ac044030 con 0x55650514af60 2024-06-10T02:55:21.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.181+0000 7f58c286ab80 10 client.25280 fetch_fsmap finished waiting for FSMap version 30 2024-06-10T02:55:21.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.181+0000 7f58c286ab80 10 client.25280 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T02:55:21.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.181+0000 7f58c286ab80 20 client.25280 populate_metadata read hostname 'smithi052' 2024-06-10T02:55:21.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.181+0000 7f58c286ab80 1 --2- 172.21.15.52:0/4133241930 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55650515f0a0 0x5565051cbd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:21.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.181+0000 7f58c286ab80 4 client.25280 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T02:55:21.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.181+0000 7f58c286ab80 1 -- 172.21.15.52:0/4133241930 --> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f58ac00f660 con 0x55650515f0a0 2024-06-10T02:55:21.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.181+0000 7f58b5582700 1 --2- 172.21.15.52:0/4133241930 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55650515f0a0 0x5565051cbd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:21.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.183+0000 7f58997fa700 10 client.25280 ms_handle_connect on v2:172.21.15.188:6838/310444836 2024-06-10T02:55:21.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.183+0000 7f58b5582700 1 --2- 172.21.15.52:0/4133241930 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55650515f0a0 0x5565051cbd40 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=54e7f8c53f5c5748 server_cookie=85c749c9265f83de in_seq=0 out_seq=0 2024-06-10T02:55:21.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.183+0000 7f58997fa700 1 -- 172.21.15.52:0/4133241930 <== mds.0 v2:172.21.15.188:6838/310444836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f58ac00f660 con 0x55650515f0a0 2024-06-10T02:55:21.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.183+0000 7f58997fa700 10 client.25280 handle_command_reply: tid=0 2024-06-10T02:55:21.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.183+0000 7f58c286ab80 1 client.25280 shutdown 2024-06-10T02:55:21.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.185+0000 7f58c286ab80 1 -- 172.21.15.52:0/4133241930 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55650514af60 msgr2=0x556505157a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:55:21.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.185+0000 7f58c286ab80 1 --2- 172.21.15.52:0/4133241930 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55650514af60 0x556505157a60 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f58ac0059a0 tx=0x7f58ac00fd90 comp rx=0 tx=0).stop 2024-06-10T02:55:21.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.186+0000 7f58c286ab80 1 -- 172.21.15.52:0/4133241930 shutdown_connections 2024-06-10T02:55:21.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.186+0000 7f58c286ab80 1 --2- 172.21.15.52:0/4133241930 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55650515f0a0 0x5565051cbd40 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:21.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.186+0000 7f58c286ab80 1 --2- 172.21.15.52:0/4133241930 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55650514b880 0x556505166150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:21.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.186+0000 7f58c286ab80 1 --2- 172.21.15.52:0/4133241930 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5565051457a0 0x556505157520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:21.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.186+0000 7f58c286ab80 1 --2- 172.21.15.52:0/4133241930 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55650514af60 0x556505157a60 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:21.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.186+0000 7f58c286ab80 1 -- 172.21.15.52:0/4133241930 >> 172.21.15.52:0/4133241930 conn(0x556505072080 msgr2=0x55650514c4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:55:21.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.186+0000 7f58997fa700 0 client.25280 ms_handle_reset on v2:172.21.15.188:6838/310444836 2024-06-10T02:55:21.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.186+0000 7f58c286ab80 1 -- 172.21.15.52:0/4133241930 shutdown_connections 2024-06-10T02:55:21.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.186+0000 7f58c286ab80 1 -- 172.21.15.52:0/4133241930 wait complete. 2024-06-10T02:55:21.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.186+0000 7f58c286ab80 20 client.25280 trim_cache size 0 max 16384 2024-06-10T02:55:21.186 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-06-10T02:55:21.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.187+0000 7f58b7fe7700 1 -- 172.21.15.52:0/2967895761 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f589c0a2d20 msgr2=0x7f589c0a51e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:55:21.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.187+0000 7f58b7fe7700 1 --2- 172.21.15.52:0/2967895761 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f589c0a2d20 0x7f589c0a51e0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f58a8097960 tx=0x7f58a800df40 comp rx=0 tx=0).stop 2024-06-10T02:55:21.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.187+0000 7f58b7fe7700 1 -- 172.21.15.52:0/2967895761 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f58b01073f0 msgr2=0x7f58b010aee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:55:21.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.187+0000 7f58b7fe7700 1 --2- 172.21.15.52:0/2967895761 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f58b01073f0 0x7f58b010aee0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f58a8007f10 tx=0x7f58a800d3b0 comp rx=0 tx=0).stop 2024-06-10T02:55:21.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.187+0000 7f58b7fe7700 1 -- 172.21.15.52:0/2967895761 shutdown_connections 2024-06-10T02:55:21.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.187+0000 7f58b7fe7700 1 --2- 172.21.15.52:0/2967895761 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f589c0a2d20 0x7f589c0a51e0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:21.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.187+0000 7f58b7fe7700 1 --2- 172.21.15.52:0/2967895761 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f58b01073f0 0x7f58b010aee0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:21.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.187+0000 7f58b7fe7700 1 --2- 172.21.15.52:0/2967895761 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f58b0101380 0x7f58b010a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:21.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.187+0000 7f58b7fe7700 1 --2- 172.21.15.52:0/2967895761 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f58b0101d40 0x7f58b010a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:21.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.187+0000 7f58b7fe7700 1 -- 172.21.15.52:0/2967895761 >> 172.21.15.52:0/2967895761 conn(0x7f58b0078300 msgr2=0x7f58b0100010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:55:21.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.188+0000 7f58b7fe7700 1 -- 172.21.15.52:0/2967895761 shutdown_connections 2024-06-10T02:55:21.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:21.188+0000 7f58b7fe7700 1 -- 172.21.15.52:0/2967895761 wait complete. 2024-06-10T02:55:21.209 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:55:22.280 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:55:22 smithi052 ceph-mon[32362]: pgmap v469: 129 pgs: 129 active+clean; 662 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 511 B/s wr, 9 op/s 2024-06-10T02:55:22.387 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:55:22 smithi120 ceph-mon[36634]: pgmap v469: 129 pgs: 129 active+clean; 662 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 511 B/s wr, 9 op/s 2024-06-10T02:55:22.453 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:55:22 smithi188 ceph-mon[36599]: pgmap v469: 129 pgs: 129 active+clean; 662 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 511 B/s wr, 9 op/s 2024-06-10T02:55:23.051 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-06-10T02:55:23.051 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-06-10T02:55:23.823 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:55:23.848 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.848+0000 7fa9d3e21700 1 -- 172.21.15.52:0/4007650833 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa9cc06b420 msgr2=0x7fa9cc10f340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:55:23.848 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.848+0000 7fa9d3e21700 1 --2- 172.21.15.52:0/4007650833 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa9cc06b420 0x7fa9cc10f340 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa9c4009f90 tx=0x7fa9c400b330 comp rx=0 tx=0).stop 2024-06-10T02:55:23.849 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.848+0000 7fa9d3e21700 1 -- 172.21.15.52:0/4007650833 shutdown_connections 2024-06-10T02:55:23.849 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.848+0000 7fa9d3e21700 1 --2- 172.21.15.52:0/4007650833 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa9cc06b420 0x7fa9cc10f340 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:23.849 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.848+0000 7fa9d3e21700 1 --2- 172.21.15.52:0/4007650833 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa9cc108520 0x7fa9cc108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:23.849 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.848+0000 7fa9d3e21700 1 --2- 172.21.15.52:0/4007650833 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa9cc06aa60 0x7fa9cc06aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:23.849 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.848+0000 7fa9d3e21700 1 -- 172.21.15.52:0/4007650833 >> 172.21.15.52:0/4007650833 conn(0x7fa9cc0783d0 msgr2=0x7fa9cc0787e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:55:23.849 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.848+0000 7fa9d3e21700 1 -- 172.21.15.52:0/4007650833 shutdown_connections 2024-06-10T02:55:23.849 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.849+0000 7fa9d3e21700 1 -- 172.21.15.52:0/4007650833 wait complete. 2024-06-10T02:55:23.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.849+0000 7fa9d3e21700 1 Processor -- start 2024-06-10T02:55:23.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.850+0000 7fa9d3e21700 1 -- start start 2024-06-10T02:55:23.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.850+0000 7fa9d3e21700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa9cc06aa60 0x7fa9cc10e520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:23.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.850+0000 7fa9d3e21700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa9cc06b420 0x7fa9cc10ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:23.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.850+0000 7fa9d3e21700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa9cc108520 0x7fa9cc19b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:23.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.850+0000 7fa9d3e21700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fa9cc1127f0 con 0x7fa9cc06b420 2024-06-10T02:55:23.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.850+0000 7fa9d3e21700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa9cc112670 con 0x7fa9cc108520 2024-06-10T02:55:23.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.850+0000 7fa9d3e21700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa9cc112970 con 0x7fa9cc06aa60 2024-06-10T02:55:23.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.850+0000 7fa9d13bc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa9cc06b420 0x7fa9cc10ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:23.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.850+0000 7fa9d13bc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa9cc06b420 0x7fa9cc10ea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48450/0 (socket says 172.21.15.52:48450) 2024-06-10T02:55:23.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.850+0000 7fa9d13bc700 1 -- 172.21.15.52:0/1914990663 learned_addr learned my addr 172.21.15.52:0/1914990663 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:55:23.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.850+0000 7fa9d1bbd700 1 --2- 172.21.15.52:0/1914990663 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa9cc06aa60 0x7fa9cc10e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:23.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.851+0000 7fa9d13bc700 1 -- 172.21.15.52:0/1914990663 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa9cc06aa60 msgr2=0x7fa9cc10e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:55:23.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.851+0000 7fa9d13bc700 1 --2- 172.21.15.52:0/1914990663 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa9cc06aa60 0x7fa9cc10e520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:23.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.851+0000 7fa9d13bc700 1 -- 172.21.15.52:0/1914990663 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa9cc108520 msgr2=0x7fa9cc19b2d0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-06-10T02:55:23.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.851+0000 7fa9d13bc700 1 --2- 172.21.15.52:0/1914990663 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa9cc108520 0x7fa9cc19b2d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:23.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.851+0000 7fa9d13bc700 1 -- 172.21.15.52:0/1914990663 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9cc19b950 con 0x7fa9cc06b420 2024-06-10T02:55:23.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.851+0000 7fa9d13bc700 1 --2- 172.21.15.52:0/1914990663 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa9cc06b420 0x7fa9cc10ea60 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fa9c000e9c0 tx=0x7fa9c000ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:55:23.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.851+0000 7fa9beffd700 1 -- 172.21.15.52:0/1914990663 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9c000cbd0 con 0x7fa9cc06b420 2024-06-10T02:55:23.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.851+0000 7fa9beffd700 1 -- 172.21.15.52:0/1914990663 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9c000cd70 con 0x7fa9cc06b420 2024-06-10T02:55:23.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.851+0000 7fa9beffd700 1 -- 172.21.15.52:0/1914990663 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9c0010430 con 0x7fa9cc06b420 2024-06-10T02:55:23.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.851+0000 7fa9d3e21700 1 -- 172.21.15.52:0/1914990663 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9cc19bc40 con 0x7fa9cc06b420 2024-06-10T02:55:23.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.851+0000 7fa9d3e21700 1 -- 172.21.15.52:0/1914990663 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9cc198200 con 0x7fa9cc06b420 2024-06-10T02:55:23.854 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.855+0000 7fa9beffd700 1 -- 172.21.15.52:0/1914990663 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fa9c0004750 con 0x7fa9cc06b420 2024-06-10T02:55:23.856 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.856+0000 7fa9beffd700 1 --2- 172.21.15.52:0/1914990663 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fa9b80a2dc0 0x7fa9b80a5280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:23.856 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.857+0000 7fa9beffd700 1 -- 172.21.15.52:0/1914990663 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7fa9c00cf690 con 0x7fa9cc06b420 2024-06-10T02:55:23.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.857+0000 7fa9d1bbd700 1 --2- 172.21.15.52:0/1914990663 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fa9b80a2dc0 0x7fa9b80a5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:23.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.857+0000 7fa9d1bbd700 1 --2- 172.21.15.52:0/1914990663 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fa9b80a2dc0 0x7fa9b80a5280 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fa9c8009750 tx=0x7fa9c8009310 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:55:23.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.859+0000 7fa9de6a4b80 1 Processor -- start 2024-06-10T02:55:23.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.859+0000 7fa9de6a4b80 1 -- start start 2024-06-10T02:55:23.860 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.860+0000 7fa9de6a4b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ecfc64dcd0 0x55ecfc64e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:23.860 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.860+0000 7fa9d1bbd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ecfc64dcd0 0x55ecfc64e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:23.860 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.860+0000 7fa9d1bbd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ecfc64dcd0 0x55ecfc64e0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48460/0 (socket says 172.21.15.52:48460) 2024-06-10T02:55:23.860 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.860+0000 7fa9d1bbd700 1 -- 172.21.15.52:0/1506045864 learned_addr learned my addr 172.21.15.52:0/1506045864 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:55:23.860 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.860+0000 7fa9de6a4b80 1 --2- 172.21.15.52:0/1506045864 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ecfc64e5f0 0x55ecfc647f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:23.860 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.860+0000 7fa9de6a4b80 1 --2- 172.21.15.52:0/1506045864 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ecfc648460 0x55ecfc6488a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:23.861 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.861+0000 7fa9de6a4b80 1 -- 172.21.15.52:0/1506045864 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55ecfc5999a0 con 0x55ecfc64dcd0 2024-06-10T02:55:23.861 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.861+0000 7fa9de6a4b80 1 -- 172.21.15.52:0/1506045864 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55ecfc5418b0 con 0x55ecfc648460 2024-06-10T02:55:23.861 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.861+0000 7fa9de6a4b80 1 -- 172.21.15.52:0/1506045864 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55ecfc595960 con 0x55ecfc64e5f0 2024-06-10T02:55:23.861 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.861+0000 7fa9d13bc700 1 --2- 172.21.15.52:0/1506045864 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ecfc64e5f0 0x55ecfc647f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:23.861 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.861+0000 7fa9d23be700 1 --2- 172.21.15.52:0/1506045864 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ecfc648460 0x55ecfc6488a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:23.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.861+0000 7fa9d1bbd700 1 -- 172.21.15.52:0/1506045864 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ecfc64e5f0 msgr2=0x55ecfc647f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:55:23.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.861+0000 7fa9d1bbd700 1 --2- 172.21.15.52:0/1506045864 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ecfc64e5f0 0x55ecfc647f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:23.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.861+0000 7fa9d1bbd700 1 -- 172.21.15.52:0/1506045864 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ecfc648460 msgr2=0x55ecfc6488a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:55:23.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.861+0000 7fa9d1bbd700 1 --2- 172.21.15.52:0/1506045864 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ecfc648460 0x55ecfc6488a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:23.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.861+0000 7fa9d1bbd700 1 -- 172.21.15.52:0/1506045864 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ecfc5380e0 con 0x55ecfc64dcd0 2024-06-10T02:55:23.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.862+0000 7fa9d1bbd700 1 --2- 172.21.15.52:0/1506045864 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ecfc64dcd0 0x55ecfc64e0b0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fa9c800e540 tx=0x7fa9c8014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7db22d198296df0c server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:55:23.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.862+0000 7fa9b7fff700 1 -- 172.21.15.52:0/1506045864 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9c8004020 con 0x55ecfc64dcd0 2024-06-10T02:55:23.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.862+0000 7fa9b7fff700 1 -- 172.21.15.52:0/1506045864 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9c8003680 con 0x55ecfc64dcd0 2024-06-10T02:55:23.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.863+0000 7fa9b7fff700 1 -- 172.21.15.52:0/1506045864 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9c8026770 con 0x55ecfc64dcd0 2024-06-10T02:55:23.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.863+0000 7fa9de6a4b80 1 -- 172.21.15.52:0/1506045864 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ecfc64dcd0 msgr2=0x55ecfc64e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:55:23.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.863+0000 7fa9de6a4b80 1 --2- 172.21.15.52:0/1506045864 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ecfc64dcd0 0x55ecfc64e0b0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fa9c800e540 tx=0x7fa9c8014000 comp rx=0 tx=0).stop 2024-06-10T02:55:23.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.863+0000 7fa9de6a4b80 1 -- 172.21.15.52:0/1506045864 shutdown_connections 2024-06-10T02:55:23.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.863+0000 7fa9de6a4b80 1 --2- 172.21.15.52:0/1506045864 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ecfc648460 0x55ecfc6488a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:23.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.864+0000 7fa9de6a4b80 1 --2- 172.21.15.52:0/1506045864 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ecfc64dcd0 0x55ecfc64e0b0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:23.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.864+0000 7fa9de6a4b80 1 --2- 172.21.15.52:0/1506045864 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ecfc64e5f0 0x55ecfc647f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:23.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.864+0000 7fa9de6a4b80 1 -- 172.21.15.52:0/1506045864 >> 172.21.15.52:0/1506045864 conn(0x55ecfc572e50 msgr2=0x55ecfc6431b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:55:23.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.864+0000 7fa9de6a4b80 1 -- 172.21.15.52:0/1506045864 shutdown_connections 2024-06-10T02:55:23.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.864+0000 7fa9de6a4b80 1 -- 172.21.15.52:0/1506045864 wait complete. 2024-06-10T02:55:23.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.864+0000 7fa9de6a4b80 1 -- start start 2024-06-10T02:55:23.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.865+0000 7fa9de6a4b80 1 Processor -- start 2024-06-10T02:55:23.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.865+0000 7fa9de6a4b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ecfc648460 0x55ecfc6542e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:23.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.866+0000 7fa9de6a4b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ecfc64dcd0 0x55ecfc6515e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:23.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.866+0000 7fa9de6a4b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ecfc64e5f0 0x55ecfc651b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:23.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.866+0000 7fa9de6a4b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55ecfc6550b0 con 0x55ecfc648460 2024-06-10T02:55:23.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.866+0000 7fa9d1bbd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ecfc648460 0x55ecfc6542e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:23.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.866+0000 7fa9d1bbd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ecfc648460 0x55ecfc6542e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48474/0 (socket says 172.21.15.52:48474) 2024-06-10T02:55:23.866 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.866+0000 7fa9de6a4b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55ecfc539d80 con 0x55ecfc64e5f0 2024-06-10T02:55:23.866 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.866+0000 7fa9d1bbd700 1 -- 172.21.15.52:0/2091944065 learned_addr learned my addr 172.21.15.52:0/2091944065 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:55:23.866 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.866+0000 7fa9de6a4b80 1 -- 172.21.15.52:0/2091944065 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55ecfc655230 con 0x55ecfc64dcd0 2024-06-10T02:55:23.866 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.866+0000 7fa9de6a4b80 10 client.0 _ll_register_callbacks cb 0x55ecfc533ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T02:55:23.866 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.866+0000 7fa9d13bc700 1 --2- 172.21.15.52:0/2091944065 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ecfc64dcd0 0x55ecfc6515e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:23.866 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.866+0000 7fa9d23be700 1 --2- 172.21.15.52:0/2091944065 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ecfc64e5f0 0x55ecfc651b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:23.867 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.866+0000 7fa9d1bbd700 1 -- 172.21.15.52:0/2091944065 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ecfc64dcd0 msgr2=0x55ecfc6515e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:55:23.867 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.866+0000 7fa9d1bbd700 1 --2- 172.21.15.52:0/2091944065 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ecfc64dcd0 0x55ecfc6515e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:23.867 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.866+0000 7fa9d1bbd700 1 -- 172.21.15.52:0/2091944065 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ecfc64e5f0 msgr2=0x55ecfc651b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:55:23.867 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.866+0000 7fa9d1bbd700 1 --2- 172.21.15.52:0/2091944065 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ecfc64e5f0 0x55ecfc651b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:23.867 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.866+0000 7fa9d1bbd700 1 -- 172.21.15.52:0/2091944065 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ecfc4e8a60 con 0x55ecfc648460 2024-06-10T02:55:23.867 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.867+0000 7fa9d1bbd700 1 --2- 172.21.15.52:0/2091944065 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ecfc648460 0x55ecfc6542e0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fa9c8009890 tx=0x7fa9c8003830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1bf5ea58ec373fa server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:55:23.867 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.867+0000 7fa9b57fa700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-10T02:55:23.867 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.867+0000 7fa9b57fa700 1 -- 172.21.15.52:0/2091944065 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9c8005270 con 0x55ecfc648460 2024-06-10T02:55:23.867 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.867+0000 7fa9b57fa700 1 -- 172.21.15.52:0/2091944065 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7fa9c8005410 con 0x55ecfc648460 2024-06-10T02:55:23.867 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.867+0000 7fa9de6a4b80 1 -- 172.21.15.52:0/2091944065 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa9c8003680 con 0x55ecfc648460 2024-06-10T02:55:23.868 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.869+0000 7fa9b57fa700 1 -- 172.21.15.52:0/2091944065 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa9c8038050 con 0x55ecfc648460 2024-06-10T02:55:23.868 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.869+0000 7fa9de6a4b80 10 client.15654 fetch_fsmap learned FSMap version 30 2024-06-10T02:55:23.868 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.869+0000 7fa9de6a4b80 1 -- 172.21.15.52:0/2091944065 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55ecfc65f910 con 0x55ecfc648460 2024-06-10T02:55:23.869 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.869+0000 7fa9b57fa700 1 -- 172.21.15.52:0/2091944065 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9c8005610 con 0x55ecfc648460 2024-06-10T02:55:23.869 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.869+0000 7fa9b57fa700 1 -- 172.21.15.52:0/2091944065 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fa9c802f550 con 0x55ecfc648460 2024-06-10T02:55:23.869 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.869+0000 7fa9de6a4b80 10 client.15654 fetch_fsmap finished waiting for FSMap version 30 2024-06-10T02:55:23.869 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.869+0000 7fa9de6a4b80 10 client.15654 resolve_mds: validated gid 24484 aka daemon mds.i 2024-06-10T02:55:23.869 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.869+0000 7fa9de6a4b80 20 client.15654 populate_metadata read hostname 'smithi052' 2024-06-10T02:55:23.869 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.869+0000 7fa9de6a4b80 1 --2- 172.21.15.52:0/2091944065 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55ecfc661e90 0x55ecfc664280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:23.869 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.869+0000 7fa9de6a4b80 4 client.15654 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T02:55:23.869 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.869+0000 7fa9de6a4b80 1 -- 172.21.15.52:0/2091944065 --> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fa9c8003680 con 0x55ecfc661e90 2024-06-10T02:55:23.870 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.870+0000 7fa9d13bc700 1 --2- 172.21.15.52:0/2091944065 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55ecfc661e90 0x55ecfc664280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:23.870 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.870+0000 7fa9b57fa700 10 client.15654 ms_handle_connect on v2:172.21.15.188:6838/310444836 2024-06-10T02:55:23.870 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.870+0000 7fa9d13bc700 1 --2- 172.21.15.52:0/2091944065 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55ecfc661e90 0x55ecfc664280 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=411debff226afe1a server_cookie=a1c7e33f8d13e1fe in_seq=0 out_seq=0 2024-06-10T02:55:23.870 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.871+0000 7fa9b57fa700 1 -- 172.21.15.52:0/2091944065 <== mds.0 v2:172.21.15.188:6838/310444836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x7fa9c8003680 con 0x55ecfc661e90 2024-06-10T02:55:23.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.871+0000 7fa9b57fa700 10 client.15654 handle_command_reply: tid=0 2024-06-10T02:55:23.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.871+0000 7fa9de6a4b80 1 client.15654 shutdown 2024-06-10T02:55:23.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.871+0000 7fa9de6a4b80 1 -- 172.21.15.52:0/2091944065 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ecfc648460 msgr2=0x55ecfc6542e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:55:23.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.871+0000 7fa9de6a4b80 1 --2- 172.21.15.52:0/2091944065 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ecfc648460 0x55ecfc6542e0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fa9c8009890 tx=0x7fa9c8003830 comp rx=0 tx=0).stop 2024-06-10T02:55:23.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.872+0000 7fa9de6a4b80 1 -- 172.21.15.52:0/2091944065 shutdown_connections 2024-06-10T02:55:23.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.872+0000 7fa9de6a4b80 1 --2- 172.21.15.52:0/2091944065 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55ecfc661e90 0x55ecfc664280 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:23.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.872+0000 7fa9de6a4b80 1 --2- 172.21.15.52:0/2091944065 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ecfc64e5f0 0x55ecfc651b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:23.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.872+0000 7fa9de6a4b80 1 --2- 172.21.15.52:0/2091944065 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ecfc648460 0x55ecfc6542e0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:23.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.872+0000 7fa9de6a4b80 1 --2- 172.21.15.52:0/2091944065 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ecfc64dcd0 0x55ecfc6515e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:23.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.872+0000 7fa9de6a4b80 1 -- 172.21.15.52:0/2091944065 >> 172.21.15.52:0/2091944065 conn(0x55ecfc572e50 msgr2=0x55ecfc642da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:55:23.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.872+0000 7fa9b57fa700 0 client.15654 ms_handle_reset on v2:172.21.15.188:6838/310444836 2024-06-10T02:55:23.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.872+0000 7fa9de6a4b80 1 -- 172.21.15.52:0/2091944065 shutdown_connections 2024-06-10T02:55:23.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.872+0000 7fa9de6a4b80 1 -- 172.21.15.52:0/2091944065 wait complete. 2024-06-10T02:55:23.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.872+0000 7fa9de6a4b80 20 client.15654 trim_cache size 0 max 16384 2024-06-10T02:55:23.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.899+0000 7fa9de6a4b80 1 Processor -- start 2024-06-10T02:55:23.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.899+0000 7fa9de6a4b80 1 -- start start 2024-06-10T02:55:23.899 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.899+0000 7fa9de6a4b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ecfc648460 0x55ecfc647f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:23.899 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.899+0000 7fa9de6a4b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ecfc64dcd0 0x55ecfc642f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:23.899 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.899+0000 7fa9de6a4b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ecfc64e5f0 0x55ecfc6515a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:23.899 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.899+0000 7fa9de6a4b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fa9cc073b70 con 0x55ecfc64dcd0 2024-06-10T02:55:23.899 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.899+0000 7fa9de6a4b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa9cc1c7510 con 0x55ecfc64e5f0 2024-06-10T02:55:23.899 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.899+0000 7fa9de6a4b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa9cc074240 con 0x55ecfc648460 2024-06-10T02:55:23.899 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.899+0000 7fa9d13bc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ecfc64dcd0 0x55ecfc642f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:23.899 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.899+0000 7fa9d13bc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ecfc64dcd0 0x55ecfc642f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48490/0 (socket says 172.21.15.52:48490) 2024-06-10T02:55:23.899 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.899+0000 7fa9d13bc700 1 -- 172.21.15.52:0/1872803400 learned_addr learned my addr 172.21.15.52:0/1872803400 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:55:23.899 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.899+0000 7fa9d23be700 1 --2- 172.21.15.52:0/1872803400 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ecfc64e5f0 0x55ecfc6515a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:23.900 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.899+0000 7fa9d1bbd700 1 --2- 172.21.15.52:0/1872803400 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ecfc648460 0x55ecfc647f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:23.900 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.899+0000 7fa9d13bc700 1 -- 172.21.15.52:0/1872803400 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ecfc648460 msgr2=0x55ecfc647f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:55:23.900 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.899+0000 7fa9d13bc700 1 --2- 172.21.15.52:0/1872803400 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ecfc648460 0x55ecfc647f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:23.900 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.899+0000 7fa9d13bc700 1 -- 172.21.15.52:0/1872803400 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ecfc64e5f0 msgr2=0x55ecfc6515a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:55:23.900 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.899+0000 7fa9d13bc700 1 --2- 172.21.15.52:0/1872803400 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ecfc64e5f0 0x55ecfc6515a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:23.900 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.899+0000 7fa9d13bc700 1 -- 172.21.15.52:0/1872803400 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ecfc64c0d0 con 0x55ecfc64dcd0 2024-06-10T02:55:23.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.900+0000 7fa9d13bc700 1 --2- 172.21.15.52:0/1872803400 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ecfc64dcd0 0x55ecfc642f90 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fa9c0097f40 tx=0x7fa9c0005190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c815d126005844b6 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:55:23.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.900+0000 7fa9b77fe700 1 -- 172.21.15.52:0/1872803400 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9c007f070 con 0x55ecfc64dcd0 2024-06-10T02:55:23.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.900+0000 7fa9b77fe700 1 -- 172.21.15.52:0/1872803400 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9c009c1e0 con 0x55ecfc64dcd0 2024-06-10T02:55:23.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.900+0000 7fa9b77fe700 1 -- 172.21.15.52:0/1872803400 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9c0004a40 con 0x55ecfc64dcd0 2024-06-10T02:55:23.902 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.901+0000 7fa9de6a4b80 1 -- 172.21.15.52:0/1872803400 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ecfc64dcd0 msgr2=0x55ecfc642f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:55:23.902 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.901+0000 7fa9de6a4b80 1 --2- 172.21.15.52:0/1872803400 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ecfc64dcd0 0x55ecfc642f90 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fa9c0097f40 tx=0x7fa9c0005190 comp rx=0 tx=0).stop 2024-06-10T02:55:23.902 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.901+0000 7fa9de6a4b80 1 -- 172.21.15.52:0/1872803400 shutdown_connections 2024-06-10T02:55:23.902 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.901+0000 7fa9de6a4b80 1 --2- 172.21.15.52:0/1872803400 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ecfc64e5f0 0x55ecfc6515a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:23.902 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.901+0000 7fa9de6a4b80 1 --2- 172.21.15.52:0/1872803400 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ecfc648460 0x55ecfc647f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:23.902 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.901+0000 7fa9de6a4b80 1 --2- 172.21.15.52:0/1872803400 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ecfc64dcd0 0x55ecfc642f90 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:23.902 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.901+0000 7fa9de6a4b80 1 -- 172.21.15.52:0/1872803400 >> 172.21.15.52:0/1872803400 conn(0x55ecfc572e50 msgr2=0x55ecfc650e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:55:23.902 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.901+0000 7fa9de6a4b80 1 -- 172.21.15.52:0/1872803400 shutdown_connections 2024-06-10T02:55:23.902 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.901+0000 7fa9de6a4b80 1 -- 172.21.15.52:0/1872803400 wait complete. 2024-06-10T02:55:23.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.901+0000 7fa9de6a4b80 1 -- start start 2024-06-10T02:55:23.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.901+0000 7fa9de6a4b80 1 Processor -- start 2024-06-10T02:55:23.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.902+0000 7fa9de6a4b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ecfc648460 0x55ecfc65ad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:23.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.902+0000 7fa9de6a4b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ecfc64dcd0 0x55ecfc657510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:23.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.902+0000 7fa9de6a4b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ecfc64e5f0 0x55ecfc657a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:23.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.902+0000 7fa9de6a4b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fa9cc1c6e40 con 0x55ecfc64dcd0 2024-06-10T02:55:23.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.902+0000 7fa9de6a4b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa9cc1c6770 con 0x55ecfc64e5f0 2024-06-10T02:55:23.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.902+0000 7fa9de6a4b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa9cc071b10 con 0x55ecfc648460 2024-06-10T02:55:23.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.902+0000 7fa9de6a4b80 10 client.0 _ll_register_callbacks cb 0x55ecfc533ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T02:55:23.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.902+0000 7fa9d13bc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ecfc64dcd0 0x55ecfc657510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:23.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.902+0000 7fa9d13bc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ecfc64dcd0 0x55ecfc657510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48492/0 (socket says 172.21.15.52:48492) 2024-06-10T02:55:23.906 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.902+0000 7fa9d13bc700 1 -- 172.21.15.52:0/1903699553 learned_addr learned my addr 172.21.15.52:0/1903699553 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:55:23.906 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.902+0000 7fa9d13bc700 1 -- 172.21.15.52:0/1903699553 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ecfc648460 msgr2=0x55ecfc65ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:55:23.906 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.902+0000 7fa9d1bbd700 1 --2- 172.21.15.52:0/1903699553 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ecfc648460 0x55ecfc65ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:23.906 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.902+0000 7fa9d13bc700 1 --2- 172.21.15.52:0/1903699553 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ecfc648460 0x55ecfc65ad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:23.906 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.902+0000 7fa9d13bc700 1 -- 172.21.15.52:0/1903699553 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ecfc64e5f0 msgr2=0x55ecfc657a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:55:23.906 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.902+0000 7fa9d23be700 1 --2- 172.21.15.52:0/1903699553 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ecfc64e5f0 0x55ecfc657a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:23.907 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.902+0000 7fa9d13bc700 1 --2- 172.21.15.52:0/1903699553 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ecfc64e5f0 0x55ecfc657a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:23.907 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.902+0000 7fa9d13bc700 1 -- 172.21.15.52:0/1903699553 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ecfc6582e0 con 0x55ecfc64dcd0 2024-06-10T02:55:23.907 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.902+0000 7fa9d1bbd700 1 --2- 172.21.15.52:0/1903699553 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ecfc648460 0x55ecfc65ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T02:55:23.907 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.903+0000 7fa9d23be700 1 --2- 172.21.15.52:0/1903699553 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ecfc64e5f0 0x55ecfc657a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T02:55:23.907 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.903+0000 7fa9d13bc700 1 --2- 172.21.15.52:0/1903699553 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ecfc64dcd0 0x55ecfc657510 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fa9c0020000 tx=0x7fa9c009b710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bfb8459eb700706 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:55:23.907 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.903+0000 7fa9b57fa700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-10T02:55:23.907 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.903+0000 7fa9b57fa700 1 -- 172.21.15.52:0/1903699553 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9c007f020 con 0x55ecfc64dcd0 2024-06-10T02:55:23.907 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.903+0000 7fa9b57fa700 1 -- 172.21.15.52:0/1903699553 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7fa9c00cce90 con 0x55ecfc64dcd0 2024-06-10T02:55:23.907 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.903+0000 7fa9de6a4b80 1 -- 172.21.15.52:0/1903699553 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa9c009c1e0 con 0x55ecfc64dcd0 2024-06-10T02:55:23.907 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.903+0000 7fa9b57fa700 1 -- 172.21.15.52:0/1903699553 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa9c009fe90 con 0x55ecfc64dcd0 2024-06-10T02:55:23.907 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.903+0000 7fa9de6a4b80 10 client.15660 fetch_fsmap learned FSMap version 30 2024-06-10T02:55:23.907 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.903+0000 7fa9de6a4b80 1 -- 172.21.15.52:0/1903699553 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55ecfc6cd850 con 0x55ecfc64dcd0 2024-06-10T02:55:23.907 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.904+0000 7fa9b57fa700 1 -- 172.21.15.52:0/1903699553 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9c0004a40 con 0x55ecfc64dcd0 2024-06-10T02:55:23.907 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.904+0000 7fa9b57fa700 1 -- 172.21.15.52:0/1903699553 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fa9c00b4ce0 con 0x55ecfc64dcd0 2024-06-10T02:55:23.908 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.904+0000 7fa9de6a4b80 10 client.15660 fetch_fsmap finished waiting for FSMap version 30 2024-06-10T02:55:23.908 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.904+0000 7fa9de6a4b80 10 client.15660 resolve_mds: validated gid 24484 aka daemon mds.i 2024-06-10T02:55:23.908 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.904+0000 7fa9de6a4b80 20 client.15660 populate_metadata read hostname 'smithi052' 2024-06-10T02:55:23.908 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.904+0000 7fa9de6a4b80 1 --2- 172.21.15.52:0/1903699553 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55ecfc6ce000 0x55ecfc6d0420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:23.908 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.904+0000 7fa9de6a4b80 4 client.15660 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}] 2024-06-10T02:55:23.908 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.904+0000 7fa9de6a4b80 1 -- 172.21.15.52:0/1903699553 --> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}) -- 0x7fa9c009c1e0 con 0x55ecfc6ce000 2024-06-10T02:55:23.908 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.904+0000 7fa9d1bbd700 1 --2- 172.21.15.52:0/1903699553 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55ecfc6ce000 0x55ecfc6d0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:23.908 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.905+0000 7fa9b57fa700 10 client.15660 ms_handle_connect on v2:172.21.15.188:6838/310444836 2024-06-10T02:55:23.908 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.905+0000 7fa9d1bbd700 1 --2- 172.21.15.52:0/1903699553 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55ecfc6ce000 0x55ecfc6d0420 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a1c500509220ce1 server_cookie=ae9f8ee3a0407892 in_seq=0 out_seq=0 2024-06-10T02:55:23.914 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.914+0000 7fa9b57fa700 1 -- 172.21.15.52:0/1903699553 <== mds.0 v2:172.21.15.188:6838/310444836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x7fa9c009c1e0 con 0x55ecfc6ce000 2024-06-10T02:55:23.914 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.914+0000 7fa9b57fa700 10 client.15660 handle_command_reply: tid=0 2024-06-10T02:55:23.914 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.914+0000 7fa9de6a4b80 1 client.15660 shutdown 2024-06-10T02:55:23.914 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.914+0000 7fa9de6a4b80 1 -- 172.21.15.52:0/1903699553 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ecfc64dcd0 msgr2=0x55ecfc657510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:55:23.914 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.914+0000 7fa9de6a4b80 1 --2- 172.21.15.52:0/1903699553 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ecfc64dcd0 0x55ecfc657510 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fa9c0020000 tx=0x7fa9c009b710 comp rx=0 tx=0).stop 2024-06-10T02:55:23.914 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.914+0000 7fa9de6a4b80 1 -- 172.21.15.52:0/1903699553 shutdown_connections 2024-06-10T02:55:23.914 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.914+0000 7fa9de6a4b80 1 --2- 172.21.15.52:0/1903699553 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55ecfc6ce000 0x55ecfc6d0420 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:23.915 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.914+0000 7fa9de6a4b80 1 --2- 172.21.15.52:0/1903699553 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ecfc64e5f0 0x55ecfc657a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:23.916 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-10T02:55:23.916 INFO:teuthology.orchestra.run.smithi052.stdout: "epoch": 30, 2024-06-10T02:55:23.916 INFO:teuthology.orchestra.run.smithi052.stdout: "leader": 24484, 2024-06-10T02:55:23.916 INFO:teuthology.orchestra.run.smithi052.stdout: "set_version": 2, 2024-06-10T02:55:23.916 INFO:teuthology.orchestra.run.smithi052.stdout: "sets": { 2024-06-10T02:55:23.916 INFO:teuthology.orchestra.run.smithi052.stdout: "57f4a9c1": { 2024-06-10T02:55:23.916 INFO:teuthology.orchestra.run.smithi052.stdout: "version": 2, 2024-06-10T02:55:23.916 INFO:teuthology.orchestra.run.smithi052.stdout: "age_ref": 0.0, 2024-06-10T02:55:23.917 INFO:teuthology.orchestra.run.smithi052.stdout: "state": { 2024-06-10T02:55:23.917 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "QUIESCED", 2024-06-10T02:55:23.917 INFO:teuthology.orchestra.run.smithi052.stdout: "age": 0.0 2024-06-10T02:55:23.917 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T02:55:23.917 INFO:teuthology.orchestra.run.smithi052.stdout: "timeout": 90.0, 2024-06-10T02:55:23.917 INFO:teuthology.orchestra.run.smithi052.stdout: "expiration": 150.0, 2024-06-10T02:55:23.917 INFO:teuthology.orchestra.run.smithi052.stdout: "members": { 2024-06-10T02:55:23.917 INFO:teuthology.orchestra.run.smithi052.stdout: "file:/": { 2024-06-10T02:55:23.917 INFO:teuthology.orchestra.run.smithi052.stdout: "excluded": false, 2024-06-10T02:55:23.917 INFO:teuthology.orchestra.run.smithi052.stdout: "state": { 2024-06-10T02:55:23.917 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "QUIESCED", 2024-06-10T02:55:23.917 INFO:teuthology.orchestra.run.smithi052.stdout: "age": 0.0 2024-06-10T02:55:23.917 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T02:55:23.917 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T02:55:23.918 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T02:55:23.918 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T02:55:23.918 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T02:55:23.918 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-10T02:55:23.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.914+0000 7fa9de6a4b80 1 --2- 172.21.15.52:0/1903699553 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ecfc648460 0x55ecfc65ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:23.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.914+0000 7fa9b57fa700 0 client.15660 ms_handle_reset on v2:172.21.15.188:6838/310444836 2024-06-10T02:55:23.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.914+0000 7fa9de6a4b80 1 --2- 172.21.15.52:0/1903699553 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ecfc64dcd0 0x55ecfc657510 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:23.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.914+0000 7fa9de6a4b80 1 -- 172.21.15.52:0/1903699553 >> 172.21.15.52:0/1903699553 conn(0x55ecfc572e50 msgr2=0x55ecfc64f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:55:23.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.915+0000 7fa9de6a4b80 1 -- 172.21.15.52:0/1903699553 shutdown_connections 2024-06-10T02:55:23.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.915+0000 7fa9de6a4b80 1 -- 172.21.15.52:0/1903699553 wait complete. 2024-06-10T02:55:23.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.915+0000 7fa9de6a4b80 20 client.15660 trim_cache size 0 max 16384 2024-06-10T02:55:23.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.917+0000 7fa9d3e21700 1 -- 172.21.15.52:0/1914990663 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fa9b80a2dc0 msgr2=0x7fa9b80a5280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:55:23.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.917+0000 7fa9d3e21700 1 --2- 172.21.15.52:0/1914990663 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fa9b80a2dc0 0x7fa9b80a5280 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fa9c8009750 tx=0x7fa9c8009310 comp rx=0 tx=0).stop 2024-06-10T02:55:23.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.917+0000 7fa9d3e21700 1 -- 172.21.15.52:0/1914990663 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa9cc06b420 msgr2=0x7fa9cc10ea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:55:23.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.917+0000 7fa9d3e21700 1 --2- 172.21.15.52:0/1914990663 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa9cc06b420 0x7fa9cc10ea60 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fa9c000e9c0 tx=0x7fa9c000ecd0 comp rx=0 tx=0).stop 2024-06-10T02:55:23.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.917+0000 7fa9d3e21700 1 -- 172.21.15.52:0/1914990663 shutdown_connections 2024-06-10T02:55:23.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.917+0000 7fa9d3e21700 1 --2- 172.21.15.52:0/1914990663 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa9cc108520 0x7fa9cc19b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:23.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.917+0000 7fa9d3e21700 1 --2- 172.21.15.52:0/1914990663 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa9cc06aa60 0x7fa9cc10e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:23.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.917+0000 7fa9d3e21700 1 --2- 172.21.15.52:0/1914990663 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fa9b80a2dc0 0x7fa9b80a5280 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:23.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.917+0000 7fa9d3e21700 1 --2- 172.21.15.52:0/1914990663 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa9cc06b420 0x7fa9cc10ea60 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:23.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.917+0000 7fa9d3e21700 1 -- 172.21.15.52:0/1914990663 >> 172.21.15.52:0/1914990663 conn(0x7fa9cc0783d0 msgr2=0x7fa9cc0ffbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:55:23.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.917+0000 7fa9d3e21700 1 -- 172.21.15.52:0/1914990663 shutdown_connections 2024-06-10T02:55:23.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:23.917+0000 7fa9d3e21700 1 -- 172.21.15.52:0/1914990663 wait complete. 2024-06-10T02:55:24.137 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:55:24 smithi052 ceph-mon[32362]: pgmap v470: 129 pgs: 129 active+clean; 667 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1023 B/s wr, 12 op/s 2024-06-10T02:55:24.388 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:55:24 smithi120 ceph-mon[36634]: pgmap v470: 129 pgs: 129 active+clean; 667 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1023 B/s wr, 12 op/s 2024-06-10T02:55:24.453 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:55:24 smithi188 ceph-mon[36599]: pgmap v470: 129 pgs: 129 active+clean; 667 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1023 B/s wr, 12 op/s 2024-06-10T02:55:24.742 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '57f4a9c1', quiesce took 9.0 seconds. Will release after: 21.0 2024-06-10T02:55:26.241 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:55:26 smithi052 ceph-mon[32362]: pgmap v471: 129 pgs: 129 active+clean; 667 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1023 B/s wr, 9 op/s 2024-06-10T02:55:26.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.306+0000 7f792ea72700 1 -- 172.21.15.52:0/267557064 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f792806ab30 msgr2=0x7f792810f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:55:26.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.306+0000 7f792ea72700 1 --2- 172.21.15.52:0/267557064 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f792806ab30 0x7f792810f320 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f792000b0a0 tx=0x7f792000b3b0 comp rx=0 tx=0).stop 2024-06-10T02:55:26.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.307+0000 7f792ea72700 1 -- 172.21.15.52:0/267557064 shutdown_connections 2024-06-10T02:55:26.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.307+0000 7f792ea72700 1 --2- 172.21.15.52:0/267557064 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f792806ab30 0x7f792810f320 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:26.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.307+0000 7f792ea72700 1 --2- 172.21.15.52:0/267557064 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7928108500 0x7f79281088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:26.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.307+0000 7f792ea72700 1 --2- 172.21.15.52:0/267557064 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f792806a170 0x7f792806a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:26.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.307+0000 7f792ea72700 1 -- 172.21.15.52:0/267557064 >> 172.21.15.52:0/267557064 conn(0x7f7928078340 msgr2=0x7f7928078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:55:26.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.307+0000 7f792ea72700 1 -- 172.21.15.52:0/267557064 shutdown_connections 2024-06-10T02:55:26.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.307+0000 7f792ea72700 1 -- 172.21.15.52:0/267557064 wait complete. 2024-06-10T02:55:26.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.308+0000 7f792ea72700 1 Processor -- start 2024-06-10T02:55:26.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.308+0000 7f792ea72700 1 -- start start 2024-06-10T02:55:26.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.308+0000 7f792ea72700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f792806a170 0x7f792819e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:26.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.308+0000 7f792ea72700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f792806ab30 0x7f792819ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:26.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.309+0000 7f792ea72700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7928108500 0x7f79281989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:26.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.309+0000 7f792c80e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f792806a170 0x7f792819e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:26.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.309+0000 7f792c80e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f792806a170 0x7f792819e810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48512/0 (socket says 172.21.15.52:48512) 2024-06-10T02:55:26.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.309+0000 7f792c80e700 1 -- 172.21.15.52:0/1977636037 learned_addr learned my addr 172.21.15.52:0/1977636037 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:55:26.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.309+0000 7f792ea72700 1 -- 172.21.15.52:0/1977636037 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f79281127b0 con 0x7f792806a170 2024-06-10T02:55:26.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.309+0000 7f792ea72700 1 -- 172.21.15.52:0/1977636037 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f7928112630 con 0x7f792806ab30 2024-06-10T02:55:26.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.309+0000 7f792ea72700 1 -- 172.21.15.52:0/1977636037 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f7928112930 con 0x7f7928108500 2024-06-10T02:55:26.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.309+0000 7f7927fff700 1 --2- 172.21.15.52:0/1977636037 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f792806ab30 0x7f792819ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:26.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.309+0000 7f792d00f700 1 --2- 172.21.15.52:0/1977636037 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7928108500 0x7f79281989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:26.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.309+0000 7f7927fff700 1 -- 172.21.15.52:0/1977636037 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7928108500 msgr2=0x7f79281989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:55:26.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.309+0000 7f7927fff700 1 --2- 172.21.15.52:0/1977636037 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7928108500 0x7f79281989d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:26.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.309+0000 7f7927fff700 1 -- 172.21.15.52:0/1977636037 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f792806a170 msgr2=0x7f792819e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:55:26.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.309+0000 7f7927fff700 1 --2- 172.21.15.52:0/1977636037 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f792806a170 0x7f792819e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:26.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.309+0000 7f7927fff700 1 -- 172.21.15.52:0/1977636037 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7928199130 con 0x7f792806ab30 2024-06-10T02:55:26.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.309+0000 7f7927fff700 1 --2- 172.21.15.52:0/1977636037 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f792806ab30 0x7f792819ed50 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f791400e9f0 tx=0x7f791400ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:55:26.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.310+0000 7f7925ffb700 1 -- 172.21.15.52:0/1977636037 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f791400cbd0 con 0x7f792806ab30 2024-06-10T02:55:26.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.310+0000 7f7925ffb700 1 -- 172.21.15.52:0/1977636037 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f791400cd70 con 0x7f792806ab30 2024-06-10T02:55:26.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.310+0000 7f7925ffb700 1 -- 172.21.15.52:0/1977636037 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7914018710 con 0x7f792806ab30 2024-06-10T02:55:26.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.310+0000 7f792ea72700 1 -- 172.21.15.52:0/1977636037 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7928199420 con 0x7f792806ab30 2024-06-10T02:55:26.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.310+0000 7f792ea72700 1 -- 172.21.15.52:0/1977636037 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79281a56e0 con 0x7f792806ab30 2024-06-10T02:55:26.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.313+0000 7f7925ffb700 1 -- 172.21.15.52:0/1977636037 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f7914010430 con 0x7f792806ab30 2024-06-10T02:55:26.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.314+0000 7f7925ffb700 1 --2- 172.21.15.52:0/1977636037 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f79180a2c20 0x7f79180a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:26.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.314+0000 7f7925ffb700 1 -- 172.21.15.52:0/1977636037 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f79140cf610 con 0x7f792806ab30 2024-06-10T02:55:26.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.314+0000 7f792c80e700 1 --2- 172.21.15.52:0/1977636037 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f79180a2c20 0x7f79180a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:26.314 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.315+0000 7f792c80e700 1 --2- 172.21.15.52:0/1977636037 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f79180a2c20 0x7f79180a50e0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f7928199b10 tx=0x7f791c007480 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:55:26.314 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.315+0000 7f79392f5b80 1 Processor -- start 2024-06-10T02:55:26.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.315+0000 7f79392f5b80 1 -- start start 2024-06-10T02:55:26.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.316+0000 7f79392f5b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56020c4996c0 0x56020c499aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:26.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.316+0000 7f79392f5b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56020c499fe0 0x56020c493a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:26.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.316+0000 7f79392f5b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56020c493fa0 0x56020c4943c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:26.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.316+0000 7f792c80e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56020c4996c0 0x56020c499aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:26.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.316+0000 7f792c80e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56020c4996c0 0x56020c499aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48526/0 (socket says 172.21.15.52:48526) 2024-06-10T02:55:26.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.316+0000 7f792c80e700 1 -- 172.21.15.52:0/3941213872 learned_addr learned my addr 172.21.15.52:0/3941213872 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:55:26.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.316+0000 7f79392f5b80 1 -- 172.21.15.52:0/3941213872 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x56020c3e5890 con 0x56020c4996c0 2024-06-10T02:55:26.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.316+0000 7f79392f5b80 1 -- 172.21.15.52:0/3941213872 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56020c38d7a0 con 0x56020c499fe0 2024-06-10T02:55:26.317 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.316+0000 7f79392f5b80 1 -- 172.21.15.52:0/3941213872 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x56020c3e1850 con 0x56020c493fa0 2024-06-10T02:55:26.317 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.316+0000 7f792d00f700 1 --2- 172.21.15.52:0/3941213872 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56020c493fa0 0x56020c4943c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:26.317 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.316+0000 7f7927fff700 1 --2- 172.21.15.52:0/3941213872 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56020c499fe0 0x56020c493a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:26.317 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.317+0000 7f792d00f700 1 -- 172.21.15.52:0/3941213872 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56020c499fe0 msgr2=0x56020c493a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:55:26.317 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.317+0000 7f792d00f700 1 --2- 172.21.15.52:0/3941213872 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56020c499fe0 0x56020c493a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:26.317 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.317+0000 7f792d00f700 1 -- 172.21.15.52:0/3941213872 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56020c4996c0 msgr2=0x56020c499aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:55:26.317 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.317+0000 7f792d00f700 1 --2- 172.21.15.52:0/3941213872 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56020c4996c0 0x56020c499aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:26.317 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.317+0000 7f792d00f700 1 -- 172.21.15.52:0/3941213872 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56020c383fd0 con 0x56020c493fa0 2024-06-10T02:55:26.317 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.317+0000 7f792c80e700 1 --2- 172.21.15.52:0/3941213872 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56020c4996c0 0x56020c499aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T02:55:26.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.317+0000 7f792d00f700 1 --2- 172.21.15.52:0/3941213872 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56020c493fa0 0x56020c4943c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f792000b780 tx=0x7f7920009df0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=753d407cbbb65ba3 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:55:26.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.317+0000 7f7912ffd700 1 -- 172.21.15.52:0/3941213872 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7920022070 con 0x56020c493fa0 2024-06-10T02:55:26.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.317+0000 7f7912ffd700 1 -- 172.21.15.52:0/3941213872 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7920009460 con 0x56020c493fa0 2024-06-10T02:55:26.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.317+0000 7f7912ffd700 1 -- 172.21.15.52:0/3941213872 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7920007c60 con 0x56020c493fa0 2024-06-10T02:55:26.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.318+0000 7f79392f5b80 1 -- 172.21.15.52:0/3941213872 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56020c493fa0 msgr2=0x56020c4943c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:55:26.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.318+0000 7f79392f5b80 1 --2- 172.21.15.52:0/3941213872 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56020c493fa0 0x56020c4943c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f792000b780 tx=0x7f7920009df0 comp rx=0 tx=0).stop 2024-06-10T02:55:26.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.318+0000 7f79392f5b80 1 -- 172.21.15.52:0/3941213872 shutdown_connections 2024-06-10T02:55:26.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.318+0000 7f79392f5b80 1 --2- 172.21.15.52:0/3941213872 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56020c493fa0 0x56020c4943c0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:26.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.318+0000 7f79392f5b80 1 --2- 172.21.15.52:0/3941213872 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56020c4996c0 0x56020c499aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:26.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.318+0000 7f79392f5b80 1 --2- 172.21.15.52:0/3941213872 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56020c499fe0 0x56020c493a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:26.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.318+0000 7f79392f5b80 1 -- 172.21.15.52:0/3941213872 >> 172.21.15.52:0/3941213872 conn(0x56020c3bed40 msgr2=0x56020c48e8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:55:26.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.318+0000 7f79392f5b80 1 -- 172.21.15.52:0/3941213872 shutdown_connections 2024-06-10T02:55:26.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.318+0000 7f79392f5b80 1 -- 172.21.15.52:0/3941213872 wait complete. 2024-06-10T02:55:26.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.319+0000 7f79392f5b80 1 -- start start 2024-06-10T02:55:26.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.319+0000 7f79392f5b80 1 Processor -- start 2024-06-10T02:55:26.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.319+0000 7f79392f5b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56020c493fa0 0x56020c49fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:26.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.319+0000 7f79392f5b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56020c4996c0 0x56020c49cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:26.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.320+0000 7f79392f5b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56020c499fe0 0x56020c49d520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:26.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.320+0000 7f792c80e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56020c493fa0 0x56020c49fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:26.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.320+0000 7f79392f5b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x56020c4a09f0 con 0x56020c493fa0 2024-06-10T02:55:26.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.320+0000 7f79392f5b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56020c385c70 con 0x56020c4996c0 2024-06-10T02:55:26.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.320+0000 7f79392f5b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x56020c4a0b70 con 0x56020c499fe0 2024-06-10T02:55:26.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.320+0000 7f792c80e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56020c493fa0 0x56020c49fce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48540/0 (socket says 172.21.15.52:48540) 2024-06-10T02:55:26.322 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.320+0000 7f792c80e700 1 -- 172.21.15.52:0/1078915740 learned_addr learned my addr 172.21.15.52:0/1078915740 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:55:26.322 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.320+0000 7f79392f5b80 10 client.0 _ll_register_callbacks cb 0x56020c37f9a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T02:55:26.322 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.320+0000 7f792d00f700 1 --2- 172.21.15.52:0/1078915740 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56020c499fe0 0x56020c49d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:26.322 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.320+0000 7f7927fff700 1 --2- 172.21.15.52:0/1078915740 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56020c4996c0 0x56020c49cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:26.322 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.320+0000 7f792c80e700 1 -- 172.21.15.52:0/1078915740 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56020c499fe0 msgr2=0x56020c49d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:55:26.322 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.320+0000 7f792c80e700 1 --2- 172.21.15.52:0/1078915740 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56020c499fe0 0x56020c49d520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:26.322 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.320+0000 7f792c80e700 1 -- 172.21.15.52:0/1078915740 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56020c4996c0 msgr2=0x56020c49cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:55:26.322 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.320+0000 7f792c80e700 1 --2- 172.21.15.52:0/1078915740 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56020c4996c0 0x56020c49cfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:26.322 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.320+0000 7f792c80e700 1 -- 172.21.15.52:0/1078915740 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56020c334950 con 0x56020c493fa0 2024-06-10T02:55:26.322 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.320+0000 7f792c80e700 1 --2- 172.21.15.52:0/1078915740 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56020c493fa0 0x56020c49fce0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f791c010760 tx=0x7f791c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6adec33ce27ab9b server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:55:26.322 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.320+0000 7f7907fff700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-10T02:55:26.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.321+0000 7f7907fff700 1 -- 172.21.15.52:0/1078915740 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f791c023840 con 0x56020c493fa0 2024-06-10T02:55:26.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.321+0000 7f7907fff700 1 -- 172.21.15.52:0/1078915740 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f791c0239e0 con 0x56020c493fa0 2024-06-10T02:55:26.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.321+0000 7f79392f5b80 1 -- 172.21.15.52:0/1078915740 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7920009460 con 0x56020c493fa0 2024-06-10T02:55:26.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.321+0000 7f7907fff700 1 -- 172.21.15.52:0/1078915740 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f791c023be0 con 0x56020c493fa0 2024-06-10T02:55:26.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.321+0000 7f79392f5b80 10 client.15672 fetch_fsmap learned FSMap version 30 2024-06-10T02:55:26.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.321+0000 7f79392f5b80 1 -- 172.21.15.52:0/1078915740 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56020c4ab2c0 con 0x56020c493fa0 2024-06-10T02:55:26.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.321+0000 7f7907fff700 1 -- 172.21.15.52:0/1078915740 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f791c027070 con 0x56020c493fa0 2024-06-10T02:55:26.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.321+0000 7f7907fff700 1 -- 172.21.15.52:0/1078915740 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f791c01e540 con 0x56020c493fa0 2024-06-10T02:55:26.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.321+0000 7f79392f5b80 10 client.15672 fetch_fsmap finished waiting for FSMap version 30 2024-06-10T02:55:26.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.321+0000 7f79392f5b80 10 client.15672 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T02:55:26.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.321+0000 7f79392f5b80 20 client.15672 populate_metadata read hostname 'smithi052' 2024-06-10T02:55:26.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.322+0000 7f79392f5b80 1 --2- 172.21.15.52:0/1078915740 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x56020c4ad900 0x56020c4afcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:26.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.322+0000 7f79392f5b80 4 client.15672 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T02:55:26.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.322+0000 7f79392f5b80 1 -- 172.21.15.52:0/1078915740 --> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f7920009460 con 0x56020c4ad900 2024-06-10T02:55:26.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.322+0000 7f7927fff700 1 --2- 172.21.15.52:0/1078915740 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x56020c4ad900 0x56020c4afcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:26.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.322+0000 7f7907fff700 10 client.15672 ms_handle_connect on v2:172.21.15.188:6838/310444836 2024-06-10T02:55:26.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.322+0000 7f7927fff700 1 --2- 172.21.15.52:0/1078915740 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x56020c4ad900 0x56020c4afcf0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b15459076c0d3c4 server_cookie=74b26b5388808095 in_seq=0 out_seq=0 2024-06-10T02:55:26.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.323+0000 7f7907fff700 1 -- 172.21.15.52:0/1078915740 <== mds.0 v2:172.21.15.188:6838/310444836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x7f7920009460 con 0x56020c4ad900 2024-06-10T02:55:26.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.323+0000 7f7907fff700 10 client.15672 handle_command_reply: tid=0 2024-06-10T02:55:26.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.323+0000 7f79392f5b80 1 client.15672 shutdown 2024-06-10T02:55:26.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.323+0000 7f79392f5b80 1 -- 172.21.15.52:0/1078915740 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56020c493fa0 msgr2=0x56020c49fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:55:26.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.323+0000 7f79392f5b80 1 --2- 172.21.15.52:0/1078915740 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56020c493fa0 0x56020c49fce0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f791c010760 tx=0x7f791c0047c0 comp rx=0 tx=0).stop 2024-06-10T02:55:26.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.323+0000 7f79392f5b80 1 -- 172.21.15.52:0/1078915740 shutdown_connections 2024-06-10T02:55:26.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.323+0000 7f79392f5b80 1 --2- 172.21.15.52:0/1078915740 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x56020c4ad900 0x56020c4afcf0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:26.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.324+0000 7f79392f5b80 1 --2- 172.21.15.52:0/1078915740 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56020c499fe0 0x56020c49d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:26.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.324+0000 7f79392f5b80 1 --2- 172.21.15.52:0/1078915740 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56020c493fa0 0x56020c49fce0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:26.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.324+0000 7f7907fff700 0 client.15672 ms_handle_reset on v2:172.21.15.188:6838/310444836 2024-06-10T02:55:26.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.324+0000 7f79392f5b80 1 --2- 172.21.15.52:0/1078915740 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56020c4996c0 0x56020c49cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:26.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.324+0000 7f79392f5b80 1 -- 172.21.15.52:0/1078915740 >> 172.21.15.52:0/1078915740 conn(0x56020c3bed40 msgr2=0x56020c492a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:55:26.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.324+0000 7f79392f5b80 1 -- 172.21.15.52:0/1078915740 shutdown_connections 2024-06-10T02:55:26.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.324+0000 7f79392f5b80 1 -- 172.21.15.52:0/1078915740 wait complete. 2024-06-10T02:55:26.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.324+0000 7f79392f5b80 20 client.15672 trim_cache size 0 max 16384 2024-06-10T02:55:26.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.345+0000 7f79392f5b80 1 Processor -- start 2024-06-10T02:55:26.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.345+0000 7f79392f5b80 1 -- start start 2024-06-10T02:55:26.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.345+0000 7f79392f5b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56020c493fa0 0x56020c493a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:26.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.345+0000 7f79392f5b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56020c4996c0 0x56020c488710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:26.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.345+0000 7f79392f5b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56020c499fe0 0x56020c4b5550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:26.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.345+0000 7f79392f5b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f79281ac180 con 0x56020c493fa0 2024-06-10T02:55:26.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.345+0000 7f79392f5b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f79281adcc0 con 0x56020c4996c0 2024-06-10T02:55:26.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.345+0000 7f79392f5b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f79281ac850 con 0x56020c499fe0 2024-06-10T02:55:26.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.345+0000 7f792c80e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56020c493fa0 0x56020c493a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:26.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.345+0000 7f792c80e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56020c493fa0 0x56020c493a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48542/0 (socket says 172.21.15.52:48542) 2024-06-10T02:55:26.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.345+0000 7f792c80e700 1 -- 172.21.15.52:0/1011326466 learned_addr learned my addr 172.21.15.52:0/1011326466 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:55:26.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.345+0000 7f792d00f700 1 --2- 172.21.15.52:0/1011326466 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56020c499fe0 0x56020c4b5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:26.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.346+0000 7f7927fff700 1 --2- 172.21.15.52:0/1011326466 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56020c4996c0 0x56020c488710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:26.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.346+0000 7f792d00f700 1 -- 172.21.15.52:0/1011326466 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56020c4996c0 msgr2=0x56020c488710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:55:26.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.346+0000 7f792d00f700 1 --2- 172.21.15.52:0/1011326466 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56020c4996c0 0x56020c488710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:26.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.346+0000 7f792d00f700 1 -- 172.21.15.52:0/1011326466 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56020c493fa0 msgr2=0x56020c493a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:55:26.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.346+0000 7f792d00f700 1 --2- 172.21.15.52:0/1011326466 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56020c493fa0 0x56020c493a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:26.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.346+0000 7f792d00f700 1 -- 172.21.15.52:0/1011326466 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56020c4a04c0 con 0x56020c499fe0 2024-06-10T02:55:26.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.346+0000 7f792c80e700 1 --2- 172.21.15.52:0/1011326466 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56020c493fa0 0x56020c493a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T02:55:26.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.346+0000 7f792d00f700 1 --2- 172.21.15.52:0/1011326466 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56020c499fe0 0x56020c4b5550 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f792000b070 tx=0x7f7920009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4b89aec4ca3c6661 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:55:26.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.346+0000 7f79127fc700 1 -- 172.21.15.52:0/1011326466 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7920022040 con 0x56020c499fe0 2024-06-10T02:55:26.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.346+0000 7f79127fc700 1 -- 172.21.15.52:0/1011326466 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f792000bb70 con 0x56020c499fe0 2024-06-10T02:55:26.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.346+0000 7f79127fc700 1 -- 172.21.15.52:0/1011326466 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f792001c6a0 con 0x56020c499fe0 2024-06-10T02:55:26.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.347+0000 7f79392f5b80 1 -- 172.21.15.52:0/1011326466 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56020c499fe0 msgr2=0x56020c4b5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:55:26.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.347+0000 7f79392f5b80 1 --2- 172.21.15.52:0/1011326466 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56020c499fe0 0x56020c4b5550 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f792000b070 tx=0x7f7920009b80 comp rx=0 tx=0).stop 2024-06-10T02:55:26.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.347+0000 7f79392f5b80 1 -- 172.21.15.52:0/1011326466 shutdown_connections 2024-06-10T02:55:26.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.347+0000 7f79392f5b80 1 --2- 172.21.15.52:0/1011326466 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56020c499fe0 0x56020c4b5550 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:26.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.347+0000 7f79392f5b80 1 --2- 172.21.15.52:0/1011326466 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56020c493fa0 0x56020c493a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:26.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.347+0000 7f79392f5b80 1 --2- 172.21.15.52:0/1011326466 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56020c4996c0 0x56020c488710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:26.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.347+0000 7f79392f5b80 1 -- 172.21.15.52:0/1011326466 >> 172.21.15.52:0/1011326466 conn(0x56020c3bed40 msgr2=0x56020c49aa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:55:26.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.347+0000 7f79392f5b80 1 -- 172.21.15.52:0/1011326466 shutdown_connections 2024-06-10T02:55:26.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.347+0000 7f79392f5b80 1 -- 172.21.15.52:0/1011326466 wait complete. 2024-06-10T02:55:26.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.348+0000 7f79392f5b80 1 -- start start 2024-06-10T02:55:26.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.348+0000 7f79392f5b80 1 Processor -- start 2024-06-10T02:55:26.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.349+0000 7f79392f5b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56020c493fa0 0x56020c4a2e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:26.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.349+0000 7f79392f5b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56020c4996c0 0x56020c4a3340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:26.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.349+0000 7f79392f5b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56020c499fe0 0x56020c4a3880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:26.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.349+0000 7f792c80e700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56020c493fa0 0x56020c4a2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:26.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.349+0000 7f792c80e700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56020c493fa0 0x56020c4a2e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.52:58050/0 (socket says 172.21.15.52:58050) 2024-06-10T02:55:26.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.349+0000 7f7927fff700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56020c4996c0 0x56020c4a3340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:26.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.349+0000 7f792d00f700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56020c499fe0 0x56020c4a3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:26.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.349+0000 7f7927fff700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56020c4996c0 0x56020c4a3340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.52:50546/0 (socket says 172.21.15.52:50546) 2024-06-10T02:55:26.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.349+0000 7f792d00f700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56020c499fe0 0x56020c4a3880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48548/0 (socket says 172.21.15.52:48548) 2024-06-10T02:55:26.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.349+0000 7f79392f5b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f79281ad5f0 con 0x56020c499fe0 2024-06-10T02:55:26.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.349+0000 7f79392f5b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f79281acf20 con 0x56020c4996c0 2024-06-10T02:55:26.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.349+0000 7f79392f5b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f79281aa120 con 0x56020c493fa0 2024-06-10T02:55:26.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.349+0000 7f79392f5b80 10 client.0 _ll_register_callbacks cb 0x56020c37f9a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T02:55:26.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.349+0000 7f792c80e700 1 -- 172.21.15.52:0/3452766751 learned_addr learned my addr 172.21.15.52:0/3452766751 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:55:26.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.350+0000 7f7927fff700 1 -- 172.21.15.52:0/3452766751 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56020c493fa0 msgr2=0x56020c4a2e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:55:26.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.350+0000 7f7927fff700 1 --2- 172.21.15.52:0/3452766751 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56020c493fa0 0x56020c4a2e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:26.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.350+0000 7f7927fff700 1 -- 172.21.15.52:0/3452766751 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56020c499fe0 msgr2=0x56020c4a3880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:55:26.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.350+0000 7f7927fff700 1 --2- 172.21.15.52:0/3452766751 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56020c499fe0 0x56020c4a3880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:26.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.350+0000 7f7927fff700 1 -- 172.21.15.52:0/3452766751 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56020c5186f0 con 0x56020c4996c0 2024-06-10T02:55:26.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.350+0000 7f792d00f700 1 --2- 172.21.15.52:0/3452766751 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56020c499fe0 0x56020c4a3880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T02:55:26.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.350+0000 7f7927fff700 1 --2- 172.21.15.52:0/3452766751 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56020c4996c0 0x56020c4a3340 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f7914000c00 tx=0x7f791400cf20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5642c76d9b5c0d3b server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:55:26.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.350+0000 7f7907fff700 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-06-10T02:55:26.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.350+0000 7f7907fff700 1 -- 172.21.15.52:0/3452766751 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79140b8070 con 0x56020c4996c0 2024-06-10T02:55:26.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.350+0000 7f7907fff700 1 -- 172.21.15.52:0/3452766751 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f7914010720 con 0x56020c4996c0 2024-06-10T02:55:26.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.350+0000 7f79392f5b80 1 -- 172.21.15.52:0/3452766751 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f792000bb70 con 0x56020c4996c0 2024-06-10T02:55:26.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.351+0000 7f7907fff700 1 -- 172.21.15.52:0/3452766751 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f791409fe70 con 0x56020c4996c0 2024-06-10T02:55:26.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.351+0000 7f79392f5b80 10 client.25307 fetch_fsmap learned FSMap version 30 2024-06-10T02:55:26.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.351+0000 7f79392f5b80 1 -- 172.21.15.52:0/3452766751 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56020c519a70 con 0x56020c4996c0 2024-06-10T02:55:26.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.351+0000 7f7907fff700 1 -- 172.21.15.52:0/3452766751 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f79140a4730 con 0x56020c4996c0 2024-06-10T02:55:26.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.351+0000 7f7907fff700 1 -- 172.21.15.52:0/3452766751 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f791409bd20 con 0x56020c4996c0 2024-06-10T02:55:26.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.351+0000 7f79392f5b80 10 client.25307 fetch_fsmap finished waiting for FSMap version 30 2024-06-10T02:55:26.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.351+0000 7f79392f5b80 10 client.25307 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T02:55:26.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.351+0000 7f79392f5b80 20 client.25307 populate_metadata read hostname 'smithi052' 2024-06-10T02:55:26.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.351+0000 7f79392f5b80 1 --2- 172.21.15.52:0/3452766751 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x56020c4ad900 0x56020c51c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:26.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.351+0000 7f79392f5b80 4 client.25307 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-06-10T02:55:26.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.351+0000 7f79392f5b80 1 -- 172.21.15.52:0/3452766751 --> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7f792000bb70 con 0x56020c4ad900 2024-06-10T02:55:26.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.352+0000 7f792c80e700 1 --2- 172.21.15.52:0/3452766751 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x56020c4ad900 0x56020c51c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:26.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.352+0000 7f792c80e700 1 --2- 172.21.15.52:0/3452766751 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x56020c4ad900 0x56020c51c1c0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c6645553b0adc0a0 server_cookie=de02090b373af40a in_seq=0 out_seq=0 2024-06-10T02:55:26.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.352+0000 7f7907fff700 10 client.25307 ms_handle_connect on v2:172.21.15.188:6838/310444836 2024-06-10T02:55:26.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.353+0000 7f7907fff700 1 -- 172.21.15.52:0/3452766751 <== mds.0 v2:172.21.15.188:6838/310444836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7f792000bb70 con 0x56020c4ad900 2024-06-10T02:55:26.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.353+0000 7f7907fff700 10 client.25307 handle_command_reply: tid=0 2024-06-10T02:55:26.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.353+0000 7f79392f5b80 1 client.25307 shutdown 2024-06-10T02:55:26.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.354+0000 7f79392f5b80 1 -- 172.21.15.52:0/3452766751 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56020c4996c0 msgr2=0x56020c4a3340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:55:26.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.354+0000 7f79392f5b80 1 --2- 172.21.15.52:0/3452766751 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56020c4996c0 0x56020c4a3340 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f7914000c00 tx=0x7f791400cf20 comp rx=0 tx=0).stop 2024-06-10T02:55:26.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.354+0000 7f79392f5b80 1 -- 172.21.15.52:0/3452766751 shutdown_connections 2024-06-10T02:55:26.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.354+0000 7f79392f5b80 1 --2- 172.21.15.52:0/3452766751 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x56020c4ad900 0x56020c51c1c0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:26.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.354+0000 7f79392f5b80 1 --2- 172.21.15.52:0/3452766751 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56020c499fe0 0x56020c4a3880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:26.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.354+0000 7f79392f5b80 1 --2- 172.21.15.52:0/3452766751 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56020c493fa0 0x56020c4a2e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:26.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.354+0000 7f79392f5b80 1 --2- 172.21.15.52:0/3452766751 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56020c4996c0 0x56020c4a3340 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:26.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.354+0000 7f79392f5b80 1 -- 172.21.15.52:0/3452766751 >> 172.21.15.52:0/3452766751 conn(0x56020c3bed40 msgr2=0x56020c498160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:55:26.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.354+0000 7f7907fff700 0 client.25307 ms_handle_reset on v2:172.21.15.188:6838/310444836 2024-06-10T02:55:26.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.354+0000 7f79392f5b80 1 -- 172.21.15.52:0/3452766751 shutdown_connections 2024-06-10T02:55:26.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.354+0000 7f79392f5b80 1 -- 172.21.15.52:0/3452766751 wait complete. 2024-06-10T02:55:26.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.354+0000 7f79392f5b80 20 client.25307 trim_cache size 0 max 16384 2024-06-10T02:55:26.357 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-10T02:55:26.357 INFO:teuthology.orchestra.run.smithi052.stdout: "return_code": 0, 2024-06-10T02:55:26.357 INFO:teuthology.orchestra.run.smithi052.stdout: "scrub_tag": "9ea45dce-3a46-4d2a-a88c-03b72edfb34d", 2024-06-10T02:55:26.357 INFO:teuthology.orchestra.run.smithi052.stdout: "mode": "asynchronous" 2024-06-10T02:55:26.357 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-10T02:55:26.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.355+0000 7f792ea72700 1 -- 172.21.15.52:0/1977636037 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f79180a2c20 msgr2=0x7f79180a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:55:26.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.355+0000 7f792ea72700 1 --2- 172.21.15.52:0/1977636037 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f79180a2c20 0x7f79180a50e0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f7928199b10 tx=0x7f791c007480 comp rx=0 tx=0).stop 2024-06-10T02:55:26.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.355+0000 7f792ea72700 1 -- 172.21.15.52:0/1977636037 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f792806ab30 msgr2=0x7f792819ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:55:26.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.355+0000 7f792ea72700 1 --2- 172.21.15.52:0/1977636037 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f792806ab30 0x7f792819ed50 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f791400e9f0 tx=0x7f791400ed00 comp rx=0 tx=0).stop 2024-06-10T02:55:26.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.355+0000 7f792ea72700 1 -- 172.21.15.52:0/1977636037 shutdown_connections 2024-06-10T02:55:26.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.355+0000 7f792ea72700 1 --2- 172.21.15.52:0/1977636037 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7928108500 0x7f79281989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:26.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.355+0000 7f792ea72700 1 --2- 172.21.15.52:0/1977636037 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f79180a2c20 0x7f79180a50e0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:26.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.355+0000 7f792ea72700 1 --2- 172.21.15.52:0/1977636037 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f792806a170 0x7f792819e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:26.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.355+0000 7f792ea72700 1 --2- 172.21.15.52:0/1977636037 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f792806ab30 0x7f792819ed50 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:26.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.355+0000 7f792ea72700 1 -- 172.21.15.52:0/1977636037 >> 172.21.15.52:0/1977636037 conn(0x7f7928078340 msgr2=0x7f79280fff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:55:26.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.355+0000 7f792ea72700 1 -- 172.21.15.52:0/1977636037 shutdown_connections 2024-06-10T02:55:26.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:26.355+0000 7f792ea72700 1 -- 172.21.15.52:0/1977636037 wait complete. 2024-06-10T02:55:26.453 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:55:26 smithi188 ceph-mon[36599]: pgmap v471: 129 pgs: 129 active+clean; 667 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1023 B/s wr, 9 op/s 2024-06-10T02:55:26.638 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:55:26 smithi120 ceph-mon[36634]: pgmap v471: 129 pgs: 129 active+clean; 667 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1023 B/s wr, 9 op/s 2024-06-10T02:55:27.180 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-10T02:55:27.562 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:55:28.453 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:55:28 smithi188 ceph-mon[36599]: scrub queued for path: / 2024-06-10T02:55:28.453 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:55:28 smithi188 ceph-mon[36599]: scrub summary: idle+waiting paths [/] 2024-06-10T02:55:28.453 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:55:28 smithi188 ceph-mon[36599]: scrub summary: active paths [/] 2024-06-10T02:55:28.453 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:55:28 smithi188 ceph-mon[36599]: scrub summary: idle+waiting paths [/] 2024-06-10T02:55:28.453 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:55:28 smithi188 ceph-mon[36599]: pgmap v472: 129 pgs: 129 active+clean; 667 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 1023 B/s wr, 12 op/s 2024-06-10T02:55:28.476 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:55:28 smithi052 ceph-mon[32362]: scrub queued for path: / 2024-06-10T02:55:28.476 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:55:28 smithi052 ceph-mon[32362]: scrub summary: idle+waiting paths [/] 2024-06-10T02:55:28.476 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:55:28 smithi052 ceph-mon[32362]: scrub summary: active paths [/] 2024-06-10T02:55:28.476 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:55:28 smithi052 ceph-mon[32362]: scrub summary: idle+waiting paths [/] 2024-06-10T02:55:28.477 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:55:28 smithi052 ceph-mon[32362]: pgmap v472: 129 pgs: 129 active+clean; 667 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 1023 B/s wr, 12 op/s 2024-06-10T02:55:28.638 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:55:28 smithi120 ceph-mon[36634]: scrub queued for path: / 2024-06-10T02:55:28.638 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:55:28 smithi120 ceph-mon[36634]: scrub summary: idle+waiting paths [/] 2024-06-10T02:55:28.638 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:55:28 smithi120 ceph-mon[36634]: scrub summary: active paths [/] 2024-06-10T02:55:28.638 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:55:28 smithi120 ceph-mon[36634]: scrub summary: idle+waiting paths [/] 2024-06-10T02:55:28.638 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:55:28 smithi120 ceph-mon[36634]: pgmap v472: 129 pgs: 129 active+clean; 667 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 1023 B/s wr, 12 op/s 2024-06-10T02:55:29.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.440+0000 7fad8407a700 1 -- 172.21.15.52:0/4217319979 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fad7c102ee0 msgr2=0x7fad7c10f390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:55:29.441 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.440+0000 7fad8407a700 1 --2- 172.21.15.52:0/4217319979 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fad7c102ee0 0x7fad7c10f390 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fad74009f90 tx=0x7fad7400b330 comp rx=0 tx=0).stop 2024-06-10T02:55:29.441 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.440+0000 7fad8407a700 1 -- 172.21.15.52:0/4217319979 shutdown_connections 2024-06-10T02:55:29.441 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.440+0000 7fad8407a700 1 --2- 172.21.15.52:0/4217319979 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fad7c102ee0 0x7fad7c10f390 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:29.441 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.440+0000 7fad8407a700 1 --2- 172.21.15.52:0/4217319979 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fad7c1085c0 0x7fad7c1089a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:29.441 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.440+0000 7fad8407a700 1 --2- 172.21.15.52:0/4217319979 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fad7c102520 0x7fad7c1029a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:29.441 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.440+0000 7fad8407a700 1 -- 172.21.15.52:0/4217319979 >> 172.21.15.52:0/4217319979 conn(0x7fad7c079680 msgr2=0x7fad7c079a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:55:29.441 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.440+0000 7fad8407a700 1 -- 172.21.15.52:0/4217319979 shutdown_connections 2024-06-10T02:55:29.441 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.440+0000 7fad8407a700 1 -- 172.21.15.52:0/4217319979 wait complete. 2024-06-10T02:55:29.442 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.442+0000 7fad8407a700 1 Processor -- start 2024-06-10T02:55:29.442 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.442+0000 7fad8407a700 1 -- start start 2024-06-10T02:55:29.442 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.443+0000 7fad8407a700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fad7c102520 0x7fad7c19d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:29.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.443+0000 7fad8407a700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fad7c102ee0 0x7fad7c19da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:29.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.443+0000 7fad81e16700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fad7c102520 0x7fad7c19d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:29.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.443+0000 7fad81e16700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fad7c102520 0x7fad7c19d530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48562/0 (socket says 172.21.15.52:48562) 2024-06-10T02:55:29.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.443+0000 7fad8407a700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fad7c1085c0 0x7fad7c1965c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:29.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.443+0000 7fad81e16700 1 -- 172.21.15.52:0/3145165677 learned_addr learned my addr 172.21.15.52:0/3145165677 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:55:29.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.443+0000 7fad8407a700 1 -- 172.21.15.52:0/3145165677 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fad7c110660 con 0x7fad7c102520 2024-06-10T02:55:29.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.443+0000 7fad8407a700 1 -- 172.21.15.52:0/3145165677 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fad7c1104e0 con 0x7fad7c1085c0 2024-06-10T02:55:29.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.443+0000 7fad8407a700 1 -- 172.21.15.52:0/3145165677 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fad7c1107e0 con 0x7fad7c102ee0 2024-06-10T02:55:29.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.443+0000 7fad82617700 1 --2- 172.21.15.52:0/3145165677 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fad7c1085c0 0x7fad7c1965c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:29.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.443+0000 7fad81615700 1 --2- 172.21.15.52:0/3145165677 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fad7c102ee0 0x7fad7c19da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:29.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.444+0000 7fad81e16700 1 -- 172.21.15.52:0/3145165677 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fad7c102ee0 msgr2=0x7fad7c19da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:55:29.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.444+0000 7fad81e16700 1 --2- 172.21.15.52:0/3145165677 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fad7c102ee0 0x7fad7c19da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:29.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.444+0000 7fad81e16700 1 -- 172.21.15.52:0/3145165677 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fad7c1085c0 msgr2=0x7fad7c1965c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:55:29.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.444+0000 7fad81e16700 1 --2- 172.21.15.52:0/3145165677 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fad7c1085c0 0x7fad7c1965c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:29.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.444+0000 7fad81e16700 1 -- 172.21.15.52:0/3145165677 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad7c196d90 con 0x7fad7c102520 2024-06-10T02:55:29.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.444+0000 7fad82617700 1 --2- 172.21.15.52:0/3145165677 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fad7c1085c0 0x7fad7c1965c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T02:55:29.445 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.444+0000 7fad81e16700 1 --2- 172.21.15.52:0/3145165677 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fad7c102520 0x7fad7c19d530 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fad78009aa0 tx=0x7fad78009db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:55:29.445 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.444+0000 7fad72ffd700 1 -- 172.21.15.52:0/3145165677 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad7801b020 con 0x7fad7c102520 2024-06-10T02:55:29.445 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.444+0000 7fad72ffd700 1 -- 172.21.15.52:0/3145165677 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad78004d10 con 0x7fad7c102520 2024-06-10T02:55:29.445 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.445+0000 7fad72ffd700 1 -- 172.21.15.52:0/3145165677 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad78005610 con 0x7fad7c102520 2024-06-10T02:55:29.445 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.445+0000 7fad8407a700 1 -- 172.21.15.52:0/3145165677 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad7c197080 con 0x7fad7c102520 2024-06-10T02:55:29.445 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.446+0000 7fad8407a700 1 -- 172.21.15.52:0/3145165677 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad7c19b060 con 0x7fad7c102520 2024-06-10T02:55:29.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.449+0000 7fad72ffd700 1 -- 172.21.15.52:0/3145165677 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fad7801e070 con 0x7fad7c102520 2024-06-10T02:55:29.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.450+0000 7fad72ffd700 1 --2- 172.21.15.52:0/3145165677 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fad680a2b80 0x7fad680a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:29.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.450+0000 7fad72ffd700 1 -- 172.21.15.52:0/3145165677 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7fad780cebc0 con 0x7fad7c102520 2024-06-10T02:55:29.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.450+0000 7fad81615700 1 --2- 172.21.15.52:0/3145165677 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fad680a2b80 0x7fad680a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:29.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.452+0000 7fad8e8fdb80 1 Processor -- start 2024-06-10T02:55:29.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.452+0000 7fad81615700 1 --2- 172.21.15.52:0/3145165677 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fad680a2b80 0x7fad680a5040 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fad6c009750 tx=0x7fad6c009310 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:55:29.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.452+0000 7fad8e8fdb80 1 -- start start 2024-06-10T02:55:29.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.452+0000 7fad8e8fdb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5559f63f5f60 0x5559f63f6340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:29.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.452+0000 7fad8e8fdb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5559f63f6880 0x5559f63f0260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:29.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.453+0000 7fad8e8fdb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5559f63f07a0 0x5559f63f0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:29.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.453+0000 7fad81e16700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5559f63f5f60 0x5559f63f6340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:29.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.453+0000 7fad81615700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5559f63f6880 0x5559f63f0260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:29.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.453+0000 7fad81e16700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5559f63f5f60 0x5559f63f6340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.52:58060/0 (socket says 172.21.15.52:58060) 2024-06-10T02:55:29.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.453+0000 7fad81615700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5559f63f6880 0x5559f63f0260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48564/0 (socket says 172.21.15.52:48564) 2024-06-10T02:55:29.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.453+0000 7fad81e16700 1 -- 172.21.15.52:0/2793238399 learned_addr learned my addr 172.21.15.52:0/2793238399 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:55:29.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.453+0000 7fad8e8fdb80 1 -- 172.21.15.52:0/2793238399 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5559f62e2f60 con 0x5559f63f6880 2024-06-10T02:55:29.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.453+0000 7fad8e8fdb80 1 -- 172.21.15.52:0/2793238399 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5559f6341850 con 0x5559f63f07a0 2024-06-10T02:55:29.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.453+0000 7fad8e8fdb80 1 -- 172.21.15.52:0/2793238399 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5559f62a9460 con 0x5559f63f5f60 2024-06-10T02:55:29.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.453+0000 7fad81e16700 1 -- 172.21.15.52:0/2793238399 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5559f63f07a0 msgr2=0x5559f63f0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:55:29.454 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.453+0000 7fad81e16700 1 --2- 172.21.15.52:0/2793238399 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5559f63f07a0 0x5559f63f0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:29.454 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.453+0000 7fad81e16700 1 -- 172.21.15.52:0/2793238399 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5559f63f6880 msgr2=0x5559f63f0260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:55:29.454 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.453+0000 7fad81e16700 1 --2- 172.21.15.52:0/2793238399 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5559f63f6880 0x5559f63f0260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:29.454 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.453+0000 7fad81e16700 1 -- 172.21.15.52:0/2793238399 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5559f62908d0 con 0x5559f63f5f60 2024-06-10T02:55:29.454 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.453+0000 7fad81e16700 1 --2- 172.21.15.52:0/2793238399 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5559f63f5f60 0x5559f63f6340 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fad780974d0 tx=0x7fad7800f5f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=92d8d795690233e3 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:55:29.454 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.454+0000 7fad67fff700 1 -- 172.21.15.52:0/2793238399 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad7809f420 con 0x5559f63f5f60 2024-06-10T02:55:29.454 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.454+0000 7fad67fff700 1 -- 172.21.15.52:0/2793238399 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad780a3ce0 con 0x5559f63f5f60 2024-06-10T02:55:29.454 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.454+0000 7fad67fff700 1 -- 172.21.15.52:0/2793238399 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad7809b530 con 0x5559f63f5f60 2024-06-10T02:55:29.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.454+0000 7fad8e8fdb80 1 -- 172.21.15.52:0/2793238399 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5559f63f5f60 msgr2=0x5559f63f6340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:55:29.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.455+0000 7fad8e8fdb80 1 --2- 172.21.15.52:0/2793238399 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5559f63f5f60 0x5559f63f6340 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fad780974d0 tx=0x7fad7800f5f0 comp rx=0 tx=0).stop 2024-06-10T02:55:29.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.455+0000 7fad8e8fdb80 1 -- 172.21.15.52:0/2793238399 shutdown_connections 2024-06-10T02:55:29.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.455+0000 7fad8e8fdb80 1 --2- 172.21.15.52:0/2793238399 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5559f63f07a0 0x5559f63f0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:29.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.455+0000 7fad8e8fdb80 1 --2- 172.21.15.52:0/2793238399 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5559f63f5f60 0x5559f63f6340 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:29.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.455+0000 7fad8e8fdb80 1 --2- 172.21.15.52:0/2793238399 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5559f63f6880 0x5559f63f0260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:29.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.455+0000 7fad8e8fdb80 1 -- 172.21.15.52:0/2793238399 >> 172.21.15.52:0/2793238399 conn(0x5559f631d080 msgr2=0x5559f63eb220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:55:29.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.455+0000 7fad8e8fdb80 1 -- 172.21.15.52:0/2793238399 shutdown_connections 2024-06-10T02:55:29.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.455+0000 7fad8e8fdb80 1 -- 172.21.15.52:0/2793238399 wait complete. 2024-06-10T02:55:29.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.456+0000 7fad8e8fdb80 1 -- start start 2024-06-10T02:55:29.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.456+0000 7fad8e8fdb80 1 Processor -- start 2024-06-10T02:55:29.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.456+0000 7fad8e8fdb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5559f63f07a0 0x5559f63fc680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:29.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.456+0000 7fad8e8fdb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5559f63f5f60 0x5559f63f9980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:29.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.456+0000 7fad8e8fdb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5559f63f6880 0x5559f63f9ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:29.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.456+0000 7fad81615700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5559f63f5f60 0x5559f63f9980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:29.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.457+0000 7fad81615700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5559f63f5f60 0x5559f63f9980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48576/0 (socket says 172.21.15.52:48576) 2024-06-10T02:55:29.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.457+0000 7fad81615700 1 -- 172.21.15.52:0/235909560 learned_addr learned my addr 172.21.15.52:0/235909560 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:55:29.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.457+0000 7fad82617700 1 --2- 172.21.15.52:0/235909560 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5559f63f6880 0x5559f63f9ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:29.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.457+0000 7fad8e8fdb80 1 -- 172.21.15.52:0/235909560 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5559f62e1fc0 con 0x5559f63f5f60 2024-06-10T02:55:29.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.457+0000 7fad8e8fdb80 1 -- 172.21.15.52:0/235909560 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5559f633d810 con 0x5559f63f07a0 2024-06-10T02:55:29.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.457+0000 7fad8e8fdb80 1 -- 172.21.15.52:0/235909560 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5559f63fd200 con 0x5559f63f6880 2024-06-10T02:55:29.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.457+0000 7fad81615700 1 -- 172.21.15.52:0/235909560 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5559f63f6880 msgr2=0x5559f63f9ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:55:29.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.457+0000 7fad81615700 1 --2- 172.21.15.52:0/235909560 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5559f63f6880 0x5559f63f9ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:29.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.457+0000 7fad81615700 1 -- 172.21.15.52:0/235909560 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5559f63f07a0 msgr2=0x5559f63fc680 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-10T02:55:29.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.457+0000 7fad81615700 1 --2- 172.21.15.52:0/235909560 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5559f63f07a0 0x5559f63fc680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:29.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.457+0000 7fad81615700 1 -- 172.21.15.52:0/235909560 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5559f6307750 con 0x5559f63f5f60 2024-06-10T02:55:29.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.457+0000 7fad81615700 1 --2- 172.21.15.52:0/235909560 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5559f63f5f60 0x5559f63f9980 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fad6c012920 tx=0x7fad6c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dcc2dca9875847e server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:55:29.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.457+0000 7fad8e8fdb80 10 client.0 _ll_register_callbacks cb 0x5559f62d75b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T02:55:29.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.457+0000 7fad657fa700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-10T02:55:29.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.457+0000 7fad8e8fdb80 1 -- 172.21.15.52:0/235909560 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fad780a3ce0 con 0x5559f63f5f60 2024-06-10T02:55:29.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.458+0000 7fad657fa700 1 -- 172.21.15.52:0/235909560 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad6c020840 con 0x5559f63f5f60 2024-06-10T02:55:29.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.458+0000 7fad657fa700 1 -- 172.21.15.52:0/235909560 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7fad6c0209e0 con 0x5559f63f5f60 2024-06-10T02:55:29.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.458+0000 7fad657fa700 1 -- 172.21.15.52:0/235909560 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fad6c020be0 con 0x5559f63f5f60 2024-06-10T02:55:29.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.458+0000 7fad8e8fdb80 10 client.15690 fetch_fsmap learned FSMap version 30 2024-06-10T02:55:29.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.458+0000 7fad8e8fdb80 1 -- 172.21.15.52:0/235909560 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({fsmap=30}) -- 0x5559f6407900 con 0x5559f63f5f60 2024-06-10T02:55:29.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.459+0000 7fad657fa700 1 -- 172.21.15.52:0/235909560 <== mon.0 v2:172.21.15.52:3300/0 4 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fad6c029470 con 0x5559f63f5f60 2024-06-10T02:55:29.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.459+0000 7fad8e8fdb80 10 client.15690 fetch_fsmap finished waiting for FSMap version 30 2024-06-10T02:55:29.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.459+0000 7fad8e8fdb80 10 client.15690 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T02:55:29.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.459+0000 7fad8e8fdb80 20 client.15690 populate_metadata read hostname 'smithi052' 2024-06-10T02:55:29.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.459+0000 7fad8e8fdb80 1 --2- 172.21.15.52:0/235909560 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x5559f640a000 0x5559f640c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:29.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.459+0000 7fad8e8fdb80 4 client.15690 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T02:55:29.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.459+0000 7fad8e8fdb80 1 -- 172.21.15.52:0/235909560 --> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5559f634ed50 con 0x5559f640a000 2024-06-10T02:55:29.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.460+0000 7fad81e16700 1 --2- 172.21.15.52:0/235909560 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x5559f640a000 0x5559f640c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:29.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.460+0000 7fad657fa700 10 client.15690 ms_handle_connect on v2:172.21.15.188:6838/310444836 2024-06-10T02:55:29.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.460+0000 7fad81e16700 1 --2- 172.21.15.52:0/235909560 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x5559f640a000 0x5559f640c4c0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ac116c0ea77b0a9 server_cookie=e05c5e771f0debac in_seq=0 out_seq=0 2024-06-10T02:55:29.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.461+0000 7fad657fa700 1 -- 172.21.15.52:0/235909560 <== mds.0 v2:172.21.15.188:6838/310444836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5559f634ed50 con 0x5559f640a000 2024-06-10T02:55:29.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.461+0000 7fad657fa700 10 client.15690 handle_command_reply: tid=0 2024-06-10T02:55:29.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.461+0000 7fad8e8fdb80 1 client.15690 shutdown 2024-06-10T02:55:29.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.461+0000 7fad8e8fdb80 1 -- 172.21.15.52:0/235909560 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5559f63f5f60 msgr2=0x5559f63f9980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:55:29.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.461+0000 7fad8e8fdb80 1 --2- 172.21.15.52:0/235909560 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5559f63f5f60 0x5559f63f9980 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fad6c012920 tx=0x7fad6c0047c0 comp rx=0 tx=0).stop 2024-06-10T02:55:29.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.461+0000 7fad8e8fdb80 1 -- 172.21.15.52:0/235909560 shutdown_connections 2024-06-10T02:55:29.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.461+0000 7fad8e8fdb80 1 --2- 172.21.15.52:0/235909560 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x5559f640a000 0x5559f640c4c0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:29.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.461+0000 7fad8e8fdb80 1 --2- 172.21.15.52:0/235909560 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5559f63f6880 0x5559f63f9ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:29.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.461+0000 7fad8e8fdb80 1 --2- 172.21.15.52:0/235909560 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5559f63f07a0 0x5559f63fc680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:29.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.461+0000 7fad8e8fdb80 1 --2- 172.21.15.52:0/235909560 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5559f63f5f60 0x5559f63f9980 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:29.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.461+0000 7fad8e8fdb80 1 -- 172.21.15.52:0/235909560 >> 172.21.15.52:0/235909560 conn(0x5559f631d080 msgr2=0x5559f63ea700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:55:29.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.461+0000 7fad657fa700 0 client.15690 ms_handle_reset on v2:172.21.15.188:6838/310444836 2024-06-10T02:55:29.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.461+0000 7fad8e8fdb80 1 -- 172.21.15.52:0/235909560 shutdown_connections 2024-06-10T02:55:29.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.462+0000 7fad8e8fdb80 1 -- 172.21.15.52:0/235909560 wait complete. 2024-06-10T02:55:29.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.462+0000 7fad8e8fdb80 20 client.15690 trim_cache size 0 max 16384 2024-06-10T02:55:29.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.502+0000 7fad8e8fdb80 1 Processor -- start 2024-06-10T02:55:29.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.502+0000 7fad8e8fdb80 1 -- start start 2024-06-10T02:55:29.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.502+0000 7fad8e8fdb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5559f63f07a0 0x5559f63f7370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:29.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.502+0000 7fad8e8fdb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5559f63f5f60 0x5559f63fc1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:29.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.502+0000 7fad8e8fdb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5559f63f6880 0x5559f63eae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:29.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.502+0000 7fad8e8fdb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fad7c1bcd40 con 0x5559f63f6880 2024-06-10T02:55:29.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.502+0000 7fad8e8fdb80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fad7c1be880 con 0x5559f63f07a0 2024-06-10T02:55:29.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.502+0000 7fad8e8fdb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fad7c1bd410 con 0x5559f63f5f60 2024-06-10T02:55:29.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.502+0000 7fad81e16700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5559f63f07a0 0x5559f63f7370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:29.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.502+0000 7fad81e16700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5559f63f07a0 0x5559f63f7370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.52:50592/0 (socket says 172.21.15.52:50592) 2024-06-10T02:55:29.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.502+0000 7fad81e16700 1 -- 172.21.15.52:0/3824732760 learned_addr learned my addr 172.21.15.52:0/3824732760 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:55:29.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.502+0000 7fad82617700 1 --2- 172.21.15.52:0/3824732760 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5559f63f6880 0x5559f63eae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:29.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.502+0000 7fad81615700 1 --2- 172.21.15.52:0/3824732760 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5559f63f5f60 0x5559f63fc1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:29.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.502+0000 7fad81e16700 1 -- 172.21.15.52:0/3824732760 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5559f63f5f60 msgr2=0x5559f63fc1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:55:29.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.502+0000 7fad81e16700 1 --2- 172.21.15.52:0/3824732760 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5559f63f5f60 0x5559f63fc1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:29.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.503+0000 7fad81e16700 1 -- 172.21.15.52:0/3824732760 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5559f63f6880 msgr2=0x5559f63eae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:55:29.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.503+0000 7fad81e16700 1 --2- 172.21.15.52:0/3824732760 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5559f63f6880 0x5559f63eae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:29.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.503+0000 7fad81e16700 1 -- 172.21.15.52:0/3824732760 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5559f63ec040 con 0x5559f63f07a0 2024-06-10T02:55:29.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.503+0000 7fad82617700 1 --2- 172.21.15.52:0/3824732760 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5559f63f6880 0x5559f63eae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T02:55:29.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.503+0000 7fad81e16700 1 --2- 172.21.15.52:0/3824732760 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5559f63f07a0 0x5559f63f7370 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fad7809ad70 tx=0x7fad7809b1f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7939c9b21cec69b1 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:55:29.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.503+0000 7fad677fe700 1 -- 172.21.15.52:0/3824732760 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad780b3070 con 0x5559f63f07a0 2024-06-10T02:55:29.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.503+0000 7fad677fe700 1 -- 172.21.15.52:0/3824732760 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad78099370 con 0x5559f63f07a0 2024-06-10T02:55:29.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.503+0000 7fad677fe700 1 -- 172.21.15.52:0/3824732760 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad780a3850 con 0x5559f63f07a0 2024-06-10T02:55:29.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.504+0000 7fad8e8fdb80 1 -- 172.21.15.52:0/3824732760 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5559f63f07a0 msgr2=0x5559f63f7370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:55:29.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.504+0000 7fad8e8fdb80 1 --2- 172.21.15.52:0/3824732760 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5559f63f07a0 0x5559f63f7370 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fad7809ad70 tx=0x7fad7809b1f0 comp rx=0 tx=0).stop 2024-06-10T02:55:29.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.504+0000 7fad8e8fdb80 1 -- 172.21.15.52:0/3824732760 shutdown_connections 2024-06-10T02:55:29.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.504+0000 7fad8e8fdb80 1 --2- 172.21.15.52:0/3824732760 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5559f63f6880 0x5559f63eae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:29.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.504+0000 7fad8e8fdb80 1 --2- 172.21.15.52:0/3824732760 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5559f63f07a0 0x5559f63f7370 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:29.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.504+0000 7fad8e8fdb80 1 --2- 172.21.15.52:0/3824732760 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5559f63f5f60 0x5559f63fc1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:29.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.504+0000 7fad8e8fdb80 1 -- 172.21.15.52:0/3824732760 >> 172.21.15.52:0/3824732760 conn(0x5559f631d080 msgr2=0x5559f63f6c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:55:29.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.505+0000 7fad8e8fdb80 1 -- 172.21.15.52:0/3824732760 shutdown_connections 2024-06-10T02:55:29.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.505+0000 7fad8e8fdb80 1 -- 172.21.15.52:0/3824732760 wait complete. 2024-06-10T02:55:29.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.505+0000 7fad8e8fdb80 1 -- start start 2024-06-10T02:55:29.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.505+0000 7fad8e8fdb80 1 Processor -- start 2024-06-10T02:55:29.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.506+0000 7fad8e8fdb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5559f63f07a0 0x5559f6411ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:29.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.507+0000 7fad81e16700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5559f63f07a0 0x5559f6411ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:29.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.507+0000 7fad81e16700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5559f63f07a0 0x5559f6411ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48602/0 (socket says 172.21.15.52:48602) 2024-06-10T02:55:29.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.507+0000 7fad81e16700 1 -- 172.21.15.52:0/1924288725 learned_addr learned my addr 172.21.15.52:0/1924288725 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:55:29.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.507+0000 7fad8e8fdb80 1 --2- 172.21.15.52:0/1924288725 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5559f63f5f60 0x5559f64120e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:29.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.507+0000 7fad8e8fdb80 1 --2- 172.21.15.52:0/1924288725 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5559f63f6880 0x5559f6412620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:29.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.507+0000 7fad8e8fdb80 1 -- 172.21.15.52:0/1924288725 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fad7c1be1b0 con 0x5559f63f07a0 2024-06-10T02:55:29.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.507+0000 7fad8e8fdb80 1 -- 172.21.15.52:0/1924288725 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fad7c1bdae0 con 0x5559f63f5f60 2024-06-10T02:55:29.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.507+0000 7fad8e8fdb80 1 -- 172.21.15.52:0/1924288725 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fad7c1bace0 con 0x5559f63f6880 2024-06-10T02:55:29.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.507+0000 7fad81615700 1 --2- 172.21.15.52:0/1924288725 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5559f63f5f60 0x5559f64120e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:29.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.507+0000 7fad82617700 1 --2- 172.21.15.52:0/1924288725 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5559f63f6880 0x5559f6412620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:29.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.507+0000 7fad81e16700 1 -- 172.21.15.52:0/1924288725 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5559f63f6880 msgr2=0x5559f6412620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:55:29.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.507+0000 7fad81e16700 1 --2- 172.21.15.52:0/1924288725 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5559f63f6880 0x5559f6412620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:29.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.507+0000 7fad8e8fdb80 10 client.0 _ll_register_callbacks cb 0x5559f62d75b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T02:55:29.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.507+0000 7fad81e16700 1 -- 172.21.15.52:0/1924288725 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5559f63f5f60 msgr2=0x5559f64120e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:55:29.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.507+0000 7fad81e16700 1 --2- 172.21.15.52:0/1924288725 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5559f63f5f60 0x5559f64120e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:29.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.507+0000 7fad81e16700 1 -- 172.21.15.52:0/1924288725 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5559f6473f50 con 0x5559f63f07a0 2024-06-10T02:55:29.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.507+0000 7fad82617700 1 --2- 172.21.15.52:0/1924288725 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5559f63f6880 0x5559f6412620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T02:55:29.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.508+0000 7fad8e8fdb80 1 -- 172.21.15.52:0/1924288725 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fad78099370 con 0x5559f63f07a0 2024-06-10T02:55:29.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.508+0000 7fad81e16700 1 --2- 172.21.15.52:0/1924288725 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5559f63f07a0 0x5559f6411ba0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fad78097a40 tx=0x7fad7800f5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ea23cf4e0175ee5 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:55:29.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.508+0000 7fad657fa700 10 client.15696 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-10T02:55:29.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.508+0000 7fad657fa700 1 -- 172.21.15.52:0/1924288725 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad7807d020 con 0x5559f63f07a0 2024-06-10T02:55:29.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.508+0000 7fad657fa700 1 -- 172.21.15.52:0/1924288725 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7fad7809b2d0 con 0x5559f63f07a0 2024-06-10T02:55:29.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.508+0000 7fad657fa700 1 -- 172.21.15.52:0/1924288725 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fad780b3070 con 0x5559f63f07a0 2024-06-10T02:55:29.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.508+0000 7fad8e8fdb80 10 client.15696 fetch_fsmap learned FSMap version 30 2024-06-10T02:55:29.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.508+0000 7fad8e8fdb80 1 -- 172.21.15.52:0/1924288725 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({fsmap=30}) -- 0x5559f6475110 con 0x5559f63f07a0 2024-06-10T02:55:29.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.509+0000 7fad657fa700 1 -- 172.21.15.52:0/1924288725 <== mon.0 v2:172.21.15.52:3300/0 4 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fad6c011d20 con 0x5559f63f07a0 2024-06-10T02:55:29.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.509+0000 7fad8e8fdb80 10 client.15696 fetch_fsmap finished waiting for FSMap version 30 2024-06-10T02:55:29.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.509+0000 7fad8e8fdb80 10 client.15696 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T02:55:29.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.509+0000 7fad8e8fdb80 20 client.15696 populate_metadata read hostname 'smithi052' 2024-06-10T02:55:29.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.509+0000 7fad8e8fdb80 1 --2- 172.21.15.52:0/1924288725 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x5559f64752e0 0x5559f64776d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:29.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.509+0000 7fad8e8fdb80 4 client.15696 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-10T02:55:29.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.509+0000 7fad8e8fdb80 1 -- 172.21.15.52:0/1924288725 --> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5559f63492d0 con 0x5559f64752e0 2024-06-10T02:55:29.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.509+0000 7fad81615700 1 --2- 172.21.15.52:0/1924288725 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x5559f64752e0 0x5559f64776d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:29.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.510+0000 7fad81615700 1 --2- 172.21.15.52:0/1924288725 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x5559f64752e0 0x5559f64776d0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8cc3180185b010dd server_cookie=224e776d91cc34a4 in_seq=0 out_seq=0 2024-06-10T02:55:29.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.510+0000 7fad657fa700 10 client.15696 ms_handle_connect on v2:172.21.15.188:6838/310444836 2024-06-10T02:55:29.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.510+0000 7fad657fa700 1 -- 172.21.15.52:0/1924288725 <== mds.0 v2:172.21.15.188:6838/310444836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5559f63492d0 con 0x5559f64752e0 2024-06-10T02:55:29.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.510+0000 7fad657fa700 10 client.15696 handle_command_reply: tid=0 2024-06-10T02:55:29.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.510+0000 7fad8e8fdb80 1 client.15696 shutdown 2024-06-10T02:55:29.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.510+0000 7fad8e8fdb80 1 -- 172.21.15.52:0/1924288725 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5559f63f07a0 msgr2=0x5559f6411ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:55:29.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.510+0000 7fad8e8fdb80 1 --2- 172.21.15.52:0/1924288725 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5559f63f07a0 0x5559f6411ba0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fad78097a40 tx=0x7fad7800f5f0 comp rx=0 tx=0).stop 2024-06-10T02:55:29.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.510+0000 7fad8e8fdb80 1 -- 172.21.15.52:0/1924288725 shutdown_connections 2024-06-10T02:55:29.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.510+0000 7fad8e8fdb80 1 --2- 172.21.15.52:0/1924288725 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x5559f64752e0 0x5559f64776d0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:29.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.510+0000 7fad8e8fdb80 1 --2- 172.21.15.52:0/1924288725 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5559f63f6880 0x5559f6412620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:29.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.510+0000 7fad8e8fdb80 1 --2- 172.21.15.52:0/1924288725 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5559f63f07a0 0x5559f6411ba0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:29.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.510+0000 7fad8e8fdb80 1 --2- 172.21.15.52:0/1924288725 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5559f63f5f60 0x5559f64120e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:29.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.510+0000 7fad8e8fdb80 1 -- 172.21.15.52:0/1924288725 >> 172.21.15.52:0/1924288725 conn(0x5559f631d080 msgr2=0x5559f63f4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:55:29.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.510+0000 7fad657fa700 0 client.15696 ms_handle_reset on v2:172.21.15.188:6838/310444836 2024-06-10T02:55:29.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.510+0000 7fad8e8fdb80 1 -- 172.21.15.52:0/1924288725 shutdown_connections 2024-06-10T02:55:29.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.511+0000 7fad8e8fdb80 1 -- 172.21.15.52:0/1924288725 wait complete. 2024-06-10T02:55:29.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.511+0000 7fad8e8fdb80 20 client.15696 trim_cache size 0 max 16384 2024-06-10T02:55:29.512 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-10T02:55:29.513 INFO:teuthology.orchestra.run.smithi052.stdout: "status": "IDLE (waiting for more scrubs)", 2024-06-10T02:55:29.513 INFO:teuthology.orchestra.run.smithi052.stdout: "scrubs": { 2024-06-10T02:55:29.513 INFO:teuthology.orchestra.run.smithi052.stdout: "9ea45dce-3a46-4d2a-a88c-03b72edfb34d": { 2024-06-10T02:55:29.513 INFO:teuthology.orchestra.run.smithi052.stdout: "path": "/", 2024-06-10T02:55:29.513 INFO:teuthology.orchestra.run.smithi052.stdout: "tag": "9ea45dce-3a46-4d2a-a88c-03b72edfb34d", 2024-06-10T02:55:29.513 INFO:teuthology.orchestra.run.smithi052.stdout: "options": "recursive,force" 2024-06-10T02:55:29.513 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T02:55:29.513 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T02:55:29.513 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-10T02:55:29.513 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.511+0000 7fad8407a700 1 -- 172.21.15.52:0/3145165677 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fad680a2b80 msgr2=0x7fad680a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:55:29.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.511+0000 7fad8407a700 1 --2- 172.21.15.52:0/3145165677 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fad680a2b80 0x7fad680a5040 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fad6c009750 tx=0x7fad6c009310 comp rx=0 tx=0).stop 2024-06-10T02:55:29.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.511+0000 7fad8407a700 1 -- 172.21.15.52:0/3145165677 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fad7c102520 msgr2=0x7fad7c19d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:55:29.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.511+0000 7fad8407a700 1 --2- 172.21.15.52:0/3145165677 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fad7c102520 0x7fad7c19d530 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fad78009aa0 tx=0x7fad78009db0 comp rx=0 tx=0).stop 2024-06-10T02:55:29.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.511+0000 7fad8407a700 1 -- 172.21.15.52:0/3145165677 shutdown_connections 2024-06-10T02:55:29.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.511+0000 7fad8407a700 1 --2- 172.21.15.52:0/3145165677 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fad7c1085c0 0x7fad7c1965c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:29.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.511+0000 7fad8407a700 1 --2- 172.21.15.52:0/3145165677 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fad680a2b80 0x7fad680a5040 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:29.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.511+0000 7fad8407a700 1 --2- 172.21.15.52:0/3145165677 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fad7c102520 0x7fad7c19d530 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:29.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.511+0000 7fad8407a700 1 --2- 172.21.15.52:0/3145165677 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fad7c102ee0 0x7fad7c19da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:29.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.511+0000 7fad8407a700 1 -- 172.21.15.52:0/3145165677 >> 172.21.15.52:0/3145165677 conn(0x7fad7c079680 msgr2=0x7fad7c100c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:55:29.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.512+0000 7fad8407a700 1 -- 172.21.15.52:0/3145165677 shutdown_connections 2024-06-10T02:55:29.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:29.512+0000 7fad8407a700 1 -- 172.21.15.52:0/3145165677 wait complete. 2024-06-10T02:55:30.453 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:55:30 smithi188 ceph-mon[36599]: pgmap v473: 129 pgs: 129 active+clean; 667 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 682 B/s wr, 9 op/s 2024-06-10T02:55:30.476 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:55:30 smithi052 ceph-mon[32362]: pgmap v473: 129 pgs: 129 active+clean; 667 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 682 B/s wr, 9 op/s 2024-06-10T02:55:30.638 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:55:30 smithi120 ceph-mon[36634]: pgmap v473: 129 pgs: 129 active+clean; 667 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 682 B/s wr, 9 op/s 2024-06-10T02:55:32.453 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:55:32 smithi188 ceph-mon[36599]: pgmap v474: 129 pgs: 129 active+clean; 667 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 682 B/s wr, 9 op/s 2024-06-10T02:55:32.453 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:55:32 smithi188 ceph-mon[36599]: scrub summary: idle 2024-06-10T02:55:32.476 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:55:32 smithi052 ceph-mon[32362]: pgmap v474: 129 pgs: 129 active+clean; 667 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 682 B/s wr, 9 op/s 2024-06-10T02:55:32.476 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:55:32 smithi052 ceph-mon[32362]: scrub summary: idle 2024-06-10T02:55:32.638 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:55:32 smithi120 ceph-mon[36634]: pgmap v474: 129 pgs: 129 active+clean; 667 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 682 B/s wr, 9 op/s 2024-06-10T02:55:32.638 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:55:32 smithi120 ceph-mon[36634]: scrub summary: idle 2024-06-10T02:55:34.453 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:55:34 smithi188 ceph-mon[36599]: pgmap v475: 129 pgs: 129 active+clean; 667 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 597 B/s wr, 12 op/s 2024-06-10T02:55:34.476 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:55:34 smithi052 ceph-mon[32362]: pgmap v475: 129 pgs: 129 active+clean; 667 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 597 B/s wr, 12 op/s 2024-06-10T02:55:34.638 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:55:34 smithi120 ceph-mon[36634]: pgmap v475: 129 pgs: 129 active+clean; 667 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 597 B/s wr, 12 op/s 2024-06-10T02:55:34.692 INFO:tasks.cephfs.filesystem:scrub status for tag:9ea45dce-3a46-4d2a-a88c-03b72edfb34d - {'path': '/', 'tag': '9ea45dce-3a46-4d2a-a88c-03b72edfb34d', 'options': 'recursive,force'} 2024-06-10T02:55:36.453 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:55:36 smithi188 ceph-mon[36599]: pgmap v476: 129 pgs: 129 active+clean; 667 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 85 B/s wr, 8 op/s 2024-06-10T02:55:36.476 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:55:36 smithi052 ceph-mon[32362]: pgmap v476: 129 pgs: 129 active+clean; 667 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 85 B/s wr, 8 op/s 2024-06-10T02:55:36.638 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:55:36 smithi120 ceph-mon[36634]: pgmap v476: 129 pgs: 129 active+clean; 667 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 85 B/s wr, 8 op/s 2024-06-10T02:55:38.453 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:55:38 smithi188 ceph-mon[36599]: pgmap v477: 129 pgs: 129 active+clean; 667 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 85 B/s wr, 12 op/s 2024-06-10T02:55:38.476 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:55:38 smithi052 ceph-mon[32362]: pgmap v477: 129 pgs: 129 active+clean; 667 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 85 B/s wr, 12 op/s 2024-06-10T02:55:38.638 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:55:38 smithi120 ceph-mon[36634]: pgmap v477: 129 pgs: 129 active+clean; 667 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 85 B/s wr, 12 op/s 2024-06-10T02:55:39.252 INFO:tasks.workunit.client.0.smithi052.stderr:Updating files: 94% (11870/12576) Updating files: 95% (11948/12576) Updating files: 96% (12073/12576) Updating files: 97% (12199/12576) Updating files: 98% (12325/12576) Updating files: 99% (12451/12576) Updating files: 100% (12576/12576) Updating files: 100% (12576/12576), done. 2024-06-10T02:55:39.752 INFO:tasks.workunit.client.0.smithi052.stderr:Note: switching to '7b7a5d375cf4e4afab96b9d4d229b699ae8304bb'. 2024-06-10T02:55:39.752 INFO:tasks.workunit.client.0.smithi052.stderr: 2024-06-10T02:55:39.753 INFO:tasks.workunit.client.0.smithi052.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-06-10T02:55:39.753 INFO:tasks.workunit.client.0.smithi052.stderr:changes and commit them, and you can discard any commits you make in this 2024-06-10T02:55:39.753 INFO:tasks.workunit.client.0.smithi052.stderr:state without impacting any branches by switching back to a branch. 2024-06-10T02:55:39.753 INFO:tasks.workunit.client.0.smithi052.stderr: 2024-06-10T02:55:39.753 INFO:tasks.workunit.client.0.smithi052.stderr:If you want to create a new branch to retain commits you create, you may 2024-06-10T02:55:39.753 INFO:tasks.workunit.client.0.smithi052.stderr:do so (now or later) by using -c with the switch command. Example: 2024-06-10T02:55:39.753 INFO:tasks.workunit.client.0.smithi052.stderr: 2024-06-10T02:55:39.753 INFO:tasks.workunit.client.0.smithi052.stderr: git switch -c 2024-06-10T02:55:39.753 INFO:tasks.workunit.client.0.smithi052.stderr: 2024-06-10T02:55:39.753 INFO:tasks.workunit.client.0.smithi052.stderr:Or undo this operation with: 2024-06-10T02:55:39.753 INFO:tasks.workunit.client.0.smithi052.stderr: 2024-06-10T02:55:39.753 INFO:tasks.workunit.client.0.smithi052.stderr: git switch - 2024-06-10T02:55:39.753 INFO:tasks.workunit.client.0.smithi052.stderr: 2024-06-10T02:55:39.754 INFO:tasks.workunit.client.0.smithi052.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-06-10T02:55:39.754 INFO:tasks.workunit.client.0.smithi052.stderr: 2024-06-10T02:55:39.754 INFO:tasks.workunit.client.0.smithi052.stderr:HEAD is now at 7b7a5d375cf Merge pull request #57941 from zdover23/wip-doc-2024-06-09-backport-57939-to-squid 2024-06-10T02:55:39.757 DEBUG:teuthology.orchestra.run.smithi052:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-06-10T02:55:39.785 INFO:tasks.workunit.client.0.smithi052.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-06-10T02:55:39.787 INFO:tasks.workunit.client.0.smithi052.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-06-10T02:55:39.787 INFO:tasks.workunit.client.0.smithi052.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-06-10T02:55:40.453 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:55:40 smithi188 ceph-mon[36599]: pgmap v478: 129 pgs: 129 active+clean; 667 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 85 B/s wr, 9 op/s 2024-06-10T02:55:40.476 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:55:40 smithi052 ceph-mon[32362]: pgmap v478: 129 pgs: 129 active+clean; 667 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 85 B/s wr, 9 op/s 2024-06-10T02:55:40.532 INFO:tasks.workunit.client.0.smithi052.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-06-10T02:55:40.587 INFO:tasks.workunit.client.0.smithi052.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-06-10T02:55:40.628 INFO:tasks.workunit.client.0.smithi052.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-06-10T02:55:40.629 INFO:tasks.workunit.client.0.smithi052.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-06-10T02:55:40.629 INFO:tasks.workunit.client.0.smithi052.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-06-10T02:55:40.637 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:55:40 smithi120 ceph-mon[36634]: pgmap v478: 129 pgs: 129 active+clean; 667 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 85 B/s wr, 9 op/s 2024-06-10T02:55:40.670 INFO:tasks.workunit.client.0.smithi052.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-06-10T02:55:40.673 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-06-10T02:55:40.673 DEBUG:teuthology.orchestra.run.smithi052:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-06-10T02:55:40.730 INFO:tasks.workunit:Running workunits matching suites/blogbench.sh on client.0... 2024-06-10T02:55:40.731 INFO:tasks.workunit:Running workunit suites/blogbench.sh... 2024-06-10T02:55:40.731 DEBUG:teuthology.orchestra.run.smithi052:workunit test suites/blogbench.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=7b7a5d375cf4e4afab96b9d4d229b699ae8304bb TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/blogbench.sh 2024-06-10T02:55:42.453 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:55:42 smithi188 ceph-mon[36599]: pgmap v479: 129 pgs: 129 active+clean; 667 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 85 B/s wr, 9 op/s 2024-06-10T02:55:42.476 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:55:42 smithi052 ceph-mon[32362]: pgmap v479: 129 pgs: 129 active+clean; 667 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 85 B/s wr, 9 op/s 2024-06-10T02:55:42.638 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:55:42 smithi120 ceph-mon[36634]: pgmap v479: 129 pgs: 129 active+clean; 667 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 85 B/s wr, 9 op/s 2024-06-10T02:55:44.454 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:55:44 smithi188 ceph-mon[36599]: pgmap v480: 129 pgs: 129 active+clean; 667 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-10T02:55:44.476 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:55:44 smithi052 ceph-mon[32362]: pgmap v480: 129 pgs: 129 active+clean; 667 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-10T02:55:44.638 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:55:44 smithi120 ceph-mon[36634]: pgmap v480: 129 pgs: 129 active+clean; 667 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-10T02:55:45.743 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T02:55:46.025 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:55:46.402 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:55:46 smithi052 ceph-mon[32362]: pgmap v481: 129 pgs: 129 active+clean; 667 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-10T02:55:46.402 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:55:46 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T02:55:46.453 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:55:46 smithi188 ceph-mon[36599]: pgmap v481: 129 pgs: 129 active+clean; 667 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-10T02:55:46.453 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:55:46 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T02:55:46.637 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:55:46 smithi120 ceph-mon[36634]: pgmap v481: 129 pgs: 129 active+clean; 667 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-10T02:55:46.638 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:55:46 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T02:55:47.421 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:55:47 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:55:47.421 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:55:47 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T02:55:47.421 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:55:47 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:55:47.453 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:55:47 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:55:47.453 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:55:47 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T02:55:47.453 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:55:47 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:55:47.638 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:55:47 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:55:47.638 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:55:47 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T02:55:47.638 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:55:47 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:55:47.789 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:47.789+0000 7ff470c34700 1 -- 172.21.15.52:0/3743644687 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff46c1019d0 msgr2=0x7ff46c101db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:55:47.790 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:47.789+0000 7ff470c34700 1 --2- 172.21.15.52:0/3743644687 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff46c1019d0 0x7ff46c101db0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7ff45c0099c0 tx=0x7ff45c009cd0 comp rx=0 tx=0).stop 2024-06-10T02:55:47.790 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:47.789+0000 7ff470c34700 1 -- 172.21.15.52:0/3743644687 shutdown_connections 2024-06-10T02:55:47.790 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:47.789+0000 7ff470c34700 1 --2- 172.21.15.52:0/3743644687 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff46c106720 0x7ff46c10e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:47.790 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:47.789+0000 7ff470c34700 1 --2- 172.21.15.52:0/3743644687 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff46c1019d0 0x7ff46c101db0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:47.790 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:47.789+0000 7ff470c34700 1 --2- 172.21.15.52:0/3743644687 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff46c102380 0x7ff46c105ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:47.790 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:47.789+0000 7ff470c34700 1 -- 172.21.15.52:0/3743644687 >> 172.21.15.52:0/3743644687 conn(0x7ff46c078340 msgr2=0x7ff46c078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:55:47.790 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:47.789+0000 7ff470c34700 1 -- 172.21.15.52:0/3743644687 shutdown_connections 2024-06-10T02:55:47.790 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:47.789+0000 7ff470c34700 1 -- 172.21.15.52:0/3743644687 wait complete. 2024-06-10T02:55:47.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:47.790+0000 7ff470c34700 1 Processor -- start 2024-06-10T02:55:47.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:47.790+0000 7ff470c34700 1 -- start start 2024-06-10T02:55:47.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:47.790+0000 7ff470c34700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff46c1019d0 0x7ff46c198d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:47.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:47.790+0000 7ff470c34700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff46c102380 0x7ff46c1992d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:47.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:47.790+0000 7ff470c34700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff46c106720 0x7ff46c19d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:47.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:47.790+0000 7ff470c34700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ff46c1117b0 con 0x7ff46c102380 2024-06-10T02:55:47.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:47.790+0000 7ff470c34700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff46c111630 con 0x7ff46c1019d0 2024-06-10T02:55:47.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:47.790+0000 7ff470c34700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ff46c111930 con 0x7ff46c106720 2024-06-10T02:55:47.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:47.791+0000 7ff469d9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff46c102380 0x7ff46c1992d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:47.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:47.791+0000 7ff469d9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff46c102380 0x7ff46c1992d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55142/0 (socket says 172.21.15.52:55142) 2024-06-10T02:55:47.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:47.791+0000 7ff469d9b700 1 -- 172.21.15.52:0/1798830656 learned_addr learned my addr 172.21.15.52:0/1798830656 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:55:47.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:47.791+0000 7ff46ad9d700 1 --2- 172.21.15.52:0/1798830656 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff46c106720 0x7ff46c19d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:47.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:47.791+0000 7ff46a59c700 1 --2- 172.21.15.52:0/1798830656 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff46c1019d0 0x7ff46c198d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:47.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:47.791+0000 7ff469d9b700 1 -- 172.21.15.52:0/1798830656 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff46c106720 msgr2=0x7ff46c19d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:55:47.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:47.791+0000 7ff469d9b700 1 --2- 172.21.15.52:0/1798830656 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff46c106720 0x7ff46c19d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:47.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:47.791+0000 7ff469d9b700 1 -- 172.21.15.52:0/1798830656 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff46c1019d0 msgr2=0x7ff46c198d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:55:47.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:47.791+0000 7ff469d9b700 1 --2- 172.21.15.52:0/1798830656 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff46c1019d0 0x7ff46c198d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:47.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:47.791+0000 7ff469d9b700 1 -- 172.21.15.52:0/1798830656 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff46c19dda0 con 0x7ff46c102380 2024-06-10T02:55:47.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:47.791+0000 7ff46a59c700 1 --2- 172.21.15.52:0/1798830656 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff46c1019d0 0x7ff46c198d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T02:55:47.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:47.791+0000 7ff469d9b700 1 --2- 172.21.15.52:0/1798830656 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff46c102380 0x7ff46c1992d0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7ff46000b4d0 tx=0x7ff46000b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:55:47.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:47.791+0000 7ff45b7fe700 1 -- 172.21.15.52:0/1798830656 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff460015020 con 0x7ff46c102380 2024-06-10T02:55:47.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:47.791+0000 7ff45b7fe700 1 -- 172.21.15.52:0/1798830656 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff460011d10 con 0x7ff46c102380 2024-06-10T02:55:47.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:47.791+0000 7ff470c34700 1 -- 172.21.15.52:0/1798830656 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff46c19e090 con 0x7ff46c102380 2024-06-10T02:55:47.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:47.791+0000 7ff45b7fe700 1 -- 172.21.15.52:0/1798830656 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff460010de0 con 0x7ff46c102380 2024-06-10T02:55:47.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:47.791+0000 7ff470c34700 1 -- 172.21.15.52:0/1798830656 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff46c102fc0 con 0x7ff46c102380 2024-06-10T02:55:47.794 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:47.792+0000 7ff470c34700 1 -- 172.21.15.52:0/1798830656 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff46c102ad0 con 0x7ff46c102380 2024-06-10T02:55:47.794 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:47.794+0000 7ff45b7fe700 1 -- 172.21.15.52:0/1798830656 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7ff460018020 con 0x7ff46c102380 2024-06-10T02:55:47.794 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:47.794+0000 7ff45b7fe700 1 --2- 172.21.15.52:0/1798830656 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7ff4540a2c20 0x7ff4540a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:47.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:47.794+0000 7ff46a59c700 1 --2- 172.21.15.52:0/1798830656 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7ff4540a2c20 0x7ff4540a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:47.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:47.794+0000 7ff45b7fe700 1 -- 172.21.15.52:0/1798830656 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7ff4600ceb30 con 0x7ff46c102380 2024-06-10T02:55:47.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:47.795+0000 7ff46a59c700 1 --2- 172.21.15.52:0/1798830656 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7ff4540a2c20 0x7ff4540a50e0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7ff45c006010 tx=0x7ff45c009f10 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:55:47.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:47.799+0000 7ff45b7fe700 1 -- 172.21.15.52:0/1798830656 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7ff46001f020 con 0x7ff46c102380 2024-06-10T02:55:48.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:48.056+0000 7ff470c34700 1 -- 172.21.15.52:0/1798830656 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff46c02dca0 con 0x7ff46c102380 2024-06-10T02:55:48.057 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:48.057+0000 7ff45b7fe700 1 -- 172.21.15.52:0/1798830656 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11938 (secure 0 0 0) 0x7ff46009b6e0 con 0x7ff46c102380 2024-06-10T02:55:48.057 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-10T02:55:48.057 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/889673247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":889673247},{"type":"v1","addr":"172.21.15.52:6835","nonce":889673247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/3615663511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":3615663511},{"type":"v1","addr":"172.21.15.52:6837","nonce":3615663511}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T02:50:03.889766+0000","modified":"2024-06-10T02:54:07.086405+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24484,"mds_1":24469,"mds_2":24463,"mds_3":24449,"mds_4":24428},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14571":{"gid":14571,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6839/3059599042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":3059599042},{"type":"v1","addr":"172.21.15.52:6839","nonce":3059599042}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14583":{"gid":14583,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6841/3039275521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":3039275521},{"type":"v1","addr":"172.21.15.52:6841","nonce":3039275521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24383":{"gid":24383,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6833/2609002896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":2609002896},{"type":"v1","addr":"172.21.15.120:6833","nonce":2609002896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6835/1996370288","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":1996370288},{"type":"v1","addr":"172.21.15.120:6835","nonce":1996370288}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6833/529564979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":529564979},{"type":"v1","addr":"172.21.15.188:6833","nonce":529564979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":25,"addr":"172.21.15.120:6837/4132440952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":4132440952},{"type":"v1","addr":"172.21.15.120:6837","nonce":4132440952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"b","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.120:6839/1672436487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1672436487},{"type":"v1","addr":"172.21.15.120:6839","nonce":1672436487}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":2,"incarnation":22,"state":"up:active","state_seq":26,"addr":"172.21.15.188:6835/3904544259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":3904544259},{"type":"v1","addr":"172.21.15.188:6835","nonce":3904544259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.188:6837/819317726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":819317726},{"type":"v1","addr":"172.21.15.188:6837","nonce":819317726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.188:6839/310444836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":310444836},{"type":"v1","addr":"172.21.15.188:6839","nonce":310444836}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24484,"qdb_cluster":[24469,24484,24463,24428,24449]},"id":1}]} 2024-06-10T02:55:48.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:48.059+0000 7ff470c34700 1 -- 172.21.15.52:0/1798830656 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7ff4540a2c20 msgr2=0x7ff4540a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:55:48.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:48.059+0000 7ff470c34700 1 --2- 172.21.15.52:0/1798830656 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7ff4540a2c20 0x7ff4540a50e0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7ff45c006010 tx=0x7ff45c009f10 comp rx=0 tx=0).stop 2024-06-10T02:55:48.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:48.059+0000 7ff470c34700 1 -- 172.21.15.52:0/1798830656 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff46c102380 msgr2=0x7ff46c1992d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:55:48.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:48.059+0000 7ff470c34700 1 --2- 172.21.15.52:0/1798830656 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff46c102380 0x7ff46c1992d0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7ff46000b4d0 tx=0x7ff46000b7e0 comp rx=0 tx=0).stop 2024-06-10T02:55:48.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:48.059+0000 7ff470c34700 1 -- 172.21.15.52:0/1798830656 shutdown_connections 2024-06-10T02:55:48.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:48.059+0000 7ff470c34700 1 --2- 172.21.15.52:0/1798830656 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff46c106720 0x7ff46c19d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:48.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:48.059+0000 7ff470c34700 1 --2- 172.21.15.52:0/1798830656 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff46c1019d0 0x7ff46c198d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:48.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:48.059+0000 7ff470c34700 1 --2- 172.21.15.52:0/1798830656 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7ff4540a2c20 0x7ff4540a50e0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:48.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:48.059+0000 7ff470c34700 1 --2- 172.21.15.52:0/1798830656 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff46c102380 0x7ff46c1992d0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:48.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:48.059+0000 7ff470c34700 1 -- 172.21.15.52:0/1798830656 >> 172.21.15.52:0/1798830656 conn(0x7ff46c078340 msgr2=0x7ff46c06af40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:55:48.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:48.059+0000 7ff470c34700 1 -- 172.21.15.52:0/1798830656 shutdown_connections 2024-06-10T02:55:48.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:48.059+0000 7ff470c34700 1 -- 172.21.15.52:0/1798830656 wait complete. 2024-06-10T02:55:48.061 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 30 2024-06-10T02:55:48.726 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:55:48 smithi052 ceph-mon[32362]: pgmap v482: 129 pgs: 129 active+clean; 667 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 85 B/s wr, 11 op/s 2024-06-10T02:55:48.727 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:55:48 smithi052 ceph-mon[32362]: from='client.? 172.21.15.52:0/1798830656' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T02:55:48.831 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24484 quiesce db --set-id 57f4a9c1 --release --await' 2024-06-10T02:55:48.831 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24484 quiesce db --set-id 57f4a9c1 --release --await 2024-06-10T02:55:48.888 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:55:48 smithi120 ceph-mon[36634]: pgmap v482: 129 pgs: 129 active+clean; 667 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 85 B/s wr, 11 op/s 2024-06-10T02:55:48.888 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:55:48 smithi120 ceph-mon[36634]: from='client.? 172.21.15.52:0/1798830656' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T02:55:48.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:55:48 smithi188 ceph-mon[36599]: pgmap v482: 129 pgs: 129 active+clean; 667 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 85 B/s wr, 11 op/s 2024-06-10T02:55:48.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:55:48 smithi188 ceph-mon[36599]: from='client.? 172.21.15.52:0/1798830656' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T02:55:49.090 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:55:50.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.647+0000 7f2a7dd2d700 1 -- 172.21.15.52:0/3565120171 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2a781066b0 msgr2=0x7f2a7810e280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:55:50.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.647+0000 7f2a7dd2d700 1 --2- 172.21.15.52:0/3565120171 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2a781066b0 0x7f2a7810e280 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f2a700099c0 tx=0x7f2a70009cd0 comp rx=0 tx=0).stop 2024-06-10T02:55:50.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.647+0000 7f2a7dd2d700 1 -- 172.21.15.52:0/3565120171 shutdown_connections 2024-06-10T02:55:50.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.647+0000 7f2a7dd2d700 1 --2- 172.21.15.52:0/3565120171 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2a781066b0 0x7f2a7810e280 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:50.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.647+0000 7f2a7dd2d700 1 --2- 172.21.15.52:0/3565120171 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2a78101620 0x7f2a78101a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:50.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.647+0000 7f2a7dd2d700 1 --2- 172.21.15.52:0/3565120171 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2a78101fd0 0x7f2a78105fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:50.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.647+0000 7f2a7dd2d700 1 -- 172.21.15.52:0/3565120171 >> 172.21.15.52:0/3565120171 conn(0x7f2a78078340 msgr2=0x7f2a78078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:55:50.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.647+0000 7f2a7dd2d700 1 -- 172.21.15.52:0/3565120171 shutdown_connections 2024-06-10T02:55:50.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.647+0000 7f2a7dd2d700 1 -- 172.21.15.52:0/3565120171 wait complete. 2024-06-10T02:55:50.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.648+0000 7f2a7dd2d700 1 Processor -- start 2024-06-10T02:55:50.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.648+0000 7f2a7dd2d700 1 -- start start 2024-06-10T02:55:50.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.649+0000 7f2a7dd2d700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2a78101620 0x7f2a7819ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:50.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.649+0000 7f2a7dd2d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2a78101fd0 0x7f2a7819b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:50.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.649+0000 7f2a7dd2d700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2a781066b0 0x7f2a7819f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:50.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.649+0000 7f2a7dd2d700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f2a78111710 con 0x7f2a78101fd0 2024-06-10T02:55:50.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.649+0000 7f2a7dd2d700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f2a78111590 con 0x7f2a78101620 2024-06-10T02:55:50.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.649+0000 7f2a7dd2d700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f2a78111890 con 0x7f2a781066b0 2024-06-10T02:55:50.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.649+0000 7f2a6effd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2a78101fd0 0x7f2a7819b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:50.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.649+0000 7f2a6effd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2a78101fd0 0x7f2a7819b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55166/0 (socket says 172.21.15.52:55166) 2024-06-10T02:55:50.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.649+0000 7f2a6effd700 1 -- 172.21.15.52:0/393482784 learned_addr learned my addr 172.21.15.52:0/393482784 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:55:50.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.649+0000 7f2a777fe700 1 --2- 172.21.15.52:0/393482784 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2a78101620 0x7f2a7819ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:50.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.649+0000 7f2a77fff700 1 --2- 172.21.15.52:0/393482784 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2a781066b0 0x7f2a7819f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:50.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.649+0000 7f2a6effd700 1 -- 172.21.15.52:0/393482784 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2a781066b0 msgr2=0x7f2a7819f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:55:50.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.649+0000 7f2a6effd700 1 --2- 172.21.15.52:0/393482784 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2a781066b0 0x7f2a7819f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:50.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.649+0000 7f2a6effd700 1 -- 172.21.15.52:0/393482784 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2a78101620 msgr2=0x7f2a7819ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:55:50.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.649+0000 7f2a6effd700 1 --2- 172.21.15.52:0/393482784 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2a78101620 0x7f2a7819ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:50.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.649+0000 7f2a6effd700 1 -- 172.21.15.52:0/393482784 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a7819fe50 con 0x7f2a78101fd0 2024-06-10T02:55:50.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.650+0000 7f2a6effd700 1 --2- 172.21.15.52:0/393482784 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2a78101fd0 0x7f2a7819b380 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f2a6000b500 tx=0x7f2a6000b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:55:50.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.650+0000 7f2a757fa700 1 -- 172.21.15.52:0/393482784 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a60015020 con 0x7f2a78101fd0 2024-06-10T02:55:50.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.650+0000 7f2a7dd2d700 1 -- 172.21.15.52:0/393482784 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a781a0140 con 0x7f2a78101fd0 2024-06-10T02:55:50.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.650+0000 7f2a757fa700 1 -- 172.21.15.52:0/393482784 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a60011d10 con 0x7f2a78101fd0 2024-06-10T02:55:50.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.650+0000 7f2a757fa700 1 -- 172.21.15.52:0/393482784 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a60010de0 con 0x7f2a78101fd0 2024-06-10T02:55:50.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.650+0000 7f2a7dd2d700 1 -- 172.21.15.52:0/393482784 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a781a79c0 con 0x7f2a78101fd0 2024-06-10T02:55:50.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.652+0000 7f2a757fa700 1 -- 172.21.15.52:0/393482784 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f2a60018020 con 0x7f2a78101fd0 2024-06-10T02:55:50.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.653+0000 7f2a757fa700 1 --2- 172.21.15.52:0/393482784 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f2a640a2ca0 0x7f2a640a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:50.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.653+0000 7f2a777fe700 1 --2- 172.21.15.52:0/393482784 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f2a640a2ca0 0x7f2a640a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:50.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.653+0000 7f2a757fa700 1 -- 172.21.15.52:0/393482784 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f2a600cea70 con 0x7f2a78101fd0 2024-06-10T02:55:50.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.653+0000 7f2a777fe700 1 --2- 172.21.15.52:0/393482784 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f2a640a2ca0 0x7f2a640a5160 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f2a68009550 tx=0x7f2a680091b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:55:50.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.653+0000 7f2a885b0b80 1 Processor -- start 2024-06-10T02:55:50.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.653+0000 7f2a885b0b80 1 -- start start 2024-06-10T02:55:50.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.654+0000 7f2a885b0b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562472c8f500 0x562472c8f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:50.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.654+0000 7f2a885b0b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562472c8fe20 0x562472c89710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:50.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.654+0000 7f2a885b0b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562472c89c50 0x562472c8a070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:50.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.654+0000 7f2a885b0b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x562472b7c060 con 0x562472c8f500 2024-06-10T02:55:50.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.654+0000 7f2a885b0b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x562472bda950 con 0x562472c89c50 2024-06-10T02:55:50.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.654+0000 7f2a885b0b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x562472b42560 con 0x562472c8fe20 2024-06-10T02:55:50.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.654+0000 7f2a777fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562472c8f500 0x562472c8f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:50.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.654+0000 7f2a777fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562472c8f500 0x562472c8f8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55178/0 (socket says 172.21.15.52:55178) 2024-06-10T02:55:50.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.654+0000 7f2a777fe700 1 -- 172.21.15.52:0/1887385696 learned_addr learned my addr 172.21.15.52:0/1887385696 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:55:50.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.654+0000 7f2a77fff700 1 --2- 172.21.15.52:0/1887385696 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562472c89c50 0x562472c8a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:50.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.654+0000 7f2a6effd700 1 --2- 172.21.15.52:0/1887385696 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562472c8fe20 0x562472c89710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:50.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.654+0000 7f2a777fe700 1 -- 172.21.15.52:0/1887385696 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562472c8fe20 msgr2=0x562472c89710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:55:50.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.654+0000 7f2a777fe700 1 --2- 172.21.15.52:0/1887385696 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562472c8fe20 0x562472c89710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:50.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.654+0000 7f2a777fe700 1 -- 172.21.15.52:0/1887385696 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562472c89c50 msgr2=0x562472c8a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:55:50.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.654+0000 7f2a777fe700 1 --2- 172.21.15.52:0/1887385696 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562472c89c50 0x562472c8a070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:50.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.654+0000 7f2a777fe700 1 -- 172.21.15.52:0/1887385696 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562472b299d0 con 0x562472c8f500 2024-06-10T02:55:50.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.654+0000 7f2a777fe700 1 --2- 172.21.15.52:0/1887385696 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562472c8f500 0x562472c8f8e0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f2a680046e0 tx=0x7f2a68013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=545c195a839a00ca server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:55:50.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.654+0000 7f2a6dffb700 1 -- 172.21.15.52:0/1887385696 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a6801c050 con 0x562472c8f500 2024-06-10T02:55:50.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.655+0000 7f2a6dffb700 1 -- 172.21.15.52:0/1887385696 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a68026070 con 0x562472c8f500 2024-06-10T02:55:50.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.655+0000 7f2a6dffb700 1 -- 172.21.15.52:0/1887385696 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a68015730 con 0x562472c8f500 2024-06-10T02:55:50.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.655+0000 7f2a885b0b80 1 -- 172.21.15.52:0/1887385696 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562472c8f500 msgr2=0x562472c8f8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:55:50.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.655+0000 7f2a885b0b80 1 --2- 172.21.15.52:0/1887385696 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562472c8f500 0x562472c8f8e0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f2a680046e0 tx=0x7f2a68013000 comp rx=0 tx=0).stop 2024-06-10T02:55:50.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.655+0000 7f2a885b0b80 1 -- 172.21.15.52:0/1887385696 shutdown_connections 2024-06-10T02:55:50.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.655+0000 7f2a885b0b80 1 --2- 172.21.15.52:0/1887385696 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562472c89c50 0x562472c8a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:50.660 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.655+0000 7f2a885b0b80 1 --2- 172.21.15.52:0/1887385696 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562472c8f500 0x562472c8f8e0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:50.660 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.655+0000 7f2a885b0b80 1 --2- 172.21.15.52:0/1887385696 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562472c8fe20 0x562472c89710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:50.660 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.655+0000 7f2a885b0b80 1 -- 172.21.15.52:0/1887385696 >> 172.21.15.52:0/1887385696 conn(0x562472bb6180 msgr2=0x562472c84a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:55:50.660 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.655+0000 7f2a885b0b80 1 -- 172.21.15.52:0/1887385696 shutdown_connections 2024-06-10T02:55:50.660 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.655+0000 7f2a885b0b80 1 -- 172.21.15.52:0/1887385696 wait complete. 2024-06-10T02:55:50.660 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.656+0000 7f2a885b0b80 1 -- start start 2024-06-10T02:55:50.660 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.656+0000 7f2a885b0b80 1 Processor -- start 2024-06-10T02:55:50.660 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.656+0000 7f2a885b0b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562472c89c50 0x562472c95b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:50.660 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.656+0000 7f2a885b0b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562472c8f500 0x562472c92e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:50.660 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.656+0000 7f2a885b0b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562472c8fe20 0x562472c933c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:50.660 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.656+0000 7f2a885b0b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x562472b7b0c0 con 0x562472c8fe20 2024-06-10T02:55:50.660 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.656+0000 7f2a885b0b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x562472bd6910 con 0x562472c8f500 2024-06-10T02:55:50.660 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.656+0000 7f2a885b0b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x562472c96800 con 0x562472c89c50 2024-06-10T02:55:50.660 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.656+0000 7f2a885b0b80 10 client.0 _ll_register_callbacks cb 0x562472b706b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T02:55:50.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.656+0000 7f2a77fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562472c8fe20 0x562472c933c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:50.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.656+0000 7f2a77fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562472c8fe20 0x562472c933c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55182/0 (socket says 172.21.15.52:55182) 2024-06-10T02:55:50.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.656+0000 7f2a77fff700 1 -- 172.21.15.52:0/2412976559 learned_addr learned my addr 172.21.15.52:0/2412976559 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:55:50.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.657+0000 7f2a777fe700 1 --2- 172.21.15.52:0/2412976559 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562472c89c50 0x562472c95b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:50.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.657+0000 7f2a6effd700 1 --2- 172.21.15.52:0/2412976559 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562472c8f500 0x562472c92e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:50.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.657+0000 7f2a77fff700 1 -- 172.21.15.52:0/2412976559 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562472c89c50 msgr2=0x562472c95b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:55:50.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.657+0000 7f2a77fff700 1 --2- 172.21.15.52:0/2412976559 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562472c89c50 0x562472c95b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:50.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.657+0000 7f2a77fff700 1 -- 172.21.15.52:0/2412976559 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562472c8f500 msgr2=0x562472c92e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:55:50.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.657+0000 7f2a77fff700 1 --2- 172.21.15.52:0/2412976559 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562472c8f500 0x562472c92e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:50.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.657+0000 7f2a77fff700 1 -- 172.21.15.52:0/2412976559 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562472ba0850 con 0x562472c8fe20 2024-06-10T02:55:50.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.657+0000 7f2a777fe700 1 --2- 172.21.15.52:0/2412976559 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562472c89c50 0x562472c95b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T02:55:50.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.657+0000 7f2a6effd700 1 --2- 172.21.15.52:0/2412976559 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562472c8f500 0x562472c92e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T02:55:50.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.657+0000 7f2a77fff700 1 --2- 172.21.15.52:0/2412976559 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562472c8fe20 0x562472c933c0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f2a70009740 tx=0x7f2a70003ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=856fabef37455480 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:55:50.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.657+0000 7f2a56ffd700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-10T02:55:50.662 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.657+0000 7f2a56ffd700 1 -- 172.21.15.52:0/2412976559 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a70028060 con 0x562472c8fe20 2024-06-10T02:55:50.662 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.657+0000 7f2a56ffd700 1 -- 172.21.15.52:0/2412976559 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f2a700052c0 con 0x562472c8fe20 2024-06-10T02:55:50.662 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.657+0000 7f2a885b0b80 1 -- 172.21.15.52:0/2412976559 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2a68026070 con 0x562472c8fe20 2024-06-10T02:55:50.662 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.657+0000 7f2a56ffd700 1 -- 172.21.15.52:0/2412976559 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f2a7002f050 con 0x562472c8fe20 2024-06-10T02:55:50.662 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.657+0000 7f2a885b0b80 10 client.15714 fetch_fsmap learned FSMap version 30 2024-06-10T02:55:50.662 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.657+0000 7f2a885b0b80 1 -- 172.21.15.52:0/2412976559 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x562472ca0f50 con 0x562472c8fe20 2024-06-10T02:55:50.662 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.658+0000 7f2a56ffd700 1 -- 172.21.15.52:0/2412976559 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a68026070 con 0x562472c8fe20 2024-06-10T02:55:50.662 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.658+0000 7f2a56ffd700 1 -- 172.21.15.52:0/2412976559 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f2a7000f760 con 0x562472c8fe20 2024-06-10T02:55:50.662 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.658+0000 7f2a885b0b80 10 client.15714 fetch_fsmap finished waiting for FSMap version 30 2024-06-10T02:55:50.662 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.658+0000 7f2a885b0b80 10 client.15714 resolve_mds: validated gid 24484 aka daemon mds.i 2024-06-10T02:55:50.662 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.658+0000 7f2a885b0b80 20 client.15714 populate_metadata read hostname 'smithi052' 2024-06-10T02:55:50.662 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.658+0000 7f2a885b0b80 1 --2- 172.21.15.52:0/2412976559 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x562472ca3590 0x562472ca5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:50.662 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.658+0000 7f2a885b0b80 4 client.15714 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T02:55:50.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.658+0000 7f2a885b0b80 1 -- 172.21.15.52:0/2412976559 --> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562472be7e50 con 0x562472ca3590 2024-06-10T02:55:50.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.658+0000 7f2a777fe700 1 --2- 172.21.15.52:0/2412976559 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x562472ca3590 0x562472ca5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:50.663 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:55:50 smithi052 ceph-mon[32362]: pgmap v483: 129 pgs: 129 active+clean; 667 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 9 op/s 2024-06-10T02:55:50.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.659+0000 7f2a777fe700 1 --2- 172.21.15.52:0/2412976559 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x562472ca3590 0x562472ca5980 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=73f9460214e301c8 server_cookie=76ab0ebc55fe79f5 in_seq=0 out_seq=0 2024-06-10T02:55:50.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.659+0000 7f2a56ffd700 10 client.15714 ms_handle_connect on v2:172.21.15.188:6838/310444836 2024-06-10T02:55:50.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.659+0000 7f2a56ffd700 1 -- 172.21.15.52:0/2412976559 <== mds.0 v2:172.21.15.188:6838/310444836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x562472be7e50 con 0x562472ca3590 2024-06-10T02:55:50.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.659+0000 7f2a56ffd700 10 client.15714 handle_command_reply: tid=0 2024-06-10T02:55:50.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.659+0000 7f2a885b0b80 1 client.15714 shutdown 2024-06-10T02:55:50.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.660+0000 7f2a885b0b80 1 -- 172.21.15.52:0/2412976559 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562472c8fe20 msgr2=0x562472c933c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:55:50.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.660+0000 7f2a885b0b80 1 --2- 172.21.15.52:0/2412976559 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562472c8fe20 0x562472c933c0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f2a70009740 tx=0x7f2a70003ef0 comp rx=0 tx=0).stop 2024-06-10T02:55:50.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.660+0000 7f2a885b0b80 1 -- 172.21.15.52:0/2412976559 shutdown_connections 2024-06-10T02:55:50.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.660+0000 7f2a885b0b80 1 --2- 172.21.15.52:0/2412976559 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x562472ca3590 0x562472ca5980 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:50.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.660+0000 7f2a885b0b80 1 --2- 172.21.15.52:0/2412976559 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562472c8fe20 0x562472c933c0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:50.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.660+0000 7f2a885b0b80 1 --2- 172.21.15.52:0/2412976559 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562472c89c50 0x562472c95b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:50.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.660+0000 7f2a56ffd700 0 client.15714 ms_handle_reset on v2:172.21.15.188:6838/310444836 2024-06-10T02:55:50.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.660+0000 7f2a885b0b80 1 --2- 172.21.15.52:0/2412976559 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562472c8f500 0x562472c92e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:50.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.660+0000 7f2a885b0b80 1 -- 172.21.15.52:0/2412976559 >> 172.21.15.52:0/2412976559 conn(0x562472bb6180 msgr2=0x562472c83320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:55:50.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.660+0000 7f2a885b0b80 1 -- 172.21.15.52:0/2412976559 shutdown_connections 2024-06-10T02:55:50.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.660+0000 7f2a885b0b80 1 -- 172.21.15.52:0/2412976559 wait complete. 2024-06-10T02:55:50.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.660+0000 7f2a885b0b80 20 client.15714 trim_cache size 0 max 16384 2024-06-10T02:55:50.683 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.682+0000 7f2a885b0b80 1 Processor -- start 2024-06-10T02:55:50.685 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.682+0000 7f2a885b0b80 1 -- start start 2024-06-10T02:55:50.685 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.682+0000 7f2a885b0b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562472c89c50 0x562472c89710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:50.685 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.682+0000 7f2a885b0b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562472c8f500 0x562472c84ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:50.685 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.682+0000 7f2a885b0b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562472c8fe20 0x562472c90910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:50.685 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.682+0000 7f2a885b0b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f2a781b0620 con 0x562472c8fe20 2024-06-10T02:55:50.685 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.682+0000 7f2a885b0b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f2a781b2160 con 0x562472c89c50 2024-06-10T02:55:50.685 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.682+0000 7f2a885b0b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f2a781b0cf0 con 0x562472c8f500 2024-06-10T02:55:50.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.682+0000 7f2a77fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562472c8fe20 0x562472c90910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:50.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.682+0000 7f2a77fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562472c8fe20 0x562472c90910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55192/0 (socket says 172.21.15.52:55192) 2024-06-10T02:55:50.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.682+0000 7f2a77fff700 1 -- 172.21.15.52:0/2323896708 learned_addr learned my addr 172.21.15.52:0/2323896708 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:55:50.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.682+0000 7f2a6effd700 1 --2- 172.21.15.52:0/2323896708 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562472c8f500 0x562472c84ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:50.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.682+0000 7f2a777fe700 1 --2- 172.21.15.52:0/2323896708 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562472c89c50 0x562472c89710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:50.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.682+0000 7f2a77fff700 1 -- 172.21.15.52:0/2323896708 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562472c8f500 msgr2=0x562472c84ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:55:50.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.682+0000 7f2a77fff700 1 --2- 172.21.15.52:0/2323896708 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562472c8f500 0x562472c84ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:50.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.682+0000 7f2a77fff700 1 -- 172.21.15.52:0/2323896708 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562472c89c50 msgr2=0x562472c89710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:55:50.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.682+0000 7f2a77fff700 1 --2- 172.21.15.52:0/2323896708 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562472c89c50 0x562472c89710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:50.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.682+0000 7f2a77fff700 1 -- 172.21.15.52:0/2323896708 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562472c937e0 con 0x562472c8fe20 2024-06-10T02:55:50.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.682+0000 7f2a777fe700 1 --2- 172.21.15.52:0/2323896708 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562472c89c50 0x562472c89710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T02:55:50.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.682+0000 7f2a77fff700 1 --2- 172.21.15.52:0/2323896708 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562472c8fe20 0x562472c90910 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f2a700098e0 tx=0x7f2a70010f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2916959a6f49870 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:55:50.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.682+0000 7f2a6d7fa700 1 -- 172.21.15.52:0/2323896708 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a70038070 con 0x562472c8fe20 2024-06-10T02:55:50.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.682+0000 7f2a6d7fa700 1 -- 172.21.15.52:0/2323896708 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a700055c0 con 0x562472c8fe20 2024-06-10T02:55:50.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.682+0000 7f2a6d7fa700 1 -- 172.21.15.52:0/2323896708 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a70028040 con 0x562472c8fe20 2024-06-10T02:55:50.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.683+0000 7f2a885b0b80 1 -- 172.21.15.52:0/2323896708 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562472c8fe20 msgr2=0x562472c90910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:55:50.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.683+0000 7f2a885b0b80 1 --2- 172.21.15.52:0/2323896708 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562472c8fe20 0x562472c90910 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f2a700098e0 tx=0x7f2a70010f70 comp rx=0 tx=0).stop 2024-06-10T02:55:50.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.683+0000 7f2a885b0b80 1 -- 172.21.15.52:0/2323896708 shutdown_connections 2024-06-10T02:55:50.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.683+0000 7f2a885b0b80 1 --2- 172.21.15.52:0/2323896708 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562472c8fe20 0x562472c90910 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:50.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.683+0000 7f2a885b0b80 1 --2- 172.21.15.52:0/2323896708 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562472c89c50 0x562472c89710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:50.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.683+0000 7f2a885b0b80 1 --2- 172.21.15.52:0/2323896708 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562472c8f500 0x562472c84ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:50.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.683+0000 7f2a885b0b80 1 -- 172.21.15.52:0/2323896708 >> 172.21.15.52:0/2323896708 conn(0x562472bb6180 msgr2=0x562472c90200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:55:50.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.683+0000 7f2a885b0b80 1 -- 172.21.15.52:0/2323896708 shutdown_connections 2024-06-10T02:55:50.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.683+0000 7f2a885b0b80 1 -- 172.21.15.52:0/2323896708 wait complete. 2024-06-10T02:55:50.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.683+0000 7f2a885b0b80 1 -- start start 2024-06-10T02:55:50.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.683+0000 7f2a885b0b80 1 Processor -- start 2024-06-10T02:55:50.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.684+0000 7f2a885b0b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562472c89c50 0x562472c9c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:50.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.684+0000 7f2a885b0b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562472c8f500 0x562472caaa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:50.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.684+0000 7f2a885b0b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562472c8fe20 0x562472caafb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:50.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.684+0000 7f2a885b0b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f2a781b1a90 con 0x562472c8f500 2024-06-10T02:55:50.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.684+0000 7f2a885b0b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f2a781b13c0 con 0x562472c8fe20 2024-06-10T02:55:50.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.684+0000 7f2a885b0b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f2a781ae5c0 con 0x562472c89c50 2024-06-10T02:55:50.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.684+0000 7f2a885b0b80 10 client.0 _ll_register_callbacks cb 0x562472b706b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T02:55:50.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.684+0000 7f2a6effd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562472c8f500 0x562472caaa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:50.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.684+0000 7f2a6effd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562472c8f500 0x562472caaa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55198/0 (socket says 172.21.15.52:55198) 2024-06-10T02:55:50.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.684+0000 7f2a6effd700 1 -- 172.21.15.52:0/3140130528 learned_addr learned my addr 172.21.15.52:0/3140130528 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:55:50.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.684+0000 7f2a77fff700 1 --2- 172.21.15.52:0/3140130528 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562472c8fe20 0x562472caafb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:50.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.684+0000 7f2a777fe700 1 --2- 172.21.15.52:0/3140130528 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562472c89c50 0x562472c9c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:50.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.684+0000 7f2a6effd700 1 -- 172.21.15.52:0/3140130528 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562472c89c50 msgr2=0x562472c9c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:55:50.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.684+0000 7f2a6effd700 1 --2- 172.21.15.52:0/3140130528 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562472c89c50 0x562472c9c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:50.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.684+0000 7f2a6effd700 1 -- 172.21.15.52:0/3140130528 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562472c8fe20 msgr2=0x562472caafb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:55:50.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.684+0000 7f2a6effd700 1 --2- 172.21.15.52:0/3140130528 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562472c8fe20 0x562472caafb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:50.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.684+0000 7f2a6effd700 1 -- 172.21.15.52:0/3140130528 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562472cab7f0 con 0x562472c8f500 2024-06-10T02:55:50.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.684+0000 7f2a6effd700 1 --2- 172.21.15.52:0/3140130528 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562472c8f500 0x562472caaa70 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f2a60097380 tx=0x7f2a60010880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fe9512dddfcc627 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:55:50.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.684+0000 7f2a56ffd700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-10T02:55:50.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.685+0000 7f2a56ffd700 1 -- 172.21.15.52:0/3140130528 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a6009f2d0 con 0x562472c8f500 2024-06-10T02:55:50.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.685+0000 7f2a56ffd700 1 -- 172.21.15.52:0/3140130528 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f2a60010480 con 0x562472c8f500 2024-06-10T02:55:50.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.685+0000 7f2a885b0b80 1 -- 172.21.15.52:0/3140130528 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2a700055c0 con 0x562472c8f500 2024-06-10T02:55:50.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.685+0000 7f2a56ffd700 1 -- 172.21.15.52:0/3140130528 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f2a6007e050 con 0x562472c8f500 2024-06-10T02:55:50.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.685+0000 7f2a885b0b80 10 client.15720 fetch_fsmap learned FSMap version 30 2024-06-10T02:55:50.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.685+0000 7f2a885b0b80 1 -- 172.21.15.52:0/3140130528 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x562472d0e170 con 0x562472c8f500 2024-06-10T02:55:50.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.685+0000 7f2a56ffd700 1 -- 172.21.15.52:0/3140130528 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a700055c0 con 0x562472c8f500 2024-06-10T02:55:50.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.685+0000 7f2a56ffd700 1 -- 172.21.15.52:0/3140130528 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f2a600b9030 con 0x562472c8f500 2024-06-10T02:55:50.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.685+0000 7f2a885b0b80 10 client.15720 fetch_fsmap finished waiting for FSMap version 30 2024-06-10T02:55:50.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.685+0000 7f2a885b0b80 10 client.15720 resolve_mds: validated gid 24484 aka daemon mds.i 2024-06-10T02:55:50.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.685+0000 7f2a885b0b80 20 client.15720 populate_metadata read hostname 'smithi052' 2024-06-10T02:55:50.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.685+0000 7f2a885b0b80 1 --2- 172.21.15.52:0/3140130528 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x562472d0e8d0 0x562472d10cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:50.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.685+0000 7f2a885b0b80 4 client.15720 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "57f4a9c1", "release": true, "await": true}] 2024-06-10T02:55:50.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.685+0000 7f2a885b0b80 1 -- 172.21.15.52:0/3140130528 --> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] -- command(tid 0: {"prefix": "quiesce db", "set_id": "57f4a9c1", "release": true, "await": true}) -- 0x562472be23d0 con 0x562472d0e8d0 2024-06-10T02:55:50.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.686+0000 7f2a777fe700 1 --2- 172.21.15.52:0/3140130528 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x562472d0e8d0 0x562472d10cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:50.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.686+0000 7f2a777fe700 1 --2- 172.21.15.52:0/3140130528 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x562472d0e8d0 0x562472d10cf0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dbde45b74bf07f90 server_cookie=91b67c4a805d74b in_seq=0 out_seq=0 2024-06-10T02:55:50.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.686+0000 7f2a56ffd700 10 client.15720 ms_handle_connect on v2:172.21.15.188:6838/310444836 2024-06-10T02:55:50.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.692+0000 7f2a56ffd700 1 -- 172.21.15.52:0/3140130528 <== mds.0 v2:172.21.15.188:6838/310444836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x562472be23d0 con 0x562472d0e8d0 2024-06-10T02:55:50.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.692+0000 7f2a56ffd700 10 client.15720 handle_command_reply: tid=0 2024-06-10T02:55:50.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.692+0000 7f2a885b0b80 1 client.15720 shutdown 2024-06-10T02:55:50.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.692+0000 7f2a885b0b80 1 -- 172.21.15.52:0/3140130528 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562472c8f500 msgr2=0x562472caaa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:55:50.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.692+0000 7f2a885b0b80 1 --2- 172.21.15.52:0/3140130528 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562472c8f500 0x562472caaa70 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f2a60097380 tx=0x7f2a60010880 comp rx=0 tx=0).stop 2024-06-10T02:55:50.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.692+0000 7f2a885b0b80 1 -- 172.21.15.52:0/3140130528 shutdown_connections 2024-06-10T02:55:50.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.692+0000 7f2a885b0b80 1 --2- 172.21.15.52:0/3140130528 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x562472d0e8d0 0x562472d10cf0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:50.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.692+0000 7f2a885b0b80 1 --2- 172.21.15.52:0/3140130528 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562472c8fe20 0x562472caafb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:50.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.692+0000 7f2a885b0b80 1 --2- 172.21.15.52:0/3140130528 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562472c89c50 0x562472c9c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:50.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.692+0000 7f2a885b0b80 1 --2- 172.21.15.52:0/3140130528 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562472c8f500 0x562472caaa70 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:50.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.692+0000 7f2a885b0b80 1 -- 172.21.15.52:0/3140130528 >> 172.21.15.52:0/3140130528 conn(0x562472bb6180 msgr2=0x562472c94ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:55:50.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.692+0000 7f2a56ffd700 0 client.15720 ms_handle_reset on v2:172.21.15.188:6838/310444836 2024-06-10T02:55:50.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.692+0000 7f2a885b0b80 1 -- 172.21.15.52:0/3140130528 shutdown_connections 2024-06-10T02:55:50.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.692+0000 7f2a885b0b80 1 -- 172.21.15.52:0/3140130528 wait complete. 2024-06-10T02:55:50.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.692+0000 7f2a885b0b80 20 client.15720 trim_cache size 0 max 16384 2024-06-10T02:55:50.695 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-10T02:55:50.695 INFO:teuthology.orchestra.run.smithi052.stdout: "epoch": 30, 2024-06-10T02:55:50.695 INFO:teuthology.orchestra.run.smithi052.stdout: "leader": 24484, 2024-06-10T02:55:50.696 INFO:teuthology.orchestra.run.smithi052.stdout: "set_version": 4, 2024-06-10T02:55:50.696 INFO:teuthology.orchestra.run.smithi052.stdout: "sets": { 2024-06-10T02:55:50.696 INFO:teuthology.orchestra.run.smithi052.stdout: "57f4a9c1": { 2024-06-10T02:55:50.696 INFO:teuthology.orchestra.run.smithi052.stdout: "version": 4, 2024-06-10T02:55:50.696 INFO:teuthology.orchestra.run.smithi052.stdout: "age_ref": 0.0, 2024-06-10T02:55:50.696 INFO:teuthology.orchestra.run.smithi052.stdout: "state": { 2024-06-10T02:55:50.696 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "RELEASED", 2024-06-10T02:55:50.696 INFO:teuthology.orchestra.run.smithi052.stdout: "age": 0.0 2024-06-10T02:55:50.696 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T02:55:50.696 INFO:teuthology.orchestra.run.smithi052.stdout: "timeout": 90.0, 2024-06-10T02:55:50.696 INFO:teuthology.orchestra.run.smithi052.stdout: "expiration": 150.0, 2024-06-10T02:55:50.696 INFO:teuthology.orchestra.run.smithi052.stdout: "members": { 2024-06-10T02:55:50.696 INFO:teuthology.orchestra.run.smithi052.stdout: "file:/": { 2024-06-10T02:55:50.696 INFO:teuthology.orchestra.run.smithi052.stdout: "excluded": false, 2024-06-10T02:55:50.697 INFO:teuthology.orchestra.run.smithi052.stdout: "state": { 2024-06-10T02:55:50.697 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "RELEASED", 2024-06-10T02:55:50.697 INFO:teuthology.orchestra.run.smithi052.stdout: "age": 0.0 2024-06-10T02:55:50.697 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T02:55:50.697 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T02:55:50.697 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T02:55:50.697 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T02:55:50.697 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T02:55:50.697 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-10T02:55:50.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.693+0000 7f2a7dd2d700 1 -- 172.21.15.52:0/393482784 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f2a640a2ca0 msgr2=0x7f2a640a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:55:50.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.693+0000 7f2a7dd2d700 1 --2- 172.21.15.52:0/393482784 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f2a640a2ca0 0x7f2a640a5160 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f2a68009550 tx=0x7f2a680091b0 comp rx=0 tx=0).stop 2024-06-10T02:55:50.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.693+0000 7f2a7dd2d700 1 -- 172.21.15.52:0/393482784 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2a78101fd0 msgr2=0x7f2a7819b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:55:50.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.693+0000 7f2a7dd2d700 1 --2- 172.21.15.52:0/393482784 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2a78101fd0 0x7f2a7819b380 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f2a6000b500 tx=0x7f2a6000b810 comp rx=0 tx=0).stop 2024-06-10T02:55:50.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.693+0000 7f2a7dd2d700 1 -- 172.21.15.52:0/393482784 shutdown_connections 2024-06-10T02:55:50.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.693+0000 7f2a7dd2d700 1 --2- 172.21.15.52:0/393482784 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2a781066b0 0x7f2a7819f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:50.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.693+0000 7f2a7dd2d700 1 --2- 172.21.15.52:0/393482784 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2a78101620 0x7f2a7819ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:50.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.693+0000 7f2a7dd2d700 1 --2- 172.21.15.52:0/393482784 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f2a640a2ca0 0x7f2a640a5160 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:50.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.693+0000 7f2a7dd2d700 1 --2- 172.21.15.52:0/393482784 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2a78101fd0 0x7f2a7819b380 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:50.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.693+0000 7f2a7dd2d700 1 -- 172.21.15.52:0/393482784 >> 172.21.15.52:0/393482784 conn(0x7f2a78078340 msgr2=0x7f2a7810cd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:55:50.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.693+0000 7f2a7dd2d700 1 -- 172.21.15.52:0/393482784 shutdown_connections 2024-06-10T02:55:50.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:50.693+0000 7f2a7dd2d700 1 -- 172.21.15.52:0/393482784 wait complete. 2024-06-10T02:55:50.699 INFO:tasks.workunit.client.0.smithi052.stdout:getting blogbench 2024-06-10T02:55:50.699 INFO:tasks.workunit.client.0.smithi052.stderr:+ echo 'getting blogbench' 2024-06-10T02:55:50.699 INFO:tasks.workunit.client.0.smithi052.stderr:+ wget http://download.ceph.com/qa/blogbench-1.0.tar.bz2 2024-06-10T02:55:50.729 INFO:tasks.workunit.client.0.smithi052.stderr:--2024-06-10 02:55:50-- http://download.ceph.com/qa/blogbench-1.0.tar.bz2 2024-06-10T02:55:50.738 INFO:tasks.workunit.client.0.smithi052.stderr:Resolving download.ceph.com (download.ceph.com)... 158.69.68.124, 2607:5300:201:2000::3:58a1 2024-06-10T02:55:50.777 INFO:tasks.workunit.client.0.smithi052.stderr:Connecting to download.ceph.com (download.ceph.com)|158.69.68.124|:80... connected. 2024-06-10T02:55:50.813 INFO:tasks.workunit.client.0.smithi052.stderr:HTTP request sent, awaiting response... 200 OK 2024-06-10T02:55:50.813 INFO:tasks.workunit.client.0.smithi052.stderr:Length: 103355 (101K) [application/octet-stream] 2024-06-10T02:55:50.815 INFO:tasks.workunit.client.0.smithi052.stderr:Saving to: ‘blogbench-1.0.tar.bz2’ 2024-06-10T02:55:50.818 INFO:tasks.workunit.client.0.smithi052.stderr: 2024-06-10T02:55:50.886 INFO:tasks.workunit.client.0.smithi052.stderr: 0K .......... .......... .......... .......... .......... 49% 719K 0s 2024-06-10T02:55:50.888 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:55:50 smithi120 ceph-mon[36634]: pgmap v483: 129 pgs: 129 active+clean; 667 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 9 op/s 2024-06-10T02:55:50.920 INFO:tasks.workunit.client.0.smithi052.stderr: 50K .......... .......... .......... .......... .......... 99% 1.37M 0s 2024-06-10T02:55:50.921 INFO:tasks.workunit.client.0.smithi052.stderr: 100K 100% 1.74T=0.1s 2024-06-10T02:55:50.921 INFO:tasks.workunit.client.0.smithi052.stderr: 2024-06-10T02:55:50.921 INFO:tasks.workunit.client.0.smithi052.stderr:2024-06-10 02:55:50 (961 KB/s) - ‘blogbench-1.0.tar.bz2’ saved [103355/103355] 2024-06-10T02:55:50.921 INFO:tasks.workunit.client.0.smithi052.stderr: 2024-06-10T02:55:50.921 INFO:tasks.workunit.client.0.smithi052.stderr:+ tar -xvf blogbench-1.0.tar.bz2 2024-06-10T02:55:50.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:55:50 smithi188 ceph-mon[36599]: pgmap v483: 129 pgs: 129 active+clean; 667 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 9 op/s 2024-06-10T02:55:50.979 INFO:tasks.workunit.client.0.smithi052.stdout:blogbench-1.0/ 2024-06-10T02:55:50.980 INFO:tasks.workunit.client.0.smithi052.stdout:blogbench-1.0/m4/ 2024-06-10T02:55:50.981 INFO:tasks.workunit.client.0.smithi052.stdout:blogbench-1.0/m4/acx_pthread.m4 2024-06-10T02:55:50.982 INFO:tasks.workunit.client.0.smithi052.stdout:blogbench-1.0/man/ 2024-06-10T02:55:50.983 INFO:tasks.workunit.client.0.smithi052.stdout:blogbench-1.0/man/blogbench.8 2024-06-10T02:55:50.984 INFO:tasks.workunit.client.0.smithi052.stdout:blogbench-1.0/man/Makefile.am 2024-06-10T02:55:50.985 INFO:tasks.workunit.client.0.smithi052.stdout:blogbench-1.0/man/Makefile.in 2024-06-10T02:55:50.986 INFO:tasks.workunit.client.0.smithi052.stdout:blogbench-1.0/src/ 2024-06-10T02:55:50.987 INFO:tasks.workunit.client.0.smithi052.stdout:blogbench-1.0/src/mysnprintf.c 2024-06-10T02:55:50.988 INFO:tasks.workunit.client.0.smithi052.stdout:blogbench-1.0/src/mysnprintf.h 2024-06-10T02:55:50.989 INFO:tasks.workunit.client.0.smithi052.stdout:blogbench-1.0/src/blogbench.c 2024-06-10T02:55:50.990 INFO:tasks.workunit.client.0.smithi052.stdout:blogbench-1.0/src/blogbench.h 2024-06-10T02:55:50.991 INFO:tasks.workunit.client.0.smithi052.stdout:blogbench-1.0/src/fakesnprintf.c 2024-06-10T02:55:50.992 INFO:tasks.workunit.client.0.smithi052.stdout:blogbench-1.0/src/fakesnprintf.h 2024-06-10T02:55:50.993 INFO:tasks.workunit.client.0.smithi052.stdout:blogbench-1.0/src/blogbench_p.h 2024-06-10T02:55:50.994 INFO:tasks.workunit.client.0.smithi052.stdout:blogbench-1.0/src/bsd-getopt_long.c 2024-06-10T02:55:50.995 INFO:tasks.workunit.client.0.smithi052.stdout:blogbench-1.0/src/bsd-getopt_long.h 2024-06-10T02:55:50.996 INFO:tasks.workunit.client.0.smithi052.stdout:blogbench-1.0/src/Makefile.am 2024-06-10T02:55:50.997 INFO:tasks.workunit.client.0.smithi052.stdout:blogbench-1.0/src/Makefile.in 2024-06-10T02:55:50.998 INFO:tasks.workunit.client.0.smithi052.stdout:blogbench-1.0/src/process.c 2024-06-10T02:55:50.999 INFO:tasks.workunit.client.0.smithi052.stdout:blogbench-1.0/src/globals.h 2024-06-10T02:55:51.000 INFO:tasks.workunit.client.0.smithi052.stdout:blogbench-1.0/src/commenter.c 2024-06-10T02:55:51.002 INFO:tasks.workunit.client.0.smithi052.stdout:blogbench-1.0/src/rewriter.c 2024-06-10T02:55:51.003 INFO:tasks.workunit.client.0.smithi052.stdout:blogbench-1.0/src/reader.c 2024-06-10T02:55:51.005 INFO:tasks.workunit.client.0.smithi052.stdout:blogbench-1.0/src/writer.c 2024-06-10T02:55:51.006 INFO:tasks.workunit.client.0.smithi052.stdout:blogbench-1.0/src/helpers.c 2024-06-10T02:55:51.007 INFO:tasks.workunit.client.0.smithi052.stdout:blogbench-1.0/NEWS 2024-06-10T02:55:51.008 INFO:tasks.workunit.client.0.smithi052.stdout:blogbench-1.0/compile 2024-06-10T02:55:51.009 INFO:tasks.workunit.client.0.smithi052.stdout:blogbench-1.0/depcomp 2024-06-10T02:55:51.010 INFO:tasks.workunit.client.0.smithi052.stdout:blogbench-1.0/aclocal.m4 2024-06-10T02:55:51.011 INFO:tasks.workunit.client.0.smithi052.stdout:blogbench-1.0/README 2024-06-10T02:55:51.012 INFO:tasks.workunit.client.0.smithi052.stdout:blogbench-1.0/configure 2024-06-10T02:55:51.016 INFO:tasks.workunit.client.0.smithi052.stdout:blogbench-1.0/configure.ac 2024-06-10T02:55:51.017 INFO:tasks.workunit.client.0.smithi052.stdout:blogbench-1.0/config.guess 2024-06-10T02:55:51.018 INFO:tasks.workunit.client.0.smithi052.stdout:blogbench-1.0/install-sh 2024-06-10T02:55:51.019 INFO:tasks.workunit.client.0.smithi052.stdout:blogbench-1.0/config.sub 2024-06-10T02:55:51.020 INFO:tasks.workunit.client.0.smithi052.stdout:blogbench-1.0/missing 2024-06-10T02:55:51.021 INFO:tasks.workunit.client.0.smithi052.stdout:blogbench-1.0/mkinstalldirs 2024-06-10T02:55:51.022 INFO:tasks.workunit.client.0.smithi052.stdout:blogbench-1.0/Makefile.am 2024-06-10T02:55:51.023 INFO:tasks.workunit.client.0.smithi052.stdout:blogbench-1.0/Makefile.in 2024-06-10T02:55:51.024 INFO:tasks.workunit.client.0.smithi052.stdout:blogbench-1.0/config.h.in 2024-06-10T02:55:51.025 INFO:tasks.workunit.client.0.smithi052.stdout:blogbench-1.0/AUTHORS 2024-06-10T02:55:51.026 INFO:tasks.workunit.client.0.smithi052.stdout:blogbench-1.0/INSTALL 2024-06-10T02:55:51.027 INFO:tasks.workunit.client.0.smithi052.stdout:blogbench-1.0/ChangeLog 2024-06-10T02:55:51.028 INFO:tasks.workunit.client.0.smithi052.stdout:blogbench-1.0/COPYING 2024-06-10T02:55:51.029 INFO:tasks.workunit.client.0.smithi052.stderr:+ cd blogbench-1.0/ 2024-06-10T02:55:51.029 INFO:tasks.workunit.client.0.smithi052.stderr:+ echo 'making blogbench' 2024-06-10T02:55:51.029 INFO:tasks.workunit.client.0.smithi052.stdout:making blogbench 2024-06-10T02:55:51.030 INFO:tasks.workunit.client.0.smithi052.stderr:+ ./configure 2024-06-10T02:55:51.175 INFO:tasks.workunit.client.0.smithi052.stdout:checking for a BSD-compatible install... /usr/bin/install -c 2024-06-10T02:55:51.381 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '57f4a9c1', total seconds elapsed: 35.6 2024-06-10T02:55:51.381 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 30.0 2024-06-10T02:55:51.382 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T02:55:51.634 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:55:52.184 INFO:tasks.workunit.client.0.smithi052.stdout:checking whether build environment is sane... yes 2024-06-10T02:55:52.193 INFO:tasks.workunit.client.0.smithi052.stdout:checking for gawk... gawk 2024-06-10T02:55:52.209 INFO:tasks.workunit.client.0.smithi052.stdout:checking whether make sets $(MAKE)... yes 2024-06-10T02:55:52.219 INFO:tasks.workunit.client.0.smithi052.stdout:checking for gcc... gcc 2024-06-10T02:55:52.269 INFO:tasks.workunit.client.0.smithi052.stdout:checking for C compiler default output file name... a.out 2024-06-10T02:55:52.271 INFO:tasks.workunit.client.0.smithi052.stdout:checking whether the C compiler works... yes 2024-06-10T02:55:52.273 INFO:tasks.workunit.client.0.smithi052.stdout:checking whether we are cross compiling... no 2024-06-10T02:55:52.310 INFO:tasks.workunit.client.0.smithi052.stdout:checking for suffix of executables... 2024-06-10T02:55:52.351 INFO:tasks.workunit.client.0.smithi052.stdout:checking for suffix of object files... o 2024-06-10T02:55:52.385 INFO:tasks.workunit.client.0.smithi052.stdout:checking whether we are using the GNU C compiler... yes 2024-06-10T02:55:52.424 INFO:tasks.workunit.client.0.smithi052.stdout:checking whether gcc accepts -g... yes 2024-06-10T02:55:52.611 INFO:tasks.workunit.client.0.smithi052.stdout:checking for gcc option to accept ANSI C... none needed 2024-06-10T02:55:52.680 INFO:tasks.workunit.client.0.smithi052.stdout:checking for style of include used by make... GNU 2024-06-10T02:55:52.726 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:55:52 smithi052 ceph-mon[32362]: pgmap v484: 129 pgs: 129 active+clean; 667 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 9 op/s 2024-06-10T02:55:52.726 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:55:52 smithi052 ceph-mon[32362]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-10T02:55:52.783 INFO:tasks.workunit.client.0.smithi052.stdout:checking dependency style of gcc... gcc3 2024-06-10T02:55:52.783 INFO:tasks.workunit.client.0.smithi052.stdout:checking for a BSD-compatible install... /usr/bin/install -c 2024-06-10T02:55:52.851 INFO:tasks.workunit.client.0.smithi052.stdout:checking how to run the C preprocessor... gcc -E 2024-06-10T02:55:52.888 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:55:52 smithi120 ceph-mon[36634]: pgmap v484: 129 pgs: 129 active+clean; 667 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 9 op/s 2024-06-10T02:55:52.888 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:55:52 smithi120 ceph-mon[36634]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-10T02:55:52.895 INFO:tasks.workunit.client.0.smithi052.stdout:checking for egrep... grep -E 2024-06-10T02:55:52.905 INFO:tasks.workunit.client.0.smithi052.stdout:checking for AIX... no 2024-06-10T02:55:52.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:55:52 smithi188 ceph-mon[36599]: pgmap v484: 129 pgs: 129 active+clean; 667 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 9 op/s 2024-06-10T02:55:52.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:55:52 smithi188 ceph-mon[36599]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-10T02:55:52.974 INFO:tasks.workunit.client.0.smithi052.stdout:checking for library containing strerror... none required 2024-06-10T02:55:53.207 INFO:tasks.workunit.client.0.smithi052.stdout:checking for ANSI C header files... yes 2024-06-10T02:55:53.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:53.246+0000 7ff1c78ed700 1 -- 172.21.15.52:0/3697482501 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff1c006ab30 msgr2=0x7ff1c010f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:55:53.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:53.246+0000 7ff1c78ed700 1 --2- 172.21.15.52:0/3697482501 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff1c006ab30 0x7ff1c010f320 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7ff1b800b0a0 tx=0x7ff1b800b3b0 comp rx=0 tx=0).stop 2024-06-10T02:55:53.247 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:53.247+0000 7ff1c78ed700 1 -- 172.21.15.52:0/3697482501 shutdown_connections 2024-06-10T02:55:53.247 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:53.247+0000 7ff1c78ed700 1 --2- 172.21.15.52:0/3697482501 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff1c006ab30 0x7ff1c010f320 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:53.247 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:53.247+0000 7ff1c78ed700 1 --2- 172.21.15.52:0/3697482501 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff1c0108500 0x7ff1c01088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:53.247 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:53.247+0000 7ff1c78ed700 1 --2- 172.21.15.52:0/3697482501 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff1c006a170 0x7ff1c006a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:53.247 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:53.247+0000 7ff1c78ed700 1 -- 172.21.15.52:0/3697482501 >> 172.21.15.52:0/3697482501 conn(0x7ff1c0078340 msgr2=0x7ff1c0078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:55:53.247 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:53.247+0000 7ff1c78ed700 1 -- 172.21.15.52:0/3697482501 shutdown_connections 2024-06-10T02:55:53.247 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:53.247+0000 7ff1c78ed700 1 -- 172.21.15.52:0/3697482501 wait complete. 2024-06-10T02:55:53.248 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:53.247+0000 7ff1c78ed700 1 Processor -- start 2024-06-10T02:55:53.248 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:53.247+0000 7ff1c78ed700 1 -- start start 2024-06-10T02:55:53.248 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:53.248+0000 7ff1c78ed700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff1c006a170 0x7ff1c019ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:53.248 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:53.248+0000 7ff1c78ed700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff1c006ab30 0x7ff1c019ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:53.248 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:53.248+0000 7ff1c78ed700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff1c0108500 0x7ff1c0198b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:53.248 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:53.248+0000 7ff1c78ed700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ff1c0112830 con 0x7ff1c0108500 2024-06-10T02:55:53.248 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:53.248+0000 7ff1c78ed700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff1c01126b0 con 0x7ff1c006ab30 2024-06-10T02:55:53.248 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:53.248+0000 7ff1c78ed700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ff1c01129b0 con 0x7ff1c006a170 2024-06-10T02:55:53.249 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:53.248+0000 7ff1c5689700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff1c006a170 0x7ff1c019ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:53.249 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:53.248+0000 7ff1c5e8a700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff1c0108500 0x7ff1c0198b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:53.249 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:53.248+0000 7ff1c5689700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff1c006a170 0x7ff1c019ea40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.52:49032/0 (socket says 172.21.15.52:49032) 2024-06-10T02:55:53.249 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:53.248+0000 7ff1c5689700 1 -- 172.21.15.52:0/3078975498 learned_addr learned my addr 172.21.15.52:0/3078975498 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:55:53.249 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:53.248+0000 7ff1c4e88700 1 --2- 172.21.15.52:0/3078975498 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff1c006ab30 0x7ff1c019ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:53.249 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:53.248+0000 7ff1c4e88700 1 -- 172.21.15.52:0/3078975498 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff1c006a170 msgr2=0x7ff1c019ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:55:53.249 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:53.248+0000 7ff1c4e88700 1 --2- 172.21.15.52:0/3078975498 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff1c006a170 0x7ff1c019ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:53.249 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:53.248+0000 7ff1c4e88700 1 -- 172.21.15.52:0/3078975498 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff1c0108500 msgr2=0x7ff1c0198b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:55:53.249 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:53.248+0000 7ff1c4e88700 1 --2- 172.21.15.52:0/3078975498 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff1c0108500 0x7ff1c0198b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:53.249 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:53.248+0000 7ff1c4e88700 1 -- 172.21.15.52:0/3078975498 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1c01993f0 con 0x7ff1c006ab30 2024-06-10T02:55:53.249 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:53.248+0000 7ff1c5689700 1 --2- 172.21.15.52:0/3078975498 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff1c006a170 0x7ff1c019ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T02:55:53.249 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:53.249+0000 7ff1c4e88700 1 --2- 172.21.15.52:0/3078975498 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff1c006ab30 0x7ff1c019ef80 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ff1b000c970 tx=0x7ff1b000cc80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:55:53.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:53.249+0000 7ff1b67fc700 1 -- 172.21.15.52:0/3078975498 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1b0014070 con 0x7ff1c006ab30 2024-06-10T02:55:53.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:53.249+0000 7ff1c78ed700 1 -- 172.21.15.52:0/3078975498 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1c01996e0 con 0x7ff1c006ab30 2024-06-10T02:55:53.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:53.249+0000 7ff1b67fc700 1 -- 172.21.15.52:0/3078975498 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff1b0004440 con 0x7ff1c006ab30 2024-06-10T02:55:53.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:53.249+0000 7ff1b67fc700 1 -- 172.21.15.52:0/3078975498 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1b0007b60 con 0x7ff1c006ab30 2024-06-10T02:55:53.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:53.249+0000 7ff1c78ed700 1 -- 172.21.15.52:0/3078975498 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1c01a59c0 con 0x7ff1c006ab30 2024-06-10T02:55:53.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:53.250+0000 7ff1c78ed700 1 -- 172.21.15.52:0/3078975498 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1c010a9e0 con 0x7ff1c006ab30 2024-06-10T02:55:53.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:53.252+0000 7ff1b67fc700 1 -- 172.21.15.52:0/3078975498 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7ff1b0017070 con 0x7ff1c006ab30 2024-06-10T02:55:53.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:53.252+0000 7ff1b67fc700 1 --2- 172.21.15.52:0/3078975498 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7ff1ac0a2ca0 0x7ff1ac0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:53.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:53.252+0000 7ff1c5689700 1 --2- 172.21.15.52:0/3078975498 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7ff1ac0a2ca0 0x7ff1ac0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:53.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:53.252+0000 7ff1b67fc700 1 -- 172.21.15.52:0/3078975498 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7ff1b00cefb0 con 0x7ff1c006ab30 2024-06-10T02:55:53.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:53.253+0000 7ff1c5689700 1 --2- 172.21.15.52:0/3078975498 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7ff1ac0a2ca0 0x7ff1ac0a5160 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7ff1bc009ea0 tx=0x7ff1bc009290 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:55:53.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:53.256+0000 7ff1b67fc700 1 -- 172.21.15.52:0/3078975498 <== mon.2 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7ff1b001e020 con 0x7ff1c006ab30 2024-06-10T02:55:53.266 INFO:tasks.workunit.client.0.smithi052.stdout:checking whether time.h and sys/time.h may both be included... yes 2024-06-10T02:55:53.315 INFO:tasks.workunit.client.0.smithi052.stdout:checking for sys/types.h... yes 2024-06-10T02:55:53.364 INFO:tasks.workunit.client.0.smithi052.stdout:checking for sys/stat.h... yes 2024-06-10T02:55:53.416 INFO:tasks.workunit.client.0.smithi052.stdout:checking for stdlib.h... yes 2024-06-10T02:55:53.470 INFO:tasks.workunit.client.0.smithi052.stdout:checking for string.h... yes 2024-06-10T02:55:53.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:53.517+0000 7ff1c78ed700 1 -- 172.21.15.52:0/3078975498 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff1c0199fd0 con 0x7ff1c006ab30 2024-06-10T02:55:53.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:53.518+0000 7ff1b67fc700 1 -- 172.21.15.52:0/3078975498 <== mon.2 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11939 (secure 0 0 0) 0x7ff1b009bbe0 con 0x7ff1c006ab30 2024-06-10T02:55:53.518 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-10T02:55:53.518 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/889673247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":889673247},{"type":"v1","addr":"172.21.15.52:6835","nonce":889673247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/3615663511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":3615663511},{"type":"v1","addr":"172.21.15.52:6837","nonce":3615663511}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":31,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T02:50:03.889766+0000","modified":"2024-06-10T02:55:51.192225+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24484,"mds_1":24469,"mds_2":24463,"mds_3":24449,"mds_4":24428},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14571":{"gid":14571,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6839/3059599042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":3059599042},{"type":"v1","addr":"172.21.15.52:6839","nonce":3059599042}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14583":{"gid":14583,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6841/3039275521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":3039275521},{"type":"v1","addr":"172.21.15.52:6841","nonce":3039275521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24383":{"gid":24383,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6833/2609002896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":2609002896},{"type":"v1","addr":"172.21.15.120:6833","nonce":2609002896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6835/1996370288","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":1996370288},{"type":"v1","addr":"172.21.15.120:6835","nonce":1996370288}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6833/529564979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":529564979},{"type":"v1","addr":"172.21.15.188:6833","nonce":529564979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":25,"addr":"172.21.15.120:6837/4132440952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":4132440952},{"type":"v1","addr":"172.21.15.120:6837","nonce":4132440952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"b","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.120:6839/1672436487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1672436487},{"type":"v1","addr":"172.21.15.120:6839","nonce":1672436487}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":2,"incarnation":22,"state":"up:active","state_seq":26,"addr":"172.21.15.188:6835/3904544259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":3904544259},{"type":"v1","addr":"172.21.15.188:6835","nonce":3904544259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.188:6837/819317726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":819317726},{"type":"v1","addr":"172.21.15.188:6837","nonce":819317726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.188:6839/310444836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":310444836},{"type":"v1","addr":"172.21.15.188:6839","nonce":310444836}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24484,"qdb_cluster":[24469,24449,24428,24463,24484]},"id":1}]} 2024-06-10T02:55:53.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:53.520+0000 7ff1c78ed700 1 -- 172.21.15.52:0/3078975498 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7ff1ac0a2ca0 msgr2=0x7ff1ac0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:55:53.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:53.520+0000 7ff1c78ed700 1 --2- 172.21.15.52:0/3078975498 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7ff1ac0a2ca0 0x7ff1ac0a5160 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7ff1bc009ea0 tx=0x7ff1bc009290 comp rx=0 tx=0).stop 2024-06-10T02:55:53.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:53.520+0000 7ff1c78ed700 1 -- 172.21.15.52:0/3078975498 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff1c006ab30 msgr2=0x7ff1c019ef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:55:53.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:53.520+0000 7ff1c78ed700 1 --2- 172.21.15.52:0/3078975498 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff1c006ab30 0x7ff1c019ef80 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ff1b000c970 tx=0x7ff1b000cc80 comp rx=0 tx=0).stop 2024-06-10T02:55:53.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:53.520+0000 7ff1c78ed700 1 -- 172.21.15.52:0/3078975498 shutdown_connections 2024-06-10T02:55:53.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:53.520+0000 7ff1c78ed700 1 --2- 172.21.15.52:0/3078975498 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7ff1ac0a2ca0 0x7ff1ac0a5160 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:53.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:53.520+0000 7ff1c78ed700 1 --2- 172.21.15.52:0/3078975498 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff1c0108500 0x7ff1c0198b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:53.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:53.520+0000 7ff1c78ed700 1 --2- 172.21.15.52:0/3078975498 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff1c006a170 0x7ff1c019ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:53.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:53.520+0000 7ff1c78ed700 1 --2- 172.21.15.52:0/3078975498 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff1c006ab30 0x7ff1c019ef80 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:53.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:53.520+0000 7ff1c78ed700 1 -- 172.21.15.52:0/3078975498 >> 172.21.15.52:0/3078975498 conn(0x7ff1c0078340 msgr2=0x7ff1c00fff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:55:53.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:53.520+0000 7ff1c78ed700 1 -- 172.21.15.52:0/3078975498 shutdown_connections 2024-06-10T02:55:53.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:53.520+0000 7ff1c78ed700 1 -- 172.21.15.52:0/3078975498 wait complete. 2024-06-10T02:55:53.521 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 31 2024-06-10T02:55:53.522 INFO:tasks.workunit.client.0.smithi052.stdout:checking for memory.h... yes 2024-06-10T02:55:53.577 INFO:tasks.workunit.client.0.smithi052.stdout:checking for strings.h... yes 2024-06-10T02:55:53.638 INFO:tasks.workunit.client.0.smithi052.stdout:checking for inttypes.h... yes 2024-06-10T02:55:53.702 INFO:tasks.workunit.client.0.smithi052.stdout:checking for stdint.h... yes 2024-06-10T02:55:53.758 INFO:tasks.workunit.client.0.smithi052.stdout:checking for unistd.h... yes 2024-06-10T02:55:53.771 INFO:tasks.workunit.client.0.smithi052.stdout:checking for memory.h... (cached) yes 2024-06-10T02:55:53.776 INFO:tasks.workunit.client.0.smithi052.stdout:checking for strings.h... (cached) yes 2024-06-10T02:55:53.781 INFO:tasks.workunit.client.0.smithi052.stdout:checking for unistd.h... (cached) yes 2024-06-10T02:55:53.859 INFO:tasks.workunit.client.0.smithi052.stdout:checking locale.h usability... yes 2024-06-10T02:55:53.878 INFO:tasks.workunit.client.0.smithi052.stdout:checking locale.h presence... yes 2024-06-10T02:55:53.879 INFO:tasks.workunit.client.0.smithi052.stdout:checking for locale.h... yes 2024-06-10T02:55:53.894 INFO:tasks.workunit.client.0.smithi052.stdout:checking for stdlib.h... (cached) yes 2024-06-10T02:55:53.899 INFO:tasks.workunit.client.0.smithi052.stdout:checking for string.h... (cached) yes 2024-06-10T02:55:54.027 INFO:tasks.workunit.client.0.smithi052.stdout:checking sys/socket.h usability... yes 2024-06-10T02:55:54.051 INFO:tasks.workunit.client.0.smithi052.stdout:checking sys/socket.h presence... yes 2024-06-10T02:55:54.052 INFO:tasks.workunit.client.0.smithi052.stdout:checking for sys/socket.h... yes 2024-06-10T02:55:54.110 INFO:tasks.workunit.client.0.smithi052.stdout:checking sys/time.h usability... yes 2024-06-10T02:55:54.132 INFO:tasks.workunit.client.0.smithi052.stdout:checking sys/time.h presence... yes 2024-06-10T02:55:54.133 INFO:tasks.workunit.client.0.smithi052.stdout:checking for sys/time.h... yes 2024-06-10T02:55:54.135 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24484 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0' 2024-06-10T02:55:54.135 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24484 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0 2024-06-10T02:55:54.194 INFO:tasks.workunit.client.0.smithi052.stdout:checking fcntl.h usability... yes 2024-06-10T02:55:54.217 INFO:tasks.workunit.client.0.smithi052.stdout:checking fcntl.h presence... yes 2024-06-10T02:55:54.218 INFO:tasks.workunit.client.0.smithi052.stdout:checking for fcntl.h... yes 2024-06-10T02:55:54.278 INFO:tasks.workunit.client.0.smithi052.stdout:checking sys/fcntl.h usability... yes 2024-06-10T02:55:54.300 INFO:tasks.workunit.client.0.smithi052.stdout:checking sys/fcntl.h presence... yes 2024-06-10T02:55:54.301 INFO:tasks.workunit.client.0.smithi052.stdout:checking for sys/fcntl.h... yes 2024-06-10T02:55:54.384 INFO:tasks.workunit.client.0.smithi052.stdout:checking sys/param.h usability... yes 2024-06-10T02:55:54.394 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:55:54.407 INFO:tasks.workunit.client.0.smithi052.stdout:checking sys/param.h presence... yes 2024-06-10T02:55:54.408 INFO:tasks.workunit.client.0.smithi052.stdout:checking for sys/param.h... yes 2024-06-10T02:55:54.467 INFO:tasks.workunit.client.0.smithi052.stdout:checking ioctl.h usability... no 2024-06-10T02:55:54.488 INFO:tasks.workunit.client.0.smithi052.stdout:checking ioctl.h presence... no 2024-06-10T02:55:54.489 INFO:tasks.workunit.client.0.smithi052.stdout:checking for ioctl.h... no 2024-06-10T02:55:54.542 INFO:tasks.workunit.client.0.smithi052.stdout:checking sys/ioctl.h usability... yes 2024-06-10T02:55:54.563 INFO:tasks.workunit.client.0.smithi052.stdout:checking sys/ioctl.h presence... yes 2024-06-10T02:55:54.564 INFO:tasks.workunit.client.0.smithi052.stdout:checking for sys/ioctl.h... yes 2024-06-10T02:55:54.617 INFO:tasks.workunit.client.0.smithi052.stdout:checking return type of signal handlers... void 2024-06-10T02:55:54.665 INFO:tasks.workunit.client.0.smithi052.stdout:checking for size_t... yes 2024-06-10T02:55:54.707 INFO:tasks.workunit.client.0.smithi052.stdout:checking for dev_t... yes 2024-06-10T02:55:54.758 INFO:tasks.workunit.client.0.smithi052.stdout:checking for short... yes 2024-06-10T02:55:54.823 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:55:54 smithi052 ceph-mon[32362]: pgmap v485: 129 pgs: 129 active+clean; 804 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-06-10T02:55:54.823 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:55:54 smithi052 ceph-mon[32362]: from='client.? 172.21.15.52:0/3078975498' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T02:55:54.860 INFO:tasks.workunit.client.0.smithi052.stdout:checking size of short... 2 2024-06-10T02:55:54.888 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:55:54 smithi120 ceph-mon[36634]: pgmap v485: 129 pgs: 129 active+clean; 804 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-06-10T02:55:54.888 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:55:54 smithi120 ceph-mon[36634]: from='client.? 172.21.15.52:0/3078975498' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T02:55:54.905 INFO:tasks.workunit.client.0.smithi052.stdout:checking for int... yes 2024-06-10T02:55:54.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:55:54 smithi188 ceph-mon[36599]: pgmap v485: 129 pgs: 129 active+clean; 804 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-06-10T02:55:54.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:55:54 smithi188 ceph-mon[36599]: from='client.? 172.21.15.52:0/3078975498' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T02:55:54.977 INFO:tasks.workunit.client.0.smithi052.stdout:checking size of int... 4 2024-06-10T02:55:55.023 INFO:tasks.workunit.client.0.smithi052.stdout:checking for long... yes 2024-06-10T02:55:55.094 INFO:tasks.workunit.client.0.smithi052.stdout:checking size of long... 8 2024-06-10T02:55:55.147 INFO:tasks.workunit.client.0.smithi052.stdout:checking for long long... yes 2024-06-10T02:55:55.222 INFO:tasks.workunit.client.0.smithi052.stdout:checking size of long long... 8 2024-06-10T02:55:55.267 INFO:tasks.workunit.client.0.smithi052.stdout:checking for ssize_t... yes 2024-06-10T02:55:55.319 INFO:tasks.workunit.client.0.smithi052.stdout:checking for sig_atomic_t... yes 2024-06-10T02:55:55.445 INFO:tasks.workunit.client.0.smithi052.stdout:checking whether gcc needs -traditional... no 2024-06-10T02:55:55.492 INFO:tasks.workunit.client.0.smithi052.stdout:checking for an ANSI C-conforming const... yes 2024-06-10T02:55:55.516 INFO:tasks.workunit.client.0.smithi052.stdout:checking for inline... inline 2024-06-10T02:55:55.579 INFO:tasks.workunit.client.0.smithi052.stdout:checking for working alloca.h... yes 2024-06-10T02:55:55.628 INFO:tasks.workunit.client.0.smithi052.stdout:checking for alloca... yes 2024-06-10T02:55:55.705 INFO:tasks.workunit.client.0.smithi052.stdout:checking for working memcmp... yes 2024-06-10T02:55:55.764 INFO:tasks.workunit.client.0.smithi052.stdout:checking for strftime... yes 2024-06-10T02:55:55.844 INFO:tasks.workunit.client.0.smithi052.stdout:checking whether lstat dereferences a symlink specified with a trailing slash... yes 2024-06-10T02:55:55.906 INFO:tasks.workunit.client.0.smithi052.stdout:checking whether stat accepts an empty string... no 2024-06-10T02:55:55.963 INFO:tasks.workunit.client.0.smithi052.stdout:checking for vprintf... yes 2024-06-10T02:55:56.085 INFO:tasks.workunit.client.0.smithi052.stdout:checking for _doprnt... no 2024-06-10T02:55:56.108 INFO:tasks.workunit.client.0.smithi052.stdout:checking build system type... x86_64-unknown-linux-gnu 2024-06-10T02:55:56.120 INFO:tasks.workunit.client.0.smithi052.stdout:checking host system type... x86_64-unknown-linux-gnu 2024-06-10T02:55:56.212 INFO:tasks.workunit.client.0.smithi052.stdout:checking for the pthreads library -lpthreads... no 2024-06-10T02:55:56.286 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.286+0000 7fc18ec3e700 1 -- 172.21.15.52:0/2346660064 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc188100a80 msgr2=0x7fc188100f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:55:56.286 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.286+0000 7fc18ec3e700 1 --2- 172.21.15.52:0/2346660064 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc188100a80 0x7fc188100f00 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fc174009990 tx=0x7fc174009ca0 comp rx=0 tx=0).stop 2024-06-10T02:55:56.286 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.287+0000 7fc18ec3e700 1 -- 172.21.15.52:0/2346660064 shutdown_connections 2024-06-10T02:55:56.286 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.287+0000 7fc18ec3e700 1 --2- 172.21.15.52:0/2346660064 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc188101440 0x7fc18810f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:56.287 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.287+0000 7fc18ec3e700 1 --2- 172.21.15.52:0/2346660064 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc188107450 0x7fc188107830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:56.287 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.287+0000 7fc18ec3e700 1 --2- 172.21.15.52:0/2346660064 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc188100a80 0x7fc188100f00 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:56.287 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.287+0000 7fc18ec3e700 1 -- 172.21.15.52:0/2346660064 >> 172.21.15.52:0/2346660064 conn(0x7fc188077a30 msgr2=0x7fc1880fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:55:56.287 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.287+0000 7fc18ec3e700 1 -- 172.21.15.52:0/2346660064 shutdown_connections 2024-06-10T02:55:56.287 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.287+0000 7fc18ec3e700 1 -- 172.21.15.52:0/2346660064 wait complete. 2024-06-10T02:55:56.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.287+0000 7fc18ec3e700 1 Processor -- start 2024-06-10T02:55:56.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.288+0000 7fc18ec3e700 1 -- start start 2024-06-10T02:55:56.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.288+0000 7fc18ec3e700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc188100a80 0x7fc18819ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:56.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.288+0000 7fc18ec3e700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc188101440 0x7fc18819b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:56.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.288+0000 7fc18ec3e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc188107450 0x7fc18819f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:56.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.288+0000 7fc18ec3e700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fc1881127e0 con 0x7fc188107450 2024-06-10T02:55:56.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.288+0000 7fc18ec3e700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc188112660 con 0x7fc188100a80 2024-06-10T02:55:56.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.288+0000 7fc18ec3e700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc188112960 con 0x7fc188101440 2024-06-10T02:55:56.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.288+0000 7fc18d1db700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc188107450 0x7fc18819f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:56.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.288+0000 7fc18c9da700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc188100a80 0x7fc18819ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:56.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.288+0000 7fc18d1db700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc188107450 0x7fc18819f5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:41846/0 (socket says 172.21.15.52:41846) 2024-06-10T02:55:56.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.288+0000 7fc18d1db700 1 -- 172.21.15.52:0/1018167810 learned_addr learned my addr 172.21.15.52:0/1018167810 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:55:56.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.288+0000 7fc187fff700 1 --2- 172.21.15.52:0/1018167810 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc188101440 0x7fc18819b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:56.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.288+0000 7fc18c9da700 1 -- 172.21.15.52:0/1018167810 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc188101440 msgr2=0x7fc18819b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:55:56.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.288+0000 7fc18c9da700 1 --2- 172.21.15.52:0/1018167810 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc188101440 0x7fc18819b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:56.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.288+0000 7fc18c9da700 1 -- 172.21.15.52:0/1018167810 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc188107450 msgr2=0x7fc18819f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:55:56.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.288+0000 7fc18c9da700 1 --2- 172.21.15.52:0/1018167810 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc188107450 0x7fc18819f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:56.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.288+0000 7fc18c9da700 1 -- 172.21.15.52:0/1018167810 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc18819fd20 con 0x7fc188100a80 2024-06-10T02:55:56.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.288+0000 7fc187fff700 1 --2- 172.21.15.52:0/1018167810 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc188101440 0x7fc18819b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T02:55:56.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.288+0000 7fc18d1db700 1 --2- 172.21.15.52:0/1018167810 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc188107450 0x7fc18819f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T02:55:56.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.289+0000 7fc18c9da700 1 --2- 172.21.15.52:0/1018167810 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc188100a80 0x7fc18819ad10 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fc17c00d750 tx=0x7fc17c00da60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:55:56.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.289+0000 7fc185ffb700 1 -- 172.21.15.52:0/1018167810 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc17c014030 con 0x7fc188100a80 2024-06-10T02:55:56.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.289+0000 7fc185ffb700 1 -- 172.21.15.52:0/1018167810 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc17c004500 con 0x7fc188100a80 2024-06-10T02:55:56.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.289+0000 7fc18ec3e700 1 -- 172.21.15.52:0/1018167810 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1881a0010 con 0x7fc188100a80 2024-06-10T02:55:56.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.289+0000 7fc185ffb700 1 -- 172.21.15.52:0/1018167810 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc17c005420 con 0x7fc188100a80 2024-06-10T02:55:56.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.289+0000 7fc18ec3e700 1 -- 172.21.15.52:0/1018167810 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1881a7890 con 0x7fc188100a80 2024-06-10T02:55:56.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.292+0000 7fc185ffb700 1 -- 172.21.15.52:0/1018167810 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fc17c004750 con 0x7fc188100a80 2024-06-10T02:55:56.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.293+0000 7fc1994c1b80 1 Processor -- start 2024-06-10T02:55:56.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.293+0000 7fc185ffb700 1 --2- 172.21.15.52:0/1018167810 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fc1780a2cc0 0x7fc1780a5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:56.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.293+0000 7fc185ffb700 1 -- 172.21.15.52:0/1018167810 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7fc17c0cf0b0 con 0x7fc188100a80 2024-06-10T02:55:56.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.293+0000 7fc1994c1b80 1 -- start start 2024-06-10T02:55:56.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.294+0000 7fc18d1db700 1 --2- 172.21.15.52:0/1018167810 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fc1780a2cc0 0x7fc1780a5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:56.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.294+0000 7fc1994c1b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55eb2eb82590 0x55eb2eb82970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:56.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.294+0000 7fc1994c1b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55eb2eb82eb0 0x55eb2eb7c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:56.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.294+0000 7fc1994c1b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eb2eb7cce0 0x55eb2eb7d100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:56.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.294+0000 7fc1994c1b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55eb2ea6f0f0 con 0x55eb2eb82590 2024-06-10T02:55:56.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.294+0000 7fc1994c1b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55eb2eacd9e0 con 0x55eb2eb82eb0 2024-06-10T02:55:56.298 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.294+0000 7fc1994c1b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55eb2ea355f0 con 0x55eb2eb7cce0 2024-06-10T02:55:56.298 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.294+0000 7fc18d1db700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eb2eb7cce0 0x55eb2eb7d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:56.298 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.294+0000 7fc187fff700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55eb2eb82eb0 0x55eb2eb7c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:56.298 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.294+0000 7fc187fff700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55eb2eb82eb0 0x55eb2eb7c7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.52:34170/0 (socket says 172.21.15.52:34170) 2024-06-10T02:55:56.298 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.294+0000 7fc187fff700 1 -- 172.21.15.52:0/4188660774 learned_addr learned my addr 172.21.15.52:0/4188660774 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:55:56.298 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.294+0000 7fc18c9da700 1 --2- 172.21.15.52:0/4188660774 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55eb2eb82590 0x55eb2eb82970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:56.298 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.294+0000 7fc187fff700 1 -- 172.21.15.52:0/4188660774 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eb2eb7cce0 msgr2=0x55eb2eb7d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:55:56.298 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.294+0000 7fc187fff700 1 --2- 172.21.15.52:0/4188660774 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eb2eb7cce0 0x55eb2eb7d100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:56.299 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.294+0000 7fc187fff700 1 -- 172.21.15.52:0/4188660774 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55eb2eb82590 msgr2=0x55eb2eb82970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:55:56.299 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.294+0000 7fc18d1db700 1 --2- 172.21.15.52:0/1018167810 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fc1780a2cc0 0x7fc1780a5180 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fc18000cb70 tx=0x7fc18000cf30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:55:56.299 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.294+0000 7fc187fff700 1 --2- 172.21.15.52:0/4188660774 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55eb2eb82590 0x55eb2eb82970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:56.299 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.294+0000 7fc187fff700 1 -- 172.21.15.52:0/4188660774 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb2ea1ca60 con 0x55eb2eb82eb0 2024-06-10T02:55:56.299 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.295+0000 7fc187fff700 1 --2- 172.21.15.52:0/4188660774 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55eb2eb82eb0 0x55eb2eb7c7a0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fc174009990 tx=0x7fc174005c70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d32b41fd2d48f33a server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:55:56.299 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.295+0000 7fc172ffd700 1 -- 172.21.15.52:0/4188660774 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc174028070 con 0x55eb2eb82eb0 2024-06-10T02:55:56.299 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.295+0000 7fc172ffd700 1 -- 172.21.15.52:0/4188660774 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc174003780 con 0x55eb2eb82eb0 2024-06-10T02:55:56.299 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.295+0000 7fc172ffd700 1 -- 172.21.15.52:0/4188660774 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1740056c0 con 0x55eb2eb82eb0 2024-06-10T02:55:56.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.296+0000 7fc1994c1b80 1 -- 172.21.15.52:0/4188660774 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55eb2eb82eb0 msgr2=0x55eb2eb7c7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:55:56.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.296+0000 7fc1994c1b80 1 --2- 172.21.15.52:0/4188660774 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55eb2eb82eb0 0x55eb2eb7c7a0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fc174009990 tx=0x7fc174005c70 comp rx=0 tx=0).stop 2024-06-10T02:55:56.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.296+0000 7fc1994c1b80 1 -- 172.21.15.52:0/4188660774 shutdown_connections 2024-06-10T02:55:56.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.296+0000 7fc1994c1b80 1 --2- 172.21.15.52:0/4188660774 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eb2eb7cce0 0x55eb2eb7d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:56.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.296+0000 7fc1994c1b80 1 --2- 172.21.15.52:0/4188660774 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55eb2eb82590 0x55eb2eb82970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:56.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.296+0000 7fc1994c1b80 1 --2- 172.21.15.52:0/4188660774 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55eb2eb82eb0 0x55eb2eb7c7a0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:56.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.296+0000 7fc1994c1b80 1 -- 172.21.15.52:0/4188660774 >> 172.21.15.52:0/4188660774 conn(0x55eb2eaa9210 msgr2=0x55eb2eb77a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:55:56.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.296+0000 7fc1994c1b80 1 -- 172.21.15.52:0/4188660774 shutdown_connections 2024-06-10T02:55:56.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.296+0000 7fc1994c1b80 1 -- 172.21.15.52:0/4188660774 wait complete. 2024-06-10T02:55:56.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.296+0000 7fc1994c1b80 1 -- start start 2024-06-10T02:55:56.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.296+0000 7fc1994c1b80 1 Processor -- start 2024-06-10T02:55:56.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.297+0000 7fc1994c1b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eb2eb7cce0 0x55eb2eb88be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:56.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.297+0000 7fc1994c1b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55eb2eb82590 0x55eb2eb85ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:56.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.297+0000 7fc1994c1b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55eb2eb82eb0 0x55eb2eb86420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:56.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.297+0000 7fc1994c1b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55eb2ea6e150 con 0x55eb2eb82eb0 2024-06-10T02:55:56.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.297+0000 7fc1994c1b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55eb2eac99a0 con 0x55eb2eb82590 2024-06-10T02:55:56.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.297+0000 7fc1994c1b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55eb2eb89830 con 0x55eb2eb7cce0 2024-06-10T02:55:56.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.297+0000 7fc1994c1b80 10 client.0 _ll_register_callbacks cb 0x55eb2ea63740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T02:55:56.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.297+0000 7fc18d1db700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55eb2eb82eb0 0x55eb2eb86420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:56.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.297+0000 7fc187fff700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55eb2eb82590 0x55eb2eb85ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:56.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.297+0000 7fc18c9da700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eb2eb7cce0 0x55eb2eb88be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:56.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.297+0000 7fc187fff700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55eb2eb82590 0x55eb2eb85ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.52:34186/0 (socket says 172.21.15.52:34186) 2024-06-10T02:55:56.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.297+0000 7fc18d1db700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55eb2eb82eb0 0x55eb2eb86420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:41876/0 (socket says 172.21.15.52:41876) 2024-06-10T02:55:56.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.297+0000 7fc187fff700 1 -- 172.21.15.52:0/2773105392 learned_addr learned my addr 172.21.15.52:0/2773105392 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:55:56.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.297+0000 7fc18c9da700 1 -- 172.21.15.52:0/2773105392 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55eb2eb82590 msgr2=0x55eb2eb85ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:55:56.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.297+0000 7fc18c9da700 1 --2- 172.21.15.52:0/2773105392 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55eb2eb82590 0x55eb2eb85ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:56.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.297+0000 7fc18c9da700 1 -- 172.21.15.52:0/2773105392 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55eb2eb82eb0 msgr2=0x55eb2eb86420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:55:56.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.297+0000 7fc18c9da700 1 --2- 172.21.15.52:0/2773105392 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55eb2eb82eb0 0x55eb2eb86420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:56.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.297+0000 7fc18c9da700 1 -- 172.21.15.52:0/2773105392 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb2ea938e0 con 0x55eb2eb7cce0 2024-06-10T02:55:56.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.297+0000 7fc18c9da700 1 --2- 172.21.15.52:0/2773105392 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eb2eb7cce0 0x55eb2eb88be0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc17c00d720 tx=0x7fc17c09bd40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ac3655629568d3cf server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:55:56.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.297+0000 7fc167fff700 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-06-10T02:55:56.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.298+0000 7fc167fff700 1 -- 172.21.15.52:0/2773105392 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc17c01d040 con 0x55eb2eb7cce0 2024-06-10T02:55:56.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.298+0000 7fc167fff700 1 -- 172.21.15.52:0/2773105392 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7fc17c0055c0 con 0x55eb2eb7cce0 2024-06-10T02:55:56.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.298+0000 7fc1994c1b80 1 -- 172.21.15.52:0/2773105392 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc174003780 con 0x55eb2eb7cce0 2024-06-10T02:55:56.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.298+0000 7fc167fff700 1 -- 172.21.15.52:0/2773105392 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fc17c0a4170 con 0x55eb2eb7cce0 2024-06-10T02:55:56.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.298+0000 7fc1994c1b80 10 client.25321 fetch_fsmap learned FSMap version 31 2024-06-10T02:55:56.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.298+0000 7fc1994c1b80 1 -- 172.21.15.52:0/2773105392 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55eb2eb93f70 con 0x55eb2eb7cce0 2024-06-10T02:55:56.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.298+0000 7fc167fff700 1 -- 172.21.15.52:0/2773105392 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc174003780 con 0x55eb2eb7cce0 2024-06-10T02:55:56.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.298+0000 7fc167fff700 1 -- 172.21.15.52:0/2773105392 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 31) ==== 8366+0+0 (secure 0 0 0) 0x7fc17c0a3de0 con 0x55eb2eb7cce0 2024-06-10T02:55:56.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.298+0000 7fc1994c1b80 10 client.25321 fetch_fsmap finished waiting for FSMap version 31 2024-06-10T02:55:56.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.298+0000 7fc1994c1b80 10 client.25321 resolve_mds: validated gid 24484 aka daemon mds.i 2024-06-10T02:55:56.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.298+0000 7fc1994c1b80 20 client.25321 populate_metadata read hostname 'smithi052' 2024-06-10T02:55:56.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.298+0000 7fc1994c1b80 1 --2- 172.21.15.52:0/2773105392 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55eb2eb96500 0x55eb2eb98930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:56.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.298+0000 7fc1994c1b80 4 client.25321 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T02:55:56.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.298+0000 7fc1994c1b80 1 -- 172.21.15.52:0/2773105392 --> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eb2eadaee0 con 0x55eb2eb96500 2024-06-10T02:55:56.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.299+0000 7fc187fff700 1 --2- 172.21.15.52:0/2773105392 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55eb2eb96500 0x55eb2eb98930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:56.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.299+0000 7fc187fff700 1 --2- 172.21.15.52:0/2773105392 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55eb2eb96500 0x55eb2eb98930 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=24a0af70d03d466 server_cookie=76a7f147bd0558c5 in_seq=0 out_seq=0 2024-06-10T02:55:56.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.299+0000 7fc167fff700 10 client.25321 ms_handle_connect on v2:172.21.15.188:6838/310444836 2024-06-10T02:55:56.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.300+0000 7fc167fff700 1 -- 172.21.15.52:0/2773105392 <== mds.0 v2:172.21.15.188:6838/310444836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55eb2eadaee0 con 0x55eb2eb96500 2024-06-10T02:55:56.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.300+0000 7fc167fff700 10 client.25321 handle_command_reply: tid=0 2024-06-10T02:55:56.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.300+0000 7fc1994c1b80 1 client.25321 shutdown 2024-06-10T02:55:56.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.300+0000 7fc1994c1b80 1 -- 172.21.15.52:0/2773105392 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eb2eb7cce0 msgr2=0x55eb2eb88be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:55:56.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.300+0000 7fc1994c1b80 1 --2- 172.21.15.52:0/2773105392 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eb2eb7cce0 0x55eb2eb88be0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc17c00d720 tx=0x7fc17c09bd40 comp rx=0 tx=0).stop 2024-06-10T02:55:56.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.300+0000 7fc1994c1b80 1 -- 172.21.15.52:0/2773105392 shutdown_connections 2024-06-10T02:55:56.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.300+0000 7fc1994c1b80 1 --2- 172.21.15.52:0/2773105392 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55eb2eb96500 0x55eb2eb98930 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:56.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.300+0000 7fc1994c1b80 1 --2- 172.21.15.52:0/2773105392 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55eb2eb82eb0 0x55eb2eb86420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:56.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.300+0000 7fc1994c1b80 1 --2- 172.21.15.52:0/2773105392 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eb2eb7cce0 0x55eb2eb88be0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:56.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.300+0000 7fc1994c1b80 1 --2- 172.21.15.52:0/2773105392 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55eb2eb82590 0x55eb2eb85ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:56.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.300+0000 7fc167fff700 0 client.25321 ms_handle_reset on v2:172.21.15.188:6838/310444836 2024-06-10T02:55:56.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.300+0000 7fc1994c1b80 1 -- 172.21.15.52:0/2773105392 >> 172.21.15.52:0/2773105392 conn(0x55eb2eaa9210 msgr2=0x55eb2eb76c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:55:56.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.300+0000 7fc1994c1b80 1 -- 172.21.15.52:0/2773105392 shutdown_connections 2024-06-10T02:55:56.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.300+0000 7fc1994c1b80 1 -- 172.21.15.52:0/2773105392 wait complete. 2024-06-10T02:55:56.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.301+0000 7fc1994c1b80 20 client.25321 trim_cache size 0 max 16384 2024-06-10T02:55:56.322 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.321+0000 7fc1994c1b80 1 Processor -- start 2024-06-10T02:55:56.322 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.322+0000 7fc1994c1b80 1 -- start start 2024-06-10T02:55:56.322 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.322+0000 7fc1994c1b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eb2eb7cce0 0x55eb2eb88150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:56.322 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.322+0000 7fc1994c1b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55eb2eb82590 0x55eb2eb7bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:56.322 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.322+0000 7fc1994c1b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55eb2eb82eb0 0x55eb2eb7c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:56.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.322+0000 7fc1994c1b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fc1881ac030 con 0x55eb2eb82eb0 2024-06-10T02:55:56.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.322+0000 7fc1994c1b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc1881adba0 con 0x55eb2eb82590 2024-06-10T02:55:56.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.322+0000 7fc1994c1b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc1881ac700 con 0x55eb2eb7cce0 2024-06-10T02:55:56.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.322+0000 7fc18c9da700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eb2eb7cce0 0x55eb2eb88150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:56.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.322+0000 7fc18d1db700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55eb2eb82eb0 0x55eb2eb7c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:56.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.322+0000 7fc18c9da700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eb2eb7cce0 0x55eb2eb88150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.52:49068/0 (socket says 172.21.15.52:49068) 2024-06-10T02:55:56.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.322+0000 7fc18c9da700 1 -- 172.21.15.52:0/1389187206 learned_addr learned my addr 172.21.15.52:0/1389187206 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:55:56.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.322+0000 7fc187fff700 1 --2- 172.21.15.52:0/1389187206 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55eb2eb82590 0x55eb2eb7bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:56.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.322+0000 7fc18c9da700 1 -- 172.21.15.52:0/1389187206 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55eb2eb82590 msgr2=0x55eb2eb7bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:55:56.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.322+0000 7fc18c9da700 1 --2- 172.21.15.52:0/1389187206 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55eb2eb82590 0x55eb2eb7bea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:56.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.322+0000 7fc18c9da700 1 -- 172.21.15.52:0/1389187206 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55eb2eb82eb0 msgr2=0x55eb2eb7c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:55:56.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.322+0000 7fc18c9da700 1 --2- 172.21.15.52:0/1389187206 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55eb2eb82eb0 0x55eb2eb7c3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:56.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.322+0000 7fc18c9da700 1 -- 172.21.15.52:0/1389187206 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb2eb6f530 con 0x55eb2eb7cce0 2024-06-10T02:55:56.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.323+0000 7fc18c9da700 1 --2- 172.21.15.52:0/1389187206 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eb2eb7cce0 0x55eb2eb88150 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc17c09b490 tx=0x7fc17c0b45f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f827e3a4250bdf9 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:55:56.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.323+0000 7fc1727fc700 1 -- 172.21.15.52:0/1389187206 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc17c01d070 con 0x55eb2eb7cce0 2024-06-10T02:55:56.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.323+0000 7fc1727fc700 1 -- 172.21.15.52:0/1389187206 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc17c0055c0 con 0x55eb2eb7cce0 2024-06-10T02:55:56.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.323+0000 7fc1727fc700 1 -- 172.21.15.52:0/1389187206 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc17c0a3f40 con 0x55eb2eb7cce0 2024-06-10T02:55:56.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.323+0000 7fc1994c1b80 1 -- 172.21.15.52:0/1389187206 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eb2eb7cce0 msgr2=0x55eb2eb88150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:55:56.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.323+0000 7fc1994c1b80 1 --2- 172.21.15.52:0/1389187206 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eb2eb7cce0 0x55eb2eb88150 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc17c09b490 tx=0x7fc17c0b45f0 comp rx=0 tx=0).stop 2024-06-10T02:55:56.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.324+0000 7fc1994c1b80 1 -- 172.21.15.52:0/1389187206 shutdown_connections 2024-06-10T02:55:56.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.324+0000 7fc1994c1b80 1 --2- 172.21.15.52:0/1389187206 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55eb2eb82eb0 0x55eb2eb7c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:56.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.324+0000 7fc1994c1b80 1 --2- 172.21.15.52:0/1389187206 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eb2eb7cce0 0x55eb2eb88150 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:56.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.324+0000 7fc1994c1b80 1 --2- 172.21.15.52:0/1389187206 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55eb2eb82590 0x55eb2eb7bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:56.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.324+0000 7fc1994c1b80 1 -- 172.21.15.52:0/1389187206 >> 172.21.15.52:0/1389187206 conn(0x55eb2eaa9210 msgr2=0x55eb2eb7b790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:55:56.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.324+0000 7fc1994c1b80 1 -- 172.21.15.52:0/1389187206 shutdown_connections 2024-06-10T02:55:56.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.324+0000 7fc1994c1b80 1 -- 172.21.15.52:0/1389187206 wait complete. 2024-06-10T02:55:56.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.324+0000 7fc1994c1b80 1 -- start start 2024-06-10T02:55:56.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.324+0000 7fc1994c1b80 1 Processor -- start 2024-06-10T02:55:56.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.324+0000 7fc1994c1b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55eb2eb7cce0 0x55eb2eb8f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:56.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.324+0000 7fc1994c1b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eb2eb82590 0x55eb2eb8bac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:56.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.325+0000 7fc1994c1b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55eb2eb82eb0 0x55eb2eb8c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:56.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.325+0000 7fc187fff700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eb2eb82590 0x55eb2eb8bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:56.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.325+0000 7fc18c9da700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55eb2eb7cce0 0x55eb2eb8f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:56.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.325+0000 7fc187fff700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eb2eb82590 0x55eb2eb8bac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.52:49084/0 (socket says 172.21.15.52:49084) 2024-06-10T02:55:56.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.325+0000 7fc187fff700 1 -- 172.21.15.52:0/96138912 learned_addr learned my addr 172.21.15.52:0/96138912 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:55:56.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.325+0000 7fc18d1db700 1 --2- 172.21.15.52:0/96138912 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55eb2eb82eb0 0x55eb2eb8c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:56.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.325+0000 7fc1994c1b80 1 -- 172.21.15.52:0/96138912 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fc1881ad4a0 con 0x55eb2eb7cce0 2024-06-10T02:55:56.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.325+0000 7fc1994c1b80 1 -- 172.21.15.52:0/96138912 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc1881acdd0 con 0x55eb2eb82eb0 2024-06-10T02:55:56.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.325+0000 7fc1994c1b80 1 -- 172.21.15.52:0/96138912 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc1881a9fd0 con 0x55eb2eb82590 2024-06-10T02:55:56.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.325+0000 7fc1994c1b80 10 client.0 _ll_register_callbacks cb 0x55eb2ea63740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T02:55:56.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.325+0000 7fc187fff700 1 -- 172.21.15.52:0/96138912 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55eb2eb82eb0 msgr2=0x55eb2eb8c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:55:56.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.325+0000 7fc187fff700 1 --2- 172.21.15.52:0/96138912 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55eb2eb82eb0 0x55eb2eb8c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:56.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.325+0000 7fc187fff700 1 -- 172.21.15.52:0/96138912 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55eb2eb7cce0 msgr2=0x55eb2eb8f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:55:56.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.325+0000 7fc187fff700 1 --2- 172.21.15.52:0/96138912 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55eb2eb7cce0 0x55eb2eb8f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:56.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.325+0000 7fc187fff700 1 -- 172.21.15.52:0/96138912 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb2eb8c640 con 0x55eb2eb82590 2024-06-10T02:55:56.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.326+0000 7fc187fff700 1 --2- 172.21.15.52:0/96138912 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eb2eb82590 0x55eb2eb8bac0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc174009990 tx=0x7fc174005900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eb7ca61cdb0c9f12 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:55:56.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.326+0000 7fc167fff700 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-06-10T02:55:56.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.326+0000 7fc167fff700 1 -- 172.21.15.52:0/96138912 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc174028040 con 0x55eb2eb82590 2024-06-10T02:55:56.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.326+0000 7fc167fff700 1 -- 172.21.15.52:0/96138912 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7fc1740052f0 con 0x55eb2eb82590 2024-06-10T02:55:56.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.326+0000 7fc1994c1b80 1 -- 172.21.15.52:0/96138912 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc17c0055c0 con 0x55eb2eb82590 2024-06-10T02:55:56.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.326+0000 7fc167fff700 1 -- 172.21.15.52:0/96138912 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fc17400bea0 con 0x55eb2eb82590 2024-06-10T02:55:56.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.326+0000 7fc1994c1b80 10 client.25327 fetch_fsmap learned FSMap version 31 2024-06-10T02:55:56.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.326+0000 7fc1994c1b80 1 -- 172.21.15.52:0/96138912 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55eb2ec02020 con 0x55eb2eb82590 2024-06-10T02:55:56.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.326+0000 7fc167fff700 1 -- 172.21.15.52:0/96138912 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc17c0055c0 con 0x55eb2eb82590 2024-06-10T02:55:56.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.326+0000 7fc167fff700 1 -- 172.21.15.52:0/96138912 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 31) ==== 8366+0+0 (secure 0 0 0) 0x7fc17400f680 con 0x55eb2eb82590 2024-06-10T02:55:56.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.326+0000 7fc1994c1b80 10 client.25327 fetch_fsmap finished waiting for FSMap version 31 2024-06-10T02:55:56.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.326+0000 7fc1994c1b80 10 client.25327 resolve_mds: validated gid 24484 aka daemon mds.i 2024-06-10T02:55:56.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.326+0000 7fc1994c1b80 20 client.25327 populate_metadata read hostname 'smithi052' 2024-06-10T02:55:56.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.327+0000 7fc1994c1b80 1 --2- 172.21.15.52:0/96138912 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55eb2ec022a0 0x55eb2ec04690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:55:56.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.327+0000 7fc1994c1b80 4 client.25327 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}] 2024-06-10T02:55:56.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.327+0000 7fc1994c1b80 1 -- 172.21.15.52:0/96138912 --> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}) -- 0x55eb2ead5460 con 0x55eb2ec022a0 2024-06-10T02:55:56.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.327+0000 7fc18c9da700 1 --2- 172.21.15.52:0/96138912 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55eb2ec022a0 0x55eb2ec04690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:55:56.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.327+0000 7fc18c9da700 1 --2- 172.21.15.52:0/96138912 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55eb2ec022a0 0x55eb2ec04690 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a383c53fb211ae3 server_cookie=f0634f6d4e5dc6de in_seq=0 out_seq=0 2024-06-10T02:55:56.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.327+0000 7fc167fff700 10 client.25327 ms_handle_connect on v2:172.21.15.188:6838/310444836 2024-06-10T02:55:56.332 INFO:tasks.workunit.client.0.smithi052.stdout:checking whether pthreads work without any flags... yes 2024-06-10T02:55:56.408 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.408+0000 7fc167fff700 1 -- 172.21.15.52:0/96138912 <== mds.0 v2:172.21.15.188:6838/310444836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x55eb2ead5460 con 0x55eb2ec022a0 2024-06-10T02:55:56.408 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.408+0000 7fc167fff700 10 client.25327 handle_command_reply: tid=0 2024-06-10T02:55:56.408 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.408+0000 7fc1994c1b80 1 client.25327 shutdown 2024-06-10T02:55:56.408 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.408+0000 7fc1994c1b80 1 -- 172.21.15.52:0/96138912 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eb2eb82590 msgr2=0x55eb2eb8bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:55:56.408 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.408+0000 7fc1994c1b80 1 --2- 172.21.15.52:0/96138912 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eb2eb82590 0x55eb2eb8bac0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc174009990 tx=0x7fc174005900 comp rx=0 tx=0).stop 2024-06-10T02:55:56.408 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.408+0000 7fc1994c1b80 1 -- 172.21.15.52:0/96138912 shutdown_connections 2024-06-10T02:55:56.408 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.408+0000 7fc1994c1b80 1 --2- 172.21.15.52:0/96138912 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55eb2ec022a0 0x55eb2ec04690 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:56.408 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.408+0000 7fc1994c1b80 1 --2- 172.21.15.52:0/96138912 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55eb2eb82eb0 0x55eb2eb8c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:56.408 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.408+0000 7fc1994c1b80 1 --2- 172.21.15.52:0/96138912 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55eb2eb7cce0 0x55eb2eb8f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:56.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.408+0000 7fc1994c1b80 1 --2- 172.21.15.52:0/96138912 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eb2eb82590 0x55eb2eb8bac0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:56.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.408+0000 7fc167fff700 0 client.25327 ms_handle_reset on v2:172.21.15.188:6838/310444836 2024-06-10T02:55:56.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.408+0000 7fc1994c1b80 1 -- 172.21.15.52:0/96138912 >> 172.21.15.52:0/96138912 conn(0x55eb2eaa9210 msgr2=0x55eb2eb80fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:55:56.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.408+0000 7fc1994c1b80 1 -- 172.21.15.52:0/96138912 shutdown_connections 2024-06-10T02:55:56.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.408+0000 7fc1994c1b80 1 -- 172.21.15.52:0/96138912 wait complete. 2024-06-10T02:55:56.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.408+0000 7fc1994c1b80 20 client.25327 trim_cache size 0 max 16384 2024-06-10T02:55:56.409 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-10T02:55:56.409 INFO:teuthology.orchestra.run.smithi052.stdout: "epoch": 31, 2024-06-10T02:55:56.409 INFO:teuthology.orchestra.run.smithi052.stdout: "leader": 24484, 2024-06-10T02:55:56.410 INFO:teuthology.orchestra.run.smithi052.stdout: "set_version": 6, 2024-06-10T02:55:56.410 INFO:teuthology.orchestra.run.smithi052.stdout: "sets": { 2024-06-10T02:55:56.410 INFO:teuthology.orchestra.run.smithi052.stdout: "cdf1e680": { 2024-06-10T02:55:56.410 INFO:teuthology.orchestra.run.smithi052.stdout: "version": 6, 2024-06-10T02:55:56.410 INFO:teuthology.orchestra.run.smithi052.stdout: "age_ref": 0.0, 2024-06-10T02:55:56.410 INFO:teuthology.orchestra.run.smithi052.stdout: "state": { 2024-06-10T02:55:56.410 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "QUIESCED", 2024-06-10T02:55:56.410 INFO:teuthology.orchestra.run.smithi052.stdout: "age": 0.0 2024-06-10T02:55:56.410 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T02:55:56.410 INFO:teuthology.orchestra.run.smithi052.stdout: "timeout": 90.0, 2024-06-10T02:55:56.410 INFO:teuthology.orchestra.run.smithi052.stdout: "expiration": 150.0, 2024-06-10T02:55:56.410 INFO:teuthology.orchestra.run.smithi052.stdout: "members": { 2024-06-10T02:55:56.410 INFO:teuthology.orchestra.run.smithi052.stdout: "file:/": { 2024-06-10T02:55:56.411 INFO:teuthology.orchestra.run.smithi052.stdout: "excluded": false, 2024-06-10T02:55:56.411 INFO:teuthology.orchestra.run.smithi052.stdout: "state": { 2024-06-10T02:55:56.411 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "QUIESCED", 2024-06-10T02:55:56.411 INFO:teuthology.orchestra.run.smithi052.stdout: "age": 0.0 2024-06-10T02:55:56.411 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T02:55:56.411 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T02:55:56.411 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T02:55:56.411 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T02:55:56.411 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T02:55:56.411 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-10T02:55:56.412 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.409+0000 7fc18ec3e700 1 -- 172.21.15.52:0/1018167810 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fc1780a2cc0 msgr2=0x7fc1780a5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:55:56.412 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.409+0000 7fc18ec3e700 1 --2- 172.21.15.52:0/1018167810 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fc1780a2cc0 0x7fc1780a5180 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fc18000cb70 tx=0x7fc18000cf30 comp rx=0 tx=0).stop 2024-06-10T02:55:56.412 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.409+0000 7fc18ec3e700 1 -- 172.21.15.52:0/1018167810 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc188100a80 msgr2=0x7fc18819ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:55:56.412 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.409+0000 7fc18ec3e700 1 --2- 172.21.15.52:0/1018167810 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc188100a80 0x7fc18819ad10 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fc17c00d750 tx=0x7fc17c00da60 comp rx=0 tx=0).stop 2024-06-10T02:55:56.412 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.409+0000 7fc18ec3e700 1 -- 172.21.15.52:0/1018167810 shutdown_connections 2024-06-10T02:55:56.412 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.409+0000 7fc18ec3e700 1 --2- 172.21.15.52:0/1018167810 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fc1780a2cc0 0x7fc1780a5180 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:56.412 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.409+0000 7fc18ec3e700 1 --2- 172.21.15.52:0/1018167810 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc188107450 0x7fc18819f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:56.412 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.409+0000 7fc18ec3e700 1 --2- 172.21.15.52:0/1018167810 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc188100a80 0x7fc18819ad10 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:56.412 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.409+0000 7fc18ec3e700 1 --2- 172.21.15.52:0/1018167810 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc188101440 0x7fc18819b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:55:56.412 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.409+0000 7fc18ec3e700 1 -- 172.21.15.52:0/1018167810 >> 172.21.15.52:0/1018167810 conn(0x7fc188077a30 msgr2=0x7fc188108970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:55:56.412 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.409+0000 7fc18ec3e700 1 -- 172.21.15.52:0/1018167810 shutdown_connections 2024-06-10T02:55:56.413 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:55:56.409+0000 7fc18ec3e700 1 -- 172.21.15.52:0/1018167810 wait complete. 2024-06-10T02:55:56.726 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:55:56 smithi052 ceph-mon[32362]: pgmap v486: 129 pgs: 129 active+clean; 804 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-06-10T02:55:56.888 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:55:56 smithi120 ceph-mon[36634]: pgmap v486: 129 pgs: 129 active+clean; 804 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-06-10T02:55:56.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:55:56 smithi188 ceph-mon[36599]: pgmap v486: 129 pgs: 129 active+clean; 804 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-06-10T02:55:57.182 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'cdf1e680', quiesce took 5.8 seconds. Will release after: 24.2 2024-06-10T02:55:58.755 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:55:58 smithi188 ceph-mon[36599]: pgmap v487: 129 pgs: 129 active+clean; 2.9 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 197 KiB/s wr, 22 op/s 2024-06-10T02:55:58.887 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:55:58 smithi120 ceph-mon[36634]: pgmap v487: 129 pgs: 129 active+clean; 2.9 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 197 KiB/s wr, 22 op/s 2024-06-10T02:55:58.976 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:55:58 smithi052 ceph-mon[32362]: pgmap v487: 129 pgs: 129 active+clean; 2.9 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 197 KiB/s wr, 22 op/s 2024-06-10T02:56:00.888 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:56:00 smithi120 ceph-mon[36634]: pgmap v488: 129 pgs: 129 active+clean; 3.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 254 KiB/s wr, 33 op/s 2024-06-10T02:56:00.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:56:00 smithi188 ceph-mon[36599]: pgmap v488: 129 pgs: 129 active+clean; 3.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 254 KiB/s wr, 33 op/s 2024-06-10T02:56:00.976 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:56:00 smithi052 ceph-mon[32362]: pgmap v488: 129 pgs: 129 active+clean; 3.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 254 KiB/s wr, 33 op/s 2024-06-10T02:56:02.888 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:56:02 smithi120 ceph-mon[36634]: pgmap v489: 129 pgs: 129 active+clean; 3.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 285 KiB/s wr, 45 op/s 2024-06-10T02:56:02.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:56:02 smithi188 ceph-mon[36599]: pgmap v489: 129 pgs: 129 active+clean; 3.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 285 KiB/s wr, 45 op/s 2024-06-10T02:56:02.976 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:56:02 smithi052 ceph-mon[32362]: pgmap v489: 129 pgs: 129 active+clean; 3.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 285 KiB/s wr, 45 op/s 2024-06-10T02:56:04.694 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-10T02:56:04.888 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:56:04 smithi120 ceph-mon[36634]: pgmap v490: 129 pgs: 129 active+clean; 3.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 296 KiB/s wr, 49 op/s 2024-06-10T02:56:04.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:56:04 smithi188 ceph-mon[36599]: pgmap v490: 129 pgs: 129 active+clean; 3.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 296 KiB/s wr, 49 op/s 2024-06-10T02:56:04.962 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:56:04.976 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:56:04 smithi052 ceph-mon[32362]: pgmap v490: 129 pgs: 129 active+clean; 3.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 296 KiB/s wr, 49 op/s 2024-06-10T02:56:06.800 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:56:06 smithi052 ceph-mon[32362]: pgmap v491: 129 pgs: 129 active+clean; 3.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 284 KiB/s wr, 45 op/s 2024-06-10T02:56:06.888 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:56:06 smithi120 ceph-mon[36634]: pgmap v491: 129 pgs: 129 active+clean; 3.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 284 KiB/s wr, 45 op/s 2024-06-10T02:56:06.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:56:06 smithi188 ceph-mon[36599]: pgmap v491: 129 pgs: 129 active+clean; 3.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 284 KiB/s wr, 45 op/s 2024-06-10T02:56:07.036 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.036+0000 7f18c6503700 1 -- 172.21.15.52:0/3438092213 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f18c0101620 msgr2=0x7f18c0101a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:56:07.036 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.036+0000 7f18c6503700 1 --2- 172.21.15.52:0/3438092213 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f18c0101620 0x7f18c0101a00 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f18b00099c0 tx=0x7f18b0009cd0 comp rx=0 tx=0).stop 2024-06-10T02:56:07.036 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.036+0000 7f18c6503700 1 -- 172.21.15.52:0/3438092213 shutdown_connections 2024-06-10T02:56:07.036 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.036+0000 7f18c6503700 1 --2- 172.21.15.52:0/3438092213 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f18c01066b0 0x7f18c010e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:07.036 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.036+0000 7f18c6503700 1 --2- 172.21.15.52:0/3438092213 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f18c0101620 0x7f18c0101a00 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:07.036 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.036+0000 7f18c6503700 1 --2- 172.21.15.52:0/3438092213 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f18c0101fd0 0x7f18c0105fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:07.036 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.036+0000 7f18c6503700 1 -- 172.21.15.52:0/3438092213 >> 172.21.15.52:0/3438092213 conn(0x7f18c0078340 msgr2=0x7f18c0078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:56:07.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.036+0000 7f18c6503700 1 -- 172.21.15.52:0/3438092213 shutdown_connections 2024-06-10T02:56:07.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.036+0000 7f18c6503700 1 -- 172.21.15.52:0/3438092213 wait complete. 2024-06-10T02:56:07.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.036+0000 7f18c6503700 1 Processor -- start 2024-06-10T02:56:07.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.036+0000 7f18c6503700 1 -- start start 2024-06-10T02:56:07.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.037+0000 7f18c6503700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f18c0101620 0x7f18c019d750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:56:07.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.037+0000 7f18c6503700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f18c0101fd0 0x7f18c019dc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:56:07.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.037+0000 7f18c6503700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f18c01066b0 0x7f18c0197960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:56:07.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.037+0000 7f18c6503700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f18c0111700 con 0x7f18c0101620 2024-06-10T02:56:07.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.037+0000 7f18c6503700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f18c0111580 con 0x7f18c0101fd0 2024-06-10T02:56:07.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.037+0000 7f18c6503700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f18c0111880 con 0x7f18c01066b0 2024-06-10T02:56:07.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.037+0000 7f18bffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f18c0101620 0x7f18c019d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:56:07.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.037+0000 7f18bffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f18c0101620 0x7f18c019d750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57142/0 (socket says 172.21.15.52:57142) 2024-06-10T02:56:07.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.037+0000 7f18bffff700 1 -- 172.21.15.52:0/2160411454 learned_addr learned my addr 172.21.15.52:0/2160411454 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:56:07.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.037+0000 7f18c4aa0700 1 --2- 172.21.15.52:0/2160411454 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f18c01066b0 0x7f18c0197960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:56:07.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.037+0000 7f18bf7fe700 1 --2- 172.21.15.52:0/2160411454 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f18c0101fd0 0x7f18c019dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:56:07.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.037+0000 7f18bffff700 1 -- 172.21.15.52:0/2160411454 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f18c01066b0 msgr2=0x7f18c0197960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:56:07.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.037+0000 7f18bffff700 1 --2- 172.21.15.52:0/2160411454 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f18c01066b0 0x7f18c0197960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:07.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.037+0000 7f18bffff700 1 -- 172.21.15.52:0/2160411454 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f18c0101fd0 msgr2=0x7f18c019dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:56:07.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.037+0000 7f18bffff700 1 --2- 172.21.15.52:0/2160411454 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f18c0101fd0 0x7f18c019dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:07.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.037+0000 7f18bffff700 1 -- 172.21.15.52:0/2160411454 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18c01980c0 con 0x7f18c0101620 2024-06-10T02:56:07.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.038+0000 7f18bffff700 1 --2- 172.21.15.52:0/2160411454 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f18c0101620 0x7f18c019d750 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f18b0005910 tx=0x7f18b000fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:56:07.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.038+0000 7f18bd7fa700 1 -- 172.21.15.52:0/2160411454 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18b0027070 con 0x7f18c0101620 2024-06-10T02:56:07.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.038+0000 7f18c6503700 1 -- 172.21.15.52:0/2160411454 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18c0198350 con 0x7f18c0101620 2024-06-10T02:56:07.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.038+0000 7f18bd7fa700 1 -- 172.21.15.52:0/2160411454 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f18b0003680 con 0x7f18c0101620 2024-06-10T02:56:07.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.038+0000 7f18bd7fa700 1 -- 172.21.15.52:0/2160411454 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18b00173f0 con 0x7f18c0101620 2024-06-10T02:56:07.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.038+0000 7f18c6503700 1 -- 172.21.15.52:0/2160411454 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18c01a4620 con 0x7f18c0101620 2024-06-10T02:56:07.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.040+0000 7f18bd7fa700 1 -- 172.21.15.52:0/2160411454 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f18b0022470 con 0x7f18c0101620 2024-06-10T02:56:07.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.041+0000 7f18bd7fa700 1 --2- 172.21.15.52:0/2160411454 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f18a80a2c20 0x7f18a80a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:56:07.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.041+0000 7f18bf7fe700 1 --2- 172.21.15.52:0/2160411454 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f18a80a2c20 0x7f18a80a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:56:07.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.041+0000 7f18bd7fa700 1 -- 172.21.15.52:0/2160411454 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f18b00cfd90 con 0x7f18c0101620 2024-06-10T02:56:07.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.041+0000 7f18bf7fe700 1 --2- 172.21.15.52:0/2160411454 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f18a80a2c20 0x7f18a80a50e0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f18c0107af0 tx=0x7f18b4009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:56:07.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.042+0000 7f18d0d86b80 1 Processor -- start 2024-06-10T02:56:07.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.042+0000 7f18d0d86b80 1 -- start start 2024-06-10T02:56:07.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.042+0000 7f18d0d86b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556b26d38030 0x556b26d36240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:56:07.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.042+0000 7f18d0d86b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556b26d36780 0x556b26d36c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:56:07.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.042+0000 7f18d0d86b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556b26d478e0 0x556b26d49cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:56:07.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.042+0000 7f18d0d86b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x556b26c2bf60 con 0x556b26d38030 2024-06-10T02:56:07.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.042+0000 7f18d0d86b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x556b26c8a850 con 0x556b26d36780 2024-06-10T02:56:07.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.042+0000 7f18d0d86b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x556b26bf2460 con 0x556b26d478e0 2024-06-10T02:56:07.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.042+0000 7f18bffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556b26d38030 0x556b26d36240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:56:07.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.042+0000 7f18bffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556b26d38030 0x556b26d36240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57146/0 (socket says 172.21.15.52:57146) 2024-06-10T02:56:07.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.042+0000 7f18bffff700 1 -- 172.21.15.52:0/3360475466 learned_addr learned my addr 172.21.15.52:0/3360475466 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:56:07.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.042+0000 7f18bf7fe700 1 --2- 172.21.15.52:0/3360475466 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556b26d36780 0x556b26d36c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:56:07.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.042+0000 7f18c4aa0700 1 --2- 172.21.15.52:0/3360475466 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556b26d478e0 0x556b26d49cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:56:07.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.042+0000 7f18bf7fe700 1 -- 172.21.15.52:0/3360475466 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556b26d478e0 msgr2=0x556b26d49cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:56:07.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.042+0000 7f18bf7fe700 1 --2- 172.21.15.52:0/3360475466 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556b26d478e0 0x556b26d49cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:07.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.042+0000 7f18bf7fe700 1 -- 172.21.15.52:0/3360475466 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556b26d38030 msgr2=0x556b26d36240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:56:07.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.042+0000 7f18bf7fe700 1 --2- 172.21.15.52:0/3360475466 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556b26d38030 0x556b26d36240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:07.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.042+0000 7f18bf7fe700 1 -- 172.21.15.52:0/3360475466 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b26bd98d0 con 0x556b26d36780 2024-06-10T02:56:07.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.043+0000 7f18bf7fe700 1 --2- 172.21.15.52:0/3360475466 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556b26d36780 0x556b26d36c00 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f18c006aa60 tx=0x7f18b4014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9cdc02e16d14a620 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:56:07.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.043+0000 7f18a67fc700 1 -- 172.21.15.52:0/3360475466 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18b4004830 con 0x556b26d36780 2024-06-10T02:56:07.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.043+0000 7f18a67fc700 1 -- 172.21.15.52:0/3360475466 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f18b4005020 con 0x556b26d36780 2024-06-10T02:56:07.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.043+0000 7f18a67fc700 1 -- 172.21.15.52:0/3360475466 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18b4003a30 con 0x556b26d36780 2024-06-10T02:56:07.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.044+0000 7f18d0d86b80 1 -- 172.21.15.52:0/3360475466 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556b26d36780 msgr2=0x556b26d36c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:56:07.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.044+0000 7f18d0d86b80 1 --2- 172.21.15.52:0/3360475466 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556b26d36780 0x556b26d36c00 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f18c006aa60 tx=0x7f18b4014000 comp rx=0 tx=0).stop 2024-06-10T02:56:07.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.044+0000 7f18d0d86b80 1 -- 172.21.15.52:0/3360475466 shutdown_connections 2024-06-10T02:56:07.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.044+0000 7f18d0d86b80 1 --2- 172.21.15.52:0/3360475466 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556b26d478e0 0x556b26d49cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:07.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.044+0000 7f18d0d86b80 1 --2- 172.21.15.52:0/3360475466 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556b26d38030 0x556b26d36240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:07.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.044+0000 7f18d0d86b80 1 --2- 172.21.15.52:0/3360475466 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556b26d36780 0x556b26d36c00 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:07.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.044+0000 7f18d0d86b80 1 -- 172.21.15.52:0/3360475466 >> 172.21.15.52:0/3360475466 conn(0x556b26c66080 msgr2=0x556b26d34220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:56:07.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.044+0000 7f18d0d86b80 1 -- 172.21.15.52:0/3360475466 shutdown_connections 2024-06-10T02:56:07.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.044+0000 7f18d0d86b80 1 -- 172.21.15.52:0/3360475466 wait complete. 2024-06-10T02:56:07.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.044+0000 7f18d0d86b80 1 -- start start 2024-06-10T02:56:07.048 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.045+0000 7f18d0d86b80 1 Processor -- start 2024-06-10T02:56:07.048 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.045+0000 7f18d0d86b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556b26d36780 0x556b26d470d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:56:07.048 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.045+0000 7f18d0d86b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556b26d38030 0x556b26d443d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:56:07.048 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.045+0000 7f18d0d86b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556b26d478e0 0x556b26d44910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:56:07.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.045+0000 7f18d0d86b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x556b26c2afc0 con 0x556b26d478e0 2024-06-10T02:56:07.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.045+0000 7f18d0d86b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x556b26c86810 con 0x556b26d38030 2024-06-10T02:56:07.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.045+0000 7f18d0d86b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x556b26d4a6c0 con 0x556b26d36780 2024-06-10T02:56:07.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.045+0000 7f18d0d86b80 10 client.0 _ll_register_callbacks cb 0x556b26c205b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T02:56:07.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.045+0000 7f18bf7fe700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556b26d38030 0x556b26d443d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:56:07.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.045+0000 7f18bf7fe700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556b26d38030 0x556b26d443d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.52:56476/0 (socket says 172.21.15.52:56476) 2024-06-10T02:56:07.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.045+0000 7f18bf7fe700 1 -- 172.21.15.52:0/573064836 learned_addr learned my addr 172.21.15.52:0/573064836 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:56:07.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.045+0000 7f18bffff700 1 --2- 172.21.15.52:0/573064836 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556b26d36780 0x556b26d470d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:56:07.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.045+0000 7f18c4aa0700 1 --2- 172.21.15.52:0/573064836 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556b26d478e0 0x556b26d44910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:56:07.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.045+0000 7f18bf7fe700 1 -- 172.21.15.52:0/573064836 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556b26d36780 msgr2=0x556b26d470d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:56:07.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.045+0000 7f18bf7fe700 1 --2- 172.21.15.52:0/573064836 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556b26d36780 0x556b26d470d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:07.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.045+0000 7f18bf7fe700 1 -- 172.21.15.52:0/573064836 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556b26d478e0 msgr2=0x556b26d44910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:56:07.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.045+0000 7f18bf7fe700 1 --2- 172.21.15.52:0/573064836 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556b26d478e0 0x556b26d44910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:07.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.045+0000 7f18bf7fe700 1 -- 172.21.15.52:0/573064836 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b26c50750 con 0x556b26d38030 2024-06-10T02:56:07.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.046+0000 7f18bf7fe700 1 --2- 172.21.15.52:0/573064836 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556b26d38030 0x556b26d443d0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f18b4005820 tx=0x7f18b4003970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a60fcbf115effe31 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:56:07.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.046+0000 7f189f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-06-10T02:56:07.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.046+0000 7f189f7fe700 1 -- 172.21.15.52:0/573064836 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18b4003d20 con 0x556b26d38030 2024-06-10T02:56:07.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.046+0000 7f189f7fe700 1 -- 172.21.15.52:0/573064836 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f18b40044d0 con 0x556b26d38030 2024-06-10T02:56:07.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.046+0000 7f18d0d86b80 1 -- 172.21.15.52:0/573064836 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f18b4005020 con 0x556b26d38030 2024-06-10T02:56:07.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.046+0000 7f189f7fe700 1 -- 172.21.15.52:0/573064836 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f18b4039050 con 0x556b26d38030 2024-06-10T02:56:07.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.046+0000 7f18d0d86b80 10 client.25364 fetch_fsmap learned FSMap version 31 2024-06-10T02:56:07.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.046+0000 7f18d0d86b80 1 -- 172.21.15.52:0/573064836 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x556b26d54e70 con 0x556b26d38030 2024-06-10T02:56:07.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.046+0000 7f189f7fe700 1 -- 172.21.15.52:0/573064836 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f18b4005020 con 0x556b26d38030 2024-06-10T02:56:07.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.046+0000 7f189f7fe700 1 -- 172.21.15.52:0/573064836 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 31) ==== 8366+0+0 (secure 0 0 0) 0x7f18b401e7f0 con 0x556b26d38030 2024-06-10T02:56:07.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.047+0000 7f18d0d86b80 10 client.25364 fetch_fsmap finished waiting for FSMap version 31 2024-06-10T02:56:07.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.047+0000 7f18d0d86b80 10 client.25364 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T02:56:07.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.047+0000 7f18d0d86b80 20 client.25364 populate_metadata read hostname 'smithi052' 2024-06-10T02:56:07.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.047+0000 7f18d0d86b80 1 --2- 172.21.15.52:0/573064836 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x556b26d57340 0x556b26d59730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:56:07.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.047+0000 7f18d0d86b80 4 client.25364 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T02:56:07.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.047+0000 7f18d0d86b80 1 -- 172.21.15.52:0/573064836 --> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556b26c97d50 con 0x556b26d57340 2024-06-10T02:56:07.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.047+0000 7f18bffff700 1 --2- 172.21.15.52:0/573064836 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x556b26d57340 0x556b26d59730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:56:07.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.047+0000 7f189f7fe700 10 client.25364 ms_handle_connect on v2:172.21.15.188:6838/310444836 2024-06-10T02:56:07.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.047+0000 7f18bffff700 1 --2- 172.21.15.52:0/573064836 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x556b26d57340 0x556b26d59730 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74d44b41e7e24729 server_cookie=1e680aaa0788b5a1 in_seq=0 out_seq=0 2024-06-10T02:56:07.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.048+0000 7f189f7fe700 1 -- 172.21.15.52:0/573064836 <== mds.0 v2:172.21.15.188:6838/310444836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x556b26c97d50 con 0x556b26d57340 2024-06-10T02:56:07.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.048+0000 7f189f7fe700 10 client.25364 handle_command_reply: tid=0 2024-06-10T02:56:07.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.048+0000 7f18d0d86b80 1 client.25364 shutdown 2024-06-10T02:56:07.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.048+0000 7f18d0d86b80 1 -- 172.21.15.52:0/573064836 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556b26d38030 msgr2=0x556b26d443d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:56:07.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.048+0000 7f18d0d86b80 1 --2- 172.21.15.52:0/573064836 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556b26d38030 0x556b26d443d0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f18b4005820 tx=0x7f18b4003970 comp rx=0 tx=0).stop 2024-06-10T02:56:07.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.048+0000 7f18d0d86b80 1 -- 172.21.15.52:0/573064836 shutdown_connections 2024-06-10T02:56:07.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.048+0000 7f18d0d86b80 1 --2- 172.21.15.52:0/573064836 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x556b26d57340 0x556b26d59730 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:07.052 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.048+0000 7f18d0d86b80 1 --2- 172.21.15.52:0/573064836 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556b26d478e0 0x556b26d44910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:07.052 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.048+0000 7f18d0d86b80 1 --2- 172.21.15.52:0/573064836 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556b26d36780 0x556b26d470d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:07.052 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.048+0000 7f189f7fe700 0 client.25364 ms_handle_reset on v2:172.21.15.188:6838/310444836 2024-06-10T02:56:07.052 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.048+0000 7f18d0d86b80 1 --2- 172.21.15.52:0/573064836 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556b26d38030 0x556b26d443d0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:07.052 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.048+0000 7f18d0d86b80 1 -- 172.21.15.52:0/573064836 >> 172.21.15.52:0/573064836 conn(0x556b26c66080 msgr2=0x556b26d32e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:56:07.052 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.049+0000 7f18d0d86b80 1 -- 172.21.15.52:0/573064836 shutdown_connections 2024-06-10T02:56:07.052 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.049+0000 7f18d0d86b80 1 -- 172.21.15.52:0/573064836 wait complete. 2024-06-10T02:56:07.052 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.049+0000 7f18d0d86b80 20 client.25364 trim_cache size 0 max 16384 2024-06-10T02:56:07.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.070+0000 7f18d0d86b80 1 Processor -- start 2024-06-10T02:56:07.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.070+0000 7f18d0d86b80 1 -- start start 2024-06-10T02:56:07.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.070+0000 7f18d0d86b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556b26d36780 0x556b26d44940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:56:07.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.070+0000 7f18d0d86b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556b26d38030 0x556b26d33e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:56:07.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.070+0000 7f18d0d86b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556b26d478e0 0x556b26d5ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:56:07.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.070+0000 7f18d0d86b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f18c01ac060 con 0x556b26d36780 2024-06-10T02:56:07.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.070+0000 7f18d0d86b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f18c01adbd0 con 0x556b26d478e0 2024-06-10T02:56:07.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.070+0000 7f18d0d86b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f18c01ac730 con 0x556b26d38030 2024-06-10T02:56:07.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.070+0000 7f18bffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556b26d36780 0x556b26d44940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:56:07.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.070+0000 7f18bffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556b26d36780 0x556b26d44940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57166/0 (socket says 172.21.15.52:57166) 2024-06-10T02:56:07.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.070+0000 7f18bffff700 1 -- 172.21.15.52:0/3244400994 learned_addr learned my addr 172.21.15.52:0/3244400994 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:56:07.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.070+0000 7f18bf7fe700 1 --2- 172.21.15.52:0/3244400994 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556b26d38030 0x556b26d33e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:56:07.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.070+0000 7f18c4aa0700 1 --2- 172.21.15.52:0/3244400994 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556b26d478e0 0x556b26d5ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:56:07.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.070+0000 7f18bffff700 1 -- 172.21.15.52:0/3244400994 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556b26d38030 msgr2=0x556b26d33e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:56:07.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.070+0000 7f18bffff700 1 --2- 172.21.15.52:0/3244400994 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556b26d38030 0x556b26d33e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:07.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.070+0000 7f18bffff700 1 -- 172.21.15.52:0/3244400994 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556b26d478e0 msgr2=0x556b26d5ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:56:07.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.070+0000 7f18bffff700 1 --2- 172.21.15.52:0/3244400994 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556b26d478e0 0x556b26d5ed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:07.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.070+0000 7f18bffff700 1 -- 172.21.15.52:0/3244400994 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b26d45310 con 0x556b26d36780 2024-06-10T02:56:07.074 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.071+0000 7f18bffff700 1 --2- 172.21.15.52:0/3244400994 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556b26d36780 0x556b26d44940 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f18b0005f50 tx=0x7f18b0000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e922d497b6debb91 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:56:07.074 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.071+0000 7f18a5ffb700 1 -- 172.21.15.52:0/3244400994 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18b00a0680 con 0x556b26d36780 2024-06-10T02:56:07.074 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.071+0000 7f18a5ffb700 1 -- 172.21.15.52:0/3244400994 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f18b00a4eb0 con 0x556b26d36780 2024-06-10T02:56:07.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.071+0000 7f18a5ffb700 1 -- 172.21.15.52:0/3244400994 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18b009c530 con 0x556b26d36780 2024-06-10T02:56:07.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.071+0000 7f18d0d86b80 1 -- 172.21.15.52:0/3244400994 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556b26d36780 msgr2=0x556b26d44940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:56:07.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.071+0000 7f18d0d86b80 1 --2- 172.21.15.52:0/3244400994 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556b26d36780 0x556b26d44940 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f18b0005f50 tx=0x7f18b0000bc0 comp rx=0 tx=0).stop 2024-06-10T02:56:07.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.072+0000 7f18d0d86b80 1 -- 172.21.15.52:0/3244400994 shutdown_connections 2024-06-10T02:56:07.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.072+0000 7f18d0d86b80 1 --2- 172.21.15.52:0/3244400994 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556b26d478e0 0x556b26d5ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:07.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.072+0000 7f18d0d86b80 1 --2- 172.21.15.52:0/3244400994 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556b26d36780 0x556b26d44940 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:07.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.072+0000 7f18d0d86b80 1 --2- 172.21.15.52:0/3244400994 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556b26d38030 0x556b26d33e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:07.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.072+0000 7f18d0d86b80 1 -- 172.21.15.52:0/3244400994 >> 172.21.15.52:0/3244400994 conn(0x556b26c66080 msgr2=0x556b26d46490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:56:07.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.072+0000 7f18d0d86b80 1 -- 172.21.15.52:0/3244400994 shutdown_connections 2024-06-10T02:56:07.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.072+0000 7f18d0d86b80 1 -- 172.21.15.52:0/3244400994 wait complete. 2024-06-10T02:56:07.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.072+0000 7f18d0d86b80 1 -- start start 2024-06-10T02:56:07.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.072+0000 7f18d0d86b80 1 Processor -- start 2024-06-10T02:56:07.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.072+0000 7f18d0d86b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556b26d38030 0x556b26d4b770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:56:07.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.072+0000 7f18d0d86b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556b26d478e0 0x556b26d4bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:56:07.076 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.073+0000 7f18d0d86b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556b26d4c1f0 0x556b26dc0d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:56:07.076 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.073+0000 7f18d0d86b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f18c01ad4d0 con 0x556b26d4c1f0 2024-06-10T02:56:07.076 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.073+0000 7f18d0d86b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f18c01ace00 con 0x556b26d38030 2024-06-10T02:56:07.076 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.073+0000 7f18d0d86b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f18c01aa000 con 0x556b26d478e0 2024-06-10T02:56:07.076 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.073+0000 7f18c4aa0700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556b26d4c1f0 0x556b26dc0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:56:07.076 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.073+0000 7f18d0d86b80 10 client.0 _ll_register_callbacks cb 0x556b26c205b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T02:56:07.076 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.073+0000 7f18c4aa0700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556b26d4c1f0 0x556b26dc0d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57182/0 (socket says 172.21.15.52:57182) 2024-06-10T02:56:07.076 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.073+0000 7f18c4aa0700 1 -- 172.21.15.52:0/1911357999 learned_addr learned my addr 172.21.15.52:0/1911357999 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:56:07.076 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.073+0000 7f18bf7fe700 1 --2- 172.21.15.52:0/1911357999 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556b26d478e0 0x556b26d4bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:56:07.076 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.073+0000 7f18bffff700 1 --2- 172.21.15.52:0/1911357999 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556b26d38030 0x556b26d4b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:56:07.076 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.073+0000 7f18c4aa0700 1 -- 172.21.15.52:0/1911357999 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556b26d478e0 msgr2=0x556b26d4bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:56:07.076 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.073+0000 7f18c4aa0700 1 --2- 172.21.15.52:0/1911357999 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556b26d478e0 0x556b26d4bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:07.076 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.073+0000 7f18c4aa0700 1 -- 172.21.15.52:0/1911357999 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556b26d38030 msgr2=0x556b26d4b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:56:07.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.073+0000 7f18c4aa0700 1 --2- 172.21.15.52:0/1911357999 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556b26d38030 0x556b26d4b770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:07.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.073+0000 7f18c4aa0700 1 -- 172.21.15.52:0/1911357999 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b26dc12a0 con 0x556b26d4c1f0 2024-06-10T02:56:07.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.073+0000 7f18c4aa0700 1 --2- 172.21.15.52:0/1911357999 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556b26d4c1f0 0x556b26dc0d60 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f18b8015dc0 tx=0x7f18b80135b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1346ab093bf3a2ac server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:56:07.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.073+0000 7f189f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-10T02:56:07.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.073+0000 7f189f7fe700 1 -- 172.21.15.52:0/1911357999 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18b8013a90 con 0x556b26d4c1f0 2024-06-10T02:56:07.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.074+0000 7f18d0d86b80 1 -- 172.21.15.52:0/1911357999 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f18b00a4eb0 con 0x556b26d4c1f0 2024-06-10T02:56:07.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.074+0000 7f189f7fe700 1 -- 172.21.15.52:0/1911357999 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f18b8013c30 con 0x556b26d4c1f0 2024-06-10T02:56:07.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.074+0000 7f189f7fe700 1 -- 172.21.15.52:0/1911357999 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f18b802a050 con 0x556b26d4c1f0 2024-06-10T02:56:07.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.074+0000 7f18d0d86b80 10 client.15762 fetch_fsmap learned FSMap version 31 2024-06-10T02:56:07.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.074+0000 7f18d0d86b80 1 -- 172.21.15.52:0/1911357999 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x556b26dc2620 con 0x556b26d4c1f0 2024-06-10T02:56:07.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.074+0000 7f189f7fe700 1 -- 172.21.15.52:0/1911357999 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f18b00a4eb0 con 0x556b26d4c1f0 2024-06-10T02:56:07.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.074+0000 7f189f7fe700 1 -- 172.21.15.52:0/1911357999 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 31) ==== 8366+0+0 (secure 0 0 0) 0x7f18b8021530 con 0x556b26d4c1f0 2024-06-10T02:56:07.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.074+0000 7f18d0d86b80 10 client.15762 fetch_fsmap finished waiting for FSMap version 31 2024-06-10T02:56:07.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.074+0000 7f18d0d86b80 10 client.15762 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T02:56:07.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.074+0000 7f18d0d86b80 20 client.15762 populate_metadata read hostname 'smithi052' 2024-06-10T02:56:07.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.074+0000 7f18d0d86b80 1 --2- 172.21.15.52:0/1911357999 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x556b26dc2d40 0x556b26dc5160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:56:07.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.074+0000 7f18d0d86b80 4 client.15762 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-10T02:56:07.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.074+0000 7f18d0d86b80 1 -- 172.21.15.52:0/1911357999 --> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556b26c922d0 con 0x556b26dc2d40 2024-06-10T02:56:07.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.075+0000 7f18bffff700 1 --2- 172.21.15.52:0/1911357999 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x556b26dc2d40 0x556b26dc5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:56:07.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.075+0000 7f18bffff700 1 --2- 172.21.15.52:0/1911357999 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x556b26dc2d40 0x556b26dc5160 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e6fde74132bedadd server_cookie=449298bfd68b72ca in_seq=0 out_seq=0 2024-06-10T02:56:07.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.075+0000 7f189f7fe700 10 client.15762 ms_handle_connect on v2:172.21.15.188:6838/310444836 2024-06-10T02:56:07.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.075+0000 7f189f7fe700 1 -- 172.21.15.52:0/1911357999 <== mds.0 v2:172.21.15.188:6838/310444836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x556b26c922d0 con 0x556b26dc2d40 2024-06-10T02:56:07.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.075+0000 7f189f7fe700 10 client.15762 handle_command_reply: tid=0 2024-06-10T02:56:07.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.075+0000 7f18d0d86b80 1 client.15762 shutdown 2024-06-10T02:56:07.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.075+0000 7f18d0d86b80 1 -- 172.21.15.52:0/1911357999 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556b26d4c1f0 msgr2=0x556b26dc0d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:56:07.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.075+0000 7f18d0d86b80 1 --2- 172.21.15.52:0/1911357999 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556b26d4c1f0 0x556b26dc0d60 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f18b8015dc0 tx=0x7f18b80135b0 comp rx=0 tx=0).stop 2024-06-10T02:56:07.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.075+0000 7f18d0d86b80 1 -- 172.21.15.52:0/1911357999 shutdown_connections 2024-06-10T02:56:07.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.075+0000 7f18d0d86b80 1 --2- 172.21.15.52:0/1911357999 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x556b26dc2d40 0x556b26dc5160 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:07.080 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.075+0000 7f18d0d86b80 1 --2- 172.21.15.52:0/1911357999 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556b26d4c1f0 0x556b26dc0d60 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:07.080 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.075+0000 7f18d0d86b80 1 --2- 172.21.15.52:0/1911357999 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556b26d38030 0x556b26d4b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:07.080 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.076+0000 7f189f7fe700 0 client.15762 ms_handle_reset on v2:172.21.15.188:6838/310444836 2024-06-10T02:56:07.080 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.076+0000 7f18d0d86b80 1 --2- 172.21.15.52:0/1911357999 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556b26d478e0 0x556b26d4bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:07.080 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.076+0000 7f18d0d86b80 1 -- 172.21.15.52:0/1911357999 >> 172.21.15.52:0/1911357999 conn(0x556b26c66080 msgr2=0x556b26d3a420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:56:07.080 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.076+0000 7f18d0d86b80 1 -- 172.21.15.52:0/1911357999 shutdown_connections 2024-06-10T02:56:07.080 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.076+0000 7f18d0d86b80 1 -- 172.21.15.52:0/1911357999 wait complete. 2024-06-10T02:56:07.080 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.076+0000 7f18d0d86b80 20 client.15762 trim_cache size 0 max 16384 2024-06-10T02:56:07.080 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-10T02:56:07.080 INFO:teuthology.orchestra.run.smithi052.stdout: "status": "no active scrubs running", 2024-06-10T02:56:07.080 INFO:teuthology.orchestra.run.smithi052.stdout: "scrubs": {} 2024-06-10T02:56:07.081 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-10T02:56:07.081 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.076+0000 7f18c6503700 1 -- 172.21.15.52:0/2160411454 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f18a80a2c20 msgr2=0x7f18a80a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:56:07.081 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.076+0000 7f18c6503700 1 --2- 172.21.15.52:0/2160411454 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f18a80a2c20 0x7f18a80a50e0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f18c0107af0 tx=0x7f18b4009260 comp rx=0 tx=0).stop 2024-06-10T02:56:07.081 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.076+0000 7f18c6503700 1 -- 172.21.15.52:0/2160411454 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f18c0101620 msgr2=0x7f18c019d750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:56:07.081 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.076+0000 7f18c6503700 1 --2- 172.21.15.52:0/2160411454 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f18c0101620 0x7f18c019d750 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f18b0005910 tx=0x7f18b000fa00 comp rx=0 tx=0).stop 2024-06-10T02:56:07.081 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.076+0000 7f18c6503700 1 -- 172.21.15.52:0/2160411454 shutdown_connections 2024-06-10T02:56:07.081 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.076+0000 7f18c6503700 1 --2- 172.21.15.52:0/2160411454 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f18c01066b0 0x7f18c0197960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:07.081 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.076+0000 7f18c6503700 1 --2- 172.21.15.52:0/2160411454 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f18a80a2c20 0x7f18a80a50e0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:07.082 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.076+0000 7f18c6503700 1 --2- 172.21.15.52:0/2160411454 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f18c0101620 0x7f18c019d750 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:07.082 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.076+0000 7f18c6503700 1 --2- 172.21.15.52:0/2160411454 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f18c0101fd0 0x7f18c019dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:07.082 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.077+0000 7f18c6503700 1 -- 172.21.15.52:0/2160411454 >> 172.21.15.52:0/2160411454 conn(0x7f18c0078340 msgr2=0x7f18c00ff410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:56:07.082 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.077+0000 7f18c6503700 1 -- 172.21.15.52:0/2160411454 shutdown_connections 2024-06-10T02:56:07.082 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:07.077+0000 7f18c6503700 1 -- 172.21.15.52:0/2160411454 wait complete. 2024-06-10T02:56:07.717 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-06-10T02:56:07.718 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T02:56:07.973 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:56:08.670 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:56:08 smithi052 ceph-mon[32362]: pgmap v492: 129 pgs: 129 active+clean; 3.5 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 285 KiB/s wr, 49 op/s 2024-06-10T02:56:08.888 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:56:08 smithi120 ceph-mon[36634]: pgmap v492: 129 pgs: 129 active+clean; 3.5 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 285 KiB/s wr, 49 op/s 2024-06-10T02:56:08.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:56:08 smithi188 ceph-mon[36599]: pgmap v492: 129 pgs: 129 active+clean; 3.5 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 285 KiB/s wr, 49 op/s 2024-06-10T02:56:09.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:09.558+0000 7f22a682b700 1 -- 172.21.15.52:0/3500606948 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f22a0102740 msgr2=0x7f22a0102b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:56:09.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:09.558+0000 7f22a682b700 1 --2- 172.21.15.52:0/3500606948 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f22a0102740 0x7f22a0102b20 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f22900099c0 tx=0x7f2290009cd0 comp rx=0 tx=0).stop 2024-06-10T02:56:09.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:09.558+0000 7f22a682b700 1 -- 172.21.15.52:0/3500606948 shutdown_connections 2024-06-10T02:56:09.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:09.558+0000 7f22a682b700 1 --2- 172.21.15.52:0/3500606948 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f22a0107840 0x7f22a010f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:09.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:09.558+0000 7f22a682b700 1 --2- 172.21.15.52:0/3500606948 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f22a0102740 0x7f22a0102b20 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:09.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:09.558+0000 7f22a682b700 1 --2- 172.21.15.52:0/3500606948 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f22a01030f0 0x7f22a0107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:09.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:09.558+0000 7f22a682b700 1 -- 172.21.15.52:0/3500606948 >> 172.21.15.52:0/3500606948 conn(0x7f22a00795f0 msgr2=0x7f22a0079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:56:09.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:09.558+0000 7f22a682b700 1 -- 172.21.15.52:0/3500606948 shutdown_connections 2024-06-10T02:56:09.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:09.558+0000 7f22a682b700 1 -- 172.21.15.52:0/3500606948 wait complete. 2024-06-10T02:56:09.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:09.559+0000 7f22a682b700 1 Processor -- start 2024-06-10T02:56:09.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:09.559+0000 7f22a682b700 1 -- start start 2024-06-10T02:56:09.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:09.559+0000 7f22a682b700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f22a0102740 0x7f22a019e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:56:09.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:09.559+0000 7f22a682b700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f22a01030f0 0x7f22a019ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:56:09.578 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:09.559+0000 7f22a682b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f22a0107840 0x7f22a0198a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:56:09.578 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:09.559+0000 7f22a682b700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f22a0112760 con 0x7f22a0107840 2024-06-10T02:56:09.578 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:09.559+0000 7f22a682b700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f22a01125e0 con 0x7f22a01030f0 2024-06-10T02:56:09.578 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:09.559+0000 7f22a682b700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f22a01128e0 con 0x7f22a0102740 2024-06-10T02:56:09.578 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:09.559+0000 7f22a4dc8700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f22a0107840 0x7f22a0198a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:56:09.578 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:09.559+0000 7f22a4dc8700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f22a0107840 0x7f22a0198a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57202/0 (socket says 172.21.15.52:57202) 2024-06-10T02:56:09.578 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:09.559+0000 7f22a4dc8700 1 -- 172.21.15.52:0/2062221148 learned_addr learned my addr 172.21.15.52:0/2062221148 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:56:09.578 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:09.560+0000 7f2297fff700 1 --2- 172.21.15.52:0/2062221148 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f22a01030f0 0x7f22a019ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:56:09.578 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:09.560+0000 7f229ffff700 1 --2- 172.21.15.52:0/2062221148 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f22a0102740 0x7f22a019e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:56:09.578 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:09.560+0000 7f22a4dc8700 1 -- 172.21.15.52:0/2062221148 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f22a0102740 msgr2=0x7f22a019e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:56:09.578 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:09.560+0000 7f22a4dc8700 1 --2- 172.21.15.52:0/2062221148 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f22a0102740 0x7f22a019e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:09.578 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:09.560+0000 7f22a4dc8700 1 -- 172.21.15.52:0/2062221148 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f22a01030f0 msgr2=0x7f22a019ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:56:09.579 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:09.560+0000 7f22a4dc8700 1 --2- 172.21.15.52:0/2062221148 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f22a01030f0 0x7f22a019ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:09.579 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:09.560+0000 7f22a4dc8700 1 -- 172.21.15.52:0/2062221148 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22a0199180 con 0x7f22a0107840 2024-06-10T02:56:09.579 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:09.560+0000 7f22a4dc8700 1 --2- 172.21.15.52:0/2062221148 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f22a0107840 0x7f22a0198a20 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f2298007ee0 tx=0x7f229800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:56:09.579 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:09.560+0000 7f229dffb700 1 -- 172.21.15.52:0/2062221148 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f229800dd20 con 0x7f22a0107840 2024-06-10T02:56:09.579 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:09.560+0000 7f229dffb700 1 -- 172.21.15.52:0/2062221148 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2298004020 con 0x7f22a0107840 2024-06-10T02:56:09.579 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:09.560+0000 7f22a682b700 1 -- 172.21.15.52:0/2062221148 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22a0199470 con 0x7f22a0107840 2024-06-10T02:56:09.579 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:09.560+0000 7f229dffb700 1 -- 172.21.15.52:0/2062221148 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2298015660 con 0x7f22a0107840 2024-06-10T02:56:09.579 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:09.560+0000 7f22a682b700 1 -- 172.21.15.52:0/2062221148 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22a01a56e0 con 0x7f22a0107840 2024-06-10T02:56:09.579 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:09.561+0000 7f22a682b700 1 -- 172.21.15.52:0/2062221148 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22a010aa80 con 0x7f22a0107840 2024-06-10T02:56:09.579 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:09.562+0000 7f229dffb700 1 -- 172.21.15.52:0/2062221148 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f22980041d0 con 0x7f22a0107840 2024-06-10T02:56:09.579 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:09.563+0000 7f229dffb700 1 --2- 172.21.15.52:0/2062221148 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f228c0a2ca0 0x7f228c0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:56:09.579 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:09.563+0000 7f229dffb700 1 -- 172.21.15.52:0/2062221148 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f22980cee10 con 0x7f22a0107840 2024-06-10T02:56:09.579 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:09.563+0000 7f229ffff700 1 --2- 172.21.15.52:0/2062221148 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f228c0a2ca0 0x7f228c0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:56:09.580 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:09.563+0000 7f229ffff700 1 --2- 172.21.15.52:0/2062221148 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f228c0a2ca0 0x7f228c0a5160 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f2290000c00 tx=0x7f2290005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:56:09.580 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:09.568+0000 7f229dffb700 1 -- 172.21.15.52:0/2062221148 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f22980d2030 con 0x7f22a0107840 2024-06-10T02:56:09.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:09.821+0000 7f22a682b700 1 -- 172.21.15.52:0/2062221148 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f22a010a460 con 0x7f22a0107840 2024-06-10T02:56:09.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:09.822+0000 7f229dffb700 1 -- 172.21.15.52:0/2062221148 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11939 (secure 0 0 0) 0x7f2298010070 con 0x7f22a0107840 2024-06-10T02:56:09.821 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-10T02:56:09.821 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/889673247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":889673247},{"type":"v1","addr":"172.21.15.52:6835","nonce":889673247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/3615663511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":3615663511},{"type":"v1","addr":"172.21.15.52:6837","nonce":3615663511}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":31,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T02:50:03.889766+0000","modified":"2024-06-10T02:55:51.192225+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24484,"mds_1":24469,"mds_2":24463,"mds_3":24449,"mds_4":24428},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14571":{"gid":14571,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6839/3059599042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":3059599042},{"type":"v1","addr":"172.21.15.52:6839","nonce":3059599042}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14583":{"gid":14583,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6841/3039275521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":3039275521},{"type":"v1","addr":"172.21.15.52:6841","nonce":3039275521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24383":{"gid":24383,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6833/2609002896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":2609002896},{"type":"v1","addr":"172.21.15.120:6833","nonce":2609002896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6835/1996370288","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":1996370288},{"type":"v1","addr":"172.21.15.120:6835","nonce":1996370288}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6833/529564979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":529564979},{"type":"v1","addr":"172.21.15.188:6833","nonce":529564979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":25,"addr":"172.21.15.120:6837/4132440952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":4132440952},{"type":"v1","addr":"172.21.15.120:6837","nonce":4132440952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"b","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.120:6839/1672436487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1672436487},{"type":"v1","addr":"172.21.15.120:6839","nonce":1672436487}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":2,"incarnation":22,"state":"up:active","state_seq":26,"addr":"172.21.15.188:6835/3904544259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":3904544259},{"type":"v1","addr":"172.21.15.188:6835","nonce":3904544259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.188:6837/819317726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":819317726},{"type":"v1","addr":"172.21.15.188:6837","nonce":819317726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.188:6839/310444836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":310444836},{"type":"v1","addr":"172.21.15.188:6839","nonce":310444836}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24484,"qdb_cluster":[24469,24449,24428,24463,24484]},"id":1}]} 2024-06-10T02:56:09.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:09.824+0000 7f22a682b700 1 -- 172.21.15.52:0/2062221148 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f228c0a2ca0 msgr2=0x7f228c0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:56:09.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:09.824+0000 7f22a682b700 1 --2- 172.21.15.52:0/2062221148 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f228c0a2ca0 0x7f228c0a5160 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f2290000c00 tx=0x7f2290005fb0 comp rx=0 tx=0).stop 2024-06-10T02:56:09.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:09.824+0000 7f22a682b700 1 -- 172.21.15.52:0/2062221148 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f22a0107840 msgr2=0x7f22a0198a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:56:09.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:09.824+0000 7f22a682b700 1 --2- 172.21.15.52:0/2062221148 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f22a0107840 0x7f22a0198a20 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f2298007ee0 tx=0x7f229800d3b0 comp rx=0 tx=0).stop 2024-06-10T02:56:09.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:09.824+0000 7f22a682b700 1 -- 172.21.15.52:0/2062221148 shutdown_connections 2024-06-10T02:56:09.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:09.824+0000 7f22a682b700 1 --2- 172.21.15.52:0/2062221148 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f228c0a2ca0 0x7f228c0a5160 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:09.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:09.824+0000 7f22a682b700 1 --2- 172.21.15.52:0/2062221148 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f22a0107840 0x7f22a0198a20 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:09.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:09.824+0000 7f22a682b700 1 --2- 172.21.15.52:0/2062221148 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f22a0102740 0x7f22a019e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:09.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:09.824+0000 7f22a682b700 1 --2- 172.21.15.52:0/2062221148 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f22a01030f0 0x7f22a019ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:09.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:09.824+0000 7f22a682b700 1 -- 172.21.15.52:0/2062221148 >> 172.21.15.52:0/2062221148 conn(0x7f22a00795f0 msgr2=0x7f22a0100730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:56:09.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:09.824+0000 7f22a682b700 1 -- 172.21.15.52:0/2062221148 shutdown_connections 2024-06-10T02:56:09.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:09.824+0000 7f22a682b700 1 -- 172.21.15.52:0/2062221148 wait complete. 2024-06-10T02:56:09.825 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 31 2024-06-10T02:56:10.572 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-06-10T02:56:10.829 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:56:10.888 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:56:10 smithi120 ceph-mon[36634]: pgmap v493: 129 pgs: 129 active+clean; 3.5 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 102 KiB/s wr, 37 op/s 2024-06-10T02:56:10.888 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:56:10 smithi120 ceph-mon[36634]: from='client.? 172.21.15.52:0/2062221148' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T02:56:10.949 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:56:10 smithi052 ceph-mon[32362]: pgmap v493: 129 pgs: 129 active+clean; 3.5 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 102 KiB/s wr, 37 op/s 2024-06-10T02:56:10.950 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:56:10 smithi052 ceph-mon[32362]: from='client.? 172.21.15.52:0/2062221148' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T02:56:10.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:56:10 smithi188 ceph-mon[36599]: pgmap v493: 129 pgs: 129 active+clean; 3.5 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 102 KiB/s wr, 37 op/s 2024-06-10T02:56:10.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:56:10 smithi188 ceph-mon[36599]: from='client.? 172.21.15.52:0/2062221148' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T02:56:12.888 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:56:12 smithi120 ceph-mon[36634]: pgmap v494: 129 pgs: 129 active+clean; 3.5 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 46 KiB/s wr, 24 op/s 2024-06-10T02:56:12.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:56:12 smithi188 ceph-mon[36599]: pgmap v494: 129 pgs: 129 active+clean; 3.5 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 46 KiB/s wr, 24 op/s 2024-06-10T02:56:12.976 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:56:12 smithi052 ceph-mon[32362]: pgmap v494: 129 pgs: 129 active+clean; 3.5 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 46 KiB/s wr, 24 op/s 2024-06-10T02:56:14.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.354+0000 7f80eb067700 1 -- 172.21.15.52:0/275171124 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f80e4102380 msgr2=0x7f80e4105ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:56:14.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.354+0000 7f80eb067700 1 --2- 172.21.15.52:0/275171124 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f80e4102380 0x7f80e4105ff0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f80d80099c0 tx=0x7f80d8009cd0 comp rx=0 tx=0).stop 2024-06-10T02:56:14.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.354+0000 7f80eb067700 1 -- 172.21.15.52:0/275171124 shutdown_connections 2024-06-10T02:56:14.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.354+0000 7f80eb067700 1 --2- 172.21.15.52:0/275171124 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f80e4106720 0x7f80e410e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:14.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.354+0000 7f80eb067700 1 --2- 172.21.15.52:0/275171124 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f80e41019d0 0x7f80e4101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:14.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.354+0000 7f80eb067700 1 --2- 172.21.15.52:0/275171124 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f80e4102380 0x7f80e4105ff0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:14.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.354+0000 7f80eb067700 1 -- 172.21.15.52:0/275171124 >> 172.21.15.52:0/275171124 conn(0x7f80e4078340 msgr2=0x7f80e4078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:56:14.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.354+0000 7f80eb067700 1 -- 172.21.15.52:0/275171124 shutdown_connections 2024-06-10T02:56:14.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.354+0000 7f80eb067700 1 -- 172.21.15.52:0/275171124 wait complete. 2024-06-10T02:56:14.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.355+0000 7f80eb067700 1 Processor -- start 2024-06-10T02:56:14.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.355+0000 7f80eb067700 1 -- start start 2024-06-10T02:56:14.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.355+0000 7f80eb067700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f80e41019d0 0x7f80e410eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:56:14.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.356+0000 7f80eb067700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f80e4102380 0x7f80e410f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:56:14.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.356+0000 7f80eb067700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f80e4106720 0x7f80e410f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:56:14.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.356+0000 7f80eb067700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f80e41127f0 con 0x7f80e41019d0 2024-06-10T02:56:14.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.356+0000 7f80eb067700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f80e4112670 con 0x7f80e4106720 2024-06-10T02:56:14.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.356+0000 7f80eb067700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f80e4112970 con 0x7f80e4102380 2024-06-10T02:56:14.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.356+0000 7f80e8e03700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f80e41019d0 0x7f80e410eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:56:14.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.356+0000 7f80e8e03700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f80e41019d0 0x7f80e410eca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:46544/0 (socket says 172.21.15.52:46544) 2024-06-10T02:56:14.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.356+0000 7f80e8e03700 1 -- 172.21.15.52:0/4048533397 learned_addr learned my addr 172.21.15.52:0/4048533397 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:56:14.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.356+0000 7f80e3fff700 1 --2- 172.21.15.52:0/4048533397 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f80e4102380 0x7f80e410f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:56:14.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.356+0000 7f80e9604700 1 --2- 172.21.15.52:0/4048533397 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f80e4106720 0x7f80e410f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:56:14.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.356+0000 7f80e8e03700 1 -- 172.21.15.52:0/4048533397 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f80e4102380 msgr2=0x7f80e410f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:56:14.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.356+0000 7f80e8e03700 1 --2- 172.21.15.52:0/4048533397 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f80e4102380 0x7f80e410f1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:14.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.356+0000 7f80e8e03700 1 -- 172.21.15.52:0/4048533397 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f80e4106720 msgr2=0x7f80e410f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:56:14.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.356+0000 7f80e8e03700 1 --2- 172.21.15.52:0/4048533397 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f80e4106720 0x7f80e410f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:14.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.356+0000 7f80e8e03700 1 -- 172.21.15.52:0/4048533397 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80e41a71c0 con 0x7f80e41019d0 2024-06-10T02:56:14.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.356+0000 7f80e8e03700 1 --2- 172.21.15.52:0/4048533397 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f80e41019d0 0x7f80e410eca0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f80d400c9b0 tx=0x7f80d400ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:56:14.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.356+0000 7f80e1ffb700 1 -- 172.21.15.52:0/4048533397 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80d4007b00 con 0x7f80e41019d0 2024-06-10T02:56:14.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.356+0000 7f80eb067700 1 -- 172.21.15.52:0/4048533397 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80e41a74b0 con 0x7f80e41019d0 2024-06-10T02:56:14.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.357+0000 7f80e1ffb700 1 -- 172.21.15.52:0/4048533397 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f80d4007ca0 con 0x7f80e41019d0 2024-06-10T02:56:14.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.357+0000 7f80e1ffb700 1 -- 172.21.15.52:0/4048533397 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80d4018730 con 0x7f80e41019d0 2024-06-10T02:56:14.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.357+0000 7f80eb067700 1 -- 172.21.15.52:0/4048533397 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80e41a7a10 con 0x7f80e41019d0 2024-06-10T02:56:14.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.359+0000 7f80e1ffb700 1 -- 172.21.15.52:0/4048533397 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f80d40188d0 con 0x7f80e41019d0 2024-06-10T02:56:14.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.360+0000 7f80e1ffb700 1 --2- 172.21.15.52:0/4048533397 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f80cc0a2ca0 0x7f80cc0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:56:14.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.360+0000 7f80e3fff700 1 --2- 172.21.15.52:0/4048533397 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f80cc0a2ca0 0x7f80cc0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:56:14.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.360+0000 7f80e1ffb700 1 -- 172.21.15.52:0/4048533397 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f80d40cf3e0 con 0x7f80e41019d0 2024-06-10T02:56:14.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.360+0000 7f80e3fff700 1 --2- 172.21.15.52:0/4048533397 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f80cc0a2ca0 0x7f80cc0a5160 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f80d8009990 tx=0x7f80d8005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:56:14.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.361+0000 7f80f58eab80 1 Processor -- start 2024-06-10T02:56:14.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.361+0000 7f80f58eab80 1 -- start start 2024-06-10T02:56:14.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.361+0000 7f80f58eab80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c081373640 0x55c081373a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:56:14.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.361+0000 7f80f58eab80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c081373f60 0x55c08136d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:56:14.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.361+0000 7f80f58eab80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c08136df20 0x55c08136e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:56:14.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.361+0000 7f80f58eab80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55c0812bf810 con 0x55c081373f60 2024-06-10T02:56:14.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.361+0000 7f80f58eab80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55c081267720 con 0x55c081373640 2024-06-10T02:56:14.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.361+0000 7f80f58eab80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55c0812bb7d0 con 0x55c08136df20 2024-06-10T02:56:14.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.361+0000 7f80e3fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c081373f60 0x55c08136d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:56:14.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.361+0000 7f80e3fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c081373f60 0x55c08136d9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:46558/0 (socket says 172.21.15.52:46558) 2024-06-10T02:56:14.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.361+0000 7f80e3fff700 1 -- 172.21.15.52:0/725867187 learned_addr learned my addr 172.21.15.52:0/725867187 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:56:14.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.361+0000 7f80e8e03700 1 --2- 172.21.15.52:0/725867187 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c081373640 0x55c081373a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:56:14.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.361+0000 7f80e9604700 1 --2- 172.21.15.52:0/725867187 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c08136df20 0x55c08136e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:56:14.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.361+0000 7f80e3fff700 1 -- 172.21.15.52:0/725867187 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c08136df20 msgr2=0x55c08136e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:56:14.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.361+0000 7f80e3fff700 1 --2- 172.21.15.52:0/725867187 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c08136df20 0x55c08136e340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:14.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.361+0000 7f80e3fff700 1 -- 172.21.15.52:0/725867187 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c081373640 msgr2=0x55c081373a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:56:14.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.361+0000 7f80e3fff700 1 --2- 172.21.15.52:0/725867187 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c081373640 0x55c081373a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:14.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.361+0000 7f80e3fff700 1 -- 172.21.15.52:0/725867187 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c08125df50 con 0x55c081373f60 2024-06-10T02:56:14.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.362+0000 7f80e3fff700 1 --2- 172.21.15.52:0/725867187 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c081373f60 0x55c08136d9e0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f80d80099c0 tx=0x7f80d8020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42d1f62041a4d063 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:56:14.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.362+0000 7f80caffd700 1 -- 172.21.15.52:0/725867187 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80d800bdc0 con 0x55c081373f60 2024-06-10T02:56:14.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.362+0000 7f80caffd700 1 -- 172.21.15.52:0/725867187 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f80d800fd20 con 0x55c081373f60 2024-06-10T02:56:14.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.362+0000 7f80caffd700 1 -- 172.21.15.52:0/725867187 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80d802f400 con 0x55c081373f60 2024-06-10T02:56:14.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.363+0000 7f80f58eab80 1 -- 172.21.15.52:0/725867187 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c081373f60 msgr2=0x55c08136d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:56:14.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.363+0000 7f80f58eab80 1 --2- 172.21.15.52:0/725867187 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c081373f60 0x55c08136d9e0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f80d80099c0 tx=0x7f80d8020000 comp rx=0 tx=0).stop 2024-06-10T02:56:14.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.363+0000 7f80f58eab80 1 -- 172.21.15.52:0/725867187 shutdown_connections 2024-06-10T02:56:14.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.363+0000 7f80f58eab80 1 --2- 172.21.15.52:0/725867187 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c08136df20 0x55c08136e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:14.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.363+0000 7f80f58eab80 1 --2- 172.21.15.52:0/725867187 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c081373640 0x55c081373a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:14.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.363+0000 7f80f58eab80 1 --2- 172.21.15.52:0/725867187 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c081373f60 0x55c08136d9e0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:14.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.363+0000 7f80f58eab80 1 -- 172.21.15.52:0/725867187 >> 172.21.15.52:0/725867187 conn(0x55c081298cc0 msgr2=0x55c081368850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:56:14.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.363+0000 7f80f58eab80 1 -- 172.21.15.52:0/725867187 shutdown_connections 2024-06-10T02:56:14.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.363+0000 7f80f58eab80 1 -- 172.21.15.52:0/725867187 wait complete. 2024-06-10T02:56:14.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.363+0000 7f80f58eab80 1 -- start start 2024-06-10T02:56:14.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.363+0000 7f80f58eab80 1 Processor -- start 2024-06-10T02:56:14.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.364+0000 7f80f58eab80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c08136df20 0x55c081379c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:56:14.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.364+0000 7f80f58eab80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c081373640 0x55c081376f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:56:14.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.364+0000 7f80f58eab80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c081373f60 0x55c0813774b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:56:14.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.364+0000 7f80f58eab80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55c08137a910 con 0x55c08136df20 2024-06-10T02:56:14.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.364+0000 7f80f58eab80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55c08125fbf0 con 0x55c081373640 2024-06-10T02:56:14.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.364+0000 7f80f58eab80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55c08137aa90 con 0x55c081373f60 2024-06-10T02:56:14.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.364+0000 7f80f58eab80 10 client.0 _ll_register_callbacks cb 0x55c081259920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T02:56:14.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.364+0000 7f80e8e03700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c08136df20 0x55c081379c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:56:14.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.364+0000 7f80e8e03700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c08136df20 0x55c081379c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:46574/0 (socket says 172.21.15.52:46574) 2024-06-10T02:56:14.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.364+0000 7f80e8e03700 1 -- 172.21.15.52:0/2561361283 learned_addr learned my addr 172.21.15.52:0/2561361283 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:56:14.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.364+0000 7f80e3fff700 1 --2- 172.21.15.52:0/2561361283 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c081373640 0x55c081376f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:56:14.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.364+0000 7f80e9604700 1 --2- 172.21.15.52:0/2561361283 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c081373f60 0x55c0813774b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:56:14.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.364+0000 7f80e8e03700 1 -- 172.21.15.52:0/2561361283 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c081373f60 msgr2=0x55c0813774b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:56:14.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.364+0000 7f80e8e03700 1 --2- 172.21.15.52:0/2561361283 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c081373f60 0x55c0813774b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:14.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.364+0000 7f80e8e03700 1 -- 172.21.15.52:0/2561361283 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c081373640 msgr2=0x55c081376f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:56:14.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.364+0000 7f80e8e03700 1 --2- 172.21.15.52:0/2561361283 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c081373640 0x55c081376f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:14.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.364+0000 7f80e8e03700 1 -- 172.21.15.52:0/2561361283 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c08120e8d0 con 0x55c08136df20 2024-06-10T02:56:14.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.364+0000 7f80e9604700 1 --2- 172.21.15.52:0/2561361283 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c081373f60 0x55c0813774b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T02:56:14.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.364+0000 7f80e3fff700 1 --2- 172.21.15.52:0/2561361283 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c081373640 0x55c081376f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T02:56:14.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.364+0000 7f80e8e03700 1 --2- 172.21.15.52:0/2561361283 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c08136df20 0x55c081379c70 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f80d400afd0 tx=0x7f80d409b190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54ebb0a6a787a155 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:56:14.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.364+0000 7f80c3fff700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-10T02:56:14.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.364+0000 7f80c3fff700 1 -- 172.21.15.52:0/2561361283 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80d40b4030 con 0x55c08136df20 2024-06-10T02:56:14.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.364+0000 7f80c3fff700 1 -- 172.21.15.52:0/2561361283 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f80d4099b30 con 0x55c08136df20 2024-06-10T02:56:14.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.364+0000 7f80f58eab80 1 -- 172.21.15.52:0/2561361283 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f80d800fd20 con 0x55c08136df20 2024-06-10T02:56:14.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.365+0000 7f80c3fff700 1 -- 172.21.15.52:0/2561361283 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f80d409b830 con 0x55c08136df20 2024-06-10T02:56:14.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.365+0000 7f80f58eab80 10 client.15780 fetch_fsmap learned FSMap version 31 2024-06-10T02:56:14.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.365+0000 7f80f58eab80 1 -- 172.21.15.52:0/2561361283 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55c081385290 con 0x55c08136df20 2024-06-10T02:56:14.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.365+0000 7f80c3fff700 1 -- 172.21.15.52:0/2561361283 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f80d800fd20 con 0x55c08136df20 2024-06-10T02:56:14.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.365+0000 7f80c3fff700 1 -- 172.21.15.52:0/2561361283 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 31) ==== 8366+0+0 (secure 0 0 0) 0x7f80d40a4010 con 0x55c08136df20 2024-06-10T02:56:14.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.365+0000 7f80f58eab80 10 client.15780 fetch_fsmap finished waiting for FSMap version 31 2024-06-10T02:56:14.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.365+0000 7f80f58eab80 10 client.15780 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-06-10T02:56:14.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.365+0000 7f80f58eab80 20 client.15780 populate_metadata read hostname 'smithi052' 2024-06-10T02:56:14.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.365+0000 7f80f58eab80 1 --2- 172.21.15.52:0/2561361283 >> [v2:172.21.15.120:6836/4132440952,v1:172.21.15.120:6837/4132440952] conn(0x55c081387820 0x55c081389c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:56:14.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.365+0000 7f80f58eab80 4 client.15780 mds_command: new command op to 24428 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T02:56:14.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.365+0000 7f80f58eab80 1 -- 172.21.15.52:0/2561361283 --> [v2:172.21.15.120:6836/4132440952,v1:172.21.15.120:6837/4132440952] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c0812ccd10 con 0x55c081387820 2024-06-10T02:56:14.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.365+0000 7f80e3fff700 1 --2- 172.21.15.52:0/2561361283 >> [v2:172.21.15.120:6836/4132440952,v1:172.21.15.120:6837/4132440952] conn(0x55c081387820 0x55c081389c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:56:14.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.366+0000 7f80e3fff700 1 --2- 172.21.15.52:0/2561361283 >> [v2:172.21.15.120:6836/4132440952,v1:172.21.15.120:6837/4132440952] conn(0x55c081387820 0x55c081389c10 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=9a23d5d064c7e96c server_cookie=f30d7c9a28ac9df6 in_seq=0 out_seq=0 2024-06-10T02:56:14.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.366+0000 7f80c3fff700 10 client.15780 ms_handle_connect on v2:172.21.15.120:6836/4132440952 2024-06-10T02:56:14.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.367+0000 7f80c3fff700 1 -- 172.21.15.52:0/2561361283 <== mds.4 v2:172.21.15.120:6836/4132440952 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55c0812ccd10 con 0x55c081387820 2024-06-10T02:56:14.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.367+0000 7f80c3fff700 10 client.15780 handle_command_reply: tid=0 2024-06-10T02:56:14.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.367+0000 7f80f58eab80 1 client.15780 shutdown 2024-06-10T02:56:14.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.367+0000 7f80f58eab80 1 -- 172.21.15.52:0/2561361283 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c08136df20 msgr2=0x55c081379c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:56:14.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.367+0000 7f80f58eab80 1 --2- 172.21.15.52:0/2561361283 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c08136df20 0x55c081379c70 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f80d400afd0 tx=0x7f80d409b190 comp rx=0 tx=0).stop 2024-06-10T02:56:14.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.367+0000 7f80f58eab80 1 -- 172.21.15.52:0/2561361283 shutdown_connections 2024-06-10T02:56:14.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.367+0000 7f80f58eab80 1 --2- 172.21.15.52:0/2561361283 >> [v2:172.21.15.120:6836/4132440952,v1:172.21.15.120:6837/4132440952] conn(0x55c081387820 0x55c081389c10 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:14.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.367+0000 7f80f58eab80 1 --2- 172.21.15.52:0/2561361283 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c081373f60 0x55c0813774b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:14.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.367+0000 7f80f58eab80 1 --2- 172.21.15.52:0/2561361283 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c08136df20 0x55c081379c70 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:14.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.367+0000 7f80f58eab80 1 --2- 172.21.15.52:0/2561361283 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c081373640 0x55c081376f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:14.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.367+0000 7f80f58eab80 1 -- 172.21.15.52:0/2561361283 >> 172.21.15.52:0/2561361283 conn(0x55c081298cc0 msgr2=0x55c0813685f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:56:14.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.367+0000 7f80c3fff700 0 client.15780 ms_handle_reset on v2:172.21.15.120:6836/4132440952 2024-06-10T02:56:14.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.367+0000 7f80f58eab80 1 -- 172.21.15.52:0/2561361283 shutdown_connections 2024-06-10T02:56:14.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.367+0000 7f80f58eab80 1 -- 172.21.15.52:0/2561361283 wait complete. 2024-06-10T02:56:14.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.367+0000 7f80f58eab80 20 client.15780 trim_cache size 0 max 16384 2024-06-10T02:56:14.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.382+0000 7f80f58eab80 1 Processor -- start 2024-06-10T02:56:14.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.382+0000 7f80f58eab80 1 -- start start 2024-06-10T02:56:14.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.382+0000 7f80f58eab80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c08136df20 0x55c08136d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:56:14.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.382+0000 7f80f58eab80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c081373640 0x55c08138f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:56:14.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.382+0000 7f80f58eab80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c081373f60 0x55c08138fa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:56:14.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.382+0000 7f80f58eab80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f80e41b03a0 con 0x55c081373f60 2024-06-10T02:56:14.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.383+0000 7f80f58eab80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f80e41b1f10 con 0x55c08136df20 2024-06-10T02:56:14.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.383+0000 7f80f58eab80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f80e41b0a70 con 0x55c081373640 2024-06-10T02:56:14.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.383+0000 7f80e9604700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c081373f60 0x55c08138fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:56:14.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.383+0000 7f80e9604700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c081373f60 0x55c08138fa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:46586/0 (socket says 172.21.15.52:46586) 2024-06-10T02:56:14.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.383+0000 7f80e9604700 1 -- 172.21.15.52:0/2598157645 learned_addr learned my addr 172.21.15.52:0/2598157645 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:56:14.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.383+0000 7f80e3fff700 1 --2- 172.21.15.52:0/2598157645 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c081373640 0x55c08138f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:56:14.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.383+0000 7f80e8e03700 1 --2- 172.21.15.52:0/2598157645 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c08136df20 0x55c08136d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:56:14.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.383+0000 7f80e9604700 1 -- 172.21.15.52:0/2598157645 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c081373640 msgr2=0x55c08138f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:56:14.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.383+0000 7f80e9604700 1 --2- 172.21.15.52:0/2598157645 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c081373640 0x55c08138f4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:14.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.383+0000 7f80e9604700 1 -- 172.21.15.52:0/2598157645 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c08136df20 msgr2=0x55c08136d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:56:14.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.383+0000 7f80e9604700 1 --2- 172.21.15.52:0/2598157645 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c08136df20 0x55c08136d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:14.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.383+0000 7f80e9604700 1 -- 172.21.15.52:0/2598157645 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c08138a270 con 0x55c081373f60 2024-06-10T02:56:14.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.383+0000 7f80e9604700 1 --2- 172.21.15.52:0/2598157645 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c081373f60 0x55c08138fa00 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f80dc009dc0 tx=0x7f80dc0076c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93baeaccda312956 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:56:14.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.383+0000 7f80ca7fc700 1 -- 172.21.15.52:0/2598157645 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80dc01b030 con 0x55c081373f60 2024-06-10T02:56:14.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.383+0000 7f80ca7fc700 1 -- 172.21.15.52:0/2598157645 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f80dc007e30 con 0x55c081373f60 2024-06-10T02:56:14.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.383+0000 7f80ca7fc700 1 -- 172.21.15.52:0/2598157645 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80dc017e00 con 0x55c081373f60 2024-06-10T02:56:14.388 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.384+0000 7f80f58eab80 1 -- 172.21.15.52:0/2598157645 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c081373f60 msgr2=0x55c08138fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:56:14.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.384+0000 7f80f58eab80 1 --2- 172.21.15.52:0/2598157645 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c081373f60 0x55c08138fa00 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f80dc009dc0 tx=0x7f80dc0076c0 comp rx=0 tx=0).stop 2024-06-10T02:56:14.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.384+0000 7f80f58eab80 1 -- 172.21.15.52:0/2598157645 shutdown_connections 2024-06-10T02:56:14.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.384+0000 7f80f58eab80 1 --2- 172.21.15.52:0/2598157645 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c081373f60 0x55c08138fa00 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:14.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.384+0000 7f80f58eab80 1 --2- 172.21.15.52:0/2598157645 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c08136df20 0x55c08136d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:14.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.384+0000 7f80f58eab80 1 --2- 172.21.15.52:0/2598157645 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c081373640 0x55c08138f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:14.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.384+0000 7f80f58eab80 1 -- 172.21.15.52:0/2598157645 >> 172.21.15.52:0/2598157645 conn(0x55c081298cc0 msgr2=0x55c0813710c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:56:14.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.384+0000 7f80f58eab80 1 -- 172.21.15.52:0/2598157645 shutdown_connections 2024-06-10T02:56:14.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.384+0000 7f80f58eab80 1 -- 172.21.15.52:0/2598157645 wait complete. 2024-06-10T02:56:14.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.384+0000 7f80f58eab80 1 -- start start 2024-06-10T02:56:14.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.385+0000 7f80f58eab80 1 Processor -- start 2024-06-10T02:56:14.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.385+0000 7f80f58eab80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c08136df20 0x55c081380320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:56:14.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.385+0000 7f80f58eab80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c081373640 0x55c08137b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:56:14.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.385+0000 7f80f58eab80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c081373f60 0x55c08137bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:56:14.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.385+0000 7f80f58eab80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f80e41b1810 con 0x55c08136df20 2024-06-10T02:56:14.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.385+0000 7f80f58eab80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f80e41b1140 con 0x55c081373f60 2024-06-10T02:56:14.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.385+0000 7f80f58eab80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f80e41ae340 con 0x55c081373640 2024-06-10T02:56:14.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.385+0000 7f80f58eab80 10 client.0 _ll_register_callbacks cb 0x55c081259920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T02:56:14.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.385+0000 7f80e8e03700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c08136df20 0x55c081380320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:56:14.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.385+0000 7f80e8e03700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c08136df20 0x55c081380320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:46598/0 (socket says 172.21.15.52:46598) 2024-06-10T02:56:14.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.385+0000 7f80e8e03700 1 -- 172.21.15.52:0/1901036823 learned_addr learned my addr 172.21.15.52:0/1901036823 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:56:14.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.385+0000 7f80e9604700 1 --2- 172.21.15.52:0/1901036823 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c081373f60 0x55c08137bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:56:14.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.385+0000 7f80e3fff700 1 --2- 172.21.15.52:0/1901036823 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c081373640 0x55c08137b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:56:14.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.385+0000 7f80e8e03700 1 -- 172.21.15.52:0/1901036823 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c081373640 msgr2=0x55c08137b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:56:14.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.385+0000 7f80e8e03700 1 --2- 172.21.15.52:0/1901036823 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c081373640 0x55c08137b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:14.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.385+0000 7f80e8e03700 1 -- 172.21.15.52:0/1901036823 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c081373f60 msgr2=0x55c08137bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:56:14.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.385+0000 7f80e8e03700 1 --2- 172.21.15.52:0/1901036823 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c081373f60 0x55c08137bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:14.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.385+0000 7f80e8e03700 1 -- 172.21.15.52:0/1901036823 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c08137c220 con 0x55c08136df20 2024-06-10T02:56:14.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.385+0000 7f80e3fff700 1 --2- 172.21.15.52:0/1901036823 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c081373640 0x55c08137b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T02:56:14.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.386+0000 7f80e8e03700 1 --2- 172.21.15.52:0/1901036823 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c08136df20 0x55c081380320 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f80d4097af0 tx=0x7f80d4099d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3bf5d1101f3f331 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:56:14.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.386+0000 7f80c3fff700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-10T02:56:14.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.386+0000 7f80c3fff700 1 -- 172.21.15.52:0/1901036823 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80d40b4040 con 0x55c08136df20 2024-06-10T02:56:14.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.386+0000 7f80c3fff700 1 -- 172.21.15.52:0/1901036823 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f80d409ac80 con 0x55c08136df20 2024-06-10T02:56:14.392 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.386+0000 7f80f58eab80 1 -- 172.21.15.52:0/1901036823 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f80dc007e30 con 0x55c08136df20 2024-06-10T02:56:14.392 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.386+0000 7f80c3fff700 1 -- 172.21.15.52:0/1901036823 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f80d40a44a0 con 0x55c08136df20 2024-06-10T02:56:14.392 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.386+0000 7f80f58eab80 10 client.15786 fetch_fsmap learned FSMap version 31 2024-06-10T02:56:14.392 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.386+0000 7f80f58eab80 1 -- 172.21.15.52:0/1901036823 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55c0813f2300 con 0x55c08136df20 2024-06-10T02:56:14.392 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.386+0000 7f80c3fff700 1 -- 172.21.15.52:0/1901036823 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f80dc007e30 con 0x55c08136df20 2024-06-10T02:56:14.392 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.386+0000 7f80c3fff700 1 -- 172.21.15.52:0/1901036823 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 31) ==== 8366+0+0 (secure 0 0 0) 0x7f80d409a6f0 con 0x55c08136df20 2024-06-10T02:56:14.392 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.386+0000 7f80f58eab80 10 client.15786 fetch_fsmap finished waiting for FSMap version 31 2024-06-10T02:56:14.392 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.386+0000 7f80f58eab80 10 client.15786 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-06-10T02:56:14.392 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.386+0000 7f80f58eab80 20 client.15786 populate_metadata read hostname 'smithi052' 2024-06-10T02:56:14.392 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.386+0000 7f80f58eab80 1 --2- 172.21.15.52:0/1901036823 >> [v2:172.21.15.120:6836/4132440952,v1:172.21.15.120:6837/4132440952] conn(0x55c0813f27d0 0x55c0813f4bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:56:14.392 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.386+0000 7f80f58eab80 4 client.15786 mds_command: new command op to 24428 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T02:56:14.393 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.386+0000 7f80f58eab80 1 -- 172.21.15.52:0/1901036823 --> [v2:172.21.15.120:6836/4132440952,v1:172.21.15.120:6837/4132440952] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c0812c7290 con 0x55c0813f27d0 2024-06-10T02:56:14.393 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.387+0000 7f80e3fff700 1 --2- 172.21.15.52:0/1901036823 >> [v2:172.21.15.120:6836/4132440952,v1:172.21.15.120:6837/4132440952] conn(0x55c0813f27d0 0x55c0813f4bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:56:14.393 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.387+0000 7f80e3fff700 1 --2- 172.21.15.52:0/1901036823 >> [v2:172.21.15.120:6836/4132440952,v1:172.21.15.120:6837/4132440952] conn(0x55c0813f27d0 0x55c0813f4bc0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=995a802323bd9d9c server_cookie=b3d530259345e0fe in_seq=0 out_seq=0 2024-06-10T02:56:14.393 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.387+0000 7f80c3fff700 10 client.15786 ms_handle_connect on v2:172.21.15.120:6836/4132440952 2024-06-10T02:56:14.393 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.387+0000 7f80c3fff700 1 -- 172.21.15.52:0/1901036823 <== mds.4 v2:172.21.15.120:6836/4132440952 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c0812c7290 con 0x55c0813f27d0 2024-06-10T02:56:14.393 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.387+0000 7f80c3fff700 10 client.15786 handle_command_reply: tid=0 2024-06-10T02:56:14.393 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.387+0000 7f80f58eab80 1 client.15786 shutdown 2024-06-10T02:56:14.393 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.387+0000 7f80f58eab80 1 -- 172.21.15.52:0/1901036823 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c08136df20 msgr2=0x55c081380320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:56:14.393 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.387+0000 7f80f58eab80 1 --2- 172.21.15.52:0/1901036823 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c08136df20 0x55c081380320 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f80d4097af0 tx=0x7f80d4099d40 comp rx=0 tx=0).stop 2024-06-10T02:56:14.393 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.388+0000 7f80f58eab80 1 -- 172.21.15.52:0/1901036823 shutdown_connections 2024-06-10T02:56:14.393 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.388+0000 7f80f58eab80 1 --2- 172.21.15.52:0/1901036823 >> [v2:172.21.15.120:6836/4132440952,v1:172.21.15.120:6837/4132440952] conn(0x55c0813f27d0 0x55c0813f4bc0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:14.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.388+0000 7f80f58eab80 1 --2- 172.21.15.52:0/1901036823 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c081373f60 0x55c08137bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:14.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.388+0000 7f80f58eab80 1 --2- 172.21.15.52:0/1901036823 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c08136df20 0x55c081380320 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:14.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.388+0000 7f80f58eab80 1 --2- 172.21.15.52:0/1901036823 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c081373640 0x55c08137b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:14.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.388+0000 7f80f58eab80 1 -- 172.21.15.52:0/1901036823 >> 172.21.15.52:0/1901036823 conn(0x55c081298cc0 msgr2=0x55c081361e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:56:14.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.388+0000 7f80c3fff700 0 client.15786 ms_handle_reset on v2:172.21.15.120:6836/4132440952 2024-06-10T02:56:14.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.388+0000 7f80f58eab80 1 -- 172.21.15.52:0/1901036823 shutdown_connections 2024-06-10T02:56:14.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.388+0000 7f80f58eab80 1 -- 172.21.15.52:0/1901036823 wait complete. 2024-06-10T02:56:14.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.388+0000 7f80f58eab80 20 client.15786 trim_cache size 0 max 16384 2024-06-10T02:56:14.395 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-06-10T02:56:14.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.388+0000 7f80eb067700 1 -- 172.21.15.52:0/4048533397 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f80cc0a2ca0 msgr2=0x7f80cc0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:56:14.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.388+0000 7f80eb067700 1 --2- 172.21.15.52:0/4048533397 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f80cc0a2ca0 0x7f80cc0a5160 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f80d8009990 tx=0x7f80d8005fb0 comp rx=0 tx=0).stop 2024-06-10T02:56:14.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.388+0000 7f80eb067700 1 -- 172.21.15.52:0/4048533397 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f80e41019d0 msgr2=0x7f80e410eca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:56:14.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.388+0000 7f80eb067700 1 --2- 172.21.15.52:0/4048533397 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f80e41019d0 0x7f80e410eca0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f80d400c9b0 tx=0x7f80d400ccc0 comp rx=0 tx=0).stop 2024-06-10T02:56:14.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.388+0000 7f80eb067700 1 -- 172.21.15.52:0/4048533397 shutdown_connections 2024-06-10T02:56:14.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.388+0000 7f80eb067700 1 --2- 172.21.15.52:0/4048533397 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f80e4106720 0x7f80e410f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:14.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.388+0000 7f80eb067700 1 --2- 172.21.15.52:0/4048533397 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f80cc0a2ca0 0x7f80cc0a5160 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:14.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.388+0000 7f80eb067700 1 --2- 172.21.15.52:0/4048533397 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f80e41019d0 0x7f80e410eca0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:14.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.388+0000 7f80eb067700 1 --2- 172.21.15.52:0/4048533397 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f80e4102380 0x7f80e410f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:14.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.388+0000 7f80eb067700 1 -- 172.21.15.52:0/4048533397 >> 172.21.15.52:0/4048533397 conn(0x7f80e4078340 msgr2=0x7f80e40ff860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:56:14.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.389+0000 7f80eb067700 1 -- 172.21.15.52:0/4048533397 shutdown_connections 2024-06-10T02:56:14.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:14.389+0000 7f80eb067700 1 -- 172.21.15.52:0/4048533397 wait complete. 2024-06-10T02:56:14.674 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:56:14 smithi052 ceph-mon[32362]: pgmap v495: 129 pgs: 129 active+clean; 3.5 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 16 KiB/s wr, 16 op/s 2024-06-10T02:56:14.888 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:56:14 smithi120 ceph-mon[36634]: pgmap v495: 129 pgs: 129 active+clean; 3.5 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 16 KiB/s wr, 16 op/s 2024-06-10T02:56:14.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:56:14 smithi188 ceph-mon[36599]: pgmap v495: 129 pgs: 129 active+clean; 3.5 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 16 KiB/s wr, 16 op/s 2024-06-10T02:56:15.167 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-06-10T02:56:15.422 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:56:16.805 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:56:16 smithi052 ceph-mon[32362]: pgmap v496: 129 pgs: 129 active+clean; 3.5 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 5.3 KiB/s wr, 10 op/s 2024-06-10T02:56:16.888 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:56:16 smithi120 ceph-mon[36634]: pgmap v496: 129 pgs: 129 active+clean; 3.5 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 5.3 KiB/s wr, 10 op/s 2024-06-10T02:56:16.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:56:16 smithi188 ceph-mon[36599]: pgmap v496: 129 pgs: 129 active+clean; 3.5 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 5.3 KiB/s wr, 10 op/s 2024-06-10T02:56:17.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.198+0000 7f3f61f8d700 1 -- 172.21.15.52:0/820488995 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3f5c101d50 msgr2=0x7f3f5c10e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:56:17.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.198+0000 7f3f61f8d700 1 --2- 172.21.15.52:0/820488995 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3f5c101d50 0x7f3f5c10e240 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f3f5400b0a0 tx=0x7f3f5400b3b0 comp rx=0 tx=0).stop 2024-06-10T02:56:17.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.198+0000 7f3f61f8d700 1 -- 172.21.15.52:0/820488995 shutdown_connections 2024-06-10T02:56:17.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.198+0000 7f3f61f8d700 1 --2- 172.21.15.52:0/820488995 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3f5c101d50 0x7f3f5c10e240 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:17.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.198+0000 7f3f61f8d700 1 --2- 172.21.15.52:0/820488995 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3f5c107400 0x7f3f5c1077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:17.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.198+0000 7f3f61f8d700 1 --2- 172.21.15.52:0/820488995 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3f5c101390 0x7f3f5c101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:17.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.198+0000 7f3f61f8d700 1 -- 172.21.15.52:0/820488995 >> 172.21.15.52:0/820488995 conn(0x7f3f5c078340 msgr2=0x7f3f5c078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:56:17.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.198+0000 7f3f61f8d700 1 -- 172.21.15.52:0/820488995 shutdown_connections 2024-06-10T02:56:17.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.198+0000 7f3f61f8d700 1 -- 172.21.15.52:0/820488995 wait complete. 2024-06-10T02:56:17.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.199+0000 7f3f61f8d700 1 Processor -- start 2024-06-10T02:56:17.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.199+0000 7f3f61f8d700 1 -- start start 2024-06-10T02:56:17.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.199+0000 7f3f61f8d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3f5c101390 0x7f3f5c19e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:56:17.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.199+0000 7f3f61f8d700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3f5c101d50 0x7f3f5c19ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:56:17.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.199+0000 7f3f61f8d700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3f5c107400 0x7f3f5c198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:56:17.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.199+0000 7f3f61f8d700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f3f5c111690 con 0x7f3f5c101390 2024-06-10T02:56:17.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.199+0000 7f3f61f8d700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f3f5c111510 con 0x7f3f5c101d50 2024-06-10T02:56:17.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.199+0000 7f3f61f8d700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f3f5c111810 con 0x7f3f5c107400 2024-06-10T02:56:17.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.199+0000 7f3f5b7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3f5c101390 0x7f3f5c19e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:56:17.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.199+0000 7f3f5b7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3f5c101390 0x7f3f5c19e800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:46618/0 (socket says 172.21.15.52:46618) 2024-06-10T02:56:17.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.200+0000 7f3f5b7fe700 1 -- 172.21.15.52:0/1277614074 learned_addr learned my addr 172.21.15.52:0/1277614074 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:56:17.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.200+0000 7f3f52ffd700 1 --2- 172.21.15.52:0/1277614074 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3f5c101d50 0x7f3f5c19ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:56:17.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.200+0000 7f3f5bfff700 1 --2- 172.21.15.52:0/1277614074 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3f5c107400 0x7f3f5c198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:56:17.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.200+0000 7f3f5b7fe700 1 -- 172.21.15.52:0/1277614074 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3f5c107400 msgr2=0x7f3f5c198a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:56:17.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.200+0000 7f3f5b7fe700 1 --2- 172.21.15.52:0/1277614074 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3f5c107400 0x7f3f5c198a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:17.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.200+0000 7f3f5b7fe700 1 -- 172.21.15.52:0/1277614074 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3f5c101d50 msgr2=0x7f3f5c19ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:56:17.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.200+0000 7f3f5b7fe700 1 --2- 172.21.15.52:0/1277614074 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3f5c101d50 0x7f3f5c19ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:17.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.200+0000 7f3f5b7fe700 1 -- 172.21.15.52:0/1277614074 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f5c199170 con 0x7f3f5c101390 2024-06-10T02:56:17.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.200+0000 7f3f52ffd700 1 --2- 172.21.15.52:0/1277614074 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3f5c101d50 0x7f3f5c19ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T02:56:17.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.200+0000 7f3f5b7fe700 1 --2- 172.21.15.52:0/1277614074 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3f5c101390 0x7f3f5c19e800 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f3f4c00c970 tx=0x7f3f4c00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:56:17.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.200+0000 7f3f597fa700 1 -- 172.21.15.52:0/1277614074 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f4c007aa0 con 0x7f3f5c101390 2024-06-10T02:56:17.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.200+0000 7f3f597fa700 1 -- 172.21.15.52:0/1277614074 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3f4c004d10 con 0x7f3f5c101390 2024-06-10T02:56:17.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.200+0000 7f3f61f8d700 1 -- 172.21.15.52:0/1277614074 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f5c199460 con 0x7f3f5c101390 2024-06-10T02:56:17.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.200+0000 7f3f61f8d700 1 -- 172.21.15.52:0/1277614074 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f5c1a56d0 con 0x7f3f5c101390 2024-06-10T02:56:17.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.200+0000 7f3f597fa700 1 -- 172.21.15.52:0/1277614074 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f4c00f450 con 0x7f3f5c101390 2024-06-10T02:56:17.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.203+0000 7f3f597fa700 1 -- 172.21.15.52:0/1277614074 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f3f4c004750 con 0x7f3f5c101390 2024-06-10T02:56:17.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.203+0000 7f3f597fa700 1 --2- 172.21.15.52:0/1277614074 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f3f480a2ca0 0x7f3f480a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:56:17.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.203+0000 7f3f597fa700 1 -- 172.21.15.52:0/1277614074 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f3f4c0cf110 con 0x7f3f5c101390 2024-06-10T02:56:17.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.203+0000 7f3f52ffd700 1 --2- 172.21.15.52:0/1277614074 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f3f480a2ca0 0x7f3f480a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:56:17.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.204+0000 7f3f52ffd700 1 --2- 172.21.15.52:0/1277614074 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f3f480a2ca0 0x7f3f480a5160 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f3f44005950 tx=0x7f3f440058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:56:17.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.204+0000 7f3f6c810b80 1 Processor -- start 2024-06-10T02:56:17.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.204+0000 7f3f6c810b80 1 -- start start 2024-06-10T02:56:17.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.204+0000 7f3f6c810b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562531988640 0x562531988a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:56:17.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.204+0000 7f3f6c810b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562531988f60 0x5625319829e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:56:17.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.204+0000 7f3f6c810b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562531982f20 0x562531983340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:56:17.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.204+0000 7f3f6c810b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5625318d4810 con 0x562531988640 2024-06-10T02:56:17.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.204+0000 7f3f6c810b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56253187c720 con 0x562531982f20 2024-06-10T02:56:17.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.204+0000 7f3f6c810b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5625318d07d0 con 0x562531988f60 2024-06-10T02:56:17.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.204+0000 7f3f5b7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562531988640 0x562531988a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:56:17.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.204+0000 7f3f5b7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562531988640 0x562531988a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:46634/0 (socket says 172.21.15.52:46634) 2024-06-10T02:56:17.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.204+0000 7f3f5b7fe700 1 -- 172.21.15.52:0/4278812993 learned_addr learned my addr 172.21.15.52:0/4278812993 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:56:17.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.205+0000 7f3f52ffd700 1 --2- 172.21.15.52:0/4278812993 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562531988f60 0x5625319829e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:56:17.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.205+0000 7f3f5bfff700 1 --2- 172.21.15.52:0/4278812993 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562531982f20 0x562531983340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:56:17.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.205+0000 7f3f52ffd700 1 -- 172.21.15.52:0/4278812993 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562531982f20 msgr2=0x562531983340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:56:17.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.205+0000 7f3f52ffd700 1 --2- 172.21.15.52:0/4278812993 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562531982f20 0x562531983340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:17.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.205+0000 7f3f52ffd700 1 -- 172.21.15.52:0/4278812993 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562531988640 msgr2=0x562531988a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:56:17.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.205+0000 7f3f52ffd700 1 --2- 172.21.15.52:0/4278812993 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562531988640 0x562531988a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:17.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.205+0000 7f3f52ffd700 1 -- 172.21.15.52:0/4278812993 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562531872f50 con 0x562531988f60 2024-06-10T02:56:17.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.205+0000 7f3f52ffd700 1 --2- 172.21.15.52:0/4278812993 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562531988f60 0x5625319829e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f3f4400fd40 tx=0x7f3f44012000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e2d9ef271863d11a server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:56:17.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.205+0000 7f3f51ffb700 1 -- 172.21.15.52:0/4278812993 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f44003880 con 0x562531988f60 2024-06-10T02:56:17.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.205+0000 7f3f51ffb700 1 -- 172.21.15.52:0/4278812993 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3f44024070 con 0x562531988f60 2024-06-10T02:56:17.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.205+0000 7f3f51ffb700 1 -- 172.21.15.52:0/4278812993 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f4401fc90 con 0x562531988f60 2024-06-10T02:56:17.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.206+0000 7f3f6c810b80 1 -- 172.21.15.52:0/4278812993 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562531988f60 msgr2=0x5625319829e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:56:17.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.206+0000 7f3f6c810b80 1 --2- 172.21.15.52:0/4278812993 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562531988f60 0x5625319829e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f3f4400fd40 tx=0x7f3f44012000 comp rx=0 tx=0).stop 2024-06-10T02:56:17.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.206+0000 7f3f6c810b80 1 -- 172.21.15.52:0/4278812993 shutdown_connections 2024-06-10T02:56:17.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.206+0000 7f3f6c810b80 1 --2- 172.21.15.52:0/4278812993 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562531982f20 0x562531983340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:17.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.206+0000 7f3f6c810b80 1 --2- 172.21.15.52:0/4278812993 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562531988640 0x562531988a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:17.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.206+0000 7f3f6c810b80 1 --2- 172.21.15.52:0/4278812993 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562531988f60 0x5625319829e0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:17.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.206+0000 7f3f6c810b80 1 -- 172.21.15.52:0/4278812993 >> 172.21.15.52:0/4278812993 conn(0x5625318adcc0 msgr2=0x56253197d850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:56:17.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.206+0000 7f3f6c810b80 1 -- 172.21.15.52:0/4278812993 shutdown_connections 2024-06-10T02:56:17.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.206+0000 7f3f6c810b80 1 -- 172.21.15.52:0/4278812993 wait complete. 2024-06-10T02:56:17.212 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.206+0000 7f3f6c810b80 1 -- start start 2024-06-10T02:56:17.212 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.207+0000 7f3f6c810b80 1 Processor -- start 2024-06-10T02:56:17.212 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.207+0000 7f3f6c810b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562531982f20 0x56253198ec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:56:17.212 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.207+0000 7f3f6c810b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562531988640 0x56253198bf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:56:17.212 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.207+0000 7f3f6c810b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562531988f60 0x56253198c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:56:17.212 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.207+0000 7f3f6c810b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x56253198f9d0 con 0x562531982f20 2024-06-10T02:56:17.212 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.207+0000 7f3f6c810b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x562531874bf0 con 0x562531988640 2024-06-10T02:56:17.212 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.207+0000 7f3f6c810b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x56253198fb50 con 0x562531988f60 2024-06-10T02:56:17.212 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.207+0000 7f3f5b7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562531982f20 0x56253198ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:56:17.212 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.207+0000 7f3f5b7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562531982f20 0x56253198ec80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:46640/0 (socket says 172.21.15.52:46640) 2024-06-10T02:56:17.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.207+0000 7f3f5b7fe700 1 -- 172.21.15.52:0/1978355336 learned_addr learned my addr 172.21.15.52:0/1978355336 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:56:17.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.207+0000 7f3f6c810b80 10 client.0 _ll_register_callbacks cb 0x56253186e920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T02:56:17.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.207+0000 7f3f52ffd700 1 --2- 172.21.15.52:0/1978355336 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562531988640 0x56253198bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:56:17.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.207+0000 7f3f5bfff700 1 --2- 172.21.15.52:0/1978355336 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562531988f60 0x56253198c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:56:17.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.207+0000 7f3f5b7fe700 1 -- 172.21.15.52:0/1978355336 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562531988f60 msgr2=0x56253198c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:56:17.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.207+0000 7f3f5b7fe700 1 --2- 172.21.15.52:0/1978355336 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562531988f60 0x56253198c4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:17.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.207+0000 7f3f5b7fe700 1 -- 172.21.15.52:0/1978355336 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562531988640 msgr2=0x56253198bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:56:17.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.207+0000 7f3f5b7fe700 1 --2- 172.21.15.52:0/1978355336 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562531988640 0x56253198bf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:17.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.207+0000 7f3f5b7fe700 1 -- 172.21.15.52:0/1978355336 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5625318238d0 con 0x562531982f20 2024-06-10T02:56:17.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.208+0000 7f3f52ffd700 1 --2- 172.21.15.52:0/1978355336 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562531988640 0x56253198bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T02:56:17.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.208+0000 7f3f5b7fe700 1 --2- 172.21.15.52:0/1978355336 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562531982f20 0x56253198ec80 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f3f4c0a20d0 tx=0x7f3f4c09b550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=738f341eaf126b18 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:56:17.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.208+0000 7f3f3affd700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-10T02:56:17.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.208+0000 7f3f3affd700 1 -- 172.21.15.52:0/1978355336 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f4c0b8020 con 0x562531982f20 2024-06-10T02:56:17.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.208+0000 7f3f3affd700 1 -- 172.21.15.52:0/1978355336 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f3f4c004a40 con 0x562531982f20 2024-06-10T02:56:17.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.208+0000 7f3f6c810b80 1 -- 172.21.15.52:0/1978355336 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3f44024070 con 0x562531982f20 2024-06-10T02:56:17.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.208+0000 7f3f3affd700 1 -- 172.21.15.52:0/1978355336 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f3f4c09afb0 con 0x562531982f20 2024-06-10T02:56:17.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.208+0000 7f3f6c810b80 10 client.15798 fetch_fsmap learned FSMap version 31 2024-06-10T02:56:17.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.208+0000 7f3f6c810b80 1 -- 172.21.15.52:0/1978355336 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x56253199a310 con 0x562531982f20 2024-06-10T02:56:17.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.208+0000 7f3f3affd700 1 -- 172.21.15.52:0/1978355336 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3f44024070 con 0x562531982f20 2024-06-10T02:56:17.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.208+0000 7f3f3affd700 1 -- 172.21.15.52:0/1978355336 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 31) ==== 8366+0+0 (secure 0 0 0) 0x7f3f4c0a3da0 con 0x562531982f20 2024-06-10T02:56:17.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.208+0000 7f3f6c810b80 10 client.15798 fetch_fsmap finished waiting for FSMap version 31 2024-06-10T02:56:17.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.208+0000 7f3f6c810b80 10 client.15798 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.b 2024-06-10T02:56:17.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.209+0000 7f3f6c810b80 20 client.15798 populate_metadata read hostname 'smithi052' 2024-06-10T02:56:17.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.209+0000 7f3f6c810b80 1 --2- 172.21.15.52:0/1978355336 >> [v2:172.21.15.120:6838/1672436487,v1:172.21.15.120:6839/1672436487] conn(0x56253199c950 0x56253199ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:56:17.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.209+0000 7f3f6c810b80 4 client.15798 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T02:56:17.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.209+0000 7f3f6c810b80 1 -- 172.21.15.52:0/1978355336 --> [v2:172.21.15.120:6838/1672436487,v1:172.21.15.120:6839/1672436487] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5625318e1d10 con 0x56253199c950 2024-06-10T02:56:17.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.209+0000 7f3f52ffd700 1 --2- 172.21.15.52:0/1978355336 >> [v2:172.21.15.120:6838/1672436487,v1:172.21.15.120:6839/1672436487] conn(0x56253199c950 0x56253199ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:56:17.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.209+0000 7f3f3affd700 10 client.15798 ms_handle_connect on v2:172.21.15.120:6838/1672436487 2024-06-10T02:56:17.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.209+0000 7f3f52ffd700 1 --2- 172.21.15.52:0/1978355336 >> [v2:172.21.15.120:6838/1672436487,v1:172.21.15.120:6839/1672436487] conn(0x56253199c950 0x56253199ed90 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ba61c03de5a72c2 server_cookie=16c66f42204b4817 in_seq=0 out_seq=0 2024-06-10T02:56:17.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.210+0000 7f3f3affd700 1 -- 172.21.15.52:0/1978355336 <== mds.3 v2:172.21.15.120:6838/1672436487 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5625318e1d10 con 0x56253199c950 2024-06-10T02:56:17.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.210+0000 7f3f3affd700 10 client.15798 handle_command_reply: tid=0 2024-06-10T02:56:17.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.210+0000 7f3f6c810b80 1 client.15798 shutdown 2024-06-10T02:56:17.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.210+0000 7f3f6c810b80 1 -- 172.21.15.52:0/1978355336 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562531982f20 msgr2=0x56253198ec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:56:17.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.210+0000 7f3f6c810b80 1 --2- 172.21.15.52:0/1978355336 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562531982f20 0x56253198ec80 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f3f4c0a20d0 tx=0x7f3f4c09b550 comp rx=0 tx=0).stop 2024-06-10T02:56:17.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.210+0000 7f3f6c810b80 1 -- 172.21.15.52:0/1978355336 shutdown_connections 2024-06-10T02:56:17.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.210+0000 7f3f6c810b80 1 --2- 172.21.15.52:0/1978355336 >> [v2:172.21.15.120:6838/1672436487,v1:172.21.15.120:6839/1672436487] conn(0x56253199c950 0x56253199ed90 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:17.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.210+0000 7f3f6c810b80 1 --2- 172.21.15.52:0/1978355336 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562531988f60 0x56253198c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:17.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.210+0000 7f3f6c810b80 1 --2- 172.21.15.52:0/1978355336 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562531982f20 0x56253198ec80 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:17.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.210+0000 7f3f6c810b80 1 --2- 172.21.15.52:0/1978355336 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562531988640 0x56253198bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:17.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.210+0000 7f3f6c810b80 1 -- 172.21.15.52:0/1978355336 >> 172.21.15.52:0/1978355336 conn(0x5625318adcc0 msgr2=0x56253197d5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:56:17.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.210+0000 7f3f3affd700 0 client.15798 ms_handle_reset on v2:172.21.15.120:6838/1672436487 2024-06-10T02:56:17.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.211+0000 7f3f6c810b80 1 -- 172.21.15.52:0/1978355336 shutdown_connections 2024-06-10T02:56:17.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.211+0000 7f3f6c810b80 1 -- 172.21.15.52:0/1978355336 wait complete. 2024-06-10T02:56:17.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.211+0000 7f3f6c810b80 20 client.15798 trim_cache size 0 max 16384 2024-06-10T02:56:17.226 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.226+0000 7f3f6c810b80 1 Processor -- start 2024-06-10T02:56:17.226 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.226+0000 7f3f6c810b80 1 -- start start 2024-06-10T02:56:17.226 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.226+0000 7f3f6c810b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562531982f20 0x5625319829e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:56:17.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.226+0000 7f3f6c810b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562531988640 0x562531989a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:56:17.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.226+0000 7f3f6c810b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562531988f60 0x5625319a4660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:56:17.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.226+0000 7f3f6c810b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f3f5c1ac1c0 con 0x562531988640 2024-06-10T02:56:17.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.226+0000 7f3f6c810b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f3f5c1add30 con 0x562531982f20 2024-06-10T02:56:17.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.226+0000 7f3f6c810b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f3f5c1ac890 con 0x562531988f60 2024-06-10T02:56:17.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.226+0000 7f3f52ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562531988640 0x562531989a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:56:17.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.226+0000 7f3f52ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562531988640 0x562531989a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:46642/0 (socket says 172.21.15.52:46642) 2024-06-10T02:56:17.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.226+0000 7f3f52ffd700 1 -- 172.21.15.52:0/2123507534 learned_addr learned my addr 172.21.15.52:0/2123507534 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:56:17.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.226+0000 7f3f5bfff700 1 --2- 172.21.15.52:0/2123507534 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562531988f60 0x5625319a4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:56:17.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.226+0000 7f3f5b7fe700 1 --2- 172.21.15.52:0/2123507534 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562531982f20 0x5625319829e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:56:17.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.226+0000 7f3f52ffd700 1 -- 172.21.15.52:0/2123507534 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562531988f60 msgr2=0x5625319a4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:56:17.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.226+0000 7f3f52ffd700 1 --2- 172.21.15.52:0/2123507534 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562531988f60 0x5625319a4660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:17.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.226+0000 7f3f52ffd700 1 -- 172.21.15.52:0/2123507534 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562531982f20 msgr2=0x5625319829e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:56:17.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.226+0000 7f3f52ffd700 1 --2- 172.21.15.52:0/2123507534 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562531982f20 0x5625319829e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:17.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.226+0000 7f3f52ffd700 1 -- 172.21.15.52:0/2123507534 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56253198f460 con 0x562531988640 2024-06-10T02:56:17.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.226+0000 7f3f5bfff700 1 --2- 172.21.15.52:0/2123507534 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562531988f60 0x5625319a4660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T02:56:17.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.226+0000 7f3f5b7fe700 1 --2- 172.21.15.52:0/2123507534 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562531982f20 0x5625319829e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T02:56:17.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.227+0000 7f3f52ffd700 1 --2- 172.21.15.52:0/2123507534 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562531988640 0x562531989a90 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f3f4400fd40 tx=0x7f3f44003a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80bae7dcdaaf2ad3 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:56:17.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.227+0000 7f3f517fa700 1 -- 172.21.15.52:0/2123507534 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f44003ba0 con 0x562531988640 2024-06-10T02:56:17.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.227+0000 7f3f517fa700 1 -- 172.21.15.52:0/2123507534 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3f44024070 con 0x562531988640 2024-06-10T02:56:17.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.227+0000 7f3f517fa700 1 -- 172.21.15.52:0/2123507534 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f44004970 con 0x562531988640 2024-06-10T02:56:17.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.227+0000 7f3f6c810b80 1 -- 172.21.15.52:0/2123507534 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562531988640 msgr2=0x562531989a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:56:17.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.227+0000 7f3f6c810b80 1 --2- 172.21.15.52:0/2123507534 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562531988640 0x562531989a90 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f3f4400fd40 tx=0x7f3f44003a30 comp rx=0 tx=0).stop 2024-06-10T02:56:17.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.227+0000 7f3f6c810b80 1 -- 172.21.15.52:0/2123507534 shutdown_connections 2024-06-10T02:56:17.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.227+0000 7f3f6c810b80 1 --2- 172.21.15.52:0/2123507534 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562531988f60 0x5625319a4660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:17.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.227+0000 7f3f6c810b80 1 --2- 172.21.15.52:0/2123507534 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562531982f20 0x5625319829e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:17.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.227+0000 7f3f6c810b80 1 --2- 172.21.15.52:0/2123507534 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562531988640 0x562531989a90 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:17.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.227+0000 7f3f6c810b80 1 -- 172.21.15.52:0/2123507534 >> 172.21.15.52:0/2123507534 conn(0x5625318adcc0 msgr2=0x562531989380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:56:17.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.228+0000 7f3f6c810b80 1 -- 172.21.15.52:0/2123507534 shutdown_connections 2024-06-10T02:56:17.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.228+0000 7f3f6c810b80 1 -- 172.21.15.52:0/2123507534 wait complete. 2024-06-10T02:56:17.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.228+0000 7f3f6c810b80 1 -- start start 2024-06-10T02:56:17.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.228+0000 7f3f6c810b80 1 Processor -- start 2024-06-10T02:56:17.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.228+0000 7f3f6c810b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562531982f20 0x562531995330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:56:17.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.228+0000 7f3f6c810b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562531988640 0x5625319a3cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:56:17.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.228+0000 7f3f6c810b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562531988f60 0x5625319a41f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:56:17.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.228+0000 7f3f6c810b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f3f5c1ad630 con 0x562531988f60 2024-06-10T02:56:17.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.228+0000 7f3f6c810b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f3f5c1acf60 con 0x562531988640 2024-06-10T02:56:17.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.228+0000 7f3f6c810b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f3f5c1aa160 con 0x562531982f20 2024-06-10T02:56:17.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.228+0000 7f3f6c810b80 10 client.0 _ll_register_callbacks cb 0x56253186e920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T02:56:17.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.229+0000 7f3f5bfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562531988f60 0x5625319a41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:56:17.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.229+0000 7f3f5bfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562531988f60 0x5625319a41f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:46648/0 (socket says 172.21.15.52:46648) 2024-06-10T02:56:17.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.229+0000 7f3f5bfff700 1 -- 172.21.15.52:0/2545703640 learned_addr learned my addr 172.21.15.52:0/2545703640 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:56:17.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.229+0000 7f3f5b7fe700 1 --2- 172.21.15.52:0/2545703640 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562531982f20 0x562531995330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:56:17.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.229+0000 7f3f52ffd700 1 --2- 172.21.15.52:0/2545703640 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562531988640 0x5625319a3cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:56:17.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.229+0000 7f3f5bfff700 1 -- 172.21.15.52:0/2545703640 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562531982f20 msgr2=0x562531995330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:56:17.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.229+0000 7f3f5bfff700 1 --2- 172.21.15.52:0/2545703640 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562531982f20 0x562531995330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:17.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.229+0000 7f3f5bfff700 1 -- 172.21.15.52:0/2545703640 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562531988640 msgr2=0x5625319a3cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:56:17.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.229+0000 7f3f5bfff700 1 --2- 172.21.15.52:0/2545703640 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562531988640 0x5625319a3cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:17.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.229+0000 7f3f5bfff700 1 -- 172.21.15.52:0/2545703640 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5625319a4730 con 0x562531988f60 2024-06-10T02:56:17.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.229+0000 7f3f52ffd700 1 --2- 172.21.15.52:0/2545703640 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562531988640 0x5625319a3cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T02:56:17.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.229+0000 7f3f5bfff700 1 --2- 172.21.15.52:0/2545703640 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562531988f60 0x5625319a41f0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f3f54015000 tx=0x7f3f54003d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d87822948c6f6af5 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:56:17.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.229+0000 7f3f3affd700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-10T02:56:17.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.229+0000 7f3f3affd700 1 -- 172.21.15.52:0/2545703640 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f540076f0 con 0x562531988f60 2024-06-10T02:56:17.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.229+0000 7f3f3affd700 1 -- 172.21.15.52:0/2545703640 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f3f54007890 con 0x562531988f60 2024-06-10T02:56:17.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.229+0000 7f3f6c810b80 1 -- 172.21.15.52:0/2545703640 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3f44024070 con 0x562531988f60 2024-06-10T02:56:17.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.229+0000 7f3f3affd700 1 -- 172.21.15.52:0/2545703640 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f3f54032050 con 0x562531988f60 2024-06-10T02:56:17.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.229+0000 7f3f6c810b80 10 client.15804 fetch_fsmap learned FSMap version 31 2024-06-10T02:56:17.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.229+0000 7f3f6c810b80 1 -- 172.21.15.52:0/2545703640 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x562531a072b0 con 0x562531988f60 2024-06-10T02:56:17.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.230+0000 7f3f3affd700 1 -- 172.21.15.52:0/2545703640 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3f44024070 con 0x562531988f60 2024-06-10T02:56:17.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.230+0000 7f3f3affd700 1 -- 172.21.15.52:0/2545703640 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 31) ==== 8366+0+0 (secure 0 0 0) 0x7f3f5402d030 con 0x562531988f60 2024-06-10T02:56:17.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.230+0000 7f3f6c810b80 10 client.15804 fetch_fsmap finished waiting for FSMap version 31 2024-06-10T02:56:17.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.230+0000 7f3f6c810b80 10 client.15804 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.b 2024-06-10T02:56:17.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.230+0000 7f3f6c810b80 20 client.15804 populate_metadata read hostname 'smithi052' 2024-06-10T02:56:17.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.230+0000 7f3f6c810b80 1 --2- 172.21.15.52:0/2545703640 >> [v2:172.21.15.120:6838/1672436487,v1:172.21.15.120:6839/1672436487] conn(0x562531a07a70 0x562531a09e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:56:17.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.230+0000 7f3f6c810b80 4 client.15804 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T02:56:17.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.230+0000 7f3f6c810b80 1 -- 172.21.15.52:0/2545703640 --> [v2:172.21.15.120:6838/1672436487,v1:172.21.15.120:6839/1672436487] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5625318dc290 con 0x562531a07a70 2024-06-10T02:56:17.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.230+0000 7f3f5b7fe700 1 --2- 172.21.15.52:0/2545703640 >> [v2:172.21.15.120:6838/1672436487,v1:172.21.15.120:6839/1672436487] conn(0x562531a07a70 0x562531a09e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:56:17.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.231+0000 7f3f5b7fe700 1 --2- 172.21.15.52:0/2545703640 >> [v2:172.21.15.120:6838/1672436487,v1:172.21.15.120:6839/1672436487] conn(0x562531a07a70 0x562531a09e90 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=e65195d4534489aa server_cookie=6ab3ff4173a80321 in_seq=0 out_seq=0 2024-06-10T02:56:17.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.231+0000 7f3f3affd700 10 client.15804 ms_handle_connect on v2:172.21.15.120:6838/1672436487 2024-06-10T02:56:17.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.231+0000 7f3f3affd700 1 -- 172.21.15.52:0/2545703640 <== mds.3 v2:172.21.15.120:6838/1672436487 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5625318dc290 con 0x562531a07a70 2024-06-10T02:56:17.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.231+0000 7f3f3affd700 10 client.15804 handle_command_reply: tid=0 2024-06-10T02:56:17.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.231+0000 7f3f6c810b80 1 client.15804 shutdown 2024-06-10T02:56:17.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.231+0000 7f3f6c810b80 1 -- 172.21.15.52:0/2545703640 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562531988f60 msgr2=0x5625319a41f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:56:17.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.231+0000 7f3f6c810b80 1 --2- 172.21.15.52:0/2545703640 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562531988f60 0x5625319a41f0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f3f54015000 tx=0x7f3f54003d70 comp rx=0 tx=0).stop 2024-06-10T02:56:17.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.231+0000 7f3f6c810b80 1 -- 172.21.15.52:0/2545703640 shutdown_connections 2024-06-10T02:56:17.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.231+0000 7f3f6c810b80 1 --2- 172.21.15.52:0/2545703640 >> [v2:172.21.15.120:6838/1672436487,v1:172.21.15.120:6839/1672436487] conn(0x562531a07a70 0x562531a09e90 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:17.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.231+0000 7f3f6c810b80 1 --2- 172.21.15.52:0/2545703640 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562531988f60 0x5625319a41f0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:17.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.231+0000 7f3f6c810b80 1 --2- 172.21.15.52:0/2545703640 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562531982f20 0x562531995330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:17.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.231+0000 7f3f3affd700 0 client.15804 ms_handle_reset on v2:172.21.15.120:6838/1672436487 2024-06-10T02:56:17.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.231+0000 7f3f6c810b80 1 --2- 172.21.15.52:0/2545703640 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562531988640 0x5625319a3cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:17.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.231+0000 7f3f6c810b80 1 -- 172.21.15.52:0/2545703640 >> 172.21.15.52:0/2545703640 conn(0x5625318adcc0 msgr2=0x562531976e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:56:17.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.231+0000 7f3f6c810b80 1 -- 172.21.15.52:0/2545703640 shutdown_connections 2024-06-10T02:56:17.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.231+0000 7f3f6c810b80 1 -- 172.21.15.52:0/2545703640 wait complete. 2024-06-10T02:56:17.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.231+0000 7f3f6c810b80 20 client.15804 trim_cache size 0 max 16384 2024-06-10T02:56:17.239 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-06-10T02:56:17.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.232+0000 7f3f61f8d700 1 -- 172.21.15.52:0/1277614074 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f3f480a2ca0 msgr2=0x7f3f480a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:56:17.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.232+0000 7f3f61f8d700 1 --2- 172.21.15.52:0/1277614074 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f3f480a2ca0 0x7f3f480a5160 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f3f44005950 tx=0x7f3f440058e0 comp rx=0 tx=0).stop 2024-06-10T02:56:17.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.232+0000 7f3f61f8d700 1 -- 172.21.15.52:0/1277614074 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3f5c101390 msgr2=0x7f3f5c19e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:56:17.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.232+0000 7f3f61f8d700 1 --2- 172.21.15.52:0/1277614074 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3f5c101390 0x7f3f5c19e800 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f3f4c00c970 tx=0x7f3f4c00cc80 comp rx=0 tx=0).stop 2024-06-10T02:56:17.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.232+0000 7f3f61f8d700 1 -- 172.21.15.52:0/1277614074 shutdown_connections 2024-06-10T02:56:17.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.232+0000 7f3f61f8d700 1 --2- 172.21.15.52:0/1277614074 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3f5c107400 0x7f3f5c198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:17.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.232+0000 7f3f61f8d700 1 --2- 172.21.15.52:0/1277614074 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f3f480a2ca0 0x7f3f480a5160 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:17.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.232+0000 7f3f61f8d700 1 --2- 172.21.15.52:0/1277614074 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3f5c101390 0x7f3f5c19e800 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:17.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.232+0000 7f3f61f8d700 1 --2- 172.21.15.52:0/1277614074 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3f5c101d50 0x7f3f5c19ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:17.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.232+0000 7f3f61f8d700 1 -- 172.21.15.52:0/1277614074 >> 172.21.15.52:0/1277614074 conn(0x7f3f5c078340 msgr2=0x7f3f5c1005d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:56:17.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.232+0000 7f3f61f8d700 1 -- 172.21.15.52:0/1277614074 shutdown_connections 2024-06-10T02:56:17.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:17.232+0000 7f3f61f8d700 1 -- 172.21.15.52:0/1277614074 wait complete. 2024-06-10T02:56:17.985 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-06-10T02:56:18.243 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:56:18.664 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:56:18 smithi052 ceph-mon[32362]: pgmap v497: 129 pgs: 129 active+clean; 3.5 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 6.6 KiB/s wr, 14 op/s 2024-06-10T02:56:18.888 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:56:18 smithi120 ceph-mon[36634]: pgmap v497: 129 pgs: 129 active+clean; 3.5 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 6.6 KiB/s wr, 14 op/s 2024-06-10T02:56:18.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:56:18 smithi188 ceph-mon[36599]: pgmap v497: 129 pgs: 129 active+clean; 3.5 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 6.6 KiB/s wr, 14 op/s 2024-06-10T02:56:19.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:19.987+0000 7fe9d264d700 1 -- 172.21.15.52:0/595563571 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe9cc06ac60 msgr2=0x7fe9cc06b0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:56:19.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:19.987+0000 7fe9d264d700 1 --2- 172.21.15.52:0/595563571 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe9cc06ac60 0x7fe9cc06b0e0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fe9c00099c0 tx=0x7fe9c0009cd0 comp rx=0 tx=0).stop 2024-06-10T02:56:19.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:19.987+0000 7fe9d264d700 1 -- 172.21.15.52:0/595563571 shutdown_connections 2024-06-10T02:56:19.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:19.987+0000 7fe9d264d700 1 --2- 172.21.15.52:0/595563571 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe9cc06b620 0x7fe9cc10f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:19.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:19.987+0000 7fe9d264d700 1 --2- 172.21.15.52:0/595563571 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe9cc108580 0x7fe9cc108960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:19.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:19.987+0000 7fe9d264d700 1 --2- 172.21.15.52:0/595563571 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe9cc06ac60 0x7fe9cc06b0e0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:19.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:19.987+0000 7fe9d264d700 1 -- 172.21.15.52:0/595563571 >> 172.21.15.52:0/595563571 conn(0x7fe9cc078430 msgr2=0x7fe9cc078840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:56:19.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:19.987+0000 7fe9d264d700 1 -- 172.21.15.52:0/595563571 shutdown_connections 2024-06-10T02:56:19.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:19.987+0000 7fe9d264d700 1 -- 172.21.15.52:0/595563571 wait complete. 2024-06-10T02:56:19.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:19.988+0000 7fe9d264d700 1 Processor -- start 2024-06-10T02:56:19.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:19.988+0000 7fe9d264d700 1 -- start start 2024-06-10T02:56:19.989 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:19.988+0000 7fe9d264d700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe9cc06ac60 0x7fe9cc19e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:56:19.989 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:19.988+0000 7fe9d264d700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe9cc06b620 0x7fe9cc19edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:56:19.989 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:19.988+0000 7fe9d264d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe9cc108580 0x7fe9cc198a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:56:19.990 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:19.988+0000 7fe9d264d700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fe9cc1127d0 con 0x7fe9cc108580 2024-06-10T02:56:19.990 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:19.988+0000 7fe9d264d700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe9cc112650 con 0x7fe9cc06ac60 2024-06-10T02:56:19.990 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:19.988+0000 7fe9d264d700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fe9cc112950 con 0x7fe9cc06b620 2024-06-10T02:56:19.990 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:19.988+0000 7fe9d0bea700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe9cc108580 0x7fe9cc198a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:56:19.990 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:19.988+0000 7fe9d0bea700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe9cc108580 0x7fe9cc198a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:46676/0 (socket says 172.21.15.52:46676) 2024-06-10T02:56:19.990 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:19.988+0000 7fe9d0bea700 1 -- 172.21.15.52:0/3000091199 learned_addr learned my addr 172.21.15.52:0/3000091199 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:56:19.990 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:19.988+0000 7fe9cb7fe700 1 --2- 172.21.15.52:0/3000091199 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe9cc06b620 0x7fe9cc19edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:56:19.990 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:19.989+0000 7fe9cbfff700 1 --2- 172.21.15.52:0/3000091199 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe9cc06ac60 0x7fe9cc19e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:56:19.990 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:19.989+0000 7fe9d0bea700 1 -- 172.21.15.52:0/3000091199 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe9cc06b620 msgr2=0x7fe9cc19edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:56:19.990 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:19.989+0000 7fe9d0bea700 1 --2- 172.21.15.52:0/3000091199 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe9cc06b620 0x7fe9cc19edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:19.990 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:19.989+0000 7fe9d0bea700 1 -- 172.21.15.52:0/3000091199 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe9cc06ac60 msgr2=0x7fe9cc19e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:56:19.990 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:19.989+0000 7fe9d0bea700 1 --2- 172.21.15.52:0/3000091199 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe9cc06ac60 0x7fe9cc19e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:19.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:19.989+0000 7fe9d0bea700 1 -- 172.21.15.52:0/3000091199 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9cc1991d0 con 0x7fe9cc108580 2024-06-10T02:56:19.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:19.989+0000 7fe9d0bea700 1 --2- 172.21.15.52:0/3000091199 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe9cc108580 0x7fe9cc198a70 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fe9c4007ee0 tx=0x7fe9c400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:56:19.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:19.989+0000 7fe9c97fa700 1 -- 172.21.15.52:0/3000091199 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9c400dbd0 con 0x7fe9cc108580 2024-06-10T02:56:19.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:19.989+0000 7fe9d264d700 1 -- 172.21.15.52:0/3000091199 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9cc1994c0 con 0x7fe9cc108580 2024-06-10T02:56:19.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:19.989+0000 7fe9c97fa700 1 -- 172.21.15.52:0/3000091199 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe9c400dd70 con 0x7fe9cc108580 2024-06-10T02:56:19.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:19.989+0000 7fe9c97fa700 1 -- 172.21.15.52:0/3000091199 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9c40133f0 con 0x7fe9cc108580 2024-06-10T02:56:19.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:19.989+0000 7fe9d264d700 1 -- 172.21.15.52:0/3000091199 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9cc1a5780 con 0x7fe9cc108580 2024-06-10T02:56:19.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:19.991+0000 7fe9c97fa700 1 -- 172.21.15.52:0/3000091199 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fe9c401a020 con 0x7fe9cc108580 2024-06-10T02:56:19.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:19.992+0000 7fe9c97fa700 1 --2- 172.21.15.52:0/3000091199 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fe9b40a2c20 0x7fe9b40a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:56:19.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:19.992+0000 7fe9cbfff700 1 --2- 172.21.15.52:0/3000091199 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fe9b40a2c20 0x7fe9b40a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:56:19.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:19.992+0000 7fe9c97fa700 1 -- 172.21.15.52:0/3000091199 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7fe9c40cec00 con 0x7fe9cc108580 2024-06-10T02:56:19.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:19.993+0000 7fe9cbfff700 1 --2- 172.21.15.52:0/3000091199 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fe9b40a2c20 0x7fe9b40a50e0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fe9cc199bb0 tx=0x7fe9bc00b5b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:56:19.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:19.993+0000 7fe9dced0b80 1 Processor -- start 2024-06-10T02:56:19.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:19.993+0000 7fe9dced0b80 1 -- start start 2024-06-10T02:56:19.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:19.993+0000 7fe9dced0b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5607faf95a70 0x5607faf95e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:56:19.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:19.993+0000 7fe9dced0b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5607faf96390 0x5607faf8fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:56:19.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:19.993+0000 7fe9dced0b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5607faf90330 0x5607faf90750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:56:19.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:19.993+0000 7fe9dced0b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5607faedd810 con 0x5607faf96390 2024-06-10T02:56:19.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:19.993+0000 7fe9dced0b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5607fae85720 con 0x5607faf95a70 2024-06-10T02:56:19.996 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:19.993+0000 7fe9dced0b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5607faed97d0 con 0x5607faf90330 2024-06-10T02:56:19.996 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:19.993+0000 7fe9cb7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5607faf96390 0x5607faf8fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:56:19.997 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:19.993+0000 7fe9cb7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5607faf96390 0x5607faf8fdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:46680/0 (socket says 172.21.15.52:46680) 2024-06-10T02:56:19.997 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:19.993+0000 7fe9cb7fe700 1 -- 172.21.15.52:0/2594907795 learned_addr learned my addr 172.21.15.52:0/2594907795 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:56:19.997 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:19.993+0000 7fe9cbfff700 1 --2- 172.21.15.52:0/2594907795 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5607faf95a70 0x5607faf95e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:56:19.997 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:19.993+0000 7fe9d0bea700 1 --2- 172.21.15.52:0/2594907795 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5607faf90330 0x5607faf90750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:56:19.997 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:19.994+0000 7fe9cb7fe700 1 -- 172.21.15.52:0/2594907795 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5607faf90330 msgr2=0x5607faf90750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:56:19.997 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:19.994+0000 7fe9cb7fe700 1 --2- 172.21.15.52:0/2594907795 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5607faf90330 0x5607faf90750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:19.997 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:19.994+0000 7fe9cb7fe700 1 -- 172.21.15.52:0/2594907795 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5607faf95a70 msgr2=0x5607faf95e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:56:19.997 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:19.994+0000 7fe9cb7fe700 1 --2- 172.21.15.52:0/2594907795 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5607faf95a70 0x5607faf95e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:19.997 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:19.994+0000 7fe9cb7fe700 1 -- 172.21.15.52:0/2594907795 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5607fae7bf50 con 0x5607faf96390 2024-06-10T02:56:19.997 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:19.994+0000 7fe9d0bea700 1 --2- 172.21.15.52:0/2594907795 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5607faf90330 0x5607faf90750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T02:56:19.998 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:19.994+0000 7fe9cb7fe700 1 --2- 172.21.15.52:0/2594907795 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5607faf96390 0x5607faf8fdf0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fe9c00098e0 tx=0x7fe9c0005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=886f198eca33a62e server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:56:19.998 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:19.994+0000 7fe9b27fc700 1 -- 172.21.15.52:0/2594907795 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9c0027070 con 0x5607faf96390 2024-06-10T02:56:19.998 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:19.994+0000 7fe9b27fc700 1 -- 172.21.15.52:0/2594907795 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe9c0003e40 con 0x5607faf96390 2024-06-10T02:56:19.998 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:19.994+0000 7fe9b27fc700 1 -- 172.21.15.52:0/2594907795 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9c000ba20 con 0x5607faf96390 2024-06-10T02:56:19.998 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:19.995+0000 7fe9dced0b80 1 -- 172.21.15.52:0/2594907795 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5607faf96390 msgr2=0x5607faf8fdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:56:19.998 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:19.995+0000 7fe9dced0b80 1 --2- 172.21.15.52:0/2594907795 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5607faf96390 0x5607faf8fdf0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fe9c00098e0 tx=0x7fe9c0005e80 comp rx=0 tx=0).stop 2024-06-10T02:56:19.998 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:19.995+0000 7fe9dced0b80 1 -- 172.21.15.52:0/2594907795 shutdown_connections 2024-06-10T02:56:19.998 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:19.995+0000 7fe9dced0b80 1 --2- 172.21.15.52:0/2594907795 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5607faf90330 0x5607faf90750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:19.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:19.995+0000 7fe9dced0b80 1 --2- 172.21.15.52:0/2594907795 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5607faf95a70 0x5607faf95e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:19.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:19.995+0000 7fe9dced0b80 1 --2- 172.21.15.52:0/2594907795 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5607faf96390 0x5607faf8fdf0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:19.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:19.995+0000 7fe9dced0b80 1 -- 172.21.15.52:0/2594907795 >> 172.21.15.52:0/2594907795 conn(0x5607faeb6cc0 msgr2=0x5607faf86850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:56:19.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:19.995+0000 7fe9dced0b80 1 -- 172.21.15.52:0/2594907795 shutdown_connections 2024-06-10T02:56:19.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:19.995+0000 7fe9dced0b80 1 -- 172.21.15.52:0/2594907795 wait complete. 2024-06-10T02:56:19.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:19.995+0000 7fe9dced0b80 1 -- start start 2024-06-10T02:56:19.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:19.996+0000 7fe9dced0b80 1 Processor -- start 2024-06-10T02:56:19.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:19.996+0000 7fe9dced0b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5607faf90330 0x5607faf9c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:56:19.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:19.996+0000 7fe9dced0b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5607faf95a70 0x5607faf99440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:56:19.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:19.996+0000 7fe9dced0b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5607faf96390 0x5607faf99980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:56:20.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:19.996+0000 7fe9dced0b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5607faf9cd70 con 0x5607faf96390 2024-06-10T02:56:20.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:19.996+0000 7fe9dced0b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5607fae7dbf0 con 0x5607faf95a70 2024-06-10T02:56:20.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:19.996+0000 7fe9dced0b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5607faf9cef0 con 0x5607faf90330 2024-06-10T02:56:20.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:19.996+0000 7fe9dced0b80 10 client.0 _ll_register_callbacks cb 0x5607fae77920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T02:56:20.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:19.996+0000 7fe9d0bea700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5607faf96390 0x5607faf99980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:56:20.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:19.996+0000 7fe9d0bea700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5607faf96390 0x5607faf99980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:46694/0 (socket says 172.21.15.52:46694) 2024-06-10T02:56:20.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:19.996+0000 7fe9d0bea700 1 -- 172.21.15.52:0/516603537 learned_addr learned my addr 172.21.15.52:0/516603537 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:56:20.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:19.996+0000 7fe9cbfff700 1 --2- 172.21.15.52:0/516603537 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5607faf90330 0x5607faf9c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:56:20.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:19.996+0000 7fe9cb7fe700 1 --2- 172.21.15.52:0/516603537 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5607faf95a70 0x5607faf99440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:56:20.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:19.997+0000 7fe9d0bea700 1 -- 172.21.15.52:0/516603537 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5607faf90330 msgr2=0x5607faf9c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:56:20.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:19.997+0000 7fe9d0bea700 1 --2- 172.21.15.52:0/516603537 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5607faf90330 0x5607faf9c140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:20.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:19.997+0000 7fe9d0bea700 1 -- 172.21.15.52:0/516603537 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5607faf95a70 msgr2=0x5607faf99440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:56:20.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:19.997+0000 7fe9d0bea700 1 --2- 172.21.15.52:0/516603537 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5607faf95a70 0x5607faf99440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:20.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:19.997+0000 7fe9d0bea700 1 -- 172.21.15.52:0/516603537 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5607fae2c8d0 con 0x5607faf96390 2024-06-10T02:56:20.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:19.997+0000 7fe9d0bea700 1 --2- 172.21.15.52:0/516603537 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5607faf96390 0x5607faf99980 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fe9c40974b0 tx=0x7fe9c409aa80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fa405e93470a484 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:56:20.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:19.997+0000 7fe9ab7fe700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-10T02:56:20.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:19.997+0000 7fe9ab7fe700 1 -- 172.21.15.52:0/516603537 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9c409f400 con 0x5607faf96390 2024-06-10T02:56:20.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:19.997+0000 7fe9ab7fe700 1 -- 172.21.15.52:0/516603537 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7fe9c4004c40 con 0x5607faf96390 2024-06-10T02:56:20.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:19.997+0000 7fe9dced0b80 1 -- 172.21.15.52:0/516603537 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5607faeead10 con 0x5607faf96390 2024-06-10T02:56:20.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:19.997+0000 7fe9ab7fe700 1 -- 172.21.15.52:0/516603537 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fe9c40a3cc0 con 0x5607faf96390 2024-06-10T02:56:20.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:19.997+0000 7fe9dced0b80 10 client.15816 fetch_fsmap learned FSMap version 31 2024-06-10T02:56:20.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:19.997+0000 7fe9dced0b80 1 -- 172.21.15.52:0/516603537 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5607fafa7720 con 0x5607faf96390 2024-06-10T02:56:20.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:19.997+0000 7fe9ab7fe700 1 -- 172.21.15.52:0/516603537 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5607faeead10 con 0x5607faf96390 2024-06-10T02:56:20.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:19.998+0000 7fe9ab7fe700 1 -- 172.21.15.52:0/516603537 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 31) ==== 8366+0+0 (secure 0 0 0) 0x7fe9c40a3830 con 0x5607faf96390 2024-06-10T02:56:20.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:19.998+0000 7fe9dced0b80 10 client.15816 fetch_fsmap finished waiting for FSMap version 31 2024-06-10T02:56:20.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:19.998+0000 7fe9dced0b80 10 client.15816 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-06-10T02:56:20.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:19.998+0000 7fe9dced0b80 20 client.15816 populate_metadata read hostname 'smithi052' 2024-06-10T02:56:20.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:19.998+0000 7fe9dced0b80 1 --2- 172.21.15.52:0/516603537 >> [v2:172.21.15.188:6834/3904544259,v1:172.21.15.188:6835/3904544259] conn(0x5607fafa9d60 0x5607fafac150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:56:20.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:19.998+0000 7fe9dced0b80 4 client.15816 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T02:56:20.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:19.998+0000 7fe9dced0b80 1 -- 172.21.15.52:0/516603537 --> [v2:172.21.15.188:6834/3904544259,v1:172.21.15.188:6835/3904544259] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5607faee5290 con 0x5607fafa9d60 2024-06-10T02:56:20.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:19.998+0000 7fe9cbfff700 1 --2- 172.21.15.52:0/516603537 >> [v2:172.21.15.188:6834/3904544259,v1:172.21.15.188:6835/3904544259] conn(0x5607fafa9d60 0x5607fafac150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:56:20.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:19.999+0000 7fe9ab7fe700 10 client.15816 ms_handle_connect on v2:172.21.15.188:6834/3904544259 2024-06-10T02:56:20.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:19.999+0000 7fe9cbfff700 1 --2- 172.21.15.52:0/516603537 >> [v2:172.21.15.188:6834/3904544259,v1:172.21.15.188:6835/3904544259] conn(0x5607fafa9d60 0x5607fafac150 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=99c66bde7c907318 server_cookie=8cf06b8f448ba072 in_seq=0 out_seq=0 2024-06-10T02:56:20.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:19.999+0000 7fe9ab7fe700 1 -- 172.21.15.52:0/516603537 <== mds.2 v2:172.21.15.188:6834/3904544259 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5607faee5290 con 0x5607fafa9d60 2024-06-10T02:56:20.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:19.999+0000 7fe9ab7fe700 10 client.15816 handle_command_reply: tid=0 2024-06-10T02:56:20.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:19.999+0000 7fe9dced0b80 1 client.15816 shutdown 2024-06-10T02:56:20.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:20.000+0000 7fe9dced0b80 1 -- 172.21.15.52:0/516603537 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5607faf96390 msgr2=0x5607faf99980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:56:20.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:20.000+0000 7fe9dced0b80 1 --2- 172.21.15.52:0/516603537 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5607faf96390 0x5607faf99980 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fe9c40974b0 tx=0x7fe9c409aa80 comp rx=0 tx=0).stop 2024-06-10T02:56:20.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:20.000+0000 7fe9dced0b80 1 -- 172.21.15.52:0/516603537 shutdown_connections 2024-06-10T02:56:20.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:20.000+0000 7fe9dced0b80 1 --2- 172.21.15.52:0/516603537 >> [v2:172.21.15.188:6834/3904544259,v1:172.21.15.188:6835/3904544259] conn(0x5607fafa9d60 0x5607fafac150 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:20.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:20.000+0000 7fe9ab7fe700 0 client.15816 ms_handle_reset on v2:172.21.15.188:6834/3904544259 2024-06-10T02:56:20.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:20.000+0000 7fe9dced0b80 1 --2- 172.21.15.52:0/516603537 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5607faf96390 0x5607faf99980 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:20.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:20.000+0000 7fe9dced0b80 1 --2- 172.21.15.52:0/516603537 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5607faf90330 0x5607faf9c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:20.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:20.000+0000 7fe9dced0b80 1 --2- 172.21.15.52:0/516603537 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5607faf95a70 0x5607faf99440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:20.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:20.000+0000 7fe9dced0b80 1 -- 172.21.15.52:0/516603537 >> 172.21.15.52:0/516603537 conn(0x5607faeb6cc0 msgr2=0x5607faf8a9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:56:20.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:20.000+0000 7fe9dced0b80 1 -- 172.21.15.52:0/516603537 shutdown_connections 2024-06-10T02:56:20.006 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:20.000+0000 7fe9dced0b80 1 -- 172.21.15.52:0/516603537 wait complete. 2024-06-10T02:56:20.006 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:20.000+0000 7fe9dced0b80 20 client.15816 trim_cache size 0 max 16384 2024-06-10T02:56:20.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:20.016+0000 7fe9dced0b80 1 Processor -- start 2024-06-10T02:56:20.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:20.016+0000 7fe9dced0b80 1 -- start start 2024-06-10T02:56:20.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:20.016+0000 7fe9dced0b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5607faf90330 0x5607faf8a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:56:20.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:20.016+0000 7fe9dced0b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5607faf95a70 0x5607faf8af10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:56:20.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:20.016+0000 7fe9dced0b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5607faf96390 0x5607faf7fe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:56:20.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:20.016+0000 7fe9dced0b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fe9cc1ac100 con 0x5607faf96390 2024-06-10T02:56:20.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:20.016+0000 7fe9dced0b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe9cc1adc70 con 0x5607faf95a70 2024-06-10T02:56:20.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:20.016+0000 7fe9dced0b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fe9cc1ac7d0 con 0x5607faf90330 2024-06-10T02:56:20.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:20.016+0000 7fe9d0bea700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5607faf96390 0x5607faf7fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:56:20.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:20.016+0000 7fe9d0bea700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5607faf96390 0x5607faf7fe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:46704/0 (socket says 172.21.15.52:46704) 2024-06-10T02:56:20.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:20.016+0000 7fe9d0bea700 1 -- 172.21.15.52:0/3721034077 learned_addr learned my addr 172.21.15.52:0/3721034077 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:56:20.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:20.016+0000 7fe9cb7fe700 1 --2- 172.21.15.52:0/3721034077 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5607faf95a70 0x5607faf8af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:56:20.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:20.016+0000 7fe9cbfff700 1 --2- 172.21.15.52:0/3721034077 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5607faf90330 0x5607faf8a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:56:20.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:20.016+0000 7fe9d0bea700 1 -- 172.21.15.52:0/3721034077 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5607faf90330 msgr2=0x5607faf8a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:56:20.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:20.016+0000 7fe9d0bea700 1 --2- 172.21.15.52:0/3721034077 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5607faf90330 0x5607faf8a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:20.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:20.016+0000 7fe9d0bea700 1 -- 172.21.15.52:0/3721034077 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5607faf95a70 msgr2=0x5607faf8af10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:56:20.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:20.016+0000 7fe9d0bea700 1 --2- 172.21.15.52:0/3721034077 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5607faf95a70 0x5607faf8af10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:20.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:20.016+0000 7fe9d0bea700 1 -- 172.21.15.52:0/3721034077 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5607faf99a70 con 0x5607faf96390 2024-06-10T02:56:20.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:20.017+0000 7fe9cb7fe700 1 --2- 172.21.15.52:0/3721034077 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5607faf95a70 0x5607faf8af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T02:56:20.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:20.017+0000 7fe9d0bea700 1 --2- 172.21.15.52:0/3721034077 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5607faf96390 0x5607faf7fe00 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fe9c4099300 tx=0x7fe9c4095e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b185135e697d1b5 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:56:20.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:20.017+0000 7fe9b1ffb700 1 -- 172.21.15.52:0/3721034077 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9c4065070 con 0x5607faf96390 2024-06-10T02:56:20.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:20.017+0000 7fe9b1ffb700 1 -- 172.21.15.52:0/3721034077 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe9c409f400 con 0x5607faf96390 2024-06-10T02:56:20.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:20.017+0000 7fe9b1ffb700 1 -- 172.21.15.52:0/3721034077 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9c409f1f0 con 0x5607faf96390 2024-06-10T02:56:20.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:20.017+0000 7fe9dced0b80 1 -- 172.21.15.52:0/3721034077 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5607faf96390 msgr2=0x5607faf7fe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:56:20.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:20.018+0000 7fe9dced0b80 1 --2- 172.21.15.52:0/3721034077 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5607faf96390 0x5607faf7fe00 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fe9c4099300 tx=0x7fe9c4095e00 comp rx=0 tx=0).stop 2024-06-10T02:56:20.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:20.018+0000 7fe9dced0b80 1 -- 172.21.15.52:0/3721034077 shutdown_connections 2024-06-10T02:56:20.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:20.018+0000 7fe9dced0b80 1 --2- 172.21.15.52:0/3721034077 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5607faf96390 0x5607faf7fe00 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:20.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:20.018+0000 7fe9dced0b80 1 --2- 172.21.15.52:0/3721034077 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5607faf90330 0x5607faf8a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:20.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:20.018+0000 7fe9dced0b80 1 --2- 172.21.15.52:0/3721034077 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5607faf95a70 0x5607faf8af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:20.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:20.018+0000 7fe9dced0b80 1 -- 172.21.15.52:0/3721034077 >> 172.21.15.52:0/3721034077 conn(0x5607faeb6cc0 msgr2=0x5607faf97d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:56:20.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:20.018+0000 7fe9dced0b80 1 -- 172.21.15.52:0/3721034077 shutdown_connections 2024-06-10T02:56:20.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:20.018+0000 7fe9dced0b80 1 -- 172.21.15.52:0/3721034077 wait complete. 2024-06-10T02:56:20.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:20.018+0000 7fe9dced0b80 1 -- start start 2024-06-10T02:56:20.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:20.018+0000 7fe9dced0b80 1 Processor -- start 2024-06-10T02:56:20.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:20.019+0000 7fe9dced0b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5607faf90330 0x5607fafa2850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:56:20.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:20.019+0000 7fe9dced0b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5607faf95a70 0x5607fafb1130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:56:20.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:20.019+0000 7fe9dced0b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5607faf96390 0x5607fafb1670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:56:20.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:20.019+0000 7fe9dced0b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fe9cc1ad570 con 0x5607faf90330 2024-06-10T02:56:20.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:20.019+0000 7fe9dced0b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe9cc1acea0 con 0x5607faf95a70 2024-06-10T02:56:20.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:20.019+0000 7fe9dced0b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fe9cc1aa0a0 con 0x5607faf96390 2024-06-10T02:56:20.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:20.019+0000 7fe9dced0b80 10 client.0 _ll_register_callbacks cb 0x5607fae77920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T02:56:20.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:20.019+0000 7fe9cbfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5607faf90330 0x5607fafa2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:56:20.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:20.019+0000 7fe9cbfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5607faf90330 0x5607fafa2850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:46714/0 (socket says 172.21.15.52:46714) 2024-06-10T02:56:20.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:20.019+0000 7fe9cbfff700 1 -- 172.21.15.52:0/271923218 learned_addr learned my addr 172.21.15.52:0/271923218 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:56:20.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:20.019+0000 7fe9cb7fe700 1 --2- 172.21.15.52:0/271923218 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5607faf95a70 0x5607fafb1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:56:20.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:20.019+0000 7fe9d0bea700 1 --2- 172.21.15.52:0/271923218 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5607faf96390 0x5607fafb1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:56:20.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:20.019+0000 7fe9cbfff700 1 -- 172.21.15.52:0/271923218 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5607faf96390 msgr2=0x5607fafb1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:56:20.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:20.019+0000 7fe9cbfff700 1 --2- 172.21.15.52:0/271923218 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5607faf96390 0x5607fafb1670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:20.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:20.019+0000 7fe9cbfff700 1 -- 172.21.15.52:0/271923218 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5607faf95a70 msgr2=0x5607fafb1130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:56:20.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:20.019+0000 7fe9cbfff700 1 --2- 172.21.15.52:0/271923218 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5607faf95a70 0x5607fafb1130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:20.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:20.019+0000 7fe9cbfff700 1 -- 172.21.15.52:0/271923218 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5607fafb1ea0 con 0x5607faf90330 2024-06-10T02:56:20.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:20.019+0000 7fe9cbfff700 1 --2- 172.21.15.52:0/271923218 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5607faf90330 0x5607fafa2850 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fe9bc00b990 tx=0x7fe9bc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbda60458c673910 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:56:20.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:20.019+0000 7fe9ab7fe700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-10T02:56:20.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:20.019+0000 7fe9ab7fe700 1 -- 172.21.15.52:0/271923218 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9bc037060 con 0x5607faf90330 2024-06-10T02:56:20.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:20.019+0000 7fe9ab7fe700 1 -- 172.21.15.52:0/271923218 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7fe9bc011d90 con 0x5607faf90330 2024-06-10T02:56:20.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:20.019+0000 7fe9dced0b80 1 -- 172.21.15.52:0/271923218 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe9c409f400 con 0x5607faf90330 2024-06-10T02:56:20.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:20.020+0000 7fe9ab7fe700 1 -- 172.21.15.52:0/271923218 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fe9bc03e050 con 0x5607faf90330 2024-06-10T02:56:20.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:20.020+0000 7fe9dced0b80 10 client.15822 fetch_fsmap learned FSMap version 31 2024-06-10T02:56:20.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:20.020+0000 7fe9dced0b80 1 -- 172.21.15.52:0/271923218 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5607fb014840 con 0x5607faf90330 2024-06-10T02:56:20.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:20.020+0000 7fe9ab7fe700 1 -- 172.21.15.52:0/271923218 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe9c409f400 con 0x5607faf90330 2024-06-10T02:56:20.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:20.020+0000 7fe9ab7fe700 1 -- 172.21.15.52:0/271923218 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 31) ==== 8366+0+0 (secure 0 0 0) 0x7fe9bc01f760 con 0x5607faf90330 2024-06-10T02:56:20.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:20.020+0000 7fe9dced0b80 10 client.15822 fetch_fsmap finished waiting for FSMap version 31 2024-06-10T02:56:20.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:20.020+0000 7fe9dced0b80 10 client.15822 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-06-10T02:56:20.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:20.020+0000 7fe9dced0b80 20 client.15822 populate_metadata read hostname 'smithi052' 2024-06-10T02:56:20.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:20.020+0000 7fe9dced0b80 1 --2- 172.21.15.52:0/271923218 >> [v2:172.21.15.188:6834/3904544259,v1:172.21.15.188:6835/3904544259] conn(0x5607fafa9d60 0x5607fb016fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:56:20.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:20.020+0000 7fe9dced0b80 4 client.15822 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T02:56:20.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:20.020+0000 7fe9dced0b80 1 -- 172.21.15.52:0/271923218 --> [v2:172.21.15.188:6834/3904544259,v1:172.21.15.188:6835/3904544259] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5607fad5f250 con 0x5607fafa9d60 2024-06-10T02:56:20.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:20.020+0000 7fe9cb7fe700 1 --2- 172.21.15.52:0/271923218 >> [v2:172.21.15.188:6834/3904544259,v1:172.21.15.188:6835/3904544259] conn(0x5607fafa9d60 0x5607fb016fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:56:20.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:20.021+0000 7fe9cb7fe700 1 --2- 172.21.15.52:0/271923218 >> [v2:172.21.15.188:6834/3904544259,v1:172.21.15.188:6835/3904544259] conn(0x5607fafa9d60 0x5607fb016fa0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e1ddea08d8ebb9c6 server_cookie=4049abe6c843ecab in_seq=0 out_seq=0 2024-06-10T02:56:20.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:20.021+0000 7fe9ab7fe700 10 client.15822 ms_handle_connect on v2:172.21.15.188:6834/3904544259 2024-06-10T02:56:20.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:20.021+0000 7fe9ab7fe700 1 -- 172.21.15.52:0/271923218 <== mds.2 v2:172.21.15.188:6834/3904544259 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5607fad5f250 con 0x5607fafa9d60 2024-06-10T02:56:20.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:20.021+0000 7fe9ab7fe700 10 client.15822 handle_command_reply: tid=0 2024-06-10T02:56:20.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:20.021+0000 7fe9dced0b80 1 client.15822 shutdown 2024-06-10T02:56:20.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:20.021+0000 7fe9dced0b80 1 -- 172.21.15.52:0/271923218 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5607faf90330 msgr2=0x5607fafa2850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:56:20.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:20.021+0000 7fe9dced0b80 1 --2- 172.21.15.52:0/271923218 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5607faf90330 0x5607fafa2850 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fe9bc00b990 tx=0x7fe9bc004750 comp rx=0 tx=0).stop 2024-06-10T02:56:20.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:20.021+0000 7fe9dced0b80 1 -- 172.21.15.52:0/271923218 shutdown_connections 2024-06-10T02:56:20.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:20.021+0000 7fe9dced0b80 1 --2- 172.21.15.52:0/271923218 >> [v2:172.21.15.188:6834/3904544259,v1:172.21.15.188:6835/3904544259] conn(0x5607fafa9d60 0x5607fb016fa0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:20.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:20.021+0000 7fe9dced0b80 1 --2- 172.21.15.52:0/271923218 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5607faf96390 0x5607fafb1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:20.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:20.021+0000 7fe9dced0b80 1 --2- 172.21.15.52:0/271923218 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5607faf90330 0x5607fafa2850 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:20.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:20.021+0000 7fe9dced0b80 1 --2- 172.21.15.52:0/271923218 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5607faf95a70 0x5607fafb1130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:20.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:20.021+0000 7fe9dced0b80 1 -- 172.21.15.52:0/271923218 >> 172.21.15.52:0/271923218 conn(0x5607faeb6cc0 msgr2=0x5607faf8a9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:56:20.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:20.021+0000 7fe9ab7fe700 0 client.15822 ms_handle_reset on v2:172.21.15.188:6834/3904544259 2024-06-10T02:56:20.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:20.021+0000 7fe9dced0b80 1 -- 172.21.15.52:0/271923218 shutdown_connections 2024-06-10T02:56:20.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:20.021+0000 7fe9dced0b80 1 -- 172.21.15.52:0/271923218 wait complete. 2024-06-10T02:56:20.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:20.021+0000 7fe9dced0b80 20 client.15822 trim_cache size 0 max 16384 2024-06-10T02:56:20.027 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-06-10T02:56:20.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:20.022+0000 7fe9d264d700 1 -- 172.21.15.52:0/3000091199 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fe9b40a2c20 msgr2=0x7fe9b40a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:56:20.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:20.022+0000 7fe9d264d700 1 --2- 172.21.15.52:0/3000091199 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fe9b40a2c20 0x7fe9b40a50e0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fe9cc199bb0 tx=0x7fe9bc00b5b0 comp rx=0 tx=0).stop 2024-06-10T02:56:20.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:20.022+0000 7fe9d264d700 1 -- 172.21.15.52:0/3000091199 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe9cc108580 msgr2=0x7fe9cc198a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:56:20.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:20.022+0000 7fe9d264d700 1 --2- 172.21.15.52:0/3000091199 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe9cc108580 0x7fe9cc198a70 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fe9c4007ee0 tx=0x7fe9c400d3b0 comp rx=0 tx=0).stop 2024-06-10T02:56:20.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:20.022+0000 7fe9d264d700 1 -- 172.21.15.52:0/3000091199 shutdown_connections 2024-06-10T02:56:20.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:20.022+0000 7fe9d264d700 1 --2- 172.21.15.52:0/3000091199 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fe9b40a2c20 0x7fe9b40a50e0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:20.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:20.022+0000 7fe9d264d700 1 --2- 172.21.15.52:0/3000091199 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe9cc108580 0x7fe9cc198a70 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:20.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:20.022+0000 7fe9d264d700 1 --2- 172.21.15.52:0/3000091199 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe9cc06ac60 0x7fe9cc19e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:20.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:20.022+0000 7fe9d264d700 1 --2- 172.21.15.52:0/3000091199 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe9cc06b620 0x7fe9cc19edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:20.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:20.022+0000 7fe9d264d700 1 -- 172.21.15.52:0/3000091199 >> 172.21.15.52:0/3000091199 conn(0x7fe9cc078430 msgr2=0x7fe9cc0fff20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:56:20.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:20.022+0000 7fe9d264d700 1 -- 172.21.15.52:0/3000091199 shutdown_connections 2024-06-10T02:56:20.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:20.022+0000 7fe9d264d700 1 -- 172.21.15.52:0/3000091199 wait complete. 2024-06-10T02:56:20.726 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:56:20 smithi052 ceph-mon[32362]: pgmap v498: 129 pgs: 129 active+clean; 3.5 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 4.7 KiB/s wr, 11 op/s 2024-06-10T02:56:20.888 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:56:20 smithi120 ceph-mon[36634]: pgmap v498: 129 pgs: 129 active+clean; 3.5 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 4.7 KiB/s wr, 11 op/s 2024-06-10T02:56:20.899 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-06-10T02:56:20.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:56:20 smithi188 ceph-mon[36599]: pgmap v498: 129 pgs: 129 active+clean; 3.5 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 4.7 KiB/s wr, 11 op/s 2024-06-10T02:56:21.160 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:56:21.384 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T02:56:22.682 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:56:22 smithi052 ceph-mon[32362]: pgmap v499: 129 pgs: 129 active+clean; 3.5 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 2.7 KiB/s wr, 10 op/s 2024-06-10T02:56:22.888 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:56:22 smithi120 ceph-mon[36634]: pgmap v499: 129 pgs: 129 active+clean; 3.5 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 2.7 KiB/s wr, 10 op/s 2024-06-10T02:56:22.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.911+0000 7f07565f9700 1 -- 172.21.15.52:0/1745347823 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f07501019d0 msgr2=0x7f0750101db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:56:22.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.911+0000 7f07565f9700 1 --2- 172.21.15.52:0/1745347823 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f07501019d0 0x7f0750101db0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f07400099c0 tx=0x7f0740009cd0 comp rx=0 tx=0).stop 2024-06-10T02:56:22.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.911+0000 7f07565f9700 1 -- 172.21.15.52:0/1745347823 shutdown_connections 2024-06-10T02:56:22.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.911+0000 7f07565f9700 1 --2- 172.21.15.52:0/1745347823 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0750106720 0x7f075010e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:22.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.911+0000 7f07565f9700 1 --2- 172.21.15.52:0/1745347823 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f07501019d0 0x7f0750101db0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:22.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.911+0000 7f07565f9700 1 --2- 172.21.15.52:0/1745347823 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0750102380 0x7f0750105ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:22.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.911+0000 7f07565f9700 1 -- 172.21.15.52:0/1745347823 >> 172.21.15.52:0/1745347823 conn(0x7f0750078340 msgr2=0x7f0750078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:56:22.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.911+0000 7f07565f9700 1 -- 172.21.15.52:0/1745347823 shutdown_connections 2024-06-10T02:56:22.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.911+0000 7f07565f9700 1 -- 172.21.15.52:0/1745347823 wait complete. 2024-06-10T02:56:22.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.911+0000 7f07565f9700 1 Processor -- start 2024-06-10T02:56:22.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.911+0000 7f07565f9700 1 -- start start 2024-06-10T02:56:22.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.912+0000 7f07565f9700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f07501019d0 0x7f0750199a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:56:22.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.912+0000 7f07565f9700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0750102380 0x7f0750199fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:56:22.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.912+0000 7f07565f9700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0750106720 0x7f07501a4af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:56:22.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.912+0000 7f07565f9700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f07501116d0 con 0x7f07501019d0 2024-06-10T02:56:22.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.912+0000 7f07565f9700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f0750111550 con 0x7f0750102380 2024-06-10T02:56:22.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.912+0000 7f07565f9700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f0750111850 con 0x7f0750106720 2024-06-10T02:56:22.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.912+0000 7f074ffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f07501019d0 0x7f0750199a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:56:22.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.912+0000 7f074ffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f07501019d0 0x7f0750199a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40246/0 (socket says 172.21.15.52:40246) 2024-06-10T02:56:22.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.912+0000 7f074ffff700 1 -- 172.21.15.52:0/1254692243 learned_addr learned my addr 172.21.15.52:0/1254692243 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:56:22.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.912+0000 7f074f7fe700 1 --2- 172.21.15.52:0/1254692243 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0750102380 0x7f0750199fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:56:22.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.912+0000 7f0754b96700 1 --2- 172.21.15.52:0/1254692243 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0750106720 0x7f07501a4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:56:22.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.912+0000 7f074ffff700 1 -- 172.21.15.52:0/1254692243 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0750106720 msgr2=0x7f07501a4af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:56:22.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.912+0000 7f074ffff700 1 --2- 172.21.15.52:0/1254692243 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0750106720 0x7f07501a4af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:22.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.912+0000 7f074ffff700 1 -- 172.21.15.52:0/1254692243 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0750102380 msgr2=0x7f0750199fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:56:22.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.912+0000 7f074ffff700 1 --2- 172.21.15.52:0/1254692243 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0750102380 0x7f0750199fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:22.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.912+0000 7f074ffff700 1 -- 172.21.15.52:0/1254692243 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07501a5030 con 0x7f07501019d0 2024-06-10T02:56:22.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.912+0000 7f0754b96700 1 --2- 172.21.15.52:0/1254692243 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0750106720 0x7f07501a4af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T02:56:22.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.912+0000 7f074ffff700 1 --2- 172.21.15.52:0/1254692243 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f07501019d0 0x7f0750199a70 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f07400059d0 tx=0x7f07400038b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:56:22.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.912+0000 7f074d7fa700 1 -- 172.21.15.52:0/1254692243 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0740027070 con 0x7f07501019d0 2024-06-10T02:56:22.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.913+0000 7f07565f9700 1 -- 172.21.15.52:0/1254692243 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07501a5200 con 0x7f07501019d0 2024-06-10T02:56:22.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.913+0000 7f074d7fa700 1 -- 172.21.15.52:0/1254692243 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0740004050 con 0x7f07501019d0 2024-06-10T02:56:22.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.913+0000 7f074d7fa700 1 -- 172.21.15.52:0/1254692243 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0740022db0 con 0x7f07501019d0 2024-06-10T02:56:22.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.913+0000 7f07565f9700 1 -- 172.21.15.52:0/1254692243 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07501a56b0 con 0x7f07501019d0 2024-06-10T02:56:22.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.916+0000 7f074d7fa700 1 -- 172.21.15.52:0/1254692243 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f074002a070 con 0x7f07501019d0 2024-06-10T02:56:22.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.917+0000 7f074d7fa700 1 --2- 172.21.15.52:0/1254692243 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f07380a2c20 0x7f07380a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:56:22.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.917+0000 7f0760e7cb80 1 Processor -- start 2024-06-10T02:56:22.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.917+0000 7f0760e7cb80 1 -- start start 2024-06-10T02:56:22.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.917+0000 7f074f7fe700 1 --2- 172.21.15.52:0/1254692243 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f07380a2c20 0x7f07380a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:56:22.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.917+0000 7f074d7fa700 1 -- 172.21.15.52:0/1254692243 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f07400d01b0 con 0x7f07501019d0 2024-06-10T02:56:22.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.917+0000 7f0760e7cb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c793edfa70 0x55c793edfe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:56:22.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.917+0000 7f0760e7cb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c793ee0390 0x55c793ed9df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:56:22.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.917+0000 7f0760e7cb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c793eda330 0x55c793eda750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:56:22.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.917+0000 7f0760e7cb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55c793e27810 con 0x55c793edfa70 2024-06-10T02:56:22.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.917+0000 7f0760e7cb80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55c793dcf720 con 0x55c793eda330 2024-06-10T02:56:22.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.917+0000 7f0760e7cb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55c793e237d0 con 0x55c793ee0390 2024-06-10T02:56:22.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.917+0000 7f074ffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c793edfa70 0x55c793edfe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:56:22.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.917+0000 7f074ffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c793edfa70 0x55c793edfe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40262/0 (socket says 172.21.15.52:40262) 2024-06-10T02:56:22.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.917+0000 7f074ffff700 1 -- 172.21.15.52:0/251732476 learned_addr learned my addr 172.21.15.52:0/251732476 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:56:22.923 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.917+0000 7f0754b96700 1 --2- 172.21.15.52:0/251732476 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c793eda330 0x55c793eda750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:56:22.923 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.917+0000 7f074f7fe700 1 --2- 172.21.15.52:0/1254692243 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f07380a2c20 0x7f07380a50e0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f0750106030 tx=0x7f07440073d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:56:22.923 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.917+0000 7f074f7fe700 1 --2- 172.21.15.52:0/251732476 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c793ee0390 0x55c793ed9df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:56:22.923 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.917+0000 7f074ffff700 1 -- 172.21.15.52:0/251732476 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c793ee0390 msgr2=0x55c793ed9df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:56:22.923 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.917+0000 7f074ffff700 1 --2- 172.21.15.52:0/251732476 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c793ee0390 0x55c793ed9df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:22.923 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.917+0000 7f074ffff700 1 -- 172.21.15.52:0/251732476 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c793eda330 msgr2=0x55c793eda750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:56:22.923 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.917+0000 7f074ffff700 1 --2- 172.21.15.52:0/251732476 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c793eda330 0x55c793eda750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:22.923 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.917+0000 7f074ffff700 1 -- 172.21.15.52:0/251732476 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c793dc5f50 con 0x55c793edfa70 2024-06-10T02:56:22.923 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.917+0000 7f074f7fe700 1 --2- 172.21.15.52:0/251732476 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c793ee0390 0x55c793ed9df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T02:56:22.923 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.918+0000 7f074ffff700 1 --2- 172.21.15.52:0/251732476 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c793edfa70 0x55c793edfe50 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f07400a4d40 tx=0x7f0740009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0e7c52cdce81ab9 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:56:22.923 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.918+0000 7f07367fc700 1 -- 172.21.15.52:0/251732476 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07400870e0 con 0x55c793edfa70 2024-06-10T02:56:22.923 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.918+0000 7f07367fc700 1 -- 172.21.15.52:0/251732476 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f074009c690 con 0x55c793edfa70 2024-06-10T02:56:22.923 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.918+0000 7f07367fc700 1 -- 172.21.15.52:0/251732476 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f074009caf0 con 0x55c793edfa70 2024-06-10T02:56:22.923 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.918+0000 7f0760e7cb80 1 -- 172.21.15.52:0/251732476 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c793edfa70 msgr2=0x55c793edfe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:56:22.924 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.918+0000 7f0760e7cb80 1 --2- 172.21.15.52:0/251732476 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c793edfa70 0x55c793edfe50 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f07400a4d40 tx=0x7f0740009cd0 comp rx=0 tx=0).stop 2024-06-10T02:56:22.924 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.919+0000 7f0760e7cb80 1 -- 172.21.15.52:0/251732476 shutdown_connections 2024-06-10T02:56:22.924 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.919+0000 7f0760e7cb80 1 --2- 172.21.15.52:0/251732476 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c793eda330 0x55c793eda750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:22.924 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.919+0000 7f0760e7cb80 1 --2- 172.21.15.52:0/251732476 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c793edfa70 0x55c793edfe50 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:22.924 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.919+0000 7f0760e7cb80 1 --2- 172.21.15.52:0/251732476 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c793ee0390 0x55c793ed9df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:22.924 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.919+0000 7f0760e7cb80 1 -- 172.21.15.52:0/251732476 >> 172.21.15.52:0/251732476 conn(0x55c793e00cc0 msgr2=0x55c793ed0850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:56:22.924 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.919+0000 7f0760e7cb80 1 -- 172.21.15.52:0/251732476 shutdown_connections 2024-06-10T02:56:22.924 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.919+0000 7f0760e7cb80 1 -- 172.21.15.52:0/251732476 wait complete. 2024-06-10T02:56:22.924 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.919+0000 7f0760e7cb80 1 -- start start 2024-06-10T02:56:22.924 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.919+0000 7f0760e7cb80 1 Processor -- start 2024-06-10T02:56:22.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.920+0000 7f0760e7cb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c793eda330 0x55c793ee6160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:56:22.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.920+0000 7f0760e7cb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c793edfa70 0x55c793ee3460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:56:22.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.920+0000 7f0760e7cb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c793ee0390 0x55c793ee39a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:56:22.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.920+0000 7f0760e7cb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55c793ee6af0 con 0x55c793edfa70 2024-06-10T02:56:22.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.920+0000 7f0760e7cb80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55c793dc7bf0 con 0x55c793eda330 2024-06-10T02:56:22.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.920+0000 7f0760e7cb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55c793ee6c70 con 0x55c793ee0390 2024-06-10T02:56:22.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.920+0000 7f074f7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c793edfa70 0x55c793ee3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:56:22.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.920+0000 7f074f7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c793edfa70 0x55c793ee3460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40276/0 (socket says 172.21.15.52:40276) 2024-06-10T02:56:22.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.920+0000 7f074ffff700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c793eda330 0x55c793ee6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:56:22.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.920+0000 7f074f7fe700 1 -- 172.21.15.52:0/1792044128 learned_addr learned my addr 172.21.15.52:0/1792044128 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:56:22.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.920+0000 7f0754b96700 1 --2- 172.21.15.52:0/1792044128 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c793ee0390 0x55c793ee39a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:56:22.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.920+0000 7f0760e7cb80 10 client.0 _ll_register_callbacks cb 0x55c793dc1920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T02:56:22.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.920+0000 7f074ffff700 1 -- 172.21.15.52:0/1792044128 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c793ee0390 msgr2=0x55c793ee39a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:56:22.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.920+0000 7f074ffff700 1 --2- 172.21.15.52:0/1792044128 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c793ee0390 0x55c793ee39a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:22.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.920+0000 7f074ffff700 1 -- 172.21.15.52:0/1792044128 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c793edfa70 msgr2=0x55c793ee3460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:56:22.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.920+0000 7f074ffff700 1 --2- 172.21.15.52:0/1792044128 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c793edfa70 0x55c793ee3460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:22.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.920+0000 7f074ffff700 1 -- 172.21.15.52:0/1792044128 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c793d768d0 con 0x55c793eda330 2024-06-10T02:56:22.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.920+0000 7f074f7fe700 1 --2- 172.21.15.52:0/1792044128 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c793edfa70 0x55c793ee3460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T02:56:22.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.921+0000 7f074ffff700 1 --2- 172.21.15.52:0/1792044128 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c793eda330 0x55c793ee6160 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f0740009990 tx=0x7f07400a0ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c6643d5e24643329 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:56:22.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.921+0000 7f072f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-06-10T02:56:22.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.921+0000 7f072f7fe700 1 -- 172.21.15.52:0/1792044128 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07400ba060 con 0x55c793eda330 2024-06-10T02:56:22.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.921+0000 7f072f7fe700 1 -- 172.21.15.52:0/1792044128 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f07400a1dd0 con 0x55c793eda330 2024-06-10T02:56:22.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.921+0000 7f0760e7cb80 1 -- 172.21.15.52:0/1792044128 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f074009c690 con 0x55c793eda330 2024-06-10T02:56:22.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.921+0000 7f072f7fe700 1 -- 172.21.15.52:0/1792044128 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f0740082050 con 0x55c793eda330 2024-06-10T02:56:22.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.921+0000 7f0760e7cb80 10 client.25427 fetch_fsmap learned FSMap version 31 2024-06-10T02:56:22.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.921+0000 7f0760e7cb80 1 -- 172.21.15.52:0/1792044128 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55c793ee6af0 con 0x55c793eda330 2024-06-10T02:56:22.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.921+0000 7f072f7fe700 1 -- 172.21.15.52:0/1792044128 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f074009c690 con 0x55c793eda330 2024-06-10T02:56:22.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.922+0000 7f072f7fe700 1 -- 172.21.15.52:0/1792044128 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 31) ==== 8366+0+0 (secure 0 0 0) 0x7f074009bae0 con 0x55c793eda330 2024-06-10T02:56:22.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.922+0000 7f0760e7cb80 10 client.25427 fetch_fsmap finished waiting for FSMap version 31 2024-06-10T02:56:22.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.922+0000 7f0760e7cb80 10 client.25427 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-06-10T02:56:22.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.922+0000 7f0760e7cb80 20 client.25427 populate_metadata read hostname 'smithi052' 2024-06-10T02:56:22.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.922+0000 7f0760e7cb80 1 --2- 172.21.15.52:0/1792044128 >> [v2:172.21.15.188:6836/819317726,v1:172.21.15.188:6837/819317726] conn(0x55c793ef36e0 0x55c793ef5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:56:22.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.922+0000 7f0760e7cb80 4 client.25427 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T02:56:22.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.922+0000 7f0760e7cb80 1 -- 172.21.15.52:0/1792044128 --> [v2:172.21.15.188:6836/819317726,v1:172.21.15.188:6837/819317726] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c793e34d10 con 0x55c793ef36e0 2024-06-10T02:56:22.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.922+0000 7f074f7fe700 1 --2- 172.21.15.52:0/1792044128 >> [v2:172.21.15.188:6836/819317726,v1:172.21.15.188:6837/819317726] conn(0x55c793ef36e0 0x55c793ef5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:56:22.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.923+0000 7f074f7fe700 1 --2- 172.21.15.52:0/1792044128 >> [v2:172.21.15.188:6836/819317726,v1:172.21.15.188:6837/819317726] conn(0x55c793ef36e0 0x55c793ef5ad0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=90ed227c3d340db1 server_cookie=9f5a4438c103527a in_seq=0 out_seq=0 2024-06-10T02:56:22.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.923+0000 7f072f7fe700 10 client.25427 ms_handle_connect on v2:172.21.15.188:6836/819317726 2024-06-10T02:56:22.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.923+0000 7f072f7fe700 1 -- 172.21.15.52:0/1792044128 <== mds.1 v2:172.21.15.188:6836/819317726 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55c793e34d10 con 0x55c793ef36e0 2024-06-10T02:56:22.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.923+0000 7f072f7fe700 10 client.25427 handle_command_reply: tid=0 2024-06-10T02:56:22.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.923+0000 7f0760e7cb80 1 client.25427 shutdown 2024-06-10T02:56:22.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.924+0000 7f0760e7cb80 1 -- 172.21.15.52:0/1792044128 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c793eda330 msgr2=0x55c793ee6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:56:22.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.924+0000 7f0760e7cb80 1 --2- 172.21.15.52:0/1792044128 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c793eda330 0x55c793ee6160 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f0740009990 tx=0x7f07400a0ab0 comp rx=0 tx=0).stop 2024-06-10T02:56:22.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.924+0000 7f0760e7cb80 1 -- 172.21.15.52:0/1792044128 shutdown_connections 2024-06-10T02:56:22.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.924+0000 7f0760e7cb80 1 --2- 172.21.15.52:0/1792044128 >> [v2:172.21.15.188:6836/819317726,v1:172.21.15.188:6837/819317726] conn(0x55c793ef36e0 0x55c793ef5ad0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:22.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.924+0000 7f0760e7cb80 1 --2- 172.21.15.52:0/1792044128 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c793ee0390 0x55c793ee39a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:22.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.924+0000 7f0760e7cb80 1 --2- 172.21.15.52:0/1792044128 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c793eda330 0x55c793ee6160 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:22.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.924+0000 7f0760e7cb80 1 --2- 172.21.15.52:0/1792044128 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c793edfa70 0x55c793ee3460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:22.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.924+0000 7f072f7fe700 0 client.25427 ms_handle_reset on v2:172.21.15.188:6836/819317726 2024-06-10T02:56:22.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.924+0000 7f0760e7cb80 1 -- 172.21.15.52:0/1792044128 >> 172.21.15.52:0/1792044128 conn(0x55c793e00cc0 msgr2=0x55c793ed49d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:56:22.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.924+0000 7f0760e7cb80 1 -- 172.21.15.52:0/1792044128 shutdown_connections 2024-06-10T02:56:22.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.924+0000 7f0760e7cb80 1 -- 172.21.15.52:0/1792044128 wait complete. 2024-06-10T02:56:22.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.924+0000 7f0760e7cb80 20 client.25427 trim_cache size 0 max 16384 2024-06-10T02:56:22.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.946+0000 7f0760e7cb80 1 Processor -- start 2024-06-10T02:56:22.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.946+0000 7f0760e7cb80 1 -- start start 2024-06-10T02:56:22.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.947+0000 7f0760e7cb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c793eda330 0x55c793eca510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:56:22.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.947+0000 7f0760e7cb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c793edfa70 0x55c793ed49d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:56:22.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.947+0000 7f0760e7cb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c793ee0390 0x55c793ed4f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:56:22.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.947+0000 7f0760e7cb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f07501ac0a0 con 0x55c793eda330 2024-06-10T02:56:22.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.947+0000 7f0760e7cb80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f07501adc10 con 0x55c793ee0390 2024-06-10T02:56:22.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.947+0000 7f0760e7cb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f07501ac770 con 0x55c793edfa70 2024-06-10T02:56:22.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.947+0000 7f074ffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c793eda330 0x55c793eca510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:56:22.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.947+0000 7f074ffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c793eda330 0x55c793eca510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40282/0 (socket says 172.21.15.52:40282) 2024-06-10T02:56:22.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.947+0000 7f074ffff700 1 -- 172.21.15.52:0/2011158049 learned_addr learned my addr 172.21.15.52:0/2011158049 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:56:22.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.947+0000 7f0754b96700 1 --2- 172.21.15.52:0/2011158049 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c793ee0390 0x55c793ed4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:56:22.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.947+0000 7f074f7fe700 1 --2- 172.21.15.52:0/2011158049 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c793edfa70 0x55c793ed49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:56:22.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.947+0000 7f074ffff700 1 -- 172.21.15.52:0/2011158049 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c793edfa70 msgr2=0x55c793ed49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:56:22.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.947+0000 7f074ffff700 1 --2- 172.21.15.52:0/2011158049 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c793edfa70 0x55c793ed49d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:22.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.947+0000 7f074ffff700 1 -- 172.21.15.52:0/2011158049 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c793ee0390 msgr2=0x55c793ed4f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:56:22.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.947+0000 7f074ffff700 1 --2- 172.21.15.52:0/2011158049 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c793ee0390 0x55c793ed4f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:22.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.947+0000 7f074ffff700 1 -- 172.21.15.52:0/2011158049 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c793ee19d0 con 0x55c793eda330 2024-06-10T02:56:22.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.948+0000 7f074ffff700 1 --2- 172.21.15.52:0/2011158049 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c793eda330 0x55c793eca510 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f0740099060 tx=0x7f074009aba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a58d08cf066c936 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:56:22.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.948+0000 7f0735ffb700 1 -- 172.21.15.52:0/2011158049 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07400ba070 con 0x55c793eda330 2024-06-10T02:56:22.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.948+0000 7f0735ffb700 1 -- 172.21.15.52:0/2011158049 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0740003e90 con 0x55c793eda330 2024-06-10T02:56:22.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.948+0000 7f0735ffb700 1 -- 172.21.15.52:0/2011158049 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07400a1bd0 con 0x55c793eda330 2024-06-10T02:56:22.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.949+0000 7f0760e7cb80 1 -- 172.21.15.52:0/2011158049 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c793eda330 msgr2=0x55c793eca510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:56:22.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.949+0000 7f0760e7cb80 1 --2- 172.21.15.52:0/2011158049 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c793eda330 0x55c793eca510 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f0740099060 tx=0x7f074009aba0 comp rx=0 tx=0).stop 2024-06-10T02:56:22.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.949+0000 7f0760e7cb80 1 -- 172.21.15.52:0/2011158049 shutdown_connections 2024-06-10T02:56:22.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.949+0000 7f0760e7cb80 1 --2- 172.21.15.52:0/2011158049 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c793ee0390 0x55c793ed4f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:22.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.949+0000 7f0760e7cb80 1 --2- 172.21.15.52:0/2011158049 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c793eda330 0x55c793eca510 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:22.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.949+0000 7f0760e7cb80 1 --2- 172.21.15.52:0/2011158049 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c793edfa70 0x55c793ed49d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:22.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.949+0000 7f0760e7cb80 1 -- 172.21.15.52:0/2011158049 >> 172.21.15.52:0/2011158049 conn(0x55c793e00cc0 msgr2=0x55c793ec9e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:56:22.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.949+0000 7f0760e7cb80 1 -- 172.21.15.52:0/2011158049 shutdown_connections 2024-06-10T02:56:22.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.949+0000 7f0760e7cb80 1 -- 172.21.15.52:0/2011158049 wait complete. 2024-06-10T02:56:22.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.949+0000 7f0760e7cb80 1 -- start start 2024-06-10T02:56:22.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.950+0000 7f0760e7cb80 1 Processor -- start 2024-06-10T02:56:22.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.950+0000 7f0760e7cb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c793eda330 0x55c793eec0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:56:22.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.950+0000 7f0760e7cb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c793edfa70 0x55c793eec620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:56:22.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.950+0000 7f0760e7cb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c793ee0390 0x55c793efa990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:56:22.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.950+0000 7f0760e7cb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f07501ad510 con 0x55c793eda330 2024-06-10T02:56:22.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.950+0000 7f0760e7cb80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f07501ace40 con 0x55c793ee0390 2024-06-10T02:56:22.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.950+0000 7f0760e7cb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f07501aa040 con 0x55c793edfa70 2024-06-10T02:56:22.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.950+0000 7f0760e7cb80 10 client.0 _ll_register_callbacks cb 0x55c793dc1920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T02:56:22.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.951+0000 7f074f7fe700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c793edfa70 0x55c793eec620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:56:22.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.951+0000 7f074ffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c793eda330 0x55c793eec0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:56:22.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.951+0000 7f074f7fe700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c793edfa70 0x55c793eec620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.52:52782/0 (socket says 172.21.15.52:52782) 2024-06-10T02:56:22.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.951+0000 7f074f7fe700 1 -- 172.21.15.52:0/2289534302 learned_addr learned my addr 172.21.15.52:0/2289534302 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:56:22.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.951+0000 7f0754b96700 1 --2- 172.21.15.52:0/2289534302 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c793ee0390 0x55c793efa990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:56:22.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.951+0000 7f074f7fe700 1 -- 172.21.15.52:0/2289534302 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c793ee0390 msgr2=0x55c793efa990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:56:22.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.951+0000 7f074f7fe700 1 --2- 172.21.15.52:0/2289534302 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c793ee0390 0x55c793efa990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:22.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.951+0000 7f074f7fe700 1 -- 172.21.15.52:0/2289534302 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c793eda330 msgr2=0x55c793eec0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:56:22.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.951+0000 7f074f7fe700 1 --2- 172.21.15.52:0/2289534302 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c793eda330 0x55c793eec0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:22.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.951+0000 7f074f7fe700 1 -- 172.21.15.52:0/2289534302 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c793efb1a0 con 0x55c793edfa70 2024-06-10T02:56:22.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.951+0000 7f0754b96700 1 --2- 172.21.15.52:0/2289534302 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c793ee0390 0x55c793efa990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T02:56:22.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.951+0000 7f074ffff700 1 --2- 172.21.15.52:0/2289534302 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c793eda330 0x55c793eec0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T02:56:22.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.951+0000 7f074f7fe700 1 --2- 172.21.15.52:0/2289534302 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c793edfa70 0x55c793eec620 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f0744007270 tx=0x7f07440047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=175f965f2b94afbc server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:56:22.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.951+0000 7f072f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-06-10T02:56:22.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.952+0000 7f072f7fe700 1 -- 172.21.15.52:0/2289534302 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0744036060 con 0x55c793edfa70 2024-06-10T02:56:22.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.952+0000 7f0760e7cb80 1 -- 172.21.15.52:0/2289534302 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55c793e2f290 con 0x55c793edfa70 2024-06-10T02:56:22.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.952+0000 7f072f7fe700 1 -- 172.21.15.52:0/2289534302 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f0744011dc0 con 0x55c793edfa70 2024-06-10T02:56:22.955 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:56:22 smithi188 ceph-mon[36599]: pgmap v499: 129 pgs: 129 active+clean; 3.5 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 2.7 KiB/s wr, 10 op/s 2024-06-10T02:56:22.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.952+0000 7f072f7fe700 1 -- 172.21.15.52:0/2289534302 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f074403d050 con 0x55c793edfa70 2024-06-10T02:56:22.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.952+0000 7f0760e7cb80 10 client.25414 fetch_fsmap learned FSMap version 31 2024-06-10T02:56:22.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.952+0000 7f0760e7cb80 1 -- 172.21.15.52:0/2289534302 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55c793f5dea0 con 0x55c793edfa70 2024-06-10T02:56:22.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.952+0000 7f072f7fe700 1 -- 172.21.15.52:0/2289534302 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55c793e2f290 con 0x55c793edfa70 2024-06-10T02:56:22.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.953+0000 7f072f7fe700 1 -- 172.21.15.52:0/2289534302 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 31) ==== 8366+0+0 (secure 0 0 0) 0x7f074401e720 con 0x55c793edfa70 2024-06-10T02:56:22.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.953+0000 7f0760e7cb80 10 client.25414 fetch_fsmap finished waiting for FSMap version 31 2024-06-10T02:56:22.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.953+0000 7f0760e7cb80 10 client.25414 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-06-10T02:56:22.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.953+0000 7f0760e7cb80 20 client.25414 populate_metadata read hostname 'smithi052' 2024-06-10T02:56:22.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.953+0000 7f0760e7cb80 1 --2- 172.21.15.52:0/2289534302 >> [v2:172.21.15.188:6836/819317726,v1:172.21.15.188:6837/819317726] conn(0x55c793f5e5e0 0x55c793f60a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:56:22.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.953+0000 7f0760e7cb80 4 client.25414 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T02:56:22.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.953+0000 7f0760e7cb80 1 -- 172.21.15.52:0/2289534302 --> [v2:172.21.15.188:6836/819317726,v1:172.21.15.188:6837/819317726] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c793ca9250 con 0x55c793f5e5e0 2024-06-10T02:56:22.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.953+0000 7f074ffff700 1 --2- 172.21.15.52:0/2289534302 >> [v2:172.21.15.188:6836/819317726,v1:172.21.15.188:6837/819317726] conn(0x55c793f5e5e0 0x55c793f60a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:56:22.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.954+0000 7f074ffff700 1 --2- 172.21.15.52:0/2289534302 >> [v2:172.21.15.188:6836/819317726,v1:172.21.15.188:6837/819317726] conn(0x55c793f5e5e0 0x55c793f60a00 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=613e82b298f4cb17 server_cookie=a9045aae95ac3c80 in_seq=0 out_seq=0 2024-06-10T02:56:22.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.954+0000 7f072f7fe700 10 client.25414 ms_handle_connect on v2:172.21.15.188:6836/819317726 2024-06-10T02:56:22.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.954+0000 7f072f7fe700 1 -- 172.21.15.52:0/2289534302 <== mds.1 v2:172.21.15.188:6836/819317726 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c793ca9250 con 0x55c793f5e5e0 2024-06-10T02:56:22.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.954+0000 7f072f7fe700 10 client.25414 handle_command_reply: tid=0 2024-06-10T02:56:22.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.954+0000 7f0760e7cb80 1 client.25414 shutdown 2024-06-10T02:56:22.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.954+0000 7f0760e7cb80 1 -- 172.21.15.52:0/2289534302 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c793edfa70 msgr2=0x55c793eec620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:56:22.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.954+0000 7f0760e7cb80 1 --2- 172.21.15.52:0/2289534302 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c793edfa70 0x55c793eec620 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f0744007270 tx=0x7f07440047c0 comp rx=0 tx=0).stop 2024-06-10T02:56:22.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.954+0000 7f0760e7cb80 1 -- 172.21.15.52:0/2289534302 shutdown_connections 2024-06-10T02:56:22.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.954+0000 7f0760e7cb80 1 --2- 172.21.15.52:0/2289534302 >> [v2:172.21.15.188:6836/819317726,v1:172.21.15.188:6837/819317726] conn(0x55c793f5e5e0 0x55c793f60a00 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:22.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.954+0000 7f0760e7cb80 1 --2- 172.21.15.52:0/2289534302 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c793ee0390 0x55c793efa990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:22.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.954+0000 7f0760e7cb80 1 --2- 172.21.15.52:0/2289534302 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c793eda330 0x55c793eec0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:22.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.954+0000 7f0760e7cb80 1 --2- 172.21.15.52:0/2289534302 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c793edfa70 0x55c793eec620 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:22.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.954+0000 7f072f7fe700 0 client.25414 ms_handle_reset on v2:172.21.15.188:6836/819317726 2024-06-10T02:56:22.958 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.954+0000 7f0760e7cb80 1 -- 172.21.15.52:0/2289534302 >> 172.21.15.52:0/2289534302 conn(0x55c793e00cc0 msgr2=0x55c793ed49d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:56:22.958 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.955+0000 7f0760e7cb80 1 -- 172.21.15.52:0/2289534302 shutdown_connections 2024-06-10T02:56:22.958 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.955+0000 7f0760e7cb80 1 -- 172.21.15.52:0/2289534302 wait complete. 2024-06-10T02:56:22.958 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.955+0000 7f0760e7cb80 20 client.25414 trim_cache size 0 max 16384 2024-06-10T02:56:22.958 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-06-10T02:56:22.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.955+0000 7f07565f9700 1 -- 172.21.15.52:0/1254692243 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f07380a2c20 msgr2=0x7f07380a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:56:22.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.955+0000 7f07565f9700 1 --2- 172.21.15.52:0/1254692243 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f07380a2c20 0x7f07380a50e0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f0750106030 tx=0x7f07440073d0 comp rx=0 tx=0).stop 2024-06-10T02:56:22.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.956+0000 7f07565f9700 1 -- 172.21.15.52:0/1254692243 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f07501019d0 msgr2=0x7f0750199a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:56:22.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.956+0000 7f07565f9700 1 --2- 172.21.15.52:0/1254692243 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f07501019d0 0x7f0750199a70 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f07400059d0 tx=0x7f07400038b0 comp rx=0 tx=0).stop 2024-06-10T02:56:22.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.956+0000 7f07565f9700 1 -- 172.21.15.52:0/1254692243 shutdown_connections 2024-06-10T02:56:22.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.956+0000 7f07565f9700 1 --2- 172.21.15.52:0/1254692243 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0750106720 0x7f07501a4af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:22.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.956+0000 7f07565f9700 1 --2- 172.21.15.52:0/1254692243 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f07380a2c20 0x7f07380a50e0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:22.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.956+0000 7f07565f9700 1 --2- 172.21.15.52:0/1254692243 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f07501019d0 0x7f0750199a70 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:22.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.956+0000 7f07565f9700 1 --2- 172.21.15.52:0/1254692243 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0750102380 0x7f0750199fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:22.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.956+0000 7f07565f9700 1 -- 172.21.15.52:0/1254692243 >> 172.21.15.52:0/1254692243 conn(0x7f0750078340 msgr2=0x7f07500ff7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:56:22.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.956+0000 7f07565f9700 1 -- 172.21.15.52:0/1254692243 shutdown_connections 2024-06-10T02:56:22.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:22.956+0000 7f07565f9700 1 -- 172.21.15.52:0/1254692243 wait complete. 2024-06-10T02:56:22.967 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:56:23.646 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-06-10T02:56:24.727 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:56:24 smithi052 ceph-mon[32362]: pgmap v500: 129 pgs: 129 active+clean; 3.5 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 2.4 KiB/s wr, 13 op/s 2024-06-10T02:56:24.888 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:56:24 smithi120 ceph-mon[36634]: pgmap v500: 129 pgs: 129 active+clean; 3.5 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 2.4 KiB/s wr, 13 op/s 2024-06-10T02:56:24.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:56:24 smithi188 ceph-mon[36599]: pgmap v500: 129 pgs: 129 active+clean; 3.5 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 2.4 KiB/s wr, 13 op/s 2024-06-10T02:56:25.326 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:56:25.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:25.335+0000 7f418efa0700 1 -- 172.21.15.52:0/1654003537 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f418810c9b0 msgr2=0x7f418810ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:56:25.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:25.335+0000 7f418efa0700 1 --2- 172.21.15.52:0/1654003537 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f418810c9b0 0x7f418810ee70 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f418000b0a0 tx=0x7f418000b3b0 comp rx=0 tx=0).stop 2024-06-10T02:56:25.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:25.335+0000 7f418efa0700 1 -- 172.21.15.52:0/1654003537 shutdown_connections 2024-06-10T02:56:25.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:25.335+0000 7f418efa0700 1 --2- 172.21.15.52:0/1654003537 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f418810c9b0 0x7f418810ee70 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:25.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:25.335+0000 7f418efa0700 1 --2- 172.21.15.52:0/1654003537 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4188103580 0x7f4188103960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:25.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:25.335+0000 7f418efa0700 1 --2- 172.21.15.52:0/1654003537 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4188103ea0 0x7f418810c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:25.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:25.335+0000 7f418efa0700 1 -- 172.21.15.52:0/1654003537 >> 172.21.15.52:0/1654003537 conn(0x7f41880fcdb0 msgr2=0x7f41880ff1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:56:25.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:25.335+0000 7f418efa0700 1 -- 172.21.15.52:0/1654003537 shutdown_connections 2024-06-10T02:56:25.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:25.335+0000 7f418efa0700 1 -- 172.21.15.52:0/1654003537 wait complete. 2024-06-10T02:56:25.336 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:25.336+0000 7f418efa0700 1 Processor -- start 2024-06-10T02:56:25.336 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:25.336+0000 7f418efa0700 1 -- start start 2024-06-10T02:56:25.336 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:25.336+0000 7f418efa0700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4188103580 0x7f418819f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:56:25.336 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:25.336+0000 7f418efa0700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4188103ea0 0x7f418819fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:56:25.336 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:25.336+0000 7f418efa0700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f418810c9b0 0x7f418810b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:56:25.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:25.336+0000 7f418d79d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4188103ea0 0x7f418819fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:56:25.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:25.336+0000 7f418d79d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4188103ea0 0x7f418819fd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40304/0 (socket says 172.21.15.52:40304) 2024-06-10T02:56:25.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:25.336+0000 7f418d79d700 1 -- 172.21.15.52:0/1998354533 learned_addr learned my addr 172.21.15.52:0/1998354533 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:56:25.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:25.336+0000 7f418efa0700 1 -- 172.21.15.52:0/1998354533 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f4188112700 con 0x7f4188103ea0 2024-06-10T02:56:25.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:25.336+0000 7f418efa0700 1 -- 172.21.15.52:0/1998354533 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f4188112580 con 0x7f418810c9b0 2024-06-10T02:56:25.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:25.336+0000 7f418efa0700 1 -- 172.21.15.52:0/1998354533 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f4188112880 con 0x7f4188103580 2024-06-10T02:56:25.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:25.336+0000 7f418df9e700 1 --2- 172.21.15.52:0/1998354533 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4188103580 0x7f418819f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:56:25.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:25.336+0000 7f418e79f700 1 --2- 172.21.15.52:0/1998354533 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f418810c9b0 0x7f418810b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:56:25.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:25.337+0000 7f418d79d700 1 -- 172.21.15.52:0/1998354533 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4188103580 msgr2=0x7f418819f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:56:25.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:25.337+0000 7f418d79d700 1 --2- 172.21.15.52:0/1998354533 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4188103580 0x7f418819f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:25.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:25.337+0000 7f418d79d700 1 -- 172.21.15.52:0/1998354533 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f418810c9b0 msgr2=0x7f418810b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:56:25.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:25.337+0000 7f418d79d700 1 --2- 172.21.15.52:0/1998354533 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f418810c9b0 0x7f418810b4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:25.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:25.337+0000 7f418d79d700 1 -- 172.21.15.52:0/1998354533 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f418810bdc0 con 0x7f4188103ea0 2024-06-10T02:56:25.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:25.337+0000 7f418e79f700 1 --2- 172.21.15.52:0/1998354533 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f418810c9b0 0x7f418810b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T02:56:25.338 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:25.337+0000 7f418d79d700 1 --2- 172.21.15.52:0/1998354533 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4188103ea0 0x7f418819fd00 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f417800e9c0 tx=0x7f417800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:56:25.338 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:25.338+0000 7f417effd700 1 -- 172.21.15.52:0/1998354533 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f417800cbd0 con 0x7f4188103ea0 2024-06-10T02:56:25.338 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:25.338+0000 7f417effd700 1 -- 172.21.15.52:0/1998354533 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f417800cd70 con 0x7f4188103ea0 2024-06-10T02:56:25.338 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:25.338+0000 7f417effd700 1 -- 172.21.15.52:0/1998354533 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41780056c0 con 0x7f4188103ea0 2024-06-10T02:56:25.339 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:25.338+0000 7f418efa0700 1 -- 172.21.15.52:0/1998354533 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f418810c0b0 con 0x7f4188103ea0 2024-06-10T02:56:25.339 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:25.338+0000 7f418efa0700 1 -- 172.21.15.52:0/1998354533 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f418819b3e0 con 0x7f4188103ea0 2024-06-10T02:56:25.339 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:25.339+0000 7f418efa0700 1 -- 172.21.15.52:0/1998354533 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f418806aa50 con 0x7f4188103ea0 2024-06-10T02:56:25.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:25.340+0000 7f417effd700 1 -- 172.21.15.52:0/1998354533 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f4178004750 con 0x7f4188103ea0 2024-06-10T02:56:25.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:25.341+0000 7f417effd700 1 --2- 172.21.15.52:0/1998354533 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f41740a2d70 0x7f41740a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:56:25.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:25.341+0000 7f417effd700 1 -- 172.21.15.52:0/1998354533 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f41780cf660 con 0x7f4188103ea0 2024-06-10T02:56:25.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:25.341+0000 7f418df9e700 1 --2- 172.21.15.52:0/1998354533 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f41740a2d70 0x7f41740a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:56:25.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:25.341+0000 7f418df9e700 1 --2- 172.21.15.52:0/1998354533 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f41740a2d70 0x7f41740a5230 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f4184009c60 tx=0x7f4184009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:56:25.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:25.345+0000 7f417effd700 1 -- 172.21.15.52:0/1998354533 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f4178053850 con 0x7f4188103ea0 2024-06-10T02:56:25.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:25.604+0000 7f418efa0700 1 -- 172.21.15.52:0/1998354533 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4188069180 con 0x7f4188103ea0 2024-06-10T02:56:25.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:25.605+0000 7f417effd700 1 -- 172.21.15.52:0/1998354533 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11939 (secure 0 0 0) 0x7f417809c230 con 0x7f4188103ea0 2024-06-10T02:56:25.605 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-10T02:56:25.605 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/889673247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":889673247},{"type":"v1","addr":"172.21.15.52:6835","nonce":889673247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/3615663511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":3615663511},{"type":"v1","addr":"172.21.15.52:6837","nonce":3615663511}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":31,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T02:50:03.889766+0000","modified":"2024-06-10T02:55:51.192225+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24484,"mds_1":24469,"mds_2":24463,"mds_3":24449,"mds_4":24428},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14571":{"gid":14571,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6839/3059599042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":3059599042},{"type":"v1","addr":"172.21.15.52:6839","nonce":3059599042}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14583":{"gid":14583,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6841/3039275521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":3039275521},{"type":"v1","addr":"172.21.15.52:6841","nonce":3039275521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24383":{"gid":24383,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6833/2609002896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":2609002896},{"type":"v1","addr":"172.21.15.120:6833","nonce":2609002896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6835/1996370288","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":1996370288},{"type":"v1","addr":"172.21.15.120:6835","nonce":1996370288}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6833/529564979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":529564979},{"type":"v1","addr":"172.21.15.188:6833","nonce":529564979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":25,"addr":"172.21.15.120:6837/4132440952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":4132440952},{"type":"v1","addr":"172.21.15.120:6837","nonce":4132440952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"b","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.120:6839/1672436487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1672436487},{"type":"v1","addr":"172.21.15.120:6839","nonce":1672436487}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":2,"incarnation":22,"state":"up:active","state_seq":26,"addr":"172.21.15.188:6835/3904544259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":3904544259},{"type":"v1","addr":"172.21.15.188:6835","nonce":3904544259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.188:6837/819317726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":819317726},{"type":"v1","addr":"172.21.15.188:6837","nonce":819317726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.188:6839/310444836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":310444836},{"type":"v1","addr":"172.21.15.188:6839","nonce":310444836}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24484,"qdb_cluster":[24469,24449,24428,24463,24484]},"id":1}]} 2024-06-10T02:56:25.607 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:25.608+0000 7f418efa0700 1 -- 172.21.15.52:0/1998354533 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f41740a2d70 msgr2=0x7f41740a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:56:25.607 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:25.608+0000 7f418efa0700 1 --2- 172.21.15.52:0/1998354533 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f41740a2d70 0x7f41740a5230 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f4184009c60 tx=0x7f4184009260 comp rx=0 tx=0).stop 2024-06-10T02:56:25.607 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:25.608+0000 7f418efa0700 1 -- 172.21.15.52:0/1998354533 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4188103ea0 msgr2=0x7f418819fd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:56:25.607 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:25.608+0000 7f418efa0700 1 --2- 172.21.15.52:0/1998354533 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4188103ea0 0x7f418819fd00 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f417800e9c0 tx=0x7f417800ecd0 comp rx=0 tx=0).stop 2024-06-10T02:56:25.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:25.608+0000 7f418efa0700 1 -- 172.21.15.52:0/1998354533 shutdown_connections 2024-06-10T02:56:25.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:25.608+0000 7f418efa0700 1 --2- 172.21.15.52:0/1998354533 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f418810c9b0 0x7f418810b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:25.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:25.608+0000 7f418efa0700 1 --2- 172.21.15.52:0/1998354533 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4188103580 0x7f418819f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:25.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:25.608+0000 7f418efa0700 1 --2- 172.21.15.52:0/1998354533 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f41740a2d70 0x7f41740a5230 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:25.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:25.608+0000 7f418efa0700 1 --2- 172.21.15.52:0/1998354533 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4188103ea0 0x7f418819fd00 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:25.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:25.608+0000 7f418efa0700 1 -- 172.21.15.52:0/1998354533 >> 172.21.15.52:0/1998354533 conn(0x7f41880fcdb0 msgr2=0x7f418810ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:56:25.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:25.608+0000 7f418efa0700 1 -- 172.21.15.52:0/1998354533 shutdown_connections 2024-06-10T02:56:25.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:25.608+0000 7f418efa0700 1 -- 172.21.15.52:0/1998354533 wait complete. 2024-06-10T02:56:25.608 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 31 2024-06-10T02:56:26.726 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:56:26 smithi052 ceph-mon[32362]: pgmap v501: 129 pgs: 129 active+clean; 3.5 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-06-10T02:56:26.726 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:56:26 smithi052 ceph-mon[32362]: from='client.? 172.21.15.52:0/1998354533' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T02:56:26.888 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:56:26 smithi120 ceph-mon[36634]: pgmap v501: 129 pgs: 129 active+clean; 3.5 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-06-10T02:56:26.888 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:56:26 smithi120 ceph-mon[36634]: from='client.? 172.21.15.52:0/1998354533' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T02:56:26.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:56:26 smithi188 ceph-mon[36599]: pgmap v501: 129 pgs: 129 active+clean; 3.5 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-06-10T02:56:26.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:56:26 smithi188 ceph-mon[36599]: from='client.? 172.21.15.52:0/1998354533' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T02:56:27.079 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24484 quiesce db --set-id cdf1e680 --release --await' 2024-06-10T02:56:27.080 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24484 quiesce db --set-id cdf1e680 --release --await 2024-06-10T02:56:27.580 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.581+0000 7f15d68a6700 1 -- 172.21.15.52:0/1888659765 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f15d0101380 msgr2=0x7f15d0101800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:56:27.580 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.581+0000 7f15d68a6700 1 --2- 172.21.15.52:0/1888659765 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f15d0101380 0x7f15d0101800 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f15c4009990 tx=0x7f15c4009ca0 comp rx=0 tx=0).stop 2024-06-10T02:56:27.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.581+0000 7f15d68a6700 1 -- 172.21.15.52:0/1888659765 shutdown_connections 2024-06-10T02:56:27.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.581+0000 7f15d68a6700 1 --2- 172.21.15.52:0/1888659765 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f15d0101d40 0x7f15d010e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:27.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.581+0000 7f15d68a6700 1 --2- 172.21.15.52:0/1888659765 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f15d01073f0 0x7f15d01077d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:27.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.581+0000 7f15d68a6700 1 --2- 172.21.15.52:0/1888659765 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f15d0101380 0x7f15d0101800 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:27.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.581+0000 7f15d68a6700 1 -- 172.21.15.52:0/1888659765 >> 172.21.15.52:0/1888659765 conn(0x7f15d0078300 msgr2=0x7f15d0078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:56:27.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.581+0000 7f15d68a6700 1 -- 172.21.15.52:0/1888659765 shutdown_connections 2024-06-10T02:56:27.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.581+0000 7f15d68a6700 1 -- 172.21.15.52:0/1888659765 wait complete. 2024-06-10T02:56:27.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.582+0000 7f15d68a6700 1 Processor -- start 2024-06-10T02:56:27.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.582+0000 7f15d68a6700 1 -- start start 2024-06-10T02:56:27.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.582+0000 7f15d68a6700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f15d0101380 0x7f15d019d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:56:27.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.582+0000 7f15d68a6700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f15d0101d40 0x7f15d019dc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:56:27.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.582+0000 7f15d68a6700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f15d01073f0 0x7f15d0197940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:56:27.583 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.582+0000 7f15d68a6700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f15d01116e0 con 0x7f15d01073f0 2024-06-10T02:56:27.583 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.582+0000 7f15d68a6700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f15d0111560 con 0x7f15d0101380 2024-06-10T02:56:27.583 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.582+0000 7f15d68a6700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f15d0111860 con 0x7f15d0101d40 2024-06-10T02:56:27.583 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.582+0000 7f15d4e43700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f15d01073f0 0x7f15d0197940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:56:27.583 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.583+0000 7f15d4e43700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f15d01073f0 0x7f15d0197940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40316/0 (socket says 172.21.15.52:40316) 2024-06-10T02:56:27.583 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.583+0000 7f15d4e43700 1 -- 172.21.15.52:0/1441275557 learned_addr learned my addr 172.21.15.52:0/1441275557 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:56:27.583 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.583+0000 7f15cffff700 1 --2- 172.21.15.52:0/1441275557 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f15d0101380 0x7f15d019d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:56:27.583 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.583+0000 7f15cf7fe700 1 --2- 172.21.15.52:0/1441275557 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f15d0101d40 0x7f15d019dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:56:27.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.583+0000 7f15cf7fe700 1 -- 172.21.15.52:0/1441275557 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f15d0101380 msgr2=0x7f15d019d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:56:27.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.583+0000 7f15cf7fe700 1 --2- 172.21.15.52:0/1441275557 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f15d0101380 0x7f15d019d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:27.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.583+0000 7f15cf7fe700 1 -- 172.21.15.52:0/1441275557 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f15d01073f0 msgr2=0x7f15d0197940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:56:27.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.583+0000 7f15cf7fe700 1 --2- 172.21.15.52:0/1441275557 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f15d01073f0 0x7f15d0197940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:27.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.583+0000 7f15cf7fe700 1 -- 172.21.15.52:0/1441275557 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15d01980a0 con 0x7f15d0101d40 2024-06-10T02:56:27.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.583+0000 7f15d4e43700 1 --2- 172.21.15.52:0/1441275557 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f15d01073f0 0x7f15d0197940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T02:56:27.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.583+0000 7f15cf7fe700 1 --2- 172.21.15.52:0/1441275557 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f15d0101d40 0x7f15d019dc70 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f15c4006010 tx=0x7f15c40038b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:56:27.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.584+0000 7f15cd7fa700 1 -- 172.21.15.52:0/1441275557 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15c4027070 con 0x7f15d0101d40 2024-06-10T02:56:27.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.584+0000 7f15cd7fa700 1 -- 172.21.15.52:0/1441275557 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f15c4004050 con 0x7f15d0101d40 2024-06-10T02:56:27.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.584+0000 7f15d68a6700 1 -- 172.21.15.52:0/1441275557 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15d0198330 con 0x7f15d0101d40 2024-06-10T02:56:27.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.584+0000 7f15cd7fa700 1 -- 172.21.15.52:0/1441275557 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15c4022cb0 con 0x7f15d0101d40 2024-06-10T02:56:27.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.584+0000 7f15d68a6700 1 -- 172.21.15.52:0/1441275557 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15d0198750 con 0x7f15d0101d40 2024-06-10T02:56:27.586 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.587+0000 7f15cd7fa700 1 -- 172.21.15.52:0/1441275557 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f15c402a070 con 0x7f15d0101d40 2024-06-10T02:56:27.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.588+0000 7f15cd7fa700 1 --2- 172.21.15.52:0/1441275557 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f15b80a2d70 0x7f15b80a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:56:27.588 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.588+0000 7f15cd7fa700 1 -- 172.21.15.52:0/1441275557 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f15c40d0260 con 0x7f15d0101d40 2024-06-10T02:56:27.588 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.588+0000 7f15cffff700 1 --2- 172.21.15.52:0/1441275557 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f15b80a2d70 0x7f15b80a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:56:27.588 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.588+0000 7f15e1129b80 1 Processor -- start 2024-06-10T02:56:27.588 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.589+0000 7f15e1129b80 1 -- start start 2024-06-10T02:56:27.588 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.589+0000 7f15e1129b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d07d1f5e50 0x55d07d1f6230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:56:27.588 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.589+0000 7f15e1129b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d07d1f6770 0x55d07d1f01f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:56:27.589 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.589+0000 7f15e1129b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d07d1f0730 0x55d07d1f0b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:56:27.589 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.589+0000 7f15e1129b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55d07d139810 con 0x55d07d1f0730 2024-06-10T02:56:27.590 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.589+0000 7f15e1129b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55d07d0e1720 con 0x55d07d1f5e50 2024-06-10T02:56:27.590 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.589+0000 7f15cffff700 1 --2- 172.21.15.52:0/1441275557 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f15b80a2d70 0x7f15b80a5230 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f15d0107bd0 tx=0x7f15c000a380 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:56:27.590 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.589+0000 7f15e1129b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55d07d1357d0 con 0x55d07d1f6770 2024-06-10T02:56:27.590 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.589+0000 7f15d4e43700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d07d1f0730 0x55d07d1f0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:56:27.590 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.589+0000 7f15d4e43700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d07d1f0730 0x55d07d1f0b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40322/0 (socket says 172.21.15.52:40322) 2024-06-10T02:56:27.590 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.589+0000 7f15d4e43700 1 -- 172.21.15.52:0/3450302902 learned_addr learned my addr 172.21.15.52:0/3450302902 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:56:27.590 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.589+0000 7f15cffff700 1 --2- 172.21.15.52:0/3450302902 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d07d1f5e50 0x55d07d1f6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:56:27.590 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.589+0000 7f15cf7fe700 1 --2- 172.21.15.52:0/3450302902 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d07d1f6770 0x55d07d1f01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:56:27.590 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.589+0000 7f15cffff700 1 -- 172.21.15.52:0/3450302902 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d07d1f6770 msgr2=0x55d07d1f01f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:56:27.590 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.589+0000 7f15cffff700 1 --2- 172.21.15.52:0/3450302902 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d07d1f6770 0x55d07d1f01f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:27.590 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.589+0000 7f15cffff700 1 -- 172.21.15.52:0/3450302902 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d07d1f0730 msgr2=0x55d07d1f0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:56:27.590 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.589+0000 7f15cffff700 1 --2- 172.21.15.52:0/3450302902 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d07d1f0730 0x55d07d1f0b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:27.590 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.589+0000 7f15cffff700 1 -- 172.21.15.52:0/3450302902 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d07d0d7f50 con 0x55d07d1f5e50 2024-06-10T02:56:27.591 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.589+0000 7f15d4e43700 1 --2- 172.21.15.52:0/3450302902 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d07d1f0730 0x55d07d1f0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T02:56:27.591 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.590+0000 7f15cffff700 1 --2- 172.21.15.52:0/3450302902 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d07d1f5e50 0x55d07d1f6230 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f15c000f750 tx=0x7f15c0012000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c12a1a246194f55 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:56:27.591 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.590+0000 7f15b67fc700 1 -- 172.21.15.52:0/3450302902 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15c0016070 con 0x55d07d1f5e50 2024-06-10T02:56:27.591 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.590+0000 7f15b67fc700 1 -- 172.21.15.52:0/3450302902 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f15c0014070 con 0x55d07d1f5e50 2024-06-10T02:56:27.593 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.591+0000 7f15b67fc700 1 -- 172.21.15.52:0/3450302902 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15c0010d40 con 0x55d07d1f5e50 2024-06-10T02:56:27.593 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.591+0000 7f15e1129b80 1 -- 172.21.15.52:0/3450302902 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d07d1f5e50 msgr2=0x55d07d1f6230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:56:27.593 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.591+0000 7f15e1129b80 1 --2- 172.21.15.52:0/3450302902 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d07d1f5e50 0x55d07d1f6230 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f15c000f750 tx=0x7f15c0012000 comp rx=0 tx=0).stop 2024-06-10T02:56:27.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.591+0000 7f15e1129b80 1 -- 172.21.15.52:0/3450302902 shutdown_connections 2024-06-10T02:56:27.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.591+0000 7f15e1129b80 1 --2- 172.21.15.52:0/3450302902 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d07d1f0730 0x55d07d1f0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:27.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.591+0000 7f15e1129b80 1 --2- 172.21.15.52:0/3450302902 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d07d1f5e50 0x55d07d1f6230 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:27.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.591+0000 7f15e1129b80 1 --2- 172.21.15.52:0/3450302902 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d07d1f6770 0x55d07d1f01f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:27.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.591+0000 7f15e1129b80 1 -- 172.21.15.52:0/3450302902 >> 172.21.15.52:0/3450302902 conn(0x55d07d112cc0 msgr2=0x55d07d1e2850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:56:27.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.591+0000 7f15e1129b80 1 -- 172.21.15.52:0/3450302902 shutdown_connections 2024-06-10T02:56:27.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.591+0000 7f15e1129b80 1 -- 172.21.15.52:0/3450302902 wait complete. 2024-06-10T02:56:27.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.591+0000 7f15e1129b80 1 -- start start 2024-06-10T02:56:27.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.591+0000 7f15e1129b80 1 Processor -- start 2024-06-10T02:56:27.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.592+0000 7f15e1129b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d07d1f0730 0x55d07d1fc490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:56:27.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.592+0000 7f15e1129b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d07d1f5e50 0x55d07d1f9790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:56:27.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.592+0000 7f15e1129b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d07d1f6770 0x55d07d1f9cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:56:27.595 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.592+0000 7f15e1129b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55d07d1fd280 con 0x55d07d1f6770 2024-06-10T02:56:27.595 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.592+0000 7f15e1129b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55d07d0d9bf0 con 0x55d07d1f0730 2024-06-10T02:56:27.595 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.592+0000 7f15e1129b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55d07d1fd400 con 0x55d07d1f5e50 2024-06-10T02:56:27.595 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.592+0000 7f15e1129b80 10 client.0 _ll_register_callbacks cb 0x55d07d0d3920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T02:56:27.595 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.592+0000 7f15d4e43700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d07d1f6770 0x55d07d1f9cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:56:27.595 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.592+0000 7f15d4e43700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d07d1f6770 0x55d07d1f9cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40326/0 (socket says 172.21.15.52:40326) 2024-06-10T02:56:27.595 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.592+0000 7f15d4e43700 1 -- 172.21.15.52:0/3726804811 learned_addr learned my addr 172.21.15.52:0/3726804811 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:56:27.595 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.592+0000 7f15cffff700 1 --2- 172.21.15.52:0/3726804811 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d07d1f0730 0x55d07d1fc490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:56:27.595 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.592+0000 7f15cf7fe700 1 --2- 172.21.15.52:0/3726804811 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d07d1f5e50 0x55d07d1f9790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:56:27.595 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.592+0000 7f15d4e43700 1 -- 172.21.15.52:0/3726804811 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d07d1f5e50 msgr2=0x55d07d1f9790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:56:27.595 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.592+0000 7f15d4e43700 1 --2- 172.21.15.52:0/3726804811 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d07d1f5e50 0x55d07d1f9790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:27.595 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.592+0000 7f15d4e43700 1 -- 172.21.15.52:0/3726804811 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d07d1f0730 msgr2=0x55d07d1fc490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:56:27.595 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.592+0000 7f15d4e43700 1 --2- 172.21.15.52:0/3726804811 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d07d1f0730 0x55d07d1fc490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:27.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.592+0000 7f15d4e43700 1 -- 172.21.15.52:0/3726804811 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d07d0888d0 con 0x55d07d1f6770 2024-06-10T02:56:27.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.592+0000 7f15cffff700 1 --2- 172.21.15.52:0/3726804811 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d07d1f0730 0x55d07d1fc490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T02:56:27.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.592+0000 7f15d4e43700 1 --2- 172.21.15.52:0/3726804811 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d07d1f6770 0x55d07d1f9cd0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f15c8009f10 tx=0x7f15c80076a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bec3866f3276e41 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:56:27.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.592+0000 7f15af7fe700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-10T02:56:27.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.593+0000 7f15af7fe700 1 -- 172.21.15.52:0/3726804811 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15c801b020 con 0x55d07d1f6770 2024-06-10T02:56:27.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.593+0000 7f15af7fe700 1 -- 172.21.15.52:0/3726804811 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f15c8017660 con 0x55d07d1f6770 2024-06-10T02:56:27.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.593+0000 7f15e1129b80 1 -- 172.21.15.52:0/3726804811 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f15c0014070 con 0x55d07d1f6770 2024-06-10T02:56:27.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.593+0000 7f15af7fe700 1 -- 172.21.15.52:0/3726804811 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f15c8026050 con 0x55d07d1f6770 2024-06-10T02:56:27.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.593+0000 7f15e1129b80 10 client.15858 fetch_fsmap learned FSMap version 31 2024-06-10T02:56:27.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.593+0000 7f15e1129b80 1 -- 172.21.15.52:0/3726804811 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55d07d207b60 con 0x55d07d1f6770 2024-06-10T02:56:27.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.593+0000 7f15af7fe700 1 -- 172.21.15.52:0/3726804811 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f15c0014070 con 0x55d07d1f6770 2024-06-10T02:56:27.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.594+0000 7f15af7fe700 1 -- 172.21.15.52:0/3726804811 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 31) ==== 8366+0+0 (secure 0 0 0) 0x7f15c80159a0 con 0x55d07d1f6770 2024-06-10T02:56:27.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.594+0000 7f15e1129b80 10 client.15858 fetch_fsmap finished waiting for FSMap version 31 2024-06-10T02:56:27.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.594+0000 7f15e1129b80 10 client.15858 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T02:56:27.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.594+0000 7f15e1129b80 20 client.15858 populate_metadata read hostname 'smithi052' 2024-06-10T02:56:27.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.594+0000 7f15e1129b80 1 --2- 172.21.15.52:0/3726804811 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55d07d20a1f0 0x55d07d20c620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:56:27.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.594+0000 7f15e1129b80 4 client.15858 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T02:56:27.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.594+0000 7f15e1129b80 1 -- 172.21.15.52:0/3726804811 --> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d07d146d10 con 0x55d07d20a1f0 2024-06-10T02:56:27.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.594+0000 7f15cffff700 1 --2- 172.21.15.52:0/3726804811 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55d07d20a1f0 0x55d07d20c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:56:27.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.595+0000 7f15cffff700 1 --2- 172.21.15.52:0/3726804811 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55d07d20a1f0 0x55d07d20c620 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=33ab84e63497166e server_cookie=550856c3d8d0be7a in_seq=0 out_seq=0 2024-06-10T02:56:27.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.595+0000 7f15af7fe700 10 client.15858 ms_handle_connect on v2:172.21.15.188:6838/310444836 2024-06-10T02:56:27.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.596+0000 7f15af7fe700 1 -- 172.21.15.52:0/3726804811 <== mds.0 v2:172.21.15.188:6838/310444836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55d07d146d10 con 0x55d07d20a1f0 2024-06-10T02:56:27.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.596+0000 7f15af7fe700 10 client.15858 handle_command_reply: tid=0 2024-06-10T02:56:27.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.596+0000 7f15e1129b80 1 client.15858 shutdown 2024-06-10T02:56:27.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.596+0000 7f15e1129b80 1 -- 172.21.15.52:0/3726804811 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d07d1f6770 msgr2=0x55d07d1f9cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:56:27.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.596+0000 7f15e1129b80 1 --2- 172.21.15.52:0/3726804811 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d07d1f6770 0x55d07d1f9cd0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f15c8009f10 tx=0x7f15c80076a0 comp rx=0 tx=0).stop 2024-06-10T02:56:27.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.596+0000 7f15e1129b80 1 -- 172.21.15.52:0/3726804811 shutdown_connections 2024-06-10T02:56:27.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.596+0000 7f15e1129b80 1 --2- 172.21.15.52:0/3726804811 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55d07d20a1f0 0x55d07d20c620 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:27.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.596+0000 7f15e1129b80 1 --2- 172.21.15.52:0/3726804811 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d07d1f6770 0x55d07d1f9cd0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:27.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.596+0000 7f15e1129b80 1 --2- 172.21.15.52:0/3726804811 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d07d1f0730 0x55d07d1fc490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:27.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.596+0000 7f15e1129b80 1 --2- 172.21.15.52:0/3726804811 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d07d1f5e50 0x55d07d1f9790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:27.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.596+0000 7f15af7fe700 0 client.15858 ms_handle_reset on v2:172.21.15.188:6838/310444836 2024-06-10T02:56:27.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.596+0000 7f15e1129b80 1 -- 172.21.15.52:0/3726804811 >> 172.21.15.52:0/3726804811 conn(0x55d07d112cc0 msgr2=0x55d07d1ef1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:56:27.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.596+0000 7f15e1129b80 1 -- 172.21.15.52:0/3726804811 shutdown_connections 2024-06-10T02:56:27.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.596+0000 7f15e1129b80 1 -- 172.21.15.52:0/3726804811 wait complete. 2024-06-10T02:56:27.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.596+0000 7f15e1129b80 20 client.15858 trim_cache size 0 max 16384 2024-06-10T02:56:27.601 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:56:27.620 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.619+0000 7f15e1129b80 1 Processor -- start 2024-06-10T02:56:27.620 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.620+0000 7f15e1129b80 1 -- start start 2024-06-10T02:56:27.620 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.620+0000 7f15e1129b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d07d1f0730 0x55d07d1f01f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:56:27.620 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.620+0000 7f15e1129b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d07d1f5e50 0x55d07d20c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:56:27.620 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.620+0000 7f15e1129b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d07d1f6770 0x55d07d1dbe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:56:27.620 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.620+0000 7f15e1129b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f15d01ac020 con 0x55d07d1f0730 2024-06-10T02:56:27.620 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.620+0000 7f15e1129b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f15d01adb90 con 0x55d07d1f5e50 2024-06-10T02:56:27.620 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.620+0000 7f15e1129b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f15d01ac6f0 con 0x55d07d1f6770 2024-06-10T02:56:27.620 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.620+0000 7f15cffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d07d1f0730 0x55d07d1f01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:56:27.620 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.620+0000 7f15cffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d07d1f0730 0x55d07d1f01f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40340/0 (socket says 172.21.15.52:40340) 2024-06-10T02:56:27.620 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.620+0000 7f15cffff700 1 -- 172.21.15.52:0/1373477032 learned_addr learned my addr 172.21.15.52:0/1373477032 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:56:27.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.620+0000 7f15cf7fe700 1 --2- 172.21.15.52:0/1373477032 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d07d1f5e50 0x55d07d20c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:56:27.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.620+0000 7f15d4e43700 1 --2- 172.21.15.52:0/1373477032 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d07d1f6770 0x55d07d1dbe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:56:27.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.620+0000 7f15cffff700 1 -- 172.21.15.52:0/1373477032 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d07d1f6770 msgr2=0x55d07d1dbe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:56:27.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.620+0000 7f15cffff700 1 --2- 172.21.15.52:0/1373477032 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d07d1f6770 0x55d07d1dbe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:27.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.620+0000 7f15cffff700 1 -- 172.21.15.52:0/1373477032 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d07d1f5e50 msgr2=0x55d07d20c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:56:27.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.620+0000 7f15cffff700 1 --2- 172.21.15.52:0/1373477032 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d07d1f5e50 0x55d07d20c5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:27.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.620+0000 7f15cffff700 1 -- 172.21.15.52:0/1373477032 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d07d1fd450 con 0x55d07d1f0730 2024-06-10T02:56:27.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.621+0000 7f15cffff700 1 --2- 172.21.15.52:0/1373477032 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d07d1f0730 0x55d07d1f01f0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f15c00045f0 tx=0x7f15c000aa10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7ea4144691fa4bb server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:56:27.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.621+0000 7f15b5ffb700 1 -- 172.21.15.52:0/1373477032 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15c0016030 con 0x55d07d1f0730 2024-06-10T02:56:27.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.621+0000 7f15b5ffb700 1 -- 172.21.15.52:0/1373477032 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f15c0014070 con 0x55d07d1f0730 2024-06-10T02:56:27.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.621+0000 7f15b5ffb700 1 -- 172.21.15.52:0/1373477032 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15c00052b0 con 0x55d07d1f0730 2024-06-10T02:56:27.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.622+0000 7f15e1129b80 1 -- 172.21.15.52:0/1373477032 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d07d1f0730 msgr2=0x55d07d1f01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:56:27.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.622+0000 7f15e1129b80 1 --2- 172.21.15.52:0/1373477032 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d07d1f0730 0x55d07d1f01f0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f15c00045f0 tx=0x7f15c000aa10 comp rx=0 tx=0).stop 2024-06-10T02:56:27.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.622+0000 7f15e1129b80 1 -- 172.21.15.52:0/1373477032 shutdown_connections 2024-06-10T02:56:27.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.622+0000 7f15e1129b80 1 --2- 172.21.15.52:0/1373477032 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d07d1f6770 0x55d07d1dbe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:27.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.622+0000 7f15e1129b80 1 --2- 172.21.15.52:0/1373477032 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d07d1f0730 0x55d07d1f01f0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:27.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.622+0000 7f15e1129b80 1 --2- 172.21.15.52:0/1373477032 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d07d1f5e50 0x55d07d20c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:27.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.622+0000 7f15e1129b80 1 -- 172.21.15.52:0/1373477032 >> 172.21.15.52:0/1373477032 conn(0x55d07d112cc0 msgr2=0x55d07d1f8080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:56:27.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.622+0000 7f15e1129b80 1 -- 172.21.15.52:0/1373477032 shutdown_connections 2024-06-10T02:56:27.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.622+0000 7f15e1129b80 1 -- 172.21.15.52:0/1373477032 wait complete. 2024-06-10T02:56:27.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.623+0000 7f15e1129b80 1 -- start start 2024-06-10T02:56:27.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.623+0000 7f15e1129b80 1 Processor -- start 2024-06-10T02:56:27.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.623+0000 7f15e1129b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d07d1f0730 0x55d07d202710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:56:27.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.623+0000 7f15e1129b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d07d1f5e50 0x55d07d202c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:56:27.625 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.623+0000 7f15e1129b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d07d1f6770 0x55d07d211000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:56:27.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.623+0000 7f15e1129b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f15d01ad490 con 0x55d07d1f0730 2024-06-10T02:56:27.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.623+0000 7f15e1129b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f15d01acdc0 con 0x55d07d1f5e50 2024-06-10T02:56:27.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.623+0000 7f15e1129b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f15d01a9fc0 con 0x55d07d1f6770 2024-06-10T02:56:27.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.623+0000 7f15cffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d07d1f0730 0x55d07d202710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:56:27.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.623+0000 7f15e1129b80 10 client.0 _ll_register_callbacks cb 0x55d07d0d3920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T02:56:27.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.624+0000 7f15cffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d07d1f0730 0x55d07d202710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40344/0 (socket says 172.21.15.52:40344) 2024-06-10T02:56:27.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.624+0000 7f15cffff700 1 -- 172.21.15.52:0/1580085710 learned_addr learned my addr 172.21.15.52:0/1580085710 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:56:27.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.624+0000 7f15d4e43700 1 --2- 172.21.15.52:0/1580085710 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d07d1f6770 0x55d07d211000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:56:27.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.624+0000 7f15cf7fe700 1 --2- 172.21.15.52:0/1580085710 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d07d1f5e50 0x55d07d202c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:56:27.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.624+0000 7f15cffff700 1 -- 172.21.15.52:0/1580085710 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d07d1f6770 msgr2=0x55d07d211000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:56:27.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.624+0000 7f15cffff700 1 --2- 172.21.15.52:0/1580085710 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d07d1f6770 0x55d07d211000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:27.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.624+0000 7f15cffff700 1 -- 172.21.15.52:0/1580085710 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d07d1f5e50 msgr2=0x55d07d202c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:56:27.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.624+0000 7f15cffff700 1 --2- 172.21.15.52:0/1580085710 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d07d1f5e50 0x55d07d202c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:27.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.624+0000 7f15cffff700 1 -- 172.21.15.52:0/1580085710 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d07d211540 con 0x55d07d1f0730 2024-06-10T02:56:27.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.624+0000 7f15cf7fe700 1 --2- 172.21.15.52:0/1580085710 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d07d1f5e50 0x55d07d202c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T02:56:27.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.624+0000 7f15cffff700 1 --2- 172.21.15.52:0/1580085710 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d07d1f0730 0x55d07d202710 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f15c0003e40 tx=0x7f15c0003f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4511f9e62a143bee server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:56:27.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.624+0000 7f15af7fe700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-10T02:56:27.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.624+0000 7f15af7fe700 1 -- 172.21.15.52:0/1580085710 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15c0016040 con 0x55d07d1f0730 2024-06-10T02:56:27.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.624+0000 7f15af7fe700 1 -- 172.21.15.52:0/1580085710 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f15c0005460 con 0x55d07d1f0730 2024-06-10T02:56:27.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.624+0000 7f15e1129b80 1 -- 172.21.15.52:0/1580085710 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f15c0014070 con 0x55d07d1f0730 2024-06-10T02:56:27.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.625+0000 7f15af7fe700 1 -- 172.21.15.52:0/1580085710 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f15c004a050 con 0x55d07d1f0730 2024-06-10T02:56:27.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.625+0000 7f15e1129b80 10 client.15864 fetch_fsmap learned FSMap version 31 2024-06-10T02:56:27.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.625+0000 7f15e1129b80 1 -- 172.21.15.52:0/1580085710 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55d07d2746a0 con 0x55d07d1f0730 2024-06-10T02:56:27.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.625+0000 7f15af7fe700 1 -- 172.21.15.52:0/1580085710 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f15c0014070 con 0x55d07d1f0730 2024-06-10T02:56:27.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.625+0000 7f15af7fe700 1 -- 172.21.15.52:0/1580085710 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 31) ==== 8366+0+0 (secure 0 0 0) 0x7f15c0039070 con 0x55d07d1f0730 2024-06-10T02:56:27.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.625+0000 7f15e1129b80 10 client.15864 fetch_fsmap finished waiting for FSMap version 31 2024-06-10T02:56:27.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.625+0000 7f15e1129b80 10 client.15864 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T02:56:27.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.625+0000 7f15e1129b80 20 client.15864 populate_metadata read hostname 'smithi052' 2024-06-10T02:56:27.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.625+0000 7f15e1129b80 1 --2- 172.21.15.52:0/1580085710 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55d07d20a1f0 0x55d07d276bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:56:27.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.625+0000 7f15e1129b80 4 client.15864 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T02:56:27.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.625+0000 7f15e1129b80 1 -- 172.21.15.52:0/1580085710 --> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d07d141290 con 0x55d07d20a1f0 2024-06-10T02:56:27.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.625+0000 7f15cf7fe700 1 --2- 172.21.15.52:0/1580085710 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55d07d20a1f0 0x55d07d276bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:56:27.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.626+0000 7f15af7fe700 10 client.15864 ms_handle_connect on v2:172.21.15.188:6838/310444836 2024-06-10T02:56:27.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.626+0000 7f15cf7fe700 1 --2- 172.21.15.52:0/1580085710 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55d07d20a1f0 0x55d07d276bc0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cfc143f073a51a9d server_cookie=b628dba51037b4bc in_seq=0 out_seq=0 2024-06-10T02:56:27.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.626+0000 7f15af7fe700 1 -- 172.21.15.52:0/1580085710 <== mds.0 v2:172.21.15.188:6838/310444836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d07d141290 con 0x55d07d20a1f0 2024-06-10T02:56:27.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.626+0000 7f15af7fe700 10 client.15864 handle_command_reply: tid=0 2024-06-10T02:56:27.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.626+0000 7f15e1129b80 1 client.15864 shutdown 2024-06-10T02:56:27.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.626+0000 7f15e1129b80 1 -- 172.21.15.52:0/1580085710 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d07d1f0730 msgr2=0x55d07d202710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:56:27.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.626+0000 7f15e1129b80 1 --2- 172.21.15.52:0/1580085710 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d07d1f0730 0x55d07d202710 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f15c0003e40 tx=0x7f15c0003f20 comp rx=0 tx=0).stop 2024-06-10T02:56:27.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.627+0000 7f15e1129b80 1 -- 172.21.15.52:0/1580085710 shutdown_connections 2024-06-10T02:56:27.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.627+0000 7f15e1129b80 1 --2- 172.21.15.52:0/1580085710 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55d07d20a1f0 0x55d07d276bc0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:27.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.627+0000 7f15af7fe700 0 client.15864 ms_handle_reset on v2:172.21.15.188:6838/310444836 2024-06-10T02:56:27.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.627+0000 7f15e1129b80 1 --2- 172.21.15.52:0/1580085710 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d07d1f6770 0x55d07d211000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:27.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.627+0000 7f15e1129b80 1 --2- 172.21.15.52:0/1580085710 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d07d1f0730 0x55d07d202710 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:27.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.627+0000 7f15e1129b80 1 --2- 172.21.15.52:0/1580085710 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d07d1f5e50 0x55d07d202c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:27.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.627+0000 7f15e1129b80 1 -- 172.21.15.52:0/1580085710 >> 172.21.15.52:0/1580085710 conn(0x55d07d112cc0 msgr2=0x55d07d1dbe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:56:27.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.627+0000 7f15e1129b80 1 -- 172.21.15.52:0/1580085710 shutdown_connections 2024-06-10T02:56:27.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.627+0000 7f15e1129b80 1 -- 172.21.15.52:0/1580085710 wait complete. 2024-06-10T02:56:27.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.627+0000 7f15e1129b80 20 client.15864 trim_cache size 0 max 16384 2024-06-10T02:56:27.631 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-06-10T02:56:27.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.628+0000 7f15d68a6700 1 -- 172.21.15.52:0/1441275557 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f15b80a2d70 msgr2=0x7f15b80a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:56:27.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.628+0000 7f15d68a6700 1 --2- 172.21.15.52:0/1441275557 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f15b80a2d70 0x7f15b80a5230 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f15d0107bd0 tx=0x7f15c000a380 comp rx=0 tx=0).stop 2024-06-10T02:56:27.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.628+0000 7f15d68a6700 1 -- 172.21.15.52:0/1441275557 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f15d0101d40 msgr2=0x7f15d019dc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:56:27.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.628+0000 7f15d68a6700 1 --2- 172.21.15.52:0/1441275557 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f15d0101d40 0x7f15d019dc70 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f15c4006010 tx=0x7f15c40038b0 comp rx=0 tx=0).stop 2024-06-10T02:56:27.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.628+0000 7f15d68a6700 1 -- 172.21.15.52:0/1441275557 shutdown_connections 2024-06-10T02:56:27.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.628+0000 7f15d68a6700 1 --2- 172.21.15.52:0/1441275557 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f15b80a2d70 0x7f15b80a5230 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:27.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.628+0000 7f15d68a6700 1 --2- 172.21.15.52:0/1441275557 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f15d01073f0 0x7f15d0197940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:27.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.628+0000 7f15d68a6700 1 --2- 172.21.15.52:0/1441275557 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f15d0101380 0x7f15d019d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:27.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.628+0000 7f15d68a6700 1 --2- 172.21.15.52:0/1441275557 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f15d0101d40 0x7f15d019dc70 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:27.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.628+0000 7f15d68a6700 1 -- 172.21.15.52:0/1441275557 >> 172.21.15.52:0/1441275557 conn(0x7f15d0078300 msgr2=0x7f15d00ffd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:56:27.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.628+0000 7f15d68a6700 1 -- 172.21.15.52:0/1441275557 shutdown_connections 2024-06-10T02:56:27.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:27.628+0000 7f15d68a6700 1 -- 172.21.15.52:0/1441275557 wait complete. 2024-06-10T02:56:28.726 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:56:28 smithi052 ceph-mon[32362]: pgmap v502: 129 pgs: 129 active+clean; 3.5 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 2.0 KiB/s wr, 12 op/s 2024-06-10T02:56:28.888 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:56:28 smithi120 ceph-mon[36634]: pgmap v502: 129 pgs: 129 active+clean; 3.5 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 2.0 KiB/s wr, 12 op/s 2024-06-10T02:56:28.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:56:28 smithi188 ceph-mon[36599]: pgmap v502: 129 pgs: 129 active+clean; 3.5 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 2.0 KiB/s wr, 12 op/s 2024-06-10T02:56:30.123 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-06-10T02:56:30.123 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-06-10T02:56:30.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.194+0000 7f1675c1c700 1 -- 172.21.15.52:0/2817196275 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f16701030f0 msgr2=0x7f1670107110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:56:30.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.194+0000 7f1675c1c700 1 --2- 172.21.15.52:0/2817196275 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f16701030f0 0x7f1670107110 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f16580099c0 tx=0x7f1658009cd0 comp rx=0 tx=0).stop 2024-06-10T02:56:30.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.194+0000 7f1675c1c700 1 -- 172.21.15.52:0/2817196275 shutdown_connections 2024-06-10T02:56:30.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.194+0000 7f1675c1c700 1 --2- 172.21.15.52:0/2817196275 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1670107840 0x7f167010f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:30.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.194+0000 7f1675c1c700 1 --2- 172.21.15.52:0/2817196275 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1670102740 0x7f1670102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:30.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.194+0000 7f1675c1c700 1 --2- 172.21.15.52:0/2817196275 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f16701030f0 0x7f1670107110 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:30.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.194+0000 7f1675c1c700 1 -- 172.21.15.52:0/2817196275 >> 172.21.15.52:0/2817196275 conn(0x7f16700795f0 msgr2=0x7f1670079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:56:30.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.194+0000 7f1675c1c700 1 -- 172.21.15.52:0/2817196275 shutdown_connections 2024-06-10T02:56:30.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.194+0000 7f1675c1c700 1 -- 172.21.15.52:0/2817196275 wait complete. 2024-06-10T02:56:30.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.195+0000 7f1675c1c700 1 Processor -- start 2024-06-10T02:56:30.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.195+0000 7f1675c1c700 1 -- start start 2024-06-10T02:56:30.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.195+0000 7f1675c1c700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1670102740 0x7f167019e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:56:30.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.195+0000 7f1675c1c700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f16701030f0 0x7f167019ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:56:30.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.195+0000 7f1675c1c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1670107840 0x7f16701989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:56:30.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.195+0000 7f1675c1c700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f1670112770 con 0x7f1670107840 2024-06-10T02:56:30.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.195+0000 7f1675c1c700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f16701125f0 con 0x7f1670102740 2024-06-10T02:56:30.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.195+0000 7f1675c1c700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f16701128f0 con 0x7f16701030f0 2024-06-10T02:56:30.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.196+0000 7f166ffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1670107840 0x7f16701989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:56:30.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.196+0000 7f1666ffd700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f16701030f0 0x7f167019ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:56:30.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.196+0000 7f166f7fe700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1670102740 0x7f167019e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:56:30.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.196+0000 7f166ffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1670107840 0x7f16701989f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40362/0 (socket says 172.21.15.52:40362) 2024-06-10T02:56:30.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.196+0000 7f166ffff700 1 -- 172.21.15.52:0/3750287905 learned_addr learned my addr 172.21.15.52:0/3750287905 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:56:30.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.196+0000 7f166ffff700 1 -- 172.21.15.52:0/3750287905 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f16701030f0 msgr2=0x7f167019ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:56:30.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.196+0000 7f166ffff700 1 --2- 172.21.15.52:0/3750287905 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f16701030f0 0x7f167019ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:30.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.196+0000 7f166ffff700 1 -- 172.21.15.52:0/3750287905 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1670102740 msgr2=0x7f167019e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:56:30.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.196+0000 7f166ffff700 1 --2- 172.21.15.52:0/3750287905 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1670102740 0x7f167019e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:30.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.196+0000 7f166ffff700 1 -- 172.21.15.52:0/3750287905 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1670199150 con 0x7f1670107840 2024-06-10T02:56:30.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.196+0000 7f1666ffd700 1 --2- 172.21.15.52:0/3750287905 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f16701030f0 0x7f167019ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T02:56:30.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.196+0000 7f166ffff700 1 --2- 172.21.15.52:0/3750287905 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1670107840 0x7f16701989f0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f1668007ee0 tx=0x7f166800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:56:30.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.196+0000 7f166d7fa700 1 -- 172.21.15.52:0/3750287905 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1668004bb0 con 0x7f1670107840 2024-06-10T02:56:30.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.197+0000 7f166d7fa700 1 -- 172.21.15.52:0/3750287905 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1668004d50 con 0x7f1670107840 2024-06-10T02:56:30.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.197+0000 7f1675c1c700 1 -- 172.21.15.52:0/3750287905 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1670199440 con 0x7f1670107840 2024-06-10T02:56:30.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.197+0000 7f166d7fa700 1 -- 172.21.15.52:0/3750287905 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1668005960 con 0x7f1670107840 2024-06-10T02:56:30.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.197+0000 7f1675c1c700 1 -- 172.21.15.52:0/3750287905 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16701a5700 con 0x7f1670107840 2024-06-10T02:56:30.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.200+0000 7f166d7fa700 1 -- 172.21.15.52:0/3750287905 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f166801a020 con 0x7f1670107840 2024-06-10T02:56:30.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.201+0000 7f166d7fa700 1 --2- 172.21.15.52:0/3750287905 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f165c0a2ca0 0x7f165c0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:56:30.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.201+0000 7f168049fb80 1 Processor -- start 2024-06-10T02:56:30.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.201+0000 7f168049fb80 1 -- start start 2024-06-10T02:56:30.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.201+0000 7f166f7fe700 1 --2- 172.21.15.52:0/3750287905 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f165c0a2ca0 0x7f165c0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:56:30.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.201+0000 7f166d7fa700 1 -- 172.21.15.52:0/3750287905 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f16680ceb60 con 0x7f1670107840 2024-06-10T02:56:30.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.201+0000 7f168049fb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a39f68b500 0x55a39f68b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:56:30.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.201+0000 7f168049fb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a39f68be20 0x55a39f685710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:56:30.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.201+0000 7f168049fb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a39f685c50 0x55a39f686070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:56:30.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.201+0000 7f168049fb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55a39f578060 con 0x55a39f68b500 2024-06-10T02:56:30.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.201+0000 7f168049fb80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55a39f5d6950 con 0x55a39f68be20 2024-06-10T02:56:30.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.201+0000 7f168049fb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55a39f53e560 con 0x55a39f685c50 2024-06-10T02:56:30.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.202+0000 7f166f7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a39f68b500 0x55a39f68b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:56:30.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.202+0000 7f166f7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a39f68b500 0x55a39f68b8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40368/0 (socket says 172.21.15.52:40368) 2024-06-10T02:56:30.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.202+0000 7f166f7fe700 1 -- 172.21.15.52:0/1002956423 learned_addr learned my addr 172.21.15.52:0/1002956423 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:56:30.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.202+0000 7f166f7fe700 1 --2- 172.21.15.52:0/3750287905 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f165c0a2ca0 0x7f165c0a5160 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f1670199b30 tx=0x7f1660009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:56:30.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.202+0000 7f1666ffd700 1 --2- 172.21.15.52:0/1002956423 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a39f68be20 0x55a39f685710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:56:30.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.202+0000 7f166ffff700 1 --2- 172.21.15.52:0/1002956423 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a39f685c50 0x55a39f686070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:56:30.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.202+0000 7f166f7fe700 1 -- 172.21.15.52:0/1002956423 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a39f685c50 msgr2=0x55a39f686070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:56:30.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.202+0000 7f166f7fe700 1 --2- 172.21.15.52:0/1002956423 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a39f685c50 0x55a39f686070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:30.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.202+0000 7f166f7fe700 1 -- 172.21.15.52:0/1002956423 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a39f68be20 msgr2=0x55a39f685710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:56:30.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.202+0000 7f166f7fe700 1 --2- 172.21.15.52:0/1002956423 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a39f68be20 0x55a39f685710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:30.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.202+0000 7f166f7fe700 1 -- 172.21.15.52:0/1002956423 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a39f5259d0 con 0x55a39f68b500 2024-06-10T02:56:30.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.202+0000 7f1666ffd700 1 --2- 172.21.15.52:0/1002956423 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a39f68be20 0x55a39f685710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T02:56:30.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.202+0000 7f166f7fe700 1 --2- 172.21.15.52:0/1002956423 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a39f68b500 0x55a39f68b8e0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f1660011d10 tx=0x7f1660014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85a12fda9c16f946 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:56:30.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.202+0000 7f1665ffb700 1 -- 172.21.15.52:0/1002956423 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f166000ebd0 con 0x55a39f68b500 2024-06-10T02:56:30.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.202+0000 7f1665ffb700 1 -- 172.21.15.52:0/1002956423 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f166000ed70 con 0x55a39f68b500 2024-06-10T02:56:30.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.202+0000 7f1665ffb700 1 -- 172.21.15.52:0/1002956423 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16600256f0 con 0x55a39f68b500 2024-06-10T02:56:30.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.203+0000 7f168049fb80 1 -- 172.21.15.52:0/1002956423 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a39f68b500 msgr2=0x55a39f68b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:56:30.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.203+0000 7f168049fb80 1 --2- 172.21.15.52:0/1002956423 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a39f68b500 0x55a39f68b8e0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f1660011d10 tx=0x7f1660014000 comp rx=0 tx=0).stop 2024-06-10T02:56:30.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.203+0000 7f168049fb80 1 -- 172.21.15.52:0/1002956423 shutdown_connections 2024-06-10T02:56:30.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.203+0000 7f168049fb80 1 --2- 172.21.15.52:0/1002956423 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a39f685c50 0x55a39f686070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:30.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.203+0000 7f168049fb80 1 --2- 172.21.15.52:0/1002956423 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a39f68b500 0x55a39f68b8e0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:30.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.203+0000 7f168049fb80 1 --2- 172.21.15.52:0/1002956423 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a39f68be20 0x55a39f685710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:30.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.203+0000 7f168049fb80 1 -- 172.21.15.52:0/1002956423 >> 172.21.15.52:0/1002956423 conn(0x55a39f5b2180 msgr2=0x55a39f680a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:56:30.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.203+0000 7f168049fb80 1 -- 172.21.15.52:0/1002956423 shutdown_connections 2024-06-10T02:56:30.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.203+0000 7f168049fb80 1 -- 172.21.15.52:0/1002956423 wait complete. 2024-06-10T02:56:30.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.204+0000 7f168049fb80 1 -- start start 2024-06-10T02:56:30.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.204+0000 7f168049fb80 1 Processor -- start 2024-06-10T02:56:30.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.204+0000 7f168049fb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a39f685c50 0x55a39f691b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:56:30.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.204+0000 7f168049fb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a39f68b500 0x55a39f68ee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:56:30.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.204+0000 7f168049fb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a39f68be20 0x55a39f68f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:56:30.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.204+0000 7f168049fb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55a39f5770c0 con 0x55a39f68b500 2024-06-10T02:56:30.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.204+0000 7f168049fb80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55a39f5d2910 con 0x55a39f68be20 2024-06-10T02:56:30.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.204+0000 7f168049fb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55a39f692800 con 0x55a39f685c50 2024-06-10T02:56:30.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.204+0000 7f1666ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a39f68b500 0x55a39f68ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:56:30.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.204+0000 7f168049fb80 10 client.0 _ll_register_callbacks cb 0x55a39f56c6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T02:56:30.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.204+0000 7f1666ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a39f68b500 0x55a39f68ee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40380/0 (socket says 172.21.15.52:40380) 2024-06-10T02:56:30.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.204+0000 7f1666ffd700 1 -- 172.21.15.52:0/3516037172 learned_addr learned my addr 172.21.15.52:0/3516037172 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:56:30.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.204+0000 7f166f7fe700 1 --2- 172.21.15.52:0/3516037172 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a39f685c50 0x55a39f691b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:56:30.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.205+0000 7f166ffff700 1 --2- 172.21.15.52:0/3516037172 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a39f68be20 0x55a39f68f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:56:30.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.205+0000 7f1666ffd700 1 -- 172.21.15.52:0/3516037172 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a39f685c50 msgr2=0x55a39f691b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:56:30.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.205+0000 7f1666ffd700 1 --2- 172.21.15.52:0/3516037172 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a39f685c50 0x55a39f691b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:30.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.205+0000 7f1666ffd700 1 -- 172.21.15.52:0/3516037172 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a39f68be20 msgr2=0x55a39f68f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:56:30.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.205+0000 7f1666ffd700 1 --2- 172.21.15.52:0/3516037172 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a39f68be20 0x55a39f68f3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:30.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.205+0000 7f1666ffd700 1 -- 172.21.15.52:0/3516037172 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a39f59c850 con 0x55a39f68b500 2024-06-10T02:56:30.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.205+0000 7f1666ffd700 1 --2- 172.21.15.52:0/3516037172 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a39f68b500 0x55a39f68ee60 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f16580099c0 tx=0x7f165800b800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a07736f8e399bf1e server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:56:30.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.205+0000 7f164effd700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-10T02:56:30.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.205+0000 7f164effd700 1 -- 172.21.15.52:0/3516037172 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16580052e0 con 0x55a39f68b500 2024-06-10T02:56:30.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.205+0000 7f164effd700 1 -- 172.21.15.52:0/3516037172 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f1658005480 con 0x55a39f68b500 2024-06-10T02:56:30.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.205+0000 7f168049fb80 1 -- 172.21.15.52:0/3516037172 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f166000ed70 con 0x55a39f68b500 2024-06-10T02:56:30.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.205+0000 7f164effd700 1 -- 172.21.15.52:0/3516037172 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f1658033050 con 0x55a39f68b500 2024-06-10T02:56:30.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.205+0000 7f168049fb80 10 client.15876 fetch_fsmap learned FSMap version 31 2024-06-10T02:56:30.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.205+0000 7f168049fb80 1 -- 172.21.15.52:0/3516037172 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55a39f69cf90 con 0x55a39f68b500 2024-06-10T02:56:30.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.206+0000 7f164effd700 1 -- 172.21.15.52:0/3516037172 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f166000ed70 con 0x55a39f68b500 2024-06-10T02:56:30.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.206+0000 7f164effd700 1 -- 172.21.15.52:0/3516037172 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 31) ==== 8366+0+0 (secure 0 0 0) 0x7f1658020820 con 0x55a39f68b500 2024-06-10T02:56:30.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.206+0000 7f168049fb80 10 client.15876 fetch_fsmap finished waiting for FSMap version 31 2024-06-10T02:56:30.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.206+0000 7f168049fb80 10 client.15876 resolve_mds: validated gid 24484 aka daemon mds.i 2024-06-10T02:56:30.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.206+0000 7f168049fb80 20 client.15876 populate_metadata read hostname 'smithi052' 2024-06-10T02:56:30.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.206+0000 7f168049fb80 1 --2- 172.21.15.52:0/3516037172 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55a39f69f5c0 0x55a39f6a19e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:56:30.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.206+0000 7f168049fb80 4 client.15876 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T02:56:30.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.206+0000 7f168049fb80 1 -- 172.21.15.52:0/3516037172 --> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a39f5e3e50 con 0x55a39f69f5c0 2024-06-10T02:56:30.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.206+0000 7f166f7fe700 1 --2- 172.21.15.52:0/3516037172 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55a39f69f5c0 0x55a39f6a19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:56:30.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.207+0000 7f166f7fe700 1 --2- 172.21.15.52:0/3516037172 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55a39f69f5c0 0x55a39f6a19e0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=40e9b539db4c184d server_cookie=baac7664367231a in_seq=0 out_seq=0 2024-06-10T02:56:30.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.207+0000 7f164effd700 10 client.15876 ms_handle_connect on v2:172.21.15.188:6838/310444836 2024-06-10T02:56:30.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.207+0000 7f164effd700 1 -- 172.21.15.52:0/3516037172 <== mds.0 v2:172.21.15.188:6838/310444836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55a39f5e3e50 con 0x55a39f69f5c0 2024-06-10T02:56:30.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.207+0000 7f164effd700 10 client.15876 handle_command_reply: tid=0 2024-06-10T02:56:30.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.208+0000 7f168049fb80 1 client.15876 shutdown 2024-06-10T02:56:30.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.208+0000 7f168049fb80 1 -- 172.21.15.52:0/3516037172 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a39f68b500 msgr2=0x55a39f68ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:56:30.212 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.208+0000 7f168049fb80 1 --2- 172.21.15.52:0/3516037172 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a39f68b500 0x55a39f68ee60 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f16580099c0 tx=0x7f165800b800 comp rx=0 tx=0).stop 2024-06-10T02:56:30.212 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.208+0000 7f168049fb80 1 -- 172.21.15.52:0/3516037172 shutdown_connections 2024-06-10T02:56:30.212 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.208+0000 7f168049fb80 1 --2- 172.21.15.52:0/3516037172 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55a39f69f5c0 0x55a39f6a19e0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:30.212 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.208+0000 7f168049fb80 1 --2- 172.21.15.52:0/3516037172 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a39f68be20 0x55a39f68f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:30.212 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.208+0000 7f164effd700 0 client.15876 ms_handle_reset on v2:172.21.15.188:6838/310444836 2024-06-10T02:56:30.212 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.208+0000 7f168049fb80 1 --2- 172.21.15.52:0/3516037172 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a39f685c50 0x55a39f691b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:30.212 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.208+0000 7f168049fb80 1 --2- 172.21.15.52:0/3516037172 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a39f68b500 0x55a39f68ee60 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:30.212 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.208+0000 7f168049fb80 1 -- 172.21.15.52:0/3516037172 >> 172.21.15.52:0/3516037172 conn(0x55a39f5b2180 msgr2=0x55a39f67fbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:56:30.212 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.208+0000 7f168049fb80 1 -- 172.21.15.52:0/3516037172 shutdown_connections 2024-06-10T02:56:30.212 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.208+0000 7f168049fb80 1 -- 172.21.15.52:0/3516037172 wait complete. 2024-06-10T02:56:30.212 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.208+0000 7f168049fb80 20 client.15876 trim_cache size 0 max 16384 2024-06-10T02:56:30.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.228+0000 7f168049fb80 1 Processor -- start 2024-06-10T02:56:30.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.228+0000 7f168049fb80 1 -- start start 2024-06-10T02:56:30.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.229+0000 7f168049fb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a39f685c50 0x55a39f68dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:56:30.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.229+0000 7f168049fb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a39f68b500 0x55a39f690d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:56:30.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.229+0000 7f168049fb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a39f68be20 0x55a39f680330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:56:30.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.229+0000 7f168049fb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f16701ac160 con 0x55a39f685c50 2024-06-10T02:56:30.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.229+0000 7f168049fb80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f16701adcd0 con 0x55a39f68b500 2024-06-10T02:56:30.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.229+0000 7f168049fb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f16701ac830 con 0x55a39f68be20 2024-06-10T02:56:30.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.229+0000 7f166f7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a39f685c50 0x55a39f68dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:56:30.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.229+0000 7f166f7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a39f685c50 0x55a39f68dfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40392/0 (socket says 172.21.15.52:40392) 2024-06-10T02:56:30.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.229+0000 7f166f7fe700 1 -- 172.21.15.52:0/1856915431 learned_addr learned my addr 172.21.15.52:0/1856915431 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:56:30.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.229+0000 7f166ffff700 1 --2- 172.21.15.52:0/1856915431 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a39f68be20 0x55a39f680330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:56:30.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.229+0000 7f1666ffd700 1 --2- 172.21.15.52:0/1856915431 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a39f68b500 0x55a39f690d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:56:30.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.229+0000 7f166f7fe700 1 -- 172.21.15.52:0/1856915431 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a39f68be20 msgr2=0x55a39f680330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:56:30.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.229+0000 7f166f7fe700 1 --2- 172.21.15.52:0/1856915431 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a39f68be20 0x55a39f680330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:30.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.229+0000 7f166f7fe700 1 -- 172.21.15.52:0/1856915431 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a39f68b500 msgr2=0x55a39f690d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:56:30.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.229+0000 7f166f7fe700 1 --2- 172.21.15.52:0/1856915431 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a39f68b500 0x55a39f690d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:30.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.229+0000 7f166f7fe700 1 -- 172.21.15.52:0/1856915431 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a39f685710 con 0x55a39f685c50 2024-06-10T02:56:30.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.229+0000 7f166ffff700 1 --2- 172.21.15.52:0/1856915431 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a39f68be20 0x55a39f680330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T02:56:30.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.229+0000 7f166f7fe700 1 --2- 172.21.15.52:0/1856915431 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a39f685c50 0x55a39f68dfe0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f1660011d10 tx=0x7f16600058e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca85db0289c6f673 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:56:30.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.230+0000 7f16657fa700 1 -- 172.21.15.52:0/1856915431 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1660036070 con 0x55a39f685c50 2024-06-10T02:56:30.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.230+0000 7f16657fa700 1 -- 172.21.15.52:0/1856915431 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f166000ed10 con 0x55a39f685c50 2024-06-10T02:56:30.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.230+0000 7f16657fa700 1 -- 172.21.15.52:0/1856915431 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f166001d410 con 0x55a39f685c50 2024-06-10T02:56:30.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.230+0000 7f168049fb80 1 -- 172.21.15.52:0/1856915431 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a39f685c50 msgr2=0x55a39f68dfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:56:30.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.230+0000 7f168049fb80 1 --2- 172.21.15.52:0/1856915431 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a39f685c50 0x55a39f68dfe0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f1660011d10 tx=0x7f16600058e0 comp rx=0 tx=0).stop 2024-06-10T02:56:30.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.230+0000 7f168049fb80 1 -- 172.21.15.52:0/1856915431 shutdown_connections 2024-06-10T02:56:30.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.230+0000 7f168049fb80 1 --2- 172.21.15.52:0/1856915431 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a39f68be20 0x55a39f680330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:30.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.230+0000 7f168049fb80 1 --2- 172.21.15.52:0/1856915431 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a39f685c50 0x55a39f68dfe0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:30.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.230+0000 7f168049fb80 1 --2- 172.21.15.52:0/1856915431 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a39f68b500 0x55a39f690d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:30.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.230+0000 7f168049fb80 1 -- 172.21.15.52:0/1856915431 >> 172.21.15.52:0/1856915431 conn(0x55a39f5b2180 msgr2=0x55a39f690640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:56:30.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.231+0000 7f168049fb80 1 -- 172.21.15.52:0/1856915431 shutdown_connections 2024-06-10T02:56:30.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.231+0000 7f168049fb80 1 -- 172.21.15.52:0/1856915431 wait complete. 2024-06-10T02:56:30.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.231+0000 7f168049fb80 1 -- start start 2024-06-10T02:56:30.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.231+0000 7f168049fb80 1 Processor -- start 2024-06-10T02:56:30.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.231+0000 7f168049fb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a39f685c50 0x55a39f697fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:56:30.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.231+0000 7f168049fb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a39f68b500 0x55a39f694ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:56:30.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.231+0000 7f168049fb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a39f68be20 0x55a39f695000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:56:30.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.231+0000 7f168049fb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f16701ad5d0 con 0x55a39f68be20 2024-06-10T02:56:30.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.231+0000 7f168049fb80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f16701acf00 con 0x55a39f68b500 2024-06-10T02:56:30.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.231+0000 7f168049fb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f16701aa120 con 0x55a39f685c50 2024-06-10T02:56:30.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.231+0000 7f168049fb80 10 client.0 _ll_register_callbacks cb 0x55a39f56c6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T02:56:30.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.232+0000 7f166ffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a39f68be20 0x55a39f695000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:56:30.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.232+0000 7f1666ffd700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a39f68b500 0x55a39f694ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:56:30.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.232+0000 7f166ffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a39f68be20 0x55a39f695000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40402/0 (socket says 172.21.15.52:40402) 2024-06-10T02:56:30.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.232+0000 7f1666ffd700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a39f68b500 0x55a39f694ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.52:46866/0 (socket says 172.21.15.52:46866) 2024-06-10T02:56:30.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.232+0000 7f166ffff700 1 -- 172.21.15.52:0/2139005520 learned_addr learned my addr 172.21.15.52:0/2139005520 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:56:30.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.232+0000 7f166f7fe700 1 --2- 172.21.15.52:0/2139005520 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a39f685c50 0x55a39f697fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:56:30.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.232+0000 7f166ffff700 1 -- 172.21.15.52:0/2139005520 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a39f685c50 msgr2=0x55a39f697fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:56:30.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.232+0000 7f166ffff700 1 --2- 172.21.15.52:0/2139005520 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a39f685c50 0x55a39f697fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:30.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.232+0000 7f166ffff700 1 -- 172.21.15.52:0/2139005520 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a39f68b500 msgr2=0x55a39f694ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:56:30.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.232+0000 7f166ffff700 1 --2- 172.21.15.52:0/2139005520 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a39f68b500 0x55a39f694ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:30.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.232+0000 7f166ffff700 1 -- 172.21.15.52:0/2139005520 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a39f695540 con 0x55a39f68be20 2024-06-10T02:56:30.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.232+0000 7f1666ffd700 1 --2- 172.21.15.52:0/2139005520 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a39f68b500 0x55a39f694ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T02:56:30.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.232+0000 7f166ffff700 1 --2- 172.21.15.52:0/2139005520 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a39f68be20 0x55a39f695000 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f16680cc250 tx=0x7f16680bb040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ace916a541f9360 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:56:30.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.232+0000 7f164effd700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-10T02:56:30.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.232+0000 7f164effd700 1 -- 172.21.15.52:0/2139005520 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1668005100 con 0x55a39f68be20 2024-06-10T02:56:30.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.232+0000 7f164effd700 1 -- 172.21.15.52:0/2139005520 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f166809b210 con 0x55a39f68be20 2024-06-10T02:56:30.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.232+0000 7f168049fb80 1 -- 172.21.15.52:0/2139005520 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f166000ed10 con 0x55a39f68be20 2024-06-10T02:56:30.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.233+0000 7f164effd700 1 -- 172.21.15.52:0/2139005520 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f166809b6b0 con 0x55a39f68be20 2024-06-10T02:56:30.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.233+0000 7f168049fb80 10 client.15882 fetch_fsmap learned FSMap version 31 2024-06-10T02:56:30.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.233+0000 7f168049fb80 1 -- 172.21.15.52:0/2139005520 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55a39f70afe0 con 0x55a39f68be20 2024-06-10T02:56:30.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.233+0000 7f164effd700 1 -- 172.21.15.52:0/2139005520 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f166000ed10 con 0x55a39f68be20 2024-06-10T02:56:30.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.233+0000 7f164effd700 1 -- 172.21.15.52:0/2139005520 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 31) ==== 8366+0+0 (secure 0 0 0) 0x7f16680a3790 con 0x55a39f68be20 2024-06-10T02:56:30.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.233+0000 7f168049fb80 10 client.15882 fetch_fsmap finished waiting for FSMap version 31 2024-06-10T02:56:30.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.233+0000 7f168049fb80 10 client.15882 resolve_mds: validated gid 24484 aka daemon mds.i 2024-06-10T02:56:30.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.233+0000 7f168049fb80 20 client.15882 populate_metadata read hostname 'smithi052' 2024-06-10T02:56:30.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.233+0000 7f168049fb80 1 --2- 172.21.15.52:0/2139005520 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55a39f69f5c0 0x55a39f70d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:56:30.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.233+0000 7f168049fb80 4 client.15882 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "cdf1e680", "release": true, "await": true}] 2024-06-10T02:56:30.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.233+0000 7f168049fb80 1 -- 172.21.15.52:0/2139005520 --> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] -- command(tid 0: {"prefix": "quiesce db", "set_id": "cdf1e680", "release": true, "await": true}) -- 0x55a39f5de3d0 con 0x55a39f69f5c0 2024-06-10T02:56:30.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.233+0000 7f166f7fe700 1 --2- 172.21.15.52:0/2139005520 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55a39f69f5c0 0x55a39f70d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:56:30.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.234+0000 7f166f7fe700 1 --2- 172.21.15.52:0/2139005520 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55a39f69f5c0 0x55a39f70d7e0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=959ca12fe5c48ce2 server_cookie=24d23758b716f226 in_seq=0 out_seq=0 2024-06-10T02:56:30.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.234+0000 7f164effd700 10 client.15882 ms_handle_connect on v2:172.21.15.188:6838/310444836 2024-06-10T02:56:30.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.240+0000 7f164effd700 1 -- 172.21.15.52:0/2139005520 <== mds.0 v2:172.21.15.188:6838/310444836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x55a39f5de3d0 con 0x55a39f69f5c0 2024-06-10T02:56:30.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.240+0000 7f164effd700 10 client.15882 handle_command_reply: tid=0 2024-06-10T02:56:30.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.240+0000 7f168049fb80 1 client.15882 shutdown 2024-06-10T02:56:30.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.240+0000 7f168049fb80 1 -- 172.21.15.52:0/2139005520 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a39f68be20 msgr2=0x55a39f695000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:56:30.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.240+0000 7f168049fb80 1 --2- 172.21.15.52:0/2139005520 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a39f68be20 0x55a39f695000 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f16680cc250 tx=0x7f16680bb040 comp rx=0 tx=0).stop 2024-06-10T02:56:30.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.240+0000 7f168049fb80 1 -- 172.21.15.52:0/2139005520 shutdown_connections 2024-06-10T02:56:30.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.240+0000 7f168049fb80 1 --2- 172.21.15.52:0/2139005520 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55a39f69f5c0 0x55a39f70d7e0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:30.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.240+0000 7f168049fb80 1 --2- 172.21.15.52:0/2139005520 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a39f68be20 0x55a39f695000 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:30.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.240+0000 7f168049fb80 1 --2- 172.21.15.52:0/2139005520 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a39f685c50 0x55a39f697fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:30.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.240+0000 7f168049fb80 1 --2- 172.21.15.52:0/2139005520 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a39f68b500 0x55a39f694ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:30.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.240+0000 7f168049fb80 1 -- 172.21.15.52:0/2139005520 >> 172.21.15.52:0/2139005520 conn(0x55a39f5b2180 msgr2=0x55a39f684f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:56:30.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.240+0000 7f164effd700 0 client.15882 ms_handle_reset on v2:172.21.15.188:6838/310444836 2024-06-10T02:56:30.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.240+0000 7f168049fb80 1 -- 172.21.15.52:0/2139005520 shutdown_connections 2024-06-10T02:56:30.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.240+0000 7f168049fb80 1 -- 172.21.15.52:0/2139005520 wait complete. 2024-06-10T02:56:30.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.240+0000 7f168049fb80 20 client.15882 trim_cache size 0 max 16384 2024-06-10T02:56:30.241 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-10T02:56:30.241 INFO:teuthology.orchestra.run.smithi052.stdout: "epoch": 31, 2024-06-10T02:56:30.241 INFO:teuthology.orchestra.run.smithi052.stdout: "leader": 24484, 2024-06-10T02:56:30.241 INFO:teuthology.orchestra.run.smithi052.stdout: "set_version": 8, 2024-06-10T02:56:30.241 INFO:teuthology.orchestra.run.smithi052.stdout: "sets": { 2024-06-10T02:56:30.241 INFO:teuthology.orchestra.run.smithi052.stdout: "cdf1e680": { 2024-06-10T02:56:30.241 INFO:teuthology.orchestra.run.smithi052.stdout: "version": 8, 2024-06-10T02:56:30.241 INFO:teuthology.orchestra.run.smithi052.stdout: "age_ref": 0.0, 2024-06-10T02:56:30.241 INFO:teuthology.orchestra.run.smithi052.stdout: "state": { 2024-06-10T02:56:30.241 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "RELEASED", 2024-06-10T02:56:30.242 INFO:teuthology.orchestra.run.smithi052.stdout: "age": 0.0 2024-06-10T02:56:30.242 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T02:56:30.242 INFO:teuthology.orchestra.run.smithi052.stdout: "timeout": 90.0, 2024-06-10T02:56:30.242 INFO:teuthology.orchestra.run.smithi052.stdout: "expiration": 150.0, 2024-06-10T02:56:30.242 INFO:teuthology.orchestra.run.smithi052.stdout: "members": { 2024-06-10T02:56:30.242 INFO:teuthology.orchestra.run.smithi052.stdout: "file:/": { 2024-06-10T02:56:30.242 INFO:teuthology.orchestra.run.smithi052.stdout: "excluded": false, 2024-06-10T02:56:30.242 INFO:teuthology.orchestra.run.smithi052.stdout: "state": { 2024-06-10T02:56:30.242 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "RELEASED", 2024-06-10T02:56:30.242 INFO:teuthology.orchestra.run.smithi052.stdout: "age": 0.0 2024-06-10T02:56:30.242 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T02:56:30.242 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T02:56:30.242 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T02:56:30.243 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T02:56:30.243 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T02:56:30.243 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-10T02:56:30.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.241+0000 7f1675c1c700 1 -- 172.21.15.52:0/3750287905 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f165c0a2ca0 msgr2=0x7f165c0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:56:30.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.241+0000 7f1675c1c700 1 --2- 172.21.15.52:0/3750287905 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f165c0a2ca0 0x7f165c0a5160 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f1670199b30 tx=0x7f1660009260 comp rx=0 tx=0).stop 2024-06-10T02:56:30.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.241+0000 7f1675c1c700 1 -- 172.21.15.52:0/3750287905 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1670107840 msgr2=0x7f16701989f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:56:30.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.241+0000 7f1675c1c700 1 --2- 172.21.15.52:0/3750287905 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1670107840 0x7f16701989f0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f1668007ee0 tx=0x7f166800d3b0 comp rx=0 tx=0).stop 2024-06-10T02:56:30.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.241+0000 7f1675c1c700 1 -- 172.21.15.52:0/3750287905 shutdown_connections 2024-06-10T02:56:30.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.241+0000 7f1675c1c700 1 --2- 172.21.15.52:0/3750287905 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f165c0a2ca0 0x7f165c0a5160 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:30.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.241+0000 7f1675c1c700 1 --2- 172.21.15.52:0/3750287905 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1670107840 0x7f16701989f0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:30.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.241+0000 7f1675c1c700 1 --2- 172.21.15.52:0/3750287905 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1670102740 0x7f167019e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:30.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.241+0000 7f1675c1c700 1 --2- 172.21.15.52:0/3750287905 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f16701030f0 0x7f167019ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:30.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.241+0000 7f1675c1c700 1 -- 172.21.15.52:0/3750287905 >> 172.21.15.52:0/3750287905 conn(0x7f16700795f0 msgr2=0x7f1670100810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:56:30.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.241+0000 7f1675c1c700 1 -- 172.21.15.52:0/3750287905 shutdown_connections 2024-06-10T02:56:30.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:30.241+0000 7f1675c1c700 1 -- 172.21.15.52:0/3750287905 wait complete. 2024-06-10T02:56:30.272 INFO:tasks.workunit.client.0.smithi052.stdout:checking for joinable pthread attribute... PTHREAD_CREATE_JOINABLE 2024-06-10T02:56:30.272 INFO:tasks.workunit.client.0.smithi052.stdout:checking if more special flags are required for pthreads... no 2024-06-10T02:56:30.273 INFO:tasks.workunit.client.0.smithi052.stdout:checking for cc_r... gcc 2024-06-10T02:56:30.352 INFO:tasks.workunit.client.0.smithi052.stdout:checking for setrlimit... yes 2024-06-10T02:56:30.417 INFO:tasks.workunit.client.0.smithi052.stdout:checking for setproctitle... no 2024-06-10T02:56:30.478 INFO:tasks.workunit.client.0.smithi052.stdout:checking for getopt_long... yes 2024-06-10T02:56:30.544 INFO:tasks.workunit.client.0.smithi052.stdout:checking for strtoull... yes 2024-06-10T02:56:30.609 INFO:tasks.workunit.client.0.smithi052.stdout:checking for strtoq... yes 2024-06-10T02:56:30.620 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:56:30.682 INFO:tasks.workunit.client.0.smithi052.stdout:checking for snprintf... yes 2024-06-10T02:56:30.746 INFO:tasks.workunit.client.0.smithi052.stdout:checking for vsnprintf... yes 2024-06-10T02:56:30.810 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:56:30 smithi052 ceph-mon[32362]: pgmap v503: 129 pgs: 129 active+clean; 3.5 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 767 B/s wr, 9 op/s 2024-06-10T02:56:30.814 INFO:tasks.workunit.client.0.smithi052.stdout:checking for vfprintf... yes 2024-06-10T02:56:30.879 INFO:tasks.workunit.client.0.smithi052.stdout:checking for gethostname... yes 2024-06-10T02:56:30.888 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:56:30 smithi120 ceph-mon[36634]: pgmap v503: 129 pgs: 129 active+clean; 3.5 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 767 B/s wr, 9 op/s 2024-06-10T02:56:30.941 INFO:tasks.workunit.client.0.smithi052.stdout:checking for setlocale... yes 2024-06-10T02:56:30.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:56:30 smithi188 ceph-mon[36599]: pgmap v503: 129 pgs: 129 active+clean; 3.5 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 767 B/s wr, 9 op/s 2024-06-10T02:56:31.006 INFO:tasks.workunit.client.0.smithi052.stdout:checking for sync... yes 2024-06-10T02:56:31.075 INFO:tasks.workunit.client.0.smithi052.stdout:checking whether snprintf is C99 conformant... done 2024-06-10T02:56:31.086 INFO:tasks.workunit.client.0.smithi052.stdout:configure: creating ./config.status 2024-06-10T02:56:31.201 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'cdf1e680', total seconds elapsed: 39.8 2024-06-10T02:56:31.255 INFO:tasks.workunit.client.0.smithi052.stdout:config.status: creating Makefile 2024-06-10T02:56:31.275 INFO:tasks.workunit.client.0.smithi052.stdout:config.status: creating src/Makefile 2024-06-10T02:56:32.660 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:56:32 smithi052 ceph-mon[32362]: pgmap v504: 129 pgs: 129 active+clean; 3.5 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 853 B/s wr, 9 op/s 2024-06-10T02:56:32.888 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:56:32 smithi120 ceph-mon[36634]: pgmap v504: 129 pgs: 129 active+clean; 3.5 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 853 B/s wr, 9 op/s 2024-06-10T02:56:32.914 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.914+0000 7faab3271700 1 -- 172.21.15.52:0/3298583726 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faaac06abc0 msgr2=0x7faaac06afa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:56:32.914 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.914+0000 7faab3271700 1 --2- 172.21.15.52:0/3298583726 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faaac06abc0 0x7faaac06afa0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7faaa8009990 tx=0x7faaa8009ca0 comp rx=0 tx=0).stop 2024-06-10T02:56:32.914 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.914+0000 7faab3271700 1 -- 172.21.15.52:0/3298583726 shutdown_connections 2024-06-10T02:56:32.914 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.914+0000 7faab3271700 1 --2- 172.21.15.52:0/3298583726 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faaac10ef70 0x7faaac111360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:32.914 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.914+0000 7faab3271700 1 --2- 172.21.15.52:0/3298583726 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faaac06abc0 0x7faaac06afa0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:32.914 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.914+0000 7faab3271700 1 --2- 172.21.15.52:0/3298583726 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faaac06b4e0 0x7faaac10ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:32.914 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.914+0000 7faab3271700 1 -- 172.21.15.52:0/3298583726 >> 172.21.15.52:0/3298583726 conn(0x7faaac078340 msgr2=0x7faaac078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:56:32.914 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.914+0000 7faab3271700 1 -- 172.21.15.52:0/3298583726 shutdown_connections 2024-06-10T02:56:32.914 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.914+0000 7faab3271700 1 -- 172.21.15.52:0/3298583726 wait complete. 2024-06-10T02:56:32.916 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.915+0000 7faab3271700 1 Processor -- start 2024-06-10T02:56:32.916 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.915+0000 7faab3271700 1 -- start start 2024-06-10T02:56:32.916 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.915+0000 7faab3271700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faaac06abc0 0x7faaac19aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:56:32.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.915+0000 7faab3271700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faaac06b4e0 0x7faaac19b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:56:32.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.915+0000 7faab3271700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faaac10ef70 0x7faaac19f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:56:32.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.915+0000 7faab3271700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7faaac1149b0 con 0x7faaac10ef70 2024-06-10T02:56:32.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.915+0000 7faab3271700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7faaac114830 con 0x7faaac06b4e0 2024-06-10T02:56:32.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.915+0000 7faab3271700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7faaac114b30 con 0x7faaac06abc0 2024-06-10T02:56:32.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.915+0000 7faab180e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faaac10ef70 0x7faaac19f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:56:32.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.915+0000 7faab180e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faaac10ef70 0x7faaac19f770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54646/0 (socket says 172.21.15.52:54646) 2024-06-10T02:56:32.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.915+0000 7faab100d700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faaac06abc0 0x7faaac19aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:56:32.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.915+0000 7faab180e700 1 -- 172.21.15.52:0/3875579289 learned_addr learned my addr 172.21.15.52:0/3875579289 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:56:32.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.916+0000 7faab080c700 1 --2- 172.21.15.52:0/3875579289 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faaac06b4e0 0x7faaac19b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:56:32.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.916+0000 7faab180e700 1 -- 172.21.15.52:0/3875579289 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faaac06abc0 msgr2=0x7faaac19aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:56:32.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.916+0000 7faab180e700 1 --2- 172.21.15.52:0/3875579289 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faaac06abc0 0x7faaac19aea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:32.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.916+0000 7faab180e700 1 -- 172.21.15.52:0/3875579289 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faaac06b4e0 msgr2=0x7faaac19b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:56:32.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.916+0000 7faab180e700 1 --2- 172.21.15.52:0/3875579289 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faaac06b4e0 0x7faaac19b3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:32.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.916+0000 7faab180e700 1 -- 172.21.15.52:0/3875579289 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faaac19feb0 con 0x7faaac10ef70 2024-06-10T02:56:32.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.916+0000 7faab080c700 1 --2- 172.21.15.52:0/3875579289 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faaac06b4e0 0x7faaac19b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T02:56:32.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.916+0000 7faab180e700 1 --2- 172.21.15.52:0/3875579289 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faaac10ef70 0x7faaac19f770 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7faaa400ada0 tx=0x7faaa400c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:56:32.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.916+0000 7faaa27fc700 1 -- 172.21.15.52:0/3875579289 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaa4017020 con 0x7faaac10ef70 2024-06-10T02:56:32.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.916+0000 7faaa27fc700 1 -- 172.21.15.52:0/3875579289 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faaa400a0d0 con 0x7faaac10ef70 2024-06-10T02:56:32.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.916+0000 7faaa27fc700 1 -- 172.21.15.52:0/3875579289 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaa4005440 con 0x7faaac10ef70 2024-06-10T02:56:32.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.916+0000 7faab3271700 1 -- 172.21.15.52:0/3875579289 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faaac1a01a0 con 0x7faaac10ef70 2024-06-10T02:56:32.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.916+0000 7faab3271700 1 -- 172.21.15.52:0/3875579289 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faaac1a7a20 con 0x7faaac10ef70 2024-06-10T02:56:32.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.919+0000 7faaa27fc700 1 -- 172.21.15.52:0/3875579289 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7faaa40055e0 con 0x7faaac10ef70 2024-06-10T02:56:32.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.919+0000 7faaa27fc700 1 --2- 172.21.15.52:0/3875579289 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7faa980a2c20 0x7faa980a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:56:32.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.920+0000 7faab100d700 1 --2- 172.21.15.52:0/3875579289 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7faa980a2c20 0x7faa980a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:56:32.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.920+0000 7faaa27fc700 1 -- 172.21.15.52:0/3875579289 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7faaa40cea10 con 0x7faaac10ef70 2024-06-10T02:56:32.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.920+0000 7faab100d700 1 --2- 172.21.15.52:0/3875579289 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7faa980a2c20 0x7faa980a50e0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7faaa800b5c0 tx=0x7faaa8005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:56:32.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.920+0000 7faabdaf4b80 1 Processor -- start 2024-06-10T02:56:32.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.920+0000 7faabdaf4b80 1 -- start start 2024-06-10T02:56:32.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.921+0000 7faabdaf4b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56405270afe0 0x56405270b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:56:32.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.921+0000 7faabdaf4b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56405270b900 0x5640527052e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:56:32.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.921+0000 7faabdaf4b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564052705820 0x564052705c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:56:32.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.921+0000 7faabdaf4b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5640525f7fe0 con 0x56405270b900 2024-06-10T02:56:32.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.921+0000 7faabdaf4b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5640526568d0 con 0x56405270afe0 2024-06-10T02:56:32.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.921+0000 7faabdaf4b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5640525be4e0 con 0x564052705820 2024-06-10T02:56:32.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.921+0000 7faab080c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56405270b900 0x5640527052e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:56:32.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.921+0000 7faab080c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56405270b900 0x5640527052e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54650/0 (socket says 172.21.15.52:54650) 2024-06-10T02:56:32.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.921+0000 7faab080c700 1 -- 172.21.15.52:0/4066259257 learned_addr learned my addr 172.21.15.52:0/4066259257 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:56:32.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.921+0000 7faab100d700 1 --2- 172.21.15.52:0/4066259257 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56405270afe0 0x56405270b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:56:32.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.921+0000 7faab180e700 1 --2- 172.21.15.52:0/4066259257 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564052705820 0x564052705c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:56:32.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.921+0000 7faab080c700 1 -- 172.21.15.52:0/4066259257 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564052705820 msgr2=0x564052705c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:56:32.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.921+0000 7faab080c700 1 --2- 172.21.15.52:0/4066259257 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564052705820 0x564052705c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:32.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.921+0000 7faab080c700 1 -- 172.21.15.52:0/4066259257 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56405270afe0 msgr2=0x56405270b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:56:32.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.921+0000 7faab080c700 1 --2- 172.21.15.52:0/4066259257 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56405270afe0 0x56405270b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:32.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.921+0000 7faab080c700 1 -- 172.21.15.52:0/4066259257 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5640525a5950 con 0x56405270b900 2024-06-10T02:56:32.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.921+0000 7faab100d700 1 --2- 172.21.15.52:0/4066259257 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56405270afe0 0x56405270b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T02:56:32.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.921+0000 7faab080c700 1 --2- 172.21.15.52:0/4066259257 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56405270b900 0x5640527052e0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7faa9c009ca0 tx=0x7faa9c00e2f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=936fdd7d5cafa18e server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:56:32.923 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.922+0000 7faa977fe700 1 -- 172.21.15.52:0/4066259257 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa9c016070 con 0x56405270b900 2024-06-10T02:56:32.923 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.922+0000 7faa977fe700 1 -- 172.21.15.52:0/4066259257 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa9c004d10 con 0x56405270b900 2024-06-10T02:56:32.923 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.922+0000 7faa977fe700 1 -- 172.21.15.52:0/4066259257 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa9c011470 con 0x56405270b900 2024-06-10T02:56:32.925 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.922+0000 7faabdaf4b80 1 -- 172.21.15.52:0/4066259257 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56405270b900 msgr2=0x5640527052e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:56:32.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.922+0000 7faabdaf4b80 1 --2- 172.21.15.52:0/4066259257 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56405270b900 0x5640527052e0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7faa9c009ca0 tx=0x7faa9c00e2f0 comp rx=0 tx=0).stop 2024-06-10T02:56:32.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.922+0000 7faabdaf4b80 1 -- 172.21.15.52:0/4066259257 shutdown_connections 2024-06-10T02:56:32.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.922+0000 7faabdaf4b80 1 --2- 172.21.15.52:0/4066259257 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564052705820 0x564052705c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:32.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.922+0000 7faabdaf4b80 1 --2- 172.21.15.52:0/4066259257 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56405270afe0 0x56405270b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:32.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.922+0000 7faabdaf4b80 1 --2- 172.21.15.52:0/4066259257 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56405270b900 0x5640527052e0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:32.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.922+0000 7faabdaf4b80 1 -- 172.21.15.52:0/4066259257 >> 172.21.15.52:0/4066259257 conn(0x564052632100 msgr2=0x5640527002a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:56:32.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.922+0000 7faabdaf4b80 1 -- 172.21.15.52:0/4066259257 shutdown_connections 2024-06-10T02:56:32.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.922+0000 7faabdaf4b80 1 -- 172.21.15.52:0/4066259257 wait complete. 2024-06-10T02:56:32.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.923+0000 7faabdaf4b80 1 -- start start 2024-06-10T02:56:32.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.923+0000 7faabdaf4b80 1 Processor -- start 2024-06-10T02:56:32.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.923+0000 7faabdaf4b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564052705820 0x564052711670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:56:32.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.923+0000 7faabdaf4b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56405270afe0 0x56405270e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:56:32.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.923+0000 7faabdaf4b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56405270b900 0x56405270eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:56:32.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.923+0000 7faabdaf4b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5640525f7040 con 0x56405270afe0 2024-06-10T02:56:32.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.923+0000 7faabdaf4b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x564052652890 con 0x56405270b900 2024-06-10T02:56:32.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.923+0000 7faabdaf4b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5640527122e0 con 0x564052705820 2024-06-10T02:56:32.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.923+0000 7faabdaf4b80 10 client.0 _ll_register_callbacks cb 0x5640525ec630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T02:56:32.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.923+0000 7faab080c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56405270afe0 0x56405270e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:56:32.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.924+0000 7faab080c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56405270afe0 0x56405270e970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54656/0 (socket says 172.21.15.52:54656) 2024-06-10T02:56:32.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.924+0000 7faab080c700 1 -- 172.21.15.52:0/3703945789 learned_addr learned my addr 172.21.15.52:0/3703945789 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:56:32.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.924+0000 7faab100d700 1 --2- 172.21.15.52:0/3703945789 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564052705820 0x564052711670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:56:32.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.924+0000 7faab180e700 1 --2- 172.21.15.52:0/3703945789 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56405270b900 0x56405270eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:56:32.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.924+0000 7faab080c700 1 -- 172.21.15.52:0/3703945789 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564052705820 msgr2=0x564052711670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:56:32.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.924+0000 7faab080c700 1 --2- 172.21.15.52:0/3703945789 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564052705820 0x564052711670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:32.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.924+0000 7faab080c700 1 -- 172.21.15.52:0/3703945789 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56405270b900 msgr2=0x56405270eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:56:32.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.924+0000 7faab080c700 1 --2- 172.21.15.52:0/3703945789 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56405270b900 0x56405270eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:32.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.924+0000 7faab080c700 1 -- 172.21.15.52:0/3703945789 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56405261c7d0 con 0x56405270afe0 2024-06-10T02:56:32.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.924+0000 7faab080c700 1 --2- 172.21.15.52:0/3703945789 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56405270afe0 0x56405270e970 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7faa9c004980 tx=0x7faa9c005990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f534478aa1d1e5b server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:56:32.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.924+0000 7faa94ff9700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-10T02:56:32.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.924+0000 7faa94ff9700 1 -- 172.21.15.52:0/3703945789 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa9c016040 con 0x56405270afe0 2024-06-10T02:56:32.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.924+0000 7faabdaf4b80 1 -- 172.21.15.52:0/3703945789 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faa9c004d10 con 0x56405270afe0 2024-06-10T02:56:32.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.924+0000 7faa94ff9700 1 -- 172.21.15.52:0/3703945789 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7faa9c00ec30 con 0x56405270afe0 2024-06-10T02:56:32.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.924+0000 7faa94ff9700 1 -- 172.21.15.52:0/3703945789 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7faa9c02e050 con 0x56405270afe0 2024-06-10T02:56:32.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.924+0000 7faabdaf4b80 10 client.15894 fetch_fsmap learned FSMap version 31 2024-06-10T02:56:32.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.924+0000 7faabdaf4b80 1 -- 172.21.15.52:0/3703945789 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x56405271cab0 con 0x56405270afe0 2024-06-10T02:56:32.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.925+0000 7faa94ff9700 1 -- 172.21.15.52:0/3703945789 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa9c004d10 con 0x56405270afe0 2024-06-10T02:56:32.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.925+0000 7faa94ff9700 1 -- 172.21.15.52:0/3703945789 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 31) ==== 8366+0+0 (secure 0 0 0) 0x7faa9c00f040 con 0x56405270afe0 2024-06-10T02:56:32.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.925+0000 7faabdaf4b80 10 client.15894 fetch_fsmap finished waiting for FSMap version 31 2024-06-10T02:56:32.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.925+0000 7faabdaf4b80 10 client.15894 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T02:56:32.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.925+0000 7faabdaf4b80 20 client.15894 populate_metadata read hostname 'smithi052' 2024-06-10T02:56:32.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.925+0000 7faabdaf4b80 1 --2- 172.21.15.52:0/3703945789 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x56405271efe0 0x5640527213d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:56:32.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.925+0000 7faabdaf4b80 4 client.15894 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T02:56:32.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.925+0000 7faabdaf4b80 1 -- 172.21.15.52:0/3703945789 --> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564052663dd0 con 0x56405271efe0 2024-06-10T02:56:32.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.925+0000 7faab100d700 1 --2- 172.21.15.52:0/3703945789 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x56405271efe0 0x5640527213d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:56:32.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.926+0000 7faab100d700 1 --2- 172.21.15.52:0/3703945789 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x56405271efe0 0x5640527213d0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e150b71b6f0b780e server_cookie=f02e0c9d7dc80687 in_seq=0 out_seq=0 2024-06-10T02:56:32.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.926+0000 7faa94ff9700 10 client.15894 ms_handle_connect on v2:172.21.15.188:6838/310444836 2024-06-10T02:56:32.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.927+0000 7faa94ff9700 1 -- 172.21.15.52:0/3703945789 <== mds.0 v2:172.21.15.188:6838/310444836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x564052663dd0 con 0x56405271efe0 2024-06-10T02:56:32.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.927+0000 7faa94ff9700 10 client.15894 handle_command_reply: tid=0 2024-06-10T02:56:32.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.927+0000 7faabdaf4b80 1 client.15894 shutdown 2024-06-10T02:56:32.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.927+0000 7faabdaf4b80 1 -- 172.21.15.52:0/3703945789 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56405270afe0 msgr2=0x56405270e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:56:32.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.927+0000 7faabdaf4b80 1 --2- 172.21.15.52:0/3703945789 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56405270afe0 0x56405270e970 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7faa9c004980 tx=0x7faa9c005990 comp rx=0 tx=0).stop 2024-06-10T02:56:32.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.927+0000 7faabdaf4b80 1 -- 172.21.15.52:0/3703945789 shutdown_connections 2024-06-10T02:56:32.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.927+0000 7faabdaf4b80 1 --2- 172.21.15.52:0/3703945789 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x56405271efe0 0x5640527213d0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:32.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.927+0000 7faabdaf4b80 1 --2- 172.21.15.52:0/3703945789 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56405270b900 0x56405270eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:32.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.927+0000 7faabdaf4b80 1 --2- 172.21.15.52:0/3703945789 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564052705820 0x564052711670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:32.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.927+0000 7faa94ff9700 0 client.15894 ms_handle_reset on v2:172.21.15.188:6838/310444836 2024-06-10T02:56:32.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.927+0000 7faabdaf4b80 1 --2- 172.21.15.52:0/3703945789 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56405270afe0 0x56405270e970 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:32.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.927+0000 7faabdaf4b80 1 -- 172.21.15.52:0/3703945789 >> 172.21.15.52:0/3703945789 conn(0x564052632100 msgr2=0x5640526ff780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:56:32.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.927+0000 7faabdaf4b80 1 -- 172.21.15.52:0/3703945789 shutdown_connections 2024-06-10T02:56:32.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.927+0000 7faabdaf4b80 1 -- 172.21.15.52:0/3703945789 wait complete. 2024-06-10T02:56:32.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.927+0000 7faabdaf4b80 20 client.15894 trim_cache size 0 max 16384 2024-06-10T02:56:32.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.948+0000 7faabdaf4b80 1 Processor -- start 2024-06-10T02:56:32.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.948+0000 7faabdaf4b80 1 -- start start 2024-06-10T02:56:32.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.948+0000 7faabdaf4b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564052705820 0x5640527108b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:56:32.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.948+0000 7faabdaf4b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56405270afe0 0x5640526fff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:56:32.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.948+0000 7faabdaf4b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56405270b900 0x564052700440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:56:32.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.948+0000 7faabdaf4b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7faaac1b06b0 con 0x56405270afe0 2024-06-10T02:56:32.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.948+0000 7faabdaf4b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7faaac1b2220 con 0x564052705820 2024-06-10T02:56:32.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.948+0000 7faabdaf4b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7faaac1b0d80 con 0x56405270b900 2024-06-10T02:56:32.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.948+0000 7faab080c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56405270afe0 0x5640526fff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:56:32.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.948+0000 7faab080c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56405270afe0 0x5640526fff00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54668/0 (socket says 172.21.15.52:54668) 2024-06-10T02:56:32.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.948+0000 7faab080c700 1 -- 172.21.15.52:0/3820525698 learned_addr learned my addr 172.21.15.52:0/3820525698 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:56:32.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.949+0000 7faab180e700 1 --2- 172.21.15.52:0/3820525698 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56405270b900 0x564052700440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:56:32.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.949+0000 7faab100d700 1 --2- 172.21.15.52:0/3820525698 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564052705820 0x5640527108b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:56:32.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.949+0000 7faab080c700 1 -- 172.21.15.52:0/3820525698 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56405270b900 msgr2=0x564052700440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:56:32.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.949+0000 7faab080c700 1 --2- 172.21.15.52:0/3820525698 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56405270b900 0x564052700440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:32.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.949+0000 7faab080c700 1 -- 172.21.15.52:0/3820525698 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564052705820 msgr2=0x5640527108b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:56:32.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.949+0000 7faab080c700 1 --2- 172.21.15.52:0/3820525698 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564052705820 0x5640527108b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:32.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.949+0000 7faab080c700 1 -- 172.21.15.52:0/3820525698 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56405270cee0 con 0x56405270afe0 2024-06-10T02:56:32.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.949+0000 7faab080c700 1 --2- 172.21.15.52:0/3820525698 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56405270afe0 0x5640526fff00 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7faa9c009bf0 tx=0x7faa9c00e2f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c51f8fe2a1c3d9a9 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:56:32.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.949+0000 7faa96ffd700 1 -- 172.21.15.52:0/3820525698 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa9c016040 con 0x56405270afe0 2024-06-10T02:56:32.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.949+0000 7faa96ffd700 1 -- 172.21.15.52:0/3820525698 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa9c023410 con 0x56405270afe0 2024-06-10T02:56:32.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.949+0000 7faa96ffd700 1 -- 172.21.15.52:0/3820525698 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa9c023750 con 0x56405270afe0 2024-06-10T02:56:32.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.950+0000 7faabdaf4b80 1 -- 172.21.15.52:0/3820525698 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56405270afe0 msgr2=0x5640526fff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:56:32.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.950+0000 7faabdaf4b80 1 --2- 172.21.15.52:0/3820525698 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56405270afe0 0x5640526fff00 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7faa9c009bf0 tx=0x7faa9c00e2f0 comp rx=0 tx=0).stop 2024-06-10T02:56:32.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.950+0000 7faabdaf4b80 1 -- 172.21.15.52:0/3820525698 shutdown_connections 2024-06-10T02:56:32.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.950+0000 7faabdaf4b80 1 --2- 172.21.15.52:0/3820525698 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56405270b900 0x564052700440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:32.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.950+0000 7faabdaf4b80 1 --2- 172.21.15.52:0/3820525698 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564052705820 0x5640527108b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:32.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.950+0000 7faabdaf4b80 1 --2- 172.21.15.52:0/3820525698 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56405270afe0 0x5640526fff00 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:32.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.950+0000 7faabdaf4b80 1 -- 172.21.15.52:0/3820525698 >> 172.21.15.52:0/3820525698 conn(0x564052632100 msgr2=0x5640527101a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:56:32.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.950+0000 7faabdaf4b80 1 -- 172.21.15.52:0/3820525698 shutdown_connections 2024-06-10T02:56:32.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.950+0000 7faabdaf4b80 1 -- 172.21.15.52:0/3820525698 wait complete. 2024-06-10T02:56:32.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.950+0000 7faabdaf4b80 1 -- start start 2024-06-10T02:56:32.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.950+0000 7faabdaf4b80 1 Processor -- start 2024-06-10T02:56:32.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.951+0000 7faabdaf4b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564052705820 0x564052717a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:56:32.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.951+0000 7faabdaf4b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56405270afe0 0x564052714570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:56:32.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.951+0000 7faabdaf4b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56405270b900 0x564052714ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:56:32.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.951+0000 7faabdaf4b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7faaac1b1b20 con 0x56405270b900 2024-06-10T02:56:32.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.951+0000 7faabdaf4b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7faaac1b1450 con 0x56405270afe0 2024-06-10T02:56:32.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.951+0000 7faabdaf4b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7faaac1ae650 con 0x564052705820 2024-06-10T02:56:32.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.951+0000 7faabdaf4b80 10 client.0 _ll_register_callbacks cb 0x5640525ec630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T02:56:32.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.951+0000 7faab180e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56405270b900 0x564052714ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:56:32.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.951+0000 7faab180e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56405270b900 0x564052714ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54676/0 (socket says 172.21.15.52:54676) 2024-06-10T02:56:32.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.951+0000 7faab180e700 1 -- 172.21.15.52:0/3704147830 learned_addr learned my addr 172.21.15.52:0/3704147830 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:56:32.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.951+0000 7faab100d700 1 --2- 172.21.15.52:0/3704147830 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564052705820 0x564052717a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:56:32.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.951+0000 7faab080c700 1 --2- 172.21.15.52:0/3704147830 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56405270afe0 0x564052714570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:56:32.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.951+0000 7faab180e700 1 -- 172.21.15.52:0/3704147830 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564052705820 msgr2=0x564052717a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:56:32.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.951+0000 7faab180e700 1 --2- 172.21.15.52:0/3704147830 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564052705820 0x564052717a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:32.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.951+0000 7faab180e700 1 -- 172.21.15.52:0/3704147830 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56405270afe0 msgr2=0x564052714570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:56:32.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.951+0000 7faab180e700 1 --2- 172.21.15.52:0/3704147830 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56405270afe0 0x564052714570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:32.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.951+0000 7faab180e700 1 -- 172.21.15.52:0/3704147830 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564052714ff0 con 0x56405270b900 2024-06-10T02:56:32.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.952+0000 7faab180e700 1 --2- 172.21.15.52:0/3704147830 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56405270b900 0x564052714ab0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7faaa4006130 tx=0x7faaa409b650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fc0a25960c4753f server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:56:32.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.952+0000 7faa94ff9700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-10T02:56:32.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.952+0000 7faa94ff9700 1 -- 172.21.15.52:0/3704147830 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaa409f300 con 0x56405270b900 2024-06-10T02:56:32.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.952+0000 7faa94ff9700 1 -- 172.21.15.52:0/3704147830 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7faaa40192b0 con 0x56405270b900 2024-06-10T02:56:32.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.952+0000 7faabdaf4b80 1 -- 172.21.15.52:0/3704147830 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faa9c023410 con 0x56405270b900 2024-06-10T02:56:32.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.952+0000 7faa94ff9700 1 -- 172.21.15.52:0/3704147830 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7faaa40194b0 con 0x56405270b900 2024-06-10T02:56:32.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.952+0000 7faabdaf4b80 10 client.15900 fetch_fsmap learned FSMap version 31 2024-06-10T02:56:32.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.952+0000 7faabdaf4b80 1 -- 172.21.15.52:0/3704147830 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x56405278a770 con 0x56405270b900 2024-06-10T02:56:32.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.952+0000 7faa94ff9700 1 -- 172.21.15.52:0/3704147830 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa9c023410 con 0x56405270b900 2024-06-10T02:56:32.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.952+0000 7faa94ff9700 1 -- 172.21.15.52:0/3704147830 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 31) ==== 8366+0+0 (secure 0 0 0) 0x7faaa409e070 con 0x56405270b900 2024-06-10T02:56:32.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.952+0000 7faabdaf4b80 10 client.15900 fetch_fsmap finished waiting for FSMap version 31 2024-06-10T02:56:32.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.952+0000 7faabdaf4b80 10 client.15900 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T02:56:32.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.952+0000 7faabdaf4b80 20 client.15900 populate_metadata read hostname 'smithi052' 2024-06-10T02:56:32.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.952+0000 7faabdaf4b80 1 --2- 172.21.15.52:0/3704147830 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x56405278ae80 0x56405278d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:56:32.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.952+0000 7faabdaf4b80 4 client.15900 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-06-10T02:56:32.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.952+0000 7faabdaf4b80 1 -- 172.21.15.52:0/3704147830 --> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56405265e350 con 0x56405278ae80 2024-06-10T02:56:32.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.953+0000 7faab100d700 1 --2- 172.21.15.52:0/3704147830 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x56405278ae80 0x56405278d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:56:32.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.953+0000 7faab100d700 1 --2- 172.21.15.52:0/3704147830 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x56405278ae80 0x56405278d2a0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5bb3e1878ea5b4 server_cookie=e58669caa100f19b in_seq=0 out_seq=0 2024-06-10T02:56:32.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.953+0000 7faa94ff9700 10 client.15900 ms_handle_connect on v2:172.21.15.188:6838/310444836 2024-06-10T02:56:32.957 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:56:32 smithi188 ceph-mon[36599]: pgmap v504: 129 pgs: 129 active+clean; 3.5 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 853 B/s wr, 9 op/s 2024-06-10T02:56:32.958 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.954+0000 7faa94ff9700 1 -- 172.21.15.52:0/3704147830 <== mds.0 v2:172.21.15.188:6838/310444836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56405265e350 con 0x56405278ae80 2024-06-10T02:56:32.958 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.954+0000 7faa94ff9700 10 client.15900 handle_command_reply: tid=0 2024-06-10T02:56:32.958 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.954+0000 7faabdaf4b80 1 client.15900 shutdown 2024-06-10T02:56:32.958 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.954+0000 7faabdaf4b80 1 -- 172.21.15.52:0/3704147830 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56405270b900 msgr2=0x564052714ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:56:32.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.954+0000 7faabdaf4b80 1 --2- 172.21.15.52:0/3704147830 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56405270b900 0x564052714ab0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7faaa4006130 tx=0x7faaa409b650 comp rx=0 tx=0).stop 2024-06-10T02:56:32.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.954+0000 7faabdaf4b80 1 -- 172.21.15.52:0/3704147830 shutdown_connections 2024-06-10T02:56:32.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.954+0000 7faabdaf4b80 1 --2- 172.21.15.52:0/3704147830 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x56405278ae80 0x56405278d2a0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:32.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.954+0000 7faabdaf4b80 1 --2- 172.21.15.52:0/3704147830 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56405270b900 0x564052714ab0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:32.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.954+0000 7faabdaf4b80 1 --2- 172.21.15.52:0/3704147830 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564052705820 0x564052717a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:32.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.954+0000 7faabdaf4b80 1 --2- 172.21.15.52:0/3704147830 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56405270afe0 0x564052714570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:32.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.954+0000 7faabdaf4b80 1 -- 172.21.15.52:0/3704147830 >> 172.21.15.52:0/3704147830 conn(0x564052632100 msgr2=0x564052710a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:56:32.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.955+0000 7faa94ff9700 0 client.15900 ms_handle_reset on v2:172.21.15.188:6838/310444836 2024-06-10T02:56:32.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.955+0000 7faabdaf4b80 1 -- 172.21.15.52:0/3704147830 shutdown_connections 2024-06-10T02:56:32.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.955+0000 7faabdaf4b80 1 -- 172.21.15.52:0/3704147830 wait complete. 2024-06-10T02:56:32.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.955+0000 7faabdaf4b80 20 client.15900 trim_cache size 0 max 16384 2024-06-10T02:56:32.960 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-10T02:56:32.960 INFO:teuthology.orchestra.run.smithi052.stdout: "return_code": 0, 2024-06-10T02:56:32.960 INFO:teuthology.orchestra.run.smithi052.stdout: "scrub_tag": "0b3b2078-b4d0-4c44-b1f6-bda30b49dd44", 2024-06-10T02:56:32.960 INFO:teuthology.orchestra.run.smithi052.stdout: "mode": "asynchronous" 2024-06-10T02:56:32.960 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-10T02:56:32.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.955+0000 7faab3271700 1 -- 172.21.15.52:0/3875579289 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7faa980a2c20 msgr2=0x7faa980a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:56:32.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.955+0000 7faab3271700 1 --2- 172.21.15.52:0/3875579289 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7faa980a2c20 0x7faa980a50e0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7faaa800b5c0 tx=0x7faaa8005960 comp rx=0 tx=0).stop 2024-06-10T02:56:32.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.955+0000 7faab3271700 1 -- 172.21.15.52:0/3875579289 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faaac10ef70 msgr2=0x7faaac19f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:56:32.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.955+0000 7faab3271700 1 --2- 172.21.15.52:0/3875579289 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faaac10ef70 0x7faaac19f770 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7faaa400ada0 tx=0x7faaa400c330 comp rx=0 tx=0).stop 2024-06-10T02:56:32.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.955+0000 7faab3271700 1 -- 172.21.15.52:0/3875579289 shutdown_connections 2024-06-10T02:56:32.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.955+0000 7faab3271700 1 --2- 172.21.15.52:0/3875579289 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7faa980a2c20 0x7faa980a50e0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:32.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.955+0000 7faab3271700 1 --2- 172.21.15.52:0/3875579289 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faaac10ef70 0x7faaac19f770 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:32.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.955+0000 7faab3271700 1 --2- 172.21.15.52:0/3875579289 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faaac06abc0 0x7faaac19aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:32.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.955+0000 7faab3271700 1 --2- 172.21.15.52:0/3875579289 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faaac06b4e0 0x7faaac19b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:32.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.955+0000 7faab3271700 1 -- 172.21.15.52:0/3875579289 >> 172.21.15.52:0/3875579289 conn(0x7faaac078340 msgr2=0x7faaac1105a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:56:32.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.956+0000 7faab3271700 1 -- 172.21.15.52:0/3875579289 shutdown_connections 2024-06-10T02:56:32.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:32.956+0000 7faab3271700 1 -- 172.21.15.52:0/3875579289 wait complete. 2024-06-10T02:56:33.742 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-10T02:56:33.888 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:56:33 smithi120 ceph-mon[36634]: scrub queued for path: / 2024-06-10T02:56:33.888 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:56:33 smithi120 ceph-mon[36634]: scrub summary: idle+waiting paths [/] 2024-06-10T02:56:33.888 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:56:33 smithi120 ceph-mon[36634]: scrub summary: active paths [/] 2024-06-10T02:56:33.888 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:56:33 smithi120 ceph-mon[36634]: scrub summary: idle+waiting paths [/] 2024-06-10T02:56:33.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:56:33 smithi188 ceph-mon[36599]: scrub queued for path: / 2024-06-10T02:56:33.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:56:33 smithi188 ceph-mon[36599]: scrub summary: idle+waiting paths [/] 2024-06-10T02:56:33.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:56:33 smithi188 ceph-mon[36599]: scrub summary: active paths [/] 2024-06-10T02:56:33.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:56:33 smithi188 ceph-mon[36599]: scrub summary: idle+waiting paths [/] 2024-06-10T02:56:33.976 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:56:33 smithi052 ceph-mon[32362]: scrub queued for path: / 2024-06-10T02:56:33.976 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:56:33 smithi052 ceph-mon[32362]: scrub summary: idle+waiting paths [/] 2024-06-10T02:56:33.976 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:56:33 smithi052 ceph-mon[32362]: scrub summary: active paths [/] 2024-06-10T02:56:33.976 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:56:33 smithi052 ceph-mon[32362]: scrub summary: idle+waiting paths [/] 2024-06-10T02:56:34.002 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:56:34.726 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:56:34 smithi052 ceph-mon[32362]: scrub summary: active paths [/] 2024-06-10T02:56:34.727 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:56:34 smithi052 ceph-mon[32362]: scrub summary: idle+waiting paths [/] 2024-06-10T02:56:34.727 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:56:34 smithi052 ceph-mon[32362]: pgmap v505: 129 pgs: 129 active+clean; 3.9 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 41 KiB/s wr, 14 op/s 2024-06-10T02:56:34.888 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:56:34 smithi120 ceph-mon[36634]: scrub summary: active paths [/] 2024-06-10T02:56:34.888 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:56:34 smithi120 ceph-mon[36634]: scrub summary: idle+waiting paths [/] 2024-06-10T02:56:34.888 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:56:34 smithi120 ceph-mon[36634]: pgmap v505: 129 pgs: 129 active+clean; 3.9 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 41 KiB/s wr, 14 op/s 2024-06-10T02:56:34.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:56:34 smithi188 ceph-mon[36599]: scrub summary: active paths [/] 2024-06-10T02:56:34.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:56:34 smithi188 ceph-mon[36599]: scrub summary: idle+waiting paths [/] 2024-06-10T02:56:34.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:56:34 smithi188 ceph-mon[36599]: pgmap v505: 129 pgs: 129 active+clean; 3.9 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 41 KiB/s wr, 14 op/s 2024-06-10T02:56:35.610 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.610+0000 7fc9a14d0700 1 -- 172.21.15.52:0/643608201 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc99c108580 msgr2=0x7fc99c108960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:56:35.611 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.610+0000 7fc9a14d0700 1 --2- 172.21.15.52:0/643608201 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc99c108580 0x7fc99c108960 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fc98c00b0d0 tx=0x7fc98c00b3e0 comp rx=0 tx=0).stop 2024-06-10T02:56:35.611 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.610+0000 7fc9a14d0700 1 -- 172.21.15.52:0/643608201 shutdown_connections 2024-06-10T02:56:35.611 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.610+0000 7fc9a14d0700 1 --2- 172.21.15.52:0/643608201 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc99c06b620 0x7fc99c10f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:35.611 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.610+0000 7fc9a14d0700 1 --2- 172.21.15.52:0/643608201 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc99c108580 0x7fc99c108960 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:35.611 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.610+0000 7fc9a14d0700 1 --2- 172.21.15.52:0/643608201 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc99c06ac60 0x7fc99c06b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:35.611 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.610+0000 7fc9a14d0700 1 -- 172.21.15.52:0/643608201 >> 172.21.15.52:0/643608201 conn(0x7fc99c078430 msgr2=0x7fc99c078840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:56:35.611 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.610+0000 7fc9a14d0700 1 -- 172.21.15.52:0/643608201 shutdown_connections 2024-06-10T02:56:35.611 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.610+0000 7fc9a14d0700 1 -- 172.21.15.52:0/643608201 wait complete. 2024-06-10T02:56:35.611 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.611+0000 7fc9a14d0700 1 Processor -- start 2024-06-10T02:56:35.611 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.611+0000 7fc9a14d0700 1 -- start start 2024-06-10T02:56:35.613 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.611+0000 7fc9a14d0700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc99c06ac60 0x7fc99c19e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:56:35.613 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.611+0000 7fc9a14d0700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc99c06b620 0x7fc99c19edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:56:35.613 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.611+0000 7fc9a14d0700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc99c108580 0x7fc99c198a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:56:35.613 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.612+0000 7fc9a14d0700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fc99c1127d0 con 0x7fc99c06ac60 2024-06-10T02:56:35.613 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.612+0000 7fc9a14d0700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc99c112650 con 0x7fc99c108580 2024-06-10T02:56:35.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.612+0000 7fc9a14d0700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc99c112950 con 0x7fc99c06b620 2024-06-10T02:56:35.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.612+0000 7fc99affd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc99c06ac60 0x7fc99c19e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:56:35.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.612+0000 7fc99affd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc99c06ac60 0x7fc99c19e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54696/0 (socket says 172.21.15.52:54696) 2024-06-10T02:56:35.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.612+0000 7fc99affd700 1 -- 172.21.15.52:0/3378674178 learned_addr learned my addr 172.21.15.52:0/3378674178 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:56:35.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.612+0000 7fc99a7fc700 1 --2- 172.21.15.52:0/3378674178 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc99c06b620 0x7fc99c19edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:56:35.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.612+0000 7fc99b7fe700 1 --2- 172.21.15.52:0/3378674178 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc99c108580 0x7fc99c198a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:56:35.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.612+0000 7fc99affd700 1 -- 172.21.15.52:0/3378674178 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc99c06b620 msgr2=0x7fc99c19edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:56:35.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.612+0000 7fc99affd700 1 --2- 172.21.15.52:0/3378674178 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc99c06b620 0x7fc99c19edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:35.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.612+0000 7fc99affd700 1 -- 172.21.15.52:0/3378674178 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc99c108580 msgr2=0x7fc99c198a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:56:35.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.612+0000 7fc99affd700 1 --2- 172.21.15.52:0/3378674178 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc99c108580 0x7fc99c198a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:35.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.612+0000 7fc99affd700 1 -- 172.21.15.52:0/3378674178 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc99c1991d0 con 0x7fc99c06ac60 2024-06-10T02:56:35.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.612+0000 7fc99affd700 1 --2- 172.21.15.52:0/3378674178 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc99c06ac60 0x7fc99c19e8b0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fc98c00b820 tx=0x7fc98c004db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:56:35.615 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.612+0000 7fc983fff700 1 -- 172.21.15.52:0/3378674178 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc98c021020 con 0x7fc99c06ac60 2024-06-10T02:56:35.615 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.613+0000 7fc9a14d0700 1 -- 172.21.15.52:0/3378674178 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc99c199460 con 0x7fc99c06ac60 2024-06-10T02:56:35.615 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.613+0000 7fc983fff700 1 -- 172.21.15.52:0/3378674178 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc98c007cc0 con 0x7fc99c06ac60 2024-06-10T02:56:35.615 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.613+0000 7fc983fff700 1 -- 172.21.15.52:0/3378674178 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc98c005280 con 0x7fc99c06ac60 2024-06-10T02:56:35.615 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.613+0000 7fc9a14d0700 1 -- 172.21.15.52:0/3378674178 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc99c1a5780 con 0x7fc99c06ac60 2024-06-10T02:56:35.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.615+0000 7fc983fff700 1 -- 172.21.15.52:0/3378674178 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fc98c024020 con 0x7fc99c06ac60 2024-06-10T02:56:35.625 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.616+0000 7fc983fff700 1 --2- 172.21.15.52:0/3378674178 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fc9840a2c20 0x7fc9840a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:56:35.625 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.616+0000 7fc99a7fc700 1 --2- 172.21.15.52:0/3378674178 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fc9840a2c20 0x7fc9840a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:56:35.625 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.616+0000 7fc983fff700 1 -- 172.21.15.52:0/3378674178 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7fc98c00f020 con 0x7fc99c06ac60 2024-06-10T02:56:35.625 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.616+0000 7fc9abd53b80 1 Processor -- start 2024-06-10T02:56:35.625 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.616+0000 7fc9abd53b80 1 -- start start 2024-06-10T02:56:35.625 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.616+0000 7fc99a7fc700 1 --2- 172.21.15.52:0/3378674178 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fc9840a2c20 0x7fc9840a50e0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fc99c0ff930 tx=0x7fc99000b3c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:56:35.625 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.617+0000 7fc9abd53b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dc2ec1cf60 0x55dc2ec1d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:56:35.625 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.617+0000 7fc9abd53b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dc2ec1d880 0x55dc2ec17260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:56:35.625 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.617+0000 7fc9abd53b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dc2ec177a0 0x55dc2ec17bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:56:35.625 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.617+0000 7fc9abd53b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55dc2eb09f60 con 0x55dc2ec1cf60 2024-06-10T02:56:35.625 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.617+0000 7fc9abd53b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55dc2eb68850 con 0x55dc2ec1d880 2024-06-10T02:56:35.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.617+0000 7fc9abd53b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55dc2ead0460 con 0x55dc2ec177a0 2024-06-10T02:56:35.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.617+0000 7fc99affd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dc2ec1cf60 0x55dc2ec1d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:56:35.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.617+0000 7fc99affd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dc2ec1cf60 0x55dc2ec1d340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54704/0 (socket says 172.21.15.52:54704) 2024-06-10T02:56:35.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.617+0000 7fc99affd700 1 -- 172.21.15.52:0/3031144439 learned_addr learned my addr 172.21.15.52:0/3031144439 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:56:35.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.617+0000 7fc99a7fc700 1 --2- 172.21.15.52:0/3031144439 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dc2ec1d880 0x55dc2ec17260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:56:35.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.617+0000 7fc99b7fe700 1 --2- 172.21.15.52:0/3031144439 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dc2ec177a0 0x55dc2ec17bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:56:35.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.617+0000 7fc99affd700 1 -- 172.21.15.52:0/3031144439 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dc2ec177a0 msgr2=0x55dc2ec17bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:56:35.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.617+0000 7fc99affd700 1 --2- 172.21.15.52:0/3031144439 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dc2ec177a0 0x55dc2ec17bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:35.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.617+0000 7fc99affd700 1 -- 172.21.15.52:0/3031144439 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dc2ec1d880 msgr2=0x55dc2ec17260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:56:35.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.617+0000 7fc99affd700 1 --2- 172.21.15.52:0/3031144439 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dc2ec1d880 0x55dc2ec17260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:35.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.617+0000 7fc99affd700 1 -- 172.21.15.52:0/3031144439 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc2eab78d0 con 0x55dc2ec1cf60 2024-06-10T02:56:35.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.617+0000 7fc99b7fe700 1 --2- 172.21.15.52:0/3031144439 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dc2ec177a0 0x55dc2ec17bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T02:56:35.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.617+0000 7fc99affd700 1 --2- 172.21.15.52:0/3031144439 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dc2ec1cf60 0x55dc2ec1d340 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fc98c004ed0 tx=0x7fc98c01dd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=724a598c155243f2 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:56:35.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.618+0000 7fc9817fa700 1 -- 172.21.15.52:0/3031144439 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc98c09cd60 con 0x55dc2ec1cf60 2024-06-10T02:56:35.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.618+0000 7fc9817fa700 1 -- 172.21.15.52:0/3031144439 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc98c0a1110 con 0x55dc2ec1cf60 2024-06-10T02:56:35.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.618+0000 7fc9817fa700 1 -- 172.21.15.52:0/3031144439 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc98c0a54b0 con 0x55dc2ec1cf60 2024-06-10T02:56:35.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.618+0000 7fc9abd53b80 1 -- 172.21.15.52:0/3031144439 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dc2ec1cf60 msgr2=0x55dc2ec1d340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:56:35.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.618+0000 7fc9abd53b80 1 --2- 172.21.15.52:0/3031144439 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dc2ec1cf60 0x55dc2ec1d340 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fc98c004ed0 tx=0x7fc98c01dd20 comp rx=0 tx=0).stop 2024-06-10T02:56:35.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.619+0000 7fc9abd53b80 1 -- 172.21.15.52:0/3031144439 shutdown_connections 2024-06-10T02:56:35.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.619+0000 7fc9abd53b80 1 --2- 172.21.15.52:0/3031144439 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dc2ec177a0 0x55dc2ec17bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:35.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.619+0000 7fc9abd53b80 1 --2- 172.21.15.52:0/3031144439 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dc2ec1cf60 0x55dc2ec1d340 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:35.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.619+0000 7fc9abd53b80 1 --2- 172.21.15.52:0/3031144439 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dc2ec1d880 0x55dc2ec17260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:35.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.619+0000 7fc9abd53b80 1 -- 172.21.15.52:0/3031144439 >> 172.21.15.52:0/3031144439 conn(0x55dc2eb44080 msgr2=0x55dc2ec12220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:56:35.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.619+0000 7fc9abd53b80 1 -- 172.21.15.52:0/3031144439 shutdown_connections 2024-06-10T02:56:35.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.619+0000 7fc9abd53b80 1 -- 172.21.15.52:0/3031144439 wait complete. 2024-06-10T02:56:35.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.619+0000 7fc9abd53b80 1 -- start start 2024-06-10T02:56:35.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.619+0000 7fc9abd53b80 1 Processor -- start 2024-06-10T02:56:35.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.620+0000 7fc9abd53b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dc2ec177a0 0x55dc2ec23630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:56:35.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.620+0000 7fc9abd53b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dc2ec1cf60 0x55dc2ec20930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:56:35.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.620+0000 7fc9abd53b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dc2ec1d880 0x55dc2ec20e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:56:35.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.620+0000 7fc9abd53b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55dc2eb08fc0 con 0x55dc2ec177a0 2024-06-10T02:56:35.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.620+0000 7fc9abd53b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55dc2eb64810 con 0x55dc2ec1d880 2024-06-10T02:56:35.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.620+0000 7fc9abd53b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55dc2eb09f60 con 0x55dc2ec1cf60 2024-06-10T02:56:35.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.620+0000 7fc99affd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dc2ec177a0 0x55dc2ec23630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:56:35.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.620+0000 7fc9abd53b80 10 client.0 _ll_register_callbacks cb 0x55dc2eafe5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T02:56:35.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.620+0000 7fc99affd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dc2ec177a0 0x55dc2ec23630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54718/0 (socket says 172.21.15.52:54718) 2024-06-10T02:56:35.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.620+0000 7fc99affd700 1 -- 172.21.15.52:0/3018150116 learned_addr learned my addr 172.21.15.52:0/3018150116 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:56:35.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.620+0000 7fc99b7fe700 1 --2- 172.21.15.52:0/3018150116 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dc2ec1d880 0x55dc2ec20e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:56:35.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.620+0000 7fc99a7fc700 1 --2- 172.21.15.52:0/3018150116 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dc2ec1cf60 0x55dc2ec20930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:56:35.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.620+0000 7fc99affd700 1 -- 172.21.15.52:0/3018150116 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dc2ec1cf60 msgr2=0x55dc2ec20930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:56:35.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.620+0000 7fc99affd700 1 --2- 172.21.15.52:0/3018150116 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dc2ec1cf60 0x55dc2ec20930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:35.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.620+0000 7fc99affd700 1 -- 172.21.15.52:0/3018150116 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dc2ec1d880 msgr2=0x55dc2ec20e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:56:35.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.620+0000 7fc99affd700 1 --2- 172.21.15.52:0/3018150116 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dc2ec1d880 0x55dc2ec20e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:35.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.620+0000 7fc99affd700 1 -- 172.21.15.52:0/3018150116 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc2eb2e750 con 0x55dc2ec177a0 2024-06-10T02:56:35.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.620+0000 7fc99b7fe700 1 --2- 172.21.15.52:0/3018150116 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dc2ec1d880 0x55dc2ec20e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T02:56:35.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.620+0000 7fc99affd700 1 --2- 172.21.15.52:0/3018150116 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dc2ec177a0 0x55dc2ec23630 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fc98c09ca60 tx=0x7fc98c0a5a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2b6b2b9864850cf server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:56:35.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.620+0000 7fc97eff5700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-10T02:56:35.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.620+0000 7fc97eff5700 1 -- 172.21.15.52:0/3018150116 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc98c09cfc0 con 0x55dc2ec177a0 2024-06-10T02:56:35.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.621+0000 7fc97eff5700 1 -- 172.21.15.52:0/3018150116 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7fc98c09d160 con 0x55dc2ec177a0 2024-06-10T02:56:35.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.621+0000 7fc9abd53b80 1 -- 172.21.15.52:0/3018150116 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc98c0a1110 con 0x55dc2ec177a0 2024-06-10T02:56:35.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.621+0000 7fc97eff5700 1 -- 172.21.15.52:0/3018150116 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fc98c083050 con 0x55dc2ec177a0 2024-06-10T02:56:35.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.621+0000 7fc9abd53b80 10 client.15912 fetch_fsmap learned FSMap version 31 2024-06-10T02:56:35.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.621+0000 7fc9abd53b80 1 -- 172.21.15.52:0/3018150116 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55dc2ec329f0 con 0x55dc2ec177a0 2024-06-10T02:56:35.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.621+0000 7fc97eff5700 1 -- 172.21.15.52:0/3018150116 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc98c0a1110 con 0x55dc2ec177a0 2024-06-10T02:56:35.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.621+0000 7fc97eff5700 1 -- 172.21.15.52:0/3018150116 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 31) ==== 8366+0+0 (secure 0 0 0) 0x7fc98c0b4020 con 0x55dc2ec177a0 2024-06-10T02:56:35.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.621+0000 7fc9abd53b80 10 client.15912 fetch_fsmap finished waiting for FSMap version 31 2024-06-10T02:56:35.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.621+0000 7fc9abd53b80 10 client.15912 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T02:56:35.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.621+0000 7fc9abd53b80 20 client.15912 populate_metadata read hostname 'smithi052' 2024-06-10T02:56:35.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.621+0000 7fc9abd53b80 1 --2- 172.21.15.52:0/3018150116 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55dc2ec34d90 0x55dc2ec371b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:56:35.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.621+0000 7fc9abd53b80 4 client.15912 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T02:56:35.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.621+0000 7fc9abd53b80 1 -- 172.21.15.52:0/3018150116 --> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dc2eb75d50 con 0x55dc2ec34d90 2024-06-10T02:56:35.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.622+0000 7fc99a7fc700 1 --2- 172.21.15.52:0/3018150116 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55dc2ec34d90 0x55dc2ec371b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:56:35.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.622+0000 7fc99a7fc700 1 --2- 172.21.15.52:0/3018150116 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55dc2ec34d90 0x55dc2ec371b0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8dec47c46f08a96c server_cookie=6f259cc6e01ae0c7 in_seq=0 out_seq=0 2024-06-10T02:56:35.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.622+0000 7fc97eff5700 10 client.15912 ms_handle_connect on v2:172.21.15.188:6838/310444836 2024-06-10T02:56:35.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.623+0000 7fc97eff5700 1 -- 172.21.15.52:0/3018150116 <== mds.0 v2:172.21.15.188:6838/310444836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55dc2eb75d50 con 0x55dc2ec34d90 2024-06-10T02:56:35.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.623+0000 7fc97eff5700 10 client.15912 handle_command_reply: tid=0 2024-06-10T02:56:35.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.623+0000 7fc9abd53b80 1 client.15912 shutdown 2024-06-10T02:56:35.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.623+0000 7fc9abd53b80 1 -- 172.21.15.52:0/3018150116 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dc2ec177a0 msgr2=0x55dc2ec23630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:56:35.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.623+0000 7fc9abd53b80 1 --2- 172.21.15.52:0/3018150116 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dc2ec177a0 0x55dc2ec23630 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fc98c09ca60 tx=0x7fc98c0a5a70 comp rx=0 tx=0).stop 2024-06-10T02:56:35.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.623+0000 7fc9abd53b80 1 -- 172.21.15.52:0/3018150116 shutdown_connections 2024-06-10T02:56:35.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.623+0000 7fc9abd53b80 1 --2- 172.21.15.52:0/3018150116 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55dc2ec34d90 0x55dc2ec371b0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:35.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.623+0000 7fc9abd53b80 1 --2- 172.21.15.52:0/3018150116 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dc2ec1d880 0x55dc2ec20e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:35.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.623+0000 7fc9abd53b80 1 --2- 172.21.15.52:0/3018150116 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dc2ec177a0 0x55dc2ec23630 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:35.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.623+0000 7fc9abd53b80 1 --2- 172.21.15.52:0/3018150116 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dc2ec1cf60 0x55dc2ec20930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:35.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.623+0000 7fc9abd53b80 1 -- 172.21.15.52:0/3018150116 >> 172.21.15.52:0/3018150116 conn(0x55dc2eb44080 msgr2=0x55dc2ec11700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:56:35.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.623+0000 7fc97eff5700 0 client.15912 ms_handle_reset on v2:172.21.15.188:6838/310444836 2024-06-10T02:56:35.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.623+0000 7fc9abd53b80 1 -- 172.21.15.52:0/3018150116 shutdown_connections 2024-06-10T02:56:35.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.623+0000 7fc9abd53b80 1 -- 172.21.15.52:0/3018150116 wait complete. 2024-06-10T02:56:35.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.623+0000 7fc9abd53b80 20 client.15912 trim_cache size 0 max 16384 2024-06-10T02:56:35.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.654+0000 7fc9abd53b80 1 Processor -- start 2024-06-10T02:56:35.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.654+0000 7fc9abd53b80 1 -- start start 2024-06-10T02:56:35.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.654+0000 7fc9abd53b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dc2ec177a0 0x55dc2ec22870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:56:35.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.655+0000 7fc9abd53b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dc2ec1cf60 0x55dc2ec11e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:56:35.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.655+0000 7fc9abd53b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dc2ec1d880 0x55dc2ec123c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:56:35.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.655+0000 7fc9abd53b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fc99c1ac100 con 0x55dc2ec1cf60 2024-06-10T02:56:35.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.655+0000 7fc9abd53b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc99c1adc70 con 0x55dc2ec177a0 2024-06-10T02:56:35.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.655+0000 7fc9abd53b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc99c1ac7d0 con 0x55dc2ec1d880 2024-06-10T02:56:35.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.655+0000 7fc99affd700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dc2ec177a0 0x55dc2ec22870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:56:35.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.655+0000 7fc99affd700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dc2ec177a0 0x55dc2ec22870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.52:60812/0 (socket says 172.21.15.52:60812) 2024-06-10T02:56:35.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.655+0000 7fc99affd700 1 -- 172.21.15.52:0/4161386834 learned_addr learned my addr 172.21.15.52:0/4161386834 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:56:35.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.655+0000 7fc99a7fc700 1 --2- 172.21.15.52:0/4161386834 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dc2ec1cf60 0x55dc2ec11e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:56:35.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.656+0000 7fc99affd700 1 -- 172.21.15.52:0/4161386834 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dc2ec1d880 msgr2=0x55dc2ec123c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-10T02:56:35.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.656+0000 7fc99affd700 1 --2- 172.21.15.52:0/4161386834 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dc2ec1d880 0x55dc2ec123c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:35.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.656+0000 7fc99affd700 1 -- 172.21.15.52:0/4161386834 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dc2ec1cf60 msgr2=0x55dc2ec11e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:56:35.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.656+0000 7fc99affd700 1 --2- 172.21.15.52:0/4161386834 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dc2ec1cf60 0x55dc2ec11e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:35.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.656+0000 7fc99affd700 1 -- 172.21.15.52:0/4161386834 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc2ec37790 con 0x55dc2ec177a0 2024-06-10T02:56:35.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.656+0000 7fc99affd700 1 --2- 172.21.15.52:0/4161386834 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dc2ec177a0 0x55dc2ec22870 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc98c09b010 tx=0x7fc98c0ce010 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b910665e6322bdd6 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:56:35.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.656+0000 7fc980ff9700 1 -- 172.21.15.52:0/4161386834 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc98c0a1110 con 0x55dc2ec177a0 2024-06-10T02:56:35.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.656+0000 7fc980ff9700 1 -- 172.21.15.52:0/4161386834 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc98c065020 con 0x55dc2ec177a0 2024-06-10T02:56:35.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.656+0000 7fc980ff9700 1 -- 172.21.15.52:0/4161386834 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc98c0a54b0 con 0x55dc2ec177a0 2024-06-10T02:56:35.660 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.657+0000 7fc9abd53b80 1 -- 172.21.15.52:0/4161386834 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dc2ec177a0 msgr2=0x55dc2ec22870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:56:35.660 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.657+0000 7fc9abd53b80 1 --2- 172.21.15.52:0/4161386834 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dc2ec177a0 0x55dc2ec22870 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc98c09b010 tx=0x7fc98c0ce010 comp rx=0 tx=0).stop 2024-06-10T02:56:35.660 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.657+0000 7fc9abd53b80 1 -- 172.21.15.52:0/4161386834 shutdown_connections 2024-06-10T02:56:35.660 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.657+0000 7fc9abd53b80 1 --2- 172.21.15.52:0/4161386834 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dc2ec1d880 0x55dc2ec123c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:35.660 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.657+0000 7fc9abd53b80 1 --2- 172.21.15.52:0/4161386834 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dc2ec177a0 0x55dc2ec22870 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:35.660 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.657+0000 7fc9abd53b80 1 --2- 172.21.15.52:0/4161386834 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dc2ec1cf60 0x55dc2ec11e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:35.660 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.657+0000 7fc9abd53b80 1 -- 172.21.15.52:0/4161386834 >> 172.21.15.52:0/4161386834 conn(0x55dc2eb44080 msgr2=0x55dc2ec22160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:56:35.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.657+0000 7fc9abd53b80 1 -- 172.21.15.52:0/4161386834 shutdown_connections 2024-06-10T02:56:35.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.657+0000 7fc9abd53b80 1 -- 172.21.15.52:0/4161386834 wait complete. 2024-06-10T02:56:35.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.658+0000 7fc9abd53b80 1 -- start start 2024-06-10T02:56:35.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.658+0000 7fc9abd53b80 1 Processor -- start 2024-06-10T02:56:35.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.658+0000 7fc9abd53b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dc2ec177a0 0x55dc2ec29660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:56:35.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.658+0000 7fc9abd53b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dc2ec1cf60 0x55dc2ec3c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:56:35.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.658+0000 7fc9abd53b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dc2ec1d880 0x55dc2ec3c690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:56:35.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.658+0000 7fc9abd53b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fc99c1ad570 con 0x55dc2ec177a0 2024-06-10T02:56:35.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.658+0000 7fc9abd53b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc99c1acea0 con 0x55dc2ec1cf60 2024-06-10T02:56:35.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.658+0000 7fc9abd53b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc99c1aa0c0 con 0x55dc2ec1d880 2024-06-10T02:56:35.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.658+0000 7fc9abd53b80 10 client.0 _ll_register_callbacks cb 0x55dc2eafe5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T02:56:35.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.658+0000 7fc99affd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dc2ec177a0 0x55dc2ec29660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:56:35.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.658+0000 7fc99affd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dc2ec177a0 0x55dc2ec29660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54744/0 (socket says 172.21.15.52:54744) 2024-06-10T02:56:35.662 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.658+0000 7fc99affd700 1 -- 172.21.15.52:0/3136008955 learned_addr learned my addr 172.21.15.52:0/3136008955 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:56:35.662 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.658+0000 7fc99a7fc700 1 --2- 172.21.15.52:0/3136008955 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dc2ec1cf60 0x55dc2ec3c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:56:35.662 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.658+0000 7fc99b7fe700 1 --2- 172.21.15.52:0/3136008955 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dc2ec1d880 0x55dc2ec3c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:56:35.662 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.658+0000 7fc99affd700 1 -- 172.21.15.52:0/3136008955 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dc2ec1d880 msgr2=0x55dc2ec3c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:56:35.662 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.658+0000 7fc99affd700 1 --2- 172.21.15.52:0/3136008955 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dc2ec1d880 0x55dc2ec3c690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:35.662 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.658+0000 7fc99affd700 1 -- 172.21.15.52:0/3136008955 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dc2ec1cf60 msgr2=0x55dc2ec3c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:56:35.662 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.658+0000 7fc99affd700 1 --2- 172.21.15.52:0/3136008955 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dc2ec1cf60 0x55dc2ec3c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:35.662 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.658+0000 7fc99affd700 1 -- 172.21.15.52:0/3136008955 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc2ec3cc80 con 0x55dc2ec177a0 2024-06-10T02:56:35.662 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.659+0000 7fc99affd700 1 --2- 172.21.15.52:0/3136008955 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dc2ec177a0 0x55dc2ec29660 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fc98c099730 tx=0x7fc98c009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60d45d6ca933326e server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:56:35.662 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.659+0000 7fc97eff5700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-10T02:56:35.662 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.659+0000 7fc97eff5700 1 -- 172.21.15.52:0/3136008955 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc98c0a9030 con 0x55dc2ec177a0 2024-06-10T02:56:35.662 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.659+0000 7fc9abd53b80 1 -- 172.21.15.52:0/3136008955 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc98c065020 con 0x55dc2ec177a0 2024-06-10T02:56:35.662 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.659+0000 7fc97eff5700 1 -- 172.21.15.52:0/3136008955 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7fc98c066070 con 0x55dc2ec177a0 2024-06-10T02:56:35.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.659+0000 7fc97eff5700 1 -- 172.21.15.52:0/3136008955 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fc98c0a1110 con 0x55dc2ec177a0 2024-06-10T02:56:35.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.659+0000 7fc9abd53b80 10 client.15918 fetch_fsmap learned FSMap version 31 2024-06-10T02:56:35.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.659+0000 7fc9abd53b80 1 -- 172.21.15.52:0/3136008955 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55dc2ec3cc80 con 0x55dc2ec177a0 2024-06-10T02:56:35.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.659+0000 7fc97eff5700 1 -- 172.21.15.52:0/3136008955 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc98c065020 con 0x55dc2ec177a0 2024-06-10T02:56:35.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.660+0000 7fc97eff5700 1 -- 172.21.15.52:0/3136008955 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 31) ==== 8366+0+0 (secure 0 0 0) 0x7fc98c0b4040 con 0x55dc2ec177a0 2024-06-10T02:56:35.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.660+0000 7fc9abd53b80 10 client.15918 fetch_fsmap finished waiting for FSMap version 31 2024-06-10T02:56:35.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.660+0000 7fc9abd53b80 10 client.15918 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T02:56:35.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.660+0000 7fc9abd53b80 20 client.15918 populate_metadata read hostname 'smithi052' 2024-06-10T02:56:35.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.660+0000 7fc9abd53b80 1 --2- 172.21.15.52:0/3136008955 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55dc2ec9fb60 0x55dc2eca1f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:56:35.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.660+0000 7fc9abd53b80 4 client.15918 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-10T02:56:35.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.660+0000 7fc9abd53b80 1 -- 172.21.15.52:0/3136008955 --> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55dc2eb702d0 con 0x55dc2ec9fb60 2024-06-10T02:56:35.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.660+0000 7fc99a7fc700 1 --2- 172.21.15.52:0/3136008955 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55dc2ec9fb60 0x55dc2eca1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:56:35.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.660+0000 7fc97eff5700 10 client.15918 ms_handle_connect on v2:172.21.15.188:6838/310444836 2024-06-10T02:56:35.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.660+0000 7fc99a7fc700 1 --2- 172.21.15.52:0/3136008955 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55dc2ec9fb60 0x55dc2eca1f80 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b886e002dba11456 server_cookie=b43f7c3d16efe6b6 in_seq=0 out_seq=0 2024-06-10T02:56:35.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.661+0000 7fc97eff5700 1 -- 172.21.15.52:0/3136008955 <== mds.0 v2:172.21.15.188:6838/310444836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55dc2eb702d0 con 0x55dc2ec9fb60 2024-06-10T02:56:35.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.661+0000 7fc97eff5700 10 client.15918 handle_command_reply: tid=0 2024-06-10T02:56:35.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.661+0000 7fc9abd53b80 1 client.15918 shutdown 2024-06-10T02:56:35.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.661+0000 7fc9abd53b80 1 -- 172.21.15.52:0/3136008955 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dc2ec177a0 msgr2=0x55dc2ec29660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:56:35.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.661+0000 7fc9abd53b80 1 --2- 172.21.15.52:0/3136008955 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dc2ec177a0 0x55dc2ec29660 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fc98c099730 tx=0x7fc98c009b80 comp rx=0 tx=0).stop 2024-06-10T02:56:35.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.661+0000 7fc9abd53b80 1 -- 172.21.15.52:0/3136008955 shutdown_connections 2024-06-10T02:56:35.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.661+0000 7fc9abd53b80 1 --2- 172.21.15.52:0/3136008955 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55dc2ec9fb60 0x55dc2eca1f80 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:35.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.661+0000 7fc9abd53b80 1 --2- 172.21.15.52:0/3136008955 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dc2ec1d880 0x55dc2ec3c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:35.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.661+0000 7fc97eff5700 0 client.15918 ms_handle_reset on v2:172.21.15.188:6838/310444836 2024-06-10T02:56:35.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.661+0000 7fc9abd53b80 1 --2- 172.21.15.52:0/3136008955 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dc2ec177a0 0x55dc2ec29660 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:35.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.661+0000 7fc9abd53b80 1 --2- 172.21.15.52:0/3136008955 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dc2ec1cf60 0x55dc2ec3c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:35.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.661+0000 7fc9abd53b80 1 -- 172.21.15.52:0/3136008955 >> 172.21.15.52:0/3136008955 conn(0x55dc2eb44080 msgr2=0x55dc2ec229f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:56:35.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.661+0000 7fc9abd53b80 1 -- 172.21.15.52:0/3136008955 shutdown_connections 2024-06-10T02:56:35.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.662+0000 7fc9abd53b80 1 -- 172.21.15.52:0/3136008955 wait complete. 2024-06-10T02:56:35.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.662+0000 7fc9abd53b80 20 client.15918 trim_cache size 0 max 16384 2024-06-10T02:56:35.667 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-10T02:56:35.667 INFO:teuthology.orchestra.run.smithi052.stdout: "status": "IDLE (waiting for more scrubs)", 2024-06-10T02:56:35.667 INFO:teuthology.orchestra.run.smithi052.stdout: "scrubs": { 2024-06-10T02:56:35.667 INFO:teuthology.orchestra.run.smithi052.stdout: "0b3b2078-b4d0-4c44-b1f6-bda30b49dd44": { 2024-06-10T02:56:35.667 INFO:teuthology.orchestra.run.smithi052.stdout: "path": "/", 2024-06-10T02:56:35.667 INFO:teuthology.orchestra.run.smithi052.stdout: "tag": "0b3b2078-b4d0-4c44-b1f6-bda30b49dd44", 2024-06-10T02:56:35.667 INFO:teuthology.orchestra.run.smithi052.stdout: "options": "recursive,force" 2024-06-10T02:56:35.667 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T02:56:35.667 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T02:56:35.667 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-10T02:56:35.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.662+0000 7fc9a14d0700 1 -- 172.21.15.52:0/3378674178 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fc9840a2c20 msgr2=0x7fc9840a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:56:35.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.662+0000 7fc9a14d0700 1 --2- 172.21.15.52:0/3378674178 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fc9840a2c20 0x7fc9840a50e0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fc99c0ff930 tx=0x7fc99000b3c0 comp rx=0 tx=0).stop 2024-06-10T02:56:35.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.662+0000 7fc9a14d0700 1 -- 172.21.15.52:0/3378674178 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc99c06ac60 msgr2=0x7fc99c19e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:56:35.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.662+0000 7fc9a14d0700 1 --2- 172.21.15.52:0/3378674178 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc99c06ac60 0x7fc99c19e8b0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fc98c00b820 tx=0x7fc98c004db0 comp rx=0 tx=0).stop 2024-06-10T02:56:35.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.662+0000 7fc9a14d0700 1 -- 172.21.15.52:0/3378674178 shutdown_connections 2024-06-10T02:56:35.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.662+0000 7fc9a14d0700 1 --2- 172.21.15.52:0/3378674178 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc99c108580 0x7fc99c198a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:35.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.662+0000 7fc9a14d0700 1 --2- 172.21.15.52:0/3378674178 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fc9840a2c20 0x7fc9840a50e0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:35.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.662+0000 7fc9a14d0700 1 --2- 172.21.15.52:0/3378674178 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc99c06ac60 0x7fc99c19e8b0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:35.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.662+0000 7fc9a14d0700 1 --2- 172.21.15.52:0/3378674178 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc99c06b620 0x7fc99c19edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:56:35.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.662+0000 7fc9a14d0700 1 -- 172.21.15.52:0/3378674178 >> 172.21.15.52:0/3378674178 conn(0x7fc99c078430 msgr2=0x7fc99c0ffef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:56:35.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.663+0000 7fc9a14d0700 1 -- 172.21.15.52:0/3378674178 shutdown_connections 2024-06-10T02:56:35.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:56:35.663+0000 7fc9a14d0700 1 -- 172.21.15.52:0/3378674178 wait complete. 2024-06-10T02:56:35.695 INFO:tasks.workunit.client.0.smithi052.stdout:config.status: creating man/Makefile 2024-06-10T02:56:35.709 INFO:tasks.workunit.client.0.smithi052.stdout:config.status: creating config.h 2024-06-10T02:56:35.771 INFO:tasks.workunit.client.0.smithi052.stdout:config.status: executing depfiles commands 2024-06-10T02:56:35.905 INFO:tasks.workunit.client.0.smithi052.stderr:+ make 2024-06-10T02:56:35.914 INFO:tasks.workunit.client.0.smithi052.stdout:make all-recursive 2024-06-10T02:56:35.917 INFO:tasks.workunit.client.0.smithi052.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0' 2024-06-10T02:56:35.919 INFO:tasks.workunit.client.0.smithi052.stdout:Making all in src 2024-06-10T02:56:35.931 INFO:tasks.workunit.client.0.smithi052.stdout:make[2]: Entering directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0/src' 2024-06-10T02:56:35.931 INFO:tasks.workunit.client.0.smithi052.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT blogbench.o -MD -MP -MF ".deps/blogbench.Tpo" -c -o blogbench.o blogbench.c; \ 2024-06-10T02:56:35.931 INFO:tasks.workunit.client.0.smithi052.stdout:then mv -f ".deps/blogbench.Tpo" ".deps/blogbench.Po"; else rm -f ".deps/blogbench.Tpo"; exit 1; fi 2024-06-10T02:56:36.155 INFO:tasks.workunit.client.0.smithi052.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT process.o -MD -MP -MF ".deps/process.Tpo" -c -o process.o process.c; \ 2024-06-10T02:56:36.155 INFO:tasks.workunit.client.0.smithi052.stdout:then mv -f ".deps/process.Tpo" ".deps/process.Po"; else rm -f ".deps/process.Tpo"; exit 1; fi 2024-06-10T02:56:36.263 INFO:tasks.workunit.client.0.smithi052.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT writer.o -MD -MP -MF ".deps/writer.Tpo" -c -o writer.o writer.c; \ 2024-06-10T02:56:36.263 INFO:tasks.workunit.client.0.smithi052.stdout:then mv -f ".deps/writer.Tpo" ".deps/writer.Po"; else rm -f ".deps/writer.Tpo"; exit 1; fi 2024-06-10T02:56:36.346 INFO:tasks.workunit.client.0.smithi052.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT rewriter.o -MD -MP -MF ".deps/rewriter.Tpo" -c -o rewriter.o rewriter.c; \ 2024-06-10T02:56:36.346 INFO:tasks.workunit.client.0.smithi052.stdout:then mv -f ".deps/rewriter.Tpo" ".deps/rewriter.Po"; else rm -f ".deps/rewriter.Tpo"; exit 1; fi 2024-06-10T02:56:36.456 INFO:tasks.workunit.client.0.smithi052.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT reader.o -MD -MP -MF ".deps/reader.Tpo" -c -o reader.o reader.c; \ 2024-06-10T02:56:36.457 INFO:tasks.workunit.client.0.smithi052.stdout:then mv -f ".deps/reader.Tpo" ".deps/reader.Po"; else rm -f ".deps/reader.Tpo"; exit 1; fi 2024-06-10T02:56:36.497 INFO:tasks.cephfs.filesystem:scrub status for tag:0b3b2078-b4d0-4c44-b1f6-bda30b49dd44 - {'path': '/', 'tag': '0b3b2078-b4d0-4c44-b1f6-bda30b49dd44', 'options': 'recursive,force'} 2024-06-10T02:56:36.536 INFO:tasks.workunit.client.0.smithi052.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT commenter.o -MD -MP -MF ".deps/commenter.Tpo" -c -o commenter.o commenter.c; \ 2024-06-10T02:56:36.536 INFO:tasks.workunit.client.0.smithi052.stdout:then mv -f ".deps/commenter.Tpo" ".deps/commenter.Po"; else rm -f ".deps/commenter.Tpo"; exit 1; fi 2024-06-10T02:56:36.615 INFO:tasks.workunit.client.0.smithi052.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT helpers.o -MD -MP -MF ".deps/helpers.Tpo" -c -o helpers.o helpers.c; \ 2024-06-10T02:56:36.615 INFO:tasks.workunit.client.0.smithi052.stdout:then mv -f ".deps/helpers.Tpo" ".deps/helpers.Po"; else rm -f ".deps/helpers.Tpo"; exit 1; fi 2024-06-10T02:56:36.726 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:56:36 smithi052 ceph-mon[32362]: pgmap v506: 129 pgs: 129 active+clean; 3.9 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 41 KiB/s wr, 11 op/s 2024-06-10T02:56:36.736 INFO:tasks.workunit.client.0.smithi052.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT bsd-getopt_long.o -MD -MP -MF ".deps/bsd-getopt_long.Tpo" -c -o bsd-getopt_long.o bsd-getopt_long.c; \ 2024-06-10T02:56:36.737 INFO:tasks.workunit.client.0.smithi052.stdout:then mv -f ".deps/bsd-getopt_long.Tpo" ".deps/bsd-getopt_long.Po"; else rm -f ".deps/bsd-getopt_long.Tpo"; exit 1; fi 2024-06-10T02:56:36.801 INFO:tasks.workunit.client.0.smithi052.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT fakesnprintf.o -MD -MP -MF ".deps/fakesnprintf.Tpo" -c -o fakesnprintf.o fakesnprintf.c; \ 2024-06-10T02:56:36.801 INFO:tasks.workunit.client.0.smithi052.stdout:then mv -f ".deps/fakesnprintf.Tpo" ".deps/fakesnprintf.Po"; else rm -f ".deps/fakesnprintf.Tpo"; exit 1; fi 2024-06-10T02:56:36.869 INFO:tasks.workunit.client.0.smithi052.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT mysnprintf.o -MD -MP -MF ".deps/mysnprintf.Tpo" -c -o mysnprintf.o mysnprintf.c; \ 2024-06-10T02:56:36.869 INFO:tasks.workunit.client.0.smithi052.stdout:then mv -f ".deps/mysnprintf.Tpo" ".deps/mysnprintf.Po"; else rm -f ".deps/mysnprintf.Tpo"; exit 1; fi 2024-06-10T02:56:36.888 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:56:36 smithi120 ceph-mon[36634]: pgmap v506: 129 pgs: 129 active+clean; 3.9 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 41 KiB/s wr, 11 op/s 2024-06-10T02:56:36.929 INFO:tasks.workunit.client.0.smithi052.stdout:gcc -g -O2 -o blogbench blogbench.o process.o writer.o rewriter.o reader.o commenter.o helpers.o bsd-getopt_long.o fakesnprintf.o mysnprintf.o 2024-06-10T02:56:36.950 INFO:tasks.workunit.client.0.smithi052.stdout:make[2]: Leaving directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0/src' 2024-06-10T02:56:36.950 INFO:tasks.workunit.client.0.smithi052.stdout:Making all in man 2024-06-10T02:56:36.952 INFO:tasks.workunit.client.0.smithi052.stdout:make[2]: Entering directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0/man' 2024-06-10T02:56:36.952 INFO:tasks.workunit.client.0.smithi052.stdout:make[2]: Nothing to be done for 'all'. 2024-06-10T02:56:36.952 INFO:tasks.workunit.client.0.smithi052.stdout:make[2]: Leaving directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0/man' 2024-06-10T02:56:36.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:56:36 smithi188 ceph-mon[36599]: pgmap v506: 129 pgs: 129 active+clean; 3.9 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 41 KiB/s wr, 11 op/s 2024-06-10T02:56:36.954 INFO:tasks.workunit.client.0.smithi052.stdout:make[2]: Entering directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0' 2024-06-10T02:56:36.955 INFO:tasks.workunit.client.0.smithi052.stdout:make[2]: Leaving directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0' 2024-06-10T02:56:36.955 INFO:tasks.workunit.client.0.smithi052.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0' 2024-06-10T02:56:36.955 INFO:tasks.workunit.client.0.smithi052.stderr:+ cd src 2024-06-10T02:56:36.962 INFO:tasks.workunit.client.0.smithi052.stderr:+ mkdir blogtest_in 2024-06-10T02:56:36.965 INFO:tasks.workunit.client.0.smithi052.stdout:running blogbench 2024-06-10T02:56:36.966 INFO:tasks.workunit.client.0.smithi052.stderr:+ echo 'running blogbench' 2024-06-10T02:56:36.966 INFO:tasks.workunit.client.0.smithi052.stderr:+ ./blogbench -d blogtest_in 2024-06-10T02:56:36.966 INFO:tasks.workunit.client.0.smithi052.stdout: 2024-06-10T02:56:36.966 INFO:tasks.workunit.client.0.smithi052.stdout:Frequency = 10 secs 2024-06-10T02:56:36.967 INFO:tasks.workunit.client.0.smithi052.stdout:Scratch dir = [blogtest_in] 2024-06-10T02:56:37.239 INFO:tasks.workunit.client.0.smithi052.stdout:Spawning 3 writers... 2024-06-10T02:56:37.239 INFO:tasks.workunit.client.0.smithi052.stdout:Spawning 1 rewriters... 2024-06-10T02:56:37.239 INFO:tasks.workunit.client.0.smithi052.stdout:Spawning 5 commenters... 2024-06-10T02:56:37.239 INFO:tasks.workunit.client.0.smithi052.stdout:Spawning 100 readers... 2024-06-10T02:56:37.241 INFO:tasks.workunit.client.0.smithi052.stdout:Benchmarking for 30 iterations. 2024-06-10T02:56:37.241 INFO:tasks.workunit.client.0.smithi052.stdout:The test will run during 5 minutes. 2024-06-10T02:56:37.241 INFO:tasks.workunit.client.0.smithi052.stdout: 2024-06-10T02:56:37.241 INFO:tasks.workunit.client.0.smithi052.stdout: Nb blogs R articles W articles R pictures W pictures R comments W comments 2024-06-10T02:56:38.811 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:56:38 smithi052 ceph-mon[32362]: pgmap v507: 129 pgs: 129 active+clean; 4.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 80 KiB/s wr, 20 op/s 2024-06-10T02:56:38.888 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:56:38 smithi120 ceph-mon[36634]: pgmap v507: 129 pgs: 129 active+clean; 4.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 80 KiB/s wr, 20 op/s 2024-06-10T02:56:38.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:56:38 smithi188 ceph-mon[36599]: pgmap v507: 129 pgs: 129 active+clean; 4.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 80 KiB/s wr, 20 op/s 2024-06-10T02:56:40.888 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:56:40 smithi120 ceph-mon[36634]: pgmap v508: 129 pgs: 129 active+clean; 4.5 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 107 KiB/s wr, 23 op/s 2024-06-10T02:56:40.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:56:40 smithi188 ceph-mon[36599]: pgmap v508: 129 pgs: 129 active+clean; 4.5 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 107 KiB/s wr, 23 op/s 2024-06-10T02:56:40.976 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:56:40 smithi052 ceph-mon[32362]: pgmap v508: 129 pgs: 129 active+clean; 4.5 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 107 KiB/s wr, 23 op/s 2024-06-10T02:56:41.888 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:56:41 smithi120 ceph-mon[36634]: scrub summary: idle 2024-06-10T02:56:41.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:56:41 smithi188 ceph-mon[36599]: scrub summary: idle 2024-06-10T02:56:41.976 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:56:41 smithi052 ceph-mon[32362]: scrub summary: idle 2024-06-10T02:56:42.888 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:56:42 smithi120 ceph-mon[36634]: pgmap v509: 129 pgs: 129 active+clean; 8.4 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 395 KiB/s rd, 463 KiB/s wr, 29 op/s 2024-06-10T02:56:42.888 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:56:42 smithi120 ceph-mon[36634]: scrub summary: idle 2024-06-10T02:56:42.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:56:42 smithi188 ceph-mon[36599]: pgmap v509: 129 pgs: 129 active+clean; 8.4 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 395 KiB/s rd, 463 KiB/s wr, 29 op/s 2024-06-10T02:56:42.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:56:42 smithi188 ceph-mon[36599]: scrub summary: idle 2024-06-10T02:56:42.976 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:56:42 smithi052 ceph-mon[32362]: pgmap v509: 129 pgs: 129 active+clean; 8.4 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 395 KiB/s rd, 463 KiB/s wr, 29 op/s 2024-06-10T02:56:42.976 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:56:42 smithi052 ceph-mon[32362]: scrub summary: idle 2024-06-10T02:56:44.888 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:56:44 smithi120 ceph-mon[36634]: pgmap v510: 129 pgs: 129 active+clean; 22 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 772 KiB/s rd, 1.6 MiB/s wr, 76 op/s 2024-06-10T02:56:44.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:56:44 smithi188 ceph-mon[36599]: pgmap v510: 129 pgs: 129 active+clean; 22 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 772 KiB/s rd, 1.6 MiB/s wr, 76 op/s 2024-06-10T02:56:44.976 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:56:44 smithi052 ceph-mon[32362]: pgmap v510: 129 pgs: 129 active+clean; 22 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 772 KiB/s rd, 1.6 MiB/s wr, 76 op/s 2024-06-10T02:56:46.888 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:56:46 smithi120 ceph-mon[36634]: pgmap v511: 129 pgs: 129 active+clean; 28 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 742 KiB/s rd, 2.1 MiB/s wr, 97 op/s 2024-06-10T02:56:46.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:56:46 smithi188 ceph-mon[36599]: pgmap v511: 129 pgs: 129 active+clean; 28 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 742 KiB/s rd, 2.1 MiB/s wr, 97 op/s 2024-06-10T02:56:46.976 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:56:46 smithi052 ceph-mon[32362]: pgmap v511: 129 pgs: 129 active+clean; 28 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 742 KiB/s rd, 2.1 MiB/s wr, 97 op/s 2024-06-10T02:56:47.241 INFO:tasks.workunit.client.0.smithi052.stdout: 13 175423 625 121328 841 54195 976 2024-06-10T02:56:47.888 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:56:47 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T02:56:47.888 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:56:47 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:56:47.888 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:56:47 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T02:56:47.888 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:56:47 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:56:47.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:56:47 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T02:56:47.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:56:47 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:56:47.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:56:47 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T02:56:47.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:56:47 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:56:47.976 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:56:47 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T02:56:47.976 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:56:47 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:56:47.976 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:56:47 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T02:56:47.977 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:56:47 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:56:48.888 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:56:48 smithi120 ceph-mon[36634]: pgmap v512: 129 pgs: 129 active+clean; 45 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 747 KiB/s rd, 3.7 MiB/s wr, 191 op/s 2024-06-10T02:56:48.940 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:56:48 smithi052 ceph-mon[32362]: pgmap v512: 129 pgs: 129 active+clean; 45 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 747 KiB/s rd, 3.7 MiB/s wr, 191 op/s 2024-06-10T02:56:48.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:56:48 smithi188 ceph-mon[36599]: pgmap v512: 129 pgs: 129 active+clean; 45 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 747 KiB/s rd, 3.7 MiB/s wr, 191 op/s 2024-06-10T02:56:50.888 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:56:50 smithi120 ceph-mon[36634]: pgmap v513: 129 pgs: 129 active+clean; 53 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.4 MiB/s wr, 203 op/s 2024-06-10T02:56:50.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:56:50 smithi188 ceph-mon[36599]: pgmap v513: 129 pgs: 129 active+clean; 53 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.4 MiB/s wr, 203 op/s 2024-06-10T02:56:50.976 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:56:50 smithi052 ceph-mon[32362]: pgmap v513: 129 pgs: 129 active+clean; 53 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.4 MiB/s wr, 203 op/s 2024-06-10T02:56:52.888 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:56:52 smithi120 ceph-mon[36634]: pgmap v514: 129 pgs: 129 active+clean; 57 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.7 MiB/s wr, 205 op/s 2024-06-10T02:56:52.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:56:52 smithi188 ceph-mon[36599]: pgmap v514: 129 pgs: 129 active+clean; 57 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.7 MiB/s wr, 205 op/s 2024-06-10T02:56:52.976 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:56:52 smithi052 ceph-mon[32362]: pgmap v514: 129 pgs: 129 active+clean; 57 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.7 MiB/s wr, 205 op/s 2024-06-10T02:56:54.888 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:56:54 smithi120 ceph-mon[36634]: pgmap v515: 129 pgs: 129 active+clean; 56 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.4 MiB/s wr, 216 op/s 2024-06-10T02:56:54.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:56:54 smithi188 ceph-mon[36599]: pgmap v515: 129 pgs: 129 active+clean; 56 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.4 MiB/s wr, 216 op/s 2024-06-10T02:56:54.976 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:56:54 smithi052 ceph-mon[32362]: pgmap v515: 129 pgs: 129 active+clean; 56 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.4 MiB/s wr, 216 op/s 2024-06-10T02:56:56.888 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:56:56 smithi120 ceph-mon[36634]: pgmap v516: 129 pgs: 129 active+clean; 60 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.6 MiB/s wr, 178 op/s 2024-06-10T02:56:56.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:56:56 smithi188 ceph-mon[36599]: pgmap v516: 129 pgs: 129 active+clean; 60 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.6 MiB/s wr, 178 op/s 2024-06-10T02:56:56.976 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:56:56 smithi052 ceph-mon[32362]: pgmap v516: 129 pgs: 129 active+clean; 60 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.6 MiB/s wr, 178 op/s 2024-06-10T02:56:57.241 INFO:tasks.workunit.client.0.smithi052.stdout: 25 474309 647 378874 585 266494 1384 2024-06-10T02:56:58.889 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:56:58 smithi120 ceph-mon[36634]: pgmap v517: 129 pgs: 129 active+clean; 63 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.4 MiB/s wr, 169 op/s 2024-06-10T02:56:58.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:56:58 smithi188 ceph-mon[36599]: pgmap v517: 129 pgs: 129 active+clean; 63 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.4 MiB/s wr, 169 op/s 2024-06-10T02:56:58.976 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:56:58 smithi052 ceph-mon[32362]: pgmap v517: 129 pgs: 129 active+clean; 63 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.4 MiB/s wr, 169 op/s 2024-06-10T02:57:00.888 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:57:00 smithi120 ceph-mon[36634]: pgmap v518: 129 pgs: 129 active+clean; 72 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.6 MiB/s wr, 101 op/s 2024-06-10T02:57:00.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:57:00 smithi188 ceph-mon[36599]: pgmap v518: 129 pgs: 129 active+clean; 72 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.6 MiB/s wr, 101 op/s 2024-06-10T02:57:00.976 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:57:00 smithi052 ceph-mon[32362]: pgmap v518: 129 pgs: 129 active+clean; 72 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.6 MiB/s wr, 101 op/s 2024-06-10T02:57:02.888 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:57:02 smithi120 ceph-mon[36634]: pgmap v519: 129 pgs: 129 active+clean; 87 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.2 MiB/s wr, 167 op/s 2024-06-10T02:57:02.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:57:02 smithi188 ceph-mon[36599]: pgmap v519: 129 pgs: 129 active+clean; 87 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.2 MiB/s wr, 167 op/s 2024-06-10T02:57:02.976 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:57:02 smithi052 ceph-mon[32362]: pgmap v519: 129 pgs: 129 active+clean; 87 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.2 MiB/s wr, 167 op/s 2024-06-10T02:57:04.888 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:57:04 smithi120 ceph-mon[36634]: pgmap v520: 129 pgs: 129 active+clean; 117 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.7 MiB/s wr, 327 op/s 2024-06-10T02:57:04.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:57:04 smithi188 ceph-mon[36599]: pgmap v520: 129 pgs: 129 active+clean; 117 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.7 MiB/s wr, 327 op/s 2024-06-10T02:57:04.976 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:57:04 smithi052 ceph-mon[32362]: pgmap v520: 129 pgs: 129 active+clean; 117 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.7 MiB/s wr, 327 op/s 2024-06-10T02:57:06.499 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-10T02:57:06.815 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:57:06.888 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:57:06 smithi120 ceph-mon[36634]: pgmap v521: 129 pgs: 129 active+clean; 122 MiB data, 537 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.2 MiB/s wr, 348 op/s 2024-06-10T02:57:06.940 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:57:06 smithi052 ceph-mon[32362]: pgmap v521: 129 pgs: 129 active+clean; 122 MiB data, 537 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.2 MiB/s wr, 348 op/s 2024-06-10T02:57:06.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:57:06 smithi188 ceph-mon[36599]: pgmap v521: 129 pgs: 129 active+clean; 122 MiB data, 537 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.2 MiB/s wr, 348 op/s 2024-06-10T02:57:07.241 INFO:tasks.workunit.client.0.smithi052.stdout: 29 460559 350 317596 278 277410 992 2024-06-10T02:57:08.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.804+0000 7f1cd3963700 1 -- 172.21.15.52:0/945232020 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1ccc06a1a0 msgr2=0x7f1ccc06a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:57:08.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.804+0000 7f1cd3963700 1 --2- 172.21.15.52:0/945232020 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1ccc06a1a0 0x7f1ccc06a580 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f1cc8009990 tx=0x7f1cc8009ca0 comp rx=0 tx=0).stop 2024-06-10T02:57:08.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.804+0000 7f1cd3963700 1 -- 172.21.15.52:0/945232020 shutdown_connections 2024-06-10T02:57:08.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.804+0000 7f1cd3963700 1 --2- 172.21.15.52:0/945232020 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1ccc10ef40 0x7f1ccc111330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:08.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.804+0000 7f1cd3963700 1 --2- 172.21.15.52:0/945232020 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1ccc06a1a0 0x7f1ccc06a580 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:08.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.804+0000 7f1cd3963700 1 --2- 172.21.15.52:0/945232020 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1ccc06aac0 0x7f1ccc10ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:08.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.804+0000 7f1cd3963700 1 -- 172.21.15.52:0/945232020 >> 172.21.15.52:0/945232020 conn(0x7f1ccc078340 msgr2=0x7f1ccc078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:57:08.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.804+0000 7f1cd3963700 1 -- 172.21.15.52:0/945232020 shutdown_connections 2024-06-10T02:57:08.805 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:57:08 smithi052 ceph-mon[32362]: pgmap v522: 129 pgs: 129 active+clean; 126 MiB data, 532 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 6.3 MiB/s wr, 415 op/s 2024-06-10T02:57:08.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.804+0000 7f1cd3963700 1 -- 172.21.15.52:0/945232020 wait complete. 2024-06-10T02:57:08.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.805+0000 7f1cd3963700 1 Processor -- start 2024-06-10T02:57:08.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.805+0000 7f1cd3963700 1 -- start start 2024-06-10T02:57:08.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.805+0000 7f1cd3963700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1ccc06a1a0 0x7f1ccc19af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:57:08.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.806+0000 7f1cd3963700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1ccc06aac0 0x7f1ccc19b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:57:08.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.806+0000 7f1cd3963700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1ccc10ef40 0x7f1ccc19f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:57:08.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.806+0000 7f1cd3963700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f1ccc1149f0 con 0x7f1ccc10ef40 2024-06-10T02:57:08.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.806+0000 7f1cd3963700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1ccc114870 con 0x7f1ccc06a1a0 2024-06-10T02:57:08.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.806+0000 7f1cd3963700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f1ccc114b70 con 0x7f1ccc06aac0 2024-06-10T02:57:08.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.806+0000 7f1cd16ff700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1ccc06a1a0 0x7f1ccc19af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:57:08.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.806+0000 7f1cd1f00700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1ccc10ef40 0x7f1ccc19f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:57:08.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.806+0000 7f1cd16ff700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1ccc06a1a0 0x7f1ccc19af00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.52:37794/0 (socket says 172.21.15.52:37794) 2024-06-10T02:57:08.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.806+0000 7f1cd16ff700 1 -- 172.21.15.52:0/1046963044 learned_addr learned my addr 172.21.15.52:0/1046963044 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:57:08.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.806+0000 7f1cd0efe700 1 --2- 172.21.15.52:0/1046963044 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1ccc06aac0 0x7f1ccc19b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:57:08.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.806+0000 7f1cd16ff700 1 -- 172.21.15.52:0/1046963044 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1ccc06aac0 msgr2=0x7f1ccc19b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:57:08.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.806+0000 7f1cd16ff700 1 --2- 172.21.15.52:0/1046963044 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1ccc06aac0 0x7f1ccc19b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:08.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.806+0000 7f1cd16ff700 1 -- 172.21.15.52:0/1046963044 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1ccc10ef40 msgr2=0x7f1ccc19f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:57:08.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.806+0000 7f1cd16ff700 1 --2- 172.21.15.52:0/1046963044 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1ccc10ef40 0x7f1ccc19f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:08.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.806+0000 7f1cd16ff700 1 -- 172.21.15.52:0/1046963044 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ccc19fec0 con 0x7f1ccc06a1a0 2024-06-10T02:57:08.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.806+0000 7f1cd1f00700 1 --2- 172.21.15.52:0/1046963044 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1ccc10ef40 0x7f1ccc19f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T02:57:08.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.806+0000 7f1cd16ff700 1 --2- 172.21.15.52:0/1046963044 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1ccc06a1a0 0x7f1ccc19af00 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f1cc8005910 tx=0x7f1cc800bab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:57:08.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.807+0000 7f1cc27fc700 1 -- 172.21.15.52:0/1046963044 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cc8027070 con 0x7f1ccc06a1a0 2024-06-10T02:57:08.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.807+0000 7f1cc27fc700 1 -- 172.21.15.52:0/1046963044 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1cc8003d70 con 0x7f1ccc06a1a0 2024-06-10T02:57:08.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.807+0000 7f1cc27fc700 1 -- 172.21.15.52:0/1046963044 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cc800f460 con 0x7f1ccc06a1a0 2024-06-10T02:57:08.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.807+0000 7f1cd3963700 1 -- 172.21.15.52:0/1046963044 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ccc1a0150 con 0x7f1ccc06a1a0 2024-06-10T02:57:08.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.807+0000 7f1cd3963700 1 -- 172.21.15.52:0/1046963044 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ccc1a7a30 con 0x7f1ccc06a1a0 2024-06-10T02:57:08.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.809+0000 7f1cc27fc700 1 -- 172.21.15.52:0/1046963044 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f1cc802a070 con 0x7f1ccc06a1a0 2024-06-10T02:57:08.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.810+0000 7f1cc27fc700 1 --2- 172.21.15.52:0/1046963044 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f1cb80a2c20 0x7f1cb80a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:57:08.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.810+0000 7f1cc27fc700 1 -- 172.21.15.52:0/1046963044 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f1cc80d0260 con 0x7f1ccc06a1a0 2024-06-10T02:57:08.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.810+0000 7f1cd0efe700 1 --2- 172.21.15.52:0/1046963044 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f1cb80a2c20 0x7f1cb80a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:57:08.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.810+0000 7f1cd0efe700 1 --2- 172.21.15.52:0/1046963044 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f1cb80a2c20 0x7f1cb80a50e0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f1ccc102880 tx=0x7f1cbc009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:57:08.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.811+0000 7f1cde1e6b80 1 Processor -- start 2024-06-10T02:57:08.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.811+0000 7f1cde1e6b80 1 -- start start 2024-06-10T02:57:08.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.811+0000 7f1cde1e6b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ebc1a0ef60 0x55ebc1a0f340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:57:08.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.811+0000 7f1cde1e6b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ebc1a0f880 0x55ebc1a09260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:57:08.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.811+0000 7f1cde1e6b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ebc1a097a0 0x55ebc1a09bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:57:08.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.811+0000 7f1cde1e6b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55ebc18fbf60 con 0x55ebc1a0f880 2024-06-10T02:57:08.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.811+0000 7f1cde1e6b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55ebc195a850 con 0x55ebc1a0ef60 2024-06-10T02:57:08.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.811+0000 7f1cde1e6b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55ebc18c2460 con 0x55ebc1a097a0 2024-06-10T02:57:08.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.811+0000 7f1cd0efe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ebc1a0f880 0x55ebc1a09260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:57:08.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.811+0000 7f1cd16ff700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ebc1a0ef60 0x55ebc1a0f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:57:08.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.811+0000 7f1cd0efe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ebc1a0f880 0x55ebc1a09260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:42568/0 (socket says 172.21.15.52:42568) 2024-06-10T02:57:08.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.811+0000 7f1cd0efe700 1 -- 172.21.15.52:0/2132749389 learned_addr learned my addr 172.21.15.52:0/2132749389 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:57:08.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.811+0000 7f1cd1f00700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ebc1a097a0 0x55ebc1a09bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:57:08.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.811+0000 7f1cd16ff700 1 -- 172.21.15.52:0/2132749389 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ebc1a097a0 msgr2=0x55ebc1a09bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:57:08.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.811+0000 7f1cd16ff700 1 --2- 172.21.15.52:0/2132749389 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ebc1a097a0 0x55ebc1a09bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:08.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.811+0000 7f1cd16ff700 1 -- 172.21.15.52:0/2132749389 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ebc1a0f880 msgr2=0x55ebc1a09260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:57:08.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.811+0000 7f1cd16ff700 1 --2- 172.21.15.52:0/2132749389 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ebc1a0f880 0x55ebc1a09260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:08.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.811+0000 7f1cd16ff700 1 -- 172.21.15.52:0/2132749389 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ebc18a98d0 con 0x55ebc1a0ef60 2024-06-10T02:57:08.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.812+0000 7f1cd16ff700 1 --2- 172.21.15.52:0/2132749389 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ebc1a0ef60 0x55ebc1a0f340 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f1cc80cda00 tx=0x7f1cc8009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f7def54d3dfd614b server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:57:08.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.812+0000 7f1cb77fe700 1 -- 172.21.15.52:0/2132749389 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cc809c100 con 0x55ebc1a0ef60 2024-06-10T02:57:08.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.812+0000 7f1cb77fe700 1 -- 172.21.15.52:0/2132749389 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1cc80043d0 con 0x55ebc1a0ef60 2024-06-10T02:57:08.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.812+0000 7f1cb77fe700 1 -- 172.21.15.52:0/2132749389 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cc80a5380 con 0x55ebc1a0ef60 2024-06-10T02:57:08.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.813+0000 7f1cde1e6b80 1 -- 172.21.15.52:0/2132749389 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ebc1a0ef60 msgr2=0x55ebc1a0f340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:57:08.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.813+0000 7f1cde1e6b80 1 --2- 172.21.15.52:0/2132749389 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ebc1a0ef60 0x55ebc1a0f340 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f1cc80cda00 tx=0x7f1cc8009ca0 comp rx=0 tx=0).stop 2024-06-10T02:57:08.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.813+0000 7f1cde1e6b80 1 -- 172.21.15.52:0/2132749389 shutdown_connections 2024-06-10T02:57:08.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.813+0000 7f1cde1e6b80 1 --2- 172.21.15.52:0/2132749389 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ebc1a097a0 0x55ebc1a09bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:08.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.813+0000 7f1cde1e6b80 1 --2- 172.21.15.52:0/2132749389 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ebc1a0ef60 0x55ebc1a0f340 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:08.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.813+0000 7f1cde1e6b80 1 --2- 172.21.15.52:0/2132749389 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ebc1a0f880 0x55ebc1a09260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:08.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.813+0000 7f1cde1e6b80 1 -- 172.21.15.52:0/2132749389 >> 172.21.15.52:0/2132749389 conn(0x55ebc1936080 msgr2=0x55ebc1a04220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:57:08.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.813+0000 7f1cde1e6b80 1 -- 172.21.15.52:0/2132749389 shutdown_connections 2024-06-10T02:57:08.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.813+0000 7f1cde1e6b80 1 -- 172.21.15.52:0/2132749389 wait complete. 2024-06-10T02:57:08.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.813+0000 7f1cde1e6b80 1 -- start start 2024-06-10T02:57:08.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.813+0000 7f1cde1e6b80 1 Processor -- start 2024-06-10T02:57:08.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.813+0000 7f1cde1e6b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ebc1a097a0 0x55ebc1a15590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:57:08.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.814+0000 7f1cde1e6b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ebc1a0ef60 0x55ebc1a12930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:57:08.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.814+0000 7f1cde1e6b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ebc1a0f880 0x55ebc1a12e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:57:08.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.814+0000 7f1cde1e6b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55ebc18fafc0 con 0x55ebc1a0ef60 2024-06-10T02:57:08.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.814+0000 7f1cde1e6b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55ebc1956810 con 0x55ebc1a097a0 2024-06-10T02:57:08.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.814+0000 7f1cde1e6b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55ebc1a16010 con 0x55ebc1a0f880 2024-06-10T02:57:08.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.814+0000 7f1cde1e6b80 10 client.0 _ll_register_callbacks cb 0x55ebc18f05b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T02:57:08.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.814+0000 7f1cd0efe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ebc1a0ef60 0x55ebc1a12930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:57:08.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.814+0000 7f1cd0efe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ebc1a0ef60 0x55ebc1a12930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:42580/0 (socket says 172.21.15.52:42580) 2024-06-10T02:57:08.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.814+0000 7f1cd1f00700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ebc1a0f880 0x55ebc1a12e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:57:08.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.814+0000 7f1cd0efe700 1 -- 172.21.15.52:0/3742754366 learned_addr learned my addr 172.21.15.52:0/3742754366 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:57:08.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.814+0000 7f1cd16ff700 1 --2- 172.21.15.52:0/3742754366 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ebc1a097a0 0x55ebc1a15590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:57:08.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.814+0000 7f1cd1f00700 1 -- 172.21.15.52:0/3742754366 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ebc1a097a0 msgr2=0x55ebc1a15590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:57:08.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.814+0000 7f1cd1f00700 1 --2- 172.21.15.52:0/3742754366 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ebc1a097a0 0x55ebc1a15590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:08.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.814+0000 7f1cd1f00700 1 -- 172.21.15.52:0/3742754366 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ebc1a0ef60 msgr2=0x55ebc1a12930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:57:08.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.814+0000 7f1cd1f00700 1 --2- 172.21.15.52:0/3742754366 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ebc1a0ef60 0x55ebc1a12930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:08.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.814+0000 7f1cd1f00700 1 -- 172.21.15.52:0/3742754366 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ebc1920750 con 0x55ebc1a0f880 2024-06-10T02:57:08.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.814+0000 7f1cd16ff700 1 --2- 172.21.15.52:0/3742754366 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ebc1a097a0 0x55ebc1a15590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T02:57:08.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.814+0000 7f1cd0efe700 1 --2- 172.21.15.52:0/3742754366 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ebc1a0ef60 0x55ebc1a12930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T02:57:08.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.814+0000 7f1cd1f00700 1 --2- 172.21.15.52:0/3742754366 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ebc1a0f880 0x55ebc1a12e70 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f1ccc069d60 tx=0x7f1cc4004630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8b2d6e93faba0d16 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:57:08.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.814+0000 7f1cb4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-06-10T02:57:08.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.814+0000 7f1cb4ff9700 1 -- 172.21.15.52:0/3742754366 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cc401a070 con 0x55ebc1a0f880 2024-06-10T02:57:08.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.815+0000 7f1cb4ff9700 1 -- 172.21.15.52:0/3742754366 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f1cc4004db0 con 0x55ebc1a0f880 2024-06-10T02:57:08.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.815+0000 7f1cde1e6b80 1 -- 172.21.15.52:0/3742754366 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1cc80043d0 con 0x55ebc1a0f880 2024-06-10T02:57:08.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.815+0000 7f1cb4ff9700 1 -- 172.21.15.52:0/3742754366 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f1cc4025050 con 0x55ebc1a0f880 2024-06-10T02:57:08.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.815+0000 7f1cde1e6b80 10 client.25495 fetch_fsmap learned FSMap version 31 2024-06-10T02:57:08.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.815+0000 7f1cde1e6b80 1 -- 172.21.15.52:0/3742754366 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55ebc1a20720 con 0x55ebc1a0f880 2024-06-10T02:57:08.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.815+0000 7f1cb4ff9700 1 -- 172.21.15.52:0/3742754366 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1cc80043d0 con 0x55ebc1a0f880 2024-06-10T02:57:08.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.815+0000 7f1cb4ff9700 1 -- 172.21.15.52:0/3742754366 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 31) ==== 8366+0+0 (secure 0 0 0) 0x7f1cc4016910 con 0x55ebc1a0f880 2024-06-10T02:57:08.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.815+0000 7f1cde1e6b80 10 client.25495 fetch_fsmap finished waiting for FSMap version 31 2024-06-10T02:57:08.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.815+0000 7f1cde1e6b80 10 client.25495 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T02:57:08.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.815+0000 7f1cde1e6b80 20 client.25495 populate_metadata read hostname 'smithi052' 2024-06-10T02:57:08.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.815+0000 7f1cde1e6b80 1 --2- 172.21.15.52:0/3742754366 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55ebc1a22d50 0x55ebc1a25140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:57:08.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.815+0000 7f1cde1e6b80 4 client.25495 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T02:57:08.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.815+0000 7f1cde1e6b80 1 -- 172.21.15.52:0/3742754366 --> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ebc1967d50 con 0x55ebc1a22d50 2024-06-10T02:57:08.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.815+0000 7f1cd16ff700 1 --2- 172.21.15.52:0/3742754366 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55ebc1a22d50 0x55ebc1a25140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:57:08.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.816+0000 7f1cd16ff700 1 --2- 172.21.15.52:0/3742754366 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55ebc1a22d50 0x55ebc1a25140 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d5062a07bb4f334 server_cookie=6bf5427235e741ef in_seq=0 out_seq=0 2024-06-10T02:57:08.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.816+0000 7f1cb4ff9700 10 client.25495 ms_handle_connect on v2:172.21.15.188:6838/310444836 2024-06-10T02:57:08.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.816+0000 7f1cb4ff9700 1 -- 172.21.15.52:0/3742754366 <== mds.0 v2:172.21.15.188:6838/310444836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55ebc1967d50 con 0x55ebc1a22d50 2024-06-10T02:57:08.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.816+0000 7f1cb4ff9700 10 client.25495 handle_command_reply: tid=0 2024-06-10T02:57:08.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.816+0000 7f1cde1e6b80 1 client.25495 shutdown 2024-06-10T02:57:08.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.817+0000 7f1cde1e6b80 1 -- 172.21.15.52:0/3742754366 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ebc1a0f880 msgr2=0x55ebc1a12e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:57:08.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.817+0000 7f1cde1e6b80 1 --2- 172.21.15.52:0/3742754366 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ebc1a0f880 0x55ebc1a12e70 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f1ccc069d60 tx=0x7f1cc4004630 comp rx=0 tx=0).stop 2024-06-10T02:57:08.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.817+0000 7f1cde1e6b80 1 -- 172.21.15.52:0/3742754366 shutdown_connections 2024-06-10T02:57:08.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.817+0000 7f1cde1e6b80 1 --2- 172.21.15.52:0/3742754366 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55ebc1a22d50 0x55ebc1a25140 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:08.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.817+0000 7f1cde1e6b80 1 --2- 172.21.15.52:0/3742754366 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ebc1a0f880 0x55ebc1a12e70 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:08.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.817+0000 7f1cde1e6b80 1 --2- 172.21.15.52:0/3742754366 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ebc1a097a0 0x55ebc1a15590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:08.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.817+0000 7f1cde1e6b80 1 --2- 172.21.15.52:0/3742754366 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ebc1a0ef60 0x55ebc1a12930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:08.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.817+0000 7f1cb4ff9700 0 client.25495 ms_handle_reset on v2:172.21.15.188:6838/310444836 2024-06-10T02:57:08.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.817+0000 7f1cde1e6b80 1 -- 172.21.15.52:0/3742754366 >> 172.21.15.52:0/3742754366 conn(0x55ebc1936080 msgr2=0x55ebc1a03700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:57:08.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.817+0000 7f1cde1e6b80 1 -- 172.21.15.52:0/3742754366 shutdown_connections 2024-06-10T02:57:08.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.817+0000 7f1cde1e6b80 1 -- 172.21.15.52:0/3742754366 wait complete. 2024-06-10T02:57:08.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.817+0000 7f1cde1e6b80 20 client.25495 trim_cache size 0 max 16384 2024-06-10T02:57:08.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.844+0000 7f1cde1e6b80 1 Processor -- start 2024-06-10T02:57:08.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.844+0000 7f1cde1e6b80 1 -- start start 2024-06-10T02:57:08.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.844+0000 7f1cde1e6b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ebc1a097a0 0x55ebc1a09260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:57:08.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.844+0000 7f1cde1e6b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ebc1a0ef60 0x55ebc1a03e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:57:08.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.844+0000 7f1cde1e6b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ebc1a0f880 0x55ebc1a043c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:57:08.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.844+0000 7f1cde1e6b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f1ccc1b0520 con 0x55ebc1a097a0 2024-06-10T02:57:08.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.844+0000 7f1cde1e6b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1ccc1b2090 con 0x55ebc1a0f880 2024-06-10T02:57:08.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.844+0000 7f1cde1e6b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f1ccc1b0bf0 con 0x55ebc1a0ef60 2024-06-10T02:57:08.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.844+0000 7f1cd16ff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ebc1a097a0 0x55ebc1a09260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:57:08.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.844+0000 7f1cd16ff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ebc1a097a0 0x55ebc1a09260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:42582/0 (socket says 172.21.15.52:42582) 2024-06-10T02:57:08.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.844+0000 7f1cd16ff700 1 -- 172.21.15.52:0/3282669629 learned_addr learned my addr 172.21.15.52:0/3282669629 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:57:08.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.844+0000 7f1cd0efe700 1 --2- 172.21.15.52:0/3282669629 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ebc1a0ef60 0x55ebc1a03e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:57:08.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.845+0000 7f1cd1f00700 1 --2- 172.21.15.52:0/3282669629 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ebc1a0f880 0x55ebc1a043c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:57:08.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.845+0000 7f1cd16ff700 1 -- 172.21.15.52:0/3282669629 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ebc1a0ef60 msgr2=0x55ebc1a03e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:57:08.846 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.845+0000 7f1cd16ff700 1 --2- 172.21.15.52:0/3282669629 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ebc1a0ef60 0x55ebc1a03e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:08.846 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.845+0000 7f1cd16ff700 1 -- 172.21.15.52:0/3282669629 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ebc1a0f880 msgr2=0x55ebc1a043c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:57:08.846 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.845+0000 7f1cd16ff700 1 --2- 172.21.15.52:0/3282669629 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ebc1a0f880 0x55ebc1a043c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:08.846 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.845+0000 7f1cd16ff700 1 -- 172.21.15.52:0/3282669629 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ebc1a118b0 con 0x55ebc1a097a0 2024-06-10T02:57:08.846 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.845+0000 7f1cd16ff700 1 --2- 172.21.15.52:0/3282669629 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ebc1a097a0 0x55ebc1a09260 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f1cc8000c00 tx=0x7f1cc801a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=897a73dce6b1ed72 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:57:08.846 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.845+0000 7f1cb6ffd700 1 -- 172.21.15.52:0/3282669629 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cc80043d0 con 0x55ebc1a097a0 2024-06-10T02:57:08.846 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.845+0000 7f1cb6ffd700 1 -- 172.21.15.52:0/3282669629 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1cc809c7a0 con 0x55ebc1a097a0 2024-06-10T02:57:08.846 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.845+0000 7f1cb6ffd700 1 -- 172.21.15.52:0/3282669629 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cc8022650 con 0x55ebc1a097a0 2024-06-10T02:57:08.849 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.846+0000 7f1cde1e6b80 1 -- 172.21.15.52:0/3282669629 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ebc1a097a0 msgr2=0x55ebc1a09260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:57:08.849 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.846+0000 7f1cde1e6b80 1 --2- 172.21.15.52:0/3282669629 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ebc1a097a0 0x55ebc1a09260 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f1cc8000c00 tx=0x7f1cc801a040 comp rx=0 tx=0).stop 2024-06-10T02:57:08.849 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.846+0000 7f1cde1e6b80 1 -- 172.21.15.52:0/3282669629 shutdown_connections 2024-06-10T02:57:08.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.846+0000 7f1cde1e6b80 1 --2- 172.21.15.52:0/3282669629 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ebc1a0f880 0x55ebc1a043c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:08.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.846+0000 7f1cde1e6b80 1 --2- 172.21.15.52:0/3282669629 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ebc1a097a0 0x55ebc1a09260 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:08.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.846+0000 7f1cde1e6b80 1 --2- 172.21.15.52:0/3282669629 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ebc1a0ef60 0x55ebc1a03e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:08.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.846+0000 7f1cde1e6b80 1 -- 172.21.15.52:0/3282669629 >> 172.21.15.52:0/3282669629 conn(0x55ebc1936080 msgr2=0x55ebc1a0e2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:57:08.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.846+0000 7f1cde1e6b80 1 -- 172.21.15.52:0/3282669629 shutdown_connections 2024-06-10T02:57:08.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.846+0000 7f1cde1e6b80 1 -- 172.21.15.52:0/3282669629 wait complete. 2024-06-10T02:57:08.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.846+0000 7f1cde1e6b80 1 -- start start 2024-06-10T02:57:08.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.846+0000 7f1cde1e6b80 1 Processor -- start 2024-06-10T02:57:08.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.847+0000 7f1cde1e6b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ebc1a097a0 0x55ebc1a2a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:57:08.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.847+0000 7f1cde1e6b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ebc1a0ef60 0x55ebc1a2abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:57:08.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.847+0000 7f1cde1e6b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ebc1a0f880 0x55ebc1a2b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:57:08.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.847+0000 7f1cde1e6b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f1ccc1b1990 con 0x55ebc1a0f880 2024-06-10T02:57:08.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.847+0000 7f1cde1e6b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1ccc1b12c0 con 0x55ebc1a0ef60 2024-06-10T02:57:08.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.847+0000 7f1cde1e6b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f1ccc1ae4c0 con 0x55ebc1a097a0 2024-06-10T02:57:08.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.847+0000 7f1cde1e6b80 10 client.0 _ll_register_callbacks cb 0x55ebc18f05b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T02:57:08.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.847+0000 7f1cd1f00700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ebc1a0f880 0x55ebc1a2b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:57:08.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.847+0000 7f1cd16ff700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ebc1a097a0 0x55ebc1a2a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:57:08.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.847+0000 7f1cd16ff700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ebc1a097a0 0x55ebc1a2a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.52:35582/0 (socket says 172.21.15.52:35582) 2024-06-10T02:57:08.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.847+0000 7f1cd1f00700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ebc1a0f880 0x55ebc1a2b120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:42598/0 (socket says 172.21.15.52:42598) 2024-06-10T02:57:08.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.847+0000 7f1cd16ff700 1 -- 172.21.15.52:0/2366356648 learned_addr learned my addr 172.21.15.52:0/2366356648 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:57:08.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.847+0000 7f1cd0efe700 1 --2- 172.21.15.52:0/2366356648 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ebc1a0ef60 0x55ebc1a2abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:57:08.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.847+0000 7f1cd1f00700 1 -- 172.21.15.52:0/2366356648 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ebc1a097a0 msgr2=0x55ebc1a2a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:57:08.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.847+0000 7f1cd1f00700 1 --2- 172.21.15.52:0/2366356648 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ebc1a097a0 0x55ebc1a2a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:08.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.847+0000 7f1cd1f00700 1 -- 172.21.15.52:0/2366356648 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ebc1a0ef60 msgr2=0x55ebc1a2abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:57:08.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.847+0000 7f1cd1f00700 1 --2- 172.21.15.52:0/2366356648 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ebc1a0ef60 0x55ebc1a2abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:08.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.847+0000 7f1cd1f00700 1 -- 172.21.15.52:0/2366356648 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ebc1a8ca20 con 0x55ebc1a0f880 2024-06-10T02:57:08.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.848+0000 7f1cd1f00700 1 --2- 172.21.15.52:0/2366356648 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ebc1a0f880 0x55ebc1a2b120 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f1cc4000f80 tx=0x7f1cc4005e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=818b34142de59a34 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:57:08.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.848+0000 7f1cb4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-10T02:57:08.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.848+0000 7f1cb4ff9700 1 -- 172.21.15.52:0/2366356648 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cc401a040 con 0x55ebc1a0f880 2024-06-10T02:57:08.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.848+0000 7f1cb4ff9700 1 -- 172.21.15.52:0/2366356648 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f1cc4004ca0 con 0x55ebc1a0f880 2024-06-10T02:57:08.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.848+0000 7f1cde1e6b80 1 -- 172.21.15.52:0/2366356648 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55ebc19622d0 con 0x55ebc1a0f880 2024-06-10T02:57:08.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.848+0000 7f1cb4ff9700 1 -- 172.21.15.52:0/2366356648 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f1cc4036050 con 0x55ebc1a0f880 2024-06-10T02:57:08.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.848+0000 7f1cde1e6b80 10 client.15936 fetch_fsmap learned FSMap version 31 2024-06-10T02:57:08.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.848+0000 7f1cde1e6b80 1 -- 172.21.15.52:0/2366356648 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55ebc1a8dd10 con 0x55ebc1a0f880 2024-06-10T02:57:08.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.848+0000 7f1cb4ff9700 1 -- 172.21.15.52:0/2366356648 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55ebc19622d0 con 0x55ebc1a0f880 2024-06-10T02:57:08.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.848+0000 7f1cb4ff9700 1 -- 172.21.15.52:0/2366356648 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 31) ==== 8366+0+0 (secure 0 0 0) 0x7f1cc4031030 con 0x55ebc1a0f880 2024-06-10T02:57:08.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.848+0000 7f1cde1e6b80 10 client.15936 fetch_fsmap finished waiting for FSMap version 31 2024-06-10T02:57:08.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.848+0000 7f1cde1e6b80 10 client.15936 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T02:57:08.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.848+0000 7f1cde1e6b80 20 client.15936 populate_metadata read hostname 'smithi052' 2024-06-10T02:57:08.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.848+0000 7f1cde1e6b80 1 --2- 172.21.15.52:0/2366356648 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55ebc1a8e4a0 0x55ebc1a908c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:57:08.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.848+0000 7f1cde1e6b80 4 client.15936 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-10T02:57:08.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.848+0000 7f1cde1e6b80 1 -- 172.21.15.52:0/2366356648 --> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ebc1915f50 con 0x55ebc1a8e4a0 2024-06-10T02:57:08.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.849+0000 7f1cd16ff700 1 --2- 172.21.15.52:0/2366356648 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55ebc1a8e4a0 0x55ebc1a908c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:57:08.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.849+0000 7f1cd16ff700 1 --2- 172.21.15.52:0/2366356648 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55ebc1a8e4a0 0x55ebc1a908c0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d1b4da6faa119a53 server_cookie=4e65f70356e18887 in_seq=0 out_seq=0 2024-06-10T02:57:08.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.849+0000 7f1cb4ff9700 10 client.15936 ms_handle_connect on v2:172.21.15.188:6838/310444836 2024-06-10T02:57:08.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.849+0000 7f1cb4ff9700 1 -- 172.21.15.52:0/2366356648 <== mds.0 v2:172.21.15.188:6838/310444836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55ebc1915f50 con 0x55ebc1a8e4a0 2024-06-10T02:57:08.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.849+0000 7f1cb4ff9700 10 client.15936 handle_command_reply: tid=0 2024-06-10T02:57:08.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.849+0000 7f1cde1e6b80 1 client.15936 shutdown 2024-06-10T02:57:08.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.849+0000 7f1cde1e6b80 1 -- 172.21.15.52:0/2366356648 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ebc1a0f880 msgr2=0x55ebc1a2b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:57:08.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.849+0000 7f1cde1e6b80 1 --2- 172.21.15.52:0/2366356648 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ebc1a0f880 0x55ebc1a2b120 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f1cc4000f80 tx=0x7f1cc4005e90 comp rx=0 tx=0).stop 2024-06-10T02:57:08.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.849+0000 7f1cde1e6b80 1 -- 172.21.15.52:0/2366356648 shutdown_connections 2024-06-10T02:57:08.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.849+0000 7f1cde1e6b80 1 --2- 172.21.15.52:0/2366356648 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55ebc1a8e4a0 0x55ebc1a908c0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:08.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.849+0000 7f1cde1e6b80 1 --2- 172.21.15.52:0/2366356648 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ebc1a0f880 0x55ebc1a2b120 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:08.854 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.849+0000 7f1cde1e6b80 1 --2- 172.21.15.52:0/2366356648 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ebc1a097a0 0x55ebc1a2a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:08.854 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.849+0000 7f1cb4ff9700 0 client.15936 ms_handle_reset on v2:172.21.15.188:6838/310444836 2024-06-10T02:57:08.854 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.849+0000 7f1cde1e6b80 1 --2- 172.21.15.52:0/2366356648 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ebc1a0ef60 0x55ebc1a2abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:08.854 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.849+0000 7f1cde1e6b80 1 -- 172.21.15.52:0/2366356648 >> 172.21.15.52:0/2366356648 conn(0x55ebc1936080 msgr2=0x55ebc1a12100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:57:08.854 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.850+0000 7f1cde1e6b80 1 -- 172.21.15.52:0/2366356648 shutdown_connections 2024-06-10T02:57:08.854 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.850+0000 7f1cde1e6b80 1 -- 172.21.15.52:0/2366356648 wait complete. 2024-06-10T02:57:08.854 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.850+0000 7f1cde1e6b80 20 client.15936 trim_cache size 0 max 16384 2024-06-10T02:57:08.854 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-10T02:57:08.855 INFO:teuthology.orchestra.run.smithi052.stdout: "status": "no active scrubs running", 2024-06-10T02:57:08.855 INFO:teuthology.orchestra.run.smithi052.stdout: "scrubs": {} 2024-06-10T02:57:08.855 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-10T02:57:08.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.850+0000 7f1cd3963700 1 -- 172.21.15.52:0/1046963044 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f1cb80a2c20 msgr2=0x7f1cb80a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:57:08.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.850+0000 7f1cd3963700 1 --2- 172.21.15.52:0/1046963044 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f1cb80a2c20 0x7f1cb80a50e0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f1ccc102880 tx=0x7f1cbc009260 comp rx=0 tx=0).stop 2024-06-10T02:57:08.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.850+0000 7f1cd3963700 1 -- 172.21.15.52:0/1046963044 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1ccc06a1a0 msgr2=0x7f1ccc19af00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:57:08.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.850+0000 7f1cd3963700 1 --2- 172.21.15.52:0/1046963044 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1ccc06a1a0 0x7f1ccc19af00 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f1cc8005910 tx=0x7f1cc800bab0 comp rx=0 tx=0).stop 2024-06-10T02:57:08.856 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.850+0000 7f1cd3963700 1 -- 172.21.15.52:0/1046963044 shutdown_connections 2024-06-10T02:57:08.856 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.850+0000 7f1cd3963700 1 --2- 172.21.15.52:0/1046963044 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f1cb80a2c20 0x7f1cb80a50e0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:08.856 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.850+0000 7f1cd3963700 1 --2- 172.21.15.52:0/1046963044 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1ccc10ef40 0x7f1ccc19f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:08.856 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.850+0000 7f1cd3963700 1 --2- 172.21.15.52:0/1046963044 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1ccc06a1a0 0x7f1ccc19af00 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:08.856 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.850+0000 7f1cd3963700 1 --2- 172.21.15.52:0/1046963044 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1ccc06aac0 0x7f1ccc19b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:08.856 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.850+0000 7f1cd3963700 1 -- 172.21.15.52:0/1046963044 >> 172.21.15.52:0/1046963044 conn(0x7f1ccc078340 msgr2=0x7f1ccc110390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:57:08.856 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.850+0000 7f1cd3963700 1 -- 172.21.15.52:0/1046963044 shutdown_connections 2024-06-10T02:57:08.856 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:08.850+0000 7f1cd3963700 1 -- 172.21.15.52:0/1046963044 wait complete. 2024-06-10T02:57:08.888 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:57:08 smithi120 ceph-mon[36634]: pgmap v522: 129 pgs: 129 active+clean; 126 MiB data, 532 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 6.3 MiB/s wr, 415 op/s 2024-06-10T02:57:08.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:57:08 smithi188 ceph-mon[36599]: pgmap v522: 129 pgs: 129 active+clean; 126 MiB data, 532 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 6.3 MiB/s wr, 415 op/s 2024-06-10T02:57:10.616 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-06-10T02:57:10.617 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T02:57:10.878 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:57:10.888 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:57:10 smithi120 ceph-mon[36634]: pgmap v523: 129 pgs: 129 active+clean; 137 MiB data, 521 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 7.0 MiB/s wr, 450 op/s 2024-06-10T02:57:10.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:57:10 smithi188 ceph-mon[36599]: pgmap v523: 129 pgs: 129 active+clean; 137 MiB data, 521 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 7.0 MiB/s wr, 450 op/s 2024-06-10T02:57:10.976 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:57:10 smithi052 ceph-mon[32362]: pgmap v523: 129 pgs: 129 active+clean; 137 MiB data, 521 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 7.0 MiB/s wr, 450 op/s 2024-06-10T02:57:12.726 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:57:12 smithi052 ceph-mon[32362]: pgmap v524: 129 pgs: 129 active+clean; 139 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 6.5 MiB/s wr, 437 op/s 2024-06-10T02:57:12.763 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:12.761+0000 7f437b9a7700 1 -- 172.21.15.52:0/3758031039 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4374111060 msgr2=0x7f43741099d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:57:12.764 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:12.761+0000 7f437b9a7700 1 --2- 172.21.15.52:0/3758031039 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4374111060 0x7f43741099d0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f43700098f0 tx=0x7f4370009c00 comp rx=0 tx=0).stop 2024-06-10T02:57:12.765 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:12.765+0000 7f437b9a7700 1 -- 172.21.15.52:0/3758031039 shutdown_connections 2024-06-10T02:57:12.765 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:12.765+0000 7f437b9a7700 1 --2- 172.21.15.52:0/3758031039 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f437411bc70 0x7f437411e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:12.765 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:12.765+0000 7f437b9a7700 1 --2- 172.21.15.52:0/3758031039 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4374110bf0 0x7f4374109490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:12.765 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:12.765+0000 7f437b9a7700 1 --2- 172.21.15.52:0/3758031039 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4374111060 0x7f43741099d0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:12.766 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:12.765+0000 7f437b9a7700 1 -- 172.21.15.52:0/3758031039 >> 172.21.15.52:0/3758031039 conn(0x7f437406d600 msgr2=0x7f437406da10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:57:12.771 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:12.770+0000 7f437b9a7700 1 -- 172.21.15.52:0/3758031039 shutdown_connections 2024-06-10T02:57:12.772 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:12.772+0000 7f437b9a7700 1 -- 172.21.15.52:0/3758031039 wait complete. 2024-06-10T02:57:12.773 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:12.773+0000 7f437b9a7700 1 Processor -- start 2024-06-10T02:57:12.773 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:12.773+0000 7f437b9a7700 1 -- start start 2024-06-10T02:57:12.774 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:12.773+0000 7f437b9a7700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4374110bf0 0x7f437411b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:57:12.774 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:12.773+0000 7f437b9a7700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f437411bc70 0x7f43741172f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:57:12.774 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:12.773+0000 7f437b9a7700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4374117830 0x7f4374117ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:57:12.774 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:12.773+0000 7f437b9a7700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f4374113dc0 con 0x7f4374110bf0 2024-06-10T02:57:12.774 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:12.773+0000 7f437b9a7700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f4374113c40 con 0x7f437411bc70 2024-06-10T02:57:12.774 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:12.773+0000 7f437b9a7700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f4374113f40 con 0x7f4374117830 2024-06-10T02:57:12.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:12.777+0000 7f4379f44700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4374117830 0x7f4374117ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:57:12.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:12.777+0000 7f4379f44700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4374117830 0x7f4374117ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.52:40198/0 (socket says 172.21.15.52:40198) 2024-06-10T02:57:12.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:12.777+0000 7f4379f44700 1 -- 172.21.15.52:0/1617684893 learned_addr learned my addr 172.21.15.52:0/1617684893 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:57:12.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:12.777+0000 7f4379f44700 1 -- 172.21.15.52:0/1617684893 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f437411bc70 msgr2=0x7f43741172f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-10T02:57:12.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:12.777+0000 7f4379f44700 1 --2- 172.21.15.52:0/1617684893 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f437411bc70 0x7f43741172f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:12.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:12.777+0000 7f4379f44700 1 -- 172.21.15.52:0/1617684893 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4374110bf0 msgr2=0x7f437411b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:57:12.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:12.777+0000 7f4379743700 1 --2- 172.21.15.52:0/1617684893 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4374110bf0 0x7f437411b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:57:12.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:12.777+0000 7f4379f44700 1 --2- 172.21.15.52:0/1617684893 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4374110bf0 0x7f437411b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:12.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:12.777+0000 7f4379f44700 1 -- 172.21.15.52:0/1617684893 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43741aba60 con 0x7f4374117830 2024-06-10T02:57:12.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:12.777+0000 7f4379743700 1 --2- 172.21.15.52:0/1617684893 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4374110bf0 0x7f437411b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T02:57:12.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:12.778+0000 7f4379f44700 1 --2- 172.21.15.52:0/1617684893 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4374117830 0x7f4374117ce0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f436400c900 tx=0x7f436400ccc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:57:12.781 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:12.781+0000 7f436a7fc700 1 -- 172.21.15.52:0/1617684893 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4364007b00 con 0x7f4374117830 2024-06-10T02:57:12.781 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:12.781+0000 7f437b9a7700 1 -- 172.21.15.52:0/1617684893 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43741abc30 con 0x7f4374117830 2024-06-10T02:57:12.781 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:12.781+0000 7f437b9a7700 1 -- 172.21.15.52:0/1617684893 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43741ac170 con 0x7f4374117830 2024-06-10T02:57:12.781 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:12.781+0000 7f436a7fc700 1 -- 172.21.15.52:0/1617684893 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4364007ca0 con 0x7f4374117830 2024-06-10T02:57:12.781 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:12.781+0000 7f436a7fc700 1 -- 172.21.15.52:0/1617684893 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f436400e6a0 con 0x7f4374117830 2024-06-10T02:57:12.789 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:12.787+0000 7f437b9a7700 1 -- 172.21.15.52:0/1617684893 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4358005330 con 0x7f4374117830 2024-06-10T02:57:12.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:12.791+0000 7f436a7fc700 1 -- 172.21.15.52:0/1617684893 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f436400f450 con 0x7f4374117830 2024-06-10T02:57:12.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:12.792+0000 7f436a7fc700 1 --2- 172.21.15.52:0/1617684893 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f43600a2e20 0x7f43600a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:57:12.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:12.793+0000 7f436a7fc700 1 -- 172.21.15.52:0/1617684893 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f43640ceea0 con 0x7f4374117830 2024-06-10T02:57:12.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:12.796+0000 7f4379743700 1 --2- 172.21.15.52:0/1617684893 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f43600a2e20 0x7f43600a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:57:12.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:12.797+0000 7f4379743700 1 --2- 172.21.15.52:0/1617684893 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f43600a2e20 0x7f43600a52e0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f436c01a3c0 tx=0x7f436c01c040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:57:12.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:12.798+0000 7f436a7fc700 1 -- 172.21.15.52:0/1617684893 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f4364014030 con 0x7f4374117830 2024-06-10T02:57:12.888 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:57:12 smithi120 ceph-mon[36634]: pgmap v524: 129 pgs: 129 active+clean; 139 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 6.5 MiB/s wr, 437 op/s 2024-06-10T02:57:12.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:57:12 smithi188 ceph-mon[36599]: pgmap v524: 129 pgs: 129 active+clean; 139 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 6.5 MiB/s wr, 437 op/s 2024-06-10T02:57:13.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:13.113+0000 7f437b9a7700 1 -- 172.21.15.52:0/1617684893 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4358005620 con 0x7f4374117830 2024-06-10T02:57:13.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:13.114+0000 7f436a7fc700 1 -- 172.21.15.52:0/1617684893 <== mon.1 v2:172.21.15.188:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11939 (secure 0 0 0) 0x7f436409b9f0 con 0x7f4374117830 2024-06-10T02:57:13.114 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-10T02:57:13.114 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/889673247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":889673247},{"type":"v1","addr":"172.21.15.52:6835","nonce":889673247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/3615663511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":3615663511},{"type":"v1","addr":"172.21.15.52:6837","nonce":3615663511}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":31,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T02:50:03.889766+0000","modified":"2024-06-10T02:55:51.192225+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24484,"mds_1":24469,"mds_2":24463,"mds_3":24449,"mds_4":24428},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14571":{"gid":14571,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6839/3059599042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":3059599042},{"type":"v1","addr":"172.21.15.52:6839","nonce":3059599042}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14583":{"gid":14583,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6841/3039275521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":3039275521},{"type":"v1","addr":"172.21.15.52:6841","nonce":3039275521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24383":{"gid":24383,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6833/2609002896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":2609002896},{"type":"v1","addr":"172.21.15.120:6833","nonce":2609002896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6835/1996370288","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":1996370288},{"type":"v1","addr":"172.21.15.120:6835","nonce":1996370288}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6833/529564979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":529564979},{"type":"v1","addr":"172.21.15.188:6833","nonce":529564979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":25,"addr":"172.21.15.120:6837/4132440952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":4132440952},{"type":"v1","addr":"172.21.15.120:6837","nonce":4132440952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"b","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.120:6839/1672436487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1672436487},{"type":"v1","addr":"172.21.15.120:6839","nonce":1672436487}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":2,"incarnation":22,"state":"up:active","state_seq":26,"addr":"172.21.15.188:6835/3904544259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":3904544259},{"type":"v1","addr":"172.21.15.188:6835","nonce":3904544259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.188:6837/819317726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":819317726},{"type":"v1","addr":"172.21.15.188:6837","nonce":819317726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.188:6839/310444836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":310444836},{"type":"v1","addr":"172.21.15.188:6839","nonce":310444836}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24484,"qdb_cluster":[24469,24449,24428,24463,24484]},"id":1}]} 2024-06-10T02:57:13.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:13.118+0000 7f437b9a7700 1 -- 172.21.15.52:0/1617684893 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f43600a2e20 msgr2=0x7f43600a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:57:13.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:13.118+0000 7f437b9a7700 1 --2- 172.21.15.52:0/1617684893 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f43600a2e20 0x7f43600a52e0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f436c01a3c0 tx=0x7f436c01c040 comp rx=0 tx=0).stop 2024-06-10T02:57:13.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:13.118+0000 7f437b9a7700 1 -- 172.21.15.52:0/1617684893 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4374117830 msgr2=0x7f4374117ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:57:13.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:13.118+0000 7f437b9a7700 1 --2- 172.21.15.52:0/1617684893 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4374117830 0x7f4374117ce0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f436400c900 tx=0x7f436400ccc0 comp rx=0 tx=0).stop 2024-06-10T02:57:13.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:13.118+0000 7f437b9a7700 1 -- 172.21.15.52:0/1617684893 shutdown_connections 2024-06-10T02:57:13.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:13.118+0000 7f437b9a7700 1 --2- 172.21.15.52:0/1617684893 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4374117830 0x7f4374117ce0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:13.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:13.118+0000 7f437b9a7700 1 --2- 172.21.15.52:0/1617684893 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f43600a2e20 0x7f43600a52e0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:13.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:13.118+0000 7f437b9a7700 1 --2- 172.21.15.52:0/1617684893 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4374110bf0 0x7f437411b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:13.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:13.118+0000 7f437b9a7700 1 --2- 172.21.15.52:0/1617684893 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f437411bc70 0x7f43741172f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:13.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:13.118+0000 7f437b9a7700 1 -- 172.21.15.52:0/1617684893 >> 172.21.15.52:0/1617684893 conn(0x7f437406d600 msgr2=0x7f437411c050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:57:13.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:13.118+0000 7f437b9a7700 1 -- 172.21.15.52:0/1617684893 shutdown_connections 2024-06-10T02:57:13.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:13.118+0000 7f437b9a7700 1 -- 172.21.15.52:0/1617684893 wait complete. 2024-06-10T02:57:13.119 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 31 2024-06-10T02:57:13.727 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:57:13 smithi052 ceph-mon[32362]: from='client.? 172.21.15.52:0/1617684893' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T02:57:13.774 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-06-10T02:57:13.888 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:57:13 smithi120 ceph-mon[36634]: from='client.? 172.21.15.52:0/1617684893' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T02:57:13.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:57:13 smithi188 ceph-mon[36599]: from='client.? 172.21.15.52:0/1617684893' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T02:57:14.045 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:57:14.726 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:57:14 smithi052 ceph-mon[32362]: pgmap v525: 129 pgs: 129 active+clean; 138 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.2 MiB/s wr, 370 op/s 2024-06-10T02:57:14.727 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:57:14 smithi052 ceph-mon[32362]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-10T02:57:14.888 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:57:14 smithi120 ceph-mon[36634]: pgmap v525: 129 pgs: 129 active+clean; 138 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.2 MiB/s wr, 370 op/s 2024-06-10T02:57:14.888 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:57:14 smithi120 ceph-mon[36634]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-10T02:57:14.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:57:14 smithi188 ceph-mon[36599]: pgmap v525: 129 pgs: 129 active+clean; 138 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.2 MiB/s wr, 370 op/s 2024-06-10T02:57:14.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:57:14 smithi188 ceph-mon[36599]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-10T02:57:16.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.032+0000 7f6b07c89700 1 -- 172.21.15.52:0/439199216 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6b001085d0 msgr2=0x7f6b001089b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:57:16.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.032+0000 7f6b07c89700 1 --2- 172.21.15.52:0/439199216 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6b001085d0 0x7f6b001089b0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f6af000b0d0 tx=0x7f6af000b3e0 comp rx=0 tx=0).stop 2024-06-10T02:57:16.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.032+0000 7f6b07c89700 1 -- 172.21.15.52:0/439199216 shutdown_connections 2024-06-10T02:57:16.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.032+0000 7f6b07c89700 1 --2- 172.21.15.52:0/439199216 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6b00102ef0 0x7f6b0010f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:16.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.032+0000 7f6b07c89700 1 --2- 172.21.15.52:0/439199216 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6b001085d0 0x7f6b001089b0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:16.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.032+0000 7f6b07c89700 1 --2- 172.21.15.52:0/439199216 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6b00102530 0x7f6b001029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:16.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.032+0000 7f6b07c89700 1 -- 172.21.15.52:0/439199216 >> 172.21.15.52:0/439199216 conn(0x7f6b00079640 msgr2=0x7f6b00079a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:57:16.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.033+0000 7f6b07c89700 1 -- 172.21.15.52:0/439199216 shutdown_connections 2024-06-10T02:57:16.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.033+0000 7f6b07c89700 1 -- 172.21.15.52:0/439199216 wait complete. 2024-06-10T02:57:16.035 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.033+0000 7f6b07c89700 1 Processor -- start 2024-06-10T02:57:16.035 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.034+0000 7f6b07c89700 1 -- start start 2024-06-10T02:57:16.035 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.034+0000 7f6b07c89700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6b00102530 0x7f6b0019af20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:57:16.035 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.034+0000 7f6b07c89700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6b00102ef0 0x7f6b0019b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:57:16.035 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.034+0000 7f6b07c89700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6b001085d0 0x7f6b0019f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:57:16.035 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.034+0000 7f6b07c89700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f6b00112810 con 0x7f6b00102ef0 2024-06-10T02:57:16.035 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.034+0000 7f6b07c89700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f6b00112690 con 0x7f6b00102530 2024-06-10T02:57:16.035 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.034+0000 7f6b07c89700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f6b00112990 con 0x7f6b001085d0 2024-06-10T02:57:16.035 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.034+0000 7f6b05224700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6b00102ef0 0x7f6b0019b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:57:16.035 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.034+0000 7f6b05224700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6b00102ef0 0x7f6b0019b460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60426/0 (socket says 172.21.15.52:60426) 2024-06-10T02:57:16.036 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.034+0000 7f6b05224700 1 -- 172.21.15.52:0/1197077765 learned_addr learned my addr 172.21.15.52:0/1197077765 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:57:16.036 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.034+0000 7f6b05a25700 1 --2- 172.21.15.52:0/1197077765 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6b00102530 0x7f6b0019af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:57:16.036 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.034+0000 7f6b06226700 1 --2- 172.21.15.52:0/1197077765 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6b001085d0 0x7f6b0019f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:57:16.036 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.034+0000 7f6b05224700 1 -- 172.21.15.52:0/1197077765 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6b001085d0 msgr2=0x7f6b0019f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:57:16.036 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.034+0000 7f6b05224700 1 --2- 172.21.15.52:0/1197077765 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6b001085d0 0x7f6b0019f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:16.036 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.034+0000 7f6b05224700 1 -- 172.21.15.52:0/1197077765 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6b00102530 msgr2=0x7f6b0019af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:57:16.036 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.034+0000 7f6b05224700 1 --2- 172.21.15.52:0/1197077765 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6b00102530 0x7f6b0019af20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:16.036 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.034+0000 7f6b05224700 1 -- 172.21.15.52:0/1197077765 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b0019ff30 con 0x7f6b00102ef0 2024-06-10T02:57:16.036 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.035+0000 7f6b05224700 1 --2- 172.21.15.52:0/1197077765 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6b00102ef0 0x7f6b0019b460 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f6af400e9a0 tx=0x7f6af400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:57:16.036 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.035+0000 7f6afeffd700 1 -- 172.21.15.52:0/1197077765 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6af400ca10 con 0x7f6b00102ef0 2024-06-10T02:57:16.036 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.035+0000 7f6afeffd700 1 -- 172.21.15.52:0/1197077765 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6af400cbb0 con 0x7f6b00102ef0 2024-06-10T02:57:16.036 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.035+0000 7f6afeffd700 1 -- 172.21.15.52:0/1197077765 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6af4018680 con 0x7f6b00102ef0 2024-06-10T02:57:16.036 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.035+0000 7f6b07c89700 1 -- 172.21.15.52:0/1197077765 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b001a0220 con 0x7f6b00102ef0 2024-06-10T02:57:16.036 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.035+0000 7f6b07c89700 1 -- 172.21.15.52:0/1197077765 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b001a7aa0 con 0x7f6b00102ef0 2024-06-10T02:57:16.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.038+0000 7f6afeffd700 1 -- 172.21.15.52:0/1197077765 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f6af4010430 con 0x7f6b00102ef0 2024-06-10T02:57:16.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.038+0000 7f6afeffd700 1 --2- 172.21.15.52:0/1197077765 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f6ae80a2c20 0x7f6ae80a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:57:16.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.038+0000 7f6b05a25700 1 --2- 172.21.15.52:0/1197077765 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f6ae80a2c20 0x7f6ae80a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:57:16.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.038+0000 7f6afeffd700 1 -- 172.21.15.52:0/1197077765 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f6af40cf630 con 0x7f6b00102ef0 2024-06-10T02:57:16.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.039+0000 7f6b05a25700 1 --2- 172.21.15.52:0/1197077765 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f6ae80a2c20 0x7f6ae80a50e0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f6af0009fd0 tx=0x7f6af000b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:57:16.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.039+0000 7f6b1250cb80 1 Processor -- start 2024-06-10T02:57:16.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.039+0000 7f6b1250cb80 1 -- start start 2024-06-10T02:57:16.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.039+0000 7f6b1250cb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5634cdce1640 0x5634cdce1a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:57:16.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.039+0000 7f6b1250cb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5634cdce1f60 0x5634cdcdb9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:57:16.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.039+0000 7f6b1250cb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5634cdcdbf20 0x5634cdcdc340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:57:16.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.039+0000 7f6b1250cb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5634cdc2d810 con 0x5634cdce1640 2024-06-10T02:57:16.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.039+0000 7f6b1250cb80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5634cdbd5720 con 0x5634cdcdbf20 2024-06-10T02:57:16.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.039+0000 7f6b1250cb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5634cdc297d0 con 0x5634cdce1f60 2024-06-10T02:57:16.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.040+0000 7f6b05a25700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5634cdce1640 0x5634cdce1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:57:16.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.040+0000 7f6b05224700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5634cdce1f60 0x5634cdcdb9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:57:16.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.040+0000 7f6b05a25700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5634cdce1640 0x5634cdce1a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60432/0 (socket says 172.21.15.52:60432) 2024-06-10T02:57:16.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.040+0000 7f6b05a25700 1 -- 172.21.15.52:0/3725667159 learned_addr learned my addr 172.21.15.52:0/3725667159 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:57:16.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.040+0000 7f6b06226700 1 --2- 172.21.15.52:0/3725667159 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5634cdcdbf20 0x5634cdcdc340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:57:16.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.040+0000 7f6b05224700 1 -- 172.21.15.52:0/3725667159 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5634cdcdbf20 msgr2=0x5634cdcdc340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:57:16.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.040+0000 7f6b05224700 1 --2- 172.21.15.52:0/3725667159 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5634cdcdbf20 0x5634cdcdc340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:16.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.040+0000 7f6b05224700 1 -- 172.21.15.52:0/3725667159 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5634cdce1640 msgr2=0x5634cdce1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:57:16.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.040+0000 7f6b05224700 1 --2- 172.21.15.52:0/3725667159 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5634cdce1640 0x5634cdce1a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:16.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.040+0000 7f6b05224700 1 -- 172.21.15.52:0/3725667159 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5634cdbcbf50 con 0x5634cdce1f60 2024-06-10T02:57:16.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.040+0000 7f6b05224700 1 --2- 172.21.15.52:0/3725667159 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5634cdce1f60 0x5634cdcdb9e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f6af400c2d0 tx=0x7f6af409c1e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e5ca5ee6fb13619e server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:57:16.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.040+0000 7f6ae7fff700 1 -- 172.21.15.52:0/3725667159 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6af409fe90 con 0x5634cdce1f60 2024-06-10T02:57:16.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.040+0000 7f6ae7fff700 1 -- 172.21.15.52:0/3725667159 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6af40a4750 con 0x5634cdce1f60 2024-06-10T02:57:16.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.040+0000 7f6ae7fff700 1 -- 172.21.15.52:0/3725667159 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6af4010720 con 0x5634cdce1f60 2024-06-10T02:57:16.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.041+0000 7f6b1250cb80 1 -- 172.21.15.52:0/3725667159 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5634cdce1f60 msgr2=0x5634cdcdb9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:57:16.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.041+0000 7f6b1250cb80 1 --2- 172.21.15.52:0/3725667159 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5634cdce1f60 0x5634cdcdb9e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f6af400c2d0 tx=0x7f6af409c1e0 comp rx=0 tx=0).stop 2024-06-10T02:57:16.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.041+0000 7f6b1250cb80 1 -- 172.21.15.52:0/3725667159 shutdown_connections 2024-06-10T02:57:16.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.041+0000 7f6b1250cb80 1 --2- 172.21.15.52:0/3725667159 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5634cdcdbf20 0x5634cdcdc340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:16.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.041+0000 7f6b1250cb80 1 --2- 172.21.15.52:0/3725667159 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5634cdce1640 0x5634cdce1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:16.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.041+0000 7f6b1250cb80 1 --2- 172.21.15.52:0/3725667159 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5634cdce1f60 0x5634cdcdb9e0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:16.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.041+0000 7f6b1250cb80 1 -- 172.21.15.52:0/3725667159 >> 172.21.15.52:0/3725667159 conn(0x5634cdc06cc0 msgr2=0x5634cdcd6850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:57:16.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.041+0000 7f6b1250cb80 1 -- 172.21.15.52:0/3725667159 shutdown_connections 2024-06-10T02:57:16.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.041+0000 7f6b1250cb80 1 -- 172.21.15.52:0/3725667159 wait complete. 2024-06-10T02:57:16.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.042+0000 7f6b1250cb80 1 -- start start 2024-06-10T02:57:16.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.042+0000 7f6b1250cb80 1 Processor -- start 2024-06-10T02:57:16.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.042+0000 7f6b1250cb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5634cdcdbf20 0x5634cdce7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:57:16.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.042+0000 7f6b1250cb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5634cdce1640 0x5634cdce4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:57:16.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.042+0000 7f6b1250cb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5634cdce1f60 0x5634cdce5510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:57:16.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.042+0000 7f6b1250cb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5634cdce89d0 con 0x5634cdce1f60 2024-06-10T02:57:16.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.042+0000 7f6b1250cb80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5634cdbcdbf0 con 0x5634cdcdbf20 2024-06-10T02:57:16.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.042+0000 7f6b1250cb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5634cdce8b50 con 0x5634cdce1640 2024-06-10T02:57:16.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.042+0000 7f6b1250cb80 10 client.0 _ll_register_callbacks cb 0x5634cdbc7920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T02:57:16.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.042+0000 7f6b06226700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5634cdce1f60 0x5634cdce5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:57:16.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.042+0000 7f6b06226700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5634cdce1f60 0x5634cdce5510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60436/0 (socket says 172.21.15.52:60436) 2024-06-10T02:57:16.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.042+0000 7f6b06226700 1 -- 172.21.15.52:0/1623477335 learned_addr learned my addr 172.21.15.52:0/1623477335 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:57:16.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.042+0000 7f6b05a25700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5634cdcdbf20 0x5634cdce7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:57:16.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.043+0000 7f6b05224700 1 --2- 172.21.15.52:0/1623477335 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5634cdce1640 0x5634cdce4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:57:16.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.043+0000 7f6b06226700 1 -- 172.21.15.52:0/1623477335 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5634cdce1640 msgr2=0x5634cdce4fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:57:16.046 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.043+0000 7f6b06226700 1 --2- 172.21.15.52:0/1623477335 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5634cdce1640 0x5634cdce4fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:16.046 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.043+0000 7f6b06226700 1 -- 172.21.15.52:0/1623477335 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5634cdcdbf20 msgr2=0x5634cdce7cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:57:16.046 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.043+0000 7f6b06226700 1 --2- 172.21.15.52:0/1623477335 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5634cdcdbf20 0x5634cdce7cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:16.046 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.043+0000 7f6b06226700 1 -- 172.21.15.52:0/1623477335 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5634cdb7c8d0 con 0x5634cdce1f60 2024-06-10T02:57:16.046 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.043+0000 7f6b05224700 1 --2- 172.21.15.52:0/1623477335 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5634cdce1640 0x5634cdce4fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T02:57:16.046 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.043+0000 7f6b06226700 1 --2- 172.21.15.52:0/1623477335 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5634cdce1f60 0x5634cdce5510 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f6af8011780 tx=0x7f6af8011b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=896ea7e9d9eb5e82 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:57:16.046 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.043+0000 7f6ae57fa700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-10T02:57:16.046 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.043+0000 7f6ae57fa700 1 -- 172.21.15.52:0/1623477335 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6af800f690 con 0x5634cdce1f60 2024-06-10T02:57:16.046 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.043+0000 7f6ae57fa700 1 -- 172.21.15.52:0/1623477335 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f6af800f830 con 0x5634cdce1f60 2024-06-10T02:57:16.046 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.043+0000 7f6b1250cb80 1 -- 172.21.15.52:0/1623477335 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6af40a4750 con 0x5634cdce1f60 2024-06-10T02:57:16.046 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.043+0000 7f6ae57fa700 1 -- 172.21.15.52:0/1623477335 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f6af8027050 con 0x5634cdce1f60 2024-06-10T02:57:16.046 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.043+0000 7f6b1250cb80 10 client.15951 fetch_fsmap learned FSMap version 32 2024-06-10T02:57:16.047 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.043+0000 7f6b1250cb80 1 -- 172.21.15.52:0/1623477335 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5634cdcf3330 con 0x5634cdce1f60 2024-06-10T02:57:16.047 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.044+0000 7f6ae57fa700 1 -- 172.21.15.52:0/1623477335 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6af40a4750 con 0x5634cdce1f60 2024-06-10T02:57:16.047 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.044+0000 7f6ae57fa700 1 -- 172.21.15.52:0/1623477335 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7f6af8022030 con 0x5634cdce1f60 2024-06-10T02:57:16.047 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.044+0000 7f6b1250cb80 10 client.15951 fetch_fsmap finished waiting for FSMap version 32 2024-06-10T02:57:16.047 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.044+0000 7f6b1250cb80 10 client.15951 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-06-10T02:57:16.047 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.044+0000 7f6b1250cb80 20 client.15951 populate_metadata read hostname 'smithi052' 2024-06-10T02:57:16.047 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.044+0000 7f6b1250cb80 1 --2- 172.21.15.52:0/1623477335 >> [v2:172.21.15.120:6836/4132440952,v1:172.21.15.120:6837/4132440952] conn(0x5634cdcf5970 0x5634cdcf7d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:57:16.047 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.044+0000 7f6b1250cb80 4 client.15951 mds_command: new command op to 24428 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T02:57:16.047 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.044+0000 7f6b1250cb80 1 -- 172.21.15.52:0/1623477335 --> [v2:172.21.15.120:6836/4132440952,v1:172.21.15.120:6837/4132440952] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5634cdc3ad10 con 0x5634cdcf5970 2024-06-10T02:57:16.047 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.044+0000 7f6b05a25700 1 --2- 172.21.15.52:0/1623477335 >> [v2:172.21.15.120:6836/4132440952,v1:172.21.15.120:6837/4132440952] conn(0x5634cdcf5970 0x5634cdcf7d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:57:16.048 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.045+0000 7f6ae57fa700 10 client.15951 ms_handle_connect on v2:172.21.15.120:6836/4132440952 2024-06-10T02:57:16.048 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.045+0000 7f6b05a25700 1 --2- 172.21.15.52:0/1623477335 >> [v2:172.21.15.120:6836/4132440952,v1:172.21.15.120:6837/4132440952] conn(0x5634cdcf5970 0x5634cdcf7d90 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=84e8648e56a40eeb server_cookie=5bea3b50ee10fd4b in_seq=0 out_seq=0 2024-06-10T02:57:16.048 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.045+0000 7f6ae57fa700 1 -- 172.21.15.52:0/1623477335 <== mds.4 v2:172.21.15.120:6836/4132440952 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5634cdc3ad10 con 0x5634cdcf5970 2024-06-10T02:57:16.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.046+0000 7f6ae57fa700 10 client.15951 handle_command_reply: tid=0 2024-06-10T02:57:16.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.046+0000 7f6b1250cb80 1 client.15951 shutdown 2024-06-10T02:57:16.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.046+0000 7f6b1250cb80 1 -- 172.21.15.52:0/1623477335 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5634cdce1f60 msgr2=0x5634cdce5510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:57:16.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.046+0000 7f6b1250cb80 1 --2- 172.21.15.52:0/1623477335 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5634cdce1f60 0x5634cdce5510 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f6af8011780 tx=0x7f6af8011b40 comp rx=0 tx=0).stop 2024-06-10T02:57:16.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.046+0000 7f6b1250cb80 1 -- 172.21.15.52:0/1623477335 shutdown_connections 2024-06-10T02:57:16.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.046+0000 7f6b1250cb80 1 --2- 172.21.15.52:0/1623477335 >> [v2:172.21.15.120:6836/4132440952,v1:172.21.15.120:6837/4132440952] conn(0x5634cdcf5970 0x5634cdcf7d90 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:16.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.046+0000 7f6b1250cb80 1 --2- 172.21.15.52:0/1623477335 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5634cdce1f60 0x5634cdce5510 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:16.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.046+0000 7f6b1250cb80 1 --2- 172.21.15.52:0/1623477335 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5634cdcdbf20 0x5634cdce7cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:16.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.046+0000 7f6b1250cb80 1 --2- 172.21.15.52:0/1623477335 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5634cdce1640 0x5634cdce4fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:16.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.046+0000 7f6ae57fa700 0 client.15951 ms_handle_reset on v2:172.21.15.120:6836/4132440952 2024-06-10T02:57:16.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.046+0000 7f6b1250cb80 1 -- 172.21.15.52:0/1623477335 >> 172.21.15.52:0/1623477335 conn(0x5634cdc06cc0 msgr2=0x5634cdcd65f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:57:16.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.046+0000 7f6b1250cb80 1 -- 172.21.15.52:0/1623477335 shutdown_connections 2024-06-10T02:57:16.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.046+0000 7f6b1250cb80 1 -- 172.21.15.52:0/1623477335 wait complete. 2024-06-10T02:57:16.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.046+0000 7f6b1250cb80 20 client.15951 trim_cache size 0 max 16384 2024-06-10T02:57:16.061 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.061+0000 7f6b1250cb80 1 Processor -- start 2024-06-10T02:57:16.062 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.062+0000 7f6b1250cb80 1 -- start start 2024-06-10T02:57:16.062 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.062+0000 7f6b1250cb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5634cdcdbf20 0x5634cdcdb9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:57:16.063 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.062+0000 7f6b1250cb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5634cdce1640 0x5634cdce38c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:57:16.063 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.062+0000 7f6b1250cb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5634cdce1f60 0x5634cdce3e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:57:16.063 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.062+0000 7f6b1250cb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f6b001b0690 con 0x5634cdce1640 2024-06-10T02:57:16.063 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.062+0000 7f6b1250cb80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f6b001b2230 con 0x5634cdcdbf20 2024-06-10T02:57:16.063 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.062+0000 7f6b1250cb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f6b001b0d60 con 0x5634cdce1f60 2024-06-10T02:57:16.063 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.062+0000 7f6b05224700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5634cdce1640 0x5634cdce38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:57:16.063 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.062+0000 7f6b05224700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5634cdce1640 0x5634cdce38c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60446/0 (socket says 172.21.15.52:60446) 2024-06-10T02:57:16.063 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.062+0000 7f6b05224700 1 -- 172.21.15.52:0/548417853 learned_addr learned my addr 172.21.15.52:0/548417853 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:57:16.063 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.062+0000 7f6b05a25700 1 --2- 172.21.15.52:0/548417853 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5634cdcdbf20 0x5634cdcdb9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:57:16.063 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.062+0000 7f6b06226700 1 --2- 172.21.15.52:0/548417853 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5634cdce1f60 0x5634cdce3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:57:16.063 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.062+0000 7f6b05224700 1 -- 172.21.15.52:0/548417853 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5634cdce1f60 msgr2=0x5634cdce3e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:57:16.063 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.062+0000 7f6b05224700 1 --2- 172.21.15.52:0/548417853 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5634cdce1f60 0x5634cdce3e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:16.063 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.062+0000 7f6b05224700 1 -- 172.21.15.52:0/548417853 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5634cdcdbf20 msgr2=0x5634cdcdb9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:57:16.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.062+0000 7f6b05224700 1 --2- 172.21.15.52:0/548417853 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5634cdcdbf20 0x5634cdcdb9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:16.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.062+0000 7f6b05224700 1 -- 172.21.15.52:0/548417853 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5634cdce3540 con 0x5634cdce1640 2024-06-10T02:57:16.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.062+0000 7f6b06226700 1 --2- 172.21.15.52:0/548417853 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5634cdce1f60 0x5634cdce3e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T02:57:16.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.062+0000 7f6b05224700 1 --2- 172.21.15.52:0/548417853 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5634cdce1640 0x5634cdce38c0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f6af409ba70 tx=0x7f6af409b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64cb20c5147df794 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:57:16.068 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.062+0000 7f6ae77fe700 1 -- 172.21.15.52:0/548417853 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6af40a4750 con 0x5634cdce1640 2024-06-10T02:57:16.068 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.063+0000 7f6ae77fe700 1 -- 172.21.15.52:0/548417853 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6af400cd60 con 0x5634cdce1640 2024-06-10T02:57:16.068 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.063+0000 7f6ae77fe700 1 -- 172.21.15.52:0/548417853 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6af40ba4c0 con 0x5634cdce1640 2024-06-10T02:57:16.068 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.063+0000 7f6b1250cb80 1 -- 172.21.15.52:0/548417853 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5634cdce1640 msgr2=0x5634cdce38c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:57:16.068 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.063+0000 7f6b1250cb80 1 --2- 172.21.15.52:0/548417853 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5634cdce1640 0x5634cdce38c0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f6af409ba70 tx=0x7f6af409b4f0 comp rx=0 tx=0).stop 2024-06-10T02:57:16.068 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.063+0000 7f6b1250cb80 1 -- 172.21.15.52:0/548417853 shutdown_connections 2024-06-10T02:57:16.068 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.063+0000 7f6b1250cb80 1 --2- 172.21.15.52:0/548417853 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5634cdce1f60 0x5634cdce3e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:16.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.063+0000 7f6b1250cb80 1 --2- 172.21.15.52:0/548417853 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5634cdcdbf20 0x5634cdcdb9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:16.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.063+0000 7f6b1250cb80 1 --2- 172.21.15.52:0/548417853 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5634cdce1640 0x5634cdce38c0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:16.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.063+0000 7f6b1250cb80 1 -- 172.21.15.52:0/548417853 >> 172.21.15.52:0/548417853 conn(0x5634cdc06cc0 msgr2=0x5634cdcd6380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:57:16.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.063+0000 7f6b1250cb80 1 -- 172.21.15.52:0/548417853 shutdown_connections 2024-06-10T02:57:16.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.063+0000 7f6b1250cb80 1 -- 172.21.15.52:0/548417853 wait complete. 2024-06-10T02:57:16.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.063+0000 7f6b1250cb80 1 -- start start 2024-06-10T02:57:16.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.064+0000 7f6b1250cb80 1 Processor -- start 2024-06-10T02:57:16.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.064+0000 7f6b1250cb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5634cdcdbf20 0x5634cdcedff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:57:16.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.064+0000 7f6b1250cb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5634cdce1640 0x5634cdcee530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:57:16.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.064+0000 7f6b1250cb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5634cdce1f60 0x5634cdcfca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:57:16.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.064+0000 7f6b1250cb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f6b001b1b30 con 0x5634cdce1f60 2024-06-10T02:57:16.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.064+0000 7f6b1250cb80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f6b001b1460 con 0x5634cdce1640 2024-06-10T02:57:16.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.064+0000 7f6b1250cb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f6b001ae630 con 0x5634cdcdbf20 2024-06-10T02:57:16.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.064+0000 7f6b1250cb80 10 client.0 _ll_register_callbacks cb 0x5634cdbc7920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T02:57:16.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.064+0000 7f6b05a25700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5634cdcdbf20 0x5634cdcedff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:57:16.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.064+0000 7f6b06226700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5634cdce1f60 0x5634cdcfca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:57:16.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.064+0000 7f6b05a25700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5634cdcdbf20 0x5634cdcedff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.52:40240/0 (socket says 172.21.15.52:40240) 2024-06-10T02:57:16.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.064+0000 7f6b05a25700 1 -- 172.21.15.52:0/4251891869 learned_addr learned my addr 172.21.15.52:0/4251891869 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:57:16.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.064+0000 7f6b05224700 1 --2- 172.21.15.52:0/4251891869 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5634cdce1640 0x5634cdcee530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:57:16.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.065+0000 7f6b06226700 1 -- 172.21.15.52:0/4251891869 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5634cdcdbf20 msgr2=0x5634cdcedff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:57:16.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.065+0000 7f6b06226700 1 --2- 172.21.15.52:0/4251891869 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5634cdcdbf20 0x5634cdcedff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:16.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.065+0000 7f6b06226700 1 -- 172.21.15.52:0/4251891869 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5634cdce1640 msgr2=0x5634cdcee530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:57:16.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.065+0000 7f6b06226700 1 --2- 172.21.15.52:0/4251891869 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5634cdce1640 0x5634cdcee530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:16.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.065+0000 7f6b06226700 1 -- 172.21.15.52:0/4251891869 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5634cdcfd270 con 0x5634cdce1f60 2024-06-10T02:57:16.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.065+0000 7f6b05224700 1 --2- 172.21.15.52:0/4251891869 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5634cdce1640 0x5634cdcee530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T02:57:16.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.065+0000 7f6b05a25700 1 --2- 172.21.15.52:0/4251891869 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5634cdcdbf20 0x5634cdcedff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T02:57:16.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.065+0000 7f6b06226700 1 --2- 172.21.15.52:0/4251891869 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5634cdce1f60 0x5634cdcfca30 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f6af801ad30 tx=0x7f6af8019640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1ff35a58c3cfcb4 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:57:16.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.065+0000 7f6ae57fa700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-10T02:57:16.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.065+0000 7f6ae57fa700 1 -- 172.21.15.52:0/4251891869 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6af80112d0 con 0x5634cdce1f60 2024-06-10T02:57:16.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.065+0000 7f6ae57fa700 1 -- 172.21.15.52:0/4251891869 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f6af80050d0 con 0x5634cdce1f60 2024-06-10T02:57:16.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.065+0000 7f6b1250cb80 1 -- 172.21.15.52:0/4251891869 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6af400cd60 con 0x5634cdce1f60 2024-06-10T02:57:16.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.065+0000 7f6ae57fa700 1 -- 172.21.15.52:0/4251891869 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f6af8038050 con 0x5634cdce1f60 2024-06-10T02:57:16.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.065+0000 7f6b1250cb80 10 client.15957 fetch_fsmap learned FSMap version 32 2024-06-10T02:57:16.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.065+0000 7f6b1250cb80 1 -- 172.21.15.52:0/4251891869 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5634cdd60050 con 0x5634cdce1f60 2024-06-10T02:57:16.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.065+0000 7f6ae57fa700 1 -- 172.21.15.52:0/4251891869 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6af400cd60 con 0x5634cdce1f60 2024-06-10T02:57:16.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.066+0000 7f6ae57fa700 1 -- 172.21.15.52:0/4251891869 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7f6af8022070 con 0x5634cdce1f60 2024-06-10T02:57:16.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.066+0000 7f6b1250cb80 10 client.15957 fetch_fsmap finished waiting for FSMap version 32 2024-06-10T02:57:16.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.066+0000 7f6b1250cb80 10 client.15957 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-06-10T02:57:16.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.066+0000 7f6b1250cb80 20 client.15957 populate_metadata read hostname 'smithi052' 2024-06-10T02:57:16.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.066+0000 7f6b1250cb80 1 --2- 172.21.15.52:0/4251891869 >> [v2:172.21.15.120:6836/4132440952,v1:172.21.15.120:6837/4132440952] conn(0x5634cdd60780 0x5634cdd62ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:57:16.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.066+0000 7f6b1250cb80 4 client.15957 mds_command: new command op to 24428 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T02:57:16.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.066+0000 7f6b1250cb80 1 -- 172.21.15.52:0/4251891869 --> [v2:172.21.15.120:6836/4132440952,v1:172.21.15.120:6837/4132440952] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5634cdc35290 con 0x5634cdd60780 2024-06-10T02:57:16.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.066+0000 7f6b05a25700 1 --2- 172.21.15.52:0/4251891869 >> [v2:172.21.15.120:6836/4132440952,v1:172.21.15.120:6837/4132440952] conn(0x5634cdd60780 0x5634cdd62ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:57:16.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.066+0000 7f6b05a25700 1 --2- 172.21.15.52:0/4251891869 >> [v2:172.21.15.120:6836/4132440952,v1:172.21.15.120:6837/4132440952] conn(0x5634cdd60780 0x5634cdd62ba0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d6ab874053ce86d5 server_cookie=d7309ed295570d91 in_seq=0 out_seq=0 2024-06-10T02:57:16.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.066+0000 7f6ae57fa700 10 client.15957 ms_handle_connect on v2:172.21.15.120:6836/4132440952 2024-06-10T02:57:16.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.066+0000 7f6ae57fa700 1 -- 172.21.15.52:0/4251891869 <== mds.4 v2:172.21.15.120:6836/4132440952 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5634cdc35290 con 0x5634cdd60780 2024-06-10T02:57:16.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.066+0000 7f6ae57fa700 10 client.15957 handle_command_reply: tid=0 2024-06-10T02:57:16.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.066+0000 7f6b1250cb80 1 client.15957 shutdown 2024-06-10T02:57:16.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.066+0000 7f6b1250cb80 1 -- 172.21.15.52:0/4251891869 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5634cdce1f60 msgr2=0x5634cdcfca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:57:16.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.067+0000 7f6b1250cb80 1 --2- 172.21.15.52:0/4251891869 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5634cdce1f60 0x5634cdcfca30 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f6af801ad30 tx=0x7f6af8019640 comp rx=0 tx=0).stop 2024-06-10T02:57:16.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.067+0000 7f6b1250cb80 1 -- 172.21.15.52:0/4251891869 shutdown_connections 2024-06-10T02:57:16.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.067+0000 7f6b1250cb80 1 --2- 172.21.15.52:0/4251891869 >> [v2:172.21.15.120:6836/4132440952,v1:172.21.15.120:6837/4132440952] conn(0x5634cdd60780 0x5634cdd62ba0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:16.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.067+0000 7f6b1250cb80 1 --2- 172.21.15.52:0/4251891869 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5634cdce1f60 0x5634cdcfca30 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:16.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.067+0000 7f6b1250cb80 1 --2- 172.21.15.52:0/4251891869 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5634cdcdbf20 0x5634cdcedff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:16.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.067+0000 7f6b1250cb80 1 --2- 172.21.15.52:0/4251891869 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5634cdce1640 0x5634cdcee530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:16.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.067+0000 7f6ae57fa700 0 client.15957 ms_handle_reset on v2:172.21.15.120:6836/4132440952 2024-06-10T02:57:16.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.067+0000 7f6b1250cb80 1 -- 172.21.15.52:0/4251891869 >> 172.21.15.52:0/4251891869 conn(0x5634cdc06cc0 msgr2=0x5634cdce3750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:57:16.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.067+0000 7f6b1250cb80 1 -- 172.21.15.52:0/4251891869 shutdown_connections 2024-06-10T02:57:16.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.067+0000 7f6b1250cb80 1 -- 172.21.15.52:0/4251891869 wait complete. 2024-06-10T02:57:16.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.067+0000 7f6b1250cb80 20 client.15957 trim_cache size 0 max 16384 2024-06-10T02:57:16.073 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-06-10T02:57:16.074 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.067+0000 7f6b07c89700 1 -- 172.21.15.52:0/1197077765 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f6ae80a2c20 msgr2=0x7f6ae80a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:57:16.074 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.067+0000 7f6b07c89700 1 --2- 172.21.15.52:0/1197077765 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f6ae80a2c20 0x7f6ae80a50e0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f6af0009fd0 tx=0x7f6af000b040 comp rx=0 tx=0).stop 2024-06-10T02:57:16.074 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.067+0000 7f6b07c89700 1 -- 172.21.15.52:0/1197077765 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6b00102ef0 msgr2=0x7f6b0019b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:57:16.074 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.067+0000 7f6b07c89700 1 --2- 172.21.15.52:0/1197077765 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6b00102ef0 0x7f6b0019b460 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f6af400e9a0 tx=0x7f6af400ecb0 comp rx=0 tx=0).stop 2024-06-10T02:57:16.074 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.067+0000 7f6b07c89700 1 -- 172.21.15.52:0/1197077765 shutdown_connections 2024-06-10T02:57:16.074 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.067+0000 7f6b07c89700 1 --2- 172.21.15.52:0/1197077765 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6b001085d0 0x7f6b0019f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:16.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.067+0000 7f6b07c89700 1 --2- 172.21.15.52:0/1197077765 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6b00102530 0x7f6b0019af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:16.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.067+0000 7f6b07c89700 1 --2- 172.21.15.52:0/1197077765 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f6ae80a2c20 0x7f6ae80a50e0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:16.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.068+0000 7f6b07c89700 1 --2- 172.21.15.52:0/1197077765 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6b00102ef0 0x7f6b0019b460 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:16.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.068+0000 7f6b07c89700 1 -- 172.21.15.52:0/1197077765 >> 172.21.15.52:0/1197077765 conn(0x7f6b00079640 msgr2=0x7f6b001040c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:57:16.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.068+0000 7f6b07c89700 1 -- 172.21.15.52:0/1197077765 shutdown_connections 2024-06-10T02:57:16.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:16.068+0000 7f6b07c89700 1 -- 172.21.15.52:0/1197077765 wait complete. 2024-06-10T02:57:16.729 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-06-10T02:57:16.753 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:57:16 smithi052 ceph-mon[32362]: pgmap v526: 129 pgs: 129 active+clean; 137 MiB data, 523 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 209 op/s 2024-06-10T02:57:16.888 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:57:16 smithi120 ceph-mon[36634]: pgmap v526: 129 pgs: 129 active+clean; 137 MiB data, 523 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 209 op/s 2024-06-10T02:57:16.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:57:16 smithi188 ceph-mon[36599]: pgmap v526: 129 pgs: 129 active+clean; 137 MiB data, 523 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 209 op/s 2024-06-10T02:57:16.997 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:57:17.241 INFO:tasks.workunit.client.0.smithi052.stdout: 38 448047 599 289603 338 269183 1508 2024-06-10T02:57:18.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.675+0000 7f73993e7700 1 -- 172.21.15.52:0/693823328 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f73941086f0 msgr2=0x7f7394108ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:57:18.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.675+0000 7f73993e7700 1 --2- 172.21.15.52:0/693823328 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f73941086f0 0x7f7394108ad0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f7384009990 tx=0x7f7384009ca0 comp rx=0 tx=0).stop 2024-06-10T02:57:18.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.676+0000 7f73993e7700 1 -- 172.21.15.52:0/693823328 shutdown_connections 2024-06-10T02:57:18.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.676+0000 7f73993e7700 1 --2- 172.21.15.52:0/693823328 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7394101440 0x7f739410f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:18.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.676+0000 7f73993e7700 1 --2- 172.21.15.52:0/693823328 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f73941086f0 0x7f7394108ad0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:18.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.676+0000 7f73993e7700 1 --2- 172.21.15.52:0/693823328 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7394100a80 0x7f7394100f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:18.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.676+0000 7f73993e7700 1 -- 172.21.15.52:0/693823328 >> 172.21.15.52:0/693823328 conn(0x7f7394077a00 msgr2=0x7f73940fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:57:18.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.676+0000 7f73993e7700 1 -- 172.21.15.52:0/693823328 shutdown_connections 2024-06-10T02:57:18.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.676+0000 7f73993e7700 1 -- 172.21.15.52:0/693823328 wait complete. 2024-06-10T02:57:18.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.676+0000 7f73993e7700 1 Processor -- start 2024-06-10T02:57:18.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.677+0000 7f73993e7700 1 -- start start 2024-06-10T02:57:18.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.677+0000 7f73993e7700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7394100a80 0x7f7394198e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:57:18.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.677+0000 7f73993e7700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7394101440 0x7f73941993b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:57:18.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.677+0000 7f73993e7700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f73941086f0 0x7f739419d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:57:18.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.677+0000 7f73993e7700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f73941129e0 con 0x7f7394100a80 2024-06-10T02:57:18.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.677+0000 7f73993e7700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f7394112860 con 0x7f73941086f0 2024-06-10T02:57:18.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.677+0000 7f73993e7700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f7394112b60 con 0x7f7394101440 2024-06-10T02:57:18.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.677+0000 7f7392ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7394100a80 0x7f7394198e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:57:18.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.677+0000 7f738a7fc700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7394101440 0x7f73941993b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:57:18.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.677+0000 7f7392ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7394100a80 0x7f7394198e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60476/0 (socket says 172.21.15.52:60476) 2024-06-10T02:57:18.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.677+0000 7f7392ffd700 1 -- 172.21.15.52:0/1024351926 learned_addr learned my addr 172.21.15.52:0/1024351926 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:57:18.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.677+0000 7f73937fe700 1 --2- 172.21.15.52:0/1024351926 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f73941086f0 0x7f739419d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:57:18.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.677+0000 7f7392ffd700 1 -- 172.21.15.52:0/1024351926 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7394101440 msgr2=0x7f73941993b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:57:18.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.677+0000 7f7392ffd700 1 --2- 172.21.15.52:0/1024351926 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7394101440 0x7f73941993b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:18.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.677+0000 7f7392ffd700 1 -- 172.21.15.52:0/1024351926 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f73941086f0 msgr2=0x7f739419d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:57:18.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.678+0000 7f7392ffd700 1 --2- 172.21.15.52:0/1024351926 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f73941086f0 0x7f739419d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:18.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.678+0000 7f7392ffd700 1 -- 172.21.15.52:0/1024351926 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f739419de80 con 0x7f7394100a80 2024-06-10T02:57:18.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.678+0000 7f738a7fc700 1 --2- 172.21.15.52:0/1024351926 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7394101440 0x7f73941993b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T02:57:18.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.678+0000 7f7392ffd700 1 --2- 172.21.15.52:0/1024351926 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7394100a80 0x7f7394198e70 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f738400b840 tx=0x7f7384005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:57:18.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.678+0000 7f7390ff9700 1 -- 172.21.15.52:0/1024351926 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f738400be50 con 0x7f7394100a80 2024-06-10T02:57:18.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.678+0000 7f7390ff9700 1 -- 172.21.15.52:0/1024351926 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7384004d10 con 0x7f7394100a80 2024-06-10T02:57:18.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.678+0000 7f7390ff9700 1 -- 172.21.15.52:0/1024351926 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7384005640 con 0x7f7394100a80 2024-06-10T02:57:18.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.678+0000 7f73993e7700 1 -- 172.21.15.52:0/1024351926 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f739419e110 con 0x7f7394100a80 2024-06-10T02:57:18.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.678+0000 7f73993e7700 1 -- 172.21.15.52:0/1024351926 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f739419e540 con 0x7f7394100a80 2024-06-10T02:57:18.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.680+0000 7f7390ff9700 1 -- 172.21.15.52:0/1024351926 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f7384009d10 con 0x7f7394100a80 2024-06-10T02:57:18.680 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:57:18 smithi052 ceph-mon[32362]: pgmap v527: 129 pgs: 129 active+clean; 143 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.4 MiB/s wr, 188 op/s 2024-06-10T02:57:18.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.681+0000 7f7390ff9700 1 --2- 172.21.15.52:0/1024351926 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f73800a2c20 0x7f73800a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:57:18.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.681+0000 7f7390ff9700 1 -- 172.21.15.52:0/1024351926 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f73840d0170 con 0x7f7394100a80 2024-06-10T02:57:18.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.681+0000 7f738a7fc700 1 --2- 172.21.15.52:0/1024351926 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f73800a2c20 0x7f73800a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:57:18.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.682+0000 7f738a7fc700 1 --2- 172.21.15.52:0/1024351926 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f73800a2c20 0x7f73800a50e0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f7394107da0 tx=0x7f737c00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:57:18.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.682+0000 7f73a3c6ab80 1 Processor -- start 2024-06-10T02:57:18.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.682+0000 7f73a3c6ab80 1 -- start start 2024-06-10T02:57:18.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.683+0000 7f73a3c6ab80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563d83808fd0 0x563d838093b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:57:18.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.683+0000 7f73a3c6ab80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563d838098f0 0x563d8380d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:57:18.683 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.683+0000 7f73a3c6ab80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563d8380dc00 0x563d8380e060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:57:18.683 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.683+0000 7f73a3c6ab80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x563d83756810 con 0x563d838098f0 2024-06-10T02:57:18.683 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.683+0000 7f73a3c6ab80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x563d836fe720 con 0x563d8380dc00 2024-06-10T02:57:18.683 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.683+0000 7f73a3c6ab80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x563d837527d0 con 0x563d83808fd0 2024-06-10T02:57:18.685 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.683+0000 7f7392ffd700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563d83808fd0 0x563d838093b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:57:18.685 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.683+0000 7f738a7fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563d838098f0 0x563d8380d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:57:18.685 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.683+0000 7f7392ffd700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563d83808fd0 0x563d838093b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.52:40280/0 (socket says 172.21.15.52:40280) 2024-06-10T02:57:18.685 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.683+0000 7f7392ffd700 1 -- 172.21.15.52:0/1492016114 learned_addr learned my addr 172.21.15.52:0/1492016114 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:57:18.685 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.683+0000 7f73937fe700 1 --2- 172.21.15.52:0/1492016114 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563d8380dc00 0x563d8380e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:57:18.685 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.683+0000 7f7392ffd700 1 -- 172.21.15.52:0/1492016114 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563d8380dc00 msgr2=0x563d8380e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:57:18.685 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.683+0000 7f7392ffd700 1 --2- 172.21.15.52:0/1492016114 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563d8380dc00 0x563d8380e060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:18.685 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.683+0000 7f7392ffd700 1 -- 172.21.15.52:0/1492016114 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563d838098f0 msgr2=0x563d8380d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:57:18.685 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.683+0000 7f7392ffd700 1 --2- 172.21.15.52:0/1492016114 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563d838098f0 0x563d8380d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:18.685 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.683+0000 7f7392ffd700 1 -- 172.21.15.52:0/1492016114 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d836f4f50 con 0x563d83808fd0 2024-06-10T02:57:18.685 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.683+0000 7f73937fe700 1 --2- 172.21.15.52:0/1492016114 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563d8380dc00 0x563d8380e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T02:57:18.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.683+0000 7f7392ffd700 1 --2- 172.21.15.52:0/1492016114 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563d83808fd0 0x563d838093b0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f738409c700 tx=0x7f7384009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=88c1bea2ac91a0f server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:57:18.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.683+0000 7f73897fa700 1 -- 172.21.15.52:0/1492016114 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73840a3070 con 0x563d83808fd0 2024-06-10T02:57:18.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.684+0000 7f73897fa700 1 -- 172.21.15.52:0/1492016114 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73840095c0 con 0x563d83808fd0 2024-06-10T02:57:18.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.684+0000 7f73897fa700 1 -- 172.21.15.52:0/1492016114 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f738409c9d0 con 0x563d83808fd0 2024-06-10T02:57:18.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.685+0000 7f73a3c6ab80 1 -- 172.21.15.52:0/1492016114 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563d83808fd0 msgr2=0x563d838093b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:57:18.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.685+0000 7f73a3c6ab80 1 --2- 172.21.15.52:0/1492016114 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563d83808fd0 0x563d838093b0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f738409c700 tx=0x7f7384009ca0 comp rx=0 tx=0).stop 2024-06-10T02:57:18.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.685+0000 7f73a3c6ab80 1 -- 172.21.15.52:0/1492016114 shutdown_connections 2024-06-10T02:57:18.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.685+0000 7f73a3c6ab80 1 --2- 172.21.15.52:0/1492016114 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563d8380dc00 0x563d8380e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:18.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.685+0000 7f73a3c6ab80 1 --2- 172.21.15.52:0/1492016114 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563d83808fd0 0x563d838093b0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:18.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.685+0000 7f73a3c6ab80 1 --2- 172.21.15.52:0/1492016114 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563d838098f0 0x563d8380d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:18.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.685+0000 7f73a3c6ab80 1 -- 172.21.15.52:0/1492016114 >> 172.21.15.52:0/1492016114 conn(0x563d8372fcc0 msgr2=0x563d837ff850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:57:18.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.685+0000 7f73a3c6ab80 1 -- 172.21.15.52:0/1492016114 shutdown_connections 2024-06-10T02:57:18.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.685+0000 7f73a3c6ab80 1 -- 172.21.15.52:0/1492016114 wait complete. 2024-06-10T02:57:18.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.685+0000 7f73a3c6ab80 1 -- start start 2024-06-10T02:57:18.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.685+0000 7f73a3c6ab80 1 Processor -- start 2024-06-10T02:57:18.689 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.686+0000 7f73a3c6ab80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563d83808fd0 0x563d838151b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:57:18.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.686+0000 7f73a3c6ab80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563d838098f0 0x563d838124b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:57:18.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.686+0000 7f73a3c6ab80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563d8380dc00 0x563d838129f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:57:18.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.686+0000 7f73a3c6ab80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x563d83815ae0 con 0x563d83808fd0 2024-06-10T02:57:18.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.686+0000 7f73a3c6ab80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x563d836f6bf0 con 0x563d8380dc00 2024-06-10T02:57:18.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.686+0000 7f7392ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563d83808fd0 0x563d838151b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:57:18.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.686+0000 7f73a3c6ab80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x563d83815c60 con 0x563d838098f0 2024-06-10T02:57:18.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.686+0000 7f7392ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563d83808fd0 0x563d838151b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60498/0 (socket says 172.21.15.52:60498) 2024-06-10T02:57:18.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.686+0000 7f7392ffd700 1 -- 172.21.15.52:0/2207486025 learned_addr learned my addr 172.21.15.52:0/2207486025 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:57:18.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.686+0000 7f73a3c6ab80 10 client.0 _ll_register_callbacks cb 0x563d836f0920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T02:57:18.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.686+0000 7f73937fe700 1 --2- 172.21.15.52:0/2207486025 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563d8380dc00 0x563d838129f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:57:18.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.686+0000 7f7392ffd700 1 -- 172.21.15.52:0/2207486025 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563d838098f0 msgr2=0x563d838124b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:57:18.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.686+0000 7f738a7fc700 1 --2- 172.21.15.52:0/2207486025 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563d838098f0 0x563d838124b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:57:18.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.686+0000 7f7392ffd700 1 --2- 172.21.15.52:0/2207486025 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563d838098f0 0x563d838124b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:18.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.686+0000 7f7392ffd700 1 -- 172.21.15.52:0/2207486025 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563d8380dc00 msgr2=0x563d838129f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:57:18.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.686+0000 7f7392ffd700 1 --2- 172.21.15.52:0/2207486025 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563d8380dc00 0x563d838129f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:18.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.686+0000 7f7392ffd700 1 -- 172.21.15.52:0/2207486025 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d836a58d0 con 0x563d83808fd0 2024-06-10T02:57:18.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.686+0000 7f738a7fc700 1 --2- 172.21.15.52:0/2207486025 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563d838098f0 0x563d838124b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T02:57:18.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.686+0000 7f73937fe700 1 --2- 172.21.15.52:0/2207486025 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563d8380dc00 0x563d838129f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T02:57:18.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.686+0000 7f7392ffd700 1 --2- 172.21.15.52:0/2207486025 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563d83808fd0 0x563d838151b0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f7384005920 tx=0x7f73840a5400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c999b783ef7b473 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:57:18.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.686+0000 7f737a7fc700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-10T02:57:18.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.687+0000 7f737a7fc700 1 -- 172.21.15.52:0/2207486025 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73840b9030 con 0x563d83808fd0 2024-06-10T02:57:18.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.687+0000 7f737a7fc700 1 -- 172.21.15.52:0/2207486025 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f738409aa70 con 0x563d83808fd0 2024-06-10T02:57:18.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.687+0000 7f73a3c6ab80 1 -- 172.21.15.52:0/2207486025 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x563d83763d10 con 0x563d83808fd0 2024-06-10T02:57:18.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.687+0000 7f737a7fc700 1 -- 172.21.15.52:0/2207486025 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f73840b4070 con 0x563d83808fd0 2024-06-10T02:57:18.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.687+0000 7f73a3c6ab80 10 client.15969 fetch_fsmap learned FSMap version 33 2024-06-10T02:57:18.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.687+0000 7f73a3c6ab80 1 -- 172.21.15.52:0/2207486025 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x563d83815ae0 con 0x563d83808fd0 2024-06-10T02:57:18.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.687+0000 7f737a7fc700 1 -- 172.21.15.52:0/2207486025 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x563d83763d10 con 0x563d83808fd0 2024-06-10T02:57:18.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.687+0000 7f737a7fc700 1 -- 172.21.15.52:0/2207486025 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 33) ==== 8382+0+0 (secure 0 0 0) 0x7f738409bbc0 con 0x563d83808fd0 2024-06-10T02:57:18.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.687+0000 7f73a3c6ab80 10 client.15969 fetch_fsmap finished waiting for FSMap version 33 2024-06-10T02:57:18.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.687+0000 7f73a3c6ab80 10 client.15969 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.b 2024-06-10T02:57:18.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.687+0000 7f73a3c6ab80 20 client.15969 populate_metadata read hostname 'smithi052' 2024-06-10T02:57:18.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.687+0000 7f73a3c6ab80 1 --2- 172.21.15.52:0/2207486025 >> [v2:172.21.15.120:6838/1672436487,v1:172.21.15.120:6839/1672436487] conn(0x563d83822860 0x563d83824c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:57:18.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.687+0000 7f73a3c6ab80 4 client.15969 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T02:57:18.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.687+0000 7f73a3c6ab80 1 -- 172.21.15.52:0/2207486025 --> [v2:172.21.15.120:6838/1672436487,v1:172.21.15.120:6839/1672436487] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563d8375e290 con 0x563d83822860 2024-06-10T02:57:18.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.688+0000 7f738a7fc700 1 --2- 172.21.15.52:0/2207486025 >> [v2:172.21.15.120:6838/1672436487,v1:172.21.15.120:6839/1672436487] conn(0x563d83822860 0x563d83824c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:57:18.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.688+0000 7f738a7fc700 1 --2- 172.21.15.52:0/2207486025 >> [v2:172.21.15.120:6838/1672436487,v1:172.21.15.120:6839/1672436487] conn(0x563d83822860 0x563d83824c50 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=2a2aca2395cf4750 server_cookie=6ed17bbb3ae02ef8 in_seq=0 out_seq=0 2024-06-10T02:57:18.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.688+0000 7f737a7fc700 10 client.15969 ms_handle_connect on v2:172.21.15.120:6838/1672436487 2024-06-10T02:57:18.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.689+0000 7f737a7fc700 1 -- 172.21.15.52:0/2207486025 <== mds.3 v2:172.21.15.120:6838/1672436487 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x563d8375e290 con 0x563d83822860 2024-06-10T02:57:18.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.689+0000 7f737a7fc700 10 client.15969 handle_command_reply: tid=0 2024-06-10T02:57:18.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.689+0000 7f73a3c6ab80 1 client.15969 shutdown 2024-06-10T02:57:18.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.689+0000 7f73a3c6ab80 1 -- 172.21.15.52:0/2207486025 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563d83808fd0 msgr2=0x563d838151b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:57:18.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.689+0000 7f73a3c6ab80 1 --2- 172.21.15.52:0/2207486025 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563d83808fd0 0x563d838151b0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f7384005920 tx=0x7f73840a5400 comp rx=0 tx=0).stop 2024-06-10T02:57:18.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.689+0000 7f73a3c6ab80 1 -- 172.21.15.52:0/2207486025 shutdown_connections 2024-06-10T02:57:18.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.689+0000 7f73a3c6ab80 1 --2- 172.21.15.52:0/2207486025 >> [v2:172.21.15.120:6838/1672436487,v1:172.21.15.120:6839/1672436487] conn(0x563d83822860 0x563d83824c50 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:18.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.689+0000 7f73a3c6ab80 1 --2- 172.21.15.52:0/2207486025 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563d8380dc00 0x563d838129f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:18.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.689+0000 7f73a3c6ab80 1 --2- 172.21.15.52:0/2207486025 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563d83808fd0 0x563d838151b0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:18.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.689+0000 7f737a7fc700 0 client.15969 ms_handle_reset on v2:172.21.15.120:6838/1672436487 2024-06-10T02:57:18.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.689+0000 7f73a3c6ab80 1 --2- 172.21.15.52:0/2207486025 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563d838098f0 0x563d838124b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:18.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.689+0000 7f73a3c6ab80 1 -- 172.21.15.52:0/2207486025 >> 172.21.15.52:0/2207486025 conn(0x563d8372fcc0 msgr2=0x563d838039d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:57:18.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.689+0000 7f73a3c6ab80 1 -- 172.21.15.52:0/2207486025 shutdown_connections 2024-06-10T02:57:18.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.689+0000 7f73a3c6ab80 1 -- 172.21.15.52:0/2207486025 wait complete. 2024-06-10T02:57:18.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.690+0000 7f73a3c6ab80 20 client.15969 trim_cache size 0 max 16384 2024-06-10T02:57:18.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.705+0000 7f73a3c6ab80 1 Processor -- start 2024-06-10T02:57:18.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.706+0000 7f73a3c6ab80 1 -- start start 2024-06-10T02:57:18.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.706+0000 7f73a3c6ab80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563d83808fd0 0x563d8381ed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:57:18.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.706+0000 7f73a3c6ab80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563d838098f0 0x563d837f9510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:57:18.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.706+0000 7f73a3c6ab80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563d8380dc00 0x563d8382a560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:57:18.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.706+0000 7f73a3c6ab80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f73941ac080 con 0x563d83808fd0 2024-06-10T02:57:18.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.706+0000 7f73a3c6ab80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f73941adcb0 con 0x563d838098f0 2024-06-10T02:57:18.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.706+0000 7f73a3c6ab80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f73941ac750 con 0x563d8380dc00 2024-06-10T02:57:18.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.706+0000 7f7392ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563d83808fd0 0x563d8381ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:57:18.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.706+0000 7f738a7fc700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563d838098f0 0x563d837f9510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:57:18.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.706+0000 7f73937fe700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563d8380dc00 0x563d8382a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:57:18.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.706+0000 7f7392ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563d83808fd0 0x563d8381ed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60500/0 (socket says 172.21.15.52:60500) 2024-06-10T02:57:18.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.706+0000 7f7392ffd700 1 -- 172.21.15.52:0/4019919571 learned_addr learned my addr 172.21.15.52:0/4019919571 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:57:18.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.706+0000 7f7392ffd700 1 -- 172.21.15.52:0/4019919571 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563d8380dc00 msgr2=0x563d8382a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:57:18.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.706+0000 7f7392ffd700 1 --2- 172.21.15.52:0/4019919571 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563d8380dc00 0x563d8382a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:18.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.706+0000 7f7392ffd700 1 -- 172.21.15.52:0/4019919571 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563d838098f0 msgr2=0x563d837f9510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:57:18.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.706+0000 7f7392ffd700 1 --2- 172.21.15.52:0/4019919571 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563d838098f0 0x563d837f9510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:18.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.706+0000 7f7392ffd700 1 -- 172.21.15.52:0/4019919571 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d838252d0 con 0x563d83808fd0 2024-06-10T02:57:18.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.707+0000 7f7392ffd700 1 --2- 172.21.15.52:0/4019919571 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563d83808fd0 0x563d8381ed80 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f738400ffb0 tx=0x7f7384005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c16c5006565eac3 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:57:18.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.707+0000 7f7388ff9700 1 -- 172.21.15.52:0/4019919571 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7384098510 con 0x563d83808fd0 2024-06-10T02:57:18.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.707+0000 7f7388ff9700 1 -- 172.21.15.52:0/4019919571 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7384098700 con 0x563d83808fd0 2024-06-10T02:57:18.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.707+0000 7f7388ff9700 1 -- 172.21.15.52:0/4019919571 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f738409c9d0 con 0x563d83808fd0 2024-06-10T02:57:18.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.707+0000 7f73a3c6ab80 1 -- 172.21.15.52:0/4019919571 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563d83808fd0 msgr2=0x563d8381ed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:57:18.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.707+0000 7f73a3c6ab80 1 --2- 172.21.15.52:0/4019919571 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563d83808fd0 0x563d8381ed80 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f738400ffb0 tx=0x7f7384005ef0 comp rx=0 tx=0).stop 2024-06-10T02:57:18.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.708+0000 7f73a3c6ab80 1 -- 172.21.15.52:0/4019919571 shutdown_connections 2024-06-10T02:57:18.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.708+0000 7f73a3c6ab80 1 --2- 172.21.15.52:0/4019919571 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563d8380dc00 0x563d8382a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:18.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.708+0000 7f73a3c6ab80 1 --2- 172.21.15.52:0/4019919571 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563d83808fd0 0x563d8381ed80 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:18.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.708+0000 7f73a3c6ab80 1 --2- 172.21.15.52:0/4019919571 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563d838098f0 0x563d837f9510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:18.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.708+0000 7f73a3c6ab80 1 -- 172.21.15.52:0/4019919571 >> 172.21.15.52:0/4019919571 conn(0x563d8372fcc0 msgr2=0x563d837f8e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:57:18.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.708+0000 7f73a3c6ab80 1 -- 172.21.15.52:0/4019919571 shutdown_connections 2024-06-10T02:57:18.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.708+0000 7f73a3c6ab80 1 -- 172.21.15.52:0/4019919571 wait complete. 2024-06-10T02:57:18.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.708+0000 7f73a3c6ab80 1 -- start start 2024-06-10T02:57:18.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.708+0000 7f73a3c6ab80 1 Processor -- start 2024-06-10T02:57:18.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.708+0000 7f73a3c6ab80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563d83808fd0 0x563d8381b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:57:18.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.708+0000 7f73a3c6ab80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563d838098f0 0x563d83829ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:57:18.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.708+0000 7f73a3c6ab80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563d8380dc00 0x563d8382a420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:57:18.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.708+0000 7f73a3c6ab80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f73941ad5b0 con 0x563d83808fd0 2024-06-10T02:57:18.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.708+0000 7f73a3c6ab80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f73941acee0 con 0x563d8380dc00 2024-06-10T02:57:18.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.708+0000 7f73a3c6ab80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f73941aa020 con 0x563d838098f0 2024-06-10T02:57:18.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.708+0000 7f73a3c6ab80 10 client.0 _ll_register_callbacks cb 0x563d836f0920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T02:57:18.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.708+0000 7f7392ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563d83808fd0 0x563d8381b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:57:18.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.708+0000 7f7392ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563d83808fd0 0x563d8381b630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60506/0 (socket says 172.21.15.52:60506) 2024-06-10T02:57:18.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.708+0000 7f7392ffd700 1 -- 172.21.15.52:0/712784629 learned_addr learned my addr 172.21.15.52:0/712784629 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:57:18.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.708+0000 7f738a7fc700 1 --2- 172.21.15.52:0/712784629 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563d838098f0 0x563d83829ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:57:18.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.709+0000 7f73937fe700 1 --2- 172.21.15.52:0/712784629 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563d8380dc00 0x563d8382a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:57:18.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.709+0000 7f7392ffd700 1 -- 172.21.15.52:0/712784629 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563d838098f0 msgr2=0x563d83829ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:57:18.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.709+0000 7f7392ffd700 1 --2- 172.21.15.52:0/712784629 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563d838098f0 0x563d83829ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:18.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.709+0000 7f7392ffd700 1 -- 172.21.15.52:0/712784629 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563d8380dc00 msgr2=0x563d8382a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:57:18.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.709+0000 7f7392ffd700 1 --2- 172.21.15.52:0/712784629 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563d8380dc00 0x563d8382a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:18.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.709+0000 7f7392ffd700 1 -- 172.21.15.52:0/712784629 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d8380da30 con 0x563d83808fd0 2024-06-10T02:57:18.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.709+0000 7f73937fe700 1 --2- 172.21.15.52:0/712784629 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563d8380dc00 0x563d8382a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T02:57:18.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.709+0000 7f7392ffd700 1 --2- 172.21.15.52:0/712784629 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563d83808fd0 0x563d8381b630 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f7384099140 tx=0x7f7384005fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f4b3d3c8c185993 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:57:18.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.709+0000 7f737a7fc700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-10T02:57:18.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.709+0000 7f737a7fc700 1 -- 172.21.15.52:0/712784629 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73840b9070 con 0x563d83808fd0 2024-06-10T02:57:18.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.709+0000 7f737a7fc700 1 -- 172.21.15.52:0/712784629 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f7384004830 con 0x563d83808fd0 2024-06-10T02:57:18.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.709+0000 7f73a3c6ab80 1 -- 172.21.15.52:0/712784629 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7384098700 con 0x563d83808fd0 2024-06-10T02:57:18.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.709+0000 7f737a7fc700 1 -- 172.21.15.52:0/712784629 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f73840a4ec0 con 0x563d83808fd0 2024-06-10T02:57:18.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.709+0000 7f73a3c6ab80 10 client.15975 fetch_fsmap learned FSMap version 33 2024-06-10T02:57:18.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.709+0000 7f73a3c6ab80 1 -- 172.21.15.52:0/712784629 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x563d8380da30 con 0x563d83808fd0 2024-06-10T02:57:18.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.709+0000 7f737a7fc700 1 -- 172.21.15.52:0/712784629 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7384098700 con 0x563d83808fd0 2024-06-10T02:57:18.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.710+0000 7f737a7fc700 1 -- 172.21.15.52:0/712784629 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 33) ==== 8382+0+0 (secure 0 0 0) 0x7f738409daa0 con 0x563d83808fd0 2024-06-10T02:57:18.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.710+0000 7f73a3c6ab80 10 client.15975 fetch_fsmap finished waiting for FSMap version 33 2024-06-10T02:57:18.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.710+0000 7f73a3c6ab80 10 client.15975 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.b 2024-06-10T02:57:18.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.710+0000 7f73a3c6ab80 20 client.15975 populate_metadata read hostname 'smithi052' 2024-06-10T02:57:18.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.710+0000 7f73a3c6ab80 1 --2- 172.21.15.52:0/712784629 >> [v2:172.21.15.120:6838/1672436487,v1:172.21.15.120:6839/1672436487] conn(0x563d8388d510 0x563d8388f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:57:18.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.710+0000 7f73a3c6ab80 4 client.15975 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T02:57:18.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.710+0000 7f73a3c6ab80 1 -- 172.21.15.52:0/712784629 --> [v2:172.21.15.120:6838/1672436487,v1:172.21.15.120:6839/1672436487] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563d835d8250 con 0x563d8388d510 2024-06-10T02:57:18.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.710+0000 7f738a7fc700 1 --2- 172.21.15.52:0/712784629 >> [v2:172.21.15.120:6838/1672436487,v1:172.21.15.120:6839/1672436487] conn(0x563d8388d510 0x563d8388f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:57:18.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.710+0000 7f738a7fc700 1 --2- 172.21.15.52:0/712784629 >> [v2:172.21.15.120:6838/1672436487,v1:172.21.15.120:6839/1672436487] conn(0x563d8388d510 0x563d8388f930 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c0e9b5895b76eaf8 server_cookie=86e7d7902255558c in_seq=0 out_seq=0 2024-06-10T02:57:18.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.710+0000 7f737a7fc700 10 client.15975 ms_handle_connect on v2:172.21.15.120:6838/1672436487 2024-06-10T02:57:18.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.711+0000 7f737a7fc700 1 -- 172.21.15.52:0/712784629 <== mds.3 v2:172.21.15.120:6838/1672436487 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563d835d8250 con 0x563d8388d510 2024-06-10T02:57:18.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.711+0000 7f737a7fc700 10 client.15975 handle_command_reply: tid=0 2024-06-10T02:57:18.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.711+0000 7f73a3c6ab80 1 client.15975 shutdown 2024-06-10T02:57:18.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.711+0000 7f73a3c6ab80 1 -- 172.21.15.52:0/712784629 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563d83808fd0 msgr2=0x563d8381b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:57:18.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.711+0000 7f73a3c6ab80 1 --2- 172.21.15.52:0/712784629 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563d83808fd0 0x563d8381b630 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f7384099140 tx=0x7f7384005fb0 comp rx=0 tx=0).stop 2024-06-10T02:57:18.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.711+0000 7f73a3c6ab80 1 -- 172.21.15.52:0/712784629 shutdown_connections 2024-06-10T02:57:18.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.711+0000 7f73a3c6ab80 1 --2- 172.21.15.52:0/712784629 >> [v2:172.21.15.120:6838/1672436487,v1:172.21.15.120:6839/1672436487] conn(0x563d8388d510 0x563d8388f930 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:18.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.711+0000 7f73a3c6ab80 1 --2- 172.21.15.52:0/712784629 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563d8380dc00 0x563d8382a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:18.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.711+0000 7f73a3c6ab80 1 --2- 172.21.15.52:0/712784629 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563d83808fd0 0x563d8381b630 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:18.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.711+0000 7f73a3c6ab80 1 --2- 172.21.15.52:0/712784629 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563d838098f0 0x563d83829ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:18.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.711+0000 7f73a3c6ab80 1 -- 172.21.15.52:0/712784629 >> 172.21.15.52:0/712784629 conn(0x563d8372fcc0 msgr2=0x563d83810010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:57:18.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.711+0000 7f737a7fc700 0 client.15975 ms_handle_reset on v2:172.21.15.120:6838/1672436487 2024-06-10T02:57:18.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.711+0000 7f73a3c6ab80 1 -- 172.21.15.52:0/712784629 shutdown_connections 2024-06-10T02:57:18.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.711+0000 7f73a3c6ab80 1 -- 172.21.15.52:0/712784629 wait complete. 2024-06-10T02:57:18.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.711+0000 7f73a3c6ab80 20 client.15975 trim_cache size 0 max 16384 2024-06-10T02:57:18.715 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-06-10T02:57:18.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.712+0000 7f73993e7700 1 -- 172.21.15.52:0/1024351926 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f73800a2c20 msgr2=0x7f73800a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:57:18.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.712+0000 7f73993e7700 1 --2- 172.21.15.52:0/1024351926 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f73800a2c20 0x7f73800a50e0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f7394107da0 tx=0x7f737c00b410 comp rx=0 tx=0).stop 2024-06-10T02:57:18.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.712+0000 7f73993e7700 1 -- 172.21.15.52:0/1024351926 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7394100a80 msgr2=0x7f7394198e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:57:18.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.712+0000 7f73993e7700 1 --2- 172.21.15.52:0/1024351926 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7394100a80 0x7f7394198e70 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f738400b840 tx=0x7f7384005e80 comp rx=0 tx=0).stop 2024-06-10T02:57:18.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.712+0000 7f73993e7700 1 -- 172.21.15.52:0/1024351926 shutdown_connections 2024-06-10T02:57:18.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.712+0000 7f73993e7700 1 --2- 172.21.15.52:0/1024351926 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f73941086f0 0x7f739419d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:18.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.712+0000 7f73993e7700 1 --2- 172.21.15.52:0/1024351926 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f73800a2c20 0x7f73800a50e0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:18.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.712+0000 7f73993e7700 1 --2- 172.21.15.52:0/1024351926 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7394100a80 0x7f7394198e70 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:18.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.712+0000 7f73993e7700 1 --2- 172.21.15.52:0/1024351926 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7394101440 0x7f73941993b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:18.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.712+0000 7f73993e7700 1 -- 172.21.15.52:0/1024351926 >> 172.21.15.52:0/1024351926 conn(0x7f7394077a00 msgr2=0x7f73940ff0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:57:18.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.712+0000 7f73993e7700 1 -- 172.21.15.52:0/1024351926 shutdown_connections 2024-06-10T02:57:18.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:18.712+0000 7f73993e7700 1 -- 172.21.15.52:0/1024351926 wait complete. 2024-06-10T02:57:18.888 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:57:18 smithi120 ceph-mon[36634]: pgmap v527: 129 pgs: 129 active+clean; 143 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.4 MiB/s wr, 188 op/s 2024-06-10T02:57:18.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:57:18 smithi188 ceph-mon[36599]: pgmap v527: 129 pgs: 129 active+clean; 143 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.4 MiB/s wr, 188 op/s 2024-06-10T02:57:19.888 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:57:19 smithi120 ceph-mon[36634]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-10T02:57:19.907 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:57:19 smithi052 ceph-mon[32362]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-10T02:57:19.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:57:19 smithi188 ceph-mon[36599]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-10T02:57:20.406 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-06-10T02:57:20.701 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:57:20.726 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:57:20 smithi052 ceph-mon[32362]: pgmap v528: 129 pgs: 129 active+clean; 145 MiB data, 517 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.3 MiB/s wr, 133 op/s 2024-06-10T02:57:20.888 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:57:20 smithi120 ceph-mon[36634]: pgmap v528: 129 pgs: 129 active+clean; 145 MiB data, 517 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.3 MiB/s wr, 133 op/s 2024-06-10T02:57:20.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:57:20 smithi188 ceph-mon[36599]: pgmap v528: 129 pgs: 129 active+clean; 145 MiB data, 517 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.3 MiB/s wr, 133 op/s 2024-06-10T02:57:22.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.355+0000 7f012a11b700 1 -- 172.21.15.52:0/1800391341 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0124107570 msgr2=0x7f0124107950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:57:22.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.355+0000 7f012a11b700 1 --2- 172.21.15.52:0/1800391341 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0124107570 0x7f0124107950 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f0114009990 tx=0x7f0114009ca0 comp rx=0 tx=0).stop 2024-06-10T02:57:22.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.356+0000 7f012a11b700 1 -- 172.21.15.52:0/1800391341 shutdown_connections 2024-06-10T02:57:22.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.356+0000 7f012a11b700 1 --2- 172.21.15.52:0/1800391341 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0124101810 0x7f012410f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:22.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.356+0000 7f012a11b700 1 --2- 172.21.15.52:0/1800391341 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0124107570 0x7f0124107950 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:22.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.356+0000 7f012a11b700 1 --2- 172.21.15.52:0/1800391341 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0124100e50 0x7f01241012d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:22.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.356+0000 7f012a11b700 1 -- 172.21.15.52:0/1800391341 >> 172.21.15.52:0/1800391341 conn(0x7f01240766b0 msgr2=0x7f0124076ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:57:22.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.356+0000 7f012a11b700 1 -- 172.21.15.52:0/1800391341 shutdown_connections 2024-06-10T02:57:22.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.356+0000 7f012a11b700 1 -- 172.21.15.52:0/1800391341 wait complete. 2024-06-10T02:57:22.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.356+0000 7f012a11b700 1 Processor -- start 2024-06-10T02:57:22.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.357+0000 7f012a11b700 1 -- start start 2024-06-10T02:57:22.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.357+0000 7f012a11b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0124100e50 0x7f0124102030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:57:22.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.357+0000 7f012a11b700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0124101810 0x7f0124102570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:57:22.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.357+0000 7f012a11b700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0124107570 0x7f0124102ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:57:22.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.357+0000 7f012a11b700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f0124112800 con 0x7f0124100e50 2024-06-10T02:57:22.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.357+0000 7f012a11b700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f0124112680 con 0x7f0124107570 2024-06-10T02:57:22.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.357+0000 7f012a11b700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f0124112980 con 0x7f0124101810 2024-06-10T02:57:22.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.357+0000 7f01237fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0124100e50 0x7f0124102030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:57:22.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.357+0000 7f01237fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0124100e50 0x7f0124102030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55470/0 (socket says 172.21.15.52:55470) 2024-06-10T02:57:22.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.357+0000 7f01237fe700 1 -- 172.21.15.52:0/3682888124 learned_addr learned my addr 172.21.15.52:0/3682888124 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:57:22.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.357+0000 7f0123fff700 1 --2- 172.21.15.52:0/3682888124 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0124107570 0x7f0124102ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:57:22.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.357+0000 7f0122ffd700 1 --2- 172.21.15.52:0/3682888124 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0124101810 0x7f0124102570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:57:22.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.357+0000 7f01237fe700 1 -- 172.21.15.52:0/3682888124 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0124101810 msgr2=0x7f0124102570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:57:22.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.357+0000 7f01237fe700 1 --2- 172.21.15.52:0/3682888124 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0124101810 0x7f0124102570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:22.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.357+0000 7f01237fe700 1 -- 172.21.15.52:0/3682888124 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0124107570 msgr2=0x7f0124102ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:57:22.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.357+0000 7f01237fe700 1 --2- 172.21.15.52:0/3682888124 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0124107570 0x7f0124102ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:22.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.357+0000 7f01237fe700 1 -- 172.21.15.52:0/3682888124 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01241a7220 con 0x7f0124100e50 2024-06-10T02:57:22.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.358+0000 7f01237fe700 1 --2- 172.21.15.52:0/3682888124 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0124100e50 0x7f0124102030 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f0114000c00 tx=0x7f0114003890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:57:22.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.358+0000 7f0120ff9700 1 -- 172.21.15.52:0/3682888124 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0114027070 con 0x7f0124100e50 2024-06-10T02:57:22.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.358+0000 7f012a11b700 1 -- 172.21.15.52:0/3682888124 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01241a74b0 con 0x7f0124100e50 2024-06-10T02:57:22.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.358+0000 7f0120ff9700 1 -- 172.21.15.52:0/3682888124 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0114003e40 con 0x7f0124100e50 2024-06-10T02:57:22.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.358+0000 7f0120ff9700 1 -- 172.21.15.52:0/3682888124 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0114022c60 con 0x7f0124100e50 2024-06-10T02:57:22.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.358+0000 7f012a11b700 1 -- 172.21.15.52:0/3682888124 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01241a7960 con 0x7f0124100e50 2024-06-10T02:57:22.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.360+0000 7f0120ff9700 1 -- 172.21.15.52:0/3682888124 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f011402a070 con 0x7f0124100e50 2024-06-10T02:57:22.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.361+0000 7f0120ff9700 1 --2- 172.21.15.52:0/3682888124 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f010c0a2c20 0x7f010c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:57:22.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.361+0000 7f0120ff9700 1 -- 172.21.15.52:0/3682888124 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f0114022e00 con 0x7f0124100e50 2024-06-10T02:57:22.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.361+0000 7f0122ffd700 1 --2- 172.21.15.52:0/3682888124 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f010c0a2c20 0x7f010c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:57:22.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.362+0000 7f0122ffd700 1 --2- 172.21.15.52:0/3682888124 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f010c0a2c20 0x7f010c0a50e0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f0124106490 tx=0x7f01180073d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:57:22.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.362+0000 7f013499eb80 1 Processor -- start 2024-06-10T02:57:22.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.362+0000 7f013499eb80 1 -- start start 2024-06-10T02:57:22.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.362+0000 7f013499eb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558ae0f79f60 0x558ae0f7a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:57:22.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.362+0000 7f013499eb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558ae0f7a880 0x558ae0f74260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:57:22.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.362+0000 7f013499eb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558ae0f747a0 0x558ae0f74bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:57:22.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.362+0000 7f013499eb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x558ae0e66f60 con 0x558ae0f747a0 2024-06-10T02:57:22.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.362+0000 7f013499eb80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x558ae0ec5850 con 0x558ae0f79f60 2024-06-10T02:57:22.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.362+0000 7f013499eb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x558ae0e2d460 con 0x558ae0f7a880 2024-06-10T02:57:22.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.362+0000 7f0123fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558ae0f747a0 0x558ae0f74bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:57:22.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.362+0000 7f0122ffd700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558ae0f7a880 0x558ae0f74260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:57:22.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.362+0000 7f0123fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558ae0f747a0 0x558ae0f74bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55480/0 (socket says 172.21.15.52:55480) 2024-06-10T02:57:22.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.362+0000 7f0123fff700 1 -- 172.21.15.52:0/279448596 learned_addr learned my addr 172.21.15.52:0/279448596 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:57:22.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.362+0000 7f01237fe700 1 --2- 172.21.15.52:0/279448596 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558ae0f79f60 0x558ae0f7a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:57:22.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.362+0000 7f0122ffd700 1 -- 172.21.15.52:0/279448596 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558ae0f79f60 msgr2=0x558ae0f7a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:57:22.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.363+0000 7f0122ffd700 1 --2- 172.21.15.52:0/279448596 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558ae0f79f60 0x558ae0f7a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:22.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.363+0000 7f0122ffd700 1 -- 172.21.15.52:0/279448596 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558ae0f747a0 msgr2=0x558ae0f74bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:57:22.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.363+0000 7f0122ffd700 1 --2- 172.21.15.52:0/279448596 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558ae0f747a0 0x558ae0f74bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:22.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.363+0000 7f0122ffd700 1 -- 172.21.15.52:0/279448596 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ae0e148d0 con 0x558ae0f7a880 2024-06-10T02:57:22.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.363+0000 7f01237fe700 1 --2- 172.21.15.52:0/279448596 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558ae0f79f60 0x558ae0f7a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T02:57:22.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.363+0000 7f0123fff700 1 --2- 172.21.15.52:0/279448596 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558ae0f747a0 0x558ae0f74bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T02:57:22.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.363+0000 7f0122ffd700 1 --2- 172.21.15.52:0/279448596 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558ae0f7a880 0x558ae0f74260 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f0124103ac0 tx=0x7f0118013000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2a125d2b8efeab2 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:57:22.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.363+0000 7f0109ffb700 1 -- 172.21.15.52:0/279448596 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0118024070 con 0x558ae0f7a880 2024-06-10T02:57:22.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.363+0000 7f0109ffb700 1 -- 172.21.15.52:0/279448596 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0118015070 con 0x558ae0f7a880 2024-06-10T02:57:22.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.363+0000 7f0109ffb700 1 -- 172.21.15.52:0/279448596 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0118011760 con 0x558ae0f7a880 2024-06-10T02:57:22.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.364+0000 7f013499eb80 1 -- 172.21.15.52:0/279448596 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558ae0f7a880 msgr2=0x558ae0f74260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:57:22.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.364+0000 7f013499eb80 1 --2- 172.21.15.52:0/279448596 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558ae0f7a880 0x558ae0f74260 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f0124103ac0 tx=0x7f0118013000 comp rx=0 tx=0).stop 2024-06-10T02:57:22.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.364+0000 7f013499eb80 1 -- 172.21.15.52:0/279448596 shutdown_connections 2024-06-10T02:57:22.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.364+0000 7f013499eb80 1 --2- 172.21.15.52:0/279448596 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558ae0f747a0 0x558ae0f74bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:22.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.364+0000 7f013499eb80 1 --2- 172.21.15.52:0/279448596 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558ae0f79f60 0x558ae0f7a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:22.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.364+0000 7f013499eb80 1 --2- 172.21.15.52:0/279448596 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558ae0f7a880 0x558ae0f74260 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:22.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.364+0000 7f013499eb80 1 -- 172.21.15.52:0/279448596 >> 172.21.15.52:0/279448596 conn(0x558ae0ea1080 msgr2=0x558ae0f6f220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:57:22.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.364+0000 7f013499eb80 1 -- 172.21.15.52:0/279448596 shutdown_connections 2024-06-10T02:57:22.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.364+0000 7f013499eb80 1 -- 172.21.15.52:0/279448596 wait complete. 2024-06-10T02:57:22.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.364+0000 7f013499eb80 1 -- start start 2024-06-10T02:57:22.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.365+0000 7f013499eb80 1 Processor -- start 2024-06-10T02:57:22.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.365+0000 7f013499eb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558ae0f747a0 0x558ae0f806c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:57:22.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.365+0000 7f013499eb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558ae0f79f60 0x558ae0f7d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:57:22.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.365+0000 7f013499eb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558ae0f7a880 0x558ae0f7df00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:57:22.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.365+0000 7f013499eb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x558ae0e65fc0 con 0x558ae0f79f60 2024-06-10T02:57:22.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.365+0000 7f013499eb80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x558ae0ec1810 con 0x558ae0f7a880 2024-06-10T02:57:22.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.365+0000 7f013499eb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x558ae0f81260 con 0x558ae0f747a0 2024-06-10T02:57:22.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.365+0000 7f013499eb80 10 client.0 _ll_register_callbacks cb 0x558ae0e5b5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T02:57:22.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.365+0000 7f01237fe700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558ae0f747a0 0x558ae0f806c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:57:22.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.365+0000 7f01237fe700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558ae0f747a0 0x558ae0f806c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.52:52108/0 (socket says 172.21.15.52:52108) 2024-06-10T02:57:22.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.365+0000 7f01237fe700 1 -- 172.21.15.52:0/1898155949 learned_addr learned my addr 172.21.15.52:0/1898155949 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:57:22.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.365+0000 7f0122ffd700 1 --2- 172.21.15.52:0/1898155949 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558ae0f79f60 0x558ae0f7d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:57:22.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.365+0000 7f0123fff700 1 --2- 172.21.15.52:0/1898155949 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558ae0f7a880 0x558ae0f7df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:57:22.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.365+0000 7f01237fe700 1 -- 172.21.15.52:0/1898155949 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558ae0f7a880 msgr2=0x558ae0f7df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:57:22.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.365+0000 7f01237fe700 1 --2- 172.21.15.52:0/1898155949 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558ae0f7a880 0x558ae0f7df00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:22.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.365+0000 7f01237fe700 1 -- 172.21.15.52:0/1898155949 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558ae0f79f60 msgr2=0x558ae0f7d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:57:22.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.365+0000 7f01237fe700 1 --2- 172.21.15.52:0/1898155949 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558ae0f79f60 0x558ae0f7d9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:22.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.365+0000 7f01237fe700 1 -- 172.21.15.52:0/1898155949 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ae0e8b750 con 0x558ae0f747a0 2024-06-10T02:57:22.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.366+0000 7f01237fe700 1 --2- 172.21.15.52:0/1898155949 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558ae0f747a0 0x558ae0f806c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f0114009550 tx=0x7f01140989e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a1c06ab0b839dba0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:57:22.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.366+0000 7f0102ffd700 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-06-10T02:57:22.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.366+0000 7f0102ffd700 1 -- 172.21.15.52:0/1898155949 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01140a0af0 con 0x558ae0f747a0 2024-06-10T02:57:22.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.366+0000 7f013499eb80 1 -- 172.21.15.52:0/1898155949 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0118015070 con 0x558ae0f747a0 2024-06-10T02:57:22.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.366+0000 7f0102ffd700 1 -- 172.21.15.52:0/1898155949 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f0114022470 con 0x558ae0f747a0 2024-06-10T02:57:22.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.366+0000 7f0102ffd700 1 -- 172.21.15.52:0/1898155949 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f01140a5320 con 0x558ae0f747a0 2024-06-10T02:57:22.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.366+0000 7f013499eb80 10 client.25552 fetch_fsmap learned FSMap version 33 2024-06-10T02:57:22.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.366+0000 7f013499eb80 1 -- 172.21.15.52:0/1898155949 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x558ae0f8bac0 con 0x558ae0f747a0 2024-06-10T02:57:22.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.366+0000 7f0102ffd700 1 -- 172.21.15.52:0/1898155949 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0118015070 con 0x558ae0f747a0 2024-06-10T02:57:22.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.366+0000 7f0102ffd700 1 -- 172.21.15.52:0/1898155949 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 33) ==== 8382+0+0 (secure 0 0 0) 0x7f01140bc040 con 0x558ae0f747a0 2024-06-10T02:57:22.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.366+0000 7f013499eb80 10 client.25552 fetch_fsmap finished waiting for FSMap version 33 2024-06-10T02:57:22.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.367+0000 7f013499eb80 10 client.25552 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-06-10T02:57:22.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.367+0000 7f013499eb80 20 client.25552 populate_metadata read hostname 'smithi052' 2024-06-10T02:57:22.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.367+0000 7f013499eb80 1 --2- 172.21.15.52:0/1898155949 >> [v2:172.21.15.188:6834/3904544259,v1:172.21.15.188:6835/3904544259] conn(0x558ae0f8e0d0 0x558ae0f904f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:57:22.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.367+0000 7f013499eb80 4 client.25552 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T02:57:22.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.367+0000 7f013499eb80 1 -- 172.21.15.52:0/1898155949 --> [v2:172.21.15.188:6834/3904544259,v1:172.21.15.188:6835/3904544259] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558ae0ed2d50 con 0x558ae0f8e0d0 2024-06-10T02:57:22.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.367+0000 7f0122ffd700 1 --2- 172.21.15.52:0/1898155949 >> [v2:172.21.15.188:6834/3904544259,v1:172.21.15.188:6835/3904544259] conn(0x558ae0f8e0d0 0x558ae0f904f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:57:22.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.367+0000 7f0122ffd700 1 --2- 172.21.15.52:0/1898155949 >> [v2:172.21.15.188:6834/3904544259,v1:172.21.15.188:6835/3904544259] conn(0x558ae0f8e0d0 0x558ae0f904f0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7fa09f2695e0c16f server_cookie=3b7673d64d84ffcc in_seq=0 out_seq=0 2024-06-10T02:57:22.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.367+0000 7f0102ffd700 10 client.25552 ms_handle_connect on v2:172.21.15.188:6834/3904544259 2024-06-10T02:57:22.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.371+0000 7f0102ffd700 1 -- 172.21.15.52:0/1898155949 <== mds.2 v2:172.21.15.188:6834/3904544259 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x558ae0ed2d50 con 0x558ae0f8e0d0 2024-06-10T02:57:22.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.371+0000 7f0102ffd700 10 client.25552 handle_command_reply: tid=0 2024-06-10T02:57:22.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.371+0000 7f013499eb80 1 client.25552 shutdown 2024-06-10T02:57:22.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.371+0000 7f013499eb80 1 -- 172.21.15.52:0/1898155949 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558ae0f747a0 msgr2=0x558ae0f806c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:57:22.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.371+0000 7f013499eb80 1 --2- 172.21.15.52:0/1898155949 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558ae0f747a0 0x558ae0f806c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f0114009550 tx=0x7f01140989e0 comp rx=0 tx=0).stop 2024-06-10T02:57:22.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.371+0000 7f013499eb80 1 -- 172.21.15.52:0/1898155949 shutdown_connections 2024-06-10T02:57:22.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.371+0000 7f013499eb80 1 --2- 172.21.15.52:0/1898155949 >> [v2:172.21.15.188:6834/3904544259,v1:172.21.15.188:6835/3904544259] conn(0x558ae0f8e0d0 0x558ae0f904f0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:22.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.371+0000 7f0102ffd700 0 client.25552 ms_handle_reset on v2:172.21.15.188:6834/3904544259 2024-06-10T02:57:22.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.371+0000 7f013499eb80 1 --2- 172.21.15.52:0/1898155949 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558ae0f7a880 0x558ae0f7df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:22.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.371+0000 7f013499eb80 1 --2- 172.21.15.52:0/1898155949 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558ae0f747a0 0x558ae0f806c0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:22.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.371+0000 7f013499eb80 1 --2- 172.21.15.52:0/1898155949 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558ae0f79f60 0x558ae0f7d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:22.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.371+0000 7f013499eb80 1 -- 172.21.15.52:0/1898155949 >> 172.21.15.52:0/1898155949 conn(0x558ae0ea1080 msgr2=0x558ae0f6e700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:57:22.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.371+0000 7f013499eb80 1 -- 172.21.15.52:0/1898155949 shutdown_connections 2024-06-10T02:57:22.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.371+0000 7f013499eb80 1 -- 172.21.15.52:0/1898155949 wait complete. 2024-06-10T02:57:22.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.371+0000 7f013499eb80 20 client.25552 trim_cache size 0 max 16384 2024-06-10T02:57:22.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.388+0000 7f013499eb80 1 Processor -- start 2024-06-10T02:57:22.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.388+0000 7f013499eb80 1 -- start start 2024-06-10T02:57:22.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.388+0000 7f013499eb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558ae0ea1080 0x558ae0f74260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:57:22.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.388+0000 7f013499eb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558ae0f747a0 0x558ae0f791c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:57:22.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.388+0000 7f013499eb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558ae0f79f60 0x558ae0f80200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:57:22.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.388+0000 7f013499eb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f01241b04d0 con 0x558ae0f747a0 2024-06-10T02:57:22.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.388+0000 7f013499eb80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f01241b2100 con 0x558ae0ea1080 2024-06-10T02:57:22.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.388+0000 7f013499eb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f01241b0ba0 con 0x558ae0f79f60 2024-06-10T02:57:22.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.388+0000 7f0122ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558ae0f747a0 0x558ae0f791c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:57:22.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.388+0000 7f0122ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558ae0f747a0 0x558ae0f791c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55496/0 (socket says 172.21.15.52:55496) 2024-06-10T02:57:22.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.389+0000 7f0122ffd700 1 -- 172.21.15.52:0/4052155885 learned_addr learned my addr 172.21.15.52:0/4052155885 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:57:22.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.389+0000 7f0123fff700 1 --2- 172.21.15.52:0/4052155885 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558ae0f79f60 0x558ae0f80200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:57:22.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.389+0000 7f01237fe700 1 --2- 172.21.15.52:0/4052155885 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558ae0ea1080 0x558ae0f74260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:57:22.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.389+0000 7f0122ffd700 1 -- 172.21.15.52:0/4052155885 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558ae0f79f60 msgr2=0x558ae0f80200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:57:22.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.389+0000 7f0122ffd700 1 --2- 172.21.15.52:0/4052155885 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558ae0f79f60 0x558ae0f80200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:22.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.389+0000 7f0122ffd700 1 -- 172.21.15.52:0/4052155885 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558ae0ea1080 msgr2=0x558ae0f74260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:57:22.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.389+0000 7f0122ffd700 1 --2- 172.21.15.52:0/4052155885 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558ae0ea1080 0x558ae0f74260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:22.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.389+0000 7f0122ffd700 1 -- 172.21.15.52:0/4052155885 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ae0f7bf30 con 0x558ae0f747a0 2024-06-10T02:57:22.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.389+0000 7f0122ffd700 1 --2- 172.21.15.52:0/4052155885 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558ae0f747a0 0x558ae0f791c0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f01180079c0 tx=0x7f01180079f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aeba2aeb45c9aafe server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:57:22.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.389+0000 7f01097fa700 1 -- 172.21.15.52:0/4052155885 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f011803a070 con 0x558ae0f747a0 2024-06-10T02:57:22.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.389+0000 7f01097fa700 1 -- 172.21.15.52:0/4052155885 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0118031070 con 0x558ae0f747a0 2024-06-10T02:57:22.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.389+0000 7f01097fa700 1 -- 172.21.15.52:0/4052155885 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0118024040 con 0x558ae0f747a0 2024-06-10T02:57:22.393 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.390+0000 7f013499eb80 1 -- 172.21.15.52:0/4052155885 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558ae0f747a0 msgr2=0x558ae0f791c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:57:22.393 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.390+0000 7f013499eb80 1 --2- 172.21.15.52:0/4052155885 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558ae0f747a0 0x558ae0f791c0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f01180079c0 tx=0x7f01180079f0 comp rx=0 tx=0).stop 2024-06-10T02:57:22.393 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.390+0000 7f013499eb80 1 -- 172.21.15.52:0/4052155885 shutdown_connections 2024-06-10T02:57:22.393 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.390+0000 7f013499eb80 1 --2- 172.21.15.52:0/4052155885 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558ae0f79f60 0x558ae0f80200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:22.393 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.390+0000 7f013499eb80 1 --2- 172.21.15.52:0/4052155885 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558ae0ea1080 0x558ae0f74260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:22.393 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.390+0000 7f013499eb80 1 --2- 172.21.15.52:0/4052155885 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558ae0f747a0 0x558ae0f791c0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:22.393 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.390+0000 7f013499eb80 1 -- 172.21.15.52:0/4052155885 >> 172.21.15.52:0/4052155885 conn(0x558ae0f7b830 msgr2=0x558ae0f78ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:57:22.393 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.390+0000 7f013499eb80 1 -- 172.21.15.52:0/4052155885 shutdown_connections 2024-06-10T02:57:22.393 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.390+0000 7f013499eb80 1 -- 172.21.15.52:0/4052155885 wait complete. 2024-06-10T02:57:22.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.390+0000 7f013499eb80 1 -- start start 2024-06-10T02:57:22.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.391+0000 7f013499eb80 1 Processor -- start 2024-06-10T02:57:22.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.391+0000 7f013499eb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558ae0ea1080 0x558ae0f86700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:57:22.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.391+0000 7f013499eb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558ae0f747a0 0x558ae0f86c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:57:22.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.391+0000 7f013499eb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558ae0f79f60 0x558ae0f953f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:57:22.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.391+0000 7f013499eb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f01241b1a00 con 0x558ae0f79f60 2024-06-10T02:57:22.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.391+0000 7f013499eb80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f01241b1330 con 0x558ae0ea1080 2024-06-10T02:57:22.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.391+0000 7f013499eb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f01241ae470 con 0x558ae0f747a0 2024-06-10T02:57:22.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.391+0000 7f0123fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558ae0f79f60 0x558ae0f953f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:57:22.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.391+0000 7f013499eb80 10 client.0 _ll_register_callbacks cb 0x558ae0e5b5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T02:57:22.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.391+0000 7f01237fe700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558ae0ea1080 0x558ae0f86700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:57:22.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.391+0000 7f0123fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558ae0f79f60 0x558ae0f953f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55500/0 (socket says 172.21.15.52:55500) 2024-06-10T02:57:22.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.391+0000 7f0123fff700 1 -- 172.21.15.52:0/1753660848 learned_addr learned my addr 172.21.15.52:0/1753660848 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:57:22.395 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.391+0000 7f0122ffd700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558ae0f747a0 0x558ae0f86c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:57:22.395 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.392+0000 7f0123fff700 1 -- 172.21.15.52:0/1753660848 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558ae0f747a0 msgr2=0x558ae0f86c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:57:22.395 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.392+0000 7f0123fff700 1 --2- 172.21.15.52:0/1753660848 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558ae0f747a0 0x558ae0f86c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:22.395 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.392+0000 7f0123fff700 1 -- 172.21.15.52:0/1753660848 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558ae0ea1080 msgr2=0x558ae0f86700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:57:22.395 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.392+0000 7f0123fff700 1 --2- 172.21.15.52:0/1753660848 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558ae0ea1080 0x558ae0f86700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:22.395 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.392+0000 7f0123fff700 1 -- 172.21.15.52:0/1753660848 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ae0f95c00 con 0x558ae0f79f60 2024-06-10T02:57:22.395 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.392+0000 7f01237fe700 1 --2- 172.21.15.52:0/1753660848 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558ae0ea1080 0x558ae0f86700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T02:57:22.395 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.392+0000 7f0123fff700 1 --2- 172.21.15.52:0/1753660848 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558ae0f79f60 0x558ae0f953f0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f011c004660 tx=0x7f011c004970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d0474d22b13e8f4 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:57:22.395 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.392+0000 7f0102ffd700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-10T02:57:22.395 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.392+0000 7f0102ffd700 1 -- 172.21.15.52:0/1753660848 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f011c004bb0 con 0x558ae0f79f60 2024-06-10T02:57:22.395 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.392+0000 7f0102ffd700 1 -- 172.21.15.52:0/1753660848 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f011c004d50 con 0x558ae0f79f60 2024-06-10T02:57:22.395 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.392+0000 7f013499eb80 1 -- 172.21.15.52:0/1753660848 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0118031070 con 0x558ae0f79f60 2024-06-10T02:57:22.395 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.392+0000 7f0102ffd700 1 -- 172.21.15.52:0/1753660848 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f011c028050 con 0x558ae0f79f60 2024-06-10T02:57:22.395 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.392+0000 7f013499eb80 10 client.15993 fetch_fsmap learned FSMap version 33 2024-06-10T02:57:22.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.392+0000 7f013499eb80 1 -- 172.21.15.52:0/1753660848 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x558ae0ff8930 con 0x558ae0f79f60 2024-06-10T02:57:22.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.392+0000 7f0102ffd700 1 -- 172.21.15.52:0/1753660848 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0118031070 con 0x558ae0f79f60 2024-06-10T02:57:22.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.393+0000 7f0102ffd700 1 -- 172.21.15.52:0/1753660848 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 33) ==== 8382+0+0 (secure 0 0 0) 0x7f011c01e030 con 0x558ae0f79f60 2024-06-10T02:57:22.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.393+0000 7f013499eb80 10 client.15993 fetch_fsmap finished waiting for FSMap version 33 2024-06-10T02:57:22.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.393+0000 7f013499eb80 10 client.15993 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-06-10T02:57:22.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.393+0000 7f013499eb80 20 client.15993 populate_metadata read hostname 'smithi052' 2024-06-10T02:57:22.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.393+0000 7f013499eb80 1 --2- 172.21.15.52:0/1753660848 >> [v2:172.21.15.188:6834/3904544259,v1:172.21.15.188:6835/3904544259] conn(0x558ae0f7a880 0x558ae0ffb170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:57:22.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.393+0000 7f013499eb80 4 client.15993 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T02:57:22.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.393+0000 7f013499eb80 1 -- 172.21.15.52:0/1753660848 --> [v2:172.21.15.188:6834/3904544259,v1:172.21.15.188:6835/3904544259] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558ae0ecd2d0 con 0x558ae0f7a880 2024-06-10T02:57:22.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.393+0000 7f01237fe700 1 --2- 172.21.15.52:0/1753660848 >> [v2:172.21.15.188:6834/3904544259,v1:172.21.15.188:6835/3904544259] conn(0x558ae0f7a880 0x558ae0ffb170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:57:22.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.393+0000 7f0102ffd700 10 client.15993 ms_handle_connect on v2:172.21.15.188:6834/3904544259 2024-06-10T02:57:22.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.393+0000 7f01237fe700 1 --2- 172.21.15.52:0/1753660848 >> [v2:172.21.15.188:6834/3904544259,v1:172.21.15.188:6835/3904544259] conn(0x558ae0f7a880 0x558ae0ffb170 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9ad6d2d42ffe194c server_cookie=c937a62ca4222d40 in_seq=0 out_seq=0 2024-06-10T02:57:22.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.398+0000 7f0102ffd700 1 -- 172.21.15.52:0/1753660848 <== mds.2 v2:172.21.15.188:6834/3904544259 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558ae0ecd2d0 con 0x558ae0f7a880 2024-06-10T02:57:22.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.398+0000 7f0102ffd700 10 client.15993 handle_command_reply: tid=0 2024-06-10T02:57:22.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.398+0000 7f013499eb80 1 client.15993 shutdown 2024-06-10T02:57:22.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.399+0000 7f013499eb80 1 -- 172.21.15.52:0/1753660848 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558ae0f79f60 msgr2=0x558ae0f953f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:57:22.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.399+0000 7f013499eb80 1 --2- 172.21.15.52:0/1753660848 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558ae0f79f60 0x558ae0f953f0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f011c004660 tx=0x7f011c004970 comp rx=0 tx=0).stop 2024-06-10T02:57:22.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.399+0000 7f013499eb80 1 -- 172.21.15.52:0/1753660848 shutdown_connections 2024-06-10T02:57:22.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.399+0000 7f013499eb80 1 --2- 172.21.15.52:0/1753660848 >> [v2:172.21.15.188:6834/3904544259,v1:172.21.15.188:6835/3904544259] conn(0x558ae0f7a880 0x558ae0ffb170 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:22.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.399+0000 7f013499eb80 1 --2- 172.21.15.52:0/1753660848 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558ae0f79f60 0x558ae0f953f0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:22.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.399+0000 7f013499eb80 1 --2- 172.21.15.52:0/1753660848 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558ae0ea1080 0x558ae0f86700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:22.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.399+0000 7f0102ffd700 0 client.15993 ms_handle_reset on v2:172.21.15.188:6834/3904544259 2024-06-10T02:57:22.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.399+0000 7f013499eb80 1 --2- 172.21.15.52:0/1753660848 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558ae0f747a0 0x558ae0f86c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:22.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.399+0000 7f013499eb80 1 -- 172.21.15.52:0/1753660848 >> 172.21.15.52:0/1753660848 conn(0x558ae0f7b830 msgr2=0x558ae0f79340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:57:22.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.399+0000 7f013499eb80 1 -- 172.21.15.52:0/1753660848 shutdown_connections 2024-06-10T02:57:22.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.399+0000 7f013499eb80 1 -- 172.21.15.52:0/1753660848 wait complete. 2024-06-10T02:57:22.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.399+0000 7f013499eb80 20 client.15993 trim_cache size 0 max 16384 2024-06-10T02:57:22.400 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-06-10T02:57:22.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.400+0000 7f012a11b700 1 -- 172.21.15.52:0/3682888124 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f010c0a2c20 msgr2=0x7f010c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:57:22.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.400+0000 7f012a11b700 1 --2- 172.21.15.52:0/3682888124 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f010c0a2c20 0x7f010c0a50e0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f0124106490 tx=0x7f01180073d0 comp rx=0 tx=0).stop 2024-06-10T02:57:22.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.400+0000 7f012a11b700 1 -- 172.21.15.52:0/3682888124 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0124100e50 msgr2=0x7f0124102030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:57:22.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.400+0000 7f012a11b700 1 --2- 172.21.15.52:0/3682888124 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0124100e50 0x7f0124102030 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f0114000c00 tx=0x7f0114003890 comp rx=0 tx=0).stop 2024-06-10T02:57:22.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.400+0000 7f012a11b700 1 -- 172.21.15.52:0/3682888124 shutdown_connections 2024-06-10T02:57:22.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.400+0000 7f012a11b700 1 --2- 172.21.15.52:0/3682888124 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0124107570 0x7f0124102ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:22.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.400+0000 7f012a11b700 1 --2- 172.21.15.52:0/3682888124 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f010c0a2c20 0x7f010c0a50e0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:22.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.400+0000 7f012a11b700 1 --2- 172.21.15.52:0/3682888124 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0124100e50 0x7f0124102030 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:22.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.400+0000 7f012a11b700 1 --2- 172.21.15.52:0/3682888124 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0124101810 0x7f0124102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:22.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.400+0000 7f012a11b700 1 -- 172.21.15.52:0/3682888124 >> 172.21.15.52:0/3682888124 conn(0x7f01240766b0 msgr2=0x7f01240ff8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:57:22.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.400+0000 7f012a11b700 1 -- 172.21.15.52:0/3682888124 shutdown_connections 2024-06-10T02:57:22.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:22.400+0000 7f012a11b700 1 -- 172.21.15.52:0/3682888124 wait complete. 2024-06-10T02:57:22.888 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:57:22 smithi120 ceph-mon[36634]: pgmap v529: 129 pgs: 129 active+clean; 149 MiB data, 521 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 90 op/s 2024-06-10T02:57:22.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:57:22 smithi188 ceph-mon[36599]: pgmap v529: 129 pgs: 129 active+clean; 149 MiB data, 521 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 90 op/s 2024-06-10T02:57:22.976 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:57:22 smithi052 ceph-mon[32362]: pgmap v529: 129 pgs: 129 active+clean; 149 MiB data, 521 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 90 op/s 2024-06-10T02:57:23.118 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-06-10T02:57:23.439 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:57:24.888 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:57:24 smithi120 ceph-mon[36634]: pgmap v530: 129 pgs: 129 active+clean; 155 MiB data, 527 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.9 MiB/s wr, 98 op/s 2024-06-10T02:57:24.909 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:57:24 smithi052 ceph-mon[32362]: pgmap v530: 129 pgs: 129 active+clean; 155 MiB data, 527 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.9 MiB/s wr, 98 op/s 2024-06-10T02:57:24.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:57:24 smithi188 ceph-mon[36599]: pgmap v530: 129 pgs: 129 active+clean; 155 MiB data, 527 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.9 MiB/s wr, 98 op/s 2024-06-10T02:57:25.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.383+0000 7f1eb2015700 1 -- 172.21.15.52:0/1967025510 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1eac109d20 msgr2=0x7f1eac10a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:57:25.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.383+0000 7f1eaa7fc700 1 -- 172.21.15.52:0/1967025510 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ea40092e0 con 0x7f1eac109d20 2024-06-10T02:57:25.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.384+0000 7f1eb2015700 1 --2- 172.21.15.52:0/1967025510 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1eac109d20 0x7f1eac10a1a0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f1ea400b0a0 tx=0x7f1ea400b3b0 comp rx=0 tx=0).stop 2024-06-10T02:57:25.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.385+0000 7f1eb2015700 1 -- 172.21.15.52:0/1967025510 shutdown_connections 2024-06-10T02:57:25.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.385+0000 7f1eb2015700 1 --2- 172.21.15.52:0/1967025510 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1eac10a6e0 0x7f1eac1161c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:25.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.385+0000 7f1eb2015700 1 --2- 172.21.15.52:0/1967025510 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1eac10f380 0x7f1eac10f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:25.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.385+0000 7f1eb2015700 1 --2- 172.21.15.52:0/1967025510 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1eac109d20 0x7f1eac10a1a0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:25.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.386+0000 7f1eb2015700 1 -- 172.21.15.52:0/1967025510 >> 172.21.15.52:0/1967025510 conn(0x7f1eac06d9e0 msgr2=0x7f1eac06ddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:57:25.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.386+0000 7f1eb2015700 1 -- 172.21.15.52:0/1967025510 shutdown_connections 2024-06-10T02:57:25.386 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.386+0000 7f1eb2015700 1 -- 172.21.15.52:0/1967025510 wait complete. 2024-06-10T02:57:25.386 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.387+0000 7f1eb2015700 1 Processor -- start 2024-06-10T02:57:25.386 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.387+0000 7f1eb2015700 1 -- start start 2024-06-10T02:57:25.387 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.388+0000 7f1eb2015700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1eac109d20 0x7f1eac19f270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:57:25.387 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.388+0000 7f1eb2015700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1eac10a6e0 0x7f1eac19f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:57:25.388 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.388+0000 7f1eb2015700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1eac10f380 0x7f1eac1a3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:57:25.388 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.388+0000 7f1eb2015700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f1eac118e20 con 0x7f1eac109d20 2024-06-10T02:57:25.388 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.388+0000 7f1eb2015700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1eac118ca0 con 0x7f1eac10a6e0 2024-06-10T02:57:25.388 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.388+0000 7f1eb2015700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f1eac118fa0 con 0x7f1eac10f380 2024-06-10T02:57:25.388 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.388+0000 7f1eaaffd700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1eac10a6e0 0x7f1eac19f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:57:25.388 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.388+0000 7f1eaaffd700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1eac10a6e0 0x7f1eac19f7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.52:49622/0 (socket says 172.21.15.52:49622) 2024-06-10T02:57:25.388 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.388+0000 7f1eaaffd700 1 -- 172.21.15.52:0/4209416483 learned_addr learned my addr 172.21.15.52:0/4209416483 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:57:25.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.389+0000 7f1eabfff700 1 --2- 172.21.15.52:0/4209416483 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1eac10f380 0x7f1eac1a3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:57:25.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.389+0000 7f1eab7fe700 1 --2- 172.21.15.52:0/4209416483 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1eac109d20 0x7f1eac19f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:57:25.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.389+0000 7f1eaaffd700 1 -- 172.21.15.52:0/4209416483 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1eac10f380 msgr2=0x7f1eac1a3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:57:25.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.389+0000 7f1eaaffd700 1 --2- 172.21.15.52:0/4209416483 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1eac10f380 0x7f1eac1a3af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:25.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.389+0000 7f1eaaffd700 1 -- 172.21.15.52:0/4209416483 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1eac109d20 msgr2=0x7f1eac19f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:57:25.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.390+0000 7f1eaaffd700 1 --2- 172.21.15.52:0/4209416483 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1eac109d20 0x7f1eac19f270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:25.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.390+0000 7f1eaaffd700 1 -- 172.21.15.52:0/4209416483 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1eac1a4230 con 0x7f1eac10a6e0 2024-06-10T02:57:25.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.390+0000 7f1eab7fe700 1 --2- 172.21.15.52:0/4209416483 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1eac109d20 0x7f1eac19f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T02:57:25.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.391+0000 7f1eaaffd700 1 --2- 172.21.15.52:0/4209416483 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1eac10a6e0 0x7f1eac19f7b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f1ea4004cc0 tx=0x7f1ea4004da0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:57:25.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.391+0000 7f1ea8ff9700 1 -- 172.21.15.52:0/4209416483 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ea4021070 con 0x7f1eac10a6e0 2024-06-10T02:57:25.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.391+0000 7f1eb2015700 1 -- 172.21.15.52:0/4209416483 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1eac1a44c0 con 0x7f1eac10a6e0 2024-06-10T02:57:25.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.391+0000 7f1eb2015700 1 -- 172.21.15.52:0/4209416483 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1eac1abda0 con 0x7f1eac10a6e0 2024-06-10T02:57:25.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.391+0000 7f1ea8ff9700 1 -- 172.21.15.52:0/4209416483 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ea4007d40 con 0x7f1eac10a6e0 2024-06-10T02:57:25.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.392+0000 7f1ea8ff9700 1 -- 172.21.15.52:0/4209416483 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ea4005370 con 0x7f1eac10a6e0 2024-06-10T02:57:25.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.397+0000 7f1ebc898b80 1 Processor -- start 2024-06-10T02:57:25.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.398+0000 7f1ebc898b80 1 -- start start 2024-06-10T02:57:25.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.399+0000 7f1ebc898b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563b3f229640 0x563b3f229a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:57:25.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.399+0000 7f1ebc898b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563b3f229f60 0x563b3f2239e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:57:25.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.399+0000 7f1ebc898b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563b3f223f20 0x563b3f224340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:57:25.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.399+0000 7f1ebc898b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x563b3f175810 con 0x563b3f229f60 2024-06-10T02:57:25.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.399+0000 7f1ebc898b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x563b3f11d720 con 0x563b3f229640 2024-06-10T02:57:25.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.399+0000 7f1ea8ff9700 1 -- 172.21.15.52:0/4209416483 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f1ea4024070 con 0x7f1eac10a6e0 2024-06-10T02:57:25.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.399+0000 7f1ebc898b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x563b3f1717d0 con 0x563b3f223f20 2024-06-10T02:57:25.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.400+0000 7f1eabfff700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563b3f229640 0x563b3f229a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:57:25.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.400+0000 7f1eabfff700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563b3f229640 0x563b3f229a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.52:49632/0 (socket says 172.21.15.52:49632) 2024-06-10T02:57:25.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.400+0000 7f1eabfff700 1 -- 172.21.15.52:0/3621827044 learned_addr learned my addr 172.21.15.52:0/3621827044 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:57:25.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.402+0000 7f1ea8ff9700 1 --2- 172.21.15.52:0/4209416483 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f1e940a7240 0x7f1e940a9700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:57:25.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.401+0000 7f1eaaffd700 1 --2- 172.21.15.52:0/3621827044 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563b3f223f20 0x563b3f224340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:57:25.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.402+0000 7f1eab7fe700 1 --2- 172.21.15.52:0/3621827044 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563b3f229f60 0x563b3f2239e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:57:25.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.402+0000 7f1eabfff700 1 -- 172.21.15.52:0/3621827044 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563b3f223f20 msgr2=0x563b3f224340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:57:25.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.402+0000 7f1eabfff700 1 --2- 172.21.15.52:0/3621827044 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563b3f223f20 0x563b3f224340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:25.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.402+0000 7f1eabfff700 1 -- 172.21.15.52:0/3621827044 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563b3f229f60 msgr2=0x563b3f2239e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:57:25.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.402+0000 7f1eabfff700 1 --2- 172.21.15.52:0/3621827044 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563b3f229f60 0x563b3f2239e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:25.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.402+0000 7f1eabfff700 1 -- 172.21.15.52:0/3621827044 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b3f113f50 con 0x563b3f229640 2024-06-10T02:57:25.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.402+0000 7f1eabfff700 1 --2- 172.21.15.52:0/4209416483 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f1e940a7240 0x7f1e940a9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:57:25.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.403+0000 7f1eab7fe700 1 --2- 172.21.15.52:0/3621827044 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563b3f229f60 0x563b3f2239e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T02:57:25.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.403+0000 7f1ea8ff9700 1 -- 172.21.15.52:0/4209416483 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f1ea40d04f0 con 0x7f1eac10a6e0 2024-06-10T02:57:25.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.403+0000 7f1eabfff700 1 --2- 172.21.15.52:0/3621827044 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563b3f229640 0x563b3f229a20 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f1ea00037d0 tx=0x7f1ea0003ae0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2dadaf1494cccb94 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:57:25.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.404+0000 7f1eabfff700 1 --2- 172.21.15.52:0/4209416483 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f1e940a7240 0x7f1e940a9700 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f1ea0003fd0 tx=0x7f1ea001b000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:57:25.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.404+0000 7f1e91ffb700 1 -- 172.21.15.52:0/3621827044 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ea0021070 con 0x563b3f229640 2024-06-10T02:57:25.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.404+0000 7f1e91ffb700 1 -- 172.21.15.52:0/3621827044 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ea001b470 con 0x563b3f229640 2024-06-10T02:57:25.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.404+0000 7f1e91ffb700 1 -- 172.21.15.52:0/3621827044 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ea000d600 con 0x563b3f229640 2024-06-10T02:57:25.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.405+0000 7f1ebc898b80 1 -- 172.21.15.52:0/3621827044 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563b3f229640 msgr2=0x563b3f229a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:57:25.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.405+0000 7f1ebc898b80 1 --2- 172.21.15.52:0/3621827044 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563b3f229640 0x563b3f229a20 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f1ea00037d0 tx=0x7f1ea0003ae0 comp rx=0 tx=0).stop 2024-06-10T02:57:25.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.405+0000 7f1ebc898b80 1 -- 172.21.15.52:0/3621827044 shutdown_connections 2024-06-10T02:57:25.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.405+0000 7f1ebc898b80 1 --2- 172.21.15.52:0/3621827044 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563b3f223f20 0x563b3f224340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:25.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.405+0000 7f1ebc898b80 1 --2- 172.21.15.52:0/3621827044 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563b3f229640 0x563b3f229a20 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:25.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.405+0000 7f1ebc898b80 1 --2- 172.21.15.52:0/3621827044 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563b3f229f60 0x563b3f2239e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:25.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.405+0000 7f1ebc898b80 1 -- 172.21.15.52:0/3621827044 >> 172.21.15.52:0/3621827044 conn(0x563b3f14ecc0 msgr2=0x563b3f21e850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:57:25.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.405+0000 7f1ebc898b80 1 -- 172.21.15.52:0/3621827044 shutdown_connections 2024-06-10T02:57:25.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.405+0000 7f1ebc898b80 1 -- 172.21.15.52:0/3621827044 wait complete. 2024-06-10T02:57:25.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.406+0000 7f1ebc898b80 1 -- start start 2024-06-10T02:57:25.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.406+0000 7f1ebc898b80 1 Processor -- start 2024-06-10T02:57:25.407 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.407+0000 7f1ebc898b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563b3f223f20 0x563b3f22fc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:57:25.407 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.407+0000 7f1ebc898b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563b3f229640 0x563b3f22d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:57:25.407 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.407+0000 7f1ebc898b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563b3f229f60 0x563b3f22d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:57:25.407 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.407+0000 7f1ebc898b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x563b3f2309d0 con 0x563b3f223f20 2024-06-10T02:57:25.407 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.407+0000 7f1ebc898b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x563b3f115bf0 con 0x563b3f229640 2024-06-10T02:57:25.407 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.407+0000 7f1ebc898b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x563b3f230b50 con 0x563b3f229f60 2024-06-10T02:57:25.407 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.407+0000 7f1ebc898b80 10 client.0 _ll_register_callbacks cb 0x563b3f10f920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T02:57:25.407 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.407+0000 7f1eab7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563b3f223f20 0x563b3f22fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:57:25.407 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.407+0000 7f1eab7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563b3f223f20 0x563b3f22fc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55546/0 (socket says 172.21.15.52:55546) 2024-06-10T02:57:25.407 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.407+0000 7f1eab7fe700 1 -- 172.21.15.52:0/3111791690 learned_addr learned my addr 172.21.15.52:0/3111791690 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:57:25.408 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.407+0000 7f1eaaffd700 1 --2- 172.21.15.52:0/3111791690 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563b3f229640 0x563b3f22d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:57:25.408 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.407+0000 7f1eabfff700 1 --2- 172.21.15.52:0/3111791690 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563b3f229f60 0x563b3f22d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:57:25.408 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.408+0000 7f1eaaffd700 1 -- 172.21.15.52:0/3111791690 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563b3f229f60 msgr2=0x563b3f22d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:57:25.408 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.408+0000 7f1eaaffd700 1 --2- 172.21.15.52:0/3111791690 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563b3f229f60 0x563b3f22d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:25.408 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.408+0000 7f1eaaffd700 1 -- 172.21.15.52:0/3111791690 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563b3f223f20 msgr2=0x563b3f22fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:57:25.408 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.408+0000 7f1eaaffd700 1 --2- 172.21.15.52:0/3111791690 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563b3f223f20 0x563b3f22fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:25.408 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.408+0000 7f1eaaffd700 1 -- 172.21.15.52:0/3111791690 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b3f0c48d0 con 0x563b3f229640 2024-06-10T02:57:25.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.409+0000 7f1eaaffd700 1 --2- 172.21.15.52:0/3111791690 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563b3f229640 0x563b3f22d020 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1ea409c6d0 tx=0x7f1ea409c3e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ab062f8d9a25398 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:57:25.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.409+0000 7f1e8affd700 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-06-10T02:57:25.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.409+0000 7f1e8affd700 1 -- 172.21.15.52:0/3111791690 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ea402b030 con 0x563b3f229640 2024-06-10T02:57:25.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.410+0000 7f1e8affd700 1 -- 172.21.15.52:0/3111791690 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f1ea4009c10 con 0x563b3f229640 2024-06-10T02:57:25.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.410+0000 7f1ebc898b80 1 -- 172.21.15.52:0/3111791690 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ea001b470 con 0x563b3f229640 2024-06-10T02:57:25.410 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.410+0000 7f1e8affd700 1 -- 172.21.15.52:0/3111791690 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f1ea40a0d80 con 0x563b3f229640 2024-06-10T02:57:25.410 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.411+0000 7f1ebc898b80 10 client.25583 fetch_fsmap learned FSMap version 33 2024-06-10T02:57:25.410 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.411+0000 7f1ebc898b80 1 -- 172.21.15.52:0/3111791690 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x563b3f23b2b0 con 0x563b3f229640 2024-06-10T02:57:25.411 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.411+0000 7f1e8affd700 1 -- 172.21.15.52:0/3111791690 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ea001b470 con 0x563b3f229640 2024-06-10T02:57:25.411 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.411+0000 7f1e8affd700 1 -- 172.21.15.52:0/3111791690 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 33) ==== 8382+0+0 (secure 0 0 0) 0x7f1ea4011070 con 0x563b3f229640 2024-06-10T02:57:25.411 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.412+0000 7f1ebc898b80 10 client.25583 fetch_fsmap finished waiting for FSMap version 33 2024-06-10T02:57:25.411 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.412+0000 7f1ebc898b80 10 client.25583 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-06-10T02:57:25.411 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.412+0000 7f1ebc898b80 20 client.25583 populate_metadata read hostname 'smithi052' 2024-06-10T02:57:25.412 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.412+0000 7f1ebc898b80 1 --2- 172.21.15.52:0/3111791690 >> [v2:172.21.15.188:6836/819317726,v1:172.21.15.188:6837/819317726] conn(0x563b3f23d7d0 0x563b3f23fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:57:25.412 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.412+0000 7f1ebc898b80 4 client.25583 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T02:57:25.412 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.412+0000 7f1ebc898b80 1 -- 172.21.15.52:0/3111791690 --> [v2:172.21.15.188:6836/819317726,v1:172.21.15.188:6837/819317726] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563b3f182d10 con 0x563b3f23d7d0 2024-06-10T02:57:25.413 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.414+0000 7f1eab7fe700 1 --2- 172.21.15.52:0/3111791690 >> [v2:172.21.15.188:6836/819317726,v1:172.21.15.188:6837/819317726] conn(0x563b3f23d7d0 0x563b3f23fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:57:25.414 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.415+0000 7f1eab7fe700 1 --2- 172.21.15.52:0/3111791690 >> [v2:172.21.15.188:6836/819317726,v1:172.21.15.188:6837/819317726] conn(0x563b3f23d7d0 0x563b3f23fbc0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d0dbf0586b9429cf server_cookie=4c900a19d6cf8ea6 in_seq=0 out_seq=0 2024-06-10T02:57:25.414 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.415+0000 7f1e8affd700 10 client.25583 ms_handle_connect on v2:172.21.15.188:6836/819317726 2024-06-10T02:57:25.417 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.417+0000 7f1e8affd700 1 -- 172.21.15.52:0/3111791690 <== mds.1 v2:172.21.15.188:6836/819317726 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x563b3f182d10 con 0x563b3f23d7d0 2024-06-10T02:57:25.417 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.417+0000 7f1e8affd700 10 client.25583 handle_command_reply: tid=0 2024-06-10T02:57:25.418 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.418+0000 7f1ebc898b80 1 client.25583 shutdown 2024-06-10T02:57:25.420 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.419+0000 7f1ebc898b80 1 -- 172.21.15.52:0/3111791690 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563b3f229640 msgr2=0x563b3f22d020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:57:25.421 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.419+0000 7f1ebc898b80 1 --2- 172.21.15.52:0/3111791690 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563b3f229640 0x563b3f22d020 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1ea409c6d0 tx=0x7f1ea409c3e0 comp rx=0 tx=0).stop 2024-06-10T02:57:25.421 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.419+0000 7f1ebc898b80 1 -- 172.21.15.52:0/3111791690 shutdown_connections 2024-06-10T02:57:25.421 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.419+0000 7f1ebc898b80 1 --2- 172.21.15.52:0/3111791690 >> [v2:172.21.15.188:6836/819317726,v1:172.21.15.188:6837/819317726] conn(0x563b3f23d7d0 0x563b3f23fbc0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:25.421 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.419+0000 7f1ebc898b80 1 --2- 172.21.15.52:0/3111791690 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563b3f229f60 0x563b3f22d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:25.421 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.419+0000 7f1ebc898b80 1 --2- 172.21.15.52:0/3111791690 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563b3f223f20 0x563b3f22fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:25.421 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.419+0000 7f1ebc898b80 1 --2- 172.21.15.52:0/3111791690 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563b3f229640 0x563b3f22d020 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:25.421 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.419+0000 7f1ebc898b80 1 -- 172.21.15.52:0/3111791690 >> 172.21.15.52:0/3111791690 conn(0x563b3f14ecc0 msgr2=0x563b3f2229d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:57:25.421 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.419+0000 7f1e8affd700 0 client.25583 ms_handle_reset on v2:172.21.15.188:6836/819317726 2024-06-10T02:57:25.421 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.419+0000 7f1ebc898b80 1 -- 172.21.15.52:0/3111791690 shutdown_connections 2024-06-10T02:57:25.421 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.419+0000 7f1ebc898b80 1 -- 172.21.15.52:0/3111791690 wait complete. 2024-06-10T02:57:25.421 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.419+0000 7f1ebc898b80 20 client.25583 trim_cache size 0 max 16384 2024-06-10T02:57:25.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.457+0000 7f1ebc898b80 1 Processor -- start 2024-06-10T02:57:25.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.458+0000 7f1ebc898b80 1 -- start start 2024-06-10T02:57:25.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.458+0000 7f1ebc898b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563b3f223f20 0x563b3f2239e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:57:25.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.458+0000 7f1ebc898b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563b3f229640 0x563b3f22b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:57:25.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.458+0000 7f1ebc898b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563b3f229f60 0x563b3f22bef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:57:25.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.458+0000 7f1ebc898b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f1eac1b48f0 con 0x563b3f229f60 2024-06-10T02:57:25.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.458+0000 7f1ebc898b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1eac1b6520 con 0x563b3f223f20 2024-06-10T02:57:25.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.458+0000 7f1ebc898b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f1eac1b4fc0 con 0x563b3f229640 2024-06-10T02:57:25.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.459+0000 7f1eaaffd700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563b3f229640 0x563b3f22b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:57:25.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.459+0000 7f1eaaffd700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563b3f229640 0x563b3f22b9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.52:52200/0 (socket says 172.21.15.52:52200) 2024-06-10T02:57:25.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.459+0000 7f1eaaffd700 1 -- 172.21.15.52:0/2035390253 learned_addr learned my addr 172.21.15.52:0/2035390253 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:57:25.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.459+0000 7f1eab7fe700 1 --2- 172.21.15.52:0/2035390253 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563b3f223f20 0x563b3f2239e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:57:25.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.459+0000 7f1eabfff700 1 --2- 172.21.15.52:0/2035390253 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563b3f229f60 0x563b3f22bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:57:25.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.460+0000 7f1eabfff700 1 -- 172.21.15.52:0/2035390253 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563b3f229640 msgr2=0x563b3f22b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:57:25.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.460+0000 7f1eabfff700 1 --2- 172.21.15.52:0/2035390253 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563b3f229640 0x563b3f22b9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:25.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.460+0000 7f1eabfff700 1 -- 172.21.15.52:0/2035390253 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563b3f223f20 msgr2=0x563b3f2239e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:57:25.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.460+0000 7f1eabfff700 1 --2- 172.21.15.52:0/2035390253 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563b3f223f20 0x563b3f2239e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:25.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.460+0000 7f1eabfff700 1 -- 172.21.15.52:0/2035390253 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b3f2401a0 con 0x563b3f229f60 2024-06-10T02:57:25.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.461+0000 7f1eabfff700 1 --2- 172.21.15.52:0/2035390253 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563b3f229f60 0x563b3f22bef0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f1ea001be30 tx=0x7f1ea001bf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d54a88494b4c6d05 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:57:25.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.461+0000 7f1e917fa700 1 -- 172.21.15.52:0/2035390253 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ea0021040 con 0x563b3f229f60 2024-06-10T02:57:25.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.462+0000 7f1e917fa700 1 -- 172.21.15.52:0/2035390253 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ea001b470 con 0x563b3f229f60 2024-06-10T02:57:25.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.463+0000 7f1ebc898b80 1 -- 172.21.15.52:0/2035390253 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563b3f229f60 msgr2=0x563b3f22bef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:57:25.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.463+0000 7f1ebc898b80 1 --2- 172.21.15.52:0/2035390253 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563b3f229f60 0x563b3f22bef0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f1ea001be30 tx=0x7f1ea001bf00 comp rx=0 tx=0).stop 2024-06-10T02:57:25.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.465+0000 7f1ebc898b80 1 -- 172.21.15.52:0/2035390253 shutdown_connections 2024-06-10T02:57:25.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.465+0000 7f1ebc898b80 1 --2- 172.21.15.52:0/2035390253 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563b3f229f60 0x563b3f22bef0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:25.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.465+0000 7f1ebc898b80 1 --2- 172.21.15.52:0/2035390253 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563b3f223f20 0x563b3f2239e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:25.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.465+0000 7f1ebc898b80 1 --2- 172.21.15.52:0/2035390253 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563b3f229640 0x563b3f22b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:25.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.465+0000 7f1ebc898b80 1 -- 172.21.15.52:0/2035390253 >> 172.21.15.52:0/2035390253 conn(0x563b3f14ecc0 msgr2=0x563b3f21e380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:57:25.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.465+0000 7f1ebc898b80 1 -- 172.21.15.52:0/2035390253 shutdown_connections 2024-06-10T02:57:25.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.467+0000 7f1ebc898b80 1 -- 172.21.15.52:0/2035390253 wait complete. 2024-06-10T02:57:25.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.468+0000 7f1ebc898b80 1 -- start start 2024-06-10T02:57:25.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.468+0000 7f1ebc898b80 1 Processor -- start 2024-06-10T02:57:25.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.468+0000 7f1ebc898b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563b3f223f20 0x563b3f236020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:57:25.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.468+0000 7f1ebc898b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563b3f229640 0x563b3f236560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:57:25.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.469+0000 7f1ebc898b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563b3f229f60 0x563b3f2447f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:57:25.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.469+0000 7f1ebc898b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f1eac1b5e20 con 0x563b3f229f60 2024-06-10T02:57:25.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.469+0000 7f1ebc898b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1eac1b5750 con 0x563b3f223f20 2024-06-10T02:57:25.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.469+0000 7f1ebc898b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f1eac1b28b0 con 0x563b3f229640 2024-06-10T02:57:25.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.469+0000 7f1ebc898b80 10 client.0 _ll_register_callbacks cb 0x563b3f10f920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T02:57:25.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.469+0000 7f1eab7fe700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563b3f223f20 0x563b3f236020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:57:25.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.469+0000 7f1eaaffd700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563b3f229640 0x563b3f236560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:57:25.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.469+0000 7f1eab7fe700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563b3f223f20 0x563b3f236020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.52:49652/0 (socket says 172.21.15.52:49652) 2024-06-10T02:57:25.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.469+0000 7f1eab7fe700 1 -- 172.21.15.52:0/2321283160 learned_addr learned my addr 172.21.15.52:0/2321283160 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:57:25.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.470+0000 7f1eabfff700 1 --2- 172.21.15.52:0/2321283160 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563b3f229f60 0x563b3f2447f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:57:25.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.470+0000 7f1eaaffd700 1 -- 172.21.15.52:0/2321283160 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563b3f223f20 msgr2=0x563b3f236020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:57:25.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.470+0000 7f1eaaffd700 1 --2- 172.21.15.52:0/2321283160 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563b3f223f20 0x563b3f236020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:25.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.470+0000 7f1eaaffd700 1 -- 172.21.15.52:0/2321283160 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563b3f229f60 msgr2=0x563b3f2447f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:57:25.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.470+0000 7f1eaaffd700 1 --2- 172.21.15.52:0/2321283160 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563b3f229f60 0x563b3f2447f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:25.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.470+0000 7f1eaaffd700 1 -- 172.21.15.52:0/2321283160 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b3f244fe0 con 0x563b3f229640 2024-06-10T02:57:25.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.471+0000 7f1eabfff700 1 --2- 172.21.15.52:0/2321283160 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563b3f229f60 0x563b3f2447f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T02:57:25.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.474+0000 7f1eaaffd700 1 --2- 172.21.15.52:0/2321283160 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563b3f229640 0x563b3f236560 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1ea40a5080 tx=0x7f1ea4029fb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f96d0712e426967a server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:57:25.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.475+0000 7f1e8affd700 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-06-10T02:57:25.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.475+0000 7f1e8affd700 1 -- 172.21.15.52:0/2321283160 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ea402b070 con 0x563b3f229640 2024-06-10T02:57:25.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.475+0000 7f1e8affd700 1 -- 172.21.15.52:0/2321283160 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f1ea40074e0 con 0x563b3f229640 2024-06-10T02:57:25.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.476+0000 7f1ebc898b80 1 -- 172.21.15.52:0/2321283160 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ea001b470 con 0x563b3f229640 2024-06-10T02:57:25.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.479+0000 7f1e8affd700 1 -- 172.21.15.52:0/2321283160 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f1ea40b0e60 con 0x563b3f229640 2024-06-10T02:57:25.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.479+0000 7f1ebc898b80 10 client.25576 fetch_fsmap learned FSMap version 33 2024-06-10T02:57:25.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.479+0000 7f1ebc898b80 1 -- 172.21.15.52:0/2321283160 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x563b3f2a0ed0 con 0x563b3f229640 2024-06-10T02:57:25.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.481+0000 7f1e8affd700 1 -- 172.21.15.52:0/2321283160 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ea001b470 con 0x563b3f229640 2024-06-10T02:57:25.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.483+0000 7f1e8affd700 1 -- 172.21.15.52:0/2321283160 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 33) ==== 8382+0+0 (secure 0 0 0) 0x7f1ea4011040 con 0x563b3f229640 2024-06-10T02:57:25.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.483+0000 7f1ebc898b80 10 client.25576 fetch_fsmap finished waiting for FSMap version 33 2024-06-10T02:57:25.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.483+0000 7f1ebc898b80 10 client.25576 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-06-10T02:57:25.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.483+0000 7f1ebc898b80 20 client.25576 populate_metadata read hostname 'smithi052' 2024-06-10T02:57:25.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.483+0000 7f1ebc898b80 1 --2- 172.21.15.52:0/2321283160 >> [v2:172.21.15.188:6836/819317726,v1:172.21.15.188:6837/819317726] conn(0x563b3f2a1450 0x563b3f2a3870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:57:25.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.483+0000 7f1ebc898b80 4 client.25576 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T02:57:25.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.483+0000 7f1ebc898b80 1 -- 172.21.15.52:0/2321283160 --> [v2:172.21.15.188:6836/819317726,v1:172.21.15.188:6837/819317726] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563b3f17d290 con 0x563b3f2a1450 2024-06-10T02:57:25.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.484+0000 7f1eab7fe700 1 --2- 172.21.15.52:0/2321283160 >> [v2:172.21.15.188:6836/819317726,v1:172.21.15.188:6837/819317726] conn(0x563b3f2a1450 0x563b3f2a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:57:25.486 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.486+0000 7f1eab7fe700 1 --2- 172.21.15.52:0/2321283160 >> [v2:172.21.15.188:6836/819317726,v1:172.21.15.188:6837/819317726] conn(0x563b3f2a1450 0x563b3f2a3870 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ea522719c5c51c5c server_cookie=e0856877ab09117b in_seq=0 out_seq=0 2024-06-10T02:57:25.486 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.487+0000 7f1e8affd700 10 client.25576 ms_handle_connect on v2:172.21.15.188:6836/819317726 2024-06-10T02:57:25.487 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.487+0000 7f1e8affd700 1 -- 172.21.15.52:0/2321283160 <== mds.1 v2:172.21.15.188:6836/819317726 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563b3f17d290 con 0x563b3f2a1450 2024-06-10T02:57:25.487 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.487+0000 7f1e8affd700 10 client.25576 handle_command_reply: tid=0 2024-06-10T02:57:25.487 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.487+0000 7f1ebc898b80 1 client.25576 shutdown 2024-06-10T02:57:25.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.488+0000 7f1ebc898b80 1 -- 172.21.15.52:0/2321283160 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563b3f229640 msgr2=0x563b3f236560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:57:25.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.488+0000 7f1ebc898b80 1 --2- 172.21.15.52:0/2321283160 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563b3f229640 0x563b3f236560 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1ea40a5080 tx=0x7f1ea4029fb0 comp rx=0 tx=0).stop 2024-06-10T02:57:25.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.488+0000 7f1ebc898b80 1 -- 172.21.15.52:0/2321283160 shutdown_connections 2024-06-10T02:57:25.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.488+0000 7f1ebc898b80 1 --2- 172.21.15.52:0/2321283160 >> [v2:172.21.15.188:6836/819317726,v1:172.21.15.188:6837/819317726] conn(0x563b3f2a1450 0x563b3f2a3870 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:25.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.488+0000 7f1ebc898b80 1 --2- 172.21.15.52:0/2321283160 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563b3f229f60 0x563b3f2447f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:25.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.488+0000 7f1ebc898b80 1 --2- 172.21.15.52:0/2321283160 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563b3f223f20 0x563b3f236020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:25.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.488+0000 7f1ebc898b80 1 --2- 172.21.15.52:0/2321283160 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563b3f229640 0x563b3f236560 secure :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1ea40a5080 tx=0x7f1ea4029fb0 comp rx=0 tx=0).stop 2024-06-10T02:57:25.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.488+0000 7f1e8affd700 0 client.25576 ms_handle_reset on v2:172.21.15.188:6836/819317726 2024-06-10T02:57:25.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.488+0000 7f1ebc898b80 1 -- 172.21.15.52:0/2321283160 >> 172.21.15.52:0/2321283160 conn(0x563b3f14ecc0 msgr2=0x563b3f22b970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:57:25.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.491+0000 7f1ebc898b80 1 -- 172.21.15.52:0/2321283160 shutdown_connections 2024-06-10T02:57:25.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.494+0000 7f1ebc898b80 1 -- 172.21.15.52:0/2321283160 wait complete. 2024-06-10T02:57:25.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.495+0000 7f1ebc898b80 20 client.25576 trim_cache size 0 max 16384 2024-06-10T02:57:25.495 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-06-10T02:57:25.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.496+0000 7f1eb2015700 1 -- 172.21.15.52:0/4209416483 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f1e940a7240 msgr2=0x7f1e940a9700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:57:25.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.496+0000 7f1eb2015700 1 --2- 172.21.15.52:0/4209416483 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f1e940a7240 0x7f1e940a9700 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f1ea0003fd0 tx=0x7f1ea001b000 comp rx=0 tx=0).stop 2024-06-10T02:57:25.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.496+0000 7f1eb2015700 1 -- 172.21.15.52:0/4209416483 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1eac10a6e0 msgr2=0x7f1eac19f7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:57:25.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.496+0000 7f1eb2015700 1 --2- 172.21.15.52:0/4209416483 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1eac10a6e0 0x7f1eac19f7b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f1ea4004cc0 tx=0x7f1ea4004da0 comp rx=0 tx=0).stop 2024-06-10T02:57:25.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.497+0000 7f1eb2015700 1 -- 172.21.15.52:0/4209416483 shutdown_connections 2024-06-10T02:57:25.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.497+0000 7f1eb2015700 1 --2- 172.21.15.52:0/4209416483 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1eac10f380 0x7f1eac1a3af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:25.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.497+0000 7f1eb2015700 1 --2- 172.21.15.52:0/4209416483 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f1e940a7240 0x7f1e940a9700 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:25.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.497+0000 7f1eb2015700 1 --2- 172.21.15.52:0/4209416483 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1eac109d20 0x7f1eac19f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:25.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.497+0000 7f1eb2015700 1 --2- 172.21.15.52:0/4209416483 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1eac10a6e0 0x7f1eac19f7b0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:25.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.497+0000 7f1eb2015700 1 -- 172.21.15.52:0/4209416483 >> 172.21.15.52:0/4209416483 conn(0x7f1eac06d9e0 msgr2=0x7f1eac072d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:57:25.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.497+0000 7f1eb2015700 1 -- 172.21.15.52:0/4209416483 shutdown_connections 2024-06-10T02:57:25.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:25.497+0000 7f1eb2015700 1 -- 172.21.15.52:0/4209416483 wait complete. 2024-06-10T02:57:26.180 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-06-10T02:57:26.436 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:57:26.872 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:57:26 smithi052 ceph-mon[32362]: pgmap v531: 129 pgs: 129 active+clean; 155 MiB data, 528 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.9 MiB/s wr, 84 op/s 2024-06-10T02:57:26.888 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:57:26 smithi120 ceph-mon[36634]: pgmap v531: 129 pgs: 129 active+clean; 155 MiB data, 528 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.9 MiB/s wr, 84 op/s 2024-06-10T02:57:26.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:57:26 smithi188 ceph-mon[36599]: pgmap v531: 129 pgs: 129 active+clean; 155 MiB data, 528 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.9 MiB/s wr, 84 op/s 2024-06-10T02:57:27.242 INFO:tasks.workunit.client.0.smithi052.stdout: 42 42767 169 27169 307 21220 1040 2024-06-10T02:57:28.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.181+0000 7f096b26a700 1 -- 172.21.15.52:0/2606083161 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0964102380 msgr2=0x7f0964105ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:57:28.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.181+0000 7f096b26a700 1 --2- 172.21.15.52:0/2606083161 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0964102380 0x7f0964105ff0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f09540099c0 tx=0x7f0954009cd0 comp rx=0 tx=0).stop 2024-06-10T02:57:28.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.181+0000 7f096b26a700 1 -- 172.21.15.52:0/2606083161 shutdown_connections 2024-06-10T02:57:28.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.181+0000 7f096b26a700 1 --2- 172.21.15.52:0/2606083161 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0964106720 0x7f096410e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:28.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.181+0000 7f096b26a700 1 --2- 172.21.15.52:0/2606083161 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f09641019d0 0x7f0964101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:28.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.181+0000 7f096b26a700 1 --2- 172.21.15.52:0/2606083161 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0964102380 0x7f0964105ff0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:28.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.181+0000 7f096b26a700 1 -- 172.21.15.52:0/2606083161 >> 172.21.15.52:0/2606083161 conn(0x7f0964078340 msgr2=0x7f0964078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:57:28.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.181+0000 7f096b26a700 1 -- 172.21.15.52:0/2606083161 shutdown_connections 2024-06-10T02:57:28.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.181+0000 7f096b26a700 1 -- 172.21.15.52:0/2606083161 wait complete. 2024-06-10T02:57:28.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.182+0000 7f096b26a700 1 Processor -- start 2024-06-10T02:57:28.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.182+0000 7f096b26a700 1 -- start start 2024-06-10T02:57:28.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.182+0000 7f096b26a700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f09641019d0 0x7f0964103090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:57:28.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.182+0000 7f096b26a700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0964102380 0x7f09641035d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:57:28.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.182+0000 7f096b26a700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0964106720 0x7f0964103b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:57:28.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.182+0000 7f096b26a700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f09641116c0 con 0x7f0964106720 2024-06-10T02:57:28.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.182+0000 7f096b26a700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f0964111540 con 0x7f0964102380 2024-06-10T02:57:28.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.182+0000 7f096b26a700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f0964111840 con 0x7f09641019d0 2024-06-10T02:57:28.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.182+0000 7f0969807700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0964106720 0x7f0964103b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:57:28.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.182+0000 7f0969006700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f09641019d0 0x7f0964103090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:57:28.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.182+0000 7f0969807700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0964106720 0x7f0964103b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55574/0 (socket says 172.21.15.52:55574) 2024-06-10T02:57:28.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.182+0000 7f0969807700 1 -- 172.21.15.52:0/2722235508 learned_addr learned my addr 172.21.15.52:0/2722235508 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:57:28.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.182+0000 7f0969006700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f09641019d0 0x7f0964103090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.52:52228/0 (socket says 172.21.15.52:52228) 2024-06-10T02:57:28.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.182+0000 7f0968805700 1 --2- 172.21.15.52:0/2722235508 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0964102380 0x7f09641035d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:57:28.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.182+0000 7f0969807700 1 -- 172.21.15.52:0/2722235508 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f09641019d0 msgr2=0x7f0964103090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:57:28.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.182+0000 7f0969807700 1 --2- 172.21.15.52:0/2722235508 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f09641019d0 0x7f0964103090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:28.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.182+0000 7f0969807700 1 -- 172.21.15.52:0/2722235508 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0964102380 msgr2=0x7f09641035d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:57:28.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.182+0000 7f0969807700 1 --2- 172.21.15.52:0/2722235508 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0964102380 0x7f09641035d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:28.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.182+0000 7f0969807700 1 -- 172.21.15.52:0/2722235508 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09641a7250 con 0x7f0964106720 2024-06-10T02:57:28.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.183+0000 7f0969006700 1 --2- 172.21.15.52:0/2722235508 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f09641019d0 0x7f0964103090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T02:57:28.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.183+0000 7f0969807700 1 --2- 172.21.15.52:0/2722235508 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0964106720 0x7f0964103b10 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f095c00ef30 tx=0x7f095c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:57:28.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.183+0000 7f095a7fc700 1 -- 172.21.15.52:0/2722235508 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f095c0092e0 con 0x7f0964106720 2024-06-10T02:57:28.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.183+0000 7f095a7fc700 1 -- 172.21.15.52:0/2722235508 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f095c004bb0 con 0x7f0964106720 2024-06-10T02:57:28.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.183+0000 7f096b26a700 1 -- 172.21.15.52:0/2722235508 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09641a74e0 con 0x7f0964106720 2024-06-10T02:57:28.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.183+0000 7f095a7fc700 1 -- 172.21.15.52:0/2722235508 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f095c004280 con 0x7f0964106720 2024-06-10T02:57:28.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.183+0000 7f096b26a700 1 -- 172.21.15.52:0/2722235508 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09641a7a40 con 0x7f0964106720 2024-06-10T02:57:28.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.185+0000 7f095a7fc700 1 -- 172.21.15.52:0/2722235508 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f095c01b030 con 0x7f0964106720 2024-06-10T02:57:28.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.186+0000 7f095a7fc700 1 --2- 172.21.15.52:0/2722235508 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f09500a2ca0 0x7f09500a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:57:28.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.186+0000 7f0969006700 1 --2- 172.21.15.52:0/2722235508 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f09500a2ca0 0x7f09500a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:57:28.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.186+0000 7f095a7fc700 1 -- 172.21.15.52:0/2722235508 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f095c0cee50 con 0x7f0964106720 2024-06-10T02:57:28.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.186+0000 7f0969006700 1 --2- 172.21.15.52:0/2722235508 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f09500a2ca0 0x7f09500a5160 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f0960007fd0 tx=0x7f09600073d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:57:28.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.187+0000 7f0975aedb80 1 Processor -- start 2024-06-10T02:57:28.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.187+0000 7f0975aedb80 1 -- start start 2024-06-10T02:57:28.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.187+0000 7f0975aedb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c628b72640 0x55c628b72a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:57:28.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.187+0000 7f0975aedb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c628b72f60 0x55c628b6c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:57:28.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.187+0000 7f0975aedb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c628b6cf20 0x55c628b6d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:57:28.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.187+0000 7f0975aedb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55c628abe810 con 0x55c628b72640 2024-06-10T02:57:28.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.187+0000 7f0975aedb80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55c628a66720 con 0x55c628b72f60 2024-06-10T02:57:28.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.187+0000 7f0975aedb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55c628aba7d0 con 0x55c628b6cf20 2024-06-10T02:57:28.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.187+0000 7f0969006700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c628b72640 0x55c628b72a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:57:28.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.187+0000 7f0969006700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c628b72640 0x55c628b72a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55588/0 (socket says 172.21.15.52:55588) 2024-06-10T02:57:28.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.187+0000 7f0969006700 1 -- 172.21.15.52:0/517538282 learned_addr learned my addr 172.21.15.52:0/517538282 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:57:28.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.187+0000 7f0969807700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c628b6cf20 0x55c628b6d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:57:28.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.188+0000 7f0968805700 1 --2- 172.21.15.52:0/517538282 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c628b72f60 0x55c628b6c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:57:28.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.188+0000 7f0969807700 1 -- 172.21.15.52:0/517538282 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c628b72f60 msgr2=0x55c628b6c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:57:28.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.188+0000 7f0969807700 1 --2- 172.21.15.52:0/517538282 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c628b72f60 0x55c628b6c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:28.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.188+0000 7f0969807700 1 -- 172.21.15.52:0/517538282 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c628b72640 msgr2=0x55c628b72a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:57:28.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.188+0000 7f0969807700 1 --2- 172.21.15.52:0/517538282 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c628b72640 0x55c628b72a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:28.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.188+0000 7f0969807700 1 -- 172.21.15.52:0/517538282 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c628a5cf50 con 0x55c628b6cf20 2024-06-10T02:57:28.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.188+0000 7f0969006700 1 --2- 172.21.15.52:0/517538282 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c628b72640 0x55c628b72a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T02:57:28.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.188+0000 7f0968805700 1 --2- 172.21.15.52:0/517538282 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c628b72f60 0x55c628b6c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T02:57:28.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.188+0000 7f0969807700 1 --2- 172.21.15.52:0/517538282 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c628b6cf20 0x55c628b6d340 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f095c0cc5f0 tx=0x7f095c097760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b86bf1648bbcfee9 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:57:28.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.188+0000 7f094f7fe700 1 -- 172.21.15.52:0/517538282 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f095c09ba90 con 0x55c628b6cf20 2024-06-10T02:57:28.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.188+0000 7f094f7fe700 1 -- 172.21.15.52:0/517538282 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f095c09b390 con 0x55c628b6cf20 2024-06-10T02:57:28.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.188+0000 7f094f7fe700 1 -- 172.21.15.52:0/517538282 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f095c0a3ae0 con 0x55c628b6cf20 2024-06-10T02:57:28.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.189+0000 7f0975aedb80 1 -- 172.21.15.52:0/517538282 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c628b6cf20 msgr2=0x55c628b6d340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:57:28.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.189+0000 7f0975aedb80 1 --2- 172.21.15.52:0/517538282 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c628b6cf20 0x55c628b6d340 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f095c0cc5f0 tx=0x7f095c097760 comp rx=0 tx=0).stop 2024-06-10T02:57:28.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.189+0000 7f0975aedb80 1 -- 172.21.15.52:0/517538282 shutdown_connections 2024-06-10T02:57:28.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.189+0000 7f0975aedb80 1 --2- 172.21.15.52:0/517538282 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c628b6cf20 0x55c628b6d340 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:28.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.189+0000 7f0975aedb80 1 --2- 172.21.15.52:0/517538282 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c628b72640 0x55c628b72a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:28.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.189+0000 7f0975aedb80 1 --2- 172.21.15.52:0/517538282 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c628b72f60 0x55c628b6c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:28.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.189+0000 7f0975aedb80 1 -- 172.21.15.52:0/517538282 >> 172.21.15.52:0/517538282 conn(0x55c628a97cc0 msgr2=0x55c628b67850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:57:28.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.189+0000 7f0975aedb80 1 -- 172.21.15.52:0/517538282 shutdown_connections 2024-06-10T02:57:28.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.189+0000 7f0975aedb80 1 -- 172.21.15.52:0/517538282 wait complete. 2024-06-10T02:57:28.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.189+0000 7f0975aedb80 1 -- start start 2024-06-10T02:57:28.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.189+0000 7f0975aedb80 1 Processor -- start 2024-06-10T02:57:28.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.190+0000 7f0975aedb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c628b6cf20 0x55c628b78ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:57:28.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.190+0000 7f0975aedb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c628b72640 0x55c628b75fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:57:28.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.190+0000 7f0975aedb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c628b72f60 0x55c628b764e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:57:28.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.190+0000 7f0975aedb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55c628b799d0 con 0x55c628b72f60 2024-06-10T02:57:28.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.190+0000 7f0975aedb80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55c628a5ebf0 con 0x55c628b72640 2024-06-10T02:57:28.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.190+0000 7f0975aedb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55c628b79b50 con 0x55c628b6cf20 2024-06-10T02:57:28.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.190+0000 7f0969807700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c628b72f60 0x55c628b764e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:57:28.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.190+0000 7f0969807700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c628b72f60 0x55c628b764e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55592/0 (socket says 172.21.15.52:55592) 2024-06-10T02:57:28.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.190+0000 7f0969807700 1 -- 172.21.15.52:0/1113311509 learned_addr learned my addr 172.21.15.52:0/1113311509 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:57:28.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.190+0000 7f0969006700 1 --2- 172.21.15.52:0/1113311509 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c628b6cf20 0x55c628b78ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:57:28.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.190+0000 7f0975aedb80 10 client.0 _ll_register_callbacks cb 0x55c628a58920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T02:57:28.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.190+0000 7f0968805700 1 --2- 172.21.15.52:0/1113311509 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c628b72640 0x55c628b75fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:57:28.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.190+0000 7f0969807700 1 -- 172.21.15.52:0/1113311509 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c628b6cf20 msgr2=0x55c628b78ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:57:28.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.190+0000 7f0969807700 1 --2- 172.21.15.52:0/1113311509 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c628b6cf20 0x55c628b78ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:28.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.190+0000 7f0969807700 1 -- 172.21.15.52:0/1113311509 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c628b72640 msgr2=0x55c628b75fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:57:28.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.190+0000 7f0969807700 1 --2- 172.21.15.52:0/1113311509 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c628b72640 0x55c628b75fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:28.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.190+0000 7f0969807700 1 -- 172.21.15.52:0/1113311509 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c628a0d8d0 con 0x55c628b72f60 2024-06-10T02:57:28.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.190+0000 7f0968805700 1 --2- 172.21.15.52:0/1113311509 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c628b72640 0x55c628b75fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T02:57:28.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.190+0000 7f0969807700 1 --2- 172.21.15.52:0/1113311509 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c628b72f60 0x55c628b764e0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f095c000f80 tx=0x7f095c0a3e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d14337dee308aa9a server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:57:28.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.190+0000 7f094cff9700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-10T02:57:28.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.190+0000 7f094cff9700 1 -- 172.21.15.52:0/1113311509 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f095c09ba90 con 0x55c628b72f60 2024-06-10T02:57:28.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.191+0000 7f094cff9700 1 -- 172.21.15.52:0/1113311509 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f095c0b4260 con 0x55c628b72f60 2024-06-10T02:57:28.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.191+0000 7f0975aedb80 1 -- 172.21.15.52:0/1113311509 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f095c09b390 con 0x55c628b72f60 2024-06-10T02:57:28.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.191+0000 7f094cff9700 1 -- 172.21.15.52:0/1113311509 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f095c082050 con 0x55c628b72f60 2024-06-10T02:57:28.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.191+0000 7f0975aedb80 10 client.16014 fetch_fsmap learned FSMap version 33 2024-06-10T02:57:28.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.191+0000 7f0975aedb80 1 -- 172.21.15.52:0/1113311509 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55c628b84200 con 0x55c628b72f60 2024-06-10T02:57:28.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.191+0000 7f094cff9700 1 -- 172.21.15.52:0/1113311509 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f095c09b390 con 0x55c628b72f60 2024-06-10T02:57:28.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.191+0000 7f094cff9700 1 -- 172.21.15.52:0/1113311509 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 33) ==== 8382+0+0 (secure 0 0 0) 0x7f095c0bb0e0 con 0x55c628b72f60 2024-06-10T02:57:28.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.191+0000 7f0975aedb80 10 client.16014 fetch_fsmap finished waiting for FSMap version 33 2024-06-10T02:57:28.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.191+0000 7f0975aedb80 10 client.16014 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T02:57:28.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.191+0000 7f0975aedb80 20 client.16014 populate_metadata read hostname 'smithi052' 2024-06-10T02:57:28.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.191+0000 7f0975aedb80 1 --2- 172.21.15.52:0/1113311509 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55c628b866d0 0x55c628b88ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:57:28.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.191+0000 7f0975aedb80 4 client.16014 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T02:57:28.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.191+0000 7f0975aedb80 1 -- 172.21.15.52:0/1113311509 --> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c628acbd10 con 0x55c628b866d0 2024-06-10T02:57:28.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.191+0000 7f0969006700 1 --2- 172.21.15.52:0/1113311509 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55c628b866d0 0x55c628b88ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:57:28.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.192+0000 7f0969006700 1 --2- 172.21.15.52:0/1113311509 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55c628b866d0 0x55c628b88ac0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a0cb56a1081e1faa server_cookie=16918e92820886a6 in_seq=0 out_seq=0 2024-06-10T02:57:28.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.192+0000 7f094cff9700 10 client.16014 ms_handle_connect on v2:172.21.15.188:6838/310444836 2024-06-10T02:57:28.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.192+0000 7f094cff9700 1 -- 172.21.15.52:0/1113311509 <== mds.0 v2:172.21.15.188:6838/310444836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55c628acbd10 con 0x55c628b866d0 2024-06-10T02:57:28.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.192+0000 7f094cff9700 10 client.16014 handle_command_reply: tid=0 2024-06-10T02:57:28.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.193+0000 7f0975aedb80 1 client.16014 shutdown 2024-06-10T02:57:28.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.193+0000 7f0975aedb80 1 -- 172.21.15.52:0/1113311509 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c628b72f60 msgr2=0x55c628b764e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:57:28.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.193+0000 7f0975aedb80 1 --2- 172.21.15.52:0/1113311509 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c628b72f60 0x55c628b764e0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f095c000f80 tx=0x7f095c0a3e80 comp rx=0 tx=0).stop 2024-06-10T02:57:28.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.193+0000 7f0975aedb80 1 -- 172.21.15.52:0/1113311509 shutdown_connections 2024-06-10T02:57:28.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.193+0000 7f0975aedb80 1 --2- 172.21.15.52:0/1113311509 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55c628b866d0 0x55c628b88ac0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:28.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.193+0000 7f0975aedb80 1 --2- 172.21.15.52:0/1113311509 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c628b72f60 0x55c628b764e0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:28.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.193+0000 7f0975aedb80 1 --2- 172.21.15.52:0/1113311509 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c628b6cf20 0x55c628b78ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:28.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.193+0000 7f0975aedb80 1 --2- 172.21.15.52:0/1113311509 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c628b72640 0x55c628b75fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:28.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.193+0000 7f0975aedb80 1 -- 172.21.15.52:0/1113311509 >> 172.21.15.52:0/1113311509 conn(0x55c628a97cc0 msgr2=0x55c628b6b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:57:28.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.193+0000 7f094cff9700 0 client.16014 ms_handle_reset on v2:172.21.15.188:6838/310444836 2024-06-10T02:57:28.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.193+0000 7f0975aedb80 1 -- 172.21.15.52:0/1113311509 shutdown_connections 2024-06-10T02:57:28.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.193+0000 7f0975aedb80 1 -- 172.21.15.52:0/1113311509 wait complete. 2024-06-10T02:57:28.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.193+0000 7f0975aedb80 20 client.16014 trim_cache size 0 max 16384 2024-06-10T02:57:28.217 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.217+0000 7f0975aedb80 1 Processor -- start 2024-06-10T02:57:28.217 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.217+0000 7f0975aedb80 1 -- start start 2024-06-10T02:57:28.217 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.218+0000 7f0975aedb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c628b6cf20 0x55c628b6c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:57:28.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.218+0000 7f0969006700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c628b6cf20 0x55c628b6c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:57:28.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.218+0000 7f0969006700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c628b6cf20 0x55c628b6c9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55598/0 (socket says 172.21.15.52:55598) 2024-06-10T02:57:28.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.218+0000 7f0975aedb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c628b72640 0x55c628b70730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:57:28.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.218+0000 7f0969006700 1 -- 172.21.15.52:0/321532192 learned_addr learned my addr 172.21.15.52:0/321532192 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:57:28.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.218+0000 7f0975aedb80 1 --2- 172.21.15.52:0/321532192 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c628b72f60 0x55c628b61510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:57:28.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.218+0000 7f0975aedb80 1 -- 172.21.15.52:0/321532192 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f09641b0630 con 0x55c628b6cf20 2024-06-10T02:57:28.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.218+0000 7f0975aedb80 1 -- 172.21.15.52:0/321532192 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f09641b2260 con 0x55c628b72f60 2024-06-10T02:57:28.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.218+0000 7f0975aedb80 1 -- 172.21.15.52:0/321532192 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f09641b0d00 con 0x55c628b72640 2024-06-10T02:57:28.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.218+0000 7f0969807700 1 --2- 172.21.15.52:0/321532192 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c628b72f60 0x55c628b61510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:57:28.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.219+0000 7f0969807700 1 -- 172.21.15.52:0/321532192 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c628b72640 msgr2=0x55c628b70730 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-10T02:57:28.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.219+0000 7f0969807700 1 --2- 172.21.15.52:0/321532192 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c628b72640 0x55c628b70730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:28.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.219+0000 7f0969807700 1 -- 172.21.15.52:0/321532192 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c628b6cf20 msgr2=0x55c628b6c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:57:28.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.219+0000 7f0969807700 1 --2- 172.21.15.52:0/321532192 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c628b6cf20 0x55c628b6c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:28.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.219+0000 7f0969807700 1 -- 172.21.15.52:0/321532192 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c628b79010 con 0x55c628b72f60 2024-06-10T02:57:28.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.219+0000 7f0969807700 1 --2- 172.21.15.52:0/321532192 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c628b72f60 0x55c628b61510 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f095c00ef00 tx=0x7f095c097760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2745bb2d08f0cd97 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:57:28.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.219+0000 7f094effd700 1 -- 172.21.15.52:0/321532192 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f095c09ba90 con 0x55c628b72f60 2024-06-10T02:57:28.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.219+0000 7f094effd700 1 -- 172.21.15.52:0/321532192 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f095c09b390 con 0x55c628b72f60 2024-06-10T02:57:28.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.219+0000 7f094effd700 1 -- 172.21.15.52:0/321532192 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f095c09a2d0 con 0x55c628b72f60 2024-06-10T02:57:28.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.220+0000 7f0975aedb80 1 -- 172.21.15.52:0/321532192 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c628b72f60 msgr2=0x55c628b61510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:57:28.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.220+0000 7f0975aedb80 1 --2- 172.21.15.52:0/321532192 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c628b72f60 0x55c628b61510 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f095c00ef00 tx=0x7f095c097760 comp rx=0 tx=0).stop 2024-06-10T02:57:28.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.220+0000 7f0975aedb80 1 -- 172.21.15.52:0/321532192 shutdown_connections 2024-06-10T02:57:28.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.220+0000 7f0975aedb80 1 --2- 172.21.15.52:0/321532192 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c628b72f60 0x55c628b61510 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:28.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.220+0000 7f0975aedb80 1 --2- 172.21.15.52:0/321532192 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c628b6cf20 0x55c628b6c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:28.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.220+0000 7f0975aedb80 1 --2- 172.21.15.52:0/321532192 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c628b72640 0x55c628b70730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:28.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.220+0000 7f0975aedb80 1 -- 172.21.15.52:0/321532192 >> 172.21.15.52:0/321532192 conn(0x55c628a97cc0 msgr2=0x55c628b60e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:57:28.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.221+0000 7f0975aedb80 1 -- 172.21.15.52:0/321532192 shutdown_connections 2024-06-10T02:57:28.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.221+0000 7f0975aedb80 1 -- 172.21.15.52:0/321532192 wait complete. 2024-06-10T02:57:28.222 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.221+0000 7f0975aedb80 1 -- start start 2024-06-10T02:57:28.222 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.221+0000 7f0975aedb80 1 Processor -- start 2024-06-10T02:57:28.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.222+0000 7f0975aedb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c628b6cf20 0x55c628b885f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:57:28.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.222+0000 7f0969006700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c628b6cf20 0x55c628b885f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:57:28.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.223+0000 7f0969006700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c628b6cf20 0x55c628b885f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.52:49710/0 (socket says 172.21.15.52:49710) 2024-06-10T02:57:28.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.222+0000 7f0975aedb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c628b72640 0x55c628b88b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:57:28.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.223+0000 7f0969006700 1 -- 172.21.15.52:0/3718985097 learned_addr learned my addr 172.21.15.52:0/3718985097 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:57:28.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.223+0000 7f0975aedb80 1 --2- 172.21.15.52:0/3718985097 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c628b72f60 0x55c628b89070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:57:28.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.223+0000 7f0975aedb80 1 -- 172.21.15.52:0/3718985097 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f09641b1b60 con 0x55c628b72640 2024-06-10T02:57:28.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.223+0000 7f0975aedb80 1 -- 172.21.15.52:0/3718985097 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f09641b1490 con 0x55c628b6cf20 2024-06-10T02:57:28.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.223+0000 7f0969807700 1 --2- 172.21.15.52:0/3718985097 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c628b72f60 0x55c628b89070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:57:28.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.223+0000 7f0975aedb80 1 -- 172.21.15.52:0/3718985097 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f09641ae5d0 con 0x55c628b72f60 2024-06-10T02:57:28.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.223+0000 7f0975aedb80 10 client.0 _ll_register_callbacks cb 0x55c628a58920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T02:57:28.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.223+0000 7f0968805700 1 --2- 172.21.15.52:0/3718985097 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c628b72640 0x55c628b88b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:57:28.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.223+0000 7f0969006700 1 -- 172.21.15.52:0/3718985097 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c628b72f60 msgr2=0x55c628b89070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:57:28.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.223+0000 7f0969006700 1 --2- 172.21.15.52:0/3718985097 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c628b72f60 0x55c628b89070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:28.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.223+0000 7f0969006700 1 -- 172.21.15.52:0/3718985097 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c628b72640 msgr2=0x55c628b88b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:57:28.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.223+0000 7f0969006700 1 --2- 172.21.15.52:0/3718985097 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c628b72640 0x55c628b88b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:28.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.224+0000 7f0969006700 1 -- 172.21.15.52:0/3718985097 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c628bf7950 con 0x55c628b6cf20 2024-06-10T02:57:28.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.224+0000 7f0969006700 1 --2- 172.21.15.52:0/3718985097 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c628b6cf20 0x55c628b885f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f09600044a0 tx=0x7f09600047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3db74a3821f72070 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:57:28.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.224+0000 7f094cff9700 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-06-10T02:57:28.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.224+0000 7f094cff9700 1 -- 172.21.15.52:0/3718985097 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f096002e060 con 0x55c628b6cf20 2024-06-10T02:57:28.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.224+0000 7f094cff9700 1 -- 172.21.15.52:0/3718985097 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f0960011440 con 0x55c628b6cf20 2024-06-10T02:57:28.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.224+0000 7f0975aedb80 1 -- 172.21.15.52:0/3718985097 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f095c09b390 con 0x55c628b6cf20 2024-06-10T02:57:28.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.225+0000 7f094cff9700 1 -- 172.21.15.52:0/3718985097 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f0960035050 con 0x55c628b6cf20 2024-06-10T02:57:28.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.225+0000 7f0975aedb80 10 client.25604 fetch_fsmap learned FSMap version 33 2024-06-10T02:57:28.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.225+0000 7f0975aedb80 1 -- 172.21.15.52:0/3718985097 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55c628bf8c70 con 0x55c628b6cf20 2024-06-10T02:57:28.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.225+0000 7f094cff9700 1 -- 172.21.15.52:0/3718985097 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f095c09b390 con 0x55c628b6cf20 2024-06-10T02:57:28.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.225+0000 7f094cff9700 1 -- 172.21.15.52:0/3718985097 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 33) ==== 8382+0+0 (secure 0 0 0) 0x7f09600206a0 con 0x55c628b6cf20 2024-06-10T02:57:28.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.225+0000 7f0975aedb80 10 client.25604 fetch_fsmap finished waiting for FSMap version 33 2024-06-10T02:57:28.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.225+0000 7f0975aedb80 10 client.25604 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T02:57:28.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.225+0000 7f0975aedb80 20 client.25604 populate_metadata read hostname 'smithi052' 2024-06-10T02:57:28.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.225+0000 7f0975aedb80 1 --2- 172.21.15.52:0/3718985097 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55c628bf9300 0x55c628bfb720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:57:28.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.225+0000 7f0975aedb80 4 client.25604 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T02:57:28.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.225+0000 7f0975aedb80 1 -- 172.21.15.52:0/3718985097 --> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c628ac6290 con 0x55c628bf9300 2024-06-10T02:57:28.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.226+0000 7f0968805700 1 --2- 172.21.15.52:0/3718985097 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55c628bf9300 0x55c628bfb720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:57:28.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.227+0000 7f094cff9700 10 client.25604 ms_handle_connect on v2:172.21.15.188:6838/310444836 2024-06-10T02:57:28.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.227+0000 7f0968805700 1 --2- 172.21.15.52:0/3718985097 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55c628bf9300 0x55c628bfb720 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b7a02a2609e06a87 server_cookie=894f2c18b475a0f8 in_seq=0 out_seq=0 2024-06-10T02:57:28.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.227+0000 7f094cff9700 1 -- 172.21.15.52:0/3718985097 <== mds.0 v2:172.21.15.188:6838/310444836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c628ac6290 con 0x55c628bf9300 2024-06-10T02:57:28.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.227+0000 7f094cff9700 10 client.25604 handle_command_reply: tid=0 2024-06-10T02:57:28.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.227+0000 7f0975aedb80 1 client.25604 shutdown 2024-06-10T02:57:28.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.228+0000 7f0975aedb80 1 -- 172.21.15.52:0/3718985097 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c628b6cf20 msgr2=0x55c628b885f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:57:28.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.228+0000 7f0975aedb80 1 --2- 172.21.15.52:0/3718985097 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c628b6cf20 0x55c628b885f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f09600044a0 tx=0x7f09600047c0 comp rx=0 tx=0).stop 2024-06-10T02:57:28.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.228+0000 7f0975aedb80 1 -- 172.21.15.52:0/3718985097 shutdown_connections 2024-06-10T02:57:28.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.228+0000 7f0975aedb80 1 --2- 172.21.15.52:0/3718985097 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55c628bf9300 0x55c628bfb720 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:28.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.228+0000 7f094cff9700 0 client.25604 ms_handle_reset on v2:172.21.15.188:6838/310444836 2024-06-10T02:57:28.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.228+0000 7f0975aedb80 1 --2- 172.21.15.52:0/3718985097 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c628b72f60 0x55c628b89070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:28.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.228+0000 7f0975aedb80 1 --2- 172.21.15.52:0/3718985097 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c628b6cf20 0x55c628b885f0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:28.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.228+0000 7f0975aedb80 1 --2- 172.21.15.52:0/3718985097 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c628b72640 0x55c628b88b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:28.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.228+0000 7f0975aedb80 1 -- 172.21.15.52:0/3718985097 >> 172.21.15.52:0/3718985097 conn(0x55c628a97cc0 msgr2=0x55c628b777d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:57:28.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.228+0000 7f0975aedb80 1 -- 172.21.15.52:0/3718985097 shutdown_connections 2024-06-10T02:57:28.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.228+0000 7f0975aedb80 1 -- 172.21.15.52:0/3718985097 wait complete. 2024-06-10T02:57:28.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.228+0000 7f0975aedb80 20 client.25604 trim_cache size 0 max 16384 2024-06-10T02:57:28.230 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-06-10T02:57:28.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.229+0000 7f096b26a700 1 -- 172.21.15.52:0/2722235508 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f09500a2ca0 msgr2=0x7f09500a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:57:28.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.229+0000 7f096b26a700 1 --2- 172.21.15.52:0/2722235508 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f09500a2ca0 0x7f09500a5160 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f0960007fd0 tx=0x7f09600073d0 comp rx=0 tx=0).stop 2024-06-10T02:57:28.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.229+0000 7f096b26a700 1 -- 172.21.15.52:0/2722235508 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0964106720 msgr2=0x7f0964103b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:57:28.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.229+0000 7f096b26a700 1 --2- 172.21.15.52:0/2722235508 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0964106720 0x7f0964103b10 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f095c00ef30 tx=0x7f095c00c3a0 comp rx=0 tx=0).stop 2024-06-10T02:57:28.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.229+0000 7f096b26a700 1 -- 172.21.15.52:0/2722235508 shutdown_connections 2024-06-10T02:57:28.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.229+0000 7f096b26a700 1 --2- 172.21.15.52:0/2722235508 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f09500a2ca0 0x7f09500a5160 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:28.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.229+0000 7f096b26a700 1 --2- 172.21.15.52:0/2722235508 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0964106720 0x7f0964103b10 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:28.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.229+0000 7f096b26a700 1 --2- 172.21.15.52:0/2722235508 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f09641019d0 0x7f0964103090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:28.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.229+0000 7f096b26a700 1 --2- 172.21.15.52:0/2722235508 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0964102380 0x7f09641035d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:28.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.229+0000 7f096b26a700 1 -- 172.21.15.52:0/2722235508 >> 172.21.15.52:0/2722235508 conn(0x7f0964078340 msgr2=0x7f0964100f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:57:28.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.229+0000 7f096b26a700 1 -- 172.21.15.52:0/2722235508 shutdown_connections 2024-06-10T02:57:28.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:28.229+0000 7f096b26a700 1 -- 172.21.15.52:0/2722235508 wait complete. 2024-06-10T02:57:28.889 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:57:28 smithi120 ceph-mon[36634]: pgmap v532: 129 pgs: 129 active+clean; 190 MiB data, 551 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.0 MiB/s wr, 332 op/s 2024-06-10T02:57:28.903 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:57:28 smithi052 ceph-mon[32362]: pgmap v532: 129 pgs: 129 active+clean; 190 MiB data, 551 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.0 MiB/s wr, 332 op/s 2024-06-10T02:57:28.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:57:28 smithi188 ceph-mon[36599]: pgmap v532: 129 pgs: 129 active+clean; 190 MiB data, 551 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.0 MiB/s wr, 332 op/s 2024-06-10T02:57:29.931 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-06-10T02:57:29.932 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-06-10T02:57:30.251 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:57:30.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:57:30 smithi188 ceph-mon[36599]: pgmap v533: 129 pgs: 129 active+clean; 220 MiB data, 585 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 7.2 MiB/s wr, 523 op/s 2024-06-10T02:57:30.976 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:57:30 smithi052 ceph-mon[32362]: pgmap v533: 129 pgs: 129 active+clean; 220 MiB data, 585 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 7.2 MiB/s wr, 523 op/s 2024-06-10T02:57:31.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:57:30 smithi120 ceph-mon[36634]: pgmap v533: 129 pgs: 129 active+clean; 220 MiB data, 585 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 7.2 MiB/s wr, 523 op/s 2024-06-10T02:57:32.807 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:57:32 smithi052 ceph-mon[32362]: pgmap v534: 129 pgs: 129 active+clean; 224 MiB data, 584 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 7.5 MiB/s wr, 577 op/s 2024-06-10T02:57:32.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:57:32 smithi188 ceph-mon[36599]: pgmap v534: 129 pgs: 129 active+clean; 224 MiB data, 584 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 7.5 MiB/s wr, 577 op/s 2024-06-10T02:57:32.995 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:32.996+0000 7f71a663e700 1 -- 172.21.15.52:0/2092309870 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f71a0101d50 msgr2=0x7f71a010e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:57:32.996 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:32.996+0000 7f71a663e700 1 --2- 172.21.15.52:0/2092309870 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f71a0101d50 0x7f71a010e240 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f719800b0a0 tx=0x7f719800b3b0 comp rx=0 tx=0).stop 2024-06-10T02:57:32.996 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:32.996+0000 7f71a663e700 1 -- 172.21.15.52:0/2092309870 shutdown_connections 2024-06-10T02:57:32.996 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:32.996+0000 7f71a663e700 1 --2- 172.21.15.52:0/2092309870 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f71a0101d50 0x7f71a010e240 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:32.996 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:32.996+0000 7f71a663e700 1 --2- 172.21.15.52:0/2092309870 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f71a0107400 0x7f71a01077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:32.996 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:32.996+0000 7f71a663e700 1 --2- 172.21.15.52:0/2092309870 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f71a0101390 0x7f71a0101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:32.996 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:32.996+0000 7f71a663e700 1 -- 172.21.15.52:0/2092309870 >> 172.21.15.52:0/2092309870 conn(0x7f71a0078340 msgr2=0x7f71a0078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:57:32.996 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:32.996+0000 7f71a663e700 1 -- 172.21.15.52:0/2092309870 shutdown_connections 2024-06-10T02:57:32.996 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:32.996+0000 7f71a663e700 1 -- 172.21.15.52:0/2092309870 wait complete. 2024-06-10T02:57:32.998 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:32.996+0000 7f71a663e700 1 Processor -- start 2024-06-10T02:57:32.998 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:32.997+0000 7f71a663e700 1 -- start start 2024-06-10T02:57:32.998 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:32.997+0000 7f71a663e700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f71a0101390 0x7f71a019d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:57:32.998 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:32.997+0000 7f71a663e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f71a0101d50 0x7f71a019dc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:57:32.998 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:32.997+0000 7f71a663e700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f71a0107400 0x7f71a01978d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:57:32.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:32.997+0000 7f71a663e700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f71a0111690 con 0x7f71a0101d50 2024-06-10T02:57:32.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:32.997+0000 7f71a663e700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f71a0111510 con 0x7f71a0107400 2024-06-10T02:57:32.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:32.997+0000 7f71a663e700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f71a0111810 con 0x7f71a0101390 2024-06-10T02:57:32.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:32.997+0000 7f719f7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f71a0101d50 0x7f71a019dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:57:32.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:32.997+0000 7f719f7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f71a0101d50 0x7f71a019dc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53472/0 (socket says 172.21.15.52:53472) 2024-06-10T02:57:32.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:32.997+0000 7f719f7fe700 1 -- 172.21.15.52:0/400252555 learned_addr learned my addr 172.21.15.52:0/400252555 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:57:32.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:32.997+0000 7f719ffff700 1 --2- 172.21.15.52:0/400252555 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f71a0101390 0x7f71a019d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:57:32.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:32.997+0000 7f71a4bdb700 1 --2- 172.21.15.52:0/400252555 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f71a0107400 0x7f71a01978d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:57:32.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:32.997+0000 7f719f7fe700 1 -- 172.21.15.52:0/400252555 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f71a0101390 msgr2=0x7f71a019d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:57:32.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:32.997+0000 7f719f7fe700 1 --2- 172.21.15.52:0/400252555 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f71a0101390 0x7f71a019d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:32.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:32.997+0000 7f719f7fe700 1 -- 172.21.15.52:0/400252555 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f71a0107400 msgr2=0x7f71a01978d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:57:32.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:32.997+0000 7f719f7fe700 1 --2- 172.21.15.52:0/400252555 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f71a0107400 0x7f71a01978d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:32.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:32.997+0000 7f719f7fe700 1 -- 172.21.15.52:0/400252555 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71a0198030 con 0x7f71a0101d50 2024-06-10T02:57:33.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:32.998+0000 7f719f7fe700 1 --2- 172.21.15.52:0/400252555 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f71a0101d50 0x7f71a019dc50 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f719400d950 tx=0x7f719400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:57:33.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:32.998+0000 7f719d7fa700 1 -- 172.21.15.52:0/400252555 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7194016070 con 0x7f71a0101d50 2024-06-10T02:57:33.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:32.998+0000 7f719d7fa700 1 -- 172.21.15.52:0/400252555 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7194009e20 con 0x7f71a0101d50 2024-06-10T02:57:33.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:32.998+0000 7f719d7fa700 1 -- 172.21.15.52:0/400252555 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f719400f620 con 0x7f71a0101d50 2024-06-10T02:57:33.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:32.998+0000 7f71a663e700 1 -- 172.21.15.52:0/400252555 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71a0198320 con 0x7f71a0101d50 2024-06-10T02:57:33.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:32.998+0000 7f71a663e700 1 -- 172.21.15.52:0/400252555 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71a01a45e0 con 0x7f71a0101d50 2024-06-10T02:57:33.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.000+0000 7f719d7fa700 1 -- 172.21.15.52:0/400252555 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f7194019070 con 0x7f71a0101d50 2024-06-10T02:57:33.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.001+0000 7f719d7fa700 1 --2- 172.21.15.52:0/400252555 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f71880a2ca0 0x7f71880a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:57:33.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.001+0000 7f719d7fa700 1 -- 172.21.15.52:0/400252555 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f71940cef40 con 0x7f71a0101d50 2024-06-10T02:57:33.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.001+0000 7f719ffff700 1 --2- 172.21.15.52:0/400252555 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f71880a2ca0 0x7f71880a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:57:33.002 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.002+0000 7f71b0ec1b80 1 Processor -- start 2024-06-10T02:57:33.002 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.002+0000 7f719ffff700 1 --2- 172.21.15.52:0/400252555 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f71880a2ca0 0x7f71880a5160 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f71a0198a10 tx=0x7f7190009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:57:33.002 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.002+0000 7f71b0ec1b80 1 -- start start 2024-06-10T02:57:33.002 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.002+0000 7f71b0ec1b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f81a5f3080 0x55f81a5f12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:57:33.002 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.002+0000 7f71b0ec1b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f81a5f1820 0x55f81a5f1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:57:33.002 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.002+0000 7f71b0ec1b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f81a5ff960 0x55f81a601d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:57:33.002 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.002+0000 7f71b0ec1b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55f81a4e3fe0 con 0x55f81a5f1820 2024-06-10T02:57:33.002 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.002+0000 7f71b0ec1b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55f81a5428d0 con 0x55f81a5ff960 2024-06-10T02:57:33.002 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.002+0000 7f71b0ec1b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55f81a4aa4e0 con 0x55f81a5f3080 2024-06-10T02:57:33.003 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.002+0000 7f719f7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f81a5f1820 0x55f81a5f1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:57:33.003 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.002+0000 7f719ffff700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f81a5f3080 0x55f81a5f12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:57:33.003 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.002+0000 7f719f7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f81a5f1820 0x55f81a5f1ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53480/0 (socket says 172.21.15.52:53480) 2024-06-10T02:57:33.003 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.002+0000 7f719f7fe700 1 -- 172.21.15.52:0/1963741285 learned_addr learned my addr 172.21.15.52:0/1963741285 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:57:33.003 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.002+0000 7f71a4bdb700 1 --2- 172.21.15.52:0/1963741285 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f81a5ff960 0x55f81a601d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:57:33.003 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.002+0000 7f719f7fe700 1 -- 172.21.15.52:0/1963741285 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f81a5f3080 msgr2=0x55f81a5f12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:57:33.003 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.002+0000 7f719f7fe700 1 --2- 172.21.15.52:0/1963741285 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f81a5f3080 0x55f81a5f12e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:33.003 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.002+0000 7f719f7fe700 1 -- 172.21.15.52:0/1963741285 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f81a5ff960 msgr2=0x55f81a601d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:57:33.003 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.002+0000 7f719f7fe700 1 --2- 172.21.15.52:0/1963741285 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f81a5ff960 0x55f81a601d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:33.003 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.002+0000 7f719f7fe700 1 -- 172.21.15.52:0/1963741285 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f81a491950 con 0x55f81a5f1820 2024-06-10T02:57:33.003 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.003+0000 7f719f7fe700 1 --2- 172.21.15.52:0/1963741285 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f81a5f1820 0x55f81a5f1ca0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f719400b2f0 tx=0x7f7194005910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76a1a67c82ed976b server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:57:33.006 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.003+0000 7f71867fc700 1 -- 172.21.15.52:0/1963741285 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7194020070 con 0x55f81a5f1820 2024-06-10T02:57:33.006 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.003+0000 7f71867fc700 1 -- 172.21.15.52:0/1963741285 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71940a4060 con 0x55f81a5f1820 2024-06-10T02:57:33.006 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.003+0000 7f71867fc700 1 -- 172.21.15.52:0/1963741285 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71940a3e30 con 0x55f81a5f1820 2024-06-10T02:57:33.006 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.003+0000 7f71b0ec1b80 1 -- 172.21.15.52:0/1963741285 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f81a5f1820 msgr2=0x55f81a5f1ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:57:33.006 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.003+0000 7f71b0ec1b80 1 --2- 172.21.15.52:0/1963741285 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f81a5f1820 0x55f81a5f1ca0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f719400b2f0 tx=0x7f7194005910 comp rx=0 tx=0).stop 2024-06-10T02:57:33.007 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.004+0000 7f71b0ec1b80 1 -- 172.21.15.52:0/1963741285 shutdown_connections 2024-06-10T02:57:33.007 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.004+0000 7f71b0ec1b80 1 --2- 172.21.15.52:0/1963741285 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f81a5ff960 0x55f81a601d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:33.007 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.004+0000 7f71b0ec1b80 1 --2- 172.21.15.52:0/1963741285 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f81a5f3080 0x55f81a5f12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:33.007 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.004+0000 7f71b0ec1b80 1 --2- 172.21.15.52:0/1963741285 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f81a5f1820 0x55f81a5f1ca0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:33.007 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.004+0000 7f71b0ec1b80 1 -- 172.21.15.52:0/1963741285 >> 172.21.15.52:0/1963741285 conn(0x55f81a51e100 msgr2=0x55f81a5ec2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:57:33.007 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.004+0000 7f71b0ec1b80 1 -- 172.21.15.52:0/1963741285 shutdown_connections 2024-06-10T02:57:33.007 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.004+0000 7f71b0ec1b80 1 -- 172.21.15.52:0/1963741285 wait complete. 2024-06-10T02:57:33.007 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.004+0000 7f71b0ec1b80 1 -- start start 2024-06-10T02:57:33.007 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.004+0000 7f71b0ec1b80 1 Processor -- start 2024-06-10T02:57:33.007 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.004+0000 7f71b0ec1b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f81a5f1820 0x55f81a5ff150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:57:33.007 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.004+0000 7f71b0ec1b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f81a5f3080 0x55f81a5fc450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:57:33.007 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.005+0000 7f71b0ec1b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f81a5ff960 0x55f81a5fc990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:57:33.007 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.005+0000 7f71b0ec1b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55f81a4e3040 con 0x55f81a5f3080 2024-06-10T02:57:33.008 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.005+0000 7f71b0ec1b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55f81a53e890 con 0x55f81a5ff960 2024-06-10T02:57:33.008 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.005+0000 7f71b0ec1b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55f81a6026a0 con 0x55f81a5f1820 2024-06-10T02:57:33.008 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.005+0000 7f71b0ec1b80 10 client.0 _ll_register_callbacks cb 0x55f81a4d8630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T02:57:33.008 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.005+0000 7f719f7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f81a5f3080 0x55f81a5fc450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:57:33.008 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.005+0000 7f719f7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f81a5f3080 0x55f81a5fc450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53490/0 (socket says 172.21.15.52:53490) 2024-06-10T02:57:33.008 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.005+0000 7f719ffff700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f81a5f1820 0x55f81a5ff150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:57:33.008 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.005+0000 7f719f7fe700 1 -- 172.21.15.52:0/104056386 learned_addr learned my addr 172.21.15.52:0/104056386 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:57:33.008 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.005+0000 7f71a4bdb700 1 --2- 172.21.15.52:0/104056386 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f81a5ff960 0x55f81a5fc990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:57:33.008 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.005+0000 7f719f7fe700 1 -- 172.21.15.52:0/104056386 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f81a5f1820 msgr2=0x55f81a5ff150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:57:33.008 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.005+0000 7f719f7fe700 1 --2- 172.21.15.52:0/104056386 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f81a5f1820 0x55f81a5ff150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:33.008 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.005+0000 7f719f7fe700 1 -- 172.21.15.52:0/104056386 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f81a5ff960 msgr2=0x55f81a5fc990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:57:33.008 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.005+0000 7f719f7fe700 1 --2- 172.21.15.52:0/104056386 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f81a5ff960 0x55f81a5fc990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:33.008 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.005+0000 7f719f7fe700 1 -- 172.21.15.52:0/104056386 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f81a5087d0 con 0x55f81a5f3080 2024-06-10T02:57:33.008 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.005+0000 7f719ffff700 1 --2- 172.21.15.52:0/104056386 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f81a5f1820 0x55f81a5ff150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T02:57:33.009 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.005+0000 7f71a4bdb700 1 --2- 172.21.15.52:0/104056386 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f81a5ff960 0x55f81a5fc990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T02:57:33.009 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.005+0000 7f719f7fe700 1 --2- 172.21.15.52:0/104056386 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f81a5f3080 0x55f81a5fc450 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f7194097850 tx=0x7f719409b650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1d1927a448c1c90 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:57:33.009 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.005+0000 7f717f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-10T02:57:33.009 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.005+0000 7f717f7fe700 1 -- 172.21.15.52:0/104056386 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7194020070 con 0x55f81a5f3080 2024-06-10T02:57:33.009 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.005+0000 7f717f7fe700 1 -- 172.21.15.52:0/104056386 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f71940b4210 con 0x55f81a5f3080 2024-06-10T02:57:33.009 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.006+0000 7f71b0ec1b80 1 -- 172.21.15.52:0/104056386 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f71940a4060 con 0x55f81a5f3080 2024-06-10T02:57:33.009 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.006+0000 7f717f7fe700 1 -- 172.21.15.52:0/104056386 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f71940a3bd0 con 0x55f81a5f3080 2024-06-10T02:57:33.009 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.006+0000 7f71b0ec1b80 10 client.16032 fetch_fsmap learned FSMap version 33 2024-06-10T02:57:33.009 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.006+0000 7f71b0ec1b80 1 -- 172.21.15.52:0/104056386 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55f81a60ce40 con 0x55f81a5f3080 2024-06-10T02:57:33.009 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.006+0000 7f717f7fe700 1 -- 172.21.15.52:0/104056386 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71940a4060 con 0x55f81a5f3080 2024-06-10T02:57:33.009 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.006+0000 7f717f7fe700 1 -- 172.21.15.52:0/104056386 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 33) ==== 8382+0+0 (secure 0 0 0) 0x7f719407c020 con 0x55f81a5f3080 2024-06-10T02:57:33.009 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.006+0000 7f71b0ec1b80 10 client.16032 fetch_fsmap finished waiting for FSMap version 33 2024-06-10T02:57:33.009 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.006+0000 7f71b0ec1b80 10 client.16032 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T02:57:33.009 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.006+0000 7f71b0ec1b80 20 client.16032 populate_metadata read hostname 'smithi052' 2024-06-10T02:57:33.009 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.006+0000 7f71b0ec1b80 1 --2- 172.21.15.52:0/104056386 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55f81a60f310 0x55f81a611700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:57:33.010 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.006+0000 7f71b0ec1b80 4 client.16032 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T02:57:33.010 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.006+0000 7f71b0ec1b80 1 -- 172.21.15.52:0/104056386 --> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f81a54fdd0 con 0x55f81a60f310 2024-06-10T02:57:33.010 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.006+0000 7f719ffff700 1 --2- 172.21.15.52:0/104056386 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55f81a60f310 0x55f81a611700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:57:33.010 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.007+0000 7f719ffff700 1 --2- 172.21.15.52:0/104056386 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55f81a60f310 0x55f81a611700 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=57790aee399694c4 server_cookie=85839d69719ad17e in_seq=0 out_seq=0 2024-06-10T02:57:33.010 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.007+0000 7f717f7fe700 10 client.16032 ms_handle_connect on v2:172.21.15.188:6838/310444836 2024-06-10T02:57:33.011 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.008+0000 7f717f7fe700 1 -- 172.21.15.52:0/104056386 <== mds.0 v2:172.21.15.188:6838/310444836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55f81a54fdd0 con 0x55f81a60f310 2024-06-10T02:57:33.011 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.008+0000 7f717f7fe700 10 client.16032 handle_command_reply: tid=0 2024-06-10T02:57:33.011 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.008+0000 7f71b0ec1b80 1 client.16032 shutdown 2024-06-10T02:57:33.011 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.008+0000 7f71b0ec1b80 1 -- 172.21.15.52:0/104056386 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f81a5f3080 msgr2=0x55f81a5fc450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:57:33.011 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.008+0000 7f71b0ec1b80 1 --2- 172.21.15.52:0/104056386 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f81a5f3080 0x55f81a5fc450 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f7194097850 tx=0x7f719409b650 comp rx=0 tx=0).stop 2024-06-10T02:57:33.011 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.008+0000 7f71b0ec1b80 1 -- 172.21.15.52:0/104056386 shutdown_connections 2024-06-10T02:57:33.011 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.008+0000 7f71b0ec1b80 1 --2- 172.21.15.52:0/104056386 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55f81a60f310 0x55f81a611700 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:33.012 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.008+0000 7f717f7fe700 0 client.16032 ms_handle_reset on v2:172.21.15.188:6838/310444836 2024-06-10T02:57:33.012 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.008+0000 7f71b0ec1b80 1 --2- 172.21.15.52:0/104056386 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f81a5ff960 0x55f81a5fc990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:33.012 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.008+0000 7f71b0ec1b80 1 --2- 172.21.15.52:0/104056386 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f81a5f1820 0x55f81a5ff150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:33.012 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.008+0000 7f71b0ec1b80 1 --2- 172.21.15.52:0/104056386 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f81a5f3080 0x55f81a5fc450 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:33.012 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.008+0000 7f71b0ec1b80 1 -- 172.21.15.52:0/104056386 >> 172.21.15.52:0/104056386 conn(0x55f81a51e100 msgr2=0x55f81a5eb780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:57:33.012 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.008+0000 7f71b0ec1b80 1 -- 172.21.15.52:0/104056386 shutdown_connections 2024-06-10T02:57:33.012 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.008+0000 7f71b0ec1b80 1 -- 172.21.15.52:0/104056386 wait complete. 2024-06-10T02:57:33.012 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.009+0000 7f71b0ec1b80 20 client.16032 trim_cache size 0 max 16384 2024-06-10T02:57:33.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.030+0000 7f71b0ec1b80 1 Processor -- start 2024-06-10T02:57:33.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.030+0000 7f71b0ec1b80 1 -- start start 2024-06-10T02:57:33.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.030+0000 7f71b0ec1b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f81a5f1820 0x55f81a5fca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:57:33.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.030+0000 7f71b0ec1b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f81a5f3080 0x55f81a5fb5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:57:33.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.030+0000 7f71b0ec1b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f81a5ff960 0x55f81a5fe390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:57:33.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.030+0000 7f71b0ec1b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f71a01ac280 con 0x55f81a5f3080 2024-06-10T02:57:33.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.030+0000 7f71b0ec1b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f71a01adeb0 con 0x55f81a5f1820 2024-06-10T02:57:33.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.030+0000 7f71b0ec1b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f71a01ac950 con 0x55f81a5ff960 2024-06-10T02:57:33.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.030+0000 7f719f7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f81a5f3080 0x55f81a5fb5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:57:33.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.030+0000 7f719f7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f81a5f3080 0x55f81a5fb5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53498/0 (socket says 172.21.15.52:53498) 2024-06-10T02:57:33.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.030+0000 7f719f7fe700 1 -- 172.21.15.52:0/3251134614 learned_addr learned my addr 172.21.15.52:0/3251134614 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:57:33.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.030+0000 7f71a4bdb700 1 --2- 172.21.15.52:0/3251134614 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f81a5ff960 0x55f81a5fe390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:57:33.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.030+0000 7f719ffff700 1 --2- 172.21.15.52:0/3251134614 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f81a5f1820 0x55f81a5fca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:57:33.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.031+0000 7f719f7fe700 1 -- 172.21.15.52:0/3251134614 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f81a5ff960 msgr2=0x55f81a5fe390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:57:33.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.031+0000 7f719f7fe700 1 --2- 172.21.15.52:0/3251134614 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f81a5ff960 0x55f81a5fe390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:33.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.031+0000 7f719f7fe700 1 -- 172.21.15.52:0/3251134614 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f81a5f1820 msgr2=0x55f81a5fca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:57:33.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.031+0000 7f719f7fe700 1 --2- 172.21.15.52:0/3251134614 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f81a5f1820 0x55f81a5fca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:33.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.031+0000 7f719f7fe700 1 -- 172.21.15.52:0/3251134614 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f81a5e8890 con 0x55f81a5f3080 2024-06-10T02:57:33.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.031+0000 7f719f7fe700 1 --2- 172.21.15.52:0/3251134614 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f81a5f3080 0x55f81a5fb5d0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f719409b380 tx=0x7f7194005910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfe4d257cfcb4b96 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:57:33.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.031+0000 7f7185ffb700 1 -- 172.21.15.52:0/3251134614 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7194020070 con 0x55f81a5f3080 2024-06-10T02:57:33.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.031+0000 7f7185ffb700 1 -- 172.21.15.52:0/3251134614 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71940a4060 con 0x55f81a5f3080 2024-06-10T02:57:33.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.031+0000 7f7185ffb700 1 -- 172.21.15.52:0/3251134614 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71940972d0 con 0x55f81a5f3080 2024-06-10T02:57:33.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.032+0000 7f71b0ec1b80 1 -- 172.21.15.52:0/3251134614 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f81a5f3080 msgr2=0x55f81a5fb5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:57:33.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.032+0000 7f71b0ec1b80 1 --2- 172.21.15.52:0/3251134614 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f81a5f3080 0x55f81a5fb5d0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f719409b380 tx=0x7f7194005910 comp rx=0 tx=0).stop 2024-06-10T02:57:33.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.032+0000 7f71b0ec1b80 1 -- 172.21.15.52:0/3251134614 shutdown_connections 2024-06-10T02:57:33.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.032+0000 7f71b0ec1b80 1 --2- 172.21.15.52:0/3251134614 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f81a5ff960 0x55f81a5fe390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:33.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.032+0000 7f71b0ec1b80 1 --2- 172.21.15.52:0/3251134614 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f81a5f1820 0x55f81a5fca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:33.035 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.032+0000 7f71b0ec1b80 1 --2- 172.21.15.52:0/3251134614 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f81a5f3080 0x55f81a5fb5d0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:33.035 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.032+0000 7f71b0ec1b80 1 -- 172.21.15.52:0/3251134614 >> 172.21.15.52:0/3251134614 conn(0x55f81a51e100 msgr2=0x55f81a5fdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:57:33.035 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.032+0000 7f71b0ec1b80 1 -- 172.21.15.52:0/3251134614 shutdown_connections 2024-06-10T02:57:33.035 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.032+0000 7f71b0ec1b80 1 -- 172.21.15.52:0/3251134614 wait complete. 2024-06-10T02:57:33.035 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.032+0000 7f71b0ec1b80 1 -- start start 2024-06-10T02:57:33.035 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.032+0000 7f71b0ec1b80 1 Processor -- start 2024-06-10T02:57:33.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.033+0000 7f71b0ec1b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f81a5f1820 0x55f81a608250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:57:33.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.033+0000 7f71b0ec1b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f81a5f3080 0x55f81a6045e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:57:33.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.033+0000 7f71b0ec1b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f81a5ff960 0x55f81a604d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:57:33.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.033+0000 7f71b0ec1b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f71a01ad7b0 con 0x55f81a5f1820 2024-06-10T02:57:33.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.033+0000 7f71b0ec1b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f71a01ad0e0 con 0x55f81a5f3080 2024-06-10T02:57:33.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.033+0000 7f71b0ec1b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f71a01aa220 con 0x55f81a5ff960 2024-06-10T02:57:33.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.033+0000 7f719ffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f81a5f1820 0x55f81a608250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:57:33.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.033+0000 7f719f7fe700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f81a5f3080 0x55f81a6045e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:57:33.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.033+0000 7f71a4bdb700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f81a5ff960 0x55f81a604d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:57:33.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.033+0000 7f719ffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f81a5f1820 0x55f81a608250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53500/0 (socket says 172.21.15.52:53500) 2024-06-10T02:57:33.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.033+0000 7f719ffff700 1 -- 172.21.15.52:0/3052024881 learned_addr learned my addr 172.21.15.52:0/3052024881 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:57:33.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.033+0000 7f71b0ec1b80 10 client.0 _ll_register_callbacks cb 0x55f81a4d8630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T02:57:33.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.033+0000 7f719ffff700 1 -- 172.21.15.52:0/3052024881 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f81a5ff960 msgr2=0x55f81a604d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:57:33.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.033+0000 7f719ffff700 1 --2- 172.21.15.52:0/3052024881 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f81a5ff960 0x55f81a604d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:33.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.033+0000 7f719ffff700 1 -- 172.21.15.52:0/3052024881 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f81a5f3080 msgr2=0x55f81a6045e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:57:33.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.033+0000 7f719ffff700 1 --2- 172.21.15.52:0/3052024881 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f81a5f3080 0x55f81a6045e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:33.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.033+0000 7f719ffff700 1 -- 172.21.15.52:0/3052024881 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f81a679c80 con 0x55f81a5f1820 2024-06-10T02:57:33.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.033+0000 7f719ffff700 1 --2- 172.21.15.52:0/3052024881 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f81a5f1820 0x55f81a608250 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f7190009fd0 tx=0x7f71900047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6914120ea50c57b8 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:57:33.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.034+0000 7f717f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-10T02:57:33.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.034+0000 7f717f7fe700 1 -- 172.21.15.52:0/3052024881 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f719001f900 con 0x55f81a5f1820 2024-06-10T02:57:33.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.034+0000 7f717f7fe700 1 -- 172.21.15.52:0/3052024881 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f719001faa0 con 0x55f81a5f1820 2024-06-10T02:57:33.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.034+0000 7f71b0ec1b80 1 -- 172.21.15.52:0/3052024881 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f71940a4060 con 0x55f81a5f1820 2024-06-10T02:57:33.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.034+0000 7f717f7fe700 1 -- 172.21.15.52:0/3052024881 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f719001fca0 con 0x55f81a5f1820 2024-06-10T02:57:33.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.034+0000 7f71b0ec1b80 10 client.16038 fetch_fsmap learned FSMap version 33 2024-06-10T02:57:33.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.034+0000 7f71b0ec1b80 1 -- 172.21.15.52:0/3052024881 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55f81a5f3610 con 0x55f81a5f1820 2024-06-10T02:57:33.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.034+0000 7f717f7fe700 1 -- 172.21.15.52:0/3052024881 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71940a4060 con 0x55f81a5f1820 2024-06-10T02:57:33.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.034+0000 7f717f7fe700 1 -- 172.21.15.52:0/3052024881 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 33) ==== 8382+0+0 (secure 0 0 0) 0x7f7190029580 con 0x55f81a5f1820 2024-06-10T02:57:33.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.034+0000 7f71b0ec1b80 10 client.16038 fetch_fsmap finished waiting for FSMap version 33 2024-06-10T02:57:33.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.034+0000 7f71b0ec1b80 10 client.16038 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T02:57:33.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.034+0000 7f71b0ec1b80 20 client.16038 populate_metadata read hostname 'smithi052' 2024-06-10T02:57:33.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.034+0000 7f71b0ec1b80 1 --2- 172.21.15.52:0/3052024881 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55f81a60f310 0x55f81a5f5d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:57:33.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.034+0000 7f71b0ec1b80 4 client.16038 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-06-10T02:57:33.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.034+0000 7f71b0ec1b80 1 -- 172.21.15.52:0/3052024881 --> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55f81a54a350 con 0x55f81a60f310 2024-06-10T02:57:33.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.035+0000 7f719f7fe700 1 --2- 172.21.15.52:0/3052024881 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55f81a60f310 0x55f81a5f5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:57:33.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.035+0000 7f717f7fe700 10 client.16038 ms_handle_connect on v2:172.21.15.188:6838/310444836 2024-06-10T02:57:33.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.035+0000 7f719f7fe700 1 --2- 172.21.15.52:0/3052024881 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55f81a60f310 0x55f81a5f5d00 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f39d5b70f14b0db0 server_cookie=7f5da3c4bbe0f0d6 in_seq=0 out_seq=0 2024-06-10T02:57:33.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.036+0000 7f717f7fe700 1 -- 172.21.15.52:0/3052024881 <== mds.0 v2:172.21.15.188:6838/310444836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55f81a54a350 con 0x55f81a60f310 2024-06-10T02:57:33.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.036+0000 7f717f7fe700 10 client.16038 handle_command_reply: tid=0 2024-06-10T02:57:33.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.036+0000 7f71b0ec1b80 1 client.16038 shutdown 2024-06-10T02:57:33.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.036+0000 7f71b0ec1b80 1 -- 172.21.15.52:0/3052024881 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f81a5f1820 msgr2=0x55f81a608250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:57:33.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.036+0000 7f71b0ec1b80 1 --2- 172.21.15.52:0/3052024881 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f81a5f1820 0x55f81a608250 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f7190009fd0 tx=0x7f71900047c0 comp rx=0 tx=0).stop 2024-06-10T02:57:33.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.036+0000 7f71b0ec1b80 1 -- 172.21.15.52:0/3052024881 shutdown_connections 2024-06-10T02:57:33.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.036+0000 7f71b0ec1b80 1 --2- 172.21.15.52:0/3052024881 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55f81a60f310 0x55f81a5f5d00 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:33.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.036+0000 7f71b0ec1b80 1 --2- 172.21.15.52:0/3052024881 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f81a5ff960 0x55f81a604d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:33.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.036+0000 7f71b0ec1b80 1 --2- 172.21.15.52:0/3052024881 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f81a5f1820 0x55f81a608250 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:33.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.036+0000 7f71b0ec1b80 1 --2- 172.21.15.52:0/3052024881 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f81a5f3080 0x55f81a6045e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:33.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.036+0000 7f71b0ec1b80 1 -- 172.21.15.52:0/3052024881 >> 172.21.15.52:0/3052024881 conn(0x55f81a51e100 msgr2=0x55f81a5fe510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:57:33.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.036+0000 7f717f7fe700 0 client.16038 ms_handle_reset on v2:172.21.15.188:6838/310444836 2024-06-10T02:57:33.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.036+0000 7f71b0ec1b80 1 -- 172.21.15.52:0/3052024881 shutdown_connections 2024-06-10T02:57:33.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.036+0000 7f71b0ec1b80 1 -- 172.21.15.52:0/3052024881 wait complete. 2024-06-10T02:57:33.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.036+0000 7f71b0ec1b80 20 client.16038 trim_cache size 0 max 16384 2024-06-10T02:57:33.043 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-10T02:57:33.043 INFO:teuthology.orchestra.run.smithi052.stdout: "return_code": 0, 2024-06-10T02:57:33.043 INFO:teuthology.orchestra.run.smithi052.stdout: "scrub_tag": "3f69b16b-280a-4a2b-870d-9350f1375f80", 2024-06-10T02:57:33.043 INFO:teuthology.orchestra.run.smithi052.stdout: "mode": "asynchronous" 2024-06-10T02:57:33.044 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-10T02:57:33.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.037+0000 7f71a663e700 1 -- 172.21.15.52:0/400252555 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f71880a2ca0 msgr2=0x7f71880a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:57:33.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.037+0000 7f71a663e700 1 --2- 172.21.15.52:0/400252555 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f71880a2ca0 0x7f71880a5160 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f71a0198a10 tx=0x7f7190009260 comp rx=0 tx=0).stop 2024-06-10T02:57:33.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.037+0000 7f71a663e700 1 -- 172.21.15.52:0/400252555 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f71a0101d50 msgr2=0x7f71a019dc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:57:33.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.037+0000 7f71a663e700 1 --2- 172.21.15.52:0/400252555 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f71a0101d50 0x7f71a019dc50 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f719400d950 tx=0x7f719400dc60 comp rx=0 tx=0).stop 2024-06-10T02:57:33.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.037+0000 7f71a663e700 1 -- 172.21.15.52:0/400252555 shutdown_connections 2024-06-10T02:57:33.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.037+0000 7f71a663e700 1 --2- 172.21.15.52:0/400252555 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f71a0107400 0x7f71a01978d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:33.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.037+0000 7f71a663e700 1 --2- 172.21.15.52:0/400252555 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f71a0101390 0x7f71a019d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:33.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.037+0000 7f71a663e700 1 --2- 172.21.15.52:0/400252555 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f71880a2ca0 0x7f71880a5160 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:33.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.037+0000 7f71a663e700 1 --2- 172.21.15.52:0/400252555 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f71a0101d50 0x7f71a019dc50 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:33.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.037+0000 7f71a663e700 1 -- 172.21.15.52:0/400252555 >> 172.21.15.52:0/400252555 conn(0x7f71a0078340 msgr2=0x7f71a00ffde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:57:33.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.037+0000 7f71a663e700 1 -- 172.21.15.52:0/400252555 shutdown_connections 2024-06-10T02:57:33.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:33.037+0000 7f71a663e700 1 -- 172.21.15.52:0/400252555 wait complete. 2024-06-10T02:57:33.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:57:32 smithi120 ceph-mon[36634]: pgmap v534: 129 pgs: 129 active+clean; 224 MiB data, 584 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 7.5 MiB/s wr, 577 op/s 2024-06-10T02:57:33.681 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:57:33 smithi052 ceph-mon[32362]: scrub queued for path: / 2024-06-10T02:57:33.681 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:57:33 smithi052 ceph-mon[32362]: scrub summary: idle+waiting paths [/] 2024-06-10T02:57:33.681 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:57:33 smithi052 ceph-mon[32362]: scrub summary: active paths [/] 2024-06-10T02:57:33.708 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-10T02:57:33.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:57:33 smithi188 ceph-mon[36599]: scrub queued for path: / 2024-06-10T02:57:33.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:57:33 smithi188 ceph-mon[36599]: scrub summary: idle+waiting paths [/] 2024-06-10T02:57:33.954 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:57:33 smithi188 ceph-mon[36599]: scrub summary: active paths [/] 2024-06-10T02:57:34.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:57:33 smithi120 ceph-mon[36634]: scrub queued for path: / 2024-06-10T02:57:34.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:57:33 smithi120 ceph-mon[36634]: scrub summary: idle+waiting paths [/] 2024-06-10T02:57:34.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:57:33 smithi120 ceph-mon[36634]: scrub summary: active paths [/] 2024-06-10T02:57:34.420 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:57:34.809 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:57:34 smithi052 ceph-mon[32362]: scrub summary: active paths [/] 2024-06-10T02:57:34.809 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:57:34 smithi052 ceph-mon[32362]: pgmap v535: 129 pgs: 129 active+clean; 226 MiB data, 592 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 7.7 MiB/s wr, 638 op/s 2024-06-10T02:57:34.809 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:57:34 smithi052 ceph-mon[32362]: scrub summary: idle+waiting paths [/] 2024-06-10T02:57:34.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:57:34 smithi188 ceph-mon[36599]: scrub summary: active paths [/] 2024-06-10T02:57:34.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:57:34 smithi188 ceph-mon[36599]: pgmap v535: 129 pgs: 129 active+clean; 226 MiB data, 592 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 7.7 MiB/s wr, 638 op/s 2024-06-10T02:57:34.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:57:34 smithi188 ceph-mon[36599]: scrub summary: idle+waiting paths [/] 2024-06-10T02:57:35.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:57:34 smithi120 ceph-mon[36634]: scrub summary: active paths [/] 2024-06-10T02:57:35.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:57:34 smithi120 ceph-mon[36634]: pgmap v535: 129 pgs: 129 active+clean; 226 MiB data, 592 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 7.7 MiB/s wr, 638 op/s 2024-06-10T02:57:35.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:57:34 smithi120 ceph-mon[36634]: scrub summary: idle+waiting paths [/] 2024-06-10T02:57:35.857 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:57:35 smithi052 ceph-mon[32362]: scrub summary: active paths [/] 2024-06-10T02:57:35.857 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:57:35 smithi052 ceph-mon[32362]: scrub summary: idle+waiting paths [/] 2024-06-10T02:57:35.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:57:35 smithi188 ceph-mon[36599]: scrub summary: active paths [/] 2024-06-10T02:57:35.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:57:35 smithi188 ceph-mon[36599]: scrub summary: idle+waiting paths [/] 2024-06-10T02:57:36.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:57:35 smithi120 ceph-mon[36634]: scrub summary: active paths [/] 2024-06-10T02:57:36.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:57:35 smithi120 ceph-mon[36634]: scrub summary: idle+waiting paths [/] 2024-06-10T02:57:36.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:57:36 smithi188 ceph-mon[36599]: pgmap v536: 129 pgs: 129 active+clean; 226 MiB data, 585 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 7.1 MiB/s wr, 627 op/s 2024-06-10T02:57:36.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:57:36 smithi188 ceph-mon[36599]: scrub summary: active paths [/] 2024-06-10T02:57:36.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:57:36 smithi188 ceph-mon[36599]: scrub summary: idle+waiting paths [/] 2024-06-10T02:57:36.973 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:57:36 smithi052 ceph-mon[32362]: pgmap v536: 129 pgs: 129 active+clean; 226 MiB data, 585 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 7.1 MiB/s wr, 627 op/s 2024-06-10T02:57:36.973 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:57:36 smithi052 ceph-mon[32362]: scrub summary: active paths [/] 2024-06-10T02:57:36.974 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:57:36 smithi052 ceph-mon[32362]: scrub summary: idle+waiting paths [/] 2024-06-10T02:57:37.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.116+0000 7f9b11d5a700 1 -- 172.21.15.52:0/2394828685 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9b0c110b80 msgr2=0x7f9b0c109490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:57:37.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.116+0000 7f9b11d5a700 1 --2- 172.21.15.52:0/2394828685 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9b0c110b80 0x7f9b0c109490 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f9afc009990 tx=0x7f9afc009ca0 comp rx=0 tx=0).stop 2024-06-10T02:57:37.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.119+0000 7f9b11d5a700 1 -- 172.21.15.52:0/2394828685 shutdown_connections 2024-06-10T02:57:37.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.119+0000 7f9b11d5a700 1 --2- 172.21.15.52:0/2394828685 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9b0c11bc70 0x7f9b0c11e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:37.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.119+0000 7f9b11d5a700 1 --2- 172.21.15.52:0/2394828685 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9b0c110b80 0x7f9b0c109490 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:37.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.119+0000 7f9b11d5a700 1 --2- 172.21.15.52:0/2394828685 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9b0c110ff0 0x7f9b0c1099d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:37.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.119+0000 7f9b11d5a700 1 -- 172.21.15.52:0/2394828685 >> 172.21.15.52:0/2394828685 conn(0x7f9b0c06d470 msgr2=0x7f9b0c06d880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:57:37.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.128+0000 7f9b11d5a700 1 -- 172.21.15.52:0/2394828685 shutdown_connections 2024-06-10T02:57:37.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.132+0000 7f9b11d5a700 1 -- 172.21.15.52:0/2394828685 wait complete. 2024-06-10T02:57:37.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.133+0000 7f9b11d5a700 1 Processor -- start 2024-06-10T02:57:37.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.133+0000 7f9b11d5a700 1 -- start start 2024-06-10T02:57:37.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.134+0000 7f9b11d5a700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9b0c110b80 0x7f9b0c11a140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:57:37.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.134+0000 7f9b11d5a700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9b0c110ff0 0x7f9b0c11a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:57:37.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.134+0000 7f9b11d5a700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9b0c11bc70 0x7f9b0c113470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:57:37.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.134+0000 7f9b11d5a700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f9b0c1216c0 con 0x7f9b0c11bc70 2024-06-10T02:57:37.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.134+0000 7f9b11d5a700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f9b0c121540 con 0x7f9b0c110ff0 2024-06-10T02:57:37.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.134+0000 7f9b11d5a700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f9b0c121840 con 0x7f9b0c110b80 2024-06-10T02:57:37.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.134+0000 7f9b11559700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9b0c11bc70 0x7f9b0c113470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:57:37.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.134+0000 7f9b11559700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9b0c11bc70 0x7f9b0c113470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53516/0 (socket says 172.21.15.52:53516) 2024-06-10T02:57:37.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.134+0000 7f9b11559700 1 -- 172.21.15.52:0/829496903 learned_addr learned my addr 172.21.15.52:0/829496903 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:57:37.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.134+0000 7f9b10d58700 1 --2- 172.21.15.52:0/829496903 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9b0c110b80 0x7f9b0c11a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:57:37.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.135+0000 7f9b11559700 1 -- 172.21.15.52:0/829496903 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9b0c110b80 msgr2=0x7f9b0c11a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:57:37.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.135+0000 7f9b11559700 1 --2- 172.21.15.52:0/829496903 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9b0c110b80 0x7f9b0c11a140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:37.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.135+0000 7f9b11559700 1 -- 172.21.15.52:0/829496903 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9b0c110ff0 msgr2=0x7f9b0c11a680 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-06-10T02:57:37.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.135+0000 7f9b11559700 1 --2- 172.21.15.52:0/829496903 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9b0c110ff0 0x7f9b0c11a680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:37.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.135+0000 7f9b11559700 1 -- 172.21.15.52:0/829496903 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b0c113d70 con 0x7f9b0c11bc70 2024-06-10T02:57:37.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.137+0000 7f9b11559700 1 --2- 172.21.15.52:0/829496903 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9b0c11bc70 0x7f9b0c113470 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f9b0400b450 tx=0x7f9b0400b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:57:37.139 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:57:36 smithi120 ceph-mon[36634]: pgmap v536: 129 pgs: 129 active+clean; 226 MiB data, 585 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 7.1 MiB/s wr, 627 op/s 2024-06-10T02:57:37.139 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:57:36 smithi120 ceph-mon[36634]: scrub summary: active paths [/] 2024-06-10T02:57:37.139 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:57:36 smithi120 ceph-mon[36634]: scrub summary: idle+waiting paths [/] 2024-06-10T02:57:37.141 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.138+0000 7f9b09ffb700 1 -- 172.21.15.52:0/829496903 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b04015020 con 0x7f9b0c11bc70 2024-06-10T02:57:37.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.138+0000 7f9b09ffb700 1 -- 172.21.15.52:0/829496903 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b04011d10 con 0x7f9b0c11bc70 2024-06-10T02:57:37.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.138+0000 7f9b09ffb700 1 -- 172.21.15.52:0/829496903 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b0400f2e0 con 0x7f9b0c11bc70 2024-06-10T02:57:37.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.139+0000 7f9b11d5a700 1 -- 172.21.15.52:0/829496903 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b0c114060 con 0x7f9b0c11bc70 2024-06-10T02:57:37.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.139+0000 7f9b11d5a700 1 -- 172.21.15.52:0/829496903 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b0c117f90 con 0x7f9b0c11bc70 2024-06-10T02:57:37.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.149+0000 7f9b09ffb700 1 -- 172.21.15.52:0/829496903 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f9b04018020 con 0x7f9b0c11bc70 2024-06-10T02:57:37.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.149+0000 7f9b1d83fb80 1 Processor -- start 2024-06-10T02:57:37.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.149+0000 7f9b1d83fb80 1 -- start start 2024-06-10T02:57:37.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.150+0000 7f9b1d83fb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c76b1a60a0 0x55c76b1a6480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:57:37.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.150+0000 7f9b1d83fb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c76b1a69c0 0x55c76b1a14b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:57:37.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.150+0000 7f9b1d83fb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c76b1a19f0 0x55c76b1a1e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:57:37.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.150+0000 7f9b1d83fb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55c76b093f60 con 0x55c76b1a60a0 2024-06-10T02:57:37.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.150+0000 7f9b1d83fb80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55c76b0f2850 con 0x55c76b1a69c0 2024-06-10T02:57:37.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.150+0000 7f9b1d83fb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55c76b05a460 con 0x55c76b1a19f0 2024-06-10T02:57:37.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.150+0000 7f9b0bfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c76b1a60a0 0x55c76b1a6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:57:37.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.150+0000 7f9b0bfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c76b1a60a0 0x55c76b1a6480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53532/0 (socket says 172.21.15.52:53532) 2024-06-10T02:57:37.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.150+0000 7f9b0bfff700 1 -- 172.21.15.52:0/1885343547 learned_addr learned my addr 172.21.15.52:0/1885343547 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:57:37.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.150+0000 7f9b0bfff700 1 --2- 172.21.15.52:0/1885343547 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c76b1a19f0 0x55c76b1a1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:57:37.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.150+0000 7f9b10d58700 1 --2- 172.21.15.52:0/1885343547 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c76b1a69c0 0x55c76b1a14b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:57:37.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.150+0000 7f9b0bfff700 1 -- 172.21.15.52:0/1885343547 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c76b1a19f0 msgr2=0x55c76b1a1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:57:37.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.150+0000 7f9b0bfff700 1 --2- 172.21.15.52:0/1885343547 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c76b1a19f0 0x55c76b1a1e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:37.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.150+0000 7f9b0bfff700 1 -- 172.21.15.52:0/1885343547 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c76b1a69c0 msgr2=0x55c76b1a14b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:57:37.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.150+0000 7f9b0bfff700 1 --2- 172.21.15.52:0/1885343547 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c76b1a69c0 0x55c76b1a14b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:37.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.150+0000 7f9b0bfff700 1 -- 172.21.15.52:0/1885343547 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c76b0418d0 con 0x55c76b1a60a0 2024-06-10T02:57:37.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.153+0000 7f9b0bfff700 1 --2- 172.21.15.52:0/1885343547 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c76b1a60a0 0x55c76b1a6480 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f9b00012770 tx=0x7f9b00009550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d441792adbf171b server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:57:37.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.156+0000 7f9b09ffb700 1 --2- 172.21.15.52:0/829496903 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f9af40a2e30 0x7f9af40a52f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:57:37.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.157+0000 7f9b10d58700 1 --2- 172.21.15.52:0/829496903 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f9af40a2e30 0x7f9af40a52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:57:37.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.157+0000 7f9af2ffd700 1 -- 172.21.15.52:0/1885343547 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b00007730 con 0x55c76b1a60a0 2024-06-10T02:57:37.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.157+0000 7f9af2ffd700 1 -- 172.21.15.52:0/1885343547 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b0000d070 con 0x55c76b1a60a0 2024-06-10T02:57:37.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.158+0000 7f9b1d83fb80 1 -- 172.21.15.52:0/1885343547 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c76b1a60a0 msgr2=0x55c76b1a6480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:57:37.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.158+0000 7f9b1d83fb80 1 --2- 172.21.15.52:0/1885343547 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c76b1a60a0 0x55c76b1a6480 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f9b00012770 tx=0x7f9b00009550 comp rx=0 tx=0).stop 2024-06-10T02:57:37.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.158+0000 7f9b10d58700 1 --2- 172.21.15.52:0/829496903 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f9af40a2e30 0x7f9af40a52f0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f9afc009480 tx=0x7f9afc005040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:57:37.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.160+0000 7f9b09ffb700 1 -- 172.21.15.52:0/829496903 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f9b040ce9c0 con 0x7f9b0c11bc70 2024-06-10T02:57:37.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.161+0000 7f9b1d83fb80 1 -- 172.21.15.52:0/1885343547 shutdown_connections 2024-06-10T02:57:37.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.161+0000 7f9b1d83fb80 1 --2- 172.21.15.52:0/1885343547 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c76b1a19f0 0x55c76b1a1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:37.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.161+0000 7f9b1d83fb80 1 --2- 172.21.15.52:0/1885343547 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c76b1a60a0 0x55c76b1a6480 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:37.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.161+0000 7f9b1d83fb80 1 --2- 172.21.15.52:0/1885343547 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c76b1a69c0 0x55c76b1a14b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:37.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.161+0000 7f9b1d83fb80 1 -- 172.21.15.52:0/1885343547 >> 172.21.15.52:0/1885343547 conn(0x55c76b0ce080 msgr2=0x55c76b19c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:57:37.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.161+0000 7f9b1d83fb80 1 -- 172.21.15.52:0/1885343547 shutdown_connections 2024-06-10T02:57:37.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.161+0000 7f9b1d83fb80 1 -- 172.21.15.52:0/1885343547 wait complete. 2024-06-10T02:57:37.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.161+0000 7f9b1d83fb80 1 -- start start 2024-06-10T02:57:37.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.162+0000 7f9b1d83fb80 1 Processor -- start 2024-06-10T02:57:37.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.166+0000 7f9b1d83fb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c76b1a19f0 0x55c76b1ad740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:57:37.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.166+0000 7f9b1d83fb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c76b1a60a0 0x55c76b1aaa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:57:37.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.166+0000 7f9b1d83fb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c76b1a69c0 0x55c76b1aaf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:57:37.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.167+0000 7f9b1d83fb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55c76b092fc0 con 0x55c76b1a60a0 2024-06-10T02:57:37.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.167+0000 7f9b1d83fb80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55c76b0ee810 con 0x55c76b1a19f0 2024-06-10T02:57:37.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.167+0000 7f9b11559700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c76b1a69c0 0x55c76b1aaf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:57:37.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.167+0000 7f9b11559700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c76b1a69c0 0x55c76b1aaf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.52:49066/0 (socket says 172.21.15.52:49066) 2024-06-10T02:57:37.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.167+0000 7f9b11559700 1 -- 172.21.15.52:0/1649377247 learned_addr learned my addr 172.21.15.52:0/1649377247 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:57:37.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.167+0000 7f9b0bfff700 1 --2- 172.21.15.52:0/1649377247 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c76b1a60a0 0x55c76b1aaa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:57:37.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.167+0000 7f9b1d83fb80 1 -- 172.21.15.52:0/1649377247 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55c76b1ae4a0 con 0x55c76b1a69c0 2024-06-10T02:57:37.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.168+0000 7f9b1d83fb80 10 client.0 _ll_register_callbacks cb 0x55c76b0885b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T02:57:37.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.168+0000 7f9b0bfff700 1 -- 172.21.15.52:0/1649377247 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c76b1a69c0 msgr2=0x55c76b1aaf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:57:37.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.168+0000 7f9b0bfff700 1 --2- 172.21.15.52:0/1649377247 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c76b1a69c0 0x55c76b1aaf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:37.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.168+0000 7f9b0bfff700 1 -- 172.21.15.52:0/1649377247 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c76b1a19f0 msgr2=0x55c76b1ad740 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-10T02:57:37.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.168+0000 7f9b0bfff700 1 --2- 172.21.15.52:0/1649377247 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c76b1a19f0 0x55c76b1ad740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:37.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.168+0000 7f9b0bfff700 1 -- 172.21.15.52:0/1649377247 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c76b0b8750 con 0x55c76b1a60a0 2024-06-10T02:57:37.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.169+0000 7f9b0bfff700 1 --2- 172.21.15.52:0/1649377247 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c76b1a60a0 0x55c76b1aaa40 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x55c76b19dbd0 tx=0x7f9b00018e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fe40b8749d40acf server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:57:37.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.170+0000 7f9aebfff700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-10T02:57:37.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.170+0000 7f9aebfff700 1 -- 172.21.15.52:0/1649377247 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b00028060 con 0x55c76b1a60a0 2024-06-10T02:57:37.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.170+0000 7f9aebfff700 1 -- 172.21.15.52:0/1649377247 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f9b00007740 con 0x55c76b1a60a0 2024-06-10T02:57:37.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.170+0000 7f9b1d83fb80 1 -- 172.21.15.52:0/1649377247 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55c76b0ffd50 con 0x55c76b1a60a0 2024-06-10T02:57:37.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.170+0000 7f9aebfff700 1 -- 172.21.15.52:0/1649377247 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f9b0002f050 con 0x55c76b1a60a0 2024-06-10T02:57:37.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.170+0000 7f9b1d83fb80 10 client.16050 fetch_fsmap learned FSMap version 33 2024-06-10T02:57:37.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.171+0000 7f9b1d83fb80 1 -- 172.21.15.52:0/1649377247 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55c76b1b8cd0 con 0x55c76b1a60a0 2024-06-10T02:57:37.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.172+0000 7f9aebfff700 1 -- 172.21.15.52:0/1649377247 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b0000d050 con 0x55c76b1a60a0 2024-06-10T02:57:37.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.172+0000 7f9aebfff700 1 -- 172.21.15.52:0/1649377247 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 33) ==== 8382+0+0 (secure 0 0 0) 0x7f9b00022070 con 0x55c76b1a60a0 2024-06-10T02:57:37.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.172+0000 7f9b1d83fb80 10 client.16050 fetch_fsmap finished waiting for FSMap version 33 2024-06-10T02:57:37.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.172+0000 7f9b1d83fb80 10 client.16050 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T02:57:37.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.172+0000 7f9b1d83fb80 20 client.16050 populate_metadata read hostname 'smithi052' 2024-06-10T02:57:37.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.172+0000 7f9b1d83fb80 1 --2- 172.21.15.52:0/1649377247 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55c76b1bb1a0 0x55c76b1bd590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:57:37.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.172+0000 7f9b1d83fb80 4 client.16050 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T02:57:37.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.172+0000 7f9b1d83fb80 1 -- 172.21.15.52:0/1649377247 --> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c76b0ffd50 con 0x55c76b1bb1a0 2024-06-10T02:57:37.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.174+0000 7f9b10d58700 1 --2- 172.21.15.52:0/1649377247 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55c76b1bb1a0 0x55c76b1bd590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:57:37.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.179+0000 7f9b10d58700 1 --2- 172.21.15.52:0/1649377247 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55c76b1bb1a0 0x55c76b1bd590 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab68de1db5fc0a0a server_cookie=64435ccc73f727b4 in_seq=0 out_seq=0 2024-06-10T02:57:37.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.179+0000 7f9aebfff700 10 client.16050 ms_handle_connect on v2:172.21.15.188:6838/310444836 2024-06-10T02:57:37.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.180+0000 7f9aebfff700 1 -- 172.21.15.52:0/1649377247 <== mds.0 v2:172.21.15.188:6838/310444836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55c76b0ffd50 con 0x55c76b1bb1a0 2024-06-10T02:57:37.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.180+0000 7f9aebfff700 10 client.16050 handle_command_reply: tid=0 2024-06-10T02:57:37.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.180+0000 7f9b1d83fb80 1 client.16050 shutdown 2024-06-10T02:57:37.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.182+0000 7f9b1d83fb80 1 -- 172.21.15.52:0/1649377247 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c76b1a60a0 msgr2=0x55c76b1aaa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:57:37.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.182+0000 7f9b1d83fb80 1 --2- 172.21.15.52:0/1649377247 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c76b1a60a0 0x55c76b1aaa40 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x55c76b19dbd0 tx=0x7f9b00018e40 comp rx=0 tx=0).stop 2024-06-10T02:57:37.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.182+0000 7f9b1d83fb80 1 -- 172.21.15.52:0/1649377247 shutdown_connections 2024-06-10T02:57:37.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.182+0000 7f9b1d83fb80 1 --2- 172.21.15.52:0/1649377247 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55c76b1bb1a0 0x55c76b1bd590 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:37.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.182+0000 7f9b1d83fb80 1 --2- 172.21.15.52:0/1649377247 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c76b1a69c0 0x55c76b1aaf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:37.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.182+0000 7f9b1d83fb80 1 --2- 172.21.15.52:0/1649377247 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c76b1a19f0 0x55c76b1ad740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:37.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.183+0000 7f9b1d83fb80 1 --2- 172.21.15.52:0/1649377247 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c76b1a60a0 0x55c76b1aaa40 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:37.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.183+0000 7f9b1d83fb80 1 -- 172.21.15.52:0/1649377247 >> 172.21.15.52:0/1649377247 conn(0x55c76b0ce080 msgr2=0x55c76b19b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:57:37.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.183+0000 7f9aebfff700 0 client.16050 ms_handle_reset on v2:172.21.15.188:6838/310444836 2024-06-10T02:57:37.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.185+0000 7f9b1d83fb80 1 -- 172.21.15.52:0/1649377247 shutdown_connections 2024-06-10T02:57:37.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.186+0000 7f9b1d83fb80 1 -- 172.21.15.52:0/1649377247 wait complete. 2024-06-10T02:57:37.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.186+0000 7f9b1d83fb80 20 client.16050 trim_cache size 0 max 16384 2024-06-10T02:57:37.241 INFO:tasks.workunit.client.0.smithi052.stdout: 45 611869 403 400243 234 330451 944 2024-06-10T02:57:37.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.234+0000 7f9b1d83fb80 1 Processor -- start 2024-06-10T02:57:37.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.235+0000 7f9b1d83fb80 1 -- start start 2024-06-10T02:57:37.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.235+0000 7f9b1d83fb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c76b1a19f0 0x55c76b1aafb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:57:37.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.235+0000 7f9b1d83fb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c76b1a60a0 0x55c76b1a5580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:57:37.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.235+0000 7f9b1d83fb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c76b1a69c0 0x55c76b19c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:57:37.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.235+0000 7f9b1d83fb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f9af800ecc0 con 0x55c76b1a60a0 2024-06-10T02:57:37.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.235+0000 7f9b1d83fb80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f9af80108f0 con 0x55c76b1a19f0 2024-06-10T02:57:37.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.235+0000 7f9b1d83fb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f9af800f390 con 0x55c76b1a69c0 2024-06-10T02:57:37.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.236+0000 7f9b10d58700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c76b1a19f0 0x55c76b1aafb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:57:37.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.236+0000 7f9b10d58700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c76b1a19f0 0x55c76b1aafb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.52:44444/0 (socket says 172.21.15.52:44444) 2024-06-10T02:57:37.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.236+0000 7f9b10d58700 1 -- 172.21.15.52:0/1881061996 learned_addr learned my addr 172.21.15.52:0/1881061996 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:57:37.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.237+0000 7f9b10d58700 1 -- 172.21.15.52:0/1881061996 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c76b1a69c0 msgr2=0x55c76b19c0d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-10T02:57:37.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.237+0000 7f9b10d58700 1 --2- 172.21.15.52:0/1881061996 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c76b1a69c0 0x55c76b19c0d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:37.247 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.237+0000 7f9b10d58700 1 -- 172.21.15.52:0/1881061996 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c76b1a60a0 msgr2=0x55c76b1a5580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:57:37.247 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.237+0000 7f9b10d58700 1 --2- 172.21.15.52:0/1881061996 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c76b1a60a0 0x55c76b1a5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:37.247 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.237+0000 7f9b10d58700 1 -- 172.21.15.52:0/1881061996 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c76b1ae570 con 0x55c76b1a19f0 2024-06-10T02:57:37.247 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.238+0000 7f9b10d58700 1 --2- 172.21.15.52:0/1881061996 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c76b1a19f0 0x55c76b1aafb0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f9afc0098e0 tx=0x7f9afc021000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=12240c0b30957070 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:57:37.247 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.242+0000 7f9af27fc700 1 -- 172.21.15.52:0/1881061996 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9afc005510 con 0x55c76b1a19f0 2024-06-10T02:57:37.247 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.242+0000 7f9af27fc700 1 -- 172.21.15.52:0/1881061996 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9afc0056b0 con 0x55c76b1a19f0 2024-06-10T02:57:37.247 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.242+0000 7f9af27fc700 1 -- 172.21.15.52:0/1881061996 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9afc02fbc0 con 0x55c76b1a19f0 2024-06-10T02:57:37.247 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.245+0000 7f9b1d83fb80 1 -- 172.21.15.52:0/1881061996 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c76b1a19f0 msgr2=0x55c76b1aafb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:57:37.247 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.245+0000 7f9b1d83fb80 1 --2- 172.21.15.52:0/1881061996 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c76b1a19f0 0x55c76b1aafb0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f9afc0098e0 tx=0x7f9afc021000 comp rx=0 tx=0).stop 2024-06-10T02:57:37.249 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.249+0000 7f9b1d83fb80 1 -- 172.21.15.52:0/1881061996 shutdown_connections 2024-06-10T02:57:37.249 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.249+0000 7f9b1d83fb80 1 --2- 172.21.15.52:0/1881061996 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c76b1a69c0 0x55c76b19c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:37.249 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.249+0000 7f9b1d83fb80 1 --2- 172.21.15.52:0/1881061996 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c76b1a19f0 0x55c76b1aafb0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:37.249 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.249+0000 7f9b1d83fb80 1 --2- 172.21.15.52:0/1881061996 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c76b1a60a0 0x55c76b1a5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:37.249 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.249+0000 7f9b1d83fb80 1 -- 172.21.15.52:0/1881061996 >> 172.21.15.52:0/1881061996 conn(0x55c76b0ce080 msgr2=0x55c76b1acb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:57:37.254 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.254+0000 7f9b1d83fb80 1 -- 172.21.15.52:0/1881061996 shutdown_connections 2024-06-10T02:57:37.254 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.254+0000 7f9b1d83fb80 1 -- 172.21.15.52:0/1881061996 wait complete. 2024-06-10T02:57:37.254 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.255+0000 7f9b1d83fb80 1 -- start start 2024-06-10T02:57:37.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.255+0000 7f9b1d83fb80 1 Processor -- start 2024-06-10T02:57:37.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.256+0000 7f9b1d83fb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c76b1a19f0 0x55c76b1b3a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:57:37.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.257+0000 7f9b1d83fb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c76b1a60a0 0x55c76b1b3fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:57:37.257 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.257+0000 7f9b10d58700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c76b1a19f0 0x55c76b1b3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:57:37.257 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.257+0000 7f9b1d83fb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c76b1a69c0 0x55c76b1a7db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:57:37.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.258+0000 7f9b0bfff700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c76b1a60a0 0x55c76b1b3fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:57:37.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.258+0000 7f9b11559700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c76b1a69c0 0x55c76b1a7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:57:37.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.258+0000 7f9b10d58700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c76b1a19f0 0x55c76b1b3a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.52:44458/0 (socket says 172.21.15.52:44458) 2024-06-10T02:57:37.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.258+0000 7f9b11559700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c76b1a69c0 0x55c76b1a7db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53552/0 (socket says 172.21.15.52:53552) 2024-06-10T02:57:37.259 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.258+0000 7f9b10d58700 1 -- 172.21.15.52:0/621903399 learned_addr learned my addr 172.21.15.52:0/621903399 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:57:37.259 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.259+0000 7f9b1d83fb80 1 -- 172.21.15.52:0/621903399 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f9af80101f0 con 0x55c76b1a69c0 2024-06-10T02:57:37.259 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.259+0000 7f9b1d83fb80 1 -- 172.21.15.52:0/621903399 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f9af800fb20 con 0x55c76b1a19f0 2024-06-10T02:57:37.259 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.259+0000 7f9b1d83fb80 1 -- 172.21.15.52:0/621903399 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f9af800cc60 con 0x55c76b1a60a0 2024-06-10T02:57:37.259 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.259+0000 7f9b1d83fb80 10 client.0 _ll_register_callbacks cb 0x55c76b0885b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T02:57:37.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.262+0000 7f9b11559700 1 -- 172.21.15.52:0/621903399 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c76b1a60a0 msgr2=0x55c76b1b3fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:57:37.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.262+0000 7f9b11559700 1 --2- 172.21.15.52:0/621903399 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c76b1a60a0 0x55c76b1b3fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:37.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.262+0000 7f9b11559700 1 -- 172.21.15.52:0/621903399 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c76b1a19f0 msgr2=0x55c76b1b3a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:57:37.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.262+0000 7f9b11559700 1 --2- 172.21.15.52:0/621903399 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c76b1a19f0 0x55c76b1b3a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:37.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.262+0000 7f9b11559700 1 -- 172.21.15.52:0/621903399 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c76b1a85f0 con 0x55c76b1a69c0 2024-06-10T02:57:37.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.263+0000 7f9b11559700 1 --2- 172.21.15.52:0/621903399 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c76b1a69c0 0x55c76b1a7db0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f9b040a3550 tx=0x7f9b0409b640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75c1cb47b50c5bd9 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:57:37.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.263+0000 7f9aebfff700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-10T02:57:37.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.265+0000 7f9aebfff700 1 -- 172.21.15.52:0/621903399 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b040a37f0 con 0x55c76b1a69c0 2024-06-10T02:57:37.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.265+0000 7f9aebfff700 1 -- 172.21.15.52:0/621903399 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f9b0409a260 con 0x55c76b1a69c0 2024-06-10T02:57:37.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.267+0000 7f9b1d83fb80 1 -- 172.21.15.52:0/621903399 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9afc0056b0 con 0x55c76b1a69c0 2024-06-10T02:57:37.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.268+0000 7f9aebfff700 1 -- 172.21.15.52:0/621903399 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f9b040a8050 con 0x55c76b1a69c0 2024-06-10T02:57:37.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.272+0000 7f9b1d83fb80 10 client.16053 fetch_fsmap learned FSMap version 33 2024-06-10T02:57:37.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.272+0000 7f9b1d83fb80 1 -- 172.21.15.52:0/621903399 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55c76b1c2d80 con 0x55c76b1a69c0 2024-06-10T02:57:37.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.274+0000 7f9aebfff700 1 -- 172.21.15.52:0/621903399 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9afc0056b0 con 0x55c76b1a69c0 2024-06-10T02:57:37.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.274+0000 7f9aebfff700 1 -- 172.21.15.52:0/621903399 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 33) ==== 8382+0+0 (secure 0 0 0) 0x7f9b04096d30 con 0x55c76b1a69c0 2024-06-10T02:57:37.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.274+0000 7f9b1d83fb80 10 client.16053 fetch_fsmap finished waiting for FSMap version 33 2024-06-10T02:57:37.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.274+0000 7f9b1d83fb80 10 client.16053 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T02:57:37.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.274+0000 7f9b1d83fb80 20 client.16053 populate_metadata read hostname 'smithi052' 2024-06-10T02:57:37.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.275+0000 7f9b1d83fb80 1 --2- 172.21.15.52:0/621903399 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55c76b225ec0 0x55c76b2282e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:57:37.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.275+0000 7f9b1d83fb80 4 client.16053 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-10T02:57:37.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.275+0000 7f9b1d83fb80 1 -- 172.21.15.52:0/621903399 --> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c76b0fa2d0 con 0x55c76b225ec0 2024-06-10T02:57:37.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.276+0000 7f9b10d58700 1 --2- 172.21.15.52:0/621903399 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55c76b225ec0 0x55c76b2282e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:57:37.287 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.285+0000 7f9b10d58700 1 --2- 172.21.15.52:0/621903399 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55c76b225ec0 0x55c76b2282e0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=129380cfd98e00d5 server_cookie=4696d4f196927150 in_seq=0 out_seq=0 2024-06-10T02:57:37.287 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.287+0000 7f9aebfff700 10 client.16053 ms_handle_connect on v2:172.21.15.188:6838/310444836 2024-06-10T02:57:37.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.289+0000 7f9aebfff700 1 -- 172.21.15.52:0/621903399 <== mds.0 v2:172.21.15.188:6838/310444836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55c76b0fa2d0 con 0x55c76b225ec0 2024-06-10T02:57:37.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.289+0000 7f9aebfff700 10 client.16053 handle_command_reply: tid=0 2024-06-10T02:57:37.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.290+0000 7f9b1d83fb80 1 client.16053 shutdown 2024-06-10T02:57:37.301 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.299+0000 7f9b1d83fb80 1 -- 172.21.15.52:0/621903399 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c76b1a69c0 msgr2=0x55c76b1a7db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:57:37.301 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.299+0000 7f9b1d83fb80 1 --2- 172.21.15.52:0/621903399 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c76b1a69c0 0x55c76b1a7db0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f9b040a3550 tx=0x7f9b0409b640 comp rx=0 tx=0).stop 2024-06-10T02:57:37.301 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.300+0000 7f9b1d83fb80 1 -- 172.21.15.52:0/621903399 shutdown_connections 2024-06-10T02:57:37.302 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.300+0000 7f9b1d83fb80 1 --2- 172.21.15.52:0/621903399 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55c76b225ec0 0x55c76b2282e0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:37.302 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.300+0000 7f9b1d83fb80 1 --2- 172.21.15.52:0/621903399 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c76b1a69c0 0x55c76b1a7db0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:37.302 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.300+0000 7f9b1d83fb80 1 --2- 172.21.15.52:0/621903399 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c76b1a19f0 0x55c76b1b3a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:37.302 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.300+0000 7f9b1d83fb80 1 --2- 172.21.15.52:0/621903399 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c76b1a60a0 0x55c76b1b3fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:37.302 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.300+0000 7f9aebfff700 0 client.16053 ms_handle_reset on v2:172.21.15.188:6838/310444836 2024-06-10T02:57:37.302 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.300+0000 7f9b1d83fb80 1 -- 172.21.15.52:0/621903399 >> 172.21.15.52:0/621903399 conn(0x55c76b0ce080 msgr2=0x55c76b1a06e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:57:37.302 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.303+0000 7f9b1d83fb80 1 -- 172.21.15.52:0/621903399 shutdown_connections 2024-06-10T02:57:37.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.307+0000 7f9b1d83fb80 1 -- 172.21.15.52:0/621903399 wait complete. 2024-06-10T02:57:37.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.308+0000 7f9b1d83fb80 20 client.16053 trim_cache size 0 max 16384 2024-06-10T02:57:37.312 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-10T02:57:37.312 INFO:teuthology.orchestra.run.smithi052.stdout: "status": "IDLE (waiting for more scrubs)", 2024-06-10T02:57:37.312 INFO:teuthology.orchestra.run.smithi052.stdout: "scrubs": { 2024-06-10T02:57:37.313 INFO:teuthology.orchestra.run.smithi052.stdout: "3f69b16b-280a-4a2b-870d-9350f1375f80": { 2024-06-10T02:57:37.313 INFO:teuthology.orchestra.run.smithi052.stdout: "path": "/", 2024-06-10T02:57:37.313 INFO:teuthology.orchestra.run.smithi052.stdout: "tag": "3f69b16b-280a-4a2b-870d-9350f1375f80", 2024-06-10T02:57:37.313 INFO:teuthology.orchestra.run.smithi052.stdout: "options": "recursive,force" 2024-06-10T02:57:37.313 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T02:57:37.313 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T02:57:37.313 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-10T02:57:37.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.314+0000 7f9b11d5a700 1 -- 172.21.15.52:0/829496903 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f9af40a2e30 msgr2=0x7f9af40a52f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:57:37.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.314+0000 7f9b11d5a700 1 --2- 172.21.15.52:0/829496903 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f9af40a2e30 0x7f9af40a52f0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f9afc009480 tx=0x7f9afc005040 comp rx=0 tx=0).stop 2024-06-10T02:57:37.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.314+0000 7f9b11d5a700 1 -- 172.21.15.52:0/829496903 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9b0c11bc70 msgr2=0x7f9b0c113470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:57:37.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.314+0000 7f9b11d5a700 1 --2- 172.21.15.52:0/829496903 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9b0c11bc70 0x7f9b0c113470 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f9b0400b450 tx=0x7f9b0400b810 comp rx=0 tx=0).stop 2024-06-10T02:57:37.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.315+0000 7f9b11d5a700 1 -- 172.21.15.52:0/829496903 shutdown_connections 2024-06-10T02:57:37.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.315+0000 7f9b11d5a700 1 --2- 172.21.15.52:0/829496903 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f9af40a2e30 0x7f9af40a52f0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:37.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.315+0000 7f9b11d5a700 1 --2- 172.21.15.52:0/829496903 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9b0c11bc70 0x7f9b0c113470 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:37.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.315+0000 7f9b11d5a700 1 --2- 172.21.15.52:0/829496903 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9b0c110b80 0x7f9b0c11a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:37.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.315+0000 7f9b11d5a700 1 --2- 172.21.15.52:0/829496903 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9b0c110ff0 0x7f9b0c11a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:57:37.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.315+0000 7f9b11d5a700 1 -- 172.21.15.52:0/829496903 >> 172.21.15.52:0/829496903 conn(0x7f9b0c06d470 msgr2=0x7f9b0c070e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:57:37.318 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.319+0000 7f9b11d5a700 1 -- 172.21.15.52:0/829496903 shutdown_connections 2024-06-10T02:57:37.318 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:57:37.319+0000 7f9b11d5a700 1 -- 172.21.15.52:0/829496903 wait complete. 2024-06-10T02:57:38.158 INFO:tasks.cephfs.filesystem:scrub status for tag:3f69b16b-280a-4a2b-870d-9350f1375f80 - {'path': '/', 'tag': '3f69b16b-280a-4a2b-870d-9350f1375f80', 'options': 'recursive,force'} 2024-06-10T02:57:38.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:57:38 smithi188 ceph-mon[36599]: pgmap v537: 129 pgs: 129 active+clean; 234 MiB data, 586 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 7.9 MiB/s wr, 649 op/s 2024-06-10T02:57:38.976 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:57:38 smithi052 ceph-mon[32362]: pgmap v537: 129 pgs: 129 active+clean; 234 MiB data, 586 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 7.9 MiB/s wr, 649 op/s 2024-06-10T02:57:39.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:57:38 smithi120 ceph-mon[36634]: pgmap v537: 129 pgs: 129 active+clean; 234 MiB data, 586 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 7.9 MiB/s wr, 649 op/s 2024-06-10T02:57:40.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:57:40 smithi188 ceph-mon[36599]: pgmap v538: 129 pgs: 129 active+clean; 233 MiB data, 586 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.8 MiB/s wr, 412 op/s 2024-06-10T02:57:40.976 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:57:40 smithi052 ceph-mon[32362]: pgmap v538: 129 pgs: 129 active+clean; 233 MiB data, 586 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.8 MiB/s wr, 412 op/s 2024-06-10T02:57:41.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:57:40 smithi120 ceph-mon[36634]: pgmap v538: 129 pgs: 129 active+clean; 233 MiB data, 586 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.8 MiB/s wr, 412 op/s 2024-06-10T02:57:42.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:57:42 smithi188 ceph-mon[36599]: pgmap v539: 129 pgs: 129 active+clean; 233 MiB data, 585 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.1 MiB/s wr, 211 op/s 2024-06-10T02:57:42.976 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:57:42 smithi052 ceph-mon[32362]: pgmap v539: 129 pgs: 129 active+clean; 233 MiB data, 585 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.1 MiB/s wr, 211 op/s 2024-06-10T02:57:43.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:57:42 smithi120 ceph-mon[36634]: pgmap v539: 129 pgs: 129 active+clean; 233 MiB data, 585 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.1 MiB/s wr, 211 op/s 2024-06-10T02:57:44.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:57:44 smithi188 ceph-mon[36599]: pgmap v540: 129 pgs: 129 active+clean; 241 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.2 MiB/s wr, 154 op/s 2024-06-10T02:57:44.976 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:57:44 smithi052 ceph-mon[32362]: pgmap v540: 129 pgs: 129 active+clean; 241 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.2 MiB/s wr, 154 op/s 2024-06-10T02:57:45.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:57:44 smithi120 ceph-mon[36634]: pgmap v540: 129 pgs: 129 active+clean; 241 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.2 MiB/s wr, 154 op/s 2024-06-10T02:57:46.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:57:46 smithi188 ceph-mon[36599]: pgmap v541: 129 pgs: 129 active+clean; 244 MiB data, 597 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.9 MiB/s wr, 91 op/s 2024-06-10T02:57:46.976 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:57:46 smithi052 ceph-mon[32362]: pgmap v541: 129 pgs: 129 active+clean; 244 MiB data, 597 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.9 MiB/s wr, 91 op/s 2024-06-10T02:57:47.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:57:46 smithi120 ceph-mon[36634]: pgmap v541: 129 pgs: 129 active+clean; 244 MiB data, 597 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.9 MiB/s wr, 91 op/s 2024-06-10T02:57:47.242 INFO:tasks.workunit.client.0.smithi052.stdout: 52 1446452 508 997015 377 792390 1230 2024-06-10T02:57:47.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:57:47 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T02:57:47.976 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:57:47 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T02:57:48.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:57:47 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T02:57:48.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:57:48 smithi188 ceph-mon[36599]: pgmap v542: 129 pgs: 129 active+clean; 265 MiB data, 659 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.8 MiB/s wr, 223 op/s 2024-06-10T02:57:48.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:57:48 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:57:48.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:57:48 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T02:57:48.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:57:48 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:57:48.976 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:57:48 smithi052 ceph-mon[32362]: pgmap v542: 129 pgs: 129 active+clean; 265 MiB data, 659 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.8 MiB/s wr, 223 op/s 2024-06-10T02:57:48.976 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:57:48 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:57:48.977 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:57:48 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T02:57:48.977 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:57:48 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:57:49.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:57:48 smithi120 ceph-mon[36634]: pgmap v542: 129 pgs: 129 active+clean; 265 MiB data, 659 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.8 MiB/s wr, 223 op/s 2024-06-10T02:57:49.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:57:48 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:57:49.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:57:48 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T02:57:49.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:57:48 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:57:50.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:57:50 smithi188 ceph-mon[36599]: pgmap v543: 129 pgs: 129 active+clean; 289 MiB data, 691 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.1 MiB/s wr, 326 op/s 2024-06-10T02:57:50.976 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:57:50 smithi052 ceph-mon[32362]: pgmap v543: 129 pgs: 129 active+clean; 289 MiB data, 691 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.1 MiB/s wr, 326 op/s 2024-06-10T02:57:51.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:57:50 smithi120 ceph-mon[36634]: pgmap v543: 129 pgs: 129 active+clean; 289 MiB data, 691 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.1 MiB/s wr, 326 op/s 2024-06-10T02:57:52.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:57:52 smithi188 ceph-mon[36599]: pgmap v544: 129 pgs: 129 active+clean; 296 MiB data, 695 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.8 MiB/s wr, 367 op/s 2024-06-10T02:57:52.976 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:57:52 smithi052 ceph-mon[32362]: pgmap v544: 129 pgs: 129 active+clean; 296 MiB data, 695 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.8 MiB/s wr, 367 op/s 2024-06-10T02:57:53.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:57:52 smithi120 ceph-mon[36634]: pgmap v544: 129 pgs: 129 active+clean; 296 MiB data, 695 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.8 MiB/s wr, 367 op/s 2024-06-10T02:57:54.954 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:57:54 smithi188 ceph-mon[36599]: pgmap v545: 129 pgs: 129 active+clean; 303 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 6.5 MiB/s wr, 410 op/s 2024-06-10T02:57:54.977 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:57:54 smithi052 ceph-mon[32362]: pgmap v545: 129 pgs: 129 active+clean; 303 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 6.5 MiB/s wr, 410 op/s 2024-06-10T02:57:55.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:57:54 smithi120 ceph-mon[36634]: pgmap v545: 129 pgs: 129 active+clean; 303 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 6.5 MiB/s wr, 410 op/s 2024-06-10T02:57:56.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:57:56 smithi188 ceph-mon[36599]: pgmap v546: 129 pgs: 129 active+clean; 307 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 6.2 MiB/s wr, 408 op/s 2024-06-10T02:57:56.976 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:57:56 smithi052 ceph-mon[32362]: pgmap v546: 129 pgs: 129 active+clean; 307 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 6.2 MiB/s wr, 408 op/s 2024-06-10T02:57:57.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:57:56 smithi120 ceph-mon[36634]: pgmap v546: 129 pgs: 129 active+clean; 307 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 6.2 MiB/s wr, 408 op/s 2024-06-10T02:57:57.243 INFO:tasks.workunit.client.0.smithi052.stdout: 56 1145737 413 775015 247 656074 1335 2024-06-10T02:57:58.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:57:58 smithi188 ceph-mon[36599]: pgmap v547: 129 pgs: 129 active+clean; 309 MiB data, 698 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 6.3 MiB/s wr, 439 op/s 2024-06-10T02:57:58.976 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:57:58 smithi052 ceph-mon[32362]: pgmap v547: 129 pgs: 129 active+clean; 309 MiB data, 698 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 6.3 MiB/s wr, 439 op/s 2024-06-10T02:57:59.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:57:58 smithi120 ceph-mon[36634]: pgmap v547: 129 pgs: 129 active+clean; 309 MiB data, 698 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 6.3 MiB/s wr, 439 op/s 2024-06-10T02:58:00.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:58:00 smithi188 ceph-mon[36599]: pgmap v548: 129 pgs: 129 active+clean; 307 MiB data, 714 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.4 MiB/s wr, 324 op/s 2024-06-10T02:58:00.976 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:58:00 smithi052 ceph-mon[32362]: pgmap v548: 129 pgs: 129 active+clean; 307 MiB data, 714 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.4 MiB/s wr, 324 op/s 2024-06-10T02:58:01.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:58:00 smithi120 ceph-mon[36634]: pgmap v548: 129 pgs: 129 active+clean; 307 MiB data, 714 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.4 MiB/s wr, 324 op/s 2024-06-10T02:58:02.976 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:58:02 smithi052 ceph-mon[32362]: pgmap v549: 129 pgs: 129 active+clean; 310 MiB data, 722 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.6 MiB/s wr, 210 op/s 2024-06-10T02:58:02.977 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:58:02 smithi052 ceph-mon[32362]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-10T02:58:03.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:58:02 smithi120 ceph-mon[36634]: pgmap v549: 129 pgs: 129 active+clean; 310 MiB data, 722 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.6 MiB/s wr, 210 op/s 2024-06-10T02:58:03.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:58:02 smithi120 ceph-mon[36634]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-10T02:58:03.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:58:02 smithi188 ceph-mon[36599]: pgmap v549: 129 pgs: 129 active+clean; 310 MiB data, 722 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.6 MiB/s wr, 210 op/s 2024-06-10T02:58:03.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:58:02 smithi188 ceph-mon[36599]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-10T02:58:04.976 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:58:04 smithi052 ceph-mon[32362]: pgmap v550: 129 pgs: 129 active+clean; 312 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.3 MiB/s wr, 187 op/s 2024-06-10T02:58:05.142 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:58:04 smithi120 ceph-mon[36634]: pgmap v550: 129 pgs: 129 active+clean; 312 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.3 MiB/s wr, 187 op/s 2024-06-10T02:58:05.204 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:58:04 smithi188 ceph-mon[36599]: pgmap v550: 129 pgs: 129 active+clean; 312 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.3 MiB/s wr, 187 op/s 2024-06-10T02:58:06.976 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:58:06 smithi052 ceph-mon[32362]: pgmap v551: 129 pgs: 129 active+clean; 311 MiB data, 723 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 143 op/s 2024-06-10T02:58:07.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:58:06 smithi120 ceph-mon[36634]: pgmap v551: 129 pgs: 129 active+clean; 311 MiB data, 723 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 143 op/s 2024-06-10T02:58:07.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:58:06 smithi188 ceph-mon[36599]: pgmap v551: 129 pgs: 129 active+clean; 311 MiB data, 723 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 143 op/s 2024-06-10T02:58:07.243 INFO:tasks.workunit.client.0.smithi052.stdout: 61 1004962 302 685937 436 560992 1130 2024-06-10T02:58:08.159 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-10T02:58:08.418 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:58:08.817 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:58:08 smithi052 ceph-mon[32362]: pgmap v552: 129 pgs: 129 active+clean; 323 MiB data, 733 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.3 MiB/s wr, 198 op/s 2024-06-10T02:58:09.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:58:08 smithi120 ceph-mon[36634]: pgmap v552: 129 pgs: 129 active+clean; 323 MiB data, 733 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.3 MiB/s wr, 198 op/s 2024-06-10T02:58:09.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:58:08 smithi188 ceph-mon[36599]: pgmap v552: 129 pgs: 129 active+clean; 323 MiB data, 733 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.3 MiB/s wr, 198 op/s 2024-06-10T02:58:10.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.200+0000 7f0ef58a7700 1 -- 172.21.15.52:0/3041746222 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0ef0102ed0 msgr2=0x7f0ef010f3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:58:10.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.200+0000 7f0ef58a7700 1 --2- 172.21.15.52:0/3041746222 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0ef0102ed0 0x7f0ef010f3b0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f0ee800b0a0 tx=0x7f0ee800b3b0 comp rx=0 tx=0).stop 2024-06-10T02:58:10.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.200+0000 7f0ef58a7700 1 -- 172.21.15.52:0/3041746222 shutdown_connections 2024-06-10T02:58:10.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.200+0000 7f0ef58a7700 1 --2- 172.21.15.52:0/3041746222 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0ef0102ed0 0x7f0ef010f3b0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:10.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.200+0000 7f0ef58a7700 1 --2- 172.21.15.52:0/3041746222 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0ef0108620 0x7f0ef0108a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:10.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.200+0000 7f0ef58a7700 1 --2- 172.21.15.52:0/3041746222 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0ef0102510 0x7f0ef0102990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:10.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.200+0000 7f0ef58a7700 1 -- 172.21.15.52:0/3041746222 >> 172.21.15.52:0/3041746222 conn(0x7f0ef0079640 msgr2=0x7f0ef0079a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:58:10.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.200+0000 7f0ef58a7700 1 -- 172.21.15.52:0/3041746222 shutdown_connections 2024-06-10T02:58:10.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.200+0000 7f0ef58a7700 1 -- 172.21.15.52:0/3041746222 wait complete. 2024-06-10T02:58:10.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.201+0000 7f0ef58a7700 1 Processor -- start 2024-06-10T02:58:10.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.201+0000 7f0ef58a7700 1 -- start start 2024-06-10T02:58:10.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.201+0000 7f0ef58a7700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0ef0102510 0x7f0ef019e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:58:10.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.201+0000 7f0ef58a7700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0ef0102ed0 0x7f0ef019edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:58:10.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.201+0000 7f0ef58a7700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0ef0108620 0x7f0ef0198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:58:10.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.201+0000 7f0ef58a7700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f0ef01127d0 con 0x7f0ef0102ed0 2024-06-10T02:58:10.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.201+0000 7f0ef58a7700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f0ef0112650 con 0x7f0ef0108620 2024-06-10T02:58:10.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.201+0000 7f0ef58a7700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f0ef0112950 con 0x7f0ef0102510 2024-06-10T02:58:10.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.201+0000 7f0eeeffd700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0ef0102510 0x7f0ef019e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:58:10.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.202+0000 7f0eeeffd700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0ef0102510 0x7f0ef019e890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.52:51412/0 (socket says 172.21.15.52:51412) 2024-06-10T02:58:10.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.202+0000 7f0eeeffd700 1 -- 172.21.15.52:0/3288571761 learned_addr learned my addr 172.21.15.52:0/3288571761 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:58:10.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.202+0000 7f0eee7fc700 1 --2- 172.21.15.52:0/3288571761 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0ef0102ed0 0x7f0ef019edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:58:10.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.202+0000 7f0eef7fe700 1 --2- 172.21.15.52:0/3288571761 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0ef0108620 0x7f0ef0198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:58:10.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.202+0000 7f0eeeffd700 1 -- 172.21.15.52:0/3288571761 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0ef0108620 msgr2=0x7f0ef0198a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:58:10.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.202+0000 7f0eeeffd700 1 --2- 172.21.15.52:0/3288571761 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0ef0108620 0x7f0ef0198a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:10.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.202+0000 7f0eeeffd700 1 -- 172.21.15.52:0/3288571761 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0ef0102ed0 msgr2=0x7f0ef019edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:58:10.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.202+0000 7f0eeeffd700 1 --2- 172.21.15.52:0/3288571761 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0ef0102ed0 0x7f0ef019edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:10.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.202+0000 7f0eeeffd700 1 -- 172.21.15.52:0/3288571761 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ef01991b0 con 0x7f0ef0102510 2024-06-10T02:58:10.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.202+0000 7f0eee7fc700 1 --2- 172.21.15.52:0/3288571761 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0ef0102ed0 0x7f0ef019edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T02:58:10.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.202+0000 7f0eeeffd700 1 --2- 172.21.15.52:0/3288571761 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0ef0102510 0x7f0ef019e890 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f0ee000d950 tx=0x7f0ee000dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:58:10.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.202+0000 7f0ef48a5700 1 -- 172.21.15.52:0/3288571761 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ee0016070 con 0x7f0ef0102510 2024-06-10T02:58:10.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.203+0000 7f0ef58a7700 1 -- 172.21.15.52:0/3288571761 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ef01994a0 con 0x7f0ef0102510 2024-06-10T02:58:10.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.203+0000 7f0ef48a5700 1 -- 172.21.15.52:0/3288571761 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ee000fd70 con 0x7f0ef0102510 2024-06-10T02:58:10.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.203+0000 7f0ef58a7700 1 -- 172.21.15.52:0/3288571761 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ef01a5760 con 0x7f0ef0102510 2024-06-10T02:58:10.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.203+0000 7f0ef48a5700 1 -- 172.21.15.52:0/3288571761 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ee000e610 con 0x7f0ef0102510 2024-06-10T02:58:10.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.205+0000 7f0ef48a5700 1 -- 172.21.15.52:0/3288571761 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f0ee0019070 con 0x7f0ef0102510 2024-06-10T02:58:10.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.206+0000 7f0ef48a5700 1 --2- 172.21.15.52:0/3288571761 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f0ed80a2c20 0x7f0ed80a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:58:10.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.206+0000 7f0eee7fc700 1 --2- 172.21.15.52:0/3288571761 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f0ed80a2c20 0x7f0ed80a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:58:10.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.206+0000 7f0ef48a5700 1 -- 172.21.15.52:0/3288571761 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f0ee00cf1e0 con 0x7f0ef0102510 2024-06-10T02:58:10.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.207+0000 7f0f0012ab80 1 Processor -- start 2024-06-10T02:58:10.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.207+0000 7f0f0012ab80 1 -- start start 2024-06-10T02:58:10.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.207+0000 7f0f0012ab80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f407439f60 0x55f40743a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:58:10.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.207+0000 7f0f0012ab80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f40743a880 0x55f407434260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:58:10.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.207+0000 7f0f0012ab80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f4074347a0 0x55f407434bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:58:10.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.207+0000 7f0f0012ab80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55f407326f60 con 0x55f40743a880 2024-06-10T02:58:10.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.207+0000 7f0f0012ab80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55f407385850 con 0x55f407439f60 2024-06-10T02:58:10.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.207+0000 7f0f0012ab80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55f4072ed460 con 0x55f4074347a0 2024-06-10T02:58:10.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.207+0000 7f0eee7fc700 1 --2- 172.21.15.52:0/3288571761 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f0ed80a2c20 0x7f0ed80a50e0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f0ef0079b40 tx=0x7f0ee4009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:58:10.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.207+0000 7f0eeeffd700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f407439f60 0x55f40743a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:58:10.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.207+0000 7f0eee7fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f40743a880 0x55f407434260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:58:10.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.207+0000 7f0eeeffd700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f407439f60 0x55f40743a340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.52:54822/0 (socket says 172.21.15.52:54822) 2024-06-10T02:58:10.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.207+0000 7f0eeeffd700 1 -- 172.21.15.52:0/549050051 learned_addr learned my addr 172.21.15.52:0/549050051 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:58:10.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.207+0000 7f0eef7fe700 1 --2- 172.21.15.52:0/549050051 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f4074347a0 0x55f407434bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:58:10.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.207+0000 7f0eeeffd700 1 -- 172.21.15.52:0/549050051 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f4074347a0 msgr2=0x55f407434bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:58:10.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.207+0000 7f0eeeffd700 1 --2- 172.21.15.52:0/549050051 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f4074347a0 0x55f407434bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:10.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.207+0000 7f0eeeffd700 1 -- 172.21.15.52:0/549050051 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f40743a880 msgr2=0x55f407434260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:58:10.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.207+0000 7f0eeeffd700 1 --2- 172.21.15.52:0/549050051 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f40743a880 0x55f407434260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:10.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.207+0000 7f0eeeffd700 1 -- 172.21.15.52:0/549050051 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4072d48d0 con 0x55f407439f60 2024-06-10T02:58:10.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.208+0000 7f0eeeffd700 1 --2- 172.21.15.52:0/549050051 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f407439f60 0x55f40743a340 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f0ee0097af0 tx=0x7f0ee009b0f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7d82acfda64dc469 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:58:10.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.208+0000 7f0ed5ffb700 1 -- 172.21.15.52:0/549050051 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ee0020070 con 0x55f407439f60 2024-06-10T02:58:10.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.208+0000 7f0ed5ffb700 1 -- 172.21.15.52:0/549050051 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ee009b690 con 0x55f407439f60 2024-06-10T02:58:10.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.208+0000 7f0ed5ffb700 1 -- 172.21.15.52:0/549050051 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ee0010460 con 0x55f407439f60 2024-06-10T02:58:10.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.209+0000 7f0f0012ab80 1 -- 172.21.15.52:0/549050051 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f407439f60 msgr2=0x55f40743a340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:58:10.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.209+0000 7f0f0012ab80 1 --2- 172.21.15.52:0/549050051 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f407439f60 0x55f40743a340 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f0ee0097af0 tx=0x7f0ee009b0f0 comp rx=0 tx=0).stop 2024-06-10T02:58:10.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.209+0000 7f0f0012ab80 1 -- 172.21.15.52:0/549050051 shutdown_connections 2024-06-10T02:58:10.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.209+0000 7f0f0012ab80 1 --2- 172.21.15.52:0/549050051 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f4074347a0 0x55f407434bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:10.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.209+0000 7f0f0012ab80 1 --2- 172.21.15.52:0/549050051 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f407439f60 0x55f40743a340 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:10.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.209+0000 7f0f0012ab80 1 --2- 172.21.15.52:0/549050051 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f40743a880 0x55f407434260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:10.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.209+0000 7f0f0012ab80 1 -- 172.21.15.52:0/549050051 >> 172.21.15.52:0/549050051 conn(0x55f407361080 msgr2=0x55f40742f220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:58:10.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.209+0000 7f0f0012ab80 1 -- 172.21.15.52:0/549050051 shutdown_connections 2024-06-10T02:58:10.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.209+0000 7f0f0012ab80 1 -- 172.21.15.52:0/549050051 wait complete. 2024-06-10T02:58:10.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.209+0000 7f0f0012ab80 1 -- start start 2024-06-10T02:58:10.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.209+0000 7f0f0012ab80 1 Processor -- start 2024-06-10T02:58:10.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.209+0000 7f0f0012ab80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f4074347a0 0x55f407440650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:58:10.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.210+0000 7f0f0012ab80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f407439f60 0x55f40743d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:58:10.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.210+0000 7f0f0012ab80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f40743a880 0x55f40743dee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:58:10.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.210+0000 7f0f0012ab80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55f407325fc0 con 0x55f4074347a0 2024-06-10T02:58:10.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.210+0000 7f0f0012ab80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55f407381810 con 0x55f407439f60 2024-06-10T02:58:10.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.210+0000 7f0f0012ab80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55f4074411a0 con 0x55f40743a880 2024-06-10T02:58:10.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.210+0000 7f0f0012ab80 10 client.0 _ll_register_callbacks cb 0x55f40731b5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T02:58:10.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.210+0000 7f0eeeffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f4074347a0 0x55f407440650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:58:10.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.210+0000 7f0eeeffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f4074347a0 0x55f407440650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57310/0 (socket says 172.21.15.52:57310) 2024-06-10T02:58:10.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.210+0000 7f0eeeffd700 1 -- 172.21.15.52:0/1903973017 learned_addr learned my addr 172.21.15.52:0/1903973017 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:58:10.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.210+0000 7f0eef7fe700 1 --2- 172.21.15.52:0/1903973017 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f40743a880 0x55f40743dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:58:10.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.210+0000 7f0eee7fc700 1 --2- 172.21.15.52:0/1903973017 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f407439f60 0x55f40743d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:58:10.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.210+0000 7f0eeeffd700 1 -- 172.21.15.52:0/1903973017 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f40743a880 msgr2=0x55f40743dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:58:10.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.210+0000 7f0eeeffd700 1 --2- 172.21.15.52:0/1903973017 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f40743a880 0x55f40743dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:10.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.210+0000 7f0eeeffd700 1 -- 172.21.15.52:0/1903973017 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f407439f60 msgr2=0x55f40743d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:58:10.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.210+0000 7f0eeeffd700 1 --2- 172.21.15.52:0/1903973017 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f407439f60 0x55f40743d9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:10.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.210+0000 7f0eeeffd700 1 -- 172.21.15.52:0/1903973017 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f40734b750 con 0x55f4074347a0 2024-06-10T02:58:10.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.210+0000 7f0eee7fc700 1 --2- 172.21.15.52:0/1903973017 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f407439f60 0x55f40743d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T02:58:10.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.210+0000 7f0eeeffd700 1 --2- 172.21.15.52:0/1903973017 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f4074347a0 0x55f407440650 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f0ee0012040 tx=0x7f0ee00a43a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21638cd097d60b51 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:58:10.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.210+0000 7f0eceffd700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-10T02:58:10.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.210+0000 7f0eceffd700 1 -- 172.21.15.52:0/1903973017 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ee00b3020 con 0x55f4074347a0 2024-06-10T02:58:10.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.210+0000 7f0eceffd700 1 -- 172.21.15.52:0/1903973017 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f0ee0099990 con 0x55f4074347a0 2024-06-10T02:58:10.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.210+0000 7f0f0012ab80 1 -- 172.21.15.52:0/1903973017 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ee009b690 con 0x55f4074347a0 2024-06-10T02:58:10.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.211+0000 7f0eceffd700 1 -- 172.21.15.52:0/1903973017 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f0ee0020070 con 0x55f4074347a0 2024-06-10T02:58:10.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.211+0000 7f0f0012ab80 10 client.16065 fetch_fsmap learned FSMap version 34 2024-06-10T02:58:10.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.211+0000 7f0f0012ab80 1 -- 172.21.15.52:0/1903973017 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55f40744b9b0 con 0x55f4074347a0 2024-06-10T02:58:10.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.211+0000 7f0eceffd700 1 -- 172.21.15.52:0/1903973017 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ee009b690 con 0x55f4074347a0 2024-06-10T02:58:10.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.211+0000 7f0eceffd700 1 -- 172.21.15.52:0/1903973017 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 34) ==== 8378+0+0 (secure 0 0 0) 0x7f0ee00a3e70 con 0x55f4074347a0 2024-06-10T02:58:10.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.211+0000 7f0f0012ab80 10 client.16065 fetch_fsmap finished waiting for FSMap version 34 2024-06-10T02:58:10.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.211+0000 7f0f0012ab80 10 client.16065 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T02:58:10.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.211+0000 7f0f0012ab80 20 client.16065 populate_metadata read hostname 'smithi052' 2024-06-10T02:58:10.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.211+0000 7f0f0012ab80 1 --2- 172.21.15.52:0/1903973017 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55f40744de80 0x55f407450270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:58:10.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.211+0000 7f0f0012ab80 4 client.16065 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T02:58:10.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.211+0000 7f0f0012ab80 1 -- 172.21.15.52:0/1903973017 --> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f407392d50 con 0x55f40744de80 2024-06-10T02:58:10.217 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.211+0000 7f0eee7fc700 1 --2- 172.21.15.52:0/1903973017 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55f40744de80 0x55f407450270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:58:10.217 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.213+0000 7f0eceffd700 10 client.16065 ms_handle_connect on v2:172.21.15.188:6838/310444836 2024-06-10T02:58:10.217 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.213+0000 7f0eee7fc700 1 --2- 172.21.15.52:0/1903973017 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55f40744de80 0x55f407450270 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=47192f9a70a521b0 server_cookie=ac55070fd333f896 in_seq=0 out_seq=0 2024-06-10T02:58:10.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.215+0000 7f0eceffd700 1 -- 172.21.15.52:0/1903973017 <== mds.0 v2:172.21.15.188:6838/310444836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55f407392d50 con 0x55f40744de80 2024-06-10T02:58:10.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.215+0000 7f0eceffd700 10 client.16065 handle_command_reply: tid=0 2024-06-10T02:58:10.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.215+0000 7f0f0012ab80 1 client.16065 shutdown 2024-06-10T02:58:10.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.215+0000 7f0f0012ab80 1 -- 172.21.15.52:0/1903973017 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f4074347a0 msgr2=0x55f407440650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:58:10.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.215+0000 7f0f0012ab80 1 --2- 172.21.15.52:0/1903973017 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f4074347a0 0x55f407440650 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f0ee0012040 tx=0x7f0ee00a43a0 comp rx=0 tx=0).stop 2024-06-10T02:58:10.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.216+0000 7f0f0012ab80 1 -- 172.21.15.52:0/1903973017 shutdown_connections 2024-06-10T02:58:10.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.216+0000 7f0f0012ab80 1 --2- 172.21.15.52:0/1903973017 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55f40744de80 0x55f407450270 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:10.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.216+0000 7f0f0012ab80 1 --2- 172.21.15.52:0/1903973017 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f40743a880 0x55f40743dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:10.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.216+0000 7f0f0012ab80 1 --2- 172.21.15.52:0/1903973017 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f4074347a0 0x55f407440650 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:10.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.216+0000 7f0eceffd700 0 client.16065 ms_handle_reset on v2:172.21.15.188:6838/310444836 2024-06-10T02:58:10.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.216+0000 7f0f0012ab80 1 --2- 172.21.15.52:0/1903973017 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f407439f60 0x55f40743d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:10.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.216+0000 7f0f0012ab80 1 -- 172.21.15.52:0/1903973017 >> 172.21.15.52:0/1903973017 conn(0x55f407361080 msgr2=0x55f40742e700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:58:10.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.216+0000 7f0f0012ab80 1 -- 172.21.15.52:0/1903973017 shutdown_connections 2024-06-10T02:58:10.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.216+0000 7f0f0012ab80 1 -- 172.21.15.52:0/1903973017 wait complete. 2024-06-10T02:58:10.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.216+0000 7f0f0012ab80 20 client.16065 trim_cache size 0 max 16384 2024-06-10T02:58:10.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.239+0000 7f0f0012ab80 1 Processor -- start 2024-06-10T02:58:10.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.239+0000 7f0f0012ab80 1 -- start start 2024-06-10T02:58:10.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.239+0000 7f0f0012ab80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f4074347a0 0x55f407434260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:58:10.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.239+0000 7f0f0012ab80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f407439f60 0x55f40743df50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:58:10.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.239+0000 7f0f0012ab80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f40743a880 0x55f40743f890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:58:10.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.239+0000 7f0f0012ab80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f0edc00eb40 con 0x55f40743a880 2024-06-10T02:58:10.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.239+0000 7f0f0012ab80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f0edc010740 con 0x55f407439f60 2024-06-10T02:58:10.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.239+0000 7f0f0012ab80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f0edc00f210 con 0x55f4074347a0 2024-06-10T02:58:10.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.239+0000 7f0eef7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f40743a880 0x55f40743f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:58:10.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.239+0000 7f0eef7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f40743a880 0x55f40743f890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57312/0 (socket says 172.21.15.52:57312) 2024-06-10T02:58:10.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.239+0000 7f0eef7fe700 1 -- 172.21.15.52:0/3113258233 learned_addr learned my addr 172.21.15.52:0/3113258233 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:58:10.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.239+0000 7f0eeeffd700 1 --2- 172.21.15.52:0/3113258233 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f4074347a0 0x55f407434260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:58:10.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.240+0000 7f0eee7fc700 1 --2- 172.21.15.52:0/3113258233 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f407439f60 0x55f40743df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:58:10.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.240+0000 7f0eef7fe700 1 -- 172.21.15.52:0/3113258233 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f4074347a0 msgr2=0x55f407434260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:58:10.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.240+0000 7f0eef7fe700 1 --2- 172.21.15.52:0/3113258233 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f4074347a0 0x55f407434260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:10.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.240+0000 7f0eef7fe700 1 -- 172.21.15.52:0/3113258233 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f407439f60 msgr2=0x55f40743df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:58:10.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.240+0000 7f0eef7fe700 1 --2- 172.21.15.52:0/3113258233 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f407439f60 0x55f40743df50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:10.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.240+0000 7f0eef7fe700 1 -- 172.21.15.52:0/3113258233 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f407430be0 con 0x55f40743a880 2024-06-10T02:58:10.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.240+0000 7f0eef7fe700 1 --2- 172.21.15.52:0/3113258233 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f40743a880 0x55f40743f890 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f0ee8009570 tx=0x7f0ee80095a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1ccd06bb3de0524 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:58:10.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.240+0000 7f0ed57fa700 1 -- 172.21.15.52:0/3113258233 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ee800bdb0 con 0x55f40743a880 2024-06-10T02:58:10.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.240+0000 7f0ed57fa700 1 -- 172.21.15.52:0/3113258233 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ee80074e0 con 0x55f40743a880 2024-06-10T02:58:10.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.240+0000 7f0ed57fa700 1 -- 172.21.15.52:0/3113258233 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ee80276c0 con 0x55f40743a880 2024-06-10T02:58:10.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.241+0000 7f0f0012ab80 1 -- 172.21.15.52:0/3113258233 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f40743a880 msgr2=0x55f40743f890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:58:10.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.241+0000 7f0f0012ab80 1 --2- 172.21.15.52:0/3113258233 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f40743a880 0x55f40743f890 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f0ee8009570 tx=0x7f0ee80095a0 comp rx=0 tx=0).stop 2024-06-10T02:58:10.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.241+0000 7f0f0012ab80 1 -- 172.21.15.52:0/3113258233 shutdown_connections 2024-06-10T02:58:10.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.241+0000 7f0f0012ab80 1 --2- 172.21.15.52:0/3113258233 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f40743a880 0x55f40743f890 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:10.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.241+0000 7f0f0012ab80 1 --2- 172.21.15.52:0/3113258233 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f4074347a0 0x55f407434260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:10.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.241+0000 7f0f0012ab80 1 --2- 172.21.15.52:0/3113258233 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f407439f60 0x55f40743df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:10.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.241+0000 7f0f0012ab80 1 -- 172.21.15.52:0/3113258233 >> 172.21.15.52:0/3113258233 conn(0x55f407361080 msgr2=0x55f40743f180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:58:10.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.241+0000 7f0f0012ab80 1 -- 172.21.15.52:0/3113258233 shutdown_connections 2024-06-10T02:58:10.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.241+0000 7f0f0012ab80 1 -- 172.21.15.52:0/3113258233 wait complete. 2024-06-10T02:58:10.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.241+0000 7f0f0012ab80 1 -- start start 2024-06-10T02:58:10.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.242+0000 7f0f0012ab80 1 Processor -- start 2024-06-10T02:58:10.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.242+0000 7f0f0012ab80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f4074347a0 0x55f407453ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:58:10.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.242+0000 7f0f0012ab80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f407439f60 0x55f407450ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:58:10.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.242+0000 7f0f0012ab80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f40743a880 0x55f4074514e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:58:10.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.242+0000 7f0f0012ab80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f0edc010070 con 0x55f407439f60 2024-06-10T02:58:10.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.242+0000 7f0f0012ab80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f0edc00f9a0 con 0x55f40743a880 2024-06-10T02:58:10.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.242+0000 7f0f0012ab80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f0edc00cae0 con 0x55f4074347a0 2024-06-10T02:58:10.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.242+0000 7f0f0012ab80 10 client.0 _ll_register_callbacks cb 0x55f40731b5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T02:58:10.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.242+0000 7f0eee7fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f407439f60 0x55f407450ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:58:10.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.242+0000 7f0eeeffd700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f4074347a0 0x55f407453ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:58:10.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.242+0000 7f0eee7fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f407439f60 0x55f407450ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57314/0 (socket says 172.21.15.52:57314) 2024-06-10T02:58:10.247 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.242+0000 7f0eee7fc700 1 -- 172.21.15.52:0/332947933 learned_addr learned my addr 172.21.15.52:0/332947933 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:58:10.247 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.242+0000 7f0eef7fe700 1 --2- 172.21.15.52:0/332947933 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f40743a880 0x55f4074514e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:58:10.247 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.242+0000 7f0eee7fc700 1 -- 172.21.15.52:0/332947933 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f4074347a0 msgr2=0x55f407453ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:58:10.247 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.242+0000 7f0eee7fc700 1 --2- 172.21.15.52:0/332947933 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f4074347a0 0x55f407453ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:10.247 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.242+0000 7f0eee7fc700 1 -- 172.21.15.52:0/332947933 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f40743a880 msgr2=0x55f4074514e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:58:10.247 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.242+0000 7f0eee7fc700 1 --2- 172.21.15.52:0/332947933 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f40743a880 0x55f4074514e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:10.247 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.242+0000 7f0eee7fc700 1 -- 172.21.15.52:0/332947933 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f40742b810 con 0x55f407439f60 2024-06-10T02:58:10.247 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.243+0000 7f0eee7fc700 1 --2- 172.21.15.52:0/332947933 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f407439f60 0x55f407450ed0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f0ee4013010 tx=0x7f0ee4013040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c974c85bfe4c5dba server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:58:10.247 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.243+0000 7f0eceffd700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-10T02:58:10.247 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.243+0000 7f0eceffd700 1 -- 172.21.15.52:0/332947933 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ee402f030 con 0x55f407439f60 2024-06-10T02:58:10.247 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.243+0000 7f0eceffd700 1 -- 172.21.15.52:0/332947933 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f0ee402a410 con 0x55f407439f60 2024-06-10T02:58:10.247 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.243+0000 7f0f0012ab80 1 -- 172.21.15.52:0/332947933 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ee80074e0 con 0x55f407439f60 2024-06-10T02:58:10.247 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.243+0000 7f0eceffd700 1 -- 172.21.15.52:0/332947933 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f0ee403c050 con 0x55f407439f60 2024-06-10T02:58:10.247 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.243+0000 7f0f0012ab80 10 client.16071 fetch_fsmap learned FSMap version 34 2024-06-10T02:58:10.248 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.243+0000 7f0f0012ab80 1 -- 172.21.15.52:0/332947933 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55f4074b87a0 con 0x55f407439f60 2024-06-10T02:58:10.248 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.243+0000 7f0eceffd700 1 -- 172.21.15.52:0/332947933 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ee80074e0 con 0x55f407439f60 2024-06-10T02:58:10.248 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.243+0000 7f0eceffd700 1 -- 172.21.15.52:0/332947933 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 34) ==== 8378+0+0 (secure 0 0 0) 0x7f0ee4034780 con 0x55f407439f60 2024-06-10T02:58:10.248 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.244+0000 7f0f0012ab80 10 client.16071 fetch_fsmap finished waiting for FSMap version 34 2024-06-10T02:58:10.248 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.244+0000 7f0f0012ab80 10 client.16071 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T02:58:10.248 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.244+0000 7f0f0012ab80 20 client.16071 populate_metadata read hostname 'smithi052' 2024-06-10T02:58:10.248 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.244+0000 7f0f0012ab80 1 --2- 172.21.15.52:0/332947933 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55f4074b8f30 0x55f4074bb350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:58:10.248 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.244+0000 7f0f0012ab80 4 client.16071 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-10T02:58:10.248 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.244+0000 7f0f0012ab80 1 -- 172.21.15.52:0/332947933 --> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f40738d2d0 con 0x55f4074b8f30 2024-06-10T02:58:10.248 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.244+0000 7f0eeeffd700 1 --2- 172.21.15.52:0/332947933 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55f4074b8f30 0x55f4074bb350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:58:10.248 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.244+0000 7f0eeeffd700 1 --2- 172.21.15.52:0/332947933 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55f4074b8f30 0x55f4074bb350 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6673233f88536030 server_cookie=6577f8a1b8ec40c in_seq=0 out_seq=0 2024-06-10T02:58:10.248 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.244+0000 7f0eceffd700 10 client.16071 ms_handle_connect on v2:172.21.15.188:6838/310444836 2024-06-10T02:58:10.249 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.246+0000 7f0eceffd700 1 -- 172.21.15.52:0/332947933 <== mds.0 v2:172.21.15.188:6838/310444836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55f40738d2d0 con 0x55f4074b8f30 2024-06-10T02:58:10.249 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.246+0000 7f0eceffd700 10 client.16071 handle_command_reply: tid=0 2024-06-10T02:58:10.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.247+0000 7f0f0012ab80 1 client.16071 shutdown 2024-06-10T02:58:10.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.247+0000 7f0f0012ab80 1 -- 172.21.15.52:0/332947933 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f407439f60 msgr2=0x55f407450ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:58:10.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.247+0000 7f0f0012ab80 1 --2- 172.21.15.52:0/332947933 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f407439f60 0x55f407450ed0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f0ee4013010 tx=0x7f0ee4013040 comp rx=0 tx=0).stop 2024-06-10T02:58:10.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.247+0000 7f0f0012ab80 1 -- 172.21.15.52:0/332947933 shutdown_connections 2024-06-10T02:58:10.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.247+0000 7f0f0012ab80 1 --2- 172.21.15.52:0/332947933 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55f4074b8f30 0x55f4074bb350 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:10.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.247+0000 7f0f0012ab80 1 --2- 172.21.15.52:0/332947933 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f40743a880 0x55f4074514e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:10.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.247+0000 7f0f0012ab80 1 --2- 172.21.15.52:0/332947933 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f4074347a0 0x55f407453ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:10.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.247+0000 7f0f0012ab80 1 --2- 172.21.15.52:0/332947933 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f407439f60 0x55f407450ed0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:10.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.247+0000 7f0f0012ab80 1 -- 172.21.15.52:0/332947933 >> 172.21.15.52:0/332947933 conn(0x55f407361080 msgr2=0x55f407439350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:58:10.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.247+0000 7f0eceffd700 0 client.16071 ms_handle_reset on v2:172.21.15.188:6838/310444836 2024-06-10T02:58:10.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.247+0000 7f0f0012ab80 1 -- 172.21.15.52:0/332947933 shutdown_connections 2024-06-10T02:58:10.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.247+0000 7f0f0012ab80 1 -- 172.21.15.52:0/332947933 wait complete. 2024-06-10T02:58:10.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.247+0000 7f0f0012ab80 20 client.16071 trim_cache size 0 max 16384 2024-06-10T02:58:10.251 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-10T02:58:10.251 INFO:teuthology.orchestra.run.smithi052.stdout: "status": "IDLE (waiting for more scrubs)", 2024-06-10T02:58:10.251 INFO:teuthology.orchestra.run.smithi052.stdout: "scrubs": { 2024-06-10T02:58:10.251 INFO:teuthology.orchestra.run.smithi052.stdout: "3f69b16b-280a-4a2b-870d-9350f1375f80": { 2024-06-10T02:58:10.251 INFO:teuthology.orchestra.run.smithi052.stdout: "path": "/", 2024-06-10T02:58:10.251 INFO:teuthology.orchestra.run.smithi052.stdout: "tag": "3f69b16b-280a-4a2b-870d-9350f1375f80", 2024-06-10T02:58:10.251 INFO:teuthology.orchestra.run.smithi052.stdout: "options": "recursive,force" 2024-06-10T02:58:10.251 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T02:58:10.251 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T02:58:10.251 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-10T02:58:10.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.247+0000 7f0ef58a7700 1 -- 172.21.15.52:0/3288571761 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f0ed80a2c20 msgr2=0x7f0ed80a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:58:10.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.247+0000 7f0ef58a7700 1 --2- 172.21.15.52:0/3288571761 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f0ed80a2c20 0x7f0ed80a50e0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f0ef0079b40 tx=0x7f0ee4009260 comp rx=0 tx=0).stop 2024-06-10T02:58:10.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.247+0000 7f0ef58a7700 1 -- 172.21.15.52:0/3288571761 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0ef0102510 msgr2=0x7f0ef019e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:58:10.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.247+0000 7f0ef58a7700 1 --2- 172.21.15.52:0/3288571761 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0ef0102510 0x7f0ef019e890 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f0ee000d950 tx=0x7f0ee000dc60 comp rx=0 tx=0).stop 2024-06-10T02:58:10.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.248+0000 7f0ef58a7700 1 -- 172.21.15.52:0/3288571761 shutdown_connections 2024-06-10T02:58:10.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.248+0000 7f0ef58a7700 1 --2- 172.21.15.52:0/3288571761 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0ef0108620 0x7f0ef0198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:10.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.248+0000 7f0ef58a7700 1 --2- 172.21.15.52:0/3288571761 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0ef0102510 0x7f0ef019e890 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:10.253 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.248+0000 7f0ef58a7700 1 --2- 172.21.15.52:0/3288571761 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f0ed80a2c20 0x7f0ed80a50e0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:10.253 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.248+0000 7f0ef58a7700 1 --2- 172.21.15.52:0/3288571761 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0ef0102ed0 0x7f0ef019edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:10.253 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.248+0000 7f0ef58a7700 1 -- 172.21.15.52:0/3288571761 >> 172.21.15.52:0/3288571761 conn(0x7f0ef0079640 msgr2=0x7f0ef0100ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:58:10.253 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.248+0000 7f0ef58a7700 1 -- 172.21.15.52:0/3288571761 shutdown_connections 2024-06-10T02:58:10.253 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:10.248+0000 7f0ef58a7700 1 -- 172.21.15.52:0/3288571761 wait complete. 2024-06-10T02:58:10.883 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:58:10 smithi052 ceph-mon[32362]: pgmap v553: 129 pgs: 129 active+clean; 339 MiB data, 808 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.3 MiB/s wr, 288 op/s 2024-06-10T02:58:10.940 INFO:tasks.cephfs.filesystem:scrub status for tag:3f69b16b-280a-4a2b-870d-9350f1375f80 - {'path': '/', 'tag': '3f69b16b-280a-4a2b-870d-9350f1375f80', 'options': 'recursive,force'} 2024-06-10T02:58:11.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:58:10 smithi120 ceph-mon[36634]: pgmap v553: 129 pgs: 129 active+clean; 339 MiB data, 808 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.3 MiB/s wr, 288 op/s 2024-06-10T02:58:11.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:58:10 smithi188 ceph-mon[36599]: pgmap v553: 129 pgs: 129 active+clean; 339 MiB data, 808 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.3 MiB/s wr, 288 op/s 2024-06-10T02:58:12.976 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:58:12 smithi052 ceph-mon[32362]: pgmap v554: 129 pgs: 129 active+clean; 351 MiB data, 820 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.4 MiB/s wr, 360 op/s 2024-06-10T02:58:13.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:58:12 smithi120 ceph-mon[36634]: pgmap v554: 129 pgs: 129 active+clean; 351 MiB data, 820 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.4 MiB/s wr, 360 op/s 2024-06-10T02:58:13.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:58:12 smithi188 ceph-mon[36599]: pgmap v554: 129 pgs: 129 active+clean; 351 MiB data, 820 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.4 MiB/s wr, 360 op/s 2024-06-10T02:58:13.976 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:58:13 smithi052 ceph-mon[32362]: scrub summary: idle+waiting paths [/] 2024-06-10T02:58:14.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:58:13 smithi120 ceph-mon[36634]: scrub summary: idle+waiting paths [/] 2024-06-10T02:58:14.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:58:13 smithi188 ceph-mon[36599]: scrub summary: idle+waiting paths [/] 2024-06-10T02:58:14.976 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:58:14 smithi052 ceph-mon[32362]: pgmap v555: 129 pgs: 129 active+clean; 368 MiB data, 871 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 6.0 MiB/s wr, 476 op/s 2024-06-10T02:58:15.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:58:14 smithi120 ceph-mon[36634]: pgmap v555: 129 pgs: 129 active+clean; 368 MiB data, 871 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 6.0 MiB/s wr, 476 op/s 2024-06-10T02:58:15.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:58:14 smithi188 ceph-mon[36599]: pgmap v555: 129 pgs: 129 active+clean; 368 MiB data, 871 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 6.0 MiB/s wr, 476 op/s 2024-06-10T02:58:17.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:58:16 smithi120 ceph-mon[36634]: pgmap v556: 129 pgs: 129 active+clean; 369 MiB data, 879 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.8 MiB/s wr, 471 op/s 2024-06-10T02:58:17.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:58:16 smithi188 ceph-mon[36599]: pgmap v556: 129 pgs: 129 active+clean; 369 MiB data, 879 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.8 MiB/s wr, 471 op/s 2024-06-10T02:58:17.226 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:58:16 smithi052 ceph-mon[32362]: pgmap v556: 129 pgs: 129 active+clean; 369 MiB data, 879 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.8 MiB/s wr, 471 op/s 2024-06-10T02:58:17.243 INFO:tasks.workunit.client.0.smithi052.stdout: 64 582542 320 402741 288 322435 1010 2024-06-10T02:58:19.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:58:18 smithi120 ceph-mon[36634]: pgmap v557: 129 pgs: 129 active+clean; 373 MiB data, 893 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 6.4 MiB/s wr, 504 op/s 2024-06-10T02:58:19.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:58:18 smithi188 ceph-mon[36599]: pgmap v557: 129 pgs: 129 active+clean; 373 MiB data, 893 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 6.4 MiB/s wr, 504 op/s 2024-06-10T02:58:19.226 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:58:18 smithi052 ceph-mon[32362]: pgmap v557: 129 pgs: 129 active+clean; 373 MiB data, 893 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 6.4 MiB/s wr, 504 op/s 2024-06-10T02:58:21.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:58:20 smithi120 ceph-mon[36634]: pgmap v558: 129 pgs: 129 active+clean; 371 MiB data, 893 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.2 MiB/s wr, 464 op/s 2024-06-10T02:58:21.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:58:20 smithi188 ceph-mon[36599]: pgmap v558: 129 pgs: 129 active+clean; 371 MiB data, 893 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.2 MiB/s wr, 464 op/s 2024-06-10T02:58:21.226 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:58:20 smithi052 ceph-mon[32362]: pgmap v558: 129 pgs: 129 active+clean; 371 MiB data, 893 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.2 MiB/s wr, 464 op/s 2024-06-10T02:58:22.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:58:21 smithi120 ceph-mon[36634]: scrub summary: idle 2024-06-10T02:58:22.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:58:21 smithi120 ceph-mon[36634]: scrub summary: idle 2024-06-10T02:58:22.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:58:21 smithi120 ceph-mon[36634]: scrub summary: idle 2024-06-10T02:58:22.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:58:21 smithi188 ceph-mon[36599]: scrub summary: idle 2024-06-10T02:58:22.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:58:21 smithi188 ceph-mon[36599]: scrub summary: idle 2024-06-10T02:58:22.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:58:21 smithi188 ceph-mon[36599]: scrub summary: idle 2024-06-10T02:58:22.226 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:58:21 smithi052 ceph-mon[32362]: scrub summary: idle 2024-06-10T02:58:22.226 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:58:21 smithi052 ceph-mon[32362]: scrub summary: idle 2024-06-10T02:58:22.226 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:58:21 smithi052 ceph-mon[32362]: scrub summary: idle 2024-06-10T02:58:22.630 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 18.2 2024-06-10T02:58:22.631 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T02:58:23.105 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:58:22 smithi052 ceph-mon[32362]: pgmap v559: 129 pgs: 129 active+clean; 370 MiB data, 896 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.8 MiB/s wr, 349 op/s 2024-06-10T02:58:23.106 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:58:22 smithi052 ceph-mon[32362]: scrub summary: idle 2024-06-10T02:58:23.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:58:22 smithi120 ceph-mon[36634]: pgmap v559: 129 pgs: 129 active+clean; 370 MiB data, 896 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.8 MiB/s wr, 349 op/s 2024-06-10T02:58:23.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:58:22 smithi120 ceph-mon[36634]: scrub summary: idle 2024-06-10T02:58:23.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:58:22 smithi188 ceph-mon[36599]: pgmap v559: 129 pgs: 129 active+clean; 370 MiB data, 896 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.8 MiB/s wr, 349 op/s 2024-06-10T02:58:23.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:58:22 smithi188 ceph-mon[36599]: scrub summary: idle 2024-06-10T02:58:24.496 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:58:24.970 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:58:24 smithi052 ceph-mon[32362]: pgmap v560: 129 pgs: 129 active+clean; 375 MiB data, 883 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.3 MiB/s wr, 290 op/s 2024-06-10T02:58:25.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:58:24 smithi120 ceph-mon[36634]: pgmap v560: 129 pgs: 129 active+clean; 375 MiB data, 883 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.3 MiB/s wr, 290 op/s 2024-06-10T02:58:25.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:58:24 smithi188 ceph-mon[36599]: pgmap v560: 129 pgs: 129 active+clean; 375 MiB data, 883 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.3 MiB/s wr, 290 op/s 2024-06-10T02:58:27.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:58:26 smithi120 ceph-mon[36634]: pgmap v561: 129 pgs: 129 active+clean; 374 MiB data, 871 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.4 MiB/s wr, 166 op/s 2024-06-10T02:58:27.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:58:26 smithi188 ceph-mon[36599]: pgmap v561: 129 pgs: 129 active+clean; 374 MiB data, 871 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.4 MiB/s wr, 166 op/s 2024-06-10T02:58:27.231 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:58:26 smithi052 ceph-mon[32362]: pgmap v561: 129 pgs: 129 active+clean; 374 MiB data, 871 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.4 MiB/s wr, 166 op/s 2024-06-10T02:58:27.246 INFO:tasks.workunit.client.0.smithi052.stdout: 72 1678374 671 1161609 404 938597 1112 2024-06-10T02:58:28.591 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:28.590+0000 7ff706081700 1 -- 172.21.15.52:0/1952572985 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff700106600 msgr2=0x7ff70010e1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:58:28.591 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:28.590+0000 7ff706081700 1 --2- 172.21.15.52:0/1952572985 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff700106600 0x7ff70010e1d0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff6f8008480 tx=0x7ff6f8008790 comp rx=0 tx=0).stop 2024-06-10T02:58:28.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:28.594+0000 7ff706081700 1 -- 172.21.15.52:0/1952572985 shutdown_connections 2024-06-10T02:58:28.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:28.594+0000 7ff706081700 1 --2- 172.21.15.52:0/1952572985 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff700106600 0x7ff70010e1d0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:28.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:28.594+0000 7ff706081700 1 --2- 172.21.15.52:0/1952572985 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff700101520 0x7ff700101900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:28.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:28.594+0000 7ff706081700 1 --2- 172.21.15.52:0/1952572985 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff700101ed0 0x7ff700105f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:28.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:28.594+0000 7ff706081700 1 -- 172.21.15.52:0/1952572985 >> 172.21.15.52:0/1952572985 conn(0x7ff7000fcdb0 msgr2=0x7ff7000ff1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:58:28.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:28.598+0000 7ff706081700 1 -- 172.21.15.52:0/1952572985 shutdown_connections 2024-06-10T02:58:28.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:28.598+0000 7ff706081700 1 -- 172.21.15.52:0/1952572985 wait complete. 2024-06-10T02:58:28.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:28.599+0000 7ff706081700 1 Processor -- start 2024-06-10T02:58:28.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:28.599+0000 7ff706081700 1 -- start start 2024-06-10T02:58:28.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:28.599+0000 7ff706081700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff700101520 0x7ff70019c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:58:28.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:28.599+0000 7ff706081700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff700101ed0 0x7ff70019cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:58:28.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:28.599+0000 7ff706081700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff700106600 0x7ff700196740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:58:28.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:28.599+0000 7ff706081700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ff7001105a0 con 0x7ff700101ed0 2024-06-10T02:58:28.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:28.599+0000 7ff706081700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff700110420 con 0x7ff700101520 2024-06-10T02:58:28.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:28.599+0000 7ff706081700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ff700110720 con 0x7ff700106600 2024-06-10T02:58:28.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:28.602+0000 7ff705880700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff700106600 0x7ff700196740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:58:28.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:28.602+0000 7ff705880700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff700106600 0x7ff700196740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.52:50650/0 (socket says 172.21.15.52:50650) 2024-06-10T02:58:28.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:28.602+0000 7ff705880700 1 -- 172.21.15.52:0/324281901 learned_addr learned my addr 172.21.15.52:0/324281901 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:58:28.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:28.602+0000 7ff70507f700 1 --2- 172.21.15.52:0/324281901 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff700101520 0x7ff70019c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:58:28.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:28.602+0000 7ff705880700 1 -- 172.21.15.52:0/324281901 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff700101520 msgr2=0x7ff70019c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:58:28.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:28.602+0000 7ff705880700 1 --2- 172.21.15.52:0/324281901 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff700101520 0x7ff70019c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:28.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:28.602+0000 7ff705880700 1 -- 172.21.15.52:0/324281901 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff700101ed0 msgr2=0x7ff70019cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:58:28.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:28.602+0000 7ff705880700 1 --2- 172.21.15.52:0/324281901 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff700101ed0 0x7ff70019cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:28.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:28.602+0000 7ff705880700 1 -- 172.21.15.52:0/324281901 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff700196f50 con 0x7ff700106600 2024-06-10T02:58:28.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:28.603+0000 7ff705880700 1 --2- 172.21.15.52:0/324281901 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff700106600 0x7ff700196740 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ff6f80095a0 tx=0x7ff6f80173e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:58:28.611 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:28.607+0000 7ff6f67fc700 1 -- 172.21.15.52:0/324281901 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6f800b2e0 con 0x7ff700106600 2024-06-10T02:58:28.611 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:28.607+0000 7ff6f67fc700 1 -- 172.21.15.52:0/324281901 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff6f8009710 con 0x7ff700106600 2024-06-10T02:58:28.611 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:28.607+0000 7ff6f67fc700 1 -- 172.21.15.52:0/324281901 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6f8017ba0 con 0x7ff700106600 2024-06-10T02:58:28.611 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:28.610+0000 7ff706081700 1 -- 172.21.15.52:0/324281901 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7001971e0 con 0x7ff700106600 2024-06-10T02:58:28.611 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:28.610+0000 7ff706081700 1 -- 172.21.15.52:0/324281901 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff70006dec0 con 0x7ff700106600 2024-06-10T02:58:28.616 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:28.616+0000 7ff6ebfff700 1 -- 172.21.15.52:0/324281901 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff700102a00 con 0x7ff700106600 2024-06-10T02:58:28.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:28.624+0000 7ff6f67fc700 1 -- 172.21.15.52:0/324281901 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7ff6f80098c0 con 0x7ff700106600 2024-06-10T02:58:28.625 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:28.625+0000 7ff6f67fc700 1 --2- 172.21.15.52:0/324281901 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7ff6ec0a2df0 0x7ff6ec0a52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:58:28.625 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:28.625+0000 7ff6f67fc700 1 -- 172.21.15.52:0/324281901 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7ff6f80cfd50 con 0x7ff700106600 2024-06-10T02:58:28.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:28.629+0000 7ff6f67fc700 1 -- 172.21.15.52:0/324281901 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7ff6f809c930 con 0x7ff700106600 2024-06-10T02:58:28.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:28.630+0000 7ff70507f700 1 --2- 172.21.15.52:0/324281901 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7ff6ec0a2df0 0x7ff6ec0a52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:58:28.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:28.632+0000 7ff70507f700 1 --2- 172.21.15.52:0/324281901 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7ff6ec0a2df0 0x7ff6ec0a52b0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7ff6fc005fd0 tx=0x7ff6fc009310 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:58:28.982 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:58:28 smithi052 ceph-mon[32362]: pgmap v562: 129 pgs: 129 active+clean; 382 MiB data, 873 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 187 op/s 2024-06-10T02:58:29.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:58:28 smithi120 ceph-mon[36634]: pgmap v562: 129 pgs: 129 active+clean; 382 MiB data, 873 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 187 op/s 2024-06-10T02:58:29.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:29.185+0000 7ff6ebfff700 1 -- 172.21.15.52:0/324281901 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff7001975e0 con 0x7ff700106600 2024-06-10T02:58:29.189 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-10T02:58:29.189 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/889673247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":889673247},{"type":"v1","addr":"172.21.15.52:6835","nonce":889673247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/3615663511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":3615663511},{"type":"v1","addr":"172.21.15.52:6837","nonce":3615663511}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T02:50:03.889766+0000","modified":"2024-06-10T02:58:01.195103+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24484,"mds_1":24469,"mds_2":24463,"mds_3":24449,"mds_4":24428},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14571":{"gid":14571,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6839/3059599042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":3059599042},{"type":"v1","addr":"172.21.15.52:6839","nonce":3059599042}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14583":{"gid":14583,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6841/3039275521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":3039275521},{"type":"v1","addr":"172.21.15.52:6841","nonce":3039275521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24383":{"gid":24383,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6833/2609002896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":2609002896},{"type":"v1","addr":"172.21.15.120:6833","nonce":2609002896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6835/1996370288","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":1996370288},{"type":"v1","addr":"172.21.15.120:6835","nonce":1996370288}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6833/529564979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":529564979},{"type":"v1","addr":"172.21.15.188:6833","nonce":529564979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":25,"addr":"172.21.15.120:6837/4132440952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":4132440952},{"type":"v1","addr":"172.21.15.120:6837","nonce":4132440952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"b","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.120:6839/1672436487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1672436487},{"type":"v1","addr":"172.21.15.120:6839","nonce":1672436487}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":2,"incarnation":22,"state":"up:active","state_seq":26,"addr":"172.21.15.188:6835/3904544259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":3904544259},{"type":"v1","addr":"172.21.15.188:6835","nonce":3904544259}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.188:6837/819317726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":819317726},{"type":"v1","addr":"172.21.15.188:6837","nonce":819317726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.188:6839/310444836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":310444836},{"type":"v1","addr":"172.21.15.188:6839","nonce":310444836}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24484,"qdb_cluster":[24469,24484,24463,24428,24449]},"id":1}]} 2024-06-10T02:58:29.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:29.186+0000 7ff6f67fc700 1 -- 172.21.15.52:0/324281901 <== mon.1 v2:172.21.15.188:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11945 (secure 0 0 0) 0x7ff6f809c230 con 0x7ff700106600 2024-06-10T02:58:29.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:29.192+0000 7ff6ebfff700 1 -- 172.21.15.52:0/324281901 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7ff6ec0a2df0 msgr2=0x7ff6ec0a52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:58:29.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:29.192+0000 7ff6ebfff700 1 --2- 172.21.15.52:0/324281901 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7ff6ec0a2df0 0x7ff6ec0a52b0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7ff6fc005fd0 tx=0x7ff6fc009310 comp rx=0 tx=0).stop 2024-06-10T02:58:29.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:29.192+0000 7ff6ebfff700 1 -- 172.21.15.52:0/324281901 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff700106600 msgr2=0x7ff700196740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:58:29.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:29.192+0000 7ff6ebfff700 1 --2- 172.21.15.52:0/324281901 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff700106600 0x7ff700196740 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ff6f80095a0 tx=0x7ff6f80173e0 comp rx=0 tx=0).stop 2024-06-10T02:58:29.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:29.198+0000 7ff6ebfff700 1 -- 172.21.15.52:0/324281901 shutdown_connections 2024-06-10T02:58:29.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:29.198+0000 7ff6ebfff700 1 --2- 172.21.15.52:0/324281901 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff700106600 0x7ff700196740 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:29.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:29.198+0000 7ff6ebfff700 1 --2- 172.21.15.52:0/324281901 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff700101520 0x7ff70019c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:29.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:29.198+0000 7ff6ebfff700 1 --2- 172.21.15.52:0/324281901 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7ff6ec0a2df0 0x7ff6ec0a52b0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:29.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:29.198+0000 7ff6ebfff700 1 --2- 172.21.15.52:0/324281901 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff700101ed0 0x7ff70019cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:29.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:29.198+0000 7ff6ebfff700 1 -- 172.21.15.52:0/324281901 >> 172.21.15.52:0/324281901 conn(0x7ff7000fcdb0 msgr2=0x7ff7000fd8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:58:29.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:58:28 smithi188 ceph-mon[36599]: pgmap v562: 129 pgs: 129 active+clean; 382 MiB data, 873 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 187 op/s 2024-06-10T02:58:29.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:29.203+0000 7ff6ebfff700 1 -- 172.21.15.52:0/324281901 shutdown_connections 2024-06-10T02:58:29.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:29.203+0000 7ff6ebfff700 1 -- 172.21.15.52:0/324281901 wait complete. 2024-06-10T02:58:29.207 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 34 2024-06-10T02:58:30.122 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:58:29 smithi052 ceph-mon[32362]: from='client.? 172.21.15.52:0/324281901' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T02:58:30.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:58:29 smithi120 ceph-mon[36634]: from='client.? 172.21.15.52:0/324281901' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T02:58:30.199 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24484 quiesce db / --timeout 90 --expiration 138.2 --await-for 45.0' 2024-06-10T02:58:30.199 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24484 quiesce db / --timeout 90 --expiration 138.2 --await-for 45.0 2024-06-10T02:58:30.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:58:29 smithi188 ceph-mon[36599]: from='client.? 172.21.15.52:0/324281901' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T02:58:30.488 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:58:30.941 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:58:30 smithi052 ceph-mon[32362]: pgmap v563: 129 pgs: 129 active+clean; 400 MiB data, 892 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.1 MiB/s wr, 255 op/s 2024-06-10T02:58:31.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:58:30 smithi120 ceph-mon[36634]: pgmap v563: 129 pgs: 129 active+clean; 400 MiB data, 892 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.1 MiB/s wr, 255 op/s 2024-06-10T02:58:31.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:58:30 smithi188 ceph-mon[36599]: pgmap v563: 129 pgs: 129 active+clean; 400 MiB data, 892 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.1 MiB/s wr, 255 op/s 2024-06-10T02:58:32.748 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.746+0000 7f5fb58d8700 1 -- 172.21.15.52:0/3992061944 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5fb01099d0 msgr2=0x7f5fb0109e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:58:32.748 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.746+0000 7f5fb58d8700 1 --2- 172.21.15.52:0/3992061944 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5fb01099d0 0x7f5fb0109e50 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f5fa801c0d0 tx=0x7f5fa801c3e0 comp rx=0 tx=0).stop 2024-06-10T02:58:32.748 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.746+0000 7f5fb58d8700 1 -- 172.21.15.52:0/3992061944 shutdown_connections 2024-06-10T02:58:32.749 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.746+0000 7f5fb58d8700 1 --2- 172.21.15.52:0/3992061944 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5fb011bc70 0x7f5fb011e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:32.749 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.746+0000 7f5fb58d8700 1 --2- 172.21.15.52:0/3992061944 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5fb0110ce0 0x7f5fb0109490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:32.749 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.746+0000 7f5fb58d8700 1 --2- 172.21.15.52:0/3992061944 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5fb01099d0 0x7f5fb0109e50 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:32.749 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.746+0000 7f5fb58d8700 1 -- 172.21.15.52:0/3992061944 >> 172.21.15.52:0/3992061944 conn(0x7f5fb006d600 msgr2=0x7f5fb006da10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:58:32.749 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.748+0000 7f5fb58d8700 1 -- 172.21.15.52:0/3992061944 shutdown_connections 2024-06-10T02:58:32.750 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.748+0000 7f5fb58d8700 1 -- 172.21.15.52:0/3992061944 wait complete. 2024-06-10T02:58:32.750 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.749+0000 7f5fb58d8700 1 Processor -- start 2024-06-10T02:58:32.750 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.749+0000 7f5fb58d8700 1 -- start start 2024-06-10T02:58:32.750 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.749+0000 7f5fb58d8700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5fb01099d0 0x7f5fb01a3d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:58:32.750 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.749+0000 7f5fb58d8700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5fb0110ce0 0x7f5fb01a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:58:32.750 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.750+0000 7f5fb58d8700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5fb011bc70 0x7f5fb01172e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:58:32.750 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.750+0000 7f5fb58d8700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f5fb0113d70 con 0x7f5fb01099d0 2024-06-10T02:58:32.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.750+0000 7f5fb58d8700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f5fb0113bf0 con 0x7f5fb0110ce0 2024-06-10T02:58:32.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.750+0000 7f5fb58d8700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f5fb0113ef0 con 0x7f5fb011bc70 2024-06-10T02:58:32.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.750+0000 7f5fae7fc700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5fb0110ce0 0x7f5fb01a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:58:32.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.750+0000 7f5fae7fc700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5fb0110ce0 0x7f5fb01a4240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.52:54670/0 (socket says 172.21.15.52:54670) 2024-06-10T02:58:32.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.750+0000 7f5fae7fc700 1 -- 172.21.15.52:0/2677639084 learned_addr learned my addr 172.21.15.52:0/2677639084 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:58:32.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.750+0000 7f5faf7fe700 1 --2- 172.21.15.52:0/2677639084 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5fb011bc70 0x7f5fb01172e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:58:32.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.750+0000 7f5faeffd700 1 --2- 172.21.15.52:0/2677639084 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5fb01099d0 0x7f5fb01a3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:58:32.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.750+0000 7f5fae7fc700 1 -- 172.21.15.52:0/2677639084 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5fb011bc70 msgr2=0x7f5fb01172e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:58:32.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.750+0000 7f5fae7fc700 1 --2- 172.21.15.52:0/2677639084 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5fb011bc70 0x7f5fb01172e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:32.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.750+0000 7f5fae7fc700 1 -- 172.21.15.52:0/2677639084 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5fb01099d0 msgr2=0x7f5fb01a3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:58:32.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.750+0000 7f5fae7fc700 1 --2- 172.21.15.52:0/2677639084 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5fb01099d0 0x7f5fb01a3d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:32.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.750+0000 7f5fae7fc700 1 -- 172.21.15.52:0/2677639084 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fb0117a40 con 0x7f5fb0110ce0 2024-06-10T02:58:32.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.751+0000 7f5fae7fc700 1 --2- 172.21.15.52:0/2677639084 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5fb0110ce0 0x7f5fb01a4240 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f5fa801abd0 tx=0x7f5fa80198c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:58:32.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.751+0000 7f5fb48d6700 1 -- 172.21.15.52:0/2677639084 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fa8030030 con 0x7f5fb0110ce0 2024-06-10T02:58:32.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.751+0000 7f5fb48d6700 1 -- 172.21.15.52:0/2677639084 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5fa8019da0 con 0x7f5fb0110ce0 2024-06-10T02:58:32.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.751+0000 7f5fb48d6700 1 -- 172.21.15.52:0/2677639084 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fa802c8f0 con 0x7f5fb0110ce0 2024-06-10T02:58:32.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.751+0000 7f5fb58d8700 1 -- 172.21.15.52:0/2677639084 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fb0117cd0 con 0x7f5fb0110ce0 2024-06-10T02:58:32.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.752+0000 7f5fb58d8700 1 -- 172.21.15.52:0/2677639084 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fb01180a0 con 0x7f5fb0110ce0 2024-06-10T02:58:32.765 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.760+0000 7f5fc015bb80 1 Processor -- start 2024-06-10T02:58:32.765 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.760+0000 7f5fb48d6700 1 -- 172.21.15.52:0/2677639084 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f5fa8033030 con 0x7f5fb0110ce0 2024-06-10T02:58:32.765 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.761+0000 7f5fc015bb80 1 -- start start 2024-06-10T02:58:32.765 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.762+0000 7f5fc015bb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fdbf8e7cd0 0x55fdbf8e80b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:58:32.765 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.762+0000 7f5fc015bb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fdbf8e85f0 0x55fdbf8e1f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:58:32.766 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.762+0000 7f5fc015bb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fdbf8e2460 0x55fdbf8e28a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:58:32.766 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.762+0000 7f5fc015bb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55fdbf8339a0 con 0x55fdbf8e2460 2024-06-10T02:58:32.766 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.762+0000 7f5fc015bb80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55fdbf7db8b0 con 0x55fdbf8e7cd0 2024-06-10T02:58:32.766 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.762+0000 7f5faf7fe700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fdbf8e7cd0 0x55fdbf8e80b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:58:32.766 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.762+0000 7f5fc015bb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55fdbf82f960 con 0x55fdbf8e85f0 2024-06-10T02:58:32.766 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.762+0000 7f5fae7fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fdbf8e2460 0x55fdbf8e28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:58:32.766 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.762+0000 7f5faf7fe700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fdbf8e7cd0 0x55fdbf8e80b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.52:54686/0 (socket says 172.21.15.52:54686) 2024-06-10T02:58:32.766 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.762+0000 7f5faf7fe700 1 -- 172.21.15.52:0/3188689372 learned_addr learned my addr 172.21.15.52:0/3188689372 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:58:32.766 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.762+0000 7f5faeffd700 1 --2- 172.21.15.52:0/3188689372 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fdbf8e85f0 0x55fdbf8e1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:58:32.766 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.762+0000 7f5fae7fc700 1 -- 172.21.15.52:0/3188689372 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fdbf8e85f0 msgr2=0x55fdbf8e1f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:58:32.766 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.762+0000 7f5fae7fc700 1 --2- 172.21.15.52:0/3188689372 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fdbf8e85f0 0x55fdbf8e1f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:32.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.762+0000 7f5fae7fc700 1 -- 172.21.15.52:0/3188689372 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fdbf8e7cd0 msgr2=0x55fdbf8e80b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:58:32.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.762+0000 7f5fae7fc700 1 --2- 172.21.15.52:0/3188689372 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fdbf8e7cd0 0x55fdbf8e80b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:32.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.762+0000 7f5fae7fc700 1 -- 172.21.15.52:0/3188689372 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fdbf7d20e0 con 0x55fdbf8e2460 2024-06-10T02:58:32.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.763+0000 7f5faf7fe700 1 --2- 172.21.15.52:0/3188689372 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fdbf8e7cd0 0x55fdbf8e80b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T02:58:32.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.763+0000 7f5fb48d6700 1 --2- 172.21.15.52:0/2677639084 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f5f9c0a2eb0 0x7f5f9c0a5370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:58:32.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.763+0000 7f5fae7fc700 1 --2- 172.21.15.52:0/3188689372 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fdbf8e2460 0x55fdbf8e28a0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f5fa80de8e0 tx=0x7f5fa8009550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42226bdfbb27d78a server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:58:32.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.763+0000 7f5faf7fe700 1 --2- 172.21.15.52:0/2677639084 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f5f9c0a2eb0 0x7f5f9c0a5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:58:32.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.764+0000 7f5faf7fe700 1 --2- 172.21.15.52:0/2677639084 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f5f9c0a2eb0 0x7f5f9c0a5370 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f5fa000dfc0 tx=0x7f5fa000d520 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:58:32.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.764+0000 7f5fb48d6700 1 -- 172.21.15.52:0/2677639084 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f5fa8020020 con 0x7f5fb0110ce0 2024-06-10T02:58:32.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.765+0000 7f5fa5ffb700 1 -- 172.21.15.52:0/3188689372 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fa80ad090 con 0x55fdbf8e2460 2024-06-10T02:58:32.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.765+0000 7f5fa5ffb700 1 -- 172.21.15.52:0/3188689372 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5fa803c300 con 0x55fdbf8e2460 2024-06-10T02:58:32.771 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.768+0000 7f5fc015bb80 1 -- 172.21.15.52:0/3188689372 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fdbf8e2460 msgr2=0x55fdbf8e28a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:58:32.772 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.768+0000 7f5fc015bb80 1 --2- 172.21.15.52:0/3188689372 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fdbf8e2460 0x55fdbf8e28a0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f5fa80de8e0 tx=0x7f5fa8009550 comp rx=0 tx=0).stop 2024-06-10T02:58:32.772 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.771+0000 7f5fc015bb80 1 -- 172.21.15.52:0/3188689372 shutdown_connections 2024-06-10T02:58:32.772 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.771+0000 7f5fc015bb80 1 --2- 172.21.15.52:0/3188689372 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fdbf8e2460 0x55fdbf8e28a0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:32.772 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.771+0000 7f5fc015bb80 1 --2- 172.21.15.52:0/3188689372 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fdbf8e7cd0 0x55fdbf8e80b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:32.772 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.771+0000 7f5fc015bb80 1 --2- 172.21.15.52:0/3188689372 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fdbf8e85f0 0x55fdbf8e1f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:32.772 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.771+0000 7f5fc015bb80 1 -- 172.21.15.52:0/3188689372 >> 172.21.15.52:0/3188689372 conn(0x55fdbf80ce50 msgr2=0x55fdbf8dd1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:58:32.774 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.773+0000 7f5fc015bb80 1 -- 172.21.15.52:0/3188689372 shutdown_connections 2024-06-10T02:58:32.774 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.773+0000 7f5fc015bb80 1 -- 172.21.15.52:0/3188689372 wait complete. 2024-06-10T02:58:32.774 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.774+0000 7f5fc015bb80 1 -- start start 2024-06-10T02:58:32.774 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.775+0000 7f5fc015bb80 1 Processor -- start 2024-06-10T02:58:32.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.775+0000 7f5fc015bb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fdbf8e2460 0x55fdbf8ee210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:58:32.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.775+0000 7f5fc015bb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fdbf8e7cd0 0x55fdbf8eb580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:58:32.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.776+0000 7f5fc015bb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fdbf8e85f0 0x55fdbf8ebac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:58:32.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.776+0000 7f5fae7fc700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fdbf8e7cd0 0x55fdbf8eb580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:58:32.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.776+0000 7f5fc015bb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55fdbf910d90 con 0x55fdbf8e2460 2024-06-10T02:58:32.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.776+0000 7f5fc015bb80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55fdbf7d3d80 con 0x55fdbf8e7cd0 2024-06-10T02:58:32.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.776+0000 7f5fc015bb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55fdbf910f10 con 0x55fdbf8e85f0 2024-06-10T02:58:32.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.776+0000 7f5fae7fc700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fdbf8e7cd0 0x55fdbf8eb580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.52:54688/0 (socket says 172.21.15.52:54688) 2024-06-10T02:58:32.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.776+0000 7f5fae7fc700 1 -- 172.21.15.52:0/92604221 learned_addr learned my addr 172.21.15.52:0/92604221 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:58:32.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.776+0000 7f5fc015bb80 10 client.0 _ll_register_callbacks cb 0x55fdbf7cdab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T02:58:32.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.776+0000 7f5faf7fe700 1 --2- 172.21.15.52:0/92604221 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fdbf8e85f0 0x55fdbf8ebac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:58:32.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.776+0000 7f5faf7fe700 1 -- 172.21.15.52:0/92604221 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fdbf8e7cd0 msgr2=0x55fdbf8eb580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:58:32.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.776+0000 7f5faf7fe700 1 --2- 172.21.15.52:0/92604221 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fdbf8e7cd0 0x55fdbf8eb580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:32.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.776+0000 7f5faf7fe700 1 -- 172.21.15.52:0/92604221 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fdbf8e2460 msgr2=0x55fdbf8ee210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:58:32.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.776+0000 7f5faf7fe700 1 --2- 172.21.15.52:0/92604221 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fdbf8e2460 0x55fdbf8ee210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:32.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.776+0000 7f5faf7fe700 1 -- 172.21.15.52:0/92604221 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fdbf782a60 con 0x55fdbf8e85f0 2024-06-10T02:58:32.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.777+0000 7f5faf7fe700 1 --2- 172.21.15.52:0/92604221 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fdbf8e85f0 0x55fdbf8ebac0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f5fa0012b70 tx=0x7f5fa0004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b02d02ab756349fa server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:58:32.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.777+0000 7f5fae7fc700 1 --2- 172.21.15.52:0/92604221 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fdbf8e7cd0 0x55fdbf8eb580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T02:58:32.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.778+0000 7f5f8effd700 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-06-10T02:58:32.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.778+0000 7f5f8effd700 1 -- 172.21.15.52:0/92604221 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fa002a060 con 0x55fdbf8e85f0 2024-06-10T02:58:32.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.778+0000 7f5fc015bb80 1 -- 172.21.15.52:0/92604221 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5fa803c300 con 0x55fdbf8e85f0 2024-06-10T02:58:32.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.778+0000 7f5f8effd700 1 -- 172.21.15.52:0/92604221 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f5fa0010820 con 0x55fdbf8e85f0 2024-06-10T02:58:32.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.778+0000 7f5f8effd700 1 -- 172.21.15.52:0/92604221 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f5fa0031050 con 0x55fdbf8e85f0 2024-06-10T02:58:32.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.778+0000 7f5fc015bb80 10 client.25657 fetch_fsmap learned FSMap version 34 2024-06-10T02:58:32.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.778+0000 7f5fc015bb80 1 -- 172.21.15.52:0/92604221 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55fdbf91b490 con 0x55fdbf8e85f0 2024-06-10T02:58:32.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.779+0000 7f5f8effd700 1 -- 172.21.15.52:0/92604221 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5fa803c300 con 0x55fdbf8e85f0 2024-06-10T02:58:32.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.779+0000 7f5f8effd700 1 -- 172.21.15.52:0/92604221 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 34) ==== 8378+0+0 (secure 0 0 0) 0x7f5fa0019040 con 0x55fdbf8e85f0 2024-06-10T02:58:32.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.779+0000 7f5fc015bb80 10 client.25657 fetch_fsmap finished waiting for FSMap version 34 2024-06-10T02:58:32.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.779+0000 7f5fc015bb80 10 client.25657 resolve_mds: validated gid 24484 aka daemon mds.i 2024-06-10T02:58:32.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.779+0000 7f5fc015bb80 20 client.25657 populate_metadata read hostname 'smithi052' 2024-06-10T02:58:32.781 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.779+0000 7f5fc015bb80 1 --2- 172.21.15.52:0/92604221 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55fdbf91df00 0x55fdbf920380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:58:32.781 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.779+0000 7f5fc015bb80 4 client.25657 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T02:58:32.781 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.779+0000 7f5fc015bb80 1 -- 172.21.15.52:0/92604221 --> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fdbf840ea0 con 0x55fdbf91df00 2024-06-10T02:58:32.781 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.781+0000 7f5faeffd700 1 --2- 172.21.15.52:0/92604221 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55fdbf91df00 0x55fdbf920380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:58:32.783 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.783+0000 7f5faeffd700 1 --2- 172.21.15.52:0/92604221 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55fdbf91df00 0x55fdbf920380 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d77f48a6ecfbe2cf server_cookie=4aa863b4bfb382da in_seq=0 out_seq=0 2024-06-10T02:58:32.784 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.784+0000 7f5f8effd700 10 client.25657 ms_handle_connect on v2:172.21.15.188:6838/310444836 2024-06-10T02:58:32.784 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.784+0000 7f5f8effd700 1 -- 172.21.15.52:0/92604221 <== mds.0 v2:172.21.15.188:6838/310444836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55fdbf840ea0 con 0x55fdbf91df00 2024-06-10T02:58:32.784 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.784+0000 7f5f8effd700 10 client.25657 handle_command_reply: tid=0 2024-06-10T02:58:32.784 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.784+0000 7f5fc015bb80 1 client.25657 shutdown 2024-06-10T02:58:32.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.785+0000 7f5fc015bb80 1 -- 172.21.15.52:0/92604221 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fdbf8e85f0 msgr2=0x55fdbf8ebac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:58:32.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.785+0000 7f5fc015bb80 1 --2- 172.21.15.52:0/92604221 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fdbf8e85f0 0x55fdbf8ebac0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f5fa0012b70 tx=0x7f5fa0004750 comp rx=0 tx=0).stop 2024-06-10T02:58:32.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.786+0000 7f5fc015bb80 1 -- 172.21.15.52:0/92604221 shutdown_connections 2024-06-10T02:58:32.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.786+0000 7f5fc015bb80 1 --2- 172.21.15.52:0/92604221 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55fdbf91df00 0x55fdbf920380 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:32.786 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.786+0000 7f5fc015bb80 1 --2- 172.21.15.52:0/92604221 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fdbf8e85f0 0x55fdbf8ebac0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:32.786 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.786+0000 7f5fc015bb80 1 --2- 172.21.15.52:0/92604221 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fdbf8e2460 0x55fdbf8ee210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:32.786 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.786+0000 7f5fc015bb80 1 --2- 172.21.15.52:0/92604221 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fdbf8e7cd0 0x55fdbf8eb580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:32.786 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.786+0000 7f5fc015bb80 1 -- 172.21.15.52:0/92604221 >> 172.21.15.52:0/92604221 conn(0x55fdbf80ce50 msgr2=0x55fdbf8dcda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:58:32.786 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.786+0000 7f5f8effd700 0 client.25657 ms_handle_reset on v2:172.21.15.188:6838/310444836 2024-06-10T02:58:32.786 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.786+0000 7f5fc015bb80 1 -- 172.21.15.52:0/92604221 shutdown_connections 2024-06-10T02:58:32.787 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.787+0000 7f5fc015bb80 1 -- 172.21.15.52:0/92604221 wait complete. 2024-06-10T02:58:32.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.791+0000 7f5fc015bb80 20 client.25657 trim_cache size 0 max 16384 2024-06-10T02:58:32.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.834+0000 7f5fc015bb80 1 Processor -- start 2024-06-10T02:58:32.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.834+0000 7f5fc015bb80 1 -- start start 2024-06-10T02:58:32.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.834+0000 7f5fc015bb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fdbf8e2460 0x55fdbf8e1f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:58:32.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.834+0000 7f5fc015bb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fdbf8e7cd0 0x55fdbf8ea540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:58:32.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.834+0000 7f5fc015bb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fdbf8e85f0 0x55fdbf8ebaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:58:32.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.834+0000 7f5fc015bb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f5fb01b4930 con 0x55fdbf8e7cd0 2024-06-10T02:58:32.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.834+0000 7f5fae7fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fdbf8e7cd0 0x55fdbf8ea540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:58:32.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.834+0000 7f5faeffd700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fdbf8e2460 0x55fdbf8e1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:58:32.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.834+0000 7f5fae7fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fdbf8e7cd0 0x55fdbf8ea540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53962/0 (socket says 172.21.15.52:53962) 2024-06-10T02:58:32.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.834+0000 7f5fae7fc700 1 -- 172.21.15.52:0/1432808853 learned_addr learned my addr 172.21.15.52:0/1432808853 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:58:32.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.834+0000 7f5faeffd700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fdbf8e2460 0x55fdbf8e1f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.52:39782/0 (socket says 172.21.15.52:39782) 2024-06-10T02:58:32.838 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.834+0000 7f5fc015bb80 1 -- 172.21.15.52:0/1432808853 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f5fb01b6530 con 0x55fdbf8e85f0 2024-06-10T02:58:32.838 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.834+0000 7f5fc015bb80 1 -- 172.21.15.52:0/1432808853 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f5fb01b5000 con 0x55fdbf8e2460 2024-06-10T02:58:32.838 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.835+0000 7f5faf7fe700 1 --2- 172.21.15.52:0/1432808853 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fdbf8e85f0 0x55fdbf8ebaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:58:32.838 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.835+0000 7f5fae7fc700 1 -- 172.21.15.52:0/1432808853 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fdbf8e2460 msgr2=0x55fdbf8e1f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:58:32.838 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.835+0000 7f5fae7fc700 1 --2- 172.21.15.52:0/1432808853 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fdbf8e2460 0x55fdbf8e1f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:32.838 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.835+0000 7f5fae7fc700 1 -- 172.21.15.52:0/1432808853 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fdbf8e85f0 msgr2=0x55fdbf8ebaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:58:32.838 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.835+0000 7f5fae7fc700 1 --2- 172.21.15.52:0/1432808853 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fdbf8e85f0 0x55fdbf8ebaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:32.838 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.835+0000 7f5fae7fc700 1 -- 172.21.15.52:0/1432808853 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fdbf8e9af0 con 0x55fdbf8e7cd0 2024-06-10T02:58:32.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.835+0000 7f5faf7fe700 1 --2- 172.21.15.52:0/1432808853 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fdbf8e85f0 0x55fdbf8ebaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T02:58:32.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.835+0000 7f5fae7fc700 1 --2- 172.21.15.52:0/1432808853 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fdbf8e7cd0 0x55fdbf8ea540 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f5fa80de8e0 tx=0x7f5fa80e4a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba866166aeb2a94b server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:58:32.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.835+0000 7f5fa57fa700 1 -- 172.21.15.52:0/1432808853 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fa80ad500 con 0x55fdbf8e7cd0 2024-06-10T02:58:32.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.835+0000 7f5fa57fa700 1 -- 172.21.15.52:0/1432808853 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5fa80ade30 con 0x55fdbf8e7cd0 2024-06-10T02:58:32.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.835+0000 7f5fa57fa700 1 -- 172.21.15.52:0/1432808853 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fa80a99d0 con 0x55fdbf8e7cd0 2024-06-10T02:58:32.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.836+0000 7f5fc015bb80 1 -- 172.21.15.52:0/1432808853 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fdbf8e7cd0 msgr2=0x55fdbf8ea540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:58:32.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.836+0000 7f5fc015bb80 1 --2- 172.21.15.52:0/1432808853 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fdbf8e7cd0 0x55fdbf8ea540 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f5fa80de8e0 tx=0x7f5fa80e4a10 comp rx=0 tx=0).stop 2024-06-10T02:58:32.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.836+0000 7f5fc015bb80 1 -- 172.21.15.52:0/1432808853 shutdown_connections 2024-06-10T02:58:32.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.836+0000 7f5fc015bb80 1 --2- 172.21.15.52:0/1432808853 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fdbf8e85f0 0x55fdbf8ebaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:32.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.836+0000 7f5fc015bb80 1 --2- 172.21.15.52:0/1432808853 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fdbf8e2460 0x55fdbf8e1f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:32.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.836+0000 7f5fc015bb80 1 --2- 172.21.15.52:0/1432808853 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fdbf8e7cd0 0x55fdbf8ea540 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:32.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.836+0000 7f5fc015bb80 1 -- 172.21.15.52:0/1432808853 >> 172.21.15.52:0/1432808853 conn(0x55fdbf80ce50 msgr2=0x55fdbf8ecdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:58:32.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.836+0000 7f5fc015bb80 1 -- 172.21.15.52:0/1432808853 shutdown_connections 2024-06-10T02:58:32.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.836+0000 7f5fc015bb80 1 -- 172.21.15.52:0/1432808853 wait complete. 2024-06-10T02:58:32.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.837+0000 7f5fc015bb80 1 -- start start 2024-06-10T02:58:32.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.837+0000 7f5fc015bb80 1 Processor -- start 2024-06-10T02:58:32.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.837+0000 7f5fc015bb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fdbf8e2460 0x55fdbf916780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:58:32.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.837+0000 7f5fc015bb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fdbf8e7cd0 0x55fdbf913240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:58:32.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.837+0000 7f5fc015bb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fdbf8e85f0 0x55fdbf913780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:58:32.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.837+0000 7f5fc015bb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f5fb01b5e60 con 0x55fdbf8e7cd0 2024-06-10T02:58:32.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.837+0000 7f5fc015bb80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f5fb01b5790 con 0x55fdbf8e2460 2024-06-10T02:58:32.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.837+0000 7f5fc015bb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f5fb01b28d0 con 0x55fdbf8e85f0 2024-06-10T02:58:32.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.837+0000 7f5fc015bb80 10 client.0 _ll_register_callbacks cb 0x55fdbf7cdab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T02:58:32.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.838+0000 7f5fae7fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fdbf8e7cd0 0x55fdbf913240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:58:32.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.838+0000 7f5faf7fe700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fdbf8e85f0 0x55fdbf913780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:58:32.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.838+0000 7f5fae7fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fdbf8e7cd0 0x55fdbf913240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53968/0 (socket says 172.21.15.52:53968) 2024-06-10T02:58:32.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.838+0000 7f5fae7fc700 1 -- 172.21.15.52:0/1033646307 learned_addr learned my addr 172.21.15.52:0/1033646307 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:58:32.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.838+0000 7f5faeffd700 1 --2- 172.21.15.52:0/1033646307 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fdbf8e2460 0x55fdbf916780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:58:32.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.838+0000 7f5fae7fc700 1 -- 172.21.15.52:0/1033646307 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fdbf8e85f0 msgr2=0x55fdbf913780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:58:32.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.838+0000 7f5fae7fc700 1 --2- 172.21.15.52:0/1033646307 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fdbf8e85f0 0x55fdbf913780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:32.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.838+0000 7f5fae7fc700 1 -- 172.21.15.52:0/1033646307 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fdbf8e2460 msgr2=0x55fdbf916780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:58:32.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.838+0000 7f5fae7fc700 1 --2- 172.21.15.52:0/1033646307 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fdbf8e2460 0x55fdbf916780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:32.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.838+0000 7f5fae7fc700 1 -- 172.21.15.52:0/1033646307 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fdbf913cf0 con 0x55fdbf8e7cd0 2024-06-10T02:58:32.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.838+0000 7f5faf7fe700 1 --2- 172.21.15.52:0/1033646307 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fdbf8e85f0 0x55fdbf913780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T02:58:32.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.838+0000 7f5faeffd700 1 --2- 172.21.15.52:0/1033646307 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fdbf8e2460 0x55fdbf916780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T02:58:32.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.838+0000 7f5fae7fc700 1 --2- 172.21.15.52:0/1033646307 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fdbf8e7cd0 0x55fdbf913240 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f5fa802e040 tx=0x7f5fa80e4700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0187793cc45bc8b server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:58:32.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.838+0000 7f5f8effd700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-10T02:58:32.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.838+0000 7f5f8effd700 1 -- 172.21.15.52:0/1033646307 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fa80a99d0 con 0x55fdbf8e7cd0 2024-06-10T02:58:32.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.838+0000 7f5f8effd700 1 -- 172.21.15.52:0/1033646307 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f5fa800f070 con 0x55fdbf8e7cd0 2024-06-10T02:58:32.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.838+0000 7f5fc015bb80 1 -- 172.21.15.52:0/1033646307 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5fa80ade30 con 0x55fdbf8e7cd0 2024-06-10T02:58:32.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.838+0000 7f5f8effd700 1 -- 172.21.15.52:0/1033646307 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f5fa80ad500 con 0x55fdbf8e7cd0 2024-06-10T02:58:32.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.838+0000 7f5fc015bb80 10 client.16089 fetch_fsmap learned FSMap version 34 2024-06-10T02:58:32.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.838+0000 7f5fc015bb80 1 -- 172.21.15.52:0/1033646307 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55fdbf989990 con 0x55fdbf8e7cd0 2024-06-10T02:58:32.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.839+0000 7f5f8effd700 1 -- 172.21.15.52:0/1033646307 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5fa803c300 con 0x55fdbf8e7cd0 2024-06-10T02:58:32.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.839+0000 7f5f8effd700 1 -- 172.21.15.52:0/1033646307 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 34) ==== 8378+0+0 (secure 0 0 0) 0x7f5fa8022070 con 0x55fdbf8e7cd0 2024-06-10T02:58:32.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.839+0000 7f5fc015bb80 10 client.16089 fetch_fsmap finished waiting for FSMap version 34 2024-06-10T02:58:32.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.839+0000 7f5fc015bb80 10 client.16089 resolve_mds: validated gid 24484 aka daemon mds.i 2024-06-10T02:58:32.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.839+0000 7f5fc015bb80 20 client.16089 populate_metadata read hostname 'smithi052' 2024-06-10T02:58:32.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.839+0000 7f5fc015bb80 1 --2- 172.21.15.52:0/1033646307 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55fdbf989df0 0x55fdbf98c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:58:32.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.839+0000 7f5fc015bb80 4 client.16089 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 138.2, "await_for": 45.0}] 2024-06-10T02:58:32.846 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.839+0000 7f5fc015bb80 1 -- 172.21.15.52:0/1033646307 --> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 138.2, "await_for": 45.0}) -- 0x7f5fa80ade30 con 0x55fdbf989df0 2024-06-10T02:58:32.846 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.839+0000 7f5faeffd700 1 --2- 172.21.15.52:0/1033646307 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55fdbf989df0 0x55fdbf98c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:58:32.846 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.840+0000 7f5faeffd700 1 --2- 172.21.15.52:0/1033646307 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55fdbf989df0 0x55fdbf98c1e0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e5ff46001c9ffa0e server_cookie=807f0ca0a832173e in_seq=0 out_seq=0 2024-06-10T02:58:32.846 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:32.840+0000 7f5f8effd700 10 client.16089 ms_handle_connect on v2:172.21.15.188:6838/310444836 2024-06-10T02:58:33.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:58:32 smithi120 ceph-mon[36634]: pgmap v564: 129 pgs: 129 active+clean; 414 MiB data, 904 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.4 MiB/s wr, 314 op/s 2024-06-10T02:58:33.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:58:32 smithi188 ceph-mon[36599]: pgmap v564: 129 pgs: 129 active+clean; 414 MiB data, 904 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.4 MiB/s wr, 314 op/s 2024-06-10T02:58:33.227 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:58:32 smithi052 ceph-mon[32362]: pgmap v564: 129 pgs: 129 active+clean; 414 MiB data, 904 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.4 MiB/s wr, 314 op/s 2024-06-10T02:58:35.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:58:34 smithi120 ceph-mon[36634]: pgmap v565: 129 pgs: 129 active+clean; 434 MiB data, 942 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 6.3 MiB/s wr, 425 op/s 2024-06-10T02:58:35.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:58:34 smithi188 ceph-mon[36599]: pgmap v565: 129 pgs: 129 active+clean; 434 MiB data, 942 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 6.3 MiB/s wr, 425 op/s 2024-06-10T02:58:35.226 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:58:34 smithi052 ceph-mon[32362]: pgmap v565: 129 pgs: 129 active+clean; 434 MiB data, 942 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 6.3 MiB/s wr, 425 op/s 2024-06-10T02:58:37.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:58:36 smithi120 ceph-mon[36634]: pgmap v566: 129 pgs: 129 active+clean; 440 MiB data, 959 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 6.2 MiB/s wr, 413 op/s 2024-06-10T02:58:37.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:58:36 smithi188 ceph-mon[36599]: pgmap v566: 129 pgs: 129 active+clean; 440 MiB data, 959 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 6.2 MiB/s wr, 413 op/s 2024-06-10T02:58:37.226 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:58:36 smithi052 ceph-mon[32362]: pgmap v566: 129 pgs: 129 active+clean; 440 MiB data, 959 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 6.2 MiB/s wr, 413 op/s 2024-06-10T02:58:37.246 INFO:tasks.workunit.client.0.smithi052.stdout: 74 880758 205 612874 149 462109 580 2024-06-10T02:58:39.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:58:38 smithi120 ceph-mon[36634]: pgmap v567: 129 pgs: 129 active+clean; 442 MiB data, 970 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.6 MiB/s wr, 465 op/s 2024-06-10T02:58:39.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:58:38 smithi188 ceph-mon[36599]: pgmap v567: 129 pgs: 129 active+clean; 442 MiB data, 970 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.6 MiB/s wr, 465 op/s 2024-06-10T02:58:39.226 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:58:38 smithi052 ceph-mon[32362]: pgmap v567: 129 pgs: 129 active+clean; 442 MiB data, 970 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.6 MiB/s wr, 465 op/s 2024-06-10T02:58:40.940 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-10T02:58:41.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:58:40 smithi120 ceph-mon[36634]: pgmap v568: 129 pgs: 129 active+clean; 442 MiB data, 962 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.8 MiB/s wr, 433 op/s 2024-06-10T02:58:41.195 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:58:41.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:58:40 smithi188 ceph-mon[36599]: pgmap v568: 129 pgs: 129 active+clean; 442 MiB data, 962 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.8 MiB/s wr, 433 op/s 2024-06-10T02:58:41.226 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:58:40 smithi052 ceph-mon[32362]: pgmap v568: 129 pgs: 129 active+clean; 442 MiB data, 962 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.8 MiB/s wr, 433 op/s 2024-06-10T02:58:42.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.873+0000 7ff3b5a6c700 1 -- 172.21.15.52:0/2632095604 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff3b0107860 msgr2=0x7ff3b010f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:58:42.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.873+0000 7ff3b5a6c700 1 --2- 172.21.15.52:0/2632095604 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff3b0107860 0x7ff3b010f3e0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7ff3a800b0d0 tx=0x7ff3a800b3e0 comp rx=0 tx=0).stop 2024-06-10T02:58:42.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.873+0000 7ff3b5a6c700 1 -- 172.21.15.52:0/2632095604 shutdown_connections 2024-06-10T02:58:42.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.873+0000 7ff3b5a6c700 1 --2- 172.21.15.52:0/2632095604 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff3b0107860 0x7ff3b010f3e0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:42.875 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.873+0000 7ff3b5a6c700 1 --2- 172.21.15.52:0/2632095604 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff3b0102770 0x7ff3b0102b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:42.875 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.873+0000 7ff3b5a6c700 1 --2- 172.21.15.52:0/2632095604 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff3b0103120 0x7ff3b0107130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:42.875 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.873+0000 7ff3b5a6c700 1 -- 172.21.15.52:0/2632095604 >> 172.21.15.52:0/2632095604 conn(0x7ff3b0079600 msgr2=0x7ff3b0079a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:58:42.875 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.873+0000 7ff3b5a6c700 1 -- 172.21.15.52:0/2632095604 shutdown_connections 2024-06-10T02:58:42.875 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.873+0000 7ff3b5a6c700 1 -- 172.21.15.52:0/2632095604 wait complete. 2024-06-10T02:58:42.875 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.874+0000 7ff3b5a6c700 1 Processor -- start 2024-06-10T02:58:42.875 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.874+0000 7ff3b5a6c700 1 -- start start 2024-06-10T02:58:42.875 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.874+0000 7ff3b5a6c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff3b0102770 0x7ff3b0074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:58:42.875 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.874+0000 7ff3b5a6c700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff3b0103120 0x7ff3b0074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:58:42.875 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.874+0000 7ff3b5a6c700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff3b0107860 0x7ff3b0075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:58:42.876 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.874+0000 7ff3b5a6c700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ff3b01127a0 con 0x7ff3b0102770 2024-06-10T02:58:42.876 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.874+0000 7ff3b5a6c700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff3b0112620 con 0x7ff3b0107860 2024-06-10T02:58:42.876 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.874+0000 7ff3b5a6c700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ff3b0112920 con 0x7ff3b0103120 2024-06-10T02:58:42.876 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.875+0000 7ff3af7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff3b0102770 0x7ff3b0074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:58:42.876 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.875+0000 7ff3af7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff3b0102770 0x7ff3b0074790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48988/0 (socket says 172.21.15.52:48988) 2024-06-10T02:58:42.876 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.875+0000 7ff3af7fe700 1 -- 172.21.15.52:0/3525598263 learned_addr learned my addr 172.21.15.52:0/3525598263 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:58:42.876 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.875+0000 7ff3affff700 1 --2- 172.21.15.52:0/3525598263 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff3b0107860 0x7ff3b0075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:58:42.876 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.875+0000 7ff3aeffd700 1 --2- 172.21.15.52:0/3525598263 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff3b0103120 0x7ff3b0074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:58:42.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.875+0000 7ff3af7fe700 1 -- 172.21.15.52:0/3525598263 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff3b0103120 msgr2=0x7ff3b0074cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:58:42.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.875+0000 7ff3af7fe700 1 --2- 172.21.15.52:0/3525598263 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff3b0103120 0x7ff3b0074cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:42.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.875+0000 7ff3af7fe700 1 -- 172.21.15.52:0/3525598263 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff3b0107860 msgr2=0x7ff3b0075210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:58:42.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.875+0000 7ff3af7fe700 1 --2- 172.21.15.52:0/3525598263 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff3b0107860 0x7ff3b0075210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:42.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.875+0000 7ff3af7fe700 1 -- 172.21.15.52:0/3525598263 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3b01a7080 con 0x7ff3b0102770 2024-06-10T02:58:42.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.875+0000 7ff3af7fe700 1 --2- 172.21.15.52:0/3525598263 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff3b0102770 0x7ff3b0074790 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7ff3a000d950 tx=0x7ff3a000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:58:42.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.875+0000 7ff3acff9700 1 -- 172.21.15.52:0/3525598263 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3a0016070 con 0x7ff3b0102770 2024-06-10T02:58:42.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.875+0000 7ff3b5a6c700 1 -- 172.21.15.52:0/3525598263 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3b01a7370 con 0x7ff3b0102770 2024-06-10T02:58:42.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.875+0000 7ff3acff9700 1 -- 172.21.15.52:0/3525598263 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff3a000fd70 con 0x7ff3b0102770 2024-06-10T02:58:42.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.875+0000 7ff3acff9700 1 -- 172.21.15.52:0/3525598263 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3a000e610 con 0x7ff3b0102770 2024-06-10T02:58:42.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.875+0000 7ff3b5a6c700 1 -- 172.21.15.52:0/3525598263 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3b01a78b0 con 0x7ff3b0102770 2024-06-10T02:58:42.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.878+0000 7ff3acff9700 1 -- 172.21.15.52:0/3525598263 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7ff3a0019070 con 0x7ff3b0102770 2024-06-10T02:58:42.880 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.879+0000 7ff3acff9700 1 --2- 172.21.15.52:0/3525598263 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7ff3980a2c20 0x7ff3980a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:58:42.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.879+0000 7ff3aeffd700 1 --2- 172.21.15.52:0/3525598263 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7ff3980a2c20 0x7ff3980a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:58:42.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.879+0000 7ff3acff9700 1 -- 172.21.15.52:0/3525598263 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7ff3a00ceee0 con 0x7ff3b0102770 2024-06-10T02:58:42.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.879+0000 7ff3aeffd700 1 --2- 172.21.15.52:0/3525598263 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7ff3980a2c20 0x7ff3980a50e0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7ff3b0108bb0 tx=0x7ff3a4009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:58:42.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.880+0000 7ff3c02efb80 1 Processor -- start 2024-06-10T02:58:42.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.880+0000 7ff3c02efb80 1 -- start start 2024-06-10T02:58:42.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.880+0000 7ff3c02efb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d4190d0640 0x55d4190d0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:58:42.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.880+0000 7ff3c02efb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d4190d0f60 0x55d4190ca9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:58:42.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.880+0000 7ff3c02efb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d4190caf20 0x55d4190cb340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:58:42.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.880+0000 7ff3c02efb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55d41901c810 con 0x55d4190caf20 2024-06-10T02:58:42.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.880+0000 7ff3c02efb80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55d418fc4720 con 0x55d4190d0f60 2024-06-10T02:58:42.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.880+0000 7ff3c02efb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55d4190187d0 con 0x55d4190d0640 2024-06-10T02:58:42.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.880+0000 7ff3affff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d4190caf20 0x55d4190cb340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:58:42.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.880+0000 7ff3affff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d4190caf20 0x55d4190cb340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49004/0 (socket says 172.21.15.52:49004) 2024-06-10T02:58:42.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.880+0000 7ff3affff700 1 -- 172.21.15.52:0/3183744561 learned_addr learned my addr 172.21.15.52:0/3183744561 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:58:42.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.880+0000 7ff3aeffd700 1 --2- 172.21.15.52:0/3183744561 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d4190d0f60 0x55d4190ca9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:58:42.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.880+0000 7ff3af7fe700 1 --2- 172.21.15.52:0/3183744561 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d4190d0640 0x55d4190d0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:58:42.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.880+0000 7ff3affff700 1 -- 172.21.15.52:0/3183744561 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d4190d0640 msgr2=0x55d4190d0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:58:42.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.880+0000 7ff3affff700 1 --2- 172.21.15.52:0/3183744561 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d4190d0640 0x55d4190d0a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:42.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.880+0000 7ff3affff700 1 -- 172.21.15.52:0/3183744561 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d4190d0f60 msgr2=0x55d4190ca9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:58:42.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.880+0000 7ff3affff700 1 --2- 172.21.15.52:0/3183744561 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d4190d0f60 0x55d4190ca9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:42.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.880+0000 7ff3affff700 1 -- 172.21.15.52:0/3183744561 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d418fbaf50 con 0x55d4190caf20 2024-06-10T02:58:42.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.880+0000 7ff3aeffd700 1 --2- 172.21.15.52:0/3183744561 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d4190d0f60 0x55d4190ca9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T02:58:42.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.880+0000 7ff3affff700 1 --2- 172.21.15.52:0/3183744561 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d4190caf20 0x55d4190cb340 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7ff3a8009d50 tx=0x7ff3a8009d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2af351eeed43ab8 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:58:42.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.881+0000 7ff395ffb700 1 -- 172.21.15.52:0/3183744561 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3a8009460 con 0x55d4190caf20 2024-06-10T02:58:42.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.881+0000 7ff395ffb700 1 -- 172.21.15.52:0/3183744561 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff3a8003a00 con 0x55d4190caf20 2024-06-10T02:58:42.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.881+0000 7ff395ffb700 1 -- 172.21.15.52:0/3183744561 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3a80057c0 con 0x55d4190caf20 2024-06-10T02:58:42.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.881+0000 7ff3c02efb80 1 -- 172.21.15.52:0/3183744561 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d4190caf20 msgr2=0x55d4190cb340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:58:42.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.881+0000 7ff3c02efb80 1 --2- 172.21.15.52:0/3183744561 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d4190caf20 0x55d4190cb340 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7ff3a8009d50 tx=0x7ff3a8009d80 comp rx=0 tx=0).stop 2024-06-10T02:58:42.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.882+0000 7ff3c02efb80 1 -- 172.21.15.52:0/3183744561 shutdown_connections 2024-06-10T02:58:42.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.882+0000 7ff3c02efb80 1 --2- 172.21.15.52:0/3183744561 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d4190caf20 0x55d4190cb340 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:42.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.882+0000 7ff3c02efb80 1 --2- 172.21.15.52:0/3183744561 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d4190d0640 0x55d4190d0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:42.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.882+0000 7ff3c02efb80 1 --2- 172.21.15.52:0/3183744561 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d4190d0f60 0x55d4190ca9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:42.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.882+0000 7ff3c02efb80 1 -- 172.21.15.52:0/3183744561 >> 172.21.15.52:0/3183744561 conn(0x55d418ff5cc0 msgr2=0x55d4190c5850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:58:42.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.882+0000 7ff3c02efb80 1 -- 172.21.15.52:0/3183744561 shutdown_connections 2024-06-10T02:58:42.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.882+0000 7ff3c02efb80 1 -- 172.21.15.52:0/3183744561 wait complete. 2024-06-10T02:58:42.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.882+0000 7ff3c02efb80 1 -- start start 2024-06-10T02:58:42.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.882+0000 7ff3c02efb80 1 Processor -- start 2024-06-10T02:58:42.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.882+0000 7ff3c02efb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d4190caf20 0x55d4190d6cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:58:42.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.882+0000 7ff3c02efb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d4190d0640 0x55d4190d3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:58:42.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.882+0000 7ff3c02efb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d4190d0f60 0x55d4190d4510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:58:42.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.882+0000 7ff3c02efb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55d4190d7970 con 0x55d4190d0f60 2024-06-10T02:58:42.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.882+0000 7ff3c02efb80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55d418fbcbf0 con 0x55d4190d0640 2024-06-10T02:58:42.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.882+0000 7ff3c02efb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55d4190d7af0 con 0x55d4190caf20 2024-06-10T02:58:42.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.883+0000 7ff3c02efb80 10 client.0 _ll_register_callbacks cb 0x55d418fb6920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T02:58:42.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.883+0000 7ff3affff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d4190d0f60 0x55d4190d4510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:58:42.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.883+0000 7ff3affff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d4190d0f60 0x55d4190d4510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49016/0 (socket says 172.21.15.52:49016) 2024-06-10T02:58:42.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.883+0000 7ff3affff700 1 -- 172.21.15.52:0/855731247 learned_addr learned my addr 172.21.15.52:0/855731247 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:58:42.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.883+0000 7ff3af7fe700 1 --2- 172.21.15.52:0/855731247 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d4190caf20 0x55d4190d6cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:58:42.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.883+0000 7ff3aeffd700 1 --2- 172.21.15.52:0/855731247 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d4190d0640 0x55d4190d3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:58:42.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.883+0000 7ff3affff700 1 -- 172.21.15.52:0/855731247 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d4190caf20 msgr2=0x55d4190d6cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:58:42.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.883+0000 7ff3affff700 1 --2- 172.21.15.52:0/855731247 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d4190caf20 0x55d4190d6cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:42.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.883+0000 7ff3affff700 1 -- 172.21.15.52:0/855731247 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d4190d0640 msgr2=0x55d4190d3fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:58:42.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.883+0000 7ff3affff700 1 --2- 172.21.15.52:0/855731247 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d4190d0640 0x55d4190d3fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:42.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.883+0000 7ff3affff700 1 -- 172.21.15.52:0/855731247 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d418f6b8d0 con 0x55d4190d0f60 2024-06-10T02:58:42.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.883+0000 7ff3af7fe700 1 --2- 172.21.15.52:0/855731247 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d4190caf20 0x55d4190d6cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T02:58:42.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.883+0000 7ff3affff700 1 --2- 172.21.15.52:0/855731247 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d4190d0f60 0x55d4190d4510 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7ff3a800d010 tx=0x7ff3a8009460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f6e5ea2b055366c server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:58:42.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.883+0000 7ff38effd700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-10T02:58:42.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.883+0000 7ff38effd700 1 -- 172.21.15.52:0/855731247 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3a802a030 con 0x55d4190d0f60 2024-06-10T02:58:42.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.883+0000 7ff3c02efb80 1 -- 172.21.15.52:0/855731247 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff3a8003a00 con 0x55d4190d0f60 2024-06-10T02:58:42.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.883+0000 7ff38effd700 1 -- 172.21.15.52:0/855731247 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7ff3a8005440 con 0x55d4190d0f60 2024-06-10T02:58:42.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.884+0000 7ff38effd700 1 -- 172.21.15.52:0/855731247 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7ff3a8037050 con 0x55d4190d0f60 2024-06-10T02:58:42.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.884+0000 7ff3c02efb80 10 client.16101 fetch_fsmap learned FSMap version 34 2024-06-10T02:58:42.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.884+0000 7ff3c02efb80 1 -- 172.21.15.52:0/855731247 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55d4190e2270 con 0x55d4190d0f60 2024-06-10T02:58:42.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.884+0000 7ff38effd700 1 -- 172.21.15.52:0/855731247 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff3a8003a00 con 0x55d4190d0f60 2024-06-10T02:58:42.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.884+0000 7ff38effd700 1 -- 172.21.15.52:0/855731247 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 34) ==== 8378+0+0 (secure 0 0 0) 0x7ff3a8011070 con 0x55d4190d0f60 2024-06-10T02:58:42.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.884+0000 7ff3c02efb80 10 client.16101 fetch_fsmap finished waiting for FSMap version 34 2024-06-10T02:58:42.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.884+0000 7ff3c02efb80 10 client.16101 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T02:58:42.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.884+0000 7ff3c02efb80 20 client.16101 populate_metadata read hostname 'smithi052' 2024-06-10T02:58:42.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.884+0000 7ff3c02efb80 1 --2- 172.21.15.52:0/855731247 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55d4190e4740 0x55d4190e6b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:58:42.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.884+0000 7ff3c02efb80 4 client.16101 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T02:58:42.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.884+0000 7ff3c02efb80 1 -- 172.21.15.52:0/855731247 --> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d419029d10 con 0x55d4190e4740 2024-06-10T02:58:42.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.884+0000 7ff3af7fe700 1 --2- 172.21.15.52:0/855731247 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55d4190e4740 0x55d4190e6b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:58:42.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.885+0000 7ff38effd700 10 client.16101 ms_handle_connect on v2:172.21.15.188:6838/310444836 2024-06-10T02:58:42.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.885+0000 7ff3af7fe700 1 --2- 172.21.15.52:0/855731247 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55d4190e4740 0x55d4190e6b30 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c05f6b1a71ec6 server_cookie=4e617d190b334b39 in_seq=0 out_seq=0 2024-06-10T02:58:42.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.885+0000 7ff38effd700 1 -- 172.21.15.52:0/855731247 <== mds.0 v2:172.21.15.188:6838/310444836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55d419029d10 con 0x55d4190e4740 2024-06-10T02:58:42.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.885+0000 7ff38effd700 10 client.16101 handle_command_reply: tid=0 2024-06-10T02:58:42.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.885+0000 7ff3c02efb80 1 client.16101 shutdown 2024-06-10T02:58:42.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.886+0000 7ff3c02efb80 1 -- 172.21.15.52:0/855731247 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d4190d0f60 msgr2=0x55d4190d4510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:58:42.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.886+0000 7ff3c02efb80 1 --2- 172.21.15.52:0/855731247 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d4190d0f60 0x55d4190d4510 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7ff3a800d010 tx=0x7ff3a8009460 comp rx=0 tx=0).stop 2024-06-10T02:58:42.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.886+0000 7ff3c02efb80 1 -- 172.21.15.52:0/855731247 shutdown_connections 2024-06-10T02:58:42.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.886+0000 7ff3c02efb80 1 --2- 172.21.15.52:0/855731247 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55d4190e4740 0x55d4190e6b30 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:42.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.886+0000 7ff3c02efb80 1 --2- 172.21.15.52:0/855731247 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d4190d0f60 0x55d4190d4510 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:42.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.886+0000 7ff3c02efb80 1 --2- 172.21.15.52:0/855731247 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d4190caf20 0x55d4190d6cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:42.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.886+0000 7ff3c02efb80 1 --2- 172.21.15.52:0/855731247 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d4190d0640 0x55d4190d3fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:42.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.886+0000 7ff3c02efb80 1 -- 172.21.15.52:0/855731247 >> 172.21.15.52:0/855731247 conn(0x55d418ff5cc0 msgr2=0x55d4190c99d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:58:42.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.886+0000 7ff38effd700 0 client.16101 ms_handle_reset on v2:172.21.15.188:6838/310444836 2024-06-10T02:58:42.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.886+0000 7ff3c02efb80 1 -- 172.21.15.52:0/855731247 shutdown_connections 2024-06-10T02:58:42.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.886+0000 7ff3c02efb80 1 -- 172.21.15.52:0/855731247 wait complete. 2024-06-10T02:58:42.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.886+0000 7ff3c02efb80 20 client.16101 trim_cache size 0 max 16384 2024-06-10T02:58:42.906 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.907+0000 7ff3c02efb80 1 Processor -- start 2024-06-10T02:58:42.907 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.907+0000 7ff3c02efb80 1 -- start start 2024-06-10T02:58:42.907 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:58:42 smithi052 ceph-mon[32362]: pgmap v569: 129 pgs: 129 active+clean; 442 MiB data, 962 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.2 MiB/s wr, 328 op/s 2024-06-10T02:58:42.909 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.907+0000 7ff3c02efb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d4190caf20 0x55d4190ca9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:58:42.909 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.907+0000 7ff3c02efb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d4190d0640 0x55d4190ce730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:58:42.909 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.907+0000 7ff3c02efb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d4190d0f60 0x55d4190d1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:58:42.909 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.907+0000 7ff3c02efb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ff3b01b0550 con 0x55d4190d0640 2024-06-10T02:58:42.909 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.907+0000 7ff3c02efb80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff3b01b2150 con 0x55d4190caf20 2024-06-10T02:58:42.909 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.907+0000 7ff3c02efb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ff3b01b0c20 con 0x55d4190d0f60 2024-06-10T02:58:42.909 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.907+0000 7ff3aeffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d4190d0640 0x55d4190ce730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:58:42.909 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.907+0000 7ff3aeffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d4190d0640 0x55d4190ce730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49018/0 (socket says 172.21.15.52:49018) 2024-06-10T02:58:42.909 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.907+0000 7ff3aeffd700 1 -- 172.21.15.52:0/1720874570 learned_addr learned my addr 172.21.15.52:0/1720874570 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:58:42.909 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.907+0000 7ff3af7fe700 1 --2- 172.21.15.52:0/1720874570 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d4190caf20 0x55d4190ca9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:58:42.910 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.907+0000 7ff3affff700 1 --2- 172.21.15.52:0/1720874570 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d4190d0f60 0x55d4190d1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:58:42.910 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.907+0000 7ff3aeffd700 1 -- 172.21.15.52:0/1720874570 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d4190d0f60 msgr2=0x55d4190d1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:58:42.910 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.907+0000 7ff3aeffd700 1 --2- 172.21.15.52:0/1720874570 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d4190d0f60 0x55d4190d1a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:42.910 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.907+0000 7ff3aeffd700 1 -- 172.21.15.52:0/1720874570 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d4190caf20 msgr2=0x55d4190ca9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:58:42.910 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.907+0000 7ff3aeffd700 1 --2- 172.21.15.52:0/1720874570 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d4190caf20 0x55d4190ca9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:42.910 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.907+0000 7ff3aeffd700 1 -- 172.21.15.52:0/1720874570 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d4190d3da0 con 0x55d4190d0640 2024-06-10T02:58:42.910 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.907+0000 7ff3af7fe700 1 --2- 172.21.15.52:0/1720874570 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d4190caf20 0x55d4190ca9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T02:58:42.910 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.908+0000 7ff3aeffd700 1 --2- 172.21.15.52:0/1720874570 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d4190d0640 0x55d4190ce730 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7ff3a4011ab0 tx=0x7ff3a4014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=972c63a9e9d733c0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:58:42.910 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.908+0000 7ff3957fa700 1 -- 172.21.15.52:0/1720874570 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3a4004050 con 0x55d4190d0640 2024-06-10T02:58:42.910 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.908+0000 7ff3957fa700 1 -- 172.21.15.52:0/1720874570 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff3a40208d0 con 0x55d4190d0640 2024-06-10T02:58:42.910 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.908+0000 7ff3957fa700 1 -- 172.21.15.52:0/1720874570 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3a4003ac0 con 0x55d4190d0640 2024-06-10T02:58:42.910 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.908+0000 7ff3c02efb80 1 -- 172.21.15.52:0/1720874570 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d4190d0640 msgr2=0x55d4190ce730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:58:42.911 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.908+0000 7ff3c02efb80 1 --2- 172.21.15.52:0/1720874570 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d4190d0640 0x55d4190ce730 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7ff3a4011ab0 tx=0x7ff3a4014000 comp rx=0 tx=0).stop 2024-06-10T02:58:42.911 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.908+0000 7ff3c02efb80 1 -- 172.21.15.52:0/1720874570 shutdown_connections 2024-06-10T02:58:42.911 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.908+0000 7ff3c02efb80 1 --2- 172.21.15.52:0/1720874570 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d4190d0f60 0x55d4190d1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:42.911 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.908+0000 7ff3c02efb80 1 --2- 172.21.15.52:0/1720874570 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d4190caf20 0x55d4190ca9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:42.911 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.908+0000 7ff3c02efb80 1 --2- 172.21.15.52:0/1720874570 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d4190d0640 0x55d4190ce730 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:42.911 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.908+0000 7ff3c02efb80 1 -- 172.21.15.52:0/1720874570 >> 172.21.15.52:0/1720874570 conn(0x55d418ff5cc0 msgr2=0x55d4190d1340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:58:42.911 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.909+0000 7ff3c02efb80 1 -- 172.21.15.52:0/1720874570 shutdown_connections 2024-06-10T02:58:42.911 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.909+0000 7ff3c02efb80 1 -- 172.21.15.52:0/1720874570 wait complete. 2024-06-10T02:58:42.911 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.909+0000 7ff3c02efb80 1 -- start start 2024-06-10T02:58:42.911 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.909+0000 7ff3c02efb80 1 Processor -- start 2024-06-10T02:58:42.914 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.909+0000 7ff3c02efb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d4190caf20 0x55d4190e6f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:58:42.914 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.909+0000 7ff3c02efb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d4190d0640 0x55d4190e7450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:58:42.914 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.909+0000 7ff3c02efb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d4190d0f60 0x55d419147040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:58:42.915 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.909+0000 7ff3c02efb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ff3b01b1a80 con 0x55d4190d0640 2024-06-10T02:58:42.915 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.909+0000 7ff3c02efb80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff3b01b13b0 con 0x55d4190caf20 2024-06-10T02:58:42.915 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.909+0000 7ff3c02efb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ff3b01ae4f0 con 0x55d4190d0f60 2024-06-10T02:58:42.915 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.910+0000 7ff3c02efb80 10 client.0 _ll_register_callbacks cb 0x55d418fb6920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T02:58:42.915 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.910+0000 7ff3aeffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d4190d0640 0x55d4190e7450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:58:42.915 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.910+0000 7ff3af7fe700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d4190caf20 0x55d4190e6f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:58:42.915 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.910+0000 7ff3affff700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d4190d0f60 0x55d419147040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:58:42.915 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.910+0000 7ff3aeffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d4190d0640 0x55d4190e7450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49022/0 (socket says 172.21.15.52:49022) 2024-06-10T02:58:42.915 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.910+0000 7ff3aeffd700 1 -- 172.21.15.52:0/1262740634 learned_addr learned my addr 172.21.15.52:0/1262740634 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:58:42.915 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.910+0000 7ff3affff700 1 -- 172.21.15.52:0/1262740634 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d4190caf20 msgr2=0x55d4190e6f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:58:42.915 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.910+0000 7ff3affff700 1 --2- 172.21.15.52:0/1262740634 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d4190caf20 0x55d4190e6f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:42.915 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.910+0000 7ff3affff700 1 -- 172.21.15.52:0/1262740634 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d4190d0640 msgr2=0x55d4190e7450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:58:42.916 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.910+0000 7ff3affff700 1 --2- 172.21.15.52:0/1262740634 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d4190d0640 0x55d4190e7450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:42.916 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.910+0000 7ff3affff700 1 -- 172.21.15.52:0/1262740634 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d4191476b0 con 0x55d4190d0f60 2024-06-10T02:58:42.916 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.910+0000 7ff3aeffd700 1 --2- 172.21.15.52:0/1262740634 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d4190d0640 0x55d4190e7450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T02:58:42.916 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.910+0000 7ff3affff700 1 --2- 172.21.15.52:0/1262740634 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d4190d0f60 0x55d419147040 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7ff3a80052d0 tx=0x7ff3a80099e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8fad8de6d7ca98b2 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:58:42.916 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.910+0000 7ff38effd700 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-06-10T02:58:42.916 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.910+0000 7ff38effd700 1 -- 172.21.15.52:0/1262740634 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3a802a070 con 0x55d4190d0f60 2024-06-10T02:58:42.916 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.910+0000 7ff3c02efb80 1 -- 172.21.15.52:0/1262740634 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff3a40208d0 con 0x55d4190d0f60 2024-06-10T02:58:42.916 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.910+0000 7ff38effd700 1 -- 172.21.15.52:0/1262740634 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7ff3a80312b0 con 0x55d4190d0f60 2024-06-10T02:58:42.916 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.910+0000 7ff38effd700 1 -- 172.21.15.52:0/1262740634 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7ff3a8003a00 con 0x55d4190d0f60 2024-06-10T02:58:42.916 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.911+0000 7ff3c02efb80 10 client.25681 fetch_fsmap learned FSMap version 34 2024-06-10T02:58:42.916 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.911+0000 7ff3c02efb80 1 -- 172.21.15.52:0/1262740634 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55d41914ea20 con 0x55d4190d0f60 2024-06-10T02:58:42.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.911+0000 7ff38effd700 1 -- 172.21.15.52:0/1262740634 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff3a40208d0 con 0x55d4190d0f60 2024-06-10T02:58:42.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.911+0000 7ff38effd700 1 -- 172.21.15.52:0/1262740634 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 34) ==== 8378+0+0 (secure 0 0 0) 0x7ff3a8011070 con 0x55d4190d0f60 2024-06-10T02:58:42.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.911+0000 7ff3c02efb80 10 client.25681 fetch_fsmap finished waiting for FSMap version 34 2024-06-10T02:58:42.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.911+0000 7ff3c02efb80 10 client.25681 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T02:58:42.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.911+0000 7ff3c02efb80 20 client.25681 populate_metadata read hostname 'smithi052' 2024-06-10T02:58:42.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.911+0000 7ff3c02efb80 1 --2- 172.21.15.52:0/1262740634 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55d41914f070 0x55d419151460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:58:42.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.911+0000 7ff3c02efb80 4 client.25681 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-10T02:58:42.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.911+0000 7ff3c02efb80 1 -- 172.21.15.52:0/1262740634 --> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d419024290 con 0x55d41914f070 2024-06-10T02:58:42.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.911+0000 7ff3af7fe700 1 --2- 172.21.15.52:0/1262740634 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55d41914f070 0x55d419151460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:58:42.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.912+0000 7ff38effd700 10 client.25681 ms_handle_connect on v2:172.21.15.188:6838/310444836 2024-06-10T02:58:42.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.912+0000 7ff3af7fe700 1 --2- 172.21.15.52:0/1262740634 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55d41914f070 0x55d419151460 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e48b7d27f1616221 server_cookie=fa37c6e8deec8d26 in_seq=0 out_seq=0 2024-06-10T02:58:42.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.912+0000 7ff38effd700 1 -- 172.21.15.52:0/1262740634 <== mds.0 v2:172.21.15.188:6838/310444836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55d419024290 con 0x55d41914f070 2024-06-10T02:58:42.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.912+0000 7ff38effd700 10 client.25681 handle_command_reply: tid=0 2024-06-10T02:58:42.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.912+0000 7ff3c02efb80 1 client.25681 shutdown 2024-06-10T02:58:42.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.912+0000 7ff3c02efb80 1 -- 172.21.15.52:0/1262740634 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d4190d0f60 msgr2=0x55d419147040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:58:42.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.912+0000 7ff3c02efb80 1 --2- 172.21.15.52:0/1262740634 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d4190d0f60 0x55d419147040 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7ff3a80052d0 tx=0x7ff3a80099e0 comp rx=0 tx=0).stop 2024-06-10T02:58:42.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.912+0000 7ff3c02efb80 1 -- 172.21.15.52:0/1262740634 shutdown_connections 2024-06-10T02:58:42.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.912+0000 7ff3c02efb80 1 --2- 172.21.15.52:0/1262740634 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55d41914f070 0x55d419151460 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:42.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.912+0000 7ff3c02efb80 1 --2- 172.21.15.52:0/1262740634 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d4190d0f60 0x55d419147040 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:42.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.912+0000 7ff3c02efb80 1 --2- 172.21.15.52:0/1262740634 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d4190caf20 0x55d4190e6f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:42.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.912+0000 7ff38effd700 0 client.25681 ms_handle_reset on v2:172.21.15.188:6838/310444836 2024-06-10T02:58:42.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.912+0000 7ff3c02efb80 1 --2- 172.21.15.52:0/1262740634 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d4190d0640 0x55d4190e7450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:42.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.912+0000 7ff3c02efb80 1 -- 172.21.15.52:0/1262740634 >> 172.21.15.52:0/1262740634 conn(0x55d418ff5cc0 msgr2=0x55d4190d1340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:58:42.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.912+0000 7ff3c02efb80 1 -- 172.21.15.52:0/1262740634 shutdown_connections 2024-06-10T02:58:42.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.912+0000 7ff3c02efb80 1 -- 172.21.15.52:0/1262740634 wait complete. 2024-06-10T02:58:42.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.912+0000 7ff3c02efb80 20 client.25681 trim_cache size 0 max 16384 2024-06-10T02:58:42.919 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-10T02:58:42.919 INFO:teuthology.orchestra.run.smithi052.stdout: "status": "no active scrubs running", 2024-06-10T02:58:42.919 INFO:teuthology.orchestra.run.smithi052.stdout: "scrubs": {} 2024-06-10T02:58:42.919 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-10T02:58:42.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.913+0000 7ff3b5a6c700 1 -- 172.21.15.52:0/3525598263 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7ff3980a2c20 msgr2=0x7ff3980a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:58:42.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.913+0000 7ff3b5a6c700 1 --2- 172.21.15.52:0/3525598263 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7ff3980a2c20 0x7ff3980a50e0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7ff3b0108bb0 tx=0x7ff3a4009260 comp rx=0 tx=0).stop 2024-06-10T02:58:42.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.913+0000 7ff3b5a6c700 1 -- 172.21.15.52:0/3525598263 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff3b0102770 msgr2=0x7ff3b0074790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:58:42.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.913+0000 7ff3b5a6c700 1 --2- 172.21.15.52:0/3525598263 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff3b0102770 0x7ff3b0074790 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7ff3a000d950 tx=0x7ff3a000dc60 comp rx=0 tx=0).stop 2024-06-10T02:58:42.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.913+0000 7ff3b5a6c700 1 -- 172.21.15.52:0/3525598263 shutdown_connections 2024-06-10T02:58:42.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.913+0000 7ff3b5a6c700 1 --2- 172.21.15.52:0/3525598263 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff3b0107860 0x7ff3b0075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:42.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.913+0000 7ff3b5a6c700 1 --2- 172.21.15.52:0/3525598263 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7ff3980a2c20 0x7ff3980a50e0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:42.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.913+0000 7ff3b5a6c700 1 --2- 172.21.15.52:0/3525598263 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff3b0102770 0x7ff3b0074790 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:42.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.913+0000 7ff3b5a6c700 1 --2- 172.21.15.52:0/3525598263 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff3b0103120 0x7ff3b0074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:42.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.913+0000 7ff3b5a6c700 1 -- 172.21.15.52:0/3525598263 >> 172.21.15.52:0/3525598263 conn(0x7ff3b0079600 msgr2=0x7ff3b0100480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:58:42.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.913+0000 7ff3b5a6c700 1 -- 172.21.15.52:0/3525598263 shutdown_connections 2024-06-10T02:58:42.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:42.913+0000 7ff3b5a6c700 1 -- 172.21.15.52:0/3525598263 wait complete. 2024-06-10T02:58:43.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:58:42 smithi120 ceph-mon[36634]: pgmap v569: 129 pgs: 129 active+clean; 442 MiB data, 962 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.2 MiB/s wr, 328 op/s 2024-06-10T02:58:43.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:58:42 smithi188 ceph-mon[36599]: pgmap v569: 129 pgs: 129 active+clean; 442 MiB data, 962 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.2 MiB/s wr, 328 op/s 2024-06-10T02:58:43.579 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-06-10T02:58:43.579 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T02:58:43.859 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:58:44.976 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:58:44 smithi052 ceph-mon[32362]: pgmap v570: 129 pgs: 129 active+clean; 445 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.3 MiB/s wr, 283 op/s 2024-06-10T02:58:45.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:58:44 smithi120 ceph-mon[36634]: pgmap v570: 129 pgs: 129 active+clean; 445 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.3 MiB/s wr, 283 op/s 2024-06-10T02:58:45.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:58:44 smithi188 ceph-mon[36599]: pgmap v570: 129 pgs: 129 active+clean; 445 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.3 MiB/s wr, 283 op/s 2024-06-10T02:58:45.734 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:45.734+0000 7f46394f0700 1 -- 172.21.15.52:0/1356873467 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f46341084e0 msgr2=0x7f46341088c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:58:45.735 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:45.734+0000 7f46394f0700 1 --2- 172.21.15.52:0/1356873467 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f46341084e0 0x7f46341088c0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f46240099c0 tx=0x7f4624009cd0 comp rx=0 tx=0).stop 2024-06-10T02:58:45.735 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:45.734+0000 7f46394f0700 1 -- 172.21.15.52:0/1356873467 shutdown_connections 2024-06-10T02:58:45.735 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:45.734+0000 7f46394f0700 1 --2- 172.21.15.52:0/1356873467 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f463406ab10 0x7f463410f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:45.735 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:45.735+0000 7f46394f0700 1 --2- 172.21.15.52:0/1356873467 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f46341084e0 0x7f46341088c0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:45.735 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:45.735+0000 7f46394f0700 1 --2- 172.21.15.52:0/1356873467 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f463406a170 0x7f463406a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:45.735 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:45.735+0000 7f46394f0700 1 -- 172.21.15.52:0/1356873467 >> 172.21.15.52:0/1356873467 conn(0x7f46340782c0 msgr2=0x7f46340786d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:58:45.735 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:45.735+0000 7f46394f0700 1 -- 172.21.15.52:0/1356873467 shutdown_connections 2024-06-10T02:58:45.735 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:45.735+0000 7f46394f0700 1 -- 172.21.15.52:0/1356873467 wait complete. 2024-06-10T02:58:45.737 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:45.735+0000 7f46394f0700 1 Processor -- start 2024-06-10T02:58:45.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:45.736+0000 7f46394f0700 1 -- start start 2024-06-10T02:58:45.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:45.736+0000 7f46394f0700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f463406a170 0x7f4634101460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:58:45.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:45.736+0000 7f46394f0700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f463406ab10 0x7f46341019a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:58:45.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:45.736+0000 7f46394f0700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f46341084e0 0x7f4634101ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:58:45.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:45.736+0000 7f46394f0700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f4634112720 con 0x7f463406a170 2024-06-10T02:58:45.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:45.736+0000 7f46394f0700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f46341125a0 con 0x7f463406ab10 2024-06-10T02:58:45.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:45.736+0000 7f46394f0700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f46341128a0 con 0x7f46341084e0 2024-06-10T02:58:45.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:45.736+0000 7f4632ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f463406a170 0x7f4634101460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:58:45.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:45.736+0000 7f46337fe700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f46341084e0 0x7f4634101ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:58:45.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:45.736+0000 7f4632ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f463406a170 0x7f4634101460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49036/0 (socket says 172.21.15.52:49036) 2024-06-10T02:58:45.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:45.736+0000 7f4632ffd700 1 -- 172.21.15.52:0/481823408 learned_addr learned my addr 172.21.15.52:0/481823408 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:58:45.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:45.736+0000 7f46337fe700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f46341084e0 0x7f4634101ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.52:54918/0 (socket says 172.21.15.52:54918) 2024-06-10T02:58:45.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:45.736+0000 7f46327fc700 1 --2- 172.21.15.52:0/481823408 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f463406ab10 0x7f46341019a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:58:45.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:45.736+0000 7f46337fe700 1 -- 172.21.15.52:0/481823408 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f463406ab10 msgr2=0x7f46341019a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:58:45.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:45.736+0000 7f46337fe700 1 --2- 172.21.15.52:0/481823408 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f463406ab10 0x7f46341019a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:45.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:45.736+0000 7f46337fe700 1 -- 172.21.15.52:0/481823408 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f463406a170 msgr2=0x7f4634101460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:58:45.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:45.736+0000 7f46337fe700 1 --2- 172.21.15.52:0/481823408 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f463406a170 0x7f4634101460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:45.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:45.736+0000 7f46337fe700 1 -- 172.21.15.52:0/481823408 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46341a7180 con 0x7f46341084e0 2024-06-10T02:58:45.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:45.737+0000 7f46337fe700 1 --2- 172.21.15.52:0/481823408 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f46341084e0 0x7f4634101ee0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f462c00efb0 tx=0x7f462c00c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:58:45.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:45.737+0000 7f461bfff700 1 -- 172.21.15.52:0/481823408 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f462c003a00 con 0x7f46341084e0 2024-06-10T02:58:45.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:45.737+0000 7f46394f0700 1 -- 172.21.15.52:0/481823408 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46341a7470 con 0x7f46341084e0 2024-06-10T02:58:45.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:45.737+0000 7f46394f0700 1 -- 172.21.15.52:0/481823408 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46341a79b0 con 0x7f46341084e0 2024-06-10T02:58:45.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:45.737+0000 7f461bfff700 1 -- 172.21.15.52:0/481823408 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f462c003ba0 con 0x7f46341084e0 2024-06-10T02:58:45.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:45.737+0000 7f461bfff700 1 -- 172.21.15.52:0/481823408 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f462c004850 con 0x7f46341084e0 2024-06-10T02:58:45.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:45.738+0000 7f46394f0700 1 -- 172.21.15.52:0/481823408 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4620005330 con 0x7f46341084e0 2024-06-10T02:58:45.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:45.739+0000 7f461bfff700 1 -- 172.21.15.52:0/481823408 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f462c012050 con 0x7f46341084e0 2024-06-10T02:58:45.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:45.740+0000 7f461bfff700 1 --2- 172.21.15.52:0/481823408 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f461c0a2c20 0x7f461c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:58:45.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:45.740+0000 7f4632ffd700 1 --2- 172.21.15.52:0/481823408 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f461c0a2c20 0x7f461c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:58:45.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:45.740+0000 7f461bfff700 1 -- 172.21.15.52:0/481823408 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f462c0cee20 con 0x7f46341084e0 2024-06-10T02:58:45.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:45.741+0000 7f4632ffd700 1 --2- 172.21.15.52:0/481823408 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f461c0a2c20 0x7f461c0a50e0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f4624009750 tx=0x7f4624005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:58:45.744 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:45.745+0000 7f461bfff700 1 -- 172.21.15.52:0/481823408 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f462c022060 con 0x7f46341084e0 2024-06-10T02:58:46.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:46.051+0000 7f46394f0700 1 -- 172.21.15.52:0/481823408 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4620005620 con 0x7f46341084e0 2024-06-10T02:58:46.052 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:46.052+0000 7f461bfff700 1 -- 172.21.15.52:0/481823408 <== mon.1 v2:172.21.15.188:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11945 (secure 0 0 0) 0x7f462c010040 con 0x7f46341084e0 2024-06-10T02:58:46.052 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-10T02:58:46.052 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/889673247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":889673247},{"type":"v1","addr":"172.21.15.52:6835","nonce":889673247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/3615663511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":3615663511},{"type":"v1","addr":"172.21.15.52:6837","nonce":3615663511}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T02:50:03.889766+0000","modified":"2024-06-10T02:58:01.195103+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24484,"mds_1":24469,"mds_2":24463,"mds_3":24449,"mds_4":24428},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14571":{"gid":14571,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6839/3059599042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":3059599042},{"type":"v1","addr":"172.21.15.52:6839","nonce":3059599042}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14583":{"gid":14583,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6841/3039275521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":3039275521},{"type":"v1","addr":"172.21.15.52:6841","nonce":3039275521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24383":{"gid":24383,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6833/2609002896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":2609002896},{"type":"v1","addr":"172.21.15.120:6833","nonce":2609002896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6835/1996370288","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":1996370288},{"type":"v1","addr":"172.21.15.120:6835","nonce":1996370288}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6833/529564979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":529564979},{"type":"v1","addr":"172.21.15.188:6833","nonce":529564979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":25,"addr":"172.21.15.120:6837/4132440952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":4132440952},{"type":"v1","addr":"172.21.15.120:6837","nonce":4132440952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"b","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.120:6839/1672436487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1672436487},{"type":"v1","addr":"172.21.15.120:6839","nonce":1672436487}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":2,"incarnation":22,"state":"up:active","state_seq":26,"addr":"172.21.15.188:6835/3904544259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":3904544259},{"type":"v1","addr":"172.21.15.188:6835","nonce":3904544259}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.188:6837/819317726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":819317726},{"type":"v1","addr":"172.21.15.188:6837","nonce":819317726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.188:6839/310444836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":310444836},{"type":"v1","addr":"172.21.15.188:6839","nonce":310444836}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24484,"qdb_cluster":[24469,24484,24463,24428,24449]},"id":1}]} 2024-06-10T02:58:46.054 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:46.054+0000 7f46394f0700 1 -- 172.21.15.52:0/481823408 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f461c0a2c20 msgr2=0x7f461c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:58:46.054 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:46.054+0000 7f46394f0700 1 --2- 172.21.15.52:0/481823408 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f461c0a2c20 0x7f461c0a50e0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f4624009750 tx=0x7f4624005960 comp rx=0 tx=0).stop 2024-06-10T02:58:46.054 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:46.054+0000 7f46394f0700 1 -- 172.21.15.52:0/481823408 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f46341084e0 msgr2=0x7f4634101ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:58:46.054 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:46.054+0000 7f46394f0700 1 --2- 172.21.15.52:0/481823408 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f46341084e0 0x7f4634101ee0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f462c00efb0 tx=0x7f462c00c3a0 comp rx=0 tx=0).stop 2024-06-10T02:58:46.054 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:46.054+0000 7f46394f0700 1 -- 172.21.15.52:0/481823408 shutdown_connections 2024-06-10T02:58:46.054 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:46.054+0000 7f46394f0700 1 --2- 172.21.15.52:0/481823408 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f46341084e0 0x7f4634101ee0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:46.054 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:46.054+0000 7f46394f0700 1 --2- 172.21.15.52:0/481823408 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f461c0a2c20 0x7f461c0a50e0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:46.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:46.054+0000 7f46394f0700 1 --2- 172.21.15.52:0/481823408 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f463406a170 0x7f4634101460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:46.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:46.054+0000 7f46394f0700 1 --2- 172.21.15.52:0/481823408 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f463406ab10 0x7f46341019a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:46.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:46.054+0000 7f46394f0700 1 -- 172.21.15.52:0/481823408 >> 172.21.15.52:0/481823408 conn(0x7f46340782c0 msgr2=0x7f46340ff4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:58:46.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:46.054+0000 7f46394f0700 1 -- 172.21.15.52:0/481823408 shutdown_connections 2024-06-10T02:58:46.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:46.054+0000 7f46394f0700 1 -- 172.21.15.52:0/481823408 wait complete. 2024-06-10T02:58:46.055 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 34 2024-06-10T02:58:46.781 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-06-10T02:58:47.120 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:58:47.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:58:46 smithi120 ceph-mon[36634]: pgmap v571: 129 pgs: 129 active+clean; 449 MiB data, 968 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.7 MiB/s wr, 194 op/s 2024-06-10T02:58:47.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:58:46 smithi120 ceph-mon[36634]: from='client.? 172.21.15.52:0/481823408' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T02:58:47.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:58:46 smithi188 ceph-mon[36599]: pgmap v571: 129 pgs: 129 active+clean; 449 MiB data, 968 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.7 MiB/s wr, 194 op/s 2024-06-10T02:58:47.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:58:46 smithi188 ceph-mon[36599]: from='client.? 172.21.15.52:0/481823408' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T02:58:47.226 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:58:46 smithi052 ceph-mon[32362]: pgmap v571: 129 pgs: 129 active+clean; 449 MiB data, 968 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.7 MiB/s wr, 194 op/s 2024-06-10T02:58:47.226 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:58:46 smithi052 ceph-mon[32362]: from='client.? 172.21.15.52:0/481823408' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T02:58:47.246 INFO:tasks.workunit.client.0.smithi052.stdout: 74 0 0 12 0 11 0 2024-06-10T02:58:48.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.862+0000 7fabfb6af700 1 -- 172.21.15.52:0/223425308 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fabf41066b0 msgr2=0x7fabf410e280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:58:48.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.862+0000 7fabfb6af700 1 --2- 172.21.15.52:0/223425308 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fabf41066b0 0x7fabf410e280 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fabec00b0d0 tx=0x7fabec00b3e0 comp rx=0 tx=0).stop 2024-06-10T02:58:48.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.862+0000 7fabfb6af700 1 -- 172.21.15.52:0/223425308 shutdown_connections 2024-06-10T02:58:48.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.862+0000 7fabfb6af700 1 --2- 172.21.15.52:0/223425308 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fabf41066b0 0x7fabf410e280 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:48.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.862+0000 7fabfb6af700 1 --2- 172.21.15.52:0/223425308 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fabf4101680 0x7fabf4101a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:48.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.862+0000 7fabfb6af700 1 --2- 172.21.15.52:0/223425308 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fabf4102030 0x7fabf4105fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:48.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.862+0000 7fabfb6af700 1 -- 172.21.15.52:0/223425308 >> 172.21.15.52:0/223425308 conn(0x7fabf40773e0 msgr2=0x7fabf40777f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:58:48.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.862+0000 7fabfb6af700 1 -- 172.21.15.52:0/223425308 shutdown_connections 2024-06-10T02:58:48.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.862+0000 7fabfb6af700 1 -- 172.21.15.52:0/223425308 wait complete. 2024-06-10T02:58:48.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.863+0000 7fabfb6af700 1 Processor -- start 2024-06-10T02:58:48.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.863+0000 7fabfb6af700 1 -- start start 2024-06-10T02:58:48.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.863+0000 7fabfb6af700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fabf4101680 0x7fabf4103070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:58:48.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.863+0000 7fabfb6af700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fabf4102030 0x7fabf41035b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:58:48.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.863+0000 7fabfb6af700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fabf41066b0 0x7fabf4103af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:58:48.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.863+0000 7fabfb6af700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fabf41116c0 con 0x7fabf4101680 2024-06-10T02:58:48.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.863+0000 7fabfb6af700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fabf4111540 con 0x7fabf41066b0 2024-06-10T02:58:48.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.863+0000 7fabfb6af700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fabf4111840 con 0x7fabf4102030 2024-06-10T02:58:48.866 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.863+0000 7fabf8c4a700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fabf4102030 0x7fabf41035b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:58:48.866 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.863+0000 7fabf8c4a700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fabf4102030 0x7fabf41035b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.52:54938/0 (socket says 172.21.15.52:54938) 2024-06-10T02:58:48.866 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.863+0000 7fabf8c4a700 1 -- 172.21.15.52:0/605185193 learned_addr learned my addr 172.21.15.52:0/605185193 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:58:48.866 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.863+0000 7fabf9c4c700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fabf41066b0 0x7fabf4103af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:58:48.866 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.864+0000 7fabf944b700 1 --2- 172.21.15.52:0/605185193 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fabf4101680 0x7fabf4103070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:58:48.866 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.864+0000 7fabf8c4a700 1 -- 172.21.15.52:0/605185193 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fabf41066b0 msgr2=0x7fabf4103af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:58:48.866 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.864+0000 7fabf8c4a700 1 --2- 172.21.15.52:0/605185193 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fabf41066b0 0x7fabf4103af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:48.866 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.864+0000 7fabf8c4a700 1 -- 172.21.15.52:0/605185193 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fabf4101680 msgr2=0x7fabf4103070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:58:48.866 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.864+0000 7fabf8c4a700 1 --2- 172.21.15.52:0/605185193 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fabf4101680 0x7fabf4103070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:48.866 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.864+0000 7fabf8c4a700 1 -- 172.21.15.52:0/605185193 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabf41a7120 con 0x7fabf4102030 2024-06-10T02:58:48.866 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.864+0000 7fabf8c4a700 1 --2- 172.21.15.52:0/605185193 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fabf4102030 0x7fabf41035b0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fabe400e9f0 tx=0x7fabe400ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:58:48.866 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.864+0000 7fabea7fc700 1 -- 172.21.15.52:0/605185193 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabe4004a30 con 0x7fabf4102030 2024-06-10T02:58:48.867 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.864+0000 7fabea7fc700 1 -- 172.21.15.52:0/605185193 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fabe400cca0 con 0x7fabf4102030 2024-06-10T02:58:48.867 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.864+0000 7fabea7fc700 1 -- 172.21.15.52:0/605185193 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabe40056c0 con 0x7fabf4102030 2024-06-10T02:58:48.867 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.864+0000 7fabfb6af700 1 -- 172.21.15.52:0/605185193 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabf41a7410 con 0x7fabf4102030 2024-06-10T02:58:48.867 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.864+0000 7fabfb6af700 1 -- 172.21.15.52:0/605185193 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabf41a7900 con 0x7fabf4102030 2024-06-10T02:58:48.867 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.868+0000 7fabea7fc700 1 -- 172.21.15.52:0/605185193 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fabe400c7c0 con 0x7fabf4102030 2024-06-10T02:58:48.868 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.868+0000 7fabea7fc700 1 --2- 172.21.15.52:0/605185193 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fabe00a2ca0 0x7fabe00a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:58:48.869 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.869+0000 7fabea7fc700 1 -- 172.21.15.52:0/605185193 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7fabe40cf520 con 0x7fabf4102030 2024-06-10T02:58:48.869 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.869+0000 7fabf944b700 1 --2- 172.21.15.52:0/605185193 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fabe00a2ca0 0x7fabe00a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:58:48.869 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.869+0000 7fac05f32b80 1 Processor -- start 2024-06-10T02:58:48.869 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.869+0000 7fac05f32b80 1 -- start start 2024-06-10T02:58:48.870 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.869+0000 7fac05f32b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55967daf9f60 0x55967dafa340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:58:48.870 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.869+0000 7fac05f32b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55967dafa880 0x55967daf4260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:58:48.870 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.869+0000 7fac05f32b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55967daf47a0 0x55967daf4bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:58:48.870 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.869+0000 7fac05f32b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55967d9e6f60 con 0x55967dafa880 2024-06-10T02:58:48.870 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.869+0000 7fac05f32b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55967da45850 con 0x55967daf47a0 2024-06-10T02:58:48.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.869+0000 7fac05f32b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55967d9ad460 con 0x55967daf9f60 2024-06-10T02:58:48.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.869+0000 7fabf944b700 1 --2- 172.21.15.52:0/605185193 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fabe00a2ca0 0x7fabe00a5160 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fabf0009c60 tx=0x7fabf0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:58:48.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.869+0000 7fabf8c4a700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55967dafa880 0x55967daf4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:58:48.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.870+0000 7fabf8c4a700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55967dafa880 0x55967daf4260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49056/0 (socket says 172.21.15.52:49056) 2024-06-10T02:58:48.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.870+0000 7fabf8c4a700 1 -- 172.21.15.52:0/2114625524 learned_addr learned my addr 172.21.15.52:0/2114625524 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:58:48.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.870+0000 7fabf944b700 1 --2- 172.21.15.52:0/2114625524 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55967daf9f60 0x55967dafa340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:58:48.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.870+0000 7fabf9c4c700 1 --2- 172.21.15.52:0/2114625524 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55967daf47a0 0x55967daf4bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:58:48.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.870+0000 7fabf8c4a700 1 -- 172.21.15.52:0/2114625524 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55967daf9f60 msgr2=0x55967dafa340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:58:48.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.870+0000 7fabf8c4a700 1 --2- 172.21.15.52:0/2114625524 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55967daf9f60 0x55967dafa340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:48.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.870+0000 7fabf8c4a700 1 -- 172.21.15.52:0/2114625524 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55967daf47a0 msgr2=0x55967daf4bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:58:48.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.870+0000 7fabf8c4a700 1 --2- 172.21.15.52:0/2114625524 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55967daf47a0 0x55967daf4bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:48.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.870+0000 7fabf8c4a700 1 -- 172.21.15.52:0/2114625524 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55967d9948d0 con 0x55967dafa880 2024-06-10T02:58:48.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.870+0000 7fabf944b700 1 --2- 172.21.15.52:0/2114625524 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55967daf9f60 0x55967dafa340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T02:58:48.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.870+0000 7fabf8c4a700 1 --2- 172.21.15.52:0/2114625524 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55967dafa880 0x55967daf4260 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fabe4098340 tx=0x7fabe409b5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cf292abff2cc910 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:58:48.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.870+0000 7fabdf7fe700 1 -- 172.21.15.52:0/2114625524 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabe40cf050 con 0x55967dafa880 2024-06-10T02:58:48.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.870+0000 7fabdf7fe700 1 -- 172.21.15.52:0/2114625524 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fabe409b970 con 0x55967dafa880 2024-06-10T02:58:48.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.870+0000 7fabdf7fe700 1 -- 172.21.15.52:0/2114625524 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabe40cf1f0 con 0x55967dafa880 2024-06-10T02:58:48.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.871+0000 7fac05f32b80 1 -- 172.21.15.52:0/2114625524 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55967dafa880 msgr2=0x55967daf4260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:58:48.875 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.871+0000 7fac05f32b80 1 --2- 172.21.15.52:0/2114625524 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55967dafa880 0x55967daf4260 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fabe4098340 tx=0x7fabe409b5d0 comp rx=0 tx=0).stop 2024-06-10T02:58:48.875 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.871+0000 7fac05f32b80 1 -- 172.21.15.52:0/2114625524 shutdown_connections 2024-06-10T02:58:48.875 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.871+0000 7fac05f32b80 1 --2- 172.21.15.52:0/2114625524 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55967daf47a0 0x55967daf4bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:48.875 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.871+0000 7fac05f32b80 1 --2- 172.21.15.52:0/2114625524 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55967daf9f60 0x55967dafa340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:48.875 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.871+0000 7fac05f32b80 1 --2- 172.21.15.52:0/2114625524 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55967dafa880 0x55967daf4260 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:48.875 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.871+0000 7fac05f32b80 1 -- 172.21.15.52:0/2114625524 >> 172.21.15.52:0/2114625524 conn(0x55967da21080 msgr2=0x55967daef220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:58:48.875 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.871+0000 7fac05f32b80 1 -- 172.21.15.52:0/2114625524 shutdown_connections 2024-06-10T02:58:48.875 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.871+0000 7fac05f32b80 1 -- 172.21.15.52:0/2114625524 wait complete. 2024-06-10T02:58:48.875 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.872+0000 7fac05f32b80 1 -- start start 2024-06-10T02:58:48.875 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.872+0000 7fac05f32b80 1 Processor -- start 2024-06-10T02:58:48.875 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.873+0000 7fac05f32b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55967daf47a0 0x55967db00590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:58:48.875 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.873+0000 7fac05f32b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55967daf9f60 0x55967dafd950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:58:48.875 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.873+0000 7fac05f32b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55967dafa880 0x55967dafde90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:58:48.876 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.873+0000 7fabf944b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55967daf47a0 0x55967db00590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:58:48.876 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.873+0000 7fac05f32b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55967d9e5fc0 con 0x55967daf47a0 2024-06-10T02:58:48.876 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.873+0000 7fabf8c4a700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55967daf9f60 0x55967dafd950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:58:48.876 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.873+0000 7fabf944b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55967daf47a0 0x55967db00590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49064/0 (socket says 172.21.15.52:49064) 2024-06-10T02:58:48.876 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.873+0000 7fabf944b700 1 -- 172.21.15.52:0/1495606063 learned_addr learned my addr 172.21.15.52:0/1495606063 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:58:48.876 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.873+0000 7fac05f32b80 1 -- 172.21.15.52:0/1495606063 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55967da41810 con 0x55967dafa880 2024-06-10T02:58:48.876 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.873+0000 7fac05f32b80 1 -- 172.21.15.52:0/1495606063 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55967db01260 con 0x55967daf9f60 2024-06-10T02:58:48.876 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.873+0000 7fac05f32b80 10 client.0 _ll_register_callbacks cb 0x55967d9db5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T02:58:48.876 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.873+0000 7fabf9c4c700 1 --2- 172.21.15.52:0/1495606063 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55967dafa880 0x55967dafde90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:58:48.876 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.873+0000 7fabf944b700 1 -- 172.21.15.52:0/1495606063 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55967daf9f60 msgr2=0x55967dafd950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:58:48.876 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.873+0000 7fabf944b700 1 --2- 172.21.15.52:0/1495606063 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55967daf9f60 0x55967dafd950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:48.876 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.873+0000 7fabf944b700 1 -- 172.21.15.52:0/1495606063 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55967dafa880 msgr2=0x55967dafde90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:58:48.876 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.873+0000 7fabf944b700 1 --2- 172.21.15.52:0/1495606063 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55967dafa880 0x55967dafde90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:48.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.873+0000 7fabf944b700 1 -- 172.21.15.52:0/1495606063 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55967da0b750 con 0x55967daf47a0 2024-06-10T02:58:48.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.873+0000 7fabf944b700 1 --2- 172.21.15.52:0/1495606063 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55967daf47a0 0x55967db00590 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fabf0009b20 tx=0x7fabf00047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e03454ba967fa586 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:58:48.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.873+0000 7fabdcff9700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-10T02:58:48.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.874+0000 7fabdcff9700 1 -- 172.21.15.52:0/1495606063 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabf000e9d0 con 0x55967daf47a0 2024-06-10T02:58:48.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.874+0000 7fabdcff9700 1 -- 172.21.15.52:0/1495606063 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7fabf0009040 con 0x55967daf47a0 2024-06-10T02:58:48.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.874+0000 7fac05f32b80 1 -- 172.21.15.52:0/1495606063 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fabe409b970 con 0x55967daf47a0 2024-06-10T02:58:48.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.874+0000 7fabdcff9700 1 -- 172.21.15.52:0/1495606063 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fabf000eb70 con 0x55967daf47a0 2024-06-10T02:58:48.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.874+0000 7fac05f32b80 10 client.16125 fetch_fsmap learned FSMap version 34 2024-06-10T02:58:48.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.874+0000 7fac05f32b80 1 -- 172.21.15.52:0/1495606063 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55967db14240 con 0x55967daf47a0 2024-06-10T02:58:48.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.874+0000 7fabdcff9700 1 -- 172.21.15.52:0/1495606063 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fabe409b970 con 0x55967daf47a0 2024-06-10T02:58:48.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.874+0000 7fabdcff9700 1 -- 172.21.15.52:0/1495606063 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 34) ==== 8378+0+0 (secure 0 0 0) 0x7fabf001c410 con 0x55967daf47a0 2024-06-10T02:58:48.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.874+0000 7fac05f32b80 10 client.16125 fetch_fsmap finished waiting for FSMap version 34 2024-06-10T02:58:48.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.874+0000 7fac05f32b80 10 client.16125 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-06-10T02:58:48.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.874+0000 7fac05f32b80 20 client.16125 populate_metadata read hostname 'smithi052' 2024-06-10T02:58:48.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.874+0000 7fac05f32b80 1 --2- 172.21.15.52:0/1495606063 >> [v2:172.21.15.120:6836/4132440952,v1:172.21.15.120:6837/4132440952] conn(0x55967db168d0 0x55967db18d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:58:48.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.875+0000 7fac05f32b80 4 client.16125 mds_command: new command op to 24428 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T02:58:48.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.875+0000 7fac05f32b80 1 -- 172.21.15.52:0/1495606063 --> [v2:172.21.15.120:6836/4132440952,v1:172.21.15.120:6837/4132440952] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55967da52d50 con 0x55967db168d0 2024-06-10T02:58:48.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.875+0000 7fabf8c4a700 1 --2- 172.21.15.52:0/1495606063 >> [v2:172.21.15.120:6836/4132440952,v1:172.21.15.120:6837/4132440952] conn(0x55967db168d0 0x55967db18d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:58:48.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.875+0000 7fabdcff9700 10 client.16125 ms_handle_connect on v2:172.21.15.120:6836/4132440952 2024-06-10T02:58:48.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.875+0000 7fabf8c4a700 1 --2- 172.21.15.52:0/1495606063 >> [v2:172.21.15.120:6836/4132440952,v1:172.21.15.120:6837/4132440952] conn(0x55967db168d0 0x55967db18d40 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=42cb18b0bc58bfa1 server_cookie=21608db58e6d7c80 in_seq=0 out_seq=0 2024-06-10T02:58:48.880 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.876+0000 7fabdcff9700 1 -- 172.21.15.52:0/1495606063 <== mds.4 v2:172.21.15.120:6836/4132440952 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55967da52d50 con 0x55967db168d0 2024-06-10T02:58:48.880 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.876+0000 7fabdcff9700 10 client.16125 handle_command_reply: tid=0 2024-06-10T02:58:48.880 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.876+0000 7fac05f32b80 1 client.16125 shutdown 2024-06-10T02:58:48.880 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.876+0000 7fac05f32b80 1 -- 172.21.15.52:0/1495606063 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55967daf47a0 msgr2=0x55967db00590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:58:48.880 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.876+0000 7fac05f32b80 1 --2- 172.21.15.52:0/1495606063 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55967daf47a0 0x55967db00590 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fabf0009b20 tx=0x7fabf00047c0 comp rx=0 tx=0).stop 2024-06-10T02:58:48.880 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.876+0000 7fac05f32b80 1 -- 172.21.15.52:0/1495606063 shutdown_connections 2024-06-10T02:58:48.880 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.876+0000 7fac05f32b80 1 --2- 172.21.15.52:0/1495606063 >> [v2:172.21.15.120:6836/4132440952,v1:172.21.15.120:6837/4132440952] conn(0x55967db168d0 0x55967db18d40 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:48.880 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.876+0000 7fac05f32b80 1 --2- 172.21.15.52:0/1495606063 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55967dafa880 0x55967dafde90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:48.880 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.876+0000 7fabdcff9700 0 client.16125 ms_handle_reset on v2:172.21.15.120:6836/4132440952 2024-06-10T02:58:48.880 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.876+0000 7fac05f32b80 1 --2- 172.21.15.52:0/1495606063 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55967daf47a0 0x55967db00590 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:48.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.876+0000 7fac05f32b80 1 --2- 172.21.15.52:0/1495606063 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55967daf9f60 0x55967dafd950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:48.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.876+0000 7fac05f32b80 1 -- 172.21.15.52:0/1495606063 >> 172.21.15.52:0/1495606063 conn(0x55967da21080 msgr2=0x55967daee700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:58:48.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.876+0000 7fac05f32b80 1 -- 172.21.15.52:0/1495606063 shutdown_connections 2024-06-10T02:58:48.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.876+0000 7fac05f32b80 1 -- 172.21.15.52:0/1495606063 wait complete. 2024-06-10T02:58:48.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.876+0000 7fac05f32b80 20 client.16125 trim_cache size 0 max 16384 2024-06-10T02:58:48.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.893+0000 7fac05f32b80 1 Processor -- start 2024-06-10T02:58:48.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.893+0000 7fac05f32b80 1 -- start start 2024-06-10T02:58:48.893 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:58:48 smithi052 ceph-mon[32362]: pgmap v572: 129 pgs: 129 active+clean; 455 MiB data, 1001 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.7 MiB/s wr, 307 op/s 2024-06-10T02:58:48.893 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:58:48 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T02:58:48.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.893+0000 7fac05f32b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55967daf47a0 0x55967daf8050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:58:48.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.893+0000 7fac05f32b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55967daf9f60 0x55967db0a380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:58:48.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.893+0000 7fac05f32b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55967dafa880 0x55967dafb3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:58:48.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.893+0000 7fac05f32b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fabf41ac210 con 0x55967dafa880 2024-06-10T02:58:48.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.893+0000 7fac05f32b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fabf41ade10 con 0x55967daf47a0 2024-06-10T02:58:48.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.893+0000 7fac05f32b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fabf41ac8e0 con 0x55967daf9f60 2024-06-10T02:58:48.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.893+0000 7fabf944b700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55967daf47a0 0x55967daf8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:58:48.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.893+0000 7fabf944b700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55967daf47a0 0x55967daf8050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.52:59518/0 (socket says 172.21.15.52:59518) 2024-06-10T02:58:48.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.893+0000 7fabf944b700 1 -- 172.21.15.52:0/1237655452 learned_addr learned my addr 172.21.15.52:0/1237655452 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:58:48.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.893+0000 7fabf8c4a700 1 --2- 172.21.15.52:0/1237655452 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55967daf9f60 0x55967db0a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:58:48.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.893+0000 7fabf9c4c700 1 --2- 172.21.15.52:0/1237655452 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55967dafa880 0x55967dafb3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:58:48.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.893+0000 7fabf944b700 1 -- 172.21.15.52:0/1237655452 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55967daf9f60 msgr2=0x55967db0a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:58:48.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.893+0000 7fabf944b700 1 --2- 172.21.15.52:0/1237655452 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55967daf9f60 0x55967db0a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:48.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.893+0000 7fabf944b700 1 -- 172.21.15.52:0/1237655452 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55967dafa880 msgr2=0x55967dafb3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:58:48.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.893+0000 7fabf944b700 1 --2- 172.21.15.52:0/1237655452 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55967dafa880 0x55967dafb3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:48.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.893+0000 7fabf944b700 1 -- 172.21.15.52:0/1237655452 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55967daefe10 con 0x55967daf47a0 2024-06-10T02:58:48.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.894+0000 7fabf944b700 1 --2- 172.21.15.52:0/1237655452 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55967daf47a0 0x55967daf8050 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fabf0011b30 tx=0x7fabf000e690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b2db2e8cdc306069 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:58:48.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.894+0000 7fabdeffd700 1 -- 172.21.15.52:0/1237655452 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabf000ed10 con 0x55967daf47a0 2024-06-10T02:58:48.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.894+0000 7fabdeffd700 1 -- 172.21.15.52:0/1237655452 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fabf0004520 con 0x55967daf47a0 2024-06-10T02:58:48.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.894+0000 7fabdeffd700 1 -- 172.21.15.52:0/1237655452 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabf0003680 con 0x55967daf47a0 2024-06-10T02:58:48.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.895+0000 7fac05f32b80 1 -- 172.21.15.52:0/1237655452 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55967daf47a0 msgr2=0x55967daf8050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:58:48.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.895+0000 7fac05f32b80 1 --2- 172.21.15.52:0/1237655452 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55967daf47a0 0x55967daf8050 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fabf0011b30 tx=0x7fabf000e690 comp rx=0 tx=0).stop 2024-06-10T02:58:48.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.895+0000 7fac05f32b80 1 -- 172.21.15.52:0/1237655452 shutdown_connections 2024-06-10T02:58:48.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.895+0000 7fac05f32b80 1 --2- 172.21.15.52:0/1237655452 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55967dafa880 0x55967dafb3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:48.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.895+0000 7fac05f32b80 1 --2- 172.21.15.52:0/1237655452 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55967daf47a0 0x55967daf8050 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:48.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.895+0000 7fac05f32b80 1 --2- 172.21.15.52:0/1237655452 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55967daf9f60 0x55967db0a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:48.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.895+0000 7fac05f32b80 1 -- 172.21.15.52:0/1237655452 >> 172.21.15.52:0/1237655452 conn(0x55967da21080 msgr2=0x55967dafaca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:58:48.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.895+0000 7fac05f32b80 1 -- 172.21.15.52:0/1237655452 shutdown_connections 2024-06-10T02:58:48.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.895+0000 7fac05f32b80 1 -- 172.21.15.52:0/1237655452 wait complete. 2024-06-10T02:58:48.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.896+0000 7fac05f32b80 1 -- start start 2024-06-10T02:58:48.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.896+0000 7fac05f32b80 1 Processor -- start 2024-06-10T02:58:48.899 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.896+0000 7fac05f32b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55967daf47a0 0x55967db067d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:58:48.899 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.896+0000 7fac05f32b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55967daf9f60 0x55967db06d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:58:48.899 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.896+0000 7fac05f32b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55967dafa880 0x55967db1dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:58:48.899 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.896+0000 7fac05f32b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fabf41ad740 con 0x55967dafa880 2024-06-10T02:58:48.899 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.896+0000 7fac05f32b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fabf41ad070 con 0x55967daf47a0 2024-06-10T02:58:48.899 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.896+0000 7fac05f32b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fabf41aa1b0 con 0x55967daf9f60 2024-06-10T02:58:48.899 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.896+0000 7fac05f32b80 10 client.0 _ll_register_callbacks cb 0x55967d9db5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T02:58:48.899 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.896+0000 7fabf8c4a700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55967daf9f60 0x55967db06d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:58:48.899 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.896+0000 7fabf8c4a700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55967daf9f60 0x55967db06d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.52:54970/0 (socket says 172.21.15.52:54970) 2024-06-10T02:58:48.899 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.897+0000 7fabf8c4a700 1 -- 172.21.15.52:0/661604323 learned_addr learned my addr 172.21.15.52:0/661604323 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:58:48.899 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.897+0000 7fabf944b700 1 --2- 172.21.15.52:0/661604323 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55967daf47a0 0x55967db067d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:58:48.899 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.897+0000 7fabf9c4c700 1 --2- 172.21.15.52:0/661604323 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55967dafa880 0x55967db1dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:58:48.899 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.897+0000 7fabf8c4a700 1 -- 172.21.15.52:0/661604323 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55967daf47a0 msgr2=0x55967db067d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:58:48.900 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.897+0000 7fabf8c4a700 1 --2- 172.21.15.52:0/661604323 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55967daf47a0 0x55967db067d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:48.900 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.897+0000 7fabf8c4a700 1 -- 172.21.15.52:0/661604323 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55967dafa880 msgr2=0x55967db1dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:58:48.900 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.897+0000 7fabf8c4a700 1 --2- 172.21.15.52:0/661604323 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55967dafa880 0x55967db1dd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:48.900 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.897+0000 7fabf8c4a700 1 -- 172.21.15.52:0/661604323 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55967db1e520 con 0x55967daf9f60 2024-06-10T02:58:48.900 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.897+0000 7fabf944b700 1 --2- 172.21.15.52:0/661604323 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55967daf47a0 0x55967db067d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T02:58:48.900 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.897+0000 7fabf8c4a700 1 --2- 172.21.15.52:0/661604323 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55967daf9f60 0x55967db06d10 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fabe4009230 tx=0x7fabe4005070 comp rx=0 tx=0).ready entity=mon.1 client_cookie=62b52ff63fec6c9a server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:58:48.900 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.897+0000 7fabdcff9700 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-06-10T02:58:48.900 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.897+0000 7fabdcff9700 1 -- 172.21.15.52:0/661604323 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabe4081070 con 0x55967daf9f60 2024-06-10T02:58:48.900 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.897+0000 7fac05f32b80 1 -- 172.21.15.52:0/661604323 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fabf0004520 con 0x55967daf9f60 2024-06-10T02:58:48.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.897+0000 7fabdcff9700 1 -- 172.21.15.52:0/661604323 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7fabe4004cf0 con 0x55967daf9f60 2024-06-10T02:58:48.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.898+0000 7fabdcff9700 1 -- 172.21.15.52:0/661604323 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fabe40cf040 con 0x55967daf9f60 2024-06-10T02:58:48.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.898+0000 7fac05f32b80 10 client.25705 fetch_fsmap learned FSMap version 34 2024-06-10T02:58:48.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.898+0000 7fac05f32b80 1 -- 172.21.15.52:0/661604323 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55967db811e0 con 0x55967daf9f60 2024-06-10T02:58:48.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.898+0000 7fabdcff9700 1 -- 172.21.15.52:0/661604323 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fabf0004520 con 0x55967daf9f60 2024-06-10T02:58:48.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.898+0000 7fabdcff9700 1 -- 172.21.15.52:0/661604323 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 34) ==== 8378+0+0 (secure 0 0 0) 0x7fabe4084020 con 0x55967daf9f60 2024-06-10T02:58:48.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.898+0000 7fac05f32b80 10 client.25705 fetch_fsmap finished waiting for FSMap version 34 2024-06-10T02:58:48.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.898+0000 7fac05f32b80 10 client.25705 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-06-10T02:58:48.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.898+0000 7fac05f32b80 20 client.25705 populate_metadata read hostname 'smithi052' 2024-06-10T02:58:48.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.898+0000 7fac05f32b80 1 --2- 172.21.15.52:0/661604323 >> [v2:172.21.15.120:6836/4132440952,v1:172.21.15.120:6837/4132440952] conn(0x55967db168d0 0x55967db833c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:58:48.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.898+0000 7fac05f32b80 4 client.25705 mds_command: new command op to 24428 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T02:58:48.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.898+0000 7fac05f32b80 1 -- 172.21.15.52:0/661604323 --> [v2:172.21.15.120:6836/4132440952,v1:172.21.15.120:6837/4132440952] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55967da4d2d0 con 0x55967db168d0 2024-06-10T02:58:48.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.898+0000 7fabf944b700 1 --2- 172.21.15.52:0/661604323 >> [v2:172.21.15.120:6836/4132440952,v1:172.21.15.120:6837/4132440952] conn(0x55967db168d0 0x55967db833c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:58:48.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.899+0000 7fabdcff9700 10 client.25705 ms_handle_connect on v2:172.21.15.120:6836/4132440952 2024-06-10T02:58:48.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.899+0000 7fabf944b700 1 --2- 172.21.15.52:0/661604323 >> [v2:172.21.15.120:6836/4132440952,v1:172.21.15.120:6837/4132440952] conn(0x55967db168d0 0x55967db833c0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=7b73616f3a37ff9b server_cookie=c0bc8d1232b45364 in_seq=0 out_seq=0 2024-06-10T02:58:48.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.899+0000 7fabdcff9700 1 -- 172.21.15.52:0/661604323 <== mds.4 v2:172.21.15.120:6836/4132440952 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55967da4d2d0 con 0x55967db168d0 2024-06-10T02:58:48.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.899+0000 7fabdcff9700 10 client.25705 handle_command_reply: tid=0 2024-06-10T02:58:48.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.899+0000 7fac05f32b80 1 client.25705 shutdown 2024-06-10T02:58:48.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.899+0000 7fac05f32b80 1 -- 172.21.15.52:0/661604323 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55967daf9f60 msgr2=0x55967db06d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:58:48.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.899+0000 7fac05f32b80 1 --2- 172.21.15.52:0/661604323 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55967daf9f60 0x55967db06d10 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fabe4009230 tx=0x7fabe4005070 comp rx=0 tx=0).stop 2024-06-10T02:58:48.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.899+0000 7fac05f32b80 1 -- 172.21.15.52:0/661604323 shutdown_connections 2024-06-10T02:58:48.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.899+0000 7fac05f32b80 1 --2- 172.21.15.52:0/661604323 >> [v2:172.21.15.120:6836/4132440952,v1:172.21.15.120:6837/4132440952] conn(0x55967db168d0 0x55967db833c0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:48.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.899+0000 7fac05f32b80 1 --2- 172.21.15.52:0/661604323 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55967dafa880 0x55967db1dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:48.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.899+0000 7fabdcff9700 0 client.25705 ms_handle_reset on v2:172.21.15.120:6836/4132440952 2024-06-10T02:58:48.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.899+0000 7fac05f32b80 1 --2- 172.21.15.52:0/661604323 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55967daf47a0 0x55967db067d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:48.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.899+0000 7fac05f32b80 1 --2- 172.21.15.52:0/661604323 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55967daf9f60 0x55967db06d10 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:48.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.899+0000 7fac05f32b80 1 -- 172.21.15.52:0/661604323 >> 172.21.15.52:0/661604323 conn(0x55967da21080 msgr2=0x55967daf9310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:58:48.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.900+0000 7fac05f32b80 1 -- 172.21.15.52:0/661604323 shutdown_connections 2024-06-10T02:58:48.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.900+0000 7fac05f32b80 1 -- 172.21.15.52:0/661604323 wait complete. 2024-06-10T02:58:48.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.900+0000 7fac05f32b80 20 client.25705 trim_cache size 0 max 16384 2024-06-10T02:58:48.904 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-06-10T02:58:48.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.900+0000 7fabfb6af700 1 -- 172.21.15.52:0/605185193 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fabe00a2ca0 msgr2=0x7fabe00a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:58:48.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.900+0000 7fabfb6af700 1 --2- 172.21.15.52:0/605185193 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fabe00a2ca0 0x7fabe00a5160 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fabf0009c60 tx=0x7fabf0009260 comp rx=0 tx=0).stop 2024-06-10T02:58:48.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.900+0000 7fabfb6af700 1 -- 172.21.15.52:0/605185193 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fabf4102030 msgr2=0x7fabf41035b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:58:48.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.900+0000 7fabfb6af700 1 --2- 172.21.15.52:0/605185193 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fabf4102030 0x7fabf41035b0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fabe400e9f0 tx=0x7fabe400ed00 comp rx=0 tx=0).stop 2024-06-10T02:58:48.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.900+0000 7fabfb6af700 1 -- 172.21.15.52:0/605185193 shutdown_connections 2024-06-10T02:58:48.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.900+0000 7fabfb6af700 1 --2- 172.21.15.52:0/605185193 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fabf41066b0 0x7fabf4103af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:48.906 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.900+0000 7fabfb6af700 1 --2- 172.21.15.52:0/605185193 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fabe00a2ca0 0x7fabe00a5160 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:48.906 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.900+0000 7fabfb6af700 1 --2- 172.21.15.52:0/605185193 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fabf4101680 0x7fabf4103070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:48.906 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.900+0000 7fabfb6af700 1 --2- 172.21.15.52:0/605185193 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fabf4102030 0x7fabf41035b0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:48.906 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.900+0000 7fabfb6af700 1 -- 172.21.15.52:0/605185193 >> 172.21.15.52:0/605185193 conn(0x7fabf40773e0 msgr2=0x7fabf406ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:58:48.906 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.900+0000 7fabfb6af700 1 -- 172.21.15.52:0/605185193 shutdown_connections 2024-06-10T02:58:48.906 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:48.900+0000 7fabfb6af700 1 -- 172.21.15.52:0/605185193 wait complete. 2024-06-10T02:58:49.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:58:48 smithi120 ceph-mon[36634]: pgmap v572: 129 pgs: 129 active+clean; 455 MiB data, 1001 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.7 MiB/s wr, 307 op/s 2024-06-10T02:58:49.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:58:48 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T02:58:49.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:58:48 smithi188 ceph-mon[36599]: pgmap v572: 129 pgs: 129 active+clean; 455 MiB data, 1001 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.7 MiB/s wr, 307 op/s 2024-06-10T02:58:49.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:58:48 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T02:58:49.711 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-06-10T02:58:49.964 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:58:49.976 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:58:49 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-10T02:58:49.977 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:58:49 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-10T02:58:49.977 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:58:49 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-10T02:58:49.977 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:58:49 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-06-10T02:58:49.977 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:58:49 smithi052 ceph-mon[32362]: Adjusting osd_memory_target on smithi120 to 176.2M 2024-06-10T02:58:49.977 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:58:49 smithi052 ceph-mon[32362]: Unable to set osd_memory_target on smithi120 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-06-10T02:58:49.977 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:58:49 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-10T02:58:49.977 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:58:49 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-10T02:58:49.977 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:58:49 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-10T02:58:49.977 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:58:49 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-06-10T02:58:49.977 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:58:49 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:58:49.977 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:58:49 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T02:58:49.978 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:58:49 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:58:50.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:58:49 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-10T02:58:50.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:58:49 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-10T02:58:50.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:58:49 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-10T02:58:50.139 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:58:49 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-06-10T02:58:50.139 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:58:49 smithi120 ceph-mon[36634]: Adjusting osd_memory_target on smithi120 to 176.2M 2024-06-10T02:58:50.139 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:58:49 smithi120 ceph-mon[36634]: Unable to set osd_memory_target on smithi120 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-06-10T02:58:50.139 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:58:49 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-10T02:58:50.139 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:58:49 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-10T02:58:50.139 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:58:49 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-10T02:58:50.139 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:58:49 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-06-10T02:58:50.139 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:58:49 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:58:50.139 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:58:49 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T02:58:50.139 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:58:49 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:58:50.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:58:49 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-10T02:58:50.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:58:49 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-10T02:58:50.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:58:49 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-10T02:58:50.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:58:49 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-06-10T02:58:50.204 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:58:49 smithi188 ceph-mon[36599]: Adjusting osd_memory_target on smithi120 to 176.2M 2024-06-10T02:58:50.204 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:58:49 smithi188 ceph-mon[36599]: Unable to set osd_memory_target on smithi120 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-06-10T02:58:50.204 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:58:49 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-10T02:58:50.204 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:58:49 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-10T02:58:50.204 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:58:49 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-10T02:58:50.204 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:58:49 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-06-10T02:58:50.204 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:58:49 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:58:50.204 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:58:49 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T02:58:50.204 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:58:49 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:58:50.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:50.954+0000 7f5f8effd700 1 -- 172.21.15.52:0/1033646307 <== mds.0 v2:172.21.15.188:6838/310444836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x7f5fa80ade30 con 0x55fdbf989df0 2024-06-10T02:58:50.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:50.954+0000 7f5f8effd700 10 client.16089 handle_command_reply: tid=0 2024-06-10T02:58:50.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:50.954+0000 7f5fc015bb80 1 client.16089 shutdown 2024-06-10T02:58:50.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:50.954+0000 7f5fc015bb80 1 -- 172.21.15.52:0/1033646307 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fdbf8e7cd0 msgr2=0x55fdbf913240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:58:50.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:50.954+0000 7f5fc015bb80 1 --2- 172.21.15.52:0/1033646307 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fdbf8e7cd0 0x55fdbf913240 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f5fa802e040 tx=0x7f5fa80e4700 comp rx=0 tx=0).stop 2024-06-10T02:58:50.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:50.954+0000 7f5fc015bb80 1 -- 172.21.15.52:0/1033646307 shutdown_connections 2024-06-10T02:58:50.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:50.954+0000 7f5fc015bb80 1 --2- 172.21.15.52:0/1033646307 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55fdbf989df0 0x55fdbf98c1e0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:50.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:50.954+0000 7f5fc015bb80 1 --2- 172.21.15.52:0/1033646307 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fdbf8e85f0 0x55fdbf913780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:50.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:50.954+0000 7f5fc015bb80 1 --2- 172.21.15.52:0/1033646307 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fdbf8e2460 0x55fdbf916780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:50.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:50.954+0000 7f5fc015bb80 1 --2- 172.21.15.52:0/1033646307 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fdbf8e7cd0 0x55fdbf913240 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:50.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:50.954+0000 7f5f8effd700 0 client.16089 ms_handle_reset on v2:172.21.15.188:6838/310444836 2024-06-10T02:58:50.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:50.954+0000 7f5fc015bb80 1 -- 172.21.15.52:0/1033646307 >> 172.21.15.52:0/1033646307 conn(0x55fdbf80ce50 msgr2=0x55fdbf8ed640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:58:50.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:50.954+0000 7f5fc015bb80 1 -- 172.21.15.52:0/1033646307 shutdown_connections 2024-06-10T02:58:50.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:50.954+0000 7f5fc015bb80 1 -- 172.21.15.52:0/1033646307 wait complete. 2024-06-10T02:58:50.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:50.954+0000 7f5fc015bb80 20 client.16089 trim_cache size 0 max 16384 2024-06-10T02:58:50.956 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-10T02:58:50.956 INFO:teuthology.orchestra.run.smithi052.stdout: "epoch": 34, 2024-06-10T02:58:50.956 INFO:teuthology.orchestra.run.smithi052.stdout: "leader": 24484, 2024-06-10T02:58:50.956 INFO:teuthology.orchestra.run.smithi052.stdout: "set_version": 10, 2024-06-10T02:58:50.957 INFO:teuthology.orchestra.run.smithi052.stdout: "sets": { 2024-06-10T02:58:50.957 INFO:teuthology.orchestra.run.smithi052.stdout: "b06ab081": { 2024-06-10T02:58:50.957 INFO:teuthology.orchestra.run.smithi052.stdout: "version": 10, 2024-06-10T02:58:50.957 INFO:teuthology.orchestra.run.smithi052.stdout: "age_ref": 0.0, 2024-06-10T02:58:50.957 INFO:teuthology.orchestra.run.smithi052.stdout: "state": { 2024-06-10T02:58:50.957 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "QUIESCED", 2024-06-10T02:58:50.957 INFO:teuthology.orchestra.run.smithi052.stdout: "age": 0.0 2024-06-10T02:58:50.957 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T02:58:50.957 INFO:teuthology.orchestra.run.smithi052.stdout: "timeout": 90.0, 2024-06-10T02:58:50.957 INFO:teuthology.orchestra.run.smithi052.stdout: "expiration": 138.2, 2024-06-10T02:58:50.957 INFO:teuthology.orchestra.run.smithi052.stdout: "members": { 2024-06-10T02:58:50.957 INFO:teuthology.orchestra.run.smithi052.stdout: "file:/": { 2024-06-10T02:58:50.957 INFO:teuthology.orchestra.run.smithi052.stdout: "excluded": false, 2024-06-10T02:58:50.958 INFO:teuthology.orchestra.run.smithi052.stdout: "state": { 2024-06-10T02:58:50.958 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "QUIESCED", 2024-06-10T02:58:50.958 INFO:teuthology.orchestra.run.smithi052.stdout: "age": 0.0 2024-06-10T02:58:50.958 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T02:58:50.958 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T02:58:50.958 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T02:58:50.958 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T02:58:50.958 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T02:58:50.958 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-10T02:58:50.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:50.955+0000 7f5fb58d8700 1 -- 172.21.15.52:0/2677639084 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f5f9c0a2eb0 msgr2=0x7f5f9c0a5370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:58:50.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:50.955+0000 7f5fb58d8700 1 --2- 172.21.15.52:0/2677639084 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f5f9c0a2eb0 0x7f5f9c0a5370 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f5fa000dfc0 tx=0x7f5fa000d520 comp rx=0 tx=0).stop 2024-06-10T02:58:50.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:50.955+0000 7f5fb58d8700 1 -- 172.21.15.52:0/2677639084 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5fb0110ce0 msgr2=0x7f5fb01a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:58:50.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:50.955+0000 7f5fb58d8700 1 --2- 172.21.15.52:0/2677639084 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5fb0110ce0 0x7f5fb01a4240 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f5fa801abd0 tx=0x7f5fa80198c0 comp rx=0 tx=0).stop 2024-06-10T02:58:50.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:50.955+0000 7f5fb58d8700 1 -- 172.21.15.52:0/2677639084 shutdown_connections 2024-06-10T02:58:50.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:50.955+0000 7f5fb58d8700 1 --2- 172.21.15.52:0/2677639084 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5fb011bc70 0x7f5fb01172e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:50.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:50.955+0000 7f5fb58d8700 1 --2- 172.21.15.52:0/2677639084 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f5f9c0a2eb0 0x7f5f9c0a5370 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:50.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:50.955+0000 7f5fb58d8700 1 --2- 172.21.15.52:0/2677639084 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5fb01099d0 0x7f5fb01a3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:50.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:50.955+0000 7f5fb58d8700 1 --2- 172.21.15.52:0/2677639084 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5fb0110ce0 0x7f5fb01a4240 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:50.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:50.955+0000 7f5fb58d8700 1 -- 172.21.15.52:0/2677639084 >> 172.21.15.52:0/2677639084 conn(0x7f5fb006d600 msgr2=0x7f5fb011c050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:58:50.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:50.956+0000 7f5fb58d8700 1 -- 172.21.15.52:0/2677639084 shutdown_connections 2024-06-10T02:58:50.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:50.956+0000 7f5fb58d8700 1 -- 172.21.15.52:0/2677639084 wait complete. 2024-06-10T02:58:51.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:58:50 smithi120 ceph-mon[36634]: Adjusting osd_memory_target on smithi052 to 177.2M 2024-06-10T02:58:51.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:58:50 smithi120 ceph-mon[36634]: Unable to set osd_memory_target on smithi052 to 185844019: error parsing value: Value '185844019' is below minimum 939524096 2024-06-10T02:58:51.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:58:50 smithi120 ceph-mon[36634]: pgmap v573: 129 pgs: 129 active+clean; 461 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.9 MiB/s wr, 306 op/s 2024-06-10T02:58:51.202 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:58:50 smithi052 ceph-mon[32362]: Adjusting osd_memory_target on smithi052 to 177.2M 2024-06-10T02:58:51.202 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:58:50 smithi052 ceph-mon[32362]: Unable to set osd_memory_target on smithi052 to 185844019: error parsing value: Value '185844019' is below minimum 939524096 2024-06-10T02:58:51.202 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:58:50 smithi052 ceph-mon[32362]: pgmap v573: 129 pgs: 129 active+clean; 461 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.9 MiB/s wr, 306 op/s 2024-06-10T02:58:51.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:58:50 smithi188 ceph-mon[36599]: Adjusting osd_memory_target on smithi052 to 177.2M 2024-06-10T02:58:51.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:58:50 smithi188 ceph-mon[36599]: Unable to set osd_memory_target on smithi052 to 185844019: error parsing value: Value '185844019' is below minimum 939524096 2024-06-10T02:58:51.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:58:50 smithi188 ceph-mon[36599]: pgmap v573: 129 pgs: 129 active+clean; 461 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.9 MiB/s wr, 306 op/s 2024-06-10T02:58:52.459 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'b06ab081', quiesce took 29.8 seconds. Will release after: -11.600000000000001 2024-06-10T02:58:52.460 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T02:58:52.742 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:58:53.097 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:58:52 smithi052 ceph-mon[32362]: pgmap v574: 129 pgs: 129 active+clean; 462 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.0 MiB/s wr, 299 op/s 2024-06-10T02:58:53.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:58:52 smithi120 ceph-mon[36634]: pgmap v574: 129 pgs: 129 active+clean; 462 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.0 MiB/s wr, 299 op/s 2024-06-10T02:58:53.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:58:52 smithi188 ceph-mon[36599]: pgmap v574: 129 pgs: 129 active+clean; 462 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.0 MiB/s wr, 299 op/s 2024-06-10T02:58:53.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.220+0000 7f6237760700 1 -- 172.21.15.52:0/878671001 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6230101d50 msgr2=0x7f623010e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:58:53.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.220+0000 7f6237760700 1 --2- 172.21.15.52:0/878671001 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6230101d50 0x7f623010e240 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f622800b0a0 tx=0x7f622800b3b0 comp rx=0 tx=0).stop 2024-06-10T02:58:53.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.220+0000 7f6237760700 1 -- 172.21.15.52:0/878671001 shutdown_connections 2024-06-10T02:58:53.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.220+0000 7f6237760700 1 --2- 172.21.15.52:0/878671001 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6230101d50 0x7f623010e240 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:53.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.220+0000 7f6237760700 1 --2- 172.21.15.52:0/878671001 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6230107400 0x7f62301077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:53.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.220+0000 7f6237760700 1 --2- 172.21.15.52:0/878671001 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6230101390 0x7f6230101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:53.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.220+0000 7f6237760700 1 -- 172.21.15.52:0/878671001 >> 172.21.15.52:0/878671001 conn(0x7f6230078340 msgr2=0x7f6230078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:58:53.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.221+0000 7f6237760700 1 -- 172.21.15.52:0/878671001 shutdown_connections 2024-06-10T02:58:53.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.221+0000 7f6237760700 1 -- 172.21.15.52:0/878671001 wait complete. 2024-06-10T02:58:53.223 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.221+0000 7f6237760700 1 Processor -- start 2024-06-10T02:58:53.223 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.221+0000 7f6237760700 1 -- start start 2024-06-10T02:58:53.223 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.222+0000 7f6237760700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6230101390 0x7f6230199ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:58:53.223 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.222+0000 7f6237760700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6230101d50 0x7f623019a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:58:53.223 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.222+0000 7f6237760700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6230107400 0x7f62301a4b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:58:53.223 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.222+0000 7f6237760700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f62301116a0 con 0x7f6230101d50 2024-06-10T02:58:53.223 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.222+0000 7f6237760700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f6230111520 con 0x7f6230101390 2024-06-10T02:58:53.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.222+0000 7f6237760700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f6230111820 con 0x7f6230107400 2024-06-10T02:58:53.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.222+0000 7f6234cfb700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6230101d50 0x7f623019a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:58:53.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.222+0000 7f6234cfb700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6230101d50 0x7f623019a000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:59614/0 (socket says 172.21.15.52:59614) 2024-06-10T02:58:53.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.222+0000 7f6234cfb700 1 -- 172.21.15.52:0/505867387 learned_addr learned my addr 172.21.15.52:0/505867387 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:58:53.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.222+0000 7f62354fc700 1 --2- 172.21.15.52:0/505867387 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6230101390 0x7f6230199ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:58:53.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.222+0000 7f6235cfd700 1 --2- 172.21.15.52:0/505867387 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6230107400 0x7f62301a4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:58:53.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.222+0000 7f6234cfb700 1 -- 172.21.15.52:0/505867387 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6230107400 msgr2=0x7f62301a4b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:58:53.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.222+0000 7f6234cfb700 1 --2- 172.21.15.52:0/505867387 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6230107400 0x7f62301a4b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:53.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.222+0000 7f6234cfb700 1 -- 172.21.15.52:0/505867387 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6230101390 msgr2=0x7f6230199ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:58:53.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.222+0000 7f6234cfb700 1 --2- 172.21.15.52:0/505867387 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6230101390 0x7f6230199ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:53.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.222+0000 7f6234cfb700 1 -- 172.21.15.52:0/505867387 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62301978b0 con 0x7f6230101d50 2024-06-10T02:58:53.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.222+0000 7f62354fc700 1 --2- 172.21.15.52:0/505867387 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6230101390 0x7f6230199ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T02:58:53.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.222+0000 7f6234cfb700 1 --2- 172.21.15.52:0/505867387 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6230101d50 0x7f623019a000 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f622000e9c0 tx=0x7f622000ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:58:53.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.223+0000 7f62267fc700 1 -- 172.21.15.52:0/505867387 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f622000cbd0 con 0x7f6230101d50 2024-06-10T02:58:53.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.223+0000 7f62267fc700 1 -- 172.21.15.52:0/505867387 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f622000ee60 con 0x7f6230101d50 2024-06-10T02:58:53.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.223+0000 7f6237760700 1 -- 172.21.15.52:0/505867387 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6230197ba0 con 0x7f6230101d50 2024-06-10T02:58:53.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.223+0000 7f62267fc700 1 -- 172.21.15.52:0/505867387 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6220005630 con 0x7f6230101d50 2024-06-10T02:58:53.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.223+0000 7f6237760700 1 -- 172.21.15.52:0/505867387 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62301980e0 con 0x7f6230101d50 2024-06-10T02:58:53.226 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.225+0000 7f62267fc700 1 -- 172.21.15.52:0/505867387 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f6220004750 con 0x7f6230101d50 2024-06-10T02:58:53.226 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.226+0000 7f62267fc700 1 --2- 172.21.15.52:0/505867387 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f621c0a2ca0 0x7f621c0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:58:53.226 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.226+0000 7f62354fc700 1 --2- 172.21.15.52:0/505867387 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f621c0a2ca0 0x7f621c0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:58:53.226 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.226+0000 7f62267fc700 1 -- 172.21.15.52:0/505867387 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f62200cf630 con 0x7f6230101d50 2024-06-10T02:58:53.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.226+0000 7f6241fe3b80 1 Processor -- start 2024-06-10T02:58:53.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.226+0000 7f6241fe3b80 1 -- start start 2024-06-10T02:58:53.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.226+0000 7f62354fc700 1 --2- 172.21.15.52:0/505867387 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f621c0a2ca0 0x7f621c0a5160 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f622c009fd0 tx=0x7f622c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:58:53.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.227+0000 7f6241fe3b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56265510cf60 0x56265510d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:58:53.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.227+0000 7f6241fe3b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56265510d880 0x562655107260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:58:53.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.227+0000 7f6241fe3b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5626551077a0 0x562655107bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:58:53.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.227+0000 7f6241fe3b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x562654ff9f60 con 0x56265510cf60 2024-06-10T02:58:53.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.227+0000 7f6241fe3b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x562655058850 con 0x5626551077a0 2024-06-10T02:58:53.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.227+0000 7f6241fe3b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x562654fc0460 con 0x56265510d880 2024-06-10T02:58:53.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.227+0000 7f62354fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56265510cf60 0x56265510d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:58:53.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.227+0000 7f6234cfb700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56265510d880 0x562655107260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:58:53.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.227+0000 7f6234cfb700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56265510d880 0x562655107260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.52:60930/0 (socket says 172.21.15.52:60930) 2024-06-10T02:58:53.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.227+0000 7f6234cfb700 1 -- 172.21.15.52:0/612390097 learned_addr learned my addr 172.21.15.52:0/612390097 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:58:53.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.227+0000 7f6235cfd700 1 --2- 172.21.15.52:0/612390097 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5626551077a0 0x562655107bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:58:53.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.227+0000 7f6234cfb700 1 -- 172.21.15.52:0/612390097 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5626551077a0 msgr2=0x562655107bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:58:53.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.227+0000 7f6234cfb700 1 --2- 172.21.15.52:0/612390097 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5626551077a0 0x562655107bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:53.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.227+0000 7f6234cfb700 1 -- 172.21.15.52:0/612390097 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56265510cf60 msgr2=0x56265510d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:58:53.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.227+0000 7f6234cfb700 1 --2- 172.21.15.52:0/612390097 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56265510cf60 0x56265510d340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:53.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.227+0000 7f6234cfb700 1 -- 172.21.15.52:0/612390097 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562654fa78d0 con 0x56265510d880 2024-06-10T02:58:53.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.227+0000 7f6234cfb700 1 --2- 172.21.15.52:0/612390097 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56265510d880 0x562655107260 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f6220097ee0 tx=0x7f622009b4d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8393ec59bab670e1 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:58:53.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.228+0000 7f621b7fe700 1 -- 172.21.15.52:0/612390097 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f622009fe30 con 0x56265510d880 2024-06-10T02:58:53.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.228+0000 7f621b7fe700 1 -- 172.21.15.52:0/612390097 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f62200a46f0 con 0x56265510d880 2024-06-10T02:58:53.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.228+0000 7f621b7fe700 1 -- 172.21.15.52:0/612390097 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62200cf3e0 con 0x56265510d880 2024-06-10T02:58:53.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.228+0000 7f6241fe3b80 1 -- 172.21.15.52:0/612390097 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56265510d880 msgr2=0x562655107260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:58:53.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.228+0000 7f6241fe3b80 1 --2- 172.21.15.52:0/612390097 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56265510d880 0x562655107260 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f6220097ee0 tx=0x7f622009b4d0 comp rx=0 tx=0).stop 2024-06-10T02:58:53.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.228+0000 7f6241fe3b80 1 -- 172.21.15.52:0/612390097 shutdown_connections 2024-06-10T02:58:53.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.228+0000 7f6241fe3b80 1 --2- 172.21.15.52:0/612390097 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5626551077a0 0x562655107bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:53.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.228+0000 7f6241fe3b80 1 --2- 172.21.15.52:0/612390097 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56265510cf60 0x56265510d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:53.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.228+0000 7f6241fe3b80 1 --2- 172.21.15.52:0/612390097 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56265510d880 0x562655107260 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:53.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.228+0000 7f6241fe3b80 1 -- 172.21.15.52:0/612390097 >> 172.21.15.52:0/612390097 conn(0x562655034080 msgr2=0x562655102220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:58:53.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.229+0000 7f6241fe3b80 1 -- 172.21.15.52:0/612390097 shutdown_connections 2024-06-10T02:58:53.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.229+0000 7f6241fe3b80 1 -- 172.21.15.52:0/612390097 wait complete. 2024-06-10T02:58:53.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.229+0000 7f6241fe3b80 1 -- start start 2024-06-10T02:58:53.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.229+0000 7f6241fe3b80 1 Processor -- start 2024-06-10T02:58:53.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.229+0000 7f6241fe3b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5626551077a0 0x5626551135c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:58:53.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.229+0000 7f6241fe3b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56265510cf60 0x5626551108c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:58:53.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.229+0000 7f6241fe3b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56265510d880 0x562655110e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:58:53.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.229+0000 7f6241fe3b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x562654ff8fc0 con 0x5626551077a0 2024-06-10T02:58:53.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.230+0000 7f6241fe3b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x562655054810 con 0x56265510d880 2024-06-10T02:58:53.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.230+0000 7f6241fe3b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x562655114200 con 0x56265510cf60 2024-06-10T02:58:53.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.230+0000 7f6241fe3b80 10 client.0 _ll_register_callbacks cb 0x562654fee5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T02:58:53.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.230+0000 7f62354fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5626551077a0 0x5626551135c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:58:53.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.230+0000 7f6234cfb700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56265510cf60 0x5626551108c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:58:53.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.230+0000 7f62354fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5626551077a0 0x5626551135c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:59630/0 (socket says 172.21.15.52:59630) 2024-06-10T02:58:53.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.230+0000 7f6234cfb700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56265510cf60 0x5626551108c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.52:60940/0 (socket says 172.21.15.52:60940) 2024-06-10T02:58:53.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.230+0000 7f62354fc700 1 -- 172.21.15.52:0/3859301564 learned_addr learned my addr 172.21.15.52:0/3859301564 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:58:53.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.230+0000 7f6235cfd700 1 --2- 172.21.15.52:0/3859301564 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56265510d880 0x562655110e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:58:53.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.230+0000 7f62354fc700 1 -- 172.21.15.52:0/3859301564 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56265510cf60 msgr2=0x5626551108c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:58:53.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.230+0000 7f62354fc700 1 --2- 172.21.15.52:0/3859301564 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56265510cf60 0x5626551108c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:53.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.230+0000 7f62354fc700 1 -- 172.21.15.52:0/3859301564 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56265510d880 msgr2=0x562655110e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:58:53.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.230+0000 7f62354fc700 1 --2- 172.21.15.52:0/3859301564 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56265510d880 0x562655110e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:53.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.230+0000 7f62354fc700 1 -- 172.21.15.52:0/3859301564 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56265501e750 con 0x5626551077a0 2024-06-10T02:58:53.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.230+0000 7f6234cfb700 1 --2- 172.21.15.52:0/3859301564 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56265510cf60 0x5626551108c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T02:58:53.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.230+0000 7f62354fc700 1 --2- 172.21.15.52:0/3859301564 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5626551077a0 0x5626551135c0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f622c00ebb0 tx=0x7f622c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9ad47da37de0e40 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:58:53.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.230+0000 7f6218ff9700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-10T02:58:53.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.230+0000 7f6218ff9700 1 -- 172.21.15.52:0/3859301564 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f622c004460 con 0x5626551077a0 2024-06-10T02:58:53.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.230+0000 7f6218ff9700 1 -- 172.21.15.52:0/3859301564 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f622c01f890 con 0x5626551077a0 2024-06-10T02:58:53.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.230+0000 7f6241fe3b80 1 -- 172.21.15.52:0/3859301564 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f62200a46f0 con 0x5626551077a0 2024-06-10T02:58:53.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.231+0000 7f6218ff9700 1 -- 172.21.15.52:0/3859301564 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f622c031050 con 0x5626551077a0 2024-06-10T02:58:53.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.231+0000 7f6241fe3b80 10 client.16143 fetch_fsmap learned FSMap version 34 2024-06-10T02:58:53.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.231+0000 7f6241fe3b80 1 -- 172.21.15.52:0/3859301564 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x56265511e9f0 con 0x5626551077a0 2024-06-10T02:58:53.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.231+0000 7f6218ff9700 1 -- 172.21.15.52:0/3859301564 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f62200a46f0 con 0x5626551077a0 2024-06-10T02:58:53.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.231+0000 7f6218ff9700 1 -- 172.21.15.52:0/3859301564 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 34) ==== 8378+0+0 (secure 0 0 0) 0x7f622c026020 con 0x5626551077a0 2024-06-10T02:58:53.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.231+0000 7f6241fe3b80 10 client.16143 fetch_fsmap finished waiting for FSMap version 34 2024-06-10T02:58:53.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.231+0000 7f6241fe3b80 10 client.16143 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.b 2024-06-10T02:58:53.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.231+0000 7f6241fe3b80 20 client.16143 populate_metadata read hostname 'smithi052' 2024-06-10T02:58:53.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.231+0000 7f6241fe3b80 1 --2- 172.21.15.52:0/3859301564 >> [v2:172.21.15.120:6838/1672436487,v1:172.21.15.120:6839/1672436487] conn(0x562655121030 0x562655123420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:58:53.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.231+0000 7f6241fe3b80 4 client.16143 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T02:58:53.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.231+0000 7f6241fe3b80 1 -- 172.21.15.52:0/3859301564 --> [v2:172.21.15.120:6838/1672436487,v1:172.21.15.120:6839/1672436487] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562655065d50 con 0x562655121030 2024-06-10T02:58:53.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.231+0000 7f6234cfb700 1 --2- 172.21.15.52:0/3859301564 >> [v2:172.21.15.120:6838/1672436487,v1:172.21.15.120:6839/1672436487] conn(0x562655121030 0x562655123420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:58:53.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.232+0000 7f6218ff9700 10 client.16143 ms_handle_connect on v2:172.21.15.120:6838/1672436487 2024-06-10T02:58:53.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.232+0000 7f6234cfb700 1 --2- 172.21.15.52:0/3859301564 >> [v2:172.21.15.120:6838/1672436487,v1:172.21.15.120:6839/1672436487] conn(0x562655121030 0x562655123420 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=15d5423070844b91 server_cookie=ef6ce8f370b3b89d in_seq=0 out_seq=0 2024-06-10T02:58:53.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.232+0000 7f6218ff9700 1 -- 172.21.15.52:0/3859301564 <== mds.3 v2:172.21.15.120:6838/1672436487 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x562655065d50 con 0x562655121030 2024-06-10T02:58:53.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.232+0000 7f6218ff9700 10 client.16143 handle_command_reply: tid=0 2024-06-10T02:58:53.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.232+0000 7f6241fe3b80 1 client.16143 shutdown 2024-06-10T02:58:53.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.233+0000 7f6241fe3b80 1 -- 172.21.15.52:0/3859301564 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5626551077a0 msgr2=0x5626551135c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:58:53.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.233+0000 7f6241fe3b80 1 --2- 172.21.15.52:0/3859301564 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5626551077a0 0x5626551135c0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f622c00ebb0 tx=0x7f622c0047c0 comp rx=0 tx=0).stop 2024-06-10T02:58:53.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.233+0000 7f6241fe3b80 1 -- 172.21.15.52:0/3859301564 shutdown_connections 2024-06-10T02:58:53.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.233+0000 7f6241fe3b80 1 --2- 172.21.15.52:0/3859301564 >> [v2:172.21.15.120:6838/1672436487,v1:172.21.15.120:6839/1672436487] conn(0x562655121030 0x562655123420 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:53.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.233+0000 7f6218ff9700 0 client.16143 ms_handle_reset on v2:172.21.15.120:6838/1672436487 2024-06-10T02:58:53.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.233+0000 7f6241fe3b80 1 --2- 172.21.15.52:0/3859301564 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56265510d880 0x562655110e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:53.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.233+0000 7f6241fe3b80 1 --2- 172.21.15.52:0/3859301564 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5626551077a0 0x5626551135c0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:53.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.233+0000 7f6241fe3b80 1 --2- 172.21.15.52:0/3859301564 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56265510cf60 0x5626551108c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:53.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.233+0000 7f6241fe3b80 1 -- 172.21.15.52:0/3859301564 >> 172.21.15.52:0/3859301564 conn(0x562655034080 msgr2=0x562655101700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:58:53.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.233+0000 7f6241fe3b80 1 -- 172.21.15.52:0/3859301564 shutdown_connections 2024-06-10T02:58:53.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.233+0000 7f6241fe3b80 1 -- 172.21.15.52:0/3859301564 wait complete. 2024-06-10T02:58:53.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.233+0000 7f6241fe3b80 20 client.16143 trim_cache size 0 max 16384 2024-06-10T02:58:53.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.250+0000 7f6241fe3b80 1 Processor -- start 2024-06-10T02:58:53.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.250+0000 7f6241fe3b80 1 -- start start 2024-06-10T02:58:53.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.250+0000 7f6241fe3b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5626551077a0 0x562655107260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:58:53.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.250+0000 7f6241fe3b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56265510cf60 0x56265510e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:58:53.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.250+0000 7f6241fe3b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56265510d880 0x562655101e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:58:53.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.250+0000 7f6241fe3b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f62301ac250 con 0x56265510d880 2024-06-10T02:58:53.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.250+0000 7f6241fe3b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f62301ade50 con 0x5626551077a0 2024-06-10T02:58:53.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.250+0000 7f6241fe3b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f62301ac920 con 0x56265510cf60 2024-06-10T02:58:53.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.250+0000 7f6234cfb700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56265510cf60 0x56265510e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:58:53.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.250+0000 7f6235cfd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56265510d880 0x562655101e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:58:53.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.250+0000 7f6234cfb700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56265510cf60 0x56265510e3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.52:60942/0 (socket says 172.21.15.52:60942) 2024-06-10T02:58:53.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.250+0000 7f6234cfb700 1 -- 172.21.15.52:0/2438556026 learned_addr learned my addr 172.21.15.52:0/2438556026 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:58:53.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.250+0000 7f6235cfd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56265510d880 0x562655101e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:59646/0 (socket says 172.21.15.52:59646) 2024-06-10T02:58:53.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.250+0000 7f62354fc700 1 --2- 172.21.15.52:0/2438556026 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5626551077a0 0x562655107260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:58:53.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.250+0000 7f6234cfb700 1 -- 172.21.15.52:0/2438556026 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5626551077a0 msgr2=0x562655107260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:58:53.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.250+0000 7f6234cfb700 1 --2- 172.21.15.52:0/2438556026 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5626551077a0 0x562655107260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:53.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.250+0000 7f6234cfb700 1 -- 172.21.15.52:0/2438556026 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56265510d880 msgr2=0x562655101e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:58:53.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.250+0000 7f6234cfb700 1 --2- 172.21.15.52:0/2438556026 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56265510d880 0x562655101e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:53.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.250+0000 7f6234cfb700 1 -- 172.21.15.52:0/2438556026 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56265510ee30 con 0x56265510cf60 2024-06-10T02:58:53.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.250+0000 7f62354fc700 1 --2- 172.21.15.52:0/2438556026 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5626551077a0 0x562655107260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T02:58:53.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.250+0000 7f6235cfd700 1 --2- 172.21.15.52:0/2438556026 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56265510d880 0x562655101e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T02:58:53.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.251+0000 7f6234cfb700 1 --2- 172.21.15.52:0/2438556026 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56265510cf60 0x56265510e3b0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f6220009fd0 tx=0x7f622000cf90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=16e7421275a8e9fe server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:58:53.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.251+0000 7f621affd700 1 -- 172.21.15.52:0/2438556026 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62200a46f0 con 0x56265510cf60 2024-06-10T02:58:53.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.251+0000 7f621affd700 1 -- 172.21.15.52:0/2438556026 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6220099d80 con 0x56265510cf60 2024-06-10T02:58:53.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.251+0000 7f621affd700 1 -- 172.21.15.52:0/2438556026 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62200a4260 con 0x56265510cf60 2024-06-10T02:58:53.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.252+0000 7f6241fe3b80 1 -- 172.21.15.52:0/2438556026 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56265510cf60 msgr2=0x56265510e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:58:53.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.252+0000 7f6241fe3b80 1 --2- 172.21.15.52:0/2438556026 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56265510cf60 0x56265510e3b0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f6220009fd0 tx=0x7f622000cf90 comp rx=0 tx=0).stop 2024-06-10T02:58:53.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.252+0000 7f6241fe3b80 1 -- 172.21.15.52:0/2438556026 shutdown_connections 2024-06-10T02:58:53.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.252+0000 7f6241fe3b80 1 --2- 172.21.15.52:0/2438556026 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56265510d880 0x562655101e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:53.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.252+0000 7f6241fe3b80 1 --2- 172.21.15.52:0/2438556026 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5626551077a0 0x562655107260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:53.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.252+0000 7f6241fe3b80 1 --2- 172.21.15.52:0/2438556026 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56265510cf60 0x56265510e3b0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:53.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.252+0000 7f6241fe3b80 1 -- 172.21.15.52:0/2438556026 >> 172.21.15.52:0/2438556026 conn(0x562655034080 msgr2=0x56265510dca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:58:53.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.252+0000 7f6241fe3b80 1 -- 172.21.15.52:0/2438556026 shutdown_connections 2024-06-10T02:58:53.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.252+0000 7f6241fe3b80 1 -- 172.21.15.52:0/2438556026 wait complete. 2024-06-10T02:58:53.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.252+0000 7f6241fe3b80 1 -- start start 2024-06-10T02:58:53.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.252+0000 7f6241fe3b80 1 Processor -- start 2024-06-10T02:58:53.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.253+0000 7f6241fe3b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5626551077a0 0x5626551196d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:58:53.257 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.253+0000 7f6241fe3b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56265510cf60 0x562655119c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:58:53.257 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.253+0000 7f6241fe3b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56265510d880 0x562655128100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:58:53.257 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.253+0000 7f6241fe3b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f62301ad780 con 0x56265510cf60 2024-06-10T02:58:53.257 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.253+0000 7f6241fe3b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f62301ad0b0 con 0x56265510d880 2024-06-10T02:58:53.257 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.253+0000 7f6241fe3b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f62301aa1f0 con 0x5626551077a0 2024-06-10T02:58:53.257 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.253+0000 7f6241fe3b80 10 client.0 _ll_register_callbacks cb 0x562654fee5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T02:58:53.257 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.253+0000 7f6234cfb700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56265510cf60 0x562655119c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:58:53.257 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.253+0000 7f62354fc700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5626551077a0 0x5626551196d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:58:53.257 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.253+0000 7f62354fc700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5626551077a0 0x5626551196d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.52:60946/0 (socket says 172.21.15.52:60946) 2024-06-10T02:58:53.257 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.253+0000 7f62354fc700 1 -- 172.21.15.52:0/2539910288 learned_addr learned my addr 172.21.15.52:0/2539910288 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:58:53.257 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.253+0000 7f6235cfd700 1 --2- 172.21.15.52:0/2539910288 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56265510d880 0x562655128100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:58:53.257 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.253+0000 7f62354fc700 1 -- 172.21.15.52:0/2539910288 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56265510d880 msgr2=0x562655128100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:58:53.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.253+0000 7f62354fc700 1 --2- 172.21.15.52:0/2539910288 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56265510d880 0x562655128100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:53.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.253+0000 7f62354fc700 1 -- 172.21.15.52:0/2539910288 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56265510cf60 msgr2=0x562655119c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:58:53.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.253+0000 7f62354fc700 1 --2- 172.21.15.52:0/2539910288 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56265510cf60 0x562655119c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:53.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.253+0000 7f62354fc700 1 -- 172.21.15.52:0/2539910288 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5626551287e0 con 0x5626551077a0 2024-06-10T02:58:53.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.253+0000 7f6234cfb700 1 --2- 172.21.15.52:0/2539910288 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56265510cf60 0x562655119c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T02:58:53.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.253+0000 7f62354fc700 1 --2- 172.21.15.52:0/2539910288 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5626551077a0 0x5626551196d0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f622c011b60 tx=0x7f622c004590 comp rx=0 tx=0).ready entity=mon.1 client_cookie=baee2f46eb0dcc75 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:58:53.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.253+0000 7f6218ff9700 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-06-10T02:58:53.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.253+0000 7f6218ff9700 1 -- 172.21.15.52:0/2539910288 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f622c03a060 con 0x5626551077a0 2024-06-10T02:58:53.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.254+0000 7f6241fe3b80 1 -- 172.21.15.52:0/2539910288 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6220099d80 con 0x5626551077a0 2024-06-10T02:58:53.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.254+0000 7f6218ff9700 1 -- 172.21.15.52:0/2539910288 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f622c00fcc0 con 0x5626551077a0 2024-06-10T02:58:53.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.254+0000 7f6218ff9700 1 -- 172.21.15.52:0/2539910288 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f622c00e460 con 0x5626551077a0 2024-06-10T02:58:53.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.254+0000 7f6241fe3b80 10 client.25723 fetch_fsmap learned FSMap version 34 2024-06-10T02:58:53.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.254+0000 7f6241fe3b80 1 -- 172.21.15.52:0/2539910288 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x56265518b740 con 0x5626551077a0 2024-06-10T02:58:53.259 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.254+0000 7f6218ff9700 1 -- 172.21.15.52:0/2539910288 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6220099d80 con 0x5626551077a0 2024-06-10T02:58:53.259 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.254+0000 7f6218ff9700 1 -- 172.21.15.52:0/2539910288 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 34) ==== 8378+0+0 (secure 0 0 0) 0x7f622c034030 con 0x5626551077a0 2024-06-10T02:58:53.259 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.254+0000 7f6241fe3b80 10 client.25723 fetch_fsmap finished waiting for FSMap version 34 2024-06-10T02:58:53.259 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.254+0000 7f6241fe3b80 10 client.25723 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.b 2024-06-10T02:58:53.259 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.254+0000 7f6241fe3b80 20 client.25723 populate_metadata read hostname 'smithi052' 2024-06-10T02:58:53.259 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.254+0000 7f6241fe3b80 1 --2- 172.21.15.52:0/2539910288 >> [v2:172.21.15.120:6838/1672436487,v1:172.21.15.120:6839/1672436487] conn(0x562655121030 0x56265518de90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:58:53.259 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.254+0000 7f6241fe3b80 4 client.25723 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T02:58:53.259 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.254+0000 7f6241fe3b80 1 -- 172.21.15.52:0/2539910288 --> [v2:172.21.15.120:6838/1672436487,v1:172.21.15.120:6839/1672436487] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5626550602d0 con 0x562655121030 2024-06-10T02:58:53.259 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.255+0000 7f6234cfb700 1 --2- 172.21.15.52:0/2539910288 >> [v2:172.21.15.120:6838/1672436487,v1:172.21.15.120:6839/1672436487] conn(0x562655121030 0x56265518de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:58:53.259 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.255+0000 7f6234cfb700 1 --2- 172.21.15.52:0/2539910288 >> [v2:172.21.15.120:6838/1672436487,v1:172.21.15.120:6839/1672436487] conn(0x562655121030 0x56265518de90 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f8659d414cdbca5d server_cookie=3ec106d0ea49af15 in_seq=0 out_seq=0 2024-06-10T02:58:53.259 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.255+0000 7f6218ff9700 10 client.25723 ms_handle_connect on v2:172.21.15.120:6838/1672436487 2024-06-10T02:58:53.259 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.255+0000 7f6218ff9700 1 -- 172.21.15.52:0/2539910288 <== mds.3 v2:172.21.15.120:6838/1672436487 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5626550602d0 con 0x562655121030 2024-06-10T02:58:53.259 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.255+0000 7f6218ff9700 10 client.25723 handle_command_reply: tid=0 2024-06-10T02:58:53.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.255+0000 7f6241fe3b80 1 client.25723 shutdown 2024-06-10T02:58:53.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.255+0000 7f6241fe3b80 1 -- 172.21.15.52:0/2539910288 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5626551077a0 msgr2=0x5626551196d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:58:53.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.255+0000 7f6241fe3b80 1 --2- 172.21.15.52:0/2539910288 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5626551077a0 0x5626551196d0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f622c011b60 tx=0x7f622c004590 comp rx=0 tx=0).stop 2024-06-10T02:58:53.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.255+0000 7f6241fe3b80 1 -- 172.21.15.52:0/2539910288 shutdown_connections 2024-06-10T02:58:53.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.255+0000 7f6241fe3b80 1 --2- 172.21.15.52:0/2539910288 >> [v2:172.21.15.120:6838/1672436487,v1:172.21.15.120:6839/1672436487] conn(0x562655121030 0x56265518de90 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:53.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.255+0000 7f6241fe3b80 1 --2- 172.21.15.52:0/2539910288 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56265510d880 0x562655128100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:53.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.255+0000 7f6241fe3b80 1 --2- 172.21.15.52:0/2539910288 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5626551077a0 0x5626551196d0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:53.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.255+0000 7f6218ff9700 0 client.25723 ms_handle_reset on v2:172.21.15.120:6838/1672436487 2024-06-10T02:58:53.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.255+0000 7f6241fe3b80 1 --2- 172.21.15.52:0/2539910288 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56265510cf60 0x562655119c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:53.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.255+0000 7f6241fe3b80 1 -- 172.21.15.52:0/2539910288 >> 172.21.15.52:0/2539910288 conn(0x562655034080 msgr2=0x5626551120f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:58:53.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.256+0000 7f6241fe3b80 1 -- 172.21.15.52:0/2539910288 shutdown_connections 2024-06-10T02:58:53.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.256+0000 7f6241fe3b80 1 -- 172.21.15.52:0/2539910288 wait complete. 2024-06-10T02:58:53.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.256+0000 7f6241fe3b80 20 client.25723 trim_cache size 0 max 16384 2024-06-10T02:58:53.261 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-06-10T02:58:53.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.256+0000 7f6237760700 1 -- 172.21.15.52:0/505867387 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f621c0a2ca0 msgr2=0x7f621c0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:58:53.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.256+0000 7f6237760700 1 --2- 172.21.15.52:0/505867387 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f621c0a2ca0 0x7f621c0a5160 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f622c009fd0 tx=0x7f622c009260 comp rx=0 tx=0).stop 2024-06-10T02:58:53.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.256+0000 7f6237760700 1 -- 172.21.15.52:0/505867387 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6230101d50 msgr2=0x7f623019a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:58:53.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.256+0000 7f6237760700 1 --2- 172.21.15.52:0/505867387 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6230101d50 0x7f623019a000 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f622000e9c0 tx=0x7f622000ecd0 comp rx=0 tx=0).stop 2024-06-10T02:58:53.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.256+0000 7f6237760700 1 -- 172.21.15.52:0/505867387 shutdown_connections 2024-06-10T02:58:53.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.256+0000 7f6237760700 1 --2- 172.21.15.52:0/505867387 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6230107400 0x7f62301a4b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:53.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.256+0000 7f6237760700 1 --2- 172.21.15.52:0/505867387 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6230101390 0x7f6230199ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:53.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.256+0000 7f6237760700 1 --2- 172.21.15.52:0/505867387 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f621c0a2ca0 0x7f621c0a5160 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:53.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.256+0000 7f6237760700 1 --2- 172.21.15.52:0/505867387 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6230101d50 0x7f623019a000 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:53.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.256+0000 7f6237760700 1 -- 172.21.15.52:0/505867387 >> 172.21.15.52:0/505867387 conn(0x7f6230078340 msgr2=0x7f62300ffe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:58:53.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.256+0000 7f6237760700 1 -- 172.21.15.52:0/505867387 shutdown_connections 2024-06-10T02:58:53.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:53.256+0000 7f6237760700 1 -- 172.21.15.52:0/505867387 wait complete. 2024-06-10T02:58:54.976 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:58:54 smithi052 ceph-mon[32362]: pgmap v575: 129 pgs: 129 active+clean; 475 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.1 MiB/s wr, 324 op/s 2024-06-10T02:58:55.051 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-06-10T02:58:55.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:58:54 smithi120 ceph-mon[36634]: pgmap v575: 129 pgs: 129 active+clean; 475 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.1 MiB/s wr, 324 op/s 2024-06-10T02:58:55.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:58:54 smithi188 ceph-mon[36599]: pgmap v575: 129 pgs: 129 active+clean; 475 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.1 MiB/s wr, 324 op/s 2024-06-10T02:58:55.552 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:55.551+0000 7f5833324700 1 -- 172.21.15.52:0/3798886542 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f582c06aac0 msgr2=0x7f582c10ea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:58:55.552 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:55.551+0000 7f5833324700 1 --2- 172.21.15.52:0/3798886542 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f582c06aac0 0x7f582c10ea00 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f581c009990 tx=0x7f581c009ca0 comp rx=0 tx=0).stop 2024-06-10T02:58:55.553 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:55.551+0000 7f5833324700 1 -- 172.21.15.52:0/3798886542 shutdown_connections 2024-06-10T02:58:55.553 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:55.551+0000 7f5833324700 1 --2- 172.21.15.52:0/3798886542 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f582c10ef40 0x7f582c111330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:55.553 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:55.551+0000 7f5833324700 1 --2- 172.21.15.52:0/3798886542 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f582c06a1a0 0x7f582c06a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:55.553 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:55.551+0000 7f5833324700 1 --2- 172.21.15.52:0/3798886542 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f582c06aac0 0x7f582c10ea00 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:55.553 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:55.551+0000 7f5833324700 1 -- 172.21.15.52:0/3798886542 >> 172.21.15.52:0/3798886542 conn(0x7f582c078340 msgr2=0x7f582c078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:58:55.553 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:55.552+0000 7f5833324700 1 -- 172.21.15.52:0/3798886542 shutdown_connections 2024-06-10T02:58:55.553 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:55.552+0000 7f5833324700 1 -- 172.21.15.52:0/3798886542 wait complete. 2024-06-10T02:58:55.553 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:55.552+0000 7f5833324700 1 Processor -- start 2024-06-10T02:58:55.553 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:55.552+0000 7f5833324700 1 -- start start 2024-06-10T02:58:55.554 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:55.553+0000 7f5833324700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f582c06a1a0 0x7f582c19af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:58:55.554 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:55.553+0000 7f5833324700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f582c06aac0 0x7f582c19b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:58:55.554 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:55.553+0000 7f5833324700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f582c10ef40 0x7f582c19f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:58:55.555 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:55.553+0000 7f58310c0700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f582c06a1a0 0x7f582c19af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:58:55.555 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:55.553+0000 7f5833324700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f582c1149d0 con 0x7f582c10ef40 2024-06-10T02:58:55.555 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:55.553+0000 7f5833324700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f582c114850 con 0x7f582c06aac0 2024-06-10T02:58:55.555 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:55.553+0000 7f5833324700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f582c114b50 con 0x7f582c06a1a0 2024-06-10T02:58:55.555 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:55.553+0000 7f58318c1700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f582c10ef40 0x7f582c19f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:58:55.555 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:55.553+0000 7f58310c0700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f582c06a1a0 0x7f582c19af40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.52:60956/0 (socket says 172.21.15.52:60956) 2024-06-10T02:58:55.556 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:55.553+0000 7f58310c0700 1 -- 172.21.15.52:0/779899378 learned_addr learned my addr 172.21.15.52:0/779899378 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:58:55.556 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:55.553+0000 7f58308bf700 1 --2- 172.21.15.52:0/779899378 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f582c06aac0 0x7f582c19b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:58:55.556 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:55.553+0000 7f58310c0700 1 -- 172.21.15.52:0/779899378 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f582c06aac0 msgr2=0x7f582c19b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:58:55.556 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:55.553+0000 7f58310c0700 1 --2- 172.21.15.52:0/779899378 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f582c06aac0 0x7f582c19b480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:55.556 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:55.553+0000 7f58310c0700 1 -- 172.21.15.52:0/779899378 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f582c10ef40 msgr2=0x7f582c19f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:58:55.556 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:55.553+0000 7f58310c0700 1 --2- 172.21.15.52:0/779899378 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f582c10ef40 0x7f582c19f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:55.556 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:55.553+0000 7f58310c0700 1 -- 172.21.15.52:0/779899378 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f582c19ff50 con 0x7f582c06a1a0 2024-06-10T02:58:55.556 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:55.553+0000 7f58308bf700 1 --2- 172.21.15.52:0/779899378 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f582c06aac0 0x7f582c19b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T02:58:55.556 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:55.553+0000 7f58318c1700 1 --2- 172.21.15.52:0/779899378 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f582c10ef40 0x7f582c19f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T02:58:55.556 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:55.554+0000 7f58310c0700 1 --2- 172.21.15.52:0/779899378 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f582c06a1a0 0x7f582c19af40 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f582800b570 tx=0x7f582800b880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:58:55.556 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:55.554+0000 7f58227fc700 1 -- 172.21.15.52:0/779899378 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5828022070 con 0x7f582c06a1a0 2024-06-10T02:58:55.556 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:55.554+0000 7f5833324700 1 -- 172.21.15.52:0/779899378 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f582c1a0240 con 0x7f582c06a1a0 2024-06-10T02:58:55.556 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:55.554+0000 7f58227fc700 1 -- 172.21.15.52:0/779899378 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f582801d8f0 con 0x7f582c06a1a0 2024-06-10T02:58:55.557 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:55.554+0000 7f58227fc700 1 -- 172.21.15.52:0/779899378 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f582801c8e0 con 0x7f582c06a1a0 2024-06-10T02:58:55.557 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:55.554+0000 7f5833324700 1 -- 172.21.15.52:0/779899378 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f582c1035a0 con 0x7f582c06a1a0 2024-06-10T02:58:55.557 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:55.556+0000 7f5833324700 1 -- 172.21.15.52:0/779899378 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f582c069180 con 0x7f582c06a1a0 2024-06-10T02:58:55.557 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:55.557+0000 7f58227fc700 1 -- 172.21.15.52:0/779899378 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f5828025070 con 0x7f582c06a1a0 2024-06-10T02:58:55.557 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:55.558+0000 7f58227fc700 1 --2- 172.21.15.52:0/779899378 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f58180a2c20 0x7f58180a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:58:55.557 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:55.558+0000 7f58227fc700 1 -- 172.21.15.52:0/779899378 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f58280d7510 con 0x7f582c06a1a0 2024-06-10T02:58:55.558 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:55.558+0000 7f58308bf700 1 --2- 172.21.15.52:0/779899378 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f58180a2c20 0x7f58180a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:58:55.558 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:55.558+0000 7f58308bf700 1 --2- 172.21.15.52:0/779899378 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f58180a2c20 0x7f58180a50e0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f581c00b2e0 tx=0x7f581c005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:58:55.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:55.565+0000 7f58227fc700 1 -- 172.21.15.52:0/779899378 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f58280a4060 con 0x7f582c06a1a0 2024-06-10T02:58:55.579 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:58:55.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:55.846+0000 7f5833324700 1 -- 172.21.15.52:0/779899378 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f582c02dca0 con 0x7f582c06a1a0 2024-06-10T02:58:55.846 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:55.846+0000 7f58227fc700 1 -- 172.21.15.52:0/779899378 <== mon.1 v2:172.21.15.188:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11945 (secure 0 0 0) 0x7f58280a3960 con 0x7f582c06a1a0 2024-06-10T02:58:55.846 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-10T02:58:55.846 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/889673247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":889673247},{"type":"v1","addr":"172.21.15.52:6835","nonce":889673247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/3615663511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":3615663511},{"type":"v1","addr":"172.21.15.52:6837","nonce":3615663511}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T02:50:03.889766+0000","modified":"2024-06-10T02:58:01.195103+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24484,"mds_1":24469,"mds_2":24463,"mds_3":24449,"mds_4":24428},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14571":{"gid":14571,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6839/3059599042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":3059599042},{"type":"v1","addr":"172.21.15.52:6839","nonce":3059599042}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14583":{"gid":14583,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6841/3039275521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":3039275521},{"type":"v1","addr":"172.21.15.52:6841","nonce":3039275521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24383":{"gid":24383,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6833/2609002896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":2609002896},{"type":"v1","addr":"172.21.15.120:6833","nonce":2609002896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6835/1996370288","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":1996370288},{"type":"v1","addr":"172.21.15.120:6835","nonce":1996370288}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6833/529564979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":529564979},{"type":"v1","addr":"172.21.15.188:6833","nonce":529564979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":25,"addr":"172.21.15.120:6837/4132440952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":4132440952},{"type":"v1","addr":"172.21.15.120:6837","nonce":4132440952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"b","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.120:6839/1672436487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1672436487},{"type":"v1","addr":"172.21.15.120:6839","nonce":1672436487}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":2,"incarnation":22,"state":"up:active","state_seq":26,"addr":"172.21.15.188:6835/3904544259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":3904544259},{"type":"v1","addr":"172.21.15.188:6835","nonce":3904544259}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.188:6837/819317726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":819317726},{"type":"v1","addr":"172.21.15.188:6837","nonce":819317726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.188:6839/310444836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":310444836},{"type":"v1","addr":"172.21.15.188:6839","nonce":310444836}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24484,"qdb_cluster":[24469,24484,24463,24428,24449]},"id":1}]} 2024-06-10T02:58:55.849 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:55.849+0000 7f5833324700 1 -- 172.21.15.52:0/779899378 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f58180a2c20 msgr2=0x7f58180a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:58:55.849 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:55.849+0000 7f5833324700 1 --2- 172.21.15.52:0/779899378 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f58180a2c20 0x7f58180a50e0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f581c00b2e0 tx=0x7f581c005960 comp rx=0 tx=0).stop 2024-06-10T02:58:55.849 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:55.849+0000 7f5833324700 1 -- 172.21.15.52:0/779899378 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f582c06a1a0 msgr2=0x7f582c19af40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:58:55.849 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:55.849+0000 7f5833324700 1 --2- 172.21.15.52:0/779899378 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f582c06a1a0 0x7f582c19af40 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f582800b570 tx=0x7f582800b880 comp rx=0 tx=0).stop 2024-06-10T02:58:55.849 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:55.849+0000 7f5833324700 1 -- 172.21.15.52:0/779899378 shutdown_connections 2024-06-10T02:58:55.849 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:55.849+0000 7f5833324700 1 --2- 172.21.15.52:0/779899378 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f58180a2c20 0x7f58180a50e0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:55.849 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:55.849+0000 7f5833324700 1 --2- 172.21.15.52:0/779899378 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f582c10ef40 0x7f582c19f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:55.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:55.849+0000 7f5833324700 1 --2- 172.21.15.52:0/779899378 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f582c06a1a0 0x7f582c19af40 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:55.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:55.849+0000 7f5833324700 1 --2- 172.21.15.52:0/779899378 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f582c06aac0 0x7f582c19b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:55.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:55.849+0000 7f5833324700 1 -- 172.21.15.52:0/779899378 >> 172.21.15.52:0/779899378 conn(0x7f582c078340 msgr2=0x7f582c110490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:58:55.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:55.849+0000 7f5833324700 1 -- 172.21.15.52:0/779899378 shutdown_connections 2024-06-10T02:58:55.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:55.849+0000 7f5833324700 1 -- 172.21.15.52:0/779899378 wait complete. 2024-06-10T02:58:55.850 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 34 2024-06-10T02:58:57.091 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:58:56 smithi052 ceph-mon[32362]: pgmap v576: 129 pgs: 129 active+clean; 475 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.7 MiB/s wr, 293 op/s 2024-06-10T02:58:57.091 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:58:56 smithi052 ceph-mon[32362]: from='client.? 172.21.15.52:0/779899378' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T02:58:57.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:58:56 smithi120 ceph-mon[36634]: pgmap v576: 129 pgs: 129 active+clean; 475 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.7 MiB/s wr, 293 op/s 2024-06-10T02:58:57.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:58:56 smithi120 ceph-mon[36634]: from='client.? 172.21.15.52:0/779899378' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T02:58:57.204 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:58:56 smithi188 ceph-mon[36599]: pgmap v576: 129 pgs: 129 active+clean; 475 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.7 MiB/s wr, 293 op/s 2024-06-10T02:58:57.204 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:58:56 smithi188 ceph-mon[36599]: from='client.? 172.21.15.52:0/779899378' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T02:58:57.246 INFO:tasks.workunit.client.0.smithi052.stdout: 74 9491 0 6516 0 5608 0 2024-06-10T02:58:57.944 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24484 quiesce db --set-id b06ab081 --release --await' 2024-06-10T02:58:57.945 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24484 quiesce db --set-id b06ab081 --release --await 2024-06-10T02:58:58.962 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:58:58.976 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:58:58 smithi052 ceph-mon[32362]: pgmap v577: 129 pgs: 129 active+clean; 477 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 2.5 MiB/s wr, 338 op/s 2024-06-10T02:58:58.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:58.982+0000 7f71e4f45700 1 -- 172.21.15.52:0/3877080675 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f71e0108500 msgr2=0x7f71e01088e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:58:58.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:58.982+0000 7f71e4f45700 1 --2- 172.21.15.52:0/3877080675 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f71e0108500 0x7f71e01088e0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f71d0009990 tx=0x7f71d0009ca0 comp rx=0 tx=0).stop 2024-06-10T02:58:58.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:58.982+0000 7f71e4f45700 1 -- 172.21.15.52:0/3877080675 shutdown_connections 2024-06-10T02:58:58.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:58.982+0000 7f71e4f45700 1 --2- 172.21.15.52:0/3877080675 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f71e006ab30 0x7f71e010f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:58.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:58.982+0000 7f71e4f45700 1 --2- 172.21.15.52:0/3877080675 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f71e0108500 0x7f71e01088e0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:58.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:58.982+0000 7f71e4f45700 1 --2- 172.21.15.52:0/3877080675 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f71e006a170 0x7f71e006a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:58.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:58.982+0000 7f71e4f45700 1 -- 172.21.15.52:0/3877080675 >> 172.21.15.52:0/3877080675 conn(0x7f71e0078340 msgr2=0x7f71e0078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:58:58.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:58.982+0000 7f71e4f45700 1 -- 172.21.15.52:0/3877080675 shutdown_connections 2024-06-10T02:58:58.983 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:58.982+0000 7f71e4f45700 1 -- 172.21.15.52:0/3877080675 wait complete. 2024-06-10T02:58:58.984 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:58.983+0000 7f71e4f45700 1 Processor -- start 2024-06-10T02:58:58.984 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:58.983+0000 7f71e4f45700 1 -- start start 2024-06-10T02:58:58.984 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:58.983+0000 7f71e4f45700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f71e006a170 0x7f71e019e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:58:58.984 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:58.983+0000 7f71e4f45700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f71e006ab30 0x7f71e019ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:58:58.984 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:58.983+0000 7f71e4f45700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f71e0108500 0x7f71e0198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:58:58.984 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:58.983+0000 7f71e4f45700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f71e0112780 con 0x7f71e0108500 2024-06-10T02:58:58.984 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:58.983+0000 7f71e4f45700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f71e0112600 con 0x7f71e006a170 2024-06-10T02:58:58.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:58.983+0000 7f71e4f45700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f71e0112900 con 0x7f71e006ab30 2024-06-10T02:58:58.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:58.984+0000 7f71de59c700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f71e006a170 0x7f71e019e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:58:58.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:58.984+0000 7f71de59c700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f71e006a170 0x7f71e019e8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.52:54490/0 (socket says 172.21.15.52:54490) 2024-06-10T02:58:58.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:58.984+0000 7f71de59c700 1 -- 172.21.15.52:0/1534350514 learned_addr learned my addr 172.21.15.52:0/1534350514 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:58:58.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:58.984+0000 7f71ddd9b700 1 --2- 172.21.15.52:0/1534350514 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f71e006ab30 0x7f71e019ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:58:58.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:58.984+0000 7f71ded9d700 1 --2- 172.21.15.52:0/1534350514 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f71e0108500 0x7f71e0198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:58:58.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:58.984+0000 7f71de59c700 1 -- 172.21.15.52:0/1534350514 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f71e006ab30 msgr2=0x7f71e019ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:58:58.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:58.984+0000 7f71de59c700 1 --2- 172.21.15.52:0/1534350514 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f71e006ab30 0x7f71e019ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:58.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:58.984+0000 7f71de59c700 1 -- 172.21.15.52:0/1534350514 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f71e0108500 msgr2=0x7f71e0198aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:58:58.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:58.984+0000 7f71de59c700 1 --2- 172.21.15.52:0/1534350514 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f71e0108500 0x7f71e0198aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:58.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:58.984+0000 7f71de59c700 1 -- 172.21.15.52:0/1534350514 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71e0199200 con 0x7f71e006a170 2024-06-10T02:58:58.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:58.984+0000 7f71de59c700 1 --2- 172.21.15.52:0/1534350514 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f71e006a170 0x7f71e019e8e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f71d0000c00 tx=0x7f71d00047b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:58:58.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:58.984+0000 7f71cf7fe700 1 -- 172.21.15.52:0/1534350514 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71d0027070 con 0x7f71e006a170 2024-06-10T02:58:58.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:58.984+0000 7f71e4f45700 1 -- 172.21.15.52:0/1534350514 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71e0199490 con 0x7f71e006a170 2024-06-10T02:58:58.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:58.984+0000 7f71cf7fe700 1 -- 172.21.15.52:0/1534350514 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71d0003e40 con 0x7f71e006a170 2024-06-10T02:58:58.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:58.984+0000 7f71e4f45700 1 -- 172.21.15.52:0/1534350514 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71e01a57b0 con 0x7f71e006a170 2024-06-10T02:58:58.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:58.985+0000 7f71cf7fe700 1 -- 172.21.15.52:0/1534350514 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71d0022d10 con 0x7f71e006a170 2024-06-10T02:58:58.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:58.987+0000 7f71cf7fe700 1 -- 172.21.15.52:0/1534350514 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f71d002a070 con 0x7f71e006a170 2024-06-10T02:58:58.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:58.988+0000 7f71cf7fe700 1 --2- 172.21.15.52:0/1534350514 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f71c80a2c20 0x7f71c80a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:58:58.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:58.988+0000 7f71cf7fe700 1 -- 172.21.15.52:0/1534350514 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f71d00d0050 con 0x7f71e006a170 2024-06-10T02:58:58.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:58.988+0000 7f71ddd9b700 1 --2- 172.21.15.52:0/1534350514 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f71c80a2c20 0x7f71c80a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:58:58.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:58.988+0000 7f71ef7c8b80 1 Processor -- start 2024-06-10T02:58:58.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:58.989+0000 7f71ef7c8b80 1 -- start start 2024-06-10T02:58:58.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:58.989+0000 7f71ddd9b700 1 --2- 172.21.15.52:0/1534350514 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f71c80a2c20 0x7f71c80a50e0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f71e00789d0 tx=0x7f71d40073d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:58:58.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:58.989+0000 7f71ef7c8b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e673ea8640 0x55e673ea8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:58:58.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:58.989+0000 7f71ef7c8b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e673ea8f60 0x55e673ea29e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:58:58.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:58.989+0000 7f71ef7c8b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e673ea2f20 0x55e673ea3340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:58:58.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:58.989+0000 7f71ef7c8b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55e673df4810 con 0x55e673ea2f20 2024-06-10T02:58:58.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:58.989+0000 7f71ef7c8b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55e673d9c720 con 0x55e673ea8f60 2024-06-10T02:58:58.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:58.989+0000 7f71ef7c8b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55e673df07d0 con 0x55e673ea8640 2024-06-10T02:58:58.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:58.989+0000 7f71ded9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e673ea2f20 0x55e673ea3340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:58:58.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:58.989+0000 7f71ded9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e673ea2f20 0x55e673ea3340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:59712/0 (socket says 172.21.15.52:59712) 2024-06-10T02:58:58.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:58.989+0000 7f71ded9d700 1 -- 172.21.15.52:0/2297698665 learned_addr learned my addr 172.21.15.52:0/2297698665 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:58:58.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:58.989+0000 7f71ddd9b700 1 --2- 172.21.15.52:0/2297698665 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e673ea8f60 0x55e673ea29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:58:58.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:58.989+0000 7f71de59c700 1 --2- 172.21.15.52:0/2297698665 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e673ea8640 0x55e673ea8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:58:58.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:58.989+0000 7f71ded9d700 1 -- 172.21.15.52:0/2297698665 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e673ea8640 msgr2=0x55e673ea8a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:58:58.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:58.989+0000 7f71ded9d700 1 --2- 172.21.15.52:0/2297698665 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e673ea8640 0x55e673ea8a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:58.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:58.989+0000 7f71ded9d700 1 -- 172.21.15.52:0/2297698665 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e673ea8f60 msgr2=0x55e673ea29e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:58:58.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:58.989+0000 7f71ded9d700 1 --2- 172.21.15.52:0/2297698665 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e673ea8f60 0x55e673ea29e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:58.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:58.989+0000 7f71ded9d700 1 -- 172.21.15.52:0/2297698665 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e673d92f50 con 0x55e673ea2f20 2024-06-10T02:58:58.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:58.989+0000 7f71ddd9b700 1 --2- 172.21.15.52:0/2297698665 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e673ea8f60 0x55e673ea29e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T02:58:58.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:58.990+0000 7f71ded9d700 1 --2- 172.21.15.52:0/2297698665 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e673ea2f20 0x55e673ea3340 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f71d8003fa0 tx=0x7f71d8009ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffcf9c13aee53251 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:58:58.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:58.990+0000 7f71ccff9700 1 -- 172.21.15.52:0/2297698665 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71d8004bb0 con 0x55e673ea2f20 2024-06-10T02:58:58.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:58.990+0000 7f71ccff9700 1 -- 172.21.15.52:0/2297698665 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71d8004d50 con 0x55e673ea2f20 2024-06-10T02:58:58.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:58.990+0000 7f71ccff9700 1 -- 172.21.15.52:0/2297698665 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71d8007de0 con 0x55e673ea2f20 2024-06-10T02:58:58.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:58.991+0000 7f71ef7c8b80 1 -- 172.21.15.52:0/2297698665 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e673ea2f20 msgr2=0x55e673ea3340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:58:58.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:58.991+0000 7f71ef7c8b80 1 --2- 172.21.15.52:0/2297698665 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e673ea2f20 0x55e673ea3340 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f71d8003fa0 tx=0x7f71d8009ba0 comp rx=0 tx=0).stop 2024-06-10T02:58:58.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:58.991+0000 7f71ef7c8b80 1 -- 172.21.15.52:0/2297698665 shutdown_connections 2024-06-10T02:58:58.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:58.991+0000 7f71ef7c8b80 1 --2- 172.21.15.52:0/2297698665 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e673ea2f20 0x55e673ea3340 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:58.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:58.991+0000 7f71ef7c8b80 1 --2- 172.21.15.52:0/2297698665 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e673ea8640 0x55e673ea8a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:58.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:58.991+0000 7f71ef7c8b80 1 --2- 172.21.15.52:0/2297698665 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e673ea8f60 0x55e673ea29e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:58.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:58.991+0000 7f71ef7c8b80 1 -- 172.21.15.52:0/2297698665 >> 172.21.15.52:0/2297698665 conn(0x55e673dcdcc0 msgr2=0x55e673e9d850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:58:58.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:58.991+0000 7f71ef7c8b80 1 -- 172.21.15.52:0/2297698665 shutdown_connections 2024-06-10T02:58:58.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:58.991+0000 7f71ef7c8b80 1 -- 172.21.15.52:0/2297698665 wait complete. 2024-06-10T02:58:58.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:58.991+0000 7f71ef7c8b80 1 -- start start 2024-06-10T02:58:58.994 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:58.991+0000 7f71ef7c8b80 1 Processor -- start 2024-06-10T02:58:58.996 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:58.992+0000 7f71ef7c8b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e673ea2f20 0x55e673eaec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:58:58.997 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:58.992+0000 7f71ef7c8b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e673ea8640 0x55e673eabfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:58:58.997 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:58.992+0000 7f71ef7c8b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e673ea8f60 0x55e673eac520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:58:58.997 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:58.992+0000 7f71ef7c8b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55e673eb3e30 con 0x55e673ea8640 2024-06-10T02:58:58.997 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:58.992+0000 7f71ef7c8b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55e673d94bf0 con 0x55e673ea8f60 2024-06-10T02:58:58.997 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:58.992+0000 7f71ddd9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e673ea8640 0x55e673eabfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:58:58.997 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:58.992+0000 7f71ef7c8b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55e673eb3fb0 con 0x55e673ea2f20 2024-06-10T02:58:58.997 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:58.992+0000 7f71ddd9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e673ea8640 0x55e673eabfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:59714/0 (socket says 172.21.15.52:59714) 2024-06-10T02:58:58.997 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:58.992+0000 7f71ddd9b700 1 -- 172.21.15.52:0/3356972364 learned_addr learned my addr 172.21.15.52:0/3356972364 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:58:58.997 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:58.992+0000 7f71de59c700 1 --2- 172.21.15.52:0/3356972364 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e673ea2f20 0x55e673eaec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:58:58.997 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:58.992+0000 7f71ef7c8b80 10 client.0 _ll_register_callbacks cb 0x55e673d8e920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T02:58:58.997 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:58.992+0000 7f71ded9d700 1 --2- 172.21.15.52:0/3356972364 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e673ea8f60 0x55e673eac520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:58:58.997 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:58.992+0000 7f71de59c700 1 -- 172.21.15.52:0/3356972364 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e673ea8f60 msgr2=0x55e673eac520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:58:58.997 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:58.992+0000 7f71de59c700 1 --2- 172.21.15.52:0/3356972364 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e673ea8f60 0x55e673eac520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:58.997 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:58.992+0000 7f71de59c700 1 -- 172.21.15.52:0/3356972364 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e673ea8640 msgr2=0x55e673eabfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:58:58.998 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:58.992+0000 7f71de59c700 1 --2- 172.21.15.52:0/3356972364 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e673ea8640 0x55e673eabfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:58.998 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:58.992+0000 7f71de59c700 1 -- 172.21.15.52:0/3356972364 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e673d438d0 con 0x55e673ea2f20 2024-06-10T02:58:58.998 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:58.992+0000 7f71ded9d700 1 --2- 172.21.15.52:0/3356972364 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e673ea8f60 0x55e673eac520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T02:58:58.998 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:58.992+0000 7f71ddd9b700 1 --2- 172.21.15.52:0/3356972364 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e673ea8640 0x55e673eabfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T02:58:58.998 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:58.992+0000 7f71de59c700 1 --2- 172.21.15.52:0/3356972364 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e673ea2f20 0x55e673eaec90 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f71d009c320 tx=0x7f71d0009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ccf09344d8ea43a7 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:58:58.998 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:58.992+0000 7f71c5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-06-10T02:58:58.998 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:58.993+0000 7f71c5ffb700 1 -- 172.21.15.52:0/3356972364 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71d0003ff0 con 0x55e673ea2f20 2024-06-10T02:58:58.998 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:58.993+0000 7f71c5ffb700 1 -- 172.21.15.52:0/3356972364 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f71d009a920 con 0x55e673ea2f20 2024-06-10T02:58:58.998 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:58.993+0000 7f71ef7c8b80 1 -- 172.21.15.52:0/3356972364 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f71d8004d50 con 0x55e673ea2f20 2024-06-10T02:58:58.998 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:58.993+0000 7f71c5ffb700 1 -- 172.21.15.52:0/3356972364 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f71d009cd20 con 0x55e673ea2f20 2024-06-10T02:58:58.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:58.993+0000 7f71ef7c8b80 10 client.25741 fetch_fsmap learned FSMap version 34 2024-06-10T02:58:58.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:58.993+0000 7f71ef7c8b80 1 -- 172.21.15.52:0/3356972364 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55e673eb3e30 con 0x55e673ea2f20 2024-06-10T02:58:58.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:58.993+0000 7f71c5ffb700 1 -- 172.21.15.52:0/3356972364 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71d009c620 con 0x55e673ea2f20 2024-06-10T02:58:58.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:58.993+0000 7f71c5ffb700 1 -- 172.21.15.52:0/3356972364 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 34) ==== 8378+0+0 (secure 0 0 0) 0x7f71d0085bb0 con 0x55e673ea2f20 2024-06-10T02:58:58.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:58.993+0000 7f71ef7c8b80 10 client.25741 fetch_fsmap finished waiting for FSMap version 34 2024-06-10T02:58:58.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:58.993+0000 7f71ef7c8b80 10 client.25741 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-06-10T02:58:58.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:58.993+0000 7f71ef7c8b80 20 client.25741 populate_metadata read hostname 'smithi052' 2024-06-10T02:58:58.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:58.993+0000 7f71ef7c8b80 1 --2- 172.21.15.52:0/3356972364 >> [v2:172.21.15.188:6834/3904544259,v1:172.21.15.188:6835/3904544259] conn(0x55e673ec0ab0 0x55e673ec2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:58:58.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:58.993+0000 7f71ef7c8b80 4 client.25741 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T02:58:58.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:58.993+0000 7f71ef7c8b80 1 -- 172.21.15.52:0/3356972364 --> [v2:172.21.15.188:6834/3904544259,v1:172.21.15.188:6835/3904544259] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f71d8004d50 con 0x55e673ec0ab0 2024-06-10T02:58:58.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:58.994+0000 7f71ddd9b700 1 --2- 172.21.15.52:0/3356972364 >> [v2:172.21.15.188:6834/3904544259,v1:172.21.15.188:6835/3904544259] conn(0x55e673ec0ab0 0x55e673ec2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:58:58.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:58.994+0000 7f71ddd9b700 1 --2- 172.21.15.52:0/3356972364 >> [v2:172.21.15.188:6834/3904544259,v1:172.21.15.188:6835/3904544259] conn(0x55e673ec0ab0 0x55e673ec2ed0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b0857a62d438cbe0 server_cookie=e9421bdc416162fe in_seq=0 out_seq=0 2024-06-10T02:58:58.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:58.994+0000 7f71c5ffb700 10 client.25741 ms_handle_connect on v2:172.21.15.188:6834/3904544259 2024-06-10T02:58:58.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:58.995+0000 7f71c5ffb700 1 -- 172.21.15.52:0/3356972364 <== mds.2 v2:172.21.15.188:6834/3904544259 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x7f71d8004d50 con 0x55e673ec0ab0 2024-06-10T02:58:58.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:58.995+0000 7f71c5ffb700 10 client.25741 handle_command_reply: tid=0 2024-06-10T02:58:59.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:58.995+0000 7f71ef7c8b80 1 client.25741 shutdown 2024-06-10T02:58:59.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:58.995+0000 7f71ef7c8b80 1 -- 172.21.15.52:0/3356972364 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e673ea2f20 msgr2=0x55e673eaec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:58:59.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:58.995+0000 7f71ef7c8b80 1 --2- 172.21.15.52:0/3356972364 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e673ea2f20 0x55e673eaec90 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f71d009c320 tx=0x7f71d0009ca0 comp rx=0 tx=0).stop 2024-06-10T02:58:59.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:58.995+0000 7f71ef7c8b80 1 -- 172.21.15.52:0/3356972364 shutdown_connections 2024-06-10T02:58:59.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:58.995+0000 7f71ef7c8b80 1 --2- 172.21.15.52:0/3356972364 >> [v2:172.21.15.188:6834/3904544259,v1:172.21.15.188:6835/3904544259] conn(0x55e673ec0ab0 0x55e673ec2ed0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:59.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:58.995+0000 7f71ef7c8b80 1 --2- 172.21.15.52:0/3356972364 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e673ea8f60 0x55e673eac520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:59.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:58.995+0000 7f71ef7c8b80 1 --2- 172.21.15.52:0/3356972364 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e673ea2f20 0x55e673eaec90 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:59.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:58.995+0000 7f71c5ffb700 0 client.25741 ms_handle_reset on v2:172.21.15.188:6834/3904544259 2024-06-10T02:58:59.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:58.995+0000 7f71ef7c8b80 1 --2- 172.21.15.52:0/3356972364 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e673ea8640 0x55e673eabfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:59.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:58.995+0000 7f71ef7c8b80 1 -- 172.21.15.52:0/3356972364 >> 172.21.15.52:0/3356972364 conn(0x55e673dcdcc0 msgr2=0x55e673ea19d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:58:59.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:58.995+0000 7f71ef7c8b80 1 -- 172.21.15.52:0/3356972364 shutdown_connections 2024-06-10T02:58:59.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:58.995+0000 7f71ef7c8b80 1 -- 172.21.15.52:0/3356972364 wait complete. 2024-06-10T02:58:59.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:58.995+0000 7f71ef7c8b80 20 client.25741 trim_cache size 0 max 16384 2024-06-10T02:58:59.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:59.011+0000 7f71ef7c8b80 1 Processor -- start 2024-06-10T02:58:59.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:59.011+0000 7f71ef7c8b80 1 -- start start 2024-06-10T02:58:59.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:59.011+0000 7f71ef7c8b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e673ea2f20 0x55e673ea29e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:58:59.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:59.011+0000 7f71ef7c8b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e673ea8640 0x55e673e97510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:58:59.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:59.011+0000 7f71ef7c8b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e673ea8f60 0x55e673ec87a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:58:59.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:59.011+0000 7f71ef7c8b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f71e01ac050 con 0x55e673ea8f60 2024-06-10T02:58:59.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:59.011+0000 7f71ef7c8b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f71e01adc50 con 0x55e673ea2f20 2024-06-10T02:58:59.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:59.011+0000 7f71ef7c8b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f71e01ac720 con 0x55e673ea8640 2024-06-10T02:58:59.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:59.011+0000 7f71ded9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e673ea8f60 0x55e673ec87a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:58:59.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:59.011+0000 7f71de59c700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e673ea2f20 0x55e673ea29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:58:59.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:59.011+0000 7f71ded9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e673ea8f60 0x55e673ec87a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:59728/0 (socket says 172.21.15.52:59728) 2024-06-10T02:58:59.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:59.011+0000 7f71ded9d700 1 -- 172.21.15.52:0/657580976 learned_addr learned my addr 172.21.15.52:0/657580976 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:58:59.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:59.011+0000 7f71ddd9b700 1 --2- 172.21.15.52:0/657580976 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e673ea8640 0x55e673e97510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:58:59.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:59.011+0000 7f71de59c700 1 -- 172.21.15.52:0/657580976 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e673ea8640 msgr2=0x55e673e97510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:58:59.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:59.011+0000 7f71de59c700 1 --2- 172.21.15.52:0/657580976 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e673ea8640 0x55e673e97510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:59.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:59.011+0000 7f71de59c700 1 -- 172.21.15.52:0/657580976 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e673ea8f60 msgr2=0x55e673ec87a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:58:59.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:59.011+0000 7f71de59c700 1 --2- 172.21.15.52:0/657580976 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e673ea8f60 0x55e673ec87a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:59.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:59.011+0000 7f71de59c700 1 -- 172.21.15.52:0/657580976 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e673ea6a40 con 0x55e673ea2f20 2024-06-10T02:58:59.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:59.012+0000 7f71de59c700 1 --2- 172.21.15.52:0/657580976 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e673ea2f20 0x55e673ea29e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f71d0098ff0 tx=0x7f71d0005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=71fbaf95bd9c4086 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:58:59.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:59.012+0000 7f71c7fff700 1 -- 172.21.15.52:0/657580976 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71d0003ff0 con 0x55e673ea2f20 2024-06-10T02:58:59.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:59.012+0000 7f71c7fff700 1 -- 172.21.15.52:0/657580976 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71d009c620 con 0x55e673ea2f20 2024-06-10T02:58:59.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:59.012+0000 7f71c7fff700 1 -- 172.21.15.52:0/657580976 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71d00a4d70 con 0x55e673ea2f20 2024-06-10T02:58:59.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:59.013+0000 7f71ef7c8b80 1 -- 172.21.15.52:0/657580976 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e673ea2f20 msgr2=0x55e673ea29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:58:59.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:59.013+0000 7f71ef7c8b80 1 --2- 172.21.15.52:0/657580976 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e673ea2f20 0x55e673ea29e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f71d0098ff0 tx=0x7f71d0005fd0 comp rx=0 tx=0).stop 2024-06-10T02:58:59.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:59.013+0000 7f71ef7c8b80 1 -- 172.21.15.52:0/657580976 shutdown_connections 2024-06-10T02:58:59.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:59.013+0000 7f71ef7c8b80 1 --2- 172.21.15.52:0/657580976 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e673ea8f60 0x55e673ec87a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:59.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:59.013+0000 7f71ef7c8b80 1 --2- 172.21.15.52:0/657580976 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e673ea2f20 0x55e673ea29e0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:59.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:59.013+0000 7f71ef7c8b80 1 --2- 172.21.15.52:0/657580976 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e673ea8640 0x55e673e97510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:59.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:59.013+0000 7f71ef7c8b80 1 -- 172.21.15.52:0/657580976 >> 172.21.15.52:0/657580976 conn(0x55e673dcdcc0 msgr2=0x55e673e96e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:58:59.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:59.013+0000 7f71ef7c8b80 1 -- 172.21.15.52:0/657580976 shutdown_connections 2024-06-10T02:58:59.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:59.013+0000 7f71ef7c8b80 1 -- 172.21.15.52:0/657580976 wait complete. 2024-06-10T02:58:59.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:59.013+0000 7f71ef7c8b80 1 -- start start 2024-06-10T02:58:59.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:59.013+0000 7f71ef7c8b80 1 Processor -- start 2024-06-10T02:58:59.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:59.013+0000 7f71ef7c8b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e673ea2f20 0x55e673eb6a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:58:59.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:59.013+0000 7f71ef7c8b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e673ea8640 0x55e673eb6f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:58:59.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:59.014+0000 7f71ef7c8b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e673ea8f60 0x55e673f237b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:58:59.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:59.014+0000 7f71ef7c8b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f71e01ad580 con 0x55e673ea8640 2024-06-10T02:58:59.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:59.014+0000 7f71ef7c8b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f71e01aceb0 con 0x55e673ea8f60 2024-06-10T02:58:59.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:59.014+0000 7f71de59c700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e673ea2f20 0x55e673eb6a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:58:59.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:59.014+0000 7f71ddd9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e673ea8640 0x55e673eb6f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:58:59.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:59.014+0000 7f71ef7c8b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f71e01a9ff0 con 0x55e673ea2f20 2024-06-10T02:58:59.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:59.014+0000 7f71ddd9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e673ea8640 0x55e673eb6f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:59732/0 (socket says 172.21.15.52:59732) 2024-06-10T02:58:59.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:59.014+0000 7f71ddd9b700 1 -- 172.21.15.52:0/3783253142 learned_addr learned my addr 172.21.15.52:0/3783253142 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:58:59.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:59.014+0000 7f71de59c700 1 --2- 172.21.15.52:0/3783253142 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e673ea2f20 0x55e673eb6a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.52:32770/0 (socket says 172.21.15.52:32770) 2024-06-10T02:58:59.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:59.014+0000 7f71ded9d700 1 --2- 172.21.15.52:0/3783253142 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e673ea8f60 0x55e673f237b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:58:59.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:59.014+0000 7f71ef7c8b80 10 client.0 _ll_register_callbacks cb 0x55e673d8e920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T02:58:59.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:59.014+0000 7f71ddd9b700 1 -- 172.21.15.52:0/3783253142 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e673ea2f20 msgr2=0x55e673eb6a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:58:59.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:59.014+0000 7f71ddd9b700 1 --2- 172.21.15.52:0/3783253142 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e673ea2f20 0x55e673eb6a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:59.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:59.014+0000 7f71ddd9b700 1 -- 172.21.15.52:0/3783253142 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e673ea8f60 msgr2=0x55e673f237b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:58:59.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:59.014+0000 7f71ddd9b700 1 --2- 172.21.15.52:0/3783253142 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e673ea8f60 0x55e673f237b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:59.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:59.014+0000 7f71ddd9b700 1 -- 172.21.15.52:0/3783253142 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e673f23df0 con 0x55e673ea8640 2024-06-10T02:58:59.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:59.014+0000 7f71ddd9b700 1 --2- 172.21.15.52:0/3783253142 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e673ea8640 0x55e673eb6f40 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x55e673ec3710 tx=0x7f71d40047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65b59c01d5558d81 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:58:59.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:59.014+0000 7f71c5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-10T02:58:59.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:59.014+0000 7f71c5ffb700 1 -- 172.21.15.52:0/3783253142 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71d4003a90 con 0x55e673ea8640 2024-06-10T02:58:59.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:59.014+0000 7f71ef7c8b80 1 -- 172.21.15.52:0/3783253142 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f71d009c620 con 0x55e673ea8640 2024-06-10T02:58:59.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:59.014+0000 7f71c5ffb700 1 -- 172.21.15.52:0/3783253142 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f71d4003c30 con 0x55e673ea8640 2024-06-10T02:58:59.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:59.015+0000 7f71c5ffb700 1 -- 172.21.15.52:0/3783253142 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f71d4038050 con 0x55e673ea8640 2024-06-10T02:58:59.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:59.015+0000 7f71ef7c8b80 10 client.16173 fetch_fsmap learned FSMap version 34 2024-06-10T02:58:59.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:59.015+0000 7f71ef7c8b80 1 -- 172.21.15.52:0/3783253142 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55e673f2b1d0 con 0x55e673ea8640 2024-06-10T02:58:59.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:59.015+0000 7f71c5ffb700 1 -- 172.21.15.52:0/3783253142 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71d009c620 con 0x55e673ea8640 2024-06-10T02:58:59.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:59.015+0000 7f71c5ffb700 1 -- 172.21.15.52:0/3783253142 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 34) ==== 8378+0+0 (secure 0 0 0) 0x7f71d4021580 con 0x55e673ea8640 2024-06-10T02:58:59.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:59.015+0000 7f71ef7c8b80 10 client.16173 fetch_fsmap finished waiting for FSMap version 34 2024-06-10T02:58:59.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:59.015+0000 7f71ef7c8b80 10 client.16173 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-06-10T02:58:59.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:59.015+0000 7f71ef7c8b80 20 client.16173 populate_metadata read hostname 'smithi052' 2024-06-10T02:58:59.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:59.015+0000 7f71ef7c8b80 1 --2- 172.21.15.52:0/3783253142 >> [v2:172.21.15.188:6834/3904544259,v1:172.21.15.188:6835/3904544259] conn(0x55e673f2b980 0x55e673f2dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:58:59.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:59.015+0000 7f71ef7c8b80 4 client.16173 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T02:58:59.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:59.015+0000 7f71ef7c8b80 1 -- 172.21.15.52:0/3783253142 --> [v2:172.21.15.188:6834/3904544259,v1:172.21.15.188:6835/3904544259] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e673e01d10 con 0x55e673f2b980 2024-06-10T02:58:59.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:59.015+0000 7f71de59c700 1 --2- 172.21.15.52:0/3783253142 >> [v2:172.21.15.188:6834/3904544259,v1:172.21.15.188:6835/3904544259] conn(0x55e673f2b980 0x55e673f2dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:58:59.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:59.016+0000 7f71de59c700 1 --2- 172.21.15.52:0/3783253142 >> [v2:172.21.15.188:6834/3904544259,v1:172.21.15.188:6835/3904544259] conn(0x55e673f2b980 0x55e673f2dda0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4cb024f54dd5a4ba server_cookie=5237f08d64454c56 in_seq=0 out_seq=0 2024-06-10T02:58:59.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:59.016+0000 7f71c5ffb700 10 client.16173 ms_handle_connect on v2:172.21.15.188:6834/3904544259 2024-06-10T02:58:59.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:59.016+0000 7f71c5ffb700 1 -- 172.21.15.52:0/3783253142 <== mds.2 v2:172.21.15.188:6834/3904544259 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e673e01d10 con 0x55e673f2b980 2024-06-10T02:58:59.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:59.016+0000 7f71c5ffb700 10 client.16173 handle_command_reply: tid=0 2024-06-10T02:58:59.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:59.016+0000 7f71ef7c8b80 1 client.16173 shutdown 2024-06-10T02:58:59.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:59.016+0000 7f71ef7c8b80 1 -- 172.21.15.52:0/3783253142 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e673ea8640 msgr2=0x55e673eb6f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:58:59.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:59.016+0000 7f71ef7c8b80 1 --2- 172.21.15.52:0/3783253142 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e673ea8640 0x55e673eb6f40 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x55e673ec3710 tx=0x7f71d40047c0 comp rx=0 tx=0).stop 2024-06-10T02:58:59.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:59.016+0000 7f71ef7c8b80 1 -- 172.21.15.52:0/3783253142 shutdown_connections 2024-06-10T02:58:59.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:59.016+0000 7f71ef7c8b80 1 --2- 172.21.15.52:0/3783253142 >> [v2:172.21.15.188:6834/3904544259,v1:172.21.15.188:6835/3904544259] conn(0x55e673f2b980 0x55e673f2dda0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:59.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:59.016+0000 7f71ef7c8b80 1 --2- 172.21.15.52:0/3783253142 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e673ea8f60 0x55e673f237b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:59.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:59.016+0000 7f71c5ffb700 0 client.16173 ms_handle_reset on v2:172.21.15.188:6834/3904544259 2024-06-10T02:58:59.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:59.016+0000 7f71ef7c8b80 1 --2- 172.21.15.52:0/3783253142 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e673ea2f20 0x55e673eb6a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:59.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:59.016+0000 7f71ef7c8b80 1 --2- 172.21.15.52:0/3783253142 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e673ea8640 0x55e673eb6f40 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:59.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:59.016+0000 7f71ef7c8b80 1 -- 172.21.15.52:0/3783253142 >> 172.21.15.52:0/3783253142 conn(0x55e673dcdcc0 msgr2=0x55e673ead7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:58:59.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:59.016+0000 7f71ef7c8b80 1 -- 172.21.15.52:0/3783253142 shutdown_connections 2024-06-10T02:58:59.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:59.016+0000 7f71ef7c8b80 1 -- 172.21.15.52:0/3783253142 wait complete. 2024-06-10T02:58:59.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:59.017+0000 7f71ef7c8b80 20 client.16173 trim_cache size 0 max 16384 2024-06-10T02:58:59.023 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-06-10T02:58:59.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:59.017+0000 7f71e4f45700 1 -- 172.21.15.52:0/1534350514 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f71c80a2c20 msgr2=0x7f71c80a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:58:59.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:59.017+0000 7f71e4f45700 1 --2- 172.21.15.52:0/1534350514 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f71c80a2c20 0x7f71c80a50e0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f71e00789d0 tx=0x7f71d40073d0 comp rx=0 tx=0).stop 2024-06-10T02:58:59.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:59.017+0000 7f71e4f45700 1 -- 172.21.15.52:0/1534350514 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f71e006a170 msgr2=0x7f71e019e8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:58:59.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:59.017+0000 7f71e4f45700 1 --2- 172.21.15.52:0/1534350514 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f71e006a170 0x7f71e019e8e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f71d0000c00 tx=0x7f71d00047b0 comp rx=0 tx=0).stop 2024-06-10T02:58:59.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:59.017+0000 7f71e4f45700 1 -- 172.21.15.52:0/1534350514 shutdown_connections 2024-06-10T02:58:59.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:59.017+0000 7f71e4f45700 1 --2- 172.21.15.52:0/1534350514 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f71c80a2c20 0x7f71c80a50e0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:59.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:59.017+0000 7f71e4f45700 1 --2- 172.21.15.52:0/1534350514 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f71e0108500 0x7f71e0198aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:59.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:59.017+0000 7f71e4f45700 1 --2- 172.21.15.52:0/1534350514 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f71e006a170 0x7f71e019e8e0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:59.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:59.017+0000 7f71e4f45700 1 --2- 172.21.15.52:0/1534350514 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f71e006ab30 0x7f71e019ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:58:59.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:59.017+0000 7f71e4f45700 1 -- 172.21.15.52:0/1534350514 >> 172.21.15.52:0/1534350514 conn(0x7f71e0078340 msgr2=0x7f71e0100090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:58:59.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:59.017+0000 7f71e4f45700 1 -- 172.21.15.52:0/1534350514 shutdown_connections 2024-06-10T02:58:59.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:58:59.017+0000 7f71e4f45700 1 -- 172.21.15.52:0/1534350514 wait complete. 2024-06-10T02:58:59.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:58:58 smithi120 ceph-mon[36634]: pgmap v577: 129 pgs: 129 active+clean; 477 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 2.5 MiB/s wr, 338 op/s 2024-06-10T02:58:59.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:58:58 smithi188 ceph-mon[36599]: pgmap v577: 129 pgs: 129 active+clean; 477 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 2.5 MiB/s wr, 338 op/s 2024-06-10T02:59:00.620 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-06-10T02:59:00.976 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:00 smithi052 ceph-mon[32362]: pgmap v578: 129 pgs: 129 active+clean; 477 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 1.9 MiB/s wr, 259 op/s 2024-06-10T02:59:01.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:00 smithi120 ceph-mon[36634]: pgmap v578: 129 pgs: 129 active+clean; 477 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 1.9 MiB/s wr, 259 op/s 2024-06-10T02:59:01.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:00 smithi188 ceph-mon[36599]: pgmap v578: 129 pgs: 129 active+clean; 477 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 1.9 MiB/s wr, 259 op/s 2024-06-10T02:59:01.507 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:59:01.523 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.523+0000 7fede4632700 1 -- 172.21.15.52:0/1221395838 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feddc101d50 msgr2=0x7feddc10e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:59:01.523 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.523+0000 7fede4632700 1 --2- 172.21.15.52:0/1221395838 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feddc101d50 0x7feddc10e240 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fedd400b0a0 tx=0x7fedd400b3b0 comp rx=0 tx=0).stop 2024-06-10T02:59:01.523 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.523+0000 7fede4632700 1 -- 172.21.15.52:0/1221395838 shutdown_connections 2024-06-10T02:59:01.524 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.523+0000 7fede4632700 1 --2- 172.21.15.52:0/1221395838 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feddc101d50 0x7feddc10e240 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:01.524 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.523+0000 7fede4632700 1 --2- 172.21.15.52:0/1221395838 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feddc107400 0x7feddc1077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:01.524 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.523+0000 7fede4632700 1 --2- 172.21.15.52:0/1221395838 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7feddc101390 0x7feddc101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:01.524 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.523+0000 7fede4632700 1 -- 172.21.15.52:0/1221395838 >> 172.21.15.52:0/1221395838 conn(0x7feddc078340 msgr2=0x7feddc078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:59:01.524 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.523+0000 7fede4632700 1 -- 172.21.15.52:0/1221395838 shutdown_connections 2024-06-10T02:59:01.524 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.523+0000 7fede4632700 1 -- 172.21.15.52:0/1221395838 wait complete. 2024-06-10T02:59:01.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.524+0000 7fede4632700 1 Processor -- start 2024-06-10T02:59:01.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.524+0000 7fede4632700 1 -- start start 2024-06-10T02:59:01.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.524+0000 7fede4632700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feddc101390 0x7feddc19e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:59:01.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.524+0000 7fede4632700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feddc101d50 0x7feddc19ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:59:01.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.524+0000 7fede4632700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7feddc107400 0x7feddc1989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:59:01.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.524+0000 7fede4632700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7feddc111670 con 0x7feddc101390 2024-06-10T02:59:01.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.524+0000 7fede4632700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7feddc1114f0 con 0x7feddc107400 2024-06-10T02:59:01.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.524+0000 7fede4632700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7feddc1117f0 con 0x7feddc101d50 2024-06-10T02:59:01.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.524+0000 7fede23ce700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feddc101390 0x7feddc19e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:59:01.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.524+0000 7fede23ce700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feddc101390 0x7feddc19e7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56644/0 (socket says 172.21.15.52:56644) 2024-06-10T02:59:01.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.524+0000 7fede23ce700 1 -- 172.21.15.52:0/2552181168 learned_addr learned my addr 172.21.15.52:0/2552181168 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:59:01.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.524+0000 7fede1bcd700 1 --2- 172.21.15.52:0/2552181168 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feddc101d50 0x7feddc19ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:59:01.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.525+0000 7fede2bcf700 1 --2- 172.21.15.52:0/2552181168 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7feddc107400 0x7feddc1989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:59:01.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.525+0000 7fede23ce700 1 -- 172.21.15.52:0/2552181168 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feddc101d50 msgr2=0x7feddc19ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:59:01.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.525+0000 7fede23ce700 1 --2- 172.21.15.52:0/2552181168 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feddc101d50 0x7feddc19ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:01.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.525+0000 7fede23ce700 1 -- 172.21.15.52:0/2552181168 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7feddc107400 msgr2=0x7feddc1989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:59:01.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.525+0000 7fede23ce700 1 --2- 172.21.15.52:0/2552181168 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7feddc107400 0x7feddc1989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:01.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.525+0000 7fede23ce700 1 -- 172.21.15.52:0/2552181168 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feddc199150 con 0x7feddc101390 2024-06-10T02:59:01.528 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.525+0000 7fede23ce700 1 --2- 172.21.15.52:0/2552181168 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feddc101390 0x7feddc19e7e0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fedd800e9a0 tx=0x7fedd800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:59:01.528 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.525+0000 7fedd37fe700 1 -- 172.21.15.52:0/2552181168 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedd800cbd0 con 0x7feddc101390 2024-06-10T02:59:01.528 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.525+0000 7fedd37fe700 1 -- 172.21.15.52:0/2552181168 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fedd800cd70 con 0x7feddc101390 2024-06-10T02:59:01.528 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.525+0000 7fedd37fe700 1 -- 172.21.15.52:0/2552181168 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedd8018750 con 0x7feddc101390 2024-06-10T02:59:01.528 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.525+0000 7fede4632700 1 -- 172.21.15.52:0/2552181168 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feddc199440 con 0x7feddc101390 2024-06-10T02:59:01.528 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.526+0000 7fede4632700 1 -- 172.21.15.52:0/2552181168 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feddc1a56b0 con 0x7feddc101390 2024-06-10T02:59:01.528 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.528+0000 7fedd37fe700 1 -- 172.21.15.52:0/2552181168 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fedd8010430 con 0x7feddc101390 2024-06-10T02:59:01.529 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.528+0000 7fedd37fe700 1 --2- 172.21.15.52:0/2552181168 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fedc80a2ca0 0x7fedc80a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:59:01.529 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.529+0000 7fedd37fe700 1 -- 172.21.15.52:0/2552181168 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7fedd80cf5e0 con 0x7feddc101390 2024-06-10T02:59:01.529 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.529+0000 7fede1bcd700 1 --2- 172.21.15.52:0/2552181168 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fedc80a2ca0 0x7fedc80a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:59:01.529 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.529+0000 7fede1bcd700 1 --2- 172.21.15.52:0/2552181168 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fedc80a2ca0 0x7fedc80a5160 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7feddc078920 tx=0x7fedcc009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:59:01.530 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.531+0000 7fedeeeb5b80 1 Processor -- start 2024-06-10T02:59:01.530 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.531+0000 7fedeeeb5b80 1 -- start start 2024-06-10T02:59:01.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.531+0000 7fedeeeb5b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5598f9f17c40 0x5598f9f18020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:59:01.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.531+0000 7fedeeeb5b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5598f9f18560 0x5598f9f11e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:59:01.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.531+0000 7fedeeeb5b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5598f9f123d0 0x5598f9f12810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:59:01.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.531+0000 7fedeeeb5b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5598f9e63910 con 0x5598f9f17c40 2024-06-10T02:59:01.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.531+0000 7fedeeeb5b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5598f9e0b820 con 0x5598f9f123d0 2024-06-10T02:59:01.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.531+0000 7fedeeeb5b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5598f9e5f8d0 con 0x5598f9f18560 2024-06-10T02:59:01.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.531+0000 7fede23ce700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5598f9f17c40 0x5598f9f18020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:59:01.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.531+0000 7fede23ce700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5598f9f17c40 0x5598f9f18020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56646/0 (socket says 172.21.15.52:56646) 2024-06-10T02:59:01.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.531+0000 7fede23ce700 1 -- 172.21.15.52:0/309244352 learned_addr learned my addr 172.21.15.52:0/309244352 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:59:01.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.531+0000 7fede1bcd700 1 --2- 172.21.15.52:0/309244352 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5598f9f18560 0x5598f9f11e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:59:01.533 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.531+0000 7fede2bcf700 1 --2- 172.21.15.52:0/309244352 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5598f9f123d0 0x5598f9f12810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:59:01.533 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.532+0000 7fede23ce700 1 -- 172.21.15.52:0/309244352 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5598f9f18560 msgr2=0x5598f9f11e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:59:01.533 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.532+0000 7fede23ce700 1 --2- 172.21.15.52:0/309244352 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5598f9f18560 0x5598f9f11e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:01.533 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.532+0000 7fede23ce700 1 -- 172.21.15.52:0/309244352 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5598f9f123d0 msgr2=0x5598f9f12810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:59:01.533 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.532+0000 7fede23ce700 1 --2- 172.21.15.52:0/309244352 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5598f9f123d0 0x5598f9f12810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:01.533 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.532+0000 7fede23ce700 1 -- 172.21.15.52:0/309244352 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5598f9e02050 con 0x5598f9f17c40 2024-06-10T02:59:01.533 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.532+0000 7fede23ce700 1 --2- 172.21.15.52:0/309244352 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5598f9f17c40 0x5598f9f18020 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fedd8098460 tx=0x7fedd809b6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96b8813d7b7a3a6c server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:59:01.533 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.532+0000 7fedd0ff9700 1 -- 172.21.15.52:0/309244352 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedd809ba90 con 0x5598f9f17c40 2024-06-10T02:59:01.533 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.532+0000 7fedd0ff9700 1 -- 172.21.15.52:0/309244352 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fedd809fe40 con 0x5598f9f17c40 2024-06-10T02:59:01.533 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.532+0000 7fedd0ff9700 1 -- 172.21.15.52:0/309244352 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedd80a4270 con 0x5598f9f17c40 2024-06-10T02:59:01.535 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.533+0000 7fedeeeb5b80 1 -- 172.21.15.52:0/309244352 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5598f9f17c40 msgr2=0x5598f9f18020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:59:01.535 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.533+0000 7fedeeeb5b80 1 --2- 172.21.15.52:0/309244352 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5598f9f17c40 0x5598f9f18020 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fedd8098460 tx=0x7fedd809b6a0 comp rx=0 tx=0).stop 2024-06-10T02:59:01.535 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.533+0000 7fedeeeb5b80 1 -- 172.21.15.52:0/309244352 shutdown_connections 2024-06-10T02:59:01.535 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.533+0000 7fedeeeb5b80 1 --2- 172.21.15.52:0/309244352 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5598f9f123d0 0x5598f9f12810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:01.535 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.533+0000 7fedeeeb5b80 1 --2- 172.21.15.52:0/309244352 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5598f9f17c40 0x5598f9f18020 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:01.536 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.533+0000 7fedeeeb5b80 1 --2- 172.21.15.52:0/309244352 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5598f9f18560 0x5598f9f11e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:01.536 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.533+0000 7fedeeeb5b80 1 -- 172.21.15.52:0/309244352 >> 172.21.15.52:0/309244352 conn(0x5598f9e3cdc0 msgr2=0x5598f9f0d120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:59:01.536 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.533+0000 7fedeeeb5b80 1 -- 172.21.15.52:0/309244352 shutdown_connections 2024-06-10T02:59:01.536 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.533+0000 7fedeeeb5b80 1 -- 172.21.15.52:0/309244352 wait complete. 2024-06-10T02:59:01.536 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.533+0000 7fedeeeb5b80 1 -- start start 2024-06-10T02:59:01.536 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.534+0000 7fedeeeb5b80 1 Processor -- start 2024-06-10T02:59:01.536 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.534+0000 7fedeeeb5b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5598f9f123d0 0x5598f9f1e310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:59:01.536 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.534+0000 7fedeeeb5b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5598f9f17c40 0x5598f9f1b610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:59:01.536 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.534+0000 7fedeeeb5b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5598f9f18560 0x5598f9f1bb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:59:01.536 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.534+0000 7fede23ce700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5598f9f123d0 0x5598f9f1e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:59:01.536 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.534+0000 7fedeeeb5b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5598f9f1ef70 con 0x5598f9f123d0 2024-06-10T02:59:01.536 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.534+0000 7fedeeeb5b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5598f9e03cf0 con 0x5598f9f17c40 2024-06-10T02:59:01.536 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.534+0000 7fede23ce700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5598f9f123d0 0x5598f9f1e310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56660/0 (socket says 172.21.15.52:56660) 2024-06-10T02:59:01.536 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.534+0000 7fede23ce700 1 -- 172.21.15.52:0/3546525780 learned_addr learned my addr 172.21.15.52:0/3546525780 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:59:01.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.534+0000 7fedeeeb5b80 1 -- 172.21.15.52:0/3546525780 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5598f9f1f0f0 con 0x5598f9f18560 2024-06-10T02:59:01.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.534+0000 7fedeeeb5b80 10 client.0 _ll_register_callbacks cb 0x5598f9dfda20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T02:59:01.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.534+0000 7fede1bcd700 1 --2- 172.21.15.52:0/3546525780 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5598f9f17c40 0x5598f9f1b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:59:01.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.534+0000 7fede2bcf700 1 --2- 172.21.15.52:0/3546525780 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5598f9f18560 0x5598f9f1bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:59:01.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.534+0000 7fede23ce700 1 -- 172.21.15.52:0/3546525780 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5598f9f18560 msgr2=0x5598f9f1bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:59:01.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.534+0000 7fede23ce700 1 --2- 172.21.15.52:0/3546525780 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5598f9f18560 0x5598f9f1bb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:01.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.534+0000 7fede23ce700 1 -- 172.21.15.52:0/3546525780 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5598f9f17c40 msgr2=0x5598f9f1b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:59:01.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.534+0000 7fede23ce700 1 --2- 172.21.15.52:0/3546525780 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5598f9f17c40 0x5598f9f1b610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:01.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.534+0000 7fede23ce700 1 -- 172.21.15.52:0/3546525780 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5598f9db29d0 con 0x5598f9f123d0 2024-06-10T02:59:01.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.535+0000 7fede23ce700 1 --2- 172.21.15.52:0/3546525780 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5598f9f123d0 0x5598f9f1e310 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fedd80ccd80 tx=0x7fedd80107d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6312302ef1eb66ce server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:59:01.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.535+0000 7fedc5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-10T02:59:01.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.535+0000 7fedc5ffb700 1 -- 172.21.15.52:0/3546525780 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedd80a4700 con 0x5598f9f123d0 2024-06-10T02:59:01.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.535+0000 7fedc5ffb700 1 -- 172.21.15.52:0/3546525780 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7fedd80109a0 con 0x5598f9f123d0 2024-06-10T02:59:01.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.535+0000 7fedeeeb5b80 1 -- 172.21.15.52:0/3546525780 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fedd809fe40 con 0x5598f9f123d0 2024-06-10T02:59:01.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.535+0000 7fedc5ffb700 1 -- 172.21.15.52:0/3546525780 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fedd80a4270 con 0x5598f9f123d0 2024-06-10T02:59:01.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.535+0000 7fedeeeb5b80 10 client.16185 fetch_fsmap learned FSMap version 34 2024-06-10T02:59:01.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.535+0000 7fedeeeb5b80 1 -- 172.21.15.52:0/3546525780 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5598f9f298f0 con 0x5598f9f123d0 2024-06-10T02:59:01.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.536+0000 7fedc5ffb700 1 -- 172.21.15.52:0/3546525780 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fedd809fe40 con 0x5598f9f123d0 2024-06-10T02:59:01.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.536+0000 7fedc5ffb700 1 -- 172.21.15.52:0/3546525780 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 34) ==== 8378+0+0 (secure 0 0 0) 0x7fedd809a950 con 0x5598f9f123d0 2024-06-10T02:59:01.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.536+0000 7fedeeeb5b80 10 client.16185 fetch_fsmap finished waiting for FSMap version 34 2024-06-10T02:59:01.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.536+0000 7fedeeeb5b80 10 client.16185 resolve_mds: validated gid 24484 aka daemon mds.i 2024-06-10T02:59:01.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.536+0000 7fedeeeb5b80 20 client.16185 populate_metadata read hostname 'smithi052' 2024-06-10T02:59:01.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.536+0000 7fedeeeb5b80 1 --2- 172.21.15.52:0/3546525780 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x5598f9f2bdc0 0x5598f9f2e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:59:01.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.536+0000 7fedeeeb5b80 4 client.16185 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T02:59:01.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.536+0000 7fedeeeb5b80 1 -- 172.21.15.52:0/3546525780 --> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5598f9e70e10 con 0x5598f9f2bdc0 2024-06-10T02:59:01.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.536+0000 7fede1bcd700 1 --2- 172.21.15.52:0/3546525780 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x5598f9f2bdc0 0x5598f9f2e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:59:01.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.537+0000 7fede1bcd700 1 --2- 172.21.15.52:0/3546525780 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x5598f9f2bdc0 0x5598f9f2e1b0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a542675ac03e9f5 server_cookie=b28393a2ee97da6b in_seq=0 out_seq=0 2024-06-10T02:59:01.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.537+0000 7fedc5ffb700 10 client.16185 ms_handle_connect on v2:172.21.15.188:6838/310444836 2024-06-10T02:59:01.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.538+0000 7fedc5ffb700 1 -- 172.21.15.52:0/3546525780 <== mds.0 v2:172.21.15.188:6838/310444836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5598f9e70e10 con 0x5598f9f2bdc0 2024-06-10T02:59:01.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.538+0000 7fedc5ffb700 10 client.16185 handle_command_reply: tid=0 2024-06-10T02:59:01.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.538+0000 7fedeeeb5b80 1 client.16185 shutdown 2024-06-10T02:59:01.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.538+0000 7fedeeeb5b80 1 -- 172.21.15.52:0/3546525780 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5598f9f123d0 msgr2=0x5598f9f1e310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:59:01.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.538+0000 7fedeeeb5b80 1 --2- 172.21.15.52:0/3546525780 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5598f9f123d0 0x5598f9f1e310 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fedd80ccd80 tx=0x7fedd80107d0 comp rx=0 tx=0).stop 2024-06-10T02:59:01.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.538+0000 7fedeeeb5b80 1 -- 172.21.15.52:0/3546525780 shutdown_connections 2024-06-10T02:59:01.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.538+0000 7fedeeeb5b80 1 --2- 172.21.15.52:0/3546525780 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x5598f9f2bdc0 0x5598f9f2e1b0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:01.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.538+0000 7fedc5ffb700 0 client.16185 ms_handle_reset on v2:172.21.15.188:6838/310444836 2024-06-10T02:59:01.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.538+0000 7fedeeeb5b80 1 --2- 172.21.15.52:0/3546525780 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5598f9f18560 0x5598f9f1bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:01.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.538+0000 7fedeeeb5b80 1 --2- 172.21.15.52:0/3546525780 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5598f9f123d0 0x5598f9f1e310 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:01.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.538+0000 7fedeeeb5b80 1 --2- 172.21.15.52:0/3546525780 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5598f9f17c40 0x5598f9f1b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:01.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.538+0000 7fedeeeb5b80 1 -- 172.21.15.52:0/3546525780 >> 172.21.15.52:0/3546525780 conn(0x5598f9e3cdc0 msgr2=0x5598f9f10e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:59:01.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.538+0000 7fedeeeb5b80 1 -- 172.21.15.52:0/3546525780 shutdown_connections 2024-06-10T02:59:01.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.538+0000 7fedeeeb5b80 1 -- 172.21.15.52:0/3546525780 wait complete. 2024-06-10T02:59:01.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.538+0000 7fedeeeb5b80 20 client.16185 trim_cache size 0 max 16384 2024-06-10T02:59:01.560 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.559+0000 7fedeeeb5b80 1 Processor -- start 2024-06-10T02:59:01.561 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.559+0000 7fedeeeb5b80 1 -- start start 2024-06-10T02:59:01.561 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.560+0000 7fedeeeb5b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5598f9f17c40 0x5598f9f1a570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:59:01.561 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.560+0000 7fedeeeb5b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5598f9f18560 0x5598f9f19050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:59:01.561 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.560+0000 7fedeeeb5b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5598f9f2bdc0 0x5598f9f0c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:59:01.561 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.560+0000 7fedeeeb5b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7feddc072850 con 0x5598f9f18560 2024-06-10T02:59:01.561 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.560+0000 7fedeeeb5b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7feddc074450 con 0x5598f9f17c40 2024-06-10T02:59:01.561 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.560+0000 7fedeeeb5b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7feddc072f20 con 0x5598f9f2bdc0 2024-06-10T02:59:01.561 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.560+0000 7fede1bcd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5598f9f18560 0x5598f9f19050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:59:01.561 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.560+0000 7fede1bcd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5598f9f18560 0x5598f9f19050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56668/0 (socket says 172.21.15.52:56668) 2024-06-10T02:59:01.562 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.560+0000 7fede1bcd700 1 -- 172.21.15.52:0/1342282410 learned_addr learned my addr 172.21.15.52:0/1342282410 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:59:01.562 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.560+0000 7fede23ce700 1 --2- 172.21.15.52:0/1342282410 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5598f9f17c40 0x5598f9f1a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:59:01.562 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.560+0000 7fede2bcf700 1 --2- 172.21.15.52:0/1342282410 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5598f9f2bdc0 0x5598f9f0c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:59:01.562 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.560+0000 7fede1bcd700 1 -- 172.21.15.52:0/1342282410 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5598f9f2bdc0 msgr2=0x5598f9f0c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:59:01.562 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.560+0000 7fede1bcd700 1 --2- 172.21.15.52:0/1342282410 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5598f9f2bdc0 0x5598f9f0c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:01.562 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.560+0000 7fede1bcd700 1 -- 172.21.15.52:0/1342282410 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5598f9f17c40 msgr2=0x5598f9f1a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:59:01.562 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.560+0000 7fede1bcd700 1 --2- 172.21.15.52:0/1342282410 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5598f9f17c40 0x5598f9f1a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:01.562 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.560+0000 7fede1bcd700 1 -- 172.21.15.52:0/1342282410 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5598f9f0ddb0 con 0x5598f9f18560 2024-06-10T02:59:01.562 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.560+0000 7fede1bcd700 1 --2- 172.21.15.52:0/1342282410 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5598f9f18560 0x5598f9f19050 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fedcc0117d0 tx=0x7fedcc014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=394ca27ec74eb3c9 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:59:01.562 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.561+0000 7fedc7fff700 1 -- 172.21.15.52:0/1342282410 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedcc003740 con 0x5598f9f18560 2024-06-10T02:59:01.562 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.561+0000 7fedc7fff700 1 -- 172.21.15.52:0/1342282410 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fedcc003dc0 con 0x5598f9f18560 2024-06-10T02:59:01.562 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.561+0000 7fedc7fff700 1 -- 172.21.15.52:0/1342282410 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedcc0293f0 con 0x5598f9f18560 2024-06-10T02:59:01.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.561+0000 7fedeeeb5b80 1 -- 172.21.15.52:0/1342282410 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5598f9f18560 msgr2=0x5598f9f19050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:59:01.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.561+0000 7fedeeeb5b80 1 --2- 172.21.15.52:0/1342282410 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5598f9f18560 0x5598f9f19050 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fedcc0117d0 tx=0x7fedcc014000 comp rx=0 tx=0).stop 2024-06-10T02:59:01.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.561+0000 7fedeeeb5b80 1 -- 172.21.15.52:0/1342282410 shutdown_connections 2024-06-10T02:59:01.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.561+0000 7fedeeeb5b80 1 --2- 172.21.15.52:0/1342282410 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5598f9f2bdc0 0x5598f9f0c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:01.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.561+0000 7fedeeeb5b80 1 --2- 172.21.15.52:0/1342282410 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5598f9f17c40 0x5598f9f1a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:01.566 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.561+0000 7fedeeeb5b80 1 --2- 172.21.15.52:0/1342282410 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5598f9f18560 0x5598f9f19050 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:01.566 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.561+0000 7fedeeeb5b80 1 -- 172.21.15.52:0/1342282410 >> 172.21.15.52:0/1342282410 conn(0x5598f9e3cdc0 msgr2=0x5598f9f18940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:59:01.566 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.561+0000 7fedeeeb5b80 1 -- 172.21.15.52:0/1342282410 shutdown_connections 2024-06-10T02:59:01.566 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.561+0000 7fedeeeb5b80 1 -- 172.21.15.52:0/1342282410 wait complete. 2024-06-10T02:59:01.566 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.562+0000 7fedeeeb5b80 1 -- start start 2024-06-10T02:59:01.566 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.562+0000 7fedeeeb5b80 1 Processor -- start 2024-06-10T02:59:01.566 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.562+0000 7fedeeeb5b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5598f9f17c40 0x5598f9f311e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:59:01.566 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.562+0000 7fedeeeb5b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5598f9f18560 0x5598f9f31720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:59:01.566 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.562+0000 7fedeeeb5b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5598f9f2bdc0 0x5598f9f2f720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:59:01.566 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.562+0000 7fedeeeb5b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7feddc073d80 con 0x5598f9f18560 2024-06-10T02:59:01.566 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.562+0000 7fedeeeb5b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7feddc0736b0 con 0x5598f9f2bdc0 2024-06-10T02:59:01.566 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.562+0000 7fedeeeb5b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7feddc0707f0 con 0x5598f9f17c40 2024-06-10T02:59:01.566 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.562+0000 7fedeeeb5b80 10 client.0 _ll_register_callbacks cb 0x5598f9dfda20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T02:59:01.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.562+0000 7fede1bcd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5598f9f18560 0x5598f9f31720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:59:01.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.562+0000 7fede1bcd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5598f9f18560 0x5598f9f31720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56674/0 (socket says 172.21.15.52:56674) 2024-06-10T02:59:01.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.562+0000 7fede1bcd700 1 -- 172.21.15.52:0/2308494011 learned_addr learned my addr 172.21.15.52:0/2308494011 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:59:01.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.562+0000 7fede23ce700 1 --2- 172.21.15.52:0/2308494011 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5598f9f17c40 0x5598f9f311e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:59:01.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.562+0000 7fede2bcf700 1 --2- 172.21.15.52:0/2308494011 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5598f9f2bdc0 0x5598f9f2f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:59:01.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.562+0000 7fede1bcd700 1 -- 172.21.15.52:0/2308494011 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5598f9f17c40 msgr2=0x5598f9f311e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:59:01.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.562+0000 7fede1bcd700 1 --2- 172.21.15.52:0/2308494011 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5598f9f17c40 0x5598f9f311e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:01.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.562+0000 7fede1bcd700 1 -- 172.21.15.52:0/2308494011 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5598f9f2bdc0 msgr2=0x5598f9f2f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:59:01.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.562+0000 7fede1bcd700 1 --2- 172.21.15.52:0/2308494011 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5598f9f2bdc0 0x5598f9f2f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:01.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.562+0000 7fede1bcd700 1 -- 172.21.15.52:0/2308494011 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5598f9f2fe20 con 0x5598f9f18560 2024-06-10T02:59:01.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.563+0000 7fede2bcf700 1 --2- 172.21.15.52:0/2308494011 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5598f9f2bdc0 0x5598f9f2f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T02:59:01.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.563+0000 7fede1bcd700 1 --2- 172.21.15.52:0/2308494011 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5598f9f18560 0x5598f9f31720 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fedcc011c80 tx=0x7fedcc01dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1cbc90e8ccd3cd8 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:59:01.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.563+0000 7fedc5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-10T02:59:01.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.563+0000 7fedc5ffb700 1 -- 172.21.15.52:0/2308494011 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedcc01de60 con 0x5598f9f18560 2024-06-10T02:59:01.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.563+0000 7fedeeeb5b80 1 -- 172.21.15.52:0/2308494011 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fedcc003dc0 con 0x5598f9f18560 2024-06-10T02:59:01.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.563+0000 7fedc5ffb700 1 -- 172.21.15.52:0/2308494011 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7fedcc01d610 con 0x5598f9f18560 2024-06-10T02:59:01.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.563+0000 7fedc5ffb700 1 -- 172.21.15.52:0/2308494011 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fedcc043050 con 0x5598f9f18560 2024-06-10T02:59:01.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.563+0000 7fedeeeb5b80 10 client.16191 fetch_fsmap learned FSMap version 34 2024-06-10T02:59:01.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.563+0000 7fedeeeb5b80 1 -- 172.21.15.52:0/2308494011 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5598f9f971c0 con 0x5598f9f18560 2024-06-10T02:59:01.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.563+0000 7fedc5ffb700 1 -- 172.21.15.52:0/2308494011 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fedcc003dc0 con 0x5598f9f18560 2024-06-10T02:59:01.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.563+0000 7fedc5ffb700 1 -- 172.21.15.52:0/2308494011 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 34) ==== 8378+0+0 (secure 0 0 0) 0x7fedcc032aa0 con 0x5598f9f18560 2024-06-10T02:59:01.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.563+0000 7fedeeeb5b80 10 client.16191 fetch_fsmap finished waiting for FSMap version 34 2024-06-10T02:59:01.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.563+0000 7fedeeeb5b80 10 client.16191 resolve_mds: validated gid 24484 aka daemon mds.i 2024-06-10T02:59:01.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.564+0000 7fedeeeb5b80 20 client.16191 populate_metadata read hostname 'smithi052' 2024-06-10T02:59:01.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.564+0000 7fedeeeb5b80 1 --2- 172.21.15.52:0/2308494011 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x5598f9f978d0 0x5598f9f99cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:59:01.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.564+0000 7fedeeeb5b80 4 client.16191 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "b06ab081", "release": true, "await": true}] 2024-06-10T02:59:01.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.564+0000 7fedeeeb5b80 1 -- 172.21.15.52:0/2308494011 --> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] -- command(tid 0: {"prefix": "quiesce db", "set_id": "b06ab081", "release": true, "await": true}) -- 0x5598f9e6b390 con 0x5598f9f978d0 2024-06-10T02:59:01.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.564+0000 7fede23ce700 1 --2- 172.21.15.52:0/2308494011 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x5598f9f978d0 0x5598f9f99cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:59:01.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.564+0000 7fedc5ffb700 10 client.16191 ms_handle_connect on v2:172.21.15.188:6838/310444836 2024-06-10T02:59:01.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:01.564+0000 7fede23ce700 1 --2- 172.21.15.52:0/2308494011 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x5598f9f978d0 0x5598f9f99cf0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c6b0d4ebec2071c server_cookie=6c00347652e984c8 in_seq=0 out_seq=0 2024-06-10T02:59:02.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:02.367+0000 7fedc5ffb700 1 -- 172.21.15.52:0/2308494011 <== mds.0 v2:172.21.15.188:6838/310444836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5598f9e6b390 con 0x5598f9f978d0 2024-06-10T02:59:02.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:02.367+0000 7fedc5ffb700 10 client.16191 handle_command_reply: tid=0 2024-06-10T02:59:02.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:02.367+0000 7fedeeeb5b80 1 client.16191 shutdown 2024-06-10T02:59:02.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:02.367+0000 7fedeeeb5b80 1 -- 172.21.15.52:0/2308494011 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5598f9f18560 msgr2=0x5598f9f31720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:59:02.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:02.367+0000 7fedeeeb5b80 1 --2- 172.21.15.52:0/2308494011 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5598f9f18560 0x5598f9f31720 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fedcc011c80 tx=0x7fedcc01dbc0 comp rx=0 tx=0).stop 2024-06-10T02:59:02.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:02.367+0000 7fedeeeb5b80 1 -- 172.21.15.52:0/2308494011 shutdown_connections 2024-06-10T02:59:02.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:02.367+0000 7fedeeeb5b80 1 --2- 172.21.15.52:0/2308494011 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x5598f9f978d0 0x5598f9f99cf0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:02.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:02.367+0000 7fedeeeb5b80 1 --2- 172.21.15.52:0/2308494011 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5598f9f2bdc0 0x5598f9f2f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:02.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:02.367+0000 7fedeeeb5b80 1 --2- 172.21.15.52:0/2308494011 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5598f9f17c40 0x5598f9f311e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:02.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:02.367+0000 7fedeeeb5b80 1 --2- 172.21.15.52:0/2308494011 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5598f9f18560 0x5598f9f31720 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:02.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:02.367+0000 7fedc5ffb700 0 client.16191 ms_handle_reset on v2:172.21.15.188:6838/310444836 2024-06-10T02:59:02.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:02.367+0000 7fedeeeb5b80 1 -- 172.21.15.52:0/2308494011 >> 172.21.15.52:0/2308494011 conn(0x5598f9e3cdc0 msgr2=0x5598f9f11c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:59:02.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:02.367+0000 7fedeeeb5b80 1 -- 172.21.15.52:0/2308494011 shutdown_connections 2024-06-10T02:59:02.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:02.368+0000 7fedeeeb5b80 1 -- 172.21.15.52:0/2308494011 wait complete. 2024-06-10T02:59:02.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:02.368+0000 7fedeeeb5b80 20 client.16191 trim_cache size 0 max 16384 2024-06-10T02:59:02.369 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-10T02:59:02.369 INFO:teuthology.orchestra.run.smithi052.stdout: "epoch": 34, 2024-06-10T02:59:02.369 INFO:teuthology.orchestra.run.smithi052.stdout: "leader": 24484, 2024-06-10T02:59:02.369 INFO:teuthology.orchestra.run.smithi052.stdout: "set_version": 12, 2024-06-10T02:59:02.369 INFO:teuthology.orchestra.run.smithi052.stdout: "sets": { 2024-06-10T02:59:02.369 INFO:teuthology.orchestra.run.smithi052.stdout: "b06ab081": { 2024-06-10T02:59:02.369 INFO:teuthology.orchestra.run.smithi052.stdout: "version": 12, 2024-06-10T02:59:02.369 INFO:teuthology.orchestra.run.smithi052.stdout: "age_ref": 0.0, 2024-06-10T02:59:02.370 INFO:teuthology.orchestra.run.smithi052.stdout: "state": { 2024-06-10T02:59:02.370 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "RELEASED", 2024-06-10T02:59:02.370 INFO:teuthology.orchestra.run.smithi052.stdout: "age": 0.0 2024-06-10T02:59:02.370 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T02:59:02.370 INFO:teuthology.orchestra.run.smithi052.stdout: "timeout": 90.0, 2024-06-10T02:59:02.370 INFO:teuthology.orchestra.run.smithi052.stdout: "expiration": 138.2, 2024-06-10T02:59:02.370 INFO:teuthology.orchestra.run.smithi052.stdout: "members": { 2024-06-10T02:59:02.370 INFO:teuthology.orchestra.run.smithi052.stdout: "file:/": { 2024-06-10T02:59:02.370 INFO:teuthology.orchestra.run.smithi052.stdout: "excluded": false, 2024-06-10T02:59:02.370 INFO:teuthology.orchestra.run.smithi052.stdout: "state": { 2024-06-10T02:59:02.370 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "RELEASED", 2024-06-10T02:59:02.370 INFO:teuthology.orchestra.run.smithi052.stdout: "age": 0.0 2024-06-10T02:59:02.370 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T02:59:02.371 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T02:59:02.371 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T02:59:02.371 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T02:59:02.371 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T02:59:02.371 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-10T02:59:02.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:02.368+0000 7fede4632700 1 -- 172.21.15.52:0/2552181168 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fedc80a2ca0 msgr2=0x7fedc80a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:59:02.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:02.368+0000 7fede4632700 1 --2- 172.21.15.52:0/2552181168 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fedc80a2ca0 0x7fedc80a5160 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7feddc078920 tx=0x7fedcc009260 comp rx=0 tx=0).stop 2024-06-10T02:59:02.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:02.368+0000 7fede4632700 1 -- 172.21.15.52:0/2552181168 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feddc101390 msgr2=0x7feddc19e7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:59:02.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:02.368+0000 7fede4632700 1 --2- 172.21.15.52:0/2552181168 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feddc101390 0x7feddc19e7e0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fedd800e9a0 tx=0x7fedd800ecb0 comp rx=0 tx=0).stop 2024-06-10T02:59:02.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:02.368+0000 7fede4632700 1 -- 172.21.15.52:0/2552181168 shutdown_connections 2024-06-10T02:59:02.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:02.368+0000 7fede4632700 1 --2- 172.21.15.52:0/2552181168 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7feddc107400 0x7feddc1989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:02.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:02.368+0000 7fede4632700 1 --2- 172.21.15.52:0/2552181168 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fedc80a2ca0 0x7fedc80a5160 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:02.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:02.368+0000 7fede4632700 1 --2- 172.21.15.52:0/2552181168 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feddc101390 0x7feddc19e7e0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:02.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:02.368+0000 7fede4632700 1 --2- 172.21.15.52:0/2552181168 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feddc101d50 0x7feddc19ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:02.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:02.368+0000 7fede4632700 1 -- 172.21.15.52:0/2552181168 >> 172.21.15.52:0/2552181168 conn(0x7feddc078340 msgr2=0x7feddc1005d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:59:02.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:02.368+0000 7fede4632700 1 -- 172.21.15.52:0/2552181168 shutdown_connections 2024-06-10T02:59:02.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:02.368+0000 7fede4632700 1 -- 172.21.15.52:0/2552181168 wait complete. 2024-06-10T02:59:02.976 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:02 smithi052 ceph-mon[32362]: pgmap v579: 129 pgs: 129 active+clean; 477 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 1.4 MiB/s wr, 222 op/s 2024-06-10T02:59:03.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:02 smithi120 ceph-mon[36634]: pgmap v579: 129 pgs: 129 active+clean; 477 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 1.4 MiB/s wr, 222 op/s 2024-06-10T02:59:03.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:02 smithi188 ceph-mon[36599]: pgmap v579: 129 pgs: 129 active+clean; 477 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 1.4 MiB/s wr, 222 op/s 2024-06-10T02:59:03.415 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'b06ab081', total seconds elapsed: 40.8 2024-06-10T02:59:03.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.808+0000 7f1d1355b700 1 -- 172.21.15.52:0/87414295 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1d0c101d40 msgr2=0x7f1d0c10e230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:59:03.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.808+0000 7f1d1355b700 1 --2- 172.21.15.52:0/87414295 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1d0c101d40 0x7f1d0c10e230 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f1d0400b0a0 tx=0x7f1d0400b3b0 comp rx=0 tx=0).stop 2024-06-10T02:59:03.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.808+0000 7f1d1355b700 1 -- 172.21.15.52:0/87414295 shutdown_connections 2024-06-10T02:59:03.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.808+0000 7f1d1355b700 1 --2- 172.21.15.52:0/87414295 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1d0c101d40 0x7f1d0c10e230 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:03.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.808+0000 7f1d1355b700 1 --2- 172.21.15.52:0/87414295 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1d0c1073f0 0x7f1d0c1077d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:03.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.808+0000 7f1d1355b700 1 --2- 172.21.15.52:0/87414295 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1d0c101380 0x7f1d0c101800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:03.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.808+0000 7f1d1355b700 1 -- 172.21.15.52:0/87414295 >> 172.21.15.52:0/87414295 conn(0x7f1d0c078300 msgr2=0x7f1d0c078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:59:03.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.808+0000 7f1d1355b700 1 -- 172.21.15.52:0/87414295 shutdown_connections 2024-06-10T02:59:03.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.808+0000 7f1d1355b700 1 -- 172.21.15.52:0/87414295 wait complete. 2024-06-10T02:59:03.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.809+0000 7f1d1355b700 1 Processor -- start 2024-06-10T02:59:03.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.809+0000 7f1d1355b700 1 -- start start 2024-06-10T02:59:03.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.809+0000 7f1d1355b700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1d0c101380 0x7f1d0c19e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:59:03.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.809+0000 7f1d1355b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1d0c101d40 0x7f1d0c19ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:59:03.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.809+0000 7f1d1355b700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1d0c1073f0 0x7f1d0c198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:59:03.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.809+0000 7f1d1355b700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f1d0c112790 con 0x7f1d0c101d40 2024-06-10T02:59:03.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.809+0000 7f1d1355b700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1d0c112610 con 0x7f1d0c1073f0 2024-06-10T02:59:03.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.809+0000 7f1d1355b700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f1d0c112910 con 0x7f1d0c101380 2024-06-10T02:59:03.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.809+0000 7f1d10af6700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1d0c101d40 0x7f1d0c19ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:59:03.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.809+0000 7f1d112f7700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1d0c101380 0x7f1d0c19e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:59:03.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.809+0000 7f1d10af6700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1d0c101d40 0x7f1d0c19ee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56704/0 (socket says 172.21.15.52:56704) 2024-06-10T02:59:03.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.809+0000 7f1d10af6700 1 -- 172.21.15.52:0/3570357480 learned_addr learned my addr 172.21.15.52:0/3570357480 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:59:03.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.809+0000 7f1d11af8700 1 --2- 172.21.15.52:0/3570357480 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1d0c1073f0 0x7f1d0c198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:59:03.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.810+0000 7f1d112f7700 1 -- 172.21.15.52:0/3570357480 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1d0c1073f0 msgr2=0x7f1d0c198af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:59:03.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.810+0000 7f1d112f7700 1 --2- 172.21.15.52:0/3570357480 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1d0c1073f0 0x7f1d0c198af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:03.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.810+0000 7f1d112f7700 1 -- 172.21.15.52:0/3570357480 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1d0c101d40 msgr2=0x7f1d0c19ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:59:03.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.810+0000 7f1d112f7700 1 --2- 172.21.15.52:0/3570357480 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1d0c101d40 0x7f1d0c19ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:03.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.810+0000 7f1d112f7700 1 -- 172.21.15.52:0/3570357480 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d0c199250 con 0x7f1d0c101380 2024-06-10T02:59:03.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.810+0000 7f1d11af8700 1 --2- 172.21.15.52:0/3570357480 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1d0c1073f0 0x7f1d0c198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T02:59:03.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.810+0000 7f1d112f7700 1 --2- 172.21.15.52:0/3570357480 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1d0c101380 0x7f1d0c19e8e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f1d0800e9f0 tx=0x7f1d0800ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:59:03.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.810+0000 7f1d027fc700 1 -- 172.21.15.52:0/3570357480 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d0800c9c0 con 0x7f1d0c101380 2024-06-10T02:59:03.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.810+0000 7f1d1355b700 1 -- 172.21.15.52:0/3570357480 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d0c199540 con 0x7f1d0c101380 2024-06-10T02:59:03.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.810+0000 7f1d027fc700 1 -- 172.21.15.52:0/3570357480 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d0800cb60 con 0x7f1d0c101380 2024-06-10T02:59:03.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.810+0000 7f1d027fc700 1 -- 172.21.15.52:0/3570357480 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d08018680 con 0x7f1d0c101380 2024-06-10T02:59:03.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.810+0000 7f1d1355b700 1 -- 172.21.15.52:0/3570357480 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d0c1a57b0 con 0x7f1d0c101380 2024-06-10T02:59:03.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.813+0000 7f1d027fc700 1 -- 172.21.15.52:0/3570357480 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f1d08010430 con 0x7f1d0c101380 2024-06-10T02:59:03.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.814+0000 7f1d027fc700 1 --2- 172.21.15.52:0/3570357480 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f1cf80a2c20 0x7f1cf80a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:59:03.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.814+0000 7f1d10af6700 1 --2- 172.21.15.52:0/3570357480 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f1cf80a2c20 0x7f1cf80a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:59:03.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.814+0000 7f1d027fc700 1 -- 172.21.15.52:0/3570357480 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f1d080cf4e0 con 0x7f1d0c101380 2024-06-10T02:59:03.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.814+0000 7f1d1dddeb80 1 Processor -- start 2024-06-10T02:59:03.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.814+0000 7f1d1dddeb80 1 -- start start 2024-06-10T02:59:03.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.814+0000 7f1d10af6700 1 --2- 172.21.15.52:0/3570357480 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f1cf80a2c20 0x7f1cf80a50e0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f1d0c078920 tx=0x7f1cfc009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:59:03.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.814+0000 7f1d1dddeb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557bc5a3ea50 0x557bc5a3ee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:59:03.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.814+0000 7f1d1dddeb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557bc5a3f370 0x557bc5a38df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:59:03.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.814+0000 7f1d1dddeb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557bc5a39330 0x557bc5a39750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:59:03.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.814+0000 7f1d1dddeb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x557bc5986810 con 0x557bc5a3ea50 2024-06-10T02:59:03.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.814+0000 7f1d1dddeb80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x557bc592e720 con 0x557bc5a39330 2024-06-10T02:59:03.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.814+0000 7f1d1dddeb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x557bc59827d0 con 0x557bc5a3f370 2024-06-10T02:59:03.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.815+0000 7f1d112f7700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557bc5a3ea50 0x557bc5a3ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:59:03.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.815+0000 7f1d112f7700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557bc5a3ea50 0x557bc5a3ee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56720/0 (socket says 172.21.15.52:56720) 2024-06-10T02:59:03.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.815+0000 7f1d112f7700 1 -- 172.21.15.52:0/1230179707 learned_addr learned my addr 172.21.15.52:0/1230179707 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:59:03.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.815+0000 7f1d10af6700 1 --2- 172.21.15.52:0/1230179707 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557bc5a3f370 0x557bc5a38df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:59:03.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.815+0000 7f1d11af8700 1 --2- 172.21.15.52:0/1230179707 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557bc5a39330 0x557bc5a39750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:59:03.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.815+0000 7f1d112f7700 1 -- 172.21.15.52:0/1230179707 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557bc5a3f370 msgr2=0x557bc5a38df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:59:03.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.815+0000 7f1d112f7700 1 --2- 172.21.15.52:0/1230179707 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557bc5a3f370 0x557bc5a38df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:03.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.815+0000 7f1d112f7700 1 -- 172.21.15.52:0/1230179707 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557bc5a39330 msgr2=0x557bc5a39750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:59:03.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.815+0000 7f1d112f7700 1 --2- 172.21.15.52:0/1230179707 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557bc5a39330 0x557bc5a39750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:03.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.815+0000 7f1d112f7700 1 -- 172.21.15.52:0/1230179707 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557bc5924f50 con 0x557bc5a3ea50 2024-06-10T02:59:03.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.815+0000 7f1d11af8700 1 --2- 172.21.15.52:0/1230179707 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557bc5a39330 0x557bc5a39750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T02:59:03.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.815+0000 7f1d112f7700 1 --2- 172.21.15.52:0/1230179707 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557bc5a3ea50 0x557bc5a3ee30 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f1d08097d90 tx=0x7f1d08097d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8c08c2bcf6687f4 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:59:03.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.815+0000 7f1cf77fe700 1 -- 172.21.15.52:0/1230179707 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d0809b930 con 0x557bc5a3ea50 2024-06-10T02:59:03.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.815+0000 7f1cf77fe700 1 -- 172.21.15.52:0/1230179707 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d0809fce0 con 0x557bc5a3ea50 2024-06-10T02:59:03.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.815+0000 7f1cf77fe700 1 -- 172.21.15.52:0/1230179707 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d080860e0 con 0x557bc5a3ea50 2024-06-10T02:59:03.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.816+0000 7f1d1dddeb80 1 -- 172.21.15.52:0/1230179707 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557bc5a3ea50 msgr2=0x557bc5a3ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:59:03.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.816+0000 7f1d1dddeb80 1 --2- 172.21.15.52:0/1230179707 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557bc5a3ea50 0x557bc5a3ee30 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f1d08097d90 tx=0x7f1d08097d00 comp rx=0 tx=0).stop 2024-06-10T02:59:03.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.816+0000 7f1d1dddeb80 1 -- 172.21.15.52:0/1230179707 shutdown_connections 2024-06-10T02:59:03.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.816+0000 7f1d1dddeb80 1 --2- 172.21.15.52:0/1230179707 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557bc5a39330 0x557bc5a39750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:03.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.816+0000 7f1d1dddeb80 1 --2- 172.21.15.52:0/1230179707 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557bc5a3ea50 0x557bc5a3ee30 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:03.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.816+0000 7f1d1dddeb80 1 --2- 172.21.15.52:0/1230179707 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557bc5a3f370 0x557bc5a38df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:03.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.816+0000 7f1d1dddeb80 1 -- 172.21.15.52:0/1230179707 >> 172.21.15.52:0/1230179707 conn(0x557bc595fcc0 msgr2=0x557bc5a2f850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:59:03.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.816+0000 7f1d1dddeb80 1 -- 172.21.15.52:0/1230179707 shutdown_connections 2024-06-10T02:59:03.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.816+0000 7f1d1dddeb80 1 -- 172.21.15.52:0/1230179707 wait complete. 2024-06-10T02:59:03.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.816+0000 7f1d1dddeb80 1 -- start start 2024-06-10T02:59:03.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.817+0000 7f1d1dddeb80 1 Processor -- start 2024-06-10T02:59:03.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.817+0000 7f1d1dddeb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557bc5a39330 0x557bc5a45140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:59:03.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.817+0000 7f1d1dddeb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557bc5a3ea50 0x557bc5a42440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:59:03.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.817+0000 7f1d1dddeb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557bc5a3f370 0x557bc5a42980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:59:03.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.817+0000 7f1d1dddeb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x557bc5a45d80 con 0x557bc5a39330 2024-06-10T02:59:03.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.817+0000 7f1d1dddeb80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x557bc5926bf0 con 0x557bc5a3ea50 2024-06-10T02:59:03.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.817+0000 7f1d1dddeb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x557bc5a45f00 con 0x557bc5a3f370 2024-06-10T02:59:03.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.817+0000 7f1d1dddeb80 10 client.0 _ll_register_callbacks cb 0x557bc5920920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T02:59:03.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.817+0000 7f1d112f7700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557bc5a39330 0x557bc5a45140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:59:03.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.817+0000 7f1d112f7700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557bc5a39330 0x557bc5a45140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56728/0 (socket says 172.21.15.52:56728) 2024-06-10T02:59:03.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.817+0000 7f1d112f7700 1 -- 172.21.15.52:0/236054097 learned_addr learned my addr 172.21.15.52:0/236054097 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:59:03.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.817+0000 7f1d11af8700 1 --2- 172.21.15.52:0/236054097 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557bc5a3f370 0x557bc5a42980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:59:03.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.817+0000 7f1d10af6700 1 --2- 172.21.15.52:0/236054097 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557bc5a3ea50 0x557bc5a42440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:59:03.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.818+0000 7f1d112f7700 1 -- 172.21.15.52:0/236054097 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557bc5a3f370 msgr2=0x557bc5a42980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:59:03.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.818+0000 7f1d112f7700 1 --2- 172.21.15.52:0/236054097 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557bc5a3f370 0x557bc5a42980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:03.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.818+0000 7f1d112f7700 1 -- 172.21.15.52:0/236054097 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557bc5a3ea50 msgr2=0x557bc5a42440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:59:03.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.818+0000 7f1d112f7700 1 --2- 172.21.15.52:0/236054097 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557bc5a3ea50 0x557bc5a42440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:03.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.818+0000 7f1d112f7700 1 -- 172.21.15.52:0/236054097 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557bc58d58d0 con 0x557bc5a39330 2024-06-10T02:59:03.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.818+0000 7f1d10af6700 1 --2- 172.21.15.52:0/236054097 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557bc5a3ea50 0x557bc5a42440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T02:59:03.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.818+0000 7f1d112f7700 1 --2- 172.21.15.52:0/236054097 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557bc5a39330 0x557bc5a45140 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f1d08009230 tx=0x7f1d08009ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5ebd5c62493db36 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:59:03.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.818+0000 7f1cf4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-10T02:59:03.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.818+0000 7f1cf4ff9700 1 -- 172.21.15.52:0/236054097 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d080b3070 con 0x557bc5a39330 2024-06-10T02:59:03.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.818+0000 7f1cf4ff9700 1 -- 172.21.15.52:0/236054097 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f1d08010720 con 0x557bc5a39330 2024-06-10T02:59:03.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.818+0000 7f1d1dddeb80 1 -- 172.21.15.52:0/236054097 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1d0809fce0 con 0x557bc5a39330 2024-06-10T02:59:03.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.818+0000 7f1cf4ff9700 1 -- 172.21.15.52:0/236054097 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f1d080ba2b0 con 0x557bc5a39330 2024-06-10T02:59:03.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.818+0000 7f1d1dddeb80 10 client.16203 fetch_fsmap learned FSMap version 34 2024-06-10T02:59:03.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.818+0000 7f1d1dddeb80 1 -- 172.21.15.52:0/236054097 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x557bc5a50650 con 0x557bc5a39330 2024-06-10T02:59:03.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.818+0000 7f1cf4ff9700 1 -- 172.21.15.52:0/236054097 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d0809fce0 con 0x557bc5a39330 2024-06-10T02:59:03.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.819+0000 7f1cf4ff9700 1 -- 172.21.15.52:0/236054097 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 34) ==== 8378+0+0 (secure 0 0 0) 0x7f1d08065020 con 0x557bc5a39330 2024-06-10T02:59:03.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.819+0000 7f1d1dddeb80 10 client.16203 fetch_fsmap finished waiting for FSMap version 34 2024-06-10T02:59:03.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.819+0000 7f1d1dddeb80 10 client.16203 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-06-10T02:59:03.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.819+0000 7f1d1dddeb80 20 client.16203 populate_metadata read hostname 'smithi052' 2024-06-10T02:59:03.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.819+0000 7f1d1dddeb80 1 --2- 172.21.15.52:0/236054097 >> [v2:172.21.15.188:6836/819317726,v1:172.21.15.188:6837/819317726] conn(0x557bc5a52b20 0x557bc5a54f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:59:03.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.819+0000 7f1d1dddeb80 4 client.16203 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T02:59:03.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.819+0000 7f1d1dddeb80 1 -- 172.21.15.52:0/236054097 --> [v2:172.21.15.188:6836/819317726,v1:172.21.15.188:6837/819317726] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557bc5993d10 con 0x557bc5a52b20 2024-06-10T02:59:03.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.819+0000 7f1d10af6700 1 --2- 172.21.15.52:0/236054097 >> [v2:172.21.15.188:6836/819317726,v1:172.21.15.188:6837/819317726] conn(0x557bc5a52b20 0x557bc5a54f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:59:03.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.819+0000 7f1cf4ff9700 10 client.16203 ms_handle_connect on v2:172.21.15.188:6836/819317726 2024-06-10T02:59:03.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.819+0000 7f1d10af6700 1 --2- 172.21.15.52:0/236054097 >> [v2:172.21.15.188:6836/819317726,v1:172.21.15.188:6837/819317726] conn(0x557bc5a52b20 0x557bc5a54f10 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6d70762daefd2a27 server_cookie=4bd3dd605587b3c6 in_seq=0 out_seq=0 2024-06-10T02:59:03.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.820+0000 7f1cf4ff9700 1 -- 172.21.15.52:0/236054097 <== mds.1 v2:172.21.15.188:6836/819317726 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x557bc5993d10 con 0x557bc5a52b20 2024-06-10T02:59:03.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.820+0000 7f1cf4ff9700 10 client.16203 handle_command_reply: tid=0 2024-06-10T02:59:03.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.820+0000 7f1d1dddeb80 1 client.16203 shutdown 2024-06-10T02:59:03.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.820+0000 7f1d1dddeb80 1 -- 172.21.15.52:0/236054097 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557bc5a39330 msgr2=0x557bc5a45140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:59:03.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.820+0000 7f1d1dddeb80 1 --2- 172.21.15.52:0/236054097 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557bc5a39330 0x557bc5a45140 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f1d08009230 tx=0x7f1d08009ef0 comp rx=0 tx=0).stop 2024-06-10T02:59:03.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.820+0000 7f1d1dddeb80 1 -- 172.21.15.52:0/236054097 shutdown_connections 2024-06-10T02:59:03.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.820+0000 7f1d1dddeb80 1 --2- 172.21.15.52:0/236054097 >> [v2:172.21.15.188:6836/819317726,v1:172.21.15.188:6837/819317726] conn(0x557bc5a52b20 0x557bc5a54f10 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:03.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.820+0000 7f1d1dddeb80 1 --2- 172.21.15.52:0/236054097 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557bc5a3f370 0x557bc5a42980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:03.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.820+0000 7f1d1dddeb80 1 --2- 172.21.15.52:0/236054097 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557bc5a39330 0x557bc5a45140 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:03.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.820+0000 7f1d1dddeb80 1 --2- 172.21.15.52:0/236054097 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557bc5a3ea50 0x557bc5a42440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:03.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.820+0000 7f1d1dddeb80 1 -- 172.21.15.52:0/236054097 >> 172.21.15.52:0/236054097 conn(0x557bc595fcc0 msgr2=0x557bc5a37de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:59:03.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.821+0000 7f1cf4ff9700 0 client.16203 ms_handle_reset on v2:172.21.15.188:6836/819317726 2024-06-10T02:59:03.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.821+0000 7f1d1dddeb80 1 -- 172.21.15.52:0/236054097 shutdown_connections 2024-06-10T02:59:03.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.821+0000 7f1d1dddeb80 1 -- 172.21.15.52:0/236054097 wait complete. 2024-06-10T02:59:03.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.821+0000 7f1d1dddeb80 20 client.16203 trim_cache size 0 max 16384 2024-06-10T02:59:03.838 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.838+0000 7f1d1dddeb80 1 Processor -- start 2024-06-10T02:59:03.838 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.838+0000 7f1d1dddeb80 1 -- start start 2024-06-10T02:59:03.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.839+0000 7f1d1dddeb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557bc5a39330 0x557bc5a38df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:59:03.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.839+0000 7f1d1dddeb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557bc5a3ea50 0x557bc5a2f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:59:03.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.839+0000 7f1d1dddeb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557bc5a3f370 0x557bc5a3fe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:59:03.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.839+0000 7f1d1dddeb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f1d0c1ac200 con 0x557bc5a3ea50 2024-06-10T02:59:03.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.839+0000 7f1d1dddeb80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1d0c1ade00 con 0x557bc5a3f370 2024-06-10T02:59:03.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.839+0000 7f1d1dddeb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f1d0c1ac8d0 con 0x557bc5a39330 2024-06-10T02:59:03.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.839+0000 7f1d10af6700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557bc5a3ea50 0x557bc5a2f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:59:03.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.839+0000 7f1d112f7700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557bc5a39330 0x557bc5a38df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:59:03.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.839+0000 7f1d10af6700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557bc5a3ea50 0x557bc5a2f9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56738/0 (socket says 172.21.15.52:56738) 2024-06-10T02:59:03.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.839+0000 7f1d112f7700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557bc5a39330 0x557bc5a38df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.52:41722/0 (socket says 172.21.15.52:41722) 2024-06-10T02:59:03.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.839+0000 7f1d10af6700 1 -- 172.21.15.52:0/3888753264 learned_addr learned my addr 172.21.15.52:0/3888753264 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:59:03.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.839+0000 7f1d11af8700 1 --2- 172.21.15.52:0/3888753264 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557bc5a3f370 0x557bc5a3fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:59:03.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.839+0000 7f1d10af6700 1 -- 172.21.15.52:0/3888753264 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557bc5a39330 msgr2=0x557bc5a38df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:59:03.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.839+0000 7f1d10af6700 1 --2- 172.21.15.52:0/3888753264 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557bc5a39330 0x557bc5a38df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:03.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.839+0000 7f1d10af6700 1 -- 172.21.15.52:0/3888753264 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557bc5a3f370 msgr2=0x557bc5a3fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:59:03.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.839+0000 7f1d10af6700 1 --2- 172.21.15.52:0/3888753264 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557bc5a3f370 0x557bc5a3fe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:03.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.839+0000 7f1d10af6700 1 -- 172.21.15.52:0/3888753264 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557bc5a3ee30 con 0x557bc5a3ea50 2024-06-10T02:59:03.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.839+0000 7f1d11af8700 1 --2- 172.21.15.52:0/3888753264 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557bc5a3f370 0x557bc5a3fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T02:59:03.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.840+0000 7f1d10af6700 1 --2- 172.21.15.52:0/3888753264 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557bc5a3ea50 0x557bc5a2f9f0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f1cfc013010 tx=0x7f1cfc014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6197ababd2b20e6c server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:59:03.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.840+0000 7f1cf6ffd700 1 -- 172.21.15.52:0/3888753264 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cfc036070 con 0x557bc5a3ea50 2024-06-10T02:59:03.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.840+0000 7f1cf6ffd700 1 -- 172.21.15.52:0/3888753264 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1cfc0318d0 con 0x557bc5a3ea50 2024-06-10T02:59:03.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.840+0000 7f1cf6ffd700 1 -- 172.21.15.52:0/3888753264 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cfc0299c0 con 0x557bc5a3ea50 2024-06-10T02:59:03.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.841+0000 7f1d1dddeb80 1 -- 172.21.15.52:0/3888753264 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557bc5a3ea50 msgr2=0x557bc5a2f9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:59:03.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.841+0000 7f1d1dddeb80 1 --2- 172.21.15.52:0/3888753264 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557bc5a3ea50 0x557bc5a2f9f0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f1cfc013010 tx=0x7f1cfc014000 comp rx=0 tx=0).stop 2024-06-10T02:59:03.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.841+0000 7f1d1dddeb80 1 -- 172.21.15.52:0/3888753264 shutdown_connections 2024-06-10T02:59:03.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.841+0000 7f1d1dddeb80 1 --2- 172.21.15.52:0/3888753264 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557bc5a3f370 0x557bc5a3fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:03.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.841+0000 7f1d1dddeb80 1 --2- 172.21.15.52:0/3888753264 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557bc5a39330 0x557bc5a38df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:03.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.841+0000 7f1d1dddeb80 1 --2- 172.21.15.52:0/3888753264 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557bc5a3ea50 0x557bc5a2f9f0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:03.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.841+0000 7f1d1dddeb80 1 -- 172.21.15.52:0/3888753264 >> 172.21.15.52:0/3888753264 conn(0x557bc595fcc0 msgr2=0x557bc5a3f750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:59:03.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.841+0000 7f1d1dddeb80 1 -- 172.21.15.52:0/3888753264 shutdown_connections 2024-06-10T02:59:03.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.841+0000 7f1d1dddeb80 1 -- 172.21.15.52:0/3888753264 wait complete. 2024-06-10T02:59:03.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.841+0000 7f1d1dddeb80 1 -- start start 2024-06-10T02:59:03.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.841+0000 7f1d1dddeb80 1 Processor -- start 2024-06-10T02:59:03.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.842+0000 7f1d1dddeb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557bc5a39330 0x557bc5a55110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:59:03.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.842+0000 7f1d1dddeb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557bc5a3ea50 0x557bc5a55650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:59:03.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.842+0000 7f1d1dddeb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557bc5a3f370 0x557bc5abb450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:59:03.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.842+0000 7f1d1dddeb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f1d0c1ad730 con 0x557bc5a3ea50 2024-06-10T02:59:03.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.842+0000 7f1d1dddeb80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1d0c1ad060 con 0x557bc5a39330 2024-06-10T02:59:03.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.842+0000 7f1d1dddeb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f1d0c1aa1a0 con 0x557bc5a3f370 2024-06-10T02:59:03.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.842+0000 7f1d1dddeb80 10 client.0 _ll_register_callbacks cb 0x557bc5920920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T02:59:03.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.842+0000 7f1d10af6700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557bc5a3ea50 0x557bc5a55650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:59:03.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.842+0000 7f1d11af8700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557bc5a3f370 0x557bc5abb450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:59:03.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.842+0000 7f1d112f7700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557bc5a39330 0x557bc5a55110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:59:03.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.842+0000 7f1d11af8700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557bc5a3f370 0x557bc5abb450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.52:41724/0 (socket says 172.21.15.52:41724) 2024-06-10T02:59:03.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.842+0000 7f1d10af6700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557bc5a3ea50 0x557bc5a55650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56752/0 (socket says 172.21.15.52:56752) 2024-06-10T02:59:03.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.842+0000 7f1d11af8700 1 -- 172.21.15.52:0/379456227 learned_addr learned my addr 172.21.15.52:0/379456227 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:59:03.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.842+0000 7f1d10af6700 1 -- 172.21.15.52:0/379456227 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557bc5a3f370 msgr2=0x557bc5abb450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:59:03.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.842+0000 7f1d10af6700 1 --2- 172.21.15.52:0/379456227 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557bc5a3f370 0x557bc5abb450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:03.846 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.842+0000 7f1d10af6700 1 -- 172.21.15.52:0/379456227 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557bc5a39330 msgr2=0x557bc5a55110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:59:03.846 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.842+0000 7f1d10af6700 1 --2- 172.21.15.52:0/379456227 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557bc5a39330 0x557bc5a55110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:03.846 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.842+0000 7f1d10af6700 1 -- 172.21.15.52:0/379456227 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557bc5abb990 con 0x557bc5a3ea50 2024-06-10T02:59:03.846 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.843+0000 7f1d10af6700 1 --2- 172.21.15.52:0/379456227 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557bc5a3ea50 0x557bc5a55650 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x557bc5a2c090 tx=0x7f1cfc01d4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a22b368afd6a1c1 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:59:03.846 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.843+0000 7f1cf4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-10T02:59:03.846 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.843+0000 7f1cf4ff9700 1 -- 172.21.15.52:0/379456227 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cfc041060 con 0x557bc5a3ea50 2024-06-10T02:59:03.846 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.843+0000 7f1d1dddeb80 1 -- 172.21.15.52:0/379456227 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1cfc0318d0 con 0x557bc5a3ea50 2024-06-10T02:59:03.846 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.843+0000 7f1cf4ff9700 1 -- 172.21.15.52:0/379456227 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f1cfc031a80 con 0x557bc5a3ea50 2024-06-10T02:59:03.846 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.843+0000 7f1cf4ff9700 1 -- 172.21.15.52:0/379456227 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f1cfc048050 con 0x557bc5a3ea50 2024-06-10T02:59:03.846 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.843+0000 7f1d1dddeb80 10 client.16209 fetch_fsmap learned FSMap version 34 2024-06-10T02:59:03.846 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.843+0000 7f1d1dddeb80 1 -- 172.21.15.52:0/379456227 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x557bc5abcc30 con 0x557bc5a3ea50 2024-06-10T02:59:03.846 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.843+0000 7f1cf4ff9700 1 -- 172.21.15.52:0/379456227 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1cfc0318d0 con 0x557bc5a3ea50 2024-06-10T02:59:03.846 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.843+0000 7f1cf4ff9700 1 -- 172.21.15.52:0/379456227 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 34) ==== 8378+0+0 (secure 0 0 0) 0x7f1cfc004750 con 0x557bc5a3ea50 2024-06-10T02:59:03.847 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.843+0000 7f1d1dddeb80 10 client.16209 fetch_fsmap finished waiting for FSMap version 34 2024-06-10T02:59:03.847 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.843+0000 7f1d1dddeb80 10 client.16209 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-06-10T02:59:03.847 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.843+0000 7f1d1dddeb80 20 client.16209 populate_metadata read hostname 'smithi052' 2024-06-10T02:59:03.847 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.843+0000 7f1d1dddeb80 1 --2- 172.21.15.52:0/379456227 >> [v2:172.21.15.188:6836/819317726,v1:172.21.15.188:6837/819317726] conn(0x557bc5abd290 0x557bc5abf6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:59:03.847 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.843+0000 7f1d1dddeb80 4 client.16209 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T02:59:03.847 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.844+0000 7f1d1dddeb80 1 -- 172.21.15.52:0/379456227 --> [v2:172.21.15.188:6836/819317726,v1:172.21.15.188:6837/819317726] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557bc598e290 con 0x557bc5abd290 2024-06-10T02:59:03.847 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.844+0000 7f1d112f7700 1 --2- 172.21.15.52:0/379456227 >> [v2:172.21.15.188:6836/819317726,v1:172.21.15.188:6837/819317726] conn(0x557bc5abd290 0x557bc5abf6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:59:03.847 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.844+0000 7f1d112f7700 1 --2- 172.21.15.52:0/379456227 >> [v2:172.21.15.188:6836/819317726,v1:172.21.15.188:6837/819317726] conn(0x557bc5abd290 0x557bc5abf6b0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a253170070cc59b1 server_cookie=8a0a0bd00af53343 in_seq=0 out_seq=0 2024-06-10T02:59:03.847 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.844+0000 7f1cf4ff9700 10 client.16209 ms_handle_connect on v2:172.21.15.188:6836/819317726 2024-06-10T02:59:03.848 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.844+0000 7f1cf4ff9700 1 -- 172.21.15.52:0/379456227 <== mds.1 v2:172.21.15.188:6836/819317726 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557bc598e290 con 0x557bc5abd290 2024-06-10T02:59:03.848 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.844+0000 7f1cf4ff9700 10 client.16209 handle_command_reply: tid=0 2024-06-10T02:59:03.848 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.844+0000 7f1d1dddeb80 1 client.16209 shutdown 2024-06-10T02:59:03.848 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.844+0000 7f1d1dddeb80 1 -- 172.21.15.52:0/379456227 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557bc5a3ea50 msgr2=0x557bc5a55650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:59:03.849 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.844+0000 7f1d1dddeb80 1 --2- 172.21.15.52:0/379456227 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557bc5a3ea50 0x557bc5a55650 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x557bc5a2c090 tx=0x7f1cfc01d4f0 comp rx=0 tx=0).stop 2024-06-10T02:59:03.849 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.844+0000 7f1d1dddeb80 1 -- 172.21.15.52:0/379456227 shutdown_connections 2024-06-10T02:59:03.849 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.844+0000 7f1d1dddeb80 1 --2- 172.21.15.52:0/379456227 >> [v2:172.21.15.188:6836/819317726,v1:172.21.15.188:6837/819317726] conn(0x557bc5abd290 0x557bc5abf6b0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:03.849 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.845+0000 7f1d1dddeb80 1 --2- 172.21.15.52:0/379456227 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557bc5a3f370 0x557bc5abb450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:03.849 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.845+0000 7f1cf4ff9700 0 client.16209 ms_handle_reset on v2:172.21.15.188:6836/819317726 2024-06-10T02:59:03.849 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.845+0000 7f1d1dddeb80 1 --2- 172.21.15.52:0/379456227 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557bc5a39330 0x557bc5a55110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:03.849 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.845+0000 7f1d1dddeb80 1 --2- 172.21.15.52:0/379456227 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557bc5a3ea50 0x557bc5a55650 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:03.849 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.845+0000 7f1d1dddeb80 1 -- 172.21.15.52:0/379456227 >> 172.21.15.52:0/379456227 conn(0x557bc595fcc0 msgr2=0x557bc5a3ffe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:59:03.849 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.845+0000 7f1d1dddeb80 1 -- 172.21.15.52:0/379456227 shutdown_connections 2024-06-10T02:59:03.849 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.845+0000 7f1d1dddeb80 1 -- 172.21.15.52:0/379456227 wait complete. 2024-06-10T02:59:03.849 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.845+0000 7f1d1dddeb80 20 client.16209 trim_cache size 0 max 16384 2024-06-10T02:59:03.850 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-06-10T02:59:03.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.845+0000 7f1d1355b700 1 -- 172.21.15.52:0/3570357480 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f1cf80a2c20 msgr2=0x7f1cf80a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:59:03.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.845+0000 7f1d1355b700 1 --2- 172.21.15.52:0/3570357480 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f1cf80a2c20 0x7f1cf80a50e0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f1d0c078920 tx=0x7f1cfc009260 comp rx=0 tx=0).stop 2024-06-10T02:59:03.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.845+0000 7f1d1355b700 1 -- 172.21.15.52:0/3570357480 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1d0c101380 msgr2=0x7f1d0c19e8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:59:03.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.845+0000 7f1d1355b700 1 --2- 172.21.15.52:0/3570357480 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1d0c101380 0x7f1d0c19e8e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f1d0800e9f0 tx=0x7f1d0800ed00 comp rx=0 tx=0).stop 2024-06-10T02:59:03.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.845+0000 7f1d1355b700 1 -- 172.21.15.52:0/3570357480 shutdown_connections 2024-06-10T02:59:03.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.845+0000 7f1d1355b700 1 --2- 172.21.15.52:0/3570357480 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1d0c1073f0 0x7f1d0c198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:03.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.845+0000 7f1d1355b700 1 --2- 172.21.15.52:0/3570357480 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1d0c101380 0x7f1d0c19e8e0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:03.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.845+0000 7f1d1355b700 1 --2- 172.21.15.52:0/3570357480 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f1cf80a2c20 0x7f1cf80a50e0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:03.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.845+0000 7f1d1355b700 1 --2- 172.21.15.52:0/3570357480 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1d0c101d40 0x7f1d0c19ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:03.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.845+0000 7f1d1355b700 1 -- 172.21.15.52:0/3570357480 >> 172.21.15.52:0/3570357480 conn(0x7f1d0c078300 msgr2=0x7f1d0c0ffef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:59:03.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.846+0000 7f1d1355b700 1 -- 172.21.15.52:0/3570357480 shutdown_connections 2024-06-10T02:59:03.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:03.846+0000 7f1d1355b700 1 -- 172.21.15.52:0/3570357480 wait complete. 2024-06-10T02:59:04.621 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-06-10T02:59:04.897 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:59:04.976 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:04 smithi052 ceph-mon[32362]: pgmap v580: 129 pgs: 129 active+clean; 479 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 1.4 MiB/s wr, 242 op/s 2024-06-10T02:59:05.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:04 smithi120 ceph-mon[36634]: pgmap v580: 129 pgs: 129 active+clean; 479 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 1.4 MiB/s wr, 242 op/s 2024-06-10T02:59:05.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:04 smithi188 ceph-mon[36599]: pgmap v580: 129 pgs: 129 active+clean; 479 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 1.4 MiB/s wr, 242 op/s 2024-06-10T02:59:06.528 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.528+0000 7fcfc4d3d700 1 -- 172.21.15.52:0/3070853846 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcfc0101390 msgr2=0x7fcfc0101810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:59:06.529 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.528+0000 7fcfc4d3d700 1 --2- 172.21.15.52:0/3070853846 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcfc0101390 0x7fcfc0101810 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fcfb4009990 tx=0x7fcfb4009ca0 comp rx=0 tx=0).stop 2024-06-10T02:59:06.529 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.529+0000 7fcfc4d3d700 1 -- 172.21.15.52:0/3070853846 shutdown_connections 2024-06-10T02:59:06.529 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.529+0000 7fcfc4d3d700 1 --2- 172.21.15.52:0/3070853846 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcfc0101d50 0x7fcfc010e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:06.529 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.529+0000 7fcfc4d3d700 1 --2- 172.21.15.52:0/3070853846 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcfc0107400 0x7fcfc01077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:06.529 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.529+0000 7fcfc4d3d700 1 --2- 172.21.15.52:0/3070853846 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcfc0101390 0x7fcfc0101810 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:06.529 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.529+0000 7fcfc4d3d700 1 -- 172.21.15.52:0/3070853846 >> 172.21.15.52:0/3070853846 conn(0x7fcfc0078340 msgr2=0x7fcfc0078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:59:06.529 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.529+0000 7fcfc4d3d700 1 -- 172.21.15.52:0/3070853846 shutdown_connections 2024-06-10T02:59:06.529 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.529+0000 7fcfc4d3d700 1 -- 172.21.15.52:0/3070853846 wait complete. 2024-06-10T02:59:06.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.529+0000 7fcfc4d3d700 1 Processor -- start 2024-06-10T02:59:06.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.529+0000 7fcfc4d3d700 1 -- start start 2024-06-10T02:59:06.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.530+0000 7fcfc4d3d700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcfc0101390 0x7fcfc0199a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:59:06.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.530+0000 7fcfc4d3d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcfc0101d50 0x7fcfc0199fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:59:06.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.530+0000 7fcfc4d3d700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcfc0107400 0x7fcfc01a4af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:59:06.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.530+0000 7fcfc4d3d700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fcfc0111690 con 0x7fcfc0101d50 2024-06-10T02:59:06.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.530+0000 7fcfc4d3d700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fcfc0111510 con 0x7fcfc0107400 2024-06-10T02:59:06.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.530+0000 7fcfc4d3d700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fcfc0111810 con 0x7fcfc0101390 2024-06-10T02:59:06.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.530+0000 7fcfbdd9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcfc0101d50 0x7fcfc0199fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:59:06.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.530+0000 7fcfbe59c700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcfc0101390 0x7fcfc0199a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:59:06.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.530+0000 7fcfbdd9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcfc0101d50 0x7fcfc0199fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56766/0 (socket says 172.21.15.52:56766) 2024-06-10T02:59:06.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.530+0000 7fcfbdd9b700 1 -- 172.21.15.52:0/3646997739 learned_addr learned my addr 172.21.15.52:0/3646997739 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:59:06.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.530+0000 7fcfbdd9b700 1 -- 172.21.15.52:0/3646997739 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcfc0101390 msgr2=0x7fcfc0199a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:59:06.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.530+0000 7fcfbdd9b700 1 --2- 172.21.15.52:0/3646997739 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcfc0101390 0x7fcfc0199a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:06.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.530+0000 7fcfbdd9b700 1 -- 172.21.15.52:0/3646997739 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcfc0107400 msgr2=0x7fcfc01a4af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:59:06.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.530+0000 7fcfbdd9b700 1 --2- 172.21.15.52:0/3646997739 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcfc0107400 0x7fcfc01a4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:06.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.530+0000 7fcfbdd9b700 1 -- 172.21.15.52:0/3646997739 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfc01a5030 con 0x7fcfc0101d50 2024-06-10T02:59:06.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.530+0000 7fcfbdd9b700 1 --2- 172.21.15.52:0/3646997739 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcfc0101d50 0x7fcfc0199fb0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fcfb4000c00 tx=0x7fcfb400bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:59:06.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.531+0000 7fcfaf7fe700 1 -- 172.21.15.52:0/3646997739 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfb4027070 con 0x7fcfc0101d50 2024-06-10T02:59:06.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.531+0000 7fcfc4d3d700 1 -- 172.21.15.52:0/3646997739 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfc01a5200 con 0x7fcfc0101d50 2024-06-10T02:59:06.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.531+0000 7fcfaf7fe700 1 -- 172.21.15.52:0/3646997739 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcfb4003d70 con 0x7fcfc0101d50 2024-06-10T02:59:06.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.531+0000 7fcfc4d3d700 1 -- 172.21.15.52:0/3646997739 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfc01a56b0 con 0x7fcfc0101d50 2024-06-10T02:59:06.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.531+0000 7fcfaf7fe700 1 -- 172.21.15.52:0/3646997739 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfb400f460 con 0x7fcfc0101d50 2024-06-10T02:59:06.533 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.533+0000 7fcfaf7fe700 1 -- 172.21.15.52:0/3646997739 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fcfb402a070 con 0x7fcfc0101d50 2024-06-10T02:59:06.533 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.534+0000 7fcfaf7fe700 1 --2- 172.21.15.52:0/3646997739 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fcfa80a2ca0 0x7fcfa80a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:59:06.534 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.534+0000 7fcfbe59c700 1 --2- 172.21.15.52:0/3646997739 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fcfa80a2ca0 0x7fcfa80a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:59:06.534 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.534+0000 7fcfaf7fe700 1 -- 172.21.15.52:0/3646997739 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7fcfb40d0080 con 0x7fcfc0101d50 2024-06-10T02:59:06.534 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.535+0000 7fcfbe59c700 1 --2- 172.21.15.52:0/3646997739 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fcfa80a2ca0 0x7fcfa80a5160 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fcfb0009be0 tx=0x7fcfb0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:59:06.534 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.535+0000 7fcfcf5c0b80 1 Processor -- start 2024-06-10T02:59:06.535 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.535+0000 7fcfcf5c0b80 1 -- start start 2024-06-10T02:59:06.536 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.535+0000 7fcfcf5c0b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fefa9a5a70 0x55fefa9a5e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:59:06.536 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.535+0000 7fcfcf5c0b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fefa9a6390 0x55fefa99fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:59:06.536 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.535+0000 7fcfcf5c0b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fefa9a0330 0x55fefa9a0750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:59:06.536 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.535+0000 7fcfcf5c0b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55fefa8ed810 con 0x55fefa9a0330 2024-06-10T02:59:06.536 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.535+0000 7fcfcf5c0b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55fefa895720 con 0x55fefa9a5a70 2024-06-10T02:59:06.536 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.535+0000 7fcfcf5c0b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55fefa8e97d0 con 0x55fefa9a6390 2024-06-10T02:59:06.536 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.535+0000 7fcfbed9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fefa9a0330 0x55fefa9a0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:59:06.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.536+0000 7fcfbed9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fefa9a0330 0x55fefa9a0750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56776/0 (socket says 172.21.15.52:56776) 2024-06-10T02:59:06.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.536+0000 7fcfbed9d700 1 -- 172.21.15.52:0/1586813004 learned_addr learned my addr 172.21.15.52:0/1586813004 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:59:06.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.536+0000 7fcfbdd9b700 1 --2- 172.21.15.52:0/1586813004 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fefa9a6390 0x55fefa99fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:59:06.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.536+0000 7fcfbe59c700 1 --2- 172.21.15.52:0/1586813004 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fefa9a5a70 0x55fefa9a5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:59:06.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.536+0000 7fcfbed9d700 1 -- 172.21.15.52:0/1586813004 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fefa9a6390 msgr2=0x55fefa99fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:59:06.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.536+0000 7fcfbed9d700 1 --2- 172.21.15.52:0/1586813004 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fefa9a6390 0x55fefa99fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:06.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.536+0000 7fcfbed9d700 1 -- 172.21.15.52:0/1586813004 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fefa9a5a70 msgr2=0x55fefa9a5e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:59:06.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.536+0000 7fcfbed9d700 1 --2- 172.21.15.52:0/1586813004 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fefa9a5a70 0x55fefa9a5e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:06.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.536+0000 7fcfbed9d700 1 -- 172.21.15.52:0/1586813004 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fefa88bf50 con 0x55fefa9a0330 2024-06-10T02:59:06.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.536+0000 7fcfbed9d700 1 --2- 172.21.15.52:0/1586813004 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fefa9a0330 0x55fefa9a0750 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fcfb800e400 tx=0x7fcfb800e710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6afbf69e5c0a1b4d server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:59:06.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.536+0000 7fcfacff9700 1 -- 172.21.15.52:0/1586813004 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfb8004bb0 con 0x55fefa9a0330 2024-06-10T02:59:06.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.536+0000 7fcfacff9700 1 -- 172.21.15.52:0/1586813004 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcfb8004d50 con 0x55fefa9a0330 2024-06-10T02:59:06.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.536+0000 7fcfacff9700 1 -- 172.21.15.52:0/1586813004 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfb80044c0 con 0x55fefa9a0330 2024-06-10T02:59:06.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.537+0000 7fcfcf5c0b80 1 -- 172.21.15.52:0/1586813004 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fefa9a0330 msgr2=0x55fefa9a0750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:59:06.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.537+0000 7fcfcf5c0b80 1 --2- 172.21.15.52:0/1586813004 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fefa9a0330 0x55fefa9a0750 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fcfb800e400 tx=0x7fcfb800e710 comp rx=0 tx=0).stop 2024-06-10T02:59:06.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.537+0000 7fcfcf5c0b80 1 -- 172.21.15.52:0/1586813004 shutdown_connections 2024-06-10T02:59:06.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.537+0000 7fcfcf5c0b80 1 --2- 172.21.15.52:0/1586813004 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fefa9a0330 0x55fefa9a0750 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:06.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.537+0000 7fcfcf5c0b80 1 --2- 172.21.15.52:0/1586813004 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fefa9a5a70 0x55fefa9a5e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:06.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.537+0000 7fcfcf5c0b80 1 --2- 172.21.15.52:0/1586813004 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fefa9a6390 0x55fefa99fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:06.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.537+0000 7fcfcf5c0b80 1 -- 172.21.15.52:0/1586813004 >> 172.21.15.52:0/1586813004 conn(0x55fefa8c6cc0 msgr2=0x55fefa996850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:59:06.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.537+0000 7fcfcf5c0b80 1 -- 172.21.15.52:0/1586813004 shutdown_connections 2024-06-10T02:59:06.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.537+0000 7fcfcf5c0b80 1 -- 172.21.15.52:0/1586813004 wait complete. 2024-06-10T02:59:06.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.538+0000 7fcfcf5c0b80 1 -- start start 2024-06-10T02:59:06.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.538+0000 7fcfcf5c0b80 1 Processor -- start 2024-06-10T02:59:06.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.538+0000 7fcfcf5c0b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fefa9a0330 0x55fefa9ac0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:59:06.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.538+0000 7fcfcf5c0b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fefa9a5a70 0x55fefa9a93f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:59:06.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.538+0000 7fcfcf5c0b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fefa9a6390 0x55fefa9a9930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:59:06.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.538+0000 7fcfcf5c0b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55fefa9acd70 con 0x55fefa9a6390 2024-06-10T02:59:06.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.538+0000 7fcfcf5c0b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55fefa88dbf0 con 0x55fefa9a0330 2024-06-10T02:59:06.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.538+0000 7fcfcf5c0b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55fefa9acef0 con 0x55fefa9a5a70 2024-06-10T02:59:06.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.538+0000 7fcfcf5c0b80 10 client.0 _ll_register_callbacks cb 0x55fefa887920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T02:59:06.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.538+0000 7fcfbed9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fefa9a6390 0x55fefa9a9930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:59:06.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.538+0000 7fcfbed9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fefa9a6390 0x55fefa9a9930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56788/0 (socket says 172.21.15.52:56788) 2024-06-10T02:59:06.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.538+0000 7fcfbed9d700 1 -- 172.21.15.52:0/3376303944 learned_addr learned my addr 172.21.15.52:0/3376303944 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:59:06.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.538+0000 7fcfbe59c700 1 --2- 172.21.15.52:0/3376303944 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fefa9a0330 0x55fefa9ac0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:59:06.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.538+0000 7fcfbdd9b700 1 --2- 172.21.15.52:0/3376303944 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fefa9a5a70 0x55fefa9a93f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:59:06.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.539+0000 7fcfbe59c700 1 -- 172.21.15.52:0/3376303944 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fefa9a5a70 msgr2=0x55fefa9a93f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:59:06.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.539+0000 7fcfbe59c700 1 --2- 172.21.15.52:0/3376303944 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fefa9a5a70 0x55fefa9a93f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:06.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.539+0000 7fcfbe59c700 1 -- 172.21.15.52:0/3376303944 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fefa9a6390 msgr2=0x55fefa9a9930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:59:06.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.539+0000 7fcfbe59c700 1 --2- 172.21.15.52:0/3376303944 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fefa9a6390 0x55fefa9a9930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:06.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.539+0000 7fcfbe59c700 1 -- 172.21.15.52:0/3376303944 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fefa83c8d0 con 0x55fefa9a0330 2024-06-10T02:59:06.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.539+0000 7fcfbed9d700 1 --2- 172.21.15.52:0/3376303944 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fefa9a6390 0x55fefa9a9930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T02:59:06.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.539+0000 7fcfbe59c700 1 --2- 172.21.15.52:0/3376303944 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fefa9a0330 0x55fefa9ac0a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fcfb00117d0 tx=0x7fcfb00047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a58d250b14c40d2f server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:59:06.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.539+0000 7fcfa5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-06-10T02:59:06.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.539+0000 7fcfa5ffb700 1 -- 172.21.15.52:0/3376303944 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfb0004030 con 0x55fefa9a0330 2024-06-10T02:59:06.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.539+0000 7fcfcf5c0b80 1 -- 172.21.15.52:0/3376303944 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcfb8004d50 con 0x55fefa9a0330 2024-06-10T02:59:06.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.539+0000 7fcfa5ffb700 1 -- 172.21.15.52:0/3376303944 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7fcfb0009040 con 0x55fefa9a0330 2024-06-10T02:59:06.543 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.539+0000 7fcfa5ffb700 1 -- 172.21.15.52:0/3376303944 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fcfb0030050 con 0x55fefa9a0330 2024-06-10T02:59:06.543 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.539+0000 7fcfcf5c0b80 10 client.25793 fetch_fsmap learned FSMap version 34 2024-06-10T02:59:06.543 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.539+0000 7fcfcf5c0b80 1 -- 172.21.15.52:0/3376303944 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55fefa9b7680 con 0x55fefa9a0330 2024-06-10T02:59:06.543 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.540+0000 7fcfa5ffb700 1 -- 172.21.15.52:0/3376303944 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcfb8004d50 con 0x55fefa9a0330 2024-06-10T02:59:06.543 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.540+0000 7fcfa5ffb700 1 -- 172.21.15.52:0/3376303944 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 34) ==== 8378+0+0 (secure 0 0 0) 0x7fcfb0005280 con 0x55fefa9a0330 2024-06-10T02:59:06.543 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.540+0000 7fcfcf5c0b80 10 client.25793 fetch_fsmap finished waiting for FSMap version 34 2024-06-10T02:59:06.543 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.540+0000 7fcfcf5c0b80 10 client.25793 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T02:59:06.543 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.540+0000 7fcfcf5c0b80 20 client.25793 populate_metadata read hostname 'smithi052' 2024-06-10T02:59:06.543 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.540+0000 7fcfcf5c0b80 1 --2- 172.21.15.52:0/3376303944 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55fefa9b9cc0 0x55fefa9bc0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:59:06.543 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.540+0000 7fcfcf5c0b80 4 client.25793 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T02:59:06.543 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.540+0000 7fcfcf5c0b80 1 -- 172.21.15.52:0/3376303944 --> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fefa8fad10 con 0x55fefa9b9cc0 2024-06-10T02:59:06.543 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.540+0000 7fcfbdd9b700 1 --2- 172.21.15.52:0/3376303944 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55fefa9b9cc0 0x55fefa9bc0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:59:06.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.541+0000 7fcfbdd9b700 1 --2- 172.21.15.52:0/3376303944 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55fefa9b9cc0 0x55fefa9bc0b0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b96a3321f0dff935 server_cookie=e65033359d66ec58 in_seq=0 out_seq=0 2024-06-10T02:59:06.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.541+0000 7fcfa5ffb700 10 client.25793 ms_handle_connect on v2:172.21.15.188:6838/310444836 2024-06-10T02:59:06.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.542+0000 7fcfa5ffb700 1 -- 172.21.15.52:0/3376303944 <== mds.0 v2:172.21.15.188:6838/310444836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55fefa8fad10 con 0x55fefa9b9cc0 2024-06-10T02:59:06.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.542+0000 7fcfa5ffb700 10 client.25793 handle_command_reply: tid=0 2024-06-10T02:59:06.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.542+0000 7fcfcf5c0b80 1 client.25793 shutdown 2024-06-10T02:59:06.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.542+0000 7fcfcf5c0b80 1 -- 172.21.15.52:0/3376303944 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fefa9a0330 msgr2=0x55fefa9ac0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:59:06.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.542+0000 7fcfcf5c0b80 1 --2- 172.21.15.52:0/3376303944 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fefa9a0330 0x55fefa9ac0a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fcfb00117d0 tx=0x7fcfb00047c0 comp rx=0 tx=0).stop 2024-06-10T02:59:06.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.543+0000 7fcfcf5c0b80 1 -- 172.21.15.52:0/3376303944 shutdown_connections 2024-06-10T02:59:06.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.543+0000 7fcfcf5c0b80 1 --2- 172.21.15.52:0/3376303944 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55fefa9b9cc0 0x55fefa9bc0b0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:06.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.543+0000 7fcfcf5c0b80 1 --2- 172.21.15.52:0/3376303944 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fefa9a6390 0x55fefa9a9930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:06.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.543+0000 7fcfcf5c0b80 1 --2- 172.21.15.52:0/3376303944 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fefa9a0330 0x55fefa9ac0a0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:06.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.543+0000 7fcfcf5c0b80 1 --2- 172.21.15.52:0/3376303944 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fefa9a5a70 0x55fefa9a93f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:06.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.543+0000 7fcfcf5c0b80 1 -- 172.21.15.52:0/3376303944 >> 172.21.15.52:0/3376303944 conn(0x55fefa8c6cc0 msgr2=0x55fefa99a9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:59:06.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.543+0000 7fcfa5ffb700 0 client.25793 ms_handle_reset on v2:172.21.15.188:6838/310444836 2024-06-10T02:59:06.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.543+0000 7fcfcf5c0b80 1 -- 172.21.15.52:0/3376303944 shutdown_connections 2024-06-10T02:59:06.556 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.549+0000 7fcfcf5c0b80 1 -- 172.21.15.52:0/3376303944 wait complete. 2024-06-10T02:59:06.556 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.553+0000 7fcfcf5c0b80 20 client.25793 trim_cache size 0 max 16384 2024-06-10T02:59:06.592 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.591+0000 7fcfcf5c0b80 1 Processor -- start 2024-06-10T02:59:06.592 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.592+0000 7fcfcf5c0b80 1 -- start start 2024-06-10T02:59:06.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.592+0000 7fcfcf5c0b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fefa9a0330 0x55fefa9a9960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:59:06.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.592+0000 7fcfcf5c0b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fefa9a5a70 0x55fefa99a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:59:06.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.592+0000 7fcfcf5c0b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fefa9a6390 0x55fefa99af10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:59:06.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.592+0000 7fcfcf5c0b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fcfc01ac170 con 0x55fefa9a0330 2024-06-10T02:59:06.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.592+0000 7fcfcf5c0b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fcfc01add70 con 0x55fefa9a6390 2024-06-10T02:59:06.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.592+0000 7fcfcf5c0b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fcfc01ac840 con 0x55fefa9a5a70 2024-06-10T02:59:06.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.592+0000 7fcfbe59c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fefa9a0330 0x55fefa9a9960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:59:06.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.592+0000 7fcfbe59c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fefa9a0330 0x55fefa9a9960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56792/0 (socket says 172.21.15.52:56792) 2024-06-10T02:59:06.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.592+0000 7fcfbe59c700 1 -- 172.21.15.52:0/311909474 learned_addr learned my addr 172.21.15.52:0/311909474 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:59:06.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.593+0000 7fcfbe59c700 1 -- 172.21.15.52:0/311909474 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fefa9a5a70 msgr2=0x55fefa99a9d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-10T02:59:06.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.593+0000 7fcfbe59c700 1 --2- 172.21.15.52:0/311909474 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fefa9a5a70 0x55fefa99a9d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:06.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.593+0000 7fcfbe59c700 1 -- 172.21.15.52:0/311909474 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fefa9a6390 msgr2=0x55fefa99af10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-10T02:59:06.595 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.593+0000 7fcfbe59c700 1 --2- 172.21.15.52:0/311909474 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fefa9a6390 0x55fefa99af10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:06.595 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.593+0000 7fcfbe59c700 1 -- 172.21.15.52:0/311909474 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fefa998130 con 0x55fefa9a0330 2024-06-10T02:59:06.595 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.593+0000 7fcfbe59c700 1 --2- 172.21.15.52:0/311909474 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fefa9a0330 0x55fefa9a9960 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fcfb0011b50 tx=0x7fcfb0011880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae6d7762efbeebba server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:59:06.595 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.594+0000 7fcfa7fff700 1 -- 172.21.15.52:0/311909474 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfb001a6b0 con 0x55fefa9a0330 2024-06-10T02:59:06.595 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.594+0000 7fcfa7fff700 1 -- 172.21.15.52:0/311909474 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcfb0004040 con 0x55fefa9a0330 2024-06-10T02:59:06.595 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.594+0000 7fcfa7fff700 1 -- 172.21.15.52:0/311909474 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfb0003a00 con 0x55fefa9a0330 2024-06-10T02:59:06.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.595+0000 7fcfcf5c0b80 1 -- 172.21.15.52:0/311909474 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fefa9a0330 msgr2=0x55fefa9a9960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:59:06.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.595+0000 7fcfcf5c0b80 1 --2- 172.21.15.52:0/311909474 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fefa9a0330 0x55fefa9a9960 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fcfb0011b50 tx=0x7fcfb0011880 comp rx=0 tx=0).stop 2024-06-10T02:59:06.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.595+0000 7fcfcf5c0b80 1 -- 172.21.15.52:0/311909474 shutdown_connections 2024-06-10T02:59:06.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.595+0000 7fcfcf5c0b80 1 --2- 172.21.15.52:0/311909474 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fefa9a6390 0x55fefa99af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:06.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.595+0000 7fcfcf5c0b80 1 --2- 172.21.15.52:0/311909474 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fefa9a0330 0x55fefa9a9960 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:06.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.595+0000 7fcfcf5c0b80 1 --2- 172.21.15.52:0/311909474 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fefa9a5a70 0x55fefa99a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:06.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.595+0000 7fcfcf5c0b80 1 -- 172.21.15.52:0/311909474 >> 172.21.15.52:0/311909474 conn(0x55fefa8c6cc0 msgr2=0x55fefa9a7ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:59:06.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.596+0000 7fcfcf5c0b80 1 -- 172.21.15.52:0/311909474 shutdown_connections 2024-06-10T02:59:06.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.596+0000 7fcfcf5c0b80 1 -- 172.21.15.52:0/311909474 wait complete. 2024-06-10T02:59:06.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.596+0000 7fcfcf5c0b80 1 -- start start 2024-06-10T02:59:06.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.597+0000 7fcfcf5c0b80 1 Processor -- start 2024-06-10T02:59:06.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.597+0000 7fcfcf5c0b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fefa9a0330 0x55fefa9c0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:59:06.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.597+0000 7fcfcf5c0b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fefa9a5a70 0x55fefa9c12d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:59:06.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.597+0000 7fcfcf5c0b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fefa9a6390 0x55fefa9c1810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:59:06.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.597+0000 7fcfcf5c0b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fcfc01ad6a0 con 0x55fefa9a0330 2024-06-10T02:59:06.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.597+0000 7fcfcf5c0b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fcfc01acfd0 con 0x55fefa9a5a70 2024-06-10T02:59:06.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.597+0000 7fcfcf5c0b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fcfc01aa110 con 0x55fefa9a6390 2024-06-10T02:59:06.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.597+0000 7fcfcf5c0b80 10 client.0 _ll_register_callbacks cb 0x55fefa887920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T02:59:06.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.597+0000 7fcfbed9d700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fefa9a6390 0x55fefa9c1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:59:06.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.597+0000 7fcfbed9d700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fefa9a6390 0x55fefa9c1810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.52:41784/0 (socket says 172.21.15.52:41784) 2024-06-10T02:59:06.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.598+0000 7fcfbed9d700 1 -- 172.21.15.52:0/1807854857 learned_addr learned my addr 172.21.15.52:0/1807854857 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:59:06.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.598+0000 7fcfbdd9b700 1 --2- 172.21.15.52:0/1807854857 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fefa9a5a70 0x55fefa9c12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:59:06.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.598+0000 7fcfbe59c700 1 --2- 172.21.15.52:0/1807854857 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fefa9a0330 0x55fefa9c0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:59:06.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.598+0000 7fcfbed9d700 1 -- 172.21.15.52:0/1807854857 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fefa9a5a70 msgr2=0x55fefa9c12d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:59:06.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.598+0000 7fcfbed9d700 1 --2- 172.21.15.52:0/1807854857 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fefa9a5a70 0x55fefa9c12d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:06.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.598+0000 7fcfbed9d700 1 -- 172.21.15.52:0/1807854857 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fefa9a0330 msgr2=0x55fefa9c0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:59:06.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.598+0000 7fcfbed9d700 1 --2- 172.21.15.52:0/1807854857 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fefa9a0330 0x55fefa9c0d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:06.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.598+0000 7fcfbed9d700 1 -- 172.21.15.52:0/1807854857 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fefa9b12d0 con 0x55fefa9a6390 2024-06-10T02:59:06.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.598+0000 7fcfbed9d700 1 --2- 172.21.15.52:0/1807854857 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fefa9a6390 0x55fefa9c1810 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fcfb800e400 tx=0x7fcfb8003f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4be75ed7d66b46e5 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:59:06.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.598+0000 7fcfa5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-06-10T02:59:06.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.598+0000 7fcfa5ffb700 1 -- 172.21.15.52:0/1807854857 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfb8023030 con 0x55fefa9a6390 2024-06-10T02:59:06.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.599+0000 7fcfa5ffb700 1 -- 172.21.15.52:0/1807854857 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7fcfb8019040 con 0x55fefa9a6390 2024-06-10T02:59:06.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.599+0000 7fcfcf5c0b80 1 -- 172.21.15.52:0/1807854857 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcfb0004040 con 0x55fefa9a6390 2024-06-10T02:59:06.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.599+0000 7fcfa5ffb700 1 -- 172.21.15.52:0/1807854857 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fcfb8030050 con 0x55fefa9a6390 2024-06-10T02:59:06.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.599+0000 7fcfcf5c0b80 10 client.25798 fetch_fsmap learned FSMap version 34 2024-06-10T02:59:06.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.599+0000 7fcfcf5c0b80 1 -- 172.21.15.52:0/1807854857 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55fefa9b2590 con 0x55fefa9a6390 2024-06-10T02:59:06.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.599+0000 7fcfa5ffb700 1 -- 172.21.15.52:0/1807854857 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcfb80092e0 con 0x55fefa9a6390 2024-06-10T02:59:06.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.599+0000 7fcfa5ffb700 1 -- 172.21.15.52:0/1807854857 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 34) ==== 8378+0+0 (secure 0 0 0) 0x7fcfb8007620 con 0x55fefa9a6390 2024-06-10T02:59:06.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.599+0000 7fcfcf5c0b80 10 client.25798 fetch_fsmap finished waiting for FSMap version 34 2024-06-10T02:59:06.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.599+0000 7fcfcf5c0b80 10 client.25798 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T02:59:06.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.599+0000 7fcfcf5c0b80 20 client.25798 populate_metadata read hostname 'smithi052' 2024-06-10T02:59:06.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.600+0000 7fcfcf5c0b80 1 --2- 172.21.15.52:0/1807854857 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55fefa9b2a10 0x55fefaa2b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:59:06.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.600+0000 7fcfcf5c0b80 4 client.25798 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T02:59:06.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.600+0000 7fcfcf5c0b80 1 -- 172.21.15.52:0/1807854857 --> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fcfb0004040 con 0x55fefa9b2a10 2024-06-10T02:59:06.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.600+0000 7fcfbe59c700 1 --2- 172.21.15.52:0/1807854857 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55fefa9b2a10 0x55fefaa2b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:59:06.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.600+0000 7fcfa5ffb700 10 client.25798 ms_handle_connect on v2:172.21.15.188:6838/310444836 2024-06-10T02:59:06.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.600+0000 7fcfbe59c700 1 --2- 172.21.15.52:0/1807854857 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55fefa9b2a10 0x55fefaa2b150 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6570cfe58d003923 server_cookie=899ab5177165b7fc in_seq=0 out_seq=0 2024-06-10T02:59:06.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.601+0000 7fcfa5ffb700 1 -- 172.21.15.52:0/1807854857 <== mds.0 v2:172.21.15.188:6838/310444836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fcfb0004040 con 0x55fefa9b2a10 2024-06-10T02:59:06.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.601+0000 7fcfa5ffb700 10 client.25798 handle_command_reply: tid=0 2024-06-10T02:59:06.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.601+0000 7fcfcf5c0b80 1 client.25798 shutdown 2024-06-10T02:59:06.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.601+0000 7fcfcf5c0b80 1 -- 172.21.15.52:0/1807854857 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fefa9a6390 msgr2=0x55fefa9c1810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:59:06.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.601+0000 7fcfcf5c0b80 1 --2- 172.21.15.52:0/1807854857 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fefa9a6390 0x55fefa9c1810 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fcfb800e400 tx=0x7fcfb8003f40 comp rx=0 tx=0).stop 2024-06-10T02:59:06.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.601+0000 7fcfcf5c0b80 1 -- 172.21.15.52:0/1807854857 shutdown_connections 2024-06-10T02:59:06.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.601+0000 7fcfcf5c0b80 1 --2- 172.21.15.52:0/1807854857 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55fefa9b2a10 0x55fefaa2b150 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:06.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.601+0000 7fcfcf5c0b80 1 --2- 172.21.15.52:0/1807854857 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fefa9a6390 0x55fefa9c1810 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:06.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.601+0000 7fcfcf5c0b80 1 --2- 172.21.15.52:0/1807854857 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fefa9a0330 0x55fefa9c0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:06.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.601+0000 7fcfcf5c0b80 1 --2- 172.21.15.52:0/1807854857 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fefa9a5a70 0x55fefa9c12d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:06.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.601+0000 7fcfcf5c0b80 1 -- 172.21.15.52:0/1807854857 >> 172.21.15.52:0/1807854857 conn(0x55fefa8c6cc0 msgr2=0x55fefa99a9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:59:06.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.601+0000 7fcfa5ffb700 0 client.25798 ms_handle_reset on v2:172.21.15.188:6838/310444836 2024-06-10T02:59:06.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.601+0000 7fcfcf5c0b80 1 -- 172.21.15.52:0/1807854857 shutdown_connections 2024-06-10T02:59:06.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.601+0000 7fcfcf5c0b80 1 -- 172.21.15.52:0/1807854857 wait complete. 2024-06-10T02:59:06.604 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-06-10T02:59:06.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.601+0000 7fcfcf5c0b80 20 client.25798 trim_cache size 0 max 16384 2024-06-10T02:59:06.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.602+0000 7fcfc4d3d700 1 -- 172.21.15.52:0/3646997739 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fcfa80a2ca0 msgr2=0x7fcfa80a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:59:06.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.602+0000 7fcfc4d3d700 1 --2- 172.21.15.52:0/3646997739 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fcfa80a2ca0 0x7fcfa80a5160 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fcfb0009be0 tx=0x7fcfb0009260 comp rx=0 tx=0).stop 2024-06-10T02:59:06.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.602+0000 7fcfc4d3d700 1 -- 172.21.15.52:0/3646997739 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcfc0101d50 msgr2=0x7fcfc0199fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:59:06.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.602+0000 7fcfc4d3d700 1 --2- 172.21.15.52:0/3646997739 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcfc0101d50 0x7fcfc0199fb0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fcfb4000c00 tx=0x7fcfb400bab0 comp rx=0 tx=0).stop 2024-06-10T02:59:06.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.602+0000 7fcfc4d3d700 1 -- 172.21.15.52:0/3646997739 shutdown_connections 2024-06-10T02:59:06.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.602+0000 7fcfc4d3d700 1 --2- 172.21.15.52:0/3646997739 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcfc0107400 0x7fcfc01a4af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:06.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.602+0000 7fcfc4d3d700 1 --2- 172.21.15.52:0/3646997739 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcfc0101390 0x7fcfc0199a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:06.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.602+0000 7fcfc4d3d700 1 --2- 172.21.15.52:0/3646997739 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fcfa80a2ca0 0x7fcfa80a5160 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:06.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.602+0000 7fcfc4d3d700 1 --2- 172.21.15.52:0/3646997739 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcfc0101d50 0x7fcfc0199fb0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:06.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.602+0000 7fcfc4d3d700 1 -- 172.21.15.52:0/3646997739 >> 172.21.15.52:0/3646997739 conn(0x7fcfc0078340 msgr2=0x7fcfc00ffe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:59:06.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.603+0000 7fcfc4d3d700 1 -- 172.21.15.52:0/3646997739 shutdown_connections 2024-06-10T02:59:06.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:06.603+0000 7fcfc4d3d700 1 -- 172.21.15.52:0/3646997739 wait complete. 2024-06-10T02:59:07.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:06 smithi120 ceph-mon[36634]: pgmap v581: 129 pgs: 129 active+clean; 479 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 374 KiB/s wr, 222 op/s 2024-06-10T02:59:07.203 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:06 smithi052 ceph-mon[32362]: pgmap v581: 129 pgs: 129 active+clean; 479 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 374 KiB/s wr, 222 op/s 2024-06-10T02:59:07.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:06 smithi188 ceph-mon[36599]: pgmap v581: 129 pgs: 129 active+clean; 479 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 374 KiB/s wr, 222 op/s 2024-06-10T02:59:07.246 INFO:tasks.workunit.client.0.smithi052.stdout: 76 80182 103 55143 126 46278 223 2024-06-10T02:59:08.234 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-06-10T02:59:08.234 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-06-10T02:59:08.811 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:59:09.129 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:08 smithi052 ceph-mon[32362]: pgmap v582: 129 pgs: 129 active+clean; 483 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 730 KiB/s wr, 246 op/s 2024-06-10T02:59:09.130 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:08 smithi052 ceph-mon[32362]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-10T02:59:09.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:08 smithi120 ceph-mon[36634]: pgmap v582: 129 pgs: 129 active+clean; 483 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 730 KiB/s wr, 246 op/s 2024-06-10T02:59:09.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:08 smithi120 ceph-mon[36634]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-10T02:59:09.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:08 smithi188 ceph-mon[36599]: pgmap v582: 129 pgs: 129 active+clean; 483 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 730 KiB/s wr, 246 op/s 2024-06-10T02:59:09.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:08 smithi188 ceph-mon[36599]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-10T02:59:11.062 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:10 smithi052 ceph-mon[32362]: pgmap v583: 129 pgs: 129 active+clean; 483 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 607 KiB/s wr, 194 op/s 2024-06-10T02:59:11.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:10 smithi120 ceph-mon[36634]: pgmap v583: 129 pgs: 129 active+clean; 483 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 607 KiB/s wr, 194 op/s 2024-06-10T02:59:11.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:10 smithi188 ceph-mon[36599]: pgmap v583: 129 pgs: 129 active+clean; 483 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 607 KiB/s wr, 194 op/s 2024-06-10T02:59:11.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.381+0000 7f2486301700 1 -- 172.21.15.52:0/2421700160 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2480101d40 msgr2=0x7f248010e230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:59:11.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.381+0000 7f2486301700 1 --2- 172.21.15.52:0/2421700160 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2480101d40 0x7f248010e230 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f247800b0a0 tx=0x7f247800b3b0 comp rx=0 tx=0).stop 2024-06-10T02:59:11.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.381+0000 7f2486301700 1 -- 172.21.15.52:0/2421700160 shutdown_connections 2024-06-10T02:59:11.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.381+0000 7f2486301700 1 --2- 172.21.15.52:0/2421700160 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2480101d40 0x7f248010e230 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:11.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.381+0000 7f2486301700 1 --2- 172.21.15.52:0/2421700160 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f24801073f0 0x7f24801077d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:11.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.381+0000 7f2486301700 1 --2- 172.21.15.52:0/2421700160 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2480101380 0x7f2480101800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:11.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.381+0000 7f2486301700 1 -- 172.21.15.52:0/2421700160 >> 172.21.15.52:0/2421700160 conn(0x7f2480078300 msgr2=0x7f2480078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:59:11.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.381+0000 7f2486301700 1 -- 172.21.15.52:0/2421700160 shutdown_connections 2024-06-10T02:59:11.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.381+0000 7f2486301700 1 -- 172.21.15.52:0/2421700160 wait complete. 2024-06-10T02:59:11.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.382+0000 7f2486301700 1 Processor -- start 2024-06-10T02:59:11.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.382+0000 7f2486301700 1 -- start start 2024-06-10T02:59:11.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.383+0000 7f2486301700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2480101380 0x7f248019e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:59:11.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.383+0000 7f2486301700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2480101d40 0x7f248019ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:59:11.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.383+0000 7f2486301700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f24801073f0 0x7f24801989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:59:11.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.383+0000 7f247ffff700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2480101380 0x7f248019e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:59:11.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.383+0000 7f247ffff700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2480101380 0x7f248019e7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.52:54124/0 (socket says 172.21.15.52:54124) 2024-06-10T02:59:11.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.383+0000 7f247ffff700 1 -- 172.21.15.52:0/3412995014 learned_addr learned my addr 172.21.15.52:0/3412995014 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:59:11.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.383+0000 7f2486301700 1 -- 172.21.15.52:0/3412995014 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f2480112790 con 0x7f2480101d40 2024-06-10T02:59:11.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.383+0000 7f2486301700 1 -- 172.21.15.52:0/3412995014 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f2480112610 con 0x7f24801073f0 2024-06-10T02:59:11.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.383+0000 7f2486301700 1 -- 172.21.15.52:0/3412995014 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f2480112910 con 0x7f2480101380 2024-06-10T02:59:11.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.383+0000 7f248489e700 1 --2- 172.21.15.52:0/3412995014 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f24801073f0 0x7f24801989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:59:11.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.384+0000 7f247f7fe700 1 --2- 172.21.15.52:0/3412995014 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2480101d40 0x7f248019ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:59:11.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.384+0000 7f247ffff700 1 -- 172.21.15.52:0/3412995014 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f24801073f0 msgr2=0x7f24801989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:59:11.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.384+0000 7f247ffff700 1 --2- 172.21.15.52:0/3412995014 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f24801073f0 0x7f24801989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:11.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.384+0000 7f247ffff700 1 -- 172.21.15.52:0/3412995014 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2480101d40 msgr2=0x7f248019ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:59:11.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.384+0000 7f247ffff700 1 --2- 172.21.15.52:0/3412995014 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2480101d40 0x7f248019ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:11.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.384+0000 7f247ffff700 1 -- 172.21.15.52:0/3412995014 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2480199110 con 0x7f2480101380 2024-06-10T02:59:11.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.384+0000 7f247f7fe700 1 --2- 172.21.15.52:0/3412995014 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2480101d40 0x7f248019ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T02:59:11.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.384+0000 7f247ffff700 1 --2- 172.21.15.52:0/3412995014 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2480101380 0x7f248019e7f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f247001f9b0 tx=0x7f247001fd70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:59:11.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.384+0000 7f247d7fa700 1 -- 172.21.15.52:0/3412995014 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f247001dbe0 con 0x7f2480101380 2024-06-10T02:59:11.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.384+0000 7f2486301700 1 -- 172.21.15.52:0/3412995014 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2480199400 con 0x7f2480101380 2024-06-10T02:59:11.386 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.385+0000 7f2486301700 1 -- 172.21.15.52:0/3412995014 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f248006beb0 con 0x7f2480101380 2024-06-10T02:59:11.386 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.385+0000 7f247d7fa700 1 -- 172.21.15.52:0/3412995014 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f247001dd80 con 0x7f2480101380 2024-06-10T02:59:11.386 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.385+0000 7f247d7fa700 1 -- 172.21.15.52:0/3412995014 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24700297c0 con 0x7f2480101380 2024-06-10T02:59:11.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.389+0000 7f247d7fa700 1 -- 172.21.15.52:0/3412995014 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f2470029960 con 0x7f2480101380 2024-06-10T02:59:11.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.390+0000 7f247d7fa700 1 --2- 172.21.15.52:0/3412995014 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f24680a2ca0 0x7f24680a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:59:11.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.391+0000 7f247d7fa700 1 -- 172.21.15.52:0/3412995014 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f24700e0660 con 0x7f2480101380 2024-06-10T02:59:11.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.391+0000 7f247f7fe700 1 --2- 172.21.15.52:0/3412995014 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f24680a2ca0 0x7f24680a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:59:11.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.392+0000 7f2490b84b80 1 Processor -- start 2024-06-10T02:59:11.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.392+0000 7f2490b84b80 1 -- start start 2024-06-10T02:59:11.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.392+0000 7f247f7fe700 1 --2- 172.21.15.52:0/3412995014 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f24680a2ca0 0x7f24680a5160 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f2480078800 tx=0x7f2474009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:59:11.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.392+0000 7f2490b84b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5572d080afe0 0x5572d080b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:59:11.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.392+0000 7f2490b84b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5572d080b900 0x5572d08052e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:59:11.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.392+0000 7f2490b84b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5572d0805820 0x5572d0805c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:59:11.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.393+0000 7f247f7fe700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5572d080b900 0x5572d08052e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:59:11.395 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.393+0000 7f247f7fe700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5572d080b900 0x5572d08052e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.52:54130/0 (socket says 172.21.15.52:54130) 2024-06-10T02:59:11.395 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.393+0000 7f247f7fe700 1 -- 172.21.15.52:0/807993103 learned_addr learned my addr 172.21.15.52:0/807993103 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:59:11.395 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.393+0000 7f2490b84b80 1 -- 172.21.15.52:0/807993103 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5572d06f7fe0 con 0x5572d0805820 2024-06-10T02:59:11.395 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.393+0000 7f248489e700 1 --2- 172.21.15.52:0/807993103 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5572d0805820 0x5572d0805c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:59:11.395 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.393+0000 7f2490b84b80 1 -- 172.21.15.52:0/807993103 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5572d07568d0 con 0x5572d080afe0 2024-06-10T02:59:11.395 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.393+0000 7f2490b84b80 1 -- 172.21.15.52:0/807993103 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5572d06be4e0 con 0x5572d080b900 2024-06-10T02:59:11.395 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.393+0000 7f247f7fe700 1 -- 172.21.15.52:0/807993103 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5572d080afe0 msgr2=0x5572d080b3c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-10T02:59:11.395 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.393+0000 7f247f7fe700 1 --2- 172.21.15.52:0/807993103 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5572d080afe0 0x5572d080b3c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:11.395 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.393+0000 7f247f7fe700 1 -- 172.21.15.52:0/807993103 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5572d0805820 msgr2=0x5572d0805c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:59:11.395 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.393+0000 7f247f7fe700 1 --2- 172.21.15.52:0/807993103 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5572d0805820 0x5572d0805c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:11.395 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.393+0000 7f247f7fe700 1 -- 172.21.15.52:0/807993103 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5572d06a5950 con 0x5572d080b900 2024-06-10T02:59:11.395 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.394+0000 7f248489e700 1 --2- 172.21.15.52:0/807993103 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5572d0805820 0x5572d0805c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T02:59:11.395 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.394+0000 7f247f7fe700 1 --2- 172.21.15.52:0/807993103 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5572d080b900 0x5572d08052e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f2474009550 tx=0x7f2474014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f7a036ac904de380 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:59:11.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.394+0000 7f24667fc700 1 -- 172.21.15.52:0/807993103 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2474012d40 con 0x5572d080b900 2024-06-10T02:59:11.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.395+0000 7f24667fc700 1 -- 172.21.15.52:0/807993103 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2474041910 con 0x5572d080b900 2024-06-10T02:59:11.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.395+0000 7f24667fc700 1 -- 172.21.15.52:0/807993103 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2474040900 con 0x5572d080b900 2024-06-10T02:59:11.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.396+0000 7f2490b84b80 1 -- 172.21.15.52:0/807993103 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5572d080b900 msgr2=0x5572d08052e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:59:11.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.396+0000 7f2490b84b80 1 --2- 172.21.15.52:0/807993103 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5572d080b900 0x5572d08052e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f2474009550 tx=0x7f2474014000 comp rx=0 tx=0).stop 2024-06-10T02:59:11.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.396+0000 7f2490b84b80 1 -- 172.21.15.52:0/807993103 shutdown_connections 2024-06-10T02:59:11.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.396+0000 7f2490b84b80 1 --2- 172.21.15.52:0/807993103 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5572d0805820 0x5572d0805c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:11.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.396+0000 7f2490b84b80 1 --2- 172.21.15.52:0/807993103 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5572d080afe0 0x5572d080b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:11.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.396+0000 7f2490b84b80 1 --2- 172.21.15.52:0/807993103 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5572d080b900 0x5572d08052e0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:11.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.396+0000 7f2490b84b80 1 -- 172.21.15.52:0/807993103 >> 172.21.15.52:0/807993103 conn(0x5572d0732100 msgr2=0x5572d08002a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:59:11.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.396+0000 7f2490b84b80 1 -- 172.21.15.52:0/807993103 shutdown_connections 2024-06-10T02:59:11.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.396+0000 7f2490b84b80 1 -- 172.21.15.52:0/807993103 wait complete. 2024-06-10T02:59:11.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.397+0000 7f2490b84b80 1 -- start start 2024-06-10T02:59:11.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.397+0000 7f2490b84b80 1 Processor -- start 2024-06-10T02:59:11.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.398+0000 7f2490b84b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5572d0805820 0x5572d0811660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:59:11.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.398+0000 7f2490b84b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5572d080afe0 0x5572d080e960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:59:11.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.398+0000 7f2490b84b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5572d080b900 0x5572d080eea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:59:11.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.398+0000 7f247ffff700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5572d0805820 0x5572d0811660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:59:11.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.398+0000 7f248489e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5572d080b900 0x5572d080eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:59:11.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.398+0000 7f247ffff700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5572d0805820 0x5572d0811660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.52:54138/0 (socket says 172.21.15.52:54138) 2024-06-10T02:59:11.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.398+0000 7f248489e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5572d080b900 0x5572d080eea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57764/0 (socket says 172.21.15.52:57764) 2024-06-10T02:59:11.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.398+0000 7f247f7fe700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5572d080afe0 0x5572d080e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:59:11.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.398+0000 7f247ffff700 1 -- 172.21.15.52:0/2985104786 learned_addr learned my addr 172.21.15.52:0/2985104786 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:59:11.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.398+0000 7f2490b84b80 1 -- 172.21.15.52:0/2985104786 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5572d06f7040 con 0x5572d080b900 2024-06-10T02:59:11.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.399+0000 7f2490b84b80 1 -- 172.21.15.52:0/2985104786 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5572d0752890 con 0x5572d080afe0 2024-06-10T02:59:11.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.399+0000 7f2490b84b80 1 -- 172.21.15.52:0/2985104786 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5572d0812380 con 0x5572d0805820 2024-06-10T02:59:11.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.399+0000 7f248489e700 1 -- 172.21.15.52:0/2985104786 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5572d0805820 msgr2=0x5572d0811660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:59:11.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.399+0000 7f248489e700 1 --2- 172.21.15.52:0/2985104786 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5572d0805820 0x5572d0811660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:11.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.399+0000 7f248489e700 1 -- 172.21.15.52:0/2985104786 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5572d080afe0 msgr2=0x5572d080e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:59:11.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.399+0000 7f248489e700 1 --2- 172.21.15.52:0/2985104786 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5572d080afe0 0x5572d080e960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:11.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.399+0000 7f248489e700 1 -- 172.21.15.52:0/2985104786 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5572d071c7d0 con 0x5572d080b900 2024-06-10T02:59:11.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.400+0000 7f248489e700 1 --2- 172.21.15.52:0/2985104786 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5572d080b900 0x5572d080eea0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f24780062a0 tx=0x7f2478009dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bd8e222c86a07ce server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:59:11.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.401+0000 7f245f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-10T02:59:11.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.401+0000 7f245f7fe700 1 -- 172.21.15.52:0/2985104786 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24780207c0 con 0x5572d080b900 2024-06-10T02:59:11.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.401+0000 7f245f7fe700 1 -- 172.21.15.52:0/2985104786 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f2478020960 con 0x5572d080b900 2024-06-10T02:59:11.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.401+0000 7f245f7fe700 10 client.0 inactive, discarding osd_map(100..100 src has 1..100) 2024-06-10T02:59:11.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.401+0000 7f2490b84b80 10 client.0 _ll_register_callbacks cb 0x5572d06ec630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T02:59:11.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.401+0000 7f2490b84b80 1 -- 172.21.15.52:0/2985104786 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2474041910 con 0x5572d080b900 2024-06-10T02:59:11.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.402+0000 7f245f7fe700 1 -- 172.21.15.52:0/2985104786 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f2478020b80 con 0x5572d080b900 2024-06-10T02:59:11.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.402+0000 7f2490b84b80 10 client.16236 fetch_fsmap learned FSMap version 35 2024-06-10T02:59:11.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.402+0000 7f2490b84b80 1 -- 172.21.15.52:0/2985104786 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({fsmap=35}) -- 0x5572d081c7b0 con 0x5572d080b900 2024-06-10T02:59:11.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.403+0000 7f245f7fe700 1 -- 172.21.15.52:0/2985104786 <== mon.0 v2:172.21.15.52:3300/0 4 ==== fsmap(e 35) ==== 8374+0+0 (secure 0 0 0) 0x7f2478011070 con 0x5572d080b900 2024-06-10T02:59:11.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.403+0000 7f2490b84b80 10 client.16236 fetch_fsmap finished waiting for FSMap version 35 2024-06-10T02:59:11.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.403+0000 7f2490b84b80 10 client.16236 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T02:59:11.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.403+0000 7f2490b84b80 20 client.16236 populate_metadata read hostname 'smithi052' 2024-06-10T02:59:11.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.403+0000 7f2490b84b80 1 --2- 172.21.15.52:0/2985104786 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x5572d081ed50 0x5572d0821180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:59:11.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.403+0000 7f2490b84b80 4 client.16236 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T02:59:11.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.403+0000 7f2490b84b80 1 -- 172.21.15.52:0/2985104786 --> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5572d0763dd0 con 0x5572d081ed50 2024-06-10T02:59:11.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.403+0000 7f247ffff700 1 --2- 172.21.15.52:0/2985104786 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x5572d081ed50 0x5572d0821180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:59:11.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.403+0000 7f247ffff700 1 --2- 172.21.15.52:0/2985104786 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x5572d081ed50 0x5572d0821180 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ab86d07f91cf626 server_cookie=5cfe02dd5ce59283 in_seq=0 out_seq=0 2024-06-10T02:59:11.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.403+0000 7f245f7fe700 10 client.16236 ms_handle_connect on v2:172.21.15.188:6838/310444836 2024-06-10T02:59:11.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.404+0000 7f245f7fe700 1 -- 172.21.15.52:0/2985104786 <== mds.0 v2:172.21.15.188:6838/310444836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5572d0763dd0 con 0x5572d081ed50 2024-06-10T02:59:11.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.404+0000 7f245f7fe700 10 client.16236 handle_command_reply: tid=0 2024-06-10T02:59:11.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.404+0000 7f2490b84b80 1 client.16236 shutdown 2024-06-10T02:59:11.407 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.406+0000 7f2490b84b80 1 -- 172.21.15.52:0/2985104786 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5572d080b900 msgr2=0x5572d080eea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:59:11.407 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.406+0000 7f2490b84b80 1 --2- 172.21.15.52:0/2985104786 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5572d080b900 0x5572d080eea0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f24780062a0 tx=0x7f2478009dc0 comp rx=0 tx=0).stop 2024-06-10T02:59:11.407 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.406+0000 7f2490b84b80 1 -- 172.21.15.52:0/2985104786 shutdown_connections 2024-06-10T02:59:11.407 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.406+0000 7f2490b84b80 1 --2- 172.21.15.52:0/2985104786 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x5572d081ed50 0x5572d0821180 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:11.407 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.406+0000 7f245f7fe700 0 client.16236 ms_handle_reset on v2:172.21.15.188:6838/310444836 2024-06-10T02:59:11.408 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.406+0000 7f2490b84b80 1 --2- 172.21.15.52:0/2985104786 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5572d080b900 0x5572d080eea0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:11.408 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.406+0000 7f2490b84b80 1 --2- 172.21.15.52:0/2985104786 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5572d0805820 0x5572d0811660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:11.408 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.406+0000 7f2490b84b80 1 --2- 172.21.15.52:0/2985104786 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5572d080afe0 0x5572d080e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:11.408 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.406+0000 7f2490b84b80 1 -- 172.21.15.52:0/2985104786 >> 172.21.15.52:0/2985104786 conn(0x5572d0732100 msgr2=0x5572d07ff780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:59:11.408 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.406+0000 7f2490b84b80 1 -- 172.21.15.52:0/2985104786 shutdown_connections 2024-06-10T02:59:11.408 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.406+0000 7f2490b84b80 1 -- 172.21.15.52:0/2985104786 wait complete. 2024-06-10T02:59:11.408 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.406+0000 7f2490b84b80 20 client.16236 trim_cache size 0 max 16384 2024-06-10T02:59:11.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.457+0000 7f2490b84b80 1 Processor -- start 2024-06-10T02:59:11.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.458+0000 7f2490b84b80 1 -- start start 2024-06-10T02:59:11.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.458+0000 7f2490b84b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5572d0805820 0x5572d08111a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:59:11.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.458+0000 7f2490b84b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5572d080afe0 0x5572d07fff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:59:11.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.458+0000 7f2490b84b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5572d080b900 0x5572d0800440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:59:11.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.458+0000 7f2490b84b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f246c00eb90 con 0x5572d080afe0 2024-06-10T02:59:11.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.458+0000 7f2490b84b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f246c010760 con 0x5572d0805820 2024-06-10T02:59:11.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.458+0000 7f2490b84b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f246c00f260 con 0x5572d080b900 2024-06-10T02:59:11.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.458+0000 7f248489e700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5572d080b900 0x5572d0800440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:59:11.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.458+0000 7f248489e700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5572d080b900 0x5572d0800440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.52:54146/0 (socket says 172.21.15.52:54146) 2024-06-10T02:59:11.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.458+0000 7f248489e700 1 -- 172.21.15.52:0/167146773 learned_addr learned my addr 172.21.15.52:0/167146773 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:59:11.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.458+0000 7f247f7fe700 1 --2- 172.21.15.52:0/167146773 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5572d080afe0 0x5572d07fff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:59:11.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.459+0000 7f248489e700 1 -- 172.21.15.52:0/167146773 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5572d0805820 msgr2=0x5572d08111a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:59:11.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.459+0000 7f248489e700 1 --2- 172.21.15.52:0/167146773 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5572d0805820 0x5572d08111a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:11.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.459+0000 7f248489e700 1 -- 172.21.15.52:0/167146773 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5572d080afe0 msgr2=0x5572d07fff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:59:11.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.459+0000 7f248489e700 1 --2- 172.21.15.52:0/167146773 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5572d080afe0 0x5572d07fff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:11.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.459+0000 7f248489e700 1 -- 172.21.15.52:0/167146773 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5572d080cef0 con 0x5572d080b900 2024-06-10T02:59:11.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.459+0000 7f247f7fe700 1 --2- 172.21.15.52:0/167146773 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5572d080afe0 0x5572d07fff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T02:59:11.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.460+0000 7f248489e700 1 --2- 172.21.15.52:0/167146773 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5572d080b900 0x5572d0800440 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f2478006200 tx=0x7f2478022000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fbf5fa05b40f2c52 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:59:11.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.460+0000 7f2465ffb700 1 -- 172.21.15.52:0/167146773 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24780207c0 con 0x5572d080b900 2024-06-10T02:59:11.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.460+0000 7f2465ffb700 1 -- 172.21.15.52:0/167146773 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2474041910 con 0x5572d080b900 2024-06-10T02:59:11.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.460+0000 7f2465ffb700 1 -- 172.21.15.52:0/167146773 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2478005040 con 0x5572d080b900 2024-06-10T02:59:11.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.461+0000 7f2490b84b80 1 -- 172.21.15.52:0/167146773 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5572d080b900 msgr2=0x5572d0800440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:59:11.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.461+0000 7f2490b84b80 1 --2- 172.21.15.52:0/167146773 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5572d080b900 0x5572d0800440 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f2478006200 tx=0x7f2478022000 comp rx=0 tx=0).stop 2024-06-10T02:59:11.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.461+0000 7f2490b84b80 1 -- 172.21.15.52:0/167146773 shutdown_connections 2024-06-10T02:59:11.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.461+0000 7f2490b84b80 1 --2- 172.21.15.52:0/167146773 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5572d080b900 0x5572d0800440 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:11.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.461+0000 7f2490b84b80 1 --2- 172.21.15.52:0/167146773 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5572d0805820 0x5572d08111a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:11.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.462+0000 7f2490b84b80 1 --2- 172.21.15.52:0/167146773 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5572d080afe0 0x5572d07fff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:11.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.462+0000 7f2490b84b80 1 -- 172.21.15.52:0/167146773 >> 172.21.15.52:0/167146773 conn(0x5572d0732100 msgr2=0x5572d080e210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:59:11.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.462+0000 7f2490b84b80 1 -- 172.21.15.52:0/167146773 shutdown_connections 2024-06-10T02:59:11.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.462+0000 7f2490b84b80 1 -- 172.21.15.52:0/167146773 wait complete. 2024-06-10T02:59:11.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.463+0000 7f2490b84b80 1 -- start start 2024-06-10T02:59:11.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.463+0000 7f2490b84b80 1 Processor -- start 2024-06-10T02:59:11.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.464+0000 7f2490b84b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5572d0805820 0x5572d08146c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:59:11.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.464+0000 7f2490b84b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5572d080afe0 0x5572d0814c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:59:11.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.464+0000 7f2490b84b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5572d080b900 0x5572d0815140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:59:11.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.464+0000 7f2490b84b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f246c010090 con 0x5572d0805820 2024-06-10T02:59:11.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.464+0000 7f2490b84b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f246c00f9c0 con 0x5572d080afe0 2024-06-10T02:59:11.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.464+0000 7f248489e700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5572d080b900 0x5572d0815140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:59:11.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.464+0000 7f248489e700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5572d080b900 0x5572d0815140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.52:54148/0 (socket says 172.21.15.52:54148) 2024-06-10T02:59:11.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.464+0000 7f248489e700 1 -- 172.21.15.52:0/3721359706 learned_addr learned my addr 172.21.15.52:0/3721359706 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:59:11.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.464+0000 7f2490b84b80 1 -- 172.21.15.52:0/3721359706 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f246c00cb30 con 0x5572d080b900 2024-06-10T02:59:11.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.465+0000 7f247ffff700 1 --2- 172.21.15.52:0/3721359706 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5572d0805820 0x5572d08146c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:59:11.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.465+0000 7f247f7fe700 1 --2- 172.21.15.52:0/3721359706 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5572d080afe0 0x5572d0814c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:59:11.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.465+0000 7f248489e700 1 -- 172.21.15.52:0/3721359706 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5572d080afe0 msgr2=0x5572d0814c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:59:11.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.465+0000 7f248489e700 1 --2- 172.21.15.52:0/3721359706 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5572d080afe0 0x5572d0814c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:11.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.465+0000 7f248489e700 1 -- 172.21.15.52:0/3721359706 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5572d0805820 msgr2=0x5572d08146c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:59:11.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.465+0000 7f248489e700 1 --2- 172.21.15.52:0/3721359706 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5572d0805820 0x5572d08146c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:11.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.465+0000 7f248489e700 1 -- 172.21.15.52:0/3721359706 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5572d08922d0 con 0x5572d080b900 2024-06-10T02:59:11.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.465+0000 7f2490b84b80 10 client.0 _ll_register_callbacks cb 0x5572d080ddf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T02:59:11.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.465+0000 7f2490b84b80 1 -- 172.21.15.52:0/3721359706 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2474041910 con 0x5572d080b900 2024-06-10T02:59:11.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.465+0000 7f248489e700 1 --2- 172.21.15.52:0/3721359706 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5572d080b900 0x5572d0815140 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f2478009ce0 tx=0x7f2478007780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f9e8cd5a5e647e3 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:59:11.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.465+0000 7f245f7fe700 10 client.25816 ms_handle_connect on v2:172.21.15.188:3300/0 2024-06-10T02:59:11.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.466+0000 7f245f7fe700 1 -- 172.21.15.52:0/3721359706 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2478020e60 con 0x5572d080b900 2024-06-10T02:59:11.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.466+0000 7f245f7fe700 1 -- 172.21.15.52:0/3721359706 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f2478007cf0 con 0x5572d080b900 2024-06-10T02:59:11.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.466+0000 7f245f7fe700 1 -- 172.21.15.52:0/3721359706 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f2478020170 con 0x5572d080b900 2024-06-10T02:59:11.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.466+0000 7f2490b84b80 10 client.25816 fetch_fsmap learned FSMap version 35 2024-06-10T02:59:11.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.466+0000 7f2490b84b80 1 -- 172.21.15.52:0/3721359706 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({fsmap=35}) -- 0x5572d08933b0 con 0x5572d080b900 2024-06-10T02:59:11.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.467+0000 7f245f7fe700 1 -- 172.21.15.52:0/3721359706 <== mon.1 v2:172.21.15.188:3300/0 4 ==== fsmap(e 35) ==== 8374+0+0 (secure 0 0 0) 0x7f2478011040 con 0x5572d080b900 2024-06-10T02:59:11.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.467+0000 7f2490b84b80 10 client.25816 fetch_fsmap finished waiting for FSMap version 35 2024-06-10T02:59:11.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.467+0000 7f2490b84b80 10 client.25816 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T02:59:11.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.467+0000 7f2490b84b80 20 client.25816 populate_metadata read hostname 'smithi052' 2024-06-10T02:59:11.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.467+0000 7f2490b84b80 1 --2- 172.21.15.52:0/3721359706 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x5572d081ed50 0x5572d0895940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:59:11.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.467+0000 7f2490b84b80 4 client.25816 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-06-10T02:59:11.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.468+0000 7f2490b84b80 1 -- 172.21.15.52:0/3721359706 --> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5572d075e350 con 0x5572d081ed50 2024-06-10T02:59:11.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.468+0000 7f247ffff700 1 --2- 172.21.15.52:0/3721359706 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x5572d081ed50 0x5572d0895940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:59:11.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.468+0000 7f247ffff700 1 --2- 172.21.15.52:0/3721359706 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x5572d081ed50 0x5572d0895940 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ba2098487b74473 server_cookie=21470c55f9d9f6a5 in_seq=0 out_seq=0 2024-06-10T02:59:11.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.468+0000 7f245f7fe700 10 client.25816 ms_handle_connect on v2:172.21.15.188:6838/310444836 2024-06-10T02:59:11.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.469+0000 7f245f7fe700 1 -- 172.21.15.52:0/3721359706 <== mds.0 v2:172.21.15.188:6838/310444836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5572d075e350 con 0x5572d081ed50 2024-06-10T02:59:11.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.469+0000 7f245f7fe700 10 client.25816 handle_command_reply: tid=0 2024-06-10T02:59:11.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.469+0000 7f2490b84b80 1 client.25816 shutdown 2024-06-10T02:59:11.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.470+0000 7f2490b84b80 1 -- 172.21.15.52:0/3721359706 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5572d080b900 msgr2=0x5572d0815140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:59:11.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.470+0000 7f2490b84b80 1 --2- 172.21.15.52:0/3721359706 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5572d080b900 0x5572d0815140 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f2478009ce0 tx=0x7f2478007780 comp rx=0 tx=0).stop 2024-06-10T02:59:11.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.470+0000 7f2490b84b80 1 -- 172.21.15.52:0/3721359706 shutdown_connections 2024-06-10T02:59:11.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.470+0000 7f2490b84b80 1 --2- 172.21.15.52:0/3721359706 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x5572d081ed50 0x5572d0895940 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:11.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.470+0000 7f2490b84b80 1 --2- 172.21.15.52:0/3721359706 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5572d080b900 0x5572d0815140 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:11.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.470+0000 7f2490b84b80 1 --2- 172.21.15.52:0/3721359706 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5572d0805820 0x5572d08146c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:11.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.470+0000 7f2490b84b80 1 --2- 172.21.15.52:0/3721359706 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5572d080afe0 0x5572d0814c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:11.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.470+0000 7f2490b84b80 1 -- 172.21.15.52:0/3721359706 >> 172.21.15.52:0/3721359706 conn(0x5572d0732100 msgr2=0x5572d080eaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:59:11.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.470+0000 7f245f7fe700 0 client.25816 ms_handle_reset on v2:172.21.15.188:6838/310444836 2024-06-10T02:59:11.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.471+0000 7f2490b84b80 1 -- 172.21.15.52:0/3721359706 shutdown_connections 2024-06-10T02:59:11.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.471+0000 7f2490b84b80 1 -- 172.21.15.52:0/3721359706 wait complete. 2024-06-10T02:59:11.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.471+0000 7f2490b84b80 20 client.25816 trim_cache size 0 max 16384 2024-06-10T02:59:11.471 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-10T02:59:11.472 INFO:teuthology.orchestra.run.smithi052.stdout: "return_code": 0, 2024-06-10T02:59:11.472 INFO:teuthology.orchestra.run.smithi052.stdout: "scrub_tag": "4f014e98-4252-4ddf-9b48-2b8c82701461", 2024-06-10T02:59:11.472 INFO:teuthology.orchestra.run.smithi052.stdout: "mode": "asynchronous" 2024-06-10T02:59:11.472 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-10T02:59:11.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.472+0000 7f2486301700 1 -- 172.21.15.52:0/3412995014 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f24680a2ca0 msgr2=0x7f24680a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:59:11.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.472+0000 7f2486301700 1 --2- 172.21.15.52:0/3412995014 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f24680a2ca0 0x7f24680a5160 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f2480078800 tx=0x7f2474009260 comp rx=0 tx=0).stop 2024-06-10T02:59:11.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.472+0000 7f2486301700 1 -- 172.21.15.52:0/3412995014 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2480101380 msgr2=0x7f248019e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:59:11.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.472+0000 7f2486301700 1 --2- 172.21.15.52:0/3412995014 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2480101380 0x7f248019e7f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f247001f9b0 tx=0x7f247001fd70 comp rx=0 tx=0).stop 2024-06-10T02:59:11.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.473+0000 7f2486301700 1 -- 172.21.15.52:0/3412995014 shutdown_connections 2024-06-10T02:59:11.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.473+0000 7f2486301700 1 --2- 172.21.15.52:0/3412995014 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f24801073f0 0x7f24801989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:11.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.473+0000 7f2486301700 1 --2- 172.21.15.52:0/3412995014 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2480101380 0x7f248019e7f0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:11.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.473+0000 7f2486301700 1 --2- 172.21.15.52:0/3412995014 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f24680a2ca0 0x7f24680a5160 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:11.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.473+0000 7f2486301700 1 --2- 172.21.15.52:0/3412995014 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2480101d40 0x7f248019ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:11.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.473+0000 7f2486301700 1 -- 172.21.15.52:0/3412995014 >> 172.21.15.52:0/3412995014 conn(0x7f2480078300 msgr2=0x7f24800ffdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:59:11.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.473+0000 7f2486301700 1 -- 172.21.15.52:0/3412995014 shutdown_connections 2024-06-10T02:59:11.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:11.473+0000 7f2486301700 1 -- 172.21.15.52:0/3412995014 wait complete. 2024-06-10T02:59:12.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:11 smithi120 ceph-mon[36634]: scrub summary: active paths [/] 2024-06-10T02:59:12.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:11 smithi188 ceph-mon[36599]: scrub summary: active paths [/] 2024-06-10T02:59:12.226 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:11 smithi052 ceph-mon[32362]: scrub summary: active paths [/] 2024-06-10T02:59:12.501 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-10T02:59:12.778 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:59:13.118 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:12 smithi052 ceph-mon[32362]: pgmap v584: 129 pgs: 129 active+clean; 483 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 599 KiB/s wr, 144 op/s 2024-06-10T02:59:13.118 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:12 smithi052 ceph-mon[32362]: scrub queued for path: / 2024-06-10T02:59:13.118 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:12 smithi052 ceph-mon[32362]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:13.118 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:12 smithi052 ceph-mon[32362]: scrub summary: active paths [/] 2024-06-10T02:59:13.118 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:12 smithi052 ceph-mon[32362]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:13.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:12 smithi120 ceph-mon[36634]: pgmap v584: 129 pgs: 129 active+clean; 483 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 599 KiB/s wr, 144 op/s 2024-06-10T02:59:13.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:12 smithi120 ceph-mon[36634]: scrub queued for path: / 2024-06-10T02:59:13.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:12 smithi120 ceph-mon[36634]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:13.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:12 smithi120 ceph-mon[36634]: scrub summary: active paths [/] 2024-06-10T02:59:13.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:12 smithi120 ceph-mon[36634]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:13.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:12 smithi188 ceph-mon[36599]: pgmap v584: 129 pgs: 129 active+clean; 483 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 599 KiB/s wr, 144 op/s 2024-06-10T02:59:13.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:12 smithi188 ceph-mon[36599]: scrub queued for path: / 2024-06-10T02:59:13.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:12 smithi188 ceph-mon[36599]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:13.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:12 smithi188 ceph-mon[36599]: scrub summary: active paths [/] 2024-06-10T02:59:13.204 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:12 smithi188 ceph-mon[36599]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:14.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.473+0000 7fbe15848700 1 -- 172.21.15.52:0/3777898964 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbe101027b0 msgr2=0x7fbe10102c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:59:14.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.473+0000 7fbe15848700 1 --2- 172.21.15.52:0/3777898964 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbe101027b0 0x7fbe10102c30 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fbe04009990 tx=0x7fbe04009ca0 comp rx=0 tx=0).stop 2024-06-10T02:59:14.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.473+0000 7fbe15848700 1 -- 172.21.15.52:0/3777898964 shutdown_connections 2024-06-10T02:59:14.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.473+0000 7fbe15848700 1 --2- 172.21.15.52:0/3777898964 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbe10103170 0x7fbe1010f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:14.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.473+0000 7fbe15848700 1 --2- 172.21.15.52:0/3777898964 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbe10108650 0x7fbe10108a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:14.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.473+0000 7fbe15848700 1 --2- 172.21.15.52:0/3777898964 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbe101027b0 0x7fbe10102c30 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:14.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.473+0000 7fbe15848700 1 -- 172.21.15.52:0/3777898964 >> 172.21.15.52:0/3777898964 conn(0x7fbe1007a420 msgr2=0x7fbe1007a830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:59:14.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.473+0000 7fbe15848700 1 -- 172.21.15.52:0/3777898964 shutdown_connections 2024-06-10T02:59:14.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.473+0000 7fbe15848700 1 -- 172.21.15.52:0/3777898964 wait complete. 2024-06-10T02:59:14.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.474+0000 7fbe15848700 1 Processor -- start 2024-06-10T02:59:14.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.474+0000 7fbe15848700 1 -- start start 2024-06-10T02:59:14.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.474+0000 7fbe15848700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbe101027b0 0x7fbe10074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:59:14.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.474+0000 7fbe15848700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbe10103170 0x7fbe10074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:59:14.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.474+0000 7fbe15848700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbe10108650 0x7fbe10075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:59:14.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.474+0000 7fbe15848700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fbe101128d0 con 0x7fbe101027b0 2024-06-10T02:59:14.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.474+0000 7fbe15848700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fbe10112750 con 0x7fbe10108650 2024-06-10T02:59:14.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.474+0000 7fbe15848700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fbe10112a50 con 0x7fbe10103170 2024-06-10T02:59:14.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.474+0000 7fbe0e7fc700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbe10103170 0x7fbe10074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:59:14.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.474+0000 7fbe0effd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbe101027b0 0x7fbe10074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:59:14.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.474+0000 7fbe0e7fc700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbe10103170 0x7fbe10074cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.52:54168/0 (socket says 172.21.15.52:54168) 2024-06-10T02:59:14.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.474+0000 7fbe0e7fc700 1 -- 172.21.15.52:0/3916617370 learned_addr learned my addr 172.21.15.52:0/3916617370 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:59:14.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.474+0000 7fbe0f7fe700 1 --2- 172.21.15.52:0/3916617370 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbe10108650 0x7fbe10075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:59:14.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.475+0000 7fbe0e7fc700 1 -- 172.21.15.52:0/3916617370 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbe10108650 msgr2=0x7fbe10075210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:59:14.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.475+0000 7fbe0e7fc700 1 --2- 172.21.15.52:0/3916617370 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbe10108650 0x7fbe10075210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:14.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.475+0000 7fbe0e7fc700 1 -- 172.21.15.52:0/3916617370 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbe101027b0 msgr2=0x7fbe10074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:59:14.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.475+0000 7fbe0e7fc700 1 --2- 172.21.15.52:0/3916617370 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbe101027b0 0x7fbe10074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:14.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.475+0000 7fbe0e7fc700 1 -- 172.21.15.52:0/3916617370 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe101a7140 con 0x7fbe10103170 2024-06-10T02:59:14.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.475+0000 7fbe0e7fc700 1 --2- 172.21.15.52:0/3916617370 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbe10103170 0x7fbe10074cd0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fbe04005910 tx=0x7fbe0400bab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:59:14.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.475+0000 7fbe14846700 1 -- 172.21.15.52:0/3916617370 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe04027070 con 0x7fbe10103170 2024-06-10T02:59:14.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.475+0000 7fbe14846700 1 -- 172.21.15.52:0/3916617370 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe04003d70 con 0x7fbe10103170 2024-06-10T02:59:14.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.475+0000 7fbe15848700 1 -- 172.21.15.52:0/3916617370 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe101a73d0 con 0x7fbe10103170 2024-06-10T02:59:14.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.475+0000 7fbe14846700 1 -- 172.21.15.52:0/3916617370 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe0400f6c0 con 0x7fbe10103170 2024-06-10T02:59:14.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.475+0000 7fbe15848700 1 -- 172.21.15.52:0/3916617370 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe101a7820 con 0x7fbe10103170 2024-06-10T02:59:14.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.477+0000 7fbe14846700 1 -- 172.21.15.52:0/3916617370 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fbe0402a070 con 0x7fbe10103170 2024-06-10T02:59:14.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.478+0000 7fbe14846700 1 --2- 172.21.15.52:0/3916617370 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fbdf80a2c20 0x7fbdf80a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:59:14.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.478+0000 7fbe0effd700 1 --2- 172.21.15.52:0/3916617370 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fbdf80a2c20 0x7fbdf80a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:59:14.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.478+0000 7fbe14846700 1 -- 172.21.15.52:0/3916617370 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7fbe040d0580 con 0x7fbe10103170 2024-06-10T02:59:14.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.479+0000 7fbe0effd700 1 --2- 172.21.15.52:0/3916617370 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fbdf80a2c20 0x7fbdf80a50e0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fbe00009680 tx=0x7fbe00009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:59:14.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.481+0000 7fbe200cbb80 1 Processor -- start 2024-06-10T02:59:14.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.481+0000 7fbe200cbb80 1 -- start start 2024-06-10T02:59:14.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.481+0000 7fbe200cbb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559c17e7ff60 0x559c17e80340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:59:14.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.481+0000 7fbe200cbb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559c17e80880 0x559c17e7a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:59:14.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.481+0000 7fbe200cbb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559c17e7a7a0 0x559c17e7abc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:59:14.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.481+0000 7fbe200cbb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x559c17d6cf60 con 0x559c17e7a7a0 2024-06-10T02:59:14.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.481+0000 7fbe200cbb80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x559c17dcb850 con 0x559c17e7ff60 2024-06-10T02:59:14.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.481+0000 7fbe200cbb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x559c17d33460 con 0x559c17e80880 2024-06-10T02:59:14.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.482+0000 7fbe0f7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559c17e7a7a0 0x559c17e7abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:59:14.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.482+0000 7fbe0e7fc700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559c17e80880 0x559c17e7a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:59:14.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.482+0000 7fbe0f7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559c17e7a7a0 0x559c17e7abc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57802/0 (socket says 172.21.15.52:57802) 2024-06-10T02:59:14.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.482+0000 7fbe0effd700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559c17e7ff60 0x559c17e80340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:59:14.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.482+0000 7fbe0f7fe700 1 -- 172.21.15.52:0/3310353889 learned_addr learned my addr 172.21.15.52:0/3310353889 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:59:14.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.482+0000 7fbe0f7fe700 1 -- 172.21.15.52:0/3310353889 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559c17e80880 msgr2=0x559c17e7a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:59:14.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.482+0000 7fbe0f7fe700 1 --2- 172.21.15.52:0/3310353889 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559c17e80880 0x559c17e7a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:14.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.482+0000 7fbe0f7fe700 1 -- 172.21.15.52:0/3310353889 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559c17e7ff60 msgr2=0x559c17e80340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:59:14.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.482+0000 7fbe0f7fe700 1 --2- 172.21.15.52:0/3310353889 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559c17e7ff60 0x559c17e80340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:14.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.482+0000 7fbe0f7fe700 1 -- 172.21.15.52:0/3310353889 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c17d1a8d0 con 0x559c17e7a7a0 2024-06-10T02:59:14.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.482+0000 7fbe0f7fe700 1 --2- 172.21.15.52:0/3310353889 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559c17e7a7a0 0x559c17e7abc0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fbe0800e160 tx=0x7fbe0800e470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18a05b1dacfb0b96 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:59:14.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.482+0000 7fbdf5ffb700 1 -- 172.21.15.52:0/3310353889 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe0800ed00 con 0x559c17e7a7a0 2024-06-10T02:59:14.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.482+0000 7fbdf5ffb700 1 -- 172.21.15.52:0/3310353889 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe08004bb0 con 0x559c17e7a7a0 2024-06-10T02:59:14.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.482+0000 7fbdf5ffb700 1 -- 172.21.15.52:0/3310353889 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe08014e40 con 0x559c17e7a7a0 2024-06-10T02:59:14.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.483+0000 7fbe200cbb80 1 -- 172.21.15.52:0/3310353889 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559c17e7a7a0 msgr2=0x559c17e7abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:59:14.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.483+0000 7fbe200cbb80 1 --2- 172.21.15.52:0/3310353889 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559c17e7a7a0 0x559c17e7abc0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fbe0800e160 tx=0x7fbe0800e470 comp rx=0 tx=0).stop 2024-06-10T02:59:14.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.483+0000 7fbe200cbb80 1 -- 172.21.15.52:0/3310353889 shutdown_connections 2024-06-10T02:59:14.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.483+0000 7fbe200cbb80 1 --2- 172.21.15.52:0/3310353889 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559c17e7a7a0 0x559c17e7abc0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:14.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.483+0000 7fbe200cbb80 1 --2- 172.21.15.52:0/3310353889 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559c17e7ff60 0x559c17e80340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:14.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.483+0000 7fbe200cbb80 1 --2- 172.21.15.52:0/3310353889 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559c17e80880 0x559c17e7a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:14.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.483+0000 7fbe200cbb80 1 -- 172.21.15.52:0/3310353889 >> 172.21.15.52:0/3310353889 conn(0x559c17da7080 msgr2=0x559c17e75220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:59:14.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.483+0000 7fbe200cbb80 1 -- 172.21.15.52:0/3310353889 shutdown_connections 2024-06-10T02:59:14.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.483+0000 7fbe200cbb80 1 -- 172.21.15.52:0/3310353889 wait complete. 2024-06-10T02:59:14.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.484+0000 7fbe200cbb80 1 -- start start 2024-06-10T02:59:14.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.484+0000 7fbe200cbb80 1 Processor -- start 2024-06-10T02:59:14.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.484+0000 7fbe200cbb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559c17e7a7a0 0x559c17e86570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:59:14.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.484+0000 7fbe200cbb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559c17e7ff60 0x559c17e83870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:59:14.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.484+0000 7fbe200cbb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559c17e80880 0x559c17e83db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:59:14.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.484+0000 7fbe200cbb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x559c17d6bfc0 con 0x559c17e80880 2024-06-10T02:59:14.486 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.484+0000 7fbe200cbb80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x559c17dc7810 con 0x559c17e7ff60 2024-06-10T02:59:14.486 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.484+0000 7fbe200cbb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x559c17e87260 con 0x559c17e7a7a0 2024-06-10T02:59:14.486 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.485+0000 7fbe200cbb80 10 client.0 _ll_register_callbacks cb 0x559c17d615b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T02:59:14.486 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.485+0000 7fbe0f7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559c17e80880 0x559c17e83db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:59:14.486 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.485+0000 7fbe0f7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559c17e80880 0x559c17e83db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57808/0 (socket says 172.21.15.52:57808) 2024-06-10T02:59:14.486 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.485+0000 7fbe0effd700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559c17e7a7a0 0x559c17e86570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:59:14.486 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.485+0000 7fbe0f7fe700 1 -- 172.21.15.52:0/1836109201 learned_addr learned my addr 172.21.15.52:0/1836109201 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:59:14.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.485+0000 7fbe0f7fe700 1 -- 172.21.15.52:0/1836109201 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559c17e7a7a0 msgr2=0x559c17e86570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:59:14.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.485+0000 7fbe0f7fe700 1 --2- 172.21.15.52:0/1836109201 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559c17e7a7a0 0x559c17e86570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:14.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.485+0000 7fbe0f7fe700 1 -- 172.21.15.52:0/1836109201 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559c17e7ff60 msgr2=0x559c17e83870 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-10T02:59:14.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.485+0000 7fbe0f7fe700 1 --2- 172.21.15.52:0/1836109201 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559c17e7ff60 0x559c17e83870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:14.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.485+0000 7fbe0f7fe700 1 -- 172.21.15.52:0/1836109201 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c17d91750 con 0x559c17e80880 2024-06-10T02:59:14.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.485+0000 7fbe0effd700 1 --2- 172.21.15.52:0/1836109201 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559c17e7a7a0 0x559c17e86570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T02:59:14.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.485+0000 7fbe0f7fe700 1 --2- 172.21.15.52:0/1836109201 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559c17e80880 0x559c17e83db0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fbe08007740 tx=0x7fbe08007c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdb68524dce7cf32 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:59:14.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.485+0000 7fbdeeffd700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-10T02:59:14.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.485+0000 7fbdeeffd700 1 -- 172.21.15.52:0/1836109201 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe08022030 con 0x559c17e80880 2024-06-10T02:59:14.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.486+0000 7fbdeeffd700 1 -- 172.21.15.52:0/1836109201 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7fbe08010070 con 0x559c17e80880 2024-06-10T02:59:14.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.486+0000 7fbe200cbb80 1 -- 172.21.15.52:0/1836109201 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbe08004bb0 con 0x559c17e80880 2024-06-10T02:59:14.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.486+0000 7fbdeeffd700 1 -- 172.21.15.52:0/1836109201 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fbe0802f050 con 0x559c17e80880 2024-06-10T02:59:14.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.486+0000 7fbe200cbb80 10 client.16254 fetch_fsmap learned FSMap version 35 2024-06-10T02:59:14.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.486+0000 7fbe200cbb80 1 -- 172.21.15.52:0/1836109201 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x559c17e91970 con 0x559c17e80880 2024-06-10T02:59:14.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.486+0000 7fbdeeffd700 1 -- 172.21.15.52:0/1836109201 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe08003a00 con 0x559c17e80880 2024-06-10T02:59:14.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.486+0000 7fbdeeffd700 1 -- 172.21.15.52:0/1836109201 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 35) ==== 8374+0+0 (secure 0 0 0) 0x7fbe08004380 con 0x559c17e80880 2024-06-10T02:59:14.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.486+0000 7fbe200cbb80 10 client.16254 fetch_fsmap finished waiting for FSMap version 35 2024-06-10T02:59:14.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.486+0000 7fbe200cbb80 10 client.16254 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T02:59:14.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.486+0000 7fbe200cbb80 20 client.16254 populate_metadata read hostname 'smithi052' 2024-06-10T02:59:14.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.486+0000 7fbe200cbb80 1 --2- 172.21.15.52:0/1836109201 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x559c17e93e40 0x559c17e96230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:59:14.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.486+0000 7fbe200cbb80 4 client.16254 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T02:59:14.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.486+0000 7fbe200cbb80 1 -- 172.21.15.52:0/1836109201 --> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fbe08004bb0 con 0x559c17e93e40 2024-06-10T02:59:14.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.487+0000 7fbe0effd700 1 --2- 172.21.15.52:0/1836109201 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x559c17e93e40 0x559c17e96230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:59:14.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.487+0000 7fbdeeffd700 10 client.16254 ms_handle_connect on v2:172.21.15.188:6838/310444836 2024-06-10T02:59:14.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.487+0000 7fbe0effd700 1 --2- 172.21.15.52:0/1836109201 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x559c17e93e40 0x559c17e96230 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=86244e43da8d7f26 server_cookie=cc7e1d10a5573a84 in_seq=0 out_seq=0 2024-06-10T02:59:14.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.488+0000 7fbdeeffd700 1 -- 172.21.15.52:0/1836109201 <== mds.0 v2:172.21.15.188:6838/310444836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x7fbe08004bb0 con 0x559c17e93e40 2024-06-10T02:59:14.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.488+0000 7fbdeeffd700 10 client.16254 handle_command_reply: tid=0 2024-06-10T02:59:14.491 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.488+0000 7fbe200cbb80 1 client.16254 shutdown 2024-06-10T02:59:14.491 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.488+0000 7fbe200cbb80 1 -- 172.21.15.52:0/1836109201 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559c17e80880 msgr2=0x559c17e83db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:59:14.491 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.488+0000 7fbe200cbb80 1 --2- 172.21.15.52:0/1836109201 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559c17e80880 0x559c17e83db0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fbe08007740 tx=0x7fbe08007c90 comp rx=0 tx=0).stop 2024-06-10T02:59:14.491 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.488+0000 7fbe200cbb80 1 -- 172.21.15.52:0/1836109201 shutdown_connections 2024-06-10T02:59:14.491 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.488+0000 7fbe200cbb80 1 --2- 172.21.15.52:0/1836109201 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x559c17e93e40 0x559c17e96230 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:14.491 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.488+0000 7fbdeeffd700 0 client.16254 ms_handle_reset on v2:172.21.15.188:6838/310444836 2024-06-10T02:59:14.491 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.488+0000 7fbe200cbb80 1 --2- 172.21.15.52:0/1836109201 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559c17e80880 0x559c17e83db0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:14.491 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.488+0000 7fbe200cbb80 1 --2- 172.21.15.52:0/1836109201 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559c17e7a7a0 0x559c17e86570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:14.491 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.488+0000 7fbe200cbb80 1 --2- 172.21.15.52:0/1836109201 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559c17e7ff60 0x559c17e83870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:14.491 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.488+0000 7fbe200cbb80 1 -- 172.21.15.52:0/1836109201 >> 172.21.15.52:0/1836109201 conn(0x559c17da7080 msgr2=0x559c17e74700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:59:14.491 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.488+0000 7fbe200cbb80 1 -- 172.21.15.52:0/1836109201 shutdown_connections 2024-06-10T02:59:14.491 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.488+0000 7fbe200cbb80 1 -- 172.21.15.52:0/1836109201 wait complete. 2024-06-10T02:59:14.491 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.488+0000 7fbe200cbb80 20 client.16254 trim_cache size 0 max 16384 2024-06-10T02:59:14.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.510+0000 7fbe200cbb80 1 Processor -- start 2024-06-10T02:59:14.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.510+0000 7fbe200cbb80 1 -- start start 2024-06-10T02:59:14.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.510+0000 7fbe200cbb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559c17e7a7a0 0x559c17e7a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:59:14.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.510+0000 7fbe200cbb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559c17e7ff60 0x559c17e83ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:59:14.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.510+0000 7fbe200cbb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559c17e80880 0x559c17e857b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:59:14.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.510+0000 7fbe200cbb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fbe101b0450 con 0x559c17e7ff60 2024-06-10T02:59:14.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.510+0000 7fbe200cbb80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fbe101b2020 con 0x559c17e7a7a0 2024-06-10T02:59:14.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.510+0000 7fbe200cbb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fbe101b0b20 con 0x559c17e80880 2024-06-10T02:59:14.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.511+0000 7fbe0e7fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559c17e7ff60 0x559c17e83ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:59:14.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.511+0000 7fbe0e7fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559c17e7ff60 0x559c17e83ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57812/0 (socket says 172.21.15.52:57812) 2024-06-10T02:59:14.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.511+0000 7fbe0e7fc700 1 -- 172.21.15.52:0/2400098111 learned_addr learned my addr 172.21.15.52:0/2400098111 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:59:14.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.511+0000 7fbe0f7fe700 1 --2- 172.21.15.52:0/2400098111 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559c17e80880 0x559c17e857b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:59:14.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.511+0000 7fbe0effd700 1 --2- 172.21.15.52:0/2400098111 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559c17e7a7a0 0x559c17e7a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:59:14.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.511+0000 7fbe0e7fc700 1 -- 172.21.15.52:0/2400098111 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559c17e80880 msgr2=0x559c17e857b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:59:14.513 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.511+0000 7fbe0e7fc700 1 --2- 172.21.15.52:0/2400098111 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559c17e80880 0x559c17e857b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:14.513 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.511+0000 7fbe0e7fc700 1 -- 172.21.15.52:0/2400098111 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559c17e7a7a0 msgr2=0x559c17e7a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:59:14.513 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.511+0000 7fbe0e7fc700 1 --2- 172.21.15.52:0/2400098111 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559c17e7a7a0 0x559c17e7a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:14.513 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.511+0000 7fbe0e7fc700 1 -- 172.21.15.52:0/2400098111 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c17e835e0 con 0x559c17e7ff60 2024-06-10T02:59:14.513 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.511+0000 7fbe0f7fe700 1 --2- 172.21.15.52:0/2400098111 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559c17e80880 0x559c17e857b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T02:59:14.513 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.511+0000 7fbe0e7fc700 1 --2- 172.21.15.52:0/2400098111 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559c17e7ff60 0x559c17e83ea0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fbe04000c00 tx=0x7fbe040cdb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b72d5920a7ba6fa0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:59:14.513 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.511+0000 7fbdf57fa700 1 -- 172.21.15.52:0/2400098111 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe040b5070 con 0x559c17e7ff60 2024-06-10T02:59:14.513 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.512+0000 7fbdf57fa700 1 -- 172.21.15.52:0/2400098111 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe040043d0 con 0x559c17e7ff60 2024-06-10T02:59:14.513 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.512+0000 7fbdf57fa700 1 -- 172.21.15.52:0/2400098111 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe0409adf0 con 0x559c17e7ff60 2024-06-10T02:59:14.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.512+0000 7fbe200cbb80 1 -- 172.21.15.52:0/2400098111 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559c17e7ff60 msgr2=0x559c17e83ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:59:14.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.512+0000 7fbe200cbb80 1 --2- 172.21.15.52:0/2400098111 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559c17e7ff60 0x559c17e83ea0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fbe04000c00 tx=0x7fbe040cdb40 comp rx=0 tx=0).stop 2024-06-10T02:59:14.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.512+0000 7fbe200cbb80 1 -- 172.21.15.52:0/2400098111 shutdown_connections 2024-06-10T02:59:14.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.512+0000 7fbe200cbb80 1 --2- 172.21.15.52:0/2400098111 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559c17e80880 0x559c17e857b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:14.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.512+0000 7fbe200cbb80 1 --2- 172.21.15.52:0/2400098111 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559c17e7a7a0 0x559c17e7a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:14.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.512+0000 7fbe200cbb80 1 --2- 172.21.15.52:0/2400098111 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559c17e7ff60 0x559c17e83ea0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:14.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.512+0000 7fbe200cbb80 1 -- 172.21.15.52:0/2400098111 >> 172.21.15.52:0/2400098111 conn(0x559c17da7080 msgr2=0x559c17e850a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:59:14.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.512+0000 7fbe200cbb80 1 -- 172.21.15.52:0/2400098111 shutdown_connections 2024-06-10T02:59:14.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.512+0000 7fbe200cbb80 1 -- 172.21.15.52:0/2400098111 wait complete. 2024-06-10T02:59:14.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.513+0000 7fbe200cbb80 1 -- start start 2024-06-10T02:59:14.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.513+0000 7fbe200cbb80 1 Processor -- start 2024-06-10T02:59:14.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.513+0000 7fbe200cbb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559c17e7a7a0 0x559c17e8ce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:59:14.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.513+0000 7fbe200cbb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559c17e80880 0x559c17e9b260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:59:14.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.513+0000 7fbe200cbb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559c17e9b900 0x559c17efd7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:59:14.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.513+0000 7fbe200cbb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fbe101b1950 con 0x559c17e9b900 2024-06-10T02:59:14.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.513+0000 7fbe200cbb80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fbe101b1280 con 0x559c17e7a7a0 2024-06-10T02:59:14.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.513+0000 7fbe200cbb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fbe101ae3f0 con 0x559c17e80880 2024-06-10T02:59:14.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.513+0000 7fbe200cbb80 10 client.0 _ll_register_callbacks cb 0x559c17d615b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T02:59:14.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.514+0000 7fbe0f7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559c17e9b900 0x559c17efd7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:59:14.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.514+0000 7fbe0f7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559c17e9b900 0x559c17efd7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57822/0 (socket says 172.21.15.52:57822) 2024-06-10T02:59:14.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.514+0000 7fbe0f7fe700 1 -- 172.21.15.52:0/274374944 learned_addr learned my addr 172.21.15.52:0/274374944 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:59:14.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.514+0000 7fbe0e7fc700 1 --2- 172.21.15.52:0/274374944 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559c17e80880 0x559c17e9b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:59:14.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.514+0000 7fbe0effd700 1 --2- 172.21.15.52:0/274374944 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559c17e7a7a0 0x559c17e8ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:59:14.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.514+0000 7fbe0f7fe700 1 -- 172.21.15.52:0/274374944 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559c17e80880 msgr2=0x559c17e9b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:59:14.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.514+0000 7fbe0f7fe700 1 --2- 172.21.15.52:0/274374944 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559c17e80880 0x559c17e9b260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:14.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.514+0000 7fbe0f7fe700 1 -- 172.21.15.52:0/274374944 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559c17e7a7a0 msgr2=0x559c17e8ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:59:14.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.514+0000 7fbe0f7fe700 1 --2- 172.21.15.52:0/274374944 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559c17e7a7a0 0x559c17e8ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:14.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.514+0000 7fbe0f7fe700 1 -- 172.21.15.52:0/274374944 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c17e9c050 con 0x559c17e9b900 2024-06-10T02:59:14.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.514+0000 7fbe0f7fe700 1 --2- 172.21.15.52:0/274374944 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559c17e9b900 0x559c17efd7d0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fbe08004780 tx=0x7fbe08007570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27d305eb7f07e4b9 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:59:14.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.514+0000 7fbdeeffd700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-10T02:59:14.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.515+0000 7fbdeeffd700 1 -- 172.21.15.52:0/274374944 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe08022040 con 0x559c17e9b900 2024-06-10T02:59:14.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.515+0000 7fbdeeffd700 1 -- 172.21.15.52:0/274374944 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7fbe08019040 con 0x559c17e9b900 2024-06-10T02:59:14.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.515+0000 7fbe200cbb80 1 -- 172.21.15.52:0/274374944 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbe040043d0 con 0x559c17e9b900 2024-06-10T02:59:14.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.515+0000 7fbdeeffd700 1 -- 172.21.15.52:0/274374944 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fbe0800e7c0 con 0x559c17e9b900 2024-06-10T02:59:14.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.515+0000 7fbe200cbb80 10 client.16260 fetch_fsmap learned FSMap version 35 2024-06-10T02:59:14.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.515+0000 7fbe200cbb80 1 -- 172.21.15.52:0/274374944 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x559c17efed90 con 0x559c17e9b900 2024-06-10T02:59:14.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.516+0000 7fbdeeffd700 1 -- 172.21.15.52:0/274374944 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe08005c60 con 0x559c17e9b900 2024-06-10T02:59:14.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.516+0000 7fbdeeffd700 1 -- 172.21.15.52:0/274374944 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 35) ==== 8374+0+0 (secure 0 0 0) 0x7fbe08014c30 con 0x559c17e9b900 2024-06-10T02:59:14.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.516+0000 7fbe200cbb80 10 client.16260 fetch_fsmap finished waiting for FSMap version 35 2024-06-10T02:59:14.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.516+0000 7fbe200cbb80 10 client.16260 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T02:59:14.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.516+0000 7fbe200cbb80 20 client.16260 populate_metadata read hostname 'smithi052' 2024-06-10T02:59:14.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.516+0000 7fbe200cbb80 1 --2- 172.21.15.52:0/274374944 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x559c17eff370 0x559c17f01790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:59:14.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.516+0000 7fbe200cbb80 4 client.16260 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-10T02:59:14.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.516+0000 7fbe200cbb80 1 -- 172.21.15.52:0/274374944 --> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7fbe040043d0 con 0x559c17eff370 2024-06-10T02:59:14.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.516+0000 7fbe0effd700 1 --2- 172.21.15.52:0/274374944 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x559c17eff370 0x559c17f01790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:59:14.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.517+0000 7fbdeeffd700 10 client.16260 ms_handle_connect on v2:172.21.15.188:6838/310444836 2024-06-10T02:59:14.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.517+0000 7fbe0effd700 1 --2- 172.21.15.52:0/274374944 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x559c17eff370 0x559c17f01790 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee0ebd8b5b44f326 server_cookie=47539f79e8c8d707 in_seq=0 out_seq=0 2024-06-10T02:59:14.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.517+0000 7fbdeeffd700 1 -- 172.21.15.52:0/274374944 <== mds.0 v2:172.21.15.188:6838/310444836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x7fbe040043d0 con 0x559c17eff370 2024-06-10T02:59:14.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.517+0000 7fbdeeffd700 10 client.16260 handle_command_reply: tid=0 2024-06-10T02:59:14.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.517+0000 7fbe200cbb80 1 client.16260 shutdown 2024-06-10T02:59:14.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.517+0000 7fbe200cbb80 1 -- 172.21.15.52:0/274374944 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559c17e9b900 msgr2=0x559c17efd7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:59:14.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.517+0000 7fbe200cbb80 1 --2- 172.21.15.52:0/274374944 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559c17e9b900 0x559c17efd7d0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fbe08004780 tx=0x7fbe08007570 comp rx=0 tx=0).stop 2024-06-10T02:59:14.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.517+0000 7fbe200cbb80 1 -- 172.21.15.52:0/274374944 shutdown_connections 2024-06-10T02:59:14.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.517+0000 7fbe200cbb80 1 --2- 172.21.15.52:0/274374944 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x559c17eff370 0x559c17f01790 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:14.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.517+0000 7fbe200cbb80 1 --2- 172.21.15.52:0/274374944 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559c17e9b900 0x559c17efd7d0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:14.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.517+0000 7fbe200cbb80 1 --2- 172.21.15.52:0/274374944 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559c17e7a7a0 0x559c17e8ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:14.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.517+0000 7fbe200cbb80 1 --2- 172.21.15.52:0/274374944 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559c17e80880 0x559c17e9b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:14.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.517+0000 7fbe200cbb80 1 -- 172.21.15.52:0/274374944 >> 172.21.15.52:0/274374944 conn(0x559c17da7080 msgr2=0x559c17e7e9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:59:14.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.517+0000 7fbdeeffd700 0 client.16260 ms_handle_reset on v2:172.21.15.188:6838/310444836 2024-06-10T02:59:14.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.518+0000 7fbe200cbb80 1 -- 172.21.15.52:0/274374944 shutdown_connections 2024-06-10T02:59:14.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.518+0000 7fbe200cbb80 1 -- 172.21.15.52:0/274374944 wait complete. 2024-06-10T02:59:14.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.518+0000 7fbe200cbb80 20 client.16260 trim_cache size 0 max 16384 2024-06-10T02:59:14.522 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-10T02:59:14.522 INFO:teuthology.orchestra.run.smithi052.stdout: "status": "IDLE (waiting for more scrubs)", 2024-06-10T02:59:14.522 INFO:teuthology.orchestra.run.smithi052.stdout: "scrubs": { 2024-06-10T02:59:14.522 INFO:teuthology.orchestra.run.smithi052.stdout: "4f014e98-4252-4ddf-9b48-2b8c82701461": { 2024-06-10T02:59:14.522 INFO:teuthology.orchestra.run.smithi052.stdout: "path": "/", 2024-06-10T02:59:14.522 INFO:teuthology.orchestra.run.smithi052.stdout: "tag": "4f014e98-4252-4ddf-9b48-2b8c82701461", 2024-06-10T02:59:14.522 INFO:teuthology.orchestra.run.smithi052.stdout: "options": "recursive,force" 2024-06-10T02:59:14.523 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T02:59:14.523 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T02:59:14.523 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-10T02:59:14.523 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.519+0000 7fbe15848700 1 -- 172.21.15.52:0/3916617370 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fbdf80a2c20 msgr2=0x7fbdf80a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:59:14.523 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.519+0000 7fbe15848700 1 --2- 172.21.15.52:0/3916617370 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fbdf80a2c20 0x7fbdf80a50e0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fbe00009680 tx=0x7fbe00009260 comp rx=0 tx=0).stop 2024-06-10T02:59:14.524 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.519+0000 7fbe15848700 1 -- 172.21.15.52:0/3916617370 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbe10103170 msgr2=0x7fbe10074cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:59:14.524 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.519+0000 7fbe15848700 1 --2- 172.21.15.52:0/3916617370 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbe10103170 0x7fbe10074cd0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fbe04005910 tx=0x7fbe0400bab0 comp rx=0 tx=0).stop 2024-06-10T02:59:14.524 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.519+0000 7fbe15848700 1 -- 172.21.15.52:0/3916617370 shutdown_connections 2024-06-10T02:59:14.524 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.519+0000 7fbe15848700 1 --2- 172.21.15.52:0/3916617370 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbe10108650 0x7fbe10075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:14.524 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.519+0000 7fbe15848700 1 --2- 172.21.15.52:0/3916617370 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fbdf80a2c20 0x7fbdf80a50e0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:14.524 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.519+0000 7fbe15848700 1 --2- 172.21.15.52:0/3916617370 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbe101027b0 0x7fbe10074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:14.524 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.519+0000 7fbe15848700 1 --2- 172.21.15.52:0/3916617370 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbe10103170 0x7fbe10074cd0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:14.524 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.519+0000 7fbe15848700 1 -- 172.21.15.52:0/3916617370 >> 172.21.15.52:0/3916617370 conn(0x7fbe1007a420 msgr2=0x7fbe10101220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:59:14.524 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.519+0000 7fbe15848700 1 -- 172.21.15.52:0/3916617370 shutdown_connections 2024-06-10T02:59:14.524 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:14.519+0000 7fbe15848700 1 -- 172.21.15.52:0/3916617370 wait complete. 2024-06-10T02:59:15.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:14 smithi120 ceph-mon[36634]: pgmap v585: 129 pgs: 129 active+clean; 488 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.1 MiB/s wr, 166 op/s 2024-06-10T02:59:15.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:14 smithi188 ceph-mon[36599]: pgmap v585: 129 pgs: 129 active+clean; 488 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.1 MiB/s wr, 166 op/s 2024-06-10T02:59:15.212 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:14 smithi052 ceph-mon[32362]: pgmap v585: 129 pgs: 129 active+clean; 488 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.1 MiB/s wr, 166 op/s 2024-06-10T02:59:15.251 INFO:tasks.cephfs.filesystem:scrub status for tag:4f014e98-4252-4ddf-9b48-2b8c82701461 - {'path': '/', 'tag': '4f014e98-4252-4ddf-9b48-2b8c82701461', 'options': 'recursive,force'} 2024-06-10T02:59:16.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:15 smithi120 ceph-mon[36634]: scrub summary: active paths [/] 2024-06-10T02:59:16.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:15 smithi120 ceph-mon[36634]: scrub summary: active paths [/] 2024-06-10T02:59:16.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:15 smithi120 ceph-mon[36634]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:16.139 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:15 smithi120 ceph-mon[36634]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:16.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:15 smithi188 ceph-mon[36599]: scrub summary: active paths [/] 2024-06-10T02:59:16.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:15 smithi188 ceph-mon[36599]: scrub summary: active paths [/] 2024-06-10T02:59:16.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:15 smithi188 ceph-mon[36599]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:16.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:15 smithi188 ceph-mon[36599]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:16.227 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:15 smithi052 ceph-mon[32362]: scrub summary: active paths [/] 2024-06-10T02:59:16.227 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:15 smithi052 ceph-mon[32362]: scrub summary: active paths [/] 2024-06-10T02:59:16.227 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:15 smithi052 ceph-mon[32362]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:16.227 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:15 smithi052 ceph-mon[32362]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:17.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:16 smithi120 ceph-mon[36634]: pgmap v586: 129 pgs: 129 active+clean; 488 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.0 MiB/s wr, 143 op/s 2024-06-10T02:59:17.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:16 smithi120 ceph-mon[36634]: scrub summary: active paths [/] 2024-06-10T02:59:17.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:16 smithi120 ceph-mon[36634]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:17.139 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:16 smithi120 ceph-mon[36634]: scrub summary: active paths [/] 2024-06-10T02:59:17.139 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:16 smithi120 ceph-mon[36634]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:17.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:16 smithi188 ceph-mon[36599]: pgmap v586: 129 pgs: 129 active+clean; 488 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.0 MiB/s wr, 143 op/s 2024-06-10T02:59:17.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:16 smithi188 ceph-mon[36599]: scrub summary: active paths [/] 2024-06-10T02:59:17.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:16 smithi188 ceph-mon[36599]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:17.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:16 smithi188 ceph-mon[36599]: scrub summary: active paths [/] 2024-06-10T02:59:17.204 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:16 smithi188 ceph-mon[36599]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:17.226 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:16 smithi052 ceph-mon[32362]: pgmap v586: 129 pgs: 129 active+clean; 488 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.0 MiB/s wr, 143 op/s 2024-06-10T02:59:17.227 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:16 smithi052 ceph-mon[32362]: scrub summary: active paths [/] 2024-06-10T02:59:17.227 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:16 smithi052 ceph-mon[32362]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:17.227 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:16 smithi052 ceph-mon[32362]: scrub summary: active paths [/] 2024-06-10T02:59:17.227 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:16 smithi052 ceph-mon[32362]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:17.246 INFO:tasks.workunit.client.0.smithi052.stdout: 82 1141740 484 815387 347 614535 1151 2024-06-10T02:59:19.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:18 smithi120 ceph-mon[36634]: pgmap v587: 129 pgs: 129 active+clean; 489 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.2 MiB/s wr, 170 op/s 2024-06-10T02:59:19.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:18 smithi188 ceph-mon[36599]: pgmap v587: 129 pgs: 129 active+clean; 489 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.2 MiB/s wr, 170 op/s 2024-06-10T02:59:19.226 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:18 smithi052 ceph-mon[32362]: pgmap v587: 129 pgs: 129 active+clean; 489 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.2 MiB/s wr, 170 op/s 2024-06-10T02:59:21.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:20 smithi120 ceph-mon[36634]: pgmap v588: 129 pgs: 129 active+clean; 503 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.2 MiB/s wr, 249 op/s 2024-06-10T02:59:21.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:20 smithi188 ceph-mon[36599]: pgmap v588: 129 pgs: 129 active+clean; 503 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.2 MiB/s wr, 249 op/s 2024-06-10T02:59:21.227 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:20 smithi052 ceph-mon[32362]: pgmap v588: 129 pgs: 129 active+clean; 503 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.2 MiB/s wr, 249 op/s 2024-06-10T02:59:23.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:22 smithi120 ceph-mon[36634]: pgmap v589: 129 pgs: 129 active+clean; 514 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.2 MiB/s wr, 307 op/s 2024-06-10T02:59:23.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:22 smithi188 ceph-mon[36599]: pgmap v589: 129 pgs: 129 active+clean; 514 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.2 MiB/s wr, 307 op/s 2024-06-10T02:59:23.226 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:22 smithi052 ceph-mon[32362]: pgmap v589: 129 pgs: 129 active+clean; 514 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.2 MiB/s wr, 307 op/s 2024-06-10T02:59:25.139 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:24 smithi120 ceph-mon[36634]: pgmap v590: 129 pgs: 129 active+clean; 529 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.6 MiB/s wr, 535 op/s 2024-06-10T02:59:25.139 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:24 smithi120 ceph-mon[36634]: scrub summary: active paths [/] 2024-06-10T02:59:25.139 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:24 smithi120 ceph-mon[36634]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.139 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:24 smithi120 ceph-mon[36634]: scrub summary: active paths [/] 2024-06-10T02:59:25.140 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:24 smithi120 ceph-mon[36634]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.140 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:24 smithi120 ceph-mon[36634]: scrub summary: active paths [/] 2024-06-10T02:59:25.140 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:24 smithi120 ceph-mon[36634]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.140 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:24 smithi120 ceph-mon[36634]: scrub summary: active paths [/] 2024-06-10T02:59:25.140 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:24 smithi120 ceph-mon[36634]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.140 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:24 smithi120 ceph-mon[36634]: scrub summary: active paths [/] 2024-06-10T02:59:25.140 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:24 smithi120 ceph-mon[36634]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.140 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:24 smithi120 ceph-mon[36634]: scrub summary: active paths [/] 2024-06-10T02:59:25.140 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:24 smithi120 ceph-mon[36634]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.140 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:24 smithi120 ceph-mon[36634]: scrub summary: active paths [/] 2024-06-10T02:59:25.140 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:24 smithi120 ceph-mon[36634]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.140 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:24 smithi120 ceph-mon[36634]: scrub summary: active paths [/] 2024-06-10T02:59:25.140 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:24 smithi120 ceph-mon[36634]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.140 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:24 smithi120 ceph-mon[36634]: scrub summary: active paths [/] 2024-06-10T02:59:25.141 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:24 smithi120 ceph-mon[36634]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.141 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:24 smithi120 ceph-mon[36634]: scrub summary: active paths [/] 2024-06-10T02:59:25.141 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:24 smithi120 ceph-mon[36634]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.141 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:24 smithi120 ceph-mon[36634]: scrub summary: active paths [/] 2024-06-10T02:59:25.141 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:24 smithi120 ceph-mon[36634]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.141 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:24 smithi120 ceph-mon[36634]: scrub summary: active paths [/] 2024-06-10T02:59:25.141 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:24 smithi120 ceph-mon[36634]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.141 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:24 smithi120 ceph-mon[36634]: scrub summary: active paths [/] 2024-06-10T02:59:25.141 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:24 smithi120 ceph-mon[36634]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.141 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:24 smithi120 ceph-mon[36634]: scrub summary: active paths [/] 2024-06-10T02:59:25.141 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:24 smithi120 ceph-mon[36634]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.141 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:24 smithi120 ceph-mon[36634]: scrub summary: active paths [/] 2024-06-10T02:59:25.141 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:24 smithi120 ceph-mon[36634]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.141 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:24 smithi120 ceph-mon[36634]: scrub summary: active paths [/] 2024-06-10T02:59:25.142 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:24 smithi120 ceph-mon[36634]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.142 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:24 smithi120 ceph-mon[36634]: scrub summary: active paths [/] 2024-06-10T02:59:25.142 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:24 smithi120 ceph-mon[36634]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.142 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:24 smithi120 ceph-mon[36634]: scrub summary: active paths [/] 2024-06-10T02:59:25.142 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:24 smithi120 ceph-mon[36634]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.142 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:24 smithi120 ceph-mon[36634]: scrub summary: active paths [/] 2024-06-10T02:59:25.142 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:24 smithi120 ceph-mon[36634]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.142 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:24 smithi120 ceph-mon[36634]: scrub summary: active paths [/] 2024-06-10T02:59:25.142 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:24 smithi120 ceph-mon[36634]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.142 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:24 smithi120 ceph-mon[36634]: scrub summary: active paths [/] 2024-06-10T02:59:25.142 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:24 smithi120 ceph-mon[36634]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.142 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:24 smithi120 ceph-mon[36634]: scrub summary: active paths [/] 2024-06-10T02:59:25.142 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:24 smithi120 ceph-mon[36634]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.142 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:24 smithi120 ceph-mon[36634]: scrub summary: active paths [/] 2024-06-10T02:59:25.143 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:24 smithi120 ceph-mon[36634]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.143 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:24 smithi120 ceph-mon[36634]: scrub summary: active paths [/] 2024-06-10T02:59:25.143 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:24 smithi120 ceph-mon[36634]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.143 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:24 smithi120 ceph-mon[36634]: scrub summary: active paths [/] 2024-06-10T02:59:25.143 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:24 smithi120 ceph-mon[36634]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.143 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:24 smithi120 ceph-mon[36634]: scrub summary: active paths [/] 2024-06-10T02:59:25.144 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:24 smithi120 ceph-mon[36634]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.144 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:24 smithi120 ceph-mon[36634]: scrub summary: active paths [/] 2024-06-10T02:59:25.144 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:24 smithi120 ceph-mon[36634]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.144 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:24 smithi120 ceph-mon[36634]: scrub summary: active paths [/] 2024-06-10T02:59:25.144 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:24 smithi120 ceph-mon[36634]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.144 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:24 smithi120 ceph-mon[36634]: scrub summary: active paths [/] 2024-06-10T02:59:25.144 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:24 smithi120 ceph-mon[36634]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.144 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:24 smithi120 ceph-mon[36634]: scrub summary: active paths [/] 2024-06-10T02:59:25.144 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:24 smithi120 ceph-mon[36634]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.144 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:24 smithi120 ceph-mon[36634]: scrub summary: active paths [/] 2024-06-10T02:59:25.144 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:24 smithi120 ceph-mon[36634]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.144 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:24 smithi120 ceph-mon[36634]: scrub summary: active paths [/] 2024-06-10T02:59:25.144 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:24 smithi120 ceph-mon[36634]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.145 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:24 smithi120 ceph-mon[36634]: scrub summary: active paths [/] 2024-06-10T02:59:25.145 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:24 smithi120 ceph-mon[36634]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.145 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:24 smithi120 ceph-mon[36634]: scrub summary: active paths [/] 2024-06-10T02:59:25.145 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:24 smithi120 ceph-mon[36634]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.145 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:24 smithi120 ceph-mon[36634]: scrub summary: active paths [/] 2024-06-10T02:59:25.145 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:24 smithi120 ceph-mon[36634]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.145 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:24 smithi120 ceph-mon[36634]: scrub summary: active paths [/] 2024-06-10T02:59:25.145 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:24 smithi120 ceph-mon[36634]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.145 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:24 smithi120 ceph-mon[36634]: scrub summary: active paths [/] 2024-06-10T02:59:25.145 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:24 smithi120 ceph-mon[36634]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.145 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:24 smithi120 ceph-mon[36634]: scrub summary: active paths [/] 2024-06-10T02:59:25.145 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:24 smithi120 ceph-mon[36634]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.145 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:24 smithi120 ceph-mon[36634]: scrub summary: active paths [/] 2024-06-10T02:59:25.146 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:24 smithi120 ceph-mon[36634]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.146 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:24 smithi120 ceph-mon[36634]: scrub summary: active paths [/] 2024-06-10T02:59:25.146 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:24 smithi120 ceph-mon[36634]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.146 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:24 smithi120 ceph-mon[36634]: scrub summary: active paths [/] 2024-06-10T02:59:25.146 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:24 smithi120 ceph-mon[36634]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.146 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:24 smithi120 ceph-mon[36634]: scrub summary: active paths [/] 2024-06-10T02:59:25.146 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:24 smithi120 ceph-mon[36634]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.146 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:24 smithi120 ceph-mon[36634]: scrub summary: active paths [/] 2024-06-10T02:59:25.146 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:24 smithi120 ceph-mon[36634]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.146 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:24 smithi120 ceph-mon[36634]: scrub summary: active paths [/] 2024-06-10T02:59:25.146 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:24 smithi120 ceph-mon[36634]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.146 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:24 smithi120 ceph-mon[36634]: scrub summary: active paths [/] 2024-06-10T02:59:25.146 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:24 smithi120 ceph-mon[36634]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.147 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:24 smithi120 ceph-mon[36634]: scrub summary: active paths [/] 2024-06-10T02:59:25.147 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:24 smithi120 ceph-mon[36634]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.147 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:24 smithi120 ceph-mon[36634]: scrub summary: active paths [/] 2024-06-10T02:59:25.147 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:24 smithi120 ceph-mon[36634]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.147 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:24 smithi120 ceph-mon[36634]: scrub summary: active paths [/] 2024-06-10T02:59:25.147 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:24 smithi120 ceph-mon[36634]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.147 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:24 smithi120 ceph-mon[36634]: scrub summary: active paths [/] 2024-06-10T02:59:25.147 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:24 smithi120 ceph-mon[36634]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.147 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:24 smithi120 ceph-mon[36634]: scrub summary: active paths [/] 2024-06-10T02:59:25.147 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:24 smithi120 ceph-mon[36634]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.147 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:24 smithi120 ceph-mon[36634]: scrub summary: active paths [/] 2024-06-10T02:59:25.147 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:24 smithi120 ceph-mon[36634]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.147 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:24 smithi120 ceph-mon[36634]: scrub summary: active paths [/] 2024-06-10T02:59:25.147 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:24 smithi120 ceph-mon[36634]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.147 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:24 smithi120 ceph-mon[36634]: scrub summary: active paths [/] 2024-06-10T02:59:25.148 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:24 smithi120 ceph-mon[36634]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.148 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:24 smithi120 ceph-mon[36634]: scrub summary: active paths [/] 2024-06-10T02:59:25.148 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:24 smithi120 ceph-mon[36634]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.148 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:24 smithi120 ceph-mon[36634]: scrub summary: active paths [/] 2024-06-10T02:59:25.148 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:24 smithi120 ceph-mon[36634]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.148 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:24 smithi120 ceph-mon[36634]: scrub summary: active paths [/] 2024-06-10T02:59:25.148 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:24 smithi120 ceph-mon[36634]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.148 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:24 smithi120 ceph-mon[36634]: scrub summary: active paths [/] 2024-06-10T02:59:25.148 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:24 smithi120 ceph-mon[36634]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.148 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:24 smithi120 ceph-mon[36634]: scrub summary: active paths [/] 2024-06-10T02:59:25.148 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:24 smithi120 ceph-mon[36634]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.149 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:24 smithi120 ceph-mon[36634]: scrub summary: active paths [/] 2024-06-10T02:59:25.149 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:24 smithi120 ceph-mon[36634]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.149 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:24 smithi120 ceph-mon[36634]: scrub summary: active paths [/] 2024-06-10T02:59:25.149 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:24 smithi120 ceph-mon[36634]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.149 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:24 smithi120 ceph-mon[36634]: scrub summary: active paths [/] 2024-06-10T02:59:25.149 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:24 smithi120 ceph-mon[36634]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.149 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:24 smithi120 ceph-mon[36634]: scrub summary: active paths [/] 2024-06-10T02:59:25.149 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:24 smithi120 ceph-mon[36634]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.149 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:24 smithi120 ceph-mon[36634]: scrub summary: active paths [/] 2024-06-10T02:59:25.149 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:24 smithi120 ceph-mon[36634]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.149 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:24 smithi120 ceph-mon[36634]: scrub summary: active paths [/] 2024-06-10T02:59:25.149 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:24 smithi120 ceph-mon[36634]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.149 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:24 smithi120 ceph-mon[36634]: scrub summary: active paths [/] 2024-06-10T02:59:25.149 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:24 smithi120 ceph-mon[36634]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.149 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:24 smithi120 ceph-mon[36634]: scrub summary: active paths [/] 2024-06-10T02:59:25.150 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:24 smithi120 ceph-mon[36634]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.150 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:24 smithi120 ceph-mon[36634]: scrub summary: active paths [/] 2024-06-10T02:59:25.150 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:24 smithi120 ceph-mon[36634]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.150 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:24 smithi120 ceph-mon[36634]: scrub summary: active paths [/] 2024-06-10T02:59:25.150 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:24 smithi120 ceph-mon[36634]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.150 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:24 smithi120 ceph-mon[36634]: scrub summary: active paths [/] 2024-06-10T02:59:25.150 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:24 smithi120 ceph-mon[36634]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.150 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:24 smithi120 ceph-mon[36634]: scrub summary: active paths [/] 2024-06-10T02:59:25.150 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:24 smithi120 ceph-mon[36634]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.150 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:24 smithi120 ceph-mon[36634]: scrub summary: active paths [/] 2024-06-10T02:59:25.150 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:24 smithi120 ceph-mon[36634]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.150 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:24 smithi120 ceph-mon[36634]: scrub summary: active paths [/] 2024-06-10T02:59:25.150 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:24 smithi120 ceph-mon[36634]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.151 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:24 smithi120 ceph-mon[36634]: scrub summary: active paths [/] 2024-06-10T02:59:25.151 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:24 smithi120 ceph-mon[36634]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.151 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:24 smithi120 ceph-mon[36634]: scrub summary: active paths [/] 2024-06-10T02:59:25.151 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:24 smithi120 ceph-mon[36634]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.206 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:24 smithi188 ceph-mon[36599]: pgmap v590: 129 pgs: 129 active+clean; 529 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.6 MiB/s wr, 535 op/s 2024-06-10T02:59:25.206 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:24 smithi188 ceph-mon[36599]: scrub summary: active paths [/] 2024-06-10T02:59:25.206 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:24 smithi188 ceph-mon[36599]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.206 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:24 smithi188 ceph-mon[36599]: scrub summary: active paths [/] 2024-06-10T02:59:25.206 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:24 smithi188 ceph-mon[36599]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.206 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:24 smithi188 ceph-mon[36599]: scrub summary: active paths [/] 2024-06-10T02:59:25.206 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:24 smithi188 ceph-mon[36599]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.206 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:24 smithi188 ceph-mon[36599]: scrub summary: active paths [/] 2024-06-10T02:59:25.206 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:24 smithi188 ceph-mon[36599]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.206 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:24 smithi188 ceph-mon[36599]: scrub summary: active paths [/] 2024-06-10T02:59:25.207 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:24 smithi188 ceph-mon[36599]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.207 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:24 smithi188 ceph-mon[36599]: scrub summary: active paths [/] 2024-06-10T02:59:25.207 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:24 smithi188 ceph-mon[36599]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.207 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:24 smithi188 ceph-mon[36599]: scrub summary: active paths [/] 2024-06-10T02:59:25.207 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:24 smithi188 ceph-mon[36599]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.207 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:24 smithi188 ceph-mon[36599]: scrub summary: active paths [/] 2024-06-10T02:59:25.207 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:24 smithi188 ceph-mon[36599]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.207 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:24 smithi188 ceph-mon[36599]: scrub summary: active paths [/] 2024-06-10T02:59:25.207 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:24 smithi188 ceph-mon[36599]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.207 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:24 smithi188 ceph-mon[36599]: scrub summary: active paths [/] 2024-06-10T02:59:25.207 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:24 smithi188 ceph-mon[36599]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.207 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:24 smithi188 ceph-mon[36599]: scrub summary: active paths [/] 2024-06-10T02:59:25.207 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:24 smithi188 ceph-mon[36599]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.207 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:24 smithi188 ceph-mon[36599]: scrub summary: active paths [/] 2024-06-10T02:59:25.208 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:24 smithi188 ceph-mon[36599]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.208 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:24 smithi188 ceph-mon[36599]: scrub summary: active paths [/] 2024-06-10T02:59:25.208 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:24 smithi188 ceph-mon[36599]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.208 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:24 smithi188 ceph-mon[36599]: scrub summary: active paths [/] 2024-06-10T02:59:25.208 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:24 smithi188 ceph-mon[36599]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.208 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:24 smithi188 ceph-mon[36599]: scrub summary: active paths [/] 2024-06-10T02:59:25.208 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:24 smithi188 ceph-mon[36599]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.208 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:24 smithi188 ceph-mon[36599]: scrub summary: active paths [/] 2024-06-10T02:59:25.208 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:24 smithi188 ceph-mon[36599]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.208 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:24 smithi188 ceph-mon[36599]: scrub summary: active paths [/] 2024-06-10T02:59:25.208 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:24 smithi188 ceph-mon[36599]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.208 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:24 smithi188 ceph-mon[36599]: scrub summary: active paths [/] 2024-06-10T02:59:25.208 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:24 smithi188 ceph-mon[36599]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.209 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:24 smithi188 ceph-mon[36599]: scrub summary: active paths [/] 2024-06-10T02:59:25.209 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:24 smithi188 ceph-mon[36599]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.209 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:24 smithi188 ceph-mon[36599]: scrub summary: active paths [/] 2024-06-10T02:59:25.209 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:24 smithi188 ceph-mon[36599]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.209 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:24 smithi188 ceph-mon[36599]: scrub summary: active paths [/] 2024-06-10T02:59:25.209 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:24 smithi188 ceph-mon[36599]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.209 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:24 smithi188 ceph-mon[36599]: scrub summary: active paths [/] 2024-06-10T02:59:25.209 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:24 smithi188 ceph-mon[36599]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.209 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:24 smithi188 ceph-mon[36599]: scrub summary: active paths [/] 2024-06-10T02:59:25.209 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:24 smithi188 ceph-mon[36599]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.209 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:24 smithi188 ceph-mon[36599]: scrub summary: active paths [/] 2024-06-10T02:59:25.209 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:24 smithi188 ceph-mon[36599]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.209 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:24 smithi188 ceph-mon[36599]: scrub summary: active paths [/] 2024-06-10T02:59:25.209 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:24 smithi188 ceph-mon[36599]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.210 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:24 smithi188 ceph-mon[36599]: scrub summary: active paths [/] 2024-06-10T02:59:25.210 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:24 smithi188 ceph-mon[36599]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.210 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:24 smithi188 ceph-mon[36599]: scrub summary: active paths [/] 2024-06-10T02:59:25.210 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:24 smithi188 ceph-mon[36599]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.210 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:24 smithi188 ceph-mon[36599]: scrub summary: active paths [/] 2024-06-10T02:59:25.210 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:24 smithi188 ceph-mon[36599]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.210 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:24 smithi188 ceph-mon[36599]: scrub summary: active paths [/] 2024-06-10T02:59:25.210 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:24 smithi188 ceph-mon[36599]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.210 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:24 smithi188 ceph-mon[36599]: scrub summary: active paths [/] 2024-06-10T02:59:25.210 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:24 smithi188 ceph-mon[36599]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.210 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:24 smithi188 ceph-mon[36599]: scrub summary: active paths [/] 2024-06-10T02:59:25.211 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:24 smithi188 ceph-mon[36599]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.211 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:24 smithi188 ceph-mon[36599]: scrub summary: active paths [/] 2024-06-10T02:59:25.211 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:24 smithi188 ceph-mon[36599]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.211 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:24 smithi188 ceph-mon[36599]: scrub summary: active paths [/] 2024-06-10T02:59:25.211 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:24 smithi188 ceph-mon[36599]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.211 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:24 smithi188 ceph-mon[36599]: scrub summary: active paths [/] 2024-06-10T02:59:25.211 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:24 smithi188 ceph-mon[36599]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.211 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:24 smithi188 ceph-mon[36599]: scrub summary: active paths [/] 2024-06-10T02:59:25.211 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:24 smithi188 ceph-mon[36599]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.211 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:24 smithi188 ceph-mon[36599]: scrub summary: active paths [/] 2024-06-10T02:59:25.211 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:24 smithi188 ceph-mon[36599]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.211 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:24 smithi188 ceph-mon[36599]: scrub summary: active paths [/] 2024-06-10T02:59:25.211 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:24 smithi188 ceph-mon[36599]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.211 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:24 smithi188 ceph-mon[36599]: scrub summary: active paths [/] 2024-06-10T02:59:25.211 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:24 smithi188 ceph-mon[36599]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.212 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:24 smithi188 ceph-mon[36599]: scrub summary: active paths [/] 2024-06-10T02:59:25.212 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:24 smithi188 ceph-mon[36599]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.212 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:24 smithi188 ceph-mon[36599]: scrub summary: active paths [/] 2024-06-10T02:59:25.212 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:24 smithi188 ceph-mon[36599]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.212 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:24 smithi188 ceph-mon[36599]: scrub summary: active paths [/] 2024-06-10T02:59:25.212 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:24 smithi188 ceph-mon[36599]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.212 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:24 smithi188 ceph-mon[36599]: scrub summary: active paths [/] 2024-06-10T02:59:25.212 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:24 smithi188 ceph-mon[36599]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.212 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:24 smithi188 ceph-mon[36599]: scrub summary: active paths [/] 2024-06-10T02:59:25.212 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:24 smithi188 ceph-mon[36599]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.212 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:24 smithi188 ceph-mon[36599]: scrub summary: active paths [/] 2024-06-10T02:59:25.212 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:24 smithi188 ceph-mon[36599]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.212 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:24 smithi188 ceph-mon[36599]: scrub summary: active paths [/] 2024-06-10T02:59:25.212 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:24 smithi188 ceph-mon[36599]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.213 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:24 smithi188 ceph-mon[36599]: scrub summary: active paths [/] 2024-06-10T02:59:25.213 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:24 smithi188 ceph-mon[36599]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.213 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:24 smithi188 ceph-mon[36599]: scrub summary: active paths [/] 2024-06-10T02:59:25.213 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:24 smithi188 ceph-mon[36599]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.213 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:24 smithi188 ceph-mon[36599]: scrub summary: active paths [/] 2024-06-10T02:59:25.213 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:24 smithi188 ceph-mon[36599]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.213 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:24 smithi188 ceph-mon[36599]: scrub summary: active paths [/] 2024-06-10T02:59:25.213 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:24 smithi188 ceph-mon[36599]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.213 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:24 smithi188 ceph-mon[36599]: scrub summary: active paths [/] 2024-06-10T02:59:25.213 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:24 smithi188 ceph-mon[36599]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.213 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:24 smithi188 ceph-mon[36599]: scrub summary: active paths [/] 2024-06-10T02:59:25.213 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:24 smithi188 ceph-mon[36599]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.213 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:24 smithi188 ceph-mon[36599]: scrub summary: active paths [/] 2024-06-10T02:59:25.213 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:24 smithi188 ceph-mon[36599]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.214 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:24 smithi188 ceph-mon[36599]: scrub summary: active paths [/] 2024-06-10T02:59:25.214 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:24 smithi188 ceph-mon[36599]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.214 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:24 smithi188 ceph-mon[36599]: scrub summary: active paths [/] 2024-06-10T02:59:25.214 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:24 smithi188 ceph-mon[36599]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.214 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:24 smithi188 ceph-mon[36599]: scrub summary: active paths [/] 2024-06-10T02:59:25.215 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:24 smithi188 ceph-mon[36599]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.215 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:24 smithi188 ceph-mon[36599]: scrub summary: active paths [/] 2024-06-10T02:59:25.215 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:24 smithi188 ceph-mon[36599]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.215 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:24 smithi188 ceph-mon[36599]: scrub summary: active paths [/] 2024-06-10T02:59:25.215 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:24 smithi188 ceph-mon[36599]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.216 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:24 smithi188 ceph-mon[36599]: scrub summary: active paths [/] 2024-06-10T02:59:25.216 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:24 smithi188 ceph-mon[36599]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.216 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:24 smithi188 ceph-mon[36599]: scrub summary: active paths [/] 2024-06-10T02:59:25.216 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:24 smithi188 ceph-mon[36599]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.216 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:24 smithi188 ceph-mon[36599]: scrub summary: active paths [/] 2024-06-10T02:59:25.216 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:24 smithi188 ceph-mon[36599]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.216 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:24 smithi188 ceph-mon[36599]: scrub summary: active paths [/] 2024-06-10T02:59:25.216 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:24 smithi188 ceph-mon[36599]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.216 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:24 smithi188 ceph-mon[36599]: scrub summary: active paths [/] 2024-06-10T02:59:25.216 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:24 smithi188 ceph-mon[36599]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.216 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:24 smithi188 ceph-mon[36599]: scrub summary: active paths [/] 2024-06-10T02:59:25.216 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:24 smithi188 ceph-mon[36599]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.216 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:24 smithi188 ceph-mon[36599]: scrub summary: active paths [/] 2024-06-10T02:59:25.216 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:24 smithi188 ceph-mon[36599]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.216 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:24 smithi188 ceph-mon[36599]: scrub summary: active paths [/] 2024-06-10T02:59:25.217 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:24 smithi188 ceph-mon[36599]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.217 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:24 smithi188 ceph-mon[36599]: scrub summary: active paths [/] 2024-06-10T02:59:25.217 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:24 smithi188 ceph-mon[36599]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.217 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:24 smithi188 ceph-mon[36599]: scrub summary: active paths [/] 2024-06-10T02:59:25.217 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:24 smithi188 ceph-mon[36599]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.217 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:24 smithi188 ceph-mon[36599]: scrub summary: active paths [/] 2024-06-10T02:59:25.217 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:24 smithi188 ceph-mon[36599]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.217 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:24 smithi188 ceph-mon[36599]: scrub summary: active paths [/] 2024-06-10T02:59:25.217 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:24 smithi188 ceph-mon[36599]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.217 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:24 smithi188 ceph-mon[36599]: scrub summary: active paths [/] 2024-06-10T02:59:25.217 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:24 smithi188 ceph-mon[36599]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.217 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:24 smithi188 ceph-mon[36599]: scrub summary: active paths [/] 2024-06-10T02:59:25.218 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:24 smithi188 ceph-mon[36599]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.218 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:24 smithi188 ceph-mon[36599]: scrub summary: active paths [/] 2024-06-10T02:59:25.218 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:24 smithi188 ceph-mon[36599]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.218 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:24 smithi188 ceph-mon[36599]: scrub summary: active paths [/] 2024-06-10T02:59:25.218 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:24 smithi188 ceph-mon[36599]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.218 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:24 smithi188 ceph-mon[36599]: scrub summary: active paths [/] 2024-06-10T02:59:25.218 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:24 smithi188 ceph-mon[36599]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.229 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:24 smithi052 ceph-mon[32362]: pgmap v590: 129 pgs: 129 active+clean; 529 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.6 MiB/s wr, 535 op/s 2024-06-10T02:59:25.229 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:24 smithi052 ceph-mon[32362]: scrub summary: active paths [/] 2024-06-10T02:59:25.229 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:24 smithi052 ceph-mon[32362]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.230 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:24 smithi052 ceph-mon[32362]: scrub summary: active paths [/] 2024-06-10T02:59:25.230 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:24 smithi052 ceph-mon[32362]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.230 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:24 smithi052 ceph-mon[32362]: scrub summary: active paths [/] 2024-06-10T02:59:25.230 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:24 smithi052 ceph-mon[32362]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.230 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:24 smithi052 ceph-mon[32362]: scrub summary: active paths [/] 2024-06-10T02:59:25.230 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:24 smithi052 ceph-mon[32362]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.230 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:24 smithi052 ceph-mon[32362]: scrub summary: active paths [/] 2024-06-10T02:59:25.230 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:24 smithi052 ceph-mon[32362]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.230 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:24 smithi052 ceph-mon[32362]: scrub summary: active paths [/] 2024-06-10T02:59:25.230 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:24 smithi052 ceph-mon[32362]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.230 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:24 smithi052 ceph-mon[32362]: scrub summary: active paths [/] 2024-06-10T02:59:25.230 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:24 smithi052 ceph-mon[32362]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.230 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:24 smithi052 ceph-mon[32362]: scrub summary: active paths [/] 2024-06-10T02:59:25.231 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:24 smithi052 ceph-mon[32362]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.231 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:24 smithi052 ceph-mon[32362]: scrub summary: active paths [/] 2024-06-10T02:59:25.231 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:24 smithi052 ceph-mon[32362]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.231 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:24 smithi052 ceph-mon[32362]: scrub summary: active paths [/] 2024-06-10T02:59:25.231 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:24 smithi052 ceph-mon[32362]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.231 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:24 smithi052 ceph-mon[32362]: scrub summary: active paths [/] 2024-06-10T02:59:25.231 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:24 smithi052 ceph-mon[32362]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.231 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:24 smithi052 ceph-mon[32362]: scrub summary: active paths [/] 2024-06-10T02:59:25.231 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:24 smithi052 ceph-mon[32362]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.231 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:24 smithi052 ceph-mon[32362]: scrub summary: active paths [/] 2024-06-10T02:59:25.231 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:24 smithi052 ceph-mon[32362]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.231 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:24 smithi052 ceph-mon[32362]: scrub summary: active paths [/] 2024-06-10T02:59:25.231 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:24 smithi052 ceph-mon[32362]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.231 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:24 smithi052 ceph-mon[32362]: scrub summary: active paths [/] 2024-06-10T02:59:25.232 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:24 smithi052 ceph-mon[32362]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.232 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:24 smithi052 ceph-mon[32362]: scrub summary: active paths [/] 2024-06-10T02:59:25.232 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:24 smithi052 ceph-mon[32362]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.232 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:24 smithi052 ceph-mon[32362]: scrub summary: active paths [/] 2024-06-10T02:59:25.232 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:24 smithi052 ceph-mon[32362]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.232 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:24 smithi052 ceph-mon[32362]: scrub summary: active paths [/] 2024-06-10T02:59:25.232 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:24 smithi052 ceph-mon[32362]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.232 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:24 smithi052 ceph-mon[32362]: scrub summary: active paths [/] 2024-06-10T02:59:25.232 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:24 smithi052 ceph-mon[32362]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.232 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:24 smithi052 ceph-mon[32362]: scrub summary: active paths [/] 2024-06-10T02:59:25.232 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:24 smithi052 ceph-mon[32362]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.232 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:24 smithi052 ceph-mon[32362]: scrub summary: active paths [/] 2024-06-10T02:59:25.232 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:24 smithi052 ceph-mon[32362]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.232 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:24 smithi052 ceph-mon[32362]: scrub summary: active paths [/] 2024-06-10T02:59:25.233 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:24 smithi052 ceph-mon[32362]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.233 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:24 smithi052 ceph-mon[32362]: scrub summary: active paths [/] 2024-06-10T02:59:25.233 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:24 smithi052 ceph-mon[32362]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.233 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:24 smithi052 ceph-mon[32362]: scrub summary: active paths [/] 2024-06-10T02:59:25.233 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:24 smithi052 ceph-mon[32362]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.233 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:24 smithi052 ceph-mon[32362]: scrub summary: active paths [/] 2024-06-10T02:59:25.233 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:24 smithi052 ceph-mon[32362]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.233 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:24 smithi052 ceph-mon[32362]: scrub summary: active paths [/] 2024-06-10T02:59:25.233 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:24 smithi052 ceph-mon[32362]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.233 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:24 smithi052 ceph-mon[32362]: scrub summary: active paths [/] 2024-06-10T02:59:25.233 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:24 smithi052 ceph-mon[32362]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.233 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:24 smithi052 ceph-mon[32362]: scrub summary: active paths [/] 2024-06-10T02:59:25.233 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:24 smithi052 ceph-mon[32362]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.233 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:24 smithi052 ceph-mon[32362]: scrub summary: active paths [/] 2024-06-10T02:59:25.233 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:24 smithi052 ceph-mon[32362]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.234 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:24 smithi052 ceph-mon[32362]: scrub summary: active paths [/] 2024-06-10T02:59:25.234 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:24 smithi052 ceph-mon[32362]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.234 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:24 smithi052 ceph-mon[32362]: scrub summary: active paths [/] 2024-06-10T02:59:25.234 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:24 smithi052 ceph-mon[32362]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.234 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:24 smithi052 ceph-mon[32362]: scrub summary: active paths [/] 2024-06-10T02:59:25.234 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:24 smithi052 ceph-mon[32362]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.234 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:24 smithi052 ceph-mon[32362]: scrub summary: active paths [/] 2024-06-10T02:59:25.234 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:24 smithi052 ceph-mon[32362]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.234 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:24 smithi052 ceph-mon[32362]: scrub summary: active paths [/] 2024-06-10T02:59:25.234 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:24 smithi052 ceph-mon[32362]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.234 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:24 smithi052 ceph-mon[32362]: scrub summary: active paths [/] 2024-06-10T02:59:25.234 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:24 smithi052 ceph-mon[32362]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.234 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:24 smithi052 ceph-mon[32362]: scrub summary: active paths [/] 2024-06-10T02:59:25.234 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:24 smithi052 ceph-mon[32362]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.235 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:24 smithi052 ceph-mon[32362]: scrub summary: active paths [/] 2024-06-10T02:59:25.235 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:24 smithi052 ceph-mon[32362]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.235 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:24 smithi052 ceph-mon[32362]: scrub summary: active paths [/] 2024-06-10T02:59:25.235 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:24 smithi052 ceph-mon[32362]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.235 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:24 smithi052 ceph-mon[32362]: scrub summary: active paths [/] 2024-06-10T02:59:25.235 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:24 smithi052 ceph-mon[32362]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.235 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:24 smithi052 ceph-mon[32362]: scrub summary: active paths [/] 2024-06-10T02:59:25.235 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:24 smithi052 ceph-mon[32362]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.235 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:24 smithi052 ceph-mon[32362]: scrub summary: active paths [/] 2024-06-10T02:59:25.235 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:24 smithi052 ceph-mon[32362]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.235 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:24 smithi052 ceph-mon[32362]: scrub summary: active paths [/] 2024-06-10T02:59:25.235 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:24 smithi052 ceph-mon[32362]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.235 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:24 smithi052 ceph-mon[32362]: scrub summary: active paths [/] 2024-06-10T02:59:25.235 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:24 smithi052 ceph-mon[32362]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.236 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:24 smithi052 ceph-mon[32362]: scrub summary: active paths [/] 2024-06-10T02:59:25.236 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:24 smithi052 ceph-mon[32362]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.236 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:24 smithi052 ceph-mon[32362]: scrub summary: active paths [/] 2024-06-10T02:59:25.236 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:24 smithi052 ceph-mon[32362]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.236 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:24 smithi052 ceph-mon[32362]: scrub summary: active paths [/] 2024-06-10T02:59:25.236 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:24 smithi052 ceph-mon[32362]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.236 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:24 smithi052 ceph-mon[32362]: scrub summary: active paths [/] 2024-06-10T02:59:25.236 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:24 smithi052 ceph-mon[32362]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.236 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:24 smithi052 ceph-mon[32362]: scrub summary: active paths [/] 2024-06-10T02:59:25.236 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:24 smithi052 ceph-mon[32362]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.236 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:24 smithi052 ceph-mon[32362]: scrub summary: active paths [/] 2024-06-10T02:59:25.236 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:24 smithi052 ceph-mon[32362]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.236 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:24 smithi052 ceph-mon[32362]: scrub summary: active paths [/] 2024-06-10T02:59:25.236 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:24 smithi052 ceph-mon[32362]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.236 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:24 smithi052 ceph-mon[32362]: scrub summary: active paths [/] 2024-06-10T02:59:25.237 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:24 smithi052 ceph-mon[32362]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.237 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:24 smithi052 ceph-mon[32362]: scrub summary: active paths [/] 2024-06-10T02:59:25.237 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:24 smithi052 ceph-mon[32362]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.237 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:24 smithi052 ceph-mon[32362]: scrub summary: active paths [/] 2024-06-10T02:59:25.237 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:24 smithi052 ceph-mon[32362]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.237 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:24 smithi052 ceph-mon[32362]: scrub summary: active paths [/] 2024-06-10T02:59:25.237 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:24 smithi052 ceph-mon[32362]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.237 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:24 smithi052 ceph-mon[32362]: scrub summary: active paths [/] 2024-06-10T02:59:25.237 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:24 smithi052 ceph-mon[32362]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.238 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:24 smithi052 ceph-mon[32362]: scrub summary: active paths [/] 2024-06-10T02:59:25.238 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:24 smithi052 ceph-mon[32362]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.238 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:24 smithi052 ceph-mon[32362]: scrub summary: active paths [/] 2024-06-10T02:59:25.238 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:24 smithi052 ceph-mon[32362]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.238 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:24 smithi052 ceph-mon[32362]: scrub summary: active paths [/] 2024-06-10T02:59:25.238 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:24 smithi052 ceph-mon[32362]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.238 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:24 smithi052 ceph-mon[32362]: scrub summary: active paths [/] 2024-06-10T02:59:25.238 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:24 smithi052 ceph-mon[32362]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.238 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:24 smithi052 ceph-mon[32362]: scrub summary: active paths [/] 2024-06-10T02:59:25.238 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:24 smithi052 ceph-mon[32362]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.238 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:24 smithi052 ceph-mon[32362]: scrub summary: active paths [/] 2024-06-10T02:59:25.238 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:24 smithi052 ceph-mon[32362]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.238 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:24 smithi052 ceph-mon[32362]: scrub summary: active paths [/] 2024-06-10T02:59:25.238 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:24 smithi052 ceph-mon[32362]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.239 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:24 smithi052 ceph-mon[32362]: scrub summary: active paths [/] 2024-06-10T02:59:25.239 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:24 smithi052 ceph-mon[32362]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.239 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:24 smithi052 ceph-mon[32362]: scrub summary: active paths [/] 2024-06-10T02:59:25.239 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:24 smithi052 ceph-mon[32362]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.239 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:24 smithi052 ceph-mon[32362]: scrub summary: active paths [/] 2024-06-10T02:59:25.239 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:24 smithi052 ceph-mon[32362]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.239 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:24 smithi052 ceph-mon[32362]: scrub summary: active paths [/] 2024-06-10T02:59:25.239 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:24 smithi052 ceph-mon[32362]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.239 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:24 smithi052 ceph-mon[32362]: scrub summary: active paths [/] 2024-06-10T02:59:25.239 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:24 smithi052 ceph-mon[32362]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.239 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:24 smithi052 ceph-mon[32362]: scrub summary: active paths [/] 2024-06-10T02:59:25.239 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:24 smithi052 ceph-mon[32362]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.239 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:24 smithi052 ceph-mon[32362]: scrub summary: active paths [/] 2024-06-10T02:59:25.239 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:24 smithi052 ceph-mon[32362]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.239 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:24 smithi052 ceph-mon[32362]: scrub summary: active paths [/] 2024-06-10T02:59:25.240 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:24 smithi052 ceph-mon[32362]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.240 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:24 smithi052 ceph-mon[32362]: scrub summary: active paths [/] 2024-06-10T02:59:25.240 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:24 smithi052 ceph-mon[32362]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.240 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:24 smithi052 ceph-mon[32362]: scrub summary: active paths [/] 2024-06-10T02:59:25.240 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:24 smithi052 ceph-mon[32362]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.240 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:24 smithi052 ceph-mon[32362]: scrub summary: active paths [/] 2024-06-10T02:59:25.240 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:24 smithi052 ceph-mon[32362]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:25.240 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:24 smithi052 ceph-mon[32362]: scrub summary: active paths [/] 2024-06-10T02:59:25.240 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:24 smithi052 ceph-mon[32362]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:27.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:26 smithi120 ceph-mon[36634]: pgmap v591: 129 pgs: 129 active+clean; 530 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.2 MiB/s wr, 560 op/s 2024-06-10T02:59:27.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:26 smithi188 ceph-mon[36599]: pgmap v591: 129 pgs: 129 active+clean; 530 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.2 MiB/s wr, 560 op/s 2024-06-10T02:59:27.226 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:26 smithi052 ceph-mon[32362]: pgmap v591: 129 pgs: 129 active+clean; 530 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.2 MiB/s wr, 560 op/s 2024-06-10T02:59:27.246 INFO:tasks.workunit.client.0.smithi052.stdout: 84 686230 290 480600 247 361679 861 2024-06-10T02:59:29.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:28 smithi188 ceph-mon[36599]: pgmap v592: 129 pgs: 129 active+clean; 532 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.4 MiB/s wr, 601 op/s 2024-06-10T02:59:29.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:28 smithi188 ceph-mon[36599]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-10T02:59:29.227 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:28 smithi052 ceph-mon[32362]: pgmap v592: 129 pgs: 129 active+clean; 532 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.4 MiB/s wr, 601 op/s 2024-06-10T02:59:29.227 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:28 smithi052 ceph-mon[32362]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-10T02:59:29.388 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:28 smithi120 ceph-mon[36634]: pgmap v592: 129 pgs: 129 active+clean; 532 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.4 MiB/s wr, 601 op/s 2024-06-10T02:59:29.388 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:28 smithi120 ceph-mon[36634]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-10T02:59:31.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:30 smithi188 ceph-mon[36599]: pgmap v593: 129 pgs: 129 active+clean; 535 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.5 MiB/s wr, 618 op/s 2024-06-10T02:59:31.226 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:30 smithi052 ceph-mon[32362]: pgmap v593: 129 pgs: 129 active+clean; 535 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.5 MiB/s wr, 618 op/s 2024-06-10T02:59:31.388 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:30 smithi120 ceph-mon[36634]: pgmap v593: 129 pgs: 129 active+clean; 535 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.5 MiB/s wr, 618 op/s 2024-06-10T02:59:33.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:32 smithi188 ceph-mon[36599]: pgmap v594: 129 pgs: 129 active+clean; 535 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.1 MiB/s wr, 527 op/s 2024-06-10T02:59:33.227 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:32 smithi052 ceph-mon[32362]: pgmap v594: 129 pgs: 129 active+clean; 535 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.1 MiB/s wr, 527 op/s 2024-06-10T02:59:33.388 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:32 smithi120 ceph-mon[36634]: pgmap v594: 129 pgs: 129 active+clean; 535 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.1 MiB/s wr, 527 op/s 2024-06-10T02:59:35.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:34 smithi188 ceph-mon[36599]: pgmap v595: 129 pgs: 129 active+clean; 539 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.6 MiB/s wr, 570 op/s 2024-06-10T02:59:35.226 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:34 smithi052 ceph-mon[32362]: pgmap v595: 129 pgs: 129 active+clean; 539 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.6 MiB/s wr, 570 op/s 2024-06-10T02:59:35.388 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:34 smithi120 ceph-mon[36634]: pgmap v595: 129 pgs: 129 active+clean; 539 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.6 MiB/s wr, 570 op/s 2024-06-10T02:59:37.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:36 smithi188 ceph-mon[36599]: pgmap v596: 129 pgs: 129 active+clean; 542 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 384 op/s 2024-06-10T02:59:37.216 INFO:tasks.quiescer.fs.[cephfs]:Total duration (58.9) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-06-10T02:59:37.216 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 29.45 2024-06-10T02:59:37.217 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T02:59:37.235 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:36 smithi052 ceph-mon[32362]: pgmap v596: 129 pgs: 129 active+clean; 542 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 384 op/s 2024-06-10T02:59:37.260 INFO:tasks.workunit.client.0.smithi052.stdout: 91 2003053 366 1425610 288 1072438 1506 2024-06-10T02:59:37.389 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:36 smithi120 ceph-mon[36634]: pgmap v596: 129 pgs: 129 active+clean; 542 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 384 op/s 2024-06-10T02:59:38.607 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:59:39.045 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:38 smithi052 ceph-mon[32362]: pgmap v597: 129 pgs: 129 active+clean; 549 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.0 MiB/s wr, 405 op/s 2024-06-10T02:59:39.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:38 smithi188 ceph-mon[36599]: pgmap v597: 129 pgs: 129 active+clean; 549 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.0 MiB/s wr, 405 op/s 2024-06-10T02:59:39.388 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:38 smithi120 ceph-mon[36634]: pgmap v597: 129 pgs: 129 active+clean; 549 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.0 MiB/s wr, 405 op/s 2024-06-10T02:59:40.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:40.397+0000 7fbdd36f0700 1 -- 172.21.15.52:0/1267479976 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbdcc101390 msgr2=0x7fbdcc101810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:59:40.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:40.397+0000 7fbdd36f0700 1 --2- 172.21.15.52:0/1267479976 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbdcc101390 0x7fbdcc101810 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fbdc0009990 tx=0x7fbdc0009ca0 comp rx=0 tx=0).stop 2024-06-10T02:59:40.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:40.397+0000 7fbdd36f0700 1 -- 172.21.15.52:0/1267479976 shutdown_connections 2024-06-10T02:59:40.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:40.397+0000 7fbdd36f0700 1 --2- 172.21.15.52:0/1267479976 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbdcc101d50 0x7fbdcc10e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:40.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:40.397+0000 7fbdd36f0700 1 --2- 172.21.15.52:0/1267479976 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbdcc107400 0x7fbdcc1077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:40.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:40.397+0000 7fbdd36f0700 1 --2- 172.21.15.52:0/1267479976 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbdcc101390 0x7fbdcc101810 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:40.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:40.397+0000 7fbdd36f0700 1 -- 172.21.15.52:0/1267479976 >> 172.21.15.52:0/1267479976 conn(0x7fbdcc078340 msgr2=0x7fbdcc078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:59:40.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:40.397+0000 7fbdd36f0700 1 -- 172.21.15.52:0/1267479976 shutdown_connections 2024-06-10T02:59:40.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:40.397+0000 7fbdd36f0700 1 -- 172.21.15.52:0/1267479976 wait complete. 2024-06-10T02:59:40.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:40.398+0000 7fbdd36f0700 1 Processor -- start 2024-06-10T02:59:40.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:40.398+0000 7fbdd36f0700 1 -- start start 2024-06-10T02:59:40.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:40.398+0000 7fbdd36f0700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbdcc101390 0x7fbdcc199a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:59:40.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:40.398+0000 7fbdd36f0700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbdcc101d50 0x7fbdcc199fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:59:40.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:40.398+0000 7fbdd36f0700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbdcc107400 0x7fbdcc1a4ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:59:40.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:40.398+0000 7fbdd36f0700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fbdcc1116a0 con 0x7fbdcc101d50 2024-06-10T02:59:40.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:40.398+0000 7fbdd36f0700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fbdcc111520 con 0x7fbdcc101390 2024-06-10T02:59:40.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:40.398+0000 7fbdd36f0700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fbdcc111820 con 0x7fbdcc107400 2024-06-10T02:59:40.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:40.399+0000 7fbdd0c8b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbdcc101d50 0x7fbdcc199fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:59:40.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:40.399+0000 7fbdd0c8b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbdcc101d50 0x7fbdcc199fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:33896/0 (socket says 172.21.15.52:33896) 2024-06-10T02:59:40.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:40.399+0000 7fbdd0c8b700 1 -- 172.21.15.52:0/231071503 learned_addr learned my addr 172.21.15.52:0/231071503 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:59:40.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:40.399+0000 7fbdd1c8d700 1 --2- 172.21.15.52:0/231071503 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbdcc107400 0x7fbdcc1a4ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:59:40.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:40.399+0000 7fbdd148c700 1 --2- 172.21.15.52:0/231071503 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbdcc101390 0x7fbdcc199a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:59:40.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:40.399+0000 7fbdd0c8b700 1 -- 172.21.15.52:0/231071503 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbdcc107400 msgr2=0x7fbdcc1a4ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:59:40.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:40.399+0000 7fbdd0c8b700 1 --2- 172.21.15.52:0/231071503 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbdcc107400 0x7fbdcc1a4ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:40.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:40.399+0000 7fbdd0c8b700 1 -- 172.21.15.52:0/231071503 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbdcc101390 msgr2=0x7fbdcc199a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:59:40.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:40.399+0000 7fbdd0c8b700 1 --2- 172.21.15.52:0/231071503 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbdcc101390 0x7fbdcc199a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:40.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:40.399+0000 7fbdd0c8b700 1 -- 172.21.15.52:0/231071503 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdcc1a5020 con 0x7fbdcc101d50 2024-06-10T02:59:40.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:40.399+0000 7fbdd1c8d700 1 --2- 172.21.15.52:0/231071503 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbdcc107400 0x7fbdcc1a4ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T02:59:40.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:40.399+0000 7fbdd0c8b700 1 --2- 172.21.15.52:0/231071503 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbdcc101d50 0x7fbdcc199fa0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fbdc0000c00 tx=0x7fbdc000bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:59:40.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:40.399+0000 7fbdca7fc700 1 -- 172.21.15.52:0/231071503 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdc0027070 con 0x7fbdcc101d50 2024-06-10T02:59:40.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:40.399+0000 7fbdca7fc700 1 -- 172.21.15.52:0/231071503 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbdc0003d70 con 0x7fbdcc101d50 2024-06-10T02:59:40.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:40.399+0000 7fbdd36f0700 1 -- 172.21.15.52:0/231071503 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbdcc1a51f0 con 0x7fbdcc101d50 2024-06-10T02:59:40.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:40.399+0000 7fbdca7fc700 1 -- 172.21.15.52:0/231071503 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdc000f460 con 0x7fbdcc101d50 2024-06-10T02:59:40.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:40.399+0000 7fbdd36f0700 1 -- 172.21.15.52:0/231071503 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbdcc1a56a0 con 0x7fbdcc101d50 2024-06-10T02:59:40.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:40.400+0000 7fbdd36f0700 1 -- 172.21.15.52:0/231071503 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbdcc069180 con 0x7fbdcc101d50 2024-06-10T02:59:40.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:40.402+0000 7fbdca7fc700 1 -- 172.21.15.52:0/231071503 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fbdc002a070 con 0x7fbdcc101d50 2024-06-10T02:59:40.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:40.402+0000 7fbdca7fc700 1 --2- 172.21.15.52:0/231071503 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fbdb40a2ca0 0x7fbdb40a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:59:40.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:40.402+0000 7fbdd148c700 1 --2- 172.21.15.52:0/231071503 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fbdb40a2ca0 0x7fbdb40a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:59:40.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:40.402+0000 7fbdca7fc700 1 -- 172.21.15.52:0/231071503 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7fbdc00d03c0 con 0x7fbdcc101d50 2024-06-10T02:59:40.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:40.403+0000 7fbdd148c700 1 --2- 172.21.15.52:0/231071503 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fbdb40a2ca0 0x7fbdb40a5160 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fbdbc009550 tx=0x7fbdbc0091f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:59:40.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:40.406+0000 7fbdca7fc700 1 -- 172.21.15.52:0/231071503 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fbdc0013070 con 0x7fbdcc101d50 2024-06-10T02:59:40.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:40.673+0000 7fbdd36f0700 1 -- 172.21.15.52:0/231071503 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbdcc02dca0 con 0x7fbdcc101d50 2024-06-10T02:59:40.673 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:40.673+0000 7fbdca7fc700 1 -- 172.21.15.52:0/231071503 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+11945 (secure 0 0 0) 0x7fbdc009d040 con 0x7fbdcc101d50 2024-06-10T02:59:40.674 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-10T02:59:40.674 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":36,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/889673247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":889673247},{"type":"v1","addr":"172.21.15.52:6835","nonce":889673247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/3615663511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":3615663511},{"type":"v1","addr":"172.21.15.52:6837","nonce":3615663511}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":36,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T02:50:03.889766+0000","modified":"2024-06-10T02:59:27.720611+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24484,"mds_1":24469,"mds_2":24463,"mds_3":24449,"mds_4":24428},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14571":{"gid":14571,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6839/3059599042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":3059599042},{"type":"v1","addr":"172.21.15.52:6839","nonce":3059599042}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14583":{"gid":14583,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6841/3039275521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":3039275521},{"type":"v1","addr":"172.21.15.52:6841","nonce":3039275521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24383":{"gid":24383,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6833/2609002896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":2609002896},{"type":"v1","addr":"172.21.15.120:6833","nonce":2609002896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6835/1996370288","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":1996370288},{"type":"v1","addr":"172.21.15.120:6835","nonce":1996370288}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6833/529564979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":529564979},{"type":"v1","addr":"172.21.15.188:6833","nonce":529564979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":25,"addr":"172.21.15.120:6837/4132440952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":4132440952},{"type":"v1","addr":"172.21.15.120:6837","nonce":4132440952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"b","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.120:6839/1672436487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1672436487},{"type":"v1","addr":"172.21.15.120:6839","nonce":1672436487}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":2,"incarnation":22,"state":"up:active","state_seq":26,"addr":"172.21.15.188:6835/3904544259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":3904544259},{"type":"v1","addr":"172.21.15.188:6835","nonce":3904544259}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.188:6837/819317726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":819317726},{"type":"v1","addr":"172.21.15.188:6837","nonce":819317726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.188:6839/310444836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":310444836},{"type":"v1","addr":"172.21.15.188:6839","nonce":310444836}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24484,"qdb_cluster":[24469,24484,24463,24428,24449]},"id":1}]} 2024-06-10T02:59:40.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:40.676+0000 7fbdd36f0700 1 -- 172.21.15.52:0/231071503 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fbdb40a2ca0 msgr2=0x7fbdb40a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:59:40.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:40.676+0000 7fbdd36f0700 1 --2- 172.21.15.52:0/231071503 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fbdb40a2ca0 0x7fbdb40a5160 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fbdbc009550 tx=0x7fbdbc0091f0 comp rx=0 tx=0).stop 2024-06-10T02:59:40.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:40.676+0000 7fbdd36f0700 1 -- 172.21.15.52:0/231071503 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbdcc101d50 msgr2=0x7fbdcc199fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:59:40.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:40.676+0000 7fbdd36f0700 1 --2- 172.21.15.52:0/231071503 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbdcc101d50 0x7fbdcc199fa0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fbdc0000c00 tx=0x7fbdc000bab0 comp rx=0 tx=0).stop 2024-06-10T02:59:40.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:40.676+0000 7fbdd36f0700 1 -- 172.21.15.52:0/231071503 shutdown_connections 2024-06-10T02:59:40.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:40.676+0000 7fbdd36f0700 1 --2- 172.21.15.52:0/231071503 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbdcc107400 0x7fbdcc1a4ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:40.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:40.676+0000 7fbdd36f0700 1 --2- 172.21.15.52:0/231071503 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbdcc101390 0x7fbdcc199a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:40.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:40.676+0000 7fbdd36f0700 1 --2- 172.21.15.52:0/231071503 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fbdb40a2ca0 0x7fbdb40a5160 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:40.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:40.676+0000 7fbdd36f0700 1 --2- 172.21.15.52:0/231071503 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbdcc101d50 0x7fbdcc199fa0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:40.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:40.676+0000 7fbdd36f0700 1 -- 172.21.15.52:0/231071503 >> 172.21.15.52:0/231071503 conn(0x7fbdcc078340 msgr2=0x7fbdcc0ffea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:59:40.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:40.676+0000 7fbdd36f0700 1 -- 172.21.15.52:0/231071503 shutdown_connections 2024-06-10T02:59:40.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:40.676+0000 7fbdd36f0700 1 -- 172.21.15.52:0/231071503 wait complete. 2024-06-10T02:59:40.677 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 36 2024-06-10T02:59:40.977 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:40 smithi052 ceph-mon[32362]: pgmap v598: 129 pgs: 129 active+clean; 541 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.6 MiB/s wr, 493 op/s 2024-06-10T02:59:40.977 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:40 smithi052 ceph-mon[32362]: from='client.? 172.21.15.52:0/231071503' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T02:59:41.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:40 smithi188 ceph-mon[36599]: pgmap v598: 129 pgs: 129 active+clean; 541 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.6 MiB/s wr, 493 op/s 2024-06-10T02:59:41.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:40 smithi188 ceph-mon[36599]: from='client.? 172.21.15.52:0/231071503' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T02:59:41.389 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:40 smithi120 ceph-mon[36634]: pgmap v598: 129 pgs: 129 active+clean; 541 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.6 MiB/s wr, 493 op/s 2024-06-10T02:59:41.390 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:40 smithi120 ceph-mon[36634]: from='client.? 172.21.15.52:0/231071503' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T02:59:41.455 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24484 quiesce db / --timeout 90 --expiration 149.45 --await-for 45.0' 2024-06-10T02:59:41.455 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24484 quiesce db / --timeout 90 --expiration 149.45 --await-for 45.0 2024-06-10T02:59:41.814 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:59:43.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:42 smithi188 ceph-mon[36599]: pgmap v599: 129 pgs: 129 active+clean; 542 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.0 MiB/s wr, 528 op/s 2024-06-10T02:59:43.229 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:42 smithi052 ceph-mon[32362]: pgmap v599: 129 pgs: 129 active+clean; 542 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.0 MiB/s wr, 528 op/s 2024-06-10T02:59:43.388 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:42 smithi120 ceph-mon[36634]: pgmap v599: 129 pgs: 129 active+clean; 542 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.0 MiB/s wr, 528 op/s 2024-06-10T02:59:45.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.111+0000 7f8a0a35b700 1 -- 172.21.15.52:0/977190214 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8a04101d80 msgr2=0x7f8a0410e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:59:45.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.111+0000 7f8a0a35b700 1 --2- 172.21.15.52:0/977190214 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8a04101d80 0x7f8a0410e220 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f89f80098f0 tx=0x7f89f8009c00 comp rx=0 tx=0).stop 2024-06-10T02:59:45.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.112+0000 7f8a0a35b700 1 -- 172.21.15.52:0/977190214 shutdown_connections 2024-06-10T02:59:45.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.112+0000 7f8a0a35b700 1 --2- 172.21.15.52:0/977190214 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8a04101d80 0x7f8a0410e220 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:45.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.112+0000 7f8a0a35b700 1 --2- 172.21.15.52:0/977190214 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8a04107470 0x7f8a04107850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:45.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.112+0000 7f8a0a35b700 1 --2- 172.21.15.52:0/977190214 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8a041013c0 0x7f8a04101840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:45.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.112+0000 7f8a0a35b700 1 -- 172.21.15.52:0/977190214 >> 172.21.15.52:0/977190214 conn(0x7f8a040783d0 msgr2=0x7f8a040787e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:59:45.113 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:44 smithi052 ceph-mon[32362]: pgmap v600: 129 pgs: 129 active+clean; 557 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.9 MiB/s wr, 710 op/s 2024-06-10T02:59:45.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.115+0000 7f8a0a35b700 1 -- 172.21.15.52:0/977190214 shutdown_connections 2024-06-10T02:59:45.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.115+0000 7f8a0a35b700 1 -- 172.21.15.52:0/977190214 wait complete. 2024-06-10T02:59:45.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.116+0000 7f8a0a35b700 1 Processor -- start 2024-06-10T02:59:45.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.116+0000 7f8a0a35b700 1 -- start start 2024-06-10T02:59:45.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.117+0000 7f8a0a35b700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8a041013c0 0x7f8a04073a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:59:45.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.117+0000 7f8a0a35b700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8a04101d80 0x7f8a04073f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:59:45.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.117+0000 7f8a0a35b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8a04107470 0x7f8a0406eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:59:45.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.117+0000 7f8a0a35b700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f8a0406d6a0 con 0x7f8a04107470 2024-06-10T02:59:45.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.117+0000 7f8a0a35b700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f8a0406d520 con 0x7f8a04101d80 2024-06-10T02:59:45.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.117+0000 7f8a0a35b700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f8a0406d820 con 0x7f8a041013c0 2024-06-10T02:59:45.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.118+0000 7f8a088f8700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8a04107470 0x7f8a0406eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:59:45.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.118+0000 7f8a088f8700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8a04107470 0x7f8a0406eb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57276/0 (socket says 172.21.15.52:57276) 2024-06-10T02:59:45.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.118+0000 7f8a088f8700 1 -- 172.21.15.52:0/4212657555 learned_addr learned my addr 172.21.15.52:0/4212657555 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:59:45.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.119+0000 7f8a037fe700 1 --2- 172.21.15.52:0/4212657555 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8a04101d80 0x7f8a04073f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:59:45.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.119+0000 7f8a03fff700 1 --2- 172.21.15.52:0/4212657555 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8a041013c0 0x7f8a04073a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:59:45.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.121+0000 7f8a037fe700 1 -- 172.21.15.52:0/4212657555 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8a041013c0 msgr2=0x7f8a04073a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:59:45.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.121+0000 7f8a037fe700 1 --2- 172.21.15.52:0/4212657555 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8a041013c0 0x7f8a04073a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:45.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.121+0000 7f8a037fe700 1 -- 172.21.15.52:0/4212657555 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8a04107470 msgr2=0x7f8a0406eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:59:45.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.121+0000 7f8a037fe700 1 --2- 172.21.15.52:0/4212657555 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8a04107470 0x7f8a0406eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:45.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.121+0000 7f8a037fe700 1 -- 172.21.15.52:0/4212657555 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a0406f490 con 0x7f8a04101d80 2024-06-10T02:59:45.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.123+0000 7f8a037fe700 1 --2- 172.21.15.52:0/4212657555 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8a04101d80 0x7f8a04073f60 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f89fc01ffc0 tx=0x7f89fc01d3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:59:45.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.123+0000 7f8a017fa700 1 -- 172.21.15.52:0/4212657555 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89fc01d9c0 con 0x7f8a04101d80 2024-06-10T02:59:45.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.123+0000 7f8a0a35b700 1 -- 172.21.15.52:0/4212657555 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a0406f720 con 0x7f8a04101d80 2024-06-10T02:59:45.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.124+0000 7f8a0a35b700 1 -- 172.21.15.52:0/4212657555 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a041a3610 con 0x7f8a04101d80 2024-06-10T02:59:45.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.133+0000 7f8a017fa700 1 -- 172.21.15.52:0/4212657555 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f89fc02a070 con 0x7f8a04101d80 2024-06-10T02:59:45.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.133+0000 7f8a017fa700 1 -- 172.21.15.52:0/4212657555 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89fc0256e0 con 0x7f8a04101d80 2024-06-10T02:59:45.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.135+0000 7f8a017fa700 1 -- 172.21.15.52:0/4212657555 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f89fc023050 con 0x7f8a04101d80 2024-06-10T02:59:45.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.140+0000 7f8a017fa700 1 --2- 172.21.15.52:0/4212657555 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f89ec0a2eb0 0x7f89ec0a5370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:59:45.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.143+0000 7f8a017fa700 1 -- 172.21.15.52:0/4212657555 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f89fc0e12e0 con 0x7f8a04101d80 2024-06-10T02:59:45.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.144+0000 7f8a14bdeb80 1 Processor -- start 2024-06-10T02:59:45.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.144+0000 7f8a03fff700 1 --2- 172.21.15.52:0/4212657555 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f89ec0a2eb0 0x7f89ec0a5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:59:45.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.145+0000 7f8a03fff700 1 --2- 172.21.15.52:0/4212657555 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f89ec0a2eb0 0x7f89ec0a5370 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f89f4009a10 tx=0x7f89f4009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:59:45.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.146+0000 7f8a14bdeb80 1 -- start start 2024-06-10T02:59:45.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.146+0000 7f8a14bdeb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5584ae46fce0 0x5584ae4700c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:59:45.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.147+0000 7f8a14bdeb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584ae470600 0x5584ae469f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:59:45.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.147+0000 7f8a03fff700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5584ae46fce0 0x5584ae4700c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:59:45.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.147+0000 7f8a03fff700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5584ae46fce0 0x5584ae4700c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.52:37298/0 (socket says 172.21.15.52:37298) 2024-06-10T02:59:45.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.147+0000 7f8a037fe700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584ae470600 0x5584ae469f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:59:45.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.147+0000 7f8a037fe700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584ae470600 0x5584ae469f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.52:33082/0 (socket says 172.21.15.52:33082) 2024-06-10T02:59:45.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.147+0000 7f8a03fff700 1 -- 172.21.15.52:0/4102000642 learned_addr learned my addr 172.21.15.52:0/4102000642 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:59:45.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.147+0000 7f8a14bdeb80 1 --2- 172.21.15.52:0/4102000642 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5584ae46a470 0x5584ae46a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:59:45.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.147+0000 7f8a14bdeb80 1 -- 172.21.15.52:0/4102000642 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5584ae3bb9b0 con 0x5584ae46a470 2024-06-10T02:59:45.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.147+0000 7f8a14bdeb80 1 -- 172.21.15.52:0/4102000642 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5584ae3638c0 con 0x5584ae470600 2024-06-10T02:59:45.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.148+0000 7f8a14bdeb80 1 -- 172.21.15.52:0/4102000642 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5584ae3b7970 con 0x5584ae46fce0 2024-06-10T02:59:45.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.148+0000 7f8a088f8700 1 --2- 172.21.15.52:0/4102000642 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5584ae46a470 0x5584ae46a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:59:45.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.148+0000 7f8a088f8700 1 -- 172.21.15.52:0/4102000642 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5584ae46fce0 msgr2=0x5584ae4700c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:59:45.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.148+0000 7f8a088f8700 1 --2- 172.21.15.52:0/4102000642 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5584ae46fce0 0x5584ae4700c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:45.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.148+0000 7f8a088f8700 1 -- 172.21.15.52:0/4102000642 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584ae470600 msgr2=0x5584ae469f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:59:45.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.148+0000 7f8a088f8700 1 --2- 172.21.15.52:0/4102000642 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584ae470600 0x5584ae469f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:45.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.148+0000 7f8a088f8700 1 -- 172.21.15.52:0/4102000642 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5584ae35a0f0 con 0x5584ae46a470 2024-06-10T02:59:45.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.149+0000 7f8a088f8700 1 --2- 172.21.15.52:0/4102000642 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5584ae46a470 0x5584ae46a8b0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f8a04074650 tx=0x7f89f800bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d53180b875310a01 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:59:45.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.154+0000 7f89ea7fc700 1 -- 172.21.15.52:0/4102000642 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89f8026070 con 0x5584ae46a470 2024-06-10T02:59:45.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.154+0000 7f89ea7fc700 1 -- 172.21.15.52:0/4102000642 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f89f800fa50 con 0x5584ae46a470 2024-06-10T02:59:45.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.154+0000 7f89ea7fc700 1 -- 172.21.15.52:0/4102000642 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89f8003da0 con 0x5584ae46a470 2024-06-10T02:59:45.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.158+0000 7f8a14bdeb80 1 -- 172.21.15.52:0/4102000642 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5584ae46a470 msgr2=0x5584ae46a8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:59:45.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.158+0000 7f8a14bdeb80 1 --2- 172.21.15.52:0/4102000642 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5584ae46a470 0x5584ae46a8b0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f8a04074650 tx=0x7f89f800bfd0 comp rx=0 tx=0).stop 2024-06-10T02:59:45.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.159+0000 7f8a14bdeb80 1 -- 172.21.15.52:0/4102000642 shutdown_connections 2024-06-10T02:59:45.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.159+0000 7f8a14bdeb80 1 --2- 172.21.15.52:0/4102000642 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5584ae46a470 0x5584ae46a8b0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:45.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.159+0000 7f8a14bdeb80 1 --2- 172.21.15.52:0/4102000642 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5584ae46fce0 0x5584ae4700c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:45.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.159+0000 7f8a14bdeb80 1 --2- 172.21.15.52:0/4102000642 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584ae470600 0x5584ae469f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:45.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.159+0000 7f8a14bdeb80 1 -- 172.21.15.52:0/4102000642 >> 172.21.15.52:0/4102000642 conn(0x5584ae394e60 msgr2=0x5584ae4651c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:59:45.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.164+0000 7f8a14bdeb80 1 -- 172.21.15.52:0/4102000642 shutdown_connections 2024-06-10T02:59:45.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.164+0000 7f8a14bdeb80 1 -- 172.21.15.52:0/4102000642 wait complete. 2024-06-10T02:59:45.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.165+0000 7f8a14bdeb80 1 -- start start 2024-06-10T02:59:45.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.165+0000 7f8a14bdeb80 1 Processor -- start 2024-06-10T02:59:45.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.165+0000 7f8a14bdeb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5584ae46a470 0x5584ae476360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:59:45.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.166+0000 7f8a14bdeb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584ae46fce0 0x5584ae473660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:59:45.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.166+0000 7f8a14bdeb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5584ae470600 0x5584ae473ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:59:45.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.166+0000 7f8a14bdeb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5584ae477010 con 0x5584ae46a470 2024-06-10T02:59:45.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.166+0000 7f8a14bdeb80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5584ae35bd90 con 0x5584ae46fce0 2024-06-10T02:59:45.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.166+0000 7f8a03fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5584ae46a470 0x5584ae476360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:59:45.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.166+0000 7f8a03fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5584ae46a470 0x5584ae476360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57290/0 (socket says 172.21.15.52:57290) 2024-06-10T02:59:45.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.166+0000 7f8a03fff700 1 -- 172.21.15.52:0/1498515327 learned_addr learned my addr 172.21.15.52:0/1498515327 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:59:45.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.166+0000 7f8a14bdeb80 1 -- 172.21.15.52:0/1498515327 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5584ae477190 con 0x5584ae470600 2024-06-10T02:59:45.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.166+0000 7f8a14bdeb80 10 client.0 _ll_register_callbacks cb 0x5584ae355ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T02:59:45.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.167+0000 7f8a03fff700 1 -- 172.21.15.52:0/1498515327 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5584ae470600 msgr2=0x5584ae473ba0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-10T02:59:45.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.167+0000 7f8a03fff700 1 --2- 172.21.15.52:0/1498515327 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5584ae470600 0x5584ae473ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:45.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.167+0000 7f8a03fff700 1 -- 172.21.15.52:0/1498515327 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584ae46fce0 msgr2=0x5584ae473660 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-10T02:59:45.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.167+0000 7f8a03fff700 1 --2- 172.21.15.52:0/1498515327 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584ae46fce0 0x5584ae473660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:45.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.167+0000 7f8a03fff700 1 -- 172.21.15.52:0/1498515327 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5584ae30aa70 con 0x5584ae46a470 2024-06-10T02:59:45.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.168+0000 7f8a03fff700 1 --2- 172.21.15.52:0/1498515327 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5584ae46a470 0x5584ae476360 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f89f4015730 tx=0x7f89f4004860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bb33b1fb912e281 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:59:45.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.168+0000 7f89e37fe700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-10T02:59:45.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.168+0000 7f89e37fe700 1 -- 172.21.15.52:0/1498515327 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89f4025020 con 0x5584ae46a470 2024-06-10T02:59:45.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.169+0000 7f89e37fe700 1 -- 172.21.15.52:0/1498515327 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f89f4015d50 con 0x5584ae46a470 2024-06-10T02:59:45.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.169+0000 7f8a14bdeb80 1 -- 172.21.15.52:0/1498515327 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f89f800fa50 con 0x5584ae46a470 2024-06-10T02:59:45.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.170+0000 7f89e37fe700 1 -- 172.21.15.52:0/1498515327 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f89f4030050 con 0x5584ae46a470 2024-06-10T02:59:45.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.173+0000 7f8a14bdeb80 10 client.16275 fetch_fsmap learned FSMap version 36 2024-06-10T02:59:45.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.173+0000 7f8a14bdeb80 1 -- 172.21.15.52:0/1498515327 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5584ae481780 con 0x5584ae46a470 2024-06-10T02:59:45.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.175+0000 7f89e37fe700 1 -- 172.21.15.52:0/1498515327 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f89f800fa50 con 0x5584ae46a470 2024-06-10T02:59:45.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.175+0000 7f89e37fe700 1 -- 172.21.15.52:0/1498515327 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 36) ==== 8378+0+0 (secure 0 0 0) 0x7f89f4013690 con 0x5584ae46a470 2024-06-10T02:59:45.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.177+0000 7f8a14bdeb80 10 client.16275 fetch_fsmap finished waiting for FSMap version 36 2024-06-10T02:59:45.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.177+0000 7f8a14bdeb80 10 client.16275 resolve_mds: validated gid 24484 aka daemon mds.i 2024-06-10T02:59:45.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.177+0000 7f8a14bdeb80 20 client.16275 populate_metadata read hostname 'smithi052' 2024-06-10T02:59:45.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.177+0000 7f8a14bdeb80 1 --2- 172.21.15.52:0/1498515327 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x5584ae483dc0 0x5584ae4861d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:59:45.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.177+0000 7f8a14bdeb80 4 client.16275 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T02:59:45.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.177+0000 7f8a14bdeb80 1 -- 172.21.15.52:0/1498515327 --> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5584ae3c8eb0 con 0x5584ae483dc0 2024-06-10T02:59:45.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.180+0000 7f8a037fe700 1 --2- 172.21.15.52:0/1498515327 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x5584ae483dc0 0x5584ae4861d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:59:45.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.182+0000 7f8a037fe700 1 --2- 172.21.15.52:0/1498515327 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x5584ae483dc0 0x5584ae4861d0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=14c23c5ae5085a40 server_cookie=dc929191c2df3cf2 in_seq=0 out_seq=0 2024-06-10T02:59:45.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.186+0000 7f89e37fe700 10 client.16275 ms_handle_connect on v2:172.21.15.188:6838/310444836 2024-06-10T02:59:45.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.187+0000 7f89e37fe700 1 -- 172.21.15.52:0/1498515327 <== mds.0 v2:172.21.15.188:6838/310444836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5584ae3c8eb0 con 0x5584ae483dc0 2024-06-10T02:59:45.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.187+0000 7f89e37fe700 10 client.16275 handle_command_reply: tid=0 2024-06-10T02:59:45.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.187+0000 7f8a14bdeb80 1 client.16275 shutdown 2024-06-10T02:59:45.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.189+0000 7f8a14bdeb80 1 -- 172.21.15.52:0/1498515327 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5584ae46a470 msgr2=0x5584ae476360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:59:45.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.189+0000 7f8a14bdeb80 1 --2- 172.21.15.52:0/1498515327 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5584ae46a470 0x5584ae476360 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f89f4015730 tx=0x7f89f4004860 comp rx=0 tx=0).stop 2024-06-10T02:59:45.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.192+0000 7f8a14bdeb80 1 -- 172.21.15.52:0/1498515327 shutdown_connections 2024-06-10T02:59:45.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.192+0000 7f8a14bdeb80 1 --2- 172.21.15.52:0/1498515327 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x5584ae483dc0 0x5584ae4861d0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:45.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.192+0000 7f8a14bdeb80 1 --2- 172.21.15.52:0/1498515327 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5584ae470600 0x5584ae473ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:45.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.192+0000 7f8a14bdeb80 1 --2- 172.21.15.52:0/1498515327 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5584ae46a470 0x5584ae476360 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:45.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.192+0000 7f8a14bdeb80 1 --2- 172.21.15.52:0/1498515327 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584ae46fce0 0x5584ae473660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:45.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.192+0000 7f8a14bdeb80 1 -- 172.21.15.52:0/1498515327 >> 172.21.15.52:0/1498515327 conn(0x5584ae394e60 msgr2=0x5584ae464db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:59:45.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.193+0000 7f89e37fe700 0 client.16275 ms_handle_reset on v2:172.21.15.188:6838/310444836 2024-06-10T02:59:45.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.193+0000 7f8a14bdeb80 1 -- 172.21.15.52:0/1498515327 shutdown_connections 2024-06-10T02:59:45.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.193+0000 7f8a14bdeb80 1 -- 172.21.15.52:0/1498515327 wait complete. 2024-06-10T02:59:45.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.195+0000 7f8a14bdeb80 20 client.16275 trim_cache size 0 max 16384 2024-06-10T02:59:45.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:44 smithi188 ceph-mon[36599]: pgmap v600: 129 pgs: 129 active+clean; 557 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.9 MiB/s wr, 710 op/s 2024-06-10T02:59:45.248 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.248+0000 7f8a14bdeb80 1 Processor -- start 2024-06-10T02:59:45.248 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.248+0000 7f8a14bdeb80 1 -- start start 2024-06-10T02:59:45.248 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.248+0000 7f8a14bdeb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5584ae394e60 0x5584ae469f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:59:45.248 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.248+0000 7f8a14bdeb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584ae46a470 0x5584ae473bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:59:45.248 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.248+0000 7f8a14bdeb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5584ae46fce0 0x5584ae4755a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:59:45.249 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.248+0000 7f8a088f8700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5584ae46fce0 0x5584ae4755a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:59:45.249 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.249+0000 7f8a088f8700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5584ae46fce0 0x5584ae4755a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57304/0 (socket says 172.21.15.52:57304) 2024-06-10T02:59:45.249 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.249+0000 7f8a088f8700 1 -- 172.21.15.52:0/2443730611 learned_addr learned my addr 172.21.15.52:0/2443730611 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:59:45.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.249+0000 7f8a14bdeb80 1 -- 172.21.15.52:0/2443730611 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f89f000ec70 con 0x5584ae46fce0 2024-06-10T02:59:45.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.249+0000 7f8a14bdeb80 1 -- 172.21.15.52:0/2443730611 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f89f0010870 con 0x5584ae46a470 2024-06-10T02:59:45.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.249+0000 7f8a14bdeb80 1 -- 172.21.15.52:0/2443730611 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f89f000f340 con 0x5584ae394e60 2024-06-10T02:59:45.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.249+0000 7f8a088f8700 1 -- 172.21.15.52:0/2443730611 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5584ae394e60 msgr2=0x5584ae469f30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-10T02:59:45.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.249+0000 7f8a088f8700 1 --2- 172.21.15.52:0/2443730611 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5584ae394e60 0x5584ae469f30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:45.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.249+0000 7f8a088f8700 1 -- 172.21.15.52:0/2443730611 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584ae46a470 msgr2=0x5584ae473bd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-10T02:59:45.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.249+0000 7f8a088f8700 1 --2- 172.21.15.52:0/2443730611 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584ae46a470 0x5584ae473bd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:45.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.249+0000 7f8a088f8700 1 -- 172.21.15.52:0/2443730611 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5584ae46f7d0 con 0x5584ae46fce0 2024-06-10T02:59:45.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.249+0000 7f8a088f8700 1 --2- 172.21.15.52:0/2443730611 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5584ae46fce0 0x5584ae4755a0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f89f8009680 tx=0x7f89f800fa80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67e2a337010a05b4 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:59:45.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.250+0000 7f89e9ffb700 1 -- 172.21.15.52:0/2443730611 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89f802f070 con 0x5584ae46fce0 2024-06-10T02:59:45.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.250+0000 7f89e9ffb700 1 -- 172.21.15.52:0/2443730611 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f89f8005270 con 0x5584ae46fce0 2024-06-10T02:59:45.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.251+0000 7f89e9ffb700 1 -- 172.21.15.52:0/2443730611 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89f8026070 con 0x5584ae46fce0 2024-06-10T02:59:45.252 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-10T02:59:45.253 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.252+0000 7f8a14bdeb80 1 -- 172.21.15.52:0/2443730611 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5584ae46fce0 msgr2=0x5584ae4755a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:59:45.253 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.252+0000 7f8a14bdeb80 1 --2- 172.21.15.52:0/2443730611 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5584ae46fce0 0x5584ae4755a0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f89f8009680 tx=0x7f89f800fa80 comp rx=0 tx=0).stop 2024-06-10T02:59:45.257 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.256+0000 7f8a14bdeb80 1 -- 172.21.15.52:0/2443730611 shutdown_connections 2024-06-10T02:59:45.257 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.256+0000 7f8a14bdeb80 1 --2- 172.21.15.52:0/2443730611 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5584ae46fce0 0x5584ae4755a0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:45.257 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.256+0000 7f8a14bdeb80 1 --2- 172.21.15.52:0/2443730611 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5584ae394e60 0x5584ae469f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:45.257 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.256+0000 7f8a14bdeb80 1 --2- 172.21.15.52:0/2443730611 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584ae46a470 0x5584ae473bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:45.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.256+0000 7f8a14bdeb80 1 -- 172.21.15.52:0/2443730611 >> 172.21.15.52:0/2443730611 conn(0x5584ae46df60 msgr2=0x5584ae474e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:59:45.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.258+0000 7f8a14bdeb80 1 -- 172.21.15.52:0/2443730611 shutdown_connections 2024-06-10T02:59:45.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.259+0000 7f8a14bdeb80 1 -- 172.21.15.52:0/2443730611 wait complete. 2024-06-10T02:59:45.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.260+0000 7f8a14bdeb80 1 -- start start 2024-06-10T02:59:45.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.260+0000 7f8a14bdeb80 1 Processor -- start 2024-06-10T02:59:45.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.262+0000 7f8a14bdeb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584ae394e60 0x5584ae479a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:59:45.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.262+0000 7f8a14bdeb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5584ae46a470 0x5584ae479fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:59:45.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.262+0000 7f8a037fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5584ae46a470 0x5584ae479fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:59:45.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.262+0000 7f8a037fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5584ae46a470 0x5584ae479fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57318/0 (socket says 172.21.15.52:57318) 2024-06-10T02:59:45.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.262+0000 7f8a037fe700 1 -- 172.21.15.52:0/3040202147 learned_addr learned my addr 172.21.15.52:0/3040202147 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:59:45.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.262+0000 7f8a14bdeb80 1 --2- 172.21.15.52:0/3040202147 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5584ae46fce0 0x5584ae4eea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:59:45.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.263+0000 7f8a03fff700 1 --2- 172.21.15.52:0/3040202147 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584ae394e60 0x5584ae479a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:59:45.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.263+0000 7f8a14bdeb80 1 -- 172.21.15.52:0/3040202147 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f89f00101a0 con 0x5584ae46a470 2024-06-10T02:59:45.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.263+0000 7f8a088f8700 1 --2- 172.21.15.52:0/3040202147 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5584ae46fce0 0x5584ae4eea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:59:45.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.264+0000 7f8a14bdeb80 1 -- 172.21.15.52:0/3040202147 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f89f000fad0 con 0x5584ae394e60 2024-06-10T02:59:45.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.264+0000 7f8a14bdeb80 1 -- 172.21.15.52:0/3040202147 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f89f000cc10 con 0x5584ae46fce0 2024-06-10T02:59:45.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.265+0000 7f8a037fe700 1 -- 172.21.15.52:0/3040202147 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5584ae46fce0 msgr2=0x5584ae4eea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:59:45.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.265+0000 7f8a037fe700 1 --2- 172.21.15.52:0/3040202147 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5584ae46fce0 0x5584ae4eea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:45.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.265+0000 7f8a037fe700 1 -- 172.21.15.52:0/3040202147 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584ae394e60 msgr2=0x5584ae479a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:59:45.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.265+0000 7f8a037fe700 1 --2- 172.21.15.52:0/3040202147 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584ae394e60 0x5584ae479a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:45.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.265+0000 7f8a037fe700 1 -- 172.21.15.52:0/3040202147 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5584ae4eeff0 con 0x5584ae46a470 2024-06-10T02:59:45.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.266+0000 7f8a03fff700 1 --2- 172.21.15.52:0/3040202147 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584ae394e60 0x5584ae479a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T02:59:45.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.266+0000 7f8a088f8700 1 --2- 172.21.15.52:0/3040202147 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5584ae46fce0 0x5584ae4eea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T02:59:45.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.266+0000 7f8a037fe700 1 --2- 172.21.15.52:0/3040202147 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5584ae46a470 0x5584ae479fc0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f89fc0aa210 tx=0x7f89fc01a970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cde2c32c0d6f5265 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:59:45.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.267+0000 7f8a14bdeb80 10 client.0 _ll_register_callbacks cb 0x5584ae355ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T02:59:45.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.267+0000 7f8a14bdeb80 1 -- 172.21.15.52:0/3040202147 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f89f8005270 con 0x5584ae46a470 2024-06-10T02:59:45.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.268+0000 7f89e37fe700 10 client.16281 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-10T02:59:45.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.268+0000 7f89e37fe700 1 -- 172.21.15.52:0/3040202147 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89fc0cb060 con 0x5584ae46a470 2024-06-10T02:59:45.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.268+0000 7f89e37fe700 1 -- 172.21.15.52:0/3040202147 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f89fc0deae0 con 0x5584ae46a470 2024-06-10T02:59:45.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.268+0000 7f89e37fe700 1 -- 172.21.15.52:0/3040202147 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f89fc0e10b0 con 0x5584ae46a470 2024-06-10T02:59:45.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.269+0000 7f8a14bdeb80 10 client.16281 fetch_fsmap learned FSMap version 36 2024-06-10T02:59:45.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.269+0000 7f8a14bdeb80 1 -- 172.21.15.52:0/3040202147 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({fsmap=36}) -- 0x5584ae4f0080 con 0x5584ae46a470 2024-06-10T02:59:45.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.273+0000 7f89e37fe700 1 -- 172.21.15.52:0/3040202147 <== mon.0 v2:172.21.15.52:3300/0 4 ==== fsmap(e 36) ==== 8378+0+0 (secure 0 0 0) 0x7f89fc0b6020 con 0x5584ae46a470 2024-06-10T02:59:45.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.274+0000 7f8a14bdeb80 10 client.16281 fetch_fsmap finished waiting for FSMap version 36 2024-06-10T02:59:45.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.274+0000 7f8a14bdeb80 10 client.16281 resolve_mds: validated gid 24484 aka daemon mds.i 2024-06-10T02:59:45.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.274+0000 7f8a14bdeb80 20 client.16281 populate_metadata read hostname 'smithi052' 2024-06-10T02:59:45.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.274+0000 7f8a14bdeb80 1 --2- 172.21.15.52:0/3040202147 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x5584ae470600 0x5584ae4f27b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:59:45.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.274+0000 7f8a14bdeb80 4 client.16281 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 149.45, "await_for": 45.0}] 2024-06-10T02:59:45.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.274+0000 7f8a14bdeb80 1 -- 172.21.15.52:0/3040202147 --> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 149.45, "await_for": 45.0}) -- 0x7f89f8005270 con 0x5584ae470600 2024-06-10T02:59:45.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.276+0000 7f8a03fff700 1 --2- 172.21.15.52:0/3040202147 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x5584ae470600 0x5584ae4f27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:59:45.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.288+0000 7f8a03fff700 1 --2- 172.21.15.52:0/3040202147 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x5584ae470600 0x5584ae4f27b0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2af53eadea34e00 server_cookie=8baa4d9404ef8396 in_seq=0 out_seq=0 2024-06-10T02:59:45.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:45.290+0000 7f89e37fe700 10 client.16281 ms_handle_connect on v2:172.21.15.188:6838/310444836 2024-06-10T02:59:45.388 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:44 smithi120 ceph-mon[36634]: pgmap v600: 129 pgs: 129 active+clean; 557 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.9 MiB/s wr, 710 op/s 2024-06-10T02:59:45.565 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T02:59:46.976 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:46 smithi052 ceph-mon[32362]: pgmap v601: 129 pgs: 129 active+clean; 573 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.9 MiB/s wr, 670 op/s 2024-06-10T02:59:47.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:46 smithi188 ceph-mon[36599]: pgmap v601: 129 pgs: 129 active+clean; 573 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.9 MiB/s wr, 670 op/s 2024-06-10T02:59:47.249 INFO:tasks.workunit.client.0.smithi052.stdout: 96 843892 320 607635 168 440361 513 2024-06-10T02:59:47.388 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:46 smithi120 ceph-mon[36634]: pgmap v601: 129 pgs: 129 active+clean; 573 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.9 MiB/s wr, 670 op/s 2024-06-10T02:59:47.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.457+0000 7f60fcc96700 1 -- 172.21.15.52:0/3336298970 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f60f8101440 msgr2=0x7f60f810f330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:59:47.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.457+0000 7f60fcc96700 1 --2- 172.21.15.52:0/3336298970 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f60f8101440 0x7f60f810f330 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f60f000b0a0 tx=0x7f60f000b3b0 comp rx=0 tx=0).stop 2024-06-10T02:59:47.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.457+0000 7f60fcc96700 1 -- 172.21.15.52:0/3336298970 shutdown_connections 2024-06-10T02:59:47.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.457+0000 7f60fcc96700 1 --2- 172.21.15.52:0/3336298970 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f60f8101440 0x7f60f810f330 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:47.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.457+0000 7f60fcc96700 1 --2- 172.21.15.52:0/3336298970 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f60f8108510 0x7f60f81088f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:47.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.457+0000 7f60fcc96700 1 --2- 172.21.15.52:0/3336298970 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f60f8100a80 0x7f60f8100f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:47.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.457+0000 7f60fcc96700 1 -- 172.21.15.52:0/3336298970 >> 172.21.15.52:0/3336298970 conn(0x7f60f8077a00 msgr2=0x7f60f80fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:59:47.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.457+0000 7f60fcc96700 1 -- 172.21.15.52:0/3336298970 shutdown_connections 2024-06-10T02:59:47.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.457+0000 7f60fcc96700 1 -- 172.21.15.52:0/3336298970 wait complete. 2024-06-10T02:59:47.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.458+0000 7f60fcc96700 1 Processor -- start 2024-06-10T02:59:47.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.458+0000 7f60fcc96700 1 -- start start 2024-06-10T02:59:47.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.458+0000 7f60fcc96700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f60f8100a80 0x7f60f8101c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:59:47.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.458+0000 7f60fcc96700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f60f8101440 0x7f60f81021a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:59:47.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.458+0000 7f60fcc96700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f60f8108510 0x7f60f81026e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:59:47.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.458+0000 7f60fcc96700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f60f81127c0 con 0x7f60f8101440 2024-06-10T02:59:47.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.458+0000 7f60fcc96700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f60f8112640 con 0x7f60f8108510 2024-06-10T02:59:47.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.458+0000 7f60fcc96700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f60f8112940 con 0x7f60f8100a80 2024-06-10T02:59:47.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.458+0000 7f60f659c700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f60f8100a80 0x7f60f8101c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:59:47.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.458+0000 7f60f5d9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f60f8101440 0x7f60f81021a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:59:47.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.458+0000 7f60f659c700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f60f8100a80 0x7f60f8101c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.52:37356/0 (socket says 172.21.15.52:37356) 2024-06-10T02:59:47.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.458+0000 7f60f659c700 1 -- 172.21.15.52:0/2407253831 learned_addr learned my addr 172.21.15.52:0/2407253831 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:59:47.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.458+0000 7f60f6d9d700 1 --2- 172.21.15.52:0/2407253831 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f60f8108510 0x7f60f81026e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:59:47.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.458+0000 7f60f5d9b700 1 -- 172.21.15.52:0/2407253831 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f60f8100a80 msgr2=0x7f60f8101c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:59:47.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.458+0000 7f60f5d9b700 1 --2- 172.21.15.52:0/2407253831 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f60f8100a80 0x7f60f8101c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:47.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.458+0000 7f60f5d9b700 1 -- 172.21.15.52:0/2407253831 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f60f8108510 msgr2=0x7f60f81026e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:59:47.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.458+0000 7f60f5d9b700 1 --2- 172.21.15.52:0/2407253831 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f60f8108510 0x7f60f81026e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:47.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.458+0000 7f60f5d9b700 1 -- 172.21.15.52:0/2407253831 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60f81a7190 con 0x7f60f8101440 2024-06-10T02:59:47.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.458+0000 7f60f659c700 1 --2- 172.21.15.52:0/2407253831 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f60f8100a80 0x7f60f8101c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T02:59:47.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.459+0000 7f60f5d9b700 1 --2- 172.21.15.52:0/2407253831 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f60f8101440 0x7f60f81021a0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f60ec00e9f0 tx=0x7f60ec00ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:59:47.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.459+0000 7f60e77fe700 1 -- 172.21.15.52:0/2407253831 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60ec00c9c0 con 0x7f60f8101440 2024-06-10T02:59:47.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.459+0000 7f60fcc96700 1 -- 172.21.15.52:0/2407253831 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60f81a7480 con 0x7f60f8101440 2024-06-10T02:59:47.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.459+0000 7f60e77fe700 1 -- 172.21.15.52:0/2407253831 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f60ec00cb60 con 0x7f60f8101440 2024-06-10T02:59:47.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.459+0000 7f60e77fe700 1 -- 172.21.15.52:0/2407253831 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60ec018680 con 0x7f60f8101440 2024-06-10T02:59:47.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.459+0000 7f60fcc96700 1 -- 172.21.15.52:0/2407253831 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60f81a7a10 con 0x7f60f8101440 2024-06-10T02:59:47.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.462+0000 7f60e77fe700 1 -- 172.21.15.52:0/2407253831 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f60ec018820 con 0x7f60f8101440 2024-06-10T02:59:47.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.463+0000 7f60e77fe700 1 --2- 172.21.15.52:0/2407253831 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f60e00a2c20 0x7f60e00a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:59:47.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.463+0000 7f60f659c700 1 --2- 172.21.15.52:0/2407253831 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f60e00a2c20 0x7f60e00a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:59:47.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.463+0000 7f60e77fe700 1 -- 172.21.15.52:0/2407253831 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f60ec0cf500 con 0x7f60f8101440 2024-06-10T02:59:47.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.463+0000 7f6107519b80 1 Processor -- start 2024-06-10T02:59:47.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.463+0000 7f6107519b80 1 -- start start 2024-06-10T02:59:47.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.464+0000 7f60f659c700 1 --2- 172.21.15.52:0/2407253831 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f60e00a2c20 0x7f60e00a50e0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f60e8005950 tx=0x7f60e800b500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:59:47.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.464+0000 7f6107519b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5612f44e0640 0x5612f44e0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:59:47.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.464+0000 7f6107519b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5612f44e0f60 0x5612f44da9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:59:47.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.464+0000 7f6107519b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5612f44daf20 0x5612f44db340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:59:47.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.464+0000 7f6107519b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5612f442c810 con 0x5612f44daf20 2024-06-10T02:59:47.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.464+0000 7f6107519b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5612f43d4720 con 0x5612f44e0640 2024-06-10T02:59:47.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.464+0000 7f6107519b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5612f44287d0 con 0x5612f44e0f60 2024-06-10T02:59:47.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.464+0000 7f60f5d9b700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5612f44e0f60 0x5612f44da9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:59:47.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.464+0000 7f60f6d9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5612f44daf20 0x5612f44db340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:59:47.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.464+0000 7f60f5d9b700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5612f44e0f60 0x5612f44da9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.52:37360/0 (socket says 172.21.15.52:37360) 2024-06-10T02:59:47.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.464+0000 7f60f5d9b700 1 -- 172.21.15.52:0/2431136178 learned_addr learned my addr 172.21.15.52:0/2431136178 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:59:47.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.464+0000 7f60f659c700 1 --2- 172.21.15.52:0/2431136178 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5612f44e0640 0x5612f44e0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:59:47.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.464+0000 7f60f5d9b700 1 -- 172.21.15.52:0/2431136178 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5612f44e0640 msgr2=0x5612f44e0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:59:47.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.464+0000 7f60f5d9b700 1 --2- 172.21.15.52:0/2431136178 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5612f44e0640 0x5612f44e0a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:47.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.464+0000 7f60f5d9b700 1 -- 172.21.15.52:0/2431136178 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5612f44daf20 msgr2=0x5612f44db340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:59:47.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.464+0000 7f60f5d9b700 1 --2- 172.21.15.52:0/2431136178 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5612f44daf20 0x5612f44db340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:47.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.464+0000 7f60f5d9b700 1 -- 172.21.15.52:0/2431136178 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5612f43caf50 con 0x5612f44e0f60 2024-06-10T02:59:47.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.464+0000 7f60f659c700 1 --2- 172.21.15.52:0/2431136178 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5612f44e0640 0x5612f44e0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T02:59:47.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.464+0000 7f60f5d9b700 1 --2- 172.21.15.52:0/2431136178 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5612f44e0f60 0x5612f44da9e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f60ec0a40f0 tx=0x7f60ec00cd10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=420b0ca33eaa5a75 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:59:47.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.464+0000 7f60e4ff9700 1 -- 172.21.15.52:0/2431136178 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60ec0cf070 con 0x5612f44e0f60 2024-06-10T02:59:47.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.464+0000 7f60e4ff9700 1 -- 172.21.15.52:0/2431136178 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f60ec09b9b0 con 0x5612f44e0f60 2024-06-10T02:59:47.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.464+0000 7f60e4ff9700 1 -- 172.21.15.52:0/2431136178 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60ec09be70 con 0x5612f44e0f60 2024-06-10T02:59:47.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.465+0000 7f6107519b80 1 -- 172.21.15.52:0/2431136178 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5612f44e0f60 msgr2=0x5612f44da9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:59:47.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.465+0000 7f6107519b80 1 --2- 172.21.15.52:0/2431136178 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5612f44e0f60 0x5612f44da9e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f60ec0a40f0 tx=0x7f60ec00cd10 comp rx=0 tx=0).stop 2024-06-10T02:59:47.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.465+0000 7f6107519b80 1 -- 172.21.15.52:0/2431136178 shutdown_connections 2024-06-10T02:59:47.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.465+0000 7f6107519b80 1 --2- 172.21.15.52:0/2431136178 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5612f44daf20 0x5612f44db340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:47.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.465+0000 7f6107519b80 1 --2- 172.21.15.52:0/2431136178 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5612f44e0640 0x5612f44e0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:47.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.465+0000 7f6107519b80 1 --2- 172.21.15.52:0/2431136178 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5612f44e0f60 0x5612f44da9e0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:47.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.465+0000 7f6107519b80 1 -- 172.21.15.52:0/2431136178 >> 172.21.15.52:0/2431136178 conn(0x5612f4405cc0 msgr2=0x5612f44d5850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:59:47.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.465+0000 7f6107519b80 1 -- 172.21.15.52:0/2431136178 shutdown_connections 2024-06-10T02:59:47.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.465+0000 7f6107519b80 1 -- 172.21.15.52:0/2431136178 wait complete. 2024-06-10T02:59:47.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.466+0000 7f6107519b80 1 -- start start 2024-06-10T02:59:47.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.466+0000 7f6107519b80 1 Processor -- start 2024-06-10T02:59:47.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.466+0000 7f6107519b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5612f44daf20 0x5612f44e6d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:59:47.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.466+0000 7f6107519b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5612f44e0640 0x5612f44e4090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:59:47.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.466+0000 7f6107519b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5612f44e0f60 0x5612f44e45d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:59:47.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.466+0000 7f6107519b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5612f44e79d0 con 0x5612f44e0640 2024-06-10T02:59:47.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.466+0000 7f6107519b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5612f43ccbf0 con 0x5612f44e0f60 2024-06-10T02:59:47.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.466+0000 7f6107519b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5612f44e7b50 con 0x5612f44daf20 2024-06-10T02:59:47.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.466+0000 7f6107519b80 10 client.0 _ll_register_callbacks cb 0x5612f43c6920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T02:59:47.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.466+0000 7f60f5d9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5612f44e0640 0x5612f44e4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:59:47.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.466+0000 7f60f659c700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5612f44daf20 0x5612f44e6d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:59:47.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.466+0000 7f60f659c700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5612f44daf20 0x5612f44e6d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.52:37372/0 (socket says 172.21.15.52:37372) 2024-06-10T02:59:47.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.466+0000 7f60f659c700 1 -- 172.21.15.52:0/2865454280 learned_addr learned my addr 172.21.15.52:0/2865454280 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:59:47.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.466+0000 7f60f6d9d700 1 --2- 172.21.15.52:0/2865454280 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5612f44e0f60 0x5612f44e45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:59:47.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.467+0000 7f60f659c700 1 -- 172.21.15.52:0/2865454280 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5612f44e0f60 msgr2=0x5612f44e45d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:59:47.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.467+0000 7f60f659c700 1 --2- 172.21.15.52:0/2865454280 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5612f44e0f60 0x5612f44e45d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:47.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.467+0000 7f60f659c700 1 -- 172.21.15.52:0/2865454280 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5612f44e0640 msgr2=0x5612f44e4090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:59:47.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.467+0000 7f60f659c700 1 --2- 172.21.15.52:0/2865454280 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5612f44e0640 0x5612f44e4090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:47.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.467+0000 7f60f659c700 1 -- 172.21.15.52:0/2865454280 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5612f437b8d0 con 0x5612f44daf20 2024-06-10T02:59:47.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.467+0000 7f60f5d9b700 1 --2- 172.21.15.52:0/2865454280 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5612f44e0640 0x5612f44e4090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T02:59:47.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.467+0000 7f60f659c700 1 --2- 172.21.15.52:0/2865454280 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5612f44daf20 0x5612f44e6d90 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f60e80119d0 tx=0x7f60e80047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f73a2d32bbf5f979 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:59:47.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.467+0000 7f60ddffb700 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-06-10T02:59:47.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.467+0000 7f60ddffb700 1 -- 172.21.15.52:0/2865454280 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60e801e850 con 0x5612f44daf20 2024-06-10T02:59:47.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.467+0000 7f60ddffb700 1 -- 172.21.15.52:0/2865454280 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f60e801e9f0 con 0x5612f44daf20 2024-06-10T02:59:47.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.467+0000 7f6107519b80 1 -- 172.21.15.52:0/2865454280 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f60ec09b9b0 con 0x5612f44daf20 2024-06-10T02:59:47.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.467+0000 7f60ddffb700 1 -- 172.21.15.52:0/2865454280 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f60e801ebf0 con 0x5612f44daf20 2024-06-10T02:59:47.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.467+0000 7f6107519b80 10 client.25861 fetch_fsmap learned FSMap version 36 2024-06-10T02:59:47.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.467+0000 7f6107519b80 1 -- 172.21.15.52:0/2865454280 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5612f44f23e0 con 0x5612f44daf20 2024-06-10T02:59:47.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.467+0000 7f60ddffb700 1 -- 172.21.15.52:0/2865454280 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f60ec09b9b0 con 0x5612f44daf20 2024-06-10T02:59:47.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.468+0000 7f60ddffb700 1 -- 172.21.15.52:0/2865454280 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 36) ==== 8378+0+0 (secure 0 0 0) 0x7f60e801f430 con 0x5612f44daf20 2024-06-10T02:59:47.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.468+0000 7f6107519b80 10 client.25861 fetch_fsmap finished waiting for FSMap version 36 2024-06-10T02:59:47.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.468+0000 7f6107519b80 10 client.25861 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T02:59:47.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.468+0000 7f6107519b80 20 client.25861 populate_metadata read hostname 'smithi052' 2024-06-10T02:59:47.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.468+0000 7f6107519b80 1 --2- 172.21.15.52:0/2865454280 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x5612f44f4a20 0x5612f44f6e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:59:47.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.468+0000 7f6107519b80 4 client.25861 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T02:59:47.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.468+0000 7f6107519b80 1 -- 172.21.15.52:0/2865454280 --> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5612f4439d10 con 0x5612f44f4a20 2024-06-10T02:59:47.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.468+0000 7f60f5d9b700 1 --2- 172.21.15.52:0/2865454280 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x5612f44f4a20 0x5612f44f6e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:59:47.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.468+0000 7f60f5d9b700 1 --2- 172.21.15.52:0/2865454280 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x5612f44f4a20 0x5612f44f6e30 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=67ba570fd66bcb9f server_cookie=347e40fec7c1ffdb in_seq=0 out_seq=0 2024-06-10T02:59:47.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.468+0000 7f60ddffb700 10 client.25861 ms_handle_connect on v2:172.21.15.188:6838/310444836 2024-06-10T02:59:47.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.469+0000 7f60ddffb700 1 -- 172.21.15.52:0/2865454280 <== mds.0 v2:172.21.15.188:6838/310444836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5612f4439d10 con 0x5612f44f4a20 2024-06-10T02:59:47.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.469+0000 7f60ddffb700 10 client.25861 handle_command_reply: tid=0 2024-06-10T02:59:47.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.469+0000 7f6107519b80 1 client.25861 shutdown 2024-06-10T02:59:47.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.469+0000 7f6107519b80 1 -- 172.21.15.52:0/2865454280 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5612f44daf20 msgr2=0x5612f44e6d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:59:47.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.469+0000 7f6107519b80 1 --2- 172.21.15.52:0/2865454280 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5612f44daf20 0x5612f44e6d90 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f60e80119d0 tx=0x7f60e80047c0 comp rx=0 tx=0).stop 2024-06-10T02:59:47.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.469+0000 7f6107519b80 1 -- 172.21.15.52:0/2865454280 shutdown_connections 2024-06-10T02:59:47.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.469+0000 7f6107519b80 1 --2- 172.21.15.52:0/2865454280 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x5612f44f4a20 0x5612f44f6e30 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:47.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.469+0000 7f6107519b80 1 --2- 172.21.15.52:0/2865454280 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5612f44e0f60 0x5612f44e45d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:47.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.469+0000 7f6107519b80 1 --2- 172.21.15.52:0/2865454280 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5612f44daf20 0x5612f44e6d90 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:47.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.469+0000 7f60ddffb700 0 client.25861 ms_handle_reset on v2:172.21.15.188:6838/310444836 2024-06-10T02:59:47.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.469+0000 7f6107519b80 1 --2- 172.21.15.52:0/2865454280 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5612f44e0640 0x5612f44e4090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:47.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.469+0000 7f6107519b80 1 -- 172.21.15.52:0/2865454280 >> 172.21.15.52:0/2865454280 conn(0x5612f4405cc0 msgr2=0x5612f44d55f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:59:47.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.469+0000 7f6107519b80 1 -- 172.21.15.52:0/2865454280 shutdown_connections 2024-06-10T02:59:47.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.469+0000 7f6107519b80 1 -- 172.21.15.52:0/2865454280 wait complete. 2024-06-10T02:59:47.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.469+0000 7f6107519b80 20 client.25861 trim_cache size 0 max 16384 2024-06-10T02:59:47.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.493+0000 7f6107519b80 1 Processor -- start 2024-06-10T02:59:47.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.493+0000 7f6107519b80 1 -- start start 2024-06-10T02:59:47.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.494+0000 7f6107519b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5612f44daf20 0x5612f44da9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:59:47.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.494+0000 7f6107519b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5612f44e0640 0x5612f44f0c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:59:47.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.494+0000 7f6107519b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5612f44e0f60 0x5612f44e1a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:59:47.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.494+0000 7f6107519b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f60f81b0610 con 0x5612f44daf20 2024-06-10T02:59:47.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.494+0000 7f6107519b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f60f81b2210 con 0x5612f44e0f60 2024-06-10T02:59:47.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.494+0000 7f6107519b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f60f81b0ce0 con 0x5612f44e0640 2024-06-10T02:59:47.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.494+0000 7f60f659c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5612f44daf20 0x5612f44da9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:59:47.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.494+0000 7f60f659c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5612f44daf20 0x5612f44da9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57374/0 (socket says 172.21.15.52:57374) 2024-06-10T02:59:47.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.494+0000 7f60f659c700 1 -- 172.21.15.52:0/4216211159 learned_addr learned my addr 172.21.15.52:0/4216211159 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:59:47.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.494+0000 7f60f5d9b700 1 --2- 172.21.15.52:0/4216211159 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5612f44e0640 0x5612f44f0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:59:47.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.494+0000 7f60f6d9d700 1 --2- 172.21.15.52:0/4216211159 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5612f44e0f60 0x5612f44e1a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:59:47.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.494+0000 7f60f659c700 1 -- 172.21.15.52:0/4216211159 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5612f44e0640 msgr2=0x5612f44f0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:59:47.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.494+0000 7f60f659c700 1 --2- 172.21.15.52:0/4216211159 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5612f44e0640 0x5612f44f0c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:47.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.494+0000 7f60f659c700 1 -- 172.21.15.52:0/4216211159 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5612f44e0f60 msgr2=0x5612f44e1a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:59:47.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.494+0000 7f60f659c700 1 --2- 172.21.15.52:0/4216211159 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5612f44e0f60 0x5612f44e1a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:47.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.494+0000 7f60f659c700 1 -- 172.21.15.52:0/4216211159 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5612f44d7050 con 0x5612f44daf20 2024-06-10T02:59:47.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.494+0000 7f60f5d9b700 1 --2- 172.21.15.52:0/4216211159 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5612f44e0640 0x5612f44f0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T02:59:47.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.494+0000 7f60f6d9d700 1 --2- 172.21.15.52:0/4216211159 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5612f44e0f60 0x5612f44e1a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T02:59:47.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.494+0000 7f60f659c700 1 --2- 172.21.15.52:0/4216211159 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5612f44daf20 0x5612f44da9e0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f60e800bf20 tx=0x7f60e80046f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=636a6851302548a2 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:59:47.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.494+0000 7f60dffff700 1 -- 172.21.15.52:0/4216211159 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60e801fd10 con 0x5612f44daf20 2024-06-10T02:59:47.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.495+0000 7f60dffff700 1 -- 172.21.15.52:0/4216211159 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f60e801f430 con 0x5612f44daf20 2024-06-10T02:59:47.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.495+0000 7f60dffff700 1 -- 172.21.15.52:0/4216211159 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60e80038e0 con 0x5612f44daf20 2024-06-10T02:59:47.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.495+0000 7f6107519b80 1 -- 172.21.15.52:0/4216211159 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5612f44daf20 msgr2=0x5612f44da9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:59:47.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.495+0000 7f6107519b80 1 --2- 172.21.15.52:0/4216211159 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5612f44daf20 0x5612f44da9e0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f60e800bf20 tx=0x7f60e80046f0 comp rx=0 tx=0).stop 2024-06-10T02:59:47.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.495+0000 7f6107519b80 1 -- 172.21.15.52:0/4216211159 shutdown_connections 2024-06-10T02:59:47.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.495+0000 7f6107519b80 1 --2- 172.21.15.52:0/4216211159 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5612f44e0f60 0x5612f44e1a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:47.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.495+0000 7f6107519b80 1 --2- 172.21.15.52:0/4216211159 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5612f44daf20 0x5612f44da9e0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:47.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.495+0000 7f6107519b80 1 --2- 172.21.15.52:0/4216211159 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5612f44e0640 0x5612f44f0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:47.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.495+0000 7f6107519b80 1 -- 172.21.15.52:0/4216211159 >> 172.21.15.52:0/4216211159 conn(0x5612f4405cc0 msgr2=0x5612f44e1380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:59:47.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.495+0000 7f6107519b80 1 -- 172.21.15.52:0/4216211159 shutdown_connections 2024-06-10T02:59:47.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.495+0000 7f6107519b80 1 -- 172.21.15.52:0/4216211159 wait complete. 2024-06-10T02:59:47.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.496+0000 7f6107519b80 1 -- start start 2024-06-10T02:59:47.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.496+0000 7f6107519b80 1 Processor -- start 2024-06-10T02:59:47.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.496+0000 7f6107519b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5612f44daf20 0x5612f44ed1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:59:47.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.496+0000 7f6107519b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5612f44e0640 0x5612f44fbd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:59:47.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.496+0000 7f6107519b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5612f44e0f60 0x5612f44fc250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:59:47.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.496+0000 7f6107519b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f60f81b1b40 con 0x5612f44e0640 2024-06-10T02:59:47.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.496+0000 7f6107519b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f60f81b1470 con 0x5612f44daf20 2024-06-10T02:59:47.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.496+0000 7f6107519b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f60f81ae5b0 con 0x5612f44e0f60 2024-06-10T02:59:47.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.496+0000 7f6107519b80 10 client.0 _ll_register_callbacks cb 0x5612f43c6920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T02:59:47.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.496+0000 7f60f5d9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5612f44e0640 0x5612f44fbd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:59:47.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.496+0000 7f60f5d9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5612f44e0640 0x5612f44fbd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57384/0 (socket says 172.21.15.52:57384) 2024-06-10T02:59:47.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.496+0000 7f60f5d9b700 1 -- 172.21.15.52:0/286867956 learned_addr learned my addr 172.21.15.52:0/286867956 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T02:59:47.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.496+0000 7f60f6d9d700 1 --2- 172.21.15.52:0/286867956 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5612f44e0f60 0x5612f44fc250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:59:47.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.496+0000 7f60f659c700 1 --2- 172.21.15.52:0/286867956 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5612f44daf20 0x5612f44ed1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:59:47.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.496+0000 7f60f5d9b700 1 -- 172.21.15.52:0/286867956 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5612f44e0f60 msgr2=0x5612f44fc250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:59:47.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.496+0000 7f60f5d9b700 1 --2- 172.21.15.52:0/286867956 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5612f44e0f60 0x5612f44fc250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:47.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.496+0000 7f60f5d9b700 1 -- 172.21.15.52:0/286867956 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5612f44daf20 msgr2=0x5612f44ed1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T02:59:47.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.496+0000 7f60f5d9b700 1 --2- 172.21.15.52:0/286867956 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5612f44daf20 0x5612f44ed1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:47.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.496+0000 7f60f5d9b700 1 -- 172.21.15.52:0/286867956 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5612f44fc790 con 0x5612f44e0640 2024-06-10T02:59:47.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.497+0000 7f60f5d9b700 1 --2- 172.21.15.52:0/286867956 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5612f44e0640 0x5612f44fbd10 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f60ec0ccfc0 tx=0x7f60ec010ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10dbab22c812b410 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T02:59:47.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.497+0000 7f60ddffb700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-10T02:59:47.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.497+0000 7f60ddffb700 1 -- 172.21.15.52:0/286867956 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60ec0cf040 con 0x5612f44e0640 2024-06-10T02:59:47.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.497+0000 7f60ddffb700 1 -- 172.21.15.52:0/286867956 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f60ec0a4190 con 0x5612f44e0640 2024-06-10T02:59:47.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.497+0000 7f6107519b80 1 -- 172.21.15.52:0/286867956 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f60e801f430 con 0x5612f44e0640 2024-06-10T02:59:47.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.497+0000 7f60ddffb700 1 -- 172.21.15.52:0/286867956 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f60ec09b9b0 con 0x5612f44e0640 2024-06-10T02:59:47.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.497+0000 7f6107519b80 10 client.16299 fetch_fsmap learned FSMap version 36 2024-06-10T02:59:47.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.497+0000 7f6107519b80 1 -- 172.21.15.52:0/286867956 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5612f455f430 con 0x5612f44e0640 2024-06-10T02:59:47.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.497+0000 7f60ddffb700 1 -- 172.21.15.52:0/286867956 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f60e801f430 con 0x5612f44e0640 2024-06-10T02:59:47.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.497+0000 7f60ddffb700 1 -- 172.21.15.52:0/286867956 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 36) ==== 8378+0+0 (secure 0 0 0) 0x7f60ec064050 con 0x5612f44e0640 2024-06-10T02:59:47.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.497+0000 7f6107519b80 10 client.16299 fetch_fsmap finished waiting for FSMap version 36 2024-06-10T02:59:47.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.497+0000 7f6107519b80 10 client.16299 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T02:59:47.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.497+0000 7f6107519b80 20 client.16299 populate_metadata read hostname 'smithi052' 2024-06-10T02:59:47.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.497+0000 7f6107519b80 1 --2- 172.21.15.52:0/286867956 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x5612f455f660 0x5612f4561a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T02:59:47.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.497+0000 7f6107519b80 4 client.16299 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-10T02:59:47.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.497+0000 7f6107519b80 1 -- 172.21.15.52:0/286867956 --> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5612f4434290 con 0x5612f455f660 2024-06-10T02:59:47.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.498+0000 7f60f659c700 1 --2- 172.21.15.52:0/286867956 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x5612f455f660 0x5612f4561a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T02:59:47.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.498+0000 7f60ddffb700 10 client.16299 ms_handle_connect on v2:172.21.15.188:6838/310444836 2024-06-10T02:59:47.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.498+0000 7f60f659c700 1 --2- 172.21.15.52:0/286867956 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x5612f455f660 0x5612f4561a50 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f1b235f08f217313 server_cookie=b15b1d1e643fc96e in_seq=0 out_seq=0 2024-06-10T02:59:47.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.498+0000 7f60ddffb700 1 -- 172.21.15.52:0/286867956 <== mds.0 v2:172.21.15.188:6838/310444836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5612f4434290 con 0x5612f455f660 2024-06-10T02:59:47.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.498+0000 7f60ddffb700 10 client.16299 handle_command_reply: tid=0 2024-06-10T02:59:47.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.498+0000 7f6107519b80 1 client.16299 shutdown 2024-06-10T02:59:47.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.498+0000 7f6107519b80 1 -- 172.21.15.52:0/286867956 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5612f44e0640 msgr2=0x5612f44fbd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:59:47.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.498+0000 7f6107519b80 1 --2- 172.21.15.52:0/286867956 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5612f44e0640 0x5612f44fbd10 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f60ec0ccfc0 tx=0x7f60ec010ca0 comp rx=0 tx=0).stop 2024-06-10T02:59:47.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.498+0000 7f6107519b80 1 -- 172.21.15.52:0/286867956 shutdown_connections 2024-06-10T02:59:47.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.498+0000 7f6107519b80 1 --2- 172.21.15.52:0/286867956 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x5612f455f660 0x5612f4561a50 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:47.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.498+0000 7f6107519b80 1 --2- 172.21.15.52:0/286867956 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5612f44e0f60 0x5612f44fc250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:47.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.498+0000 7f6107519b80 1 --2- 172.21.15.52:0/286867956 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5612f44daf20 0x5612f44ed1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:47.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.498+0000 7f60ddffb700 0 client.16299 ms_handle_reset on v2:172.21.15.188:6838/310444836 2024-06-10T02:59:47.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.498+0000 7f6107519b80 1 --2- 172.21.15.52:0/286867956 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5612f44e0640 0x5612f44fbd10 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:47.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.498+0000 7f6107519b80 1 -- 172.21.15.52:0/286867956 >> 172.21.15.52:0/286867956 conn(0x5612f4405cc0 msgr2=0x5612f44e1340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:59:47.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.499+0000 7f6107519b80 1 -- 172.21.15.52:0/286867956 shutdown_connections 2024-06-10T02:59:47.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.499+0000 7f6107519b80 1 -- 172.21.15.52:0/286867956 wait complete. 2024-06-10T02:59:47.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.499+0000 7f6107519b80 20 client.16299 trim_cache size 0 max 16384 2024-06-10T02:59:47.509 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-10T02:59:47.509 INFO:teuthology.orchestra.run.smithi052.stdout: "status": "IDLE (waiting for more scrubs)", 2024-06-10T02:59:47.509 INFO:teuthology.orchestra.run.smithi052.stdout: "scrubs": { 2024-06-10T02:59:47.509 INFO:teuthology.orchestra.run.smithi052.stdout: "4f014e98-4252-4ddf-9b48-2b8c82701461": { 2024-06-10T02:59:47.509 INFO:teuthology.orchestra.run.smithi052.stdout: "path": "/", 2024-06-10T02:59:47.509 INFO:teuthology.orchestra.run.smithi052.stdout: "tag": "4f014e98-4252-4ddf-9b48-2b8c82701461", 2024-06-10T02:59:47.509 INFO:teuthology.orchestra.run.smithi052.stdout: "options": "recursive,force" 2024-06-10T02:59:47.509 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T02:59:47.509 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T02:59:47.509 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-10T02:59:47.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.499+0000 7f60fcc96700 1 -- 172.21.15.52:0/2407253831 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f60e00a2c20 msgr2=0x7f60e00a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:59:47.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.499+0000 7f60fcc96700 1 --2- 172.21.15.52:0/2407253831 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f60e00a2c20 0x7f60e00a50e0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f60e8005950 tx=0x7f60e800b500 comp rx=0 tx=0).stop 2024-06-10T02:59:47.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.499+0000 7f60fcc96700 1 -- 172.21.15.52:0/2407253831 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f60f8101440 msgr2=0x7f60f81021a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T02:59:47.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.499+0000 7f60fcc96700 1 --2- 172.21.15.52:0/2407253831 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f60f8101440 0x7f60f81021a0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f60ec00e9f0 tx=0x7f60ec00ed00 comp rx=0 tx=0).stop 2024-06-10T02:59:47.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.499+0000 7f60fcc96700 1 -- 172.21.15.52:0/2407253831 shutdown_connections 2024-06-10T02:59:47.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.499+0000 7f60fcc96700 1 --2- 172.21.15.52:0/2407253831 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f60f8108510 0x7f60f81026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:47.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.499+0000 7f60fcc96700 1 --2- 172.21.15.52:0/2407253831 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f60f8100a80 0x7f60f8101c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:47.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.499+0000 7f60fcc96700 1 --2- 172.21.15.52:0/2407253831 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f60e00a2c20 0x7f60e00a50e0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:47.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.499+0000 7f60fcc96700 1 --2- 172.21.15.52:0/2407253831 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f60f8101440 0x7f60f81021a0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T02:59:47.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.499+0000 7f60fcc96700 1 -- 172.21.15.52:0/2407253831 >> 172.21.15.52:0/2407253831 conn(0x7f60f8077a00 msgr2=0x7f60f80ff130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T02:59:47.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.499+0000 7f60fcc96700 1 -- 172.21.15.52:0/2407253831 shutdown_connections 2024-06-10T02:59:47.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T02:59:47.499+0000 7f60fcc96700 1 -- 172.21.15.52:0/2407253831 wait complete. 2024-06-10T02:59:48.950 INFO:tasks.cephfs.filesystem:scrub status for tag:4f014e98-4252-4ddf-9b48-2b8c82701461 - {'path': '/', 'tag': '4f014e98-4252-4ddf-9b48-2b8c82701461', 'options': 'recursive,force'} 2024-06-10T02:59:49.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:48 smithi188 ceph-mon[36599]: pgmap v602: 129 pgs: 129 active+clean; 574 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.9 MiB/s wr, 689 op/s 2024-06-10T02:59:49.226 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:48 smithi052 ceph-mon[32362]: pgmap v602: 129 pgs: 129 active+clean; 574 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.9 MiB/s wr, 689 op/s 2024-06-10T02:59:49.388 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:48 smithi120 ceph-mon[36634]: pgmap v602: 129 pgs: 129 active+clean; 574 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.9 MiB/s wr, 689 op/s 2024-06-10T02:59:50.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:49 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T02:59:50.226 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:49 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T02:59:50.390 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:49 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T02:59:51.205 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:50 smithi188 ceph-mon[36599]: pgmap v603: 129 pgs: 129 active+clean; 574 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.3 MiB/s wr, 658 op/s 2024-06-10T02:59:51.206 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:50 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-10T02:59:51.206 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:50 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-10T02:59:51.206 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:50 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-10T02:59:51.206 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:50 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-06-10T02:59:51.206 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:50 smithi188 ceph-mon[36599]: Adjusting osd_memory_target on smithi188 to 176.2M 2024-06-10T02:59:51.206 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:50 smithi188 ceph-mon[36599]: Unable to set osd_memory_target on smithi188 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-06-10T02:59:51.206 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:50 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:59:51.206 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:50 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T02:59:51.206 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:50 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:59:51.226 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:50 smithi052 ceph-mon[32362]: pgmap v603: 129 pgs: 129 active+clean; 574 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.3 MiB/s wr, 658 op/s 2024-06-10T02:59:51.227 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:50 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-10T02:59:51.227 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:50 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-10T02:59:51.227 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:50 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-10T02:59:51.227 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:50 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-06-10T02:59:51.227 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:50 smithi052 ceph-mon[32362]: Adjusting osd_memory_target on smithi188 to 176.2M 2024-06-10T02:59:51.227 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:50 smithi052 ceph-mon[32362]: Unable to set osd_memory_target on smithi188 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-06-10T02:59:51.227 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:50 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:59:51.227 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:50 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T02:59:51.227 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:50 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:59:51.388 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:50 smithi120 ceph-mon[36634]: pgmap v603: 129 pgs: 129 active+clean; 574 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.3 MiB/s wr, 658 op/s 2024-06-10T02:59:51.388 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:50 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-10T02:59:51.388 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:50 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-10T02:59:51.388 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:50 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-10T02:59:51.388 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:50 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-06-10T02:59:51.389 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:50 smithi120 ceph-mon[36634]: Adjusting osd_memory_target on smithi188 to 176.2M 2024-06-10T02:59:51.389 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:50 smithi120 ceph-mon[36634]: Unable to set osd_memory_target on smithi188 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-06-10T02:59:51.389 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:50 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T02:59:51.389 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:50 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T02:59:51.389 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:50 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T02:59:52.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:51 smithi188 ceph-mon[36599]: pgmap v604: 129 pgs: 129 active+clean; 573 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.6 MiB/s wr, 572 op/s 2024-06-10T02:59:52.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:51 smithi188 ceph-mon[36599]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:52.227 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:51 smithi052 ceph-mon[32362]: pgmap v604: 129 pgs: 129 active+clean; 573 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.6 MiB/s wr, 572 op/s 2024-06-10T02:59:52.227 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:51 smithi052 ceph-mon[32362]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:52.388 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:51 smithi120 ceph-mon[36634]: pgmap v604: 129 pgs: 129 active+clean; 573 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.6 MiB/s wr, 572 op/s 2024-06-10T02:59:52.388 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:51 smithi120 ceph-mon[36634]: scrub summary: idle+waiting paths [/] 2024-06-10T02:59:54.477 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:54 smithi052 ceph-mon[32362]: pgmap v605: 129 pgs: 129 active+clean; 573 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.0 MiB/s wr, 628 op/s 2024-06-10T02:59:54.638 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:54 smithi120 ceph-mon[36634]: pgmap v605: 129 pgs: 129 active+clean; 573 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.0 MiB/s wr, 628 op/s 2024-06-10T02:59:54.704 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:54 smithi188 ceph-mon[36599]: pgmap v605: 129 pgs: 129 active+clean; 573 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.0 MiB/s wr, 628 op/s 2024-06-10T02:59:56.476 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:56 smithi052 ceph-mon[32362]: pgmap v606: 129 pgs: 129 active+clean; 572 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.2 MiB/s wr, 517 op/s 2024-06-10T02:59:56.638 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:56 smithi120 ceph-mon[36634]: pgmap v606: 129 pgs: 129 active+clean; 572 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.2 MiB/s wr, 517 op/s 2024-06-10T02:59:56.703 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:56 smithi188 ceph-mon[36599]: pgmap v606: 129 pgs: 129 active+clean; 572 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.2 MiB/s wr, 517 op/s 2024-06-10T02:59:57.249 INFO:tasks.workunit.client.0.smithi052.stdout: 96 175 0 142 0 114 0 2024-06-10T02:59:58.477 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 02:59:58 smithi052 ceph-mon[32362]: pgmap v607: 129 pgs: 129 active+clean; 569 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 536 op/s 2024-06-10T02:59:58.638 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 02:59:58 smithi120 ceph-mon[36634]: pgmap v607: 129 pgs: 129 active+clean; 569 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 536 op/s 2024-06-10T02:59:58.703 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 02:59:58 smithi188 ceph-mon[36599]: pgmap v607: 129 pgs: 129 active+clean; 569 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 536 op/s 2024-06-10T03:00:00.418 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:00.418+0000 7f89e37fe700 1 -- 172.21.15.52:0/3040202147 <== mds.0 v2:172.21.15.188:6838/310444836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x7f89f8005270 con 0x5584ae470600 2024-06-10T03:00:00.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:00.418+0000 7f89e37fe700 10 client.16281 handle_command_reply: tid=0 2024-06-10T03:00:00.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:00.418+0000 7f8a14bdeb80 1 client.16281 shutdown 2024-06-10T03:00:00.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:00.418+0000 7f8a14bdeb80 1 -- 172.21.15.52:0/3040202147 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5584ae46a470 msgr2=0x5584ae479fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:00:00.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:00.418+0000 7f8a14bdeb80 1 --2- 172.21.15.52:0/3040202147 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5584ae46a470 0x5584ae479fc0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f89fc0aa210 tx=0x7f89fc01a970 comp rx=0 tx=0).stop 2024-06-10T03:00:00.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:00.418+0000 7f8a14bdeb80 1 -- 172.21.15.52:0/3040202147 shutdown_connections 2024-06-10T03:00:00.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:00.418+0000 7f8a14bdeb80 1 --2- 172.21.15.52:0/3040202147 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x5584ae470600 0x5584ae4f27b0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:00.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:00.418+0000 7f8a14bdeb80 1 --2- 172.21.15.52:0/3040202147 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5584ae46fce0 0x5584ae4eea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:00.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:00.418+0000 7f8a14bdeb80 1 --2- 172.21.15.52:0/3040202147 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584ae394e60 0x5584ae479a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:00.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:00.418+0000 7f8a14bdeb80 1 --2- 172.21.15.52:0/3040202147 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5584ae46a470 0x5584ae479fc0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:00.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:00.418+0000 7f8a14bdeb80 1 -- 172.21.15.52:0/3040202147 >> 172.21.15.52:0/3040202147 conn(0x5584ae46df60 msgr2=0x5584ae475720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:00:00.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:00.418+0000 7f89e37fe700 0 client.16281 ms_handle_reset on v2:172.21.15.188:6838/310444836 2024-06-10T03:00:00.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:00.418+0000 7f8a14bdeb80 1 -- 172.21.15.52:0/3040202147 shutdown_connections 2024-06-10T03:00:00.420 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:00.418+0000 7f8a14bdeb80 1 -- 172.21.15.52:0/3040202147 wait complete. 2024-06-10T03:00:00.420 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:00.418+0000 7f8a14bdeb80 20 client.16281 trim_cache size 0 max 16384 2024-06-10T03:00:00.420 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-10T03:00:00.420 INFO:teuthology.orchestra.run.smithi052.stdout: "epoch": 36, 2024-06-10T03:00:00.420 INFO:teuthology.orchestra.run.smithi052.stdout: "leader": 24484, 2024-06-10T03:00:00.420 INFO:teuthology.orchestra.run.smithi052.stdout: "set_version": 14, 2024-06-10T03:00:00.420 INFO:teuthology.orchestra.run.smithi052.stdout: "sets": { 2024-06-10T03:00:00.420 INFO:teuthology.orchestra.run.smithi052.stdout: "3240692c": { 2024-06-10T03:00:00.420 INFO:teuthology.orchestra.run.smithi052.stdout: "version": 14, 2024-06-10T03:00:00.421 INFO:teuthology.orchestra.run.smithi052.stdout: "age_ref": 0.0, 2024-06-10T03:00:00.421 INFO:teuthology.orchestra.run.smithi052.stdout: "state": { 2024-06-10T03:00:00.421 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "QUIESCED", 2024-06-10T03:00:00.421 INFO:teuthology.orchestra.run.smithi052.stdout: "age": 0.0 2024-06-10T03:00:00.421 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:00:00.421 INFO:teuthology.orchestra.run.smithi052.stdout: "timeout": 90.0, 2024-06-10T03:00:00.421 INFO:teuthology.orchestra.run.smithi052.stdout: "expiration": 149.4, 2024-06-10T03:00:00.421 INFO:teuthology.orchestra.run.smithi052.stdout: "members": { 2024-06-10T03:00:00.421 INFO:teuthology.orchestra.run.smithi052.stdout: "file:/": { 2024-06-10T03:00:00.421 INFO:teuthology.orchestra.run.smithi052.stdout: "excluded": false, 2024-06-10T03:00:00.421 INFO:teuthology.orchestra.run.smithi052.stdout: "state": { 2024-06-10T03:00:00.421 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "QUIESCED", 2024-06-10T03:00:00.421 INFO:teuthology.orchestra.run.smithi052.stdout: "age": 0.0 2024-06-10T03:00:00.421 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:00:00.421 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:00:00.422 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:00:00.422 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:00:00.422 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:00:00.422 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-10T03:00:00.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:00.419+0000 7f8a0a35b700 1 -- 172.21.15.52:0/4212657555 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f89ec0a2eb0 msgr2=0x7f89ec0a5370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:00:00.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:00.419+0000 7f8a0a35b700 1 --2- 172.21.15.52:0/4212657555 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f89ec0a2eb0 0x7f89ec0a5370 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f89f4009a10 tx=0x7f89f4009190 comp rx=0 tx=0).stop 2024-06-10T03:00:00.423 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:00.419+0000 7f8a0a35b700 1 -- 172.21.15.52:0/4212657555 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8a04101d80 msgr2=0x7f8a04073f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:00:00.423 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:00.419+0000 7f8a0a35b700 1 --2- 172.21.15.52:0/4212657555 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8a04101d80 0x7f8a04073f60 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f89fc01ffc0 tx=0x7f89fc01d3a0 comp rx=0 tx=0).stop 2024-06-10T03:00:00.423 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:00.419+0000 7f8a0a35b700 1 -- 172.21.15.52:0/4212657555 shutdown_connections 2024-06-10T03:00:00.423 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:00.419+0000 7f8a0a35b700 1 --2- 172.21.15.52:0/4212657555 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f89ec0a2eb0 0x7f89ec0a5370 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:00.423 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:00.419+0000 7f8a0a35b700 1 --2- 172.21.15.52:0/4212657555 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8a04107470 0x7f8a0406eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:00.423 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:00.419+0000 7f8a0a35b700 1 --2- 172.21.15.52:0/4212657555 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8a041013c0 0x7f8a04073a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:00.423 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:00.419+0000 7f8a0a35b700 1 --2- 172.21.15.52:0/4212657555 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8a04101d80 0x7f8a04073f60 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:00.423 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:00.419+0000 7f8a0a35b700 1 -- 172.21.15.52:0/4212657555 >> 172.21.15.52:0/4212657555 conn(0x7f8a040783d0 msgr2=0x7f8a040ffdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:00:00.423 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:00.420+0000 7f8a0a35b700 1 -- 172.21.15.52:0/4212657555 shutdown_connections 2024-06-10T03:00:00.423 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:00.420+0000 7f8a0a35b700 1 -- 172.21.15.52:0/4212657555 wait complete. 2024-06-10T03:00:00.477 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:00:00 smithi052 ceph-mon[32362]: pgmap v608: 129 pgs: 129 active+clean; 551 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 935 KiB/s rd, 1.1 MiB/s wr, 585 op/s 2024-06-10T03:00:00.477 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:00:00 smithi052 ceph-mon[32362]: overall HEALTH_OK 2024-06-10T03:00:00.505 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:00:00 smithi188 ceph-mon[36599]: pgmap v608: 129 pgs: 129 active+clean; 551 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 935 KiB/s rd, 1.1 MiB/s wr, 585 op/s 2024-06-10T03:00:00.506 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:00:00 smithi188 ceph-mon[36599]: overall HEALTH_OK 2024-06-10T03:00:00.638 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:00:00 smithi120 ceph-mon[36634]: pgmap v608: 129 pgs: 129 active+clean; 551 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 935 KiB/s rd, 1.1 MiB/s wr, 585 op/s 2024-06-10T03:00:00.638 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:00:00 smithi120 ceph-mon[36634]: overall HEALTH_OK 2024-06-10T03:00:01.164 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '3240692c', quiesce took 23.9 seconds. Will release after: 5.550000000000001 2024-06-10T03:00:02.476 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:00:02 smithi052 ceph-mon[32362]: pgmap v609: 129 pgs: 129 active+clean; 553 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 558 op/s 2024-06-10T03:00:02.477 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:00:02 smithi052 ceph-mon[32362]: scrub summary: idle 2024-06-10T03:00:02.477 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:00:02 smithi052 ceph-mon[32362]: scrub summary: idle 2024-06-10T03:00:02.477 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:00:02 smithi052 ceph-mon[32362]: scrub summary: idle 2024-06-10T03:00:02.477 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:00:02 smithi052 ceph-mon[32362]: scrub summary: idle 2024-06-10T03:00:02.638 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:00:02 smithi120 ceph-mon[36634]: pgmap v609: 129 pgs: 129 active+clean; 553 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 558 op/s 2024-06-10T03:00:02.638 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:00:02 smithi120 ceph-mon[36634]: scrub summary: idle 2024-06-10T03:00:02.638 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:00:02 smithi120 ceph-mon[36634]: scrub summary: idle 2024-06-10T03:00:02.638 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:00:02 smithi120 ceph-mon[36634]: scrub summary: idle 2024-06-10T03:00:02.638 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:00:02 smithi120 ceph-mon[36634]: scrub summary: idle 2024-06-10T03:00:02.704 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:00:02 smithi188 ceph-mon[36599]: pgmap v609: 129 pgs: 129 active+clean; 553 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 558 op/s 2024-06-10T03:00:02.704 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:00:02 smithi188 ceph-mon[36599]: scrub summary: idle 2024-06-10T03:00:02.704 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:00:02 smithi188 ceph-mon[36599]: scrub summary: idle 2024-06-10T03:00:02.704 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:00:02 smithi188 ceph-mon[36599]: scrub summary: idle 2024-06-10T03:00:02.704 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:00:02 smithi188 ceph-mon[36599]: scrub summary: idle 2024-06-10T03:00:04.476 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:00:04 smithi052 ceph-mon[32362]: pgmap v610: 129 pgs: 129 active+clean; 559 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 627 op/s 2024-06-10T03:00:04.638 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:00:04 smithi120 ceph-mon[36634]: pgmap v610: 129 pgs: 129 active+clean; 559 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 627 op/s 2024-06-10T03:00:04.703 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:00:04 smithi188 ceph-mon[36599]: pgmap v610: 129 pgs: 129 active+clean; 559 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 627 op/s 2024-06-10T03:00:06.638 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:00:06 smithi120 ceph-mon[36634]: pgmap v611: 129 pgs: 129 active+clean; 566 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.1 MiB/s wr, 573 op/s 2024-06-10T03:00:06.703 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:00:06 smithi188 ceph-mon[36599]: pgmap v611: 129 pgs: 129 active+clean; 566 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.1 MiB/s wr, 573 op/s 2024-06-10T03:00:06.715 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T03:00:06.726 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:00:06 smithi052 ceph-mon[32362]: pgmap v611: 129 pgs: 129 active+clean; 566 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.1 MiB/s wr, 573 op/s 2024-06-10T03:00:06.986 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T03:00:07.249 INFO:tasks.workunit.client.0.smithi052.stdout: 96 8206 0 5746 0 5338 0 2024-06-10T03:00:08.477 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:00:08 smithi052 ceph-mon[32362]: pgmap v612: 129 pgs: 129 active+clean; 566 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 2.1 MiB/s wr, 582 op/s 2024-06-10T03:00:08.638 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:00:08 smithi120 ceph-mon[36634]: pgmap v612: 129 pgs: 129 active+clean; 566 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 2.1 MiB/s wr, 582 op/s 2024-06-10T03:00:08.703 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:00:08 smithi188 ceph-mon[36599]: pgmap v612: 129 pgs: 129 active+clean; 566 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 2.1 MiB/s wr, 582 op/s 2024-06-10T03:00:08.736 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:08.733+0000 7f7c5364a700 1 -- 172.21.15.52:0/4060998524 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7c4c06ab30 msgr2=0x7f7c4c10f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:00:08.736 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:08.733+0000 7f7c5364a700 1 --2- 172.21.15.52:0/4060998524 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7c4c06ab30 0x7f7c4c10f320 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f7c4400b0a0 tx=0x7f7c4400b3b0 comp rx=0 tx=0).stop 2024-06-10T03:00:08.737 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:08.733+0000 7f7c5364a700 1 -- 172.21.15.52:0/4060998524 shutdown_connections 2024-06-10T03:00:08.737 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:08.733+0000 7f7c5364a700 1 --2- 172.21.15.52:0/4060998524 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7c4c06ab30 0x7f7c4c10f320 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:08.737 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:08.733+0000 7f7c5364a700 1 --2- 172.21.15.52:0/4060998524 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7c4c108500 0x7f7c4c1088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:08.737 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:08.733+0000 7f7c5364a700 1 --2- 172.21.15.52:0/4060998524 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7c4c06a170 0x7f7c4c06a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:08.737 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:08.733+0000 7f7c5364a700 1 -- 172.21.15.52:0/4060998524 >> 172.21.15.52:0/4060998524 conn(0x7f7c4c078340 msgr2=0x7f7c4c078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:00:08.737 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:08.734+0000 7f7c5364a700 1 -- 172.21.15.52:0/4060998524 shutdown_connections 2024-06-10T03:00:08.737 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:08.734+0000 7f7c5364a700 1 -- 172.21.15.52:0/4060998524 wait complete. 2024-06-10T03:00:08.737 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:08.734+0000 7f7c5364a700 1 Processor -- start 2024-06-10T03:00:08.737 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:08.734+0000 7f7c5364a700 1 -- start start 2024-06-10T03:00:08.737 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:08.734+0000 7f7c5364a700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7c4c06a170 0x7f7c4c196b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:08.737 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:08.734+0000 7f7c5364a700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7c4c06ab30 0x7f7c4c197040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:08.737 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:08.735+0000 7f7c5364a700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7c4c108500 0x7f7c4c19b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:08.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:08.735+0000 7f7c5364a700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f7c4c1105d0 con 0x7f7c4c06a170 2024-06-10T03:00:08.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:08.735+0000 7f7c5364a700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f7c4c110450 con 0x7f7c4c108500 2024-06-10T03:00:08.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:08.735+0000 7f7c5364a700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f7c4c110750 con 0x7f7c4c06ab30 2024-06-10T03:00:08.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:08.735+0000 7f7c513e6700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7c4c06a170 0x7f7c4c196b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:08.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:08.735+0000 7f7c50be5700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7c4c06ab30 0x7f7c4c197040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:08.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:08.735+0000 7f7c513e6700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7c4c06a170 0x7f7c4c196b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37238/0 (socket says 172.21.15.52:37238) 2024-06-10T03:00:08.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:08.735+0000 7f7c513e6700 1 -- 172.21.15.52:0/2444165153 learned_addr learned my addr 172.21.15.52:0/2444165153 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:00:08.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:08.735+0000 7f7c51be7700 1 --2- 172.21.15.52:0/2444165153 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7c4c108500 0x7f7c4c19b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:08.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:08.735+0000 7f7c50be5700 1 -- 172.21.15.52:0/2444165153 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7c4c108500 msgr2=0x7f7c4c19b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:00:08.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:08.735+0000 7f7c50be5700 1 --2- 172.21.15.52:0/2444165153 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7c4c108500 0x7f7c4c19b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:08.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:08.735+0000 7f7c50be5700 1 -- 172.21.15.52:0/2444165153 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7c4c06a170 msgr2=0x7f7c4c196b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:00:08.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:08.735+0000 7f7c50be5700 1 --2- 172.21.15.52:0/2444165153 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7c4c06a170 0x7f7c4c196b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:08.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:08.735+0000 7f7c50be5700 1 -- 172.21.15.52:0/2444165153 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c4c19bab0 con 0x7f7c4c06ab30 2024-06-10T03:00:08.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:08.735+0000 7f7c513e6700 1 --2- 172.21.15.52:0/2444165153 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7c4c06a170 0x7f7c4c196b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T03:00:08.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:08.735+0000 7f7c50be5700 1 --2- 172.21.15.52:0/2444165153 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7c4c06ab30 0x7f7c4c197040 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f7c3c00e9f0 tx=0x7f7c3c00ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:00:08.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:08.735+0000 7f7c427fc700 1 -- 172.21.15.52:0/2444165153 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c3c00cb70 con 0x7f7c4c06ab30 2024-06-10T03:00:08.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:08.736+0000 7f7c5364a700 1 -- 172.21.15.52:0/2444165153 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c4c19bda0 con 0x7f7c4c06ab30 2024-06-10T03:00:08.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:08.736+0000 7f7c5364a700 1 -- 172.21.15.52:0/2444165153 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c4c1a3660 con 0x7f7c4c06ab30 2024-06-10T03:00:08.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:08.736+0000 7f7c427fc700 1 -- 172.21.15.52:0/2444165153 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c3c004500 con 0x7f7c4c06ab30 2024-06-10T03:00:08.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:08.736+0000 7f7c427fc700 1 -- 172.21.15.52:0/2444165153 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c3c003e10 con 0x7f7c4c06ab30 2024-06-10T03:00:08.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:08.737+0000 7f7c5364a700 1 -- 172.21.15.52:0/2444165153 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c4c10a9e0 con 0x7f7c4c06ab30 2024-06-10T03:00:08.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:08.738+0000 7f7c427fc700 1 -- 172.21.15.52:0/2444165153 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f7c3c004020 con 0x7f7c4c06ab30 2024-06-10T03:00:08.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:08.739+0000 7f7c427fc700 1 --2- 172.21.15.52:0/2444165153 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f7c380a2c20 0x7f7c380a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:08.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:08.739+0000 7f7c513e6700 1 --2- 172.21.15.52:0/2444165153 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f7c380a2c20 0x7f7c380a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:08.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:08.739+0000 7f7c427fc700 1 -- 172.21.15.52:0/2444165153 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f7c3c0cf670 con 0x7f7c4c06ab30 2024-06-10T03:00:08.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:08.739+0000 7f7c513e6700 1 --2- 172.21.15.52:0/2444165153 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f7c380a2c20 0x7f7c380a50e0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f7c4800f760 tx=0x7f7c48011040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:00:08.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:08.743+0000 7f7c427fc700 1 -- 172.21.15.52:0/2444165153 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f7c3c09c250 con 0x7f7c4c06ab30 2024-06-10T03:00:09.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:09.018+0000 7f7c5364a700 1 -- 172.21.15.52:0/2444165153 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7c4c197e10 con 0x7f7c4c06ab30 2024-06-10T03:00:09.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:09.019+0000 7f7c427fc700 1 -- 172.21.15.52:0/2444165153 <== mon.1 v2:172.21.15.188:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+11945 (secure 0 0 0) 0x7f7c3c09bb50 con 0x7f7c4c06ab30 2024-06-10T03:00:09.018 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-10T03:00:09.019 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":36,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/889673247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":889673247},{"type":"v1","addr":"172.21.15.52:6835","nonce":889673247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/3615663511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":3615663511},{"type":"v1","addr":"172.21.15.52:6837","nonce":3615663511}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":36,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T02:50:03.889766+0000","modified":"2024-06-10T02:59:27.720611+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24484,"mds_1":24469,"mds_2":24463,"mds_3":24449,"mds_4":24428},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14571":{"gid":14571,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6839/3059599042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":3059599042},{"type":"v1","addr":"172.21.15.52:6839","nonce":3059599042}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14583":{"gid":14583,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6841/3039275521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":3039275521},{"type":"v1","addr":"172.21.15.52:6841","nonce":3039275521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24383":{"gid":24383,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6833/2609002896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":2609002896},{"type":"v1","addr":"172.21.15.120:6833","nonce":2609002896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6835/1996370288","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":1996370288},{"type":"v1","addr":"172.21.15.120:6835","nonce":1996370288}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6833/529564979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":529564979},{"type":"v1","addr":"172.21.15.188:6833","nonce":529564979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":25,"addr":"172.21.15.120:6837/4132440952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":4132440952},{"type":"v1","addr":"172.21.15.120:6837","nonce":4132440952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"b","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.120:6839/1672436487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1672436487},{"type":"v1","addr":"172.21.15.120:6839","nonce":1672436487}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":2,"incarnation":22,"state":"up:active","state_seq":26,"addr":"172.21.15.188:6835/3904544259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":3904544259},{"type":"v1","addr":"172.21.15.188:6835","nonce":3904544259}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.188:6837/819317726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":819317726},{"type":"v1","addr":"172.21.15.188:6837","nonce":819317726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.188:6839/310444836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":310444836},{"type":"v1","addr":"172.21.15.188:6839","nonce":310444836}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24484,"qdb_cluster":[24469,24484,24463,24428,24449]},"id":1}]} 2024-06-10T03:00:09.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:09.021+0000 7f7c5364a700 1 -- 172.21.15.52:0/2444165153 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f7c380a2c20 msgr2=0x7f7c380a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:00:09.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:09.021+0000 7f7c5364a700 1 --2- 172.21.15.52:0/2444165153 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f7c380a2c20 0x7f7c380a50e0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f7c4800f760 tx=0x7f7c48011040 comp rx=0 tx=0).stop 2024-06-10T03:00:09.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:09.021+0000 7f7c5364a700 1 -- 172.21.15.52:0/2444165153 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7c4c06ab30 msgr2=0x7f7c4c197040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:00:09.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:09.021+0000 7f7c5364a700 1 --2- 172.21.15.52:0/2444165153 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7c4c06ab30 0x7f7c4c197040 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f7c3c00e9f0 tx=0x7f7c3c00ed00 comp rx=0 tx=0).stop 2024-06-10T03:00:09.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:09.021+0000 7f7c5364a700 1 -- 172.21.15.52:0/2444165153 shutdown_connections 2024-06-10T03:00:09.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:09.021+0000 7f7c5364a700 1 --2- 172.21.15.52:0/2444165153 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7c4c108500 0x7f7c4c19b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:09.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:09.021+0000 7f7c5364a700 1 --2- 172.21.15.52:0/2444165153 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f7c380a2c20 0x7f7c380a50e0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:09.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:09.021+0000 7f7c5364a700 1 --2- 172.21.15.52:0/2444165153 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7c4c06a170 0x7f7c4c196b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:09.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:09.021+0000 7f7c5364a700 1 --2- 172.21.15.52:0/2444165153 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7c4c06ab30 0x7f7c4c197040 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:09.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:09.021+0000 7f7c5364a700 1 -- 172.21.15.52:0/2444165153 >> 172.21.15.52:0/2444165153 conn(0x7f7c4c078340 msgr2=0x7f7c4c0fffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:00:09.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:09.021+0000 7f7c5364a700 1 -- 172.21.15.52:0/2444165153 shutdown_connections 2024-06-10T03:00:09.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:09.021+0000 7f7c5364a700 1 -- 172.21.15.52:0/2444165153 wait complete. 2024-06-10T03:00:09.022 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 36 2024-06-10T03:00:09.347 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:00:09 smithi052 ceph-mon[32362]: from='client.? 172.21.15.52:0/2444165153' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T03:00:09.638 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:00:09 smithi120 ceph-mon[36634]: from='client.? 172.21.15.52:0/2444165153' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T03:00:09.703 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:00:09 smithi188 ceph-mon[36599]: from='client.? 172.21.15.52:0/2444165153' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T03:00:09.776 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24484 quiesce db --set-id 3240692c --release --await' 2024-06-10T03:00:09.776 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24484 quiesce db --set-id 3240692c --release --await 2024-06-10T03:00:10.085 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T03:00:10.477 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:00:10 smithi052 ceph-mon[32362]: pgmap v613: 129 pgs: 129 active+clean; 568 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 1.7 MiB/s wr, 567 op/s 2024-06-10T03:00:10.638 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:00:10 smithi120 ceph-mon[36634]: pgmap v613: 129 pgs: 129 active+clean; 568 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 1.7 MiB/s wr, 567 op/s 2024-06-10T03:00:10.703 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:00:10 smithi188 ceph-mon[36599]: pgmap v613: 129 pgs: 129 active+clean; 568 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 1.7 MiB/s wr, 567 op/s 2024-06-10T03:00:11.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.809+0000 7efe5dc46700 1 -- 172.21.15.52:0/2807487665 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efe58101390 msgr2=0x7efe58101810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:00:11.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.809+0000 7efe5dc46700 1 --2- 172.21.15.52:0/2807487665 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efe58101390 0x7efe58101810 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7efe4c009990 tx=0x7efe4c009ca0 comp rx=0 tx=0).stop 2024-06-10T03:00:11.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.809+0000 7efe5dc46700 1 -- 172.21.15.52:0/2807487665 shutdown_connections 2024-06-10T03:00:11.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.809+0000 7efe5dc46700 1 --2- 172.21.15.52:0/2807487665 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efe58101d50 0x7efe5810e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:11.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.809+0000 7efe5dc46700 1 --2- 172.21.15.52:0/2807487665 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efe58107400 0x7efe581077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:11.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.809+0000 7efe5dc46700 1 --2- 172.21.15.52:0/2807487665 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efe58101390 0x7efe58101810 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:11.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.809+0000 7efe5dc46700 1 -- 172.21.15.52:0/2807487665 >> 172.21.15.52:0/2807487665 conn(0x7efe58078340 msgr2=0x7efe58078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:00:11.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.809+0000 7efe5dc46700 1 -- 172.21.15.52:0/2807487665 shutdown_connections 2024-06-10T03:00:11.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.809+0000 7efe5dc46700 1 -- 172.21.15.52:0/2807487665 wait complete. 2024-06-10T03:00:11.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.810+0000 7efe5dc46700 1 Processor -- start 2024-06-10T03:00:11.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.810+0000 7efe5dc46700 1 -- start start 2024-06-10T03:00:11.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.810+0000 7efe5dc46700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efe58101390 0x7efe5819e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:11.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.810+0000 7efe5dc46700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efe58101d50 0x7efe5819ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:11.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.810+0000 7efe5dc46700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efe58107400 0x7efe58198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:11.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.811+0000 7efe5dc46700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7efe58111670 con 0x7efe58101d50 2024-06-10T03:00:11.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.811+0000 7efe5dc46700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7efe581114f0 con 0x7efe58107400 2024-06-10T03:00:11.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.811+0000 7efe5dc46700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7efe581117f0 con 0x7efe58101390 2024-06-10T03:00:11.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.811+0000 7efe56ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efe58101d50 0x7efe5819ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:11.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.811+0000 7efe56ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efe58101d50 0x7efe5819ed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:50236/0 (socket says 172.21.15.52:50236) 2024-06-10T03:00:11.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.811+0000 7efe56ffd700 1 -- 172.21.15.52:0/3540095069 learned_addr learned my addr 172.21.15.52:0/3540095069 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:00:11.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.811+0000 7efe577fe700 1 --2- 172.21.15.52:0/3540095069 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efe58101390 0x7efe5819e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:11.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.811+0000 7efe57fff700 1 --2- 172.21.15.52:0/3540095069 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efe58107400 0x7efe58198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:11.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.811+0000 7efe56ffd700 1 -- 172.21.15.52:0/3540095069 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efe58101390 msgr2=0x7efe5819e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:00:11.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.811+0000 7efe56ffd700 1 --2- 172.21.15.52:0/3540095069 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efe58101390 0x7efe5819e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:11.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.811+0000 7efe56ffd700 1 -- 172.21.15.52:0/3540095069 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efe58107400 msgr2=0x7efe58198a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:00:11.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.811+0000 7efe56ffd700 1 --2- 172.21.15.52:0/3540095069 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efe58107400 0x7efe58198a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:11.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.811+0000 7efe56ffd700 1 -- 172.21.15.52:0/3540095069 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe58199170 con 0x7efe58101d50 2024-06-10T03:00:11.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.811+0000 7efe577fe700 1 --2- 172.21.15.52:0/3540095069 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efe58101390 0x7efe5819e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T03:00:11.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.811+0000 7efe56ffd700 1 --2- 172.21.15.52:0/3540095069 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efe58101d50 0x7efe5819ed90 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7efe4c000c00 tx=0x7efe4c0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:00:11.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.811+0000 7efe54ff9700 1 -- 172.21.15.52:0/3540095069 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe4c027070 con 0x7efe58101d50 2024-06-10T03:00:11.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.811+0000 7efe5dc46700 1 -- 172.21.15.52:0/3540095069 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe58199400 con 0x7efe58101d50 2024-06-10T03:00:11.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.811+0000 7efe54ff9700 1 -- 172.21.15.52:0/3540095069 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe4c004050 con 0x7efe58101d50 2024-06-10T03:00:11.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.811+0000 7efe54ff9700 1 -- 172.21.15.52:0/3540095069 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe4c00f460 con 0x7efe58101d50 2024-06-10T03:00:11.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.812+0000 7efe5dc46700 1 -- 172.21.15.52:0/3540095069 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe581a5720 con 0x7efe58101d50 2024-06-10T03:00:11.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.814+0000 7efe54ff9700 1 -- 172.21.15.52:0/3540095069 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7efe4c02a070 con 0x7efe58101d50 2024-06-10T03:00:11.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.815+0000 7efe54ff9700 1 --2- 172.21.15.52:0/3540095069 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7efe400a2c20 0x7efe400a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:11.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.815+0000 7efe577fe700 1 --2- 172.21.15.52:0/3540095069 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7efe400a2c20 0x7efe400a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:11.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.815+0000 7efe54ff9700 1 -- 172.21.15.52:0/3540095069 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7efe4c0d0360 con 0x7efe58101d50 2024-06-10T03:00:11.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.815+0000 7efe577fe700 1 --2- 172.21.15.52:0/3540095069 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7efe400a2c20 0x7efe400a50e0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7efe58199b50 tx=0x7efe48005d80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:00:11.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.815+0000 7efe684c9b80 1 Processor -- start 2024-06-10T03:00:11.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.816+0000 7efe684c9b80 1 -- start start 2024-06-10T03:00:11.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.816+0000 7efe684c9b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5632d7633050 0x5632d7633430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:11.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.816+0000 7efe684c9b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5632d7633970 0x5632d762d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:11.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.816+0000 7efe684c9b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5632d762d7e0 0x5632d762dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:11.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.816+0000 7efe684c9b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5632d757a910 con 0x5632d7633050 2024-06-10T03:00:11.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.816+0000 7efe684c9b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5632d7522820 con 0x5632d7633970 2024-06-10T03:00:11.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.816+0000 7efe684c9b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5632d75768d0 con 0x5632d762d7e0 2024-06-10T03:00:11.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.816+0000 7efe56ffd700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5632d7633970 0x5632d762d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:11.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.816+0000 7efe577fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5632d7633050 0x5632d7633430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:11.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.816+0000 7efe56ffd700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5632d7633970 0x5632d762d2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.52:58268/0 (socket says 172.21.15.52:58268) 2024-06-10T03:00:11.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.816+0000 7efe56ffd700 1 -- 172.21.15.52:0/3168574724 learned_addr learned my addr 172.21.15.52:0/3168574724 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:00:11.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.816+0000 7efe57fff700 1 --2- 172.21.15.52:0/3168574724 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5632d762d7e0 0x5632d762dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:11.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.816+0000 7efe577fe700 1 -- 172.21.15.52:0/3168574724 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5632d762d7e0 msgr2=0x5632d762dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:00:11.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.816+0000 7efe577fe700 1 --2- 172.21.15.52:0/3168574724 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5632d762d7e0 0x5632d762dc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:11.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.816+0000 7efe577fe700 1 -- 172.21.15.52:0/3168574724 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5632d7633970 msgr2=0x5632d762d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:00:11.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.816+0000 7efe577fe700 1 --2- 172.21.15.52:0/3168574724 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5632d7633970 0x5632d762d2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:11.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.816+0000 7efe577fe700 1 -- 172.21.15.52:0/3168574724 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5632d7519050 con 0x5632d7633050 2024-06-10T03:00:11.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.816+0000 7efe56ffd700 1 --2- 172.21.15.52:0/3168574724 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5632d7633970 0x5632d762d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T03:00:11.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.816+0000 7efe577fe700 1 --2- 172.21.15.52:0/3168574724 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5632d7633050 0x5632d7633430 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7efe4800fbc0 tx=0x7efe48012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=936287de0607986 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:00:11.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.817+0000 7efe3dffb700 1 -- 172.21.15.52:0/3168574724 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe48022070 con 0x5632d7633050 2024-06-10T03:00:11.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.817+0000 7efe3dffb700 1 -- 172.21.15.52:0/3168574724 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe48011db0 con 0x5632d7633050 2024-06-10T03:00:11.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.817+0000 7efe3dffb700 1 -- 172.21.15.52:0/3168574724 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe48003be0 con 0x5632d7633050 2024-06-10T03:00:11.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.817+0000 7efe684c9b80 1 -- 172.21.15.52:0/3168574724 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5632d7633050 msgr2=0x5632d7633430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:00:11.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.817+0000 7efe684c9b80 1 --2- 172.21.15.52:0/3168574724 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5632d7633050 0x5632d7633430 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7efe4800fbc0 tx=0x7efe48012000 comp rx=0 tx=0).stop 2024-06-10T03:00:11.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.817+0000 7efe684c9b80 1 -- 172.21.15.52:0/3168574724 shutdown_connections 2024-06-10T03:00:11.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.817+0000 7efe684c9b80 1 --2- 172.21.15.52:0/3168574724 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5632d762d7e0 0x5632d762dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:11.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.817+0000 7efe684c9b80 1 --2- 172.21.15.52:0/3168574724 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5632d7633050 0x5632d7633430 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:11.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.817+0000 7efe684c9b80 1 --2- 172.21.15.52:0/3168574724 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5632d7633970 0x5632d762d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:11.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.817+0000 7efe684c9b80 1 -- 172.21.15.52:0/3168574724 >> 172.21.15.52:0/3168574724 conn(0x5632d7553dc0 msgr2=0x5632d7624120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:00:11.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.817+0000 7efe684c9b80 1 -- 172.21.15.52:0/3168574724 shutdown_connections 2024-06-10T03:00:11.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.818+0000 7efe684c9b80 1 -- 172.21.15.52:0/3168574724 wait complete. 2024-06-10T03:00:11.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.818+0000 7efe684c9b80 1 -- start start 2024-06-10T03:00:11.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.818+0000 7efe684c9b80 1 Processor -- start 2024-06-10T03:00:11.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.818+0000 7efe684c9b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5632d762d7e0 0x5632d7639600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:11.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.818+0000 7efe684c9b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5632d7633050 0x5632d7636970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:11.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.818+0000 7efe684c9b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5632d7633970 0x5632d7636eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:11.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.818+0000 7efe684c9b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5632d763a3e0 con 0x5632d7633970 2024-06-10T03:00:11.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.818+0000 7efe684c9b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5632d751acf0 con 0x5632d762d7e0 2024-06-10T03:00:11.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.818+0000 7efe684c9b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5632d763a560 con 0x5632d7633050 2024-06-10T03:00:11.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.818+0000 7efe684c9b80 10 client.0 _ll_register_callbacks cb 0x5632d7514a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T03:00:11.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.819+0000 7efe57fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5632d7633970 0x5632d7636eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:11.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.819+0000 7efe57fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5632d7633970 0x5632d7636eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:50248/0 (socket says 172.21.15.52:50248) 2024-06-10T03:00:11.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.819+0000 7efe57fff700 1 -- 172.21.15.52:0/3588456486 learned_addr learned my addr 172.21.15.52:0/3588456486 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:00:11.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.819+0000 7efe577fe700 1 --2- 172.21.15.52:0/3588456486 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5632d762d7e0 0x5632d7639600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:11.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.819+0000 7efe56ffd700 1 --2- 172.21.15.52:0/3588456486 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5632d7633050 0x5632d7636970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:11.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.819+0000 7efe57fff700 1 -- 172.21.15.52:0/3588456486 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5632d7633050 msgr2=0x5632d7636970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:00:11.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.819+0000 7efe57fff700 1 --2- 172.21.15.52:0/3588456486 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5632d7633050 0x5632d7636970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:11.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.819+0000 7efe57fff700 1 -- 172.21.15.52:0/3588456486 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5632d762d7e0 msgr2=0x5632d7639600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:00:11.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.819+0000 7efe57fff700 1 --2- 172.21.15.52:0/3588456486 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5632d762d7e0 0x5632d7639600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:11.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.819+0000 7efe57fff700 1 -- 172.21.15.52:0/3588456486 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5632d74c99d0 con 0x5632d7633970 2024-06-10T03:00:11.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.819+0000 7efe57fff700 1 --2- 172.21.15.52:0/3588456486 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5632d7633970 0x5632d7636eb0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7efe5000f930 tx=0x7efe5000fc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16c2ba2ab98e2a6e server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:00:11.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.819+0000 7efe36ffd700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-10T03:00:11.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.819+0000 7efe36ffd700 1 -- 172.21.15.52:0/3588456486 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe5000d070 con 0x5632d7633970 2024-06-10T03:00:11.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.819+0000 7efe36ffd700 1 -- 172.21.15.52:0/3588456486 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7efe50003c00 con 0x5632d7633970 2024-06-10T03:00:11.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.819+0000 7efe684c9b80 1 -- 172.21.15.52:0/3588456486 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efe48011db0 con 0x5632d7633970 2024-06-10T03:00:11.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.819+0000 7efe36ffd700 1 -- 172.21.15.52:0/3588456486 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7efe50022050 con 0x5632d7633970 2024-06-10T03:00:11.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.819+0000 7efe684c9b80 10 client.16317 fetch_fsmap learned FSMap version 36 2024-06-10T03:00:11.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.819+0000 7efe684c9b80 1 -- 172.21.15.52:0/3588456486 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5632d7644cc0 con 0x5632d7633970 2024-06-10T03:00:11.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.820+0000 7efe36ffd700 1 -- 172.21.15.52:0/3588456486 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe48011db0 con 0x5632d7633970 2024-06-10T03:00:11.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.820+0000 7efe36ffd700 1 -- 172.21.15.52:0/3588456486 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 36) ==== 8378+0+0 (secure 0 0 0) 0x7efe50013050 con 0x5632d7633970 2024-06-10T03:00:11.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.820+0000 7efe684c9b80 10 client.16317 fetch_fsmap finished waiting for FSMap version 36 2024-06-10T03:00:11.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.820+0000 7efe684c9b80 10 client.16317 resolve_mds: validated gid 24484 aka daemon mds.i 2024-06-10T03:00:11.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.820+0000 7efe684c9b80 20 client.16317 populate_metadata read hostname 'smithi052' 2024-06-10T03:00:11.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.820+0000 7efe684c9b80 1 --2- 172.21.15.52:0/3588456486 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x5632d7647300 0x5632d7649710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:11.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.820+0000 7efe684c9b80 4 client.16317 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T03:00:11.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.820+0000 7efe684c9b80 1 -- 172.21.15.52:0/3588456486 --> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5632d7587e10 con 0x5632d7647300 2024-06-10T03:00:11.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.820+0000 7efe577fe700 1 --2- 172.21.15.52:0/3588456486 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x5632d7647300 0x5632d7649710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:11.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.821+0000 7efe577fe700 1 --2- 172.21.15.52:0/3588456486 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x5632d7647300 0x5632d7649710 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4991fed1f99a70c4 server_cookie=4c5aec12991709c4 in_seq=0 out_seq=0 2024-06-10T03:00:11.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.821+0000 7efe36ffd700 10 client.16317 ms_handle_connect on v2:172.21.15.188:6838/310444836 2024-06-10T03:00:11.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.821+0000 7efe36ffd700 1 -- 172.21.15.52:0/3588456486 <== mds.0 v2:172.21.15.188:6838/310444836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5632d7587e10 con 0x5632d7647300 2024-06-10T03:00:11.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.821+0000 7efe36ffd700 10 client.16317 handle_command_reply: tid=0 2024-06-10T03:00:11.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.821+0000 7efe684c9b80 1 client.16317 shutdown 2024-06-10T03:00:11.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.822+0000 7efe684c9b80 1 -- 172.21.15.52:0/3588456486 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5632d7633970 msgr2=0x5632d7636eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:00:11.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.822+0000 7efe684c9b80 1 --2- 172.21.15.52:0/3588456486 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5632d7633970 0x5632d7636eb0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7efe5000f930 tx=0x7efe5000fc40 comp rx=0 tx=0).stop 2024-06-10T03:00:11.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.822+0000 7efe684c9b80 1 -- 172.21.15.52:0/3588456486 shutdown_connections 2024-06-10T03:00:11.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.822+0000 7efe684c9b80 1 --2- 172.21.15.52:0/3588456486 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x5632d7647300 0x5632d7649710 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:11.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.822+0000 7efe684c9b80 1 --2- 172.21.15.52:0/3588456486 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5632d7633970 0x5632d7636eb0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:11.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.822+0000 7efe684c9b80 1 --2- 172.21.15.52:0/3588456486 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5632d762d7e0 0x5632d7639600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:11.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.822+0000 7efe684c9b80 1 --2- 172.21.15.52:0/3588456486 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5632d7633050 0x5632d7636970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:11.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.822+0000 7efe684c9b80 1 -- 172.21.15.52:0/3588456486 >> 172.21.15.52:0/3588456486 conn(0x5632d7553dc0 msgr2=0x5632d7623d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:00:11.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.822+0000 7efe36ffd700 0 client.16317 ms_handle_reset on v2:172.21.15.188:6838/310444836 2024-06-10T03:00:11.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.822+0000 7efe684c9b80 1 -- 172.21.15.52:0/3588456486 shutdown_connections 2024-06-10T03:00:11.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.822+0000 7efe684c9b80 1 -- 172.21.15.52:0/3588456486 wait complete. 2024-06-10T03:00:11.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.822+0000 7efe684c9b80 20 client.16317 trim_cache size 0 max 16384 2024-06-10T03:00:11.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.843+0000 7efe684c9b80 1 Processor -- start 2024-06-10T03:00:11.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.843+0000 7efe684c9b80 1 -- start start 2024-06-10T03:00:11.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.843+0000 7efe684c9b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5632d762d7e0 0x5632d7636ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:11.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.843+0000 7efe684c9b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5632d7633050 0x5632d764f020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:11.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.843+0000 7efe684c9b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5632d7633970 0x5632d764f560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:11.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.843+0000 7efe684c9b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7efe581ac070 con 0x5632d762d7e0 2024-06-10T03:00:11.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.843+0000 7efe684c9b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7efe581adc70 con 0x5632d7633970 2024-06-10T03:00:11.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.843+0000 7efe684c9b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7efe581ac740 con 0x5632d7633050 2024-06-10T03:00:11.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.843+0000 7efe577fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5632d762d7e0 0x5632d7636ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:11.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.843+0000 7efe577fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5632d762d7e0 0x5632d7636ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:50258/0 (socket says 172.21.15.52:50258) 2024-06-10T03:00:11.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.843+0000 7efe577fe700 1 -- 172.21.15.52:0/423487195 learned_addr learned my addr 172.21.15.52:0/423487195 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:00:11.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.843+0000 7efe56ffd700 1 --2- 172.21.15.52:0/423487195 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5632d7633050 0x5632d764f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:11.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.844+0000 7efe577fe700 1 -- 172.21.15.52:0/423487195 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5632d7633050 msgr2=0x5632d764f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:00:11.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.844+0000 7efe577fe700 1 --2- 172.21.15.52:0/423487195 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5632d7633050 0x5632d764f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:11.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.844+0000 7efe577fe700 1 -- 172.21.15.52:0/423487195 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5632d7633970 msgr2=0x5632d764f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:00:11.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.844+0000 7efe577fe700 1 --2- 172.21.15.52:0/423487195 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5632d7633970 0x5632d764f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:11.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.844+0000 7efe577fe700 1 -- 172.21.15.52:0/423487195 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5632d7634ee0 con 0x5632d762d7e0 2024-06-10T03:00:11.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.844+0000 7efe56ffd700 1 --2- 172.21.15.52:0/423487195 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5632d7633050 0x5632d764f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T03:00:11.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.844+0000 7efe577fe700 1 --2- 172.21.15.52:0/423487195 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5632d762d7e0 0x5632d7636ee0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7efe48003810 tx=0x7efe48011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e66067d3ed0dc15 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:00:11.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.844+0000 7efe3d7fa700 1 -- 172.21.15.52:0/423487195 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe48038020 con 0x5632d762d7e0 2024-06-10T03:00:11.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.844+0000 7efe3d7fa700 1 -- 172.21.15.52:0/423487195 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe48011d80 con 0x5632d762d7e0 2024-06-10T03:00:11.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.844+0000 7efe3d7fa700 1 -- 172.21.15.52:0/423487195 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe48015040 con 0x5632d762d7e0 2024-06-10T03:00:11.848 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.845+0000 7efe684c9b80 1 -- 172.21.15.52:0/423487195 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5632d762d7e0 msgr2=0x5632d7636ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:00:11.848 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.845+0000 7efe684c9b80 1 --2- 172.21.15.52:0/423487195 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5632d762d7e0 0x5632d7636ee0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7efe48003810 tx=0x7efe48011b60 comp rx=0 tx=0).stop 2024-06-10T03:00:11.848 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.845+0000 7efe684c9b80 1 -- 172.21.15.52:0/423487195 shutdown_connections 2024-06-10T03:00:11.848 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.845+0000 7efe684c9b80 1 --2- 172.21.15.52:0/423487195 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5632d7633970 0x5632d764f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:11.848 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.845+0000 7efe684c9b80 1 --2- 172.21.15.52:0/423487195 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5632d762d7e0 0x5632d7636ee0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:11.848 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.845+0000 7efe684c9b80 1 --2- 172.21.15.52:0/423487195 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5632d7633050 0x5632d764f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:11.848 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.845+0000 7efe684c9b80 1 -- 172.21.15.52:0/423487195 >> 172.21.15.52:0/423487195 conn(0x5632d7553dc0 msgr2=0x5632d7630ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:00:11.849 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.845+0000 7efe684c9b80 1 -- 172.21.15.52:0/423487195 shutdown_connections 2024-06-10T03:00:11.849 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.845+0000 7efe684c9b80 1 -- 172.21.15.52:0/423487195 wait complete. 2024-06-10T03:00:11.849 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.845+0000 7efe684c9b80 1 -- start start 2024-06-10T03:00:11.849 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.845+0000 7efe684c9b80 1 Processor -- start 2024-06-10T03:00:11.849 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.846+0000 7efe684c9b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5632d762d7e0 0x5632d763fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:11.849 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.846+0000 7efe684c9b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5632d7633050 0x5632d763c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:11.849 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.846+0000 7efe684c9b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5632d7633970 0x5632d763cd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:11.849 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.846+0000 7efe684c9b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7efe581ad5a0 con 0x5632d762d7e0 2024-06-10T03:00:11.849 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.846+0000 7efe684c9b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7efe581aced0 con 0x5632d7633050 2024-06-10T03:00:11.849 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.846+0000 7efe684c9b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7efe581aa010 con 0x5632d7633970 2024-06-10T03:00:11.849 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.846+0000 7efe684c9b80 10 client.0 _ll_register_callbacks cb 0x5632d7514a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T03:00:11.849 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.846+0000 7efe577fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5632d762d7e0 0x5632d763fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:11.849 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.846+0000 7efe577fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5632d762d7e0 0x5632d763fcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:50260/0 (socket says 172.21.15.52:50260) 2024-06-10T03:00:11.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.846+0000 7efe577fe700 1 -- 172.21.15.52:0/1814562179 learned_addr learned my addr 172.21.15.52:0/1814562179 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:00:11.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.846+0000 7efe56ffd700 1 --2- 172.21.15.52:0/1814562179 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5632d7633050 0x5632d763c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:11.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.846+0000 7efe57fff700 1 --2- 172.21.15.52:0/1814562179 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5632d7633970 0x5632d763cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:11.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.846+0000 7efe577fe700 1 -- 172.21.15.52:0/1814562179 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5632d7633970 msgr2=0x5632d763cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:00:11.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.846+0000 7efe577fe700 1 --2- 172.21.15.52:0/1814562179 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5632d7633970 0x5632d763cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:11.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.846+0000 7efe577fe700 1 -- 172.21.15.52:0/1814562179 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5632d7633050 msgr2=0x5632d763c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:00:11.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.846+0000 7efe577fe700 1 --2- 172.21.15.52:0/1814562179 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5632d7633050 0x5632d763c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:11.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.846+0000 7efe577fe700 1 -- 172.21.15.52:0/1814562179 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5632d763d270 con 0x5632d762d7e0 2024-06-10T03:00:11.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.846+0000 7efe577fe700 1 --2- 172.21.15.52:0/1814562179 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5632d762d7e0 0x5632d763fcf0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7efe48014040 tx=0x7efe480050e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77f997c366d0ac52 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:00:11.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.846+0000 7efe36ffd700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-10T03:00:11.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.846+0000 7efe36ffd700 1 -- 172.21.15.52:0/1814562179 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe48041060 con 0x5632d762d7e0 2024-06-10T03:00:11.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.847+0000 7efe36ffd700 1 -- 172.21.15.52:0/1814562179 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7efe48004910 con 0x5632d762d7e0 2024-06-10T03:00:11.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.847+0000 7efe684c9b80 1 -- 172.21.15.52:0/1814562179 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efe48011d80 con 0x5632d762d7e0 2024-06-10T03:00:11.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.847+0000 7efe36ffd700 1 -- 172.21.15.52:0/1814562179 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7efe48048050 con 0x5632d762d7e0 2024-06-10T03:00:11.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.847+0000 7efe684c9b80 10 client.16323 fetch_fsmap learned FSMap version 36 2024-06-10T03:00:11.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.847+0000 7efe684c9b80 1 -- 172.21.15.52:0/1814562179 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5632d76b2d50 con 0x5632d762d7e0 2024-06-10T03:00:11.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.847+0000 7efe36ffd700 1 -- 172.21.15.52:0/1814562179 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe48011d80 con 0x5632d762d7e0 2024-06-10T03:00:11.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.847+0000 7efe36ffd700 1 -- 172.21.15.52:0/1814562179 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 36) ==== 8378+0+0 (secure 0 0 0) 0x7efe4801ebe0 con 0x5632d762d7e0 2024-06-10T03:00:11.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.847+0000 7efe684c9b80 10 client.16323 fetch_fsmap finished waiting for FSMap version 36 2024-06-10T03:00:11.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.847+0000 7efe684c9b80 10 client.16323 resolve_mds: validated gid 24484 aka daemon mds.i 2024-06-10T03:00:11.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.847+0000 7efe684c9b80 20 client.16323 populate_metadata read hostname 'smithi052' 2024-06-10T03:00:11.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.847+0000 7efe684c9b80 1 --2- 172.21.15.52:0/1814562179 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x5632d76b3180 0x5632d76b55a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:11.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.847+0000 7efe684c9b80 4 client.16323 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "3240692c", "release": true, "await": true}] 2024-06-10T03:00:11.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.847+0000 7efe684c9b80 1 -- 172.21.15.52:0/1814562179 --> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] -- command(tid 0: {"prefix": "quiesce db", "set_id": "3240692c", "release": true, "await": true}) -- 0x5632d7582390 con 0x5632d76b3180 2024-06-10T03:00:11.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.847+0000 7efe56ffd700 1 --2- 172.21.15.52:0/1814562179 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x5632d76b3180 0x5632d76b55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:11.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.848+0000 7efe36ffd700 10 client.16323 ms_handle_connect on v2:172.21.15.188:6838/310444836 2024-06-10T03:00:11.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:11.848+0000 7efe56ffd700 1 --2- 172.21.15.52:0/1814562179 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x5632d76b3180 0x5632d76b55a0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=54a2d402498feda3 server_cookie=bc09258408da3239 in_seq=0 out_seq=0 2024-06-10T03:00:12.638 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:00:12 smithi120 ceph-mon[36634]: pgmap v614: 129 pgs: 129 active+clean; 568 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 1.5 MiB/s wr, 477 op/s 2024-06-10T03:00:12.703 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:00:12 smithi188 ceph-mon[36599]: pgmap v614: 129 pgs: 129 active+clean; 568 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 1.5 MiB/s wr, 477 op/s 2024-06-10T03:00:12.726 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:00:12 smithi052 ceph-mon[32362]: pgmap v614: 129 pgs: 129 active+clean; 568 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 1.5 MiB/s wr, 477 op/s 2024-06-10T03:00:12.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:12.876+0000 7efe36ffd700 1 -- 172.21.15.52:0/1814562179 <== mds.0 v2:172.21.15.188:6838/310444836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5632d7582390 con 0x5632d76b3180 2024-06-10T03:00:12.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:12.876+0000 7efe36ffd700 10 client.16323 handle_command_reply: tid=0 2024-06-10T03:00:12.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:12.876+0000 7efe684c9b80 1 client.16323 shutdown 2024-06-10T03:00:12.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:12.876+0000 7efe684c9b80 1 -- 172.21.15.52:0/1814562179 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5632d762d7e0 msgr2=0x5632d763fcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:00:12.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:12.876+0000 7efe684c9b80 1 --2- 172.21.15.52:0/1814562179 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5632d762d7e0 0x5632d763fcf0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7efe48014040 tx=0x7efe480050e0 comp rx=0 tx=0).stop 2024-06-10T03:00:12.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:12.877+0000 7efe684c9b80 1 -- 172.21.15.52:0/1814562179 shutdown_connections 2024-06-10T03:00:12.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:12.877+0000 7efe684c9b80 1 --2- 172.21.15.52:0/1814562179 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x5632d76b3180 0x5632d76b55a0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:12.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:12.877+0000 7efe684c9b80 1 --2- 172.21.15.52:0/1814562179 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5632d7633970 0x5632d763cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:12.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:12.877+0000 7efe684c9b80 1 --2- 172.21.15.52:0/1814562179 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5632d762d7e0 0x5632d763fcf0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:12.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:12.877+0000 7efe684c9b80 1 --2- 172.21.15.52:0/1814562179 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5632d7633050 0x5632d763c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:12.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:12.877+0000 7efe36ffd700 0 client.16323 ms_handle_reset on v2:172.21.15.188:6838/310444836 2024-06-10T03:00:12.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:12.877+0000 7efe684c9b80 1 -- 172.21.15.52:0/1814562179 >> 172.21.15.52:0/1814562179 conn(0x5632d7553dc0 msgr2=0x5632d7633d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:00:12.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:12.877+0000 7efe684c9b80 1 -- 172.21.15.52:0/1814562179 shutdown_connections 2024-06-10T03:00:12.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:12.877+0000 7efe684c9b80 1 -- 172.21.15.52:0/1814562179 wait complete. 2024-06-10T03:00:12.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:12.877+0000 7efe684c9b80 20 client.16323 trim_cache size 0 max 16384 2024-06-10T03:00:12.879 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-10T03:00:12.879 INFO:teuthology.orchestra.run.smithi052.stdout: "epoch": 36, 2024-06-10T03:00:12.879 INFO:teuthology.orchestra.run.smithi052.stdout: "leader": 24484, 2024-06-10T03:00:12.879 INFO:teuthology.orchestra.run.smithi052.stdout: "set_version": 16, 2024-06-10T03:00:12.879 INFO:teuthology.orchestra.run.smithi052.stdout: "sets": { 2024-06-10T03:00:12.879 INFO:teuthology.orchestra.run.smithi052.stdout: "3240692c": { 2024-06-10T03:00:12.879 INFO:teuthology.orchestra.run.smithi052.stdout: "version": 16, 2024-06-10T03:00:12.879 INFO:teuthology.orchestra.run.smithi052.stdout: "age_ref": 0.0, 2024-06-10T03:00:12.879 INFO:teuthology.orchestra.run.smithi052.stdout: "state": { 2024-06-10T03:00:12.879 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "RELEASED", 2024-06-10T03:00:12.879 INFO:teuthology.orchestra.run.smithi052.stdout: "age": 0.0 2024-06-10T03:00:12.880 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:00:12.880 INFO:teuthology.orchestra.run.smithi052.stdout: "timeout": 90.0, 2024-06-10T03:00:12.880 INFO:teuthology.orchestra.run.smithi052.stdout: "expiration": 149.4, 2024-06-10T03:00:12.880 INFO:teuthology.orchestra.run.smithi052.stdout: "members": { 2024-06-10T03:00:12.880 INFO:teuthology.orchestra.run.smithi052.stdout: "file:/": { 2024-06-10T03:00:12.880 INFO:teuthology.orchestra.run.smithi052.stdout: "excluded": false, 2024-06-10T03:00:12.880 INFO:teuthology.orchestra.run.smithi052.stdout: "state": { 2024-06-10T03:00:12.880 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "RELEASED", 2024-06-10T03:00:12.880 INFO:teuthology.orchestra.run.smithi052.stdout: "age": 0.0 2024-06-10T03:00:12.880 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:00:12.880 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:00:12.880 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:00:12.880 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:00:12.880 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:00:12.881 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-10T03:00:12.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:12.879+0000 7efe5dc46700 1 -- 172.21.15.52:0/3540095069 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7efe400a2c20 msgr2=0x7efe400a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:00:12.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:12.879+0000 7efe5dc46700 1 --2- 172.21.15.52:0/3540095069 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7efe400a2c20 0x7efe400a50e0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7efe58199b50 tx=0x7efe48005d80 comp rx=0 tx=0).stop 2024-06-10T03:00:12.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:12.879+0000 7efe5dc46700 1 -- 172.21.15.52:0/3540095069 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efe58101d50 msgr2=0x7efe5819ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:00:12.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:12.879+0000 7efe5dc46700 1 --2- 172.21.15.52:0/3540095069 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efe58101d50 0x7efe5819ed90 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7efe4c000c00 tx=0x7efe4c0047b0 comp rx=0 tx=0).stop 2024-06-10T03:00:12.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:12.879+0000 7efe5dc46700 1 -- 172.21.15.52:0/3540095069 shutdown_connections 2024-06-10T03:00:12.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:12.879+0000 7efe5dc46700 1 --2- 172.21.15.52:0/3540095069 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efe58107400 0x7efe58198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:12.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:12.879+0000 7efe5dc46700 1 --2- 172.21.15.52:0/3540095069 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efe58101390 0x7efe5819e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:12.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:12.879+0000 7efe5dc46700 1 --2- 172.21.15.52:0/3540095069 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7efe400a2c20 0x7efe400a50e0 secure :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7efe58199b50 tx=0x7efe48005d80 comp rx=0 tx=0).stop 2024-06-10T03:00:12.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:12.879+0000 7efe5dc46700 1 --2- 172.21.15.52:0/3540095069 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efe58101d50 0x7efe5819ed90 secure :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7efe4c000c00 tx=0x7efe4c0047b0 comp rx=0 tx=0).stop 2024-06-10T03:00:12.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:12.879+0000 7efe5dc46700 1 -- 172.21.15.52:0/3540095069 >> 172.21.15.52:0/3540095069 conn(0x7efe58078340 msgr2=0x7efe58100550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:00:12.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:12.880+0000 7efe5dc46700 1 -- 172.21.15.52:0/3540095069 shutdown_connections 2024-06-10T03:00:12.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:12.881+0000 7efe5dc46700 1 -- 172.21.15.52:0/3540095069 wait complete. 2024-06-10T03:00:13.673 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '3240692c', total seconds elapsed: 36.5 2024-06-10T03:00:13.673 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 29.45 2024-06-10T03:00:13.673 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T03:00:14.044 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T03:00:14.476 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:00:14 smithi052 ceph-mon[32362]: pgmap v615: 129 pgs: 129 active+clean; 569 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 1.4 MiB/s wr, 506 op/s 2024-06-10T03:00:14.638 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:00:14 smithi120 ceph-mon[36634]: pgmap v615: 129 pgs: 129 active+clean; 569 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 1.4 MiB/s wr, 506 op/s 2024-06-10T03:00:14.707 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:00:14 smithi188 ceph-mon[36599]: pgmap v615: 129 pgs: 129 active+clean; 569 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 1.4 MiB/s wr, 506 op/s 2024-06-10T03:00:16.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:16.043+0000 7fe90349b700 1 -- 172.21.15.52:0/2422342277 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe8fc103120 msgr2=0x7fe8fc107130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:00:16.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:16.043+0000 7fe90349b700 1 --2- 172.21.15.52:0/2422342277 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe8fc103120 0x7fe8fc107130 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fe8ec0099c0 tx=0x7fe8ec009cd0 comp rx=0 tx=0).stop 2024-06-10T03:00:16.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:16.043+0000 7fe90349b700 1 -- 172.21.15.52:0/2422342277 shutdown_connections 2024-06-10T03:00:16.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:16.043+0000 7fe90349b700 1 --2- 172.21.15.52:0/2422342277 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe8fc107860 0x7fe8fc10f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:16.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:16.043+0000 7fe90349b700 1 --2- 172.21.15.52:0/2422342277 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe8fc102770 0x7fe8fc102b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:16.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:16.043+0000 7fe90349b700 1 --2- 172.21.15.52:0/2422342277 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe8fc103120 0x7fe8fc107130 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:16.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:16.043+0000 7fe90349b700 1 -- 172.21.15.52:0/2422342277 >> 172.21.15.52:0/2422342277 conn(0x7fe8fc079600 msgr2=0x7fe8fc079a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:00:16.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:16.043+0000 7fe90349b700 1 -- 172.21.15.52:0/2422342277 shutdown_connections 2024-06-10T03:00:16.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:16.043+0000 7fe90349b700 1 -- 172.21.15.52:0/2422342277 wait complete. 2024-06-10T03:00:16.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:16.043+0000 7fe90349b700 1 Processor -- start 2024-06-10T03:00:16.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:16.043+0000 7fe90349b700 1 -- start start 2024-06-10T03:00:16.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:16.044+0000 7fe90349b700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe8fc102770 0x7fe8fc19e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:16.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:16.044+0000 7fe90349b700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe8fc103120 0x7fe8fc19ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:16.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:16.044+0000 7fe90349b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe8fc107860 0x7fe8fc1989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:16.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:16.044+0000 7fe90349b700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fe8fc112790 con 0x7fe8fc107860 2024-06-10T03:00:16.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:16.044+0000 7fe90349b700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe8fc112610 con 0x7fe8fc102770 2024-06-10T03:00:16.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:16.044+0000 7fe90349b700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fe8fc112910 con 0x7fe8fc103120 2024-06-10T03:00:16.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:16.044+0000 7fe901a38700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe8fc107860 0x7fe8fc1989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:16.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:16.044+0000 7fe901a38700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe8fc107860 0x7fe8fc1989f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:50280/0 (socket says 172.21.15.52:50280) 2024-06-10T03:00:16.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:16.044+0000 7fe901a38700 1 -- 172.21.15.52:0/2888699648 learned_addr learned my addr 172.21.15.52:0/2888699648 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:00:16.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:16.044+0000 7fe900a36700 1 --2- 172.21.15.52:0/2888699648 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe8fc103120 0x7fe8fc19ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:16.046 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:16.044+0000 7fe901237700 1 --2- 172.21.15.52:0/2888699648 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe8fc102770 0x7fe8fc19e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:16.046 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:16.044+0000 7fe901a38700 1 -- 172.21.15.52:0/2888699648 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe8fc103120 msgr2=0x7fe8fc19ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:00:16.046 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:16.044+0000 7fe901a38700 1 --2- 172.21.15.52:0/2888699648 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe8fc103120 0x7fe8fc19ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:16.046 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:16.044+0000 7fe901a38700 1 -- 172.21.15.52:0/2888699648 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe8fc102770 msgr2=0x7fe8fc19e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:00:16.046 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:16.044+0000 7fe901a38700 1 --2- 172.21.15.52:0/2888699648 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe8fc102770 0x7fe8fc19e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:16.046 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:16.044+0000 7fe901a38700 1 -- 172.21.15.52:0/2888699648 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8fc199150 con 0x7fe8fc107860 2024-06-10T03:00:16.046 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:16.044+0000 7fe900a36700 1 --2- 172.21.15.52:0/2888699648 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe8fc103120 0x7fe8fc19ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T03:00:16.046 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:16.044+0000 7fe901a38700 1 --2- 172.21.15.52:0/2888699648 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe8fc107860 0x7fe8fc1989f0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fe8f400e940 tx=0x7fe8f400ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:00:16.047 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:16.044+0000 7fe8f27fc700 1 -- 172.21.15.52:0/2888699648 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8f4004a30 con 0x7fe8fc107860 2024-06-10T03:00:16.047 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:16.045+0000 7fe90349b700 1 -- 172.21.15.52:0/2888699648 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8fc199440 con 0x7fe8fc107860 2024-06-10T03:00:16.047 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:16.045+0000 7fe8f27fc700 1 -- 172.21.15.52:0/2888699648 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe8f4005500 con 0x7fe8fc107860 2024-06-10T03:00:16.047 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:16.045+0000 7fe8f27fc700 1 -- 172.21.15.52:0/2888699648 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8f400ce20 con 0x7fe8fc107860 2024-06-10T03:00:16.047 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:16.045+0000 7fe90349b700 1 -- 172.21.15.52:0/2888699648 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8fc1a56b0 con 0x7fe8fc107860 2024-06-10T03:00:16.047 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:16.046+0000 7fe90349b700 1 -- 172.21.15.52:0/2888699648 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8fc069180 con 0x7fe8fc107860 2024-06-10T03:00:16.047 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:16.047+0000 7fe8f27fc700 1 -- 172.21.15.52:0/2888699648 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fe8f4005000 con 0x7fe8fc107860 2024-06-10T03:00:16.048 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:16.048+0000 7fe8f27fc700 1 --2- 172.21.15.52:0/2888699648 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fe8e80a2ca0 0x7fe8e80a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:16.048 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:16.048+0000 7fe8f27fc700 1 -- 172.21.15.52:0/2888699648 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7fe8f40cf6c0 con 0x7fe8fc107860 2024-06-10T03:00:16.048 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:16.048+0000 7fe901237700 1 --2- 172.21.15.52:0/2888699648 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fe8e80a2ca0 0x7fe8e80a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:16.048 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:16.048+0000 7fe901237700 1 --2- 172.21.15.52:0/2888699648 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fe8e80a2ca0 0x7fe8e80a5160 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fe8fc107460 tx=0x7fe8f800b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:00:16.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:16.051+0000 7fe8f27fc700 1 -- 172.21.15.52:0/2888699648 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fe8f4053850 con 0x7fe8fc107860 2024-06-10T03:00:16.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:16.314+0000 7fe90349b700 1 -- 172.21.15.52:0/2888699648 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe8fc02dca0 con 0x7fe8fc107860 2024-06-10T03:00:16.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:16.315+0000 7fe8f27fc700 1 -- 172.21.15.52:0/2888699648 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+11945 (secure 0 0 0) 0x7fe8f409c270 con 0x7fe8fc107860 2024-06-10T03:00:16.315 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-10T03:00:16.316 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":36,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/889673247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":889673247},{"type":"v1","addr":"172.21.15.52:6835","nonce":889673247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/3615663511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":3615663511},{"type":"v1","addr":"172.21.15.52:6837","nonce":3615663511}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":36,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T02:50:03.889766+0000","modified":"2024-06-10T02:59:27.720611+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24484,"mds_1":24469,"mds_2":24463,"mds_3":24449,"mds_4":24428},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14571":{"gid":14571,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6839/3059599042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":3059599042},{"type":"v1","addr":"172.21.15.52:6839","nonce":3059599042}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14583":{"gid":14583,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6841/3039275521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":3039275521},{"type":"v1","addr":"172.21.15.52:6841","nonce":3039275521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24383":{"gid":24383,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6833/2609002896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":2609002896},{"type":"v1","addr":"172.21.15.120:6833","nonce":2609002896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6835/1996370288","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":1996370288},{"type":"v1","addr":"172.21.15.120:6835","nonce":1996370288}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6833/529564979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":529564979},{"type":"v1","addr":"172.21.15.188:6833","nonce":529564979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":25,"addr":"172.21.15.120:6837/4132440952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":4132440952},{"type":"v1","addr":"172.21.15.120:6837","nonce":4132440952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"b","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.120:6839/1672436487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1672436487},{"type":"v1","addr":"172.21.15.120:6839","nonce":1672436487}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":2,"incarnation":22,"state":"up:active","state_seq":26,"addr":"172.21.15.188:6835/3904544259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":3904544259},{"type":"v1","addr":"172.21.15.188:6835","nonce":3904544259}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.188:6837/819317726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":819317726},{"type":"v1","addr":"172.21.15.188:6837","nonce":819317726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.188:6839/310444836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":310444836},{"type":"v1","addr":"172.21.15.188:6839","nonce":310444836}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24484,"qdb_cluster":[24469,24484,24463,24428,24449]},"id":1}]} 2024-06-10T03:00:16.317 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:16.317+0000 7fe90349b700 1 -- 172.21.15.52:0/2888699648 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fe8e80a2ca0 msgr2=0x7fe8e80a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:00:16.317 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:16.317+0000 7fe90349b700 1 --2- 172.21.15.52:0/2888699648 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fe8e80a2ca0 0x7fe8e80a5160 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fe8fc107460 tx=0x7fe8f800b040 comp rx=0 tx=0).stop 2024-06-10T03:00:16.318 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:16.317+0000 7fe90349b700 1 -- 172.21.15.52:0/2888699648 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe8fc107860 msgr2=0x7fe8fc1989f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:00:16.318 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:16.317+0000 7fe90349b700 1 --2- 172.21.15.52:0/2888699648 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe8fc107860 0x7fe8fc1989f0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fe8f400e940 tx=0x7fe8f400ec50 comp rx=0 tx=0).stop 2024-06-10T03:00:16.318 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:16.317+0000 7fe90349b700 1 -- 172.21.15.52:0/2888699648 shutdown_connections 2024-06-10T03:00:16.318 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:16.317+0000 7fe90349b700 1 --2- 172.21.15.52:0/2888699648 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fe8e80a2ca0 0x7fe8e80a5160 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:16.318 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:16.317+0000 7fe90349b700 1 --2- 172.21.15.52:0/2888699648 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe8fc107860 0x7fe8fc1989f0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:16.318 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:16.317+0000 7fe90349b700 1 --2- 172.21.15.52:0/2888699648 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe8fc102770 0x7fe8fc19e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:16.318 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:16.317+0000 7fe90349b700 1 --2- 172.21.15.52:0/2888699648 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe8fc103120 0x7fe8fc19ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:16.318 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:16.317+0000 7fe90349b700 1 -- 172.21.15.52:0/2888699648 >> 172.21.15.52:0/2888699648 conn(0x7fe8fc079600 msgr2=0x7fe8fc100560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:00:16.318 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:16.318+0000 7fe90349b700 1 -- 172.21.15.52:0/2888699648 shutdown_connections 2024-06-10T03:00:16.318 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:16.318+0000 7fe90349b700 1 -- 172.21.15.52:0/2888699648 wait complete. 2024-06-10T03:00:16.319 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 36 2024-06-10T03:00:16.638 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:00:16 smithi120 ceph-mon[36634]: pgmap v616: 129 pgs: 129 active+clean; 571 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 1.1 MiB/s wr, 421 op/s 2024-06-10T03:00:16.697 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:00:16 smithi052 ceph-mon[32362]: pgmap v616: 129 pgs: 129 active+clean; 571 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 1.1 MiB/s wr, 421 op/s 2024-06-10T03:00:16.703 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:00:16 smithi188 ceph-mon[36599]: pgmap v616: 129 pgs: 129 active+clean; 571 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 1.1 MiB/s wr, 421 op/s 2024-06-10T03:00:17.105 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24484 quiesce db / --timeout 90 --expiration 149.45 --await-for 45.0' 2024-06-10T03:00:17.105 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24484 quiesce db / --timeout 90 --expiration 149.45 --await-for 45.0 2024-06-10T03:00:17.249 INFO:tasks.workunit.client.0.smithi052.stdout: 97 164491 86 114991 92 88435 195 2024-06-10T03:00:17.378 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T03:00:17.476 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:00:17 smithi052 ceph-mon[32362]: from='client.? 172.21.15.52:0/2888699648' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T03:00:17.638 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:00:17 smithi120 ceph-mon[36634]: from='client.? 172.21.15.52:0/2888699648' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T03:00:17.703 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:00:17 smithi188 ceph-mon[36599]: from='client.? 172.21.15.52:0/2888699648' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T03:00:18.638 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:00:18 smithi120 ceph-mon[36634]: pgmap v617: 129 pgs: 129 active+clean; 574 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 792 KiB/s wr, 373 op/s 2024-06-10T03:00:18.703 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:00:18 smithi188 ceph-mon[36599]: pgmap v617: 129 pgs: 129 active+clean; 574 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 792 KiB/s wr, 373 op/s 2024-06-10T03:00:18.726 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:00:18 smithi052 ceph-mon[32362]: pgmap v617: 129 pgs: 129 active+clean; 574 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 792 KiB/s wr, 373 op/s 2024-06-10T03:00:18.951 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-10T03:00:19.348 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T03:00:19.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.349+0000 7f3b4ace2700 1 -- 172.21.15.52:0/681537476 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3b44107850 msgr2=0x7f3b4410f3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:00:19.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.349+0000 7f3b4ace2700 1 --2- 172.21.15.52:0/681537476 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3b44107850 0x7f3b4410f3d0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f3b3c00b0d0 tx=0x7f3b3c00b3e0 comp rx=0 tx=0).stop 2024-06-10T03:00:19.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.349+0000 7f3b4ace2700 1 -- 172.21.15.52:0/681537476 shutdown_connections 2024-06-10T03:00:19.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.349+0000 7f3b4ace2700 1 --2- 172.21.15.52:0/681537476 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3b44107850 0x7f3b4410f3d0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:19.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.349+0000 7f3b4ace2700 1 --2- 172.21.15.52:0/681537476 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3b4406a3d0 0x7f3b4406a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:19.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.349+0000 7f3b4ace2700 1 --2- 172.21.15.52:0/681537476 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3b4406ad80 0x7f3b44107120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:19.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.349+0000 7f3b4ace2700 1 -- 172.21.15.52:0/681537476 >> 172.21.15.52:0/681537476 conn(0x7f3b44078330 msgr2=0x7f3b44078740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:00:19.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.349+0000 7f3b4ace2700 1 -- 172.21.15.52:0/681537476 shutdown_connections 2024-06-10T03:00:19.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.349+0000 7f3b4ace2700 1 -- 172.21.15.52:0/681537476 wait complete. 2024-06-10T03:00:19.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.350+0000 7f3b4ace2700 1 Processor -- start 2024-06-10T03:00:19.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.350+0000 7f3b4ace2700 1 -- start start 2024-06-10T03:00:19.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.350+0000 7f3b4ace2700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3b4406a3d0 0x7f3b441a0880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:19.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.350+0000 7f3b4ace2700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3b4406ad80 0x7f3b441a0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:19.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.350+0000 7f3b4ace2700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3b44107850 0x7f3b4419aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:19.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.350+0000 7f3b4ace2700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f3b44112770 con 0x7f3b4406ad80 2024-06-10T03:00:19.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.350+0000 7f3b4ace2700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f3b441125f0 con 0x7f3b4406a3d0 2024-06-10T03:00:19.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.350+0000 7f3b4ace2700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f3b441128f0 con 0x7f3b44107850 2024-06-10T03:00:19.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.350+0000 7f3b4927f700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3b44107850 0x7f3b4419aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:19.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.350+0000 7f3b4927f700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3b44107850 0x7f3b4419aa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.52:33868/0 (socket says 172.21.15.52:33868) 2024-06-10T03:00:19.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.350+0000 7f3b4927f700 1 -- 172.21.15.52:0/1352397341 learned_addr learned my addr 172.21.15.52:0/1352397341 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:00:19.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.351+0000 7f3b48a7e700 1 --2- 172.21.15.52:0/1352397341 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3b4406a3d0 0x7f3b441a0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:19.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.351+0000 7f3b43fff700 1 --2- 172.21.15.52:0/1352397341 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3b4406ad80 0x7f3b441a0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:19.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.351+0000 7f3b4927f700 1 -- 172.21.15.52:0/1352397341 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3b4406a3d0 msgr2=0x7f3b441a0880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:00:19.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.351+0000 7f3b4927f700 1 --2- 172.21.15.52:0/1352397341 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3b4406a3d0 0x7f3b441a0880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:19.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.351+0000 7f3b4927f700 1 -- 172.21.15.52:0/1352397341 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3b4406ad80 msgr2=0x7f3b441a0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:00:19.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.351+0000 7f3b4927f700 1 --2- 172.21.15.52:0/1352397341 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3b4406ad80 0x7f3b441a0dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:19.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.351+0000 7f3b4927f700 1 -- 172.21.15.52:0/1352397341 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b4419b1f0 con 0x7f3b44107850 2024-06-10T03:00:19.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.351+0000 7f3b4927f700 1 --2- 172.21.15.52:0/1352397341 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3b44107850 0x7f3b4419aa90 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f3b3c006070 tx=0x7f3b3c007a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:00:19.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.351+0000 7f3b41ffb700 1 -- 172.21.15.52:0/1352397341 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b3c003c00 con 0x7f3b44107850 2024-06-10T03:00:19.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.351+0000 7f3b4ace2700 1 -- 172.21.15.52:0/1352397341 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b4419b480 con 0x7f3b44107850 2024-06-10T03:00:19.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.351+0000 7f3b41ffb700 1 -- 172.21.15.52:0/1352397341 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b3c022070 con 0x7f3b44107850 2024-06-10T03:00:19.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.351+0000 7f3b41ffb700 1 -- 172.21.15.52:0/1352397341 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b3c01b6c0 con 0x7f3b44107850 2024-06-10T03:00:19.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.352+0000 7f3b4ace2700 1 -- 172.21.15.52:0/1352397341 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b441a7750 con 0x7f3b44107850 2024-06-10T03:00:19.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.355+0000 7f3b41ffb700 1 -- 172.21.15.52:0/1352397341 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f3b3c01d470 con 0x7f3b44107850 2024-06-10T03:00:19.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.356+0000 7f3b41ffb700 1 --2- 172.21.15.52:0/1352397341 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f3b340a2cc0 0x7f3b340a5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:19.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.356+0000 7f3b48a7e700 1 --2- 172.21.15.52:0/1352397341 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f3b340a2cc0 0x7f3b340a5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:19.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.356+0000 7f3b41ffb700 1 -- 172.21.15.52:0/1352397341 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f3b3c0d0310 con 0x7f3b44107850 2024-06-10T03:00:19.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.357+0000 7f3b48a7e700 1 --2- 172.21.15.52:0/1352397341 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f3b340a2cc0 0x7f3b340a5180 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f3b44107220 tx=0x7f3b38009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:00:19.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.357+0000 7f3b55565b80 1 Processor -- start 2024-06-10T03:00:19.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.357+0000 7f3b55565b80 1 -- start start 2024-06-10T03:00:19.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.357+0000 7f3b55565b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e5068ccce0 0x55e5068cd0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:19.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.357+0000 7f3b55565b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e5068cd600 0x55e5068c6f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:19.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.357+0000 7f3b55565b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e5068c7470 0x55e5068c78b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:19.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.357+0000 7f3b55565b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55e5068189b0 con 0x55e5068c7470 2024-06-10T03:00:19.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.357+0000 7f3b55565b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55e5067c08c0 con 0x55e5068ccce0 2024-06-10T03:00:19.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.357+0000 7f3b55565b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55e506814970 con 0x55e5068cd600 2024-06-10T03:00:19.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.357+0000 7f3b43fff700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e5068cd600 0x55e5068c6f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:19.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.357+0000 7f3b4927f700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e5068c7470 0x55e5068c78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:19.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.357+0000 7f3b43fff700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e5068cd600 0x55e5068c6f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.52:33880/0 (socket says 172.21.15.52:33880) 2024-06-10T03:00:19.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.357+0000 7f3b43fff700 1 -- 172.21.15.52:0/2138808135 learned_addr learned my addr 172.21.15.52:0/2138808135 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:00:19.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.357+0000 7f3b48a7e700 1 --2- 172.21.15.52:0/2138808135 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e5068ccce0 0x55e5068cd0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:19.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.358+0000 7f3b4927f700 1 -- 172.21.15.52:0/2138808135 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e5068cd600 msgr2=0x55e5068c6f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:00:19.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.358+0000 7f3b4927f700 1 --2- 172.21.15.52:0/2138808135 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e5068cd600 0x55e5068c6f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:19.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.358+0000 7f3b4927f700 1 -- 172.21.15.52:0/2138808135 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e5068ccce0 msgr2=0x55e5068cd0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:00:19.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.358+0000 7f3b4927f700 1 --2- 172.21.15.52:0/2138808135 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e5068ccce0 0x55e5068cd0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:19.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.358+0000 7f3b4927f700 1 -- 172.21.15.52:0/2138808135 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e5067b70f0 con 0x55e5068c7470 2024-06-10T03:00:19.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.358+0000 7f3b48a7e700 1 --2- 172.21.15.52:0/2138808135 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e5068ccce0 0x55e5068cd0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T03:00:19.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.358+0000 7f3b4927f700 1 --2- 172.21.15.52:0/2138808135 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e5068c7470 0x55e5068c78b0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f3b3c0cdab0 tx=0x7f3b3c01dcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dd38f15e1620bcb server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:00:19.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.359+0000 7f3b2effd700 1 -- 172.21.15.52:0/2138808135 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b3c09c1b0 con 0x55e5068c7470 2024-06-10T03:00:19.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.359+0000 7f3b2effd700 1 -- 172.21.15.52:0/2138808135 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b3c09cf50 con 0x55e5068c7470 2024-06-10T03:00:19.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.359+0000 7f3b2effd700 1 -- 172.21.15.52:0/2138808135 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b3c0d0170 con 0x55e5068c7470 2024-06-10T03:00:19.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.359+0000 7f3b55565b80 1 -- 172.21.15.52:0/2138808135 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e5068c7470 msgr2=0x55e5068c78b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:00:19.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.359+0000 7f3b55565b80 1 --2- 172.21.15.52:0/2138808135 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e5068c7470 0x55e5068c78b0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f3b3c0cdab0 tx=0x7f3b3c01dcb0 comp rx=0 tx=0).stop 2024-06-10T03:00:19.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.360+0000 7f3b55565b80 1 -- 172.21.15.52:0/2138808135 shutdown_connections 2024-06-10T03:00:19.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.360+0000 7f3b55565b80 1 --2- 172.21.15.52:0/2138808135 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e5068c7470 0x55e5068c78b0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:19.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.360+0000 7f3b55565b80 1 --2- 172.21.15.52:0/2138808135 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e5068ccce0 0x55e5068cd0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:19.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.360+0000 7f3b55565b80 1 --2- 172.21.15.52:0/2138808135 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e5068cd600 0x55e5068c6f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:19.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.360+0000 7f3b55565b80 1 -- 172.21.15.52:0/2138808135 >> 172.21.15.52:0/2138808135 conn(0x55e5067f1e60 msgr2=0x55e5068c21c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:00:19.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.360+0000 7f3b55565b80 1 -- 172.21.15.52:0/2138808135 shutdown_connections 2024-06-10T03:00:19.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.360+0000 7f3b55565b80 1 -- 172.21.15.52:0/2138808135 wait complete. 2024-06-10T03:00:19.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.360+0000 7f3b55565b80 1 -- start start 2024-06-10T03:00:19.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.360+0000 7f3b55565b80 1 Processor -- start 2024-06-10T03:00:19.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.361+0000 7f3b55565b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e5068c7470 0x55e5068d3390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:19.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.361+0000 7f3b55565b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e5068ccce0 0x55e5068d0690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:19.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.361+0000 7f3b55565b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e5068cd600 0x55e5068d0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:19.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.361+0000 7f3b55565b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55e5068d3d90 con 0x55e5068ccce0 2024-06-10T03:00:19.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.361+0000 7f3b55565b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55e5067b8d90 con 0x55e5068cd600 2024-06-10T03:00:19.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.361+0000 7f3b55565b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55e5068d3f10 con 0x55e5068c7470 2024-06-10T03:00:19.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.361+0000 7f3b43fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e5068ccce0 0x55e5068d0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:19.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.361+0000 7f3b43fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e5068ccce0 0x55e5068d0690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:50312/0 (socket says 172.21.15.52:50312) 2024-06-10T03:00:19.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.361+0000 7f3b43fff700 1 -- 172.21.15.52:0/3107656491 learned_addr learned my addr 172.21.15.52:0/3107656491 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:00:19.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.361+0000 7f3b48a7e700 1 --2- 172.21.15.52:0/3107656491 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e5068c7470 0x55e5068d3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:19.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.361+0000 7f3b55565b80 10 client.0 _ll_register_callbacks cb 0x55e5067b2ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T03:00:19.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.361+0000 7f3b4927f700 1 --2- 172.21.15.52:0/3107656491 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e5068cd600 0x55e5068d0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:19.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.361+0000 7f3b4927f700 1 -- 172.21.15.52:0/3107656491 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e5068c7470 msgr2=0x55e5068d3390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:00:19.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.361+0000 7f3b4927f700 1 --2- 172.21.15.52:0/3107656491 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e5068c7470 0x55e5068d3390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:19.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.361+0000 7f3b4927f700 1 -- 172.21.15.52:0/3107656491 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e5068ccce0 msgr2=0x55e5068d0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:00:19.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.361+0000 7f3b4927f700 1 --2- 172.21.15.52:0/3107656491 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e5068ccce0 0x55e5068d0690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:19.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.361+0000 7f3b4927f700 1 -- 172.21.15.52:0/3107656491 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e506767a70 con 0x55e5068cd600 2024-06-10T03:00:19.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.361+0000 7f3b43fff700 1 --2- 172.21.15.52:0/3107656491 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e5068ccce0 0x55e5068d0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T03:00:19.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.362+0000 7f3b4927f700 1 --2- 172.21.15.52:0/3107656491 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e5068cd600 0x55e5068d0bd0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f3b3c0cdab0 tx=0x7f3b3c01dbb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc4b3de1ee6d57a9 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:00:19.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.362+0000 7f3b23fff700 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-06-10T03:00:19.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.362+0000 7f3b23fff700 1 -- 172.21.15.52:0/3107656491 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b3c09c1b0 con 0x55e5068cd600 2024-06-10T03:00:19.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.362+0000 7f3b23fff700 1 -- 172.21.15.52:0/3107656491 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f3b3c0a3050 con 0x55e5068cd600 2024-06-10T03:00:19.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.362+0000 7f3b55565b80 1 -- 172.21.15.52:0/3107656491 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3b3c09cf50 con 0x55e5068cd600 2024-06-10T03:00:19.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.362+0000 7f3b23fff700 1 -- 172.21.15.52:0/3107656491 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f3b3c09c850 con 0x55e5068cd600 2024-06-10T03:00:19.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.362+0000 7f3b55565b80 10 client.25886 fetch_fsmap learned FSMap version 36 2024-06-10T03:00:19.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.362+0000 7f3b55565b80 1 -- 172.21.15.52:0/3107656491 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55e5068d3d90 con 0x55e5068cd600 2024-06-10T03:00:19.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.362+0000 7f3b23fff700 1 -- 172.21.15.52:0/3107656491 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b3c09cf50 con 0x55e5068cd600 2024-06-10T03:00:19.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.362+0000 7f3b23fff700 1 -- 172.21.15.52:0/3107656491 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 36) ==== 8378+0+0 (secure 0 0 0) 0x7f3b3c0b0ce0 con 0x55e5068cd600 2024-06-10T03:00:19.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.363+0000 7f3b55565b80 10 client.25886 fetch_fsmap finished waiting for FSMap version 36 2024-06-10T03:00:19.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.363+0000 7f3b55565b80 10 client.25886 resolve_mds: validated gid 24484 aka daemon mds.i 2024-06-10T03:00:19.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.363+0000 7f3b55565b80 20 client.25886 populate_metadata read hostname 'smithi052' 2024-06-10T03:00:19.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.363+0000 7f3b55565b80 1 --2- 172.21.15.52:0/3107656491 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55e5068e07c0 0x55e5068e2bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:19.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.363+0000 7f3b55565b80 4 client.25886 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T03:00:19.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.363+0000 7f3b55565b80 1 -- 172.21.15.52:0/3107656491 --> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e506825eb0 con 0x55e5068e07c0 2024-06-10T03:00:19.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.363+0000 7f3b48a7e700 1 --2- 172.21.15.52:0/3107656491 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55e5068e07c0 0x55e5068e2bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:19.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.363+0000 7f3b48a7e700 1 --2- 172.21.15.52:0/3107656491 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55e5068e07c0 0x55e5068e2bb0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7465c9653b7ce77b server_cookie=6b44df6ac03dca8f in_seq=0 out_seq=0 2024-06-10T03:00:19.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.363+0000 7f3b23fff700 10 client.25886 ms_handle_connect on v2:172.21.15.188:6838/310444836 2024-06-10T03:00:19.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.364+0000 7f3b23fff700 1 -- 172.21.15.52:0/3107656491 <== mds.0 v2:172.21.15.188:6838/310444836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55e506825eb0 con 0x55e5068e07c0 2024-06-10T03:00:19.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.364+0000 7f3b23fff700 10 client.25886 handle_command_reply: tid=0 2024-06-10T03:00:19.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.364+0000 7f3b55565b80 1 client.25886 shutdown 2024-06-10T03:00:19.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.365+0000 7f3b55565b80 1 -- 172.21.15.52:0/3107656491 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e5068cd600 msgr2=0x55e5068d0bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:00:19.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.365+0000 7f3b55565b80 1 --2- 172.21.15.52:0/3107656491 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e5068cd600 0x55e5068d0bd0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f3b3c0cdab0 tx=0x7f3b3c01dbb0 comp rx=0 tx=0).stop 2024-06-10T03:00:19.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.365+0000 7f3b55565b80 1 -- 172.21.15.52:0/3107656491 shutdown_connections 2024-06-10T03:00:19.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.365+0000 7f3b55565b80 1 --2- 172.21.15.52:0/3107656491 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55e5068e07c0 0x55e5068e2bb0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:19.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.365+0000 7f3b55565b80 1 --2- 172.21.15.52:0/3107656491 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e5068cd600 0x55e5068d0bd0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:19.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.365+0000 7f3b55565b80 1 --2- 172.21.15.52:0/3107656491 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e5068c7470 0x55e5068d3390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:19.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.365+0000 7f3b55565b80 1 --2- 172.21.15.52:0/3107656491 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e5068ccce0 0x55e5068d0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:19.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.365+0000 7f3b55565b80 1 -- 172.21.15.52:0/3107656491 >> 172.21.15.52:0/3107656491 conn(0x55e5067f1e60 msgr2=0x55e5068c5f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:00:19.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.365+0000 7f3b23fff700 0 client.25886 ms_handle_reset on v2:172.21.15.188:6838/310444836 2024-06-10T03:00:19.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.365+0000 7f3b55565b80 1 -- 172.21.15.52:0/3107656491 shutdown_connections 2024-06-10T03:00:19.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.365+0000 7f3b55565b80 1 -- 172.21.15.52:0/3107656491 wait complete. 2024-06-10T03:00:19.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.365+0000 7f3b55565b80 20 client.25886 trim_cache size 0 max 16384 2024-06-10T03:00:19.388 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.388+0000 7f3b55565b80 1 Processor -- start 2024-06-10T03:00:19.388 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.388+0000 7f3b55565b80 1 -- start start 2024-06-10T03:00:19.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.389+0000 7f3b55565b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e5068c7470 0x55e5068cdac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:19.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.389+0000 7f3b55565b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e5068ccce0 0x55e5068ce000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:19.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.389+0000 7f3b48a7e700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e5068c7470 0x55e5068cdac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:19.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.389+0000 7f3b48a7e700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e5068c7470 0x55e5068cdac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.52:58382/0 (socket says 172.21.15.52:58382) 2024-06-10T03:00:19.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.389+0000 7f3b55565b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e5068cd600 0x55e5068e8440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:19.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.389+0000 7f3b48a7e700 1 -- 172.21.15.52:0/1759780271 learned_addr learned my addr 172.21.15.52:0/1759780271 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:00:19.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.389+0000 7f3b43fff700 1 --2- 172.21.15.52:0/1759780271 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e5068ccce0 0x55e5068ce000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:19.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.389+0000 7f3b55565b80 1 -- 172.21.15.52:0/1759780271 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f3b441b0350 con 0x55e5068cd600 2024-06-10T03:00:19.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.389+0000 7f3b55565b80 1 -- 172.21.15.52:0/1759780271 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f3b441b1f50 con 0x55e5068c7470 2024-06-10T03:00:19.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.389+0000 7f3b55565b80 1 -- 172.21.15.52:0/1759780271 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f3b441b0a20 con 0x55e5068ccce0 2024-06-10T03:00:19.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.390+0000 7f3b48a7e700 1 -- 172.21.15.52:0/1759780271 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e5068ccce0 msgr2=0x55e5068ce000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:00:19.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.390+0000 7f3b4927f700 1 --2- 172.21.15.52:0/1759780271 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e5068cd600 0x55e5068e8440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:19.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.390+0000 7f3b48a7e700 1 --2- 172.21.15.52:0/1759780271 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e5068ccce0 0x55e5068ce000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:19.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.390+0000 7f3b48a7e700 1 -- 172.21.15.52:0/1759780271 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e5068cd600 msgr2=0x55e5068e8440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:00:19.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.390+0000 7f3b48a7e700 1 --2- 172.21.15.52:0/1759780271 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e5068cd600 0x55e5068e8440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:19.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.390+0000 7f3b48a7e700 1 -- 172.21.15.52:0/1759780271 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e5068e3190 con 0x55e5068c7470 2024-06-10T03:00:19.392 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.390+0000 7f3b43fff700 1 --2- 172.21.15.52:0/1759780271 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e5068ccce0 0x55e5068ce000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T03:00:19.392 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.390+0000 7f3b4927f700 1 --2- 172.21.15.52:0/1759780271 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e5068cd600 0x55e5068e8440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T03:00:19.392 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.390+0000 7f3b48a7e700 1 --2- 172.21.15.52:0/1759780271 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e5068c7470 0x55e5068cdac0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f3b38011c60 tx=0x7f3b38017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ef46906e4654204 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:00:19.392 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.390+0000 7f3b2e7fc700 1 -- 172.21.15.52:0/1759780271 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b3801dd00 con 0x55e5068c7470 2024-06-10T03:00:19.392 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.390+0000 7f3b2e7fc700 1 -- 172.21.15.52:0/1759780271 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b38004530 con 0x55e5068c7470 2024-06-10T03:00:19.392 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.390+0000 7f3b2e7fc700 1 -- 172.21.15.52:0/1759780271 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b38005280 con 0x55e5068c7470 2024-06-10T03:00:19.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.391+0000 7f3b55565b80 1 -- 172.21.15.52:0/1759780271 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e5068c7470 msgr2=0x55e5068cdac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:00:19.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.391+0000 7f3b55565b80 1 --2- 172.21.15.52:0/1759780271 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e5068c7470 0x55e5068cdac0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f3b38011c60 tx=0x7f3b38017000 comp rx=0 tx=0).stop 2024-06-10T03:00:19.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.391+0000 7f3b55565b80 1 -- 172.21.15.52:0/1759780271 shutdown_connections 2024-06-10T03:00:19.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.391+0000 7f3b55565b80 1 --2- 172.21.15.52:0/1759780271 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e5068cd600 0x55e5068e8440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:19.395 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.391+0000 7f3b55565b80 1 --2- 172.21.15.52:0/1759780271 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e5068c7470 0x55e5068cdac0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:19.395 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.391+0000 7f3b55565b80 1 --2- 172.21.15.52:0/1759780271 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e5068ccce0 0x55e5068ce000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:19.395 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.391+0000 7f3b55565b80 1 -- 172.21.15.52:0/1759780271 >> 172.21.15.52:0/1759780271 conn(0x55e5067f1e60 msgr2=0x55e5068d0650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:00:19.395 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.391+0000 7f3b55565b80 1 -- 172.21.15.52:0/1759780271 shutdown_connections 2024-06-10T03:00:19.395 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.391+0000 7f3b55565b80 1 -- 172.21.15.52:0/1759780271 wait complete. 2024-06-10T03:00:19.395 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.391+0000 7f3b55565b80 1 -- start start 2024-06-10T03:00:19.395 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.391+0000 7f3b55565b80 1 Processor -- start 2024-06-10T03:00:19.395 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.392+0000 7f3b55565b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e5068c7470 0x55e5068d91a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:19.395 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.392+0000 7f3b55565b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e5068ccce0 0x55e5068d96e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:19.395 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.392+0000 7f3b55565b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e5068cd600 0x55e5068d6170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:19.395 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.392+0000 7f3b55565b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f3b441b1880 con 0x55e5068cd600 2024-06-10T03:00:19.395 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.392+0000 7f3b55565b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f3b441b11b0 con 0x55e5068ccce0 2024-06-10T03:00:19.395 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.392+0000 7f3b55565b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f3b441ae2f0 con 0x55e5068c7470 2024-06-10T03:00:19.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.392+0000 7f3b4927f700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e5068cd600 0x55e5068d6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:19.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.392+0000 7f3b55565b80 10 client.0 _ll_register_callbacks cb 0x55e5067b2ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T03:00:19.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.392+0000 7f3b48a7e700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e5068c7470 0x55e5068d91a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:19.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.392+0000 7f3b4927f700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e5068cd600 0x55e5068d6170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:50334/0 (socket says 172.21.15.52:50334) 2024-06-10T03:00:19.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.392+0000 7f3b4927f700 1 -- 172.21.15.52:0/2946735471 learned_addr learned my addr 172.21.15.52:0/2946735471 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:00:19.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.392+0000 7f3b43fff700 1 --2- 172.21.15.52:0/2946735471 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e5068ccce0 0x55e5068d96e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:19.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.392+0000 7f3b48a7e700 1 -- 172.21.15.52:0/2946735471 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e5068ccce0 msgr2=0x55e5068d96e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:00:19.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.392+0000 7f3b48a7e700 1 --2- 172.21.15.52:0/2946735471 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e5068ccce0 0x55e5068d96e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:19.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.392+0000 7f3b48a7e700 1 -- 172.21.15.52:0/2946735471 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e5068cd600 msgr2=0x55e5068d6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:00:19.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.392+0000 7f3b48a7e700 1 --2- 172.21.15.52:0/2946735471 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e5068cd600 0x55e5068d6170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:19.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.392+0000 7f3b48a7e700 1 -- 172.21.15.52:0/2946735471 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e5068d9e30 con 0x55e5068c7470 2024-06-10T03:00:19.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.393+0000 7f3b43fff700 1 --2- 172.21.15.52:0/2946735471 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e5068ccce0 0x55e5068d96e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T03:00:19.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.393+0000 7f3b48a7e700 1 --2- 172.21.15.52:0/2946735471 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e5068c7470 0x55e5068d91a0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f3b38000c00 tx=0x7f3b380042c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f10df1e950d63365 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:00:19.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.393+0000 7f3b23fff700 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-06-10T03:00:19.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.393+0000 7f3b23fff700 1 -- 172.21.15.52:0/2946735471 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b3801db00 con 0x55e5068c7470 2024-06-10T03:00:19.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.393+0000 7f3b55565b80 1 -- 172.21.15.52:0/2946735471 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3b38004530 con 0x55e5068c7470 2024-06-10T03:00:19.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.393+0000 7f3b23fff700 1 -- 172.21.15.52:0/2946735471 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f3b3801dca0 con 0x55e5068c7470 2024-06-10T03:00:19.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.393+0000 7f3b23fff700 1 -- 172.21.15.52:0/2946735471 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f3b38041050 con 0x55e5068c7470 2024-06-10T03:00:19.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.393+0000 7f3b55565b80 10 client.25909 fetch_fsmap learned FSMap version 36 2024-06-10T03:00:19.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.393+0000 7f3b55565b80 1 -- 172.21.15.52:0/2946735471 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55e5069541e0 con 0x55e5068c7470 2024-06-10T03:00:19.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.393+0000 7f3b23fff700 1 -- 172.21.15.52:0/2946735471 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b38004530 con 0x55e5068c7470 2024-06-10T03:00:19.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.394+0000 7f3b23fff700 1 -- 172.21.15.52:0/2946735471 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 36) ==== 8378+0+0 (secure 0 0 0) 0x7f3b38028630 con 0x55e5068c7470 2024-06-10T03:00:19.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.394+0000 7f3b55565b80 10 client.25909 fetch_fsmap finished waiting for FSMap version 36 2024-06-10T03:00:19.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.394+0000 7f3b55565b80 10 client.25909 resolve_mds: validated gid 24484 aka daemon mds.i 2024-06-10T03:00:19.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.394+0000 7f3b55565b80 20 client.25909 populate_metadata read hostname 'smithi052' 2024-06-10T03:00:19.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.394+0000 7f3b55565b80 1 --2- 172.21.15.52:0/2946735471 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55e506954900 0x55e506956cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:19.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.394+0000 7f3b55565b80 4 client.25909 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 149.45, "await_for": 45.0}] 2024-06-10T03:00:19.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.394+0000 7f3b55565b80 1 -- 172.21.15.52:0/2946735471 --> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 149.45, "await_for": 45.0}) -- 0x55e506820430 con 0x55e506954900 2024-06-10T03:00:19.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.394+0000 7f3b43fff700 1 --2- 172.21.15.52:0/2946735471 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55e506954900 0x55e506956cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:19.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.394+0000 7f3b23fff700 10 client.25909 ms_handle_connect on v2:172.21.15.188:6838/310444836 2024-06-10T03:00:19.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:19.394+0000 7f3b43fff700 1 --2- 172.21.15.52:0/2946735471 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55e506954900 0x55e506956cf0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e2203177d6a22d20 server_cookie=e5b3a12c5169d878 in_seq=0 out_seq=0 2024-06-10T03:00:20.638 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:00:20 smithi120 ceph-mon[36634]: pgmap v618: 129 pgs: 129 active+clean; 578 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 1.2 MiB/s wr, 306 op/s 2024-06-10T03:00:20.703 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:00:20 smithi188 ceph-mon[36599]: pgmap v618: 129 pgs: 129 active+clean; 578 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 1.2 MiB/s wr, 306 op/s 2024-06-10T03:00:20.727 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:00:20 smithi052 ceph-mon[32362]: pgmap v618: 129 pgs: 129 active+clean; 578 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 1.2 MiB/s wr, 306 op/s 2024-06-10T03:00:21.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:21.968+0000 7f45bae0b700 1 -- 172.21.15.52:0/281020662 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f45b406aa90 msgr2=0x7f45b410ea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:00:21.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:21.968+0000 7f45bae0b700 1 --2- 172.21.15.52:0/281020662 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f45b406aa90 0x7f45b410ea00 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f45a0009990 tx=0x7f45a0009ca0 comp rx=0 tx=0).stop 2024-06-10T03:00:21.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:21.968+0000 7f45bae0b700 1 -- 172.21.15.52:0/281020662 shutdown_connections 2024-06-10T03:00:21.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:21.968+0000 7f45bae0b700 1 --2- 172.21.15.52:0/281020662 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f45b410ef40 0x7f45b4111330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:21.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:21.968+0000 7f45bae0b700 1 --2- 172.21.15.52:0/281020662 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f45b406a170 0x7f45b406a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:21.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:21.968+0000 7f45bae0b700 1 --2- 172.21.15.52:0/281020662 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f45b406aa90 0x7f45b410ea00 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:21.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:21.968+0000 7f45bae0b700 1 -- 172.21.15.52:0/281020662 >> 172.21.15.52:0/281020662 conn(0x7f45b40782c0 msgr2=0x7f45b40786d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:00:21.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:21.968+0000 7f45bae0b700 1 -- 172.21.15.52:0/281020662 shutdown_connections 2024-06-10T03:00:21.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:21.968+0000 7f45bae0b700 1 -- 172.21.15.52:0/281020662 wait complete. 2024-06-10T03:00:21.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:21.968+0000 7f45bae0b700 1 Processor -- start 2024-06-10T03:00:21.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:21.968+0000 7f45bae0b700 1 -- start start 2024-06-10T03:00:21.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:21.969+0000 7f45bae0b700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f45b406a170 0x7f45b419ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:21.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:21.969+0000 7f45bae0b700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f45b406aa90 0x7f45b419b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:21.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:21.969+0000 7f45bae0b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f45b410ef40 0x7f45b419f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:21.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:21.969+0000 7f45bae0b700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f45b4114960 con 0x7f45b410ef40 2024-06-10T03:00:21.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:21.969+0000 7f45bae0b700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f45b41147e0 con 0x7f45b406aa90 2024-06-10T03:00:21.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:21.969+0000 7f45bae0b700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f45b4114ae0 con 0x7f45b406a170 2024-06-10T03:00:21.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:21.969+0000 7f45b93a8700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f45b410ef40 0x7f45b419f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:21.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:21.969+0000 7f45b8ba7700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f45b406a170 0x7f45b419ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:21.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:21.969+0000 7f45b93a8700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f45b410ef40 0x7f45b419f700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49262/0 (socket says 172.21.15.52:49262) 2024-06-10T03:00:21.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:21.969+0000 7f45b93a8700 1 -- 172.21.15.52:0/66791785 learned_addr learned my addr 172.21.15.52:0/66791785 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:00:21.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:21.969+0000 7f45b3fff700 1 --2- 172.21.15.52:0/66791785 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f45b406aa90 0x7f45b419b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:21.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:21.969+0000 7f45b93a8700 1 -- 172.21.15.52:0/66791785 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f45b406a170 msgr2=0x7f45b419ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:00:21.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:21.969+0000 7f45b93a8700 1 --2- 172.21.15.52:0/66791785 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f45b406a170 0x7f45b419ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:21.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:21.969+0000 7f45b93a8700 1 -- 172.21.15.52:0/66791785 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f45b406aa90 msgr2=0x7f45b419b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:00:21.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:21.969+0000 7f45b93a8700 1 --2- 172.21.15.52:0/66791785 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f45b406aa90 0x7f45b419b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:21.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:21.969+0000 7f45b93a8700 1 -- 172.21.15.52:0/66791785 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45b419fe40 con 0x7f45b410ef40 2024-06-10T03:00:21.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:21.969+0000 7f45b3fff700 1 --2- 172.21.15.52:0/66791785 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f45b406aa90 0x7f45b419b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T03:00:21.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:21.969+0000 7f45b93a8700 1 --2- 172.21.15.52:0/66791785 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f45b410ef40 0x7f45b419f700 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f45ac0094f0 tx=0x7f45ac00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:00:21.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:21.969+0000 7f45b1ffb700 1 -- 172.21.15.52:0/66791785 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45ac00ddd0 con 0x7f45b410ef40 2024-06-10T03:00:21.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:21.970+0000 7f45b1ffb700 1 -- 172.21.15.52:0/66791785 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45ac009e20 con 0x7f45b410ef40 2024-06-10T03:00:21.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:21.970+0000 7f45bae0b700 1 -- 172.21.15.52:0/66791785 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45b41a0130 con 0x7f45b410ef40 2024-06-10T03:00:21.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:21.970+0000 7f45b1ffb700 1 -- 172.21.15.52:0/66791785 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45ac015660 con 0x7f45b410ef40 2024-06-10T03:00:21.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:21.970+0000 7f45bae0b700 1 -- 172.21.15.52:0/66791785 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45b41a79b0 con 0x7f45b410ef40 2024-06-10T03:00:21.973 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:21.972+0000 7f45b1ffb700 1 -- 172.21.15.52:0/66791785 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f45ac004ad0 con 0x7f45b410ef40 2024-06-10T03:00:21.973 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:21.973+0000 7f45b1ffb700 1 --2- 172.21.15.52:0/66791785 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f45a40a2ca0 0x7f45a40a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:21.973 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:21.973+0000 7f45b1ffb700 1 -- 172.21.15.52:0/66791785 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f45ac0cede0 con 0x7f45b410ef40 2024-06-10T03:00:21.973 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:21.973+0000 7f45b8ba7700 1 --2- 172.21.15.52:0/66791785 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f45a40a2ca0 0x7f45a40a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:21.973 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:21.973+0000 7f45b8ba7700 1 --2- 172.21.15.52:0/66791785 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f45a40a2ca0 0x7f45a40a5160 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f45a8009c60 tx=0x7f45a8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:00:21.975 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:21.974+0000 7f45c568eb80 1 Processor -- start 2024-06-10T03:00:21.975 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:21.974+0000 7f45c568eb80 1 -- start start 2024-06-10T03:00:21.975 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:21.974+0000 7f45c568eb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559b68231f60 0x559b68232340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:21.975 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:21.974+0000 7f45c568eb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559b68232880 0x559b6822c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:21.975 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:21.974+0000 7f45c568eb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559b6822c7a0 0x559b6822cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:21.975 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:21.974+0000 7f45c568eb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x559b6811ef60 con 0x559b68232880 2024-06-10T03:00:21.975 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:21.974+0000 7f45c568eb80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x559b6817d850 con 0x559b6822c7a0 2024-06-10T03:00:21.975 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:21.974+0000 7f45c568eb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x559b680e5460 con 0x559b68231f60 2024-06-10T03:00:21.975 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:21.975+0000 7f45b8ba7700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559b68231f60 0x559b68232340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:21.975 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:21.975+0000 7f45b3fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559b68232880 0x559b6822c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:21.975 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:21.975+0000 7f45b8ba7700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559b68231f60 0x559b68232340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.52:33972/0 (socket says 172.21.15.52:33972) 2024-06-10T03:00:21.976 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:21.975+0000 7f45b8ba7700 1 -- 172.21.15.52:0/656641108 learned_addr learned my addr 172.21.15.52:0/656641108 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:00:21.976 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:21.975+0000 7f45b93a8700 1 --2- 172.21.15.52:0/656641108 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559b6822c7a0 0x559b6822cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:21.976 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:21.975+0000 7f45b8ba7700 1 -- 172.21.15.52:0/656641108 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559b6822c7a0 msgr2=0x559b6822cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:00:21.976 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:21.975+0000 7f45b8ba7700 1 --2- 172.21.15.52:0/656641108 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559b6822c7a0 0x559b6822cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:21.976 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:21.975+0000 7f45b8ba7700 1 -- 172.21.15.52:0/656641108 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559b68232880 msgr2=0x559b6822c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:00:21.976 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:21.975+0000 7f45b8ba7700 1 --2- 172.21.15.52:0/656641108 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559b68232880 0x559b6822c260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:21.976 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:21.975+0000 7f45b8ba7700 1 -- 172.21.15.52:0/656641108 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b680cc8d0 con 0x559b68231f60 2024-06-10T03:00:21.976 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:21.975+0000 7f45b93a8700 1 --2- 172.21.15.52:0/656641108 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559b6822c7a0 0x559b6822cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T03:00:21.976 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:21.975+0000 7f45b3fff700 1 --2- 172.21.15.52:0/656641108 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559b68232880 0x559b6822c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T03:00:21.976 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:21.975+0000 7f45b8ba7700 1 --2- 172.21.15.52:0/656641108 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559b68231f60 0x559b68232340 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f45a800e4e0 tx=0x7f45a8014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4206309ac0aa667b server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:00:21.979 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:21.975+0000 7f459effd700 1 -- 172.21.15.52:0/656641108 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45a800ee00 con 0x559b68231f60 2024-06-10T03:00:21.979 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:21.975+0000 7f459effd700 1 -- 172.21.15.52:0/656641108 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45a801c8f0 con 0x559b68231f60 2024-06-10T03:00:21.979 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:21.975+0000 7f459effd700 1 -- 172.21.15.52:0/656641108 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45a801dae0 con 0x559b68231f60 2024-06-10T03:00:21.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:21.976+0000 7f45c568eb80 1 -- 172.21.15.52:0/656641108 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559b68231f60 msgr2=0x559b68232340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:00:21.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:21.976+0000 7f45c568eb80 1 --2- 172.21.15.52:0/656641108 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559b68231f60 0x559b68232340 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f45a800e4e0 tx=0x7f45a8014000 comp rx=0 tx=0).stop 2024-06-10T03:00:21.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:21.976+0000 7f45c568eb80 1 -- 172.21.15.52:0/656641108 shutdown_connections 2024-06-10T03:00:21.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:21.976+0000 7f45c568eb80 1 --2- 172.21.15.52:0/656641108 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559b6822c7a0 0x559b6822cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:21.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:21.976+0000 7f45c568eb80 1 --2- 172.21.15.52:0/656641108 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559b68231f60 0x559b68232340 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:21.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:21.976+0000 7f45c568eb80 1 --2- 172.21.15.52:0/656641108 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559b68232880 0x559b6822c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:21.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:21.976+0000 7f45c568eb80 1 -- 172.21.15.52:0/656641108 >> 172.21.15.52:0/656641108 conn(0x559b68159080 msgr2=0x559b68227220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:00:21.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:21.976+0000 7f45c568eb80 1 -- 172.21.15.52:0/656641108 shutdown_connections 2024-06-10T03:00:21.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:21.976+0000 7f45c568eb80 1 -- 172.21.15.52:0/656641108 wait complete. 2024-06-10T03:00:21.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:21.977+0000 7f45c568eb80 1 -- start start 2024-06-10T03:00:21.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:21.977+0000 7f45c568eb80 1 Processor -- start 2024-06-10T03:00:21.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:21.977+0000 7f45c568eb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559b6822c7a0 0x559b68238660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:21.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:21.977+0000 7f45c568eb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559b68231f60 0x559b68235960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:21.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:21.977+0000 7f45c568eb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559b68232880 0x559b68235ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:21.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:21.977+0000 7f45c568eb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x559b6811dfc0 con 0x559b68231f60 2024-06-10T03:00:21.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:21.977+0000 7f45c568eb80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x559b68179810 con 0x559b6822c7a0 2024-06-10T03:00:21.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:21.977+0000 7f45c568eb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x559b68239260 con 0x559b68232880 2024-06-10T03:00:21.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:21.977+0000 7f45c568eb80 10 client.0 _ll_register_callbacks cb 0x559b681135b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T03:00:21.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:21.977+0000 7f45b3fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559b68231f60 0x559b68235960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:21.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:21.977+0000 7f45b3fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559b68231f60 0x559b68235960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49274/0 (socket says 172.21.15.52:49274) 2024-06-10T03:00:21.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:21.977+0000 7f45b3fff700 1 -- 172.21.15.52:0/951789831 learned_addr learned my addr 172.21.15.52:0/951789831 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:00:21.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:21.977+0000 7f45b93a8700 1 --2- 172.21.15.52:0/951789831 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559b68232880 0x559b68235ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:21.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:21.977+0000 7f45b8ba7700 1 --2- 172.21.15.52:0/951789831 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559b6822c7a0 0x559b68238660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:21.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:21.978+0000 7f45b3fff700 1 -- 172.21.15.52:0/951789831 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559b68232880 msgr2=0x559b68235ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:00:21.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:21.978+0000 7f45b3fff700 1 --2- 172.21.15.52:0/951789831 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559b68232880 0x559b68235ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:21.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:21.978+0000 7f45b3fff700 1 -- 172.21.15.52:0/951789831 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559b6822c7a0 msgr2=0x559b68238660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:00:21.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:21.978+0000 7f45b3fff700 1 --2- 172.21.15.52:0/951789831 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559b6822c7a0 0x559b68238660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:21.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:21.978+0000 7f45b3fff700 1 -- 172.21.15.52:0/951789831 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559b68143750 con 0x559b68231f60 2024-06-10T03:00:21.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:21.978+0000 7f45b3fff700 1 --2- 172.21.15.52:0/951789831 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559b68231f60 0x559b68235960 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f45a0009990 tx=0x7f45a000b610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2f5c9a8e140388d server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:00:21.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:21.978+0000 7f4593fff700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-10T03:00:21.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:21.978+0000 7f4593fff700 1 -- 172.21.15.52:0/951789831 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45a0027030 con 0x559b68231f60 2024-06-10T03:00:21.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:21.978+0000 7f4593fff700 1 -- 172.21.15.52:0/951789831 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f45a000f8c0 con 0x559b68231f60 2024-06-10T03:00:21.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:21.978+0000 7f45c568eb80 1 -- 172.21.15.52:0/951789831 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f45a801c8f0 con 0x559b68231f60 2024-06-10T03:00:21.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:21.978+0000 7f4593fff700 1 -- 172.21.15.52:0/951789831 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f45a0031050 con 0x559b68231f60 2024-06-10T03:00:21.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:21.978+0000 7f45c568eb80 10 client.16356 fetch_fsmap learned FSMap version 36 2024-06-10T03:00:21.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:21.978+0000 7f45c568eb80 1 -- 172.21.15.52:0/951789831 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x559b68243a20 con 0x559b68231f60 2024-06-10T03:00:21.983 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:21.979+0000 7f4593fff700 1 -- 172.21.15.52:0/951789831 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45a801c8f0 con 0x559b68231f60 2024-06-10T03:00:21.983 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:21.979+0000 7f4593fff700 1 -- 172.21.15.52:0/951789831 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 36) ==== 8378+0+0 (secure 0 0 0) 0x7f45a0023c00 con 0x559b68231f60 2024-06-10T03:00:21.983 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:21.979+0000 7f45c568eb80 10 client.16356 fetch_fsmap finished waiting for FSMap version 36 2024-06-10T03:00:21.983 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:21.979+0000 7f45c568eb80 10 client.16356 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T03:00:21.983 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:21.979+0000 7f45c568eb80 20 client.16356 populate_metadata read hostname 'smithi052' 2024-06-10T03:00:21.983 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:21.979+0000 7f45c568eb80 1 --2- 172.21.15.52:0/951789831 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x559b68246060 0x559b68248490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:21.983 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:21.979+0000 7f45c568eb80 4 client.16356 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T03:00:21.983 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:21.979+0000 7f45c568eb80 1 -- 172.21.15.52:0/951789831 --> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559b6818ad50 con 0x559b68246060 2024-06-10T03:00:21.983 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:21.979+0000 7f45b8ba7700 1 --2- 172.21.15.52:0/951789831 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x559b68246060 0x559b68248490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:21.983 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:21.980+0000 7f45b8ba7700 1 --2- 172.21.15.52:0/951789831 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x559b68246060 0x559b68248490 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=35aa88a9473af7d8 server_cookie=8e303fa0ca8424ef in_seq=0 out_seq=0 2024-06-10T03:00:21.983 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:21.980+0000 7f4593fff700 10 client.16356 ms_handle_connect on v2:172.21.15.188:6838/310444836 2024-06-10T03:00:21.984 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:21.980+0000 7f4593fff700 1 -- 172.21.15.52:0/951789831 <== mds.0 v2:172.21.15.188:6838/310444836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x559b6818ad50 con 0x559b68246060 2024-06-10T03:00:21.984 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:21.980+0000 7f4593fff700 10 client.16356 handle_command_reply: tid=0 2024-06-10T03:00:21.984 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:21.980+0000 7f45c568eb80 1 client.16356 shutdown 2024-06-10T03:00:21.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:21.980+0000 7f45c568eb80 1 -- 172.21.15.52:0/951789831 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559b68231f60 msgr2=0x559b68235960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:00:21.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:21.980+0000 7f45c568eb80 1 --2- 172.21.15.52:0/951789831 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559b68231f60 0x559b68235960 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f45a0009990 tx=0x7f45a000b610 comp rx=0 tx=0).stop 2024-06-10T03:00:21.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:21.981+0000 7f45c568eb80 1 -- 172.21.15.52:0/951789831 shutdown_connections 2024-06-10T03:00:21.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:21.981+0000 7f45c568eb80 1 --2- 172.21.15.52:0/951789831 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x559b68246060 0x559b68248490 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:21.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:21.981+0000 7f45c568eb80 1 --2- 172.21.15.52:0/951789831 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559b68232880 0x559b68235ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:21.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:21.981+0000 7f45c568eb80 1 --2- 172.21.15.52:0/951789831 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559b6822c7a0 0x559b68238660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:21.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:21.981+0000 7f45c568eb80 1 --2- 172.21.15.52:0/951789831 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559b68231f60 0x559b68235960 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:21.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:21.981+0000 7f45c568eb80 1 -- 172.21.15.52:0/951789831 >> 172.21.15.52:0/951789831 conn(0x559b68159080 msgr2=0x559b68226700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:00:21.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:21.981+0000 7f4593fff700 0 client.16356 ms_handle_reset on v2:172.21.15.188:6838/310444836 2024-06-10T03:00:21.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:21.981+0000 7f45c568eb80 1 -- 172.21.15.52:0/951789831 shutdown_connections 2024-06-10T03:00:21.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:21.981+0000 7f45c568eb80 1 -- 172.21.15.52:0/951789831 wait complete. 2024-06-10T03:00:21.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:21.981+0000 7f45c568eb80 20 client.16356 trim_cache size 0 max 16384 2024-06-10T03:00:22.009 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:22.008+0000 7f45c568eb80 1 Processor -- start 2024-06-10T03:00:22.009 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:22.008+0000 7f45c568eb80 1 -- start start 2024-06-10T03:00:22.010 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:22.009+0000 7f45c568eb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559b6822c7a0 0x559b68233370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:22.010 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:22.009+0000 7f45c568eb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559b68231f60 0x559b682381a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:22.010 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:22.009+0000 7f45c568eb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559b68232880 0x559b68226e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:22.010 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:22.009+0000 7f45c568eb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f45b41b05c0 con 0x559b68231f60 2024-06-10T03:00:22.010 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:22.009+0000 7f45c568eb80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f45b41b21c0 con 0x559b6822c7a0 2024-06-10T03:00:22.010 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:22.009+0000 7f45c568eb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f45b41b0c90 con 0x559b68232880 2024-06-10T03:00:22.010 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:22.009+0000 7f45b3fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559b68231f60 0x559b682381a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:22.010 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:22.009+0000 7f45b8ba7700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559b6822c7a0 0x559b68233370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:22.010 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:22.009+0000 7f45b3fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559b68231f60 0x559b682381a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49286/0 (socket says 172.21.15.52:49286) 2024-06-10T03:00:22.010 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:22.009+0000 7f45b3fff700 1 -- 172.21.15.52:0/477077189 learned_addr learned my addr 172.21.15.52:0/477077189 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:00:22.010 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:22.009+0000 7f45b93a8700 1 --2- 172.21.15.52:0/477077189 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559b68232880 0x559b68226e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:22.010 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:22.009+0000 7f45b3fff700 1 -- 172.21.15.52:0/477077189 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559b68232880 msgr2=0x559b68226e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:00:22.010 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:22.009+0000 7f45b3fff700 1 --2- 172.21.15.52:0/477077189 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559b68232880 0x559b68226e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:22.011 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:22.009+0000 7f45b3fff700 1 -- 172.21.15.52:0/477077189 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559b6822c7a0 msgr2=0x559b68233370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:00:22.011 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:22.009+0000 7f45b3fff700 1 --2- 172.21.15.52:0/477077189 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559b6822c7a0 0x559b68233370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:22.011 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:22.009+0000 7f45b3fff700 1 -- 172.21.15.52:0/477077189 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b6822c5a0 con 0x559b68231f60 2024-06-10T03:00:22.011 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:22.009+0000 7f45b8ba7700 1 --2- 172.21.15.52:0/477077189 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559b6822c7a0 0x559b68233370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T03:00:22.011 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:22.009+0000 7f45b3fff700 1 --2- 172.21.15.52:0/477077189 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559b68231f60 0x559b682381a0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f45a000bdc0 tx=0x7f45a0009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ce465983a03cd77 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:00:22.011 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:22.010+0000 7f459e7fc700 1 -- 172.21.15.52:0/477077189 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45a0027040 con 0x559b68231f60 2024-06-10T03:00:22.011 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:22.010+0000 7f459e7fc700 1 -- 172.21.15.52:0/477077189 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45a002c840 con 0x559b68231f60 2024-06-10T03:00:22.011 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:22.010+0000 7f459e7fc700 1 -- 172.21.15.52:0/477077189 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45a000fe30 con 0x559b68231f60 2024-06-10T03:00:22.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:22.010+0000 7f45c568eb80 1 -- 172.21.15.52:0/477077189 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559b68231f60 msgr2=0x559b682381a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:00:22.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:22.010+0000 7f45c568eb80 1 --2- 172.21.15.52:0/477077189 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559b68231f60 0x559b682381a0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f45a000bdc0 tx=0x7f45a0009ca0 comp rx=0 tx=0).stop 2024-06-10T03:00:22.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:22.010+0000 7f45c568eb80 1 -- 172.21.15.52:0/477077189 shutdown_connections 2024-06-10T03:00:22.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:22.010+0000 7f45c568eb80 1 --2- 172.21.15.52:0/477077189 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559b68232880 0x559b68226e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:22.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:22.010+0000 7f45c568eb80 1 --2- 172.21.15.52:0/477077189 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559b6822c7a0 0x559b68233370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:22.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:22.010+0000 7f45c568eb80 1 --2- 172.21.15.52:0/477077189 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559b68231f60 0x559b682381a0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:22.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:22.010+0000 7f45c568eb80 1 -- 172.21.15.52:0/477077189 >> 172.21.15.52:0/477077189 conn(0x559b68159080 msgr2=0x559b68232c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:00:22.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:22.011+0000 7f45c568eb80 1 -- 172.21.15.52:0/477077189 shutdown_connections 2024-06-10T03:00:22.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:22.011+0000 7f45c568eb80 1 -- 172.21.15.52:0/477077189 wait complete. 2024-06-10T03:00:22.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:22.011+0000 7f45c568eb80 1 -- start start 2024-06-10T03:00:22.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:22.011+0000 7f45c568eb80 1 Processor -- start 2024-06-10T03:00:22.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:22.011+0000 7f45c568eb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559b6822c7a0 0x559b6823ea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:22.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:22.012+0000 7f45c568eb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559b68231f60 0x559b6824d450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:22.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:22.012+0000 7f45c568eb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559b68232880 0x559b6824d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:22.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:22.012+0000 7f45c568eb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f45b41b1af0 con 0x559b6822c7a0 2024-06-10T03:00:22.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:22.012+0000 7f45c568eb80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f45b41b1420 con 0x559b68232880 2024-06-10T03:00:22.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:22.012+0000 7f45c568eb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f45b41ae560 con 0x559b68231f60 2024-06-10T03:00:22.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:22.012+0000 7f45b8ba7700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559b6822c7a0 0x559b6823ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:22.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:22.012+0000 7f45b8ba7700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559b6822c7a0 0x559b6823ea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49288/0 (socket says 172.21.15.52:49288) 2024-06-10T03:00:22.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:22.012+0000 7f45b8ba7700 1 -- 172.21.15.52:0/1277828195 learned_addr learned my addr 172.21.15.52:0/1277828195 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:00:22.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:22.012+0000 7f45b3fff700 1 --2- 172.21.15.52:0/1277828195 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559b68231f60 0x559b6824d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:22.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:22.012+0000 7f45c568eb80 10 client.0 _ll_register_callbacks cb 0x559b681135b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T03:00:22.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:22.012+0000 7f45b93a8700 1 --2- 172.21.15.52:0/1277828195 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559b68232880 0x559b6824d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:22.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:22.012+0000 7f45b8ba7700 1 -- 172.21.15.52:0/1277828195 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559b68231f60 msgr2=0x559b6824d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:00:22.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:22.012+0000 7f45b8ba7700 1 --2- 172.21.15.52:0/1277828195 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559b68231f60 0x559b6824d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:22.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:22.012+0000 7f45b8ba7700 1 -- 172.21.15.52:0/1277828195 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559b68232880 msgr2=0x559b6824d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:00:22.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:22.012+0000 7f45b8ba7700 1 --2- 172.21.15.52:0/1277828195 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559b68232880 0x559b6824d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:22.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:22.012+0000 7f45b8ba7700 1 -- 172.21.15.52:0/1277828195 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559b6824df50 con 0x559b6822c7a0 2024-06-10T03:00:22.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:22.012+0000 7f45b3fff700 1 --2- 172.21.15.52:0/1277828195 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559b68231f60 0x559b6824d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T03:00:22.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:22.012+0000 7f45b8ba7700 1 --2- 172.21.15.52:0/1277828195 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559b6822c7a0 0x559b6823ea70 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f45a800e4b0 tx=0x7f45a8009e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=542222d449b22bd3 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:00:22.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:22.012+0000 7f4593fff700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-10T03:00:22.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:22.012+0000 7f4593fff700 1 -- 172.21.15.52:0/1277828195 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45a800ed60 con 0x559b6822c7a0 2024-06-10T03:00:22.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:22.012+0000 7f4593fff700 1 -- 172.21.15.52:0/1277828195 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f45a80050d0 con 0x559b6822c7a0 2024-06-10T03:00:22.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:22.012+0000 7f45c568eb80 1 -- 172.21.15.52:0/1277828195 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f45a002c840 con 0x559b6822c7a0 2024-06-10T03:00:22.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:22.013+0000 7f4593fff700 1 -- 172.21.15.52:0/1277828195 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f45a8041050 con 0x559b6822c7a0 2024-06-10T03:00:22.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:22.013+0000 7f45c568eb80 10 client.16362 fetch_fsmap learned FSMap version 36 2024-06-10T03:00:22.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:22.013+0000 7f45c568eb80 1 -- 172.21.15.52:0/1277828195 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x559b682b0b10 con 0x559b6822c7a0 2024-06-10T03:00:22.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:22.013+0000 7f4593fff700 1 -- 172.21.15.52:0/1277828195 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45a002c840 con 0x559b6822c7a0 2024-06-10T03:00:22.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:22.013+0000 7f4593fff700 1 -- 172.21.15.52:0/1277828195 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 36) ==== 8378+0+0 (secure 0 0 0) 0x7f45a8003680 con 0x559b6822c7a0 2024-06-10T03:00:22.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:22.013+0000 7f45c568eb80 10 client.16362 fetch_fsmap finished waiting for FSMap version 36 2024-06-10T03:00:22.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:22.013+0000 7f45c568eb80 10 client.16362 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T03:00:22.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:22.013+0000 7f45c568eb80 20 client.16362 populate_metadata read hostname 'smithi052' 2024-06-10T03:00:22.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:22.013+0000 7f45c568eb80 1 --2- 172.21.15.52:0/1277828195 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x559b682b0ee0 0x559b682b32d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:22.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:22.013+0000 7f45c568eb80 4 client.16362 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-10T03:00:22.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:22.013+0000 7f45c568eb80 1 -- 172.21.15.52:0/1277828195 --> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559b681852d0 con 0x559b682b0ee0 2024-06-10T03:00:22.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:22.013+0000 7f45b3fff700 1 --2- 172.21.15.52:0/1277828195 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x559b682b0ee0 0x559b682b32d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:22.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:22.014+0000 7f45b3fff700 1 --2- 172.21.15.52:0/1277828195 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x559b682b0ee0 0x559b682b32d0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aceae163ad3dcfd2 server_cookie=4bb031270a167e82 in_seq=0 out_seq=0 2024-06-10T03:00:22.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:22.014+0000 7f4593fff700 10 client.16362 ms_handle_connect on v2:172.21.15.188:6838/310444836 2024-06-10T03:00:22.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:22.014+0000 7f4593fff700 1 -- 172.21.15.52:0/1277828195 <== mds.0 v2:172.21.15.188:6838/310444836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x559b681852d0 con 0x559b682b0ee0 2024-06-10T03:00:22.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:22.014+0000 7f4593fff700 10 client.16362 handle_command_reply: tid=0 2024-06-10T03:00:22.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:22.014+0000 7f45c568eb80 1 client.16362 shutdown 2024-06-10T03:00:22.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:22.014+0000 7f45c568eb80 1 -- 172.21.15.52:0/1277828195 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559b6822c7a0 msgr2=0x559b6823ea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:00:22.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:22.014+0000 7f45c568eb80 1 --2- 172.21.15.52:0/1277828195 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559b6822c7a0 0x559b6823ea70 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f45a800e4b0 tx=0x7f45a8009e70 comp rx=0 tx=0).stop 2024-06-10T03:00:22.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:22.014+0000 7f45c568eb80 1 -- 172.21.15.52:0/1277828195 shutdown_connections 2024-06-10T03:00:22.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:22.014+0000 7f45c568eb80 1 --2- 172.21.15.52:0/1277828195 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x559b682b0ee0 0x559b682b32d0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:22.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:22.014+0000 7f4593fff700 0 client.16362 ms_handle_reset on v2:172.21.15.188:6838/310444836 2024-06-10T03:00:22.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:22.014+0000 7f45c568eb80 1 --2- 172.21.15.52:0/1277828195 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559b68232880 0x559b6824d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:22.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:22.014+0000 7f45c568eb80 1 --2- 172.21.15.52:0/1277828195 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559b6822c7a0 0x559b6823ea70 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:22.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:22.014+0000 7f45c568eb80 1 --2- 172.21.15.52:0/1277828195 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559b68231f60 0x559b6824d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:22.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:22.014+0000 7f45c568eb80 1 -- 172.21.15.52:0/1277828195 >> 172.21.15.52:0/1277828195 conn(0x559b68159080 msgr2=0x559b68230a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:00:22.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:22.014+0000 7f45c568eb80 1 -- 172.21.15.52:0/1277828195 shutdown_connections 2024-06-10T03:00:22.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:22.014+0000 7f45c568eb80 1 -- 172.21.15.52:0/1277828195 wait complete. 2024-06-10T03:00:22.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:22.015+0000 7f45c568eb80 20 client.16362 trim_cache size 0 max 16384 2024-06-10T03:00:22.020 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-10T03:00:22.020 INFO:teuthology.orchestra.run.smithi052.stdout: "status": "no active scrubs running", 2024-06-10T03:00:22.020 INFO:teuthology.orchestra.run.smithi052.stdout: "scrubs": {} 2024-06-10T03:00:22.020 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-10T03:00:22.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:22.015+0000 7f45bae0b700 1 -- 172.21.15.52:0/66791785 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f45a40a2ca0 msgr2=0x7f45a40a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:00:22.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:22.015+0000 7f45bae0b700 1 --2- 172.21.15.52:0/66791785 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f45a40a2ca0 0x7f45a40a5160 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f45a8009c60 tx=0x7f45a8009260 comp rx=0 tx=0).stop 2024-06-10T03:00:22.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:22.015+0000 7f45bae0b700 1 -- 172.21.15.52:0/66791785 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f45b410ef40 msgr2=0x7f45b419f700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:00:22.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:22.015+0000 7f45bae0b700 1 --2- 172.21.15.52:0/66791785 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f45b410ef40 0x7f45b419f700 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f45ac0094f0 tx=0x7f45ac00d3b0 comp rx=0 tx=0).stop 2024-06-10T03:00:22.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:22.016+0000 7f45bae0b700 1 -- 172.21.15.52:0/66791785 shutdown_connections 2024-06-10T03:00:22.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:22.016+0000 7f45bae0b700 1 --2- 172.21.15.52:0/66791785 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f45a40a2ca0 0x7f45a40a5160 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:22.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:22.016+0000 7f45bae0b700 1 --2- 172.21.15.52:0/66791785 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f45b410ef40 0x7f45b419f700 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:22.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:22.016+0000 7f45bae0b700 1 --2- 172.21.15.52:0/66791785 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f45b406a170 0x7f45b419ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:22.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:22.016+0000 7f45bae0b700 1 --2- 172.21.15.52:0/66791785 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f45b406aa90 0x7f45b419b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:22.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:22.016+0000 7f45bae0b700 1 -- 172.21.15.52:0/66791785 >> 172.21.15.52:0/66791785 conn(0x7f45b40782c0 msgr2=0x7f45b41103b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:00:22.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:22.016+0000 7f45bae0b700 1 -- 172.21.15.52:0/66791785 shutdown_connections 2024-06-10T03:00:22.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:22.016+0000 7f45bae0b700 1 -- 172.21.15.52:0/66791785 wait complete. 2024-06-10T03:00:22.476 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:00:22 smithi052 ceph-mon[32362]: pgmap v619: 129 pgs: 129 active+clean; 581 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.2 MiB/s wr, 220 op/s 2024-06-10T03:00:22.638 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:00:22 smithi120 ceph-mon[36634]: pgmap v619: 129 pgs: 129 active+clean; 581 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.2 MiB/s wr, 220 op/s 2024-06-10T03:00:22.703 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:00:22 smithi188 ceph-mon[36599]: pgmap v619: 129 pgs: 129 active+clean; 581 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.2 MiB/s wr, 220 op/s 2024-06-10T03:00:22.832 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-06-10T03:00:22.833 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T03:00:23.091 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T03:00:24.538 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:00:24 smithi052 ceph-mon[32362]: pgmap v620: 129 pgs: 129 active+clean; 581 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.3 MiB/s wr, 208 op/s 2024-06-10T03:00:24.638 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:00:24 smithi120 ceph-mon[36634]: pgmap v620: 129 pgs: 129 active+clean; 581 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.3 MiB/s wr, 208 op/s 2024-06-10T03:00:24.703 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:00:24 smithi188 ceph-mon[36599]: pgmap v620: 129 pgs: 129 active+clean; 581 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.3 MiB/s wr, 208 op/s 2024-06-10T03:00:24.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:24.876+0000 7fa717b97700 1 -- 172.21.15.52:0/810290795 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa710101a40 msgr2=0x7fa710105800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:00:24.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:24.876+0000 7fa717b97700 1 --2- 172.21.15.52:0/810290795 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa710101a40 0x7fa710105800 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fa700009990 tx=0x7fa700009ca0 comp rx=0 tx=0).stop 2024-06-10T03:00:24.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:24.876+0000 7fa717b97700 1 -- 172.21.15.52:0/810290795 shutdown_connections 2024-06-10T03:00:24.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:24.876+0000 7fa717b97700 1 --2- 172.21.15.52:0/810290795 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa710105f30 0x7fa71010da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:24.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:24.876+0000 7fa717b97700 1 --2- 172.21.15.52:0/810290795 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa710101090 0x7fa710101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:24.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:24.876+0000 7fa717b97700 1 --2- 172.21.15.52:0/810290795 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa710101a40 0x7fa710105800 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:24.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:24.876+0000 7fa717b97700 1 -- 172.21.15.52:0/810290795 >> 172.21.15.52:0/810290795 conn(0x7fa7100766b0 msgr2=0x7fa710076ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:00:24.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:24.876+0000 7fa717b97700 1 -- 172.21.15.52:0/810290795 shutdown_connections 2024-06-10T03:00:24.880 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:24.876+0000 7fa717b97700 1 -- 172.21.15.52:0/810290795 wait complete. 2024-06-10T03:00:24.880 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:24.877+0000 7fa717b97700 1 Processor -- start 2024-06-10T03:00:24.880 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:24.877+0000 7fa717b97700 1 -- start start 2024-06-10T03:00:24.880 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:24.877+0000 7fa717b97700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa710101090 0x7fa7101967f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:24.880 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:24.877+0000 7fa717b97700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa710101a40 0x7fa710196d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:24.880 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:24.877+0000 7fa717b97700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa710105f30 0x7fa710197270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:24.880 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:24.877+0000 7fa717b97700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fa7101105f0 con 0x7fa710101a40 2024-06-10T03:00:24.880 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:24.877+0000 7fa717b97700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa710110470 con 0x7fa710105f30 2024-06-10T03:00:24.880 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:24.877+0000 7fa717b97700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa710110770 con 0x7fa710101090 2024-06-10T03:00:24.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:24.877+0000 7fa715132700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa710101a40 0x7fa710196d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:24.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:24.877+0000 7fa715933700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa710101090 0x7fa7101967f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:24.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:24.877+0000 7fa715132700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa710101a40 0x7fa710196d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49302/0 (socket says 172.21.15.52:49302) 2024-06-10T03:00:24.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:24.877+0000 7fa715933700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa710101090 0x7fa7101967f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.52:34022/0 (socket says 172.21.15.52:34022) 2024-06-10T03:00:24.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:24.877+0000 7fa715132700 1 -- 172.21.15.52:0/1394082197 learned_addr learned my addr 172.21.15.52:0/1394082197 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:00:24.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:24.877+0000 7fa716134700 1 --2- 172.21.15.52:0/1394082197 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa710105f30 0x7fa710197270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:24.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:24.878+0000 7fa715132700 1 -- 172.21.15.52:0/1394082197 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa710101090 msgr2=0x7fa7101967f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:00:24.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:24.878+0000 7fa715132700 1 --2- 172.21.15.52:0/1394082197 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa710101090 0x7fa7101967f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:24.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:24.878+0000 7fa715132700 1 -- 172.21.15.52:0/1394082197 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa710105f30 msgr2=0x7fa710197270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:00:24.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:24.878+0000 7fa715132700 1 --2- 172.21.15.52:0/1394082197 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa710105f30 0x7fa710197270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:24.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:24.878+0000 7fa715132700 1 -- 172.21.15.52:0/1394082197 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7101a7140 con 0x7fa710101a40 2024-06-10T03:00:24.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:24.878+0000 7fa715933700 1 --2- 172.21.15.52:0/1394082197 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa710101090 0x7fa7101967f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T03:00:24.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:24.878+0000 7fa716134700 1 --2- 172.21.15.52:0/1394082197 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa710105f30 0x7fa710197270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T03:00:24.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:24.878+0000 7fa715132700 1 --2- 172.21.15.52:0/1394082197 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa710101a40 0x7fa710196d30 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fa700006010 tx=0x7fa70000bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:00:24.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:24.878+0000 7fa706ffd700 1 -- 172.21.15.52:0/1394082197 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa700027070 con 0x7fa710101a40 2024-06-10T03:00:24.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:24.878+0000 7fa706ffd700 1 -- 172.21.15.52:0/1394082197 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa700003d70 con 0x7fa710101a40 2024-06-10T03:00:24.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:24.878+0000 7fa717b97700 1 -- 172.21.15.52:0/1394082197 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7101a73d0 con 0x7fa710101a40 2024-06-10T03:00:24.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:24.878+0000 7fa706ffd700 1 -- 172.21.15.52:0/1394082197 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa70000f460 con 0x7fa710101a40 2024-06-10T03:00:24.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:24.878+0000 7fa717b97700 1 -- 172.21.15.52:0/1394082197 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7101a7880 con 0x7fa710101a40 2024-06-10T03:00:24.884 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:24.879+0000 7fa717b97700 1 -- 172.21.15.52:0/1394082197 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa710069180 con 0x7fa710101a40 2024-06-10T03:00:24.884 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:24.881+0000 7fa706ffd700 1 -- 172.21.15.52:0/1394082197 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fa70002a070 con 0x7fa710101a40 2024-06-10T03:00:24.884 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:24.882+0000 7fa706ffd700 1 --2- 172.21.15.52:0/1394082197 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fa6fc0a2ca0 0x7fa6fc0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:24.884 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:24.882+0000 7fa715933700 1 --2- 172.21.15.52:0/1394082197 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fa6fc0a2ca0 0x7fa6fc0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:24.884 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:24.882+0000 7fa706ffd700 1 -- 172.21.15.52:0/1394082197 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7fa7000d09e0 con 0x7fa710101a40 2024-06-10T03:00:24.885 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:24.882+0000 7fa715933700 1 --2- 172.21.15.52:0/1394082197 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fa6fc0a2ca0 0x7fa6fc0a5160 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fa70c005950 tx=0x7fa70c00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:00:24.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:24.886+0000 7fa706ffd700 1 -- 172.21.15.52:0/1394082197 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fa700031070 con 0x7fa710101a40 2024-06-10T03:00:25.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:25.167+0000 7fa717b97700 1 -- 172.21.15.52:0/1394082197 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa71002de60 con 0x7fa710101a40 2024-06-10T03:00:25.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:25.168+0000 7fa706ffd700 1 -- 172.21.15.52:0/1394082197 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+11945 (secure 0 0 0) 0x7fa700013070 con 0x7fa710101a40 2024-06-10T03:00:25.168 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-10T03:00:25.169 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":36,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/889673247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":889673247},{"type":"v1","addr":"172.21.15.52:6835","nonce":889673247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/3615663511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":3615663511},{"type":"v1","addr":"172.21.15.52:6837","nonce":3615663511}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":36,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T02:50:03.889766+0000","modified":"2024-06-10T02:59:27.720611+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24484,"mds_1":24469,"mds_2":24463,"mds_3":24449,"mds_4":24428},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14571":{"gid":14571,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6839/3059599042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":3059599042},{"type":"v1","addr":"172.21.15.52:6839","nonce":3059599042}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14583":{"gid":14583,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6841/3039275521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":3039275521},{"type":"v1","addr":"172.21.15.52:6841","nonce":3039275521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24383":{"gid":24383,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6833/2609002896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":2609002896},{"type":"v1","addr":"172.21.15.120:6833","nonce":2609002896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6835/1996370288","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":1996370288},{"type":"v1","addr":"172.21.15.120:6835","nonce":1996370288}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6833/529564979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":529564979},{"type":"v1","addr":"172.21.15.188:6833","nonce":529564979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":25,"addr":"172.21.15.120:6837/4132440952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":4132440952},{"type":"v1","addr":"172.21.15.120:6837","nonce":4132440952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"b","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.120:6839/1672436487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1672436487},{"type":"v1","addr":"172.21.15.120:6839","nonce":1672436487}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":2,"incarnation":22,"state":"up:active","state_seq":26,"addr":"172.21.15.188:6835/3904544259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":3904544259},{"type":"v1","addr":"172.21.15.188:6835","nonce":3904544259}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.188:6837/819317726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":819317726},{"type":"v1","addr":"172.21.15.188:6837","nonce":819317726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.188:6839/310444836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":310444836},{"type":"v1","addr":"172.21.15.188:6839","nonce":310444836}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24484,"qdb_cluster":[24469,24484,24463,24428,24449]},"id":1}]} 2024-06-10T03:00:25.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:25.170+0000 7fa717b97700 1 -- 172.21.15.52:0/1394082197 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fa6fc0a2ca0 msgr2=0x7fa6fc0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:00:25.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:25.170+0000 7fa717b97700 1 --2- 172.21.15.52:0/1394082197 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fa6fc0a2ca0 0x7fa6fc0a5160 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fa70c005950 tx=0x7fa70c00b410 comp rx=0 tx=0).stop 2024-06-10T03:00:25.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:25.170+0000 7fa717b97700 1 -- 172.21.15.52:0/1394082197 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa710101a40 msgr2=0x7fa710196d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:00:25.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:25.170+0000 7fa717b97700 1 --2- 172.21.15.52:0/1394082197 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa710101a40 0x7fa710196d30 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fa700006010 tx=0x7fa70000bab0 comp rx=0 tx=0).stop 2024-06-10T03:00:25.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:25.171+0000 7fa717b97700 1 -- 172.21.15.52:0/1394082197 shutdown_connections 2024-06-10T03:00:25.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:25.171+0000 7fa717b97700 1 --2- 172.21.15.52:0/1394082197 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa710105f30 0x7fa710197270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:25.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:25.171+0000 7fa717b97700 1 --2- 172.21.15.52:0/1394082197 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa710101090 0x7fa7101967f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:25.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:25.171+0000 7fa717b97700 1 --2- 172.21.15.52:0/1394082197 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fa6fc0a2ca0 0x7fa6fc0a5160 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:25.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:25.171+0000 7fa717b97700 1 --2- 172.21.15.52:0/1394082197 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa710101a40 0x7fa710196d30 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:25.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:25.171+0000 7fa717b97700 1 -- 172.21.15.52:0/1394082197 >> 172.21.15.52:0/1394082197 conn(0x7fa7100766b0 msgr2=0x7fa7100feff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:00:25.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:25.171+0000 7fa717b97700 1 -- 172.21.15.52:0/1394082197 shutdown_connections 2024-06-10T03:00:25.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:25.171+0000 7fa717b97700 1 -- 172.21.15.52:0/1394082197 wait complete. 2024-06-10T03:00:25.172 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 36 2024-06-10T03:00:25.477 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:00:25 smithi052 ceph-mon[32362]: from='client.? 172.21.15.52:0/1394082197' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T03:00:25.638 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:00:25 smithi120 ceph-mon[36634]: from='client.? 172.21.15.52:0/1394082197' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T03:00:25.703 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:00:25 smithi188 ceph-mon[36599]: from='client.? 172.21.15.52:0/1394082197' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T03:00:25.933 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-06-10T03:00:26.211 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T03:00:26.514 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:00:26 smithi052 ceph-mon[32362]: pgmap v621: 129 pgs: 129 active+clean; 577 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 150 op/s 2024-06-10T03:00:26.539 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:00:26 smithi120 ceph-mon[36634]: pgmap v621: 129 pgs: 129 active+clean; 577 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 150 op/s 2024-06-10T03:00:26.703 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:00:26 smithi188 ceph-mon[36599]: pgmap v621: 129 pgs: 129 active+clean; 577 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 150 op/s 2024-06-10T03:00:27.249 INFO:tasks.workunit.client.0.smithi052.stdout: 98 80623 70 57361 84 43046 189 2024-06-10T03:00:28.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.198+0000 7f79f15d9700 1 -- 172.21.15.52:0/3298420636 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f79ec06ab30 msgr2=0x7f79ec10f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:00:28.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.198+0000 7f79f15d9700 1 --2- 172.21.15.52:0/3298420636 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f79ec06ab30 0x7f79ec10f320 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f79e400b0a0 tx=0x7f79e400b3b0 comp rx=0 tx=0).stop 2024-06-10T03:00:28.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.198+0000 7f79f15d9700 1 -- 172.21.15.52:0/3298420636 shutdown_connections 2024-06-10T03:00:28.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.198+0000 7f79f15d9700 1 --2- 172.21.15.52:0/3298420636 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f79ec06ab30 0x7f79ec10f320 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:28.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.198+0000 7f79f15d9700 1 --2- 172.21.15.52:0/3298420636 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f79ec108500 0x7f79ec1088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:28.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.198+0000 7f79f15d9700 1 --2- 172.21.15.52:0/3298420636 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f79ec06a170 0x7f79ec06a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:28.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.198+0000 7f79f15d9700 1 -- 172.21.15.52:0/3298420636 >> 172.21.15.52:0/3298420636 conn(0x7f79ec078340 msgr2=0x7f79ec078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:00:28.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.198+0000 7f79f15d9700 1 -- 172.21.15.52:0/3298420636 shutdown_connections 2024-06-10T03:00:28.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.198+0000 7f79f15d9700 1 -- 172.21.15.52:0/3298420636 wait complete. 2024-06-10T03:00:28.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.199+0000 7f79f15d9700 1 Processor -- start 2024-06-10T03:00:28.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.199+0000 7f79f15d9700 1 -- start start 2024-06-10T03:00:28.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.199+0000 7f79f15d9700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f79ec06a170 0x7f79ec101460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:28.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.199+0000 7f79f15d9700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f79ec06ab30 0x7f79ec1019a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:28.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.199+0000 7f79f15d9700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f79ec108500 0x7f79ec101ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:28.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.199+0000 7f79f15d9700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f79ec112760 con 0x7f79ec108500 2024-06-10T03:00:28.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.199+0000 7f79f15d9700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f79ec1125e0 con 0x7f79ec06a170 2024-06-10T03:00:28.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.199+0000 7f79f15d9700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f79ec1128e0 con 0x7f79ec06ab30 2024-06-10T03:00:28.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.200+0000 7f79eaffd700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f79ec06a170 0x7f79ec101460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:28.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.200+0000 7f79e27fc700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f79ec06ab30 0x7f79ec1019a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:28.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.200+0000 7f79eaffd700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f79ec06a170 0x7f79ec101460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.52:49982/0 (socket says 172.21.15.52:49982) 2024-06-10T03:00:28.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.200+0000 7f79eaffd700 1 -- 172.21.15.52:0/2586795643 learned_addr learned my addr 172.21.15.52:0/2586795643 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:00:28.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.200+0000 7f79eb7fe700 1 --2- 172.21.15.52:0/2586795643 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f79ec108500 0x7f79ec101ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:28.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.200+0000 7f79eaffd700 1 -- 172.21.15.52:0/2586795643 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f79ec06ab30 msgr2=0x7f79ec1019a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:00:28.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.200+0000 7f79eaffd700 1 --2- 172.21.15.52:0/2586795643 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f79ec06ab30 0x7f79ec1019a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:28.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.200+0000 7f79eaffd700 1 -- 172.21.15.52:0/2586795643 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f79ec108500 msgr2=0x7f79ec101ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:00:28.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.200+0000 7f79eaffd700 1 --2- 172.21.15.52:0/2586795643 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f79ec108500 0x7f79ec101ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:28.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.200+0000 7f79eaffd700 1 -- 172.21.15.52:0/2586795643 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79ec1a7130 con 0x7f79ec06a170 2024-06-10T03:00:28.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.200+0000 7f79eaffd700 1 --2- 172.21.15.52:0/2586795643 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f79ec06a170 0x7f79ec101460 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f79dc00d980 tx=0x7f79dc00dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:00:28.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.200+0000 7f79e8ff9700 1 -- 172.21.15.52:0/2586795643 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79dc016070 con 0x7f79ec06a170 2024-06-10T03:00:28.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.200+0000 7f79e8ff9700 1 -- 172.21.15.52:0/2586795643 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f79dc004d10 con 0x7f79ec06a170 2024-06-10T03:00:28.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.200+0000 7f79f15d9700 1 -- 172.21.15.52:0/2586795643 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79ec1a7420 con 0x7f79ec06a170 2024-06-10T03:00:28.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.200+0000 7f79e8ff9700 1 -- 172.21.15.52:0/2586795643 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79dc005670 con 0x7f79ec06a170 2024-06-10T03:00:28.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.200+0000 7f79f15d9700 1 -- 172.21.15.52:0/2586795643 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79ec1a79b0 con 0x7f79ec06a170 2024-06-10T03:00:28.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.203+0000 7f79e8ff9700 1 -- 172.21.15.52:0/2586795643 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f79dc004750 con 0x7f79ec06a170 2024-06-10T03:00:28.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.203+0000 7f79e8ff9700 1 --2- 172.21.15.52:0/2586795643 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f79d80a2ca0 0x7f79d80a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:28.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.204+0000 7f79e8ff9700 1 -- 172.21.15.52:0/2586795643 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f79dc0cf0a0 con 0x7f79ec06a170 2024-06-10T03:00:28.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.204+0000 7f79e27fc700 1 --2- 172.21.15.52:0/2586795643 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f79d80a2ca0 0x7f79d80a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:28.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.204+0000 7f79fbe5cb80 1 Processor -- start 2024-06-10T03:00:28.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.204+0000 7f79e27fc700 1 --2- 172.21.15.52:0/2586795643 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f79d80a2ca0 0x7f79d80a5160 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f79ec078840 tx=0x7f79d400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:00:28.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.204+0000 7f79fbe5cb80 1 -- start start 2024-06-10T03:00:28.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.205+0000 7f79fbe5cb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558bd99b9f60 0x558bd99ba340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:28.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.205+0000 7f79fbe5cb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558bd99ba880 0x558bd99b4260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:28.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.205+0000 7f79fbe5cb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558bd99b47a0 0x558bd99b4bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:28.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.205+0000 7f79fbe5cb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x558bd98a6f60 con 0x558bd99b9f60 2024-06-10T03:00:28.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.205+0000 7f79fbe5cb80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x558bd9905850 con 0x558bd99ba880 2024-06-10T03:00:28.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.205+0000 7f79fbe5cb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x558bd986d460 con 0x558bd99b47a0 2024-06-10T03:00:28.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.205+0000 7f79eaffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558bd99b9f60 0x558bd99ba340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:28.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.205+0000 7f79eaffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558bd99b9f60 0x558bd99ba340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49344/0 (socket says 172.21.15.52:49344) 2024-06-10T03:00:28.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.205+0000 7f79eaffd700 1 -- 172.21.15.52:0/952899896 learned_addr learned my addr 172.21.15.52:0/952899896 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:00:28.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.205+0000 7f79e27fc700 1 --2- 172.21.15.52:0/952899896 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558bd99ba880 0x558bd99b4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:28.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.205+0000 7f79eb7fe700 1 --2- 172.21.15.52:0/952899896 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558bd99b47a0 0x558bd99b4bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:28.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.205+0000 7f79eaffd700 1 -- 172.21.15.52:0/952899896 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558bd99b47a0 msgr2=0x558bd99b4bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:00:28.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.205+0000 7f79eaffd700 1 --2- 172.21.15.52:0/952899896 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558bd99b47a0 0x558bd99b4bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:28.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.205+0000 7f79eaffd700 1 -- 172.21.15.52:0/952899896 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558bd99ba880 msgr2=0x558bd99b4260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:00:28.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.205+0000 7f79eaffd700 1 --2- 172.21.15.52:0/952899896 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558bd99ba880 0x558bd99b4260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:28.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.205+0000 7f79eaffd700 1 -- 172.21.15.52:0/952899896 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558bd98548d0 con 0x558bd99b9f60 2024-06-10T03:00:28.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.205+0000 7f79e27fc700 1 --2- 172.21.15.52:0/952899896 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558bd99ba880 0x558bd99b4260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T03:00:28.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.205+0000 7f79eaffd700 1 --2- 172.21.15.52:0/952899896 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558bd99b9f60 0x558bd99ba340 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f79dc0979b0 tx=0x7f79dc09aeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b72e87c7c2fa9c4 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:00:28.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.205+0000 7f79e17fa700 1 -- 172.21.15.52:0/952899896 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79dc09bc50 con 0x558bd99b9f60 2024-06-10T03:00:28.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.206+0000 7f79e17fa700 1 -- 172.21.15.52:0/952899896 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f79dc09b550 con 0x558bd99b9f60 2024-06-10T03:00:28.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.206+0000 7f79e17fa700 1 -- 172.21.15.52:0/952899896 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79dc004340 con 0x558bd99b9f60 2024-06-10T03:00:28.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.206+0000 7f79fbe5cb80 1 -- 172.21.15.52:0/952899896 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558bd99b9f60 msgr2=0x558bd99ba340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:00:28.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.206+0000 7f79fbe5cb80 1 --2- 172.21.15.52:0/952899896 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558bd99b9f60 0x558bd99ba340 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f79dc0979b0 tx=0x7f79dc09aeb0 comp rx=0 tx=0).stop 2024-06-10T03:00:28.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.206+0000 7f79fbe5cb80 1 -- 172.21.15.52:0/952899896 shutdown_connections 2024-06-10T03:00:28.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.206+0000 7f79fbe5cb80 1 --2- 172.21.15.52:0/952899896 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558bd99b47a0 0x558bd99b4bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:28.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.206+0000 7f79fbe5cb80 1 --2- 172.21.15.52:0/952899896 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558bd99b9f60 0x558bd99ba340 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:28.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.206+0000 7f79fbe5cb80 1 --2- 172.21.15.52:0/952899896 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558bd99ba880 0x558bd99b4260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:28.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.206+0000 7f79fbe5cb80 1 -- 172.21.15.52:0/952899896 >> 172.21.15.52:0/952899896 conn(0x558bd98e1080 msgr2=0x558bd99af220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:00:28.212 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.206+0000 7f79fbe5cb80 1 -- 172.21.15.52:0/952899896 shutdown_connections 2024-06-10T03:00:28.212 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.206+0000 7f79fbe5cb80 1 -- 172.21.15.52:0/952899896 wait complete. 2024-06-10T03:00:28.212 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.207+0000 7f79fbe5cb80 1 -- start start 2024-06-10T03:00:28.212 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.207+0000 7f79fbe5cb80 1 Processor -- start 2024-06-10T03:00:28.212 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.207+0000 7f79fbe5cb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558bd99b47a0 0x558bd99c0610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:28.212 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.207+0000 7f79fbe5cb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558bd99b9f60 0x558bd99bd910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:28.212 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.207+0000 7f79fbe5cb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558bd99ba880 0x558bd99bde50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:28.212 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.207+0000 7f79fbe5cb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x558bd98a5fc0 con 0x558bd99ba880 2024-06-10T03:00:28.212 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.207+0000 7f79fbe5cb80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x558bd9901810 con 0x558bd99b9f60 2024-06-10T03:00:28.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.207+0000 7f79fbe5cb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x558bd99c1200 con 0x558bd99b47a0 2024-06-10T03:00:28.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.207+0000 7f79fbe5cb80 10 client.0 _ll_register_callbacks cb 0x558bd989b5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T03:00:28.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.207+0000 7f79eb7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558bd99ba880 0x558bd99bde50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:28.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.207+0000 7f79eaffd700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558bd99b47a0 0x558bd99c0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:28.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.207+0000 7f79eaffd700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558bd99b47a0 0x558bd99c0610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.52:34062/0 (socket says 172.21.15.52:34062) 2024-06-10T03:00:28.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.207+0000 7f79eb7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558bd99ba880 0x558bd99bde50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49354/0 (socket says 172.21.15.52:49354) 2024-06-10T03:00:28.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.207+0000 7f79eaffd700 1 -- 172.21.15.52:0/1019501111 learned_addr learned my addr 172.21.15.52:0/1019501111 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:00:28.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.208+0000 7f79e27fc700 1 --2- 172.21.15.52:0/1019501111 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558bd99b9f60 0x558bd99bd910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:28.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.208+0000 7f79eaffd700 1 -- 172.21.15.52:0/1019501111 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558bd99b9f60 msgr2=0x558bd99bd910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:00:28.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.208+0000 7f79eaffd700 1 --2- 172.21.15.52:0/1019501111 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558bd99b9f60 0x558bd99bd910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:28.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.208+0000 7f79eaffd700 1 -- 172.21.15.52:0/1019501111 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558bd99ba880 msgr2=0x558bd99bde50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:00:28.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.208+0000 7f79eaffd700 1 --2- 172.21.15.52:0/1019501111 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558bd99ba880 0x558bd99bde50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:28.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.208+0000 7f79eaffd700 1 -- 172.21.15.52:0/1019501111 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558bd98cb750 con 0x558bd99b47a0 2024-06-10T03:00:28.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.208+0000 7f79eb7fe700 1 --2- 172.21.15.52:0/1019501111 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558bd99ba880 0x558bd99bde50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T03:00:28.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.208+0000 7f79eaffd700 1 --2- 172.21.15.52:0/1019501111 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558bd99b47a0 0x558bd99c0610 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f79dc00d950 tx=0x7f79dc004290 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8fce2a6d1150698f server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:00:28.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.208+0000 7f79d27fc700 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-06-10T03:00:28.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.208+0000 7f79d27fc700 1 -- 172.21.15.52:0/1019501111 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79dc0b8070 con 0x558bd99b47a0 2024-06-10T03:00:28.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.208+0000 7f79d27fc700 1 -- 172.21.15.52:0/1019501111 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f79dc099850 con 0x558bd99b47a0 2024-06-10T03:00:28.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.208+0000 7f79fbe5cb80 1 -- 172.21.15.52:0/1019501111 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f79dc09b550 con 0x558bd99b47a0 2024-06-10T03:00:28.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.208+0000 7f79d27fc700 1 -- 172.21.15.52:0/1019501111 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f79dc0a41c0 con 0x558bd99b47a0 2024-06-10T03:00:28.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.208+0000 7f79fbe5cb80 10 client.25945 fetch_fsmap learned FSMap version 36 2024-06-10T03:00:28.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.208+0000 7f79fbe5cb80 1 -- 172.21.15.52:0/1019501111 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x558bd99cb9b0 con 0x558bd99b47a0 2024-06-10T03:00:28.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.209+0000 7f79d27fc700 1 -- 172.21.15.52:0/1019501111 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f79dc09b550 con 0x558bd99b47a0 2024-06-10T03:00:28.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.209+0000 7f79d27fc700 1 -- 172.21.15.52:0/1019501111 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 36) ==== 8378+0+0 (secure 0 0 0) 0x7f79dc0b4280 con 0x558bd99b47a0 2024-06-10T03:00:28.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.209+0000 7f79fbe5cb80 10 client.25945 fetch_fsmap finished waiting for FSMap version 36 2024-06-10T03:00:28.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.209+0000 7f79fbe5cb80 10 client.25945 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-06-10T03:00:28.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.209+0000 7f79fbe5cb80 20 client.25945 populate_metadata read hostname 'smithi052' 2024-06-10T03:00:28.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.209+0000 7f79fbe5cb80 1 --2- 172.21.15.52:0/1019501111 >> [v2:172.21.15.120:6836/4132440952,v1:172.21.15.120:6837/4132440952] conn(0x558bd99cde80 0x558bd99d0270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:28.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.209+0000 7f79fbe5cb80 4 client.25945 mds_command: new command op to 24428 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T03:00:28.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.209+0000 7f79fbe5cb80 1 -- 172.21.15.52:0/1019501111 --> [v2:172.21.15.120:6836/4132440952,v1:172.21.15.120:6837/4132440952] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558bd9912d50 con 0x558bd99cde80 2024-06-10T03:00:28.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.209+0000 7f79e27fc700 1 --2- 172.21.15.52:0/1019501111 >> [v2:172.21.15.120:6836/4132440952,v1:172.21.15.120:6837/4132440952] conn(0x558bd99cde80 0x558bd99d0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:28.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.210+0000 7f79e27fc700 1 --2- 172.21.15.52:0/1019501111 >> [v2:172.21.15.120:6836/4132440952,v1:172.21.15.120:6837/4132440952] conn(0x558bd99cde80 0x558bd99d0270 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=949a8a090950e593 server_cookie=9bb208221ba26cb9 in_seq=0 out_seq=0 2024-06-10T03:00:28.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.210+0000 7f79d27fc700 10 client.25945 ms_handle_connect on v2:172.21.15.120:6836/4132440952 2024-06-10T03:00:28.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.210+0000 7f79d27fc700 1 -- 172.21.15.52:0/1019501111 <== mds.4 v2:172.21.15.120:6836/4132440952 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x558bd9912d50 con 0x558bd99cde80 2024-06-10T03:00:28.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.210+0000 7f79d27fc700 10 client.25945 handle_command_reply: tid=0 2024-06-10T03:00:28.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.210+0000 7f79fbe5cb80 1 client.25945 shutdown 2024-06-10T03:00:28.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.210+0000 7f79fbe5cb80 1 -- 172.21.15.52:0/1019501111 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558bd99b47a0 msgr2=0x558bd99c0610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:00:28.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.210+0000 7f79fbe5cb80 1 --2- 172.21.15.52:0/1019501111 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558bd99b47a0 0x558bd99c0610 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f79dc00d950 tx=0x7f79dc004290 comp rx=0 tx=0).stop 2024-06-10T03:00:28.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.210+0000 7f79fbe5cb80 1 -- 172.21.15.52:0/1019501111 shutdown_connections 2024-06-10T03:00:28.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.210+0000 7f79fbe5cb80 1 --2- 172.21.15.52:0/1019501111 >> [v2:172.21.15.120:6836/4132440952,v1:172.21.15.120:6837/4132440952] conn(0x558bd99cde80 0x558bd99d0270 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:28.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.210+0000 7f79fbe5cb80 1 --2- 172.21.15.52:0/1019501111 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558bd99ba880 0x558bd99bde50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:28.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.210+0000 7f79fbe5cb80 1 --2- 172.21.15.52:0/1019501111 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558bd99b47a0 0x558bd99c0610 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:28.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.210+0000 7f79fbe5cb80 1 --2- 172.21.15.52:0/1019501111 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558bd99b9f60 0x558bd99bd910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:28.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.210+0000 7f79fbe5cb80 1 -- 172.21.15.52:0/1019501111 >> 172.21.15.52:0/1019501111 conn(0x558bd98e1080 msgr2=0x558bd99ae700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:00:28.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.211+0000 7f79d27fc700 0 client.25945 ms_handle_reset on v2:172.21.15.120:6836/4132440952 2024-06-10T03:00:28.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.211+0000 7f79fbe5cb80 1 -- 172.21.15.52:0/1019501111 shutdown_connections 2024-06-10T03:00:28.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.211+0000 7f79fbe5cb80 1 -- 172.21.15.52:0/1019501111 wait complete. 2024-06-10T03:00:28.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.211+0000 7f79fbe5cb80 20 client.25945 trim_cache size 0 max 16384 2024-06-10T03:00:28.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.226+0000 7f79fbe5cb80 1 Processor -- start 2024-06-10T03:00:28.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.227+0000 7f79fbe5cb80 1 -- start start 2024-06-10T03:00:28.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.227+0000 7f79fbe5cb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558bd99b47a0 0x558bd99b4260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:28.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.227+0000 7f79fbe5cb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558bd99b9f60 0x558bd99c0150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:28.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.227+0000 7f79fbe5cb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558bd99ba880 0x558bd99b9150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:28.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.227+0000 7f79fbe5cb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f79ec1b05a0 con 0x558bd99ba880 2024-06-10T03:00:28.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.227+0000 7f79fbe5cb80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f79ec1b21a0 con 0x558bd99b47a0 2024-06-10T03:00:28.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.227+0000 7f79fbe5cb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f79ec1b0c70 con 0x558bd99b9f60 2024-06-10T03:00:28.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.227+0000 7f79eb7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558bd99ba880 0x558bd99b9150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:28.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.227+0000 7f79eb7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558bd99ba880 0x558bd99b9150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49356/0 (socket says 172.21.15.52:49356) 2024-06-10T03:00:28.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.227+0000 7f79eb7fe700 1 -- 172.21.15.52:0/1979373055 learned_addr learned my addr 172.21.15.52:0/1979373055 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:00:28.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.227+0000 7f79eaffd700 1 --2- 172.21.15.52:0/1979373055 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558bd99b47a0 0x558bd99b4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:28.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.227+0000 7f79e27fc700 1 --2- 172.21.15.52:0/1979373055 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558bd99b9f60 0x558bd99c0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:28.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.227+0000 7f79eb7fe700 1 -- 172.21.15.52:0/1979373055 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558bd99b9f60 msgr2=0x558bd99c0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:00:28.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.227+0000 7f79eb7fe700 1 --2- 172.21.15.52:0/1979373055 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558bd99b9f60 0x558bd99c0150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:28.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.227+0000 7f79eb7fe700 1 -- 172.21.15.52:0/1979373055 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558bd99b47a0 msgr2=0x558bd99b4260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:00:28.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.227+0000 7f79eb7fe700 1 --2- 172.21.15.52:0/1979373055 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558bd99b47a0 0x558bd99b4260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:28.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.227+0000 7f79eb7fe700 1 -- 172.21.15.52:0/1979373055 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558bd99be850 con 0x558bd99ba880 2024-06-10T03:00:28.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.228+0000 7f79eb7fe700 1 --2- 172.21.15.52:0/1979373055 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558bd99ba880 0x558bd99b9150 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f79e4015000 tx=0x7f79e4003ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba1a69e24fd44f0f server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:00:28.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.228+0000 7f79e0ff9700 1 -- 172.21.15.52:0/1979373055 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79e4003d10 con 0x558bd99ba880 2024-06-10T03:00:28.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.228+0000 7f79e0ff9700 1 -- 172.21.15.52:0/1979373055 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f79e40079c0 con 0x558bd99ba880 2024-06-10T03:00:28.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.228+0000 7f79e0ff9700 1 -- 172.21.15.52:0/1979373055 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79e4005500 con 0x558bd99ba880 2024-06-10T03:00:28.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.228+0000 7f79fbe5cb80 1 -- 172.21.15.52:0/1979373055 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558bd99ba880 msgr2=0x558bd99b9150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:00:28.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.228+0000 7f79fbe5cb80 1 --2- 172.21.15.52:0/1979373055 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558bd99ba880 0x558bd99b9150 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f79e4015000 tx=0x7f79e4003ab0 comp rx=0 tx=0).stop 2024-06-10T03:00:28.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.229+0000 7f79fbe5cb80 1 -- 172.21.15.52:0/1979373055 shutdown_connections 2024-06-10T03:00:28.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.229+0000 7f79fbe5cb80 1 --2- 172.21.15.52:0/1979373055 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558bd99ba880 0x558bd99b9150 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:28.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.229+0000 7f79fbe5cb80 1 --2- 172.21.15.52:0/1979373055 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558bd99b47a0 0x558bd99b4260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:28.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.229+0000 7f79fbe5cb80 1 --2- 172.21.15.52:0/1979373055 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558bd99b9f60 0x558bd99c0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:28.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.229+0000 7f79fbe5cb80 1 -- 172.21.15.52:0/1979373055 >> 172.21.15.52:0/1979373055 conn(0x558bd98e1080 msgr2=0x558bd99b8a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:00:28.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.229+0000 7f79fbe5cb80 1 -- 172.21.15.52:0/1979373055 shutdown_connections 2024-06-10T03:00:28.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.229+0000 7f79fbe5cb80 1 -- 172.21.15.52:0/1979373055 wait complete. 2024-06-10T03:00:28.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.229+0000 7f79fbe5cb80 1 -- start start 2024-06-10T03:00:28.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.229+0000 7f79fbe5cb80 1 Processor -- start 2024-06-10T03:00:28.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.229+0000 7f79fbe5cb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558bd99b47a0 0x558bd99c6490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:28.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.229+0000 7f79fbe5cb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558bd99b9f60 0x558bd99c69d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:28.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.229+0000 7f79fbe5cb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558bd99ba880 0x558bd99d49c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:28.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.230+0000 7f79fbe5cb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f79ec1b1ad0 con 0x558bd99ba880 2024-06-10T03:00:28.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.230+0000 7f79fbe5cb80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f79ec1b1400 con 0x558bd99b47a0 2024-06-10T03:00:28.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.230+0000 7f79fbe5cb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f79ec1ae540 con 0x558bd99b9f60 2024-06-10T03:00:28.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.230+0000 7f79eb7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558bd99ba880 0x558bd99d49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:28.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.230+0000 7f79eb7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558bd99ba880 0x558bd99d49c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49368/0 (socket says 172.21.15.52:49368) 2024-06-10T03:00:28.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.230+0000 7f79eb7fe700 1 -- 172.21.15.52:0/597796780 learned_addr learned my addr 172.21.15.52:0/597796780 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:00:28.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.230+0000 7f79fbe5cb80 10 client.0 _ll_register_callbacks cb 0x558bd989b5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T03:00:28.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.230+0000 7f79e27fc700 1 --2- 172.21.15.52:0/597796780 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558bd99b9f60 0x558bd99c69d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:28.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.230+0000 7f79eaffd700 1 --2- 172.21.15.52:0/597796780 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558bd99b47a0 0x558bd99c6490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:28.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.230+0000 7f79eb7fe700 1 -- 172.21.15.52:0/597796780 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558bd99b9f60 msgr2=0x558bd99c69d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:00:28.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.230+0000 7f79eb7fe700 1 --2- 172.21.15.52:0/597796780 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558bd99b9f60 0x558bd99c69d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:28.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.230+0000 7f79eb7fe700 1 -- 172.21.15.52:0/597796780 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558bd99b47a0 msgr2=0x558bd99c6490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:00:28.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.230+0000 7f79eb7fe700 1 --2- 172.21.15.52:0/597796780 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558bd99b47a0 0x558bd99c6490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:28.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.230+0000 7f79eb7fe700 1 -- 172.21.15.52:0/597796780 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558bd99c7120 con 0x558bd99ba880 2024-06-10T03:00:28.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.230+0000 7f79eb7fe700 1 --2- 172.21.15.52:0/597796780 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558bd99ba880 0x558bd99d49c0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f79e400d010 tx=0x7f79e4006130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b196c641fd0bd86e server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:00:28.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.230+0000 7f79d27fc700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-10T03:00:28.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.230+0000 7f79d27fc700 1 -- 172.21.15.52:0/597796780 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79e4005180 con 0x558bd99ba880 2024-06-10T03:00:28.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.230+0000 7f79d27fc700 1 -- 172.21.15.52:0/597796780 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f79e4005320 con 0x558bd99ba880 2024-06-10T03:00:28.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.230+0000 7f79fbe5cb80 1 -- 172.21.15.52:0/597796780 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f79e40079c0 con 0x558bd99ba880 2024-06-10T03:00:28.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.230+0000 7f79d27fc700 1 -- 172.21.15.52:0/597796780 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f79e4005520 con 0x558bd99ba880 2024-06-10T03:00:28.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.230+0000 7f79fbe5cb80 10 client.16386 fetch_fsmap learned FSMap version 36 2024-06-10T03:00:28.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.231+0000 7f79fbe5cb80 1 -- 172.21.15.52:0/597796780 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x558bd9a37fe0 con 0x558bd99ba880 2024-06-10T03:00:28.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.231+0000 7f79d27fc700 1 -- 172.21.15.52:0/597796780 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f79e40079c0 con 0x558bd99ba880 2024-06-10T03:00:28.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.231+0000 7f79d27fc700 1 -- 172.21.15.52:0/597796780 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 36) ==== 8378+0+0 (secure 0 0 0) 0x7f79e4011070 con 0x558bd99ba880 2024-06-10T03:00:28.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.231+0000 7f79fbe5cb80 10 client.16386 fetch_fsmap finished waiting for FSMap version 36 2024-06-10T03:00:28.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.231+0000 7f79fbe5cb80 10 client.16386 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-06-10T03:00:28.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.231+0000 7f79fbe5cb80 20 client.16386 populate_metadata read hostname 'smithi052' 2024-06-10T03:00:28.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.231+0000 7f79fbe5cb80 1 --2- 172.21.15.52:0/597796780 >> [v2:172.21.15.120:6836/4132440952,v1:172.21.15.120:6837/4132440952] conn(0x558bd9a385d0 0x558bd9a3a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:28.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.231+0000 7f79fbe5cb80 4 client.16386 mds_command: new command op to 24428 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T03:00:28.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.231+0000 7f79fbe5cb80 1 -- 172.21.15.52:0/597796780 --> [v2:172.21.15.120:6836/4132440952,v1:172.21.15.120:6837/4132440952] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558bd990d2d0 con 0x558bd9a385d0 2024-06-10T03:00:28.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.231+0000 7f79eaffd700 1 --2- 172.21.15.52:0/597796780 >> [v2:172.21.15.120:6836/4132440952,v1:172.21.15.120:6837/4132440952] conn(0x558bd9a385d0 0x558bd9a3a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:28.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.232+0000 7f79eaffd700 1 --2- 172.21.15.52:0/597796780 >> [v2:172.21.15.120:6836/4132440952,v1:172.21.15.120:6837/4132440952] conn(0x558bd9a385d0 0x558bd9a3a9c0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d14e66ae9f386120 server_cookie=e031611312f68098 in_seq=0 out_seq=0 2024-06-10T03:00:28.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.232+0000 7f79d27fc700 10 client.16386 ms_handle_connect on v2:172.21.15.120:6836/4132440952 2024-06-10T03:00:28.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.232+0000 7f79d27fc700 1 -- 172.21.15.52:0/597796780 <== mds.4 v2:172.21.15.120:6836/4132440952 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558bd990d2d0 con 0x558bd9a385d0 2024-06-10T03:00:28.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.232+0000 7f79d27fc700 10 client.16386 handle_command_reply: tid=0 2024-06-10T03:00:28.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.232+0000 7f79fbe5cb80 1 client.16386 shutdown 2024-06-10T03:00:28.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.232+0000 7f79fbe5cb80 1 -- 172.21.15.52:0/597796780 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558bd99ba880 msgr2=0x558bd99d49c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:00:28.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.232+0000 7f79fbe5cb80 1 --2- 172.21.15.52:0/597796780 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558bd99ba880 0x558bd99d49c0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f79e400d010 tx=0x7f79e4006130 comp rx=0 tx=0).stop 2024-06-10T03:00:28.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.232+0000 7f79fbe5cb80 1 -- 172.21.15.52:0/597796780 shutdown_connections 2024-06-10T03:00:28.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.232+0000 7f79fbe5cb80 1 --2- 172.21.15.52:0/597796780 >> [v2:172.21.15.120:6836/4132440952,v1:172.21.15.120:6837/4132440952] conn(0x558bd9a385d0 0x558bd9a3a9c0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:28.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.232+0000 7f79fbe5cb80 1 --2- 172.21.15.52:0/597796780 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558bd99ba880 0x558bd99d49c0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:28.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.232+0000 7f79d27fc700 0 client.16386 ms_handle_reset on v2:172.21.15.120:6836/4132440952 2024-06-10T03:00:28.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.232+0000 7f79fbe5cb80 1 --2- 172.21.15.52:0/597796780 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558bd99b47a0 0x558bd99c6490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:28.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.232+0000 7f79fbe5cb80 1 --2- 172.21.15.52:0/597796780 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558bd99b9f60 0x558bd99c69d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:28.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.232+0000 7f79fbe5cb80 1 -- 172.21.15.52:0/597796780 >> 172.21.15.52:0/597796780 conn(0x558bd98e1080 msgr2=0x558bd99b92d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:00:28.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.232+0000 7f79fbe5cb80 1 -- 172.21.15.52:0/597796780 shutdown_connections 2024-06-10T03:00:28.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.232+0000 7f79fbe5cb80 1 -- 172.21.15.52:0/597796780 wait complete. 2024-06-10T03:00:28.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.232+0000 7f79fbe5cb80 20 client.16386 trim_cache size 0 max 16384 2024-06-10T03:00:28.241 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-06-10T03:00:28.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.233+0000 7f79f15d9700 1 -- 172.21.15.52:0/2586795643 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f79d80a2ca0 msgr2=0x7f79d80a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:00:28.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.233+0000 7f79f15d9700 1 --2- 172.21.15.52:0/2586795643 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f79d80a2ca0 0x7f79d80a5160 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f79ec078840 tx=0x7f79d400b410 comp rx=0 tx=0).stop 2024-06-10T03:00:28.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.233+0000 7f79f15d9700 1 -- 172.21.15.52:0/2586795643 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f79ec06a170 msgr2=0x7f79ec101460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:00:28.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.233+0000 7f79f15d9700 1 --2- 172.21.15.52:0/2586795643 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f79ec06a170 0x7f79ec101460 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f79dc00d980 tx=0x7f79dc00dc90 comp rx=0 tx=0).stop 2024-06-10T03:00:28.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.233+0000 7f79f15d9700 1 -- 172.21.15.52:0/2586795643 shutdown_connections 2024-06-10T03:00:28.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.233+0000 7f79f15d9700 1 --2- 172.21.15.52:0/2586795643 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f79d80a2ca0 0x7f79d80a5160 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:28.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.233+0000 7f79f15d9700 1 --2- 172.21.15.52:0/2586795643 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f79ec108500 0x7f79ec101ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:28.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.233+0000 7f79f15d9700 1 --2- 172.21.15.52:0/2586795643 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f79ec06a170 0x7f79ec101460 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:28.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.233+0000 7f79f15d9700 1 --2- 172.21.15.52:0/2586795643 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f79ec06ab30 0x7f79ec1019a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:28.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.233+0000 7f79f15d9700 1 -- 172.21.15.52:0/2586795643 >> 172.21.15.52:0/2586795643 conn(0x7f79ec078340 msgr2=0x7f79ec0fffa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:00:28.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.233+0000 7f79f15d9700 1 -- 172.21.15.52:0/2586795643 shutdown_connections 2024-06-10T03:00:28.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:28.233+0000 7f79f15d9700 1 -- 172.21.15.52:0/2586795643 wait complete. 2024-06-10T03:00:28.476 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:00:28 smithi052 ceph-mon[32362]: pgmap v622: 129 pgs: 129 active+clean; 569 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 141 op/s 2024-06-10T03:00:28.638 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:00:28 smithi120 ceph-mon[36634]: pgmap v622: 129 pgs: 129 active+clean; 569 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 141 op/s 2024-06-10T03:00:28.703 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:00:28 smithi188 ceph-mon[36599]: pgmap v622: 129 pgs: 129 active+clean; 569 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 141 op/s 2024-06-10T03:00:28.979 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-06-10T03:00:29.280 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T03:00:30.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:30.069+0000 7f3b23fff700 1 -- 172.21.15.52:0/2946735471 <== mds.0 v2:172.21.15.188:6838/310444836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55e506820430 con 0x55e506954900 2024-06-10T03:00:30.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:30.069+0000 7f3b23fff700 10 client.25909 handle_command_reply: tid=0 2024-06-10T03:00:30.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:30.069+0000 7f3b55565b80 1 client.25909 shutdown 2024-06-10T03:00:30.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:30.069+0000 7f3b55565b80 1 -- 172.21.15.52:0/2946735471 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e5068c7470 msgr2=0x55e5068d91a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:00:30.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:30.069+0000 7f3b55565b80 1 --2- 172.21.15.52:0/2946735471 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e5068c7470 0x55e5068d91a0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f3b38000c00 tx=0x7f3b380042c0 comp rx=0 tx=0).stop 2024-06-10T03:00:30.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:30.069+0000 7f3b55565b80 1 -- 172.21.15.52:0/2946735471 shutdown_connections 2024-06-10T03:00:30.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:30.069+0000 7f3b55565b80 1 --2- 172.21.15.52:0/2946735471 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55e506954900 0x55e506956cf0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:30.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:30.069+0000 7f3b55565b80 1 --2- 172.21.15.52:0/2946735471 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e5068cd600 0x55e5068d6170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:30.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:30.069+0000 7f3b55565b80 1 --2- 172.21.15.52:0/2946735471 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e5068c7470 0x55e5068d91a0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:30.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:30.069+0000 7f3b23fff700 0 client.25909 ms_handle_reset on v2:172.21.15.188:6838/310444836 2024-06-10T03:00:30.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:30.069+0000 7f3b55565b80 1 --2- 172.21.15.52:0/2946735471 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e5068ccce0 0x55e5068d96e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:30.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:30.069+0000 7f3b55565b80 1 -- 172.21.15.52:0/2946735471 >> 172.21.15.52:0/2946735471 conn(0x55e5067f1e60 msgr2=0x55e5068c6590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:00:30.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:30.070+0000 7f3b55565b80 1 -- 172.21.15.52:0/2946735471 shutdown_connections 2024-06-10T03:00:30.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:30.070+0000 7f3b55565b80 1 -- 172.21.15.52:0/2946735471 wait complete. 2024-06-10T03:00:30.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:30.070+0000 7f3b55565b80 20 client.25909 trim_cache size 0 max 16384 2024-06-10T03:00:30.071 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-10T03:00:30.071 INFO:teuthology.orchestra.run.smithi052.stdout: "epoch": 36, 2024-06-10T03:00:30.071 INFO:teuthology.orchestra.run.smithi052.stdout: "leader": 24484, 2024-06-10T03:00:30.071 INFO:teuthology.orchestra.run.smithi052.stdout: "set_version": 18, 2024-06-10T03:00:30.072 INFO:teuthology.orchestra.run.smithi052.stdout: "sets": { 2024-06-10T03:00:30.072 INFO:teuthology.orchestra.run.smithi052.stdout: "c8954a8e": { 2024-06-10T03:00:30.072 INFO:teuthology.orchestra.run.smithi052.stdout: "version": 18, 2024-06-10T03:00:30.072 INFO:teuthology.orchestra.run.smithi052.stdout: "age_ref": 0.0, 2024-06-10T03:00:30.072 INFO:teuthology.orchestra.run.smithi052.stdout: "state": { 2024-06-10T03:00:30.072 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "QUIESCED", 2024-06-10T03:00:30.072 INFO:teuthology.orchestra.run.smithi052.stdout: "age": 0.0 2024-06-10T03:00:30.072 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:00:30.072 INFO:teuthology.orchestra.run.smithi052.stdout: "timeout": 90.0, 2024-06-10T03:00:30.072 INFO:teuthology.orchestra.run.smithi052.stdout: "expiration": 149.4, 2024-06-10T03:00:30.072 INFO:teuthology.orchestra.run.smithi052.stdout: "members": { 2024-06-10T03:00:30.073 INFO:teuthology.orchestra.run.smithi052.stdout: "file:/": { 2024-06-10T03:00:30.073 INFO:teuthology.orchestra.run.smithi052.stdout: "excluded": false, 2024-06-10T03:00:30.073 INFO:teuthology.orchestra.run.smithi052.stdout: "state": { 2024-06-10T03:00:30.073 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "QUIESCED", 2024-06-10T03:00:30.073 INFO:teuthology.orchestra.run.smithi052.stdout: "age": 0.0 2024-06-10T03:00:30.073 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:00:30.073 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:00:30.073 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:00:30.073 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:00:30.073 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:00:30.073 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-10T03:00:30.074 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:30.070+0000 7f3b4ace2700 1 -- 172.21.15.52:0/1352397341 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f3b340a2cc0 msgr2=0x7f3b340a5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:00:30.074 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:30.070+0000 7f3b4ace2700 1 --2- 172.21.15.52:0/1352397341 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f3b340a2cc0 0x7f3b340a5180 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f3b44107220 tx=0x7f3b38009260 comp rx=0 tx=0).stop 2024-06-10T03:00:30.074 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:30.070+0000 7f3b4ace2700 1 -- 172.21.15.52:0/1352397341 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3b44107850 msgr2=0x7f3b4419aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:00:30.074 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:30.070+0000 7f3b4ace2700 1 --2- 172.21.15.52:0/1352397341 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3b44107850 0x7f3b4419aa90 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f3b3c006070 tx=0x7f3b3c007a20 comp rx=0 tx=0).stop 2024-06-10T03:00:30.074 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:30.070+0000 7f3b4ace2700 1 -- 172.21.15.52:0/1352397341 shutdown_connections 2024-06-10T03:00:30.074 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:30.070+0000 7f3b4ace2700 1 --2- 172.21.15.52:0/1352397341 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3b44107850 0x7f3b4419aa90 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:30.074 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:30.070+0000 7f3b4ace2700 1 --2- 172.21.15.52:0/1352397341 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3b4406a3d0 0x7f3b441a0880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:30.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:30.070+0000 7f3b4ace2700 1 --2- 172.21.15.52:0/1352397341 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f3b340a2cc0 0x7f3b340a5180 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:30.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:30.070+0000 7f3b4ace2700 1 --2- 172.21.15.52:0/1352397341 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3b4406ad80 0x7f3b441a0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:30.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:30.070+0000 7f3b4ace2700 1 -- 172.21.15.52:0/1352397341 >> 172.21.15.52:0/1352397341 conn(0x7f3b44078330 msgr2=0x7f3b4410d760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:00:30.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:30.070+0000 7f3b4ace2700 1 -- 172.21.15.52:0/1352397341 shutdown_connections 2024-06-10T03:00:30.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:30.071+0000 7f3b4ace2700 1 -- 172.21.15.52:0/1352397341 wait complete. 2024-06-10T03:00:30.477 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:00:30 smithi052 ceph-mon[32362]: pgmap v623: 129 pgs: 129 active+clean; 558 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 130 op/s 2024-06-10T03:00:30.638 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:00:30 smithi120 ceph-mon[36634]: pgmap v623: 129 pgs: 129 active+clean; 558 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 130 op/s 2024-06-10T03:00:30.704 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:00:30 smithi188 ceph-mon[36599]: pgmap v623: 129 pgs: 129 active+clean; 558 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 130 op/s 2024-06-10T03:00:31.162 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'c8954a8e', quiesce took 17.5 seconds. Will release after: 11.95 2024-06-10T03:00:31.765 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.764+0000 7f45c572a700 1 -- 172.21.15.52:0/1904279944 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f45c0108500 msgr2=0x7f45c01088e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:00:31.765 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.764+0000 7f45c572a700 1 --2- 172.21.15.52:0/1904279944 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f45c0108500 0x7f45c01088e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f45b00098f0 tx=0x7f45b0009c00 comp rx=0 tx=0).stop 2024-06-10T03:00:31.765 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.764+0000 7f45c572a700 1 -- 172.21.15.52:0/1904279944 shutdown_connections 2024-06-10T03:00:31.765 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.764+0000 7f45c572a700 1 --2- 172.21.15.52:0/1904279944 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f45c006ab30 0x7f45c010f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:31.765 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.764+0000 7f45c572a700 1 --2- 172.21.15.52:0/1904279944 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f45c0108500 0x7f45c01088e0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:31.765 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.764+0000 7f45c572a700 1 --2- 172.21.15.52:0/1904279944 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f45c006a170 0x7f45c006a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:31.765 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.764+0000 7f45c572a700 1 -- 172.21.15.52:0/1904279944 >> 172.21.15.52:0/1904279944 conn(0x7f45c0078340 msgr2=0x7f45c0078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:00:31.765 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.764+0000 7f45c572a700 1 -- 172.21.15.52:0/1904279944 shutdown_connections 2024-06-10T03:00:31.765 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.765+0000 7f45c572a700 1 -- 172.21.15.52:0/1904279944 wait complete. 2024-06-10T03:00:31.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.765+0000 7f45c572a700 1 Processor -- start 2024-06-10T03:00:31.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.765+0000 7f45c572a700 1 -- start start 2024-06-10T03:00:31.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.766+0000 7f45c572a700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f45c006a170 0x7f45c019e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:31.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.766+0000 7f45c572a700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f45c006ab30 0x7f45c019ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:31.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.766+0000 7f45c572a700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f45c0108500 0x7f45c0198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:31.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.766+0000 7f45c572a700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f45c0112760 con 0x7f45c0108500 2024-06-10T03:00:31.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.766+0000 7f45c572a700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f45c01125e0 con 0x7f45c006ab30 2024-06-10T03:00:31.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.766+0000 7f45c572a700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f45c01128e0 con 0x7f45c006a170 2024-06-10T03:00:31.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.766+0000 7f45be7fc700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f45c006ab30 0x7f45c019ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:31.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.766+0000 7f45be7fc700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f45c006ab30 0x7f45c019ee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.52:50268/0 (socket says 172.21.15.52:50268) 2024-06-10T03:00:31.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.766+0000 7f45be7fc700 1 -- 172.21.15.52:0/2795121775 learned_addr learned my addr 172.21.15.52:0/2795121775 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:00:31.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.766+0000 7f45beffd700 1 --2- 172.21.15.52:0/2795121775 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f45c006a170 0x7f45c019e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:31.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.766+0000 7f45bf7fe700 1 --2- 172.21.15.52:0/2795121775 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f45c0108500 0x7f45c0198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:31.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.766+0000 7f45be7fc700 1 -- 172.21.15.52:0/2795121775 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f45c006a170 msgr2=0x7f45c019e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:00:31.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.766+0000 7f45be7fc700 1 --2- 172.21.15.52:0/2795121775 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f45c006a170 0x7f45c019e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:31.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.766+0000 7f45be7fc700 1 -- 172.21.15.52:0/2795121775 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f45c0108500 msgr2=0x7f45c0198b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:00:31.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.766+0000 7f45be7fc700 1 --2- 172.21.15.52:0/2795121775 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f45c0108500 0x7f45c0198b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:31.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.766+0000 7f45be7fc700 1 -- 172.21.15.52:0/2795121775 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45c0199260 con 0x7f45c006ab30 2024-06-10T03:00:31.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.766+0000 7f45be7fc700 1 --2- 172.21.15.52:0/2795121775 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f45c006ab30 0x7f45c019ee80 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f45b400ec10 tx=0x7f45b400ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:00:31.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.767+0000 7f45a7fff700 1 -- 172.21.15.52:0/2795121775 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45b400cc90 con 0x7f45c006ab30 2024-06-10T03:00:31.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.767+0000 7f45a7fff700 1 -- 172.21.15.52:0/2795121775 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45b400ce30 con 0x7f45c006ab30 2024-06-10T03:00:31.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.767+0000 7f45c572a700 1 -- 172.21.15.52:0/2795121775 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45c0199550 con 0x7f45c006ab30 2024-06-10T03:00:31.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.767+0000 7f45a7fff700 1 -- 172.21.15.52:0/2795121775 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45b4010690 con 0x7f45c006ab30 2024-06-10T03:00:31.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.767+0000 7f45c572a700 1 -- 172.21.15.52:0/2795121775 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45c01a5810 con 0x7f45c006ab30 2024-06-10T03:00:31.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.770+0000 7f45a7fff700 1 -- 172.21.15.52:0/2795121775 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f45b4003680 con 0x7f45c006ab30 2024-06-10T03:00:31.770 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.770+0000 7f45a7fff700 1 --2- 172.21.15.52:0/2795121775 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f45a80a2ca0 0x7f45a80a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:31.770 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.770+0000 7f45beffd700 1 --2- 172.21.15.52:0/2795121775 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f45a80a2ca0 0x7f45a80a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:31.770 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.770+0000 7f45a7fff700 1 -- 172.21.15.52:0/2795121775 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f45b40cf6e0 con 0x7f45c006ab30 2024-06-10T03:00:31.772 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.771+0000 7f45beffd700 1 --2- 172.21.15.52:0/2795121775 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f45a80a2ca0 0x7f45a80a5160 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f45b00098c0 tx=0x7f45b0019040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:00:31.772 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.771+0000 7f45cffadb80 1 Processor -- start 2024-06-10T03:00:31.773 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.771+0000 7f45cffadb80 1 -- start start 2024-06-10T03:00:31.773 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.771+0000 7f45cffadb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5631ee082a70 0x5631ee082e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:31.773 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.772+0000 7f45cffadb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5631ee083390 0x5631ee07cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:31.773 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.772+0000 7f45cffadb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5631ee07d330 0x5631ee07d750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:31.773 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.772+0000 7f45cffadb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5631edfca810 con 0x5631ee07d330 2024-06-10T03:00:31.773 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.772+0000 7f45cffadb80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5631edf72720 con 0x5631ee082a70 2024-06-10T03:00:31.773 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.772+0000 7f45cffadb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5631edfc67d0 con 0x5631ee083390 2024-06-10T03:00:31.773 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.772+0000 7f45beffd700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5631ee082a70 0x5631ee082e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:31.773 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.772+0000 7f45beffd700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5631ee082a70 0x5631ee082e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.52:50282/0 (socket says 172.21.15.52:50282) 2024-06-10T03:00:31.773 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.772+0000 7f45beffd700 1 -- 172.21.15.52:0/3979301014 learned_addr learned my addr 172.21.15.52:0/3979301014 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:00:31.773 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.772+0000 7f45bf7fe700 1 --2- 172.21.15.52:0/3979301014 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5631ee07d330 0x5631ee07d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:31.773 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.772+0000 7f45be7fc700 1 --2- 172.21.15.52:0/3979301014 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5631ee083390 0x5631ee07cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:31.773 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.772+0000 7f45beffd700 1 -- 172.21.15.52:0/3979301014 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5631ee083390 msgr2=0x5631ee07cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:00:31.774 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.772+0000 7f45beffd700 1 --2- 172.21.15.52:0/3979301014 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5631ee083390 0x5631ee07cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:31.774 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.772+0000 7f45beffd700 1 -- 172.21.15.52:0/3979301014 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5631ee07d330 msgr2=0x5631ee07d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:00:31.774 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.772+0000 7f45beffd700 1 --2- 172.21.15.52:0/3979301014 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5631ee07d330 0x5631ee07d750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:31.774 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.772+0000 7f45beffd700 1 -- 172.21.15.52:0/3979301014 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5631edf68f50 con 0x5631ee082a70 2024-06-10T03:00:31.774 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.772+0000 7f45beffd700 1 --2- 172.21.15.52:0/3979301014 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5631ee082a70 0x5631ee082e50 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f45b000f910 tx=0x7f45b001e000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a19266e11ef992f6 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:00:31.774 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.772+0000 7f45a57fa700 1 -- 172.21.15.52:0/3979301014 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45b0005240 con 0x5631ee082a70 2024-06-10T03:00:31.774 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.772+0000 7f45a57fa700 1 -- 172.21.15.52:0/3979301014 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45b0003b60 con 0x5631ee082a70 2024-06-10T03:00:31.774 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.772+0000 7f45a57fa700 1 -- 172.21.15.52:0/3979301014 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45b000b8e0 con 0x5631ee082a70 2024-06-10T03:00:31.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.773+0000 7f45cffadb80 1 -- 172.21.15.52:0/3979301014 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5631ee082a70 msgr2=0x5631ee082e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:00:31.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.773+0000 7f45cffadb80 1 --2- 172.21.15.52:0/3979301014 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5631ee082a70 0x5631ee082e50 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f45b000f910 tx=0x7f45b001e000 comp rx=0 tx=0).stop 2024-06-10T03:00:31.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.773+0000 7f45cffadb80 1 -- 172.21.15.52:0/3979301014 shutdown_connections 2024-06-10T03:00:31.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.773+0000 7f45cffadb80 1 --2- 172.21.15.52:0/3979301014 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5631ee07d330 0x5631ee07d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:31.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.773+0000 7f45cffadb80 1 --2- 172.21.15.52:0/3979301014 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5631ee082a70 0x5631ee082e50 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:31.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.773+0000 7f45cffadb80 1 --2- 172.21.15.52:0/3979301014 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5631ee083390 0x5631ee07cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:31.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.773+0000 7f45cffadb80 1 -- 172.21.15.52:0/3979301014 >> 172.21.15.52:0/3979301014 conn(0x5631edfa3cc0 msgr2=0x5631ee073850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:00:31.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.774+0000 7f45cffadb80 1 -- 172.21.15.52:0/3979301014 shutdown_connections 2024-06-10T03:00:31.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.774+0000 7f45cffadb80 1 -- 172.21.15.52:0/3979301014 wait complete. 2024-06-10T03:00:31.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.774+0000 7f45cffadb80 1 -- start start 2024-06-10T03:00:31.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.774+0000 7f45cffadb80 1 Processor -- start 2024-06-10T03:00:31.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.774+0000 7f45cffadb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5631ee07d330 0x5631ee089160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:31.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.774+0000 7f45cffadb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5631ee082a70 0x5631ee086460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:31.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.774+0000 7f45cffadb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5631ee083390 0x5631ee0869a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:31.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.774+0000 7f45cffadb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5631ee089d70 con 0x5631ee083390 2024-06-10T03:00:31.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.774+0000 7f45cffadb80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5631edf6abf0 con 0x5631ee082a70 2024-06-10T03:00:31.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.774+0000 7f45cffadb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5631ee089ef0 con 0x5631ee07d330 2024-06-10T03:00:31.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.774+0000 7f45cffadb80 10 client.0 _ll_register_callbacks cb 0x5631edf64920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T03:00:31.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.775+0000 7f45bf7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5631ee083390 0x5631ee0869a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:31.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.775+0000 7f45beffd700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5631ee07d330 0x5631ee089160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:31.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.775+0000 7f45bf7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5631ee083390 0x5631ee0869a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:42730/0 (socket says 172.21.15.52:42730) 2024-06-10T03:00:31.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.775+0000 7f45be7fc700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5631ee082a70 0x5631ee086460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:31.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.775+0000 7f45bf7fe700 1 -- 172.21.15.52:0/3757344446 learned_addr learned my addr 172.21.15.52:0/3757344446 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:00:31.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.775+0000 7f45bf7fe700 1 -- 172.21.15.52:0/3757344446 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5631ee07d330 msgr2=0x5631ee089160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:00:31.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.775+0000 7f45bf7fe700 1 --2- 172.21.15.52:0/3757344446 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5631ee07d330 0x5631ee089160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:31.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.775+0000 7f45bf7fe700 1 -- 172.21.15.52:0/3757344446 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5631ee082a70 msgr2=0x5631ee086460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:00:31.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.775+0000 7f45bf7fe700 1 --2- 172.21.15.52:0/3757344446 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5631ee082a70 0x5631ee086460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:31.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.775+0000 7f45bf7fe700 1 -- 172.21.15.52:0/3757344446 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5631edf198d0 con 0x5631ee083390 2024-06-10T03:00:31.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.775+0000 7f45beffd700 1 --2- 172.21.15.52:0/3757344446 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5631ee07d330 0x5631ee089160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T03:00:31.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.775+0000 7f45be7fc700 1 --2- 172.21.15.52:0/3757344446 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5631ee082a70 0x5631ee086460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T03:00:31.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.775+0000 7f45bf7fe700 1 --2- 172.21.15.52:0/3757344446 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5631ee083390 0x5631ee0869a0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f45b8011e80 tx=0x7f45b8014600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e76bfdc9a8193843 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:00:31.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.775+0000 7f45a2ff5700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-10T03:00:31.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.775+0000 7f45a2ff5700 1 -- 172.21.15.52:0/3757344446 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45b801e420 con 0x5631ee083390 2024-06-10T03:00:31.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.775+0000 7f45a2ff5700 1 -- 172.21.15.52:0/3757344446 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f45b801e5c0 con 0x5631ee083390 2024-06-10T03:00:31.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.775+0000 7f45cffadb80 1 -- 172.21.15.52:0/3757344446 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f45b0003b60 con 0x5631ee083390 2024-06-10T03:00:31.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.776+0000 7f45a2ff5700 1 -- 172.21.15.52:0/3757344446 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f45b8029050 con 0x5631ee083390 2024-06-10T03:00:31.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.776+0000 7f45cffadb80 10 client.16398 fetch_fsmap learned FSMap version 36 2024-06-10T03:00:31.781 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.776+0000 7f45cffadb80 1 -- 172.21.15.52:0/3757344446 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5631ee094750 con 0x5631ee083390 2024-06-10T03:00:31.781 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.776+0000 7f45a2ff5700 1 -- 172.21.15.52:0/3757344446 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45b0003b60 con 0x5631ee083390 2024-06-10T03:00:31.781 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.776+0000 7f45a2ff5700 1 -- 172.21.15.52:0/3757344446 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 36) ==== 8378+0+0 (secure 0 0 0) 0x7f45b80168b0 con 0x5631ee083390 2024-06-10T03:00:31.781 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.776+0000 7f45cffadb80 10 client.16398 fetch_fsmap finished waiting for FSMap version 36 2024-06-10T03:00:31.781 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.776+0000 7f45cffadb80 10 client.16398 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.b 2024-06-10T03:00:31.781 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.776+0000 7f45cffadb80 20 client.16398 populate_metadata read hostname 'smithi052' 2024-06-10T03:00:31.781 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.776+0000 7f45cffadb80 1 --2- 172.21.15.52:0/3757344446 >> [v2:172.21.15.120:6838/1672436487,v1:172.21.15.120:6839/1672436487] conn(0x5631ee096d90 0x5631ee099180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:31.781 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.776+0000 7f45cffadb80 4 client.16398 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T03:00:31.781 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.776+0000 7f45cffadb80 1 -- 172.21.15.52:0/3757344446 --> [v2:172.21.15.120:6838/1672436487,v1:172.21.15.120:6839/1672436487] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5631edfd7d10 con 0x5631ee096d90 2024-06-10T03:00:31.781 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.776+0000 7f45beffd700 1 --2- 172.21.15.52:0/3757344446 >> [v2:172.21.15.120:6838/1672436487,v1:172.21.15.120:6839/1672436487] conn(0x5631ee096d90 0x5631ee099180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:31.781 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.777+0000 7f45beffd700 1 --2- 172.21.15.52:0/3757344446 >> [v2:172.21.15.120:6838/1672436487,v1:172.21.15.120:6839/1672436487] conn(0x5631ee096d90 0x5631ee099180 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=7ebe126d66612667 server_cookie=f1bcb1cdef8b99a5 in_seq=0 out_seq=0 2024-06-10T03:00:31.781 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.777+0000 7f45a2ff5700 10 client.16398 ms_handle_connect on v2:172.21.15.120:6838/1672436487 2024-06-10T03:00:31.781 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.777+0000 7f45a2ff5700 1 -- 172.21.15.52:0/3757344446 <== mds.3 v2:172.21.15.120:6838/1672436487 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5631edfd7d10 con 0x5631ee096d90 2024-06-10T03:00:31.782 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.777+0000 7f45a2ff5700 10 client.16398 handle_command_reply: tid=0 2024-06-10T03:00:31.782 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.778+0000 7f45cffadb80 1 client.16398 shutdown 2024-06-10T03:00:31.782 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.778+0000 7f45cffadb80 1 -- 172.21.15.52:0/3757344446 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5631ee083390 msgr2=0x5631ee0869a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:00:31.782 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.778+0000 7f45cffadb80 1 --2- 172.21.15.52:0/3757344446 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5631ee083390 0x5631ee0869a0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f45b8011e80 tx=0x7f45b8014600 comp rx=0 tx=0).stop 2024-06-10T03:00:31.782 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.778+0000 7f45cffadb80 1 -- 172.21.15.52:0/3757344446 shutdown_connections 2024-06-10T03:00:31.782 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.778+0000 7f45cffadb80 1 --2- 172.21.15.52:0/3757344446 >> [v2:172.21.15.120:6838/1672436487,v1:172.21.15.120:6839/1672436487] conn(0x5631ee096d90 0x5631ee099180 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:31.782 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.778+0000 7f45a2ff5700 0 client.16398 ms_handle_reset on v2:172.21.15.120:6838/1672436487 2024-06-10T03:00:31.782 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.778+0000 7f45cffadb80 1 --2- 172.21.15.52:0/3757344446 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5631ee083390 0x5631ee0869a0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:31.782 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.778+0000 7f45cffadb80 1 --2- 172.21.15.52:0/3757344446 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5631ee07d330 0x5631ee089160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:31.782 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.778+0000 7f45cffadb80 1 --2- 172.21.15.52:0/3757344446 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5631ee082a70 0x5631ee086460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:31.782 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.778+0000 7f45cffadb80 1 -- 172.21.15.52:0/3757344446 >> 172.21.15.52:0/3757344446 conn(0x5631edfa3cc0 msgr2=0x5631ee0779d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:00:31.782 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.778+0000 7f45cffadb80 1 -- 172.21.15.52:0/3757344446 shutdown_connections 2024-06-10T03:00:31.782 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.778+0000 7f45cffadb80 1 -- 172.21.15.52:0/3757344446 wait complete. 2024-06-10T03:00:31.783 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.778+0000 7f45cffadb80 20 client.16398 trim_cache size 0 max 16384 2024-06-10T03:00:31.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.795+0000 7f45cffadb80 1 Processor -- start 2024-06-10T03:00:31.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.795+0000 7f45cffadb80 1 -- start start 2024-06-10T03:00:31.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.795+0000 7f45cffadb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5631ee07d330 0x5631ee082e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:31.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.795+0000 7f45cffadb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5631ee082a70 0x5631ee084d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:31.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.795+0000 7f45cffadb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5631ee083390 0x5631ee085290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:31.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.795+0000 7f45cffadb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f45c01ac1d0 con 0x5631ee082a70 2024-06-10T03:00:31.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.795+0000 7f45cffadb80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f45c01addd0 con 0x5631ee083390 2024-06-10T03:00:31.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.795+0000 7f45cffadb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f45c01ac8a0 con 0x5631ee07d330 2024-06-10T03:00:31.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.795+0000 7f45be7fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5631ee082a70 0x5631ee084d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:31.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.795+0000 7f45be7fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5631ee082a70 0x5631ee084d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:42732/0 (socket says 172.21.15.52:42732) 2024-06-10T03:00:31.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.795+0000 7f45be7fc700 1 -- 172.21.15.52:0/3067275321 learned_addr learned my addr 172.21.15.52:0/3067275321 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:00:31.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.795+0000 7f45beffd700 1 --2- 172.21.15.52:0/3067275321 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5631ee07d330 0x5631ee082e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:31.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.795+0000 7f45bf7fe700 1 --2- 172.21.15.52:0/3067275321 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5631ee083390 0x5631ee085290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:31.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.795+0000 7f45be7fc700 1 -- 172.21.15.52:0/3067275321 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5631ee07d330 msgr2=0x5631ee082e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:00:31.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.795+0000 7f45be7fc700 1 --2- 172.21.15.52:0/3067275321 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5631ee07d330 0x5631ee082e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:31.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.795+0000 7f45be7fc700 1 -- 172.21.15.52:0/3067275321 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5631ee083390 msgr2=0x5631ee085290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:00:31.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.795+0000 7f45be7fc700 1 --2- 172.21.15.52:0/3067275321 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5631ee083390 0x5631ee085290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:31.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.795+0000 7f45be7fc700 1 -- 172.21.15.52:0/3067275321 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5631ee0849d0 con 0x5631ee082a70 2024-06-10T03:00:31.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.796+0000 7f45be7fc700 1 --2- 172.21.15.52:0/3067275321 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5631ee082a70 0x5631ee084d50 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f45b4098510 tx=0x7f45b400efb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1d228921185c781 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:00:31.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.796+0000 7f45a4ff9700 1 -- 172.21.15.52:0/3067275321 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45b40b8020 con 0x5631ee082a70 2024-06-10T03:00:31.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.796+0000 7f45a4ff9700 1 -- 172.21.15.52:0/3067275321 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45b409c290 con 0x5631ee082a70 2024-06-10T03:00:31.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.796+0000 7f45a4ff9700 1 -- 172.21.15.52:0/3067275321 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45b409c050 con 0x5631ee082a70 2024-06-10T03:00:31.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.797+0000 7f45cffadb80 1 -- 172.21.15.52:0/3067275321 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5631ee082a70 msgr2=0x5631ee084d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:00:31.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.797+0000 7f45cffadb80 1 --2- 172.21.15.52:0/3067275321 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5631ee082a70 0x5631ee084d50 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f45b4098510 tx=0x7f45b400efb0 comp rx=0 tx=0).stop 2024-06-10T03:00:31.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.797+0000 7f45cffadb80 1 -- 172.21.15.52:0/3067275321 shutdown_connections 2024-06-10T03:00:31.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.797+0000 7f45cffadb80 1 --2- 172.21.15.52:0/3067275321 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5631ee083390 0x5631ee085290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:31.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.797+0000 7f45cffadb80 1 --2- 172.21.15.52:0/3067275321 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5631ee07d330 0x5631ee082e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:31.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.797+0000 7f45cffadb80 1 --2- 172.21.15.52:0/3067275321 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5631ee082a70 0x5631ee084d50 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:31.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.797+0000 7f45cffadb80 1 -- 172.21.15.52:0/3067275321 >> 172.21.15.52:0/3067275321 conn(0x5631edfa3cc0 msgr2=0x5631ee0804b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:00:31.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.797+0000 7f45cffadb80 1 -- 172.21.15.52:0/3067275321 shutdown_connections 2024-06-10T03:00:31.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.797+0000 7f45cffadb80 1 -- 172.21.15.52:0/3067275321 wait complete. 2024-06-10T03:00:31.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.797+0000 7f45cffadb80 1 -- start start 2024-06-10T03:00:31.801 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.797+0000 7f45cffadb80 1 Processor -- start 2024-06-10T03:00:31.801 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.798+0000 7f45cffadb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5631ee07d330 0x5631ee08f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:31.801 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.798+0000 7f45cffadb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5631ee082a70 0x5631ee08f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:31.801 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.798+0000 7f45cffadb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5631ee083390 0x5631ee09de00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:31.801 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.798+0000 7f45cffadb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f45c01ad700 con 0x5631ee083390 2024-06-10T03:00:31.801 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.798+0000 7f45cffadb80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f45c01ad030 con 0x5631ee082a70 2024-06-10T03:00:31.801 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.798+0000 7f45cffadb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f45c01aa190 con 0x5631ee07d330 2024-06-10T03:00:31.801 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.798+0000 7f45cffadb80 10 client.0 _ll_register_callbacks cb 0x5631edf64920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T03:00:31.801 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.798+0000 7f45beffd700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5631ee07d330 0x5631ee08f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:31.801 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.798+0000 7f45beffd700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5631ee07d330 0x5631ee08f360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.52:34032/0 (socket says 172.21.15.52:34032) 2024-06-10T03:00:31.801 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.798+0000 7f45be7fc700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5631ee082a70 0x5631ee08f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:31.801 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.798+0000 7f45beffd700 1 -- 172.21.15.52:0/2400489662 learned_addr learned my addr 172.21.15.52:0/2400489662 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:00:31.801 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.798+0000 7f45bf7fe700 1 --2- 172.21.15.52:0/2400489662 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5631ee083390 0x5631ee09de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:31.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.798+0000 7f45beffd700 1 -- 172.21.15.52:0/2400489662 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5631ee082a70 msgr2=0x5631ee08f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:00:31.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.798+0000 7f45beffd700 1 --2- 172.21.15.52:0/2400489662 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5631ee082a70 0x5631ee08f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:31.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.798+0000 7f45beffd700 1 -- 172.21.15.52:0/2400489662 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5631ee083390 msgr2=0x5631ee09de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:00:31.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.798+0000 7f45beffd700 1 --2- 172.21.15.52:0/2400489662 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5631ee083390 0x5631ee09de00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:31.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.798+0000 7f45beffd700 1 -- 172.21.15.52:0/2400489662 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5631ee09e470 con 0x5631ee07d330 2024-06-10T03:00:31.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.798+0000 7f45be7fc700 1 --2- 172.21.15.52:0/2400489662 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5631ee082a70 0x5631ee08f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T03:00:31.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.799+0000 7f45a2ff5700 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-06-10T03:00:31.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.799+0000 7f45beffd700 1 --2- 172.21.15.52:0/2400489662 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5631ee07d330 0x5631ee08f360 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f45b0000c00 tx=0x7f45b00057a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8789e352bd3143eb server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:00:31.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.799+0000 7f45a2ff5700 1 -- 172.21.15.52:0/2400489662 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45b003c070 con 0x5631ee07d330 2024-06-10T03:00:31.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.799+0000 7f45a2ff5700 1 -- 172.21.15.52:0/2400489662 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f45b0003a90 con 0x5631ee07d330 2024-06-10T03:00:31.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.799+0000 7f45cffadb80 1 -- 172.21.15.52:0/2400489662 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f45b409c290 con 0x5631ee07d330 2024-06-10T03:00:31.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.799+0000 7f45a2ff5700 1 -- 172.21.15.52:0/2400489662 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f45b0045050 con 0x5631ee07d330 2024-06-10T03:00:31.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.799+0000 7f45cffadb80 10 client.25969 fetch_fsmap learned FSMap version 36 2024-06-10T03:00:31.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.799+0000 7f45cffadb80 1 -- 172.21.15.52:0/2400489662 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5631ee1012e0 con 0x5631ee07d330 2024-06-10T03:00:31.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.799+0000 7f45a2ff5700 1 -- 172.21.15.52:0/2400489662 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45b409c290 con 0x5631ee07d330 2024-06-10T03:00:31.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.799+0000 7f45a2ff5700 1 -- 172.21.15.52:0/2400489662 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 36) ==== 8378+0+0 (secure 0 0 0) 0x7f45b003f020 con 0x5631ee07d330 2024-06-10T03:00:31.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.799+0000 7f45cffadb80 10 client.25969 fetch_fsmap finished waiting for FSMap version 36 2024-06-10T03:00:31.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.799+0000 7f45cffadb80 10 client.25969 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.b 2024-06-10T03:00:31.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.800+0000 7f45cffadb80 20 client.25969 populate_metadata read hostname 'smithi052' 2024-06-10T03:00:31.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.800+0000 7f45cffadb80 1 --2- 172.21.15.52:0/2400489662 >> [v2:172.21.15.120:6838/1672436487,v1:172.21.15.120:6839/1672436487] conn(0x5631ee096d90 0x5631ee1034c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:31.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.800+0000 7f45cffadb80 4 client.25969 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T03:00:31.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.800+0000 7f45cffadb80 1 -- 172.21.15.52:0/2400489662 --> [v2:172.21.15.120:6838/1672436487,v1:172.21.15.120:6839/1672436487] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5631edfd2290 con 0x5631ee096d90 2024-06-10T03:00:31.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.800+0000 7f45be7fc700 1 --2- 172.21.15.52:0/2400489662 >> [v2:172.21.15.120:6838/1672436487,v1:172.21.15.120:6839/1672436487] conn(0x5631ee096d90 0x5631ee1034c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:31.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.800+0000 7f45a2ff5700 10 client.25969 ms_handle_connect on v2:172.21.15.120:6838/1672436487 2024-06-10T03:00:31.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.800+0000 7f45be7fc700 1 --2- 172.21.15.52:0/2400489662 >> [v2:172.21.15.120:6838/1672436487,v1:172.21.15.120:6839/1672436487] conn(0x5631ee096d90 0x5631ee1034c0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c4baa5f5e0e3f3bf server_cookie=5545bdda56510eba in_seq=0 out_seq=0 2024-06-10T03:00:31.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.800+0000 7f45a2ff5700 1 -- 172.21.15.52:0/2400489662 <== mds.3 v2:172.21.15.120:6838/1672436487 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5631edfd2290 con 0x5631ee096d90 2024-06-10T03:00:31.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.800+0000 7f45a2ff5700 10 client.25969 handle_command_reply: tid=0 2024-06-10T03:00:31.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.800+0000 7f45cffadb80 1 client.25969 shutdown 2024-06-10T03:00:31.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.800+0000 7f45cffadb80 1 -- 172.21.15.52:0/2400489662 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5631ee07d330 msgr2=0x5631ee08f360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:00:31.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.800+0000 7f45cffadb80 1 --2- 172.21.15.52:0/2400489662 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5631ee07d330 0x5631ee08f360 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f45b0000c00 tx=0x7f45b00057a0 comp rx=0 tx=0).stop 2024-06-10T03:00:31.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.801+0000 7f45cffadb80 1 -- 172.21.15.52:0/2400489662 shutdown_connections 2024-06-10T03:00:31.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.801+0000 7f45cffadb80 1 --2- 172.21.15.52:0/2400489662 >> [v2:172.21.15.120:6838/1672436487,v1:172.21.15.120:6839/1672436487] conn(0x5631ee096d90 0x5631ee1034c0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:31.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.801+0000 7f45cffadb80 1 --2- 172.21.15.52:0/2400489662 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5631ee083390 0x5631ee09de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:31.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.801+0000 7f45a2ff5700 0 client.25969 ms_handle_reset on v2:172.21.15.120:6838/1672436487 2024-06-10T03:00:31.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.801+0000 7f45cffadb80 1 --2- 172.21.15.52:0/2400489662 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5631ee07d330 0x5631ee08f360 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:31.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.801+0000 7f45cffadb80 1 --2- 172.21.15.52:0/2400489662 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5631ee082a70 0x5631ee08f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:31.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.801+0000 7f45cffadb80 1 -- 172.21.15.52:0/2400489662 >> 172.21.15.52:0/2400489662 conn(0x5631edfa3cc0 msgr2=0x5631ee084c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:00:31.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.801+0000 7f45cffadb80 1 -- 172.21.15.52:0/2400489662 shutdown_connections 2024-06-10T03:00:31.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.801+0000 7f45cffadb80 1 -- 172.21.15.52:0/2400489662 wait complete. 2024-06-10T03:00:31.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.801+0000 7f45cffadb80 20 client.25969 trim_cache size 0 max 16384 2024-06-10T03:00:31.806 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-06-10T03:00:31.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.802+0000 7f45c572a700 1 -- 172.21.15.52:0/2795121775 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f45a80a2ca0 msgr2=0x7f45a80a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:00:31.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.802+0000 7f45c572a700 1 --2- 172.21.15.52:0/2795121775 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f45a80a2ca0 0x7f45a80a5160 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f45b00098c0 tx=0x7f45b0019040 comp rx=0 tx=0).stop 2024-06-10T03:00:31.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.802+0000 7f45c572a700 1 -- 172.21.15.52:0/2795121775 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f45c006ab30 msgr2=0x7f45c019ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:00:31.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.802+0000 7f45c572a700 1 --2- 172.21.15.52:0/2795121775 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f45c006ab30 0x7f45c019ee80 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f45b400ec10 tx=0x7f45b400ef20 comp rx=0 tx=0).stop 2024-06-10T03:00:31.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.802+0000 7f45c572a700 1 -- 172.21.15.52:0/2795121775 shutdown_connections 2024-06-10T03:00:31.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.802+0000 7f45c572a700 1 --2- 172.21.15.52:0/2795121775 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f45a80a2ca0 0x7f45a80a5160 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:31.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.802+0000 7f45c572a700 1 --2- 172.21.15.52:0/2795121775 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f45c0108500 0x7f45c0198b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:31.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.802+0000 7f45c572a700 1 --2- 172.21.15.52:0/2795121775 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f45c006a170 0x7f45c019e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:31.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.802+0000 7f45c572a700 1 --2- 172.21.15.52:0/2795121775 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f45c006ab30 0x7f45c019ee80 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:31.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.802+0000 7f45c572a700 1 -- 172.21.15.52:0/2795121775 >> 172.21.15.52:0/2795121775 conn(0x7f45c0078340 msgr2=0x7f45c0100130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:00:31.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.802+0000 7f45c572a700 1 -- 172.21.15.52:0/2795121775 shutdown_connections 2024-06-10T03:00:31.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:31.802+0000 7f45c572a700 1 -- 172.21.15.52:0/2795121775 wait complete. 2024-06-10T03:00:32.476 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:00:32 smithi052 ceph-mon[32362]: pgmap v624: 129 pgs: 129 active+clean; 558 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 892 KiB/s rd, 915 KiB/s wr, 119 op/s 2024-06-10T03:00:32.538 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-06-10T03:00:32.638 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:00:32 smithi120 ceph-mon[36634]: pgmap v624: 129 pgs: 129 active+clean; 558 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 892 KiB/s rd, 915 KiB/s wr, 119 op/s 2024-06-10T03:00:32.703 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:00:32 smithi188 ceph-mon[36599]: pgmap v624: 129 pgs: 129 active+clean; 558 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 892 KiB/s rd, 915 KiB/s wr, 119 op/s 2024-06-10T03:00:32.812 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T03:00:34.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.499+0000 7fb5359f4700 1 -- 172.21.15.52:0/1509330070 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb530102ef0 msgr2=0x7fb53010f3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:00:34.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.499+0000 7fb5359f4700 1 --2- 172.21.15.52:0/1509330070 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb530102ef0 0x7fb53010f3a0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fb52800b0d0 tx=0x7fb52800b3e0 comp rx=0 tx=0).stop 2024-06-10T03:00:34.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.499+0000 7fb5359f4700 1 -- 172.21.15.52:0/1509330070 shutdown_connections 2024-06-10T03:00:34.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.499+0000 7fb5359f4700 1 --2- 172.21.15.52:0/1509330070 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb530102ef0 0x7fb53010f3a0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:34.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.499+0000 7fb5359f4700 1 --2- 172.21.15.52:0/1509330070 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb5301085d0 0x7fb5301089b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:34.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.499+0000 7fb5359f4700 1 --2- 172.21.15.52:0/1509330070 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb530102530 0x7fb5301029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:34.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.499+0000 7fb5359f4700 1 -- 172.21.15.52:0/1509330070 >> 172.21.15.52:0/1509330070 conn(0x7fb530079640 msgr2=0x7fb530079a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:00:34.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.499+0000 7fb5359f4700 1 -- 172.21.15.52:0/1509330070 shutdown_connections 2024-06-10T03:00:34.500 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.499+0000 7fb5359f4700 1 -- 172.21.15.52:0/1509330070 wait complete. 2024-06-10T03:00:34.500 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:00:34 smithi052 ceph-mon[32362]: pgmap v625: 129 pgs: 129 active+clean; 562 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.0 MiB/s wr, 242 op/s 2024-06-10T03:00:34.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.499+0000 7fb5359f4700 1 Processor -- start 2024-06-10T03:00:34.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.500+0000 7fb5359f4700 1 -- start start 2024-06-10T03:00:34.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.500+0000 7fb5359f4700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb530102530 0x7fb53019e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:34.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.500+0000 7fb5359f4700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb530102ef0 0x7fb53019ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:34.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.500+0000 7fb5359f4700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb5301085d0 0x7fb5301989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:34.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.500+0000 7fb5359f4700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb530112830 con 0x7fb5301085d0 2024-06-10T03:00:34.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.500+0000 7fb5359f4700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fb5301126b0 con 0x7fb530102530 2024-06-10T03:00:34.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.500+0000 7fb5359f4700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fb5301129b0 con 0x7fb530102ef0 2024-06-10T03:00:34.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.500+0000 7fb52effd700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb530102530 0x7fb53019e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:34.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.500+0000 7fb52effd700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb530102530 0x7fb53019e810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.52:50344/0 (socket says 172.21.15.52:50344) 2024-06-10T03:00:34.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.500+0000 7fb52effd700 1 -- 172.21.15.52:0/3380538897 learned_addr learned my addr 172.21.15.52:0/3380538897 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:00:34.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.500+0000 7fb52e7fc700 1 --2- 172.21.15.52:0/3380538897 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb530102ef0 0x7fb53019ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:34.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.500+0000 7fb52f7fe700 1 --2- 172.21.15.52:0/3380538897 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb5301085d0 0x7fb5301989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:34.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.500+0000 7fb52effd700 1 -- 172.21.15.52:0/3380538897 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb530102ef0 msgr2=0x7fb53019ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:00:34.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.500+0000 7fb52effd700 1 --2- 172.21.15.52:0/3380538897 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb530102ef0 0x7fb53019ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:34.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.500+0000 7fb52effd700 1 -- 172.21.15.52:0/3380538897 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb5301085d0 msgr2=0x7fb5301989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:00:34.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.500+0000 7fb52effd700 1 --2- 172.21.15.52:0/3380538897 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb5301085d0 0x7fb5301989d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:34.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.500+0000 7fb52effd700 1 -- 172.21.15.52:0/3380538897 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb530199130 con 0x7fb530102530 2024-06-10T03:00:34.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.501+0000 7fb52e7fc700 1 --2- 172.21.15.52:0/3380538897 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb530102ef0 0x7fb53019ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T03:00:34.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.501+0000 7fb52effd700 1 --2- 172.21.15.52:0/3380538897 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb530102530 0x7fb53019e810 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fb52000e9c0 tx=0x7fb52000ed80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:00:34.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.501+0000 7fb5349f2700 1 -- 172.21.15.52:0/3380538897 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb52000cbd0 con 0x7fb530102530 2024-06-10T03:00:34.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.501+0000 7fb5349f2700 1 -- 172.21.15.52:0/3380538897 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb52000cd70 con 0x7fb530102530 2024-06-10T03:00:34.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.501+0000 7fb5359f4700 1 -- 172.21.15.52:0/3380538897 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb530199420 con 0x7fb530102530 2024-06-10T03:00:34.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.501+0000 7fb5349f2700 1 -- 172.21.15.52:0/3380538897 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb520010430 con 0x7fb530102530 2024-06-10T03:00:34.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.501+0000 7fb5359f4700 1 -- 172.21.15.52:0/3380538897 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5301a56e0 con 0x7fb530102530 2024-06-10T03:00:34.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.504+0000 7fb5349f2700 1 -- 172.21.15.52:0/3380538897 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fb520004830 con 0x7fb530102530 2024-06-10T03:00:34.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.505+0000 7fb5349f2700 1 --2- 172.21.15.52:0/3380538897 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fb5180a2c20 0x7fb5180a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:34.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.505+0000 7fb52e7fc700 1 --2- 172.21.15.52:0/3380538897 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fb5180a2c20 0x7fb5180a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:34.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.505+0000 7fb5349f2700 1 -- 172.21.15.52:0/3380538897 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7fb5200cf900 con 0x7fb530102530 2024-06-10T03:00:34.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.505+0000 7fb52e7fc700 1 --2- 172.21.15.52:0/3380538897 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fb5180a2c20 0x7fb5180a50e0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fb530079b40 tx=0x7fb524009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:00:34.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.506+0000 7fb540277b80 1 Processor -- start 2024-06-10T03:00:34.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.506+0000 7fb540277b80 1 -- start start 2024-06-10T03:00:34.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.506+0000 7fb540277b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fc39a88370 0x55fc39a88750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:34.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.506+0000 7fb540277b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fc39a88c90 0x55fc39a82670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:34.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.506+0000 7fb540277b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fc39a82bb0 0x55fc39a82ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:34.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.506+0000 7fb540277b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55fc39970f60 con 0x55fc39a88c90 2024-06-10T03:00:34.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.506+0000 7fb540277b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55fc399cf850 con 0x55fc39a82bb0 2024-06-10T03:00:34.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.506+0000 7fb540277b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55fc39937460 con 0x55fc39a88370 2024-06-10T03:00:34.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.506+0000 7fb52e7fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fc39a88c90 0x55fc39a82670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:34.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.506+0000 7fb52effd700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fc39a88370 0x55fc39a88750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:34.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.506+0000 7fb52f7fe700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fc39a82bb0 0x55fc39a82ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:34.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.506+0000 7fb52e7fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fc39a88c90 0x55fc39a82670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:42770/0 (socket says 172.21.15.52:42770) 2024-06-10T03:00:34.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.506+0000 7fb52effd700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fc39a88370 0x55fc39a88750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.52:34044/0 (socket says 172.21.15.52:34044) 2024-06-10T03:00:34.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.506+0000 7fb52e7fc700 1 -- 172.21.15.52:0/1593157717 learned_addr learned my addr 172.21.15.52:0/1593157717 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:00:34.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.506+0000 7fb52f7fe700 1 -- 172.21.15.52:0/1593157717 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fc39a88370 msgr2=0x55fc39a88750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:00:34.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.506+0000 7fb52f7fe700 1 --2- 172.21.15.52:0/1593157717 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fc39a88370 0x55fc39a88750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:34.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.506+0000 7fb52f7fe700 1 -- 172.21.15.52:0/1593157717 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fc39a88c90 msgr2=0x55fc39a82670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:00:34.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.506+0000 7fb52f7fe700 1 --2- 172.21.15.52:0/1593157717 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fc39a88c90 0x55fc39a82670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:34.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.506+0000 7fb52f7fe700 1 -- 172.21.15.52:0/1593157717 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc3991e8d0 con 0x55fc39a82bb0 2024-06-10T03:00:34.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.507+0000 7fb52f7fe700 1 --2- 172.21.15.52:0/1593157717 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fc39a82bb0 0x55fc39a82ff0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fb52800d010 tx=0x7fb528009db0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d6a9d897f6f1d09e server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:00:34.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.507+0000 7fb515ffb700 1 -- 172.21.15.52:0/1593157717 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb528020030 con 0x55fc39a82bb0 2024-06-10T03:00:34.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.507+0000 7fb515ffb700 1 -- 172.21.15.52:0/1593157717 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb528009460 con 0x55fc39a82bb0 2024-06-10T03:00:34.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.507+0000 7fb515ffb700 1 -- 172.21.15.52:0/1593157717 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb528007980 con 0x55fc39a82bb0 2024-06-10T03:00:34.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.508+0000 7fb540277b80 1 -- 172.21.15.52:0/1593157717 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fc39a82bb0 msgr2=0x55fc39a82ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:00:34.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.508+0000 7fb540277b80 1 --2- 172.21.15.52:0/1593157717 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fc39a82bb0 0x55fc39a82ff0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fb52800d010 tx=0x7fb528009db0 comp rx=0 tx=0).stop 2024-06-10T03:00:34.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.508+0000 7fb540277b80 1 -- 172.21.15.52:0/1593157717 shutdown_connections 2024-06-10T03:00:34.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.508+0000 7fb540277b80 1 --2- 172.21.15.52:0/1593157717 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fc39a82bb0 0x55fc39a82ff0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:34.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.508+0000 7fb540277b80 1 --2- 172.21.15.52:0/1593157717 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fc39a88370 0x55fc39a88750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:34.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.508+0000 7fb540277b80 1 --2- 172.21.15.52:0/1593157717 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fc39a88c90 0x55fc39a82670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:34.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.508+0000 7fb540277b80 1 -- 172.21.15.52:0/1593157717 >> 172.21.15.52:0/1593157717 conn(0x55fc399ab080 msgr2=0x55fc39a79220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:00:34.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.508+0000 7fb540277b80 1 -- 172.21.15.52:0/1593157717 shutdown_connections 2024-06-10T03:00:34.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.508+0000 7fb540277b80 1 -- 172.21.15.52:0/1593157717 wait complete. 2024-06-10T03:00:34.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.508+0000 7fb540277b80 1 -- start start 2024-06-10T03:00:34.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.509+0000 7fb540277b80 1 Processor -- start 2024-06-10T03:00:34.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.509+0000 7fb540277b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fc39a82bb0 0x55fc39a8e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:34.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.509+0000 7fb540277b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fc39a88370 0x55fc39a8bc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:34.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.509+0000 7fb540277b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fc39a88c90 0x55fc39a8c180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:34.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.509+0000 7fb540277b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55fc3996ffc0 con 0x55fc39a82bb0 2024-06-10T03:00:34.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.509+0000 7fb540277b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55fc399cb810 con 0x55fc39a88c90 2024-06-10T03:00:34.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.509+0000 7fb540277b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55fc39a8f620 con 0x55fc39a88370 2024-06-10T03:00:34.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.509+0000 7fb540277b80 10 client.0 _ll_register_callbacks cb 0x55fc399655b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T03:00:34.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.509+0000 7fb52e7fc700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fc39a88370 0x55fc39a8bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:34.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.509+0000 7fb52effd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fc39a82bb0 0x55fc39a8e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:34.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.509+0000 7fb52effd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fc39a82bb0 0x55fc39a8e940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:42784/0 (socket says 172.21.15.52:42784) 2024-06-10T03:00:34.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.509+0000 7fb52effd700 1 -- 172.21.15.52:0/1706116916 learned_addr learned my addr 172.21.15.52:0/1706116916 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:00:34.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.509+0000 7fb52f7fe700 1 --2- 172.21.15.52:0/1706116916 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fc39a88c90 0x55fc39a8c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:34.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.510+0000 7fb52effd700 1 -- 172.21.15.52:0/1706116916 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fc39a88370 msgr2=0x55fc39a8bc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:00:34.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.510+0000 7fb52effd700 1 --2- 172.21.15.52:0/1706116916 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fc39a88370 0x55fc39a8bc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:34.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.510+0000 7fb52effd700 1 -- 172.21.15.52:0/1706116916 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fc39a88c90 msgr2=0x55fc39a8c180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:00:34.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.510+0000 7fb52effd700 1 --2- 172.21.15.52:0/1706116916 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fc39a88c90 0x55fc39a8c180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:34.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.510+0000 7fb52effd700 1 -- 172.21.15.52:0/1706116916 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc39995750 con 0x55fc39a82bb0 2024-06-10T03:00:34.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.510+0000 7fb52effd700 1 --2- 172.21.15.52:0/1706116916 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fc39a82bb0 0x55fc39a8e940 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fb5200051e0 tx=0x7fb52009b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27a225ba6e2e035d server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:00:34.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.510+0000 7fb50effd700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-10T03:00:34.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.510+0000 7fb50effd700 1 -- 172.21.15.52:0/1706116916 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5200b9060 con 0x55fc39a82bb0 2024-06-10T03:00:34.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.510+0000 7fb50effd700 1 -- 172.21.15.52:0/1706116916 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7fb52009c010 con 0x55fc39a82bb0 2024-06-10T03:00:34.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.510+0000 7fb540277b80 1 -- 172.21.15.52:0/1706116916 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb528009460 con 0x55fc39a82bb0 2024-06-10T03:00:34.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.510+0000 7fb50effd700 1 -- 172.21.15.52:0/1706116916 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fb52009bdb0 con 0x55fc39a82bb0 2024-06-10T03:00:34.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.510+0000 7fb540277b80 10 client.16416 fetch_fsmap learned FSMap version 36 2024-06-10T03:00:34.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.510+0000 7fb540277b80 1 -- 172.21.15.52:0/1706116916 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55fc39a99d50 con 0x55fc39a82bb0 2024-06-10T03:00:34.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.511+0000 7fb50effd700 1 -- 172.21.15.52:0/1706116916 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb528009460 con 0x55fc39a82bb0 2024-06-10T03:00:34.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.511+0000 7fb50effd700 1 -- 172.21.15.52:0/1706116916 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 36) ==== 8378+0+0 (secure 0 0 0) 0x7fb52009b200 con 0x55fc39a82bb0 2024-06-10T03:00:34.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.511+0000 7fb540277b80 10 client.16416 fetch_fsmap finished waiting for FSMap version 36 2024-06-10T03:00:34.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.511+0000 7fb540277b80 10 client.16416 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-06-10T03:00:34.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.511+0000 7fb540277b80 20 client.16416 populate_metadata read hostname 'smithi052' 2024-06-10T03:00:34.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.511+0000 7fb540277b80 1 --2- 172.21.15.52:0/1706116916 >> [v2:172.21.15.188:6834/3904544259,v1:172.21.15.188:6835/3904544259] conn(0x55fc39a9c390 0x55fc39a9e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:34.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.511+0000 7fb540277b80 4 client.16416 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T03:00:34.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.511+0000 7fb540277b80 1 -- 172.21.15.52:0/1706116916 --> [v2:172.21.15.188:6834/3904544259,v1:172.21.15.188:6835/3904544259] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fc399dcd50 con 0x55fc39a9c390 2024-06-10T03:00:34.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.511+0000 7fb52e7fc700 1 --2- 172.21.15.52:0/1706116916 >> [v2:172.21.15.188:6834/3904544259,v1:172.21.15.188:6835/3904544259] conn(0x55fc39a9c390 0x55fc39a9e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:34.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.512+0000 7fb52e7fc700 1 --2- 172.21.15.52:0/1706116916 >> [v2:172.21.15.188:6834/3904544259,v1:172.21.15.188:6835/3904544259] conn(0x55fc39a9c390 0x55fc39a9e7a0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4bb4dcd76f4b710c server_cookie=cbbfc309863f9ebb in_seq=0 out_seq=0 2024-06-10T03:00:34.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.512+0000 7fb50effd700 10 client.16416 ms_handle_connect on v2:172.21.15.188:6834/3904544259 2024-06-10T03:00:34.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.514+0000 7fb50effd700 1 -- 172.21.15.52:0/1706116916 <== mds.2 v2:172.21.15.188:6834/3904544259 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55fc399dcd50 con 0x55fc39a9c390 2024-06-10T03:00:34.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.514+0000 7fb50effd700 10 client.16416 handle_command_reply: tid=0 2024-06-10T03:00:34.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.514+0000 7fb540277b80 1 client.16416 shutdown 2024-06-10T03:00:34.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.514+0000 7fb540277b80 1 -- 172.21.15.52:0/1706116916 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fc39a82bb0 msgr2=0x55fc39a8e940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:00:34.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.514+0000 7fb540277b80 1 --2- 172.21.15.52:0/1706116916 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fc39a82bb0 0x55fc39a8e940 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fb5200051e0 tx=0x7fb52009b8f0 comp rx=0 tx=0).stop 2024-06-10T03:00:34.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.514+0000 7fb540277b80 1 -- 172.21.15.52:0/1706116916 shutdown_connections 2024-06-10T03:00:34.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.514+0000 7fb540277b80 1 --2- 172.21.15.52:0/1706116916 >> [v2:172.21.15.188:6834/3904544259,v1:172.21.15.188:6835/3904544259] conn(0x55fc39a9c390 0x55fc39a9e7a0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:34.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.514+0000 7fb540277b80 1 --2- 172.21.15.52:0/1706116916 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fc39a88c90 0x55fc39a8c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:34.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.514+0000 7fb540277b80 1 --2- 172.21.15.52:0/1706116916 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fc39a82bb0 0x55fc39a8e940 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:34.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.514+0000 7fb540277b80 1 --2- 172.21.15.52:0/1706116916 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fc39a88370 0x55fc39a8bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:34.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.514+0000 7fb50effd700 0 client.16416 ms_handle_reset on v2:172.21.15.188:6834/3904544259 2024-06-10T03:00:34.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.514+0000 7fb540277b80 1 -- 172.21.15.52:0/1706116916 >> 172.21.15.52:0/1706116916 conn(0x55fc399ab080 msgr2=0x55fc39a78700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:00:34.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.514+0000 7fb540277b80 1 -- 172.21.15.52:0/1706116916 shutdown_connections 2024-06-10T03:00:34.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.514+0000 7fb540277b80 1 -- 172.21.15.52:0/1706116916 wait complete. 2024-06-10T03:00:34.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.514+0000 7fb540277b80 20 client.16416 trim_cache size 0 max 16384 2024-06-10T03:00:34.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.530+0000 7fb540277b80 1 Processor -- start 2024-06-10T03:00:34.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.531+0000 7fb540277b80 1 -- start start 2024-06-10T03:00:34.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.531+0000 7fb540277b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fc39a88370 0x55fc39a89780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:34.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.531+0000 7fb540277b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fc39a88c90 0x55fc39a78e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:34.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.531+0000 7fb540277b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fc39a87db0 0x55fc39a793c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:34.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.531+0000 7fb540277b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb5301abfe0 con 0x55fc39a87db0 2024-06-10T03:00:34.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.531+0000 7fb540277b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fb5301adbe0 con 0x55fc39a88c90 2024-06-10T03:00:34.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.531+0000 7fb540277b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fb5301ac6b0 con 0x55fc39a88370 2024-06-10T03:00:34.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.531+0000 7fb52f7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fc39a87db0 0x55fc39a793c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:34.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.531+0000 7fb52f7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fc39a87db0 0x55fc39a793c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:42790/0 (socket says 172.21.15.52:42790) 2024-06-10T03:00:34.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.531+0000 7fb52f7fe700 1 -- 172.21.15.52:0/1610098358 learned_addr learned my addr 172.21.15.52:0/1610098358 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:00:34.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.531+0000 7fb52effd700 1 --2- 172.21.15.52:0/1610098358 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fc39a88370 0x55fc39a89780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:34.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.531+0000 7fb52e7fc700 1 --2- 172.21.15.52:0/1610098358 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fc39a88c90 0x55fc39a78e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:34.533 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.531+0000 7fb52f7fe700 1 -- 172.21.15.52:0/1610098358 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fc39a88370 msgr2=0x55fc39a89780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:00:34.533 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.531+0000 7fb52f7fe700 1 --2- 172.21.15.52:0/1610098358 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fc39a88370 0x55fc39a89780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:34.533 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.531+0000 7fb52f7fe700 1 -- 172.21.15.52:0/1610098358 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fc39a88c90 msgr2=0x55fc39a78e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:00:34.533 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.531+0000 7fb52f7fe700 1 --2- 172.21.15.52:0/1610098358 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fc39a88c90 0x55fc39a78e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:34.533 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.531+0000 7fb52f7fe700 1 -- 172.21.15.52:0/1610098358 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc39a8f120 con 0x55fc39a87db0 2024-06-10T03:00:34.533 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.531+0000 7fb52e7fc700 1 --2- 172.21.15.52:0/1610098358 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fc39a88c90 0x55fc39a78e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T03:00:34.533 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.531+0000 7fb52f7fe700 1 --2- 172.21.15.52:0/1610098358 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fc39a87db0 0x55fc39a793c0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fb52800b0d0 tx=0x7fb52800b7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a117754d91d498d server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:00:34.533 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.532+0000 7fb5157fa700 1 -- 172.21.15.52:0/1610098358 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb528020040 con 0x55fc39a87db0 2024-06-10T03:00:34.533 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.532+0000 7fb5157fa700 1 -- 172.21.15.52:0/1610098358 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb52800bd20 con 0x55fc39a87db0 2024-06-10T03:00:34.533 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.532+0000 7fb5157fa700 1 -- 172.21.15.52:0/1610098358 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb52801c440 con 0x55fc39a87db0 2024-06-10T03:00:34.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.532+0000 7fb540277b80 1 -- 172.21.15.52:0/1610098358 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fc39a87db0 msgr2=0x55fc39a793c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:00:34.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.532+0000 7fb540277b80 1 --2- 172.21.15.52:0/1610098358 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fc39a87db0 0x55fc39a793c0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fb52800b0d0 tx=0x7fb52800b7a0 comp rx=0 tx=0).stop 2024-06-10T03:00:34.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.532+0000 7fb540277b80 1 -- 172.21.15.52:0/1610098358 shutdown_connections 2024-06-10T03:00:34.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.532+0000 7fb540277b80 1 --2- 172.21.15.52:0/1610098358 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fc39a87db0 0x55fc39a793c0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:34.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.532+0000 7fb540277b80 1 --2- 172.21.15.52:0/1610098358 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fc39a88370 0x55fc39a89780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:34.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.532+0000 7fb540277b80 1 --2- 172.21.15.52:0/1610098358 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fc39a88c90 0x55fc39a78e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:34.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.532+0000 7fb540277b80 1 -- 172.21.15.52:0/1610098358 >> 172.21.15.52:0/1610098358 conn(0x55fc399ab080 msgr2=0x55fc39a89070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:00:34.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.533+0000 7fb540277b80 1 -- 172.21.15.52:0/1610098358 shutdown_connections 2024-06-10T03:00:34.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.533+0000 7fb540277b80 1 -- 172.21.15.52:0/1610098358 wait complete. 2024-06-10T03:00:34.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.533+0000 7fb540277b80 1 -- start start 2024-06-10T03:00:34.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.533+0000 7fb540277b80 1 Processor -- start 2024-06-10T03:00:34.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.533+0000 7fb540277b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fc39a87db0 0x55fc39a95000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:34.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.533+0000 7fb540277b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fc39a88370 0x55fc39aa39c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:34.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.533+0000 7fb540277b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fc39a88c90 0x55fc39aa3f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:34.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.533+0000 7fb540277b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb5301ad510 con 0x55fc39a88370 2024-06-10T03:00:34.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.533+0000 7fb540277b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fb5301ace40 con 0x55fc39a88c90 2024-06-10T03:00:34.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.533+0000 7fb540277b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fb5301a9f80 con 0x55fc39a87db0 2024-06-10T03:00:34.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.533+0000 7fb540277b80 10 client.0 _ll_register_callbacks cb 0x55fc399655b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T03:00:34.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.533+0000 7fb52e7fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fc39a88370 0x55fc39aa39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:34.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.533+0000 7fb52e7fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fc39a88370 0x55fc39aa39c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:42802/0 (socket says 172.21.15.52:42802) 2024-06-10T03:00:34.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.534+0000 7fb52effd700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fc39a87db0 0x55fc39a95000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:34.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.534+0000 7fb52e7fc700 1 -- 172.21.15.52:0/2257162761 learned_addr learned my addr 172.21.15.52:0/2257162761 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:00:34.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.534+0000 7fb52f7fe700 1 --2- 172.21.15.52:0/2257162761 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fc39a88c90 0x55fc39aa3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:34.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.534+0000 7fb52e7fc700 1 -- 172.21.15.52:0/2257162761 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fc39a87db0 msgr2=0x55fc39a95000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:00:34.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.534+0000 7fb52e7fc700 1 --2- 172.21.15.52:0/2257162761 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fc39a87db0 0x55fc39a95000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:34.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.534+0000 7fb52e7fc700 1 -- 172.21.15.52:0/2257162761 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fc39a88c90 msgr2=0x55fc39aa3f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:00:34.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.534+0000 7fb52e7fc700 1 --2- 172.21.15.52:0/2257162761 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fc39a88c90 0x55fc39aa3f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:34.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.534+0000 7fb52e7fc700 1 -- 172.21.15.52:0/2257162761 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc39aa46f0 con 0x55fc39a88370 2024-06-10T03:00:34.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.534+0000 7fb52e7fc700 1 --2- 172.21.15.52:0/2257162761 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fc39a88370 0x55fc39aa39c0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fb524011a50 tx=0x7fb5240047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f572fafcc9afd20a server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:00:34.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.534+0000 7fb50effd700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-10T03:00:34.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.534+0000 7fb50effd700 1 -- 172.21.15.52:0/2257162761 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb52402d070 con 0x55fc39a88370 2024-06-10T03:00:34.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.534+0000 7fb50effd700 1 -- 172.21.15.52:0/2257162761 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7fb524003b50 con 0x55fc39a88370 2024-06-10T03:00:34.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.534+0000 7fb540277b80 1 -- 172.21.15.52:0/2257162761 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb52800bd20 con 0x55fc39a88370 2024-06-10T03:00:34.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.534+0000 7fb50effd700 1 -- 172.21.15.52:0/2257162761 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fb524039050 con 0x55fc39a88370 2024-06-10T03:00:34.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.535+0000 7fb540277b80 10 client.16422 fetch_fsmap learned FSMap version 36 2024-06-10T03:00:34.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.535+0000 7fb540277b80 1 -- 172.21.15.52:0/2257162761 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55fc39b07100 con 0x55fc39a88370 2024-06-10T03:00:34.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.535+0000 7fb50effd700 1 -- 172.21.15.52:0/2257162761 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb52800bd20 con 0x55fc39a88370 2024-06-10T03:00:34.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.535+0000 7fb50effd700 1 -- 172.21.15.52:0/2257162761 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 36) ==== 8378+0+0 (secure 0 0 0) 0x7fb5240292c0 con 0x55fc39a88370 2024-06-10T03:00:34.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.535+0000 7fb540277b80 10 client.16422 fetch_fsmap finished waiting for FSMap version 36 2024-06-10T03:00:34.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.535+0000 7fb540277b80 10 client.16422 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-06-10T03:00:34.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.535+0000 7fb540277b80 20 client.16422 populate_metadata read hostname 'smithi052' 2024-06-10T03:00:34.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.535+0000 7fb540277b80 1 --2- 172.21.15.52:0/2257162761 >> [v2:172.21.15.188:6834/3904544259,v1:172.21.15.188:6835/3904544259] conn(0x55fc39b078b0 0x55fc39b09cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:34.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.535+0000 7fb540277b80 4 client.16422 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T03:00:34.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.535+0000 7fb540277b80 1 -- 172.21.15.52:0/2257162761 --> [v2:172.21.15.188:6834/3904544259,v1:172.21.15.188:6835/3904544259] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fc399d72d0 con 0x55fc39b078b0 2024-06-10T03:00:34.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.535+0000 7fb52effd700 1 --2- 172.21.15.52:0/2257162761 >> [v2:172.21.15.188:6834/3904544259,v1:172.21.15.188:6835/3904544259] conn(0x55fc39b078b0 0x55fc39b09cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:34.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.536+0000 7fb52effd700 1 --2- 172.21.15.52:0/2257162761 >> [v2:172.21.15.188:6834/3904544259,v1:172.21.15.188:6835/3904544259] conn(0x55fc39b078b0 0x55fc39b09cd0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8d440a5e26fb759e server_cookie=45ebc6e05ced6b11 in_seq=0 out_seq=0 2024-06-10T03:00:34.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.536+0000 7fb50effd700 10 client.16422 ms_handle_connect on v2:172.21.15.188:6834/3904544259 2024-06-10T03:00:34.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.537+0000 7fb50effd700 1 -- 172.21.15.52:0/2257162761 <== mds.2 v2:172.21.15.188:6834/3904544259 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fc399d72d0 con 0x55fc39b078b0 2024-06-10T03:00:34.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.537+0000 7fb50effd700 10 client.16422 handle_command_reply: tid=0 2024-06-10T03:00:34.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.537+0000 7fb540277b80 1 client.16422 shutdown 2024-06-10T03:00:34.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.538+0000 7fb540277b80 1 -- 172.21.15.52:0/2257162761 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fc39a88370 msgr2=0x55fc39aa39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:00:34.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.538+0000 7fb540277b80 1 --2- 172.21.15.52:0/2257162761 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fc39a88370 0x55fc39aa39c0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fb524011a50 tx=0x7fb5240047c0 comp rx=0 tx=0).stop 2024-06-10T03:00:34.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.538+0000 7fb540277b80 1 -- 172.21.15.52:0/2257162761 shutdown_connections 2024-06-10T03:00:34.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.538+0000 7fb540277b80 1 --2- 172.21.15.52:0/2257162761 >> [v2:172.21.15.188:6834/3904544259,v1:172.21.15.188:6835/3904544259] conn(0x55fc39b078b0 0x55fc39b09cd0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:34.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.538+0000 7fb540277b80 1 --2- 172.21.15.52:0/2257162761 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fc39a88c90 0x55fc39aa3f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:34.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.538+0000 7fb540277b80 1 --2- 172.21.15.52:0/2257162761 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fc39a87db0 0x55fc39a95000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:34.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.538+0000 7fb540277b80 1 --2- 172.21.15.52:0/2257162761 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fc39a88370 0x55fc39aa39c0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:34.543 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.538+0000 7fb540277b80 1 -- 172.21.15.52:0/2257162761 >> 172.21.15.52:0/2257162761 conn(0x55fc399ab080 msgr2=0x55fc39a8d470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:00:34.543 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.538+0000 7fb50effd700 0 client.16422 ms_handle_reset on v2:172.21.15.188:6834/3904544259 2024-06-10T03:00:34.543 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.538+0000 7fb540277b80 1 -- 172.21.15.52:0/2257162761 shutdown_connections 2024-06-10T03:00:34.543 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.538+0000 7fb540277b80 1 -- 172.21.15.52:0/2257162761 wait complete. 2024-06-10T03:00:34.543 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.538+0000 7fb540277b80 20 client.16422 trim_cache size 0 max 16384 2024-06-10T03:00:34.543 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-06-10T03:00:34.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.538+0000 7fb5359f4700 1 -- 172.21.15.52:0/3380538897 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fb5180a2c20 msgr2=0x7fb5180a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:00:34.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.538+0000 7fb5359f4700 1 --2- 172.21.15.52:0/3380538897 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fb5180a2c20 0x7fb5180a50e0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fb530079b40 tx=0x7fb524009260 comp rx=0 tx=0).stop 2024-06-10T03:00:34.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.538+0000 7fb5359f4700 1 -- 172.21.15.52:0/3380538897 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb530102530 msgr2=0x7fb53019e810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:00:34.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.538+0000 7fb5359f4700 1 --2- 172.21.15.52:0/3380538897 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb530102530 0x7fb53019e810 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fb52000e9c0 tx=0x7fb52000ed80 comp rx=0 tx=0).stop 2024-06-10T03:00:34.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.539+0000 7fb5359f4700 1 -- 172.21.15.52:0/3380538897 shutdown_connections 2024-06-10T03:00:34.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.539+0000 7fb5359f4700 1 --2- 172.21.15.52:0/3380538897 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fb5180a2c20 0x7fb5180a50e0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:34.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.539+0000 7fb5359f4700 1 --2- 172.21.15.52:0/3380538897 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb5301085d0 0x7fb5301989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:34.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.539+0000 7fb5359f4700 1 --2- 172.21.15.52:0/3380538897 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb530102530 0x7fb53019e810 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:34.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.539+0000 7fb5359f4700 1 --2- 172.21.15.52:0/3380538897 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb530102ef0 0x7fb53019ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:34.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.539+0000 7fb5359f4700 1 -- 172.21.15.52:0/3380538897 >> 172.21.15.52:0/3380538897 conn(0x7fb530079640 msgr2=0x7fb530100f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:00:34.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.539+0000 7fb5359f4700 1 -- 172.21.15.52:0/3380538897 shutdown_connections 2024-06-10T03:00:34.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:34.539+0000 7fb5359f4700 1 -- 172.21.15.52:0/3380538897 wait complete. 2024-06-10T03:00:34.638 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:00:34 smithi120 ceph-mon[36634]: pgmap v625: 129 pgs: 129 active+clean; 562 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.0 MiB/s wr, 242 op/s 2024-06-10T03:00:34.703 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:00:34 smithi188 ceph-mon[36599]: pgmap v625: 129 pgs: 129 active+clean; 562 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.0 MiB/s wr, 242 op/s 2024-06-10T03:00:35.216 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-06-10T03:00:35.490 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T03:00:36.550 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:00:36 smithi052 ceph-mon[32362]: pgmap v626: 129 pgs: 129 active+clean; 567 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.4 MiB/s wr, 334 op/s 2024-06-10T03:00:36.638 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:00:36 smithi120 ceph-mon[36634]: pgmap v626: 129 pgs: 129 active+clean; 567 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.4 MiB/s wr, 334 op/s 2024-06-10T03:00:36.703 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:00:36 smithi188 ceph-mon[36599]: pgmap v626: 129 pgs: 129 active+clean; 567 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.4 MiB/s wr, 334 op/s 2024-06-10T03:00:37.250 INFO:tasks.workunit.client.0.smithi052.stdout: 98 8190 0 6348 0 5591 0 2024-06-10T03:00:37.338 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.338+0000 7f8ee88fa700 1 -- 172.21.15.52:0/1088092749 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8ee4108520 msgr2=0x7f8ee4108900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:00:37.338 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.338+0000 7f8ee88fa700 1 --2- 172.21.15.52:0/1088092749 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8ee4108520 0x7f8ee4108900 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f8ed4009990 tx=0x7f8ed4009ca0 comp rx=0 tx=0).stop 2024-06-10T03:00:37.338 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.338+0000 7f8ee88fa700 1 -- 172.21.15.52:0/1088092749 shutdown_connections 2024-06-10T03:00:37.339 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.338+0000 7f8ee88fa700 1 --2- 172.21.15.52:0/1088092749 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8ee4102ed0 0x7f8ee410f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:37.339 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.338+0000 7f8ee88fa700 1 --2- 172.21.15.52:0/1088092749 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8ee4108520 0x7f8ee4108900 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:37.339 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.338+0000 7f8ee88fa700 1 --2- 172.21.15.52:0/1088092749 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8ee4102510 0x7f8ee4102990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:37.339 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.338+0000 7f8ee88fa700 1 -- 172.21.15.52:0/1088092749 >> 172.21.15.52:0/1088092749 conn(0x7f8ee4079560 msgr2=0x7f8ee4079970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:00:37.339 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.338+0000 7f8ee88fa700 1 -- 172.21.15.52:0/1088092749 shutdown_connections 2024-06-10T03:00:37.339 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.338+0000 7f8ee88fa700 1 -- 172.21.15.52:0/1088092749 wait complete. 2024-06-10T03:00:37.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.339+0000 7f8ee88fa700 1 Processor -- start 2024-06-10T03:00:37.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.339+0000 7f8ee88fa700 1 -- start start 2024-06-10T03:00:37.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.340+0000 7f8ee88fa700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8ee4102510 0x7f8ee419e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:37.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.340+0000 7f8ee88fa700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8ee4102ed0 0x7f8ee419ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:37.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.340+0000 7f8ee88fa700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8ee4108520 0x7f8ee41989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:37.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.340+0000 7f8ee88fa700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f8ee4112800 con 0x7f8ee4102ed0 2024-06-10T03:00:37.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.340+0000 7f8ee88fa700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f8ee4112680 con 0x7f8ee4102510 2024-06-10T03:00:37.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.340+0000 7f8ee88fa700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f8ee4112980 con 0x7f8ee4108520 2024-06-10T03:00:37.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.340+0000 7f8ee1d9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8ee4102ed0 0x7f8ee419ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:37.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.340+0000 7f8ee2d9d700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8ee4108520 0x7f8ee41989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:37.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.340+0000 7f8ee1d9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8ee4102ed0 0x7f8ee419ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:42820/0 (socket says 172.21.15.52:42820) 2024-06-10T03:00:37.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.340+0000 7f8ee2d9d700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8ee4108520 0x7f8ee41989e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.52:34086/0 (socket says 172.21.15.52:34086) 2024-06-10T03:00:37.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.340+0000 7f8ee1d9b700 1 -- 172.21.15.52:0/3597064699 learned_addr learned my addr 172.21.15.52:0/3597064699 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:00:37.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.340+0000 7f8ee259c700 1 --2- 172.21.15.52:0/3597064699 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8ee4102510 0x7f8ee419e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:37.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.340+0000 7f8ee2d9d700 1 -- 172.21.15.52:0/3597064699 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8ee4102510 msgr2=0x7f8ee419e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:00:37.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.340+0000 7f8ee2d9d700 1 --2- 172.21.15.52:0/3597064699 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8ee4102510 0x7f8ee419e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:37.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.340+0000 7f8ee2d9d700 1 -- 172.21.15.52:0/3597064699 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8ee4102ed0 msgr2=0x7f8ee419ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:00:37.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.340+0000 7f8ee2d9d700 1 --2- 172.21.15.52:0/3597064699 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8ee4102ed0 0x7f8ee419ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:37.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.340+0000 7f8ee2d9d700 1 -- 172.21.15.52:0/3597064699 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ee4199140 con 0x7f8ee4108520 2024-06-10T03:00:37.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.340+0000 7f8ee1d9b700 1 --2- 172.21.15.52:0/3597064699 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8ee4102ed0 0x7f8ee419ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T03:00:37.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.341+0000 7f8ee2d9d700 1 --2- 172.21.15.52:0/3597064699 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8ee4108520 0x7f8ee41989e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f8edc00efc0 tx=0x7f8edc00c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:00:37.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.341+0000 7f8ed37fe700 1 -- 172.21.15.52:0/3597064699 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8edc0092e0 con 0x7f8ee4108520 2024-06-10T03:00:37.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.341+0000 7f8ed37fe700 1 -- 172.21.15.52:0/3597064699 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8edc004bb0 con 0x7f8ee4108520 2024-06-10T03:00:37.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.341+0000 7f8ee88fa700 1 -- 172.21.15.52:0/3597064699 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ee41993d0 con 0x7f8ee4108520 2024-06-10T03:00:37.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.341+0000 7f8ed37fe700 1 -- 172.21.15.52:0/3597064699 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8edc004280 con 0x7f8ee4108520 2024-06-10T03:00:37.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.341+0000 7f8ee88fa700 1 -- 172.21.15.52:0/3597064699 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ee40746c0 con 0x7f8ee4108520 2024-06-10T03:00:37.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.344+0000 7f8ed37fe700 1 -- 172.21.15.52:0/3597064699 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f8edc01b030 con 0x7f8ee4108520 2024-06-10T03:00:37.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.344+0000 7f8ed37fe700 1 --2- 172.21.15.52:0/3597064699 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f8ecc0a2c20 0x7f8ecc0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:37.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.344+0000 7f8ee259c700 1 --2- 172.21.15.52:0/3597064699 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f8ecc0a2c20 0x7f8ecc0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:37.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.344+0000 7f8ed37fe700 1 -- 172.21.15.52:0/3597064699 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f8edc0cef60 con 0x7f8ee4108520 2024-06-10T03:00:37.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.345+0000 7f8ef317db80 1 Processor -- start 2024-06-10T03:00:37.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.345+0000 7f8ef317db80 1 -- start start 2024-06-10T03:00:37.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.345+0000 7f8ee259c700 1 --2- 172.21.15.52:0/3597064699 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f8ecc0a2c20 0x7f8ecc0a50e0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f8ee4199b20 tx=0x7f8ed4005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:00:37.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.345+0000 7f8ef317db80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55da51089f60 0x55da5108a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:37.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.345+0000 7f8ef317db80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55da5108a880 0x55da51084260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:37.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.345+0000 7f8ef317db80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55da510847a0 0x55da51084bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:37.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.345+0000 7f8ef317db80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55da50f76f60 con 0x55da5108a880 2024-06-10T03:00:37.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.345+0000 7f8ef317db80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55da50fd5850 con 0x55da51089f60 2024-06-10T03:00:37.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.345+0000 7f8ef317db80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55da50f3d460 con 0x55da510847a0 2024-06-10T03:00:37.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.345+0000 7f8ee1d9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55da5108a880 0x55da51084260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:37.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.345+0000 7f8ee1d9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55da5108a880 0x55da51084260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:42830/0 (socket says 172.21.15.52:42830) 2024-06-10T03:00:37.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.345+0000 7f8ee1d9b700 1 -- 172.21.15.52:0/371172309 learned_addr learned my addr 172.21.15.52:0/371172309 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:00:37.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.345+0000 7f8ee259c700 1 --2- 172.21.15.52:0/371172309 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55da51089f60 0x55da5108a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:37.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.345+0000 7f8ee2d9d700 1 --2- 172.21.15.52:0/371172309 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55da510847a0 0x55da51084bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:37.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.346+0000 7f8ee2d9d700 1 -- 172.21.15.52:0/371172309 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55da51089f60 msgr2=0x55da5108a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:00:37.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.346+0000 7f8ee2d9d700 1 --2- 172.21.15.52:0/371172309 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55da51089f60 0x55da5108a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:37.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.346+0000 7f8ee2d9d700 1 -- 172.21.15.52:0/371172309 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55da5108a880 msgr2=0x55da51084260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:00:37.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.346+0000 7f8ee2d9d700 1 --2- 172.21.15.52:0/371172309 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55da5108a880 0x55da51084260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:37.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.346+0000 7f8ee2d9d700 1 -- 172.21.15.52:0/371172309 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da50f248d0 con 0x55da510847a0 2024-06-10T03:00:37.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.346+0000 7f8ee1d9b700 1 --2- 172.21.15.52:0/371172309 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55da5108a880 0x55da51084260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T03:00:37.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.346+0000 7f8ee259c700 1 --2- 172.21.15.52:0/371172309 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55da51089f60 0x55da5108a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T03:00:37.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.346+0000 7f8ee2d9d700 1 --2- 172.21.15.52:0/371172309 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55da510847a0 0x55da51084bc0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f8edc0cc650 tx=0x7f8edc09af80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9e1faa0f692f84b8 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:00:37.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.346+0000 7f8ed0ff9700 1 -- 172.21.15.52:0/371172309 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8edc09bb40 con 0x55da510847a0 2024-06-10T03:00:37.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.346+0000 7f8ed0ff9700 1 -- 172.21.15.52:0/371172309 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8edc09b440 con 0x55da510847a0 2024-06-10T03:00:37.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.346+0000 7f8ed0ff9700 1 -- 172.21.15.52:0/371172309 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8edc0a3b90 con 0x55da510847a0 2024-06-10T03:00:37.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.347+0000 7f8ef317db80 1 -- 172.21.15.52:0/371172309 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55da510847a0 msgr2=0x55da51084bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:00:37.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.347+0000 7f8ef317db80 1 --2- 172.21.15.52:0/371172309 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55da510847a0 0x55da51084bc0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f8edc0cc650 tx=0x7f8edc09af80 comp rx=0 tx=0).stop 2024-06-10T03:00:37.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.347+0000 7f8ef317db80 1 -- 172.21.15.52:0/371172309 shutdown_connections 2024-06-10T03:00:37.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.347+0000 7f8ef317db80 1 --2- 172.21.15.52:0/371172309 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55da510847a0 0x55da51084bc0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:37.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.347+0000 7f8ef317db80 1 --2- 172.21.15.52:0/371172309 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55da51089f60 0x55da5108a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:37.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.347+0000 7f8ef317db80 1 --2- 172.21.15.52:0/371172309 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55da5108a880 0x55da51084260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:37.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.347+0000 7f8ef317db80 1 -- 172.21.15.52:0/371172309 >> 172.21.15.52:0/371172309 conn(0x55da50fb1080 msgr2=0x55da5107f220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:00:37.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.347+0000 7f8ef317db80 1 -- 172.21.15.52:0/371172309 shutdown_connections 2024-06-10T03:00:37.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.347+0000 7f8ef317db80 1 -- 172.21.15.52:0/371172309 wait complete. 2024-06-10T03:00:37.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.347+0000 7f8ef317db80 1 -- start start 2024-06-10T03:00:37.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.347+0000 7f8ef317db80 1 Processor -- start 2024-06-10T03:00:37.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.348+0000 7f8ef317db80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55da510847a0 0x55da51090630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:37.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.348+0000 7f8ef317db80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55da51089f60 0x55da5108d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:37.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.348+0000 7f8ef317db80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55da5108a880 0x55da5108de70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:37.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.348+0000 7f8ef317db80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55da50f75fc0 con 0x55da5108a880 2024-06-10T03:00:37.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.348+0000 7f8ef317db80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55da50fd1810 con 0x55da51089f60 2024-06-10T03:00:37.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.348+0000 7f8ef317db80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55da51091200 con 0x55da510847a0 2024-06-10T03:00:37.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.348+0000 7f8ef317db80 10 client.0 _ll_register_callbacks cb 0x55da50f6b5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T03:00:37.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.348+0000 7f8ee2d9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55da5108a880 0x55da5108de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:37.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.348+0000 7f8ee2d9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55da5108a880 0x55da5108de70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:42836/0 (socket says 172.21.15.52:42836) 2024-06-10T03:00:37.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.348+0000 7f8ee259c700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55da510847a0 0x55da51090630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:37.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.348+0000 7f8ee2d9d700 1 -- 172.21.15.52:0/1188299181 learned_addr learned my addr 172.21.15.52:0/1188299181 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:00:37.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.348+0000 7f8ee1d9b700 1 --2- 172.21.15.52:0/1188299181 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55da51089f60 0x55da5108d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:37.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.348+0000 7f8ee2d9d700 1 -- 172.21.15.52:0/1188299181 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55da510847a0 msgr2=0x55da51090630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:00:37.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.348+0000 7f8ee2d9d700 1 --2- 172.21.15.52:0/1188299181 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55da510847a0 0x55da51090630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:37.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.348+0000 7f8ee2d9d700 1 -- 172.21.15.52:0/1188299181 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55da51089f60 msgr2=0x55da5108d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:00:37.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.348+0000 7f8ee2d9d700 1 --2- 172.21.15.52:0/1188299181 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55da51089f60 0x55da5108d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:37.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.348+0000 7f8ee2d9d700 1 -- 172.21.15.52:0/1188299181 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da50f9b750 con 0x55da5108a880 2024-06-10T03:00:37.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.348+0000 7f8ee2d9d700 1 --2- 172.21.15.52:0/1188299181 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55da5108a880 0x55da5108de70 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f8edc095fd0 tx=0x7f8edc004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30bca11c9cf8dd07 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:00:37.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.348+0000 7f8ec9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-10T03:00:37.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.349+0000 7f8ec9ffb700 1 -- 172.21.15.52:0/1188299181 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8edc09bb40 con 0x55da5108a880 2024-06-10T03:00:37.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.349+0000 7f8ec9ffb700 1 -- 172.21.15.52:0/1188299181 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f8edc0994e0 con 0x55da5108a880 2024-06-10T03:00:37.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.349+0000 7f8ef317db80 1 -- 172.21.15.52:0/1188299181 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8edc09b440 con 0x55da5108a880 2024-06-10T03:00:37.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.349+0000 7f8ec9ffb700 1 -- 172.21.15.52:0/1188299181 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f8edc0b42a0 con 0x55da5108a880 2024-06-10T03:00:37.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.349+0000 7f8ef317db80 10 client.16434 fetch_fsmap learned FSMap version 36 2024-06-10T03:00:37.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.349+0000 7f8ef317db80 1 -- 172.21.15.52:0/1188299181 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55da5109b990 con 0x55da5108a880 2024-06-10T03:00:37.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.349+0000 7f8ec9ffb700 1 -- 172.21.15.52:0/1188299181 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8edc09b440 con 0x55da5108a880 2024-06-10T03:00:37.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.349+0000 7f8ec9ffb700 1 -- 172.21.15.52:0/1188299181 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 36) ==== 8378+0+0 (secure 0 0 0) 0x7f8edc097270 con 0x55da5108a880 2024-06-10T03:00:37.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.349+0000 7f8ef317db80 10 client.16434 fetch_fsmap finished waiting for FSMap version 36 2024-06-10T03:00:37.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.349+0000 7f8ef317db80 10 client.16434 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-06-10T03:00:37.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.349+0000 7f8ef317db80 20 client.16434 populate_metadata read hostname 'smithi052' 2024-06-10T03:00:37.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.349+0000 7f8ef317db80 1 --2- 172.21.15.52:0/1188299181 >> [v2:172.21.15.188:6836/819317726,v1:172.21.15.188:6837/819317726] conn(0x55da5109de60 0x55da510a0250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:37.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.349+0000 7f8ef317db80 4 client.16434 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T03:00:37.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.349+0000 7f8ef317db80 1 -- 172.21.15.52:0/1188299181 --> [v2:172.21.15.188:6836/819317726,v1:172.21.15.188:6837/819317726] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55da50fe2d50 con 0x55da5109de60 2024-06-10T03:00:37.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.350+0000 7f8ee259c700 1 --2- 172.21.15.52:0/1188299181 >> [v2:172.21.15.188:6836/819317726,v1:172.21.15.188:6837/819317726] conn(0x55da5109de60 0x55da510a0250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:37.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.350+0000 7f8ee259c700 1 --2- 172.21.15.52:0/1188299181 >> [v2:172.21.15.188:6836/819317726,v1:172.21.15.188:6837/819317726] conn(0x55da5109de60 0x55da510a0250 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=24b42dda4f8c77f6 server_cookie=eef43de40e495726 in_seq=0 out_seq=0 2024-06-10T03:00:37.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.350+0000 7f8ec9ffb700 10 client.16434 ms_handle_connect on v2:172.21.15.188:6836/819317726 2024-06-10T03:00:37.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.351+0000 7f8ec9ffb700 1 -- 172.21.15.52:0/1188299181 <== mds.1 v2:172.21.15.188:6836/819317726 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55da50fe2d50 con 0x55da5109de60 2024-06-10T03:00:37.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.351+0000 7f8ec9ffb700 10 client.16434 handle_command_reply: tid=0 2024-06-10T03:00:37.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.351+0000 7f8ef317db80 1 client.16434 shutdown 2024-06-10T03:00:37.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.351+0000 7f8ef317db80 1 -- 172.21.15.52:0/1188299181 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55da5108a880 msgr2=0x55da5108de70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:00:37.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.351+0000 7f8ef317db80 1 --2- 172.21.15.52:0/1188299181 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55da5108a880 0x55da5108de70 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f8edc095fd0 tx=0x7f8edc004b40 comp rx=0 tx=0).stop 2024-06-10T03:00:37.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.351+0000 7f8ef317db80 1 -- 172.21.15.52:0/1188299181 shutdown_connections 2024-06-10T03:00:37.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.351+0000 7f8ef317db80 1 --2- 172.21.15.52:0/1188299181 >> [v2:172.21.15.188:6836/819317726,v1:172.21.15.188:6837/819317726] conn(0x55da5109de60 0x55da510a0250 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:37.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.351+0000 7f8ef317db80 1 --2- 172.21.15.52:0/1188299181 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55da5108a880 0x55da5108de70 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:37.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.351+0000 7f8ef317db80 1 --2- 172.21.15.52:0/1188299181 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55da510847a0 0x55da51090630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:37.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.351+0000 7f8ec9ffb700 0 client.16434 ms_handle_reset on v2:172.21.15.188:6836/819317726 2024-06-10T03:00:37.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.351+0000 7f8ef317db80 1 --2- 172.21.15.52:0/1188299181 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55da51089f60 0x55da5108d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:37.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.352+0000 7f8ef317db80 1 -- 172.21.15.52:0/1188299181 >> 172.21.15.52:0/1188299181 conn(0x55da50fb1080 msgr2=0x55da5107e700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:00:37.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.352+0000 7f8ef317db80 1 -- 172.21.15.52:0/1188299181 shutdown_connections 2024-06-10T03:00:37.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.352+0000 7f8ef317db80 1 -- 172.21.15.52:0/1188299181 wait complete. 2024-06-10T03:00:37.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.352+0000 7f8ef317db80 20 client.16434 trim_cache size 0 max 16384 2024-06-10T03:00:37.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.369+0000 7f8ef317db80 1 Processor -- start 2024-06-10T03:00:37.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.369+0000 7f8ef317db80 1 -- start start 2024-06-10T03:00:37.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.369+0000 7f8ef317db80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55da510847a0 0x55da51084260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:37.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.369+0000 7f8ef317db80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55da51089f60 0x55da5108cab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:37.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.369+0000 7f8ef317db80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55da5108a880 0x55da5108f870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:37.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.369+0000 7f8ef317db80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f8ee41ac1f0 con 0x55da51089f60 2024-06-10T03:00:37.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.369+0000 7f8ef317db80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f8ee41addf0 con 0x55da5108a880 2024-06-10T03:00:37.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.369+0000 7f8ef317db80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f8ee41ac8c0 con 0x55da510847a0 2024-06-10T03:00:37.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.369+0000 7f8ee1d9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55da51089f60 0x55da5108cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:37.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.369+0000 7f8ee1d9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55da51089f60 0x55da5108cab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:42850/0 (socket says 172.21.15.52:42850) 2024-06-10T03:00:37.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.369+0000 7f8ee1d9b700 1 -- 172.21.15.52:0/1504806048 learned_addr learned my addr 172.21.15.52:0/1504806048 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:00:37.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.369+0000 7f8ee259c700 1 --2- 172.21.15.52:0/1504806048 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55da510847a0 0x55da51084260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:37.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.370+0000 7f8ee2d9d700 1 --2- 172.21.15.52:0/1504806048 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55da5108a880 0x55da5108f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:37.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.370+0000 7f8ee1d9b700 1 -- 172.21.15.52:0/1504806048 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55da510847a0 msgr2=0x55da51084260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:00:37.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.370+0000 7f8ee1d9b700 1 --2- 172.21.15.52:0/1504806048 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55da510847a0 0x55da51084260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:37.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.370+0000 7f8ee1d9b700 1 -- 172.21.15.52:0/1504806048 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55da5108a880 msgr2=0x55da5108f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:00:37.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.370+0000 7f8ee1d9b700 1 --2- 172.21.15.52:0/1504806048 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55da5108a880 0x55da5108f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:37.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.370+0000 7f8ee1d9b700 1 -- 172.21.15.52:0/1504806048 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da5108a680 con 0x55da51089f60 2024-06-10T03:00:37.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.370+0000 7f8ee259c700 1 --2- 172.21.15.52:0/1504806048 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55da510847a0 0x55da51084260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T03:00:37.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.370+0000 7f8ee1d9b700 1 --2- 172.21.15.52:0/1504806048 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55da51089f60 0x55da5108cab0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f8ed80107d0 tx=0x7f8ed800a400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e6a7f4f89ef8073 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:00:37.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.370+0000 7f8ecbfff700 1 -- 172.21.15.52:0/1504806048 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ed801e030 con 0x55da51089f60 2024-06-10T03:00:37.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.370+0000 7f8ecbfff700 1 -- 172.21.15.52:0/1504806048 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ed8004d10 con 0x55da51089f60 2024-06-10T03:00:37.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.370+0000 7f8ecbfff700 1 -- 172.21.15.52:0/1504806048 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ed800fa10 con 0x55da51089f60 2024-06-10T03:00:37.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.371+0000 7f8ef317db80 1 -- 172.21.15.52:0/1504806048 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55da51089f60 msgr2=0x55da5108cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:00:37.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.371+0000 7f8ef317db80 1 --2- 172.21.15.52:0/1504806048 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55da51089f60 0x55da5108cab0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f8ed80107d0 tx=0x7f8ed800a400 comp rx=0 tx=0).stop 2024-06-10T03:00:37.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.371+0000 7f8ef317db80 1 -- 172.21.15.52:0/1504806048 shutdown_connections 2024-06-10T03:00:37.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.371+0000 7f8ef317db80 1 --2- 172.21.15.52:0/1504806048 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55da5108a880 0x55da5108f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:37.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.371+0000 7f8ef317db80 1 --2- 172.21.15.52:0/1504806048 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55da510847a0 0x55da51084260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:37.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.371+0000 7f8ef317db80 1 --2- 172.21.15.52:0/1504806048 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55da51089f60 0x55da5108cab0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:37.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.371+0000 7f8ef317db80 1 -- 172.21.15.52:0/1504806048 >> 172.21.15.52:0/1504806048 conn(0x55da50fb1080 msgr2=0x55da5108f160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:00:37.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.371+0000 7f8ef317db80 1 -- 172.21.15.52:0/1504806048 shutdown_connections 2024-06-10T03:00:37.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.371+0000 7f8ef317db80 1 -- 172.21.15.52:0/1504806048 wait complete. 2024-06-10T03:00:37.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.371+0000 7f8ef317db80 1 -- start start 2024-06-10T03:00:37.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.372+0000 7f8ef317db80 1 Processor -- start 2024-06-10T03:00:37.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.372+0000 7f8ef317db80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55da510847a0 0x55da510a3680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:37.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.372+0000 7f8ef317db80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55da51089f60 0x55da510a0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:37.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.372+0000 7f8ef317db80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55da5108a880 0x55da510a0f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:37.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.372+0000 7f8ef317db80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f8ee41ad720 con 0x55da5108a880 2024-06-10T03:00:37.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.372+0000 7f8ef317db80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f8ee41ad050 con 0x55da51089f60 2024-06-10T03:00:37.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.372+0000 7f8ef317db80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f8ee41aa190 con 0x55da510847a0 2024-06-10T03:00:37.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.372+0000 7f8ef317db80 10 client.0 _ll_register_callbacks cb 0x55da50f6b5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T03:00:37.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.372+0000 7f8ee2d9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55da5108a880 0x55da510a0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:37.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.372+0000 7f8ee2d9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55da5108a880 0x55da510a0f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:42854/0 (socket says 172.21.15.52:42854) 2024-06-10T03:00:37.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.372+0000 7f8ee259c700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55da510847a0 0x55da510a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:37.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.372+0000 7f8ee2d9d700 1 -- 172.21.15.52:0/513632944 learned_addr learned my addr 172.21.15.52:0/513632944 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:00:37.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.372+0000 7f8ee1d9b700 1 --2- 172.21.15.52:0/513632944 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55da51089f60 0x55da510a0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:37.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.372+0000 7f8ee2d9d700 1 -- 172.21.15.52:0/513632944 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55da510847a0 msgr2=0x55da510a3680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:00:37.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.372+0000 7f8ee2d9d700 1 --2- 172.21.15.52:0/513632944 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55da510847a0 0x55da510a3680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:37.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.372+0000 7f8ee2d9d700 1 -- 172.21.15.52:0/513632944 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55da51089f60 msgr2=0x55da510a0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:00:37.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.372+0000 7f8ee2d9d700 1 --2- 172.21.15.52:0/513632944 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55da51089f60 0x55da510a0a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:37.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.373+0000 7f8ee2d9d700 1 -- 172.21.15.52:0/513632944 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da510a14a0 con 0x55da5108a880 2024-06-10T03:00:37.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.373+0000 7f8ee259c700 1 --2- 172.21.15.52:0/513632944 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55da510847a0 0x55da510a3680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T03:00:37.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.373+0000 7f8ee1d9b700 1 --2- 172.21.15.52:0/513632944 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55da51089f60 0x55da510a0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T03:00:37.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.373+0000 7f8ee2d9d700 1 --2- 172.21.15.52:0/513632944 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55da5108a880 0x55da510a0f60 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f8edc00c6f0 tx=0x7f8edc00ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfeddd69b31210b4 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:00:37.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.373+0000 7f8ec9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-10T03:00:37.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.373+0000 7f8ec9ffb700 1 -- 172.21.15.52:0/513632944 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8edc09af80 con 0x55da5108a880 2024-06-10T03:00:37.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.373+0000 7f8ec9ffb700 1 -- 172.21.15.52:0/513632944 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f8edc09aaa0 con 0x55da5108a880 2024-06-10T03:00:37.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.373+0000 7f8ef317db80 1 -- 172.21.15.52:0/513632944 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ed8004d10 con 0x55da5108a880 2024-06-10T03:00:37.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.373+0000 7f8ec9ffb700 1 -- 172.21.15.52:0/513632944 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f8edc09bb40 con 0x55da5108a880 2024-06-10T03:00:37.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.373+0000 7f8ef317db80 10 client.16440 fetch_fsmap learned FSMap version 36 2024-06-10T03:00:37.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.373+0000 7f8ef317db80 1 -- 172.21.15.52:0/513632944 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55da51110c90 con 0x55da5108a880 2024-06-10T03:00:37.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.373+0000 7f8ec9ffb700 1 -- 172.21.15.52:0/513632944 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ed8004d10 con 0x55da5108a880 2024-06-10T03:00:37.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.373+0000 7f8ec9ffb700 1 -- 172.21.15.52:0/513632944 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 36) ==== 8378+0+0 (secure 0 0 0) 0x7f8edc003ad0 con 0x55da5108a880 2024-06-10T03:00:37.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.374+0000 7f8ef317db80 10 client.16440 fetch_fsmap finished waiting for FSMap version 36 2024-06-10T03:00:37.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.374+0000 7f8ef317db80 10 client.16440 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-06-10T03:00:37.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.374+0000 7f8ef317db80 20 client.16440 populate_metadata read hostname 'smithi052' 2024-06-10T03:00:37.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.374+0000 7f8ef317db80 1 --2- 172.21.15.52:0/513632944 >> [v2:172.21.15.188:6836/819317726,v1:172.21.15.188:6837/819317726] conn(0x55da511111a0 0x55da51113590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:37.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.374+0000 7f8ef317db80 4 client.16440 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T03:00:37.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.374+0000 7f8ef317db80 1 -- 172.21.15.52:0/513632944 --> [v2:172.21.15.188:6836/819317726,v1:172.21.15.188:6837/819317726] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55da50fdd2d0 con 0x55da511111a0 2024-06-10T03:00:37.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.374+0000 7f8ee259c700 1 --2- 172.21.15.52:0/513632944 >> [v2:172.21.15.188:6836/819317726,v1:172.21.15.188:6837/819317726] conn(0x55da511111a0 0x55da51113590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:37.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.374+0000 7f8ee259c700 1 --2- 172.21.15.52:0/513632944 >> [v2:172.21.15.188:6836/819317726,v1:172.21.15.188:6837/819317726] conn(0x55da511111a0 0x55da51113590 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f01ab6481303513c server_cookie=2d19ef3a4eeac7f9 in_seq=0 out_seq=0 2024-06-10T03:00:37.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.374+0000 7f8ec9ffb700 10 client.16440 ms_handle_connect on v2:172.21.15.188:6836/819317726 2024-06-10T03:00:37.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.374+0000 7f8ec9ffb700 1 -- 172.21.15.52:0/513632944 <== mds.1 v2:172.21.15.188:6836/819317726 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55da50fdd2d0 con 0x55da511111a0 2024-06-10T03:00:37.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.374+0000 7f8ec9ffb700 10 client.16440 handle_command_reply: tid=0 2024-06-10T03:00:37.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.374+0000 7f8ef317db80 1 client.16440 shutdown 2024-06-10T03:00:37.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.375+0000 7f8ef317db80 1 -- 172.21.15.52:0/513632944 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55da5108a880 msgr2=0x55da510a0f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:00:37.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.375+0000 7f8ef317db80 1 --2- 172.21.15.52:0/513632944 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55da5108a880 0x55da510a0f60 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f8edc00c6f0 tx=0x7f8edc00ceb0 comp rx=0 tx=0).stop 2024-06-10T03:00:37.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.375+0000 7f8ef317db80 1 -- 172.21.15.52:0/513632944 shutdown_connections 2024-06-10T03:00:37.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.375+0000 7f8ef317db80 1 --2- 172.21.15.52:0/513632944 >> [v2:172.21.15.188:6836/819317726,v1:172.21.15.188:6837/819317726] conn(0x55da511111a0 0x55da51113590 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:37.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.375+0000 7f8ef317db80 1 --2- 172.21.15.52:0/513632944 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55da5108a880 0x55da510a0f60 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:37.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.375+0000 7f8ef317db80 1 --2- 172.21.15.52:0/513632944 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55da510847a0 0x55da510a3680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:37.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.375+0000 7f8ec9ffb700 0 client.16440 ms_handle_reset on v2:172.21.15.188:6836/819317726 2024-06-10T03:00:37.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.375+0000 7f8ef317db80 1 --2- 172.21.15.52:0/513632944 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55da51089f60 0x55da510a0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:37.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.375+0000 7f8ef317db80 1 -- 172.21.15.52:0/513632944 >> 172.21.15.52:0/513632944 conn(0x55da50fb1080 msgr2=0x55da5108f160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:00:37.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.375+0000 7f8ef317db80 1 -- 172.21.15.52:0/513632944 shutdown_connections 2024-06-10T03:00:37.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.375+0000 7f8ef317db80 1 -- 172.21.15.52:0/513632944 wait complete. 2024-06-10T03:00:37.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.375+0000 7f8ef317db80 20 client.16440 trim_cache size 0 max 16384 2024-06-10T03:00:37.380 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-06-10T03:00:37.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.375+0000 7f8ee88fa700 1 -- 172.21.15.52:0/3597064699 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f8ecc0a2c20 msgr2=0x7f8ecc0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:00:37.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.375+0000 7f8ee88fa700 1 --2- 172.21.15.52:0/3597064699 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f8ecc0a2c20 0x7f8ecc0a50e0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f8ee4199b20 tx=0x7f8ed4005fd0 comp rx=0 tx=0).stop 2024-06-10T03:00:37.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.375+0000 7f8ee88fa700 1 -- 172.21.15.52:0/3597064699 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8ee4108520 msgr2=0x7f8ee41989e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:00:37.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.375+0000 7f8ee88fa700 1 --2- 172.21.15.52:0/3597064699 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8ee4108520 0x7f8ee41989e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f8edc00efc0 tx=0x7f8edc00c3a0 comp rx=0 tx=0).stop 2024-06-10T03:00:37.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.375+0000 7f8ee88fa700 1 -- 172.21.15.52:0/3597064699 shutdown_connections 2024-06-10T03:00:37.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.375+0000 7f8ee88fa700 1 --2- 172.21.15.52:0/3597064699 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8ee4108520 0x7f8ee41989e0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:37.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.376+0000 7f8ee88fa700 1 --2- 172.21.15.52:0/3597064699 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8ee4102510 0x7f8ee419e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:37.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.376+0000 7f8ee88fa700 1 --2- 172.21.15.52:0/3597064699 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f8ecc0a2c20 0x7f8ecc0a50e0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:37.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.376+0000 7f8ee88fa700 1 --2- 172.21.15.52:0/3597064699 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8ee4102ed0 0x7f8ee419ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:37.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.376+0000 7f8ee88fa700 1 -- 172.21.15.52:0/3597064699 >> 172.21.15.52:0/3597064699 conn(0x7f8ee4079560 msgr2=0x7f8ee4101020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:00:37.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.376+0000 7f8ee88fa700 1 -- 172.21.15.52:0/3597064699 shutdown_connections 2024-06-10T03:00:37.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:37.376+0000 7f8ee88fa700 1 -- 172.21.15.52:0/3597064699 wait complete. 2024-06-10T03:00:38.155 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-06-10T03:00:38.436 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T03:00:38.477 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:00:38 smithi052 ceph-mon[32362]: pgmap v627: 129 pgs: 129 active+clean; 568 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 1.5 MiB/s wr, 550 op/s 2024-06-10T03:00:38.638 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:00:38 smithi120 ceph-mon[36634]: pgmap v627: 129 pgs: 129 active+clean; 568 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 1.5 MiB/s wr, 550 op/s 2024-06-10T03:00:38.703 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:00:38 smithi188 ceph-mon[36599]: pgmap v627: 129 pgs: 129 active+clean; 568 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 1.5 MiB/s wr, 550 op/s 2024-06-10T03:00:40.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.260+0000 7fb1d6711700 1 -- 172.21.15.52:0/2368228999 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb1d0102510 msgr2=0x7fb1d0102990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:00:40.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.260+0000 7fb1d6711700 1 --2- 172.21.15.52:0/2368228999 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb1d0102510 0x7fb1d0102990 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fb1c4009990 tx=0x7fb1c4009ca0 comp rx=0 tx=0).stop 2024-06-10T03:00:40.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.260+0000 7fb1d6711700 1 -- 172.21.15.52:0/2368228999 shutdown_connections 2024-06-10T03:00:40.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.260+0000 7fb1d6711700 1 --2- 172.21.15.52:0/2368228999 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb1d0102ed0 0x7fb1d010f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:40.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.260+0000 7fb1d6711700 1 --2- 172.21.15.52:0/2368228999 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb1d0108520 0x7fb1d0108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:40.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.260+0000 7fb1d6711700 1 --2- 172.21.15.52:0/2368228999 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb1d0102510 0x7fb1d0102990 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:40.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.260+0000 7fb1d6711700 1 -- 172.21.15.52:0/2368228999 >> 172.21.15.52:0/2368228999 conn(0x7fb1d0079560 msgr2=0x7fb1d0079970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:00:40.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.260+0000 7fb1d6711700 1 -- 172.21.15.52:0/2368228999 shutdown_connections 2024-06-10T03:00:40.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.260+0000 7fb1d6711700 1 -- 172.21.15.52:0/2368228999 wait complete. 2024-06-10T03:00:40.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.261+0000 7fb1d6711700 1 Processor -- start 2024-06-10T03:00:40.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.261+0000 7fb1d6711700 1 -- start start 2024-06-10T03:00:40.267 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.261+0000 7fb1d6711700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb1d0102510 0x7fb1d019e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:40.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.261+0000 7fb1d6711700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb1d0102ed0 0x7fb1d019ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:40.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.261+0000 7fb1d6711700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb1d0108520 0x7fb1d01989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:40.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.261+0000 7fb1d6711700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb1d01127b0 con 0x7fb1d0102ed0 2024-06-10T03:00:40.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.261+0000 7fb1d6711700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fb1d0112630 con 0x7fb1d0102510 2024-06-10T03:00:40.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.261+0000 7fb1d6711700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fb1d0112930 con 0x7fb1d0108520 2024-06-10T03:00:40.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.261+0000 7fb1cf7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb1d0102ed0 0x7fb1d019ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:40.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.261+0000 7fb1cf7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb1d0102ed0 0x7fb1d019ed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:42864/0 (socket says 172.21.15.52:42864) 2024-06-10T03:00:40.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.261+0000 7fb1cf7fe700 1 -- 172.21.15.52:0/179752865 learned_addr learned my addr 172.21.15.52:0/179752865 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:00:40.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.261+0000 7fb1d4cae700 1 --2- 172.21.15.52:0/179752865 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb1d0108520 0x7fb1d01989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:40.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.261+0000 7fb1cffff700 1 --2- 172.21.15.52:0/179752865 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb1d0102510 0x7fb1d019e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:40.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.262+0000 7fb1cf7fe700 1 -- 172.21.15.52:0/179752865 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb1d0108520 msgr2=0x7fb1d01989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:00:40.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.262+0000 7fb1cf7fe700 1 --2- 172.21.15.52:0/179752865 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb1d0108520 0x7fb1d01989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:40.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.262+0000 7fb1cf7fe700 1 -- 172.21.15.52:0/179752865 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb1d0102510 msgr2=0x7fb1d019e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:00:40.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.262+0000 7fb1cf7fe700 1 --2- 172.21.15.52:0/179752865 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb1d0102510 0x7fb1d019e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:40.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.262+0000 7fb1cf7fe700 1 -- 172.21.15.52:0/179752865 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1d0199150 con 0x7fb1d0102ed0 2024-06-10T03:00:40.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.262+0000 7fb1cf7fe700 1 --2- 172.21.15.52:0/179752865 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb1d0102ed0 0x7fb1d019ed20 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fb1c4000c00 tx=0x7fb1c400bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:00:40.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.262+0000 7fb1cd7fa700 1 -- 172.21.15.52:0/179752865 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1c4027070 con 0x7fb1d0102ed0 2024-06-10T03:00:40.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.262+0000 7fb1d6711700 1 -- 172.21.15.52:0/179752865 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1d01993e0 con 0x7fb1d0102ed0 2024-06-10T03:00:40.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.262+0000 7fb1cd7fa700 1 -- 172.21.15.52:0/179752865 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb1c4003d70 con 0x7fb1d0102ed0 2024-06-10T03:00:40.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.262+0000 7fb1cd7fa700 1 -- 172.21.15.52:0/179752865 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1c400f460 con 0x7fb1d0102ed0 2024-06-10T03:00:40.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.262+0000 7fb1d6711700 1 -- 172.21.15.52:0/179752865 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1d01a56b0 con 0x7fb1d0102ed0 2024-06-10T03:00:40.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.265+0000 7fb1cd7fa700 1 -- 172.21.15.52:0/179752865 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fb1c402a070 con 0x7fb1d0102ed0 2024-06-10T03:00:40.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.265+0000 7fb1cd7fa700 1 --2- 172.21.15.52:0/179752865 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fb1b80a2c20 0x7fb1b80a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:40.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.265+0000 7fb1cffff700 1 --2- 172.21.15.52:0/179752865 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fb1b80a2c20 0x7fb1b80a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:40.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.265+0000 7fb1cd7fa700 1 -- 172.21.15.52:0/179752865 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7fb1c40d0240 con 0x7fb1d0102ed0 2024-06-10T03:00:40.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.266+0000 7fb1cffff700 1 --2- 172.21.15.52:0/179752865 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fb1b80a2c20 0x7fb1b80a50e0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fb1d0100cd0 tx=0x7fb1c000a380 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:00:40.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.266+0000 7fb1e0f94b80 1 Processor -- start 2024-06-10T03:00:40.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.266+0000 7fb1e0f94b80 1 -- start start 2024-06-10T03:00:40.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.266+0000 7fb1e0f94b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fb2f312f60 0x55fb2f313340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:40.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.266+0000 7fb1e0f94b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fb2f313880 0x55fb2f30d260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:40.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.266+0000 7fb1e0f94b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fb2f30d7a0 0x55fb2f30dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:40.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.266+0000 7fb1e0f94b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55fb2f1fff60 con 0x55fb2f312f60 2024-06-10T03:00:40.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.266+0000 7fb1e0f94b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55fb2f25e850 con 0x55fb2f30d7a0 2024-06-10T03:00:40.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.266+0000 7fb1e0f94b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55fb2f1c6460 con 0x55fb2f313880 2024-06-10T03:00:40.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.267+0000 7fb1cffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fb2f312f60 0x55fb2f313340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:40.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.267+0000 7fb1cf7fe700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fb2f313880 0x55fb2f30d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:40.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.267+0000 7fb1cffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fb2f312f60 0x55fb2f313340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:42876/0 (socket says 172.21.15.52:42876) 2024-06-10T03:00:40.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.267+0000 7fb1d4cae700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fb2f30d7a0 0x55fb2f30dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:40.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.267+0000 7fb1cffff700 1 -- 172.21.15.52:0/3390348399 learned_addr learned my addr 172.21.15.52:0/3390348399 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:00:40.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.267+0000 7fb1cffff700 1 -- 172.21.15.52:0/3390348399 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fb2f313880 msgr2=0x55fb2f30d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:00:40.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.267+0000 7fb1cffff700 1 --2- 172.21.15.52:0/3390348399 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fb2f313880 0x55fb2f30d260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:40.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.267+0000 7fb1cffff700 1 -- 172.21.15.52:0/3390348399 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fb2f30d7a0 msgr2=0x55fb2f30dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:00:40.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.267+0000 7fb1cffff700 1 --2- 172.21.15.52:0/3390348399 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fb2f30d7a0 0x55fb2f30dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:40.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.267+0000 7fb1cffff700 1 -- 172.21.15.52:0/3390348399 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb2f1ad8d0 con 0x55fb2f312f60 2024-06-10T03:00:40.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.267+0000 7fb1d4cae700 1 --2- 172.21.15.52:0/3390348399 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fb2f30d7a0 0x55fb2f30dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T03:00:40.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.267+0000 7fb1cf7fe700 1 --2- 172.21.15.52:0/3390348399 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fb2f313880 0x55fb2f30d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T03:00:40.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.267+0000 7fb1cffff700 1 --2- 172.21.15.52:0/3390348399 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fb2f312f60 0x55fb2f313340 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fb1c0004530 tx=0x7fb1c0012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6840ce594c54e3e8 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:00:40.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.268+0000 7fb1b67fc700 1 -- 172.21.15.52:0/3390348399 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1c0016070 con 0x55fb2f312f60 2024-06-10T03:00:40.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.268+0000 7fb1b67fc700 1 -- 172.21.15.52:0/3390348399 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb1c0014070 con 0x55fb2f312f60 2024-06-10T03:00:40.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.268+0000 7fb1b67fc700 1 -- 172.21.15.52:0/3390348399 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1c001d400 con 0x55fb2f312f60 2024-06-10T03:00:40.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.269+0000 7fb1e0f94b80 1 -- 172.21.15.52:0/3390348399 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fb2f312f60 msgr2=0x55fb2f313340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:00:40.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.269+0000 7fb1e0f94b80 1 --2- 172.21.15.52:0/3390348399 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fb2f312f60 0x55fb2f313340 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fb1c0004530 tx=0x7fb1c0012000 comp rx=0 tx=0).stop 2024-06-10T03:00:40.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.269+0000 7fb1e0f94b80 1 -- 172.21.15.52:0/3390348399 shutdown_connections 2024-06-10T03:00:40.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.269+0000 7fb1e0f94b80 1 --2- 172.21.15.52:0/3390348399 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fb2f30d7a0 0x55fb2f30dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:40.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.269+0000 7fb1e0f94b80 1 --2- 172.21.15.52:0/3390348399 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fb2f312f60 0x55fb2f313340 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:40.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.269+0000 7fb1e0f94b80 1 --2- 172.21.15.52:0/3390348399 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fb2f313880 0x55fb2f30d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:40.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.269+0000 7fb1e0f94b80 1 -- 172.21.15.52:0/3390348399 >> 172.21.15.52:0/3390348399 conn(0x55fb2f23a080 msgr2=0x55fb2f308220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:00:40.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.269+0000 7fb1e0f94b80 1 -- 172.21.15.52:0/3390348399 shutdown_connections 2024-06-10T03:00:40.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.269+0000 7fb1e0f94b80 1 -- 172.21.15.52:0/3390348399 wait complete. 2024-06-10T03:00:40.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.270+0000 7fb1e0f94b80 1 -- start start 2024-06-10T03:00:40.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.270+0000 7fb1e0f94b80 1 Processor -- start 2024-06-10T03:00:40.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.270+0000 7fb1e0f94b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fb2f30d7a0 0x55fb2f3196a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:40.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.270+0000 7fb1e0f94b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fb2f312f60 0x55fb2f3169a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:40.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.270+0000 7fb1e0f94b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fb2f313880 0x55fb2f316ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:40.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.270+0000 7fb1e0f94b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55fb2f1fefc0 con 0x55fb2f313880 2024-06-10T03:00:40.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.270+0000 7fb1e0f94b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55fb2f25a810 con 0x55fb2f312f60 2024-06-10T03:00:40.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.270+0000 7fb1e0f94b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55fb2f31a260 con 0x55fb2f30d7a0 2024-06-10T03:00:40.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.270+0000 7fb1e0f94b80 10 client.0 _ll_register_callbacks cb 0x55fb2f1f45b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T03:00:40.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.270+0000 7fb1cf7fe700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fb2f312f60 0x55fb2f3169a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:40.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.270+0000 7fb1d4cae700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fb2f313880 0x55fb2f316ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:40.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.270+0000 7fb1cffff700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fb2f30d7a0 0x55fb2f3196a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:40.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.270+0000 7fb1d4cae700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fb2f313880 0x55fb2f316ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:42888/0 (socket says 172.21.15.52:42888) 2024-06-10T03:00:40.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.270+0000 7fb1cffff700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fb2f30d7a0 0x55fb2f3196a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.52:34172/0 (socket says 172.21.15.52:34172) 2024-06-10T03:00:40.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.270+0000 7fb1cf7fe700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fb2f312f60 0x55fb2f3169a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.52:50480/0 (socket says 172.21.15.52:50480) 2024-06-10T03:00:40.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.270+0000 7fb1d4cae700 1 -- 172.21.15.52:0/2341987295 learned_addr learned my addr 172.21.15.52:0/2341987295 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:00:40.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.271+0000 7fb1cf7fe700 1 -- 172.21.15.52:0/2341987295 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fb2f30d7a0 msgr2=0x55fb2f3196a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:00:40.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.271+0000 7fb1cf7fe700 1 --2- 172.21.15.52:0/2341987295 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fb2f30d7a0 0x55fb2f3196a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:40.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.271+0000 7fb1cf7fe700 1 -- 172.21.15.52:0/2341987295 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fb2f313880 msgr2=0x55fb2f316ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:00:40.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.271+0000 7fb1cf7fe700 1 --2- 172.21.15.52:0/2341987295 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fb2f313880 0x55fb2f316ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:40.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.271+0000 7fb1cf7fe700 1 -- 172.21.15.52:0/2341987295 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb2f224750 con 0x55fb2f312f60 2024-06-10T03:00:40.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.271+0000 7fb1cf7fe700 1 --2- 172.21.15.52:0/2341987295 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fb2f312f60 0x55fb2f3169a0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fb1c40cdd00 tx=0x7fb1c40229e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=35084c1b13588a80 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:00:40.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.271+0000 7fb1af7fe700 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-06-10T03:00:40.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.271+0000 7fb1af7fe700 1 -- 172.21.15.52:0/2341987295 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1c40a5360 con 0x55fb2f312f60 2024-06-10T03:00:40.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.271+0000 7fb1e0f94b80 1 -- 172.21.15.52:0/2341987295 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb1c0014070 con 0x55fb2f312f60 2024-06-10T03:00:40.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.271+0000 7fb1af7fe700 1 -- 172.21.15.52:0/2341987295 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7fb1c40094b0 con 0x55fb2f312f60 2024-06-10T03:00:40.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.271+0000 7fb1af7fe700 1 -- 172.21.15.52:0/2341987295 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fb1c40870e0 con 0x55fb2f312f60 2024-06-10T03:00:40.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.271+0000 7fb1e0f94b80 10 client.25991 fetch_fsmap learned FSMap version 36 2024-06-10T03:00:40.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.271+0000 7fb1e0f94b80 1 -- 172.21.15.52:0/2341987295 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55fb2f324ac0 con 0x55fb2f312f60 2024-06-10T03:00:40.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.272+0000 7fb1af7fe700 1 -- 172.21.15.52:0/2341987295 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb1c0014070 con 0x55fb2f312f60 2024-06-10T03:00:40.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.272+0000 7fb1af7fe700 1 -- 172.21.15.52:0/2341987295 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 36) ==== 8378+0+0 (secure 0 0 0) 0x7fb1c40985b0 con 0x55fb2f312f60 2024-06-10T03:00:40.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.272+0000 7fb1e0f94b80 10 client.25991 fetch_fsmap finished waiting for FSMap version 36 2024-06-10T03:00:40.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.272+0000 7fb1e0f94b80 10 client.25991 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T03:00:40.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.272+0000 7fb1e0f94b80 20 client.25991 populate_metadata read hostname 'smithi052' 2024-06-10T03:00:40.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.272+0000 7fb1e0f94b80 1 --2- 172.21.15.52:0/2341987295 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55fb2f327170 0x55fb2f329590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:40.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.272+0000 7fb1e0f94b80 4 client.25991 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T03:00:40.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.272+0000 7fb1e0f94b80 1 -- 172.21.15.52:0/2341987295 --> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fb2f26bd50 con 0x55fb2f327170 2024-06-10T03:00:40.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.272+0000 7fb1cffff700 1 --2- 172.21.15.52:0/2341987295 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55fb2f327170 0x55fb2f329590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:40.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.273+0000 7fb1cffff700 1 --2- 172.21.15.52:0/2341987295 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55fb2f327170 0x55fb2f329590 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1153e4da705c4a16 server_cookie=9d83bad670b2329e in_seq=0 out_seq=0 2024-06-10T03:00:40.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.273+0000 7fb1af7fe700 10 client.25991 ms_handle_connect on v2:172.21.15.188:6838/310444836 2024-06-10T03:00:40.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.273+0000 7fb1af7fe700 1 -- 172.21.15.52:0/2341987295 <== mds.0 v2:172.21.15.188:6838/310444836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55fb2f26bd50 con 0x55fb2f327170 2024-06-10T03:00:40.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.273+0000 7fb1af7fe700 10 client.25991 handle_command_reply: tid=0 2024-06-10T03:00:40.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.273+0000 7fb1e0f94b80 1 client.25991 shutdown 2024-06-10T03:00:40.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.274+0000 7fb1e0f94b80 1 -- 172.21.15.52:0/2341987295 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fb2f312f60 msgr2=0x55fb2f3169a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:00:40.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.274+0000 7fb1e0f94b80 1 --2- 172.21.15.52:0/2341987295 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fb2f312f60 0x55fb2f3169a0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fb1c40cdd00 tx=0x7fb1c40229e0 comp rx=0 tx=0).stop 2024-06-10T03:00:40.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.274+0000 7fb1e0f94b80 1 -- 172.21.15.52:0/2341987295 shutdown_connections 2024-06-10T03:00:40.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.274+0000 7fb1e0f94b80 1 --2- 172.21.15.52:0/2341987295 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55fb2f327170 0x55fb2f329590 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:40.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.274+0000 7fb1e0f94b80 1 --2- 172.21.15.52:0/2341987295 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fb2f313880 0x55fb2f316ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:40.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.274+0000 7fb1af7fe700 0 client.25991 ms_handle_reset on v2:172.21.15.188:6838/310444836 2024-06-10T03:00:40.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.274+0000 7fb1e0f94b80 1 --2- 172.21.15.52:0/2341987295 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fb2f30d7a0 0x55fb2f3196a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:40.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.274+0000 7fb1e0f94b80 1 --2- 172.21.15.52:0/2341987295 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fb2f312f60 0x55fb2f3169a0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:40.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.274+0000 7fb1e0f94b80 1 -- 172.21.15.52:0/2341987295 >> 172.21.15.52:0/2341987295 conn(0x55fb2f23a080 msgr2=0x55fb2f307700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:00:40.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.274+0000 7fb1e0f94b80 1 -- 172.21.15.52:0/2341987295 shutdown_connections 2024-06-10T03:00:40.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.274+0000 7fb1e0f94b80 1 -- 172.21.15.52:0/2341987295 wait complete. 2024-06-10T03:00:40.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.274+0000 7fb1e0f94b80 20 client.25991 trim_cache size 0 max 16384 2024-06-10T03:00:40.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.291+0000 7fb1e0f94b80 1 Processor -- start 2024-06-10T03:00:40.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.291+0000 7fb1e0f94b80 1 -- start start 2024-06-10T03:00:40.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.291+0000 7fb1e0f94b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fb2f30d7a0 0x55fb2f3233b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:40.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.291+0000 7fb1e0f94b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fb2f312f60 0x55fb2f312190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:40.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.291+0000 7fb1e0f94b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fb2f313880 0x55fb2f3191e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:40.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.291+0000 7fb1e0f94b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb1d01ae210 con 0x55fb2f313880 2024-06-10T03:00:40.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.291+0000 7fb1e0f94b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fb1d01afe10 con 0x55fb2f312f60 2024-06-10T03:00:40.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.291+0000 7fb1e0f94b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fb1d01ae8e0 con 0x55fb2f30d7a0 2024-06-10T03:00:40.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.292+0000 7fb1d4cae700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fb2f313880 0x55fb2f3191e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:40.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.292+0000 7fb1d4cae700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fb2f313880 0x55fb2f3191e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:42896/0 (socket says 172.21.15.52:42896) 2024-06-10T03:00:40.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.292+0000 7fb1d4cae700 1 -- 172.21.15.52:0/2751075807 learned_addr learned my addr 172.21.15.52:0/2751075807 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:00:40.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.292+0000 7fb1cffff700 1 --2- 172.21.15.52:0/2751075807 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fb2f30d7a0 0x55fb2f3233b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:40.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.292+0000 7fb1cf7fe700 1 --2- 172.21.15.52:0/2751075807 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fb2f312f60 0x55fb2f312190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:40.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.292+0000 7fb1d4cae700 1 -- 172.21.15.52:0/2751075807 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fb2f30d7a0 msgr2=0x55fb2f3233b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:00:40.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.292+0000 7fb1d4cae700 1 --2- 172.21.15.52:0/2751075807 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fb2f30d7a0 0x55fb2f3233b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:40.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.292+0000 7fb1d4cae700 1 -- 172.21.15.52:0/2751075807 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fb2f312f60 msgr2=0x55fb2f312190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:00:40.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.292+0000 7fb1d4cae700 1 --2- 172.21.15.52:0/2751075807 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fb2f312f60 0x55fb2f312190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:40.295 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.292+0000 7fb1d4cae700 1 -- 172.21.15.52:0/2751075807 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb2f309040 con 0x55fb2f313880 2024-06-10T03:00:40.295 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.292+0000 7fb1cf7fe700 1 --2- 172.21.15.52:0/2751075807 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fb2f312f60 0x55fb2f312190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T03:00:40.295 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.292+0000 7fb1d4cae700 1 --2- 172.21.15.52:0/2751075807 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fb2f313880 0x55fb2f3191e0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fb1c8011930 tx=0x7fb1c8009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a5c34f0adbfb1e4 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:00:40.295 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.293+0000 7fb1b5ffb700 1 -- 172.21.15.52:0/2751075807 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1c80175b0 con 0x55fb2f313880 2024-06-10T03:00:40.295 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.293+0000 7fb1b5ffb700 1 -- 172.21.15.52:0/2751075807 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb1c8017c30 con 0x55fb2f313880 2024-06-10T03:00:40.295 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.293+0000 7fb1b5ffb700 1 -- 172.21.15.52:0/2751075807 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1c8007e60 con 0x55fb2f313880 2024-06-10T03:00:40.297 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.293+0000 7fb1e0f94b80 1 -- 172.21.15.52:0/2751075807 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fb2f313880 msgr2=0x55fb2f3191e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:00:40.298 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.293+0000 7fb1e0f94b80 1 --2- 172.21.15.52:0/2751075807 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fb2f313880 0x55fb2f3191e0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fb1c8011930 tx=0x7fb1c8009920 comp rx=0 tx=0).stop 2024-06-10T03:00:40.298 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.293+0000 7fb1e0f94b80 1 -- 172.21.15.52:0/2751075807 shutdown_connections 2024-06-10T03:00:40.298 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.293+0000 7fb1e0f94b80 1 --2- 172.21.15.52:0/2751075807 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fb2f313880 0x55fb2f3191e0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:40.298 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.293+0000 7fb1e0f94b80 1 --2- 172.21.15.52:0/2751075807 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fb2f30d7a0 0x55fb2f3233b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:40.298 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.293+0000 7fb1e0f94b80 1 --2- 172.21.15.52:0/2751075807 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fb2f312f60 0x55fb2f312190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:40.298 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.293+0000 7fb1e0f94b80 1 -- 172.21.15.52:0/2751075807 >> 172.21.15.52:0/2751075807 conn(0x55fb2f23a080 msgr2=0x55fb2f311a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:00:40.298 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.293+0000 7fb1e0f94b80 1 -- 172.21.15.52:0/2751075807 shutdown_connections 2024-06-10T03:00:40.298 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.293+0000 7fb1e0f94b80 1 -- 172.21.15.52:0/2751075807 wait complete. 2024-06-10T03:00:40.298 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.294+0000 7fb1e0f94b80 1 -- start start 2024-06-10T03:00:40.298 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.294+0000 7fb1e0f94b80 1 Processor -- start 2024-06-10T03:00:40.298 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.294+0000 7fb1e0f94b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fb2f30d7a0 0x55fb2f31fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:40.298 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.294+0000 7fb1e0f94b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fb2f312f60 0x55fb2f32e870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:40.299 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.294+0000 7fb1e0f94b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fb2f313880 0x55fb2f32edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:40.299 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.294+0000 7fb1e0f94b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb1d01af740 con 0x55fb2f313880 2024-06-10T03:00:40.299 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.294+0000 7fb1e0f94b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fb1d01af070 con 0x55fb2f312f60 2024-06-10T03:00:40.299 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.294+0000 7fb1e0f94b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fb1d01ac1b0 con 0x55fb2f30d7a0 2024-06-10T03:00:40.299 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.295+0000 7fb1e0f94b80 10 client.0 _ll_register_callbacks cb 0x55fb2f1f45b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T03:00:40.299 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.295+0000 7fb1cf7fe700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fb2f312f60 0x55fb2f32e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:40.299 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.295+0000 7fb1d4cae700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fb2f313880 0x55fb2f32edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:40.299 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.295+0000 7fb1cf7fe700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fb2f312f60 0x55fb2f32e870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.52:50494/0 (socket says 172.21.15.52:50494) 2024-06-10T03:00:40.299 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.295+0000 7fb1cf7fe700 1 -- 172.21.15.52:0/471453211 learned_addr learned my addr 172.21.15.52:0/471453211 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:00:40.299 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.295+0000 7fb1cffff700 1 --2- 172.21.15.52:0/471453211 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fb2f30d7a0 0x55fb2f31fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:40.299 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.295+0000 7fb1cf7fe700 1 -- 172.21.15.52:0/471453211 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fb2f30d7a0 msgr2=0x55fb2f31fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:00:40.299 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.295+0000 7fb1cf7fe700 1 --2- 172.21.15.52:0/471453211 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fb2f30d7a0 0x55fb2f31fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:40.299 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.295+0000 7fb1cf7fe700 1 -- 172.21.15.52:0/471453211 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fb2f313880 msgr2=0x55fb2f32edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:00:40.300 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.295+0000 7fb1cf7fe700 1 --2- 172.21.15.52:0/471453211 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fb2f313880 0x55fb2f32edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:40.300 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.295+0000 7fb1cf7fe700 1 -- 172.21.15.52:0/471453211 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb2f32f680 con 0x55fb2f312f60 2024-06-10T03:00:40.300 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.295+0000 7fb1cf7fe700 1 --2- 172.21.15.52:0/471453211 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fb2f312f60 0x55fb2f32e870 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fb1c4098cc0 tx=0x7fb1c4005b00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9f4f7e6bbaae7fe server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:00:40.300 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.295+0000 7fb1af7fe700 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-06-10T03:00:40.300 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.295+0000 7fb1af7fe700 1 -- 172.21.15.52:0/471453211 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1c40a5360 con 0x55fb2f312f60 2024-06-10T03:00:40.300 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.295+0000 7fb1af7fe700 1 -- 172.21.15.52:0/471453211 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7fb1c407fbc0 con 0x55fb2f312f60 2024-06-10T03:00:40.300 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.295+0000 7fb1e0f94b80 1 -- 172.21.15.52:0/471453211 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb1c8017c30 con 0x55fb2f312f60 2024-06-10T03:00:40.300 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.296+0000 7fb1af7fe700 1 -- 172.21.15.52:0/471453211 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fb1c409dab0 con 0x55fb2f312f60 2024-06-10T03:00:40.300 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.296+0000 7fb1e0f94b80 10 client.25997 fetch_fsmap learned FSMap version 36 2024-06-10T03:00:40.300 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.296+0000 7fb1e0f94b80 1 -- 172.21.15.52:0/471453211 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55fb2f391ed0 con 0x55fb2f312f60 2024-06-10T03:00:40.300 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.296+0000 7fb1af7fe700 1 -- 172.21.15.52:0/471453211 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb1c8017c30 con 0x55fb2f312f60 2024-06-10T03:00:40.300 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.296+0000 7fb1af7fe700 1 -- 172.21.15.52:0/471453211 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 36) ==== 8378+0+0 (secure 0 0 0) 0x7fb1c409b720 con 0x55fb2f312f60 2024-06-10T03:00:40.300 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.296+0000 7fb1e0f94b80 10 client.25997 fetch_fsmap finished waiting for FSMap version 36 2024-06-10T03:00:40.301 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.296+0000 7fb1e0f94b80 10 client.25997 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T03:00:40.301 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.296+0000 7fb1e0f94b80 20 client.25997 populate_metadata read hostname 'smithi052' 2024-06-10T03:00:40.301 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.296+0000 7fb1e0f94b80 1 --2- 172.21.15.52:0/471453211 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55fb2f327170 0x55fb2f394460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:40.301 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.296+0000 7fb1e0f94b80 4 client.25997 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T03:00:40.301 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.296+0000 7fb1e0f94b80 1 -- 172.21.15.52:0/471453211 --> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fb2f2662d0 con 0x55fb2f327170 2024-06-10T03:00:40.301 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.296+0000 7fb1cffff700 1 --2- 172.21.15.52:0/471453211 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55fb2f327170 0x55fb2f394460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:40.301 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.297+0000 7fb1af7fe700 10 client.25997 ms_handle_connect on v2:172.21.15.188:6838/310444836 2024-06-10T03:00:40.301 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.297+0000 7fb1cffff700 1 --2- 172.21.15.52:0/471453211 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55fb2f327170 0x55fb2f394460 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3feda0be93f2f916 server_cookie=f0eadf6dd43ff7f in_seq=0 out_seq=0 2024-06-10T03:00:40.301 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.297+0000 7fb1af7fe700 1 -- 172.21.15.52:0/471453211 <== mds.0 v2:172.21.15.188:6838/310444836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fb2f2662d0 con 0x55fb2f327170 2024-06-10T03:00:40.301 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.297+0000 7fb1af7fe700 10 client.25997 handle_command_reply: tid=0 2024-06-10T03:00:40.301 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.297+0000 7fb1e0f94b80 1 client.25997 shutdown 2024-06-10T03:00:40.302 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.297+0000 7fb1e0f94b80 1 -- 172.21.15.52:0/471453211 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fb2f312f60 msgr2=0x55fb2f32e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:00:40.302 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.297+0000 7fb1e0f94b80 1 --2- 172.21.15.52:0/471453211 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fb2f312f60 0x55fb2f32e870 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fb1c4098cc0 tx=0x7fb1c4005b00 comp rx=0 tx=0).stop 2024-06-10T03:00:40.302 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.297+0000 7fb1e0f94b80 1 -- 172.21.15.52:0/471453211 shutdown_connections 2024-06-10T03:00:40.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.297+0000 7fb1e0f94b80 1 --2- 172.21.15.52:0/471453211 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55fb2f327170 0x55fb2f394460 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:40.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.297+0000 7fb1e0f94b80 1 --2- 172.21.15.52:0/471453211 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fb2f313880 0x55fb2f32edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:40.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.297+0000 7fb1e0f94b80 1 --2- 172.21.15.52:0/471453211 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fb2f30d7a0 0x55fb2f31fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:40.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.297+0000 7fb1af7fe700 0 client.25997 ms_handle_reset on v2:172.21.15.188:6838/310444836 2024-06-10T03:00:40.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.297+0000 7fb1e0f94b80 1 --2- 172.21.15.52:0/471453211 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fb2f312f60 0x55fb2f32e870 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:40.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.297+0000 7fb1e0f94b80 1 -- 172.21.15.52:0/471453211 >> 172.21.15.52:0/471453211 conn(0x55fb2f23a080 msgr2=0x55fb2f30c250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:00:40.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.297+0000 7fb1e0f94b80 1 -- 172.21.15.52:0/471453211 shutdown_connections 2024-06-10T03:00:40.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.297+0000 7fb1e0f94b80 1 -- 172.21.15.52:0/471453211 wait complete. 2024-06-10T03:00:40.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.298+0000 7fb1e0f94b80 20 client.25997 trim_cache size 0 max 16384 2024-06-10T03:00:40.303 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-06-10T03:00:40.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.298+0000 7fb1d6711700 1 -- 172.21.15.52:0/179752865 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fb1b80a2c20 msgr2=0x7fb1b80a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:00:40.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.298+0000 7fb1d6711700 1 --2- 172.21.15.52:0/179752865 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fb1b80a2c20 0x7fb1b80a50e0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fb1d0100cd0 tx=0x7fb1c000a380 comp rx=0 tx=0).stop 2024-06-10T03:00:40.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.298+0000 7fb1d6711700 1 -- 172.21.15.52:0/179752865 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb1d0102ed0 msgr2=0x7fb1d019ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:00:40.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.298+0000 7fb1d6711700 1 --2- 172.21.15.52:0/179752865 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb1d0102ed0 0x7fb1d019ed20 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fb1c4000c00 tx=0x7fb1c400bab0 comp rx=0 tx=0).stop 2024-06-10T03:00:40.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.298+0000 7fb1d6711700 1 -- 172.21.15.52:0/179752865 shutdown_connections 2024-06-10T03:00:40.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.298+0000 7fb1d6711700 1 --2- 172.21.15.52:0/179752865 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb1d0108520 0x7fb1d01989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:40.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.298+0000 7fb1d6711700 1 --2- 172.21.15.52:0/179752865 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb1d0102510 0x7fb1d019e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:40.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.298+0000 7fb1d6711700 1 --2- 172.21.15.52:0/179752865 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fb1b80a2c20 0x7fb1b80a50e0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:40.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.298+0000 7fb1d6711700 1 --2- 172.21.15.52:0/179752865 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb1d0102ed0 0x7fb1d019ed20 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:40.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.298+0000 7fb1d6711700 1 -- 172.21.15.52:0/179752865 >> 172.21.15.52:0/179752865 conn(0x7fb1d0079560 msgr2=0x7fb1d0100fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:00:40.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.299+0000 7fb1d6711700 1 -- 172.21.15.52:0/179752865 shutdown_connections 2024-06-10T03:00:40.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:40.299+0000 7fb1d6711700 1 -- 172.21.15.52:0/179752865 wait complete. 2024-06-10T03:00:40.476 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:00:40 smithi052 ceph-mon[32362]: pgmap v628: 129 pgs: 129 active+clean; 568 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 1.2 MiB/s wr, 659 op/s 2024-06-10T03:00:40.638 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:00:40 smithi120 ceph-mon[36634]: pgmap v628: 129 pgs: 129 active+clean; 568 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 1.2 MiB/s wr, 659 op/s 2024-06-10T03:00:40.772 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:00:40 smithi188 ceph-mon[36599]: pgmap v628: 129 pgs: 129 active+clean; 568 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 1.2 MiB/s wr, 659 op/s 2024-06-10T03:00:41.947 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-06-10T03:00:41.947 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-06-10T03:00:42.244 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T03:00:42.278 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:00:42 smithi052 ceph-mon[32362]: pgmap v629: 129 pgs: 129 active+clean; 568 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 1.0 MiB/s wr, 677 op/s 2024-06-10T03:00:42.638 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:00:42 smithi120 ceph-mon[36634]: pgmap v629: 129 pgs: 129 active+clean; 568 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 1.0 MiB/s wr, 677 op/s 2024-06-10T03:00:42.703 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:00:42 smithi188 ceph-mon[36599]: pgmap v629: 129 pgs: 129 active+clean; 568 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 1.0 MiB/s wr, 677 op/s 2024-06-10T03:00:43.113 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T03:00:43.449 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T03:00:43.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.931+0000 7f678f441700 1 -- 172.21.15.52:0/1290776248 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6788102030 msgr2=0x7f6788105ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:00:43.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.931+0000 7f678f441700 1 --2- 172.21.15.52:0/1290776248 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6788102030 0x7f6788105ff0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f67780099c0 tx=0x7f6778009cd0 comp rx=0 tx=0).stop 2024-06-10T03:00:43.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.931+0000 7f678f441700 1 -- 172.21.15.52:0/1290776248 shutdown_connections 2024-06-10T03:00:43.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.931+0000 7f678f441700 1 --2- 172.21.15.52:0/1290776248 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f67881066d0 0x7f678810e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:43.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.931+0000 7f678f441700 1 --2- 172.21.15.52:0/1290776248 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6788101680 0x7f6788101a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:43.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.931+0000 7f678f441700 1 --2- 172.21.15.52:0/1290776248 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6788102030 0x7f6788105ff0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:43.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.931+0000 7f678f441700 1 -- 172.21.15.52:0/1290776248 >> 172.21.15.52:0/1290776248 conn(0x7f67880773e0 msgr2=0x7f67880777f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:00:43.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.931+0000 7f678f441700 1 -- 172.21.15.52:0/1290776248 shutdown_connections 2024-06-10T03:00:43.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.931+0000 7f678f441700 1 -- 172.21.15.52:0/1290776248 wait complete. 2024-06-10T03:00:43.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.932+0000 7f678f441700 1 Processor -- start 2024-06-10T03:00:43.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.932+0000 7f678f441700 1 -- start start 2024-06-10T03:00:43.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.932+0000 7f678f441700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6788101680 0x7f6788103090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:43.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.932+0000 7f678f441700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6788102030 0x7f67881035d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:43.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.932+0000 7f678f441700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f67881066d0 0x7f6788103b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:43.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.932+0000 7f678f441700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f6788111710 con 0x7f67881066d0 2024-06-10T03:00:43.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.932+0000 7f678f441700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f6788111590 con 0x7f6788101680 2024-06-10T03:00:43.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.932+0000 7f678f441700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f6788111890 con 0x7f6788102030 2024-06-10T03:00:43.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.932+0000 7f678d9de700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f67881066d0 0x7f6788103b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:43.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.932+0000 7f678d9de700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f67881066d0 0x7f6788103b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58708/0 (socket says 172.21.15.52:58708) 2024-06-10T03:00:43.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.932+0000 7f678d9de700 1 -- 172.21.15.52:0/993208735 learned_addr learned my addr 172.21.15.52:0/993208735 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:00:43.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.933+0000 7f678d1dd700 1 --2- 172.21.15.52:0/993208735 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6788101680 0x7f6788103090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:43.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.933+0000 7f678c9dc700 1 --2- 172.21.15.52:0/993208735 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6788102030 0x7f67881035d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:43.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.933+0000 7f678d9de700 1 -- 172.21.15.52:0/993208735 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6788102030 msgr2=0x7f67881035d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:00:43.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.933+0000 7f678d9de700 1 --2- 172.21.15.52:0/993208735 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6788102030 0x7f67881035d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:43.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.933+0000 7f678d9de700 1 -- 172.21.15.52:0/993208735 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6788101680 msgr2=0x7f6788103090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:00:43.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.933+0000 7f678d9de700 1 --2- 172.21.15.52:0/993208735 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6788101680 0x7f6788103090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:43.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.933+0000 7f678d9de700 1 -- 172.21.15.52:0/993208735 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67881a7220 con 0x7f67881066d0 2024-06-10T03:00:43.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.933+0000 7f678d1dd700 1 --2- 172.21.15.52:0/993208735 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6788101680 0x7f6788103090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T03:00:43.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.933+0000 7f678d9de700 1 --2- 172.21.15.52:0/993208735 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f67881066d0 0x7f6788103b10 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f678000b970 tx=0x7f678000bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:00:43.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.933+0000 7f677e7fc700 1 -- 172.21.15.52:0/993208735 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6780019070 con 0x7f67881066d0 2024-06-10T03:00:43.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.933+0000 7f677e7fc700 1 -- 172.21.15.52:0/993208735 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6780014440 con 0x7f67881066d0 2024-06-10T03:00:43.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.933+0000 7f678f441700 1 -- 172.21.15.52:0/993208735 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67881a7510 con 0x7f67881066d0 2024-06-10T03:00:43.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.933+0000 7f677e7fc700 1 -- 172.21.15.52:0/993208735 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67800075d0 con 0x7f67881066d0 2024-06-10T03:00:43.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.933+0000 7f678f441700 1 -- 172.21.15.52:0/993208735 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67881a7a20 con 0x7f67881066d0 2024-06-10T03:00:43.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.936+0000 7f677e7fc700 1 -- 172.21.15.52:0/993208735 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f678001c070 con 0x7f67881066d0 2024-06-10T03:00:43.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.936+0000 7f677e7fc700 1 --2- 172.21.15.52:0/993208735 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f67740a2c20 0x7f67740a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:43.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.936+0000 7f677e7fc700 1 -- 172.21.15.52:0/993208735 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f67800ce800 con 0x7f67881066d0 2024-06-10T03:00:43.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.936+0000 7f678d1dd700 1 --2- 172.21.15.52:0/993208735 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f67740a2c20 0x7f67740a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:43.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.937+0000 7f678d1dd700 1 --2- 172.21.15.52:0/993208735 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f67740a2c20 0x7f67740a50e0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f6784005fd0 tx=0x7f6784005e30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:00:43.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.937+0000 7f6799cc4b80 1 Processor -- start 2024-06-10T03:00:43.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.937+0000 7f6799cc4b80 1 -- start start 2024-06-10T03:00:43.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.937+0000 7f6799cc4b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559d2f53bfe0 0x559d2f53c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:43.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.937+0000 7f6799cc4b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559d2f53c900 0x559d2f5362e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:43.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.938+0000 7f6799cc4b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559d2f536820 0x559d2f536c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:43.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.938+0000 7f6799cc4b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x559d2f428fe0 con 0x559d2f53c900 2024-06-10T03:00:43.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.938+0000 7f6799cc4b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x559d2f4878d0 con 0x559d2f53bfe0 2024-06-10T03:00:43.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.938+0000 7f6799cc4b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x559d2f3ef4e0 con 0x559d2f536820 2024-06-10T03:00:43.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.938+0000 7f678c9dc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559d2f53c900 0x559d2f5362e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:43.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.938+0000 7f678c9dc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559d2f53c900 0x559d2f5362e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58710/0 (socket says 172.21.15.52:58710) 2024-06-10T03:00:43.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.938+0000 7f678c9dc700 1 -- 172.21.15.52:0/1264194522 learned_addr learned my addr 172.21.15.52:0/1264194522 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:00:43.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.938+0000 7f678d1dd700 1 --2- 172.21.15.52:0/1264194522 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559d2f53bfe0 0x559d2f53c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:43.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.938+0000 7f678d9de700 1 --2- 172.21.15.52:0/1264194522 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559d2f536820 0x559d2f536c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:43.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.938+0000 7f678c9dc700 1 -- 172.21.15.52:0/1264194522 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559d2f536820 msgr2=0x559d2f536c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:00:43.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.938+0000 7f678c9dc700 1 --2- 172.21.15.52:0/1264194522 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559d2f536820 0x559d2f536c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:43.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.938+0000 7f678c9dc700 1 -- 172.21.15.52:0/1264194522 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559d2f53bfe0 msgr2=0x559d2f53c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:00:43.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.938+0000 7f678c9dc700 1 --2- 172.21.15.52:0/1264194522 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559d2f53bfe0 0x559d2f53c3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:43.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.938+0000 7f678c9dc700 1 -- 172.21.15.52:0/1264194522 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d2f3d6950 con 0x559d2f53c900 2024-06-10T03:00:43.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.938+0000 7f678c9dc700 1 --2- 172.21.15.52:0/1264194522 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559d2f53c900 0x559d2f5362e0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f67780099c0 tx=0x7f6778005c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4cd5a1f587cf241 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:00:43.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.938+0000 7f67737fe700 1 -- 172.21.15.52:0/1264194522 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6778027070 con 0x559d2f53c900 2024-06-10T03:00:43.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.938+0000 7f67737fe700 1 -- 172.21.15.52:0/1264194522 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6778005550 con 0x559d2f53c900 2024-06-10T03:00:43.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.938+0000 7f67737fe700 1 -- 172.21.15.52:0/1264194522 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f677800bb20 con 0x559d2f53c900 2024-06-10T03:00:43.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.939+0000 7f6799cc4b80 1 -- 172.21.15.52:0/1264194522 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559d2f53c900 msgr2=0x559d2f5362e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:00:43.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.939+0000 7f6799cc4b80 1 --2- 172.21.15.52:0/1264194522 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559d2f53c900 0x559d2f5362e0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f67780099c0 tx=0x7f6778005c80 comp rx=0 tx=0).stop 2024-06-10T03:00:43.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.939+0000 7f6799cc4b80 1 -- 172.21.15.52:0/1264194522 shutdown_connections 2024-06-10T03:00:43.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.939+0000 7f6799cc4b80 1 --2- 172.21.15.52:0/1264194522 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559d2f536820 0x559d2f536c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:43.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.939+0000 7f6799cc4b80 1 --2- 172.21.15.52:0/1264194522 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559d2f53bfe0 0x559d2f53c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:43.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.939+0000 7f6799cc4b80 1 --2- 172.21.15.52:0/1264194522 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559d2f53c900 0x559d2f5362e0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:43.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.939+0000 7f6799cc4b80 1 -- 172.21.15.52:0/1264194522 >> 172.21.15.52:0/1264194522 conn(0x559d2f463100 msgr2=0x559d2f5312a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:00:43.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.939+0000 7f6799cc4b80 1 -- 172.21.15.52:0/1264194522 shutdown_connections 2024-06-10T03:00:43.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.939+0000 7f6799cc4b80 1 -- 172.21.15.52:0/1264194522 wait complete. 2024-06-10T03:00:43.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.940+0000 7f6799cc4b80 1 -- start start 2024-06-10T03:00:43.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.940+0000 7f6799cc4b80 1 Processor -- start 2024-06-10T03:00:43.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.940+0000 7f6799cc4b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559d2f536820 0x559d2f542700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:43.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.940+0000 7f6799cc4b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559d2f53bfe0 0x559d2f53fa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:43.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.940+0000 7f6799cc4b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559d2f53c900 0x559d2f53ff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:43.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.940+0000 7f6799cc4b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x559d2f428040 con 0x559d2f53c900 2024-06-10T03:00:43.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.940+0000 7f6799cc4b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x559d2f483890 con 0x559d2f53bfe0 2024-06-10T03:00:43.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.940+0000 7f6799cc4b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x559d2f543280 con 0x559d2f536820 2024-06-10T03:00:43.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.940+0000 7f6799cc4b80 10 client.0 _ll_register_callbacks cb 0x559d2f41d630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T03:00:43.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.940+0000 7f678d9de700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559d2f53c900 0x559d2f53ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:43.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.940+0000 7f678d9de700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559d2f53c900 0x559d2f53ff40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58726/0 (socket says 172.21.15.52:58726) 2024-06-10T03:00:43.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.940+0000 7f678d9de700 1 -- 172.21.15.52:0/3295381328 learned_addr learned my addr 172.21.15.52:0/3295381328 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:00:43.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.940+0000 7f678c9dc700 1 --2- 172.21.15.52:0/3295381328 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559d2f53bfe0 0x559d2f53fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:43.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.940+0000 7f678d1dd700 1 --2- 172.21.15.52:0/3295381328 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559d2f536820 0x559d2f542700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:43.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.941+0000 7f678d9de700 1 -- 172.21.15.52:0/3295381328 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559d2f536820 msgr2=0x559d2f542700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:00:43.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.941+0000 7f678d9de700 1 --2- 172.21.15.52:0/3295381328 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559d2f536820 0x559d2f542700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:43.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.941+0000 7f678d9de700 1 -- 172.21.15.52:0/3295381328 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559d2f53bfe0 msgr2=0x559d2f53fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:00:43.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.941+0000 7f678d9de700 1 --2- 172.21.15.52:0/3295381328 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559d2f53bfe0 0x559d2f53fa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:43.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.941+0000 7f678d9de700 1 -- 172.21.15.52:0/3295381328 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d2f44d7d0 con 0x559d2f53c900 2024-06-10T03:00:43.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.941+0000 7f678d9de700 1 --2- 172.21.15.52:0/3295381328 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559d2f53c900 0x559d2f53ff40 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f678009ac40 tx=0x7f67800148d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b305dbb93f2434f2 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:00:43.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.941+0000 7f6770ff9700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-10T03:00:43.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.941+0000 7f6770ff9700 1 -- 172.21.15.52:0/3295381328 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f678009f060 con 0x559d2f53c900 2024-06-10T03:00:43.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.941+0000 7f6770ff9700 1 -- 172.21.15.52:0/3295381328 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f6780014960 con 0x559d2f53c900 2024-06-10T03:00:43.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.941+0000 7f6799cc4b80 1 -- 172.21.15.52:0/3295381328 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6778005550 con 0x559d2f53c900 2024-06-10T03:00:43.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.941+0000 7f6770ff9700 1 -- 172.21.15.52:0/3295381328 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f67800a3920 con 0x559d2f53c900 2024-06-10T03:00:43.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.941+0000 7f6799cc4b80 10 client.16470 fetch_fsmap learned FSMap version 36 2024-06-10T03:00:43.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.941+0000 7f6799cc4b80 1 -- 172.21.15.52:0/3295381328 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x559d2f54db40 con 0x559d2f53c900 2024-06-10T03:00:43.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.941+0000 7f6770ff9700 1 -- 172.21.15.52:0/3295381328 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6778005550 con 0x559d2f53c900 2024-06-10T03:00:43.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.942+0000 7f6770ff9700 1 -- 172.21.15.52:0/3295381328 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 36) ==== 8378+0+0 (secure 0 0 0) 0x7f678009a100 con 0x559d2f53c900 2024-06-10T03:00:43.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.942+0000 7f6799cc4b80 10 client.16470 fetch_fsmap finished waiting for FSMap version 36 2024-06-10T03:00:43.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.942+0000 7f6799cc4b80 10 client.16470 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T03:00:43.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.942+0000 7f6799cc4b80 20 client.16470 populate_metadata read hostname 'smithi052' 2024-06-10T03:00:43.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.942+0000 7f6799cc4b80 1 --2- 172.21.15.52:0/3295381328 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x559d2f550180 0x559d2f552570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:43.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.942+0000 7f6799cc4b80 4 client.16470 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T03:00:43.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.942+0000 7f6799cc4b80 1 -- 172.21.15.52:0/3295381328 --> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559d2f494dd0 con 0x559d2f550180 2024-06-10T03:00:43.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.942+0000 7f678d1dd700 1 --2- 172.21.15.52:0/3295381328 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x559d2f550180 0x559d2f552570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:43.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.942+0000 7f678d1dd700 1 --2- 172.21.15.52:0/3295381328 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x559d2f550180 0x559d2f552570 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=90b22f4a81b72b00 server_cookie=d7a0de902667b4f1 in_seq=0 out_seq=0 2024-06-10T03:00:43.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.942+0000 7f6770ff9700 10 client.16470 ms_handle_connect on v2:172.21.15.188:6838/310444836 2024-06-10T03:00:43.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.943+0000 7f6770ff9700 1 -- 172.21.15.52:0/3295381328 <== mds.0 v2:172.21.15.188:6838/310444836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x559d2f494dd0 con 0x559d2f550180 2024-06-10T03:00:43.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.943+0000 7f6770ff9700 10 client.16470 handle_command_reply: tid=0 2024-06-10T03:00:43.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.943+0000 7f6799cc4b80 1 client.16470 shutdown 2024-06-10T03:00:43.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.943+0000 7f6799cc4b80 1 -- 172.21.15.52:0/3295381328 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559d2f53c900 msgr2=0x559d2f53ff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:00:43.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.943+0000 7f6799cc4b80 1 --2- 172.21.15.52:0/3295381328 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559d2f53c900 0x559d2f53ff40 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f678009ac40 tx=0x7f67800148d0 comp rx=0 tx=0).stop 2024-06-10T03:00:43.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.943+0000 7f6799cc4b80 1 -- 172.21.15.52:0/3295381328 shutdown_connections 2024-06-10T03:00:43.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.943+0000 7f6799cc4b80 1 --2- 172.21.15.52:0/3295381328 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x559d2f550180 0x559d2f552570 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:43.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.943+0000 7f6799cc4b80 1 --2- 172.21.15.52:0/3295381328 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559d2f53c900 0x559d2f53ff40 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:43.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.943+0000 7f6799cc4b80 1 --2- 172.21.15.52:0/3295381328 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559d2f536820 0x559d2f542700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:43.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.943+0000 7f6799cc4b80 1 --2- 172.21.15.52:0/3295381328 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559d2f53bfe0 0x559d2f53fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:43.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.943+0000 7f6799cc4b80 1 -- 172.21.15.52:0/3295381328 >> 172.21.15.52:0/3295381328 conn(0x559d2f463100 msgr2=0x559d2f530780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:00:43.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.943+0000 7f6770ff9700 0 client.16470 ms_handle_reset on v2:172.21.15.188:6838/310444836 2024-06-10T03:00:43.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.943+0000 7f6799cc4b80 1 -- 172.21.15.52:0/3295381328 shutdown_connections 2024-06-10T03:00:43.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.944+0000 7f6799cc4b80 1 -- 172.21.15.52:0/3295381328 wait complete. 2024-06-10T03:00:43.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.944+0000 7f6799cc4b80 20 client.16470 trim_cache size 0 max 16384 2024-06-10T03:00:43.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.965+0000 7f6799cc4b80 1 Processor -- start 2024-06-10T03:00:43.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.965+0000 7f6799cc4b80 1 -- start start 2024-06-10T03:00:43.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.965+0000 7f6799cc4b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559d2f536820 0x559d2f542240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:43.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.965+0000 7f6799cc4b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559d2f53bfe0 0x559d2f530f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:43.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.965+0000 7f6799cc4b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559d2f53c900 0x559d2f531440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:43.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.965+0000 7f6799cc4b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f67881b04e0 con 0x559d2f53c900 2024-06-10T03:00:43.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.965+0000 7f6799cc4b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f67881b20e0 con 0x559d2f536820 2024-06-10T03:00:43.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.965+0000 7f6799cc4b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f67881b0bb0 con 0x559d2f53bfe0 2024-06-10T03:00:43.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.965+0000 7f678d9de700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559d2f53c900 0x559d2f531440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:43.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.965+0000 7f678d9de700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559d2f53c900 0x559d2f531440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58740/0 (socket says 172.21.15.52:58740) 2024-06-10T03:00:43.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.965+0000 7f678d9de700 1 -- 172.21.15.52:0/3934448839 learned_addr learned my addr 172.21.15.52:0/3934448839 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:00:43.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.965+0000 7f678d1dd700 1 --2- 172.21.15.52:0/3934448839 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559d2f536820 0x559d2f542240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:43.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.965+0000 7f678c9dc700 1 --2- 172.21.15.52:0/3934448839 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559d2f53bfe0 0x559d2f530f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:43.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.965+0000 7f678d9de700 1 -- 172.21.15.52:0/3934448839 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559d2f53bfe0 msgr2=0x559d2f530f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:00:43.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.965+0000 7f678d9de700 1 --2- 172.21.15.52:0/3934448839 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559d2f53bfe0 0x559d2f530f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:43.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.965+0000 7f678d9de700 1 -- 172.21.15.52:0/3934448839 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559d2f536820 msgr2=0x559d2f542240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:00:43.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.965+0000 7f678d9de700 1 --2- 172.21.15.52:0/3934448839 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559d2f536820 0x559d2f542240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:43.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.965+0000 7f678d9de700 1 -- 172.21.15.52:0/3934448839 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d2f552b50 con 0x559d2f53c900 2024-06-10T03:00:43.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.965+0000 7f678d9de700 1 --2- 172.21.15.52:0/3934448839 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559d2f53c900 0x559d2f531440 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f678000b940 tx=0x7f67800991c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4795759bac63e179 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:00:43.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.966+0000 7f6772ffd700 1 -- 172.21.15.52:0/3934448839 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f678009f060 con 0x559d2f53c900 2024-06-10T03:00:43.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.966+0000 7f6772ffd700 1 -- 172.21.15.52:0/3934448839 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6780014960 con 0x559d2f53c900 2024-06-10T03:00:43.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.966+0000 7f6772ffd700 1 -- 172.21.15.52:0/3934448839 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6780096a00 con 0x559d2f53c900 2024-06-10T03:00:43.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.966+0000 7f6799cc4b80 1 -- 172.21.15.52:0/3934448839 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559d2f53c900 msgr2=0x559d2f531440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:00:43.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.966+0000 7f6799cc4b80 1 --2- 172.21.15.52:0/3934448839 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559d2f53c900 0x559d2f531440 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f678000b940 tx=0x7f67800991c0 comp rx=0 tx=0).stop 2024-06-10T03:00:43.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.967+0000 7f6799cc4b80 1 -- 172.21.15.52:0/3934448839 shutdown_connections 2024-06-10T03:00:43.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.967+0000 7f6799cc4b80 1 --2- 172.21.15.52:0/3934448839 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559d2f53c900 0x559d2f531440 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:43.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.967+0000 7f6799cc4b80 1 --2- 172.21.15.52:0/3934448839 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559d2f536820 0x559d2f542240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:43.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.967+0000 7f6799cc4b80 1 --2- 172.21.15.52:0/3934448839 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559d2f53bfe0 0x559d2f530f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:43.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.967+0000 7f6799cc4b80 1 -- 172.21.15.52:0/3934448839 >> 172.21.15.52:0/3934448839 conn(0x559d2f463100 msgr2=0x559d2f53abf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:00:43.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.967+0000 7f6799cc4b80 1 -- 172.21.15.52:0/3934448839 shutdown_connections 2024-06-10T03:00:43.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.967+0000 7f6799cc4b80 1 -- 172.21.15.52:0/3934448839 wait complete. 2024-06-10T03:00:43.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.967+0000 7f6799cc4b80 1 -- start start 2024-06-10T03:00:43.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.967+0000 7f6799cc4b80 1 Processor -- start 2024-06-10T03:00:43.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.967+0000 7f6799cc4b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559d2f536820 0x559d2f548cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:43.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.967+0000 7f6799cc4b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559d2f53bfe0 0x559d2f545540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:43.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.967+0000 7f6799cc4b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559d2f53c900 0x559d2f545a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:43.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.967+0000 7f6799cc4b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f67881b1a10 con 0x559d2f53c900 2024-06-10T03:00:43.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.967+0000 7f6799cc4b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f67881b1340 con 0x559d2f53bfe0 2024-06-10T03:00:43.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.967+0000 7f6799cc4b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f67881ae480 con 0x559d2f536820 2024-06-10T03:00:43.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.967+0000 7f6799cc4b80 10 client.0 _ll_register_callbacks cb 0x559d2f41d630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T03:00:43.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.968+0000 7f678d9de700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559d2f53c900 0x559d2f545a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:43.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.968+0000 7f678d1dd700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559d2f536820 0x559d2f548cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:43.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.968+0000 7f678c9dc700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559d2f53bfe0 0x559d2f545540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:43.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.968+0000 7f678d1dd700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559d2f536820 0x559d2f548cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.52:56712/0 (socket says 172.21.15.52:56712) 2024-06-10T03:00:43.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.968+0000 7f678d1dd700 1 -- 172.21.15.52:0/1208570000 learned_addr learned my addr 172.21.15.52:0/1208570000 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:00:43.973 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.968+0000 7f678c9dc700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559d2f53bfe0 0x559d2f545540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.52:56236/0 (socket says 172.21.15.52:56236) 2024-06-10T03:00:43.973 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.968+0000 7f678d1dd700 1 -- 172.21.15.52:0/1208570000 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559d2f53bfe0 msgr2=0x559d2f545540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:00:43.973 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.968+0000 7f678d1dd700 1 --2- 172.21.15.52:0/1208570000 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559d2f53bfe0 0x559d2f545540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:43.973 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.968+0000 7f678d1dd700 1 -- 172.21.15.52:0/1208570000 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559d2f53c900 msgr2=0x559d2f545a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:00:43.973 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.968+0000 7f678d1dd700 1 --2- 172.21.15.52:0/1208570000 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559d2f53c900 0x559d2f545a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:43.973 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.968+0000 7f678d1dd700 1 -- 172.21.15.52:0/1208570000 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d2f5462c0 con 0x559d2f536820 2024-06-10T03:00:43.973 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.968+0000 7f678c9dc700 1 --2- 172.21.15.52:0/1208570000 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559d2f53bfe0 0x559d2f545540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T03:00:43.973 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.968+0000 7f678d9de700 1 --2- 172.21.15.52:0/1208570000 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559d2f53c900 0x559d2f545a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T03:00:43.974 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.968+0000 7f678d1dd700 1 --2- 172.21.15.52:0/1208570000 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559d2f536820 0x559d2f548cf0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f6784005ea0 tx=0x7f6784004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cb001aeebbd33528 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:00:43.974 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.968+0000 7f6770ff9700 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-06-10T03:00:43.974 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.968+0000 7f6770ff9700 1 -- 172.21.15.52:0/1208570000 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f678401ee20 con 0x559d2f536820 2024-06-10T03:00:43.974 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.968+0000 7f6770ff9700 1 -- 172.21.15.52:0/1208570000 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f678402a630 con 0x559d2f536820 2024-06-10T03:00:43.974 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.968+0000 7f6799cc4b80 1 -- 172.21.15.52:0/1208570000 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6780014960 con 0x559d2f536820 2024-06-10T03:00:43.974 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.969+0000 7f6770ff9700 1 -- 172.21.15.52:0/1208570000 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f678403b050 con 0x559d2f536820 2024-06-10T03:00:43.974 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.969+0000 7f6799cc4b80 10 client.26035 fetch_fsmap learned FSMap version 36 2024-06-10T03:00:43.974 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.969+0000 7f6799cc4b80 1 -- 172.21.15.52:0/1208570000 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x559d2f5bbf50 con 0x559d2f536820 2024-06-10T03:00:43.974 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.969+0000 7f6770ff9700 1 -- 172.21.15.52:0/1208570000 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6780014960 con 0x559d2f536820 2024-06-10T03:00:43.974 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.969+0000 7f6770ff9700 1 -- 172.21.15.52:0/1208570000 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 36) ==== 8378+0+0 (secure 0 0 0) 0x7f6784010ba0 con 0x559d2f536820 2024-06-10T03:00:43.974 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.969+0000 7f6799cc4b80 10 client.26035 fetch_fsmap finished waiting for FSMap version 36 2024-06-10T03:00:43.974 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.969+0000 7f6799cc4b80 10 client.26035 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T03:00:43.975 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.969+0000 7f6799cc4b80 20 client.26035 populate_metadata read hostname 'smithi052' 2024-06-10T03:00:43.975 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.969+0000 7f6799cc4b80 1 --2- 172.21.15.52:0/1208570000 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x559d2f5bc600 0x559d2f5bea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:43.975 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.969+0000 7f6799cc4b80 4 client.26035 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-06-10T03:00:43.975 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.969+0000 7f6799cc4b80 1 -- 172.21.15.52:0/1208570000 --> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x559d2f48f350 con 0x559d2f5bc600 2024-06-10T03:00:43.975 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.969+0000 7f678c9dc700 1 --2- 172.21.15.52:0/1208570000 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x559d2f5bc600 0x559d2f5bea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:43.975 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.970+0000 7f678c9dc700 1 --2- 172.21.15.52:0/1208570000 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x559d2f5bc600 0x559d2f5bea20 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bccf14f10b962f86 server_cookie=24c9d635eaf6ba0 in_seq=0 out_seq=0 2024-06-10T03:00:43.975 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.970+0000 7f6770ff9700 10 client.26035 ms_handle_connect on v2:172.21.15.188:6838/310444836 2024-06-10T03:00:43.976 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.971+0000 7f6770ff9700 1 -- 172.21.15.52:0/1208570000 <== mds.0 v2:172.21.15.188:6838/310444836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x559d2f48f350 con 0x559d2f5bc600 2024-06-10T03:00:43.976 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.971+0000 7f6770ff9700 10 client.26035 handle_command_reply: tid=0 2024-06-10T03:00:43.976 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.971+0000 7f6799cc4b80 1 client.26035 shutdown 2024-06-10T03:00:43.976 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.971+0000 7f6799cc4b80 1 -- 172.21.15.52:0/1208570000 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559d2f536820 msgr2=0x559d2f548cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:00:43.976 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.971+0000 7f6799cc4b80 1 --2- 172.21.15.52:0/1208570000 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559d2f536820 0x559d2f548cf0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f6784005ea0 tx=0x7f6784004750 comp rx=0 tx=0).stop 2024-06-10T03:00:43.976 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.971+0000 7f6799cc4b80 1 -- 172.21.15.52:0/1208570000 shutdown_connections 2024-06-10T03:00:43.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.972+0000 7f6799cc4b80 1 --2- 172.21.15.52:0/1208570000 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x559d2f5bc600 0x559d2f5bea20 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:43.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.972+0000 7f6799cc4b80 1 --2- 172.21.15.52:0/1208570000 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559d2f53c900 0x559d2f545a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:43.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.972+0000 7f6799cc4b80 1 --2- 172.21.15.52:0/1208570000 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559d2f536820 0x559d2f548cf0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:43.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.972+0000 7f6799cc4b80 1 --2- 172.21.15.52:0/1208570000 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559d2f53bfe0 0x559d2f545540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:43.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.972+0000 7f6799cc4b80 1 -- 172.21.15.52:0/1208570000 >> 172.21.15.52:0/1208570000 conn(0x559d2f463100 msgr2=0x559d2f53b480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:00:43.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.972+0000 7f6770ff9700 0 client.26035 ms_handle_reset on v2:172.21.15.188:6838/310444836 2024-06-10T03:00:43.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.972+0000 7f6799cc4b80 1 -- 172.21.15.52:0/1208570000 shutdown_connections 2024-06-10T03:00:43.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.972+0000 7f6799cc4b80 1 -- 172.21.15.52:0/1208570000 wait complete. 2024-06-10T03:00:43.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.972+0000 7f6799cc4b80 20 client.26035 trim_cache size 0 max 16384 2024-06-10T03:00:43.977 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-10T03:00:43.978 INFO:teuthology.orchestra.run.smithi052.stdout: "return_code": 0, 2024-06-10T03:00:43.978 INFO:teuthology.orchestra.run.smithi052.stdout: "scrub_tag": "83aa1863-a6f4-45c9-a634-54ed2a6509bb", 2024-06-10T03:00:43.978 INFO:teuthology.orchestra.run.smithi052.stdout: "mode": "asynchronous" 2024-06-10T03:00:43.978 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-10T03:00:43.978 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.972+0000 7f678f441700 1 -- 172.21.15.52:0/993208735 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f67740a2c20 msgr2=0x7f67740a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:00:43.978 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.972+0000 7f678f441700 1 --2- 172.21.15.52:0/993208735 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f67740a2c20 0x7f67740a50e0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f6784005fd0 tx=0x7f6784005e30 comp rx=0 tx=0).stop 2024-06-10T03:00:43.979 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.972+0000 7f678f441700 1 -- 172.21.15.52:0/993208735 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f67881066d0 msgr2=0x7f6788103b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:00:43.979 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.972+0000 7f678f441700 1 --2- 172.21.15.52:0/993208735 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f67881066d0 0x7f6788103b10 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f678000b970 tx=0x7f678000bd30 comp rx=0 tx=0).stop 2024-06-10T03:00:43.979 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.973+0000 7f678f441700 1 -- 172.21.15.52:0/993208735 shutdown_connections 2024-06-10T03:00:43.979 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.973+0000 7f678f441700 1 --2- 172.21.15.52:0/993208735 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f67740a2c20 0x7f67740a50e0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:43.979 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.973+0000 7f678f441700 1 --2- 172.21.15.52:0/993208735 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f67881066d0 0x7f6788103b10 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:43.979 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.973+0000 7f678f441700 1 --2- 172.21.15.52:0/993208735 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6788101680 0x7f6788103090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:43.979 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.973+0000 7f678f441700 1 --2- 172.21.15.52:0/993208735 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6788102030 0x7f67881035d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:43.979 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.973+0000 7f678f441700 1 -- 172.21.15.52:0/993208735 >> 172.21.15.52:0/993208735 conn(0x7f67880773e0 msgr2=0x7f678806ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:00:43.979 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.973+0000 7f678f441700 1 -- 172.21.15.52:0/993208735 shutdown_connections 2024-06-10T03:00:43.979 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:43.973+0000 7f678f441700 1 -- 172.21.15.52:0/993208735 wait complete. 2024-06-10T03:00:44.476 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:00:44 smithi052 ceph-mon[32362]: pgmap v630: 129 pgs: 129 active+clean; 571 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 1.2 MiB/s wr, 805 op/s 2024-06-10T03:00:44.639 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:00:44 smithi120 ceph-mon[36634]: pgmap v630: 129 pgs: 129 active+clean; 571 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 1.2 MiB/s wr, 805 op/s 2024-06-10T03:00:44.703 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:00:44 smithi188 ceph-mon[36599]: pgmap v630: 129 pgs: 129 active+clean; 571 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 1.2 MiB/s wr, 805 op/s 2024-06-10T03:00:45.549 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:00:45 smithi052 ceph-mon[32362]: scrub queued for path: / 2024-06-10T03:00:45.550 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:00:45 smithi052 ceph-mon[32362]: scrub summary: idle+waiting paths [/] 2024-06-10T03:00:45.550 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:00:45 smithi052 ceph-mon[32362]: scrub summary: active paths [/] 2024-06-10T03:00:45.550 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:00:45 smithi052 ceph-mon[32362]: scrub summary: active paths [/] 2024-06-10T03:00:45.550 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:00:45 smithi052 ceph-mon[32362]: scrub summary: idle+waiting paths [/] 2024-06-10T03:00:45.550 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:00:45 smithi052 ceph-mon[32362]: scrub summary: active paths [/] 2024-06-10T03:00:45.550 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:00:45 smithi052 ceph-mon[32362]: scrub summary: idle+waiting paths [/] 2024-06-10T03:00:45.550 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:00:45 smithi052 ceph-mon[32362]: scrub summary: active paths [/] 2024-06-10T03:00:45.550 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:00:45 smithi052 ceph-mon[32362]: scrub summary: active paths [/] 2024-06-10T03:00:45.550 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:00:45 smithi052 ceph-mon[32362]: scrub summary: idle+waiting paths [/] 2024-06-10T03:00:45.550 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:00:45 smithi052 ceph-mon[32362]: scrub summary: idle+waiting paths [/] 2024-06-10T03:00:45.550 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:00:45 smithi052 ceph-mon[32362]: scrub summary: active paths [/] 2024-06-10T03:00:45.550 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:00:45 smithi052 ceph-mon[32362]: scrub summary: idle+waiting paths [/] 2024-06-10T03:00:45.550 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:00:45 smithi052 ceph-mon[32362]: scrub summary: active paths [/] 2024-06-10T03:00:45.551 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:00:45 smithi052 ceph-mon[32362]: scrub summary: idle+waiting paths [/] 2024-06-10T03:00:45.551 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:00:45 smithi052 ceph-mon[32362]: scrub summary: active paths [/] 2024-06-10T03:00:45.551 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:00:45 smithi052 ceph-mon[32362]: scrub summary: idle+waiting paths [/] 2024-06-10T03:00:45.551 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:00:45 smithi052 ceph-mon[32362]: scrub summary: active paths [/] 2024-06-10T03:00:45.638 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:00:45 smithi120 ceph-mon[36634]: scrub queued for path: / 2024-06-10T03:00:45.638 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:00:45 smithi120 ceph-mon[36634]: scrub summary: idle+waiting paths [/] 2024-06-10T03:00:45.638 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:00:45 smithi120 ceph-mon[36634]: scrub summary: active paths [/] 2024-06-10T03:00:45.639 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:00:45 smithi120 ceph-mon[36634]: scrub summary: active paths [/] 2024-06-10T03:00:45.639 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:00:45 smithi120 ceph-mon[36634]: scrub summary: idle+waiting paths [/] 2024-06-10T03:00:45.639 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:00:45 smithi120 ceph-mon[36634]: scrub summary: active paths [/] 2024-06-10T03:00:45.639 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:00:45 smithi120 ceph-mon[36634]: scrub summary: idle+waiting paths [/] 2024-06-10T03:00:45.639 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:00:45 smithi120 ceph-mon[36634]: scrub summary: active paths [/] 2024-06-10T03:00:45.639 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:00:45 smithi120 ceph-mon[36634]: scrub summary: active paths [/] 2024-06-10T03:00:45.639 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:00:45 smithi120 ceph-mon[36634]: scrub summary: idle+waiting paths [/] 2024-06-10T03:00:45.639 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:00:45 smithi120 ceph-mon[36634]: scrub summary: idle+waiting paths [/] 2024-06-10T03:00:45.639 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:00:45 smithi120 ceph-mon[36634]: scrub summary: active paths [/] 2024-06-10T03:00:45.639 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:00:45 smithi120 ceph-mon[36634]: scrub summary: idle+waiting paths [/] 2024-06-10T03:00:45.639 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:00:45 smithi120 ceph-mon[36634]: scrub summary: active paths [/] 2024-06-10T03:00:45.639 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:00:45 smithi120 ceph-mon[36634]: scrub summary: idle+waiting paths [/] 2024-06-10T03:00:45.639 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:00:45 smithi120 ceph-mon[36634]: scrub summary: active paths [/] 2024-06-10T03:00:45.640 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:00:45 smithi120 ceph-mon[36634]: scrub summary: idle+waiting paths [/] 2024-06-10T03:00:45.640 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:00:45 smithi120 ceph-mon[36634]: scrub summary: active paths [/] 2024-06-10T03:00:45.704 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:00:45 smithi188 ceph-mon[36599]: scrub queued for path: / 2024-06-10T03:00:45.704 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:00:45 smithi188 ceph-mon[36599]: scrub summary: idle+waiting paths [/] 2024-06-10T03:00:45.704 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:00:45 smithi188 ceph-mon[36599]: scrub summary: active paths [/] 2024-06-10T03:00:45.704 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:00:45 smithi188 ceph-mon[36599]: scrub summary: active paths [/] 2024-06-10T03:00:45.704 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:00:45 smithi188 ceph-mon[36599]: scrub summary: idle+waiting paths [/] 2024-06-10T03:00:45.704 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:00:45 smithi188 ceph-mon[36599]: scrub summary: active paths [/] 2024-06-10T03:00:45.704 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:00:45 smithi188 ceph-mon[36599]: scrub summary: idle+waiting paths [/] 2024-06-10T03:00:45.704 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:00:45 smithi188 ceph-mon[36599]: scrub summary: active paths [/] 2024-06-10T03:00:45.704 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:00:45 smithi188 ceph-mon[36599]: scrub summary: active paths [/] 2024-06-10T03:00:45.704 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:00:45 smithi188 ceph-mon[36599]: scrub summary: idle+waiting paths [/] 2024-06-10T03:00:45.704 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:00:45 smithi188 ceph-mon[36599]: scrub summary: idle+waiting paths [/] 2024-06-10T03:00:45.704 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:00:45 smithi188 ceph-mon[36599]: scrub summary: active paths [/] 2024-06-10T03:00:45.705 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:00:45 smithi188 ceph-mon[36599]: scrub summary: idle+waiting paths [/] 2024-06-10T03:00:45.705 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:00:45 smithi188 ceph-mon[36599]: scrub summary: active paths [/] 2024-06-10T03:00:45.705 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:00:45 smithi188 ceph-mon[36599]: scrub summary: idle+waiting paths [/] 2024-06-10T03:00:45.705 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:00:45 smithi188 ceph-mon[36599]: scrub summary: active paths [/] 2024-06-10T03:00:45.705 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:00:45 smithi188 ceph-mon[36599]: scrub summary: idle+waiting paths [/] 2024-06-10T03:00:45.705 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:00:45 smithi188 ceph-mon[36599]: scrub summary: active paths [/] 2024-06-10T03:00:45.872 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-10T03:00:46.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:46.502+0000 7f4e95530700 1 -- 172.21.15.52:0/276245668 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4e90100a80 msgr2=0x7f4e90100f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:00:46.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:46.502+0000 7f4e95530700 1 --2- 172.21.15.52:0/276245668 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4e90100a80 0x7f4e90100f00 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f4e84009990 tx=0x7f4e84009ca0 comp rx=0 tx=0).stop 2024-06-10T03:00:46.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:46.502+0000 7f4e95530700 1 -- 172.21.15.52:0/276245668 shutdown_connections 2024-06-10T03:00:46.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:46.502+0000 7f4e95530700 1 --2- 172.21.15.52:0/276245668 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4e90101440 0x7f4e9010f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:46.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:46.502+0000 7f4e95530700 1 --2- 172.21.15.52:0/276245668 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4e90108510 0x7f4e901088f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:46.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:46.502+0000 7f4e95530700 1 --2- 172.21.15.52:0/276245668 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4e90100a80 0x7f4e90100f00 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:46.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:46.502+0000 7f4e95530700 1 -- 172.21.15.52:0/276245668 >> 172.21.15.52:0/276245668 conn(0x7f4e90077a00 msgr2=0x7f4e900fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:00:46.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:46.502+0000 7f4e95530700 1 -- 172.21.15.52:0/276245668 shutdown_connections 2024-06-10T03:00:46.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:46.502+0000 7f4e95530700 1 -- 172.21.15.52:0/276245668 wait complete. 2024-06-10T03:00:46.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:46.503+0000 7f4e95530700 1 Processor -- start 2024-06-10T03:00:46.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:46.503+0000 7f4e95530700 1 -- start start 2024-06-10T03:00:46.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:46.503+0000 7f4e95530700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4e90100a80 0x7f4e9019e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:46.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:46.503+0000 7f4e95530700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4e90101440 0x7f4e9019ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:46.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:46.503+0000 7f4e95530700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4e90108510 0x7f4e90198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:46.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:46.503+0000 7f4e95530700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f4e90112790 con 0x7f4e90101440 2024-06-10T03:00:46.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:46.503+0000 7f4e95530700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f4e90112610 con 0x7f4e90100a80 2024-06-10T03:00:46.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:46.503+0000 7f4e95530700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f4e90112910 con 0x7f4e90108510 2024-06-10T03:00:46.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:46.503+0000 7f4e8e7fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4e90101440 0x7f4e9019ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:46.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:46.504+0000 7f4e8e7fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4e90101440 0x7f4e9019ed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58772/0 (socket says 172.21.15.52:58772) 2024-06-10T03:00:46.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:46.504+0000 7f4e8e7fc700 1 -- 172.21.15.52:0/849646902 learned_addr learned my addr 172.21.15.52:0/849646902 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:00:46.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:46.504+0000 7f4e8f7fe700 1 --2- 172.21.15.52:0/849646902 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4e90108510 0x7f4e90198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:46.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:46.504+0000 7f4e8effd700 1 --2- 172.21.15.52:0/849646902 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4e90100a80 0x7f4e9019e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:46.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:46.504+0000 7f4e8e7fc700 1 -- 172.21.15.52:0/849646902 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4e90108510 msgr2=0x7f4e90198a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:00:46.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:46.504+0000 7f4e8e7fc700 1 --2- 172.21.15.52:0/849646902 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4e90108510 0x7f4e90198a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:46.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:46.504+0000 7f4e8e7fc700 1 -- 172.21.15.52:0/849646902 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4e90100a80 msgr2=0x7f4e9019e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:00:46.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:46.504+0000 7f4e8e7fc700 1 --2- 172.21.15.52:0/849646902 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4e90100a80 0x7f4e9019e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:46.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:46.504+0000 7f4e8e7fc700 1 -- 172.21.15.52:0/849646902 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e901991b0 con 0x7f4e90101440 2024-06-10T03:00:46.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:46.504+0000 7f4e8effd700 1 --2- 172.21.15.52:0/849646902 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4e90100a80 0x7f4e9019e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T03:00:46.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:46.504+0000 7f4e8e7fc700 1 --2- 172.21.15.52:0/849646902 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4e90101440 0x7f4e9019ed80 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f4e84000c00 tx=0x7f4e8400bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:00:46.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:46.504+0000 7f4e77fff700 1 -- 172.21.15.52:0/849646902 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e84027070 con 0x7f4e90101440 2024-06-10T03:00:46.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:46.504+0000 7f4e95530700 1 -- 172.21.15.52:0/849646902 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e90199440 con 0x7f4e90101440 2024-06-10T03:00:46.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:46.504+0000 7f4e77fff700 1 -- 172.21.15.52:0/849646902 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e84003d70 con 0x7f4e90101440 2024-06-10T03:00:46.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:46.504+0000 7f4e77fff700 1 -- 172.21.15.52:0/849646902 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e8400f460 con 0x7f4e90101440 2024-06-10T03:00:46.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:46.504+0000 7f4e95530700 1 -- 172.21.15.52:0/849646902 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e90101b90 con 0x7f4e90101440 2024-06-10T03:00:46.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:46.505+0000 7f4e95530700 1 -- 172.21.15.52:0/849646902 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e9010a9f0 con 0x7f4e90101440 2024-06-10T03:00:46.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:46.506+0000 7f4e77fff700 1 -- 172.21.15.52:0/849646902 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f4e8402a070 con 0x7f4e90101440 2024-06-10T03:00:46.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:46.507+0000 7f4e77fff700 1 --2- 172.21.15.52:0/849646902 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f4e780a2ca0 0x7f4e780a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:46.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:46.507+0000 7f4e8effd700 1 --2- 172.21.15.52:0/849646902 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f4e780a2ca0 0x7f4e780a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:46.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:46.507+0000 7f4e77fff700 1 -- 172.21.15.52:0/849646902 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f4e840d0050 con 0x7f4e90101440 2024-06-10T03:00:46.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:46.508+0000 7f4e8effd700 1 --2- 172.21.15.52:0/849646902 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f4e780a2ca0 0x7f4e780a5160 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f4e90107640 tx=0x7f4e80009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:00:46.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:46.512+0000 7f4e77fff700 1 -- 172.21.15.52:0/849646902 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f4e84013070 con 0x7f4e90101440 2024-06-10T03:00:46.564 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T03:00:46.638 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:00:46 smithi120 ceph-mon[36634]: scrub summary: idle+waiting paths [/] 2024-06-10T03:00:46.638 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:00:46 smithi120 ceph-mon[36634]: pgmap v631: 129 pgs: 129 active+clean; 571 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 865 KiB/s wr, 738 op/s 2024-06-10T03:00:46.687 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:00:46 smithi052 ceph-mon[32362]: scrub summary: idle+waiting paths [/] 2024-06-10T03:00:46.687 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:00:46 smithi052 ceph-mon[32362]: pgmap v631: 129 pgs: 129 active+clean; 571 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 865 KiB/s wr, 738 op/s 2024-06-10T03:00:46.703 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:00:46 smithi188 ceph-mon[36599]: scrub summary: idle+waiting paths [/] 2024-06-10T03:00:46.703 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:00:46 smithi188 ceph-mon[36599]: pgmap v631: 129 pgs: 129 active+clean; 571 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 865 KiB/s wr, 738 op/s 2024-06-10T03:00:46.771 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:46.771+0000 7f4e95530700 1 -- 172.21.15.52:0/849646902 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4e9010a3d0 con 0x7f4e90101440 2024-06-10T03:00:46.772 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:46.771+0000 7f4e77fff700 1 -- 172.21.15.52:0/849646902 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+11945 (secure 0 0 0) 0x7f4e8409cc90 con 0x7f4e90101440 2024-06-10T03:00:46.772 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-10T03:00:46.773 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":36,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/889673247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":889673247},{"type":"v1","addr":"172.21.15.52:6835","nonce":889673247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/3615663511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":3615663511},{"type":"v1","addr":"172.21.15.52:6837","nonce":3615663511}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":36,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T02:50:03.889766+0000","modified":"2024-06-10T02:59:27.720611+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24484,"mds_1":24469,"mds_2":24463,"mds_3":24449,"mds_4":24428},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14571":{"gid":14571,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6839/3059599042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":3059599042},{"type":"v1","addr":"172.21.15.52:6839","nonce":3059599042}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14583":{"gid":14583,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6841/3039275521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":3039275521},{"type":"v1","addr":"172.21.15.52:6841","nonce":3039275521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24383":{"gid":24383,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6833/2609002896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":2609002896},{"type":"v1","addr":"172.21.15.120:6833","nonce":2609002896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6835/1996370288","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":1996370288},{"type":"v1","addr":"172.21.15.120:6835","nonce":1996370288}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6833/529564979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":529564979},{"type":"v1","addr":"172.21.15.188:6833","nonce":529564979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":25,"addr":"172.21.15.120:6837/4132440952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":4132440952},{"type":"v1","addr":"172.21.15.120:6837","nonce":4132440952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"b","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.120:6839/1672436487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1672436487},{"type":"v1","addr":"172.21.15.120:6839","nonce":1672436487}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":2,"incarnation":22,"state":"up:active","state_seq":26,"addr":"172.21.15.188:6835/3904544259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":3904544259},{"type":"v1","addr":"172.21.15.188:6835","nonce":3904544259}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.188:6837/819317726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":819317726},{"type":"v1","addr":"172.21.15.188:6837","nonce":819317726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.188:6839/310444836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":310444836},{"type":"v1","addr":"172.21.15.188:6839","nonce":310444836}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24484,"qdb_cluster":[24469,24484,24463,24428,24449]},"id":1}]} 2024-06-10T03:00:46.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:46.774+0000 7f4e95530700 1 -- 172.21.15.52:0/849646902 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f4e780a2ca0 msgr2=0x7f4e780a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:00:46.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:46.774+0000 7f4e95530700 1 --2- 172.21.15.52:0/849646902 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f4e780a2ca0 0x7f4e780a5160 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f4e90107640 tx=0x7f4e80009260 comp rx=0 tx=0).stop 2024-06-10T03:00:46.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:46.774+0000 7f4e95530700 1 -- 172.21.15.52:0/849646902 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4e90101440 msgr2=0x7f4e9019ed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:00:46.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:46.774+0000 7f4e95530700 1 --2- 172.21.15.52:0/849646902 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4e90101440 0x7f4e9019ed80 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f4e84000c00 tx=0x7f4e8400bab0 comp rx=0 tx=0).stop 2024-06-10T03:00:46.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:46.774+0000 7f4e95530700 1 -- 172.21.15.52:0/849646902 shutdown_connections 2024-06-10T03:00:46.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:46.774+0000 7f4e95530700 1 --2- 172.21.15.52:0/849646902 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4e90108510 0x7f4e90198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:46.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:46.774+0000 7f4e95530700 1 --2- 172.21.15.52:0/849646902 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4e90100a80 0x7f4e9019e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:46.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:46.774+0000 7f4e95530700 1 --2- 172.21.15.52:0/849646902 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f4e780a2ca0 0x7f4e780a5160 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:46.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:46.774+0000 7f4e95530700 1 --2- 172.21.15.52:0/849646902 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4e90101440 0x7f4e9019ed80 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:46.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:46.774+0000 7f4e95530700 1 -- 172.21.15.52:0/849646902 >> 172.21.15.52:0/849646902 conn(0x7f4e90077a00 msgr2=0x7f4e900ff0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:00:46.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:46.774+0000 7f4e95530700 1 -- 172.21.15.52:0/849646902 shutdown_connections 2024-06-10T03:00:46.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:46.774+0000 7f4e95530700 1 -- 172.21.15.52:0/849646902 wait complete. 2024-06-10T03:00:46.776 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 36 2024-06-10T03:00:47.250 INFO:tasks.workunit.client.0.smithi052.stdout: 98 60013 0 43228 0 31706 0 2024-06-10T03:00:47.566 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:00:47 smithi052 ceph-mon[32362]: scrub summary: idle+waiting paths [/] 2024-06-10T03:00:47.566 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:00:47 smithi052 ceph-mon[32362]: from='client.? 172.21.15.52:0/849646902' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T03:00:47.567 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24484 quiesce db --set-id c8954a8e --release --await' 2024-06-10T03:00:47.567 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24484 quiesce db --set-id c8954a8e --release --await 2024-06-10T03:00:47.638 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:00:47 smithi120 ceph-mon[36634]: scrub summary: idle+waiting paths [/] 2024-06-10T03:00:47.638 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:00:47 smithi120 ceph-mon[36634]: from='client.? 172.21.15.52:0/849646902' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T03:00:47.703 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:00:47 smithi188 ceph-mon[36599]: scrub summary: idle+waiting paths [/] 2024-06-10T03:00:47.703 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:00:47 smithi188 ceph-mon[36599]: from='client.? 172.21.15.52:0/849646902' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T03:00:48.638 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:00:48 smithi120 ceph-mon[36634]: pgmap v632: 129 pgs: 129 active+clean; 572 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 515 KiB/s wr, 938 op/s 2024-06-10T03:00:48.703 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:00:48 smithi188 ceph-mon[36599]: pgmap v632: 129 pgs: 129 active+clean; 572 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 515 KiB/s wr, 938 op/s 2024-06-10T03:00:48.706 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:00:48 smithi052 ceph-mon[32362]: pgmap v632: 129 pgs: 129 active+clean; 572 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 515 KiB/s wr, 938 op/s 2024-06-10T03:00:49.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.251+0000 7f7533951700 1 -- 172.21.15.52:0/1582333389 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f752c101680 msgr2=0x7f752c101a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:00:49.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.251+0000 7f7533951700 1 --2- 172.21.15.52:0/1582333389 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f752c101680 0x7f752c101a60 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f75280099c0 tx=0x7f7528009cd0 comp rx=0 tx=0).stop 2024-06-10T03:00:49.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.251+0000 7f7533951700 1 -- 172.21.15.52:0/1582333389 shutdown_connections 2024-06-10T03:00:49.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.251+0000 7f7533951700 1 --2- 172.21.15.52:0/1582333389 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f752c1066d0 0x7f752c10e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:49.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.251+0000 7f7533951700 1 --2- 172.21.15.52:0/1582333389 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f752c101680 0x7f752c101a60 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:49.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.251+0000 7f7533951700 1 --2- 172.21.15.52:0/1582333389 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f752c102030 0x7f752c105ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:49.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.251+0000 7f7533951700 1 -- 172.21.15.52:0/1582333389 >> 172.21.15.52:0/1582333389 conn(0x7f752c0773e0 msgr2=0x7f752c0777f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:00:49.253 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.251+0000 7f7533951700 1 -- 172.21.15.52:0/1582333389 shutdown_connections 2024-06-10T03:00:49.253 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.251+0000 7f7533951700 1 -- 172.21.15.52:0/1582333389 wait complete. 2024-06-10T03:00:49.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.252+0000 7f7533951700 1 Processor -- start 2024-06-10T03:00:49.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.252+0000 7f7533951700 1 -- start start 2024-06-10T03:00:49.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.252+0000 7f7533951700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f752c101680 0x7f752c199b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:49.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.252+0000 7f7533951700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f752c102030 0x7f752c19a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:49.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.252+0000 7f7533951700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f752c1066d0 0x7f752c1a4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:49.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.252+0000 7f7533951700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f752c111730 con 0x7f752c1066d0 2024-06-10T03:00:49.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.252+0000 7f7533951700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f752c1115b0 con 0x7f752c102030 2024-06-10T03:00:49.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.252+0000 7f7533951700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f752c1118b0 con 0x7f752c101680 2024-06-10T03:00:49.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.252+0000 7f7531eee700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f752c1066d0 0x7f752c1a4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:49.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.252+0000 7f7531eee700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f752c1066d0 0x7f752c1a4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58788/0 (socket says 172.21.15.52:58788) 2024-06-10T03:00:49.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.252+0000 7f7531eee700 1 -- 172.21.15.52:0/3659512289 learned_addr learned my addr 172.21.15.52:0/3659512289 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:00:49.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.252+0000 7f75316ed700 1 --2- 172.21.15.52:0/3659512289 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f752c101680 0x7f752c199b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:49.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.253+0000 7f7530eec700 1 --2- 172.21.15.52:0/3659512289 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f752c102030 0x7f752c19a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:49.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.253+0000 7f7531eee700 1 -- 172.21.15.52:0/3659512289 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f752c101680 msgr2=0x7f752c199b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:00:49.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.253+0000 7f7531eee700 1 --2- 172.21.15.52:0/3659512289 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f752c101680 0x7f752c199b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:49.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.253+0000 7f7531eee700 1 -- 172.21.15.52:0/3659512289 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f752c102030 msgr2=0x7f752c19a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:00:49.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.253+0000 7f7531eee700 1 --2- 172.21.15.52:0/3659512289 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f752c102030 0x7f752c19a0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:49.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.253+0000 7f7531eee700 1 -- 172.21.15.52:0/3659512289 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f752c1a5130 con 0x7f752c1066d0 2024-06-10T03:00:49.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.253+0000 7f75316ed700 1 --2- 172.21.15.52:0/3659512289 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f752c101680 0x7f752c199b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T03:00:49.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.253+0000 7f7530eec700 1 --2- 172.21.15.52:0/3659512289 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f752c102030 0x7f752c19a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T03:00:49.257 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.253+0000 7f7531eee700 1 --2- 172.21.15.52:0/3659512289 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f752c1066d0 0x7f752c1a4bf0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f7524007ee0 tx=0x7f752400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:00:49.257 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.253+0000 7f75227fc700 1 -- 172.21.15.52:0/3659512289 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f752400dbd0 con 0x7f752c1066d0 2024-06-10T03:00:49.257 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.253+0000 7f7533951700 1 -- 172.21.15.52:0/3659512289 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f752c1a5360 con 0x7f752c1066d0 2024-06-10T03:00:49.257 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.253+0000 7f75227fc700 1 -- 172.21.15.52:0/3659512289 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f752400dd70 con 0x7f752c1066d0 2024-06-10T03:00:49.257 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.253+0000 7f75227fc700 1 -- 172.21.15.52:0/3659512289 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75240133f0 con 0x7f752c1066d0 2024-06-10T03:00:49.257 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.253+0000 7f7533951700 1 -- 172.21.15.52:0/3659512289 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f752c1a5870 con 0x7f752c1066d0 2024-06-10T03:00:49.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.255+0000 7f75227fc700 1 -- 172.21.15.52:0/3659512289 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f752401a020 con 0x7f752c1066d0 2024-06-10T03:00:49.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.256+0000 7f75227fc700 1 --2- 172.21.15.52:0/3659512289 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f75180a2c20 0x7f75180a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:49.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.257+0000 7f75316ed700 1 --2- 172.21.15.52:0/3659512289 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f75180a2c20 0x7f75180a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:49.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.257+0000 7f75227fc700 1 -- 172.21.15.52:0/3659512289 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f75240cec00 con 0x7f752c1066d0 2024-06-10T03:00:49.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.257+0000 7f75316ed700 1 --2- 172.21.15.52:0/3659512289 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f75180a2c20 0x7f75180a50e0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f7528000c00 tx=0x7f7528005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:00:49.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.257+0000 7f753e1d4b80 1 Processor -- start 2024-06-10T03:00:49.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.257+0000 7f753e1d4b80 1 -- start start 2024-06-10T03:00:49.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.258+0000 7f753e1d4b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5569c7f61640 0x5569c7f61a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:49.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.258+0000 7f753e1d4b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5569c7f61f60 0x5569c7f5b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:49.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.258+0000 7f753e1d4b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5569c7f5bf20 0x5569c7f5c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:49.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.258+0000 7f753e1d4b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5569c7ead810 con 0x5569c7f61f60 2024-06-10T03:00:49.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.258+0000 7f753e1d4b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5569c7e55720 con 0x5569c7f5bf20 2024-06-10T03:00:49.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.258+0000 7f753e1d4b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5569c7ea97d0 con 0x5569c7f61640 2024-06-10T03:00:49.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.258+0000 7f7530eec700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5569c7f61f60 0x5569c7f5b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:49.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.258+0000 7f7530eec700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5569c7f61f60 0x5569c7f5b9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58798/0 (socket says 172.21.15.52:58798) 2024-06-10T03:00:49.267 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.258+0000 7f7530eec700 1 -- 172.21.15.52:0/1878664534 learned_addr learned my addr 172.21.15.52:0/1878664534 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:00:49.267 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.258+0000 7f75316ed700 1 --2- 172.21.15.52:0/1878664534 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5569c7f61640 0x5569c7f61a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:49.267 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.258+0000 7f7531eee700 1 --2- 172.21.15.52:0/1878664534 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5569c7f5bf20 0x5569c7f5c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:49.267 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.258+0000 7f7530eec700 1 -- 172.21.15.52:0/1878664534 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5569c7f61640 msgr2=0x5569c7f61a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:00:49.267 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.258+0000 7f7530eec700 1 --2- 172.21.15.52:0/1878664534 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5569c7f61640 0x5569c7f61a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:49.267 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.258+0000 7f7530eec700 1 -- 172.21.15.52:0/1878664534 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5569c7f5bf20 msgr2=0x5569c7f5c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:00:49.267 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.258+0000 7f7530eec700 1 --2- 172.21.15.52:0/1878664534 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5569c7f5bf20 0x5569c7f5c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:49.267 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.258+0000 7f7530eec700 1 -- 172.21.15.52:0/1878664534 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5569c7e4bf50 con 0x5569c7f61f60 2024-06-10T03:00:49.267 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.258+0000 7f7531eee700 1 --2- 172.21.15.52:0/1878664534 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5569c7f5bf20 0x5569c7f5c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T03:00:49.267 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.258+0000 7f7530eec700 1 --2- 172.21.15.52:0/1878664534 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5569c7f61f60 0x5569c7f5b9e0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f751c00aa10 tx=0x7f751c00ad20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b80fc9df4904ab9 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:00:49.267 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.258+0000 7f75177fe700 1 -- 172.21.15.52:0/1878664534 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f751c015070 con 0x5569c7f61f60 2024-06-10T03:00:49.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.258+0000 7f75177fe700 1 -- 172.21.15.52:0/1878664534 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f751c0113f0 con 0x5569c7f61f60 2024-06-10T03:00:49.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.258+0000 7f75177fe700 1 -- 172.21.15.52:0/1878664534 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f751c0185c0 con 0x5569c7f61f60 2024-06-10T03:00:49.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.259+0000 7f753e1d4b80 1 -- 172.21.15.52:0/1878664534 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5569c7f61f60 msgr2=0x5569c7f5b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:00:49.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.259+0000 7f753e1d4b80 1 --2- 172.21.15.52:0/1878664534 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5569c7f61f60 0x5569c7f5b9e0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f751c00aa10 tx=0x7f751c00ad20 comp rx=0 tx=0).stop 2024-06-10T03:00:49.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.259+0000 7f753e1d4b80 1 -- 172.21.15.52:0/1878664534 shutdown_connections 2024-06-10T03:00:49.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.259+0000 7f753e1d4b80 1 --2- 172.21.15.52:0/1878664534 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5569c7f5bf20 0x5569c7f5c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:49.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.259+0000 7f753e1d4b80 1 --2- 172.21.15.52:0/1878664534 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5569c7f61640 0x5569c7f61a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:49.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.259+0000 7f753e1d4b80 1 --2- 172.21.15.52:0/1878664534 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5569c7f61f60 0x5569c7f5b9e0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:49.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.259+0000 7f753e1d4b80 1 -- 172.21.15.52:0/1878664534 >> 172.21.15.52:0/1878664534 conn(0x5569c7e86cc0 msgr2=0x5569c7f56850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:00:49.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.259+0000 7f753e1d4b80 1 -- 172.21.15.52:0/1878664534 shutdown_connections 2024-06-10T03:00:49.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.259+0000 7f753e1d4b80 1 -- 172.21.15.52:0/1878664534 wait complete. 2024-06-10T03:00:49.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.260+0000 7f753e1d4b80 1 -- start start 2024-06-10T03:00:49.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.260+0000 7f753e1d4b80 1 Processor -- start 2024-06-10T03:00:49.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.260+0000 7f753e1d4b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5569c7f5bf20 0x5569c7f67cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:49.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.260+0000 7f753e1d4b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5569c7f61640 0x5569c7f64fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:49.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.260+0000 7f753e1d4b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5569c7f61f60 0x5569c7f654f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:49.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.260+0000 7f753e1d4b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5569c7f689d0 con 0x5569c7f61f60 2024-06-10T03:00:49.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.260+0000 7f753e1d4b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5569c7e4dbf0 con 0x5569c7f5bf20 2024-06-10T03:00:49.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.260+0000 7f753e1d4b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5569c7f68b50 con 0x5569c7f61640 2024-06-10T03:00:49.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.260+0000 7f753e1d4b80 10 client.0 _ll_register_callbacks cb 0x5569c7e47920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T03:00:49.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.260+0000 7f7531eee700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5569c7f61f60 0x5569c7f654f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:49.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.260+0000 7f7531eee700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5569c7f61f60 0x5569c7f654f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58806/0 (socket says 172.21.15.52:58806) 2024-06-10T03:00:49.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.260+0000 7f7531eee700 1 -- 172.21.15.52:0/2587929423 learned_addr learned my addr 172.21.15.52:0/2587929423 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:00:49.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.261+0000 7f7530eec700 1 --2- 172.21.15.52:0/2587929423 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5569c7f61640 0x5569c7f64fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:49.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.261+0000 7f75316ed700 1 --2- 172.21.15.52:0/2587929423 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5569c7f5bf20 0x5569c7f67cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:49.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.261+0000 7f7531eee700 1 -- 172.21.15.52:0/2587929423 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5569c7f61640 msgr2=0x5569c7f64fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:00:49.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.261+0000 7f7531eee700 1 --2- 172.21.15.52:0/2587929423 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5569c7f61640 0x5569c7f64fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:49.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.261+0000 7f7531eee700 1 -- 172.21.15.52:0/2587929423 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5569c7f5bf20 msgr2=0x5569c7f67cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:00:49.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.261+0000 7f7531eee700 1 --2- 172.21.15.52:0/2587929423 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5569c7f5bf20 0x5569c7f67cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:49.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.261+0000 7f7531eee700 1 -- 172.21.15.52:0/2587929423 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5569c7dfc8d0 con 0x5569c7f61f60 2024-06-10T03:00:49.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.261+0000 7f7530eec700 1 --2- 172.21.15.52:0/2587929423 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5569c7f61640 0x5569c7f64fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T03:00:49.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.261+0000 7f7531eee700 1 --2- 172.21.15.52:0/2587929423 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5569c7f61f60 0x5569c7f654f0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f75240974b0 tx=0x7f752409b800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef50b152781440f6 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:00:49.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.261+0000 7f7514ff9700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-10T03:00:49.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.261+0000 7f7514ff9700 1 -- 172.21.15.52:0/2587929423 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75240a3cc0 con 0x5569c7f61f60 2024-06-10T03:00:49.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.261+0000 7f7514ff9700 1 -- 172.21.15.52:0/2587929423 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f7524004c40 con 0x5569c7f61f60 2024-06-10T03:00:49.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.261+0000 7f753e1d4b80 1 -- 172.21.15.52:0/2587929423 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f751c0113f0 con 0x5569c7f61f60 2024-06-10T03:00:49.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.261+0000 7f7514ff9700 1 -- 172.21.15.52:0/2587929423 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f75240b4020 con 0x5569c7f61f60 2024-06-10T03:00:49.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.261+0000 7f753e1d4b80 10 client.16494 fetch_fsmap learned FSMap version 36 2024-06-10T03:00:49.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.261+0000 7f753e1d4b80 1 -- 172.21.15.52:0/2587929423 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5569c7f73310 con 0x5569c7f61f60 2024-06-10T03:00:49.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.261+0000 7f7514ff9700 1 -- 172.21.15.52:0/2587929423 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f751c0113f0 con 0x5569c7f61f60 2024-06-10T03:00:49.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.262+0000 7f7514ff9700 1 -- 172.21.15.52:0/2587929423 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 36) ==== 8378+0+0 (secure 0 0 0) 0x7f75240a3830 con 0x5569c7f61f60 2024-06-10T03:00:49.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.262+0000 7f753e1d4b80 10 client.16494 fetch_fsmap finished waiting for FSMap version 36 2024-06-10T03:00:49.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.262+0000 7f753e1d4b80 10 client.16494 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T03:00:49.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.262+0000 7f753e1d4b80 20 client.16494 populate_metadata read hostname 'smithi052' 2024-06-10T03:00:49.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.262+0000 7f753e1d4b80 1 --2- 172.21.15.52:0/2587929423 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x5569c7f75950 0x5569c7f77d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:49.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.262+0000 7f753e1d4b80 4 client.16494 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T03:00:49.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.262+0000 7f753e1d4b80 1 -- 172.21.15.52:0/2587929423 --> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5569c7ebad10 con 0x5569c7f75950 2024-06-10T03:00:49.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.262+0000 7f75316ed700 1 --2- 172.21.15.52:0/2587929423 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x5569c7f75950 0x5569c7f77d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:49.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.263+0000 7f75316ed700 1 --2- 172.21.15.52:0/2587929423 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x5569c7f75950 0x5569c7f77d40 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4523c397c96e5dcf server_cookie=408d8603947dd053 in_seq=0 out_seq=0 2024-06-10T03:00:49.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.263+0000 7f7514ff9700 10 client.16494 ms_handle_connect on v2:172.21.15.188:6838/310444836 2024-06-10T03:00:49.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.263+0000 7f7514ff9700 1 -- 172.21.15.52:0/2587929423 <== mds.0 v2:172.21.15.188:6838/310444836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5569c7ebad10 con 0x5569c7f75950 2024-06-10T03:00:49.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.263+0000 7f7514ff9700 10 client.16494 handle_command_reply: tid=0 2024-06-10T03:00:49.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.263+0000 7f753e1d4b80 1 client.16494 shutdown 2024-06-10T03:00:49.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.263+0000 7f753e1d4b80 1 -- 172.21.15.52:0/2587929423 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5569c7f61f60 msgr2=0x5569c7f654f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:00:49.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.263+0000 7f753e1d4b80 1 --2- 172.21.15.52:0/2587929423 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5569c7f61f60 0x5569c7f654f0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f75240974b0 tx=0x7f752409b800 comp rx=0 tx=0).stop 2024-06-10T03:00:49.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.264+0000 7f753e1d4b80 1 -- 172.21.15.52:0/2587929423 shutdown_connections 2024-06-10T03:00:49.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.264+0000 7f753e1d4b80 1 --2- 172.21.15.52:0/2587929423 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x5569c7f75950 0x5569c7f77d40 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:49.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.264+0000 7f753e1d4b80 1 --2- 172.21.15.52:0/2587929423 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5569c7f61f60 0x5569c7f654f0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:49.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.264+0000 7f753e1d4b80 1 --2- 172.21.15.52:0/2587929423 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5569c7f5bf20 0x5569c7f67cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:49.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.264+0000 7f753e1d4b80 1 --2- 172.21.15.52:0/2587929423 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5569c7f61640 0x5569c7f64fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:49.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.264+0000 7f753e1d4b80 1 -- 172.21.15.52:0/2587929423 >> 172.21.15.52:0/2587929423 conn(0x5569c7e86cc0 msgr2=0x5569c7f5a9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:00:49.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.264+0000 7f7514ff9700 0 client.16494 ms_handle_reset on v2:172.21.15.188:6838/310444836 2024-06-10T03:00:49.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.264+0000 7f753e1d4b80 1 -- 172.21.15.52:0/2587929423 shutdown_connections 2024-06-10T03:00:49.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.264+0000 7f753e1d4b80 1 -- 172.21.15.52:0/2587929423 wait complete. 2024-06-10T03:00:49.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.264+0000 7f753e1d4b80 20 client.16494 trim_cache size 0 max 16384 2024-06-10T03:00:49.287 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.286+0000 7f753e1d4b80 1 Processor -- start 2024-06-10T03:00:49.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.286+0000 7f753e1d4b80 1 -- start start 2024-06-10T03:00:49.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.286+0000 7f753e1d4b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5569c7f5bf20 0x5569c7f5b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:49.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.286+0000 7f753e1d4b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5569c7f61640 0x5569c7f62400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:49.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.286+0000 7f753e1d4b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5569c7f61f60 0x5569c7f62940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:49.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.286+0000 7f753e1d4b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f752c1ae4b0 con 0x5569c7f61640 2024-06-10T03:00:49.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.286+0000 7f753e1d4b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f752c1b00b0 con 0x5569c7f61f60 2024-06-10T03:00:49.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.286+0000 7f753e1d4b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f752c1aeb80 con 0x5569c7f5bf20 2024-06-10T03:00:49.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.286+0000 7f7530eec700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5569c7f61640 0x5569c7f62400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:49.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.286+0000 7f7530eec700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5569c7f61640 0x5569c7f62400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58818/0 (socket says 172.21.15.52:58818) 2024-06-10T03:00:49.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.286+0000 7f7530eec700 1 -- 172.21.15.52:0/1992581812 learned_addr learned my addr 172.21.15.52:0/1992581812 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:00:49.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.286+0000 7f7531eee700 1 --2- 172.21.15.52:0/1992581812 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5569c7f61f60 0x5569c7f62940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:49.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.286+0000 7f75316ed700 1 --2- 172.21.15.52:0/1992581812 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5569c7f5bf20 0x5569c7f5b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:49.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.286+0000 7f7530eec700 1 -- 172.21.15.52:0/1992581812 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5569c7f5bf20 msgr2=0x5569c7f5b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:00:49.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.286+0000 7f7530eec700 1 --2- 172.21.15.52:0/1992581812 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5569c7f5bf20 0x5569c7f5b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:49.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.286+0000 7f7530eec700 1 -- 172.21.15.52:0/1992581812 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5569c7f61f60 msgr2=0x5569c7f62940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:00:49.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.286+0000 7f7530eec700 1 --2- 172.21.15.52:0/1992581812 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5569c7f61f60 0x5569c7f62940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:49.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.286+0000 7f7530eec700 1 -- 172.21.15.52:0/1992581812 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5569c7f64d80 con 0x5569c7f61640 2024-06-10T03:00:49.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.286+0000 7f7531eee700 1 --2- 172.21.15.52:0/1992581812 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5569c7f61f60 0x5569c7f62940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T03:00:49.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.286+0000 7f7530eec700 1 --2- 172.21.15.52:0/1992581812 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5569c7f61640 0x5569c7f62400 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f751c00a9e0 tx=0x7f751c00caa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39e4c0604a91bc14 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:00:49.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.287+0000 7f7516ffd700 1 -- 172.21.15.52:0/1992581812 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f751c028070 con 0x5569c7f61640 2024-06-10T03:00:49.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.287+0000 7f7516ffd700 1 -- 172.21.15.52:0/1992581812 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f751c00cc90 con 0x5569c7f61640 2024-06-10T03:00:49.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.287+0000 7f7516ffd700 1 -- 172.21.15.52:0/1992581812 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f751c015040 con 0x5569c7f61640 2024-06-10T03:00:49.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.288+0000 7f753e1d4b80 1 -- 172.21.15.52:0/1992581812 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5569c7f61640 msgr2=0x5569c7f62400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:00:49.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.288+0000 7f753e1d4b80 1 --2- 172.21.15.52:0/1992581812 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5569c7f61640 0x5569c7f62400 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f751c00a9e0 tx=0x7f751c00caa0 comp rx=0 tx=0).stop 2024-06-10T03:00:49.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.288+0000 7f753e1d4b80 1 -- 172.21.15.52:0/1992581812 shutdown_connections 2024-06-10T03:00:49.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.288+0000 7f753e1d4b80 1 --2- 172.21.15.52:0/1992581812 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5569c7f61f60 0x5569c7f62940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:49.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.288+0000 7f753e1d4b80 1 --2- 172.21.15.52:0/1992581812 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5569c7f5bf20 0x5569c7f5b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:49.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.288+0000 7f753e1d4b80 1 --2- 172.21.15.52:0/1992581812 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5569c7f61640 0x5569c7f62400 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:49.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.288+0000 7f753e1d4b80 1 -- 172.21.15.52:0/1992581812 >> 172.21.15.52:0/1992581812 conn(0x5569c7e86cc0 msgr2=0x5569c7f56380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:00:49.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.288+0000 7f753e1d4b80 1 -- 172.21.15.52:0/1992581812 shutdown_connections 2024-06-10T03:00:49.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.288+0000 7f753e1d4b80 1 -- 172.21.15.52:0/1992581812 wait complete. 2024-06-10T03:00:49.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.288+0000 7f753e1d4b80 1 -- start start 2024-06-10T03:00:49.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.288+0000 7f753e1d4b80 1 Processor -- start 2024-06-10T03:00:49.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.289+0000 7f753e1d4b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5569c7f5bf20 0x5569c7f6dff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:49.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.289+0000 7f753e1d4b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5569c7f61640 0x5569c7f6e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:49.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.289+0000 7f753e1d4b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5569c7f61f60 0x5569c7f6ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:49.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.289+0000 7f753e1d4b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f752c1af9e0 con 0x5569c7f61640 2024-06-10T03:00:49.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.289+0000 7f753e1d4b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f752c1af310 con 0x5569c7f61f60 2024-06-10T03:00:49.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.289+0000 7f753e1d4b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f752c1ac450 con 0x5569c7f5bf20 2024-06-10T03:00:49.295 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.289+0000 7f753e1d4b80 10 client.0 _ll_register_callbacks cb 0x5569c7e47920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T03:00:49.295 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.289+0000 7f7530eec700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5569c7f61640 0x5569c7f6e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:49.295 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.289+0000 7f7530eec700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5569c7f61640 0x5569c7f6e530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58830/0 (socket says 172.21.15.52:58830) 2024-06-10T03:00:49.295 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.289+0000 7f7530eec700 1 -- 172.21.15.52:0/2644496967 learned_addr learned my addr 172.21.15.52:0/2644496967 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:00:49.295 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.289+0000 7f75316ed700 1 --2- 172.21.15.52:0/2644496967 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5569c7f5bf20 0x5569c7f6dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:49.295 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.289+0000 7f7531eee700 1 --2- 172.21.15.52:0/2644496967 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5569c7f61f60 0x5569c7f6ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:49.295 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.289+0000 7f7530eec700 1 -- 172.21.15.52:0/2644496967 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5569c7f5bf20 msgr2=0x5569c7f6dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:00:49.295 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.289+0000 7f7530eec700 1 --2- 172.21.15.52:0/2644496967 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5569c7f5bf20 0x5569c7f6dff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:49.295 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.289+0000 7f7530eec700 1 -- 172.21.15.52:0/2644496967 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5569c7f61f60 msgr2=0x5569c7f6ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:00:49.295 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.289+0000 7f7530eec700 1 --2- 172.21.15.52:0/2644496967 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5569c7f61f60 0x5569c7f6ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:49.295 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.289+0000 7f7530eec700 1 -- 172.21.15.52:0/2644496967 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5569c7f6b420 con 0x5569c7f61640 2024-06-10T03:00:49.295 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.290+0000 7f7530eec700 1 --2- 172.21.15.52:0/2644496967 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5569c7f61640 0x5569c7f6e530 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f751c0048a0 tx=0x7f751c011620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f7f76ec94b46bf1 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:00:49.295 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.290+0000 7f7514ff9700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-10T03:00:49.296 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.290+0000 7f7514ff9700 1 -- 172.21.15.52:0/2644496967 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f751c031060 con 0x5569c7f61640 2024-06-10T03:00:49.296 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.290+0000 7f7514ff9700 1 -- 172.21.15.52:0/2644496967 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f751c00ce40 con 0x5569c7f61640 2024-06-10T03:00:49.296 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.290+0000 7f753e1d4b80 1 -- 172.21.15.52:0/2644496967 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f751c00cc90 con 0x5569c7f61640 2024-06-10T03:00:49.296 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.290+0000 7f7514ff9700 1 -- 172.21.15.52:0/2644496967 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f751c038050 con 0x5569c7f61640 2024-06-10T03:00:49.296 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.290+0000 7f753e1d4b80 10 client.16500 fetch_fsmap learned FSMap version 36 2024-06-10T03:00:49.296 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.290+0000 7f753e1d4b80 1 -- 172.21.15.52:0/2644496967 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5569c7fe0ef0 con 0x5569c7f61640 2024-06-10T03:00:49.296 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.290+0000 7f7514ff9700 1 -- 172.21.15.52:0/2644496967 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f751c00cc90 con 0x5569c7f61640 2024-06-10T03:00:49.296 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.291+0000 7f7514ff9700 1 -- 172.21.15.52:0/2644496967 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 36) ==== 8378+0+0 (secure 0 0 0) 0x7f751c023410 con 0x5569c7f61640 2024-06-10T03:00:49.296 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.291+0000 7f753e1d4b80 10 client.16500 fetch_fsmap finished waiting for FSMap version 36 2024-06-10T03:00:49.296 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.291+0000 7f753e1d4b80 10 client.16500 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T03:00:49.296 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.291+0000 7f753e1d4b80 20 client.16500 populate_metadata read hostname 'smithi052' 2024-06-10T03:00:49.296 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.291+0000 7f753e1d4b80 1 --2- 172.21.15.52:0/2644496967 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x5569c7fe1160 0x5569c7fe3580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:49.296 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.291+0000 7f753e1d4b80 4 client.16500 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-10T03:00:49.297 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.291+0000 7f753e1d4b80 1 -- 172.21.15.52:0/2644496967 --> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5569c7eb5290 con 0x5569c7fe1160 2024-06-10T03:00:49.297 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.291+0000 7f75316ed700 1 --2- 172.21.15.52:0/2644496967 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x5569c7fe1160 0x5569c7fe3580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:49.297 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.291+0000 7f7514ff9700 10 client.16500 ms_handle_connect on v2:172.21.15.188:6838/310444836 2024-06-10T03:00:49.297 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.291+0000 7f75316ed700 1 --2- 172.21.15.52:0/2644496967 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x5569c7fe1160 0x5569c7fe3580 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=53f14281b8c71c48 server_cookie=721dbf3435729e54 in_seq=0 out_seq=0 2024-06-10T03:00:49.297 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.292+0000 7f7514ff9700 1 -- 172.21.15.52:0/2644496967 <== mds.0 v2:172.21.15.188:6838/310444836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5569c7eb5290 con 0x5569c7fe1160 2024-06-10T03:00:49.297 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.292+0000 7f7514ff9700 10 client.16500 handle_command_reply: tid=0 2024-06-10T03:00:49.297 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.292+0000 7f753e1d4b80 1 client.16500 shutdown 2024-06-10T03:00:49.297 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.292+0000 7f753e1d4b80 1 -- 172.21.15.52:0/2644496967 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5569c7f61640 msgr2=0x5569c7f6e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:00:49.297 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.292+0000 7f753e1d4b80 1 --2- 172.21.15.52:0/2644496967 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5569c7f61640 0x5569c7f6e530 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f751c0048a0 tx=0x7f751c011620 comp rx=0 tx=0).stop 2024-06-10T03:00:49.297 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.292+0000 7f753e1d4b80 1 -- 172.21.15.52:0/2644496967 shutdown_connections 2024-06-10T03:00:49.297 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.292+0000 7f753e1d4b80 1 --2- 172.21.15.52:0/2644496967 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x5569c7fe1160 0x5569c7fe3580 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:49.297 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.292+0000 7f753e1d4b80 1 --2- 172.21.15.52:0/2644496967 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5569c7f61f60 0x5569c7f6ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:49.297 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.292+0000 7f7514ff9700 0 client.16500 ms_handle_reset on v2:172.21.15.188:6838/310444836 2024-06-10T03:00:49.297 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.292+0000 7f753e1d4b80 1 --2- 172.21.15.52:0/2644496967 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5569c7f5bf20 0x5569c7f6dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:49.297 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.292+0000 7f753e1d4b80 1 --2- 172.21.15.52:0/2644496967 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5569c7f61640 0x5569c7f6e530 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:49.298 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.292+0000 7f753e1d4b80 1 -- 172.21.15.52:0/2644496967 >> 172.21.15.52:0/2644496967 conn(0x5569c7e86cc0 msgr2=0x5569c7f62400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:00:49.298 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.292+0000 7f753e1d4b80 1 -- 172.21.15.52:0/2644496967 shutdown_connections 2024-06-10T03:00:49.298 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.292+0000 7f753e1d4b80 1 -- 172.21.15.52:0/2644496967 wait complete. 2024-06-10T03:00:49.298 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.292+0000 7f753e1d4b80 20 client.16500 trim_cache size 0 max 16384 2024-06-10T03:00:49.298 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-10T03:00:49.298 INFO:teuthology.orchestra.run.smithi052.stdout: "status": "IDLE (waiting for more scrubs)", 2024-06-10T03:00:49.298 INFO:teuthology.orchestra.run.smithi052.stdout: "scrubs": { 2024-06-10T03:00:49.298 INFO:teuthology.orchestra.run.smithi052.stdout: "83aa1863-a6f4-45c9-a634-54ed2a6509bb": { 2024-06-10T03:00:49.298 INFO:teuthology.orchestra.run.smithi052.stdout: "path": "/", 2024-06-10T03:00:49.298 INFO:teuthology.orchestra.run.smithi052.stdout: "tag": "83aa1863-a6f4-45c9-a634-54ed2a6509bb", 2024-06-10T03:00:49.299 INFO:teuthology.orchestra.run.smithi052.stdout: "options": "recursive,force" 2024-06-10T03:00:49.299 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:00:49.299 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:00:49.299 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-10T03:00:49.299 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.293+0000 7f7533951700 1 -- 172.21.15.52:0/3659512289 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f75180a2c20 msgr2=0x7f75180a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:00:49.299 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.293+0000 7f7533951700 1 --2- 172.21.15.52:0/3659512289 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f75180a2c20 0x7f75180a50e0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f7528000c00 tx=0x7f7528005fd0 comp rx=0 tx=0).stop 2024-06-10T03:00:49.300 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.293+0000 7f7533951700 1 -- 172.21.15.52:0/3659512289 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f752c1066d0 msgr2=0x7f752c1a4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:00:49.300 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.293+0000 7f7533951700 1 --2- 172.21.15.52:0/3659512289 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f752c1066d0 0x7f752c1a4bf0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f7524007ee0 tx=0x7f752400d3b0 comp rx=0 tx=0).stop 2024-06-10T03:00:49.300 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.293+0000 7f7533951700 1 -- 172.21.15.52:0/3659512289 shutdown_connections 2024-06-10T03:00:49.300 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.293+0000 7f7533951700 1 --2- 172.21.15.52:0/3659512289 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f75180a2c20 0x7f75180a50e0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:49.300 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.293+0000 7f7533951700 1 --2- 172.21.15.52:0/3659512289 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f752c1066d0 0x7f752c1a4bf0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:49.300 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.293+0000 7f7533951700 1 --2- 172.21.15.52:0/3659512289 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f752c101680 0x7f752c199b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:49.300 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.293+0000 7f7533951700 1 --2- 172.21.15.52:0/3659512289 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f752c102030 0x7f752c19a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:49.300 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.293+0000 7f7533951700 1 -- 172.21.15.52:0/3659512289 >> 172.21.15.52:0/3659512289 conn(0x7f752c0773e0 msgr2=0x7f752c100bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:00:49.300 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.293+0000 7f7533951700 1 -- 172.21.15.52:0/3659512289 shutdown_connections 2024-06-10T03:00:49.300 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:49.293+0000 7f7533951700 1 -- 172.21.15.52:0/3659512289 wait complete. 2024-06-10T03:00:50.019 INFO:tasks.cephfs.filesystem:scrub status for tag:83aa1863-a6f4-45c9-a634-54ed2a6509bb - {'path': '/', 'tag': '83aa1863-a6f4-45c9-a634-54ed2a6509bb', 'options': 'recursive,force'} 2024-06-10T03:00:50.044 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T03:00:50.476 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:00:50 smithi052 ceph-mon[32362]: pgmap v633: 129 pgs: 129 active+clean; 572 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 359 KiB/s wr, 910 op/s 2024-06-10T03:00:50.477 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:00:50 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T03:00:50.638 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:00:50 smithi120 ceph-mon[36634]: pgmap v633: 129 pgs: 129 active+clean; 572 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 359 KiB/s wr, 910 op/s 2024-06-10T03:00:50.638 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:00:50 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T03:00:50.703 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:00:50 smithi188 ceph-mon[36599]: pgmap v633: 129 pgs: 129 active+clean; 572 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 359 KiB/s wr, 910 op/s 2024-06-10T03:00:50.704 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:00:50 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T03:00:51.638 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:00:51 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T03:00:51.638 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:00:51 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T03:00:51.639 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:00:51 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T03:00:51.703 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:00:51 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T03:00:51.704 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:00:51 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T03:00:51.704 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:00:51 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T03:00:51.727 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:00:51 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T03:00:51.727 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:00:51 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T03:00:51.727 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:00:51 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T03:00:52.580 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:00:52 smithi052 ceph-mon[32362]: pgmap v634: 129 pgs: 129 active+clean; 572 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 394 KiB/s wr, 788 op/s 2024-06-10T03:00:52.619 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.619+0000 7f7d69e59700 1 -- 172.21.15.52:0/592253978 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7d64107840 msgr2=0x7f7d6410f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:00:52.619 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.619+0000 7f7d69e59700 1 --2- 172.21.15.52:0/592253978 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7d64107840 0x7f7d6410f3c0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c00b0d0 tx=0x7f7d5c00b3e0 comp rx=0 tx=0).stop 2024-06-10T03:00:52.619 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.619+0000 7f7d69e59700 1 -- 172.21.15.52:0/592253978 shutdown_connections 2024-06-10T03:00:52.619 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.619+0000 7f7d69e59700 1 --2- 172.21.15.52:0/592253978 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7d64107840 0x7f7d6410f3c0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:52.619 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.619+0000 7f7d69e59700 1 --2- 172.21.15.52:0/592253978 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7d64102740 0x7f7d64102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:52.619 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.619+0000 7f7d69e59700 1 --2- 172.21.15.52:0/592253978 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7d641030f0 0x7f7d64107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:52.619 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.619+0000 7f7d69e59700 1 -- 172.21.15.52:0/592253978 >> 172.21.15.52:0/592253978 conn(0x7f7d640795f0 msgr2=0x7f7d64079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:00:52.620 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.619+0000 7f7d69e59700 1 -- 172.21.15.52:0/592253978 shutdown_connections 2024-06-10T03:00:52.620 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.619+0000 7f7d69e59700 1 -- 172.21.15.52:0/592253978 wait complete. 2024-06-10T03:00:52.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.619+0000 7f7d69e59700 1 Processor -- start 2024-06-10T03:00:52.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.619+0000 7f7d69e59700 1 -- start start 2024-06-10T03:00:52.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.620+0000 7f7d69e59700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7d64102740 0x7f7d6419ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:52.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.620+0000 7f7d69e59700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7d641030f0 0x7f7d6419b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:52.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.620+0000 7f7d69e59700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7d64107840 0x7f7d6419f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:52.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.620+0000 7f7d69e59700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f7d64112780 con 0x7f7d64107840 2024-06-10T03:00:52.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.620+0000 7f7d69e59700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f7d64112600 con 0x7f7d641030f0 2024-06-10T03:00:52.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.620+0000 7f7d69e59700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f7d64112900 con 0x7f7d64102740 2024-06-10T03:00:52.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.620+0000 7f7d63fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7d64107840 0x7f7d6419f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:52.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.620+0000 7f7d63fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7d64107840 0x7f7d6419f610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:59334/0 (socket says 172.21.15.52:59334) 2024-06-10T03:00:52.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.620+0000 7f7d63fff700 1 -- 172.21.15.52:0/241323056 learned_addr learned my addr 172.21.15.52:0/241323056 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:00:52.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.620+0000 7f7d637fe700 1 --2- 172.21.15.52:0/241323056 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7d64102740 0x7f7d6419ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:52.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.620+0000 7f7d5affd700 1 --2- 172.21.15.52:0/241323056 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7d641030f0 0x7f7d6419b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:52.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.620+0000 7f7d63fff700 1 -- 172.21.15.52:0/241323056 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7d64102740 msgr2=0x7f7d6419ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:00:52.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.620+0000 7f7d63fff700 1 --2- 172.21.15.52:0/241323056 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7d64102740 0x7f7d6419ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:52.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.620+0000 7f7d63fff700 1 -- 172.21.15.52:0/241323056 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7d641030f0 msgr2=0x7f7d6419b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:00:52.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.620+0000 7f7d63fff700 1 --2- 172.21.15.52:0/241323056 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7d641030f0 0x7f7d6419b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:52.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.620+0000 7f7d63fff700 1 -- 172.21.15.52:0/241323056 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d6419fd50 con 0x7f7d64107840 2024-06-10T03:00:52.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.620+0000 7f7d637fe700 1 --2- 172.21.15.52:0/241323056 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7d64102740 0x7f7d6419ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T03:00:52.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.621+0000 7f7d63fff700 1 --2- 172.21.15.52:0/241323056 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7d64107840 0x7f7d6419f610 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c006070 tx=0x7f7d5c004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:00:52.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.621+0000 7f7d617fa700 1 -- 172.21.15.52:0/241323056 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d5c021070 con 0x7f7d64107840 2024-06-10T03:00:52.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.621+0000 7f7d617fa700 1 -- 172.21.15.52:0/241323056 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d5c007b80 con 0x7f7d64107840 2024-06-10T03:00:52.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.621+0000 7f7d69e59700 1 -- 172.21.15.52:0/241323056 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d6419ffe0 con 0x7f7d64107840 2024-06-10T03:00:52.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.621+0000 7f7d617fa700 1 -- 172.21.15.52:0/241323056 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d5c005330 con 0x7f7d64107840 2024-06-10T03:00:52.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.621+0000 7f7d69e59700 1 -- 172.21.15.52:0/241323056 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d641a78c0 con 0x7f7d64107840 2024-06-10T03:00:52.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.623+0000 7f7d617fa700 1 -- 172.21.15.52:0/241323056 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f7d5c024070 con 0x7f7d64107840 2024-06-10T03:00:52.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.624+0000 7f7d617fa700 1 --2- 172.21.15.52:0/241323056 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f7d500a2c20 0x7f7d500a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:52.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.624+0000 7f7d637fe700 1 --2- 172.21.15.52:0/241323056 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f7d500a2c20 0x7f7d500a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:52.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.624+0000 7f7d617fa700 1 -- 172.21.15.52:0/241323056 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f7d5c00f020 con 0x7f7d64107840 2024-06-10T03:00:52.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.624+0000 7f7d637fe700 1 --2- 172.21.15.52:0/241323056 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f7d500a2c20 0x7f7d500a50e0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f7d54005950 tx=0x7f7d540058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:00:52.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.625+0000 7f7d746dcb80 1 Processor -- start 2024-06-10T03:00:52.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.625+0000 7f7d746dcb80 1 -- start start 2024-06-10T03:00:52.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.625+0000 7f7d746dcb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5607aeede500 0x5607aeede8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:52.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.625+0000 7f7d746dcb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5607aeedee20 0x5607aeed8710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:52.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.625+0000 7f7d746dcb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5607aeed8c50 0x5607aeed9070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:52.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.625+0000 7f7d746dcb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5607aedcb060 con 0x5607aeedee20 2024-06-10T03:00:52.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.625+0000 7f7d746dcb80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5607aee29950 con 0x5607aeede500 2024-06-10T03:00:52.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.625+0000 7f7d746dcb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5607aed91560 con 0x5607aeed8c50 2024-06-10T03:00:52.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.625+0000 7f7d5affd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5607aeedee20 0x5607aeed8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:52.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.625+0000 7f7d5affd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5607aeedee20 0x5607aeed8710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:59336/0 (socket says 172.21.15.52:59336) 2024-06-10T03:00:52.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.625+0000 7f7d5affd700 1 -- 172.21.15.52:0/1453758268 learned_addr learned my addr 172.21.15.52:0/1453758268 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:00:52.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.625+0000 7f7d637fe700 1 --2- 172.21.15.52:0/1453758268 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5607aeede500 0x5607aeede8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:52.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.625+0000 7f7d63fff700 1 --2- 172.21.15.52:0/1453758268 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5607aeed8c50 0x5607aeed9070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:52.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.625+0000 7f7d5affd700 1 -- 172.21.15.52:0/1453758268 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5607aeed8c50 msgr2=0x5607aeed9070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:00:52.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.625+0000 7f7d5affd700 1 --2- 172.21.15.52:0/1453758268 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5607aeed8c50 0x5607aeed9070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:52.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.625+0000 7f7d5affd700 1 -- 172.21.15.52:0/1453758268 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5607aeede500 msgr2=0x5607aeede8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:00:52.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.625+0000 7f7d5affd700 1 --2- 172.21.15.52:0/1453758268 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5607aeede500 0x5607aeede8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:52.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.625+0000 7f7d5affd700 1 -- 172.21.15.52:0/1453758268 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5607aed789d0 con 0x5607aeedee20 2024-06-10T03:00:52.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.625+0000 7f7d637fe700 1 --2- 172.21.15.52:0/1453758268 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5607aeede500 0x5607aeede8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T03:00:52.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.625+0000 7f7d5affd700 1 --2- 172.21.15.52:0/1453758268 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5607aeedee20 0x5607aeed8710 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f7d4c00ac80 tx=0x7f7d4c00af90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfd8bfa7febf4e3 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:00:52.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.626+0000 7f7d59ffb700 1 -- 172.21.15.52:0/1453758268 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d4c01d070 con 0x5607aeedee20 2024-06-10T03:00:52.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.626+0000 7f7d59ffb700 1 -- 172.21.15.52:0/1453758268 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d4c00fe90 con 0x5607aeedee20 2024-06-10T03:00:52.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.626+0000 7f7d59ffb700 1 -- 172.21.15.52:0/1453758268 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d4c010880 con 0x5607aeedee20 2024-06-10T03:00:52.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.626+0000 7f7d746dcb80 1 -- 172.21.15.52:0/1453758268 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5607aeedee20 msgr2=0x5607aeed8710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:00:52.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.626+0000 7f7d746dcb80 1 --2- 172.21.15.52:0/1453758268 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5607aeedee20 0x5607aeed8710 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f7d4c00ac80 tx=0x7f7d4c00af90 comp rx=0 tx=0).stop 2024-06-10T03:00:52.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.626+0000 7f7d746dcb80 1 -- 172.21.15.52:0/1453758268 shutdown_connections 2024-06-10T03:00:52.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.626+0000 7f7d746dcb80 1 --2- 172.21.15.52:0/1453758268 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5607aeed8c50 0x5607aeed9070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:52.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.626+0000 7f7d746dcb80 1 --2- 172.21.15.52:0/1453758268 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5607aeede500 0x5607aeede8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:52.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.626+0000 7f7d746dcb80 1 --2- 172.21.15.52:0/1453758268 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5607aeedee20 0x5607aeed8710 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:52.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.626+0000 7f7d746dcb80 1 -- 172.21.15.52:0/1453758268 >> 172.21.15.52:0/1453758268 conn(0x5607aee05180 msgr2=0x5607aeed3a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:00:52.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.627+0000 7f7d746dcb80 1 -- 172.21.15.52:0/1453758268 shutdown_connections 2024-06-10T03:00:52.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.627+0000 7f7d746dcb80 1 -- 172.21.15.52:0/1453758268 wait complete. 2024-06-10T03:00:52.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.627+0000 7f7d746dcb80 1 -- start start 2024-06-10T03:00:52.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.627+0000 7f7d746dcb80 1 Processor -- start 2024-06-10T03:00:52.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.627+0000 7f7d746dcb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5607aeed8c50 0x5607aeee4b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:52.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.628+0000 7f7d746dcb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5607aeede500 0x5607aeee1e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:52.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.628+0000 7f7d746dcb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5607aeedee20 0x5607aeee2360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:52.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.628+0000 7f7d746dcb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5607aedca0c0 con 0x5607aeede500 2024-06-10T03:00:52.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.628+0000 7f7d746dcb80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5607aee25910 con 0x5607aeed8c50 2024-06-10T03:00:52.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.628+0000 7f7d746dcb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5607aeee5800 con 0x5607aeedee20 2024-06-10T03:00:52.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.628+0000 7f7d746dcb80 10 client.0 _ll_register_callbacks cb 0x5607aedbf6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T03:00:52.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.628+0000 7f7d5affd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5607aeede500 0x5607aeee1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:52.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.628+0000 7f7d63fff700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5607aeedee20 0x5607aeee2360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:52.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.628+0000 7f7d5affd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5607aeede500 0x5607aeee1e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:59340/0 (socket says 172.21.15.52:59340) 2024-06-10T03:00:52.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.628+0000 7f7d5affd700 1 -- 172.21.15.52:0/2092986578 learned_addr learned my addr 172.21.15.52:0/2092986578 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:00:52.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.628+0000 7f7d63fff700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5607aeedee20 0x5607aeee2360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.52:32858/0 (socket says 172.21.15.52:32858) 2024-06-10T03:00:52.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.628+0000 7f7d637fe700 1 --2- 172.21.15.52:0/2092986578 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5607aeed8c50 0x5607aeee4b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:52.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.628+0000 7f7d5affd700 1 -- 172.21.15.52:0/2092986578 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5607aeedee20 msgr2=0x5607aeee2360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:00:52.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.628+0000 7f7d5affd700 1 --2- 172.21.15.52:0/2092986578 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5607aeedee20 0x5607aeee2360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:52.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.628+0000 7f7d5affd700 1 -- 172.21.15.52:0/2092986578 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5607aeed8c50 msgr2=0x5607aeee4b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:00:52.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.628+0000 7f7d5affd700 1 --2- 172.21.15.52:0/2092986578 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5607aeed8c50 0x5607aeee4b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:52.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.628+0000 7f7d5affd700 1 -- 172.21.15.52:0/2092986578 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5607aedef850 con 0x5607aeede500 2024-06-10T03:00:52.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.628+0000 7f7d5affd700 1 --2- 172.21.15.52:0/2092986578 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5607aeede500 0x5607aeee1e20 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f7d4c00a850 tx=0x7f7d4c010b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e0342cb3820864c server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:00:52.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.628+0000 7f7d42ffd700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-10T03:00:52.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.628+0000 7f7d42ffd700 1 -- 172.21.15.52:0/2092986578 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d4c01d040 con 0x5607aeede500 2024-06-10T03:00:52.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.629+0000 7f7d42ffd700 1 -- 172.21.15.52:0/2092986578 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f7d4c010df0 con 0x5607aeede500 2024-06-10T03:00:52.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.629+0000 7f7d746dcb80 1 -- 172.21.15.52:0/2092986578 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7d4c00fe90 con 0x5607aeede500 2024-06-10T03:00:52.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.629+0000 7f7d42ffd700 1 -- 172.21.15.52:0/2092986578 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f7d4c004870 con 0x5607aeede500 2024-06-10T03:00:52.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.629+0000 7f7d746dcb80 10 client.16512 fetch_fsmap learned FSMap version 36 2024-06-10T03:00:52.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.629+0000 7f7d746dcb80 1 -- 172.21.15.52:0/2092986578 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5607aeeeff40 con 0x5607aeede500 2024-06-10T03:00:52.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.629+0000 7f7d42ffd700 1 -- 172.21.15.52:0/2092986578 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d4c00fe90 con 0x5607aeede500 2024-06-10T03:00:52.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.629+0000 7f7d42ffd700 1 -- 172.21.15.52:0/2092986578 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 36) ==== 8378+0+0 (secure 0 0 0) 0x7f7d4c020030 con 0x5607aeede500 2024-06-10T03:00:52.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.629+0000 7f7d746dcb80 10 client.16512 fetch_fsmap finished waiting for FSMap version 36 2024-06-10T03:00:52.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.629+0000 7f7d746dcb80 10 client.16512 resolve_mds: validated gid 24484 aka daemon mds.i 2024-06-10T03:00:52.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.629+0000 7f7d746dcb80 20 client.16512 populate_metadata read hostname 'smithi052' 2024-06-10T03:00:52.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.629+0000 7f7d746dcb80 1 --2- 172.21.15.52:0/2092986578 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x5607aeef2410 0x5607aeef4800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:52.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.629+0000 7f7d746dcb80 4 client.16512 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T03:00:52.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.629+0000 7f7d746dcb80 1 -- 172.21.15.52:0/2092986578 --> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5607aee36e50 con 0x5607aeef2410 2024-06-10T03:00:52.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.630+0000 7f7d637fe700 1 --2- 172.21.15.52:0/2092986578 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x5607aeef2410 0x5607aeef4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:52.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.630+0000 7f7d637fe700 1 --2- 172.21.15.52:0/2092986578 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x5607aeef2410 0x5607aeef4800 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6719dca009615840 server_cookie=8705afbb709414be in_seq=0 out_seq=0 2024-06-10T03:00:52.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.630+0000 7f7d42ffd700 10 client.16512 ms_handle_connect on v2:172.21.15.188:6838/310444836 2024-06-10T03:00:52.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.631+0000 7f7d42ffd700 1 -- 172.21.15.52:0/2092986578 <== mds.0 v2:172.21.15.188:6838/310444836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5607aee36e50 con 0x5607aeef2410 2024-06-10T03:00:52.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.631+0000 7f7d42ffd700 10 client.16512 handle_command_reply: tid=0 2024-06-10T03:00:52.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.631+0000 7f7d746dcb80 1 client.16512 shutdown 2024-06-10T03:00:52.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.631+0000 7f7d746dcb80 1 -- 172.21.15.52:0/2092986578 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5607aeede500 msgr2=0x5607aeee1e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:00:52.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.631+0000 7f7d746dcb80 1 --2- 172.21.15.52:0/2092986578 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5607aeede500 0x5607aeee1e20 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f7d4c00a850 tx=0x7f7d4c010b80 comp rx=0 tx=0).stop 2024-06-10T03:00:52.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.631+0000 7f7d746dcb80 1 -- 172.21.15.52:0/2092986578 shutdown_connections 2024-06-10T03:00:52.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.631+0000 7f7d746dcb80 1 --2- 172.21.15.52:0/2092986578 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x5607aeef2410 0x5607aeef4800 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:52.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.631+0000 7f7d42ffd700 0 client.16512 ms_handle_reset on v2:172.21.15.188:6838/310444836 2024-06-10T03:00:52.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.631+0000 7f7d746dcb80 1 --2- 172.21.15.52:0/2092986578 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5607aeedee20 0x5607aeee2360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:52.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.631+0000 7f7d746dcb80 1 --2- 172.21.15.52:0/2092986578 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5607aeed8c50 0x5607aeee4b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:52.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.631+0000 7f7d746dcb80 1 --2- 172.21.15.52:0/2092986578 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5607aeede500 0x5607aeee1e20 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:52.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.631+0000 7f7d746dcb80 1 -- 172.21.15.52:0/2092986578 >> 172.21.15.52:0/2092986578 conn(0x5607aee05180 msgr2=0x5607aeed2bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:00:52.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.631+0000 7f7d746dcb80 1 -- 172.21.15.52:0/2092986578 shutdown_connections 2024-06-10T03:00:52.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.631+0000 7f7d746dcb80 1 -- 172.21.15.52:0/2092986578 wait complete. 2024-06-10T03:00:52.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.631+0000 7f7d746dcb80 20 client.16512 trim_cache size 0 max 16384 2024-06-10T03:00:52.638 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:00:52 smithi120 ceph-mon[36634]: pgmap v634: 129 pgs: 129 active+clean; 572 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 394 KiB/s wr, 788 op/s 2024-06-10T03:00:52.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.651+0000 7f7d746dcb80 1 Processor -- start 2024-06-10T03:00:52.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.651+0000 7f7d746dcb80 1 -- start start 2024-06-10T03:00:52.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.651+0000 7f7d746dcb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5607aeed8c50 0x5607aeedf9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:52.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.651+0000 7f7d746dcb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5607aeede500 0x5607aeed3330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:52.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.651+0000 7f7d746dcb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5607aeedee20 0x5607aeed3870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:52.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.651+0000 7f7d746dcb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f7d641b0490 con 0x5607aeedee20 2024-06-10T03:00:52.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.651+0000 7f7d746dcb80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f7d641b2090 con 0x5607aeed8c50 2024-06-10T03:00:52.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.651+0000 7f7d746dcb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f7d641b0b60 con 0x5607aeede500 2024-06-10T03:00:52.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.652+0000 7f7d63fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5607aeedee20 0x5607aeed3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:52.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.652+0000 7f7d63fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5607aeedee20 0x5607aeed3870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:59356/0 (socket says 172.21.15.52:59356) 2024-06-10T03:00:52.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.652+0000 7f7d63fff700 1 -- 172.21.15.52:0/88251984 learned_addr learned my addr 172.21.15.52:0/88251984 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:00:52.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.652+0000 7f7d5affd700 1 --2- 172.21.15.52:0/88251984 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5607aeede500 0x5607aeed3330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:52.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.652+0000 7f7d63fff700 1 -- 172.21.15.52:0/88251984 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5607aeede500 msgr2=0x5607aeed3330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:00:52.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.652+0000 7f7d63fff700 1 --2- 172.21.15.52:0/88251984 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5607aeede500 0x5607aeed3330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:52.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.652+0000 7f7d63fff700 1 -- 172.21.15.52:0/88251984 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5607aeed8c50 msgr2=0x5607aeedf9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:00:52.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.652+0000 7f7d63fff700 1 --2- 172.21.15.52:0/88251984 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5607aeed8c50 0x5607aeedf9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:52.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.652+0000 7f7d63fff700 1 -- 172.21.15.52:0/88251984 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5607aeeeeb00 con 0x5607aeedee20 2024-06-10T03:00:52.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.652+0000 7f7d63fff700 1 --2- 172.21.15.52:0/88251984 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5607aeedee20 0x5607aeed3870 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c009e60 tx=0x7f7d5c009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c6c2503ee361abd server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:00:52.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.652+0000 7f7d597fa700 1 -- 172.21.15.52:0/88251984 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d5c09cf10 con 0x5607aeedee20 2024-06-10T03:00:52.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.652+0000 7f7d597fa700 1 -- 172.21.15.52:0/88251984 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d5c0a12c0 con 0x5607aeedee20 2024-06-10T03:00:52.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.652+0000 7f7d597fa700 1 -- 172.21.15.52:0/88251984 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d5c09d3d0 con 0x5607aeedee20 2024-06-10T03:00:52.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.653+0000 7f7d746dcb80 1 -- 172.21.15.52:0/88251984 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5607aeedee20 msgr2=0x5607aeed3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:00:52.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.653+0000 7f7d746dcb80 1 --2- 172.21.15.52:0/88251984 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5607aeedee20 0x5607aeed3870 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c009e60 tx=0x7f7d5c009040 comp rx=0 tx=0).stop 2024-06-10T03:00:52.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.653+0000 7f7d746dcb80 1 -- 172.21.15.52:0/88251984 shutdown_connections 2024-06-10T03:00:52.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.653+0000 7f7d746dcb80 1 --2- 172.21.15.52:0/88251984 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5607aeedee20 0x5607aeed3870 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:52.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.653+0000 7f7d746dcb80 1 --2- 172.21.15.52:0/88251984 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5607aeed8c50 0x5607aeedf9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:52.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.653+0000 7f7d746dcb80 1 --2- 172.21.15.52:0/88251984 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5607aeede500 0x5607aeed3330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:52.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.653+0000 7f7d746dcb80 1 -- 172.21.15.52:0/88251984 >> 172.21.15.52:0/88251984 conn(0x5607aee05180 msgr2=0x5607aeedf2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:00:52.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.653+0000 7f7d746dcb80 1 -- 172.21.15.52:0/88251984 shutdown_connections 2024-06-10T03:00:52.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.653+0000 7f7d746dcb80 1 -- 172.21.15.52:0/88251984 wait complete. 2024-06-10T03:00:52.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.653+0000 7f7d746dcb80 1 -- start start 2024-06-10T03:00:52.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.654+0000 7f7d746dcb80 1 Processor -- start 2024-06-10T03:00:52.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.654+0000 7f7d746dcb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5607aeed8c50 0x5607aeef7d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:52.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.654+0000 7f7d746dcb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5607aeede500 0x5607aeef6130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:52.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.654+0000 7f7d746dcb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5607aeedee20 0x5607aeef6670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:52.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.654+0000 7f7d746dcb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f7d641b19c0 con 0x5607aeedee20 2024-06-10T03:00:52.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.654+0000 7f7d746dcb80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f7d641b12f0 con 0x5607aeede500 2024-06-10T03:00:52.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.654+0000 7f7d746dcb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f7d641ae430 con 0x5607aeed8c50 2024-06-10T03:00:52.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.654+0000 7f7d746dcb80 10 client.0 _ll_register_callbacks cb 0x5607aedbf6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T03:00:52.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.654+0000 7f7d63fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5607aeedee20 0x5607aeef6670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:52.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.654+0000 7f7d63fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5607aeedee20 0x5607aeef6670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:59358/0 (socket says 172.21.15.52:59358) 2024-06-10T03:00:52.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.654+0000 7f7d63fff700 1 -- 172.21.15.52:0/2532002885 learned_addr learned my addr 172.21.15.52:0/2532002885 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:00:52.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.654+0000 7f7d5affd700 1 --2- 172.21.15.52:0/2532002885 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5607aeede500 0x5607aeef6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:52.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.654+0000 7f7d637fe700 1 --2- 172.21.15.52:0/2532002885 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5607aeed8c50 0x5607aeef7d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:52.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.654+0000 7f7d63fff700 1 -- 172.21.15.52:0/2532002885 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5607aeed8c50 msgr2=0x5607aeef7d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:00:52.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.654+0000 7f7d63fff700 1 --2- 172.21.15.52:0/2532002885 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5607aeed8c50 0x5607aeef7d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:52.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.654+0000 7f7d63fff700 1 -- 172.21.15.52:0/2532002885 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5607aeede500 msgr2=0x5607aeef6130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:00:52.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.654+0000 7f7d63fff700 1 --2- 172.21.15.52:0/2532002885 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5607aeede500 0x5607aeef6130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:52.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.654+0000 7f7d63fff700 1 -- 172.21.15.52:0/2532002885 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5607aeef6ce0 con 0x5607aeedee20 2024-06-10T03:00:52.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.655+0000 7f7d63fff700 1 --2- 172.21.15.52:0/2532002885 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5607aeedee20 0x5607aeef6670 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c09cea0 tx=0x7f7d5c0a5af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4a36e3e38aa4dfd server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:00:52.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.655+0000 7f7d42ffd700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-10T03:00:52.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.655+0000 7f7d42ffd700 1 -- 172.21.15.52:0/2532002885 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d5c09d170 con 0x5607aeedee20 2024-06-10T03:00:52.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.655+0000 7f7d42ffd700 1 -- 172.21.15.52:0/2532002885 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f7d5c09d310 con 0x5607aeedee20 2024-06-10T03:00:52.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.655+0000 7f7d746dcb80 1 -- 172.21.15.52:0/2532002885 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7d5c0a12c0 con 0x5607aeedee20 2024-06-10T03:00:52.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.655+0000 7f7d42ffd700 1 -- 172.21.15.52:0/2532002885 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f7d5c086120 con 0x5607aeedee20 2024-06-10T03:00:52.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.655+0000 7f7d746dcb80 10 client.16518 fetch_fsmap learned FSMap version 36 2024-06-10T03:00:52.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.655+0000 7f7d746dcb80 1 -- 172.21.15.52:0/2532002885 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5607aef5dc50 con 0x5607aeedee20 2024-06-10T03:00:52.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.655+0000 7f7d42ffd700 1 -- 172.21.15.52:0/2532002885 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d5c0a12c0 con 0x5607aeedee20 2024-06-10T03:00:52.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.655+0000 7f7d42ffd700 1 -- 172.21.15.52:0/2532002885 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 36) ==== 8378+0+0 (secure 0 0 0) 0x7f7d5c083030 con 0x5607aeedee20 2024-06-10T03:00:52.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.655+0000 7f7d746dcb80 10 client.16518 fetch_fsmap finished waiting for FSMap version 36 2024-06-10T03:00:52.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.655+0000 7f7d746dcb80 10 client.16518 resolve_mds: validated gid 24484 aka daemon mds.i 2024-06-10T03:00:52.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.655+0000 7f7d746dcb80 20 client.16518 populate_metadata read hostname 'smithi052' 2024-06-10T03:00:52.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.655+0000 7f7d746dcb80 1 --2- 172.21.15.52:0/2532002885 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x5607aeef2410 0x5607aef60170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:00:52.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.655+0000 7f7d746dcb80 4 client.16518 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "c8954a8e", "release": true, "await": true}] 2024-06-10T03:00:52.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.655+0000 7f7d746dcb80 1 -- 172.21.15.52:0/2532002885 --> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] -- command(tid 0: {"prefix": "quiesce db", "set_id": "c8954a8e", "release": true, "await": true}) -- 0x5607aee313d0 con 0x5607aeef2410 2024-06-10T03:00:52.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.656+0000 7f7d637fe700 1 --2- 172.21.15.52:0/2532002885 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x5607aeef2410 0x5607aef60170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:00:52.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.656+0000 7f7d637fe700 1 --2- 172.21.15.52:0/2532002885 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x5607aeef2410 0x5607aef60170 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b3a0b68529f8c5a server_cookie=6d755129ac736b04 in_seq=0 out_seq=0 2024-06-10T03:00:52.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:52.656+0000 7f7d42ffd700 10 client.16518 ms_handle_connect on v2:172.21.15.188:6838/310444836 2024-06-10T03:00:52.703 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:00:52 smithi188 ceph-mon[36599]: pgmap v634: 129 pgs: 129 active+clean; 572 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 394 KiB/s wr, 788 op/s 2024-06-10T03:00:53.673 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:53.672+0000 7f7d42ffd700 1 -- 172.21.15.52:0/2532002885 <== mds.0 v2:172.21.15.188:6838/310444836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5607aee313d0 con 0x5607aeef2410 2024-06-10T03:00:53.673 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:53.672+0000 7f7d42ffd700 10 client.16518 handle_command_reply: tid=0 2024-06-10T03:00:53.673 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:53.673+0000 7f7d746dcb80 1 client.16518 shutdown 2024-06-10T03:00:53.673 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:53.673+0000 7f7d746dcb80 1 -- 172.21.15.52:0/2532002885 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5607aeedee20 msgr2=0x5607aeef6670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:00:53.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:53.673+0000 7f7d746dcb80 1 --2- 172.21.15.52:0/2532002885 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5607aeedee20 0x5607aeef6670 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c09cea0 tx=0x7f7d5c0a5af0 comp rx=0 tx=0).stop 2024-06-10T03:00:53.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:53.673+0000 7f7d746dcb80 1 -- 172.21.15.52:0/2532002885 shutdown_connections 2024-06-10T03:00:53.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:53.673+0000 7f7d746dcb80 1 --2- 172.21.15.52:0/2532002885 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x5607aeef2410 0x5607aef60170 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:53.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:53.673+0000 7f7d746dcb80 1 --2- 172.21.15.52:0/2532002885 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5607aeedee20 0x5607aeef6670 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:53.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:53.673+0000 7f7d746dcb80 1 --2- 172.21.15.52:0/2532002885 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5607aeed8c50 0x5607aeef7d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:53.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:53.673+0000 7f7d746dcb80 1 --2- 172.21.15.52:0/2532002885 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5607aeede500 0x5607aeef6130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:53.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:53.673+0000 7f7d746dcb80 1 -- 172.21.15.52:0/2532002885 >> 172.21.15.52:0/2532002885 conn(0x5607aee05180 msgr2=0x5607aeee3650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:00:53.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:53.673+0000 7f7d42ffd700 0 client.16518 ms_handle_reset on v2:172.21.15.188:6838/310444836 2024-06-10T03:00:53.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:53.673+0000 7f7d746dcb80 1 -- 172.21.15.52:0/2532002885 shutdown_connections 2024-06-10T03:00:53.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:53.673+0000 7f7d746dcb80 1 -- 172.21.15.52:0/2532002885 wait complete. 2024-06-10T03:00:53.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:53.673+0000 7f7d746dcb80 20 client.16518 trim_cache size 0 max 16384 2024-06-10T03:00:53.675 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-10T03:00:53.675 INFO:teuthology.orchestra.run.smithi052.stdout: "epoch": 36, 2024-06-10T03:00:53.675 INFO:teuthology.orchestra.run.smithi052.stdout: "leader": 24484, 2024-06-10T03:00:53.675 INFO:teuthology.orchestra.run.smithi052.stdout: "set_version": 20, 2024-06-10T03:00:53.675 INFO:teuthology.orchestra.run.smithi052.stdout: "sets": { 2024-06-10T03:00:53.675 INFO:teuthology.orchestra.run.smithi052.stdout: "c8954a8e": { 2024-06-10T03:00:53.675 INFO:teuthology.orchestra.run.smithi052.stdout: "version": 20, 2024-06-10T03:00:53.675 INFO:teuthology.orchestra.run.smithi052.stdout: "age_ref": 0.0, 2024-06-10T03:00:53.675 INFO:teuthology.orchestra.run.smithi052.stdout: "state": { 2024-06-10T03:00:53.675 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "RELEASED", 2024-06-10T03:00:53.676 INFO:teuthology.orchestra.run.smithi052.stdout: "age": 0.0 2024-06-10T03:00:53.676 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:00:53.676 INFO:teuthology.orchestra.run.smithi052.stdout: "timeout": 90.0, 2024-06-10T03:00:53.676 INFO:teuthology.orchestra.run.smithi052.stdout: "expiration": 149.4, 2024-06-10T03:00:53.676 INFO:teuthology.orchestra.run.smithi052.stdout: "members": { 2024-06-10T03:00:53.676 INFO:teuthology.orchestra.run.smithi052.stdout: "file:/": { 2024-06-10T03:00:53.676 INFO:teuthology.orchestra.run.smithi052.stdout: "excluded": false, 2024-06-10T03:00:53.676 INFO:teuthology.orchestra.run.smithi052.stdout: "state": { 2024-06-10T03:00:53.676 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "RELEASED", 2024-06-10T03:00:53.676 INFO:teuthology.orchestra.run.smithi052.stdout: "age": 0.0 2024-06-10T03:00:53.676 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:00:53.676 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:00:53.676 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:00:53.676 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:00:53.677 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:00:53.677 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-10T03:00:53.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:53.674+0000 7f7d69e59700 1 -- 172.21.15.52:0/241323056 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f7d500a2c20 msgr2=0x7f7d500a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:00:53.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:53.674+0000 7f7d69e59700 1 --2- 172.21.15.52:0/241323056 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f7d500a2c20 0x7f7d500a50e0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f7d54005950 tx=0x7f7d540058e0 comp rx=0 tx=0).stop 2024-06-10T03:00:53.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:53.674+0000 7f7d69e59700 1 -- 172.21.15.52:0/241323056 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7d64107840 msgr2=0x7f7d6419f610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:00:53.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:53.674+0000 7f7d69e59700 1 --2- 172.21.15.52:0/241323056 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7d64107840 0x7f7d6419f610 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c006070 tx=0x7f7d5c004d00 comp rx=0 tx=0).stop 2024-06-10T03:00:53.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:53.674+0000 7f7d69e59700 1 -- 172.21.15.52:0/241323056 shutdown_connections 2024-06-10T03:00:53.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:53.674+0000 7f7d69e59700 1 --2- 172.21.15.52:0/241323056 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f7d500a2c20 0x7f7d500a50e0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:53.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:53.674+0000 7f7d69e59700 1 --2- 172.21.15.52:0/241323056 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7d64107840 0x7f7d6419f610 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:53.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:53.674+0000 7f7d69e59700 1 --2- 172.21.15.52:0/241323056 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7d64102740 0x7f7d6419ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:53.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:53.674+0000 7f7d69e59700 1 --2- 172.21.15.52:0/241323056 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7d641030f0 0x7f7d6419b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:00:53.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:53.674+0000 7f7d69e59700 1 -- 172.21.15.52:0/241323056 >> 172.21.15.52:0/241323056 conn(0x7f7d640795f0 msgr2=0x7f7d64100690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:00:53.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:53.674+0000 7f7d69e59700 1 -- 172.21.15.52:0/241323056 shutdown_connections 2024-06-10T03:00:53.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:00:53.674+0000 7f7d69e59700 1 -- 172.21.15.52:0/241323056 wait complete. 2024-06-10T03:00:54.461 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:00:54 smithi052 ceph-mon[32362]: pgmap v635: 129 pgs: 129 active+clean; 575 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 595 KiB/s wr, 770 op/s 2024-06-10T03:00:54.491 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'c8954a8e', total seconds elapsed: 40.8 2024-06-10T03:00:54.638 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:00:54 smithi120 ceph-mon[36634]: pgmap v635: 129 pgs: 129 active+clean; 575 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 595 KiB/s wr, 770 op/s 2024-06-10T03:00:54.703 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:00:54 smithi188 ceph-mon[36599]: pgmap v635: 129 pgs: 129 active+clean; 575 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 595 KiB/s wr, 770 op/s 2024-06-10T03:00:56.642 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:00:56 smithi120 ceph-mon[36634]: pgmap v636: 129 pgs: 129 active+clean; 575 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 418 KiB/s wr, 631 op/s 2024-06-10T03:00:56.703 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:00:56 smithi188 ceph-mon[36599]: pgmap v636: 129 pgs: 129 active+clean; 575 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 418 KiB/s wr, 631 op/s 2024-06-10T03:00:56.726 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:00:56 smithi052 ceph-mon[32362]: pgmap v636: 129 pgs: 129 active+clean; 575 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 418 KiB/s wr, 631 op/s 2024-06-10T03:00:57.250 INFO:tasks.workunit.client.0.smithi052.stdout: 101 9909 104 7352 31 5600 141 2024-06-10T03:00:57.639 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:00:57 smithi120 ceph-mon[36634]: scrub summary: idle 2024-06-10T03:00:57.640 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:00:57 smithi120 ceph-mon[36634]: scrub summary: idle 2024-06-10T03:00:57.640 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:00:57 smithi120 ceph-mon[36634]: scrub summary: idle 2024-06-10T03:00:57.640 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:00:57 smithi120 ceph-mon[36634]: scrub summary: idle 2024-06-10T03:00:57.703 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:00:57 smithi188 ceph-mon[36599]: scrub summary: idle 2024-06-10T03:00:57.703 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:00:57 smithi188 ceph-mon[36599]: scrub summary: idle 2024-06-10T03:00:57.704 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:00:57 smithi188 ceph-mon[36599]: scrub summary: idle 2024-06-10T03:00:57.704 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:00:57 smithi188 ceph-mon[36599]: scrub summary: idle 2024-06-10T03:00:57.726 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:00:57 smithi052 ceph-mon[32362]: scrub summary: idle 2024-06-10T03:00:57.727 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:00:57 smithi052 ceph-mon[32362]: scrub summary: idle 2024-06-10T03:00:57.727 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:00:57 smithi052 ceph-mon[32362]: scrub summary: idle 2024-06-10T03:00:57.727 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:00:57 smithi052 ceph-mon[32362]: scrub summary: idle 2024-06-10T03:00:58.638 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:00:58 smithi120 ceph-mon[36634]: pgmap v637: 129 pgs: 129 active+clean; 576 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 475 KiB/s wr, 580 op/s 2024-06-10T03:00:58.703 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:00:58 smithi188 ceph-mon[36599]: pgmap v637: 129 pgs: 129 active+clean; 576 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 475 KiB/s wr, 580 op/s 2024-06-10T03:00:58.726 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:00:58 smithi052 ceph-mon[32362]: pgmap v637: 129 pgs: 129 active+clean; 576 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 475 KiB/s wr, 580 op/s 2024-06-10T03:01:00.638 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:01:00 smithi120 ceph-mon[36634]: pgmap v638: 129 pgs: 129 active+clean; 576 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 436 KiB/s rd, 418 KiB/s wr, 279 op/s 2024-06-10T03:01:00.703 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:01:00 smithi188 ceph-mon[36599]: pgmap v638: 129 pgs: 129 active+clean; 576 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 436 KiB/s rd, 418 KiB/s wr, 279 op/s 2024-06-10T03:01:00.727 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:01:00 smithi052 ceph-mon[32362]: pgmap v638: 129 pgs: 129 active+clean; 576 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 436 KiB/s rd, 418 KiB/s wr, 279 op/s 2024-06-10T03:01:02.638 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:01:02 smithi120 ceph-mon[36634]: pgmap v639: 129 pgs: 129 active+clean; 576 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 416 KiB/s wr, 105 op/s 2024-06-10T03:01:02.703 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:01:02 smithi188 ceph-mon[36599]: pgmap v639: 129 pgs: 129 active+clean; 576 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 416 KiB/s wr, 105 op/s 2024-06-10T03:01:02.727 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:01:02 smithi052 ceph-mon[32362]: pgmap v639: 129 pgs: 129 active+clean; 576 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 416 KiB/s wr, 105 op/s 2024-06-10T03:01:04.638 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:01:04 smithi120 ceph-mon[36634]: pgmap v640: 129 pgs: 129 active+clean; 554 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 517 KiB/s rd, 631 KiB/s wr, 160 op/s 2024-06-10T03:01:04.703 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:01:04 smithi188 ceph-mon[36599]: pgmap v640: 129 pgs: 129 active+clean; 554 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 517 KiB/s rd, 631 KiB/s wr, 160 op/s 2024-06-10T03:01:04.727 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:01:04 smithi052 ceph-mon[32362]: pgmap v640: 129 pgs: 129 active+clean; 554 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 517 KiB/s rd, 631 KiB/s wr, 160 op/s 2024-06-10T03:01:06.638 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:01:06 smithi120 ceph-mon[36634]: pgmap v641: 129 pgs: 129 active+clean; 549 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 485 KiB/s rd, 451 KiB/s wr, 155 op/s 2024-06-10T03:01:06.704 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:01:06 smithi188 ceph-mon[36599]: pgmap v641: 129 pgs: 129 active+clean; 549 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 485 KiB/s rd, 451 KiB/s wr, 155 op/s 2024-06-10T03:01:06.727 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:01:06 smithi052 ceph-mon[32362]: pgmap v641: 129 pgs: 129 active+clean; 549 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 485 KiB/s rd, 451 KiB/s wr, 155 op/s 2024-06-10T03:01:07.250 INFO:tasks.workunit.client.0.smithi052.stdout: 104 665750 237 469608 206 364361 653 2024-06-10T03:01:08.638 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:01:08 smithi120 ceph-mon[36634]: pgmap v642: 129 pgs: 129 active+clean; 559 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 600 KiB/s rd, 1.3 MiB/s wr, 203 op/s 2024-06-10T03:01:08.703 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:01:08 smithi188 ceph-mon[36599]: pgmap v642: 129 pgs: 129 active+clean; 559 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 600 KiB/s rd, 1.3 MiB/s wr, 203 op/s 2024-06-10T03:01:08.726 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:01:08 smithi052 ceph-mon[32362]: pgmap v642: 129 pgs: 129 active+clean; 559 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 600 KiB/s rd, 1.3 MiB/s wr, 203 op/s 2024-06-10T03:01:10.638 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:01:10 smithi120 ceph-mon[36634]: pgmap v643: 129 pgs: 129 active+clean; 568 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 763 KiB/s rd, 2.0 MiB/s wr, 240 op/s 2024-06-10T03:01:10.703 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:01:10 smithi188 ceph-mon[36599]: pgmap v643: 129 pgs: 129 active+clean; 568 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 763 KiB/s rd, 2.0 MiB/s wr, 240 op/s 2024-06-10T03:01:10.728 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:01:10 smithi052 ceph-mon[32362]: pgmap v643: 129 pgs: 129 active+clean; 568 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 763 KiB/s rd, 2.0 MiB/s wr, 240 op/s 2024-06-10T03:01:12.638 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:01:12 smithi120 ceph-mon[36634]: pgmap v644: 129 pgs: 129 active+clean; 571 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1014 KiB/s rd, 2.3 MiB/s wr, 247 op/s 2024-06-10T03:01:12.703 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:01:12 smithi188 ceph-mon[36599]: pgmap v644: 129 pgs: 129 active+clean; 571 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1014 KiB/s rd, 2.3 MiB/s wr, 247 op/s 2024-06-10T03:01:12.726 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:01:12 smithi052 ceph-mon[32362]: pgmap v644: 129 pgs: 129 active+clean; 571 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1014 KiB/s rd, 2.3 MiB/s wr, 247 op/s 2024-06-10T03:01:14.638 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:01:14 smithi120 ceph-mon[36634]: pgmap v645: 129 pgs: 129 active+clean; 573 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.7 MiB/s wr, 301 op/s 2024-06-10T03:01:14.703 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:01:14 smithi188 ceph-mon[36599]: pgmap v645: 129 pgs: 129 active+clean; 573 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.7 MiB/s wr, 301 op/s 2024-06-10T03:01:14.727 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:01:14 smithi052 ceph-mon[32362]: pgmap v645: 129 pgs: 129 active+clean; 573 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.7 MiB/s wr, 301 op/s 2024-06-10T03:01:16.638 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:01:16 smithi120 ceph-mon[36634]: pgmap v646: 129 pgs: 129 active+clean; 572 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.4 MiB/s wr, 275 op/s 2024-06-10T03:01:16.703 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:01:16 smithi188 ceph-mon[36599]: pgmap v646: 129 pgs: 129 active+clean; 572 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.4 MiB/s wr, 275 op/s 2024-06-10T03:01:16.727 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:01:16 smithi052 ceph-mon[32362]: pgmap v646: 129 pgs: 129 active+clean; 572 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.4 MiB/s wr, 275 op/s 2024-06-10T03:01:17.250 INFO:tasks.workunit.client.0.smithi052.stdout: 111 1960005 704 1384036 600 1049055 1245 2024-06-10T03:01:18.638 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:01:18 smithi120 ceph-mon[36634]: pgmap v647: 129 pgs: 129 active+clean; 577 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.1 MiB/s wr, 339 op/s 2024-06-10T03:01:18.703 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:01:18 smithi188 ceph-mon[36599]: pgmap v647: 129 pgs: 129 active+clean; 577 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.1 MiB/s wr, 339 op/s 2024-06-10T03:01:18.727 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:01:18 smithi052 ceph-mon[32362]: pgmap v647: 129 pgs: 129 active+clean; 577 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.1 MiB/s wr, 339 op/s 2024-06-10T03:01:20.020 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-10T03:01:20.317 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T03:01:20.638 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:01:20 smithi120 ceph-mon[36634]: pgmap v648: 129 pgs: 129 active+clean; 576 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.3 MiB/s wr, 365 op/s 2024-06-10T03:01:20.679 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:01:20 smithi052 ceph-mon[32362]: pgmap v648: 129 pgs: 129 active+clean; 576 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.3 MiB/s wr, 365 op/s 2024-06-10T03:01:20.703 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:01:20 smithi188 ceph-mon[36599]: pgmap v648: 129 pgs: 129 active+clean; 576 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.3 MiB/s wr, 365 op/s 2024-06-10T03:01:22.638 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:01:22 smithi120 ceph-mon[36634]: pgmap v649: 129 pgs: 129 active+clean; 577 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.6 MiB/s wr, 339 op/s 2024-06-10T03:01:22.703 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:01:22 smithi188 ceph-mon[36599]: pgmap v649: 129 pgs: 129 active+clean; 577 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.6 MiB/s wr, 339 op/s 2024-06-10T03:01:22.727 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:01:22 smithi052 ceph-mon[32362]: pgmap v649: 129 pgs: 129 active+clean; 577 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.6 MiB/s wr, 339 op/s 2024-06-10T03:01:23.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.574+0000 7f0f40abc700 1 -- 172.21.15.52:0/1634152597 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0f3c109e50 msgr2=0x7f0f3c115d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:01:23.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.574+0000 7f0f40abc700 1 --2- 172.21.15.52:0/1634152597 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0f3c109e50 0x7f0f3c115d80 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f0f34009f90 tx=0x7f0f3400b330 comp rx=0 tx=0).stop 2024-06-10T03:01:23.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.576+0000 7f0f40abc700 1 -- 172.21.15.52:0/1634152597 shutdown_connections 2024-06-10T03:01:23.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.576+0000 7f0f40abc700 1 --2- 172.21.15.52:0/1634152597 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0f3c109e50 0x7f0f3c115d80 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:23.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.576+0000 7f0f40abc700 1 --2- 172.21.15.52:0/1634152597 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0f3c10dee0 0x7f0f3c10e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:23.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.576+0000 7f0f40abc700 1 --2- 172.21.15.52:0/1634152597 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0f3c109490 0x7f0f3c109910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:23.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.576+0000 7f0f40abc700 1 -- 172.21.15.52:0/1634152597 >> 172.21.15.52:0/1634152597 conn(0x7f0f3c06d9e0 msgr2=0x7f0f3c06ddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:01:23.578 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.577+0000 7f0f40abc700 1 -- 172.21.15.52:0/1634152597 shutdown_connections 2024-06-10T03:01:23.579 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.577+0000 7f0f40abc700 1 -- 172.21.15.52:0/1634152597 wait complete. 2024-06-10T03:01:23.579 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.578+0000 7f0f40abc700 1 Processor -- start 2024-06-10T03:01:23.580 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.578+0000 7f0f40abc700 1 -- start start 2024-06-10T03:01:23.580 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.579+0000 7f0f40abc700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0f3c109490 0x7f0f3c11f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:01:23.580 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.579+0000 7f0f40abc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0f3c109e50 0x7f0f3c11f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:01:23.580 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.579+0000 7f0f40abc700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0f3c10dee0 0x7f0f3c126f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:01:23.580 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.579+0000 7f0f40abc700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f0f3c118e60 con 0x7f0f3c109e50 2024-06-10T03:01:23.580 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.579+0000 7f0f40abc700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f0f3c118ce0 con 0x7f0f3c10dee0 2024-06-10T03:01:23.580 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.579+0000 7f0f40abc700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f0f3c118fe0 con 0x7f0f3c109490 2024-06-10T03:01:23.580 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.579+0000 7f0f3ad9d700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0f3c10dee0 0x7f0f3c126f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:01:23.580 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.580+0000 7f0f3ad9d700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0f3c10dee0 0x7f0f3c126f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.52:49764/0 (socket says 172.21.15.52:49764) 2024-06-10T03:01:23.580 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.580+0000 7f0f3ad9d700 1 -- 172.21.15.52:0/2278103587 learned_addr learned my addr 172.21.15.52:0/2278103587 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:01:23.583 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.583+0000 7f0f3ad9d700 1 -- 172.21.15.52:0/2278103587 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0f3c109490 msgr2=0x7f0f3c11f1f0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-06-10T03:01:23.583 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.583+0000 7f0f3ad9d700 1 --2- 172.21.15.52:0/2278103587 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0f3c109490 0x7f0f3c11f1f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:23.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.583+0000 7f0f3ad9d700 1 -- 172.21.15.52:0/2278103587 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0f3c109e50 msgr2=0x7f0f3c11f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:01:23.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.583+0000 7f0f3ad9d700 1 --2- 172.21.15.52:0/2278103587 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0f3c109e50 0x7f0f3c11f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:23.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.583+0000 7f0f3ad9d700 1 -- 172.21.15.52:0/2278103587 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f3c127820 con 0x7f0f3c10dee0 2024-06-10T03:01:23.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.584+0000 7f0f3ad9d700 1 --2- 172.21.15.52:0/2278103587 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0f3c10dee0 0x7f0f3c126f20 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f0f34004fd0 tx=0x7f0f34009430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:01:23.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.584+0000 7f0f2b7fe700 1 -- 172.21.15.52:0/2278103587 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f34021070 con 0x7f0f3c10dee0 2024-06-10T03:01:23.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.584+0000 7f0f40abc700 1 -- 172.21.15.52:0/2278103587 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f3c11a310 con 0x7f0f3c10dee0 2024-06-10T03:01:23.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.584+0000 7f0f40abc700 1 -- 172.21.15.52:0/2278103587 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f3c11a850 con 0x7f0f3c10dee0 2024-06-10T03:01:23.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.585+0000 7f0f2b7fe700 1 -- 172.21.15.52:0/2278103587 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f34007c00 con 0x7f0f3c10dee0 2024-06-10T03:01:23.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.585+0000 7f0f2b7fe700 1 -- 172.21.15.52:0/2278103587 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f34005240 con 0x7f0f3c10dee0 2024-06-10T03:01:23.595 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.595+0000 7f0f2b7fe700 1 -- 172.21.15.52:0/2278103587 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f0f34024070 con 0x7f0f3c10dee0 2024-06-10T03:01:23.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.596+0000 7f0f2b7fe700 1 --2- 172.21.15.52:0/2278103587 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f0f240a2d70 0x7f0f240a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:01:23.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.596+0000 7f0f2b7fe700 1 -- 172.21.15.52:0/2278103587 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f0f340d0430 con 0x7f0f3c10dee0 2024-06-10T03:01:23.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.597+0000 7f0f3a59c700 1 --2- 172.21.15.52:0/2278103587 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f0f240a2d70 0x7f0f240a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:01:23.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.599+0000 7f0f4b33fb80 1 Processor -- start 2024-06-10T03:01:23.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.599+0000 7f0f4b33fb80 1 -- start start 2024-06-10T03:01:23.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.600+0000 7f0f4b33fb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d257db2f60 0x55d257db3340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:01:23.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.600+0000 7f0f4b33fb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d257db3880 0x55d257dad260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:01:23.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.600+0000 7f0f4b33fb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d257dad7a0 0x55d257dadbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:01:23.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.600+0000 7f0f4b33fb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55d257c9ff60 con 0x55d257dad7a0 2024-06-10T03:01:23.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.600+0000 7f0f4b33fb80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55d257cfe850 con 0x55d257db2f60 2024-06-10T03:01:23.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.600+0000 7f0f4b33fb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55d257c66460 con 0x55d257db3880 2024-06-10T03:01:23.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.601+0000 7f0f39d9b700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d257db3880 0x55d257dad260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:01:23.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.601+0000 7f0f39d9b700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d257db3880 0x55d257dad260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.52:38410/0 (socket says 172.21.15.52:38410) 2024-06-10T03:01:23.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.601+0000 7f0f39d9b700 1 -- 172.21.15.52:0/2177221288 learned_addr learned my addr 172.21.15.52:0/2177221288 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:01:23.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.601+0000 7f0f3ad9d700 1 --2- 172.21.15.52:0/2177221288 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d257dad7a0 0x55d257dadbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:01:23.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.601+0000 7f0f3ad9d700 1 -- 172.21.15.52:0/2177221288 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d257db3880 msgr2=0x55d257dad260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:01:23.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.601+0000 7f0f3ad9d700 1 --2- 172.21.15.52:0/2177221288 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d257db3880 0x55d257dad260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:23.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.602+0000 7f0f3ad9d700 1 -- 172.21.15.52:0/2177221288 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d257db2f60 msgr2=0x55d257db3340 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-10T03:01:23.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.602+0000 7f0f3ad9d700 1 --2- 172.21.15.52:0/2177221288 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d257db2f60 0x55d257db3340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:23.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.602+0000 7f0f3ad9d700 1 -- 172.21.15.52:0/2177221288 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d257c4d8d0 con 0x55d257dad7a0 2024-06-10T03:01:23.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.602+0000 7f0f3ad9d700 1 --2- 172.21.15.52:0/2177221288 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d257dad7a0 0x55d257dadbc0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f0f34009af0 tx=0x7f0f3409d090 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c494d166fa493bc server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:01:23.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.603+0000 7f0f28ff9700 1 -- 172.21.15.52:0/2177221288 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f3402b070 con 0x55d257dad7a0 2024-06-10T03:01:23.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.603+0000 7f0f28ff9700 1 -- 172.21.15.52:0/2177221288 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f340a1e50 con 0x55d257dad7a0 2024-06-10T03:01:23.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.603+0000 7f0f28ff9700 1 -- 172.21.15.52:0/2177221288 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f340cdd10 con 0x55d257dad7a0 2024-06-10T03:01:23.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.605+0000 7f0f4b33fb80 1 -- 172.21.15.52:0/2177221288 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d257dad7a0 msgr2=0x55d257dadbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:01:23.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.605+0000 7f0f4b33fb80 1 --2- 172.21.15.52:0/2177221288 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d257dad7a0 0x55d257dadbc0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f0f34009af0 tx=0x7f0f3409d090 comp rx=0 tx=0).stop 2024-06-10T03:01:23.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.606+0000 7f0f3a59c700 1 --2- 172.21.15.52:0/2278103587 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f0f240a2d70 0x7f0f240a5230 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f0f30009710 tx=0x7f0f30006c60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:01:23.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.606+0000 7f0f4b33fb80 1 -- 172.21.15.52:0/2177221288 shutdown_connections 2024-06-10T03:01:23.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.606+0000 7f0f4b33fb80 1 --2- 172.21.15.52:0/2177221288 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d257dad7a0 0x55d257dadbc0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:23.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.606+0000 7f0f4b33fb80 1 --2- 172.21.15.52:0/2177221288 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d257db2f60 0x55d257db3340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:23.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.606+0000 7f0f4b33fb80 1 --2- 172.21.15.52:0/2177221288 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d257db3880 0x55d257dad260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:23.607 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.606+0000 7f0f4b33fb80 1 -- 172.21.15.52:0/2177221288 >> 172.21.15.52:0/2177221288 conn(0x55d257cda080 msgr2=0x55d257da8220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:01:23.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.608+0000 7f0f4b33fb80 1 -- 172.21.15.52:0/2177221288 shutdown_connections 2024-06-10T03:01:23.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.609+0000 7f0f4b33fb80 1 -- 172.21.15.52:0/2177221288 wait complete. 2024-06-10T03:01:23.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.609+0000 7f0f4b33fb80 1 -- start start 2024-06-10T03:01:23.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.609+0000 7f0f4b33fb80 1 Processor -- start 2024-06-10T03:01:23.611 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.610+0000 7f0f4b33fb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d257dad7a0 0x55d257db9690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:01:23.611 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.610+0000 7f0f4b33fb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d257db2f60 0x55d257db6a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:01:23.611 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.610+0000 7f0f3a59c700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d257dad7a0 0x55d257db9690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:01:23.611 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.610+0000 7f0f3a59c700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d257dad7a0 0x55d257db9690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.52:38422/0 (socket says 172.21.15.52:38422) 2024-06-10T03:01:23.611 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.610+0000 7f0f39d9b700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d257db2f60 0x55d257db6a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:01:23.611 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.610+0000 7f0f39d9b700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d257db2f60 0x55d257db6a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.52:49772/0 (socket says 172.21.15.52:49772) 2024-06-10T03:01:23.611 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.610+0000 7f0f3a59c700 1 -- 172.21.15.52:0/1020152225 learned_addr learned my addr 172.21.15.52:0/1020152225 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:01:23.611 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.611+0000 7f0f4b33fb80 1 --2- 172.21.15.52:0/1020152225 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d257db6f40 0x55d257db73c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:01:23.611 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.611+0000 7f0f4b33fb80 1 -- 172.21.15.52:0/1020152225 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55d257c9efc0 con 0x55d257db6f40 2024-06-10T03:01:23.611 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.611+0000 7f0f4b33fb80 1 -- 172.21.15.52:0/1020152225 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55d257cfa810 con 0x55d257db2f60 2024-06-10T03:01:23.611 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.611+0000 7f0f4b33fb80 1 -- 172.21.15.52:0/1020152225 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55d257dba010 con 0x55d257dad7a0 2024-06-10T03:01:23.611 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.611+0000 7f0f3ad9d700 1 --2- 172.21.15.52:0/1020152225 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d257db6f40 0x55d257db73c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:01:23.613 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.611+0000 7f0f4b33fb80 10 client.0 _ll_register_callbacks cb 0x55d257c945b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T03:01:23.613 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.612+0000 7f0f3ad9d700 1 -- 172.21.15.52:0/1020152225 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d257dad7a0 msgr2=0x55d257db9690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:01:23.613 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.612+0000 7f0f3ad9d700 1 --2- 172.21.15.52:0/1020152225 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d257dad7a0 0x55d257db9690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:23.613 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.612+0000 7f0f3ad9d700 1 -- 172.21.15.52:0/1020152225 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d257db2f60 msgr2=0x55d257db6a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:01:23.613 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.612+0000 7f0f3ad9d700 1 --2- 172.21.15.52:0/1020152225 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d257db2f60 0x55d257db6a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:23.613 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.612+0000 7f0f3ad9d700 1 -- 172.21.15.52:0/1020152225 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d257cc4750 con 0x55d257db6f40 2024-06-10T03:01:23.613 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.612+0000 7f0f39d9b700 1 --2- 172.21.15.52:0/1020152225 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d257db2f60 0x55d257db6a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T03:01:23.613 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.612+0000 7f0f3ad9d700 1 --2- 172.21.15.52:0/1020152225 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d257db6f40 0x55d257db73c0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f0f340d03a0 tx=0x7f0f340a0d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eab8363d1d04a788 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:01:23.613 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.612+0000 7f0f21ffb700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-10T03:01:23.613 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.612+0000 7f0f21ffb700 1 -- 172.21.15.52:0/1020152225 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f3402b070 con 0x55d257db6f40 2024-06-10T03:01:23.613 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.612+0000 7f0f21ffb700 1 -- 172.21.15.52:0/1020152225 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f0f340cdba0 con 0x55d257db6f40 2024-06-10T03:01:23.613 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.613+0000 7f0f4b33fb80 1 -- 172.21.15.52:0/1020152225 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0f340a1e50 con 0x55d257db6f40 2024-06-10T03:01:23.615 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.613+0000 7f0f21ffb700 1 -- 172.21.15.52:0/1020152225 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f0f340bb050 con 0x55d257db6f40 2024-06-10T03:01:23.615 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.613+0000 7f0f4b33fb80 10 client.16527 fetch_fsmap learned FSMap version 36 2024-06-10T03:01:23.615 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.613+0000 7f0f4b33fb80 1 -- 172.21.15.52:0/1020152225 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55d257dc4aa0 con 0x55d257db6f40 2024-06-10T03:01:23.615 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.614+0000 7f0f21ffb700 1 -- 172.21.15.52:0/1020152225 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f340bb1f0 con 0x55d257db6f40 2024-06-10T03:01:23.615 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.614+0000 7f0f21ffb700 1 -- 172.21.15.52:0/1020152225 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 36) ==== 8378+0+0 (secure 0 0 0) 0x7f0f34011070 con 0x55d257db6f40 2024-06-10T03:01:23.615 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.614+0000 7f0f4b33fb80 10 client.16527 fetch_fsmap finished waiting for FSMap version 36 2024-06-10T03:01:23.615 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.614+0000 7f0f4b33fb80 10 client.16527 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T03:01:23.615 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.614+0000 7f0f4b33fb80 20 client.16527 populate_metadata read hostname 'smithi052' 2024-06-10T03:01:23.615 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.614+0000 7f0f4b33fb80 1 --2- 172.21.15.52:0/1020152225 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55d257dc6f70 0x55d257dc9360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:01:23.615 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.614+0000 7f0f4b33fb80 4 client.16527 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T03:01:23.615 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.614+0000 7f0f4b33fb80 1 -- 172.21.15.52:0/1020152225 --> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f0f340a1e50 con 0x55d257dc6f70 2024-06-10T03:01:23.616 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.616+0000 7f0f3a59c700 1 --2- 172.21.15.52:0/1020152225 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55d257dc6f70 0x55d257dc9360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:01:23.618 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.616+0000 7f0f3a59c700 1 --2- 172.21.15.52:0/1020152225 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55d257dc6f70 0x55d257dc9360 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e1a590e2f308a39 server_cookie=34246cb5362905c0 in_seq=0 out_seq=0 2024-06-10T03:01:23.618 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.616+0000 7f0f21ffb700 10 client.16527 ms_handle_connect on v2:172.21.15.188:6838/310444836 2024-06-10T03:01:23.618 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.618+0000 7f0f21ffb700 1 -- 172.21.15.52:0/1020152225 <== mds.0 v2:172.21.15.188:6838/310444836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x7f0f340a1e50 con 0x55d257dc6f70 2024-06-10T03:01:23.618 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.618+0000 7f0f21ffb700 10 client.16527 handle_command_reply: tid=0 2024-06-10T03:01:23.618 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.618+0000 7f0f4b33fb80 1 client.16527 shutdown 2024-06-10T03:01:23.619 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.618+0000 7f0f4b33fb80 1 -- 172.21.15.52:0/1020152225 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d257db6f40 msgr2=0x55d257db73c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:01:23.619 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.618+0000 7f0f4b33fb80 1 --2- 172.21.15.52:0/1020152225 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d257db6f40 0x55d257db73c0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f0f340d03a0 tx=0x7f0f340a0d50 comp rx=0 tx=0).stop 2024-06-10T03:01:23.619 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.619+0000 7f0f4b33fb80 1 -- 172.21.15.52:0/1020152225 shutdown_connections 2024-06-10T03:01:23.619 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.619+0000 7f0f4b33fb80 1 --2- 172.21.15.52:0/1020152225 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55d257dc6f70 0x55d257dc9360 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:23.619 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.619+0000 7f0f4b33fb80 1 --2- 172.21.15.52:0/1020152225 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d257db6f40 0x55d257db73c0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:23.619 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.619+0000 7f0f4b33fb80 1 --2- 172.21.15.52:0/1020152225 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d257dad7a0 0x55d257db9690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:23.619 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.619+0000 7f0f21ffb700 0 client.16527 ms_handle_reset on v2:172.21.15.188:6838/310444836 2024-06-10T03:01:23.619 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.619+0000 7f0f4b33fb80 1 --2- 172.21.15.52:0/1020152225 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d257db2f60 0x55d257db6a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:23.619 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.619+0000 7f0f4b33fb80 1 -- 172.21.15.52:0/1020152225 >> 172.21.15.52:0/1020152225 conn(0x55d257cda080 msgr2=0x55d257da7700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:01:23.620 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.620+0000 7f0f4b33fb80 1 -- 172.21.15.52:0/1020152225 shutdown_connections 2024-06-10T03:01:23.620 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.620+0000 7f0f4b33fb80 1 -- 172.21.15.52:0/1020152225 wait complete. 2024-06-10T03:01:23.620 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.620+0000 7f0f4b33fb80 20 client.16527 trim_cache size 0 max 16384 2024-06-10T03:01:23.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.663+0000 7f0f4b33fb80 1 Processor -- start 2024-06-10T03:01:23.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.664+0000 7f0f4b33fb80 1 -- start start 2024-06-10T03:01:23.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.664+0000 7f0f4b33fb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d257dad7a0 0x55d257dad260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:01:23.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.664+0000 7f0f4b33fb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d257db2f60 0x55d257db9240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:01:23.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.664+0000 7f0f4b33fb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d257db6f40 0x55d257da7e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:01:23.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.664+0000 7f0f4b33fb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f0f1c00ec20 con 0x55d257dad7a0 2024-06-10T03:01:23.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.664+0000 7f0f4b33fb80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f0f1c010820 con 0x55d257db6f40 2024-06-10T03:01:23.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.664+0000 7f0f4b33fb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f0f1c00f2f0 con 0x55d257db2f60 2024-06-10T03:01:23.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.664+0000 7f0f3a59c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d257dad7a0 0x55d257dad260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:01:23.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.664+0000 7f0f3ad9d700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d257db6f40 0x55d257da7e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:01:23.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.664+0000 7f0f3a59c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d257dad7a0 0x55d257dad260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:35994/0 (socket says 172.21.15.52:35994) 2024-06-10T03:01:23.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.664+0000 7f0f3a59c700 1 -- 172.21.15.52:0/2989918008 learned_addr learned my addr 172.21.15.52:0/2989918008 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:01:23.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.665+0000 7f0f3ad9d700 1 -- 172.21.15.52:0/2989918008 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d257db2f60 msgr2=0x55d257db9240 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-10T03:01:23.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.665+0000 7f0f3ad9d700 1 --2- 172.21.15.52:0/2989918008 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d257db2f60 0x55d257db9240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:23.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.665+0000 7f0f3ad9d700 1 -- 172.21.15.52:0/2989918008 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d257dad7a0 msgr2=0x55d257dad260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:01:23.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.665+0000 7f0f3ad9d700 1 --2- 172.21.15.52:0/2989918008 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d257dad7a0 0x55d257dad260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:23.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.665+0000 7f0f3ad9d700 1 -- 172.21.15.52:0/2989918008 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d257db7a10 con 0x55d257db6f40 2024-06-10T03:01:23.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.665+0000 7f0f3a59c700 1 --2- 172.21.15.52:0/2989918008 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d257dad7a0 0x55d257dad260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T03:01:23.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.665+0000 7f0f3ad9d700 1 --2- 172.21.15.52:0/2989918008 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d257db6f40 0x55d257da7e80 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f0f34098d70 tx=0x7f0f3409ae20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fd12e29e5170730b server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:01:23.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.667+0000 7f0f23fff700 1 -- 172.21.15.52:0/2989918008 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f3402b070 con 0x55d257db6f40 2024-06-10T03:01:23.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.667+0000 7f0f23fff700 1 -- 172.21.15.52:0/2989918008 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f340a1e50 con 0x55d257db6f40 2024-06-10T03:01:23.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.667+0000 7f0f23fff700 1 -- 172.21.15.52:0/2989918008 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f34080080 con 0x55d257db6f40 2024-06-10T03:01:23.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.668+0000 7f0f4b33fb80 1 -- 172.21.15.52:0/2989918008 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d257db6f40 msgr2=0x55d257da7e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:01:23.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.668+0000 7f0f4b33fb80 1 --2- 172.21.15.52:0/2989918008 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d257db6f40 0x55d257da7e80 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f0f34098d70 tx=0x7f0f3409ae20 comp rx=0 tx=0).stop 2024-06-10T03:01:23.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.668+0000 7f0f4b33fb80 1 -- 172.21.15.52:0/2989918008 shutdown_connections 2024-06-10T03:01:23.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.668+0000 7f0f4b33fb80 1 --2- 172.21.15.52:0/2989918008 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d257db6f40 0x55d257da7e80 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:23.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.668+0000 7f0f4b33fb80 1 --2- 172.21.15.52:0/2989918008 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d257dad7a0 0x55d257dad260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:23.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.668+0000 7f0f4b33fb80 1 --2- 172.21.15.52:0/2989918008 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d257db2f60 0x55d257db9240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:23.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.668+0000 7f0f4b33fb80 1 -- 172.21.15.52:0/2989918008 >> 172.21.15.52:0/2989918008 conn(0x55d257cda080 msgr2=0x55d257db6810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:01:23.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.669+0000 7f0f4b33fb80 1 -- 172.21.15.52:0/2989918008 shutdown_connections 2024-06-10T03:01:23.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.669+0000 7f0f4b33fb80 1 -- 172.21.15.52:0/2989918008 wait complete. 2024-06-10T03:01:23.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.670+0000 7f0f4b33fb80 1 -- start start 2024-06-10T03:01:23.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.670+0000 7f0f4b33fb80 1 Processor -- start 2024-06-10T03:01:23.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.671+0000 7f0f4b33fb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d257dad7a0 0x55d257dbf2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:01:23.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.671+0000 7f0f4b33fb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d257db2f60 0x55d257dbf820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:01:23.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.671+0000 7f0f4b33fb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d257db6f40 0x55d257dbc2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:01:23.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.671+0000 7f0f4b33fb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f0f1c010150 con 0x55d257db6f40 2024-06-10T03:01:23.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.671+0000 7f0f4b33fb80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f0f1c00fa80 con 0x55d257dad7a0 2024-06-10T03:01:23.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.671+0000 7f0f4b33fb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f0f1c00cbc0 con 0x55d257db2f60 2024-06-10T03:01:23.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.671+0000 7f0f4b33fb80 10 client.0 _ll_register_callbacks cb 0x55d257c945b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T03:01:23.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.671+0000 7f0f3a59c700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d257dad7a0 0x55d257dbf2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:01:23.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.671+0000 7f0f3ad9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d257db6f40 0x55d257dbc2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:01:23.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.671+0000 7f0f3a59c700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d257dad7a0 0x55d257dbf2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.52:49784/0 (socket says 172.21.15.52:49784) 2024-06-10T03:01:23.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.671+0000 7f0f3a59c700 1 -- 172.21.15.52:0/3611316059 learned_addr learned my addr 172.21.15.52:0/3611316059 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:01:23.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.671+0000 7f0f39d9b700 1 --2- 172.21.15.52:0/3611316059 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d257db2f60 0x55d257dbf820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:01:23.673 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.672+0000 7f0f3a59c700 1 -- 172.21.15.52:0/3611316059 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d257db2f60 msgr2=0x55d257dbf820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:01:23.673 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.672+0000 7f0f3a59c700 1 --2- 172.21.15.52:0/3611316059 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d257db2f60 0x55d257dbf820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:23.673 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.672+0000 7f0f3a59c700 1 -- 172.21.15.52:0/3611316059 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d257db6f40 msgr2=0x55d257dbc2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:01:23.673 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.672+0000 7f0f3a59c700 1 --2- 172.21.15.52:0/3611316059 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d257db6f40 0x55d257dbc2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:23.673 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.672+0000 7f0f3a59c700 1 -- 172.21.15.52:0/3611316059 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d257dcf4c0 con 0x55d257dad7a0 2024-06-10T03:01:23.673 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.672+0000 7f0f3ad9d700 1 --2- 172.21.15.52:0/3611316059 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d257db6f40 0x55d257dbc2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T03:01:23.673 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.672+0000 7f0f3a59c700 1 --2- 172.21.15.52:0/3611316059 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d257dad7a0 0x55d257dbf2e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f0f30015dc0 tx=0x7f0f300047e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=965b2f659f9f099c server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:01:23.673 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.672+0000 7f0f21ffb700 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-06-10T03:01:23.673 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.674+0000 7f0f21ffb700 1 -- 172.21.15.52:0/3611316059 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f30022ce0 con 0x55d257dad7a0 2024-06-10T03:01:23.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.674+0000 7f0f4b33fb80 1 -- 172.21.15.52:0/3611316059 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0f340a1e50 con 0x55d257dad7a0 2024-06-10T03:01:23.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.674+0000 7f0f21ffb700 1 -- 172.21.15.52:0/3611316059 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f0f30005a00 con 0x55d257dad7a0 2024-06-10T03:01:23.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.674+0000 7f0f21ffb700 1 -- 172.21.15.52:0/3611316059 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f0f30037050 con 0x55d257dad7a0 2024-06-10T03:01:23.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.674+0000 7f0f4b33fb80 10 client.26054 fetch_fsmap learned FSMap version 36 2024-06-10T03:01:23.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.674+0000 7f0f4b33fb80 1 -- 172.21.15.52:0/3611316059 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55d257e31ee0 con 0x55d257dad7a0 2024-06-10T03:01:23.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.674+0000 7f0f21ffb700 1 -- 172.21.15.52:0/3611316059 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f340a1e50 con 0x55d257dad7a0 2024-06-10T03:01:23.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.675+0000 7f0f21ffb700 1 -- 172.21.15.52:0/3611316059 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 36) ==== 8378+0+0 (secure 0 0 0) 0x7f0f300066d0 con 0x55d257dad7a0 2024-06-10T03:01:23.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.676+0000 7f0f4b33fb80 10 client.26054 fetch_fsmap finished waiting for FSMap version 36 2024-06-10T03:01:23.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.676+0000 7f0f4b33fb80 10 client.26054 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T03:01:23.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.676+0000 7f0f4b33fb80 20 client.26054 populate_metadata read hostname 'smithi052' 2024-06-10T03:01:23.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.676+0000 7f0f4b33fb80 1 --2- 172.21.15.52:0/3611316059 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55d257dc6f70 0x55d257e34630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:01:23.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.676+0000 7f0f4b33fb80 4 client.26054 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-10T03:01:23.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.676+0000 7f0f4b33fb80 1 -- 172.21.15.52:0/3611316059 --> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d257d0bd50 con 0x55d257dc6f70 2024-06-10T03:01:23.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.676+0000 7f0f39d9b700 1 --2- 172.21.15.52:0/3611316059 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55d257dc6f70 0x55d257e34630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:01:23.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.678+0000 7f0f39d9b700 1 --2- 172.21.15.52:0/3611316059 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55d257dc6f70 0x55d257e34630 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c0b4c87c36dcbe00 server_cookie=b9c668b2baf0a87e in_seq=0 out_seq=0 2024-06-10T03:01:23.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.678+0000 7f0f21ffb700 10 client.26054 ms_handle_connect on v2:172.21.15.188:6838/310444836 2024-06-10T03:01:23.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.678+0000 7f0f21ffb700 1 -- 172.21.15.52:0/3611316059 <== mds.0 v2:172.21.15.188:6838/310444836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55d257d0bd50 con 0x55d257dc6f70 2024-06-10T03:01:23.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.678+0000 7f0f21ffb700 10 client.26054 handle_command_reply: tid=0 2024-06-10T03:01:23.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.678+0000 7f0f4b33fb80 1 client.26054 shutdown 2024-06-10T03:01:23.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.679+0000 7f0f4b33fb80 1 -- 172.21.15.52:0/3611316059 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d257dad7a0 msgr2=0x55d257dbf2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:01:23.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.679+0000 7f0f4b33fb80 1 --2- 172.21.15.52:0/3611316059 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d257dad7a0 0x55d257dbf2e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f0f30015dc0 tx=0x7f0f300047e0 comp rx=0 tx=0).stop 2024-06-10T03:01:23.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.681+0000 7f0f4b33fb80 1 -- 172.21.15.52:0/3611316059 shutdown_connections 2024-06-10T03:01:23.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.681+0000 7f0f4b33fb80 1 --2- 172.21.15.52:0/3611316059 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55d257dc6f70 0x55d257e34630 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:23.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.681+0000 7f0f4b33fb80 1 --2- 172.21.15.52:0/3611316059 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d257db6f40 0x55d257dbc2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:23.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.681+0000 7f0f4b33fb80 1 --2- 172.21.15.52:0/3611316059 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d257dad7a0 0x55d257dbf2e0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:23.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.681+0000 7f0f4b33fb80 1 --2- 172.21.15.52:0/3611316059 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d257db2f60 0x55d257dbf820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:23.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.681+0000 7f0f4b33fb80 1 -- 172.21.15.52:0/3611316059 >> 172.21.15.52:0/3611316059 conn(0x55d257cda080 msgr2=0x55d257db1ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:01:23.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.681+0000 7f0f21ffb700 0 client.26054 ms_handle_reset on v2:172.21.15.188:6838/310444836 2024-06-10T03:01:23.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.681+0000 7f0f4b33fb80 1 -- 172.21.15.52:0/3611316059 shutdown_connections 2024-06-10T03:01:23.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.681+0000 7f0f4b33fb80 1 -- 172.21.15.52:0/3611316059 wait complete. 2024-06-10T03:01:23.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.681+0000 7f0f4b33fb80 20 client.26054 trim_cache size 0 max 16384 2024-06-10T03:01:23.682 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-10T03:01:23.682 INFO:teuthology.orchestra.run.smithi052.stdout: "status": "no active scrubs running", 2024-06-10T03:01:23.682 INFO:teuthology.orchestra.run.smithi052.stdout: "scrubs": {} 2024-06-10T03:01:23.683 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-10T03:01:23.683 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.682+0000 7f0f40abc700 1 -- 172.21.15.52:0/2278103587 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f0f240a2d70 msgr2=0x7f0f240a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:01:23.683 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.682+0000 7f0f40abc700 1 --2- 172.21.15.52:0/2278103587 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f0f240a2d70 0x7f0f240a5230 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f0f30009710 tx=0x7f0f30006c60 comp rx=0 tx=0).stop 2024-06-10T03:01:23.684 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.682+0000 7f0f40abc700 1 -- 172.21.15.52:0/2278103587 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0f3c10dee0 msgr2=0x7f0f3c126f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:01:23.684 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.682+0000 7f0f40abc700 1 --2- 172.21.15.52:0/2278103587 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0f3c10dee0 0x7f0f3c126f20 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f0f34004fd0 tx=0x7f0f34009430 comp rx=0 tx=0).stop 2024-06-10T03:01:23.684 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.683+0000 7f0f40abc700 1 -- 172.21.15.52:0/2278103587 shutdown_connections 2024-06-10T03:01:23.684 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.683+0000 7f0f40abc700 1 --2- 172.21.15.52:0/2278103587 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0f3c10dee0 0x7f0f3c126f20 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:23.684 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.683+0000 7f0f40abc700 1 --2- 172.21.15.52:0/2278103587 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0f3c109490 0x7f0f3c11f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:23.684 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.683+0000 7f0f40abc700 1 --2- 172.21.15.52:0/2278103587 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f0f240a2d70 0x7f0f240a5230 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:23.684 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.683+0000 7f0f40abc700 1 --2- 172.21.15.52:0/2278103587 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0f3c109e50 0x7f0f3c11f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:23.684 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.683+0000 7f0f40abc700 1 -- 172.21.15.52:0/2278103587 >> 172.21.15.52:0/2278103587 conn(0x7f0f3c06d9e0 msgr2=0x7f0f3c071f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:01:23.684 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.683+0000 7f0f40abc700 1 -- 172.21.15.52:0/2278103587 shutdown_connections 2024-06-10T03:01:23.684 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:23.683+0000 7f0f40abc700 1 -- 172.21.15.52:0/2278103587 wait complete. 2024-06-10T03:01:24.638 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:01:24 smithi120 ceph-mon[36634]: pgmap v650: 129 pgs: 129 active+clean; 599 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.3 MiB/s wr, 479 op/s 2024-06-10T03:01:24.703 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:01:24 smithi188 ceph-mon[36599]: pgmap v650: 129 pgs: 129 active+clean; 599 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.3 MiB/s wr, 479 op/s 2024-06-10T03:01:24.727 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:01:24 smithi052 ceph-mon[32362]: pgmap v650: 129 pgs: 129 active+clean; 599 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.3 MiB/s wr, 479 op/s 2024-06-10T03:01:25.364 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-06-10T03:01:25.364 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T03:01:26.703 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:01:26 smithi188 ceph-mon[36599]: pgmap v651: 129 pgs: 129 active+clean; 610 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.9 MiB/s wr, 506 op/s 2024-06-10T03:01:26.727 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:01:26 smithi052 ceph-mon[32362]: pgmap v651: 129 pgs: 129 active+clean; 610 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.9 MiB/s wr, 506 op/s 2024-06-10T03:01:26.888 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:01:26 smithi120 ceph-mon[36634]: pgmap v651: 129 pgs: 129 active+clean; 610 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.9 MiB/s wr, 506 op/s 2024-06-10T03:01:27.251 INFO:tasks.workunit.client.0.smithi052.stdout: 115 3369476 244 1011062 406 739764 865 2024-06-10T03:01:28.703 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:01:28 smithi188 ceph-mon[36599]: pgmap v652: 129 pgs: 129 active+clean; 625 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 6.7 MiB/s wr, 703 op/s 2024-06-10T03:01:28.727 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:01:28 smithi052 ceph-mon[32362]: pgmap v652: 129 pgs: 129 active+clean; 625 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 6.7 MiB/s wr, 703 op/s 2024-06-10T03:01:28.888 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:01:28 smithi120 ceph-mon[36634]: pgmap v652: 129 pgs: 129 active+clean; 625 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 6.7 MiB/s wr, 703 op/s 2024-06-10T03:01:30.593 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T03:01:30.703 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:01:30 smithi188 ceph-mon[36599]: pgmap v653: 129 pgs: 129 active+clean; 618 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 6.7 MiB/s wr, 766 op/s 2024-06-10T03:01:30.727 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:01:30 smithi052 ceph-mon[32362]: pgmap v653: 129 pgs: 129 active+clean; 618 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 6.7 MiB/s wr, 766 op/s 2024-06-10T03:01:30.888 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:01:30 smithi120 ceph-mon[36634]: pgmap v653: 129 pgs: 129 active+clean; 618 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 6.7 MiB/s wr, 766 op/s 2024-06-10T03:01:32.409 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:01:32 smithi052 ceph-mon[32362]: pgmap v654: 129 pgs: 129 active+clean; 617 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 6.7 MiB/s wr, 699 op/s 2024-06-10T03:01:32.703 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:01:32 smithi188 ceph-mon[36599]: pgmap v654: 129 pgs: 129 active+clean; 617 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 6.7 MiB/s wr, 699 op/s 2024-06-10T03:01:32.888 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:01:32 smithi120 ceph-mon[36634]: pgmap v654: 129 pgs: 129 active+clean; 617 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 6.7 MiB/s wr, 699 op/s 2024-06-10T03:01:33.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:33.217+0000 7f9331374700 1 -- 172.21.15.52:0/2153714541 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f932c10de50 msgr2=0x7f932c1159e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:01:33.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:33.217+0000 7f9331374700 1 --2- 172.21.15.52:0/2153714541 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f932c10de50 0x7f932c1159e0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f9324009fc0 tx=0x7f932400b330 comp rx=0 tx=0).stop 2024-06-10T03:01:33.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:33.218+0000 7f9331374700 1 -- 172.21.15.52:0/2153714541 shutdown_connections 2024-06-10T03:01:33.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:33.218+0000 7f9331374700 1 --2- 172.21.15.52:0/2153714541 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f932c10de50 0x7f932c1159e0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:33.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:33.218+0000 7f9331374700 1 --2- 172.21.15.52:0/2153714541 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f932c109530 0x7f932c109910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:33.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:33.218+0000 7f9331374700 1 --2- 172.21.15.52:0/2153714541 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f932c109ee0 0x7f932c10d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:33.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:33.218+0000 7f9331374700 1 -- 172.21.15.52:0/2153714541 >> 172.21.15.52:0/2153714541 conn(0x7f932c078360 msgr2=0x7f932c07a780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:01:33.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:33.219+0000 7f9331374700 1 -- 172.21.15.52:0/2153714541 shutdown_connections 2024-06-10T03:01:33.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:33.219+0000 7f9331374700 1 -- 172.21.15.52:0/2153714541 wait complete. 2024-06-10T03:01:33.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:33.220+0000 7f9331374700 1 Processor -- start 2024-06-10T03:01:33.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:33.221+0000 7f9331374700 1 -- start start 2024-06-10T03:01:33.222 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:33.221+0000 7f9331374700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f932c109530 0x7f932c19ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:01:33.222 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:33.221+0000 7f9331374700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f932c109ee0 0x7f932c19f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:01:33.222 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:33.222+0000 7f9331374700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f932c1a35e0 0x7f932c1a3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:01:33.222 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:33.222+0000 7f9331374700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f932c118ea0 con 0x7f932c109530 2024-06-10T03:01:33.222 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:33.222+0000 7f9331374700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f932c118d20 con 0x7f932c109ee0 2024-06-10T03:01:33.222 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:33.222+0000 7f9331374700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f932c119020 con 0x7f932c1a35e0 2024-06-10T03:01:33.222 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:33.222+0000 7f93227fc700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f932c109ee0 0x7f932c19f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:01:33.223 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:33.222+0000 7f932b7fe700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f932c1a35e0 0x7f932c1a3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:01:33.223 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:33.222+0000 7f932b7fe700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f932c1a35e0 0x7f932c1a3a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.52:55134/0 (socket says 172.21.15.52:55134) 2024-06-10T03:01:33.223 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:33.222+0000 7f932b7fe700 1 -- 172.21.15.52:0/2485116300 learned_addr learned my addr 172.21.15.52:0/2485116300 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:01:33.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:33.225+0000 7f932b7fe700 1 -- 172.21.15.52:0/2485116300 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f932c109ee0 msgr2=0x7f932c19f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:01:33.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:33.225+0000 7f932b7fe700 1 --2- 172.21.15.52:0/2485116300 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f932c109ee0 0x7f932c19f250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:33.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:33.225+0000 7f932b7fe700 1 -- 172.21.15.52:0/2485116300 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f932c109530 msgr2=0x7f932c19ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:01:33.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:33.225+0000 7f932b7fe700 1 --2- 172.21.15.52:0/2485116300 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f932c109530 0x7f932c19ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:33.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:33.225+0000 7f932b7fe700 1 -- 172.21.15.52:0/2485116300 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f932c1a4190 con 0x7f932c1a35e0 2024-06-10T03:01:33.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:33.231+0000 7f932b7fe700 1 --2- 172.21.15.52:0/2485116300 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f932c1a35e0 0x7f932c1a3a60 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f9324006050 tx=0x7f932400bcd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:01:33.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:33.234+0000 7f9328ff9700 1 -- 172.21.15.52:0/2485116300 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9324012cc0 con 0x7f932c1a35e0 2024-06-10T03:01:33.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:33.234+0000 7f9331374700 1 -- 172.21.15.52:0/2485116300 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f932c1a4420 con 0x7f932c1a35e0 2024-06-10T03:01:33.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:33.234+0000 7f9331374700 1 -- 172.21.15.52:0/2485116300 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f932c1abb10 con 0x7f932c1a35e0 2024-06-10T03:01:33.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:33.236+0000 7f9328ff9700 1 -- 172.21.15.52:0/2485116300 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9324022070 con 0x7f932c1a35e0 2024-06-10T03:01:33.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:33.236+0000 7f9328ff9700 1 -- 172.21.15.52:0/2485116300 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f932401d760 con 0x7f932c1a35e0 2024-06-10T03:01:33.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:33.237+0000 7f9331374700 1 -- 172.21.15.52:0/2485116300 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f932c069180 con 0x7f932c1a35e0 2024-06-10T03:01:33.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:33.245+0000 7f9328ff9700 1 -- 172.21.15.52:0/2485116300 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f932401d900 con 0x7f932c1a35e0 2024-06-10T03:01:33.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:33.246+0000 7f9328ff9700 1 --2- 172.21.15.52:0/2485116300 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f93180a2db0 0x7f93180a5270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:01:33.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:33.247+0000 7f932affd700 1 --2- 172.21.15.52:0/2485116300 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f93180a2db0 0x7f93180a5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:01:33.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:33.247+0000 7f932affd700 1 --2- 172.21.15.52:0/2485116300 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f93180a2db0 0x7f93180a5270 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f9314005950 tx=0x7f93140058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:01:33.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:33.247+0000 7f9328ff9700 1 -- 172.21.15.52:0/2485116300 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f93240d0570 con 0x7f932c1a35e0 2024-06-10T03:01:33.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:33.248+0000 7f9328ff9700 1 -- 172.21.15.52:0/2485116300 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f93240d0a90 con 0x7f932c1a35e0 2024-06-10T03:01:33.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:33.798+0000 7f9331374700 1 -- 172.21.15.52:0/2485116300 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f932c1abe30 con 0x7f932c1a35e0 2024-06-10T03:01:33.809 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-10T03:01:33.809 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":36,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/889673247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":889673247},{"type":"v1","addr":"172.21.15.52:6835","nonce":889673247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/3615663511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":3615663511},{"type":"v1","addr":"172.21.15.52:6837","nonce":3615663511}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":36,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T02:50:03.889766+0000","modified":"2024-06-10T02:59:27.720611+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24484,"mds_1":24469,"mds_2":24463,"mds_3":24449,"mds_4":24428},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14571":{"gid":14571,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6839/3059599042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":3059599042},{"type":"v1","addr":"172.21.15.52:6839","nonce":3059599042}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14583":{"gid":14583,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6841/3039275521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":3039275521},{"type":"v1","addr":"172.21.15.52:6841","nonce":3039275521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24383":{"gid":24383,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6833/2609002896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":2609002896},{"type":"v1","addr":"172.21.15.120:6833","nonce":2609002896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6835/1996370288","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":1996370288},{"type":"v1","addr":"172.21.15.120:6835","nonce":1996370288}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6833/529564979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":529564979},{"type":"v1","addr":"172.21.15.188:6833","nonce":529564979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":25,"addr":"172.21.15.120:6837/4132440952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":4132440952},{"type":"v1","addr":"172.21.15.120:6837","nonce":4132440952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"b","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.120:6839/1672436487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1672436487},{"type":"v1","addr":"172.21.15.120:6839","nonce":1672436487}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":2,"incarnation":22,"state":"up:active","state_seq":26,"addr":"172.21.15.188:6835/3904544259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":3904544259},{"type":"v1","addr":"172.21.15.188:6835","nonce":3904544259}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.188:6837/819317726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":819317726},{"type":"v1","addr":"172.21.15.188:6837","nonce":819317726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.188:6839/310444836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":310444836},{"type":"v1","addr":"172.21.15.188:6839","nonce":310444836}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24484,"qdb_cluster":[24469,24484,24463,24428,24449]},"id":1}]} 2024-06-10T03:01:33.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:33.802+0000 7f9328ff9700 1 -- 172.21.15.52:0/2485116300 <== mon.1 v2:172.21.15.188:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+11945 (secure 0 0 0) 0x7f932409d230 con 0x7f932c1a35e0 2024-06-10T03:01:33.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:33.809+0000 7f9331374700 1 -- 172.21.15.52:0/2485116300 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f93180a2db0 msgr2=0x7f93180a5270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:01:33.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:33.809+0000 7f9331374700 1 --2- 172.21.15.52:0/2485116300 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f93180a2db0 0x7f93180a5270 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f9314005950 tx=0x7f93140058e0 comp rx=0 tx=0).stop 2024-06-10T03:01:33.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:33.809+0000 7f9331374700 1 -- 172.21.15.52:0/2485116300 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f932c1a35e0 msgr2=0x7f932c1a3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:01:33.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:33.809+0000 7f9331374700 1 --2- 172.21.15.52:0/2485116300 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f932c1a35e0 0x7f932c1a3a60 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f9324006050 tx=0x7f932400bcd0 comp rx=0 tx=0).stop 2024-06-10T03:01:33.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:33.809+0000 7f9331374700 1 -- 172.21.15.52:0/2485116300 shutdown_connections 2024-06-10T03:01:33.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:33.809+0000 7f9331374700 1 --2- 172.21.15.52:0/2485116300 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f932c1a35e0 0x7f932c1a3a60 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:33.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:33.809+0000 7f9331374700 1 --2- 172.21.15.52:0/2485116300 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f93180a2db0 0x7f93180a5270 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:33.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:33.809+0000 7f9331374700 1 --2- 172.21.15.52:0/2485116300 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f932c109530 0x7f932c19ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:33.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:33.809+0000 7f9331374700 1 --2- 172.21.15.52:0/2485116300 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f932c109ee0 0x7f932c19f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:33.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:33.809+0000 7f9331374700 1 -- 172.21.15.52:0/2485116300 >> 172.21.15.52:0/2485116300 conn(0x7f932c078360 msgr2=0x7f932c10b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:01:33.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:33.809+0000 7f9331374700 1 -- 172.21.15.52:0/2485116300 shutdown_connections 2024-06-10T03:01:33.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:33.809+0000 7f9331374700 1 -- 172.21.15.52:0/2485116300 wait complete. 2024-06-10T03:01:33.811 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 36 2024-06-10T03:01:34.664 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:01:34 smithi052 ceph-mon[32362]: pgmap v655: 129 pgs: 129 active+clean; 618 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 6.9 MiB/s wr, 737 op/s 2024-06-10T03:01:34.665 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:01:34 smithi052 ceph-mon[32362]: from='client.? 172.21.15.52:0/2485116300' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T03:01:34.703 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:01:34 smithi188 ceph-mon[36599]: pgmap v655: 129 pgs: 129 active+clean; 618 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 6.9 MiB/s wr, 737 op/s 2024-06-10T03:01:34.703 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:01:34 smithi188 ceph-mon[36599]: from='client.? 172.21.15.52:0/2485116300' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T03:01:34.888 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:01:34 smithi120 ceph-mon[36634]: pgmap v655: 129 pgs: 129 active+clean; 618 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 6.9 MiB/s wr, 737 op/s 2024-06-10T03:01:34.888 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:01:34 smithi120 ceph-mon[36634]: from='client.? 172.21.15.52:0/2485116300' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T03:01:35.132 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-06-10T03:01:36.696 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T03:01:36.759 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:01:36 smithi188 ceph-mon[36599]: pgmap v656: 129 pgs: 129 active+clean; 619 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.1 MiB/s wr, 607 op/s 2024-06-10T03:01:36.759 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:01:36 smithi052 ceph-mon[32362]: pgmap v656: 129 pgs: 129 active+clean; 619 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.1 MiB/s wr, 607 op/s 2024-06-10T03:01:36.888 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:01:36 smithi120 ceph-mon[36634]: pgmap v656: 129 pgs: 129 active+clean; 619 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.1 MiB/s wr, 607 op/s 2024-06-10T03:01:37.252 INFO:tasks.workunit.client.0.smithi052.stdout: 123 1866220 470 1360950 599 1013945 1423 2024-06-10T03:01:38.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.494+0000 7fc33f2a6700 1 -- 172.21.15.52:0/3060285189 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc338102ef0 msgr2=0x7fc33810f3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:01:38.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.494+0000 7fc33f2a6700 1 --2- 172.21.15.52:0/3060285189 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc338102ef0 0x7fc33810f3a0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fc33000b0d0 tx=0x7fc33000b3e0 comp rx=0 tx=0).stop 2024-06-10T03:01:38.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.494+0000 7fc33f2a6700 1 -- 172.21.15.52:0/3060285189 shutdown_connections 2024-06-10T03:01:38.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.494+0000 7fc33f2a6700 1 --2- 172.21.15.52:0/3060285189 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc338102ef0 0x7fc33810f3a0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:38.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.494+0000 7fc33f2a6700 1 --2- 172.21.15.52:0/3060285189 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc3381085d0 0x7fc3381089b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:38.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.494+0000 7fc33f2a6700 1 --2- 172.21.15.52:0/3060285189 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc338102530 0x7fc3381029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:38.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.494+0000 7fc33f2a6700 1 -- 172.21.15.52:0/3060285189 >> 172.21.15.52:0/3060285189 conn(0x7fc338079640 msgr2=0x7fc338079a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:01:38.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.494+0000 7fc33f2a6700 1 -- 172.21.15.52:0/3060285189 shutdown_connections 2024-06-10T03:01:38.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.494+0000 7fc33f2a6700 1 -- 172.21.15.52:0/3060285189 wait complete. 2024-06-10T03:01:38.495 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:01:38 smithi052 ceph-mon[32362]: pgmap v657: 129 pgs: 129 active+clean; 626 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.8 MiB/s wr, 612 op/s 2024-06-10T03:01:38.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.495+0000 7fc33f2a6700 1 Processor -- start 2024-06-10T03:01:38.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.495+0000 7fc33f2a6700 1 -- start start 2024-06-10T03:01:38.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.495+0000 7fc33f2a6700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc338102530 0x7fc338074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:01:38.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.495+0000 7fc33f2a6700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc338102ef0 0x7fc338074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:01:38.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.495+0000 7fc33f2a6700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc3381085d0 0x7fc338075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:01:38.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.495+0000 7fc33f2a6700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fc338112810 con 0x7fc338102530 2024-06-10T03:01:38.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.495+0000 7fc33f2a6700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc338112690 con 0x7fc3381085d0 2024-06-10T03:01:38.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.495+0000 7fc33f2a6700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc338112990 con 0x7fc338102ef0 2024-06-10T03:01:38.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.495+0000 7fc33d042700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc338102530 0x7fc338074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:01:38.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.495+0000 7fc33d042700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc338102530 0x7fc338074790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47360/0 (socket says 172.21.15.52:47360) 2024-06-10T03:01:38.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.496+0000 7fc33d042700 1 -- 172.21.15.52:0/4243894105 learned_addr learned my addr 172.21.15.52:0/4243894105 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:01:38.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.496+0000 7fc33d843700 1 --2- 172.21.15.52:0/4243894105 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc3381085d0 0x7fc338075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:01:38.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.496+0000 7fc33c841700 1 --2- 172.21.15.52:0/4243894105 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc338102ef0 0x7fc338074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:01:38.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.496+0000 7fc33d843700 1 -- 172.21.15.52:0/4243894105 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc338102ef0 msgr2=0x7fc338074cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:01:38.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.496+0000 7fc33d843700 1 --2- 172.21.15.52:0/4243894105 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc338102ef0 0x7fc338074cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:38.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.496+0000 7fc33d843700 1 -- 172.21.15.52:0/4243894105 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc338102530 msgr2=0x7fc338074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:01:38.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.496+0000 7fc33d843700 1 --2- 172.21.15.52:0/4243894105 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc338102530 0x7fc338074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:38.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.496+0000 7fc33d843700 1 -- 172.21.15.52:0/4243894105 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3381a7140 con 0x7fc3381085d0 2024-06-10T03:01:38.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.496+0000 7fc33d843700 1 --2- 172.21.15.52:0/4243894105 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc3381085d0 0x7fc338075210 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fc33000b770 tx=0x7fc330007a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:01:38.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.496+0000 7fc32e7fc700 1 -- 172.21.15.52:0/4243894105 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc330021020 con 0x7fc3381085d0 2024-06-10T03:01:38.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.496+0000 7fc32e7fc700 1 -- 172.21.15.52:0/4243894105 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc330004c50 con 0x7fc3381085d0 2024-06-10T03:01:38.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.496+0000 7fc32e7fc700 1 -- 172.21.15.52:0/4243894105 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3300052b0 con 0x7fc3381085d0 2024-06-10T03:01:38.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.496+0000 7fc33f2a6700 1 -- 172.21.15.52:0/4243894105 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3381a73d0 con 0x7fc3381085d0 2024-06-10T03:01:38.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.497+0000 7fc33f2a6700 1 -- 172.21.15.52:0/4243894105 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3381a7830 con 0x7fc3381085d0 2024-06-10T03:01:38.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.499+0000 7fc32e7fc700 1 -- 172.21.15.52:0/4243894105 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fc330024020 con 0x7fc3381085d0 2024-06-10T03:01:38.500 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.500+0000 7fc32e7fc700 1 --2- 172.21.15.52:0/4243894105 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fc3240a2c20 0x7fc3240a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:01:38.500 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.500+0000 7fc33d042700 1 --2- 172.21.15.52:0/4243894105 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fc3240a2c20 0x7fc3240a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:01:38.500 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.500+0000 7fc32e7fc700 1 -- 172.21.15.52:0/4243894105 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7fc33000f020 con 0x7fc3381085d0 2024-06-10T03:01:38.500 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.500+0000 7fc33d042700 1 --2- 172.21.15.52:0/4243894105 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fc3240a2c20 0x7fc3240a50e0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fc334009ca0 tx=0x7fc334009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:01:38.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.501+0000 7fc349b29b80 1 Processor -- start 2024-06-10T03:01:38.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.501+0000 7fc349b29b80 1 -- start start 2024-06-10T03:01:38.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.501+0000 7fc349b29b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d4bb27af60 0x55d4bb27b340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:01:38.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.501+0000 7fc349b29b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d4bb27b880 0x55d4bb275260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:01:38.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.501+0000 7fc349b29b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d4bb2757a0 0x55d4bb275bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:01:38.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.501+0000 7fc349b29b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55d4bb167f60 con 0x55d4bb27b880 2024-06-10T03:01:38.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.501+0000 7fc349b29b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55d4bb1c6850 con 0x55d4bb27af60 2024-06-10T03:01:38.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.501+0000 7fc349b29b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55d4bb12e460 con 0x55d4bb2757a0 2024-06-10T03:01:38.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.501+0000 7fc33d042700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d4bb27af60 0x55d4bb27b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:01:38.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.501+0000 7fc33d042700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d4bb27af60 0x55d4bb27b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.52:46594/0 (socket says 172.21.15.52:46594) 2024-06-10T03:01:38.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.501+0000 7fc33d042700 1 -- 172.21.15.52:0/2314562429 learned_addr learned my addr 172.21.15.52:0/2314562429 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:01:38.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.501+0000 7fc33c841700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d4bb27b880 0x55d4bb275260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:01:38.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.501+0000 7fc33d843700 1 --2- 172.21.15.52:0/2314562429 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d4bb2757a0 0x55d4bb275bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:01:38.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.502+0000 7fc33d042700 1 -- 172.21.15.52:0/2314562429 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d4bb2757a0 msgr2=0x55d4bb275bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:01:38.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.502+0000 7fc33d042700 1 --2- 172.21.15.52:0/2314562429 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d4bb2757a0 0x55d4bb275bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:38.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.502+0000 7fc33d042700 1 -- 172.21.15.52:0/2314562429 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d4bb27b880 msgr2=0x55d4bb275260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:01:38.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.502+0000 7fc33d042700 1 --2- 172.21.15.52:0/2314562429 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d4bb27b880 0x55d4bb275260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:38.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.502+0000 7fc33d042700 1 -- 172.21.15.52:0/2314562429 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d4bb1158d0 con 0x55d4bb27af60 2024-06-10T03:01:38.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.502+0000 7fc33d042700 1 --2- 172.21.15.52:0/2314562429 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d4bb27af60 0x55d4bb27b340 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fc33400e4b0 tx=0x7fc334014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6ac7fe27b4df63b1 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:01:38.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.502+0000 7fc3237fe700 1 -- 172.21.15.52:0/2314562429 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc33400ec50 con 0x55d4bb27af60 2024-06-10T03:01:38.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.502+0000 7fc3237fe700 1 -- 172.21.15.52:0/2314562429 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc33400edf0 con 0x55d4bb27af60 2024-06-10T03:01:38.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.502+0000 7fc3237fe700 1 -- 172.21.15.52:0/2314562429 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc33401c960 con 0x55d4bb27af60 2024-06-10T03:01:38.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.503+0000 7fc349b29b80 1 -- 172.21.15.52:0/2314562429 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d4bb27af60 msgr2=0x55d4bb27b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:01:38.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.503+0000 7fc349b29b80 1 --2- 172.21.15.52:0/2314562429 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d4bb27af60 0x55d4bb27b340 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fc33400e4b0 tx=0x7fc334014000 comp rx=0 tx=0).stop 2024-06-10T03:01:38.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.503+0000 7fc349b29b80 1 -- 172.21.15.52:0/2314562429 shutdown_connections 2024-06-10T03:01:38.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.503+0000 7fc349b29b80 1 --2- 172.21.15.52:0/2314562429 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d4bb2757a0 0x55d4bb275bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:38.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.503+0000 7fc349b29b80 1 --2- 172.21.15.52:0/2314562429 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d4bb27af60 0x55d4bb27b340 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:38.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.503+0000 7fc349b29b80 1 --2- 172.21.15.52:0/2314562429 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d4bb27b880 0x55d4bb275260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:38.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.503+0000 7fc349b29b80 1 -- 172.21.15.52:0/2314562429 >> 172.21.15.52:0/2314562429 conn(0x55d4bb1a2080 msgr2=0x55d4bb270220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:01:38.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.503+0000 7fc349b29b80 1 -- 172.21.15.52:0/2314562429 shutdown_connections 2024-06-10T03:01:38.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.503+0000 7fc349b29b80 1 -- 172.21.15.52:0/2314562429 wait complete. 2024-06-10T03:01:38.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.503+0000 7fc349b29b80 1 -- start start 2024-06-10T03:01:38.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.503+0000 7fc349b29b80 1 Processor -- start 2024-06-10T03:01:38.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.504+0000 7fc349b29b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d4bb2757a0 0x55d4bb281680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:01:38.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.504+0000 7fc349b29b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d4bb27af60 0x55d4bb27e980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:01:38.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.504+0000 7fc349b29b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d4bb27b880 0x55d4bb27eec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:01:38.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.504+0000 7fc349b29b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55d4bb166fc0 con 0x55d4bb2757a0 2024-06-10T03:01:38.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.504+0000 7fc349b29b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55d4bb1c2810 con 0x55d4bb27b880 2024-06-10T03:01:38.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.504+0000 7fc349b29b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55d4bb282260 con 0x55d4bb27af60 2024-06-10T03:01:38.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.504+0000 7fc349b29b80 10 client.0 _ll_register_callbacks cb 0x55d4bb15c5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T03:01:38.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.504+0000 7fc33d042700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d4bb2757a0 0x55d4bb281680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:01:38.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.504+0000 7fc33d042700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d4bb2757a0 0x55d4bb281680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47372/0 (socket says 172.21.15.52:47372) 2024-06-10T03:01:38.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.504+0000 7fc33d843700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d4bb27b880 0x55d4bb27eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:01:38.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.504+0000 7fc33d042700 1 -- 172.21.15.52:0/3469960921 learned_addr learned my addr 172.21.15.52:0/3469960921 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:01:38.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.504+0000 7fc33c841700 1 --2- 172.21.15.52:0/3469960921 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d4bb27af60 0x55d4bb27e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:01:38.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.504+0000 7fc33d042700 1 -- 172.21.15.52:0/3469960921 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d4bb27af60 msgr2=0x55d4bb27e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:01:38.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.504+0000 7fc33d042700 1 --2- 172.21.15.52:0/3469960921 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d4bb27af60 0x55d4bb27e980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:38.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.504+0000 7fc33d042700 1 -- 172.21.15.52:0/3469960921 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d4bb27b880 msgr2=0x55d4bb27eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:01:38.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.504+0000 7fc33d042700 1 --2- 172.21.15.52:0/3469960921 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d4bb27b880 0x55d4bb27eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:38.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.504+0000 7fc33d042700 1 -- 172.21.15.52:0/3469960921 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d4bb18c750 con 0x55d4bb2757a0 2024-06-10T03:01:38.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.504+0000 7fc33c841700 1 --2- 172.21.15.52:0/3469960921 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d4bb27af60 0x55d4bb27e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T03:01:38.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.504+0000 7fc33d843700 1 --2- 172.21.15.52:0/3469960921 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d4bb27b880 0x55d4bb27eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T03:01:38.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.505+0000 7fc33d042700 1 --2- 172.21.15.52:0/3469960921 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d4bb2757a0 0x55d4bb281680 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fc33401c560 tx=0x7fc334009a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ad6dcbe751310d0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:01:38.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.505+0000 7fc320ff9700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-10T03:01:38.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.505+0000 7fc320ff9700 1 -- 172.21.15.52:0/3469960921 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc334005450 con 0x55d4bb2757a0 2024-06-10T03:01:38.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.505+0000 7fc320ff9700 1 -- 172.21.15.52:0/3469960921 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7fc3340055f0 con 0x55d4bb2757a0 2024-06-10T03:01:38.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.505+0000 7fc349b29b80 1 -- 172.21.15.52:0/3469960921 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc33400edf0 con 0x55d4bb2757a0 2024-06-10T03:01:38.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.505+0000 7fc320ff9700 1 -- 172.21.15.52:0/3469960921 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fc334038050 con 0x55d4bb2757a0 2024-06-10T03:01:38.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.505+0000 7fc349b29b80 10 client.16548 fetch_fsmap learned FSMap version 36 2024-06-10T03:01:38.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.505+0000 7fc349b29b80 1 -- 172.21.15.52:0/3469960921 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55d4bb28ca90 con 0x55d4bb2757a0 2024-06-10T03:01:38.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.505+0000 7fc320ff9700 1 -- 172.21.15.52:0/3469960921 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc33400edf0 con 0x55d4bb2757a0 2024-06-10T03:01:38.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.505+0000 7fc320ff9700 1 -- 172.21.15.52:0/3469960921 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 36) ==== 8378+0+0 (secure 0 0 0) 0x7fc3340269e0 con 0x55d4bb2757a0 2024-06-10T03:01:38.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.505+0000 7fc349b29b80 10 client.16548 fetch_fsmap finished waiting for FSMap version 36 2024-06-10T03:01:38.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.505+0000 7fc349b29b80 10 client.16548 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-06-10T03:01:38.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.505+0000 7fc349b29b80 20 client.16548 populate_metadata read hostname 'smithi052' 2024-06-10T03:01:38.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.505+0000 7fc349b29b80 1 --2- 172.21.15.52:0/3469960921 >> [v2:172.21.15.120:6836/4132440952,v1:172.21.15.120:6837/4132440952] conn(0x55d4bb28ef60 0x55d4bb291350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:01:38.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.505+0000 7fc349b29b80 4 client.16548 mds_command: new command op to 24428 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T03:01:38.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.505+0000 7fc349b29b80 1 -- 172.21.15.52:0/3469960921 --> [v2:172.21.15.120:6836/4132440952,v1:172.21.15.120:6837/4132440952] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d4bb1d3d50 con 0x55d4bb28ef60 2024-06-10T03:01:38.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.506+0000 7fc33c841700 1 --2- 172.21.15.52:0/3469960921 >> [v2:172.21.15.120:6836/4132440952,v1:172.21.15.120:6837/4132440952] conn(0x55d4bb28ef60 0x55d4bb291350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:01:38.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.506+0000 7fc33c841700 1 --2- 172.21.15.52:0/3469960921 >> [v2:172.21.15.120:6836/4132440952,v1:172.21.15.120:6837/4132440952] conn(0x55d4bb28ef60 0x55d4bb291350 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a6de677e8b425e6c server_cookie=790201ec01921696 in_seq=0 out_seq=0 2024-06-10T03:01:38.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.506+0000 7fc320ff9700 10 client.16548 ms_handle_connect on v2:172.21.15.120:6836/4132440952 2024-06-10T03:01:38.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.507+0000 7fc320ff9700 1 -- 172.21.15.52:0/3469960921 <== mds.4 v2:172.21.15.120:6836/4132440952 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55d4bb1d3d50 con 0x55d4bb28ef60 2024-06-10T03:01:38.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.507+0000 7fc320ff9700 10 client.16548 handle_command_reply: tid=0 2024-06-10T03:01:38.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.507+0000 7fc349b29b80 1 client.16548 shutdown 2024-06-10T03:01:38.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.507+0000 7fc349b29b80 1 -- 172.21.15.52:0/3469960921 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d4bb2757a0 msgr2=0x55d4bb281680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:01:38.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.507+0000 7fc349b29b80 1 --2- 172.21.15.52:0/3469960921 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d4bb2757a0 0x55d4bb281680 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fc33401c560 tx=0x7fc334009a50 comp rx=0 tx=0).stop 2024-06-10T03:01:38.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.507+0000 7fc349b29b80 1 -- 172.21.15.52:0/3469960921 shutdown_connections 2024-06-10T03:01:38.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.507+0000 7fc349b29b80 1 --2- 172.21.15.52:0/3469960921 >> [v2:172.21.15.120:6836/4132440952,v1:172.21.15.120:6837/4132440952] conn(0x55d4bb28ef60 0x55d4bb291350 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:38.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.507+0000 7fc349b29b80 1 --2- 172.21.15.52:0/3469960921 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d4bb27b880 0x55d4bb27eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:38.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.507+0000 7fc349b29b80 1 --2- 172.21.15.52:0/3469960921 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d4bb2757a0 0x55d4bb281680 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:38.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.507+0000 7fc320ff9700 0 client.16548 ms_handle_reset on v2:172.21.15.120:6836/4132440952 2024-06-10T03:01:38.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.507+0000 7fc349b29b80 1 --2- 172.21.15.52:0/3469960921 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d4bb27af60 0x55d4bb27e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:38.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.507+0000 7fc349b29b80 1 -- 172.21.15.52:0/3469960921 >> 172.21.15.52:0/3469960921 conn(0x55d4bb1a2080 msgr2=0x55d4bb26f700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:01:38.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.508+0000 7fc349b29b80 1 -- 172.21.15.52:0/3469960921 shutdown_connections 2024-06-10T03:01:38.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.508+0000 7fc349b29b80 1 -- 172.21.15.52:0/3469960921 wait complete. 2024-06-10T03:01:38.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.508+0000 7fc349b29b80 20 client.16548 trim_cache size 0 max 16384 2024-06-10T03:01:38.524 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.523+0000 7fc349b29b80 1 Processor -- start 2024-06-10T03:01:38.524 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.523+0000 7fc349b29b80 1 -- start start 2024-06-10T03:01:38.524 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.523+0000 7fc349b29b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d4bb2757a0 0x55d4bb2811c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:01:38.524 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.523+0000 7fc349b29b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d4bb27af60 0x55d4bb27a200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:01:38.524 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.523+0000 7fc349b29b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d4bb27b880 0x55d4bb26fe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:01:38.524 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.523+0000 7fc349b29b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fc3381b0550 con 0x55d4bb2757a0 2024-06-10T03:01:38.524 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.523+0000 7fc349b29b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc3381b2150 con 0x55d4bb27af60 2024-06-10T03:01:38.524 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.523+0000 7fc349b29b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc3381b0c20 con 0x55d4bb27b880 2024-06-10T03:01:38.524 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.524+0000 7fc33d042700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d4bb2757a0 0x55d4bb2811c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:01:38.524 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.524+0000 7fc33d042700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d4bb2757a0 0x55d4bb2811c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47386/0 (socket says 172.21.15.52:47386) 2024-06-10T03:01:38.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.524+0000 7fc33d042700 1 -- 172.21.15.52:0/3493298949 learned_addr learned my addr 172.21.15.52:0/3493298949 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:01:38.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.524+0000 7fc33c841700 1 --2- 172.21.15.52:0/3493298949 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d4bb27af60 0x55d4bb27a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:01:38.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.524+0000 7fc33d843700 1 --2- 172.21.15.52:0/3493298949 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d4bb27b880 0x55d4bb26fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:01:38.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.524+0000 7fc33d042700 1 -- 172.21.15.52:0/3493298949 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d4bb27b880 msgr2=0x55d4bb26fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:01:38.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.524+0000 7fc33d042700 1 --2- 172.21.15.52:0/3493298949 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d4bb27b880 0x55d4bb26fe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:38.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.524+0000 7fc33d042700 1 -- 172.21.15.52:0/3493298949 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d4bb27af60 msgr2=0x55d4bb27a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:01:38.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.524+0000 7fc33d042700 1 --2- 172.21.15.52:0/3493298949 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d4bb27af60 0x55d4bb27a200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:38.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.524+0000 7fc33d042700 1 -- 172.21.15.52:0/3493298949 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d4bb27cf30 con 0x55d4bb2757a0 2024-06-10T03:01:38.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.524+0000 7fc33d042700 1 --2- 172.21.15.52:0/3493298949 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d4bb2757a0 0x55d4bb2811c0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fc334005d50 tx=0x7fc334005ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0fafac756e3fb05 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:01:38.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.524+0000 7fc322ffd700 1 -- 172.21.15.52:0/3493298949 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc334035020 con 0x55d4bb2757a0 2024-06-10T03:01:38.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.524+0000 7fc322ffd700 1 -- 172.21.15.52:0/3493298949 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc33400eca0 con 0x55d4bb2757a0 2024-06-10T03:01:38.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.524+0000 7fc322ffd700 1 -- 172.21.15.52:0/3493298949 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc334004a40 con 0x55d4bb2757a0 2024-06-10T03:01:38.528 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.525+0000 7fc349b29b80 1 -- 172.21.15.52:0/3493298949 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d4bb2757a0 msgr2=0x55d4bb2811c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:01:38.529 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.525+0000 7fc349b29b80 1 --2- 172.21.15.52:0/3493298949 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d4bb2757a0 0x55d4bb2811c0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fc334005d50 tx=0x7fc334005ce0 comp rx=0 tx=0).stop 2024-06-10T03:01:38.529 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.525+0000 7fc349b29b80 1 -- 172.21.15.52:0/3493298949 shutdown_connections 2024-06-10T03:01:38.529 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.525+0000 7fc349b29b80 1 --2- 172.21.15.52:0/3493298949 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d4bb27b880 0x55d4bb26fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:38.529 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.525+0000 7fc349b29b80 1 --2- 172.21.15.52:0/3493298949 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d4bb2757a0 0x55d4bb2811c0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:38.529 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.525+0000 7fc349b29b80 1 --2- 172.21.15.52:0/3493298949 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d4bb27af60 0x55d4bb27a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:38.529 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.525+0000 7fc349b29b80 1 -- 172.21.15.52:0/3493298949 >> 172.21.15.52:0/3493298949 conn(0x55d4bb1a2080 msgr2=0x55d4bb279af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:01:38.529 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.525+0000 7fc349b29b80 1 -- 172.21.15.52:0/3493298949 shutdown_connections 2024-06-10T03:01:38.529 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.525+0000 7fc349b29b80 1 -- 172.21.15.52:0/3493298949 wait complete. 2024-06-10T03:01:38.529 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.525+0000 7fc349b29b80 1 -- start start 2024-06-10T03:01:38.529 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.525+0000 7fc349b29b80 1 Processor -- start 2024-06-10T03:01:38.529 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.526+0000 7fc349b29b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d4bb2757a0 0x55d4bb287700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:01:38.529 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.526+0000 7fc349b29b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d4bb27af60 0x55d4bb287c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:01:38.529 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.526+0000 7fc349b29b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d4bb27b880 0x55d4bb295d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:01:38.529 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.526+0000 7fc349b29b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fc3381b1a80 con 0x55d4bb27b880 2024-06-10T03:01:38.530 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.526+0000 7fc349b29b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc3381b13b0 con 0x55d4bb2757a0 2024-06-10T03:01:38.530 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.526+0000 7fc349b29b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc3381ae4f0 con 0x55d4bb27af60 2024-06-10T03:01:38.530 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.526+0000 7fc349b29b80 10 client.0 _ll_register_callbacks cb 0x55d4bb15c5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T03:01:38.530 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.526+0000 7fc33d843700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d4bb27b880 0x55d4bb295d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:01:38.530 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.526+0000 7fc33d042700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d4bb2757a0 0x55d4bb287700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:01:38.530 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.526+0000 7fc33d843700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d4bb27b880 0x55d4bb295d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47398/0 (socket says 172.21.15.52:47398) 2024-06-10T03:01:38.530 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.526+0000 7fc33d843700 1 -- 172.21.15.52:0/2897135865 learned_addr learned my addr 172.21.15.52:0/2897135865 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:01:38.530 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.526+0000 7fc33c841700 1 --2- 172.21.15.52:0/2897135865 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d4bb27af60 0x55d4bb287c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:01:38.530 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.526+0000 7fc33d843700 1 -- 172.21.15.52:0/2897135865 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d4bb27af60 msgr2=0x55d4bb287c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:01:38.530 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.526+0000 7fc33d843700 1 --2- 172.21.15.52:0/2897135865 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d4bb27af60 0x55d4bb287c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:38.530 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.526+0000 7fc33d843700 1 -- 172.21.15.52:0/2897135865 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d4bb2757a0 msgr2=0x55d4bb287700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:01:38.530 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.526+0000 7fc33d843700 1 --2- 172.21.15.52:0/2897135865 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d4bb2757a0 0x55d4bb287700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:38.530 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.526+0000 7fc33d843700 1 -- 172.21.15.52:0/2897135865 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d4bb2963b0 con 0x55d4bb27b880 2024-06-10T03:01:38.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.526+0000 7fc33c841700 1 --2- 172.21.15.52:0/2897135865 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d4bb27af60 0x55d4bb287c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T03:01:38.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.527+0000 7fc33d843700 1 --2- 172.21.15.52:0/2897135865 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d4bb27b880 0x55d4bb295d70 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fc330098c40 tx=0x7fc330018f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c454883e0234347 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:01:38.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.527+0000 7fc320ff9700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-10T03:01:38.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.527+0000 7fc320ff9700 1 -- 172.21.15.52:0/2897135865 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3300b4020 con 0x55d4bb27b880 2024-06-10T03:01:38.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.527+0000 7fc320ff9700 1 -- 172.21.15.52:0/2897135865 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7fc330009c10 con 0x55d4bb27b880 2024-06-10T03:01:38.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.527+0000 7fc349b29b80 1 -- 172.21.15.52:0/2897135865 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc33400eca0 con 0x55d4bb27b880 2024-06-10T03:01:38.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.527+0000 7fc320ff9700 1 -- 172.21.15.52:0/2897135865 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fc33009c7e0 con 0x55d4bb27b880 2024-06-10T03:01:38.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.527+0000 7fc349b29b80 10 client.16554 fetch_fsmap learned FSMap version 36 2024-06-10T03:01:38.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.527+0000 7fc349b29b80 1 -- 172.21.15.52:0/2897135865 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55d4bb2f92f0 con 0x55d4bb27b880 2024-06-10T03:01:38.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.527+0000 7fc320ff9700 1 -- 172.21.15.52:0/2897135865 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc33400eca0 con 0x55d4bb27b880 2024-06-10T03:01:38.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.527+0000 7fc320ff9700 1 -- 172.21.15.52:0/2897135865 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 36) ==== 8378+0+0 (secure 0 0 0) 0x7fc33002e060 con 0x55d4bb27b880 2024-06-10T03:01:38.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.527+0000 7fc349b29b80 10 client.16554 fetch_fsmap finished waiting for FSMap version 36 2024-06-10T03:01:38.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.527+0000 7fc349b29b80 10 client.16554 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-06-10T03:01:38.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.527+0000 7fc349b29b80 20 client.16554 populate_metadata read hostname 'smithi052' 2024-06-10T03:01:38.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.527+0000 7fc349b29b80 1 --2- 172.21.15.52:0/2897135865 >> [v2:172.21.15.120:6836/4132440952,v1:172.21.15.120:6837/4132440952] conn(0x55d4bb2f9870 0x55d4bb2fbc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:01:38.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.527+0000 7fc349b29b80 4 client.16554 mds_command: new command op to 24428 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T03:01:38.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.527+0000 7fc349b29b80 1 -- 172.21.15.52:0/2897135865 --> [v2:172.21.15.120:6836/4132440952,v1:172.21.15.120:6837/4132440952] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d4bb1ce2d0 con 0x55d4bb2f9870 2024-06-10T03:01:38.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.528+0000 7fc33d042700 1 --2- 172.21.15.52:0/2897135865 >> [v2:172.21.15.120:6836/4132440952,v1:172.21.15.120:6837/4132440952] conn(0x55d4bb2f9870 0x55d4bb2fbc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:01:38.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.528+0000 7fc320ff9700 10 client.16554 ms_handle_connect on v2:172.21.15.120:6836/4132440952 2024-06-10T03:01:38.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.528+0000 7fc33d042700 1 --2- 172.21.15.52:0/2897135865 >> [v2:172.21.15.120:6836/4132440952,v1:172.21.15.120:6837/4132440952] conn(0x55d4bb2f9870 0x55d4bb2fbc90 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=2a332c95629c124d server_cookie=5b8509e20884e5cd in_seq=0 out_seq=0 2024-06-10T03:01:38.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.528+0000 7fc320ff9700 1 -- 172.21.15.52:0/2897135865 <== mds.4 v2:172.21.15.120:6836/4132440952 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d4bb1ce2d0 con 0x55d4bb2f9870 2024-06-10T03:01:38.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.528+0000 7fc320ff9700 10 client.16554 handle_command_reply: tid=0 2024-06-10T03:01:38.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.528+0000 7fc349b29b80 1 client.16554 shutdown 2024-06-10T03:01:38.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.528+0000 7fc349b29b80 1 -- 172.21.15.52:0/2897135865 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d4bb27b880 msgr2=0x55d4bb295d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:01:38.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.528+0000 7fc349b29b80 1 --2- 172.21.15.52:0/2897135865 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d4bb27b880 0x55d4bb295d70 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fc330098c40 tx=0x7fc330018f90 comp rx=0 tx=0).stop 2024-06-10T03:01:38.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.528+0000 7fc349b29b80 1 -- 172.21.15.52:0/2897135865 shutdown_connections 2024-06-10T03:01:38.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.528+0000 7fc349b29b80 1 --2- 172.21.15.52:0/2897135865 >> [v2:172.21.15.120:6836/4132440952,v1:172.21.15.120:6837/4132440952] conn(0x55d4bb2f9870 0x55d4bb2fbc90 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:38.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.528+0000 7fc349b29b80 1 --2- 172.21.15.52:0/2897135865 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d4bb27b880 0x55d4bb295d70 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:38.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.528+0000 7fc349b29b80 1 --2- 172.21.15.52:0/2897135865 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d4bb2757a0 0x55d4bb287700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:38.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.528+0000 7fc349b29b80 1 --2- 172.21.15.52:0/2897135865 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d4bb27af60 0x55d4bb287c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:38.533 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.528+0000 7fc349b29b80 1 -- 172.21.15.52:0/2897135865 >> 172.21.15.52:0/2897135865 conn(0x55d4bb1a2080 msgr2=0x55d4bb27eac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:01:38.533 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.528+0000 7fc320ff9700 0 client.16554 ms_handle_reset on v2:172.21.15.120:6836/4132440952 2024-06-10T03:01:38.533 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.529+0000 7fc349b29b80 1 -- 172.21.15.52:0/2897135865 shutdown_connections 2024-06-10T03:01:38.533 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.529+0000 7fc349b29b80 1 -- 172.21.15.52:0/2897135865 wait complete. 2024-06-10T03:01:38.533 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.529+0000 7fc349b29b80 20 client.16554 trim_cache size 0 max 16384 2024-06-10T03:01:38.533 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-06-10T03:01:38.534 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.529+0000 7fc33f2a6700 1 -- 172.21.15.52:0/4243894105 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fc3240a2c20 msgr2=0x7fc3240a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:01:38.534 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.529+0000 7fc33f2a6700 1 --2- 172.21.15.52:0/4243894105 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fc3240a2c20 0x7fc3240a50e0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fc334009ca0 tx=0x7fc334009260 comp rx=0 tx=0).stop 2024-06-10T03:01:38.534 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.529+0000 7fc33f2a6700 1 -- 172.21.15.52:0/4243894105 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc3381085d0 msgr2=0x7fc338075210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:01:38.534 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.529+0000 7fc33f2a6700 1 --2- 172.21.15.52:0/4243894105 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc3381085d0 0x7fc338075210 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fc33000b770 tx=0x7fc330007a20 comp rx=0 tx=0).stop 2024-06-10T03:01:38.534 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.529+0000 7fc33f2a6700 1 -- 172.21.15.52:0/4243894105 shutdown_connections 2024-06-10T03:01:38.534 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.529+0000 7fc33f2a6700 1 --2- 172.21.15.52:0/4243894105 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc3381085d0 0x7fc338075210 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:38.534 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.529+0000 7fc33f2a6700 1 --2- 172.21.15.52:0/4243894105 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fc3240a2c20 0x7fc3240a50e0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:38.534 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.529+0000 7fc33f2a6700 1 --2- 172.21.15.52:0/4243894105 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc338102530 0x7fc338074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:38.534 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.529+0000 7fc33f2a6700 1 --2- 172.21.15.52:0/4243894105 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc338102ef0 0x7fc338074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:38.534 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.529+0000 7fc33f2a6700 1 -- 172.21.15.52:0/4243894105 >> 172.21.15.52:0/4243894105 conn(0x7fc338079640 msgr2=0x7fc338101010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:01:38.534 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.529+0000 7fc33f2a6700 1 -- 172.21.15.52:0/4243894105 shutdown_connections 2024-06-10T03:01:38.534 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:38.529+0000 7fc33f2a6700 1 -- 172.21.15.52:0/4243894105 wait complete. 2024-06-10T03:01:38.621 INFO:tasks.workunit.client.0.smithi052.stdout: 2024-06-10T03:01:38.621 INFO:tasks.workunit.client.0.smithi052.stdout:Final score for writes: 123 2024-06-10T03:01:38.621 INFO:tasks.workunit.client.0.smithi052.stdout:Final score for reads : 386867 2024-06-10T03:01:38.621 INFO:tasks.workunit.client.0.smithi052.stdout: 2024-06-10T03:01:38.622 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-06-10T03:01:38.622 DEBUG:teuthology.orchestra.run.smithi052:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-06-10T03:01:38.703 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:01:38 smithi188 ceph-mon[36599]: pgmap v657: 129 pgs: 129 active+clean; 626 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.8 MiB/s wr, 612 op/s 2024-06-10T03:01:38.888 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:01:38 smithi120 ceph-mon[36634]: pgmap v657: 129 pgs: 129 active+clean; 626 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.8 MiB/s wr, 612 op/s 2024-06-10T03:01:39.217 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-06-10T03:01:39.473 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T03:01:40.658 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:01:40 smithi052 ceph-mon[32362]: pgmap v658: 129 pgs: 129 active+clean; 625 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.1 MiB/s wr, 511 op/s 2024-06-10T03:01:40.703 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:01:40 smithi188 ceph-mon[36599]: pgmap v658: 129 pgs: 129 active+clean; 625 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.1 MiB/s wr, 511 op/s 2024-06-10T03:01:40.888 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:01:40 smithi120 ceph-mon[36634]: pgmap v658: 129 pgs: 129 active+clean; 625 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.1 MiB/s wr, 511 op/s 2024-06-10T03:01:41.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.283+0000 7fd4a6d91700 1 -- 172.21.15.52:0/1266840207 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd4980a6ae0 msgr2=0x7fd4980a8ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:01:41.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.283+0000 7fd4a6d91700 1 --2- 172.21.15.52:0/1266840207 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd4980a6ae0 0x7fd4980a8ed0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fd4a0067f10 tx=0x7fd4a0068aa0 comp rx=0 tx=0).stop 2024-06-10T03:01:41.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.283+0000 7fd4a6d91700 1 -- 172.21.15.52:0/1266840207 shutdown_connections 2024-06-10T03:01:41.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.283+0000 7fd4a6d91700 1 --2- 172.21.15.52:0/1266840207 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd4980a6ae0 0x7fd4980a8ed0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:41.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.283+0000 7fd4a6d91700 1 --2- 172.21.15.52:0/1266840207 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd498094650 0x7fd498094a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:41.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.283+0000 7fd4a6d91700 1 --2- 172.21.15.52:0/1266840207 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd498094f70 0x7fd4980a65a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:41.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.283+0000 7fd4a6d91700 1 -- 172.21.15.52:0/1266840207 >> 172.21.15.52:0/1266840207 conn(0x7fd498009f10 msgr2=0x7fd49800a320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:01:41.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.285+0000 7fd4a6d91700 1 -- 172.21.15.52:0/1266840207 shutdown_connections 2024-06-10T03:01:41.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.285+0000 7fd4a6d91700 1 -- 172.21.15.52:0/1266840207 wait complete. 2024-06-10T03:01:41.286 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.286+0000 7fd4a6d91700 1 Processor -- start 2024-06-10T03:01:41.286 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.286+0000 7fd4a6d91700 1 -- start start 2024-06-10T03:01:41.286 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.286+0000 7fd4a6d91700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd498094650 0x7fd4980a4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:01:41.286 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.286+0000 7fd4a6d91700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd498094f70 0x7fd49809dc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:01:41.287 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.287+0000 7fd4a6d91700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd4980a6ae0 0x7fd49809e170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:01:41.287 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.287+0000 7fd4a6d91700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fd4980ac650 con 0x7fd498094650 2024-06-10T03:01:41.287 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.287+0000 7fd4a6d91700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd4980ac4d0 con 0x7fd4980a6ae0 2024-06-10T03:01:41.287 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.287+0000 7fd4a6d91700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fd4980ac7d0 con 0x7fd498094f70 2024-06-10T03:01:41.287 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.287+0000 7fd4a5d8f700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd498094650 0x7fd4980a4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:01:41.287 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.287+0000 7fd4a5d8f700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd498094650 0x7fd4980a4ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:35728/0 (socket says 172.21.15.52:35728) 2024-06-10T03:01:41.287 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.287+0000 7fd4a5d8f700 1 -- 172.21.15.52:0/957300453 learned_addr learned my addr 172.21.15.52:0/957300453 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:01:41.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.289+0000 7fd4a558e700 1 --2- 172.21.15.52:0/957300453 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd498094f70 0x7fd49809dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:01:41.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.289+0000 7fd4a6590700 1 --2- 172.21.15.52:0/957300453 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd4980a6ae0 0x7fd49809e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:01:41.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.289+0000 7fd4a5d8f700 1 -- 172.21.15.52:0/957300453 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd498094f70 msgr2=0x7fd49809dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:01:41.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.289+0000 7fd4a5d8f700 1 --2- 172.21.15.52:0/957300453 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd498094f70 0x7fd49809dc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:41.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.289+0000 7fd4a5d8f700 1 -- 172.21.15.52:0/957300453 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd4980a6ae0 msgr2=0x7fd49809e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:01:41.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.289+0000 7fd4a5d8f700 1 --2- 172.21.15.52:0/957300453 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd4980a6ae0 0x7fd49809e170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:41.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.289+0000 7fd4a5d8f700 1 -- 172.21.15.52:0/957300453 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd49809e780 con 0x7fd498094650 2024-06-10T03:01:41.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.289+0000 7fd4a558e700 1 --2- 172.21.15.52:0/957300453 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd498094f70 0x7fd49809dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T03:01:41.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.289+0000 7fd4a5d8f700 1 --2- 172.21.15.52:0/957300453 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd498094650 0x7fd4980a4ba0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fd49c00c920 tx=0x7fd49c00cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:01:41.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.290+0000 7fd496ffd700 1 -- 172.21.15.52:0/957300453 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd49c007b00 con 0x7fd498094650 2024-06-10T03:01:41.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.290+0000 7fd496ffd700 1 -- 172.21.15.52:0/957300453 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd49c004d10 con 0x7fd498094650 2024-06-10T03:01:41.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.290+0000 7fd496ffd700 1 -- 172.21.15.52:0/957300453 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd49c00f450 con 0x7fd498094650 2024-06-10T03:01:41.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.290+0000 7fd4a6d91700 1 -- 172.21.15.52:0/957300453 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd49809ea70 con 0x7fd498094650 2024-06-10T03:01:41.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.291+0000 7fd4a6d91700 1 -- 172.21.15.52:0/957300453 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4980a29f0 con 0x7fd498094650 2024-06-10T03:01:41.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.294+0000 7fd496ffd700 1 -- 172.21.15.52:0/957300453 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fd49c004750 con 0x7fd498094650 2024-06-10T03:01:41.295 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.295+0000 7fd496ffd700 1 --2- 172.21.15.52:0/957300453 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fd48c0a2da0 0x7fd48c0a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:01:41.295 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.295+0000 7fd4a558e700 1 --2- 172.21.15.52:0/957300453 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fd48c0a2da0 0x7fd48c0a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:01:41.295 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.295+0000 7fd496ffd700 1 -- 172.21.15.52:0/957300453 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7fd49c0cf2a0 con 0x7fd498094650 2024-06-10T03:01:41.295 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.295+0000 7fd4b2876b80 1 Processor -- start 2024-06-10T03:01:41.295 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.295+0000 7fd4b2876b80 1 -- start start 2024-06-10T03:01:41.297 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.296+0000 7fd4a558e700 1 --2- 172.21.15.52:0/957300453 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fd48c0a2da0 0x7fd48c0a5260 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fd490006fd0 tx=0x7fd490008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:01:41.297 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.296+0000 7fd4b2876b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b7329460a0 0x55b732946480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:01:41.297 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.296+0000 7fd4b2876b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b7329469c0 0x55b7329414b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:01:41.297 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.296+0000 7fd4b2876b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b7329419f0 0x55b732941e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:01:41.297 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.296+0000 7fd4b2876b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55b732833f60 con 0x55b7329419f0 2024-06-10T03:01:41.297 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.296+0000 7fd4b2876b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55b732892850 con 0x55b7329460a0 2024-06-10T03:01:41.297 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.296+0000 7fd4b2876b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55b7327fa460 con 0x55b7329469c0 2024-06-10T03:01:41.297 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.296+0000 7fd4a6590700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b7329419f0 0x55b732941e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:01:41.297 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.296+0000 7fd4a5d8f700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b7329460a0 0x55b732946480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:01:41.297 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.296+0000 7fd4a558e700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b7329469c0 0x55b7329414b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:01:41.298 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.296+0000 7fd4a5d8f700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b7329460a0 0x55b732946480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.52:48174/0 (socket says 172.21.15.52:48174) 2024-06-10T03:01:41.298 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.296+0000 7fd4a5d8f700 1 -- 172.21.15.52:0/3446561961 learned_addr learned my addr 172.21.15.52:0/3446561961 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:01:41.298 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.296+0000 7fd4a5d8f700 1 -- 172.21.15.52:0/3446561961 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b7329469c0 msgr2=0x55b7329414b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:01:41.298 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.296+0000 7fd4a5d8f700 1 --2- 172.21.15.52:0/3446561961 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b7329469c0 0x55b7329414b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:41.298 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.296+0000 7fd4a5d8f700 1 -- 172.21.15.52:0/3446561961 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b7329419f0 msgr2=0x55b732941e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:01:41.298 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.296+0000 7fd4a5d8f700 1 --2- 172.21.15.52:0/3446561961 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b7329419f0 0x55b732941e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:41.298 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.296+0000 7fd4a5d8f700 1 -- 172.21.15.52:0/3446561961 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b7327e18d0 con 0x55b7329460a0 2024-06-10T03:01:41.298 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.297+0000 7fd4a5d8f700 1 --2- 172.21.15.52:0/3446561961 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b7329460a0 0x55b732946480 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd49c0981a0 tx=0x7fd49c09b390 comp rx=0 tx=0).ready entity=mon.2 client_cookie=97271c5bb4d3ad2f server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:01:41.298 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.297+0000 7fd48bfff700 1 -- 172.21.15.52:0/3446561961 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd49c0b3070 con 0x55b7329460a0 2024-06-10T03:01:41.298 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.297+0000 7fd48bfff700 1 -- 172.21.15.52:0/3446561961 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd49c09b7d0 con 0x55b7329460a0 2024-06-10T03:01:41.298 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.297+0000 7fd48bfff700 1 -- 172.21.15.52:0/3446561961 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd49c099ad0 con 0x55b7329460a0 2024-06-10T03:01:41.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.298+0000 7fd4b2876b80 1 -- 172.21.15.52:0/3446561961 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b7329460a0 msgr2=0x55b732946480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:01:41.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.298+0000 7fd4b2876b80 1 --2- 172.21.15.52:0/3446561961 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b7329460a0 0x55b732946480 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd49c0981a0 tx=0x7fd49c09b390 comp rx=0 tx=0).stop 2024-06-10T03:01:41.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.298+0000 7fd4b2876b80 1 -- 172.21.15.52:0/3446561961 shutdown_connections 2024-06-10T03:01:41.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.298+0000 7fd4b2876b80 1 --2- 172.21.15.52:0/3446561961 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b7329419f0 0x55b732941e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:41.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.298+0000 7fd4b2876b80 1 --2- 172.21.15.52:0/3446561961 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b7329460a0 0x55b732946480 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:41.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.298+0000 7fd4b2876b80 1 --2- 172.21.15.52:0/3446561961 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b7329469c0 0x55b7329414b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:41.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.298+0000 7fd4b2876b80 1 -- 172.21.15.52:0/3446561961 >> 172.21.15.52:0/3446561961 conn(0x55b73286e080 msgr2=0x55b73293c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:01:41.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.298+0000 7fd4b2876b80 1 -- 172.21.15.52:0/3446561961 shutdown_connections 2024-06-10T03:01:41.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.298+0000 7fd4b2876b80 1 -- 172.21.15.52:0/3446561961 wait complete. 2024-06-10T03:01:41.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.298+0000 7fd4b2876b80 1 -- start start 2024-06-10T03:01:41.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.298+0000 7fd4b2876b80 1 Processor -- start 2024-06-10T03:01:41.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.299+0000 7fd4b2876b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b7329419f0 0x55b73294d770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:01:41.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.299+0000 7fd4b2876b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b7329460a0 0x55b73294aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:01:41.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.299+0000 7fd4b2876b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b7329469c0 0x55b73294afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:01:41.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.299+0000 7fd4b2876b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55b732832fc0 con 0x55b7329460a0 2024-06-10T03:01:41.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.299+0000 7fd4b2876b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55b73288e810 con 0x55b7329419f0 2024-06-10T03:01:41.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.299+0000 7fd4b2876b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55b73294e450 con 0x55b7329469c0 2024-06-10T03:01:41.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.299+0000 7fd4b2876b80 10 client.0 _ll_register_callbacks cb 0x55b7328285b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T03:01:41.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.299+0000 7fd4a5d8f700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b7329419f0 0x55b73294d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:01:41.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.299+0000 7fd4a5d8f700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b7329419f0 0x55b73294d770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.52:48188/0 (socket says 172.21.15.52:48188) 2024-06-10T03:01:41.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.299+0000 7fd4a5d8f700 1 -- 172.21.15.52:0/524513311 learned_addr learned my addr 172.21.15.52:0/524513311 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:01:41.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.299+0000 7fd4a558e700 1 --2- 172.21.15.52:0/524513311 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b7329460a0 0x55b73294aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:01:41.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.299+0000 7fd4a6590700 1 --2- 172.21.15.52:0/524513311 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b7329469c0 0x55b73294afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:01:41.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.299+0000 7fd4a5d8f700 1 -- 172.21.15.52:0/524513311 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b7329469c0 msgr2=0x55b73294afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:01:41.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.299+0000 7fd4a5d8f700 1 --2- 172.21.15.52:0/524513311 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b7329469c0 0x55b73294afb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:41.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.299+0000 7fd4a5d8f700 1 -- 172.21.15.52:0/524513311 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b7329460a0 msgr2=0x55b73294aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:01:41.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.299+0000 7fd4a5d8f700 1 --2- 172.21.15.52:0/524513311 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b7329460a0 0x55b73294aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:41.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.299+0000 7fd4a5d8f700 1 -- 172.21.15.52:0/524513311 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b732858750 con 0x55b7329419f0 2024-06-10T03:01:41.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.299+0000 7fd4a558e700 1 --2- 172.21.15.52:0/524513311 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b7329460a0 0x55b73294aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T03:01:41.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.300+0000 7fd4a5d8f700 1 --2- 172.21.15.52:0/524513311 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b7329419f0 0x55b73294d770 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fd49c097c30 tx=0x7fd49c0cf150 comp rx=0 tx=0).ready entity=mon.2 client_cookie=620d23fe11390e3e server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:01:41.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.300+0000 7fd4897fa700 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-06-10T03:01:41.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.300+0000 7fd4897fa700 1 -- 172.21.15.52:0/524513311 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd49c0ba070 con 0x55b7329419f0 2024-06-10T03:01:41.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.300+0000 7fd4b2876b80 1 -- 172.21.15.52:0/524513311 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd49c09b7d0 con 0x55b7329419f0 2024-06-10T03:01:41.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.300+0000 7fd4897fa700 1 -- 172.21.15.52:0/524513311 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7fd49c099ad0 con 0x55b7329419f0 2024-06-10T03:01:41.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.300+0000 7fd4897fa700 1 -- 172.21.15.52:0/524513311 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fd49c0b3040 con 0x55b7329419f0 2024-06-10T03:01:41.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.300+0000 7fd4b2876b80 10 client.26084 fetch_fsmap learned FSMap version 36 2024-06-10T03:01:41.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.300+0000 7fd4b2876b80 1 -- 172.21.15.52:0/524513311 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55b732958b50 con 0x55b7329419f0 2024-06-10T03:01:41.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.300+0000 7fd4897fa700 1 -- 172.21.15.52:0/524513311 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd49c09b7d0 con 0x55b7329419f0 2024-06-10T03:01:41.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.300+0000 7fd4897fa700 1 -- 172.21.15.52:0/524513311 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 36) ==== 8378+0+0 (secure 0 0 0) 0x7fd49c0a3fb0 con 0x55b7329419f0 2024-06-10T03:01:41.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.300+0000 7fd4b2876b80 10 client.26084 fetch_fsmap finished waiting for FSMap version 36 2024-06-10T03:01:41.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.300+0000 7fd4b2876b80 10 client.26084 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.b 2024-06-10T03:01:41.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.300+0000 7fd4b2876b80 20 client.26084 populate_metadata read hostname 'smithi052' 2024-06-10T03:01:41.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.300+0000 7fd4b2876b80 1 --2- 172.21.15.52:0/524513311 >> [v2:172.21.15.120:6838/1672436487,v1:172.21.15.120:6839/1672436487] conn(0x55b73295b020 0x55b73295d410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:01:41.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.300+0000 7fd4b2876b80 4 client.26084 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T03:01:41.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.300+0000 7fd4b2876b80 1 -- 172.21.15.52:0/524513311 --> [v2:172.21.15.120:6838/1672436487,v1:172.21.15.120:6839/1672436487] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b73289fd50 con 0x55b73295b020 2024-06-10T03:01:41.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.301+0000 7fd4a558e700 1 --2- 172.21.15.52:0/524513311 >> [v2:172.21.15.120:6838/1672436487,v1:172.21.15.120:6839/1672436487] conn(0x55b73295b020 0x55b73295d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:01:41.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.301+0000 7fd4a558e700 1 --2- 172.21.15.52:0/524513311 >> [v2:172.21.15.120:6838/1672436487,v1:172.21.15.120:6839/1672436487] conn(0x55b73295b020 0x55b73295d410 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=edd1b1758c70328b server_cookie=8cbc412583c3613e in_seq=0 out_seq=0 2024-06-10T03:01:41.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.301+0000 7fd4897fa700 10 client.26084 ms_handle_connect on v2:172.21.15.120:6838/1672436487 2024-06-10T03:01:41.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.302+0000 7fd4897fa700 1 -- 172.21.15.52:0/524513311 <== mds.3 v2:172.21.15.120:6838/1672436487 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55b73289fd50 con 0x55b73295b020 2024-06-10T03:01:41.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.302+0000 7fd4897fa700 10 client.26084 handle_command_reply: tid=0 2024-06-10T03:01:41.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.302+0000 7fd4b2876b80 1 client.26084 shutdown 2024-06-10T03:01:41.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.302+0000 7fd4b2876b80 1 -- 172.21.15.52:0/524513311 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b7329419f0 msgr2=0x55b73294d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:01:41.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.302+0000 7fd4b2876b80 1 --2- 172.21.15.52:0/524513311 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b7329419f0 0x55b73294d770 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fd49c097c30 tx=0x7fd49c0cf150 comp rx=0 tx=0).stop 2024-06-10T03:01:41.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.302+0000 7fd4b2876b80 1 -- 172.21.15.52:0/524513311 shutdown_connections 2024-06-10T03:01:41.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.302+0000 7fd4b2876b80 1 --2- 172.21.15.52:0/524513311 >> [v2:172.21.15.120:6838/1672436487,v1:172.21.15.120:6839/1672436487] conn(0x55b73295b020 0x55b73295d410 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:41.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.302+0000 7fd4b2876b80 1 --2- 172.21.15.52:0/524513311 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b7329469c0 0x55b73294afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:41.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.302+0000 7fd4b2876b80 1 --2- 172.21.15.52:0/524513311 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b7329419f0 0x55b73294d770 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:41.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.302+0000 7fd4897fa700 0 client.26084 ms_handle_reset on v2:172.21.15.120:6838/1672436487 2024-06-10T03:01:41.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.302+0000 7fd4b2876b80 1 --2- 172.21.15.52:0/524513311 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b7329460a0 0x55b73294aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:41.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.302+0000 7fd4b2876b80 1 -- 172.21.15.52:0/524513311 >> 172.21.15.52:0/524513311 conn(0x55b73286e080 msgr2=0x55b73293b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:01:41.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.302+0000 7fd4b2876b80 1 -- 172.21.15.52:0/524513311 shutdown_connections 2024-06-10T03:01:41.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.302+0000 7fd4b2876b80 1 -- 172.21.15.52:0/524513311 wait complete. 2024-06-10T03:01:41.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.302+0000 7fd4b2876b80 20 client.26084 trim_cache size 0 max 16384 2024-06-10T03:01:41.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.318+0000 7fd4b2876b80 1 Processor -- start 2024-06-10T03:01:41.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.318+0000 7fd4b2876b80 1 -- start start 2024-06-10T03:01:41.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.318+0000 7fd4b2876b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b7329419f0 0x55b73293c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:01:41.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.318+0000 7fd4b2876b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b7329460a0 0x55b73293f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:01:41.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.318+0000 7fd4b2876b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b7329469c0 0x55b73293f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:01:41.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.318+0000 7fd4b2876b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fd498150c40 con 0x55b7329469c0 2024-06-10T03:01:41.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.318+0000 7fd4b2876b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd498152840 con 0x55b7329460a0 2024-06-10T03:01:41.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.318+0000 7fd4b2876b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fd498151310 con 0x55b7329419f0 2024-06-10T03:01:41.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.318+0000 7fd4a6590700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b7329469c0 0x55b73293f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:01:41.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.318+0000 7fd4a6590700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b7329469c0 0x55b73293f9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:35750/0 (socket says 172.21.15.52:35750) 2024-06-10T03:01:41.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.318+0000 7fd4a6590700 1 -- 172.21.15.52:0/42217027 learned_addr learned my addr 172.21.15.52:0/42217027 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:01:41.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.318+0000 7fd4a5d8f700 1 --2- 172.21.15.52:0/42217027 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b7329419f0 0x55b73293c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:01:41.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.318+0000 7fd4a558e700 1 --2- 172.21.15.52:0/42217027 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b7329460a0 0x55b73293f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:01:41.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.319+0000 7fd4a6590700 1 -- 172.21.15.52:0/42217027 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b7329419f0 msgr2=0x55b73293c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:01:41.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.319+0000 7fd4a6590700 1 --2- 172.21.15.52:0/42217027 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b7329419f0 0x55b73293c0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:41.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.319+0000 7fd4a6590700 1 -- 172.21.15.52:0/42217027 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b7329460a0 msgr2=0x55b73293f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:01:41.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.319+0000 7fd4a6590700 1 --2- 172.21.15.52:0/42217027 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b7329460a0 0x55b73293f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:41.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.319+0000 7fd4a6590700 1 -- 172.21.15.52:0/42217027 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b732957750 con 0x55b7329469c0 2024-06-10T03:01:41.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.319+0000 7fd4a558e700 1 --2- 172.21.15.52:0/42217027 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b7329460a0 0x55b73293f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T03:01:41.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.319+0000 7fd4a6590700 1 --2- 172.21.15.52:0/42217027 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b7329469c0 0x55b73293f9e0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fd4a0051b50 tx=0x7fd4a0068610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6da47ca6dd5735fc server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:01:41.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.319+0000 7fd48b7fe700 1 -- 172.21.15.52:0/42217027 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4a0084070 con 0x55b7329469c0 2024-06-10T03:01:41.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.319+0000 7fd48b7fe700 1 -- 172.21.15.52:0/42217027 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd4a0074cb0 con 0x55b7329469c0 2024-06-10T03:01:41.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.319+0000 7fd48b7fe700 1 -- 172.21.15.52:0/42217027 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4a006a6d0 con 0x55b7329469c0 2024-06-10T03:01:41.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.320+0000 7fd4b2876b80 1 -- 172.21.15.52:0/42217027 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b7329469c0 msgr2=0x55b73293f9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:01:41.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.320+0000 7fd4b2876b80 1 --2- 172.21.15.52:0/42217027 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b7329469c0 0x55b73293f9e0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fd4a0051b50 tx=0x7fd4a0068610 comp rx=0 tx=0).stop 2024-06-10T03:01:41.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.320+0000 7fd4b2876b80 1 -- 172.21.15.52:0/42217027 shutdown_connections 2024-06-10T03:01:41.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.320+0000 7fd4b2876b80 1 --2- 172.21.15.52:0/42217027 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b7329469c0 0x55b73293f9e0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:41.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.320+0000 7fd4b2876b80 1 --2- 172.21.15.52:0/42217027 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b7329419f0 0x55b73293c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:41.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.320+0000 7fd4b2876b80 1 --2- 172.21.15.52:0/42217027 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b7329460a0 0x55b73293f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:41.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.320+0000 7fd4b2876b80 1 -- 172.21.15.52:0/42217027 >> 172.21.15.52:0/42217027 conn(0x55b73286e080 msgr2=0x55b73294cb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:01:41.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.320+0000 7fd4b2876b80 1 -- 172.21.15.52:0/42217027 shutdown_connections 2024-06-10T03:01:41.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.320+0000 7fd4b2876b80 1 -- 172.21.15.52:0/42217027 wait complete. 2024-06-10T03:01:41.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.321+0000 7fd4b2876b80 1 -- start start 2024-06-10T03:01:41.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.321+0000 7fd4b2876b80 1 Processor -- start 2024-06-10T03:01:41.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.321+0000 7fd4b2876b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b7329419f0 0x55b732960440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:01:41.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.321+0000 7fd4b2876b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b7329460a0 0x55b732960980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:01:41.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.321+0000 7fd4a5d8f700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b7329419f0 0x55b732960440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:01:41.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.321+0000 7fd4a5d8f700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b7329419f0 0x55b732960440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:35754/0 (socket says 172.21.15.52:35754) 2024-06-10T03:01:41.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.322+0000 7fd4a5d8f700 1 -- 172.21.15.52:0/1096128930 learned_addr learned my addr 172.21.15.52:0/1096128930 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:01:41.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.322+0000 7fd4b2876b80 1 --2- 172.21.15.52:0/1096128930 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b7329469c0 0x55b732946de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:01:41.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.322+0000 7fd4b2876b80 1 -- 172.21.15.52:0/1096128930 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fd498152170 con 0x55b7329419f0 2024-06-10T03:01:41.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.322+0000 7fd4b2876b80 1 -- 172.21.15.52:0/1096128930 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd498151aa0 con 0x55b7329460a0 2024-06-10T03:01:41.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.322+0000 7fd4b2876b80 1 -- 172.21.15.52:0/1096128930 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fd49814ebe0 con 0x55b7329469c0 2024-06-10T03:01:41.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.322+0000 7fd4b2876b80 10 client.0 _ll_register_callbacks cb 0x55b7328285b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T03:01:41.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.322+0000 7fd4a6590700 1 --2- 172.21.15.52:0/1096128930 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b7329469c0 0x55b732946de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:01:41.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.322+0000 7fd4a558e700 1 --2- 172.21.15.52:0/1096128930 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b7329460a0 0x55b732960980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:01:41.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.322+0000 7fd4a5d8f700 1 -- 172.21.15.52:0/1096128930 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b7329469c0 msgr2=0x55b732946de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:01:41.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.322+0000 7fd4a5d8f700 1 --2- 172.21.15.52:0/1096128930 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b7329469c0 0x55b732946de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:41.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.322+0000 7fd4a5d8f700 1 -- 172.21.15.52:0/1096128930 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b7329460a0 msgr2=0x55b732960980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:01:41.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.322+0000 7fd4a5d8f700 1 --2- 172.21.15.52:0/1096128930 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b7329460a0 0x55b732960980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:41.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.322+0000 7fd4a5d8f700 1 -- 172.21.15.52:0/1096128930 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b732961080 con 0x55b7329419f0 2024-06-10T03:01:41.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.322+0000 7fd4a558e700 1 --2- 172.21.15.52:0/1096128930 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b7329460a0 0x55b732960980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T03:01:41.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.322+0000 7fd4a5d8f700 1 --2- 172.21.15.52:0/1096128930 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b7329419f0 0x55b732960440 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fd49c09b4d0 tx=0x7fd49c09b1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b530335e0a5fa7a server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:01:41.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.322+0000 7fd4897fa700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-10T03:01:41.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.322+0000 7fd4897fa700 1 -- 172.21.15.52:0/1096128930 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd49c0af030 con 0x55b7329419f0 2024-06-10T03:01:41.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.322+0000 7fd4897fa700 1 -- 172.21.15.52:0/1096128930 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7fd49c004a40 con 0x55b7329419f0 2024-06-10T03:01:41.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.322+0000 7fd4b2876b80 1 -- 172.21.15.52:0/1096128930 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd4a0074cb0 con 0x55b7329419f0 2024-06-10T03:01:41.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.322+0000 7fd4897fa700 1 -- 172.21.15.52:0/1096128930 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fd49c0ba040 con 0x55b7329419f0 2024-06-10T03:01:41.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.323+0000 7fd4b2876b80 10 client.16572 fetch_fsmap learned FSMap version 36 2024-06-10T03:01:41.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.323+0000 7fd4b2876b80 1 -- 172.21.15.52:0/1096128930 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55b7329c4ea0 con 0x55b7329419f0 2024-06-10T03:01:41.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.323+0000 7fd4897fa700 1 -- 172.21.15.52:0/1096128930 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd4a0074cb0 con 0x55b7329419f0 2024-06-10T03:01:41.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.323+0000 7fd4897fa700 1 -- 172.21.15.52:0/1096128930 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 36) ==== 8378+0+0 (secure 0 0 0) 0x7fd49c09ba30 con 0x55b7329419f0 2024-06-10T03:01:41.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.323+0000 7fd4b2876b80 10 client.16572 fetch_fsmap finished waiting for FSMap version 36 2024-06-10T03:01:41.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.323+0000 7fd4b2876b80 10 client.16572 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.b 2024-06-10T03:01:41.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.323+0000 7fd4b2876b80 20 client.16572 populate_metadata read hostname 'smithi052' 2024-06-10T03:01:41.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.323+0000 7fd4b2876b80 1 --2- 172.21.15.52:0/1096128930 >> [v2:172.21.15.120:6838/1672436487,v1:172.21.15.120:6839/1672436487] conn(0x55b7329c5310 0x55b7329c7700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:01:41.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.323+0000 7fd4b2876b80 4 client.16572 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T03:01:41.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.323+0000 7fd4b2876b80 1 -- 172.21.15.52:0/1096128930 --> [v2:172.21.15.120:6838/1672436487,v1:172.21.15.120:6839/1672436487] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b73289a2d0 con 0x55b7329c5310 2024-06-10T03:01:41.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.323+0000 7fd4a558e700 1 --2- 172.21.15.52:0/1096128930 >> [v2:172.21.15.120:6838/1672436487,v1:172.21.15.120:6839/1672436487] conn(0x55b7329c5310 0x55b7329c7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:01:41.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.324+0000 7fd4a558e700 1 --2- 172.21.15.52:0/1096128930 >> [v2:172.21.15.120:6838/1672436487,v1:172.21.15.120:6839/1672436487] conn(0x55b7329c5310 0x55b7329c7700 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=6c9d9d4e0d101a34 server_cookie=3201124400a8ca3f in_seq=0 out_seq=0 2024-06-10T03:01:41.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.324+0000 7fd4897fa700 10 client.16572 ms_handle_connect on v2:172.21.15.120:6838/1672436487 2024-06-10T03:01:41.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.324+0000 7fd4897fa700 1 -- 172.21.15.52:0/1096128930 <== mds.3 v2:172.21.15.120:6838/1672436487 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b73289a2d0 con 0x55b7329c5310 2024-06-10T03:01:41.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.324+0000 7fd4897fa700 10 client.16572 handle_command_reply: tid=0 2024-06-10T03:01:41.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.324+0000 7fd4b2876b80 1 client.16572 shutdown 2024-06-10T03:01:41.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.324+0000 7fd4b2876b80 1 -- 172.21.15.52:0/1096128930 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b7329419f0 msgr2=0x55b732960440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:01:41.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.324+0000 7fd4b2876b80 1 --2- 172.21.15.52:0/1096128930 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b7329419f0 0x55b732960440 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fd49c09b4d0 tx=0x7fd49c09b1c0 comp rx=0 tx=0).stop 2024-06-10T03:01:41.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.324+0000 7fd4b2876b80 1 -- 172.21.15.52:0/1096128930 shutdown_connections 2024-06-10T03:01:41.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.324+0000 7fd4b2876b80 1 --2- 172.21.15.52:0/1096128930 >> [v2:172.21.15.120:6838/1672436487,v1:172.21.15.120:6839/1672436487] conn(0x55b7329c5310 0x55b7329c7700 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:41.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.324+0000 7fd4b2876b80 1 --2- 172.21.15.52:0/1096128930 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b7329469c0 0x55b732946de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:41.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.324+0000 7fd4b2876b80 1 --2- 172.21.15.52:0/1096128930 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b7329419f0 0x55b732960440 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:41.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.324+0000 7fd4897fa700 0 client.16572 ms_handle_reset on v2:172.21.15.120:6838/1672436487 2024-06-10T03:01:41.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.324+0000 7fd4b2876b80 1 --2- 172.21.15.52:0/1096128930 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b7329460a0 0x55b732960980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:41.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.324+0000 7fd4b2876b80 1 -- 172.21.15.52:0/1096128930 >> 172.21.15.52:0/1096128930 conn(0x55b73286e080 msgr2=0x55b732956740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:01:41.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.324+0000 7fd4b2876b80 1 -- 172.21.15.52:0/1096128930 shutdown_connections 2024-06-10T03:01:41.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.324+0000 7fd4b2876b80 1 -- 172.21.15.52:0/1096128930 wait complete. 2024-06-10T03:01:41.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.324+0000 7fd4b2876b80 20 client.16572 trim_cache size 0 max 16384 2024-06-10T03:01:41.329 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-06-10T03:01:41.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.325+0000 7fd4a6d91700 1 -- 172.21.15.52:0/957300453 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fd48c0a2da0 msgr2=0x7fd48c0a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:01:41.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.325+0000 7fd4a6d91700 1 --2- 172.21.15.52:0/957300453 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fd48c0a2da0 0x7fd48c0a5260 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fd490006fd0 tx=0x7fd490008040 comp rx=0 tx=0).stop 2024-06-10T03:01:41.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.325+0000 7fd4a6d91700 1 -- 172.21.15.52:0/957300453 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd498094650 msgr2=0x7fd4980a4ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:01:41.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.325+0000 7fd4a6d91700 1 --2- 172.21.15.52:0/957300453 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd498094650 0x7fd4980a4ba0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fd49c00c920 tx=0x7fd49c00cce0 comp rx=0 tx=0).stop 2024-06-10T03:01:41.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.325+0000 7fd4a6d91700 1 -- 172.21.15.52:0/957300453 shutdown_connections 2024-06-10T03:01:41.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.325+0000 7fd4a6d91700 1 --2- 172.21.15.52:0/957300453 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd4980a6ae0 0x7fd49809e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:41.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.325+0000 7fd4a6d91700 1 --2- 172.21.15.52:0/957300453 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fd48c0a2da0 0x7fd48c0a5260 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:41.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.325+0000 7fd4a6d91700 1 --2- 172.21.15.52:0/957300453 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd498094650 0x7fd4980a4ba0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:41.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.325+0000 7fd4a6d91700 1 --2- 172.21.15.52:0/957300453 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd498094f70 0x7fd49809dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:41.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.325+0000 7fd4a6d91700 1 -- 172.21.15.52:0/957300453 >> 172.21.15.52:0/957300453 conn(0x7fd498009f10 msgr2=0x7fd498091630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:01:41.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.325+0000 7fd4a6d91700 1 -- 172.21.15.52:0/957300453 shutdown_connections 2024-06-10T03:01:41.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:41.325+0000 7fd4a6d91700 1 -- 172.21.15.52:0/957300453 wait complete. 2024-06-10T03:01:42.032 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-06-10T03:01:42.377 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T03:01:42.703 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:01:42 smithi188 ceph-mon[36599]: pgmap v659: 129 pgs: 129 active+clean; 626 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.6 MiB/s wr, 401 op/s 2024-06-10T03:01:42.726 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:01:42 smithi052 ceph-mon[32362]: pgmap v659: 129 pgs: 129 active+clean; 626 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.6 MiB/s wr, 401 op/s 2024-06-10T03:01:42.888 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:01:42 smithi120 ceph-mon[36634]: pgmap v659: 129 pgs: 129 active+clean; 626 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.6 MiB/s wr, 401 op/s 2024-06-10T03:01:44.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.435+0000 7f85bc92c700 1 -- 172.21.15.52:0/1666183508 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f85b8108500 msgr2=0x7f85b81088e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:01:44.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.435+0000 7f85bc92c700 1 --2- 172.21.15.52:0/1666183508 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f85b8108500 0x7f85b81088e0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f85a8009990 tx=0x7f85a8009ca0 comp rx=0 tx=0).stop 2024-06-10T03:01:44.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.436+0000 7f85bc92c700 1 -- 172.21.15.52:0/1666183508 shutdown_connections 2024-06-10T03:01:44.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.436+0000 7f85bc92c700 1 --2- 172.21.15.52:0/1666183508 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f85b806ab30 0x7f85b810f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:44.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.436+0000 7f85bc92c700 1 --2- 172.21.15.52:0/1666183508 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f85b8108500 0x7f85b81088e0 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:44.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.436+0000 7f85bc92c700 1 --2- 172.21.15.52:0/1666183508 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f85b806a170 0x7f85b806a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:44.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.436+0000 7f85bc92c700 1 -- 172.21.15.52:0/1666183508 >> 172.21.15.52:0/1666183508 conn(0x7f85b8078340 msgr2=0x7f85b8078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:01:44.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.436+0000 7f85bc92c700 1 -- 172.21.15.52:0/1666183508 shutdown_connections 2024-06-10T03:01:44.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.436+0000 7f85bc92c700 1 -- 172.21.15.52:0/1666183508 wait complete. 2024-06-10T03:01:44.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.436+0000 7f85bc92c700 1 Processor -- start 2024-06-10T03:01:44.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.437+0000 7f85bc92c700 1 -- start start 2024-06-10T03:01:44.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.437+0000 7f85bc92c700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f85b806a170 0x7f85b819e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:01:44.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.437+0000 7f85bc92c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f85b806ab30 0x7f85b819edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:01:44.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.437+0000 7f85bc92c700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f85b8108500 0x7f85b8198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:01:44.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.437+0000 7f85bc92c700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f85b8112780 con 0x7f85b806ab30 2024-06-10T03:01:44.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.437+0000 7f85bc92c700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f85b8112600 con 0x7f85b806a170 2024-06-10T03:01:44.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.437+0000 7f85bc92c700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f85b8112900 con 0x7f85b8108500 2024-06-10T03:01:44.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.437+0000 7f85b5d9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f85b806ab30 0x7f85b819edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:01:44.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.437+0000 7f85b5d9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f85b806ab30 0x7f85b819edd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:35772/0 (socket says 172.21.15.52:35772) 2024-06-10T03:01:44.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.437+0000 7f85b5d9b700 1 -- 172.21.15.52:0/3249878233 learned_addr learned my addr 172.21.15.52:0/3249878233 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:01:44.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.437+0000 7f85b6d9d700 1 --2- 172.21.15.52:0/3249878233 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f85b8108500 0x7f85b8198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:01:44.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.437+0000 7f85b659c700 1 --2- 172.21.15.52:0/3249878233 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f85b806a170 0x7f85b819e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:01:44.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.437+0000 7f85b5d9b700 1 -- 172.21.15.52:0/3249878233 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f85b8108500 msgr2=0x7f85b8198aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:01:44.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.437+0000 7f85b5d9b700 1 --2- 172.21.15.52:0/3249878233 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f85b8108500 0x7f85b8198aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:44.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.437+0000 7f85b5d9b700 1 -- 172.21.15.52:0/3249878233 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f85b806a170 msgr2=0x7f85b819e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:01:44.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.437+0000 7f85b5d9b700 1 --2- 172.21.15.52:0/3249878233 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f85b806a170 0x7f85b819e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:44.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.437+0000 7f85b5d9b700 1 -- 172.21.15.52:0/3249878233 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85b8199200 con 0x7f85b806ab30 2024-06-10T03:01:44.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.437+0000 7f85b659c700 1 --2- 172.21.15.52:0/3249878233 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f85b806a170 0x7f85b819e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T03:01:44.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.437+0000 7f85b6d9d700 1 --2- 172.21.15.52:0/3249878233 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f85b8108500 0x7f85b8198aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T03:01:44.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.437+0000 7f85b5d9b700 1 --2- 172.21.15.52:0/3249878233 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f85b806ab30 0x7f85b819edd0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f85ac00e9c0 tx=0x7f85ac00ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:01:44.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.438+0000 7f85a77fe700 1 -- 172.21.15.52:0/3249878233 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85ac00cbd0 con 0x7f85b806ab30 2024-06-10T03:01:44.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.438+0000 7f85bc92c700 1 -- 172.21.15.52:0/3249878233 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85b81994f0 con 0x7f85b806ab30 2024-06-10T03:01:44.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.438+0000 7f85a77fe700 1 -- 172.21.15.52:0/3249878233 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85ac00ee50 con 0x7f85b806ab30 2024-06-10T03:01:44.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.438+0000 7f85a77fe700 1 -- 172.21.15.52:0/3249878233 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85ac005680 con 0x7f85b806ab30 2024-06-10T03:01:44.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.438+0000 7f85bc92c700 1 -- 172.21.15.52:0/3249878233 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85b81a5760 con 0x7f85b806ab30 2024-06-10T03:01:44.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.440+0000 7f85a77fe700 1 -- 172.21.15.52:0/3249878233 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f85ac004750 con 0x7f85b806ab30 2024-06-10T03:01:44.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.442+0000 7f85a77fe700 1 --2- 172.21.15.52:0/3249878233 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f85a00a2c20 0x7f85a00a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:01:44.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.442+0000 7f85b659c700 1 --2- 172.21.15.52:0/3249878233 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f85a00a2c20 0x7f85a00a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:01:44.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.442+0000 7f85a77fe700 1 -- 172.21.15.52:0/3249878233 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f85ac0cf6a0 con 0x7f85b806ab30 2024-06-10T03:01:44.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.442+0000 7f85c71afb80 1 Processor -- start 2024-06-10T03:01:44.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.442+0000 7f85b659c700 1 --2- 172.21.15.52:0/3249878233 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f85a00a2c20 0x7f85a00a50e0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f85a800b2e0 tx=0x7f85a8005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:01:44.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.442+0000 7f85c71afb80 1 -- start start 2024-06-10T03:01:44.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.443+0000 7f85c71afb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cd0fd19640 0x55cd0fd19a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:01:44.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.443+0000 7f85c71afb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd0fd19f60 0x55cd0fd139e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:01:44.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.443+0000 7f85c71afb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cd0fd13f20 0x55cd0fd14340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:01:44.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.443+0000 7f85c71afb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55cd0fc65810 con 0x55cd0fd19f60 2024-06-10T03:01:44.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.443+0000 7f85c71afb80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55cd0fc0d720 con 0x55cd0fd19640 2024-06-10T03:01:44.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.443+0000 7f85c71afb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55cd0fc617d0 con 0x55cd0fd13f20 2024-06-10T03:01:44.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.443+0000 7f85b5d9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd0fd19f60 0x55cd0fd139e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:01:44.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.443+0000 7f85b5d9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd0fd19f60 0x55cd0fd139e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:35786/0 (socket says 172.21.15.52:35786) 2024-06-10T03:01:44.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.443+0000 7f85b5d9b700 1 -- 172.21.15.52:0/3544250198 learned_addr learned my addr 172.21.15.52:0/3544250198 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:01:44.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.443+0000 7f85b659c700 1 --2- 172.21.15.52:0/3544250198 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cd0fd19640 0x55cd0fd19a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:01:44.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.443+0000 7f85b6d9d700 1 --2- 172.21.15.52:0/3544250198 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cd0fd13f20 0x55cd0fd14340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:01:44.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.443+0000 7f85b5d9b700 1 -- 172.21.15.52:0/3544250198 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cd0fd13f20 msgr2=0x55cd0fd14340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:01:44.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.443+0000 7f85b5d9b700 1 --2- 172.21.15.52:0/3544250198 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cd0fd13f20 0x55cd0fd14340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:44.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.443+0000 7f85b5d9b700 1 -- 172.21.15.52:0/3544250198 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cd0fd19640 msgr2=0x55cd0fd19a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:01:44.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.443+0000 7f85b5d9b700 1 --2- 172.21.15.52:0/3544250198 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cd0fd19640 0x55cd0fd19a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:44.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.443+0000 7f85b5d9b700 1 -- 172.21.15.52:0/3544250198 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd0fc03f50 con 0x55cd0fd19f60 2024-06-10T03:01:44.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.443+0000 7f85b659c700 1 --2- 172.21.15.52:0/3544250198 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cd0fd19640 0x55cd0fd19a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T03:01:44.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.443+0000 7f85b5d9b700 1 --2- 172.21.15.52:0/3544250198 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd0fd19f60 0x55cd0fd139e0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f85ac09ba80 tx=0x7f85ac09b520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7cf3bbd2d7c0703 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:01:44.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.444+0000 7f85a4ff9700 1 -- 172.21.15.52:0/3544250198 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85ac09fea0 con 0x55cd0fd19f60 2024-06-10T03:01:44.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.444+0000 7f85a4ff9700 1 -- 172.21.15.52:0/3544250198 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85ac0a4760 con 0x55cd0fd19f60 2024-06-10T03:01:44.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.444+0000 7f85a4ff9700 1 -- 172.21.15.52:0/3544250198 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85ac0a42d0 con 0x55cd0fd19f60 2024-06-10T03:01:44.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.444+0000 7f85c71afb80 1 -- 172.21.15.52:0/3544250198 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd0fd19f60 msgr2=0x55cd0fd139e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:01:44.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.444+0000 7f85c71afb80 1 --2- 172.21.15.52:0/3544250198 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd0fd19f60 0x55cd0fd139e0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f85ac09ba80 tx=0x7f85ac09b520 comp rx=0 tx=0).stop 2024-06-10T03:01:44.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.444+0000 7f85c71afb80 1 -- 172.21.15.52:0/3544250198 shutdown_connections 2024-06-10T03:01:44.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.444+0000 7f85c71afb80 1 --2- 172.21.15.52:0/3544250198 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cd0fd13f20 0x55cd0fd14340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:44.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.444+0000 7f85c71afb80 1 --2- 172.21.15.52:0/3544250198 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cd0fd19640 0x55cd0fd19a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:44.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.444+0000 7f85c71afb80 1 --2- 172.21.15.52:0/3544250198 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd0fd19f60 0x55cd0fd139e0 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:44.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.444+0000 7f85c71afb80 1 -- 172.21.15.52:0/3544250198 >> 172.21.15.52:0/3544250198 conn(0x55cd0fc3ecc0 msgr2=0x55cd0fd0e850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:01:44.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.445+0000 7f85c71afb80 1 -- 172.21.15.52:0/3544250198 shutdown_connections 2024-06-10T03:01:44.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.445+0000 7f85c71afb80 1 -- 172.21.15.52:0/3544250198 wait complete. 2024-06-10T03:01:44.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.445+0000 7f85c71afb80 1 -- start start 2024-06-10T03:01:44.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.445+0000 7f85c71afb80 1 Processor -- start 2024-06-10T03:01:44.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.445+0000 7f85c71afb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd0fd13f20 0x55cd0fd1fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:01:44.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.445+0000 7f85c71afb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cd0fd19640 0x55cd0fd1d000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:01:44.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.445+0000 7f85c71afb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cd0fd19f60 0x55cd0fd1d540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:01:44.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.445+0000 7f85c71afb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55cd0fd209d0 con 0x55cd0fd13f20 2024-06-10T03:01:44.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.445+0000 7f85c71afb80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55cd0fc05bf0 con 0x55cd0fd19640 2024-06-10T03:01:44.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.445+0000 7f85c71afb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55cd0fd20b50 con 0x55cd0fd19f60 2024-06-10T03:01:44.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.445+0000 7f85c71afb80 10 client.0 _ll_register_callbacks cb 0x55cd0fbff920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T03:01:44.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.446+0000 7f85b659c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd0fd13f20 0x55cd0fd1fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:01:44.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.446+0000 7f85b6d9d700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cd0fd19f60 0x55cd0fd1d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:01:44.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.446+0000 7f85b659c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd0fd13f20 0x55cd0fd1fd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:35792/0 (socket says 172.21.15.52:35792) 2024-06-10T03:01:44.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.446+0000 7f85b659c700 1 -- 172.21.15.52:0/1524531196 learned_addr learned my addr 172.21.15.52:0/1524531196 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:01:44.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.446+0000 7f85b6d9d700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cd0fd19f60 0x55cd0fd1d540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.52:50490/0 (socket says 172.21.15.52:50490) 2024-06-10T03:01:44.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.446+0000 7f85b5d9b700 1 --2- 172.21.15.52:0/1524531196 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cd0fd19640 0x55cd0fd1d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:01:44.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.446+0000 7f85b659c700 1 -- 172.21.15.52:0/1524531196 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cd0fd19f60 msgr2=0x55cd0fd1d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:01:44.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.446+0000 7f85b659c700 1 --2- 172.21.15.52:0/1524531196 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cd0fd19f60 0x55cd0fd1d540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:44.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.446+0000 7f85b659c700 1 -- 172.21.15.52:0/1524531196 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cd0fd19640 msgr2=0x55cd0fd1d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:01:44.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.446+0000 7f85b659c700 1 --2- 172.21.15.52:0/1524531196 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cd0fd19640 0x55cd0fd1d000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:44.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.446+0000 7f85b659c700 1 -- 172.21.15.52:0/1524531196 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd0fbb48d0 con 0x55cd0fd13f20 2024-06-10T03:01:44.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.446+0000 7f85b6d9d700 1 --2- 172.21.15.52:0/1524531196 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cd0fd19f60 0x55cd0fd1d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T03:01:44.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.446+0000 7f85b5d9b700 1 --2- 172.21.15.52:0/1524531196 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cd0fd19640 0x55cd0fd1d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T03:01:44.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.446+0000 7f85b659c700 1 --2- 172.21.15.52:0/1524531196 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd0fd13f20 0x55cd0fd1fd00 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f85a8009990 tx=0x7f85a8020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f037d3b00192e95 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:01:44.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.446+0000 7f859dffb700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-10T03:01:44.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.446+0000 7f859dffb700 1 -- 172.21.15.52:0/1524531196 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85a800f870 con 0x55cd0fd13f20 2024-06-10T03:01:44.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.446+0000 7f859dffb700 1 -- 172.21.15.52:0/1524531196 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f85a800fa10 con 0x55cd0fd13f20 2024-06-10T03:01:44.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.446+0000 7f85c71afb80 1 -- 172.21.15.52:0/1524531196 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f85ac0a4760 con 0x55cd0fd13f20 2024-06-10T03:01:44.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.446+0000 7f859dffb700 1 -- 172.21.15.52:0/1524531196 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f85a800fc10 con 0x55cd0fd13f20 2024-06-10T03:01:44.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.447+0000 7f85c71afb80 10 client.16584 fetch_fsmap learned FSMap version 36 2024-06-10T03:01:44.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.447+0000 7f85c71afb80 1 -- 172.21.15.52:0/1524531196 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55cd0fd2b300 con 0x55cd0fd13f20 2024-06-10T03:01:44.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.447+0000 7f859dffb700 1 -- 172.21.15.52:0/1524531196 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85ac0a4760 con 0x55cd0fd13f20 2024-06-10T03:01:44.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.447+0000 7f859dffb700 1 -- 172.21.15.52:0/1524531196 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 36) ==== 8378+0+0 (secure 0 0 0) 0x7f85a8024410 con 0x55cd0fd13f20 2024-06-10T03:01:44.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.447+0000 7f85c71afb80 10 client.16584 fetch_fsmap finished waiting for FSMap version 36 2024-06-10T03:01:44.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.447+0000 7f85c71afb80 10 client.16584 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-06-10T03:01:44.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.447+0000 7f85c71afb80 20 client.16584 populate_metadata read hostname 'smithi052' 2024-06-10T03:01:44.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.447+0000 7f85c71afb80 1 --2- 172.21.15.52:0/1524531196 >> [v2:172.21.15.188:6834/3904544259,v1:172.21.15.188:6835/3904544259] conn(0x55cd0fd2d990 0x55cd0fd2fd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:01:44.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.447+0000 7f85c71afb80 4 client.16584 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T03:01:44.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.447+0000 7f85c71afb80 1 -- 172.21.15.52:0/1524531196 --> [v2:172.21.15.188:6834/3904544259,v1:172.21.15.188:6835/3904544259] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cd0fc72d10 con 0x55cd0fd2d990 2024-06-10T03:01:44.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.447+0000 7f85b5d9b700 1 --2- 172.21.15.52:0/1524531196 >> [v2:172.21.15.188:6834/3904544259,v1:172.21.15.188:6835/3904544259] conn(0x55cd0fd2d990 0x55cd0fd2fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:01:44.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.448+0000 7f85b5d9b700 1 --2- 172.21.15.52:0/1524531196 >> [v2:172.21.15.188:6834/3904544259,v1:172.21.15.188:6835/3904544259] conn(0x55cd0fd2d990 0x55cd0fd2fd80 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cb91a31ade93581b server_cookie=cede9837343f8995 in_seq=0 out_seq=0 2024-06-10T03:01:44.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.448+0000 7f859dffb700 10 client.16584 ms_handle_connect on v2:172.21.15.188:6834/3904544259 2024-06-10T03:01:44.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.448+0000 7f859dffb700 1 -- 172.21.15.52:0/1524531196 <== mds.2 v2:172.21.15.188:6834/3904544259 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55cd0fc72d10 con 0x55cd0fd2d990 2024-06-10T03:01:44.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.448+0000 7f859dffb700 10 client.16584 handle_command_reply: tid=0 2024-06-10T03:01:44.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.448+0000 7f85c71afb80 1 client.16584 shutdown 2024-06-10T03:01:44.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.448+0000 7f85c71afb80 1 -- 172.21.15.52:0/1524531196 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd0fd13f20 msgr2=0x55cd0fd1fd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:01:44.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.448+0000 7f85c71afb80 1 --2- 172.21.15.52:0/1524531196 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd0fd13f20 0x55cd0fd1fd00 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f85a8009990 tx=0x7f85a8020000 comp rx=0 tx=0).stop 2024-06-10T03:01:44.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.449+0000 7f85c71afb80 1 -- 172.21.15.52:0/1524531196 shutdown_connections 2024-06-10T03:01:44.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.449+0000 7f85c71afb80 1 --2- 172.21.15.52:0/1524531196 >> [v2:172.21.15.188:6834/3904544259,v1:172.21.15.188:6835/3904544259] conn(0x55cd0fd2d990 0x55cd0fd2fd80 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:44.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.449+0000 7f85c71afb80 1 --2- 172.21.15.52:0/1524531196 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cd0fd19f60 0x55cd0fd1d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:44.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.449+0000 7f859dffb700 0 client.16584 ms_handle_reset on v2:172.21.15.188:6834/3904544259 2024-06-10T03:01:44.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.449+0000 7f85c71afb80 1 --2- 172.21.15.52:0/1524531196 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd0fd13f20 0x55cd0fd1fd00 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:44.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.449+0000 7f85c71afb80 1 --2- 172.21.15.52:0/1524531196 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cd0fd19640 0x55cd0fd1d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:44.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.449+0000 7f85c71afb80 1 -- 172.21.15.52:0/1524531196 >> 172.21.15.52:0/1524531196 conn(0x55cd0fc3ecc0 msgr2=0x55cd0fd0e5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:01:44.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.449+0000 7f85c71afb80 1 -- 172.21.15.52:0/1524531196 shutdown_connections 2024-06-10T03:01:44.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.449+0000 7f85c71afb80 1 -- 172.21.15.52:0/1524531196 wait complete. 2024-06-10T03:01:44.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.449+0000 7f85c71afb80 20 client.16584 trim_cache size 0 max 16384 2024-06-10T03:01:44.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.464+0000 7f85c71afb80 1 Processor -- start 2024-06-10T03:01:44.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.464+0000 7f85c71afb80 1 -- start start 2024-06-10T03:01:44.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.464+0000 7f85c71afb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cd0fd13f20 0x55cd0fd139e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:01:44.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.464+0000 7f85c71afb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd0fd19640 0x55cd0fd1aa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:01:44.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.464+0000 7f85c71afb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cd0fd19f60 0x55cd0fd35650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:01:44.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.464+0000 7f85c71afb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f85b81ae300 con 0x55cd0fd19640 2024-06-10T03:01:44.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.464+0000 7f85c71afb80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f85b81aff00 con 0x55cd0fd19f60 2024-06-10T03:01:44.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.464+0000 7f85c71afb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f85b81ae9d0 con 0x55cd0fd13f20 2024-06-10T03:01:44.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.465+0000 7f85b5d9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd0fd19640 0x55cd0fd1aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:01:44.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.465+0000 7f85b5d9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd0fd19640 0x55cd0fd1aa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:35808/0 (socket says 172.21.15.52:35808) 2024-06-10T03:01:44.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.465+0000 7f85b5d9b700 1 -- 172.21.15.52:0/4281880817 learned_addr learned my addr 172.21.15.52:0/4281880817 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:01:44.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.465+0000 7f85b659c700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cd0fd13f20 0x55cd0fd139e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:01:44.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.465+0000 7f85b6d9d700 1 --2- 172.21.15.52:0/4281880817 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cd0fd19f60 0x55cd0fd35650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:01:44.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.465+0000 7f85b5d9b700 1 -- 172.21.15.52:0/4281880817 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cd0fd13f20 msgr2=0x55cd0fd139e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:01:44.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.465+0000 7f85b5d9b700 1 --2- 172.21.15.52:0/4281880817 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cd0fd13f20 0x55cd0fd139e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:44.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.465+0000 7f85b5d9b700 1 -- 172.21.15.52:0/4281880817 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cd0fd19f60 msgr2=0x55cd0fd35650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:01:44.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.465+0000 7f85b5d9b700 1 --2- 172.21.15.52:0/4281880817 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cd0fd19f60 0x55cd0fd35650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:44.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.465+0000 7f85b5d9b700 1 -- 172.21.15.52:0/4281880817 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd0fd30400 con 0x55cd0fd19640 2024-06-10T03:01:44.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.465+0000 7f85b5d9b700 1 --2- 172.21.15.52:0/4281880817 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd0fd19640 0x55cd0fd1aa90 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f85ac0984c0 tx=0x7f85ac00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e03737051144d3c server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:01:44.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.465+0000 7f859ffff700 1 -- 172.21.15.52:0/4281880817 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85ac0a4760 con 0x55cd0fd19640 2024-06-10T03:01:44.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.465+0000 7f859ffff700 1 -- 172.21.15.52:0/4281880817 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85ac00cd70 con 0x55cd0fd19640 2024-06-10T03:01:44.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.465+0000 7f859ffff700 1 -- 172.21.15.52:0/4281880817 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85ac09bd50 con 0x55cd0fd19640 2024-06-10T03:01:44.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.466+0000 7f85c71afb80 1 -- 172.21.15.52:0/4281880817 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd0fd19640 msgr2=0x55cd0fd1aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:01:44.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.466+0000 7f85c71afb80 1 --2- 172.21.15.52:0/4281880817 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd0fd19640 0x55cd0fd1aa90 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f85ac0984c0 tx=0x7f85ac00cf90 comp rx=0 tx=0).stop 2024-06-10T03:01:44.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.466+0000 7f85c71afb80 1 -- 172.21.15.52:0/4281880817 shutdown_connections 2024-06-10T03:01:44.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.466+0000 7f85c71afb80 1 --2- 172.21.15.52:0/4281880817 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cd0fd19f60 0x55cd0fd35650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:44.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.466+0000 7f85c71afb80 1 --2- 172.21.15.52:0/4281880817 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cd0fd13f20 0x55cd0fd139e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:44.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.466+0000 7f85c71afb80 1 --2- 172.21.15.52:0/4281880817 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd0fd19640 0x55cd0fd1aa90 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:44.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.466+0000 7f85c71afb80 1 -- 172.21.15.52:0/4281880817 >> 172.21.15.52:0/4281880817 conn(0x55cd0fc3ecc0 msgr2=0x55cd0fd1a380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:01:44.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.466+0000 7f85c71afb80 1 -- 172.21.15.52:0/4281880817 shutdown_connections 2024-06-10T03:01:44.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.466+0000 7f85c71afb80 1 -- 172.21.15.52:0/4281880817 wait complete. 2024-06-10T03:01:44.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.466+0000 7f85c71afb80 1 -- start start 2024-06-10T03:01:44.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.467+0000 7f85c71afb80 1 Processor -- start 2024-06-10T03:01:44.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.467+0000 7f85c71afb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cd0fd13f20 0x55cd0fd260c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:01:44.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.467+0000 7f85c71afb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd0fd19640 0x55cd0fd26600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:01:44.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.467+0000 7f85c71afb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cd0fd19f60 0x55cd0fd34c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:01:44.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.467+0000 7f85c71afb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f85b81af830 con 0x55cd0fd19640 2024-06-10T03:01:44.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.467+0000 7f85c71afb80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f85b81af160 con 0x55cd0fd13f20 2024-06-10T03:01:44.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.467+0000 7f85c71afb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f85b81ac2a0 con 0x55cd0fd19f60 2024-06-10T03:01:44.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.467+0000 7f85c71afb80 10 client.0 _ll_register_callbacks cb 0x55cd0fbff920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T03:01:44.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.467+0000 7f85b5d9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd0fd19640 0x55cd0fd26600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:01:44.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.467+0000 7f85b5d9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd0fd19640 0x55cd0fd26600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:35818/0 (socket says 172.21.15.52:35818) 2024-06-10T03:01:44.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.467+0000 7f85b5d9b700 1 -- 172.21.15.52:0/2186908365 learned_addr learned my addr 172.21.15.52:0/2186908365 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:01:44.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.467+0000 7f85b6d9d700 1 --2- 172.21.15.52:0/2186908365 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cd0fd19f60 0x55cd0fd34c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:01:44.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.467+0000 7f85b659c700 1 --2- 172.21.15.52:0/2186908365 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cd0fd13f20 0x55cd0fd260c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:01:44.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.467+0000 7f85b6d9d700 1 -- 172.21.15.52:0/2186908365 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cd0fd13f20 msgr2=0x55cd0fd260c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:01:44.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.467+0000 7f85b6d9d700 1 --2- 172.21.15.52:0/2186908365 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cd0fd13f20 0x55cd0fd260c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:44.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.467+0000 7f85b6d9d700 1 -- 172.21.15.52:0/2186908365 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd0fd19640 msgr2=0x55cd0fd26600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:01:44.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.467+0000 7f85b6d9d700 1 --2- 172.21.15.52:0/2186908365 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd0fd19640 0x55cd0fd26600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:44.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.467+0000 7f85b6d9d700 1 -- 172.21.15.52:0/2186908365 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd0fd354c0 con 0x55cd0fd19f60 2024-06-10T03:01:44.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.467+0000 7f85b5d9b700 1 --2- 172.21.15.52:0/2186908365 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd0fd19640 0x55cd0fd26600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T03:01:44.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.467+0000 7f85b659c700 1 --2- 172.21.15.52:0/2186908365 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cd0fd13f20 0x55cd0fd260c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T03:01:44.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.468+0000 7f85b6d9d700 1 --2- 172.21.15.52:0/2186908365 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cd0fd19f60 0x55cd0fd34c80 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x55cd0fd2a790 tx=0x7f85b000e7d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7a1a769ed65b6c74 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:01:44.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.468+0000 7f859dffb700 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-06-10T03:01:44.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.468+0000 7f859dffb700 1 -- 172.21.15.52:0/2186908365 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85b0004db0 con 0x55cd0fd19f60 2024-06-10T03:01:44.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.468+0000 7f85c71afb80 1 -- 172.21.15.52:0/2186908365 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f85ac00cd70 con 0x55cd0fd19f60 2024-06-10T03:01:44.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.468+0000 7f859dffb700 1 -- 172.21.15.52:0/2186908365 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f85b0005020 con 0x55cd0fd19f60 2024-06-10T03:01:44.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.468+0000 7f859dffb700 1 -- 172.21.15.52:0/2186908365 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f85b0005220 con 0x55cd0fd19f60 2024-06-10T03:01:44.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.468+0000 7f85c71afb80 10 client.26137 fetch_fsmap learned FSMap version 36 2024-06-10T03:01:44.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.468+0000 7f85c71afb80 1 -- 172.21.15.52:0/2186908365 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55cd0fd982a0 con 0x55cd0fd19f60 2024-06-10T03:01:44.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.468+0000 7f859dffb700 1 -- 172.21.15.52:0/2186908365 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85ac00cd70 con 0x55cd0fd19f60 2024-06-10T03:01:44.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.468+0000 7f859dffb700 1 -- 172.21.15.52:0/2186908365 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 36) ==== 8378+0+0 (secure 0 0 0) 0x7f85b001e070 con 0x55cd0fd19f60 2024-06-10T03:01:44.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.468+0000 7f85c71afb80 10 client.26137 fetch_fsmap finished waiting for FSMap version 36 2024-06-10T03:01:44.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.468+0000 7f85c71afb80 10 client.26137 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-06-10T03:01:44.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.468+0000 7f85c71afb80 20 client.26137 populate_metadata read hostname 'smithi052' 2024-06-10T03:01:44.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.469+0000 7f85c71afb80 1 --2- 172.21.15.52:0/2186908365 >> [v2:172.21.15.188:6834/3904544259,v1:172.21.15.188:6835/3904544259] conn(0x55cd0fd98ad0 0x55cd0fd9aef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:01:44.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.469+0000 7f85c71afb80 4 client.26137 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T03:01:44.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.469+0000 7f85c71afb80 1 -- 172.21.15.52:0/2186908365 --> [v2:172.21.15.188:6834/3904544259,v1:172.21.15.188:6835/3904544259] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55cd0fc6d290 con 0x55cd0fd98ad0 2024-06-10T03:01:44.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.469+0000 7f85b659c700 1 --2- 172.21.15.52:0/2186908365 >> [v2:172.21.15.188:6834/3904544259,v1:172.21.15.188:6835/3904544259] conn(0x55cd0fd98ad0 0x55cd0fd9aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:01:44.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.469+0000 7f85b659c700 1 --2- 172.21.15.52:0/2186908365 >> [v2:172.21.15.188:6834/3904544259,v1:172.21.15.188:6835/3904544259] conn(0x55cd0fd98ad0 0x55cd0fd9aef0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6d8e4dd37cf77242 server_cookie=182b26a1113de7fb in_seq=0 out_seq=0 2024-06-10T03:01:44.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.469+0000 7f859dffb700 10 client.26137 ms_handle_connect on v2:172.21.15.188:6834/3904544259 2024-06-10T03:01:44.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.471+0000 7f859dffb700 1 -- 172.21.15.52:0/2186908365 <== mds.2 v2:172.21.15.188:6834/3904544259 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55cd0fc6d290 con 0x55cd0fd98ad0 2024-06-10T03:01:44.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.471+0000 7f859dffb700 10 client.26137 handle_command_reply: tid=0 2024-06-10T03:01:44.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.471+0000 7f85c71afb80 1 client.26137 shutdown 2024-06-10T03:01:44.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.471+0000 7f85c71afb80 1 -- 172.21.15.52:0/2186908365 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cd0fd19f60 msgr2=0x55cd0fd34c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:01:44.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.471+0000 7f85c71afb80 1 --2- 172.21.15.52:0/2186908365 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cd0fd19f60 0x55cd0fd34c80 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x55cd0fd2a790 tx=0x7f85b000e7d0 comp rx=0 tx=0).stop 2024-06-10T03:01:44.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.471+0000 7f85c71afb80 1 -- 172.21.15.52:0/2186908365 shutdown_connections 2024-06-10T03:01:44.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.471+0000 7f85c71afb80 1 --2- 172.21.15.52:0/2186908365 >> [v2:172.21.15.188:6834/3904544259,v1:172.21.15.188:6835/3904544259] conn(0x55cd0fd98ad0 0x55cd0fd9aef0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:44.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.471+0000 7f85c71afb80 1 --2- 172.21.15.52:0/2186908365 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cd0fd19f60 0x55cd0fd34c80 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:44.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.471+0000 7f85c71afb80 1 --2- 172.21.15.52:0/2186908365 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cd0fd13f20 0x55cd0fd260c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:44.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.471+0000 7f85c71afb80 1 --2- 172.21.15.52:0/2186908365 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd0fd19640 0x55cd0fd26600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:44.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.471+0000 7f859dffb700 0 client.26137 ms_handle_reset on v2:172.21.15.188:6834/3904544259 2024-06-10T03:01:44.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.471+0000 7f85c71afb80 1 -- 172.21.15.52:0/2186908365 >> 172.21.15.52:0/2186908365 conn(0x55cd0fc3ecc0 msgr2=0x55cd0fd1a380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:01:44.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.471+0000 7f85c71afb80 1 -- 172.21.15.52:0/2186908365 shutdown_connections 2024-06-10T03:01:44.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.471+0000 7f85c71afb80 1 -- 172.21.15.52:0/2186908365 wait complete. 2024-06-10T03:01:44.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.471+0000 7f85c71afb80 20 client.26137 trim_cache size 0 max 16384 2024-06-10T03:01:44.479 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-06-10T03:01:44.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.472+0000 7f85bc92c700 1 -- 172.21.15.52:0/3249878233 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f85a00a2c20 msgr2=0x7f85a00a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:01:44.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.472+0000 7f85bc92c700 1 --2- 172.21.15.52:0/3249878233 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f85a00a2c20 0x7f85a00a50e0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f85a800b2e0 tx=0x7f85a8005960 comp rx=0 tx=0).stop 2024-06-10T03:01:44.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.472+0000 7f85bc92c700 1 -- 172.21.15.52:0/3249878233 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f85b806ab30 msgr2=0x7f85b819edd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:01:44.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.472+0000 7f85bc92c700 1 --2- 172.21.15.52:0/3249878233 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f85b806ab30 0x7f85b819edd0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f85ac00e9c0 tx=0x7f85ac00ecd0 comp rx=0 tx=0).stop 2024-06-10T03:01:44.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.472+0000 7f85bc92c700 1 -- 172.21.15.52:0/3249878233 shutdown_connections 2024-06-10T03:01:44.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.472+0000 7f85bc92c700 1 --2- 172.21.15.52:0/3249878233 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f85b8108500 0x7f85b8198aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:44.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.472+0000 7f85bc92c700 1 --2- 172.21.15.52:0/3249878233 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f85b806a170 0x7f85b819e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:44.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.472+0000 7f85bc92c700 1 --2- 172.21.15.52:0/3249878233 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f85a00a2c20 0x7f85a00a50e0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:44.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.472+0000 7f85bc92c700 1 --2- 172.21.15.52:0/3249878233 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f85b806ab30 0x7f85b819edd0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:44.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.472+0000 7f85bc92c700 1 -- 172.21.15.52:0/3249878233 >> 172.21.15.52:0/3249878233 conn(0x7f85b8078340 msgr2=0x7f85b8100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:01:44.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.472+0000 7f85bc92c700 1 -- 172.21.15.52:0/3249878233 shutdown_connections 2024-06-10T03:01:44.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:44.472+0000 7f85bc92c700 1 -- 172.21.15.52:0/3249878233 wait complete. 2024-06-10T03:01:44.703 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:01:44 smithi188 ceph-mon[36599]: pgmap v660: 129 pgs: 129 active+clean; 623 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.9 MiB/s wr, 542 op/s 2024-06-10T03:01:44.727 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:01:44 smithi052 ceph-mon[32362]: pgmap v660: 129 pgs: 129 active+clean; 623 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.9 MiB/s wr, 542 op/s 2024-06-10T03:01:44.888 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:01:44 smithi120 ceph-mon[36634]: pgmap v660: 129 pgs: 129 active+clean; 623 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.9 MiB/s wr, 542 op/s 2024-06-10T03:01:45.168 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-06-10T03:01:45.494 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T03:01:46.703 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:01:46 smithi188 ceph-mon[36599]: pgmap v661: 129 pgs: 129 active+clean; 618 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.6 MiB/s wr, 540 op/s 2024-06-10T03:01:46.727 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:01:46 smithi052 ceph-mon[32362]: pgmap v661: 129 pgs: 129 active+clean; 618 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.6 MiB/s wr, 540 op/s 2024-06-10T03:01:46.888 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:01:46 smithi120 ceph-mon[36634]: pgmap v661: 129 pgs: 129 active+clean; 618 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.6 MiB/s wr, 540 op/s 2024-06-10T03:01:47.420 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.419+0000 7fe5336d2700 1 -- 172.21.15.52:0/4125093790 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe52c10a6d0 msgr2=0x7fe52c10aab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:01:47.420 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.419+0000 7fe5336d2700 1 --2- 172.21.15.52:0/4125093790 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe52c10a6d0 0x7fe52c10aab0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fe52401c0d0 tx=0x7fe52401c3e0 comp rx=0 tx=0).stop 2024-06-10T03:01:47.421 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.420+0000 7fe5336d2700 1 -- 172.21.15.52:0/4125093790 shutdown_connections 2024-06-10T03:01:47.421 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.420+0000 7fe5336d2700 1 --2- 172.21.15.52:0/4125093790 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe52c105090 0x7fe52c1115a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:47.421 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.420+0000 7fe5336d2700 1 --2- 172.21.15.52:0/4125093790 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe52c10a6d0 0x7fe52c10aab0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:47.421 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.420+0000 7fe5336d2700 1 --2- 172.21.15.52:0/4125093790 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe52c1046d0 0x7fe52c104b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:47.421 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.420+0000 7fe5336d2700 1 -- 172.21.15.52:0/4125093790 >> 172.21.15.52:0/4125093790 conn(0x7fe52c1001c0 msgr2=0x7fe52c1025e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:01:47.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.422+0000 7fe5336d2700 1 -- 172.21.15.52:0/4125093790 shutdown_connections 2024-06-10T03:01:47.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.424+0000 7fe5336d2700 1 -- 172.21.15.52:0/4125093790 wait complete. 2024-06-10T03:01:47.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.425+0000 7fe5336d2700 1 Processor -- start 2024-06-10T03:01:47.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.427+0000 7fe5336d2700 1 -- start start 2024-06-10T03:01:47.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.427+0000 7fe5336d2700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe52c1046d0 0x7fe52c073990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:01:47.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.428+0000 7fe5336d2700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe52c105090 0x7fe52c073ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:01:47.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.428+0000 7fe53146e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe52c1046d0 0x7fe52c073990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:01:47.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.428+0000 7fe53146e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe52c1046d0 0x7fe52c073990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:35830/0 (socket says 172.21.15.52:35830) 2024-06-10T03:01:47.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.428+0000 7fe5336d2700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe52c10a6d0 0x7fe52c06beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:01:47.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.428+0000 7fe53146e700 1 -- 172.21.15.52:0/4173851160 learned_addr learned my addr 172.21.15.52:0/4173851160 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:01:47.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.428+0000 7fe5336d2700 1 -- 172.21.15.52:0/4173851160 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fe52c079240 con 0x7fe52c1046d0 2024-06-10T03:01:47.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.428+0000 7fe5336d2700 1 -- 172.21.15.52:0/4173851160 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe52c0790c0 con 0x7fe52c105090 2024-06-10T03:01:47.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.428+0000 7fe5336d2700 1 -- 172.21.15.52:0/4173851160 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fe52c0793c0 con 0x7fe52c10a6d0 2024-06-10T03:01:47.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.428+0000 7fe530c6d700 1 --2- 172.21.15.52:0/4173851160 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe52c105090 0x7fe52c073ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:01:47.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.429+0000 7fe530c6d700 1 -- 172.21.15.52:0/4173851160 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe52c10a6d0 msgr2=0x7fe52c06beb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-10T03:01:47.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.429+0000 7fe530c6d700 1 --2- 172.21.15.52:0/4173851160 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe52c10a6d0 0x7fe52c06beb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:47.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.429+0000 7fe530c6d700 1 -- 172.21.15.52:0/4173851160 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe52c1046d0 msgr2=0x7fe52c073990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:01:47.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.429+0000 7fe530c6d700 1 --2- 172.21.15.52:0/4173851160 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe52c1046d0 0x7fe52c073990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:47.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.429+0000 7fe530c6d700 1 -- 172.21.15.52:0/4173851160 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe52c06c780 con 0x7fe52c105090 2024-06-10T03:01:47.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.430+0000 7fe530c6d700 1 --2- 172.21.15.52:0/4173851160 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe52c105090 0x7fe52c073ed0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fe528006fd0 tx=0x7fe52800b730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:01:47.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.431+0000 7fe5227fc700 1 -- 172.21.15.52:0/4173851160 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe528015020 con 0x7fe52c105090 2024-06-10T03:01:47.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.431+0000 7fe5336d2700 1 -- 172.21.15.52:0/4173851160 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe52c06ca70 con 0x7fe52c105090 2024-06-10T03:01:47.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.431+0000 7fe5336d2700 1 -- 172.21.15.52:0/4173851160 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe52c19b090 con 0x7fe52c105090 2024-06-10T03:01:47.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.431+0000 7fe5227fc700 1 -- 172.21.15.52:0/4173851160 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe528011d10 con 0x7fe52c105090 2024-06-10T03:01:47.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.431+0000 7fe5227fc700 1 -- 172.21.15.52:0/4173851160 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe528010e30 con 0x7fe52c105090 2024-06-10T03:01:47.441 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.437+0000 7fe53df55b80 1 Processor -- start 2024-06-10T03:01:47.441 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.437+0000 7fe5227fc700 1 -- 172.21.15.52:0/4173851160 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fe528018020 con 0x7fe52c105090 2024-06-10T03:01:47.445 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.442+0000 7fe5227fc700 1 --2- 172.21.15.52:0/4173851160 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fe5180a2d60 0x7fe5180a5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:01:47.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.443+0000 7fe5227fc700 1 -- 172.21.15.52:0/4173851160 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7fe5280cee10 con 0x7fe52c105090 2024-06-10T03:01:47.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.443+0000 7fe53df55b80 1 -- start start 2024-06-10T03:01:47.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.443+0000 7fe53df55b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a3f93adf60 0x55a3f93ae340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:01:47.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.443+0000 7fe53df55b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a3f93ae880 0x55a3f93a8260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:01:47.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.443+0000 7fe53df55b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a3f93a87a0 0x55a3f93a8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:01:47.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.443+0000 7fe53df55b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55a3f929af60 con 0x55a3f93a87a0 2024-06-10T03:01:47.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.443+0000 7fe53df55b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55a3f92f9850 con 0x55a3f93adf60 2024-06-10T03:01:47.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.443+0000 7fe53df55b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55a3f9261460 con 0x55a3f93ae880 2024-06-10T03:01:47.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.445+0000 7fe53146e700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a3f93adf60 0x55a3f93ae340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:01:47.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.445+0000 7fe530c6d700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a3f93ae880 0x55a3f93a8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:01:47.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.445+0000 7fe53146e700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a3f93adf60 0x55a3f93ae340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.52:48272/0 (socket says 172.21.15.52:48272) 2024-06-10T03:01:47.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.445+0000 7fe53146e700 1 -- 172.21.15.52:0/2067648396 learned_addr learned my addr 172.21.15.52:0/2067648396 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:01:47.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.445+0000 7fe531c6f700 1 --2- 172.21.15.52:0/2067648396 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a3f93a87a0 0x55a3f93a8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:01:47.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.445+0000 7fe531c6f700 1 --2- 172.21.15.52:0/4173851160 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fe5180a2d60 0x7fe5180a5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:01:47.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.446+0000 7fe530c6d700 1 -- 172.21.15.52:0/2067648396 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a3f93adf60 msgr2=0x55a3f93ae340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:01:47.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.446+0000 7fe530c6d700 1 --2- 172.21.15.52:0/2067648396 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a3f93adf60 0x55a3f93ae340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:47.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.446+0000 7fe530c6d700 1 -- 172.21.15.52:0/2067648396 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a3f93a87a0 msgr2=0x55a3f93a8bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:01:47.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.446+0000 7fe530c6d700 1 --2- 172.21.15.52:0/2067648396 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a3f93a87a0 0x55a3f93a8bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:47.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.446+0000 7fe530c6d700 1 -- 172.21.15.52:0/2067648396 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a3f92488d0 con 0x55a3f93ae880 2024-06-10T03:01:47.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.446+0000 7fe530c6d700 1 --2- 172.21.15.52:0/2067648396 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a3f93ae880 0x55a3f93a8260 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fe5280cc560 tx=0x7fe52809b920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5e919b23587b6a5d server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:01:47.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.446+0000 7fe5177fe700 1 -- 172.21.15.52:0/2067648396 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5280b4070 con 0x55a3f93ae880 2024-06-10T03:01:47.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.447+0000 7fe5177fe700 1 -- 172.21.15.52:0/2067648396 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe52809bf40 con 0x55a3f93ae880 2024-06-10T03:01:47.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.447+0000 7fe5177fe700 1 -- 172.21.15.52:0/2067648396 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5280bb480 con 0x55a3f93ae880 2024-06-10T03:01:47.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.448+0000 7fe53df55b80 1 -- 172.21.15.52:0/2067648396 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a3f93ae880 msgr2=0x55a3f93a8260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:01:47.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.448+0000 7fe53df55b80 1 --2- 172.21.15.52:0/2067648396 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a3f93ae880 0x55a3f93a8260 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fe5280cc560 tx=0x7fe52809b920 comp rx=0 tx=0).stop 2024-06-10T03:01:47.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.448+0000 7fe531c6f700 1 --2- 172.21.15.52:0/4173851160 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fe5180a2d60 0x7fe5180a5220 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fe51c00d950 tx=0x7fe51c006940 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:01:47.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.448+0000 7fe53df55b80 1 -- 172.21.15.52:0/2067648396 shutdown_connections 2024-06-10T03:01:47.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.448+0000 7fe53df55b80 1 --2- 172.21.15.52:0/2067648396 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a3f93a87a0 0x55a3f93a8bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:47.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.448+0000 7fe53df55b80 1 --2- 172.21.15.52:0/2067648396 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a3f93adf60 0x55a3f93ae340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:47.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.448+0000 7fe53df55b80 1 --2- 172.21.15.52:0/2067648396 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a3f93ae880 0x55a3f93a8260 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:47.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.448+0000 7fe53df55b80 1 -- 172.21.15.52:0/2067648396 >> 172.21.15.52:0/2067648396 conn(0x55a3f92d5080 msgr2=0x55a3f93a3220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:01:47.454 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.454+0000 7fe53df55b80 1 -- 172.21.15.52:0/2067648396 shutdown_connections 2024-06-10T03:01:47.454 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.454+0000 7fe53df55b80 1 -- 172.21.15.52:0/2067648396 wait complete. 2024-06-10T03:01:47.454 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.454+0000 7fe53df55b80 1 -- start start 2024-06-10T03:01:47.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.454+0000 7fe53df55b80 1 Processor -- start 2024-06-10T03:01:47.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.455+0000 7fe53df55b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a3f93a87a0 0x55a3f93b4350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:01:47.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.455+0000 7fe53df55b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a3f93adf60 0x55a3f93b4890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:01:47.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.455+0000 7fe53df55b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a3f93b1800 0x55a3f93b1c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:01:47.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.455+0000 7fe53df55b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55a3f9299fc0 con 0x55a3f93b1800 2024-06-10T03:01:47.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.455+0000 7fe53df55b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55a3f92f5810 con 0x55a3f93a87a0 2024-06-10T03:01:47.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.455+0000 7fe53df55b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55a3f93b5260 con 0x55a3f93adf60 2024-06-10T03:01:47.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.455+0000 7fe53df55b80 10 client.0 _ll_register_callbacks cb 0x55a3f928f5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T03:01:47.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.455+0000 7fe530c6d700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a3f93adf60 0x55a3f93b4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:01:47.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.455+0000 7fe530c6d700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a3f93adf60 0x55a3f93b4890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.52:50530/0 (socket says 172.21.15.52:50530) 2024-06-10T03:01:47.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.455+0000 7fe530c6d700 1 -- 172.21.15.52:0/1862188600 learned_addr learned my addr 172.21.15.52:0/1862188600 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:01:47.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.456+0000 7fe53146e700 1 --2- 172.21.15.52:0/1862188600 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a3f93a87a0 0x55a3f93b4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:01:47.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.456+0000 7fe531c6f700 1 --2- 172.21.15.52:0/1862188600 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a3f93b1800 0x55a3f93b1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:01:47.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.456+0000 7fe53146e700 1 -- 172.21.15.52:0/1862188600 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a3f93adf60 msgr2=0x55a3f93b4890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:01:47.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.456+0000 7fe53146e700 1 --2- 172.21.15.52:0/1862188600 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a3f93adf60 0x55a3f93b4890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:47.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.456+0000 7fe53146e700 1 -- 172.21.15.52:0/1862188600 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a3f93b1800 msgr2=0x55a3f93b1c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:01:47.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.456+0000 7fe53146e700 1 --2- 172.21.15.52:0/1862188600 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a3f93b1800 0x55a3f93b1c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:47.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.457+0000 7fe53146e700 1 -- 172.21.15.52:0/1862188600 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a3f92bf750 con 0x55a3f93a87a0 2024-06-10T03:01:47.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.461+0000 7fe53146e700 1 --2- 172.21.15.52:0/1862188600 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a3f93a87a0 0x55a3f93b4350 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fe524003d70 tx=0x7fe524003da0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=76b06b0915c64d97 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:01:47.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.463+0000 7fe514ff9700 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-06-10T03:01:47.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.468+0000 7fe514ff9700 1 -- 172.21.15.52:0/1862188600 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5240189c0 con 0x55a3f93a87a0 2024-06-10T03:01:47.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.468+0000 7fe514ff9700 1 -- 172.21.15.52:0/1862188600 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7fe524018b60 con 0x55a3f93a87a0 2024-06-10T03:01:47.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.470+0000 7fe53df55b80 1 -- 172.21.15.52:0/1862188600 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe52809bf40 con 0x55a3f93a87a0 2024-06-10T03:01:47.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.471+0000 7fe514ff9700 1 -- 172.21.15.52:0/1862188600 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fe524018d60 con 0x55a3f93a87a0 2024-06-10T03:01:47.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.475+0000 7fe53df55b80 10 client.26108 fetch_fsmap learned FSMap version 37 2024-06-10T03:01:47.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.475+0000 7fe53df55b80 1 -- 172.21.15.52:0/1862188600 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55a3f93bfb30 con 0x55a3f93a87a0 2024-06-10T03:01:47.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.478+0000 7fe514ff9700 1 -- 172.21.15.52:0/1862188600 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe524042070 con 0x55a3f93a87a0 2024-06-10T03:01:47.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.478+0000 7fe514ff9700 1 -- 172.21.15.52:0/1862188600 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 37) ==== 8382+0+0 (secure 0 0 0) 0x7fe524022070 con 0x55a3f93a87a0 2024-06-10T03:01:47.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.479+0000 7fe53df55b80 10 client.26108 fetch_fsmap finished waiting for FSMap version 37 2024-06-10T03:01:47.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.479+0000 7fe53df55b80 10 client.26108 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-06-10T03:01:47.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.479+0000 7fe53df55b80 20 client.26108 populate_metadata read hostname 'smithi052' 2024-06-10T03:01:47.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.479+0000 7fe53df55b80 1 --2- 172.21.15.52:0/1862188600 >> [v2:172.21.15.188:6836/819317726,v1:172.21.15.188:6837/819317726] conn(0x55a3f93c2170 0x55a3f93c4560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:01:47.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.479+0000 7fe53df55b80 4 client.26108 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T03:01:47.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.479+0000 7fe53df55b80 1 -- 172.21.15.52:0/1862188600 --> [v2:172.21.15.188:6836/819317726,v1:172.21.15.188:6837/819317726] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fe52809bf40 con 0x55a3f93c2170 2024-06-10T03:01:47.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.480+0000 7fe530c6d700 1 --2- 172.21.15.52:0/1862188600 >> [v2:172.21.15.188:6836/819317726,v1:172.21.15.188:6837/819317726] conn(0x55a3f93c2170 0x55a3f93c4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:01:47.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.484+0000 7fe530c6d700 1 --2- 172.21.15.52:0/1862188600 >> [v2:172.21.15.188:6836/819317726,v1:172.21.15.188:6837/819317726] conn(0x55a3f93c2170 0x55a3f93c4560 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6322cc3df69471db server_cookie=5d546ed4af206ee7 in_seq=0 out_seq=0 2024-06-10T03:01:47.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.484+0000 7fe514ff9700 10 client.26108 ms_handle_connect on v2:172.21.15.188:6836/819317726 2024-06-10T03:01:47.486 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.486+0000 7fe514ff9700 1 -- 172.21.15.52:0/1862188600 <== mds.1 v2:172.21.15.188:6836/819317726 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x7fe52809bf40 con 0x55a3f93c2170 2024-06-10T03:01:47.487 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.486+0000 7fe514ff9700 10 client.26108 handle_command_reply: tid=0 2024-06-10T03:01:47.487 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.487+0000 7fe53df55b80 1 client.26108 shutdown 2024-06-10T03:01:47.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.489+0000 7fe53df55b80 1 -- 172.21.15.52:0/1862188600 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a3f93a87a0 msgr2=0x55a3f93b4350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:01:47.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.489+0000 7fe53df55b80 1 --2- 172.21.15.52:0/1862188600 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a3f93a87a0 0x55a3f93b4350 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fe524003d70 tx=0x7fe524003da0 comp rx=0 tx=0).stop 2024-06-10T03:01:47.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.490+0000 7fe53df55b80 1 -- 172.21.15.52:0/1862188600 shutdown_connections 2024-06-10T03:01:47.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.490+0000 7fe53df55b80 1 --2- 172.21.15.52:0/1862188600 >> [v2:172.21.15.188:6836/819317726,v1:172.21.15.188:6837/819317726] conn(0x55a3f93c2170 0x55a3f93c4560 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:47.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.490+0000 7fe53df55b80 1 --2- 172.21.15.52:0/1862188600 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a3f93b1800 0x55a3f93b1c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:47.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.490+0000 7fe53df55b80 1 --2- 172.21.15.52:0/1862188600 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a3f93a87a0 0x55a3f93b4350 secure :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fe524003d70 tx=0x7fe524003da0 comp rx=0 tx=0).stop 2024-06-10T03:01:47.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.490+0000 7fe53df55b80 1 --2- 172.21.15.52:0/1862188600 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a3f93adf60 0x55a3f93b4890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:47.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.490+0000 7fe53df55b80 1 -- 172.21.15.52:0/1862188600 >> 172.21.15.52:0/1862188600 conn(0x55a3f92d5080 msgr2=0x55a3f93a1e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:01:47.491 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.490+0000 7fe514ff9700 0 client.26108 ms_handle_reset on v2:172.21.15.188:6836/819317726 2024-06-10T03:01:47.491 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.492+0000 7fe53df55b80 1 -- 172.21.15.52:0/1862188600 shutdown_connections 2024-06-10T03:01:47.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.493+0000 7fe53df55b80 1 -- 172.21.15.52:0/1862188600 wait complete. 2024-06-10T03:01:47.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.493+0000 7fe53df55b80 20 client.26108 trim_cache size 0 max 16384 2024-06-10T03:01:47.524 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.524+0000 7fe53df55b80 1 Processor -- start 2024-06-10T03:01:47.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.525+0000 7fe53df55b80 1 -- start start 2024-06-10T03:01:47.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.525+0000 7fe53df55b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a3f93adf60 0x55a3f93be3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:01:47.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.525+0000 7fe53df55b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a3f93b1800 0x55a3f93affb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:01:47.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.525+0000 7fe53df55b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a3f93ad770 0x55a3f93b04f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:01:47.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.526+0000 7fe53df55b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fe51000ec70 con 0x55a3f93adf60 2024-06-10T03:01:47.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.526+0000 7fe53df55b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe5100108a0 con 0x55a3f93ad770 2024-06-10T03:01:47.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.526+0000 7fe53146e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a3f93adf60 0x55a3f93be3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:01:47.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.526+0000 7fe531c6f700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a3f93ad770 0x55a3f93b04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:01:47.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.526+0000 7fe53146e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a3f93adf60 0x55a3f93be3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:35846/0 (socket says 172.21.15.52:35846) 2024-06-10T03:01:47.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.526+0000 7fe531c6f700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a3f93ad770 0x55a3f93b04f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.52:48278/0 (socket says 172.21.15.52:48278) 2024-06-10T03:01:47.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.526+0000 7fe53146e700 1 -- 172.21.15.52:0/720078177 learned_addr learned my addr 172.21.15.52:0/720078177 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:01:47.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.526+0000 7fe53df55b80 1 -- 172.21.15.52:0/720078177 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fe51000f340 con 0x55a3f93b1800 2024-06-10T03:01:47.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.526+0000 7fe530c6d700 1 --2- 172.21.15.52:0/720078177 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a3f93b1800 0x55a3f93affb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:01:47.533 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.533+0000 7fe53146e700 1 -- 172.21.15.52:0/720078177 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a3f93b1800 msgr2=0x55a3f93affb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:01:47.534 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.533+0000 7fe53146e700 1 --2- 172.21.15.52:0/720078177 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a3f93b1800 0x55a3f93affb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:47.534 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.533+0000 7fe53146e700 1 -- 172.21.15.52:0/720078177 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a3f93ad770 msgr2=0x55a3f93b04f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:01:47.534 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.533+0000 7fe53146e700 1 --2- 172.21.15.52:0/720078177 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a3f93ad770 0x55a3f93b04f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:47.534 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.533+0000 7fe53146e700 1 -- 172.21.15.52:0/720078177 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a3f93b2280 con 0x55a3f93adf60 2024-06-10T03:01:47.534 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.534+0000 7fe53146e700 1 --2- 172.21.15.52:0/720078177 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a3f93adf60 0x55a3f93be3b0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fe524050960 tx=0x7fe524050650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe266dc7214e32f server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:01:47.534 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.534+0000 7fe516ffd700 1 -- 172.21.15.52:0/720078177 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe524054070 con 0x55a3f93adf60 2024-06-10T03:01:47.535 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.535+0000 7fe516ffd700 1 -- 172.21.15.52:0/720078177 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe524042040 con 0x55a3f93adf60 2024-06-10T03:01:47.535 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.535+0000 7fe516ffd700 1 -- 172.21.15.52:0/720078177 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5240054b0 con 0x55a3f93adf60 2024-06-10T03:01:47.535 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.536+0000 7fe53df55b80 1 -- 172.21.15.52:0/720078177 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a3f93adf60 msgr2=0x55a3f93be3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:01:47.536 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.536+0000 7fe53df55b80 1 --2- 172.21.15.52:0/720078177 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a3f93adf60 0x55a3f93be3b0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fe524050960 tx=0x7fe524050650 comp rx=0 tx=0).stop 2024-06-10T03:01:47.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.539+0000 7fe53df55b80 1 -- 172.21.15.52:0/720078177 shutdown_connections 2024-06-10T03:01:47.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.539+0000 7fe53df55b80 1 --2- 172.21.15.52:0/720078177 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a3f93ad770 0x55a3f93b04f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:47.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.539+0000 7fe53df55b80 1 --2- 172.21.15.52:0/720078177 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a3f93adf60 0x55a3f93be3b0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:47.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.539+0000 7fe53df55b80 1 --2- 172.21.15.52:0/720078177 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a3f93b1800 0x55a3f93affb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:47.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.539+0000 7fe53df55b80 1 -- 172.21.15.52:0/720078177 >> 172.21.15.52:0/720078177 conn(0x55a3f92d5080 msgr2=0x55a3f93c4180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:01:47.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.540+0000 7fe53df55b80 1 -- 172.21.15.52:0/720078177 shutdown_connections 2024-06-10T03:01:47.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.540+0000 7fe53df55b80 1 -- 172.21.15.52:0/720078177 wait complete. 2024-06-10T03:01:47.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.541+0000 7fe53df55b80 1 -- start start 2024-06-10T03:01:47.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.541+0000 7fe53df55b80 1 Processor -- start 2024-06-10T03:01:47.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.543+0000 7fe53df55b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a3f93ad770 0x55a3f93ba8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:01:47.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.543+0000 7fe53df55b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a3f93adf60 0x55a3f93bae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:01:47.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.543+0000 7fe53df55b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a3f93b1800 0x55a3f93b7f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:01:47.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.543+0000 7fe53df55b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fe5100101d0 con 0x55a3f93b1800 2024-06-10T03:01:47.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.543+0000 7fe53df55b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe51000fad0 con 0x55a3f93adf60 2024-06-10T03:01:47.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.543+0000 7fe53df55b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fe51000cc10 con 0x55a3f93ad770 2024-06-10T03:01:47.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.543+0000 7fe53df55b80 10 client.0 _ll_register_callbacks cb 0x55a3f928f5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T03:01:47.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.543+0000 7fe530c6d700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a3f93adf60 0x55a3f93bae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:01:47.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.543+0000 7fe530c6d700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a3f93adf60 0x55a3f93bae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.52:48290/0 (socket says 172.21.15.52:48290) 2024-06-10T03:01:47.546 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.543+0000 7fe530c6d700 1 -- 172.21.15.52:0/915872835 learned_addr learned my addr 172.21.15.52:0/915872835 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:01:47.546 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.544+0000 7fe53146e700 1 --2- 172.21.15.52:0/915872835 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a3f93ad770 0x55a3f93ba8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:01:47.546 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.544+0000 7fe530c6d700 1 -- 172.21.15.52:0/915872835 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a3f93ad770 msgr2=0x55a3f93ba8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:01:47.546 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.544+0000 7fe530c6d700 1 --2- 172.21.15.52:0/915872835 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a3f93ad770 0x55a3f93ba8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:47.546 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.544+0000 7fe530c6d700 1 -- 172.21.15.52:0/915872835 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a3f93b1800 msgr2=0x55a3f93b7f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:01:47.546 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.544+0000 7fe530c6d700 1 --2- 172.21.15.52:0/915872835 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a3f93b1800 0x55a3f93b7f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:47.546 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.544+0000 7fe530c6d700 1 -- 172.21.15.52:0/915872835 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a3f93b8750 con 0x55a3f93adf60 2024-06-10T03:01:47.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.545+0000 7fe530c6d700 1 --2- 172.21.15.52:0/915872835 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a3f93adf60 0x55a3f93bae30 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fe52809bc40 tx=0x7fe528006dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c2566b1fe5cf6d6f server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:01:47.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.545+0000 7fe514ff9700 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-06-10T03:01:47.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.545+0000 7fe514ff9700 1 -- 172.21.15.52:0/915872835 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5280b4040 con 0x55a3f93adf60 2024-06-10T03:01:47.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.546+0000 7fe514ff9700 1 -- 172.21.15.52:0/915872835 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7fe5280b00c0 con 0x55a3f93adf60 2024-06-10T03:01:47.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.546+0000 7fe53df55b80 1 -- 172.21.15.52:0/915872835 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe524042040 con 0x55a3f93adf60 2024-06-10T03:01:47.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.546+0000 7fe514ff9700 1 -- 172.21.15.52:0/915872835 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fe52809bf40 con 0x55a3f93adf60 2024-06-10T03:01:47.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.546+0000 7fe53df55b80 10 client.26114 fetch_fsmap learned FSMap version 37 2024-06-10T03:01:47.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.546+0000 7fe53df55b80 1 -- 172.21.15.52:0/915872835 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55a3f942cac0 con 0x55a3f93adf60 2024-06-10T03:01:47.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.547+0000 7fe514ff9700 1 -- 172.21.15.52:0/915872835 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe52809a060 con 0x55a3f93adf60 2024-06-10T03:01:47.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.547+0000 7fe514ff9700 1 -- 172.21.15.52:0/915872835 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 37) ==== 8382+0+0 (secure 0 0 0) 0x7fe52809f070 con 0x55a3f93adf60 2024-06-10T03:01:47.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.547+0000 7fe53df55b80 10 client.26114 fetch_fsmap finished waiting for FSMap version 37 2024-06-10T03:01:47.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.547+0000 7fe53df55b80 10 client.26114 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-06-10T03:01:47.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.547+0000 7fe53df55b80 20 client.26114 populate_metadata read hostname 'smithi052' 2024-06-10T03:01:47.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.547+0000 7fe53df55b80 1 --2- 172.21.15.52:0/915872835 >> [v2:172.21.15.188:6836/819317726,v1:172.21.15.188:6837/819317726] conn(0x55a3f942cfc0 0x55a3f942f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:01:47.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.547+0000 7fe53df55b80 4 client.26114 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T03:01:47.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.547+0000 7fe53df55b80 1 -- 172.21.15.52:0/915872835 --> [v2:172.21.15.188:6836/819317726,v1:172.21.15.188:6837/819317726] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fe524042040 con 0x55a3f942cfc0 2024-06-10T03:01:47.550 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.549+0000 7fe53146e700 1 --2- 172.21.15.52:0/915872835 >> [v2:172.21.15.188:6836/819317726,v1:172.21.15.188:6837/819317726] conn(0x55a3f942cfc0 0x55a3f942f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:01:47.560 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.555+0000 7fe53146e700 1 --2- 172.21.15.52:0/915872835 >> [v2:172.21.15.188:6836/819317726,v1:172.21.15.188:6837/819317726] conn(0x55a3f942cfc0 0x55a3f942f3e0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4801459a76fd7e9e server_cookie=b940a81922b86f16 in_seq=0 out_seq=0 2024-06-10T03:01:47.560 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.555+0000 7fe514ff9700 10 client.26114 ms_handle_connect on v2:172.21.15.188:6836/819317726 2024-06-10T03:01:47.560 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.556+0000 7fe514ff9700 1 -- 172.21.15.52:0/915872835 <== mds.1 v2:172.21.15.188:6836/819317726 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fe524042040 con 0x55a3f942cfc0 2024-06-10T03:01:47.560 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.556+0000 7fe514ff9700 10 client.26114 handle_command_reply: tid=0 2024-06-10T03:01:47.560 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.556+0000 7fe53df55b80 1 client.26114 shutdown 2024-06-10T03:01:47.560 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.556+0000 7fe53df55b80 1 -- 172.21.15.52:0/915872835 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a3f93adf60 msgr2=0x55a3f93bae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:01:47.560 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.556+0000 7fe53df55b80 1 --2- 172.21.15.52:0/915872835 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a3f93adf60 0x55a3f93bae30 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fe52809bc40 tx=0x7fe528006dc0 comp rx=0 tx=0).stop 2024-06-10T03:01:47.560 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.556+0000 7fe53df55b80 1 -- 172.21.15.52:0/915872835 shutdown_connections 2024-06-10T03:01:47.561 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.556+0000 7fe53df55b80 1 --2- 172.21.15.52:0/915872835 >> [v2:172.21.15.188:6836/819317726,v1:172.21.15.188:6837/819317726] conn(0x55a3f942cfc0 0x55a3f942f3e0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:47.561 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.556+0000 7fe53df55b80 1 --2- 172.21.15.52:0/915872835 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a3f93b1800 0x55a3f93b7f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:47.561 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.556+0000 7fe53df55b80 1 --2- 172.21.15.52:0/915872835 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a3f93ad770 0x55a3f93ba8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:47.561 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.556+0000 7fe53df55b80 1 --2- 172.21.15.52:0/915872835 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a3f93adf60 0x55a3f93bae30 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:47.561 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.556+0000 7fe53df55b80 1 -- 172.21.15.52:0/915872835 >> 172.21.15.52:0/915872835 conn(0x55a3f92d5080 msgr2=0x55a3f93c4180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:01:47.561 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.556+0000 7fe514ff9700 0 client.26114 ms_handle_reset on v2:172.21.15.188:6836/819317726 2024-06-10T03:01:47.561 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.559+0000 7fe53df55b80 1 -- 172.21.15.52:0/915872835 shutdown_connections 2024-06-10T03:01:47.561 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.560+0000 7fe53df55b80 1 -- 172.21.15.52:0/915872835 wait complete. 2024-06-10T03:01:47.561 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.560+0000 7fe53df55b80 20 client.26114 trim_cache size 0 max 16384 2024-06-10T03:01:47.561 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-06-10T03:01:47.563 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.563+0000 7fe5336d2700 1 -- 172.21.15.52:0/4173851160 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fe5180a2d60 msgr2=0x7fe5180a5220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:01:47.563 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.563+0000 7fe5336d2700 1 --2- 172.21.15.52:0/4173851160 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fe5180a2d60 0x7fe5180a5220 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fe51c00d950 tx=0x7fe51c006940 comp rx=0 tx=0).stop 2024-06-10T03:01:47.563 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.563+0000 7fe5336d2700 1 -- 172.21.15.52:0/4173851160 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe52c105090 msgr2=0x7fe52c073ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:01:47.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.563+0000 7fe5336d2700 1 --2- 172.21.15.52:0/4173851160 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe52c105090 0x7fe52c073ed0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fe528006fd0 tx=0x7fe52800b730 comp rx=0 tx=0).stop 2024-06-10T03:01:47.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.564+0000 7fe5336d2700 1 -- 172.21.15.52:0/4173851160 shutdown_connections 2024-06-10T03:01:47.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.564+0000 7fe5336d2700 1 --2- 172.21.15.52:0/4173851160 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe52c10a6d0 0x7fe52c06beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:47.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.564+0000 7fe5336d2700 1 --2- 172.21.15.52:0/4173851160 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fe5180a2d60 0x7fe5180a5220 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:47.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.564+0000 7fe5336d2700 1 --2- 172.21.15.52:0/4173851160 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe52c1046d0 0x7fe52c073990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:47.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.564+0000 7fe5336d2700 1 --2- 172.21.15.52:0/4173851160 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe52c105090 0x7fe52c073ed0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:47.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.564+0000 7fe5336d2700 1 -- 172.21.15.52:0/4173851160 >> 172.21.15.52:0/4173851160 conn(0x7fe52c1001c0 msgr2=0x7fe52c079aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:01:47.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.565+0000 7fe5336d2700 1 -- 172.21.15.52:0/4173851160 shutdown_connections 2024-06-10T03:01:47.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:47.565+0000 7fe5336d2700 1 -- 172.21.15.52:0/4173851160 wait complete. 2024-06-10T03:01:47.590 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:01:47 smithi052 ceph-mon[32362]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-10T03:01:47.888 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:01:47 smithi120 ceph-mon[36634]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-10T03:01:47.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:01:47 smithi188 ceph-mon[36599]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-10T03:01:48.303 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-06-10T03:01:48.633 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T03:01:48.727 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:01:48 smithi052 ceph-mon[32362]: pgmap v662: 129 pgs: 129 active+clean; 592 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.1 MiB/s wr, 665 op/s 2024-06-10T03:01:48.888 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:01:48 smithi120 ceph-mon[36634]: pgmap v662: 129 pgs: 129 active+clean; 592 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.1 MiB/s wr, 665 op/s 2024-06-10T03:01:48.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:01:48 smithi188 ceph-mon[36599]: pgmap v662: 129 pgs: 129 active+clean; 592 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.1 MiB/s wr, 665 op/s 2024-06-10T03:01:50.727 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:01:50 smithi052 ceph-mon[32362]: pgmap v663: 129 pgs: 129 active+clean; 602 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.4 MiB/s wr, 852 op/s 2024-06-10T03:01:50.888 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:01:50 smithi120 ceph-mon[36634]: pgmap v663: 129 pgs: 129 active+clean; 602 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.4 MiB/s wr, 852 op/s 2024-06-10T03:01:50.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:01:50 smithi188 ceph-mon[36599]: pgmap v663: 129 pgs: 129 active+clean; 602 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.4 MiB/s wr, 852 op/s 2024-06-10T03:01:51.727 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:01:51 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T03:01:51.888 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:01:51 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T03:01:51.954 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:01:51 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T03:01:52.642 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:01:52 smithi052 ceph-mon[32362]: pgmap v664: 129 pgs: 129 active+clean; 614 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.6 MiB/s wr, 805 op/s 2024-06-10T03:01:52.888 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:01:52 smithi120 ceph-mon[36634]: pgmap v664: 129 pgs: 129 active+clean; 614 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.6 MiB/s wr, 805 op/s 2024-06-10T03:01:52.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:01:52 smithi188 ceph-mon[36599]: pgmap v664: 129 pgs: 129 active+clean; 614 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.6 MiB/s wr, 805 op/s 2024-06-10T03:01:53.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.178+0000 7f623005d700 1 -- 172.21.15.52:0/3280635605 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f622810a930 msgr2=0x7f622810dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:01:53.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.178+0000 7f623005d700 1 --2- 172.21.15.52:0/3280635605 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f622810a930 0x7f622810dfb0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f6218009f90 tx=0x7f621800b330 comp rx=0 tx=0).stop 2024-06-10T03:01:53.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.178+0000 7f623005d700 1 -- 172.21.15.52:0/3280635605 shutdown_connections 2024-06-10T03:01:53.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.178+0000 7f623005d700 1 --2- 172.21.15.52:0/3280635605 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f622810e690 0x7f6228116220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:53.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.178+0000 7f623005d700 1 --2- 172.21.15.52:0/3280635605 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6228109f80 0x7f622810a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:53.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.178+0000 7f623005d700 1 --2- 172.21.15.52:0/3280635605 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f622810a930 0x7f622810dfb0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:53.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.178+0000 7f623005d700 1 -- 172.21.15.52:0/3280635605 >> 172.21.15.52:0/3280635605 conn(0x7f622806d9e0 msgr2=0x7f622806ddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:01:53.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.178+0000 7f623005d700 1 -- 172.21.15.52:0/3280635605 shutdown_connections 2024-06-10T03:01:53.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.178+0000 7f623005d700 1 -- 172.21.15.52:0/3280635605 wait complete. 2024-06-10T03:01:53.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.179+0000 7f623005d700 1 Processor -- start 2024-06-10T03:01:53.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.179+0000 7f623005d700 1 -- start start 2024-06-10T03:01:53.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.180+0000 7f623005d700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6228109f80 0x7f62281ad770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:01:53.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.180+0000 7f623005d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f622810a930 0x7f62281adcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:01:53.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.180+0000 7f623005d700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f622810e690 0x7f62281a7930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:01:53.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.180+0000 7f623005d700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f6228118e40 con 0x7f622810a930 2024-06-10T03:01:53.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.180+0000 7f623005d700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f6228118cc0 con 0x7f622810e690 2024-06-10T03:01:53.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.180+0000 7f623005d700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f6228118fc0 con 0x7f6228109f80 2024-06-10T03:01:53.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.180+0000 7f622e5fa700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f622810e690 0x7f62281a7930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:01:53.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.180+0000 7f622e5fa700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f622810e690 0x7f62281a7930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.52:39972/0 (socket says 172.21.15.52:39972) 2024-06-10T03:01:53.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.180+0000 7f622e5fa700 1 -- 172.21.15.52:0/543869333 learned_addr learned my addr 172.21.15.52:0/543869333 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:01:53.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.180+0000 7f622ddf9700 1 --2- 172.21.15.52:0/543869333 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6228109f80 0x7f62281ad770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:01:53.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.181+0000 7f622d5f8700 1 --2- 172.21.15.52:0/543869333 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f622810a930 0x7f62281adcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:01:53.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.181+0000 7f622e5fa700 1 -- 172.21.15.52:0/543869333 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6228109f80 msgr2=0x7f62281ad770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:01:53.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.181+0000 7f622e5fa700 1 --2- 172.21.15.52:0/543869333 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6228109f80 0x7f62281ad770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:53.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.181+0000 7f622e5fa700 1 -- 172.21.15.52:0/543869333 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f622810a930 msgr2=0x7f62281adcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:01:53.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.181+0000 7f622e5fa700 1 --2- 172.21.15.52:0/543869333 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f622810a930 0x7f62281adcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:53.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.181+0000 7f622e5fa700 1 -- 172.21.15.52:0/543869333 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62281a8090 con 0x7f622810e690 2024-06-10T03:01:53.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.181+0000 7f622d5f8700 1 --2- 172.21.15.52:0/543869333 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f622810a930 0x7f62281adcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T03:01:53.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.182+0000 7f622e5fa700 1 --2- 172.21.15.52:0/543869333 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f622810e690 0x7f62281a7930 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f621c00e940 tx=0x7f621c00ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:01:53.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.182+0000 7f6226ffd700 1 -- 172.21.15.52:0/543869333 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f621c00cbd0 con 0x7f622810e690 2024-06-10T03:01:53.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.182+0000 7f6226ffd700 1 -- 172.21.15.52:0/543869333 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f621c00ee80 con 0x7f622810e690 2024-06-10T03:01:53.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.182+0000 7f623005d700 1 -- 172.21.15.52:0/543869333 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62281a8380 con 0x7f622810e690 2024-06-10T03:01:53.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.182+0000 7f623005d700 1 -- 172.21.15.52:0/543869333 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62281b4640 con 0x7f622810e690 2024-06-10T03:01:53.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.182+0000 7f6226ffd700 1 -- 172.21.15.52:0/543869333 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f621c0056c0 con 0x7f622810e690 2024-06-10T03:01:53.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.186+0000 7f6226ffd700 1 -- 172.21.15.52:0/543869333 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f621c004750 con 0x7f622810e690 2024-06-10T03:01:53.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.188+0000 7f623a8e0b80 1 Processor -- start 2024-06-10T03:01:53.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.188+0000 7f6226ffd700 1 --2- 172.21.15.52:0/543869333 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f62100a2d20 0x7f62100a51e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:01:53.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.189+0000 7f6226ffd700 1 -- 172.21.15.52:0/543869333 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f621c0cf5c0 con 0x7f622810e690 2024-06-10T03:01:53.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.189+0000 7f622d5f8700 1 --2- 172.21.15.52:0/543869333 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f62100a2d20 0x7f62100a51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:01:53.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.189+0000 7f623a8e0b80 1 -- start start 2024-06-10T03:01:53.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.189+0000 7f623a8e0b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557c7da29f60 0x557c7da2a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:01:53.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.189+0000 7f623a8e0b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557c7da2a880 0x557c7da24260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:01:53.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.189+0000 7f623a8e0b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557c7da247a0 0x557c7da24bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:01:53.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.190+0000 7f622d5f8700 1 --2- 172.21.15.52:0/543869333 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f62100a2d20 0x7f62100a51e0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f6218009da0 tx=0x7f62180094b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:01:53.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.190+0000 7f622d5f8700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557c7da2a880 0x557c7da24260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:01:53.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.190+0000 7f622d5f8700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557c7da2a880 0x557c7da24260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52488/0 (socket says 172.21.15.52:52488) 2024-06-10T03:01:53.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.190+0000 7f622d5f8700 1 -- 172.21.15.52:0/913286009 learned_addr learned my addr 172.21.15.52:0/913286009 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:01:53.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.190+0000 7f623a8e0b80 1 -- 172.21.15.52:0/913286009 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x557c7d916f60 con 0x557c7da2a880 2024-06-10T03:01:53.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.190+0000 7f623a8e0b80 1 -- 172.21.15.52:0/913286009 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x557c7d975850 con 0x557c7da29f60 2024-06-10T03:01:53.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.190+0000 7f623a8e0b80 1 -- 172.21.15.52:0/913286009 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x557c7d8dd460 con 0x557c7da247a0 2024-06-10T03:01:53.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.190+0000 7f622e5fa700 1 --2- 172.21.15.52:0/913286009 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557c7da247a0 0x557c7da24bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:01:53.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.191+0000 7f622e5fa700 1 -- 172.21.15.52:0/913286009 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557c7da29f60 msgr2=0x557c7da2a340 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-10T03:01:53.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.191+0000 7f622e5fa700 1 --2- 172.21.15.52:0/913286009 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557c7da29f60 0x557c7da2a340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:53.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.191+0000 7f622e5fa700 1 -- 172.21.15.52:0/913286009 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557c7da2a880 msgr2=0x557c7da24260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:01:53.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.191+0000 7f622e5fa700 1 --2- 172.21.15.52:0/913286009 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557c7da2a880 0x557c7da24260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:53.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.191+0000 7f622e5fa700 1 -- 172.21.15.52:0/913286009 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c7d8c48d0 con 0x557c7da247a0 2024-06-10T03:01:53.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.191+0000 7f622e5fa700 1 --2- 172.21.15.52:0/913286009 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557c7da247a0 0x557c7da24bc0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f621c09b9a0 tx=0x7f621c09b370 comp rx=0 tx=0).ready entity=mon.1 client_cookie=324416785f2edf31 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:01:53.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.191+0000 7f620ffff700 1 -- 172.21.15.52:0/913286009 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f621c0ce070 con 0x557c7da247a0 2024-06-10T03:01:53.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.191+0000 7f622d5f8700 1 --2- 172.21.15.52:0/913286009 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557c7da2a880 0x557c7da24260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T03:01:53.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.192+0000 7f620ffff700 1 -- 172.21.15.52:0/913286009 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f621c09fdc0 con 0x557c7da247a0 2024-06-10T03:01:53.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.193+0000 7f623a8e0b80 1 -- 172.21.15.52:0/913286009 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557c7da247a0 msgr2=0x557c7da24bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:01:53.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.193+0000 7f623a8e0b80 1 --2- 172.21.15.52:0/913286009 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557c7da247a0 0x557c7da24bc0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f621c09b9a0 tx=0x7f621c09b370 comp rx=0 tx=0).stop 2024-06-10T03:01:53.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.194+0000 7f623a8e0b80 1 -- 172.21.15.52:0/913286009 shutdown_connections 2024-06-10T03:01:53.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.194+0000 7f623a8e0b80 1 --2- 172.21.15.52:0/913286009 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557c7da247a0 0x557c7da24bc0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:53.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.194+0000 7f623a8e0b80 1 --2- 172.21.15.52:0/913286009 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557c7da29f60 0x557c7da2a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:53.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.194+0000 7f623a8e0b80 1 --2- 172.21.15.52:0/913286009 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557c7da2a880 0x557c7da24260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:53.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.194+0000 7f623a8e0b80 1 -- 172.21.15.52:0/913286009 >> 172.21.15.52:0/913286009 conn(0x557c7d951080 msgr2=0x557c7da1f220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:01:53.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.194+0000 7f623a8e0b80 1 -- 172.21.15.52:0/913286009 shutdown_connections 2024-06-10T03:01:53.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.195+0000 7f623a8e0b80 1 -- 172.21.15.52:0/913286009 wait complete. 2024-06-10T03:01:53.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.195+0000 7f623a8e0b80 1 -- start start 2024-06-10T03:01:53.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.195+0000 7f623a8e0b80 1 Processor -- start 2024-06-10T03:01:53.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.196+0000 7f623a8e0b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557c7da247a0 0x557c7da30650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:01:53.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.196+0000 7f623a8e0b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557c7da29f60 0x557c7da2d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:01:53.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.196+0000 7f623a8e0b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557c7da2a880 0x557c7da2de90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:01:53.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.196+0000 7f623a8e0b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x557c7d915fc0 con 0x557c7da2a880 2024-06-10T03:01:53.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.196+0000 7f623a8e0b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x557c7d971810 con 0x557c7da29f60 2024-06-10T03:01:53.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.196+0000 7f623a8e0b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x557c7da31200 con 0x557c7da247a0 2024-06-10T03:01:53.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.196+0000 7f623a8e0b80 10 client.0 _ll_register_callbacks cb 0x557c7d90b5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T03:01:53.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.197+0000 7f622e5fa700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557c7da2a880 0x557c7da2de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:01:53.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.197+0000 7f622e5fa700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557c7da2a880 0x557c7da2de90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52490/0 (socket says 172.21.15.52:52490) 2024-06-10T03:01:53.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.197+0000 7f622e5fa700 1 -- 172.21.15.52:0/3335104573 learned_addr learned my addr 172.21.15.52:0/3335104573 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:01:53.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.197+0000 7f622ddf9700 1 --2- 172.21.15.52:0/3335104573 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557c7da247a0 0x557c7da30650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:01:53.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.198+0000 7f622e5fa700 1 -- 172.21.15.52:0/3335104573 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557c7da247a0 msgr2=0x557c7da30650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:01:53.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.198+0000 7f622e5fa700 1 --2- 172.21.15.52:0/3335104573 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557c7da247a0 0x557c7da30650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:53.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.198+0000 7f622e5fa700 1 -- 172.21.15.52:0/3335104573 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557c7da29f60 msgr2=0x557c7da2d950 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-10T03:01:53.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.198+0000 7f622e5fa700 1 --2- 172.21.15.52:0/3335104573 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557c7da29f60 0x557c7da2d950 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:53.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.198+0000 7f622e5fa700 1 -- 172.21.15.52:0/3335104573 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c7d93b750 con 0x557c7da2a880 2024-06-10T03:01:53.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.198+0000 7f622e5fa700 1 --2- 172.21.15.52:0/3335104573 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557c7da2a880 0x557c7da2de90 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f621c0a4150 tx=0x7f621c0ccfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c2f9bed829006e7 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:01:53.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.198+0000 7f620d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-10T03:01:53.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.199+0000 7f620d7fa700 1 -- 172.21.15.52:0/3335104573 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f621c0ce070 con 0x557c7da2a880 2024-06-10T03:01:53.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.199+0000 7f620d7fa700 1 -- 172.21.15.52:0/3335104573 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f621c0a41f0 con 0x557c7da2a880 2024-06-10T03:01:53.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.201+0000 7f623a8e0b80 1 -- 172.21.15.52:0/3335104573 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f621c09fdc0 con 0x557c7da2a880 2024-06-10T03:01:53.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.203+0000 7f620d7fa700 1 -- 172.21.15.52:0/3335104573 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f621c0a4680 con 0x557c7da2a880 2024-06-10T03:01:53.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.204+0000 7f623a8e0b80 10 client.16611 fetch_fsmap learned FSMap version 37 2024-06-10T03:01:53.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.204+0000 7f623a8e0b80 1 -- 172.21.15.52:0/3335104573 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x557c7da3b6c0 con 0x557c7da2a880 2024-06-10T03:01:53.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.205+0000 7f620d7fa700 1 -- 172.21.15.52:0/3335104573 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f621c09fdc0 con 0x557c7da2a880 2024-06-10T03:01:53.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.205+0000 7f620d7fa700 1 -- 172.21.15.52:0/3335104573 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 37) ==== 8382+0+0 (secure 0 0 0) 0x7f621c097760 con 0x557c7da2a880 2024-06-10T03:01:53.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.205+0000 7f623a8e0b80 10 client.16611 fetch_fsmap finished waiting for FSMap version 37 2024-06-10T03:01:53.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.205+0000 7f623a8e0b80 10 client.16611 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T03:01:53.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.205+0000 7f623a8e0b80 20 client.16611 populate_metadata read hostname 'smithi052' 2024-06-10T03:01:53.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.205+0000 7f623a8e0b80 1 --2- 172.21.15.52:0/3335104573 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x557c7da3ddc0 0x557c7da401e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:01:53.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.205+0000 7f623a8e0b80 4 client.16611 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T03:01:53.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.205+0000 7f623a8e0b80 1 -- 172.21.15.52:0/3335104573 --> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557c7d982d50 con 0x557c7da3ddc0 2024-06-10T03:01:53.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.206+0000 7f622ddf9700 1 --2- 172.21.15.52:0/3335104573 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x557c7da3ddc0 0x557c7da401e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:01:53.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.208+0000 7f622ddf9700 1 --2- 172.21.15.52:0/3335104573 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x557c7da3ddc0 0x557c7da401e0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bcdf2be4cbfbae1f server_cookie=ceba1a29fd3272ae in_seq=0 out_seq=0 2024-06-10T03:01:53.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.208+0000 7f620d7fa700 10 client.16611 ms_handle_connect on v2:172.21.15.188:6838/310444836 2024-06-10T03:01:53.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.210+0000 7f620d7fa700 1 -- 172.21.15.52:0/3335104573 <== mds.0 v2:172.21.15.188:6838/310444836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x557c7d982d50 con 0x557c7da3ddc0 2024-06-10T03:01:53.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.210+0000 7f620d7fa700 10 client.16611 handle_command_reply: tid=0 2024-06-10T03:01:53.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.210+0000 7f623a8e0b80 1 client.16611 shutdown 2024-06-10T03:01:53.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.211+0000 7f623a8e0b80 1 -- 172.21.15.52:0/3335104573 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557c7da2a880 msgr2=0x557c7da2de90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:01:53.212 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.211+0000 7f623a8e0b80 1 --2- 172.21.15.52:0/3335104573 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557c7da2a880 0x557c7da2de90 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f621c0a4150 tx=0x7f621c0ccfd0 comp rx=0 tx=0).stop 2024-06-10T03:01:53.212 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.211+0000 7f623a8e0b80 1 -- 172.21.15.52:0/3335104573 shutdown_connections 2024-06-10T03:01:53.212 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.211+0000 7f623a8e0b80 1 --2- 172.21.15.52:0/3335104573 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x557c7da3ddc0 0x557c7da401e0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:53.212 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.211+0000 7f623a8e0b80 1 --2- 172.21.15.52:0/3335104573 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557c7da2a880 0x557c7da2de90 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:53.212 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.211+0000 7f623a8e0b80 1 --2- 172.21.15.52:0/3335104573 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557c7da247a0 0x557c7da30650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:53.212 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.211+0000 7f623a8e0b80 1 --2- 172.21.15.52:0/3335104573 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557c7da29f60 0x557c7da2d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:53.212 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.211+0000 7f620d7fa700 0 client.16611 ms_handle_reset on v2:172.21.15.188:6838/310444836 2024-06-10T03:01:53.212 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.211+0000 7f623a8e0b80 1 -- 172.21.15.52:0/3335104573 >> 172.21.15.52:0/3335104573 conn(0x557c7d951080 msgr2=0x557c7da1e700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:01:53.212 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.211+0000 7f623a8e0b80 1 -- 172.21.15.52:0/3335104573 shutdown_connections 2024-06-10T03:01:53.212 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.211+0000 7f623a8e0b80 1 -- 172.21.15.52:0/3335104573 wait complete. 2024-06-10T03:01:53.212 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.212+0000 7f623a8e0b80 20 client.16611 trim_cache size 0 max 16384 2024-06-10T03:01:53.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.246+0000 7f623a8e0b80 1 Processor -- start 2024-06-10T03:01:53.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.247+0000 7f623a8e0b80 1 -- start start 2024-06-10T03:01:53.247 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.247+0000 7f623a8e0b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557c7da247a0 0x557c7da24260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:01:53.247 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.247+0000 7f623a8e0b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557c7da29f60 0x557c7da2b450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:01:53.247 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.247+0000 7f623a8e0b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557c7da2a880 0x557c7da1ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:01:53.248 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.248+0000 7f623a8e0b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f62281bd230 con 0x557c7da2a880 2024-06-10T03:01:53.248 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.248+0000 7f622e5fa700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557c7da2a880 0x557c7da1ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:01:53.248 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.248+0000 7f622e5fa700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557c7da2a880 0x557c7da1ee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52504/0 (socket says 172.21.15.52:52504) 2024-06-10T03:01:53.248 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.248+0000 7f622e5fa700 1 -- 172.21.15.52:0/2365912547 learned_addr learned my addr 172.21.15.52:0/2365912547 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:01:53.249 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.248+0000 7f623a8e0b80 1 -- 172.21.15.52:0/2365912547 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f62281bee60 con 0x557c7da247a0 2024-06-10T03:01:53.249 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.248+0000 7f623a8e0b80 1 -- 172.21.15.52:0/2365912547 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f62281bd900 con 0x557c7da29f60 2024-06-10T03:01:53.249 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.249+0000 7f622e5fa700 1 -- 172.21.15.52:0/2365912547 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557c7da29f60 msgr2=0x557c7da2b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:01:53.249 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.249+0000 7f622e5fa700 1 --2- 172.21.15.52:0/2365912547 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557c7da29f60 0x557c7da2b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:53.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.249+0000 7f622e5fa700 1 -- 172.21.15.52:0/2365912547 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557c7da247a0 msgr2=0x557c7da24260 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-10T03:01:53.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.249+0000 7f622e5fa700 1 --2- 172.21.15.52:0/2365912547 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557c7da247a0 0x557c7da24260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:53.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.249+0000 7f622e5fa700 1 -- 172.21.15.52:0/2365912547 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c7da31200 con 0x557c7da2a880 2024-06-10T03:01:53.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.249+0000 7f622e5fa700 1 --2- 172.21.15.52:0/2365912547 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557c7da2a880 0x557c7da1ee80 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f621c09b6c0 tx=0x7f621c09b370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35d5d593fc1b334d server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:01:53.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.250+0000 7f620f7fe700 1 -- 172.21.15.52:0/2365912547 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f621c0ce070 con 0x557c7da2a880 2024-06-10T03:01:53.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.250+0000 7f620f7fe700 1 -- 172.21.15.52:0/2365912547 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f621c004d90 con 0x557c7da2a880 2024-06-10T03:01:53.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.250+0000 7f620f7fe700 1 -- 172.21.15.52:0/2365912547 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f621c0ba290 con 0x557c7da2a880 2024-06-10T03:01:53.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.251+0000 7f623a8e0b80 1 -- 172.21.15.52:0/2365912547 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557c7da2a880 msgr2=0x557c7da1ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:01:53.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.251+0000 7f623a8e0b80 1 --2- 172.21.15.52:0/2365912547 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557c7da2a880 0x557c7da1ee80 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f621c09b6c0 tx=0x7f621c09b370 comp rx=0 tx=0).stop 2024-06-10T03:01:53.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.251+0000 7f623a8e0b80 1 -- 172.21.15.52:0/2365912547 shutdown_connections 2024-06-10T03:01:53.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.251+0000 7f623a8e0b80 1 --2- 172.21.15.52:0/2365912547 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557c7da2a880 0x557c7da1ee80 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:53.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.251+0000 7f623a8e0b80 1 --2- 172.21.15.52:0/2365912547 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557c7da247a0 0x557c7da24260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:53.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.252+0000 7f623a8e0b80 1 --2- 172.21.15.52:0/2365912547 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557c7da29f60 0x557c7da2b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:53.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.252+0000 7f623a8e0b80 1 -- 172.21.15.52:0/2365912547 >> 172.21.15.52:0/2365912547 conn(0x557c7d951080 msgr2=0x557c7da2ad40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:01:53.253 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.253+0000 7f623a8e0b80 1 -- 172.21.15.52:0/2365912547 shutdown_connections 2024-06-10T03:01:53.254 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.253+0000 7f623a8e0b80 1 -- 172.21.15.52:0/2365912547 wait complete. 2024-06-10T03:01:53.254 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.254+0000 7f623a8e0b80 1 -- start start 2024-06-10T03:01:53.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.254+0000 7f623a8e0b80 1 Processor -- start 2024-06-10T03:01:53.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.255+0000 7f623a8e0b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557c7da247a0 0x557c7da366a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:01:53.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.255+0000 7f623a8e0b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557c7da29f60 0x557c7da36be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:01:53.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.255+0000 7f623a8e0b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557c7da2a880 0x557c7da44ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:01:53.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.256+0000 7f623a8e0b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f62281be790 con 0x557c7da2a880 2024-06-10T03:01:53.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.256+0000 7f623a8e0b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f62281be090 con 0x557c7da247a0 2024-06-10T03:01:53.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.256+0000 7f623a8e0b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f62281bb1d0 con 0x557c7da29f60 2024-06-10T03:01:53.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.256+0000 7f622d5f8700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557c7da29f60 0x557c7da36be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:01:53.257 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.256+0000 7f622ddf9700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557c7da247a0 0x557c7da366a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:01:53.257 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.256+0000 7f622d5f8700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557c7da29f60 0x557c7da36be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.52:33282/0 (socket says 172.21.15.52:33282) 2024-06-10T03:01:53.257 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.256+0000 7f622d5f8700 1 -- 172.21.15.52:0/2875019775 learned_addr learned my addr 172.21.15.52:0/2875019775 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:01:53.257 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.256+0000 7f622e5fa700 1 --2- 172.21.15.52:0/2875019775 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557c7da2a880 0x557c7da44ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:01:53.257 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.256+0000 7f623a8e0b80 10 client.0 _ll_register_callbacks cb 0x557c7d90b5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T03:01:53.257 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.257+0000 7f622d5f8700 1 -- 172.21.15.52:0/2875019775 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557c7da247a0 msgr2=0x557c7da366a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:01:53.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.257+0000 7f622d5f8700 1 --2- 172.21.15.52:0/2875019775 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557c7da247a0 0x557c7da366a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:53.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.257+0000 7f622d5f8700 1 -- 172.21.15.52:0/2875019775 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557c7da2a880 msgr2=0x557c7da44ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:01:53.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.257+0000 7f622d5f8700 1 --2- 172.21.15.52:0/2875019775 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557c7da2a880 0x557c7da44ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:53.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.257+0000 7f622d5f8700 1 -- 172.21.15.52:0/2875019775 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c7da45530 con 0x557c7da29f60 2024-06-10T03:01:53.259 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.259+0000 7f622d5f8700 1 --2- 172.21.15.52:0/2875019775 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557c7da29f60 0x557c7da36be0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f6218000f80 tx=0x7f621801c440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=944e7ad0878a9570 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:01:53.259 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.259+0000 7f620d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-06-10T03:01:53.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.260+0000 7f620d7fa700 1 -- 172.21.15.52:0/2875019775 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f621801cca0 con 0x557c7da29f60 2024-06-10T03:01:53.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.260+0000 7f620d7fa700 1 -- 172.21.15.52:0/2875019775 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f6218005040 con 0x557c7da29f60 2024-06-10T03:01:53.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.261+0000 7f623a8e0b80 1 -- 172.21.15.52:0/2875019775 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f621c004d90 con 0x557c7da29f60 2024-06-10T03:01:53.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.261+0000 7f620d7fa700 1 -- 172.21.15.52:0/2875019775 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f621801ce40 con 0x557c7da29f60 2024-06-10T03:01:53.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.261+0000 7f623a8e0b80 10 client.26167 fetch_fsmap learned FSMap version 37 2024-06-10T03:01:53.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.261+0000 7f623a8e0b80 1 -- 172.21.15.52:0/2875019775 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x557c7daa83d0 con 0x557c7da29f60 2024-06-10T03:01:53.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.262+0000 7f620d7fa700 1 -- 172.21.15.52:0/2875019775 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6218022070 con 0x557c7da29f60 2024-06-10T03:01:53.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.262+0000 7f620d7fa700 1 -- 172.21.15.52:0/2875019775 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 37) ==== 8382+0+0 (secure 0 0 0) 0x7f6218027070 con 0x557c7da29f60 2024-06-10T03:01:53.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.262+0000 7f623a8e0b80 10 client.26167 fetch_fsmap finished waiting for FSMap version 37 2024-06-10T03:01:53.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.262+0000 7f623a8e0b80 10 client.26167 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T03:01:53.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.262+0000 7f623a8e0b80 20 client.26167 populate_metadata read hostname 'smithi052' 2024-06-10T03:01:53.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.262+0000 7f623a8e0b80 1 --2- 172.21.15.52:0/2875019775 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x557c7daa8b10 0x557c7daaaf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:01:53.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.262+0000 7f623a8e0b80 4 client.26167 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T03:01:53.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.262+0000 7f623a8e0b80 1 -- 172.21.15.52:0/2875019775 --> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f621c004d90 con 0x557c7daa8b10 2024-06-10T03:01:53.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.263+0000 7f622ddf9700 1 --2- 172.21.15.52:0/2875019775 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x557c7daa8b10 0x557c7daaaf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:01:53.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.265+0000 7f622ddf9700 1 --2- 172.21.15.52:0/2875019775 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x557c7daa8b10 0x557c7daaaf30 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b9da9b209697105 server_cookie=f05b0e5e92b04b7f in_seq=0 out_seq=0 2024-06-10T03:01:53.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.265+0000 7f620d7fa700 10 client.26167 ms_handle_connect on v2:172.21.15.188:6838/310444836 2024-06-10T03:01:53.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.265+0000 7f620d7fa700 1 -- 172.21.15.52:0/2875019775 <== mds.0 v2:172.21.15.188:6838/310444836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f621c004d90 con 0x557c7daa8b10 2024-06-10T03:01:53.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.265+0000 7f620d7fa700 10 client.26167 handle_command_reply: tid=0 2024-06-10T03:01:53.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.266+0000 7f623a8e0b80 1 client.26167 shutdown 2024-06-10T03:01:53.267 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.266+0000 7f623a8e0b80 1 -- 172.21.15.52:0/2875019775 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557c7da29f60 msgr2=0x557c7da36be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:01:53.267 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.266+0000 7f623a8e0b80 1 --2- 172.21.15.52:0/2875019775 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557c7da29f60 0x557c7da36be0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f6218000f80 tx=0x7f621801c440 comp rx=0 tx=0).stop 2024-06-10T03:01:53.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.266+0000 7f623a8e0b80 1 -- 172.21.15.52:0/2875019775 shutdown_connections 2024-06-10T03:01:53.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.266+0000 7f623a8e0b80 1 --2- 172.21.15.52:0/2875019775 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x557c7daa8b10 0x557c7daaaf30 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:53.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.267+0000 7f620d7fa700 0 client.26167 ms_handle_reset on v2:172.21.15.188:6838/310444836 2024-06-10T03:01:53.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.267+0000 7f623a8e0b80 1 --2- 172.21.15.52:0/2875019775 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557c7da2a880 0x557c7da44ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:53.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.267+0000 7f623a8e0b80 1 --2- 172.21.15.52:0/2875019775 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557c7da247a0 0x557c7da366a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:53.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.267+0000 7f623a8e0b80 1 --2- 172.21.15.52:0/2875019775 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557c7da29f60 0x557c7da36be0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:53.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.267+0000 7f623a8e0b80 1 -- 172.21.15.52:0/2875019775 >> 172.21.15.52:0/2875019775 conn(0x557c7d951080 msgr2=0x557c7da2f180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:01:53.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.267+0000 7f623a8e0b80 1 -- 172.21.15.52:0/2875019775 shutdown_connections 2024-06-10T03:01:53.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.267+0000 7f623a8e0b80 1 -- 172.21.15.52:0/2875019775 wait complete. 2024-06-10T03:01:53.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.267+0000 7f623a8e0b80 20 client.26167 trim_cache size 0 max 16384 2024-06-10T03:01:53.268 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-06-10T03:01:53.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.269+0000 7f623005d700 1 -- 172.21.15.52:0/543869333 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f62100a2d20 msgr2=0x7f62100a51e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:01:53.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.269+0000 7f623005d700 1 --2- 172.21.15.52:0/543869333 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f62100a2d20 0x7f62100a51e0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f6218009da0 tx=0x7f62180094b0 comp rx=0 tx=0).stop 2024-06-10T03:01:53.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.269+0000 7f623005d700 1 -- 172.21.15.52:0/543869333 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f622810e690 msgr2=0x7f62281a7930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:01:53.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.269+0000 7f623005d700 1 --2- 172.21.15.52:0/543869333 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f622810e690 0x7f62281a7930 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f621c00e940 tx=0x7f621c00ed00 comp rx=0 tx=0).stop 2024-06-10T03:01:53.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.269+0000 7f623005d700 1 -- 172.21.15.52:0/543869333 shutdown_connections 2024-06-10T03:01:53.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.269+0000 7f623005d700 1 --2- 172.21.15.52:0/543869333 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f622810e690 0x7f62281a7930 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:53.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.269+0000 7f623005d700 1 --2- 172.21.15.52:0/543869333 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6228109f80 0x7f62281ad770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:53.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.269+0000 7f623005d700 1 --2- 172.21.15.52:0/543869333 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f62100a2d20 0x7f62100a51e0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:53.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.269+0000 7f623005d700 1 --2- 172.21.15.52:0/543869333 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f622810a930 0x7f62281adcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:53.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.269+0000 7f623005d700 1 -- 172.21.15.52:0/543869333 >> 172.21.15.52:0/543869333 conn(0x7f622806d9e0 msgr2=0x7f6228115a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:01:53.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.269+0000 7f623005d700 1 -- 172.21.15.52:0/543869333 shutdown_connections 2024-06-10T03:01:53.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:53.269+0000 7f623005d700 1 -- 172.21.15.52:0/543869333 wait complete. 2024-06-10T03:01:54.727 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:01:54 smithi052 ceph-mon[32362]: pgmap v665: 129 pgs: 129 active+clean; 625 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 6.2 MiB/s wr, 971 op/s 2024-06-10T03:01:54.728 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:01:54 smithi052 ceph-mon[32362]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-10T03:01:54.888 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:01:54 smithi120 ceph-mon[36634]: pgmap v665: 129 pgs: 129 active+clean; 625 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 6.2 MiB/s wr, 971 op/s 2024-06-10T03:01:54.888 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:01:54 smithi120 ceph-mon[36634]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-10T03:01:54.923 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-06-10T03:01:54.923 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-06-10T03:01:54.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:01:54 smithi188 ceph-mon[36599]: pgmap v665: 129 pgs: 129 active+clean; 625 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 6.2 MiB/s wr, 971 op/s 2024-06-10T03:01:54.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:01:54 smithi188 ceph-mon[36599]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-10T03:01:55.176 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T03:01:56.542 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:01:56 smithi052 ceph-mon[32362]: pgmap v666: 129 pgs: 129 active+clean; 624 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 6.1 MiB/s wr, 845 op/s 2024-06-10T03:01:56.888 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:01:56 smithi120 ceph-mon[36634]: pgmap v666: 129 pgs: 129 active+clean; 624 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 6.1 MiB/s wr, 845 op/s 2024-06-10T03:01:56.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:01:56 smithi188 ceph-mon[36599]: pgmap v666: 129 pgs: 129 active+clean; 624 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 6.1 MiB/s wr, 845 op/s 2024-06-10T03:01:57.600 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:01:57 smithi052 ceph-mon[32362]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-10T03:01:57.888 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:01:57 smithi120 ceph-mon[36634]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-10T03:01:57.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:01:57 smithi188 ceph-mon[36599]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-10T03:01:57.990 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:57.989+0000 7f092ea57700 1 -- 172.21.15.52:0/263258246 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0928107840 msgr2=0x7f092810f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:01:57.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:57.989+0000 7f092ea57700 1 --2- 172.21.15.52:0/263258246 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0928107840 0x7f092810f3c0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f092000b0d0 tx=0x7f092000b3e0 comp rx=0 tx=0).stop 2024-06-10T03:01:57.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:57.989+0000 7f092ea57700 1 -- 172.21.15.52:0/263258246 shutdown_connections 2024-06-10T03:01:57.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:57.989+0000 7f092ea57700 1 --2- 172.21.15.52:0/263258246 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0928107840 0x7f092810f3c0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:57.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:57.989+0000 7f092ea57700 1 --2- 172.21.15.52:0/263258246 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0928102740 0x7f0928102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:57.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:57.989+0000 7f092ea57700 1 --2- 172.21.15.52:0/263258246 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f09281030f0 0x7f0928107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:57.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:57.989+0000 7f092ea57700 1 -- 172.21.15.52:0/263258246 >> 172.21.15.52:0/263258246 conn(0x7f09280795f0 msgr2=0x7f0928079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:01:57.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:57.989+0000 7f092ea57700 1 -- 172.21.15.52:0/263258246 shutdown_connections 2024-06-10T03:01:57.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:57.989+0000 7f092ea57700 1 -- 172.21.15.52:0/263258246 wait complete. 2024-06-10T03:01:57.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:57.990+0000 7f092ea57700 1 Processor -- start 2024-06-10T03:01:57.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:57.990+0000 7f092ea57700 1 -- start start 2024-06-10T03:01:57.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:57.990+0000 7f092ea57700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0928102740 0x7f092819e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:01:57.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:57.990+0000 7f092ea57700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f09281030f0 0x7f092819ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:01:57.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:57.990+0000 7f092ea57700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0928107840 0x7f0928198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:01:57.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:57.991+0000 7f092ea57700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f0928112760 con 0x7f0928102740 2024-06-10T03:01:57.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:57.991+0000 7f0927fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0928102740 0x7f092819e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:01:57.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:57.991+0000 7f092ea57700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f09281125e0 con 0x7f09281030f0 2024-06-10T03:01:57.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:57.991+0000 7f092ea57700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f09281128e0 con 0x7f0928107840 2024-06-10T03:01:57.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:57.991+0000 7f0927fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0928102740 0x7f092819e7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52534/0 (socket says 172.21.15.52:52534) 2024-06-10T03:01:57.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:57.991+0000 7f0927fff700 1 -- 172.21.15.52:0/4222940973 learned_addr learned my addr 172.21.15.52:0/4222940973 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:01:57.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:57.991+0000 7f091ffff700 1 --2- 172.21.15.52:0/4222940973 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f09281030f0 0x7f092819ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:01:57.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:57.991+0000 7f092cff4700 1 --2- 172.21.15.52:0/4222940973 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0928107840 0x7f0928198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:01:57.994 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:57.991+0000 7f0927fff700 1 -- 172.21.15.52:0/4222940973 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0928107840 msgr2=0x7f0928198960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:01:57.994 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:57.991+0000 7f0927fff700 1 --2- 172.21.15.52:0/4222940973 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0928107840 0x7f0928198960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:57.994 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:57.991+0000 7f0927fff700 1 -- 172.21.15.52:0/4222940973 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f09281030f0 msgr2=0x7f092819ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:01:57.994 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:57.991+0000 7f0927fff700 1 --2- 172.21.15.52:0/4222940973 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f09281030f0 0x7f092819ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:57.994 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:57.991+0000 7f0927fff700 1 -- 172.21.15.52:0/4222940973 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09281990c0 con 0x7f0928102740 2024-06-10T03:01:57.994 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:57.991+0000 7f0927fff700 1 --2- 172.21.15.52:0/4222940973 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0928102740 0x7f092819e7a0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f091800e9a0 tx=0x7f091800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:01:57.994 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:57.991+0000 7f0925ffb700 1 -- 172.21.15.52:0/4222940973 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f091800ca10 con 0x7f0928102740 2024-06-10T03:01:57.994 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:57.991+0000 7f092ea57700 1 -- 172.21.15.52:0/4222940973 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09281993b0 con 0x7f0928102740 2024-06-10T03:01:57.994 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:57.991+0000 7f0925ffb700 1 -- 172.21.15.52:0/4222940973 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f091800cbb0 con 0x7f0928102740 2024-06-10T03:01:57.994 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:57.991+0000 7f0925ffb700 1 -- 172.21.15.52:0/4222940973 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0918018680 con 0x7f0928102740 2024-06-10T03:01:57.994 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:57.991+0000 7f092ea57700 1 -- 172.21.15.52:0/4222940973 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09281a5670 con 0x7f0928102740 2024-06-10T03:01:57.995 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:57.994+0000 7f0925ffb700 1 -- 172.21.15.52:0/4222940973 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f0918010430 con 0x7f0928102740 2024-06-10T03:01:57.995 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:57.994+0000 7f0925ffb700 1 --2- 172.21.15.52:0/4222940973 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f09140a2ca0 0x7f09140a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:01:57.996 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:57.995+0000 7f091ffff700 1 --2- 172.21.15.52:0/4222940973 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f09140a2ca0 0x7f09140a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:01:57.996 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:57.995+0000 7f0925ffb700 1 -- 172.21.15.52:0/4222940973 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f09180cf570 con 0x7f0928102740 2024-06-10T03:01:57.996 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:57.995+0000 7f091ffff700 1 --2- 172.21.15.52:0/4222940973 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f09140a2ca0 0x7f09140a5160 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f0928108d00 tx=0x7f0910009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:01:57.997 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:57.995+0000 7f09392dab80 1 Processor -- start 2024-06-10T03:01:57.997 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:57.995+0000 7f09392dab80 1 -- start start 2024-06-10T03:01:57.997 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:57.996+0000 7f09392dab80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558ee0cdb3f0 0x558ee0cdb7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:01:57.997 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:57.996+0000 7f09392dab80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558ee0cdbd10 0x558ee0cd56f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:01:57.997 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:57.996+0000 7f09392dab80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558ee0cd5c30 0x558ee0cd6070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:01:57.997 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:57.996+0000 7f09392dab80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x558ee0bc3fe0 con 0x558ee0cd5c30 2024-06-10T03:01:57.997 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:57.996+0000 7f09392dab80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x558ee0c228d0 con 0x558ee0cdb3f0 2024-06-10T03:01:57.998 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:57.996+0000 7f09392dab80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x558ee0b8a4e0 con 0x558ee0cdbd10 2024-06-10T03:01:57.998 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:57.996+0000 7f092cff4700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558ee0cd5c30 0x558ee0cd6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:01:57.998 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:57.996+0000 7f0927fff700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558ee0cdb3f0 0x558ee0cdb7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:01:57.998 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:57.996+0000 7f092cff4700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558ee0cd5c30 0x558ee0cd6070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52540/0 (socket says 172.21.15.52:52540) 2024-06-10T03:01:57.998 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:57.996+0000 7f0927fff700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558ee0cdb3f0 0x558ee0cdb7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.52:40048/0 (socket says 172.21.15.52:40048) 2024-06-10T03:01:57.998 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:57.996+0000 7f092cff4700 1 -- 172.21.15.52:0/3158532125 learned_addr learned my addr 172.21.15.52:0/3158532125 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:01:57.998 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:57.996+0000 7f091ffff700 1 --2- 172.21.15.52:0/3158532125 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558ee0cdbd10 0x558ee0cd56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:01:57.998 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:57.996+0000 7f0927fff700 1 -- 172.21.15.52:0/3158532125 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558ee0cdbd10 msgr2=0x558ee0cd56f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:01:57.998 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:57.996+0000 7f0927fff700 1 --2- 172.21.15.52:0/3158532125 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558ee0cdbd10 0x558ee0cd56f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:57.998 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:57.996+0000 7f0927fff700 1 -- 172.21.15.52:0/3158532125 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558ee0cd5c30 msgr2=0x558ee0cd6070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:01:57.998 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:57.996+0000 7f0927fff700 1 --2- 172.21.15.52:0/3158532125 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558ee0cd5c30 0x558ee0cd6070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:57.998 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:57.996+0000 7f0927fff700 1 -- 172.21.15.52:0/3158532125 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ee0b71950 con 0x558ee0cdb3f0 2024-06-10T03:01:57.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:57.996+0000 7f0927fff700 1 --2- 172.21.15.52:0/3158532125 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558ee0cdb3f0 0x558ee0cdb7d0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f09180cccc0 tx=0x7f091809b690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ea83969c201f1fc8 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:01:58.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:57.997+0000 7f091e7fc700 1 -- 172.21.15.52:0/3158532125 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09180a4690 con 0x558ee0cdb3f0 2024-06-10T03:01:58.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:57.997+0000 7f091e7fc700 1 -- 172.21.15.52:0/3158532125 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f09180ce070 con 0x558ee0cdb3f0 2024-06-10T03:01:58.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:57.997+0000 7f091e7fc700 1 -- 172.21.15.52:0/3158532125 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0918010720 con 0x558ee0cdb3f0 2024-06-10T03:01:58.002 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:57.997+0000 7f09392dab80 1 -- 172.21.15.52:0/3158532125 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558ee0cdb3f0 msgr2=0x558ee0cdb7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:01:58.002 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:57.997+0000 7f09392dab80 1 --2- 172.21.15.52:0/3158532125 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558ee0cdb3f0 0x558ee0cdb7d0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f09180cccc0 tx=0x7f091809b690 comp rx=0 tx=0).stop 2024-06-10T03:01:58.002 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:57.997+0000 7f09392dab80 1 -- 172.21.15.52:0/3158532125 shutdown_connections 2024-06-10T03:01:58.002 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:57.997+0000 7f09392dab80 1 --2- 172.21.15.52:0/3158532125 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558ee0cd5c30 0x558ee0cd6070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:58.002 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:57.997+0000 7f09392dab80 1 --2- 172.21.15.52:0/3158532125 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558ee0cdb3f0 0x558ee0cdb7d0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:58.002 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:57.997+0000 7f09392dab80 1 --2- 172.21.15.52:0/3158532125 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558ee0cdbd10 0x558ee0cd56f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:58.002 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:57.997+0000 7f09392dab80 1 -- 172.21.15.52:0/3158532125 >> 172.21.15.52:0/3158532125 conn(0x558ee0bfe100 msgr2=0x558ee0ccc2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:01:58.002 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:57.997+0000 7f09392dab80 1 -- 172.21.15.52:0/3158532125 shutdown_connections 2024-06-10T03:01:58.002 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:57.998+0000 7f09392dab80 1 -- 172.21.15.52:0/3158532125 wait complete. 2024-06-10T03:01:58.002 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:57.998+0000 7f09392dab80 1 -- start start 2024-06-10T03:01:58.002 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:57.998+0000 7f09392dab80 1 Processor -- start 2024-06-10T03:01:58.003 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:57.998+0000 7f09392dab80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558ee0cd5c30 0x558ee0ce1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:01:58.003 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:57.998+0000 7f09392dab80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558ee0cdb3f0 0x558ee0cded50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:01:58.003 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:57.998+0000 7f09392dab80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558ee0cdbd10 0x558ee0cdf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:01:58.003 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:57.998+0000 7f09392dab80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x558ee0bc3040 con 0x558ee0cd5c30 2024-06-10T03:01:58.003 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:57.999+0000 7f09392dab80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x558ee0c1e890 con 0x558ee0cdb3f0 2024-06-10T03:01:58.003 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:57.999+0000 7f09392dab80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x558ee0ce2700 con 0x558ee0cdbd10 2024-06-10T03:01:58.003 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:57.999+0000 7f0927fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558ee0cd5c30 0x558ee0ce1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:01:58.003 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:57.999+0000 7f09392dab80 10 client.0 _ll_register_callbacks cb 0x558ee0bb8630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T03:01:58.003 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:57.999+0000 7f0927fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558ee0cd5c30 0x558ee0ce1a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52548/0 (socket says 172.21.15.52:52548) 2024-06-10T03:01:58.003 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:57.999+0000 7f0927fff700 1 -- 172.21.15.52:0/993806727 learned_addr learned my addr 172.21.15.52:0/993806727 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:01:58.003 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:57.999+0000 7f091ffff700 1 --2- 172.21.15.52:0/993806727 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558ee0cdb3f0 0x558ee0cded50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:01:58.003 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:57.999+0000 7f092cff4700 1 --2- 172.21.15.52:0/993806727 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558ee0cdbd10 0x558ee0cdf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:01:58.003 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:57.999+0000 7f0927fff700 1 -- 172.21.15.52:0/993806727 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558ee0cdbd10 msgr2=0x558ee0cdf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:01:58.003 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:57.999+0000 7f0927fff700 1 --2- 172.21.15.52:0/993806727 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558ee0cdbd10 0x558ee0cdf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:58.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:57.999+0000 7f0927fff700 1 -- 172.21.15.52:0/993806727 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558ee0cdb3f0 msgr2=0x558ee0cded50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:01:58.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:57.999+0000 7f0927fff700 1 --2- 172.21.15.52:0/993806727 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558ee0cdb3f0 0x558ee0cded50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:58.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:57.999+0000 7f0927fff700 1 -- 172.21.15.52:0/993806727 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ee0be87d0 con 0x558ee0cd5c30 2024-06-10T03:01:58.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:57.999+0000 7f0927fff700 1 --2- 172.21.15.52:0/993806727 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558ee0cd5c30 0x558ee0ce1a50 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f09180a4160 tx=0x7f091809fe90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad86aca7934a5e1a server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:01:58.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:57.999+0000 7f09077fe700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-10T03:01:58.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:57.999+0000 7f09077fe700 1 -- 172.21.15.52:0/993806727 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09180b8020 con 0x558ee0cd5c30 2024-06-10T03:01:58.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:57.999+0000 7f09077fe700 1 -- 172.21.15.52:0/993806727 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f0918099d20 con 0x558ee0cd5c30 2024-06-10T03:01:58.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:57.999+0000 7f09392dab80 1 -- 172.21.15.52:0/993806727 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f09180ce070 con 0x558ee0cd5c30 2024-06-10T03:01:58.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:57.999+0000 7f09077fe700 1 -- 172.21.15.52:0/993806727 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f09180ba2a0 con 0x558ee0cd5c30 2024-06-10T03:01:58.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:57.999+0000 7f09392dab80 10 client.16629 fetch_fsmap learned FSMap version 39 2024-06-10T03:01:58.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:58.000+0000 7f09392dab80 1 -- 172.21.15.52:0/993806727 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x558ee0cece70 con 0x558ee0cd5c30 2024-06-10T03:01:58.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:58.000+0000 7f09077fe700 1 -- 172.21.15.52:0/993806727 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f09180ce070 con 0x558ee0cd5c30 2024-06-10T03:01:58.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:58.000+0000 7f09077fe700 1 -- 172.21.15.52:0/993806727 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 39) ==== 8390+0+0 (secure 0 0 0) 0x7f091809af00 con 0x558ee0cd5c30 2024-06-10T03:01:58.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:58.000+0000 7f09392dab80 10 client.16629 fetch_fsmap finished waiting for FSMap version 39 2024-06-10T03:01:58.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:58.000+0000 7f09392dab80 10 client.16629 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T03:01:58.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:58.000+0000 7f09392dab80 20 client.16629 populate_metadata read hostname 'smithi052' 2024-06-10T03:01:58.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:58.000+0000 7f09392dab80 1 --2- 172.21.15.52:0/993806727 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x558ee0cef340 0x558ee0cf1730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:01:58.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:58.000+0000 7f09392dab80 4 client.16629 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T03:01:58.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:58.000+0000 7f09392dab80 1 -- 172.21.15.52:0/993806727 --> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558ee0c2fdd0 con 0x558ee0cef340 2024-06-10T03:01:58.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:58.000+0000 7f091ffff700 1 --2- 172.21.15.52:0/993806727 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x558ee0cef340 0x558ee0cf1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:01:58.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:58.001+0000 7f091ffff700 1 --2- 172.21.15.52:0/993806727 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x558ee0cef340 0x558ee0cf1730 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e225b282a9421b54 server_cookie=69f3944dc54e3bd3 in_seq=0 out_seq=0 2024-06-10T03:01:58.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:58.001+0000 7f09077fe700 10 client.16629 ms_handle_connect on v2:172.21.15.188:6838/310444836 2024-06-10T03:01:58.006 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:58.001+0000 7f09077fe700 1 -- 172.21.15.52:0/993806727 <== mds.0 v2:172.21.15.188:6838/310444836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x558ee0c2fdd0 con 0x558ee0cef340 2024-06-10T03:01:58.006 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:58.001+0000 7f09077fe700 10 client.16629 handle_command_reply: tid=0 2024-06-10T03:01:58.006 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:58.001+0000 7f09392dab80 1 client.16629 shutdown 2024-06-10T03:01:58.006 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:58.002+0000 7f09392dab80 1 -- 172.21.15.52:0/993806727 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558ee0cd5c30 msgr2=0x558ee0ce1a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:01:58.006 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:58.002+0000 7f09392dab80 1 --2- 172.21.15.52:0/993806727 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558ee0cd5c30 0x558ee0ce1a50 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f09180a4160 tx=0x7f091809fe90 comp rx=0 tx=0).stop 2024-06-10T03:01:58.006 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:58.002+0000 7f09392dab80 1 -- 172.21.15.52:0/993806727 shutdown_connections 2024-06-10T03:01:58.006 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:58.002+0000 7f09392dab80 1 --2- 172.21.15.52:0/993806727 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x558ee0cef340 0x558ee0cf1730 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:58.006 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:58.002+0000 7f09077fe700 0 client.16629 ms_handle_reset on v2:172.21.15.188:6838/310444836 2024-06-10T03:01:58.007 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:58.002+0000 7f09392dab80 1 --2- 172.21.15.52:0/993806727 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558ee0cdbd10 0x558ee0cdf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:58.007 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:58.002+0000 7f09392dab80 1 --2- 172.21.15.52:0/993806727 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558ee0cd5c30 0x558ee0ce1a50 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:58.007 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:58.002+0000 7f09392dab80 1 --2- 172.21.15.52:0/993806727 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558ee0cdb3f0 0x558ee0cded50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:58.007 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:58.002+0000 7f09392dab80 1 -- 172.21.15.52:0/993806727 >> 172.21.15.52:0/993806727 conn(0x558ee0bfe100 msgr2=0x558ee0ccb780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:01:58.007 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:58.002+0000 7f09392dab80 1 -- 172.21.15.52:0/993806727 shutdown_connections 2024-06-10T03:01:58.007 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:58.002+0000 7f09392dab80 1 -- 172.21.15.52:0/993806727 wait complete. 2024-06-10T03:01:58.007 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:58.002+0000 7f09392dab80 20 client.16629 trim_cache size 0 max 16384 2024-06-10T03:01:58.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:58.023+0000 7f09392dab80 1 Processor -- start 2024-06-10T03:01:58.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:58.023+0000 7f09392dab80 1 -- start start 2024-06-10T03:01:58.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:58.023+0000 7f09392dab80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558ee0cd5c30 0x558ee0ce0c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:01:58.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:58.024+0000 7f09392dab80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558ee0cdb3f0 0x558ee0ccbf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:01:58.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:58.024+0000 7f09392dab80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558ee0cdbd10 0x558ee0ccc440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:01:58.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:58.024+0000 7f09392dab80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f09281ac220 con 0x558ee0cdb3f0 2024-06-10T03:01:58.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:58.024+0000 7f09392dab80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f09281adeb0 con 0x558ee0cd5c30 2024-06-10T03:01:58.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:58.024+0000 7f09392dab80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f09281ac920 con 0x558ee0cdbd10 2024-06-10T03:01:58.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:58.024+0000 7f091ffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558ee0cdb3f0 0x558ee0ccbf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:01:58.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:58.024+0000 7f091ffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558ee0cdb3f0 0x558ee0ccbf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52552/0 (socket says 172.21.15.52:52552) 2024-06-10T03:01:58.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:58.024+0000 7f091ffff700 1 -- 172.21.15.52:0/1419257880 learned_addr learned my addr 172.21.15.52:0/1419257880 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:01:58.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:58.024+0000 7f092cff4700 1 --2- 172.21.15.52:0/1419257880 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558ee0cdbd10 0x558ee0ccc440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:01:58.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:58.024+0000 7f0927fff700 1 --2- 172.21.15.52:0/1419257880 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558ee0cd5c30 0x558ee0ce0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:01:58.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:58.024+0000 7f091ffff700 1 -- 172.21.15.52:0/1419257880 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558ee0cdbd10 msgr2=0x558ee0ccc440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:01:58.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:58.024+0000 7f091ffff700 1 --2- 172.21.15.52:0/1419257880 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558ee0cdbd10 0x558ee0ccc440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:58.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:58.024+0000 7f091ffff700 1 -- 172.21.15.52:0/1419257880 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558ee0cd5c30 msgr2=0x558ee0ce0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:01:58.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:58.024+0000 7f091ffff700 1 --2- 172.21.15.52:0/1419257880 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558ee0cd5c30 0x558ee0ce0c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:58.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:58.024+0000 7f091ffff700 1 -- 172.21.15.52:0/1419257880 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ee0ce02b0 con 0x558ee0cdb3f0 2024-06-10T03:01:58.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:58.024+0000 7f091ffff700 1 --2- 172.21.15.52:0/1419257880 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558ee0cdb3f0 0x558ee0ccbf00 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f09100117d0 tx=0x7f0910014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=482fa599bf84bb83 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:01:58.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:58.024+0000 7f091dffb700 1 -- 172.21.15.52:0/1419257880 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0910003900 con 0x558ee0cdb3f0 2024-06-10T03:01:58.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:58.024+0000 7f091dffb700 1 -- 172.21.15.52:0/1419257880 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f091001d410 con 0x558ee0cdb3f0 2024-06-10T03:01:58.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:58.024+0000 7f091dffb700 1 -- 172.21.15.52:0/1419257880 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0910029650 con 0x558ee0cdb3f0 2024-06-10T03:01:58.030 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:58.025+0000 7f09392dab80 1 -- 172.21.15.52:0/1419257880 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558ee0cdb3f0 msgr2=0x558ee0ccbf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:01:58.030 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:58.025+0000 7f09392dab80 1 --2- 172.21.15.52:0/1419257880 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558ee0cdb3f0 0x558ee0ccbf00 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f09100117d0 tx=0x7f0910014000 comp rx=0 tx=0).stop 2024-06-10T03:01:58.030 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:58.025+0000 7f09392dab80 1 -- 172.21.15.52:0/1419257880 shutdown_connections 2024-06-10T03:01:58.030 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:58.025+0000 7f09392dab80 1 --2- 172.21.15.52:0/1419257880 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558ee0cdbd10 0x558ee0ccc440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:58.030 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:58.025+0000 7f09392dab80 1 --2- 172.21.15.52:0/1419257880 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558ee0cd5c30 0x558ee0ce0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:58.030 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:58.025+0000 7f09392dab80 1 --2- 172.21.15.52:0/1419257880 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558ee0cdb3f0 0x558ee0ccbf00 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:58.030 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:58.025+0000 7f09392dab80 1 -- 172.21.15.52:0/1419257880 >> 172.21.15.52:0/1419257880 conn(0x558ee0bfe100 msgr2=0x558ee0ce0580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:01:58.030 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:58.025+0000 7f09392dab80 1 -- 172.21.15.52:0/1419257880 shutdown_connections 2024-06-10T03:01:58.030 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:58.025+0000 7f09392dab80 1 -- 172.21.15.52:0/1419257880 wait complete. 2024-06-10T03:01:58.030 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:58.025+0000 7f09392dab80 1 -- start start 2024-06-10T03:01:58.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:58.026+0000 7f09392dab80 1 Processor -- start 2024-06-10T03:01:58.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:58.026+0000 7f09392dab80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558ee0cd5c30 0x558ee0cf4750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:01:58.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:58.026+0000 7f09392dab80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558ee0cdb3f0 0x558ee0cf4c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:01:58.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:58.026+0000 7f09392dab80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558ee0cdbd10 0x558ee0cf2c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:01:58.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:58.026+0000 7f09392dab80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f09281ad7b0 con 0x558ee0cdb3f0 2024-06-10T03:01:58.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:58.026+0000 7f09392dab80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f09281ad0b0 con 0x558ee0cd5c30 2024-06-10T03:01:58.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:58.026+0000 7f09392dab80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f09281aa1c0 con 0x558ee0cdbd10 2024-06-10T03:01:58.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:58.026+0000 7f09392dab80 10 client.0 _ll_register_callbacks cb 0x558ee0bb8630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T03:01:58.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:58.026+0000 7f0927fff700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558ee0cd5c30 0x558ee0cf4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:01:58.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:58.026+0000 7f0927fff700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558ee0cd5c30 0x558ee0cf4750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.52:40082/0 (socket says 172.21.15.52:40082) 2024-06-10T03:01:58.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:58.026+0000 7f0927fff700 1 -- 172.21.15.52:0/2017849695 learned_addr learned my addr 172.21.15.52:0/2017849695 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:01:58.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:58.026+0000 7f092cff4700 1 --2- 172.21.15.52:0/2017849695 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558ee0cdbd10 0x558ee0cf2c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:01:58.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:58.026+0000 7f091ffff700 1 --2- 172.21.15.52:0/2017849695 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558ee0cdb3f0 0x558ee0cf4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:01:58.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:58.026+0000 7f0927fff700 1 -- 172.21.15.52:0/2017849695 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558ee0cdbd10 msgr2=0x558ee0cf2c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:01:58.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:58.026+0000 7f0927fff700 1 --2- 172.21.15.52:0/2017849695 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558ee0cdbd10 0x558ee0cf2c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:58.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:58.026+0000 7f0927fff700 1 -- 172.21.15.52:0/2017849695 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558ee0cdb3f0 msgr2=0x558ee0cf4c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:01:58.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:58.026+0000 7f0927fff700 1 --2- 172.21.15.52:0/2017849695 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558ee0cdb3f0 0x558ee0cf4c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:58.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:58.026+0000 7f0927fff700 1 -- 172.21.15.52:0/2017849695 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ee0cf3410 con 0x558ee0cd5c30 2024-06-10T03:01:58.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:58.027+0000 7f0927fff700 1 --2- 172.21.15.52:0/2017849695 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558ee0cd5c30 0x558ee0cf4750 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f091809b9b0 tx=0x7f091809ff00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd9d571d9312fc7c server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:01:58.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:58.027+0000 7f09077fe700 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-06-10T03:01:58.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:58.027+0000 7f09077fe700 1 -- 172.21.15.52:0/2017849695 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09180b8070 con 0x558ee0cd5c30 2024-06-10T03:01:58.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:58.027+0000 7f09392dab80 1 -- 172.21.15.52:0/2017849695 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f091001d410 con 0x558ee0cd5c30 2024-06-10T03:01:58.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:58.027+0000 7f09077fe700 1 -- 172.21.15.52:0/2017849695 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f0918010930 con 0x558ee0cd5c30 2024-06-10T03:01:58.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:58.027+0000 7f09077fe700 1 -- 172.21.15.52:0/2017849695 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f091809b7f0 con 0x558ee0cd5c30 2024-06-10T03:01:58.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:58.027+0000 7f09392dab80 10 client.26141 fetch_fsmap learned FSMap version 39 2024-06-10T03:01:58.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:58.027+0000 7f09392dab80 1 -- 172.21.15.52:0/2017849695 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x558ee0cf3410 con 0x558ee0cd5c30 2024-06-10T03:01:58.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:58.027+0000 7f09077fe700 1 -- 172.21.15.52:0/2017849695 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f091001d410 con 0x558ee0cd5c30 2024-06-10T03:01:58.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:58.027+0000 7f09077fe700 1 -- 172.21.15.52:0/2017849695 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8390+0+0 (secure 0 0 0) 0x7f09180860b0 con 0x558ee0cd5c30 2024-06-10T03:01:58.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:58.027+0000 7f09392dab80 10 client.26141 fetch_fsmap finished waiting for FSMap version 39 2024-06-10T03:01:58.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:58.027+0000 7f09392dab80 10 client.26141 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T03:01:58.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:58.027+0000 7f09392dab80 20 client.26141 populate_metadata read hostname 'smithi052' 2024-06-10T03:01:58.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:58.027+0000 7f09392dab80 1 --2- 172.21.15.52:0/2017849695 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x558ee0d5aa50 0x558ee0d5ce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:01:58.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:58.027+0000 7f09392dab80 4 client.26141 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-06-10T03:01:58.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:58.027+0000 7f09392dab80 1 -- 172.21.15.52:0/2017849695 --> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x558ee0c2a350 con 0x558ee0d5aa50 2024-06-10T03:01:58.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:58.028+0000 7f091ffff700 1 --2- 172.21.15.52:0/2017849695 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x558ee0d5aa50 0x558ee0d5ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:01:58.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:58.028+0000 7f091ffff700 1 --2- 172.21.15.52:0/2017849695 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x558ee0d5aa50 0x558ee0d5ce40 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=abb3164ec6315b80 server_cookie=c558833d56c4956 in_seq=0 out_seq=0 2024-06-10T03:01:58.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:58.028+0000 7f09077fe700 10 client.26141 ms_handle_connect on v2:172.21.15.188:6838/310444836 2024-06-10T03:01:58.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:58.029+0000 7f09077fe700 1 -- 172.21.15.52:0/2017849695 <== mds.0 v2:172.21.15.188:6838/310444836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x558ee0c2a350 con 0x558ee0d5aa50 2024-06-10T03:01:58.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:58.029+0000 7f09077fe700 10 client.26141 handle_command_reply: tid=0 2024-06-10T03:01:58.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:58.029+0000 7f09392dab80 1 client.26141 shutdown 2024-06-10T03:01:58.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:58.029+0000 7f09392dab80 1 -- 172.21.15.52:0/2017849695 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558ee0cd5c30 msgr2=0x558ee0cf4750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:01:58.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:58.029+0000 7f09392dab80 1 --2- 172.21.15.52:0/2017849695 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558ee0cd5c30 0x558ee0cf4750 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f091809b9b0 tx=0x7f091809ff00 comp rx=0 tx=0).stop 2024-06-10T03:01:58.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:58.029+0000 7f09392dab80 1 -- 172.21.15.52:0/2017849695 shutdown_connections 2024-06-10T03:01:58.035 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:58.029+0000 7f09392dab80 1 --2- 172.21.15.52:0/2017849695 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x558ee0d5aa50 0x558ee0d5ce40 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:58.035 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:58.030+0000 7f09392dab80 1 --2- 172.21.15.52:0/2017849695 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558ee0cdbd10 0x558ee0cf2c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:58.035 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:58.030+0000 7f09392dab80 1 --2- 172.21.15.52:0/2017849695 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558ee0cd5c30 0x558ee0cf4750 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:58.035 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:58.030+0000 7f09392dab80 1 --2- 172.21.15.52:0/2017849695 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558ee0cdb3f0 0x558ee0cf4c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:58.035 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:58.030+0000 7f09392dab80 1 -- 172.21.15.52:0/2017849695 >> 172.21.15.52:0/2017849695 conn(0x558ee0bfe100 msgr2=0x558ee0cd0b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:01:58.035 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:58.030+0000 7f09077fe700 0 client.26141 ms_handle_reset on v2:172.21.15.188:6838/310444836 2024-06-10T03:01:58.035 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:58.030+0000 7f09392dab80 1 -- 172.21.15.52:0/2017849695 shutdown_connections 2024-06-10T03:01:58.035 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:58.030+0000 7f09392dab80 1 -- 172.21.15.52:0/2017849695 wait complete. 2024-06-10T03:01:58.035 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:58.030+0000 7f09392dab80 20 client.26141 trim_cache size 0 max 16384 2024-06-10T03:01:58.035 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-10T03:01:58.035 INFO:teuthology.orchestra.run.smithi052.stdout: "return_code": 0, 2024-06-10T03:01:58.036 INFO:teuthology.orchestra.run.smithi052.stdout: "scrub_tag": "cde34f71-67b8-4825-a70c-65a33edb0bcd", 2024-06-10T03:01:58.036 INFO:teuthology.orchestra.run.smithi052.stdout: "mode": "asynchronous" 2024-06-10T03:01:58.036 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-10T03:01:58.036 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:58.030+0000 7f092ea57700 1 -- 172.21.15.52:0/4222940973 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f09140a2ca0 msgr2=0x7f09140a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:01:58.036 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:58.030+0000 7f092ea57700 1 --2- 172.21.15.52:0/4222940973 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f09140a2ca0 0x7f09140a5160 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f0928108d00 tx=0x7f0910009260 comp rx=0 tx=0).stop 2024-06-10T03:01:58.036 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:58.030+0000 7f092ea57700 1 -- 172.21.15.52:0/4222940973 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0928102740 msgr2=0x7f092819e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:01:58.036 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:58.030+0000 7f092ea57700 1 --2- 172.21.15.52:0/4222940973 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0928102740 0x7f092819e7a0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f091800e9a0 tx=0x7f091800ecb0 comp rx=0 tx=0).stop 2024-06-10T03:01:58.036 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:58.030+0000 7f092ea57700 1 -- 172.21.15.52:0/4222940973 shutdown_connections 2024-06-10T03:01:58.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:58.030+0000 7f092ea57700 1 --2- 172.21.15.52:0/4222940973 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0928107840 0x7f0928198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:58.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:58.030+0000 7f092ea57700 1 --2- 172.21.15.52:0/4222940973 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f09140a2ca0 0x7f09140a5160 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:58.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:58.030+0000 7f092ea57700 1 --2- 172.21.15.52:0/4222940973 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0928102740 0x7f092819e7a0 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:58.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:58.030+0000 7f092ea57700 1 --2- 172.21.15.52:0/4222940973 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f09281030f0 0x7f092819ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:01:58.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:58.030+0000 7f092ea57700 1 -- 172.21.15.52:0/4222940973 >> 172.21.15.52:0/4222940973 conn(0x7f09280795f0 msgr2=0x7f0928100490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:01:58.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:58.031+0000 7f092ea57700 1 -- 172.21.15.52:0/4222940973 shutdown_connections 2024-06-10T03:01:58.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:01:58.031+0000 7f092ea57700 1 -- 172.21.15.52:0/4222940973 wait complete. 2024-06-10T03:01:58.623 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:01:58 smithi052 ceph-mon[32362]: pgmap v667: 129 pgs: 129 active+clean; 614 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 6.1 MiB/s wr, 886 op/s 2024-06-10T03:01:58.623 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:01:58 smithi052 ceph-mon[32362]: scrub queued for path: / 2024-06-10T03:01:58.623 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:01:58 smithi052 ceph-mon[32362]: scrub summary: idle+waiting paths [/] 2024-06-10T03:01:58.623 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:01:58 smithi052 ceph-mon[32362]: scrub summary: active paths [/] 2024-06-10T03:01:58.623 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:01:58 smithi052 ceph-mon[32362]: scrub summary: idle+waiting paths [/] 2024-06-10T03:01:58.624 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:01:58 smithi052 ceph-mon[32362]: scrub summary: active paths [/] 2024-06-10T03:01:58.624 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:01:58 smithi052 ceph-mon[32362]: scrub summary: idle+waiting paths [/] 2024-06-10T03:01:58.624 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:01:58 smithi052 ceph-mon[32362]: scrub summary: active paths [/] 2024-06-10T03:01:58.624 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:01:58 smithi052 ceph-mon[32362]: scrub summary: idle+waiting paths [/] 2024-06-10T03:01:58.624 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:01:58 smithi052 ceph-mon[32362]: scrub summary: active paths [/] 2024-06-10T03:01:58.624 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:01:58 smithi052 ceph-mon[32362]: scrub summary: idle+waiting paths [/] 2024-06-10T03:01:58.671 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-10T03:01:58.888 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:01:58 smithi120 ceph-mon[36634]: pgmap v667: 129 pgs: 129 active+clean; 614 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 6.1 MiB/s wr, 886 op/s 2024-06-10T03:01:58.888 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:01:58 smithi120 ceph-mon[36634]: scrub queued for path: / 2024-06-10T03:01:58.888 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:01:58 smithi120 ceph-mon[36634]: scrub summary: idle+waiting paths [/] 2024-06-10T03:01:58.888 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:01:58 smithi120 ceph-mon[36634]: scrub summary: active paths [/] 2024-06-10T03:01:58.888 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:01:58 smithi120 ceph-mon[36634]: scrub summary: idle+waiting paths [/] 2024-06-10T03:01:58.888 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:01:58 smithi120 ceph-mon[36634]: scrub summary: active paths [/] 2024-06-10T03:01:58.888 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:01:58 smithi120 ceph-mon[36634]: scrub summary: idle+waiting paths [/] 2024-06-10T03:01:58.889 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:01:58 smithi120 ceph-mon[36634]: scrub summary: active paths [/] 2024-06-10T03:01:58.889 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:01:58 smithi120 ceph-mon[36634]: scrub summary: idle+waiting paths [/] 2024-06-10T03:01:58.889 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:01:58 smithi120 ceph-mon[36634]: scrub summary: active paths [/] 2024-06-10T03:01:58.889 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:01:58 smithi120 ceph-mon[36634]: scrub summary: idle+waiting paths [/] 2024-06-10T03:01:58.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:01:58 smithi188 ceph-mon[36599]: pgmap v667: 129 pgs: 129 active+clean; 614 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 6.1 MiB/s wr, 886 op/s 2024-06-10T03:01:58.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:01:58 smithi188 ceph-mon[36599]: scrub queued for path: / 2024-06-10T03:01:58.954 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:01:58 smithi188 ceph-mon[36599]: scrub summary: idle+waiting paths [/] 2024-06-10T03:01:58.954 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:01:58 smithi188 ceph-mon[36599]: scrub summary: active paths [/] 2024-06-10T03:01:58.954 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:01:58 smithi188 ceph-mon[36599]: scrub summary: idle+waiting paths [/] 2024-06-10T03:01:58.954 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:01:58 smithi188 ceph-mon[36599]: scrub summary: active paths [/] 2024-06-10T03:01:58.954 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:01:58 smithi188 ceph-mon[36599]: scrub summary: idle+waiting paths [/] 2024-06-10T03:01:58.954 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:01:58 smithi188 ceph-mon[36599]: scrub summary: active paths [/] 2024-06-10T03:01:58.954 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:01:58 smithi188 ceph-mon[36599]: scrub summary: idle+waiting paths [/] 2024-06-10T03:01:58.954 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:01:58 smithi188 ceph-mon[36599]: scrub summary: active paths [/] 2024-06-10T03:01:58.954 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:01:58 smithi188 ceph-mon[36599]: scrub summary: idle+waiting paths [/] 2024-06-10T03:01:58.955 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T03:01:59.727 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:01:59 smithi052 ceph-mon[32362]: scrub summary: active paths [/] 2024-06-10T03:01:59.727 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:01:59 smithi052 ceph-mon[32362]: scrub summary: active paths [/] 2024-06-10T03:01:59.727 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:01:59 smithi052 ceph-mon[32362]: scrub summary: idle+waiting paths [/] 2024-06-10T03:01:59.727 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:01:59 smithi052 ceph-mon[32362]: scrub summary: active paths [/] 2024-06-10T03:01:59.727 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:01:59 smithi052 ceph-mon[32362]: scrub summary: idle+waiting paths [/] 2024-06-10T03:01:59.727 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:01:59 smithi052 ceph-mon[32362]: scrub summary: active paths [/] 2024-06-10T03:01:59.727 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:01:59 smithi052 ceph-mon[32362]: scrub summary: idle+waiting paths [/] 2024-06-10T03:01:59.888 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:01:59 smithi120 ceph-mon[36634]: scrub summary: active paths [/] 2024-06-10T03:01:59.888 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:01:59 smithi120 ceph-mon[36634]: scrub summary: active paths [/] 2024-06-10T03:01:59.888 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:01:59 smithi120 ceph-mon[36634]: scrub summary: idle+waiting paths [/] 2024-06-10T03:01:59.889 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:01:59 smithi120 ceph-mon[36634]: scrub summary: active paths [/] 2024-06-10T03:01:59.889 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:01:59 smithi120 ceph-mon[36634]: scrub summary: idle+waiting paths [/] 2024-06-10T03:01:59.889 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:01:59 smithi120 ceph-mon[36634]: scrub summary: active paths [/] 2024-06-10T03:01:59.889 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:01:59 smithi120 ceph-mon[36634]: scrub summary: idle+waiting paths [/] 2024-06-10T03:01:59.954 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:01:59 smithi188 ceph-mon[36599]: scrub summary: active paths [/] 2024-06-10T03:01:59.954 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:01:59 smithi188 ceph-mon[36599]: scrub summary: active paths [/] 2024-06-10T03:01:59.954 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:01:59 smithi188 ceph-mon[36599]: scrub summary: idle+waiting paths [/] 2024-06-10T03:01:59.954 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:01:59 smithi188 ceph-mon[36599]: scrub summary: active paths [/] 2024-06-10T03:01:59.954 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:01:59 smithi188 ceph-mon[36599]: scrub summary: idle+waiting paths [/] 2024-06-10T03:01:59.954 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:01:59 smithi188 ceph-mon[36599]: scrub summary: active paths [/] 2024-06-10T03:01:59.954 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:01:59 smithi188 ceph-mon[36599]: scrub summary: idle+waiting paths [/] 2024-06-10T03:02:00.671 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:02:00 smithi052 ceph-mon[32362]: pgmap v668: 129 pgs: 129 active+clean; 606 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.9 MiB/s wr, 877 op/s 2024-06-10T03:02:00.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.670+0000 7f26b5437700 1 -- 172.21.15.52:0/2394862006 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f26b01027a0 msgr2=0x7f26b0102b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:02:00.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.670+0000 7f26b5437700 1 --2- 172.21.15.52:0/2394862006 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f26b01027a0 0x7f26b0102b80 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f26a00099c0 tx=0x7f26a0009cd0 comp rx=0 tx=0).stop 2024-06-10T03:02:00.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.670+0000 7f26b5437700 1 -- 172.21.15.52:0/2394862006 shutdown_connections 2024-06-10T03:02:00.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.670+0000 7f26b5437700 1 --2- 172.21.15.52:0/2394862006 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f26b0107870 0x7f26b010f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:00.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.670+0000 7f26b5437700 1 --2- 172.21.15.52:0/2394862006 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f26b01027a0 0x7f26b0102b80 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:00.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.670+0000 7f26b5437700 1 --2- 172.21.15.52:0/2394862006 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f26b0103150 0x7f26b0107140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:00.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.670+0000 7f26b5437700 1 -- 172.21.15.52:0/2394862006 >> 172.21.15.52:0/2394862006 conn(0x7f26b0079570 msgr2=0x7f26b0079980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:02:00.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.670+0000 7f26b5437700 1 -- 172.21.15.52:0/2394862006 shutdown_connections 2024-06-10T03:02:00.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.670+0000 7f26b5437700 1 -- 172.21.15.52:0/2394862006 wait complete. 2024-06-10T03:02:00.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.671+0000 7f26b5437700 1 Processor -- start 2024-06-10T03:02:00.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.671+0000 7f26b5437700 1 -- start start 2024-06-10T03:02:00.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.671+0000 7f26b5437700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f26b01027a0 0x7f26b019e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:02:00.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.671+0000 7f26b5437700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f26b0103150 0x7f26b019ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:02:00.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.671+0000 7f26b5437700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f26b0107870 0x7f26b0198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:02:00.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.671+0000 7f26b5437700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f26b0112790 con 0x7f26b0103150 2024-06-10T03:02:00.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.671+0000 7f26b5437700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f26b0112610 con 0x7f26b01027a0 2024-06-10T03:02:00.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.671+0000 7f26b5437700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f26b0112910 con 0x7f26b0107870 2024-06-10T03:02:00.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.672+0000 7f26ae7fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f26b0103150 0x7f26b019ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:02:00.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.672+0000 7f26ae7fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f26b0103150 0x7f26b019ed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52568/0 (socket says 172.21.15.52:52568) 2024-06-10T03:02:00.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.672+0000 7f26ae7fc700 1 -- 172.21.15.52:0/836353904 learned_addr learned my addr 172.21.15.52:0/836353904 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:02:00.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.672+0000 7f26aeffd700 1 --2- 172.21.15.52:0/836353904 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f26b01027a0 0x7f26b019e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:02:00.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.672+0000 7f26af7fe700 1 --2- 172.21.15.52:0/836353904 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f26b0107870 0x7f26b0198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:02:00.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.672+0000 7f26ae7fc700 1 -- 172.21.15.52:0/836353904 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f26b0107870 msgr2=0x7f26b0198a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:02:00.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.672+0000 7f26ae7fc700 1 --2- 172.21.15.52:0/836353904 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f26b0107870 0x7f26b0198a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:00.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.672+0000 7f26ae7fc700 1 -- 172.21.15.52:0/836353904 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f26b01027a0 msgr2=0x7f26b019e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:02:00.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.672+0000 7f26ae7fc700 1 --2- 172.21.15.52:0/836353904 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f26b01027a0 0x7f26b019e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:00.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.672+0000 7f26ae7fc700 1 -- 172.21.15.52:0/836353904 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26b0199170 con 0x7f26b0103150 2024-06-10T03:02:00.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.672+0000 7f26aeffd700 1 --2- 172.21.15.52:0/836353904 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f26b01027a0 0x7f26b019e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T03:02:00.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.672+0000 7f26ae7fc700 1 --2- 172.21.15.52:0/836353904 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f26b0103150 0x7f26b019ed90 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f26a400e9a0 tx=0x7f26a400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:02:00.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.673+0000 7f2697fff700 1 -- 172.21.15.52:0/836353904 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26a400ca10 con 0x7f26b0103150 2024-06-10T03:02:00.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.673+0000 7f2697fff700 1 -- 172.21.15.52:0/836353904 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26a400cbb0 con 0x7f26b0103150 2024-06-10T03:02:00.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.673+0000 7f26b5437700 1 -- 172.21.15.52:0/836353904 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26b0199460 con 0x7f26b0103150 2024-06-10T03:02:00.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.673+0000 7f2697fff700 1 -- 172.21.15.52:0/836353904 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26a4018680 con 0x7f26b0103150 2024-06-10T03:02:00.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.673+0000 7f26b5437700 1 -- 172.21.15.52:0/836353904 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26b01a5720 con 0x7f26b0103150 2024-06-10T03:02:00.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.675+0000 7f2697fff700 1 -- 172.21.15.52:0/836353904 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f26a4010430 con 0x7f26b0103150 2024-06-10T03:02:00.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.676+0000 7f2697fff700 1 --2- 172.21.15.52:0/836353904 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f26980a2ca0 0x7f26980a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:02:00.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.676+0000 7f2697fff700 1 -- 172.21.15.52:0/836353904 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f26a40cf5d0 con 0x7f26b0103150 2024-06-10T03:02:00.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.676+0000 7f26aeffd700 1 --2- 172.21.15.52:0/836353904 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f26980a2ca0 0x7f26980a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:02:00.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.676+0000 7f26aeffd700 1 --2- 172.21.15.52:0/836353904 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f26980a2ca0 0x7f26980a5160 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f26a0000c00 tx=0x7f26a0005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:02:00.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.677+0000 7f26bfcbab80 1 Processor -- start 2024-06-10T03:02:00.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.677+0000 7f26bfcbab80 1 -- start start 2024-06-10T03:02:00.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.677+0000 7f26bfcbab80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560efe37c850 0x560efe37cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:02:00.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.677+0000 7f26bfcbab80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560efe37d170 0x560efe381010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:02:00.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.677+0000 7f26bfcbab80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560efe381550 0x560efe381990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:02:00.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.677+0000 7f26bfcbab80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x560efe26af60 con 0x560efe381550 2024-06-10T03:02:00.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.677+0000 7f26bfcbab80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x560efe2c9850 con 0x560efe37d170 2024-06-10T03:02:00.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.677+0000 7f26bfcbab80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x560efe231460 con 0x560efe37c850 2024-06-10T03:02:00.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.677+0000 7f26aeffd700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560efe37c850 0x560efe37cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:02:00.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.677+0000 7f26af7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560efe381550 0x560efe381990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:02:00.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.677+0000 7f26aeffd700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560efe37c850 0x560efe37cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.52:33354/0 (socket says 172.21.15.52:33354) 2024-06-10T03:02:00.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.677+0000 7f26aeffd700 1 -- 172.21.15.52:0/934513258 learned_addr learned my addr 172.21.15.52:0/934513258 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:02:00.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.677+0000 7f26ae7fc700 1 --2- 172.21.15.52:0/934513258 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560efe37d170 0x560efe381010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:02:00.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.677+0000 7f26aeffd700 1 -- 172.21.15.52:0/934513258 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560efe37d170 msgr2=0x560efe381010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:02:00.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.677+0000 7f26aeffd700 1 --2- 172.21.15.52:0/934513258 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560efe37d170 0x560efe381010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:00.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.677+0000 7f26aeffd700 1 -- 172.21.15.52:0/934513258 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560efe381550 msgr2=0x560efe381990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:02:00.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.677+0000 7f26aeffd700 1 --2- 172.21.15.52:0/934513258 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560efe381550 0x560efe381990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:00.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.677+0000 7f26aeffd700 1 -- 172.21.15.52:0/934513258 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560efe2188d0 con 0x560efe37c850 2024-06-10T03:02:00.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.678+0000 7f26aeffd700 1 --2- 172.21.15.52:0/934513258 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560efe37c850 0x560efe37cc30 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f26a0015000 tx=0x7f26a0020000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=95f095ed7446447f server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:02:00.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.678+0000 7f26957fa700 1 -- 172.21.15.52:0/934513258 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26a0003c40 con 0x560efe37c850 2024-06-10T03:02:00.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.678+0000 7f26957fa700 1 -- 172.21.15.52:0/934513258 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26a0003de0 con 0x560efe37c850 2024-06-10T03:02:00.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.678+0000 7f26957fa700 1 -- 172.21.15.52:0/934513258 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26a000fc80 con 0x560efe37c850 2024-06-10T03:02:00.683 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.678+0000 7f26bfcbab80 1 -- 172.21.15.52:0/934513258 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560efe37c850 msgr2=0x560efe37cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:02:00.683 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.678+0000 7f26bfcbab80 1 --2- 172.21.15.52:0/934513258 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560efe37c850 0x560efe37cc30 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f26a0015000 tx=0x7f26a0020000 comp rx=0 tx=0).stop 2024-06-10T03:02:00.683 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.679+0000 7f26bfcbab80 1 -- 172.21.15.52:0/934513258 shutdown_connections 2024-06-10T03:02:00.683 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.679+0000 7f26bfcbab80 1 --2- 172.21.15.52:0/934513258 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560efe381550 0x560efe381990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:00.684 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.679+0000 7f26bfcbab80 1 --2- 172.21.15.52:0/934513258 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560efe37c850 0x560efe37cc30 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:00.684 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.679+0000 7f26bfcbab80 1 --2- 172.21.15.52:0/934513258 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560efe37d170 0x560efe381010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:00.684 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.679+0000 7f26bfcbab80 1 -- 172.21.15.52:0/934513258 >> 172.21.15.52:0/934513258 conn(0x560efe2a5080 msgr2=0x560efe373220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:02:00.684 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.679+0000 7f26bfcbab80 1 -- 172.21.15.52:0/934513258 shutdown_connections 2024-06-10T03:02:00.684 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.679+0000 7f26bfcbab80 1 -- 172.21.15.52:0/934513258 wait complete. 2024-06-10T03:02:00.684 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.679+0000 7f26bfcbab80 1 -- start start 2024-06-10T03:02:00.684 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.679+0000 7f26bfcbab80 1 Processor -- start 2024-06-10T03:02:00.684 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.679+0000 7f26bfcbab80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560efe37c850 0x560efe388b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:02:00.684 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.679+0000 7f26bfcbab80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560efe37d170 0x560efe385e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:02:00.684 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.679+0000 7f26bfcbab80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560efe381550 0x560efe386370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:02:00.684 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.679+0000 7f26bfcbab80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x560efe269fc0 con 0x560efe37c850 2024-06-10T03:02:00.684 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.679+0000 7f26bfcbab80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x560efe2c5810 con 0x560efe381550 2024-06-10T03:02:00.684 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.679+0000 7f26bfcbab80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x560efe389620 con 0x560efe37d170 2024-06-10T03:02:00.684 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.679+0000 7f26bfcbab80 10 client.0 _ll_register_callbacks cb 0x560efe25f5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T03:02:00.685 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.680+0000 7f26ae7fc700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560efe37d170 0x560efe385e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:02:00.685 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.680+0000 7f26aeffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560efe37c850 0x560efe388b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:02:00.685 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.680+0000 7f26ae7fc700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560efe37d170 0x560efe385e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.52:33360/0 (socket says 172.21.15.52:33360) 2024-06-10T03:02:00.685 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.680+0000 7f26ae7fc700 1 -- 172.21.15.52:0/2351412292 learned_addr learned my addr 172.21.15.52:0/2351412292 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:02:00.685 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.680+0000 7f26af7fe700 1 --2- 172.21.15.52:0/2351412292 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560efe381550 0x560efe386370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:02:00.685 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.680+0000 7f26ae7fc700 1 -- 172.21.15.52:0/2351412292 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560efe381550 msgr2=0x560efe386370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:02:00.685 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.680+0000 7f26ae7fc700 1 --2- 172.21.15.52:0/2351412292 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560efe381550 0x560efe386370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:00.685 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.680+0000 7f26ae7fc700 1 -- 172.21.15.52:0/2351412292 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560efe37c850 msgr2=0x560efe388b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:02:00.685 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.680+0000 7f26ae7fc700 1 --2- 172.21.15.52:0/2351412292 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560efe37c850 0x560efe388b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:00.685 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.680+0000 7f26ae7fc700 1 -- 172.21.15.52:0/2351412292 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560efe28f750 con 0x560efe37d170 2024-06-10T03:02:00.685 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.680+0000 7f26aeffd700 1 --2- 172.21.15.52:0/2351412292 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560efe37c850 0x560efe388b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T03:02:00.685 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.680+0000 7f26ae7fc700 1 --2- 172.21.15.52:0/2351412292 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560efe37d170 0x560efe385e30 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f26a40983f0 tx=0x7f26a40cf4e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f9157a7c7de6ef1f server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:02:00.685 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.680+0000 7f2692ff5700 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-06-10T03:02:00.685 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.680+0000 7f2692ff5700 1 -- 172.21.15.52:0/2351412292 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26a40b4060 con 0x560efe37d170 2024-06-10T03:02:00.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.680+0000 7f26bfcbab80 1 -- 172.21.15.52:0/2351412292 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f26a0003de0 con 0x560efe37d170 2024-06-10T03:02:00.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.680+0000 7f2692ff5700 1 -- 172.21.15.52:0/2351412292 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f26a40a4200 con 0x560efe37d170 2024-06-10T03:02:00.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.681+0000 7f2692ff5700 1 -- 172.21.15.52:0/2351412292 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f26a40a4690 con 0x560efe37d170 2024-06-10T03:02:00.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.681+0000 7f26bfcbab80 10 client.26197 fetch_fsmap learned FSMap version 39 2024-06-10T03:02:00.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.681+0000 7f26bfcbab80 1 -- 172.21.15.52:0/2351412292 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x560efe393e50 con 0x560efe37d170 2024-06-10T03:02:00.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.681+0000 7f2692ff5700 1 -- 172.21.15.52:0/2351412292 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26a0003de0 con 0x560efe37d170 2024-06-10T03:02:00.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.681+0000 7f2692ff5700 1 -- 172.21.15.52:0/2351412292 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 39) ==== 8390+0+0 (secure 0 0 0) 0x7f26a4010720 con 0x560efe37d170 2024-06-10T03:02:00.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.681+0000 7f26bfcbab80 10 client.26197 fetch_fsmap finished waiting for FSMap version 39 2024-06-10T03:02:00.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.681+0000 7f26bfcbab80 10 client.26197 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T03:02:00.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.681+0000 7f26bfcbab80 20 client.26197 populate_metadata read hostname 'smithi052' 2024-06-10T03:02:00.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.681+0000 7f26bfcbab80 1 --2- 172.21.15.52:0/2351412292 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x560efe396490 0x560efe398880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:02:00.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.681+0000 7f26bfcbab80 4 client.26197 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T03:02:00.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.681+0000 7f26bfcbab80 1 -- 172.21.15.52:0/2351412292 --> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560efe2d6d50 con 0x560efe396490 2024-06-10T03:02:00.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.681+0000 7f26aeffd700 1 --2- 172.21.15.52:0/2351412292 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x560efe396490 0x560efe398880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:02:00.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.682+0000 7f26aeffd700 1 --2- 172.21.15.52:0/2351412292 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x560efe396490 0x560efe398880 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=841f069f915ddbac server_cookie=82dd1393f4eeb68f in_seq=0 out_seq=0 2024-06-10T03:02:00.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.682+0000 7f2692ff5700 10 client.26197 ms_handle_connect on v2:172.21.15.188:6838/310444836 2024-06-10T03:02:00.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.683+0000 7f2692ff5700 1 -- 172.21.15.52:0/2351412292 <== mds.0 v2:172.21.15.188:6838/310444836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x560efe2d6d50 con 0x560efe396490 2024-06-10T03:02:00.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.683+0000 7f2692ff5700 10 client.26197 handle_command_reply: tid=0 2024-06-10T03:02:00.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.683+0000 7f26bfcbab80 1 client.26197 shutdown 2024-06-10T03:02:00.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.683+0000 7f26bfcbab80 1 -- 172.21.15.52:0/2351412292 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560efe37d170 msgr2=0x560efe385e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:02:00.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.683+0000 7f26bfcbab80 1 --2- 172.21.15.52:0/2351412292 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560efe37d170 0x560efe385e30 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f26a40983f0 tx=0x7f26a40cf4e0 comp rx=0 tx=0).stop 2024-06-10T03:02:00.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.683+0000 7f26bfcbab80 1 -- 172.21.15.52:0/2351412292 shutdown_connections 2024-06-10T03:02:00.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.683+0000 7f26bfcbab80 1 --2- 172.21.15.52:0/2351412292 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x560efe396490 0x560efe398880 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:00.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.683+0000 7f26bfcbab80 1 --2- 172.21.15.52:0/2351412292 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560efe381550 0x560efe386370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:00.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.683+0000 7f2692ff5700 0 client.26197 ms_handle_reset on v2:172.21.15.188:6838/310444836 2024-06-10T03:02:00.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.683+0000 7f26bfcbab80 1 --2- 172.21.15.52:0/2351412292 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560efe37c850 0x560efe388b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:00.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.683+0000 7f26bfcbab80 1 --2- 172.21.15.52:0/2351412292 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560efe37d170 0x560efe385e30 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:00.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.683+0000 7f26bfcbab80 1 -- 172.21.15.52:0/2351412292 >> 172.21.15.52:0/2351412292 conn(0x560efe2a5080 msgr2=0x560efe382a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:02:00.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.683+0000 7f26bfcbab80 1 -- 172.21.15.52:0/2351412292 shutdown_connections 2024-06-10T03:02:00.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.683+0000 7f26bfcbab80 1 -- 172.21.15.52:0/2351412292 wait complete. 2024-06-10T03:02:00.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.683+0000 7f26bfcbab80 20 client.26197 trim_cache size 0 max 16384 2024-06-10T03:02:00.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.704+0000 7f26bfcbab80 1 Processor -- start 2024-06-10T03:02:00.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.704+0000 7f26bfcbab80 1 -- start start 2024-06-10T03:02:00.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.705+0000 7f26bfcbab80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560efe37c850 0x560efe384fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:02:00.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.705+0000 7f26bfcbab80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560efe37d170 0x560efe387d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:02:00.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.705+0000 7f26bfcbab80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560efe381550 0x560efe39e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:02:00.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.705+0000 7f26bfcbab80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f26b01ac0a0 con 0x560efe37c850 2024-06-10T03:02:00.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.705+0000 7f26bfcbab80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f26b01add30 con 0x560efe37d170 2024-06-10T03:02:00.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.705+0000 7f26bfcbab80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f26b01ac7a0 con 0x560efe381550 2024-06-10T03:02:00.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.705+0000 7f26aeffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560efe37c850 0x560efe384fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:02:00.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.705+0000 7f26aeffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560efe37c850 0x560efe384fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52588/0 (socket says 172.21.15.52:52588) 2024-06-10T03:02:00.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.705+0000 7f26aeffd700 1 -- 172.21.15.52:0/377831969 learned_addr learned my addr 172.21.15.52:0/377831969 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:02:00.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.705+0000 7f26af7fe700 1 --2- 172.21.15.52:0/377831969 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560efe381550 0x560efe39e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:02:00.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.705+0000 7f26ae7fc700 1 --2- 172.21.15.52:0/377831969 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560efe37d170 0x560efe387d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:02:00.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.705+0000 7f26aeffd700 1 -- 172.21.15.52:0/377831969 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560efe381550 msgr2=0x560efe39e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:02:00.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.705+0000 7f26aeffd700 1 --2- 172.21.15.52:0/377831969 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560efe381550 0x560efe39e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:00.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.705+0000 7f26aeffd700 1 -- 172.21.15.52:0/377831969 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560efe37d170 msgr2=0x560efe387d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:02:00.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.705+0000 7f26aeffd700 1 --2- 172.21.15.52:0/377831969 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560efe37d170 0x560efe387d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:00.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.705+0000 7f26aeffd700 1 -- 172.21.15.52:0/377831969 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560efe373760 con 0x560efe37c850 2024-06-10T03:02:00.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.705+0000 7f26ae7fc700 1 --2- 172.21.15.52:0/377831969 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560efe37d170 0x560efe387d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T03:02:00.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.705+0000 7f26aeffd700 1 --2- 172.21.15.52:0/377831969 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560efe37c850 0x560efe384fb0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f26a0009d10 tx=0x7f26a0023000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=303734102f99f161 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:02:00.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.706+0000 7f2694ff9700 1 -- 172.21.15.52:0/377831969 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26a000fc40 con 0x560efe37c850 2024-06-10T03:02:00.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.706+0000 7f2694ff9700 1 -- 172.21.15.52:0/377831969 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26a0005590 con 0x560efe37c850 2024-06-10T03:02:00.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.706+0000 7f2694ff9700 1 -- 172.21.15.52:0/377831969 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26a0038770 con 0x560efe37c850 2024-06-10T03:02:00.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.706+0000 7f26bfcbab80 1 -- 172.21.15.52:0/377831969 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560efe37c850 msgr2=0x560efe384fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:02:00.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.706+0000 7f26bfcbab80 1 --2- 172.21.15.52:0/377831969 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560efe37c850 0x560efe384fb0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f26a0009d10 tx=0x7f26a0023000 comp rx=0 tx=0).stop 2024-06-10T03:02:00.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.706+0000 7f26bfcbab80 1 -- 172.21.15.52:0/377831969 shutdown_connections 2024-06-10T03:02:00.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.706+0000 7f26bfcbab80 1 --2- 172.21.15.52:0/377831969 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560efe381550 0x560efe39e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:00.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.706+0000 7f26bfcbab80 1 --2- 172.21.15.52:0/377831969 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560efe37c850 0x560efe384fb0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:00.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.706+0000 7f26bfcbab80 1 --2- 172.21.15.52:0/377831969 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560efe37d170 0x560efe387d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:00.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.706+0000 7f26bfcbab80 1 -- 172.21.15.52:0/377831969 >> 172.21.15.52:0/377831969 conn(0x560efe2a5080 msgr2=0x560efe387660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:02:00.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.706+0000 7f26bfcbab80 1 -- 172.21.15.52:0/377831969 shutdown_connections 2024-06-10T03:02:00.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.706+0000 7f26bfcbab80 1 -- 172.21.15.52:0/377831969 wait complete. 2024-06-10T03:02:00.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.707+0000 7f26bfcbab80 1 -- start start 2024-06-10T03:02:00.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.707+0000 7f26bfcbab80 1 Processor -- start 2024-06-10T03:02:00.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.707+0000 7f26bfcbab80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560efe37c850 0x560efe38f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:02:00.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.707+0000 7f26bfcbab80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560efe37d170 0x560efe38c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:02:00.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.707+0000 7f26bfcbab80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560efe381550 0x560efe38c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:02:00.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.707+0000 7f26bfcbab80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f26b01ad630 con 0x560efe37d170 2024-06-10T03:02:00.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.707+0000 7f26bfcbab80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f26b01acf30 con 0x560efe37c850 2024-06-10T03:02:00.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.707+0000 7f26bfcbab80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f26b01aa040 con 0x560efe381550 2024-06-10T03:02:00.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.707+0000 7f26bfcbab80 10 client.0 _ll_register_callbacks cb 0x560efe25f5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T03:02:00.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.707+0000 7f26ae7fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560efe37d170 0x560efe38c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:02:00.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.707+0000 7f26af7fe700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560efe381550 0x560efe38c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:02:00.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.707+0000 7f26ae7fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560efe37d170 0x560efe38c3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52590/0 (socket says 172.21.15.52:52590) 2024-06-10T03:02:00.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.707+0000 7f26ae7fc700 1 -- 172.21.15.52:0/4066909249 learned_addr learned my addr 172.21.15.52:0/4066909249 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:02:00.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.707+0000 7f26aeffd700 1 --2- 172.21.15.52:0/4066909249 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560efe37c850 0x560efe38f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:02:00.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.708+0000 7f26af7fe700 1 -- 172.21.15.52:0/4066909249 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560efe37c850 msgr2=0x560efe38f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:02:00.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.708+0000 7f26af7fe700 1 --2- 172.21.15.52:0/4066909249 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560efe37c850 0x560efe38f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:00.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.708+0000 7f26af7fe700 1 -- 172.21.15.52:0/4066909249 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560efe37d170 msgr2=0x560efe38c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:02:00.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.708+0000 7f26af7fe700 1 --2- 172.21.15.52:0/4066909249 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560efe37d170 0x560efe38c3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:00.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.708+0000 7f26af7fe700 1 -- 172.21.15.52:0/4066909249 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560efe38d190 con 0x560efe381550 2024-06-10T03:02:00.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.708+0000 7f26ae7fc700 1 --2- 172.21.15.52:0/4066909249 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560efe37d170 0x560efe38c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T03:02:00.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.708+0000 7f26aeffd700 1 --2- 172.21.15.52:0/4066909249 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560efe37c850 0x560efe38f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T03:02:00.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.708+0000 7f26af7fe700 1 --2- 172.21.15.52:0/4066909249 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560efe381550 0x560efe38c950 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f26a8015fb0 tx=0x7f26a8018700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2c6236e6167c414f server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:02:00.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.708+0000 7f2692ff5700 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-06-10T03:02:00.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.708+0000 7f2692ff5700 1 -- 172.21.15.52:0/4066909249 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26a8004bb0 con 0x560efe381550 2024-06-10T03:02:00.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.708+0000 7f26bfcbab80 1 -- 172.21.15.52:0/4066909249 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f26a0005590 con 0x560efe381550 2024-06-10T03:02:00.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.708+0000 7f2692ff5700 1 -- 172.21.15.52:0/4066909249 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f26a8004d50 con 0x560efe381550 2024-06-10T03:02:00.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.708+0000 7f2692ff5700 1 -- 172.21.15.52:0/4066909249 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f26a802b050 con 0x560efe381550 2024-06-10T03:02:00.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.708+0000 7f26bfcbab80 10 client.26203 fetch_fsmap learned FSMap version 39 2024-06-10T03:02:00.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.708+0000 7f26bfcbab80 1 -- 172.21.15.52:0/4066909249 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x560efe401140 con 0x560efe381550 2024-06-10T03:02:00.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.709+0000 7f2692ff5700 1 -- 172.21.15.52:0/4066909249 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26a0005590 con 0x560efe381550 2024-06-10T03:02:00.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.709+0000 7f2692ff5700 1 -- 172.21.15.52:0/4066909249 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 39) ==== 8390+0+0 (secure 0 0 0) 0x7f26a8016520 con 0x560efe381550 2024-06-10T03:02:00.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.709+0000 7f26bfcbab80 10 client.26203 fetch_fsmap finished waiting for FSMap version 39 2024-06-10T03:02:00.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.709+0000 7f26bfcbab80 10 client.26203 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T03:02:00.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.709+0000 7f26bfcbab80 20 client.26203 populate_metadata read hostname 'smithi052' 2024-06-10T03:02:00.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.709+0000 7f26bfcbab80 1 --2- 172.21.15.52:0/4066909249 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x560efe401860 0x560efe403c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:02:00.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.709+0000 7f26bfcbab80 4 client.26203 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-10T03:02:00.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.709+0000 7f26bfcbab80 1 -- 172.21.15.52:0/4066909249 --> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560efe2d12d0 con 0x560efe401860 2024-06-10T03:02:00.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.709+0000 7f26aeffd700 1 --2- 172.21.15.52:0/4066909249 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x560efe401860 0x560efe403c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:02:00.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.710+0000 7f26aeffd700 1 --2- 172.21.15.52:0/4066909249 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x560efe401860 0x560efe403c80 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c6bcfab426ed6d2 server_cookie=eabd7cdfd1b03a06 in_seq=0 out_seq=0 2024-06-10T03:02:00.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.710+0000 7f2692ff5700 10 client.26203 ms_handle_connect on v2:172.21.15.188:6838/310444836 2024-06-10T03:02:00.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.710+0000 7f2692ff5700 1 -- 172.21.15.52:0/4066909249 <== mds.0 v2:172.21.15.188:6838/310444836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x560efe2d12d0 con 0x560efe401860 2024-06-10T03:02:00.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.710+0000 7f2692ff5700 10 client.26203 handle_command_reply: tid=0 2024-06-10T03:02:00.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.710+0000 7f26bfcbab80 1 client.26203 shutdown 2024-06-10T03:02:00.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.710+0000 7f26bfcbab80 1 -- 172.21.15.52:0/4066909249 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560efe381550 msgr2=0x560efe38c950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:02:00.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.710+0000 7f26bfcbab80 1 --2- 172.21.15.52:0/4066909249 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560efe381550 0x560efe38c950 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f26a8015fb0 tx=0x7f26a8018700 comp rx=0 tx=0).stop 2024-06-10T03:02:00.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.710+0000 7f26bfcbab80 1 -- 172.21.15.52:0/4066909249 shutdown_connections 2024-06-10T03:02:00.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.710+0000 7f26bfcbab80 1 --2- 172.21.15.52:0/4066909249 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x560efe401860 0x560efe403c80 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:00.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.710+0000 7f26bfcbab80 1 --2- 172.21.15.52:0/4066909249 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560efe381550 0x560efe38c950 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:00.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.710+0000 7f2692ff5700 0 client.26203 ms_handle_reset on v2:172.21.15.188:6838/310444836 2024-06-10T03:02:00.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.710+0000 7f26bfcbab80 1 --2- 172.21.15.52:0/4066909249 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560efe37c850 0x560efe38f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:00.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.710+0000 7f26bfcbab80 1 --2- 172.21.15.52:0/4066909249 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560efe37d170 0x560efe38c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:00.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.710+0000 7f26bfcbab80 1 -- 172.21.15.52:0/4066909249 >> 172.21.15.52:0/4066909249 conn(0x560efe2a5080 msgr2=0x560efe377ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:02:00.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.710+0000 7f26bfcbab80 1 -- 172.21.15.52:0/4066909249 shutdown_connections 2024-06-10T03:02:00.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.710+0000 7f26bfcbab80 1 -- 172.21.15.52:0/4066909249 wait complete. 2024-06-10T03:02:00.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.710+0000 7f26bfcbab80 20 client.26203 trim_cache size 0 max 16384 2024-06-10T03:02:00.717 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-10T03:02:00.718 INFO:teuthology.orchestra.run.smithi052.stdout: "status": "IDLE (waiting for more scrubs)", 2024-06-10T03:02:00.718 INFO:teuthology.orchestra.run.smithi052.stdout: "scrubs": { 2024-06-10T03:02:00.718 INFO:teuthology.orchestra.run.smithi052.stdout: "cde34f71-67b8-4825-a70c-65a33edb0bcd": { 2024-06-10T03:02:00.718 INFO:teuthology.orchestra.run.smithi052.stdout: "path": "/", 2024-06-10T03:02:00.718 INFO:teuthology.orchestra.run.smithi052.stdout: "tag": "cde34f71-67b8-4825-a70c-65a33edb0bcd", 2024-06-10T03:02:00.718 INFO:teuthology.orchestra.run.smithi052.stdout: "options": "recursive,force" 2024-06-10T03:02:00.718 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:02:00.718 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:02:00.718 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-10T03:02:00.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.711+0000 7f26b5437700 1 -- 172.21.15.52:0/836353904 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f26980a2ca0 msgr2=0x7f26980a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:02:00.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.711+0000 7f26b5437700 1 --2- 172.21.15.52:0/836353904 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f26980a2ca0 0x7f26980a5160 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f26a0000c00 tx=0x7f26a0005fb0 comp rx=0 tx=0).stop 2024-06-10T03:02:00.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.711+0000 7f26b5437700 1 -- 172.21.15.52:0/836353904 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f26b0103150 msgr2=0x7f26b019ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:02:00.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.711+0000 7f26b5437700 1 --2- 172.21.15.52:0/836353904 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f26b0103150 0x7f26b019ed90 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f26a400e9a0 tx=0x7f26a400ecb0 comp rx=0 tx=0).stop 2024-06-10T03:02:00.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.711+0000 7f26b5437700 1 -- 172.21.15.52:0/836353904 shutdown_connections 2024-06-10T03:02:00.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.711+0000 7f26b5437700 1 --2- 172.21.15.52:0/836353904 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f26b0107870 0x7f26b0198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:00.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.711+0000 7f26b5437700 1 --2- 172.21.15.52:0/836353904 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f26b01027a0 0x7f26b019e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:00.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.711+0000 7f26b5437700 1 --2- 172.21.15.52:0/836353904 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f26980a2ca0 0x7f26980a5160 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:00.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.711+0000 7f26b5437700 1 --2- 172.21.15.52:0/836353904 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f26b0103150 0x7f26b019ed90 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:00.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.711+0000 7f26b5437700 1 -- 172.21.15.52:0/836353904 >> 172.21.15.52:0/836353904 conn(0x7f26b0079570 msgr2=0x7f26b0101d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:02:00.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.711+0000 7f26b5437700 1 -- 172.21.15.52:0/836353904 shutdown_connections 2024-06-10T03:02:00.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:00.711+0000 7f26b5437700 1 -- 172.21.15.52:0/836353904 wait complete. 2024-06-10T03:02:00.854 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:02:00 smithi188 ceph-mon[36599]: pgmap v668: 129 pgs: 129 active+clean; 606 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.9 MiB/s wr, 877 op/s 2024-06-10T03:02:00.888 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:02:00 smithi120 ceph-mon[36634]: pgmap v668: 129 pgs: 129 active+clean; 606 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.9 MiB/s wr, 877 op/s 2024-06-10T03:02:01.742 INFO:tasks.cephfs.filesystem:scrub status for tag:cde34f71-67b8-4825-a70c-65a33edb0bcd - {'path': '/', 'tag': 'cde34f71-67b8-4825-a70c-65a33edb0bcd', 'options': 'recursive,force'} 2024-06-10T03:02:02.888 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:02:02 smithi120 ceph-mon[36634]: pgmap v669: 129 pgs: 129 active+clean; 600 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.9 MiB/s wr, 639 op/s 2024-06-10T03:02:02.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:02:02 smithi188 ceph-mon[36599]: pgmap v669: 129 pgs: 129 active+clean; 600 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.9 MiB/s wr, 639 op/s 2024-06-10T03:02:02.977 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:02:02 smithi052 ceph-mon[32362]: pgmap v669: 129 pgs: 129 active+clean; 600 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.9 MiB/s wr, 639 op/s 2024-06-10T03:02:04.782 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:02:04 smithi052 ceph-mon[32362]: pgmap v670: 129 pgs: 129 active+clean; 562 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.0 MiB/s wr, 781 op/s 2024-06-10T03:02:04.783 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:02:04 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T03:02:04.784 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:02:04 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T03:02:04.784 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:02:04 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T03:02:04.888 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:02:04 smithi120 ceph-mon[36634]: pgmap v670: 129 pgs: 129 active+clean; 562 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.0 MiB/s wr, 781 op/s 2024-06-10T03:02:04.888 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:02:04 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T03:02:04.888 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:02:04 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T03:02:04.889 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:02:04 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T03:02:04.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:02:04 smithi188 ceph-mon[36599]: pgmap v670: 129 pgs: 129 active+clean; 562 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.0 MiB/s wr, 781 op/s 2024-06-10T03:02:04.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:02:04 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T03:02:04.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:02:04 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T03:02:04.954 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:02:04 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T03:02:06.888 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:02:06 smithi120 ceph-mon[36634]: pgmap v671: 129 pgs: 129 active+clean; 557 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.1 MiB/s wr, 643 op/s 2024-06-10T03:02:06.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:02:06 smithi188 ceph-mon[36599]: pgmap v671: 129 pgs: 129 active+clean; 557 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.1 MiB/s wr, 643 op/s 2024-06-10T03:02:06.976 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:02:06 smithi052 ceph-mon[32362]: pgmap v671: 129 pgs: 129 active+clean; 557 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.1 MiB/s wr, 643 op/s 2024-06-10T03:02:08.888 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:02:08 smithi120 ceph-mon[36634]: pgmap v672: 129 pgs: 129 active+clean; 549 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.2 MiB/s wr, 777 op/s 2024-06-10T03:02:08.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:02:08 smithi188 ceph-mon[36599]: pgmap v672: 129 pgs: 129 active+clean; 549 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.2 MiB/s wr, 777 op/s 2024-06-10T03:02:08.976 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:02:08 smithi052 ceph-mon[32362]: pgmap v672: 129 pgs: 129 active+clean; 549 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.2 MiB/s wr, 777 op/s 2024-06-10T03:02:10.888 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:02:10 smithi120 ceph-mon[36634]: pgmap v673: 129 pgs: 129 active+clean; 539 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.3 MiB/s wr, 905 op/s 2024-06-10T03:02:10.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:02:10 smithi188 ceph-mon[36599]: pgmap v673: 129 pgs: 129 active+clean; 539 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.3 MiB/s wr, 905 op/s 2024-06-10T03:02:10.976 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:02:10 smithi052 ceph-mon[32362]: pgmap v673: 129 pgs: 129 active+clean; 539 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.3 MiB/s wr, 905 op/s 2024-06-10T03:02:12.888 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:02:12 smithi120 ceph-mon[36634]: pgmap v674: 129 pgs: 129 active+clean; 534 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 839 KiB/s wr, 836 op/s 2024-06-10T03:02:12.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:02:12 smithi188 ceph-mon[36599]: pgmap v674: 129 pgs: 129 active+clean; 534 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 839 KiB/s wr, 836 op/s 2024-06-10T03:02:12.977 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:02:12 smithi052 ceph-mon[32362]: pgmap v674: 129 pgs: 129 active+clean; 534 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 839 KiB/s wr, 836 op/s 2024-06-10T03:02:14.888 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:02:14 smithi120 ceph-mon[36634]: pgmap v675: 129 pgs: 129 active+clean; 524 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.1 MiB/s wr, 1.03k op/s 2024-06-10T03:02:14.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:02:14 smithi188 ceph-mon[36599]: pgmap v675: 129 pgs: 129 active+clean; 524 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.1 MiB/s wr, 1.03k op/s 2024-06-10T03:02:14.977 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:02:14 smithi052 ceph-mon[32362]: pgmap v675: 129 pgs: 129 active+clean; 524 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.1 MiB/s wr, 1.03k op/s 2024-06-10T03:02:15.888 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:02:15 smithi120 ceph-mon[36634]: scrub summary: idle+waiting paths [/] 2024-06-10T03:02:15.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:02:15 smithi188 ceph-mon[36599]: scrub summary: idle+waiting paths [/] 2024-06-10T03:02:15.976 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:02:15 smithi052 ceph-mon[32362]: scrub summary: idle+waiting paths [/] 2024-06-10T03:02:16.888 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:02:16 smithi120 ceph-mon[36634]: pgmap v676: 129 pgs: 129 active+clean; 525 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.0 MiB/s wr, 854 op/s 2024-06-10T03:02:16.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:02:16 smithi188 ceph-mon[36599]: pgmap v676: 129 pgs: 129 active+clean; 525 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.0 MiB/s wr, 854 op/s 2024-06-10T03:02:16.977 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:02:16 smithi052 ceph-mon[32362]: pgmap v676: 129 pgs: 129 active+clean; 525 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.0 MiB/s wr, 854 op/s 2024-06-10T03:02:18.888 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:02:18 smithi120 ceph-mon[36634]: pgmap v677: 129 pgs: 129 active+clean; 502 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.5 MiB/s wr, 919 op/s 2024-06-10T03:02:18.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:02:18 smithi188 ceph-mon[36599]: pgmap v677: 129 pgs: 129 active+clean; 502 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.5 MiB/s wr, 919 op/s 2024-06-10T03:02:18.976 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:02:18 smithi052 ceph-mon[32362]: pgmap v677: 129 pgs: 129 active+clean; 502 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.5 MiB/s wr, 919 op/s 2024-06-10T03:02:19.888 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:02:19 smithi120 ceph-mon[36634]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-10T03:02:19.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:02:19 smithi188 ceph-mon[36599]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-10T03:02:19.976 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:02:19 smithi052 ceph-mon[32362]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-10T03:02:20.890 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:02:20 smithi120 ceph-mon[36634]: pgmap v678: 129 pgs: 129 active+clean; 482 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.1 MiB/s wr, 926 op/s 2024-06-10T03:02:20.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:02:20 smithi188 ceph-mon[36599]: pgmap v678: 129 pgs: 129 active+clean; 482 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.1 MiB/s wr, 926 op/s 2024-06-10T03:02:20.976 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:02:20 smithi052 ceph-mon[32362]: pgmap v678: 129 pgs: 129 active+clean; 482 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.1 MiB/s wr, 926 op/s 2024-06-10T03:02:21.888 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:02:21 smithi120 ceph-mon[36634]: scrub summary: idle 2024-06-10T03:02:21.888 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:02:21 smithi120 ceph-mon[36634]: scrub summary: idle 2024-06-10T03:02:21.888 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:02:21 smithi120 ceph-mon[36634]: scrub summary: idle 2024-06-10T03:02:21.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:02:21 smithi188 ceph-mon[36599]: scrub summary: idle 2024-06-10T03:02:21.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:02:21 smithi188 ceph-mon[36599]: scrub summary: idle 2024-06-10T03:02:21.954 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:02:21 smithi188 ceph-mon[36599]: scrub summary: idle 2024-06-10T03:02:21.976 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:02:21 smithi052 ceph-mon[32362]: scrub summary: idle 2024-06-10T03:02:21.977 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:02:21 smithi052 ceph-mon[32362]: scrub summary: idle 2024-06-10T03:02:21.977 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:02:21 smithi052 ceph-mon[32362]: scrub summary: idle 2024-06-10T03:02:22.888 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:02:22 smithi120 ceph-mon[36634]: scrub summary: idle 2024-06-10T03:02:22.888 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:02:22 smithi120 ceph-mon[36634]: scrub summary: idle 2024-06-10T03:02:22.889 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:02:22 smithi120 ceph-mon[36634]: pgmap v679: 129 pgs: 129 active+clean; 478 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.0 MiB/s wr, 734 op/s 2024-06-10T03:02:22.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:02:22 smithi188 ceph-mon[36599]: scrub summary: idle 2024-06-10T03:02:22.954 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:02:22 smithi188 ceph-mon[36599]: scrub summary: idle 2024-06-10T03:02:22.954 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:02:22 smithi188 ceph-mon[36599]: pgmap v679: 129 pgs: 129 active+clean; 478 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.0 MiB/s wr, 734 op/s 2024-06-10T03:02:22.976 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:02:22 smithi052 ceph-mon[32362]: scrub summary: idle 2024-06-10T03:02:22.977 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:02:22 smithi052 ceph-mon[32362]: scrub summary: idle 2024-06-10T03:02:22.977 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:02:22 smithi052 ceph-mon[32362]: pgmap v679: 129 pgs: 129 active+clean; 478 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.0 MiB/s wr, 734 op/s 2024-06-10T03:02:23.888 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:02:23 smithi120 ceph-mon[36634]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-10T03:02:23.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:02:23 smithi188 ceph-mon[36599]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-10T03:02:23.976 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:02:23 smithi052 ceph-mon[32362]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-10T03:02:24.878 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:02:24 smithi052 ceph-mon[32362]: pgmap v680: 129 pgs: 129 active+clean; 471 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.5 MiB/s wr, 809 op/s 2024-06-10T03:02:24.888 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:02:24 smithi120 ceph-mon[36634]: pgmap v680: 129 pgs: 129 active+clean; 471 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.5 MiB/s wr, 809 op/s 2024-06-10T03:02:24.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:02:24 smithi188 ceph-mon[36599]: pgmap v680: 129 pgs: 129 active+clean; 471 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.5 MiB/s wr, 809 op/s 2024-06-10T03:02:26.889 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:02:26 smithi120 ceph-mon[36634]: pgmap v681: 129 pgs: 129 active+clean; 467 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 611 op/s 2024-06-10T03:02:26.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:02:26 smithi188 ceph-mon[36599]: pgmap v681: 129 pgs: 129 active+clean; 467 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 611 op/s 2024-06-10T03:02:26.976 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:02:26 smithi052 ceph-mon[32362]: pgmap v681: 129 pgs: 129 active+clean; 467 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 611 op/s 2024-06-10T03:02:28.888 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:02:28 smithi120 ceph-mon[36634]: pgmap v682: 129 pgs: 129 active+clean; 461 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 661 op/s 2024-06-10T03:02:28.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:02:28 smithi188 ceph-mon[36599]: pgmap v682: 129 pgs: 129 active+clean; 461 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 661 op/s 2024-06-10T03:02:28.976 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:02:28 smithi052 ceph-mon[32362]: pgmap v682: 129 pgs: 129 active+clean; 461 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 661 op/s 2024-06-10T03:02:30.888 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:02:30 smithi120 ceph-mon[36634]: pgmap v683: 129 pgs: 129 active+clean; 446 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 667 op/s 2024-06-10T03:02:30.954 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:02:30 smithi188 ceph-mon[36599]: pgmap v683: 129 pgs: 129 active+clean; 446 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 667 op/s 2024-06-10T03:02:30.977 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:02:30 smithi052 ceph-mon[32362]: pgmap v683: 129 pgs: 129 active+clean; 446 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 667 op/s 2024-06-10T03:02:31.743 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-10T03:02:31.996 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T03:02:32.727 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:02:32 smithi052 ceph-mon[32362]: pgmap v684: 129 pgs: 129 active+clean; 441 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 526 op/s 2024-06-10T03:02:32.888 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:02:32 smithi120 ceph-mon[36634]: pgmap v684: 129 pgs: 129 active+clean; 441 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 526 op/s 2024-06-10T03:02:32.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:02:32 smithi188 ceph-mon[36599]: pgmap v684: 129 pgs: 129 active+clean; 441 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 526 op/s 2024-06-10T03:02:33.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.751+0000 7f9f47332700 1 -- 172.21.15.52:0/1345889326 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9f40103150 msgr2=0x7f9f40107140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:02:33.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.751+0000 7f9f47332700 1 --2- 172.21.15.52:0/1345889326 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9f40103150 0x7f9f40107140 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f9f300099c0 tx=0x7f9f30009cd0 comp rx=0 tx=0).stop 2024-06-10T03:02:33.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.751+0000 7f9f47332700 1 -- 172.21.15.52:0/1345889326 shutdown_connections 2024-06-10T03:02:33.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.751+0000 7f9f47332700 1 --2- 172.21.15.52:0/1345889326 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9f40107870 0x7f9f4010f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:33.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.751+0000 7f9f47332700 1 --2- 172.21.15.52:0/1345889326 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9f401027a0 0x7f9f40102b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:33.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.751+0000 7f9f47332700 1 --2- 172.21.15.52:0/1345889326 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9f40103150 0x7f9f40107140 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:33.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.751+0000 7f9f47332700 1 -- 172.21.15.52:0/1345889326 >> 172.21.15.52:0/1345889326 conn(0x7f9f40079570 msgr2=0x7f9f40079980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:02:33.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.751+0000 7f9f47332700 1 -- 172.21.15.52:0/1345889326 shutdown_connections 2024-06-10T03:02:33.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.751+0000 7f9f47332700 1 -- 172.21.15.52:0/1345889326 wait complete. 2024-06-10T03:02:33.754 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.752+0000 7f9f47332700 1 Processor -- start 2024-06-10T03:02:33.754 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.752+0000 7f9f47332700 1 -- start start 2024-06-10T03:02:33.754 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.753+0000 7f9f47332700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9f401027a0 0x7f9f4019e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:02:33.754 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.753+0000 7f9f47332700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9f40103150 0x7f9f4019ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:02:33.754 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.753+0000 7f9f47332700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9f40107870 0x7f9f40198ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:02:33.754 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.753+0000 7f9f47332700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f9f401127f0 con 0x7f9f40103150 2024-06-10T03:02:33.754 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.753+0000 7f9f47332700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f9f40112670 con 0x7f9f401027a0 2024-06-10T03:02:33.754 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.753+0000 7f9f47332700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f9f40112970 con 0x7f9f40107870 2024-06-10T03:02:33.754 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.753+0000 7f9f458cf700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9f40107870 0x7f9f40198ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:02:33.754 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.753+0000 7f9f450ce700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9f401027a0 0x7f9f4019e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:02:33.754 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.753+0000 7f9f448cd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9f40103150 0x7f9f4019ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:02:33.754 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.753+0000 7f9f458cf700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9f40107870 0x7f9f40198ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.52:36486/0 (socket says 172.21.15.52:36486) 2024-06-10T03:02:33.754 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.753+0000 7f9f458cf700 1 -- 172.21.15.52:0/3171846830 learned_addr learned my addr 172.21.15.52:0/3171846830 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:02:33.755 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.753+0000 7f9f450ce700 1 -- 172.21.15.52:0/3171846830 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9f40107870 msgr2=0x7f9f40198ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:02:33.755 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.753+0000 7f9f450ce700 1 --2- 172.21.15.52:0/3171846830 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9f40107870 0x7f9f40198ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:33.755 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.753+0000 7f9f450ce700 1 -- 172.21.15.52:0/3171846830 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9f40103150 msgr2=0x7f9f4019ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:02:33.755 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.753+0000 7f9f450ce700 1 --2- 172.21.15.52:0/3171846830 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9f40103150 0x7f9f4019ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:33.756 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.753+0000 7f9f450ce700 1 -- 172.21.15.52:0/3171846830 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f40199230 con 0x7f9f401027a0 2024-06-10T03:02:33.756 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.753+0000 7f9f448cd700 1 --2- 172.21.15.52:0/3171846830 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9f40103150 0x7f9f4019ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T03:02:33.756 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.753+0000 7f9f450ce700 1 --2- 172.21.15.52:0/3171846830 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9f401027a0 0x7f9f4019e8c0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f9f3c00e9f0 tx=0x7f9f3c00ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:02:33.756 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.753+0000 7f9f367fc700 1 -- 172.21.15.52:0/3171846830 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f3c00c9c0 con 0x7f9f401027a0 2024-06-10T03:02:33.756 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.753+0000 7f9f367fc700 1 -- 172.21.15.52:0/3171846830 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f3c00cb60 con 0x7f9f401027a0 2024-06-10T03:02:33.756 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.754+0000 7f9f367fc700 1 -- 172.21.15.52:0/3171846830 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f3c018680 con 0x7f9f401027a0 2024-06-10T03:02:33.756 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.754+0000 7f9f47332700 1 -- 172.21.15.52:0/3171846830 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f40199520 con 0x7f9f401027a0 2024-06-10T03:02:33.756 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.754+0000 7f9f47332700 1 -- 172.21.15.52:0/3171846830 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f401a5790 con 0x7f9f401027a0 2024-06-10T03:02:33.756 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.757+0000 7f9f367fc700 1 -- 172.21.15.52:0/3171846830 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f9f3c010430 con 0x7f9f401027a0 2024-06-10T03:02:33.757 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.757+0000 7f9f367fc700 1 --2- 172.21.15.52:0/3171846830 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f9f2c0a2ca0 0x7f9f2c0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:02:33.757 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.758+0000 7f9f448cd700 1 --2- 172.21.15.52:0/3171846830 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f9f2c0a2ca0 0x7f9f2c0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:02:33.758 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.758+0000 7f9f367fc700 1 -- 172.21.15.52:0/3171846830 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f9f3c0cf470 con 0x7f9f401027a0 2024-06-10T03:02:33.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.758+0000 7f9f51bb5b80 1 Processor -- start 2024-06-10T03:02:33.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.758+0000 7f9f51bb5b80 1 -- start start 2024-06-10T03:02:33.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.758+0000 7f9f448cd700 1 --2- 172.21.15.52:0/3171846830 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f9f2c0a2ca0 0x7f9f2c0a5160 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f9f401070f0 tx=0x7f9f30005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:02:33.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.758+0000 7f9f51bb5b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559c4af1a640 0x559c4af1aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:02:33.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.758+0000 7f9f51bb5b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559c4af1af60 0x559c4af149e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:02:33.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.758+0000 7f9f51bb5b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559c4af14f20 0x559c4af15340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:02:33.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.758+0000 7f9f51bb5b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x559c4ae66810 con 0x559c4af1af60 2024-06-10T03:02:33.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.758+0000 7f9f51bb5b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x559c4ae0e720 con 0x559c4af1a640 2024-06-10T03:02:33.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.758+0000 7f9f51bb5b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x559c4ae627d0 con 0x559c4af14f20 2024-06-10T03:02:33.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.759+0000 7f9f448cd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559c4af1af60 0x559c4af149e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:02:33.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.759+0000 7f9f448cd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559c4af1af60 0x559c4af149e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54616/0 (socket says 172.21.15.52:54616) 2024-06-10T03:02:33.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.759+0000 7f9f448cd700 1 -- 172.21.15.52:0/2519706874 learned_addr learned my addr 172.21.15.52:0/2519706874 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:02:33.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.759+0000 7f9f458cf700 1 --2- 172.21.15.52:0/2519706874 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559c4af14f20 0x559c4af15340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:02:33.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.759+0000 7f9f450ce700 1 --2- 172.21.15.52:0/2519706874 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559c4af1a640 0x559c4af1aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:02:33.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.759+0000 7f9f448cd700 1 -- 172.21.15.52:0/2519706874 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559c4af14f20 msgr2=0x559c4af15340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:02:33.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.759+0000 7f9f448cd700 1 --2- 172.21.15.52:0/2519706874 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559c4af14f20 0x559c4af15340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:33.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.759+0000 7f9f448cd700 1 -- 172.21.15.52:0/2519706874 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559c4af1a640 msgr2=0x559c4af1aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:02:33.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.759+0000 7f9f448cd700 1 --2- 172.21.15.52:0/2519706874 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559c4af1a640 0x559c4af1aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:33.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.759+0000 7f9f448cd700 1 -- 172.21.15.52:0/2519706874 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c4ae04f50 con 0x559c4af1af60 2024-06-10T03:02:33.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.759+0000 7f9f448cd700 1 --2- 172.21.15.52:0/2519706874 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559c4af1af60 0x559c4af149e0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f9f300099c0 tx=0x7f9f30020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7098d266bfdc980 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:02:33.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.759+0000 7f9f2b7fe700 1 -- 172.21.15.52:0/2519706874 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f30003c40 con 0x559c4af1af60 2024-06-10T03:02:33.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.759+0000 7f9f2b7fe700 1 -- 172.21.15.52:0/2519706874 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f30003de0 con 0x559c4af1af60 2024-06-10T03:02:33.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.759+0000 7f9f2b7fe700 1 -- 172.21.15.52:0/2519706874 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f3000fc80 con 0x559c4af1af60 2024-06-10T03:02:33.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.760+0000 7f9f51bb5b80 1 -- 172.21.15.52:0/2519706874 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559c4af1af60 msgr2=0x559c4af149e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:02:33.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.760+0000 7f9f51bb5b80 1 --2- 172.21.15.52:0/2519706874 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559c4af1af60 0x559c4af149e0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f9f300099c0 tx=0x7f9f30020000 comp rx=0 tx=0).stop 2024-06-10T03:02:33.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.760+0000 7f9f51bb5b80 1 -- 172.21.15.52:0/2519706874 shutdown_connections 2024-06-10T03:02:33.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.760+0000 7f9f51bb5b80 1 --2- 172.21.15.52:0/2519706874 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559c4af14f20 0x559c4af15340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:33.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.760+0000 7f9f51bb5b80 1 --2- 172.21.15.52:0/2519706874 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559c4af1a640 0x559c4af1aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:33.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.760+0000 7f9f51bb5b80 1 --2- 172.21.15.52:0/2519706874 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559c4af1af60 0x559c4af149e0 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:33.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.760+0000 7f9f51bb5b80 1 -- 172.21.15.52:0/2519706874 >> 172.21.15.52:0/2519706874 conn(0x559c4ae3fcc0 msgr2=0x559c4af0f850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:02:33.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.760+0000 7f9f51bb5b80 1 -- 172.21.15.52:0/2519706874 shutdown_connections 2024-06-10T03:02:33.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.760+0000 7f9f51bb5b80 1 -- 172.21.15.52:0/2519706874 wait complete. 2024-06-10T03:02:33.766 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.761+0000 7f9f51bb5b80 1 -- start start 2024-06-10T03:02:33.766 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.761+0000 7f9f51bb5b80 1 Processor -- start 2024-06-10T03:02:33.766 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.761+0000 7f9f51bb5b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559c4af14f20 0x559c4af20d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:02:33.766 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.761+0000 7f9f51bb5b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559c4af1a640 0x559c4af1e060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:02:33.766 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.761+0000 7f9f51bb5b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559c4af1af60 0x559c4af1e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:02:33.766 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.761+0000 7f9f51bb5b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x559c4af21970 con 0x559c4af14f20 2024-06-10T03:02:33.766 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.761+0000 7f9f51bb5b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x559c4ae06bf0 con 0x559c4af1a640 2024-06-10T03:02:33.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.761+0000 7f9f51bb5b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x559c4af21af0 con 0x559c4af1af60 2024-06-10T03:02:33.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.762+0000 7f9f450ce700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559c4af14f20 0x559c4af20d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:02:33.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.762+0000 7f9f51bb5b80 10 client.0 _ll_register_callbacks cb 0x559c4ae00920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T03:02:33.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.762+0000 7f9f450ce700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559c4af14f20 0x559c4af20d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54626/0 (socket says 172.21.15.52:54626) 2024-06-10T03:02:33.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.762+0000 7f9f450ce700 1 -- 172.21.15.52:0/1690790079 learned_addr learned my addr 172.21.15.52:0/1690790079 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:02:33.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.762+0000 7f9f458cf700 1 --2- 172.21.15.52:0/1690790079 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559c4af1af60 0x559c4af1e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:02:33.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.762+0000 7f9f448cd700 1 --2- 172.21.15.52:0/1690790079 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559c4af1a640 0x559c4af1e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:02:33.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.762+0000 7f9f450ce700 1 -- 172.21.15.52:0/1690790079 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559c4af1af60 msgr2=0x559c4af1e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:02:33.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.762+0000 7f9f450ce700 1 --2- 172.21.15.52:0/1690790079 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559c4af1af60 0x559c4af1e5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:33.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.762+0000 7f9f450ce700 1 -- 172.21.15.52:0/1690790079 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559c4af1a640 msgr2=0x559c4af1e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:02:33.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.762+0000 7f9f450ce700 1 --2- 172.21.15.52:0/1690790079 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559c4af1a640 0x559c4af1e060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:33.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.762+0000 7f9f450ce700 1 -- 172.21.15.52:0/1690790079 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c4adb58d0 con 0x559c4af14f20 2024-06-10T03:02:33.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.762+0000 7f9f448cd700 1 --2- 172.21.15.52:0/1690790079 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559c4af1a640 0x559c4af1e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T03:02:33.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.762+0000 7f9f458cf700 1 --2- 172.21.15.52:0/1690790079 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559c4af1af60 0x559c4af1e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T03:02:33.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.762+0000 7f9f450ce700 1 --2- 172.21.15.52:0/1690790079 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559c4af14f20 0x559c4af20d60 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f9f3c0ccc10 tx=0x7f9f3c09be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbbda253cee5bca4 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:02:33.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.762+0000 7f9f28ff9700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-10T03:02:33.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.762+0000 7f9f28ff9700 1 -- 172.21.15.52:0/1690790079 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f3c0b4060 con 0x559c4af14f20 2024-06-10T03:02:33.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.762+0000 7f9f51bb5b80 1 -- 172.21.15.52:0/1690790079 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f30003de0 con 0x559c4af14f20 2024-06-10T03:02:33.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.762+0000 7f9f28ff9700 1 -- 172.21.15.52:0/1690790079 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f9f3c010720 con 0x559c4af14f20 2024-06-10T03:02:33.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.762+0000 7f9f28ff9700 1 -- 172.21.15.52:0/1690790079 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f9f3c09b920 con 0x559c4af14f20 2024-06-10T03:02:33.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.762+0000 7f9f51bb5b80 10 client.16665 fetch_fsmap learned FSMap version 41 2024-06-10T03:02:33.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.762+0000 7f9f51bb5b80 1 -- 172.21.15.52:0/1690790079 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x559c4af2c3a0 con 0x559c4af14f20 2024-06-10T03:02:33.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.763+0000 7f9f28ff9700 1 -- 172.21.15.52:0/1690790079 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f30003de0 con 0x559c4af14f20 2024-06-10T03:02:33.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.763+0000 7f9f28ff9700 1 -- 172.21.15.52:0/1690790079 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 41) ==== 8398+0+0 (secure 0 0 0) 0x7f9f3c0860b0 con 0x559c4af14f20 2024-06-10T03:02:33.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.763+0000 7f9f51bb5b80 10 client.16665 fetch_fsmap finished waiting for FSMap version 41 2024-06-10T03:02:33.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.763+0000 7f9f51bb5b80 10 client.16665 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T03:02:33.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.763+0000 7f9f51bb5b80 20 client.16665 populate_metadata read hostname 'smithi052' 2024-06-10T03:02:33.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.763+0000 7f9f51bb5b80 1 --2- 172.21.15.52:0/1690790079 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x559c4af2ed80 0x559c4af311a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:02:33.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.763+0000 7f9f51bb5b80 4 client.16665 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T03:02:33.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.763+0000 7f9f51bb5b80 1 -- 172.21.15.52:0/1690790079 --> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559c4ae73d10 con 0x559c4af2ed80 2024-06-10T03:02:33.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.763+0000 7f9f448cd700 1 --2- 172.21.15.52:0/1690790079 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x559c4af2ed80 0x559c4af311a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:02:33.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.763+0000 7f9f448cd700 1 --2- 172.21.15.52:0/1690790079 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x559c4af2ed80 0x559c4af311a0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7487dd35357a11be server_cookie=239aafadf29d7a1e in_seq=0 out_seq=0 2024-06-10T03:02:33.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.763+0000 7f9f28ff9700 10 client.16665 ms_handle_connect on v2:172.21.15.188:6838/310444836 2024-06-10T03:02:33.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.764+0000 7f9f28ff9700 1 -- 172.21.15.52:0/1690790079 <== mds.0 v2:172.21.15.188:6838/310444836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x559c4ae73d10 con 0x559c4af2ed80 2024-06-10T03:02:33.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.764+0000 7f9f28ff9700 10 client.16665 handle_command_reply: tid=0 2024-06-10T03:02:33.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.764+0000 7f9f51bb5b80 1 client.16665 shutdown 2024-06-10T03:02:33.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.764+0000 7f9f51bb5b80 1 -- 172.21.15.52:0/1690790079 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559c4af14f20 msgr2=0x559c4af20d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:02:33.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.764+0000 7f9f51bb5b80 1 --2- 172.21.15.52:0/1690790079 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559c4af14f20 0x559c4af20d60 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f9f3c0ccc10 tx=0x7f9f3c09be60 comp rx=0 tx=0).stop 2024-06-10T03:02:33.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.764+0000 7f9f51bb5b80 1 -- 172.21.15.52:0/1690790079 shutdown_connections 2024-06-10T03:02:33.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.764+0000 7f9f51bb5b80 1 --2- 172.21.15.52:0/1690790079 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x559c4af2ed80 0x559c4af311a0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:33.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.764+0000 7f9f51bb5b80 1 --2- 172.21.15.52:0/1690790079 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559c4af1af60 0x559c4af1e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:33.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.764+0000 7f9f51bb5b80 1 --2- 172.21.15.52:0/1690790079 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559c4af14f20 0x559c4af20d60 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:33.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.764+0000 7f9f51bb5b80 1 --2- 172.21.15.52:0/1690790079 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559c4af1a640 0x559c4af1e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:33.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.764+0000 7f9f51bb5b80 1 -- 172.21.15.52:0/1690790079 >> 172.21.15.52:0/1690790079 conn(0x559c4ae3fcc0 msgr2=0x559c4af0f5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:02:33.770 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.764+0000 7f9f28ff9700 0 client.16665 ms_handle_reset on v2:172.21.15.188:6838/310444836 2024-06-10T03:02:33.770 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.765+0000 7f9f51bb5b80 1 -- 172.21.15.52:0/1690790079 shutdown_connections 2024-06-10T03:02:33.770 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.765+0000 7f9f51bb5b80 1 -- 172.21.15.52:0/1690790079 wait complete. 2024-06-10T03:02:33.770 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.765+0000 7f9f51bb5b80 20 client.16665 trim_cache size 0 max 16384 2024-06-10T03:02:33.787 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.786+0000 7f9f51bb5b80 1 Processor -- start 2024-06-10T03:02:33.787 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.786+0000 7f9f51bb5b80 1 -- start start 2024-06-10T03:02:33.787 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.786+0000 7f9f51bb5b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559c4af14f20 0x559c4af149e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:02:33.787 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.786+0000 7f9f51bb5b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559c4af1a640 0x559c4af1ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:02:33.787 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.786+0000 7f9f51bb5b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559c4af1af60 0x559c4af18730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:02:33.787 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.786+0000 7f9f51bb5b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f9f401ae3b0 con 0x559c4af1a640 2024-06-10T03:02:33.787 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.786+0000 7f9f51bb5b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f9f401b0070 con 0x559c4af14f20 2024-06-10T03:02:33.787 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.786+0000 7f9f51bb5b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f9f401aeae0 con 0x559c4af1af60 2024-06-10T03:02:33.787 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.786+0000 7f9f448cd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559c4af1a640 0x559c4af1ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:02:33.787 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.786+0000 7f9f448cd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559c4af1a640 0x559c4af1ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54628/0 (socket says 172.21.15.52:54628) 2024-06-10T03:02:33.788 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.786+0000 7f9f448cd700 1 -- 172.21.15.52:0/3974793513 learned_addr learned my addr 172.21.15.52:0/3974793513 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:02:33.788 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.786+0000 7f9f458cf700 1 --2- 172.21.15.52:0/3974793513 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559c4af1af60 0x559c4af18730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:02:33.788 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.786+0000 7f9f450ce700 1 --2- 172.21.15.52:0/3974793513 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559c4af14f20 0x559c4af149e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:02:33.788 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.786+0000 7f9f448cd700 1 -- 172.21.15.52:0/3974793513 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559c4af1af60 msgr2=0x559c4af18730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:02:33.788 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.786+0000 7f9f448cd700 1 --2- 172.21.15.52:0/3974793513 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559c4af1af60 0x559c4af18730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:33.788 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.786+0000 7f9f448cd700 1 -- 172.21.15.52:0/3974793513 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559c4af14f20 msgr2=0x559c4af149e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:02:33.788 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.786+0000 7f9f448cd700 1 --2- 172.21.15.52:0/3974793513 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559c4af14f20 0x559c4af149e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:33.788 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.786+0000 7f9f448cd700 1 -- 172.21.15.52:0/3974793513 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c4af114c0 con 0x559c4af1a640 2024-06-10T03:02:33.788 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.787+0000 7f9f448cd700 1 --2- 172.21.15.52:0/3974793513 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559c4af1a640 0x559c4af1ccc0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f9f3000b910 tx=0x7f9f3001a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67c8425979d8ad12 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:02:33.788 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.787+0000 7f9f2affd700 1 -- 172.21.15.52:0/3974793513 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f3003c070 con 0x559c4af1a640 2024-06-10T03:02:33.788 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.787+0000 7f9f2affd700 1 -- 172.21.15.52:0/3974793513 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f30037340 con 0x559c4af1a640 2024-06-10T03:02:33.788 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.787+0000 7f9f2affd700 1 -- 172.21.15.52:0/3974793513 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f3002f670 con 0x559c4af1a640 2024-06-10T03:02:33.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.787+0000 7f9f51bb5b80 1 -- 172.21.15.52:0/3974793513 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559c4af1a640 msgr2=0x559c4af1ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:02:33.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.787+0000 7f9f51bb5b80 1 --2- 172.21.15.52:0/3974793513 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559c4af1a640 0x559c4af1ccc0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f9f3000b910 tx=0x7f9f3001a040 comp rx=0 tx=0).stop 2024-06-10T03:02:33.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.788+0000 7f9f51bb5b80 1 -- 172.21.15.52:0/3974793513 shutdown_connections 2024-06-10T03:02:33.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.788+0000 7f9f51bb5b80 1 --2- 172.21.15.52:0/3974793513 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559c4af1af60 0x559c4af18730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:33.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.788+0000 7f9f51bb5b80 1 --2- 172.21.15.52:0/3974793513 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559c4af14f20 0x559c4af149e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:33.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.788+0000 7f9f51bb5b80 1 --2- 172.21.15.52:0/3974793513 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559c4af1a640 0x559c4af1ccc0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:33.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.788+0000 7f9f51bb5b80 1 -- 172.21.15.52:0/3974793513 >> 172.21.15.52:0/3974793513 conn(0x559c4ae3fcc0 msgr2=0x559c4af08e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:02:33.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.788+0000 7f9f51bb5b80 1 -- 172.21.15.52:0/3974793513 shutdown_connections 2024-06-10T03:02:33.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.788+0000 7f9f51bb5b80 1 -- 172.21.15.52:0/3974793513 wait complete. 2024-06-10T03:02:33.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.788+0000 7f9f51bb5b80 1 -- start start 2024-06-10T03:02:33.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.788+0000 7f9f51bb5b80 1 Processor -- start 2024-06-10T03:02:33.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.789+0000 7f9f51bb5b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559c4af14f20 0x559c4af26de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:02:33.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.789+0000 7f9f51bb5b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559c4af1af60 0x559c4af27320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:02:33.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.789+0000 7f9f51bb5b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559c4af35b80 0x559c4af36000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:02:33.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.789+0000 7f9f51bb5b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f9f401af970 con 0x559c4af14f20 2024-06-10T03:02:33.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.789+0000 7f9f51bb5b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f9f401af270 con 0x559c4af35b80 2024-06-10T03:02:33.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.789+0000 7f9f51bb5b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f9f401ac350 con 0x559c4af1af60 2024-06-10T03:02:33.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.789+0000 7f9f51bb5b80 10 client.0 _ll_register_callbacks cb 0x559c4ae00920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T03:02:33.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.789+0000 7f9f450ce700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559c4af14f20 0x559c4af26de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:02:33.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.789+0000 7f9f450ce700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559c4af14f20 0x559c4af26de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54642/0 (socket says 172.21.15.52:54642) 2024-06-10T03:02:33.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.789+0000 7f9f450ce700 1 -- 172.21.15.52:0/1631777771 learned_addr learned my addr 172.21.15.52:0/1631777771 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:02:33.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.789+0000 7f9f458cf700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559c4af35b80 0x559c4af36000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:02:33.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.789+0000 7f9f448cd700 1 --2- 172.21.15.52:0/1631777771 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559c4af1af60 0x559c4af27320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:02:33.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.789+0000 7f9f450ce700 1 -- 172.21.15.52:0/1631777771 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559c4af1af60 msgr2=0x559c4af27320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:02:33.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.789+0000 7f9f450ce700 1 --2- 172.21.15.52:0/1631777771 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559c4af1af60 0x559c4af27320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:33.794 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.789+0000 7f9f450ce700 1 -- 172.21.15.52:0/1631777771 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559c4af35b80 msgr2=0x559c4af36000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:02:33.794 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.789+0000 7f9f450ce700 1 --2- 172.21.15.52:0/1631777771 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559c4af35b80 0x559c4af36000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:33.794 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.789+0000 7f9f450ce700 1 -- 172.21.15.52:0/1631777771 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c4af36540 con 0x559c4af14f20 2024-06-10T03:02:33.794 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.789+0000 7f9f450ce700 1 --2- 172.21.15.52:0/1631777771 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559c4af14f20 0x559c4af26de0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f9f3c09b8b0 tx=0x7f9f3c009ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca661bf141d3c6c3 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:02:33.794 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.789+0000 7f9f28ff9700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-10T03:02:33.794 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.789+0000 7f9f28ff9700 1 -- 172.21.15.52:0/1631777771 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f3c0b4040 con 0x559c4af14f20 2024-06-10T03:02:33.794 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.790+0000 7f9f28ff9700 1 -- 172.21.15.52:0/1631777771 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f9f3c010720 con 0x559c4af14f20 2024-06-10T03:02:33.794 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.790+0000 7f9f51bb5b80 1 -- 172.21.15.52:0/1631777771 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f30037340 con 0x559c4af14f20 2024-06-10T03:02:33.794 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.790+0000 7f9f28ff9700 1 -- 172.21.15.52:0/1631777771 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f9f3c09fcd0 con 0x559c4af14f20 2024-06-10T03:02:33.794 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.790+0000 7f9f51bb5b80 10 client.16671 fetch_fsmap learned FSMap version 41 2024-06-10T03:02:33.794 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.790+0000 7f9f51bb5b80 1 -- 172.21.15.52:0/1631777771 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x559c4af996b0 con 0x559c4af14f20 2024-06-10T03:02:33.794 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.790+0000 7f9f28ff9700 1 -- 172.21.15.52:0/1631777771 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f30037340 con 0x559c4af14f20 2024-06-10T03:02:33.794 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.790+0000 7f9f28ff9700 1 -- 172.21.15.52:0/1631777771 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 41) ==== 8398+0+0 (secure 0 0 0) 0x7f9f3c081050 con 0x559c4af14f20 2024-06-10T03:02:33.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.790+0000 7f9f51bb5b80 10 client.16671 fetch_fsmap finished waiting for FSMap version 41 2024-06-10T03:02:33.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.790+0000 7f9f51bb5b80 10 client.16671 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T03:02:33.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.790+0000 7f9f51bb5b80 20 client.16671 populate_metadata read hostname 'smithi052' 2024-06-10T03:02:33.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.790+0000 7f9f51bb5b80 1 --2- 172.21.15.52:0/1631777771 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x559c4af99d00 0x559c4af9c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:02:33.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.790+0000 7f9f51bb5b80 4 client.16671 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-10T03:02:33.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.790+0000 7f9f51bb5b80 1 -- 172.21.15.52:0/1631777771 --> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559c4ae6e290 con 0x559c4af99d00 2024-06-10T03:02:33.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.790+0000 7f9f448cd700 1 --2- 172.21.15.52:0/1631777771 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x559c4af99d00 0x559c4af9c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:02:33.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.791+0000 7f9f448cd700 1 --2- 172.21.15.52:0/1631777771 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x559c4af99d00 0x559c4af9c0f0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8dc7543a7bbaa562 server_cookie=ad289a5054b68b94 in_seq=0 out_seq=0 2024-06-10T03:02:33.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.791+0000 7f9f28ff9700 10 client.16671 ms_handle_connect on v2:172.21.15.188:6838/310444836 2024-06-10T03:02:33.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.791+0000 7f9f28ff9700 1 -- 172.21.15.52:0/1631777771 <== mds.0 v2:172.21.15.188:6838/310444836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x559c4ae6e290 con 0x559c4af99d00 2024-06-10T03:02:33.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.791+0000 7f9f28ff9700 10 client.16671 handle_command_reply: tid=0 2024-06-10T03:02:33.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.791+0000 7f9f51bb5b80 1 client.16671 shutdown 2024-06-10T03:02:33.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.791+0000 7f9f51bb5b80 1 -- 172.21.15.52:0/1631777771 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559c4af14f20 msgr2=0x559c4af26de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:02:33.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.791+0000 7f9f51bb5b80 1 --2- 172.21.15.52:0/1631777771 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559c4af14f20 0x559c4af26de0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f9f3c09b8b0 tx=0x7f9f3c009ef0 comp rx=0 tx=0).stop 2024-06-10T03:02:33.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.791+0000 7f9f51bb5b80 1 -- 172.21.15.52:0/1631777771 shutdown_connections 2024-06-10T03:02:33.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.791+0000 7f9f51bb5b80 1 --2- 172.21.15.52:0/1631777771 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x559c4af99d00 0x559c4af9c0f0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:33.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.791+0000 7f9f51bb5b80 1 --2- 172.21.15.52:0/1631777771 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559c4af35b80 0x559c4af36000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:33.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.791+0000 7f9f51bb5b80 1 --2- 172.21.15.52:0/1631777771 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559c4af14f20 0x559c4af26de0 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:33.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.791+0000 7f9f51bb5b80 1 --2- 172.21.15.52:0/1631777771 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559c4af1af60 0x559c4af27320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:33.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.791+0000 7f9f51bb5b80 1 -- 172.21.15.52:0/1631777771 >> 172.21.15.52:0/1631777771 conn(0x559c4ae3fcc0 msgr2=0x559c4af09510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:02:33.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.791+0000 7f9f28ff9700 0 client.16671 ms_handle_reset on v2:172.21.15.188:6838/310444836 2024-06-10T03:02:33.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.791+0000 7f9f51bb5b80 1 -- 172.21.15.52:0/1631777771 shutdown_connections 2024-06-10T03:02:33.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.791+0000 7f9f51bb5b80 1 -- 172.21.15.52:0/1631777771 wait complete. 2024-06-10T03:02:33.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.791+0000 7f9f51bb5b80 20 client.16671 trim_cache size 0 max 16384 2024-06-10T03:02:33.796 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-10T03:02:33.796 INFO:teuthology.orchestra.run.smithi052.stdout: "status": "no active scrubs running", 2024-06-10T03:02:33.797 INFO:teuthology.orchestra.run.smithi052.stdout: "scrubs": {} 2024-06-10T03:02:33.797 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-10T03:02:33.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.792+0000 7f9f47332700 1 -- 172.21.15.52:0/3171846830 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f9f2c0a2ca0 msgr2=0x7f9f2c0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:02:33.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.792+0000 7f9f47332700 1 --2- 172.21.15.52:0/3171846830 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f9f2c0a2ca0 0x7f9f2c0a5160 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f9f401070f0 tx=0x7f9f30005fb0 comp rx=0 tx=0).stop 2024-06-10T03:02:33.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.792+0000 7f9f47332700 1 -- 172.21.15.52:0/3171846830 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9f401027a0 msgr2=0x7f9f4019e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:02:33.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.792+0000 7f9f47332700 1 --2- 172.21.15.52:0/3171846830 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9f401027a0 0x7f9f4019e8c0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f9f3c00e9f0 tx=0x7f9f3c00ed00 comp rx=0 tx=0).stop 2024-06-10T03:02:33.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.792+0000 7f9f47332700 1 -- 172.21.15.52:0/3171846830 shutdown_connections 2024-06-10T03:02:33.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.792+0000 7f9f47332700 1 --2- 172.21.15.52:0/3171846830 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9f40107870 0x7f9f40198ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:33.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.792+0000 7f9f47332700 1 --2- 172.21.15.52:0/3171846830 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9f401027a0 0x7f9f4019e8c0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:33.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.792+0000 7f9f47332700 1 --2- 172.21.15.52:0/3171846830 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f9f2c0a2ca0 0x7f9f2c0a5160 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:33.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.792+0000 7f9f47332700 1 --2- 172.21.15.52:0/3171846830 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9f40103150 0x7f9f4019ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:33.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.792+0000 7f9f47332700 1 -- 172.21.15.52:0/3171846830 >> 172.21.15.52:0/3171846830 conn(0x7f9f40079570 msgr2=0x7f9f401004f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:02:33.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.792+0000 7f9f47332700 1 -- 172.21.15.52:0/3171846830 shutdown_connections 2024-06-10T03:02:33.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:33.792+0000 7f9f47332700 1 -- 172.21.15.52:0/3171846830 wait complete. 2024-06-10T03:02:34.802 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-06-10T03:02:34.802 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T03:02:34.842 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:02:34 smithi052 ceph-mon[32362]: pgmap v685: 129 pgs: 129 active+clean; 431 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 677 op/s 2024-06-10T03:02:34.888 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:02:34 smithi120 ceph-mon[36634]: pgmap v685: 129 pgs: 129 active+clean; 431 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 677 op/s 2024-06-10T03:02:34.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:02:34 smithi188 ceph-mon[36599]: pgmap v685: 129 pgs: 129 active+clean; 431 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 677 op/s 2024-06-10T03:02:35.087 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T03:02:35.888 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:02:35 smithi120 ceph-mon[36634]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-10T03:02:35.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:02:35 smithi052 ceph-mon[32362]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-10T03:02:35.954 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:02:35 smithi188 ceph-mon[36599]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-10T03:02:36.888 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:02:36 smithi120 ceph-mon[36634]: pgmap v686: 129 pgs: 129 active+clean; 426 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 567 op/s 2024-06-10T03:02:36.929 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:02:36 smithi052 ceph-mon[32362]: pgmap v686: 129 pgs: 129 active+clean; 426 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 567 op/s 2024-06-10T03:02:36.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:02:36 smithi188 ceph-mon[36599]: pgmap v686: 129 pgs: 129 active+clean; 426 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 567 op/s 2024-06-10T03:02:37.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:37.198+0000 7f6d9418b700 1 -- 172.21.15.52:0/2009672806 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6d8c107400 msgr2=0x7f6d8c1077e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:02:37.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:37.198+0000 7f6d9418b700 1 --2- 172.21.15.52:0/2009672806 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6d8c107400 0x7f6d8c1077e0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f6d84009990 tx=0x7f6d84009ca0 comp rx=0 tx=0).stop 2024-06-10T03:02:37.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:37.198+0000 7f6d9418b700 1 -- 172.21.15.52:0/2009672806 shutdown_connections 2024-06-10T03:02:37.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:37.198+0000 7f6d9418b700 1 --2- 172.21.15.52:0/2009672806 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6d8c101d50 0x7f6d8c10e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:37.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:37.198+0000 7f6d9418b700 1 --2- 172.21.15.52:0/2009672806 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6d8c107400 0x7f6d8c1077e0 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:37.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:37.198+0000 7f6d9418b700 1 --2- 172.21.15.52:0/2009672806 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6d8c101390 0x7f6d8c101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:37.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:37.198+0000 7f6d9418b700 1 -- 172.21.15.52:0/2009672806 >> 172.21.15.52:0/2009672806 conn(0x7f6d8c078340 msgr2=0x7f6d8c078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:02:37.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:37.198+0000 7f6d9418b700 1 -- 172.21.15.52:0/2009672806 shutdown_connections 2024-06-10T03:02:37.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:37.198+0000 7f6d9418b700 1 -- 172.21.15.52:0/2009672806 wait complete. 2024-06-10T03:02:37.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:37.199+0000 7f6d9418b700 1 Processor -- start 2024-06-10T03:02:37.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:37.199+0000 7f6d9418b700 1 -- start start 2024-06-10T03:02:37.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:37.199+0000 7f6d9418b700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6d8c101390 0x7f6d8c199af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:02:37.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:37.199+0000 7f6d9418b700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6d8c101d50 0x7f6d8c19a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:02:37.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:37.199+0000 7f6d9418b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6d8c107400 0x7f6d8c1a4b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:02:37.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:37.199+0000 7f6d9418b700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f6d8c111670 con 0x7f6d8c107400 2024-06-10T03:02:37.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:37.199+0000 7f6d9418b700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f6d8c1114f0 con 0x7f6d8c101390 2024-06-10T03:02:37.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:37.199+0000 7f6d9418b700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f6d8c1117f0 con 0x7f6d8c101d50 2024-06-10T03:02:37.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:37.200+0000 7f6d92728700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6d8c107400 0x7f6d8c1a4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:02:37.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:37.200+0000 7f6d92728700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6d8c107400 0x7f6d8c1a4b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54654/0 (socket says 172.21.15.52:54654) 2024-06-10T03:02:37.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:37.200+0000 7f6d92728700 1 -- 172.21.15.52:0/1182392840 learned_addr learned my addr 172.21.15.52:0/1182392840 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:02:37.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:37.200+0000 7f6d91726700 1 --2- 172.21.15.52:0/1182392840 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6d8c101d50 0x7f6d8c19a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:02:37.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:37.200+0000 7f6d91f27700 1 --2- 172.21.15.52:0/1182392840 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6d8c101390 0x7f6d8c199af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:02:37.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:37.200+0000 7f6d92728700 1 -- 172.21.15.52:0/1182392840 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6d8c101d50 msgr2=0x7f6d8c19a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:02:37.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:37.200+0000 7f6d92728700 1 --2- 172.21.15.52:0/1182392840 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6d8c101d50 0x7f6d8c19a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:37.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:37.200+0000 7f6d92728700 1 -- 172.21.15.52:0/1182392840 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6d8c101390 msgr2=0x7f6d8c199af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:02:37.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:37.200+0000 7f6d92728700 1 --2- 172.21.15.52:0/1182392840 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6d8c101390 0x7f6d8c199af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:37.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:37.200+0000 7f6d92728700 1 -- 172.21.15.52:0/1182392840 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d8c1a50b0 con 0x7f6d8c107400 2024-06-10T03:02:37.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:37.200+0000 7f6d92728700 1 --2- 172.21.15.52:0/1182392840 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6d8c107400 0x7f6d8c1a4b70 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f6d7c00d980 tx=0x7f6d7c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:02:37.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:37.200+0000 7f6d82ffd700 1 -- 172.21.15.52:0/1182392840 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d7c017070 con 0x7f6d8c107400 2024-06-10T03:02:37.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:37.200+0000 7f6d82ffd700 1 -- 172.21.15.52:0/1182392840 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d7c005020 con 0x7f6d8c107400 2024-06-10T03:02:37.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:37.200+0000 7f6d9418b700 1 -- 172.21.15.52:0/1182392840 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d8c1a52e0 con 0x7f6d8c107400 2024-06-10T03:02:37.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:37.200+0000 7f6d82ffd700 1 -- 172.21.15.52:0/1182392840 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d7c00fbf0 con 0x7f6d8c107400 2024-06-10T03:02:37.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:37.200+0000 7f6d9418b700 1 -- 172.21.15.52:0/1182392840 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d8c1a57d0 con 0x7f6d8c107400 2024-06-10T03:02:37.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:37.201+0000 7f6d9418b700 1 -- 172.21.15.52:0/1182392840 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d8c069180 con 0x7f6d8c107400 2024-06-10T03:02:37.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:37.203+0000 7f6d82ffd700 1 -- 172.21.15.52:0/1182392840 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f6d7c0051d0 con 0x7f6d8c107400 2024-06-10T03:02:37.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:37.203+0000 7f6d82ffd700 1 --2- 172.21.15.52:0/1182392840 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f6d780a2c20 0x7f6d780a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:02:37.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:37.203+0000 7f6d91f27700 1 --2- 172.21.15.52:0/1182392840 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f6d780a2c20 0x7f6d780a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:02:37.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:37.203+0000 7f6d82ffd700 1 -- 172.21.15.52:0/1182392840 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f6d7c0cf100 con 0x7f6d8c107400 2024-06-10T03:02:37.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:37.204+0000 7f6d91f27700 1 --2- 172.21.15.52:0/1182392840 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f6d780a2c20 0x7f6d780a50e0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f6d8400b2e0 tx=0x7f6d84005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:02:37.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:37.208+0000 7f6d82ffd700 1 -- 172.21.15.52:0/1182392840 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f6d7c020020 con 0x7f6d8c107400 2024-06-10T03:02:37.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:37.477+0000 7f6d9418b700 1 -- 172.21.15.52:0/1182392840 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6d8c02dca0 con 0x7f6d8c107400 2024-06-10T03:02:37.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:37.478+0000 7f6d82ffd700 1 -- 172.21.15.52:0/1182392840 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) ==== 94+0+11949 (secure 0 0 0) 0x7f6d7c09bcb0 con 0x7f6d8c107400 2024-06-10T03:02:37.478 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-10T03:02:37.478 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":42,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/889673247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":889673247},{"type":"v1","addr":"172.21.15.52:6835","nonce":889673247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/3615663511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":3615663511},{"type":"v1","addr":"172.21.15.52:6837","nonce":3615663511}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":42,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T02:50:03.889766+0000","modified":"2024-06-10T03:02:33.697178+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24484,"mds_1":24469,"mds_2":24463,"mds_3":24449,"mds_4":24428},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14571":{"gid":14571,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6839/3059599042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":3059599042},{"type":"v1","addr":"172.21.15.52:6839","nonce":3059599042}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14583":{"gid":14583,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6841/3039275521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":3039275521},{"type":"v1","addr":"172.21.15.52:6841","nonce":3039275521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24383":{"gid":24383,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6833/2609002896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":2609002896},{"type":"v1","addr":"172.21.15.120:6833","nonce":2609002896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6835/1996370288","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":1996370288},{"type":"v1","addr":"172.21.15.120:6835","nonce":1996370288}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6833/529564979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":529564979},{"type":"v1","addr":"172.21.15.188:6833","nonce":529564979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":25,"addr":"172.21.15.120:6837/4132440952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":4132440952},{"type":"v1","addr":"172.21.15.120:6837","nonce":4132440952}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"b","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.120:6839/1672436487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1672436487},{"type":"v1","addr":"172.21.15.120:6839","nonce":1672436487}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":2,"incarnation":22,"state":"up:active","state_seq":26,"addr":"172.21.15.188:6835/3904544259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":3904544259},{"type":"v1","addr":"172.21.15.188:6835","nonce":3904544259}]},"join_fscid":-1,"export_targets":[1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.188:6837/819317726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":819317726},{"type":"v1","addr":"172.21.15.188:6837","nonce":819317726}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.188:6839/310444836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":310444836},{"type":"v1","addr":"172.21.15.188:6839","nonce":310444836}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24484,"qdb_cluster":[24469,24484,24463,24428,24449]},"id":1}]} 2024-06-10T03:02:37.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:37.480+0000 7f6d9418b700 1 -- 172.21.15.52:0/1182392840 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f6d780a2c20 msgr2=0x7f6d780a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:02:37.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:37.480+0000 7f6d9418b700 1 --2- 172.21.15.52:0/1182392840 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f6d780a2c20 0x7f6d780a50e0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f6d8400b2e0 tx=0x7f6d84005960 comp rx=0 tx=0).stop 2024-06-10T03:02:37.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:37.480+0000 7f6d9418b700 1 -- 172.21.15.52:0/1182392840 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6d8c107400 msgr2=0x7f6d8c1a4b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:02:37.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:37.480+0000 7f6d9418b700 1 --2- 172.21.15.52:0/1182392840 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6d8c107400 0x7f6d8c1a4b70 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f6d7c00d980 tx=0x7f6d7c00dc90 comp rx=0 tx=0).stop 2024-06-10T03:02:37.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:37.480+0000 7f6d9418b700 1 -- 172.21.15.52:0/1182392840 shutdown_connections 2024-06-10T03:02:37.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:37.480+0000 7f6d9418b700 1 --2- 172.21.15.52:0/1182392840 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f6d780a2c20 0x7f6d780a50e0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:37.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:37.480+0000 7f6d9418b700 1 --2- 172.21.15.52:0/1182392840 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6d8c107400 0x7f6d8c1a4b70 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:37.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:37.480+0000 7f6d9418b700 1 --2- 172.21.15.52:0/1182392840 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6d8c101390 0x7f6d8c199af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:37.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:37.480+0000 7f6d9418b700 1 --2- 172.21.15.52:0/1182392840 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6d8c101d50 0x7f6d8c19a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:37.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:37.480+0000 7f6d9418b700 1 -- 172.21.15.52:0/1182392840 >> 172.21.15.52:0/1182392840 conn(0x7f6d8c078340 msgr2=0x7f6d8c100130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:02:37.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:37.480+0000 7f6d9418b700 1 -- 172.21.15.52:0/1182392840 shutdown_connections 2024-06-10T03:02:37.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:37.480+0000 7f6d9418b700 1 -- 172.21.15.52:0/1182392840 wait complete. 2024-06-10T03:02:37.481 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 42 2024-06-10T03:02:37.863 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:02:37 smithi052 ceph-mon[32362]: from='client.? 172.21.15.52:0/1182392840' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T03:02:37.888 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:02:37 smithi120 ceph-mon[36634]: from='client.? 172.21.15.52:0/1182392840' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T03:02:37.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:02:37 smithi188 ceph-mon[36599]: from='client.? 172.21.15.52:0/1182392840' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T03:02:38.275 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-06-10T03:02:38.567 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T03:02:38.890 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:02:38 smithi120 ceph-mon[36634]: pgmap v687: 129 pgs: 129 active+clean; 407 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 619 op/s 2024-06-10T03:02:38.936 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:02:38 smithi052 ceph-mon[32362]: pgmap v687: 129 pgs: 129 active+clean; 407 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 619 op/s 2024-06-10T03:02:38.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:02:38 smithi188 ceph-mon[36599]: pgmap v687: 129 pgs: 129 active+clean; 407 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 619 op/s 2024-06-10T03:02:40.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.430+0000 7f36320d1700 1 -- 172.21.15.52:0/2854563250 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f362c101d50 msgr2=0x7f362c10e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:02:40.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.430+0000 7f36320d1700 1 --2- 172.21.15.52:0/2854563250 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f362c101d50 0x7f362c10e240 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f362400b0a0 tx=0x7f362400b3b0 comp rx=0 tx=0).stop 2024-06-10T03:02:40.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.430+0000 7f36320d1700 1 -- 172.21.15.52:0/2854563250 shutdown_connections 2024-06-10T03:02:40.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.430+0000 7f36320d1700 1 --2- 172.21.15.52:0/2854563250 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f362c101d50 0x7f362c10e240 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:40.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.430+0000 7f36320d1700 1 --2- 172.21.15.52:0/2854563250 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f362c107400 0x7f362c1077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:40.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.430+0000 7f36320d1700 1 --2- 172.21.15.52:0/2854563250 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f362c101390 0x7f362c101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:40.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.430+0000 7f36320d1700 1 -- 172.21.15.52:0/2854563250 >> 172.21.15.52:0/2854563250 conn(0x7f362c078340 msgr2=0x7f362c078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:02:40.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.430+0000 7f36320d1700 1 -- 172.21.15.52:0/2854563250 shutdown_connections 2024-06-10T03:02:40.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.430+0000 7f36320d1700 1 -- 172.21.15.52:0/2854563250 wait complete. 2024-06-10T03:02:40.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.431+0000 7f36320d1700 1 Processor -- start 2024-06-10T03:02:40.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.431+0000 7f36320d1700 1 -- start start 2024-06-10T03:02:40.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.431+0000 7f36320d1700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f362c101390 0x7f362c19e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:02:40.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.431+0000 7f36320d1700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f362c101d50 0x7f362c19ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:02:40.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.431+0000 7f36320d1700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f362c107400 0x7f362c1989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:02:40.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.431+0000 7f36320d1700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f362c111670 con 0x7f362c101390 2024-06-10T03:02:40.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.431+0000 7f36320d1700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f362c1114f0 con 0x7f362c101d50 2024-06-10T03:02:40.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.431+0000 7f36320d1700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f362c1117f0 con 0x7f362c107400 2024-06-10T03:02:40.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.431+0000 7f362b7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f362c101390 0x7f362c19e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:02:40.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.431+0000 7f362b7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f362c101390 0x7f362c19e7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54684/0 (socket says 172.21.15.52:54684) 2024-06-10T03:02:40.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.431+0000 7f362b7fe700 1 -- 172.21.15.52:0/1521963006 learned_addr learned my addr 172.21.15.52:0/1521963006 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:02:40.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.431+0000 7f362bfff700 1 --2- 172.21.15.52:0/1521963006 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f362c107400 0x7f362c1989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:02:40.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.431+0000 7f3622ffd700 1 --2- 172.21.15.52:0/1521963006 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f362c101d50 0x7f362c19ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:02:40.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.432+0000 7f362bfff700 1 -- 172.21.15.52:0/1521963006 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f362c101d50 msgr2=0x7f362c19ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:02:40.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.432+0000 7f362bfff700 1 --2- 172.21.15.52:0/1521963006 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f362c101d50 0x7f362c19ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:40.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.432+0000 7f362bfff700 1 -- 172.21.15.52:0/1521963006 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f362c101390 msgr2=0x7f362c19e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:02:40.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.432+0000 7f362bfff700 1 --2- 172.21.15.52:0/1521963006 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f362c101390 0x7f362c19e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:40.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.432+0000 7f362bfff700 1 -- 172.21.15.52:0/1521963006 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f362c199150 con 0x7f362c107400 2024-06-10T03:02:40.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.432+0000 7f362b7fe700 1 --2- 172.21.15.52:0/1521963006 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f362c101390 0x7f362c19e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T03:02:40.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.432+0000 7f362bfff700 1 --2- 172.21.15.52:0/1521963006 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f362c107400 0x7f362c1989f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f362400b0a0 tx=0x7f3624004d00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:02:40.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.432+0000 7f36297fa700 1 -- 172.21.15.52:0/1521963006 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3624021070 con 0x7f362c107400 2024-06-10T03:02:40.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.432+0000 7f36320d1700 1 -- 172.21.15.52:0/1521963006 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f362c1993e0 con 0x7f362c107400 2024-06-10T03:02:40.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.432+0000 7f36297fa700 1 -- 172.21.15.52:0/1521963006 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3624007b80 con 0x7f362c107400 2024-06-10T03:02:40.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.432+0000 7f36297fa700 1 -- 172.21.15.52:0/1521963006 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3624005040 con 0x7f362c107400 2024-06-10T03:02:40.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.432+0000 7f36320d1700 1 -- 172.21.15.52:0/1521963006 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f362c1a56b0 con 0x7f362c107400 2024-06-10T03:02:40.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.434+0000 7f36297fa700 1 -- 172.21.15.52:0/1521963006 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f3624024070 con 0x7f362c107400 2024-06-10T03:02:40.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.435+0000 7f36297fa700 1 --2- 172.21.15.52:0/1521963006 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f36180a2c20 0x7f36180a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:02:40.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.435+0000 7f362b7fe700 1 --2- 172.21.15.52:0/1521963006 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f36180a2c20 0x7f36180a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:02:40.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.435+0000 7f36297fa700 1 -- 172.21.15.52:0/1521963006 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f36240d02f0 con 0x7f362c107400 2024-06-10T03:02:40.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.435+0000 7f362b7fe700 1 --2- 172.21.15.52:0/1521963006 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f36180a2c20 0x7f36180a50e0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f362c106c80 tx=0x7f361c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:02:40.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.436+0000 7f363c954b80 1 Processor -- start 2024-06-10T03:02:40.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.436+0000 7f363c954b80 1 -- start start 2024-06-10T03:02:40.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.436+0000 7f363c954b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b2331ba640 0x55b2331baa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:02:40.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.436+0000 7f363c954b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b2331baf60 0x55b2331b49e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:02:40.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.436+0000 7f363c954b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b2331b4f20 0x55b2331b5340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:02:40.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.436+0000 7f363c954b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55b233106810 con 0x55b2331baf60 2024-06-10T03:02:40.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.436+0000 7f363c954b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55b2330ae720 con 0x55b2331ba640 2024-06-10T03:02:40.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.436+0000 7f363c954b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55b2331027d0 con 0x55b2331b4f20 2024-06-10T03:02:40.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.437+0000 7f3622ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b2331baf60 0x55b2331b49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:02:40.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.437+0000 7f3622ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b2331baf60 0x55b2331b49e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54686/0 (socket says 172.21.15.52:54686) 2024-06-10T03:02:40.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.437+0000 7f3622ffd700 1 -- 172.21.15.52:0/752862064 learned_addr learned my addr 172.21.15.52:0/752862064 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:02:40.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.437+0000 7f362bfff700 1 --2- 172.21.15.52:0/752862064 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b2331b4f20 0x55b2331b5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:02:40.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.437+0000 7f362b7fe700 1 --2- 172.21.15.52:0/752862064 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b2331ba640 0x55b2331baa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:02:40.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.437+0000 7f3622ffd700 1 -- 172.21.15.52:0/752862064 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b2331b4f20 msgr2=0x55b2331b5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:02:40.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.437+0000 7f3622ffd700 1 --2- 172.21.15.52:0/752862064 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b2331b4f20 0x55b2331b5340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:40.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.437+0000 7f3622ffd700 1 -- 172.21.15.52:0/752862064 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b2331ba640 msgr2=0x55b2331baa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:02:40.439 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.437+0000 7f3622ffd700 1 --2- 172.21.15.52:0/752862064 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b2331ba640 0x55b2331baa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:40.439 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.437+0000 7f3622ffd700 1 -- 172.21.15.52:0/752862064 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b2330a4f50 con 0x55b2331baf60 2024-06-10T03:02:40.439 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.437+0000 7f362b7fe700 1 --2- 172.21.15.52:0/752862064 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b2331ba640 0x55b2331baa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T03:02:40.439 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.437+0000 7f3622ffd700 1 --2- 172.21.15.52:0/752862064 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b2331baf60 0x55b2331b49e0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f3614009d20 tx=0x7f3614011720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc0f16cad509e9e5 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:02:40.439 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.437+0000 7f3621ffb700 1 -- 172.21.15.52:0/752862064 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f361401e070 con 0x55b2331baf60 2024-06-10T03:02:40.439 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.437+0000 7f3621ffb700 1 -- 172.21.15.52:0/752862064 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3614019b10 con 0x55b2331baf60 2024-06-10T03:02:40.439 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.437+0000 7f3621ffb700 1 -- 172.21.15.52:0/752862064 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f361400eb60 con 0x55b2331baf60 2024-06-10T03:02:40.442 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.438+0000 7f363c954b80 1 -- 172.21.15.52:0/752862064 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b2331baf60 msgr2=0x55b2331b49e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:02:40.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.438+0000 7f363c954b80 1 --2- 172.21.15.52:0/752862064 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b2331baf60 0x55b2331b49e0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f3614009d20 tx=0x7f3614011720 comp rx=0 tx=0).stop 2024-06-10T03:02:40.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.438+0000 7f363c954b80 1 -- 172.21.15.52:0/752862064 shutdown_connections 2024-06-10T03:02:40.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.438+0000 7f363c954b80 1 --2- 172.21.15.52:0/752862064 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b2331b4f20 0x55b2331b5340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:40.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.438+0000 7f363c954b80 1 --2- 172.21.15.52:0/752862064 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b2331ba640 0x55b2331baa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:40.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.438+0000 7f363c954b80 1 --2- 172.21.15.52:0/752862064 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b2331baf60 0x55b2331b49e0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:40.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.438+0000 7f363c954b80 1 -- 172.21.15.52:0/752862064 >> 172.21.15.52:0/752862064 conn(0x55b2330dfcc0 msgr2=0x55b2331af850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:02:40.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.438+0000 7f363c954b80 1 -- 172.21.15.52:0/752862064 shutdown_connections 2024-06-10T03:02:40.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.438+0000 7f363c954b80 1 -- 172.21.15.52:0/752862064 wait complete. 2024-06-10T03:02:40.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.438+0000 7f363c954b80 1 -- start start 2024-06-10T03:02:40.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.439+0000 7f363c954b80 1 Processor -- start 2024-06-10T03:02:40.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.439+0000 7f363c954b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b2331b4f20 0x55b2331c0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:02:40.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.439+0000 7f363c954b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b2331ba640 0x55b2331bdfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:02:40.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.439+0000 7f363c954b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b2331baf60 0x55b2331be510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:02:40.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.439+0000 7f363c954b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55b2331c19d0 con 0x55b2331b4f20 2024-06-10T03:02:40.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.439+0000 7f363c954b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55b2330a6bf0 con 0x55b2331ba640 2024-06-10T03:02:40.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.439+0000 7f363c954b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55b2331c1b50 con 0x55b2331baf60 2024-06-10T03:02:40.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.439+0000 7f362b7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b2331b4f20 0x55b2331c0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:02:40.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.439+0000 7f362b7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b2331b4f20 0x55b2331c0cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54690/0 (socket says 172.21.15.52:54690) 2024-06-10T03:02:40.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.439+0000 7f362b7fe700 1 -- 172.21.15.52:0/1940263921 learned_addr learned my addr 172.21.15.52:0/1940263921 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:02:40.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.439+0000 7f363c954b80 10 client.0 _ll_register_callbacks cb 0x55b2330a0920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T03:02:40.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.439+0000 7f362bfff700 1 --2- 172.21.15.52:0/1940263921 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b2331baf60 0x55b2331be510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:02:40.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.439+0000 7f3622ffd700 1 --2- 172.21.15.52:0/1940263921 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b2331ba640 0x55b2331bdfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:02:40.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.439+0000 7f362b7fe700 1 -- 172.21.15.52:0/1940263921 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b2331baf60 msgr2=0x55b2331be510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:02:40.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.439+0000 7f362b7fe700 1 --2- 172.21.15.52:0/1940263921 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b2331baf60 0x55b2331be510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:40.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.439+0000 7f362b7fe700 1 -- 172.21.15.52:0/1940263921 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b2331ba640 msgr2=0x55b2331bdfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:02:40.445 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.439+0000 7f362b7fe700 1 --2- 172.21.15.52:0/1940263921 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b2331ba640 0x55b2331bdfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:40.445 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.439+0000 7f362b7fe700 1 -- 172.21.15.52:0/1940263921 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b2330558d0 con 0x55b2331b4f20 2024-06-10T03:02:40.445 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.440+0000 7f362b7fe700 1 --2- 172.21.15.52:0/1940263921 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b2331b4f20 0x55b2331c0cd0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f361c003b30 tx=0x7f361c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a9fecfbe880c7fd server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:02:40.445 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.440+0000 7f360affd700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-10T03:02:40.445 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.440+0000 7f360affd700 1 -- 172.21.15.52:0/1940263921 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f361c011e60 con 0x55b2331b4f20 2024-06-10T03:02:40.445 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.440+0000 7f360affd700 1 -- 172.21.15.52:0/1940263921 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f361c01dc50 con 0x55b2331b4f20 2024-06-10T03:02:40.445 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.440+0000 7f363c954b80 1 -- 172.21.15.52:0/1940263921 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3614019b10 con 0x55b2331b4f20 2024-06-10T03:02:40.445 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.440+0000 7f360affd700 1 -- 172.21.15.52:0/1940263921 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f361c031050 con 0x55b2331b4f20 2024-06-10T03:02:40.445 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.440+0000 7f363c954b80 10 client.16689 fetch_fsmap learned FSMap version 42 2024-06-10T03:02:40.445 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.440+0000 7f363c954b80 1 -- 172.21.15.52:0/1940263921 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55b2331cc2a0 con 0x55b2331b4f20 2024-06-10T03:02:40.445 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.440+0000 7f360affd700 1 -- 172.21.15.52:0/1940263921 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3614019b10 con 0x55b2331b4f20 2024-06-10T03:02:40.445 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.440+0000 7f360affd700 1 -- 172.21.15.52:0/1940263921 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 42) ==== 8394+0+0 (secure 0 0 0) 0x7f361c01b410 con 0x55b2331b4f20 2024-06-10T03:02:40.445 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.441+0000 7f363c954b80 10 client.16689 fetch_fsmap finished waiting for FSMap version 42 2024-06-10T03:02:40.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.441+0000 7f363c954b80 10 client.16689 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-06-10T03:02:40.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.441+0000 7f363c954b80 20 client.16689 populate_metadata read hostname 'smithi052' 2024-06-10T03:02:40.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.441+0000 7f363c954b80 1 --2- 172.21.15.52:0/1940263921 >> [v2:172.21.15.120:6836/4132440952,v1:172.21.15.120:6837/4132440952] conn(0x55b2331ce9b0 0x55b2331d0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:02:40.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.441+0000 7f363c954b80 4 client.16689 mds_command: new command op to 24428 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T03:02:40.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.441+0000 7f363c954b80 1 -- 172.21.15.52:0/1940263921 --> [v2:172.21.15.120:6836/4132440952,v1:172.21.15.120:6837/4132440952] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b233113d10 con 0x55b2331ce9b0 2024-06-10T03:02:40.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.441+0000 7f3622ffd700 1 --2- 172.21.15.52:0/1940263921 >> [v2:172.21.15.120:6836/4132440952,v1:172.21.15.120:6837/4132440952] conn(0x55b2331ce9b0 0x55b2331d0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:02:40.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.441+0000 7f3622ffd700 1 --2- 172.21.15.52:0/1940263921 >> [v2:172.21.15.120:6836/4132440952,v1:172.21.15.120:6837/4132440952] conn(0x55b2331ce9b0 0x55b2331d0e50 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=be46084ed681a433 server_cookie=8d8dad74eab95f90 in_seq=0 out_seq=0 2024-06-10T03:02:40.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.441+0000 7f360affd700 10 client.16689 ms_handle_connect on v2:172.21.15.120:6836/4132440952 2024-06-10T03:02:40.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.442+0000 7f360affd700 1 -- 172.21.15.52:0/1940263921 <== mds.4 v2:172.21.15.120:6836/4132440952 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55b233113d10 con 0x55b2331ce9b0 2024-06-10T03:02:40.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.442+0000 7f360affd700 10 client.16689 handle_command_reply: tid=0 2024-06-10T03:02:40.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.442+0000 7f363c954b80 1 client.16689 shutdown 2024-06-10T03:02:40.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.442+0000 7f363c954b80 1 -- 172.21.15.52:0/1940263921 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b2331b4f20 msgr2=0x55b2331c0cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:02:40.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.442+0000 7f363c954b80 1 --2- 172.21.15.52:0/1940263921 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b2331b4f20 0x55b2331c0cd0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f361c003b30 tx=0x7f361c0047c0 comp rx=0 tx=0).stop 2024-06-10T03:02:40.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.442+0000 7f363c954b80 1 -- 172.21.15.52:0/1940263921 shutdown_connections 2024-06-10T03:02:40.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.442+0000 7f363c954b80 1 --2- 172.21.15.52:0/1940263921 >> [v2:172.21.15.120:6836/4132440952,v1:172.21.15.120:6837/4132440952] conn(0x55b2331ce9b0 0x55b2331d0e50 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:40.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.442+0000 7f363c954b80 1 --2- 172.21.15.52:0/1940263921 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b2331baf60 0x55b2331be510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:40.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.442+0000 7f363c954b80 1 --2- 172.21.15.52:0/1940263921 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b2331b4f20 0x55b2331c0cd0 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:40.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.442+0000 7f360affd700 0 client.16689 ms_handle_reset on v2:172.21.15.120:6836/4132440952 2024-06-10T03:02:40.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.442+0000 7f363c954b80 1 --2- 172.21.15.52:0/1940263921 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b2331ba640 0x55b2331bdfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:40.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.442+0000 7f363c954b80 1 -- 172.21.15.52:0/1940263921 >> 172.21.15.52:0/1940263921 conn(0x55b2330dfcc0 msgr2=0x55b2331b39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:02:40.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.442+0000 7f363c954b80 1 -- 172.21.15.52:0/1940263921 shutdown_connections 2024-06-10T03:02:40.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.442+0000 7f363c954b80 1 -- 172.21.15.52:0/1940263921 wait complete. 2024-06-10T03:02:40.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.442+0000 7f363c954b80 20 client.16689 trim_cache size 0 max 16384 2024-06-10T03:02:40.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.458+0000 7f363c954b80 1 Processor -- start 2024-06-10T03:02:40.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.458+0000 7f363c954b80 1 -- start start 2024-06-10T03:02:40.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.458+0000 7f363c954b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b2331b4f20 0x55b2331b49e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:02:40.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.458+0000 7f363c954b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b2331ba640 0x55b2331bba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:02:40.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.458+0000 7f363c954b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b2331baf60 0x55b2331d66d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:02:40.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.458+0000 7f363c954b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f362c1ae250 con 0x55b2331baf60 2024-06-10T03:02:40.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.458+0000 7f363c954b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f362c1afee0 con 0x55b2331ba640 2024-06-10T03:02:40.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.458+0000 7f363c954b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f362c1ae980 con 0x55b2331b4f20 2024-06-10T03:02:40.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.458+0000 7f362bfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b2331baf60 0x55b2331d66d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:02:40.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.458+0000 7f362b7fe700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b2331b4f20 0x55b2331b49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:02:40.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.458+0000 7f362bfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b2331baf60 0x55b2331d66d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54692/0 (socket says 172.21.15.52:54692) 2024-06-10T03:02:40.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.458+0000 7f362bfff700 1 -- 172.21.15.52:0/3515369666 learned_addr learned my addr 172.21.15.52:0/3515369666 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:02:40.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.458+0000 7f3622ffd700 1 --2- 172.21.15.52:0/3515369666 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b2331ba640 0x55b2331bba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:02:40.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.458+0000 7f362b7fe700 1 -- 172.21.15.52:0/3515369666 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b2331ba640 msgr2=0x55b2331bba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:02:40.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.458+0000 7f362b7fe700 1 --2- 172.21.15.52:0/3515369666 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b2331ba640 0x55b2331bba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:40.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.458+0000 7f362b7fe700 1 -- 172.21.15.52:0/3515369666 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b2331baf60 msgr2=0x55b2331d66d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:02:40.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.458+0000 7f362b7fe700 1 --2- 172.21.15.52:0/3515369666 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b2331baf60 0x55b2331d66d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:40.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.458+0000 7f362b7fe700 1 -- 172.21.15.52:0/3515369666 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b2331bad70 con 0x55b2331b4f20 2024-06-10T03:02:40.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.458+0000 7f362bfff700 1 --2- 172.21.15.52:0/3515369666 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b2331baf60 0x55b2331d66d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T03:02:40.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.458+0000 7f362b7fe700 1 --2- 172.21.15.52:0/3515369666 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b2331b4f20 0x55b2331b49e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f361c009340 tx=0x7f361c009fc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2fe10d2f00cd90ff server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:02:40.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.459+0000 7f36217fa700 1 -- 172.21.15.52:0/3515369666 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f361c011e60 con 0x55b2331b4f20 2024-06-10T03:02:40.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.459+0000 7f36217fa700 1 -- 172.21.15.52:0/3515369666 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f361c01d690 con 0x55b2331b4f20 2024-06-10T03:02:40.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.459+0000 7f36217fa700 1 -- 172.21.15.52:0/3515369666 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f361c01dd40 con 0x55b2331b4f20 2024-06-10T03:02:40.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.459+0000 7f363c954b80 1 -- 172.21.15.52:0/3515369666 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b2331b4f20 msgr2=0x55b2331b49e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:02:40.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.459+0000 7f363c954b80 1 --2- 172.21.15.52:0/3515369666 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b2331b4f20 0x55b2331b49e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f361c009340 tx=0x7f361c009fc0 comp rx=0 tx=0).stop 2024-06-10T03:02:40.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.459+0000 7f363c954b80 1 -- 172.21.15.52:0/3515369666 shutdown_connections 2024-06-10T03:02:40.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.459+0000 7f363c954b80 1 --2- 172.21.15.52:0/3515369666 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b2331baf60 0x55b2331d66d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:40.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.459+0000 7f363c954b80 1 --2- 172.21.15.52:0/3515369666 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b2331b4f20 0x55b2331b49e0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:40.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.459+0000 7f363c954b80 1 --2- 172.21.15.52:0/3515369666 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b2331ba640 0x55b2331bba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:40.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.459+0000 7f363c954b80 1 -- 172.21.15.52:0/3515369666 >> 172.21.15.52:0/3515369666 conn(0x55b2330dfcc0 msgr2=0x55b2331bb340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:02:40.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.459+0000 7f363c954b80 1 -- 172.21.15.52:0/3515369666 shutdown_connections 2024-06-10T03:02:40.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.459+0000 7f363c954b80 1 -- 172.21.15.52:0/3515369666 wait complete. 2024-06-10T03:02:40.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.460+0000 7f363c954b80 1 -- start start 2024-06-10T03:02:40.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.460+0000 7f363c954b80 1 Processor -- start 2024-06-10T03:02:40.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.460+0000 7f363c954b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b2331b4f20 0x55b2331c7440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:02:40.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.460+0000 7f363c954b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b2331ba640 0x55b2331d5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:02:40.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.460+0000 7f363c954b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b2331baf60 0x55b2331d6410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:02:40.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.460+0000 7f363c954b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f362c1af7e0 con 0x55b2331ba640 2024-06-10T03:02:40.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.460+0000 7f363c954b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f362c1af0e0 con 0x55b2331baf60 2024-06-10T03:02:40.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.460+0000 7f363c954b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f362c1ac1f0 con 0x55b2331b4f20 2024-06-10T03:02:40.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.460+0000 7f363c954b80 10 client.0 _ll_register_callbacks cb 0x55b2330a0920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T03:02:40.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.460+0000 7f3622ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b2331ba640 0x55b2331d5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:02:40.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.460+0000 7f362b7fe700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b2331b4f20 0x55b2331c7440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:02:40.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.460+0000 7f362b7fe700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b2331b4f20 0x55b2331c7440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.52:36606/0 (socket says 172.21.15.52:36606) 2024-06-10T03:02:40.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.460+0000 7f362b7fe700 1 -- 172.21.15.52:0/193606514 learned_addr learned my addr 172.21.15.52:0/193606514 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:02:40.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.461+0000 7f362bfff700 1 --2- 172.21.15.52:0/193606514 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b2331baf60 0x55b2331d6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:02:40.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.461+0000 7f3622ffd700 1 -- 172.21.15.52:0/193606514 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b2331b4f20 msgr2=0x55b2331c7440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:02:40.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.461+0000 7f3622ffd700 1 --2- 172.21.15.52:0/193606514 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b2331b4f20 0x55b2331c7440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:40.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.461+0000 7f3622ffd700 1 -- 172.21.15.52:0/193606514 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b2331baf60 msgr2=0x55b2331d6410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:02:40.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.461+0000 7f3622ffd700 1 --2- 172.21.15.52:0/193606514 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b2331baf60 0x55b2331d6410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:40.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.461+0000 7f3622ffd700 1 -- 172.21.15.52:0/193606514 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b2331d6ac0 con 0x55b2331ba640 2024-06-10T03:02:40.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.461+0000 7f362bfff700 1 --2- 172.21.15.52:0/193606514 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b2331baf60 0x55b2331d6410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T03:02:40.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.461+0000 7f362b7fe700 1 --2- 172.21.15.52:0/193606514 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b2331b4f20 0x55b2331c7440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T03:02:40.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.461+0000 7f3622ffd700 1 --2- 172.21.15.52:0/193606514 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b2331ba640 0x55b2331d5ed0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f3614011a70 tx=0x7f361400e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1a09f4f89f78577 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:02:40.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.461+0000 7f360affd700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-10T03:02:40.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.461+0000 7f360affd700 1 -- 172.21.15.52:0/193606514 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3614019e60 con 0x55b2331ba640 2024-06-10T03:02:40.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.461+0000 7f363c954b80 1 -- 172.21.15.52:0/193606514 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f361c01d690 con 0x55b2331ba640 2024-06-10T03:02:40.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.461+0000 7f360affd700 1 -- 172.21.15.52:0/193606514 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f3614005040 con 0x55b2331ba640 2024-06-10T03:02:40.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.461+0000 7f360affd700 1 -- 172.21.15.52:0/193606514 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f3614038050 con 0x55b2331ba640 2024-06-10T03:02:40.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.461+0000 7f363c954b80 10 client.16695 fetch_fsmap learned FSMap version 42 2024-06-10T03:02:40.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.461+0000 7f363c954b80 1 -- 172.21.15.52:0/193606514 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55b23323d970 con 0x55b2331ba640 2024-06-10T03:02:40.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.462+0000 7f360affd700 1 -- 172.21.15.52:0/193606514 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f361c01d690 con 0x55b2331ba640 2024-06-10T03:02:40.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.462+0000 7f360affd700 1 -- 172.21.15.52:0/193606514 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 42) ==== 8394+0+0 (secure 0 0 0) 0x7f361400e790 con 0x55b2331ba640 2024-06-10T03:02:40.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.462+0000 7f363c954b80 10 client.16695 fetch_fsmap finished waiting for FSMap version 42 2024-06-10T03:02:40.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.462+0000 7f363c954b80 10 client.16695 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-06-10T03:02:40.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.462+0000 7f363c954b80 20 client.16695 populate_metadata read hostname 'smithi052' 2024-06-10T03:02:40.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.462+0000 7f363c954b80 1 --2- 172.21.15.52:0/193606514 >> [v2:172.21.15.120:6836/4132440952,v1:172.21.15.120:6837/4132440952] conn(0x55b23323df10 0x55b233240330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:02:40.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.462+0000 7f363c954b80 4 client.16695 mds_command: new command op to 24428 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T03:02:40.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.462+0000 7f363c954b80 1 -- 172.21.15.52:0/193606514 --> [v2:172.21.15.120:6836/4132440952,v1:172.21.15.120:6837/4132440952] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b23310e290 con 0x55b23323df10 2024-06-10T03:02:40.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.462+0000 7f362b7fe700 1 --2- 172.21.15.52:0/193606514 >> [v2:172.21.15.120:6836/4132440952,v1:172.21.15.120:6837/4132440952] conn(0x55b23323df10 0x55b233240330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:02:40.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.462+0000 7f360affd700 10 client.16695 ms_handle_connect on v2:172.21.15.120:6836/4132440952 2024-06-10T03:02:40.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.462+0000 7f362b7fe700 1 --2- 172.21.15.52:0/193606514 >> [v2:172.21.15.120:6836/4132440952,v1:172.21.15.120:6837/4132440952] conn(0x55b23323df10 0x55b233240330 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=66a8f8c22424dfa0 server_cookie=6a6dfa030abb8348 in_seq=0 out_seq=0 2024-06-10T03:02:40.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.463+0000 7f360affd700 1 -- 172.21.15.52:0/193606514 <== mds.4 v2:172.21.15.120:6836/4132440952 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b23310e290 con 0x55b23323df10 2024-06-10T03:02:40.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.463+0000 7f360affd700 10 client.16695 handle_command_reply: tid=0 2024-06-10T03:02:40.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.463+0000 7f363c954b80 1 client.16695 shutdown 2024-06-10T03:02:40.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.463+0000 7f363c954b80 1 -- 172.21.15.52:0/193606514 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b2331ba640 msgr2=0x55b2331d5ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:02:40.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.463+0000 7f363c954b80 1 --2- 172.21.15.52:0/193606514 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b2331ba640 0x55b2331d5ed0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f3614011a70 tx=0x7f361400e720 comp rx=0 tx=0).stop 2024-06-10T03:02:40.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.463+0000 7f363c954b80 1 -- 172.21.15.52:0/193606514 shutdown_connections 2024-06-10T03:02:40.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.463+0000 7f363c954b80 1 --2- 172.21.15.52:0/193606514 >> [v2:172.21.15.120:6836/4132440952,v1:172.21.15.120:6837/4132440952] conn(0x55b23323df10 0x55b233240330 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:40.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.463+0000 7f363c954b80 1 --2- 172.21.15.52:0/193606514 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b2331baf60 0x55b2331d6410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:40.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.463+0000 7f360affd700 0 client.16695 ms_handle_reset on v2:172.21.15.120:6836/4132440952 2024-06-10T03:02:40.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.463+0000 7f363c954b80 1 --2- 172.21.15.52:0/193606514 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b2331b4f20 0x55b2331c7440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:40.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.463+0000 7f363c954b80 1 --2- 172.21.15.52:0/193606514 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b2331ba640 0x55b2331d5ed0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:40.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.463+0000 7f363c954b80 1 -- 172.21.15.52:0/193606514 >> 172.21.15.52:0/193606514 conn(0x55b2330dfcc0 msgr2=0x55b2331bb340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:02:40.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.463+0000 7f363c954b80 1 -- 172.21.15.52:0/193606514 shutdown_connections 2024-06-10T03:02:40.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.463+0000 7f363c954b80 1 -- 172.21.15.52:0/193606514 wait complete. 2024-06-10T03:02:40.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.463+0000 7f363c954b80 20 client.16695 trim_cache size 0 max 16384 2024-06-10T03:02:40.469 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-06-10T03:02:40.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.463+0000 7f36320d1700 1 -- 172.21.15.52:0/1521963006 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f36180a2c20 msgr2=0x7f36180a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:02:40.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.463+0000 7f36320d1700 1 --2- 172.21.15.52:0/1521963006 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f36180a2c20 0x7f36180a50e0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f362c106c80 tx=0x7f361c009260 comp rx=0 tx=0).stop 2024-06-10T03:02:40.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.464+0000 7f36320d1700 1 -- 172.21.15.52:0/1521963006 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f362c107400 msgr2=0x7f362c1989f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:02:40.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.464+0000 7f36320d1700 1 --2- 172.21.15.52:0/1521963006 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f362c107400 0x7f362c1989f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f362400b0a0 tx=0x7f3624004d00 comp rx=0 tx=0).stop 2024-06-10T03:02:40.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.464+0000 7f36320d1700 1 -- 172.21.15.52:0/1521963006 shutdown_connections 2024-06-10T03:02:40.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.464+0000 7f36320d1700 1 --2- 172.21.15.52:0/1521963006 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f362c107400 0x7f362c1989f0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:40.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.464+0000 7f36320d1700 1 --2- 172.21.15.52:0/1521963006 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f36180a2c20 0x7f36180a50e0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:40.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.464+0000 7f36320d1700 1 --2- 172.21.15.52:0/1521963006 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f362c101390 0x7f362c19e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:40.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.464+0000 7f36320d1700 1 --2- 172.21.15.52:0/1521963006 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f362c101d50 0x7f362c19ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:40.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.464+0000 7f36320d1700 1 -- 172.21.15.52:0/1521963006 >> 172.21.15.52:0/1521963006 conn(0x7f362c078340 msgr2=0x7f362c1005d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:02:40.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.464+0000 7f36320d1700 1 -- 172.21.15.52:0/1521963006 shutdown_connections 2024-06-10T03:02:40.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:40.464+0000 7f36320d1700 1 -- 172.21.15.52:0/1521963006 wait complete. 2024-06-10T03:02:40.888 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:02:40 smithi120 ceph-mon[36634]: pgmap v688: 129 pgs: 129 active+clean; 373 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 668 op/s 2024-06-10T03:02:40.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:02:40 smithi188 ceph-mon[36599]: pgmap v688: 129 pgs: 129 active+clean; 373 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 668 op/s 2024-06-10T03:02:40.976 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:02:40 smithi052 ceph-mon[32362]: pgmap v688: 129 pgs: 129 active+clean; 373 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 668 op/s 2024-06-10T03:02:41.115 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-06-10T03:02:41.412 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T03:02:42.677 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:02:42 smithi052 ceph-mon[32362]: pgmap v689: 129 pgs: 129 active+clean; 373 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 568 op/s 2024-06-10T03:02:42.888 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:02:42 smithi120 ceph-mon[36634]: pgmap v689: 129 pgs: 129 active+clean; 373 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 568 op/s 2024-06-10T03:02:42.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:02:42 smithi188 ceph-mon[36599]: pgmap v689: 129 pgs: 129 active+clean; 373 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 568 op/s 2024-06-10T03:02:43.003 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.002+0000 7fcb6c4cc700 1 -- 172.21.15.52:0/1823394539 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcb64102ed0 msgr2=0x7fcb6410f360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:02:43.003 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.002+0000 7fcb6c4cc700 1 --2- 172.21.15.52:0/1823394539 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcb64102ed0 0x7fcb6410f360 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fcb5c00b0a0 tx=0x7fcb5c00b3b0 comp rx=0 tx=0).stop 2024-06-10T03:02:43.003 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.002+0000 7fcb6c4cc700 1 -- 172.21.15.52:0/1823394539 shutdown_connections 2024-06-10T03:02:43.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.002+0000 7fcb6c4cc700 1 --2- 172.21.15.52:0/1823394539 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcb64102ed0 0x7fcb6410f360 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:43.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.002+0000 7fcb6c4cc700 1 --2- 172.21.15.52:0/1823394539 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcb64108520 0x7fcb64108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:43.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.002+0000 7fcb6c4cc700 1 --2- 172.21.15.52:0/1823394539 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcb64102510 0x7fcb64102990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:43.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.002+0000 7fcb6c4cc700 1 -- 172.21.15.52:0/1823394539 >> 172.21.15.52:0/1823394539 conn(0x7fcb64079560 msgr2=0x7fcb64079970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:02:43.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.003+0000 7fcb6c4cc700 1 -- 172.21.15.52:0/1823394539 shutdown_connections 2024-06-10T03:02:43.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.003+0000 7fcb6c4cc700 1 -- 172.21.15.52:0/1823394539 wait complete. 2024-06-10T03:02:43.006 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.003+0000 7fcb6c4cc700 1 Processor -- start 2024-06-10T03:02:43.006 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.004+0000 7fcb6c4cc700 1 -- start start 2024-06-10T03:02:43.006 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.004+0000 7fcb6c4cc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcb64102510 0x7fcb6419e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:02:43.006 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.004+0000 7fcb6c4cc700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcb64102ed0 0x7fcb6419ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:02:43.007 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.004+0000 7fcb6c4cc700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcb64108520 0x7fcb64198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:02:43.007 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.004+0000 7fcb6c4cc700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fcb641127b0 con 0x7fcb64102510 2024-06-10T03:02:43.007 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.004+0000 7fcb6c4cc700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fcb64112630 con 0x7fcb64102ed0 2024-06-10T03:02:43.007 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.004+0000 7fcb6c4cc700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fcb64112930 con 0x7fcb64108520 2024-06-10T03:02:43.007 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.004+0000 7fcb6a268700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcb64102510 0x7fcb6419e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:02:43.007 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.004+0000 7fcb6a268700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcb64102510 0x7fcb6419e790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:46160/0 (socket says 172.21.15.52:46160) 2024-06-10T03:02:43.007 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.004+0000 7fcb6a268700 1 -- 172.21.15.52:0/1756857531 learned_addr learned my addr 172.21.15.52:0/1756857531 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:02:43.007 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.004+0000 7fcb6aa69700 1 --2- 172.21.15.52:0/1756857531 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcb64108520 0x7fcb64198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:02:43.007 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.005+0000 7fcb6a268700 1 -- 172.21.15.52:0/1756857531 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcb64108520 msgr2=0x7fcb64198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:02:43.007 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.005+0000 7fcb6a268700 1 --2- 172.21.15.52:0/1756857531 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcb64108520 0x7fcb64198950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:43.007 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.005+0000 7fcb6a268700 1 -- 172.21.15.52:0/1756857531 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcb64102ed0 msgr2=0x7fcb6419ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:02:43.007 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.005+0000 7fcb6a268700 1 --2- 172.21.15.52:0/1756857531 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcb64102ed0 0x7fcb6419ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:43.007 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.005+0000 7fcb6a268700 1 -- 172.21.15.52:0/1756857531 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb641990b0 con 0x7fcb64102510 2024-06-10T03:02:43.007 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.005+0000 7fcb6a268700 1 --2- 172.21.15.52:0/1756857531 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcb64102510 0x7fcb6419e790 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fcb5400eb30 tx=0x7fcb5400ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:02:43.008 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.005+0000 7fcb637fe700 1 -- 172.21.15.52:0/1756857531 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb5400cc90 con 0x7fcb64102510 2024-06-10T03:02:43.008 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.005+0000 7fcb6c4cc700 1 -- 172.21.15.52:0/1756857531 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb641993a0 con 0x7fcb64102510 2024-06-10T03:02:43.008 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.005+0000 7fcb6c4cc700 1 -- 172.21.15.52:0/1756857531 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb640746c0 con 0x7fcb64102510 2024-06-10T03:02:43.008 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.006+0000 7fcb637fe700 1 -- 172.21.15.52:0/1756857531 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb5400ce30 con 0x7fcb64102510 2024-06-10T03:02:43.008 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.006+0000 7fcb637fe700 1 -- 172.21.15.52:0/1756857531 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb54010690 con 0x7fcb64102510 2024-06-10T03:02:43.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.012+0000 7fcb637fe700 1 -- 172.21.15.52:0/1756857531 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fcb54021430 con 0x7fcb64102510 2024-06-10T03:02:43.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.013+0000 7fcb637fe700 1 --2- 172.21.15.52:0/1756857531 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fcb4c0a2d20 0x7fcb4c0a51e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:02:43.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.013+0000 7fcb637fe700 1 -- 172.21.15.52:0/1756857531 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7fcb540cf5c0 con 0x7fcb64102510 2024-06-10T03:02:43.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.014+0000 7fcb76d4fb80 1 Processor -- start 2024-06-10T03:02:43.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.014+0000 7fcb69a67700 1 --2- 172.21.15.52:0/1756857531 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fcb4c0a2d20 0x7fcb4c0a51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:02:43.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.016+0000 7fcb69a67700 1 --2- 172.21.15.52:0/1756857531 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fcb4c0a2d20 0x7fcb4c0a51e0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fcb58005950 tx=0x7fcb580058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:02:43.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.019+0000 7fcb76d4fb80 1 -- start start 2024-06-10T03:02:43.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.019+0000 7fcb76d4fb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a71ac99770 0x55a71ac99b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:02:43.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.019+0000 7fcb76d4fb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a71ac9a090 0x55a71ac93a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:02:43.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.019+0000 7fcb76d4fb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a71ac93fb0 0x55a71ac943f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:02:43.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.019+0000 7fcb76d4fb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55a71ab7df60 con 0x55a71ac99770 2024-06-10T03:02:43.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.019+0000 7fcb6a268700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a71ac99770 0x55a71ac99b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:02:43.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.019+0000 7fcb6a268700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a71ac99770 0x55a71ac99b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:46164/0 (socket says 172.21.15.52:46164) 2024-06-10T03:02:43.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.019+0000 7fcb6a268700 1 -- 172.21.15.52:0/640750179 learned_addr learned my addr 172.21.15.52:0/640750179 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:02:43.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.019+0000 7fcb76d4fb80 1 -- 172.21.15.52:0/640750179 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55a71abdc850 con 0x55a71ac9a090 2024-06-10T03:02:43.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.020+0000 7fcb6aa69700 1 --2- 172.21.15.52:0/640750179 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a71ac93fb0 0x55a71ac943f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:02:43.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.020+0000 7fcb76d4fb80 1 -- 172.21.15.52:0/640750179 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55a71ab44460 con 0x55a71ac93fb0 2024-06-10T03:02:43.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.020+0000 7fcb69a67700 1 --2- 172.21.15.52:0/640750179 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a71ac9a090 0x55a71ac93a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:02:43.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.021+0000 7fcb69a67700 1 -- 172.21.15.52:0/640750179 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a71ac93fb0 msgr2=0x55a71ac943f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:02:43.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.021+0000 7fcb69a67700 1 --2- 172.21.15.52:0/640750179 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a71ac93fb0 0x55a71ac943f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:43.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.021+0000 7fcb69a67700 1 -- 172.21.15.52:0/640750179 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a71ac99770 msgr2=0x55a71ac99b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:02:43.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.021+0000 7fcb69a67700 1 --2- 172.21.15.52:0/640750179 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a71ac99770 0x55a71ac99b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:43.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.021+0000 7fcb69a67700 1 -- 172.21.15.52:0/640750179 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a71ab2b8d0 con 0x55a71ac9a090 2024-06-10T03:02:43.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.021+0000 7fcb69a67700 1 --2- 172.21.15.52:0/640750179 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a71ac9a090 0x55a71ac93a70 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fcb64079a60 tx=0x7fcb58011000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f743a6db17fa020 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:02:43.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.021+0000 7fcb6a268700 1 --2- 172.21.15.52:0/640750179 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a71ac99770 0x55a71ac99b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T03:02:43.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.021+0000 7fcb6aa69700 1 --2- 172.21.15.52:0/640750179 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a71ac93fb0 0x55a71ac943f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T03:02:43.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.021+0000 7fcb60d70700 1 -- 172.21.15.52:0/640750179 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb58020070 con 0x55a71ac9a090 2024-06-10T03:02:43.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.022+0000 7fcb60d70700 1 -- 172.21.15.52:0/640750179 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb58025460 con 0x55a71ac9a090 2024-06-10T03:02:43.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.022+0000 7fcb60d70700 1 -- 172.21.15.52:0/640750179 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb5800eea0 con 0x55a71ac9a090 2024-06-10T03:02:43.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.022+0000 7fcb76d4fb80 1 -- 172.21.15.52:0/640750179 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a71ac9a090 msgr2=0x55a71ac93a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:02:43.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.022+0000 7fcb76d4fb80 1 --2- 172.21.15.52:0/640750179 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a71ac9a090 0x55a71ac93a70 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fcb64079a60 tx=0x7fcb58011000 comp rx=0 tx=0).stop 2024-06-10T03:02:43.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.023+0000 7fcb76d4fb80 1 -- 172.21.15.52:0/640750179 shutdown_connections 2024-06-10T03:02:43.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.023+0000 7fcb76d4fb80 1 --2- 172.21.15.52:0/640750179 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a71ac93fb0 0x55a71ac943f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:43.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.023+0000 7fcb76d4fb80 1 --2- 172.21.15.52:0/640750179 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a71ac99770 0x55a71ac99b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:43.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.023+0000 7fcb76d4fb80 1 --2- 172.21.15.52:0/640750179 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a71ac9a090 0x55a71ac93a70 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:43.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.023+0000 7fcb76d4fb80 1 -- 172.21.15.52:0/640750179 >> 172.21.15.52:0/640750179 conn(0x55a71abb8080 msgr2=0x55a71ac86220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:02:43.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.023+0000 7fcb76d4fb80 1 -- 172.21.15.52:0/640750179 shutdown_connections 2024-06-10T03:02:43.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.023+0000 7fcb76d4fb80 1 -- 172.21.15.52:0/640750179 wait complete. 2024-06-10T03:02:43.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.024+0000 7fcb76d4fb80 1 -- start start 2024-06-10T03:02:43.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.024+0000 7fcb76d4fb80 1 Processor -- start 2024-06-10T03:02:43.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.024+0000 7fcb76d4fb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a71ac93fb0 0x55a71ac9fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:02:43.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.024+0000 7fcb76d4fb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a71ac99770 0x55a71ac9d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:02:43.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.024+0000 7fcb76d4fb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a71ac9a090 0x55a71ac9d610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:02:43.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.024+0000 7fcb6a268700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a71ac93fb0 0x55a71ac9fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:02:43.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.024+0000 7fcb6a268700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a71ac93fb0 0x55a71ac9fdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.52:40846/0 (socket says 172.21.15.52:40846) 2024-06-10T03:02:43.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.024+0000 7fcb6a268700 1 -- 172.21.15.52:0/4018684305 learned_addr learned my addr 172.21.15.52:0/4018684305 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:02:43.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.024+0000 7fcb76d4fb80 1 -- 172.21.15.52:0/4018684305 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55a71ab7cfc0 con 0x55a71ac9a090 2024-06-10T03:02:43.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.024+0000 7fcb76d4fb80 1 -- 172.21.15.52:0/4018684305 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55a71abd8810 con 0x55a71ac93fb0 2024-06-10T03:02:43.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.024+0000 7fcb76d4fb80 1 -- 172.21.15.52:0/4018684305 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55a71aca0a80 con 0x55a71ac99770 2024-06-10T03:02:43.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.025+0000 7fcb69a67700 1 --2- 172.21.15.52:0/4018684305 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a71ac99770 0x55a71ac9d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:02:43.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.025+0000 7fcb76d4fb80 10 client.0 _ll_register_callbacks cb 0x55a71ab725b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T03:02:43.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.025+0000 7fcb69a67700 1 -- 172.21.15.52:0/4018684305 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a71ac93fb0 msgr2=0x55a71ac9fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:02:43.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.025+0000 7fcb69a67700 1 --2- 172.21.15.52:0/4018684305 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a71ac93fb0 0x55a71ac9fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:43.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.025+0000 7fcb69a67700 1 -- 172.21.15.52:0/4018684305 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a71ac9a090 msgr2=0x55a71ac9d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:02:43.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.025+0000 7fcb69a67700 1 --2- 172.21.15.52:0/4018684305 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a71ac9a090 0x55a71ac9d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:43.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.025+0000 7fcb69a67700 1 -- 172.21.15.52:0/4018684305 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a71aba2750 con 0x55a71ac99770 2024-06-10T03:02:43.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.026+0000 7fcb69a67700 1 --2- 172.21.15.52:0/4018684305 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a71ac99770 0x55a71ac9d0d0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fcb58009fd0 tx=0x7fcb5800eea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4b8ab6a8d1925a4a server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:02:43.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.026+0000 7fcb49ffb700 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-06-10T03:02:43.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.026+0000 7fcb49ffb700 1 -- 172.21.15.52:0/4018684305 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb58020050 con 0x55a71ac99770 2024-06-10T03:02:43.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.026+0000 7fcb76d4fb80 1 -- 172.21.15.52:0/4018684305 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcb58025460 con 0x55a71ac99770 2024-06-10T03:02:43.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.026+0000 7fcb49ffb700 1 -- 172.21.15.52:0/4018684305 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7fcb5801c420 con 0x55a71ac99770 2024-06-10T03:02:43.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.027+0000 7fcb49ffb700 1 -- 172.21.15.52:0/4018684305 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fcb5801c620 con 0x55a71ac99770 2024-06-10T03:02:43.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.027+0000 7fcb76d4fb80 10 client.26254 fetch_fsmap learned FSMap version 42 2024-06-10T03:02:43.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.027+0000 7fcb76d4fb80 1 -- 172.21.15.52:0/4018684305 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55a71acbc180 con 0x55a71ac99770 2024-06-10T03:02:43.030 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.027+0000 7fcb49ffb700 1 -- 172.21.15.52:0/4018684305 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb58025460 con 0x55a71ac99770 2024-06-10T03:02:43.030 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.028+0000 7fcb49ffb700 1 -- 172.21.15.52:0/4018684305 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 42) ==== 8394+0+0 (secure 0 0 0) 0x7fcb58037030 con 0x55a71ac99770 2024-06-10T03:02:43.030 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.028+0000 7fcb76d4fb80 10 client.26254 fetch_fsmap finished waiting for FSMap version 42 2024-06-10T03:02:43.030 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.028+0000 7fcb76d4fb80 10 client.26254 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.b 2024-06-10T03:02:43.030 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.028+0000 7fcb76d4fb80 20 client.26254 populate_metadata read hostname 'smithi052' 2024-06-10T03:02:43.030 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.028+0000 7fcb76d4fb80 1 --2- 172.21.15.52:0/4018684305 >> [v2:172.21.15.120:6838/1672436487,v1:172.21.15.120:6839/1672436487] conn(0x55a71acbe700 0x55a71acc0af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:02:43.030 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.028+0000 7fcb76d4fb80 4 client.26254 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T03:02:43.030 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.028+0000 7fcb76d4fb80 1 -- 172.21.15.52:0/4018684305 --> [v2:172.21.15.120:6838/1672436487,v1:172.21.15.120:6839/1672436487] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a71abe9d50 con 0x55a71acbe700 2024-06-10T03:02:43.030 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.029+0000 7fcb6a268700 1 --2- 172.21.15.52:0/4018684305 >> [v2:172.21.15.120:6838/1672436487,v1:172.21.15.120:6839/1672436487] conn(0x55a71acbe700 0x55a71acc0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:02:43.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.029+0000 7fcb6a268700 1 --2- 172.21.15.52:0/4018684305 >> [v2:172.21.15.120:6838/1672436487,v1:172.21.15.120:6839/1672436487] conn(0x55a71acbe700 0x55a71acc0af0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=2ceebdfda83c552a server_cookie=74b97e2e2931c345 in_seq=0 out_seq=0 2024-06-10T03:02:43.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.029+0000 7fcb49ffb700 10 client.26254 ms_handle_connect on v2:172.21.15.120:6838/1672436487 2024-06-10T03:02:43.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.030+0000 7fcb49ffb700 1 -- 172.21.15.52:0/4018684305 <== mds.3 v2:172.21.15.120:6838/1672436487 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55a71abe9d50 con 0x55a71acbe700 2024-06-10T03:02:43.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.030+0000 7fcb49ffb700 10 client.26254 handle_command_reply: tid=0 2024-06-10T03:02:43.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.030+0000 7fcb76d4fb80 1 client.26254 shutdown 2024-06-10T03:02:43.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.030+0000 7fcb76d4fb80 1 -- 172.21.15.52:0/4018684305 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a71ac99770 msgr2=0x55a71ac9d0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:02:43.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.030+0000 7fcb76d4fb80 1 --2- 172.21.15.52:0/4018684305 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a71ac99770 0x55a71ac9d0d0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fcb58009fd0 tx=0x7fcb5800eea0 comp rx=0 tx=0).stop 2024-06-10T03:02:43.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.030+0000 7fcb76d4fb80 1 -- 172.21.15.52:0/4018684305 shutdown_connections 2024-06-10T03:02:43.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.030+0000 7fcb76d4fb80 1 --2- 172.21.15.52:0/4018684305 >> [v2:172.21.15.120:6838/1672436487,v1:172.21.15.120:6839/1672436487] conn(0x55a71acbe700 0x55a71acc0af0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:43.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.030+0000 7fcb49ffb700 0 client.26254 ms_handle_reset on v2:172.21.15.120:6838/1672436487 2024-06-10T03:02:43.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.030+0000 7fcb76d4fb80 1 --2- 172.21.15.52:0/4018684305 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a71ac9a090 0x55a71ac9d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:43.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.030+0000 7fcb76d4fb80 1 --2- 172.21.15.52:0/4018684305 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a71ac93fb0 0x55a71ac9fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:43.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.030+0000 7fcb76d4fb80 1 --2- 172.21.15.52:0/4018684305 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a71ac99770 0x55a71ac9d0d0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:43.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.030+0000 7fcb76d4fb80 1 -- 172.21.15.52:0/4018684305 >> 172.21.15.52:0/4018684305 conn(0x55a71abb8080 msgr2=0x55a71ac85700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:02:43.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.031+0000 7fcb76d4fb80 1 -- 172.21.15.52:0/4018684305 shutdown_connections 2024-06-10T03:02:43.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.031+0000 7fcb76d4fb80 1 -- 172.21.15.52:0/4018684305 wait complete. 2024-06-10T03:02:43.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.031+0000 7fcb76d4fb80 20 client.26254 trim_cache size 0 max 16384 2024-06-10T03:02:43.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.055+0000 7fcb76d4fb80 1 Processor -- start 2024-06-10T03:02:43.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.055+0000 7fcb76d4fb80 1 -- start start 2024-06-10T03:02:43.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.055+0000 7fcb76d4fb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a71ac93fb0 0x55a71ac97840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:02:43.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.055+0000 7fcb76d4fb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a71ac99770 0x55a71ac85e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:02:43.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.056+0000 7fcb76d4fb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a71ac9a090 0x55a71ac863c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:02:43.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.056+0000 7fcb76d4fb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fcb5000eb40 con 0x55a71ac93fb0 2024-06-10T03:02:43.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.056+0000 7fcb76d4fb80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fcb500107d0 con 0x55a71ac99770 2024-06-10T03:02:43.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.056+0000 7fcb76d4fb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fcb5000f270 con 0x55a71ac9a090 2024-06-10T03:02:43.057 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.056+0000 7fcb6a268700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a71ac93fb0 0x55a71ac97840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:02:43.057 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.056+0000 7fcb6a268700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a71ac93fb0 0x55a71ac97840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:46176/0 (socket says 172.21.15.52:46176) 2024-06-10T03:02:43.057 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.056+0000 7fcb6a268700 1 -- 172.21.15.52:0/1649245192 learned_addr learned my addr 172.21.15.52:0/1649245192 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:02:43.057 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.056+0000 7fcb6aa69700 1 --2- 172.21.15.52:0/1649245192 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a71ac9a090 0x55a71ac863c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:02:43.057 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.056+0000 7fcb69a67700 1 --2- 172.21.15.52:0/1649245192 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a71ac99770 0x55a71ac85e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:02:43.057 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.057+0000 7fcb6aa69700 1 -- 172.21.15.52:0/1649245192 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a71ac99770 msgr2=0x55a71ac85e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:02:43.057 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.057+0000 7fcb6aa69700 1 --2- 172.21.15.52:0/1649245192 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a71ac99770 0x55a71ac85e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:43.058 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.057+0000 7fcb6aa69700 1 -- 172.21.15.52:0/1649245192 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a71ac93fb0 msgr2=0x55a71ac97840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:02:43.058 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.057+0000 7fcb6aa69700 1 --2- 172.21.15.52:0/1649245192 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a71ac93fb0 0x55a71ac97840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:43.058 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.057+0000 7fcb6aa69700 1 -- 172.21.15.52:0/1649245192 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a71aca0b00 con 0x55a71ac9a090 2024-06-10T03:02:43.058 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.057+0000 7fcb6a268700 1 --2- 172.21.15.52:0/1649245192 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a71ac93fb0 0x55a71ac97840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T03:02:43.058 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.058+0000 7fcb6aa69700 1 --2- 172.21.15.52:0/1649245192 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a71ac9a090 0x55a71ac863c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fcb5c009fd0 tx=0x7fcb5c009d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e05bd84dda3fe72c server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:02:43.059 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.058+0000 7fcb4bfff700 1 -- 172.21.15.52:0/1649245192 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb5c0092e0 con 0x55a71ac9a090 2024-06-10T03:02:43.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.059+0000 7fcb4bfff700 1 -- 172.21.15.52:0/1649245192 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb5c022070 con 0x55a71ac9a090 2024-06-10T03:02:43.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.059+0000 7fcb4bfff700 1 -- 172.21.15.52:0/1649245192 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb5c007940 con 0x55a71ac9a090 2024-06-10T03:02:43.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.060+0000 7fcb76d4fb80 1 -- 172.21.15.52:0/1649245192 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a71ac9a090 msgr2=0x55a71ac863c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:02:43.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.060+0000 7fcb76d4fb80 1 --2- 172.21.15.52:0/1649245192 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a71ac9a090 0x55a71ac863c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fcb5c009fd0 tx=0x7fcb5c009d50 comp rx=0 tx=0).stop 2024-06-10T03:02:43.061 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.061+0000 7fcb76d4fb80 1 -- 172.21.15.52:0/1649245192 shutdown_connections 2024-06-10T03:02:43.061 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.061+0000 7fcb76d4fb80 1 --2- 172.21.15.52:0/1649245192 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a71ac9a090 0x55a71ac863c0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:43.062 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.061+0000 7fcb76d4fb80 1 --2- 172.21.15.52:0/1649245192 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a71ac93fb0 0x55a71ac97840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:43.062 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.061+0000 7fcb76d4fb80 1 --2- 172.21.15.52:0/1649245192 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a71ac99770 0x55a71ac85e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:43.062 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.061+0000 7fcb76d4fb80 1 -- 172.21.15.52:0/1649245192 >> 172.21.15.52:0/1649245192 conn(0x55a71abb8080 msgr2=0x55a71ac9d090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:02:43.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.064+0000 7fcb76d4fb80 1 -- 172.21.15.52:0/1649245192 shutdown_connections 2024-06-10T03:02:43.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.064+0000 7fcb76d4fb80 1 -- 172.21.15.52:0/1649245192 wait complete. 2024-06-10T03:02:43.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.065+0000 7fcb76d4fb80 1 -- start start 2024-06-10T03:02:43.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.065+0000 7fcb76d4fb80 1 Processor -- start 2024-06-10T03:02:43.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.065+0000 7fcb76d4fb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a71ac93fb0 0x55a71aca5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:02:43.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.066+0000 7fcb76d4fb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a71ac99770 0x55a71aca62e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:02:43.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.066+0000 7fcb76d4fb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a71ac9a090 0x55a71aca2d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:02:43.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.066+0000 7fcb76d4fb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fcb500100d0 con 0x55a71ac93fb0 2024-06-10T03:02:43.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.066+0000 7fcb76d4fb80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fcb5000f9d0 con 0x55a71ac9a090 2024-06-10T03:02:43.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.066+0000 7fcb76d4fb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fcb5000cae0 con 0x55a71ac99770 2024-06-10T03:02:43.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.066+0000 7fcb6a268700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a71ac93fb0 0x55a71aca5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:02:43.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.066+0000 7fcb6a268700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a71ac93fb0 0x55a71aca5da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:46182/0 (socket says 172.21.15.52:46182) 2024-06-10T03:02:43.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.066+0000 7fcb6a268700 1 -- 172.21.15.52:0/2413162200 learned_addr learned my addr 172.21.15.52:0/2413162200 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:02:43.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.066+0000 7fcb76d4fb80 10 client.0 _ll_register_callbacks cb 0x55a71ab725b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T03:02:43.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.066+0000 7fcb6aa69700 1 --2- 172.21.15.52:0/2413162200 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a71ac9a090 0x55a71aca2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:02:43.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.066+0000 7fcb6a268700 1 -- 172.21.15.52:0/2413162200 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a71ac99770 msgr2=0x55a71aca62e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:02:43.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.066+0000 7fcb6a268700 1 --2- 172.21.15.52:0/2413162200 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a71ac99770 0x55a71aca62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:43.068 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.066+0000 7fcb6a268700 1 -- 172.21.15.52:0/2413162200 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a71ac9a090 msgr2=0x55a71aca2d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:02:43.068 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.066+0000 7fcb6a268700 1 --2- 172.21.15.52:0/2413162200 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a71ac9a090 0x55a71aca2d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:43.068 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.066+0000 7fcb6a268700 1 -- 172.21.15.52:0/2413162200 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a71aca32b0 con 0x55a71ac93fb0 2024-06-10T03:02:43.068 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.066+0000 7fcb6aa69700 1 --2- 172.21.15.52:0/2413162200 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a71ac9a090 0x55a71aca2d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T03:02:43.068 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.067+0000 7fcb6a268700 1 --2- 172.21.15.52:0/2413162200 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a71ac93fb0 0x55a71aca5da0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fcb54010880 tx=0x7fcb540a4730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d76b89624c66cd2 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:02:43.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.067+0000 7fcb49ffb700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-10T03:02:43.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.067+0000 7fcb49ffb700 1 -- 172.21.15.52:0/2413162200 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb540ce040 con 0x55a71ac93fb0 2024-06-10T03:02:43.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.068+0000 7fcb49ffb700 1 -- 172.21.15.52:0/2413162200 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7fcb540ccdc0 con 0x55a71ac93fb0 2024-06-10T03:02:43.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.068+0000 7fcb76d4fb80 1 -- 172.21.15.52:0/2413162200 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcb5c022070 con 0x55a71ac93fb0 2024-06-10T03:02:43.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.068+0000 7fcb49ffb700 1 -- 172.21.15.52:0/2413162200 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fcb540ba2b0 con 0x55a71ac93fb0 2024-06-10T03:02:43.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.069+0000 7fcb76d4fb80 10 client.16707 fetch_fsmap learned FSMap version 42 2024-06-10T03:02:43.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.069+0000 7fcb76d4fb80 1 -- 172.21.15.52:0/2413162200 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55a71ac92af0 con 0x55a71ac93fb0 2024-06-10T03:02:43.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.070+0000 7fcb49ffb700 1 -- 172.21.15.52:0/2413162200 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb5c022070 con 0x55a71ac93fb0 2024-06-10T03:02:43.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.070+0000 7fcb49ffb700 1 -- 172.21.15.52:0/2413162200 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 42) ==== 8394+0+0 (secure 0 0 0) 0x7fcb5409bd80 con 0x55a71ac93fb0 2024-06-10T03:02:43.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.070+0000 7fcb76d4fb80 10 client.16707 fetch_fsmap finished waiting for FSMap version 42 2024-06-10T03:02:43.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.070+0000 7fcb76d4fb80 10 client.16707 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.b 2024-06-10T03:02:43.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.070+0000 7fcb76d4fb80 20 client.16707 populate_metadata read hostname 'smithi052' 2024-06-10T03:02:43.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.070+0000 7fcb76d4fb80 1 --2- 172.21.15.52:0/2413162200 >> [v2:172.21.15.120:6838/1672436487,v1:172.21.15.120:6839/1672436487] conn(0x55a71ac93250 0x55a71ad34130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:02:43.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.070+0000 7fcb76d4fb80 4 client.16707 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T03:02:43.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.070+0000 7fcb76d4fb80 1 -- 172.21.15.52:0/2413162200 --> [v2:172.21.15.120:6838/1672436487,v1:172.21.15.120:6839/1672436487] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a71abe42d0 con 0x55a71ac93250 2024-06-10T03:02:43.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.071+0000 7fcb69a67700 1 --2- 172.21.15.52:0/2413162200 >> [v2:172.21.15.120:6838/1672436487,v1:172.21.15.120:6839/1672436487] conn(0x55a71ac93250 0x55a71ad34130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:02:43.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.072+0000 7fcb69a67700 1 --2- 172.21.15.52:0/2413162200 >> [v2:172.21.15.120:6838/1672436487,v1:172.21.15.120:6839/1672436487] conn(0x55a71ac93250 0x55a71ad34130 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=e27549c0faa8cf37 server_cookie=26df2183eaa6e754 in_seq=0 out_seq=0 2024-06-10T03:02:43.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.072+0000 7fcb49ffb700 10 client.16707 ms_handle_connect on v2:172.21.15.120:6838/1672436487 2024-06-10T03:02:43.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.072+0000 7fcb49ffb700 1 -- 172.21.15.52:0/2413162200 <== mds.3 v2:172.21.15.120:6838/1672436487 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a71abe42d0 con 0x55a71ac93250 2024-06-10T03:02:43.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.072+0000 7fcb49ffb700 10 client.16707 handle_command_reply: tid=0 2024-06-10T03:02:43.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.072+0000 7fcb76d4fb80 1 client.16707 shutdown 2024-06-10T03:02:43.074 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.073+0000 7fcb76d4fb80 1 -- 172.21.15.52:0/2413162200 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a71ac93fb0 msgr2=0x55a71aca5da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:02:43.074 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.073+0000 7fcb76d4fb80 1 --2- 172.21.15.52:0/2413162200 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a71ac93fb0 0x55a71aca5da0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fcb54010880 tx=0x7fcb540a4730 comp rx=0 tx=0).stop 2024-06-10T03:02:43.074 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.073+0000 7fcb76d4fb80 1 -- 172.21.15.52:0/2413162200 shutdown_connections 2024-06-10T03:02:43.074 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.073+0000 7fcb76d4fb80 1 --2- 172.21.15.52:0/2413162200 >> [v2:172.21.15.120:6838/1672436487,v1:172.21.15.120:6839/1672436487] conn(0x55a71ac93250 0x55a71ad34130 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:43.074 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.073+0000 7fcb76d4fb80 1 --2- 172.21.15.52:0/2413162200 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a71ac9a090 0x55a71aca2d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:43.074 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.073+0000 7fcb76d4fb80 1 --2- 172.21.15.52:0/2413162200 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a71ac93fb0 0x55a71aca5da0 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:43.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.073+0000 7fcb76d4fb80 1 --2- 172.21.15.52:0/2413162200 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a71ac99770 0x55a71aca62e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:43.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.073+0000 7fcb76d4fb80 1 -- 172.21.15.52:0/2413162200 >> 172.21.15.52:0/2413162200 conn(0x55a71abb8080 msgr2=0x55a71ac9ccf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:02:43.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.074+0000 7fcb49ffb700 0 client.16707 ms_handle_reset on v2:172.21.15.120:6838/1672436487 2024-06-10T03:02:43.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.074+0000 7fcb76d4fb80 1 -- 172.21.15.52:0/2413162200 shutdown_connections 2024-06-10T03:02:43.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.074+0000 7fcb76d4fb80 1 -- 172.21.15.52:0/2413162200 wait complete. 2024-06-10T03:02:43.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.074+0000 7fcb76d4fb80 20 client.16707 trim_cache size 0 max 16384 2024-06-10T03:02:43.075 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-06-10T03:02:43.076 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.075+0000 7fcb6c4cc700 1 -- 172.21.15.52:0/1756857531 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fcb4c0a2d20 msgr2=0x7fcb4c0a51e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:02:43.076 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.075+0000 7fcb6c4cc700 1 --2- 172.21.15.52:0/1756857531 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fcb4c0a2d20 0x7fcb4c0a51e0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fcb58005950 tx=0x7fcb580058e0 comp rx=0 tx=0).stop 2024-06-10T03:02:43.076 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.075+0000 7fcb6c4cc700 1 -- 172.21.15.52:0/1756857531 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcb64102510 msgr2=0x7fcb6419e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:02:43.076 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.075+0000 7fcb6c4cc700 1 --2- 172.21.15.52:0/1756857531 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcb64102510 0x7fcb6419e790 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fcb5400eb30 tx=0x7fcb5400ee40 comp rx=0 tx=0).stop 2024-06-10T03:02:43.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.077+0000 7fcb6c4cc700 1 -- 172.21.15.52:0/1756857531 shutdown_connections 2024-06-10T03:02:43.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.077+0000 7fcb6c4cc700 1 --2- 172.21.15.52:0/1756857531 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcb64108520 0x7fcb64198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:43.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.077+0000 7fcb6c4cc700 1 --2- 172.21.15.52:0/1756857531 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fcb4c0a2d20 0x7fcb4c0a51e0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:43.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.077+0000 7fcb6c4cc700 1 --2- 172.21.15.52:0/1756857531 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcb64102510 0x7fcb6419e790 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:43.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.077+0000 7fcb6c4cc700 1 --2- 172.21.15.52:0/1756857531 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcb64102ed0 0x7fcb6419ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:43.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.077+0000 7fcb6c4cc700 1 -- 172.21.15.52:0/1756857531 >> 172.21.15.52:0/1756857531 conn(0x7fcb64079560 msgr2=0x7fcb64100ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:02:43.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.078+0000 7fcb6c4cc700 1 -- 172.21.15.52:0/1756857531 shutdown_connections 2024-06-10T03:02:43.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:43.078+0000 7fcb6c4cc700 1 -- 172.21.15.52:0/1756857531 wait complete. 2024-06-10T03:02:43.858 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-06-10T03:02:43.877 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 16.0 2024-06-10T03:02:43.877 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T03:02:44.136 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T03:02:44.174 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T03:02:44.865 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:02:44 smithi052 ceph-mon[32362]: pgmap v690: 129 pgs: 129 active+clean; 365 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 723 op/s 2024-06-10T03:02:44.888 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:02:44 smithi120 ceph-mon[36634]: pgmap v690: 129 pgs: 129 active+clean; 365 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 723 op/s 2024-06-10T03:02:44.954 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:02:44 smithi188 ceph-mon[36599]: pgmap v690: 129 pgs: 129 active+clean; 365 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 723 op/s 2024-06-10T03:02:46.888 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:02:46 smithi120 ceph-mon[36634]: pgmap v691: 129 pgs: 129 active+clean; 361 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 574 op/s 2024-06-10T03:02:46.942 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:02:46 smithi052 ceph-mon[32362]: pgmap v691: 129 pgs: 129 active+clean; 361 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 574 op/s 2024-06-10T03:02:46.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:02:46 smithi188 ceph-mon[36599]: pgmap v691: 129 pgs: 129 active+clean; 361 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 574 op/s 2024-06-10T03:02:48.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.201+0000 7f13e3bfa700 1 -- 172.21.15.52:0/2428645462 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f13dc102740 msgr2=0x7f13dc102b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:02:48.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.201+0000 7f13e3bfa700 1 --2- 172.21.15.52:0/2428645462 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f13dc102740 0x7f13dc102b20 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f13d80099c0 tx=0x7f13d8009cd0 comp rx=0 tx=0).stop 2024-06-10T03:02:48.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.201+0000 7f13e3bfa700 1 -- 172.21.15.52:0/2428645462 shutdown_connections 2024-06-10T03:02:48.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.201+0000 7f13e3bfa700 1 --2- 172.21.15.52:0/2428645462 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f13dc107840 0x7f13dc10f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:48.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.201+0000 7f13e3bfa700 1 --2- 172.21.15.52:0/2428645462 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f13dc102740 0x7f13dc102b20 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:48.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.201+0000 7f13e3bfa700 1 --2- 172.21.15.52:0/2428645462 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f13dc1030f0 0x7f13dc107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:48.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.201+0000 7f13e3bfa700 1 -- 172.21.15.52:0/2428645462 >> 172.21.15.52:0/2428645462 conn(0x7f13dc0795f0 msgr2=0x7f13dc079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:02:48.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.201+0000 7f13e3bfa700 1 -- 172.21.15.52:0/2428645462 shutdown_connections 2024-06-10T03:02:48.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.201+0000 7f13e3bfa700 1 -- 172.21.15.52:0/2428645462 wait complete. 2024-06-10T03:02:48.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.202+0000 7f13e3bfa700 1 Processor -- start 2024-06-10T03:02:48.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.202+0000 7f13e3bfa700 1 -- start start 2024-06-10T03:02:48.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.202+0000 7f13e3bfa700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f13dc102740 0x7f13dc074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:02:48.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.202+0000 7f13e3bfa700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f13dc1030f0 0x7f13dc074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:02:48.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.202+0000 7f13e3bfa700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f13dc107840 0x7f13dc075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:02:48.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.202+0000 7f13e3bfa700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f13dc112780 con 0x7f13dc102740 2024-06-10T03:02:48.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.202+0000 7f13e3bfa700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f13dc112600 con 0x7f13dc1030f0 2024-06-10T03:02:48.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.202+0000 7f13e3bfa700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f13dc112900 con 0x7f13dc107840 2024-06-10T03:02:48.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.202+0000 7f13e1996700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f13dc102740 0x7f13dc074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:02:48.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.202+0000 7f13e1996700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f13dc102740 0x7f13dc074790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:46208/0 (socket says 172.21.15.52:46208) 2024-06-10T03:02:48.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.202+0000 7f13e1996700 1 -- 172.21.15.52:0/2649008740 learned_addr learned my addr 172.21.15.52:0/2649008740 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:02:48.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.202+0000 7f13e2197700 1 --2- 172.21.15.52:0/2649008740 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f13dc107840 0x7f13dc075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:02:48.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.202+0000 7f13e1195700 1 --2- 172.21.15.52:0/2649008740 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f13dc1030f0 0x7f13dc074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:02:48.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.202+0000 7f13e1996700 1 -- 172.21.15.52:0/2649008740 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f13dc107840 msgr2=0x7f13dc075210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:02:48.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.202+0000 7f13e1996700 1 --2- 172.21.15.52:0/2649008740 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f13dc107840 0x7f13dc075210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:48.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.202+0000 7f13e1996700 1 -- 172.21.15.52:0/2649008740 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f13dc1030f0 msgr2=0x7f13dc074cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:02:48.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.202+0000 7f13e1996700 1 --2- 172.21.15.52:0/2649008740 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f13dc1030f0 0x7f13dc074cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:48.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.202+0000 7f13e1996700 1 -- 172.21.15.52:0/2649008740 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13dc1a7110 con 0x7f13dc102740 2024-06-10T03:02:48.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.203+0000 7f13e1996700 1 --2- 172.21.15.52:0/2649008740 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f13dc102740 0x7f13dc074790 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f13d80059d0 tx=0x7f13d800fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:02:48.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.203+0000 7f13d2ffd700 1 -- 172.21.15.52:0/2649008740 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13d8027070 con 0x7f13dc102740 2024-06-10T03:02:48.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.203+0000 7f13d2ffd700 1 -- 172.21.15.52:0/2649008740 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f13d8003680 con 0x7f13dc102740 2024-06-10T03:02:48.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.203+0000 7f13e3bfa700 1 -- 172.21.15.52:0/2649008740 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13dc1a73a0 con 0x7f13dc102740 2024-06-10T03:02:48.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.203+0000 7f13d2ffd700 1 -- 172.21.15.52:0/2649008740 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13d80173f0 con 0x7f13dc102740 2024-06-10T03:02:48.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.203+0000 7f13e3bfa700 1 -- 172.21.15.52:0/2649008740 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13dc1a77f0 con 0x7f13dc102740 2024-06-10T03:02:48.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.205+0000 7f13d2ffd700 1 -- 172.21.15.52:0/2649008740 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f13d802a070 con 0x7f13dc102740 2024-06-10T03:02:48.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.206+0000 7f13d2ffd700 1 --2- 172.21.15.52:0/2649008740 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f13c80a2c20 0x7f13c80a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:02:48.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.206+0000 7f13e1195700 1 --2- 172.21.15.52:0/2649008740 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f13c80a2c20 0x7f13c80a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:02:48.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.206+0000 7f13d2ffd700 1 -- 172.21.15.52:0/2649008740 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f13d80cff50 con 0x7f13dc102740 2024-06-10T03:02:48.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.207+0000 7f13e1195700 1 --2- 172.21.15.52:0/2649008740 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f13c80a2c20 0x7f13c80a50e0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f13dc101fd0 tx=0x7f13cc009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:02:48.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.207+0000 7f13ee47db80 1 Processor -- start 2024-06-10T03:02:48.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.207+0000 7f13ee47db80 1 -- start start 2024-06-10T03:02:48.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.207+0000 7f13ee47db80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ef69c52f60 0x55ef69c53340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:02:48.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.207+0000 7f13ee47db80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ef69c53880 0x55ef69c4d260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:02:48.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.207+0000 7f13ee47db80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ef69c4d7a0 0x55ef69c4dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:02:48.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.207+0000 7f13ee47db80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55ef69b3ff60 con 0x55ef69c53880 2024-06-10T03:02:48.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.207+0000 7f13ee47db80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55ef69b9e850 con 0x55ef69c4d7a0 2024-06-10T03:02:48.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.207+0000 7f13ee47db80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55ef69b06460 con 0x55ef69c52f60 2024-06-10T03:02:48.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.207+0000 7f13e1195700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ef69c53880 0x55ef69c4d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:02:48.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.207+0000 7f13e1996700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ef69c52f60 0x55ef69c53340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:02:48.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.207+0000 7f13e1195700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ef69c53880 0x55ef69c4d260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:46212/0 (socket says 172.21.15.52:46212) 2024-06-10T03:02:48.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.207+0000 7f13e1195700 1 -- 172.21.15.52:0/3874019980 learned_addr learned my addr 172.21.15.52:0/3874019980 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:02:48.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.207+0000 7f13e1996700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ef69c52f60 0x55ef69c53340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.52:35634/0 (socket says 172.21.15.52:35634) 2024-06-10T03:02:48.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.208+0000 7f13e2197700 1 --2- 172.21.15.52:0/3874019980 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ef69c4d7a0 0x55ef69c4dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:02:48.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.208+0000 7f13e1996700 1 -- 172.21.15.52:0/3874019980 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ef69c4d7a0 msgr2=0x55ef69c4dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:02:48.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.208+0000 7f13e1996700 1 --2- 172.21.15.52:0/3874019980 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ef69c4d7a0 0x55ef69c4dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:48.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.208+0000 7f13e1996700 1 -- 172.21.15.52:0/3874019980 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ef69c53880 msgr2=0x55ef69c4d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:02:48.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.208+0000 7f13e1996700 1 --2- 172.21.15.52:0/3874019980 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ef69c53880 0x55ef69c4d260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:48.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.208+0000 7f13e1996700 1 -- 172.21.15.52:0/3874019980 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef69aed8d0 con 0x55ef69c52f60 2024-06-10T03:02:48.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.208+0000 7f13e1195700 1 --2- 172.21.15.52:0/3874019980 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ef69c53880 0x55ef69c4d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T03:02:48.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.208+0000 7f13e1996700 1 --2- 172.21.15.52:0/3874019980 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ef69c52f60 0x55ef69c53340 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f13d80cd6f0 tx=0x7f13d8098040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cd560349de8f3481 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:02:48.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.208+0000 7f13c7fff700 1 -- 172.21.15.52:0/3874019980 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13d800fe40 con 0x55ef69c52f60 2024-06-10T03:02:48.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.208+0000 7f13c7fff700 1 -- 172.21.15.52:0/3874019980 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f13d809cb90 con 0x55ef69c52f60 2024-06-10T03:02:48.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.208+0000 7f13c7fff700 1 -- 172.21.15.52:0/3874019980 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13d809b8e0 con 0x55ef69c52f60 2024-06-10T03:02:48.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.209+0000 7f13ee47db80 1 -- 172.21.15.52:0/3874019980 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ef69c52f60 msgr2=0x55ef69c53340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:02:48.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.209+0000 7f13ee47db80 1 --2- 172.21.15.52:0/3874019980 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ef69c52f60 0x55ef69c53340 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f13d80cd6f0 tx=0x7f13d8098040 comp rx=0 tx=0).stop 2024-06-10T03:02:48.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.209+0000 7f13ee47db80 1 -- 172.21.15.52:0/3874019980 shutdown_connections 2024-06-10T03:02:48.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.209+0000 7f13ee47db80 1 --2- 172.21.15.52:0/3874019980 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ef69c4d7a0 0x55ef69c4dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:48.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.209+0000 7f13ee47db80 1 --2- 172.21.15.52:0/3874019980 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ef69c52f60 0x55ef69c53340 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:48.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.209+0000 7f13ee47db80 1 --2- 172.21.15.52:0/3874019980 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ef69c53880 0x55ef69c4d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:48.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.209+0000 7f13ee47db80 1 -- 172.21.15.52:0/3874019980 >> 172.21.15.52:0/3874019980 conn(0x55ef69b7a080 msgr2=0x55ef69c48220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:02:48.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.209+0000 7f13ee47db80 1 -- 172.21.15.52:0/3874019980 shutdown_connections 2024-06-10T03:02:48.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.209+0000 7f13ee47db80 1 -- 172.21.15.52:0/3874019980 wait complete. 2024-06-10T03:02:48.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.209+0000 7f13ee47db80 1 -- start start 2024-06-10T03:02:48.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.209+0000 7f13ee47db80 1 Processor -- start 2024-06-10T03:02:48.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.210+0000 7f13ee47db80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ef69c4d7a0 0x55ef69c59680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:02:48.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.210+0000 7f13ee47db80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ef69c52f60 0x55ef69c56980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:02:48.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.210+0000 7f13ee47db80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ef69c53880 0x55ef69c56ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:02:48.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.210+0000 7f13ee47db80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55ef69b3efc0 con 0x55ef69c52f60 2024-06-10T03:02:48.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.210+0000 7f13ee47db80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55ef69b9a810 con 0x55ef69c4d7a0 2024-06-10T03:02:48.217 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.210+0000 7f13ee47db80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55ef69c5a010 con 0x55ef69c53880 2024-06-10T03:02:48.217 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.210+0000 7f13ee47db80 10 client.0 _ll_register_callbacks cb 0x55ef69b345b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T03:02:48.217 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.210+0000 7f13e1195700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ef69c52f60 0x55ef69c56980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:02:48.217 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.210+0000 7f13e2197700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ef69c53880 0x55ef69c56ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:02:48.217 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.210+0000 7f13e1195700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ef69c52f60 0x55ef69c56980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:46224/0 (socket says 172.21.15.52:46224) 2024-06-10T03:02:48.217 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.210+0000 7f13e1195700 1 -- 172.21.15.52:0/66443602 learned_addr learned my addr 172.21.15.52:0/66443602 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:02:48.217 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.210+0000 7f13e2197700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ef69c53880 0x55ef69c56ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.52:35650/0 (socket says 172.21.15.52:35650) 2024-06-10T03:02:48.217 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.210+0000 7f13e1996700 1 --2- 172.21.15.52:0/66443602 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ef69c4d7a0 0x55ef69c59680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:02:48.217 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.210+0000 7f13e1195700 1 -- 172.21.15.52:0/66443602 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ef69c53880 msgr2=0x55ef69c56ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:02:48.217 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.210+0000 7f13e1195700 1 --2- 172.21.15.52:0/66443602 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ef69c53880 0x55ef69c56ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:48.217 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.210+0000 7f13e1195700 1 -- 172.21.15.52:0/66443602 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ef69c4d7a0 msgr2=0x55ef69c59680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:02:48.217 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.210+0000 7f13e1195700 1 --2- 172.21.15.52:0/66443602 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ef69c4d7a0 0x55ef69c59680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:48.217 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.210+0000 7f13e1195700 1 -- 172.21.15.52:0/66443602 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef69b64750 con 0x55ef69c52f60 2024-06-10T03:02:48.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.210+0000 7f13e1996700 1 --2- 172.21.15.52:0/66443602 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ef69c4d7a0 0x55ef69c59680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T03:02:48.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.210+0000 7f13e2197700 1 --2- 172.21.15.52:0/66443602 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ef69c53880 0x55ef69c56ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T03:02:48.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.211+0000 7f13e1195700 1 --2- 172.21.15.52:0/66443602 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ef69c52f60 0x55ef69c56980 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f13cc005950 tx=0x7f13cc0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34e3f135ee531529 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:02:48.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.211+0000 7f13c57fa700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-10T03:02:48.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.211+0000 7f13c57fa700 1 -- 172.21.15.52:0/66443602 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13cc01e630 con 0x55ef69c52f60 2024-06-10T03:02:48.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.211+0000 7f13ee47db80 1 -- 172.21.15.52:0/66443602 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f13d809cb90 con 0x55ef69c52f60 2024-06-10T03:02:48.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.211+0000 7f13c57fa700 1 -- 172.21.15.52:0/66443602 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f13cc01e7d0 con 0x55ef69c52f60 2024-06-10T03:02:48.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.211+0000 7f13c57fa700 1 -- 172.21.15.52:0/66443602 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f13cc01e9d0 con 0x55ef69c52f60 2024-06-10T03:02:48.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.211+0000 7f13ee47db80 10 client.16719 fetch_fsmap learned FSMap version 42 2024-06-10T03:02:48.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.211+0000 7f13ee47db80 1 -- 172.21.15.52:0/66443602 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55ef69c64840 con 0x55ef69c52f60 2024-06-10T03:02:48.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.211+0000 7f13c57fa700 1 -- 172.21.15.52:0/66443602 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f13d809cb90 con 0x55ef69c52f60 2024-06-10T03:02:48.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.212+0000 7f13c57fa700 1 -- 172.21.15.52:0/66443602 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 42) ==== 8394+0+0 (secure 0 0 0) 0x7f13cc026020 con 0x55ef69c52f60 2024-06-10T03:02:48.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.212+0000 7f13ee47db80 10 client.16719 fetch_fsmap finished waiting for FSMap version 42 2024-06-10T03:02:48.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.212+0000 7f13ee47db80 10 client.16719 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-06-10T03:02:48.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.212+0000 7f13ee47db80 20 client.16719 populate_metadata read hostname 'smithi052' 2024-06-10T03:02:48.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.212+0000 7f13ee47db80 1 --2- 172.21.15.52:0/66443602 >> [v2:172.21.15.188:6834/3904544259,v1:172.21.15.188:6835/3904544259] conn(0x55ef69c66e20 0x55ef69c69240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:02:48.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.212+0000 7f13ee47db80 4 client.16719 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T03:02:48.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.212+0000 7f13ee47db80 1 -- 172.21.15.52:0/66443602 --> [v2:172.21.15.188:6834/3904544259,v1:172.21.15.188:6835/3904544259] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ef69babd50 con 0x55ef69c66e20 2024-06-10T03:02:48.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.212+0000 7f13e1996700 1 --2- 172.21.15.52:0/66443602 >> [v2:172.21.15.188:6834/3904544259,v1:172.21.15.188:6835/3904544259] conn(0x55ef69c66e20 0x55ef69c69240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:02:48.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.212+0000 7f13e1996700 1 --2- 172.21.15.52:0/66443602 >> [v2:172.21.15.188:6834/3904544259,v1:172.21.15.188:6835/3904544259] conn(0x55ef69c66e20 0x55ef69c69240 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4ed3ab6e99ea70e5 server_cookie=fa3dcc2b755b19e5 in_seq=0 out_seq=0 2024-06-10T03:02:48.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.212+0000 7f13c57fa700 10 client.16719 ms_handle_connect on v2:172.21.15.188:6834/3904544259 2024-06-10T03:02:48.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.214+0000 7f13c57fa700 1 -- 172.21.15.52:0/66443602 <== mds.2 v2:172.21.15.188:6834/3904544259 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55ef69babd50 con 0x55ef69c66e20 2024-06-10T03:02:48.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.214+0000 7f13c57fa700 10 client.16719 handle_command_reply: tid=0 2024-06-10T03:02:48.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.214+0000 7f13ee47db80 1 client.16719 shutdown 2024-06-10T03:02:48.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.215+0000 7f13ee47db80 1 -- 172.21.15.52:0/66443602 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ef69c52f60 msgr2=0x55ef69c56980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:02:48.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.215+0000 7f13ee47db80 1 --2- 172.21.15.52:0/66443602 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ef69c52f60 0x55ef69c56980 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f13cc005950 tx=0x7f13cc0047c0 comp rx=0 tx=0).stop 2024-06-10T03:02:48.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.215+0000 7f13ee47db80 1 -- 172.21.15.52:0/66443602 shutdown_connections 2024-06-10T03:02:48.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.215+0000 7f13ee47db80 1 --2- 172.21.15.52:0/66443602 >> [v2:172.21.15.188:6834/3904544259,v1:172.21.15.188:6835/3904544259] conn(0x55ef69c66e20 0x55ef69c69240 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:48.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.215+0000 7f13ee47db80 1 --2- 172.21.15.52:0/66443602 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ef69c53880 0x55ef69c56ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:48.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.215+0000 7f13ee47db80 1 --2- 172.21.15.52:0/66443602 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ef69c4d7a0 0x55ef69c59680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:48.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.215+0000 7f13ee47db80 1 --2- 172.21.15.52:0/66443602 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ef69c52f60 0x55ef69c56980 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:48.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.215+0000 7f13c57fa700 0 client.16719 ms_handle_reset on v2:172.21.15.188:6834/3904544259 2024-06-10T03:02:48.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.215+0000 7f13ee47db80 1 -- 172.21.15.52:0/66443602 >> 172.21.15.52:0/66443602 conn(0x55ef69b7a080 msgr2=0x55ef69c47700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:02:48.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.215+0000 7f13ee47db80 1 -- 172.21.15.52:0/66443602 shutdown_connections 2024-06-10T03:02:48.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.215+0000 7f13ee47db80 1 -- 172.21.15.52:0/66443602 wait complete. 2024-06-10T03:02:48.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.215+0000 7f13ee47db80 20 client.16719 trim_cache size 0 max 16384 2024-06-10T03:02:48.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.231+0000 7f13ee47db80 1 Processor -- start 2024-06-10T03:02:48.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.232+0000 7f13ee47db80 1 -- start start 2024-06-10T03:02:48.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.232+0000 7f13ee47db80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ef69c4d7a0 0x55ef69c4d260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:02:48.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.232+0000 7f13ee47db80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ef69c52f60 0x55ef69c52190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:02:48.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.232+0000 7f13ee47db80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ef69c53880 0x55ef69c591c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:02:48.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.232+0000 7f13ee47db80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f13dc1b0520 con 0x55ef69c4d7a0 2024-06-10T03:02:48.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.232+0000 7f13ee47db80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f13dc1b21b0 con 0x55ef69c52f60 2024-06-10T03:02:48.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.232+0000 7f13ee47db80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f13dc1b0c50 con 0x55ef69c53880 2024-06-10T03:02:48.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.232+0000 7f13e1996700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ef69c4d7a0 0x55ef69c4d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:02:48.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.232+0000 7f13e1996700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ef69c4d7a0 0x55ef69c4d260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:46232/0 (socket says 172.21.15.52:46232) 2024-06-10T03:02:48.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.232+0000 7f13e1996700 1 -- 172.21.15.52:0/3690523762 learned_addr learned my addr 172.21.15.52:0/3690523762 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:02:48.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.232+0000 7f13e2197700 1 --2- 172.21.15.52:0/3690523762 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ef69c53880 0x55ef69c591c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:02:48.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.232+0000 7f13e1195700 1 --2- 172.21.15.52:0/3690523762 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ef69c52f60 0x55ef69c52190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:02:48.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.232+0000 7f13e1996700 1 -- 172.21.15.52:0/3690523762 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ef69c53880 msgr2=0x55ef69c591c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:02:48.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.232+0000 7f13e1996700 1 --2- 172.21.15.52:0/3690523762 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ef69c53880 0x55ef69c591c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:48.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.232+0000 7f13e1996700 1 -- 172.21.15.52:0/3690523762 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ef69c52f60 msgr2=0x55ef69c52190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:02:48.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.232+0000 7f13e1996700 1 --2- 172.21.15.52:0/3690523762 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ef69c52f60 0x55ef69c52190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:48.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.232+0000 7f13e1996700 1 -- 172.21.15.52:0/3690523762 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef69c57990 con 0x55ef69c4d7a0 2024-06-10T03:02:48.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.232+0000 7f13e2197700 1 --2- 172.21.15.52:0/3690523762 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ef69c53880 0x55ef69c591c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T03:02:48.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.232+0000 7f13e1195700 1 --2- 172.21.15.52:0/3690523762 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ef69c52f60 0x55ef69c52190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T03:02:48.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.232+0000 7f13e1996700 1 --2- 172.21.15.52:0/3690523762 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ef69c4d7a0 0x55ef69c4d260 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f13d8098e60 tx=0x7f13d801a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c330d220e7a6b799 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:02:48.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.233+0000 7f13c77fe700 1 -- 172.21.15.52:0/3690523762 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13d80a5070 con 0x55ef69c4d7a0 2024-06-10T03:02:48.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.233+0000 7f13c77fe700 1 -- 172.21.15.52:0/3690523762 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f13d809a790 con 0x55ef69c4d7a0 2024-06-10T03:02:48.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.233+0000 7f13c77fe700 1 -- 172.21.15.52:0/3690523762 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13d809bb40 con 0x55ef69c4d7a0 2024-06-10T03:02:48.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.233+0000 7f13ee47db80 1 -- 172.21.15.52:0/3690523762 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ef69c4d7a0 msgr2=0x55ef69c4d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:02:48.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.233+0000 7f13ee47db80 1 --2- 172.21.15.52:0/3690523762 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ef69c4d7a0 0x55ef69c4d260 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f13d8098e60 tx=0x7f13d801a040 comp rx=0 tx=0).stop 2024-06-10T03:02:48.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.233+0000 7f13ee47db80 1 -- 172.21.15.52:0/3690523762 shutdown_connections 2024-06-10T03:02:48.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.233+0000 7f13ee47db80 1 --2- 172.21.15.52:0/3690523762 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ef69c53880 0x55ef69c591c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:48.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.233+0000 7f13ee47db80 1 --2- 172.21.15.52:0/3690523762 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ef69c4d7a0 0x55ef69c4d260 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:48.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.233+0000 7f13ee47db80 1 --2- 172.21.15.52:0/3690523762 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ef69c52f60 0x55ef69c52190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:48.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.233+0000 7f13ee47db80 1 -- 172.21.15.52:0/3690523762 >> 172.21.15.52:0/3690523762 conn(0x55ef69b7a080 msgr2=0x55ef69c51a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:02:48.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.233+0000 7f13ee47db80 1 -- 172.21.15.52:0/3690523762 shutdown_connections 2024-06-10T03:02:48.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.233+0000 7f13ee47db80 1 -- 172.21.15.52:0/3690523762 wait complete. 2024-06-10T03:02:48.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.234+0000 7f13ee47db80 1 -- start start 2024-06-10T03:02:48.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.234+0000 7f13ee47db80 1 Processor -- start 2024-06-10T03:02:48.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.234+0000 7f13ee47db80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ef69c4d7a0 0x55ef69c5f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:02:48.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.234+0000 7f13ee47db80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ef69c52f60 0x55ef69c5f980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:02:48.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.234+0000 7f13ee47db80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ef69c53880 0x55ef69c6dee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:02:48.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.234+0000 7f13ee47db80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f13dc1b1ab0 con 0x55ef69c53880 2024-06-10T03:02:48.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.234+0000 7f13ee47db80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f13dc1b13b0 con 0x55ef69c4d7a0 2024-06-10T03:02:48.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.234+0000 7f13ee47db80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f13dc1ae4c0 con 0x55ef69c52f60 2024-06-10T03:02:48.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.234+0000 7f13ee47db80 10 client.0 _ll_register_callbacks cb 0x55ef69b345b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T03:02:48.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.234+0000 7f13e1996700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ef69c4d7a0 0x55ef69c5f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:02:48.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.234+0000 7f13e2197700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ef69c53880 0x55ef69c6dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:02:48.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.234+0000 7f13e1195700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ef69c52f60 0x55ef69c5f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:02:48.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.234+0000 7f13e1996700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ef69c4d7a0 0x55ef69c5f440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.52:40900/0 (socket says 172.21.15.52:40900) 2024-06-10T03:02:48.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.234+0000 7f13e1996700 1 -- 172.21.15.52:0/4220232436 learned_addr learned my addr 172.21.15.52:0/4220232436 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:02:48.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.234+0000 7f13e2197700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ef69c53880 0x55ef69c6dee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:46238/0 (socket says 172.21.15.52:46238) 2024-06-10T03:02:48.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.235+0000 7f13e1996700 1 -- 172.21.15.52:0/4220232436 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ef69c52f60 msgr2=0x55ef69c5f980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:02:48.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.235+0000 7f13e1996700 1 --2- 172.21.15.52:0/4220232436 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ef69c52f60 0x55ef69c5f980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:48.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.235+0000 7f13e1996700 1 -- 172.21.15.52:0/4220232436 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ef69c53880 msgr2=0x55ef69c6dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:02:48.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.235+0000 7f13e1996700 1 --2- 172.21.15.52:0/4220232436 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ef69c53880 0x55ef69c6dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:48.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.235+0000 7f13e1996700 1 -- 172.21.15.52:0/4220232436 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef69c6e580 con 0x55ef69c4d7a0 2024-06-10T03:02:48.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.235+0000 7f13e2197700 1 --2- 172.21.15.52:0/4220232436 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ef69c53880 0x55ef69c6dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T03:02:48.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.235+0000 7f13e1996700 1 --2- 172.21.15.52:0/4220232436 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ef69c4d7a0 0x55ef69c5f440 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f13d8006010 tx=0x7f13d80cdbf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=82b63e2a0a7fc7c3 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:02:48.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.235+0000 7f13c57fa700 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-06-10T03:02:48.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.235+0000 7f13c57fa700 1 -- 172.21.15.52:0/4220232436 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13d809bfd0 con 0x55ef69c4d7a0 2024-06-10T03:02:48.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.235+0000 7f13c57fa700 1 -- 172.21.15.52:0/4220232436 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f13d80b5230 con 0x55ef69c4d7a0 2024-06-10T03:02:48.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.235+0000 7f13ee47db80 1 -- 172.21.15.52:0/4220232436 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f13d809a790 con 0x55ef69c4d7a0 2024-06-10T03:02:48.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.235+0000 7f13c57fa700 1 -- 172.21.15.52:0/4220232436 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f13d809cb90 con 0x55ef69c4d7a0 2024-06-10T03:02:48.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.235+0000 7f13ee47db80 10 client.26222 fetch_fsmap learned FSMap version 42 2024-06-10T03:02:48.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.235+0000 7f13ee47db80 1 -- 172.21.15.52:0/4220232436 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55ef69cd13b0 con 0x55ef69c4d7a0 2024-06-10T03:02:48.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.235+0000 7f13c57fa700 1 -- 172.21.15.52:0/4220232436 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f13d809a790 con 0x55ef69c4d7a0 2024-06-10T03:02:48.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.236+0000 7f13c57fa700 1 -- 172.21.15.52:0/4220232436 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 42) ==== 8394+0+0 (secure 0 0 0) 0x7f13d8085070 con 0x55ef69c4d7a0 2024-06-10T03:02:48.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.236+0000 7f13ee47db80 10 client.26222 fetch_fsmap finished waiting for FSMap version 42 2024-06-10T03:02:48.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.236+0000 7f13ee47db80 10 client.26222 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-06-10T03:02:48.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.236+0000 7f13ee47db80 20 client.26222 populate_metadata read hostname 'smithi052' 2024-06-10T03:02:48.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.236+0000 7f13ee47db80 1 --2- 172.21.15.52:0/4220232436 >> [v2:172.21.15.188:6834/3904544259,v1:172.21.15.188:6835/3904544259] conn(0x55ef69cd1680 0x55ef69cd3a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:02:48.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.236+0000 7f13ee47db80 4 client.26222 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T03:02:48.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.236+0000 7f13ee47db80 1 -- 172.21.15.52:0/4220232436 --> [v2:172.21.15.188:6834/3904544259,v1:172.21.15.188:6835/3904544259] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ef69ba62d0 con 0x55ef69cd1680 2024-06-10T03:02:48.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.236+0000 7f13e1195700 1 --2- 172.21.15.52:0/4220232436 >> [v2:172.21.15.188:6834/3904544259,v1:172.21.15.188:6835/3904544259] conn(0x55ef69cd1680 0x55ef69cd3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:02:48.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.236+0000 7f13e1195700 1 --2- 172.21.15.52:0/4220232436 >> [v2:172.21.15.188:6834/3904544259,v1:172.21.15.188:6835/3904544259] conn(0x55ef69cd1680 0x55ef69cd3a70 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e45bfacb13f01c0e server_cookie=9f0bbec2fc055c0d in_seq=0 out_seq=0 2024-06-10T03:02:48.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.236+0000 7f13c57fa700 10 client.26222 ms_handle_connect on v2:172.21.15.188:6834/3904544259 2024-06-10T03:02:48.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.238+0000 7f13c57fa700 1 -- 172.21.15.52:0/4220232436 <== mds.2 v2:172.21.15.188:6834/3904544259 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ef69ba62d0 con 0x55ef69cd1680 2024-06-10T03:02:48.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.238+0000 7f13c57fa700 10 client.26222 handle_command_reply: tid=0 2024-06-10T03:02:48.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.238+0000 7f13ee47db80 1 client.26222 shutdown 2024-06-10T03:02:48.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.238+0000 7f13ee47db80 1 -- 172.21.15.52:0/4220232436 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ef69c4d7a0 msgr2=0x55ef69c5f440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:02:48.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.238+0000 7f13ee47db80 1 --2- 172.21.15.52:0/4220232436 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ef69c4d7a0 0x55ef69c5f440 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f13d8006010 tx=0x7f13d80cdbf0 comp rx=0 tx=0).stop 2024-06-10T03:02:48.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.238+0000 7f13ee47db80 1 -- 172.21.15.52:0/4220232436 shutdown_connections 2024-06-10T03:02:48.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.238+0000 7f13ee47db80 1 --2- 172.21.15.52:0/4220232436 >> [v2:172.21.15.188:6834/3904544259,v1:172.21.15.188:6835/3904544259] conn(0x55ef69cd1680 0x55ef69cd3a70 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:48.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.238+0000 7f13ee47db80 1 --2- 172.21.15.52:0/4220232436 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ef69c53880 0x55ef69c6dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:48.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.238+0000 7f13ee47db80 1 --2- 172.21.15.52:0/4220232436 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ef69c4d7a0 0x55ef69c5f440 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:48.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.238+0000 7f13ee47db80 1 --2- 172.21.15.52:0/4220232436 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ef69c52f60 0x55ef69c5f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:48.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.238+0000 7f13c57fa700 0 client.26222 ms_handle_reset on v2:172.21.15.188:6834/3904544259 2024-06-10T03:02:48.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.238+0000 7f13ee47db80 1 -- 172.21.15.52:0/4220232436 >> 172.21.15.52:0/4220232436 conn(0x55ef69b7a080 msgr2=0x55ef69c52310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:02:48.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.238+0000 7f13ee47db80 1 -- 172.21.15.52:0/4220232436 shutdown_connections 2024-06-10T03:02:48.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.238+0000 7f13ee47db80 1 -- 172.21.15.52:0/4220232436 wait complete. 2024-06-10T03:02:48.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.239+0000 7f13ee47db80 20 client.26222 trim_cache size 0 max 16384 2024-06-10T03:02:48.243 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-06-10T03:02:48.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.239+0000 7f13e3bfa700 1 -- 172.21.15.52:0/2649008740 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f13c80a2c20 msgr2=0x7f13c80a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:02:48.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.239+0000 7f13e3bfa700 1 --2- 172.21.15.52:0/2649008740 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f13c80a2c20 0x7f13c80a50e0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f13dc101fd0 tx=0x7f13cc009260 comp rx=0 tx=0).stop 2024-06-10T03:02:48.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.239+0000 7f13e3bfa700 1 -- 172.21.15.52:0/2649008740 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f13dc102740 msgr2=0x7f13dc074790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:02:48.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.239+0000 7f13e3bfa700 1 --2- 172.21.15.52:0/2649008740 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f13dc102740 0x7f13dc074790 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f13d80059d0 tx=0x7f13d800fa00 comp rx=0 tx=0).stop 2024-06-10T03:02:48.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.239+0000 7f13e3bfa700 1 -- 172.21.15.52:0/2649008740 shutdown_connections 2024-06-10T03:02:48.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.239+0000 7f13e3bfa700 1 --2- 172.21.15.52:0/2649008740 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f13dc107840 0x7f13dc075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:48.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.239+0000 7f13e3bfa700 1 --2- 172.21.15.52:0/2649008740 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f13c80a2c20 0x7f13c80a50e0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:48.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.239+0000 7f13e3bfa700 1 --2- 172.21.15.52:0/2649008740 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f13dc102740 0x7f13dc074790 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:48.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.239+0000 7f13e3bfa700 1 --2- 172.21.15.52:0/2649008740 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f13dc1030f0 0x7f13dc074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:48.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.239+0000 7f13e3bfa700 1 -- 172.21.15.52:0/2649008740 >> 172.21.15.52:0/2649008740 conn(0x7f13dc0795f0 msgr2=0x7f13dc108990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:02:48.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.239+0000 7f13e3bfa700 1 -- 172.21.15.52:0/2649008740 shutdown_connections 2024-06-10T03:02:48.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.239+0000 7f13e3bfa700 1 -- 172.21.15.52:0/2649008740 wait complete. 2024-06-10T03:02:48.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.382+0000 7f61e0a88700 1 -- 172.21.15.52:0/3527565776 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f61dc06ab30 msgr2=0x7f61dc10f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:02:48.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.382+0000 7f61e0a88700 1 --2- 172.21.15.52:0/3527565776 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f61dc06ab30 0x7f61dc10f320 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f61d400b0a0 tx=0x7f61d400b3b0 comp rx=0 tx=0).stop 2024-06-10T03:02:48.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.383+0000 7f61e0a88700 1 -- 172.21.15.52:0/3527565776 shutdown_connections 2024-06-10T03:02:48.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.383+0000 7f61e0a88700 1 --2- 172.21.15.52:0/3527565776 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f61dc06ab30 0x7f61dc10f320 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:48.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.383+0000 7f61e0a88700 1 --2- 172.21.15.52:0/3527565776 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f61dc108500 0x7f61dc1088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:48.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.383+0000 7f61e0a88700 1 --2- 172.21.15.52:0/3527565776 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f61dc06a170 0x7f61dc06a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:48.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.383+0000 7f61e0a88700 1 -- 172.21.15.52:0/3527565776 >> 172.21.15.52:0/3527565776 conn(0x7f61dc078340 msgr2=0x7f61dc078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:02:48.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.383+0000 7f61e0a88700 1 -- 172.21.15.52:0/3527565776 shutdown_connections 2024-06-10T03:02:48.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.383+0000 7f61e0a88700 1 -- 172.21.15.52:0/3527565776 wait complete. 2024-06-10T03:02:48.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.383+0000 7f61e0a88700 1 Processor -- start 2024-06-10T03:02:48.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.383+0000 7f61e0a88700 1 -- start start 2024-06-10T03:02:48.386 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.384+0000 7f61e0a88700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f61dc06a170 0x7f61dc101460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:02:48.386 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.384+0000 7f61e0a88700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f61dc06ab30 0x7f61dc1019a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:02:48.386 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.384+0000 7f61e0a88700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f61dc108500 0x7f61dc101ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:02:48.386 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.384+0000 7f61e0a88700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f61dc112760 con 0x7f61dc06a170 2024-06-10T03:02:48.386 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.384+0000 7f61e0a88700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f61dc1125e0 con 0x7f61dc108500 2024-06-10T03:02:48.386 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.384+0000 7f61e0a88700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f61dc1128e0 con 0x7f61dc06ab30 2024-06-10T03:02:48.386 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.384+0000 7f61da59c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f61dc06a170 0x7f61dc101460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:02:48.386 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.384+0000 7f61d9d9b700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f61dc06ab30 0x7f61dc1019a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:02:48.386 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.384+0000 7f61d9d9b700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f61dc06ab30 0x7f61dc1019a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.52:35688/0 (socket says 172.21.15.52:35688) 2024-06-10T03:02:48.386 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.384+0000 7f61da59c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f61dc06a170 0x7f61dc101460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:46268/0 (socket says 172.21.15.52:46268) 2024-06-10T03:02:48.386 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.384+0000 7f61d9d9b700 1 -- 172.21.15.52:0/916962357 learned_addr learned my addr 172.21.15.52:0/916962357 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:02:48.386 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.384+0000 7f61dad9d700 1 --2- 172.21.15.52:0/916962357 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f61dc108500 0x7f61dc101ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:02:48.387 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.384+0000 7f61d9d9b700 1 -- 172.21.15.52:0/916962357 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f61dc108500 msgr2=0x7f61dc101ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:02:48.387 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.384+0000 7f61d9d9b700 1 --2- 172.21.15.52:0/916962357 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f61dc108500 0x7f61dc101ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:48.387 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.384+0000 7f61d9d9b700 1 -- 172.21.15.52:0/916962357 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f61dc06a170 msgr2=0x7f61dc101460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:02:48.387 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.384+0000 7f61d9d9b700 1 --2- 172.21.15.52:0/916962357 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f61dc06a170 0x7f61dc101460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:48.387 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.384+0000 7f61d9d9b700 1 -- 172.21.15.52:0/916962357 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61dc1a7130 con 0x7f61dc06ab30 2024-06-10T03:02:48.387 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.384+0000 7f61da59c700 1 --2- 172.21.15.52:0/916962357 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f61dc06a170 0x7f61dc101460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T03:02:48.387 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.384+0000 7f61d9d9b700 1 --2- 172.21.15.52:0/916962357 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f61dc06ab30 0x7f61dc1019a0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f61d000d950 tx=0x7f61d000dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:02:48.387 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.385+0000 7f61cb7fe700 1 -- 172.21.15.52:0/916962357 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d0016070 con 0x7f61dc06ab30 2024-06-10T03:02:48.387 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.385+0000 7f61cb7fe700 1 -- 172.21.15.52:0/916962357 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f61d000fd70 con 0x7f61dc06ab30 2024-06-10T03:02:48.387 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.385+0000 7f61e0a88700 1 -- 172.21.15.52:0/916962357 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61dc1a7420 con 0x7f61dc06ab30 2024-06-10T03:02:48.387 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.385+0000 7f61cb7fe700 1 -- 172.21.15.52:0/916962357 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d000e610 con 0x7f61dc06ab30 2024-06-10T03:02:48.387 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.385+0000 7f61e0a88700 1 -- 172.21.15.52:0/916962357 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61dc1a7960 con 0x7f61dc06ab30 2024-06-10T03:02:48.387 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.386+0000 7f61e0a88700 1 -- 172.21.15.52:0/916962357 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61dc069180 con 0x7f61dc06ab30 2024-06-10T03:02:48.388 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.387+0000 7f61cb7fe700 1 -- 172.21.15.52:0/916962357 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f61d0019070 con 0x7f61dc06ab30 2024-06-10T03:02:48.388 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.388+0000 7f61cb7fe700 1 --2- 172.21.15.52:0/916962357 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f61c40a2c20 0x7f61c40a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:02:48.388 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.388+0000 7f61da59c700 1 --2- 172.21.15.52:0/916962357 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f61c40a2c20 0x7f61c40a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:02:48.388 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.388+0000 7f61cb7fe700 1 -- 172.21.15.52:0/916962357 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f61d00ceec0 con 0x7f61dc06ab30 2024-06-10T03:02:48.388 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.388+0000 7f61da59c700 1 --2- 172.21.15.52:0/916962357 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f61c40a2c20 0x7f61c40a50e0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f61cc005950 tx=0x7f61cc00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:02:48.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.391+0000 7f61cb7fe700 1 -- 172.21.15.52:0/916962357 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f61d009ba10 con 0x7f61dc06ab30 2024-06-10T03:02:48.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.649+0000 7f61e0a88700 1 -- 172.21.15.52:0/916962357 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f61dc102940 con 0x7f61dc06ab30 2024-06-10T03:02:48.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.650+0000 7f61cb7fe700 1 -- 172.21.15.52:0/916962357 <== mon.1 v2:172.21.15.188:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) ==== 94+0+11949 (secure 0 0 0) 0x7f61d009b310 con 0x7f61dc06ab30 2024-06-10T03:02:48.651 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-10T03:02:48.651 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":42,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/889673247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":889673247},{"type":"v1","addr":"172.21.15.52:6835","nonce":889673247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/3615663511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":3615663511},{"type":"v1","addr":"172.21.15.52:6837","nonce":3615663511}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":42,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T02:50:03.889766+0000","modified":"2024-06-10T03:02:33.697178+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24484,"mds_1":24469,"mds_2":24463,"mds_3":24449,"mds_4":24428},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14571":{"gid":14571,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6839/3059599042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":3059599042},{"type":"v1","addr":"172.21.15.52:6839","nonce":3059599042}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14583":{"gid":14583,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6841/3039275521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":3039275521},{"type":"v1","addr":"172.21.15.52:6841","nonce":3039275521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24383":{"gid":24383,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6833/2609002896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":2609002896},{"type":"v1","addr":"172.21.15.120:6833","nonce":2609002896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6835/1996370288","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":1996370288},{"type":"v1","addr":"172.21.15.120:6835","nonce":1996370288}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6833/529564979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":529564979},{"type":"v1","addr":"172.21.15.188:6833","nonce":529564979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":25,"addr":"172.21.15.120:6837/4132440952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":4132440952},{"type":"v1","addr":"172.21.15.120:6837","nonce":4132440952}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"b","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.120:6839/1672436487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1672436487},{"type":"v1","addr":"172.21.15.120:6839","nonce":1672436487}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":2,"incarnation":22,"state":"up:active","state_seq":26,"addr":"172.21.15.188:6835/3904544259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":3904544259},{"type":"v1","addr":"172.21.15.188:6835","nonce":3904544259}]},"join_fscid":-1,"export_targets":[1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.188:6837/819317726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":819317726},{"type":"v1","addr":"172.21.15.188:6837","nonce":819317726}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.188:6839/310444836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":310444836},{"type":"v1","addr":"172.21.15.188:6839","nonce":310444836}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24484,"qdb_cluster":[24469,24484,24463,24428,24449]},"id":1}]} 2024-06-10T03:02:48.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.652+0000 7f61e0a88700 1 -- 172.21.15.52:0/916962357 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f61c40a2c20 msgr2=0x7f61c40a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:02:48.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.652+0000 7f61e0a88700 1 --2- 172.21.15.52:0/916962357 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f61c40a2c20 0x7f61c40a50e0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f61cc005950 tx=0x7f61cc00b410 comp rx=0 tx=0).stop 2024-06-10T03:02:48.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.652+0000 7f61e0a88700 1 -- 172.21.15.52:0/916962357 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f61dc06ab30 msgr2=0x7f61dc1019a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:02:48.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.652+0000 7f61e0a88700 1 --2- 172.21.15.52:0/916962357 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f61dc06ab30 0x7f61dc1019a0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f61d000d950 tx=0x7f61d000dc60 comp rx=0 tx=0).stop 2024-06-10T03:02:48.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.652+0000 7f61e0a88700 1 -- 172.21.15.52:0/916962357 shutdown_connections 2024-06-10T03:02:48.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.652+0000 7f61e0a88700 1 --2- 172.21.15.52:0/916962357 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f61dc108500 0x7f61dc101ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:48.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.652+0000 7f61e0a88700 1 --2- 172.21.15.52:0/916962357 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f61c40a2c20 0x7f61c40a50e0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:48.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.652+0000 7f61e0a88700 1 --2- 172.21.15.52:0/916962357 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f61dc06a170 0x7f61dc101460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:48.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.652+0000 7f61e0a88700 1 --2- 172.21.15.52:0/916962357 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f61dc06ab30 0x7f61dc1019a0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:48.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.652+0000 7f61e0a88700 1 -- 172.21.15.52:0/916962357 >> 172.21.15.52:0/916962357 conn(0x7f61dc078340 msgr2=0x7f61dc0fffa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:02:48.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.652+0000 7f61e0a88700 1 -- 172.21.15.52:0/916962357 shutdown_connections 2024-06-10T03:02:48.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:48.652+0000 7f61e0a88700 1 -- 172.21.15.52:0/916962357 wait complete. 2024-06-10T03:02:48.654 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 42 2024-06-10T03:02:48.888 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:02:48 smithi120 ceph-mon[36634]: pgmap v692: 129 pgs: 129 active+clean; 351 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 619 op/s 2024-06-10T03:02:48.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:02:48 smithi188 ceph-mon[36599]: pgmap v692: 129 pgs: 129 active+clean; 351 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 619 op/s 2024-06-10T03:02:48.976 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:02:48 smithi052 ceph-mon[32362]: pgmap v692: 129 pgs: 129 active+clean; 351 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 619 op/s 2024-06-10T03:02:49.070 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-06-10T03:02:49.691 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:02:49 smithi052 ceph-mon[32362]: from='client.? 172.21.15.52:0/916962357' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T03:02:49.719 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24484 quiesce db / --timeout 90 --expiration 136.0 --await-for 45.0' 2024-06-10T03:02:49.719 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24484 quiesce db / --timeout 90 --expiration 136.0 --await-for 45.0 2024-06-10T03:02:49.857 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T03:02:49.888 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:02:49 smithi120 ceph-mon[36634]: from='client.? 172.21.15.52:0/916962357' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T03:02:49.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:02:49 smithi188 ceph-mon[36599]: from='client.? 172.21.15.52:0/916962357' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T03:02:50.888 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:02:50 smithi120 ceph-mon[36634]: pgmap v693: 129 pgs: 129 active+clean; 342 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 657 op/s 2024-06-10T03:02:50.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:02:50 smithi188 ceph-mon[36599]: pgmap v693: 129 pgs: 129 active+clean; 342 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 657 op/s 2024-06-10T03:02:50.977 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:02:50 smithi052 ceph-mon[32362]: pgmap v693: 129 pgs: 129 active+clean; 342 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 657 op/s 2024-06-10T03:02:51.507 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T03:02:52.884 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.884+0000 7f38e25a6700 1 -- 172.21.15.52:0/176567855 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f38dc107880 msgr2=0x7f38dc10f400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:02:52.884 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.884+0000 7f38e25a6700 1 --2- 172.21.15.52:0/176567855 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f38dc107880 0x7f38dc10f400 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f38d400b0d0 tx=0x7f38d400b3e0 comp rx=0 tx=0).stop 2024-06-10T03:02:52.884 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.884+0000 7f38e25a6700 1 -- 172.21.15.52:0/176567855 shutdown_connections 2024-06-10T03:02:52.884 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.884+0000 7f38e25a6700 1 --2- 172.21.15.52:0/176567855 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f38dc107880 0x7f38dc10f400 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:52.884 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.884+0000 7f38e25a6700 1 --2- 172.21.15.52:0/176567855 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f38dc100c40 0x7f38dc101020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:52.885 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.884+0000 7f38e25a6700 1 --2- 172.21.15.52:0/176567855 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f38dc1015f0 0x7f38dc107150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:52.885 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.884+0000 7f38e25a6700 1 -- 172.21.15.52:0/176567855 >> 172.21.15.52:0/176567855 conn(0x7f38dc077990 msgr2=0x7f38dc0fed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:02:52.885 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.884+0000 7f38e25a6700 1 -- 172.21.15.52:0/176567855 shutdown_connections 2024-06-10T03:02:52.885 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.884+0000 7f38e25a6700 1 -- 172.21.15.52:0/176567855 wait complete. 2024-06-10T03:02:52.885 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.884+0000 7f38e25a6700 1 Processor -- start 2024-06-10T03:02:52.885 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:02:52 smithi052 ceph-mon[32362]: pgmap v694: 129 pgs: 129 active+clean; 337 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 525 op/s 2024-06-10T03:02:52.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.884+0000 7f38e25a6700 1 -- start start 2024-06-10T03:02:52.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.885+0000 7f38e25a6700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f38dc100c40 0x7f38dc19ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:02:52.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.885+0000 7f38e25a6700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f38dc1015f0 0x7f38dc19b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:02:52.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.885+0000 7f38e25a6700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f38dc107880 0x7f38dc19f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:02:52.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.885+0000 7f38e25a6700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f38dc112810 con 0x7f38dc107880 2024-06-10T03:02:52.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.885+0000 7f38e25a6700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f38dc112690 con 0x7f38dc1015f0 2024-06-10T03:02:52.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.885+0000 7f38e25a6700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f38dc112990 con 0x7f38dc100c40 2024-06-10T03:02:52.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.885+0000 7f38e0b43700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f38dc107880 0x7f38dc19f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:02:52.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.885+0000 7f38dbfff700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f38dc100c40 0x7f38dc19ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:02:52.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.885+0000 7f38e0b43700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f38dc107880 0x7f38dc19f630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58148/0 (socket says 172.21.15.52:58148) 2024-06-10T03:02:52.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.885+0000 7f38e0b43700 1 -- 172.21.15.52:0/1417996955 learned_addr learned my addr 172.21.15.52:0/1417996955 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:02:52.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.885+0000 7f38dbfff700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f38dc100c40 0x7f38dc19ad60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.52:36252/0 (socket says 172.21.15.52:36252) 2024-06-10T03:02:52.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.885+0000 7f38db7fe700 1 --2- 172.21.15.52:0/1417996955 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f38dc1015f0 0x7f38dc19b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:02:52.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.885+0000 7f38e0b43700 1 -- 172.21.15.52:0/1417996955 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f38dc100c40 msgr2=0x7f38dc19ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:02:52.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.885+0000 7f38e0b43700 1 --2- 172.21.15.52:0/1417996955 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f38dc100c40 0x7f38dc19ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:52.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.885+0000 7f38e0b43700 1 -- 172.21.15.52:0/1417996955 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f38dc1015f0 msgr2=0x7f38dc19b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:02:52.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.885+0000 7f38e0b43700 1 --2- 172.21.15.52:0/1417996955 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f38dc1015f0 0x7f38dc19b2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:52.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.885+0000 7f38e0b43700 1 -- 172.21.15.52:0/1417996955 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38dc19fd70 con 0x7f38dc107880 2024-06-10T03:02:52.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.886+0000 7f38e0b43700 1 --2- 172.21.15.52:0/1417996955 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f38dc107880 0x7f38dc19f630 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f38d400d010 tx=0x7f38d40079a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:02:52.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.886+0000 7f38d97fa700 1 -- 172.21.15.52:0/1417996955 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38d4003c00 con 0x7f38dc107880 2024-06-10T03:02:52.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.886+0000 7f38d97fa700 1 -- 172.21.15.52:0/1417996955 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f38d4022070 con 0x7f38dc107880 2024-06-10T03:02:52.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.886+0000 7f38e25a6700 1 -- 172.21.15.52:0/1417996955 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38dc1a0000 con 0x7f38dc107880 2024-06-10T03:02:52.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.886+0000 7f38d97fa700 1 -- 172.21.15.52:0/1417996955 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38d401b6c0 con 0x7f38dc107880 2024-06-10T03:02:52.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.886+0000 7f38e25a6700 1 -- 172.21.15.52:0/1417996955 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38dc1a78e0 con 0x7f38dc107880 2024-06-10T03:02:52.890 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:02:52 smithi120 ceph-mon[36634]: pgmap v694: 129 pgs: 129 active+clean; 337 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 525 op/s 2024-06-10T03:02:52.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.889+0000 7f38d97fa700 1 -- 172.21.15.52:0/1417996955 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f38d4009ba0 con 0x7f38dc107880 2024-06-10T03:02:52.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.890+0000 7f38d97fa700 1 --2- 172.21.15.52:0/1417996955 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f38c40a2ca0 0x7f38c40a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:02:52.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.890+0000 7f38dbfff700 1 --2- 172.21.15.52:0/1417996955 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f38c40a2ca0 0x7f38c40a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:02:52.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.890+0000 7f38d97fa700 1 -- 172.21.15.52:0/1417996955 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f38d400f020 con 0x7f38dc107880 2024-06-10T03:02:52.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.890+0000 7f38ece29b80 1 Processor -- start 2024-06-10T03:02:52.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.890+0000 7f38dbfff700 1 --2- 172.21.15.52:0/1417996955 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f38c40a2ca0 0x7f38c40a5160 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f38cc009c60 tx=0x7f38cc009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:02:52.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.890+0000 7f38ece29b80 1 -- start start 2024-06-10T03:02:52.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.891+0000 7f38ece29b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f953ab6f60 0x55f953ab7340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:02:52.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.891+0000 7f38ece29b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f953ab7880 0x55f953ab1260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:02:52.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.891+0000 7f38ece29b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f953ab17a0 0x55f953ab1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:02:52.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.891+0000 7f38ece29b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55f9539a3f60 con 0x55f953ab6f60 2024-06-10T03:02:52.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.891+0000 7f38ece29b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55f953a02850 con 0x55f953ab17a0 2024-06-10T03:02:52.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.891+0000 7f38ece29b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55f95396a460 con 0x55f953ab7880 2024-06-10T03:02:52.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.891+0000 7f38dbfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f953ab6f60 0x55f953ab7340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:02:52.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.891+0000 7f38dbfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f953ab6f60 0x55f953ab7340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58158/0 (socket says 172.21.15.52:58158) 2024-06-10T03:02:52.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.891+0000 7f38dbfff700 1 -- 172.21.15.52:0/1660311710 learned_addr learned my addr 172.21.15.52:0/1660311710 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:02:52.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.891+0000 7f38db7fe700 1 --2- 172.21.15.52:0/1660311710 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f953ab7880 0x55f953ab1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:02:52.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.891+0000 7f38e0b43700 1 --2- 172.21.15.52:0/1660311710 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f953ab17a0 0x55f953ab1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:02:52.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.891+0000 7f38dbfff700 1 -- 172.21.15.52:0/1660311710 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f953ab7880 msgr2=0x55f953ab1260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:02:52.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.891+0000 7f38dbfff700 1 --2- 172.21.15.52:0/1660311710 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f953ab7880 0x55f953ab1260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:52.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.891+0000 7f38dbfff700 1 -- 172.21.15.52:0/1660311710 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f953ab17a0 msgr2=0x55f953ab1bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:02:52.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.891+0000 7f38dbfff700 1 --2- 172.21.15.52:0/1660311710 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f953ab17a0 0x55f953ab1bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:52.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.891+0000 7f38dbfff700 1 -- 172.21.15.52:0/1660311710 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9539518d0 con 0x55f953ab6f60 2024-06-10T03:02:52.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.891+0000 7f38dbfff700 1 --2- 172.21.15.52:0/1660311710 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f953ab6f60 0x55f953ab7340 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f38cc004140 tx=0x7f38cc014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2397ff3111867ef9 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:02:52.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.892+0000 7f38c27fc700 1 -- 172.21.15.52:0/1660311710 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38cc003b10 con 0x55f953ab6f60 2024-06-10T03:02:52.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.892+0000 7f38c27fc700 1 -- 172.21.15.52:0/1660311710 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f38cc004d10 con 0x55f953ab6f60 2024-06-10T03:02:52.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.892+0000 7f38c27fc700 1 -- 172.21.15.52:0/1660311710 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38cc01e420 con 0x55f953ab6f60 2024-06-10T03:02:52.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.892+0000 7f38ece29b80 1 -- 172.21.15.52:0/1660311710 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f953ab6f60 msgr2=0x55f953ab7340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:02:52.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.892+0000 7f38ece29b80 1 --2- 172.21.15.52:0/1660311710 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f953ab6f60 0x55f953ab7340 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f38cc004140 tx=0x7f38cc014000 comp rx=0 tx=0).stop 2024-06-10T03:02:52.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.892+0000 7f38ece29b80 1 -- 172.21.15.52:0/1660311710 shutdown_connections 2024-06-10T03:02:52.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.892+0000 7f38ece29b80 1 --2- 172.21.15.52:0/1660311710 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f953ab17a0 0x55f953ab1bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:52.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.892+0000 7f38ece29b80 1 --2- 172.21.15.52:0/1660311710 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f953ab6f60 0x55f953ab7340 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:52.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.892+0000 7f38ece29b80 1 --2- 172.21.15.52:0/1660311710 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f953ab7880 0x55f953ab1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:52.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.892+0000 7f38ece29b80 1 -- 172.21.15.52:0/1660311710 >> 172.21.15.52:0/1660311710 conn(0x55f9539de080 msgr2=0x55f953aac220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:02:52.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.892+0000 7f38ece29b80 1 -- 172.21.15.52:0/1660311710 shutdown_connections 2024-06-10T03:02:52.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.892+0000 7f38ece29b80 1 -- 172.21.15.52:0/1660311710 wait complete. 2024-06-10T03:02:52.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.893+0000 7f38ece29b80 1 -- start start 2024-06-10T03:02:52.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.893+0000 7f38ece29b80 1 Processor -- start 2024-06-10T03:02:52.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.893+0000 7f38ece29b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f953ab17a0 0x55f953abd5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:02:52.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.893+0000 7f38ece29b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f953ab6f60 0x55f953aba8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:02:52.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.893+0000 7f38ece29b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f953ab7880 0x55f953abae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:02:52.899 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.893+0000 7f38ece29b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55f9539a2fc0 con 0x55f953ab7880 2024-06-10T03:02:52.899 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.893+0000 7f38ece29b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55f9539fe810 con 0x55f953ab6f60 2024-06-10T03:02:52.899 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.893+0000 7f38ece29b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55f953abe260 con 0x55f953ab17a0 2024-06-10T03:02:52.899 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.893+0000 7f38ece29b80 10 client.0 _ll_register_callbacks cb 0x55f9539985b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T03:02:52.899 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.893+0000 7f38e0b43700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f953ab7880 0x55f953abae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:02:52.899 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.894+0000 7f38dbfff700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f953ab17a0 0x55f953abd5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:02:52.899 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.894+0000 7f38e0b43700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f953ab7880 0x55f953abae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58168/0 (socket says 172.21.15.52:58168) 2024-06-10T03:02:52.899 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.894+0000 7f38e0b43700 1 -- 172.21.15.52:0/2196195782 learned_addr learned my addr 172.21.15.52:0/2196195782 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:02:52.899 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.894+0000 7f38db7fe700 1 --2- 172.21.15.52:0/2196195782 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f953ab6f60 0x55f953aba8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:02:52.899 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.894+0000 7f38e0b43700 1 -- 172.21.15.52:0/2196195782 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f953ab17a0 msgr2=0x55f953abd5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:02:52.899 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.894+0000 7f38e0b43700 1 --2- 172.21.15.52:0/2196195782 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f953ab17a0 0x55f953abd5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:52.899 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.894+0000 7f38e0b43700 1 -- 172.21.15.52:0/2196195782 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f953ab6f60 msgr2=0x55f953aba8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:02:52.899 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.894+0000 7f38e0b43700 1 --2- 172.21.15.52:0/2196195782 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f953ab6f60 0x55f953aba8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:52.900 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.894+0000 7f38e0b43700 1 -- 172.21.15.52:0/2196195782 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9539c8750 con 0x55f953ab7880 2024-06-10T03:02:52.900 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.894+0000 7f38e0b43700 1 --2- 172.21.15.52:0/2196195782 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f953ab7880 0x55f953abae00 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f38d40cdb90 tx=0x7f38d401db70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29f967744208f1e2 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:02:52.900 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.894+0000 7f38bb7fe700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-10T03:02:52.900 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.894+0000 7f38bb7fe700 1 -- 172.21.15.52:0/2196195782 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38d409bf70 con 0x55f953ab7880 2024-06-10T03:02:52.900 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.894+0000 7f38bb7fe700 1 -- 172.21.15.52:0/2196195782 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f38d409c870 con 0x55f953ab7880 2024-06-10T03:02:52.900 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.894+0000 7f38ece29b80 1 -- 172.21.15.52:0/2196195782 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f38cc004d10 con 0x55f953ab7880 2024-06-10T03:02:52.900 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.894+0000 7f38bb7fe700 1 -- 172.21.15.52:0/2196195782 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f38d409cd10 con 0x55f953ab7880 2024-06-10T03:02:52.900 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.894+0000 7f38ece29b80 10 client.16743 fetch_fsmap learned FSMap version 42 2024-06-10T03:02:52.900 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.894+0000 7f38ece29b80 1 -- 172.21.15.52:0/2196195782 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55f953ac89c0 con 0x55f953ab7880 2024-06-10T03:02:52.900 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.895+0000 7f38bb7fe700 1 -- 172.21.15.52:0/2196195782 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f38cc004d10 con 0x55f953ab7880 2024-06-10T03:02:52.900 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.895+0000 7f38bb7fe700 1 -- 172.21.15.52:0/2196195782 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 42) ==== 8394+0+0 (secure 0 0 0) 0x7f38d40cdce0 con 0x55f953ab7880 2024-06-10T03:02:52.900 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.895+0000 7f38ece29b80 10 client.16743 fetch_fsmap finished waiting for FSMap version 42 2024-06-10T03:02:52.900 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.895+0000 7f38ece29b80 10 client.16743 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-06-10T03:02:52.900 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.895+0000 7f38ece29b80 20 client.16743 populate_metadata read hostname 'smithi052' 2024-06-10T03:02:52.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.895+0000 7f38ece29b80 1 --2- 172.21.15.52:0/2196195782 >> [v2:172.21.15.188:6836/819317726,v1:172.21.15.188:6837/819317726] conn(0x55f953acae90 0x55f953acd2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:02:52.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.895+0000 7f38ece29b80 4 client.16743 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T03:02:52.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.895+0000 7f38ece29b80 1 -- 172.21.15.52:0/2196195782 --> [v2:172.21.15.188:6836/819317726,v1:172.21.15.188:6837/819317726] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f953a0fd50 con 0x55f953acae90 2024-06-10T03:02:52.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.895+0000 7f38dbfff700 1 --2- 172.21.15.52:0/2196195782 >> [v2:172.21.15.188:6836/819317726,v1:172.21.15.188:6837/819317726] conn(0x55f953acae90 0x55f953acd2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:02:52.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.895+0000 7f38dbfff700 1 --2- 172.21.15.52:0/2196195782 >> [v2:172.21.15.188:6836/819317726,v1:172.21.15.188:6837/819317726] conn(0x55f953acae90 0x55f953acd2b0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f9071ad19ecb0848 server_cookie=1d325b19352fd55 in_seq=0 out_seq=0 2024-06-10T03:02:52.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.895+0000 7f38bb7fe700 10 client.16743 ms_handle_connect on v2:172.21.15.188:6836/819317726 2024-06-10T03:02:52.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.896+0000 7f38bb7fe700 1 -- 172.21.15.52:0/2196195782 <== mds.1 v2:172.21.15.188:6836/819317726 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55f953a0fd50 con 0x55f953acae90 2024-06-10T03:02:52.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.896+0000 7f38bb7fe700 10 client.16743 handle_command_reply: tid=0 2024-06-10T03:02:52.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.896+0000 7f38ece29b80 1 client.16743 shutdown 2024-06-10T03:02:52.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.897+0000 7f38ece29b80 1 -- 172.21.15.52:0/2196195782 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f953ab7880 msgr2=0x55f953abae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:02:52.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.897+0000 7f38ece29b80 1 --2- 172.21.15.52:0/2196195782 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f953ab7880 0x55f953abae00 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f38d40cdb90 tx=0x7f38d401db70 comp rx=0 tx=0).stop 2024-06-10T03:02:52.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.897+0000 7f38ece29b80 1 -- 172.21.15.52:0/2196195782 shutdown_connections 2024-06-10T03:02:52.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.897+0000 7f38ece29b80 1 --2- 172.21.15.52:0/2196195782 >> [v2:172.21.15.188:6836/819317726,v1:172.21.15.188:6837/819317726] conn(0x55f953acae90 0x55f953acd2b0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:52.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.897+0000 7f38ece29b80 1 --2- 172.21.15.52:0/2196195782 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f953ab7880 0x55f953abae00 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:52.902 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.897+0000 7f38ece29b80 1 --2- 172.21.15.52:0/2196195782 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f953ab17a0 0x55f953abd5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:52.902 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.897+0000 7f38ece29b80 1 --2- 172.21.15.52:0/2196195782 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f953ab6f60 0x55f953aba8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:52.902 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.897+0000 7f38bb7fe700 0 client.16743 ms_handle_reset on v2:172.21.15.188:6836/819317726 2024-06-10T03:02:52.902 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.897+0000 7f38ece29b80 1 -- 172.21.15.52:0/2196195782 >> 172.21.15.52:0/2196195782 conn(0x55f9539de080 msgr2=0x55f953aab700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:02:52.902 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.897+0000 7f38ece29b80 1 -- 172.21.15.52:0/2196195782 shutdown_connections 2024-06-10T03:02:52.902 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.897+0000 7f38ece29b80 1 -- 172.21.15.52:0/2196195782 wait complete. 2024-06-10T03:02:52.902 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.897+0000 7f38ece29b80 20 client.16743 trim_cache size 0 max 16384 2024-06-10T03:02:52.913 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.912+0000 7f38ece29b80 1 Processor -- start 2024-06-10T03:02:52.913 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.912+0000 7f38ece29b80 1 -- start start 2024-06-10T03:02:52.913 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.912+0000 7f38ece29b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f953ab17a0 0x55f953abd100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:02:52.913 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.912+0000 7f38ece29b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f953ab6f60 0x55f953aabe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:02:52.913 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.912+0000 7f38ece29b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f953ab7880 0x55f953aac3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:02:52.913 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.912+0000 7f38ece29b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f38dc1b0460 con 0x55f953ab7880 2024-06-10T03:02:52.914 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.912+0000 7f38ece29b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f38dc1b20f0 con 0x55f953ab17a0 2024-06-10T03:02:52.914 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.912+0000 7f38ece29b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f38dc1b0b90 con 0x55f953ab6f60 2024-06-10T03:02:52.914 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.913+0000 7f38e0b43700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f953ab7880 0x55f953aac3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:02:52.914 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.913+0000 7f38db7fe700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f953ab6f60 0x55f953aabe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:02:52.914 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.913+0000 7f38e0b43700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f953ab7880 0x55f953aac3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58184/0 (socket says 172.21.15.52:58184) 2024-06-10T03:02:52.914 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.913+0000 7f38db7fe700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f953ab6f60 0x55f953aabe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.52:36284/0 (socket says 172.21.15.52:36284) 2024-06-10T03:02:52.914 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.913+0000 7f38e0b43700 1 -- 172.21.15.52:0/1896246197 learned_addr learned my addr 172.21.15.52:0/1896246197 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:02:52.914 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.913+0000 7f38dbfff700 1 --2- 172.21.15.52:0/1896246197 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f953ab17a0 0x55f953abd100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:02:52.914 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.913+0000 7f38e0b43700 1 -- 172.21.15.52:0/1896246197 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f953ab6f60 msgr2=0x55f953aabe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:02:52.914 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.913+0000 7f38e0b43700 1 --2- 172.21.15.52:0/1896246197 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f953ab6f60 0x55f953aabe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:52.914 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.913+0000 7f38e0b43700 1 -- 172.21.15.52:0/1896246197 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f953ab17a0 msgr2=0x55f953abd100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:02:52.914 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.913+0000 7f38e0b43700 1 --2- 172.21.15.52:0/1896246197 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f953ab17a0 0x55f953abd100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:52.914 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.913+0000 7f38e0b43700 1 -- 172.21.15.52:0/1896246197 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f953ab8e50 con 0x55f953ab7880 2024-06-10T03:02:52.915 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.913+0000 7f38dbfff700 1 --2- 172.21.15.52:0/1896246197 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f953ab17a0 0x55f953abd100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T03:02:52.915 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.913+0000 7f38e0b43700 1 --2- 172.21.15.52:0/1896246197 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f953ab7880 0x55f953aac3c0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f38d4098f90 tx=0x7f38d409ab20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ee1437552a45a72 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:02:52.915 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.913+0000 7f38c1ffb700 1 -- 172.21.15.52:0/1896246197 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38d409bf70 con 0x55f953ab7880 2024-06-10T03:02:52.915 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.913+0000 7f38c1ffb700 1 -- 172.21.15.52:0/1896246197 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f38d4098040 con 0x55f953ab7880 2024-06-10T03:02:52.915 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.913+0000 7f38c1ffb700 1 -- 172.21.15.52:0/1896246197 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38d409c870 con 0x55f953ab7880 2024-06-10T03:02:52.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.914+0000 7f38ece29b80 1 -- 172.21.15.52:0/1896246197 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f953ab7880 msgr2=0x55f953aac3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:02:52.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.914+0000 7f38ece29b80 1 --2- 172.21.15.52:0/1896246197 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f953ab7880 0x55f953aac3c0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f38d4098f90 tx=0x7f38d409ab20 comp rx=0 tx=0).stop 2024-06-10T03:02:52.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.914+0000 7f38ece29b80 1 -- 172.21.15.52:0/1896246197 shutdown_connections 2024-06-10T03:02:52.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.914+0000 7f38ece29b80 1 --2- 172.21.15.52:0/1896246197 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f953ab7880 0x55f953aac3c0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:52.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.914+0000 7f38ece29b80 1 --2- 172.21.15.52:0/1896246197 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f953ab17a0 0x55f953abd100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:52.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.914+0000 7f38ece29b80 1 --2- 172.21.15.52:0/1896246197 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f953ab6f60 0x55f953aabe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:52.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.914+0000 7f38ece29b80 1 -- 172.21.15.52:0/1896246197 >> 172.21.15.52:0/1896246197 conn(0x55f9539de080 msgr2=0x55f953ab4e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:02:52.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.914+0000 7f38ece29b80 1 -- 172.21.15.52:0/1896246197 shutdown_connections 2024-06-10T03:02:52.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.914+0000 7f38ece29b80 1 -- 172.21.15.52:0/1896246197 wait complete. 2024-06-10T03:02:52.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.914+0000 7f38ece29b80 1 -- start start 2024-06-10T03:02:52.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.914+0000 7f38ece29b80 1 Processor -- start 2024-06-10T03:02:52.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.915+0000 7f38ece29b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f953ab17a0 0x55f953ac3520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:02:52.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.915+0000 7f38ece29b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f953ab6f60 0x55f953ac3a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:02:52.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.915+0000 7f38ece29b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f953ab7880 0x55f953ad1f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:02:52.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.915+0000 7f38ece29b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f38dc1b19f0 con 0x55f953ab7880 2024-06-10T03:02:52.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.915+0000 7f38ece29b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f38dc1b12f0 con 0x55f953ab6f60 2024-06-10T03:02:52.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.915+0000 7f38ece29b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f38dc1ae400 con 0x55f953ab17a0 2024-06-10T03:02:52.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.915+0000 7f38ece29b80 10 client.0 _ll_register_callbacks cb 0x55f9539985b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T03:02:52.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.915+0000 7f38dbfff700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f953ab17a0 0x55f953ac3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:02:52.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.915+0000 7f38e0b43700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f953ab7880 0x55f953ad1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:02:52.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.915+0000 7f38dbfff700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f953ab17a0 0x55f953ac3520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.52:36286/0 (socket says 172.21.15.52:36286) 2024-06-10T03:02:52.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.915+0000 7f38dbfff700 1 -- 172.21.15.52:0/2960253565 learned_addr learned my addr 172.21.15.52:0/2960253565 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:02:52.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.915+0000 7f38db7fe700 1 --2- 172.21.15.52:0/2960253565 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f953ab6f60 0x55f953ac3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:02:52.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.915+0000 7f38e0b43700 1 -- 172.21.15.52:0/2960253565 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f953ab17a0 msgr2=0x55f953ac3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:02:52.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.915+0000 7f38e0b43700 1 --2- 172.21.15.52:0/2960253565 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f953ab17a0 0x55f953ac3520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:52.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.915+0000 7f38e0b43700 1 -- 172.21.15.52:0/2960253565 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f953ab6f60 msgr2=0x55f953ac3a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:02:52.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.915+0000 7f38e0b43700 1 --2- 172.21.15.52:0/2960253565 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f953ab6f60 0x55f953ac3a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:52.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.915+0000 7f38e0b43700 1 -- 172.21.15.52:0/2960253565 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f953ad2590 con 0x55f953ab7880 2024-06-10T03:02:52.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.915+0000 7f38db7fe700 1 --2- 172.21.15.52:0/2960253565 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f953ab6f60 0x55f953ac3a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T03:02:52.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.916+0000 7f38e0b43700 1 --2- 172.21.15.52:0/2960253565 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f953ab7880 0x55f953ad1f20 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f38d4098a70 tx=0x7f38d40cd870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80881c924df94796 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:02:52.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.916+0000 7f38bb7fe700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-10T03:02:52.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.916+0000 7f38bb7fe700 1 -- 172.21.15.52:0/2960253565 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38d409c610 con 0x55f953ab7880 2024-06-10T03:02:52.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.916+0000 7f38bb7fe700 1 -- 172.21.15.52:0/2960253565 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f38d40a1b50 con 0x55f953ab7880 2024-06-10T03:02:52.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.916+0000 7f38ece29b80 1 -- 172.21.15.52:0/2960253565 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f38d4098040 con 0x55f953ab7880 2024-06-10T03:02:52.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.916+0000 7f38bb7fe700 1 -- 172.21.15.52:0/2960253565 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f38d40a51f0 con 0x55f953ab7880 2024-06-10T03:02:52.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.916+0000 7f38ece29b80 10 client.16749 fetch_fsmap learned FSMap version 42 2024-06-10T03:02:52.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.916+0000 7f38ece29b80 1 -- 172.21.15.52:0/2960253565 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55f953ad2590 con 0x55f953ab7880 2024-06-10T03:02:52.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.916+0000 7f38bb7fe700 1 -- 172.21.15.52:0/2960253565 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f38d4098040 con 0x55f953ab7880 2024-06-10T03:02:52.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.916+0000 7f38bb7fe700 1 -- 172.21.15.52:0/2960253565 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 42) ==== 8394+0+0 (secure 0 0 0) 0x7f38d40cdce0 con 0x55f953ab7880 2024-06-10T03:02:52.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.916+0000 7f38ece29b80 10 client.16749 fetch_fsmap finished waiting for FSMap version 42 2024-06-10T03:02:52.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.916+0000 7f38ece29b80 10 client.16749 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-06-10T03:02:52.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.916+0000 7f38ece29b80 20 client.16749 populate_metadata read hostname 'smithi052' 2024-06-10T03:02:52.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.916+0000 7f38ece29b80 1 --2- 172.21.15.52:0/2960253565 >> [v2:172.21.15.188:6836/819317726,v1:172.21.15.188:6837/819317726] conn(0x55f953b35780 0x55f953b37ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:02:52.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.916+0000 7f38ece29b80 4 client.16749 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T03:02:52.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.916+0000 7f38ece29b80 1 -- 172.21.15.52:0/2960253565 --> [v2:172.21.15.188:6836/819317726,v1:172.21.15.188:6837/819317726] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f953a0a2d0 con 0x55f953b35780 2024-06-10T03:02:52.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.916+0000 7f38dbfff700 1 --2- 172.21.15.52:0/2960253565 >> [v2:172.21.15.188:6836/819317726,v1:172.21.15.188:6837/819317726] conn(0x55f953b35780 0x55f953b37ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:02:52.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.917+0000 7f38dbfff700 1 --2- 172.21.15.52:0/2960253565 >> [v2:172.21.15.188:6836/819317726,v1:172.21.15.188:6837/819317726] conn(0x55f953b35780 0x55f953b37ba0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c3fc5f2fcbac2bc0 server_cookie=dced8677687b7826 in_seq=0 out_seq=0 2024-06-10T03:02:52.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.917+0000 7f38bb7fe700 10 client.16749 ms_handle_connect on v2:172.21.15.188:6836/819317726 2024-06-10T03:02:52.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.917+0000 7f38bb7fe700 1 -- 172.21.15.52:0/2960253565 <== mds.1 v2:172.21.15.188:6836/819317726 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f953a0a2d0 con 0x55f953b35780 2024-06-10T03:02:52.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.917+0000 7f38bb7fe700 10 client.16749 handle_command_reply: tid=0 2024-06-10T03:02:52.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.918+0000 7f38ece29b80 1 client.16749 shutdown 2024-06-10T03:02:52.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.918+0000 7f38ece29b80 1 -- 172.21.15.52:0/2960253565 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f953ab7880 msgr2=0x55f953ad1f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:02:52.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.918+0000 7f38ece29b80 1 --2- 172.21.15.52:0/2960253565 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f953ab7880 0x55f953ad1f20 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f38d4098a70 tx=0x7f38d40cd870 comp rx=0 tx=0).stop 2024-06-10T03:02:52.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.918+0000 7f38ece29b80 1 -- 172.21.15.52:0/2960253565 shutdown_connections 2024-06-10T03:02:52.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.918+0000 7f38ece29b80 1 --2- 172.21.15.52:0/2960253565 >> [v2:172.21.15.188:6836/819317726,v1:172.21.15.188:6837/819317726] conn(0x55f953b35780 0x55f953b37ba0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:52.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.918+0000 7f38ece29b80 1 --2- 172.21.15.52:0/2960253565 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f953ab7880 0x55f953ad1f20 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:52.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.918+0000 7f38ece29b80 1 --2- 172.21.15.52:0/2960253565 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f953ab17a0 0x55f953ac3520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:52.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.918+0000 7f38ece29b80 1 --2- 172.21.15.52:0/2960253565 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f953ab6f60 0x55f953ac3a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:52.923 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.918+0000 7f38ece29b80 1 -- 172.21.15.52:0/2960253565 >> 172.21.15.52:0/2960253565 conn(0x55f9539de080 msgr2=0x55f953ab9060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:02:52.923 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.918+0000 7f38bb7fe700 0 client.16749 ms_handle_reset on v2:172.21.15.188:6836/819317726 2024-06-10T03:02:52.923 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.918+0000 7f38ece29b80 1 -- 172.21.15.52:0/2960253565 shutdown_connections 2024-06-10T03:02:52.923 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.918+0000 7f38ece29b80 1 -- 172.21.15.52:0/2960253565 wait complete. 2024-06-10T03:02:52.923 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.918+0000 7f38ece29b80 20 client.16749 trim_cache size 0 max 16384 2024-06-10T03:02:52.923 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-06-10T03:02:52.924 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.918+0000 7f38e25a6700 1 -- 172.21.15.52:0/1417996955 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f38c40a2ca0 msgr2=0x7f38c40a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:02:52.924 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.918+0000 7f38e25a6700 1 --2- 172.21.15.52:0/1417996955 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f38c40a2ca0 0x7f38c40a5160 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f38cc009c60 tx=0x7f38cc009260 comp rx=0 tx=0).stop 2024-06-10T03:02:52.924 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.918+0000 7f38e25a6700 1 -- 172.21.15.52:0/1417996955 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f38dc107880 msgr2=0x7f38dc19f630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:02:52.924 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.918+0000 7f38e25a6700 1 --2- 172.21.15.52:0/1417996955 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f38dc107880 0x7f38dc19f630 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f38d400d010 tx=0x7f38d40079a0 comp rx=0 tx=0).stop 2024-06-10T03:02:52.924 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.919+0000 7f38e25a6700 1 -- 172.21.15.52:0/1417996955 shutdown_connections 2024-06-10T03:02:52.924 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.919+0000 7f38e25a6700 1 --2- 172.21.15.52:0/1417996955 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f38c40a2ca0 0x7f38c40a5160 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:52.924 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.919+0000 7f38e25a6700 1 --2- 172.21.15.52:0/1417996955 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f38dc107880 0x7f38dc19f630 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:52.924 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.919+0000 7f38e25a6700 1 --2- 172.21.15.52:0/1417996955 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f38dc100c40 0x7f38dc19ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:52.924 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.919+0000 7f38e25a6700 1 --2- 172.21.15.52:0/1417996955 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f38dc1015f0 0x7f38dc19b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:52.924 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.919+0000 7f38e25a6700 1 -- 172.21.15.52:0/1417996955 >> 172.21.15.52:0/1417996955 conn(0x7f38dc077990 msgr2=0x7f38dc105e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:02:52.924 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.919+0000 7f38e25a6700 1 -- 172.21.15.52:0/1417996955 shutdown_connections 2024-06-10T03:02:52.925 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:52.919+0000 7f38e25a6700 1 -- 172.21.15.52:0/1417996955 wait complete. 2024-06-10T03:02:52.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:02:52 smithi188 ceph-mon[36599]: pgmap v694: 129 pgs: 129 active+clean; 337 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 525 op/s 2024-06-10T03:02:53.987 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-06-10T03:02:54.810 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:02:54 smithi052 ceph-mon[32362]: pgmap v695: 129 pgs: 129 active+clean; 306 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 614 op/s 2024-06-10T03:02:54.888 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:02:54 smithi120 ceph-mon[36634]: pgmap v695: 129 pgs: 129 active+clean; 306 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 614 op/s 2024-06-10T03:02:54.912 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T03:02:54.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:02:54 smithi188 ceph-mon[36599]: pgmap v695: 129 pgs: 129 active+clean; 306 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 614 op/s 2024-06-10T03:02:55.010 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.008+0000 7f0531c5e700 1 -- 172.21.15.52:0/3253685006 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f052c11bc70 msgr2=0x7f052c11e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:02:55.010 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.008+0000 7f0531c5e700 1 --2- 172.21.15.52:0/3253685006 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f052c11bc70 0x7f052c11e060 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f052400b0d0 tx=0x7f052400b3e0 comp rx=0 tx=0).stop 2024-06-10T03:02:55.010 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.010+0000 7f0531c5e700 1 -- 172.21.15.52:0/3253685006 shutdown_connections 2024-06-10T03:02:55.011 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.010+0000 7f0531c5e700 1 --2- 172.21.15.52:0/3253685006 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f052c11bc70 0x7f052c11e060 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:55.011 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.010+0000 7f0531c5e700 1 --2- 172.21.15.52:0/3253685006 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f052c110ce0 0x7f052c109490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:55.011 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.010+0000 7f0531c5e700 1 --2- 172.21.15.52:0/3253685006 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f052c1099d0 0x7f052c109e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:55.011 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.010+0000 7f0531c5e700 1 -- 172.21.15.52:0/3253685006 >> 172.21.15.52:0/3253685006 conn(0x7f052c06d600 msgr2=0x7f052c06da10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:02:55.011 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.010+0000 7f0531c5e700 1 -- 172.21.15.52:0/3253685006 shutdown_connections 2024-06-10T03:02:55.011 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.010+0000 7f0531c5e700 1 -- 172.21.15.52:0/3253685006 wait complete. 2024-06-10T03:02:55.013 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.011+0000 7f0531c5e700 1 Processor -- start 2024-06-10T03:02:55.013 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.012+0000 7f0531c5e700 1 -- start start 2024-06-10T03:02:55.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.012+0000 7f0531c5e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f052c1099d0 0x7f052c1ad640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:02:55.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.012+0000 7f0531c5e700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f052c110ce0 0x7f052c1adb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:02:55.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.012+0000 7f0531c5e700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f052c11bc70 0x7f052c1a77c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:02:55.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.012+0000 7f0531c5e700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f052c1215e0 con 0x7f052c1099d0 2024-06-10T03:02:55.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.012+0000 7f0531c5e700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f052c121460 con 0x7f052c110ce0 2024-06-10T03:02:55.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.012+0000 7f0531c5e700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f052c121760 con 0x7f052c11bc70 2024-06-10T03:02:55.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.013+0000 7f052bfff700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f052c11bc70 0x7f052c1a77c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:02:55.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.013+0000 7f052bfff700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f052c11bc70 0x7f052c1a77c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.52:36292/0 (socket says 172.21.15.52:36292) 2024-06-10T03:02:55.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.013+0000 7f052bfff700 1 -- 172.21.15.52:0/2030872684 learned_addr learned my addr 172.21.15.52:0/2030872684 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:02:55.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.013+0000 7f052b7fe700 1 --2- 172.21.15.52:0/2030872684 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f052c1099d0 0x7f052c1ad640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:02:55.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.013+0000 7f052affd700 1 --2- 172.21.15.52:0/2030872684 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f052c110ce0 0x7f052c1adb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:02:55.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.013+0000 7f052bfff700 1 -- 172.21.15.52:0/2030872684 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f052c110ce0 msgr2=0x7f052c1adb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:02:55.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.013+0000 7f052bfff700 1 --2- 172.21.15.52:0/2030872684 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f052c110ce0 0x7f052c1adb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:55.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.013+0000 7f052bfff700 1 -- 172.21.15.52:0/2030872684 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f052c1099d0 msgr2=0x7f052c1ad640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:02:55.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.013+0000 7f052bfff700 1 --2- 172.21.15.52:0/2030872684 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f052c1099d0 0x7f052c1ad640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:55.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.013+0000 7f052bfff700 1 -- 172.21.15.52:0/2030872684 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f052c1a7fa0 con 0x7f052c11bc70 2024-06-10T03:02:55.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.013+0000 7f052affd700 1 --2- 172.21.15.52:0/2030872684 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f052c110ce0 0x7f052c1adb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T03:02:55.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.013+0000 7f052bfff700 1 --2- 172.21.15.52:0/2030872684 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f052c11bc70 0x7f052c1a77c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f0524006070 tx=0x7f0524004d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:02:55.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.014+0000 7f0528ff9700 1 -- 172.21.15.52:0/2030872684 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0524021070 con 0x7f052c11bc70 2024-06-10T03:02:55.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.014+0000 7f0528ff9700 1 -- 172.21.15.52:0/2030872684 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0524003c00 con 0x7f052c11bc70 2024-06-10T03:02:55.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.015+0000 7f0531c5e700 1 -- 172.21.15.52:0/2030872684 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f052c1a8230 con 0x7f052c11bc70 2024-06-10T03:02:55.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.015+0000 7f0531c5e700 1 -- 172.21.15.52:0/2030872684 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f052c1b44a0 con 0x7f052c11bc70 2024-06-10T03:02:55.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.015+0000 7f0528ff9700 1 -- 172.21.15.52:0/2030872684 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f052401d450 con 0x7f052c11bc70 2024-06-10T03:02:55.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.022+0000 7f053c4e1b80 1 Processor -- start 2024-06-10T03:02:55.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.022+0000 7f053c4e1b80 1 -- start start 2024-06-10T03:02:55.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.022+0000 7f053c4e1b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f83dedd590 0x55f83dedd970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:02:55.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.022+0000 7f053c4e1b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f83deddeb0 0x55f83ded77a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:02:55.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.022+0000 7f053c4e1b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f83ded7ce0 0x55f83ded8100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:02:55.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.022+0000 7f053c4e1b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55f83ddca0f0 con 0x55f83dedd590 2024-06-10T03:02:55.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.022+0000 7f053c4e1b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55f83de289e0 con 0x55f83deddeb0 2024-06-10T03:02:55.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.022+0000 7f053c4e1b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55f83dd905f0 con 0x55f83ded7ce0 2024-06-10T03:02:55.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.022+0000 7f0528ff9700 1 -- 172.21.15.52:0/2030872684 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f052401d5f0 con 0x7f052c11bc70 2024-06-10T03:02:55.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.023+0000 7f052b7fe700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f83deddeb0 0x55f83ded77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:02:55.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.023+0000 7f052b7fe700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f83deddeb0 0x55f83ded77a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.52:56650/0 (socket says 172.21.15.52:56650) 2024-06-10T03:02:55.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.023+0000 7f052b7fe700 1 -- 172.21.15.52:0/1878318317 learned_addr learned my addr 172.21.15.52:0/1878318317 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:02:55.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.023+0000 7f052affd700 1 --2- 172.21.15.52:0/1878318317 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f83dedd590 0x55f83dedd970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:02:55.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.023+0000 7f052affd700 1 --2- 172.21.15.52:0/1878318317 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f83ded7ce0 0x55f83ded8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:02:55.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.023+0000 7f052affd700 1 -- 172.21.15.52:0/1878318317 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f83ded7ce0 msgr2=0x55f83ded8100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:02:55.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.023+0000 7f052affd700 1 --2- 172.21.15.52:0/1878318317 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f83ded7ce0 0x55f83ded8100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:55.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.023+0000 7f052affd700 1 -- 172.21.15.52:0/1878318317 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f83deddeb0 msgr2=0x55f83ded77a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:02:55.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.023+0000 7f052affd700 1 --2- 172.21.15.52:0/1878318317 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f83deddeb0 0x55f83ded77a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:55.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.023+0000 7f052affd700 1 -- 172.21.15.52:0/1878318317 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f83dd77a60 con 0x55f83dedd590 2024-06-10T03:02:55.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.024+0000 7f052affd700 1 --2- 172.21.15.52:0/1878318317 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f83dedd590 0x55f83dedd970 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f0520005d90 tx=0x7f052000b440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb5902b61421ae27 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:02:55.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.024+0000 7f0528ff9700 1 --2- 172.21.15.52:0/2030872684 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f05140a2eb0 0x7f05140a5370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:02:55.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.024+0000 7f0528ff9700 1 -- 172.21.15.52:0/2030872684 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f05240d03c0 con 0x7f052c11bc70 2024-06-10T03:02:55.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.024+0000 7f0511ffb700 1 -- 172.21.15.52:0/1878318317 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0520004a30 con 0x55f83dedd590 2024-06-10T03:02:55.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.024+0000 7f0511ffb700 1 -- 172.21.15.52:0/1878318317 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0520017030 con 0x55f83dedd590 2024-06-10T03:02:55.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.024+0000 7f052b7fe700 1 --2- 172.21.15.52:0/2030872684 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f05140a2eb0 0x7f05140a5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:02:55.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.025+0000 7f053c4e1b80 1 -- 172.21.15.52:0/1878318317 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f83dedd590 msgr2=0x55f83dedd970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:02:55.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.025+0000 7f053c4e1b80 1 --2- 172.21.15.52:0/1878318317 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f83dedd590 0x55f83dedd970 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f0520005d90 tx=0x7f052000b440 comp rx=0 tx=0).stop 2024-06-10T03:02:55.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.026+0000 7f053c4e1b80 1 -- 172.21.15.52:0/1878318317 shutdown_connections 2024-06-10T03:02:55.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.026+0000 7f053c4e1b80 1 --2- 172.21.15.52:0/1878318317 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f83ded7ce0 0x55f83ded8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:55.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.026+0000 7f053c4e1b80 1 --2- 172.21.15.52:0/1878318317 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f83dedd590 0x55f83dedd970 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:55.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.026+0000 7f053c4e1b80 1 --2- 172.21.15.52:0/1878318317 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f83deddeb0 0x55f83ded77a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:55.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.026+0000 7f053c4e1b80 1 -- 172.21.15.52:0/1878318317 >> 172.21.15.52:0/1878318317 conn(0x55f83de04210 msgr2=0x55f83ded2a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:02:55.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.026+0000 7f052b7fe700 1 --2- 172.21.15.52:0/2030872684 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f05140a2eb0 0x7f05140a5370 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f051c0045f0 tx=0x7f051c00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:02:55.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.027+0000 7f053c4e1b80 1 -- 172.21.15.52:0/1878318317 shutdown_connections 2024-06-10T03:02:55.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.027+0000 7f053c4e1b80 1 -- 172.21.15.52:0/1878318317 wait complete. 2024-06-10T03:02:55.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.028+0000 7f053c4e1b80 1 -- start start 2024-06-10T03:02:55.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.028+0000 7f053c4e1b80 1 Processor -- start 2024-06-10T03:02:55.030 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.030+0000 7f053c4e1b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f83ded7ce0 0x55f83dee3a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:02:55.030 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.030+0000 7f053c4e1b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f83dedd590 0x55f83dee0df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:02:55.030 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.030+0000 7f053c4e1b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f83deddeb0 0x55f83dee1330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:02:55.030 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.030+0000 7f053c4e1b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55f83ddc9150 con 0x55f83ded7ce0 2024-06-10T03:02:55.030 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.030+0000 7f053c4e1b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55f83de249a0 con 0x55f83deddeb0 2024-06-10T03:02:55.030 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.030+0000 7f053c4e1b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55f83dee4880 con 0x55f83dedd590 2024-06-10T03:02:55.030 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.030+0000 7f053c4e1b80 10 client.0 _ll_register_callbacks cb 0x55f83ddbe740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T03:02:55.030 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.030+0000 7f052b7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f83ded7ce0 0x55f83dee3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:02:55.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.030+0000 7f052b7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f83ded7ce0 0x55f83dee3a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58224/0 (socket says 172.21.15.52:58224) 2024-06-10T03:02:55.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.030+0000 7f052b7fe700 1 -- 172.21.15.52:0/1648274712 learned_addr learned my addr 172.21.15.52:0/1648274712 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:02:55.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.031+0000 7f052bfff700 1 --2- 172.21.15.52:0/1648274712 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f83deddeb0 0x55f83dee1330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:02:55.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.031+0000 7f052bfff700 1 -- 172.21.15.52:0/1648274712 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f83dedd590 msgr2=0x55f83dee0df0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-10T03:02:55.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.031+0000 7f052bfff700 1 --2- 172.21.15.52:0/1648274712 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f83dedd590 0x55f83dee0df0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:55.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.031+0000 7f052bfff700 1 -- 172.21.15.52:0/1648274712 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f83ded7ce0 msgr2=0x55f83dee3a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:02:55.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.031+0000 7f052bfff700 1 --2- 172.21.15.52:0/1648274712 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f83ded7ce0 0x55f83dee3a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:55.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.031+0000 7f052bfff700 1 -- 172.21.15.52:0/1648274712 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f83ddee8e0 con 0x55f83deddeb0 2024-06-10T03:02:55.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.032+0000 7f052bfff700 1 --2- 172.21.15.52:0/1648274712 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f83deddeb0 0x55f83dee1330 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f0524009fd0 tx=0x7f052409c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=378a9b00fb630d8f server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:02:55.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.032+0000 7f050affd700 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-06-10T03:02:55.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.032+0000 7f050affd700 1 -- 172.21.15.52:0/1648274712 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f052402a050 con 0x55f83deddeb0 2024-06-10T03:02:55.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.032+0000 7f050affd700 1 -- 172.21.15.52:0/1648274712 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f05240cdbc0 con 0x55f83deddeb0 2024-06-10T03:02:55.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.032+0000 7f053c4e1b80 1 -- 172.21.15.52:0/1648274712 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0520017030 con 0x55f83deddeb0 2024-06-10T03:02:55.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.032+0000 7f050affd700 1 -- 172.21.15.52:0/1648274712 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f052409c8a0 con 0x55f83deddeb0 2024-06-10T03:02:55.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.033+0000 7f053c4e1b80 10 client.26255 fetch_fsmap learned FSMap version 42 2024-06-10T03:02:55.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.033+0000 7f053c4e1b80 1 -- 172.21.15.52:0/1648274712 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55f83defff50 con 0x55f83deddeb0 2024-06-10T03:02:55.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.034+0000 7f050affd700 1 -- 172.21.15.52:0/1648274712 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0520017030 con 0x55f83deddeb0 2024-06-10T03:02:55.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.034+0000 7f050affd700 1 -- 172.21.15.52:0/1648274712 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 42) ==== 8394+0+0 (secure 0 0 0) 0x7f0524011070 con 0x55f83deddeb0 2024-06-10T03:02:55.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.034+0000 7f053c4e1b80 10 client.26255 fetch_fsmap finished waiting for FSMap version 42 2024-06-10T03:02:55.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.034+0000 7f053c4e1b80 10 client.26255 resolve_mds: validated gid 24484 aka daemon mds.i 2024-06-10T03:02:55.035 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.034+0000 7f053c4e1b80 20 client.26255 populate_metadata read hostname 'smithi052' 2024-06-10T03:02:55.035 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.034+0000 7f053c4e1b80 1 --2- 172.21.15.52:0/1648274712 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55f83df02510 0x55f83df04970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:02:55.035 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.034+0000 7f053c4e1b80 4 client.26255 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T03:02:55.035 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.034+0000 7f053c4e1b80 1 -- 172.21.15.52:0/1648274712 --> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f83de35ee0 con 0x55f83df02510 2024-06-10T03:02:55.035 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.034+0000 7f052b7fe700 1 --2- 172.21.15.52:0/1648274712 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55f83df02510 0x55f83df04970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:02:55.035 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.035+0000 7f050affd700 10 client.26255 ms_handle_connect on v2:172.21.15.188:6838/310444836 2024-06-10T03:02:55.035 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.035+0000 7f052b7fe700 1 --2- 172.21.15.52:0/1648274712 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55f83df02510 0x55f83df04970 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e18fece9da2a38 server_cookie=545a053ef2b61e55 in_seq=0 out_seq=0 2024-06-10T03:02:55.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.037+0000 7f050affd700 1 -- 172.21.15.52:0/1648274712 <== mds.0 v2:172.21.15.188:6838/310444836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55f83de35ee0 con 0x55f83df02510 2024-06-10T03:02:55.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.037+0000 7f050affd700 10 client.26255 handle_command_reply: tid=0 2024-06-10T03:02:55.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.037+0000 7f053c4e1b80 1 client.26255 shutdown 2024-06-10T03:02:55.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.037+0000 7f053c4e1b80 1 -- 172.21.15.52:0/1648274712 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f83deddeb0 msgr2=0x55f83dee1330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:02:55.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.037+0000 7f053c4e1b80 1 --2- 172.21.15.52:0/1648274712 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f83deddeb0 0x55f83dee1330 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f0524009fd0 tx=0x7f052409c490 comp rx=0 tx=0).stop 2024-06-10T03:02:55.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.038+0000 7f053c4e1b80 1 -- 172.21.15.52:0/1648274712 shutdown_connections 2024-06-10T03:02:55.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.038+0000 7f053c4e1b80 1 --2- 172.21.15.52:0/1648274712 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55f83df02510 0x55f83df04970 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:55.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.038+0000 7f053c4e1b80 1 --2- 172.21.15.52:0/1648274712 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f83deddeb0 0x55f83dee1330 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:55.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.038+0000 7f053c4e1b80 1 --2- 172.21.15.52:0/1648274712 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f83ded7ce0 0x55f83dee3a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:55.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.038+0000 7f050affd700 0 client.26255 ms_handle_reset on v2:172.21.15.188:6838/310444836 2024-06-10T03:02:55.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.038+0000 7f053c4e1b80 1 --2- 172.21.15.52:0/1648274712 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f83dedd590 0x55f83dee0df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:55.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.038+0000 7f053c4e1b80 1 -- 172.21.15.52:0/1648274712 >> 172.21.15.52:0/1648274712 conn(0x55f83de04210 msgr2=0x55f83ded13b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:02:55.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.038+0000 7f053c4e1b80 1 -- 172.21.15.52:0/1648274712 shutdown_connections 2024-06-10T03:02:55.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.038+0000 7f053c4e1b80 1 -- 172.21.15.52:0/1648274712 wait complete. 2024-06-10T03:02:55.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.038+0000 7f053c4e1b80 20 client.26255 trim_cache size 0 max 16384 2024-06-10T03:02:55.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.079+0000 7f053c4e1b80 1 Processor -- start 2024-06-10T03:02:55.081 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.081+0000 7f053c4e1b80 1 -- start start 2024-06-10T03:02:55.082 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.081+0000 7f053c4e1b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f83ded7ce0 0x55f83ded77a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:02:55.082 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.081+0000 7f053c4e1b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f83dedd590 0x55f83df0a240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:02:55.082 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.081+0000 7f053c4e1b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f83deddeb0 0x55f83df0a780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:02:55.082 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.081+0000 7f053c4e1b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f052c1b48f0 con 0x55f83ded7ce0 2024-06-10T03:02:55.082 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.081+0000 7f052b7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f83ded7ce0 0x55f83ded77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:02:55.082 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.082+0000 7f052b7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f83ded7ce0 0x55f83ded77a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58230/0 (socket says 172.21.15.52:58230) 2024-06-10T03:02:55.082 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.082+0000 7f052b7fe700 1 -- 172.21.15.52:0/152643750 learned_addr learned my addr 172.21.15.52:0/152643750 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:02:55.082 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.082+0000 7f052affd700 1 --2- 172.21.15.52:0/152643750 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f83dedd590 0x55f83df0a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:02:55.082 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.082+0000 7f053c4e1b80 1 -- 172.21.15.52:0/152643750 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f052c1b6580 con 0x55f83dedd590 2024-06-10T03:02:55.082 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.082+0000 7f053c4e1b80 1 -- 172.21.15.52:0/152643750 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f052c1b5020 con 0x55f83deddeb0 2024-06-10T03:02:55.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.082+0000 7f052b7fe700 1 -- 172.21.15.52:0/152643750 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f83deddeb0 msgr2=0x55f83df0a780 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-10T03:02:55.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.082+0000 7f052b7fe700 1 --2- 172.21.15.52:0/152643750 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f83deddeb0 0x55f83df0a780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:55.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.082+0000 7f052b7fe700 1 -- 172.21.15.52:0/152643750 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f83dedd590 msgr2=0x55f83df0a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:02:55.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.082+0000 7f052b7fe700 1 --2- 172.21.15.52:0/152643750 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f83dedd590 0x55f83df0a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:55.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.082+0000 7f052b7fe700 1 -- 172.21.15.52:0/152643750 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f83ded2be0 con 0x55f83ded7ce0 2024-06-10T03:02:55.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.083+0000 7f052b7fe700 1 --2- 172.21.15.52:0/152643750 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f83ded7ce0 0x55f83ded77a0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f051c00bf90 tx=0x7f051c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4efb83eb0936ba8b server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:02:55.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.084+0000 7f05117fa700 1 -- 172.21.15.52:0/152643750 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f051c003ea0 con 0x55f83ded7ce0 2024-06-10T03:02:55.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.084+0000 7f05117fa700 1 -- 172.21.15.52:0/152643750 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f051c0200d0 con 0x55f83ded7ce0 2024-06-10T03:02:55.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.084+0000 7f05117fa700 1 -- 172.21.15.52:0/152643750 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f051c029d20 con 0x55f83ded7ce0 2024-06-10T03:02:55.087 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.086+0000 7f053c4e1b80 1 -- 172.21.15.52:0/152643750 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f83ded7ce0 msgr2=0x55f83ded77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:02:55.087 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.087+0000 7f053c4e1b80 1 --2- 172.21.15.52:0/152643750 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f83ded7ce0 0x55f83ded77a0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f051c00bf90 tx=0x7f051c016000 comp rx=0 tx=0).stop 2024-06-10T03:02:55.088 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.088+0000 7f053c4e1b80 1 -- 172.21.15.52:0/152643750 shutdown_connections 2024-06-10T03:02:55.088 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.088+0000 7f053c4e1b80 1 --2- 172.21.15.52:0/152643750 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f83deddeb0 0x55f83df0a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:55.088 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.088+0000 7f053c4e1b80 1 --2- 172.21.15.52:0/152643750 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f83ded7ce0 0x55f83ded77a0 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:55.088 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.088+0000 7f053c4e1b80 1 --2- 172.21.15.52:0/152643750 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f83dedd590 0x55f83df0a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:55.088 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.088+0000 7f053c4e1b80 1 -- 172.21.15.52:0/152643750 >> 172.21.15.52:0/152643750 conn(0x55f83de04210 msgr2=0x55f83dee2eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:02:55.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.088+0000 7f053c4e1b80 1 -- 172.21.15.52:0/152643750 shutdown_connections 2024-06-10T03:02:55.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.089+0000 7f053c4e1b80 1 -- 172.21.15.52:0/152643750 wait complete. 2024-06-10T03:02:55.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.089+0000 7f053c4e1b80 1 -- start start 2024-06-10T03:02:55.091 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.090+0000 7f053c4e1b80 1 Processor -- start 2024-06-10T03:02:55.091 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.090+0000 7f053c4e1b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f83ded7ce0 0x55f83dee7d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:02:55.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.090+0000 7f053c4e1b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f83dedd590 0x55f83dee82d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:02:55.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.091+0000 7f052b7fe700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f83ded7ce0 0x55f83dee7d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:02:55.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.091+0000 7f052b7fe700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f83ded7ce0 0x55f83dee7d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.52:56686/0 (socket says 172.21.15.52:56686) 2024-06-10T03:02:55.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.091+0000 7f053c4e1b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f83deddeb0 0x55f83df6c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:02:55.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.091+0000 7f052b7fe700 1 -- 172.21.15.52:0/3362375871 learned_addr learned my addr 172.21.15.52:0/3362375871 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:02:55.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.091+0000 7f052bfff700 1 --2- 172.21.15.52:0/3362375871 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f83deddeb0 0x55f83df6c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:02:55.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.091+0000 7f053c4e1b80 1 -- 172.21.15.52:0/3362375871 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f052c1b5e80 con 0x55f83dedd590 2024-06-10T03:02:55.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.092+0000 7f053c4e1b80 1 -- 172.21.15.52:0/3362375871 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f052c1b5780 con 0x55f83ded7ce0 2024-06-10T03:02:55.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.092+0000 7f053c4e1b80 1 -- 172.21.15.52:0/3362375871 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f052c117e70 con 0x55f83deddeb0 2024-06-10T03:02:55.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.092+0000 7f053c4e1b80 10 client.0 _ll_register_callbacks cb 0x55f83ddbe740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T03:02:55.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.092+0000 7f052b7fe700 1 -- 172.21.15.52:0/3362375871 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f83deddeb0 msgr2=0x55f83df6c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:02:55.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.092+0000 7f052b7fe700 1 --2- 172.21.15.52:0/3362375871 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f83deddeb0 0x55f83df6c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:55.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.092+0000 7f052b7fe700 1 -- 172.21.15.52:0/3362375871 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f83dedd590 msgr2=0x55f83dee82d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:02:55.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.092+0000 7f052b7fe700 1 --2- 172.21.15.52:0/3362375871 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f83dedd590 0x55f83dee82d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:55.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.092+0000 7f052b7fe700 1 -- 172.21.15.52:0/3362375871 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f83df6c760 con 0x55f83ded7ce0 2024-06-10T03:02:55.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.093+0000 7f052b7fe700 1 --2- 172.21.15.52:0/3362375871 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f83ded7ce0 0x55f83dee7d90 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f051c003a50 tx=0x7f051c003ea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bcf8aed9e15feeb server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:02:55.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.093+0000 7f050affd700 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-06-10T03:02:55.094 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.094+0000 7f050affd700 1 -- 172.21.15.52:0/3362375871 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f051c029620 con 0x55f83ded7ce0 2024-06-10T03:02:55.094 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.094+0000 7f050affd700 1 -- 172.21.15.52:0/3362375871 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f051c020ab0 con 0x55f83ded7ce0 2024-06-10T03:02:55.095 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.094+0000 7f053c4e1b80 1 -- 172.21.15.52:0/3362375871 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f051c0200d0 con 0x55f83ded7ce0 2024-06-10T03:02:55.095 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.094+0000 7f050affd700 1 -- 172.21.15.52:0/3362375871 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f051c040050 con 0x55f83ded7ce0 2024-06-10T03:02:55.095 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.094+0000 7f053c4e1b80 10 client.26261 fetch_fsmap learned FSMap version 42 2024-06-10T03:02:55.095 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.094+0000 7f053c4e1b80 1 -- 172.21.15.52:0/3362375871 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55f83df6d910 con 0x55f83ded7ce0 2024-06-10T03:02:55.096 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.095+0000 7f050affd700 1 -- 172.21.15.52:0/3362375871 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f051c0200d0 con 0x55f83ded7ce0 2024-06-10T03:02:55.096 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.095+0000 7f050affd700 1 -- 172.21.15.52:0/3362375871 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 42) ==== 8394+0+0 (secure 0 0 0) 0x7f051c03b030 con 0x55f83ded7ce0 2024-06-10T03:02:55.096 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.095+0000 7f053c4e1b80 10 client.26261 fetch_fsmap finished waiting for FSMap version 42 2024-06-10T03:02:55.096 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.095+0000 7f053c4e1b80 10 client.26261 resolve_mds: validated gid 24484 aka daemon mds.i 2024-06-10T03:02:55.096 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.095+0000 7f053c4e1b80 20 client.26261 populate_metadata read hostname 'smithi052' 2024-06-10T03:02:55.096 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.095+0000 7f053c4e1b80 1 --2- 172.21.15.52:0/3362375871 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55f83df6dfe0 0x55f83df70400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:02:55.096 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.095+0000 7f053c4e1b80 4 client.26261 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 136.0, "await_for": 45.0}] 2024-06-10T03:02:55.096 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.095+0000 7f053c4e1b80 1 -- 172.21.15.52:0/3362375871 --> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 136.0, "await_for": 45.0}) -- 0x55f83de30460 con 0x55f83df6dfe0 2024-06-10T03:02:55.096 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.096+0000 7f052affd700 1 --2- 172.21.15.52:0/3362375871 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55f83df6dfe0 0x55f83df70400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:02:55.097 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.097+0000 7f050affd700 10 client.26261 ms_handle_connect on v2:172.21.15.188:6838/310444836 2024-06-10T03:02:55.097 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:55.097+0000 7f052affd700 1 --2- 172.21.15.52:0/3362375871 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55f83df6dfe0 0x55f83df70400 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4226440bb3e2d846 server_cookie=5d09aea2d92b4a26 in_seq=0 out_seq=0 2024-06-10T03:02:56.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:02:56 smithi188 ceph-mon[36599]: pgmap v696: 129 pgs: 129 active+clean; 298 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 450 op/s 2024-06-10T03:02:56.977 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:02:56 smithi052 ceph-mon[32362]: pgmap v696: 129 pgs: 129 active+clean; 298 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 450 op/s 2024-06-10T03:02:57.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:02:56 smithi120 ceph-mon[36634]: pgmap v696: 129 pgs: 129 active+clean; 298 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 450 op/s 2024-06-10T03:02:57.583 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.581+0000 7f050affd700 1 -- 172.21.15.52:0/3362375871 <== mds.0 v2:172.21.15.188:6838/310444836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55f83de30460 con 0x55f83df6dfe0 2024-06-10T03:02:57.583 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.581+0000 7f050affd700 10 client.26261 handle_command_reply: tid=0 2024-06-10T03:02:57.583 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.582+0000 7f053c4e1b80 1 client.26261 shutdown 2024-06-10T03:02:57.583 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.582+0000 7f053c4e1b80 1 -- 172.21.15.52:0/3362375871 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f83ded7ce0 msgr2=0x55f83dee7d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:02:57.583 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.582+0000 7f053c4e1b80 1 --2- 172.21.15.52:0/3362375871 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f83ded7ce0 0x55f83dee7d90 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f051c003a50 tx=0x7f051c003ea0 comp rx=0 tx=0).stop 2024-06-10T03:02:57.583 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.582+0000 7f053c4e1b80 1 -- 172.21.15.52:0/3362375871 shutdown_connections 2024-06-10T03:02:57.583 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.582+0000 7f053c4e1b80 1 --2- 172.21.15.52:0/3362375871 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55f83df6dfe0 0x55f83df70400 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:57.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.582+0000 7f053c4e1b80 1 --2- 172.21.15.52:0/3362375871 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f83deddeb0 0x55f83df6c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:57.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.582+0000 7f053c4e1b80 1 --2- 172.21.15.52:0/3362375871 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f83ded7ce0 0x55f83dee7d90 secure :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f051c003a50 tx=0x7f051c003ea0 comp rx=0 tx=0).stop 2024-06-10T03:02:57.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.582+0000 7f053c4e1b80 1 --2- 172.21.15.52:0/3362375871 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f83dedd590 0x55f83dee82d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:57.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.582+0000 7f053c4e1b80 1 -- 172.21.15.52:0/3362375871 >> 172.21.15.52:0/3362375871 conn(0x55f83de04210 msgr2=0x55f83dee2620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:02:57.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.582+0000 7f050affd700 0 client.26261 ms_handle_reset on v2:172.21.15.188:6838/310444836 2024-06-10T03:02:57.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.582+0000 7f053c4e1b80 1 -- 172.21.15.52:0/3362375871 shutdown_connections 2024-06-10T03:02:57.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.582+0000 7f053c4e1b80 1 -- 172.21.15.52:0/3362375871 wait complete. 2024-06-10T03:02:57.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.583+0000 7f053c4e1b80 20 client.26261 trim_cache size 0 max 16384 2024-06-10T03:02:57.584 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-10T03:02:57.584 INFO:teuthology.orchestra.run.smithi052.stdout: "epoch": 42, 2024-06-10T03:02:57.585 INFO:teuthology.orchestra.run.smithi052.stdout: "leader": 24484, 2024-06-10T03:02:57.585 INFO:teuthology.orchestra.run.smithi052.stdout: "set_version": 22, 2024-06-10T03:02:57.585 INFO:teuthology.orchestra.run.smithi052.stdout: "sets": { 2024-06-10T03:02:57.585 INFO:teuthology.orchestra.run.smithi052.stdout: "6a4162d1": { 2024-06-10T03:02:57.585 INFO:teuthology.orchestra.run.smithi052.stdout: "version": 22, 2024-06-10T03:02:57.585 INFO:teuthology.orchestra.run.smithi052.stdout: "age_ref": 0.0, 2024-06-10T03:02:57.585 INFO:teuthology.orchestra.run.smithi052.stdout: "state": { 2024-06-10T03:02:57.585 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "QUIESCED", 2024-06-10T03:02:57.585 INFO:teuthology.orchestra.run.smithi052.stdout: "age": 0.0 2024-06-10T03:02:57.585 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:02:57.585 INFO:teuthology.orchestra.run.smithi052.stdout: "timeout": 90.0, 2024-06-10T03:02:57.585 INFO:teuthology.orchestra.run.smithi052.stdout: "expiration": 136.0, 2024-06-10T03:02:57.586 INFO:teuthology.orchestra.run.smithi052.stdout: "members": { 2024-06-10T03:02:57.586 INFO:teuthology.orchestra.run.smithi052.stdout: "file:/": { 2024-06-10T03:02:57.586 INFO:teuthology.orchestra.run.smithi052.stdout: "excluded": false, 2024-06-10T03:02:57.586 INFO:teuthology.orchestra.run.smithi052.stdout: "state": { 2024-06-10T03:02:57.586 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "QUIESCED", 2024-06-10T03:02:57.586 INFO:teuthology.orchestra.run.smithi052.stdout: "age": 0.0 2024-06-10T03:02:57.586 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:02:57.586 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:02:57.586 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:02:57.586 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:02:57.586 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:02:57.586 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-10T03:02:57.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.583+0000 7f0531c5e700 1 -- 172.21.15.52:0/2030872684 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f05140a2eb0 msgr2=0x7f05140a5370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:02:57.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.583+0000 7f0531c5e700 1 --2- 172.21.15.52:0/2030872684 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f05140a2eb0 0x7f05140a5370 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f051c0045f0 tx=0x7f051c00b410 comp rx=0 tx=0).stop 2024-06-10T03:02:57.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.583+0000 7f0531c5e700 1 -- 172.21.15.52:0/2030872684 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f052c11bc70 msgr2=0x7f052c1a77c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:02:57.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.583+0000 7f0531c5e700 1 --2- 172.21.15.52:0/2030872684 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f052c11bc70 0x7f052c1a77c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f0524006070 tx=0x7f0524004d50 comp rx=0 tx=0).stop 2024-06-10T03:02:57.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.584+0000 7f0531c5e700 1 -- 172.21.15.52:0/2030872684 shutdown_connections 2024-06-10T03:02:57.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.584+0000 7f0531c5e700 1 --2- 172.21.15.52:0/2030872684 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f052c11bc70 0x7f052c1a77c0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:57.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.584+0000 7f0531c5e700 1 --2- 172.21.15.52:0/2030872684 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f05140a2eb0 0x7f05140a5370 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:57.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.584+0000 7f0531c5e700 1 --2- 172.21.15.52:0/2030872684 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f052c1099d0 0x7f052c1ad640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:57.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.584+0000 7f0531c5e700 1 --2- 172.21.15.52:0/2030872684 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f052c110ce0 0x7f052c1adb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:57.588 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.584+0000 7f0531c5e700 1 -- 172.21.15.52:0/2030872684 >> 172.21.15.52:0/2030872684 conn(0x7f052c06d600 msgr2=0x7f052c119c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:02:57.588 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.584+0000 7f0531c5e700 1 -- 172.21.15.52:0/2030872684 shutdown_connections 2024-06-10T03:02:57.588 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.584+0000 7f0531c5e700 1 -- 172.21.15.52:0/2030872684 wait complete. 2024-06-10T03:02:57.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.667+0000 7f4935b3a700 1 -- 172.21.15.52:0/1308373056 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f49301099d0 msgr2=0x7f4930109e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:02:57.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.667+0000 7f4935b3a700 1 --2- 172.21.15.52:0/1308373056 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f49301099d0 0x7f4930109e50 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f4924009990 tx=0x7f4924009ca0 comp rx=0 tx=0).stop 2024-06-10T03:02:57.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.667+0000 7f4935b3a700 1 -- 172.21.15.52:0/1308373056 shutdown_connections 2024-06-10T03:02:57.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.667+0000 7f4935b3a700 1 --2- 172.21.15.52:0/1308373056 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f493011bc70 0x7f493011e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:57.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.667+0000 7f4935b3a700 1 --2- 172.21.15.52:0/1308373056 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4930110c70 0x7f4930109490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:57.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.667+0000 7f4935b3a700 1 --2- 172.21.15.52:0/1308373056 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f49301099d0 0x7f4930109e50 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:57.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.667+0000 7f4935b3a700 1 -- 172.21.15.52:0/1308373056 >> 172.21.15.52:0/1308373056 conn(0x7f493006d600 msgr2=0x7f493006da10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:02:57.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.667+0000 7f4935b3a700 1 -- 172.21.15.52:0/1308373056 shutdown_connections 2024-06-10T03:02:57.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.667+0000 7f4935b3a700 1 -- 172.21.15.52:0/1308373056 wait complete. 2024-06-10T03:02:57.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.668+0000 7f4935b3a700 1 Processor -- start 2024-06-10T03:02:57.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.668+0000 7f4935b3a700 1 -- start start 2024-06-10T03:02:57.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.669+0000 7f4935b3a700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f49301099d0 0x7f493011a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:02:57.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.669+0000 7f4935b3a700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4930110c70 0x7f493011a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:02:57.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.669+0000 7f4935b3a700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f493011bc70 0x7f4930113390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:02:57.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.669+0000 7f4935b3a700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f49301217e0 con 0x7f49301099d0 2024-06-10T03:02:57.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.669+0000 7f4935b3a700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f4930121660 con 0x7f4930110c70 2024-06-10T03:02:57.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.669+0000 7f4935b3a700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f4930121960 con 0x7f493011bc70 2024-06-10T03:02:57.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.670+0000 7f492effd700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4930110c70 0x7f493011a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:02:57.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.670+0000 7f492effd700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4930110c70 0x7f493011a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.52:56708/0 (socket says 172.21.15.52:56708) 2024-06-10T03:02:57.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.670+0000 7f492effd700 1 -- 172.21.15.52:0/2188220574 learned_addr learned my addr 172.21.15.52:0/2188220574 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:02:57.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.670+0000 7f492f7fe700 1 --2- 172.21.15.52:0/2188220574 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f49301099d0 0x7f493011a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:02:57.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.671+0000 7f492effd700 1 -- 172.21.15.52:0/2188220574 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f493011bc70 msgr2=0x7f4930113390 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-06-10T03:02:57.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.671+0000 7f492effd700 1 --2- 172.21.15.52:0/2188220574 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f493011bc70 0x7f4930113390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:57.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.671+0000 7f492effd700 1 -- 172.21.15.52:0/2188220574 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f49301099d0 msgr2=0x7f493011a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:02:57.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.671+0000 7f492effd700 1 --2- 172.21.15.52:0/2188220574 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f49301099d0 0x7f493011a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:57.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.671+0000 7f492effd700 1 -- 172.21.15.52:0/2188220574 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4930113c90 con 0x7f4930110c70 2024-06-10T03:02:57.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.671+0000 7f492effd700 1 --2- 172.21.15.52:0/2188220574 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4930110c70 0x7f493011a660 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f49240057c0 tx=0x7f4924003730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:02:57.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.672+0000 7f492cff9700 1 -- 172.21.15.52:0/2188220574 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4924026070 con 0x7f4930110c70 2024-06-10T03:02:57.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.672+0000 7f492cff9700 1 -- 172.21.15.52:0/2188220574 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f49240041b0 con 0x7f4930110c70 2024-06-10T03:02:57.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.672+0000 7f492cff9700 1 -- 172.21.15.52:0/2188220574 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4924021410 con 0x7f4930110c70 2024-06-10T03:02:57.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.672+0000 7f4935b3a700 1 -- 172.21.15.52:0/2188220574 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4930113f20 con 0x7f4930110c70 2024-06-10T03:02:57.673 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.672+0000 7f4935b3a700 1 -- 172.21.15.52:0/2188220574 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4930117e20 con 0x7f4930110c70 2024-06-10T03:02:57.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.678+0000 7f49403bdb80 1 Processor -- start 2024-06-10T03:02:57.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.679+0000 7f49403bdb80 1 -- start start 2024-06-10T03:02:57.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.679+0000 7f49403bdb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55762612af60 0x55762612b340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:02:57.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.679+0000 7f49403bdb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55762612b880 0x557626125260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:02:57.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.679+0000 7f49403bdb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5576261257a0 0x557626125bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:02:57.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.679+0000 7f49403bdb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x557626017f60 con 0x55762612b880 2024-06-10T03:02:57.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.679+0000 7f49403bdb80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x557626076850 con 0x55762612af60 2024-06-10T03:02:57.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.679+0000 7f49403bdb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x557625fde460 con 0x5576261257a0 2024-06-10T03:02:57.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.679+0000 7f492cff9700 1 -- 172.21.15.52:0/2188220574 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f492400f5d0 con 0x7f4930110c70 2024-06-10T03:02:57.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.679+0000 7f492f7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55762612b880 0x557626125260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:02:57.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.679+0000 7f492f7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55762612b880 0x557626125260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58254/0 (socket says 172.21.15.52:58254) 2024-06-10T03:02:57.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.679+0000 7f492f7fe700 1 -- 172.21.15.52:0/2156272659 learned_addr learned my addr 172.21.15.52:0/2156272659 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:02:57.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.680+0000 7f492ffff700 1 --2- 172.21.15.52:0/2156272659 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55762612af60 0x55762612b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:02:57.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.680+0000 7f492effd700 1 --2- 172.21.15.52:0/2156272659 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5576261257a0 0x557626125bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:02:57.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.681+0000 7f492ffff700 1 -- 172.21.15.52:0/2156272659 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5576261257a0 msgr2=0x557626125bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:02:57.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.681+0000 7f492ffff700 1 --2- 172.21.15.52:0/2156272659 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5576261257a0 0x557626125bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:57.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.681+0000 7f492ffff700 1 -- 172.21.15.52:0/2156272659 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55762612b880 msgr2=0x557626125260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:02:57.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.681+0000 7f492ffff700 1 --2- 172.21.15.52:0/2156272659 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55762612b880 0x557626125260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:57.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.681+0000 7f492ffff700 1 -- 172.21.15.52:0/2156272659 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557625fc58d0 con 0x55762612af60 2024-06-10T03:02:57.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.681+0000 7f492cff9700 1 --2- 172.21.15.52:0/2188220574 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f49180a2da0 0x7f49180a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:02:57.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.681+0000 7f492cff9700 1 -- 172.21.15.52:0/2188220574 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f49240d0040 con 0x7f4930110c70 2024-06-10T03:02:57.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.682+0000 7f492f7fe700 1 --2- 172.21.15.52:0/2188220574 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f49180a2da0 0x7f49180a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:02:57.683 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.683+0000 7f492f7fe700 1 --2- 172.21.15.52:0/2188220574 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f49180a2da0 0x7f49180a5260 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f492800b600 tx=0x7f492800e040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:02:57.684 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.685+0000 7f492ffff700 1 --2- 172.21.15.52:0/2156272659 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55762612af60 0x55762612b340 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f4920009690 tx=0x7f49200099a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=483f9bf56f33062c server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:02:57.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.686+0000 7f4915ffb700 1 -- 172.21.15.52:0/2156272659 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49200107b0 con 0x55762612af60 2024-06-10T03:02:57.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.686+0000 7f4915ffb700 1 -- 172.21.15.52:0/2156272659 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4920016070 con 0x55762612af60 2024-06-10T03:02:57.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.686+0000 7f4915ffb700 1 -- 172.21.15.52:0/2156272659 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4920011e40 con 0x55762612af60 2024-06-10T03:02:57.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.688+0000 7f49403bdb80 1 -- 172.21.15.52:0/2156272659 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55762612af60 msgr2=0x55762612b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:02:57.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.688+0000 7f49403bdb80 1 --2- 172.21.15.52:0/2156272659 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55762612af60 0x55762612b340 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f4920009690 tx=0x7f49200099a0 comp rx=0 tx=0).stop 2024-06-10T03:02:57.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.688+0000 7f49403bdb80 1 -- 172.21.15.52:0/2156272659 shutdown_connections 2024-06-10T03:02:57.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.688+0000 7f49403bdb80 1 --2- 172.21.15.52:0/2156272659 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5576261257a0 0x557626125bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:57.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.688+0000 7f49403bdb80 1 --2- 172.21.15.52:0/2156272659 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55762612af60 0x55762612b340 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:57.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.688+0000 7f49403bdb80 1 --2- 172.21.15.52:0/2156272659 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55762612b880 0x557626125260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:57.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.688+0000 7f49403bdb80 1 -- 172.21.15.52:0/2156272659 >> 172.21.15.52:0/2156272659 conn(0x557626052080 msgr2=0x557626120220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:02:57.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.688+0000 7f49403bdb80 1 -- 172.21.15.52:0/2156272659 shutdown_connections 2024-06-10T03:02:57.689 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.688+0000 7f49403bdb80 1 -- 172.21.15.52:0/2156272659 wait complete. 2024-06-10T03:02:57.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.689+0000 7f49403bdb80 1 -- start start 2024-06-10T03:02:57.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.689+0000 7f49403bdb80 1 Processor -- start 2024-06-10T03:02:57.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.689+0000 7f49403bdb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5576261257a0 0x557626131700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:02:57.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.689+0000 7f49403bdb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55762612af60 0x55762612ea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:02:57.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.689+0000 7f49403bdb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55762612b880 0x55762612ef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:02:57.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.689+0000 7f49403bdb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x557626016fc0 con 0x55762612b880 2024-06-10T03:02:57.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.689+0000 7f49403bdb80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x557626072810 con 0x55762612af60 2024-06-10T03:02:57.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.689+0000 7f49403bdb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x557626132260 con 0x5576261257a0 2024-06-10T03:02:57.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.689+0000 7f49403bdb80 10 client.0 _ll_register_callbacks cb 0x55762600c5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T03:02:57.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.690+0000 7f492ffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55762612b880 0x55762612ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:02:57.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.690+0000 7f492effd700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55762612af60 0x55762612ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:02:57.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.690+0000 7f492ffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55762612b880 0x55762612ef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58258/0 (socket says 172.21.15.52:58258) 2024-06-10T03:02:57.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.690+0000 7f492ffff700 1 -- 172.21.15.52:0/2098564076 learned_addr learned my addr 172.21.15.52:0/2098564076 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:02:57.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.691+0000 7f492f7fe700 1 --2- 172.21.15.52:0/2098564076 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5576261257a0 0x557626131700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:02:57.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.691+0000 7f492ffff700 1 -- 172.21.15.52:0/2098564076 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5576261257a0 msgr2=0x557626131700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:02:57.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.691+0000 7f492ffff700 1 --2- 172.21.15.52:0/2098564076 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5576261257a0 0x557626131700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:57.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.691+0000 7f492ffff700 1 -- 172.21.15.52:0/2098564076 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55762612af60 msgr2=0x55762612ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:02:57.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.691+0000 7f492ffff700 1 --2- 172.21.15.52:0/2098564076 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55762612af60 0x55762612ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:57.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.691+0000 7f492ffff700 1 -- 172.21.15.52:0/2098564076 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55762603c750 con 0x55762612b880 2024-06-10T03:02:57.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.691+0000 7f492effd700 1 --2- 172.21.15.52:0/2098564076 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55762612af60 0x55762612ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T03:02:57.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.692+0000 7f492ffff700 1 --2- 172.21.15.52:0/2098564076 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55762612b880 0x55762612ef40 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f4920011c40 tx=0x7f4920011d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e62164eb9c4f6cc4 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:02:57.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.692+0000 7f490effd700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-10T03:02:57.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.693+0000 7f490effd700 1 -- 172.21.15.52:0/2098564076 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4920010a40 con 0x55762612b880 2024-06-10T03:02:57.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.693+0000 7f49403bdb80 1 -- 172.21.15.52:0/2098564076 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4920016070 con 0x55762612b880 2024-06-10T03:02:57.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.693+0000 7f490effd700 1 -- 172.21.15.52:0/2098564076 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f4920010be0 con 0x55762612b880 2024-06-10T03:02:57.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.694+0000 7f490effd700 1 -- 172.21.15.52:0/2098564076 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f492002d050 con 0x55762612b880 2024-06-10T03:02:57.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.694+0000 7f49403bdb80 10 client.16773 fetch_fsmap learned FSMap version 42 2024-06-10T03:02:57.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.694+0000 7f49403bdb80 1 -- 172.21.15.52:0/2098564076 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55762613cb20 con 0x55762612b880 2024-06-10T03:02:57.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.695+0000 7f490effd700 1 -- 172.21.15.52:0/2098564076 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4920016070 con 0x55762612b880 2024-06-10T03:02:57.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.695+0000 7f490effd700 1 -- 172.21.15.52:0/2098564076 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 42) ==== 8394+0+0 (secure 0 0 0) 0x7f4920019380 con 0x55762612b880 2024-06-10T03:02:57.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.695+0000 7f49403bdb80 10 client.16773 fetch_fsmap finished waiting for FSMap version 42 2024-06-10T03:02:57.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.695+0000 7f49403bdb80 10 client.16773 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T03:02:57.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.695+0000 7f49403bdb80 20 client.16773 populate_metadata read hostname 'smithi052' 2024-06-10T03:02:57.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.695+0000 7f49403bdb80 1 --2- 172.21.15.52:0/2098564076 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55762613f490 0x5576261418b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:02:57.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.695+0000 7f49403bdb80 4 client.16773 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T03:02:57.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.695+0000 7f49403bdb80 1 -- 172.21.15.52:0/2098564076 --> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557626083d50 con 0x55762613f490 2024-06-10T03:02:57.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.696+0000 7f492f7fe700 1 --2- 172.21.15.52:0/2098564076 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55762613f490 0x5576261418b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:02:57.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.699+0000 7f492f7fe700 1 --2- 172.21.15.52:0/2098564076 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55762613f490 0x5576261418b0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=601a5365404a7504 server_cookie=9ccbaa4f0b081436 in_seq=0 out_seq=0 2024-06-10T03:02:57.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.699+0000 7f490effd700 10 client.16773 ms_handle_connect on v2:172.21.15.188:6838/310444836 2024-06-10T03:02:57.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.700+0000 7f490effd700 1 -- 172.21.15.52:0/2098564076 <== mds.0 v2:172.21.15.188:6838/310444836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x557626083d50 con 0x55762613f490 2024-06-10T03:02:57.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.700+0000 7f490effd700 10 client.16773 handle_command_reply: tid=0 2024-06-10T03:02:57.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.700+0000 7f49403bdb80 1 client.16773 shutdown 2024-06-10T03:02:57.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.701+0000 7f49403bdb80 1 -- 172.21.15.52:0/2098564076 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55762612b880 msgr2=0x55762612ef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:02:57.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.701+0000 7f49403bdb80 1 --2- 172.21.15.52:0/2098564076 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55762612b880 0x55762612ef40 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f4920011c40 tx=0x7f4920011d20 comp rx=0 tx=0).stop 2024-06-10T03:02:57.702 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.702+0000 7f49403bdb80 1 -- 172.21.15.52:0/2098564076 shutdown_connections 2024-06-10T03:02:57.702 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.702+0000 7f49403bdb80 1 --2- 172.21.15.52:0/2098564076 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55762613f490 0x5576261418b0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:57.702 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.702+0000 7f49403bdb80 1 --2- 172.21.15.52:0/2098564076 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55762612b880 0x55762612ef40 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:57.702 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.702+0000 7f49403bdb80 1 --2- 172.21.15.52:0/2098564076 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5576261257a0 0x557626131700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:57.702 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.702+0000 7f49403bdb80 1 --2- 172.21.15.52:0/2098564076 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55762612af60 0x55762612ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:57.703 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.702+0000 7f49403bdb80 1 -- 172.21.15.52:0/2098564076 >> 172.21.15.52:0/2098564076 conn(0x557626052080 msgr2=0x55762611f700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:02:57.703 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.702+0000 7f490effd700 0 client.16773 ms_handle_reset on v2:172.21.15.188:6838/310444836 2024-06-10T03:02:57.703 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.702+0000 7f49403bdb80 1 -- 172.21.15.52:0/2098564076 shutdown_connections 2024-06-10T03:02:57.703 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.702+0000 7f49403bdb80 1 -- 172.21.15.52:0/2098564076 wait complete. 2024-06-10T03:02:57.703 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.703+0000 7f49403bdb80 20 client.16773 trim_cache size 0 max 16384 2024-06-10T03:02:57.734 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.734+0000 7f49403bdb80 1 Processor -- start 2024-06-10T03:02:57.735 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.734+0000 7f49403bdb80 1 -- start start 2024-06-10T03:02:57.735 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.735+0000 7f49403bdb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557626052080 0x55762613b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:02:57.735 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.735+0000 7f49403bdb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5576261257a0 0x55762612d980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:02:57.735 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.735+0000 7f49403bdb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55762612af60 0x557626131240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:02:57.736 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.735+0000 7f49403bdb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f49301c5a00 con 0x557626052080 2024-06-10T03:02:57.736 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.735+0000 7f49403bdb80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f49301c7690 con 0x55762612af60 2024-06-10T03:02:57.736 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.735+0000 7f49403bdb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f49301c6130 con 0x5576261257a0 2024-06-10T03:02:57.736 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.735+0000 7f492ffff700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55762612af60 0x557626131240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:02:57.736 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.735+0000 7f492f7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557626052080 0x55762613b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:02:57.736 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.735+0000 7f492ffff700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55762612af60 0x557626131240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.52:56728/0 (socket says 172.21.15.52:56728) 2024-06-10T03:02:57.736 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.735+0000 7f492ffff700 1 -- 172.21.15.52:0/2375842996 learned_addr learned my addr 172.21.15.52:0/2375842996 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:02:57.736 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.735+0000 7f492effd700 1 --2- 172.21.15.52:0/2375842996 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5576261257a0 0x55762612d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:02:57.736 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.736+0000 7f492f7fe700 1 -- 172.21.15.52:0/2375842996 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5576261257a0 msgr2=0x55762612d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:02:57.737 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.736+0000 7f492f7fe700 1 --2- 172.21.15.52:0/2375842996 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5576261257a0 0x55762612d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:57.737 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.736+0000 7f492f7fe700 1 -- 172.21.15.52:0/2375842996 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55762612af60 msgr2=0x557626131240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:02:57.737 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.736+0000 7f492f7fe700 1 --2- 172.21.15.52:0/2375842996 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55762612af60 0x557626131240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:57.737 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.736+0000 7f492f7fe700 1 -- 172.21.15.52:0/2375842996 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55762611c810 con 0x557626052080 2024-06-10T03:02:57.737 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.736+0000 7f492f7fe700 1 --2- 172.21.15.52:0/2375842996 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557626052080 0x55762613b3b0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x557626142100 tx=0x7f4928013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b0f339899cc92a3 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:02:57.737 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.737+0000 7f49157fa700 1 -- 172.21.15.52:0/2375842996 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4928031070 con 0x557626052080 2024-06-10T03:02:57.737 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.737+0000 7f49157fa700 1 -- 172.21.15.52:0/2375842996 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4928018c40 con 0x557626052080 2024-06-10T03:02:57.737 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.737+0000 7f49157fa700 1 -- 172.21.15.52:0/2375842996 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f492802c650 con 0x557626052080 2024-06-10T03:02:57.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.738+0000 7f49403bdb80 1 -- 172.21.15.52:0/2375842996 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557626052080 msgr2=0x55762613b3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:02:57.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.738+0000 7f49403bdb80 1 --2- 172.21.15.52:0/2375842996 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557626052080 0x55762613b3b0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x557626142100 tx=0x7f4928013000 comp rx=0 tx=0).stop 2024-06-10T03:02:57.741 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.740+0000 7f49403bdb80 1 -- 172.21.15.52:0/2375842996 shutdown_connections 2024-06-10T03:02:57.741 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.740+0000 7f49403bdb80 1 --2- 172.21.15.52:0/2375842996 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55762612af60 0x557626131240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:57.741 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.740+0000 7f49403bdb80 1 --2- 172.21.15.52:0/2375842996 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557626052080 0x55762613b3b0 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:57.741 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.740+0000 7f49403bdb80 1 --2- 172.21.15.52:0/2375842996 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5576261257a0 0x55762612d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:57.741 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.740+0000 7f49403bdb80 1 -- 172.21.15.52:0/2375842996 >> 172.21.15.52:0/2375842996 conn(0x55762612c7f0 msgr2=0x55762612a360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:02:57.741 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.740+0000 7f49403bdb80 1 -- 172.21.15.52:0/2375842996 shutdown_connections 2024-06-10T03:02:57.741 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.740+0000 7f49403bdb80 1 -- 172.21.15.52:0/2375842996 wait complete. 2024-06-10T03:02:57.741 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.741+0000 7f49403bdb80 1 -- start start 2024-06-10T03:02:57.741 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.741+0000 7f49403bdb80 1 Processor -- start 2024-06-10T03:02:57.744 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.742+0000 7f49403bdb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557626052080 0x557626147910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:02:57.744 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.742+0000 7f49403bdb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5576261257a0 0x557626137520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:02:57.744 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.742+0000 7f49403bdb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55762612af60 0x557626137a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:02:57.744 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.742+0000 7f492f7fe700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557626052080 0x557626147910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:02:57.744 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.742+0000 7f492f7fe700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557626052080 0x557626147910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.52:36326/0 (socket says 172.21.15.52:36326) 2024-06-10T03:02:57.744 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.742+0000 7f492effd700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5576261257a0 0x557626137520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:02:57.745 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.742+0000 7f492effd700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5576261257a0 0x557626137520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.52:56742/0 (socket says 172.21.15.52:56742) 2024-06-10T03:02:57.745 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.742+0000 7f492f7fe700 1 -- 172.21.15.52:0/452210820 learned_addr learned my addr 172.21.15.52:0/452210820 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:02:57.745 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.742+0000 7f49403bdb80 1 -- 172.21.15.52:0/452210820 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f49301c6f90 con 0x55762612af60 2024-06-10T03:02:57.745 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.742+0000 7f49403bdb80 1 -- 172.21.15.52:0/452210820 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f49301c6890 con 0x5576261257a0 2024-06-10T03:02:57.745 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.742+0000 7f49403bdb80 1 -- 172.21.15.52:0/452210820 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f49301c39a0 con 0x557626052080 2024-06-10T03:02:57.745 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.742+0000 7f49403bdb80 10 client.0 _ll_register_callbacks cb 0x55762600c5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T03:02:57.745 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.742+0000 7f492ffff700 1 --2- 172.21.15.52:0/452210820 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55762612af60 0x557626137a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:02:57.745 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.743+0000 7f492f7fe700 1 -- 172.21.15.52:0/452210820 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5576261257a0 msgr2=0x557626137520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:02:57.745 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.743+0000 7f492f7fe700 1 --2- 172.21.15.52:0/452210820 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5576261257a0 0x557626137520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:57.745 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.743+0000 7f492f7fe700 1 -- 172.21.15.52:0/452210820 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55762612af60 msgr2=0x557626137a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:02:57.745 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.743+0000 7f492f7fe700 1 --2- 172.21.15.52:0/452210820 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55762612af60 0x557626137a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:57.745 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.743+0000 7f492f7fe700 1 -- 172.21.15.52:0/452210820 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5576261461b0 con 0x557626052080 2024-06-10T03:02:57.745 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.743+0000 7f492f7fe700 1 --2- 172.21.15.52:0/452210820 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557626052080 0x557626147910 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f4928000f80 tx=0x7f49280098a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5b56cd295214eec9 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:02:57.745 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.743+0000 7f490effd700 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-06-10T03:02:57.746 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.743+0000 7f490effd700 1 -- 172.21.15.52:0/452210820 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4928031070 con 0x557626052080 2024-06-10T03:02:57.746 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.743+0000 7f49403bdb80 1 -- 172.21.15.52:0/452210820 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4928018c40 con 0x557626052080 2024-06-10T03:02:57.746 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.743+0000 7f490effd700 1 -- 172.21.15.52:0/452210820 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f4928024830 con 0x557626052080 2024-06-10T03:02:57.746 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.744+0000 7f490effd700 1 -- 172.21.15.52:0/452210820 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f4928042050 con 0x557626052080 2024-06-10T03:02:57.746 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.744+0000 7f49403bdb80 10 client.26320 fetch_fsmap learned FSMap version 42 2024-06-10T03:02:57.746 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.744+0000 7f49403bdb80 1 -- 172.21.15.52:0/452210820 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x5576261a93a0 con 0x557626052080 2024-06-10T03:02:57.747 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.745+0000 7f490effd700 1 -- 172.21.15.52:0/452210820 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4928018c40 con 0x557626052080 2024-06-10T03:02:57.747 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.745+0000 7f490effd700 1 -- 172.21.15.52:0/452210820 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 42) ==== 8394+0+0 (secure 0 0 0) 0x7f492802b030 con 0x557626052080 2024-06-10T03:02:57.747 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.745+0000 7f49403bdb80 10 client.26320 fetch_fsmap finished waiting for FSMap version 42 2024-06-10T03:02:57.747 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.745+0000 7f49403bdb80 10 client.26320 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T03:02:57.747 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.745+0000 7f49403bdb80 20 client.26320 populate_metadata read hostname 'smithi052' 2024-06-10T03:02:57.747 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.745+0000 7f49403bdb80 1 --2- 172.21.15.52:0/452210820 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55762612b880 0x5576261abaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:02:57.747 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.745+0000 7f49403bdb80 4 client.26320 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T03:02:57.747 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.745+0000 7f49403bdb80 1 -- 172.21.15.52:0/452210820 --> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55762607e2d0 con 0x55762612b880 2024-06-10T03:02:57.747 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.745+0000 7f492effd700 1 --2- 172.21.15.52:0/452210820 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55762612b880 0x5576261abaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:02:57.748 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.747+0000 7f492effd700 1 --2- 172.21.15.52:0/452210820 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55762612b880 0x5576261abaa0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=93577d04dc461520 server_cookie=a6860c913b308cea in_seq=0 out_seq=0 2024-06-10T03:02:57.748 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.747+0000 7f490effd700 10 client.26320 ms_handle_connect on v2:172.21.15.188:6838/310444836 2024-06-10T03:02:57.749 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.747+0000 7f490effd700 1 -- 172.21.15.52:0/452210820 <== mds.0 v2:172.21.15.188:6838/310444836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55762607e2d0 con 0x55762612b880 2024-06-10T03:02:57.749 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.747+0000 7f490effd700 10 client.26320 handle_command_reply: tid=0 2024-06-10T03:02:57.749 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.747+0000 7f49403bdb80 1 client.26320 shutdown 2024-06-10T03:02:57.749 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.747+0000 7f49403bdb80 1 -- 172.21.15.52:0/452210820 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557626052080 msgr2=0x557626147910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:02:57.749 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.747+0000 7f49403bdb80 1 --2- 172.21.15.52:0/452210820 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557626052080 0x557626147910 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f4928000f80 tx=0x7f49280098a0 comp rx=0 tx=0).stop 2024-06-10T03:02:57.749 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.748+0000 7f49403bdb80 1 -- 172.21.15.52:0/452210820 shutdown_connections 2024-06-10T03:02:57.749 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.748+0000 7f49403bdb80 1 --2- 172.21.15.52:0/452210820 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55762612b880 0x5576261abaa0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:57.749 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.748+0000 7f49403bdb80 1 --2- 172.21.15.52:0/452210820 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55762612af60 0x557626137a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:57.749 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.748+0000 7f49403bdb80 1 --2- 172.21.15.52:0/452210820 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557626052080 0x557626147910 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:57.749 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.748+0000 7f49403bdb80 1 --2- 172.21.15.52:0/452210820 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5576261257a0 0x557626137520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:57.749 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.748+0000 7f49403bdb80 1 -- 172.21.15.52:0/452210820 >> 172.21.15.52:0/452210820 conn(0x55762612c7f0 msgr2=0x557626124250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:02:57.749 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.748+0000 7f490effd700 0 client.26320 ms_handle_reset on v2:172.21.15.188:6838/310444836 2024-06-10T03:02:57.750 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.749+0000 7f49403bdb80 1 -- 172.21.15.52:0/452210820 shutdown_connections 2024-06-10T03:02:57.750 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.749+0000 7f49403bdb80 1 -- 172.21.15.52:0/452210820 wait complete. 2024-06-10T03:02:57.750 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.750+0000 7f49403bdb80 20 client.26320 trim_cache size 0 max 16384 2024-06-10T03:02:57.750 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-06-10T03:02:57.751 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.751+0000 7f4935b3a700 1 -- 172.21.15.52:0/2188220574 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f49180a2da0 msgr2=0x7f49180a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:02:57.751 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.751+0000 7f4935b3a700 1 --2- 172.21.15.52:0/2188220574 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f49180a2da0 0x7f49180a5260 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f492800b600 tx=0x7f492800e040 comp rx=0 tx=0).stop 2024-06-10T03:02:57.751 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.751+0000 7f4935b3a700 1 -- 172.21.15.52:0/2188220574 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4930110c70 msgr2=0x7f493011a660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:02:57.751 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.751+0000 7f4935b3a700 1 --2- 172.21.15.52:0/2188220574 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4930110c70 0x7f493011a660 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f49240057c0 tx=0x7f4924003730 comp rx=0 tx=0).stop 2024-06-10T03:02:57.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.751+0000 7f4935b3a700 1 -- 172.21.15.52:0/2188220574 shutdown_connections 2024-06-10T03:02:57.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.751+0000 7f4935b3a700 1 --2- 172.21.15.52:0/2188220574 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f493011bc70 0x7f4930113390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:57.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.751+0000 7f4935b3a700 1 --2- 172.21.15.52:0/2188220574 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f49180a2da0 0x7f49180a5260 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:57.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.751+0000 7f4935b3a700 1 --2- 172.21.15.52:0/2188220574 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f49301099d0 0x7f493011a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:57.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.751+0000 7f4935b3a700 1 --2- 172.21.15.52:0/2188220574 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4930110c70 0x7f493011a660 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:02:57.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.751+0000 7f4935b3a700 1 -- 172.21.15.52:0/2188220574 >> 172.21.15.52:0/2188220574 conn(0x7f493006d600 msgr2=0x7f4930071030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:02:57.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.752+0000 7f4935b3a700 1 -- 172.21.15.52:0/2188220574 shutdown_connections 2024-06-10T03:02:57.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:02:57.752+0000 7f4935b3a700 1 -- 172.21.15.52:0/2188220574 wait complete. 2024-06-10T03:02:58.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:02:58 smithi188 ceph-mon[36599]: pgmap v697: 129 pgs: 129 active+clean; 285 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 492 op/s 2024-06-10T03:02:58.977 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:02:58 smithi052 ceph-mon[32362]: pgmap v697: 129 pgs: 129 active+clean; 285 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 492 op/s 2024-06-10T03:02:59.017 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '6a4162d1', quiesce took 15.1 seconds. Will release after: 0.9000000000000004 2024-06-10T03:02:59.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:02:58 smithi120 ceph-mon[36634]: pgmap v697: 129 pgs: 129 active+clean; 285 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 492 op/s 2024-06-10T03:02:59.563 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-06-10T03:02:59.563 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-06-10T03:02:59.830 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T03:02:59.918 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T03:03:00.934 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:03:00 smithi052 ceph-mon[32362]: pgmap v698: 129 pgs: 129 active+clean; 279 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 490 op/s 2024-06-10T03:03:00.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:03:00 smithi188 ceph-mon[36599]: pgmap v698: 129 pgs: 129 active+clean; 279 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 490 op/s 2024-06-10T03:03:00.984 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T03:03:01.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:03:00 smithi120 ceph-mon[36634]: pgmap v698: 129 pgs: 129 active+clean; 279 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 490 op/s 2024-06-10T03:03:02.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.148+0000 7fa6067dc700 1 -- 172.21.15.52:0/336024622 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa600102740 msgr2=0x7fa600102b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:03:02.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.148+0000 7fa6067dc700 1 --2- 172.21.15.52:0/336024622 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa600102740 0x7fa600102b20 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fa5f00099c0 tx=0x7fa5f0009cd0 comp rx=0 tx=0).stop 2024-06-10T03:03:02.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.149+0000 7fa6067dc700 1 -- 172.21.15.52:0/336024622 shutdown_connections 2024-06-10T03:03:02.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.149+0000 7fa6067dc700 1 --2- 172.21.15.52:0/336024622 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa600107840 0x7fa60010f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:02.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.149+0000 7fa6067dc700 1 --2- 172.21.15.52:0/336024622 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa600102740 0x7fa600102b20 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:02.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.149+0000 7fa6067dc700 1 --2- 172.21.15.52:0/336024622 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa6001030f0 0x7fa600107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:02.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.149+0000 7fa6067dc700 1 -- 172.21.15.52:0/336024622 >> 172.21.15.52:0/336024622 conn(0x7fa6000795f0 msgr2=0x7fa600079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:03:02.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.149+0000 7fa6067dc700 1 -- 172.21.15.52:0/336024622 shutdown_connections 2024-06-10T03:03:02.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.149+0000 7fa6067dc700 1 -- 172.21.15.52:0/336024622 wait complete. 2024-06-10T03:03:02.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.149+0000 7fa6067dc700 1 Processor -- start 2024-06-10T03:03:02.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.149+0000 7fa6067dc700 1 -- start start 2024-06-10T03:03:02.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.149+0000 7fa6067dc700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa600102740 0x7fa600074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:02.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.149+0000 7fa6067dc700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa6001030f0 0x7fa600074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:02.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.149+0000 7fa6067dc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa600107840 0x7fa600075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:02.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.149+0000 7fa6067dc700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fa600112780 con 0x7fa600107840 2024-06-10T03:03:02.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.149+0000 7fa6067dc700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa600112600 con 0x7fa6001030f0 2024-06-10T03:03:02.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.149+0000 7fa6067dc700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa600112900 con 0x7fa600102740 2024-06-10T03:03:02.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.150+0000 7fa604d79700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa600107840 0x7fa600075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:02.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.150+0000 7fa604d79700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa600107840 0x7fa600075210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49328/0 (socket says 172.21.15.52:49328) 2024-06-10T03:03:02.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.150+0000 7fa604d79700 1 -- 172.21.15.52:0/1084389659 learned_addr learned my addr 172.21.15.52:0/1084389659 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:03:02.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.150+0000 7fa5fffff700 1 --2- 172.21.15.52:0/1084389659 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa600102740 0x7fa600074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:02.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.150+0000 7fa5f7fff700 1 --2- 172.21.15.52:0/1084389659 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa6001030f0 0x7fa600074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:02.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.150+0000 7fa604d79700 1 -- 172.21.15.52:0/1084389659 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa600102740 msgr2=0x7fa600074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:03:02.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.150+0000 7fa604d79700 1 --2- 172.21.15.52:0/1084389659 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa600102740 0x7fa600074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:02.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.150+0000 7fa604d79700 1 -- 172.21.15.52:0/1084389659 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa6001030f0 msgr2=0x7fa600074cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:03:02.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.150+0000 7fa604d79700 1 --2- 172.21.15.52:0/1084389659 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa6001030f0 0x7fa600074cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:02.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.150+0000 7fa604d79700 1 -- 172.21.15.52:0/1084389659 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6001a7110 con 0x7fa600107840 2024-06-10T03:03:02.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.150+0000 7fa604d79700 1 --2- 172.21.15.52:0/1084389659 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa600107840 0x7fa600075210 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fa5f8007ee0 tx=0x7fa5f800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:03:02.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.150+0000 7fa5fdffb700 1 -- 172.21.15.52:0/1084389659 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5f8004bb0 con 0x7fa600107840 2024-06-10T03:03:02.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.150+0000 7fa5fdffb700 1 -- 172.21.15.52:0/1084389659 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5f8004d50 con 0x7fa600107840 2024-06-10T03:03:02.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.150+0000 7fa6067dc700 1 -- 172.21.15.52:0/1084389659 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6001a7400 con 0x7fa600107840 2024-06-10T03:03:02.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.150+0000 7fa5fdffb700 1 -- 172.21.15.52:0/1084389659 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5f8005960 con 0x7fa600107840 2024-06-10T03:03:02.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.150+0000 7fa6067dc700 1 -- 172.21.15.52:0/1084389659 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6001a78f0 con 0x7fa600107840 2024-06-10T03:03:02.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.153+0000 7fa5fdffb700 1 -- 172.21.15.52:0/1084389659 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fa5f801a020 con 0x7fa600107840 2024-06-10T03:03:02.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.154+0000 7fa5fdffb700 1 --2- 172.21.15.52:0/1084389659 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fa5ec0a2c20 0x7fa5ec0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:02.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.154+0000 7fa5fdffb700 1 -- 172.21.15.52:0/1084389659 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7fa5f80ceac0 con 0x7fa600107840 2024-06-10T03:03:02.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.154+0000 7fa5fffff700 1 --2- 172.21.15.52:0/1084389659 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fa5ec0a2c20 0x7fa5ec0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:02.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.154+0000 7fa5fffff700 1 --2- 172.21.15.52:0/1084389659 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fa5ec0a2c20 0x7fa5ec0a50e0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fa5f0000c00 tx=0x7fa5f0005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:03:02.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.154+0000 7fa61105fb80 1 Processor -- start 2024-06-10T03:03:02.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.154+0000 7fa61105fb80 1 -- start start 2024-06-10T03:03:02.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.155+0000 7fa61105fb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560a3f49bfe0 0x560a3f49c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:02.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.155+0000 7fa61105fb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560a3f49c900 0x560a3f4962e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:02.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.155+0000 7fa61105fb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560a3f496820 0x560a3f496c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:02.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.155+0000 7fa61105fb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x560a3f388fe0 con 0x560a3f49c900 2024-06-10T03:03:02.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.155+0000 7fa61105fb80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x560a3f3e78d0 con 0x560a3f49bfe0 2024-06-10T03:03:02.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.155+0000 7fa61105fb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x560a3f34f4e0 con 0x560a3f496820 2024-06-10T03:03:02.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.155+0000 7fa5f7fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560a3f49c900 0x560a3f4962e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:02.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.155+0000 7fa5f7fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560a3f49c900 0x560a3f4962e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49340/0 (socket says 172.21.15.52:49340) 2024-06-10T03:03:02.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.155+0000 7fa5f7fff700 1 -- 172.21.15.52:0/2616991879 learned_addr learned my addr 172.21.15.52:0/2616991879 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:03:02.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.155+0000 7fa5fffff700 1 --2- 172.21.15.52:0/2616991879 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560a3f49bfe0 0x560a3f49c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:02.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.155+0000 7fa604d79700 1 --2- 172.21.15.52:0/2616991879 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560a3f496820 0x560a3f496c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:02.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.155+0000 7fa5f7fff700 1 -- 172.21.15.52:0/2616991879 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560a3f496820 msgr2=0x560a3f496c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:03:02.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.155+0000 7fa5f7fff700 1 --2- 172.21.15.52:0/2616991879 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560a3f496820 0x560a3f496c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:02.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.155+0000 7fa5f7fff700 1 -- 172.21.15.52:0/2616991879 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560a3f49bfe0 msgr2=0x560a3f49c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:03:02.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.155+0000 7fa5f7fff700 1 --2- 172.21.15.52:0/2616991879 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560a3f49bfe0 0x560a3f49c3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:02.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.155+0000 7fa5f7fff700 1 -- 172.21.15.52:0/2616991879 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a3f336950 con 0x560a3f49c900 2024-06-10T03:03:02.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.155+0000 7fa604d79700 1 --2- 172.21.15.52:0/2616991879 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560a3f496820 0x560a3f496c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T03:03:02.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.155+0000 7fa5f7fff700 1 --2- 172.21.15.52:0/2616991879 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560a3f49c900 0x560a3f4962e0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fa5e800ac40 tx=0x7fa5e800af50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbaa21484e388625 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:03:02.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.155+0000 7fa5f67fc700 1 -- 172.21.15.52:0/2616991879 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5e801d070 con 0x560a3f49c900 2024-06-10T03:03:02.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.155+0000 7fa5f67fc700 1 -- 172.21.15.52:0/2616991879 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5e800fe90 con 0x560a3f49c900 2024-06-10T03:03:02.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.156+0000 7fa5f67fc700 1 -- 172.21.15.52:0/2616991879 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5e8010a00 con 0x560a3f49c900 2024-06-10T03:03:02.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.156+0000 7fa61105fb80 1 -- 172.21.15.52:0/2616991879 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560a3f49c900 msgr2=0x560a3f4962e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:03:02.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.156+0000 7fa61105fb80 1 --2- 172.21.15.52:0/2616991879 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560a3f49c900 0x560a3f4962e0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fa5e800ac40 tx=0x7fa5e800af50 comp rx=0 tx=0).stop 2024-06-10T03:03:02.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.156+0000 7fa61105fb80 1 -- 172.21.15.52:0/2616991879 shutdown_connections 2024-06-10T03:03:02.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.156+0000 7fa61105fb80 1 --2- 172.21.15.52:0/2616991879 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560a3f496820 0x560a3f496c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:02.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.156+0000 7fa61105fb80 1 --2- 172.21.15.52:0/2616991879 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560a3f49bfe0 0x560a3f49c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:02.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.156+0000 7fa61105fb80 1 --2- 172.21.15.52:0/2616991879 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560a3f49c900 0x560a3f4962e0 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:02.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.156+0000 7fa61105fb80 1 -- 172.21.15.52:0/2616991879 >> 172.21.15.52:0/2616991879 conn(0x560a3f3c3100 msgr2=0x560a3f4912a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:03:02.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.156+0000 7fa61105fb80 1 -- 172.21.15.52:0/2616991879 shutdown_connections 2024-06-10T03:03:02.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.156+0000 7fa61105fb80 1 -- 172.21.15.52:0/2616991879 wait complete. 2024-06-10T03:03:02.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.157+0000 7fa61105fb80 1 -- start start 2024-06-10T03:03:02.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.157+0000 7fa61105fb80 1 Processor -- start 2024-06-10T03:03:02.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.157+0000 7fa61105fb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560a3f496820 0x560a3f4a26c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:02.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.157+0000 7fa61105fb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560a3f49bfe0 0x560a3f49fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:02.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.157+0000 7fa61105fb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560a3f49c900 0x560a3f49ff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:02.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.157+0000 7fa61105fb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x560a3f388040 con 0x560a3f49bfe0 2024-06-10T03:03:02.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.157+0000 7fa61105fb80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x560a3f3e3890 con 0x560a3f49c900 2024-06-10T03:03:02.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.157+0000 7fa61105fb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x560a3f4a32e0 con 0x560a3f496820 2024-06-10T03:03:02.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.157+0000 7fa61105fb80 10 client.0 _ll_register_callbacks cb 0x560a3f37d630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T03:03:02.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.157+0000 7fa5f7fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560a3f49bfe0 0x560a3f49fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:02.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.157+0000 7fa5f7fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560a3f49bfe0 0x560a3f49fa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49356/0 (socket says 172.21.15.52:49356) 2024-06-10T03:03:02.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.157+0000 7fa5f7fff700 1 -- 172.21.15.52:0/1388916076 learned_addr learned my addr 172.21.15.52:0/1388916076 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:03:02.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.157+0000 7fa5fffff700 1 --2- 172.21.15.52:0/1388916076 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560a3f496820 0x560a3f4a26c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:02.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.158+0000 7fa604d79700 1 --2- 172.21.15.52:0/1388916076 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560a3f49c900 0x560a3f49ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:02.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.158+0000 7fa5f7fff700 1 -- 172.21.15.52:0/1388916076 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560a3f496820 msgr2=0x560a3f4a26c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:03:02.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.158+0000 7fa5f7fff700 1 --2- 172.21.15.52:0/1388916076 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560a3f496820 0x560a3f4a26c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:02.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.158+0000 7fa5f7fff700 1 -- 172.21.15.52:0/1388916076 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560a3f49c900 msgr2=0x560a3f49ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:03:02.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.158+0000 7fa5f7fff700 1 --2- 172.21.15.52:0/1388916076 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560a3f49c900 0x560a3f49ff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:02.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.158+0000 7fa5f7fff700 1 -- 172.21.15.52:0/1388916076 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a3f3ad7d0 con 0x560a3f49bfe0 2024-06-10T03:03:02.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.158+0000 7fa5f7fff700 1 --2- 172.21.15.52:0/1388916076 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560a3f49bfe0 0x560a3f49fa10 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fa5e800a630 tx=0x7fa5e8018b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9cd55c62842df31 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:03:02.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.158+0000 7fa5df7fe700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-10T03:03:02.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.158+0000 7fa5df7fe700 1 -- 172.21.15.52:0/1388916076 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5e8025020 con 0x560a3f49bfe0 2024-06-10T03:03:02.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.158+0000 7fa5df7fe700 1 -- 172.21.15.52:0/1388916076 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7fa5e8005050 con 0x560a3f49bfe0 2024-06-10T03:03:02.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.158+0000 7fa61105fb80 1 -- 172.21.15.52:0/1388916076 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa5e800fe90 con 0x560a3f49bfe0 2024-06-10T03:03:02.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.158+0000 7fa5df7fe700 1 -- 172.21.15.52:0/1388916076 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fa5e8030050 con 0x560a3f49bfe0 2024-06-10T03:03:02.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.158+0000 7fa61105fb80 10 client.16791 fetch_fsmap learned FSMap version 42 2024-06-10T03:03:02.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.158+0000 7fa61105fb80 1 -- 172.21.15.52:0/1388916076 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x560a3f4ada30 con 0x560a3f49bfe0 2024-06-10T03:03:02.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.159+0000 7fa5df7fe700 1 -- 172.21.15.52:0/1388916076 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5e800fe90 con 0x560a3f49bfe0 2024-06-10T03:03:02.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.159+0000 7fa5df7fe700 1 -- 172.21.15.52:0/1388916076 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 42) ==== 8394+0+0 (secure 0 0 0) 0x7fa5e8020030 con 0x560a3f49bfe0 2024-06-10T03:03:02.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.159+0000 7fa61105fb80 10 client.16791 fetch_fsmap finished waiting for FSMap version 42 2024-06-10T03:03:02.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.159+0000 7fa61105fb80 10 client.16791 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T03:03:02.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.159+0000 7fa61105fb80 20 client.16791 populate_metadata read hostname 'smithi052' 2024-06-10T03:03:02.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.159+0000 7fa61105fb80 1 --2- 172.21.15.52:0/1388916076 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x560a3f4aff00 0x560a3f4b22f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:02.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.159+0000 7fa61105fb80 4 client.16791 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T03:03:02.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.159+0000 7fa61105fb80 1 -- 172.21.15.52:0/1388916076 --> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560a3f3f4dd0 con 0x560a3f4aff00 2024-06-10T03:03:02.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.159+0000 7fa5fffff700 1 --2- 172.21.15.52:0/1388916076 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x560a3f4aff00 0x560a3f4b22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:02.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.160+0000 7fa5fffff700 1 --2- 172.21.15.52:0/1388916076 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x560a3f4aff00 0x560a3f4b22f0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e0006440f36db40c server_cookie=546b443f4aef960c in_seq=0 out_seq=0 2024-06-10T03:03:02.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.160+0000 7fa5df7fe700 10 client.16791 ms_handle_connect on v2:172.21.15.188:6838/310444836 2024-06-10T03:03:02.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.160+0000 7fa5df7fe700 1 -- 172.21.15.52:0/1388916076 <== mds.0 v2:172.21.15.188:6838/310444836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x560a3f3f4dd0 con 0x560a3f4aff00 2024-06-10T03:03:02.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.160+0000 7fa5df7fe700 10 client.16791 handle_command_reply: tid=0 2024-06-10T03:03:02.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.160+0000 7fa61105fb80 1 client.16791 shutdown 2024-06-10T03:03:02.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.161+0000 7fa61105fb80 1 -- 172.21.15.52:0/1388916076 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560a3f49bfe0 msgr2=0x560a3f49fa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:03:02.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.161+0000 7fa61105fb80 1 --2- 172.21.15.52:0/1388916076 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560a3f49bfe0 0x560a3f49fa10 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fa5e800a630 tx=0x7fa5e8018b20 comp rx=0 tx=0).stop 2024-06-10T03:03:02.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.161+0000 7fa61105fb80 1 -- 172.21.15.52:0/1388916076 shutdown_connections 2024-06-10T03:03:02.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.161+0000 7fa61105fb80 1 --2- 172.21.15.52:0/1388916076 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x560a3f4aff00 0x560a3f4b22f0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:02.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.161+0000 7fa61105fb80 1 --2- 172.21.15.52:0/1388916076 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560a3f49c900 0x560a3f49ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:02.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.161+0000 7fa61105fb80 1 --2- 172.21.15.52:0/1388916076 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560a3f496820 0x560a3f4a26c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:02.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.161+0000 7fa61105fb80 1 --2- 172.21.15.52:0/1388916076 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560a3f49bfe0 0x560a3f49fa10 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:02.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.161+0000 7fa61105fb80 1 -- 172.21.15.52:0/1388916076 >> 172.21.15.52:0/1388916076 conn(0x560a3f3c3100 msgr2=0x560a3f490780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:03:02.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.161+0000 7fa5df7fe700 0 client.16791 ms_handle_reset on v2:172.21.15.188:6838/310444836 2024-06-10T03:03:02.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.161+0000 7fa61105fb80 1 -- 172.21.15.52:0/1388916076 shutdown_connections 2024-06-10T03:03:02.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.161+0000 7fa61105fb80 1 -- 172.21.15.52:0/1388916076 wait complete. 2024-06-10T03:03:02.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.161+0000 7fa61105fb80 20 client.16791 trim_cache size 0 max 16384 2024-06-10T03:03:02.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.182+0000 7fa61105fb80 1 Processor -- start 2024-06-10T03:03:02.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.182+0000 7fa61105fb80 1 -- start start 2024-06-10T03:03:02.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.182+0000 7fa61105fb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560a3f496820 0x560a3f4962e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:02.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.182+0000 7fa61105fb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560a3f49bfe0 0x560a3f49e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:02.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.182+0000 7fa61105fb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560a3f49c900 0x560a3f490f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:02.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.182+0000 7fa61105fb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fa6001b05b0 con 0x560a3f496820 2024-06-10T03:03:02.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.182+0000 7fa61105fb80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa6001b2240 con 0x560a3f49c900 2024-06-10T03:03:02.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.182+0000 7fa61105fb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa6001b0ce0 con 0x560a3f49bfe0 2024-06-10T03:03:02.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.182+0000 7fa5fffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560a3f496820 0x560a3f4962e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:02.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.182+0000 7fa5fffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560a3f496820 0x560a3f4962e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49366/0 (socket says 172.21.15.52:49366) 2024-06-10T03:03:02.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.182+0000 7fa5fffff700 1 -- 172.21.15.52:0/1709789370 learned_addr learned my addr 172.21.15.52:0/1709789370 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:03:02.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.182+0000 7fa5f7fff700 1 --2- 172.21.15.52:0/1709789370 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560a3f49bfe0 0x560a3f49e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:02.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.182+0000 7fa604d79700 1 --2- 172.21.15.52:0/1709789370 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560a3f49c900 0x560a3f490f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:02.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.182+0000 7fa5fffff700 1 -- 172.21.15.52:0/1709789370 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560a3f49bfe0 msgr2=0x560a3f49e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:03:02.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.182+0000 7fa5fffff700 1 --2- 172.21.15.52:0/1709789370 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560a3f49bfe0 0x560a3f49e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:02.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.182+0000 7fa5fffff700 1 -- 172.21.15.52:0/1709789370 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560a3f49c900 msgr2=0x560a3f490f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:03:02.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.182+0000 7fa5fffff700 1 --2- 172.21.15.52:0/1709789370 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560a3f49c900 0x560a3f490f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:02.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.182+0000 7fa5fffff700 1 -- 172.21.15.52:0/1709789370 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a3f49baa0 con 0x560a3f496820 2024-06-10T03:03:02.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.182+0000 7fa604d79700 1 --2- 172.21.15.52:0/1709789370 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560a3f49c900 0x560a3f490f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T03:03:02.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.183+0000 7fa5fffff700 1 --2- 172.21.15.52:0/1709789370 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560a3f496820 0x560a3f4962e0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fa5f00099c0 tx=0x7fa5f0023000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8620782962e3d79 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:03:02.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.183+0000 7fa5f5ffb700 1 -- 172.21.15.52:0/1709789370 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5f000f760 con 0x560a3f496820 2024-06-10T03:03:02.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.183+0000 7fa5f5ffb700 1 -- 172.21.15.52:0/1709789370 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5f000fde0 con 0x560a3f496820 2024-06-10T03:03:02.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.183+0000 7fa5f5ffb700 1 -- 172.21.15.52:0/1709789370 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5f0024410 con 0x560a3f496820 2024-06-10T03:03:02.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.183+0000 7fa61105fb80 1 -- 172.21.15.52:0/1709789370 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560a3f496820 msgr2=0x560a3f4962e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:03:02.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.183+0000 7fa61105fb80 1 --2- 172.21.15.52:0/1709789370 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560a3f496820 0x560a3f4962e0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fa5f00099c0 tx=0x7fa5f0023000 comp rx=0 tx=0).stop 2024-06-10T03:03:02.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.183+0000 7fa61105fb80 1 -- 172.21.15.52:0/1709789370 shutdown_connections 2024-06-10T03:03:02.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.183+0000 7fa61105fb80 1 --2- 172.21.15.52:0/1709789370 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560a3f49c900 0x560a3f490f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:02.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.183+0000 7fa61105fb80 1 --2- 172.21.15.52:0/1709789370 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560a3f496820 0x560a3f4962e0 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:02.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.183+0000 7fa61105fb80 1 --2- 172.21.15.52:0/1709789370 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560a3f49bfe0 0x560a3f49e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:02.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.183+0000 7fa61105fb80 1 -- 172.21.15.52:0/1709789370 >> 172.21.15.52:0/1709789370 conn(0x560a3f3c3100 msgr2=0x560a3f49e1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:03:02.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.184+0000 7fa61105fb80 1 -- 172.21.15.52:0/1709789370 shutdown_connections 2024-06-10T03:03:02.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.184+0000 7fa61105fb80 1 -- 172.21.15.52:0/1709789370 wait complete. 2024-06-10T03:03:02.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.184+0000 7fa61105fb80 1 -- start start 2024-06-10T03:03:02.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.184+0000 7fa61105fb80 1 Processor -- start 2024-06-10T03:03:02.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.184+0000 7fa61105fb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560a3f496820 0x560a3f4a86e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:02.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.184+0000 7fa61105fb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560a3f49bfe0 0x560a3f4a8c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:02.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.184+0000 7fa61105fb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560a3f49c900 0x560a3f4a4b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:02.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.184+0000 7fa61105fb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fa6001b1b40 con 0x560a3f496820 2024-06-10T03:03:02.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.184+0000 7fa61105fb80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa6001b1440 con 0x560a3f49bfe0 2024-06-10T03:03:02.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.184+0000 7fa61105fb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa6001ae550 con 0x560a3f49c900 2024-06-10T03:03:02.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.185+0000 7fa61105fb80 10 client.0 _ll_register_callbacks cb 0x560a3f37d630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T03:03:02.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.185+0000 7fa5fffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560a3f496820 0x560a3f4a86e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:02.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.185+0000 7fa5fffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560a3f496820 0x560a3f4a86e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49382/0 (socket says 172.21.15.52:49382) 2024-06-10T03:03:02.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.185+0000 7fa5fffff700 1 -- 172.21.15.52:0/472628267 learned_addr learned my addr 172.21.15.52:0/472628267 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:03:02.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.185+0000 7fa5f7fff700 1 --2- 172.21.15.52:0/472628267 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560a3f49bfe0 0x560a3f4a8c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:02.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.185+0000 7fa604d79700 1 --2- 172.21.15.52:0/472628267 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560a3f49c900 0x560a3f4a4b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:02.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.185+0000 7fa5fffff700 1 -- 172.21.15.52:0/472628267 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560a3f49c900 msgr2=0x560a3f4a4b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:03:02.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.185+0000 7fa5fffff700 1 --2- 172.21.15.52:0/472628267 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560a3f49c900 0x560a3f4a4b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:02.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.185+0000 7fa5fffff700 1 -- 172.21.15.52:0/472628267 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560a3f49bfe0 msgr2=0x560a3f4a8c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:03:02.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.185+0000 7fa5fffff700 1 --2- 172.21.15.52:0/472628267 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560a3f49bfe0 0x560a3f4a8c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:02.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.185+0000 7fa5fffff700 1 -- 172.21.15.52:0/472628267 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a3f4a5390 con 0x560a3f496820 2024-06-10T03:03:02.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.185+0000 7fa5fffff700 1 --2- 172.21.15.52:0/472628267 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560a3f496820 0x560a3f4a86e0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fa5f000ffd0 tx=0x7fa5f000b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49e350e6ae1ca884 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:03:02.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.185+0000 7fa5df7fe700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-10T03:03:02.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.185+0000 7fa5df7fe700 1 -- 172.21.15.52:0/472628267 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5f0033430 con 0x560a3f496820 2024-06-10T03:03:02.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.185+0000 7fa5df7fe700 1 -- 172.21.15.52:0/472628267 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7fa5f00335d0 con 0x560a3f496820 2024-06-10T03:03:02.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.185+0000 7fa61105fb80 1 -- 172.21.15.52:0/472628267 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa5f000fde0 con 0x560a3f496820 2024-06-10T03:03:02.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.185+0000 7fa5df7fe700 1 -- 172.21.15.52:0/472628267 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fa5f004c050 con 0x560a3f496820 2024-06-10T03:03:02.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.185+0000 7fa61105fb80 10 client.16797 fetch_fsmap learned FSMap version 42 2024-06-10T03:03:02.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.186+0000 7fa61105fb80 1 -- 172.21.15.52:0/472628267 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x560a3f51b3c0 con 0x560a3f496820 2024-06-10T03:03:02.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.186+0000 7fa5df7fe700 1 -- 172.21.15.52:0/472628267 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5f000fde0 con 0x560a3f496820 2024-06-10T03:03:02.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.186+0000 7fa5df7fe700 1 -- 172.21.15.52:0/472628267 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 42) ==== 8394+0+0 (secure 0 0 0) 0x7fa5f003a020 con 0x560a3f496820 2024-06-10T03:03:02.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.186+0000 7fa61105fb80 10 client.16797 fetch_fsmap finished waiting for FSMap version 42 2024-06-10T03:03:02.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.186+0000 7fa61105fb80 10 client.16797 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T03:03:02.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.186+0000 7fa61105fb80 20 client.16797 populate_metadata read hostname 'smithi052' 2024-06-10T03:03:02.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.186+0000 7fa61105fb80 1 --2- 172.21.15.52:0/472628267 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x560a3f51b960 0x560a3f51dd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:02.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.186+0000 7fa61105fb80 4 client.16797 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-06-10T03:03:02.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.186+0000 7fa61105fb80 1 -- 172.21.15.52:0/472628267 --> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x560a3f3ef350 con 0x560a3f51b960 2024-06-10T03:03:02.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.186+0000 7fa5f7fff700 1 --2- 172.21.15.52:0/472628267 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x560a3f51b960 0x560a3f51dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:02.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.187+0000 7fa5f7fff700 1 --2- 172.21.15.52:0/472628267 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x560a3f51b960 0x560a3f51dd50 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec8118a36cf08e7e server_cookie=afc6c21804c9246e in_seq=0 out_seq=0 2024-06-10T03:03:02.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.187+0000 7fa5df7fe700 10 client.16797 ms_handle_connect on v2:172.21.15.188:6838/310444836 2024-06-10T03:03:02.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.188+0000 7fa5df7fe700 1 -- 172.21.15.52:0/472628267 <== mds.0 v2:172.21.15.188:6838/310444836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x560a3f3ef350 con 0x560a3f51b960 2024-06-10T03:03:02.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.188+0000 7fa5df7fe700 10 client.16797 handle_command_reply: tid=0 2024-06-10T03:03:02.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.188+0000 7fa61105fb80 1 client.16797 shutdown 2024-06-10T03:03:02.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.188+0000 7fa61105fb80 1 -- 172.21.15.52:0/472628267 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560a3f496820 msgr2=0x560a3f4a86e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:03:02.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.188+0000 7fa61105fb80 1 --2- 172.21.15.52:0/472628267 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560a3f496820 0x560a3f4a86e0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fa5f000ffd0 tx=0x7fa5f000b890 comp rx=0 tx=0).stop 2024-06-10T03:03:02.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.188+0000 7fa61105fb80 1 -- 172.21.15.52:0/472628267 shutdown_connections 2024-06-10T03:03:02.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.188+0000 7fa61105fb80 1 --2- 172.21.15.52:0/472628267 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x560a3f51b960 0x560a3f51dd50 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:02.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.188+0000 7fa61105fb80 1 --2- 172.21.15.52:0/472628267 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560a3f49c900 0x560a3f4a4b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:02.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.188+0000 7fa61105fb80 1 --2- 172.21.15.52:0/472628267 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560a3f496820 0x560a3f4a86e0 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:02.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.188+0000 7fa5df7fe700 0 client.16797 ms_handle_reset on v2:172.21.15.188:6838/310444836 2024-06-10T03:03:02.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.188+0000 7fa61105fb80 1 --2- 172.21.15.52:0/472628267 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560a3f49bfe0 0x560a3f4a8c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:02.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.188+0000 7fa61105fb80 1 -- 172.21.15.52:0/472628267 >> 172.21.15.52:0/472628267 conn(0x560a3f3c3100 msgr2=0x560a3f49b320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:03:02.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.188+0000 7fa61105fb80 1 -- 172.21.15.52:0/472628267 shutdown_connections 2024-06-10T03:03:02.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.188+0000 7fa61105fb80 1 -- 172.21.15.52:0/472628267 wait complete. 2024-06-10T03:03:02.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.188+0000 7fa61105fb80 20 client.16797 trim_cache size 0 max 16384 2024-06-10T03:03:02.193 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-10T03:03:02.193 INFO:teuthology.orchestra.run.smithi052.stdout: "return_code": 0, 2024-06-10T03:03:02.193 INFO:teuthology.orchestra.run.smithi052.stdout: "scrub_tag": "9ba0b5c7-4aa7-4870-a07e-1c62d90b40dc", 2024-06-10T03:03:02.193 INFO:teuthology.orchestra.run.smithi052.stdout: "mode": "asynchronous" 2024-06-10T03:03:02.193 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-10T03:03:02.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.189+0000 7fa6067dc700 1 -- 172.21.15.52:0/1084389659 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fa5ec0a2c20 msgr2=0x7fa5ec0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:03:02.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.189+0000 7fa6067dc700 1 --2- 172.21.15.52:0/1084389659 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fa5ec0a2c20 0x7fa5ec0a50e0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fa5f0000c00 tx=0x7fa5f0005fb0 comp rx=0 tx=0).stop 2024-06-10T03:03:02.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.189+0000 7fa6067dc700 1 -- 172.21.15.52:0/1084389659 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa600107840 msgr2=0x7fa600075210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:03:02.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.189+0000 7fa6067dc700 1 --2- 172.21.15.52:0/1084389659 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa600107840 0x7fa600075210 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fa5f8007ee0 tx=0x7fa5f800d3b0 comp rx=0 tx=0).stop 2024-06-10T03:03:02.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.189+0000 7fa6067dc700 1 -- 172.21.15.52:0/1084389659 shutdown_connections 2024-06-10T03:03:02.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.189+0000 7fa6067dc700 1 --2- 172.21.15.52:0/1084389659 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fa5ec0a2c20 0x7fa5ec0a50e0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:02.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.189+0000 7fa6067dc700 1 --2- 172.21.15.52:0/1084389659 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa600107840 0x7fa600075210 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:02.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.189+0000 7fa6067dc700 1 --2- 172.21.15.52:0/1084389659 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa600102740 0x7fa600074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:02.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.189+0000 7fa6067dc700 1 --2- 172.21.15.52:0/1084389659 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa6001030f0 0x7fa600074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:02.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.189+0000 7fa6067dc700 1 -- 172.21.15.52:0/1084389659 >> 172.21.15.52:0/1084389659 conn(0x7fa6000795f0 msgr2=0x7fa600108990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:03:02.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.189+0000 7fa6067dc700 1 -- 172.21.15.52:0/1084389659 shutdown_connections 2024-06-10T03:03:02.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:02.189+0000 7fa6067dc700 1 -- 172.21.15.52:0/1084389659 wait complete. 2024-06-10T03:03:02.954 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:03:02 smithi188 ceph-mon[36599]: pgmap v699: 129 pgs: 129 active+clean; 279 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 984 KiB/s rd, 925 KiB/s wr, 355 op/s 2024-06-10T03:03:02.954 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:03:02 smithi188 ceph-mon[36599]: scrub queued for path: / 2024-06-10T03:03:02.954 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:03:02 smithi188 ceph-mon[36599]: scrub summary: idle+waiting paths [/] 2024-06-10T03:03:02.954 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:03:02 smithi188 ceph-mon[36599]: scrub summary: active paths [/] 2024-06-10T03:03:02.954 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:03:02 smithi188 ceph-mon[36599]: scrub summary: idle+waiting paths [/] 2024-06-10T03:03:02.954 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:03:02 smithi188 ceph-mon[36599]: scrub summary: active paths [/] 2024-06-10T03:03:02.954 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:03:02 smithi188 ceph-mon[36599]: scrub summary: idle+waiting paths [/] 2024-06-10T03:03:02.954 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:03:02 smithi188 ceph-mon[36599]: scrub summary: active paths [/] 2024-06-10T03:03:02.954 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:03:02 smithi188 ceph-mon[36599]: scrub summary: idle+waiting paths [/] 2024-06-10T03:03:02.977 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:03:02 smithi052 ceph-mon[32362]: pgmap v699: 129 pgs: 129 active+clean; 279 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 984 KiB/s rd, 925 KiB/s wr, 355 op/s 2024-06-10T03:03:02.977 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:03:02 smithi052 ceph-mon[32362]: scrub queued for path: / 2024-06-10T03:03:02.977 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:03:02 smithi052 ceph-mon[32362]: scrub summary: idle+waiting paths [/] 2024-06-10T03:03:02.977 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:03:02 smithi052 ceph-mon[32362]: scrub summary: active paths [/] 2024-06-10T03:03:02.977 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:03:02 smithi052 ceph-mon[32362]: scrub summary: idle+waiting paths [/] 2024-06-10T03:03:02.977 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:03:02 smithi052 ceph-mon[32362]: scrub summary: active paths [/] 2024-06-10T03:03:02.977 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:03:02 smithi052 ceph-mon[32362]: scrub summary: idle+waiting paths [/] 2024-06-10T03:03:02.977 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:03:02 smithi052 ceph-mon[32362]: scrub summary: active paths [/] 2024-06-10T03:03:02.977 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:03:02 smithi052 ceph-mon[32362]: scrub summary: idle+waiting paths [/] 2024-06-10T03:03:03.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:03:02 smithi120 ceph-mon[36634]: pgmap v699: 129 pgs: 129 active+clean; 279 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 984 KiB/s rd, 925 KiB/s wr, 355 op/s 2024-06-10T03:03:03.139 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:03:02 smithi120 ceph-mon[36634]: scrub queued for path: / 2024-06-10T03:03:03.139 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:03:02 smithi120 ceph-mon[36634]: scrub summary: idle+waiting paths [/] 2024-06-10T03:03:03.139 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:03:02 smithi120 ceph-mon[36634]: scrub summary: active paths [/] 2024-06-10T03:03:03.139 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:03:02 smithi120 ceph-mon[36634]: scrub summary: idle+waiting paths [/] 2024-06-10T03:03:03.139 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:03:02 smithi120 ceph-mon[36634]: scrub summary: active paths [/] 2024-06-10T03:03:03.139 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:03:02 smithi120 ceph-mon[36634]: scrub summary: idle+waiting paths [/] 2024-06-10T03:03:03.139 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:03:02 smithi120 ceph-mon[36634]: scrub summary: active paths [/] 2024-06-10T03:03:03.139 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:03:02 smithi120 ceph-mon[36634]: scrub summary: idle+waiting paths [/] 2024-06-10T03:03:03.891 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:03:03 smithi052 ceph-mon[32362]: scrub summary: active paths [/] 2024-06-10T03:03:03.891 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:03:03 smithi052 ceph-mon[32362]: scrub summary: idle+waiting paths [/] 2024-06-10T03:03:03.919 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-10T03:03:03.954 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:03:03 smithi188 ceph-mon[36599]: scrub summary: active paths [/] 2024-06-10T03:03:03.954 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:03:03 smithi188 ceph-mon[36599]: scrub summary: idle+waiting paths [/] 2024-06-10T03:03:04.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:03.709+0000 7fde0b5c4700 1 -- 172.21.15.52:0/3448863459 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fde0406b5c0 msgr2=0x7fde0410f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:03:04.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:03.709+0000 7fde0b5c4700 1 --2- 172.21.15.52:0/3448863459 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fde0406b5c0 0x7fde0410f320 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fddfc00b0a0 tx=0x7fddfc00b3b0 comp rx=0 tx=0).stop 2024-06-10T03:03:04.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:03.709+0000 7fde0b5c4700 1 -- 172.21.15.52:0/3448863459 shutdown_connections 2024-06-10T03:03:04.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:03.709+0000 7fde0b5c4700 1 --2- 172.21.15.52:0/3448863459 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fde0406b5c0 0x7fde0410f320 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:04.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:03.709+0000 7fde0b5c4700 1 --2- 172.21.15.52:0/3448863459 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fde04108500 0x7fde041088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:04.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:03.709+0000 7fde0b5c4700 1 --2- 172.21.15.52:0/3448863459 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fde0406ac00 0x7fde0406b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:04.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:03.709+0000 7fde0b5c4700 1 -- 172.21.15.52:0/3448863459 >> 172.21.15.52:0/3448863459 conn(0x7fde04078340 msgr2=0x7fde04078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:03:04.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:03.709+0000 7fde0b5c4700 1 -- 172.21.15.52:0/3448863459 shutdown_connections 2024-06-10T03:03:04.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:03.709+0000 7fde0b5c4700 1 -- 172.21.15.52:0/3448863459 wait complete. 2024-06-10T03:03:04.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:03.710+0000 7fde0b5c4700 1 Processor -- start 2024-06-10T03:03:04.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:03.710+0000 7fde0b5c4700 1 -- start start 2024-06-10T03:03:04.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:03.710+0000 7fde0b5c4700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fde0406ac00 0x7fde0419e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:04.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:03.710+0000 7fde0b5c4700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fde0406b5c0 0x7fde0419ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:04.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:03.710+0000 7fde0b5c4700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fde04108500 0x7fde041989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:04.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:03.710+0000 7fde0b5c4700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fde04112760 con 0x7fde0406ac00 2024-06-10T03:03:04.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:03.710+0000 7fde0b5c4700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fde041125e0 con 0x7fde0406b5c0 2024-06-10T03:03:04.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:03.710+0000 7fde0b5c4700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fde041128e0 con 0x7fde04108500 2024-06-10T03:03:04.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:03.710+0000 7fde09360700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fde0406ac00 0x7fde0419e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:04.019 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-10T03:03:04.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:03.710+0000 7fde09360700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fde0406ac00 0x7fde0419e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49400/0 (socket says 172.21.15.52:49400) 2024-06-10T03:03:04.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:03.710+0000 7fde09360700 1 -- 172.21.15.52:0/1884536748 learned_addr learned my addr 172.21.15.52:0/1884536748 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:03:04.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:03.710+0000 7fde09b61700 1 --2- 172.21.15.52:0/1884536748 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fde04108500 0x7fde041989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:04.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:03.711+0000 7fde08b5f700 1 --2- 172.21.15.52:0/1884536748 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fde0406b5c0 0x7fde0419ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:04.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:03.711+0000 7fde09360700 1 -- 172.21.15.52:0/1884536748 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fde04108500 msgr2=0x7fde041989e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:03:04.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:03.711+0000 7fde09360700 1 --2- 172.21.15.52:0/1884536748 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fde04108500 0x7fde041989e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:04.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:03.711+0000 7fde09360700 1 -- 172.21.15.52:0/1884536748 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fde0406b5c0 msgr2=0x7fde0419ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:03:04.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:03.711+0000 7fde09360700 1 --2- 172.21.15.52:0/1884536748 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fde0406b5c0 0x7fde0419ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:04.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:03.711+0000 7fde09360700 1 -- 172.21.15.52:0/1884536748 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde04199140 con 0x7fde0406ac00 2024-06-10T03:03:04.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:03.711+0000 7fde09360700 1 --2- 172.21.15.52:0/1884536748 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fde0406ac00 0x7fde0419e820 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fde0000d980 tx=0x7fde0000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:03:04.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:03.711+0000 7fddfa7fc700 1 -- 172.21.15.52:0/1884536748 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde00016070 con 0x7fde0406ac00 2024-06-10T03:03:04.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:03.711+0000 7fddfa7fc700 1 -- 172.21.15.52:0/1884536748 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde00004d10 con 0x7fde0406ac00 2024-06-10T03:03:04.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:03.711+0000 7fde0b5c4700 1 -- 172.21.15.52:0/1884536748 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde04199430 con 0x7fde0406ac00 2024-06-10T03:03:04.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:03.711+0000 7fddfa7fc700 1 -- 172.21.15.52:0/1884536748 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde00005670 con 0x7fde0406ac00 2024-06-10T03:03:04.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:03.711+0000 7fde0b5c4700 1 -- 172.21.15.52:0/1884536748 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde041a56f0 con 0x7fde0406ac00 2024-06-10T03:03:04.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:03.712+0000 7fde0b5c4700 1 -- 172.21.15.52:0/1884536748 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde04103bc0 con 0x7fde0406ac00 2024-06-10T03:03:04.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:03.714+0000 7fddfa7fc700 1 -- 172.21.15.52:0/1884536748 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fde00004750 con 0x7fde0406ac00 2024-06-10T03:03:04.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:03.715+0000 7fddfa7fc700 1 --2- 172.21.15.52:0/1884536748 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fddf00a2c20 0x7fddf00a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:04.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:03.715+0000 7fde08b5f700 1 --2- 172.21.15.52:0/1884536748 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fddf00a2c20 0x7fddf00a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:04.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:03.715+0000 7fddfa7fc700 1 -- 172.21.15.52:0/1884536748 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7fde000cf6c0 con 0x7fde0406ac00 2024-06-10T03:03:04.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:03.715+0000 7fde08b5f700 1 --2- 172.21.15.52:0/1884536748 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fddf00a2c20 0x7fddf00a50e0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fde04078940 tx=0x7fddf4009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:03:04.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:03.719+0000 7fddfa7fc700 1 -- 172.21.15.52:0/1884536748 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fde0001f020 con 0x7fde0406ac00 2024-06-10T03:03:04.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:03.974+0000 7fde0b5c4700 1 -- 172.21.15.52:0/1884536748 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fde04199f10 con 0x7fde0406ac00 2024-06-10T03:03:04.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:03.975+0000 7fddfa7fc700 1 -- 172.21.15.52:0/1884536748 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) ==== 94+0+11949 (secure 0 0 0) 0x7fde0009c210 con 0x7fde0406ac00 2024-06-10T03:03:04.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:03.978+0000 7fde0b5c4700 1 -- 172.21.15.52:0/1884536748 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fddf00a2c20 msgr2=0x7fddf00a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:03:04.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:03.978+0000 7fde0b5c4700 1 --2- 172.21.15.52:0/1884536748 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fddf00a2c20 0x7fddf00a50e0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fde04078940 tx=0x7fddf4009260 comp rx=0 tx=0).stop 2024-06-10T03:03:04.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:03.978+0000 7fde0b5c4700 1 -- 172.21.15.52:0/1884536748 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fde0406ac00 msgr2=0x7fde0419e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:03:04.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:03.978+0000 7fde0b5c4700 1 --2- 172.21.15.52:0/1884536748 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fde0406ac00 0x7fde0419e820 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fde0000d980 tx=0x7fde0000dc90 comp rx=0 tx=0).stop 2024-06-10T03:03:04.022 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":42,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/889673247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":889673247},{"type":"v1","addr":"172.21.15.52:6835","nonce":889673247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/3615663511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":3615663511},{"type":"v1","addr":"172.21.15.52:6837","nonce":3615663511}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":42,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T02:50:03.889766+0000","modified":"2024-06-10T03:02:33.697178+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24484,"mds_1":24469,"mds_2":24463,"mds_3":24449,"mds_4":24428},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14571":{"gid":14571,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6839/3059599042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":3059599042},{"type":"v1","addr":"172.21.15.52:6839","nonce":3059599042}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14583":{"gid":14583,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6841/3039275521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":3039275521},{"type":"v1","addr":"172.21.15.52:6841","nonce":3039275521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24383":{"gid":24383,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6833/2609002896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":2609002896},{"type":"v1","addr":"172.21.15.120:6833","nonce":2609002896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6835/1996370288","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":1996370288},{"type":"v1","addr":"172.21.15.120:6835","nonce":1996370288}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6833/529564979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":529564979},{"type":"v1","addr":"172.21.15.188:6833","nonce":529564979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":25,"addr":"172.21.15.120:6837/4132440952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":4132440952},{"type":"v1","addr":"172.21.15.120:6837","nonce":4132440952}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"b","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.120:6839/1672436487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1672436487},{"type":"v1","addr":"172.21.15.120:6839","nonce":1672436487}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":2,"incarnation":22,"state":"up:active","state_seq":26,"addr":"172.21.15.188:6835/3904544259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":3904544259},{"type":"v1","addr":"172.21.15.188:6835","nonce":3904544259}]},"join_fscid":-1,"export_targets":[1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.188:6837/819317726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":819317726},{"type":"v1","addr":"172.21.15.188:6837","nonce":819317726}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.188:6839/310444836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":310444836},{"type":"v1","addr":"172.21.15.188:6839","nonce":310444836}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24484,"qdb_cluster":[24469,24484,24463,24428,24449]},"id":1}]} 2024-06-10T03:03:04.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:03.978+0000 7fde0b5c4700 1 -- 172.21.15.52:0/1884536748 shutdown_connections 2024-06-10T03:03:04.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:03.978+0000 7fde0b5c4700 1 --2- 172.21.15.52:0/1884536748 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fde04108500 0x7fde041989e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:04.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:03.978+0000 7fde0b5c4700 1 --2- 172.21.15.52:0/1884536748 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fddf00a2c20 0x7fddf00a50e0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:04.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:03.978+0000 7fde0b5c4700 1 --2- 172.21.15.52:0/1884536748 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fde0406ac00 0x7fde0419e820 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:04.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:03.978+0000 7fde0b5c4700 1 --2- 172.21.15.52:0/1884536748 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fde0406b5c0 0x7fde0419ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:04.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:03.978+0000 7fde0b5c4700 1 -- 172.21.15.52:0/1884536748 >> 172.21.15.52:0/1884536748 conn(0x7fde04078340 msgr2=0x7fde040fff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:03:04.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:03.978+0000 7fde0b5c4700 1 -- 172.21.15.52:0/1884536748 shutdown_connections 2024-06-10T03:03:04.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:03.978+0000 7fde0b5c4700 1 -- 172.21.15.52:0/1884536748 wait complete. 2024-06-10T03:03:04.024 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 42 2024-06-10T03:03:04.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:03:03 smithi120 ceph-mon[36634]: scrub summary: active paths [/] 2024-06-10T03:03:04.139 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:03:03 smithi120 ceph-mon[36634]: scrub summary: idle+waiting paths [/] 2024-06-10T03:03:04.660 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24484 quiesce db --set-id 6a4162d1 --release --await' 2024-06-10T03:03:04.660 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24484 quiesce db --set-id 6a4162d1 --release --await 2024-06-10T03:03:04.715 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T03:03:04.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:03:04 smithi052 ceph-mon[32362]: pgmap v700: 129 pgs: 129 active+clean; 277 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 925 KiB/s wr, 392 op/s 2024-06-10T03:03:04.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:03:04 smithi052 ceph-mon[32362]: from='client.? 172.21.15.52:0/1884536748' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T03:03:04.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:03:04 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T03:03:04.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:03:04 smithi188 ceph-mon[36599]: pgmap v700: 129 pgs: 129 active+clean; 277 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 925 KiB/s wr, 392 op/s 2024-06-10T03:03:04.954 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:03:04 smithi188 ceph-mon[36599]: from='client.? 172.21.15.52:0/1884536748' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T03:03:04.954 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:03:04 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T03:03:05.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:03:04 smithi120 ceph-mon[36634]: pgmap v700: 129 pgs: 129 active+clean; 277 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 925 KiB/s wr, 392 op/s 2024-06-10T03:03:05.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:03:04 smithi120 ceph-mon[36634]: from='client.? 172.21.15.52:0/1884536748' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T03:03:05.139 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:03:04 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T03:03:05.905 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T03:03:06.727 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:03:06 smithi052 ceph-mon[32362]: pgmap v701: 129 pgs: 129 active+clean; 277 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 523 KiB/s rd, 377 KiB/s wr, 282 op/s 2024-06-10T03:03:06.727 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:03:06 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T03:03:06.727 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:03:06 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T03:03:06.727 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:03:06 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T03:03:06.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:03:06 smithi188 ceph-mon[36599]: pgmap v701: 129 pgs: 129 active+clean; 277 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 523 KiB/s rd, 377 KiB/s wr, 282 op/s 2024-06-10T03:03:06.954 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:03:06 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T03:03:06.954 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:03:06 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T03:03:06.954 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:03:06 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T03:03:07.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:03:06 smithi120 ceph-mon[36634]: pgmap v701: 129 pgs: 129 active+clean; 277 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 523 KiB/s rd, 377 KiB/s wr, 282 op/s 2024-06-10T03:03:07.139 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:03:06 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T03:03:07.139 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:03:06 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T03:03:07.139 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:03:06 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T03:03:07.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.467+0000 7f2eef6ff700 1 -- 172.21.15.52:0/665536471 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2ee8107870 msgr2=0x7f2ee810f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:03:07.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.467+0000 7f2eef6ff700 1 --2- 172.21.15.52:0/665536471 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2ee8107870 0x7f2ee810f3f0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f2ee000b0d0 tx=0x7f2ee000b3e0 comp rx=0 tx=0).stop 2024-06-10T03:03:07.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.467+0000 7f2eef6ff700 1 -- 172.21.15.52:0/665536471 shutdown_connections 2024-06-10T03:03:07.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.467+0000 7f2eef6ff700 1 --2- 172.21.15.52:0/665536471 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2ee8107870 0x7f2ee810f3f0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:07.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.467+0000 7f2eef6ff700 1 --2- 172.21.15.52:0/665536471 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2ee81027a0 0x7f2ee8102b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:07.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.467+0000 7f2eef6ff700 1 --2- 172.21.15.52:0/665536471 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2ee8103150 0x7f2ee8107140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:07.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.467+0000 7f2eef6ff700 1 -- 172.21.15.52:0/665536471 >> 172.21.15.52:0/665536471 conn(0x7f2ee8079570 msgr2=0x7f2ee8079980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:03:07.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.467+0000 7f2eef6ff700 1 -- 172.21.15.52:0/665536471 shutdown_connections 2024-06-10T03:03:07.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.468+0000 7f2eef6ff700 1 -- 172.21.15.52:0/665536471 wait complete. 2024-06-10T03:03:07.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.468+0000 7f2eef6ff700 1 Processor -- start 2024-06-10T03:03:07.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.468+0000 7f2eef6ff700 1 -- start start 2024-06-10T03:03:07.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.468+0000 7f2eef6ff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2ee81027a0 0x7f2ee819adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:07.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.468+0000 7f2eef6ff700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2ee8103150 0x7f2ee819b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:07.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.468+0000 7f2eef6ff700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2ee8107870 0x7f2ee819f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:07.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.468+0000 7f2eef6ff700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee81127d0 con 0x7f2ee81027a0 2024-06-10T03:03:07.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.468+0000 7f2eef6ff700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee8112650 con 0x7f2ee8103150 2024-06-10T03:03:07.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.468+0000 7f2eef6ff700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee8112950 con 0x7f2ee8107870 2024-06-10T03:03:07.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.468+0000 7f2eed49b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2ee81027a0 0x7f2ee819adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:07.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.469+0000 7f2eed49b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2ee81027a0 0x7f2ee819adb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49418/0 (socket says 172.21.15.52:49418) 2024-06-10T03:03:07.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.469+0000 7f2eed49b700 1 -- 172.21.15.52:0/1770543968 learned_addr learned my addr 172.21.15.52:0/1770543968 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:03:07.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.469+0000 7f2eedc9c700 1 --2- 172.21.15.52:0/1770543968 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2ee8107870 0x7f2ee819f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:07.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.469+0000 7f2eecc9a700 1 --2- 172.21.15.52:0/1770543968 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2ee8103150 0x7f2ee819b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:07.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.469+0000 7f2eed49b700 1 -- 172.21.15.52:0/1770543968 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2ee8107870 msgr2=0x7f2ee819f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:03:07.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.469+0000 7f2eed49b700 1 --2- 172.21.15.52:0/1770543968 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2ee8107870 0x7f2ee819f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:07.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.469+0000 7f2eed49b700 1 -- 172.21.15.52:0/1770543968 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2ee8103150 msgr2=0x7f2ee819b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:03:07.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.469+0000 7f2eed49b700 1 --2- 172.21.15.52:0/1770543968 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2ee8103150 0x7f2ee819b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:07.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.469+0000 7f2eed49b700 1 -- 172.21.15.52:0/1770543968 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ee819fd70 con 0x7f2ee81027a0 2024-06-10T03:03:07.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.469+0000 7f2eedc9c700 1 --2- 172.21.15.52:0/1770543968 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2ee8107870 0x7f2ee819f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T03:03:07.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.469+0000 7f2eed49b700 1 --2- 172.21.15.52:0/1770543968 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2ee81027a0 0x7f2ee819adb0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f2ee400d980 tx=0x7f2ee400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:03:07.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.469+0000 7f2ede7fc700 1 -- 172.21.15.52:0/1770543968 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ee4017070 con 0x7f2ee81027a0 2024-06-10T03:03:07.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.469+0000 7f2ede7fc700 1 -- 172.21.15.52:0/1770543968 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ee4005020 con 0x7f2ee81027a0 2024-06-10T03:03:07.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.469+0000 7f2eef6ff700 1 -- 172.21.15.52:0/1770543968 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ee81a0060 con 0x7f2ee81027a0 2024-06-10T03:03:07.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.469+0000 7f2ede7fc700 1 -- 172.21.15.52:0/1770543968 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ee400fbf0 con 0x7f2ee81027a0 2024-06-10T03:03:07.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.469+0000 7f2eef6ff700 1 -- 172.21.15.52:0/1770543968 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ee81a78e0 con 0x7f2ee81027a0 2024-06-10T03:03:07.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.472+0000 7f2ede7fc700 1 -- 172.21.15.52:0/1770543968 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f2ee40051d0 con 0x7f2ee81027a0 2024-06-10T03:03:07.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.472+0000 7f2ede7fc700 1 --2- 172.21.15.52:0/1770543968 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f2ed40a2c20 0x7f2ed40a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:07.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.472+0000 7f2eecc9a700 1 --2- 172.21.15.52:0/1770543968 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f2ed40a2c20 0x7f2ed40a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:07.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.472+0000 7f2ede7fc700 1 -- 172.21.15.52:0/1770543968 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f2ee40cf220 con 0x7f2ee81027a0 2024-06-10T03:03:07.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.473+0000 7f2ef9f82b80 1 Processor -- start 2024-06-10T03:03:07.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.473+0000 7f2eecc9a700 1 --2- 172.21.15.52:0/1770543968 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f2ed40a2c20 0x7f2ed40a50e0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f2ee81090a0 tx=0x7f2ed800b3c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:03:07.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.473+0000 7f2ef9f82b80 1 -- start start 2024-06-10T03:03:07.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.473+0000 7f2ef9f82b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5640a774cf60 0x5640a774d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:07.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.473+0000 7f2ef9f82b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5640a774d880 0x5640a7747260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:07.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.473+0000 7f2ef9f82b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5640a77477a0 0x5640a7747bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:07.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.473+0000 7f2ef9f82b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5640a7639f60 con 0x5640a774d880 2024-06-10T03:03:07.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.473+0000 7f2ef9f82b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5640a7698850 con 0x5640a77477a0 2024-06-10T03:03:07.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.473+0000 7f2ef9f82b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5640a7600460 con 0x5640a774cf60 2024-06-10T03:03:07.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.473+0000 7f2eecc9a700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5640a774d880 0x5640a7747260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:07.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.473+0000 7f2eed49b700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5640a774cf60 0x5640a774d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:07.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.473+0000 7f2eecc9a700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5640a774d880 0x5640a7747260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49430/0 (socket says 172.21.15.52:49430) 2024-06-10T03:03:07.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.473+0000 7f2eecc9a700 1 -- 172.21.15.52:0/2657403258 learned_addr learned my addr 172.21.15.52:0/2657403258 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:03:07.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.473+0000 7f2eed49b700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5640a774cf60 0x5640a774d340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.52:35490/0 (socket says 172.21.15.52:35490) 2024-06-10T03:03:07.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.474+0000 7f2eedc9c700 1 --2- 172.21.15.52:0/2657403258 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5640a77477a0 0x5640a7747bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:07.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.474+0000 7f2eecc9a700 1 -- 172.21.15.52:0/2657403258 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5640a774cf60 msgr2=0x5640a774d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:03:07.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.474+0000 7f2eecc9a700 1 --2- 172.21.15.52:0/2657403258 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5640a774cf60 0x5640a774d340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:07.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.474+0000 7f2eecc9a700 1 -- 172.21.15.52:0/2657403258 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5640a77477a0 msgr2=0x5640a7747bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:03:07.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.474+0000 7f2eecc9a700 1 --2- 172.21.15.52:0/2657403258 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5640a77477a0 0x5640a7747bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:07.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.474+0000 7f2eecc9a700 1 -- 172.21.15.52:0/2657403258 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5640a75e78d0 con 0x5640a774d880 2024-06-10T03:03:07.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.474+0000 7f2eed49b700 1 --2- 172.21.15.52:0/2657403258 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5640a774cf60 0x5640a774d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T03:03:07.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.474+0000 7f2eecc9a700 1 --2- 172.21.15.52:0/2657403258 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5640a774d880 0x5640a7747260 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f2ed800b670 tx=0x7f2ed8013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2123def8efaddb3 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:03:07.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.474+0000 7f2ed37fe700 1 -- 172.21.15.52:0/2657403258 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ed80118d0 con 0x5640a774d880 2024-06-10T03:03:07.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.474+0000 7f2ed37fe700 1 -- 172.21.15.52:0/2657403258 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ed8004830 con 0x5640a774d880 2024-06-10T03:03:07.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.474+0000 7f2ed37fe700 1 -- 172.21.15.52:0/2657403258 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ed80054a0 con 0x5640a774d880 2024-06-10T03:03:07.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.475+0000 7f2ef9f82b80 1 -- 172.21.15.52:0/2657403258 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5640a774d880 msgr2=0x5640a7747260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:03:07.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.475+0000 7f2ef9f82b80 1 --2- 172.21.15.52:0/2657403258 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5640a774d880 0x5640a7747260 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f2ed800b670 tx=0x7f2ed8013000 comp rx=0 tx=0).stop 2024-06-10T03:03:07.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.475+0000 7f2ef9f82b80 1 -- 172.21.15.52:0/2657403258 shutdown_connections 2024-06-10T03:03:07.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.475+0000 7f2ef9f82b80 1 --2- 172.21.15.52:0/2657403258 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5640a77477a0 0x5640a7747bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:07.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.475+0000 7f2ef9f82b80 1 --2- 172.21.15.52:0/2657403258 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5640a774cf60 0x5640a774d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:07.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.475+0000 7f2ef9f82b80 1 --2- 172.21.15.52:0/2657403258 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5640a774d880 0x5640a7747260 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:07.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.475+0000 7f2ef9f82b80 1 -- 172.21.15.52:0/2657403258 >> 172.21.15.52:0/2657403258 conn(0x5640a7674080 msgr2=0x5640a7742220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:03:07.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.475+0000 7f2ef9f82b80 1 -- 172.21.15.52:0/2657403258 shutdown_connections 2024-06-10T03:03:07.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.475+0000 7f2ef9f82b80 1 -- 172.21.15.52:0/2657403258 wait complete. 2024-06-10T03:03:07.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.475+0000 7f2ef9f82b80 1 -- start start 2024-06-10T03:03:07.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.476+0000 7f2ef9f82b80 1 Processor -- start 2024-06-10T03:03:07.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.476+0000 7f2ef9f82b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5640a77477a0 0x5640a7753620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:07.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.476+0000 7f2ef9f82b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5640a774cf60 0x5640a7750920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:07.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.476+0000 7f2ef9f82b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5640a774d880 0x5640a7750e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:07.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.476+0000 7f2ef9f82b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5640a7638fc0 con 0x5640a774cf60 2024-06-10T03:03:07.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.476+0000 7f2ef9f82b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5640a7694810 con 0x5640a774d880 2024-06-10T03:03:07.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.476+0000 7f2ef9f82b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5640a7754260 con 0x5640a77477a0 2024-06-10T03:03:07.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.476+0000 7f2ef9f82b80 10 client.0 _ll_register_callbacks cb 0x5640a762e5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T03:03:07.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.476+0000 7f2eecc9a700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5640a774cf60 0x5640a7750920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:07.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.476+0000 7f2eecc9a700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5640a774cf60 0x5640a7750920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49444/0 (socket says 172.21.15.52:49444) 2024-06-10T03:03:07.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.476+0000 7f2eecc9a700 1 -- 172.21.15.52:0/826236815 learned_addr learned my addr 172.21.15.52:0/826236815 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:03:07.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.476+0000 7f2eed49b700 1 --2- 172.21.15.52:0/826236815 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5640a77477a0 0x5640a7753620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:07.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.476+0000 7f2eedc9c700 1 --2- 172.21.15.52:0/826236815 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5640a774d880 0x5640a7750e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:07.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.477+0000 7f2eecc9a700 1 -- 172.21.15.52:0/826236815 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5640a77477a0 msgr2=0x5640a7753620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:03:07.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.477+0000 7f2eecc9a700 1 --2- 172.21.15.52:0/826236815 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5640a77477a0 0x5640a7753620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:07.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.477+0000 7f2eecc9a700 1 -- 172.21.15.52:0/826236815 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5640a774d880 msgr2=0x5640a7750e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:03:07.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.477+0000 7f2eecc9a700 1 --2- 172.21.15.52:0/826236815 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5640a774d880 0x5640a7750e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:07.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.477+0000 7f2eecc9a700 1 -- 172.21.15.52:0/826236815 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5640a765e750 con 0x5640a774cf60 2024-06-10T03:03:07.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.477+0000 7f2eed49b700 1 --2- 172.21.15.52:0/826236815 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5640a77477a0 0x5640a7753620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T03:03:07.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.477+0000 7f2eecc9a700 1 --2- 172.21.15.52:0/826236815 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5640a774cf60 0x5640a7750920 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f2ed8004c90 tx=0x7f2ed8004cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f99b7bdeb7711a4 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:03:07.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.477+0000 7f2ed0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-10T03:03:07.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.477+0000 7f2ed0ff9700 1 -- 172.21.15.52:0/826236815 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ed8035060 con 0x5640a774cf60 2024-06-10T03:03:07.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.477+0000 7f2ef9f82b80 1 -- 172.21.15.52:0/826236815 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ed8004830 con 0x5640a774cf60 2024-06-10T03:03:07.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.477+0000 7f2ed0ff9700 1 -- 172.21.15.52:0/826236815 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f2ed8003de0 con 0x5640a774cf60 2024-06-10T03:03:07.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.477+0000 7f2ed0ff9700 1 -- 172.21.15.52:0/826236815 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f2ed803c050 con 0x5640a774cf60 2024-06-10T03:03:07.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.477+0000 7f2ef9f82b80 10 client.16815 fetch_fsmap learned FSMap version 42 2024-06-10T03:03:07.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.477+0000 7f2ef9f82b80 1 -- 172.21.15.52:0/826236815 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x5640a775ea10 con 0x5640a774cf60 2024-06-10T03:03:07.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.477+0000 7f2ed0ff9700 1 -- 172.21.15.52:0/826236815 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ed8004830 con 0x5640a774cf60 2024-06-10T03:03:07.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.478+0000 7f2ed0ff9700 1 -- 172.21.15.52:0/826236815 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 42) ==== 8394+0+0 (secure 0 0 0) 0x7f2ed8026460 con 0x5640a774cf60 2024-06-10T03:03:07.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.478+0000 7f2ef9f82b80 10 client.16815 fetch_fsmap finished waiting for FSMap version 42 2024-06-10T03:03:07.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.478+0000 7f2ef9f82b80 10 client.16815 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T03:03:07.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.478+0000 7f2ef9f82b80 20 client.16815 populate_metadata read hostname 'smithi052' 2024-06-10T03:03:07.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.478+0000 7f2ef9f82b80 1 --2- 172.21.15.52:0/826236815 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x5640a7760f90 0x5640a7763380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:07.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.478+0000 7f2ef9f82b80 4 client.16815 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T03:03:07.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.478+0000 7f2ef9f82b80 1 -- 172.21.15.52:0/826236815 --> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5640a76a5d50 con 0x5640a7760f90 2024-06-10T03:03:07.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.478+0000 7f2eed49b700 1 --2- 172.21.15.52:0/826236815 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x5640a7760f90 0x5640a7763380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:07.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.479+0000 7f2ed0ff9700 10 client.16815 ms_handle_connect on v2:172.21.15.188:6838/310444836 2024-06-10T03:03:07.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.479+0000 7f2eed49b700 1 --2- 172.21.15.52:0/826236815 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x5640a7760f90 0x5640a7763380 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2077909b118fc268 server_cookie=90baa6d3d9551548 in_seq=0 out_seq=0 2024-06-10T03:03:07.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.479+0000 7f2ed0ff9700 1 -- 172.21.15.52:0/826236815 <== mds.0 v2:172.21.15.188:6838/310444836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5640a76a5d50 con 0x5640a7760f90 2024-06-10T03:03:07.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.479+0000 7f2ed0ff9700 10 client.16815 handle_command_reply: tid=0 2024-06-10T03:03:07.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.479+0000 7f2ef9f82b80 1 client.16815 shutdown 2024-06-10T03:03:07.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.480+0000 7f2ef9f82b80 1 -- 172.21.15.52:0/826236815 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5640a774cf60 msgr2=0x5640a7750920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:03:07.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.480+0000 7f2ef9f82b80 1 --2- 172.21.15.52:0/826236815 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5640a774cf60 0x5640a7750920 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f2ed8004c90 tx=0x7f2ed8004cc0 comp rx=0 tx=0).stop 2024-06-10T03:03:07.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.480+0000 7f2ef9f82b80 1 -- 172.21.15.52:0/826236815 shutdown_connections 2024-06-10T03:03:07.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.480+0000 7f2ef9f82b80 1 --2- 172.21.15.52:0/826236815 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x5640a7760f90 0x5640a7763380 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:07.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.480+0000 7f2ef9f82b80 1 --2- 172.21.15.52:0/826236815 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5640a774d880 0x5640a7750e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:07.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.480+0000 7f2ef9f82b80 1 --2- 172.21.15.52:0/826236815 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5640a77477a0 0x5640a7753620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:07.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.480+0000 7f2ed0ff9700 0 client.16815 ms_handle_reset on v2:172.21.15.188:6838/310444836 2024-06-10T03:03:07.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.480+0000 7f2ef9f82b80 1 --2- 172.21.15.52:0/826236815 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5640a774cf60 0x5640a7750920 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:07.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.480+0000 7f2ef9f82b80 1 -- 172.21.15.52:0/826236815 >> 172.21.15.52:0/826236815 conn(0x5640a7674080 msgr2=0x5640a7741700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:03:07.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.480+0000 7f2ef9f82b80 1 -- 172.21.15.52:0/826236815 shutdown_connections 2024-06-10T03:03:07.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.480+0000 7f2ef9f82b80 1 -- 172.21.15.52:0/826236815 wait complete. 2024-06-10T03:03:07.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.480+0000 7f2ef9f82b80 20 client.16815 trim_cache size 0 max 16384 2024-06-10T03:03:07.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.501+0000 7f2ef9f82b80 1 Processor -- start 2024-06-10T03:03:07.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.501+0000 7f2ef9f82b80 1 -- start start 2024-06-10T03:03:07.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.501+0000 7f2ef9f82b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5640a77477a0 0x5640a774e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:07.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.501+0000 7f2ef9f82b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5640a774cf60 0x5640a7753160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:07.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.501+0000 7f2ef9f82b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5640a774d880 0x5640a774c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:07.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.501+0000 7f2ef9f82b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee81b0590 con 0x5640a774d880 2024-06-10T03:03:07.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.501+0000 7f2ef9f82b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee81b2220 con 0x5640a77477a0 2024-06-10T03:03:07.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.501+0000 7f2ef9f82b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee81b0cc0 con 0x5640a774cf60 2024-06-10T03:03:07.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.502+0000 7f2eecc9a700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5640a774cf60 0x5640a7753160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:07.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.502+0000 7f2eed49b700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5640a77477a0 0x5640a774e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:07.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.502+0000 7f2eecc9a700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5640a774cf60 0x5640a7753160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.52:35510/0 (socket says 172.21.15.52:35510) 2024-06-10T03:03:07.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.502+0000 7f2eecc9a700 1 -- 172.21.15.52:0/111385138 learned_addr learned my addr 172.21.15.52:0/111385138 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:03:07.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.502+0000 7f2eedc9c700 1 --2- 172.21.15.52:0/111385138 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5640a774d880 0x5640a774c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:07.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.502+0000 7f2eecc9a700 1 -- 172.21.15.52:0/111385138 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5640a77477a0 msgr2=0x5640a774e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:03:07.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.502+0000 7f2eecc9a700 1 --2- 172.21.15.52:0/111385138 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5640a77477a0 0x5640a774e5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:07.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.502+0000 7f2eecc9a700 1 -- 172.21.15.52:0/111385138 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5640a774d880 msgr2=0x5640a774c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:03:07.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.502+0000 7f2eecc9a700 1 --2- 172.21.15.52:0/111385138 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5640a774d880 0x5640a774c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:07.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.502+0000 7f2eecc9a700 1 -- 172.21.15.52:0/111385138 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5640a7742e10 con 0x5640a774cf60 2024-06-10T03:03:07.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.502+0000 7f2eed49b700 1 --2- 172.21.15.52:0/111385138 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5640a77477a0 0x5640a774e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T03:03:07.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.502+0000 7f2eecc9a700 1 --2- 172.21.15.52:0/111385138 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5640a774cf60 0x5640a7753160 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f2ed8003ad0 tx=0x7f2ed8003720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4b8415886b346704 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:03:07.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.502+0000 7f2ed2ffd700 1 -- 172.21.15.52:0/111385138 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ed8035050 con 0x5640a774cf60 2024-06-10T03:03:07.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.503+0000 7f2ed2ffd700 1 -- 172.21.15.52:0/111385138 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ed8003de0 con 0x5640a774cf60 2024-06-10T03:03:07.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.503+0000 7f2ed2ffd700 1 -- 172.21.15.52:0/111385138 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ed80055b0 con 0x5640a774cf60 2024-06-10T03:03:07.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.503+0000 7f2ef9f82b80 1 -- 172.21.15.52:0/111385138 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5640a774cf60 msgr2=0x5640a7753160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:03:07.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.503+0000 7f2ef9f82b80 1 --2- 172.21.15.52:0/111385138 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5640a774cf60 0x5640a7753160 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f2ed8003ad0 tx=0x7f2ed8003720 comp rx=0 tx=0).stop 2024-06-10T03:03:07.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.503+0000 7f2ef9f82b80 1 -- 172.21.15.52:0/111385138 shutdown_connections 2024-06-10T03:03:07.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.503+0000 7f2ef9f82b80 1 --2- 172.21.15.52:0/111385138 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5640a774d880 0x5640a774c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:07.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.503+0000 7f2ef9f82b80 1 --2- 172.21.15.52:0/111385138 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5640a77477a0 0x5640a774e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:07.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.503+0000 7f2ef9f82b80 1 --2- 172.21.15.52:0/111385138 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5640a774cf60 0x5640a7753160 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:07.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.503+0000 7f2ef9f82b80 1 -- 172.21.15.52:0/111385138 >> 172.21.15.52:0/111385138 conn(0x5640a7674080 msgr2=0x5640a774baa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:03:07.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.503+0000 7f2ef9f82b80 1 -- 172.21.15.52:0/111385138 shutdown_connections 2024-06-10T03:03:07.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.503+0000 7f2ef9f82b80 1 -- 172.21.15.52:0/111385138 wait complete. 2024-06-10T03:03:07.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.504+0000 7f2ef9f82b80 1 -- start start 2024-06-10T03:03:07.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.504+0000 7f2ef9f82b80 1 Processor -- start 2024-06-10T03:03:07.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.504+0000 7f2ef9f82b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5640a77477a0 0x5640a7759aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:07.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.504+0000 7f2ef9f82b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5640a774cf60 0x5640a7768010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:07.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.504+0000 7f2ef9f82b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5640a774d880 0x5640a7768550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:07.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.504+0000 7f2ef9f82b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee81b1b20 con 0x5640a774cf60 2024-06-10T03:03:07.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.504+0000 7f2ef9f82b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee81b1420 con 0x5640a77477a0 2024-06-10T03:03:07.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.504+0000 7f2ef9f82b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee81ae530 con 0x5640a774d880 2024-06-10T03:03:07.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.504+0000 7f2ef9f82b80 10 client.0 _ll_register_callbacks cb 0x5640a774b570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T03:03:07.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.504+0000 7f2eecc9a700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5640a774cf60 0x5640a7768010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:07.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.504+0000 7f2eedc9c700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5640a774d880 0x5640a7768550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:07.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.504+0000 7f2eecc9a700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5640a774cf60 0x5640a7768010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49462/0 (socket says 172.21.15.52:49462) 2024-06-10T03:03:07.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.504+0000 7f2eecc9a700 1 -- 172.21.15.52:0/1090392829 learned_addr learned my addr 172.21.15.52:0/1090392829 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:03:07.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.504+0000 7f2eed49b700 1 --2- 172.21.15.52:0/1090392829 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5640a77477a0 0x5640a7759aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:07.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.505+0000 7f2eedc9c700 1 -- 172.21.15.52:0/1090392829 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5640a77477a0 msgr2=0x5640a7759aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:03:07.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.505+0000 7f2eedc9c700 1 --2- 172.21.15.52:0/1090392829 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5640a77477a0 0x5640a7759aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:07.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.505+0000 7f2eedc9c700 1 -- 172.21.15.52:0/1090392829 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5640a774cf60 msgr2=0x5640a7768010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:03:07.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.505+0000 7f2eedc9c700 1 --2- 172.21.15.52:0/1090392829 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5640a774cf60 0x5640a7768010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:07.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.505+0000 7f2eedc9c700 1 -- 172.21.15.52:0/1090392829 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5640a7768b40 con 0x5640a774d880 2024-06-10T03:03:07.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.505+0000 7f2eecc9a700 1 --2- 172.21.15.52:0/1090392829 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5640a774cf60 0x5640a7768010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T03:03:07.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.505+0000 7f2eedc9c700 1 --2- 172.21.15.52:0/1090392829 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5640a774d880 0x5640a7768550 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f2ee0009fd0 tx=0x7f2ee0007710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=46f9806630d49ee2 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:03:07.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.505+0000 7f2ed0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-06-10T03:03:07.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.505+0000 7f2ed0ff9700 1 -- 172.21.15.52:0/1090392829 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ee002b060 con 0x5640a774d880 2024-06-10T03:03:07.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.505+0000 7f2ef9f82b80 1 -- 172.21.15.52:0/1090392829 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ed8003de0 con 0x5640a774d880 2024-06-10T03:03:07.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.505+0000 7f2ed0ff9700 1 -- 172.21.15.52:0/1090392829 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f2ee0007ac0 con 0x5640a774d880 2024-06-10T03:03:07.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.505+0000 7f2ed0ff9700 1 -- 172.21.15.52:0/1090392829 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f2ee0032050 con 0x5640a774d880 2024-06-10T03:03:07.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.505+0000 7f2ef9f82b80 10 client.26356 fetch_fsmap learned FSMap version 42 2024-06-10T03:03:07.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.505+0000 7f2ef9f82b80 1 -- 172.21.15.52:0/1090392829 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x5640a77cb700 con 0x5640a774d880 2024-06-10T03:03:07.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.506+0000 7f2ed0ff9700 1 -- 172.21.15.52:0/1090392829 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ed8003de0 con 0x5640a774d880 2024-06-10T03:03:07.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.506+0000 7f2ed0ff9700 1 -- 172.21.15.52:0/1090392829 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 42) ==== 8394+0+0 (secure 0 0 0) 0x7f2ee0025070 con 0x5640a774d880 2024-06-10T03:03:07.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.506+0000 7f2ef9f82b80 10 client.26356 fetch_fsmap finished waiting for FSMap version 42 2024-06-10T03:03:07.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.506+0000 7f2ef9f82b80 10 client.26356 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T03:03:07.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.506+0000 7f2ef9f82b80 20 client.26356 populate_metadata read hostname 'smithi052' 2024-06-10T03:03:07.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.506+0000 7f2ef9f82b80 1 --2- 172.21.15.52:0/1090392829 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x5640a77cbdf0 0x5640a77ce210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:07.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.506+0000 7f2ef9f82b80 4 client.26356 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-10T03:03:07.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.506+0000 7f2ef9f82b80 1 -- 172.21.15.52:0/1090392829 --> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5640a76a02d0 con 0x5640a77cbdf0 2024-06-10T03:03:07.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.506+0000 7f2eed49b700 1 --2- 172.21.15.52:0/1090392829 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x5640a77cbdf0 0x5640a77ce210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:07.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.506+0000 7f2ed0ff9700 10 client.26356 ms_handle_connect on v2:172.21.15.188:6838/310444836 2024-06-10T03:03:07.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.506+0000 7f2eed49b700 1 --2- 172.21.15.52:0/1090392829 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x5640a77cbdf0 0x5640a77ce210 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=34f980f4fd1d5f75 server_cookie=17b86217b57efcec in_seq=0 out_seq=0 2024-06-10T03:03:07.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.507+0000 7f2ed0ff9700 1 -- 172.21.15.52:0/1090392829 <== mds.0 v2:172.21.15.188:6838/310444836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5640a76a02d0 con 0x5640a77cbdf0 2024-06-10T03:03:07.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.507+0000 7f2ed0ff9700 10 client.26356 handle_command_reply: tid=0 2024-06-10T03:03:07.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.507+0000 7f2ef9f82b80 1 client.26356 shutdown 2024-06-10T03:03:07.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.507+0000 7f2ef9f82b80 1 -- 172.21.15.52:0/1090392829 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5640a774d880 msgr2=0x5640a7768550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:03:07.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.507+0000 7f2ef9f82b80 1 --2- 172.21.15.52:0/1090392829 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5640a774d880 0x5640a7768550 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f2ee0009fd0 tx=0x7f2ee0007710 comp rx=0 tx=0).stop 2024-06-10T03:03:07.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.507+0000 7f2ef9f82b80 1 -- 172.21.15.52:0/1090392829 shutdown_connections 2024-06-10T03:03:07.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.507+0000 7f2ef9f82b80 1 --2- 172.21.15.52:0/1090392829 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x5640a77cbdf0 0x5640a77ce210 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:07.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.507+0000 7f2ef9f82b80 1 --2- 172.21.15.52:0/1090392829 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5640a774d880 0x5640a7768550 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:07.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.507+0000 7f2ef9f82b80 1 --2- 172.21.15.52:0/1090392829 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5640a77477a0 0x5640a7759aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:07.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.507+0000 7f2ed0ff9700 0 client.26356 ms_handle_reset on v2:172.21.15.188:6838/310444836 2024-06-10T03:03:07.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.507+0000 7f2ef9f82b80 1 --2- 172.21.15.52:0/1090392829 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5640a774cf60 0x5640a7768010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:07.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.507+0000 7f2ef9f82b80 1 -- 172.21.15.52:0/1090392829 >> 172.21.15.52:0/1090392829 conn(0x5640a7674080 msgr2=0x5640a774e3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:03:07.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.507+0000 7f2ef9f82b80 1 -- 172.21.15.52:0/1090392829 shutdown_connections 2024-06-10T03:03:07.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.507+0000 7f2ef9f82b80 1 -- 172.21.15.52:0/1090392829 wait complete. 2024-06-10T03:03:07.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.507+0000 7f2ef9f82b80 20 client.26356 trim_cache size 0 max 16384 2024-06-10T03:03:07.512 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-10T03:03:07.512 INFO:teuthology.orchestra.run.smithi052.stdout: "status": "IDLE (waiting for more scrubs)", 2024-06-10T03:03:07.512 INFO:teuthology.orchestra.run.smithi052.stdout: "scrubs": { 2024-06-10T03:03:07.512 INFO:teuthology.orchestra.run.smithi052.stdout: "9ba0b5c7-4aa7-4870-a07e-1c62d90b40dc": { 2024-06-10T03:03:07.512 INFO:teuthology.orchestra.run.smithi052.stdout: "path": "/", 2024-06-10T03:03:07.513 INFO:teuthology.orchestra.run.smithi052.stdout: "tag": "9ba0b5c7-4aa7-4870-a07e-1c62d90b40dc", 2024-06-10T03:03:07.513 INFO:teuthology.orchestra.run.smithi052.stdout: "options": "recursive,force" 2024-06-10T03:03:07.513 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:03:07.513 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:03:07.513 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-10T03:03:07.513 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.507+0000 7f2eef6ff700 1 -- 172.21.15.52:0/1770543968 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f2ed40a2c20 msgr2=0x7f2ed40a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:03:07.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.507+0000 7f2eef6ff700 1 --2- 172.21.15.52:0/1770543968 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f2ed40a2c20 0x7f2ed40a50e0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f2ee81090a0 tx=0x7f2ed800b3c0 comp rx=0 tx=0).stop 2024-06-10T03:03:07.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.507+0000 7f2eef6ff700 1 -- 172.21.15.52:0/1770543968 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2ee81027a0 msgr2=0x7f2ee819adb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:03:07.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.507+0000 7f2eef6ff700 1 --2- 172.21.15.52:0/1770543968 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2ee81027a0 0x7f2ee819adb0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f2ee400d980 tx=0x7f2ee400dc90 comp rx=0 tx=0).stop 2024-06-10T03:03:07.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.508+0000 7f2eef6ff700 1 -- 172.21.15.52:0/1770543968 shutdown_connections 2024-06-10T03:03:07.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.508+0000 7f2eef6ff700 1 --2- 172.21.15.52:0/1770543968 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2ee8107870 0x7f2ee819f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:07.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.508+0000 7f2eef6ff700 1 --2- 172.21.15.52:0/1770543968 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f2ed40a2c20 0x7f2ed40a50e0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:07.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.508+0000 7f2eef6ff700 1 --2- 172.21.15.52:0/1770543968 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2ee81027a0 0x7f2ee819adb0 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:07.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.508+0000 7f2eef6ff700 1 --2- 172.21.15.52:0/1770543968 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2ee8103150 0x7f2ee819b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:07.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.508+0000 7f2eef6ff700 1 -- 172.21.15.52:0/1770543968 >> 172.21.15.52:0/1770543968 conn(0x7f2ee8079570 msgr2=0x7f2ee8100790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:03:07.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.508+0000 7f2eef6ff700 1 -- 172.21.15.52:0/1770543968 shutdown_connections 2024-06-10T03:03:07.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:07.508+0000 7f2eef6ff700 1 -- 172.21.15.52:0/1770543968 wait complete. 2024-06-10T03:03:07.954 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:03:07 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T03:03:07.977 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:03:07 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T03:03:08.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:03:07 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T03:03:08.733 INFO:tasks.cephfs.filesystem:scrub status for tag:9ba0b5c7-4aa7-4870-a07e-1c62d90b40dc - {'path': '/', 'tag': '9ba0b5c7-4aa7-4870-a07e-1c62d90b40dc', 'options': 'recursive,force'} 2024-06-10T03:03:08.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:03:08 smithi188 ceph-mon[36599]: pgmap v702: 129 pgs: 129 active+clean; 277 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 600 KiB/s rd, 376 KiB/s wr, 333 op/s 2024-06-10T03:03:08.964 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:03:08 smithi052 ceph-mon[32362]: pgmap v702: 129 pgs: 129 active+clean; 277 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 600 KiB/s rd, 376 KiB/s wr, 333 op/s 2024-06-10T03:03:08.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:08.985+0000 7f5fb3dc2700 1 -- 172.21.15.52:0/3613261546 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5fac107840 msgr2=0x7f5fac10f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:03:08.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:08.985+0000 7f5fb3dc2700 1 --2- 172.21.15.52:0/3613261546 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5fac107840 0x7f5fac10f3c0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f5fa400b0d0 tx=0x7f5fa400b3e0 comp rx=0 tx=0).stop 2024-06-10T03:03:08.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:08.985+0000 7f5fb3dc2700 1 -- 172.21.15.52:0/3613261546 shutdown_connections 2024-06-10T03:03:08.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:08.985+0000 7f5fb3dc2700 1 --2- 172.21.15.52:0/3613261546 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5fac107840 0x7f5fac10f3c0 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:08.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:08.985+0000 7f5fb3dc2700 1 --2- 172.21.15.52:0/3613261546 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5fac102740 0x7f5fac102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:08.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:08.985+0000 7f5fb3dc2700 1 --2- 172.21.15.52:0/3613261546 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5fac1030f0 0x7f5fac107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:08.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:08.985+0000 7f5fb3dc2700 1 -- 172.21.15.52:0/3613261546 >> 172.21.15.52:0/3613261546 conn(0x7f5fac0795f0 msgr2=0x7f5fac079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:03:08.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:08.985+0000 7f5fb3dc2700 1 -- 172.21.15.52:0/3613261546 shutdown_connections 2024-06-10T03:03:08.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:08.986+0000 7f5fb3dc2700 1 -- 172.21.15.52:0/3613261546 wait complete. 2024-06-10T03:03:08.989 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:08.986+0000 7f5fb3dc2700 1 Processor -- start 2024-06-10T03:03:08.989 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:08.986+0000 7f5fb3dc2700 1 -- start start 2024-06-10T03:03:08.989 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:08.986+0000 7f5fb3dc2700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5fac102740 0x7f5fac19e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:08.989 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:08.986+0000 7f5fb3dc2700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5fac1030f0 0x7f5fac19ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:08.989 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:08.986+0000 7f5fb3dc2700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5fac107840 0x7f5fac198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:08.990 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:08.986+0000 7f5fb3dc2700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f5fac1127d0 con 0x7f5fac102740 2024-06-10T03:03:08.990 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:08.986+0000 7f5fb3dc2700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f5fac112650 con 0x7f5fac107840 2024-06-10T03:03:08.990 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:08.986+0000 7f5fb3dc2700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f5fac112950 con 0x7f5fac1030f0 2024-06-10T03:03:08.990 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:08.987+0000 7f5fb1b5e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5fac102740 0x7f5fac19e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:08.990 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:08.987+0000 7f5fb1b5e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5fac102740 0x7f5fac19e7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49484/0 (socket says 172.21.15.52:49484) 2024-06-10T03:03:08.990 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:08.987+0000 7f5fb1b5e700 1 -- 172.21.15.52:0/705653107 learned_addr learned my addr 172.21.15.52:0/705653107 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:03:08.990 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:08.987+0000 7f5fb135d700 1 --2- 172.21.15.52:0/705653107 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5fac1030f0 0x7f5fac19ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:08.990 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:08.987+0000 7f5fb235f700 1 --2- 172.21.15.52:0/705653107 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5fac107840 0x7f5fac198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:08.990 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:08.987+0000 7f5fb1b5e700 1 -- 172.21.15.52:0/705653107 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5fac1030f0 msgr2=0x7f5fac19ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:03:08.990 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:08.987+0000 7f5fb1b5e700 1 --2- 172.21.15.52:0/705653107 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5fac1030f0 0x7f5fac19ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:08.990 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:08.987+0000 7f5fb1b5e700 1 -- 172.21.15.52:0/705653107 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5fac107840 msgr2=0x7f5fac198990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:03:08.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:08.987+0000 7f5fb1b5e700 1 --2- 172.21.15.52:0/705653107 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5fac107840 0x7f5fac198990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:08.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:08.987+0000 7f5fb1b5e700 1 -- 172.21.15.52:0/705653107 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fac1990f0 con 0x7f5fac102740 2024-06-10T03:03:08.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:08.987+0000 7f5fb1b5e700 1 --2- 172.21.15.52:0/705653107 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5fac102740 0x7f5fac19e7d0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f5fa800c920 tx=0x7f5fa800cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:03:08.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:08.987+0000 7f5fa2ffd700 1 -- 172.21.15.52:0/705653107 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fa8004a30 con 0x7f5fac102740 2024-06-10T03:03:08.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:08.987+0000 7f5fa2ffd700 1 -- 172.21.15.52:0/705653107 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5fa8005500 con 0x7f5fac102740 2024-06-10T03:03:08.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:08.987+0000 7f5fb3dc2700 1 -- 172.21.15.52:0/705653107 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fac1993e0 con 0x7f5fac102740 2024-06-10T03:03:08.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:08.987+0000 7f5fa2ffd700 1 -- 172.21.15.52:0/705653107 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fa8007d50 con 0x7f5fac102740 2024-06-10T03:03:08.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:08.987+0000 7f5fb3dc2700 1 -- 172.21.15.52:0/705653107 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fac1a5720 con 0x7f5fac102740 2024-06-10T03:03:08.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:08.990+0000 7f5fa2ffd700 1 -- 172.21.15.52:0/705653107 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f5fa8004d60 con 0x7f5fac102740 2024-06-10T03:03:08.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:08.990+0000 7f5fa2ffd700 1 --2- 172.21.15.52:0/705653107 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f5f980a2ca0 0x7f5f980a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:08.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:08.990+0000 7f5fb135d700 1 --2- 172.21.15.52:0/705653107 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f5f980a2ca0 0x7f5f980a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:08.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:08.990+0000 7f5fa2ffd700 1 -- 172.21.15.52:0/705653107 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f5fa80cf270 con 0x7f5fac102740 2024-06-10T03:03:08.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:08.991+0000 7f5fb135d700 1 --2- 172.21.15.52:0/705653107 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f5f980a2ca0 0x7f5f980a5160 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f5fac108c60 tx=0x7f5f9c009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:03:08.994 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:08.991+0000 7f5fbe645b80 1 Processor -- start 2024-06-10T03:03:08.994 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:08.992+0000 7f5fbe645b80 1 -- start start 2024-06-10T03:03:08.994 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:08.992+0000 7f5fbe645b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555d99876910 0x555d99876cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:08.994 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:08.992+0000 7f5fbe645b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555d99877230 0x555d99870b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:08.994 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:08.992+0000 7f5fbe645b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555d99871060 0x555d99871480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:08.994 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:08.992+0000 7f5fbe645b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x555d9975f060 con 0x555d99877230 2024-06-10T03:03:08.994 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:08.992+0000 7f5fbe645b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x555d997bd950 con 0x555d99871060 2024-06-10T03:03:08.995 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:08.992+0000 7f5fbe645b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x555d99725560 con 0x555d99876910 2024-06-10T03:03:08.995 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:08.992+0000 7f5fb135d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555d99877230 0x555d99870b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:08.995 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:08.992+0000 7f5fb135d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555d99877230 0x555d99870b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49486/0 (socket says 172.21.15.52:49486) 2024-06-10T03:03:08.995 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:08.992+0000 7f5fb135d700 1 -- 172.21.15.52:0/696635721 learned_addr learned my addr 172.21.15.52:0/696635721 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:03:08.995 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:08.992+0000 7f5fb1b5e700 1 --2- 172.21.15.52:0/696635721 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555d99876910 0x555d99876cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:08.995 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:08.992+0000 7f5fb235f700 1 --2- 172.21.15.52:0/696635721 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555d99871060 0x555d99871480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:08.995 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:08.992+0000 7f5fb135d700 1 -- 172.21.15.52:0/696635721 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555d99876910 msgr2=0x555d99876cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:03:08.995 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:08.992+0000 7f5fb135d700 1 --2- 172.21.15.52:0/696635721 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555d99876910 0x555d99876cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:08.995 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:08.992+0000 7f5fb135d700 1 -- 172.21.15.52:0/696635721 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555d99871060 msgr2=0x555d99871480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:03:08.995 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:08.992+0000 7f5fb135d700 1 --2- 172.21.15.52:0/696635721 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555d99871060 0x555d99871480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:08.995 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:08.992+0000 7f5fb135d700 1 -- 172.21.15.52:0/696635721 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555d9970c9d0 con 0x555d99877230 2024-06-10T03:03:08.995 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:08.992+0000 7f5fb235f700 1 --2- 172.21.15.52:0/696635721 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555d99871060 0x555d99871480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T03:03:08.995 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:08.993+0000 7f5fb135d700 1 --2- 172.21.15.52:0/696635721 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555d99877230 0x555d99870b20 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f5f9c009fd0 tx=0x7f5f9c011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a05c843db2908c7 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:03:08.996 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:08.993+0000 7f5f97fff700 1 -- 172.21.15.52:0/696635721 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f9c01b040 con 0x555d99877230 2024-06-10T03:03:08.996 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:08.993+0000 7f5f97fff700 1 -- 172.21.15.52:0/696635721 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f9c024070 con 0x555d99877230 2024-06-10T03:03:08.996 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:08.993+0000 7f5f97fff700 1 -- 172.21.15.52:0/696635721 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f9c013740 con 0x555d99877230 2024-06-10T03:03:08.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:08.993+0000 7f5fbe645b80 1 -- 172.21.15.52:0/696635721 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555d99877230 msgr2=0x555d99870b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:03:08.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:08.993+0000 7f5fbe645b80 1 --2- 172.21.15.52:0/696635721 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555d99877230 0x555d99870b20 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f5f9c009fd0 tx=0x7f5f9c011000 comp rx=0 tx=0).stop 2024-06-10T03:03:08.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:08.994+0000 7f5fbe645b80 1 -- 172.21.15.52:0/696635721 shutdown_connections 2024-06-10T03:03:08.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:08.994+0000 7f5fbe645b80 1 --2- 172.21.15.52:0/696635721 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555d99871060 0x555d99871480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:08.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:08.994+0000 7f5fbe645b80 1 --2- 172.21.15.52:0/696635721 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555d99876910 0x555d99876cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:08.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:08.994+0000 7f5fbe645b80 1 --2- 172.21.15.52:0/696635721 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555d99877230 0x555d99870b20 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:09.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:08.994+0000 7f5fbe645b80 1 -- 172.21.15.52:0/696635721 >> 172.21.15.52:0/696635721 conn(0x555d99799180 msgr2=0x555d99867a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:03:09.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:08.994+0000 7f5fbe645b80 1 -- 172.21.15.52:0/696635721 shutdown_connections 2024-06-10T03:03:09.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:08.994+0000 7f5fbe645b80 1 -- 172.21.15.52:0/696635721 wait complete. 2024-06-10T03:03:09.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:08.994+0000 7f5fbe645b80 1 -- start start 2024-06-10T03:03:09.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:08.994+0000 7f5fbe645b80 1 Processor -- start 2024-06-10T03:03:09.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:08.995+0000 7f5fbe645b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555d99871060 0x555d9987cf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:09.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:08.995+0000 7f5fbe645b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555d99876910 0x555d9987a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:09.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:08.995+0000 7f5fbe645b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555d99877230 0x555d9987a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:09.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:08.995+0000 7f5fbe645b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x555d9975e0c0 con 0x555d99877230 2024-06-10T03:03:09.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:08.995+0000 7f5fbe645b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x555d997b9910 con 0x555d99876910 2024-06-10T03:03:09.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:08.995+0000 7f5fbe645b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x555d9987dc10 con 0x555d99871060 2024-06-10T03:03:09.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:08.995+0000 7f5fbe645b80 10 client.0 _ll_register_callbacks cb 0x555d997536b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T03:03:09.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:08.995+0000 7f5fb235f700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555d99877230 0x555d9987a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:09.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:08.995+0000 7f5fb235f700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555d99877230 0x555d9987a7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49496/0 (socket says 172.21.15.52:49496) 2024-06-10T03:03:09.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:08.995+0000 7f5fb235f700 1 -- 172.21.15.52:0/3571628140 learned_addr learned my addr 172.21.15.52:0/3571628140 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:03:09.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:08.995+0000 7f5fb135d700 1 --2- 172.21.15.52:0/3571628140 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555d99876910 0x555d9987a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:09.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:08.995+0000 7f5fb1b5e700 1 --2- 172.21.15.52:0/3571628140 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555d99871060 0x555d9987cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:09.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:08.995+0000 7f5fb235f700 1 -- 172.21.15.52:0/3571628140 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555d99871060 msgr2=0x555d9987cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:03:09.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:08.995+0000 7f5fb235f700 1 --2- 172.21.15.52:0/3571628140 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555d99871060 0x555d9987cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:09.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:08.995+0000 7f5fb235f700 1 -- 172.21.15.52:0/3571628140 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555d99876910 msgr2=0x555d9987a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:03:09.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:08.995+0000 7f5fb235f700 1 --2- 172.21.15.52:0/3571628140 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555d99876910 0x555d9987a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:09.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:08.995+0000 7f5fb235f700 1 -- 172.21.15.52:0/3571628140 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555d99783850 con 0x555d99877230 2024-06-10T03:03:09.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:08.995+0000 7f5fb1b5e700 1 --2- 172.21.15.52:0/3571628140 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555d99871060 0x555d9987cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T03:03:09.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:08.995+0000 7f5fb135d700 1 --2- 172.21.15.52:0/3571628140 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555d99876910 0x555d9987a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T03:03:09.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:08.995+0000 7f5fb235f700 1 --2- 172.21.15.52:0/3571628140 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555d99877230 0x555d9987a7b0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f5fa4004ce0 tx=0x7f5fa4004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e37acd05987587a server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:03:09.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:08.995+0000 7f5f957fa700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-10T03:03:09.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:08.995+0000 7f5f957fa700 1 -- 172.21.15.52:0/3571628140 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fa40076e0 con 0x555d99877230 2024-06-10T03:03:09.002 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:08.995+0000 7f5f957fa700 1 -- 172.21.15.52:0/3571628140 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f5fa4007880 con 0x555d99877230 2024-06-10T03:03:09.002 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:08.995+0000 7f5fbe645b80 1 -- 172.21.15.52:0/3571628140 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f9c024070 con 0x555d99877230 2024-06-10T03:03:09.002 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:08.996+0000 7f5f957fa700 1 -- 172.21.15.52:0/3571628140 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f5fa4007ac0 con 0x555d99877230 2024-06-10T03:03:09.002 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:08.996+0000 7f5fbe645b80 10 client.16833 fetch_fsmap learned FSMap version 42 2024-06-10T03:03:09.002 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:08.996+0000 7f5fbe645b80 1 -- 172.21.15.52:0/3571628140 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x555d99888370 con 0x555d99877230 2024-06-10T03:03:09.002 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:08.996+0000 7f5f957fa700 1 -- 172.21.15.52:0/3571628140 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f9c024070 con 0x555d99877230 2024-06-10T03:03:09.002 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:08.996+0000 7f5f957fa700 1 -- 172.21.15.52:0/3571628140 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 42) ==== 8394+0+0 (secure 0 0 0) 0x7f5fa4011070 con 0x555d99877230 2024-06-10T03:03:09.002 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:08.996+0000 7f5fbe645b80 10 client.16833 fetch_fsmap finished waiting for FSMap version 42 2024-06-10T03:03:09.002 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:08.996+0000 7f5fbe645b80 10 client.16833 resolve_mds: validated gid 24484 aka daemon mds.i 2024-06-10T03:03:09.002 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:08.996+0000 7f5fbe645b80 20 client.16833 populate_metadata read hostname 'smithi052' 2024-06-10T03:03:09.002 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:08.996+0000 7f5fbe645b80 1 --2- 172.21.15.52:0/3571628140 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x555d9988aa50 0x555d9988ce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:09.002 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:08.996+0000 7f5fbe645b80 4 client.16833 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T03:03:09.002 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:08.996+0000 7f5fbe645b80 1 -- 172.21.15.52:0/3571628140 --> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555d997cae50 con 0x555d9988aa50 2024-06-10T03:03:09.003 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:08.996+0000 7f5fb1b5e700 1 --2- 172.21.15.52:0/3571628140 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x555d9988aa50 0x555d9988ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:09.003 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:08.997+0000 7f5fb1b5e700 1 --2- 172.21.15.52:0/3571628140 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x555d9988aa50 0x555d9988ce40 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f8d48a3f3032463b server_cookie=dea36ff47d8bef09 in_seq=0 out_seq=0 2024-06-10T03:03:09.003 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:08.997+0000 7f5f957fa700 10 client.16833 ms_handle_connect on v2:172.21.15.188:6838/310444836 2024-06-10T03:03:09.003 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:08.998+0000 7f5f957fa700 1 -- 172.21.15.52:0/3571628140 <== mds.0 v2:172.21.15.188:6838/310444836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x555d997cae50 con 0x555d9988aa50 2024-06-10T03:03:09.003 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:08.998+0000 7f5f957fa700 10 client.16833 handle_command_reply: tid=0 2024-06-10T03:03:09.003 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:08.998+0000 7f5fbe645b80 1 client.16833 shutdown 2024-06-10T03:03:09.003 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:08.998+0000 7f5fbe645b80 1 -- 172.21.15.52:0/3571628140 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555d99877230 msgr2=0x555d9987a7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:03:09.003 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:08.998+0000 7f5fbe645b80 1 --2- 172.21.15.52:0/3571628140 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555d99877230 0x555d9987a7b0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f5fa4004ce0 tx=0x7f5fa4004d10 comp rx=0 tx=0).stop 2024-06-10T03:03:09.003 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:08.998+0000 7f5fbe645b80 1 -- 172.21.15.52:0/3571628140 shutdown_connections 2024-06-10T03:03:09.003 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:08.998+0000 7f5fbe645b80 1 --2- 172.21.15.52:0/3571628140 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x555d9988aa50 0x555d9988ce40 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:09.003 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:08.998+0000 7f5fbe645b80 1 --2- 172.21.15.52:0/3571628140 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555d99877230 0x555d9987a7b0 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:09.003 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:08.998+0000 7f5fbe645b80 1 --2- 172.21.15.52:0/3571628140 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555d99871060 0x555d9987cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:09.003 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:08.998+0000 7f5f957fa700 0 client.16833 ms_handle_reset on v2:172.21.15.188:6838/310444836 2024-06-10T03:03:09.003 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:08.998+0000 7f5fbe645b80 1 --2- 172.21.15.52:0/3571628140 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555d99876910 0x555d9987a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:09.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:08.998+0000 7f5fbe645b80 1 -- 172.21.15.52:0/3571628140 >> 172.21.15.52:0/3571628140 conn(0x555d99799180 msgr2=0x555d99866bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:03:09.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:08.998+0000 7f5fbe645b80 1 -- 172.21.15.52:0/3571628140 shutdown_connections 2024-06-10T03:03:09.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:08.998+0000 7f5fbe645b80 1 -- 172.21.15.52:0/3571628140 wait complete. 2024-06-10T03:03:09.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:08.998+0000 7f5fbe645b80 20 client.16833 trim_cache size 0 max 16384 2024-06-10T03:03:09.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:09.019+0000 7f5fbe645b80 1 Processor -- start 2024-06-10T03:03:09.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:09.019+0000 7f5fbe645b80 1 -- start start 2024-06-10T03:03:09.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:09.019+0000 7f5fbe645b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555d99871060 0x555d9987cab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:09.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:09.019+0000 7f5fbe645b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555d99876910 0x555d99875ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:09.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:09.019+0000 7f5fbe645b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555d99877230 0x555d99876020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:09.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:09.019+0000 7f5fbe645b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f5fac1ac150 con 0x555d99871060 2024-06-10T03:03:09.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:09.019+0000 7f5fbe645b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f5fac1adde0 con 0x555d99876910 2024-06-10T03:03:09.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:09.019+0000 7f5fbe645b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f5fac1ac880 con 0x555d99877230 2024-06-10T03:03:09.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:09.019+0000 7f5fb1b5e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555d99871060 0x555d9987cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:09.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:09.019+0000 7f5fb1b5e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555d99871060 0x555d9987cab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49502/0 (socket says 172.21.15.52:49502) 2024-06-10T03:03:09.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:09.019+0000 7f5fb1b5e700 1 -- 172.21.15.52:0/3904463549 learned_addr learned my addr 172.21.15.52:0/3904463549 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:03:09.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:09.019+0000 7f5fb1b5e700 1 -- 172.21.15.52:0/3904463549 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555d99877230 msgr2=0x555d99876020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:03:09.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:09.019+0000 7f5fb1b5e700 1 --2- 172.21.15.52:0/3904463549 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555d99877230 0x555d99876020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:09.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:09.019+0000 7f5fb1b5e700 1 -- 172.21.15.52:0/3904463549 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555d99876910 msgr2=0x555d99875ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:03:09.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:09.019+0000 7f5fb1b5e700 1 --2- 172.21.15.52:0/3904463549 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555d99876910 0x555d99875ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:09.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:09.019+0000 7f5fb1b5e700 1 -- 172.21.15.52:0/3904463549 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555d9985f4a0 con 0x555d99871060 2024-06-10T03:03:09.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:09.019+0000 7f5fb1b5e700 1 --2- 172.21.15.52:0/3904463549 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555d99871060 0x555d9987cab0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f5fa80cc960 tx=0x7f5fa80cc830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3e49967da9703b8 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:03:09.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:09.020+0000 7f5f977fe700 1 -- 172.21.15.52:0/3904463549 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fa801d040 con 0x555d99871060 2024-06-10T03:03:09.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:09.020+0000 7f5f977fe700 1 -- 172.21.15.52:0/3904463549 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5fa809fa70 con 0x555d99871060 2024-06-10T03:03:09.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:09.020+0000 7f5f977fe700 1 -- 172.21.15.52:0/3904463549 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fa80999c0 con 0x555d99871060 2024-06-10T03:03:09.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:09.020+0000 7f5fbe645b80 1 -- 172.21.15.52:0/3904463549 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555d99871060 msgr2=0x555d9987cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:03:09.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:09.020+0000 7f5fbe645b80 1 --2- 172.21.15.52:0/3904463549 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555d99871060 0x555d9987cab0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f5fa80cc960 tx=0x7f5fa80cc830 comp rx=0 tx=0).stop 2024-06-10T03:03:09.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:09.020+0000 7f5fbe645b80 1 -- 172.21.15.52:0/3904463549 shutdown_connections 2024-06-10T03:03:09.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:09.020+0000 7f5fbe645b80 1 --2- 172.21.15.52:0/3904463549 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555d99877230 0x555d99876020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:09.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:09.020+0000 7f5fbe645b80 1 --2- 172.21.15.52:0/3904463549 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555d99871060 0x555d9987cab0 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:09.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:09.020+0000 7f5fbe645b80 1 --2- 172.21.15.52:0/3904463549 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555d99876910 0x555d99875ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:09.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:09.020+0000 7f5fbe645b80 1 -- 172.21.15.52:0/3904463549 >> 172.21.15.52:0/3904463549 conn(0x555d99799180 msgr2=0x555d998753d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:03:09.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:09.020+0000 7f5fbe645b80 1 -- 172.21.15.52:0/3904463549 shutdown_connections 2024-06-10T03:03:09.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:09.020+0000 7f5fbe645b80 1 -- 172.21.15.52:0/3904463549 wait complete. 2024-06-10T03:03:09.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:09.021+0000 7f5fbe645b80 1 -- start start 2024-06-10T03:03:09.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:09.021+0000 7f5fbe645b80 1 Processor -- start 2024-06-10T03:03:09.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:09.021+0000 7f5fbe645b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555d99871060 0x555d998832b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:09.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:09.021+0000 7f5fbe645b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555d99876910 0x555d998837f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:09.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:09.021+0000 7f5fbe645b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555d99877230 0x555d99880080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:09.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:09.021+0000 7f5fbe645b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f5fac1ad6e0 con 0x555d99876910 2024-06-10T03:03:09.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:09.021+0000 7f5fbe645b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f5fac1acfe0 con 0x555d99871060 2024-06-10T03:03:09.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:09.021+0000 7f5fbe645b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f5fac1aa0f0 con 0x555d99877230 2024-06-10T03:03:09.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:09.021+0000 7f5fbe645b80 10 client.0 _ll_register_callbacks cb 0x555d997536b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T03:03:09.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:09.021+0000 7f5fb135d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555d99876910 0x555d998837f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:09.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:09.021+0000 7f5fb135d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555d99876910 0x555d998837f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49512/0 (socket says 172.21.15.52:49512) 2024-06-10T03:03:09.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:09.021+0000 7f5fb135d700 1 -- 172.21.15.52:0/4023619936 learned_addr learned my addr 172.21.15.52:0/4023619936 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:03:09.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:09.021+0000 7f5fb235f700 1 --2- 172.21.15.52:0/4023619936 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555d99877230 0x555d99880080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:09.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:09.021+0000 7f5fb1b5e700 1 --2- 172.21.15.52:0/4023619936 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555d99871060 0x555d998832b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:09.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:09.022+0000 7f5fb135d700 1 -- 172.21.15.52:0/4023619936 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555d99877230 msgr2=0x555d99880080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:03:09.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:09.022+0000 7f5fb135d700 1 --2- 172.21.15.52:0/4023619936 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555d99877230 0x555d99880080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:09.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:09.022+0000 7f5fb135d700 1 -- 172.21.15.52:0/4023619936 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555d99871060 msgr2=0x555d998832b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:03:09.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:09.022+0000 7f5fb135d700 1 --2- 172.21.15.52:0/4023619936 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555d99871060 0x555d998832b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:09.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:09.022+0000 7f5fb135d700 1 -- 172.21.15.52:0/4023619936 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555d998808c0 con 0x555d99876910 2024-06-10T03:03:09.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:09.022+0000 7f5fb235f700 1 --2- 172.21.15.52:0/4023619936 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555d99877230 0x555d99880080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T03:03:09.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:09.022+0000 7f5fb135d700 1 --2- 172.21.15.52:0/4023619936 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555d99876910 0x555d998837f0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f5f9c009fd0 tx=0x7f5f9c015d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2e934220d1a4321 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:03:09.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:09.022+0000 7f5f957fa700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-10T03:03:09.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:09.022+0000 7f5f957fa700 1 -- 172.21.15.52:0/4023619936 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f9c013860 con 0x555d99876910 2024-06-10T03:03:09.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:09.022+0000 7f5f957fa700 1 -- 172.21.15.52:0/4023619936 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f5f9c013a00 con 0x555d99876910 2024-06-10T03:03:09.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:09.022+0000 7f5fbe645b80 1 -- 172.21.15.52:0/4023619936 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5fa809fa70 con 0x555d99876910 2024-06-10T03:03:09.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:09.022+0000 7f5f957fa700 1 -- 172.21.15.52:0/4023619936 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f5f9c013c20 con 0x555d99876910 2024-06-10T03:03:09.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:09.022+0000 7f5fbe645b80 10 client.16839 fetch_fsmap learned FSMap version 42 2024-06-10T03:03:09.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:09.022+0000 7f5fbe645b80 1 -- 172.21.15.52:0/4023619936 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x555d998f63e0 con 0x555d99876910 2024-06-10T03:03:09.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:09.022+0000 7f5f957fa700 1 -- 172.21.15.52:0/4023619936 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5fa809fa70 con 0x555d99876910 2024-06-10T03:03:09.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:09.023+0000 7f5f957fa700 1 -- 172.21.15.52:0/4023619936 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 42) ==== 8394+0+0 (secure 0 0 0) 0x7f5f9c01e030 con 0x555d99876910 2024-06-10T03:03:09.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:09.023+0000 7f5fbe645b80 10 client.16839 fetch_fsmap finished waiting for FSMap version 42 2024-06-10T03:03:09.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:09.023+0000 7f5fbe645b80 10 client.16839 resolve_mds: validated gid 24484 aka daemon mds.i 2024-06-10T03:03:09.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:09.023+0000 7f5fbe645b80 20 client.16839 populate_metadata read hostname 'smithi052' 2024-06-10T03:03:09.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:09.023+0000 7f5fbe645b80 1 --2- 172.21.15.52:0/4023619936 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x555d998f6850 0x555d998f8c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:09.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:09.023+0000 7f5fbe645b80 4 client.16839 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "6a4162d1", "release": true, "await": true}] 2024-06-10T03:03:09.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:09.023+0000 7f5fbe645b80 1 -- 172.21.15.52:0/4023619936 --> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] -- command(tid 0: {"prefix": "quiesce db", "set_id": "6a4162d1", "release": true, "await": true}) -- 0x555d997c53d0 con 0x555d998f6850 2024-06-10T03:03:09.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:09.023+0000 7f5fb1b5e700 1 --2- 172.21.15.52:0/4023619936 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x555d998f6850 0x555d998f8c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:09.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:09.023+0000 7f5fb1b5e700 1 --2- 172.21.15.52:0/4023619936 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x555d998f6850 0x555d998f8c70 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb89675005989aae server_cookie=cb5a1ac263c0a4cd in_seq=0 out_seq=0 2024-06-10T03:03:09.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:09.023+0000 7f5f957fa700 10 client.16839 ms_handle_connect on v2:172.21.15.188:6838/310444836 2024-06-10T03:03:09.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:03:08 smithi120 ceph-mon[36634]: pgmap v702: 129 pgs: 129 active+clean; 277 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 600 KiB/s rd, 376 KiB/s wr, 333 op/s 2024-06-10T03:03:09.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:09.217+0000 7f5f957fa700 1 -- 172.21.15.52:0/4023619936 <== mds.0 v2:172.21.15.188:6838/310444836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x555d997c53d0 con 0x555d998f6850 2024-06-10T03:03:09.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:09.217+0000 7f5f957fa700 10 client.16839 handle_command_reply: tid=0 2024-06-10T03:03:09.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:09.217+0000 7f5fbe645b80 1 client.16839 shutdown 2024-06-10T03:03:09.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:09.217+0000 7f5fbe645b80 1 -- 172.21.15.52:0/4023619936 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555d99876910 msgr2=0x555d998837f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:03:09.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:09.217+0000 7f5fbe645b80 1 --2- 172.21.15.52:0/4023619936 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555d99876910 0x555d998837f0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f5f9c009fd0 tx=0x7f5f9c015d20 comp rx=0 tx=0).stop 2024-06-10T03:03:09.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:09.217+0000 7f5fbe645b80 1 -- 172.21.15.52:0/4023619936 shutdown_connections 2024-06-10T03:03:09.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:09.217+0000 7f5fbe645b80 1 --2- 172.21.15.52:0/4023619936 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x555d998f6850 0x555d998f8c70 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:09.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:09.217+0000 7f5fbe645b80 1 --2- 172.21.15.52:0/4023619936 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555d99877230 0x555d99880080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:09.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:09.217+0000 7f5fbe645b80 1 --2- 172.21.15.52:0/4023619936 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555d99871060 0x555d998832b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:09.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:09.217+0000 7f5f957fa700 0 client.16839 ms_handle_reset on v2:172.21.15.188:6838/310444836 2024-06-10T03:03:09.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:09.217+0000 7f5fbe645b80 1 --2- 172.21.15.52:0/4023619936 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555d99876910 0x555d998837f0 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:09.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:09.217+0000 7f5fbe645b80 1 -- 172.21.15.52:0/4023619936 >> 172.21.15.52:0/4023619936 conn(0x555d99799180 msgr2=0x555d99875c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:03:09.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:09.217+0000 7f5fbe645b80 1 -- 172.21.15.52:0/4023619936 shutdown_connections 2024-06-10T03:03:09.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:09.217+0000 7f5fbe645b80 1 -- 172.21.15.52:0/4023619936 wait complete. 2024-06-10T03:03:09.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:09.217+0000 7f5fbe645b80 20 client.16839 trim_cache size 0 max 16384 2024-06-10T03:03:09.220 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-10T03:03:09.220 INFO:teuthology.orchestra.run.smithi052.stdout: "epoch": 42, 2024-06-10T03:03:09.220 INFO:teuthology.orchestra.run.smithi052.stdout: "leader": 24484, 2024-06-10T03:03:09.220 INFO:teuthology.orchestra.run.smithi052.stdout: "set_version": 24, 2024-06-10T03:03:09.220 INFO:teuthology.orchestra.run.smithi052.stdout: "sets": { 2024-06-10T03:03:09.220 INFO:teuthology.orchestra.run.smithi052.stdout: "6a4162d1": { 2024-06-10T03:03:09.220 INFO:teuthology.orchestra.run.smithi052.stdout: "version": 24, 2024-06-10T03:03:09.220 INFO:teuthology.orchestra.run.smithi052.stdout: "age_ref": 0.0, 2024-06-10T03:03:09.221 INFO:teuthology.orchestra.run.smithi052.stdout: "state": { 2024-06-10T03:03:09.221 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "RELEASED", 2024-06-10T03:03:09.221 INFO:teuthology.orchestra.run.smithi052.stdout: "age": 0.0 2024-06-10T03:03:09.221 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:09.221 INFO:teuthology.orchestra.run.smithi052.stdout: "timeout": 90.0, 2024-06-10T03:03:09.221 INFO:teuthology.orchestra.run.smithi052.stdout: "expiration": 136.0, 2024-06-10T03:03:09.222 INFO:teuthology.orchestra.run.smithi052.stdout: "members": { 2024-06-10T03:03:09.222 INFO:teuthology.orchestra.run.smithi052.stdout: "file:/": { 2024-06-10T03:03:09.222 INFO:teuthology.orchestra.run.smithi052.stdout: "excluded": false, 2024-06-10T03:03:09.222 INFO:teuthology.orchestra.run.smithi052.stdout: "state": { 2024-06-10T03:03:09.222 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "RELEASED", 2024-06-10T03:03:09.222 INFO:teuthology.orchestra.run.smithi052.stdout: "age": 0.0 2024-06-10T03:03:09.222 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:03:09.222 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:03:09.222 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:03:09.222 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:03:09.222 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:03:09.223 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-10T03:03:09.223 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:09.217+0000 7f5fb3dc2700 1 -- 172.21.15.52:0/705653107 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f5f980a2ca0 msgr2=0x7f5f980a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:03:09.223 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:09.217+0000 7f5fb3dc2700 1 --2- 172.21.15.52:0/705653107 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f5f980a2ca0 0x7f5f980a5160 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f5fac108c60 tx=0x7f5f9c009190 comp rx=0 tx=0).stop 2024-06-10T03:03:09.223 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:09.217+0000 7f5fb3dc2700 1 -- 172.21.15.52:0/705653107 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5fac102740 msgr2=0x7f5fac19e7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:03:09.223 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:09.217+0000 7f5fb3dc2700 1 --2- 172.21.15.52:0/705653107 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5fac102740 0x7f5fac19e7d0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f5fa800c920 tx=0x7f5fa800cc30 comp rx=0 tx=0).stop 2024-06-10T03:03:09.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:09.218+0000 7f5fb3dc2700 1 -- 172.21.15.52:0/705653107 shutdown_connections 2024-06-10T03:03:09.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:09.218+0000 7f5fb3dc2700 1 --2- 172.21.15.52:0/705653107 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5fac107840 0x7f5fac198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:09.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:09.218+0000 7f5fb3dc2700 1 --2- 172.21.15.52:0/705653107 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f5f980a2ca0 0x7f5f980a5160 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:09.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:09.218+0000 7f5fb3dc2700 1 --2- 172.21.15.52:0/705653107 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5fac102740 0x7f5fac19e7d0 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:09.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:09.218+0000 7f5fb3dc2700 1 --2- 172.21.15.52:0/705653107 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5fac1030f0 0x7f5fac19ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:09.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:09.218+0000 7f5fb3dc2700 1 -- 172.21.15.52:0/705653107 >> 172.21.15.52:0/705653107 conn(0x7f5fac0795f0 msgr2=0x7f5fac1004f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:03:09.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:09.218+0000 7f5fb3dc2700 1 -- 172.21.15.52:0/705653107 shutdown_connections 2024-06-10T03:03:09.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:09.218+0000 7f5fb3dc2700 1 -- 172.21.15.52:0/705653107 wait complete. 2024-06-10T03:03:10.071 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '6a4162d1', total seconds elapsed: 26.2 2024-06-10T03:03:10.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:03:10 smithi188 ceph-mon[36599]: pgmap v703: 129 pgs: 129 active+clean; 277 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 485 KiB/s rd, 306 KiB/s wr, 255 op/s 2024-06-10T03:03:10.954 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:03:10 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T03:03:10.954 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:03:10 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T03:03:10.977 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:03:10 smithi052 ceph-mon[32362]: pgmap v703: 129 pgs: 129 active+clean; 277 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 485 KiB/s rd, 306 KiB/s wr, 255 op/s 2024-06-10T03:03:10.977 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:03:10 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T03:03:10.977 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:03:10 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T03:03:11.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:03:10 smithi120 ceph-mon[36634]: pgmap v703: 129 pgs: 129 active+clean; 277 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 485 KiB/s rd, 306 KiB/s wr, 255 op/s 2024-06-10T03:03:11.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:03:10 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T03:03:11.139 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:03:10 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T03:03:11.954 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:03:11 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T03:03:11.954 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:03:11 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T03:03:11.954 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:03:11 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T03:03:11.977 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:03:11 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T03:03:11.977 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:03:11 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T03:03:11.977 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:03:11 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T03:03:12.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:03:11 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T03:03:12.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:03:11 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T03:03:12.139 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:03:11 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T03:03:12.954 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:03:12 smithi188 ceph-mon[36599]: pgmap v704: 129 pgs: 129 active+clean; 276 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 5.3 KiB/s wr, 175 op/s 2024-06-10T03:03:12.977 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:03:12 smithi052 ceph-mon[32362]: pgmap v704: 129 pgs: 129 active+clean; 276 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 5.3 KiB/s wr, 175 op/s 2024-06-10T03:03:13.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:03:12 smithi120 ceph-mon[36634]: pgmap v704: 129 pgs: 129 active+clean; 276 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 5.3 KiB/s wr, 175 op/s 2024-06-10T03:03:14.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:03:14 smithi188 ceph-mon[36599]: pgmap v705: 129 pgs: 129 active+clean; 271 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 174 KiB/s wr, 226 op/s 2024-06-10T03:03:14.977 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:03:14 smithi052 ceph-mon[32362]: pgmap v705: 129 pgs: 129 active+clean; 271 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 174 KiB/s wr, 226 op/s 2024-06-10T03:03:15.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:03:14 smithi120 ceph-mon[36634]: pgmap v705: 129 pgs: 129 active+clean; 271 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 174 KiB/s wr, 226 op/s 2024-06-10T03:03:16.954 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:03:16 smithi188 ceph-mon[36599]: pgmap v706: 129 pgs: 129 active+clean; 269 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 338 KiB/s rd, 340 KiB/s wr, 191 op/s 2024-06-10T03:03:16.954 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:03:16 smithi188 ceph-mon[36599]: scrub summary: idle 2024-06-10T03:03:16.954 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:03:16 smithi188 ceph-mon[36599]: scrub summary: idle 2024-06-10T03:03:16.977 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:03:16 smithi052 ceph-mon[32362]: pgmap v706: 129 pgs: 129 active+clean; 269 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 338 KiB/s rd, 340 KiB/s wr, 191 op/s 2024-06-10T03:03:16.977 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:03:16 smithi052 ceph-mon[32362]: scrub summary: idle 2024-06-10T03:03:16.977 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:03:16 smithi052 ceph-mon[32362]: scrub summary: idle 2024-06-10T03:03:17.141 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:03:16 smithi120 ceph-mon[36634]: pgmap v706: 129 pgs: 129 active+clean; 269 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 338 KiB/s rd, 340 KiB/s wr, 191 op/s 2024-06-10T03:03:17.141 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:03:16 smithi120 ceph-mon[36634]: scrub summary: idle 2024-06-10T03:03:17.141 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:03:16 smithi120 ceph-mon[36634]: scrub summary: idle 2024-06-10T03:03:17.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:03:17 smithi188 ceph-mon[36599]: scrub summary: idle 2024-06-10T03:03:17.978 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:03:17 smithi052 ceph-mon[32362]: scrub summary: idle 2024-06-10T03:03:18.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:03:17 smithi120 ceph-mon[36634]: scrub summary: idle 2024-06-10T03:03:18.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:03:18 smithi188 ceph-mon[36599]: pgmap v707: 129 pgs: 129 active+clean; 253 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 333 KiB/s rd, 397 KiB/s wr, 271 op/s 2024-06-10T03:03:18.977 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:03:18 smithi052 ceph-mon[32362]: pgmap v707: 129 pgs: 129 active+clean; 253 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 333 KiB/s rd, 397 KiB/s wr, 271 op/s 2024-06-10T03:03:19.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:03:18 smithi120 ceph-mon[36634]: pgmap v707: 129 pgs: 129 active+clean; 253 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 333 KiB/s rd, 397 KiB/s wr, 271 op/s 2024-06-10T03:03:20.237 INFO:tasks.workunit:Stopping ['suites/blogbench.sh'] on client.0... 2024-06-10T03:03:20.237 DEBUG:teuthology.orchestra.run.smithi052:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-06-10T03:03:20.544 DEBUG:teuthology.parallel:result is None 2024-06-10T03:03:20.545 DEBUG:teuthology.orchestra.run.smithi052:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-06-10T03:03:20.582 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-06-10T03:03:20.583 DEBUG:teuthology.run_tasks:Unwinding manager quiescer 2024-06-10T03:03:20.594 INFO:tasks.quiescer:joining Quiescers 2024-06-10T03:03:20.594 WARNING:tasks.quiescer:The quiescer is requested to stop, running cancel all 2024-06-10T03:03:20.595 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T03:03:20.947 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T03:03:20.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:03:20 smithi188 ceph-mon[36599]: pgmap v708: 129 pgs: 129 active+clean; 245 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 688 KiB/s rd, 806 KiB/s wr, 307 op/s 2024-06-10T03:03:20.977 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:03:20 smithi052 ceph-mon[32362]: pgmap v708: 129 pgs: 129 active+clean; 245 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 688 KiB/s rd, 806 KiB/s wr, 307 op/s 2024-06-10T03:03:21.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:03:20 smithi120 ceph-mon[36634]: pgmap v708: 129 pgs: 129 active+clean; 245 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 688 KiB/s rd, 806 KiB/s wr, 307 op/s 2024-06-10T03:03:22.780 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:03:22 smithi052 ceph-mon[32362]: pgmap v709: 129 pgs: 129 active+clean; 239 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 686 KiB/s rd, 808 KiB/s wr, 340 op/s 2024-06-10T03:03:22.781 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:03:22 smithi052 ceph-mon[32362]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-10T03:03:22.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:22.888+0000 7f6d06bdd700 1 -- 172.21.15.52:0/714224872 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6d00101390 msgr2=0x7f6d00101810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:03:22.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:22.888+0000 7f6d06bdd700 1 --2- 172.21.15.52:0/714224872 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6d00101390 0x7f6d00101810 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f6cf0009990 tx=0x7f6cf0009ca0 comp rx=0 tx=0).stop 2024-06-10T03:03:22.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:22.888+0000 7f6d06bdd700 1 -- 172.21.15.52:0/714224872 shutdown_connections 2024-06-10T03:03:22.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:22.888+0000 7f6d06bdd700 1 --2- 172.21.15.52:0/714224872 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6d00101d50 0x7f6d0010e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:22.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:22.888+0000 7f6d06bdd700 1 --2- 172.21.15.52:0/714224872 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6d00107400 0x7f6d001077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:22.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:22.888+0000 7f6d06bdd700 1 --2- 172.21.15.52:0/714224872 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6d00101390 0x7f6d00101810 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:22.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:22.888+0000 7f6d06bdd700 1 -- 172.21.15.52:0/714224872 >> 172.21.15.52:0/714224872 conn(0x7f6d00078340 msgr2=0x7f6d00078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:03:22.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:22.888+0000 7f6d06bdd700 1 -- 172.21.15.52:0/714224872 shutdown_connections 2024-06-10T03:03:22.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:22.888+0000 7f6d06bdd700 1 -- 172.21.15.52:0/714224872 wait complete. 2024-06-10T03:03:22.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:22.889+0000 7f6d06bdd700 1 Processor -- start 2024-06-10T03:03:22.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:22.889+0000 7f6d06bdd700 1 -- start start 2024-06-10T03:03:22.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:22.889+0000 7f6d06bdd700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6d00101390 0x7f6d0019d820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:22.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:22.890+0000 7f6d06bdd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6d00101d50 0x7f6d0019dd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:22.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:22.890+0000 7f6d06bdd700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6d00107400 0x7f6d00197a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:22.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:22.890+0000 7f6d06bdd700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f6d00111670 con 0x7f6d00101d50 2024-06-10T03:03:22.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:22.890+0000 7f6d06bdd700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f6d001114f0 con 0x7f6d00107400 2024-06-10T03:03:22.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:22.890+0000 7f6d06bdd700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f6d001117f0 con 0x7f6d00101390 2024-06-10T03:03:22.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:22.890+0000 7f6cf7fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6d00101d50 0x7f6d0019dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:22.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:22.890+0000 7f6cf7fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6d00101d50 0x7f6d0019dd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:43356/0 (socket says 172.21.15.52:43356) 2024-06-10T03:03:22.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:22.890+0000 7f6cf7fff700 1 -- 172.21.15.52:0/1560830776 learned_addr learned my addr 172.21.15.52:0/1560830776 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:03:22.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:22.890+0000 7f6d04979700 1 --2- 172.21.15.52:0/1560830776 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6d00101390 0x7f6d0019d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:22.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:22.890+0000 7f6d0517a700 1 --2- 172.21.15.52:0/1560830776 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6d00107400 0x7f6d00197a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:22.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:22.890+0000 7f6cf7fff700 1 -- 172.21.15.52:0/1560830776 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6d00101390 msgr2=0x7f6d0019d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:03:22.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:22.890+0000 7f6cf7fff700 1 --2- 172.21.15.52:0/1560830776 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6d00101390 0x7f6d0019d820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:22.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:22.890+0000 7f6cf7fff700 1 -- 172.21.15.52:0/1560830776 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6d00107400 msgr2=0x7f6d00197a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:03:22.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:22.890+0000 7f6cf7fff700 1 --2- 172.21.15.52:0/1560830776 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6d00107400 0x7f6d00197a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:22.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:22.890+0000 7f6cf7fff700 1 -- 172.21.15.52:0/1560830776 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d00198190 con 0x7f6d00101d50 2024-06-10T03:03:22.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:22.890+0000 7f6d0517a700 1 --2- 172.21.15.52:0/1560830776 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6d00107400 0x7f6d00197a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T03:03:22.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:22.890+0000 7f6cf7fff700 1 --2- 172.21.15.52:0/1560830776 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6d00101d50 0x7f6d0019dd60 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f6cf00038b0 tx=0x7f6cf0009f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:03:22.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:22.890+0000 7f6cf5ffb700 1 -- 172.21.15.52:0/1560830776 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cf0027070 con 0x7f6d00101d50 2024-06-10T03:03:22.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:22.891+0000 7f6d06bdd700 1 -- 172.21.15.52:0/1560830776 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d00198420 con 0x7f6d00101d50 2024-06-10T03:03:22.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:22.891+0000 7f6cf5ffb700 1 -- 172.21.15.52:0/1560830776 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6cf0004050 con 0x7f6d00101d50 2024-06-10T03:03:22.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:22.891+0000 7f6cf5ffb700 1 -- 172.21.15.52:0/1560830776 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cf0022cd0 con 0x7f6d00101d50 2024-06-10T03:03:22.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:22.891+0000 7f6d06bdd700 1 -- 172.21.15.52:0/1560830776 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d001a46f0 con 0x7f6d00101d50 2024-06-10T03:03:22.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:22.892+0000 7f6d06bdd700 1 -- 172.21.15.52:0/1560830776 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d00069180 con 0x7f6d00101d50 2024-06-10T03:03:22.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:22.893+0000 7f6cf5ffb700 1 -- 172.21.15.52:0/1560830776 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f6cf002a070 con 0x7f6d00101d50 2024-06-10T03:03:22.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:22.893+0000 7f6cf5ffb700 1 --2- 172.21.15.52:0/1560830776 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f6ce80a2ca0 0x7f6ce80a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:22.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:22.894+0000 7f6d04979700 1 --2- 172.21.15.52:0/1560830776 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f6ce80a2ca0 0x7f6ce80a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:22.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:22.894+0000 7f6cf5ffb700 1 -- 172.21.15.52:0/1560830776 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f6cf00d01f0 con 0x7f6d00101d50 2024-06-10T03:03:22.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:22.894+0000 7f6d04979700 1 --2- 172.21.15.52:0/1560830776 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f6ce80a2ca0 0x7f6ce80a5160 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f6d00106d70 tx=0x7f6cfc0073d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:03:22.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:22.898+0000 7f6cf5ffb700 1 -- 172.21.15.52:0/1560830776 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f6cf005e850 con 0x7f6d00101d50 2024-06-10T03:03:22.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:03:22 smithi188 ceph-mon[36599]: pgmap v709: 129 pgs: 129 active+clean; 239 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 686 KiB/s rd, 808 KiB/s wr, 340 op/s 2024-06-10T03:03:22.954 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:03:22 smithi188 ceph-mon[36599]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-10T03:03:23.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:03:22 smithi120 ceph-mon[36634]: pgmap v709: 129 pgs: 129 active+clean; 239 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 686 KiB/s rd, 808 KiB/s wr, 340 op/s 2024-06-10T03:03:23.139 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:03:22 smithi120 ceph-mon[36634]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-10T03:03:23.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:23.151+0000 7f6d06bdd700 1 -- 172.21.15.52:0/1560830776 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6d00198f40 con 0x7f6d00101d50 2024-06-10T03:03:23.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:23.152+0000 7f6cf5ffb700 1 -- 172.21.15.52:0/1560830776 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) ==== 94+0+11955 (secure 0 0 0) 0x7f6d00198f40 con 0x7f6d00101d50 2024-06-10T03:03:23.152 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-10T03:03:23.153 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":43,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/889673247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":889673247},{"type":"v1","addr":"172.21.15.52:6835","nonce":889673247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/3615663511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":3615663511},{"type":"v1","addr":"172.21.15.52:6837","nonce":3615663511}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":43,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T02:50:03.889766+0000","modified":"2024-06-10T03:03:21.199409+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24484,"mds_1":24469,"mds_2":24463,"mds_3":24449,"mds_4":24428},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14571":{"gid":14571,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6839/3059599042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":3059599042},{"type":"v1","addr":"172.21.15.52:6839","nonce":3059599042}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14583":{"gid":14583,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6841/3039275521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":3039275521},{"type":"v1","addr":"172.21.15.52:6841","nonce":3039275521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24383":{"gid":24383,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6833/2609002896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":2609002896},{"type":"v1","addr":"172.21.15.120:6833","nonce":2609002896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6835/1996370288","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":1996370288},{"type":"v1","addr":"172.21.15.120:6835","nonce":1996370288}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6833/529564979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":529564979},{"type":"v1","addr":"172.21.15.188:6833","nonce":529564979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":25,"addr":"172.21.15.120:6837/4132440952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":4132440952},{"type":"v1","addr":"172.21.15.120:6837","nonce":4132440952}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"b","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.120:6839/1672436487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1672436487},{"type":"v1","addr":"172.21.15.120:6839","nonce":1672436487}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":2,"incarnation":22,"state":"up:active","state_seq":26,"addr":"172.21.15.188:6835/3904544259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":3904544259},{"type":"v1","addr":"172.21.15.188:6835","nonce":3904544259}]},"join_fscid":-1,"export_targets":[1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.188:6837/819317726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":819317726},{"type":"v1","addr":"172.21.15.188:6837","nonce":819317726}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.188:6839/310444836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":310444836},{"type":"v1","addr":"172.21.15.188:6839","nonce":310444836}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24484,"qdb_cluster":[24469,24449,24428,24463,24484]},"id":1}]} 2024-06-10T03:03:23.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:23.154+0000 7f6d06bdd700 1 -- 172.21.15.52:0/1560830776 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f6ce80a2ca0 msgr2=0x7f6ce80a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:03:23.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:23.154+0000 7f6d06bdd700 1 --2- 172.21.15.52:0/1560830776 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f6ce80a2ca0 0x7f6ce80a5160 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f6d00106d70 tx=0x7f6cfc0073d0 comp rx=0 tx=0).stop 2024-06-10T03:03:23.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:23.154+0000 7f6d06bdd700 1 -- 172.21.15.52:0/1560830776 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6d00101d50 msgr2=0x7f6d0019dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:03:23.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:23.154+0000 7f6d06bdd700 1 --2- 172.21.15.52:0/1560830776 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6d00101d50 0x7f6d0019dd60 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f6cf00038b0 tx=0x7f6cf0009f50 comp rx=0 tx=0).stop 2024-06-10T03:03:23.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:23.155+0000 7f6d06bdd700 1 -- 172.21.15.52:0/1560830776 shutdown_connections 2024-06-10T03:03:23.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:23.155+0000 7f6d06bdd700 1 --2- 172.21.15.52:0/1560830776 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6d00107400 0x7f6d00197a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:23.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:23.155+0000 7f6d06bdd700 1 --2- 172.21.15.52:0/1560830776 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6d00101390 0x7f6d0019d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:23.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:23.155+0000 7f6d06bdd700 1 --2- 172.21.15.52:0/1560830776 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f6ce80a2ca0 0x7f6ce80a5160 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:23.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:23.155+0000 7f6d06bdd700 1 --2- 172.21.15.52:0/1560830776 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6d00101d50 0x7f6d0019dd60 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:23.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:23.155+0000 7f6d06bdd700 1 -- 172.21.15.52:0/1560830776 >> 172.21.15.52:0/1560830776 conn(0x7f6d00078340 msgr2=0x7f6d000fff20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:03:23.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:23.155+0000 7f6d06bdd700 1 -- 172.21.15.52:0/1560830776 shutdown_connections 2024-06-10T03:03:23.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:23.155+0000 7f6d06bdd700 1 -- 172.21.15.52:0/1560830776 wait complete. 2024-06-10T03:03:23.156 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 43 2024-06-10T03:03:23.727 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:03:23 smithi052 ceph-mon[32362]: from='client.? 172.21.15.52:0/1560830776' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T03:03:23.954 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:03:23 smithi188 ceph-mon[36599]: from='client.? 172.21.15.52:0/1560830776' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T03:03:24.071 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24484 quiesce db --cancel --all' 2024-06-10T03:03:24.071 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24484 quiesce db --cancel --all 2024-06-10T03:03:24.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:03:23 smithi120 ceph-mon[36634]: from='client.? 172.21.15.52:0/1560830776' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T03:03:24.331 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T03:03:24.727 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:03:24 smithi052 ceph-mon[32362]: pgmap v710: 129 pgs: 129 active+clean; 231 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 435 op/s 2024-06-10T03:03:24.954 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:03:24 smithi188 ceph-mon[36599]: pgmap v710: 129 pgs: 129 active+clean; 231 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 435 op/s 2024-06-10T03:03:25.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:03:24 smithi120 ceph-mon[36634]: pgmap v710: 129 pgs: 129 active+clean; 231 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 435 op/s 2024-06-10T03:03:25.719 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:03:25 smithi052 ceph-mon[32362]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-10T03:03:26.040 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:03:25 smithi120 ceph-mon[36634]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-10T03:03:26.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:03:25 smithi188 ceph-mon[36599]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-10T03:03:26.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.260+0000 7fc3d2199700 1 -- 172.21.15.52:0/2411310542 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc3cc06aa00 msgr2=0x7fc3cc06ae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:03:26.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.260+0000 7fc3d2199700 1 --2- 172.21.15.52:0/2411310542 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc3cc06aa00 0x7fc3cc06ae80 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fc3b4009990 tx=0x7fc3b4009ca0 comp rx=0 tx=0).stop 2024-06-10T03:03:26.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.260+0000 7fc3d2199700 1 -- 172.21.15.52:0/2411310542 shutdown_connections 2024-06-10T03:03:26.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.260+0000 7fc3d2199700 1 --2- 172.21.15.52:0/2411310542 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc3cc06b3c0 0x7fc3cc10f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:26.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.260+0000 7fc3d2199700 1 --2- 172.21.15.52:0/2411310542 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc3cc1084f0 0x7fc3cc1088d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:26.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.260+0000 7fc3d2199700 1 --2- 172.21.15.52:0/2411310542 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc3cc06aa00 0x7fc3cc06ae80 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:26.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.260+0000 7fc3d2199700 1 -- 172.21.15.52:0/2411310542 >> 172.21.15.52:0/2411310542 conn(0x7fc3cc078300 msgr2=0x7fc3cc078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:03:26.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.260+0000 7fc3d2199700 1 -- 172.21.15.52:0/2411310542 shutdown_connections 2024-06-10T03:03:26.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.260+0000 7fc3d2199700 1 -- 172.21.15.52:0/2411310542 wait complete. 2024-06-10T03:03:26.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.261+0000 7fc3d2199700 1 Processor -- start 2024-06-10T03:03:26.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.261+0000 7fc3d2199700 1 -- start start 2024-06-10T03:03:26.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.261+0000 7fc3d2199700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc3cc06aa00 0x7fc3cc101450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:26.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.261+0000 7fc3d2199700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc3cc06b3c0 0x7fc3cc101990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:26.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.261+0000 7fc3d2199700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc3cc1084f0 0x7fc3cc101ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:26.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.262+0000 7fc3d2199700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fc3cc1127d0 con 0x7fc3cc06aa00 2024-06-10T03:03:26.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.262+0000 7fc3d2199700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc3cc112650 con 0x7fc3cc1084f0 2024-06-10T03:03:26.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.262+0000 7fc3d2199700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc3cc112950 con 0x7fc3cc06b3c0 2024-06-10T03:03:26.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.262+0000 7fc3cb7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc3cc06aa00 0x7fc3cc101450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:26.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.262+0000 7fc3cb7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc3cc06aa00 0x7fc3cc101450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:43378/0 (socket says 172.21.15.52:43378) 2024-06-10T03:03:26.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.262+0000 7fc3cb7fe700 1 -- 172.21.15.52:0/1430461131 learned_addr learned my addr 172.21.15.52:0/1430461131 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:03:26.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.262+0000 7fc3c2ffd700 1 --2- 172.21.15.52:0/1430461131 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc3cc06b3c0 0x7fc3cc101990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:26.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.262+0000 7fc3cbfff700 1 --2- 172.21.15.52:0/1430461131 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc3cc1084f0 0x7fc3cc101ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:26.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.262+0000 7fc3cb7fe700 1 -- 172.21.15.52:0/1430461131 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc3cc06b3c0 msgr2=0x7fc3cc101990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:03:26.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.262+0000 7fc3cb7fe700 1 --2- 172.21.15.52:0/1430461131 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc3cc06b3c0 0x7fc3cc101990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:26.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.262+0000 7fc3cb7fe700 1 -- 172.21.15.52:0/1430461131 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc3cc1084f0 msgr2=0x7fc3cc101ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:03:26.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.262+0000 7fc3cb7fe700 1 --2- 172.21.15.52:0/1430461131 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc3cc1084f0 0x7fc3cc101ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:26.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.262+0000 7fc3cb7fe700 1 -- 172.21.15.52:0/1430461131 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3cc1a71b0 con 0x7fc3cc06aa00 2024-06-10T03:03:26.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.262+0000 7fc3cbfff700 1 --2- 172.21.15.52:0/1430461131 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc3cc1084f0 0x7fc3cc101ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T03:03:26.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.262+0000 7fc3c2ffd700 1 --2- 172.21.15.52:0/1430461131 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc3cc06b3c0 0x7fc3cc101990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T03:03:26.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.262+0000 7fc3cb7fe700 1 --2- 172.21.15.52:0/1430461131 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc3cc06aa00 0x7fc3cc101450 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fc3bc00c920 tx=0x7fc3bc00cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:03:26.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.262+0000 7fc3c97fa700 1 -- 172.21.15.52:0/1430461131 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3bc007b00 con 0x7fc3cc06aa00 2024-06-10T03:03:26.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.263+0000 7fc3d2199700 1 -- 172.21.15.52:0/1430461131 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3cc1a74a0 con 0x7fc3cc06aa00 2024-06-10T03:03:26.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.263+0000 7fc3d2199700 1 -- 172.21.15.52:0/1430461131 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3cc1a7a30 con 0x7fc3cc06aa00 2024-06-10T03:03:26.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.263+0000 7fc3c97fa700 1 -- 172.21.15.52:0/1430461131 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc3bc004d10 con 0x7fc3cc06aa00 2024-06-10T03:03:26.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.263+0000 7fc3c97fa700 1 -- 172.21.15.52:0/1430461131 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3bc01d060 con 0x7fc3cc06aa00 2024-06-10T03:03:26.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.265+0000 7fc3c97fa700 1 -- 172.21.15.52:0/1430461131 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fc3bc00f450 con 0x7fc3cc06aa00 2024-06-10T03:03:26.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.266+0000 7fc3c97fa700 1 --2- 172.21.15.52:0/1430461131 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fc3b80a2c20 0x7fc3b80a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:26.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.266+0000 7fc3c2ffd700 1 --2- 172.21.15.52:0/1430461131 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fc3b80a2c20 0x7fc3b80a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:26.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.266+0000 7fc3c97fa700 1 -- 172.21.15.52:0/1430461131 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7fc3bc0cebc0 con 0x7fc3cc06aa00 2024-06-10T03:03:26.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.266+0000 7fc3c2ffd700 1 --2- 172.21.15.52:0/1430461131 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fc3b80a2c20 0x7fc3b80a50e0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fc3b4006010 tx=0x7fc3b400b540 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:03:26.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.267+0000 7fc3dca1cb80 1 Processor -- start 2024-06-10T03:03:26.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.267+0000 7fc3dca1cb80 1 -- start start 2024-06-10T03:03:26.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.267+0000 7fc3dca1cb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e23ee35470 0x55e23ee35850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:26.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.267+0000 7fc3dca1cb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e23ee35d90 0x55e23ee2f680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:26.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.267+0000 7fc3dca1cb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e23ee2fbc0 0x55e23ee2ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:26.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.267+0000 7fc3dca1cb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55e23ed21fd0 con 0x55e23ee35470 2024-06-10T03:03:26.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.267+0000 7fc3dca1cb80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55e23ed808c0 con 0x55e23ee2fbc0 2024-06-10T03:03:26.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.267+0000 7fc3dca1cb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55e23ece84d0 con 0x55e23ee35d90 2024-06-10T03:03:26.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.267+0000 7fc3cb7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e23ee35470 0x55e23ee35850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:26.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.267+0000 7fc3cb7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e23ee35470 0x55e23ee35850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:43386/0 (socket says 172.21.15.52:43386) 2024-06-10T03:03:26.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.267+0000 7fc3cb7fe700 1 -- 172.21.15.52:0/1036097802 learned_addr learned my addr 172.21.15.52:0/1036097802 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:03:26.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.267+0000 7fc3c2ffd700 1 --2- 172.21.15.52:0/1036097802 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e23ee35d90 0x55e23ee2f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:26.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.267+0000 7fc3cbfff700 1 --2- 172.21.15.52:0/1036097802 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e23ee2fbc0 0x55e23ee2ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:26.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.267+0000 7fc3cb7fe700 1 -- 172.21.15.52:0/1036097802 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e23ee35d90 msgr2=0x55e23ee2f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:03:26.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.267+0000 7fc3cb7fe700 1 --2- 172.21.15.52:0/1036097802 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e23ee35d90 0x55e23ee2f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:26.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.267+0000 7fc3cb7fe700 1 -- 172.21.15.52:0/1036097802 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e23ee2fbc0 msgr2=0x55e23ee2ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:03:26.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.267+0000 7fc3cb7fe700 1 --2- 172.21.15.52:0/1036097802 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e23ee2fbc0 0x55e23ee2ffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:26.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.267+0000 7fc3cb7fe700 1 -- 172.21.15.52:0/1036097802 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e23eccf940 con 0x55e23ee35470 2024-06-10T03:03:26.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.267+0000 7fc3cbfff700 1 --2- 172.21.15.52:0/1036097802 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e23ee2fbc0 0x55e23ee2ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T03:03:26.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.268+0000 7fc3cb7fe700 1 --2- 172.21.15.52:0/1036097802 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e23ee35470 0x55e23ee35850 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fc3bc09ad70 tx=0x7fc3bc09aa40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78e079bacf2fa54a server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:03:26.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.268+0000 7fc3c1ffb700 1 -- 172.21.15.52:0/1036097802 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3bc09f420 con 0x55e23ee35470 2024-06-10T03:03:26.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.268+0000 7fc3c1ffb700 1 -- 172.21.15.52:0/1036097802 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc3bc0a3ce0 con 0x55e23ee35470 2024-06-10T03:03:26.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.268+0000 7fc3c1ffb700 1 -- 172.21.15.52:0/1036097802 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3bc09f1d0 con 0x55e23ee35470 2024-06-10T03:03:26.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.269+0000 7fc3dca1cb80 1 -- 172.21.15.52:0/1036097802 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e23ee35470 msgr2=0x55e23ee35850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:03:26.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.269+0000 7fc3dca1cb80 1 --2- 172.21.15.52:0/1036097802 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e23ee35470 0x55e23ee35850 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fc3bc09ad70 tx=0x7fc3bc09aa40 comp rx=0 tx=0).stop 2024-06-10T03:03:26.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.269+0000 7fc3dca1cb80 1 -- 172.21.15.52:0/1036097802 shutdown_connections 2024-06-10T03:03:26.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.269+0000 7fc3dca1cb80 1 --2- 172.21.15.52:0/1036097802 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e23ee2fbc0 0x55e23ee2ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:26.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.269+0000 7fc3dca1cb80 1 --2- 172.21.15.52:0/1036097802 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e23ee35470 0x55e23ee35850 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:26.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.269+0000 7fc3dca1cb80 1 --2- 172.21.15.52:0/1036097802 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e23ee35d90 0x55e23ee2f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:26.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.269+0000 7fc3dca1cb80 1 -- 172.21.15.52:0/1036097802 >> 172.21.15.52:0/1036097802 conn(0x55e23ed5c0f0 msgr2=0x55e23ee2a970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:03:26.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.269+0000 7fc3dca1cb80 1 -- 172.21.15.52:0/1036097802 shutdown_connections 2024-06-10T03:03:26.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.269+0000 7fc3dca1cb80 1 -- 172.21.15.52:0/1036097802 wait complete. 2024-06-10T03:03:26.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.269+0000 7fc3dca1cb80 1 -- start start 2024-06-10T03:03:26.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.269+0000 7fc3dca1cb80 1 Processor -- start 2024-06-10T03:03:26.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.270+0000 7fc3dca1cb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e23ee2fbc0 0x55e23ee3baf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:26.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.270+0000 7fc3dca1cb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e23ee35470 0x55e23ee38df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:26.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.270+0000 7fc3dca1cb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e23ee35d90 0x55e23ee39330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:26.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.270+0000 7fc3dca1cb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55e23ed21030 con 0x55e23ee35470 2024-06-10T03:03:26.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.270+0000 7fc3dca1cb80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55e23ed7c880 con 0x55e23ee35d90 2024-06-10T03:03:26.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.270+0000 7fc3dca1cb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55e23ee3c710 con 0x55e23ee2fbc0 2024-06-10T03:03:26.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.270+0000 7fc3dca1cb80 10 client.0 _ll_register_callbacks cb 0x55e23ed16620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T03:03:26.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.270+0000 7fc3c2ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e23ee35470 0x55e23ee38df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:26.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.270+0000 7fc3c2ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e23ee35470 0x55e23ee38df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:43396/0 (socket says 172.21.15.52:43396) 2024-06-10T03:03:26.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.270+0000 7fc3c2ffd700 1 -- 172.21.15.52:0/1717817453 learned_addr learned my addr 172.21.15.52:0/1717817453 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:03:26.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.270+0000 7fc3cb7fe700 1 --2- 172.21.15.52:0/1717817453 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e23ee2fbc0 0x55e23ee3baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:26.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.270+0000 7fc3cbfff700 1 --2- 172.21.15.52:0/1717817453 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e23ee35d90 0x55e23ee39330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:26.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.270+0000 7fc3c2ffd700 1 -- 172.21.15.52:0/1717817453 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e23ee2fbc0 msgr2=0x55e23ee3baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:03:26.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.270+0000 7fc3c2ffd700 1 --2- 172.21.15.52:0/1717817453 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e23ee2fbc0 0x55e23ee3baf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:26.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.270+0000 7fc3c2ffd700 1 -- 172.21.15.52:0/1717817453 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e23ee35d90 msgr2=0x55e23ee39330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:03:26.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.270+0000 7fc3c2ffd700 1 --2- 172.21.15.52:0/1717817453 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e23ee35d90 0x55e23ee39330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:26.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.270+0000 7fc3c2ffd700 1 -- 172.21.15.52:0/1717817453 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e23ed467c0 con 0x55e23ee35470 2024-06-10T03:03:26.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.270+0000 7fc3c2ffd700 1 --2- 172.21.15.52:0/1717817453 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e23ee35470 0x55e23ee38df0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fc3b4000c00 tx=0x7fc3b401a050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e090526ac408ddd server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:03:26.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.270+0000 7fc3aaffd700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-10T03:03:26.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.270+0000 7fc3aaffd700 1 -- 172.21.15.52:0/1717817453 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3b400fcd0 con 0x55e23ee35470 2024-06-10T03:03:26.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.271+0000 7fc3dca1cb80 1 -- 172.21.15.52:0/1717817453 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc3bc0a3ce0 con 0x55e23ee35470 2024-06-10T03:03:26.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.271+0000 7fc3aaffd700 1 -- 172.21.15.52:0/1717817453 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7fc3b40349c0 con 0x55e23ee35470 2024-06-10T03:03:26.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.271+0000 7fc3aaffd700 1 -- 172.21.15.52:0/1717817453 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fc3b4043050 con 0x55e23ee35470 2024-06-10T03:03:26.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.271+0000 7fc3dca1cb80 10 client.16857 fetch_fsmap learned FSMap version 44 2024-06-10T03:03:26.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.271+0000 7fc3dca1cb80 1 -- 172.21.15.52:0/1717817453 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55e23ee46f70 con 0x55e23ee35470 2024-06-10T03:03:26.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.271+0000 7fc3aaffd700 1 -- 172.21.15.52:0/1717817453 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc3bc0a3ce0 con 0x55e23ee35470 2024-06-10T03:03:26.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.271+0000 7fc3aaffd700 1 -- 172.21.15.52:0/1717817453 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 44) ==== 8410+0+0 (secure 0 0 0) 0x7fc3b4005290 con 0x55e23ee35470 2024-06-10T03:03:26.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.271+0000 7fc3dca1cb80 10 client.16857 fetch_fsmap finished waiting for FSMap version 44 2024-06-10T03:03:26.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.271+0000 7fc3dca1cb80 10 client.16857 resolve_mds: validated gid 24484 aka daemon mds.i 2024-06-10T03:03:26.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.271+0000 7fc3dca1cb80 20 client.16857 populate_metadata read hostname 'smithi052' 2024-06-10T03:03:26.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.271+0000 7fc3dca1cb80 1 --2- 172.21.15.52:0/1717817453 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55e23ee49a70 0x55e23ee4be90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:26.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.271+0000 7fc3dca1cb80 4 client.16857 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T03:03:26.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.271+0000 7fc3dca1cb80 1 -- 172.21.15.52:0/1717817453 --> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e23ed8ddc0 con 0x55e23ee49a70 2024-06-10T03:03:26.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.271+0000 7fc3cb7fe700 1 --2- 172.21.15.52:0/1717817453 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55e23ee49a70 0x55e23ee4be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:26.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.272+0000 7fc3aaffd700 10 client.16857 ms_handle_connect on v2:172.21.15.188:6838/310444836 2024-06-10T03:03:26.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.272+0000 7fc3cb7fe700 1 --2- 172.21.15.52:0/1717817453 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55e23ee49a70 0x55e23ee4be90 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74931056f3e0fdf8 server_cookie=77a29e9acc2cecc4 in_seq=0 out_seq=0 2024-06-10T03:03:26.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.273+0000 7fc3aaffd700 1 -- 172.21.15.52:0/1717817453 <== mds.0 v2:172.21.15.188:6838/310444836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55e23ed8ddc0 con 0x55e23ee49a70 2024-06-10T03:03:26.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.273+0000 7fc3aaffd700 10 client.16857 handle_command_reply: tid=0 2024-06-10T03:03:26.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.273+0000 7fc3dca1cb80 1 client.16857 shutdown 2024-06-10T03:03:26.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.273+0000 7fc3dca1cb80 1 -- 172.21.15.52:0/1717817453 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e23ee35470 msgr2=0x55e23ee38df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:03:26.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.273+0000 7fc3dca1cb80 1 --2- 172.21.15.52:0/1717817453 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e23ee35470 0x55e23ee38df0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fc3b4000c00 tx=0x7fc3b401a050 comp rx=0 tx=0).stop 2024-06-10T03:03:26.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.273+0000 7fc3dca1cb80 1 -- 172.21.15.52:0/1717817453 shutdown_connections 2024-06-10T03:03:26.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.273+0000 7fc3dca1cb80 1 --2- 172.21.15.52:0/1717817453 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55e23ee49a70 0x55e23ee4be90 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:26.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.273+0000 7fc3dca1cb80 1 --2- 172.21.15.52:0/1717817453 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e23ee35d90 0x55e23ee39330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:26.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.273+0000 7fc3dca1cb80 1 --2- 172.21.15.52:0/1717817453 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e23ee2fbc0 0x55e23ee3baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:26.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.273+0000 7fc3dca1cb80 1 --2- 172.21.15.52:0/1717817453 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e23ee35470 0x55e23ee38df0 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:26.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.273+0000 7fc3dca1cb80 1 -- 172.21.15.52:0/1717817453 >> 172.21.15.52:0/1717817453 conn(0x55e23ed5c0f0 msgr2=0x55e23ee29b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:03:26.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.273+0000 7fc3aaffd700 0 client.16857 ms_handle_reset on v2:172.21.15.188:6838/310444836 2024-06-10T03:03:26.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.273+0000 7fc3dca1cb80 1 -- 172.21.15.52:0/1717817453 shutdown_connections 2024-06-10T03:03:26.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.273+0000 7fc3dca1cb80 1 -- 172.21.15.52:0/1717817453 wait complete. 2024-06-10T03:03:26.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.273+0000 7fc3dca1cb80 20 client.16857 trim_cache size 0 max 16384 2024-06-10T03:03:26.295 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.293+0000 7fc3dca1cb80 1 Processor -- start 2024-06-10T03:03:26.295 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.293+0000 7fc3dca1cb80 1 -- start start 2024-06-10T03:03:26.295 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.293+0000 7fc3dca1cb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e23ee2fbc0 0x55e23ee39360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:26.295 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.294+0000 7fc3dca1cb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e23ee35470 0x55e23ee3b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:26.295 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.294+0000 7fc3dca1cb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e23ee35d90 0x55e23ee2a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:26.295 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.294+0000 7fc3dca1cb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fc3ac00eb40 con 0x55e23ee2fbc0 2024-06-10T03:03:26.295 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.294+0000 7fc3dca1cb80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc3ac010890 con 0x55e23ee35470 2024-06-10T03:03:26.295 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.294+0000 7fc3dca1cb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc3ac00f270 con 0x55e23ee35d90 2024-06-10T03:03:26.295 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.294+0000 7fc3cb7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e23ee2fbc0 0x55e23ee39360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:26.295 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.294+0000 7fc3cb7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e23ee2fbc0 0x55e23ee39360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:43400/0 (socket says 172.21.15.52:43400) 2024-06-10T03:03:26.295 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.294+0000 7fc3cb7fe700 1 -- 172.21.15.52:0/3296614398 learned_addr learned my addr 172.21.15.52:0/3296614398 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:03:26.295 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.294+0000 7fc3cbfff700 1 --2- 172.21.15.52:0/3296614398 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e23ee35d90 0x55e23ee2a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:26.296 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.294+0000 7fc3c2ffd700 1 --2- 172.21.15.52:0/3296614398 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e23ee35470 0x55e23ee3b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:26.296 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.294+0000 7fc3cb7fe700 1 -- 172.21.15.52:0/3296614398 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e23ee35d90 msgr2=0x55e23ee2a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:03:26.296 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.294+0000 7fc3cb7fe700 1 --2- 172.21.15.52:0/3296614398 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e23ee35d90 0x55e23ee2a2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:26.296 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.294+0000 7fc3cb7fe700 1 -- 172.21.15.52:0/3296614398 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e23ee35470 msgr2=0x55e23ee3b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:03:26.296 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.294+0000 7fc3cb7fe700 1 --2- 172.21.15.52:0/3296614398 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e23ee35470 0x55e23ee3b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:26.296 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.294+0000 7fc3cb7fe700 1 -- 172.21.15.52:0/3296614398 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e23ee37360 con 0x55e23ee2fbc0 2024-06-10T03:03:26.296 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.294+0000 7fc3c2ffd700 1 --2- 172.21.15.52:0/3296614398 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e23ee35470 0x55e23ee3b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T03:03:26.296 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.294+0000 7fc3cb7fe700 1 --2- 172.21.15.52:0/3296614398 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e23ee2fbc0 0x55e23ee39360 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fc3bc00c8f0 tx=0x7fc3bc0ce040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95b04579f883e85a server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:03:26.296 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.295+0000 7fc3c17fa700 1 -- 172.21.15.52:0/3296614398 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3bc0a3ce0 con 0x55e23ee2fbc0 2024-06-10T03:03:26.296 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.295+0000 7fc3c17fa700 1 -- 172.21.15.52:0/3296614398 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc3bc099370 con 0x55e23ee2fbc0 2024-06-10T03:03:26.296 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.295+0000 7fc3c17fa700 1 -- 172.21.15.52:0/3296614398 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3bc095be0 con 0x55e23ee2fbc0 2024-06-10T03:03:26.300 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.295+0000 7fc3dca1cb80 1 -- 172.21.15.52:0/3296614398 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e23ee2fbc0 msgr2=0x55e23ee39360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:03:26.300 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.295+0000 7fc3dca1cb80 1 --2- 172.21.15.52:0/3296614398 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e23ee2fbc0 0x55e23ee39360 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fc3bc00c8f0 tx=0x7fc3bc0ce040 comp rx=0 tx=0).stop 2024-06-10T03:03:26.300 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.295+0000 7fc3dca1cb80 1 -- 172.21.15.52:0/3296614398 shutdown_connections 2024-06-10T03:03:26.300 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.295+0000 7fc3dca1cb80 1 --2- 172.21.15.52:0/3296614398 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e23ee35d90 0x55e23ee2a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:26.300 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.295+0000 7fc3dca1cb80 1 --2- 172.21.15.52:0/3296614398 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e23ee2fbc0 0x55e23ee39360 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:26.300 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.295+0000 7fc3dca1cb80 1 --2- 172.21.15.52:0/3296614398 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e23ee35470 0x55e23ee3b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:26.300 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.295+0000 7fc3dca1cb80 1 -- 172.21.15.52:0/3296614398 >> 172.21.15.52:0/3296614398 conn(0x55e23ed5c0f0 msgr2=0x55e23ee2ef00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:03:26.300 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.295+0000 7fc3dca1cb80 1 -- 172.21.15.52:0/3296614398 shutdown_connections 2024-06-10T03:03:26.300 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.296+0000 7fc3dca1cb80 1 -- 172.21.15.52:0/3296614398 wait complete. 2024-06-10T03:03:26.300 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.296+0000 7fc3dca1cb80 1 -- start start 2024-06-10T03:03:26.300 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.296+0000 7fc3dca1cb80 1 Processor -- start 2024-06-10T03:03:26.301 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.297+0000 7fc3dca1cb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e23ee2fbc0 0x55e23ee41ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:26.301 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.297+0000 7fc3dca1cb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e23ee35470 0x55e23ee50e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:26.301 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.297+0000 7fc3dca1cb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e23ee35d90 0x55e23ee51350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:26.301 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.297+0000 7fc3dca1cb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fc3ac010100 con 0x55e23ee2fbc0 2024-06-10T03:03:26.301 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.297+0000 7fc3dca1cb80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc3ac00fa00 con 0x55e23ee35d90 2024-06-10T03:03:26.301 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.297+0000 7fc3dca1cb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc3ac00cae0 con 0x55e23ee35470 2024-06-10T03:03:26.301 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.297+0000 7fc3cb7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e23ee2fbc0 0x55e23ee41ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:26.301 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.297+0000 7fc3cb7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e23ee2fbc0 0x55e23ee41ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:43404/0 (socket says 172.21.15.52:43404) 2024-06-10T03:03:26.301 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.297+0000 7fc3cb7fe700 1 -- 172.21.15.52:0/2858631914 learned_addr learned my addr 172.21.15.52:0/2858631914 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:03:26.301 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.297+0000 7fc3dca1cb80 10 client.0 _ll_register_callbacks cb 0x55e23ed16620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T03:03:26.301 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.297+0000 7fc3cbfff700 1 --2- 172.21.15.52:0/2858631914 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e23ee35d90 0x55e23ee51350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:26.301 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.297+0000 7fc3c2ffd700 1 --2- 172.21.15.52:0/2858631914 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e23ee35470 0x55e23ee50e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:26.301 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.297+0000 7fc3cb7fe700 1 -- 172.21.15.52:0/2858631914 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e23ee35470 msgr2=0x55e23ee50e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:03:26.302 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.297+0000 7fc3cb7fe700 1 --2- 172.21.15.52:0/2858631914 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e23ee35470 0x55e23ee50e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:26.302 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.297+0000 7fc3cb7fe700 1 -- 172.21.15.52:0/2858631914 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e23ee35d90 msgr2=0x55e23ee51350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:03:26.302 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.297+0000 7fc3cb7fe700 1 --2- 172.21.15.52:0/2858631914 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e23ee35d90 0x55e23ee51350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:26.302 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.297+0000 7fc3cb7fe700 1 -- 172.21.15.52:0/2858631914 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e23ee51890 con 0x55e23ee2fbc0 2024-06-10T03:03:26.302 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.297+0000 7fc3cb7fe700 1 --2- 172.21.15.52:0/2858631914 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e23ee2fbc0 0x55e23ee41ed0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fc3bc09ad70 tx=0x7fc3bc000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef3823ba60868aed server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:03:26.302 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.297+0000 7fc3aaffd700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-10T03:03:26.302 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.297+0000 7fc3aaffd700 1 -- 172.21.15.52:0/2858631914 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3bc0b3070 con 0x55e23ee2fbc0 2024-06-10T03:03:26.302 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.298+0000 7fc3aaffd700 1 -- 172.21.15.52:0/2858631914 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7fc3bc007ca0 con 0x55e23ee2fbc0 2024-06-10T03:03:26.302 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.298+0000 7fc3dca1cb80 1 -- 172.21.15.52:0/2858631914 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc3bc099370 con 0x55e23ee2fbc0 2024-06-10T03:03:26.302 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.298+0000 7fc3aaffd700 1 -- 172.21.15.52:0/2858631914 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fc3bc0bb130 con 0x55e23ee2fbc0 2024-06-10T03:03:26.302 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.298+0000 7fc3dca1cb80 10 client.16863 fetch_fsmap learned FSMap version 44 2024-06-10T03:03:26.302 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.298+0000 7fc3dca1cb80 1 -- 172.21.15.52:0/2858631914 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55e23eeb4400 con 0x55e23ee2fbc0 2024-06-10T03:03:26.302 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.298+0000 7fc3aaffd700 1 -- 172.21.15.52:0/2858631914 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc3bc099370 con 0x55e23ee2fbc0 2024-06-10T03:03:26.302 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.298+0000 7fc3aaffd700 1 -- 172.21.15.52:0/2858631914 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 44) ==== 8410+0+0 (secure 0 0 0) 0x7fc3bc020060 con 0x55e23ee2fbc0 2024-06-10T03:03:26.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.298+0000 7fc3dca1cb80 10 client.16863 fetch_fsmap finished waiting for FSMap version 44 2024-06-10T03:03:26.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.298+0000 7fc3dca1cb80 10 client.16863 resolve_mds: validated gid 24484 aka daemon mds.i 2024-06-10T03:03:26.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.298+0000 7fc3dca1cb80 20 client.16863 populate_metadata read hostname 'smithi052' 2024-06-10T03:03:26.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.298+0000 7fc3dca1cb80 1 --2- 172.21.15.52:0/2858631914 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55e23eeb4690 0x55e23eeb6a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:26.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.298+0000 7fc3dca1cb80 4 client.16863 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "quiesce db", "cancel": true, "all": true}] 2024-06-10T03:03:26.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.298+0000 7fc3dca1cb80 1 -- 172.21.15.52:0/2858631914 --> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] -- command(tid 0: {"prefix": "quiesce db", "cancel": true, "all": true}) -- 0x55e23ed88340 con 0x55e23eeb4690 2024-06-10T03:03:26.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.298+0000 7fc3c2ffd700 1 --2- 172.21.15.52:0/2858631914 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55e23eeb4690 0x55e23eeb6a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:26.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.299+0000 7fc3c2ffd700 1 --2- 172.21.15.52:0/2858631914 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55e23eeb4690 0x55e23eeb6a80 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=11762d2c0546e648 server_cookie=31d97e82b49c0fa7 in_seq=0 out_seq=0 2024-06-10T03:03:26.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.299+0000 7fc3aaffd700 10 client.16863 ms_handle_connect on v2:172.21.15.188:6838/310444836 2024-06-10T03:03:26.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.299+0000 7fc3aaffd700 1 -- 172.21.15.52:0/2858631914 <== mds.0 v2:172.21.15.188:6838/310444836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+80 (crc 0 0 0) 0x55e23ed88340 con 0x55e23eeb4690 2024-06-10T03:03:26.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.299+0000 7fc3aaffd700 10 client.16863 handle_command_reply: tid=0 2024-06-10T03:03:26.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.299+0000 7fc3dca1cb80 1 client.16863 shutdown 2024-06-10T03:03:26.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.299+0000 7fc3dca1cb80 1 -- 172.21.15.52:0/2858631914 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e23ee2fbc0 msgr2=0x55e23ee41ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:03:26.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.299+0000 7fc3dca1cb80 1 --2- 172.21.15.52:0/2858631914 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e23ee2fbc0 0x55e23ee41ed0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fc3bc09ad70 tx=0x7fc3bc000bc0 comp rx=0 tx=0).stop 2024-06-10T03:03:26.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.299+0000 7fc3dca1cb80 1 -- 172.21.15.52:0/2858631914 shutdown_connections 2024-06-10T03:03:26.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.299+0000 7fc3dca1cb80 1 --2- 172.21.15.52:0/2858631914 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x55e23eeb4690 0x55e23eeb6a80 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:26.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.300+0000 7fc3aaffd700 0 client.16863 ms_handle_reset on v2:172.21.15.188:6838/310444836 2024-06-10T03:03:26.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.300+0000 7fc3dca1cb80 1 --2- 172.21.15.52:0/2858631914 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e23ee35d90 0x55e23ee51350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:26.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.300+0000 7fc3dca1cb80 1 --2- 172.21.15.52:0/2858631914 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e23ee2fbc0 0x55e23ee41ed0 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:26.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.300+0000 7fc3dca1cb80 1 --2- 172.21.15.52:0/2858631914 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e23ee35470 0x55e23ee50e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:26.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.300+0000 7fc3dca1cb80 1 -- 172.21.15.52:0/2858631914 >> 172.21.15.52:0/2858631914 conn(0x55e23ed5c0f0 msgr2=0x55e23ee38350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:03:26.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.300+0000 7fc3dca1cb80 1 -- 172.21.15.52:0/2858631914 shutdown_connections 2024-06-10T03:03:26.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.300+0000 7fc3dca1cb80 1 -- 172.21.15.52:0/2858631914 wait complete. 2024-06-10T03:03:26.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.300+0000 7fc3dca1cb80 20 client.16863 trim_cache size 0 max 16384 2024-06-10T03:03:26.304 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-10T03:03:26.305 INFO:teuthology.orchestra.run.smithi052.stdout: "epoch": 44, 2024-06-10T03:03:26.305 INFO:teuthology.orchestra.run.smithi052.stdout: "leader": 24484, 2024-06-10T03:03:26.305 INFO:teuthology.orchestra.run.smithi052.stdout: "set_version": 24, 2024-06-10T03:03:26.305 INFO:teuthology.orchestra.run.smithi052.stdout: "sets": {} 2024-06-10T03:03:26.305 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-10T03:03:26.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.300+0000 7fc3d2199700 1 -- 172.21.15.52:0/1430461131 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fc3b80a2c20 msgr2=0x7fc3b80a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:03:26.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.300+0000 7fc3d2199700 1 --2- 172.21.15.52:0/1430461131 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fc3b80a2c20 0x7fc3b80a50e0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fc3b4006010 tx=0x7fc3b400b540 comp rx=0 tx=0).stop 2024-06-10T03:03:26.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.300+0000 7fc3d2199700 1 -- 172.21.15.52:0/1430461131 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc3cc06aa00 msgr2=0x7fc3cc101450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:03:26.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.300+0000 7fc3d2199700 1 --2- 172.21.15.52:0/1430461131 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc3cc06aa00 0x7fc3cc101450 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fc3bc00c920 tx=0x7fc3bc00cce0 comp rx=0 tx=0).stop 2024-06-10T03:03:26.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.301+0000 7fc3d2199700 1 -- 172.21.15.52:0/1430461131 shutdown_connections 2024-06-10T03:03:26.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.301+0000 7fc3d2199700 1 --2- 172.21.15.52:0/1430461131 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc3cc1084f0 0x7fc3cc101ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:26.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.301+0000 7fc3d2199700 1 --2- 172.21.15.52:0/1430461131 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fc3b80a2c20 0x7fc3b80a50e0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:26.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.301+0000 7fc3d2199700 1 --2- 172.21.15.52:0/1430461131 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc3cc06aa00 0x7fc3cc101450 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:26.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.301+0000 7fc3d2199700 1 --2- 172.21.15.52:0/1430461131 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc3cc06b3c0 0x7fc3cc101990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:26.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.301+0000 7fc3d2199700 1 -- 172.21.15.52:0/1430461131 >> 172.21.15.52:0/1430461131 conn(0x7fc3cc078300 msgr2=0x7fc3cc100140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:03:26.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.301+0000 7fc3d2199700 1 -- 172.21.15.52:0/1430461131 shutdown_connections 2024-06-10T03:03:26.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:26.301+0000 7fc3d2199700 1 -- 172.21.15.52:0/1430461131 wait complete. 2024-06-10T03:03:26.970 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:03:26 smithi052 ceph-mon[32362]: pgmap v711: 129 pgs: 129 active+clean; 229 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 995 KiB/s rd, 897 KiB/s wr, 394 op/s 2024-06-10T03:03:27.003 INFO:tasks.quiescer:done joining Quiescers 2024-06-10T03:03:27.003 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-06-10T03:03:27.012 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-06-10T03:03:27.059 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:03:26 smithi120 ceph-mon[36634]: pgmap v711: 129 pgs: 129 active+clean; 229 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 995 KiB/s rd, 897 KiB/s wr, 394 op/s 2024-06-10T03:03:27.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:03:26 smithi188 ceph-mon[36599]: pgmap v711: 129 pgs: 129 active+clean; 229 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 995 KiB/s rd, 897 KiB/s wr, 394 op/s 2024-06-10T03:03:28.977 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:03:28 smithi052 ceph-mon[32362]: pgmap v712: 129 pgs: 129 active+clean; 229 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 832 KiB/s rd, 731 KiB/s wr, 374 op/s 2024-06-10T03:03:29.139 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:03:28 smithi120 ceph-mon[36634]: pgmap v712: 129 pgs: 129 active+clean; 229 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 832 KiB/s rd, 731 KiB/s wr, 374 op/s 2024-06-10T03:03:29.205 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:03:28 smithi188 ceph-mon[36599]: pgmap v712: 129 pgs: 129 active+clean; 229 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 832 KiB/s rd, 731 KiB/s wr, 374 op/s 2024-06-10T03:03:30.977 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:03:30 smithi052 ceph-mon[32362]: pgmap v713: 129 pgs: 129 active+clean; 228 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 818 KiB/s rd, 682 KiB/s wr, 275 op/s 2024-06-10T03:03:31.006 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:03:30 smithi188 ceph-mon[36599]: pgmap v713: 129 pgs: 129 active+clean; 228 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 818 KiB/s rd, 682 KiB/s wr, 275 op/s 2024-06-10T03:03:31.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:03:30 smithi120 ceph-mon[36634]: pgmap v713: 129 pgs: 129 active+clean; 228 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 818 KiB/s rd, 682 KiB/s wr, 275 op/s 2024-06-10T03:03:32.953 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:03:32 smithi188 ceph-mon[36599]: pgmap v714: 129 pgs: 129 active+clean; 228 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 386 KiB/s rd, 272 KiB/s wr, 159 op/s 2024-06-10T03:03:32.977 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:03:32 smithi052 ceph-mon[32362]: pgmap v714: 129 pgs: 129 active+clean; 228 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 386 KiB/s rd, 272 KiB/s wr, 159 op/s 2024-06-10T03:03:33.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:03:32 smithi120 ceph-mon[36634]: pgmap v714: 129 pgs: 129 active+clean; 228 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 386 KiB/s rd, 272 KiB/s wr, 159 op/s 2024-06-10T03:03:34.977 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:03:34 smithi052 ceph-mon[32362]: pgmap v715: 129 pgs: 129 active+clean; 228 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 388 KiB/s rd, 271 KiB/s wr, 128 op/s 2024-06-10T03:03:35.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:03:34 smithi120 ceph-mon[36634]: pgmap v715: 129 pgs: 129 active+clean; 228 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 388 KiB/s rd, 271 KiB/s wr, 128 op/s 2024-06-10T03:03:35.204 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:03:34 smithi188 ceph-mon[36599]: pgmap v715: 129 pgs: 129 active+clean; 228 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 388 KiB/s rd, 271 KiB/s wr, 128 op/s 2024-06-10T03:03:36.977 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:03:36 smithi052 ceph-mon[32362]: pgmap v716: 129 pgs: 129 active+clean; 228 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 28 op/s 2024-06-10T03:03:37.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:03:36 smithi120 ceph-mon[36634]: pgmap v716: 129 pgs: 129 active+clean; 228 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 28 op/s 2024-06-10T03:03:37.204 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:03:36 smithi188 ceph-mon[36599]: pgmap v716: 129 pgs: 129 active+clean; 228 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 28 op/s 2024-06-10T03:03:38.735 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-10T03:03:39.001 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T03:03:39.124 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:03:38 smithi052 ceph-mon[32362]: pgmap v717: 129 pgs: 129 active+clean; 228 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 13 KiB/s wr, 18 op/s 2024-06-10T03:03:39.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:03:38 smithi120 ceph-mon[36634]: pgmap v717: 129 pgs: 129 active+clean; 228 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 13 KiB/s wr, 18 op/s 2024-06-10T03:03:39.203 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:03:38 smithi188 ceph-mon[36599]: pgmap v717: 129 pgs: 129 active+clean; 228 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 13 KiB/s wr, 18 op/s 2024-06-10T03:03:40.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.715+0000 7ff86e196700 1 -- 172.21.15.52:0/823868571 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff868102740 msgr2=0x7ff868102b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:03:40.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.715+0000 7ff86e196700 1 --2- 172.21.15.52:0/823868571 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff868102740 0x7ff868102b20 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7ff8580099c0 tx=0x7ff858009cd0 comp rx=0 tx=0).stop 2024-06-10T03:03:40.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.715+0000 7ff86e196700 1 -- 172.21.15.52:0/823868571 shutdown_connections 2024-06-10T03:03:40.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.715+0000 7ff86e196700 1 --2- 172.21.15.52:0/823868571 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff868107840 0x7ff86810f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:40.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.715+0000 7ff86e196700 1 --2- 172.21.15.52:0/823868571 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff868102740 0x7ff868102b20 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:40.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.716+0000 7ff86e196700 1 --2- 172.21.15.52:0/823868571 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff8681030f0 0x7ff868107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:40.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.716+0000 7ff86e196700 1 -- 172.21.15.52:0/823868571 >> 172.21.15.52:0/823868571 conn(0x7ff8680795f0 msgr2=0x7ff868079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:03:40.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.716+0000 7ff86e196700 1 -- 172.21.15.52:0/823868571 shutdown_connections 2024-06-10T03:03:40.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.716+0000 7ff86e196700 1 -- 172.21.15.52:0/823868571 wait complete. 2024-06-10T03:03:40.718 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.716+0000 7ff86e196700 1 Processor -- start 2024-06-10T03:03:40.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.716+0000 7ff86e196700 1 -- start start 2024-06-10T03:03:40.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.717+0000 7ff86e196700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff868102740 0x7ff86819ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:40.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.717+0000 7ff86e196700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff8681030f0 0x7ff86819b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:40.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.717+0000 7ff86e196700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff868107840 0x7ff86819f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:40.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.717+0000 7ff86e196700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ff8681127d0 con 0x7ff8681030f0 2024-06-10T03:03:40.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.717+0000 7ff86e196700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff868112650 con 0x7ff868107840 2024-06-10T03:03:40.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.717+0000 7ff86e196700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ff868112950 con 0x7ff868102740 2024-06-10T03:03:40.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.717+0000 7ff85effd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff8681030f0 0x7ff86819b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:40.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.717+0000 7ff85effd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff8681030f0 0x7ff86819b2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36224/0 (socket says 172.21.15.52:36224) 2024-06-10T03:03:40.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.717+0000 7ff85effd700 1 -- 172.21.15.52:0/555747301 learned_addr learned my addr 172.21.15.52:0/555747301 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:03:40.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.717+0000 7ff8677fe700 1 --2- 172.21.15.52:0/555747301 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff868102740 0x7ff86819ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:40.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.717+0000 7ff867fff700 1 --2- 172.21.15.52:0/555747301 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff868107840 0x7ff86819f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:40.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.717+0000 7ff85effd700 1 -- 172.21.15.52:0/555747301 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff868102740 msgr2=0x7ff86819ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:03:40.720 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.717+0000 7ff85effd700 1 --2- 172.21.15.52:0/555747301 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff868102740 0x7ff86819ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:40.720 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.717+0000 7ff85effd700 1 -- 172.21.15.52:0/555747301 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff868107840 msgr2=0x7ff86819f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:03:40.720 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.717+0000 7ff85effd700 1 --2- 172.21.15.52:0/555747301 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff868107840 0x7ff86819f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:40.720 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.717+0000 7ff85effd700 1 -- 172.21.15.52:0/555747301 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff86819fdb0 con 0x7ff8681030f0 2024-06-10T03:03:40.720 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.718+0000 7ff85effd700 1 --2- 172.21.15.52:0/555747301 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff8681030f0 0x7ff86819b2e0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7ff85000c9b0 tx=0x7ff85000ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:03:40.720 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.718+0000 7ff8657fa700 1 -- 172.21.15.52:0/555747301 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff850007b00 con 0x7ff8681030f0 2024-06-10T03:03:40.720 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.718+0000 7ff86e196700 1 -- 172.21.15.52:0/555747301 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8681a00a0 con 0x7ff8681030f0 2024-06-10T03:03:40.720 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.718+0000 7ff8657fa700 1 -- 172.21.15.52:0/555747301 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff85000ce50 con 0x7ff8681030f0 2024-06-10T03:03:40.720 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.718+0000 7ff8657fa700 1 -- 172.21.15.52:0/555747301 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff850018660 con 0x7ff8681030f0 2024-06-10T03:03:40.720 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.718+0000 7ff86e196700 1 -- 172.21.15.52:0/555747301 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8681a7920 con 0x7ff8681030f0 2024-06-10T03:03:40.721 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.720+0000 7ff8657fa700 1 -- 172.21.15.52:0/555747301 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7ff850004830 con 0x7ff8681030f0 2024-06-10T03:03:40.721 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.721+0000 7ff8657fa700 1 --2- 172.21.15.52:0/555747301 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7ff8540a2c20 0x7ff8540a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:40.721 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.721+0000 7ff8677fe700 1 --2- 172.21.15.52:0/555747301 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7ff8540a2c20 0x7ff8540a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:40.721 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.721+0000 7ff8657fa700 1 -- 172.21.15.52:0/555747301 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7ff8500cf390 con 0x7ff8681030f0 2024-06-10T03:03:40.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.722+0000 7ff8677fe700 1 --2- 172.21.15.52:0/555747301 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7ff8540a2c20 0x7ff8540a50e0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7ff858000c00 tx=0x7ff858005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:03:40.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.722+0000 7ff878a19b80 1 Processor -- start 2024-06-10T03:03:40.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.722+0000 7ff878a19b80 1 -- start start 2024-06-10T03:03:40.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.722+0000 7ff878a19b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56424ca6b640 0x56424ca6ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:40.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.722+0000 7ff878a19b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56424ca6bf60 0x56424ca659e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:40.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.722+0000 7ff878a19b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56424ca65f20 0x56424ca66340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:40.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.722+0000 7ff878a19b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x56424c9b7810 con 0x56424ca6bf60 2024-06-10T03:03:40.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.722+0000 7ff878a19b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56424c95f720 con 0x56424ca65f20 2024-06-10T03:03:40.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.722+0000 7ff878a19b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x56424c9b37d0 con 0x56424ca6b640 2024-06-10T03:03:40.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.722+0000 7ff85effd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56424ca6bf60 0x56424ca659e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:40.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.722+0000 7ff85effd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56424ca6bf60 0x56424ca659e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36232/0 (socket says 172.21.15.52:36232) 2024-06-10T03:03:40.724 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.722+0000 7ff85effd700 1 -- 172.21.15.52:0/2126308565 learned_addr learned my addr 172.21.15.52:0/2126308565 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:03:40.724 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.722+0000 7ff867fff700 1 --2- 172.21.15.52:0/2126308565 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56424ca65f20 0x56424ca66340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:40.724 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.722+0000 7ff8677fe700 1 --2- 172.21.15.52:0/2126308565 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56424ca6b640 0x56424ca6ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:40.724 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.722+0000 7ff85effd700 1 -- 172.21.15.52:0/2126308565 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56424ca6b640 msgr2=0x56424ca6ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:03:40.724 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.722+0000 7ff85effd700 1 --2- 172.21.15.52:0/2126308565 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56424ca6b640 0x56424ca6ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:40.724 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.722+0000 7ff85effd700 1 -- 172.21.15.52:0/2126308565 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56424ca65f20 msgr2=0x56424ca66340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:03:40.724 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.722+0000 7ff85effd700 1 --2- 172.21.15.52:0/2126308565 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56424ca65f20 0x56424ca66340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:40.724 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.722+0000 7ff85effd700 1 -- 172.21.15.52:0/2126308565 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56424c955f50 con 0x56424ca6bf60 2024-06-10T03:03:40.724 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.722+0000 7ff85effd700 1 --2- 172.21.15.52:0/2126308565 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56424ca6bf60 0x56424ca659e0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7ff85009b770 tx=0x7ff850007f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bc2ad2464e07aa6 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:03:40.724 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.723+0000 7ff85dffb700 1 -- 172.21.15.52:0/2126308565 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8500cf160 con 0x56424ca6bf60 2024-06-10T03:03:40.724 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.723+0000 7ff85dffb700 1 -- 172.21.15.52:0/2126308565 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff85009b7e0 con 0x56424ca6bf60 2024-06-10T03:03:40.724 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.723+0000 7ff85dffb700 1 -- 172.21.15.52:0/2126308565 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff850097e50 con 0x56424ca6bf60 2024-06-10T03:03:40.728 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.723+0000 7ff878a19b80 1 -- 172.21.15.52:0/2126308565 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56424ca6bf60 msgr2=0x56424ca659e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:03:40.728 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.723+0000 7ff878a19b80 1 --2- 172.21.15.52:0/2126308565 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56424ca6bf60 0x56424ca659e0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7ff85009b770 tx=0x7ff850007f30 comp rx=0 tx=0).stop 2024-06-10T03:03:40.728 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.723+0000 7ff878a19b80 1 -- 172.21.15.52:0/2126308565 shutdown_connections 2024-06-10T03:03:40.728 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.723+0000 7ff878a19b80 1 --2- 172.21.15.52:0/2126308565 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56424ca65f20 0x56424ca66340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:40.728 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.723+0000 7ff878a19b80 1 --2- 172.21.15.52:0/2126308565 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56424ca6b640 0x56424ca6ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:40.728 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.723+0000 7ff878a19b80 1 --2- 172.21.15.52:0/2126308565 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56424ca6bf60 0x56424ca659e0 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:40.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.723+0000 7ff878a19b80 1 -- 172.21.15.52:0/2126308565 >> 172.21.15.52:0/2126308565 conn(0x56424c990cc0 msgr2=0x56424ca60850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:03:40.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.723+0000 7ff878a19b80 1 -- 172.21.15.52:0/2126308565 shutdown_connections 2024-06-10T03:03:40.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.724+0000 7ff878a19b80 1 -- 172.21.15.52:0/2126308565 wait complete. 2024-06-10T03:03:40.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.724+0000 7ff878a19b80 1 -- start start 2024-06-10T03:03:40.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.724+0000 7ff878a19b80 1 Processor -- start 2024-06-10T03:03:40.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.724+0000 7ff878a19b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56424ca65f20 0x56424ca71d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:40.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.724+0000 7ff878a19b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56424ca6b640 0x56424ca6f020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:40.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.724+0000 7ff878a19b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56424ca6bf60 0x56424ca6f560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:40.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.724+0000 7ff878a19b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x56424ca729d0 con 0x56424ca6bf60 2024-06-10T03:03:40.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.724+0000 7ff878a19b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56424c957bf0 con 0x56424ca65f20 2024-06-10T03:03:40.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.724+0000 7ff878a19b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x56424ca72b50 con 0x56424ca6b640 2024-06-10T03:03:40.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.724+0000 7ff878a19b80 10 client.0 _ll_register_callbacks cb 0x56424c951920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T03:03:40.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.725+0000 7ff867fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56424ca6bf60 0x56424ca6f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:40.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.725+0000 7ff867fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56424ca6bf60 0x56424ca6f560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36248/0 (socket says 172.21.15.52:36248) 2024-06-10T03:03:40.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.725+0000 7ff867fff700 1 -- 172.21.15.52:0/1786004399 learned_addr learned my addr 172.21.15.52:0/1786004399 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:03:40.730 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.725+0000 7ff8677fe700 1 --2- 172.21.15.52:0/1786004399 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56424ca65f20 0x56424ca71d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:40.730 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.725+0000 7ff85effd700 1 --2- 172.21.15.52:0/1786004399 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56424ca6b640 0x56424ca6f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:40.730 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.725+0000 7ff867fff700 1 -- 172.21.15.52:0/1786004399 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56424ca6b640 msgr2=0x56424ca6f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:03:40.730 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.725+0000 7ff867fff700 1 --2- 172.21.15.52:0/1786004399 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56424ca6b640 0x56424ca6f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:40.730 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.725+0000 7ff867fff700 1 -- 172.21.15.52:0/1786004399 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56424ca65f20 msgr2=0x56424ca71d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:03:40.730 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.725+0000 7ff867fff700 1 --2- 172.21.15.52:0/1786004399 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56424ca65f20 0x56424ca71d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:40.730 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.725+0000 7ff867fff700 1 -- 172.21.15.52:0/1786004399 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56424c9068d0 con 0x56424ca6bf60 2024-06-10T03:03:40.730 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.725+0000 7ff85effd700 1 --2- 172.21.15.52:0/1786004399 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56424ca6b640 0x56424ca6f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T03:03:40.730 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.725+0000 7ff867fff700 1 --2- 172.21.15.52:0/1786004399 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56424ca6bf60 0x56424ca6f560 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7ff8600042f0 tx=0x7ff8600043d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49e2f944812b6db0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:03:40.730 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.725+0000 7ff846ffd700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-10T03:03:40.730 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.725+0000 7ff846ffd700 1 -- 172.21.15.52:0/1786004399 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff86001a070 con 0x56424ca6bf60 2024-06-10T03:03:40.730 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.725+0000 7ff846ffd700 1 -- 172.21.15.52:0/1786004399 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7ff860007710 con 0x56424ca6bf60 2024-06-10T03:03:40.730 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.725+0000 7ff878a19b80 1 -- 172.21.15.52:0/1786004399 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff85009b7e0 con 0x56424ca6bf60 2024-06-10T03:03:40.731 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.726+0000 7ff846ffd700 1 -- 172.21.15.52:0/1786004399 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7ff860026050 con 0x56424ca6bf60 2024-06-10T03:03:40.731 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.726+0000 7ff878a19b80 10 client.16875 fetch_fsmap learned FSMap version 44 2024-06-10T03:03:40.731 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.726+0000 7ff878a19b80 1 -- 172.21.15.52:0/1786004399 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x56424ca7d3e0 con 0x56424ca6bf60 2024-06-10T03:03:40.731 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.726+0000 7ff846ffd700 1 -- 172.21.15.52:0/1786004399 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff85009b7e0 con 0x56424ca6bf60 2024-06-10T03:03:40.731 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.726+0000 7ff846ffd700 1 -- 172.21.15.52:0/1786004399 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 44) ==== 8410+0+0 (secure 0 0 0) 0x7ff860016b30 con 0x56424ca6bf60 2024-06-10T03:03:40.731 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.726+0000 7ff878a19b80 10 client.16875 fetch_fsmap finished waiting for FSMap version 44 2024-06-10T03:03:40.731 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.726+0000 7ff878a19b80 10 client.16875 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T03:03:40.731 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.726+0000 7ff878a19b80 20 client.16875 populate_metadata read hostname 'smithi052' 2024-06-10T03:03:40.731 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.726+0000 7ff878a19b80 1 --2- 172.21.15.52:0/1786004399 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x56424ca7f9c0 0x56424ca81db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:40.731 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.726+0000 7ff878a19b80 4 client.16875 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T03:03:40.731 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.726+0000 7ff878a19b80 1 -- 172.21.15.52:0/1786004399 --> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56424c9c4d10 con 0x56424ca7f9c0 2024-06-10T03:03:40.731 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.726+0000 7ff8677fe700 1 --2- 172.21.15.52:0/1786004399 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x56424ca7f9c0 0x56424ca81db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:40.731 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.727+0000 7ff8677fe700 1 --2- 172.21.15.52:0/1786004399 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x56424ca7f9c0 0x56424ca81db0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc63a6ceb32d9764 server_cookie=3998c021d07c3ec in_seq=0 out_seq=0 2024-06-10T03:03:40.731 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.727+0000 7ff846ffd700 10 client.16875 ms_handle_connect on v2:172.21.15.188:6838/310444836 2024-06-10T03:03:40.732 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.727+0000 7ff846ffd700 1 -- 172.21.15.52:0/1786004399 <== mds.0 v2:172.21.15.188:6838/310444836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x56424c9c4d10 con 0x56424ca7f9c0 2024-06-10T03:03:40.732 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.727+0000 7ff846ffd700 10 client.16875 handle_command_reply: tid=0 2024-06-10T03:03:40.732 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.727+0000 7ff878a19b80 1 client.16875 shutdown 2024-06-10T03:03:40.732 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.728+0000 7ff878a19b80 1 -- 172.21.15.52:0/1786004399 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56424ca6bf60 msgr2=0x56424ca6f560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:03:40.732 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.728+0000 7ff878a19b80 1 --2- 172.21.15.52:0/1786004399 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56424ca6bf60 0x56424ca6f560 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7ff8600042f0 tx=0x7ff8600043d0 comp rx=0 tx=0).stop 2024-06-10T03:03:40.732 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.728+0000 7ff878a19b80 1 -- 172.21.15.52:0/1786004399 shutdown_connections 2024-06-10T03:03:40.732 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.728+0000 7ff878a19b80 1 --2- 172.21.15.52:0/1786004399 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x56424ca7f9c0 0x56424ca81db0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:40.732 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.728+0000 7ff878a19b80 1 --2- 172.21.15.52:0/1786004399 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56424ca6bf60 0x56424ca6f560 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:40.732 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.728+0000 7ff878a19b80 1 --2- 172.21.15.52:0/1786004399 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56424ca65f20 0x56424ca71d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:40.732 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.728+0000 7ff878a19b80 1 --2- 172.21.15.52:0/1786004399 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56424ca6b640 0x56424ca6f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:40.732 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.728+0000 7ff846ffd700 0 client.16875 ms_handle_reset on v2:172.21.15.188:6838/310444836 2024-06-10T03:03:40.732 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.728+0000 7ff878a19b80 1 -- 172.21.15.52:0/1786004399 >> 172.21.15.52:0/1786004399 conn(0x56424c990cc0 msgr2=0x56424ca605f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:03:40.732 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.728+0000 7ff878a19b80 1 -- 172.21.15.52:0/1786004399 shutdown_connections 2024-06-10T03:03:40.732 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.728+0000 7ff878a19b80 1 -- 172.21.15.52:0/1786004399 wait complete. 2024-06-10T03:03:40.732 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.728+0000 7ff878a19b80 20 client.16875 trim_cache size 0 max 16384 2024-06-10T03:03:40.750 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.749+0000 7ff878a19b80 1 Processor -- start 2024-06-10T03:03:40.750 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.749+0000 7ff878a19b80 1 -- start start 2024-06-10T03:03:40.750 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.749+0000 7ff878a19b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56424ca65f20 0x56424ca659e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:40.750 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.749+0000 7ff878a19b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56424ca6b640 0x56424ca6cc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:40.751 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.749+0000 7ff878a19b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56424ca6bf60 0x56424ca609f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:40.751 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.749+0000 7ff878a19b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ff8681b0580 con 0x56424ca65f20 2024-06-10T03:03:40.751 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.749+0000 7ff878a19b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff8681b22d0 con 0x56424ca6b640 2024-06-10T03:03:40.751 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.749+0000 7ff878a19b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ff8681b0cb0 con 0x56424ca6bf60 2024-06-10T03:03:40.751 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.749+0000 7ff8677fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56424ca65f20 0x56424ca659e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:40.751 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.749+0000 7ff8677fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56424ca65f20 0x56424ca659e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36260/0 (socket says 172.21.15.52:36260) 2024-06-10T03:03:40.751 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.749+0000 7ff8677fe700 1 -- 172.21.15.52:0/1797295719 learned_addr learned my addr 172.21.15.52:0/1797295719 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:03:40.751 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.750+0000 7ff867fff700 1 --2- 172.21.15.52:0/1797295719 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56424ca6bf60 0x56424ca609f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:40.751 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.750+0000 7ff85effd700 1 --2- 172.21.15.52:0/1797295719 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56424ca6b640 0x56424ca6cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:40.751 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.750+0000 7ff8677fe700 1 -- 172.21.15.52:0/1797295719 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56424ca6bf60 msgr2=0x56424ca609f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:03:40.751 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.750+0000 7ff8677fe700 1 --2- 172.21.15.52:0/1797295719 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56424ca6bf60 0x56424ca609f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:40.751 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.750+0000 7ff8677fe700 1 -- 172.21.15.52:0/1797295719 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56424ca6b640 msgr2=0x56424ca6cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:03:40.751 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.750+0000 7ff8677fe700 1 --2- 172.21.15.52:0/1797295719 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56424ca6b640 0x56424ca6cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:40.751 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.750+0000 7ff8677fe700 1 -- 172.21.15.52:0/1797295719 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56424ca6f6b0 con 0x56424ca65f20 2024-06-10T03:03:40.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.750+0000 7ff85effd700 1 --2- 172.21.15.52:0/1797295719 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56424ca6b640 0x56424ca6cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T03:03:40.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.750+0000 7ff8677fe700 1 --2- 172.21.15.52:0/1797295719 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56424ca65f20 0x56424ca659e0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7ff858009fd0 tx=0x7ff858023000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9324777831af5329 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:03:40.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.750+0000 7ff85d7fa700 1 -- 172.21.15.52:0/1797295719 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff85800fa40 con 0x56424ca65f20 2024-06-10T03:03:40.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.750+0000 7ff85d7fa700 1 -- 172.21.15.52:0/1797295719 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff85800fbe0 con 0x56424ca65f20 2024-06-10T03:03:40.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.750+0000 7ff85d7fa700 1 -- 172.21.15.52:0/1797295719 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff858024410 con 0x56424ca65f20 2024-06-10T03:03:40.755 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.751+0000 7ff878a19b80 1 -- 172.21.15.52:0/1797295719 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56424ca65f20 msgr2=0x56424ca659e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:03:40.755 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.751+0000 7ff878a19b80 1 --2- 172.21.15.52:0/1797295719 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56424ca65f20 0x56424ca659e0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7ff858009fd0 tx=0x7ff858023000 comp rx=0 tx=0).stop 2024-06-10T03:03:40.755 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.751+0000 7ff878a19b80 1 -- 172.21.15.52:0/1797295719 shutdown_connections 2024-06-10T03:03:40.755 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.751+0000 7ff878a19b80 1 --2- 172.21.15.52:0/1797295719 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56424ca6bf60 0x56424ca609f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:40.755 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.751+0000 7ff878a19b80 1 --2- 172.21.15.52:0/1797295719 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56424ca65f20 0x56424ca659e0 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:40.755 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.751+0000 7ff878a19b80 1 --2- 172.21.15.52:0/1797295719 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56424ca6b640 0x56424ca6cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:40.755 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.751+0000 7ff878a19b80 1 -- 172.21.15.52:0/1797295719 >> 172.21.15.52:0/1797295719 conn(0x56424c990cc0 msgr2=0x56424ca59e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:03:40.755 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.751+0000 7ff878a19b80 1 -- 172.21.15.52:0/1797295719 shutdown_connections 2024-06-10T03:03:40.755 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.751+0000 7ff878a19b80 1 -- 172.21.15.52:0/1797295719 wait complete. 2024-06-10T03:03:40.756 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.751+0000 7ff878a19b80 1 -- start start 2024-06-10T03:03:40.756 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.751+0000 7ff878a19b80 1 Processor -- start 2024-06-10T03:03:40.756 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.752+0000 7ff878a19b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56424ca65f20 0x56424ca786b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:40.756 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.752+0000 7ff878a19b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56424ca6b640 0x56424ca87130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:40.756 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.752+0000 7ff878a19b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56424ca6bf60 0x56424ca87740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:40.756 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.752+0000 7ff878a19b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ff8681b1b40 con 0x56424ca6b640 2024-06-10T03:03:40.756 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.752+0000 7ff878a19b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff8681b1440 con 0x56424ca6bf60 2024-06-10T03:03:40.756 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.752+0000 7ff878a19b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ff8681ae520 con 0x56424ca65f20 2024-06-10T03:03:40.756 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.752+0000 7ff878a19b80 10 client.0 _ll_register_callbacks cb 0x56424c951920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T03:03:40.756 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.752+0000 7ff85effd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56424ca6b640 0x56424ca87130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:40.756 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.752+0000 7ff85effd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56424ca6b640 0x56424ca87130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36266/0 (socket says 172.21.15.52:36266) 2024-06-10T03:03:40.756 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.752+0000 7ff85effd700 1 -- 172.21.15.52:0/3318960999 learned_addr learned my addr 172.21.15.52:0/3318960999 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:03:40.756 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.752+0000 7ff867fff700 1 --2- 172.21.15.52:0/3318960999 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56424ca6bf60 0x56424ca87740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:40.756 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.752+0000 7ff8677fe700 1 --2- 172.21.15.52:0/3318960999 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56424ca65f20 0x56424ca786b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:40.757 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.752+0000 7ff85effd700 1 -- 172.21.15.52:0/3318960999 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56424ca65f20 msgr2=0x56424ca786b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:03:40.757 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.752+0000 7ff85effd700 1 --2- 172.21.15.52:0/3318960999 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56424ca65f20 0x56424ca786b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:40.757 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.752+0000 7ff85effd700 1 -- 172.21.15.52:0/3318960999 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56424ca6bf60 msgr2=0x56424ca87740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:03:40.757 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.752+0000 7ff85effd700 1 --2- 172.21.15.52:0/3318960999 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56424ca6bf60 0x56424ca87740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:40.757 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.752+0000 7ff85effd700 1 -- 172.21.15.52:0/3318960999 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56424cae9690 con 0x56424ca6b640 2024-06-10T03:03:40.757 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.752+0000 7ff85effd700 1 --2- 172.21.15.52:0/3318960999 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56424ca6b640 0x56424ca87130 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7ff8500074a0 tx=0x7ff850004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d604a1d03c40c3b server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:03:40.757 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.752+0000 7ff846ffd700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-10T03:03:40.757 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.753+0000 7ff846ffd700 1 -- 172.21.15.52:0/3318960999 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff85009fb90 con 0x56424ca6b640 2024-06-10T03:03:40.757 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.753+0000 7ff846ffd700 1 -- 172.21.15.52:0/3318960999 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7ff8500860d0 con 0x56424ca6b640 2024-06-10T03:03:40.757 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.753+0000 7ff878a19b80 1 -- 172.21.15.52:0/3318960999 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff85800fbe0 con 0x56424ca6b640 2024-06-10T03:03:40.757 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.753+0000 7ff846ffd700 1 -- 172.21.15.52:0/3318960999 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7ff8500a4450 con 0x56424ca6b640 2024-06-10T03:03:40.757 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.753+0000 7ff878a19b80 10 client.16881 fetch_fsmap learned FSMap version 44 2024-06-10T03:03:40.757 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.753+0000 7ff878a19b80 1 -- 172.21.15.52:0/3318960999 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x56424caea750 con 0x56424ca6b640 2024-06-10T03:03:40.757 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.753+0000 7ff846ffd700 1 -- 172.21.15.52:0/3318960999 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff85800fbe0 con 0x56424ca6b640 2024-06-10T03:03:40.757 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.753+0000 7ff846ffd700 1 -- 172.21.15.52:0/3318960999 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 44) ==== 8410+0+0 (secure 0 0 0) 0x7ff8500a3fc0 con 0x56424ca6b640 2024-06-10T03:03:40.758 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.753+0000 7ff878a19b80 10 client.16881 fetch_fsmap finished waiting for FSMap version 44 2024-06-10T03:03:40.758 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.753+0000 7ff878a19b80 10 client.16881 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T03:03:40.758 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.753+0000 7ff878a19b80 20 client.16881 populate_metadata read hostname 'smithi052' 2024-06-10T03:03:40.758 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.753+0000 7ff878a19b80 1 --2- 172.21.15.52:0/3318960999 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x56424caeaa90 0x56424caece80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:40.758 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.753+0000 7ff878a19b80 4 client.16881 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-10T03:03:40.758 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.753+0000 7ff878a19b80 1 -- 172.21.15.52:0/3318960999 --> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56424c9bf290 con 0x56424caeaa90 2024-06-10T03:03:40.758 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.753+0000 7ff8677fe700 1 --2- 172.21.15.52:0/3318960999 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x56424caeaa90 0x56424caece80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:40.758 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.754+0000 7ff8677fe700 1 --2- 172.21.15.52:0/3318960999 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x56424caeaa90 0x56424caece80 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=88bf2f06dc564330 server_cookie=bfbb2f7d356bc4a3 in_seq=0 out_seq=0 2024-06-10T03:03:40.758 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.754+0000 7ff846ffd700 10 client.16881 ms_handle_connect on v2:172.21.15.188:6838/310444836 2024-06-10T03:03:40.758 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.754+0000 7ff846ffd700 1 -- 172.21.15.52:0/3318960999 <== mds.0 v2:172.21.15.188:6838/310444836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56424c9bf290 con 0x56424caeaa90 2024-06-10T03:03:40.758 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.754+0000 7ff846ffd700 10 client.16881 handle_command_reply: tid=0 2024-06-10T03:03:40.758 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.754+0000 7ff878a19b80 1 client.16881 shutdown 2024-06-10T03:03:40.758 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.754+0000 7ff878a19b80 1 -- 172.21.15.52:0/3318960999 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56424ca6b640 msgr2=0x56424ca87130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:03:40.758 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.754+0000 7ff878a19b80 1 --2- 172.21.15.52:0/3318960999 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56424ca6b640 0x56424ca87130 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7ff8500074a0 tx=0x7ff850004750 comp rx=0 tx=0).stop 2024-06-10T03:03:40.759 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.754+0000 7ff878a19b80 1 -- 172.21.15.52:0/3318960999 shutdown_connections 2024-06-10T03:03:40.759 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.754+0000 7ff878a19b80 1 --2- 172.21.15.52:0/3318960999 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x56424caeaa90 0x56424caece80 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:40.759 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.754+0000 7ff878a19b80 1 --2- 172.21.15.52:0/3318960999 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56424ca6bf60 0x56424ca87740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:40.759 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.754+0000 7ff878a19b80 1 --2- 172.21.15.52:0/3318960999 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56424ca65f20 0x56424ca786b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:40.759 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.754+0000 7ff878a19b80 1 --2- 172.21.15.52:0/3318960999 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56424ca6b640 0x56424ca87130 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:40.759 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.754+0000 7ff878a19b80 1 -- 172.21.15.52:0/3318960999 >> 172.21.15.52:0/3318960999 conn(0x56424c990cc0 msgr2=0x56424ca6cb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:03:40.759 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.754+0000 7ff846ffd700 0 client.16881 ms_handle_reset on v2:172.21.15.188:6838/310444836 2024-06-10T03:03:40.759 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.755+0000 7ff878a19b80 1 -- 172.21.15.52:0/3318960999 shutdown_connections 2024-06-10T03:03:40.759 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.755+0000 7ff878a19b80 1 -- 172.21.15.52:0/3318960999 wait complete. 2024-06-10T03:03:40.759 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.755+0000 7ff878a19b80 20 client.16881 trim_cache size 0 max 16384 2024-06-10T03:03:40.760 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-10T03:03:40.760 INFO:teuthology.orchestra.run.smithi052.stdout: "status": "no active scrubs running", 2024-06-10T03:03:40.760 INFO:teuthology.orchestra.run.smithi052.stdout: "scrubs": {} 2024-06-10T03:03:40.760 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-10T03:03:40.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.755+0000 7ff86e196700 1 -- 172.21.15.52:0/555747301 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7ff8540a2c20 msgr2=0x7ff8540a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:03:40.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.755+0000 7ff86e196700 1 --2- 172.21.15.52:0/555747301 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7ff8540a2c20 0x7ff8540a50e0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7ff858000c00 tx=0x7ff858005fb0 comp rx=0 tx=0).stop 2024-06-10T03:03:40.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.755+0000 7ff86e196700 1 -- 172.21.15.52:0/555747301 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff8681030f0 msgr2=0x7ff86819b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:03:40.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.755+0000 7ff86e196700 1 --2- 172.21.15.52:0/555747301 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff8681030f0 0x7ff86819b2e0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7ff85000c9b0 tx=0x7ff85000ccc0 comp rx=0 tx=0).stop 2024-06-10T03:03:40.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.755+0000 7ff86e196700 1 -- 172.21.15.52:0/555747301 shutdown_connections 2024-06-10T03:03:40.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.755+0000 7ff86e196700 1 --2- 172.21.15.52:0/555747301 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff868107840 0x7ff86819f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:40.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.755+0000 7ff86e196700 1 --2- 172.21.15.52:0/555747301 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff868102740 0x7ff86819ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:40.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.755+0000 7ff86e196700 1 --2- 172.21.15.52:0/555747301 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7ff8540a2c20 0x7ff8540a50e0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:40.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.755+0000 7ff86e196700 1 --2- 172.21.15.52:0/555747301 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff8681030f0 0x7ff86819b2e0 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:40.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.755+0000 7ff86e196700 1 -- 172.21.15.52:0/555747301 >> 172.21.15.52:0/555747301 conn(0x7ff8680795f0 msgr2=0x7ff8681006f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:03:40.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.755+0000 7ff86e196700 1 -- 172.21.15.52:0/555747301 shutdown_connections 2024-06-10T03:03:40.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:40.755+0000 7ff86e196700 1 -- 172.21.15.52:0/555747301 wait complete. 2024-06-10T03:03:40.977 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:03:40 smithi052 ceph-mon[32362]: pgmap v718: 129 pgs: 129 active+clean; 228 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 10 KiB/s wr, 12 op/s 2024-06-10T03:03:41.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:03:40 smithi120 ceph-mon[36634]: pgmap v718: 129 pgs: 129 active+clean; 228 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 10 KiB/s wr, 12 op/s 2024-06-10T03:03:41.204 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:03:40 smithi188 ceph-mon[36599]: pgmap v718: 129 pgs: 129 active+clean; 228 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 10 KiB/s wr, 12 op/s 2024-06-10T03:03:41.509 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-06-10T03:03:41.510 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T03:03:41.769 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T03:03:42.946 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:03:42 smithi052 ceph-mon[32362]: pgmap v719: 129 pgs: 129 active+clean; 228 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 255 B/s wr, 9 op/s 2024-06-10T03:03:43.138 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:03:42 smithi120 ceph-mon[36634]: pgmap v719: 129 pgs: 129 active+clean; 228 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 255 B/s wr, 9 op/s 2024-06-10T03:03:43.204 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:03:42 smithi188 ceph-mon[36599]: pgmap v719: 129 pgs: 129 active+clean; 228 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 255 B/s wr, 9 op/s 2024-06-10T03:03:43.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:43.465+0000 7f9959a3d700 1 -- 172.21.15.52:0/3799916344 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f99541019d0 msgr2=0x7f9954107110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:03:43.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:43.465+0000 7f9959a3d700 1 --2- 172.21.15.52:0/3799916344 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f99541019d0 0x7f9954107110 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f99440099c0 tx=0x7f9944009cd0 comp rx=0 tx=0).stop 2024-06-10T03:03:43.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:43.465+0000 7f9959a3d700 1 -- 172.21.15.52:0/3799916344 shutdown_connections 2024-06-10T03:03:43.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:43.465+0000 7f9959a3d700 1 --2- 172.21.15.52:0/3799916344 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9954107840 0x7f995410f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:43.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:43.465+0000 7f9959a3d700 1 --2- 172.21.15.52:0/3799916344 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9954101020 0x7f9954101400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:43.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:43.465+0000 7f9959a3d700 1 --2- 172.21.15.52:0/3799916344 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f99541019d0 0x7f9954107110 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:43.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:43.465+0000 7f9959a3d700 1 -- 172.21.15.52:0/3799916344 >> 172.21.15.52:0/3799916344 conn(0x7f99540766c0 msgr2=0x7f9954076ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:03:43.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:43.465+0000 7f9959a3d700 1 -- 172.21.15.52:0/3799916344 shutdown_connections 2024-06-10T03:03:43.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:43.465+0000 7f9959a3d700 1 -- 172.21.15.52:0/3799916344 wait complete. 2024-06-10T03:03:43.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:43.466+0000 7f9959a3d700 1 Processor -- start 2024-06-10T03:03:43.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:43.466+0000 7f9959a3d700 1 -- start start 2024-06-10T03:03:43.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:43.466+0000 7f9959a3d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9954101020 0x7f9954101fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:43.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:43.466+0000 7f9959a3d700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f99541019d0 0x7f99541024e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:43.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:43.466+0000 7f9959a3d700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9954107840 0x7f9954102a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:43.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:43.466+0000 7f9959a3d700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f9954112790 con 0x7f9954101020 2024-06-10T03:03:43.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:43.466+0000 7f9959a3d700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f9954112610 con 0x7f99541019d0 2024-06-10T03:03:43.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:43.466+0000 7f9959a3d700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f9954112910 con 0x7f9954107840 2024-06-10T03:03:43.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:43.466+0000 7f9952ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9954101020 0x7f9954101fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:43.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:43.466+0000 7f9952ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9954101020 0x7f9954101fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58908/0 (socket says 172.21.15.52:58908) 2024-06-10T03:03:43.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:43.466+0000 7f9952ffd700 1 -- 172.21.15.52:0/237585079 learned_addr learned my addr 172.21.15.52:0/237585079 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:03:43.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:43.466+0000 7f99527fc700 1 --2- 172.21.15.52:0/237585079 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f99541019d0 0x7f99541024e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:43.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:43.466+0000 7f99537fe700 1 --2- 172.21.15.52:0/237585079 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9954107840 0x7f9954102a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:43.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:43.466+0000 7f9952ffd700 1 -- 172.21.15.52:0/237585079 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9954107840 msgr2=0x7f9954102a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:03:43.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:43.466+0000 7f9952ffd700 1 --2- 172.21.15.52:0/237585079 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9954107840 0x7f9954102a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:43.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:43.466+0000 7f9952ffd700 1 -- 172.21.15.52:0/237585079 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f99541019d0 msgr2=0x7f99541024e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:03:43.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:43.466+0000 7f9952ffd700 1 --2- 172.21.15.52:0/237585079 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f99541019d0 0x7f99541024e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:43.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:43.467+0000 7f9952ffd700 1 -- 172.21.15.52:0/237585079 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99541a7170 con 0x7f9954101020 2024-06-10T03:03:43.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:43.467+0000 7f9952ffd700 1 --2- 172.21.15.52:0/237585079 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9954101020 0x7f9954101fa0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f993c00d950 tx=0x7f993c00dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:03:43.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:43.467+0000 7f9958a3b700 1 -- 172.21.15.52:0/237585079 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f993c016070 con 0x7f9954101020 2024-06-10T03:03:43.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:43.467+0000 7f9958a3b700 1 -- 172.21.15.52:0/237585079 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f993c009b40 con 0x7f9954101020 2024-06-10T03:03:43.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:43.467+0000 7f9959a3d700 1 -- 172.21.15.52:0/237585079 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99541a7460 con 0x7f9954101020 2024-06-10T03:03:43.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:43.467+0000 7f9958a3b700 1 -- 172.21.15.52:0/237585079 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f993c00f620 con 0x7f9954101020 2024-06-10T03:03:43.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:43.467+0000 7f9959a3d700 1 -- 172.21.15.52:0/237585079 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99541a79c0 con 0x7f9954101020 2024-06-10T03:03:43.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:43.468+0000 7f9959a3d700 1 -- 172.21.15.52:0/237585079 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f995410aa80 con 0x7f9954101020 2024-06-10T03:03:43.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:43.469+0000 7f9958a3b700 1 -- 172.21.15.52:0/237585079 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f993c019070 con 0x7f9954101020 2024-06-10T03:03:43.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:43.470+0000 7f9958a3b700 1 --2- 172.21.15.52:0/237585079 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f99400a2ca0 0x7f99400a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:43.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:43.470+0000 7f99527fc700 1 --2- 172.21.15.52:0/237585079 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f99400a2ca0 0x7f99400a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:43.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:43.470+0000 7f9958a3b700 1 -- 172.21.15.52:0/237585079 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f993c0cf3b0 con 0x7f9954101020 2024-06-10T03:03:43.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:43.470+0000 7f99527fc700 1 --2- 172.21.15.52:0/237585079 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f99400a2ca0 0x7f99400a5160 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f9944009990 tx=0x7f9944005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:03:43.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:43.474+0000 7f9958a3b700 1 -- 172.21.15.52:0/237585079 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f993c09bf60 con 0x7f9954101020 2024-06-10T03:03:43.733 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:43.733+0000 7f9959a3d700 1 -- 172.21.15.52:0/237585079 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f99541034d0 con 0x7f9954101020 2024-06-10T03:03:43.734 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:43.734+0000 7f9958a3b700 1 -- 172.21.15.52:0/237585079 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) ==== 94+0+11957 (secure 0 0 0) 0x7f993c09b860 con 0x7f9954101020 2024-06-10T03:03:43.735 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-10T03:03:43.735 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":44,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/889673247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":889673247},{"type":"v1","addr":"172.21.15.52:6835","nonce":889673247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/3615663511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":3615663511},{"type":"v1","addr":"172.21.15.52:6837","nonce":3615663511}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":44,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T02:50:03.889766+0000","modified":"2024-06-10T03:03:23.701877+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24484,"mds_1":24469,"mds_2":24463,"mds_3":24449,"mds_4":24428},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14571":{"gid":14571,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6839/3059599042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":3059599042},{"type":"v1","addr":"172.21.15.52:6839","nonce":3059599042}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14583":{"gid":14583,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6841/3039275521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":3039275521},{"type":"v1","addr":"172.21.15.52:6841","nonce":3039275521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24383":{"gid":24383,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6833/2609002896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":2609002896},{"type":"v1","addr":"172.21.15.120:6833","nonce":2609002896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6835/1996370288","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":1996370288},{"type":"v1","addr":"172.21.15.120:6835","nonce":1996370288}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6833/529564979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":529564979},{"type":"v1","addr":"172.21.15.188:6833","nonce":529564979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":25,"addr":"172.21.15.120:6837/4132440952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":4132440952},{"type":"v1","addr":"172.21.15.120:6837","nonce":4132440952}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"b","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.120:6839/1672436487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1672436487},{"type":"v1","addr":"172.21.15.120:6839","nonce":1672436487}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":2,"incarnation":22,"state":"up:active","state_seq":26,"addr":"172.21.15.188:6835/3904544259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":3904544259},{"type":"v1","addr":"172.21.15.188:6835","nonce":3904544259}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.188:6837/819317726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":819317726},{"type":"v1","addr":"172.21.15.188:6837","nonce":819317726}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.188:6839/310444836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":310444836},{"type":"v1","addr":"172.21.15.188:6839","nonce":310444836}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24484,"qdb_cluster":[24469,24484,24463,24428,24449]},"id":1}]} 2024-06-10T03:03:43.737 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:43.736+0000 7f9959a3d700 1 -- 172.21.15.52:0/237585079 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f99400a2ca0 msgr2=0x7f99400a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:03:43.737 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:43.736+0000 7f9959a3d700 1 --2- 172.21.15.52:0/237585079 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f99400a2ca0 0x7f99400a5160 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f9944009990 tx=0x7f9944005fb0 comp rx=0 tx=0).stop 2024-06-10T03:03:43.737 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:43.736+0000 7f9959a3d700 1 -- 172.21.15.52:0/237585079 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9954101020 msgr2=0x7f9954101fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:03:43.737 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:43.736+0000 7f9959a3d700 1 --2- 172.21.15.52:0/237585079 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9954101020 0x7f9954101fa0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f993c00d950 tx=0x7f993c00dd10 comp rx=0 tx=0).stop 2024-06-10T03:03:43.737 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:43.736+0000 7f9959a3d700 1 -- 172.21.15.52:0/237585079 shutdown_connections 2024-06-10T03:03:43.737 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:43.736+0000 7f9959a3d700 1 --2- 172.21.15.52:0/237585079 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9954107840 0x7f9954102a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:43.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:43.736+0000 7f9959a3d700 1 --2- 172.21.15.52:0/237585079 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f99400a2ca0 0x7f99400a5160 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:43.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:43.736+0000 7f9959a3d700 1 --2- 172.21.15.52:0/237585079 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9954101020 0x7f9954101fa0 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:43.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:43.736+0000 7f9959a3d700 1 --2- 172.21.15.52:0/237585079 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f99541019d0 0x7f99541024e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:43.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:43.736+0000 7f9959a3d700 1 -- 172.21.15.52:0/237585079 >> 172.21.15.52:0/237585079 conn(0x7f99540766c0 msgr2=0x7f99540fee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:03:43.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:43.737+0000 7f9959a3d700 1 -- 172.21.15.52:0/237585079 shutdown_connections 2024-06-10T03:03:43.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:43.737+0000 7f9959a3d700 1 -- 172.21.15.52:0/237585079 wait complete. 2024-06-10T03:03:43.738 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 44 2024-06-10T03:03:44.428 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:03:44 smithi052 ceph-mon[32362]: pgmap v720: 129 pgs: 129 active+clean; 228 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 255 B/s wr, 12 op/s 2024-06-10T03:03:44.429 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:03:44 smithi052 ceph-mon[32362]: from='client.? 172.21.15.52:0/237585079' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T03:03:44.458 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-06-10T03:03:44.638 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:03:44 smithi120 ceph-mon[36634]: pgmap v720: 129 pgs: 129 active+clean; 228 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 255 B/s wr, 12 op/s 2024-06-10T03:03:44.639 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:03:44 smithi120 ceph-mon[36634]: from='client.? 172.21.15.52:0/237585079' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T03:03:44.704 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:03:44 smithi188 ceph-mon[36599]: pgmap v720: 129 pgs: 129 active+clean; 228 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 255 B/s wr, 12 op/s 2024-06-10T03:03:44.704 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:03:44 smithi188 ceph-mon[36599]: from='client.? 172.21.15.52:0/237585079' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T03:03:44.710 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T03:03:45.477 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:03:45 smithi052 ceph-mon[32362]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-10T03:03:45.638 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:03:45 smithi120 ceph-mon[36634]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-10T03:03:45.703 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:03:45 smithi188 ceph-mon[36599]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-10T03:03:46.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.355+0000 7f275b3bd700 1 -- 172.21.15.52:0/2288568241 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2754102510 msgr2=0x7f2754102990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:03:46.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.355+0000 7f275b3bd700 1 --2- 172.21.15.52:0/2288568241 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2754102510 0x7f2754102990 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f2744009990 tx=0x7f2744009ca0 comp rx=0 tx=0).stop 2024-06-10T03:03:46.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.355+0000 7f275b3bd700 1 -- 172.21.15.52:0/2288568241 shutdown_connections 2024-06-10T03:03:46.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.355+0000 7f275b3bd700 1 --2- 172.21.15.52:0/2288568241 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2754102ed0 0x7f275410f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:46.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.355+0000 7f275b3bd700 1 --2- 172.21.15.52:0/2288568241 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2754108510 0x7f27541088f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:46.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.355+0000 7f275b3bd700 1 --2- 172.21.15.52:0/2288568241 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2754102510 0x7f2754102990 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:46.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.355+0000 7f275b3bd700 1 -- 172.21.15.52:0/2288568241 >> 172.21.15.52:0/2288568241 conn(0x7f27540795b0 msgr2=0x7f27540799c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:03:46.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.355+0000 7f275b3bd700 1 -- 172.21.15.52:0/2288568241 shutdown_connections 2024-06-10T03:03:46.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.355+0000 7f275b3bd700 1 -- 172.21.15.52:0/2288568241 wait complete. 2024-06-10T03:03:46.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.356+0000 7f275b3bd700 1 Processor -- start 2024-06-10T03:03:46.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.356+0000 7f275b3bd700 1 -- start start 2024-06-10T03:03:46.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.356+0000 7f275b3bd700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2754102510 0x7f275419e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:46.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.356+0000 7f275b3bd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2754102ed0 0x7f275419ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:46.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.356+0000 7f275b3bd700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2754108510 0x7f2754198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:46.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.356+0000 7f275b3bd700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f27541127e0 con 0x7f2754102ed0 2024-06-10T03:03:46.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.356+0000 7f275b3bd700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f2754112660 con 0x7f2754108510 2024-06-10T03:03:46.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.356+0000 7f275b3bd700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f2754112960 con 0x7f2754102510 2024-06-10T03:03:46.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.356+0000 7f2758958700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2754102ed0 0x7f275419ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:46.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.356+0000 7f2758958700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2754102ed0 0x7f275419ede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58928/0 (socket says 172.21.15.52:58928) 2024-06-10T03:03:46.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.356+0000 7f2758958700 1 -- 172.21.15.52:0/2274453913 learned_addr learned my addr 172.21.15.52:0/2274453913 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:03:46.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.356+0000 7f2758958700 1 -- 172.21.15.52:0/2274453913 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2754102510 msgr2=0x7f275419e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:03:46.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.356+0000 7f2759159700 1 --2- 172.21.15.52:0/2274453913 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2754102510 0x7f275419e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:46.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.356+0000 7f275995a700 1 --2- 172.21.15.52:0/2274453913 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2754108510 0x7f2754198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:46.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.356+0000 7f2758958700 1 --2- 172.21.15.52:0/2274453913 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2754102510 0x7f275419e8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:46.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.356+0000 7f2758958700 1 -- 172.21.15.52:0/2274453913 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2754108510 msgr2=0x7f2754198a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:03:46.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.356+0000 7f2758958700 1 --2- 172.21.15.52:0/2274453913 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2754108510 0x7f2754198a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:46.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.356+0000 7f2758958700 1 -- 172.21.15.52:0/2274453913 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27541991c0 con 0x7f2754102ed0 2024-06-10T03:03:46.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.356+0000 7f275995a700 1 --2- 172.21.15.52:0/2274453913 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2754108510 0x7f2754198a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T03:03:46.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.357+0000 7f2758958700 1 --2- 172.21.15.52:0/2274453913 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2754102ed0 0x7f275419ede0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f2744000c00 tx=0x7f27440047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:03:46.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.357+0000 7f274a7fc700 1 -- 172.21.15.52:0/2274453913 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2744027070 con 0x7f2754102ed0 2024-06-10T03:03:46.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.357+0000 7f274a7fc700 1 -- 172.21.15.52:0/2274453913 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2744004050 con 0x7f2754102ed0 2024-06-10T03:03:46.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.357+0000 7f275b3bd700 1 -- 172.21.15.52:0/2274453913 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2754199450 con 0x7f2754102ed0 2024-06-10T03:03:46.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.357+0000 7f274a7fc700 1 -- 172.21.15.52:0/2274453913 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f274400f460 con 0x7f2754102ed0 2024-06-10T03:03:46.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.357+0000 7f275b3bd700 1 -- 172.21.15.52:0/2274453913 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27541a5770 con 0x7f2754102ed0 2024-06-10T03:03:46.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.360+0000 7f274a7fc700 1 -- 172.21.15.52:0/2274453913 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f274402a070 con 0x7f2754102ed0 2024-06-10T03:03:46.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.361+0000 7f274a7fc700 1 --2- 172.21.15.52:0/2274453913 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f27400a2ca0 0x7f27400a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:46.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.361+0000 7f2759159700 1 --2- 172.21.15.52:0/2274453913 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f27400a2ca0 0x7f27400a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:46.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.361+0000 7f274a7fc700 1 -- 172.21.15.52:0/2274453913 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f27440d0340 con 0x7f2754102ed0 2024-06-10T03:03:46.361 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:03:46 smithi052 ceph-mon[32362]: pgmap v721: 129 pgs: 129 active+clean; 228 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-10T03:03:46.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.361+0000 7f2765c40b80 1 Processor -- start 2024-06-10T03:03:46.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.361+0000 7f2765c40b80 1 -- start start 2024-06-10T03:03:46.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.362+0000 7f2759159700 1 --2- 172.21.15.52:0/2274453913 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f27400a2ca0 0x7f27400a5160 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f2750005950 tx=0x7f27500058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:03:46.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.362+0000 7f2765c40b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555b83b0b470 0x555b83b0b850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:46.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.362+0000 7f2765c40b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555b83b05670 0x555b83b05af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:46.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.362+0000 7f2765c40b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555b83b06030 0x555b83b12210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:46.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.362+0000 7f2765c40b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x555b839f3f60 con 0x555b83b05670 2024-06-10T03:03:46.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.362+0000 7f2765c40b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x555b83a52850 con 0x555b83b06030 2024-06-10T03:03:46.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.362+0000 7f2765c40b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x555b839ba460 con 0x555b83b0b470 2024-06-10T03:03:46.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.362+0000 7f2758958700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555b83b05670 0x555b83b05af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:46.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.362+0000 7f2758958700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555b83b05670 0x555b83b05af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58942/0 (socket says 172.21.15.52:58942) 2024-06-10T03:03:46.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.362+0000 7f2758958700 1 -- 172.21.15.52:0/1543488198 learned_addr learned my addr 172.21.15.52:0/1543488198 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:03:46.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.362+0000 7f2759159700 1 --2- 172.21.15.52:0/1543488198 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555b83b0b470 0x555b83b0b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:46.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.362+0000 7f275995a700 1 --2- 172.21.15.52:0/1543488198 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555b83b06030 0x555b83b12210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:46.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.362+0000 7f2758958700 1 -- 172.21.15.52:0/1543488198 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555b83b0b470 msgr2=0x555b83b0b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:03:46.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.362+0000 7f2758958700 1 --2- 172.21.15.52:0/1543488198 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555b83b0b470 0x555b83b0b850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:46.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.362+0000 7f2758958700 1 -- 172.21.15.52:0/1543488198 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555b83b06030 msgr2=0x555b83b12210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:03:46.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.362+0000 7f2758958700 1 --2- 172.21.15.52:0/1543488198 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555b83b06030 0x555b83b12210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:46.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.362+0000 7f2758958700 1 -- 172.21.15.52:0/1543488198 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b839a18d0 con 0x555b83b05670 2024-06-10T03:03:46.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.363+0000 7f2758958700 1 --2- 172.21.15.52:0/1543488198 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555b83b05670 0x555b83b05af0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f2744098c80 tx=0x7f27440228a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f97980a54a10d3f server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:03:46.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.363+0000 7f273f7fe700 1 -- 172.21.15.52:0/1543488198 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27440a0bd0 con 0x555b83b05670 2024-06-10T03:03:46.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.363+0000 7f273f7fe700 1 -- 172.21.15.52:0/1543488198 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27440a5400 con 0x555b83b05670 2024-06-10T03:03:46.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.363+0000 7f273f7fe700 1 -- 172.21.15.52:0/1543488198 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f274409cc80 con 0x555b83b05670 2024-06-10T03:03:46.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.363+0000 7f2765c40b80 1 -- 172.21.15.52:0/1543488198 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555b83b05670 msgr2=0x555b83b05af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:03:46.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.363+0000 7f2765c40b80 1 --2- 172.21.15.52:0/1543488198 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555b83b05670 0x555b83b05af0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f2744098c80 tx=0x7f27440228a0 comp rx=0 tx=0).stop 2024-06-10T03:03:46.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.364+0000 7f2765c40b80 1 -- 172.21.15.52:0/1543488198 shutdown_connections 2024-06-10T03:03:46.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.364+0000 7f2765c40b80 1 --2- 172.21.15.52:0/1543488198 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555b83b06030 0x555b83b12210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:46.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.364+0000 7f2765c40b80 1 --2- 172.21.15.52:0/1543488198 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555b83b0b470 0x555b83b0b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:46.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.364+0000 7f2765c40b80 1 --2- 172.21.15.52:0/1543488198 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555b83b05670 0x555b83b05af0 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:46.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.364+0000 7f2765c40b80 1 -- 172.21.15.52:0/1543488198 >> 172.21.15.52:0/1543488198 conn(0x555b83a2e080 msgr2=0x555b83afc220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:03:46.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.364+0000 7f2765c40b80 1 -- 172.21.15.52:0/1543488198 shutdown_connections 2024-06-10T03:03:46.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.364+0000 7f2765c40b80 1 -- 172.21.15.52:0/1543488198 wait complete. 2024-06-10T03:03:46.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.364+0000 7f2765c40b80 1 -- start start 2024-06-10T03:03:46.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.364+0000 7f2765c40b80 1 Processor -- start 2024-06-10T03:03:46.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.365+0000 7f2765c40b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555b83b05670 0x555b83b14c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:46.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.365+0000 7f2765c40b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555b83b06030 0x555b83b15160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:46.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.365+0000 7f2765c40b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555b83b0b470 0x555b83b0f040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:46.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.365+0000 7f2765c40b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x555b839f2fc0 con 0x555b83b06030 2024-06-10T03:03:46.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.365+0000 7f2765c40b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x555b83a4e810 con 0x555b83b0b470 2024-06-10T03:03:46.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.365+0000 7f2765c40b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x555b839f3f60 con 0x555b83b05670 2024-06-10T03:03:46.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.365+0000 7f2765c40b80 10 client.0 _ll_register_callbacks cb 0x555b839e85b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T03:03:46.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.365+0000 7f2758958700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555b83b06030 0x555b83b15160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:46.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.365+0000 7f2758958700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555b83b06030 0x555b83b15160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58952/0 (socket says 172.21.15.52:58952) 2024-06-10T03:03:46.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.365+0000 7f2758958700 1 -- 172.21.15.52:0/4052742472 learned_addr learned my addr 172.21.15.52:0/4052742472 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:03:46.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.365+0000 7f275995a700 1 --2- 172.21.15.52:0/4052742472 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555b83b0b470 0x555b83b0f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:46.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.365+0000 7f2759159700 1 --2- 172.21.15.52:0/4052742472 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555b83b05670 0x555b83b14c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:46.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.365+0000 7f2758958700 1 -- 172.21.15.52:0/4052742472 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555b83b05670 msgr2=0x555b83b14c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:03:46.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.365+0000 7f2758958700 1 --2- 172.21.15.52:0/4052742472 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555b83b05670 0x555b83b14c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:46.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.365+0000 7f2758958700 1 -- 172.21.15.52:0/4052742472 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555b83b0b470 msgr2=0x555b83b0f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:03:46.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.365+0000 7f2758958700 1 --2- 172.21.15.52:0/4052742472 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555b83b0b470 0x555b83b0f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:46.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.365+0000 7f2758958700 1 -- 172.21.15.52:0/4052742472 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b83a18750 con 0x555b83b06030 2024-06-10T03:03:46.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.365+0000 7f2758958700 1 --2- 172.21.15.52:0/4052742472 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555b83b06030 0x555b83b15160 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f274409c7b0 tx=0x7f2744009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5aa6ff3fa8e4ec16 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:03:46.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.365+0000 7f273cff9700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-10T03:03:46.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.365+0000 7f273cff9700 1 -- 172.21.15.52:0/4052742472 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27440094b0 con 0x555b83b06030 2024-06-10T03:03:46.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.366+0000 7f273cff9700 1 -- 172.21.15.52:0/4052742472 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f2744087180 con 0x555b83b06030 2024-06-10T03:03:46.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.366+0000 7f2765c40b80 1 -- 172.21.15.52:0/4052742472 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f27440a5400 con 0x555b83b06030 2024-06-10T03:03:46.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.366+0000 7f273cff9700 1 -- 172.21.15.52:0/4052742472 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f27440a0bd0 con 0x555b83b06030 2024-06-10T03:03:46.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.366+0000 7f2765c40b80 10 client.16899 fetch_fsmap learned FSMap version 45 2024-06-10T03:03:46.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.366+0000 7f2765c40b80 1 -- 172.21.15.52:0/4052742472 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x555b83b1c980 con 0x555b83b06030 2024-06-10T03:03:46.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.366+0000 7f273cff9700 1 -- 172.21.15.52:0/4052742472 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27440a5400 con 0x555b83b06030 2024-06-10T03:03:46.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.366+0000 7f273cff9700 1 -- 172.21.15.52:0/4052742472 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 45) ==== 8406+0+0 (secure 0 0 0) 0x7f274409db50 con 0x555b83b06030 2024-06-10T03:03:46.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.366+0000 7f2765c40b80 10 client.16899 fetch_fsmap finished waiting for FSMap version 45 2024-06-10T03:03:46.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.366+0000 7f2765c40b80 10 client.16899 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-06-10T03:03:46.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.366+0000 7f2765c40b80 20 client.16899 populate_metadata read hostname 'smithi052' 2024-06-10T03:03:46.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.366+0000 7f2765c40b80 1 --2- 172.21.15.52:0/4052742472 >> [v2:172.21.15.120:6836/4132440952,v1:172.21.15.120:6837/4132440952] conn(0x555b83b1efc0 0x555b83b213e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:46.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.366+0000 7f2765c40b80 4 client.16899 mds_command: new command op to 24428 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T03:03:46.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.366+0000 7f2765c40b80 1 -- 172.21.15.52:0/4052742472 --> [v2:172.21.15.120:6836/4132440952,v1:172.21.15.120:6837/4132440952] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555b83a5fd50 con 0x555b83b1efc0 2024-06-10T03:03:46.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.366+0000 7f2759159700 1 --2- 172.21.15.52:0/4052742472 >> [v2:172.21.15.120:6836/4132440952,v1:172.21.15.120:6837/4132440952] conn(0x555b83b1efc0 0x555b83b213e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:46.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.367+0000 7f2759159700 1 --2- 172.21.15.52:0/4052742472 >> [v2:172.21.15.120:6836/4132440952,v1:172.21.15.120:6837/4132440952] conn(0x555b83b1efc0 0x555b83b213e0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=da253610996fc42b server_cookie=358e552050b4f721 in_seq=0 out_seq=0 2024-06-10T03:03:46.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.367+0000 7f273cff9700 10 client.16899 ms_handle_connect on v2:172.21.15.120:6836/4132440952 2024-06-10T03:03:46.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.368+0000 7f273cff9700 1 -- 172.21.15.52:0/4052742472 <== mds.4 v2:172.21.15.120:6836/4132440952 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x555b83a5fd50 con 0x555b83b1efc0 2024-06-10T03:03:46.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.368+0000 7f273cff9700 10 client.16899 handle_command_reply: tid=0 2024-06-10T03:03:46.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.368+0000 7f2765c40b80 1 client.16899 shutdown 2024-06-10T03:03:46.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.368+0000 7f2765c40b80 1 -- 172.21.15.52:0/4052742472 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555b83b06030 msgr2=0x555b83b15160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:03:46.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.368+0000 7f2765c40b80 1 --2- 172.21.15.52:0/4052742472 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555b83b06030 0x555b83b15160 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f274409c7b0 tx=0x7f2744009ca0 comp rx=0 tx=0).stop 2024-06-10T03:03:46.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.368+0000 7f2765c40b80 1 -- 172.21.15.52:0/4052742472 shutdown_connections 2024-06-10T03:03:46.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.368+0000 7f2765c40b80 1 --2- 172.21.15.52:0/4052742472 >> [v2:172.21.15.120:6836/4132440952,v1:172.21.15.120:6837/4132440952] conn(0x555b83b1efc0 0x555b83b213e0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:46.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.368+0000 7f2765c40b80 1 --2- 172.21.15.52:0/4052742472 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555b83b0b470 0x555b83b0f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:46.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.368+0000 7f2765c40b80 1 --2- 172.21.15.52:0/4052742472 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555b83b05670 0x555b83b14c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:46.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.368+0000 7f2765c40b80 1 --2- 172.21.15.52:0/4052742472 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555b83b06030 0x555b83b15160 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:46.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.368+0000 7f2765c40b80 1 -- 172.21.15.52:0/4052742472 >> 172.21.15.52:0/4052742472 conn(0x555b83a2e080 msgr2=0x555b83afb700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:03:46.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.368+0000 7f273cff9700 0 client.16899 ms_handle_reset on v2:172.21.15.120:6836/4132440952 2024-06-10T03:03:46.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.368+0000 7f2765c40b80 1 -- 172.21.15.52:0/4052742472 shutdown_connections 2024-06-10T03:03:46.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.368+0000 7f2765c40b80 1 -- 172.21.15.52:0/4052742472 wait complete. 2024-06-10T03:03:46.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.368+0000 7f2765c40b80 20 client.16899 trim_cache size 0 max 16384 2024-06-10T03:03:46.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.384+0000 7f2765c40b80 1 Processor -- start 2024-06-10T03:03:46.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.384+0000 7f2765c40b80 1 -- start start 2024-06-10T03:03:46.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.384+0000 7f2765c40b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555b83b05670 0x555b83b0f180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:46.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.384+0000 7f2765c40b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555b83b06030 0x555b83b0e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:46.386 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.384+0000 7f2765c40b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555b83b0b470 0x555b83afbe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:46.386 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.384+0000 7f2765c40b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f27541ac0c0 con 0x555b83b05670 2024-06-10T03:03:46.386 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.384+0000 7f2765c40b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f27541adde0 con 0x555b83b0b470 2024-06-10T03:03:46.386 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.384+0000 7f2765c40b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f27541ac7f0 con 0x555b83b06030 2024-06-10T03:03:46.386 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.384+0000 7f2759159700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555b83b05670 0x555b83b0f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:46.386 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.384+0000 7f2759159700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555b83b05670 0x555b83b0f180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58956/0 (socket says 172.21.15.52:58956) 2024-06-10T03:03:46.386 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.384+0000 7f2759159700 1 -- 172.21.15.52:0/2757826097 learned_addr learned my addr 172.21.15.52:0/2757826097 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:03:46.386 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.384+0000 7f275995a700 1 --2- 172.21.15.52:0/2757826097 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555b83b0b470 0x555b83afbe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:46.386 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.384+0000 7f2758958700 1 --2- 172.21.15.52:0/2757826097 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555b83b06030 0x555b83b0e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:46.386 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.384+0000 7f2759159700 1 -- 172.21.15.52:0/2757826097 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555b83b06030 msgr2=0x555b83b0e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:03:46.386 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.385+0000 7f2759159700 1 --2- 172.21.15.52:0/2757826097 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555b83b06030 0x555b83b0e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:46.386 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.385+0000 7f2759159700 1 -- 172.21.15.52:0/2757826097 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555b83b0b470 msgr2=0x555b83afbe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:03:46.387 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.385+0000 7f2759159700 1 --2- 172.21.15.52:0/2757826097 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555b83b0b470 0x555b83afbe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:46.387 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.385+0000 7f2759159700 1 -- 172.21.15.52:0/2757826097 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b83b1b980 con 0x555b83b05670 2024-06-10T03:03:46.387 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.385+0000 7f275995a700 1 --2- 172.21.15.52:0/2757826097 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555b83b0b470 0x555b83afbe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T03:03:46.387 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.385+0000 7f2758958700 1 --2- 172.21.15.52:0/2757826097 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555b83b06030 0x555b83b0e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T03:03:46.387 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.385+0000 7f2759159700 1 --2- 172.21.15.52:0/2757826097 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555b83b05670 0x555b83b0f180 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f275000ed90 tx=0x7f2750014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5997c7c3911a2b82 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:03:46.387 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.385+0000 7f273effd700 1 -- 172.21.15.52:0/2757826097 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f275000ee70 con 0x555b83b05670 2024-06-10T03:03:46.387 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.385+0000 7f273effd700 1 -- 172.21.15.52:0/2757826097 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2750017040 con 0x555b83b05670 2024-06-10T03:03:46.387 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.385+0000 7f273effd700 1 -- 172.21.15.52:0/2757826097 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27500056d0 con 0x555b83b05670 2024-06-10T03:03:46.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.386+0000 7f2765c40b80 1 -- 172.21.15.52:0/2757826097 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555b83b05670 msgr2=0x555b83b0f180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:03:46.395 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.386+0000 7f2765c40b80 1 --2- 172.21.15.52:0/2757826097 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555b83b05670 0x555b83b0f180 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f275000ed90 tx=0x7f2750014000 comp rx=0 tx=0).stop 2024-06-10T03:03:46.395 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.386+0000 7f2765c40b80 1 -- 172.21.15.52:0/2757826097 shutdown_connections 2024-06-10T03:03:46.395 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.386+0000 7f2765c40b80 1 --2- 172.21.15.52:0/2757826097 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555b83b0b470 0x555b83afbe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:46.395 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.386+0000 7f2765c40b80 1 --2- 172.21.15.52:0/2757826097 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555b83b05670 0x555b83b0f180 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:46.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.386+0000 7f2765c40b80 1 --2- 172.21.15.52:0/2757826097 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555b83b06030 0x555b83b0e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:46.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.386+0000 7f2765c40b80 1 -- 172.21.15.52:0/2757826097 >> 172.21.15.52:0/2757826097 conn(0x555b83a2e080 msgr2=0x555b83b14f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:03:46.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.386+0000 7f2765c40b80 1 -- 172.21.15.52:0/2757826097 shutdown_connections 2024-06-10T03:03:46.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.386+0000 7f2765c40b80 1 -- 172.21.15.52:0/2757826097 wait complete. 2024-06-10T03:03:46.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.386+0000 7f2765c40b80 1 -- start start 2024-06-10T03:03:46.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.386+0000 7f2765c40b80 1 Processor -- start 2024-06-10T03:03:46.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.387+0000 7f2765c40b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555b83b05670 0x555b83b1b420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:46.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.387+0000 7f2765c40b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555b83b06030 0x555b83b17950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:46.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.387+0000 7f2765c40b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555b83b0b470 0x555b83b18090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:46.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.387+0000 7f2765c40b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f27541ad650 con 0x555b83b0b470 2024-06-10T03:03:46.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.387+0000 7f2765c40b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f27541acf50 con 0x555b83b06030 2024-06-10T03:03:46.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.387+0000 7f2765c40b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f27541aa060 con 0x555b83b05670 2024-06-10T03:03:46.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.387+0000 7f2765c40b80 10 client.0 _ll_register_callbacks cb 0x555b839e85b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T03:03:46.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.387+0000 7f275995a700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555b83b0b470 0x555b83b18090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:46.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.387+0000 7f275995a700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555b83b0b470 0x555b83b18090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58960/0 (socket says 172.21.15.52:58960) 2024-06-10T03:03:46.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.387+0000 7f275995a700 1 -- 172.21.15.52:0/2823382635 learned_addr learned my addr 172.21.15.52:0/2823382635 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:03:46.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.387+0000 7f2758958700 1 --2- 172.21.15.52:0/2823382635 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555b83b06030 0x555b83b17950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:46.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.387+0000 7f2759159700 1 --2- 172.21.15.52:0/2823382635 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555b83b05670 0x555b83b1b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:46.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.387+0000 7f275995a700 1 -- 172.21.15.52:0/2823382635 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555b83b05670 msgr2=0x555b83b1b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:03:46.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.387+0000 7f275995a700 1 --2- 172.21.15.52:0/2823382635 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555b83b05670 0x555b83b1b420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:46.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.387+0000 7f275995a700 1 -- 172.21.15.52:0/2823382635 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555b83b06030 msgr2=0x555b83b17950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:03:46.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.387+0000 7f275995a700 1 --2- 172.21.15.52:0/2823382635 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555b83b06030 0x555b83b17950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:46.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.387+0000 7f275995a700 1 -- 172.21.15.52:0/2823382635 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b83b87c80 con 0x555b83b0b470 2024-06-10T03:03:46.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.387+0000 7f275995a700 1 --2- 172.21.15.52:0/2823382635 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555b83b0b470 0x555b83b18090 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f274c011c00 tx=0x7f274c011f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=118442b2be5d345d server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:03:46.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.387+0000 7f273cff9700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-10T03:03:46.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.387+0000 7f273cff9700 1 -- 172.21.15.52:0/2823382635 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f274c01f060 con 0x555b83b0b470 2024-06-10T03:03:46.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.388+0000 7f273cff9700 1 -- 172.21.15.52:0/2823382635 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f274c00fb80 con 0x555b83b0b470 2024-06-10T03:03:46.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.388+0000 7f2765c40b80 1 -- 172.21.15.52:0/2823382635 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2750017040 con 0x555b83b0b470 2024-06-10T03:03:46.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.388+0000 7f273cff9700 1 -- 172.21.15.52:0/2823382635 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f274c026050 con 0x555b83b0b470 2024-06-10T03:03:46.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.388+0000 7f2765c40b80 10 client.16905 fetch_fsmap learned FSMap version 45 2024-06-10T03:03:46.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.388+0000 7f2765c40b80 1 -- 172.21.15.52:0/2823382635 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x555b83b88db0 con 0x555b83b0b470 2024-06-10T03:03:46.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.388+0000 7f273cff9700 1 -- 172.21.15.52:0/2823382635 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2750017040 con 0x555b83b0b470 2024-06-10T03:03:46.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.388+0000 7f273cff9700 1 -- 172.21.15.52:0/2823382635 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 45) ==== 8406+0+0 (secure 0 0 0) 0x7f274c019370 con 0x555b83b0b470 2024-06-10T03:03:46.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.388+0000 7f2765c40b80 10 client.16905 fetch_fsmap finished waiting for FSMap version 45 2024-06-10T03:03:46.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.388+0000 7f2765c40b80 10 client.16905 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-06-10T03:03:46.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.388+0000 7f2765c40b80 20 client.16905 populate_metadata read hostname 'smithi052' 2024-06-10T03:03:46.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.388+0000 7f2765c40b80 1 --2- 172.21.15.52:0/2823382635 >> [v2:172.21.15.120:6836/4132440952,v1:172.21.15.120:6837/4132440952] conn(0x555b83b89580 0x555b83b8b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:46.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.388+0000 7f2765c40b80 4 client.16905 mds_command: new command op to 24428 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T03:03:46.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.388+0000 7f2765c40b80 1 -- 172.21.15.52:0/2823382635 --> [v2:172.21.15.120:6836/4132440952,v1:172.21.15.120:6837/4132440952] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555b83a5a2d0 con 0x555b83b89580 2024-06-10T03:03:46.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.389+0000 7f2759159700 1 --2- 172.21.15.52:0/2823382635 >> [v2:172.21.15.120:6836/4132440952,v1:172.21.15.120:6837/4132440952] conn(0x555b83b89580 0x555b83b8b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:46.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.389+0000 7f273cff9700 10 client.16905 ms_handle_connect on v2:172.21.15.120:6836/4132440952 2024-06-10T03:03:46.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.389+0000 7f2759159700 1 --2- 172.21.15.52:0/2823382635 >> [v2:172.21.15.120:6836/4132440952,v1:172.21.15.120:6837/4132440952] conn(0x555b83b89580 0x555b83b8b9a0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=8dc1ee3536f65745 server_cookie=7f1417fd8417b405 in_seq=0 out_seq=0 2024-06-10T03:03:46.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.389+0000 7f273cff9700 1 -- 172.21.15.52:0/2823382635 <== mds.4 v2:172.21.15.120:6836/4132440952 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555b83a5a2d0 con 0x555b83b89580 2024-06-10T03:03:46.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.389+0000 7f273cff9700 10 client.16905 handle_command_reply: tid=0 2024-06-10T03:03:46.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.389+0000 7f2765c40b80 1 client.16905 shutdown 2024-06-10T03:03:46.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.389+0000 7f2765c40b80 1 -- 172.21.15.52:0/2823382635 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555b83b0b470 msgr2=0x555b83b18090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:03:46.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.389+0000 7f2765c40b80 1 --2- 172.21.15.52:0/2823382635 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555b83b0b470 0x555b83b18090 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f274c011c00 tx=0x7f274c011f10 comp rx=0 tx=0).stop 2024-06-10T03:03:46.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.389+0000 7f2765c40b80 1 -- 172.21.15.52:0/2823382635 shutdown_connections 2024-06-10T03:03:46.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.389+0000 7f2765c40b80 1 --2- 172.21.15.52:0/2823382635 >> [v2:172.21.15.120:6836/4132440952,v1:172.21.15.120:6837/4132440952] conn(0x555b83b89580 0x555b83b8b9a0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:46.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.389+0000 7f2765c40b80 1 --2- 172.21.15.52:0/2823382635 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555b83b0b470 0x555b83b18090 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:46.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.389+0000 7f2765c40b80 1 --2- 172.21.15.52:0/2823382635 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555b83b05670 0x555b83b1b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:46.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.389+0000 7f2765c40b80 1 --2- 172.21.15.52:0/2823382635 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555b83b06030 0x555b83b17950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:46.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.389+0000 7f2765c40b80 1 -- 172.21.15.52:0/2823382635 >> 172.21.15.52:0/2823382635 conn(0x555b83a2e080 msgr2=0x555b83b0a2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:03:46.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.389+0000 7f273cff9700 0 client.16905 ms_handle_reset on v2:172.21.15.120:6836/4132440952 2024-06-10T03:03:46.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.390+0000 7f2765c40b80 1 -- 172.21.15.52:0/2823382635 shutdown_connections 2024-06-10T03:03:46.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.390+0000 7f2765c40b80 1 -- 172.21.15.52:0/2823382635 wait complete. 2024-06-10T03:03:46.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.390+0000 7f2765c40b80 20 client.16905 trim_cache size 0 max 16384 2024-06-10T03:03:46.401 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-06-10T03:03:46.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.390+0000 7f275b3bd700 1 -- 172.21.15.52:0/2274453913 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f27400a2ca0 msgr2=0x7f27400a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:03:46.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.390+0000 7f275b3bd700 1 --2- 172.21.15.52:0/2274453913 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f27400a2ca0 0x7f27400a5160 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f2750005950 tx=0x7f27500058e0 comp rx=0 tx=0).stop 2024-06-10T03:03:46.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.390+0000 7f275b3bd700 1 -- 172.21.15.52:0/2274453913 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2754102ed0 msgr2=0x7f275419ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:03:46.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.390+0000 7f275b3bd700 1 --2- 172.21.15.52:0/2274453913 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2754102ed0 0x7f275419ede0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f2744000c00 tx=0x7f27440047b0 comp rx=0 tx=0).stop 2024-06-10T03:03:46.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.390+0000 7f275b3bd700 1 -- 172.21.15.52:0/2274453913 shutdown_connections 2024-06-10T03:03:46.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.390+0000 7f275b3bd700 1 --2- 172.21.15.52:0/2274453913 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2754108510 0x7f2754198a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:46.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.390+0000 7f275b3bd700 1 --2- 172.21.15.52:0/2274453913 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2754102510 0x7f275419e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:46.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.390+0000 7f275b3bd700 1 --2- 172.21.15.52:0/2274453913 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f27400a2ca0 0x7f27400a5160 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:46.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.390+0000 7f275b3bd700 1 --2- 172.21.15.52:0/2274453913 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2754102ed0 0x7f275419ede0 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:46.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.390+0000 7f275b3bd700 1 -- 172.21.15.52:0/2274453913 >> 172.21.15.52:0/2274453913 conn(0x7f27540795b0 msgr2=0x7f27541013a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:03:46.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.391+0000 7f275b3bd700 1 -- 172.21.15.52:0/2274453913 shutdown_connections 2024-06-10T03:03:46.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:46.391+0000 7f275b3bd700 1 -- 172.21.15.52:0/2274453913 wait complete. 2024-06-10T03:03:46.638 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:03:46 smithi120 ceph-mon[36634]: pgmap v721: 129 pgs: 129 active+clean; 228 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-10T03:03:46.704 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:03:46 smithi188 ceph-mon[36599]: pgmap v721: 129 pgs: 129 active+clean; 228 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-10T03:03:47.107 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-06-10T03:03:47.361 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T03:03:48.638 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:03:48 smithi120 ceph-mon[36634]: pgmap v722: 129 pgs: 129 active+clean; 228 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-10T03:03:48.653 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:03:48 smithi052 ceph-mon[32362]: pgmap v722: 129 pgs: 129 active+clean; 228 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-10T03:03:48.704 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:03:48 smithi188 ceph-mon[36599]: pgmap v722: 129 pgs: 129 active+clean; 228 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-10T03:03:49.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.015+0000 7f0194181700 1 -- 172.21.15.52:0/1910903173 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f018c102740 msgr2=0x7f018c102b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:03:49.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.015+0000 7f0194181700 1 --2- 172.21.15.52:0/1910903173 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f018c102740 0x7f018c102b20 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f01880099c0 tx=0x7f0188009cd0 comp rx=0 tx=0).stop 2024-06-10T03:03:49.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.015+0000 7f0194181700 1 -- 172.21.15.52:0/1910903173 shutdown_connections 2024-06-10T03:03:49.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.015+0000 7f0194181700 1 --2- 172.21.15.52:0/1910903173 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f018c107840 0x7f018c10f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:49.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.015+0000 7f0194181700 1 --2- 172.21.15.52:0/1910903173 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f018c102740 0x7f018c102b20 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:49.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.015+0000 7f0194181700 1 --2- 172.21.15.52:0/1910903173 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f018c1030f0 0x7f018c107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:49.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.015+0000 7f0194181700 1 -- 172.21.15.52:0/1910903173 >> 172.21.15.52:0/1910903173 conn(0x7f018c0795f0 msgr2=0x7f018c079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:03:49.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.015+0000 7f0194181700 1 -- 172.21.15.52:0/1910903173 shutdown_connections 2024-06-10T03:03:49.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.015+0000 7f0194181700 1 -- 172.21.15.52:0/1910903173 wait complete. 2024-06-10T03:03:49.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.016+0000 7f0194181700 1 Processor -- start 2024-06-10T03:03:49.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.016+0000 7f0194181700 1 -- start start 2024-06-10T03:03:49.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.016+0000 7f0194181700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f018c102740 0x7f018c19e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:49.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.016+0000 7f0194181700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f018c1030f0 0x7f018c19ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:49.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.016+0000 7f0194181700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f018c107840 0x7f018c1989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:49.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.016+0000 7f0194181700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f018c112760 con 0x7f018c102740 2024-06-10T03:03:49.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.016+0000 7f0194181700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f018c1125e0 con 0x7f018c1030f0 2024-06-10T03:03:49.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.016+0000 7f0194181700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f018c1128e0 con 0x7f018c107840 2024-06-10T03:03:49.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.016+0000 7f0191f1d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f018c102740 0x7f018c19e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:49.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.016+0000 7f0191f1d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f018c102740 0x7f018c19e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58976/0 (socket says 172.21.15.52:58976) 2024-06-10T03:03:49.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.016+0000 7f0191f1d700 1 -- 172.21.15.52:0/444455414 learned_addr learned my addr 172.21.15.52:0/444455414 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:03:49.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.016+0000 7f019171c700 1 --2- 172.21.15.52:0/444455414 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f018c1030f0 0x7f018c19ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:49.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.016+0000 7f019271e700 1 --2- 172.21.15.52:0/444455414 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f018c107840 0x7f018c1989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:49.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.017+0000 7f0191f1d700 1 -- 172.21.15.52:0/444455414 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f018c107840 msgr2=0x7f018c1989e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:03:49.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.017+0000 7f0191f1d700 1 --2- 172.21.15.52:0/444455414 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f018c107840 0x7f018c1989e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:49.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.017+0000 7f0191f1d700 1 -- 172.21.15.52:0/444455414 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f018c1030f0 msgr2=0x7f018c19ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:03:49.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.017+0000 7f0191f1d700 1 --2- 172.21.15.52:0/444455414 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f018c1030f0 0x7f018c19ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:49.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.017+0000 7f0191f1d700 1 -- 172.21.15.52:0/444455414 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f018c199140 con 0x7f018c102740 2024-06-10T03:03:49.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.017+0000 7f0191f1d700 1 --2- 172.21.15.52:0/444455414 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f018c102740 0x7f018c19e820 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f0188006010 tx=0x7f0188009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:03:49.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.017+0000 7f0182ffd700 1 -- 172.21.15.52:0/444455414 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0188026070 con 0x7f018c102740 2024-06-10T03:03:49.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.017+0000 7f0182ffd700 1 -- 172.21.15.52:0/444455414 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0188004cd0 con 0x7f018c102740 2024-06-10T03:03:49.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.017+0000 7f0194181700 1 -- 172.21.15.52:0/444455414 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f018c1993d0 con 0x7f018c102740 2024-06-10T03:03:49.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.017+0000 7f0182ffd700 1 -- 172.21.15.52:0/444455414 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0188003ba0 con 0x7f018c102740 2024-06-10T03:03:49.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.017+0000 7f0194181700 1 -- 172.21.15.52:0/444455414 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f018c1a56f0 con 0x7f018c102740 2024-06-10T03:03:49.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.019+0000 7f0182ffd700 1 -- 172.21.15.52:0/444455414 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f0188029070 con 0x7f018c102740 2024-06-10T03:03:49.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.020+0000 7f0182ffd700 1 --2- 172.21.15.52:0/444455414 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f01780a2c20 0x7f01780a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:49.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.020+0000 7f019171c700 1 --2- 172.21.15.52:0/444455414 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f01780a2c20 0x7f01780a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:49.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.020+0000 7f0182ffd700 1 -- 172.21.15.52:0/444455414 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f01880cfd40 con 0x7f018c102740 2024-06-10T03:03:49.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.021+0000 7f019171c700 1 --2- 172.21.15.52:0/444455414 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f01780a2c20 0x7f01780a50e0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f018c101f60 tx=0x7f017c00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:03:49.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.021+0000 7f019ea04b80 1 Processor -- start 2024-06-10T03:03:49.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.021+0000 7f019ea04b80 1 -- start start 2024-06-10T03:03:49.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.021+0000 7f019ea04b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5579fc9f2f60 0x5579fc9f3340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:49.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.021+0000 7f019ea04b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5579fc9f3880 0x5579fc9ed260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:49.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.021+0000 7f019ea04b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5579fc9ed7a0 0x5579fc9edbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:49.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.021+0000 7f019ea04b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5579fc8dff60 con 0x5579fc9f3880 2024-06-10T03:03:49.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.021+0000 7f019ea04b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5579fc93e850 con 0x5579fc9f2f60 2024-06-10T03:03:49.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.021+0000 7f019ea04b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5579fc8a6460 con 0x5579fc9ed7a0 2024-06-10T03:03:49.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.021+0000 7f019171c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5579fc9f3880 0x5579fc9ed260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:49.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.021+0000 7f019171c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5579fc9f3880 0x5579fc9ed260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58986/0 (socket says 172.21.15.52:58986) 2024-06-10T03:03:49.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.021+0000 7f019171c700 1 -- 172.21.15.52:0/2591586252 learned_addr learned my addr 172.21.15.52:0/2591586252 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:03:49.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.021+0000 7f0191f1d700 1 --2- 172.21.15.52:0/2591586252 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5579fc9f2f60 0x5579fc9f3340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:49.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.021+0000 7f019271e700 1 --2- 172.21.15.52:0/2591586252 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5579fc9ed7a0 0x5579fc9edbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:49.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.022+0000 7f019171c700 1 -- 172.21.15.52:0/2591586252 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5579fc9ed7a0 msgr2=0x5579fc9edbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:03:49.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.022+0000 7f019171c700 1 --2- 172.21.15.52:0/2591586252 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5579fc9ed7a0 0x5579fc9edbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:49.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.022+0000 7f019171c700 1 -- 172.21.15.52:0/2591586252 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5579fc9f2f60 msgr2=0x5579fc9f3340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:03:49.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.022+0000 7f019171c700 1 --2- 172.21.15.52:0/2591586252 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5579fc9f2f60 0x5579fc9f3340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:49.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.022+0000 7f019171c700 1 -- 172.21.15.52:0/2591586252 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5579fc88d8d0 con 0x5579fc9f3880 2024-06-10T03:03:49.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.022+0000 7f0191f1d700 1 --2- 172.21.15.52:0/2591586252 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5579fc9f2f60 0x5579fc9f3340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T03:03:49.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.022+0000 7f019171c700 1 --2- 172.21.15.52:0/2591586252 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5579fc9f3880 0x5579fc9ed260 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f017c010f70 tx=0x7f017c013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb8a7d99b4513c24 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:03:49.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.022+0000 7f0177fff700 1 -- 172.21.15.52:0/2591586252 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f017c00be60 con 0x5579fc9f3880 2024-06-10T03:03:49.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.022+0000 7f0177fff700 1 -- 172.21.15.52:0/2591586252 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f017c01eb00 con 0x5579fc9f3880 2024-06-10T03:03:49.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.022+0000 7f0177fff700 1 -- 172.21.15.52:0/2591586252 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f017c01cbf0 con 0x5579fc9f3880 2024-06-10T03:03:49.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.023+0000 7f019ea04b80 1 -- 172.21.15.52:0/2591586252 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5579fc9f3880 msgr2=0x5579fc9ed260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:03:49.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.023+0000 7f019ea04b80 1 --2- 172.21.15.52:0/2591586252 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5579fc9f3880 0x5579fc9ed260 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f017c010f70 tx=0x7f017c013000 comp rx=0 tx=0).stop 2024-06-10T03:03:49.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.023+0000 7f019ea04b80 1 -- 172.21.15.52:0/2591586252 shutdown_connections 2024-06-10T03:03:49.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.023+0000 7f019ea04b80 1 --2- 172.21.15.52:0/2591586252 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5579fc9ed7a0 0x5579fc9edbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:49.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.023+0000 7f019ea04b80 1 --2- 172.21.15.52:0/2591586252 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5579fc9f2f60 0x5579fc9f3340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:49.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.023+0000 7f019ea04b80 1 --2- 172.21.15.52:0/2591586252 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5579fc9f3880 0x5579fc9ed260 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:49.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.023+0000 7f019ea04b80 1 -- 172.21.15.52:0/2591586252 >> 172.21.15.52:0/2591586252 conn(0x5579fc91a080 msgr2=0x5579fc9e8220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:03:49.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.023+0000 7f019ea04b80 1 -- 172.21.15.52:0/2591586252 shutdown_connections 2024-06-10T03:03:49.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.023+0000 7f019ea04b80 1 -- 172.21.15.52:0/2591586252 wait complete. 2024-06-10T03:03:49.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.023+0000 7f019ea04b80 1 -- start start 2024-06-10T03:03:49.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.023+0000 7f019ea04b80 1 Processor -- start 2024-06-10T03:03:49.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.024+0000 7f019ea04b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5579fc9ed7a0 0x5579fc9f9600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:49.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.024+0000 7f019ea04b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5579fc9f2f60 0x5579fc9f6950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:49.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.024+0000 7f019ea04b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5579fc9f3880 0x5579fc9f6e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:49.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.024+0000 7f019ea04b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5579fc8defc0 con 0x5579fc9ed7a0 2024-06-10T03:03:49.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.024+0000 7f019ea04b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5579fc93a810 con 0x5579fc9f2f60 2024-06-10T03:03:49.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.024+0000 7f019ea04b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5579fc9fa260 con 0x5579fc9f3880 2024-06-10T03:03:49.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.024+0000 7f019ea04b80 10 client.0 _ll_register_callbacks cb 0x5579fc8d45b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T03:03:49.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.024+0000 7f0191f1d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5579fc9ed7a0 0x5579fc9f9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:49.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.024+0000 7f0191f1d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5579fc9ed7a0 0x5579fc9f9600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58994/0 (socket says 172.21.15.52:58994) 2024-06-10T03:03:49.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.024+0000 7f0191f1d700 1 -- 172.21.15.52:0/3999320776 learned_addr learned my addr 172.21.15.52:0/3999320776 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:03:49.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.024+0000 7f019171c700 1 --2- 172.21.15.52:0/3999320776 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5579fc9f2f60 0x5579fc9f6950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:49.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.024+0000 7f019271e700 1 --2- 172.21.15.52:0/3999320776 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5579fc9f3880 0x5579fc9f6e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:49.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.024+0000 7f0191f1d700 1 -- 172.21.15.52:0/3999320776 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5579fc9f3880 msgr2=0x5579fc9f6e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:03:49.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.024+0000 7f0191f1d700 1 --2- 172.21.15.52:0/3999320776 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5579fc9f3880 0x5579fc9f6e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:49.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.024+0000 7f0191f1d700 1 -- 172.21.15.52:0/3999320776 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5579fc9f2f60 msgr2=0x5579fc9f6950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:03:49.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.024+0000 7f0191f1d700 1 --2- 172.21.15.52:0/3999320776 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5579fc9f2f60 0x5579fc9f6950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:49.030 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.024+0000 7f0191f1d700 1 -- 172.21.15.52:0/3999320776 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5579fc904750 con 0x5579fc9ed7a0 2024-06-10T03:03:49.030 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.024+0000 7f0191f1d700 1 --2- 172.21.15.52:0/3999320776 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5579fc9ed7a0 0x5579fc9f9600 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f0188098680 tx=0x7f01880054a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e1846431efb47b9 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:03:49.030 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.024+0000 7f01757fa700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-10T03:03:49.030 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.024+0000 7f01757fa700 1 -- 172.21.15.52:0/3999320776 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01880bc270 con 0x5579fc9ed7a0 2024-06-10T03:03:49.030 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.025+0000 7f01757fa700 1 -- 172.21.15.52:0/3999320776 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f01880a4970 con 0x5579fc9ed7a0 2024-06-10T03:03:49.030 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.025+0000 7f019ea04b80 1 -- 172.21.15.52:0/3999320776 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f017c01eb00 con 0x5579fc9ed7a0 2024-06-10T03:03:49.030 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.025+0000 7f01757fa700 1 -- 172.21.15.52:0/3999320776 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f018809c920 con 0x5579fc9ed7a0 2024-06-10T03:03:49.030 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.025+0000 7f019ea04b80 10 client.16917 fetch_fsmap learned FSMap version 45 2024-06-10T03:03:49.030 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.025+0000 7f019ea04b80 1 -- 172.21.15.52:0/3999320776 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x5579fca049f0 con 0x5579fc9ed7a0 2024-06-10T03:03:49.030 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.025+0000 7f01757fa700 1 -- 172.21.15.52:0/3999320776 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f017c01eb00 con 0x5579fc9ed7a0 2024-06-10T03:03:49.030 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.025+0000 7f01757fa700 1 -- 172.21.15.52:0/3999320776 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 45) ==== 8406+0+0 (secure 0 0 0) 0x7f01880a0120 con 0x5579fc9ed7a0 2024-06-10T03:03:49.030 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.025+0000 7f019ea04b80 10 client.16917 fetch_fsmap finished waiting for FSMap version 45 2024-06-10T03:03:49.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.025+0000 7f019ea04b80 10 client.16917 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.b 2024-06-10T03:03:49.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.025+0000 7f019ea04b80 20 client.16917 populate_metadata read hostname 'smithi052' 2024-06-10T03:03:49.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.025+0000 7f019ea04b80 1 --2- 172.21.15.52:0/3999320776 >> [v2:172.21.15.120:6838/1672436487,v1:172.21.15.120:6839/1672436487] conn(0x5579fca06fb0 0x5579fca09400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:49.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.025+0000 7f019ea04b80 4 client.16917 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T03:03:49.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.025+0000 7f019ea04b80 1 -- 172.21.15.52:0/3999320776 --> [v2:172.21.15.120:6838/1672436487,v1:172.21.15.120:6839/1672436487] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5579fc94bd50 con 0x5579fca06fb0 2024-06-10T03:03:49.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.025+0000 7f019171c700 1 --2- 172.21.15.52:0/3999320776 >> [v2:172.21.15.120:6838/1672436487,v1:172.21.15.120:6839/1672436487] conn(0x5579fca06fb0 0x5579fca09400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:49.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.026+0000 7f019171c700 1 --2- 172.21.15.52:0/3999320776 >> [v2:172.21.15.120:6838/1672436487,v1:172.21.15.120:6839/1672436487] conn(0x5579fca06fb0 0x5579fca09400 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=e7734eab689a3f9f server_cookie=fd33f540c788fda6 in_seq=0 out_seq=0 2024-06-10T03:03:49.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.026+0000 7f01757fa700 10 client.16917 ms_handle_connect on v2:172.21.15.120:6838/1672436487 2024-06-10T03:03:49.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.027+0000 7f01757fa700 1 -- 172.21.15.52:0/3999320776 <== mds.3 v2:172.21.15.120:6838/1672436487 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5579fc94bd50 con 0x5579fca06fb0 2024-06-10T03:03:49.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.027+0000 7f01757fa700 10 client.16917 handle_command_reply: tid=0 2024-06-10T03:03:49.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.027+0000 7f019ea04b80 1 client.16917 shutdown 2024-06-10T03:03:49.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.027+0000 7f019ea04b80 1 -- 172.21.15.52:0/3999320776 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5579fc9ed7a0 msgr2=0x5579fc9f9600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:03:49.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.027+0000 7f019ea04b80 1 --2- 172.21.15.52:0/3999320776 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5579fc9ed7a0 0x5579fc9f9600 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f0188098680 tx=0x7f01880054a0 comp rx=0 tx=0).stop 2024-06-10T03:03:49.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.027+0000 7f019ea04b80 1 -- 172.21.15.52:0/3999320776 shutdown_connections 2024-06-10T03:03:49.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.027+0000 7f019ea04b80 1 --2- 172.21.15.52:0/3999320776 >> [v2:172.21.15.120:6838/1672436487,v1:172.21.15.120:6839/1672436487] conn(0x5579fca06fb0 0x5579fca09400 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:49.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.027+0000 7f019ea04b80 1 --2- 172.21.15.52:0/3999320776 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5579fc9f3880 0x5579fc9f6e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:49.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.027+0000 7f019ea04b80 1 --2- 172.21.15.52:0/3999320776 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5579fc9ed7a0 0x5579fc9f9600 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:49.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.027+0000 7f01757fa700 0 client.16917 ms_handle_reset on v2:172.21.15.120:6838/1672436487 2024-06-10T03:03:49.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.027+0000 7f019ea04b80 1 --2- 172.21.15.52:0/3999320776 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5579fc9f2f60 0x5579fc9f6950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:49.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.027+0000 7f019ea04b80 1 -- 172.21.15.52:0/3999320776 >> 172.21.15.52:0/3999320776 conn(0x5579fc91a080 msgr2=0x5579fc9e7700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:03:49.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.027+0000 7f019ea04b80 1 -- 172.21.15.52:0/3999320776 shutdown_connections 2024-06-10T03:03:49.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.027+0000 7f019ea04b80 1 -- 172.21.15.52:0/3999320776 wait complete. 2024-06-10T03:03:49.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.027+0000 7f019ea04b80 20 client.16917 trim_cache size 0 max 16384 2024-06-10T03:03:49.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.042+0000 7f019ea04b80 1 Processor -- start 2024-06-10T03:03:49.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.042+0000 7f019ea04b80 1 -- start start 2024-06-10T03:03:49.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.043+0000 7f019ea04b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5579fc9ed7a0 0x5579fc9ed260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:49.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.043+0000 7f019ea04b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5579fc9f2f60 0x5579fc9f9140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:49.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.043+0000 7f019ea04b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5579fc9f3880 0x5579fc9e7e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:49.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.043+0000 7f019ea04b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f018c1ae2c0 con 0x5579fc9ed7a0 2024-06-10T03:03:49.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.043+0000 7f019ea04b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f018c1affe0 con 0x5579fc9f3880 2024-06-10T03:03:49.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.043+0000 7f019ea04b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f018c1ae9f0 con 0x5579fc9f2f60 2024-06-10T03:03:49.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.043+0000 7f0191f1d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5579fc9ed7a0 0x5579fc9ed260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:49.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.043+0000 7f0191f1d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5579fc9ed7a0 0x5579fc9ed260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58998/0 (socket says 172.21.15.52:58998) 2024-06-10T03:03:49.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.043+0000 7f0191f1d700 1 -- 172.21.15.52:0/1048721538 learned_addr learned my addr 172.21.15.52:0/1048721538 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:03:49.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.043+0000 7f019171c700 1 --2- 172.21.15.52:0/1048721538 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5579fc9f2f60 0x5579fc9f9140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:49.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.043+0000 7f019271e700 1 --2- 172.21.15.52:0/1048721538 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5579fc9f3880 0x5579fc9e7e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:49.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.043+0000 7f0191f1d700 1 -- 172.21.15.52:0/1048721538 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5579fc9f2f60 msgr2=0x5579fc9f9140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:03:49.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.043+0000 7f0191f1d700 1 --2- 172.21.15.52:0/1048721538 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5579fc9f2f60 0x5579fc9f9140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:49.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.043+0000 7f0191f1d700 1 -- 172.21.15.52:0/1048721538 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5579fc9f3880 msgr2=0x5579fc9e7e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:03:49.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.043+0000 7f0191f1d700 1 --2- 172.21.15.52:0/1048721538 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5579fc9f3880 0x5579fc9e7e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:49.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.043+0000 7f0191f1d700 1 -- 172.21.15.52:0/1048721538 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5579fc9f3680 con 0x5579fc9ed7a0 2024-06-10T03:03:49.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.043+0000 7f0191f1d700 1 --2- 172.21.15.52:0/1048721538 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5579fc9ed7a0 0x5579fc9ed260 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f0188098ba0 tx=0x7f0188098570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b172431ec36cc5d server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:03:49.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.043+0000 7f01777fe700 1 -- 172.21.15.52:0/1048721538 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f018809c220 con 0x5579fc9ed7a0 2024-06-10T03:03:49.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.043+0000 7f01777fe700 1 -- 172.21.15.52:0/1048721538 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f01880a05d0 con 0x5579fc9ed7a0 2024-06-10T03:03:49.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.043+0000 7f01777fe700 1 -- 172.21.15.52:0/1048721538 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01880a4970 con 0x5579fc9ed7a0 2024-06-10T03:03:49.048 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.044+0000 7f019ea04b80 1 -- 172.21.15.52:0/1048721538 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5579fc9ed7a0 msgr2=0x5579fc9ed260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:03:49.048 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.044+0000 7f019ea04b80 1 --2- 172.21.15.52:0/1048721538 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5579fc9ed7a0 0x5579fc9ed260 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f0188098ba0 tx=0x7f0188098570 comp rx=0 tx=0).stop 2024-06-10T03:03:49.048 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.044+0000 7f019ea04b80 1 -- 172.21.15.52:0/1048721538 shutdown_connections 2024-06-10T03:03:49.048 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.044+0000 7f019ea04b80 1 --2- 172.21.15.52:0/1048721538 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5579fc9f3880 0x5579fc9e7e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:49.048 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.044+0000 7f019ea04b80 1 --2- 172.21.15.52:0/1048721538 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5579fc9ed7a0 0x5579fc9ed260 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:49.048 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.044+0000 7f019ea04b80 1 --2- 172.21.15.52:0/1048721538 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5579fc9f2f60 0x5579fc9f9140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:49.048 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.044+0000 7f019ea04b80 1 -- 172.21.15.52:0/1048721538 >> 172.21.15.52:0/1048721538 conn(0x5579fc91a080 msgr2=0x5579fc9f1a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:03:49.048 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.044+0000 7f019ea04b80 1 -- 172.21.15.52:0/1048721538 shutdown_connections 2024-06-10T03:03:49.048 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.044+0000 7f019ea04b80 1 -- 172.21.15.52:0/1048721538 wait complete. 2024-06-10T03:03:49.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.045+0000 7f019ea04b80 1 -- start start 2024-06-10T03:03:49.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.045+0000 7f019ea04b80 1 Processor -- start 2024-06-10T03:03:49.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.045+0000 7f019ea04b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5579fc9ed7a0 0x5579fc9ffc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:49.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.045+0000 7f019ea04b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5579fc9f2f60 0x5579fca0e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:49.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.045+0000 7f019ea04b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5579fc9f3880 0x5579fca0ea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:49.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.045+0000 7f019ea04b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f018c1af850 con 0x5579fc9f2f60 2024-06-10T03:03:49.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.045+0000 7f019ea04b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f018c1af150 con 0x5579fc9ed7a0 2024-06-10T03:03:49.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.045+0000 7f019ea04b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f018c1ac260 con 0x5579fc9f3880 2024-06-10T03:03:49.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.045+0000 7f019ea04b80 10 client.0 _ll_register_callbacks cb 0x5579fc8d45b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T03:03:49.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.045+0000 7f019171c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5579fc9f2f60 0x5579fca0e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:49.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.045+0000 7f019171c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5579fc9f2f60 0x5579fca0e530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:59010/0 (socket says 172.21.15.52:59010) 2024-06-10T03:03:49.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.045+0000 7f019171c700 1 -- 172.21.15.52:0/2366849407 learned_addr learned my addr 172.21.15.52:0/2366849407 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:03:49.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.045+0000 7f019271e700 1 --2- 172.21.15.52:0/2366849407 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5579fc9f3880 0x5579fca0ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:49.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.045+0000 7f0191f1d700 1 --2- 172.21.15.52:0/2366849407 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5579fc9ed7a0 0x5579fc9ffc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:49.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.046+0000 7f019171c700 1 -- 172.21.15.52:0/2366849407 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5579fc9f3880 msgr2=0x5579fca0ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:03:49.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.046+0000 7f019171c700 1 --2- 172.21.15.52:0/2366849407 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5579fc9f3880 0x5579fca0ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:49.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.046+0000 7f019171c700 1 -- 172.21.15.52:0/2366849407 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5579fc9ed7a0 msgr2=0x5579fc9ffc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:03:49.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.046+0000 7f019171c700 1 --2- 172.21.15.52:0/2366849407 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5579fc9ed7a0 0x5579fc9ffc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:49.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.046+0000 7f019171c700 1 -- 172.21.15.52:0/2366849407 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5579fca0f260 con 0x5579fc9f2f60 2024-06-10T03:03:49.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.046+0000 7f019171c700 1 --2- 172.21.15.52:0/2366849407 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5579fc9f2f60 0x5579fca0e530 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f017c01c720 tx=0x7f017c01c5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f38434630b19c17 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:03:49.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.046+0000 7f01757fa700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-10T03:03:49.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.046+0000 7f01757fa700 1 -- 172.21.15.52:0/2366849407 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f017c01cbd0 con 0x5579fc9f2f60 2024-06-10T03:03:49.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.046+0000 7f01757fa700 1 -- 172.21.15.52:0/2366849407 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f017c01cd70 con 0x5579fc9f2f60 2024-06-10T03:03:49.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.046+0000 7f019ea04b80 1 -- 172.21.15.52:0/2366849407 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f01880a05d0 con 0x5579fc9f2f60 2024-06-10T03:03:49.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.046+0000 7f01757fa700 1 -- 172.21.15.52:0/2366849407 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f017c045050 con 0x5579fc9f2f60 2024-06-10T03:03:49.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.047+0000 7f019ea04b80 10 client.16923 fetch_fsmap learned FSMap version 45 2024-06-10T03:03:49.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.047+0000 7f019ea04b80 1 -- 172.21.15.52:0/2366849407 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x5579fca71be0 con 0x5579fc9f2f60 2024-06-10T03:03:49.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.047+0000 7f01757fa700 1 -- 172.21.15.52:0/2366849407 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f01880a05d0 con 0x5579fc9f2f60 2024-06-10T03:03:49.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.047+0000 7f01757fa700 1 -- 172.21.15.52:0/2366849407 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 45) ==== 8406+0+0 (secure 0 0 0) 0x7f017c004a90 con 0x5579fc9f2f60 2024-06-10T03:03:49.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.047+0000 7f019ea04b80 10 client.16923 fetch_fsmap finished waiting for FSMap version 45 2024-06-10T03:03:49.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.047+0000 7f019ea04b80 10 client.16923 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.b 2024-06-10T03:03:49.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.047+0000 7f019ea04b80 20 client.16923 populate_metadata read hostname 'smithi052' 2024-06-10T03:03:49.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.047+0000 7f019ea04b80 1 --2- 172.21.15.52:0/2366849407 >> [v2:172.21.15.120:6838/1672436487,v1:172.21.15.120:6839/1672436487] conn(0x5579fca72000 0x5579fca74420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:49.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.047+0000 7f019ea04b80 4 client.16923 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T03:03:49.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.047+0000 7f019ea04b80 1 -- 172.21.15.52:0/2366849407 --> [v2:172.21.15.120:6838/1672436487,v1:172.21.15.120:6839/1672436487] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5579fc9462d0 con 0x5579fca72000 2024-06-10T03:03:49.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.047+0000 7f0191f1d700 1 --2- 172.21.15.52:0/2366849407 >> [v2:172.21.15.120:6838/1672436487,v1:172.21.15.120:6839/1672436487] conn(0x5579fca72000 0x5579fca74420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:49.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.048+0000 7f0191f1d700 1 --2- 172.21.15.52:0/2366849407 >> [v2:172.21.15.120:6838/1672436487,v1:172.21.15.120:6839/1672436487] conn(0x5579fca72000 0x5579fca74420 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=9909efd3d29b37ce server_cookie=6c1c667dc8a98fea in_seq=0 out_seq=0 2024-06-10T03:03:49.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.048+0000 7f01757fa700 10 client.16923 ms_handle_connect on v2:172.21.15.120:6838/1672436487 2024-06-10T03:03:49.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.048+0000 7f01757fa700 1 -- 172.21.15.52:0/2366849407 <== mds.3 v2:172.21.15.120:6838/1672436487 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5579fc9462d0 con 0x5579fca72000 2024-06-10T03:03:49.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.048+0000 7f01757fa700 10 client.16923 handle_command_reply: tid=0 2024-06-10T03:03:49.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.048+0000 7f019ea04b80 1 client.16923 shutdown 2024-06-10T03:03:49.052 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.048+0000 7f019ea04b80 1 -- 172.21.15.52:0/2366849407 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5579fc9f2f60 msgr2=0x5579fca0e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:03:49.052 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.048+0000 7f019ea04b80 1 --2- 172.21.15.52:0/2366849407 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5579fc9f2f60 0x5579fca0e530 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f017c01c720 tx=0x7f017c01c5f0 comp rx=0 tx=0).stop 2024-06-10T03:03:49.052 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.048+0000 7f019ea04b80 1 -- 172.21.15.52:0/2366849407 shutdown_connections 2024-06-10T03:03:49.052 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.048+0000 7f019ea04b80 1 --2- 172.21.15.52:0/2366849407 >> [v2:172.21.15.120:6838/1672436487,v1:172.21.15.120:6839/1672436487] conn(0x5579fca72000 0x5579fca74420 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:49.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.048+0000 7f019ea04b80 1 --2- 172.21.15.52:0/2366849407 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5579fc9f3880 0x5579fca0ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:49.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.048+0000 7f019ea04b80 1 --2- 172.21.15.52:0/2366849407 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5579fc9ed7a0 0x5579fc9ffc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:49.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.048+0000 7f019ea04b80 1 --2- 172.21.15.52:0/2366849407 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5579fc9f2f60 0x5579fca0e530 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:49.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.048+0000 7f01757fa700 0 client.16923 ms_handle_reset on v2:172.21.15.120:6838/1672436487 2024-06-10T03:03:49.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.048+0000 7f019ea04b80 1 -- 172.21.15.52:0/2366849407 >> 172.21.15.52:0/2366849407 conn(0x5579fc91a080 msgr2=0x5579fc9f22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:03:49.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.048+0000 7f019ea04b80 1 -- 172.21.15.52:0/2366849407 shutdown_connections 2024-06-10T03:03:49.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.048+0000 7f019ea04b80 1 -- 172.21.15.52:0/2366849407 wait complete. 2024-06-10T03:03:49.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.049+0000 7f019ea04b80 20 client.16923 trim_cache size 0 max 16384 2024-06-10T03:03:49.053 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-06-10T03:03:49.054 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.049+0000 7f0194181700 1 -- 172.21.15.52:0/444455414 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f01780a2c20 msgr2=0x7f01780a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:03:49.054 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.049+0000 7f0194181700 1 --2- 172.21.15.52:0/444455414 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f01780a2c20 0x7f01780a50e0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f018c101f60 tx=0x7f017c00b410 comp rx=0 tx=0).stop 2024-06-10T03:03:49.054 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.049+0000 7f0194181700 1 -- 172.21.15.52:0/444455414 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f018c102740 msgr2=0x7f018c19e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:03:49.054 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.049+0000 7f0194181700 1 --2- 172.21.15.52:0/444455414 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f018c102740 0x7f018c19e820 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f0188006010 tx=0x7f0188009f80 comp rx=0 tx=0).stop 2024-06-10T03:03:49.054 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.049+0000 7f0194181700 1 -- 172.21.15.52:0/444455414 shutdown_connections 2024-06-10T03:03:49.054 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.049+0000 7f0194181700 1 --2- 172.21.15.52:0/444455414 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f018c107840 0x7f018c1989e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:49.054 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.049+0000 7f0194181700 1 --2- 172.21.15.52:0/444455414 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f01780a2c20 0x7f01780a50e0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:49.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.049+0000 7f0194181700 1 --2- 172.21.15.52:0/444455414 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f018c102740 0x7f018c19e820 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:49.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.049+0000 7f0194181700 1 --2- 172.21.15.52:0/444455414 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f018c1030f0 0x7f018c19ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:49.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.049+0000 7f0194181700 1 -- 172.21.15.52:0/444455414 >> 172.21.15.52:0/444455414 conn(0x7f018c0795f0 msgr2=0x7f018c108990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:03:49.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.049+0000 7f0194181700 1 -- 172.21.15.52:0/444455414 shutdown_connections 2024-06-10T03:03:49.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:49.049+0000 7f0194181700 1 -- 172.21.15.52:0/444455414 wait complete. 2024-06-10T03:03:49.787 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-06-10T03:03:50.044 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T03:03:50.477 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:03:50 smithi052 ceph-mon[32362]: pgmap v723: 129 pgs: 129 active+clean; 228 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-10T03:03:50.638 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:03:50 smithi120 ceph-mon[36634]: pgmap v723: 129 pgs: 129 active+clean; 228 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-10T03:03:50.704 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:03:50 smithi188 ceph-mon[36599]: pgmap v723: 129 pgs: 129 active+clean; 228 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-10T03:03:51.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.697+0000 7f16b3a7c700 1 -- 172.21.15.52:0/3996490531 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f16ac107840 msgr2=0x7f16ac10f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:03:51.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.697+0000 7f16b3a7c700 1 --2- 172.21.15.52:0/3996490531 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f16ac107840 0x7f16ac10f3c0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f16a400b0d0 tx=0x7f16a400b3e0 comp rx=0 tx=0).stop 2024-06-10T03:03:51.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.698+0000 7f16b3a7c700 1 -- 172.21.15.52:0/3996490531 shutdown_connections 2024-06-10T03:03:51.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.698+0000 7f16b3a7c700 1 --2- 172.21.15.52:0/3996490531 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f16ac107840 0x7f16ac10f3c0 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:51.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.698+0000 7f16b3a7c700 1 --2- 172.21.15.52:0/3996490531 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f16ac102740 0x7f16ac102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:51.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.698+0000 7f16b3a7c700 1 --2- 172.21.15.52:0/3996490531 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f16ac1030f0 0x7f16ac107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:51.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.698+0000 7f16b3a7c700 1 -- 172.21.15.52:0/3996490531 >> 172.21.15.52:0/3996490531 conn(0x7f16ac0795f0 msgr2=0x7f16ac079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:03:51.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.698+0000 7f16b3a7c700 1 -- 172.21.15.52:0/3996490531 shutdown_connections 2024-06-10T03:03:51.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.698+0000 7f16b3a7c700 1 -- 172.21.15.52:0/3996490531 wait complete. 2024-06-10T03:03:51.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.698+0000 7f16b3a7c700 1 Processor -- start 2024-06-10T03:03:51.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.699+0000 7f16b3a7c700 1 -- start start 2024-06-10T03:03:51.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.699+0000 7f16b3a7c700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f16ac102740 0x7f16ac19e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:51.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.699+0000 7f16b3a7c700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f16ac1030f0 0x7f16ac19ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:51.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.699+0000 7f16b3a7c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f16ac107840 0x7f16ac198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:51.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.699+0000 7f16b3a7c700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f16ac112760 con 0x7f16ac107840 2024-06-10T03:03:51.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.699+0000 7f16b3a7c700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f16ac1125e0 con 0x7f16ac102740 2024-06-10T03:03:51.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.699+0000 7f16b3a7c700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f16ac1128e0 con 0x7f16ac1030f0 2024-06-10T03:03:51.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.699+0000 7f16b2019700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f16ac107840 0x7f16ac198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:51.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.699+0000 7f16b2019700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f16ac107840 0x7f16ac198900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51886/0 (socket says 172.21.15.52:51886) 2024-06-10T03:03:51.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.699+0000 7f16b2019700 1 -- 172.21.15.52:0/3617057447 learned_addr learned my addr 172.21.15.52:0/3617057447 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:03:51.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.699+0000 7f16b1017700 1 --2- 172.21.15.52:0/3617057447 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f16ac1030f0 0x7f16ac19ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:51.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.699+0000 7f16b1818700 1 --2- 172.21.15.52:0/3617057447 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f16ac102740 0x7f16ac19e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:51.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.699+0000 7f16b2019700 1 -- 172.21.15.52:0/3617057447 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f16ac1030f0 msgr2=0x7f16ac19ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:03:51.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.699+0000 7f16b2019700 1 --2- 172.21.15.52:0/3617057447 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f16ac1030f0 0x7f16ac19ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:51.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.699+0000 7f16b2019700 1 -- 172.21.15.52:0/3617057447 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f16ac102740 msgr2=0x7f16ac19e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:03:51.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.699+0000 7f16b2019700 1 --2- 172.21.15.52:0/3617057447 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f16ac102740 0x7f16ac19e740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:51.702 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.699+0000 7f16b2019700 1 -- 172.21.15.52:0/3617057447 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16ac199060 con 0x7f16ac107840 2024-06-10T03:03:51.702 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.700+0000 7f16b2019700 1 --2- 172.21.15.52:0/3617057447 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f16ac107840 0x7f16ac198900 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f16a4006070 tx=0x7f16a4004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:03:51.702 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.700+0000 7f16a2ffd700 1 -- 172.21.15.52:0/3617057447 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16a4021070 con 0x7f16ac107840 2024-06-10T03:03:51.702 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.700+0000 7f16a2ffd700 1 -- 172.21.15.52:0/3617057447 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16a4007b80 con 0x7f16ac107840 2024-06-10T03:03:51.702 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.700+0000 7f16b3a7c700 1 -- 172.21.15.52:0/3617057447 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16ac1992f0 con 0x7f16ac107840 2024-06-10T03:03:51.702 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.700+0000 7f16a2ffd700 1 -- 172.21.15.52:0/3617057447 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16a40052a0 con 0x7f16ac107840 2024-06-10T03:03:51.702 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.700+0000 7f16b3a7c700 1 -- 172.21.15.52:0/3617057447 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16ac1a5610 con 0x7f16ac107840 2024-06-10T03:03:51.702 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.702+0000 7f16a2ffd700 1 -- 172.21.15.52:0/3617057447 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f16a4024070 con 0x7f16ac107840 2024-06-10T03:03:51.703 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.703+0000 7f16a2ffd700 1 --2- 172.21.15.52:0/3617057447 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f16980a2ca0 0x7f16980a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:51.703 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.703+0000 7f16b1818700 1 --2- 172.21.15.52:0/3617057447 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f16980a2ca0 0x7f16980a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:51.703 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.703+0000 7f16a2ffd700 1 -- 172.21.15.52:0/3617057447 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f16a400f020 con 0x7f16ac107840 2024-06-10T03:03:51.704 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.704+0000 7f16b1818700 1 --2- 172.21.15.52:0/3617057447 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f16980a2ca0 0x7f16980a5160 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f16ac199a40 tx=0x7f16a800a380 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:03:51.704 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.704+0000 7f16be2ffb80 1 Processor -- start 2024-06-10T03:03:51.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.704+0000 7f16be2ffb80 1 -- start start 2024-06-10T03:03:51.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.704+0000 7f16be2ffb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c0c2d79640 0x55c0c2d79a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:51.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.704+0000 7f16be2ffb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c0c2d79f60 0x55c0c2d739e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:51.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.704+0000 7f16be2ffb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c0c2d73f20 0x55c0c2d74340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:51.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.704+0000 7f16be2ffb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55c0c2cc5810 con 0x55c0c2d73f20 2024-06-10T03:03:51.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.704+0000 7f16be2ffb80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55c0c2c6d720 con 0x55c0c2d79640 2024-06-10T03:03:51.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.704+0000 7f16be2ffb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55c0c2cc17d0 con 0x55c0c2d79f60 2024-06-10T03:03:51.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.704+0000 7f16b2019700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c0c2d73f20 0x55c0c2d74340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:51.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.705+0000 7f16b2019700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c0c2d73f20 0x55c0c2d74340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51892/0 (socket says 172.21.15.52:51892) 2024-06-10T03:03:51.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.705+0000 7f16b2019700 1 -- 172.21.15.52:0/2354412905 learned_addr learned my addr 172.21.15.52:0/2354412905 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:03:51.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.705+0000 7f16b1017700 1 --2- 172.21.15.52:0/2354412905 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c0c2d79f60 0x55c0c2d739e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:51.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.705+0000 7f16b1818700 1 --2- 172.21.15.52:0/2354412905 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c0c2d79640 0x55c0c2d79a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:51.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.705+0000 7f16b2019700 1 -- 172.21.15.52:0/2354412905 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c0c2d79f60 msgr2=0x55c0c2d739e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:03:51.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.705+0000 7f16b2019700 1 --2- 172.21.15.52:0/2354412905 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c0c2d79f60 0x55c0c2d739e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:51.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.705+0000 7f16b2019700 1 -- 172.21.15.52:0/2354412905 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c0c2d79640 msgr2=0x55c0c2d79a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:03:51.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.705+0000 7f16b2019700 1 --2- 172.21.15.52:0/2354412905 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c0c2d79640 0x55c0c2d79a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:51.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.705+0000 7f16b2019700 1 -- 172.21.15.52:0/2354412905 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c0c2c63f50 con 0x55c0c2d73f20 2024-06-10T03:03:51.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.705+0000 7f16b2019700 1 --2- 172.21.15.52:0/2354412905 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c0c2d73f20 0x55c0c2d74340 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f16a4099820 tx=0x7f16a401dcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b25b3e31bf365498 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:03:51.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.705+0000 7f1697fff700 1 -- 172.21.15.52:0/2354412905 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16a40a1200 con 0x55c0c2d73f20 2024-06-10T03:03:51.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.705+0000 7f1697fff700 1 -- 172.21.15.52:0/2354412905 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16a409f030 con 0x55c0c2d73f20 2024-06-10T03:03:51.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.705+0000 7f1697fff700 1 -- 172.21.15.52:0/2354412905 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16a409d0b0 con 0x55c0c2d73f20 2024-06-10T03:03:51.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.706+0000 7f16be2ffb80 1 -- 172.21.15.52:0/2354412905 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c0c2d73f20 msgr2=0x55c0c2d74340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:03:51.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.706+0000 7f16be2ffb80 1 --2- 172.21.15.52:0/2354412905 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c0c2d73f20 0x55c0c2d74340 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f16a4099820 tx=0x7f16a401dcd0 comp rx=0 tx=0).stop 2024-06-10T03:03:51.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.706+0000 7f16be2ffb80 1 -- 172.21.15.52:0/2354412905 shutdown_connections 2024-06-10T03:03:51.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.706+0000 7f16be2ffb80 1 --2- 172.21.15.52:0/2354412905 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c0c2d73f20 0x55c0c2d74340 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:51.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.706+0000 7f16be2ffb80 1 --2- 172.21.15.52:0/2354412905 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c0c2d79640 0x55c0c2d79a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:51.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.706+0000 7f16be2ffb80 1 --2- 172.21.15.52:0/2354412905 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c0c2d79f60 0x55c0c2d739e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:51.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.706+0000 7f16be2ffb80 1 -- 172.21.15.52:0/2354412905 >> 172.21.15.52:0/2354412905 conn(0x55c0c2c9ecc0 msgr2=0x55c0c2d6e850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:03:51.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.706+0000 7f16be2ffb80 1 -- 172.21.15.52:0/2354412905 shutdown_connections 2024-06-10T03:03:51.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.706+0000 7f16be2ffb80 1 -- 172.21.15.52:0/2354412905 wait complete. 2024-06-10T03:03:51.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.706+0000 7f16be2ffb80 1 -- start start 2024-06-10T03:03:51.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.707+0000 7f16be2ffb80 1 Processor -- start 2024-06-10T03:03:51.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.707+0000 7f16be2ffb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c0c2d73f20 0x55c0c2d7fd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:51.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.707+0000 7f16be2ffb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c0c2d79640 0x55c0c2d7d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:51.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.707+0000 7f16be2ffb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c0c2d79f60 0x55c0c2d7d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:51.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.707+0000 7f16be2ffb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55c0c2d806f0 con 0x55c0c2d79f60 2024-06-10T03:03:51.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.707+0000 7f16be2ffb80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55c0c2c65bf0 con 0x55c0c2d79640 2024-06-10T03:03:51.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.707+0000 7f16be2ffb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55c0c2d80870 con 0x55c0c2d73f20 2024-06-10T03:03:51.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.707+0000 7f16be2ffb80 10 client.0 _ll_register_callbacks cb 0x55c0c2c5f920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T03:03:51.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.707+0000 7f16b2019700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c0c2d79f60 0x55c0c2d7d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:51.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.707+0000 7f16b2019700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c0c2d79f60 0x55c0c2d7d590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51896/0 (socket says 172.21.15.52:51896) 2024-06-10T03:03:51.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.707+0000 7f16b2019700 1 -- 172.21.15.52:0/2853123849 learned_addr learned my addr 172.21.15.52:0/2853123849 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:03:51.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.707+0000 7f16b1818700 1 --2- 172.21.15.52:0/2853123849 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c0c2d73f20 0x55c0c2d7fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:51.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.707+0000 7f16b1017700 1 --2- 172.21.15.52:0/2853123849 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c0c2d79640 0x55c0c2d7d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:51.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.707+0000 7f16b2019700 1 -- 172.21.15.52:0/2853123849 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c0c2d73f20 msgr2=0x55c0c2d7fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:03:51.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.707+0000 7f16b2019700 1 --2- 172.21.15.52:0/2853123849 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c0c2d73f20 0x55c0c2d7fd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:51.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.707+0000 7f16b2019700 1 -- 172.21.15.52:0/2853123849 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c0c2d79640 msgr2=0x55c0c2d7d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:03:51.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.707+0000 7f16b2019700 1 --2- 172.21.15.52:0/2853123849 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c0c2d79640 0x55c0c2d7d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:51.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.707+0000 7f16b2019700 1 -- 172.21.15.52:0/2853123849 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c0c2c148d0 con 0x55c0c2d79f60 2024-06-10T03:03:51.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.708+0000 7f16b2019700 1 --2- 172.21.15.52:0/2853123849 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c0c2d79f60 0x55c0c2d7d590 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f16a40992b0 tx=0x7f16a40a5b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce9418e67f908a84 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:03:51.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.708+0000 7f16957fa700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-10T03:03:51.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.708+0000 7f16957fa700 1 -- 172.21.15.52:0/2853123849 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16a40074e0 con 0x55c0c2d79f60 2024-06-10T03:03:51.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.708+0000 7f16957fa700 1 -- 172.21.15.52:0/2853123849 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f16a4007d30 con 0x55c0c2d79f60 2024-06-10T03:03:51.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.708+0000 7f16be2ffb80 1 -- 172.21.15.52:0/2853123849 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f16a409f030 con 0x55c0c2d79f60 2024-06-10T03:03:51.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.708+0000 7f16957fa700 1 -- 172.21.15.52:0/2853123849 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f16a4086120 con 0x55c0c2d79f60 2024-06-10T03:03:51.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.708+0000 7f16be2ffb80 10 client.16935 fetch_fsmap learned FSMap version 45 2024-06-10T03:03:51.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.708+0000 7f16be2ffb80 1 -- 172.21.15.52:0/2853123849 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55c0c2d806f0 con 0x55c0c2d79f60 2024-06-10T03:03:51.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.708+0000 7f16957fa700 1 -- 172.21.15.52:0/2853123849 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16a409f030 con 0x55c0c2d79f60 2024-06-10T03:03:51.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.708+0000 7f16957fa700 1 -- 172.21.15.52:0/2853123849 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 45) ==== 8406+0+0 (secure 0 0 0) 0x7f16a4083030 con 0x55c0c2d79f60 2024-06-10T03:03:51.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.708+0000 7f16be2ffb80 10 client.16935 fetch_fsmap finished waiting for FSMap version 45 2024-06-10T03:03:51.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.708+0000 7f16be2ffb80 10 client.16935 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-06-10T03:03:51.718 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.708+0000 7f16be2ffb80 20 client.16935 populate_metadata read hostname 'smithi052' 2024-06-10T03:03:51.718 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.708+0000 7f16be2ffb80 1 --2- 172.21.15.52:0/2853123849 >> [v2:172.21.15.188:6834/3904544259,v1:172.21.15.188:6835/3904544259] conn(0x55c0c2d8d5a0 0x55c0c2d8f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:51.718 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.708+0000 7f16be2ffb80 4 client.16935 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T03:03:51.718 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.708+0000 7f16be2ffb80 1 -- 172.21.15.52:0/2853123849 --> [v2:172.21.15.188:6834/3904544259,v1:172.21.15.188:6835/3904544259] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c0c2cd2d10 con 0x55c0c2d8d5a0 2024-06-10T03:03:51.718 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.709+0000 7f16b1818700 1 --2- 172.21.15.52:0/2853123849 >> [v2:172.21.15.188:6834/3904544259,v1:172.21.15.188:6835/3904544259] conn(0x55c0c2d8d5a0 0x55c0c2d8f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:51.718 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.709+0000 7f16b1818700 1 --2- 172.21.15.52:0/2853123849 >> [v2:172.21.15.188:6834/3904544259,v1:172.21.15.188:6835/3904544259] conn(0x55c0c2d8d5a0 0x55c0c2d8f9c0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=fd53b14b0deeb850 server_cookie=bd349ce29d8051e3 in_seq=0 out_seq=0 2024-06-10T03:03:51.718 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.709+0000 7f16957fa700 10 client.16935 ms_handle_connect on v2:172.21.15.188:6834/3904544259 2024-06-10T03:03:51.718 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.710+0000 7f16957fa700 1 -- 172.21.15.52:0/2853123849 <== mds.2 v2:172.21.15.188:6834/3904544259 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55c0c2cd2d10 con 0x55c0c2d8d5a0 2024-06-10T03:03:51.718 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.710+0000 7f16957fa700 10 client.16935 handle_command_reply: tid=0 2024-06-10T03:03:51.718 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.710+0000 7f16be2ffb80 1 client.16935 shutdown 2024-06-10T03:03:51.718 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.710+0000 7f16be2ffb80 1 -- 172.21.15.52:0/2853123849 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c0c2d79f60 msgr2=0x55c0c2d7d590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:03:51.718 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.710+0000 7f16be2ffb80 1 --2- 172.21.15.52:0/2853123849 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c0c2d79f60 0x55c0c2d7d590 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f16a40992b0 tx=0x7f16a40a5b40 comp rx=0 tx=0).stop 2024-06-10T03:03:51.718 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.710+0000 7f16be2ffb80 1 -- 172.21.15.52:0/2853123849 shutdown_connections 2024-06-10T03:03:51.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.710+0000 7f16be2ffb80 1 --2- 172.21.15.52:0/2853123849 >> [v2:172.21.15.188:6834/3904544259,v1:172.21.15.188:6835/3904544259] conn(0x55c0c2d8d5a0 0x55c0c2d8f9c0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:51.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.710+0000 7f16be2ffb80 1 --2- 172.21.15.52:0/2853123849 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c0c2d79f60 0x55c0c2d7d590 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:51.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.710+0000 7f16be2ffb80 1 --2- 172.21.15.52:0/2853123849 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c0c2d73f20 0x55c0c2d7fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:51.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.710+0000 7f16be2ffb80 1 --2- 172.21.15.52:0/2853123849 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c0c2d79640 0x55c0c2d7d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:51.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.710+0000 7f16957fa700 0 client.16935 ms_handle_reset on v2:172.21.15.188:6834/3904544259 2024-06-10T03:03:51.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.710+0000 7f16be2ffb80 1 -- 172.21.15.52:0/2853123849 >> 172.21.15.52:0/2853123849 conn(0x55c0c2c9ecc0 msgr2=0x55c0c2d729d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:03:51.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.710+0000 7f16be2ffb80 1 -- 172.21.15.52:0/2853123849 shutdown_connections 2024-06-10T03:03:51.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.710+0000 7f16be2ffb80 1 -- 172.21.15.52:0/2853123849 wait complete. 2024-06-10T03:03:51.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.711+0000 7f16be2ffb80 20 client.16935 trim_cache size 0 max 16384 2024-06-10T03:03:51.727 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.726+0000 7f16be2ffb80 1 Processor -- start 2024-06-10T03:03:51.727 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.726+0000 7f16be2ffb80 1 -- start start 2024-06-10T03:03:51.727 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.726+0000 7f16be2ffb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c0c2d73f20 0x55c0c2d899c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:51.727 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.726+0000 7f16be2ffb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c0c2d79640 0x55c0c2d739e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:51.727 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.726+0000 7f16be2ffb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c0c2d79f60 0x55c0c2d7aa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:51.728 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.726+0000 7f16be2ffb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f16ac1ae240 con 0x55c0c2d79f60 2024-06-10T03:03:51.728 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.726+0000 7f16be2ffb80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f16ac1aff60 con 0x55c0c2d73f20 2024-06-10T03:03:51.728 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.726+0000 7f16be2ffb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f16ac1ae970 con 0x55c0c2d79640 2024-06-10T03:03:51.728 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.726+0000 7f16b2019700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c0c2d79f60 0x55c0c2d7aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:51.728 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.726+0000 7f16b2019700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c0c2d79f60 0x55c0c2d7aa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51904/0 (socket says 172.21.15.52:51904) 2024-06-10T03:03:51.728 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.726+0000 7f16b2019700 1 -- 172.21.15.52:0/3596536227 learned_addr learned my addr 172.21.15.52:0/3596536227 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:03:51.728 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.727+0000 7f16b1017700 1 --2- 172.21.15.52:0/3596536227 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c0c2d79640 0x55c0c2d739e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:51.728 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.727+0000 7f16b1818700 1 --2- 172.21.15.52:0/3596536227 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c0c2d73f20 0x55c0c2d899c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:51.728 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.727+0000 7f16b2019700 1 -- 172.21.15.52:0/3596536227 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c0c2d79640 msgr2=0x55c0c2d739e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:03:51.728 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.727+0000 7f16b2019700 1 --2- 172.21.15.52:0/3596536227 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c0c2d79640 0x55c0c2d739e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:51.728 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.727+0000 7f16b2019700 1 -- 172.21.15.52:0/3596536227 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c0c2d73f20 msgr2=0x55c0c2d899c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:03:51.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.727+0000 7f16b2019700 1 --2- 172.21.15.52:0/3596536227 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c0c2d73f20 0x55c0c2d899c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:51.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.727+0000 7f16b2019700 1 -- 172.21.15.52:0/3596536227 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c0c2d7b5c0 con 0x55c0c2d79f60 2024-06-10T03:03:51.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.727+0000 7f16b1818700 1 --2- 172.21.15.52:0/3596536227 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c0c2d73f20 0x55c0c2d899c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T03:03:51.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.727+0000 7f16b2019700 1 --2- 172.21.15.52:0/3596536227 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c0c2d79f60 0x55c0c2d7aa50 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f16a409cb00 tx=0x7f16a409b360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ac7877e5804083 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:03:51.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.727+0000 7f16977fe700 1 -- 172.21.15.52:0/3596536227 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16a409c5a0 con 0x55c0c2d79f60 2024-06-10T03:03:51.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.727+0000 7f16977fe700 1 -- 172.21.15.52:0/3596536227 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16a409f050 con 0x55c0c2d79f60 2024-06-10T03:03:51.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.727+0000 7f16977fe700 1 -- 172.21.15.52:0/3596536227 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16a409d0b0 con 0x55c0c2d79f60 2024-06-10T03:03:51.733 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.728+0000 7f16be2ffb80 1 -- 172.21.15.52:0/3596536227 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c0c2d79f60 msgr2=0x55c0c2d7aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:03:51.733 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.728+0000 7f16be2ffb80 1 --2- 172.21.15.52:0/3596536227 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c0c2d79f60 0x55c0c2d7aa50 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f16a409cb00 tx=0x7f16a409b360 comp rx=0 tx=0).stop 2024-06-10T03:03:51.733 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.728+0000 7f16be2ffb80 1 -- 172.21.15.52:0/3596536227 shutdown_connections 2024-06-10T03:03:51.733 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.728+0000 7f16be2ffb80 1 --2- 172.21.15.52:0/3596536227 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c0c2d79f60 0x55c0c2d7aa50 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:51.733 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.728+0000 7f16be2ffb80 1 --2- 172.21.15.52:0/3596536227 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c0c2d73f20 0x55c0c2d899c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:51.733 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.728+0000 7f16be2ffb80 1 --2- 172.21.15.52:0/3596536227 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c0c2d79640 0x55c0c2d739e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:51.733 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.728+0000 7f16be2ffb80 1 -- 172.21.15.52:0/3596536227 >> 172.21.15.52:0/3596536227 conn(0x55c0c2c9ecc0 msgr2=0x55c0c2d7a340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:03:51.733 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.728+0000 7f16be2ffb80 1 -- 172.21.15.52:0/3596536227 shutdown_connections 2024-06-10T03:03:51.733 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.728+0000 7f16be2ffb80 1 -- 172.21.15.52:0/3596536227 wait complete. 2024-06-10T03:03:51.733 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.728+0000 7f16be2ffb80 1 -- start start 2024-06-10T03:03:51.733 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.728+0000 7f16be2ffb80 1 Processor -- start 2024-06-10T03:03:51.733 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.729+0000 7f16be2ffb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c0c2d73f20 0x55c0c2d85d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:51.733 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.729+0000 7f16be2ffb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c0c2d79640 0x55c0c2d86250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:51.734 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.729+0000 7f16be2ffb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c0c2d79f60 0x55c0c2d94460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:51.734 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.729+0000 7f16be2ffb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f16ac1af7d0 con 0x55c0c2d79f60 2024-06-10T03:03:51.734 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.729+0000 7f16be2ffb80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f16ac1af0d0 con 0x55c0c2d79640 2024-06-10T03:03:51.734 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.729+0000 7f16be2ffb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f16ac1ac1e0 con 0x55c0c2d73f20 2024-06-10T03:03:51.734 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.729+0000 7f16be2ffb80 10 client.0 _ll_register_callbacks cb 0x55c0c2c5f920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T03:03:51.734 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.729+0000 7f16b2019700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c0c2d79f60 0x55c0c2d94460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:51.734 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.729+0000 7f16b2019700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c0c2d79f60 0x55c0c2d94460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51918/0 (socket says 172.21.15.52:51918) 2024-06-10T03:03:51.734 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.729+0000 7f16b2019700 1 -- 172.21.15.52:0/2217042674 learned_addr learned my addr 172.21.15.52:0/2217042674 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:03:51.734 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.729+0000 7f16b1017700 1 --2- 172.21.15.52:0/2217042674 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c0c2d79640 0x55c0c2d86250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:51.734 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.729+0000 7f16b1818700 1 --2- 172.21.15.52:0/2217042674 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c0c2d73f20 0x55c0c2d85d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:51.734 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.729+0000 7f16b2019700 1 -- 172.21.15.52:0/2217042674 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c0c2d73f20 msgr2=0x55c0c2d85d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:03:51.734 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.729+0000 7f16b2019700 1 --2- 172.21.15.52:0/2217042674 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c0c2d73f20 0x55c0c2d85d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:51.734 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.729+0000 7f16b2019700 1 -- 172.21.15.52:0/2217042674 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c0c2d79640 msgr2=0x55c0c2d86250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:03:51.734 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.729+0000 7f16b2019700 1 --2- 172.21.15.52:0/2217042674 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c0c2d79640 0x55c0c2d86250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:51.735 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.729+0000 7f16b2019700 1 -- 172.21.15.52:0/2217042674 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c0c2d94ad0 con 0x55c0c2d79f60 2024-06-10T03:03:51.735 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.729+0000 7f16b2019700 1 --2- 172.21.15.52:0/2217042674 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c0c2d79f60 0x55c0c2d94460 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f16a400d010 tx=0x7f16a401dcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=816255de385fbc6e server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:03:51.735 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.729+0000 7f16957fa700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-10T03:03:51.735 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.729+0000 7f16957fa700 1 -- 172.21.15.52:0/2217042674 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16a409c2a0 con 0x55c0c2d79f60 2024-06-10T03:03:51.735 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.729+0000 7f16957fa700 1 -- 172.21.15.52:0/2217042674 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f16a4007d30 con 0x55c0c2d79f60 2024-06-10T03:03:51.735 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.729+0000 7f16be2ffb80 1 -- 172.21.15.52:0/2217042674 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f16a409f050 con 0x55c0c2d79f60 2024-06-10T03:03:51.735 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.730+0000 7f16957fa700 1 -- 172.21.15.52:0/2217042674 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f16a40b1080 con 0x55c0c2d79f60 2024-06-10T03:03:51.735 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.730+0000 7f16be2ffb80 10 client.16941 fetch_fsmap learned FSMap version 45 2024-06-10T03:03:51.735 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.730+0000 7f16be2ffb80 1 -- 172.21.15.52:0/2217042674 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55c0c2d94ad0 con 0x55c0c2d79f60 2024-06-10T03:03:51.735 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.730+0000 7f16957fa700 1 -- 172.21.15.52:0/2217042674 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16a409f050 con 0x55c0c2d79f60 2024-06-10T03:03:51.735 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.730+0000 7f16957fa700 1 -- 172.21.15.52:0/2217042674 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 45) ==== 8406+0+0 (secure 0 0 0) 0x7f16a4083070 con 0x55c0c2d79f60 2024-06-10T03:03:51.735 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.730+0000 7f16be2ffb80 10 client.16941 fetch_fsmap finished waiting for FSMap version 45 2024-06-10T03:03:51.735 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.730+0000 7f16be2ffb80 10 client.16941 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-06-10T03:03:51.735 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.730+0000 7f16be2ffb80 20 client.16941 populate_metadata read hostname 'smithi052' 2024-06-10T03:03:51.736 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.730+0000 7f16be2ffb80 1 --2- 172.21.15.52:0/2217042674 >> [v2:172.21.15.188:6834/3904544259,v1:172.21.15.188:6835/3904544259] conn(0x55c0c2df7d40 0x55c0c2dfa160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:51.736 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.730+0000 7f16be2ffb80 4 client.16941 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T03:03:51.736 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.730+0000 7f16be2ffb80 1 -- 172.21.15.52:0/2217042674 --> [v2:172.21.15.188:6834/3904544259,v1:172.21.15.188:6835/3904544259] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c0c2ccd290 con 0x55c0c2df7d40 2024-06-10T03:03:51.736 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.730+0000 7f16b1818700 1 --2- 172.21.15.52:0/2217042674 >> [v2:172.21.15.188:6834/3904544259,v1:172.21.15.188:6835/3904544259] conn(0x55c0c2df7d40 0x55c0c2dfa160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:51.736 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.731+0000 7f16b1818700 1 --2- 172.21.15.52:0/2217042674 >> [v2:172.21.15.188:6834/3904544259,v1:172.21.15.188:6835/3904544259] conn(0x55c0c2df7d40 0x55c0c2dfa160 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=49b44e01120a617d server_cookie=b9e53f098b88e11e in_seq=0 out_seq=0 2024-06-10T03:03:51.736 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.731+0000 7f16957fa700 10 client.16941 ms_handle_connect on v2:172.21.15.188:6834/3904544259 2024-06-10T03:03:51.736 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.731+0000 7f16957fa700 1 -- 172.21.15.52:0/2217042674 <== mds.2 v2:172.21.15.188:6834/3904544259 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c0c2ccd290 con 0x55c0c2df7d40 2024-06-10T03:03:51.736 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.731+0000 7f16957fa700 10 client.16941 handle_command_reply: tid=0 2024-06-10T03:03:51.736 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.731+0000 7f16be2ffb80 1 client.16941 shutdown 2024-06-10T03:03:51.736 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.731+0000 7f16be2ffb80 1 -- 172.21.15.52:0/2217042674 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c0c2d79f60 msgr2=0x55c0c2d94460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:03:51.736 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.731+0000 7f16be2ffb80 1 --2- 172.21.15.52:0/2217042674 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c0c2d79f60 0x55c0c2d94460 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f16a400d010 tx=0x7f16a401dcd0 comp rx=0 tx=0).stop 2024-06-10T03:03:51.736 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.731+0000 7f16be2ffb80 1 -- 172.21.15.52:0/2217042674 shutdown_connections 2024-06-10T03:03:51.736 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.731+0000 7f16be2ffb80 1 --2- 172.21.15.52:0/2217042674 >> [v2:172.21.15.188:6834/3904544259,v1:172.21.15.188:6835/3904544259] conn(0x55c0c2df7d40 0x55c0c2dfa160 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:51.736 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.731+0000 7f16be2ffb80 1 --2- 172.21.15.52:0/2217042674 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c0c2d79f60 0x55c0c2d94460 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:51.737 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.731+0000 7f16be2ffb80 1 --2- 172.21.15.52:0/2217042674 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c0c2d73f20 0x55c0c2d85d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:51.737 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.731+0000 7f16be2ffb80 1 --2- 172.21.15.52:0/2217042674 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c0c2d79640 0x55c0c2d86250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:51.737 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.731+0000 7f16957fa700 0 client.16941 ms_handle_reset on v2:172.21.15.188:6834/3904544259 2024-06-10T03:03:51.737 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.731+0000 7f16be2ffb80 1 -- 172.21.15.52:0/2217042674 >> 172.21.15.52:0/2217042674 conn(0x55c0c2c9ecc0 msgr2=0x55c0c2d7a340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:03:51.737 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.731+0000 7f16be2ffb80 1 -- 172.21.15.52:0/2217042674 shutdown_connections 2024-06-10T03:03:51.737 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.731+0000 7f16be2ffb80 1 -- 172.21.15.52:0/2217042674 wait complete. 2024-06-10T03:03:51.737 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.731+0000 7f16be2ffb80 20 client.16941 trim_cache size 0 max 16384 2024-06-10T03:03:51.737 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-06-10T03:03:51.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.732+0000 7f16b3a7c700 1 -- 172.21.15.52:0/3617057447 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f16980a2ca0 msgr2=0x7f16980a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:03:51.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.732+0000 7f16b3a7c700 1 --2- 172.21.15.52:0/3617057447 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f16980a2ca0 0x7f16980a5160 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f16ac199a40 tx=0x7f16a800a380 comp rx=0 tx=0).stop 2024-06-10T03:03:51.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.732+0000 7f16b3a7c700 1 -- 172.21.15.52:0/3617057447 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f16ac107840 msgr2=0x7f16ac198900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:03:51.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.732+0000 7f16b3a7c700 1 --2- 172.21.15.52:0/3617057447 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f16ac107840 0x7f16ac198900 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f16a4006070 tx=0x7f16a4004d00 comp rx=0 tx=0).stop 2024-06-10T03:03:51.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.732+0000 7f16b3a7c700 1 -- 172.21.15.52:0/3617057447 shutdown_connections 2024-06-10T03:03:51.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.732+0000 7f16b3a7c700 1 --2- 172.21.15.52:0/3617057447 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f16980a2ca0 0x7f16980a5160 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:51.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.732+0000 7f16b3a7c700 1 --2- 172.21.15.52:0/3617057447 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f16ac107840 0x7f16ac198900 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:51.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.732+0000 7f16b3a7c700 1 --2- 172.21.15.52:0/3617057447 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f16ac102740 0x7f16ac19e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:51.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.732+0000 7f16b3a7c700 1 --2- 172.21.15.52:0/3617057447 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f16ac1030f0 0x7f16ac19ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:51.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.732+0000 7f16b3a7c700 1 -- 172.21.15.52:0/3617057447 >> 172.21.15.52:0/3617057447 conn(0x7f16ac0795f0 msgr2=0x7f16ac1003b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:03:51.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.732+0000 7f16b3a7c700 1 -- 172.21.15.52:0/3617057447 shutdown_connections 2024-06-10T03:03:51.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:51.732+0000 7f16b3a7c700 1 -- 172.21.15.52:0/3617057447 wait complete. 2024-06-10T03:03:52.477 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:03:52 smithi052 ceph-mon[32362]: pgmap v724: 129 pgs: 129 active+clean; 228 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-06-10T03:03:52.508 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-06-10T03:03:52.638 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:03:52 smithi120 ceph-mon[36634]: pgmap v724: 129 pgs: 129 active+clean; 228 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-06-10T03:03:52.704 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:03:52 smithi188 ceph-mon[36599]: pgmap v724: 129 pgs: 129 active+clean; 228 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-06-10T03:03:52.759 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T03:03:54.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.433+0000 7f5b966f5700 1 -- 172.21.15.52:0/2108285131 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5b90101d50 msgr2=0x7f5b9010e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:03:54.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.433+0000 7f5b966f5700 1 --2- 172.21.15.52:0/2108285131 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5b90101d50 0x7f5b9010e240 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f5b8800b0a0 tx=0x7f5b8800b3b0 comp rx=0 tx=0).stop 2024-06-10T03:03:54.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.433+0000 7f5b966f5700 1 -- 172.21.15.52:0/2108285131 shutdown_connections 2024-06-10T03:03:54.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.433+0000 7f5b966f5700 1 --2- 172.21.15.52:0/2108285131 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5b90101d50 0x7f5b9010e240 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:54.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.433+0000 7f5b966f5700 1 --2- 172.21.15.52:0/2108285131 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5b90107400 0x7f5b901077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:54.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.433+0000 7f5b966f5700 1 --2- 172.21.15.52:0/2108285131 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5b90101390 0x7f5b90101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:54.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.433+0000 7f5b966f5700 1 -- 172.21.15.52:0/2108285131 >> 172.21.15.52:0/2108285131 conn(0x7f5b90078340 msgr2=0x7f5b90078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:03:54.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.433+0000 7f5b966f5700 1 -- 172.21.15.52:0/2108285131 shutdown_connections 2024-06-10T03:03:54.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.433+0000 7f5b966f5700 1 -- 172.21.15.52:0/2108285131 wait complete. 2024-06-10T03:03:54.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.434+0000 7f5b966f5700 1 Processor -- start 2024-06-10T03:03:54.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.434+0000 7f5b966f5700 1 -- start start 2024-06-10T03:03:54.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.434+0000 7f5b966f5700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5b90101390 0x7f5b9019e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:54.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.434+0000 7f5b966f5700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5b90101d50 0x7f5b9019ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:54.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.434+0000 7f5b966f5700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5b90107400 0x7f5b90198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:54.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.435+0000 7f5b966f5700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f5b901116a0 con 0x7f5b90107400 2024-06-10T03:03:54.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.435+0000 7f5b966f5700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f5b90111520 con 0x7f5b90101390 2024-06-10T03:03:54.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.435+0000 7f5b966f5700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f5b90111820 con 0x7f5b90101d50 2024-06-10T03:03:54.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.435+0000 7f5b94c92700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5b90107400 0x7f5b90198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:54.439 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.435+0000 7f5b94c92700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5b90107400 0x7f5b90198990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51938/0 (socket says 172.21.15.52:51938) 2024-06-10T03:03:54.439 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.435+0000 7f5b94c92700 1 -- 172.21.15.52:0/856879438 learned_addr learned my addr 172.21.15.52:0/856879438 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:03:54.439 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.435+0000 7f5b8ffff700 1 --2- 172.21.15.52:0/856879438 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5b90101390 0x7f5b9019e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:54.439 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.435+0000 7f5b8f7fe700 1 --2- 172.21.15.52:0/856879438 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5b90101d50 0x7f5b9019ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:54.439 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.435+0000 7f5b94c92700 1 -- 172.21.15.52:0/856879438 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5b90101d50 msgr2=0x7f5b9019ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:03:54.439 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.435+0000 7f5b94c92700 1 --2- 172.21.15.52:0/856879438 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5b90101d50 0x7f5b9019ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:54.439 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.435+0000 7f5b94c92700 1 -- 172.21.15.52:0/856879438 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5b90101390 msgr2=0x7f5b9019e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:03:54.439 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.435+0000 7f5b94c92700 1 --2- 172.21.15.52:0/856879438 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5b90101390 0x7f5b9019e7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:54.439 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.435+0000 7f5b94c92700 1 -- 172.21.15.52:0/856879438 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b901990f0 con 0x7f5b90107400 2024-06-10T03:03:54.439 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.435+0000 7f5b8f7fe700 1 --2- 172.21.15.52:0/856879438 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5b90101d50 0x7f5b9019ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T03:03:54.439 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.435+0000 7f5b94c92700 1 --2- 172.21.15.52:0/856879438 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5b90107400 0x7f5b90198990 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f5b880062a0 tx=0x7f5b8800bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:03:54.439 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.435+0000 7f5b8d7fa700 1 -- 172.21.15.52:0/856879438 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b88021070 con 0x7f5b90107400 2024-06-10T03:03:54.439 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.435+0000 7f5b8d7fa700 1 -- 172.21.15.52:0/856879438 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b88007db0 con 0x7f5b90107400 2024-06-10T03:03:54.439 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.435+0000 7f5b966f5700 1 -- 172.21.15.52:0/856879438 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b90199380 con 0x7f5b90107400 2024-06-10T03:03:54.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.435+0000 7f5b8d7fa700 1 -- 172.21.15.52:0/856879438 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b88005460 con 0x7f5b90107400 2024-06-10T03:03:54.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.435+0000 7f5b966f5700 1 -- 172.21.15.52:0/856879438 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b901a56a0 con 0x7f5b90107400 2024-06-10T03:03:54.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.438+0000 7f5b8d7fa700 1 -- 172.21.15.52:0/856879438 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f5b88024070 con 0x7f5b90107400 2024-06-10T03:03:54.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.438+0000 7f5b8d7fa700 1 --2- 172.21.15.52:0/856879438 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f5b780a2ca0 0x7f5b780a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:54.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.439+0000 7f5b8d7fa700 1 -- 172.21.15.52:0/856879438 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f5b8800f020 con 0x7f5b90107400 2024-06-10T03:03:54.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.439+0000 7f5b8ffff700 1 --2- 172.21.15.52:0/856879438 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f5b780a2ca0 0x7f5b780a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:54.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.439+0000 7f5b8ffff700 1 --2- 172.21.15.52:0/856879438 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f5b780a2ca0 0x7f5b780a5160 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f5b80005fd0 tx=0x7f5b8000af00 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:03:54.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.439+0000 7f5ba0f78b80 1 Processor -- start 2024-06-10T03:03:54.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.439+0000 7f5ba0f78b80 1 -- start start 2024-06-10T03:03:54.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.439+0000 7f5ba0f78b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559e7e4d5640 0x559e7e4d5a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:54.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.440+0000 7f5ba0f78b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559e7e4d5f60 0x559e7e4cf9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:54.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.440+0000 7f5ba0f78b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559e7e4cff20 0x559e7e4d0340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:54.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.440+0000 7f5ba0f78b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x559e7e421810 con 0x559e7e4cff20 2024-06-10T03:03:54.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.440+0000 7f5ba0f78b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x559e7e3c9720 con 0x559e7e4d5f60 2024-06-10T03:03:54.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.440+0000 7f5ba0f78b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x559e7e41d7d0 con 0x559e7e4d5640 2024-06-10T03:03:54.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.440+0000 7f5b94c92700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559e7e4cff20 0x559e7e4d0340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:54.445 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.440+0000 7f5b8ffff700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559e7e4d5640 0x559e7e4d5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:54.445 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.440+0000 7f5b8ffff700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559e7e4d5640 0x559e7e4d5a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.52:53632/0 (socket says 172.21.15.52:53632) 2024-06-10T03:03:54.445 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.440+0000 7f5b8ffff700 1 -- 172.21.15.52:0/1351266120 learned_addr learned my addr 172.21.15.52:0/1351266120 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:03:54.445 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.440+0000 7f5b94c92700 1 --2- 172.21.15.52:0/1351266120 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559e7e4cff20 0x559e7e4d0340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51952/0 (socket says 172.21.15.52:51952) 2024-06-10T03:03:54.445 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.440+0000 7f5b8f7fe700 1 --2- 172.21.15.52:0/1351266120 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559e7e4d5f60 0x559e7e4cf9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:54.445 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.440+0000 7f5b8ffff700 1 -- 172.21.15.52:0/1351266120 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559e7e4d5f60 msgr2=0x559e7e4cf9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:03:54.445 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.440+0000 7f5b8ffff700 1 --2- 172.21.15.52:0/1351266120 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559e7e4d5f60 0x559e7e4cf9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:54.445 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.440+0000 7f5b8ffff700 1 -- 172.21.15.52:0/1351266120 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559e7e4cff20 msgr2=0x559e7e4d0340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:03:54.445 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.440+0000 7f5b8ffff700 1 --2- 172.21.15.52:0/1351266120 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559e7e4cff20 0x559e7e4d0340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:54.445 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.440+0000 7f5b8ffff700 1 -- 172.21.15.52:0/1351266120 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e7e3bff50 con 0x559e7e4d5640 2024-06-10T03:03:54.445 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.440+0000 7f5b94c92700 1 --2- 172.21.15.52:0/1351266120 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559e7e4cff20 0x559e7e4d0340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T03:03:54.445 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.440+0000 7f5b8ffff700 1 --2- 172.21.15.52:0/1351266120 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559e7e4d5640 0x559e7e4d5a20 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f5b8000a4a0 tx=0x7f5b80011000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ffbbc6e90314d474 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:03:54.445 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.441+0000 7f5b767fc700 1 -- 172.21.15.52:0/1351266120 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b80021070 con 0x559e7e4d5640 2024-06-10T03:03:54.445 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.441+0000 7f5b767fc700 1 -- 172.21.15.52:0/1351266120 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b80010ca0 con 0x559e7e4d5640 2024-06-10T03:03:54.445 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.441+0000 7f5b767fc700 1 -- 172.21.15.52:0/1351266120 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b8000f620 con 0x559e7e4d5640 2024-06-10T03:03:54.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.441+0000 7f5ba0f78b80 1 -- 172.21.15.52:0/1351266120 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559e7e4d5640 msgr2=0x559e7e4d5a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:03:54.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.441+0000 7f5ba0f78b80 1 --2- 172.21.15.52:0/1351266120 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559e7e4d5640 0x559e7e4d5a20 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f5b8000a4a0 tx=0x7f5b80011000 comp rx=0 tx=0).stop 2024-06-10T03:03:54.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.441+0000 7f5ba0f78b80 1 -- 172.21.15.52:0/1351266120 shutdown_connections 2024-06-10T03:03:54.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.441+0000 7f5ba0f78b80 1 --2- 172.21.15.52:0/1351266120 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559e7e4cff20 0x559e7e4d0340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:54.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.441+0000 7f5ba0f78b80 1 --2- 172.21.15.52:0/1351266120 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559e7e4d5640 0x559e7e4d5a20 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:54.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.441+0000 7f5ba0f78b80 1 --2- 172.21.15.52:0/1351266120 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559e7e4d5f60 0x559e7e4cf9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:54.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.441+0000 7f5ba0f78b80 1 -- 172.21.15.52:0/1351266120 >> 172.21.15.52:0/1351266120 conn(0x559e7e3facc0 msgr2=0x559e7e4ca850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:03:54.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.441+0000 7f5ba0f78b80 1 -- 172.21.15.52:0/1351266120 shutdown_connections 2024-06-10T03:03:54.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.441+0000 7f5ba0f78b80 1 -- 172.21.15.52:0/1351266120 wait complete. 2024-06-10T03:03:54.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.442+0000 7f5ba0f78b80 1 -- start start 2024-06-10T03:03:54.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.442+0000 7f5ba0f78b80 1 Processor -- start 2024-06-10T03:03:54.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.442+0000 7f5ba0f78b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559e7e4cff20 0x559e7e4dbd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:54.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.442+0000 7f5ba0f78b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559e7e4d5640 0x559e7e4d9010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:54.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.442+0000 7f5ba0f78b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559e7e4d5f60 0x559e7e4d9550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:54.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.442+0000 7f5ba0f78b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x559e7e4dc9d0 con 0x559e7e4d5640 2024-06-10T03:03:54.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.442+0000 7f5ba0f78b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x559e7e3c1bf0 con 0x559e7e4d5f60 2024-06-10T03:03:54.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.442+0000 7f5ba0f78b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x559e7e4dcb50 con 0x559e7e4cff20 2024-06-10T03:03:54.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.442+0000 7f5ba0f78b80 10 client.0 _ll_register_callbacks cb 0x559e7e3bb920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T03:03:54.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.443+0000 7f5b8f7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559e7e4d5640 0x559e7e4d9010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:54.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.443+0000 7f5b8f7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559e7e4d5640 0x559e7e4d9010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51962/0 (socket says 172.21.15.52:51962) 2024-06-10T03:03:54.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.443+0000 7f5b8f7fe700 1 -- 172.21.15.52:0/3569491258 learned_addr learned my addr 172.21.15.52:0/3569491258 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:03:54.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.443+0000 7f5b8ffff700 1 --2- 172.21.15.52:0/3569491258 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559e7e4cff20 0x559e7e4dbd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:54.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.443+0000 7f5b94c92700 1 --2- 172.21.15.52:0/3569491258 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559e7e4d5f60 0x559e7e4d9550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:54.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.443+0000 7f5b8f7fe700 1 -- 172.21.15.52:0/3569491258 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559e7e4cff20 msgr2=0x559e7e4dbd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:03:54.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.443+0000 7f5b8f7fe700 1 --2- 172.21.15.52:0/3569491258 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559e7e4cff20 0x559e7e4dbd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:54.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.443+0000 7f5b8f7fe700 1 -- 172.21.15.52:0/3569491258 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559e7e4d5f60 msgr2=0x559e7e4d9550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:03:54.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.443+0000 7f5b8f7fe700 1 --2- 172.21.15.52:0/3569491258 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559e7e4d5f60 0x559e7e4d9550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:54.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.443+0000 7f5b8f7fe700 1 -- 172.21.15.52:0/3569491258 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e7e3708d0 con 0x559e7e4d5640 2024-06-10T03:03:54.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.443+0000 7f5b8f7fe700 1 --2- 172.21.15.52:0/3569491258 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559e7e4d5640 0x559e7e4d9010 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f5b84010a50 tx=0x7f5b84010d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aad9fc096efe73b0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:03:54.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.443+0000 7f5b6f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-10T03:03:54.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.443+0000 7f5b6f7fe700 1 -- 172.21.15.52:0/3569491258 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b8401d060 con 0x559e7e4d5640 2024-06-10T03:03:54.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.443+0000 7f5b6f7fe700 1 -- 172.21.15.52:0/3569491258 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f5b8400ecb0 con 0x559e7e4d5640 2024-06-10T03:03:54.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.443+0000 7f5ba0f78b80 1 -- 172.21.15.52:0/3569491258 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5b80010ca0 con 0x559e7e4d5640 2024-06-10T03:03:54.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.443+0000 7f5b6f7fe700 1 -- 172.21.15.52:0/3569491258 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f5b84024050 con 0x559e7e4d5640 2024-06-10T03:03:54.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.443+0000 7f5ba0f78b80 10 client.16953 fetch_fsmap learned FSMap version 45 2024-06-10T03:03:54.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.443+0000 7f5ba0f78b80 1 -- 172.21.15.52:0/3569491258 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x559e7e4e7370 con 0x559e7e4d5640 2024-06-10T03:03:54.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.444+0000 7f5b6f7fe700 1 -- 172.21.15.52:0/3569491258 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b80010ca0 con 0x559e7e4d5640 2024-06-10T03:03:54.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.444+0000 7f5b6f7fe700 1 -- 172.21.15.52:0/3569491258 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 45) ==== 8406+0+0 (secure 0 0 0) 0x7f5b840163c0 con 0x559e7e4d5640 2024-06-10T03:03:54.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.444+0000 7f5ba0f78b80 10 client.16953 fetch_fsmap finished waiting for FSMap version 45 2024-06-10T03:03:54.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.444+0000 7f5ba0f78b80 10 client.16953 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-06-10T03:03:54.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.444+0000 7f5ba0f78b80 20 client.16953 populate_metadata read hostname 'smithi052' 2024-06-10T03:03:54.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.444+0000 7f5ba0f78b80 1 --2- 172.21.15.52:0/3569491258 >> [v2:172.21.15.188:6836/819317726,v1:172.21.15.188:6837/819317726] conn(0x559e7e4e9a00 0x559e7e4ebdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:54.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.444+0000 7f5ba0f78b80 4 client.16953 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T03:03:54.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.444+0000 7f5ba0f78b80 1 -- 172.21.15.52:0/3569491258 --> [v2:172.21.15.188:6836/819317726,v1:172.21.15.188:6837/819317726] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559e7e42ed10 con 0x559e7e4e9a00 2024-06-10T03:03:54.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.444+0000 7f5b8ffff700 1 --2- 172.21.15.52:0/3569491258 >> [v2:172.21.15.188:6836/819317726,v1:172.21.15.188:6837/819317726] conn(0x559e7e4e9a00 0x559e7e4ebdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:54.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.445+0000 7f5b8ffff700 1 --2- 172.21.15.52:0/3569491258 >> [v2:172.21.15.188:6836/819317726,v1:172.21.15.188:6837/819317726] conn(0x559e7e4e9a00 0x559e7e4ebdf0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f7b3b61e1027cee3 server_cookie=95a1504e4cbbc738 in_seq=0 out_seq=0 2024-06-10T03:03:54.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.445+0000 7f5b6f7fe700 10 client.16953 ms_handle_connect on v2:172.21.15.188:6836/819317726 2024-06-10T03:03:54.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.446+0000 7f5b6f7fe700 1 -- 172.21.15.52:0/3569491258 <== mds.1 v2:172.21.15.188:6836/819317726 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x559e7e42ed10 con 0x559e7e4e9a00 2024-06-10T03:03:54.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.446+0000 7f5b6f7fe700 10 client.16953 handle_command_reply: tid=0 2024-06-10T03:03:54.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.446+0000 7f5ba0f78b80 1 client.16953 shutdown 2024-06-10T03:03:54.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.446+0000 7f5ba0f78b80 1 -- 172.21.15.52:0/3569491258 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559e7e4d5640 msgr2=0x559e7e4d9010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:03:54.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.446+0000 7f5ba0f78b80 1 --2- 172.21.15.52:0/3569491258 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559e7e4d5640 0x559e7e4d9010 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f5b84010a50 tx=0x7f5b84010d60 comp rx=0 tx=0).stop 2024-06-10T03:03:54.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.446+0000 7f5ba0f78b80 1 -- 172.21.15.52:0/3569491258 shutdown_connections 2024-06-10T03:03:54.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.446+0000 7f5ba0f78b80 1 --2- 172.21.15.52:0/3569491258 >> [v2:172.21.15.188:6836/819317726,v1:172.21.15.188:6837/819317726] conn(0x559e7e4e9a00 0x559e7e4ebdf0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:54.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.446+0000 7f5ba0f78b80 1 --2- 172.21.15.52:0/3569491258 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559e7e4d5f60 0x559e7e4d9550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:54.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.446+0000 7f5ba0f78b80 1 --2- 172.21.15.52:0/3569491258 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559e7e4cff20 0x559e7e4dbd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:54.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.446+0000 7f5ba0f78b80 1 --2- 172.21.15.52:0/3569491258 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559e7e4d5640 0x559e7e4d9010 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:54.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.446+0000 7f5ba0f78b80 1 -- 172.21.15.52:0/3569491258 >> 172.21.15.52:0/3569491258 conn(0x559e7e3facc0 msgr2=0x559e7e4ce9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:03:54.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.446+0000 7f5b6f7fe700 0 client.16953 ms_handle_reset on v2:172.21.15.188:6836/819317726 2024-06-10T03:03:54.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.446+0000 7f5ba0f78b80 1 -- 172.21.15.52:0/3569491258 shutdown_connections 2024-06-10T03:03:54.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.446+0000 7f5ba0f78b80 1 -- 172.21.15.52:0/3569491258 wait complete. 2024-06-10T03:03:54.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.446+0000 7f5ba0f78b80 20 client.16953 trim_cache size 0 max 16384 2024-06-10T03:03:54.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.461+0000 7f5ba0f78b80 1 Processor -- start 2024-06-10T03:03:54.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.462+0000 7f5ba0f78b80 1 -- start start 2024-06-10T03:03:54.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.462+0000 7f5ba0f78b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559e7e4cff20 0x559e7e4cf9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:54.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.462+0000 7f5ba0f78b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559e7e4d5640 0x559e7e4e5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:54.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.462+0000 7f5ba0f78b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559e7e4d5f60 0x559e7e4d6a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:54.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.462+0000 7f5ba0f78b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f5b901ae230 con 0x559e7e4d5640 2024-06-10T03:03:54.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.462+0000 7f5ba0f78b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f5b901aff50 con 0x559e7e4d5f60 2024-06-10T03:03:54.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.462+0000 7f5ba0f78b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f5b901ae960 con 0x559e7e4cff20 2024-06-10T03:03:54.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.462+0000 7f5b8f7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559e7e4d5640 0x559e7e4e5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:54.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.462+0000 7f5b8f7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559e7e4d5640 0x559e7e4e5ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51972/0 (socket says 172.21.15.52:51972) 2024-06-10T03:03:54.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.462+0000 7f5b8f7fe700 1 -- 172.21.15.52:0/1587773482 learned_addr learned my addr 172.21.15.52:0/1587773482 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:03:54.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.462+0000 7f5b94c92700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559e7e4d5f60 0x559e7e4d6a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:54.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.462+0000 7f5b8ffff700 1 --2- 172.21.15.52:0/1587773482 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559e7e4cff20 0x559e7e4cf9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:54.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.462+0000 7f5b8f7fe700 1 -- 172.21.15.52:0/1587773482 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559e7e4cff20 msgr2=0x559e7e4cf9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:03:54.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.462+0000 7f5b8f7fe700 1 --2- 172.21.15.52:0/1587773482 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559e7e4cff20 0x559e7e4cf9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:54.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.462+0000 7f5b8f7fe700 1 -- 172.21.15.52:0/1587773482 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559e7e4d5f60 msgr2=0x559e7e4d6a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:03:54.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.462+0000 7f5b8f7fe700 1 --2- 172.21.15.52:0/1587773482 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559e7e4d5f60 0x559e7e4d6a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:54.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.462+0000 7f5b8f7fe700 1 -- 172.21.15.52:0/1587773482 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e7e4d3a90 con 0x559e7e4d5640 2024-06-10T03:03:54.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.462+0000 7f5b94c92700 1 --2- 172.21.15.52:0/1587773482 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559e7e4d5f60 0x559e7e4d6a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T03:03:54.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.462+0000 7f5b8f7fe700 1 --2- 172.21.15.52:0/1587773482 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559e7e4d5640 0x559e7e4e5ca0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f5b84005950 tx=0x7f5b8400b830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22aa3eec23699b13 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:03:54.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.463+0000 7f5b75ffb700 1 -- 172.21.15.52:0/1587773482 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b84016a00 con 0x559e7e4d5640 2024-06-10T03:03:54.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.463+0000 7f5b75ffb700 1 -- 172.21.15.52:0/1587773482 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b8402d070 con 0x559e7e4d5640 2024-06-10T03:03:54.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.463+0000 7f5b75ffb700 1 -- 172.21.15.52:0/1587773482 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b84028990 con 0x559e7e4d5640 2024-06-10T03:03:54.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.463+0000 7f5ba0f78b80 1 -- 172.21.15.52:0/1587773482 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559e7e4d5640 msgr2=0x559e7e4e5ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:03:54.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.463+0000 7f5ba0f78b80 1 --2- 172.21.15.52:0/1587773482 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559e7e4d5640 0x559e7e4e5ca0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f5b84005950 tx=0x7f5b8400b830 comp rx=0 tx=0).stop 2024-06-10T03:03:54.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.463+0000 7f5ba0f78b80 1 -- 172.21.15.52:0/1587773482 shutdown_connections 2024-06-10T03:03:54.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.463+0000 7f5ba0f78b80 1 --2- 172.21.15.52:0/1587773482 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559e7e4d5f60 0x559e7e4d6a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:54.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.463+0000 7f5ba0f78b80 1 --2- 172.21.15.52:0/1587773482 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559e7e4cff20 0x559e7e4cf9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:54.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.463+0000 7f5ba0f78b80 1 --2- 172.21.15.52:0/1587773482 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559e7e4d5640 0x559e7e4e5ca0 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:54.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.463+0000 7f5ba0f78b80 1 -- 172.21.15.52:0/1587773482 >> 172.21.15.52:0/1587773482 conn(0x559e7e3facc0 msgr2=0x559e7e4d6340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:03:54.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.463+0000 7f5ba0f78b80 1 -- 172.21.15.52:0/1587773482 shutdown_connections 2024-06-10T03:03:54.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.463+0000 7f5ba0f78b80 1 -- 172.21.15.52:0/1587773482 wait complete. 2024-06-10T03:03:54.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.464+0000 7f5ba0f78b80 1 -- start start 2024-06-10T03:03:54.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.464+0000 7f5ba0f78b80 1 Processor -- start 2024-06-10T03:03:54.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.464+0000 7f5ba0f78b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559e7e4cff20 0x559e7e4e1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:54.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.464+0000 7f5ba0f78b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559e7e4d5640 0x559e7e4e1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:54.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.464+0000 7f5ba0f78b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559e7e4d5f60 0x559e7e4e2520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:54.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.464+0000 7f5ba0f78b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f5b901af7c0 con 0x559e7e4d5f60 2024-06-10T03:03:54.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.464+0000 7f5ba0f78b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f5b901af0c0 con 0x559e7e4cff20 2024-06-10T03:03:54.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.464+0000 7f5ba0f78b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f5b901ac1d0 con 0x559e7e4d5640 2024-06-10T03:03:54.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.464+0000 7f5ba0f78b80 10 client.0 _ll_register_callbacks cb 0x559e7e3bb920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T03:03:54.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.464+0000 7f5b94c92700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559e7e4d5f60 0x559e7e4e2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:54.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.464+0000 7f5b94c92700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559e7e4d5f60 0x559e7e4e2520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51980/0 (socket says 172.21.15.52:51980) 2024-06-10T03:03:54.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.464+0000 7f5b94c92700 1 -- 172.21.15.52:0/2051049719 learned_addr learned my addr 172.21.15.52:0/2051049719 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:03:54.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.464+0000 7f5b8ffff700 1 --2- 172.21.15.52:0/2051049719 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559e7e4cff20 0x559e7e4e1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:54.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.465+0000 7f5b8f7fe700 1 --2- 172.21.15.52:0/2051049719 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559e7e4d5640 0x559e7e4e1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:54.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.465+0000 7f5b94c92700 1 -- 172.21.15.52:0/2051049719 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559e7e4d5640 msgr2=0x559e7e4e1fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:03:54.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.465+0000 7f5b94c92700 1 --2- 172.21.15.52:0/2051049719 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559e7e4d5640 0x559e7e4e1fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:54.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.465+0000 7f5b94c92700 1 -- 172.21.15.52:0/2051049719 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559e7e4cff20 msgr2=0x559e7e4e1aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:03:54.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.465+0000 7f5b94c92700 1 --2- 172.21.15.52:0/2051049719 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559e7e4cff20 0x559e7e4e1aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:54.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.465+0000 7f5b94c92700 1 -- 172.21.15.52:0/2051049719 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e7e5576e0 con 0x559e7e4d5f60 2024-06-10T03:03:54.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.465+0000 7f5b94c92700 1 --2- 172.21.15.52:0/2051049719 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559e7e4d5f60 0x559e7e4e2520 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f5b8809cce0 tx=0x7f5b8801def0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f1ee6cacf0b0671 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:03:54.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.465+0000 7f5b6f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-10T03:03:54.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.465+0000 7f5b6f7fe700 1 -- 172.21.15.52:0/2051049719 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b8809cfb0 con 0x559e7e4d5f60 2024-06-10T03:03:54.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.465+0000 7f5b6f7fe700 1 -- 172.21.15.52:0/2051049719 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f5b8809d150 con 0x559e7e4d5f60 2024-06-10T03:03:54.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.465+0000 7f5ba0f78b80 1 -- 172.21.15.52:0/2051049719 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5b8402d070 con 0x559e7e4d5f60 2024-06-10T03:03:54.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.465+0000 7f5b6f7fe700 1 -- 172.21.15.52:0/2051049719 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f5b880a54a0 con 0x559e7e4d5f60 2024-06-10T03:03:54.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.465+0000 7f5ba0f78b80 10 client.16959 fetch_fsmap learned FSMap version 45 2024-06-10T03:03:54.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.465+0000 7f5ba0f78b80 1 -- 172.21.15.52:0/2051049719 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x559e7e558a20 con 0x559e7e4d5f60 2024-06-10T03:03:54.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.465+0000 7f5b6f7fe700 1 -- 172.21.15.52:0/2051049719 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b8402d070 con 0x559e7e4d5f60 2024-06-10T03:03:54.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.466+0000 7f5b6f7fe700 1 -- 172.21.15.52:0/2051049719 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 45) ==== 8406+0+0 (secure 0 0 0) 0x7f5b88011070 con 0x559e7e4d5f60 2024-06-10T03:03:54.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.466+0000 7f5ba0f78b80 10 client.16959 fetch_fsmap finished waiting for FSMap version 45 2024-06-10T03:03:54.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.466+0000 7f5ba0f78b80 10 client.16959 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-06-10T03:03:54.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.466+0000 7f5ba0f78b80 20 client.16959 populate_metadata read hostname 'smithi052' 2024-06-10T03:03:54.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.466+0000 7f5ba0f78b80 1 --2- 172.21.15.52:0/2051049719 >> [v2:172.21.15.188:6836/819317726,v1:172.21.15.188:6837/819317726] conn(0x559e7e559190 0x559e7e55b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:54.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.466+0000 7f5ba0f78b80 4 client.16959 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T03:03:54.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.466+0000 7f5ba0f78b80 1 -- 172.21.15.52:0/2051049719 --> [v2:172.21.15.188:6836/819317726,v1:172.21.15.188:6837/819317726] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559e7e429290 con 0x559e7e559190 2024-06-10T03:03:54.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.466+0000 7f5b8ffff700 1 --2- 172.21.15.52:0/2051049719 >> [v2:172.21.15.188:6836/819317726,v1:172.21.15.188:6837/819317726] conn(0x559e7e559190 0x559e7e55b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:54.471 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:03:54 smithi052 ceph-mon[32362]: pgmap v725: 129 pgs: 129 active+clean; 228 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-06-10T03:03:54.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.467+0000 7f5b6f7fe700 10 client.16959 ms_handle_connect on v2:172.21.15.188:6836/819317726 2024-06-10T03:03:54.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.467+0000 7f5b8ffff700 1 --2- 172.21.15.52:0/2051049719 >> [v2:172.21.15.188:6836/819317726,v1:172.21.15.188:6837/819317726] conn(0x559e7e559190 0x559e7e55b5b0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4b6f08ca9f997e02 server_cookie=1361e06f3024029e in_seq=0 out_seq=0 2024-06-10T03:03:54.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.467+0000 7f5b6f7fe700 1 -- 172.21.15.52:0/2051049719 <== mds.1 v2:172.21.15.188:6836/819317726 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559e7e429290 con 0x559e7e559190 2024-06-10T03:03:54.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.467+0000 7f5b6f7fe700 10 client.16959 handle_command_reply: tid=0 2024-06-10T03:03:54.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.467+0000 7f5ba0f78b80 1 client.16959 shutdown 2024-06-10T03:03:54.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.467+0000 7f5ba0f78b80 1 -- 172.21.15.52:0/2051049719 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559e7e4d5f60 msgr2=0x559e7e4e2520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:03:54.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.467+0000 7f5ba0f78b80 1 --2- 172.21.15.52:0/2051049719 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559e7e4d5f60 0x559e7e4e2520 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f5b8809cce0 tx=0x7f5b8801def0 comp rx=0 tx=0).stop 2024-06-10T03:03:54.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.467+0000 7f5ba0f78b80 1 -- 172.21.15.52:0/2051049719 shutdown_connections 2024-06-10T03:03:54.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.467+0000 7f5ba0f78b80 1 --2- 172.21.15.52:0/2051049719 >> [v2:172.21.15.188:6836/819317726,v1:172.21.15.188:6837/819317726] conn(0x559e7e559190 0x559e7e55b5b0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:54.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.467+0000 7f5ba0f78b80 1 --2- 172.21.15.52:0/2051049719 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559e7e4d5f60 0x559e7e4e2520 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:54.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.467+0000 7f5ba0f78b80 1 --2- 172.21.15.52:0/2051049719 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559e7e4cff20 0x559e7e4e1aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:54.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.467+0000 7f5ba0f78b80 1 --2- 172.21.15.52:0/2051049719 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559e7e4d5640 0x559e7e4e1fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:54.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.467+0000 7f5b6f7fe700 0 client.16959 ms_handle_reset on v2:172.21.15.188:6836/819317726 2024-06-10T03:03:54.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.467+0000 7f5ba0f78b80 1 -- 172.21.15.52:0/2051049719 >> 172.21.15.52:0/2051049719 conn(0x559e7e3facc0 msgr2=0x559e7e4d6340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:03:54.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.467+0000 7f5ba0f78b80 1 -- 172.21.15.52:0/2051049719 shutdown_connections 2024-06-10T03:03:54.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.467+0000 7f5ba0f78b80 1 -- 172.21.15.52:0/2051049719 wait complete. 2024-06-10T03:03:54.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.467+0000 7f5ba0f78b80 20 client.16959 trim_cache size 0 max 16384 2024-06-10T03:03:54.473 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-06-10T03:03:54.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.468+0000 7f5b966f5700 1 -- 172.21.15.52:0/856879438 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f5b780a2ca0 msgr2=0x7f5b780a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:03:54.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.468+0000 7f5b966f5700 1 --2- 172.21.15.52:0/856879438 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f5b780a2ca0 0x7f5b780a5160 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f5b80005fd0 tx=0x7f5b8000af00 comp rx=0 tx=0).stop 2024-06-10T03:03:54.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.468+0000 7f5b966f5700 1 -- 172.21.15.52:0/856879438 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5b90107400 msgr2=0x7f5b90198990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:03:54.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.468+0000 7f5b966f5700 1 --2- 172.21.15.52:0/856879438 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5b90107400 0x7f5b90198990 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f5b880062a0 tx=0x7f5b8800bf10 comp rx=0 tx=0).stop 2024-06-10T03:03:54.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.468+0000 7f5b966f5700 1 -- 172.21.15.52:0/856879438 shutdown_connections 2024-06-10T03:03:54.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.468+0000 7f5b966f5700 1 --2- 172.21.15.52:0/856879438 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f5b780a2ca0 0x7f5b780a5160 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:54.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.468+0000 7f5b966f5700 1 --2- 172.21.15.52:0/856879438 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5b90107400 0x7f5b90198990 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:54.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.468+0000 7f5b966f5700 1 --2- 172.21.15.52:0/856879438 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5b90101390 0x7f5b9019e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:54.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.468+0000 7f5b966f5700 1 --2- 172.21.15.52:0/856879438 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5b90101d50 0x7f5b9019ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:54.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.468+0000 7f5b966f5700 1 -- 172.21.15.52:0/856879438 >> 172.21.15.52:0/856879438 conn(0x7f5b90078340 msgr2=0x7f5b90100540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:03:54.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.468+0000 7f5b966f5700 1 -- 172.21.15.52:0/856879438 shutdown_connections 2024-06-10T03:03:54.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:54.468+0000 7f5b966f5700 1 -- 172.21.15.52:0/856879438 wait complete. 2024-06-10T03:03:54.638 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:03:54 smithi120 ceph-mon[36634]: pgmap v725: 129 pgs: 129 active+clean; 228 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-06-10T03:03:54.703 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:03:54 smithi188 ceph-mon[36599]: pgmap v725: 129 pgs: 129 active+clean; 228 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-06-10T03:03:55.162 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-06-10T03:03:55.424 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T03:03:56.455 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:03:56 smithi052 ceph-mon[32362]: pgmap v726: 129 pgs: 129 active+clean; 228 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-10T03:03:56.639 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:03:56 smithi120 ceph-mon[36634]: pgmap v726: 129 pgs: 129 active+clean; 228 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-10T03:03:56.703 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:03:56 smithi188 ceph-mon[36599]: pgmap v726: 129 pgs: 129 active+clean; 228 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-10T03:03:57.061 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.061+0000 7f3eb9614700 1 -- 172.21.15.52:0/2460378435 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3eb406b5b0 msgr2=0x7f3eb410f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:03:57.061 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.061+0000 7f3eb9614700 1 --2- 172.21.15.52:0/2460378435 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3eb406b5b0 0x7f3eb410f2e0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f3eac00b0d0 tx=0x7f3eac00b3e0 comp rx=0 tx=0).stop 2024-06-10T03:03:57.061 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.061+0000 7f3eb9614700 1 -- 172.21.15.52:0/2460378435 shutdown_connections 2024-06-10T03:03:57.061 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.061+0000 7f3eb9614700 1 --2- 172.21.15.52:0/2460378435 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3eb406b5b0 0x7f3eb410f2e0 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:57.061 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.061+0000 7f3eb9614700 1 --2- 172.21.15.52:0/2460378435 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3eb4107390 0x7f3eb4107770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:57.062 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.061+0000 7f3eb9614700 1 --2- 172.21.15.52:0/2460378435 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3eb406abf0 0x7f3eb406b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:57.062 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.061+0000 7f3eb9614700 1 -- 172.21.15.52:0/2460378435 >> 172.21.15.52:0/2460378435 conn(0x7f3eb4078340 msgr2=0x7f3eb4078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:03:57.062 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.061+0000 7f3eb9614700 1 -- 172.21.15.52:0/2460378435 shutdown_connections 2024-06-10T03:03:57.062 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.061+0000 7f3eb9614700 1 -- 172.21.15.52:0/2460378435 wait complete. 2024-06-10T03:03:57.063 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.061+0000 7f3eb9614700 1 Processor -- start 2024-06-10T03:03:57.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.062+0000 7f3eb9614700 1 -- start start 2024-06-10T03:03:57.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.062+0000 7f3eb9614700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3eb406abf0 0x7f3eb4101440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:57.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.062+0000 7f3eb9614700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3eb406b5b0 0x7f3eb4101980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:57.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.062+0000 7f3eb9614700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3eb4107390 0x7f3eb4101ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:57.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.062+0000 7f3eb9614700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f3eb4112750 con 0x7f3eb4107390 2024-06-10T03:03:57.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.062+0000 7f3eb9614700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f3eb41125d0 con 0x7f3eb406abf0 2024-06-10T03:03:57.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.062+0000 7f3eb9614700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f3eb41128d0 con 0x7f3eb406b5b0 2024-06-10T03:03:57.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.062+0000 7f3eb37fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3eb4107390 0x7f3eb4101ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:57.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.062+0000 7f3eb37fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3eb4107390 0x7f3eb4101ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51996/0 (socket says 172.21.15.52:51996) 2024-06-10T03:03:57.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.062+0000 7f3eb37fe700 1 -- 172.21.15.52:0/1789005604 learned_addr learned my addr 172.21.15.52:0/1789005604 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:03:57.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.062+0000 7f3eb2ffd700 1 --2- 172.21.15.52:0/1789005604 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3eb406abf0 0x7f3eb4101440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:57.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.062+0000 7f3eb27fc700 1 --2- 172.21.15.52:0/1789005604 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3eb406b5b0 0x7f3eb4101980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:57.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.062+0000 7f3eb37fe700 1 -- 172.21.15.52:0/1789005604 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3eb406b5b0 msgr2=0x7f3eb4101980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:03:57.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.062+0000 7f3eb37fe700 1 --2- 172.21.15.52:0/1789005604 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3eb406b5b0 0x7f3eb4101980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:57.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.062+0000 7f3eb37fe700 1 -- 172.21.15.52:0/1789005604 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3eb406abf0 msgr2=0x7f3eb4101440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:03:57.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.062+0000 7f3eb37fe700 1 --2- 172.21.15.52:0/1789005604 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3eb406abf0 0x7f3eb4101440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:57.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.062+0000 7f3eb37fe700 1 -- 172.21.15.52:0/1789005604 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3eb41a70d0 con 0x7f3eb4107390 2024-06-10T03:03:57.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.062+0000 7f3eb27fc700 1 --2- 172.21.15.52:0/1789005604 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3eb406b5b0 0x7f3eb4101980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T03:03:57.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.062+0000 7f3eb37fe700 1 --2- 172.21.15.52:0/1789005604 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3eb4107390 0x7f3eb4101ec0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f3eac006070 tx=0x7f3eac004e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:03:57.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.063+0000 7f3e9bfff700 1 -- 172.21.15.52:0/1789005604 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3eac021070 con 0x7f3eb4107390 2024-06-10T03:03:57.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.063+0000 7f3e9bfff700 1 -- 172.21.15.52:0/1789005604 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3eac007d00 con 0x7f3eb4107390 2024-06-10T03:03:57.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.063+0000 7f3eb9614700 1 -- 172.21.15.52:0/1789005604 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3eb41a7360 con 0x7f3eb4107390 2024-06-10T03:03:57.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.063+0000 7f3e9bfff700 1 -- 172.21.15.52:0/1789005604 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3eac005470 con 0x7f3eb4107390 2024-06-10T03:03:57.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.063+0000 7f3eb9614700 1 -- 172.21.15.52:0/1789005604 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3eb41a7810 con 0x7f3eb4107390 2024-06-10T03:03:57.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.065+0000 7f3e9bfff700 1 -- 172.21.15.52:0/1789005604 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f3eac024070 con 0x7f3eb4107390 2024-06-10T03:03:57.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.066+0000 7f3e9bfff700 1 --2- 172.21.15.52:0/1789005604 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f3e9c0a2ca0 0x7f3e9c0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:57.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.066+0000 7f3eb2ffd700 1 --2- 172.21.15.52:0/1789005604 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f3e9c0a2ca0 0x7f3e9c0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:57.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.066+0000 7f3e9bfff700 1 -- 172.21.15.52:0/1789005604 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f3eac0d0630 con 0x7f3eb4107390 2024-06-10T03:03:57.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.066+0000 7f3eb2ffd700 1 --2- 172.21.15.52:0/1789005604 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f3e9c0a2ca0 0x7f3e9c0a5160 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f3ea4005950 tx=0x7f3ea400a300 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:03:57.068 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.067+0000 7f3ec3e97b80 1 Processor -- start 2024-06-10T03:03:57.068 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.067+0000 7f3ec3e97b80 1 -- start start 2024-06-10T03:03:57.068 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.067+0000 7f3ec3e97b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555ffdcb1640 0x555ffdcb1a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:57.068 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.067+0000 7f3ec3e97b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555ffdcb1f60 0x555ffdcab9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:57.068 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.067+0000 7f3ec3e97b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555ffdcabf20 0x555ffdcac340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:57.068 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.067+0000 7f3ec3e97b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x555ffdbfd810 con 0x555ffdcb1640 2024-06-10T03:03:57.068 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.067+0000 7f3ec3e97b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x555ffdba5720 con 0x555ffdcb1f60 2024-06-10T03:03:57.068 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.067+0000 7f3ec3e97b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x555ffdbf97d0 con 0x555ffdcabf20 2024-06-10T03:03:57.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.067+0000 7f3eb2ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555ffdcb1640 0x555ffdcb1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:57.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.067+0000 7f3eb2ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555ffdcb1640 0x555ffdcb1a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52010/0 (socket says 172.21.15.52:52010) 2024-06-10T03:03:57.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.067+0000 7f3eb2ffd700 1 -- 172.21.15.52:0/4158959734 learned_addr learned my addr 172.21.15.52:0/4158959734 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:03:57.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.067+0000 7f3eb37fe700 1 --2- 172.21.15.52:0/4158959734 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555ffdcabf20 0x555ffdcac340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:57.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.067+0000 7f3eb27fc700 1 --2- 172.21.15.52:0/4158959734 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555ffdcb1f60 0x555ffdcab9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:57.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.067+0000 7f3eb2ffd700 1 -- 172.21.15.52:0/4158959734 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555ffdcabf20 msgr2=0x555ffdcac340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:03:57.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.068+0000 7f3eb2ffd700 1 --2- 172.21.15.52:0/4158959734 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555ffdcabf20 0x555ffdcac340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:57.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.068+0000 7f3eb2ffd700 1 -- 172.21.15.52:0/4158959734 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555ffdcb1f60 msgr2=0x555ffdcab9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:03:57.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.068+0000 7f3eb2ffd700 1 --2- 172.21.15.52:0/4158959734 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555ffdcb1f60 0x555ffdcab9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:57.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.068+0000 7f3eb2ffd700 1 -- 172.21.15.52:0/4158959734 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ffdb9bf50 con 0x555ffdcb1640 2024-06-10T03:03:57.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.068+0000 7f3eb2ffd700 1 --2- 172.21.15.52:0/4158959734 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555ffdcb1640 0x555ffdcb1a20 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f3ea400fd10 tx=0x7f3ea4012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6043245c2f32221 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:03:57.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.068+0000 7f3e997fa700 1 -- 172.21.15.52:0/4158959734 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ea4016070 con 0x555ffdcb1640 2024-06-10T03:03:57.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.068+0000 7f3e997fa700 1 -- 172.21.15.52:0/4158959734 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ea4010cc0 con 0x555ffdcb1640 2024-06-10T03:03:57.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.068+0000 7f3e997fa700 1 -- 172.21.15.52:0/4158959734 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ea401b420 con 0x555ffdcb1640 2024-06-10T03:03:57.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.069+0000 7f3ec3e97b80 1 -- 172.21.15.52:0/4158959734 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555ffdcb1640 msgr2=0x555ffdcb1a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:03:57.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.069+0000 7f3ec3e97b80 1 --2- 172.21.15.52:0/4158959734 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555ffdcb1640 0x555ffdcb1a20 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f3ea400fd10 tx=0x7f3ea4012000 comp rx=0 tx=0).stop 2024-06-10T03:03:57.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.069+0000 7f3ec3e97b80 1 -- 172.21.15.52:0/4158959734 shutdown_connections 2024-06-10T03:03:57.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.069+0000 7f3ec3e97b80 1 --2- 172.21.15.52:0/4158959734 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555ffdcabf20 0x555ffdcac340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:57.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.069+0000 7f3ec3e97b80 1 --2- 172.21.15.52:0/4158959734 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555ffdcb1640 0x555ffdcb1a20 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:57.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.069+0000 7f3ec3e97b80 1 --2- 172.21.15.52:0/4158959734 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555ffdcb1f60 0x555ffdcab9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:57.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.069+0000 7f3ec3e97b80 1 -- 172.21.15.52:0/4158959734 >> 172.21.15.52:0/4158959734 conn(0x555ffdbd6cc0 msgr2=0x555ffdca6850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:03:57.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.069+0000 7f3ec3e97b80 1 -- 172.21.15.52:0/4158959734 shutdown_connections 2024-06-10T03:03:57.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.069+0000 7f3ec3e97b80 1 -- 172.21.15.52:0/4158959734 wait complete. 2024-06-10T03:03:57.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.069+0000 7f3ec3e97b80 1 -- start start 2024-06-10T03:03:57.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.069+0000 7f3ec3e97b80 1 Processor -- start 2024-06-10T03:03:57.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.070+0000 7f3ec3e97b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555ffdcabf20 0x555ffdcb7d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:57.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.070+0000 7f3ec3e97b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555ffdcb1640 0x555ffdcb5060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:57.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.070+0000 7f3ec3e97b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555ffdcb1f60 0x555ffdcb55a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:57.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.070+0000 7f3ec3e97b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x555ffdcb89d0 con 0x555ffdcb1640 2024-06-10T03:03:57.074 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.070+0000 7f3ec3e97b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x555ffdb9dbf0 con 0x555ffdcabf20 2024-06-10T03:03:57.074 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.070+0000 7f3ec3e97b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x555ffdcb8b50 con 0x555ffdcb1f60 2024-06-10T03:03:57.074 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.070+0000 7f3ec3e97b80 10 client.0 _ll_register_callbacks cb 0x555ffdb97920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T03:03:57.074 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.070+0000 7f3eb27fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555ffdcb1640 0x555ffdcb5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:57.074 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.070+0000 7f3eb27fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555ffdcb1640 0x555ffdcb5060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52024/0 (socket says 172.21.15.52:52024) 2024-06-10T03:03:57.074 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.070+0000 7f3eb27fc700 1 -- 172.21.15.52:0/1632940367 learned_addr learned my addr 172.21.15.52:0/1632940367 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:03:57.074 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.070+0000 7f3eb2ffd700 1 --2- 172.21.15.52:0/1632940367 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555ffdcabf20 0x555ffdcb7d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:57.074 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.070+0000 7f3eb37fe700 1 --2- 172.21.15.52:0/1632940367 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555ffdcb1f60 0x555ffdcb55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:57.074 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.070+0000 7f3eb27fc700 1 -- 172.21.15.52:0/1632940367 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555ffdcb1f60 msgr2=0x555ffdcb55a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:03:57.074 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.070+0000 7f3eb27fc700 1 --2- 172.21.15.52:0/1632940367 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555ffdcb1f60 0x555ffdcb55a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:57.074 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.070+0000 7f3eb27fc700 1 -- 172.21.15.52:0/1632940367 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555ffdcabf20 msgr2=0x555ffdcb7d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:03:57.074 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.070+0000 7f3eb27fc700 1 --2- 172.21.15.52:0/1632940367 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555ffdcabf20 0x555ffdcb7d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:57.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.070+0000 7f3eb27fc700 1 -- 172.21.15.52:0/1632940367 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ffdb4c8d0 con 0x555ffdcb1640 2024-06-10T03:03:57.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.070+0000 7f3eb37fe700 1 --2- 172.21.15.52:0/1632940367 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555ffdcb1f60 0x555ffdcb55a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T03:03:57.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.070+0000 7f3eb27fc700 1 --2- 172.21.15.52:0/1632940367 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555ffdcb1640 0x555ffdcb5060 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f3ea8010830 tx=0x7f3ea8010b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c441b69e5b98610 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:03:57.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.070+0000 7f3e96ff5700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-10T03:03:57.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.071+0000 7f3e96ff5700 1 -- 172.21.15.52:0/1632940367 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ea801e060 con 0x555ffdcb1640 2024-06-10T03:03:57.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.071+0000 7f3e96ff5700 1 -- 172.21.15.52:0/1632940367 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f3ea800e9e0 con 0x555ffdcb1640 2024-06-10T03:03:57.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.071+0000 7f3ec3e97b80 1 -- 172.21.15.52:0/1632940367 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ea4010cc0 con 0x555ffdcb1640 2024-06-10T03:03:57.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.071+0000 7f3e96ff5700 1 -- 172.21.15.52:0/1632940367 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f3ea8025050 con 0x555ffdcb1640 2024-06-10T03:03:57.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.071+0000 7f3ec3e97b80 10 client.16971 fetch_fsmap learned FSMap version 45 2024-06-10T03:03:57.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.071+0000 7f3ec3e97b80 1 -- 172.21.15.52:0/1632940367 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x555ffdcc33e0 con 0x555ffdcb1640 2024-06-10T03:03:57.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.071+0000 7f3e96ff5700 1 -- 172.21.15.52:0/1632940367 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ea4010cc0 con 0x555ffdcb1640 2024-06-10T03:03:57.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.071+0000 7f3e96ff5700 1 -- 172.21.15.52:0/1632940367 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 45) ==== 8406+0+0 (secure 0 0 0) 0x7f3ea8017340 con 0x555ffdcb1640 2024-06-10T03:03:57.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.071+0000 7f3ec3e97b80 10 client.16971 fetch_fsmap finished waiting for FSMap version 45 2024-06-10T03:03:57.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.071+0000 7f3ec3e97b80 10 client.16971 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T03:03:57.076 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.071+0000 7f3ec3e97b80 20 client.16971 populate_metadata read hostname 'smithi052' 2024-06-10T03:03:57.076 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.071+0000 7f3ec3e97b80 1 --2- 172.21.15.52:0/1632940367 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x555ffdcc5c10 0x555ffdcc8000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:57.076 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.071+0000 7f3ec3e97b80 4 client.16971 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T03:03:57.076 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.071+0000 7f3ec3e97b80 1 -- 172.21.15.52:0/1632940367 --> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555ffdc0ad10 con 0x555ffdcc5c10 2024-06-10T03:03:57.076 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.072+0000 7f3eb2ffd700 1 --2- 172.21.15.52:0/1632940367 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x555ffdcc5c10 0x555ffdcc8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:57.076 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.072+0000 7f3e96ff5700 10 client.16971 ms_handle_connect on v2:172.21.15.188:6838/310444836 2024-06-10T03:03:57.076 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.072+0000 7f3eb2ffd700 1 --2- 172.21.15.52:0/1632940367 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x555ffdcc5c10 0x555ffdcc8000 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=73b0f81b001f47b8 server_cookie=60084aeb00bd93ae in_seq=0 out_seq=0 2024-06-10T03:03:57.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.073+0000 7f3e96ff5700 1 -- 172.21.15.52:0/1632940367 <== mds.0 v2:172.21.15.188:6838/310444836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x555ffdc0ad10 con 0x555ffdcc5c10 2024-06-10T03:03:57.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.073+0000 7f3e96ff5700 10 client.16971 handle_command_reply: tid=0 2024-06-10T03:03:57.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.073+0000 7f3ec3e97b80 1 client.16971 shutdown 2024-06-10T03:03:57.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.073+0000 7f3ec3e97b80 1 -- 172.21.15.52:0/1632940367 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555ffdcb1640 msgr2=0x555ffdcb5060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:03:57.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.073+0000 7f3ec3e97b80 1 --2- 172.21.15.52:0/1632940367 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555ffdcb1640 0x555ffdcb5060 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f3ea8010830 tx=0x7f3ea8010b40 comp rx=0 tx=0).stop 2024-06-10T03:03:57.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.073+0000 7f3ec3e97b80 1 -- 172.21.15.52:0/1632940367 shutdown_connections 2024-06-10T03:03:57.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.073+0000 7f3ec3e97b80 1 --2- 172.21.15.52:0/1632940367 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x555ffdcc5c10 0x555ffdcc8000 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:57.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.073+0000 7f3ec3e97b80 1 --2- 172.21.15.52:0/1632940367 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555ffdcb1f60 0x555ffdcb55a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:57.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.073+0000 7f3ec3e97b80 1 --2- 172.21.15.52:0/1632940367 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555ffdcabf20 0x555ffdcb7d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:57.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.073+0000 7f3e96ff5700 0 client.16971 ms_handle_reset on v2:172.21.15.188:6838/310444836 2024-06-10T03:03:57.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.073+0000 7f3ec3e97b80 1 --2- 172.21.15.52:0/1632940367 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555ffdcb1640 0x555ffdcb5060 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:57.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.073+0000 7f3ec3e97b80 1 -- 172.21.15.52:0/1632940367 >> 172.21.15.52:0/1632940367 conn(0x555ffdbd6cc0 msgr2=0x555ffdcaa9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:03:57.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.073+0000 7f3ec3e97b80 1 -- 172.21.15.52:0/1632940367 shutdown_connections 2024-06-10T03:03:57.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.073+0000 7f3ec3e97b80 1 -- 172.21.15.52:0/1632940367 wait complete. 2024-06-10T03:03:57.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.074+0000 7f3ec3e97b80 20 client.16971 trim_cache size 0 max 16384 2024-06-10T03:03:57.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.089+0000 7f3ec3e97b80 1 Processor -- start 2024-06-10T03:03:57.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.089+0000 7f3ec3e97b80 1 -- start start 2024-06-10T03:03:57.091 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.089+0000 7f3ec3e97b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555ffdcabf20 0x555ffdcab9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:57.091 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.089+0000 7f3ec3e97b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555ffdcb1640 0x555ffdcb49a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:57.091 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.089+0000 7f3ec3e97b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555ffdcb1f60 0x555ffdcb2a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:57.091 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.089+0000 7f3ec3e97b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f3eb41b04e0 con 0x555ffdcb1640 2024-06-10T03:03:57.091 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.089+0000 7f3eb27fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555ffdcb1640 0x555ffdcb49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:57.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.089+0000 7f3eb27fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555ffdcb1640 0x555ffdcb49a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52028/0 (socket says 172.21.15.52:52028) 2024-06-10T03:03:57.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.090+0000 7f3eb27fc700 1 -- 172.21.15.52:0/1237503743 learned_addr learned my addr 172.21.15.52:0/1237503743 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:03:57.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.090+0000 7f3eb2ffd700 1 --2- 172.21.15.52:0/1237503743 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555ffdcabf20 0x555ffdcab9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:57.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.090+0000 7f3ec3e97b80 1 -- 172.21.15.52:0/1237503743 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f3eb41b2200 con 0x555ffdcb1f60 2024-06-10T03:03:57.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.090+0000 7f3ec3e97b80 1 -- 172.21.15.52:0/1237503743 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f3eb41b0c10 con 0x555ffdcabf20 2024-06-10T03:03:57.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.090+0000 7f3eb37fe700 1 --2- 172.21.15.52:0/1237503743 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555ffdcb1f60 0x555ffdcb2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:57.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.090+0000 7f3eb27fc700 1 -- 172.21.15.52:0/1237503743 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555ffdcabf20 msgr2=0x555ffdcab9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:03:57.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.090+0000 7f3eb27fc700 1 --2- 172.21.15.52:0/1237503743 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555ffdcabf20 0x555ffdcab9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:57.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.090+0000 7f3eb27fc700 1 -- 172.21.15.52:0/1237503743 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555ffdcb1f60 msgr2=0x555ffdcb2a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:03:57.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.090+0000 7f3eb27fc700 1 --2- 172.21.15.52:0/1237503743 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555ffdcb1f60 0x555ffdcb2a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:57.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.090+0000 7f3eb27fc700 1 -- 172.21.15.52:0/1237503743 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ffdca13b0 con 0x555ffdcb1640 2024-06-10T03:03:57.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.090+0000 7f3eb2ffd700 1 --2- 172.21.15.52:0/1237503743 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555ffdcabf20 0x555ffdcab9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T03:03:57.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.090+0000 7f3eb37fe700 1 --2- 172.21.15.52:0/1237503743 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555ffdcb1f60 0x555ffdcb2a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T03:03:57.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.090+0000 7f3eb27fc700 1 --2- 172.21.15.52:0/1237503743 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555ffdcb1640 0x555ffdcb49a0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f3ea8000c00 tx=0x7f3ea80057a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11d8157bbb74fdcd server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:03:57.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.090+0000 7f3e98ff9700 1 -- 172.21.15.52:0/1237503743 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ea802e070 con 0x555ffdcb1640 2024-06-10T03:03:57.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.090+0000 7f3e98ff9700 1 -- 172.21.15.52:0/1237503743 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ea8017e30 con 0x555ffdcb1640 2024-06-10T03:03:57.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.090+0000 7f3e98ff9700 1 -- 172.21.15.52:0/1237503743 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ea80116a0 con 0x555ffdcb1640 2024-06-10T03:03:57.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.091+0000 7f3ec3e97b80 1 -- 172.21.15.52:0/1237503743 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555ffdcb1640 msgr2=0x555ffdcb49a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:03:57.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.091+0000 7f3ec3e97b80 1 --2- 172.21.15.52:0/1237503743 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555ffdcb1640 0x555ffdcb49a0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f3ea8000c00 tx=0x7f3ea80057a0 comp rx=0 tx=0).stop 2024-06-10T03:03:57.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.091+0000 7f3ec3e97b80 1 -- 172.21.15.52:0/1237503743 shutdown_connections 2024-06-10T03:03:57.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.091+0000 7f3ec3e97b80 1 --2- 172.21.15.52:0/1237503743 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555ffdcb1f60 0x555ffdcb2a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:57.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.091+0000 7f3ec3e97b80 1 --2- 172.21.15.52:0/1237503743 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555ffdcabf20 0x555ffdcab9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:57.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.091+0000 7f3ec3e97b80 1 --2- 172.21.15.52:0/1237503743 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555ffdcb1640 0x555ffdcb49a0 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:57.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.091+0000 7f3ec3e97b80 1 -- 172.21.15.52:0/1237503743 >> 172.21.15.52:0/1237503743 conn(0x555ffdbd6cc0 msgr2=0x555ffdcb2340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:03:57.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.091+0000 7f3ec3e97b80 1 -- 172.21.15.52:0/1237503743 shutdown_connections 2024-06-10T03:03:57.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.091+0000 7f3ec3e97b80 1 -- 172.21.15.52:0/1237503743 wait complete. 2024-06-10T03:03:57.094 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.091+0000 7f3ec3e97b80 1 -- start start 2024-06-10T03:03:57.094 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.091+0000 7f3ec3e97b80 1 Processor -- start 2024-06-10T03:03:57.097 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.092+0000 7f3ec3e97b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555ffdcabf20 0x555ffdcbe600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:57.097 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.092+0000 7f3ec3e97b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555ffdcb1640 0x555ffdccd080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:57.097 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.092+0000 7f3ec3e97b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555ffdcb1f60 0x555ffdccd5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:57.097 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.092+0000 7f3ec3e97b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f3eb41b1a70 con 0x555ffdcabf20 2024-06-10T03:03:57.097 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.092+0000 7f3ec3e97b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f3eb41b1370 con 0x555ffdcb1640 2024-06-10T03:03:57.097 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.092+0000 7f3ec3e97b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f3eb41ae480 con 0x555ffdcb1f60 2024-06-10T03:03:57.097 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.092+0000 7f3ec3e97b80 10 client.0 _ll_register_callbacks cb 0x555ffdb97920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T03:03:57.097 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.092+0000 7f3eb2ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555ffdcabf20 0x555ffdcbe600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:57.097 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.092+0000 7f3eb2ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555ffdcabf20 0x555ffdcbe600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52036/0 (socket says 172.21.15.52:52036) 2024-06-10T03:03:57.097 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.092+0000 7f3eb2ffd700 1 -- 172.21.15.52:0/1913453841 learned_addr learned my addr 172.21.15.52:0/1913453841 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:03:57.097 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.092+0000 7f3eb27fc700 1 --2- 172.21.15.52:0/1913453841 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555ffdcb1640 0x555ffdccd080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:57.097 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.092+0000 7f3eb37fe700 1 --2- 172.21.15.52:0/1913453841 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555ffdcb1f60 0x555ffdccd5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:57.098 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.092+0000 7f3eb2ffd700 1 -- 172.21.15.52:0/1913453841 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555ffdcb1f60 msgr2=0x555ffdccd5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:03:57.098 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.092+0000 7f3eb2ffd700 1 --2- 172.21.15.52:0/1913453841 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555ffdcb1f60 0x555ffdccd5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:57.098 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.092+0000 7f3eb2ffd700 1 -- 172.21.15.52:0/1913453841 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555ffdcb1640 msgr2=0x555ffdccd080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T03:03:57.098 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.092+0000 7f3eb2ffd700 1 --2- 172.21.15.52:0/1913453841 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555ffdcb1640 0x555ffdccd080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:57.098 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.092+0000 7f3eb2ffd700 1 -- 172.21.15.52:0/1913453841 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ffdccde70 con 0x555ffdcabf20 2024-06-10T03:03:57.098 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.092+0000 7f3eb37fe700 1 --2- 172.21.15.52:0/1913453841 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555ffdcb1f60 0x555ffdccd5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T03:03:57.098 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.092+0000 7f3eb2ffd700 1 --2- 172.21.15.52:0/1913453841 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555ffdcabf20 0x555ffdcbe600 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f3ea400fce0 tx=0x7f3ea4025950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0758d95497181f9 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:03:57.098 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.092+0000 7f3e96ff5700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-10T03:03:57.098 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.092+0000 7f3e96ff5700 1 -- 172.21.15.52:0/1913453841 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ea4016050 con 0x555ffdcabf20 2024-06-10T03:03:57.098 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.092+0000 7f3e96ff5700 1 -- 172.21.15.52:0/1913453841 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f3ea4005240 con 0x555ffdcabf20 2024-06-10T03:03:57.098 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.092+0000 7f3ec3e97b80 1 -- 172.21.15.52:0/1913453841 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ea8017e30 con 0x555ffdcabf20 2024-06-10T03:03:57.098 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.093+0000 7f3e96ff5700 1 -- 172.21.15.52:0/1913453841 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f3ea403d050 con 0x555ffdcabf20 2024-06-10T03:03:57.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.093+0000 7f3ec3e97b80 10 client.16977 fetch_fsmap learned FSMap version 45 2024-06-10T03:03:57.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.093+0000 7f3ec3e97b80 1 -- 172.21.15.52:0/1913453841 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x555ffdd307c0 con 0x555ffdcabf20 2024-06-10T03:03:57.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.093+0000 7f3e96ff5700 1 -- 172.21.15.52:0/1913453841 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ea8017e30 con 0x555ffdcabf20 2024-06-10T03:03:57.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.093+0000 7f3e96ff5700 1 -- 172.21.15.52:0/1913453841 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 45) ==== 8406+0+0 (secure 0 0 0) 0x7f3ea4038030 con 0x555ffdcabf20 2024-06-10T03:03:57.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.093+0000 7f3ec3e97b80 10 client.16977 fetch_fsmap finished waiting for FSMap version 45 2024-06-10T03:03:57.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.093+0000 7f3ec3e97b80 10 client.16977 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T03:03:57.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.093+0000 7f3ec3e97b80 20 client.16977 populate_metadata read hostname 'smithi052' 2024-06-10T03:03:57.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.093+0000 7f3ec3e97b80 1 --2- 172.21.15.52:0/1913453841 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x555ffdcc5c10 0x555ffdd32d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:57.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.093+0000 7f3ec3e97b80 4 client.16977 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T03:03:57.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.093+0000 7f3ec3e97b80 1 -- 172.21.15.52:0/1913453841 --> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555ffdc05290 con 0x555ffdcc5c10 2024-06-10T03:03:57.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.093+0000 7f3eb27fc700 1 --2- 172.21.15.52:0/1913453841 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x555ffdcc5c10 0x555ffdd32d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:57.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.094+0000 7f3e96ff5700 10 client.16977 ms_handle_connect on v2:172.21.15.188:6838/310444836 2024-06-10T03:03:57.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.094+0000 7f3eb27fc700 1 --2- 172.21.15.52:0/1913453841 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x555ffdcc5c10 0x555ffdd32d20 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=24a24e70a50ee6e5 server_cookie=61b59b0a2cd69b5c in_seq=0 out_seq=0 2024-06-10T03:03:57.100 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.094+0000 7f3e96ff5700 1 -- 172.21.15.52:0/1913453841 <== mds.0 v2:172.21.15.188:6838/310444836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555ffdc05290 con 0x555ffdcc5c10 2024-06-10T03:03:57.100 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.094+0000 7f3e96ff5700 10 client.16977 handle_command_reply: tid=0 2024-06-10T03:03:57.100 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.094+0000 7f3ec3e97b80 1 client.16977 shutdown 2024-06-10T03:03:57.100 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.094+0000 7f3ec3e97b80 1 -- 172.21.15.52:0/1913453841 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555ffdcabf20 msgr2=0x555ffdcbe600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:03:57.100 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.094+0000 7f3ec3e97b80 1 --2- 172.21.15.52:0/1913453841 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555ffdcabf20 0x555ffdcbe600 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f3ea400fce0 tx=0x7f3ea4025950 comp rx=0 tx=0).stop 2024-06-10T03:03:57.100 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.094+0000 7f3ec3e97b80 1 -- 172.21.15.52:0/1913453841 shutdown_connections 2024-06-10T03:03:57.100 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.094+0000 7f3ec3e97b80 1 --2- 172.21.15.52:0/1913453841 >> [v2:172.21.15.188:6838/310444836,v1:172.21.15.188:6839/310444836] conn(0x555ffdcc5c10 0x555ffdd32d20 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:57.100 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.094+0000 7f3e96ff5700 0 client.16977 ms_handle_reset on v2:172.21.15.188:6838/310444836 2024-06-10T03:03:57.100 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.094+0000 7f3ec3e97b80 1 --2- 172.21.15.52:0/1913453841 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555ffdcb1f60 0x555ffdccd5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:57.100 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.094+0000 7f3ec3e97b80 1 --2- 172.21.15.52:0/1913453841 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555ffdcabf20 0x555ffdcbe600 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:57.100 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.094+0000 7f3ec3e97b80 1 --2- 172.21.15.52:0/1913453841 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555ffdcb1640 0x555ffdccd080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:57.100 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.094+0000 7f3ec3e97b80 1 -- 172.21.15.52:0/1913453841 >> 172.21.15.52:0/1913453841 conn(0x555ffdbd6cc0 msgr2=0x555ffdcb2bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:03:57.100 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.094+0000 7f3ec3e97b80 1 -- 172.21.15.52:0/1913453841 shutdown_connections 2024-06-10T03:03:57.100 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.094+0000 7f3ec3e97b80 1 -- 172.21.15.52:0/1913453841 wait complete. 2024-06-10T03:03:57.100 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.094+0000 7f3ec3e97b80 20 client.16977 trim_cache size 0 max 16384 2024-06-10T03:03:57.101 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-06-10T03:03:57.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.095+0000 7f3eb9614700 1 -- 172.21.15.52:0/1789005604 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f3e9c0a2ca0 msgr2=0x7f3e9c0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:03:57.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.095+0000 7f3eb9614700 1 --2- 172.21.15.52:0/1789005604 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f3e9c0a2ca0 0x7f3e9c0a5160 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f3ea4005950 tx=0x7f3ea400a300 comp rx=0 tx=0).stop 2024-06-10T03:03:57.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.095+0000 7f3eb9614700 1 -- 172.21.15.52:0/1789005604 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3eb4107390 msgr2=0x7f3eb4101ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:03:57.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.095+0000 7f3eb9614700 1 --2- 172.21.15.52:0/1789005604 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3eb4107390 0x7f3eb4101ec0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f3eac006070 tx=0x7f3eac004e40 comp rx=0 tx=0).stop 2024-06-10T03:03:57.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.095+0000 7f3eb9614700 1 -- 172.21.15.52:0/1789005604 shutdown_connections 2024-06-10T03:03:57.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.095+0000 7f3eb9614700 1 --2- 172.21.15.52:0/1789005604 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f3e9c0a2ca0 0x7f3e9c0a5160 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:57.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.095+0000 7f3eb9614700 1 --2- 172.21.15.52:0/1789005604 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3eb4107390 0x7f3eb4101ec0 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:57.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.095+0000 7f3eb9614700 1 --2- 172.21.15.52:0/1789005604 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3eb406abf0 0x7f3eb4101440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:57.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.095+0000 7f3eb9614700 1 --2- 172.21.15.52:0/1789005604 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3eb406b5b0 0x7f3eb4101980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:57.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.095+0000 7f3eb9614700 1 -- 172.21.15.52:0/1789005604 >> 172.21.15.52:0/1789005604 conn(0x7f3eb4078340 msgr2=0x7f3eb40ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:03:57.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.095+0000 7f3eb9614700 1 -- 172.21.15.52:0/1789005604 shutdown_connections 2024-06-10T03:03:57.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.095+0000 7f3eb9614700 1 -- 172.21.15.52:0/1789005604 wait complete. 2024-06-10T03:03:57.784 ERROR:tasks.fwd_scrub.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph_7b7a5d375cf4e4afab96b9d4d229b699ae8304bb/qa/tasks/fwd_scrub.py", line 35, in _run self.do_scrub() File "/home/teuthworker/src/git.ceph.com_ceph_7b7a5d375cf4e4afab96b9d4d229b699ae8304bb/qa/tasks/fwd_scrub.py", line 49, in do_scrub self.sleep_unless_stopped(self.sleep_between_iterations) File "/home/teuthworker/src/git.ceph.com_ceph_7b7a5d375cf4e4afab96b9d4d229b699ae8304bb/qa/tasks/thrasher.py", line 47, in sleep_unless_stopped raise self.Stopped() tasks.thrasher.ThrasherGreenlet.Stopped 2024-06-10T03:03:57.785 INFO:tasks.fwd_scrub:done joining 2024-06-10T03:03:57.785 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-06-10T03:03:57.796 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-06-10T03:03:57.909 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.907+0000 7f7da40ba640 1 -- 172.21.15.52:0/3921255717 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7d9c1024b0 msgr2=0x7f7d9c108d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:03:57.910 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.907+0000 7f7da40ba640 1 --2- 172.21.15.52:0/3921255717 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7d9c1024b0 0x7f7d9c108d40 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f7d94009ca0 tx=0x7f7d9402a8c0 comp rx=0 tx=0).stop 2024-06-10T03:03:57.910 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.908+0000 7f7da40ba640 1 -- 172.21.15.52:0/3921255717 shutdown_connections 2024-06-10T03:03:57.910 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.908+0000 7f7da40ba640 1 --2- 172.21.15.52:0/3921255717 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7d9c1024b0 0x7f7d9c108d40 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:57.910 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.908+0000 7f7da40ba640 1 --2- 172.21.15.52:0/3921255717 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7d9c101af0 0x7f7d9c101f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:57.910 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.908+0000 7f7da40ba640 1 --2- 172.21.15.52:0/3921255717 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7d9c100900 0x7f7d9c100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:57.910 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.908+0000 7f7da40ba640 1 -- 172.21.15.52:0/3921255717 >> 172.21.15.52:0/3921255717 conn(0x7f7d9c0fbe90 msgr2=0x7f7d9c0fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:03:57.910 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.908+0000 7f7da40ba640 1 -- 172.21.15.52:0/3921255717 shutdown_connections 2024-06-10T03:03:57.910 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.908+0000 7f7da40ba640 1 -- 172.21.15.52:0/3921255717 wait complete. 2024-06-10T03:03:57.910 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.908+0000 7f7da40ba640 1 Processor -- start 2024-06-10T03:03:57.910 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.908+0000 7f7da40ba640 1 -- start start 2024-06-10T03:03:57.910 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.908+0000 7f7da40ba640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7d9c100900 0x7f7d9c104960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:57.911 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.908+0000 7f7da40ba640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7d9c101af0 0x7f7d9c102fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:57.911 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.908+0000 7f7da40ba640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7d9c1024b0 0x7f7d9c1034e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:57.911 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.908+0000 7f7da40ba640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f7d9c10c560 con 0x7f7d9c1024b0 2024-06-10T03:03:57.911 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.908+0000 7f7da40ba640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f7d9c10c3e0 con 0x7f7d9c100900 2024-06-10T03:03:57.911 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.908+0000 7f7da40ba640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f7d9c10c6e0 con 0x7f7d9c101af0 2024-06-10T03:03:57.911 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.909+0000 7f7da2657640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7d9c1024b0 0x7f7d9c1034e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:57.911 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.909+0000 7f7da2657640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7d9c1024b0 0x7f7d9c1034e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52052/0 (socket says 172.21.15.52:52052) 2024-06-10T03:03:57.911 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.909+0000 7f7da2657640 1 -- 172.21.15.52:0/876298862 learned_addr learned my addr 172.21.15.52:0/876298862 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:03:57.911 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.909+0000 7f7da1655640 1 --2- 172.21.15.52:0/876298862 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7d9c101af0 0x7f7d9c102fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:57.911 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.909+0000 7f7da1e56640 1 --2- 172.21.15.52:0/876298862 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7d9c100900 0x7f7d9c104960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:57.911 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.909+0000 7f7da2657640 1 -- 172.21.15.52:0/876298862 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7d9c101af0 msgr2=0x7f7d9c102fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:03:57.911 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.909+0000 7f7da2657640 1 --2- 172.21.15.52:0/876298862 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7d9c101af0 0x7f7d9c102fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:57.911 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.909+0000 7f7da2657640 1 -- 172.21.15.52:0/876298862 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7d9c100900 msgr2=0x7f7d9c104960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:03:57.911 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.909+0000 7f7da2657640 1 --2- 172.21.15.52:0/876298862 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7d9c100900 0x7f7d9c104960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:57.912 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.909+0000 7f7da2657640 1 -- 172.21.15.52:0/876298862 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d9c103d70 con 0x7f7d9c1024b0 2024-06-10T03:03:57.912 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.909+0000 7f7da1e56640 1 --2- 172.21.15.52:0/876298862 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7d9c100900 0x7f7d9c104960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T03:03:57.912 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.909+0000 7f7da2657640 1 --2- 172.21.15.52:0/876298862 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7d9c1024b0 0x7f7d9c1034e0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f7d94005e30 tx=0x7f7d94031ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:03:57.912 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.909+0000 7f7d92ffd640 1 -- 172.21.15.52:0/876298862 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d94003c30 con 0x7f7d9c1024b0 2024-06-10T03:03:57.912 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.909+0000 7f7da40ba640 1 -- 172.21.15.52:0/876298862 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d9c1a1df0 con 0x7f7d9c1024b0 2024-06-10T03:03:57.912 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.909+0000 7f7d92ffd640 1 -- 172.21.15.52:0/876298862 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d94003dd0 con 0x7f7d9c1024b0 2024-06-10T03:03:57.912 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.909+0000 7f7d92ffd640 1 -- 172.21.15.52:0/876298862 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d9402fde0 con 0x7f7d9c1024b0 2024-06-10T03:03:57.912 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.909+0000 7f7da40ba640 1 -- 172.21.15.52:0/876298862 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d9c1a22b0 con 0x7f7d9c1024b0 2024-06-10T03:03:57.913 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.910+0000 7f7da40ba640 1 -- 172.21.15.52:0/876298862 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d9c1a25a0 con 0x7f7d9c1024b0 2024-06-10T03:03:57.913 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.912+0000 7f7d92ffd640 1 -- 172.21.15.52:0/876298862 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f7d94004050 con 0x7f7d9c1024b0 2024-06-10T03:03:57.913 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.912+0000 7f7d92ffd640 1 --2- 172.21.15.52:0/876298862 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f7d880a2b90 0x7f7d880a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:57.913 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.913+0000 7f7da1e56640 1 --2- 172.21.15.52:0/876298862 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f7d880a2b90 0x7f7d880a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:57.913 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.913+0000 7f7d92ffd640 1 -- 172.21.15.52:0/876298862 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f7d940f0940 con 0x7f7d9c1024b0 2024-06-10T03:03:57.913 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.913+0000 7f7da1e56640 1 --2- 172.21.15.52:0/876298862 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f7d880a2b90 0x7f7d880a5050 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f7d9c101950 tx=0x7f7d9800a560 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:03:57.916 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:57.916+0000 7f7d92ffd640 1 -- 172.21.15.52:0/876298862 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f7d940bd4f0 con 0x7f7d9c1024b0 2024-06-10T03:03:58.105 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:58.104+0000 7f7da40ba640 1 -- 172.21.15.52:0/876298862 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7d9c1a2890 con 0x7f7d9c1024b0 2024-06-10T03:03:58.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:58.106+0000 7f7d92ffd640 1 -- 172.21.15.52:0/876298862 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v100) ==== 74+0+24739 (secure 0 0 0) 0x7f7d940bcdf0 con 0x7f7d9c1024b0 2024-06-10T03:03:58.106 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-10T03:03:58.107 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":100,"fsid":"d8676e9e-26d2-11ef-bc9b-c7b262605968","created":"2024-06-10T02:41:43.076837+0000","modified":"2024-06-10T02:51:45.708105+0000","last_up_change":"2024-06-10T02:48:57.297829+0000","last_in_change":"2024-06-10T02:48:36.865802+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-10T02:45:07.394690+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-10T02:49:57.926838+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-10T02:50:01.130393+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-10T02:50:10.655588+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e03c0691-cc5a-497a-bbc2-ce18c337f074","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":4255833408},{"type":"v1","addr":"172.21.15.52:6803","nonce":4255833408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":4255833408},{"type":"v1","addr":"172.21.15.52:6805","nonce":4255833408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":4255833408},{"type":"v1","addr":"172.21.15.52:6809","nonce":4255833408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":4255833408},{"type":"v1","addr":"172.21.15.52:6807","nonce":4255833408}]},"public_addr":"172.21.15.52:6803/4255833408","cluster_addr":"172.21.15.52:6805/4255833408","heartbeat_back_addr":"172.21.15.52:6809/4255833408","heartbeat_front_addr":"172.21.15.52:6807/4255833408","state":["exists","up"]},{"osd":1,"uuid":"a24d8da8-c5ec-44d7-ab30-56f6ce3efda7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":4116540053},{"type":"v1","addr":"172.21.15.120:6801","nonce":4116540053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":4116540053},{"type":"v1","addr":"172.21.15.120:6803","nonce":4116540053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":4116540053},{"type":"v1","addr":"172.21.15.120:6807","nonce":4116540053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":4116540053},{"type":"v1","addr":"172.21.15.120:6805","nonce":4116540053}]},"public_addr":"172.21.15.120:6801/4116540053","cluster_addr":"172.21.15.120:6803/4116540053","heartbeat_back_addr":"172.21.15.120:6807/4116540053","heartbeat_front_addr":"172.21.15.120:6805/4116540053","state":["exists","up"]},{"osd":2,"uuid":"a7162106-ccbf-45bd-bc0e-051ee43f4214","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":2624326999},{"type":"v1","addr":"172.21.15.188:6801","nonce":2624326999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":2624326999},{"type":"v1","addr":"172.21.15.188:6803","nonce":2624326999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":2624326999},{"type":"v1","addr":"172.21.15.188:6807","nonce":2624326999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":2624326999},{"type":"v1","addr":"172.21.15.188:6805","nonce":2624326999}]},"public_addr":"172.21.15.188:6801/2624326999","cluster_addr":"172.21.15.188:6803/2624326999","heartbeat_back_addr":"172.21.15.188:6807/2624326999","heartbeat_front_addr":"172.21.15.188:6805/2624326999","state":["exists","up"]},{"osd":3,"uuid":"53680244-b49a-4120-b24a-9ad7a6341dc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":116897011},{"type":"v1","addr":"172.21.15.52:6811","nonce":116897011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":116897011},{"type":"v1","addr":"172.21.15.52:6813","nonce":116897011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":116897011},{"type":"v1","addr":"172.21.15.52:6817","nonce":116897011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":116897011},{"type":"v1","addr":"172.21.15.52:6815","nonce":116897011}]},"public_addr":"172.21.15.52:6811/116897011","cluster_addr":"172.21.15.52:6813/116897011","heartbeat_back_addr":"172.21.15.52:6817/116897011","heartbeat_front_addr":"172.21.15.52:6815/116897011","state":["exists","up"]},{"osd":4,"uuid":"9f72f1f4-2ef3-49c5-8410-c108767d2ef5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":1320690621},{"type":"v1","addr":"172.21.15.120:6809","nonce":1320690621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":1320690621},{"type":"v1","addr":"172.21.15.120:6811","nonce":1320690621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":1320690621},{"type":"v1","addr":"172.21.15.120:6815","nonce":1320690621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":1320690621},{"type":"v1","addr":"172.21.15.120:6813","nonce":1320690621}]},"public_addr":"172.21.15.120:6809/1320690621","cluster_addr":"172.21.15.120:6811/1320690621","heartbeat_back_addr":"172.21.15.120:6815/1320690621","heartbeat_front_addr":"172.21.15.120:6813/1320690621","state":["exists","up"]},{"osd":5,"uuid":"001b5706-7f27-4fa4-9b69-969e6d765abc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3951947483},{"type":"v1","addr":"172.21.15.188:6809","nonce":3951947483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3951947483},{"type":"v1","addr":"172.21.15.188:6811","nonce":3951947483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3951947483},{"type":"v1","addr":"172.21.15.188:6815","nonce":3951947483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3951947483},{"type":"v1","addr":"172.21.15.188:6813","nonce":3951947483}]},"public_addr":"172.21.15.188:6809/3951947483","cluster_addr":"172.21.15.188:6811/3951947483","heartbeat_back_addr":"172.21.15.188:6815/3951947483","heartbeat_front_addr":"172.21.15.188:6813/3951947483","state":["exists","up"]},{"osd":6,"uuid":"140c903e-9b31-4a81-9ccd-2edc0a2decf9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":359084919},{"type":"v1","addr":"172.21.15.52:6819","nonce":359084919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":359084919},{"type":"v1","addr":"172.21.15.52:6821","nonce":359084919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":359084919},{"type":"v1","addr":"172.21.15.52:6825","nonce":359084919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":359084919},{"type":"v1","addr":"172.21.15.52:6823","nonce":359084919}]},"public_addr":"172.21.15.52:6819/359084919","cluster_addr":"172.21.15.52:6821/359084919","heartbeat_back_addr":"172.21.15.52:6825/359084919","heartbeat_front_addr":"172.21.15.52:6823/359084919","state":["exists","up"]},{"osd":7,"uuid":"57b735d4-5072-4ca2-a3ff-3763ad84f15a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":3737722011},{"type":"v1","addr":"172.21.15.120:6817","nonce":3737722011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6818","nonce":3737722011},{"type":"v1","addr":"172.21.15.120:6819","nonce":3737722011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":3737722011},{"type":"v1","addr":"172.21.15.120:6823","nonce":3737722011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":3737722011},{"type":"v1","addr":"172.21.15.120:6821","nonce":3737722011}]},"public_addr":"172.21.15.120:6817/3737722011","cluster_addr":"172.21.15.120:6819/3737722011","heartbeat_back_addr":"172.21.15.120:6823/3737722011","heartbeat_front_addr":"172.21.15.120:6821/3737722011","state":["exists","up"]},{"osd":8,"uuid":"4ffbb43b-854a-4eac-99cf-7af89bc65409","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1444828441},{"type":"v1","addr":"172.21.15.188:6817","nonce":1444828441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1444828441},{"type":"v1","addr":"172.21.15.188:6819","nonce":1444828441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1444828441},{"type":"v1","addr":"172.21.15.188:6823","nonce":1444828441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1444828441},{"type":"v1","addr":"172.21.15.188:6821","nonce":1444828441}]},"public_addr":"172.21.15.188:6817/1444828441","cluster_addr":"172.21.15.188:6819/1444828441","heartbeat_back_addr":"172.21.15.188:6823/1444828441","heartbeat_front_addr":"172.21.15.188:6821/1444828441","state":["exists","up"]},{"osd":9,"uuid":"5bfd985c-8acc-4d00-8561-40bbd541c07b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":924440658},{"type":"v1","addr":"172.21.15.52:6827","nonce":924440658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":924440658},{"type":"v1","addr":"172.21.15.52:6829","nonce":924440658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":924440658},{"type":"v1","addr":"172.21.15.52:6833","nonce":924440658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":924440658},{"type":"v1","addr":"172.21.15.52:6831","nonce":924440658}]},"public_addr":"172.21.15.52:6827/924440658","cluster_addr":"172.21.15.52:6829/924440658","heartbeat_back_addr":"172.21.15.52:6833/924440658","heartbeat_front_addr":"172.21.15.52:6831/924440658","state":["exists","up"]},{"osd":10,"uuid":"0180cddf-e109-42cc-829a-185604276f02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6824","nonce":1531769083},{"type":"v1","addr":"172.21.15.120:6825","nonce":1531769083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":1531769083},{"type":"v1","addr":"172.21.15.120:6827","nonce":1531769083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":1531769083},{"type":"v1","addr":"172.21.15.120:6831","nonce":1531769083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6828","nonce":1531769083},{"type":"v1","addr":"172.21.15.120:6829","nonce":1531769083}]},"public_addr":"172.21.15.120:6825/1531769083","cluster_addr":"172.21.15.120:6827/1531769083","heartbeat_back_addr":"172.21.15.120:6831/1531769083","heartbeat_front_addr":"172.21.15.120:6829/1531769083","state":["exists","up"]},{"osd":11,"uuid":"73aec750-82d1-4800-b6a0-4e305912f9ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":3769050170},{"type":"v1","addr":"172.21.15.188:6825","nonce":3769050170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":3769050170},{"type":"v1","addr":"172.21.15.188:6827","nonce":3769050170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":3769050170},{"type":"v1","addr":"172.21.15.188:6831","nonce":3769050170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":3769050170},{"type":"v1","addr":"172.21.15.188:6829","nonce":3769050170}]},"public_addr":"172.21.15.188:6825/3769050170","cluster_addr":"172.21.15.188:6827/3769050170","heartbeat_back_addr":"172.21.15.188:6831/3769050170","heartbeat_front_addr":"172.21.15.188:6829/3769050170","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:44:16.455752+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:44:40.219735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:45:03.558899+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:45:29.155004+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:45:53.981098+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:46:19.412020+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:46:45.462640+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:47:11.484578+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:47:36.879054+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:48:03.392446+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:48:29.422173+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:48:55.523423+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":4}]},{"pgid":"3.6","mappings":[{"from":3,"to":7}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.9","mappings":[{"from":5,"to":8}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.52:0/3535271671":"2024-06-11T02:42:42.872312+0000","172.21.15.52:0/1092934380":"2024-06-11T02:42:42.872312+0000","172.21.15.52:6800/896317057":"2024-06-11T02:42:42.872312+0000","172.21.15.52:6801/3934479642":"2024-06-11T02:42:15.498118+0000","172.21.15.52:6800/3934479642":"2024-06-11T02:42:15.498118+0000","172.21.15.52:0/2759694132":"2024-06-11T02:42:15.498118+0000","172.21.15.52:0/2593665631":"2024-06-11T02:42:15.498118+0000","172.21.15.52:0/3151504634":"2024-06-11T02:42:15.498118+0000","172.21.15.52:6801/896317057":"2024-06-11T02:42:42.872312+0000","172.21.15.52:0/1254480982":"2024-06-11T02:42:42.872312+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-10T03:03:58.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:58.108+0000 7f7da40ba640 1 -- 172.21.15.52:0/876298862 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f7d880a2b90 msgr2=0x7f7d880a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:03:58.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:58.108+0000 7f7da40ba640 1 --2- 172.21.15.52:0/876298862 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f7d880a2b90 0x7f7d880a5050 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f7d9c101950 tx=0x7f7d9800a560 comp rx=0 tx=0).stop 2024-06-10T03:03:58.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:58.108+0000 7f7da40ba640 1 -- 172.21.15.52:0/876298862 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7d9c1024b0 msgr2=0x7f7d9c1034e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:03:58.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:58.108+0000 7f7da40ba640 1 --2- 172.21.15.52:0/876298862 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7d9c1024b0 0x7f7d9c1034e0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f7d94005e30 tx=0x7f7d94031ee0 comp rx=0 tx=0).stop 2024-06-10T03:03:58.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:58.108+0000 7f7da40ba640 1 -- 172.21.15.52:0/876298862 shutdown_connections 2024-06-10T03:03:58.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:58.108+0000 7f7da40ba640 1 --2- 172.21.15.52:0/876298862 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7d9c1024b0 0x7f7d9c1034e0 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:58.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:58.108+0000 7f7da40ba640 1 --2- 172.21.15.52:0/876298862 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f7d880a2b90 0x7f7d880a5050 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:58.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:58.108+0000 7f7da40ba640 1 --2- 172.21.15.52:0/876298862 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7d9c101af0 0x7f7d9c102fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:58.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:58.108+0000 7f7da40ba640 1 --2- 172.21.15.52:0/876298862 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7d9c100900 0x7f7d9c104960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:58.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:58.108+0000 7f7da40ba640 1 -- 172.21.15.52:0/876298862 >> 172.21.15.52:0/876298862 conn(0x7f7d9c0fbe90 msgr2=0x7f7d9c06a040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:03:58.110 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:58.109+0000 7f7da40ba640 1 -- 172.21.15.52:0/876298862 shutdown_connections 2024-06-10T03:03:58.110 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:58.109+0000 7f7da40ba640 1 -- 172.21.15.52:0/876298862 wait complete. 2024-06-10T03:03:58.120 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-06-10T02:45:07.394690+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-06-10T02:49:57.926838+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-06-10T02:50:01.130393+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-06-10T02:50:10.655588+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '96', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-06-10T03:03:58.120 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-06-10T03:03:58.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:58.227+0000 7fe399064640 1 -- 172.21.15.52:0/941253012 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe394102490 msgr2=0x7fe394108d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:03:58.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:58.227+0000 7fe399064640 1 --2- 172.21.15.52:0/941253012 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe394102490 0x7fe394108d20 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fe38c009ca0 tx=0x7fe38c02a8c0 comp rx=0 tx=0).stop 2024-06-10T03:03:58.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:58.227+0000 7fe399064640 1 -- 172.21.15.52:0/941253012 shutdown_connections 2024-06-10T03:03:58.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:58.227+0000 7fe399064640 1 --2- 172.21.15.52:0/941253012 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe394102490 0x7fe394108d20 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:58.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:58.227+0000 7fe399064640 1 --2- 172.21.15.52:0/941253012 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe394101ad0 0x7fe394101f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:58.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:58.227+0000 7fe399064640 1 --2- 172.21.15.52:0/941253012 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe3941008e0 0x7fe394100d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:58.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:58.227+0000 7fe399064640 1 -- 172.21.15.52:0/941253012 >> 172.21.15.52:0/941253012 conn(0x7fe3940fbe50 msgr2=0x7fe3940fe290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:03:58.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:58.227+0000 7fe399064640 1 -- 172.21.15.52:0/941253012 shutdown_connections 2024-06-10T03:03:58.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:58.227+0000 7fe399064640 1 -- 172.21.15.52:0/941253012 wait complete. 2024-06-10T03:03:58.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:58.228+0000 7fe399064640 1 Processor -- start 2024-06-10T03:03:58.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:58.228+0000 7fe399064640 1 -- start start 2024-06-10T03:03:58.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:58.228+0000 7fe399064640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe3941008e0 0x7fe3940696c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:58.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:58.228+0000 7fe399064640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe394101ad0 0x7fe39419ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:58.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:58.229+0000 7fe399064640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe394102490 0x7fe39419f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:58.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:58.229+0000 7fe399064640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fe39410c560 con 0x7fe3941008e0 2024-06-10T03:03:58.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:58.229+0000 7fe399064640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe39410c3e0 con 0x7fe394102490 2024-06-10T03:03:58.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:58.229+0000 7fe399064640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fe39410c6e0 con 0x7fe394101ad0 2024-06-10T03:03:58.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:58.229+0000 7fe392d9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe3941008e0 0x7fe3940696c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:58.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:58.229+0000 7fe392d9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe3941008e0 0x7fe3940696c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52072/0 (socket says 172.21.15.52:52072) 2024-06-10T03:03:58.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:58.229+0000 7fe392d9d640 1 -- 172.21.15.52:0/284519069 learned_addr learned my addr 172.21.15.52:0/284519069 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:03:58.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:58.229+0000 7fe39359e640 1 --2- 172.21.15.52:0/284519069 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe394102490 0x7fe39419f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:58.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:58.229+0000 7fe39259c640 1 --2- 172.21.15.52:0/284519069 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe394101ad0 0x7fe39419ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:58.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:58.229+0000 7fe392d9d640 1 -- 172.21.15.52:0/284519069 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe394101ad0 msgr2=0x7fe39419ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:03:58.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:58.229+0000 7fe392d9d640 1 --2- 172.21.15.52:0/284519069 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe394101ad0 0x7fe39419ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:58.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:58.229+0000 7fe392d9d640 1 -- 172.21.15.52:0/284519069 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe394102490 msgr2=0x7fe39419f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:03:58.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:58.229+0000 7fe392d9d640 1 --2- 172.21.15.52:0/284519069 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe394102490 0x7fe39419f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:58.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:58.229+0000 7fe392d9d640 1 -- 172.21.15.52:0/284519069 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe39419f900 con 0x7fe3941008e0 2024-06-10T03:03:58.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:58.229+0000 7fe39259c640 1 --2- 172.21.15.52:0/284519069 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe394101ad0 0x7fe39419ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T03:03:58.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:58.229+0000 7fe392d9d640 1 --2- 172.21.15.52:0/284519069 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe3941008e0 0x7fe3940696c0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fe38400d950 tx=0x7fe38400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:03:58.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:58.229+0000 7fe37bfff640 1 -- 172.21.15.52:0/284519069 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe384017070 con 0x7fe3941008e0 2024-06-10T03:03:58.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:58.229+0000 7fe399064640 1 -- 172.21.15.52:0/284519069 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe39419fbf0 con 0x7fe3941008e0 2024-06-10T03:03:58.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:58.229+0000 7fe37bfff640 1 -- 172.21.15.52:0/284519069 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe384009c30 con 0x7fe3941008e0 2024-06-10T03:03:58.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:58.229+0000 7fe37bfff640 1 -- 172.21.15.52:0/284519069 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe38400f600 con 0x7fe3941008e0 2024-06-10T03:03:58.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:58.229+0000 7fe399064640 1 -- 172.21.15.52:0/284519069 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3941a0140 con 0x7fe3941008e0 2024-06-10T03:03:58.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:58.230+0000 7fe399064640 1 -- 172.21.15.52:0/284519069 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe394107c80 con 0x7fe3941008e0 2024-06-10T03:03:58.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:58.232+0000 7fe37bfff640 1 -- 172.21.15.52:0/284519069 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fe38401a070 con 0x7fe3941008e0 2024-06-10T03:03:58.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:58.232+0000 7fe37bfff640 1 --2- 172.21.15.52:0/284519069 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fe37c0a2b90 0x7fe37c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:58.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:58.232+0000 7fe39259c640 1 --2- 172.21.15.52:0/284519069 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fe37c0a2b90 0x7fe37c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:58.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:58.232+0000 7fe37bfff640 1 -- 172.21.15.52:0/284519069 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7fe3840cee90 con 0x7fe3941008e0 2024-06-10T03:03:58.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:58.233+0000 7fe39259c640 1 --2- 172.21.15.52:0/284519069 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fe37c0a2b90 0x7fe37c0a5050 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fe394069430 tx=0x7fe388005e50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:03:58.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:58.237+0000 7fe37bfff640 1 -- 172.21.15.52:0/284519069 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fe384009de0 con 0x7fe3941008e0 2024-06-10T03:03:58.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:58.422+0000 7fe399064640 1 -- 172.21.15.52:0/284519069 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fe3940625d0 con 0x7fe3941008e0 2024-06-10T03:03:58.423 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:58.422+0000 7fe37bfff640 1 -- 172.21.15.52:0/284519069 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v100) ==== 93+0+10 (secure 0 0 0) 0x7fe38409ba40 con 0x7fe3941008e0 2024-06-10T03:03:58.423 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 1 2024-06-10T03:03:58.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:58.424+0000 7fe399064640 1 -- 172.21.15.52:0/284519069 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fe37c0a2b90 msgr2=0x7fe37c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:03:58.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:58.424+0000 7fe399064640 1 --2- 172.21.15.52:0/284519069 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fe37c0a2b90 0x7fe37c0a5050 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fe394069430 tx=0x7fe388005e50 comp rx=0 tx=0).stop 2024-06-10T03:03:58.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:58.424+0000 7fe399064640 1 -- 172.21.15.52:0/284519069 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe3941008e0 msgr2=0x7fe3940696c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:03:58.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:58.424+0000 7fe399064640 1 --2- 172.21.15.52:0/284519069 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe3941008e0 0x7fe3940696c0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fe38400d950 tx=0x7fe38400de10 comp rx=0 tx=0).stop 2024-06-10T03:03:58.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:58.424+0000 7fe399064640 1 -- 172.21.15.52:0/284519069 shutdown_connections 2024-06-10T03:03:58.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:58.424+0000 7fe399064640 1 --2- 172.21.15.52:0/284519069 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe394102490 0x7fe39419f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:58.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:58.424+0000 7fe399064640 1 --2- 172.21.15.52:0/284519069 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fe37c0a2b90 0x7fe37c0a5050 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:58.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:58.424+0000 7fe399064640 1 --2- 172.21.15.52:0/284519069 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe394101ad0 0x7fe39419ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:58.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:58.424+0000 7fe399064640 1 --2- 172.21.15.52:0/284519069 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe3941008e0 0x7fe3940696c0 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:58.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:58.424+0000 7fe399064640 1 -- 172.21.15.52:0/284519069 >> 172.21.15.52:0/284519069 conn(0x7fe3940fbe50 msgr2=0x7fe394104e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:03:58.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:58.424+0000 7fe399064640 1 -- 172.21.15.52:0/284519069 shutdown_connections 2024-06-10T03:03:58.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:58.424+0000 7fe399064640 1 -- 172.21.15.52:0/284519069 wait complete. 2024-06-10T03:03:58.435 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:03:58 smithi052 ceph-mon[32362]: pgmap v727: 129 pgs: 129 active+clean; 228 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-10T03:03:58.435 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:03:58 smithi052 ceph-mon[32362]: from='client.? 172.21.15.52:0/876298862' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T03:03:58.436 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-06-10T03:03:58.546 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:58.544+0000 7f99b11a2640 1 -- 172.21.15.52:0/2780685771 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f99ac100900 msgr2=0x7f99ac100d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:03:58.547 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:58.544+0000 7f99b11a2640 1 --2- 172.21.15.52:0/2780685771 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f99ac100900 0x7f99ac100d20 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f999c009900 tx=0x7f999c02a410 comp rx=0 tx=0).stop 2024-06-10T03:03:58.547 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:58.544+0000 7f99b11a2640 1 -- 172.21.15.52:0/2780685771 shutdown_connections 2024-06-10T03:03:58.547 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:58.544+0000 7f99b11a2640 1 --2- 172.21.15.52:0/2780685771 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f99ac1024b0 0x7f99ac108d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:58.547 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:58.544+0000 7f99b11a2640 1 --2- 172.21.15.52:0/2780685771 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f99ac101af0 0x7f99ac101f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:58.547 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:58.544+0000 7f99b11a2640 1 --2- 172.21.15.52:0/2780685771 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f99ac100900 0x7f99ac100d20 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:58.547 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:58.544+0000 7f99b11a2640 1 -- 172.21.15.52:0/2780685771 >> 172.21.15.52:0/2780685771 conn(0x7f99ac0fbe90 msgr2=0x7f99ac0fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:03:58.547 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:58.544+0000 7f99b11a2640 1 -- 172.21.15.52:0/2780685771 shutdown_connections 2024-06-10T03:03:58.547 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:58.544+0000 7f99b11a2640 1 -- 172.21.15.52:0/2780685771 wait complete. 2024-06-10T03:03:58.547 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:58.544+0000 7f99b11a2640 1 Processor -- start 2024-06-10T03:03:58.547 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:58.545+0000 7f99b11a2640 1 -- start start 2024-06-10T03:03:58.547 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:58.545+0000 7f99b11a2640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f99ac100900 0x7f99ac197e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:58.547 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:58.545+0000 7f99b11a2640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f99ac101af0 0x7f99ac198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:58.547 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:58.545+0000 7f99b11a2640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f99ac1024b0 0x7f99ac19f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:58.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:58.545+0000 7f99b11a2640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f99ac10d6a0 con 0x7f99ac100900 2024-06-10T03:03:58.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:58.545+0000 7f99b11a2640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f99ac10d520 con 0x7f99ac101af0 2024-06-10T03:03:58.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:58.545+0000 7f99b11a2640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f99ac10d820 con 0x7f99ac1024b0 2024-06-10T03:03:58.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:58.545+0000 7f99aad9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f99ac100900 0x7f99ac197e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:58.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:58.545+0000 7f99aad9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f99ac100900 0x7f99ac197e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52096/0 (socket says 172.21.15.52:52096) 2024-06-10T03:03:58.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:58.545+0000 7f99aad9d640 1 -- 172.21.15.52:0/1353692372 learned_addr learned my addr 172.21.15.52:0/1353692372 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:03:58.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:58.545+0000 7f99aa59c640 1 --2- 172.21.15.52:0/1353692372 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f99ac101af0 0x7f99ac198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:58.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:58.545+0000 7f99ab59e640 1 --2- 172.21.15.52:0/1353692372 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f99ac1024b0 0x7f99ac19f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:58.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:58.545+0000 7f99aad9d640 1 -- 172.21.15.52:0/1353692372 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f99ac1024b0 msgr2=0x7f99ac19f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:03:58.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:58.545+0000 7f99aad9d640 1 --2- 172.21.15.52:0/1353692372 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f99ac1024b0 0x7f99ac19f310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:58.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:58.545+0000 7f99aad9d640 1 -- 172.21.15.52:0/1353692372 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f99ac101af0 msgr2=0x7f99ac198380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:03:58.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:58.545+0000 7f99aad9d640 1 --2- 172.21.15.52:0/1353692372 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f99ac101af0 0x7f99ac198380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:58.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:58.545+0000 7f99aad9d640 1 -- 172.21.15.52:0/1353692372 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99ac19f9b0 con 0x7f99ac100900 2024-06-10T03:03:58.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:58.545+0000 7f99ab59e640 1 --2- 172.21.15.52:0/1353692372 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f99ac1024b0 0x7f99ac19f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T03:03:58.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:58.545+0000 7f99aa59c640 1 --2- 172.21.15.52:0/1353692372 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f99ac101af0 0x7f99ac198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T03:03:58.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:58.545+0000 7f99aad9d640 1 --2- 172.21.15.52:0/1353692372 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f99ac100900 0x7f99ac197e40 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f999c031610 tx=0x7f999c0043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:03:58.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:58.545+0000 7f9993fff640 1 -- 172.21.15.52:0/1353692372 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f999c03f030 con 0x7f99ac100900 2024-06-10T03:03:58.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:58.546+0000 7f99b11a2640 1 -- 172.21.15.52:0/1353692372 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99ac19fc40 con 0x7f99ac100900 2024-06-10T03:03:58.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:58.546+0000 7f9993fff640 1 -- 172.21.15.52:0/1353692372 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f999c003ce0 con 0x7f99ac100900 2024-06-10T03:03:58.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:58.546+0000 7f9993fff640 1 -- 172.21.15.52:0/1353692372 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f999c035070 con 0x7f99ac100900 2024-06-10T03:03:58.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:58.546+0000 7f99b11a2640 1 -- 172.21.15.52:0/1353692372 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99ac1a0130 con 0x7f99ac100900 2024-06-10T03:03:58.550 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:58.547+0000 7f99b11a2640 1 -- 172.21.15.52:0/1353692372 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99ac107c40 con 0x7f99ac100900 2024-06-10T03:03:58.550 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:58.548+0000 7f9993fff640 1 -- 172.21.15.52:0/1353692372 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f999c042030 con 0x7f99ac100900 2024-06-10T03:03:58.550 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:58.548+0000 7f9993fff640 1 --2- 172.21.15.52:0/1353692372 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f99940a2b90 0x7f99940a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:58.550 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:58.548+0000 7f9993fff640 1 -- 172.21.15.52:0/1353692372 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f999c0efb00 con 0x7f99ac100900 2024-06-10T03:03:58.550 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:58.548+0000 7f99aa59c640 1 --2- 172.21.15.52:0/1353692372 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f99940a2b90 0x7f99940a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:58.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:58.549+0000 7f99aa59c640 1 --2- 172.21.15.52:0/1353692372 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f99940a2b90 0x7f99940a5050 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f99a00058c0 tx=0x7f99a0005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:03:58.553 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:58.553+0000 7f9993fff640 1 -- 172.21.15.52:0/1353692372 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f999c0bc6b0 con 0x7f99ac100900 2024-06-10T03:03:58.638 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:03:58 smithi120 ceph-mon[36634]: pgmap v727: 129 pgs: 129 active+clean; 228 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-10T03:03:58.638 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:03:58 smithi120 ceph-mon[36634]: from='client.? 172.21.15.52:0/876298862' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T03:03:58.704 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:03:58 smithi188 ceph-mon[36599]: pgmap v727: 129 pgs: 129 active+clean; 228 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-10T03:03:58.704 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:03:58 smithi188 ceph-mon[36599]: from='client.? 172.21.15.52:0/876298862' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T03:03:58.737 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:58.736+0000 7f99b11a2640 1 -- 172.21.15.52:0/1353692372 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f99ac0625a0 con 0x7f99ac100900 2024-06-10T03:03:58.737 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:58.737+0000 7f9993fff640 1 -- 172.21.15.52:0/1353692372 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v100) ==== 104+0+11 (secure 0 0 0) 0x7f999c0bbfb0 con 0x7f99ac100900 2024-06-10T03:03:58.737 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 32 2024-06-10T03:03:58.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:58.739+0000 7f99b11a2640 1 -- 172.21.15.52:0/1353692372 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f99940a2b90 msgr2=0x7f99940a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:03:58.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:58.739+0000 7f99b11a2640 1 --2- 172.21.15.52:0/1353692372 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f99940a2b90 0x7f99940a5050 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f99a00058c0 tx=0x7f99a0005850 comp rx=0 tx=0).stop 2024-06-10T03:03:58.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:58.739+0000 7f99b11a2640 1 -- 172.21.15.52:0/1353692372 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f99ac100900 msgr2=0x7f99ac197e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:03:58.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:58.739+0000 7f99b11a2640 1 --2- 172.21.15.52:0/1353692372 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f99ac100900 0x7f99ac197e40 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f999c031610 tx=0x7f999c0043b0 comp rx=0 tx=0).stop 2024-06-10T03:03:58.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:58.739+0000 7f99b11a2640 1 -- 172.21.15.52:0/1353692372 shutdown_connections 2024-06-10T03:03:58.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:58.739+0000 7f99b11a2640 1 --2- 172.21.15.52:0/1353692372 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f99940a2b90 0x7f99940a5050 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:58.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:58.739+0000 7f99b11a2640 1 --2- 172.21.15.52:0/1353692372 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f99ac1024b0 0x7f99ac19f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:58.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:58.739+0000 7f99b11a2640 1 --2- 172.21.15.52:0/1353692372 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f99ac101af0 0x7f99ac198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:58.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:58.739+0000 7f99b11a2640 1 --2- 172.21.15.52:0/1353692372 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f99ac100900 0x7f99ac197e40 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:58.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:58.739+0000 7f99b11a2640 1 -- 172.21.15.52:0/1353692372 >> 172.21.15.52:0/1353692372 conn(0x7f99ac0fbe90 msgr2=0x7f99ac104e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:03:58.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:58.739+0000 7f99b11a2640 1 -- 172.21.15.52:0/1353692372 shutdown_connections 2024-06-10T03:03:58.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:58.739+0000 7f99b11a2640 1 -- 172.21.15.52:0/1353692372 wait complete. 2024-06-10T03:03:58.750 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-06-10T03:03:58.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:58.858+0000 7f529b038640 1 -- 172.21.15.52:0/861126656 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5294101af0 msgr2=0x7f5294101f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:03:58.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:58.858+0000 7f529b038640 1 --2- 172.21.15.52:0/861126656 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5294101af0 0x7f5294101f70 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f5288009900 tx=0x7f528802a410 comp rx=0 tx=0).stop 2024-06-10T03:03:58.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:58.858+0000 7f529b038640 1 -- 172.21.15.52:0/861126656 shutdown_connections 2024-06-10T03:03:58.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:58.858+0000 7f529b038640 1 --2- 172.21.15.52:0/861126656 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f52941024b0 0x7f5294108d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:58.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:58.858+0000 7f529b038640 1 --2- 172.21.15.52:0/861126656 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5294101af0 0x7f5294101f70 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:58.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:58.858+0000 7f529b038640 1 --2- 172.21.15.52:0/861126656 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5294100900 0x7f5294100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:58.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:58.858+0000 7f529b038640 1 -- 172.21.15.52:0/861126656 >> 172.21.15.52:0/861126656 conn(0x7f52940fbe90 msgr2=0x7f52940fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:03:58.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:58.858+0000 7f529b038640 1 -- 172.21.15.52:0/861126656 shutdown_connections 2024-06-10T03:03:58.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:58.858+0000 7f529b038640 1 -- 172.21.15.52:0/861126656 wait complete. 2024-06-10T03:03:58.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:58.858+0000 7f529b038640 1 Processor -- start 2024-06-10T03:03:58.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:58.858+0000 7f529b038640 1 -- start start 2024-06-10T03:03:58.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:58.858+0000 7f529b038640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5294100900 0x7f5294197d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:58.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:58.858+0000 7f529b038640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5294101af0 0x7f5294198250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:58.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:58.858+0000 7f529b038640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f52941024b0 0x7f529419f280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:58.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:58.858+0000 7f529b038640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f529410c580 con 0x7f52941024b0 2024-06-10T03:03:58.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:58.858+0000 7f529b038640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f529410c400 con 0x7f5294101af0 2024-06-10T03:03:58.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:58.858+0000 7f529b038640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f529410c700 con 0x7f5294100900 2024-06-10T03:03:58.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:58.859+0000 7f52995d5640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f52941024b0 0x7f529419f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:58.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:58.859+0000 7f52995d5640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f52941024b0 0x7f529419f280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52112/0 (socket says 172.21.15.52:52112) 2024-06-10T03:03:58.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:58.859+0000 7f52995d5640 1 -- 172.21.15.52:0/1673184048 learned_addr learned my addr 172.21.15.52:0/1673184048 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:03:58.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:58.859+0000 7f5298dd4640 1 --2- 172.21.15.52:0/1673184048 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5294100900 0x7f5294197d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:58.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:58.859+0000 7f5293fff640 1 --2- 172.21.15.52:0/1673184048 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5294101af0 0x7f5294198250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:58.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:58.859+0000 7f52995d5640 1 -- 172.21.15.52:0/1673184048 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5294100900 msgr2=0x7f5294197d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:03:58.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:58.859+0000 7f52995d5640 1 --2- 172.21.15.52:0/1673184048 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5294100900 0x7f5294197d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:58.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:58.859+0000 7f52995d5640 1 -- 172.21.15.52:0/1673184048 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5294101af0 msgr2=0x7f5294198250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:03:58.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:58.859+0000 7f52995d5640 1 --2- 172.21.15.52:0/1673184048 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5294101af0 0x7f5294198250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:58.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:58.859+0000 7f52995d5640 1 -- 172.21.15.52:0/1673184048 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f529419f920 con 0x7f52941024b0 2024-06-10T03:03:58.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:58.859+0000 7f5293fff640 1 --2- 172.21.15.52:0/1673184048 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5294101af0 0x7f5294198250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T03:03:58.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:58.859+0000 7f52995d5640 1 --2- 172.21.15.52:0/1673184048 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f52941024b0 0x7f529419f280 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f528c00cce0 tx=0x7f528c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:03:58.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:58.859+0000 7f5291ffb640 1 -- 172.21.15.52:0/1673184048 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f528c00f450 con 0x7f52941024b0 2024-06-10T03:03:58.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:58.859+0000 7f5291ffb640 1 -- 172.21.15.52:0/1673184048 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f528c00fad0 con 0x7f52941024b0 2024-06-10T03:03:58.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:58.859+0000 7f529b038640 1 -- 172.21.15.52:0/1673184048 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f529419fc10 con 0x7f52941024b0 2024-06-10T03:03:58.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:58.859+0000 7f5291ffb640 1 -- 172.21.15.52:0/1673184048 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f528c018c10 con 0x7f52941024b0 2024-06-10T03:03:58.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:58.859+0000 7f529b038640 1 -- 172.21.15.52:0/1673184048 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52941a0160 con 0x7f52941024b0 2024-06-10T03:03:58.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:58.860+0000 7f529b038640 1 -- 172.21.15.52:0/1673184048 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52941a0330 con 0x7f52941024b0 2024-06-10T03:03:58.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:58.861+0000 7f5291ffb640 1 -- 172.21.15.52:0/1673184048 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f528c00f5f0 con 0x7f52941024b0 2024-06-10T03:03:58.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:58.862+0000 7f5291ffb640 1 --2- 172.21.15.52:0/1673184048 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f527c0a2c10 0x7f527c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:58.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:58.862+0000 7f5298dd4640 1 --2- 172.21.15.52:0/1673184048 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f527c0a2c10 0x7f527c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:58.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:58.862+0000 7f5291ffb640 1 -- 172.21.15.52:0/1673184048 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f528c0cfb50 con 0x7f52941024b0 2024-06-10T03:03:58.866 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:58.863+0000 7f5298dd4640 1 --2- 172.21.15.52:0/1673184048 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f527c0a2c10 0x7f527c0a50d0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f52941991e0 tx=0x7f5284005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:03:58.866 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:58.867+0000 7f5291ffb640 1 -- 172.21.15.52:0/1673184048 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f528c014030 con 0x7f52941024b0 2024-06-10T03:03:59.052 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:59.052+0000 7f529b038640 1 -- 172.21.15.52:0/1673184048 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f52941a05a0 con 0x7f52941024b0 2024-06-10T03:03:59.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:59.052+0000 7f5291ffb640 1 -- 172.21.15.52:0/1673184048 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v100) ==== 100+0+11 (secure 0 0 0) 0x7f528c09c6a0 con 0x7f52941024b0 2024-06-10T03:03:59.053 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 64 2024-06-10T03:03:59.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:59.054+0000 7f529b038640 1 -- 172.21.15.52:0/1673184048 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f527c0a2c10 msgr2=0x7f527c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:03:59.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:59.054+0000 7f529b038640 1 --2- 172.21.15.52:0/1673184048 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f527c0a2c10 0x7f527c0a50d0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f52941991e0 tx=0x7f5284005f90 comp rx=0 tx=0).stop 2024-06-10T03:03:59.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:59.054+0000 7f529b038640 1 -- 172.21.15.52:0/1673184048 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f52941024b0 msgr2=0x7f529419f280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:03:59.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:59.054+0000 7f529b038640 1 --2- 172.21.15.52:0/1673184048 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f52941024b0 0x7f529419f280 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f528c00cce0 tx=0x7f528c00a620 comp rx=0 tx=0).stop 2024-06-10T03:03:59.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:59.054+0000 7f529b038640 1 -- 172.21.15.52:0/1673184048 shutdown_connections 2024-06-10T03:03:59.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:59.054+0000 7f529b038640 1 --2- 172.21.15.52:0/1673184048 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f52941024b0 0x7f529419f280 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:59.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:59.054+0000 7f529b038640 1 --2- 172.21.15.52:0/1673184048 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5294101af0 0x7f5294198250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:59.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:59.054+0000 7f529b038640 1 --2- 172.21.15.52:0/1673184048 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f527c0a2c10 0x7f527c0a50d0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:59.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:59.054+0000 7f529b038640 1 --2- 172.21.15.52:0/1673184048 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5294100900 0x7f5294197d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:59.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:59.054+0000 7f529b038640 1 -- 172.21.15.52:0/1673184048 >> 172.21.15.52:0/1673184048 conn(0x7f52940fbe90 msgr2=0x7f5294069fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:03:59.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:59.054+0000 7f529b038640 1 -- 172.21.15.52:0/1673184048 shutdown_connections 2024-06-10T03:03:59.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:59.054+0000 7f529b038640 1 -- 172.21.15.52:0/1673184048 wait complete. 2024-06-10T03:03:59.066 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-06-10T03:03:59.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:59.174+0000 7f51927b7640 1 -- 172.21.15.52:0/3164129871 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f518c107780 msgr2=0x7f518c109b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:03:59.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:59.174+0000 7f51927b7640 1 --2- 172.21.15.52:0/3164129871 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f518c107780 0x7f518c109b70 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f5184009ca0 tx=0x7f518402a8c0 comp rx=0 tx=0).stop 2024-06-10T03:03:59.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:59.174+0000 7f51927b7640 1 -- 172.21.15.52:0/3164129871 shutdown_connections 2024-06-10T03:03:59.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:59.174+0000 7f51927b7640 1 --2- 172.21.15.52:0/3164129871 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f518c107780 0x7f518c109b70 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:59.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:59.174+0000 7f51927b7640 1 --2- 172.21.15.52:0/3164129871 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f518c104e50 0x7f518c107240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:59.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:59.174+0000 7f51927b7640 1 --2- 172.21.15.52:0/3164129871 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f518c102520 0x7f518c104910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:59.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:59.174+0000 7f51927b7640 1 -- 172.21.15.52:0/3164129871 >> 172.21.15.52:0/3164129871 conn(0x7f518c0fbe90 msgr2=0x7f518c0fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:03:59.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:59.174+0000 7f51927b7640 1 -- 172.21.15.52:0/3164129871 shutdown_connections 2024-06-10T03:03:59.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:59.174+0000 7f51927b7640 1 -- 172.21.15.52:0/3164129871 wait complete. 2024-06-10T03:03:59.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:59.174+0000 7f51927b7640 1 Processor -- start 2024-06-10T03:03:59.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:59.174+0000 7f51927b7640 1 -- start start 2024-06-10T03:03:59.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:59.175+0000 7f51927b7640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f518c102520 0x7f518c197d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:59.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:59.175+0000 7f51927b7640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f518c104e50 0x7f518c198240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:59.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:59.175+0000 7f51927b7640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f518c107780 0x7f518c19f270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:59.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:59.175+0000 7f51927b7640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f518c10d6a0 con 0x7f518c107780 2024-06-10T03:03:59.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:59.175+0000 7f51927b7640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f518c10d520 con 0x7f518c104e50 2024-06-10T03:03:59.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:59.175+0000 7f51927b7640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f518c10d820 con 0x7f518c102520 2024-06-10T03:03:59.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:59.175+0000 7f5190d54640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f518c107780 0x7f518c19f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:59.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:59.175+0000 7f5190d54640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f518c107780 0x7f518c19f270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52118/0 (socket says 172.21.15.52:52118) 2024-06-10T03:03:59.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:59.175+0000 7f5190d54640 1 -- 172.21.15.52:0/1334580849 learned_addr learned my addr 172.21.15.52:0/1334580849 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:03:59.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:59.175+0000 7f518bfff640 1 --2- 172.21.15.52:0/1334580849 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f518c102520 0x7f518c197d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:59.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:59.175+0000 7f518b7fe640 1 --2- 172.21.15.52:0/1334580849 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f518c104e50 0x7f518c198240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:59.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:59.175+0000 7f5190d54640 1 -- 172.21.15.52:0/1334580849 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f518c102520 msgr2=0x7f518c197d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:03:59.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:59.175+0000 7f5190d54640 1 --2- 172.21.15.52:0/1334580849 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f518c102520 0x7f518c197d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:59.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:59.175+0000 7f5190d54640 1 -- 172.21.15.52:0/1334580849 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f518c104e50 msgr2=0x7f518c198240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:03:59.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:59.175+0000 7f5190d54640 1 --2- 172.21.15.52:0/1334580849 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f518c104e50 0x7f518c198240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:59.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:59.175+0000 7f5190d54640 1 -- 172.21.15.52:0/1334580849 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f518c19f910 con 0x7f518c107780 2024-06-10T03:03:59.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:59.175+0000 7f5190d54640 1 --2- 172.21.15.52:0/1334580849 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f518c107780 0x7f518c19f270 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f5184031ae0 tx=0x7f5184031ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:03:59.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:59.175+0000 7f51897fa640 1 -- 172.21.15.52:0/1334580849 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5184003b70 con 0x7f518c107780 2024-06-10T03:03:59.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:59.175+0000 7f51927b7640 1 -- 172.21.15.52:0/1334580849 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f518c19fba0 con 0x7f518c107780 2024-06-10T03:03:59.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:59.175+0000 7f51897fa640 1 -- 172.21.15.52:0/1334580849 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5184003d10 con 0x7f518c107780 2024-06-10T03:03:59.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:59.175+0000 7f51897fa640 1 -- 172.21.15.52:0/1334580849 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f518402fc40 con 0x7f518c107780 2024-06-10T03:03:59.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:59.176+0000 7f51927b7640 1 -- 172.21.15.52:0/1334580849 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f518c1a0140 con 0x7f518c107780 2024-06-10T03:03:59.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:59.176+0000 7f51927b7640 1 -- 172.21.15.52:0/1334580849 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f518c0fd980 con 0x7f518c107780 2024-06-10T03:03:59.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:59.178+0000 7f51897fa640 1 -- 172.21.15.52:0/1334580849 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f5184004050 con 0x7f518c107780 2024-06-10T03:03:59.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:59.179+0000 7f51897fa640 1 --2- 172.21.15.52:0/1334580849 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f51740a2b90 0x7f51740a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:59.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:59.179+0000 7f51897fa640 1 -- 172.21.15.52:0/1334580849 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f51840f08c0 con 0x7f518c107780 2024-06-10T03:03:59.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:59.179+0000 7f518bfff640 1 --2- 172.21.15.52:0/1334580849 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f51740a2b90 0x7f51740a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:59.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:59.179+0000 7f518bfff640 1 --2- 172.21.15.52:0/1334580849 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f51740a2b90 0x7f51740a5050 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f518c1991d0 tx=0x7f517c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:03:59.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:59.183+0000 7f51897fa640 1 -- 172.21.15.52:0/1334580849 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f51840bd470 con 0x7f518c107780 2024-06-10T03:03:59.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:59.370+0000 7f51927b7640 1 -- 172.21.15.52:0/1334580849 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f518c02d470 con 0x7f518c107780 2024-06-10T03:03:59.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:59.370+0000 7f51897fa640 1 -- 172.21.15.52:0/1334580849 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v100) ==== 103+0+11 (secure 0 0 0) 0x7f51840bcd70 con 0x7f518c107780 2024-06-10T03:03:59.371 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 32 2024-06-10T03:03:59.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:59.372+0000 7f51927b7640 1 -- 172.21.15.52:0/1334580849 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f51740a2b90 msgr2=0x7f51740a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:03:59.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:59.372+0000 7f51927b7640 1 --2- 172.21.15.52:0/1334580849 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f51740a2b90 0x7f51740a5050 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f518c1991d0 tx=0x7f517c005f90 comp rx=0 tx=0).stop 2024-06-10T03:03:59.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:59.372+0000 7f51927b7640 1 -- 172.21.15.52:0/1334580849 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f518c107780 msgr2=0x7f518c19f270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:03:59.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:59.372+0000 7f51927b7640 1 --2- 172.21.15.52:0/1334580849 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f518c107780 0x7f518c19f270 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f5184031ae0 tx=0x7f5184031ee0 comp rx=0 tx=0).stop 2024-06-10T03:03:59.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:59.372+0000 7f51927b7640 1 -- 172.21.15.52:0/1334580849 shutdown_connections 2024-06-10T03:03:59.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:59.372+0000 7f51927b7640 1 --2- 172.21.15.52:0/1334580849 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f518c107780 0x7f518c19f270 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:59.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:59.372+0000 7f51927b7640 1 --2- 172.21.15.52:0/1334580849 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f518c104e50 0x7f518c198240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:59.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:59.372+0000 7f51927b7640 1 --2- 172.21.15.52:0/1334580849 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f51740a2b90 0x7f51740a5050 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:59.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:59.372+0000 7f51927b7640 1 --2- 172.21.15.52:0/1334580849 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f518c102520 0x7f518c197d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:59.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:59.372+0000 7f51927b7640 1 -- 172.21.15.52:0/1334580849 >> 172.21.15.52:0/1334580849 conn(0x7f518c0fbe90 msgr2=0x7f518c102e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:03:59.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:59.372+0000 7f51927b7640 1 -- 172.21.15.52:0/1334580849 shutdown_connections 2024-06-10T03:03:59.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:59.372+0000 7f51927b7640 1 -- 172.21.15.52:0/1334580849 wait complete. 2024-06-10T03:03:59.384 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-06-10T03:03:59.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:59.488+0000 7fd840c9e640 1 -- 172.21.15.52:0/3988787955 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd83c101540 msgr2=0x7fd83c108de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:03:59.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:59.488+0000 7fd840c9e640 1 --2- 172.21.15.52:0/3988787955 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd83c101540 0x7fd83c108de0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fd834009cd0 tx=0x7fd83402a8e0 comp rx=0 tx=0).stop 2024-06-10T03:03:59.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:59.489+0000 7fd840c9e640 1 -- 172.21.15.52:0/3988787955 shutdown_connections 2024-06-10T03:03:59.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:59.489+0000 7fd840c9e640 1 --2- 172.21.15.52:0/3988787955 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd83c101540 0x7fd83c108de0 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:59.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:59.489+0000 7fd840c9e640 1 --2- 172.21.15.52:0/3988787955 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd83c100b80 0x7fd83c101000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:59.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:59.489+0000 7fd840c9e640 1 --2- 172.21.15.52:0/3988787955 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd83c101ee0 0x7fd83c102300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:59.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:59.489+0000 7fd840c9e640 1 -- 172.21.15.52:0/3988787955 >> 172.21.15.52:0/3988787955 conn(0x7fd83c0fb610 msgr2=0x7fd83c0fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:03:59.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:59.489+0000 7fd840c9e640 1 -- 172.21.15.52:0/3988787955 shutdown_connections 2024-06-10T03:03:59.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:59.489+0000 7fd840c9e640 1 -- 172.21.15.52:0/3988787955 wait complete. 2024-06-10T03:03:59.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:59.489+0000 7fd840c9e640 1 Processor -- start 2024-06-10T03:03:59.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:59.489+0000 7fd840c9e640 1 -- start start 2024-06-10T03:03:59.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:59.490+0000 7fd840c9e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd83c100b80 0x7fd83c10b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:59.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:59.490+0000 7fd840c9e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd83c101540 0x7fd83c1098b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:59.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:59.490+0000 7fd840c9e640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd83c101ee0 0x7fd83c109df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:59.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:59.490+0000 7fd840c9e640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fd83c10d720 con 0x7fd83c101ee0 2024-06-10T03:03:59.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:59.490+0000 7fd840c9e640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd83c10d5a0 con 0x7fd83c101540 2024-06-10T03:03:59.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:59.490+0000 7fd840c9e640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fd83c10d8a0 con 0x7fd83c100b80 2024-06-10T03:03:59.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:59.490+0000 7fd83ad9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd83c101ee0 0x7fd83c109df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:59.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:59.490+0000 7fd83ad9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd83c101ee0 0x7fd83c109df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52148/0 (socket says 172.21.15.52:52148) 2024-06-10T03:03:59.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:59.490+0000 7fd83ad9d640 1 -- 172.21.15.52:0/3899309631 learned_addr learned my addr 172.21.15.52:0/3899309631 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:03:59.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:59.490+0000 7fd839d9b640 1 --2- 172.21.15.52:0/3899309631 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd83c101540 0x7fd83c1098b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:59.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:59.490+0000 7fd83a59c640 1 --2- 172.21.15.52:0/3899309631 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd83c100b80 0x7fd83c10b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:59.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:59.490+0000 7fd83ad9d640 1 -- 172.21.15.52:0/3899309631 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd83c100b80 msgr2=0x7fd83c10b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:03:59.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:59.490+0000 7fd83ad9d640 1 --2- 172.21.15.52:0/3899309631 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd83c100b80 0x7fd83c10b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:59.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:59.490+0000 7fd83ad9d640 1 -- 172.21.15.52:0/3899309631 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd83c101540 msgr2=0x7fd83c1098b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:03:59.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:59.490+0000 7fd83ad9d640 1 --2- 172.21.15.52:0/3899309631 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd83c101540 0x7fd83c1098b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:03:59.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:59.490+0000 7fd83ad9d640 1 -- 172.21.15.52:0/3899309631 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd83c10a680 con 0x7fd83c101ee0 2024-06-10T03:03:59.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:59.490+0000 7fd83ad9d640 1 --2- 172.21.15.52:0/3899309631 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd83c101ee0 0x7fd83c109df0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fd83402ae00 tx=0x7fd834003b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:03:59.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:59.491+0000 7fd82b7fe640 1 -- 172.21.15.52:0/3899309631 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd834004240 con 0x7fd83c101ee0 2024-06-10T03:03:59.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:59.491+0000 7fd840c9e640 1 -- 172.21.15.52:0/3899309631 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd83c1a1d80 con 0x7fd83c101ee0 2024-06-10T03:03:59.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:59.491+0000 7fd82b7fe640 1 -- 172.21.15.52:0/3899309631 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd8340305a0 con 0x7fd83c101ee0 2024-06-10T03:03:59.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:59.491+0000 7fd82b7fe640 1 -- 172.21.15.52:0/3899309631 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8340466e0 con 0x7fd83c101ee0 2024-06-10T03:03:59.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:59.491+0000 7fd840c9e640 1 -- 172.21.15.52:0/3899309631 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd83c1a21e0 con 0x7fd83c101ee0 2024-06-10T03:03:59.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:59.492+0000 7fd840c9e640 1 -- 172.21.15.52:0/3899309631 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd83c107bf0 con 0x7fd83c101ee0 2024-06-10T03:03:59.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:59.493+0000 7fd82b7fe640 1 -- 172.21.15.52:0/3899309631 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fd834030ae0 con 0x7fd83c101ee0 2024-06-10T03:03:59.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:59.494+0000 7fd82b7fe640 1 --2- 172.21.15.52:0/3899309631 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fd8240a2b90 0x7fd8240a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:03:59.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:59.494+0000 7fd83a59c640 1 --2- 172.21.15.52:0/3899309631 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fd8240a2b90 0x7fd8240a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:03:59.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:59.494+0000 7fd82b7fe640 1 -- 172.21.15.52:0/3899309631 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7fd83402e030 con 0x7fd83c101ee0 2024-06-10T03:03:59.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:59.494+0000 7fd83a59c640 1 --2- 172.21.15.52:0/3899309631 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fd8240a2b90 0x7fd8240a5050 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fd82c0058c0 tx=0x7fd82c005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:03:59.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:59.499+0000 7fd82b7fe640 1 -- 172.21.15.52:0/3899309631 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fd83404b060 con 0x7fd83c101ee0 2024-06-10T03:03:59.638 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:03:59 smithi120 ceph-mon[36634]: from='client.? 172.21.15.52:0/284519069' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-06-10T03:03:59.638 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:03:59 smithi120 ceph-mon[36634]: from='client.? 172.21.15.52:0/1353692372' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-06-10T03:03:59.639 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:03:59 smithi120 ceph-mon[36634]: from='client.? 172.21.15.52:0/1673184048' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-06-10T03:03:59.704 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:03:59 smithi188 ceph-mon[36599]: from='client.? 172.21.15.52:0/284519069' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-06-10T03:03:59.704 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:03:59 smithi188 ceph-mon[36599]: from='client.? 172.21.15.52:0/1353692372' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-06-10T03:03:59.704 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:03:59 smithi188 ceph-mon[36599]: from='client.? 172.21.15.52:0/1673184048' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-06-10T03:03:59.727 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:03:59 smithi052 ceph-mon[32362]: from='client.? 172.21.15.52:0/284519069' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-06-10T03:03:59.727 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:03:59 smithi052 ceph-mon[32362]: from='client.? 172.21.15.52:0/1353692372' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-06-10T03:03:59.727 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:03:59 smithi052 ceph-mon[32362]: from='client.? 172.21.15.52:0/1673184048' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-06-10T03:03:59.733 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:59.733+0000 7fd840c9e640 1 -- 172.21.15.52:0/3899309631 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7fd83c0625a0 con 0x7fd83c101ee0 2024-06-10T03:03:59.741 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:59.742+0000 7fd82b7fe640 1 -- 172.21.15.52:0/3899309631 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v20) ==== 81+0+740371 (secure 0 0 0) 0x7fd8340bda00 con 0x7fd83c101ee0 2024-06-10T03:03:59.751 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-10T03:03:59.751 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-10T03:03:59.751 INFO:teuthology.orchestra.run.smithi052.stdout: "epoch": 20, 2024-06-10T03:03:59.752 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:03:59.752 INFO:teuthology.orchestra.run.smithi052.stdout: "active_gid": 14150, 2024-06-10T03:03:59.752 INFO:teuthology.orchestra.run.smithi052.stdout: "active_name": "x", 2024-06-10T03:03:59.752 INFO:teuthology.orchestra.run.smithi052.stdout: "active_addrs": { 2024-06-10T03:03:59.752 INFO:teuthology.orchestra.run.smithi052.stdout: "addrvec": [ 2024-06-10T03:03:59.752 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-10T03:03:59.752 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "v2", 2024-06-10T03:03:59.752 INFO:teuthology.orchestra.run.smithi052.stdout: "addr": "172.21.15.52:6800", 2024-06-10T03:03:59.752 INFO:teuthology.orchestra.run.smithi052.stdout: "nonce": 3214763969 2024-06-10T03:03:59.752 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.752 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-10T03:03:59.752 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "v1", 2024-06-10T03:03:59.752 INFO:teuthology.orchestra.run.smithi052.stdout: "addr": "172.21.15.52:6801", 2024-06-10T03:03:59.753 INFO:teuthology.orchestra.run.smithi052.stdout: "nonce": 3214763969 2024-06-10T03:03:59.753 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:03:59.753 INFO:teuthology.orchestra.run.smithi052.stdout: ] 2024-06-10T03:03:59.753 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.753 INFO:teuthology.orchestra.run.smithi052.stdout: "active_addr": "172.21.15.52:6801/3214763969", 2024-06-10T03:03:59.753 INFO:teuthology.orchestra.run.smithi052.stdout: "active_change": "2024-06-10T02:42:42.872425+0000", 2024-06-10T03:03:59.753 INFO:teuthology.orchestra.run.smithi052.stdout: "active_mgr_features": 4540701547738038271, 2024-06-10T03:03:59.753 INFO:teuthology.orchestra.run.smithi052.stdout: "available": true, 2024-06-10T03:03:59.753 INFO:teuthology.orchestra.run.smithi052.stdout: "standbys": [ 2024-06-10T03:03:59.753 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-10T03:03:59.753 INFO:teuthology.orchestra.run.smithi052.stdout: "gid": 24109, 2024-06-10T03:03:59.753 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "z", 2024-06-10T03:03:59.753 INFO:teuthology.orchestra.run.smithi052.stdout: "mgr_features": 4540701547738038271, 2024-06-10T03:03:59.754 INFO:teuthology.orchestra.run.smithi052.stdout: "available_modules": [ 2024-06-10T03:03:59.754 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-10T03:03:59.754 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "alerts", 2024-06-10T03:03:59.754 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-10T03:03:59.754 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-10T03:03:59.754 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-10T03:03:59.754 INFO:teuthology.orchestra.run.smithi052.stdout: "interval": { 2024-06-10T03:03:59.754 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "interval", 2024-06-10T03:03:59.754 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-06-10T03:03:59.754 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.754 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:03:59.754 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "60", 2024-06-10T03:03:59.754 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.755 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.755 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.755 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "How frequently to reexamine health status", 2024-06-10T03:03:59.755 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.755 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.755 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.755 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.755 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-10T03:03:59.755 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-10T03:03:59.755 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:03:59.755 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.755 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:03:59.755 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:03:59.756 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.756 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.756 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:03:59.756 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:03:59.756 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:03:59.756 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:03:59.756 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:03:59.756 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:03:59.756 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:03:59.756 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:03:59.756 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:03:59.756 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.756 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.757 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.757 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.757 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-10T03:03:59.757 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-10T03:03:59.757 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:03:59.757 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.757 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:03:59.757 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:03:59.757 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.757 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.757 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.757 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:03:59.757 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.758 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.758 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.758 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.758 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-10T03:03:59.758 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-10T03:03:59.758 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:03:59.758 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.758 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:03:59.758 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-10T03:03:59.758 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.758 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.758 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:03:59.758 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:03:59.759 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:03:59.759 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:03:59.759 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:03:59.759 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:03:59.759 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:03:59.759 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:03:59.759 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:03:59.759 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.759 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.759 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.759 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.759 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-10T03:03:59.759 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-10T03:03:59.759 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:03:59.760 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.760 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:03:59.760 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:03:59.760 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.760 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.760 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.760 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:03:59.760 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.760 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.760 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.760 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.760 INFO:teuthology.orchestra.run.smithi052.stdout: "smtp_destination": { 2024-06-10T03:03:59.760 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "smtp_destination", 2024-06-10T03:03:59.760 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:03:59.761 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.761 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:03:59.761 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:03:59.761 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.761 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.761 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.761 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Email address to send alerts to", 2024-06-10T03:03:59.761 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.761 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.761 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.761 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.761 INFO:teuthology.orchestra.run.smithi052.stdout: "smtp_from_name": { 2024-06-10T03:03:59.761 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "smtp_from_name", 2024-06-10T03:03:59.762 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:03:59.762 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.762 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:03:59.762 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "Ceph", 2024-06-10T03:03:59.762 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.762 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.762 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.762 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Email From: name", 2024-06-10T03:03:59.762 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.762 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.762 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.762 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.762 INFO:teuthology.orchestra.run.smithi052.stdout: "smtp_host": { 2024-06-10T03:03:59.763 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "smtp_host", 2024-06-10T03:03:59.763 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:03:59.763 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.763 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:03:59.763 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:03:59.763 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.763 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.763 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.763 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "SMTP server", 2024-06-10T03:03:59.763 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.763 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.763 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.763 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.763 INFO:teuthology.orchestra.run.smithi052.stdout: "smtp_password": { 2024-06-10T03:03:59.764 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "smtp_password", 2024-06-10T03:03:59.764 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:03:59.764 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.764 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:03:59.764 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:03:59.764 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.764 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.764 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.764 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Password to authenticate with", 2024-06-10T03:03:59.764 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.764 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.764 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.764 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.765 INFO:teuthology.orchestra.run.smithi052.stdout: "smtp_port": { 2024-06-10T03:03:59.765 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "smtp_port", 2024-06-10T03:03:59.765 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-10T03:03:59.765 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.765 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:03:59.765 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "465", 2024-06-10T03:03:59.765 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.765 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.765 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.765 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "SMTP port", 2024-06-10T03:03:59.765 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.765 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.765 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.765 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.766 INFO:teuthology.orchestra.run.smithi052.stdout: "smtp_sender": { 2024-06-10T03:03:59.766 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "smtp_sender", 2024-06-10T03:03:59.766 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:03:59.766 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.766 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:03:59.766 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:03:59.766 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.766 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.766 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.766 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "SMTP envelope sender", 2024-06-10T03:03:59.766 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.766 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.767 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.767 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.767 INFO:teuthology.orchestra.run.smithi052.stdout: "smtp_ssl": { 2024-06-10T03:03:59.767 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "smtp_ssl", 2024-06-10T03:03:59.767 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:03:59.767 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.767 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:03:59.767 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-10T03:03:59.767 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.767 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.767 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.767 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Use SSL to connect to SMTP server", 2024-06-10T03:03:59.768 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.768 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.768 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.768 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.768 INFO:teuthology.orchestra.run.smithi052.stdout: "smtp_user": { 2024-06-10T03:03:59.768 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "smtp_user", 2024-06-10T03:03:59.768 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:03:59.768 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.768 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:03:59.768 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:03:59.768 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.768 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.769 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.769 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "User to authenticate as", 2024-06-10T03:03:59.769 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.769 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.769 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.769 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:03:59.769 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:03:59.769 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.769 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-10T03:03:59.769 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "balancer", 2024-06-10T03:03:59.769 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-10T03:03:59.769 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-10T03:03:59.769 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-10T03:03:59.769 INFO:teuthology.orchestra.run.smithi052.stdout: "active": { 2024-06-10T03:03:59.770 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "active", 2024-06-10T03:03:59.770 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:03:59.770 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.770 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:03:59.770 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-10T03:03:59.770 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.770 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.770 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.770 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "automatically balance PGs across cluster", 2024-06-10T03:03:59.770 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.770 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.770 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.770 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.771 INFO:teuthology.orchestra.run.smithi052.stdout: "begin_time": { 2024-06-10T03:03:59.771 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "begin_time", 2024-06-10T03:03:59.771 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:03:59.771 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.771 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:03:59.771 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "0000", 2024-06-10T03:03:59.771 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.771 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.771 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.771 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "beginning time of day to automatically balance", 2024-06-10T03:03:59.771 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-06-10T03:03:59.771 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.771 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.772 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.772 INFO:teuthology.orchestra.run.smithi052.stdout: "begin_weekday": { 2024-06-10T03:03:59.772 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "begin_weekday", 2024-06-10T03:03:59.772 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "uint", 2024-06-10T03:03:59.772 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.772 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:03:59.772 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "0", 2024-06-10T03:03:59.772 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "0", 2024-06-10T03:03:59.772 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "6", 2024-06-10T03:03:59.772 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.772 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-06-10T03:03:59.772 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-06-10T03:03:59.772 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.772 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.773 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.773 INFO:teuthology.orchestra.run.smithi052.stdout: "crush_compat_max_iterations": { 2024-06-10T03:03:59.773 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "crush_compat_max_iterations", 2024-06-10T03:03:59.773 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "uint", 2024-06-10T03:03:59.773 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.773 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:03:59.773 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "25", 2024-06-10T03:03:59.773 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "1", 2024-06-10T03:03:59.773 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "250", 2024-06-10T03:03:59.773 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.773 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-06-10T03:03:59.773 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.773 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.774 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.774 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.774 INFO:teuthology.orchestra.run.smithi052.stdout: "crush_compat_metrics": { 2024-06-10T03:03:59.774 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "crush_compat_metrics", 2024-06-10T03:03:59.774 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:03:59.774 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.774 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:03:59.774 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "pgs,objects,bytes", 2024-06-10T03:03:59.774 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.774 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.774 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.774 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-06-10T03:03:59.774 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-06-10T03:03:59.775 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.775 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.775 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.775 INFO:teuthology.orchestra.run.smithi052.stdout: "crush_compat_step": { 2024-06-10T03:03:59.775 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "crush_compat_step", 2024-06-10T03:03:59.775 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "float", 2024-06-10T03:03:59.775 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.775 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:03:59.775 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "0.5", 2024-06-10T03:03:59.775 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "0.001", 2024-06-10T03:03:59.775 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "0.999", 2024-06-10T03:03:59.775 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.775 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "aggressiveness of optimization", 2024-06-10T03:03:59.776 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-06-10T03:03:59.776 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.776 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.776 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.776 INFO:teuthology.orchestra.run.smithi052.stdout: "end_time": { 2024-06-10T03:03:59.776 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "end_time", 2024-06-10T03:03:59.776 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:03:59.776 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.776 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:03:59.776 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "2359", 2024-06-10T03:03:59.776 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.776 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.776 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.776 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "ending time of day to automatically balance", 2024-06-10T03:03:59.777 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-06-10T03:03:59.777 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.777 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.777 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.777 INFO:teuthology.orchestra.run.smithi052.stdout: "end_weekday": { 2024-06-10T03:03:59.777 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "end_weekday", 2024-06-10T03:03:59.777 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "uint", 2024-06-10T03:03:59.777 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.777 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:03:59.777 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "0", 2024-06-10T03:03:59.777 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "0", 2024-06-10T03:03:59.777 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "6", 2024-06-10T03:03:59.777 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.778 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-06-10T03:03:59.778 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-06-10T03:03:59.778 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.778 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.778 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.778 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-10T03:03:59.778 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-10T03:03:59.778 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:03:59.778 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.778 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:03:59.778 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:03:59.778 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.778 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.779 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:03:59.779 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:03:59.779 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:03:59.779 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:03:59.779 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:03:59.779 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:03:59.779 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:03:59.779 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:03:59.779 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:03:59.779 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.779 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.779 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.779 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.779 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-10T03:03:59.780 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-10T03:03:59.780 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:03:59.780 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.780 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:03:59.780 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:03:59.780 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.780 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.780 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.780 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:03:59.780 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.780 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.780 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.781 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.781 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-10T03:03:59.781 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-10T03:03:59.781 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:03:59.781 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.781 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:03:59.781 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-10T03:03:59.781 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.781 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.781 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:03:59.781 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:03:59.781 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:03:59.781 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:03:59.781 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:03:59.782 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:03:59.782 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:03:59.782 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:03:59.782 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:03:59.782 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.782 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.782 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.782 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.782 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-10T03:03:59.782 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-10T03:03:59.782 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:03:59.782 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.783 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:03:59.783 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:03:59.783 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.783 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.783 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.783 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:03:59.783 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.783 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.783 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.783 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.783 INFO:teuthology.orchestra.run.smithi052.stdout: "min_score": { 2024-06-10T03:03:59.783 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "min_score", 2024-06-10T03:03:59.783 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "float", 2024-06-10T03:03:59.783 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.784 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:03:59.784 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "0", 2024-06-10T03:03:59.784 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.784 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.784 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.784 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-06-10T03:03:59.784 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.784 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.784 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.784 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.784 INFO:teuthology.orchestra.run.smithi052.stdout: "mode": { 2024-06-10T03:03:59.784 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "mode", 2024-06-10T03:03:59.784 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:03:59.785 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.785 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:03:59.785 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "upmap", 2024-06-10T03:03:59.785 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.785 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.785 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:03:59.785 INFO:teuthology.orchestra.run.smithi052.stdout: "crush-compat", 2024-06-10T03:03:59.785 INFO:teuthology.orchestra.run.smithi052.stdout: "none", 2024-06-10T03:03:59.785 INFO:teuthology.orchestra.run.smithi052.stdout: "read", 2024-06-10T03:03:59.785 INFO:teuthology.orchestra.run.smithi052.stdout: "upmap", 2024-06-10T03:03:59.785 INFO:teuthology.orchestra.run.smithi052.stdout: "upmap-read" 2024-06-10T03:03:59.785 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:03:59.785 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Balancer mode", 2024-06-10T03:03:59.785 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.786 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.786 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.786 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.786 INFO:teuthology.orchestra.run.smithi052.stdout: "pool_ids": { 2024-06-10T03:03:59.786 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "pool_ids", 2024-06-10T03:03:59.786 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:03:59.786 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.786 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:03:59.786 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:03:59.786 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.786 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.786 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.787 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-06-10T03:03:59.787 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.787 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.787 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.787 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.787 INFO:teuthology.orchestra.run.smithi052.stdout: "sleep_interval": { 2024-06-10T03:03:59.787 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "sleep_interval", 2024-06-10T03:03:59.787 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-06-10T03:03:59.787 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.787 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:03:59.787 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "60", 2024-06-10T03:03:59.787 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.787 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.788 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.788 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-06-10T03:03:59.788 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.788 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.788 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.788 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.788 INFO:teuthology.orchestra.run.smithi052.stdout: "upmap_max_deviation": { 2024-06-10T03:03:59.788 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "upmap_max_deviation", 2024-06-10T03:03:59.788 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-10T03:03:59.788 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.788 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:03:59.788 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "5", 2024-06-10T03:03:59.788 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "1", 2024-06-10T03:03:59.788 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.789 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.789 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "deviation below which no optimization is attempted", 2024-06-10T03:03:59.789 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-06-10T03:03:59.789 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.789 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.789 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.789 INFO:teuthology.orchestra.run.smithi052.stdout: "upmap_max_optimizations": { 2024-06-10T03:03:59.789 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "upmap_max_optimizations", 2024-06-10T03:03:59.789 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "uint", 2024-06-10T03:03:59.789 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.789 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:03:59.789 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "10", 2024-06-10T03:03:59.789 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.790 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.790 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.790 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-06-10T03:03:59.790 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.790 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.790 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.790 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:03:59.790 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:03:59.790 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.790 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-10T03:03:59.790 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "cephadm", 2024-06-10T03:03:59.790 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-10T03:03:59.791 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-10T03:03:59.791 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-10T03:03:59.791 INFO:teuthology.orchestra.run.smithi052.stdout: "agent_down_multiplier": { 2024-06-10T03:03:59.791 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "agent_down_multiplier", 2024-06-10T03:03:59.791 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "float", 2024-06-10T03:03:59.791 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.791 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:03:59.791 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "3.0", 2024-06-10T03:03:59.791 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.791 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.791 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.791 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-06-10T03:03:59.792 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.792 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.792 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.792 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.792 INFO:teuthology.orchestra.run.smithi052.stdout: "agent_refresh_rate": { 2024-06-10T03:03:59.792 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "agent_refresh_rate", 2024-06-10T03:03:59.792 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-06-10T03:03:59.792 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.792 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:03:59.792 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "20", 2024-06-10T03:03:59.792 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.792 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.792 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.792 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-06-10T03:03:59.793 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.793 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.793 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.793 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.793 INFO:teuthology.orchestra.run.smithi052.stdout: "agent_starting_port": { 2024-06-10T03:03:59.793 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "agent_starting_port", 2024-06-10T03:03:59.793 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-10T03:03:59.793 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.793 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:03:59.793 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "4721", 2024-06-10T03:03:59.793 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.793 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.793 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.793 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-06-10T03:03:59.794 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.794 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.794 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.794 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.794 INFO:teuthology.orchestra.run.smithi052.stdout: "allow_ptrace": { 2024-06-10T03:03:59.794 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "allow_ptrace", 2024-06-10T03:03:59.794 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:03:59.794 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.794 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:03:59.794 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:03:59.794 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.795 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.795 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.795 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-06-10T03:03:59.795 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-06-10T03:03:59.795 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.795 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.795 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.795 INFO:teuthology.orchestra.run.smithi052.stdout: "autotune_interval": { 2024-06-10T03:03:59.795 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "autotune_interval", 2024-06-10T03:03:59.795 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-06-10T03:03:59.795 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.795 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:03:59.795 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "600", 2024-06-10T03:03:59.796 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.796 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.796 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.796 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "how frequently to autotune daemon memory", 2024-06-10T03:03:59.796 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.796 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.796 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.796 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.796 INFO:teuthology.orchestra.run.smithi052.stdout: "autotune_memory_target_ratio": { 2024-06-10T03:03:59.796 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "autotune_memory_target_ratio", 2024-06-10T03:03:59.796 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "float", 2024-06-10T03:03:59.796 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.796 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:03:59.797 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "0.7", 2024-06-10T03:03:59.797 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.797 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.797 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.797 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-06-10T03:03:59.797 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.797 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.797 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.797 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.797 INFO:teuthology.orchestra.run.smithi052.stdout: "cephadm_log_destination": { 2024-06-10T03:03:59.797 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "cephadm_log_destination", 2024-06-10T03:03:59.797 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:03:59.797 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.798 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:03:59.798 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:03:59.798 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.798 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.798 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:03:59.798 INFO:teuthology.orchestra.run.smithi052.stdout: "file", 2024-06-10T03:03:59.798 INFO:teuthology.orchestra.run.smithi052.stdout: "file,syslog", 2024-06-10T03:03:59.798 INFO:teuthology.orchestra.run.smithi052.stdout: "syslog" 2024-06-10T03:03:59.798 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:03:59.798 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-06-10T03:03:59.798 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.798 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.798 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.799 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.799 INFO:teuthology.orchestra.run.smithi052.stdout: "cgroups_split": { 2024-06-10T03:03:59.799 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "cgroups_split", 2024-06-10T03:03:59.799 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:03:59.799 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.799 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:03:59.799 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-10T03:03:59.799 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.799 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.799 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.799 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-06-10T03:03:59.799 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.800 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.800 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.800 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.800 INFO:teuthology.orchestra.run.smithi052.stdout: "config_checks_enabled": { 2024-06-10T03:03:59.800 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "config_checks_enabled", 2024-06-10T03:03:59.800 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:03:59.800 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.800 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:03:59.800 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:03:59.800 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.800 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.800 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.800 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-06-10T03:03:59.800 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.801 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.801 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.801 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.801 INFO:teuthology.orchestra.run.smithi052.stdout: "config_dashboard": { 2024-06-10T03:03:59.801 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "config_dashboard", 2024-06-10T03:03:59.801 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:03:59.801 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.801 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:03:59.801 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-10T03:03:59.801 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.801 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.801 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.801 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-06-10T03:03:59.802 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.802 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.802 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.802 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.802 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_alertmanager": { 2024-06-10T03:03:59.802 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_alertmanager", 2024-06-10T03:03:59.802 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:03:59.802 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.802 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:03:59.802 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-06-10T03:03:59.802 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.802 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.802 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.803 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Prometheus container image", 2024-06-10T03:03:59.803 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.803 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.803 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.803 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.803 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_base": { 2024-06-10T03:03:59.803 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_base", 2024-06-10T03:03:59.803 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:03:59.803 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.803 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:03:59.803 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/ceph/ceph", 2024-06-10T03:03:59.803 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.803 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.804 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.804 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Container image name, without the tag", 2024-06-10T03:03:59.804 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.804 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.804 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.804 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.804 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_elasticsearch": { 2024-06-10T03:03:59.804 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_elasticsearch", 2024-06-10T03:03:59.804 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:03:59.804 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.804 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:03:59.804 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-06-10T03:03:59.804 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.805 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.805 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.805 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "elasticsearch container image", 2024-06-10T03:03:59.805 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.805 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.805 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.805 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.805 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_grafana": { 2024-06-10T03:03:59.805 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_grafana", 2024-06-10T03:03:59.805 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:03:59.805 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.805 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:03:59.805 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-06-10T03:03:59.805 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.806 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.806 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.806 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Prometheus container image", 2024-06-10T03:03:59.806 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.806 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.806 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.806 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.806 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_haproxy": { 2024-06-10T03:03:59.806 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_haproxy", 2024-06-10T03:03:59.806 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:03:59.806 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.806 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:03:59.806 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-06-10T03:03:59.807 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.807 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.807 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.807 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "HAproxy container image", 2024-06-10T03:03:59.807 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.807 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.807 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.807 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.807 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_jaeger_agent": { 2024-06-10T03:03:59.807 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_jaeger_agent", 2024-06-10T03:03:59.807 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:03:59.808 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.808 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:03:59.808 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-06-10T03:03:59.808 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.808 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.808 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.808 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Jaeger agent container image", 2024-06-10T03:03:59.808 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.808 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.808 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.808 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.808 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_jaeger_collector": { 2024-06-10T03:03:59.808 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_jaeger_collector", 2024-06-10T03:03:59.809 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:03:59.809 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.809 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:03:59.809 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-06-10T03:03:59.809 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.809 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.809 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.809 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Jaeger collector container image", 2024-06-10T03:03:59.809 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.809 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.809 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.809 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.809 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_jaeger_query": { 2024-06-10T03:03:59.809 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_jaeger_query", 2024-06-10T03:03:59.810 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:03:59.810 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.810 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:03:59.810 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-06-10T03:03:59.810 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.810 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.810 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.810 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Jaeger query container image", 2024-06-10T03:03:59.810 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.810 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.810 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.810 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.811 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_keepalived": { 2024-06-10T03:03:59.811 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_keepalived", 2024-06-10T03:03:59.811 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:03:59.811 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.811 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:03:59.811 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-06-10T03:03:59.811 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.811 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.811 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.811 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Keepalived container image", 2024-06-10T03:03:59.811 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.811 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.811 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.811 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.812 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_loki": { 2024-06-10T03:03:59.812 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_loki", 2024-06-10T03:03:59.812 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:03:59.812 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.812 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:03:59.812 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-06-10T03:03:59.812 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.812 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.812 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.812 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Loki container image", 2024-06-10T03:03:59.812 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.812 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.812 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.813 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.813 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_node_exporter": { 2024-06-10T03:03:59.813 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_node_exporter", 2024-06-10T03:03:59.813 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:03:59.813 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.813 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:03:59.813 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-06-10T03:03:59.813 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.813 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.813 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.813 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Prometheus container image", 2024-06-10T03:03:59.813 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.813 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.814 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.814 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.814 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_nvmeof": { 2024-06-10T03:03:59.814 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_nvmeof", 2024-06-10T03:03:59.814 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:03:59.814 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.814 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:03:59.814 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-06-10T03:03:59.814 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.814 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.814 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.814 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Nvme-of container image", 2024-06-10T03:03:59.814 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.815 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.815 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.815 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.815 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_prometheus": { 2024-06-10T03:03:59.815 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_prometheus", 2024-06-10T03:03:59.815 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:03:59.815 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.815 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:03:59.815 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-06-10T03:03:59.815 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.815 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.815 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.815 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Prometheus container image", 2024-06-10T03:03:59.816 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.816 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.816 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.816 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.816 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_promtail": { 2024-06-10T03:03:59.816 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_promtail", 2024-06-10T03:03:59.816 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:03:59.816 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.816 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:03:59.816 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-06-10T03:03:59.816 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.816 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.816 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.817 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Promtail container image", 2024-06-10T03:03:59.817 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.817 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.817 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.817 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.817 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_samba": { 2024-06-10T03:03:59.817 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_samba", 2024-06-10T03:03:59.817 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:03:59.817 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.817 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:03:59.817 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-06-10T03:03:59.817 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.817 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.818 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.818 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Samba/SMB container image", 2024-06-10T03:03:59.818 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.818 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.818 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.818 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.818 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_snmp_gateway": { 2024-06-10T03:03:59.818 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_snmp_gateway", 2024-06-10T03:03:59.818 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:03:59.818 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.818 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:03:59.818 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-06-10T03:03:59.818 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.818 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.819 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.819 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "SNMP Gateway container image", 2024-06-10T03:03:59.819 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.819 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.819 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.819 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.819 INFO:teuthology.orchestra.run.smithi052.stdout: "container_init": { 2024-06-10T03:03:59.819 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_init", 2024-06-10T03:03:59.819 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:03:59.819 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.819 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:03:59.819 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-10T03:03:59.820 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.820 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.820 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.820 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Run podman/docker with `--init`", 2024-06-10T03:03:59.820 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.820 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.820 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.820 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.820 INFO:teuthology.orchestra.run.smithi052.stdout: "daemon_cache_timeout": { 2024-06-10T03:03:59.820 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "daemon_cache_timeout", 2024-06-10T03:03:59.820 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-06-10T03:03:59.820 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.820 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:03:59.821 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "600", 2024-06-10T03:03:59.821 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.821 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.821 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.821 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-06-10T03:03:59.821 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.821 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.821 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.821 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.821 INFO:teuthology.orchestra.run.smithi052.stdout: "default_cephadm_command_timeout": { 2024-06-10T03:03:59.821 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "default_cephadm_command_timeout", 2024-06-10T03:03:59.821 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-10T03:03:59.821 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.822 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:03:59.822 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "900", 2024-06-10T03:03:59.822 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.822 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.822 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.822 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-06-10T03:03:59.822 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.822 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.822 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.822 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.822 INFO:teuthology.orchestra.run.smithi052.stdout: "default_registry": { 2024-06-10T03:03:59.822 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "default_registry", 2024-06-10T03:03:59.823 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:03:59.823 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.823 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:03:59.823 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "docker.io", 2024-06-10T03:03:59.823 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.823 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.823 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.823 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-06-10T03:03:59.823 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.823 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.823 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.823 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.823 INFO:teuthology.orchestra.run.smithi052.stdout: "device_cache_timeout": { 2024-06-10T03:03:59.824 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "device_cache_timeout", 2024-06-10T03:03:59.824 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-06-10T03:03:59.824 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.824 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:03:59.824 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "1800", 2024-06-10T03:03:59.824 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.824 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.824 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.824 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "seconds to cache device inventory", 2024-06-10T03:03:59.824 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.824 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.824 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.824 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.825 INFO:teuthology.orchestra.run.smithi052.stdout: "device_enhanced_scan": { 2024-06-10T03:03:59.825 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "device_enhanced_scan", 2024-06-10T03:03:59.825 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:03:59.825 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.825 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:03:59.825 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:03:59.825 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.825 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.825 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.825 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Use libstoragemgmt during device scans", 2024-06-10T03:03:59.825 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.825 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.825 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.825 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.826 INFO:teuthology.orchestra.run.smithi052.stdout: "facts_cache_timeout": { 2024-06-10T03:03:59.826 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "facts_cache_timeout", 2024-06-10T03:03:59.826 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-06-10T03:03:59.826 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.826 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:03:59.826 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "60", 2024-06-10T03:03:59.826 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.826 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.826 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.826 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "seconds to cache host facts data", 2024-06-10T03:03:59.827 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.827 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.827 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.827 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.827 INFO:teuthology.orchestra.run.smithi052.stdout: "grafana_dashboards_path": { 2024-06-10T03:03:59.827 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "grafana_dashboards_path", 2024-06-10T03:03:59.827 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:03:59.827 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.827 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:03:59.827 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-06-10T03:03:59.827 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.827 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.827 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.827 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-06-10T03:03:59.828 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.828 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.828 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.828 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.828 INFO:teuthology.orchestra.run.smithi052.stdout: "host_check_interval": { 2024-06-10T03:03:59.828 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "host_check_interval", 2024-06-10T03:03:59.828 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-06-10T03:03:59.828 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.828 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:03:59.828 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "600", 2024-06-10T03:03:59.828 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.828 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.828 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.829 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "how frequently to perform a host check", 2024-06-10T03:03:59.829 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.829 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.829 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.829 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.829 INFO:teuthology.orchestra.run.smithi052.stdout: "hw_monitoring": { 2024-06-10T03:03:59.829 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "hw_monitoring", 2024-06-10T03:03:59.829 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:03:59.829 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.829 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:03:59.829 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:03:59.829 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.829 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.830 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.830 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-06-10T03:03:59.830 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.830 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.830 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.830 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.830 INFO:teuthology.orchestra.run.smithi052.stdout: "inventory_list_all": { 2024-06-10T03:03:59.830 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "inventory_list_all", 2024-06-10T03:03:59.830 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:03:59.830 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.830 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:03:59.830 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:03:59.830 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.831 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.831 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.831 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-06-10T03:03:59.831 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.831 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.831 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.831 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.831 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-10T03:03:59.831 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-10T03:03:59.831 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:03:59.831 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.831 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:03:59.831 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:03:59.832 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.832 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.832 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:03:59.832 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:03:59.832 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:03:59.832 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:03:59.832 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:03:59.832 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:03:59.832 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:03:59.832 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:03:59.832 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:03:59.832 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.832 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.833 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.833 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.833 INFO:teuthology.orchestra.run.smithi052.stdout: "log_refresh_metadata": { 2024-06-10T03:03:59.833 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_refresh_metadata", 2024-06-10T03:03:59.833 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:03:59.833 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.833 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:03:59.833 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:03:59.833 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.833 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.833 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.833 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-06-10T03:03:59.834 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.834 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.834 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.834 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.834 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-10T03:03:59.834 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-10T03:03:59.834 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:03:59.834 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.834 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:03:59.834 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-10T03:03:59.834 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.834 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.834 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.835 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-06-10T03:03:59.835 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.835 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.835 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.835 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.835 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-10T03:03:59.835 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-10T03:03:59.835 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:03:59.835 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.835 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:03:59.835 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-10T03:03:59.835 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.835 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.836 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:03:59.836 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:03:59.836 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:03:59.836 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:03:59.836 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:03:59.836 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:03:59.836 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:03:59.836 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:03:59.836 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:03:59.836 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.836 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.836 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.836 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.837 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-10T03:03:59.837 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-10T03:03:59.837 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:03:59.837 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.837 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:03:59.837 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:03:59.837 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.837 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.837 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.837 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:03:59.837 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.837 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.837 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.837 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.838 INFO:teuthology.orchestra.run.smithi052.stdout: "manage_etc_ceph_ceph_conf": { 2024-06-10T03:03:59.838 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-06-10T03:03:59.838 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:03:59.838 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.838 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:03:59.838 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:03:59.838 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.838 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.838 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.838 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-06-10T03:03:59.838 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.838 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.838 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.839 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.839 INFO:teuthology.orchestra.run.smithi052.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-06-10T03:03:59.839 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-06-10T03:03:59.839 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:03:59.839 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.839 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:03:59.839 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "*", 2024-06-10T03:03:59.839 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.839 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.839 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.839 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-06-10T03:03:59.839 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.839 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.839 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.840 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.840 INFO:teuthology.orchestra.run.smithi052.stdout: "max_count_per_host": { 2024-06-10T03:03:59.840 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "max_count_per_host", 2024-06-10T03:03:59.840 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-10T03:03:59.840 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.840 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:03:59.840 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "10", 2024-06-10T03:03:59.840 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.840 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.840 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.840 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "max number of daemons per service per host", 2024-06-10T03:03:59.840 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.840 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.841 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.841 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.841 INFO:teuthology.orchestra.run.smithi052.stdout: "max_osd_draining_count": { 2024-06-10T03:03:59.841 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "max_osd_draining_count", 2024-06-10T03:03:59.841 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-10T03:03:59.841 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.841 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:03:59.841 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "10", 2024-06-10T03:03:59.841 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.841 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.841 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.841 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-06-10T03:03:59.841 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.841 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.842 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.842 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.842 INFO:teuthology.orchestra.run.smithi052.stdout: "migration_current": { 2024-06-10T03:03:59.842 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "migration_current", 2024-06-10T03:03:59.842 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-10T03:03:59.842 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.842 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:03:59.842 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:03:59.842 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.842 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.842 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.842 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "internal - do not modify", 2024-06-10T03:03:59.842 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.842 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.843 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.843 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.843 INFO:teuthology.orchestra.run.smithi052.stdout: "mode": { 2024-06-10T03:03:59.843 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "mode", 2024-06-10T03:03:59.843 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:03:59.843 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.843 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:03:59.843 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "root", 2024-06-10T03:03:59.843 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.843 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.843 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:03:59.843 INFO:teuthology.orchestra.run.smithi052.stdout: "cephadm-package", 2024-06-10T03:03:59.843 INFO:teuthology.orchestra.run.smithi052.stdout: "root" 2024-06-10T03:03:59.844 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:03:59.844 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "mode for remote execution of cephadm", 2024-06-10T03:03:59.844 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.844 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.844 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.844 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.844 INFO:teuthology.orchestra.run.smithi052.stdout: "oob_default_addr": { 2024-06-10T03:03:59.844 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "oob_default_addr", 2024-06-10T03:03:59.844 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:03:59.844 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.844 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:03:59.844 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "169.254.1.1", 2024-06-10T03:03:59.844 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.844 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.845 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.845 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Default address for RedFish API (oob management).", 2024-06-10T03:03:59.845 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.845 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.845 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.845 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.845 INFO:teuthology.orchestra.run.smithi052.stdout: "prometheus_alerts_path": { 2024-06-10T03:03:59.845 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "prometheus_alerts_path", 2024-06-10T03:03:59.845 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:03:59.845 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.845 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:03:59.845 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-06-10T03:03:59.845 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.845 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.846 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.846 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-06-10T03:03:59.846 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.846 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.846 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.846 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.846 INFO:teuthology.orchestra.run.smithi052.stdout: "registry_insecure": { 2024-06-10T03:03:59.846 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "registry_insecure", 2024-06-10T03:03:59.846 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:03:59.846 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.846 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:03:59.846 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:03:59.847 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.847 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.847 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.847 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-06-10T03:03:59.847 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.847 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.847 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.847 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.847 INFO:teuthology.orchestra.run.smithi052.stdout: "registry_password": { 2024-06-10T03:03:59.847 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "registry_password", 2024-06-10T03:03:59.847 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:03:59.847 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.847 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:03:59.848 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:03:59.848 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.848 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.848 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.848 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-06-10T03:03:59.848 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.848 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.848 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.848 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.848 INFO:teuthology.orchestra.run.smithi052.stdout: "registry_url": { 2024-06-10T03:03:59.848 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "registry_url", 2024-06-10T03:03:59.848 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:03:59.848 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.849 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:03:59.849 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:03:59.849 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.849 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.849 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.849 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-06-10T03:03:59.849 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.849 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.849 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.849 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.849 INFO:teuthology.orchestra.run.smithi052.stdout: "registry_username": { 2024-06-10T03:03:59.849 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "registry_username", 2024-06-10T03:03:59.849 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:03:59.849 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.850 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:03:59.850 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:03:59.850 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.850 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.850 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.850 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-06-10T03:03:59.850 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.850 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.850 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.850 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.850 INFO:teuthology.orchestra.run.smithi052.stdout: "secure_monitoring_stack": { 2024-06-10T03:03:59.850 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "secure_monitoring_stack", 2024-06-10T03:03:59.851 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:03:59.851 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.851 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:03:59.851 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:03:59.851 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.851 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.851 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.851 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-06-10T03:03:59.851 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.851 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.851 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.851 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.851 INFO:teuthology.orchestra.run.smithi052.stdout: "service_discovery_port": { 2024-06-10T03:03:59.851 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "service_discovery_port", 2024-06-10T03:03:59.852 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-10T03:03:59.852 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.852 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:03:59.852 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "8765", 2024-06-10T03:03:59.852 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.852 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.852 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.852 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "cephadm service discovery port", 2024-06-10T03:03:59.852 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.852 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.852 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.852 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.853 INFO:teuthology.orchestra.run.smithi052.stdout: "ssh_config_file": { 2024-06-10T03:03:59.853 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "ssh_config_file", 2024-06-10T03:03:59.853 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:03:59.853 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.853 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:03:59.853 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:03:59.853 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.853 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.853 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.853 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-06-10T03:03:59.853 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.853 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.853 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.854 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.854 INFO:teuthology.orchestra.run.smithi052.stdout: "use_agent": { 2024-06-10T03:03:59.854 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "use_agent", 2024-06-10T03:03:59.854 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:03:59.854 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.854 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:03:59.854 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:03:59.854 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.854 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.854 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.854 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-06-10T03:03:59.854 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.854 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.855 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.855 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.855 INFO:teuthology.orchestra.run.smithi052.stdout: "use_repo_digest": { 2024-06-10T03:03:59.855 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "use_repo_digest", 2024-06-10T03:03:59.855 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:03:59.855 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.855 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:03:59.855 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-10T03:03:59.855 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.855 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.855 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.855 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-06-10T03:03:59.855 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.856 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.856 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.856 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.856 INFO:teuthology.orchestra.run.smithi052.stdout: "warn_on_failed_host_check": { 2024-06-10T03:03:59.856 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "warn_on_failed_host_check", 2024-06-10T03:03:59.856 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:03:59.856 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.856 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:03:59.856 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-10T03:03:59.856 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.856 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.856 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.856 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "raise a health warning if the host check fails", 2024-06-10T03:03:59.856 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.857 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.857 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.857 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.857 INFO:teuthology.orchestra.run.smithi052.stdout: "warn_on_stray_daemons": { 2024-06-10T03:03:59.857 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "warn_on_stray_daemons", 2024-06-10T03:03:59.857 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:03:59.857 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.857 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:03:59.857 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-10T03:03:59.857 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.857 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.857 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.857 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-06-10T03:03:59.858 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.858 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.858 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.858 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.858 INFO:teuthology.orchestra.run.smithi052.stdout: "warn_on_stray_hosts": { 2024-06-10T03:03:59.858 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "warn_on_stray_hosts", 2024-06-10T03:03:59.858 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:03:59.858 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.858 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:03:59.858 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-10T03:03:59.858 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.858 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.858 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.859 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-06-10T03:03:59.859 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.859 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.859 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.859 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:03:59.859 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:03:59.859 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.859 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-10T03:03:59.859 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "crash", 2024-06-10T03:03:59.859 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-10T03:03:59.859 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-10T03:03:59.859 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-10T03:03:59.860 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-10T03:03:59.860 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-10T03:03:59.860 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:03:59.860 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.860 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:03:59.860 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:03:59.860 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.860 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.860 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:03:59.860 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:03:59.860 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:03:59.860 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:03:59.860 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:03:59.860 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:03:59.861 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:03:59.861 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:03:59.861 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:03:59.861 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.861 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.861 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.861 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.861 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-10T03:03:59.861 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-10T03:03:59.861 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:03:59.861 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.861 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:03:59.861 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:03:59.862 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.862 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.862 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.862 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:03:59.862 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.862 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.862 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.862 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.862 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-10T03:03:59.862 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-10T03:03:59.862 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:03:59.862 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.862 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:03:59.863 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-10T03:03:59.863 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.863 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.863 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:03:59.863 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:03:59.863 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:03:59.863 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:03:59.863 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:03:59.863 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:03:59.863 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:03:59.863 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:03:59.863 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:03:59.863 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.863 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.864 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.864 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.864 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-10T03:03:59.864 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-10T03:03:59.864 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:03:59.864 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.864 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:03:59.864 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:03:59.864 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.864 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.864 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.864 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:03:59.864 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.865 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.865 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.865 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.865 INFO:teuthology.orchestra.run.smithi052.stdout: "retain_interval": { 2024-06-10T03:03:59.865 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "retain_interval", 2024-06-10T03:03:59.865 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-06-10T03:03:59.865 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.865 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:03:59.865 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "31536000", 2024-06-10T03:03:59.865 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.865 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.865 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.865 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "how long to retain crashes before pruning them", 2024-06-10T03:03:59.865 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.866 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.866 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.866 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.866 INFO:teuthology.orchestra.run.smithi052.stdout: "warn_recent_interval": { 2024-06-10T03:03:59.866 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "warn_recent_interval", 2024-06-10T03:03:59.866 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-06-10T03:03:59.866 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.866 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:03:59.866 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "1209600", 2024-06-10T03:03:59.866 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.866 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.866 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.866 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "time interval in which to warn about recent crashes", 2024-06-10T03:03:59.867 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.867 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.867 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.867 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:03:59.867 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:03:59.867 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.867 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-10T03:03:59.867 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "dashboard", 2024-06-10T03:03:59.867 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-10T03:03:59.867 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-10T03:03:59.867 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-10T03:03:59.867 INFO:teuthology.orchestra.run.smithi052.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-06-10T03:03:59.867 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-06-10T03:03:59.868 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-10T03:03:59.868 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.868 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:03:59.868 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "10", 2024-06-10T03:03:59.868 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.868 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.868 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.868 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:03:59.868 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.868 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.868 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.868 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.868 INFO:teuthology.orchestra.run.smithi052.stdout: "ALERTMANAGER_API_HOST": { 2024-06-10T03:03:59.869 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "ALERTMANAGER_API_HOST", 2024-06-10T03:03:59.869 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:03:59.869 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.869 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:03:59.869 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:03:59.869 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.869 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.869 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.869 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:03:59.869 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.869 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.869 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.869 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.869 INFO:teuthology.orchestra.run.smithi052.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-06-10T03:03:59.870 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-06-10T03:03:59.870 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:03:59.870 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.870 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:03:59.870 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-10T03:03:59.870 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.870 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.870 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.870 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:03:59.870 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.870 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.870 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.871 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.871 INFO:teuthology.orchestra.run.smithi052.stdout: "AUDIT_API_ENABLED": { 2024-06-10T03:03:59.871 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "AUDIT_API_ENABLED", 2024-06-10T03:03:59.871 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:03:59.871 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.871 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:03:59.871 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:03:59.871 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.871 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.871 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.871 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:03:59.871 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.871 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.871 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.872 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.872 INFO:teuthology.orchestra.run.smithi052.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-06-10T03:03:59.872 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-06-10T03:03:59.872 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:03:59.872 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.872 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:03:59.872 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-10T03:03:59.872 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.872 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.872 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.872 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:03:59.872 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.872 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.873 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.873 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.873 INFO:teuthology.orchestra.run.smithi052.stdout: "ENABLE_BROWSABLE_API": { 2024-06-10T03:03:59.873 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "ENABLE_BROWSABLE_API", 2024-06-10T03:03:59.873 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:03:59.873 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.873 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:03:59.873 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-10T03:03:59.873 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.873 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.873 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.873 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:03:59.873 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.874 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.874 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.874 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.874 INFO:teuthology.orchestra.run.smithi052.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-06-10T03:03:59.874 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-06-10T03:03:59.874 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:03:59.874 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.874 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:03:59.874 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-10T03:03:59.874 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.874 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.874 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.875 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:03:59.875 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.875 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.875 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.875 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.875 INFO:teuthology.orchestra.run.smithi052.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-06-10T03:03:59.875 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-06-10T03:03:59.875 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:03:59.875 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.875 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:03:59.875 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-10T03:03:59.875 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.875 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.876 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.876 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:03:59.876 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.876 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.876 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.876 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.876 INFO:teuthology.orchestra.run.smithi052.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-06-10T03:03:59.876 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-06-10T03:03:59.876 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:03:59.876 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.876 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:03:59.876 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-10T03:03:59.876 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.876 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.877 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.877 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:03:59.877 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.877 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.877 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.877 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.877 INFO:teuthology.orchestra.run.smithi052.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-06-10T03:03:59.877 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-06-10T03:03:59.877 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:03:59.877 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.877 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:03:59.877 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-10T03:03:59.877 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.878 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.878 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.878 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:03:59.878 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.878 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.878 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.878 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.878 INFO:teuthology.orchestra.run.smithi052.stdout: "FEATURE_TOGGLE_NFS": { 2024-06-10T03:03:59.878 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-06-10T03:03:59.878 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:03:59.878 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.878 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:03:59.878 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-10T03:03:59.879 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.879 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.879 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.879 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:03:59.879 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.879 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.879 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.879 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.879 INFO:teuthology.orchestra.run.smithi052.stdout: "FEATURE_TOGGLE_RBD": { 2024-06-10T03:03:59.879 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-06-10T03:03:59.879 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:03:59.879 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.879 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:03:59.880 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-10T03:03:59.880 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.880 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.880 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.880 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:03:59.880 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.880 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.880 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.880 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.880 INFO:teuthology.orchestra.run.smithi052.stdout: "FEATURE_TOGGLE_RGW": { 2024-06-10T03:03:59.880 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-06-10T03:03:59.880 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:03:59.880 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.880 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:03:59.881 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-10T03:03:59.881 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.881 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.881 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.881 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:03:59.881 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.881 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.881 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.881 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.881 INFO:teuthology.orchestra.run.smithi052.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-06-10T03:03:59.881 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-06-10T03:03:59.881 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:03:59.881 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.881 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:03:59.882 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:03:59.882 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.882 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.882 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.882 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:03:59.882 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.882 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.882 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.882 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.882 INFO:teuthology.orchestra.run.smithi052.stdout: "GRAFANA_API_PASSWORD": { 2024-06-10T03:03:59.882 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "GRAFANA_API_PASSWORD", 2024-06-10T03:03:59.882 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:03:59.882 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.883 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:03:59.883 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "admin", 2024-06-10T03:03:59.883 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.883 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.883 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.883 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:03:59.883 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.883 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.883 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.883 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.883 INFO:teuthology.orchestra.run.smithi052.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-06-10T03:03:59.883 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-06-10T03:03:59.883 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:03:59.883 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.884 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:03:59.884 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-10T03:03:59.884 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.884 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.884 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.884 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:03:59.884 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.884 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.884 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.884 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.884 INFO:teuthology.orchestra.run.smithi052.stdout: "GRAFANA_API_URL": { 2024-06-10T03:03:59.884 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "GRAFANA_API_URL", 2024-06-10T03:03:59.884 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:03:59.885 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.885 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:03:59.885 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:03:59.885 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.885 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.885 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.885 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:03:59.885 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.885 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.885 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.885 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.885 INFO:teuthology.orchestra.run.smithi052.stdout: "GRAFANA_API_USERNAME": { 2024-06-10T03:03:59.885 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "GRAFANA_API_USERNAME", 2024-06-10T03:03:59.885 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:03:59.886 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.886 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:03:59.886 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "admin", 2024-06-10T03:03:59.886 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.886 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.886 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.886 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:03:59.886 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.886 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.886 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.886 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.886 INFO:teuthology.orchestra.run.smithi052.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-06-10T03:03:59.886 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-06-10T03:03:59.887 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:03:59.887 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.887 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:03:59.887 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:03:59.887 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.887 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.887 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.887 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:03:59.887 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.887 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.887 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.887 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.887 INFO:teuthology.orchestra.run.smithi052.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-06-10T03:03:59.888 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-06-10T03:03:59.888 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:03:59.888 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.888 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:03:59.888 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:03:59.888 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.888 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.888 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.888 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:03:59.888 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.888 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.888 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.889 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.889 INFO:teuthology.orchestra.run.smithi052.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-06-10T03:03:59.889 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-06-10T03:03:59.889 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:03:59.889 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.889 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:03:59.889 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-10T03:03:59.889 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.889 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.889 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.889 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:03:59.889 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.889 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.890 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.890 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.890 INFO:teuthology.orchestra.run.smithi052.stdout: "ISSUE_TRACKER_API_KEY": { 2024-06-10T03:03:59.890 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-06-10T03:03:59.890 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:03:59.890 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.890 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:03:59.890 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:03:59.890 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.890 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.890 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.890 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:03:59.890 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.891 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.891 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.891 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.891 INFO:teuthology.orchestra.run.smithi052.stdout: "PROMETHEUS_API_HOST": { 2024-06-10T03:03:59.891 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PROMETHEUS_API_HOST", 2024-06-10T03:03:59.891 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:03:59.891 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.891 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:03:59.891 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:03:59.891 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.891 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.891 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.891 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:03:59.891 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.892 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.892 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.892 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.892 INFO:teuthology.orchestra.run.smithi052.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-06-10T03:03:59.892 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-06-10T03:03:59.892 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:03:59.892 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.892 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:03:59.892 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-10T03:03:59.892 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.892 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.892 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.892 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:03:59.893 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.893 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.893 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.893 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.893 INFO:teuthology.orchestra.run.smithi052.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-06-10T03:03:59.893 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-06-10T03:03:59.893 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:03:59.893 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.893 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:03:59.893 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:03:59.893 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.893 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.893 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.893 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:03:59.894 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.894 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.894 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.894 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.894 INFO:teuthology.orchestra.run.smithi052.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-06-10T03:03:59.894 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-06-10T03:03:59.894 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:03:59.894 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.894 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:03:59.894 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:03:59.894 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.894 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.894 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.895 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:03:59.895 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.895 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.895 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.895 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.895 INFO:teuthology.orchestra.run.smithi052.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-06-10T03:03:59.895 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-06-10T03:03:59.895 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:03:59.895 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.895 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:03:59.895 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-10T03:03:59.895 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.895 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.896 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.896 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:03:59.896 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.896 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.896 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.896 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.896 INFO:teuthology.orchestra.run.smithi052.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-06-10T03:03:59.896 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-06-10T03:03:59.896 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:03:59.896 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.896 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:03:59.896 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-10T03:03:59.896 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.896 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.897 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.897 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:03:59.897 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.897 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.897 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.897 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.897 INFO:teuthology.orchestra.run.smithi052.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-06-10T03:03:59.897 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-06-10T03:03:59.897 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:03:59.897 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.897 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:03:59.897 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:03:59.897 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.898 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.898 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.898 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:03:59.898 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.898 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.898 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.898 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.898 INFO:teuthology.orchestra.run.smithi052.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-06-10T03:03:59.898 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-06-10T03:03:59.898 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:03:59.898 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.898 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:03:59.898 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:03:59.899 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.899 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.899 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.899 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:03:59.899 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.899 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.899 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.899 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.899 INFO:teuthology.orchestra.run.smithi052.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-06-10T03:03:59.899 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-06-10T03:03:59.899 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:03:59.899 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.899 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:03:59.899 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:03:59.900 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.900 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.900 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.900 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:03:59.900 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.900 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.900 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.900 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.900 INFO:teuthology.orchestra.run.smithi052.stdout: "PWD_POLICY_ENABLED": { 2024-06-10T03:03:59.900 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PWD_POLICY_ENABLED", 2024-06-10T03:03:59.900 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:03:59.900 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.900 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:03:59.901 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-10T03:03:59.901 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.901 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.901 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.901 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:03:59.901 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.901 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.901 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.901 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.901 INFO:teuthology.orchestra.run.smithi052.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-06-10T03:03:59.901 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-06-10T03:03:59.901 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:03:59.902 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.902 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:03:59.902 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-06-10T03:03:59.902 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.902 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.902 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.902 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:03:59.902 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.902 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.902 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.902 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.902 INFO:teuthology.orchestra.run.smithi052.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-06-10T03:03:59.903 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-06-10T03:03:59.903 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-10T03:03:59.903 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.903 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:03:59.903 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "10", 2024-06-10T03:03:59.903 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.903 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.903 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.903 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:03:59.903 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.903 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.903 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.903 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.903 INFO:teuthology.orchestra.run.smithi052.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-06-10T03:03:59.904 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-06-10T03:03:59.904 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-10T03:03:59.904 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.904 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:03:59.904 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "8", 2024-06-10T03:03:59.904 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.904 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.904 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.904 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:03:59.904 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.904 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.904 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.904 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.904 INFO:teuthology.orchestra.run.smithi052.stdout: "REST_REQUESTS_TIMEOUT": { 2024-06-10T03:03:59.904 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-06-10T03:03:59.905 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-10T03:03:59.905 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.905 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:03:59.905 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "45", 2024-06-10T03:03:59.905 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.905 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.905 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.905 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:03:59.905 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.905 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.905 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.905 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.906 INFO:teuthology.orchestra.run.smithi052.stdout: "RGW_API_ACCESS_KEY": { 2024-06-10T03:03:59.906 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "RGW_API_ACCESS_KEY", 2024-06-10T03:03:59.906 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:03:59.906 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.906 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:03:59.906 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:03:59.906 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.906 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.906 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.906 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:03:59.906 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.906 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.906 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.907 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.907 INFO:teuthology.orchestra.run.smithi052.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-06-10T03:03:59.907 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-06-10T03:03:59.907 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:03:59.907 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.907 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:03:59.907 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "admin", 2024-06-10T03:03:59.907 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.907 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.907 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.907 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:03:59.907 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.907 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.907 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.908 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.908 INFO:teuthology.orchestra.run.smithi052.stdout: "RGW_API_SECRET_KEY": { 2024-06-10T03:03:59.908 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "RGW_API_SECRET_KEY", 2024-06-10T03:03:59.908 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:03:59.908 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.908 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:03:59.908 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:03:59.908 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.908 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.908 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.908 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:03:59.908 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.908 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.908 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.909 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.909 INFO:teuthology.orchestra.run.smithi052.stdout: "RGW_API_SSL_VERIFY": { 2024-06-10T03:03:59.909 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "RGW_API_SSL_VERIFY", 2024-06-10T03:03:59.909 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:03:59.909 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.909 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:03:59.909 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-10T03:03:59.909 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.909 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.909 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.909 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:03:59.909 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.909 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.910 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.910 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.910 INFO:teuthology.orchestra.run.smithi052.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-06-10T03:03:59.910 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-06-10T03:03:59.910 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-10T03:03:59.910 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.910 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:03:59.910 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "0", 2024-06-10T03:03:59.910 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.914 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.914 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.914 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:03:59.914 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.914 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.914 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.914 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.914 INFO:teuthology.orchestra.run.smithi052.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-06-10T03:03:59.914 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-06-10T03:03:59.914 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-10T03:03:59.914 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.915 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:03:59.915 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "10", 2024-06-10T03:03:59.915 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.915 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.915 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.915 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:03:59.915 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.915 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.915 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.915 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.915 INFO:teuthology.orchestra.run.smithi052.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-06-10T03:03:59.915 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-06-10T03:03:59.915 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-10T03:03:59.916 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.916 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:03:59.916 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "5", 2024-06-10T03:03:59.916 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.916 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.916 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.916 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:03:59.916 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.916 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.916 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.916 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.916 INFO:teuthology.orchestra.run.smithi052.stdout: "cross_origin_url": { 2024-06-10T03:03:59.916 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "cross_origin_url", 2024-06-10T03:03:59.916 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:03:59.917 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.917 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:03:59.917 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:03:59.917 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.917 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.917 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.917 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:03:59.917 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.917 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.917 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.917 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.917 INFO:teuthology.orchestra.run.smithi052.stdout: "crt_file": { 2024-06-10T03:03:59.917 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "crt_file", 2024-06-10T03:03:59.918 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:03:59.918 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.918 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:03:59.918 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:03:59.918 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.918 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.918 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.918 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:03:59.918 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.918 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.918 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.918 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.918 INFO:teuthology.orchestra.run.smithi052.stdout: "debug": { 2024-06-10T03:03:59.919 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "debug", 2024-06-10T03:03:59.919 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:03:59.919 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.919 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:03:59.919 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:03:59.919 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.919 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.919 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.919 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Enable/disable debug options", 2024-06-10T03:03:59.919 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.919 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.919 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.920 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.920 INFO:teuthology.orchestra.run.smithi052.stdout: "jwt_token_ttl": { 2024-06-10T03:03:59.920 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "jwt_token_ttl", 2024-06-10T03:03:59.920 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-10T03:03:59.920 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.920 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:03:59.920 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "28800", 2024-06-10T03:03:59.920 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.920 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.920 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.920 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:03:59.920 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.920 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.921 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.921 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.921 INFO:teuthology.orchestra.run.smithi052.stdout: "key_file": { 2024-06-10T03:03:59.921 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "key_file", 2024-06-10T03:03:59.921 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:03:59.921 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.921 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:03:59.921 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:03:59.921 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.921 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.921 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.921 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:03:59.921 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.922 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.922 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.922 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.922 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-10T03:03:59.922 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-10T03:03:59.922 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:03:59.922 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.922 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:03:59.922 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:03:59.922 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.922 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.922 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:03:59.923 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:03:59.923 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:03:59.923 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:03:59.923 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:03:59.923 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:03:59.923 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:03:59.923 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:03:59.923 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:03:59.923 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.923 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.923 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.923 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.923 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-10T03:03:59.923 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-10T03:03:59.924 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:03:59.924 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.924 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:03:59.924 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:03:59.924 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.924 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.924 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.924 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:03:59.924 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.924 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.924 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.924 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.925 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-10T03:03:59.925 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-10T03:03:59.925 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:03:59.925 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.925 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:03:59.925 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-10T03:03:59.925 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.925 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.925 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:03:59.925 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:03:59.925 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:03:59.925 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:03:59.925 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:03:59.926 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:03:59.926 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:03:59.926 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:03:59.926 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:03:59.926 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.926 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.926 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.926 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.926 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-10T03:03:59.926 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-10T03:03:59.927 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:03:59.927 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.927 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:03:59.927 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:03:59.927 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.927 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.927 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.927 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:03:59.927 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.927 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.927 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.927 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.927 INFO:teuthology.orchestra.run.smithi052.stdout: "motd": { 2024-06-10T03:03:59.927 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "motd", 2024-06-10T03:03:59.928 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:03:59.928 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.928 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:03:59.928 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:03:59.928 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.928 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.928 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.928 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "The message of the day", 2024-06-10T03:03:59.928 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.928 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.928 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.928 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.929 INFO:teuthology.orchestra.run.smithi052.stdout: "redirect_resolve_ip_addr": { 2024-06-10T03:03:59.929 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "redirect_resolve_ip_addr", 2024-06-10T03:03:59.929 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:03:59.929 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.929 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:03:59.929 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:03:59.929 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.929 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.929 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.929 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:03:59.929 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.929 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.929 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.930 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.930 INFO:teuthology.orchestra.run.smithi052.stdout: "server_addr": { 2024-06-10T03:03:59.930 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "server_addr", 2024-06-10T03:03:59.930 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:03:59.930 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.930 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:03:59.930 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "::", 2024-06-10T03:03:59.930 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.930 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.930 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.930 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:03:59.930 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.930 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.931 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.931 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.931 INFO:teuthology.orchestra.run.smithi052.stdout: "server_port": { 2024-06-10T03:03:59.931 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "server_port", 2024-06-10T03:03:59.931 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-10T03:03:59.931 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.931 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:03:59.931 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "8080", 2024-06-10T03:03:59.931 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.931 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.931 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.931 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:03:59.931 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.932 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.932 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.932 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.932 INFO:teuthology.orchestra.run.smithi052.stdout: "ssl": { 2024-06-10T03:03:59.932 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "ssl", 2024-06-10T03:03:59.932 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:03:59.932 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.932 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:03:59.932 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-10T03:03:59.932 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.932 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.932 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.932 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:03:59.933 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.933 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.933 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.933 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.933 INFO:teuthology.orchestra.run.smithi052.stdout: "ssl_server_port": { 2024-06-10T03:03:59.933 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "ssl_server_port", 2024-06-10T03:03:59.933 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-10T03:03:59.933 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.933 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:03:59.933 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "8443", 2024-06-10T03:03:59.933 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.933 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.933 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.934 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:03:59.934 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.934 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.934 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.934 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.934 INFO:teuthology.orchestra.run.smithi052.stdout: "standby_behaviour": { 2024-06-10T03:03:59.934 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "standby_behaviour", 2024-06-10T03:03:59.934 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:03:59.934 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.934 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:03:59.934 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "redirect", 2024-06-10T03:03:59.935 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.935 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.935 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:03:59.935 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:03:59.935 INFO:teuthology.orchestra.run.smithi052.stdout: "redirect" 2024-06-10T03:03:59.935 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:03:59.935 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:03:59.935 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.935 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.935 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.935 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.935 INFO:teuthology.orchestra.run.smithi052.stdout: "standby_error_status_code": { 2024-06-10T03:03:59.935 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "standby_error_status_code", 2024-06-10T03:03:59.936 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-10T03:03:59.936 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.936 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:03:59.936 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "500", 2024-06-10T03:03:59.936 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "400", 2024-06-10T03:03:59.936 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "599", 2024-06-10T03:03:59.936 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.936 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:03:59.936 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.936 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.936 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.936 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.936 INFO:teuthology.orchestra.run.smithi052.stdout: "url_prefix": { 2024-06-10T03:03:59.936 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "url_prefix", 2024-06-10T03:03:59.937 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:03:59.937 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.937 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:03:59.937 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:03:59.937 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.937 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.937 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.937 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:03:59.937 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.937 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.937 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.937 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:03:59.937 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:03:59.937 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.938 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-10T03:03:59.938 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "devicehealth", 2024-06-10T03:03:59.938 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-10T03:03:59.938 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-10T03:03:59.938 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-10T03:03:59.938 INFO:teuthology.orchestra.run.smithi052.stdout: "enable_monitoring": { 2024-06-10T03:03:59.938 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "enable_monitoring", 2024-06-10T03:03:59.938 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:03:59.938 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.938 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:03:59.938 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-10T03:03:59.938 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.939 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.939 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.939 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "monitor device health metrics", 2024-06-10T03:03:59.939 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.939 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.939 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.939 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.939 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-10T03:03:59.939 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-10T03:03:59.939 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:03:59.939 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.939 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:03:59.939 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:03:59.940 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.940 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.940 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:03:59.940 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:03:59.940 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:03:59.940 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:03:59.940 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:03:59.940 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:03:59.940 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:03:59.940 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:03:59.940 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:03:59.940 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.940 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.940 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.941 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.941 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-10T03:03:59.941 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-10T03:03:59.941 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:03:59.941 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.941 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:03:59.941 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:03:59.941 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.941 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.941 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.941 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:03:59.941 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.941 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.942 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.942 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.942 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-10T03:03:59.942 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-10T03:03:59.942 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:03:59.942 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.942 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:03:59.942 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-10T03:03:59.942 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.942 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.942 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:03:59.942 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:03:59.942 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:03:59.943 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:03:59.943 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:03:59.943 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:03:59.943 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:03:59.943 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:03:59.943 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:03:59.943 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.943 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.943 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.943 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.943 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-10T03:03:59.943 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-10T03:03:59.943 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:03:59.943 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.944 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:03:59.944 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:03:59.944 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.944 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.944 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.944 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:03:59.944 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.944 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.944 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.944 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.944 INFO:teuthology.orchestra.run.smithi052.stdout: "mark_out_threshold": { 2024-06-10T03:03:59.944 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "mark_out_threshold", 2024-06-10T03:03:59.944 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-06-10T03:03:59.944 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.945 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:03:59.945 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "2419200", 2024-06-10T03:03:59.945 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.945 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.945 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.945 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-06-10T03:03:59.945 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.945 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.945 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.945 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.945 INFO:teuthology.orchestra.run.smithi052.stdout: "pool_name": { 2024-06-10T03:03:59.945 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "pool_name", 2024-06-10T03:03:59.945 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:03:59.946 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.946 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:03:59.946 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "device_health_metrics", 2024-06-10T03:03:59.946 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.946 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.946 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.946 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "name of pool in which to store device health metrics", 2024-06-10T03:03:59.946 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.946 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.946 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.946 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.946 INFO:teuthology.orchestra.run.smithi052.stdout: "retention_period": { 2024-06-10T03:03:59.946 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "retention_period", 2024-06-10T03:03:59.947 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-06-10T03:03:59.947 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.947 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:03:59.947 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "15552000", 2024-06-10T03:03:59.947 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.947 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.947 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.947 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "how long to retain device health metrics", 2024-06-10T03:03:59.947 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.947 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.947 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.947 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.947 INFO:teuthology.orchestra.run.smithi052.stdout: "scrape_frequency": { 2024-06-10T03:03:59.948 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "scrape_frequency", 2024-06-10T03:03:59.948 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-06-10T03:03:59.948 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.948 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:03:59.948 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "86400", 2024-06-10T03:03:59.948 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.948 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.948 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.948 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "how frequently to scrape device health metrics", 2024-06-10T03:03:59.948 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.948 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.948 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.949 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.949 INFO:teuthology.orchestra.run.smithi052.stdout: "self_heal": { 2024-06-10T03:03:59.949 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "self_heal", 2024-06-10T03:03:59.949 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:03:59.949 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.949 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:03:59.949 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-10T03:03:59.949 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.949 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.949 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.949 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-06-10T03:03:59.949 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.949 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.949 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.950 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.950 INFO:teuthology.orchestra.run.smithi052.stdout: "sleep_interval": { 2024-06-10T03:03:59.950 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "sleep_interval", 2024-06-10T03:03:59.950 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-06-10T03:03:59.950 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.950 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:03:59.950 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "600", 2024-06-10T03:03:59.950 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.950 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.950 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.950 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "how frequently to wake up and check device health", 2024-06-10T03:03:59.950 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.950 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.951 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.951 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.951 INFO:teuthology.orchestra.run.smithi052.stdout: "warn_threshold": { 2024-06-10T03:03:59.951 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "warn_threshold", 2024-06-10T03:03:59.951 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-06-10T03:03:59.951 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.951 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:03:59.951 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "7257600", 2024-06-10T03:03:59.951 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.951 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.951 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.951 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-06-10T03:03:59.951 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.952 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.952 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.952 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:03:59.952 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:03:59.952 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.952 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-10T03:03:59.952 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "diskprediction_local", 2024-06-10T03:03:59.952 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-10T03:03:59.952 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-10T03:03:59.952 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-10T03:03:59.952 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-10T03:03:59.952 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-10T03:03:59.952 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:03:59.953 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.953 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:03:59.953 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:03:59.953 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.953 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.953 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:03:59.953 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:03:59.953 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:03:59.953 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:03:59.953 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:03:59.953 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:03:59.953 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:03:59.953 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:03:59.953 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:03:59.954 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.954 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.954 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.954 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.954 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-10T03:03:59.954 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-10T03:03:59.954 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:03:59.954 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.954 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:03:59.954 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:03:59.954 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.954 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.955 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.955 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:03:59.955 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.955 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.955 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.955 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.955 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-10T03:03:59.955 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-10T03:03:59.955 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:03:59.955 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.955 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:03:59.955 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-10T03:03:59.956 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.956 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.956 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:03:59.956 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:03:59.956 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:03:59.956 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:03:59.956 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:03:59.956 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:03:59.956 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:03:59.956 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:03:59.956 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:03:59.956 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.956 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.956 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.957 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.957 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-10T03:03:59.957 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-10T03:03:59.957 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:03:59.957 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.957 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:03:59.957 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:03:59.957 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.957 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.957 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.957 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:03:59.957 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.957 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.957 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.957 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.958 INFO:teuthology.orchestra.run.smithi052.stdout: "predict_interval": { 2024-06-10T03:03:59.958 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "predict_interval", 2024-06-10T03:03:59.958 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:03:59.958 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.958 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:03:59.958 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "86400", 2024-06-10T03:03:59.958 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.958 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.958 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.958 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:03:59.958 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.958 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.959 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.959 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.959 INFO:teuthology.orchestra.run.smithi052.stdout: "predictor_model": { 2024-06-10T03:03:59.959 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "predictor_model", 2024-06-10T03:03:59.959 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:03:59.959 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.959 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:03:59.959 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "prophetstor", 2024-06-10T03:03:59.959 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.959 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.959 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.959 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:03:59.959 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.959 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.960 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.960 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.960 INFO:teuthology.orchestra.run.smithi052.stdout: "sleep_interval": { 2024-06-10T03:03:59.960 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "sleep_interval", 2024-06-10T03:03:59.960 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:03:59.960 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.960 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:03:59.960 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "600", 2024-06-10T03:03:59.960 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.960 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.960 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.960 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:03:59.960 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.961 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.961 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.961 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:03:59.961 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:03:59.961 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.961 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-10T03:03:59.961 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "influx", 2024-06-10T03:03:59.961 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": false, 2024-06-10T03:03:59.961 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "influxdb python module not found", 2024-06-10T03:03:59.961 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-10T03:03:59.961 INFO:teuthology.orchestra.run.smithi052.stdout: "batch_size": { 2024-06-10T03:03:59.961 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "batch_size", 2024-06-10T03:03:59.961 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-10T03:03:59.961 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.962 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:03:59.962 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "5000", 2024-06-10T03:03:59.962 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.962 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.962 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.962 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-06-10T03:03:59.962 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.962 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.962 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.962 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.962 INFO:teuthology.orchestra.run.smithi052.stdout: "database": { 2024-06-10T03:03:59.962 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "database", 2024-06-10T03:03:59.963 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:03:59.963 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.963 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:03:59.963 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "ceph", 2024-06-10T03:03:59.963 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.963 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.963 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.963 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-06-10T03:03:59.963 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.963 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.963 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.963 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.963 INFO:teuthology.orchestra.run.smithi052.stdout: "hostname": { 2024-06-10T03:03:59.964 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "hostname", 2024-06-10T03:03:59.964 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:03:59.964 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.964 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:03:59.964 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:03:59.964 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.964 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.964 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.964 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "InfluxDB server hostname", 2024-06-10T03:03:59.964 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.964 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.964 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.964 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.965 INFO:teuthology.orchestra.run.smithi052.stdout: "interval": { 2024-06-10T03:03:59.965 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "interval", 2024-06-10T03:03:59.965 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-06-10T03:03:59.965 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.965 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:03:59.965 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "30", 2024-06-10T03:03:59.965 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "5", 2024-06-10T03:03:59.965 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.965 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.965 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-06-10T03:03:59.965 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.965 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.965 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.965 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.966 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-10T03:03:59.966 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-10T03:03:59.966 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:03:59.966 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.966 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:03:59.966 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:03:59.966 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.966 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.966 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:03:59.966 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:03:59.966 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:03:59.966 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:03:59.966 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:03:59.967 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:03:59.967 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:03:59.967 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:03:59.967 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:03:59.967 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.967 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.967 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.967 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.967 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-10T03:03:59.967 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-10T03:03:59.967 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:03:59.967 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.967 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:03:59.968 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:03:59.968 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.968 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.968 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.968 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:03:59.968 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.968 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.968 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.968 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.968 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-10T03:03:59.968 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-10T03:03:59.968 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:03:59.968 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.968 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:03:59.969 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-10T03:03:59.969 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.969 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.969 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:03:59.969 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:03:59.969 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:03:59.969 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:03:59.969 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:03:59.969 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:03:59.969 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:03:59.969 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:03:59.969 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:03:59.969 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.970 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.970 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.970 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.970 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-10T03:03:59.970 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-10T03:03:59.970 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:03:59.970 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.970 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:03:59.970 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:03:59.970 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.970 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.970 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.970 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:03:59.971 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.971 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.971 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.971 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.971 INFO:teuthology.orchestra.run.smithi052.stdout: "password": { 2024-06-10T03:03:59.971 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "password", 2024-06-10T03:03:59.971 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:03:59.971 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.971 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:03:59.971 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:03:59.971 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.971 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.971 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.972 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "password of InfluxDB server user", 2024-06-10T03:03:59.972 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.972 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.972 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.972 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.972 INFO:teuthology.orchestra.run.smithi052.stdout: "port": { 2024-06-10T03:03:59.972 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "port", 2024-06-10T03:03:59.972 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-10T03:03:59.972 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.972 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:03:59.972 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "8086", 2024-06-10T03:03:59.972 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.972 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.973 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.973 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "InfluxDB server port", 2024-06-10T03:03:59.973 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.973 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.973 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.973 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.973 INFO:teuthology.orchestra.run.smithi052.stdout: "ssl": { 2024-06-10T03:03:59.973 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "ssl", 2024-06-10T03:03:59.973 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:03:59.973 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.973 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:03:59.973 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "false", 2024-06-10T03:03:59.973 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.973 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.974 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.974 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-06-10T03:03:59.974 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.974 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.974 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.974 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.974 INFO:teuthology.orchestra.run.smithi052.stdout: "threads": { 2024-06-10T03:03:59.974 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "threads", 2024-06-10T03:03:59.974 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-10T03:03:59.974 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.974 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:03:59.974 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "5", 2024-06-10T03:03:59.975 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "1", 2024-06-10T03:03:59.975 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "32", 2024-06-10T03:03:59.975 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.975 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-06-10T03:03:59.975 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.975 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.975 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.975 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.975 INFO:teuthology.orchestra.run.smithi052.stdout: "username": { 2024-06-10T03:03:59.975 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "username", 2024-06-10T03:03:59.975 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:03:59.975 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.975 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:03:59.975 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:03:59.976 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.976 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.976 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.976 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "username of InfluxDB server user", 2024-06-10T03:03:59.976 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.976 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.976 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.976 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.976 INFO:teuthology.orchestra.run.smithi052.stdout: "verify_ssl": { 2024-06-10T03:03:59.976 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "verify_ssl", 2024-06-10T03:03:59.976 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:03:59.976 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.977 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:03:59.977 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "true", 2024-06-10T03:03:59.977 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.977 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.977 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.977 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-06-10T03:03:59.977 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.977 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.977 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.977 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:03:59.977 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:03:59.977 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.977 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-10T03:03:59.978 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "insights", 2024-06-10T03:03:59.978 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-10T03:03:59.978 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-10T03:03:59.978 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-10T03:03:59.978 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-10T03:03:59.978 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-10T03:03:59.978 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:03:59.978 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.978 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:03:59.978 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:03:59.978 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.978 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.979 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:03:59.979 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:03:59.979 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:03:59.979 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:03:59.979 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:03:59.979 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:03:59.979 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:03:59.979 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:03:59.979 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:03:59.979 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.979 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.979 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.979 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.979 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-10T03:03:59.980 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-10T03:03:59.980 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:03:59.980 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.980 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:03:59.980 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:03:59.980 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.980 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.980 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.980 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:03:59.980 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.980 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.980 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.980 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.981 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-10T03:03:59.981 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-10T03:03:59.981 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:03:59.981 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.981 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:03:59.981 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-10T03:03:59.981 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.981 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.981 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:03:59.981 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:03:59.981 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:03:59.981 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:03:59.982 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:03:59.982 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:03:59.982 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:03:59.982 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:03:59.982 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:03:59.982 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.982 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.982 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.982 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.982 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-10T03:03:59.982 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-10T03:03:59.982 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:03:59.982 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.983 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:03:59.983 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:03:59.983 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.983 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.983 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.983 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:03:59.983 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.983 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.983 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.983 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:03:59.983 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:03:59.983 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.983 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-10T03:03:59.983 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "iostat", 2024-06-10T03:03:59.984 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-10T03:03:59.984 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-10T03:03:59.984 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-10T03:03:59.984 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-10T03:03:59.984 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-10T03:03:59.984 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:03:59.984 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.984 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:03:59.984 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:03:59.984 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.984 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.984 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:03:59.984 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:03:59.985 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:03:59.985 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:03:59.985 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:03:59.985 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:03:59.985 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:03:59.985 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:03:59.985 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:03:59.985 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.985 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.985 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.985 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.985 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-10T03:03:59.985 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-10T03:03:59.986 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:03:59.986 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.986 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:03:59.986 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:03:59.986 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.986 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.986 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.986 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:03:59.986 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.986 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.986 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.986 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.986 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-10T03:03:59.987 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-10T03:03:59.987 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:03:59.987 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.987 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:03:59.987 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-10T03:03:59.987 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.987 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.987 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:03:59.987 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:03:59.987 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:03:59.987 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:03:59.987 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:03:59.987 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:03:59.988 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:03:59.988 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:03:59.988 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:03:59.988 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.988 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.988 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.988 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.988 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-10T03:03:59.988 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-10T03:03:59.988 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:03:59.988 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.988 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:03:59.988 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:03:59.988 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.989 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.989 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.989 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:03:59.989 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.989 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.989 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.989 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:03:59.989 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:03:59.989 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.989 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-10T03:03:59.989 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "k8sevents", 2024-06-10T03:03:59.989 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-10T03:03:59.989 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-10T03:03:59.990 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-10T03:03:59.990 INFO:teuthology.orchestra.run.smithi052.stdout: "ceph_event_retention_days": { 2024-06-10T03:03:59.990 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "ceph_event_retention_days", 2024-06-10T03:03:59.990 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-10T03:03:59.990 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.990 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:03:59.990 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "7", 2024-06-10T03:03:59.990 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.990 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.990 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.990 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Days to hold ceph event information within local cache", 2024-06-10T03:03:59.990 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.990 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.991 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.991 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.991 INFO:teuthology.orchestra.run.smithi052.stdout: "config_check_secs": { 2024-06-10T03:03:59.991 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "config_check_secs", 2024-06-10T03:03:59.991 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-10T03:03:59.991 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.991 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:03:59.991 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "10", 2024-06-10T03:03:59.991 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "10", 2024-06-10T03:03:59.991 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.991 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.991 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-06-10T03:03:59.991 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.992 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.992 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.992 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.992 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-10T03:03:59.992 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-10T03:03:59.992 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:03:59.992 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.992 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:03:59.992 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:03:59.992 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.992 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.992 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:03:59.992 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:03:59.993 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:03:59.993 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:03:59.993 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:03:59.993 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:03:59.993 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:03:59.993 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:03:59.993 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:03:59.993 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.993 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.993 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.993 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.993 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-10T03:03:59.993 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-10T03:03:59.994 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:03:59.994 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.994 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:03:59.994 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:03:59.994 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.994 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.994 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.994 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:03:59.994 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.994 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.994 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.994 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.994 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-10T03:03:59.994 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-10T03:03:59.995 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:03:59.995 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.995 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:03:59.995 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-10T03:03:59.995 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.995 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.995 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:03:59.995 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:03:59.995 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:03:59.995 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:03:59.995 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:03:59.995 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:03:59.995 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:03:59.996 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:03:59.996 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:03:59.996 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.996 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.996 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.996 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.996 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-10T03:03:59.996 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-10T03:03:59.996 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:03:59.996 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.996 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:03:59.996 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:03:59.996 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.997 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.997 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.997 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:03:59.997 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:03:59.997 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.997 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.997 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:03:59.997 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:03:59.997 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.997 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-10T03:03:59.997 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "localpool", 2024-06-10T03:03:59.997 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-10T03:03:59.997 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-10T03:03:59.997 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-10T03:03:59.998 INFO:teuthology.orchestra.run.smithi052.stdout: "failure_domain": { 2024-06-10T03:03:59.998 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "failure_domain", 2024-06-10T03:03:59.998 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:03:59.998 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.998 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:03:59.998 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "host", 2024-06-10T03:03:59.998 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.998 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.998 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:03:59.998 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "failure domain for any created local pool", 2024-06-10T03:03:59.998 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-06-10T03:03:59.998 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:03:59.998 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:03:59.999 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:03:59.999 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-10T03:03:59.999 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-10T03:03:59.999 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:03:59.999 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:03:59.999 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:03:59.999 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:03:59.999 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:03:59.999 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:03:59.999 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:03:59.999 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:03:59.999 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:03:59.999 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:03:59.999 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:00.000 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:00.000 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:00.000 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:00.000 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.000 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.000 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.000 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.000 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.000 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-10T03:04:00.000 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-10T03:04:00.000 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.000 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.000 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.001 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.001 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.001 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.001 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.001 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.001 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.001 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.001 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.001 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.001 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-10T03:04:00.001 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-10T03:04:00.001 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.001 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.001 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.002 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-10T03:04:00.002 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.002 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.002 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:00.002 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:04:00.002 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:04:00.002 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:04:00.002 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:00.002 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:00.002 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:00.002 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:00.002 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.002 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.003 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.003 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.003 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.003 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-10T03:04:00.003 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-10T03:04:00.003 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.003 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.003 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.003 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.003 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.003 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.003 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.003 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.003 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.004 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.004 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.004 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.004 INFO:teuthology.orchestra.run.smithi052.stdout: "min_size": { 2024-06-10T03:04:00.004 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "min_size", 2024-06-10T03:04:00.004 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-10T03:04:00.004 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.004 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.004 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.004 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.004 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.004 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.004 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "default min_size for any created local pool", 2024-06-10T03:04:00.005 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-06-10T03:04:00.005 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.005 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.005 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.005 INFO:teuthology.orchestra.run.smithi052.stdout: "num_rep": { 2024-06-10T03:04:00.005 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "num_rep", 2024-06-10T03:04:00.005 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-10T03:04:00.005 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.005 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.005 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "3", 2024-06-10T03:04:00.005 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.005 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.005 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.006 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "default replica count for any created local pool", 2024-06-10T03:04:00.006 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.006 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.006 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.006 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.006 INFO:teuthology.orchestra.run.smithi052.stdout: "pg_num": { 2024-06-10T03:04:00.006 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "pg_num", 2024-06-10T03:04:00.006 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-10T03:04:00.006 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.006 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.006 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "128", 2024-06-10T03:04:00.006 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.007 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.007 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.007 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "default pg_num for any created local pool", 2024-06-10T03:04:00.007 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.007 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.007 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.007 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.007 INFO:teuthology.orchestra.run.smithi052.stdout: "prefix": { 2024-06-10T03:04:00.007 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "prefix", 2024-06-10T03:04:00.007 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.007 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.007 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.007 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.007 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.008 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.008 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.008 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "name prefix for any created local pool", 2024-06-10T03:04:00.008 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.008 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.008 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.008 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.008 INFO:teuthology.orchestra.run.smithi052.stdout: "subtree": { 2024-06-10T03:04:00.008 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "subtree", 2024-06-10T03:04:00.008 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.008 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.008 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.008 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "rack", 2024-06-10T03:04:00.009 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.009 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.009 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.009 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "CRUSH level for which to create a local pool", 2024-06-10T03:04:00.009 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-06-10T03:04:00.009 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.009 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.009 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:00.009 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:00.009 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.009 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-10T03:04:00.009 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "mds_autoscaler", 2024-06-10T03:04:00.009 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-10T03:04:00.010 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-10T03:04:00.010 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-10T03:04:00.010 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-10T03:04:00.010 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-10T03:04:00.010 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.010 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.010 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.010 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.010 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.010 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.010 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:00.010 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:04:00.010 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:04:00.011 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:04:00.011 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:00.011 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:00.011 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:00.011 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:00.011 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.011 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.011 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.011 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.011 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.011 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-10T03:04:00.011 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-10T03:04:00.011 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.011 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.012 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.012 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.012 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.012 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.012 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.012 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.012 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.012 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.012 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.012 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.012 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-10T03:04:00.012 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-10T03:04:00.012 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.012 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.013 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.013 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-10T03:04:00.013 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.013 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.013 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:00.013 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:04:00.013 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:04:00.013 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:04:00.013 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:00.013 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:00.013 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:00.013 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:00.013 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.013 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.014 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.014 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.014 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.014 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-10T03:04:00.014 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-10T03:04:00.014 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.014 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.014 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.014 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.014 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.014 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.014 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.014 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.015 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.015 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.015 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.015 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:00.015 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:00.015 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.015 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-10T03:04:00.015 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "mirroring", 2024-06-10T03:04:00.015 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-10T03:04:00.015 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-10T03:04:00.015 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-10T03:04:00.015 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-10T03:04:00.015 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-10T03:04:00.015 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.016 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.016 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.016 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.016 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.016 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.016 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:00.016 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:04:00.016 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:04:00.016 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:04:00.016 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:00.016 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:00.016 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:00.016 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:00.017 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.017 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.017 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.017 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.017 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.017 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-10T03:04:00.017 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-10T03:04:00.017 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.017 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.017 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.017 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.017 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.017 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.018 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.018 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.018 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.018 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.018 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.018 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.018 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-10T03:04:00.018 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-10T03:04:00.018 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.018 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.018 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.018 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-10T03:04:00.018 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.019 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.019 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:00.019 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:04:00.019 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:04:00.019 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:04:00.019 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:00.019 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:00.019 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:00.019 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:00.019 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.019 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.019 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.019 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.020 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.020 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-10T03:04:00.020 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-10T03:04:00.020 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.020 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.020 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.020 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.020 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.020 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.020 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.020 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.020 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.021 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.021 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.021 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:00.021 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:00.021 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.021 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-10T03:04:00.021 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "nfs", 2024-06-10T03:04:00.021 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-10T03:04:00.021 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-10T03:04:00.021 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-10T03:04:00.021 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-10T03:04:00.021 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-10T03:04:00.021 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.022 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.022 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.022 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.022 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.022 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.022 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:00.022 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:04:00.022 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:04:00.022 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:04:00.022 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:00.022 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:00.022 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:00.022 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:00.023 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.023 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.023 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.023 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.023 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.023 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-10T03:04:00.023 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-10T03:04:00.023 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.023 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.023 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.023 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.023 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.023 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.024 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.024 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.024 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.024 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.024 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.024 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.024 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-10T03:04:00.024 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-10T03:04:00.024 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.024 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.024 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.024 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-10T03:04:00.024 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.025 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.025 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:00.025 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:04:00.025 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:04:00.025 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:04:00.025 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:00.025 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:00.025 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:00.025 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:00.025 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.025 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.025 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.025 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.025 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.026 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-10T03:04:00.026 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-10T03:04:00.026 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.026 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.026 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.026 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.026 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.026 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.026 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.026 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.026 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.026 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.027 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.027 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:00.027 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:00.027 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.027 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-10T03:04:00.027 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "orchestrator", 2024-06-10T03:04:00.027 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-10T03:04:00.027 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-10T03:04:00.027 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-10T03:04:00.027 INFO:teuthology.orchestra.run.smithi052.stdout: "fail_fs": { 2024-06-10T03:04:00.027 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "fail_fs", 2024-06-10T03:04:00.027 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.027 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.028 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.028 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.028 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.028 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.028 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.028 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-06-10T03:04:00.028 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.028 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.028 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.028 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.028 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-10T03:04:00.028 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-10T03:04:00.028 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.029 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.029 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.029 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.029 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.029 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.029 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:00.029 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:04:00.029 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:04:00.029 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:04:00.029 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:00.029 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:00.029 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:00.029 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:00.029 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.030 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.030 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.030 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.030 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.030 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-10T03:04:00.030 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-10T03:04:00.030 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.030 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.030 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.030 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.030 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.030 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.030 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.031 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.031 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.031 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.031 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.031 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.031 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-10T03:04:00.031 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-10T03:04:00.031 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.031 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.031 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.031 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-10T03:04:00.031 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.031 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.032 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:00.032 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:04:00.032 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:04:00.032 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:04:00.032 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:00.032 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:00.032 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:00.032 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:00.032 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.032 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.032 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.032 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.032 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.033 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-10T03:04:00.033 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-10T03:04:00.033 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.033 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.033 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.033 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.033 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.033 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.033 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.033 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.033 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.033 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.033 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.034 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.034 INFO:teuthology.orchestra.run.smithi052.stdout: "orchestrator": { 2024-06-10T03:04:00.034 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "orchestrator", 2024-06-10T03:04:00.034 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.034 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.034 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.034 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.034 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.034 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.034 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:00.034 INFO:teuthology.orchestra.run.smithi052.stdout: "cephadm", 2024-06-10T03:04:00.034 INFO:teuthology.orchestra.run.smithi052.stdout: "rook", 2024-06-10T03:04:00.035 INFO:teuthology.orchestra.run.smithi052.stdout: "test_orchestrator" 2024-06-10T03:04:00.035 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:00.035 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Orchestrator backend", 2024-06-10T03:04:00.035 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.035 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.035 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.035 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:00.035 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:00.035 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.035 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-10T03:04:00.035 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "osd_perf_query", 2024-06-10T03:04:00.035 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-10T03:04:00.036 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-10T03:04:00.036 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-10T03:04:00.036 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-10T03:04:00.036 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-10T03:04:00.036 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.036 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.036 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.036 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.036 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.036 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.036 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:00.036 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:04:00.037 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:04:00.037 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:04:00.037 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:00.037 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:00.037 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:00.037 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:00.037 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.037 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.037 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.037 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.037 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.037 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-10T03:04:00.037 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-10T03:04:00.038 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.038 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.038 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.038 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.038 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.038 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.038 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.038 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.038 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.038 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.038 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.038 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.038 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-10T03:04:00.039 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-10T03:04:00.039 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.039 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.039 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.039 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-10T03:04:00.039 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.039 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.039 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:00.039 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:04:00.039 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:04:00.039 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:04:00.039 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:00.040 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:00.040 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:00.040 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:00.040 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.040 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.040 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.040 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.040 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.040 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-10T03:04:00.040 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-10T03:04:00.040 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.040 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.040 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.041 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.041 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.041 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.041 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.041 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.041 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.041 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.041 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.041 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:00.041 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:00.041 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.041 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-10T03:04:00.041 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "osd_support", 2024-06-10T03:04:00.042 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-10T03:04:00.042 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-10T03:04:00.042 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-10T03:04:00.042 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-10T03:04:00.042 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-10T03:04:00.042 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.042 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.042 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.042 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.042 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.042 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.042 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:00.043 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:04:00.043 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:04:00.043 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:04:00.043 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:00.043 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:00.043 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:00.043 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:00.043 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.043 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.043 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.043 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.043 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.043 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-10T03:04:00.044 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-10T03:04:00.044 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.044 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.044 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.044 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.044 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.044 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.044 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.044 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.044 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.044 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.044 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.044 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.045 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-10T03:04:00.045 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-10T03:04:00.045 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.045 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.045 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.045 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-10T03:04:00.045 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.045 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.045 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:00.045 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:04:00.045 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:04:00.045 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:04:00.045 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:00.046 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:00.046 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:00.046 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:00.046 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.046 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.046 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.046 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.046 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.046 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-10T03:04:00.046 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-10T03:04:00.046 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.046 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.047 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.047 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.047 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.047 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.047 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.047 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.047 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.047 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.047 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.047 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:00.047 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:00.047 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.047 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-10T03:04:00.048 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "pg_autoscaler", 2024-06-10T03:04:00.048 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-10T03:04:00.048 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-10T03:04:00.048 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-10T03:04:00.048 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-10T03:04:00.048 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-10T03:04:00.048 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.048 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.048 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.048 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.048 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.048 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.048 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:00.048 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:04:00.049 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:04:00.049 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:04:00.049 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:00.049 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:00.049 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:00.049 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:00.049 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.049 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.049 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.049 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.049 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.049 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-10T03:04:00.049 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-10T03:04:00.049 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.050 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.050 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.050 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.050 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.050 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.050 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.050 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.050 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.050 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.050 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.050 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.050 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-10T03:04:00.050 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-10T03:04:00.051 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.051 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.051 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.051 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-10T03:04:00.051 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.051 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.051 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:00.051 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:04:00.051 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:04:00.051 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:04:00.051 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:00.051 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:00.052 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:00.052 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:00.052 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.052 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.052 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.052 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.052 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.052 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-10T03:04:00.052 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-10T03:04:00.052 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.052 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.052 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.052 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.053 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.053 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.053 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.053 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.053 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.053 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.053 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.053 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.053 INFO:teuthology.orchestra.run.smithi052.stdout: "sleep_interval": { 2024-06-10T03:04:00.053 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "sleep_interval", 2024-06-10T03:04:00.053 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-06-10T03:04:00.053 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.054 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.054 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "60", 2024-06-10T03:04:00.054 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.054 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.054 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.054 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.054 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.054 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.054 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.054 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.054 INFO:teuthology.orchestra.run.smithi052.stdout: "threshold": { 2024-06-10T03:04:00.054 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "threshold", 2024-06-10T03:04:00.054 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "float", 2024-06-10T03:04:00.055 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.055 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.055 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "3.0", 2024-06-10T03:04:00.055 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "1.0", 2024-06-10T03:04:00.055 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.055 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.055 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "scaling threshold", 2024-06-10T03:04:00.055 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-06-10T03:04:00.055 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.055 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.055 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:00.055 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:00.056 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.056 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-10T03:04:00.056 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "progress", 2024-06-10T03:04:00.056 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-10T03:04:00.056 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-10T03:04:00.056 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-10T03:04:00.056 INFO:teuthology.orchestra.run.smithi052.stdout: "allow_pg_recovery_event": { 2024-06-10T03:04:00.056 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "allow_pg_recovery_event", 2024-06-10T03:04:00.056 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.056 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.056 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.056 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.056 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.057 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.057 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.057 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "allow the module to show pg recovery progress", 2024-06-10T03:04:00.057 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.057 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.057 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.057 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.057 INFO:teuthology.orchestra.run.smithi052.stdout: "enabled": { 2024-06-10T03:04:00.057 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "enabled", 2024-06-10T03:04:00.057 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.057 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.057 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.057 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-10T03:04:00.057 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.058 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.058 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.058 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.058 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.058 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.058 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.058 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.058 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-10T03:04:00.058 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-10T03:04:00.058 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.058 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.058 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.059 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.059 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.059 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.059 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:00.059 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:04:00.059 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:04:00.059 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:04:00.059 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:00.059 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:00.059 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:00.059 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:00.059 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.059 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.059 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.060 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.060 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.060 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-10T03:04:00.060 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-10T03:04:00.060 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.060 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.060 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.060 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.060 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.060 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.060 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.060 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.060 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.061 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.061 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.061 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.061 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-10T03:04:00.061 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-10T03:04:00.061 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.061 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.061 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.061 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-10T03:04:00.061 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.061 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.061 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:00.061 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:04:00.062 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:04:00.062 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:04:00.062 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:00.062 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:00.062 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:00.062 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:00.062 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.062 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.062 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.062 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.062 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.062 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-10T03:04:00.062 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-10T03:04:00.063 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.063 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.063 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.063 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.063 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.063 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.063 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.063 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.063 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.063 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.063 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.063 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.064 INFO:teuthology.orchestra.run.smithi052.stdout: "max_completed_events": { 2024-06-10T03:04:00.064 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "max_completed_events", 2024-06-10T03:04:00.064 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-10T03:04:00.064 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.064 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.064 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "50", 2024-06-10T03:04:00.064 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.064 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.064 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.064 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "number of past completed events to remember", 2024-06-10T03:04:00.064 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.064 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.064 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.064 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.065 INFO:teuthology.orchestra.run.smithi052.stdout: "sleep_interval": { 2024-06-10T03:04:00.065 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "sleep_interval", 2024-06-10T03:04:00.065 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-06-10T03:04:00.065 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.065 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.065 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "5", 2024-06-10T03:04:00.065 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.065 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.065 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.065 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "how long the module is going to sleep", 2024-06-10T03:04:00.065 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.065 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.066 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.066 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:00.066 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:00.066 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.066 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-10T03:04:00.066 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "prometheus", 2024-06-10T03:04:00.066 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-10T03:04:00.066 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-10T03:04:00.066 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-10T03:04:00.066 INFO:teuthology.orchestra.run.smithi052.stdout: "cache": { 2024-06-10T03:04:00.066 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "cache", 2024-06-10T03:04:00.066 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.066 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.067 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.067 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-10T03:04:00.067 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.067 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.067 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.067 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.067 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.067 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.067 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.067 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.067 INFO:teuthology.orchestra.run.smithi052.stdout: "exclude_perf_counters": { 2024-06-10T03:04:00.067 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "exclude_perf_counters", 2024-06-10T03:04:00.067 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.068 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.068 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.068 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-10T03:04:00.068 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.068 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.068 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.068 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-06-10T03:04:00.068 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-06-10T03:04:00.068 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.068 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.068 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.068 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-10T03:04:00.068 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-10T03:04:00.069 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.069 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.069 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.069 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.069 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.069 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.069 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:00.069 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:04:00.069 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:04:00.069 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:04:00.069 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:00.069 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:00.069 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:00.069 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:00.070 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.070 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.070 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.070 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.070 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.070 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-10T03:04:00.070 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-10T03:04:00.070 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.070 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.070 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.070 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.070 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.070 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.071 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.071 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.071 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.071 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.071 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.071 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.071 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-10T03:04:00.071 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-10T03:04:00.071 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.071 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.071 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.071 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-10T03:04:00.071 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.072 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.072 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:00.072 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:04:00.072 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:04:00.072 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:04:00.072 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:00.072 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:00.072 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:00.072 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:00.072 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.072 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.072 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.072 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.073 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.073 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-10T03:04:00.073 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-10T03:04:00.073 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.073 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.073 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.073 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.073 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.073 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.073 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.073 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.073 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.073 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.074 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.074 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.074 INFO:teuthology.orchestra.run.smithi052.stdout: "rbd_stats_pools": { 2024-06-10T03:04:00.074 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rbd_stats_pools", 2024-06-10T03:04:00.074 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.074 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.074 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.074 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.074 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.074 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.074 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.074 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.074 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.074 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.075 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.075 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.075 INFO:teuthology.orchestra.run.smithi052.stdout: "rbd_stats_pools_refresh_interval": { 2024-06-10T03:04:00.075 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-06-10T03:04:00.075 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-10T03:04:00.075 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.075 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.075 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "300", 2024-06-10T03:04:00.075 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.075 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.075 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.075 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.075 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.076 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.076 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.076 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.076 INFO:teuthology.orchestra.run.smithi052.stdout: "scrape_interval": { 2024-06-10T03:04:00.076 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "scrape_interval", 2024-06-10T03:04:00.076 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "float", 2024-06-10T03:04:00.076 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.076 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.076 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "15.0", 2024-06-10T03:04:00.076 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.076 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.076 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.076 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.077 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.077 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.077 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.077 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.077 INFO:teuthology.orchestra.run.smithi052.stdout: "server_addr": { 2024-06-10T03:04:00.077 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "server_addr", 2024-06-10T03:04:00.077 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.077 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.077 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.077 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "::", 2024-06-10T03:04:00.077 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.077 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.077 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.077 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-06-10T03:04:00.078 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.078 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.078 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.078 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.078 INFO:teuthology.orchestra.run.smithi052.stdout: "server_port": { 2024-06-10T03:04:00.078 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "server_port", 2024-06-10T03:04:00.078 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-10T03:04:00.078 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.078 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.078 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "9283", 2024-06-10T03:04:00.078 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.078 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.078 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.079 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-06-10T03:04:00.079 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.079 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.079 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.079 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.079 INFO:teuthology.orchestra.run.smithi052.stdout: "stale_cache_strategy": { 2024-06-10T03:04:00.079 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "stale_cache_strategy", 2024-06-10T03:04:00.079 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.079 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.079 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.079 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "log", 2024-06-10T03:04:00.079 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.080 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.080 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.080 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.080 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.080 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.080 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.080 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.080 INFO:teuthology.orchestra.run.smithi052.stdout: "standby_behaviour": { 2024-06-10T03:04:00.080 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "standby_behaviour", 2024-06-10T03:04:00.080 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.080 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.080 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.080 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "default", 2024-06-10T03:04:00.081 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.081 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.081 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:00.081 INFO:teuthology.orchestra.run.smithi052.stdout: "default", 2024-06-10T03:04:00.081 INFO:teuthology.orchestra.run.smithi052.stdout: "error" 2024-06-10T03:04:00.081 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:00.081 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.081 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.081 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.081 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.081 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.081 INFO:teuthology.orchestra.run.smithi052.stdout: "standby_error_status_code": { 2024-06-10T03:04:00.081 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "standby_error_status_code", 2024-06-10T03:04:00.081 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-10T03:04:00.082 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.082 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.082 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "500", 2024-06-10T03:04:00.082 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "400", 2024-06-10T03:04:00.082 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "599", 2024-06-10T03:04:00.082 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.082 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.082 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.082 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.082 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.082 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:00.082 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:00.082 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.083 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-10T03:04:00.083 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rbd_support", 2024-06-10T03:04:00.083 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-10T03:04:00.083 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-10T03:04:00.083 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-10T03:04:00.083 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-10T03:04:00.083 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-10T03:04:00.083 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.083 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.083 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.083 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.083 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.083 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.083 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:00.084 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:04:00.084 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:04:00.084 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:04:00.084 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:00.084 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:00.084 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:00.084 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:00.084 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.084 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.084 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.084 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.084 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.084 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-10T03:04:00.085 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-10T03:04:00.085 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.085 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.085 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.085 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.085 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.085 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.085 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.085 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.085 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.085 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.085 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.085 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.085 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-10T03:04:00.086 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-10T03:04:00.086 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.086 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.086 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.086 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-10T03:04:00.086 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.086 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.086 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:00.086 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:04:00.086 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:04:00.086 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:04:00.086 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:00.086 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:00.087 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:00.087 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:00.087 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.087 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.087 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.087 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.087 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.087 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-10T03:04:00.087 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-10T03:04:00.087 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.087 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.087 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.087 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.087 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.088 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.088 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.088 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.088 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.088 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.088 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.088 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.088 INFO:teuthology.orchestra.run.smithi052.stdout: "max_concurrent_snap_create": { 2024-06-10T03:04:00.088 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "max_concurrent_snap_create", 2024-06-10T03:04:00.088 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-10T03:04:00.088 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.088 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.088 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "10", 2024-06-10T03:04:00.089 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.089 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.089 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.089 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.089 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.089 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.089 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.089 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.089 INFO:teuthology.orchestra.run.smithi052.stdout: "mirror_snapshot_schedule": { 2024-06-10T03:04:00.089 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "mirror_snapshot_schedule", 2024-06-10T03:04:00.089 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.089 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.089 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.089 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.090 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.090 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.090 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.090 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.090 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.090 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.090 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.090 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.090 INFO:teuthology.orchestra.run.smithi052.stdout: "trash_purge_schedule": { 2024-06-10T03:04:00.090 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "trash_purge_schedule", 2024-06-10T03:04:00.090 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.090 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.091 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.091 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.091 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.091 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.091 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.091 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.091 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.091 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.091 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.091 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:00.091 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:00.091 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.091 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-10T03:04:00.092 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "restful", 2024-06-10T03:04:00.092 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-10T03:04:00.092 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-10T03:04:00.092 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-10T03:04:00.092 INFO:teuthology.orchestra.run.smithi052.stdout: "enable_auth": { 2024-06-10T03:04:00.092 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "enable_auth", 2024-06-10T03:04:00.092 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.092 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.092 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.092 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-10T03:04:00.092 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.092 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.092 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.092 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.093 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.093 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.093 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.093 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.093 INFO:teuthology.orchestra.run.smithi052.stdout: "key_file": { 2024-06-10T03:04:00.093 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "key_file", 2024-06-10T03:04:00.093 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.093 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.093 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.093 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.093 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.093 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.093 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.093 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.094 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.094 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.094 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.094 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.094 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-10T03:04:00.094 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-10T03:04:00.094 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.094 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.095 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.095 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.095 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.095 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.095 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:00.095 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:04:00.095 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:04:00.095 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:04:00.095 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:00.095 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:00.095 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:00.095 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:00.095 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.096 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.096 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.096 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.096 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.096 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-10T03:04:00.096 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-10T03:04:00.096 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.096 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.096 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.096 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.096 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.096 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.097 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.097 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.097 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.097 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.097 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.097 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.097 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-10T03:04:00.097 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-10T03:04:00.097 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.097 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.097 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.097 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-10T03:04:00.097 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.098 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.098 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:00.098 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:04:00.098 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:04:00.098 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:04:00.098 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:00.098 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:00.098 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:00.098 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:00.098 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.098 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.098 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.099 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.099 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.099 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-10T03:04:00.099 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-10T03:04:00.099 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.099 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.099 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.099 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.099 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.099 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.099 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.099 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.099 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.100 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.100 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.100 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.100 INFO:teuthology.orchestra.run.smithi052.stdout: "server_addr": { 2024-06-10T03:04:00.100 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "server_addr", 2024-06-10T03:04:00.100 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.100 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.100 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.100 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.100 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.100 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.100 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.100 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.101 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.101 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.101 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.101 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.101 INFO:teuthology.orchestra.run.smithi052.stdout: "server_port": { 2024-06-10T03:04:00.101 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "server_port", 2024-06-10T03:04:00.101 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.101 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.101 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.101 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.101 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.101 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.101 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.102 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.102 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.102 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.102 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.102 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:00.102 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:00.102 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.102 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-10T03:04:00.102 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rgw", 2024-06-10T03:04:00.102 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-10T03:04:00.102 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-10T03:04:00.102 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-10T03:04:00.102 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-10T03:04:00.103 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-10T03:04:00.103 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.103 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.103 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.103 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.103 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.103 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.103 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:00.103 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:04:00.103 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:04:00.103 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:04:00.103 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:00.103 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:00.103 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:00.104 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:00.104 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.104 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.104 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.104 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.104 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.104 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-10T03:04:00.104 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-10T03:04:00.104 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.104 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.104 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.104 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.104 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.105 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.105 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.105 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.105 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.105 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.105 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.105 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.105 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-10T03:04:00.105 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-10T03:04:00.105 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.105 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.105 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.105 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-10T03:04:00.106 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.106 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.106 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:00.106 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:04:00.106 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:04:00.106 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:04:00.106 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:00.106 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:00.106 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:00.106 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:00.106 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.106 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.106 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.107 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.107 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.107 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-10T03:04:00.107 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-10T03:04:00.107 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.107 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.107 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.107 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.107 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.107 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.107 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.107 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.107 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.108 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.108 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.108 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:00.108 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:00.108 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.108 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-10T03:04:00.108 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rook", 2024-06-10T03:04:00.108 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-10T03:04:00.108 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-10T03:04:00.108 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-10T03:04:00.108 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-10T03:04:00.108 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-10T03:04:00.108 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.108 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.109 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.109 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.109 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.109 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.109 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:00.109 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:04:00.109 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:04:00.109 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:04:00.109 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:00.109 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:00.109 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:00.109 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:00.110 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.110 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.110 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.110 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.110 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.110 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-10T03:04:00.110 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-10T03:04:00.110 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.110 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.110 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.110 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.110 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.111 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.111 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.111 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.111 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.111 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.111 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.111 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.111 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-10T03:04:00.111 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-10T03:04:00.111 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.111 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.111 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.111 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-10T03:04:00.111 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.112 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.112 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:00.112 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:04:00.112 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:04:00.112 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:04:00.112 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:00.112 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:00.112 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:00.112 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:00.112 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.112 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.112 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.112 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.113 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.113 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-10T03:04:00.113 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-10T03:04:00.113 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.113 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.113 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.113 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.113 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.113 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.113 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.113 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.113 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.113 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.113 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.114 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.114 INFO:teuthology.orchestra.run.smithi052.stdout: "storage_class": { 2024-06-10T03:04:00.114 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "storage_class", 2024-06-10T03:04:00.114 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.114 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.114 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.114 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "local", 2024-06-10T03:04:00.114 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.114 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.114 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.114 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-06-10T03:04:00.114 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.118 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.118 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.118 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:00.118 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:00.118 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.118 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-10T03:04:00.118 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "selftest", 2024-06-10T03:04:00.118 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-10T03:04:00.118 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-10T03:04:00.118 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-10T03:04:00.118 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-10T03:04:00.118 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-10T03:04:00.119 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.119 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.119 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.119 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.119 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.119 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.119 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:00.119 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:04:00.119 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:04:00.119 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:04:00.119 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:00.119 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:00.119 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:00.120 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:00.120 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.120 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.120 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.120 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.120 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.120 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-10T03:04:00.120 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-10T03:04:00.120 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.120 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.120 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.120 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.120 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.121 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.121 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.121 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.121 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.121 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.121 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.121 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.121 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-10T03:04:00.121 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-10T03:04:00.121 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.121 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.121 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.121 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-10T03:04:00.122 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.122 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.122 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:00.122 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:04:00.122 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:04:00.122 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:04:00.122 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:00.122 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:00.122 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:00.122 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:00.122 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.122 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.122 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.123 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.123 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.123 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-10T03:04:00.123 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-10T03:04:00.123 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.123 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.123 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.123 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.123 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.123 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.123 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.123 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.123 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.123 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.124 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.124 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.124 INFO:teuthology.orchestra.run.smithi052.stdout: "roption1": { 2024-06-10T03:04:00.124 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "roption1", 2024-06-10T03:04:00.124 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.124 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.124 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.124 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.124 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.124 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.124 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.124 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.124 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.125 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.125 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.125 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.125 INFO:teuthology.orchestra.run.smithi052.stdout: "roption2": { 2024-06-10T03:04:00.125 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "roption2", 2024-06-10T03:04:00.125 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.125 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.125 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.125 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "xyz", 2024-06-10T03:04:00.125 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.125 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.125 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.125 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.126 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.126 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.126 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.126 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.126 INFO:teuthology.orchestra.run.smithi052.stdout: "rwoption1": { 2024-06-10T03:04:00.126 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rwoption1", 2024-06-10T03:04:00.126 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.126 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.126 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.126 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.126 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.126 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.126 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.127 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.127 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.127 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.127 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.127 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.127 INFO:teuthology.orchestra.run.smithi052.stdout: "rwoption2": { 2024-06-10T03:04:00.127 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rwoption2", 2024-06-10T03:04:00.127 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-10T03:04:00.127 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.127 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.127 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.128 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.128 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.128 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.128 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.128 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.128 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.128 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.128 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.128 INFO:teuthology.orchestra.run.smithi052.stdout: "rwoption3": { 2024-06-10T03:04:00.128 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rwoption3", 2024-06-10T03:04:00.128 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "float", 2024-06-10T03:04:00.128 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.128 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.129 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.129 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.129 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.129 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.129 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.129 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.129 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.129 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.129 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.129 INFO:teuthology.orchestra.run.smithi052.stdout: "rwoption4": { 2024-06-10T03:04:00.129 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rwoption4", 2024-06-10T03:04:00.129 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.129 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.130 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.130 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.130 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.130 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.130 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.130 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.130 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.130 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.130 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.130 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.130 INFO:teuthology.orchestra.run.smithi052.stdout: "rwoption5": { 2024-06-10T03:04:00.130 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rwoption5", 2024-06-10T03:04:00.131 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.131 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.131 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.131 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.131 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.131 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.131 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.131 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.131 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.131 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.131 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.131 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.131 INFO:teuthology.orchestra.run.smithi052.stdout: "rwoption6": { 2024-06-10T03:04:00.131 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rwoption6", 2024-06-10T03:04:00.132 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.132 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.132 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.132 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-10T03:04:00.132 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.132 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.132 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.132 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.132 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.132 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.132 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.132 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.132 INFO:teuthology.orchestra.run.smithi052.stdout: "rwoption7": { 2024-06-10T03:04:00.132 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rwoption7", 2024-06-10T03:04:00.133 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-10T03:04:00.133 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.133 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.133 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.133 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "1", 2024-06-10T03:04:00.133 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "42", 2024-06-10T03:04:00.133 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.133 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.133 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.133 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.133 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.133 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.133 INFO:teuthology.orchestra.run.smithi052.stdout: "testkey": { 2024-06-10T03:04:00.134 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "testkey", 2024-06-10T03:04:00.134 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.134 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.134 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.134 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.134 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.134 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.134 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.134 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.134 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.134 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.134 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.134 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.135 INFO:teuthology.orchestra.run.smithi052.stdout: "testlkey": { 2024-06-10T03:04:00.135 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "testlkey", 2024-06-10T03:04:00.135 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.135 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.135 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.135 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.135 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.135 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.135 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.135 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.135 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.135 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.135 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.136 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.136 INFO:teuthology.orchestra.run.smithi052.stdout: "testnewline": { 2024-06-10T03:04:00.136 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "testnewline", 2024-06-10T03:04:00.136 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.136 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.136 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.136 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.136 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.136 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.136 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.136 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.136 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.136 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.136 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.136 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:00.137 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:00.137 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.137 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-10T03:04:00.137 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "snap_schedule", 2024-06-10T03:04:00.137 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-10T03:04:00.137 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-10T03:04:00.137 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-10T03:04:00.137 INFO:teuthology.orchestra.run.smithi052.stdout: "allow_m_granularity": { 2024-06-10T03:04:00.137 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "allow_m_granularity", 2024-06-10T03:04:00.137 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.137 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.137 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.137 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.137 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.137 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.138 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.138 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "allow minute scheduled snapshots", 2024-06-10T03:04:00.138 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.138 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.138 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.138 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.138 INFO:teuthology.orchestra.run.smithi052.stdout: "dump_on_update": { 2024-06-10T03:04:00.138 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "dump_on_update", 2024-06-10T03:04:00.138 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.138 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.138 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.138 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.138 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.139 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.139 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.139 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "dump database to debug log on update", 2024-06-10T03:04:00.139 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.139 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.139 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.139 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.139 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-10T03:04:00.139 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-10T03:04:00.139 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.139 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.139 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.139 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.139 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.139 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.140 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:00.140 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:04:00.140 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:04:00.140 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:04:00.140 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:00.140 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:00.140 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:00.140 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:00.140 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.140 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.140 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.140 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.140 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.140 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-10T03:04:00.141 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-10T03:04:00.141 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.141 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.141 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.141 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.141 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.141 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.141 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.141 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.141 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.141 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.141 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.141 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.141 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-10T03:04:00.141 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-10T03:04:00.142 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.142 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.142 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.142 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-10T03:04:00.142 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.142 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.142 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:00.142 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:04:00.142 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:04:00.142 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:04:00.142 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:00.142 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:00.142 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:00.143 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:00.143 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.143 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.143 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.143 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.143 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.143 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-10T03:04:00.143 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-10T03:04:00.143 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.143 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.143 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.143 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.143 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.143 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.144 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.144 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.144 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.144 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.144 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.144 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:00.144 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:00.144 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.144 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-10T03:04:00.144 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "stats", 2024-06-10T03:04:00.144 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-10T03:04:00.144 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-10T03:04:00.144 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-10T03:04:00.144 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-10T03:04:00.144 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-10T03:04:00.145 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.145 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.145 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.145 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.145 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.145 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.145 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:00.145 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:04:00.145 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:04:00.145 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:04:00.145 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:00.145 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:00.145 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:00.145 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:00.145 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.146 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.146 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.146 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.146 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.146 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-10T03:04:00.146 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-10T03:04:00.146 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.146 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.146 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.146 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.146 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.146 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.146 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.147 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.147 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.147 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.147 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.147 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.147 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-10T03:04:00.147 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-10T03:04:00.147 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.147 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.147 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.147 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-10T03:04:00.147 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.147 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.148 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:00.148 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:04:00.148 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:04:00.148 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:04:00.148 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:00.148 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:00.148 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:00.148 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:00.148 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.148 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.148 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.148 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.148 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.149 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-10T03:04:00.149 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-10T03:04:00.149 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.149 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.149 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.149 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.149 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.149 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.149 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.149 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.149 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.149 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.149 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.149 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:00.150 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:00.150 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.150 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-10T03:04:00.150 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "status", 2024-06-10T03:04:00.150 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-10T03:04:00.150 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-10T03:04:00.150 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-10T03:04:00.150 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-10T03:04:00.150 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-10T03:04:00.150 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.150 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.150 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.150 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.150 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.151 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.151 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:00.151 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:04:00.151 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:04:00.151 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:04:00.151 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:00.151 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:00.151 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:00.151 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:00.151 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.151 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.151 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.151 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.151 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.152 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-10T03:04:00.152 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-10T03:04:00.152 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.152 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.152 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.152 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.152 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.152 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.152 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.152 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.152 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.152 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.152 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.152 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.153 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-10T03:04:00.153 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-10T03:04:00.153 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.153 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.153 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.153 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-10T03:04:00.153 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.153 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.153 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:00.153 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:04:00.153 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:04:00.153 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:04:00.153 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:00.153 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:00.154 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:00.154 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:00.154 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.154 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.154 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.154 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.154 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.154 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-10T03:04:00.154 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-10T03:04:00.154 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.154 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.154 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.154 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.154 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.155 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.155 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.155 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.155 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.155 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.155 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.155 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:00.155 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:00.155 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.155 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-10T03:04:00.155 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "telegraf", 2024-06-10T03:04:00.155 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-10T03:04:00.155 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-10T03:04:00.155 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-10T03:04:00.155 INFO:teuthology.orchestra.run.smithi052.stdout: "address": { 2024-06-10T03:04:00.156 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "address", 2024-06-10T03:04:00.156 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.156 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.156 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.156 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-06-10T03:04:00.156 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.156 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.156 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.156 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.156 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.156 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.156 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.156 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.157 INFO:teuthology.orchestra.run.smithi052.stdout: "interval": { 2024-06-10T03:04:00.157 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "interval", 2024-06-10T03:04:00.157 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-06-10T03:04:00.157 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.157 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.157 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "15", 2024-06-10T03:04:00.157 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.157 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.157 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.157 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.157 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.157 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.157 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.157 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.158 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-10T03:04:00.158 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-10T03:04:00.158 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.158 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.158 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.158 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.158 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.158 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.158 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:00.158 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:04:00.158 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:04:00.158 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:04:00.158 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:00.159 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:00.159 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:00.159 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:00.159 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.159 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.159 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.159 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.159 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.159 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-10T03:04:00.159 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-10T03:04:00.159 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.159 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.159 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.160 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.160 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.160 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.160 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.160 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.160 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.160 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.160 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.160 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.160 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-10T03:04:00.160 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-10T03:04:00.160 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.160 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.161 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.161 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-10T03:04:00.161 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.161 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.161 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:00.161 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:04:00.161 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:04:00.161 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:04:00.161 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:00.161 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:00.161 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:00.161 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:00.161 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.162 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.162 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.162 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.162 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.162 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-10T03:04:00.162 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-10T03:04:00.162 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.162 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.162 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.162 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.162 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.162 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.162 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.163 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.163 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.163 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.163 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.163 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:00.163 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:00.163 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.163 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-10T03:04:00.163 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "telemetry", 2024-06-10T03:04:00.163 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-10T03:04:00.163 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-10T03:04:00.163 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-10T03:04:00.163 INFO:teuthology.orchestra.run.smithi052.stdout: "channel_basic": { 2024-06-10T03:04:00.164 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "channel_basic", 2024-06-10T03:04:00.164 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.164 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.164 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.164 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-10T03:04:00.164 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.164 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.164 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.164 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Share basic cluster information (size, version)", 2024-06-10T03:04:00.164 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.164 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.164 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.164 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.165 INFO:teuthology.orchestra.run.smithi052.stdout: "channel_crash": { 2024-06-10T03:04:00.165 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "channel_crash", 2024-06-10T03:04:00.165 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.165 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.165 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.165 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-10T03:04:00.165 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.165 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.165 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.165 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-06-10T03:04:00.165 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.165 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.165 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.166 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.166 INFO:teuthology.orchestra.run.smithi052.stdout: "channel_device": { 2024-06-10T03:04:00.166 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "channel_device", 2024-06-10T03:04:00.166 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.166 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.166 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.166 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-10T03:04:00.166 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.166 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.166 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.166 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-06-10T03:04:00.166 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.166 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.167 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.167 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.167 INFO:teuthology.orchestra.run.smithi052.stdout: "channel_ident": { 2024-06-10T03:04:00.167 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "channel_ident", 2024-06-10T03:04:00.167 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.167 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.167 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.167 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.167 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.167 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.167 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.167 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-06-10T03:04:00.167 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.168 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.168 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.168 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.168 INFO:teuthology.orchestra.run.smithi052.stdout: "channel_perf": { 2024-06-10T03:04:00.168 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "channel_perf", 2024-06-10T03:04:00.168 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.168 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.168 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.168 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.168 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.168 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.168 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.168 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Share various performance metrics of a cluster", 2024-06-10T03:04:00.169 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.169 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.169 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.169 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.169 INFO:teuthology.orchestra.run.smithi052.stdout: "contact": { 2024-06-10T03:04:00.169 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "contact", 2024-06-10T03:04:00.169 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.169 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.169 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.169 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.169 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.169 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.169 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.169 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.170 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.170 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.170 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.170 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.170 INFO:teuthology.orchestra.run.smithi052.stdout: "description": { 2024-06-10T03:04:00.170 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "description", 2024-06-10T03:04:00.170 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.170 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.170 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.170 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.170 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.170 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.171 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.171 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.171 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.171 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.171 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.171 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.171 INFO:teuthology.orchestra.run.smithi052.stdout: "device_url": { 2024-06-10T03:04:00.171 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "device_url", 2024-06-10T03:04:00.171 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.171 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.171 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.171 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-06-10T03:04:00.171 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.172 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.172 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.172 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.172 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.172 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.172 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.172 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.172 INFO:teuthology.orchestra.run.smithi052.stdout: "enabled": { 2024-06-10T03:04:00.172 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "enabled", 2024-06-10T03:04:00.172 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.172 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.172 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.172 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.172 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.172 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.173 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.173 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.173 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.173 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.173 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.173 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.173 INFO:teuthology.orchestra.run.smithi052.stdout: "interval": { 2024-06-10T03:04:00.173 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "interval", 2024-06-10T03:04:00.173 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-10T03:04:00.173 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.173 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.173 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "24", 2024-06-10T03:04:00.173 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "8", 2024-06-10T03:04:00.174 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.174 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.174 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.174 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.174 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.174 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.174 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.174 INFO:teuthology.orchestra.run.smithi052.stdout: "last_opt_revision": { 2024-06-10T03:04:00.174 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "last_opt_revision", 2024-06-10T03:04:00.174 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-10T03:04:00.174 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.174 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.174 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "1", 2024-06-10T03:04:00.175 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.175 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.175 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.175 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.175 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.175 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.175 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.175 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.175 INFO:teuthology.orchestra.run.smithi052.stdout: "leaderboard": { 2024-06-10T03:04:00.175 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "leaderboard", 2024-06-10T03:04:00.175 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.175 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.175 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.175 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.176 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.176 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.176 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.176 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.176 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.176 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.176 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.176 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.176 INFO:teuthology.orchestra.run.smithi052.stdout: "leaderboard_description": { 2024-06-10T03:04:00.176 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "leaderboard_description", 2024-06-10T03:04:00.176 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.176 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.176 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.176 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.177 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.177 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.177 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.177 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.177 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.177 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.177 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.177 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.177 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-10T03:04:00.177 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-10T03:04:00.177 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.177 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.177 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.177 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.178 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.178 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.178 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:00.178 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:04:00.178 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:04:00.178 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:04:00.178 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:00.178 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:00.178 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:00.178 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:00.178 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.178 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.178 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.178 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.179 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.179 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-10T03:04:00.179 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-10T03:04:00.179 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.179 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.179 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.179 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.179 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.179 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.179 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.179 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.179 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.179 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.179 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.180 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.180 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-10T03:04:00.180 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-10T03:04:00.180 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.180 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.180 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.180 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-10T03:04:00.180 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.180 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.180 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:00.180 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:04:00.180 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:04:00.180 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:04:00.180 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:00.181 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:00.181 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:00.181 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:00.181 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.181 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.181 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.181 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.181 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.181 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-10T03:04:00.181 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-10T03:04:00.181 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.181 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.181 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.181 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.182 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.182 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.182 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.182 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.182 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.182 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.182 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.182 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.182 INFO:teuthology.orchestra.run.smithi052.stdout: "organization": { 2024-06-10T03:04:00.182 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "organization", 2024-06-10T03:04:00.182 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.182 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.182 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.182 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.183 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.183 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.183 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.183 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.183 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.183 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.183 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.183 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.183 INFO:teuthology.orchestra.run.smithi052.stdout: "proxy": { 2024-06-10T03:04:00.183 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "proxy", 2024-06-10T03:04:00.183 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.183 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.183 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.183 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.183 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.184 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.184 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.184 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.184 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.184 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.184 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.184 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.184 INFO:teuthology.orchestra.run.smithi052.stdout: "url": { 2024-06-10T03:04:00.184 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "url", 2024-06-10T03:04:00.184 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.184 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.184 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.184 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-06-10T03:04:00.184 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.184 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.185 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.185 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.185 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.185 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.185 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.185 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:00.185 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:00.185 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.185 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-10T03:04:00.185 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "test_orchestrator", 2024-06-10T03:04:00.185 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-10T03:04:00.185 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-10T03:04:00.185 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-10T03:04:00.185 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-10T03:04:00.185 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-10T03:04:00.186 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.186 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.186 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.186 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.186 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.186 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.186 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:00.186 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:04:00.186 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:04:00.186 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:04:00.186 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:00.186 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:00.186 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:00.186 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:00.187 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.187 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.187 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.187 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.187 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.187 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-10T03:04:00.187 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-10T03:04:00.187 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.187 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.187 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.187 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.187 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.187 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.187 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.187 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.188 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.188 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.188 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.188 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.188 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-10T03:04:00.188 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-10T03:04:00.188 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.188 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.188 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.188 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-10T03:04:00.188 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.188 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.188 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:00.188 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:04:00.188 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:04:00.189 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:04:00.189 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:00.189 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:00.189 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:00.189 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:00.189 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.189 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.189 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.189 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.189 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.189 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-10T03:04:00.189 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-10T03:04:00.189 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.189 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.190 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.190 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.190 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.190 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.190 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.190 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.190 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.190 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.190 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.190 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:00.190 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:00.190 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.190 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-10T03:04:00.190 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "volumes", 2024-06-10T03:04:00.191 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-10T03:04:00.191 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-10T03:04:00.191 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-10T03:04:00.191 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-10T03:04:00.191 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-10T03:04:00.191 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.191 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.191 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.191 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.191 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.191 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.191 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:00.191 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:04:00.191 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:04:00.191 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:04:00.192 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:00.192 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:00.192 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:00.192 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:00.192 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.192 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.192 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.192 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.192 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.192 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-10T03:04:00.192 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-10T03:04:00.192 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.192 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.192 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.192 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.193 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.193 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.193 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.193 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.193 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.193 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.193 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.193 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.193 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-10T03:04:00.193 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-10T03:04:00.193 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.193 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.193 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.193 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-10T03:04:00.193 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.194 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.194 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:00.194 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:04:00.194 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:04:00.194 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:04:00.194 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:00.194 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:00.194 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:00.194 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:00.194 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.194 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.194 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.194 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.194 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.195 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-10T03:04:00.195 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-10T03:04:00.195 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.195 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.195 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.195 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.195 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.195 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.195 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.195 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.195 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.196 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.196 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.196 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.196 INFO:teuthology.orchestra.run.smithi052.stdout: "max_concurrent_clones": { 2024-06-10T03:04:00.196 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "max_concurrent_clones", 2024-06-10T03:04:00.196 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-10T03:04:00.196 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.196 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.196 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "4", 2024-06-10T03:04:00.196 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.196 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.196 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.196 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Number of asynchronous cloner threads", 2024-06-10T03:04:00.196 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.197 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.197 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.197 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.197 INFO:teuthology.orchestra.run.smithi052.stdout: "periodic_async_work": { 2024-06-10T03:04:00.197 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "periodic_async_work", 2024-06-10T03:04:00.197 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.197 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.197 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.197 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.197 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.197 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.197 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.197 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Periodically check for async work", 2024-06-10T03:04:00.197 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.198 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.198 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.198 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.198 INFO:teuthology.orchestra.run.smithi052.stdout: "snapshot_clone_delay": { 2024-06-10T03:04:00.198 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "snapshot_clone_delay", 2024-06-10T03:04:00.198 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-10T03:04:00.198 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.198 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.198 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "0", 2024-06-10T03:04:00.198 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.198 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.198 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.199 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-06-10T03:04:00.199 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.199 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.199 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.199 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.199 INFO:teuthology.orchestra.run.smithi052.stdout: "snapshot_clone_no_wait": { 2024-06-10T03:04:00.199 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "snapshot_clone_no_wait", 2024-06-10T03:04:00.199 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.199 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.199 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.199 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-10T03:04:00.199 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.199 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.199 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.199 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-06-10T03:04:00.200 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.200 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.200 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.200 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:00.200 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:00.200 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.200 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-10T03:04:00.200 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "zabbix", 2024-06-10T03:04:00.200 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-10T03:04:00.200 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-10T03:04:00.200 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-10T03:04:00.200 INFO:teuthology.orchestra.run.smithi052.stdout: "discovery_interval": { 2024-06-10T03:04:00.200 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "discovery_interval", 2024-06-10T03:04:00.201 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "uint", 2024-06-10T03:04:00.201 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.201 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.201 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "100", 2024-06-10T03:04:00.201 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.201 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.201 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.201 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.201 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.201 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.201 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.201 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.201 INFO:teuthology.orchestra.run.smithi052.stdout: "identifier": { 2024-06-10T03:04:00.201 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "identifier", 2024-06-10T03:04:00.201 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.202 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.202 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.202 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.202 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.202 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.202 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.202 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.202 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.202 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.202 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.202 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.202 INFO:teuthology.orchestra.run.smithi052.stdout: "interval": { 2024-06-10T03:04:00.203 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "interval", 2024-06-10T03:04:00.203 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-06-10T03:04:00.203 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.203 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.203 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "60", 2024-06-10T03:04:00.203 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.203 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.203 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.203 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.203 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.203 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.203 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.203 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.203 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-10T03:04:00.204 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-10T03:04:00.204 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.204 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.204 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.204 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.204 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.204 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.204 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:00.204 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:04:00.204 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:04:00.204 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:04:00.204 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:00.204 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:00.204 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:00.205 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:00.205 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.205 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.205 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.205 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.205 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.205 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-10T03:04:00.205 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-10T03:04:00.205 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.205 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.205 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.205 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.205 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.205 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.206 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.206 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.206 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.206 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.206 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.206 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.206 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-10T03:04:00.206 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-10T03:04:00.206 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.206 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.206 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.206 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-10T03:04:00.206 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.206 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.207 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:00.207 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:04:00.207 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:04:00.207 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:04:00.207 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:00.207 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:00.207 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:00.207 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:00.207 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.207 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.207 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.207 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.207 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.207 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-10T03:04:00.208 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-10T03:04:00.208 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.208 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.208 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.208 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.208 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.208 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.208 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.208 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.208 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.208 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.208 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.208 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.208 INFO:teuthology.orchestra.run.smithi052.stdout: "zabbix_host": { 2024-06-10T03:04:00.208 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "zabbix_host", 2024-06-10T03:04:00.209 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.209 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.209 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.209 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.209 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.209 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.209 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.209 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.209 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.209 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.209 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.209 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.209 INFO:teuthology.orchestra.run.smithi052.stdout: "zabbix_port": { 2024-06-10T03:04:00.209 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "zabbix_port", 2024-06-10T03:04:00.209 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-10T03:04:00.210 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.210 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.210 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "10051", 2024-06-10T03:04:00.210 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.210 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.210 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.210 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.210 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.210 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.210 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.210 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.210 INFO:teuthology.orchestra.run.smithi052.stdout: "zabbix_sender": { 2024-06-10T03:04:00.210 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "zabbix_sender", 2024-06-10T03:04:00.211 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.211 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.211 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.211 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-06-10T03:04:00.211 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.211 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.211 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.211 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.211 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.211 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.211 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.211 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:00.211 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:00.212 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:00.212 INFO:teuthology.orchestra.run.smithi052.stdout: ] 2024-06-10T03:04:00.212 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.212 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-10T03:04:00.212 INFO:teuthology.orchestra.run.smithi052.stdout: "gid": 24119, 2024-06-10T03:04:00.212 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "y", 2024-06-10T03:04:00.212 INFO:teuthology.orchestra.run.smithi052.stdout: "mgr_features": 4540701547738038271, 2024-06-10T03:04:00.212 INFO:teuthology.orchestra.run.smithi052.stdout: "available_modules": [ 2024-06-10T03:04:00.212 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-10T03:04:00.212 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "alerts", 2024-06-10T03:04:00.212 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-10T03:04:00.212 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-10T03:04:00.212 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-10T03:04:00.213 INFO:teuthology.orchestra.run.smithi052.stdout: "interval": { 2024-06-10T03:04:00.213 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "interval", 2024-06-10T03:04:00.213 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-06-10T03:04:00.213 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.213 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.213 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "60", 2024-06-10T03:04:00.213 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.213 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.213 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.213 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "How frequently to reexamine health status", 2024-06-10T03:04:00.213 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.213 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.213 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.213 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.214 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-10T03:04:00.214 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-10T03:04:00.214 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.214 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.214 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.214 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.214 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.214 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.214 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:00.214 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:04:00.214 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:04:00.214 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:04:00.214 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:00.215 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:00.215 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:00.215 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:00.215 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.215 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.215 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.215 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.215 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.215 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-10T03:04:00.215 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-10T03:04:00.215 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.215 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.215 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.216 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.216 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.216 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.216 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.216 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.216 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.216 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.216 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.216 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.216 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-10T03:04:00.216 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-10T03:04:00.216 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.216 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.216 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.216 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-10T03:04:00.217 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.217 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.217 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:00.217 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:04:00.217 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:04:00.217 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:04:00.217 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:00.217 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:00.217 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:00.217 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:00.217 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.217 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.217 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.218 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.218 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.218 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-10T03:04:00.218 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-10T03:04:00.218 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.218 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.218 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.218 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.218 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.218 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.218 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.218 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.218 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.219 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.219 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.219 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.219 INFO:teuthology.orchestra.run.smithi052.stdout: "smtp_destination": { 2024-06-10T03:04:00.219 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "smtp_destination", 2024-06-10T03:04:00.219 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.219 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.219 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.219 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.219 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.219 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.219 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.219 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Email address to send alerts to", 2024-06-10T03:04:00.219 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.220 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.220 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.220 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.220 INFO:teuthology.orchestra.run.smithi052.stdout: "smtp_from_name": { 2024-06-10T03:04:00.220 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "smtp_from_name", 2024-06-10T03:04:00.220 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.220 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.220 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.220 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "Ceph", 2024-06-10T03:04:00.220 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.220 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.220 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.220 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Email From: name", 2024-06-10T03:04:00.220 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.221 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.221 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.221 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.221 INFO:teuthology.orchestra.run.smithi052.stdout: "smtp_host": { 2024-06-10T03:04:00.221 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "smtp_host", 2024-06-10T03:04:00.221 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.221 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.221 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.221 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.221 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.221 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.221 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.221 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "SMTP server", 2024-06-10T03:04:00.221 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.222 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.222 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.222 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.222 INFO:teuthology.orchestra.run.smithi052.stdout: "smtp_password": { 2024-06-10T03:04:00.222 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "smtp_password", 2024-06-10T03:04:00.222 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.222 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.222 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.222 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.222 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.222 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.222 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.222 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Password to authenticate with", 2024-06-10T03:04:00.222 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.223 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.223 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.223 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.223 INFO:teuthology.orchestra.run.smithi052.stdout: "smtp_port": { 2024-06-10T03:04:00.223 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "smtp_port", 2024-06-10T03:04:00.223 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-10T03:04:00.223 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.223 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.223 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "465", 2024-06-10T03:04:00.223 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.223 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.223 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.223 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "SMTP port", 2024-06-10T03:04:00.223 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.224 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.224 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.224 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.224 INFO:teuthology.orchestra.run.smithi052.stdout: "smtp_sender": { 2024-06-10T03:04:00.224 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "smtp_sender", 2024-06-10T03:04:00.224 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.224 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.224 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.224 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.224 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.224 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.224 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.224 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "SMTP envelope sender", 2024-06-10T03:04:00.225 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.225 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.225 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.225 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.225 INFO:teuthology.orchestra.run.smithi052.stdout: "smtp_ssl": { 2024-06-10T03:04:00.225 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "smtp_ssl", 2024-06-10T03:04:00.225 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.225 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.225 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.225 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-10T03:04:00.225 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.225 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.225 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.226 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Use SSL to connect to SMTP server", 2024-06-10T03:04:00.226 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.226 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.226 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.226 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.226 INFO:teuthology.orchestra.run.smithi052.stdout: "smtp_user": { 2024-06-10T03:04:00.226 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "smtp_user", 2024-06-10T03:04:00.226 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.226 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.226 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.226 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.226 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.226 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.227 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.227 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "User to authenticate as", 2024-06-10T03:04:00.227 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.227 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.227 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.227 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:00.227 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:00.227 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.227 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-10T03:04:00.227 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "balancer", 2024-06-10T03:04:00.227 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-10T03:04:00.227 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-10T03:04:00.228 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-10T03:04:00.228 INFO:teuthology.orchestra.run.smithi052.stdout: "active": { 2024-06-10T03:04:00.228 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "active", 2024-06-10T03:04:00.228 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.228 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.228 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.228 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-10T03:04:00.228 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.228 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.228 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.228 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "automatically balance PGs across cluster", 2024-06-10T03:04:00.228 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.228 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.229 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.229 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.229 INFO:teuthology.orchestra.run.smithi052.stdout: "begin_time": { 2024-06-10T03:04:00.229 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "begin_time", 2024-06-10T03:04:00.229 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.229 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.229 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.229 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "0000", 2024-06-10T03:04:00.229 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.229 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.229 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.229 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "beginning time of day to automatically balance", 2024-06-10T03:04:00.229 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-06-10T03:04:00.229 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.230 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.230 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.230 INFO:teuthology.orchestra.run.smithi052.stdout: "begin_weekday": { 2024-06-10T03:04:00.230 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "begin_weekday", 2024-06-10T03:04:00.230 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "uint", 2024-06-10T03:04:00.230 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.230 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.230 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "0", 2024-06-10T03:04:00.230 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "0", 2024-06-10T03:04:00.230 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "6", 2024-06-10T03:04:00.230 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.230 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-06-10T03:04:00.231 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-06-10T03:04:00.231 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.231 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.231 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.231 INFO:teuthology.orchestra.run.smithi052.stdout: "crush_compat_max_iterations": { 2024-06-10T03:04:00.231 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "crush_compat_max_iterations", 2024-06-10T03:04:00.231 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "uint", 2024-06-10T03:04:00.231 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.231 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.231 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "25", 2024-06-10T03:04:00.231 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "1", 2024-06-10T03:04:00.231 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "250", 2024-06-10T03:04:00.231 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.231 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-06-10T03:04:00.232 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.232 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.232 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.232 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.232 INFO:teuthology.orchestra.run.smithi052.stdout: "crush_compat_metrics": { 2024-06-10T03:04:00.232 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "crush_compat_metrics", 2024-06-10T03:04:00.232 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.232 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.232 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.232 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "pgs,objects,bytes", 2024-06-10T03:04:00.232 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.232 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.232 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.233 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-06-10T03:04:00.233 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-06-10T03:04:00.233 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.233 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.233 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.233 INFO:teuthology.orchestra.run.smithi052.stdout: "crush_compat_step": { 2024-06-10T03:04:00.233 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "crush_compat_step", 2024-06-10T03:04:00.233 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "float", 2024-06-10T03:04:00.233 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.233 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.233 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "0.5", 2024-06-10T03:04:00.233 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "0.001", 2024-06-10T03:04:00.233 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "0.999", 2024-06-10T03:04:00.234 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.234 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "aggressiveness of optimization", 2024-06-10T03:04:00.234 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-06-10T03:04:00.234 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.234 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.234 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.234 INFO:teuthology.orchestra.run.smithi052.stdout: "end_time": { 2024-06-10T03:04:00.234 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "end_time", 2024-06-10T03:04:00.234 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.234 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.234 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.234 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "2359", 2024-06-10T03:04:00.234 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.235 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.235 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.235 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "ending time of day to automatically balance", 2024-06-10T03:04:00.235 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-06-10T03:04:00.235 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.235 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.235 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.235 INFO:teuthology.orchestra.run.smithi052.stdout: "end_weekday": { 2024-06-10T03:04:00.235 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "end_weekday", 2024-06-10T03:04:00.235 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "uint", 2024-06-10T03:04:00.235 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.235 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.235 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "0", 2024-06-10T03:04:00.236 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "0", 2024-06-10T03:04:00.236 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "6", 2024-06-10T03:04:00.236 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.236 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-06-10T03:04:00.236 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-06-10T03:04:00.236 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.236 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.236 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.236 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-10T03:04:00.236 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-10T03:04:00.236 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.236 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.236 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.237 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.237 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.237 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.237 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:00.237 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:04:00.237 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:04:00.237 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:04:00.237 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:00.237 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:00.237 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:00.237 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:00.237 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.237 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.238 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.238 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.238 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.238 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-10T03:04:00.238 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-10T03:04:00.238 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.238 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.238 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.238 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.238 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.238 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.238 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.238 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.239 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.239 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.239 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.239 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.239 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-10T03:04:00.239 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-10T03:04:00.239 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.239 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.239 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.239 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-10T03:04:00.239 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.239 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.239 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:00.240 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:04:00.240 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:04:00.240 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:04:00.240 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:00.240 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:00.240 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:00.240 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:00.240 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.240 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.240 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.240 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.240 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.241 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-10T03:04:00.241 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-10T03:04:00.241 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.241 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.241 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.241 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.241 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.241 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.241 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.241 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.241 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.241 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.241 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.242 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.242 INFO:teuthology.orchestra.run.smithi052.stdout: "min_score": { 2024-06-10T03:04:00.242 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "min_score", 2024-06-10T03:04:00.242 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "float", 2024-06-10T03:04:00.242 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.242 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.242 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "0", 2024-06-10T03:04:00.242 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.242 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.242 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.242 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-06-10T03:04:00.242 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.242 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.243 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.243 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.243 INFO:teuthology.orchestra.run.smithi052.stdout: "mode": { 2024-06-10T03:04:00.243 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "mode", 2024-06-10T03:04:00.243 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.243 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.243 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.243 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "upmap", 2024-06-10T03:04:00.243 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.243 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.243 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:00.243 INFO:teuthology.orchestra.run.smithi052.stdout: "crush-compat", 2024-06-10T03:04:00.243 INFO:teuthology.orchestra.run.smithi052.stdout: "none", 2024-06-10T03:04:00.243 INFO:teuthology.orchestra.run.smithi052.stdout: "read", 2024-06-10T03:04:00.244 INFO:teuthology.orchestra.run.smithi052.stdout: "upmap", 2024-06-10T03:04:00.244 INFO:teuthology.orchestra.run.smithi052.stdout: "upmap-read" 2024-06-10T03:04:00.244 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:00.244 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Balancer mode", 2024-06-10T03:04:00.244 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.244 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.244 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.244 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.244 INFO:teuthology.orchestra.run.smithi052.stdout: "pool_ids": { 2024-06-10T03:04:00.244 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "pool_ids", 2024-06-10T03:04:00.244 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.244 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.244 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.244 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.244 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.245 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.245 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.245 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-06-10T03:04:00.245 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.245 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.245 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.245 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.245 INFO:teuthology.orchestra.run.smithi052.stdout: "sleep_interval": { 2024-06-10T03:04:00.245 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "sleep_interval", 2024-06-10T03:04:00.245 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-06-10T03:04:00.245 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.245 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.245 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "60", 2024-06-10T03:04:00.245 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.245 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.246 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.246 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-06-10T03:04:00.246 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.246 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.246 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.246 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.246 INFO:teuthology.orchestra.run.smithi052.stdout: "upmap_max_deviation": { 2024-06-10T03:04:00.246 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "upmap_max_deviation", 2024-06-10T03:04:00.246 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-10T03:04:00.246 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.246 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.246 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "5", 2024-06-10T03:04:00.246 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "1", 2024-06-10T03:04:00.247 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.247 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.247 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "deviation below which no optimization is attempted", 2024-06-10T03:04:00.247 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-06-10T03:04:00.247 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.247 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.247 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.247 INFO:teuthology.orchestra.run.smithi052.stdout: "upmap_max_optimizations": { 2024-06-10T03:04:00.247 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "upmap_max_optimizations", 2024-06-10T03:04:00.247 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "uint", 2024-06-10T03:04:00.247 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.247 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.247 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "10", 2024-06-10T03:04:00.247 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.247 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.248 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.248 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-06-10T03:04:00.248 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.248 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.248 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.248 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:00.248 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:00.248 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.248 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-10T03:04:00.248 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "cephadm", 2024-06-10T03:04:00.248 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-10T03:04:00.248 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-10T03:04:00.248 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-10T03:04:00.248 INFO:teuthology.orchestra.run.smithi052.stdout: "agent_down_multiplier": { 2024-06-10T03:04:00.248 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "agent_down_multiplier", 2024-06-10T03:04:00.249 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "float", 2024-06-10T03:04:00.249 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.249 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.249 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "3.0", 2024-06-10T03:04:00.249 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.249 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.249 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.249 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-06-10T03:04:00.249 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.249 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.249 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.249 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.249 INFO:teuthology.orchestra.run.smithi052.stdout: "agent_refresh_rate": { 2024-06-10T03:04:00.249 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "agent_refresh_rate", 2024-06-10T03:04:00.250 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-06-10T03:04:00.250 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.250 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.250 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "20", 2024-06-10T03:04:00.250 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.250 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.250 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.250 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-06-10T03:04:00.250 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.250 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.250 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.250 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.251 INFO:teuthology.orchestra.run.smithi052.stdout: "agent_starting_port": { 2024-06-10T03:04:00.251 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "agent_starting_port", 2024-06-10T03:04:00.251 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-10T03:04:00.251 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.251 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.251 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "4721", 2024-06-10T03:04:00.251 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.251 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.251 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.251 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-06-10T03:04:00.251 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.251 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.251 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.251 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.251 INFO:teuthology.orchestra.run.smithi052.stdout: "allow_ptrace": { 2024-06-10T03:04:00.252 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "allow_ptrace", 2024-06-10T03:04:00.252 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.252 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.252 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.252 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.252 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.252 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.252 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.252 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-06-10T03:04:00.252 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-06-10T03:04:00.252 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.252 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.252 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.252 INFO:teuthology.orchestra.run.smithi052.stdout: "autotune_interval": { 2024-06-10T03:04:00.253 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "autotune_interval", 2024-06-10T03:04:00.253 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-06-10T03:04:00.253 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.253 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.253 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "600", 2024-06-10T03:04:00.253 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.253 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.253 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.253 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "how frequently to autotune daemon memory", 2024-06-10T03:04:00.253 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.253 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.253 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.253 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.253 INFO:teuthology.orchestra.run.smithi052.stdout: "autotune_memory_target_ratio": { 2024-06-10T03:04:00.254 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "autotune_memory_target_ratio", 2024-06-10T03:04:00.254 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "float", 2024-06-10T03:04:00.254 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.254 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.254 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "0.7", 2024-06-10T03:04:00.254 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.254 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.254 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.254 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-06-10T03:04:00.254 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.254 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.254 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.254 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.254 INFO:teuthology.orchestra.run.smithi052.stdout: "cephadm_log_destination": { 2024-06-10T03:04:00.255 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "cephadm_log_destination", 2024-06-10T03:04:00.255 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.255 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.255 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.255 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.255 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.255 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.255 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:00.255 INFO:teuthology.orchestra.run.smithi052.stdout: "file", 2024-06-10T03:04:00.255 INFO:teuthology.orchestra.run.smithi052.stdout: "file,syslog", 2024-06-10T03:04:00.255 INFO:teuthology.orchestra.run.smithi052.stdout: "syslog" 2024-06-10T03:04:00.255 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:00.255 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-06-10T03:04:00.255 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.255 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.256 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.256 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.256 INFO:teuthology.orchestra.run.smithi052.stdout: "cgroups_split": { 2024-06-10T03:04:00.256 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "cgroups_split", 2024-06-10T03:04:00.256 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.256 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.256 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.256 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-10T03:04:00.256 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.256 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.256 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.256 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-06-10T03:04:00.256 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.256 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.256 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.257 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.257 INFO:teuthology.orchestra.run.smithi052.stdout: "config_checks_enabled": { 2024-06-10T03:04:00.257 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "config_checks_enabled", 2024-06-10T03:04:00.257 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.257 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.257 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.257 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.257 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.257 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.257 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.257 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-06-10T03:04:00.257 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.257 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.257 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.258 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.258 INFO:teuthology.orchestra.run.smithi052.stdout: "config_dashboard": { 2024-06-10T03:04:00.258 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "config_dashboard", 2024-06-10T03:04:00.258 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.258 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.258 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.258 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-10T03:04:00.258 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.258 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.258 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.258 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-06-10T03:04:00.258 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.259 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.259 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.259 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.259 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_alertmanager": { 2024-06-10T03:04:00.259 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_alertmanager", 2024-06-10T03:04:00.259 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.259 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.259 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.259 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-06-10T03:04:00.259 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.259 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.259 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.259 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Prometheus container image", 2024-06-10T03:04:00.259 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.260 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.260 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.260 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.260 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_base": { 2024-06-10T03:04:00.260 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_base", 2024-06-10T03:04:00.260 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.260 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.260 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.260 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/ceph/ceph", 2024-06-10T03:04:00.260 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.260 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.260 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.260 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Container image name, without the tag", 2024-06-10T03:04:00.260 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.261 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.261 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.261 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.261 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_elasticsearch": { 2024-06-10T03:04:00.261 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_elasticsearch", 2024-06-10T03:04:00.261 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.261 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.261 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.261 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-06-10T03:04:00.261 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.261 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.261 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.261 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "elasticsearch container image", 2024-06-10T03:04:00.262 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.262 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.262 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.262 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.262 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_grafana": { 2024-06-10T03:04:00.262 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_grafana", 2024-06-10T03:04:00.262 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.262 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.262 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.262 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-06-10T03:04:00.262 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.262 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.262 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.263 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Prometheus container image", 2024-06-10T03:04:00.263 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.263 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.263 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.263 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.263 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_haproxy": { 2024-06-10T03:04:00.263 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_haproxy", 2024-06-10T03:04:00.263 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.263 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.263 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.263 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-06-10T03:04:00.263 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.263 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.264 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.264 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "HAproxy container image", 2024-06-10T03:04:00.264 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.264 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.264 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.264 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.264 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_jaeger_agent": { 2024-06-10T03:04:00.264 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_jaeger_agent", 2024-06-10T03:04:00.264 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.264 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.264 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.264 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-06-10T03:04:00.264 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.264 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.265 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.265 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Jaeger agent container image", 2024-06-10T03:04:00.265 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.265 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.265 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.265 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.265 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_jaeger_collector": { 2024-06-10T03:04:00.265 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_jaeger_collector", 2024-06-10T03:04:00.265 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.265 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.265 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.265 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-06-10T03:04:00.266 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.266 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.266 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.266 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Jaeger collector container image", 2024-06-10T03:04:00.266 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.266 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.266 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.266 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.266 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_jaeger_query": { 2024-06-10T03:04:00.266 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_jaeger_query", 2024-06-10T03:04:00.266 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.266 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.267 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.267 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-06-10T03:04:00.267 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.267 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.267 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.267 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Jaeger query container image", 2024-06-10T03:04:00.267 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.267 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.267 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.267 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.267 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_keepalived": { 2024-06-10T03:04:00.267 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_keepalived", 2024-06-10T03:04:00.267 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.267 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.268 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.268 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-06-10T03:04:00.268 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.268 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.268 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.268 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Keepalived container image", 2024-06-10T03:04:00.268 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.268 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.268 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.268 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.268 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_loki": { 2024-06-10T03:04:00.268 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_loki", 2024-06-10T03:04:00.268 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.269 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.269 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.269 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-06-10T03:04:00.269 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.269 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.269 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.269 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Loki container image", 2024-06-10T03:04:00.269 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.269 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.269 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.269 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.269 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_node_exporter": { 2024-06-10T03:04:00.269 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_node_exporter", 2024-06-10T03:04:00.270 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.270 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.270 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.270 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-06-10T03:04:00.270 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.270 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.270 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.270 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Prometheus container image", 2024-06-10T03:04:00.270 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.270 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.270 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.270 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.270 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_nvmeof": { 2024-06-10T03:04:00.271 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_nvmeof", 2024-06-10T03:04:00.271 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.271 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.271 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.271 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-06-10T03:04:00.271 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.271 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.271 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.271 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Nvme-of container image", 2024-06-10T03:04:00.271 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.271 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.271 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.271 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.272 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_prometheus": { 2024-06-10T03:04:00.272 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_prometheus", 2024-06-10T03:04:00.272 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.272 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.272 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.272 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-06-10T03:04:00.272 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.272 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.272 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.272 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Prometheus container image", 2024-06-10T03:04:00.272 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.272 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.272 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.272 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.273 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_promtail": { 2024-06-10T03:04:00.273 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_promtail", 2024-06-10T03:04:00.273 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.273 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.273 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.273 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-06-10T03:04:00.273 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.273 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.273 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.273 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Promtail container image", 2024-06-10T03:04:00.273 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.273 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.273 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.274 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.274 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_samba": { 2024-06-10T03:04:00.274 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_samba", 2024-06-10T03:04:00.274 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.274 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.274 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.274 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-06-10T03:04:00.274 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.274 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.274 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.274 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Samba/SMB container image", 2024-06-10T03:04:00.274 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.275 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.275 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.275 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.275 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_snmp_gateway": { 2024-06-10T03:04:00.275 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_snmp_gateway", 2024-06-10T03:04:00.275 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.275 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.275 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.275 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-06-10T03:04:00.275 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.275 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.275 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.275 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "SNMP Gateway container image", 2024-06-10T03:04:00.276 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.276 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.276 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.276 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.276 INFO:teuthology.orchestra.run.smithi052.stdout: "container_init": { 2024-06-10T03:04:00.276 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_init", 2024-06-10T03:04:00.276 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.276 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.276 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.276 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-10T03:04:00.276 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.276 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.276 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.276 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Run podman/docker with `--init`", 2024-06-10T03:04:00.277 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.277 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.277 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.277 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.277 INFO:teuthology.orchestra.run.smithi052.stdout: "daemon_cache_timeout": { 2024-06-10T03:04:00.277 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "daemon_cache_timeout", 2024-06-10T03:04:00.277 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-06-10T03:04:00.277 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.277 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.277 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "600", 2024-06-10T03:04:00.277 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.277 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.277 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.278 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-06-10T03:04:00.278 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.278 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.278 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.278 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.278 INFO:teuthology.orchestra.run.smithi052.stdout: "default_cephadm_command_timeout": { 2024-06-10T03:04:00.278 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "default_cephadm_command_timeout", 2024-06-10T03:04:00.278 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-10T03:04:00.278 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.278 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.278 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "900", 2024-06-10T03:04:00.278 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.278 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.279 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.279 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-06-10T03:04:00.279 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.279 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.279 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.279 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.279 INFO:teuthology.orchestra.run.smithi052.stdout: "default_registry": { 2024-06-10T03:04:00.279 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "default_registry", 2024-06-10T03:04:00.279 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.279 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.279 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.279 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "docker.io", 2024-06-10T03:04:00.280 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.280 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.280 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.280 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-06-10T03:04:00.280 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.280 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.280 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.280 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.280 INFO:teuthology.orchestra.run.smithi052.stdout: "device_cache_timeout": { 2024-06-10T03:04:00.280 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "device_cache_timeout", 2024-06-10T03:04:00.280 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-06-10T03:04:00.280 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.280 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.281 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "1800", 2024-06-10T03:04:00.281 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.281 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.281 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.281 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "seconds to cache device inventory", 2024-06-10T03:04:00.281 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.281 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.281 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.281 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.281 INFO:teuthology.orchestra.run.smithi052.stdout: "device_enhanced_scan": { 2024-06-10T03:04:00.281 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "device_enhanced_scan", 2024-06-10T03:04:00.281 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.281 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.282 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.282 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.282 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.282 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.282 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.282 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Use libstoragemgmt during device scans", 2024-06-10T03:04:00.282 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.282 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.282 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.282 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.282 INFO:teuthology.orchestra.run.smithi052.stdout: "facts_cache_timeout": { 2024-06-10T03:04:00.282 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "facts_cache_timeout", 2024-06-10T03:04:00.283 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-06-10T03:04:00.283 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.283 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.283 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "60", 2024-06-10T03:04:00.283 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.283 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.283 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.283 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "seconds to cache host facts data", 2024-06-10T03:04:00.283 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.283 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.283 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.283 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.283 INFO:teuthology.orchestra.run.smithi052.stdout: "grafana_dashboards_path": { 2024-06-10T03:04:00.283 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "grafana_dashboards_path", 2024-06-10T03:04:00.284 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.284 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.284 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.284 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-06-10T03:04:00.284 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.284 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.284 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.284 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-06-10T03:04:00.284 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.284 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.284 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.284 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.284 INFO:teuthology.orchestra.run.smithi052.stdout: "host_check_interval": { 2024-06-10T03:04:00.285 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "host_check_interval", 2024-06-10T03:04:00.285 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-06-10T03:04:00.285 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.285 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.285 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "600", 2024-06-10T03:04:00.285 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.285 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.285 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.285 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "how frequently to perform a host check", 2024-06-10T03:04:00.285 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.285 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.285 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.285 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.286 INFO:teuthology.orchestra.run.smithi052.stdout: "hw_monitoring": { 2024-06-10T03:04:00.286 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "hw_monitoring", 2024-06-10T03:04:00.286 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.286 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.286 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.286 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.286 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.286 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.286 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.286 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-06-10T03:04:00.286 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.286 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.287 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.287 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.287 INFO:teuthology.orchestra.run.smithi052.stdout: "inventory_list_all": { 2024-06-10T03:04:00.287 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "inventory_list_all", 2024-06-10T03:04:00.287 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.287 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.287 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.287 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.287 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.287 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.287 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.287 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-06-10T03:04:00.287 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.288 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.288 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.288 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.288 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-10T03:04:00.288 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-10T03:04:00.288 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.288 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.288 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.288 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.288 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.288 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.288 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:00.288 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:04:00.288 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:04:00.289 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:04:00.289 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:00.289 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:00.289 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:00.289 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:00.289 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.289 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.289 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.289 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.289 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.289 INFO:teuthology.orchestra.run.smithi052.stdout: "log_refresh_metadata": { 2024-06-10T03:04:00.289 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_refresh_metadata", 2024-06-10T03:04:00.289 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.290 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.290 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.290 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.290 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.290 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.290 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.290 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-06-10T03:04:00.290 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.290 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.290 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.290 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.290 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-10T03:04:00.291 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-10T03:04:00.291 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.291 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.291 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.291 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-10T03:04:00.291 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.291 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.291 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.291 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-06-10T03:04:00.291 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.291 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.291 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.291 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.292 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-10T03:04:00.292 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-10T03:04:00.292 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.292 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.292 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.292 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-10T03:04:00.292 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.292 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.292 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:00.292 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:04:00.292 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:04:00.292 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:04:00.292 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:00.293 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:00.293 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:00.293 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:00.293 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.293 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.293 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.293 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.293 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.293 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-10T03:04:00.293 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-10T03:04:00.293 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.293 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.293 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.294 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.294 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.294 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.294 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.294 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.294 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.294 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.294 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.294 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.294 INFO:teuthology.orchestra.run.smithi052.stdout: "manage_etc_ceph_ceph_conf": { 2024-06-10T03:04:00.294 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-06-10T03:04:00.294 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.295 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.295 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.295 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.295 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.295 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.295 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.295 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-06-10T03:04:00.295 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.295 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.295 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.295 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.295 INFO:teuthology.orchestra.run.smithi052.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-06-10T03:04:00.295 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-06-10T03:04:00.296 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.296 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.296 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.296 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "*", 2024-06-10T03:04:00.296 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.296 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.296 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.296 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-06-10T03:04:00.296 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.296 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.296 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.296 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.296 INFO:teuthology.orchestra.run.smithi052.stdout: "max_count_per_host": { 2024-06-10T03:04:00.296 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "max_count_per_host", 2024-06-10T03:04:00.297 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-10T03:04:00.297 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.297 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.297 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "10", 2024-06-10T03:04:00.297 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.297 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.297 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.297 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "max number of daemons per service per host", 2024-06-10T03:04:00.297 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.297 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.297 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.297 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.297 INFO:teuthology.orchestra.run.smithi052.stdout: "max_osd_draining_count": { 2024-06-10T03:04:00.297 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "max_osd_draining_count", 2024-06-10T03:04:00.298 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-10T03:04:00.298 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.298 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.298 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "10", 2024-06-10T03:04:00.298 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.298 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.298 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.298 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-06-10T03:04:00.298 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.298 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.298 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.298 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.298 INFO:teuthology.orchestra.run.smithi052.stdout: "migration_current": { 2024-06-10T03:04:00.299 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "migration_current", 2024-06-10T03:04:00.299 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-10T03:04:00.299 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.299 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.299 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.299 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.299 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.299 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.299 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "internal - do not modify", 2024-06-10T03:04:00.299 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.299 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.299 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.299 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.299 INFO:teuthology.orchestra.run.smithi052.stdout: "mode": { 2024-06-10T03:04:00.300 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "mode", 2024-06-10T03:04:00.300 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.300 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.300 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.300 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "root", 2024-06-10T03:04:00.300 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.300 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.300 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:00.300 INFO:teuthology.orchestra.run.smithi052.stdout: "cephadm-package", 2024-06-10T03:04:00.300 INFO:teuthology.orchestra.run.smithi052.stdout: "root" 2024-06-10T03:04:00.300 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:00.300 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "mode for remote execution of cephadm", 2024-06-10T03:04:00.300 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.301 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.301 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.301 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.301 INFO:teuthology.orchestra.run.smithi052.stdout: "oob_default_addr": { 2024-06-10T03:04:00.301 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "oob_default_addr", 2024-06-10T03:04:00.301 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.301 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.301 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.301 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "169.254.1.1", 2024-06-10T03:04:00.301 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.301 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.301 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.301 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Default address for RedFish API (oob management).", 2024-06-10T03:04:00.301 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.302 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.302 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.302 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.302 INFO:teuthology.orchestra.run.smithi052.stdout: "prometheus_alerts_path": { 2024-06-10T03:04:00.302 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "prometheus_alerts_path", 2024-06-10T03:04:00.302 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.302 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.302 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.302 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-06-10T03:04:00.302 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.302 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.302 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.302 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-06-10T03:04:00.303 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.303 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.303 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.303 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.303 INFO:teuthology.orchestra.run.smithi052.stdout: "registry_insecure": { 2024-06-10T03:04:00.303 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "registry_insecure", 2024-06-10T03:04:00.303 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.303 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.303 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.303 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.303 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.303 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.303 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.303 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-06-10T03:04:00.304 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.304 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.304 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.304 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.304 INFO:teuthology.orchestra.run.smithi052.stdout: "registry_password": { 2024-06-10T03:04:00.304 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "registry_password", 2024-06-10T03:04:00.304 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.304 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.304 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.304 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.304 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.304 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.305 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.305 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-06-10T03:04:00.305 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.305 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.305 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.305 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.305 INFO:teuthology.orchestra.run.smithi052.stdout: "registry_url": { 2024-06-10T03:04:00.305 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "registry_url", 2024-06-10T03:04:00.305 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.305 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.305 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.305 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.305 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.306 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.306 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.306 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-06-10T03:04:00.306 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.306 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.306 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.306 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.306 INFO:teuthology.orchestra.run.smithi052.stdout: "registry_username": { 2024-06-10T03:04:00.306 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "registry_username", 2024-06-10T03:04:00.306 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.306 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.306 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.306 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.307 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.307 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.307 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.307 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-06-10T03:04:00.307 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.307 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.307 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.307 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.307 INFO:teuthology.orchestra.run.smithi052.stdout: "secure_monitoring_stack": { 2024-06-10T03:04:00.307 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "secure_monitoring_stack", 2024-06-10T03:04:00.307 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.307 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.307 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.308 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.308 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.308 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.308 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.308 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-06-10T03:04:00.308 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.308 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.308 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.308 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.308 INFO:teuthology.orchestra.run.smithi052.stdout: "service_discovery_port": { 2024-06-10T03:04:00.308 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "service_discovery_port", 2024-06-10T03:04:00.308 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-10T03:04:00.308 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.308 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.309 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "8765", 2024-06-10T03:04:00.309 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.309 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.309 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.309 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "cephadm service discovery port", 2024-06-10T03:04:00.309 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.309 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.309 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.309 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.309 INFO:teuthology.orchestra.run.smithi052.stdout: "ssh_config_file": { 2024-06-10T03:04:00.309 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "ssh_config_file", 2024-06-10T03:04:00.309 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.309 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.310 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.310 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.310 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.310 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.310 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.310 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-06-10T03:04:00.310 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.310 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.310 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.310 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.310 INFO:teuthology.orchestra.run.smithi052.stdout: "use_agent": { 2024-06-10T03:04:00.310 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "use_agent", 2024-06-10T03:04:00.310 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.310 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.311 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.311 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.311 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.311 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.311 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.311 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-06-10T03:04:00.311 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.311 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.311 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.311 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.311 INFO:teuthology.orchestra.run.smithi052.stdout: "use_repo_digest": { 2024-06-10T03:04:00.311 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "use_repo_digest", 2024-06-10T03:04:00.311 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.312 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.312 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.312 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-10T03:04:00.312 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.312 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.312 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.312 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-06-10T03:04:00.312 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.312 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.312 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.312 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.312 INFO:teuthology.orchestra.run.smithi052.stdout: "warn_on_failed_host_check": { 2024-06-10T03:04:00.312 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "warn_on_failed_host_check", 2024-06-10T03:04:00.313 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.313 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.313 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.313 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-10T03:04:00.313 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.313 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.313 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.313 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "raise a health warning if the host check fails", 2024-06-10T03:04:00.313 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.313 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.313 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.313 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.313 INFO:teuthology.orchestra.run.smithi052.stdout: "warn_on_stray_daemons": { 2024-06-10T03:04:00.313 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "warn_on_stray_daemons", 2024-06-10T03:04:00.314 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.314 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.314 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.314 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-10T03:04:00.314 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.314 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.314 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.314 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-06-10T03:04:00.314 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.314 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.314 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.314 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.314 INFO:teuthology.orchestra.run.smithi052.stdout: "warn_on_stray_hosts": { 2024-06-10T03:04:00.315 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "warn_on_stray_hosts", 2024-06-10T03:04:00.315 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.315 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.315 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.315 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-10T03:04:00.315 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.315 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.315 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.315 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-06-10T03:04:00.315 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.315 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.315 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.315 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:00.315 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:00.316 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.316 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-10T03:04:00.316 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "crash", 2024-06-10T03:04:00.316 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-10T03:04:00.316 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-10T03:04:00.316 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-10T03:04:00.316 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-10T03:04:00.316 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-10T03:04:00.316 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.316 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.316 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.316 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.316 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.316 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.317 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:00.317 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:04:00.317 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:04:00.317 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:04:00.317 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:00.317 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:00.317 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:00.317 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:00.317 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.317 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.317 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.317 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.318 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.318 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-10T03:04:00.318 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-10T03:04:00.318 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.318 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.318 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.318 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.318 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.322 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.322 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.322 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.322 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.322 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.322 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.322 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.322 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-10T03:04:00.322 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-10T03:04:00.322 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.322 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.322 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.323 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-10T03:04:00.323 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.323 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.323 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:00.323 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:04:00.323 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:04:00.323 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:04:00.323 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:00.323 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:00.323 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:00.323 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:00.323 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.323 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.324 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.324 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.324 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.324 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-10T03:04:00.324 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-10T03:04:00.324 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.324 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.324 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.324 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.324 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.324 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.324 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.324 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.324 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.325 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.325 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.325 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.325 INFO:teuthology.orchestra.run.smithi052.stdout: "retain_interval": { 2024-06-10T03:04:00.325 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "retain_interval", 2024-06-10T03:04:00.325 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-06-10T03:04:00.325 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.325 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.325 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "31536000", 2024-06-10T03:04:00.325 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.325 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.325 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.326 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "how long to retain crashes before pruning them", 2024-06-10T03:04:00.326 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.326 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.326 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.326 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.326 INFO:teuthology.orchestra.run.smithi052.stdout: "warn_recent_interval": { 2024-06-10T03:04:00.326 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "warn_recent_interval", 2024-06-10T03:04:00.326 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-06-10T03:04:00.326 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.326 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.326 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "1209600", 2024-06-10T03:04:00.326 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.326 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.327 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.327 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "time interval in which to warn about recent crashes", 2024-06-10T03:04:00.327 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.327 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.327 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.327 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:00.327 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:00.327 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.327 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-10T03:04:00.327 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "dashboard", 2024-06-10T03:04:00.327 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-10T03:04:00.327 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-10T03:04:00.327 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-10T03:04:00.328 INFO:teuthology.orchestra.run.smithi052.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-06-10T03:04:00.328 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-06-10T03:04:00.328 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-10T03:04:00.328 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.328 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.328 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "10", 2024-06-10T03:04:00.328 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.328 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.328 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.328 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.328 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.328 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.328 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.328 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.329 INFO:teuthology.orchestra.run.smithi052.stdout: "ALERTMANAGER_API_HOST": { 2024-06-10T03:04:00.329 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "ALERTMANAGER_API_HOST", 2024-06-10T03:04:00.329 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.329 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.329 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.329 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.329 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.329 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.329 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.329 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.329 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.329 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.329 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.329 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.330 INFO:teuthology.orchestra.run.smithi052.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-06-10T03:04:00.330 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-06-10T03:04:00.330 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.330 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.330 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.330 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-10T03:04:00.330 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.330 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.330 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.330 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.330 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.330 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.330 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.331 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.331 INFO:teuthology.orchestra.run.smithi052.stdout: "AUDIT_API_ENABLED": { 2024-06-10T03:04:00.331 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "AUDIT_API_ENABLED", 2024-06-10T03:04:00.331 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.331 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.331 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.331 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.331 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.331 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.331 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.331 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.331 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.331 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.332 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.332 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.332 INFO:teuthology.orchestra.run.smithi052.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-06-10T03:04:00.332 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-06-10T03:04:00.332 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.332 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.332 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.332 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-10T03:04:00.332 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.332 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.332 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.332 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.332 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.333 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.333 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.333 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.333 INFO:teuthology.orchestra.run.smithi052.stdout: "ENABLE_BROWSABLE_API": { 2024-06-10T03:04:00.333 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "ENABLE_BROWSABLE_API", 2024-06-10T03:04:00.333 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.333 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.333 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.333 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-10T03:04:00.333 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.333 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.333 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.333 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.333 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.334 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.334 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.334 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.334 INFO:teuthology.orchestra.run.smithi052.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-06-10T03:04:00.334 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-06-10T03:04:00.334 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.334 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.334 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.334 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-10T03:04:00.334 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.334 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.334 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.335 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.335 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.335 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.335 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.335 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.335 INFO:teuthology.orchestra.run.smithi052.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-06-10T03:04:00.335 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-06-10T03:04:00.335 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.335 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.335 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.335 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-10T03:04:00.335 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.336 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.336 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.336 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.336 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.336 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.336 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.336 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.336 INFO:teuthology.orchestra.run.smithi052.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-06-10T03:04:00.336 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-06-10T03:04:00.336 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.336 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.336 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.336 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-10T03:04:00.337 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.337 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.337 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.337 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.337 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.337 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.337 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.337 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.337 INFO:teuthology.orchestra.run.smithi052.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-06-10T03:04:00.337 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-06-10T03:04:00.337 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.337 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.337 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.337 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-10T03:04:00.338 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.338 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.338 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.338 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.338 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.338 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.338 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.338 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.338 INFO:teuthology.orchestra.run.smithi052.stdout: "FEATURE_TOGGLE_NFS": { 2024-06-10T03:04:00.338 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-06-10T03:04:00.338 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.338 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.338 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.339 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-10T03:04:00.339 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.339 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.339 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.339 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.339 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.339 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.339 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.339 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.339 INFO:teuthology.orchestra.run.smithi052.stdout: "FEATURE_TOGGLE_RBD": { 2024-06-10T03:04:00.339 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-06-10T03:04:00.339 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.339 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.340 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.340 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-10T03:04:00.340 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.340 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.340 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.340 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.340 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.340 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.340 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.340 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.340 INFO:teuthology.orchestra.run.smithi052.stdout: "FEATURE_TOGGLE_RGW": { 2024-06-10T03:04:00.340 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-06-10T03:04:00.340 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.340 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.341 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.341 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-10T03:04:00.341 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.341 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.341 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.341 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.341 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.341 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.341 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.341 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.341 INFO:teuthology.orchestra.run.smithi052.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-06-10T03:04:00.341 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-06-10T03:04:00.341 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.342 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.342 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.342 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.342 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.342 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.342 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.342 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.342 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.342 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.342 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.342 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.342 INFO:teuthology.orchestra.run.smithi052.stdout: "GRAFANA_API_PASSWORD": { 2024-06-10T03:04:00.342 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "GRAFANA_API_PASSWORD", 2024-06-10T03:04:00.342 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.343 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.343 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.343 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "admin", 2024-06-10T03:04:00.343 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.343 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.343 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.343 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.343 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.343 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.343 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.343 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.343 INFO:teuthology.orchestra.run.smithi052.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-06-10T03:04:00.343 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-06-10T03:04:00.344 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.344 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.344 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.344 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-10T03:04:00.344 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.344 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.344 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.344 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.344 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.344 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.344 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.344 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.344 INFO:teuthology.orchestra.run.smithi052.stdout: "GRAFANA_API_URL": { 2024-06-10T03:04:00.344 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "GRAFANA_API_URL", 2024-06-10T03:04:00.345 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.345 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.345 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.345 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.345 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.345 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.345 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.345 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.345 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.345 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.345 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.345 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.345 INFO:teuthology.orchestra.run.smithi052.stdout: "GRAFANA_API_USERNAME": { 2024-06-10T03:04:00.346 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "GRAFANA_API_USERNAME", 2024-06-10T03:04:00.346 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.346 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.346 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.346 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "admin", 2024-06-10T03:04:00.346 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.346 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.346 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.346 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.346 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.346 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.346 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.346 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.347 INFO:teuthology.orchestra.run.smithi052.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-06-10T03:04:00.347 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-06-10T03:04:00.347 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.347 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.347 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.347 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.347 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.347 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.347 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.347 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.347 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.347 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.347 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.347 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.348 INFO:teuthology.orchestra.run.smithi052.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-06-10T03:04:00.348 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-06-10T03:04:00.348 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.348 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.348 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.348 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.348 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.348 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.348 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.348 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.348 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.348 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.349 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.349 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.349 INFO:teuthology.orchestra.run.smithi052.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-06-10T03:04:00.349 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-06-10T03:04:00.349 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.349 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.349 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.349 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-10T03:04:00.349 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.349 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.349 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.349 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.349 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.350 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.350 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.350 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.350 INFO:teuthology.orchestra.run.smithi052.stdout: "ISSUE_TRACKER_API_KEY": { 2024-06-10T03:04:00.350 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-06-10T03:04:00.350 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.350 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.350 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.350 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.350 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.350 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.350 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.350 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.351 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.351 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.351 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.351 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.351 INFO:teuthology.orchestra.run.smithi052.stdout: "PROMETHEUS_API_HOST": { 2024-06-10T03:04:00.351 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PROMETHEUS_API_HOST", 2024-06-10T03:04:00.351 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.351 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.351 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.351 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.351 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.351 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.351 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.352 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.352 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.352 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.352 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.352 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.352 INFO:teuthology.orchestra.run.smithi052.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-06-10T03:04:00.352 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-06-10T03:04:00.352 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.352 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.352 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.352 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-10T03:04:00.352 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.352 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.353 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.353 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.353 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.353 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.353 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.353 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.353 INFO:teuthology.orchestra.run.smithi052.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-06-10T03:04:00.353 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-06-10T03:04:00.353 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.353 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.353 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.353 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.353 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.354 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.354 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.354 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.354 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.354 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.354 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.354 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.354 INFO:teuthology.orchestra.run.smithi052.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-06-10T03:04:00.354 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-06-10T03:04:00.354 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.354 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.354 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.354 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.355 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.355 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.355 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.355 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.355 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.355 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.355 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.355 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.355 INFO:teuthology.orchestra.run.smithi052.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-06-10T03:04:00.355 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-06-10T03:04:00.355 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.355 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.355 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.356 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-10T03:04:00.356 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.356 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.356 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.356 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.356 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.356 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.356 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.356 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.356 INFO:teuthology.orchestra.run.smithi052.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-06-10T03:04:00.356 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-06-10T03:04:00.356 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.356 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.356 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.357 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-10T03:04:00.357 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.357 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.357 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.357 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.357 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.357 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.357 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.357 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.357 INFO:teuthology.orchestra.run.smithi052.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-06-10T03:04:00.357 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-06-10T03:04:00.357 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.357 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.357 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.358 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.358 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.358 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.358 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.358 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.358 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.358 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.358 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.358 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.358 INFO:teuthology.orchestra.run.smithi052.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-06-10T03:04:00.358 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-06-10T03:04:00.358 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.358 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.359 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.359 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.359 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.359 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.359 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.359 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.359 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.359 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.359 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.359 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.359 INFO:teuthology.orchestra.run.smithi052.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-06-10T03:04:00.359 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-06-10T03:04:00.359 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.360 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.360 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.360 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.360 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.360 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.360 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.360 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.360 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.360 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.360 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.360 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.360 INFO:teuthology.orchestra.run.smithi052.stdout: "PWD_POLICY_ENABLED": { 2024-06-10T03:04:00.360 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PWD_POLICY_ENABLED", 2024-06-10T03:04:00.360 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.361 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.361 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.361 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-10T03:04:00.361 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.361 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.361 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.361 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.361 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.361 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.361 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.361 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.361 INFO:teuthology.orchestra.run.smithi052.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-06-10T03:04:00.361 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-06-10T03:04:00.362 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.362 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.362 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.362 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-06-10T03:04:00.362 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.362 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.362 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.362 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.362 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.362 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.362 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.362 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.363 INFO:teuthology.orchestra.run.smithi052.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-06-10T03:04:00.363 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-06-10T03:04:00.363 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-10T03:04:00.363 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.363 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.363 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "10", 2024-06-10T03:04:00.363 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.363 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.363 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.363 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.363 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.363 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.363 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.364 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.364 INFO:teuthology.orchestra.run.smithi052.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-06-10T03:04:00.364 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-06-10T03:04:00.364 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-10T03:04:00.364 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.364 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.364 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "8", 2024-06-10T03:04:00.364 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.364 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.364 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.364 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.364 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.364 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.365 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.365 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.365 INFO:teuthology.orchestra.run.smithi052.stdout: "REST_REQUESTS_TIMEOUT": { 2024-06-10T03:04:00.365 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-06-10T03:04:00.365 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-10T03:04:00.365 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.365 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.365 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "45", 2024-06-10T03:04:00.365 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.365 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.365 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.365 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.365 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.365 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.366 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.366 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.366 INFO:teuthology.orchestra.run.smithi052.stdout: "RGW_API_ACCESS_KEY": { 2024-06-10T03:04:00.366 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "RGW_API_ACCESS_KEY", 2024-06-10T03:04:00.366 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.366 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.366 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.366 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.366 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.366 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.366 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.366 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.366 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.367 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.367 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.367 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.367 INFO:teuthology.orchestra.run.smithi052.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-06-10T03:04:00.367 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-06-10T03:04:00.367 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.367 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.367 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.367 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "admin", 2024-06-10T03:04:00.367 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.367 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.367 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.367 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.368 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.368 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.368 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.368 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.368 INFO:teuthology.orchestra.run.smithi052.stdout: "RGW_API_SECRET_KEY": { 2024-06-10T03:04:00.368 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "RGW_API_SECRET_KEY", 2024-06-10T03:04:00.368 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.368 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.368 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.368 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.368 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.368 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.368 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.369 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.369 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.369 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.369 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.369 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.369 INFO:teuthology.orchestra.run.smithi052.stdout: "RGW_API_SSL_VERIFY": { 2024-06-10T03:04:00.369 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "RGW_API_SSL_VERIFY", 2024-06-10T03:04:00.369 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.369 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.369 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.369 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-10T03:04:00.369 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.369 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.369 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.370 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.370 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.370 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.370 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.370 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.370 INFO:teuthology.orchestra.run.smithi052.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-06-10T03:04:00.370 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-06-10T03:04:00.370 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-10T03:04:00.370 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.370 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.370 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "0", 2024-06-10T03:04:00.370 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.370 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.371 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.371 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.371 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.371 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.371 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.371 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.371 INFO:teuthology.orchestra.run.smithi052.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-06-10T03:04:00.371 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-06-10T03:04:00.371 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-10T03:04:00.371 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.371 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.371 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "10", 2024-06-10T03:04:00.371 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.372 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.372 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.372 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.372 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.372 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.372 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.372 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.372 INFO:teuthology.orchestra.run.smithi052.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-06-10T03:04:00.372 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-06-10T03:04:00.372 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-10T03:04:00.372 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.372 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.372 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "5", 2024-06-10T03:04:00.373 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.373 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.373 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.373 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.373 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.373 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.373 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.373 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.373 INFO:teuthology.orchestra.run.smithi052.stdout: "cross_origin_url": { 2024-06-10T03:04:00.373 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "cross_origin_url", 2024-06-10T03:04:00.373 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.373 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.373 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.373 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.374 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.374 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.374 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.374 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.374 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.374 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.374 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.374 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.374 INFO:teuthology.orchestra.run.smithi052.stdout: "crt_file": { 2024-06-10T03:04:00.374 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "crt_file", 2024-06-10T03:04:00.374 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.374 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.374 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.375 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.375 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.375 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.375 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.375 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.375 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.375 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.375 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.375 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.375 INFO:teuthology.orchestra.run.smithi052.stdout: "debug": { 2024-06-10T03:04:00.375 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "debug", 2024-06-10T03:04:00.375 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.375 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.376 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.376 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.376 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.376 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.376 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.376 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Enable/disable debug options", 2024-06-10T03:04:00.376 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.376 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.376 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.376 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.376 INFO:teuthology.orchestra.run.smithi052.stdout: "jwt_token_ttl": { 2024-06-10T03:04:00.376 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "jwt_token_ttl", 2024-06-10T03:04:00.377 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-10T03:04:00.377 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.377 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.377 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "28800", 2024-06-10T03:04:00.377 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.377 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.377 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.377 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.377 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.377 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.377 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.377 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.377 INFO:teuthology.orchestra.run.smithi052.stdout: "key_file": { 2024-06-10T03:04:00.377 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "key_file", 2024-06-10T03:04:00.378 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.378 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.378 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.378 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.378 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.378 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.378 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.378 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.378 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.378 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.378 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.378 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.379 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-10T03:04:00.379 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-10T03:04:00.379 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.379 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.379 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.379 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.379 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.379 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.379 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:00.379 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:04:00.379 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:04:00.379 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:04:00.379 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:00.379 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:00.380 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:00.380 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:00.380 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.380 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.380 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.380 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.380 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.380 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-10T03:04:00.380 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-10T03:04:00.380 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.380 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.380 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.380 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.380 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.381 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.381 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.381 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.381 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.381 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.381 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.381 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.381 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-10T03:04:00.381 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-10T03:04:00.381 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.381 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.381 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.381 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-10T03:04:00.382 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.382 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.382 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:00.382 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:04:00.382 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:04:00.382 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:04:00.382 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:00.382 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:00.382 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:00.382 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:00.382 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.382 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.383 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.383 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.383 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.383 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-10T03:04:00.383 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-10T03:04:00.383 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.383 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.383 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.383 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.383 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.383 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.383 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.383 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.383 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.384 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.384 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.384 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.384 INFO:teuthology.orchestra.run.smithi052.stdout: "motd": { 2024-06-10T03:04:00.384 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "motd", 2024-06-10T03:04:00.384 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.384 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.384 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.384 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.384 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.384 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.384 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.384 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "The message of the day", 2024-06-10T03:04:00.385 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.385 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.385 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.385 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.385 INFO:teuthology.orchestra.run.smithi052.stdout: "redirect_resolve_ip_addr": { 2024-06-10T03:04:00.385 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "redirect_resolve_ip_addr", 2024-06-10T03:04:00.385 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.385 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.385 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.385 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.385 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.385 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.385 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.386 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.386 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.386 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.386 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.386 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.386 INFO:teuthology.orchestra.run.smithi052.stdout: "server_addr": { 2024-06-10T03:04:00.386 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "server_addr", 2024-06-10T03:04:00.386 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.386 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.386 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.386 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "::", 2024-06-10T03:04:00.386 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.386 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.387 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.387 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.387 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.387 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.387 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.387 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.387 INFO:teuthology.orchestra.run.smithi052.stdout: "server_port": { 2024-06-10T03:04:00.387 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "server_port", 2024-06-10T03:04:00.387 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-10T03:04:00.387 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.387 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.387 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "8080", 2024-06-10T03:04:00.387 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.387 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.388 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.388 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.388 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.388 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.388 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.388 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.388 INFO:teuthology.orchestra.run.smithi052.stdout: "ssl": { 2024-06-10T03:04:00.388 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "ssl", 2024-06-10T03:04:00.388 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.388 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.388 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.388 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-10T03:04:00.388 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.389 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.389 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.389 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.389 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.389 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.389 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.389 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.389 INFO:teuthology.orchestra.run.smithi052.stdout: "ssl_server_port": { 2024-06-10T03:04:00.389 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "ssl_server_port", 2024-06-10T03:04:00.389 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-10T03:04:00.389 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.389 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.389 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "8443", 2024-06-10T03:04:00.390 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.390 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.390 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.390 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.390 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.390 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.390 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.390 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.390 INFO:teuthology.orchestra.run.smithi052.stdout: "standby_behaviour": { 2024-06-10T03:04:00.390 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "standby_behaviour", 2024-06-10T03:04:00.390 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.390 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.390 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.391 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "redirect", 2024-06-10T03:04:00.391 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.391 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.391 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:00.391 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:00.391 INFO:teuthology.orchestra.run.smithi052.stdout: "redirect" 2024-06-10T03:04:00.391 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:00.391 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.391 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.391 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.391 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.391 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.391 INFO:teuthology.orchestra.run.smithi052.stdout: "standby_error_status_code": { 2024-06-10T03:04:00.392 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "standby_error_status_code", 2024-06-10T03:04:00.392 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-10T03:04:00.392 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.392 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.392 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "500", 2024-06-10T03:04:00.392 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "400", 2024-06-10T03:04:00.392 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "599", 2024-06-10T03:04:00.392 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.392 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.392 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.392 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.392 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.392 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.393 INFO:teuthology.orchestra.run.smithi052.stdout: "url_prefix": { 2024-06-10T03:04:00.393 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "url_prefix", 2024-06-10T03:04:00.393 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.393 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.393 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.393 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.393 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.393 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.393 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.393 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.393 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.393 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.393 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.393 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:00.394 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:00.394 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.394 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-10T03:04:00.394 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "devicehealth", 2024-06-10T03:04:00.394 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-10T03:04:00.394 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-10T03:04:00.394 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-10T03:04:00.394 INFO:teuthology.orchestra.run.smithi052.stdout: "enable_monitoring": { 2024-06-10T03:04:00.394 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "enable_monitoring", 2024-06-10T03:04:00.394 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.394 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.394 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.394 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-10T03:04:00.395 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.395 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.395 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.395 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "monitor device health metrics", 2024-06-10T03:04:00.395 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.395 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.395 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.395 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.395 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-10T03:04:00.395 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-10T03:04:00.395 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.395 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.395 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.396 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.396 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.396 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.396 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:00.396 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:04:00.396 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:04:00.396 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:04:00.396 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:00.396 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:00.396 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:00.396 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:00.396 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.396 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.397 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.397 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.397 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.397 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-10T03:04:00.397 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-10T03:04:00.397 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.397 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.397 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.397 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.397 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.397 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.397 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.397 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.398 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.398 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.398 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.398 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.398 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-10T03:04:00.398 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-10T03:04:00.398 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.398 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.398 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.398 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-10T03:04:00.398 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.398 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.398 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:00.399 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:04:00.399 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:04:00.399 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:04:00.399 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:00.399 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:00.399 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:00.399 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:00.399 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.399 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.399 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.399 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.399 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.399 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-10T03:04:00.399 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-10T03:04:00.400 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.400 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.400 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.400 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.400 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.400 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.400 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.400 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.400 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.400 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.400 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.400 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.400 INFO:teuthology.orchestra.run.smithi052.stdout: "mark_out_threshold": { 2024-06-10T03:04:00.401 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "mark_out_threshold", 2024-06-10T03:04:00.401 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-06-10T03:04:00.401 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.401 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.401 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "2419200", 2024-06-10T03:04:00.401 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.401 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.401 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.401 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-06-10T03:04:00.401 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.401 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.401 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.401 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.401 INFO:teuthology.orchestra.run.smithi052.stdout: "pool_name": { 2024-06-10T03:04:00.402 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "pool_name", 2024-06-10T03:04:00.402 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.402 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.402 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.402 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "device_health_metrics", 2024-06-10T03:04:00.402 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.402 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.402 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.402 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "name of pool in which to store device health metrics", 2024-06-10T03:04:00.402 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.402 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.402 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.402 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.403 INFO:teuthology.orchestra.run.smithi052.stdout: "retention_period": { 2024-06-10T03:04:00.403 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "retention_period", 2024-06-10T03:04:00.403 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-06-10T03:04:00.403 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.403 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.403 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "15552000", 2024-06-10T03:04:00.403 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.403 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.403 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.403 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "how long to retain device health metrics", 2024-06-10T03:04:00.403 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.403 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.403 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.403 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.404 INFO:teuthology.orchestra.run.smithi052.stdout: "scrape_frequency": { 2024-06-10T03:04:00.404 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "scrape_frequency", 2024-06-10T03:04:00.404 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-06-10T03:04:00.404 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.404 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.404 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "86400", 2024-06-10T03:04:00.404 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.404 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.404 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.404 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "how frequently to scrape device health metrics", 2024-06-10T03:04:00.404 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.404 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.404 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.404 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.405 INFO:teuthology.orchestra.run.smithi052.stdout: "self_heal": { 2024-06-10T03:04:00.405 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "self_heal", 2024-06-10T03:04:00.405 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.405 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.405 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.405 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-10T03:04:00.405 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.405 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.405 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.405 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-06-10T03:04:00.405 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.405 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.406 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.406 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.406 INFO:teuthology.orchestra.run.smithi052.stdout: "sleep_interval": { 2024-06-10T03:04:00.406 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "sleep_interval", 2024-06-10T03:04:00.406 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-06-10T03:04:00.406 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.406 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.406 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "600", 2024-06-10T03:04:00.406 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.406 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.406 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.406 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "how frequently to wake up and check device health", 2024-06-10T03:04:00.406 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.407 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.407 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.407 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.407 INFO:teuthology.orchestra.run.smithi052.stdout: "warn_threshold": { 2024-06-10T03:04:00.407 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "warn_threshold", 2024-06-10T03:04:00.407 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-06-10T03:04:00.407 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.407 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.407 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "7257600", 2024-06-10T03:04:00.407 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.407 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.407 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.407 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-06-10T03:04:00.407 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.408 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.408 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.408 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:00.408 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:00.408 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.408 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-10T03:04:00.408 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "diskprediction_local", 2024-06-10T03:04:00.408 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-10T03:04:00.408 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-10T03:04:00.408 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-10T03:04:00.408 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-10T03:04:00.408 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-10T03:04:00.408 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.408 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.409 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.409 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.409 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.409 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.409 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:00.409 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:04:00.409 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:04:00.409 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:04:00.409 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:00.409 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:00.409 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:00.409 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:00.409 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.410 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.410 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.410 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.410 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.410 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-10T03:04:00.410 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-10T03:04:00.410 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.410 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.410 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.410 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.410 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.410 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.410 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.410 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.411 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.411 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.411 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.411 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.411 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-10T03:04:00.411 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-10T03:04:00.411 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.411 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.411 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.411 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-10T03:04:00.411 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.411 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.411 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:00.412 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:04:00.412 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:04:00.412 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:04:00.412 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:00.412 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:00.412 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:00.412 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:00.412 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.412 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.412 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.412 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.412 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.412 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-10T03:04:00.412 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-10T03:04:00.413 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.413 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.413 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.413 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.413 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.413 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.413 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.413 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.413 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.413 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.413 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.413 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.413 INFO:teuthology.orchestra.run.smithi052.stdout: "predict_interval": { 2024-06-10T03:04:00.413 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "predict_interval", 2024-06-10T03:04:00.414 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.414 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.414 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.414 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "86400", 2024-06-10T03:04:00.414 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.414 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.414 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.414 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.414 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.414 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.414 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.414 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.415 INFO:teuthology.orchestra.run.smithi052.stdout: "predictor_model": { 2024-06-10T03:04:00.415 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "predictor_model", 2024-06-10T03:04:00.415 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.415 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.415 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.415 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "prophetstor", 2024-06-10T03:04:00.415 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.415 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.415 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.415 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.415 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.415 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.415 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.415 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.416 INFO:teuthology.orchestra.run.smithi052.stdout: "sleep_interval": { 2024-06-10T03:04:00.416 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "sleep_interval", 2024-06-10T03:04:00.416 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.416 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.416 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.416 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "600", 2024-06-10T03:04:00.416 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.416 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.416 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.416 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.416 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.416 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.416 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.417 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:00.417 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:00.417 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.417 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-10T03:04:00.417 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "influx", 2024-06-10T03:04:00.417 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": false, 2024-06-10T03:04:00.417 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "influxdb python module not found", 2024-06-10T03:04:00.417 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-10T03:04:00.417 INFO:teuthology.orchestra.run.smithi052.stdout: "batch_size": { 2024-06-10T03:04:00.417 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "batch_size", 2024-06-10T03:04:00.417 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-10T03:04:00.417 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.417 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.418 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "5000", 2024-06-10T03:04:00.418 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.418 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.418 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.418 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-06-10T03:04:00.418 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.418 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.418 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.418 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.418 INFO:teuthology.orchestra.run.smithi052.stdout: "database": { 2024-06-10T03:04:00.418 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "database", 2024-06-10T03:04:00.418 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.418 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.418 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.419 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "ceph", 2024-06-10T03:04:00.419 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.419 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.419 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.419 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-06-10T03:04:00.419 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.419 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.419 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.419 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.419 INFO:teuthology.orchestra.run.smithi052.stdout: "hostname": { 2024-06-10T03:04:00.419 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "hostname", 2024-06-10T03:04:00.419 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.420 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.420 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.420 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.420 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.420 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.420 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.420 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "InfluxDB server hostname", 2024-06-10T03:04:00.420 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.420 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.420 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.420 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.420 INFO:teuthology.orchestra.run.smithi052.stdout: "interval": { 2024-06-10T03:04:00.420 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "interval", 2024-06-10T03:04:00.421 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-06-10T03:04:00.421 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.421 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.421 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "30", 2024-06-10T03:04:00.421 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "5", 2024-06-10T03:04:00.421 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.421 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.421 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-06-10T03:04:00.421 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.421 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.421 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.421 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.421 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-10T03:04:00.422 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-10T03:04:00.422 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.422 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.422 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.422 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.422 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.422 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.422 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:00.422 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:04:00.422 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:04:00.422 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:04:00.422 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:00.422 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:00.423 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:00.423 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:00.423 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.423 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.423 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.423 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.423 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.423 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-10T03:04:00.423 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-10T03:04:00.423 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.423 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.423 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.423 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.423 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.424 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.424 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.424 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.424 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.424 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.424 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.424 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.424 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-10T03:04:00.424 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-10T03:04:00.424 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.424 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.424 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.424 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-10T03:04:00.425 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.425 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.425 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:00.425 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:04:00.425 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:04:00.425 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:04:00.425 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:00.425 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:00.425 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:00.425 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:00.425 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.425 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.425 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.425 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.426 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.426 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-10T03:04:00.426 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-10T03:04:00.426 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.426 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.426 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.426 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.426 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.426 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.426 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.426 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.426 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.426 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.427 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.427 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.427 INFO:teuthology.orchestra.run.smithi052.stdout: "password": { 2024-06-10T03:04:00.427 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "password", 2024-06-10T03:04:00.427 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.427 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.427 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.427 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.427 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.427 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.427 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.427 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "password of InfluxDB server user", 2024-06-10T03:04:00.427 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.428 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.428 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.428 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.428 INFO:teuthology.orchestra.run.smithi052.stdout: "port": { 2024-06-10T03:04:00.428 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "port", 2024-06-10T03:04:00.428 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-10T03:04:00.428 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.428 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.428 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "8086", 2024-06-10T03:04:00.428 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.428 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.428 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.428 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "InfluxDB server port", 2024-06-10T03:04:00.429 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.429 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.429 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.429 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.429 INFO:teuthology.orchestra.run.smithi052.stdout: "ssl": { 2024-06-10T03:04:00.429 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "ssl", 2024-06-10T03:04:00.429 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.429 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.429 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.429 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "false", 2024-06-10T03:04:00.429 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.429 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.429 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.429 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-06-10T03:04:00.430 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.430 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.430 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.430 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.430 INFO:teuthology.orchestra.run.smithi052.stdout: "threads": { 2024-06-10T03:04:00.430 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "threads", 2024-06-10T03:04:00.430 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-10T03:04:00.430 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.430 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.430 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "5", 2024-06-10T03:04:00.430 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "1", 2024-06-10T03:04:00.430 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "32", 2024-06-10T03:04:00.430 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.431 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-06-10T03:04:00.431 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.431 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.431 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.431 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.431 INFO:teuthology.orchestra.run.smithi052.stdout: "username": { 2024-06-10T03:04:00.431 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "username", 2024-06-10T03:04:00.431 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.431 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.431 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.431 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.431 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.431 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.432 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.432 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "username of InfluxDB server user", 2024-06-10T03:04:00.432 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.432 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.432 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.432 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.432 INFO:teuthology.orchestra.run.smithi052.stdout: "verify_ssl": { 2024-06-10T03:04:00.432 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "verify_ssl", 2024-06-10T03:04:00.432 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.432 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.432 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.432 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "true", 2024-06-10T03:04:00.432 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.432 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.433 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.433 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-06-10T03:04:00.433 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.433 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.433 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.433 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:00.433 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:00.433 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.433 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-10T03:04:00.433 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "insights", 2024-06-10T03:04:00.433 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-10T03:04:00.433 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-10T03:04:00.433 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-10T03:04:00.434 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-10T03:04:00.434 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-10T03:04:00.434 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.434 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.434 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.434 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.434 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.434 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.434 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:00.434 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:04:00.434 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:04:00.434 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:04:00.434 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:00.435 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:00.435 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:00.435 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:00.435 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.435 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.435 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.435 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.435 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.435 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-10T03:04:00.435 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-10T03:04:00.435 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.435 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.435 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.436 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.436 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.436 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.436 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.436 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.436 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.436 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.436 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.436 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.436 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-10T03:04:00.436 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-10T03:04:00.436 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.436 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.436 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.437 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-10T03:04:00.437 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.437 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.437 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:00.437 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:04:00.437 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:04:00.437 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:04:00.437 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:00.437 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:00.437 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:00.437 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:00.437 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.437 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.437 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.438 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.438 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.438 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-10T03:04:00.438 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-10T03:04:00.438 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.438 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.438 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.438 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.438 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.438 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.438 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.438 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.438 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.439 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.439 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.439 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:00.439 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:00.439 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.439 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-10T03:04:00.439 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "iostat", 2024-06-10T03:04:00.439 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-10T03:04:00.439 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-10T03:04:00.439 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-10T03:04:00.439 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-10T03:04:00.439 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-10T03:04:00.439 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.440 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.440 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.440 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.440 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.440 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.440 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:00.440 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:04:00.440 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:04:00.440 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:04:00.440 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:00.440 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:00.440 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:00.440 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:00.440 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.441 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.441 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.441 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.441 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.441 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-10T03:04:00.441 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-10T03:04:00.441 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.441 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.441 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.441 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.441 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.441 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.441 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.442 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.442 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.442 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.442 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.442 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.442 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-10T03:04:00.442 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-10T03:04:00.442 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.442 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.442 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.442 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-10T03:04:00.442 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.442 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.442 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:00.443 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:04:00.443 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:04:00.443 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:04:00.443 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:00.443 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:00.443 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:00.443 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:00.443 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.443 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.443 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.443 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.443 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.443 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-10T03:04:00.444 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-10T03:04:00.444 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.444 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.444 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.444 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.444 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.444 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.444 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.444 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.444 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.444 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.444 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.444 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:00.444 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:00.445 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.445 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-10T03:04:00.445 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "k8sevents", 2024-06-10T03:04:00.445 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-10T03:04:00.445 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-10T03:04:00.445 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-10T03:04:00.445 INFO:teuthology.orchestra.run.smithi052.stdout: "ceph_event_retention_days": { 2024-06-10T03:04:00.445 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "ceph_event_retention_days", 2024-06-10T03:04:00.445 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-10T03:04:00.445 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.445 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.445 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "7", 2024-06-10T03:04:00.445 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.446 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.446 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.446 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Days to hold ceph event information within local cache", 2024-06-10T03:04:00.446 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.446 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.446 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.446 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.446 INFO:teuthology.orchestra.run.smithi052.stdout: "config_check_secs": { 2024-06-10T03:04:00.446 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "config_check_secs", 2024-06-10T03:04:00.446 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-10T03:04:00.446 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.446 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.446 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "10", 2024-06-10T03:04:00.447 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "10", 2024-06-10T03:04:00.447 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.447 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.447 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-06-10T03:04:00.447 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.447 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.447 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.447 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.447 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-10T03:04:00.447 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-10T03:04:00.447 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.447 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.447 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.448 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.448 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.448 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.448 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:00.448 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:04:00.448 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:04:00.448 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:04:00.448 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:00.448 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:00.448 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:00.448 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:00.448 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.448 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.449 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.449 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.449 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.449 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-10T03:04:00.449 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-10T03:04:00.449 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.449 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.449 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.449 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.449 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.449 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.449 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.449 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.449 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.450 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.450 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.450 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.450 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-10T03:04:00.450 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-10T03:04:00.450 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.450 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.450 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.450 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-10T03:04:00.450 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.450 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.450 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:00.451 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:04:00.451 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:04:00.451 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:04:00.451 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:00.451 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:00.451 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:00.451 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:00.451 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.451 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.451 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.451 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.451 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.451 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-10T03:04:00.451 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-10T03:04:00.452 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.452 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.452 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.452 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.452 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.452 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.452 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.452 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.452 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.452 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.452 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.452 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:00.452 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:00.453 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.453 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-10T03:04:00.453 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "localpool", 2024-06-10T03:04:00.453 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-10T03:04:00.453 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-10T03:04:00.453 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-10T03:04:00.453 INFO:teuthology.orchestra.run.smithi052.stdout: "failure_domain": { 2024-06-10T03:04:00.453 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "failure_domain", 2024-06-10T03:04:00.453 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.453 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.453 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.453 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "host", 2024-06-10T03:04:00.453 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.453 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.454 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.454 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "failure domain for any created local pool", 2024-06-10T03:04:00.454 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-06-10T03:04:00.454 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.454 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.454 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.454 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-10T03:04:00.454 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-10T03:04:00.454 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.454 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.454 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.454 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.454 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.455 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.455 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:00.455 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:04:00.455 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:04:00.455 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:04:00.455 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:00.455 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:00.455 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:00.455 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:00.455 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.455 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.455 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.455 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.455 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.456 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-10T03:04:00.456 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-10T03:04:00.456 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.456 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.456 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.456 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.456 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.456 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.456 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.456 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.456 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.456 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.456 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.457 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.457 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-10T03:04:00.457 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-10T03:04:00.457 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.457 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.457 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.457 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-10T03:04:00.457 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.457 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.457 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:00.457 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:04:00.457 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:04:00.457 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:04:00.457 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:00.458 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:00.458 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:00.458 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:00.458 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.458 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.458 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.458 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.458 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.458 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-10T03:04:00.458 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-10T03:04:00.458 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.458 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.458 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.459 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.459 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.459 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.459 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.459 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.459 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.459 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.459 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.459 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.459 INFO:teuthology.orchestra.run.smithi052.stdout: "min_size": { 2024-06-10T03:04:00.459 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "min_size", 2024-06-10T03:04:00.459 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-10T03:04:00.459 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.460 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.460 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.460 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.460 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.460 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.460 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "default min_size for any created local pool", 2024-06-10T03:04:00.460 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-06-10T03:04:00.460 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.460 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.460 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.460 INFO:teuthology.orchestra.run.smithi052.stdout: "num_rep": { 2024-06-10T03:04:00.460 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "num_rep", 2024-06-10T03:04:00.460 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-10T03:04:00.460 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.461 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.461 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "3", 2024-06-10T03:04:00.461 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.461 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.461 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.461 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "default replica count for any created local pool", 2024-06-10T03:04:00.461 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.461 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.461 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.461 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.461 INFO:teuthology.orchestra.run.smithi052.stdout: "pg_num": { 2024-06-10T03:04:00.461 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "pg_num", 2024-06-10T03:04:00.461 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-10T03:04:00.462 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.462 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.462 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "128", 2024-06-10T03:04:00.462 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.462 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.462 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.462 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "default pg_num for any created local pool", 2024-06-10T03:04:00.462 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.462 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.463 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.463 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.463 INFO:teuthology.orchestra.run.smithi052.stdout: "prefix": { 2024-06-10T03:04:00.463 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "prefix", 2024-06-10T03:04:00.463 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.463 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.463 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.463 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.463 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.463 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.463 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.463 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "name prefix for any created local pool", 2024-06-10T03:04:00.463 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.464 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.464 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.464 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.464 INFO:teuthology.orchestra.run.smithi052.stdout: "subtree": { 2024-06-10T03:04:00.464 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "subtree", 2024-06-10T03:04:00.464 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.464 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.464 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.464 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "rack", 2024-06-10T03:04:00.464 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.464 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.464 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.464 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "CRUSH level for which to create a local pool", 2024-06-10T03:04:00.465 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-06-10T03:04:00.465 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.465 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.465 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:00.465 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:00.465 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.465 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-10T03:04:00.465 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "mds_autoscaler", 2024-06-10T03:04:00.465 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-10T03:04:00.465 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-10T03:04:00.465 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-10T03:04:00.465 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-10T03:04:00.465 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-10T03:04:00.466 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.466 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.466 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.466 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.466 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.466 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.466 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:00.466 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:04:00.466 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:04:00.466 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:04:00.466 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:00.466 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:00.466 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:00.467 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:00.467 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.467 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.467 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.467 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.467 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.467 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-10T03:04:00.467 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-10T03:04:00.467 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.467 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.467 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.467 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.467 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.468 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.468 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.468 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.468 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.468 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.468 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.468 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.468 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-10T03:04:00.468 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-10T03:04:00.468 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.468 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.468 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.468 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-10T03:04:00.469 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.469 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.469 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:00.469 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:04:00.469 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:04:00.469 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:04:00.469 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:00.469 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:00.469 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:00.469 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:00.469 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.469 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.469 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.469 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.470 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.470 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-10T03:04:00.470 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-10T03:04:00.470 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.470 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.470 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.470 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.470 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.470 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.470 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.470 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.470 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.470 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.471 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.471 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:00.471 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:00.471 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.471 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-10T03:04:00.471 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "mirroring", 2024-06-10T03:04:00.471 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-10T03:04:00.471 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-10T03:04:00.471 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-10T03:04:00.471 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-10T03:04:00.471 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-10T03:04:00.471 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.471 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.471 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.472 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.472 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.472 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.472 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:00.472 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:04:00.472 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:04:00.472 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:04:00.472 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:00.472 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:00.472 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:00.472 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:00.472 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.472 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.472 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.473 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.473 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.473 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-10T03:04:00.473 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-10T03:04:00.473 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.473 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.473 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.473 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.473 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.473 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.473 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.473 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.473 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.474 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.474 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.474 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.474 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-10T03:04:00.474 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-10T03:04:00.474 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.474 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.474 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.474 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-10T03:04:00.474 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.474 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.474 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:00.474 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:04:00.475 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:04:00.475 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:04:00.475 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:00.475 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:00.475 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:00.475 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:00.475 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.475 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.475 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.475 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.475 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.475 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-10T03:04:00.475 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-10T03:04:00.475 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.476 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.476 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.476 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.476 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.476 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.476 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.476 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.476 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.476 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.476 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.476 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:00.476 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:00.476 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.477 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-10T03:04:00.477 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "nfs", 2024-06-10T03:04:00.477 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-10T03:04:00.477 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-10T03:04:00.477 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-10T03:04:00.477 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-10T03:04:00.477 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-10T03:04:00.477 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.477 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.477 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.477 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.477 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.477 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.478 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:00.478 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:04:00.478 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:04:00.478 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:04:00.478 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:00.478 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:00.478 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:00.478 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:00.478 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.478 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.478 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.478 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.478 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.479 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-10T03:04:00.479 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-10T03:04:00.479 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.479 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.479 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.479 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.479 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.479 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.479 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.479 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.479 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.479 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.479 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.480 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.480 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-10T03:04:00.480 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-10T03:04:00.480 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.480 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.480 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.480 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-10T03:04:00.480 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.480 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.480 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:00.480 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:04:00.480 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:04:00.480 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:04:00.481 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:00.481 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:00.481 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:00.481 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:00.481 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.481 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.481 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.481 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.481 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.481 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-10T03:04:00.481 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-10T03:04:00.481 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.481 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.482 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.482 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.482 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.482 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.482 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.482 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.482 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.482 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.482 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.482 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:00.482 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:00.482 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.482 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-10T03:04:00.483 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "orchestrator", 2024-06-10T03:04:00.483 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-10T03:04:00.483 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-10T03:04:00.483 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-10T03:04:00.483 INFO:teuthology.orchestra.run.smithi052.stdout: "fail_fs": { 2024-06-10T03:04:00.483 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "fail_fs", 2024-06-10T03:04:00.483 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.483 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.483 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.483 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.483 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.483 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.483 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.484 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-06-10T03:04:00.484 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.484 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.484 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.484 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.484 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-10T03:04:00.484 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-10T03:04:00.484 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.484 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.484 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.484 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.484 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.484 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.485 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:00.485 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:04:00.485 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:04:00.485 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:04:00.485 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:00.485 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:00.485 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:00.485 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:00.485 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.485 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.485 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.485 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.485 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.485 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-10T03:04:00.486 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-10T03:04:00.486 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.486 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.486 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.486 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.486 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.486 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.486 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.486 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.486 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.486 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.486 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.487 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.487 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-10T03:04:00.487 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-10T03:04:00.487 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.487 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.487 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.487 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-10T03:04:00.487 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.487 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.487 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:00.487 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:04:00.487 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:04:00.487 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:04:00.487 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:00.488 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:00.488 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:00.488 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:00.488 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.488 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.488 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.488 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.488 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.488 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-10T03:04:00.488 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-10T03:04:00.488 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.488 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.488 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.488 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.489 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.489 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.489 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.489 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.489 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.489 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.489 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.489 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.489 INFO:teuthology.orchestra.run.smithi052.stdout: "orchestrator": { 2024-06-10T03:04:00.489 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "orchestrator", 2024-06-10T03:04:00.489 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.489 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.489 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.490 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.490 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.490 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.490 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:00.490 INFO:teuthology.orchestra.run.smithi052.stdout: "cephadm", 2024-06-10T03:04:00.490 INFO:teuthology.orchestra.run.smithi052.stdout: "rook", 2024-06-10T03:04:00.490 INFO:teuthology.orchestra.run.smithi052.stdout: "test_orchestrator" 2024-06-10T03:04:00.490 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:00.490 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Orchestrator backend", 2024-06-10T03:04:00.490 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.490 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.490 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.490 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:00.491 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:00.491 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.491 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-10T03:04:00.491 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "osd_perf_query", 2024-06-10T03:04:00.491 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-10T03:04:00.491 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-10T03:04:00.491 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-10T03:04:00.491 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-10T03:04:00.491 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-10T03:04:00.491 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.491 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.491 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.491 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.491 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.492 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.492 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:00.492 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:04:00.492 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:04:00.492 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:04:00.492 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:00.492 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:00.492 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:00.492 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:00.492 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.492 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.492 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.492 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.493 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.493 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-10T03:04:00.493 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-10T03:04:00.493 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.493 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.493 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.493 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.493 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.493 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.493 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.493 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.493 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.494 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.494 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.494 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.494 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-10T03:04:00.494 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-10T03:04:00.494 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.494 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.494 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.494 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-10T03:04:00.494 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.494 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.494 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:00.495 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:04:00.495 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:04:00.495 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:04:00.495 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:00.495 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:00.495 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:00.495 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:00.495 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.495 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.495 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.495 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.495 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.496 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-10T03:04:00.496 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-10T03:04:00.496 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.496 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.496 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.496 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.496 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.496 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.496 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.496 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.496 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.496 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.496 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.496 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:00.497 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:00.497 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.497 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-10T03:04:00.497 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "osd_support", 2024-06-10T03:04:00.497 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-10T03:04:00.497 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-10T03:04:00.497 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-10T03:04:00.497 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-10T03:04:00.497 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-10T03:04:00.497 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.497 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.497 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.498 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.498 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.498 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.498 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:00.498 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:04:00.498 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:04:00.498 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:04:00.498 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:00.498 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:00.498 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:00.498 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:00.498 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.498 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.499 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.499 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.499 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.499 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-10T03:04:00.499 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-10T03:04:00.499 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.499 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.499 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.499 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.499 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.499 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.499 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.499 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.500 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.500 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.500 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.500 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.500 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-10T03:04:00.500 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-10T03:04:00.500 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.500 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.500 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.500 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-10T03:04:00.500 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.500 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.500 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:00.501 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:04:00.501 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:04:00.501 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:04:00.501 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:00.501 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:00.501 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:00.501 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:00.501 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.501 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.501 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.501 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.501 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.501 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-10T03:04:00.502 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-10T03:04:00.502 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.502 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.502 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.502 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.502 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.502 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.502 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.502 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.502 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.502 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.502 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.502 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:00.503 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:00.503 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.503 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-10T03:04:00.503 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "pg_autoscaler", 2024-06-10T03:04:00.503 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-10T03:04:00.503 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-10T03:04:00.503 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-10T03:04:00.503 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-10T03:04:00.503 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-10T03:04:00.503 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.503 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.503 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.503 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.503 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.504 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.504 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:00.504 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:04:00.504 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:04:00.504 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:04:00.504 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:00.504 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:00.504 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:00.504 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:00.504 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.504 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.504 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.504 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.505 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.505 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-10T03:04:00.505 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-10T03:04:00.505 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.505 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.505 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.505 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.505 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.505 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.505 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.505 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.505 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.505 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.505 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.506 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.506 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-10T03:04:00.506 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-10T03:04:00.506 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.506 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.506 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.506 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-10T03:04:00.506 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.506 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.506 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:00.506 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:04:00.506 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:04:00.506 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:04:00.507 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:00.507 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:00.507 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:00.507 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:00.507 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.507 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.507 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.507 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.507 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.507 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-10T03:04:00.507 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-10T03:04:00.507 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.508 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.508 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.508 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.508 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.508 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.508 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.508 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.508 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.508 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.508 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.508 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.508 INFO:teuthology.orchestra.run.smithi052.stdout: "sleep_interval": { 2024-06-10T03:04:00.509 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "sleep_interval", 2024-06-10T03:04:00.509 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-06-10T03:04:00.509 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.509 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.509 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "60", 2024-06-10T03:04:00.509 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.509 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.509 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.509 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.509 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.509 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.509 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.509 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.510 INFO:teuthology.orchestra.run.smithi052.stdout: "threshold": { 2024-06-10T03:04:00.510 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "threshold", 2024-06-10T03:04:00.510 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "float", 2024-06-10T03:04:00.510 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.510 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.510 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "3.0", 2024-06-10T03:04:00.510 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "1.0", 2024-06-10T03:04:00.510 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.510 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.510 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "scaling threshold", 2024-06-10T03:04:00.510 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-06-10T03:04:00.510 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.511 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.511 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:00.511 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:00.511 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.511 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-10T03:04:00.511 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "progress", 2024-06-10T03:04:00.511 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-10T03:04:00.511 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-10T03:04:00.511 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-10T03:04:00.511 INFO:teuthology.orchestra.run.smithi052.stdout: "allow_pg_recovery_event": { 2024-06-10T03:04:00.511 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "allow_pg_recovery_event", 2024-06-10T03:04:00.511 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.512 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.512 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.512 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.512 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.512 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.512 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.512 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "allow the module to show pg recovery progress", 2024-06-10T03:04:00.512 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.512 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.512 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.512 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.512 INFO:teuthology.orchestra.run.smithi052.stdout: "enabled": { 2024-06-10T03:04:00.512 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "enabled", 2024-06-10T03:04:00.513 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.513 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.513 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.513 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-10T03:04:00.513 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.513 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.513 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.513 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.513 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.513 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.513 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.513 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.514 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-10T03:04:00.514 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-10T03:04:00.514 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.514 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.514 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.514 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.514 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.514 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.514 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:00.514 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:04:00.514 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:04:00.514 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:04:00.515 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:00.515 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:00.515 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:00.515 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:00.515 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.515 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.515 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.515 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.515 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.515 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-10T03:04:00.515 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-10T03:04:00.515 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.516 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.516 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.516 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.516 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.516 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.516 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.516 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.516 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.516 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.516 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.516 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.516 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-10T03:04:00.517 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-10T03:04:00.517 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.517 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.517 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.517 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-10T03:04:00.517 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.517 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.517 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:00.517 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:04:00.517 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:04:00.517 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:04:00.517 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:00.517 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:00.517 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:00.518 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:00.518 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.518 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.518 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.518 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.518 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.518 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-10T03:04:00.518 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-10T03:04:00.518 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.518 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.518 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.518 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.518 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.518 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.519 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.519 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.519 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.519 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.519 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.519 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.519 INFO:teuthology.orchestra.run.smithi052.stdout: "max_completed_events": { 2024-06-10T03:04:00.519 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "max_completed_events", 2024-06-10T03:04:00.519 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-10T03:04:00.519 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.519 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.519 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "50", 2024-06-10T03:04:00.519 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.519 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.520 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.520 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "number of past completed events to remember", 2024-06-10T03:04:00.520 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.520 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.520 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.520 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.520 INFO:teuthology.orchestra.run.smithi052.stdout: "sleep_interval": { 2024-06-10T03:04:00.520 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "sleep_interval", 2024-06-10T03:04:00.520 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-06-10T03:04:00.520 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.520 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.520 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "5", 2024-06-10T03:04:00.520 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.520 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.521 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.521 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "how long the module is going to sleep", 2024-06-10T03:04:00.521 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.521 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.521 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.521 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:00.521 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:00.521 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.521 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-10T03:04:00.521 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "prometheus", 2024-06-10T03:04:00.521 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-10T03:04:00.521 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-10T03:04:00.521 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-10T03:04:00.521 INFO:teuthology.orchestra.run.smithi052.stdout: "cache": { 2024-06-10T03:04:00.522 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "cache", 2024-06-10T03:04:00.522 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.522 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.522 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.522 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-10T03:04:00.522 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.522 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.522 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.526 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.526 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.526 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.526 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.526 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.526 INFO:teuthology.orchestra.run.smithi052.stdout: "exclude_perf_counters": { 2024-06-10T03:04:00.526 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "exclude_perf_counters", 2024-06-10T03:04:00.526 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.526 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.526 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.526 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-10T03:04:00.527 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.527 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.527 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.527 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-06-10T03:04:00.527 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-06-10T03:04:00.527 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.527 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.527 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.527 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-10T03:04:00.527 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-10T03:04:00.527 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.527 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.527 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.527 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.528 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.528 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.528 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:00.528 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:04:00.528 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:04:00.528 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:04:00.528 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:00.528 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:00.528 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:00.528 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:00.528 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.528 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.528 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.529 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.529 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.529 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-10T03:04:00.529 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-10T03:04:00.529 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.529 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.529 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.529 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.529 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.529 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.529 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.529 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.529 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.529 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.530 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.530 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.530 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-10T03:04:00.530 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-10T03:04:00.530 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.530 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.530 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.530 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-10T03:04:00.530 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.530 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.530 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:00.530 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:04:00.530 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:04:00.530 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:04:00.531 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:00.531 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:00.531 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:00.531 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:00.531 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.531 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.531 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.531 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.531 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.531 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-10T03:04:00.531 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-10T03:04:00.531 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.532 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.532 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.532 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.532 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.532 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.532 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.532 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.532 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.532 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.532 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.532 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.532 INFO:teuthology.orchestra.run.smithi052.stdout: "rbd_stats_pools": { 2024-06-10T03:04:00.532 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rbd_stats_pools", 2024-06-10T03:04:00.533 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.533 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.533 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.533 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.533 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.533 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.533 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.533 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.533 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.533 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.533 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.533 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.533 INFO:teuthology.orchestra.run.smithi052.stdout: "rbd_stats_pools_refresh_interval": { 2024-06-10T03:04:00.534 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-06-10T03:04:00.534 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-10T03:04:00.534 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.534 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.534 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "300", 2024-06-10T03:04:00.534 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.534 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.534 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.534 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.534 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.534 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.534 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.534 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.535 INFO:teuthology.orchestra.run.smithi052.stdout: "scrape_interval": { 2024-06-10T03:04:00.535 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "scrape_interval", 2024-06-10T03:04:00.535 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "float", 2024-06-10T03:04:00.535 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.535 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.535 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "15.0", 2024-06-10T03:04:00.535 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.535 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.535 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.535 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.535 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.535 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.536 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.536 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.536 INFO:teuthology.orchestra.run.smithi052.stdout: "server_addr": { 2024-06-10T03:04:00.536 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "server_addr", 2024-06-10T03:04:00.536 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.536 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.536 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.536 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "::", 2024-06-10T03:04:00.536 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.536 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.536 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.536 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-06-10T03:04:00.536 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.537 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.537 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.537 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.537 INFO:teuthology.orchestra.run.smithi052.stdout: "server_port": { 2024-06-10T03:04:00.537 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "server_port", 2024-06-10T03:04:00.537 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-10T03:04:00.537 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.537 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.537 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "9283", 2024-06-10T03:04:00.537 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.537 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.537 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.537 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-06-10T03:04:00.538 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.538 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.538 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.538 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.538 INFO:teuthology.orchestra.run.smithi052.stdout: "stale_cache_strategy": { 2024-06-10T03:04:00.538 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "stale_cache_strategy", 2024-06-10T03:04:00.538 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.538 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.538 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.538 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "log", 2024-06-10T03:04:00.538 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.538 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.538 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.539 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.539 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.539 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.539 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.539 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.539 INFO:teuthology.orchestra.run.smithi052.stdout: "standby_behaviour": { 2024-06-10T03:04:00.539 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "standby_behaviour", 2024-06-10T03:04:00.539 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.539 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.539 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.539 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "default", 2024-06-10T03:04:00.539 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.539 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.540 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:00.540 INFO:teuthology.orchestra.run.smithi052.stdout: "default", 2024-06-10T03:04:00.540 INFO:teuthology.orchestra.run.smithi052.stdout: "error" 2024-06-10T03:04:00.540 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:00.540 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.540 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.540 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.540 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.540 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.540 INFO:teuthology.orchestra.run.smithi052.stdout: "standby_error_status_code": { 2024-06-10T03:04:00.540 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "standby_error_status_code", 2024-06-10T03:04:00.540 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-10T03:04:00.540 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.541 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.541 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "500", 2024-06-10T03:04:00.541 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "400", 2024-06-10T03:04:00.541 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "599", 2024-06-10T03:04:00.541 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.541 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.541 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.541 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.541 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.541 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:00.541 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:00.541 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.541 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-10T03:04:00.541 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rbd_support", 2024-06-10T03:04:00.542 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-10T03:04:00.542 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-10T03:04:00.542 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-10T03:04:00.542 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-10T03:04:00.542 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-10T03:04:00.542 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.542 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.542 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.542 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.542 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.542 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.542 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:00.543 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:04:00.543 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:04:00.543 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:04:00.543 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:00.543 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:00.543 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:00.543 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:00.543 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.543 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.543 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.543 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.543 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.543 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-10T03:04:00.543 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-10T03:04:00.544 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.544 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.544 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.544 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.544 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.544 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.544 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.544 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.544 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.544 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.544 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.544 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.544 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-10T03:04:00.545 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-10T03:04:00.545 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.545 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.545 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.545 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-10T03:04:00.545 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.545 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.545 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:00.545 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:04:00.545 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:04:00.545 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:04:00.545 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:00.545 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:00.546 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:00.546 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:00.546 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.546 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.546 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.546 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.546 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.546 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-10T03:04:00.546 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-10T03:04:00.546 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.546 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.546 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.547 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.547 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.547 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.547 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.547 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.547 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.547 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.547 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.547 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.547 INFO:teuthology.orchestra.run.smithi052.stdout: "max_concurrent_snap_create": { 2024-06-10T03:04:00.547 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "max_concurrent_snap_create", 2024-06-10T03:04:00.547 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-10T03:04:00.547 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.548 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.548 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "10", 2024-06-10T03:04:00.548 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.548 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.548 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.548 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.548 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.548 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.548 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.548 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.548 INFO:teuthology.orchestra.run.smithi052.stdout: "mirror_snapshot_schedule": { 2024-06-10T03:04:00.548 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "mirror_snapshot_schedule", 2024-06-10T03:04:00.548 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.549 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.549 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.549 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.549 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.549 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.549 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.549 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.549 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.549 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.549 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.549 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.549 INFO:teuthology.orchestra.run.smithi052.stdout: "trash_purge_schedule": { 2024-06-10T03:04:00.549 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "trash_purge_schedule", 2024-06-10T03:04:00.550 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.550 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.550 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.550 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.550 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.550 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.550 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.550 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.550 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.550 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.550 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.550 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:00.550 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:00.551 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.551 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-10T03:04:00.551 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "restful", 2024-06-10T03:04:00.551 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-10T03:04:00.551 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-10T03:04:00.551 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-10T03:04:00.551 INFO:teuthology.orchestra.run.smithi052.stdout: "enable_auth": { 2024-06-10T03:04:00.551 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "enable_auth", 2024-06-10T03:04:00.551 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.551 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.551 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.551 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-10T03:04:00.551 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.551 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.552 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.552 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.552 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.552 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.552 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.552 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.552 INFO:teuthology.orchestra.run.smithi052.stdout: "key_file": { 2024-06-10T03:04:00.552 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "key_file", 2024-06-10T03:04:00.552 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.552 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.552 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.552 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.552 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.553 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.553 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.553 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.553 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.553 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.553 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.553 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.553 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-10T03:04:00.553 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-10T03:04:00.553 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.553 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.553 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.553 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.554 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.554 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.554 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:00.554 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:04:00.554 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:04:00.554 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:04:00.554 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:00.554 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:00.554 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:00.554 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:00.554 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.554 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.555 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.555 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.555 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.555 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-10T03:04:00.555 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-10T03:04:00.555 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.555 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.555 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.555 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.555 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.555 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.555 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.555 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.556 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.556 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.556 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.556 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.556 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-10T03:04:00.556 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-10T03:04:00.556 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.556 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.556 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.556 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-10T03:04:00.556 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.556 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.556 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:00.556 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:04:00.557 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:04:00.557 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:04:00.557 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:00.557 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:00.557 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:00.557 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:00.557 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.557 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.557 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.557 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.557 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.557 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-10T03:04:00.557 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-10T03:04:00.557 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.558 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.558 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.558 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.558 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.558 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.558 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.558 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.558 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.558 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.558 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.558 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.558 INFO:teuthology.orchestra.run.smithi052.stdout: "server_addr": { 2024-06-10T03:04:00.559 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "server_addr", 2024-06-10T03:04:00.559 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.559 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.559 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.559 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.559 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.559 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.559 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.559 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.559 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.559 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.559 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.559 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.559 INFO:teuthology.orchestra.run.smithi052.stdout: "server_port": { 2024-06-10T03:04:00.560 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "server_port", 2024-06-10T03:04:00.560 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.560 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.560 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.560 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.560 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.560 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.560 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.560 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.560 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.560 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.560 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.560 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:00.560 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:00.561 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.561 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-10T03:04:00.561 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rgw", 2024-06-10T03:04:00.561 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-10T03:04:00.561 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-10T03:04:00.561 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-10T03:04:00.561 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-10T03:04:00.561 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-10T03:04:00.561 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.561 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.561 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.561 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.561 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.562 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.562 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:00.562 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:04:00.562 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:04:00.562 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:04:00.562 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:00.562 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:00.562 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:00.562 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:00.562 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.562 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.562 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.563 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.563 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.563 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-10T03:04:00.563 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-10T03:04:00.563 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.563 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.563 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.563 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.563 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.563 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.563 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.563 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.563 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.564 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.564 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.564 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.564 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-10T03:04:00.564 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-10T03:04:00.564 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.564 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.564 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.564 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-10T03:04:00.564 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.564 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.564 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:00.564 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:04:00.564 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:04:00.565 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:04:00.565 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:00.565 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:00.565 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:00.565 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:00.565 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.565 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.565 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.565 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.565 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.565 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-10T03:04:00.565 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-10T03:04:00.565 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.565 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.566 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.566 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.566 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.566 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.566 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.566 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.566 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.566 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.566 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.566 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:00.566 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:00.566 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.567 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-10T03:04:00.567 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rook", 2024-06-10T03:04:00.567 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-10T03:04:00.567 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-10T03:04:00.567 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-10T03:04:00.567 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-10T03:04:00.567 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-10T03:04:00.567 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.567 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.567 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.567 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.567 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.567 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.567 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:00.568 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:04:00.568 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:04:00.568 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:04:00.568 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:00.568 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:00.568 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:00.568 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:00.568 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.568 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.568 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.568 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.568 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.568 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-10T03:04:00.569 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-10T03:04:00.569 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.569 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.569 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.569 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.569 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.569 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.569 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.569 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.569 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.569 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.569 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.569 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.570 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-10T03:04:00.570 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-10T03:04:00.570 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.570 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.570 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.570 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-10T03:04:00.570 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.570 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.570 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:00.570 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:04:00.570 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:04:00.570 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:04:00.571 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:00.571 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:00.571 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:00.571 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:00.571 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.571 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.571 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.571 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.571 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.571 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-10T03:04:00.571 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-10T03:04:00.571 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.572 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.572 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.572 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.572 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.572 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.572 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.572 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.572 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.572 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.572 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.572 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.572 INFO:teuthology.orchestra.run.smithi052.stdout: "storage_class": { 2024-06-10T03:04:00.572 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "storage_class", 2024-06-10T03:04:00.573 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.573 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.573 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.573 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "local", 2024-06-10T03:04:00.573 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.573 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.573 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.573 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-06-10T03:04:00.573 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.573 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.573 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.573 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:00.573 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:00.574 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.574 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-10T03:04:00.574 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "selftest", 2024-06-10T03:04:00.574 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-10T03:04:00.574 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-10T03:04:00.574 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-10T03:04:00.574 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-10T03:04:00.574 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-10T03:04:00.574 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.574 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.574 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.574 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.575 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.575 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.575 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:00.575 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:04:00.575 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:04:00.575 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:04:00.575 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:00.575 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:00.575 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:00.575 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:00.575 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.575 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.575 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.576 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.576 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.576 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-10T03:04:00.576 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-10T03:04:00.576 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.576 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.576 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.576 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.576 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.576 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.576 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.576 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.576 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.577 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.577 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.577 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.577 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-10T03:04:00.577 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-10T03:04:00.577 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.577 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.577 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.577 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-10T03:04:00.577 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.577 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.577 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:00.577 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:04:00.577 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:04:00.578 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:04:00.578 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:00.578 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:00.578 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:00.578 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:00.578 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.578 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.578 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.578 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.578 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.578 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-10T03:04:00.578 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-10T03:04:00.578 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.579 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.579 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.579 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.579 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.579 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.579 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.579 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.579 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.579 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.579 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.579 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.579 INFO:teuthology.orchestra.run.smithi052.stdout: "roption1": { 2024-06-10T03:04:00.579 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "roption1", 2024-06-10T03:04:00.580 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.580 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.580 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.580 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.580 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.580 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.580 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.580 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.580 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.580 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.580 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.580 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.580 INFO:teuthology.orchestra.run.smithi052.stdout: "roption2": { 2024-06-10T03:04:00.581 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "roption2", 2024-06-10T03:04:00.581 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.581 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.581 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.581 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "xyz", 2024-06-10T03:04:00.581 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.581 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.581 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.581 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.581 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.581 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.581 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.581 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.582 INFO:teuthology.orchestra.run.smithi052.stdout: "rwoption1": { 2024-06-10T03:04:00.582 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rwoption1", 2024-06-10T03:04:00.582 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.582 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.582 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.582 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.582 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.582 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.582 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.582 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.582 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.582 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.582 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.583 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.583 INFO:teuthology.orchestra.run.smithi052.stdout: "rwoption2": { 2024-06-10T03:04:00.583 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rwoption2", 2024-06-10T03:04:00.583 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-10T03:04:00.583 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.583 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.583 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.583 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.583 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.583 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.583 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.583 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.583 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.583 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.584 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.584 INFO:teuthology.orchestra.run.smithi052.stdout: "rwoption3": { 2024-06-10T03:04:00.584 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rwoption3", 2024-06-10T03:04:00.584 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "float", 2024-06-10T03:04:00.584 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.584 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.584 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.584 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.584 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.584 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.584 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.584 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.585 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.585 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.585 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.585 INFO:teuthology.orchestra.run.smithi052.stdout: "rwoption4": { 2024-06-10T03:04:00.585 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rwoption4", 2024-06-10T03:04:00.585 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.585 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.585 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.585 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.585 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.585 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.585 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.586 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.586 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.586 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.586 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.586 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.586 INFO:teuthology.orchestra.run.smithi052.stdout: "rwoption5": { 2024-06-10T03:04:00.586 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rwoption5", 2024-06-10T03:04:00.586 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.586 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.586 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.586 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.586 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.587 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.587 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.587 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.587 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.587 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.587 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.587 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.587 INFO:teuthology.orchestra.run.smithi052.stdout: "rwoption6": { 2024-06-10T03:04:00.587 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rwoption6", 2024-06-10T03:04:00.587 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.587 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.587 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.588 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-10T03:04:00.588 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.588 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.588 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.588 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.588 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.588 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.588 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.588 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.588 INFO:teuthology.orchestra.run.smithi052.stdout: "rwoption7": { 2024-06-10T03:04:00.588 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rwoption7", 2024-06-10T03:04:00.588 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-10T03:04:00.588 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.588 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.589 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.589 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "1", 2024-06-10T03:04:00.589 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "42", 2024-06-10T03:04:00.589 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.589 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.589 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.589 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.589 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.589 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.589 INFO:teuthology.orchestra.run.smithi052.stdout: "testkey": { 2024-06-10T03:04:00.589 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "testkey", 2024-06-10T03:04:00.589 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.589 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.590 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.590 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.590 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.590 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.590 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.590 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.590 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.590 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.590 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.590 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.590 INFO:teuthology.orchestra.run.smithi052.stdout: "testlkey": { 2024-06-10T03:04:00.590 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "testlkey", 2024-06-10T03:04:00.591 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.591 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.591 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.591 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.591 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.591 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.591 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.591 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.591 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.591 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.591 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.591 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.591 INFO:teuthology.orchestra.run.smithi052.stdout: "testnewline": { 2024-06-10T03:04:00.591 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "testnewline", 2024-06-10T03:04:00.592 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.592 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.592 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.592 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.592 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.592 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.592 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.592 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.592 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.592 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.592 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.592 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:00.592 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:00.592 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.593 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-10T03:04:00.593 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "snap_schedule", 2024-06-10T03:04:00.593 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-10T03:04:00.593 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-10T03:04:00.593 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-10T03:04:00.593 INFO:teuthology.orchestra.run.smithi052.stdout: "allow_m_granularity": { 2024-06-10T03:04:00.593 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "allow_m_granularity", 2024-06-10T03:04:00.593 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.593 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.593 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.593 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.593 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.593 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.594 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.594 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "allow minute scheduled snapshots", 2024-06-10T03:04:00.594 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.594 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.594 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.594 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.594 INFO:teuthology.orchestra.run.smithi052.stdout: "dump_on_update": { 2024-06-10T03:04:00.594 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "dump_on_update", 2024-06-10T03:04:00.594 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.594 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.594 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.594 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.595 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.595 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.595 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.595 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "dump database to debug log on update", 2024-06-10T03:04:00.595 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.595 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.595 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.595 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.595 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-10T03:04:00.595 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-10T03:04:00.595 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.595 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.595 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.595 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.596 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.596 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.596 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:00.596 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:04:00.596 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:04:00.596 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:04:00.596 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:00.596 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:00.596 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:00.596 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:00.596 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.596 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.597 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.597 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.597 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.597 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-10T03:04:00.597 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-10T03:04:00.597 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.597 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.597 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.597 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.597 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.597 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.597 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.597 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.598 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.598 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.598 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.598 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.598 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-10T03:04:00.598 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-10T03:04:00.598 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.598 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.598 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.598 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-10T03:04:00.598 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.598 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.598 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:00.599 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:04:00.599 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:04:00.599 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:04:00.599 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:00.599 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:00.599 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:00.599 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:00.599 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.599 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.599 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.599 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.599 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.600 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-10T03:04:00.600 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-10T03:04:00.600 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.600 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.600 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.600 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.600 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.600 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.600 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.600 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.600 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.600 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.601 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.601 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:00.601 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:00.601 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.601 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-10T03:04:00.601 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "stats", 2024-06-10T03:04:00.601 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-10T03:04:00.601 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-10T03:04:00.601 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-10T03:04:00.601 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-10T03:04:00.601 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-10T03:04:00.601 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.601 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.602 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.602 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.602 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.602 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.602 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:00.602 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:04:00.602 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:04:00.602 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:04:00.602 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:00.602 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:00.602 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:00.602 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:00.603 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.603 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.603 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.603 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.603 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.603 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-10T03:04:00.603 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-10T03:04:00.603 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.603 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.603 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.603 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.603 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.603 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.603 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.604 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.604 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.604 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.604 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.604 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.604 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-10T03:04:00.604 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-10T03:04:00.604 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.604 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.604 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.604 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-10T03:04:00.604 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.604 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.605 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:00.605 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:04:00.605 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:04:00.605 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:04:00.605 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:00.605 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:00.605 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:00.605 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:00.605 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.605 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.605 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.605 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.605 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.605 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-10T03:04:00.606 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-10T03:04:00.606 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.606 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.606 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.606 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.606 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.606 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.606 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.606 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.606 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.606 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.606 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.606 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:00.607 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:00.607 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.607 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-10T03:04:00.607 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "status", 2024-06-10T03:04:00.607 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-10T03:04:00.607 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-10T03:04:00.607 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-10T03:04:00.607 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-10T03:04:00.607 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-10T03:04:00.607 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.607 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.607 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.607 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.607 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.608 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.608 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:00.608 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:04:00.608 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:04:00.608 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:04:00.608 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:00.608 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:00.608 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:00.608 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:00.608 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.608 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.608 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.608 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.608 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.609 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-10T03:04:00.609 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-10T03:04:00.609 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.609 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.609 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.609 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.609 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.609 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.609 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.609 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.609 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.609 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.609 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.609 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.609 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-10T03:04:00.610 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-10T03:04:00.610 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.610 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.610 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.610 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-10T03:04:00.610 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.610 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.610 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:00.610 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:04:00.610 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:04:00.610 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:04:00.610 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:00.610 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:00.611 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:00.611 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:00.611 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.611 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.611 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.611 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.611 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.611 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-10T03:04:00.611 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-10T03:04:00.611 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.611 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.611 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.611 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.611 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.612 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.612 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.612 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.612 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.612 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.612 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.612 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:00.612 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:00.612 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.612 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-10T03:04:00.612 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "telegraf", 2024-06-10T03:04:00.612 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-10T03:04:00.612 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-10T03:04:00.613 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-10T03:04:00.613 INFO:teuthology.orchestra.run.smithi052.stdout: "address": { 2024-06-10T03:04:00.613 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "address", 2024-06-10T03:04:00.613 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.613 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.613 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.613 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-06-10T03:04:00.613 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.613 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.613 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.613 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.613 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.613 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.614 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.614 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.614 INFO:teuthology.orchestra.run.smithi052.stdout: "interval": { 2024-06-10T03:04:00.614 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "interval", 2024-06-10T03:04:00.614 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-06-10T03:04:00.614 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.614 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.614 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "15", 2024-06-10T03:04:00.614 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.614 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.614 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.614 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.614 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.615 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.615 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.615 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.615 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-10T03:04:00.615 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-10T03:04:00.615 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.615 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.615 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.615 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.615 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.615 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.615 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:00.615 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:04:00.616 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:04:00.616 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:04:00.616 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:00.616 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:00.616 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:00.616 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:00.616 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.616 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.616 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.616 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.616 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.616 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-10T03:04:00.616 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-10T03:04:00.617 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.617 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.617 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.617 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.617 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.617 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.617 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.617 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.617 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.617 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.617 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.617 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.617 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-10T03:04:00.617 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-10T03:04:00.618 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.618 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.618 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.618 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-10T03:04:00.618 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.618 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.618 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:00.618 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:04:00.618 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:04:00.618 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:04:00.618 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:00.618 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:00.618 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:00.619 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:00.619 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.619 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.619 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.619 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.619 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.619 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-10T03:04:00.619 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-10T03:04:00.619 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.619 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.619 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.619 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.619 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.619 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.620 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.620 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.620 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.620 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.620 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.620 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:00.620 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:00.620 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.620 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-10T03:04:00.620 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "telemetry", 2024-06-10T03:04:00.620 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-10T03:04:00.620 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-10T03:04:00.620 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-10T03:04:00.620 INFO:teuthology.orchestra.run.smithi052.stdout: "channel_basic": { 2024-06-10T03:04:00.621 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "channel_basic", 2024-06-10T03:04:00.621 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.621 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.621 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.621 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-10T03:04:00.621 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.621 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.621 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.621 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Share basic cluster information (size, version)", 2024-06-10T03:04:00.621 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.621 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.621 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.621 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.621 INFO:teuthology.orchestra.run.smithi052.stdout: "channel_crash": { 2024-06-10T03:04:00.621 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "channel_crash", 2024-06-10T03:04:00.622 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.622 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.622 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.622 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-10T03:04:00.622 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.622 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.622 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.622 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-06-10T03:04:00.622 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.622 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.622 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.622 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.622 INFO:teuthology.orchestra.run.smithi052.stdout: "channel_device": { 2024-06-10T03:04:00.623 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "channel_device", 2024-06-10T03:04:00.623 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.623 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.623 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.623 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-10T03:04:00.623 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.623 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.623 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.623 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-06-10T03:04:00.623 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.623 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.623 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.623 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.624 INFO:teuthology.orchestra.run.smithi052.stdout: "channel_ident": { 2024-06-10T03:04:00.624 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "channel_ident", 2024-06-10T03:04:00.624 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.624 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.624 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.624 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.624 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.624 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.624 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.624 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-06-10T03:04:00.624 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.624 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.624 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.624 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.625 INFO:teuthology.orchestra.run.smithi052.stdout: "channel_perf": { 2024-06-10T03:04:00.625 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "channel_perf", 2024-06-10T03:04:00.625 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.625 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.625 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.625 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.625 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.625 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.625 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.625 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Share various performance metrics of a cluster", 2024-06-10T03:04:00.625 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.625 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.625 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.625 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.626 INFO:teuthology.orchestra.run.smithi052.stdout: "contact": { 2024-06-10T03:04:00.626 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "contact", 2024-06-10T03:04:00.626 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.626 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.626 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.626 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.626 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.626 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.626 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.626 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.626 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.626 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.626 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.627 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.627 INFO:teuthology.orchestra.run.smithi052.stdout: "description": { 2024-06-10T03:04:00.627 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "description", 2024-06-10T03:04:00.627 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.627 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.627 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.627 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.627 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.627 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.627 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.627 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.627 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.627 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.628 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.628 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.628 INFO:teuthology.orchestra.run.smithi052.stdout: "device_url": { 2024-06-10T03:04:00.628 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "device_url", 2024-06-10T03:04:00.628 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.628 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.628 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.628 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-06-10T03:04:00.628 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.628 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.628 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.628 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.629 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.629 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.629 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.629 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.629 INFO:teuthology.orchestra.run.smithi052.stdout: "enabled": { 2024-06-10T03:04:00.629 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "enabled", 2024-06-10T03:04:00.629 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.629 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.629 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.629 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.629 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.629 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.629 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.629 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.630 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.630 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.630 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.630 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.630 INFO:teuthology.orchestra.run.smithi052.stdout: "interval": { 2024-06-10T03:04:00.630 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "interval", 2024-06-10T03:04:00.630 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-10T03:04:00.630 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.630 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.630 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "24", 2024-06-10T03:04:00.630 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "8", 2024-06-10T03:04:00.630 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.630 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.631 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.631 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.631 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.631 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.631 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.631 INFO:teuthology.orchestra.run.smithi052.stdout: "last_opt_revision": { 2024-06-10T03:04:00.631 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "last_opt_revision", 2024-06-10T03:04:00.631 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-10T03:04:00.631 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.631 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.631 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "1", 2024-06-10T03:04:00.631 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.631 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.632 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.632 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.632 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.632 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.632 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.632 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.632 INFO:teuthology.orchestra.run.smithi052.stdout: "leaderboard": { 2024-06-10T03:04:00.632 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "leaderboard", 2024-06-10T03:04:00.632 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.632 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.632 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.632 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.632 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.633 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.633 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.633 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.633 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.633 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.633 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.633 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.633 INFO:teuthology.orchestra.run.smithi052.stdout: "leaderboard_description": { 2024-06-10T03:04:00.633 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "leaderboard_description", 2024-06-10T03:04:00.633 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.633 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.633 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.633 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.634 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.634 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.634 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.634 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.634 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.634 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.634 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.634 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.634 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-10T03:04:00.634 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-10T03:04:00.634 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.634 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.634 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.635 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.635 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.635 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.635 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:00.635 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:04:00.635 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:04:00.635 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:04:00.635 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:00.635 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:00.635 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:00.635 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:00.635 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.635 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.635 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.636 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.636 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.636 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-10T03:04:00.636 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-10T03:04:00.636 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.636 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.636 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.636 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.636 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.636 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.636 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.636 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.636 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.637 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.637 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.637 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.637 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-10T03:04:00.637 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-10T03:04:00.637 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.637 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.637 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.637 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-10T03:04:00.637 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.637 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.637 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:00.637 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:04:00.638 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:04:00.638 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:04:00.638 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:00.638 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:00.638 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:00.638 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:00.638 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.638 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.638 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.638 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.638 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.638 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-10T03:04:00.638 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-10T03:04:00.639 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.639 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.639 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.639 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.639 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.639 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.639 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.639 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.639 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.639 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.639 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.639 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.639 INFO:teuthology.orchestra.run.smithi052.stdout: "organization": { 2024-06-10T03:04:00.640 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "organization", 2024-06-10T03:04:00.640 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.640 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.640 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.640 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.640 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.640 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.640 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.640 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.640 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.640 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.640 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.640 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.640 INFO:teuthology.orchestra.run.smithi052.stdout: "proxy": { 2024-06-10T03:04:00.641 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "proxy", 2024-06-10T03:04:00.641 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.641 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.641 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.641 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.641 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.641 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.641 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.641 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.641 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.641 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.641 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.641 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.641 INFO:teuthology.orchestra.run.smithi052.stdout: "url": { 2024-06-10T03:04:00.642 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "url", 2024-06-10T03:04:00.642 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.642 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.642 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.642 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-06-10T03:04:00.642 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.642 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.642 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.642 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.642 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.642 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.642 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.643 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:00.643 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:00.643 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.643 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-10T03:04:00.643 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "test_orchestrator", 2024-06-10T03:04:00.643 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-10T03:04:00.643 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-10T03:04:00.643 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-10T03:04:00.643 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-10T03:04:00.643 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-10T03:04:00.643 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.643 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.643 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.643 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.644 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.644 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.644 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:00.644 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:04:00.644 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:04:00.644 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:04:00.644 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:00.644 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:00.644 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:00.644 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:00.644 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.644 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.645 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.645 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.645 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.645 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-10T03:04:00.645 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-10T03:04:00.645 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.645 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.645 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.645 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.645 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.645 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.645 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.645 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.646 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.646 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.646 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.646 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.646 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-10T03:04:00.646 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-10T03:04:00.646 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.646 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.646 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.646 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-10T03:04:00.646 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.646 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.647 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:00.647 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:04:00.647 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:04:00.647 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:04:00.647 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:00.647 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:00.647 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:00.647 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:00.647 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.647 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.647 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.647 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.647 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.647 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-10T03:04:00.648 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-10T03:04:00.648 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.648 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.648 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.648 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.648 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.648 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.648 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.648 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.648 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.648 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.648 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.648 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:00.648 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:00.649 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.649 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-10T03:04:00.649 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "volumes", 2024-06-10T03:04:00.649 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-10T03:04:00.649 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-10T03:04:00.649 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-10T03:04:00.649 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-10T03:04:00.649 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-10T03:04:00.649 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.649 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.649 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.649 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.649 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.650 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.650 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:00.650 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:04:00.650 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:04:00.650 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:04:00.650 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:00.650 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:00.650 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:00.650 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:00.650 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.650 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.650 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.650 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.651 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.651 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-10T03:04:00.651 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-10T03:04:00.651 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.651 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.651 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.651 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.651 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.651 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.651 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.651 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.651 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.651 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.651 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.651 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.652 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-10T03:04:00.652 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-10T03:04:00.652 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.652 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.652 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.652 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-10T03:04:00.652 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.652 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.652 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:00.652 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:04:00.652 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:04:00.652 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:04:00.652 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:00.653 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:00.653 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:00.653 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:00.653 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.653 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.653 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.653 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.653 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.653 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-10T03:04:00.653 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-10T03:04:00.653 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.653 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.653 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.653 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.654 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.654 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.654 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.654 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.654 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.654 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.654 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.654 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.654 INFO:teuthology.orchestra.run.smithi052.stdout: "max_concurrent_clones": { 2024-06-10T03:04:00.654 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "max_concurrent_clones", 2024-06-10T03:04:00.654 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-10T03:04:00.654 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.654 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.655 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "4", 2024-06-10T03:04:00.655 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.655 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.655 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.655 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Number of asynchronous cloner threads", 2024-06-10T03:04:00.655 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.655 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.655 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.655 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.655 INFO:teuthology.orchestra.run.smithi052.stdout: "periodic_async_work": { 2024-06-10T03:04:00.655 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "periodic_async_work", 2024-06-10T03:04:00.655 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.655 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.656 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.656 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.656 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.656 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.656 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.656 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Periodically check for async work", 2024-06-10T03:04:00.656 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.656 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.656 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.656 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.656 INFO:teuthology.orchestra.run.smithi052.stdout: "snapshot_clone_delay": { 2024-06-10T03:04:00.656 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "snapshot_clone_delay", 2024-06-10T03:04:00.656 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-10T03:04:00.657 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.657 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.657 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "0", 2024-06-10T03:04:00.657 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.657 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.657 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.657 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-06-10T03:04:00.657 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.657 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.657 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.657 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.657 INFO:teuthology.orchestra.run.smithi052.stdout: "snapshot_clone_no_wait": { 2024-06-10T03:04:00.657 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "snapshot_clone_no_wait", 2024-06-10T03:04:00.658 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.658 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.658 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.658 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-10T03:04:00.658 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.658 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.658 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.658 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-06-10T03:04:00.658 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.658 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.658 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.658 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:00.658 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:00.659 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.659 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-10T03:04:00.659 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "zabbix", 2024-06-10T03:04:00.659 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-10T03:04:00.659 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-10T03:04:00.659 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-10T03:04:00.659 INFO:teuthology.orchestra.run.smithi052.stdout: "discovery_interval": { 2024-06-10T03:04:00.659 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "discovery_interval", 2024-06-10T03:04:00.659 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "uint", 2024-06-10T03:04:00.659 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.659 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.659 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "100", 2024-06-10T03:04:00.660 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.660 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.660 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.660 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.660 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.660 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.660 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.660 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.660 INFO:teuthology.orchestra.run.smithi052.stdout: "identifier": { 2024-06-10T03:04:00.660 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "identifier", 2024-06-10T03:04:00.660 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.660 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.660 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.661 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.661 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.661 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.661 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.661 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.661 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.661 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.661 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.661 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.661 INFO:teuthology.orchestra.run.smithi052.stdout: "interval": { 2024-06-10T03:04:00.661 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "interval", 2024-06-10T03:04:00.661 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-06-10T03:04:00.661 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.662 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.662 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "60", 2024-06-10T03:04:00.662 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.662 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.662 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.662 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.662 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.662 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.662 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.662 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.662 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-10T03:04:00.662 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-10T03:04:00.663 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.663 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.663 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.663 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.663 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.663 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.663 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:00.663 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:04:00.663 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:04:00.663 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:04:00.663 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:00.663 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:00.663 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:00.664 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:00.664 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.664 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.664 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.664 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.664 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.664 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-10T03:04:00.664 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-10T03:04:00.664 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.664 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.664 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.664 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.664 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.665 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.665 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.665 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.665 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.665 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.665 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.665 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.665 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-10T03:04:00.665 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-10T03:04:00.665 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.665 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.665 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.665 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-10T03:04:00.666 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.666 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.666 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:00.666 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:04:00.666 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:04:00.666 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:04:00.666 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:00.666 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:00.666 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:00.666 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:00.666 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.666 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.666 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.667 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.667 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.667 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-10T03:04:00.667 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-10T03:04:00.667 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.667 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.667 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.667 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.667 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.667 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.667 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.667 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.667 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.668 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.668 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.668 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.668 INFO:teuthology.orchestra.run.smithi052.stdout: "zabbix_host": { 2024-06-10T03:04:00.668 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "zabbix_host", 2024-06-10T03:04:00.668 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.668 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.668 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.668 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.668 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.668 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.668 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.668 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.668 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.669 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.669 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.669 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.669 INFO:teuthology.orchestra.run.smithi052.stdout: "zabbix_port": { 2024-06-10T03:04:00.669 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "zabbix_port", 2024-06-10T03:04:00.669 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-10T03:04:00.669 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.669 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.669 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "10051", 2024-06-10T03:04:00.669 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.669 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.669 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.669 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.670 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.670 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.670 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.670 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.670 INFO:teuthology.orchestra.run.smithi052.stdout: "zabbix_sender": { 2024-06-10T03:04:00.670 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "zabbix_sender", 2024-06-10T03:04:00.670 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.670 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.670 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.670 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-06-10T03:04:00.670 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.670 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.671 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.671 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.671 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.671 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.671 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.671 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:00.671 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:00.671 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:00.671 INFO:teuthology.orchestra.run.smithi052.stdout: ] 2024-06-10T03:04:00.671 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:00.671 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:00.671 INFO:teuthology.orchestra.run.smithi052.stdout: "modules": [ 2024-06-10T03:04:00.671 INFO:teuthology.orchestra.run.smithi052.stdout: "cephadm", 2024-06-10T03:04:00.671 INFO:teuthology.orchestra.run.smithi052.stdout: "dashboard", 2024-06-10T03:04:00.672 INFO:teuthology.orchestra.run.smithi052.stdout: "iostat", 2024-06-10T03:04:00.672 INFO:teuthology.orchestra.run.smithi052.stdout: "nfs", 2024-06-10T03:04:00.672 INFO:teuthology.orchestra.run.smithi052.stdout: "restful" 2024-06-10T03:04:00.672 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:00.672 INFO:teuthology.orchestra.run.smithi052.stdout: "available_modules": [ 2024-06-10T03:04:00.672 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-10T03:04:00.672 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "alerts", 2024-06-10T03:04:00.672 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-10T03:04:00.672 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-10T03:04:00.672 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-10T03:04:00.672 INFO:teuthology.orchestra.run.smithi052.stdout: "interval": { 2024-06-10T03:04:00.672 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "interval", 2024-06-10T03:04:00.672 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-06-10T03:04:00.672 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.673 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.673 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "60", 2024-06-10T03:04:00.673 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.673 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.673 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.673 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "How frequently to reexamine health status", 2024-06-10T03:04:00.673 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.673 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.673 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.673 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.673 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-10T03:04:00.673 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-10T03:04:00.673 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.674 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.674 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.674 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.674 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.674 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.674 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:00.674 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:04:00.674 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:04:00.674 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:04:00.674 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:00.674 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:00.674 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:00.674 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:00.675 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.675 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.675 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.675 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.675 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.675 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-10T03:04:00.675 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-10T03:04:00.675 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.675 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.675 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.675 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.675 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.675 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.676 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.676 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.676 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.676 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.676 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.676 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.676 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-10T03:04:00.676 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-10T03:04:00.676 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.676 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.676 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.676 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-10T03:04:00.676 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.676 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.677 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:00.677 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:04:00.677 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:04:00.677 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:04:00.677 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:00.677 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:00.677 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:00.677 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:00.677 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.677 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.677 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.677 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.677 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.678 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-10T03:04:00.678 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-10T03:04:00.678 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.678 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.678 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.678 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.678 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.678 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.678 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.678 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.678 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.678 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.678 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.679 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.679 INFO:teuthology.orchestra.run.smithi052.stdout: "smtp_destination": { 2024-06-10T03:04:00.679 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "smtp_destination", 2024-06-10T03:04:00.679 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.679 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.679 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.679 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.679 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.679 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.679 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.679 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Email address to send alerts to", 2024-06-10T03:04:00.679 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.679 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.680 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.680 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.680 INFO:teuthology.orchestra.run.smithi052.stdout: "smtp_from_name": { 2024-06-10T03:04:00.680 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "smtp_from_name", 2024-06-10T03:04:00.680 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.680 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.680 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.680 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "Ceph", 2024-06-10T03:04:00.680 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.680 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.680 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.680 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Email From: name", 2024-06-10T03:04:00.680 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.681 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.681 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.681 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.681 INFO:teuthology.orchestra.run.smithi052.stdout: "smtp_host": { 2024-06-10T03:04:00.681 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "smtp_host", 2024-06-10T03:04:00.681 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.681 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.681 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.681 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.681 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.681 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.681 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.681 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "SMTP server", 2024-06-10T03:04:00.682 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.682 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.682 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.682 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.682 INFO:teuthology.orchestra.run.smithi052.stdout: "smtp_password": { 2024-06-10T03:04:00.682 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "smtp_password", 2024-06-10T03:04:00.682 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.682 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.682 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.682 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.682 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.682 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.682 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.683 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Password to authenticate with", 2024-06-10T03:04:00.683 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.683 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.683 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.683 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.683 INFO:teuthology.orchestra.run.smithi052.stdout: "smtp_port": { 2024-06-10T03:04:00.683 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "smtp_port", 2024-06-10T03:04:00.683 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-10T03:04:00.683 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.683 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.683 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "465", 2024-06-10T03:04:00.683 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.683 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.683 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.684 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "SMTP port", 2024-06-10T03:04:00.684 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.684 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.684 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.684 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.684 INFO:teuthology.orchestra.run.smithi052.stdout: "smtp_sender": { 2024-06-10T03:04:00.684 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "smtp_sender", 2024-06-10T03:04:00.684 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.684 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.684 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.684 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.684 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.684 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.685 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.685 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "SMTP envelope sender", 2024-06-10T03:04:00.685 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.685 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.685 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.685 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.685 INFO:teuthology.orchestra.run.smithi052.stdout: "smtp_ssl": { 2024-06-10T03:04:00.685 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "smtp_ssl", 2024-06-10T03:04:00.685 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.685 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.685 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.685 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-10T03:04:00.685 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.686 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.686 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.686 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Use SSL to connect to SMTP server", 2024-06-10T03:04:00.686 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.686 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.686 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.686 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.686 INFO:teuthology.orchestra.run.smithi052.stdout: "smtp_user": { 2024-06-10T03:04:00.686 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "smtp_user", 2024-06-10T03:04:00.686 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.686 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.686 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.686 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.687 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.687 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.687 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.687 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "User to authenticate as", 2024-06-10T03:04:00.687 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.687 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.687 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.687 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:00.687 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:00.687 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.687 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-10T03:04:00.687 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "balancer", 2024-06-10T03:04:00.688 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-10T03:04:00.688 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-10T03:04:00.688 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-10T03:04:00.688 INFO:teuthology.orchestra.run.smithi052.stdout: "active": { 2024-06-10T03:04:00.688 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "active", 2024-06-10T03:04:00.688 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.688 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.688 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.688 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-10T03:04:00.688 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.688 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.688 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.688 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "automatically balance PGs across cluster", 2024-06-10T03:04:00.688 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.689 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.689 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.689 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.689 INFO:teuthology.orchestra.run.smithi052.stdout: "begin_time": { 2024-06-10T03:04:00.689 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "begin_time", 2024-06-10T03:04:00.689 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.689 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.689 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.689 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "0000", 2024-06-10T03:04:00.689 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.689 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.689 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.689 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "beginning time of day to automatically balance", 2024-06-10T03:04:00.690 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-06-10T03:04:00.690 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.690 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.690 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.690 INFO:teuthology.orchestra.run.smithi052.stdout: "begin_weekday": { 2024-06-10T03:04:00.690 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "begin_weekday", 2024-06-10T03:04:00.690 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "uint", 2024-06-10T03:04:00.690 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.690 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.690 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "0", 2024-06-10T03:04:00.690 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "0", 2024-06-10T03:04:00.690 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "6", 2024-06-10T03:04:00.691 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.691 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-06-10T03:04:00.691 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-06-10T03:04:00.691 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.691 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.691 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.691 INFO:teuthology.orchestra.run.smithi052.stdout: "crush_compat_max_iterations": { 2024-06-10T03:04:00.691 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "crush_compat_max_iterations", 2024-06-10T03:04:00.691 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "uint", 2024-06-10T03:04:00.691 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.691 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.691 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "25", 2024-06-10T03:04:00.691 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "1", 2024-06-10T03:04:00.691 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "250", 2024-06-10T03:04:00.692 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.692 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-06-10T03:04:00.692 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.692 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.692 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.692 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.692 INFO:teuthology.orchestra.run.smithi052.stdout: "crush_compat_metrics": { 2024-06-10T03:04:00.692 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "crush_compat_metrics", 2024-06-10T03:04:00.692 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.692 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.692 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.692 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "pgs,objects,bytes", 2024-06-10T03:04:00.692 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.693 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.693 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.693 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-06-10T03:04:00.693 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-06-10T03:04:00.693 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.693 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.693 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.693 INFO:teuthology.orchestra.run.smithi052.stdout: "crush_compat_step": { 2024-06-10T03:04:00.693 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "crush_compat_step", 2024-06-10T03:04:00.693 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "float", 2024-06-10T03:04:00.693 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.693 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.693 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "0.5", 2024-06-10T03:04:00.694 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "0.001", 2024-06-10T03:04:00.694 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "0.999", 2024-06-10T03:04:00.694 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.694 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "aggressiveness of optimization", 2024-06-10T03:04:00.694 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-06-10T03:04:00.694 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.694 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.694 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.694 INFO:teuthology.orchestra.run.smithi052.stdout: "end_time": { 2024-06-10T03:04:00.694 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "end_time", 2024-06-10T03:04:00.694 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.694 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.695 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.695 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "2359", 2024-06-10T03:04:00.695 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.695 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.695 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.695 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "ending time of day to automatically balance", 2024-06-10T03:04:00.695 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-06-10T03:04:00.695 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.695 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.695 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.695 INFO:teuthology.orchestra.run.smithi052.stdout: "end_weekday": { 2024-06-10T03:04:00.695 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "end_weekday", 2024-06-10T03:04:00.695 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "uint", 2024-06-10T03:04:00.696 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.696 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.696 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "0", 2024-06-10T03:04:00.696 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "0", 2024-06-10T03:04:00.696 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "6", 2024-06-10T03:04:00.696 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.696 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-06-10T03:04:00.696 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-06-10T03:04:00.696 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.696 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.696 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.696 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-10T03:04:00.696 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-10T03:04:00.697 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.697 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.697 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.697 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.697 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.697 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.697 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:00.697 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:04:00.697 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:04:00.697 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:04:00.697 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:00.697 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:00.697 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:00.698 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:00.698 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.698 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.698 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.698 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.698 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.698 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-10T03:04:00.698 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-10T03:04:00.698 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.698 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.698 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.698 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.698 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.699 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.699 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.699 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.699 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.699 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.699 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.699 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.699 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-10T03:04:00.699 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-10T03:04:00.699 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.699 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.699 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.700 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-10T03:04:00.700 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.700 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.700 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:00.700 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:04:00.700 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:04:00.700 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:04:00.700 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:00.700 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:00.700 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:00.700 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:00.700 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.700 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.701 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.701 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.701 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.701 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-10T03:04:00.701 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-10T03:04:00.701 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.701 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.701 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.701 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.701 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.701 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.701 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.701 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.701 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.702 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.702 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.702 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.702 INFO:teuthology.orchestra.run.smithi052.stdout: "min_score": { 2024-06-10T03:04:00.702 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "min_score", 2024-06-10T03:04:00.702 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "float", 2024-06-10T03:04:00.702 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.702 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.702 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "0", 2024-06-10T03:04:00.702 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.702 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.702 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.703 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-06-10T03:04:00.703 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.703 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.703 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.703 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.703 INFO:teuthology.orchestra.run.smithi052.stdout: "mode": { 2024-06-10T03:04:00.703 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "mode", 2024-06-10T03:04:00.703 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.703 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.703 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.703 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "upmap", 2024-06-10T03:04:00.703 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.703 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.704 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:00.704 INFO:teuthology.orchestra.run.smithi052.stdout: "crush-compat", 2024-06-10T03:04:00.704 INFO:teuthology.orchestra.run.smithi052.stdout: "none", 2024-06-10T03:04:00.704 INFO:teuthology.orchestra.run.smithi052.stdout: "read", 2024-06-10T03:04:00.704 INFO:teuthology.orchestra.run.smithi052.stdout: "upmap", 2024-06-10T03:04:00.704 INFO:teuthology.orchestra.run.smithi052.stdout: "upmap-read" 2024-06-10T03:04:00.704 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:00.704 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Balancer mode", 2024-06-10T03:04:00.704 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.704 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.704 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.704 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.704 INFO:teuthology.orchestra.run.smithi052.stdout: "pool_ids": { 2024-06-10T03:04:00.705 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "pool_ids", 2024-06-10T03:04:00.705 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.705 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.705 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.705 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.705 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.705 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.705 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.705 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-06-10T03:04:00.705 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.705 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.705 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.705 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.705 INFO:teuthology.orchestra.run.smithi052.stdout: "sleep_interval": { 2024-06-10T03:04:00.706 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "sleep_interval", 2024-06-10T03:04:00.706 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-06-10T03:04:00.706 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.706 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.706 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "60", 2024-06-10T03:04:00.706 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.706 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.706 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.706 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-06-10T03:04:00.706 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.706 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.706 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.706 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.706 INFO:teuthology.orchestra.run.smithi052.stdout: "upmap_max_deviation": { 2024-06-10T03:04:00.707 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "upmap_max_deviation", 2024-06-10T03:04:00.707 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-10T03:04:00.707 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.707 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.707 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "5", 2024-06-10T03:04:00.707 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "1", 2024-06-10T03:04:00.707 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.707 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.707 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "deviation below which no optimization is attempted", 2024-06-10T03:04:00.707 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-06-10T03:04:00.707 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.707 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.708 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.708 INFO:teuthology.orchestra.run.smithi052.stdout: "upmap_max_optimizations": { 2024-06-10T03:04:00.708 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "upmap_max_optimizations", 2024-06-10T03:04:00.708 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "uint", 2024-06-10T03:04:00.708 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.708 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.708 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "10", 2024-06-10T03:04:00.708 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.708 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.708 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.708 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-06-10T03:04:00.708 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.708 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.709 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.709 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:00.709 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:00.709 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.709 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-10T03:04:00.709 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "cephadm", 2024-06-10T03:04:00.709 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-10T03:04:00.709 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-10T03:04:00.709 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-10T03:04:00.709 INFO:teuthology.orchestra.run.smithi052.stdout: "agent_down_multiplier": { 2024-06-10T03:04:00.709 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "agent_down_multiplier", 2024-06-10T03:04:00.709 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "float", 2024-06-10T03:04:00.709 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.710 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.710 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "3.0", 2024-06-10T03:04:00.710 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.710 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.710 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.710 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-06-10T03:04:00.710 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.710 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.710 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.710 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.710 INFO:teuthology.orchestra.run.smithi052.stdout: "agent_refresh_rate": { 2024-06-10T03:04:00.710 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "agent_refresh_rate", 2024-06-10T03:04:00.710 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-06-10T03:04:00.711 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.711 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.711 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "20", 2024-06-10T03:04:00.711 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.711 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.711 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.711 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-06-10T03:04:00.711 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.711 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.711 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.711 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.711 INFO:teuthology.orchestra.run.smithi052.stdout: "agent_starting_port": { 2024-06-10T03:04:00.712 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "agent_starting_port", 2024-06-10T03:04:00.712 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-10T03:04:00.712 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.712 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.712 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "4721", 2024-06-10T03:04:00.712 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.712 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.712 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.712 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-06-10T03:04:00.712 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.712 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.712 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.712 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.713 INFO:teuthology.orchestra.run.smithi052.stdout: "allow_ptrace": { 2024-06-10T03:04:00.713 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "allow_ptrace", 2024-06-10T03:04:00.713 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.713 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.713 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.713 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.713 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.713 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.713 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.713 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-06-10T03:04:00.713 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-06-10T03:04:00.713 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.713 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.714 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.714 INFO:teuthology.orchestra.run.smithi052.stdout: "autotune_interval": { 2024-06-10T03:04:00.714 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "autotune_interval", 2024-06-10T03:04:00.714 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-06-10T03:04:00.714 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.714 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.714 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "600", 2024-06-10T03:04:00.714 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.714 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.714 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.714 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "how frequently to autotune daemon memory", 2024-06-10T03:04:00.714 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.714 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.715 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.715 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.715 INFO:teuthology.orchestra.run.smithi052.stdout: "autotune_memory_target_ratio": { 2024-06-10T03:04:00.715 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "autotune_memory_target_ratio", 2024-06-10T03:04:00.715 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "float", 2024-06-10T03:04:00.715 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.715 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.715 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "0.7", 2024-06-10T03:04:00.715 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.715 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.715 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.715 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-06-10T03:04:00.715 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.715 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.716 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.716 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.716 INFO:teuthology.orchestra.run.smithi052.stdout: "cephadm_log_destination": { 2024-06-10T03:04:00.716 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "cephadm_log_destination", 2024-06-10T03:04:00.716 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.716 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.716 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.716 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.716 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.716 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.716 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:00.716 INFO:teuthology.orchestra.run.smithi052.stdout: "file", 2024-06-10T03:04:00.716 INFO:teuthology.orchestra.run.smithi052.stdout: "file,syslog", 2024-06-10T03:04:00.717 INFO:teuthology.orchestra.run.smithi052.stdout: "syslog" 2024-06-10T03:04:00.717 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:00.717 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-06-10T03:04:00.717 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.717 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.717 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.717 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.717 INFO:teuthology.orchestra.run.smithi052.stdout: "cgroups_split": { 2024-06-10T03:04:00.717 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "cgroups_split", 2024-06-10T03:04:00.717 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.717 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.717 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.717 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-10T03:04:00.717 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.718 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.718 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.718 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-06-10T03:04:00.718 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.718 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.718 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.718 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.718 INFO:teuthology.orchestra.run.smithi052.stdout: "config_checks_enabled": { 2024-06-10T03:04:00.718 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "config_checks_enabled", 2024-06-10T03:04:00.718 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.718 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.718 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.718 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.718 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.719 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.719 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.719 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-06-10T03:04:00.719 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.719 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.719 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.719 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.719 INFO:teuthology.orchestra.run.smithi052.stdout: "config_dashboard": { 2024-06-10T03:04:00.719 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "config_dashboard", 2024-06-10T03:04:00.719 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.719 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.719 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.719 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-10T03:04:00.720 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.720 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.720 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.720 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-06-10T03:04:00.720 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.720 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.720 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.720 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.720 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_alertmanager": { 2024-06-10T03:04:00.720 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_alertmanager", 2024-06-10T03:04:00.720 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.720 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.720 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.720 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-06-10T03:04:00.721 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.721 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.721 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.721 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Prometheus container image", 2024-06-10T03:04:00.721 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.721 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.721 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.721 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.721 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_base": { 2024-06-10T03:04:00.721 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_base", 2024-06-10T03:04:00.721 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.721 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.721 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.721 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/ceph/ceph", 2024-06-10T03:04:00.722 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.722 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.722 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.722 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Container image name, without the tag", 2024-06-10T03:04:00.722 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.722 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.722 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.722 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.722 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_elasticsearch": { 2024-06-10T03:04:00.722 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_elasticsearch", 2024-06-10T03:04:00.722 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.722 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.722 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.723 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-06-10T03:04:00.723 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.723 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.723 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.723 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "elasticsearch container image", 2024-06-10T03:04:00.723 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.723 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.723 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.723 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.723 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_grafana": { 2024-06-10T03:04:00.723 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_grafana", 2024-06-10T03:04:00.723 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.723 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.723 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.724 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-06-10T03:04:00.724 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.724 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.724 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.724 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Prometheus container image", 2024-06-10T03:04:00.724 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.724 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.724 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.724 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.724 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_haproxy": { 2024-06-10T03:04:00.724 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_haproxy", 2024-06-10T03:04:00.724 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.724 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.725 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.725 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-06-10T03:04:00.725 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.725 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.725 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.725 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "HAproxy container image", 2024-06-10T03:04:00.725 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.725 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.725 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.725 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.725 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_jaeger_agent": { 2024-06-10T03:04:00.725 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_jaeger_agent", 2024-06-10T03:04:00.725 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.725 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.726 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.726 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-06-10T03:04:00.726 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.726 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.726 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.726 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Jaeger agent container image", 2024-06-10T03:04:00.726 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.726 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.726 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.726 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.726 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_jaeger_collector": { 2024-06-10T03:04:00.726 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_jaeger_collector", 2024-06-10T03:04:00.726 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.727 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.727 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.727 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-06-10T03:04:00.727 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.727 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.727 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.727 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Jaeger collector container image", 2024-06-10T03:04:00.727 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.727 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.727 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.727 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.727 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_jaeger_query": { 2024-06-10T03:04:00.727 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_jaeger_query", 2024-06-10T03:04:00.728 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.728 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.728 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.728 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-06-10T03:04:00.728 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.728 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.728 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.728 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Jaeger query container image", 2024-06-10T03:04:00.728 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.728 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.728 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.728 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.728 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_keepalived": { 2024-06-10T03:04:00.728 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_keepalived", 2024-06-10T03:04:00.729 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.729 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.729 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.729 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-06-10T03:04:00.729 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.729 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.730 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.730 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Keepalived container image", 2024-06-10T03:04:00.730 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.730 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.730 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.730 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.730 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_loki": { 2024-06-10T03:04:00.730 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_loki", 2024-06-10T03:04:00.730 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.730 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.730 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.730 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-06-10T03:04:00.730 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.731 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.731 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.731 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Loki container image", 2024-06-10T03:04:00.731 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.731 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.731 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.731 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.731 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_node_exporter": { 2024-06-10T03:04:00.731 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_node_exporter", 2024-06-10T03:04:00.731 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.731 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.731 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.731 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-06-10T03:04:00.732 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.732 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.732 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.732 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Prometheus container image", 2024-06-10T03:04:00.732 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.732 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.732 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.732 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.732 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_nvmeof": { 2024-06-10T03:04:00.732 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_nvmeof", 2024-06-10T03:04:00.732 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.732 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.732 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.733 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-06-10T03:04:00.733 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.733 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.733 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.733 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Nvme-of container image", 2024-06-10T03:04:00.733 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.733 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.733 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.733 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.733 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_prometheus": { 2024-06-10T03:04:00.733 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_prometheus", 2024-06-10T03:04:00.733 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.733 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.733 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.734 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-06-10T03:04:00.734 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.734 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.734 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.734 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Prometheus container image", 2024-06-10T03:04:00.734 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.734 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.734 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.734 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.734 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_promtail": { 2024-06-10T03:04:00.734 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_promtail", 2024-06-10T03:04:00.734 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.734 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.734 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.735 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-06-10T03:04:00.735 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.735 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.735 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.735 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Promtail container image", 2024-06-10T03:04:00.735 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.735 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.735 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.735 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.735 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_samba": { 2024-06-10T03:04:00.735 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_samba", 2024-06-10T03:04:00.735 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.735 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.736 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.736 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-06-10T03:04:00.736 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.736 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.736 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.736 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Samba/SMB container image", 2024-06-10T03:04:00.736 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.736 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.736 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.736 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.736 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_snmp_gateway": { 2024-06-10T03:04:00.736 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_snmp_gateway", 2024-06-10T03:04:00.736 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.737 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.737 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.737 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-06-10T03:04:00.737 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.737 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.737 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.737 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "SNMP Gateway container image", 2024-06-10T03:04:00.737 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.737 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.737 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.737 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.737 INFO:teuthology.orchestra.run.smithi052.stdout: "container_init": { 2024-06-10T03:04:00.737 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_init", 2024-06-10T03:04:00.738 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.738 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.738 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.738 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-10T03:04:00.738 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.738 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.738 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.738 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Run podman/docker with `--init`", 2024-06-10T03:04:00.738 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.738 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.738 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.738 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.738 INFO:teuthology.orchestra.run.smithi052.stdout: "daemon_cache_timeout": { 2024-06-10T03:04:00.739 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "daemon_cache_timeout", 2024-06-10T03:04:00.739 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-06-10T03:04:00.739 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.739 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.739 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "600", 2024-06-10T03:04:00.739 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.739 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.739 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.739 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-06-10T03:04:00.739 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.739 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.739 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.739 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.740 INFO:teuthology.orchestra.run.smithi052.stdout: "default_cephadm_command_timeout": { 2024-06-10T03:04:00.740 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "default_cephadm_command_timeout", 2024-06-10T03:04:00.740 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-10T03:04:00.740 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.740 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.740 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "900", 2024-06-10T03:04:00.740 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.740 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.740 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.740 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-06-10T03:04:00.740 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.740 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.740 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.741 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.741 INFO:teuthology.orchestra.run.smithi052.stdout: "default_registry": { 2024-06-10T03:04:00.741 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "default_registry", 2024-06-10T03:04:00.741 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.741 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.741 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.741 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "docker.io", 2024-06-10T03:04:00.741 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.741 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.741 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.741 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-06-10T03:04:00.741 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.741 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.742 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.742 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.742 INFO:teuthology.orchestra.run.smithi052.stdout: "device_cache_timeout": { 2024-06-10T03:04:00.742 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "device_cache_timeout", 2024-06-10T03:04:00.742 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-06-10T03:04:00.742 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.742 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.742 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "1800", 2024-06-10T03:04:00.742 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.742 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.742 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.742 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "seconds to cache device inventory", 2024-06-10T03:04:00.743 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.743 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.743 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.743 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.743 INFO:teuthology.orchestra.run.smithi052.stdout: "device_enhanced_scan": { 2024-06-10T03:04:00.743 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "device_enhanced_scan", 2024-06-10T03:04:00.743 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.743 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.743 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.743 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.743 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.743 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.743 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.743 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Use libstoragemgmt during device scans", 2024-06-10T03:04:00.744 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.744 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.744 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.744 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.744 INFO:teuthology.orchestra.run.smithi052.stdout: "facts_cache_timeout": { 2024-06-10T03:04:00.744 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "facts_cache_timeout", 2024-06-10T03:04:00.744 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-06-10T03:04:00.744 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.744 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.744 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "60", 2024-06-10T03:04:00.744 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.744 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.744 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.745 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "seconds to cache host facts data", 2024-06-10T03:04:00.745 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.745 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.745 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.745 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.745 INFO:teuthology.orchestra.run.smithi052.stdout: "grafana_dashboards_path": { 2024-06-10T03:04:00.745 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "grafana_dashboards_path", 2024-06-10T03:04:00.745 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.745 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.745 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.745 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-06-10T03:04:00.745 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.745 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.746 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.746 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-06-10T03:04:00.746 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.746 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.746 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.746 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.746 INFO:teuthology.orchestra.run.smithi052.stdout: "host_check_interval": { 2024-06-10T03:04:00.746 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "host_check_interval", 2024-06-10T03:04:00.746 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-06-10T03:04:00.746 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.746 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.746 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "600", 2024-06-10T03:04:00.746 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.746 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.747 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.747 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "how frequently to perform a host check", 2024-06-10T03:04:00.747 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.747 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.747 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.747 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.747 INFO:teuthology.orchestra.run.smithi052.stdout: "hw_monitoring": { 2024-06-10T03:04:00.747 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "hw_monitoring", 2024-06-10T03:04:00.747 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.747 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.747 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.747 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.747 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.748 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.748 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.748 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-06-10T03:04:00.748 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.748 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.748 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.748 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.748 INFO:teuthology.orchestra.run.smithi052.stdout: "inventory_list_all": { 2024-06-10T03:04:00.748 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "inventory_list_all", 2024-06-10T03:04:00.748 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.748 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.748 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.748 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.749 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.749 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.749 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.749 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-06-10T03:04:00.749 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.749 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.749 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.749 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.749 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-10T03:04:00.749 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-10T03:04:00.749 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.749 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.749 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.749 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.750 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.750 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.750 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:00.750 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:04:00.750 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:04:00.750 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:04:00.750 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:00.750 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:00.750 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:00.750 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:00.750 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.750 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.750 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.751 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.751 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.751 INFO:teuthology.orchestra.run.smithi052.stdout: "log_refresh_metadata": { 2024-06-10T03:04:00.751 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_refresh_metadata", 2024-06-10T03:04:00.751 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.751 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.751 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.751 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.751 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.751 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.751 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.751 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-06-10T03:04:00.751 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.751 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.751 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.752 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.752 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-10T03:04:00.752 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-10T03:04:00.752 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.752 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.752 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.752 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-10T03:04:00.752 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.752 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.752 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.752 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-06-10T03:04:00.752 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.752 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.753 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.753 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.753 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-10T03:04:00.753 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-10T03:04:00.753 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.753 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.753 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.753 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-10T03:04:00.753 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.753 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.753 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:00.753 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:04:00.753 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:04:00.754 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:04:00.754 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:00.754 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:00.754 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:00.754 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:00.754 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.754 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.754 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.754 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.754 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.754 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-10T03:04:00.754 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-10T03:04:00.754 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.755 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.755 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.755 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.755 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.755 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.755 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.755 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.755 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.755 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.755 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.755 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.755 INFO:teuthology.orchestra.run.smithi052.stdout: "manage_etc_ceph_ceph_conf": { 2024-06-10T03:04:00.755 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-06-10T03:04:00.755 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.756 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.756 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.756 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.756 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.756 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.756 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.756 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-06-10T03:04:00.756 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.756 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.756 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.756 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.756 INFO:teuthology.orchestra.run.smithi052.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-06-10T03:04:00.756 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-06-10T03:04:00.757 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.757 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.757 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.757 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "*", 2024-06-10T03:04:00.757 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.757 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.757 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.757 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-06-10T03:04:00.757 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.757 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.757 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.757 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.757 INFO:teuthology.orchestra.run.smithi052.stdout: "max_count_per_host": { 2024-06-10T03:04:00.758 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "max_count_per_host", 2024-06-10T03:04:00.758 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-10T03:04:00.758 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.758 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.758 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "10", 2024-06-10T03:04:00.758 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.758 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.758 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.758 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "max number of daemons per service per host", 2024-06-10T03:04:00.758 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.758 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.758 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.758 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.759 INFO:teuthology.orchestra.run.smithi052.stdout: "max_osd_draining_count": { 2024-06-10T03:04:00.759 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "max_osd_draining_count", 2024-06-10T03:04:00.759 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-10T03:04:00.759 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.759 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.759 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "10", 2024-06-10T03:04:00.759 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.759 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.759 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.759 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-06-10T03:04:00.759 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.759 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.759 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.759 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.760 INFO:teuthology.orchestra.run.smithi052.stdout: "migration_current": { 2024-06-10T03:04:00.760 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "migration_current", 2024-06-10T03:04:00.760 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-10T03:04:00.760 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.760 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.760 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.760 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.760 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.760 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.760 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "internal - do not modify", 2024-06-10T03:04:00.760 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.760 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.760 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.760 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.760 INFO:teuthology.orchestra.run.smithi052.stdout: "mode": { 2024-06-10T03:04:00.761 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "mode", 2024-06-10T03:04:00.761 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.761 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.761 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.761 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "root", 2024-06-10T03:04:00.761 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.761 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.761 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:00.761 INFO:teuthology.orchestra.run.smithi052.stdout: "cephadm-package", 2024-06-10T03:04:00.761 INFO:teuthology.orchestra.run.smithi052.stdout: "root" 2024-06-10T03:04:00.761 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:00.761 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "mode for remote execution of cephadm", 2024-06-10T03:04:00.761 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.761 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.762 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.762 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.762 INFO:teuthology.orchestra.run.smithi052.stdout: "oob_default_addr": { 2024-06-10T03:04:00.762 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "oob_default_addr", 2024-06-10T03:04:00.762 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.762 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.762 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.762 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "169.254.1.1", 2024-06-10T03:04:00.762 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.762 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.762 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.762 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Default address for RedFish API (oob management).", 2024-06-10T03:04:00.762 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.763 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.763 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.763 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.763 INFO:teuthology.orchestra.run.smithi052.stdout: "prometheus_alerts_path": { 2024-06-10T03:04:00.763 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "prometheus_alerts_path", 2024-06-10T03:04:00.763 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.763 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.763 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.763 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-06-10T03:04:00.763 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.763 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.763 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.763 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-06-10T03:04:00.763 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.763 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.764 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.764 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.764 INFO:teuthology.orchestra.run.smithi052.stdout: "registry_insecure": { 2024-06-10T03:04:00.764 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "registry_insecure", 2024-06-10T03:04:00.764 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.764 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.764 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.764 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.764 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.764 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.764 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.764 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-06-10T03:04:00.764 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.764 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.765 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.765 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.765 INFO:teuthology.orchestra.run.smithi052.stdout: "registry_password": { 2024-06-10T03:04:00.765 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "registry_password", 2024-06-10T03:04:00.765 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.765 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.765 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.765 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.765 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.765 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.765 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.765 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-06-10T03:04:00.765 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.765 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.766 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.766 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.766 INFO:teuthology.orchestra.run.smithi052.stdout: "registry_url": { 2024-06-10T03:04:00.766 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "registry_url", 2024-06-10T03:04:00.766 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.766 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.766 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.766 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.766 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.766 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.766 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.766 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-06-10T03:04:00.767 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.767 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.767 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.767 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.767 INFO:teuthology.orchestra.run.smithi052.stdout: "registry_username": { 2024-06-10T03:04:00.767 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "registry_username", 2024-06-10T03:04:00.767 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.767 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.767 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.767 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.767 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.767 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.767 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.767 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-06-10T03:04:00.768 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.768 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.768 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.768 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.768 INFO:teuthology.orchestra.run.smithi052.stdout: "secure_monitoring_stack": { 2024-06-10T03:04:00.768 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "secure_monitoring_stack", 2024-06-10T03:04:00.768 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.768 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.768 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.768 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.768 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.768 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.768 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.769 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-06-10T03:04:00.769 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.769 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.769 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.769 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.769 INFO:teuthology.orchestra.run.smithi052.stdout: "service_discovery_port": { 2024-06-10T03:04:00.769 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "service_discovery_port", 2024-06-10T03:04:00.769 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-10T03:04:00.769 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.769 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.769 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "8765", 2024-06-10T03:04:00.769 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.769 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.770 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.770 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "cephadm service discovery port", 2024-06-10T03:04:00.770 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.770 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.770 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.770 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.770 INFO:teuthology.orchestra.run.smithi052.stdout: "ssh_config_file": { 2024-06-10T03:04:00.770 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "ssh_config_file", 2024-06-10T03:04:00.770 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.770 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.770 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.770 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.770 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.771 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.771 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.771 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-06-10T03:04:00.771 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.771 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.771 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.771 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.771 INFO:teuthology.orchestra.run.smithi052.stdout: "use_agent": { 2024-06-10T03:04:00.771 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "use_agent", 2024-06-10T03:04:00.771 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.771 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.771 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.771 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.772 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.772 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.772 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.772 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-06-10T03:04:00.772 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.772 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.772 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.772 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.772 INFO:teuthology.orchestra.run.smithi052.stdout: "use_repo_digest": { 2024-06-10T03:04:00.772 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "use_repo_digest", 2024-06-10T03:04:00.772 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.772 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.772 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.773 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-10T03:04:00.773 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.773 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.773 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.773 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-06-10T03:04:00.773 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.773 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.773 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.773 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.773 INFO:teuthology.orchestra.run.smithi052.stdout: "warn_on_failed_host_check": { 2024-06-10T03:04:00.773 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "warn_on_failed_host_check", 2024-06-10T03:04:00.773 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.773 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.774 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.774 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-10T03:04:00.774 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.774 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.774 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.774 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "raise a health warning if the host check fails", 2024-06-10T03:04:00.774 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.774 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.774 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.774 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.774 INFO:teuthology.orchestra.run.smithi052.stdout: "warn_on_stray_daemons": { 2024-06-10T03:04:00.774 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "warn_on_stray_daemons", 2024-06-10T03:04:00.775 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.775 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.775 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.775 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-10T03:04:00.775 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.775 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.775 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.775 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-06-10T03:04:00.775 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.775 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.775 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.775 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.775 INFO:teuthology.orchestra.run.smithi052.stdout: "warn_on_stray_hosts": { 2024-06-10T03:04:00.775 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "warn_on_stray_hosts", 2024-06-10T03:04:00.776 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.776 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.776 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.776 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-10T03:04:00.776 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.776 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.776 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.776 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-06-10T03:04:00.776 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.776 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.776 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.776 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:00.776 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:00.776 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.777 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-10T03:04:00.777 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "crash", 2024-06-10T03:04:00.777 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-10T03:04:00.777 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-10T03:04:00.777 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-10T03:04:00.777 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-10T03:04:00.777 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-10T03:04:00.777 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.777 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.777 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.777 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.777 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.777 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.777 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:00.777 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:04:00.778 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:04:00.778 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:04:00.778 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:00.778 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:00.778 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:00.778 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:00.778 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.778 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.778 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.778 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.778 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.778 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-10T03:04:00.779 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-10T03:04:00.779 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.779 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.779 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.779 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.779 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.779 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.779 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.779 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.779 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.779 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.779 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.779 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.779 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-10T03:04:00.779 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-10T03:04:00.780 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.780 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.780 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.780 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-10T03:04:00.780 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.780 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.780 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:00.780 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:04:00.780 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:04:00.780 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:04:00.780 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:00.780 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:00.780 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:00.780 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:00.781 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.781 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.781 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.781 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.781 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.781 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-10T03:04:00.781 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-10T03:04:00.781 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.781 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.781 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.781 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.781 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.781 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.781 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.781 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.782 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.782 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.782 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.782 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.782 INFO:teuthology.orchestra.run.smithi052.stdout: "retain_interval": { 2024-06-10T03:04:00.782 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "retain_interval", 2024-06-10T03:04:00.782 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-06-10T03:04:00.782 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.782 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.782 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "31536000", 2024-06-10T03:04:00.782 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.782 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.782 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.783 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "how long to retain crashes before pruning them", 2024-06-10T03:04:00.783 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.783 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.783 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.783 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.783 INFO:teuthology.orchestra.run.smithi052.stdout: "warn_recent_interval": { 2024-06-10T03:04:00.783 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "warn_recent_interval", 2024-06-10T03:04:00.783 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-06-10T03:04:00.783 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.783 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.783 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "1209600", 2024-06-10T03:04:00.783 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.783 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.783 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.784 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "time interval in which to warn about recent crashes", 2024-06-10T03:04:00.784 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.784 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.784 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.784 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:00.784 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:00.784 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.784 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-10T03:04:00.784 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "dashboard", 2024-06-10T03:04:00.784 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-10T03:04:00.784 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-10T03:04:00.784 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-10T03:04:00.784 INFO:teuthology.orchestra.run.smithi052.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-06-10T03:04:00.784 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-06-10T03:04:00.784 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-10T03:04:00.785 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.785 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.785 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "10", 2024-06-10T03:04:00.785 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.785 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.785 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.785 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.785 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.785 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.785 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.785 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.785 INFO:teuthology.orchestra.run.smithi052.stdout: "ALERTMANAGER_API_HOST": { 2024-06-10T03:04:00.786 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "ALERTMANAGER_API_HOST", 2024-06-10T03:04:00.786 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.786 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.786 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.786 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.786 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.786 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.786 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.786 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.786 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.786 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.786 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.787 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.787 INFO:teuthology.orchestra.run.smithi052.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-06-10T03:04:00.787 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-06-10T03:04:00.787 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.787 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.787 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.787 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-10T03:04:00.787 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.787 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.787 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.787 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.787 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.787 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.788 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.788 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.788 INFO:teuthology.orchestra.run.smithi052.stdout: "AUDIT_API_ENABLED": { 2024-06-10T03:04:00.788 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "AUDIT_API_ENABLED", 2024-06-10T03:04:00.788 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.788 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.788 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.788 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.788 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.788 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.788 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.788 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.788 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.789 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.789 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.789 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.789 INFO:teuthology.orchestra.run.smithi052.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-06-10T03:04:00.789 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-06-10T03:04:00.789 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.789 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.789 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.789 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-10T03:04:00.789 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.789 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.789 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.789 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.790 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.790 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.790 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.790 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.790 INFO:teuthology.orchestra.run.smithi052.stdout: "ENABLE_BROWSABLE_API": { 2024-06-10T03:04:00.790 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "ENABLE_BROWSABLE_API", 2024-06-10T03:04:00.790 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.790 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.790 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.790 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-10T03:04:00.790 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.790 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.790 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.791 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.791 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.791 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.791 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.791 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.791 INFO:teuthology.orchestra.run.smithi052.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-06-10T03:04:00.791 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-06-10T03:04:00.791 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.791 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.791 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.791 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-10T03:04:00.791 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.792 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.792 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.792 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.792 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.792 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.792 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.792 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.792 INFO:teuthology.orchestra.run.smithi052.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-06-10T03:04:00.792 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-06-10T03:04:00.792 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.792 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.792 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.792 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-10T03:04:00.792 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.793 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.793 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.793 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.793 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.793 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.793 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.793 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.793 INFO:teuthology.orchestra.run.smithi052.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-06-10T03:04:00.793 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-06-10T03:04:00.793 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.793 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.793 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.793 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-10T03:04:00.794 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.794 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.794 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.794 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.794 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.794 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.794 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.794 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.794 INFO:teuthology.orchestra.run.smithi052.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-06-10T03:04:00.794 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-06-10T03:04:00.794 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.794 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.794 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.795 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-10T03:04:00.795 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.795 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.795 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.795 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.795 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.795 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.795 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.795 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.795 INFO:teuthology.orchestra.run.smithi052.stdout: "FEATURE_TOGGLE_NFS": { 2024-06-10T03:04:00.795 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-06-10T03:04:00.795 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.795 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.796 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.796 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-10T03:04:00.796 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.796 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.796 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.796 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.796 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.796 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.796 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.796 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.796 INFO:teuthology.orchestra.run.smithi052.stdout: "FEATURE_TOGGLE_RBD": { 2024-06-10T03:04:00.796 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-06-10T03:04:00.796 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.797 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.797 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.797 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-10T03:04:00.797 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.797 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.797 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.797 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.797 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.797 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.797 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.797 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.797 INFO:teuthology.orchestra.run.smithi052.stdout: "FEATURE_TOGGLE_RGW": { 2024-06-10T03:04:00.797 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-06-10T03:04:00.798 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.798 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.798 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.798 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-10T03:04:00.798 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.798 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.798 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.798 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.798 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.798 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.798 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.798 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.798 INFO:teuthology.orchestra.run.smithi052.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-06-10T03:04:00.799 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-06-10T03:04:00.799 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.799 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.799 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.799 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.799 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.799 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.799 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.799 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.799 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.799 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.799 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.799 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.800 INFO:teuthology.orchestra.run.smithi052.stdout: "GRAFANA_API_PASSWORD": { 2024-06-10T03:04:00.800 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "GRAFANA_API_PASSWORD", 2024-06-10T03:04:00.800 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.800 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.800 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.800 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "admin", 2024-06-10T03:04:00.800 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.800 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.800 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.800 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.800 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.800 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.800 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.800 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.801 INFO:teuthology.orchestra.run.smithi052.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-06-10T03:04:00.801 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-06-10T03:04:00.801 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.801 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.801 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.801 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-10T03:04:00.801 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.801 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.801 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.801 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.801 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.801 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.801 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.801 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.801 INFO:teuthology.orchestra.run.smithi052.stdout: "GRAFANA_API_URL": { 2024-06-10T03:04:00.802 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "GRAFANA_API_URL", 2024-06-10T03:04:00.802 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.802 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.802 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.802 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.802 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.802 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.802 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.802 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.802 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.802 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.802 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.803 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.803 INFO:teuthology.orchestra.run.smithi052.stdout: "GRAFANA_API_USERNAME": { 2024-06-10T03:04:00.803 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "GRAFANA_API_USERNAME", 2024-06-10T03:04:00.803 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.803 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.803 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.803 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "admin", 2024-06-10T03:04:00.803 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.803 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.803 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.803 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.803 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.803 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.803 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.804 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.804 INFO:teuthology.orchestra.run.smithi052.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-06-10T03:04:00.804 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-06-10T03:04:00.804 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.804 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.804 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.804 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.804 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.804 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.804 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.804 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.804 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.804 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.805 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.805 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.805 INFO:teuthology.orchestra.run.smithi052.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-06-10T03:04:00.805 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-06-10T03:04:00.805 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.805 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.805 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.805 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.805 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.805 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.805 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.805 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.805 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.805 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.806 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.806 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.806 INFO:teuthology.orchestra.run.smithi052.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-06-10T03:04:00.806 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-06-10T03:04:00.806 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.806 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.806 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.806 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-10T03:04:00.806 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.806 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.806 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.806 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.806 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.807 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.807 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.807 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.807 INFO:teuthology.orchestra.run.smithi052.stdout: "ISSUE_TRACKER_API_KEY": { 2024-06-10T03:04:00.807 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-06-10T03:04:00.807 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.807 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.807 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.807 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.807 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.807 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.807 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.807 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.808 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.808 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.808 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.808 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.808 INFO:teuthology.orchestra.run.smithi052.stdout: "PROMETHEUS_API_HOST": { 2024-06-10T03:04:00.808 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PROMETHEUS_API_HOST", 2024-06-10T03:04:00.808 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.808 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.808 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.808 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.808 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.808 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.808 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.808 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.809 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.809 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.809 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.809 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.809 INFO:teuthology.orchestra.run.smithi052.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-06-10T03:04:00.809 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-06-10T03:04:00.809 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.809 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.809 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.809 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-10T03:04:00.809 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.809 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.809 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.809 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.809 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.810 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.810 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.810 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.810 INFO:teuthology.orchestra.run.smithi052.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-06-10T03:04:00.810 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-06-10T03:04:00.810 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.810 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.810 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.810 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.810 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.810 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.810 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.810 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.811 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.811 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.811 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.811 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.811 INFO:teuthology.orchestra.run.smithi052.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-06-10T03:04:00.811 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-06-10T03:04:00.811 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.811 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.811 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.811 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.811 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.811 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.811 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.811 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.811 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.812 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.812 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.812 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.812 INFO:teuthology.orchestra.run.smithi052.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-06-10T03:04:00.812 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-06-10T03:04:00.812 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.812 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.812 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.812 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-10T03:04:00.812 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.812 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.812 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.812 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.813 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.813 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.813 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.813 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.813 INFO:teuthology.orchestra.run.smithi052.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-06-10T03:04:00.813 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-06-10T03:04:00.813 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.813 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.813 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.813 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-10T03:04:00.813 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.813 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.813 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.813 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.814 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.814 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.814 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.814 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.814 INFO:teuthology.orchestra.run.smithi052.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-06-10T03:04:00.814 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-06-10T03:04:00.814 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.814 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.814 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.814 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.814 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.814 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.814 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.815 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.815 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.815 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.815 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.815 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.815 INFO:teuthology.orchestra.run.smithi052.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-06-10T03:04:00.815 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-06-10T03:04:00.815 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.815 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.815 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.815 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.815 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.815 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.815 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.816 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.816 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.816 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.816 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.816 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.816 INFO:teuthology.orchestra.run.smithi052.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-06-10T03:04:00.816 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-06-10T03:04:00.816 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.816 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.816 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.816 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.816 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.816 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.817 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.817 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.817 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.817 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.817 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.817 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.817 INFO:teuthology.orchestra.run.smithi052.stdout: "PWD_POLICY_ENABLED": { 2024-06-10T03:04:00.817 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PWD_POLICY_ENABLED", 2024-06-10T03:04:00.817 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.817 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.817 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.817 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-10T03:04:00.817 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.817 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.817 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.818 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.818 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.818 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.818 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.818 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.818 INFO:teuthology.orchestra.run.smithi052.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-06-10T03:04:00.818 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-06-10T03:04:00.818 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.818 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.818 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.818 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-06-10T03:04:00.818 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.818 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.819 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.819 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.819 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.819 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.819 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.819 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.819 INFO:teuthology.orchestra.run.smithi052.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-06-10T03:04:00.819 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-06-10T03:04:00.819 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-10T03:04:00.819 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.819 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.819 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "10", 2024-06-10T03:04:00.819 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.819 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.820 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.820 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.820 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.820 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.820 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.820 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.820 INFO:teuthology.orchestra.run.smithi052.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-06-10T03:04:00.820 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-06-10T03:04:00.820 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-10T03:04:00.820 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.820 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.820 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "8", 2024-06-10T03:04:00.820 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.820 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.821 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.821 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.821 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.821 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.821 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.821 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.821 INFO:teuthology.orchestra.run.smithi052.stdout: "REST_REQUESTS_TIMEOUT": { 2024-06-10T03:04:00.821 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-06-10T03:04:00.821 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-10T03:04:00.821 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.821 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.821 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "45", 2024-06-10T03:04:00.821 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.821 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.822 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.822 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.822 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.822 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.822 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.822 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.822 INFO:teuthology.orchestra.run.smithi052.stdout: "RGW_API_ACCESS_KEY": { 2024-06-10T03:04:00.822 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "RGW_API_ACCESS_KEY", 2024-06-10T03:04:00.822 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.822 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.822 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.822 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.823 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.823 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.823 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.823 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.823 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.823 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.823 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.823 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.823 INFO:teuthology.orchestra.run.smithi052.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-06-10T03:04:00.823 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-06-10T03:04:00.823 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.823 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.823 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.823 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "admin", 2024-06-10T03:04:00.824 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.824 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.824 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.824 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.824 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.824 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.824 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.824 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.824 INFO:teuthology.orchestra.run.smithi052.stdout: "RGW_API_SECRET_KEY": { 2024-06-10T03:04:00.824 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "RGW_API_SECRET_KEY", 2024-06-10T03:04:00.824 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.824 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.824 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.824 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.825 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.825 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.825 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.825 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.825 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.825 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.825 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.825 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.825 INFO:teuthology.orchestra.run.smithi052.stdout: "RGW_API_SSL_VERIFY": { 2024-06-10T03:04:00.825 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "RGW_API_SSL_VERIFY", 2024-06-10T03:04:00.825 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.825 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.825 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.826 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-10T03:04:00.826 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.826 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.826 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.826 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.826 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.826 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.826 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.826 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.826 INFO:teuthology.orchestra.run.smithi052.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-06-10T03:04:00.826 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-06-10T03:04:00.826 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-10T03:04:00.826 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.827 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.827 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "0", 2024-06-10T03:04:00.827 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.827 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.827 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.827 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.827 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.827 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.827 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.827 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.827 INFO:teuthology.orchestra.run.smithi052.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-06-10T03:04:00.827 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-06-10T03:04:00.828 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-10T03:04:00.828 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.828 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.828 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "10", 2024-06-10T03:04:00.828 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.828 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.828 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.828 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.828 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.828 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.828 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.828 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.828 INFO:teuthology.orchestra.run.smithi052.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-06-10T03:04:00.828 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-06-10T03:04:00.829 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-10T03:04:00.829 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.829 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.829 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "5", 2024-06-10T03:04:00.829 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.829 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.829 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.829 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.829 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.829 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.829 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.829 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.829 INFO:teuthology.orchestra.run.smithi052.stdout: "cross_origin_url": { 2024-06-10T03:04:00.830 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "cross_origin_url", 2024-06-10T03:04:00.830 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.830 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.830 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.830 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.830 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.830 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.830 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.830 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.830 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.830 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.830 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.830 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.831 INFO:teuthology.orchestra.run.smithi052.stdout: "crt_file": { 2024-06-10T03:04:00.831 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "crt_file", 2024-06-10T03:04:00.831 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.831 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.831 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.831 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.831 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.831 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.831 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.831 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.831 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.831 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.831 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.832 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.832 INFO:teuthology.orchestra.run.smithi052.stdout: "debug": { 2024-06-10T03:04:00.832 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "debug", 2024-06-10T03:04:00.832 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.832 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.832 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.832 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.832 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.832 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.832 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.832 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Enable/disable debug options", 2024-06-10T03:04:00.832 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.832 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.833 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.833 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.833 INFO:teuthology.orchestra.run.smithi052.stdout: "jwt_token_ttl": { 2024-06-10T03:04:00.833 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "jwt_token_ttl", 2024-06-10T03:04:00.833 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-10T03:04:00.833 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.833 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.833 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "28800", 2024-06-10T03:04:00.833 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.833 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.833 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.833 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.833 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.833 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.833 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.834 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.834 INFO:teuthology.orchestra.run.smithi052.stdout: "key_file": { 2024-06-10T03:04:00.834 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "key_file", 2024-06-10T03:04:00.834 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.834 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.834 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.834 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.834 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.834 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.834 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.834 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.834 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.835 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.835 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.835 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.835 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-10T03:04:00.835 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-10T03:04:00.835 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.835 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.835 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.835 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.835 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.835 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.835 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:00.835 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:04:00.836 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:04:00.836 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:04:00.836 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:00.836 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:00.836 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:00.836 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:00.836 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.836 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.836 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.836 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.836 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.836 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-10T03:04:00.836 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-10T03:04:00.836 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.837 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.837 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.837 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.837 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.837 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.837 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.837 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.837 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.837 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.837 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.837 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.837 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-10T03:04:00.837 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-10T03:04:00.837 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.838 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.838 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.838 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-10T03:04:00.838 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.838 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.838 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:00.838 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:04:00.838 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:04:00.838 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:04:00.838 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:00.838 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:00.838 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:00.838 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:00.839 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.839 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.839 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.839 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.839 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.839 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-10T03:04:00.839 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-10T03:04:00.839 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.839 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.839 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.839 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.839 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.839 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.839 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.839 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.840 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.840 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.840 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.840 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.840 INFO:teuthology.orchestra.run.smithi052.stdout: "motd": { 2024-06-10T03:04:00.840 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "motd", 2024-06-10T03:04:00.840 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.840 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.840 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.840 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.840 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.840 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.840 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.841 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "The message of the day", 2024-06-10T03:04:00.841 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.841 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.841 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.841 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.841 INFO:teuthology.orchestra.run.smithi052.stdout: "redirect_resolve_ip_addr": { 2024-06-10T03:04:00.841 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "redirect_resolve_ip_addr", 2024-06-10T03:04:00.841 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.841 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.841 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.841 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.841 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.841 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.841 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.841 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.841 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.842 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.842 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.842 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.842 INFO:teuthology.orchestra.run.smithi052.stdout: "server_addr": { 2024-06-10T03:04:00.842 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "server_addr", 2024-06-10T03:04:00.842 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.842 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.842 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.842 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "::", 2024-06-10T03:04:00.842 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.842 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.842 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.843 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.843 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.843 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.843 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.843 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.843 INFO:teuthology.orchestra.run.smithi052.stdout: "server_port": { 2024-06-10T03:04:00.843 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "server_port", 2024-06-10T03:04:00.843 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-10T03:04:00.843 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.843 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.843 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "8080", 2024-06-10T03:04:00.843 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.843 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.844 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.844 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.844 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.844 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.844 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.844 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.844 INFO:teuthology.orchestra.run.smithi052.stdout: "ssl": { 2024-06-10T03:04:00.844 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "ssl", 2024-06-10T03:04:00.844 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.844 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.844 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.844 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-10T03:04:00.844 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.844 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.845 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.845 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.845 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.845 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.845 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.845 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.845 INFO:teuthology.orchestra.run.smithi052.stdout: "ssl_server_port": { 2024-06-10T03:04:00.845 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "ssl_server_port", 2024-06-10T03:04:00.845 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-10T03:04:00.845 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.845 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.845 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "8443", 2024-06-10T03:04:00.845 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.845 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.846 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.846 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.846 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.846 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.846 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.846 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.846 INFO:teuthology.orchestra.run.smithi052.stdout: "standby_behaviour": { 2024-06-10T03:04:00.846 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "standby_behaviour", 2024-06-10T03:04:00.846 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.846 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.846 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.846 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "redirect", 2024-06-10T03:04:00.846 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.847 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.847 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:00.847 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:00.847 INFO:teuthology.orchestra.run.smithi052.stdout: "redirect" 2024-06-10T03:04:00.847 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:00.847 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.847 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.847 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.847 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.847 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.847 INFO:teuthology.orchestra.run.smithi052.stdout: "standby_error_status_code": { 2024-06-10T03:04:00.847 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "standby_error_status_code", 2024-06-10T03:04:00.848 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-10T03:04:00.848 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.848 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.848 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "500", 2024-06-10T03:04:00.848 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "400", 2024-06-10T03:04:00.848 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "599", 2024-06-10T03:04:00.848 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.848 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.848 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.848 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.848 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.848 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.848 INFO:teuthology.orchestra.run.smithi052.stdout: "url_prefix": { 2024-06-10T03:04:00.849 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "url_prefix", 2024-06-10T03:04:00.849 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.849 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.849 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.849 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.849 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.849 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.849 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.849 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.849 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.849 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.849 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.849 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:00.849 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:00.850 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.850 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-10T03:04:00.850 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "devicehealth", 2024-06-10T03:04:00.850 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-10T03:04:00.850 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-10T03:04:00.850 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-10T03:04:00.850 INFO:teuthology.orchestra.run.smithi052.stdout: "enable_monitoring": { 2024-06-10T03:04:00.850 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "enable_monitoring", 2024-06-10T03:04:00.850 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.850 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.850 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.850 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-10T03:04:00.850 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.851 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.851 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.851 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "monitor device health metrics", 2024-06-10T03:04:00.851 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.851 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.851 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.851 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.851 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-10T03:04:00.851 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-10T03:04:00.851 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.851 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.851 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.851 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.851 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.851 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.852 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:00.852 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:04:00.852 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:04:00.852 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:04:00.852 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:00.852 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:00.852 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:00.852 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:00.852 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.852 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.852 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.852 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.852 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.852 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-10T03:04:00.852 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-10T03:04:00.852 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.853 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.853 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.853 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.853 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.853 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.853 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.853 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.853 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.853 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.853 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.853 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.853 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-10T03:04:00.853 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-10T03:04:00.853 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.854 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.854 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.854 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-10T03:04:00.854 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.854 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.854 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:00.854 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:04:00.854 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:04:00.854 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:04:00.854 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:00.854 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:00.854 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:00.854 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:00.855 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.855 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.855 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.855 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.855 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.855 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-10T03:04:00.855 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-10T03:04:00.855 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.855 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.855 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.855 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.855 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.855 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.855 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.856 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.856 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.856 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.856 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.856 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.856 INFO:teuthology.orchestra.run.smithi052.stdout: "mark_out_threshold": { 2024-06-10T03:04:00.856 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "mark_out_threshold", 2024-06-10T03:04:00.856 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-06-10T03:04:00.856 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.856 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.856 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "2419200", 2024-06-10T03:04:00.856 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.856 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.856 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.857 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-06-10T03:04:00.857 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.857 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.857 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.857 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.857 INFO:teuthology.orchestra.run.smithi052.stdout: "pool_name": { 2024-06-10T03:04:00.857 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "pool_name", 2024-06-10T03:04:00.857 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.857 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.857 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.857 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "device_health_metrics", 2024-06-10T03:04:00.857 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.857 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.857 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.857 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "name of pool in which to store device health metrics", 2024-06-10T03:04:00.858 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.858 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.858 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.858 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.858 INFO:teuthology.orchestra.run.smithi052.stdout: "retention_period": { 2024-06-10T03:04:00.858 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "retention_period", 2024-06-10T03:04:00.858 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-06-10T03:04:00.858 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.858 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.858 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "15552000", 2024-06-10T03:04:00.858 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.858 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.858 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.859 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "how long to retain device health metrics", 2024-06-10T03:04:00.859 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.859 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.859 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.859 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.859 INFO:teuthology.orchestra.run.smithi052.stdout: "scrape_frequency": { 2024-06-10T03:04:00.859 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "scrape_frequency", 2024-06-10T03:04:00.859 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-06-10T03:04:00.859 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.859 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.859 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "86400", 2024-06-10T03:04:00.859 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.859 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.859 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.860 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "how frequently to scrape device health metrics", 2024-06-10T03:04:00.860 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.860 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.860 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.860 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.860 INFO:teuthology.orchestra.run.smithi052.stdout: "self_heal": { 2024-06-10T03:04:00.860 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "self_heal", 2024-06-10T03:04:00.860 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.860 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.860 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.860 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-10T03:04:00.860 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.860 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.860 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.861 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-06-10T03:04:00.861 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.861 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.861 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.861 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.861 INFO:teuthology.orchestra.run.smithi052.stdout: "sleep_interval": { 2024-06-10T03:04:00.861 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "sleep_interval", 2024-06-10T03:04:00.861 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-06-10T03:04:00.861 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.861 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.861 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "600", 2024-06-10T03:04:00.861 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.861 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.861 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.861 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "how frequently to wake up and check device health", 2024-06-10T03:04:00.862 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.862 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.862 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.862 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.862 INFO:teuthology.orchestra.run.smithi052.stdout: "warn_threshold": { 2024-06-10T03:04:00.862 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "warn_threshold", 2024-06-10T03:04:00.862 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-06-10T03:04:00.862 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.862 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.862 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "7257600", 2024-06-10T03:04:00.862 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.862 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.862 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.863 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-06-10T03:04:00.863 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.863 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.863 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.863 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:00.863 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:00.863 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.863 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-10T03:04:00.863 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "diskprediction_local", 2024-06-10T03:04:00.863 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-10T03:04:00.863 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-10T03:04:00.863 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-10T03:04:00.863 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-10T03:04:00.863 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-10T03:04:00.863 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.864 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.864 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.864 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.864 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.864 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.864 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:00.864 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:04:00.864 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:04:00.864 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:04:00.864 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:00.864 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:00.864 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:00.864 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:00.864 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.865 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.865 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.865 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.865 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.865 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-10T03:04:00.865 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-10T03:04:00.865 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.865 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.865 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.865 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.865 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.865 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.865 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.866 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.866 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.866 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.866 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.866 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.866 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-10T03:04:00.866 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-10T03:04:00.866 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.866 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.866 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.866 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-10T03:04:00.866 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.866 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.867 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:00.867 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:04:00.867 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:04:00.867 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:04:00.867 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:00.867 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:00.867 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:00.867 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:00.867 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.867 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.867 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.867 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.867 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.867 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-10T03:04:00.868 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-10T03:04:00.868 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.868 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.868 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.868 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.868 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.868 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.868 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.868 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.868 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.868 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.868 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.868 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.869 INFO:teuthology.orchestra.run.smithi052.stdout: "predict_interval": { 2024-06-10T03:04:00.869 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "predict_interval", 2024-06-10T03:04:00.869 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.869 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.869 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.869 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "86400", 2024-06-10T03:04:00.869 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.869 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.869 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.869 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.870 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.870 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.870 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.870 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.870 INFO:teuthology.orchestra.run.smithi052.stdout: "predictor_model": { 2024-06-10T03:04:00.870 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "predictor_model", 2024-06-10T03:04:00.870 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.870 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.870 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.870 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "prophetstor", 2024-06-10T03:04:00.870 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.870 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.871 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.871 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.871 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.871 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.871 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.871 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.871 INFO:teuthology.orchestra.run.smithi052.stdout: "sleep_interval": { 2024-06-10T03:04:00.871 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "sleep_interval", 2024-06-10T03:04:00.871 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.871 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.871 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.871 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "600", 2024-06-10T03:04:00.871 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.872 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.872 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.872 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.872 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.872 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.872 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.872 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:00.872 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:00.872 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.872 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-10T03:04:00.872 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "influx", 2024-06-10T03:04:00.872 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": false, 2024-06-10T03:04:00.872 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "influxdb python module not found", 2024-06-10T03:04:00.873 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-10T03:04:00.873 INFO:teuthology.orchestra.run.smithi052.stdout: "batch_size": { 2024-06-10T03:04:00.873 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "batch_size", 2024-06-10T03:04:00.873 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-10T03:04:00.873 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.873 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.873 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "5000", 2024-06-10T03:04:00.873 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.873 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.873 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.873 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-06-10T03:04:00.873 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.873 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.874 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.874 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.874 INFO:teuthology.orchestra.run.smithi052.stdout: "database": { 2024-06-10T03:04:00.874 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "database", 2024-06-10T03:04:00.874 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.874 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.874 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.874 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "ceph", 2024-06-10T03:04:00.874 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.874 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.874 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.874 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-06-10T03:04:00.875 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.875 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.875 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.875 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.875 INFO:teuthology.orchestra.run.smithi052.stdout: "hostname": { 2024-06-10T03:04:00.875 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "hostname", 2024-06-10T03:04:00.875 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.875 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.875 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.875 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.875 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.875 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.875 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.875 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "InfluxDB server hostname", 2024-06-10T03:04:00.876 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.876 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.876 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.876 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.876 INFO:teuthology.orchestra.run.smithi052.stdout: "interval": { 2024-06-10T03:04:00.876 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "interval", 2024-06-10T03:04:00.876 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-06-10T03:04:00.876 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.876 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.876 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "30", 2024-06-10T03:04:00.876 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "5", 2024-06-10T03:04:00.876 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.877 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.877 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-06-10T03:04:00.877 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.877 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.877 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.877 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.877 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-10T03:04:00.877 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-10T03:04:00.877 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.877 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.877 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.877 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.877 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.877 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.877 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:00.878 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:04:00.878 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:04:00.878 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:04:00.878 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:00.878 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:00.878 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:00.878 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:00.878 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.878 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.878 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.878 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.878 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.878 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-10T03:04:00.879 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-10T03:04:00.879 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.879 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.879 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.879 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.879 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.879 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.879 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.879 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.879 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.879 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.879 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.879 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.880 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-10T03:04:00.880 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-10T03:04:00.880 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.880 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.880 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.880 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-10T03:04:00.880 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.880 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.880 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:00.880 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:04:00.880 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:04:00.880 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:04:00.880 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:00.881 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:00.881 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:00.881 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:00.881 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.881 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.881 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.881 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.881 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.881 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-10T03:04:00.881 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-10T03:04:00.881 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.881 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.881 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.881 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.881 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.882 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.882 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.882 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.882 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.882 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.882 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.882 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.882 INFO:teuthology.orchestra.run.smithi052.stdout: "password": { 2024-06-10T03:04:00.882 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "password", 2024-06-10T03:04:00.882 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.882 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.882 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.882 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.882 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.883 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.883 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.883 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "password of InfluxDB server user", 2024-06-10T03:04:00.883 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.883 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.883 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.883 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.883 INFO:teuthology.orchestra.run.smithi052.stdout: "port": { 2024-06-10T03:04:00.883 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "port", 2024-06-10T03:04:00.883 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-10T03:04:00.883 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.883 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.883 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "8086", 2024-06-10T03:04:00.883 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.883 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.884 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.884 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "InfluxDB server port", 2024-06-10T03:04:00.884 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.884 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.884 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.884 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.884 INFO:teuthology.orchestra.run.smithi052.stdout: "ssl": { 2024-06-10T03:04:00.884 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "ssl", 2024-06-10T03:04:00.884 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.884 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.884 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.884 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "false", 2024-06-10T03:04:00.884 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.884 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.884 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.885 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-06-10T03:04:00.885 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.885 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.885 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.885 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.885 INFO:teuthology.orchestra.run.smithi052.stdout: "threads": { 2024-06-10T03:04:00.885 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "threads", 2024-06-10T03:04:00.885 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-10T03:04:00.885 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.885 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.885 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "5", 2024-06-10T03:04:00.885 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "1", 2024-06-10T03:04:00.885 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "32", 2024-06-10T03:04:00.885 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.886 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-06-10T03:04:00.886 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.886 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.886 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.886 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.886 INFO:teuthology.orchestra.run.smithi052.stdout: "username": { 2024-06-10T03:04:00.886 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "username", 2024-06-10T03:04:00.886 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.886 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.886 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.886 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.886 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.887 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.887 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.887 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "username of InfluxDB server user", 2024-06-10T03:04:00.887 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.887 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.887 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.887 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.887 INFO:teuthology.orchestra.run.smithi052.stdout: "verify_ssl": { 2024-06-10T03:04:00.887 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "verify_ssl", 2024-06-10T03:04:00.887 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.887 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.887 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.887 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "true", 2024-06-10T03:04:00.887 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.887 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.888 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.888 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-06-10T03:04:00.888 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.888 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.888 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.888 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:00.888 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:00.888 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.888 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-10T03:04:00.888 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "insights", 2024-06-10T03:04:00.888 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-10T03:04:00.888 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-10T03:04:00.888 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-10T03:04:00.888 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-10T03:04:00.889 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-10T03:04:00.889 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.889 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.889 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.889 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.889 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.889 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.889 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:00.889 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:04:00.889 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:04:00.889 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:04:00.889 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:00.889 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:00.890 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:00.890 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:00.890 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.890 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.890 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.890 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.890 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.890 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-10T03:04:00.890 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-10T03:04:00.890 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.890 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.890 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.891 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.891 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.891 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.891 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.891 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.891 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.891 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.891 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.891 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.891 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-10T03:04:00.891 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-10T03:04:00.891 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.891 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.891 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.892 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-10T03:04:00.892 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.892 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.892 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:00.892 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:04:00.892 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:04:00.892 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:04:00.892 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:00.892 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:00.892 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:00.892 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:00.892 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.892 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.892 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.892 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.893 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.893 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-10T03:04:00.893 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-10T03:04:00.893 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.893 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.893 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.893 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.893 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.893 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.893 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.893 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.893 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.893 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.893 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.894 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:00.894 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:00.894 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.894 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-10T03:04:00.894 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "iostat", 2024-06-10T03:04:00.894 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-10T03:04:00.894 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-10T03:04:00.894 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-10T03:04:00.894 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-10T03:04:00.894 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-10T03:04:00.894 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.894 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.894 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.895 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.895 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.895 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.895 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:00.895 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:04:00.895 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:04:00.895 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:04:00.895 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:00.895 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:00.895 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:00.895 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:00.895 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.895 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.895 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.896 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.896 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.896 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-10T03:04:00.896 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-10T03:04:00.896 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.896 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.896 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.896 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.896 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.896 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.896 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.896 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.896 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.896 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.897 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.897 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.897 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-10T03:04:00.897 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-10T03:04:00.897 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.897 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.897 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.897 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-10T03:04:00.897 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.897 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.897 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:00.897 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:04:00.897 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:04:00.898 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:04:00.898 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:00.898 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:00.898 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:00.898 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:00.898 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.898 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.898 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.898 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.898 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.898 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-10T03:04:00.898 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-10T03:04:00.898 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.899 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.899 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.899 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.899 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.899 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.899 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.899 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.899 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.899 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.899 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.899 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:00.899 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:00.899 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.899 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-10T03:04:00.900 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "k8sevents", 2024-06-10T03:04:00.900 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-10T03:04:00.900 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-10T03:04:00.900 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-10T03:04:00.900 INFO:teuthology.orchestra.run.smithi052.stdout: "ceph_event_retention_days": { 2024-06-10T03:04:00.900 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "ceph_event_retention_days", 2024-06-10T03:04:00.900 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-10T03:04:00.900 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.900 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.900 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "7", 2024-06-10T03:04:00.900 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.900 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.900 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.901 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Days to hold ceph event information within local cache", 2024-06-10T03:04:00.901 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.901 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.901 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.901 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.901 INFO:teuthology.orchestra.run.smithi052.stdout: "config_check_secs": { 2024-06-10T03:04:00.901 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "config_check_secs", 2024-06-10T03:04:00.901 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-10T03:04:00.901 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.901 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.901 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "10", 2024-06-10T03:04:00.901 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "10", 2024-06-10T03:04:00.901 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.901 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.901 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-06-10T03:04:00.902 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.902 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.902 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.902 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.902 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-10T03:04:00.902 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-10T03:04:00.902 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.902 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.902 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.902 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.902 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.902 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.902 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:00.903 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:04:00.903 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:04:00.903 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:04:00.903 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:00.903 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:00.903 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:00.903 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:00.903 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.903 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.903 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.903 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.903 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.903 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-10T03:04:00.903 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-10T03:04:00.904 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.904 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.904 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.904 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.904 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.904 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.904 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.904 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.904 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.904 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.904 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.904 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.904 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-10T03:04:00.904 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-10T03:04:00.904 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.905 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.905 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.905 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-10T03:04:00.905 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.905 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.905 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:00.905 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:04:00.905 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:04:00.905 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:04:00.905 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:00.905 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:00.905 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:00.905 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:00.906 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.906 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.906 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.906 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.906 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.906 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-10T03:04:00.906 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-10T03:04:00.906 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.906 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.906 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.906 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.906 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.906 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.907 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.907 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.907 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.907 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.907 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.907 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:00.907 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:00.907 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.907 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-10T03:04:00.907 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "localpool", 2024-06-10T03:04:00.907 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-10T03:04:00.907 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-10T03:04:00.907 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-10T03:04:00.907 INFO:teuthology.orchestra.run.smithi052.stdout: "failure_domain": { 2024-06-10T03:04:00.907 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "failure_domain", 2024-06-10T03:04:00.908 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.908 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.908 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.908 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "host", 2024-06-10T03:04:00.908 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.908 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.908 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.908 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "failure domain for any created local pool", 2024-06-10T03:04:00.908 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-06-10T03:04:00.908 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.908 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.908 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.908 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-10T03:04:00.908 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-10T03:04:00.909 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.909 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.909 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.909 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.909 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.909 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.909 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:00.909 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:04:00.909 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:04:00.909 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:04:00.909 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:00.909 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:00.909 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:00.909 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:00.910 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.910 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.910 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.910 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.910 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.910 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-10T03:04:00.910 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-10T03:04:00.910 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.910 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.910 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.910 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.910 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.910 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.911 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.911 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.911 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.911 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.911 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.911 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.911 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-10T03:04:00.911 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-10T03:04:00.911 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.911 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.911 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.911 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-10T03:04:00.911 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.912 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.912 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:00.912 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:04:00.912 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:04:00.912 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:04:00.912 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:00.912 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:00.912 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:00.912 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:00.912 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.912 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.912 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.913 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.913 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.913 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-10T03:04:00.913 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-10T03:04:00.913 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.913 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.913 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.913 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.913 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.913 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.913 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.913 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.913 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.914 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.914 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.914 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.914 INFO:teuthology.orchestra.run.smithi052.stdout: "min_size": { 2024-06-10T03:04:00.914 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "min_size", 2024-06-10T03:04:00.914 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-10T03:04:00.914 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.914 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.914 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.914 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.914 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.914 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.914 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "default min_size for any created local pool", 2024-06-10T03:04:00.915 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-06-10T03:04:00.915 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.915 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.915 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.915 INFO:teuthology.orchestra.run.smithi052.stdout: "num_rep": { 2024-06-10T03:04:00.915 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "num_rep", 2024-06-10T03:04:00.915 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-10T03:04:00.915 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.915 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.915 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "3", 2024-06-10T03:04:00.915 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.915 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.915 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.916 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "default replica count for any created local pool", 2024-06-10T03:04:00.916 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.916 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.916 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.916 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.916 INFO:teuthology.orchestra.run.smithi052.stdout: "pg_num": { 2024-06-10T03:04:00.916 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "pg_num", 2024-06-10T03:04:00.916 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-10T03:04:00.916 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.916 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.916 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "128", 2024-06-10T03:04:00.916 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.916 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.917 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.917 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "default pg_num for any created local pool", 2024-06-10T03:04:00.917 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.917 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.917 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.917 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.917 INFO:teuthology.orchestra.run.smithi052.stdout: "prefix": { 2024-06-10T03:04:00.917 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "prefix", 2024-06-10T03:04:00.917 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.917 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.917 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.917 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.917 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.917 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.918 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.918 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "name prefix for any created local pool", 2024-06-10T03:04:00.918 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.918 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.918 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.918 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.918 INFO:teuthology.orchestra.run.smithi052.stdout: "subtree": { 2024-06-10T03:04:00.918 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "subtree", 2024-06-10T03:04:00.918 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.918 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.918 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.918 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "rack", 2024-06-10T03:04:00.918 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.919 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.919 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.919 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "CRUSH level for which to create a local pool", 2024-06-10T03:04:00.919 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-06-10T03:04:00.919 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.919 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.919 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:00.919 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:00.919 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.919 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-10T03:04:00.919 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "mds_autoscaler", 2024-06-10T03:04:00.919 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-10T03:04:00.919 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-10T03:04:00.919 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-10T03:04:00.920 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-10T03:04:00.920 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-10T03:04:00.920 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.920 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.920 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.920 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.920 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.920 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.920 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:00.920 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:04:00.920 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:04:00.920 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:04:00.920 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:00.920 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:00.920 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:00.921 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:00.921 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.921 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.921 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.921 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.921 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.921 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-10T03:04:00.921 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-10T03:04:00.921 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.921 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.921 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.921 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.921 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.922 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.922 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.922 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.922 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.922 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.922 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.922 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.922 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-10T03:04:00.922 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-10T03:04:00.922 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.922 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.922 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.922 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-10T03:04:00.923 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.923 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.923 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:00.923 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:04:00.923 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:04:00.923 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:04:00.923 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:00.923 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:00.923 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:00.923 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:00.923 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.923 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.923 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.923 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.924 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.924 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-10T03:04:00.924 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-10T03:04:00.924 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.924 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.924 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.924 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.924 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.924 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.924 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.924 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.924 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.924 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.924 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.924 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:00.925 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:00.925 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.925 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-10T03:04:00.925 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "mirroring", 2024-06-10T03:04:00.925 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-10T03:04:00.925 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-10T03:04:00.925 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-10T03:04:00.925 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-10T03:04:00.925 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-10T03:04:00.925 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.925 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.925 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.925 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.925 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.926 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.926 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:00.926 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:04:00.926 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:04:00.926 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:04:00.926 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:00.926 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:00.926 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:00.926 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:00.926 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.926 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.926 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.926 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.927 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.927 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-10T03:04:00.927 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-10T03:04:00.927 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.927 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.927 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.927 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.927 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.927 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.927 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.927 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.927 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.927 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.928 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.928 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.928 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-10T03:04:00.928 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-10T03:04:00.928 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.928 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.928 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.928 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-10T03:04:00.928 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.928 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.928 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:00.928 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:04:00.928 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:04:00.928 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:04:00.929 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:00.929 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:00.929 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:00.929 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:00.929 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.929 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.929 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.929 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.929 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.929 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-10T03:04:00.929 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-10T03:04:00.929 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.929 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.929 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.929 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.930 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.930 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.930 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.930 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.930 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.930 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.930 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.930 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:00.930 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:00.930 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.930 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-10T03:04:00.930 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "nfs", 2024-06-10T03:04:00.930 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-10T03:04:00.931 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-10T03:04:00.931 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-10T03:04:00.931 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-10T03:04:00.931 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-10T03:04:00.931 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.931 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.931 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.931 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.931 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.931 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.931 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:00.931 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:04:00.931 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:04:00.931 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:04:00.932 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:00.932 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:00.934 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:00.935 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:00.935 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.935 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.935 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.935 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.935 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.935 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-10T03:04:00.935 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-10T03:04:00.935 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.935 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.935 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.935 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.936 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.936 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.936 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.936 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.936 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.936 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.936 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.936 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.936 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-10T03:04:00.936 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-10T03:04:00.936 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.936 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.936 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.937 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-10T03:04:00.937 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.937 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.937 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:00.937 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:04:00.937 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:04:00.937 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:04:00.937 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:00.937 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:00.937 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:00.937 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:00.937 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.937 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.938 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.938 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.938 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.938 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-10T03:04:00.938 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-10T03:04:00.938 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.938 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.938 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.938 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.938 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.938 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.938 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.939 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.939 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.939 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.939 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.939 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:00.939 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:00.939 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.939 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-10T03:04:00.939 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "orchestrator", 2024-06-10T03:04:00.939 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-10T03:04:00.939 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-10T03:04:00.939 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-10T03:04:00.939 INFO:teuthology.orchestra.run.smithi052.stdout: "fail_fs": { 2024-06-10T03:04:00.940 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "fail_fs", 2024-06-10T03:04:00.940 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.940 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.940 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.940 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.940 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.940 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.940 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.940 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-06-10T03:04:00.940 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.940 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.940 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.940 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.941 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-10T03:04:00.941 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-10T03:04:00.941 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.941 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.941 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.941 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.941 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.941 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.941 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:00.941 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:04:00.941 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:04:00.941 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:04:00.941 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:00.942 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:00.942 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:00.942 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:00.942 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.942 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.942 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.942 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.942 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.942 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-10T03:04:00.942 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-10T03:04:00.942 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.942 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.943 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.943 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.943 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.943 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.943 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.943 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.943 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.943 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.943 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.943 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.943 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-10T03:04:00.943 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-10T03:04:00.943 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.944 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.944 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.944 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-10T03:04:00.944 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.944 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.944 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:00.944 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:04:00.944 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:04:00.944 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:04:00.944 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:00.944 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:00.944 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:00.944 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:00.945 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.945 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.945 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.945 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.945 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.945 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-10T03:04:00.945 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-10T03:04:00.945 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.945 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.945 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.945 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.945 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.945 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.945 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.946 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.946 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.946 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.946 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.946 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.946 INFO:teuthology.orchestra.run.smithi052.stdout: "orchestrator": { 2024-06-10T03:04:00.946 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "orchestrator", 2024-06-10T03:04:00.946 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.946 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.946 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.946 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.946 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.947 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.947 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:00.947 INFO:teuthology.orchestra.run.smithi052.stdout: "cephadm", 2024-06-10T03:04:00.947 INFO:teuthology.orchestra.run.smithi052.stdout: "rook", 2024-06-10T03:04:00.947 INFO:teuthology.orchestra.run.smithi052.stdout: "test_orchestrator" 2024-06-10T03:04:00.947 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:00.947 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Orchestrator backend", 2024-06-10T03:04:00.947 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.947 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.947 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.947 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:00.947 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:00.947 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.948 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-10T03:04:00.948 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "osd_perf_query", 2024-06-10T03:04:00.948 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-10T03:04:00.948 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-10T03:04:00.948 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-10T03:04:00.948 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-10T03:04:00.948 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-10T03:04:00.948 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.948 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.948 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.948 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.948 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.948 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.949 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:00.949 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:04:00.949 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:04:00.949 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:04:00.949 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:00.949 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:00.949 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:00.949 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:00.949 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.949 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.949 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.949 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.949 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.949 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-10T03:04:00.950 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-10T03:04:00.950 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.950 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.950 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.950 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.950 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.950 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.950 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.950 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.950 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.950 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.950 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.950 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.951 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-10T03:04:00.951 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-10T03:04:00.951 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.951 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.951 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.951 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-10T03:04:00.951 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.951 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.951 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:00.951 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:04:00.951 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:04:00.951 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:04:00.951 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:00.952 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:00.952 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:00.952 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:00.952 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.952 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.952 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.952 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.952 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.952 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-10T03:04:00.952 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-10T03:04:00.952 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.952 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.952 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.952 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.953 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.953 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.953 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.953 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.953 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.953 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.953 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.953 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:00.953 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:00.953 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.953 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-10T03:04:00.953 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "osd_support", 2024-06-10T03:04:00.953 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-10T03:04:00.954 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-10T03:04:00.954 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-10T03:04:00.954 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-10T03:04:00.954 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-10T03:04:00.954 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.954 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.954 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.954 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.954 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.954 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.954 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:00.954 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:04:00.955 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:04:00.955 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:04:00.955 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:00.955 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:00.955 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:00.955 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:00.955 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.955 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.955 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.955 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.955 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.955 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-10T03:04:00.955 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-10T03:04:00.955 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.956 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.956 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.956 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.956 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.956 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.956 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.956 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.956 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.956 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.956 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.956 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.956 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-10T03:04:00.956 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-10T03:04:00.957 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.957 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.957 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.957 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-10T03:04:00.957 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.957 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.957 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:00.957 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:04:00.957 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:04:00.957 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:04:00.957 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:00.957 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:00.957 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:00.958 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:00.958 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.958 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.958 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.958 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.958 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.958 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-10T03:04:00.958 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-10T03:04:00.958 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.958 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.958 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.958 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.958 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.959 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.959 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.959 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.959 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.959 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.959 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.959 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:00.959 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:00.959 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.959 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-10T03:04:00.959 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "pg_autoscaler", 2024-06-10T03:04:00.959 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-10T03:04:00.959 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-10T03:04:00.959 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-10T03:04:00.960 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-10T03:04:00.960 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-10T03:04:00.960 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.960 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.960 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.960 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.960 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.960 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.960 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:00.960 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:04:00.960 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:04:00.960 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:04:00.960 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:00.961 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:00.961 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:00.961 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:00.961 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.961 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.961 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.961 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.961 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.961 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-10T03:04:00.961 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-10T03:04:00.961 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.961 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.961 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.961 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.962 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.962 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.962 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.962 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.962 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.962 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.962 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.962 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.962 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-10T03:04:00.962 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-10T03:04:00.962 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.962 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.962 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.963 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-10T03:04:00.963 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.963 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.963 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:00.963 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:04:00.963 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:04:00.963 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:04:00.963 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:00.963 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:00.963 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:00.963 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:00.963 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.963 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.964 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.964 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.964 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.964 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-10T03:04:00.964 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-10T03:04:00.964 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.964 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.964 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.964 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.964 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.964 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.964 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.964 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.964 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.965 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.965 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.965 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.965 INFO:teuthology.orchestra.run.smithi052.stdout: "sleep_interval": { 2024-06-10T03:04:00.965 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "sleep_interval", 2024-06-10T03:04:00.965 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-06-10T03:04:00.965 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.965 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.965 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "60", 2024-06-10T03:04:00.965 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.965 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.965 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.965 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.965 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.966 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.966 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.966 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.966 INFO:teuthology.orchestra.run.smithi052.stdout: "threshold": { 2024-06-10T03:04:00.966 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "threshold", 2024-06-10T03:04:00.966 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "float", 2024-06-10T03:04:00.966 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.966 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.966 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "3.0", 2024-06-10T03:04:00.966 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "1.0", 2024-06-10T03:04:00.966 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.966 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.966 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "scaling threshold", 2024-06-10T03:04:00.967 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-06-10T03:04:00.967 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.967 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.967 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:00.967 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:00.967 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.967 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-10T03:04:00.967 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "progress", 2024-06-10T03:04:00.967 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-10T03:04:00.967 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-10T03:04:00.967 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-10T03:04:00.967 INFO:teuthology.orchestra.run.smithi052.stdout: "allow_pg_recovery_event": { 2024-06-10T03:04:00.967 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "allow_pg_recovery_event", 2024-06-10T03:04:00.967 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.968 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.968 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.968 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.968 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.968 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.968 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.968 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "allow the module to show pg recovery progress", 2024-06-10T03:04:00.968 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.968 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.968 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.968 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.968 INFO:teuthology.orchestra.run.smithi052.stdout: "enabled": { 2024-06-10T03:04:00.969 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "enabled", 2024-06-10T03:04:00.969 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.969 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.969 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.969 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-10T03:04:00.969 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.969 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.969 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.969 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.969 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.969 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.969 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.969 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.970 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-10T03:04:00.970 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-10T03:04:00.970 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.970 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.970 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.970 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.970 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.970 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.970 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:00.970 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:04:00.970 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:04:00.970 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:04:00.970 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:00.971 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:00.971 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:00.971 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:00.971 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.971 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.971 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.971 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.971 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.971 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-10T03:04:00.971 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-10T03:04:00.971 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.971 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.971 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.971 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.972 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.972 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.972 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.972 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.972 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.972 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.972 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.972 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.972 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-10T03:04:00.972 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-10T03:04:00.972 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.972 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.972 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.972 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-10T03:04:00.973 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.973 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.973 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:00.973 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:04:00.973 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:04:00.973 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:04:00.973 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:00.973 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:00.973 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:00.973 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:00.973 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.973 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.973 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.974 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.974 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.974 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-10T03:04:00.974 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-10T03:04:00.974 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.974 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.974 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.974 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.974 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.974 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.974 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.974 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.974 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.975 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.975 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.975 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.975 INFO:teuthology.orchestra.run.smithi052.stdout: "max_completed_events": { 2024-06-10T03:04:00.975 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "max_completed_events", 2024-06-10T03:04:00.975 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-10T03:04:00.975 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.975 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.975 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "50", 2024-06-10T03:04:00.975 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.975 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.975 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.975 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "number of past completed events to remember", 2024-06-10T03:04:00.976 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.976 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.976 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.976 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.976 INFO:teuthology.orchestra.run.smithi052.stdout: "sleep_interval": { 2024-06-10T03:04:00.976 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "sleep_interval", 2024-06-10T03:04:00.976 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-06-10T03:04:00.976 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.976 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.976 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "5", 2024-06-10T03:04:00.976 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.976 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.976 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.976 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "how long the module is going to sleep", 2024-06-10T03:04:00.977 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.977 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.977 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.977 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:00.977 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:00.977 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.977 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-10T03:04:00.977 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "prometheus", 2024-06-10T03:04:00.977 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-10T03:04:00.977 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-10T03:04:00.977 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-10T03:04:00.977 INFO:teuthology.orchestra.run.smithi052.stdout: "cache": { 2024-06-10T03:04:00.977 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "cache", 2024-06-10T03:04:00.977 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.977 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.978 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.978 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-10T03:04:00.978 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.978 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.978 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.978 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.978 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.978 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.978 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.978 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.978 INFO:teuthology.orchestra.run.smithi052.stdout: "exclude_perf_counters": { 2024-06-10T03:04:00.978 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "exclude_perf_counters", 2024-06-10T03:04:00.978 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.979 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.979 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.979 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-10T03:04:00.979 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.979 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.979 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.979 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-06-10T03:04:00.979 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-06-10T03:04:00.979 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.979 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.979 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.979 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-10T03:04:00.979 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-10T03:04:00.980 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.980 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.980 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.980 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.980 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.980 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.980 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:00.980 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:04:00.980 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:04:00.980 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:04:00.980 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:00.980 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:00.980 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:00.981 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:00.981 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.981 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.981 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.981 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.981 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.981 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-10T03:04:00.981 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-10T03:04:00.981 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.981 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.981 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.981 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.981 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.981 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.982 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.982 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.982 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.982 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.982 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.982 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.982 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-10T03:04:00.982 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-10T03:04:00.982 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.982 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.982 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.982 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-10T03:04:00.982 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.983 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.983 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:00.983 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:04:00.983 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:04:00.983 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:04:00.983 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:00.983 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:00.983 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:00.983 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:00.983 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.983 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.983 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.983 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.983 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.983 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-10T03:04:00.984 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-10T03:04:00.984 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.984 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.984 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.984 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.984 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.984 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.984 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.984 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.984 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.984 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.984 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.984 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.985 INFO:teuthology.orchestra.run.smithi052.stdout: "rbd_stats_pools": { 2024-06-10T03:04:00.985 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rbd_stats_pools", 2024-06-10T03:04:00.985 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.985 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.985 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.985 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.985 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.985 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.985 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.985 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.985 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.985 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.985 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.985 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.985 INFO:teuthology.orchestra.run.smithi052.stdout: "rbd_stats_pools_refresh_interval": { 2024-06-10T03:04:00.986 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-06-10T03:04:00.986 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-10T03:04:00.986 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.986 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.986 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "300", 2024-06-10T03:04:00.986 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.986 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.986 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.986 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.986 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.986 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.986 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.987 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.987 INFO:teuthology.orchestra.run.smithi052.stdout: "scrape_interval": { 2024-06-10T03:04:00.987 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "scrape_interval", 2024-06-10T03:04:00.987 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "float", 2024-06-10T03:04:00.987 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.987 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.987 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "15.0", 2024-06-10T03:04:00.987 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.987 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.987 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.987 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.987 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.987 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.987 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.988 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.988 INFO:teuthology.orchestra.run.smithi052.stdout: "server_addr": { 2024-06-10T03:04:00.988 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "server_addr", 2024-06-10T03:04:00.988 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.988 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.988 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.988 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "::", 2024-06-10T03:04:00.988 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.988 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.988 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.988 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-06-10T03:04:00.988 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.988 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.988 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.989 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.989 INFO:teuthology.orchestra.run.smithi052.stdout: "server_port": { 2024-06-10T03:04:00.989 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "server_port", 2024-06-10T03:04:00.989 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-10T03:04:00.989 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.989 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.989 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "9283", 2024-06-10T03:04:00.989 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.989 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.989 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.989 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-06-10T03:04:00.989 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.989 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.989 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.989 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.990 INFO:teuthology.orchestra.run.smithi052.stdout: "stale_cache_strategy": { 2024-06-10T03:04:00.990 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "stale_cache_strategy", 2024-06-10T03:04:00.990 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.990 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.990 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.990 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "log", 2024-06-10T03:04:00.990 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.990 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.990 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.990 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.990 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.990 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.990 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.991 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.991 INFO:teuthology.orchestra.run.smithi052.stdout: "standby_behaviour": { 2024-06-10T03:04:00.991 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "standby_behaviour", 2024-06-10T03:04:00.991 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.991 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.991 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.991 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "default", 2024-06-10T03:04:00.991 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.991 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.991 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:00.991 INFO:teuthology.orchestra.run.smithi052.stdout: "default", 2024-06-10T03:04:00.991 INFO:teuthology.orchestra.run.smithi052.stdout: "error" 2024-06-10T03:04:00.991 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:00.991 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.991 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.992 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.992 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.992 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.992 INFO:teuthology.orchestra.run.smithi052.stdout: "standby_error_status_code": { 2024-06-10T03:04:00.992 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "standby_error_status_code", 2024-06-10T03:04:00.992 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-10T03:04:00.992 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.992 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.992 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "500", 2024-06-10T03:04:00.992 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "400", 2024-06-10T03:04:00.992 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "599", 2024-06-10T03:04:00.992 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.992 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.992 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.993 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.993 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.993 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:00.993 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:00.993 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.993 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-10T03:04:00.993 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rbd_support", 2024-06-10T03:04:00.993 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-10T03:04:00.993 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-10T03:04:00.993 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-10T03:04:00.993 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-10T03:04:00.993 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-10T03:04:00.993 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.994 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.994 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.994 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:00.994 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.994 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.994 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:00.994 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:04:00.994 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:04:00.994 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:04:00.994 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:00.994 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:00.994 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:00.994 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:00.995 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.995 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.995 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.995 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.995 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.995 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-10T03:04:00.995 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-10T03:04:00.995 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.995 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.995 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.995 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.995 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.996 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.996 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.996 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.996 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.996 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.996 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.996 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.996 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-10T03:04:00.996 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-10T03:04:00.996 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:00.996 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.996 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.997 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-10T03:04:00.997 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.997 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.997 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:00.997 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:04:00.997 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:04:00.997 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:04:00.997 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:00.997 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:00.997 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:00.997 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:00.997 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.997 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.998 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.998 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.998 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.998 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-10T03:04:00.998 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-10T03:04:00.998 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:00.998 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.998 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:00.998 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:00.998 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.998 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.998 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:00.998 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:00.999 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:00.999 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:00.999 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:00.999 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:00.999 INFO:teuthology.orchestra.run.smithi052.stdout: "max_concurrent_snap_create": { 2024-06-10T03:04:00.999 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "max_concurrent_snap_create", 2024-06-10T03:04:00.999 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-10T03:04:00.999 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:00.999 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:00.999 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "10", 2024-06-10T03:04:00.999 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:00.999 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:00.999 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:01.000 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:01.000 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:01.000 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:01.000 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:01.000 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:01.000 INFO:teuthology.orchestra.run.smithi052.stdout: "mirror_snapshot_schedule": { 2024-06-10T03:04:01.000 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "mirror_snapshot_schedule", 2024-06-10T03:04:01.000 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:01.000 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:01.000 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:01.000 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:01.000 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:01.000 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:01.001 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:01.001 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:01.001 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:01.001 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:01.001 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:01.001 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:01.001 INFO:teuthology.orchestra.run.smithi052.stdout: "trash_purge_schedule": { 2024-06-10T03:04:01.001 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "trash_purge_schedule", 2024-06-10T03:04:01.001 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:01.001 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:01.001 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:01.001 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:01.001 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:01.002 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:01.002 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:01.002 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:01.002 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:01.002 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:01.002 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:01.002 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:01.002 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:01.002 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:01.002 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-10T03:04:01.002 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "restful", 2024-06-10T03:04:01.002 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-10T03:04:01.002 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-10T03:04:01.003 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-10T03:04:01.003 INFO:teuthology.orchestra.run.smithi052.stdout: "enable_auth": { 2024-06-10T03:04:01.003 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "enable_auth", 2024-06-10T03:04:01.003 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:01.003 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:01.003 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:01.003 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-10T03:04:01.003 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:01.003 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:01.003 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:01.003 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:01.003 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:01.003 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:01.004 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:01.004 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:01.004 INFO:teuthology.orchestra.run.smithi052.stdout: "key_file": { 2024-06-10T03:04:01.004 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "key_file", 2024-06-10T03:04:01.004 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:01.004 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:01.004 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:01.004 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:01.004 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:01.004 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:01.004 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:01.004 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:01.004 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:01.004 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:01.005 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:01.005 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:01.005 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-10T03:04:01.005 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-10T03:04:01.005 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:01.005 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:01.005 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:01.005 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:01.005 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:01.005 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:01.005 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:01.005 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:04:01.005 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:04:01.005 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:04:01.006 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:01.006 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:01.006 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:01.006 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:01.006 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:01.006 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:01.006 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:01.006 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:01.006 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:01.006 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-10T03:04:01.006 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-10T03:04:01.006 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:01.007 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:01.007 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:01.007 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:01.007 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:01.007 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:01.007 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:01.007 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:01.007 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:01.007 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:01.007 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:01.007 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:01.007 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-10T03:04:01.007 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-10T03:04:01.008 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:01.008 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:01.008 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:01.008 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-10T03:04:01.008 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:01.008 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:01.008 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:01.008 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:04:01.008 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:04:01.008 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:04:01.008 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:01.008 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:01.008 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:01.008 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:01.009 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:01.009 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:01.009 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:01.009 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:01.009 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:01.009 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-10T03:04:01.009 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-10T03:04:01.009 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:01.009 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:01.009 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:01.009 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:01.009 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:01.009 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:01.009 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:01.010 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:01.010 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:01.010 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:01.010 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:01.010 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:01.010 INFO:teuthology.orchestra.run.smithi052.stdout: "server_addr": { 2024-06-10T03:04:01.010 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "server_addr", 2024-06-10T03:04:01.010 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:01.010 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:01.010 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:01.010 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:01.010 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:01.010 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:01.011 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:01.011 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:01.011 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:01.011 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:01.011 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:01.011 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:01.011 INFO:teuthology.orchestra.run.smithi052.stdout: "server_port": { 2024-06-10T03:04:01.011 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "server_port", 2024-06-10T03:04:01.011 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:01.011 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:01.011 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:01.011 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:01.011 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:01.011 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:01.012 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:01.012 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:01.012 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:01.012 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:01.012 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:01.012 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:01.012 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:01.012 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:01.012 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-10T03:04:01.012 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rgw", 2024-06-10T03:04:01.012 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-10T03:04:01.012 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-10T03:04:01.012 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-10T03:04:01.012 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-10T03:04:01.013 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-10T03:04:01.013 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:01.013 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:01.013 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:01.013 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:01.013 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:01.013 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:01.013 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:01.013 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:04:01.013 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:04:01.013 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:04:01.013 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:01.013 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:01.013 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:01.014 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:01.014 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:01.014 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:01.014 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:01.014 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:01.014 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:01.014 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-10T03:04:01.014 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-10T03:04:01.014 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:01.014 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:01.014 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:01.014 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:01.014 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:01.015 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:01.015 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:01.015 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:01.015 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:01.015 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:01.015 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:01.015 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:01.015 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-10T03:04:01.015 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-10T03:04:01.015 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:01.015 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:01.015 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:01.015 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-10T03:04:01.016 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:01.016 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:01.016 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:01.016 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:04:01.016 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:04:01.016 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:04:01.016 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:01.016 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:01.016 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:01.016 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:01.016 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:01.016 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:01.016 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:01.016 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:01.016 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:01.017 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-10T03:04:01.017 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-10T03:04:01.017 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:01.017 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:01.017 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:01.017 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:01.017 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:01.017 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:01.017 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:01.017 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:01.017 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:01.017 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:01.017 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:01.018 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:01.018 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:01.018 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:01.018 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-10T03:04:01.018 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rook", 2024-06-10T03:04:01.018 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-10T03:04:01.018 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-10T03:04:01.018 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-10T03:04:01.018 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-10T03:04:01.018 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-10T03:04:01.018 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:01.018 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:01.018 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:01.019 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:01.019 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:01.019 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:01.019 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:01.019 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:04:01.019 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:04:01.019 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:04:01.019 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:01.019 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:01.019 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:01.019 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:01.019 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:01.019 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:01.019 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:01.020 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:01.020 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:01.020 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-10T03:04:01.020 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-10T03:04:01.020 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:01.020 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:01.020 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:01.020 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:01.020 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:01.020 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:01.020 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:01.020 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:01.020 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:01.020 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:01.021 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:01.021 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:01.021 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-10T03:04:01.021 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-10T03:04:01.021 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:01.021 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:01.021 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:01.021 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-10T03:04:01.021 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:01.021 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:01.021 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:01.021 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:04:01.021 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:04:01.022 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:04:01.022 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:01.022 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:01.022 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:01.022 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:01.022 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:01.022 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:01.022 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:01.022 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:01.022 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:01.022 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-10T03:04:01.022 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-10T03:04:01.023 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:01.023 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:01.023 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:01.023 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:01.023 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:01.023 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:01.023 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:01.023 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:01.023 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:01.023 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:01.023 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:01.023 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:01.023 INFO:teuthology.orchestra.run.smithi052.stdout: "storage_class": { 2024-06-10T03:04:01.024 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "storage_class", 2024-06-10T03:04:01.024 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:01.024 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:01.024 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:01.024 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "local", 2024-06-10T03:04:01.024 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:01.024 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:01.024 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:01.024 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-06-10T03:04:01.024 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:01.024 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:01.024 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:01.024 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:01.024 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:01.025 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:01.025 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-10T03:04:01.025 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "selftest", 2024-06-10T03:04:01.025 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-10T03:04:01.025 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-10T03:04:01.025 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-10T03:04:01.025 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-10T03:04:01.025 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-10T03:04:01.025 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:01.025 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:01.025 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:01.025 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:01.025 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:01.025 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:01.026 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:01.026 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:04:01.026 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:04:01.026 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:04:01.026 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:01.026 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:01.026 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:01.026 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:01.026 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:01.026 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:01.026 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:01.026 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:01.026 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:01.027 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-10T03:04:01.027 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-10T03:04:01.027 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:01.027 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:01.027 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:01.027 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:01.027 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:01.027 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:01.027 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:01.027 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:01.027 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:01.027 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:01.027 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:01.028 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:01.028 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-10T03:04:01.028 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-10T03:04:01.028 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:01.028 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:01.028 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:01.028 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-10T03:04:01.028 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:01.028 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:01.028 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:01.028 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:04:01.028 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:04:01.028 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:04:01.029 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:01.029 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:01.029 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:01.029 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:01.029 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:01.029 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:01.029 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:01.029 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:01.029 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:01.029 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-10T03:04:01.029 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-10T03:04:01.029 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:01.029 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:01.029 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:01.029 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:01.030 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:01.030 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:01.030 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:01.030 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:01.030 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:01.030 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:01.030 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:01.030 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:01.030 INFO:teuthology.orchestra.run.smithi052.stdout: "roption1": { 2024-06-10T03:04:01.030 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "roption1", 2024-06-10T03:04:01.030 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:01.030 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:01.030 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:01.031 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:01.031 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:01.031 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:01.031 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:01.031 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:01.031 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:01.031 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:01.031 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:01.031 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:01.031 INFO:teuthology.orchestra.run.smithi052.stdout: "roption2": { 2024-06-10T03:04:01.031 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "roption2", 2024-06-10T03:04:01.031 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:01.031 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:01.032 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:01.032 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "xyz", 2024-06-10T03:04:01.032 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:01.032 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:01.032 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:01.032 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:01.032 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:01.032 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:01.032 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:01.032 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:01.032 INFO:teuthology.orchestra.run.smithi052.stdout: "rwoption1": { 2024-06-10T03:04:01.032 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rwoption1", 2024-06-10T03:04:01.032 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:01.033 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:01.033 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:01.033 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:01.033 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:01.033 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:01.033 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:01.033 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:01.033 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:01.033 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:01.033 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:01.033 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:01.033 INFO:teuthology.orchestra.run.smithi052.stdout: "rwoption2": { 2024-06-10T03:04:01.033 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rwoption2", 2024-06-10T03:04:01.034 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-10T03:04:01.034 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:01.034 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:01.034 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:01.034 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:01.034 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:01.034 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:01.034 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:01.034 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:01.034 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:01.034 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:01.034 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:01.034 INFO:teuthology.orchestra.run.smithi052.stdout: "rwoption3": { 2024-06-10T03:04:01.035 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rwoption3", 2024-06-10T03:04:01.035 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "float", 2024-06-10T03:04:01.035 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:01.035 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:01.035 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:01.035 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:01.035 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:01.035 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:01.035 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:01.035 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:01.035 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:01.035 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:01.035 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:01.036 INFO:teuthology.orchestra.run.smithi052.stdout: "rwoption4": { 2024-06-10T03:04:01.036 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rwoption4", 2024-06-10T03:04:01.036 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:01.036 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:01.036 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:01.036 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:01.036 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:01.036 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:01.036 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:01.036 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:01.036 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:01.036 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:01.036 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:01.037 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:01.037 INFO:teuthology.orchestra.run.smithi052.stdout: "rwoption5": { 2024-06-10T03:04:01.037 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rwoption5", 2024-06-10T03:04:01.037 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:01.037 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:01.037 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:01.037 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:01.037 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:01.037 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:01.037 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:01.037 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:01.037 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:01.037 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:01.037 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:01.037 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:01.038 INFO:teuthology.orchestra.run.smithi052.stdout: "rwoption6": { 2024-06-10T03:04:01.038 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rwoption6", 2024-06-10T03:04:01.038 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:01.038 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:01.038 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:01.038 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-10T03:04:01.038 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:01.038 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:01.038 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:01.038 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:01.038 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:01.038 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:01.039 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:01.039 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:01.039 INFO:teuthology.orchestra.run.smithi052.stdout: "rwoption7": { 2024-06-10T03:04:01.039 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rwoption7", 2024-06-10T03:04:01.039 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-10T03:04:01.039 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:01.039 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:01.039 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:01.039 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "1", 2024-06-10T03:04:01.039 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "42", 2024-06-10T03:04:01.039 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:01.039 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:01.039 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:01.039 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:01.040 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:01.040 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:01.040 INFO:teuthology.orchestra.run.smithi052.stdout: "testkey": { 2024-06-10T03:04:01.040 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "testkey", 2024-06-10T03:04:01.040 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:01.040 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:01.040 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:01.040 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:01.040 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:01.040 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:01.040 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:01.040 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:01.041 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:01.041 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:01.041 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:01.041 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:01.041 INFO:teuthology.orchestra.run.smithi052.stdout: "testlkey": { 2024-06-10T03:04:01.041 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "testlkey", 2024-06-10T03:04:01.041 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:01.041 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:01.041 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:01.041 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:01.041 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:01.041 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:01.041 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:01.041 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:01.042 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:01.042 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:01.042 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:01.042 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:01.042 INFO:teuthology.orchestra.run.smithi052.stdout: "testnewline": { 2024-06-10T03:04:01.042 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "testnewline", 2024-06-10T03:04:01.042 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:01.042 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:01.042 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:01.042 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:01.042 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:01.042 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:01.042 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:01.043 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:01.043 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:01.043 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:01.043 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:01.043 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:01.043 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:01.043 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:01.043 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-10T03:04:01.043 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "snap_schedule", 2024-06-10T03:04:01.043 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-10T03:04:01.043 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-10T03:04:01.043 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-10T03:04:01.043 INFO:teuthology.orchestra.run.smithi052.stdout: "allow_m_granularity": { 2024-06-10T03:04:01.044 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "allow_m_granularity", 2024-06-10T03:04:01.044 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:01.044 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:01.044 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:01.044 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:01.044 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:01.044 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:01.044 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:01.044 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "allow minute scheduled snapshots", 2024-06-10T03:04:01.044 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:01.044 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:01.044 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:01.044 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:01.044 INFO:teuthology.orchestra.run.smithi052.stdout: "dump_on_update": { 2024-06-10T03:04:01.045 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "dump_on_update", 2024-06-10T03:04:01.045 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:01.045 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:01.045 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:01.045 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:01.045 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:01.045 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:01.045 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:01.045 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "dump database to debug log on update", 2024-06-10T03:04:01.045 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:01.045 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:01.045 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:01.045 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:01.046 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-10T03:04:01.046 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-10T03:04:01.046 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:01.046 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:01.046 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:01.046 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:01.046 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:01.046 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:01.046 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:01.046 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:04:01.046 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:04:01.046 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:04:01.046 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:01.047 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:01.047 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:01.047 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:01.047 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:01.047 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:01.047 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:01.047 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:01.047 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:01.047 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-10T03:04:01.047 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-10T03:04:01.047 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:01.047 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:01.047 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:01.047 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:01.048 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:01.048 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:01.048 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:01.048 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:01.048 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:01.048 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:01.048 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:01.048 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:01.048 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-10T03:04:01.048 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-10T03:04:01.048 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:01.048 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:01.048 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:01.049 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-10T03:04:01.049 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:01.049 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:01.049 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:01.049 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:04:01.049 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:04:01.049 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:04:01.049 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:01.049 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:01.049 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:01.049 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:01.049 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:01.049 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:01.050 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:01.050 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:01.050 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:01.050 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-10T03:04:01.050 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-10T03:04:01.050 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:01.050 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:01.050 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:01.050 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:01.050 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:01.050 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:01.050 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:01.050 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:01.051 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:01.051 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:01.051 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:01.051 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:01.051 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:01.051 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:01.051 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-10T03:04:01.051 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "stats", 2024-06-10T03:04:01.051 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-10T03:04:01.051 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-10T03:04:01.051 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-10T03:04:01.051 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-10T03:04:01.051 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-10T03:04:01.052 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:01.052 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:01.052 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:01.052 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:01.052 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:01.052 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:01.052 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:01.052 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:04:01.052 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:04:01.052 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:04:01.052 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:01.052 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:01.052 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:01.053 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:01.053 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:01.053 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:01.053 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:01.053 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:01.053 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:01.053 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-10T03:04:01.053 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-10T03:04:01.053 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:01.053 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:01.053 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:01.053 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:01.053 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:01.054 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:01.054 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:01.054 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:01.054 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:01.054 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:01.054 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:01.054 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:01.054 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-10T03:04:01.054 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-10T03:04:01.054 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:01.054 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:01.054 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:01.054 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-10T03:04:01.055 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:01.055 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:01.058 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:01.058 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:04:01.058 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:04:01.058 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:04:01.058 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:01.058 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:01.058 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:01.058 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:01.058 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:01.059 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:01.059 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:01.059 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:01.059 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:01.059 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-10T03:04:01.059 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-10T03:04:01.059 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:01.059 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:01.059 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:01.059 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:01.059 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:01.059 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:01.059 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:01.060 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:01.060 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:01.060 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:01.060 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:01.060 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:01.060 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:01.060 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:01.060 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-10T03:04:01.060 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "status", 2024-06-10T03:04:01.060 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-10T03:04:01.060 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-10T03:04:01.060 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-10T03:04:01.060 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-10T03:04:01.061 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-10T03:04:01.061 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:01.061 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:01.061 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:01.061 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:01.061 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:01.061 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:01.061 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:01.061 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:04:01.061 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:04:01.061 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:04:01.061 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:01.061 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:01.061 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:01.061 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:01.062 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:01.062 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:01.062 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:01.062 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:01.062 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:01.062 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-10T03:04:01.062 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-10T03:04:01.062 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:01.062 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:01.062 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:01.062 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:01.062 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:01.063 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:01.063 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:01.063 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:01.063 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:01.063 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:01.063 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:01.063 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:01.063 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-10T03:04:01.063 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-10T03:04:01.063 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:01.063 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:01.063 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:01.063 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-10T03:04:01.063 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:01.063 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:01.064 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:01.064 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:04:01.064 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:04:01.064 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:04:01.064 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:01.064 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:01.064 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:01.064 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:01.064 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:01.064 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:01.064 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:01.064 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:01.064 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:01.064 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-10T03:04:01.065 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-10T03:04:01.065 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:01.065 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:01.065 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:01.065 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:01.065 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:01.065 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:01.065 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:01.065 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:01.065 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:01.065 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:01.065 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:01.065 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:01.065 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:01.066 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:01.066 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-10T03:04:01.066 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "telegraf", 2024-06-10T03:04:01.066 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-10T03:04:01.066 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-10T03:04:01.066 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-10T03:04:01.066 INFO:teuthology.orchestra.run.smithi052.stdout: "address": { 2024-06-10T03:04:01.066 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "address", 2024-06-10T03:04:01.066 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:01.066 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:01.066 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:01.066 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-06-10T03:04:01.066 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:01.067 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:01.067 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:01.067 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:01.067 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:01.067 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:01.067 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:01.067 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:01.067 INFO:teuthology.orchestra.run.smithi052.stdout: "interval": { 2024-06-10T03:04:01.067 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "interval", 2024-06-10T03:04:01.067 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-06-10T03:04:01.067 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:01.067 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:01.067 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "15", 2024-06-10T03:04:01.068 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:01.068 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:01.068 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:01.068 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:01.068 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:01.068 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:01.068 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:01.068 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:01.068 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-10T03:04:01.068 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-10T03:04:01.068 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:01.068 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:01.068 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:01.069 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:01.069 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:01.069 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:01.069 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:01.069 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:04:01.069 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:04:01.069 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:04:01.069 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:01.069 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:01.069 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:01.069 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:01.069 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:01.069 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:01.070 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:01.070 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:01.070 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:01.070 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-10T03:04:01.070 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-10T03:04:01.070 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:01.070 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:01.070 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:01.070 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:01.070 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:01.070 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:01.070 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:01.071 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:01.071 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:01.071 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:01.071 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:01.071 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:01.071 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-10T03:04:01.071 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-10T03:04:01.071 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:01.071 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:01.071 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:01.071 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-10T03:04:01.071 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:01.071 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:01.072 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:01.072 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:04:01.072 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:04:01.072 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:04:01.072 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:01.072 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:01.072 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:01.072 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:01.072 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:01.072 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:01.072 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:01.072 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:01.072 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:01.073 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-10T03:04:01.073 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-10T03:04:01.073 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:01.073 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:01.073 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:01.073 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:01.073 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:01.073 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:01.073 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:01.073 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:01.073 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:01.073 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:01.073 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:01.074 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:01.074 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:01.074 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:01.074 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-10T03:04:01.074 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "telemetry", 2024-06-10T03:04:01.074 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-10T03:04:01.074 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-10T03:04:01.074 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-10T03:04:01.074 INFO:teuthology.orchestra.run.smithi052.stdout: "channel_basic": { 2024-06-10T03:04:01.074 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "channel_basic", 2024-06-10T03:04:01.074 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:01.074 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:01.074 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:01.075 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-10T03:04:01.075 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:01.075 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:01.075 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:01.075 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Share basic cluster information (size, version)", 2024-06-10T03:04:01.075 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:01.075 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:01.075 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:01.075 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:01.075 INFO:teuthology.orchestra.run.smithi052.stdout: "channel_crash": { 2024-06-10T03:04:01.075 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "channel_crash", 2024-06-10T03:04:01.075 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:01.075 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:01.075 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:01.076 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-10T03:04:01.076 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:01.076 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:01.076 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:01.076 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-06-10T03:04:01.076 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:01.076 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:01.076 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:01.076 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:01.076 INFO:teuthology.orchestra.run.smithi052.stdout: "channel_device": { 2024-06-10T03:04:01.076 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "channel_device", 2024-06-10T03:04:01.076 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:01.076 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:01.077 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:01.077 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-10T03:04:01.077 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:01.077 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:01.077 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:01.077 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-06-10T03:04:01.077 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:01.077 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:01.077 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:01.077 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:01.077 INFO:teuthology.orchestra.run.smithi052.stdout: "channel_ident": { 2024-06-10T03:04:01.077 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "channel_ident", 2024-06-10T03:04:01.077 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:01.077 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:01.078 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:01.078 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:01.078 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:01.078 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:01.078 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:01.078 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-06-10T03:04:01.078 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:01.078 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:01.078 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:01.078 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:01.078 INFO:teuthology.orchestra.run.smithi052.stdout: "channel_perf": { 2024-06-10T03:04:01.078 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "channel_perf", 2024-06-10T03:04:01.078 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:01.079 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:01.079 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:01.079 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:01.079 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:01.079 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:01.079 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:01.079 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Share various performance metrics of a cluster", 2024-06-10T03:04:01.079 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:01.079 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:01.079 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:01.079 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:01.079 INFO:teuthology.orchestra.run.smithi052.stdout: "contact": { 2024-06-10T03:04:01.079 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "contact", 2024-06-10T03:04:01.079 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:01.080 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:01.080 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:01.080 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:01.080 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:01.080 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:01.080 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:01.080 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:01.080 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:01.080 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:01.080 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:01.080 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:01.080 INFO:teuthology.orchestra.run.smithi052.stdout: "description": { 2024-06-10T03:04:01.080 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "description", 2024-06-10T03:04:01.081 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:01.081 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:01.081 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:01.081 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:01.081 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:01.081 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:01.081 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:01.081 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:01.081 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:01.081 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:01.081 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:01.081 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:01.081 INFO:teuthology.orchestra.run.smithi052.stdout: "device_url": { 2024-06-10T03:04:01.082 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "device_url", 2024-06-10T03:04:01.082 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:01.082 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:01.082 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:01.082 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-06-10T03:04:01.082 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:01.082 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:01.082 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:01.082 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:01.082 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:01.082 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:01.082 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:01.082 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:01.083 INFO:teuthology.orchestra.run.smithi052.stdout: "enabled": { 2024-06-10T03:04:01.083 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "enabled", 2024-06-10T03:04:01.083 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:01.083 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:01.083 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:01.083 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:01.083 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:01.083 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:01.083 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:01.083 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:01.083 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:01.083 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:01.083 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:01.083 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:01.084 INFO:teuthology.orchestra.run.smithi052.stdout: "interval": { 2024-06-10T03:04:01.084 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "interval", 2024-06-10T03:04:01.084 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-10T03:04:01.084 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:01.084 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:01.084 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "24", 2024-06-10T03:04:01.084 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "8", 2024-06-10T03:04:01.084 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:01.084 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:01.084 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:01.084 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:01.084 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:01.084 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:01.085 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:01.085 INFO:teuthology.orchestra.run.smithi052.stdout: "last_opt_revision": { 2024-06-10T03:04:01.085 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "last_opt_revision", 2024-06-10T03:04:01.085 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-10T03:04:01.085 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:01.085 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:01.085 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "1", 2024-06-10T03:04:01.085 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:01.085 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:01.085 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:01.085 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:01.085 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:01.085 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:01.086 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:01.086 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:01.086 INFO:teuthology.orchestra.run.smithi052.stdout: "leaderboard": { 2024-06-10T03:04:01.086 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "leaderboard", 2024-06-10T03:04:01.086 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:01.086 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:01.086 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:01.086 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:01.086 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:01.086 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:01.086 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:01.086 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:01.086 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:01.086 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:01.087 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:01.087 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:01.087 INFO:teuthology.orchestra.run.smithi052.stdout: "leaderboard_description": { 2024-06-10T03:04:01.087 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "leaderboard_description", 2024-06-10T03:04:01.087 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:01.087 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:01.087 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:01.087 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:01.087 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:01.087 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:01.087 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:01.087 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:01.087 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:01.088 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:01.088 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:01.088 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:01.088 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-10T03:04:01.088 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-10T03:04:01.088 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:01.088 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:01.088 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:01.088 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:01.088 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:01.088 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:01.088 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:01.088 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:04:01.089 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:04:01.089 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:04:01.089 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:01.089 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:01.089 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:01.089 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:01.089 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:01.089 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:01.089 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:01.089 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:01.089 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:01.089 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-10T03:04:01.089 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-10T03:04:01.090 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:01.090 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:01.090 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:01.090 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:01.090 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:01.090 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:01.090 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:01.090 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:01.090 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:01.090 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:01.090 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:01.090 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:01.090 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-10T03:04:01.091 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-10T03:04:01.091 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:01.091 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:01.091 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:01.091 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-10T03:04:01.091 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:01.091 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:01.091 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:01.091 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:04:01.091 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:04:01.091 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:04:01.091 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:01.091 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:01.092 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:01.092 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:01.092 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:01.092 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:01.092 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:01.092 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:01.092 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:01.092 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-10T03:04:01.092 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-10T03:04:01.092 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:01.092 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:01.092 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:01.092 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:01.092 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:01.093 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:01.093 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:01.093 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:01.093 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:01.093 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:01.093 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:01.093 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:01.093 INFO:teuthology.orchestra.run.smithi052.stdout: "organization": { 2024-06-10T03:04:01.093 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "organization", 2024-06-10T03:04:01.093 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:01.093 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:01.093 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:01.093 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:01.094 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:01.094 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:01.094 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:01.094 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:01.094 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:01.094 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:01.094 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:01.094 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:01.094 INFO:teuthology.orchestra.run.smithi052.stdout: "proxy": { 2024-06-10T03:04:01.094 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "proxy", 2024-06-10T03:04:01.094 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:01.094 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:01.095 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:01.095 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:01.095 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:01.095 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:01.095 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:01.095 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:01.095 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:01.095 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:01.095 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:01.095 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:01.095 INFO:teuthology.orchestra.run.smithi052.stdout: "url": { 2024-06-10T03:04:01.095 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "url", 2024-06-10T03:04:01.095 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:01.095 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:01.096 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:01.096 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-06-10T03:04:01.096 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:01.096 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:01.096 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:01.096 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:01.096 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:01.096 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:01.096 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:01.096 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:01.096 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:01.096 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:01.097 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-10T03:04:01.097 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "test_orchestrator", 2024-06-10T03:04:01.097 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-10T03:04:01.097 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-10T03:04:01.097 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-10T03:04:01.097 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-10T03:04:01.097 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-10T03:04:01.097 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:01.097 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:01.097 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:01.097 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:01.097 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:01.097 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:01.097 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:01.098 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:04:01.098 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:04:01.098 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:04:01.098 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:01.098 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:01.098 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:01.098 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:01.098 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:01.098 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:01.098 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:01.098 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:01.098 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:01.099 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-10T03:04:01.099 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-10T03:04:01.099 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:01.099 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:01.099 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:01.099 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:01.099 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:01.099 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:01.099 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:01.099 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:01.099 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:01.099 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:01.099 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:01.099 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:01.099 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-10T03:04:01.100 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-10T03:04:01.100 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:01.100 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:01.100 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:01.100 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-10T03:04:01.100 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:01.100 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:01.100 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:01.100 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:04:01.100 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:04:01.100 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:04:01.100 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:01.100 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:01.101 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:01.101 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:01.101 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:01.101 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:01.101 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:01.101 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:01.101 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:01.101 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-10T03:04:01.101 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-10T03:04:01.101 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:01.101 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:01.101 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:01.101 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:01.101 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:01.102 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:01.102 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:01.102 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:01.102 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:01.102 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:01.102 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:01.102 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:01.102 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:01.102 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:01.102 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-10T03:04:01.102 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "volumes", 2024-06-10T03:04:01.102 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-10T03:04:01.102 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-10T03:04:01.103 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-10T03:04:01.103 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-10T03:04:01.103 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-10T03:04:01.103 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:01.103 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:01.103 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:01.103 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:01.103 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:01.103 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:01.103 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:01.103 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:04:01.103 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:04:01.103 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:04:01.103 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:01.104 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:01.104 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:01.104 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:01.104 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:01.104 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:01.104 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:01.104 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:01.104 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:01.104 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-10T03:04:01.104 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-10T03:04:01.104 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:01.104 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:01.104 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:01.104 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:01.105 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:01.105 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:01.105 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:01.105 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:01.105 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:01.105 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:01.105 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:01.105 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:01.105 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-10T03:04:01.105 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-10T03:04:01.105 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:01.105 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:01.105 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:01.105 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-10T03:04:01.106 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:01.106 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:01.106 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:01.106 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:04:01.106 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:04:01.106 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:04:01.106 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:01.106 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:01.106 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:01.106 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:01.106 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:01.106 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:01.106 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:01.107 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:01.107 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:01.107 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-10T03:04:01.107 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-10T03:04:01.107 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:01.107 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:01.107 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:01.107 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:01.107 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:01.107 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:01.107 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:01.107 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:01.107 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:01.108 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:01.108 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:01.108 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:01.108 INFO:teuthology.orchestra.run.smithi052.stdout: "max_concurrent_clones": { 2024-06-10T03:04:01.108 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "max_concurrent_clones", 2024-06-10T03:04:01.108 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-10T03:04:01.108 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:01.108 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:01.108 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "4", 2024-06-10T03:04:01.108 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:01.108 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:01.108 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:01.108 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Number of asynchronous cloner threads", 2024-06-10T03:04:01.108 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:01.109 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:01.109 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:01.109 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:01.109 INFO:teuthology.orchestra.run.smithi052.stdout: "periodic_async_work": { 2024-06-10T03:04:01.109 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "periodic_async_work", 2024-06-10T03:04:01.109 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:01.109 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:01.109 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:01.109 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:01.109 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:01.109 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:01.109 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:01.109 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Periodically check for async work", 2024-06-10T03:04:01.109 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:01.110 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:01.110 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:01.110 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:01.110 INFO:teuthology.orchestra.run.smithi052.stdout: "snapshot_clone_delay": { 2024-06-10T03:04:01.110 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "snapshot_clone_delay", 2024-06-10T03:04:01.110 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-10T03:04:01.110 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:01.110 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:01.110 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "0", 2024-06-10T03:04:01.110 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:01.110 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:01.110 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:01.110 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-06-10T03:04:01.110 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:01.111 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:01.111 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:01.111 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:01.111 INFO:teuthology.orchestra.run.smithi052.stdout: "snapshot_clone_no_wait": { 2024-06-10T03:04:01.111 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "snapshot_clone_no_wait", 2024-06-10T03:04:01.111 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:01.111 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:01.111 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:01.111 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-10T03:04:01.111 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:01.111 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:01.111 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:01.111 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-06-10T03:04:01.111 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:01.112 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:01.112 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:01.112 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:01.112 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:01.112 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:01.112 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-10T03:04:01.112 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "zabbix", 2024-06-10T03:04:01.112 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-10T03:04:01.112 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-10T03:04:01.112 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-10T03:04:01.112 INFO:teuthology.orchestra.run.smithi052.stdout: "discovery_interval": { 2024-06-10T03:04:01.112 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "discovery_interval", 2024-06-10T03:04:01.112 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "uint", 2024-06-10T03:04:01.112 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:01.113 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:01.113 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "100", 2024-06-10T03:04:01.113 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:01.113 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:01.113 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:01.113 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:01.113 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:01.113 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:01.113 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:01.113 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:01.113 INFO:teuthology.orchestra.run.smithi052.stdout: "identifier": { 2024-06-10T03:04:01.113 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "identifier", 2024-06-10T03:04:01.113 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:01.113 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:01.114 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:01.114 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:01.114 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:01.114 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:01.114 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:01.114 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:01.114 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:01.114 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:01.114 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:01.114 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:01.114 INFO:teuthology.orchestra.run.smithi052.stdout: "interval": { 2024-06-10T03:04:01.114 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "interval", 2024-06-10T03:04:01.114 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-06-10T03:04:01.115 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:01.115 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:01.115 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "60", 2024-06-10T03:04:01.115 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:01.115 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:01.115 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:01.115 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:01.115 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:01.115 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:01.115 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:01.115 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:01.115 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-10T03:04:01.115 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-10T03:04:01.115 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:01.116 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:01.116 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:01.116 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:01.116 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:01.116 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:01.116 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:01.116 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:04:01.116 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:04:01.116 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:04:01.116 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:01.116 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:01.116 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:01.116 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:01.116 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:01.117 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:01.117 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:01.117 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:01.117 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:01.117 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-10T03:04:01.117 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-10T03:04:01.117 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:01.117 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:01.117 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:01.117 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:01.117 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:01.117 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:01.117 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:01.117 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:01.118 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:01.118 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:01.118 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:01.118 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:01.118 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-10T03:04:01.118 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-10T03:04:01.118 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:01.118 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:01.118 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:01.118 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-10T03:04:01.118 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:01.118 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:01.118 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-10T03:04:01.118 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-10T03:04:01.119 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-10T03:04:01.119 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-10T03:04:01.119 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-10T03:04:01.119 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-10T03:04:01.119 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-10T03:04:01.119 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:01.119 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:01.119 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:01.119 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:01.119 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:01.119 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:01.119 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-10T03:04:01.119 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-10T03:04:01.119 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-10T03:04:01.120 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:01.120 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-10T03:04:01.120 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-10T03:04:01.120 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:01.120 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:01.120 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:01.120 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:01.120 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:01.120 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:01.120 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:01.120 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:01.120 INFO:teuthology.orchestra.run.smithi052.stdout: "zabbix_host": { 2024-06-10T03:04:01.120 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "zabbix_host", 2024-06-10T03:04:01.120 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:01.121 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:01.121 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:01.121 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-10T03:04:01.121 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:01.121 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:01.121 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:01.121 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:01.121 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:01.121 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:01.121 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:01.121 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:01.121 INFO:teuthology.orchestra.run.smithi052.stdout: "zabbix_port": { 2024-06-10T03:04:01.121 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "zabbix_port", 2024-06-10T03:04:01.121 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-10T03:04:01.122 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:01.122 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:01.122 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "10051", 2024-06-10T03:04:01.122 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:01.122 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:01.122 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:01.122 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:01.122 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:01.122 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:01.122 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:01.122 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:01.122 INFO:teuthology.orchestra.run.smithi052.stdout: "zabbix_sender": { 2024-06-10T03:04:01.122 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "zabbix_sender", 2024-06-10T03:04:01.122 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-10T03:04:01.123 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-10T03:04:01.123 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-10T03:04:01.123 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-06-10T03:04:01.123 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-10T03:04:01.123 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-10T03:04:01.123 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-10T03:04:01.123 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-10T03:04:01.123 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-10T03:04:01.123 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-10T03:04:01.123 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-10T03:04:01.123 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:01.123 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:01.123 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:01.123 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:01.124 INFO:teuthology.orchestra.run.smithi052.stdout: "services": { 2024-06-10T03:04:01.124 INFO:teuthology.orchestra.run.smithi052.stdout: "dashboard": "https://172.21.15.52:8443/" 2024-06-10T03:04:01.124 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:01.124 INFO:teuthology.orchestra.run.smithi052.stdout: "always_on_modules": { 2024-06-10T03:04:01.124 INFO:teuthology.orchestra.run.smithi052.stdout: "octopus": [ 2024-06-10T03:04:01.124 INFO:teuthology.orchestra.run.smithi052.stdout: "balancer", 2024-06-10T03:04:01.124 INFO:teuthology.orchestra.run.smithi052.stdout: "crash", 2024-06-10T03:04:01.124 INFO:teuthology.orchestra.run.smithi052.stdout: "devicehealth", 2024-06-10T03:04:01.124 INFO:teuthology.orchestra.run.smithi052.stdout: "orchestrator", 2024-06-10T03:04:01.124 INFO:teuthology.orchestra.run.smithi052.stdout: "pg_autoscaler", 2024-06-10T03:04:01.124 INFO:teuthology.orchestra.run.smithi052.stdout: "progress", 2024-06-10T03:04:01.124 INFO:teuthology.orchestra.run.smithi052.stdout: "rbd_support", 2024-06-10T03:04:01.124 INFO:teuthology.orchestra.run.smithi052.stdout: "status", 2024-06-10T03:04:01.124 INFO:teuthology.orchestra.run.smithi052.stdout: "telemetry", 2024-06-10T03:04:01.125 INFO:teuthology.orchestra.run.smithi052.stdout: "volumes" 2024-06-10T03:04:01.125 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:01.125 INFO:teuthology.orchestra.run.smithi052.stdout: "pacific": [ 2024-06-10T03:04:01.125 INFO:teuthology.orchestra.run.smithi052.stdout: "balancer", 2024-06-10T03:04:01.125 INFO:teuthology.orchestra.run.smithi052.stdout: "crash", 2024-06-10T03:04:01.125 INFO:teuthology.orchestra.run.smithi052.stdout: "devicehealth", 2024-06-10T03:04:01.125 INFO:teuthology.orchestra.run.smithi052.stdout: "orchestrator", 2024-06-10T03:04:01.125 INFO:teuthology.orchestra.run.smithi052.stdout: "pg_autoscaler", 2024-06-10T03:04:01.125 INFO:teuthology.orchestra.run.smithi052.stdout: "progress", 2024-06-10T03:04:01.125 INFO:teuthology.orchestra.run.smithi052.stdout: "rbd_support", 2024-06-10T03:04:01.125 INFO:teuthology.orchestra.run.smithi052.stdout: "status", 2024-06-10T03:04:01.125 INFO:teuthology.orchestra.run.smithi052.stdout: "telemetry", 2024-06-10T03:04:01.125 INFO:teuthology.orchestra.run.smithi052.stdout: "volumes" 2024-06-10T03:04:01.126 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:01.126 INFO:teuthology.orchestra.run.smithi052.stdout: "quincy": [ 2024-06-10T03:04:01.126 INFO:teuthology.orchestra.run.smithi052.stdout: "balancer", 2024-06-10T03:04:01.126 INFO:teuthology.orchestra.run.smithi052.stdout: "crash", 2024-06-10T03:04:01.126 INFO:teuthology.orchestra.run.smithi052.stdout: "devicehealth", 2024-06-10T03:04:01.126 INFO:teuthology.orchestra.run.smithi052.stdout: "orchestrator", 2024-06-10T03:04:01.126 INFO:teuthology.orchestra.run.smithi052.stdout: "pg_autoscaler", 2024-06-10T03:04:01.126 INFO:teuthology.orchestra.run.smithi052.stdout: "progress", 2024-06-10T03:04:01.126 INFO:teuthology.orchestra.run.smithi052.stdout: "rbd_support", 2024-06-10T03:04:01.126 INFO:teuthology.orchestra.run.smithi052.stdout: "status", 2024-06-10T03:04:01.126 INFO:teuthology.orchestra.run.smithi052.stdout: "telemetry", 2024-06-10T03:04:01.126 INFO:teuthology.orchestra.run.smithi052.stdout: "volumes" 2024-06-10T03:04:01.126 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:01.126 INFO:teuthology.orchestra.run.smithi052.stdout: "reef": [ 2024-06-10T03:04:01.127 INFO:teuthology.orchestra.run.smithi052.stdout: "balancer", 2024-06-10T03:04:01.127 INFO:teuthology.orchestra.run.smithi052.stdout: "crash", 2024-06-10T03:04:01.127 INFO:teuthology.orchestra.run.smithi052.stdout: "devicehealth", 2024-06-10T03:04:01.127 INFO:teuthology.orchestra.run.smithi052.stdout: "orchestrator", 2024-06-10T03:04:01.127 INFO:teuthology.orchestra.run.smithi052.stdout: "pg_autoscaler", 2024-06-10T03:04:01.127 INFO:teuthology.orchestra.run.smithi052.stdout: "progress", 2024-06-10T03:04:01.127 INFO:teuthology.orchestra.run.smithi052.stdout: "rbd_support", 2024-06-10T03:04:01.127 INFO:teuthology.orchestra.run.smithi052.stdout: "status", 2024-06-10T03:04:01.127 INFO:teuthology.orchestra.run.smithi052.stdout: "telemetry", 2024-06-10T03:04:01.127 INFO:teuthology.orchestra.run.smithi052.stdout: "volumes" 2024-06-10T03:04:01.127 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-10T03:04:01.127 INFO:teuthology.orchestra.run.smithi052.stdout: "squid": [ 2024-06-10T03:04:01.127 INFO:teuthology.orchestra.run.smithi052.stdout: "balancer", 2024-06-10T03:04:01.127 INFO:teuthology.orchestra.run.smithi052.stdout: "crash", 2024-06-10T03:04:01.128 INFO:teuthology.orchestra.run.smithi052.stdout: "devicehealth", 2024-06-10T03:04:01.128 INFO:teuthology.orchestra.run.smithi052.stdout: "orchestrator", 2024-06-10T03:04:01.128 INFO:teuthology.orchestra.run.smithi052.stdout: "pg_autoscaler", 2024-06-10T03:04:01.128 INFO:teuthology.orchestra.run.smithi052.stdout: "progress", 2024-06-10T03:04:01.128 INFO:teuthology.orchestra.run.smithi052.stdout: "rbd_support", 2024-06-10T03:04:01.128 INFO:teuthology.orchestra.run.smithi052.stdout: "status", 2024-06-10T03:04:01.128 INFO:teuthology.orchestra.run.smithi052.stdout: "telemetry", 2024-06-10T03:04:01.128 INFO:teuthology.orchestra.run.smithi052.stdout: "volumes" 2024-06-10T03:04:01.128 INFO:teuthology.orchestra.run.smithi052.stdout: ] 2024-06-10T03:04:01.128 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:01.128 INFO:teuthology.orchestra.run.smithi052.stdout: "last_failure_osd_epoch": 3, 2024-06-10T03:04:01.128 INFO:teuthology.orchestra.run.smithi052.stdout: "active_clients": [ 2024-06-10T03:04:01.128 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-10T03:04:01.128 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "devicehealth", 2024-06-10T03:04:01.129 INFO:teuthology.orchestra.run.smithi052.stdout: "addrvec": [ 2024-06-10T03:04:01.129 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-10T03:04:01.129 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "v2", 2024-06-10T03:04:01.129 INFO:teuthology.orchestra.run.smithi052.stdout: "addr": "172.21.15.52:0", 2024-06-10T03:04:01.129 INFO:teuthology.orchestra.run.smithi052.stdout: "nonce": 2021062140 2024-06-10T03:04:01.129 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:01.129 INFO:teuthology.orchestra.run.smithi052.stdout: ] 2024-06-10T03:04:01.129 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:01.129 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-10T03:04:01.129 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "libcephsqlite", 2024-06-10T03:04:01.129 INFO:teuthology.orchestra.run.smithi052.stdout: "addrvec": [ 2024-06-10T03:04:01.129 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-10T03:04:01.129 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "v2", 2024-06-10T03:04:01.129 INFO:teuthology.orchestra.run.smithi052.stdout: "addr": "172.21.15.52:0", 2024-06-10T03:04:01.130 INFO:teuthology.orchestra.run.smithi052.stdout: "nonce": 2646182408 2024-06-10T03:04:01.130 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:01.130 INFO:teuthology.orchestra.run.smithi052.stdout: ] 2024-06-10T03:04:01.130 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:01.130 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-10T03:04:01.130 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rbd_support", 2024-06-10T03:04:01.130 INFO:teuthology.orchestra.run.smithi052.stdout: "addrvec": [ 2024-06-10T03:04:01.130 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-10T03:04:01.130 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "v2", 2024-06-10T03:04:01.130 INFO:teuthology.orchestra.run.smithi052.stdout: "addr": "172.21.15.52:0", 2024-06-10T03:04:01.130 INFO:teuthology.orchestra.run.smithi052.stdout: "nonce": 2736987751 2024-06-10T03:04:01.130 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:01.130 INFO:teuthology.orchestra.run.smithi052.stdout: ] 2024-06-10T03:04:01.130 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:01.131 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-10T03:04:01.131 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "volumes", 2024-06-10T03:04:01.131 INFO:teuthology.orchestra.run.smithi052.stdout: "addrvec": [ 2024-06-10T03:04:01.131 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-10T03:04:01.131 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "v2", 2024-06-10T03:04:01.131 INFO:teuthology.orchestra.run.smithi052.stdout: "addr": "172.21.15.52:0", 2024-06-10T03:04:01.131 INFO:teuthology.orchestra.run.smithi052.stdout: "nonce": 1363984630 2024-06-10T03:04:01.131 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:01.131 INFO:teuthology.orchestra.run.smithi052.stdout: ] 2024-06-10T03:04:01.131 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:01.131 INFO:teuthology.orchestra.run.smithi052.stdout: ] 2024-06-10T03:04:01.131 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-10T03:04:01.133 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:04:00 smithi188 ceph-mon[36599]: pgmap v728: 129 pgs: 129 active+clean; 228 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-10T03:04:01.133 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:04:00 smithi188 ceph-mon[36599]: from='client.? 172.21.15.52:0/1334580849' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-06-10T03:04:01.133 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:04:00 smithi188 ceph-mon[36599]: from='client.? 172.21.15.52:0/3899309631' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-06-10T03:04:01.133 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:04:00 smithi120 ceph-mon[36634]: pgmap v728: 129 pgs: 129 active+clean; 228 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-10T03:04:01.133 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:04:00 smithi120 ceph-mon[36634]: from='client.? 172.21.15.52:0/1334580849' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-06-10T03:04:01.133 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:04:00 smithi120 ceph-mon[36634]: from='client.? 172.21.15.52:0/3899309631' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-06-10T03:04:01.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:59.747+0000 7fd840c9e640 1 -- 172.21.15.52:0/3899309631 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fd8240a2b90 msgr2=0x7fd8240a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:04:01.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:59.747+0000 7fd840c9e640 1 --2- 172.21.15.52:0/3899309631 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fd8240a2b90 0x7fd8240a5050 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fd82c0058c0 tx=0x7fd82c005850 comp rx=0 tx=0).stop 2024-06-10T03:04:01.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:59.747+0000 7fd840c9e640 1 -- 172.21.15.52:0/3899309631 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd83c101ee0 msgr2=0x7fd83c109df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:04:01.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:59.747+0000 7fd840c9e640 1 --2- 172.21.15.52:0/3899309631 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd83c101ee0 0x7fd83c109df0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fd83402ae00 tx=0x7fd834003b80 comp rx=0 tx=0).stop 2024-06-10T03:04:01.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:59.747+0000 7fd840c9e640 1 -- 172.21.15.52:0/3899309631 shutdown_connections 2024-06-10T03:04:01.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:59.747+0000 7fd840c9e640 1 --2- 172.21.15.52:0/3899309631 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd83c101ee0 0x7fd83c109df0 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:04:01.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:59.747+0000 7fd840c9e640 1 --2- 172.21.15.52:0/3899309631 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd83c101540 0x7fd83c1098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:04:01.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:59.747+0000 7fd840c9e640 1 --2- 172.21.15.52:0/3899309631 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fd8240a2b90 0x7fd8240a5050 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:04:01.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:59.747+0000 7fd840c9e640 1 --2- 172.21.15.52:0/3899309631 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd83c100b80 0x7fd83c10b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:04:01.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:59.747+0000 7fd840c9e640 1 -- 172.21.15.52:0/3899309631 >> 172.21.15.52:0/3899309631 conn(0x7fd83c0fb610 msgr2=0x7fd83c104dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:04:01.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:59.747+0000 7fd840c9e640 1 -- 172.21.15.52:0/3899309631 shutdown_connections 2024-06-10T03:04:01.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:03:59.747+0000 7fd840c9e640 1 -- 172.21.15.52:0/3899309631 wait complete. 2024-06-10T03:04:01.182 DEBUG:tasks.check_counter:Getting stats from a 2024-06-10T03:04:01.183 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph daemon mds.a perf dump 2024-06-10T03:04:01.183 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:04:00 smithi052 ceph-mon[32362]: pgmap v728: 129 pgs: 129 active+clean; 228 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-10T03:04:01.183 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:04:00 smithi052 ceph-mon[32362]: from='client.? 172.21.15.52:0/1334580849' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-06-10T03:04:01.183 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:04:00 smithi052 ceph-mon[32362]: from='client.? 172.21.15.52:0/3899309631' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-06-10T03:04:01.448 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T03:04:02.599 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:04:02 smithi052 ceph-mon[32362]: pgmap v729: 129 pgs: 129 active+clean; 228 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-06-10T03:04:02.638 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:04:02 smithi120 ceph-mon[36634]: pgmap v729: 129 pgs: 129 active+clean; 228 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-06-10T03:04:02.704 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:04:02 smithi188 ceph-mon[36599]: pgmap v729: 129 pgs: 129 active+clean; 228 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-06-10T03:04:03.169 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-10T03:04:03.169 INFO:teuthology.orchestra.run.smithi052.stdout: "AsyncMessenger::Worker-0": { 2024-06-10T03:04:03.169 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_messages": 1556, 2024-06-10T03:04:03.169 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_messages": 1556, 2024-06-10T03:04:03.169 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_bytes": 1092080, 2024-06-10T03:04:03.169 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_bytes": 493985, 2024-06-10T03:04:03.169 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_created_connections": 6, 2024-06-10T03:04:03.169 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_active_connections": 2, 2024-06-10T03:04:03.169 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_total_time": 0.130562836, 2024-06-10T03:04:03.170 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_send_time": 0.042046582, 2024-06-10T03:04:03.170 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_recv_time": 0.034258194, 2024-06-10T03:04:03.170 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_fast_dispatch_time": 0.042274423, 2024-06-10T03:04:03.170 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_messages_queue_lat": { 2024-06-10T03:04:03.170 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 1555, 2024-06-10T03:04:03.170 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.060740947, 2024-06-10T03:04:03.170 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000039061 2024-06-10T03:04:03.170 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:03.170 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_handle_ack_lat": { 2024-06-10T03:04:03.170 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-10T03:04:03.170 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-10T03:04:03.170 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-10T03:04:03.170 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:03.171 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_encrypted_bytes": 3072, 2024-06-10T03:04:03.171 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_encrypted_bytes": 448 2024-06-10T03:04:03.171 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:03.171 INFO:teuthology.orchestra.run.smithi052.stdout: "AsyncMessenger::Worker-1": { 2024-06-10T03:04:03.171 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_messages": 680, 2024-06-10T03:04:03.171 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_messages": 854, 2024-06-10T03:04:03.171 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_bytes": 167846, 2024-06-10T03:04:03.171 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_bytes": 582806, 2024-06-10T03:04:03.171 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_created_connections": 5, 2024-06-10T03:04:03.171 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_active_connections": 4, 2024-06-10T03:04:03.171 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_total_time": 0.071633109, 2024-06-10T03:04:03.171 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_send_time": 0.025818161, 2024-06-10T03:04:03.171 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_recv_time": 0.014295799, 2024-06-10T03:04:03.172 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_fast_dispatch_time": 0.023364548, 2024-06-10T03:04:03.172 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_messages_queue_lat": { 2024-06-10T03:04:03.172 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 852, 2024-06-10T03:04:03.172 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.031837457, 2024-06-10T03:04:03.172 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000037367 2024-06-10T03:04:03.172 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:03.172 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_handle_ack_lat": { 2024-06-10T03:04:03.172 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-10T03:04:03.172 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-10T03:04:03.172 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-10T03:04:03.172 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:03.172 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_encrypted_bytes": 160, 2024-06-10T03:04:03.172 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_encrypted_bytes": 366816 2024-06-10T03:04:03.172 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:03.173 INFO:teuthology.orchestra.run.smithi052.stdout: "AsyncMessenger::Worker-2": { 2024-06-10T03:04:03.173 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_messages": 260, 2024-06-10T03:04:03.173 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_messages": 238, 2024-06-10T03:04:03.173 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_bytes": 890748, 2024-06-10T03:04:03.173 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_bytes": 143006, 2024-06-10T03:04:03.173 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_created_connections": 3, 2024-06-10T03:04:03.173 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_active_connections": 2, 2024-06-10T03:04:03.173 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_total_time": 0.034839941, 2024-06-10T03:04:03.173 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_send_time": 0.013268059, 2024-06-10T03:04:03.173 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_recv_time": 0.019474977, 2024-06-10T03:04:03.173 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_fast_dispatch_time": 0.004571831, 2024-06-10T03:04:03.173 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_messages_queue_lat": { 2024-06-10T03:04:03.173 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 237, 2024-06-10T03:04:03.173 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.013721241, 2024-06-10T03:04:03.174 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000057895 2024-06-10T03:04:03.174 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:03.174 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_handle_ack_lat": { 2024-06-10T03:04:03.174 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-10T03:04:03.174 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-10T03:04:03.174 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-10T03:04:03.174 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:03.174 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_encrypted_bytes": 890480, 2024-06-10T03:04:03.174 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_encrypted_bytes": 142688 2024-06-10T03:04:03.174 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:03.174 INFO:teuthology.orchestra.run.smithi052.stdout: "cct": { 2024-06-10T03:04:03.174 INFO:teuthology.orchestra.run.smithi052.stdout: "total_workers": 1, 2024-06-10T03:04:03.175 INFO:teuthology.orchestra.run.smithi052.stdout: "unhealthy_workers": 0 2024-06-10T03:04:03.175 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:03.175 INFO:teuthology.orchestra.run.smithi052.stdout: "finisher-MDSRank": { 2024-06-10T03:04:03.175 INFO:teuthology.orchestra.run.smithi052.stdout: "queue_len": 0, 2024-06-10T03:04:03.175 INFO:teuthology.orchestra.run.smithi052.stdout: "complete_latency": { 2024-06-10T03:04:03.175 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 3676, 2024-06-10T03:04:03.175 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.100276438, 2024-06-10T03:04:03.175 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000027278 2024-06-10T03:04:03.175 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:03.175 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:03.175 INFO:teuthology.orchestra.run.smithi052.stdout: "finisher-PurgeQueue": { 2024-06-10T03:04:03.175 INFO:teuthology.orchestra.run.smithi052.stdout: "queue_len": 0, 2024-06-10T03:04:03.175 INFO:teuthology.orchestra.run.smithi052.stdout: "complete_latency": { 2024-06-10T03:04:03.175 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-10T03:04:03.176 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-10T03:04:03.176 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-10T03:04:03.176 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:03.176 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:03.176 INFO:teuthology.orchestra.run.smithi052.stdout: "mds": { 2024-06-10T03:04:03.176 INFO:teuthology.orchestra.run.smithi052.stdout: "request": 0, 2024-06-10T03:04:03.176 INFO:teuthology.orchestra.run.smithi052.stdout: "reply": 0, 2024-06-10T03:04:03.176 INFO:teuthology.orchestra.run.smithi052.stdout: "reply_latency": { 2024-06-10T03:04:03.176 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-10T03:04:03.176 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-10T03:04:03.176 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-10T03:04:03.176 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:03.176 INFO:teuthology.orchestra.run.smithi052.stdout: "slow_reply": 0, 2024-06-10T03:04:03.176 INFO:teuthology.orchestra.run.smithi052.stdout: "forward": 0, 2024-06-10T03:04:03.177 INFO:teuthology.orchestra.run.smithi052.stdout: "dir_fetch_complete": 0, 2024-06-10T03:04:03.177 INFO:teuthology.orchestra.run.smithi052.stdout: "dir_fetch_keys": 0, 2024-06-10T03:04:03.177 INFO:teuthology.orchestra.run.smithi052.stdout: "dir_commit": 0, 2024-06-10T03:04:03.177 INFO:teuthology.orchestra.run.smithi052.stdout: "dir_split": 0, 2024-06-10T03:04:03.177 INFO:teuthology.orchestra.run.smithi052.stdout: "dir_merge": 0, 2024-06-10T03:04:03.177 INFO:teuthology.orchestra.run.smithi052.stdout: "inodes": 92, 2024-06-10T03:04:03.177 INFO:teuthology.orchestra.run.smithi052.stdout: "inodes_top": 0, 2024-06-10T03:04:03.177 INFO:teuthology.orchestra.run.smithi052.stdout: "inodes_bottom": 92, 2024-06-10T03:04:03.177 INFO:teuthology.orchestra.run.smithi052.stdout: "inodes_pin_tail": 0, 2024-06-10T03:04:03.177 INFO:teuthology.orchestra.run.smithi052.stdout: "inodes_pinned": 92, 2024-06-10T03:04:03.178 INFO:teuthology.orchestra.run.smithi052.stdout: "inodes_expired": 0, 2024-06-10T03:04:03.178 INFO:teuthology.orchestra.run.smithi052.stdout: "inodes_with_caps": 0, 2024-06-10T03:04:03.178 INFO:teuthology.orchestra.run.smithi052.stdout: "caps": 0, 2024-06-10T03:04:03.178 INFO:teuthology.orchestra.run.smithi052.stdout: "subtrees": 1, 2024-06-10T03:04:03.178 INFO:teuthology.orchestra.run.smithi052.stdout: "traverse": 0, 2024-06-10T03:04:03.178 INFO:teuthology.orchestra.run.smithi052.stdout: "traverse_hit": 0, 2024-06-10T03:04:03.178 INFO:teuthology.orchestra.run.smithi052.stdout: "traverse_forward": 0, 2024-06-10T03:04:03.178 INFO:teuthology.orchestra.run.smithi052.stdout: "traverse_discover": 0, 2024-06-10T03:04:03.178 INFO:teuthology.orchestra.run.smithi052.stdout: "traverse_dir_fetch": 0, 2024-06-10T03:04:03.178 INFO:teuthology.orchestra.run.smithi052.stdout: "traverse_remote_ino": 0, 2024-06-10T03:04:03.178 INFO:teuthology.orchestra.run.smithi052.stdout: "traverse_lock": 0, 2024-06-10T03:04:03.178 INFO:teuthology.orchestra.run.smithi052.stdout: "load_cent": 0, 2024-06-10T03:04:03.178 INFO:teuthology.orchestra.run.smithi052.stdout: "q": 0, 2024-06-10T03:04:03.179 INFO:teuthology.orchestra.run.smithi052.stdout: "exported": 0, 2024-06-10T03:04:03.179 INFO:teuthology.orchestra.run.smithi052.stdout: "exported_inodes": 0, 2024-06-10T03:04:03.179 INFO:teuthology.orchestra.run.smithi052.stdout: "imported": 0, 2024-06-10T03:04:03.179 INFO:teuthology.orchestra.run.smithi052.stdout: "imported_inodes": 0, 2024-06-10T03:04:03.179 INFO:teuthology.orchestra.run.smithi052.stdout: "openino_dir_fetch": 0, 2024-06-10T03:04:03.179 INFO:teuthology.orchestra.run.smithi052.stdout: "openino_backtrace_fetch": 0, 2024-06-10T03:04:03.179 INFO:teuthology.orchestra.run.smithi052.stdout: "openino_peer_discover": 0, 2024-06-10T03:04:03.179 INFO:teuthology.orchestra.run.smithi052.stdout: "root_rfiles": 0, 2024-06-10T03:04:03.179 INFO:teuthology.orchestra.run.smithi052.stdout: "root_rbytes": 0, 2024-06-10T03:04:03.182 INFO:teuthology.orchestra.run.smithi052.stdout: "root_rsnaps": 0, 2024-06-10T03:04:03.182 INFO:teuthology.orchestra.run.smithi052.stdout: "scrub_backtrace_fetch": 0, 2024-06-10T03:04:03.182 INFO:teuthology.orchestra.run.smithi052.stdout: "scrub_set_tag": 0, 2024-06-10T03:04:03.182 INFO:teuthology.orchestra.run.smithi052.stdout: "scrub_backtrace_repaired": 0, 2024-06-10T03:04:03.183 INFO:teuthology.orchestra.run.smithi052.stdout: "scrub_inotable_repaired": 0, 2024-06-10T03:04:03.183 INFO:teuthology.orchestra.run.smithi052.stdout: "scrub_dir_inodes": 0, 2024-06-10T03:04:03.183 INFO:teuthology.orchestra.run.smithi052.stdout: "scrub_dir_base_inodes": 0, 2024-06-10T03:04:03.183 INFO:teuthology.orchestra.run.smithi052.stdout: "scrub_dirfrag_rstats": 0, 2024-06-10T03:04:03.183 INFO:teuthology.orchestra.run.smithi052.stdout: "scrub_file_inodes": 0, 2024-06-10T03:04:03.183 INFO:teuthology.orchestra.run.smithi052.stdout: "handle_inode_file_caps": 0, 2024-06-10T03:04:03.183 INFO:teuthology.orchestra.run.smithi052.stdout: "ceph_cap_op_revoke": 0, 2024-06-10T03:04:03.183 INFO:teuthology.orchestra.run.smithi052.stdout: "ceph_cap_op_grant": 0, 2024-06-10T03:04:03.183 INFO:teuthology.orchestra.run.smithi052.stdout: "ceph_cap_op_trunc": 0, 2024-06-10T03:04:03.183 INFO:teuthology.orchestra.run.smithi052.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-06-10T03:04:03.183 INFO:teuthology.orchestra.run.smithi052.stdout: "ceph_cap_op_flush_ack": 0, 2024-06-10T03:04:03.183 INFO:teuthology.orchestra.run.smithi052.stdout: "handle_client_caps": 0, 2024-06-10T03:04:03.184 INFO:teuthology.orchestra.run.smithi052.stdout: "handle_client_caps_dirty": 0, 2024-06-10T03:04:03.184 INFO:teuthology.orchestra.run.smithi052.stdout: "handle_client_cap_release": 0, 2024-06-10T03:04:03.184 INFO:teuthology.orchestra.run.smithi052.stdout: "process_request_cap_release": 0 2024-06-10T03:04:03.184 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:03.184 INFO:teuthology.orchestra.run.smithi052.stdout: "mds_cache": { 2024-06-10T03:04:03.184 INFO:teuthology.orchestra.run.smithi052.stdout: "dir_update": 0, 2024-06-10T03:04:03.184 INFO:teuthology.orchestra.run.smithi052.stdout: "dir_update_receipt": 0, 2024-06-10T03:04:03.184 INFO:teuthology.orchestra.run.smithi052.stdout: "dir_try_discover": 0, 2024-06-10T03:04:03.184 INFO:teuthology.orchestra.run.smithi052.stdout: "dir_send_discover": 0, 2024-06-10T03:04:03.184 INFO:teuthology.orchestra.run.smithi052.stdout: "dir_handle_discover": 0, 2024-06-10T03:04:03.184 INFO:teuthology.orchestra.run.smithi052.stdout: "num_strays": 0, 2024-06-10T03:04:03.184 INFO:teuthology.orchestra.run.smithi052.stdout: "num_strays_delayed": 0, 2024-06-10T03:04:03.184 INFO:teuthology.orchestra.run.smithi052.stdout: "num_strays_enqueuing": 0, 2024-06-10T03:04:03.185 INFO:teuthology.orchestra.run.smithi052.stdout: "strays_created": 0, 2024-06-10T03:04:03.185 INFO:teuthology.orchestra.run.smithi052.stdout: "strays_enqueued": 0, 2024-06-10T03:04:03.185 INFO:teuthology.orchestra.run.smithi052.stdout: "strays_reintegrated": 0, 2024-06-10T03:04:03.185 INFO:teuthology.orchestra.run.smithi052.stdout: "strays_migrated": 0, 2024-06-10T03:04:03.185 INFO:teuthology.orchestra.run.smithi052.stdout: "num_recovering_processing": 0, 2024-06-10T03:04:03.185 INFO:teuthology.orchestra.run.smithi052.stdout: "num_recovering_enqueued": 0, 2024-06-10T03:04:03.185 INFO:teuthology.orchestra.run.smithi052.stdout: "num_recovering_prioritized": 0, 2024-06-10T03:04:03.185 INFO:teuthology.orchestra.run.smithi052.stdout: "recovery_started": 0, 2024-06-10T03:04:03.185 INFO:teuthology.orchestra.run.smithi052.stdout: "recovery_completed": 0, 2024-06-10T03:04:03.185 INFO:teuthology.orchestra.run.smithi052.stdout: "ireq_quiesce_path": 0, 2024-06-10T03:04:03.186 INFO:teuthology.orchestra.run.smithi052.stdout: "ireq_quiesce_inode": 0, 2024-06-10T03:04:03.186 INFO:teuthology.orchestra.run.smithi052.stdout: "ireq_enqueue_scrub": 0, 2024-06-10T03:04:03.186 INFO:teuthology.orchestra.run.smithi052.stdout: "ireq_exportdir": 0, 2024-06-10T03:04:03.186 INFO:teuthology.orchestra.run.smithi052.stdout: "ireq_flush": 0, 2024-06-10T03:04:03.186 INFO:teuthology.orchestra.run.smithi052.stdout: "ireq_fragmentdir": 0, 2024-06-10T03:04:03.186 INFO:teuthology.orchestra.run.smithi052.stdout: "ireq_fragstats": 0, 2024-06-10T03:04:03.186 INFO:teuthology.orchestra.run.smithi052.stdout: "ireq_inodestats": 0 2024-06-10T03:04:03.186 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:03.186 INFO:teuthology.orchestra.run.smithi052.stdout: "mds_log": { 2024-06-10T03:04:03.186 INFO:teuthology.orchestra.run.smithi052.stdout: "evlrg": 0, 2024-06-10T03:04:03.186 INFO:teuthology.orchestra.run.smithi052.stdout: "evadd": 0, 2024-06-10T03:04:03.186 INFO:teuthology.orchestra.run.smithi052.stdout: "evex": 0, 2024-06-10T03:04:03.186 INFO:teuthology.orchestra.run.smithi052.stdout: "evtrm": 0, 2024-06-10T03:04:03.187 INFO:teuthology.orchestra.run.smithi052.stdout: "ev": 378, 2024-06-10T03:04:03.187 INFO:teuthology.orchestra.run.smithi052.stdout: "evexg": 0, 2024-06-10T03:04:03.187 INFO:teuthology.orchestra.run.smithi052.stdout: "evexd": 0, 2024-06-10T03:04:03.187 INFO:teuthology.orchestra.run.smithi052.stdout: "segadd": 0, 2024-06-10T03:04:03.187 INFO:teuthology.orchestra.run.smithi052.stdout: "segex": 0, 2024-06-10T03:04:03.187 INFO:teuthology.orchestra.run.smithi052.stdout: "segtrm": 0, 2024-06-10T03:04:03.187 INFO:teuthology.orchestra.run.smithi052.stdout: "seg": 2, 2024-06-10T03:04:03.187 INFO:teuthology.orchestra.run.smithi052.stdout: "segmjr": 2, 2024-06-10T03:04:03.187 INFO:teuthology.orchestra.run.smithi052.stdout: "segexg": 0, 2024-06-10T03:04:03.187 INFO:teuthology.orchestra.run.smithi052.stdout: "segexd": 0, 2024-06-10T03:04:03.187 INFO:teuthology.orchestra.run.smithi052.stdout: "expos": 4194304, 2024-06-10T03:04:03.188 INFO:teuthology.orchestra.run.smithi052.stdout: "wrpos": 4819865, 2024-06-10T03:04:03.188 INFO:teuthology.orchestra.run.smithi052.stdout: "rdpos": 4818214, 2024-06-10T03:04:03.188 INFO:teuthology.orchestra.run.smithi052.stdout: "jlat": { 2024-06-10T03:04:03.188 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-10T03:04:03.188 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-10T03:04:03.188 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-10T03:04:03.188 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:03.188 INFO:teuthology.orchestra.run.smithi052.stdout: "replayed": 378 2024-06-10T03:04:03.188 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:03.188 INFO:teuthology.orchestra.run.smithi052.stdout: "mds_mem": { 2024-06-10T03:04:03.188 INFO:teuthology.orchestra.run.smithi052.stdout: "ino": 4, 2024-06-10T03:04:03.188 INFO:teuthology.orchestra.run.smithi052.stdout: "ino+": 107, 2024-06-10T03:04:03.189 INFO:teuthology.orchestra.run.smithi052.stdout: "ino-": 103, 2024-06-10T03:04:03.189 INFO:teuthology.orchestra.run.smithi052.stdout: "dir": 2, 2024-06-10T03:04:03.189 INFO:teuthology.orchestra.run.smithi052.stdout: "dir+": 15, 2024-06-10T03:04:03.189 INFO:teuthology.orchestra.run.smithi052.stdout: "dir-": 13, 2024-06-10T03:04:03.189 INFO:teuthology.orchestra.run.smithi052.stdout: "dn": 92, 2024-06-10T03:04:03.189 INFO:teuthology.orchestra.run.smithi052.stdout: "dn+": 286, 2024-06-10T03:04:03.189 INFO:teuthology.orchestra.run.smithi052.stdout: "dn-": 194, 2024-06-10T03:04:03.189 INFO:teuthology.orchestra.run.smithi052.stdout: "cap": 0, 2024-06-10T03:04:03.189 INFO:teuthology.orchestra.run.smithi052.stdout: "cap+": 0, 2024-06-10T03:04:03.189 INFO:teuthology.orchestra.run.smithi052.stdout: "cap-": 0, 2024-06-10T03:04:03.189 INFO:teuthology.orchestra.run.smithi052.stdout: "rss": 54812, 2024-06-10T03:04:03.189 INFO:teuthology.orchestra.run.smithi052.stdout: "heap": 215420 2024-06-10T03:04:03.189 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:03.189 INFO:teuthology.orchestra.run.smithi052.stdout: "mds_server": { 2024-06-10T03:04:03.190 INFO:teuthology.orchestra.run.smithi052.stdout: "dispatch_client_request": 0, 2024-06-10T03:04:03.190 INFO:teuthology.orchestra.run.smithi052.stdout: "dispatch_server_request": 0, 2024-06-10T03:04:03.190 INFO:teuthology.orchestra.run.smithi052.stdout: "handle_client_request": 0, 2024-06-10T03:04:03.190 INFO:teuthology.orchestra.run.smithi052.stdout: "handle_client_session": 0, 2024-06-10T03:04:03.190 INFO:teuthology.orchestra.run.smithi052.stdout: "handle_peer_request": 0, 2024-06-10T03:04:03.190 INFO:teuthology.orchestra.run.smithi052.stdout: "req_create_latency": { 2024-06-10T03:04:03.190 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-10T03:04:03.190 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-10T03:04:03.190 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-10T03:04:03.190 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:03.190 INFO:teuthology.orchestra.run.smithi052.stdout: "req_getattr_latency": { 2024-06-10T03:04:03.191 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-10T03:04:03.191 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-10T03:04:03.191 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-10T03:04:03.191 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:03.191 INFO:teuthology.orchestra.run.smithi052.stdout: "req_getfilelock_latency": { 2024-06-10T03:04:03.191 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-10T03:04:03.191 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-10T03:04:03.191 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-10T03:04:03.191 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:03.191 INFO:teuthology.orchestra.run.smithi052.stdout: "req_link_latency": { 2024-06-10T03:04:03.191 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-10T03:04:03.191 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-10T03:04:03.191 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-10T03:04:03.192 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:03.192 INFO:teuthology.orchestra.run.smithi052.stdout: "req_lookup_latency": { 2024-06-10T03:04:03.192 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-10T03:04:03.192 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-10T03:04:03.192 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-10T03:04:03.192 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:03.192 INFO:teuthology.orchestra.run.smithi052.stdout: "req_lookuphash_latency": { 2024-06-10T03:04:03.192 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-10T03:04:03.192 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-10T03:04:03.192 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-10T03:04:03.192 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:03.193 INFO:teuthology.orchestra.run.smithi052.stdout: "req_lookupino_latency": { 2024-06-10T03:04:03.193 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-10T03:04:03.193 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-10T03:04:03.193 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-10T03:04:03.193 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:03.193 INFO:teuthology.orchestra.run.smithi052.stdout: "req_lookupname_latency": { 2024-06-10T03:04:03.193 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-10T03:04:03.193 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-10T03:04:03.193 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-10T03:04:03.193 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:03.194 INFO:teuthology.orchestra.run.smithi052.stdout: "req_lookupparent_latency": { 2024-06-10T03:04:03.194 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-10T03:04:03.194 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-10T03:04:03.194 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-10T03:04:03.194 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:03.194 INFO:teuthology.orchestra.run.smithi052.stdout: "req_lookupsnap_latency": { 2024-06-10T03:04:03.194 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-10T03:04:03.194 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-10T03:04:03.194 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-10T03:04:03.194 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:03.194 INFO:teuthology.orchestra.run.smithi052.stdout: "req_lssnap_latency": { 2024-06-10T03:04:03.194 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-10T03:04:03.194 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-10T03:04:03.195 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-10T03:04:03.195 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:03.195 INFO:teuthology.orchestra.run.smithi052.stdout: "req_mkdir_latency": { 2024-06-10T03:04:03.195 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-10T03:04:03.195 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-10T03:04:03.195 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-10T03:04:03.196 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:03.196 INFO:teuthology.orchestra.run.smithi052.stdout: "req_mknod_latency": { 2024-06-10T03:04:03.196 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-10T03:04:03.196 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-10T03:04:03.196 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-10T03:04:03.196 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:03.196 INFO:teuthology.orchestra.run.smithi052.stdout: "req_mksnap_latency": { 2024-06-10T03:04:03.196 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-10T03:04:03.196 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-10T03:04:03.196 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-10T03:04:03.196 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:03.196 INFO:teuthology.orchestra.run.smithi052.stdout: "req_open_latency": { 2024-06-10T03:04:03.197 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-10T03:04:03.197 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-10T03:04:03.197 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-10T03:04:03.197 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:03.197 INFO:teuthology.orchestra.run.smithi052.stdout: "req_readdir_latency": { 2024-06-10T03:04:03.197 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-10T03:04:03.197 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-10T03:04:03.197 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-10T03:04:03.197 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:03.197 INFO:teuthology.orchestra.run.smithi052.stdout: "req_rename_latency": { 2024-06-10T03:04:03.197 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-10T03:04:03.197 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-10T03:04:03.197 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-10T03:04:03.197 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:03.197 INFO:teuthology.orchestra.run.smithi052.stdout: "req_renamesnap_latency": { 2024-06-10T03:04:03.198 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-10T03:04:03.198 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-10T03:04:03.198 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-10T03:04:03.198 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:03.198 INFO:teuthology.orchestra.run.smithi052.stdout: "req_snapdiff_latency": { 2024-06-10T03:04:03.198 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-10T03:04:03.198 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-10T03:04:03.198 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-10T03:04:03.198 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:03.198 INFO:teuthology.orchestra.run.smithi052.stdout: "req_rmdir_latency": { 2024-06-10T03:04:03.198 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-10T03:04:03.198 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-10T03:04:03.198 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-10T03:04:03.199 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:03.199 INFO:teuthology.orchestra.run.smithi052.stdout: "req_rmsnap_latency": { 2024-06-10T03:04:03.199 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-10T03:04:03.199 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-10T03:04:03.199 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-10T03:04:03.199 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:03.199 INFO:teuthology.orchestra.run.smithi052.stdout: "req_rmxattr_latency": { 2024-06-10T03:04:03.199 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-10T03:04:03.199 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-10T03:04:03.199 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-10T03:04:03.199 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:03.199 INFO:teuthology.orchestra.run.smithi052.stdout: "req_setattr_latency": { 2024-06-10T03:04:03.199 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-10T03:04:03.199 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-10T03:04:03.200 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-10T03:04:03.200 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:03.200 INFO:teuthology.orchestra.run.smithi052.stdout: "req_setdirlayout_latency": { 2024-06-10T03:04:03.200 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-10T03:04:03.200 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-10T03:04:03.200 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-10T03:04:03.200 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:03.200 INFO:teuthology.orchestra.run.smithi052.stdout: "req_setfilelock_latency": { 2024-06-10T03:04:03.200 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-10T03:04:03.200 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-10T03:04:03.200 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-10T03:04:03.200 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:03.200 INFO:teuthology.orchestra.run.smithi052.stdout: "req_setlayout_latency": { 2024-06-10T03:04:03.200 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-10T03:04:03.201 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-10T03:04:03.201 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-10T03:04:03.201 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:03.201 INFO:teuthology.orchestra.run.smithi052.stdout: "req_setxattr_latency": { 2024-06-10T03:04:03.201 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-10T03:04:03.201 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-10T03:04:03.201 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-10T03:04:03.201 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:03.201 INFO:teuthology.orchestra.run.smithi052.stdout: "req_symlink_latency": { 2024-06-10T03:04:03.201 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-10T03:04:03.201 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-10T03:04:03.201 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-10T03:04:03.201 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:03.201 INFO:teuthology.orchestra.run.smithi052.stdout: "req_unlink_latency": { 2024-06-10T03:04:03.202 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-10T03:04:03.202 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-10T03:04:03.202 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-10T03:04:03.202 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:03.202 INFO:teuthology.orchestra.run.smithi052.stdout: "cap_revoke_eviction": 0, 2024-06-10T03:04:03.202 INFO:teuthology.orchestra.run.smithi052.stdout: "cap_acquisition_throttle": 0, 2024-06-10T03:04:03.202 INFO:teuthology.orchestra.run.smithi052.stdout: "req_getvxattr_latency": { 2024-06-10T03:04:03.202 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-10T03:04:03.202 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-10T03:04:03.202 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-10T03:04:03.202 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:03.202 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:03.202 INFO:teuthology.orchestra.run.smithi052.stdout: "mds_sessions": { 2024-06-10T03:04:03.203 INFO:teuthology.orchestra.run.smithi052.stdout: "session_count": 1, 2024-06-10T03:04:03.203 INFO:teuthology.orchestra.run.smithi052.stdout: "session_add": 2, 2024-06-10T03:04:03.203 INFO:teuthology.orchestra.run.smithi052.stdout: "session_remove": 1, 2024-06-10T03:04:03.203 INFO:teuthology.orchestra.run.smithi052.stdout: "sessions_open": 1, 2024-06-10T03:04:03.203 INFO:teuthology.orchestra.run.smithi052.stdout: "sessions_stale": 0, 2024-06-10T03:04:03.203 INFO:teuthology.orchestra.run.smithi052.stdout: "total_load": 0, 2024-06-10T03:04:03.203 INFO:teuthology.orchestra.run.smithi052.stdout: "average_load": 0, 2024-06-10T03:04:03.203 INFO:teuthology.orchestra.run.smithi052.stdout: "avg_session_uptime": 2110, 2024-06-10T03:04:03.203 INFO:teuthology.orchestra.run.smithi052.stdout: "mdthresh_evicted": 0 2024-06-10T03:04:03.203 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:03.203 INFO:teuthology.orchestra.run.smithi052.stdout: "mempool": { 2024-06-10T03:04:03.203 INFO:teuthology.orchestra.run.smithi052.stdout: "bloom_filter_bytes": 0, 2024-06-10T03:04:03.203 INFO:teuthology.orchestra.run.smithi052.stdout: "bloom_filter_items": 0, 2024-06-10T03:04:03.203 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_alloc_bytes": 0, 2024-06-10T03:04:03.203 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_alloc_items": 0, 2024-06-10T03:04:03.204 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_data_bytes": 0, 2024-06-10T03:04:03.204 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_data_items": 0, 2024-06-10T03:04:03.204 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-10T03:04:03.204 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_onode_items": 0, 2024-06-10T03:04:03.204 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-10T03:04:03.204 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_meta_items": 0, 2024-06-10T03:04:03.204 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_other_bytes": 0, 2024-06-10T03:04:03.204 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_other_items": 0, 2024-06-10T03:04:03.204 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-10T03:04:03.204 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_buffer_items": 0, 2024-06-10T03:04:03.204 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_extent_bytes": 0, 2024-06-10T03:04:03.204 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_extent_items": 0, 2024-06-10T03:04:03.204 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_blob_bytes": 0, 2024-06-10T03:04:03.205 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_blob_items": 0, 2024-06-10T03:04:03.205 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-10T03:04:03.205 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_shared_blob_items": 0, 2024-06-10T03:04:03.205 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-10T03:04:03.205 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_inline_bl_items": 0, 2024-06-10T03:04:03.205 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_fsck_bytes": 0, 2024-06-10T03:04:03.205 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_fsck_items": 0, 2024-06-10T03:04:03.205 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_txc_bytes": 0, 2024-06-10T03:04:03.205 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_txc_items": 0, 2024-06-10T03:04:03.205 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-10T03:04:03.205 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_writing_deferred_items": 0, 2024-06-10T03:04:03.205 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_writing_bytes": 0, 2024-06-10T03:04:03.205 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_writing_items": 0, 2024-06-10T03:04:03.205 INFO:teuthology.orchestra.run.smithi052.stdout: "bluefs_bytes": 0, 2024-06-10T03:04:03.206 INFO:teuthology.orchestra.run.smithi052.stdout: "bluefs_items": 0, 2024-06-10T03:04:03.206 INFO:teuthology.orchestra.run.smithi052.stdout: "bluefs_file_reader_bytes": 0, 2024-06-10T03:04:03.206 INFO:teuthology.orchestra.run.smithi052.stdout: "bluefs_file_reader_items": 0, 2024-06-10T03:04:03.206 INFO:teuthology.orchestra.run.smithi052.stdout: "bluefs_file_writer_bytes": 0, 2024-06-10T03:04:03.206 INFO:teuthology.orchestra.run.smithi052.stdout: "bluefs_file_writer_items": 0, 2024-06-10T03:04:03.206 INFO:teuthology.orchestra.run.smithi052.stdout: "buffer_anon_bytes": 189423, 2024-06-10T03:04:03.206 INFO:teuthology.orchestra.run.smithi052.stdout: "buffer_anon_items": 51, 2024-06-10T03:04:03.206 INFO:teuthology.orchestra.run.smithi052.stdout: "buffer_meta_bytes": 0, 2024-06-10T03:04:03.206 INFO:teuthology.orchestra.run.smithi052.stdout: "buffer_meta_items": 0, 2024-06-10T03:04:03.206 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_bytes": 0, 2024-06-10T03:04:03.206 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_items": 0, 2024-06-10T03:04:03.206 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_mapbl_bytes": 0, 2024-06-10T03:04:03.206 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_mapbl_items": 0, 2024-06-10T03:04:03.207 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_pglog_bytes": 0, 2024-06-10T03:04:03.207 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_pglog_items": 0, 2024-06-10T03:04:03.207 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_bytes": 8984, 2024-06-10T03:04:03.207 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_items": 172, 2024-06-10T03:04:03.207 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_mapping_bytes": 0, 2024-06-10T03:04:03.207 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_mapping_items": 0, 2024-06-10T03:04:03.207 INFO:teuthology.orchestra.run.smithi052.stdout: "pgmap_bytes": 0, 2024-06-10T03:04:03.207 INFO:teuthology.orchestra.run.smithi052.stdout: "pgmap_items": 0, 2024-06-10T03:04:03.207 INFO:teuthology.orchestra.run.smithi052.stdout: "mds_co_bytes": 76752, 2024-06-10T03:04:03.207 INFO:teuthology.orchestra.run.smithi052.stdout: "mds_co_items": 396, 2024-06-10T03:04:03.207 INFO:teuthology.orchestra.run.smithi052.stdout: "unittest_1_bytes": 0, 2024-06-10T03:04:03.207 INFO:teuthology.orchestra.run.smithi052.stdout: "unittest_1_items": 0, 2024-06-10T03:04:03.207 INFO:teuthology.orchestra.run.smithi052.stdout: "unittest_2_bytes": 0, 2024-06-10T03:04:03.207 INFO:teuthology.orchestra.run.smithi052.stdout: "unittest_2_items": 0 2024-06-10T03:04:03.207 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:03.208 INFO:teuthology.orchestra.run.smithi052.stdout: "objecter": { 2024-06-10T03:04:03.208 INFO:teuthology.orchestra.run.smithi052.stdout: "op_active": 0, 2024-06-10T03:04:03.208 INFO:teuthology.orchestra.run.smithi052.stdout: "op_laggy": 0, 2024-06-10T03:04:03.208 INFO:teuthology.orchestra.run.smithi052.stdout: "op_send": 2232, 2024-06-10T03:04:03.208 INFO:teuthology.orchestra.run.smithi052.stdout: "op_send_bytes": 21, 2024-06-10T03:04:03.208 INFO:teuthology.orchestra.run.smithi052.stdout: "op_resend": 3, 2024-06-10T03:04:03.208 INFO:teuthology.orchestra.run.smithi052.stdout: "op_reply": 2232, 2024-06-10T03:04:03.208 INFO:teuthology.orchestra.run.smithi052.stdout: "op_latency": { 2024-06-10T03:04:03.208 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 2232, 2024-06-10T03:04:03.208 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 4.352941841, 2024-06-10T03:04:03.208 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.001950242 2024-06-10T03:04:03.208 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:03.208 INFO:teuthology.orchestra.run.smithi052.stdout: "op_inflight": 0, 2024-06-10T03:04:03.208 INFO:teuthology.orchestra.run.smithi052.stdout: "oplen_avg": { 2024-06-10T03:04:03.209 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 2232, 2024-06-10T03:04:03.209 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 2233 2024-06-10T03:04:03.209 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:03.209 INFO:teuthology.orchestra.run.smithi052.stdout: "op": 2232, 2024-06-10T03:04:03.209 INFO:teuthology.orchestra.run.smithi052.stdout: "op_r": 2232, 2024-06-10T03:04:03.209 INFO:teuthology.orchestra.run.smithi052.stdout: "op_w": 0, 2024-06-10T03:04:03.209 INFO:teuthology.orchestra.run.smithi052.stdout: "op_rmw": 0, 2024-06-10T03:04:03.209 INFO:teuthology.orchestra.run.smithi052.stdout: "op_pg": 0, 2024-06-10T03:04:03.209 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_stat": 1446, 2024-06-10T03:04:03.209 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_create": 0, 2024-06-10T03:04:03.209 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_read": 784, 2024-06-10T03:04:03.209 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_write": 0, 2024-06-10T03:04:03.209 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_writefull": 0, 2024-06-10T03:04:03.209 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_writesame": 0, 2024-06-10T03:04:03.210 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_append": 0, 2024-06-10T03:04:03.210 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_zero": 0, 2024-06-10T03:04:03.210 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_truncate": 0, 2024-06-10T03:04:03.210 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_delete": 0, 2024-06-10T03:04:03.210 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_mapext": 0, 2024-06-10T03:04:03.210 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_sparse_read": 0, 2024-06-10T03:04:03.210 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_clonerange": 0, 2024-06-10T03:04:03.210 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_getxattr": 1, 2024-06-10T03:04:03.210 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_setxattr": 0, 2024-06-10T03:04:03.210 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_cmpxattr": 0, 2024-06-10T03:04:03.210 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_rmxattr": 0, 2024-06-10T03:04:03.210 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_resetxattrs": 0, 2024-06-10T03:04:03.210 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_call": 0, 2024-06-10T03:04:03.210 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_watch": 0, 2024-06-10T03:04:03.211 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_notify": 0, 2024-06-10T03:04:03.211 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_src_cmpxattr": 0, 2024-06-10T03:04:03.211 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_pgls": 0, 2024-06-10T03:04:03.211 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_pgls_filter": 0, 2024-06-10T03:04:03.211 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_other": 0, 2024-06-10T03:04:03.211 INFO:teuthology.orchestra.run.smithi052.stdout: "linger_active": 0, 2024-06-10T03:04:03.211 INFO:teuthology.orchestra.run.smithi052.stdout: "linger_send": 0, 2024-06-10T03:04:03.211 INFO:teuthology.orchestra.run.smithi052.stdout: "linger_resend": 0, 2024-06-10T03:04:03.211 INFO:teuthology.orchestra.run.smithi052.stdout: "linger_ping": 0, 2024-06-10T03:04:03.211 INFO:teuthology.orchestra.run.smithi052.stdout: "poolop_active": 0, 2024-06-10T03:04:03.211 INFO:teuthology.orchestra.run.smithi052.stdout: "poolop_send": 0, 2024-06-10T03:04:03.211 INFO:teuthology.orchestra.run.smithi052.stdout: "poolop_resend": 0, 2024-06-10T03:04:03.211 INFO:teuthology.orchestra.run.smithi052.stdout: "poolstat_active": 0, 2024-06-10T03:04:03.211 INFO:teuthology.orchestra.run.smithi052.stdout: "poolstat_send": 0, 2024-06-10T03:04:03.212 INFO:teuthology.orchestra.run.smithi052.stdout: "poolstat_resend": 0, 2024-06-10T03:04:03.212 INFO:teuthology.orchestra.run.smithi052.stdout: "statfs_active": 0, 2024-06-10T03:04:03.212 INFO:teuthology.orchestra.run.smithi052.stdout: "statfs_send": 0, 2024-06-10T03:04:03.212 INFO:teuthology.orchestra.run.smithi052.stdout: "statfs_resend": 0, 2024-06-10T03:04:03.212 INFO:teuthology.orchestra.run.smithi052.stdout: "command_active": 0, 2024-06-10T03:04:03.212 INFO:teuthology.orchestra.run.smithi052.stdout: "command_send": 0, 2024-06-10T03:04:03.212 INFO:teuthology.orchestra.run.smithi052.stdout: "command_resend": 0, 2024-06-10T03:04:03.212 INFO:teuthology.orchestra.run.smithi052.stdout: "map_epoch": 100, 2024-06-10T03:04:03.212 INFO:teuthology.orchestra.run.smithi052.stdout: "map_full": 0, 2024-06-10T03:04:03.212 INFO:teuthology.orchestra.run.smithi052.stdout: "map_inc": 2, 2024-06-10T03:04:03.212 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_sessions": 5, 2024-06-10T03:04:03.212 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_session_open": 5, 2024-06-10T03:04:03.212 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_session_close": 0, 2024-06-10T03:04:03.212 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_laggy": 0, 2024-06-10T03:04:03.213 INFO:teuthology.orchestra.run.smithi052.stdout: "omap_wr": 0, 2024-06-10T03:04:03.213 INFO:teuthology.orchestra.run.smithi052.stdout: "omap_rd": 2, 2024-06-10T03:04:03.213 INFO:teuthology.orchestra.run.smithi052.stdout: "omap_del": 0 2024-06-10T03:04:03.213 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:03.213 INFO:teuthology.orchestra.run.smithi052.stdout: "oft": { 2024-06-10T03:04:03.213 INFO:teuthology.orchestra.run.smithi052.stdout: "omap_total_objs": 0, 2024-06-10T03:04:03.213 INFO:teuthology.orchestra.run.smithi052.stdout: "omap_total_kv_pairs": 0, 2024-06-10T03:04:03.213 INFO:teuthology.orchestra.run.smithi052.stdout: "omap_total_updates": 0, 2024-06-10T03:04:03.213 INFO:teuthology.orchestra.run.smithi052.stdout: "omap_total_removes": 0 2024-06-10T03:04:03.213 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:03.213 INFO:teuthology.orchestra.run.smithi052.stdout: "osd-slow-ops": { 2024-06-10T03:04:03.213 INFO:teuthology.orchestra.run.smithi052.stdout: "slow_ops_count": 0 2024-06-10T03:04:03.214 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:03.214 INFO:teuthology.orchestra.run.smithi052.stdout: "purge_queue": { 2024-06-10T03:04:03.214 INFO:teuthology.orchestra.run.smithi052.stdout: "pq_executing_ops": 0, 2024-06-10T03:04:03.214 INFO:teuthology.orchestra.run.smithi052.stdout: "pq_executing_ops_high_water": 0, 2024-06-10T03:04:03.214 INFO:teuthology.orchestra.run.smithi052.stdout: "pq_executing": 0, 2024-06-10T03:04:03.214 INFO:teuthology.orchestra.run.smithi052.stdout: "pq_executing_high_water": 0, 2024-06-10T03:04:03.214 INFO:teuthology.orchestra.run.smithi052.stdout: "pq_executed_ops": 0, 2024-06-10T03:04:03.214 INFO:teuthology.orchestra.run.smithi052.stdout: "pq_executed": 0, 2024-06-10T03:04:03.214 INFO:teuthology.orchestra.run.smithi052.stdout: "pq_item_in_journal": 0 2024-06-10T03:04:03.214 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:03.214 INFO:teuthology.orchestra.run.smithi052.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-10T03:04:03.215 INFO:teuthology.orchestra.run.smithi052.stdout: "val": 0, 2024-06-10T03:04:03.215 INFO:teuthology.orchestra.run.smithi052.stdout: "max": 104857600, 2024-06-10T03:04:03.215 INFO:teuthology.orchestra.run.smithi052.stdout: "get_started": 0, 2024-06-10T03:04:03.215 INFO:teuthology.orchestra.run.smithi052.stdout: "get": 2492, 2024-06-10T03:04:03.215 INFO:teuthology.orchestra.run.smithi052.stdout: "get_sum": 1910114, 2024-06-10T03:04:03.215 INFO:teuthology.orchestra.run.smithi052.stdout: "get_or_fail_fail": 0, 2024-06-10T03:04:03.215 INFO:teuthology.orchestra.run.smithi052.stdout: "get_or_fail_success": 2492, 2024-06-10T03:04:03.215 INFO:teuthology.orchestra.run.smithi052.stdout: "take": 0, 2024-06-10T03:04:03.215 INFO:teuthology.orchestra.run.smithi052.stdout: "take_sum": 0, 2024-06-10T03:04:03.215 INFO:teuthology.orchestra.run.smithi052.stdout: "put": 2492, 2024-06-10T03:04:03.215 INFO:teuthology.orchestra.run.smithi052.stdout: "put_sum": 1910114, 2024-06-10T03:04:03.215 INFO:teuthology.orchestra.run.smithi052.stdout: "wait": { 2024-06-10T03:04:03.215 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-10T03:04:03.215 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-10T03:04:03.216 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-10T03:04:03.216 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:03.216 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:03.216 INFO:teuthology.orchestra.run.smithi052.stdout: "throttle-objecter_bytes": { 2024-06-10T03:04:03.216 INFO:teuthology.orchestra.run.smithi052.stdout: "val": 0, 2024-06-10T03:04:03.216 INFO:teuthology.orchestra.run.smithi052.stdout: "max": 104857600, 2024-06-10T03:04:03.216 INFO:teuthology.orchestra.run.smithi052.stdout: "get_started": 0, 2024-06-10T03:04:03.216 INFO:teuthology.orchestra.run.smithi052.stdout: "get": 0, 2024-06-10T03:04:03.216 INFO:teuthology.orchestra.run.smithi052.stdout: "get_sum": 0, 2024-06-10T03:04:03.216 INFO:teuthology.orchestra.run.smithi052.stdout: "get_or_fail_fail": 0, 2024-06-10T03:04:03.216 INFO:teuthology.orchestra.run.smithi052.stdout: "get_or_fail_success": 0, 2024-06-10T03:04:03.216 INFO:teuthology.orchestra.run.smithi052.stdout: "take": 2232, 2024-06-10T03:04:03.216 INFO:teuthology.orchestra.run.smithi052.stdout: "take_sum": 625566, 2024-06-10T03:04:03.216 INFO:teuthology.orchestra.run.smithi052.stdout: "put": 2232, 2024-06-10T03:04:03.217 INFO:teuthology.orchestra.run.smithi052.stdout: "put_sum": 625566, 2024-06-10T03:04:03.217 INFO:teuthology.orchestra.run.smithi052.stdout: "wait": { 2024-06-10T03:04:03.217 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-10T03:04:03.217 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-10T03:04:03.217 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-10T03:04:03.217 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:03.217 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:03.217 INFO:teuthology.orchestra.run.smithi052.stdout: "throttle-objecter_ops": { 2024-06-10T03:04:03.217 INFO:teuthology.orchestra.run.smithi052.stdout: "val": 0, 2024-06-10T03:04:03.217 INFO:teuthology.orchestra.run.smithi052.stdout: "max": 1024, 2024-06-10T03:04:03.217 INFO:teuthology.orchestra.run.smithi052.stdout: "get_started": 0, 2024-06-10T03:04:03.217 INFO:teuthology.orchestra.run.smithi052.stdout: "get": 0, 2024-06-10T03:04:03.217 INFO:teuthology.orchestra.run.smithi052.stdout: "get_sum": 0, 2024-06-10T03:04:03.217 INFO:teuthology.orchestra.run.smithi052.stdout: "get_or_fail_fail": 0, 2024-06-10T03:04:03.218 INFO:teuthology.orchestra.run.smithi052.stdout: "get_or_fail_success": 0, 2024-06-10T03:04:03.218 INFO:teuthology.orchestra.run.smithi052.stdout: "take": 2232, 2024-06-10T03:04:03.218 INFO:teuthology.orchestra.run.smithi052.stdout: "take_sum": 2232, 2024-06-10T03:04:03.218 INFO:teuthology.orchestra.run.smithi052.stdout: "put": 2232, 2024-06-10T03:04:03.218 INFO:teuthology.orchestra.run.smithi052.stdout: "put_sum": 2232, 2024-06-10T03:04:03.218 INFO:teuthology.orchestra.run.smithi052.stdout: "wait": { 2024-06-10T03:04:03.218 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-10T03:04:03.218 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-10T03:04:03.218 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-10T03:04:03.218 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:03.218 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:03.218 INFO:teuthology.orchestra.run.smithi052.stdout: "throttle-write_buf_throttle": { 2024-06-10T03:04:03.219 INFO:teuthology.orchestra.run.smithi052.stdout: "val": 0, 2024-06-10T03:04:03.219 INFO:teuthology.orchestra.run.smithi052.stdout: "max": 3758096384, 2024-06-10T03:04:03.219 INFO:teuthology.orchestra.run.smithi052.stdout: "get_started": 0, 2024-06-10T03:04:03.219 INFO:teuthology.orchestra.run.smithi052.stdout: "get": 0, 2024-06-10T03:04:03.219 INFO:teuthology.orchestra.run.smithi052.stdout: "get_sum": 0, 2024-06-10T03:04:03.219 INFO:teuthology.orchestra.run.smithi052.stdout: "get_or_fail_fail": 0, 2024-06-10T03:04:03.219 INFO:teuthology.orchestra.run.smithi052.stdout: "get_or_fail_success": 0, 2024-06-10T03:04:03.219 INFO:teuthology.orchestra.run.smithi052.stdout: "take": 0, 2024-06-10T03:04:03.219 INFO:teuthology.orchestra.run.smithi052.stdout: "take_sum": 0, 2024-06-10T03:04:03.219 INFO:teuthology.orchestra.run.smithi052.stdout: "put": 0, 2024-06-10T03:04:03.220 INFO:teuthology.orchestra.run.smithi052.stdout: "put_sum": 0, 2024-06-10T03:04:03.220 INFO:teuthology.orchestra.run.smithi052.stdout: "wait": { 2024-06-10T03:04:03.220 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-10T03:04:03.220 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-10T03:04:03.220 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-10T03:04:03.220 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:03.220 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:03.220 INFO:teuthology.orchestra.run.smithi052.stdout: "throttle-write_buf_throttle-0x56291396a6e0": { 2024-06-10T03:04:03.220 INFO:teuthology.orchestra.run.smithi052.stdout: "val": 0, 2024-06-10T03:04:03.220 INFO:teuthology.orchestra.run.smithi052.stdout: "max": 3758096384, 2024-06-10T03:04:03.220 INFO:teuthology.orchestra.run.smithi052.stdout: "get_started": 0, 2024-06-10T03:04:03.220 INFO:teuthology.orchestra.run.smithi052.stdout: "get": 0, 2024-06-10T03:04:03.220 INFO:teuthology.orchestra.run.smithi052.stdout: "get_sum": 0, 2024-06-10T03:04:03.220 INFO:teuthology.orchestra.run.smithi052.stdout: "get_or_fail_fail": 0, 2024-06-10T03:04:03.220 INFO:teuthology.orchestra.run.smithi052.stdout: "get_or_fail_success": 0, 2024-06-10T03:04:03.221 INFO:teuthology.orchestra.run.smithi052.stdout: "take": 0, 2024-06-10T03:04:03.221 INFO:teuthology.orchestra.run.smithi052.stdout: "take_sum": 0, 2024-06-10T03:04:03.221 INFO:teuthology.orchestra.run.smithi052.stdout: "put": 0, 2024-06-10T03:04:03.221 INFO:teuthology.orchestra.run.smithi052.stdout: "put_sum": 0, 2024-06-10T03:04:03.221 INFO:teuthology.orchestra.run.smithi052.stdout: "wait": { 2024-06-10T03:04:03.221 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-10T03:04:03.221 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-10T03:04:03.221 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-10T03:04:03.221 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:03.221 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:03.221 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-10T03:04:04.031 INFO:tasks.check_counter:Daemon mds.a mds.exported=0 2024-06-10T03:04:04.031 INFO:tasks.check_counter:Daemon mds.a mds.imported=0 2024-06-10T03:04:04.031 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update=0 2024-06-10T03:04:04.031 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update_receipt=0 2024-06-10T03:04:04.031 INFO:tasks.check_counter:Daemon mds.a mds.dir_split=0 2024-06-10T03:04:04.031 DEBUG:tasks.check_counter:Getting stats from d 2024-06-10T03:04:04.031 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph daemon mds.d perf dump 2024-06-10T03:04:04.291 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T03:04:04.638 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:04:04 smithi120 ceph-mon[36634]: pgmap v730: 129 pgs: 129 active+clean; 228 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-06-10T03:04:04.642 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:04:04 smithi052 ceph-mon[32362]: pgmap v730: 129 pgs: 129 active+clean; 228 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-06-10T03:04:04.704 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:04:04 smithi188 ceph-mon[36599]: pgmap v730: 129 pgs: 129 active+clean; 228 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-06-10T03:04:05.638 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:04:05 smithi120 ceph-mon[36634]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-10T03:04:05.645 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:04:05 smithi052 ceph-mon[32362]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-10T03:04:05.704 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:04:05 smithi188 ceph-mon[36599]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-10T03:04:05.930 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-10T03:04:05.930 INFO:teuthology.orchestra.run.smithi052.stdout: "AsyncMessenger::Worker-0": { 2024-06-10T03:04:05.930 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_messages": 241, 2024-06-10T03:04:05.931 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_messages": 236, 2024-06-10T03:04:05.931 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_bytes": 759744, 2024-06-10T03:04:05.931 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_bytes": 145344, 2024-06-10T03:04:05.931 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_created_connections": 4, 2024-06-10T03:04:05.931 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_active_connections": 0, 2024-06-10T03:04:05.931 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_total_time": 0.037464753, 2024-06-10T03:04:05.931 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_send_time": 0.014238201, 2024-06-10T03:04:05.931 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_recv_time": 0.019719283, 2024-06-10T03:04:05.931 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_fast_dispatch_time": 0.004570006, 2024-06-10T03:04:05.931 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_messages_queue_lat": { 2024-06-10T03:04:05.931 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 235, 2024-06-10T03:04:05.931 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.009731051, 2024-06-10T03:04:05.931 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000041408 2024-06-10T03:04:05.932 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:05.932 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_handle_ack_lat": { 2024-06-10T03:04:05.932 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-10T03:04:05.932 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-10T03:04:05.932 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-10T03:04:05.932 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:05.932 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_encrypted_bytes": 759744, 2024-06-10T03:04:05.932 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_encrypted_bytes": 145344 2024-06-10T03:04:05.932 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:05.932 INFO:teuthology.orchestra.run.smithi052.stdout: "AsyncMessenger::Worker-1": { 2024-06-10T03:04:05.932 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_messages": 5, 2024-06-10T03:04:05.932 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_messages": 182, 2024-06-10T03:04:05.932 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_bytes": 3232, 2024-06-10T03:04:05.933 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_bytes": 110000, 2024-06-10T03:04:05.933 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_created_connections": 3, 2024-06-10T03:04:05.933 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_active_connections": 2, 2024-06-10T03:04:05.933 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_total_time": 0.013784671, 2024-06-10T03:04:05.933 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_send_time": 0.010625517, 2024-06-10T03:04:05.933 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_recv_time": 0.001484952, 2024-06-10T03:04:05.933 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-06-10T03:04:05.933 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_messages_queue_lat": { 2024-06-10T03:04:05.933 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 181, 2024-06-10T03:04:05.933 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.011706243, 2024-06-10T03:04:05.933 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000064675 2024-06-10T03:04:05.933 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:05.933 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_handle_ack_lat": { 2024-06-10T03:04:05.934 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-10T03:04:05.934 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-10T03:04:05.934 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-10T03:04:05.934 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:05.934 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_encrypted_bytes": 3232, 2024-06-10T03:04:05.934 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_encrypted_bytes": 110000 2024-06-10T03:04:05.934 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:05.934 INFO:teuthology.orchestra.run.smithi052.stdout: "AsyncMessenger::Worker-2": { 2024-06-10T03:04:05.934 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_messages": 0, 2024-06-10T03:04:05.934 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_messages": 2, 2024-06-10T03:04:05.934 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_bytes": 0, 2024-06-10T03:04:05.934 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_bytes": 0, 2024-06-10T03:04:05.934 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_created_connections": 2, 2024-06-10T03:04:05.935 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_active_connections": 1, 2024-06-10T03:04:05.935 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_total_time": 0.000359359, 2024-06-10T03:04:05.935 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_send_time": 0.000000000, 2024-06-10T03:04:05.935 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_recv_time": 0.000276008, 2024-06-10T03:04:05.935 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-06-10T03:04:05.935 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_messages_queue_lat": { 2024-06-10T03:04:05.935 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-10T03:04:05.935 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-10T03:04:05.935 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-10T03:04:05.935 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:05.935 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_handle_ack_lat": { 2024-06-10T03:04:05.935 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-10T03:04:05.935 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-10T03:04:05.936 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-10T03:04:05.936 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:05.936 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_encrypted_bytes": 0, 2024-06-10T03:04:05.936 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_encrypted_bytes": 0 2024-06-10T03:04:05.936 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:05.936 INFO:teuthology.orchestra.run.smithi052.stdout: "cct": { 2024-06-10T03:04:05.936 INFO:teuthology.orchestra.run.smithi052.stdout: "total_workers": 0, 2024-06-10T03:04:05.936 INFO:teuthology.orchestra.run.smithi052.stdout: "unhealthy_workers": 0 2024-06-10T03:04:05.936 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:05.936 INFO:teuthology.orchestra.run.smithi052.stdout: "mempool": { 2024-06-10T03:04:05.936 INFO:teuthology.orchestra.run.smithi052.stdout: "bloom_filter_bytes": 0, 2024-06-10T03:04:05.936 INFO:teuthology.orchestra.run.smithi052.stdout: "bloom_filter_items": 0, 2024-06-10T03:04:05.936 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_alloc_bytes": 0, 2024-06-10T03:04:05.936 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_alloc_items": 0, 2024-06-10T03:04:05.937 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_data_bytes": 0, 2024-06-10T03:04:05.937 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_data_items": 0, 2024-06-10T03:04:05.937 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-10T03:04:05.937 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_onode_items": 0, 2024-06-10T03:04:05.937 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-10T03:04:05.937 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_meta_items": 0, 2024-06-10T03:04:05.937 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_other_bytes": 0, 2024-06-10T03:04:05.937 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_other_items": 0, 2024-06-10T03:04:05.937 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-10T03:04:05.937 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_buffer_items": 0, 2024-06-10T03:04:05.937 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_extent_bytes": 0, 2024-06-10T03:04:05.937 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_extent_items": 0, 2024-06-10T03:04:05.937 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_blob_bytes": 0, 2024-06-10T03:04:05.938 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_blob_items": 0, 2024-06-10T03:04:05.938 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-10T03:04:05.938 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_shared_blob_items": 0, 2024-06-10T03:04:05.938 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-10T03:04:05.938 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_inline_bl_items": 0, 2024-06-10T03:04:05.938 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_fsck_bytes": 0, 2024-06-10T03:04:05.938 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_fsck_items": 0, 2024-06-10T03:04:05.938 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_txc_bytes": 0, 2024-06-10T03:04:05.938 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_txc_items": 0, 2024-06-10T03:04:05.938 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-10T03:04:05.938 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_writing_deferred_items": 0, 2024-06-10T03:04:05.938 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_writing_bytes": 0, 2024-06-10T03:04:05.939 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_writing_items": 0, 2024-06-10T03:04:05.939 INFO:teuthology.orchestra.run.smithi052.stdout: "bluefs_bytes": 0, 2024-06-10T03:04:05.939 INFO:teuthology.orchestra.run.smithi052.stdout: "bluefs_items": 0, 2024-06-10T03:04:05.939 INFO:teuthology.orchestra.run.smithi052.stdout: "bluefs_file_reader_bytes": 0, 2024-06-10T03:04:05.939 INFO:teuthology.orchestra.run.smithi052.stdout: "bluefs_file_reader_items": 0, 2024-06-10T03:04:05.939 INFO:teuthology.orchestra.run.smithi052.stdout: "bluefs_file_writer_bytes": 0, 2024-06-10T03:04:05.939 INFO:teuthology.orchestra.run.smithi052.stdout: "bluefs_file_writer_items": 0, 2024-06-10T03:04:05.939 INFO:teuthology.orchestra.run.smithi052.stdout: "buffer_anon_bytes": 148967, 2024-06-10T03:04:05.939 INFO:teuthology.orchestra.run.smithi052.stdout: "buffer_anon_items": 29, 2024-06-10T03:04:05.939 INFO:teuthology.orchestra.run.smithi052.stdout: "buffer_meta_bytes": 0, 2024-06-10T03:04:05.939 INFO:teuthology.orchestra.run.smithi052.stdout: "buffer_meta_items": 0, 2024-06-10T03:04:05.939 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_bytes": 0, 2024-06-10T03:04:05.939 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_items": 0, 2024-06-10T03:04:05.939 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_mapbl_bytes": 0, 2024-06-10T03:04:05.940 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_mapbl_items": 0, 2024-06-10T03:04:05.940 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_pglog_bytes": 0, 2024-06-10T03:04:05.940 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_pglog_items": 0, 2024-06-10T03:04:05.940 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_bytes": 0, 2024-06-10T03:04:05.940 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_items": 0, 2024-06-10T03:04:05.940 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_mapping_bytes": 0, 2024-06-10T03:04:05.940 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_mapping_items": 0, 2024-06-10T03:04:05.940 INFO:teuthology.orchestra.run.smithi052.stdout: "pgmap_bytes": 0, 2024-06-10T03:04:05.940 INFO:teuthology.orchestra.run.smithi052.stdout: "pgmap_items": 0, 2024-06-10T03:04:05.940 INFO:teuthology.orchestra.run.smithi052.stdout: "mds_co_bytes": 928, 2024-06-10T03:04:05.940 INFO:teuthology.orchestra.run.smithi052.stdout: "mds_co_items": 2, 2024-06-10T03:04:05.940 INFO:teuthology.orchestra.run.smithi052.stdout: "unittest_1_bytes": 0, 2024-06-10T03:04:05.940 INFO:teuthology.orchestra.run.smithi052.stdout: "unittest_1_items": 0, 2024-06-10T03:04:05.940 INFO:teuthology.orchestra.run.smithi052.stdout: "unittest_2_bytes": 0, 2024-06-10T03:04:05.941 INFO:teuthology.orchestra.run.smithi052.stdout: "unittest_2_items": 0 2024-06-10T03:04:05.941 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:05.941 INFO:teuthology.orchestra.run.smithi052.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-10T03:04:05.941 INFO:teuthology.orchestra.run.smithi052.stdout: "val": 0, 2024-06-10T03:04:05.941 INFO:teuthology.orchestra.run.smithi052.stdout: "max": 104857600, 2024-06-10T03:04:05.941 INFO:teuthology.orchestra.run.smithi052.stdout: "get_started": 0, 2024-06-10T03:04:05.941 INFO:teuthology.orchestra.run.smithi052.stdout: "get": 242, 2024-06-10T03:04:05.941 INFO:teuthology.orchestra.run.smithi052.stdout: "get_sum": 725695, 2024-06-10T03:04:05.941 INFO:teuthology.orchestra.run.smithi052.stdout: "get_or_fail_fail": 0, 2024-06-10T03:04:05.941 INFO:teuthology.orchestra.run.smithi052.stdout: "get_or_fail_success": 242, 2024-06-10T03:04:05.941 INFO:teuthology.orchestra.run.smithi052.stdout: "take": 0, 2024-06-10T03:04:05.941 INFO:teuthology.orchestra.run.smithi052.stdout: "take_sum": 0, 2024-06-10T03:04:05.941 INFO:teuthology.orchestra.run.smithi052.stdout: "put": 242, 2024-06-10T03:04:05.942 INFO:teuthology.orchestra.run.smithi052.stdout: "put_sum": 725695, 2024-06-10T03:04:05.942 INFO:teuthology.orchestra.run.smithi052.stdout: "wait": { 2024-06-10T03:04:05.942 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-10T03:04:05.942 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-10T03:04:05.942 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-10T03:04:05.942 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:05.942 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:05.942 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-10T03:04:06.520 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:04:06 smithi052 ceph-mon[32362]: pgmap v731: 129 pgs: 129 active+clean; 228 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-10T03:04:06.638 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:04:06 smithi120 ceph-mon[36634]: pgmap v731: 129 pgs: 129 active+clean; 228 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-10T03:04:06.693 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-06-10T03:04:06.693 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-06-10T03:04:06.693 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.d 2024-06-10T03:04:06.693 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.d 2024-06-10T03:04:06.693 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.d 2024-06-10T03:04:06.694 DEBUG:tasks.check_counter:Getting stats from g 2024-06-10T03:04:06.694 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph daemon mds.g perf dump 2024-06-10T03:04:06.703 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:04:06 smithi188 ceph-mon[36599]: pgmap v731: 129 pgs: 129 active+clean; 228 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-10T03:04:06.945 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T03:04:08.638 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:04:08 smithi120 ceph-mon[36634]: pgmap v732: 129 pgs: 129 active+clean; 228 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-10T03:04:08.660 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:04:08 smithi052 ceph-mon[32362]: pgmap v732: 129 pgs: 129 active+clean; 228 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-10T03:04:08.701 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-10T03:04:08.702 INFO:teuthology.orchestra.run.smithi052.stdout: "AsyncMessenger::Worker-0": { 2024-06-10T03:04:08.702 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_messages": 868, 2024-06-10T03:04:08.702 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_messages": 868, 2024-06-10T03:04:08.702 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_bytes": 284591, 2024-06-10T03:04:08.702 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_bytes": 275201, 2024-06-10T03:04:08.702 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_created_connections": 6, 2024-06-10T03:04:08.702 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_active_connections": 2, 2024-06-10T03:04:08.702 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_total_time": 0.091663362, 2024-06-10T03:04:08.702 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_send_time": 0.035754558, 2024-06-10T03:04:08.702 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_recv_time": 0.021231621, 2024-06-10T03:04:08.702 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_fast_dispatch_time": 0.026230232, 2024-06-10T03:04:08.702 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_messages_queue_lat": { 2024-06-10T03:04:08.703 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 867, 2024-06-10T03:04:08.703 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.045126691, 2024-06-10T03:04:08.703 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000052049 2024-06-10T03:04:08.703 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:08.703 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_handle_ack_lat": { 2024-06-10T03:04:08.703 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-10T03:04:08.703 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-10T03:04:08.703 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-10T03:04:08.703 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:08.703 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_encrypted_bytes": 3072, 2024-06-10T03:04:08.703 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_encrypted_bytes": 448 2024-06-10T03:04:08.703 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:08.704 INFO:teuthology.orchestra.run.smithi052.stdout: "AsyncMessenger::Worker-1": { 2024-06-10T03:04:08.704 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_messages": 1716, 2024-06-10T03:04:08.704 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_messages": 1693, 2024-06-10T03:04:08.704 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_bytes": 10997310, 2024-06-10T03:04:08.704 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_bytes": 606719, 2024-06-10T03:04:08.704 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_created_connections": 4, 2024-06-10T03:04:08.704 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_active_connections": 3, 2024-06-10T03:04:08.704 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_total_time": 0.139653869, 2024-06-10T03:04:08.704 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_send_time": 0.040815500, 2024-06-10T03:04:08.704 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_recv_time": 0.054200144, 2024-06-10T03:04:08.704 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_fast_dispatch_time": 0.040487763, 2024-06-10T03:04:08.704 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_messages_queue_lat": { 2024-06-10T03:04:08.704 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 1692, 2024-06-10T03:04:08.705 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.058837498, 2024-06-10T03:04:08.705 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000034773 2024-06-10T03:04:08.705 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:08.705 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_handle_ack_lat": { 2024-06-10T03:04:08.705 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-10T03:04:08.705 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-10T03:04:08.705 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-10T03:04:08.705 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:08.705 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_encrypted_bytes": 898080, 2024-06-10T03:04:08.705 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_encrypted_bytes": 144608 2024-06-10T03:04:08.705 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:08.705 INFO:teuthology.orchestra.run.smithi052.stdout: "AsyncMessenger::Worker-2": { 2024-06-10T03:04:08.705 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_messages": 4, 2024-06-10T03:04:08.706 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_messages": 180, 2024-06-10T03:04:08.706 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_bytes": 1508, 2024-06-10T03:04:08.706 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_bytes": 369749, 2024-06-10T03:04:08.706 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_created_connections": 5, 2024-06-10T03:04:08.706 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_active_connections": 4, 2024-06-10T03:04:08.706 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_total_time": 0.013458460, 2024-06-10T03:04:08.706 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_send_time": 0.009988761, 2024-06-10T03:04:08.706 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_recv_time": 0.001901584, 2024-06-10T03:04:08.706 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_fast_dispatch_time": 0.000091941, 2024-06-10T03:04:08.706 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_messages_queue_lat": { 2024-06-10T03:04:08.706 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 178, 2024-06-10T03:04:08.706 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.008237746, 2024-06-10T03:04:08.707 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000046279 2024-06-10T03:04:08.707 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:08.707 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_handle_ack_lat": { 2024-06-10T03:04:08.707 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-10T03:04:08.707 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-10T03:04:08.707 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-10T03:04:08.707 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:08.707 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_encrypted_bytes": 160, 2024-06-10T03:04:08.707 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_encrypted_bytes": 368784 2024-06-10T03:04:08.707 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:08.707 INFO:teuthology.orchestra.run.smithi052.stdout: "cct": { 2024-06-10T03:04:08.707 INFO:teuthology.orchestra.run.smithi052.stdout: "total_workers": 1, 2024-06-10T03:04:08.707 INFO:teuthology.orchestra.run.smithi052.stdout: "unhealthy_workers": 0 2024-06-10T03:04:08.707 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:08.708 INFO:teuthology.orchestra.run.smithi052.stdout: "finisher-MDSRank": { 2024-06-10T03:04:08.708 INFO:teuthology.orchestra.run.smithi052.stdout: "queue_len": 0, 2024-06-10T03:04:08.708 INFO:teuthology.orchestra.run.smithi052.stdout: "complete_latency": { 2024-06-10T03:04:08.708 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 3893, 2024-06-10T03:04:08.708 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.100494186, 2024-06-10T03:04:08.708 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000025814 2024-06-10T03:04:08.708 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:08.708 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:08.708 INFO:teuthology.orchestra.run.smithi052.stdout: "finisher-PurgeQueue": { 2024-06-10T03:04:08.708 INFO:teuthology.orchestra.run.smithi052.stdout: "queue_len": 0, 2024-06-10T03:04:08.708 INFO:teuthology.orchestra.run.smithi052.stdout: "complete_latency": { 2024-06-10T03:04:08.708 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-10T03:04:08.708 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-10T03:04:08.708 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-10T03:04:08.709 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:08.709 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:08.709 INFO:teuthology.orchestra.run.smithi052.stdout: "mds": { 2024-06-10T03:04:08.709 INFO:teuthology.orchestra.run.smithi052.stdout: "request": 0, 2024-06-10T03:04:08.709 INFO:teuthology.orchestra.run.smithi052.stdout: "reply": 0, 2024-06-10T03:04:08.709 INFO:teuthology.orchestra.run.smithi052.stdout: "reply_latency": { 2024-06-10T03:04:08.709 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-10T03:04:08.709 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-10T03:04:08.709 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-10T03:04:08.709 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:08.709 INFO:teuthology.orchestra.run.smithi052.stdout: "slow_reply": 0, 2024-06-10T03:04:08.709 INFO:teuthology.orchestra.run.smithi052.stdout: "forward": 0, 2024-06-10T03:04:08.710 INFO:teuthology.orchestra.run.smithi052.stdout: "dir_fetch_complete": 0, 2024-06-10T03:04:08.710 INFO:teuthology.orchestra.run.smithi052.stdout: "dir_fetch_keys": 0, 2024-06-10T03:04:08.710 INFO:teuthology.orchestra.run.smithi052.stdout: "dir_commit": 0, 2024-06-10T03:04:08.710 INFO:teuthology.orchestra.run.smithi052.stdout: "dir_split": 0, 2024-06-10T03:04:08.710 INFO:teuthology.orchestra.run.smithi052.stdout: "dir_merge": 0, 2024-06-10T03:04:08.710 INFO:teuthology.orchestra.run.smithi052.stdout: "inodes": 1326, 2024-06-10T03:04:08.710 INFO:teuthology.orchestra.run.smithi052.stdout: "inodes_top": 0, 2024-06-10T03:04:08.710 INFO:teuthology.orchestra.run.smithi052.stdout: "inodes_bottom": 1326, 2024-06-10T03:04:08.710 INFO:teuthology.orchestra.run.smithi052.stdout: "inodes_pin_tail": 0, 2024-06-10T03:04:08.710 INFO:teuthology.orchestra.run.smithi052.stdout: "inodes_pinned": 1326, 2024-06-10T03:04:08.710 INFO:teuthology.orchestra.run.smithi052.stdout: "inodes_expired": 0, 2024-06-10T03:04:08.710 INFO:teuthology.orchestra.run.smithi052.stdout: "inodes_with_caps": 0, 2024-06-10T03:04:08.710 INFO:teuthology.orchestra.run.smithi052.stdout: "caps": 0, 2024-06-10T03:04:08.711 INFO:teuthology.orchestra.run.smithi052.stdout: "subtrees": 1, 2024-06-10T03:04:08.711 INFO:teuthology.orchestra.run.smithi052.stdout: "traverse": 0, 2024-06-10T03:04:08.711 INFO:teuthology.orchestra.run.smithi052.stdout: "traverse_hit": 0, 2024-06-10T03:04:08.711 INFO:teuthology.orchestra.run.smithi052.stdout: "traverse_forward": 0, 2024-06-10T03:04:08.711 INFO:teuthology.orchestra.run.smithi052.stdout: "traverse_discover": 0, 2024-06-10T03:04:08.711 INFO:teuthology.orchestra.run.smithi052.stdout: "traverse_dir_fetch": 0, 2024-06-10T03:04:08.711 INFO:teuthology.orchestra.run.smithi052.stdout: "traverse_remote_ino": 0, 2024-06-10T03:04:08.711 INFO:teuthology.orchestra.run.smithi052.stdout: "traverse_lock": 0, 2024-06-10T03:04:08.711 INFO:teuthology.orchestra.run.smithi052.stdout: "load_cent": 0, 2024-06-10T03:04:08.711 INFO:teuthology.orchestra.run.smithi052.stdout: "q": 0, 2024-06-10T03:04:08.711 INFO:teuthology.orchestra.run.smithi052.stdout: "exported": 0, 2024-06-10T03:04:08.711 INFO:teuthology.orchestra.run.smithi052.stdout: "exported_inodes": 0, 2024-06-10T03:04:08.711 INFO:teuthology.orchestra.run.smithi052.stdout: "imported": 0, 2024-06-10T03:04:08.711 INFO:teuthology.orchestra.run.smithi052.stdout: "imported_inodes": 0, 2024-06-10T03:04:08.711 INFO:teuthology.orchestra.run.smithi052.stdout: "openino_dir_fetch": 0, 2024-06-10T03:04:08.712 INFO:teuthology.orchestra.run.smithi052.stdout: "openino_backtrace_fetch": 0, 2024-06-10T03:04:08.712 INFO:teuthology.orchestra.run.smithi052.stdout: "openino_peer_discover": 0, 2024-06-10T03:04:08.712 INFO:teuthology.orchestra.run.smithi052.stdout: "root_rfiles": 0, 2024-06-10T03:04:08.712 INFO:teuthology.orchestra.run.smithi052.stdout: "root_rbytes": 0, 2024-06-10T03:04:08.712 INFO:teuthology.orchestra.run.smithi052.stdout: "root_rsnaps": 0, 2024-06-10T03:04:08.712 INFO:teuthology.orchestra.run.smithi052.stdout: "scrub_backtrace_fetch": 0, 2024-06-10T03:04:08.712 INFO:teuthology.orchestra.run.smithi052.stdout: "scrub_set_tag": 0, 2024-06-10T03:04:08.712 INFO:teuthology.orchestra.run.smithi052.stdout: "scrub_backtrace_repaired": 0, 2024-06-10T03:04:08.712 INFO:teuthology.orchestra.run.smithi052.stdout: "scrub_inotable_repaired": 0, 2024-06-10T03:04:08.712 INFO:teuthology.orchestra.run.smithi052.stdout: "scrub_dir_inodes": 0, 2024-06-10T03:04:08.712 INFO:teuthology.orchestra.run.smithi052.stdout: "scrub_dir_base_inodes": 0, 2024-06-10T03:04:08.712 INFO:teuthology.orchestra.run.smithi052.stdout: "scrub_dirfrag_rstats": 0, 2024-06-10T03:04:08.712 INFO:teuthology.orchestra.run.smithi052.stdout: "scrub_file_inodes": 0, 2024-06-10T03:04:08.712 INFO:teuthology.orchestra.run.smithi052.stdout: "handle_inode_file_caps": 0, 2024-06-10T03:04:08.713 INFO:teuthology.orchestra.run.smithi052.stdout: "ceph_cap_op_revoke": 0, 2024-06-10T03:04:08.713 INFO:teuthology.orchestra.run.smithi052.stdout: "ceph_cap_op_grant": 0, 2024-06-10T03:04:08.713 INFO:teuthology.orchestra.run.smithi052.stdout: "ceph_cap_op_trunc": 0, 2024-06-10T03:04:08.713 INFO:teuthology.orchestra.run.smithi052.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-06-10T03:04:08.713 INFO:teuthology.orchestra.run.smithi052.stdout: "ceph_cap_op_flush_ack": 0, 2024-06-10T03:04:08.713 INFO:teuthology.orchestra.run.smithi052.stdout: "handle_client_caps": 0, 2024-06-10T03:04:08.713 INFO:teuthology.orchestra.run.smithi052.stdout: "handle_client_caps_dirty": 0, 2024-06-10T03:04:08.713 INFO:teuthology.orchestra.run.smithi052.stdout: "handle_client_cap_release": 0, 2024-06-10T03:04:08.713 INFO:teuthology.orchestra.run.smithi052.stdout: "process_request_cap_release": 0 2024-06-10T03:04:08.713 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:08.713 INFO:teuthology.orchestra.run.smithi052.stdout: "mds_cache": { 2024-06-10T03:04:08.713 INFO:teuthology.orchestra.run.smithi052.stdout: "dir_update": 0, 2024-06-10T03:04:08.713 INFO:teuthology.orchestra.run.smithi052.stdout: "dir_update_receipt": 0, 2024-06-10T03:04:08.713 INFO:teuthology.orchestra.run.smithi052.stdout: "dir_try_discover": 0, 2024-06-10T03:04:08.713 INFO:teuthology.orchestra.run.smithi052.stdout: "dir_send_discover": 0, 2024-06-10T03:04:08.714 INFO:teuthology.orchestra.run.smithi052.stdout: "dir_handle_discover": 0, 2024-06-10T03:04:08.714 INFO:teuthology.orchestra.run.smithi052.stdout: "num_strays": 0, 2024-06-10T03:04:08.714 INFO:teuthology.orchestra.run.smithi052.stdout: "num_strays_delayed": 0, 2024-06-10T03:04:08.714 INFO:teuthology.orchestra.run.smithi052.stdout: "num_strays_enqueuing": 0, 2024-06-10T03:04:08.714 INFO:teuthology.orchestra.run.smithi052.stdout: "strays_created": 0, 2024-06-10T03:04:08.714 INFO:teuthology.orchestra.run.smithi052.stdout: "strays_enqueued": 0, 2024-06-10T03:04:08.714 INFO:teuthology.orchestra.run.smithi052.stdout: "strays_reintegrated": 0, 2024-06-10T03:04:08.714 INFO:teuthology.orchestra.run.smithi052.stdout: "strays_migrated": 0, 2024-06-10T03:04:08.714 INFO:teuthology.orchestra.run.smithi052.stdout: "num_recovering_processing": 0, 2024-06-10T03:04:08.714 INFO:teuthology.orchestra.run.smithi052.stdout: "num_recovering_enqueued": 0, 2024-06-10T03:04:08.714 INFO:teuthology.orchestra.run.smithi052.stdout: "num_recovering_prioritized": 0, 2024-06-10T03:04:08.714 INFO:teuthology.orchestra.run.smithi052.stdout: "recovery_started": 0, 2024-06-10T03:04:08.714 INFO:teuthology.orchestra.run.smithi052.stdout: "recovery_completed": 0, 2024-06-10T03:04:08.714 INFO:teuthology.orchestra.run.smithi052.stdout: "ireq_quiesce_path": 0, 2024-06-10T03:04:08.715 INFO:teuthology.orchestra.run.smithi052.stdout: "ireq_quiesce_inode": 0, 2024-06-10T03:04:08.715 INFO:teuthology.orchestra.run.smithi052.stdout: "ireq_enqueue_scrub": 0, 2024-06-10T03:04:08.715 INFO:teuthology.orchestra.run.smithi052.stdout: "ireq_exportdir": 0, 2024-06-10T03:04:08.715 INFO:teuthology.orchestra.run.smithi052.stdout: "ireq_flush": 0, 2024-06-10T03:04:08.715 INFO:teuthology.orchestra.run.smithi052.stdout: "ireq_fragmentdir": 0, 2024-06-10T03:04:08.715 INFO:teuthology.orchestra.run.smithi052.stdout: "ireq_fragstats": 0, 2024-06-10T03:04:08.715 INFO:teuthology.orchestra.run.smithi052.stdout: "ireq_inodestats": 0 2024-06-10T03:04:08.715 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:08.715 INFO:teuthology.orchestra.run.smithi052.stdout: "mds_log": { 2024-06-10T03:04:08.715 INFO:teuthology.orchestra.run.smithi052.stdout: "evlrg": 0, 2024-06-10T03:04:08.715 INFO:teuthology.orchestra.run.smithi052.stdout: "evadd": 0, 2024-06-10T03:04:08.715 INFO:teuthology.orchestra.run.smithi052.stdout: "evex": 0, 2024-06-10T03:04:08.715 INFO:teuthology.orchestra.run.smithi052.stdout: "evtrm": 0, 2024-06-10T03:04:08.715 INFO:teuthology.orchestra.run.smithi052.stdout: "ev": 6535, 2024-06-10T03:04:08.716 INFO:teuthology.orchestra.run.smithi052.stdout: "evexg": 0, 2024-06-10T03:04:08.716 INFO:teuthology.orchestra.run.smithi052.stdout: "evexd": 0, 2024-06-10T03:04:08.716 INFO:teuthology.orchestra.run.smithi052.stdout: "segadd": 0, 2024-06-10T03:04:08.716 INFO:teuthology.orchestra.run.smithi052.stdout: "segex": 0, 2024-06-10T03:04:08.716 INFO:teuthology.orchestra.run.smithi052.stdout: "segtrm": 0, 2024-06-10T03:04:08.716 INFO:teuthology.orchestra.run.smithi052.stdout: "seg": 8, 2024-06-10T03:04:08.716 INFO:teuthology.orchestra.run.smithi052.stdout: "segmjr": 2, 2024-06-10T03:04:08.716 INFO:teuthology.orchestra.run.smithi052.stdout: "segexg": 0, 2024-06-10T03:04:08.716 INFO:teuthology.orchestra.run.smithi052.stdout: "segexd": 0, 2024-06-10T03:04:08.716 INFO:teuthology.orchestra.run.smithi052.stdout: "expos": 4194304, 2024-06-10T03:04:08.716 INFO:teuthology.orchestra.run.smithi052.stdout: "wrpos": 13923504, 2024-06-10T03:04:08.716 INFO:teuthology.orchestra.run.smithi052.stdout: "rdpos": 13922366, 2024-06-10T03:04:08.716 INFO:teuthology.orchestra.run.smithi052.stdout: "jlat": { 2024-06-10T03:04:08.716 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-10T03:04:08.716 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-10T03:04:08.717 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-10T03:04:08.717 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:08.717 INFO:teuthology.orchestra.run.smithi052.stdout: "replayed": 6535 2024-06-10T03:04:08.717 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:08.717 INFO:teuthology.orchestra.run.smithi052.stdout: "mds_mem": { 2024-06-10T03:04:08.717 INFO:teuthology.orchestra.run.smithi052.stdout: "ino": 10, 2024-06-10T03:04:08.717 INFO:teuthology.orchestra.run.smithi052.stdout: "ino+": 1356, 2024-06-10T03:04:08.717 INFO:teuthology.orchestra.run.smithi052.stdout: "ino-": 1346, 2024-06-10T03:04:08.717 INFO:teuthology.orchestra.run.smithi052.stdout: "dir": 8, 2024-06-10T03:04:08.717 INFO:teuthology.orchestra.run.smithi052.stdout: "dir+": 84, 2024-06-10T03:04:08.717 INFO:teuthology.orchestra.run.smithi052.stdout: "dir-": 76, 2024-06-10T03:04:08.717 INFO:teuthology.orchestra.run.smithi052.stdout: "dn": 1326, 2024-06-10T03:04:08.717 INFO:teuthology.orchestra.run.smithi052.stdout: "dn+": 3376, 2024-06-10T03:04:08.717 INFO:teuthology.orchestra.run.smithi052.stdout: "dn-": 2050, 2024-06-10T03:04:08.717 INFO:teuthology.orchestra.run.smithi052.stdout: "cap": 0, 2024-06-10T03:04:08.718 INFO:teuthology.orchestra.run.smithi052.stdout: "cap+": 0, 2024-06-10T03:04:08.718 INFO:teuthology.orchestra.run.smithi052.stdout: "cap-": 0, 2024-06-10T03:04:08.718 INFO:teuthology.orchestra.run.smithi052.stdout: "rss": 55632, 2024-06-10T03:04:08.718 INFO:teuthology.orchestra.run.smithi052.stdout: "heap": 215420 2024-06-10T03:04:08.718 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:08.718 INFO:teuthology.orchestra.run.smithi052.stdout: "mds_server": { 2024-06-10T03:04:08.718 INFO:teuthology.orchestra.run.smithi052.stdout: "dispatch_client_request": 0, 2024-06-10T03:04:08.718 INFO:teuthology.orchestra.run.smithi052.stdout: "dispatch_server_request": 0, 2024-06-10T03:04:08.718 INFO:teuthology.orchestra.run.smithi052.stdout: "handle_client_request": 0, 2024-06-10T03:04:08.718 INFO:teuthology.orchestra.run.smithi052.stdout: "handle_client_session": 0, 2024-06-10T03:04:08.718 INFO:teuthology.orchestra.run.smithi052.stdout: "handle_peer_request": 0, 2024-06-10T03:04:08.718 INFO:teuthology.orchestra.run.smithi052.stdout: "req_create_latency": { 2024-06-10T03:04:08.718 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-10T03:04:08.718 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-10T03:04:08.719 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-10T03:04:08.719 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:08.719 INFO:teuthology.orchestra.run.smithi052.stdout: "req_getattr_latency": { 2024-06-10T03:04:08.719 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-10T03:04:08.719 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-10T03:04:08.719 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-10T03:04:08.719 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:08.719 INFO:teuthology.orchestra.run.smithi052.stdout: "req_getfilelock_latency": { 2024-06-10T03:04:08.719 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-10T03:04:08.719 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-10T03:04:08.719 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-10T03:04:08.719 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:08.719 INFO:teuthology.orchestra.run.smithi052.stdout: "req_link_latency": { 2024-06-10T03:04:08.719 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-10T03:04:08.719 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-10T03:04:08.720 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-10T03:04:08.720 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:08.720 INFO:teuthology.orchestra.run.smithi052.stdout: "req_lookup_latency": { 2024-06-10T03:04:08.720 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-10T03:04:08.720 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-10T03:04:08.720 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-10T03:04:08.720 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:08.720 INFO:teuthology.orchestra.run.smithi052.stdout: "req_lookuphash_latency": { 2024-06-10T03:04:08.720 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-10T03:04:08.720 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-10T03:04:08.720 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-10T03:04:08.720 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:08.720 INFO:teuthology.orchestra.run.smithi052.stdout: "req_lookupino_latency": { 2024-06-10T03:04:08.720 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-10T03:04:08.720 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-10T03:04:08.721 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-10T03:04:08.721 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:08.721 INFO:teuthology.orchestra.run.smithi052.stdout: "req_lookupname_latency": { 2024-06-10T03:04:08.721 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-10T03:04:08.721 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-10T03:04:08.721 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-10T03:04:08.721 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:08.721 INFO:teuthology.orchestra.run.smithi052.stdout: "req_lookupparent_latency": { 2024-06-10T03:04:08.721 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-10T03:04:08.721 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-10T03:04:08.721 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-10T03:04:08.721 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:08.721 INFO:teuthology.orchestra.run.smithi052.stdout: "req_lookupsnap_latency": { 2024-06-10T03:04:08.721 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-10T03:04:08.721 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-10T03:04:08.721 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-10T03:04:08.722 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:08.722 INFO:teuthology.orchestra.run.smithi052.stdout: "req_lssnap_latency": { 2024-06-10T03:04:08.722 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-10T03:04:08.722 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-10T03:04:08.722 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-10T03:04:08.722 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:08.722 INFO:teuthology.orchestra.run.smithi052.stdout: "req_mkdir_latency": { 2024-06-10T03:04:08.723 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:04:08 smithi188 ceph-mon[36599]: pgmap v732: 129 pgs: 129 active+clean; 228 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-10T03:04:08.723 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-10T03:04:08.723 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-10T03:04:08.723 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-10T03:04:08.723 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:08.723 INFO:teuthology.orchestra.run.smithi052.stdout: "req_mknod_latency": { 2024-06-10T03:04:08.723 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-10T03:04:08.723 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-10T03:04:08.723 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-10T03:04:08.723 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:08.723 INFO:teuthology.orchestra.run.smithi052.stdout: "req_mksnap_latency": { 2024-06-10T03:04:08.723 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-10T03:04:08.724 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-10T03:04:08.724 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-10T03:04:08.724 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:08.724 INFO:teuthology.orchestra.run.smithi052.stdout: "req_open_latency": { 2024-06-10T03:04:08.724 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-10T03:04:08.724 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-10T03:04:08.724 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-10T03:04:08.724 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:08.724 INFO:teuthology.orchestra.run.smithi052.stdout: "req_readdir_latency": { 2024-06-10T03:04:08.724 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-10T03:04:08.724 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-10T03:04:08.724 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-10T03:04:08.724 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:08.724 INFO:teuthology.orchestra.run.smithi052.stdout: "req_rename_latency": { 2024-06-10T03:04:08.725 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-10T03:04:08.725 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-10T03:04:08.725 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-10T03:04:08.725 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:08.725 INFO:teuthology.orchestra.run.smithi052.stdout: "req_renamesnap_latency": { 2024-06-10T03:04:08.725 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-10T03:04:08.725 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-10T03:04:08.725 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-10T03:04:08.725 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:08.725 INFO:teuthology.orchestra.run.smithi052.stdout: "req_snapdiff_latency": { 2024-06-10T03:04:08.725 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-10T03:04:08.725 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-10T03:04:08.725 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-10T03:04:08.725 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:08.726 INFO:teuthology.orchestra.run.smithi052.stdout: "req_rmdir_latency": { 2024-06-10T03:04:08.726 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-10T03:04:08.726 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-10T03:04:08.726 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-10T03:04:08.726 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:08.726 INFO:teuthology.orchestra.run.smithi052.stdout: "req_rmsnap_latency": { 2024-06-10T03:04:08.726 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-10T03:04:08.726 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-10T03:04:08.726 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-10T03:04:08.726 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:08.726 INFO:teuthology.orchestra.run.smithi052.stdout: "req_rmxattr_latency": { 2024-06-10T03:04:08.726 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-10T03:04:08.726 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-10T03:04:08.727 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-10T03:04:08.727 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:08.727 INFO:teuthology.orchestra.run.smithi052.stdout: "req_setattr_latency": { 2024-06-10T03:04:08.727 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-10T03:04:08.727 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-10T03:04:08.727 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-10T03:04:08.727 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:08.727 INFO:teuthology.orchestra.run.smithi052.stdout: "req_setdirlayout_latency": { 2024-06-10T03:04:08.727 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-10T03:04:08.727 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-10T03:04:08.727 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-10T03:04:08.727 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:08.727 INFO:teuthology.orchestra.run.smithi052.stdout: "req_setfilelock_latency": { 2024-06-10T03:04:08.727 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-10T03:04:08.728 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-10T03:04:08.728 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-10T03:04:08.728 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:08.728 INFO:teuthology.orchestra.run.smithi052.stdout: "req_setlayout_latency": { 2024-06-10T03:04:08.728 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-10T03:04:08.728 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-10T03:04:08.728 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-10T03:04:08.728 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:08.728 INFO:teuthology.orchestra.run.smithi052.stdout: "req_setxattr_latency": { 2024-06-10T03:04:08.728 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-10T03:04:08.728 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-10T03:04:08.728 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-10T03:04:08.728 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:08.728 INFO:teuthology.orchestra.run.smithi052.stdout: "req_symlink_latency": { 2024-06-10T03:04:08.728 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-10T03:04:08.729 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-10T03:04:08.729 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-10T03:04:08.729 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:08.729 INFO:teuthology.orchestra.run.smithi052.stdout: "req_unlink_latency": { 2024-06-10T03:04:08.729 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-10T03:04:08.729 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-10T03:04:08.729 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-10T03:04:08.729 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:08.729 INFO:teuthology.orchestra.run.smithi052.stdout: "cap_revoke_eviction": 0, 2024-06-10T03:04:08.729 INFO:teuthology.orchestra.run.smithi052.stdout: "cap_acquisition_throttle": 0, 2024-06-10T03:04:08.729 INFO:teuthology.orchestra.run.smithi052.stdout: "req_getvxattr_latency": { 2024-06-10T03:04:08.729 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-10T03:04:08.729 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-10T03:04:08.730 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-10T03:04:08.730 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:08.730 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:08.730 INFO:teuthology.orchestra.run.smithi052.stdout: "mds_sessions": { 2024-06-10T03:04:08.730 INFO:teuthology.orchestra.run.smithi052.stdout: "session_count": 1, 2024-06-10T03:04:08.730 INFO:teuthology.orchestra.run.smithi052.stdout: "session_add": 3, 2024-06-10T03:04:08.730 INFO:teuthology.orchestra.run.smithi052.stdout: "session_remove": 2, 2024-06-10T03:04:08.730 INFO:teuthology.orchestra.run.smithi052.stdout: "sessions_open": 1, 2024-06-10T03:04:08.730 INFO:teuthology.orchestra.run.smithi052.stdout: "sessions_stale": 0, 2024-06-10T03:04:08.730 INFO:teuthology.orchestra.run.smithi052.stdout: "total_load": 0, 2024-06-10T03:04:08.730 INFO:teuthology.orchestra.run.smithi052.stdout: "average_load": 0, 2024-06-10T03:04:08.730 INFO:teuthology.orchestra.run.smithi052.stdout: "avg_session_uptime": 2116, 2024-06-10T03:04:08.730 INFO:teuthology.orchestra.run.smithi052.stdout: "mdthresh_evicted": 0 2024-06-10T03:04:08.731 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:08.731 INFO:teuthology.orchestra.run.smithi052.stdout: "mempool": { 2024-06-10T03:04:08.731 INFO:teuthology.orchestra.run.smithi052.stdout: "bloom_filter_bytes": 0, 2024-06-10T03:04:08.731 INFO:teuthology.orchestra.run.smithi052.stdout: "bloom_filter_items": 0, 2024-06-10T03:04:08.731 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_alloc_bytes": 0, 2024-06-10T03:04:08.731 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_alloc_items": 0, 2024-06-10T03:04:08.731 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_data_bytes": 0, 2024-06-10T03:04:08.731 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_data_items": 0, 2024-06-10T03:04:08.731 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-10T03:04:08.731 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_onode_items": 0, 2024-06-10T03:04:08.731 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-10T03:04:08.731 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_meta_items": 0, 2024-06-10T03:04:08.731 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_other_bytes": 0, 2024-06-10T03:04:08.732 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_other_items": 0, 2024-06-10T03:04:08.732 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-10T03:04:08.732 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_buffer_items": 0, 2024-06-10T03:04:08.732 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_extent_bytes": 0, 2024-06-10T03:04:08.732 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_extent_items": 0, 2024-06-10T03:04:08.732 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_blob_bytes": 0, 2024-06-10T03:04:08.732 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_blob_items": 0, 2024-06-10T03:04:08.732 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-10T03:04:08.732 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_shared_blob_items": 0, 2024-06-10T03:04:08.732 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-10T03:04:08.732 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_inline_bl_items": 0, 2024-06-10T03:04:08.732 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_fsck_bytes": 0, 2024-06-10T03:04:08.732 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_fsck_items": 0, 2024-06-10T03:04:08.732 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_txc_bytes": 0, 2024-06-10T03:04:08.733 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_txc_items": 0, 2024-06-10T03:04:08.733 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-10T03:04:08.733 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_writing_deferred_items": 0, 2024-06-10T03:04:08.733 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_writing_bytes": 0, 2024-06-10T03:04:08.733 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_writing_items": 0, 2024-06-10T03:04:08.733 INFO:teuthology.orchestra.run.smithi052.stdout: "bluefs_bytes": 0, 2024-06-10T03:04:08.733 INFO:teuthology.orchestra.run.smithi052.stdout: "bluefs_items": 0, 2024-06-10T03:04:08.733 INFO:teuthology.orchestra.run.smithi052.stdout: "bluefs_file_reader_bytes": 0, 2024-06-10T03:04:08.733 INFO:teuthology.orchestra.run.smithi052.stdout: "bluefs_file_reader_items": 0, 2024-06-10T03:04:08.733 INFO:teuthology.orchestra.run.smithi052.stdout: "bluefs_file_writer_bytes": 0, 2024-06-10T03:04:08.733 INFO:teuthology.orchestra.run.smithi052.stdout: "bluefs_file_writer_items": 0, 2024-06-10T03:04:08.733 INFO:teuthology.orchestra.run.smithi052.stdout: "buffer_anon_bytes": 197463, 2024-06-10T03:04:08.733 INFO:teuthology.orchestra.run.smithi052.stdout: "buffer_anon_items": 54, 2024-06-10T03:04:08.734 INFO:teuthology.orchestra.run.smithi052.stdout: "buffer_meta_bytes": 0, 2024-06-10T03:04:08.734 INFO:teuthology.orchestra.run.smithi052.stdout: "buffer_meta_items": 0, 2024-06-10T03:04:08.734 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_bytes": 0, 2024-06-10T03:04:08.734 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_items": 0, 2024-06-10T03:04:08.734 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_mapbl_bytes": 0, 2024-06-10T03:04:08.734 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_mapbl_items": 0, 2024-06-10T03:04:08.734 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_pglog_bytes": 0, 2024-06-10T03:04:08.734 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_pglog_items": 0, 2024-06-10T03:04:08.734 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_bytes": 8984, 2024-06-10T03:04:08.734 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_items": 172, 2024-06-10T03:04:08.734 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_mapping_bytes": 0, 2024-06-10T03:04:08.734 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_mapping_items": 0, 2024-06-10T03:04:08.734 INFO:teuthology.orchestra.run.smithi052.stdout: "pgmap_bytes": 0, 2024-06-10T03:04:08.735 INFO:teuthology.orchestra.run.smithi052.stdout: "pgmap_items": 0, 2024-06-10T03:04:08.735 INFO:teuthology.orchestra.run.smithi052.stdout: "mds_co_bytes": 966424, 2024-06-10T03:04:08.735 INFO:teuthology.orchestra.run.smithi052.stdout: "mds_co_items": 5399, 2024-06-10T03:04:08.735 INFO:teuthology.orchestra.run.smithi052.stdout: "unittest_1_bytes": 0, 2024-06-10T03:04:08.735 INFO:teuthology.orchestra.run.smithi052.stdout: "unittest_1_items": 0, 2024-06-10T03:04:08.735 INFO:teuthology.orchestra.run.smithi052.stdout: "unittest_2_bytes": 0, 2024-06-10T03:04:08.735 INFO:teuthology.orchestra.run.smithi052.stdout: "unittest_2_items": 0 2024-06-10T03:04:08.735 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:08.735 INFO:teuthology.orchestra.run.smithi052.stdout: "objecter": { 2024-06-10T03:04:08.735 INFO:teuthology.orchestra.run.smithi052.stdout: "op_active": 0, 2024-06-10T03:04:08.735 INFO:teuthology.orchestra.run.smithi052.stdout: "op_laggy": 0, 2024-06-10T03:04:08.735 INFO:teuthology.orchestra.run.smithi052.stdout: "op_send": 2320, 2024-06-10T03:04:08.735 INFO:teuthology.orchestra.run.smithi052.stdout: "op_send_bytes": 21, 2024-06-10T03:04:08.736 INFO:teuthology.orchestra.run.smithi052.stdout: "op_resend": 3, 2024-06-10T03:04:08.736 INFO:teuthology.orchestra.run.smithi052.stdout: "op_reply": 2320, 2024-06-10T03:04:08.736 INFO:teuthology.orchestra.run.smithi052.stdout: "op_latency": { 2024-06-10T03:04:08.736 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 2320, 2024-06-10T03:04:08.736 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 12.411834125, 2024-06-10T03:04:08.736 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.005349928 2024-06-10T03:04:08.736 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:08.736 INFO:teuthology.orchestra.run.smithi052.stdout: "op_inflight": 0, 2024-06-10T03:04:08.736 INFO:teuthology.orchestra.run.smithi052.stdout: "oplen_avg": { 2024-06-10T03:04:08.736 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 2320, 2024-06-10T03:04:08.736 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 2321 2024-06-10T03:04:08.736 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:08.736 INFO:teuthology.orchestra.run.smithi052.stdout: "op": 2320, 2024-06-10T03:04:08.737 INFO:teuthology.orchestra.run.smithi052.stdout: "op_r": 2320, 2024-06-10T03:04:08.737 INFO:teuthology.orchestra.run.smithi052.stdout: "op_w": 0, 2024-06-10T03:04:08.737 INFO:teuthology.orchestra.run.smithi052.stdout: "op_rmw": 0, 2024-06-10T03:04:08.737 INFO:teuthology.orchestra.run.smithi052.stdout: "op_pg": 0, 2024-06-10T03:04:08.737 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_stat": 1438, 2024-06-10T03:04:08.737 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_create": 0, 2024-06-10T03:04:08.737 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_read": 880, 2024-06-10T03:04:08.737 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_write": 0, 2024-06-10T03:04:08.737 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_writefull": 0, 2024-06-10T03:04:08.737 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_writesame": 0, 2024-06-10T03:04:08.737 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_append": 0, 2024-06-10T03:04:08.737 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_zero": 0, 2024-06-10T03:04:08.737 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_truncate": 0, 2024-06-10T03:04:08.738 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_delete": 0, 2024-06-10T03:04:08.738 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_mapext": 0, 2024-06-10T03:04:08.738 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_sparse_read": 0, 2024-06-10T03:04:08.738 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_clonerange": 0, 2024-06-10T03:04:08.738 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_getxattr": 1, 2024-06-10T03:04:08.738 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_setxattr": 0, 2024-06-10T03:04:08.738 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_cmpxattr": 0, 2024-06-10T03:04:08.738 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_rmxattr": 0, 2024-06-10T03:04:08.738 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_resetxattrs": 0, 2024-06-10T03:04:08.738 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_call": 0, 2024-06-10T03:04:08.738 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_watch": 0, 2024-06-10T03:04:08.738 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_notify": 0, 2024-06-10T03:04:08.738 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_src_cmpxattr": 0, 2024-06-10T03:04:08.738 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_pgls": 0, 2024-06-10T03:04:08.739 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_pgls_filter": 0, 2024-06-10T03:04:08.739 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_other": 0, 2024-06-10T03:04:08.739 INFO:teuthology.orchestra.run.smithi052.stdout: "linger_active": 0, 2024-06-10T03:04:08.739 INFO:teuthology.orchestra.run.smithi052.stdout: "linger_send": 0, 2024-06-10T03:04:08.739 INFO:teuthology.orchestra.run.smithi052.stdout: "linger_resend": 0, 2024-06-10T03:04:08.739 INFO:teuthology.orchestra.run.smithi052.stdout: "linger_ping": 0, 2024-06-10T03:04:08.739 INFO:teuthology.orchestra.run.smithi052.stdout: "poolop_active": 0, 2024-06-10T03:04:08.739 INFO:teuthology.orchestra.run.smithi052.stdout: "poolop_send": 0, 2024-06-10T03:04:08.739 INFO:teuthology.orchestra.run.smithi052.stdout: "poolop_resend": 0, 2024-06-10T03:04:08.739 INFO:teuthology.orchestra.run.smithi052.stdout: "poolstat_active": 0, 2024-06-10T03:04:08.739 INFO:teuthology.orchestra.run.smithi052.stdout: "poolstat_send": 0, 2024-06-10T03:04:08.739 INFO:teuthology.orchestra.run.smithi052.stdout: "poolstat_resend": 0, 2024-06-10T03:04:08.739 INFO:teuthology.orchestra.run.smithi052.stdout: "statfs_active": 0, 2024-06-10T03:04:08.739 INFO:teuthology.orchestra.run.smithi052.stdout: "statfs_send": 0, 2024-06-10T03:04:08.740 INFO:teuthology.orchestra.run.smithi052.stdout: "statfs_resend": 0, 2024-06-10T03:04:08.740 INFO:teuthology.orchestra.run.smithi052.stdout: "command_active": 0, 2024-06-10T03:04:08.740 INFO:teuthology.orchestra.run.smithi052.stdout: "command_send": 0, 2024-06-10T03:04:08.740 INFO:teuthology.orchestra.run.smithi052.stdout: "command_resend": 0, 2024-06-10T03:04:08.740 INFO:teuthology.orchestra.run.smithi052.stdout: "map_epoch": 100, 2024-06-10T03:04:08.740 INFO:teuthology.orchestra.run.smithi052.stdout: "map_full": 0, 2024-06-10T03:04:08.740 INFO:teuthology.orchestra.run.smithi052.stdout: "map_inc": 2, 2024-06-10T03:04:08.740 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_sessions": 6, 2024-06-10T03:04:08.740 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_session_open": 6, 2024-06-10T03:04:08.740 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_session_close": 0, 2024-06-10T03:04:08.740 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_laggy": 0, 2024-06-10T03:04:08.740 INFO:teuthology.orchestra.run.smithi052.stdout: "omap_wr": 0, 2024-06-10T03:04:08.740 INFO:teuthology.orchestra.run.smithi052.stdout: "omap_rd": 2, 2024-06-10T03:04:08.740 INFO:teuthology.orchestra.run.smithi052.stdout: "omap_del": 0 2024-06-10T03:04:08.741 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:08.741 INFO:teuthology.orchestra.run.smithi052.stdout: "oft": { 2024-06-10T03:04:08.741 INFO:teuthology.orchestra.run.smithi052.stdout: "omap_total_objs": 0, 2024-06-10T03:04:08.741 INFO:teuthology.orchestra.run.smithi052.stdout: "omap_total_kv_pairs": 0, 2024-06-10T03:04:08.741 INFO:teuthology.orchestra.run.smithi052.stdout: "omap_total_updates": 0, 2024-06-10T03:04:08.741 INFO:teuthology.orchestra.run.smithi052.stdout: "omap_total_removes": 0 2024-06-10T03:04:08.741 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:08.741 INFO:teuthology.orchestra.run.smithi052.stdout: "osd-slow-ops": { 2024-06-10T03:04:08.741 INFO:teuthology.orchestra.run.smithi052.stdout: "slow_ops_count": 0 2024-06-10T03:04:08.741 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:08.741 INFO:teuthology.orchestra.run.smithi052.stdout: "purge_queue": { 2024-06-10T03:04:08.742 INFO:teuthology.orchestra.run.smithi052.stdout: "pq_executing_ops": 0, 2024-06-10T03:04:08.742 INFO:teuthology.orchestra.run.smithi052.stdout: "pq_executing_ops_high_water": 0, 2024-06-10T03:04:08.742 INFO:teuthology.orchestra.run.smithi052.stdout: "pq_executing": 0, 2024-06-10T03:04:08.742 INFO:teuthology.orchestra.run.smithi052.stdout: "pq_executing_high_water": 0, 2024-06-10T03:04:08.742 INFO:teuthology.orchestra.run.smithi052.stdout: "pq_executed_ops": 0, 2024-06-10T03:04:08.742 INFO:teuthology.orchestra.run.smithi052.stdout: "pq_executed": 0, 2024-06-10T03:04:08.742 INFO:teuthology.orchestra.run.smithi052.stdout: "pq_item_in_journal": 0 2024-06-10T03:04:08.742 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:08.742 INFO:teuthology.orchestra.run.smithi052.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-10T03:04:08.742 INFO:teuthology.orchestra.run.smithi052.stdout: "val": 0, 2024-06-10T03:04:08.742 INFO:teuthology.orchestra.run.smithi052.stdout: "max": 104857600, 2024-06-10T03:04:08.742 INFO:teuthology.orchestra.run.smithi052.stdout: "get_started": 0, 2024-06-10T03:04:08.743 INFO:teuthology.orchestra.run.smithi052.stdout: "get": 2584, 2024-06-10T03:04:08.743 INFO:teuthology.orchestra.run.smithi052.stdout: "get_sum": 11034364, 2024-06-10T03:04:08.743 INFO:teuthology.orchestra.run.smithi052.stdout: "get_or_fail_fail": 0, 2024-06-10T03:04:08.743 INFO:teuthology.orchestra.run.smithi052.stdout: "get_or_fail_success": 2584, 2024-06-10T03:04:08.743 INFO:teuthology.orchestra.run.smithi052.stdout: "take": 0, 2024-06-10T03:04:08.743 INFO:teuthology.orchestra.run.smithi052.stdout: "take_sum": 0, 2024-06-10T03:04:08.743 INFO:teuthology.orchestra.run.smithi052.stdout: "put": 2584, 2024-06-10T03:04:08.743 INFO:teuthology.orchestra.run.smithi052.stdout: "put_sum": 11034364, 2024-06-10T03:04:08.743 INFO:teuthology.orchestra.run.smithi052.stdout: "wait": { 2024-06-10T03:04:08.743 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-10T03:04:08.743 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-10T03:04:08.743 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-10T03:04:08.743 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:08.743 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:08.744 INFO:teuthology.orchestra.run.smithi052.stdout: "throttle-objecter_bytes": { 2024-06-10T03:04:08.744 INFO:teuthology.orchestra.run.smithi052.stdout: "val": 0, 2024-06-10T03:04:08.744 INFO:teuthology.orchestra.run.smithi052.stdout: "max": 104857600, 2024-06-10T03:04:08.744 INFO:teuthology.orchestra.run.smithi052.stdout: "get_started": 0, 2024-06-10T03:04:08.744 INFO:teuthology.orchestra.run.smithi052.stdout: "get": 0, 2024-06-10T03:04:08.744 INFO:teuthology.orchestra.run.smithi052.stdout: "get_sum": 0, 2024-06-10T03:04:08.744 INFO:teuthology.orchestra.run.smithi052.stdout: "get_or_fail_fail": 0, 2024-06-10T03:04:08.744 INFO:teuthology.orchestra.run.smithi052.stdout: "get_or_fail_success": 0, 2024-06-10T03:04:08.744 INFO:teuthology.orchestra.run.smithi052.stdout: "take": 2320, 2024-06-10T03:04:08.744 INFO:teuthology.orchestra.run.smithi052.stdout: "take_sum": 9729205, 2024-06-10T03:04:08.744 INFO:teuthology.orchestra.run.smithi052.stdout: "put": 2320, 2024-06-10T03:04:08.744 INFO:teuthology.orchestra.run.smithi052.stdout: "put_sum": 9729205, 2024-06-10T03:04:08.744 INFO:teuthology.orchestra.run.smithi052.stdout: "wait": { 2024-06-10T03:04:08.744 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-10T03:04:08.745 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-10T03:04:08.745 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-10T03:04:08.745 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:08.745 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:08.745 INFO:teuthology.orchestra.run.smithi052.stdout: "throttle-objecter_ops": { 2024-06-10T03:04:08.745 INFO:teuthology.orchestra.run.smithi052.stdout: "val": 0, 2024-06-10T03:04:08.745 INFO:teuthology.orchestra.run.smithi052.stdout: "max": 1024, 2024-06-10T03:04:08.745 INFO:teuthology.orchestra.run.smithi052.stdout: "get_started": 0, 2024-06-10T03:04:08.745 INFO:teuthology.orchestra.run.smithi052.stdout: "get": 0, 2024-06-10T03:04:08.745 INFO:teuthology.orchestra.run.smithi052.stdout: "get_sum": 0, 2024-06-10T03:04:08.745 INFO:teuthology.orchestra.run.smithi052.stdout: "get_or_fail_fail": 0, 2024-06-10T03:04:08.745 INFO:teuthology.orchestra.run.smithi052.stdout: "get_or_fail_success": 0, 2024-06-10T03:04:08.745 INFO:teuthology.orchestra.run.smithi052.stdout: "take": 2320, 2024-06-10T03:04:08.746 INFO:teuthology.orchestra.run.smithi052.stdout: "take_sum": 2320, 2024-06-10T03:04:08.746 INFO:teuthology.orchestra.run.smithi052.stdout: "put": 2320, 2024-06-10T03:04:08.746 INFO:teuthology.orchestra.run.smithi052.stdout: "put_sum": 2320, 2024-06-10T03:04:08.746 INFO:teuthology.orchestra.run.smithi052.stdout: "wait": { 2024-06-10T03:04:08.746 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-10T03:04:08.746 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-10T03:04:08.746 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-10T03:04:08.746 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:08.746 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:08.746 INFO:teuthology.orchestra.run.smithi052.stdout: "throttle-write_buf_throttle": { 2024-06-10T03:04:08.746 INFO:teuthology.orchestra.run.smithi052.stdout: "val": 0, 2024-06-10T03:04:08.746 INFO:teuthology.orchestra.run.smithi052.stdout: "max": 3758096384, 2024-06-10T03:04:08.746 INFO:teuthology.orchestra.run.smithi052.stdout: "get_started": 0, 2024-06-10T03:04:08.746 INFO:teuthology.orchestra.run.smithi052.stdout: "get": 0, 2024-06-10T03:04:08.747 INFO:teuthology.orchestra.run.smithi052.stdout: "get_sum": 0, 2024-06-10T03:04:08.747 INFO:teuthology.orchestra.run.smithi052.stdout: "get_or_fail_fail": 0, 2024-06-10T03:04:08.747 INFO:teuthology.orchestra.run.smithi052.stdout: "get_or_fail_success": 0, 2024-06-10T03:04:08.747 INFO:teuthology.orchestra.run.smithi052.stdout: "take": 0, 2024-06-10T03:04:08.747 INFO:teuthology.orchestra.run.smithi052.stdout: "take_sum": 0, 2024-06-10T03:04:08.747 INFO:teuthology.orchestra.run.smithi052.stdout: "put": 0, 2024-06-10T03:04:08.747 INFO:teuthology.orchestra.run.smithi052.stdout: "put_sum": 0, 2024-06-10T03:04:08.747 INFO:teuthology.orchestra.run.smithi052.stdout: "wait": { 2024-06-10T03:04:08.747 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-10T03:04:08.747 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-10T03:04:08.747 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-10T03:04:08.747 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:08.747 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:08.747 INFO:teuthology.orchestra.run.smithi052.stdout: "throttle-write_buf_throttle-0x55bfd9ab46e0": { 2024-06-10T03:04:08.748 INFO:teuthology.orchestra.run.smithi052.stdout: "val": 0, 2024-06-10T03:04:08.748 INFO:teuthology.orchestra.run.smithi052.stdout: "max": 3758096384, 2024-06-10T03:04:08.748 INFO:teuthology.orchestra.run.smithi052.stdout: "get_started": 0, 2024-06-10T03:04:08.748 INFO:teuthology.orchestra.run.smithi052.stdout: "get": 0, 2024-06-10T03:04:08.748 INFO:teuthology.orchestra.run.smithi052.stdout: "get_sum": 0, 2024-06-10T03:04:08.748 INFO:teuthology.orchestra.run.smithi052.stdout: "get_or_fail_fail": 0, 2024-06-10T03:04:08.748 INFO:teuthology.orchestra.run.smithi052.stdout: "get_or_fail_success": 0, 2024-06-10T03:04:08.748 INFO:teuthology.orchestra.run.smithi052.stdout: "take": 0, 2024-06-10T03:04:08.748 INFO:teuthology.orchestra.run.smithi052.stdout: "take_sum": 0, 2024-06-10T03:04:08.748 INFO:teuthology.orchestra.run.smithi052.stdout: "put": 0, 2024-06-10T03:04:08.748 INFO:teuthology.orchestra.run.smithi052.stdout: "put_sum": 0, 2024-06-10T03:04:08.748 INFO:teuthology.orchestra.run.smithi052.stdout: "wait": { 2024-06-10T03:04:08.748 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-10T03:04:08.748 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-10T03:04:08.749 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-10T03:04:08.749 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:08.749 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:08.749 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-10T03:04:09.578 INFO:tasks.check_counter:Daemon mds.g mds.exported=0 2024-06-10T03:04:09.578 INFO:tasks.check_counter:Daemon mds.g mds.imported=0 2024-06-10T03:04:09.579 INFO:tasks.check_counter:Daemon mds.g mds_cache.dir_update=0 2024-06-10T03:04:09.579 INFO:tasks.check_counter:Daemon mds.g mds_cache.dir_update_receipt=0 2024-06-10T03:04:09.579 INFO:tasks.check_counter:Daemon mds.g mds.dir_split=0 2024-06-10T03:04:09.579 DEBUG:tasks.check_counter:Getting stats from j 2024-06-10T03:04:09.579 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph daemon mds.j perf dump 2024-06-10T03:04:09.830 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T03:04:10.533 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:04:10 smithi052 ceph-mon[32362]: pgmap v733: 129 pgs: 129 active+clean; 228 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-10T03:04:10.533 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:04:10 smithi052 ceph-mon[32362]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-10T03:04:10.641 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:04:10 smithi120 ceph-mon[36634]: pgmap v733: 129 pgs: 129 active+clean; 228 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-10T03:04:10.642 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:04:10 smithi120 ceph-mon[36634]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-10T03:04:10.704 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:04:10 smithi188 ceph-mon[36599]: pgmap v733: 129 pgs: 129 active+clean; 228 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-10T03:04:10.704 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:04:10 smithi188 ceph-mon[36599]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-10T03:04:11.337 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-10T03:04:11.337 INFO:teuthology.orchestra.run.smithi052.stdout: "AsyncMessenger::Worker-0": { 2024-06-10T03:04:11.337 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_messages": 244, 2024-06-10T03:04:11.337 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_messages": 239, 2024-06-10T03:04:11.337 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_bytes": 760560, 2024-06-10T03:04:11.337 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_bytes": 147264, 2024-06-10T03:04:11.337 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_created_connections": 4, 2024-06-10T03:04:11.337 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_active_connections": 0, 2024-06-10T03:04:11.338 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_total_time": 0.039064678, 2024-06-10T03:04:11.338 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_send_time": 0.014605920, 2024-06-10T03:04:11.338 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_recv_time": 0.021577436, 2024-06-10T03:04:11.338 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_fast_dispatch_time": 0.004888183, 2024-06-10T03:04:11.338 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_messages_queue_lat": { 2024-06-10T03:04:11.338 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 238, 2024-06-10T03:04:11.338 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.010151457, 2024-06-10T03:04:11.338 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000042653 2024-06-10T03:04:11.338 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:11.338 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_handle_ack_lat": { 2024-06-10T03:04:11.338 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-10T03:04:11.338 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-10T03:04:11.339 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-10T03:04:11.339 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:11.339 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_encrypted_bytes": 760560, 2024-06-10T03:04:11.339 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_encrypted_bytes": 147264 2024-06-10T03:04:11.339 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:11.339 INFO:teuthology.orchestra.run.smithi052.stdout: "AsyncMessenger::Worker-1": { 2024-06-10T03:04:11.339 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_messages": 5, 2024-06-10T03:04:11.339 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_messages": 184, 2024-06-10T03:04:11.339 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_bytes": 3232, 2024-06-10T03:04:11.339 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_bytes": 110352, 2024-06-10T03:04:11.339 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_created_connections": 3, 2024-06-10T03:04:11.339 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_active_connections": 2, 2024-06-10T03:04:11.339 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_total_time": 0.013011999, 2024-06-10T03:04:11.339 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_send_time": 0.009627770, 2024-06-10T03:04:11.339 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_recv_time": 0.001584036, 2024-06-10T03:04:11.340 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-06-10T03:04:11.340 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_messages_queue_lat": { 2024-06-10T03:04:11.340 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 183, 2024-06-10T03:04:11.340 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.011164791, 2024-06-10T03:04:11.340 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000061009 2024-06-10T03:04:11.340 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:11.340 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_handle_ack_lat": { 2024-06-10T03:04:11.340 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-10T03:04:11.340 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-10T03:04:11.340 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-10T03:04:11.340 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:11.340 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_encrypted_bytes": 3232, 2024-06-10T03:04:11.340 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_encrypted_bytes": 110352 2024-06-10T03:04:11.340 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:11.340 INFO:teuthology.orchestra.run.smithi052.stdout: "AsyncMessenger::Worker-2": { 2024-06-10T03:04:11.341 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_messages": 0, 2024-06-10T03:04:11.341 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_messages": 2, 2024-06-10T03:04:11.341 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_bytes": 0, 2024-06-10T03:04:11.341 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_bytes": 0, 2024-06-10T03:04:11.341 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_created_connections": 2, 2024-06-10T03:04:11.341 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_active_connections": 1, 2024-06-10T03:04:11.341 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_total_time": 0.000277648, 2024-06-10T03:04:11.341 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_send_time": 0.000000000, 2024-06-10T03:04:11.341 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_recv_time": 0.000210206, 2024-06-10T03:04:11.341 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-06-10T03:04:11.341 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_messages_queue_lat": { 2024-06-10T03:04:11.341 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-10T03:04:11.341 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-10T03:04:11.341 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-10T03:04:11.342 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:11.342 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_handle_ack_lat": { 2024-06-10T03:04:11.342 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-10T03:04:11.342 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-10T03:04:11.342 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-10T03:04:11.342 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:11.342 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_encrypted_bytes": 0, 2024-06-10T03:04:11.342 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_encrypted_bytes": 0 2024-06-10T03:04:11.342 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:11.342 INFO:teuthology.orchestra.run.smithi052.stdout: "cct": { 2024-06-10T03:04:11.342 INFO:teuthology.orchestra.run.smithi052.stdout: "total_workers": 0, 2024-06-10T03:04:11.342 INFO:teuthology.orchestra.run.smithi052.stdout: "unhealthy_workers": 0 2024-06-10T03:04:11.342 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:11.343 INFO:teuthology.orchestra.run.smithi052.stdout: "mempool": { 2024-06-10T03:04:11.343 INFO:teuthology.orchestra.run.smithi052.stdout: "bloom_filter_bytes": 0, 2024-06-10T03:04:11.343 INFO:teuthology.orchestra.run.smithi052.stdout: "bloom_filter_items": 0, 2024-06-10T03:04:11.343 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_alloc_bytes": 0, 2024-06-10T03:04:11.343 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_alloc_items": 0, 2024-06-10T03:04:11.343 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_data_bytes": 0, 2024-06-10T03:04:11.343 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_data_items": 0, 2024-06-10T03:04:11.343 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-10T03:04:11.343 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_onode_items": 0, 2024-06-10T03:04:11.343 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-10T03:04:11.343 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_meta_items": 0, 2024-06-10T03:04:11.343 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_other_bytes": 0, 2024-06-10T03:04:11.343 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_other_items": 0, 2024-06-10T03:04:11.343 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-10T03:04:11.343 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_buffer_items": 0, 2024-06-10T03:04:11.344 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_extent_bytes": 0, 2024-06-10T03:04:11.344 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_extent_items": 0, 2024-06-10T03:04:11.344 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_blob_bytes": 0, 2024-06-10T03:04:11.344 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_blob_items": 0, 2024-06-10T03:04:11.344 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-10T03:04:11.344 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_shared_blob_items": 0, 2024-06-10T03:04:11.344 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-10T03:04:11.344 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_inline_bl_items": 0, 2024-06-10T03:04:11.344 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_fsck_bytes": 0, 2024-06-10T03:04:11.344 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_fsck_items": 0, 2024-06-10T03:04:11.344 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_txc_bytes": 0, 2024-06-10T03:04:11.344 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_txc_items": 0, 2024-06-10T03:04:11.344 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-10T03:04:11.344 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_writing_deferred_items": 0, 2024-06-10T03:04:11.345 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_writing_bytes": 0, 2024-06-10T03:04:11.345 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_writing_items": 0, 2024-06-10T03:04:11.345 INFO:teuthology.orchestra.run.smithi052.stdout: "bluefs_bytes": 0, 2024-06-10T03:04:11.345 INFO:teuthology.orchestra.run.smithi052.stdout: "bluefs_items": 0, 2024-06-10T03:04:11.345 INFO:teuthology.orchestra.run.smithi052.stdout: "bluefs_file_reader_bytes": 0, 2024-06-10T03:04:11.345 INFO:teuthology.orchestra.run.smithi052.stdout: "bluefs_file_reader_items": 0, 2024-06-10T03:04:11.345 INFO:teuthology.orchestra.run.smithi052.stdout: "bluefs_file_writer_bytes": 0, 2024-06-10T03:04:11.345 INFO:teuthology.orchestra.run.smithi052.stdout: "bluefs_file_writer_items": 0, 2024-06-10T03:04:11.345 INFO:teuthology.orchestra.run.smithi052.stdout: "buffer_anon_bytes": 149007, 2024-06-10T03:04:11.345 INFO:teuthology.orchestra.run.smithi052.stdout: "buffer_anon_items": 30, 2024-06-10T03:04:11.345 INFO:teuthology.orchestra.run.smithi052.stdout: "buffer_meta_bytes": 0, 2024-06-10T03:04:11.345 INFO:teuthology.orchestra.run.smithi052.stdout: "buffer_meta_items": 0, 2024-06-10T03:04:11.345 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_bytes": 0, 2024-06-10T03:04:11.345 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_items": 0, 2024-06-10T03:04:11.346 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_mapbl_bytes": 0, 2024-06-10T03:04:11.346 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_mapbl_items": 0, 2024-06-10T03:04:11.346 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_pglog_bytes": 0, 2024-06-10T03:04:11.346 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_pglog_items": 0, 2024-06-10T03:04:11.346 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_bytes": 0, 2024-06-10T03:04:11.346 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_items": 0, 2024-06-10T03:04:11.346 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_mapping_bytes": 0, 2024-06-10T03:04:11.346 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_mapping_items": 0, 2024-06-10T03:04:11.346 INFO:teuthology.orchestra.run.smithi052.stdout: "pgmap_bytes": 0, 2024-06-10T03:04:11.346 INFO:teuthology.orchestra.run.smithi052.stdout: "pgmap_items": 0, 2024-06-10T03:04:11.346 INFO:teuthology.orchestra.run.smithi052.stdout: "mds_co_bytes": 928, 2024-06-10T03:04:11.346 INFO:teuthology.orchestra.run.smithi052.stdout: "mds_co_items": 2, 2024-06-10T03:04:11.346 INFO:teuthology.orchestra.run.smithi052.stdout: "unittest_1_bytes": 0, 2024-06-10T03:04:11.347 INFO:teuthology.orchestra.run.smithi052.stdout: "unittest_1_items": 0, 2024-06-10T03:04:11.347 INFO:teuthology.orchestra.run.smithi052.stdout: "unittest_2_bytes": 0, 2024-06-10T03:04:11.347 INFO:teuthology.orchestra.run.smithi052.stdout: "unittest_2_items": 0 2024-06-10T03:04:11.347 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-10T03:04:11.347 INFO:teuthology.orchestra.run.smithi052.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-10T03:04:11.347 INFO:teuthology.orchestra.run.smithi052.stdout: "val": 0, 2024-06-10T03:04:11.347 INFO:teuthology.orchestra.run.smithi052.stdout: "max": 104857600, 2024-06-10T03:04:11.347 INFO:teuthology.orchestra.run.smithi052.stdout: "get_started": 0, 2024-06-10T03:04:11.347 INFO:teuthology.orchestra.run.smithi052.stdout: "get": 245, 2024-06-10T03:04:11.347 INFO:teuthology.orchestra.run.smithi052.stdout: "get_sum": 726085, 2024-06-10T03:04:11.347 INFO:teuthology.orchestra.run.smithi052.stdout: "get_or_fail_fail": 0, 2024-06-10T03:04:11.347 INFO:teuthology.orchestra.run.smithi052.stdout: "get_or_fail_success": 245, 2024-06-10T03:04:11.347 INFO:teuthology.orchestra.run.smithi052.stdout: "take": 0, 2024-06-10T03:04:11.347 INFO:teuthology.orchestra.run.smithi052.stdout: "take_sum": 0, 2024-06-10T03:04:11.348 INFO:teuthology.orchestra.run.smithi052.stdout: "put": 245, 2024-06-10T03:04:11.348 INFO:teuthology.orchestra.run.smithi052.stdout: "put_sum": 726085, 2024-06-10T03:04:11.348 INFO:teuthology.orchestra.run.smithi052.stdout: "wait": { 2024-06-10T03:04:11.348 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-10T03:04:11.348 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-10T03:04:11.348 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-10T03:04:11.348 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:11.348 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-10T03:04:11.348 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-10T03:04:12.120 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-06-10T03:04:12.120 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-06-10T03:04:12.120 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.j 2024-06-10T03:04:12.120 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.j 2024-06-10T03:04:12.120 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.j 2024-06-10T03:04:12.120 DEBUG:tasks.check_counter:Getting stats from b 2024-06-10T03:04:12.120 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph daemon mds.b perf dump 2024-06-10T03:04:12.378 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.b/config 2024-06-10T03:04:12.500 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:04:12 smithi120 ceph-mon[36634]: pgmap v734: 129 pgs: 129 active+clean; 228 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-06-10T03:04:12.500 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:04:12 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T03:04:12.501 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:04:12 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T03:04:12.501 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:04:12 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T03:04:12.501 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:04:12 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T03:04:12.501 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:04:12 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T03:04:12.501 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:04:12 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T03:04:12.501 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:04:12 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T03:04:12.704 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:04:12 smithi188 ceph-mon[36599]: pgmap v734: 129 pgs: 129 active+clean; 228 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-06-10T03:04:12.704 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:04:12 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T03:04:12.704 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:04:12 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T03:04:12.704 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:04:12 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T03:04:12.704 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:04:12 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T03:04:12.704 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:04:12 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T03:04:12.704 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:04:12 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T03:04:12.704 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:04:12 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T03:04:12.727 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:04:12 smithi052 ceph-mon[32362]: pgmap v734: 129 pgs: 129 active+clean; 228 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-06-10T03:04:12.727 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:04:12 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T03:04:12.727 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:04:12 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T03:04:12.727 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:04:12 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T03:04:12.727 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:04:12 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T03:04:12.727 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:04:12 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T03:04:12.727 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:04:12 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T03:04:12.727 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:04:12 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T03:04:13.638 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:04:13 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T03:04:13.639 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:04:13 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T03:04:13.639 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:04:13 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T03:04:13.704 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:04:13 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T03:04:13.704 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:04:13 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T03:04:13.704 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:04:13 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T03:04:13.727 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:04:13 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T03:04:13.727 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:04:13 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T03:04:13.727 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:04:13 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' 2024-06-10T03:04:14.215 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-06-10T03:04:14.215 INFO:teuthology.orchestra.run.smithi120.stdout: "AsyncMessenger::Worker-0": { 2024-06-10T03:04:14.215 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_messages": 10288, 2024-06-10T03:04:14.215 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_messages": 10452, 2024-06-10T03:04:14.215 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_bytes": 3521219, 2024-06-10T03:04:14.216 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_bytes": 2543876, 2024-06-10T03:04:14.216 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_created_connections": 17, 2024-06-10T03:04:14.216 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_active_connections": 5, 2024-06-10T03:04:14.216 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_total_time": 0.703404754, 2024-06-10T03:04:14.216 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_send_time": 0.264116997, 2024-06-10T03:04:14.216 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_recv_time": 0.626036067, 2024-06-10T03:04:14.216 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_fast_dispatch_time": 0.003237516, 2024-06-10T03:04:14.216 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_messages_queue_lat": { 2024-06-10T03:04:14.216 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 10450, 2024-06-10T03:04:14.216 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.187732556, 2024-06-10T03:04:14.216 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000017964 2024-06-10T03:04:14.216 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:14.216 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_handle_ack_lat": { 2024-06-10T03:04:14.216 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 20096, 2024-06-10T03:04:14.217 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.002992749, 2024-06-10T03:04:14.217 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000148 2024-06-10T03:04:14.217 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:14.217 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_encrypted_bytes": 3314832, 2024-06-10T03:04:14.217 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_encrypted_bytes": 1711696 2024-06-10T03:04:14.217 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:14.217 INFO:teuthology.orchestra.run.smithi120.stdout: "AsyncMessenger::Worker-1": { 2024-06-10T03:04:14.217 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_messages": 8836, 2024-06-10T03:04:14.217 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_messages": 4977, 2024-06-10T03:04:14.217 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_bytes": 5208312, 2024-06-10T03:04:14.217 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_bytes": 1173487, 2024-06-10T03:04:14.217 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_created_connections": 20, 2024-06-10T03:04:14.217 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_active_connections": 7, 2024-06-10T03:04:14.218 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_total_time": 0.469671781, 2024-06-10T03:04:14.218 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_send_time": 0.170030158, 2024-06-10T03:04:14.218 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_recv_time": 0.411445011, 2024-06-10T03:04:14.218 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_fast_dispatch_time": 0.008853629, 2024-06-10T03:04:14.218 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_messages_queue_lat": { 2024-06-10T03:04:14.218 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 4977, 2024-06-10T03:04:14.218 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.126047476, 2024-06-10T03:04:14.218 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000025325 2024-06-10T03:04:14.218 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:14.218 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_handle_ack_lat": { 2024-06-10T03:04:14.218 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 12885, 2024-06-10T03:04:14.218 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.002017564, 2024-06-10T03:04:14.218 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000156 2024-06-10T03:04:14.219 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:14.219 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_encrypted_bytes": 926816, 2024-06-10T03:04:14.219 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_encrypted_bytes": 150704 2024-06-10T03:04:14.219 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:14.219 INFO:teuthology.orchestra.run.smithi120.stdout: "AsyncMessenger::Worker-2": { 2024-06-10T03:04:14.219 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_messages": 1835, 2024-06-10T03:04:14.219 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_messages": 1499, 2024-06-10T03:04:14.219 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_bytes": 646487, 2024-06-10T03:04:14.219 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_bytes": 891373, 2024-06-10T03:04:14.219 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_created_connections": 12, 2024-06-10T03:04:14.219 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_active_connections": 6, 2024-06-10T03:04:14.219 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_total_time": 0.131338104, 2024-06-10T03:04:14.219 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_send_time": 0.053593977, 2024-06-10T03:04:14.219 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_recv_time": 0.085162905, 2024-06-10T03:04:14.220 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_fast_dispatch_time": 0.006255453, 2024-06-10T03:04:14.220 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_messages_queue_lat": { 2024-06-10T03:04:14.220 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 1497, 2024-06-10T03:04:14.220 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.062418181, 2024-06-10T03:04:14.220 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000041695 2024-06-10T03:04:14.220 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:14.220 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_handle_ack_lat": { 2024-06-10T03:04:14.220 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 2937, 2024-06-10T03:04:14.220 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000596008, 2024-06-10T03:04:14.220 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000202 2024-06-10T03:04:14.220 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:14.220 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_encrypted_bytes": 160, 2024-06-10T03:04:14.221 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_encrypted_bytes": 372144 2024-06-10T03:04:14.221 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:14.221 INFO:teuthology.orchestra.run.smithi120.stdout: "cct": { 2024-06-10T03:04:14.221 INFO:teuthology.orchestra.run.smithi120.stdout: "total_workers": 1, 2024-06-10T03:04:14.221 INFO:teuthology.orchestra.run.smithi120.stdout: "unhealthy_workers": 0 2024-06-10T03:04:14.221 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:14.221 INFO:teuthology.orchestra.run.smithi120.stdout: "finisher-MDSRank": { 2024-06-10T03:04:14.221 INFO:teuthology.orchestra.run.smithi120.stdout: "queue_len": 0, 2024-06-10T03:04:14.221 INFO:teuthology.orchestra.run.smithi120.stdout: "complete_latency": { 2024-06-10T03:04:14.221 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 136, 2024-06-10T03:04:14.221 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.099844612, 2024-06-10T03:04:14.221 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000734151 2024-06-10T03:04:14.221 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-06-10T03:04:14.221 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:14.222 INFO:teuthology.orchestra.run.smithi120.stdout: "finisher-PurgeQueue": { 2024-06-10T03:04:14.222 INFO:teuthology.orchestra.run.smithi120.stdout: "queue_len": 0, 2024-06-10T03:04:14.222 INFO:teuthology.orchestra.run.smithi120.stdout: "complete_latency": { 2024-06-10T03:04:14.222 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 186, 2024-06-10T03:04:14.222 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.011822678, 2024-06-10T03:04:14.222 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000063562 2024-06-10T03:04:14.222 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-06-10T03:04:14.222 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:14.222 INFO:teuthology.orchestra.run.smithi120.stdout: "mds": { 2024-06-10T03:04:14.222 INFO:teuthology.orchestra.run.smithi120.stdout: "request": 9738, 2024-06-10T03:04:14.222 INFO:teuthology.orchestra.run.smithi120.stdout: "reply": 184, 2024-06-10T03:04:14.222 INFO:teuthology.orchestra.run.smithi120.stdout: "reply_latency": { 2024-06-10T03:04:14.222 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 184, 2024-06-10T03:04:14.223 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.180645450, 2024-06-10T03:04:14.223 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000981768 2024-06-10T03:04:14.223 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:14.223 INFO:teuthology.orchestra.run.smithi120.stdout: "slow_reply": 0, 2024-06-10T03:04:14.223 INFO:teuthology.orchestra.run.smithi120.stdout: "forward": 9554, 2024-06-10T03:04:14.223 INFO:teuthology.orchestra.run.smithi120.stdout: "dir_fetch_complete": 0, 2024-06-10T03:04:14.223 INFO:teuthology.orchestra.run.smithi120.stdout: "dir_fetch_keys": 0, 2024-06-10T03:04:14.223 INFO:teuthology.orchestra.run.smithi120.stdout: "dir_commit": 11, 2024-06-10T03:04:14.223 INFO:teuthology.orchestra.run.smithi120.stdout: "dir_split": 0, 2024-06-10T03:04:14.223 INFO:teuthology.orchestra.run.smithi120.stdout: "dir_merge": 0, 2024-06-10T03:04:14.223 INFO:teuthology.orchestra.run.smithi120.stdout: "inodes": 29, 2024-06-10T03:04:14.223 INFO:teuthology.orchestra.run.smithi120.stdout: "inodes_top": 9, 2024-06-10T03:04:14.223 INFO:teuthology.orchestra.run.smithi120.stdout: "inodes_bottom": 20, 2024-06-10T03:04:14.223 INFO:teuthology.orchestra.run.smithi120.stdout: "inodes_pin_tail": 0, 2024-06-10T03:04:14.224 INFO:teuthology.orchestra.run.smithi120.stdout: "inodes_pinned": 13, 2024-06-10T03:04:14.224 INFO:teuthology.orchestra.run.smithi120.stdout: "inodes_expired": 276, 2024-06-10T03:04:14.224 INFO:teuthology.orchestra.run.smithi120.stdout: "inodes_with_caps": 1, 2024-06-10T03:04:14.224 INFO:teuthology.orchestra.run.smithi120.stdout: "caps": 1, 2024-06-10T03:04:14.224 INFO:teuthology.orchestra.run.smithi120.stdout: "subtrees": 4, 2024-06-10T03:04:14.224 INFO:teuthology.orchestra.run.smithi120.stdout: "traverse": 11564, 2024-06-10T03:04:14.224 INFO:teuthology.orchestra.run.smithi120.stdout: "traverse_hit": 1765, 2024-06-10T03:04:14.224 INFO:teuthology.orchestra.run.smithi120.stdout: "traverse_forward": 9552, 2024-06-10T03:04:14.224 INFO:teuthology.orchestra.run.smithi120.stdout: "traverse_discover": 152, 2024-06-10T03:04:14.224 INFO:teuthology.orchestra.run.smithi120.stdout: "traverse_dir_fetch": 0, 2024-06-10T03:04:14.224 INFO:teuthology.orchestra.run.smithi120.stdout: "traverse_remote_ino": 0, 2024-06-10T03:04:14.224 INFO:teuthology.orchestra.run.smithi120.stdout: "traverse_lock": 0, 2024-06-10T03:04:14.224 INFO:teuthology.orchestra.run.smithi120.stdout: "load_cent": 0, 2024-06-10T03:04:14.225 INFO:teuthology.orchestra.run.smithi120.stdout: "q": 0, 2024-06-10T03:04:14.225 INFO:teuthology.orchestra.run.smithi120.stdout: "exported": 1, 2024-06-10T03:04:14.225 INFO:teuthology.orchestra.run.smithi120.stdout: "exported_inodes": 0, 2024-06-10T03:04:14.225 INFO:teuthology.orchestra.run.smithi120.stdout: "imported": 1, 2024-06-10T03:04:14.225 INFO:teuthology.orchestra.run.smithi120.stdout: "imported_inodes": 182, 2024-06-10T03:04:14.225 INFO:teuthology.orchestra.run.smithi120.stdout: "openino_dir_fetch": 0, 2024-06-10T03:04:14.225 INFO:teuthology.orchestra.run.smithi120.stdout: "openino_backtrace_fetch": 91, 2024-06-10T03:04:14.225 INFO:teuthology.orchestra.run.smithi120.stdout: "openino_peer_discover": 0, 2024-06-10T03:04:14.225 INFO:teuthology.orchestra.run.smithi120.stdout: "root_rfiles": 3367, 2024-06-10T03:04:14.225 INFO:teuthology.orchestra.run.smithi120.stdout: "root_rbytes": 68050036, 2024-06-10T03:04:14.225 INFO:teuthology.orchestra.run.smithi120.stdout: "root_rsnaps": 0, 2024-06-10T03:04:14.225 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_backtrace_fetch": 91, 2024-06-10T03:04:14.225 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_set_tag": 0, 2024-06-10T03:04:14.225 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_backtrace_repaired": 0, 2024-06-10T03:04:14.226 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_inotable_repaired": 0, 2024-06-10T03:04:14.226 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_dir_inodes": 0, 2024-06-10T03:04:14.226 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_dir_base_inodes": 0, 2024-06-10T03:04:14.226 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_dirfrag_rstats": 0, 2024-06-10T03:04:14.226 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_file_inodes": 91, 2024-06-10T03:04:14.226 INFO:teuthology.orchestra.run.smithi120.stdout: "handle_inode_file_caps": 0, 2024-06-10T03:04:14.226 INFO:teuthology.orchestra.run.smithi120.stdout: "ceph_cap_op_revoke": 3, 2024-06-10T03:04:14.226 INFO:teuthology.orchestra.run.smithi120.stdout: "ceph_cap_op_grant": 92, 2024-06-10T03:04:14.226 INFO:teuthology.orchestra.run.smithi120.stdout: "ceph_cap_op_trunc": 0, 2024-06-10T03:04:14.226 INFO:teuthology.orchestra.run.smithi120.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-06-10T03:04:14.226 INFO:teuthology.orchestra.run.smithi120.stdout: "ceph_cap_op_flush_ack": 0, 2024-06-10T03:04:14.226 INFO:teuthology.orchestra.run.smithi120.stdout: "handle_client_caps": 93, 2024-06-10T03:04:14.226 INFO:teuthology.orchestra.run.smithi120.stdout: "handle_client_caps_dirty": 91, 2024-06-10T03:04:14.227 INFO:teuthology.orchestra.run.smithi120.stdout: "handle_client_cap_release": 1, 2024-06-10T03:04:14.227 INFO:teuthology.orchestra.run.smithi120.stdout: "process_request_cap_release": 184 2024-06-10T03:04:14.227 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:14.227 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_cache": { 2024-06-10T03:04:14.227 INFO:teuthology.orchestra.run.smithi120.stdout: "dir_update": 3, 2024-06-10T03:04:14.227 INFO:teuthology.orchestra.run.smithi120.stdout: "dir_update_receipt": 2736, 2024-06-10T03:04:14.227 INFO:teuthology.orchestra.run.smithi120.stdout: "dir_try_discover": 1603, 2024-06-10T03:04:14.227 INFO:teuthology.orchestra.run.smithi120.stdout: "dir_send_discover": 1974, 2024-06-10T03:04:14.227 INFO:teuthology.orchestra.run.smithi120.stdout: "dir_handle_discover": 3, 2024-06-10T03:04:14.227 INFO:teuthology.orchestra.run.smithi120.stdout: "num_strays": 0, 2024-06-10T03:04:14.227 INFO:teuthology.orchestra.run.smithi120.stdout: "num_strays_delayed": 0, 2024-06-10T03:04:14.227 INFO:teuthology.orchestra.run.smithi120.stdout: "num_strays_enqueuing": 0, 2024-06-10T03:04:14.227 INFO:teuthology.orchestra.run.smithi120.stdout: "strays_created": 91, 2024-06-10T03:04:14.227 INFO:teuthology.orchestra.run.smithi120.stdout: "strays_enqueued": 91, 2024-06-10T03:04:14.227 INFO:teuthology.orchestra.run.smithi120.stdout: "strays_reintegrated": 0, 2024-06-10T03:04:14.228 INFO:teuthology.orchestra.run.smithi120.stdout: "strays_migrated": 0, 2024-06-10T03:04:14.228 INFO:teuthology.orchestra.run.smithi120.stdout: "num_recovering_processing": 0, 2024-06-10T03:04:14.228 INFO:teuthology.orchestra.run.smithi120.stdout: "num_recovering_enqueued": 0, 2024-06-10T03:04:14.228 INFO:teuthology.orchestra.run.smithi120.stdout: "num_recovering_prioritized": 0, 2024-06-10T03:04:14.228 INFO:teuthology.orchestra.run.smithi120.stdout: "recovery_started": 0, 2024-06-10T03:04:14.228 INFO:teuthology.orchestra.run.smithi120.stdout: "recovery_completed": 0, 2024-06-10T03:04:14.228 INFO:teuthology.orchestra.run.smithi120.stdout: "ireq_quiesce_path": 6, 2024-06-10T03:04:14.228 INFO:teuthology.orchestra.run.smithi120.stdout: "ireq_quiesce_inode": 359, 2024-06-10T03:04:14.228 INFO:teuthology.orchestra.run.smithi120.stdout: "ireq_enqueue_scrub": 0, 2024-06-10T03:04:14.228 INFO:teuthology.orchestra.run.smithi120.stdout: "ireq_exportdir": 1, 2024-06-10T03:04:14.228 INFO:teuthology.orchestra.run.smithi120.stdout: "ireq_flush": 0, 2024-06-10T03:04:14.228 INFO:teuthology.orchestra.run.smithi120.stdout: "ireq_fragmentdir": 0, 2024-06-10T03:04:14.228 INFO:teuthology.orchestra.run.smithi120.stdout: "ireq_fragstats": 0, 2024-06-10T03:04:14.228 INFO:teuthology.orchestra.run.smithi120.stdout: "ireq_inodestats": 0 2024-06-10T03:04:14.229 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:14.229 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_log": { 2024-06-10T03:04:14.229 INFO:teuthology.orchestra.run.smithi120.stdout: "evlrg": 0, 2024-06-10T03:04:14.229 INFO:teuthology.orchestra.run.smithi120.stdout: "evadd": 378, 2024-06-10T03:04:14.229 INFO:teuthology.orchestra.run.smithi120.stdout: "evex": 0, 2024-06-10T03:04:14.229 INFO:teuthology.orchestra.run.smithi120.stdout: "evtrm": 0, 2024-06-10T03:04:14.229 INFO:teuthology.orchestra.run.smithi120.stdout: "ev": 378, 2024-06-10T03:04:14.229 INFO:teuthology.orchestra.run.smithi120.stdout: "evexg": 0, 2024-06-10T03:04:14.229 INFO:teuthology.orchestra.run.smithi120.stdout: "evexd": 0, 2024-06-10T03:04:14.229 INFO:teuthology.orchestra.run.smithi120.stdout: "segadd": 2, 2024-06-10T03:04:14.229 INFO:teuthology.orchestra.run.smithi120.stdout: "segex": 0, 2024-06-10T03:04:14.229 INFO:teuthology.orchestra.run.smithi120.stdout: "segtrm": 0, 2024-06-10T03:04:14.229 INFO:teuthology.orchestra.run.smithi120.stdout: "seg": 2, 2024-06-10T03:04:14.229 INFO:teuthology.orchestra.run.smithi120.stdout: "segmjr": 2, 2024-06-10T03:04:14.230 INFO:teuthology.orchestra.run.smithi120.stdout: "segexg": 0, 2024-06-10T03:04:14.230 INFO:teuthology.orchestra.run.smithi120.stdout: "segexd": 0, 2024-06-10T03:04:14.230 INFO:teuthology.orchestra.run.smithi120.stdout: "expos": 4194304, 2024-06-10T03:04:14.230 INFO:teuthology.orchestra.run.smithi120.stdout: "wrpos": 4819865, 2024-06-10T03:04:14.230 INFO:teuthology.orchestra.run.smithi120.stdout: "rdpos": 0, 2024-06-10T03:04:14.230 INFO:teuthology.orchestra.run.smithi120.stdout: "jlat": { 2024-06-10T03:04:14.230 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 14, 2024-06-10T03:04:14.230 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.041413257, 2024-06-10T03:04:14.230 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.002958089 2024-06-10T03:04:14.230 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:14.230 INFO:teuthology.orchestra.run.smithi120.stdout: "replayed": 0 2024-06-10T03:04:14.230 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:14.230 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_mem": { 2024-06-10T03:04:14.231 INFO:teuthology.orchestra.run.smithi120.stdout: "ino": 34, 2024-06-10T03:04:14.231 INFO:teuthology.orchestra.run.smithi120.stdout: "ino+": 263, 2024-06-10T03:04:14.231 INFO:teuthology.orchestra.run.smithi120.stdout: "ino-": 229, 2024-06-10T03:04:14.231 INFO:teuthology.orchestra.run.smithi120.stdout: "dir": 33, 2024-06-10T03:04:14.231 INFO:teuthology.orchestra.run.smithi120.stdout: "dir+": 1292, 2024-06-10T03:04:14.231 INFO:teuthology.orchestra.run.smithi120.stdout: "dir-": 1259, 2024-06-10T03:04:14.231 INFO:teuthology.orchestra.run.smithi120.stdout: "dn": 29, 2024-06-10T03:04:14.231 INFO:teuthology.orchestra.run.smithi120.stdout: "dn+": 578, 2024-06-10T03:04:14.231 INFO:teuthology.orchestra.run.smithi120.stdout: "dn-": 549, 2024-06-10T03:04:14.231 INFO:teuthology.orchestra.run.smithi120.stdout: "cap": 1, 2024-06-10T03:04:14.231 INFO:teuthology.orchestra.run.smithi120.stdout: "cap+": 94, 2024-06-10T03:04:14.231 INFO:teuthology.orchestra.run.smithi120.stdout: "cap-": 93, 2024-06-10T03:04:14.231 INFO:teuthology.orchestra.run.smithi120.stdout: "rss": 59000, 2024-06-10T03:04:14.231 INFO:teuthology.orchestra.run.smithi120.stdout: "heap": 223612 2024-06-10T03:04:14.232 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:14.232 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_server": { 2024-06-10T03:04:14.232 INFO:teuthology.orchestra.run.smithi120.stdout: "dispatch_client_request": 9844, 2024-06-10T03:04:14.232 INFO:teuthology.orchestra.run.smithi120.stdout: "dispatch_server_request": 14, 2024-06-10T03:04:14.232 INFO:teuthology.orchestra.run.smithi120.stdout: "handle_client_request": 9738, 2024-06-10T03:04:14.232 INFO:teuthology.orchestra.run.smithi120.stdout: "handle_client_session": 12, 2024-06-10T03:04:14.232 INFO:teuthology.orchestra.run.smithi120.stdout: "handle_peer_request": 120, 2024-06-10T03:04:14.232 INFO:teuthology.orchestra.run.smithi120.stdout: "req_create_latency": { 2024-06-10T03:04:14.232 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-10T03:04:14.232 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-10T03:04:14.232 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-10T03:04:14.232 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:14.232 INFO:teuthology.orchestra.run.smithi120.stdout: "req_getattr_latency": { 2024-06-10T03:04:14.232 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 1, 2024-06-10T03:04:14.233 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000203394, 2024-06-10T03:04:14.233 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000203394 2024-06-10T03:04:14.233 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:14.233 INFO:teuthology.orchestra.run.smithi120.stdout: "req_getfilelock_latency": { 2024-06-10T03:04:14.233 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-10T03:04:14.233 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-10T03:04:14.233 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-10T03:04:14.233 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:14.233 INFO:teuthology.orchestra.run.smithi120.stdout: "req_link_latency": { 2024-06-10T03:04:14.233 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-10T03:04:14.233 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-10T03:04:14.233 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-10T03:04:14.233 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:14.233 INFO:teuthology.orchestra.run.smithi120.stdout: "req_lookup_latency": { 2024-06-10T03:04:14.234 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 90, 2024-06-10T03:04:14.234 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.028244294, 2024-06-10T03:04:14.234 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000313825 2024-06-10T03:04:14.234 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:14.234 INFO:teuthology.orchestra.run.smithi120.stdout: "req_lookuphash_latency": { 2024-06-10T03:04:14.234 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-10T03:04:14.234 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-10T03:04:14.234 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-10T03:04:14.234 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:14.234 INFO:teuthology.orchestra.run.smithi120.stdout: "req_lookupino_latency": { 2024-06-10T03:04:14.234 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 1, 2024-06-10T03:04:14.234 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000134592, 2024-06-10T03:04:14.234 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000134592 2024-06-10T03:04:14.235 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:14.235 INFO:teuthology.orchestra.run.smithi120.stdout: "req_lookupname_latency": { 2024-06-10T03:04:14.235 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-10T03:04:14.235 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-10T03:04:14.235 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-10T03:04:14.235 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:14.235 INFO:teuthology.orchestra.run.smithi120.stdout: "req_lookupparent_latency": { 2024-06-10T03:04:14.235 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-10T03:04:14.235 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-10T03:04:14.235 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-10T03:04:14.235 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:14.235 INFO:teuthology.orchestra.run.smithi120.stdout: "req_lookupsnap_latency": { 2024-06-10T03:04:14.235 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-10T03:04:14.235 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-10T03:04:14.235 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-10T03:04:14.236 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:14.236 INFO:teuthology.orchestra.run.smithi120.stdout: "req_lssnap_latency": { 2024-06-10T03:04:14.236 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-10T03:04:14.236 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-10T03:04:14.236 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-10T03:04:14.236 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:14.237 INFO:teuthology.orchestra.run.smithi120.stdout: "req_mkdir_latency": { 2024-06-10T03:04:14.237 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-10T03:04:14.237 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-10T03:04:14.237 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-10T03:04:14.237 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:14.237 INFO:teuthology.orchestra.run.smithi120.stdout: "req_mknod_latency": { 2024-06-10T03:04:14.237 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-10T03:04:14.237 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-10T03:04:14.237 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-10T03:04:14.237 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:14.237 INFO:teuthology.orchestra.run.smithi120.stdout: "req_mksnap_latency": { 2024-06-10T03:04:14.237 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-10T03:04:14.237 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-10T03:04:14.238 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-10T03:04:14.238 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:14.238 INFO:teuthology.orchestra.run.smithi120.stdout: "req_open_latency": { 2024-06-10T03:04:14.238 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-10T03:04:14.238 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-10T03:04:14.238 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-10T03:04:14.238 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:14.238 INFO:teuthology.orchestra.run.smithi120.stdout: "req_readdir_latency": { 2024-06-10T03:04:14.238 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 1, 2024-06-10T03:04:14.238 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.003051942, 2024-06-10T03:04:14.238 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.003051942 2024-06-10T03:04:14.238 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:14.239 INFO:teuthology.orchestra.run.smithi120.stdout: "req_rename_latency": { 2024-06-10T03:04:14.239 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-10T03:04:14.239 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-10T03:04:14.239 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-10T03:04:14.239 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:14.239 INFO:teuthology.orchestra.run.smithi120.stdout: "req_renamesnap_latency": { 2024-06-10T03:04:14.239 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-10T03:04:14.239 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-10T03:04:14.239 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-10T03:04:14.239 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:14.239 INFO:teuthology.orchestra.run.smithi120.stdout: "req_snapdiff_latency": { 2024-06-10T03:04:14.239 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-10T03:04:14.239 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-10T03:04:14.239 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-10T03:04:14.240 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:14.240 INFO:teuthology.orchestra.run.smithi120.stdout: "req_rmdir_latency": { 2024-06-10T03:04:14.240 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-10T03:04:14.240 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-10T03:04:14.240 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-10T03:04:14.240 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:14.240 INFO:teuthology.orchestra.run.smithi120.stdout: "req_rmsnap_latency": { 2024-06-10T03:04:14.240 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-10T03:04:14.240 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-10T03:04:14.240 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-10T03:04:14.240 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:14.240 INFO:teuthology.orchestra.run.smithi120.stdout: "req_rmxattr_latency": { 2024-06-10T03:04:14.240 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-10T03:04:14.241 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-10T03:04:14.241 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-10T03:04:14.241 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:14.241 INFO:teuthology.orchestra.run.smithi120.stdout: "req_setattr_latency": { 2024-06-10T03:04:14.241 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-10T03:04:14.241 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-10T03:04:14.241 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-10T03:04:14.241 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:14.241 INFO:teuthology.orchestra.run.smithi120.stdout: "req_setdirlayout_latency": { 2024-06-10T03:04:14.241 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-10T03:04:14.241 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-10T03:04:14.241 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-10T03:04:14.241 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:14.242 INFO:teuthology.orchestra.run.smithi120.stdout: "req_setfilelock_latency": { 2024-06-10T03:04:14.242 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-10T03:04:14.242 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-10T03:04:14.242 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-10T03:04:14.242 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:14.242 INFO:teuthology.orchestra.run.smithi120.stdout: "req_setlayout_latency": { 2024-06-10T03:04:14.242 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-10T03:04:14.242 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-10T03:04:14.242 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-10T03:04:14.242 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:14.242 INFO:teuthology.orchestra.run.smithi120.stdout: "req_setxattr_latency": { 2024-06-10T03:04:14.242 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-10T03:04:14.243 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-10T03:04:14.243 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-10T03:04:14.243 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:14.243 INFO:teuthology.orchestra.run.smithi120.stdout: "req_symlink_latency": { 2024-06-10T03:04:14.243 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-10T03:04:14.243 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-10T03:04:14.243 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-10T03:04:14.243 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:14.243 INFO:teuthology.orchestra.run.smithi120.stdout: "req_unlink_latency": { 2024-06-10T03:04:14.243 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 91, 2024-06-10T03:04:14.243 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.149011228, 2024-06-10T03:04:14.243 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.001637486 2024-06-10T03:04:14.243 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:14.243 INFO:teuthology.orchestra.run.smithi120.stdout: "cap_revoke_eviction": 0, 2024-06-10T03:04:14.244 INFO:teuthology.orchestra.run.smithi120.stdout: "cap_acquisition_throttle": 0, 2024-06-10T03:04:14.244 INFO:teuthology.orchestra.run.smithi120.stdout: "req_getvxattr_latency": { 2024-06-10T03:04:14.244 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-10T03:04:14.244 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-10T03:04:14.244 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-10T03:04:14.244 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-06-10T03:04:14.244 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:14.244 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_sessions": { 2024-06-10T03:04:14.244 INFO:teuthology.orchestra.run.smithi120.stdout: "session_count": 1, 2024-06-10T03:04:14.244 INFO:teuthology.orchestra.run.smithi120.stdout: "session_add": 2, 2024-06-10T03:04:14.244 INFO:teuthology.orchestra.run.smithi120.stdout: "session_remove": 1, 2024-06-10T03:04:14.244 INFO:teuthology.orchestra.run.smithi120.stdout: "sessions_open": 1, 2024-06-10T03:04:14.244 INFO:teuthology.orchestra.run.smithi120.stdout: "sessions_stale": 0, 2024-06-10T03:04:14.245 INFO:teuthology.orchestra.run.smithi120.stdout: "total_load": 181, 2024-06-10T03:04:14.245 INFO:teuthology.orchestra.run.smithi120.stdout: "average_load": 181, 2024-06-10T03:04:14.245 INFO:teuthology.orchestra.run.smithi120.stdout: "avg_session_uptime": 568, 2024-06-10T03:04:14.245 INFO:teuthology.orchestra.run.smithi120.stdout: "mdthresh_evicted": 0 2024-06-10T03:04:14.245 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:14.245 INFO:teuthology.orchestra.run.smithi120.stdout: "mempool": { 2024-06-10T03:04:14.245 INFO:teuthology.orchestra.run.smithi120.stdout: "bloom_filter_bytes": 0, 2024-06-10T03:04:14.245 INFO:teuthology.orchestra.run.smithi120.stdout: "bloom_filter_items": 0, 2024-06-10T03:04:14.245 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_alloc_bytes": 0, 2024-06-10T03:04:14.245 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_alloc_items": 0, 2024-06-10T03:04:14.245 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_data_bytes": 0, 2024-06-10T03:04:14.245 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_data_items": 0, 2024-06-10T03:04:14.245 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-10T03:04:14.246 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_onode_items": 0, 2024-06-10T03:04:14.246 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-10T03:04:14.246 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_meta_items": 0, 2024-06-10T03:04:14.246 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_other_bytes": 0, 2024-06-10T03:04:14.246 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_other_items": 0, 2024-06-10T03:04:14.246 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-10T03:04:14.246 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_buffer_items": 0, 2024-06-10T03:04:14.246 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_extent_bytes": 0, 2024-06-10T03:04:14.246 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_extent_items": 0, 2024-06-10T03:04:14.246 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_blob_bytes": 0, 2024-06-10T03:04:14.246 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_blob_items": 0, 2024-06-10T03:04:14.246 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-10T03:04:14.246 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_shared_blob_items": 0, 2024-06-10T03:04:14.247 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-10T03:04:14.247 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_inline_bl_items": 0, 2024-06-10T03:04:14.247 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_fsck_bytes": 0, 2024-06-10T03:04:14.247 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_fsck_items": 0, 2024-06-10T03:04:14.247 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_txc_bytes": 0, 2024-06-10T03:04:14.247 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_txc_items": 0, 2024-06-10T03:04:14.247 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-10T03:04:14.247 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_writing_deferred_items": 0, 2024-06-10T03:04:14.247 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_writing_bytes": 0, 2024-06-10T03:04:14.247 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_writing_items": 0, 2024-06-10T03:04:14.247 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_bytes": 0, 2024-06-10T03:04:14.247 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_items": 0, 2024-06-10T03:04:14.247 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_file_reader_bytes": 0, 2024-06-10T03:04:14.247 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_file_reader_items": 0, 2024-06-10T03:04:14.248 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_file_writer_bytes": 0, 2024-06-10T03:04:14.248 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_file_writer_items": 0, 2024-06-10T03:04:14.248 INFO:teuthology.orchestra.run.smithi120.stdout: "buffer_anon_bytes": 287511, 2024-06-10T03:04:14.248 INFO:teuthology.orchestra.run.smithi120.stdout: "buffer_anon_items": 102, 2024-06-10T03:04:14.248 INFO:teuthology.orchestra.run.smithi120.stdout: "buffer_meta_bytes": 0, 2024-06-10T03:04:14.248 INFO:teuthology.orchestra.run.smithi120.stdout: "buffer_meta_items": 0, 2024-06-10T03:04:14.248 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_bytes": 0, 2024-06-10T03:04:14.248 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_items": 0, 2024-06-10T03:04:14.248 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_mapbl_bytes": 0, 2024-06-10T03:04:14.248 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_mapbl_items": 0, 2024-06-10T03:04:14.248 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_pglog_bytes": 0, 2024-06-10T03:04:14.248 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_pglog_items": 0, 2024-06-10T03:04:14.248 INFO:teuthology.orchestra.run.smithi120.stdout: "osdmap_bytes": 8984, 2024-06-10T03:04:14.248 INFO:teuthology.orchestra.run.smithi120.stdout: "osdmap_items": 172, 2024-06-10T03:04:14.249 INFO:teuthology.orchestra.run.smithi120.stdout: "osdmap_mapping_bytes": 0, 2024-06-10T03:04:14.249 INFO:teuthology.orchestra.run.smithi120.stdout: "osdmap_mapping_items": 0, 2024-06-10T03:04:14.249 INFO:teuthology.orchestra.run.smithi120.stdout: "pgmap_bytes": 0, 2024-06-10T03:04:14.249 INFO:teuthology.orchestra.run.smithi120.stdout: "pgmap_items": 0, 2024-06-10T03:04:14.249 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_co_bytes": 138637, 2024-06-10T03:04:14.249 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_co_items": 490, 2024-06-10T03:04:14.249 INFO:teuthology.orchestra.run.smithi120.stdout: "unittest_1_bytes": 0, 2024-06-10T03:04:14.249 INFO:teuthology.orchestra.run.smithi120.stdout: "unittest_1_items": 0, 2024-06-10T03:04:14.249 INFO:teuthology.orchestra.run.smithi120.stdout: "unittest_2_bytes": 0, 2024-06-10T03:04:14.249 INFO:teuthology.orchestra.run.smithi120.stdout: "unittest_2_items": 0 2024-06-10T03:04:14.249 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:14.249 INFO:teuthology.orchestra.run.smithi120.stdout: "objecter": { 2024-06-10T03:04:14.249 INFO:teuthology.orchestra.run.smithi120.stdout: "op_active": 0, 2024-06-10T03:04:14.249 INFO:teuthology.orchestra.run.smithi120.stdout: "op_laggy": 0, 2024-06-10T03:04:14.250 INFO:teuthology.orchestra.run.smithi120.stdout: "op_send": 341, 2024-06-10T03:04:14.250 INFO:teuthology.orchestra.run.smithi120.stdout: "op_send_bytes": 655682, 2024-06-10T03:04:14.250 INFO:teuthology.orchestra.run.smithi120.stdout: "op_resend": 33, 2024-06-10T03:04:14.250 INFO:teuthology.orchestra.run.smithi120.stdout: "op_reply": 341, 2024-06-10T03:04:14.250 INFO:teuthology.orchestra.run.smithi120.stdout: "op_latency": { 2024-06-10T03:04:14.250 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 341, 2024-06-10T03:04:14.250 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 1.413968325, 2024-06-10T03:04:14.250 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.004146534 2024-06-10T03:04:14.250 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:14.250 INFO:teuthology.orchestra.run.smithi120.stdout: "op_inflight": 0, 2024-06-10T03:04:14.250 INFO:teuthology.orchestra.run.smithi120.stdout: "oplen_avg": { 2024-06-10T03:04:14.250 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 341, 2024-06-10T03:04:14.250 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 369 2024-06-10T03:04:14.251 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:14.251 INFO:teuthology.orchestra.run.smithi120.stdout: "op": 341, 2024-06-10T03:04:14.251 INFO:teuthology.orchestra.run.smithi120.stdout: "op_r": 92, 2024-06-10T03:04:14.251 INFO:teuthology.orchestra.run.smithi120.stdout: "op_w": 249, 2024-06-10T03:04:14.251 INFO:teuthology.orchestra.run.smithi120.stdout: "op_rmw": 0, 2024-06-10T03:04:14.251 INFO:teuthology.orchestra.run.smithi120.stdout: "op_pg": 0, 2024-06-10T03:04:14.251 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_stat": 0, 2024-06-10T03:04:14.251 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_create": 10, 2024-06-10T03:04:14.251 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_read": 1, 2024-06-10T03:04:14.251 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_write": 15, 2024-06-10T03:04:14.251 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_writefull": 12, 2024-06-10T03:04:14.251 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_writesame": 0, 2024-06-10T03:04:14.251 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_append": 0, 2024-06-10T03:04:14.251 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_zero": 0, 2024-06-10T03:04:14.252 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_truncate": 0, 2024-06-10T03:04:14.252 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_delete": 194, 2024-06-10T03:04:14.252 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_mapext": 0, 2024-06-10T03:04:14.252 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_sparse_read": 0, 2024-06-10T03:04:14.252 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_clonerange": 0, 2024-06-10T03:04:14.252 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_getxattr": 91, 2024-06-10T03:04:14.252 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_setxattr": 20, 2024-06-10T03:04:14.252 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_cmpxattr": 0, 2024-06-10T03:04:14.252 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_rmxattr": 0, 2024-06-10T03:04:14.252 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_resetxattrs": 0, 2024-06-10T03:04:14.252 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_call": 0, 2024-06-10T03:04:14.252 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_watch": 0, 2024-06-10T03:04:14.252 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_notify": 0, 2024-06-10T03:04:14.253 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_src_cmpxattr": 0, 2024-06-10T03:04:14.253 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_pgls": 0, 2024-06-10T03:04:14.253 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_pgls_filter": 0, 2024-06-10T03:04:14.253 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_other": 0, 2024-06-10T03:04:14.253 INFO:teuthology.orchestra.run.smithi120.stdout: "linger_active": 0, 2024-06-10T03:04:14.253 INFO:teuthology.orchestra.run.smithi120.stdout: "linger_send": 0, 2024-06-10T03:04:14.253 INFO:teuthology.orchestra.run.smithi120.stdout: "linger_resend": 0, 2024-06-10T03:04:14.253 INFO:teuthology.orchestra.run.smithi120.stdout: "linger_ping": 0, 2024-06-10T03:04:14.253 INFO:teuthology.orchestra.run.smithi120.stdout: "poolop_active": 0, 2024-06-10T03:04:14.253 INFO:teuthology.orchestra.run.smithi120.stdout: "poolop_send": 0, 2024-06-10T03:04:14.253 INFO:teuthology.orchestra.run.smithi120.stdout: "poolop_resend": 0, 2024-06-10T03:04:14.253 INFO:teuthology.orchestra.run.smithi120.stdout: "poolstat_active": 0, 2024-06-10T03:04:14.253 INFO:teuthology.orchestra.run.smithi120.stdout: "poolstat_send": 0, 2024-06-10T03:04:14.253 INFO:teuthology.orchestra.run.smithi120.stdout: "poolstat_resend": 0, 2024-06-10T03:04:14.254 INFO:teuthology.orchestra.run.smithi120.stdout: "statfs_active": 0, 2024-06-10T03:04:14.254 INFO:teuthology.orchestra.run.smithi120.stdout: "statfs_send": 0, 2024-06-10T03:04:14.254 INFO:teuthology.orchestra.run.smithi120.stdout: "statfs_resend": 0, 2024-06-10T03:04:14.254 INFO:teuthology.orchestra.run.smithi120.stdout: "command_active": 0, 2024-06-10T03:04:14.254 INFO:teuthology.orchestra.run.smithi120.stdout: "command_send": 0, 2024-06-10T03:04:14.254 INFO:teuthology.orchestra.run.smithi120.stdout: "command_resend": 0, 2024-06-10T03:04:14.254 INFO:teuthology.orchestra.run.smithi120.stdout: "map_epoch": 100, 2024-06-10T03:04:14.254 INFO:teuthology.orchestra.run.smithi120.stdout: "map_full": 0, 2024-06-10T03:04:14.254 INFO:teuthology.orchestra.run.smithi120.stdout: "map_inc": 2, 2024-06-10T03:04:14.254 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_sessions": 12, 2024-06-10T03:04:14.254 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_session_open": 12, 2024-06-10T03:04:14.254 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_session_close": 0, 2024-06-10T03:04:14.255 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_laggy": 0, 2024-06-10T03:04:14.255 INFO:teuthology.orchestra.run.smithi120.stdout: "omap_wr": 21, 2024-06-10T03:04:14.255 INFO:teuthology.orchestra.run.smithi120.stdout: "omap_rd": 0, 2024-06-10T03:04:14.255 INFO:teuthology.orchestra.run.smithi120.stdout: "omap_del": 5 2024-06-10T03:04:14.255 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:14.255 INFO:teuthology.orchestra.run.smithi120.stdout: "oft": { 2024-06-10T03:04:14.255 INFO:teuthology.orchestra.run.smithi120.stdout: "omap_total_objs": 1, 2024-06-10T03:04:14.255 INFO:teuthology.orchestra.run.smithi120.stdout: "omap_total_kv_pairs": 0, 2024-06-10T03:04:14.255 INFO:teuthology.orchestra.run.smithi120.stdout: "omap_total_updates": 0, 2024-06-10T03:04:14.255 INFO:teuthology.orchestra.run.smithi120.stdout: "omap_total_removes": 0 2024-06-10T03:04:14.255 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:14.255 INFO:teuthology.orchestra.run.smithi120.stdout: "osd-slow-ops": { 2024-06-10T03:04:14.255 INFO:teuthology.orchestra.run.smithi120.stdout: "slow_ops_count": 349 2024-06-10T03:04:14.255 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:14.256 INFO:teuthology.orchestra.run.smithi120.stdout: "purge_queue": { 2024-06-10T03:04:14.256 INFO:teuthology.orchestra.run.smithi120.stdout: "pq_executing_ops": 0, 2024-06-10T03:04:14.256 INFO:teuthology.orchestra.run.smithi120.stdout: "pq_executing_ops_high_water": 10, 2024-06-10T03:04:14.256 INFO:teuthology.orchestra.run.smithi120.stdout: "pq_executing": 0, 2024-06-10T03:04:14.256 INFO:teuthology.orchestra.run.smithi120.stdout: "pq_executing_high_water": 5, 2024-06-10T03:04:14.256 INFO:teuthology.orchestra.run.smithi120.stdout: "pq_executed_ops": 182, 2024-06-10T03:04:14.256 INFO:teuthology.orchestra.run.smithi120.stdout: "pq_executed": 91, 2024-06-10T03:04:14.256 INFO:teuthology.orchestra.run.smithi120.stdout: "pq_item_in_journal": 0 2024-06-10T03:04:14.256 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:14.256 INFO:teuthology.orchestra.run.smithi120.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-10T03:04:14.256 INFO:teuthology.orchestra.run.smithi120.stdout: "val": 0, 2024-06-10T03:04:14.256 INFO:teuthology.orchestra.run.smithi120.stdout: "max": 104857600, 2024-06-10T03:04:14.256 INFO:teuthology.orchestra.run.smithi120.stdout: "get_started": 0, 2024-06-10T03:04:14.256 INFO:teuthology.orchestra.run.smithi120.stdout: "get": 20955, 2024-06-10T03:04:14.257 INFO:teuthology.orchestra.run.smithi120.stdout: "get_sum": 6962488, 2024-06-10T03:04:14.257 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_fail": 0, 2024-06-10T03:04:14.257 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_success": 20955, 2024-06-10T03:04:14.257 INFO:teuthology.orchestra.run.smithi120.stdout: "take": 0, 2024-06-10T03:04:14.257 INFO:teuthology.orchestra.run.smithi120.stdout: "take_sum": 0, 2024-06-10T03:04:14.257 INFO:teuthology.orchestra.run.smithi120.stdout: "put": 20955, 2024-06-10T03:04:14.257 INFO:teuthology.orchestra.run.smithi120.stdout: "put_sum": 6962488, 2024-06-10T03:04:14.257 INFO:teuthology.orchestra.run.smithi120.stdout: "wait": { 2024-06-10T03:04:14.257 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-10T03:04:14.257 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-10T03:04:14.257 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-10T03:04:14.257 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-06-10T03:04:14.257 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:14.257 INFO:teuthology.orchestra.run.smithi120.stdout: "throttle-objecter_bytes": { 2024-06-10T03:04:14.257 INFO:teuthology.orchestra.run.smithi120.stdout: "val": 0, 2024-06-10T03:04:14.258 INFO:teuthology.orchestra.run.smithi120.stdout: "max": 104857600, 2024-06-10T03:04:14.258 INFO:teuthology.orchestra.run.smithi120.stdout: "get_started": 0, 2024-06-10T03:04:14.258 INFO:teuthology.orchestra.run.smithi120.stdout: "get": 0, 2024-06-10T03:04:14.258 INFO:teuthology.orchestra.run.smithi120.stdout: "get_sum": 0, 2024-06-10T03:04:14.258 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_fail": 0, 2024-06-10T03:04:14.258 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_success": 0, 2024-06-10T03:04:14.258 INFO:teuthology.orchestra.run.smithi120.stdout: "take": 341, 2024-06-10T03:04:14.258 INFO:teuthology.orchestra.run.smithi120.stdout: "take_sum": 665601, 2024-06-10T03:04:14.258 INFO:teuthology.orchestra.run.smithi120.stdout: "put": 341, 2024-06-10T03:04:14.258 INFO:teuthology.orchestra.run.smithi120.stdout: "put_sum": 665601, 2024-06-10T03:04:14.258 INFO:teuthology.orchestra.run.smithi120.stdout: "wait": { 2024-06-10T03:04:14.258 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-10T03:04:14.258 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-10T03:04:14.258 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-10T03:04:14.259 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-06-10T03:04:14.259 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:14.259 INFO:teuthology.orchestra.run.smithi120.stdout: "throttle-objecter_ops": { 2024-06-10T03:04:14.259 INFO:teuthology.orchestra.run.smithi120.stdout: "val": 0, 2024-06-10T03:04:14.259 INFO:teuthology.orchestra.run.smithi120.stdout: "max": 1024, 2024-06-10T03:04:14.259 INFO:teuthology.orchestra.run.smithi120.stdout: "get_started": 0, 2024-06-10T03:04:14.259 INFO:teuthology.orchestra.run.smithi120.stdout: "get": 0, 2024-06-10T03:04:14.259 INFO:teuthology.orchestra.run.smithi120.stdout: "get_sum": 0, 2024-06-10T03:04:14.259 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_fail": 0, 2024-06-10T03:04:14.259 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_success": 0, 2024-06-10T03:04:14.259 INFO:teuthology.orchestra.run.smithi120.stdout: "take": 341, 2024-06-10T03:04:14.259 INFO:teuthology.orchestra.run.smithi120.stdout: "take_sum": 341, 2024-06-10T03:04:14.259 INFO:teuthology.orchestra.run.smithi120.stdout: "put": 341, 2024-06-10T03:04:14.260 INFO:teuthology.orchestra.run.smithi120.stdout: "put_sum": 341, 2024-06-10T03:04:14.260 INFO:teuthology.orchestra.run.smithi120.stdout: "wait": { 2024-06-10T03:04:14.260 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-10T03:04:14.260 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-10T03:04:14.260 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-10T03:04:14.260 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-06-10T03:04:14.260 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:14.260 INFO:teuthology.orchestra.run.smithi120.stdout: "throttle-write_buf_throttle": { 2024-06-10T03:04:14.260 INFO:teuthology.orchestra.run.smithi120.stdout: "val": 0, 2024-06-10T03:04:14.260 INFO:teuthology.orchestra.run.smithi120.stdout: "max": 3758096384, 2024-06-10T03:04:14.260 INFO:teuthology.orchestra.run.smithi120.stdout: "get_started": 0, 2024-06-10T03:04:14.260 INFO:teuthology.orchestra.run.smithi120.stdout: "get": 91, 2024-06-10T03:04:14.260 INFO:teuthology.orchestra.run.smithi120.stdout: "get_sum": 9919, 2024-06-10T03:04:14.261 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_fail": 0, 2024-06-10T03:04:14.261 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_success": 91, 2024-06-10T03:04:14.261 INFO:teuthology.orchestra.run.smithi120.stdout: "take": 0, 2024-06-10T03:04:14.261 INFO:teuthology.orchestra.run.smithi120.stdout: "take_sum": 0, 2024-06-10T03:04:14.261 INFO:teuthology.orchestra.run.smithi120.stdout: "put": 1, 2024-06-10T03:04:14.261 INFO:teuthology.orchestra.run.smithi120.stdout: "put_sum": 9919, 2024-06-10T03:04:14.261 INFO:teuthology.orchestra.run.smithi120.stdout: "wait": { 2024-06-10T03:04:14.261 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-10T03:04:14.261 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-10T03:04:14.261 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-10T03:04:14.261 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-06-10T03:04:14.261 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:14.261 INFO:teuthology.orchestra.run.smithi120.stdout: "throttle-write_buf_throttle-0x5610a4bbe500": { 2024-06-10T03:04:14.261 INFO:teuthology.orchestra.run.smithi120.stdout: "val": 0, 2024-06-10T03:04:14.261 INFO:teuthology.orchestra.run.smithi120.stdout: "max": 3758096384, 2024-06-10T03:04:14.262 INFO:teuthology.orchestra.run.smithi120.stdout: "get_started": 0, 2024-06-10T03:04:14.262 INFO:teuthology.orchestra.run.smithi120.stdout: "get": 378, 2024-06-10T03:04:14.262 INFO:teuthology.orchestra.run.smithi120.stdout: "get_sum": 625561, 2024-06-10T03:04:14.262 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_fail": 0, 2024-06-10T03:04:14.262 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_success": 378, 2024-06-10T03:04:14.262 INFO:teuthology.orchestra.run.smithi120.stdout: "take": 0, 2024-06-10T03:04:14.262 INFO:teuthology.orchestra.run.smithi120.stdout: "take_sum": 0, 2024-06-10T03:04:14.262 INFO:teuthology.orchestra.run.smithi120.stdout: "put": 14, 2024-06-10T03:04:14.262 INFO:teuthology.orchestra.run.smithi120.stdout: "put_sum": 625561, 2024-06-10T03:04:14.262 INFO:teuthology.orchestra.run.smithi120.stdout: "wait": { 2024-06-10T03:04:14.262 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-10T03:04:14.262 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-10T03:04:14.262 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-10T03:04:14.262 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-06-10T03:04:14.263 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-06-10T03:04:14.263 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-06-10T03:04:15.033 INFO:tasks.check_counter:Daemon mds.b mds.exported=1 2024-06-10T03:04:15.033 INFO:tasks.check_counter:Daemon mds.b mds.imported=1 2024-06-10T03:04:15.033 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update=3 2024-06-10T03:04:15.033 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update_receipt=2736 2024-06-10T03:04:15.033 INFO:tasks.check_counter:Daemon mds.b mds.dir_split=0 2024-06-10T03:04:15.033 DEBUG:tasks.check_counter:Getting stats from e 2024-06-10T03:04:15.034 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph daemon mds.e perf dump 2024-06-10T03:04:15.204 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:04:14 smithi188 ceph-mon[36599]: pgmap v735: 129 pgs: 129 active+clean; 228 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-06-10T03:04:15.227 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:04:14 smithi052 ceph-mon[32362]: pgmap v735: 129 pgs: 129 active+clean; 228 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-06-10T03:04:15.273 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.b/config 2024-06-10T03:04:15.388 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:04:14 smithi120 ceph-mon[36634]: pgmap v735: 129 pgs: 129 active+clean; 228 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-06-10T03:04:17.015 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:04:16 smithi120 ceph-mon[36634]: pgmap v736: 129 pgs: 129 active+clean; 228 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-06-10T03:04:17.016 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-06-10T03:04:17.016 INFO:teuthology.orchestra.run.smithi120.stdout: "AsyncMessenger::Worker-0": { 2024-06-10T03:04:17.016 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_messages": 2363, 2024-06-10T03:04:17.016 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_messages": 2365, 2024-06-10T03:04:17.016 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_bytes": 7973960, 2024-06-10T03:04:17.016 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_bytes": 751441, 2024-06-10T03:04:17.017 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_created_connections": 6, 2024-06-10T03:04:17.017 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_active_connections": 2, 2024-06-10T03:04:17.017 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_total_time": 0.168404912, 2024-06-10T03:04:17.017 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_send_time": 0.051403912, 2024-06-10T03:04:17.017 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_recv_time": 0.045569253, 2024-06-10T03:04:17.017 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_fast_dispatch_time": 0.059093574, 2024-06-10T03:04:17.017 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_messages_queue_lat": { 2024-06-10T03:04:17.017 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 2363, 2024-06-10T03:04:17.017 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.065651798, 2024-06-10T03:04:17.017 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000027783 2024-06-10T03:04:17.017 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:17.017 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_handle_ack_lat": { 2024-06-10T03:04:17.017 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-10T03:04:17.017 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-10T03:04:17.017 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-10T03:04:17.018 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:17.018 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_encrypted_bytes": 0, 2024-06-10T03:04:17.018 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_encrypted_bytes": 0 2024-06-10T03:04:17.018 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:17.018 INFO:teuthology.orchestra.run.smithi120.stdout: "AsyncMessenger::Worker-1": { 2024-06-10T03:04:17.018 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_messages": 272, 2024-06-10T03:04:17.018 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_messages": 248, 2024-06-10T03:04:17.018 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_bytes": 907052, 2024-06-10T03:04:17.018 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_bytes": 149146, 2024-06-10T03:04:17.018 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_created_connections": 3, 2024-06-10T03:04:17.018 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_active_connections": 2, 2024-06-10T03:04:17.018 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_total_time": 0.040546828, 2024-06-10T03:04:17.018 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_send_time": 0.014476188, 2024-06-10T03:04:17.019 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_recv_time": 0.022908463, 2024-06-10T03:04:17.019 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_fast_dispatch_time": 0.005766658, 2024-06-10T03:04:17.019 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_messages_queue_lat": { 2024-06-10T03:04:17.019 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 247, 2024-06-10T03:04:17.019 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.012543742, 2024-06-10T03:04:17.019 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000050784 2024-06-10T03:04:17.019 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:17.019 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_handle_ack_lat": { 2024-06-10T03:04:17.019 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-10T03:04:17.019 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-10T03:04:17.019 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-10T03:04:17.019 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:17.019 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_encrypted_bytes": 906496, 2024-06-10T03:04:17.019 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_encrypted_bytes": 148448 2024-06-10T03:04:17.019 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:17.020 INFO:teuthology.orchestra.run.smithi120.stdout: "AsyncMessenger::Worker-2": { 2024-06-10T03:04:17.020 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_messages": 6, 2024-06-10T03:04:17.020 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_messages": 185, 2024-06-10T03:04:17.020 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_bytes": 3500, 2024-06-10T03:04:17.020 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_bytes": 375278, 2024-06-10T03:04:17.020 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_created_connections": 4, 2024-06-10T03:04:17.020 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_active_connections": 3, 2024-06-10T03:04:17.020 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_total_time": 0.010134724, 2024-06-10T03:04:17.020 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_send_time": 0.007261560, 2024-06-10T03:04:17.020 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_recv_time": 0.001314759, 2024-06-10T03:04:17.020 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_fast_dispatch_time": 0.000025830, 2024-06-10T03:04:17.020 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_messages_queue_lat": { 2024-06-10T03:04:17.020 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 184, 2024-06-10T03:04:17.021 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.011447535, 2024-06-10T03:04:17.021 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000062214 2024-06-10T03:04:17.021 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:17.021 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_handle_ack_lat": { 2024-06-10T03:04:17.021 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-10T03:04:17.021 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-10T03:04:17.021 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-10T03:04:17.021 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:17.021 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_encrypted_bytes": 3232, 2024-06-10T03:04:17.021 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_encrypted_bytes": 374960 2024-06-10T03:04:17.021 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:17.021 INFO:teuthology.orchestra.run.smithi120.stdout: "cct": { 2024-06-10T03:04:17.021 INFO:teuthology.orchestra.run.smithi120.stdout: "total_workers": 1, 2024-06-10T03:04:17.021 INFO:teuthology.orchestra.run.smithi120.stdout: "unhealthy_workers": 0 2024-06-10T03:04:17.022 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:17.022 INFO:teuthology.orchestra.run.smithi120.stdout: "finisher-MDSRank": { 2024-06-10T03:04:17.022 INFO:teuthology.orchestra.run.smithi120.stdout: "queue_len": 0, 2024-06-10T03:04:17.022 INFO:teuthology.orchestra.run.smithi120.stdout: "complete_latency": { 2024-06-10T03:04:17.022 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 4001, 2024-06-10T03:04:17.022 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.087508744, 2024-06-10T03:04:17.022 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000021871 2024-06-10T03:04:17.022 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-06-10T03:04:17.022 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:17.022 INFO:teuthology.orchestra.run.smithi120.stdout: "finisher-PurgeQueue": { 2024-06-10T03:04:17.022 INFO:teuthology.orchestra.run.smithi120.stdout: "queue_len": 0, 2024-06-10T03:04:17.022 INFO:teuthology.orchestra.run.smithi120.stdout: "complete_latency": { 2024-06-10T03:04:17.022 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-10T03:04:17.023 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-10T03:04:17.023 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-10T03:04:17.023 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-06-10T03:04:17.023 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:17.023 INFO:teuthology.orchestra.run.smithi120.stdout: "mds": { 2024-06-10T03:04:17.023 INFO:teuthology.orchestra.run.smithi120.stdout: "request": 0, 2024-06-10T03:04:17.023 INFO:teuthology.orchestra.run.smithi120.stdout: "reply": 0, 2024-06-10T03:04:17.023 INFO:teuthology.orchestra.run.smithi120.stdout: "reply_latency": { 2024-06-10T03:04:17.023 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-10T03:04:17.023 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-10T03:04:17.023 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-10T03:04:17.023 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:17.023 INFO:teuthology.orchestra.run.smithi120.stdout: "slow_reply": 0, 2024-06-10T03:04:17.023 INFO:teuthology.orchestra.run.smithi120.stdout: "forward": 0, 2024-06-10T03:04:17.023 INFO:teuthology.orchestra.run.smithi120.stdout: "dir_fetch_complete": 0, 2024-06-10T03:04:17.023 INFO:teuthology.orchestra.run.smithi120.stdout: "dir_fetch_keys": 0, 2024-06-10T03:04:17.024 INFO:teuthology.orchestra.run.smithi120.stdout: "dir_commit": 0, 2024-06-10T03:04:17.024 INFO:teuthology.orchestra.run.smithi120.stdout: "dir_split": 0, 2024-06-10T03:04:17.024 INFO:teuthology.orchestra.run.smithi120.stdout: "dir_merge": 0, 2024-06-10T03:04:17.024 INFO:teuthology.orchestra.run.smithi120.stdout: "inodes": 907, 2024-06-10T03:04:17.024 INFO:teuthology.orchestra.run.smithi120.stdout: "inodes_top": 0, 2024-06-10T03:04:17.024 INFO:teuthology.orchestra.run.smithi120.stdout: "inodes_bottom": 907, 2024-06-10T03:04:17.024 INFO:teuthology.orchestra.run.smithi120.stdout: "inodes_pin_tail": 0, 2024-06-10T03:04:17.024 INFO:teuthology.orchestra.run.smithi120.stdout: "inodes_pinned": 907, 2024-06-10T03:04:17.024 INFO:teuthology.orchestra.run.smithi120.stdout: "inodes_expired": 0, 2024-06-10T03:04:17.024 INFO:teuthology.orchestra.run.smithi120.stdout: "inodes_with_caps": 0, 2024-06-10T03:04:17.024 INFO:teuthology.orchestra.run.smithi120.stdout: "caps": 0, 2024-06-10T03:04:17.024 INFO:teuthology.orchestra.run.smithi120.stdout: "subtrees": 1, 2024-06-10T03:04:17.024 INFO:teuthology.orchestra.run.smithi120.stdout: "traverse": 0, 2024-06-10T03:04:17.024 INFO:teuthology.orchestra.run.smithi120.stdout: "traverse_hit": 0, 2024-06-10T03:04:17.024 INFO:teuthology.orchestra.run.smithi120.stdout: "traverse_forward": 0, 2024-06-10T03:04:17.025 INFO:teuthology.orchestra.run.smithi120.stdout: "traverse_discover": 0, 2024-06-10T03:04:17.025 INFO:teuthology.orchestra.run.smithi120.stdout: "traverse_dir_fetch": 0, 2024-06-10T03:04:17.025 INFO:teuthology.orchestra.run.smithi120.stdout: "traverse_remote_ino": 0, 2024-06-10T03:04:17.025 INFO:teuthology.orchestra.run.smithi120.stdout: "traverse_lock": 0, 2024-06-10T03:04:17.025 INFO:teuthology.orchestra.run.smithi120.stdout: "load_cent": 0, 2024-06-10T03:04:17.025 INFO:teuthology.orchestra.run.smithi120.stdout: "q": 0, 2024-06-10T03:04:17.025 INFO:teuthology.orchestra.run.smithi120.stdout: "exported": 0, 2024-06-10T03:04:17.025 INFO:teuthology.orchestra.run.smithi120.stdout: "exported_inodes": 0, 2024-06-10T03:04:17.025 INFO:teuthology.orchestra.run.smithi120.stdout: "imported": 0, 2024-06-10T03:04:17.025 INFO:teuthology.orchestra.run.smithi120.stdout: "imported_inodes": 0, 2024-06-10T03:04:17.025 INFO:teuthology.orchestra.run.smithi120.stdout: "openino_dir_fetch": 0, 2024-06-10T03:04:17.025 INFO:teuthology.orchestra.run.smithi120.stdout: "openino_backtrace_fetch": 0, 2024-06-10T03:04:17.025 INFO:teuthology.orchestra.run.smithi120.stdout: "openino_peer_discover": 0, 2024-06-10T03:04:17.025 INFO:teuthology.orchestra.run.smithi120.stdout: "root_rfiles": 0, 2024-06-10T03:04:17.025 INFO:teuthology.orchestra.run.smithi120.stdout: "root_rbytes": 0, 2024-06-10T03:04:17.025 INFO:teuthology.orchestra.run.smithi120.stdout: "root_rsnaps": 0, 2024-06-10T03:04:17.026 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_backtrace_fetch": 0, 2024-06-10T03:04:17.026 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_set_tag": 0, 2024-06-10T03:04:17.026 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_backtrace_repaired": 0, 2024-06-10T03:04:17.026 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_inotable_repaired": 0, 2024-06-10T03:04:17.026 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_dir_inodes": 0, 2024-06-10T03:04:17.026 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_dir_base_inodes": 0, 2024-06-10T03:04:17.026 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_dirfrag_rstats": 0, 2024-06-10T03:04:17.026 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_file_inodes": 0, 2024-06-10T03:04:17.026 INFO:teuthology.orchestra.run.smithi120.stdout: "handle_inode_file_caps": 0, 2024-06-10T03:04:17.026 INFO:teuthology.orchestra.run.smithi120.stdout: "ceph_cap_op_revoke": 0, 2024-06-10T03:04:17.026 INFO:teuthology.orchestra.run.smithi120.stdout: "ceph_cap_op_grant": 0, 2024-06-10T03:04:17.026 INFO:teuthology.orchestra.run.smithi120.stdout: "ceph_cap_op_trunc": 0, 2024-06-10T03:04:17.026 INFO:teuthology.orchestra.run.smithi120.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-06-10T03:04:17.026 INFO:teuthology.orchestra.run.smithi120.stdout: "ceph_cap_op_flush_ack": 0, 2024-06-10T03:04:17.027 INFO:teuthology.orchestra.run.smithi120.stdout: "handle_client_caps": 0, 2024-06-10T03:04:17.027 INFO:teuthology.orchestra.run.smithi120.stdout: "handle_client_caps_dirty": 0, 2024-06-10T03:04:17.027 INFO:teuthology.orchestra.run.smithi120.stdout: "handle_client_cap_release": 0, 2024-06-10T03:04:17.027 INFO:teuthology.orchestra.run.smithi120.stdout: "process_request_cap_release": 0 2024-06-10T03:04:17.027 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:17.027 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_cache": { 2024-06-10T03:04:17.027 INFO:teuthology.orchestra.run.smithi120.stdout: "dir_update": 0, 2024-06-10T03:04:17.027 INFO:teuthology.orchestra.run.smithi120.stdout: "dir_update_receipt": 0, 2024-06-10T03:04:17.027 INFO:teuthology.orchestra.run.smithi120.stdout: "dir_try_discover": 0, 2024-06-10T03:04:17.027 INFO:teuthology.orchestra.run.smithi120.stdout: "dir_send_discover": 0, 2024-06-10T03:04:17.027 INFO:teuthology.orchestra.run.smithi120.stdout: "dir_handle_discover": 0, 2024-06-10T03:04:17.027 INFO:teuthology.orchestra.run.smithi120.stdout: "num_strays": 0, 2024-06-10T03:04:17.027 INFO:teuthology.orchestra.run.smithi120.stdout: "num_strays_delayed": 0, 2024-06-10T03:04:17.027 INFO:teuthology.orchestra.run.smithi120.stdout: "num_strays_enqueuing": 0, 2024-06-10T03:04:17.027 INFO:teuthology.orchestra.run.smithi120.stdout: "strays_created": 0, 2024-06-10T03:04:17.027 INFO:teuthology.orchestra.run.smithi120.stdout: "strays_enqueued": 0, 2024-06-10T03:04:17.028 INFO:teuthology.orchestra.run.smithi120.stdout: "strays_reintegrated": 0, 2024-06-10T03:04:17.028 INFO:teuthology.orchestra.run.smithi120.stdout: "strays_migrated": 0, 2024-06-10T03:04:17.028 INFO:teuthology.orchestra.run.smithi120.stdout: "num_recovering_processing": 0, 2024-06-10T03:04:17.028 INFO:teuthology.orchestra.run.smithi120.stdout: "num_recovering_enqueued": 0, 2024-06-10T03:04:17.028 INFO:teuthology.orchestra.run.smithi120.stdout: "num_recovering_prioritized": 0, 2024-06-10T03:04:17.028 INFO:teuthology.orchestra.run.smithi120.stdout: "recovery_started": 0, 2024-06-10T03:04:17.028 INFO:teuthology.orchestra.run.smithi120.stdout: "recovery_completed": 0, 2024-06-10T03:04:17.028 INFO:teuthology.orchestra.run.smithi120.stdout: "ireq_quiesce_path": 0, 2024-06-10T03:04:17.028 INFO:teuthology.orchestra.run.smithi120.stdout: "ireq_quiesce_inode": 0, 2024-06-10T03:04:17.028 INFO:teuthology.orchestra.run.smithi120.stdout: "ireq_enqueue_scrub": 0, 2024-06-10T03:04:17.028 INFO:teuthology.orchestra.run.smithi120.stdout: "ireq_exportdir": 0, 2024-06-10T03:04:17.028 INFO:teuthology.orchestra.run.smithi120.stdout: "ireq_flush": 0, 2024-06-10T03:04:17.028 INFO:teuthology.orchestra.run.smithi120.stdout: "ireq_fragmentdir": 0, 2024-06-10T03:04:17.028 INFO:teuthology.orchestra.run.smithi120.stdout: "ireq_fragstats": 0, 2024-06-10T03:04:17.028 INFO:teuthology.orchestra.run.smithi120.stdout: "ireq_inodestats": 0 2024-06-10T03:04:17.029 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:17.029 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_log": { 2024-06-10T03:04:17.029 INFO:teuthology.orchestra.run.smithi120.stdout: "evlrg": 0, 2024-06-10T03:04:17.029 INFO:teuthology.orchestra.run.smithi120.stdout: "evadd": 0, 2024-06-10T03:04:17.029 INFO:teuthology.orchestra.run.smithi120.stdout: "evex": 0, 2024-06-10T03:04:17.029 INFO:teuthology.orchestra.run.smithi120.stdout: "evtrm": 0, 2024-06-10T03:04:17.029 INFO:teuthology.orchestra.run.smithi120.stdout: "ev": 4935, 2024-06-10T03:04:17.029 INFO:teuthology.orchestra.run.smithi120.stdout: "evexg": 0, 2024-06-10T03:04:17.029 INFO:teuthology.orchestra.run.smithi120.stdout: "evexd": 0, 2024-06-10T03:04:17.029 INFO:teuthology.orchestra.run.smithi120.stdout: "segadd": 0, 2024-06-10T03:04:17.029 INFO:teuthology.orchestra.run.smithi120.stdout: "segex": 0, 2024-06-10T03:04:17.029 INFO:teuthology.orchestra.run.smithi120.stdout: "segtrm": 0, 2024-06-10T03:04:17.029 INFO:teuthology.orchestra.run.smithi120.stdout: "seg": 6, 2024-06-10T03:04:17.029 INFO:teuthology.orchestra.run.smithi120.stdout: "segmjr": 2, 2024-06-10T03:04:17.029 INFO:teuthology.orchestra.run.smithi120.stdout: "segexg": 0, 2024-06-10T03:04:17.030 INFO:teuthology.orchestra.run.smithi120.stdout: "segexd": 0, 2024-06-10T03:04:17.030 INFO:teuthology.orchestra.run.smithi120.stdout: "expos": 4194304, 2024-06-10T03:04:17.030 INFO:teuthology.orchestra.run.smithi120.stdout: "wrpos": 11503137, 2024-06-10T03:04:17.030 INFO:teuthology.orchestra.run.smithi120.stdout: "rdpos": 11501999, 2024-06-10T03:04:17.030 INFO:teuthology.orchestra.run.smithi120.stdout: "jlat": { 2024-06-10T03:04:17.030 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-10T03:04:17.030 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-10T03:04:17.030 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-10T03:04:17.030 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:17.030 INFO:teuthology.orchestra.run.smithi120.stdout: "replayed": 4935 2024-06-10T03:04:17.030 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:17.030 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_mem": { 2024-06-10T03:04:17.030 INFO:teuthology.orchestra.run.smithi120.stdout: "ino": 8, 2024-06-10T03:04:17.030 INFO:teuthology.orchestra.run.smithi120.stdout: "ino+": 928, 2024-06-10T03:04:17.031 INFO:teuthology.orchestra.run.smithi120.stdout: "ino-": 920, 2024-06-10T03:04:17.031 INFO:teuthology.orchestra.run.smithi120.stdout: "dir": 6, 2024-06-10T03:04:17.031 INFO:teuthology.orchestra.run.smithi120.stdout: "dir+": 57, 2024-06-10T03:04:17.031 INFO:teuthology.orchestra.run.smithi120.stdout: "dir-": 51, 2024-06-10T03:04:17.031 INFO:teuthology.orchestra.run.smithi120.stdout: "dn": 907, 2024-06-10T03:04:17.031 INFO:teuthology.orchestra.run.smithi120.stdout: "dn+": 2226, 2024-06-10T03:04:17.031 INFO:teuthology.orchestra.run.smithi120.stdout: "dn-": 1319, 2024-06-10T03:04:17.031 INFO:teuthology.orchestra.run.smithi120.stdout: "cap": 0, 2024-06-10T03:04:17.031 INFO:teuthology.orchestra.run.smithi120.stdout: "cap+": 0, 2024-06-10T03:04:17.031 INFO:teuthology.orchestra.run.smithi120.stdout: "cap-": 0, 2024-06-10T03:04:17.031 INFO:teuthology.orchestra.run.smithi120.stdout: "rss": 52792, 2024-06-10T03:04:17.031 INFO:teuthology.orchestra.run.smithi120.stdout: "heap": 215420 2024-06-10T03:04:17.031 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:17.031 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_server": { 2024-06-10T03:04:17.032 INFO:teuthology.orchestra.run.smithi120.stdout: "dispatch_client_request": 0, 2024-06-10T03:04:17.032 INFO:teuthology.orchestra.run.smithi120.stdout: "dispatch_server_request": 0, 2024-06-10T03:04:17.032 INFO:teuthology.orchestra.run.smithi120.stdout: "handle_client_request": 0, 2024-06-10T03:04:17.032 INFO:teuthology.orchestra.run.smithi120.stdout: "handle_client_session": 0, 2024-06-10T03:04:17.032 INFO:teuthology.orchestra.run.smithi120.stdout: "handle_peer_request": 0, 2024-06-10T03:04:17.032 INFO:teuthology.orchestra.run.smithi120.stdout: "req_create_latency": { 2024-06-10T03:04:17.032 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-10T03:04:17.032 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-10T03:04:17.032 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-10T03:04:17.032 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:17.032 INFO:teuthology.orchestra.run.smithi120.stdout: "req_getattr_latency": { 2024-06-10T03:04:17.032 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-10T03:04:17.032 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-10T03:04:17.032 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-10T03:04:17.032 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:17.033 INFO:teuthology.orchestra.run.smithi120.stdout: "req_getfilelock_latency": { 2024-06-10T03:04:17.033 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-10T03:04:17.033 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-10T03:04:17.033 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-10T03:04:17.033 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:17.033 INFO:teuthology.orchestra.run.smithi120.stdout: "req_link_latency": { 2024-06-10T03:04:17.033 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-10T03:04:17.033 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-10T03:04:17.033 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-10T03:04:17.033 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:17.033 INFO:teuthology.orchestra.run.smithi120.stdout: "req_lookup_latency": { 2024-06-10T03:04:17.033 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-10T03:04:17.033 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-10T03:04:17.033 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-10T03:04:17.033 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:17.034 INFO:teuthology.orchestra.run.smithi120.stdout: "req_lookuphash_latency": { 2024-06-10T03:04:17.034 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-10T03:04:17.034 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-10T03:04:17.034 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-10T03:04:17.034 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:17.034 INFO:teuthology.orchestra.run.smithi120.stdout: "req_lookupino_latency": { 2024-06-10T03:04:17.034 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-10T03:04:17.034 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-10T03:04:17.034 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-10T03:04:17.034 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:17.034 INFO:teuthology.orchestra.run.smithi120.stdout: "req_lookupname_latency": { 2024-06-10T03:04:17.034 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-10T03:04:17.034 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-10T03:04:17.034 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-10T03:04:17.035 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:17.035 INFO:teuthology.orchestra.run.smithi120.stdout: "req_lookupparent_latency": { 2024-06-10T03:04:17.035 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-10T03:04:17.035 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-10T03:04:17.035 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-10T03:04:17.035 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:17.035 INFO:teuthology.orchestra.run.smithi120.stdout: "req_lookupsnap_latency": { 2024-06-10T03:04:17.035 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-10T03:04:17.035 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-10T03:04:17.035 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-10T03:04:17.035 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:17.035 INFO:teuthology.orchestra.run.smithi120.stdout: "req_lssnap_latency": { 2024-06-10T03:04:17.035 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-10T03:04:17.035 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-10T03:04:17.036 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-10T03:04:17.036 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:17.036 INFO:teuthology.orchestra.run.smithi120.stdout: "req_mkdir_latency": { 2024-06-10T03:04:17.036 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-10T03:04:17.036 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-10T03:04:17.036 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-10T03:04:17.036 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:17.036 INFO:teuthology.orchestra.run.smithi120.stdout: "req_mknod_latency": { 2024-06-10T03:04:17.036 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-10T03:04:17.037 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-10T03:04:17.037 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-10T03:04:17.037 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:17.037 INFO:teuthology.orchestra.run.smithi120.stdout: "req_mksnap_latency": { 2024-06-10T03:04:17.037 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-10T03:04:17.037 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-10T03:04:17.037 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-10T03:04:17.037 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:17.037 INFO:teuthology.orchestra.run.smithi120.stdout: "req_open_latency": { 2024-06-10T03:04:17.037 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-10T03:04:17.037 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-10T03:04:17.037 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-10T03:04:17.037 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:17.037 INFO:teuthology.orchestra.run.smithi120.stdout: "req_readdir_latency": { 2024-06-10T03:04:17.037 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-10T03:04:17.038 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-10T03:04:17.038 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-10T03:04:17.038 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:17.038 INFO:teuthology.orchestra.run.smithi120.stdout: "req_rename_latency": { 2024-06-10T03:04:17.038 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-10T03:04:17.038 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-10T03:04:17.038 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-10T03:04:17.038 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:17.038 INFO:teuthology.orchestra.run.smithi120.stdout: "req_renamesnap_latency": { 2024-06-10T03:04:17.038 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-10T03:04:17.038 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-10T03:04:17.038 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-10T03:04:17.038 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:17.039 INFO:teuthology.orchestra.run.smithi120.stdout: "req_snapdiff_latency": { 2024-06-10T03:04:17.039 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-10T03:04:17.039 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-10T03:04:17.039 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-10T03:04:17.039 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:17.039 INFO:teuthology.orchestra.run.smithi120.stdout: "req_rmdir_latency": { 2024-06-10T03:04:17.039 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-10T03:04:17.039 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-10T03:04:17.039 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-10T03:04:17.039 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:17.039 INFO:teuthology.orchestra.run.smithi120.stdout: "req_rmsnap_latency": { 2024-06-10T03:04:17.039 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-10T03:04:17.039 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-10T03:04:17.039 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-10T03:04:17.039 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:17.040 INFO:teuthology.orchestra.run.smithi120.stdout: "req_rmxattr_latency": { 2024-06-10T03:04:17.040 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-10T03:04:17.040 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-10T03:04:17.040 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-10T03:04:17.040 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:17.040 INFO:teuthology.orchestra.run.smithi120.stdout: "req_setattr_latency": { 2024-06-10T03:04:17.040 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-10T03:04:17.040 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-10T03:04:17.040 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-10T03:04:17.040 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:17.040 INFO:teuthology.orchestra.run.smithi120.stdout: "req_setdirlayout_latency": { 2024-06-10T03:04:17.040 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-10T03:04:17.040 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-10T03:04:17.040 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-10T03:04:17.041 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:17.041 INFO:teuthology.orchestra.run.smithi120.stdout: "req_setfilelock_latency": { 2024-06-10T03:04:17.041 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-10T03:04:17.041 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-10T03:04:17.041 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-10T03:04:17.041 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:17.041 INFO:teuthology.orchestra.run.smithi120.stdout: "req_setlayout_latency": { 2024-06-10T03:04:17.041 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-10T03:04:17.041 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-10T03:04:17.041 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-10T03:04:17.041 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:17.041 INFO:teuthology.orchestra.run.smithi120.stdout: "req_setxattr_latency": { 2024-06-10T03:04:17.041 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-10T03:04:17.041 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-10T03:04:17.041 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-10T03:04:17.042 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:17.042 INFO:teuthology.orchestra.run.smithi120.stdout: "req_symlink_latency": { 2024-06-10T03:04:17.042 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-10T03:04:17.042 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-10T03:04:17.042 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-10T03:04:17.042 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:17.042 INFO:teuthology.orchestra.run.smithi120.stdout: "req_unlink_latency": { 2024-06-10T03:04:17.042 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-10T03:04:17.042 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-10T03:04:17.042 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-10T03:04:17.042 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:17.042 INFO:teuthology.orchestra.run.smithi120.stdout: "cap_revoke_eviction": 0, 2024-06-10T03:04:17.042 INFO:teuthology.orchestra.run.smithi120.stdout: "cap_acquisition_throttle": 0, 2024-06-10T03:04:17.042 INFO:teuthology.orchestra.run.smithi120.stdout: "req_getvxattr_latency": { 2024-06-10T03:04:17.043 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-10T03:04:17.043 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-10T03:04:17.043 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-10T03:04:17.043 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-06-10T03:04:17.043 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:17.043 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_sessions": { 2024-06-10T03:04:17.043 INFO:teuthology.orchestra.run.smithi120.stdout: "session_count": 1, 2024-06-10T03:04:17.043 INFO:teuthology.orchestra.run.smithi120.stdout: "session_add": 2, 2024-06-10T03:04:17.043 INFO:teuthology.orchestra.run.smithi120.stdout: "session_remove": 1, 2024-06-10T03:04:17.043 INFO:teuthology.orchestra.run.smithi120.stdout: "sessions_open": 1, 2024-06-10T03:04:17.043 INFO:teuthology.orchestra.run.smithi120.stdout: "sessions_stale": 0, 2024-06-10T03:04:17.043 INFO:teuthology.orchestra.run.smithi120.stdout: "total_load": 0, 2024-06-10T03:04:17.043 INFO:teuthology.orchestra.run.smithi120.stdout: "average_load": 0, 2024-06-10T03:04:17.043 INFO:teuthology.orchestra.run.smithi120.stdout: "avg_session_uptime": 2340, 2024-06-10T03:04:17.044 INFO:teuthology.orchestra.run.smithi120.stdout: "mdthresh_evicted": 0 2024-06-10T03:04:17.044 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:17.044 INFO:teuthology.orchestra.run.smithi120.stdout: "mempool": { 2024-06-10T03:04:17.044 INFO:teuthology.orchestra.run.smithi120.stdout: "bloom_filter_bytes": 0, 2024-06-10T03:04:17.044 INFO:teuthology.orchestra.run.smithi120.stdout: "bloom_filter_items": 0, 2024-06-10T03:04:17.044 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_alloc_bytes": 0, 2024-06-10T03:04:17.044 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_alloc_items": 0, 2024-06-10T03:04:17.044 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_data_bytes": 0, 2024-06-10T03:04:17.044 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_data_items": 0, 2024-06-10T03:04:17.044 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-10T03:04:17.044 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_onode_items": 0, 2024-06-10T03:04:17.044 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-10T03:04:17.044 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_meta_items": 0, 2024-06-10T03:04:17.044 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_other_bytes": 0, 2024-06-10T03:04:17.045 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_other_items": 0, 2024-06-10T03:04:17.045 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-10T03:04:17.045 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_buffer_items": 0, 2024-06-10T03:04:17.045 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_extent_bytes": 0, 2024-06-10T03:04:17.045 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_extent_items": 0, 2024-06-10T03:04:17.045 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_blob_bytes": 0, 2024-06-10T03:04:17.045 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_blob_items": 0, 2024-06-10T03:04:17.045 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-10T03:04:17.045 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_shared_blob_items": 0, 2024-06-10T03:04:17.045 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-10T03:04:17.045 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_inline_bl_items": 0, 2024-06-10T03:04:17.045 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_fsck_bytes": 0, 2024-06-10T03:04:17.045 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_fsck_items": 0, 2024-06-10T03:04:17.045 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_txc_bytes": 0, 2024-06-10T03:04:17.046 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_txc_items": 0, 2024-06-10T03:04:17.046 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-10T03:04:17.046 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_writing_deferred_items": 0, 2024-06-10T03:04:17.046 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_writing_bytes": 0, 2024-06-10T03:04:17.046 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_writing_items": 0, 2024-06-10T03:04:17.046 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_bytes": 0, 2024-06-10T03:04:17.046 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_items": 0, 2024-06-10T03:04:17.046 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_file_reader_bytes": 0, 2024-06-10T03:04:17.046 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_file_reader_items": 0, 2024-06-10T03:04:17.046 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_file_writer_bytes": 0, 2024-06-10T03:04:17.046 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_file_writer_items": 0, 2024-06-10T03:04:17.046 INFO:teuthology.orchestra.run.smithi120.stdout: "buffer_anon_bytes": 181303, 2024-06-10T03:04:17.046 INFO:teuthology.orchestra.run.smithi120.stdout: "buffer_anon_items": 46, 2024-06-10T03:04:17.047 INFO:teuthology.orchestra.run.smithi120.stdout: "buffer_meta_bytes": 0, 2024-06-10T03:04:17.047 INFO:teuthology.orchestra.run.smithi120.stdout: "buffer_meta_items": 0, 2024-06-10T03:04:17.047 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_bytes": 0, 2024-06-10T03:04:17.047 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_items": 0, 2024-06-10T03:04:17.047 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_mapbl_bytes": 0, 2024-06-10T03:04:17.047 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_mapbl_items": 0, 2024-06-10T03:04:17.047 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_pglog_bytes": 0, 2024-06-10T03:04:17.047 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_pglog_items": 0, 2024-06-10T03:04:17.047 INFO:teuthology.orchestra.run.smithi120.stdout: "osdmap_bytes": 8984, 2024-06-10T03:04:17.047 INFO:teuthology.orchestra.run.smithi120.stdout: "osdmap_items": 172, 2024-06-10T03:04:17.047 INFO:teuthology.orchestra.run.smithi120.stdout: "osdmap_mapping_bytes": 0, 2024-06-10T03:04:17.047 INFO:teuthology.orchestra.run.smithi120.stdout: "osdmap_mapping_items": 0, 2024-06-10T03:04:17.047 INFO:teuthology.orchestra.run.smithi120.stdout: "pgmap_bytes": 0, 2024-06-10T03:04:17.047 INFO:teuthology.orchestra.run.smithi120.stdout: "pgmap_items": 0, 2024-06-10T03:04:17.048 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_co_bytes": 664472, 2024-06-10T03:04:17.048 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_co_items": 3701, 2024-06-10T03:04:17.048 INFO:teuthology.orchestra.run.smithi120.stdout: "unittest_1_bytes": 0, 2024-06-10T03:04:17.048 INFO:teuthology.orchestra.run.smithi120.stdout: "unittest_1_items": 0, 2024-06-10T03:04:17.048 INFO:teuthology.orchestra.run.smithi120.stdout: "unittest_2_bytes": 0, 2024-06-10T03:04:17.048 INFO:teuthology.orchestra.run.smithi120.stdout: "unittest_2_items": 0 2024-06-10T03:04:17.048 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:17.048 INFO:teuthology.orchestra.run.smithi120.stdout: "objecter": { 2024-06-10T03:04:17.048 INFO:teuthology.orchestra.run.smithi120.stdout: "op_active": 0, 2024-06-10T03:04:17.048 INFO:teuthology.orchestra.run.smithi120.stdout: "op_laggy": 0, 2024-06-10T03:04:17.048 INFO:teuthology.orchestra.run.smithi120.stdout: "op_send": 2366, 2024-06-10T03:04:17.048 INFO:teuthology.orchestra.run.smithi120.stdout: "op_send_bytes": 21, 2024-06-10T03:04:17.048 INFO:teuthology.orchestra.run.smithi120.stdout: "op_resend": 3, 2024-06-10T03:04:17.048 INFO:teuthology.orchestra.run.smithi120.stdout: "op_reply": 2366, 2024-06-10T03:04:17.049 INFO:teuthology.orchestra.run.smithi120.stdout: "op_latency": { 2024-06-10T03:04:17.049 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 2366, 2024-06-10T03:04:17.049 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 6.555853013, 2024-06-10T03:04:17.049 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.002770859 2024-06-10T03:04:17.049 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:17.049 INFO:teuthology.orchestra.run.smithi120.stdout: "op_inflight": 0, 2024-06-10T03:04:17.049 INFO:teuthology.orchestra.run.smithi120.stdout: "oplen_avg": { 2024-06-10T03:04:17.049 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 2366, 2024-06-10T03:04:17.049 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 2367 2024-06-10T03:04:17.049 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:17.049 INFO:teuthology.orchestra.run.smithi120.stdout: "op": 2366, 2024-06-10T03:04:17.049 INFO:teuthology.orchestra.run.smithi120.stdout: "op_r": 2366, 2024-06-10T03:04:17.049 INFO:teuthology.orchestra.run.smithi120.stdout: "op_w": 0, 2024-06-10T03:04:17.050 INFO:teuthology.orchestra.run.smithi120.stdout: "op_rmw": 0, 2024-06-10T03:04:17.050 INFO:teuthology.orchestra.run.smithi120.stdout: "op_pg": 0, 2024-06-10T03:04:17.050 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_stat": 1472, 2024-06-10T03:04:17.050 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_create": 0, 2024-06-10T03:04:17.050 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_read": 892, 2024-06-10T03:04:17.050 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_write": 0, 2024-06-10T03:04:17.050 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_writefull": 0, 2024-06-10T03:04:17.050 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_writesame": 0, 2024-06-10T03:04:17.050 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_append": 0, 2024-06-10T03:04:17.050 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_zero": 0, 2024-06-10T03:04:17.050 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_truncate": 0, 2024-06-10T03:04:17.050 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_delete": 0, 2024-06-10T03:04:17.050 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_mapext": 0, 2024-06-10T03:04:17.051 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_sparse_read": 0, 2024-06-10T03:04:17.051 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_clonerange": 0, 2024-06-10T03:04:17.051 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_getxattr": 1, 2024-06-10T03:04:17.051 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_setxattr": 0, 2024-06-10T03:04:17.051 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_cmpxattr": 0, 2024-06-10T03:04:17.051 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_rmxattr": 0, 2024-06-10T03:04:17.051 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_resetxattrs": 0, 2024-06-10T03:04:17.051 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_call": 0, 2024-06-10T03:04:17.051 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_watch": 0, 2024-06-10T03:04:17.051 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_notify": 0, 2024-06-10T03:04:17.051 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_src_cmpxattr": 0, 2024-06-10T03:04:17.051 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_pgls": 0, 2024-06-10T03:04:17.051 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_pgls_filter": 0, 2024-06-10T03:04:17.052 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_other": 0, 2024-06-10T03:04:17.052 INFO:teuthology.orchestra.run.smithi120.stdout: "linger_active": 0, 2024-06-10T03:04:17.052 INFO:teuthology.orchestra.run.smithi120.stdout: "linger_send": 0, 2024-06-10T03:04:17.052 INFO:teuthology.orchestra.run.smithi120.stdout: "linger_resend": 0, 2024-06-10T03:04:17.052 INFO:teuthology.orchestra.run.smithi120.stdout: "linger_ping": 0, 2024-06-10T03:04:17.052 INFO:teuthology.orchestra.run.smithi120.stdout: "poolop_active": 0, 2024-06-10T03:04:17.052 INFO:teuthology.orchestra.run.smithi120.stdout: "poolop_send": 0, 2024-06-10T03:04:17.052 INFO:teuthology.orchestra.run.smithi120.stdout: "poolop_resend": 0, 2024-06-10T03:04:17.052 INFO:teuthology.orchestra.run.smithi120.stdout: "poolstat_active": 0, 2024-06-10T03:04:17.052 INFO:teuthology.orchestra.run.smithi120.stdout: "poolstat_send": 0, 2024-06-10T03:04:17.052 INFO:teuthology.orchestra.run.smithi120.stdout: "poolstat_resend": 0, 2024-06-10T03:04:17.052 INFO:teuthology.orchestra.run.smithi120.stdout: "statfs_active": 0, 2024-06-10T03:04:17.052 INFO:teuthology.orchestra.run.smithi120.stdout: "statfs_send": 0, 2024-06-10T03:04:17.052 INFO:teuthology.orchestra.run.smithi120.stdout: "statfs_resend": 0, 2024-06-10T03:04:17.052 INFO:teuthology.orchestra.run.smithi120.stdout: "command_active": 0, 2024-06-10T03:04:17.053 INFO:teuthology.orchestra.run.smithi120.stdout: "command_send": 0, 2024-06-10T03:04:17.053 INFO:teuthology.orchestra.run.smithi120.stdout: "command_resend": 0, 2024-06-10T03:04:17.053 INFO:teuthology.orchestra.run.smithi120.stdout: "map_epoch": 100, 2024-06-10T03:04:17.053 INFO:teuthology.orchestra.run.smithi120.stdout: "map_full": 0, 2024-06-10T03:04:17.053 INFO:teuthology.orchestra.run.smithi120.stdout: "map_inc": 2, 2024-06-10T03:04:17.053 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_sessions": 4, 2024-06-10T03:04:17.053 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_session_open": 4, 2024-06-10T03:04:17.053 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_session_close": 0, 2024-06-10T03:04:17.053 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_laggy": 0, 2024-06-10T03:04:17.053 INFO:teuthology.orchestra.run.smithi120.stdout: "omap_wr": 0, 2024-06-10T03:04:17.053 INFO:teuthology.orchestra.run.smithi120.stdout: "omap_rd": 2, 2024-06-10T03:04:17.053 INFO:teuthology.orchestra.run.smithi120.stdout: "omap_del": 0 2024-06-10T03:04:17.053 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:17.053 INFO:teuthology.orchestra.run.smithi120.stdout: "oft": { 2024-06-10T03:04:17.054 INFO:teuthology.orchestra.run.smithi120.stdout: "omap_total_objs": 0, 2024-06-10T03:04:17.054 INFO:teuthology.orchestra.run.smithi120.stdout: "omap_total_kv_pairs": 0, 2024-06-10T03:04:17.054 INFO:teuthology.orchestra.run.smithi120.stdout: "omap_total_updates": 0, 2024-06-10T03:04:17.054 INFO:teuthology.orchestra.run.smithi120.stdout: "omap_total_removes": 0 2024-06-10T03:04:17.054 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:17.054 INFO:teuthology.orchestra.run.smithi120.stdout: "osd-slow-ops": { 2024-06-10T03:04:17.054 INFO:teuthology.orchestra.run.smithi120.stdout: "slow_ops_count": 0 2024-06-10T03:04:17.054 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:17.054 INFO:teuthology.orchestra.run.smithi120.stdout: "purge_queue": { 2024-06-10T03:04:17.054 INFO:teuthology.orchestra.run.smithi120.stdout: "pq_executing_ops": 0, 2024-06-10T03:04:17.054 INFO:teuthology.orchestra.run.smithi120.stdout: "pq_executing_ops_high_water": 0, 2024-06-10T03:04:17.054 INFO:teuthology.orchestra.run.smithi120.stdout: "pq_executing": 0, 2024-06-10T03:04:17.054 INFO:teuthology.orchestra.run.smithi120.stdout: "pq_executing_high_water": 0, 2024-06-10T03:04:17.055 INFO:teuthology.orchestra.run.smithi120.stdout: "pq_executed_ops": 0, 2024-06-10T03:04:17.055 INFO:teuthology.orchestra.run.smithi120.stdout: "pq_executed": 0, 2024-06-10T03:04:17.055 INFO:teuthology.orchestra.run.smithi120.stdout: "pq_item_in_journal": 0 2024-06-10T03:04:17.055 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:17.055 INFO:teuthology.orchestra.run.smithi120.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-10T03:04:17.055 INFO:teuthology.orchestra.run.smithi120.stdout: "val": 0, 2024-06-10T03:04:17.055 INFO:teuthology.orchestra.run.smithi120.stdout: "max": 104857600, 2024-06-10T03:04:17.055 INFO:teuthology.orchestra.run.smithi120.stdout: "get_started": 0, 2024-06-10T03:04:17.055 INFO:teuthology.orchestra.run.smithi120.stdout: "get": 2637, 2024-06-10T03:04:17.055 INFO:teuthology.orchestra.run.smithi120.stdout: "get_sum": 8630332, 2024-06-10T03:04:17.055 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_fail": 0, 2024-06-10T03:04:17.055 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_success": 2637, 2024-06-10T03:04:17.055 INFO:teuthology.orchestra.run.smithi120.stdout: "take": 0, 2024-06-10T03:04:17.055 INFO:teuthology.orchestra.run.smithi120.stdout: "take_sum": 0, 2024-06-10T03:04:17.056 INFO:teuthology.orchestra.run.smithi120.stdout: "put": 2637, 2024-06-10T03:04:17.056 INFO:teuthology.orchestra.run.smithi120.stdout: "put_sum": 8630332, 2024-06-10T03:04:17.056 INFO:teuthology.orchestra.run.smithi120.stdout: "wait": { 2024-06-10T03:04:17.056 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-10T03:04:17.056 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-10T03:04:17.056 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-10T03:04:17.056 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-06-10T03:04:17.056 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:17.056 INFO:teuthology.orchestra.run.smithi120.stdout: "throttle-objecter_bytes": { 2024-06-10T03:04:17.056 INFO:teuthology.orchestra.run.smithi120.stdout: "val": 0, 2024-06-10T03:04:17.056 INFO:teuthology.orchestra.run.smithi120.stdout: "max": 104857600, 2024-06-10T03:04:17.056 INFO:teuthology.orchestra.run.smithi120.stdout: "get_started": 0, 2024-06-10T03:04:17.056 INFO:teuthology.orchestra.run.smithi120.stdout: "get": 0, 2024-06-10T03:04:17.056 INFO:teuthology.orchestra.run.smithi120.stdout: "get_sum": 0, 2024-06-10T03:04:17.057 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_fail": 0, 2024-06-10T03:04:17.057 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_success": 0, 2024-06-10T03:04:17.057 INFO:teuthology.orchestra.run.smithi120.stdout: "take": 2366, 2024-06-10T03:04:17.057 INFO:teuthology.orchestra.run.smithi120.stdout: "take_sum": 7308838, 2024-06-10T03:04:17.057 INFO:teuthology.orchestra.run.smithi120.stdout: "put": 2366, 2024-06-10T03:04:17.057 INFO:teuthology.orchestra.run.smithi120.stdout: "put_sum": 7308838, 2024-06-10T03:04:17.057 INFO:teuthology.orchestra.run.smithi120.stdout: "wait": { 2024-06-10T03:04:17.057 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-10T03:04:17.057 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-10T03:04:17.057 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-10T03:04:17.057 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-06-10T03:04:17.057 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:17.057 INFO:teuthology.orchestra.run.smithi120.stdout: "throttle-objecter_ops": { 2024-06-10T03:04:17.057 INFO:teuthology.orchestra.run.smithi120.stdout: "val": 0, 2024-06-10T03:04:17.057 INFO:teuthology.orchestra.run.smithi120.stdout: "max": 1024, 2024-06-10T03:04:17.058 INFO:teuthology.orchestra.run.smithi120.stdout: "get_started": 0, 2024-06-10T03:04:17.058 INFO:teuthology.orchestra.run.smithi120.stdout: "get": 0, 2024-06-10T03:04:17.058 INFO:teuthology.orchestra.run.smithi120.stdout: "get_sum": 0, 2024-06-10T03:04:17.058 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_fail": 0, 2024-06-10T03:04:17.058 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_success": 0, 2024-06-10T03:04:17.058 INFO:teuthology.orchestra.run.smithi120.stdout: "take": 2366, 2024-06-10T03:04:17.058 INFO:teuthology.orchestra.run.smithi120.stdout: "take_sum": 2366, 2024-06-10T03:04:17.058 INFO:teuthology.orchestra.run.smithi120.stdout: "put": 2366, 2024-06-10T03:04:17.058 INFO:teuthology.orchestra.run.smithi120.stdout: "put_sum": 2366, 2024-06-10T03:04:17.058 INFO:teuthology.orchestra.run.smithi120.stdout: "wait": { 2024-06-10T03:04:17.058 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-10T03:04:17.058 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-10T03:04:17.058 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-10T03:04:17.058 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-06-10T03:04:17.059 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:17.059 INFO:teuthology.orchestra.run.smithi120.stdout: "throttle-write_buf_throttle": { 2024-06-10T03:04:17.059 INFO:teuthology.orchestra.run.smithi120.stdout: "val": 0, 2024-06-10T03:04:17.059 INFO:teuthology.orchestra.run.smithi120.stdout: "max": 3758096384, 2024-06-10T03:04:17.059 INFO:teuthology.orchestra.run.smithi120.stdout: "get_started": 0, 2024-06-10T03:04:17.059 INFO:teuthology.orchestra.run.smithi120.stdout: "get": 0, 2024-06-10T03:04:17.059 INFO:teuthology.orchestra.run.smithi120.stdout: "get_sum": 0, 2024-06-10T03:04:17.059 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_fail": 0, 2024-06-10T03:04:17.059 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_success": 0, 2024-06-10T03:04:17.059 INFO:teuthology.orchestra.run.smithi120.stdout: "take": 0, 2024-06-10T03:04:17.059 INFO:teuthology.orchestra.run.smithi120.stdout: "take_sum": 0, 2024-06-10T03:04:17.059 INFO:teuthology.orchestra.run.smithi120.stdout: "put": 0, 2024-06-10T03:04:17.059 INFO:teuthology.orchestra.run.smithi120.stdout: "put_sum": 0, 2024-06-10T03:04:17.059 INFO:teuthology.orchestra.run.smithi120.stdout: "wait": { 2024-06-10T03:04:17.060 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-10T03:04:17.060 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-10T03:04:17.060 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-10T03:04:17.060 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-06-10T03:04:17.060 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:17.060 INFO:teuthology.orchestra.run.smithi120.stdout: "throttle-write_buf_throttle-0x55792a5f06e0": { 2024-06-10T03:04:17.060 INFO:teuthology.orchestra.run.smithi120.stdout: "val": 0, 2024-06-10T03:04:17.060 INFO:teuthology.orchestra.run.smithi120.stdout: "max": 3758096384, 2024-06-10T03:04:17.060 INFO:teuthology.orchestra.run.smithi120.stdout: "get_started": 0, 2024-06-10T03:04:17.060 INFO:teuthology.orchestra.run.smithi120.stdout: "get": 0, 2024-06-10T03:04:17.060 INFO:teuthology.orchestra.run.smithi120.stdout: "get_sum": 0, 2024-06-10T03:04:17.060 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_fail": 0, 2024-06-10T03:04:17.060 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_success": 0, 2024-06-10T03:04:17.060 INFO:teuthology.orchestra.run.smithi120.stdout: "take": 0, 2024-06-10T03:04:17.060 INFO:teuthology.orchestra.run.smithi120.stdout: "take_sum": 0, 2024-06-10T03:04:17.061 INFO:teuthology.orchestra.run.smithi120.stdout: "put": 0, 2024-06-10T03:04:17.061 INFO:teuthology.orchestra.run.smithi120.stdout: "put_sum": 0, 2024-06-10T03:04:17.061 INFO:teuthology.orchestra.run.smithi120.stdout: "wait": { 2024-06-10T03:04:17.061 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-10T03:04:17.061 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-10T03:04:17.061 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-10T03:04:17.061 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-06-10T03:04:17.061 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-06-10T03:04:17.061 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-06-10T03:04:17.204 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:04:16 smithi188 ceph-mon[36599]: pgmap v736: 129 pgs: 129 active+clean; 228 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-06-10T03:04:17.227 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:04:16 smithi052 ceph-mon[32362]: pgmap v736: 129 pgs: 129 active+clean; 228 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-06-10T03:04:17.948 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-06-10T03:04:17.949 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-06-10T03:04:17.949 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update=0 2024-06-10T03:04:17.949 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update_receipt=0 2024-06-10T03:04:17.949 INFO:tasks.check_counter:Daemon mds.e mds.dir_split=0 2024-06-10T03:04:17.949 DEBUG:tasks.check_counter:Getting stats from h 2024-06-10T03:04:17.949 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph daemon mds.h perf dump 2024-06-10T03:04:18.186 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.b/config 2024-06-10T03:04:19.204 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:04:18 smithi188 ceph-mon[36599]: pgmap v737: 129 pgs: 129 active+clean; 228 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-06-10T03:04:19.227 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:04:18 smithi052 ceph-mon[32362]: pgmap v737: 129 pgs: 129 active+clean; 228 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-06-10T03:04:19.314 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:04:18 smithi120 ceph-mon[36634]: pgmap v737: 129 pgs: 129 active+clean; 228 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-06-10T03:04:19.912 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-06-10T03:04:19.912 INFO:teuthology.orchestra.run.smithi120.stdout: "AsyncMessenger::Worker-0": { 2024-06-10T03:04:19.913 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_messages": 40334, 2024-06-10T03:04:19.913 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_messages": 47045, 2024-06-10T03:04:19.913 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_bytes": 15939583, 2024-06-10T03:04:19.913 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_bytes": 30250950, 2024-06-10T03:04:19.913 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_created_connections": 21, 2024-06-10T03:04:19.913 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_active_connections": 6, 2024-06-10T03:04:19.913 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_total_time": 1.955288235, 2024-06-10T03:04:19.913 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_send_time": 0.857656993, 2024-06-10T03:04:19.913 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_recv_time": 1.545005669, 2024-06-10T03:04:19.913 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_fast_dispatch_time": 0.035645944, 2024-06-10T03:04:19.913 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_messages_queue_lat": { 2024-06-10T03:04:19.913 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 47041, 2024-06-10T03:04:19.913 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 1.692991053, 2024-06-10T03:04:19.913 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000035989 2024-06-10T03:04:19.914 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:19.914 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_handle_ack_lat": { 2024-06-10T03:04:19.914 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 77530, 2024-06-10T03:04:19.914 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.010524979, 2024-06-10T03:04:19.914 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000135 2024-06-10T03:04:19.914 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:19.914 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_encrypted_bytes": 9648752, 2024-06-10T03:04:19.914 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_encrypted_bytes": 21801552 2024-06-10T03:04:19.914 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:19.914 INFO:teuthology.orchestra.run.smithi120.stdout: "AsyncMessenger::Worker-1": { 2024-06-10T03:04:19.914 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_messages": 3635, 2024-06-10T03:04:19.914 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_messages": 3718, 2024-06-10T03:04:19.914 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_bytes": 1053708, 2024-06-10T03:04:19.915 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_bytes": 1736822, 2024-06-10T03:04:19.915 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_created_connections": 9, 2024-06-10T03:04:19.915 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_active_connections": 6, 2024-06-10T03:04:19.915 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_total_time": 0.250461683, 2024-06-10T03:04:19.915 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_send_time": 0.092464749, 2024-06-10T03:04:19.915 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_recv_time": 0.114211154, 2024-06-10T03:04:19.915 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_fast_dispatch_time": 0.046261678, 2024-06-10T03:04:19.915 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_messages_queue_lat": { 2024-06-10T03:04:19.915 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 3719, 2024-06-10T03:04:19.915 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.122987485, 2024-06-10T03:04:19.915 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000033070 2024-06-10T03:04:19.915 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:19.915 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_handle_ack_lat": { 2024-06-10T03:04:19.916 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 3309, 2024-06-10T03:04:19.916 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000611658, 2024-06-10T03:04:19.916 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000184 2024-06-10T03:04:19.916 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:19.916 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_encrypted_bytes": 3232, 2024-06-10T03:04:19.916 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_encrypted_bytes": 377296 2024-06-10T03:04:19.916 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:19.916 INFO:teuthology.orchestra.run.smithi120.stdout: "AsyncMessenger::Worker-2": { 2024-06-10T03:04:19.916 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_messages": 2897, 2024-06-10T03:04:19.916 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_messages": 2799, 2024-06-10T03:04:19.916 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_bytes": 1649759, 2024-06-10T03:04:19.916 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_bytes": 6995673, 2024-06-10T03:04:19.916 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_created_connections": 21, 2024-06-10T03:04:19.916 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_active_connections": 8, 2024-06-10T03:04:19.917 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_total_time": 0.212269209, 2024-06-10T03:04:19.917 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_send_time": 0.072479051, 2024-06-10T03:04:19.917 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_recv_time": 0.086080442, 2024-06-10T03:04:19.917 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_fast_dispatch_time": 0.053798339, 2024-06-10T03:04:19.917 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_messages_queue_lat": { 2024-06-10T03:04:19.917 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 2798, 2024-06-10T03:04:19.917 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.118372661, 2024-06-10T03:04:19.917 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000042306 2024-06-10T03:04:19.917 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:19.917 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_handle_ack_lat": { 2024-06-10T03:04:19.917 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 1161, 2024-06-10T03:04:19.917 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000237077, 2024-06-10T03:04:19.917 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000204 2024-06-10T03:04:19.918 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:19.918 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_encrypted_bytes": 919376, 2024-06-10T03:04:19.918 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_encrypted_bytes": 181728 2024-06-10T03:04:19.918 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:19.918 INFO:teuthology.orchestra.run.smithi120.stdout: "cct": { 2024-06-10T03:04:19.918 INFO:teuthology.orchestra.run.smithi120.stdout: "total_workers": 1, 2024-06-10T03:04:19.918 INFO:teuthology.orchestra.run.smithi120.stdout: "unhealthy_workers": 0 2024-06-10T03:04:19.918 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:19.918 INFO:teuthology.orchestra.run.smithi120.stdout: "finisher-MDSRank": { 2024-06-10T03:04:19.918 INFO:teuthology.orchestra.run.smithi120.stdout: "queue_len": 0, 2024-06-10T03:04:19.918 INFO:teuthology.orchestra.run.smithi120.stdout: "complete_latency": { 2024-06-10T03:04:19.918 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 2168, 2024-06-10T03:04:19.918 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 2.653358866, 2024-06-10T03:04:19.919 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.001223874 2024-06-10T03:04:19.919 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-06-10T03:04:19.919 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:19.919 INFO:teuthology.orchestra.run.smithi120.stdout: "finisher-PurgeQueue": { 2024-06-10T03:04:19.919 INFO:teuthology.orchestra.run.smithi120.stdout: "queue_len": 0, 2024-06-10T03:04:19.919 INFO:teuthology.orchestra.run.smithi120.stdout: "complete_latency": { 2024-06-10T03:04:19.919 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 2661, 2024-06-10T03:04:19.919 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.216779495, 2024-06-10T03:04:19.919 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000081465 2024-06-10T03:04:19.919 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-06-10T03:04:19.919 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:19.919 INFO:teuthology.orchestra.run.smithi120.stdout: "mds": { 2024-06-10T03:04:19.919 INFO:teuthology.orchestra.run.smithi120.stdout: "request": 18662, 2024-06-10T03:04:19.919 INFO:teuthology.orchestra.run.smithi120.stdout: "reply": 18648, 2024-06-10T03:04:19.920 INFO:teuthology.orchestra.run.smithi120.stdout: "reply_latency": { 2024-06-10T03:04:19.920 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 18648, 2024-06-10T03:04:19.920 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 264.971519734, 2024-06-10T03:04:19.920 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.014209111 2024-06-10T03:04:19.920 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:19.920 INFO:teuthology.orchestra.run.smithi120.stdout: "slow_reply": 0, 2024-06-10T03:04:19.920 INFO:teuthology.orchestra.run.smithi120.stdout: "forward": 14, 2024-06-10T03:04:19.920 INFO:teuthology.orchestra.run.smithi120.stdout: "dir_fetch_complete": 0, 2024-06-10T03:04:19.920 INFO:teuthology.orchestra.run.smithi120.stdout: "dir_fetch_keys": 0, 2024-06-10T03:04:19.920 INFO:teuthology.orchestra.run.smithi120.stdout: "dir_commit": 47, 2024-06-10T03:04:19.920 INFO:teuthology.orchestra.run.smithi120.stdout: "dir_split": 4, 2024-06-10T03:04:19.920 INFO:teuthology.orchestra.run.smithi120.stdout: "dir_merge": 0, 2024-06-10T03:04:19.920 INFO:teuthology.orchestra.run.smithi120.stdout: "inodes": 26, 2024-06-10T03:04:19.920 INFO:teuthology.orchestra.run.smithi120.stdout: "inodes_top": 7, 2024-06-10T03:04:19.921 INFO:teuthology.orchestra.run.smithi120.stdout: "inodes_bottom": 9, 2024-06-10T03:04:19.921 INFO:teuthology.orchestra.run.smithi120.stdout: "inodes_pin_tail": 10, 2024-06-10T03:04:19.921 INFO:teuthology.orchestra.run.smithi120.stdout: "inodes_pinned": 13, 2024-06-10T03:04:19.921 INFO:teuthology.orchestra.run.smithi120.stdout: "inodes_expired": 766, 2024-06-10T03:04:19.921 INFO:teuthology.orchestra.run.smithi120.stdout: "inodes_with_caps": 0, 2024-06-10T03:04:19.921 INFO:teuthology.orchestra.run.smithi120.stdout: "caps": 0, 2024-06-10T03:04:19.921 INFO:teuthology.orchestra.run.smithi120.stdout: "subtrees": 4, 2024-06-10T03:04:19.921 INFO:teuthology.orchestra.run.smithi120.stdout: "traverse": 40677, 2024-06-10T03:04:19.921 INFO:teuthology.orchestra.run.smithi120.stdout: "traverse_hit": 28300, 2024-06-10T03:04:19.921 INFO:teuthology.orchestra.run.smithi120.stdout: "traverse_forward": 14, 2024-06-10T03:04:19.921 INFO:teuthology.orchestra.run.smithi120.stdout: "traverse_discover": 207, 2024-06-10T03:04:19.921 INFO:teuthology.orchestra.run.smithi120.stdout: "traverse_dir_fetch": 0, 2024-06-10T03:04:19.921 INFO:teuthology.orchestra.run.smithi120.stdout: "traverse_remote_ino": 0, 2024-06-10T03:04:19.921 INFO:teuthology.orchestra.run.smithi120.stdout: "traverse_lock": 203, 2024-06-10T03:04:19.922 INFO:teuthology.orchestra.run.smithi120.stdout: "load_cent": 0, 2024-06-10T03:04:19.922 INFO:teuthology.orchestra.run.smithi120.stdout: "q": 0, 2024-06-10T03:04:19.922 INFO:teuthology.orchestra.run.smithi120.stdout: "exported": 41, 2024-06-10T03:04:19.922 INFO:teuthology.orchestra.run.smithi120.stdout: "exported_inodes": 246, 2024-06-10T03:04:19.922 INFO:teuthology.orchestra.run.smithi120.stdout: "imported": 13, 2024-06-10T03:04:19.922 INFO:teuthology.orchestra.run.smithi120.stdout: "imported_inodes": 1101, 2024-06-10T03:04:19.922 INFO:teuthology.orchestra.run.smithi120.stdout: "openino_dir_fetch": 0, 2024-06-10T03:04:19.922 INFO:teuthology.orchestra.run.smithi120.stdout: "openino_backtrace_fetch": 1776, 2024-06-10T03:04:19.922 INFO:teuthology.orchestra.run.smithi120.stdout: "openino_peer_discover": 0, 2024-06-10T03:04:19.922 INFO:teuthology.orchestra.run.smithi120.stdout: "root_rfiles": 3367, 2024-06-10T03:04:19.922 INFO:teuthology.orchestra.run.smithi120.stdout: "root_rbytes": 68050036, 2024-06-10T03:04:19.922 INFO:teuthology.orchestra.run.smithi120.stdout: "root_rsnaps": 0, 2024-06-10T03:04:19.922 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_backtrace_fetch": 1776, 2024-06-10T03:04:19.923 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_set_tag": 0, 2024-06-10T03:04:19.923 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_backtrace_repaired": 0, 2024-06-10T03:04:19.923 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_inotable_repaired": 0, 2024-06-10T03:04:19.923 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_dir_inodes": 0, 2024-06-10T03:04:19.923 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_dir_base_inodes": 0, 2024-06-10T03:04:19.923 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_dirfrag_rstats": 0, 2024-06-10T03:04:19.923 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_file_inodes": 1776, 2024-06-10T03:04:19.923 INFO:teuthology.orchestra.run.smithi120.stdout: "handle_inode_file_caps": 36, 2024-06-10T03:04:19.923 INFO:teuthology.orchestra.run.smithi120.stdout: "ceph_cap_op_revoke": 2098, 2024-06-10T03:04:19.923 INFO:teuthology.orchestra.run.smithi120.stdout: "ceph_cap_op_grant": 3410, 2024-06-10T03:04:19.923 INFO:teuthology.orchestra.run.smithi120.stdout: "ceph_cap_op_trunc": 0, 2024-06-10T03:04:19.923 INFO:teuthology.orchestra.run.smithi120.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-06-10T03:04:19.923 INFO:teuthology.orchestra.run.smithi120.stdout: "ceph_cap_op_flush_ack": 0, 2024-06-10T03:04:19.923 INFO:teuthology.orchestra.run.smithi120.stdout: "handle_client_caps": 3367, 2024-06-10T03:04:19.924 INFO:teuthology.orchestra.run.smithi120.stdout: "handle_client_caps_dirty": 1157, 2024-06-10T03:04:19.924 INFO:teuthology.orchestra.run.smithi120.stdout: "handle_client_cap_release": 118, 2024-06-10T03:04:19.924 INFO:teuthology.orchestra.run.smithi120.stdout: "process_request_cap_release": 4808 2024-06-10T03:04:19.924 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:19.924 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_cache": { 2024-06-10T03:04:19.924 INFO:teuthology.orchestra.run.smithi120.stdout: "dir_update": 475, 2024-06-10T03:04:19.924 INFO:teuthology.orchestra.run.smithi120.stdout: "dir_update_receipt": 3362, 2024-06-10T03:04:19.924 INFO:teuthology.orchestra.run.smithi120.stdout: "dir_try_discover": 1926, 2024-06-10T03:04:19.924 INFO:teuthology.orchestra.run.smithi120.stdout: "dir_send_discover": 2125, 2024-06-10T03:04:19.924 INFO:teuthology.orchestra.run.smithi120.stdout: "dir_handle_discover": 472, 2024-06-10T03:04:19.924 INFO:teuthology.orchestra.run.smithi120.stdout: "num_strays": 0, 2024-06-10T03:04:19.924 INFO:teuthology.orchestra.run.smithi120.stdout: "num_strays_delayed": 0, 2024-06-10T03:04:19.924 INFO:teuthology.orchestra.run.smithi120.stdout: "num_strays_enqueuing": 0, 2024-06-10T03:04:19.924 INFO:teuthology.orchestra.run.smithi120.stdout: "strays_created": 1319, 2024-06-10T03:04:19.925 INFO:teuthology.orchestra.run.smithi120.stdout: "strays_enqueued": 1319, 2024-06-10T03:04:19.925 INFO:teuthology.orchestra.run.smithi120.stdout: "strays_reintegrated": 0, 2024-06-10T03:04:19.925 INFO:teuthology.orchestra.run.smithi120.stdout: "strays_migrated": 0, 2024-06-10T03:04:19.925 INFO:teuthology.orchestra.run.smithi120.stdout: "num_recovering_processing": 0, 2024-06-10T03:04:19.925 INFO:teuthology.orchestra.run.smithi120.stdout: "num_recovering_enqueued": 0, 2024-06-10T03:04:19.925 INFO:teuthology.orchestra.run.smithi120.stdout: "num_recovering_prioritized": 0, 2024-06-10T03:04:19.925 INFO:teuthology.orchestra.run.smithi120.stdout: "recovery_started": 0, 2024-06-10T03:04:19.925 INFO:teuthology.orchestra.run.smithi120.stdout: "recovery_completed": 0, 2024-06-10T03:04:19.925 INFO:teuthology.orchestra.run.smithi120.stdout: "ireq_quiesce_path": 6, 2024-06-10T03:04:19.925 INFO:teuthology.orchestra.run.smithi120.stdout: "ireq_quiesce_inode": 1514, 2024-06-10T03:04:19.925 INFO:teuthology.orchestra.run.smithi120.stdout: "ireq_enqueue_scrub": 0, 2024-06-10T03:04:19.925 INFO:teuthology.orchestra.run.smithi120.stdout: "ireq_exportdir": 41, 2024-06-10T03:04:19.925 INFO:teuthology.orchestra.run.smithi120.stdout: "ireq_flush": 0, 2024-06-10T03:04:19.925 INFO:teuthology.orchestra.run.smithi120.stdout: "ireq_fragmentdir": 4, 2024-06-10T03:04:19.926 INFO:teuthology.orchestra.run.smithi120.stdout: "ireq_fragstats": 0, 2024-06-10T03:04:19.926 INFO:teuthology.orchestra.run.smithi120.stdout: "ireq_inodestats": 0 2024-06-10T03:04:19.926 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:19.926 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_log": { 2024-06-10T03:04:19.926 INFO:teuthology.orchestra.run.smithi120.stdout: "evlrg": 0, 2024-06-10T03:04:19.926 INFO:teuthology.orchestra.run.smithi120.stdout: "evadd": 6535, 2024-06-10T03:04:19.926 INFO:teuthology.orchestra.run.smithi120.stdout: "evex": 0, 2024-06-10T03:04:19.926 INFO:teuthology.orchestra.run.smithi120.stdout: "evtrm": 0, 2024-06-10T03:04:19.926 INFO:teuthology.orchestra.run.smithi120.stdout: "ev": 6535, 2024-06-10T03:04:19.926 INFO:teuthology.orchestra.run.smithi120.stdout: "evexg": 0, 2024-06-10T03:04:19.926 INFO:teuthology.orchestra.run.smithi120.stdout: "evexd": 0, 2024-06-10T03:04:19.926 INFO:teuthology.orchestra.run.smithi120.stdout: "segadd": 8, 2024-06-10T03:04:19.926 INFO:teuthology.orchestra.run.smithi120.stdout: "segex": 0, 2024-06-10T03:04:19.927 INFO:teuthology.orchestra.run.smithi120.stdout: "segtrm": 0, 2024-06-10T03:04:19.927 INFO:teuthology.orchestra.run.smithi120.stdout: "seg": 8, 2024-06-10T03:04:19.927 INFO:teuthology.orchestra.run.smithi120.stdout: "segmjr": 2, 2024-06-10T03:04:19.927 INFO:teuthology.orchestra.run.smithi120.stdout: "segexg": 0, 2024-06-10T03:04:19.927 INFO:teuthology.orchestra.run.smithi120.stdout: "segexd": 0, 2024-06-10T03:04:19.927 INFO:teuthology.orchestra.run.smithi120.stdout: "expos": 4194304, 2024-06-10T03:04:19.927 INFO:teuthology.orchestra.run.smithi120.stdout: "wrpos": 13923504, 2024-06-10T03:04:19.927 INFO:teuthology.orchestra.run.smithi120.stdout: "rdpos": 0, 2024-06-10T03:04:19.927 INFO:teuthology.orchestra.run.smithi120.stdout: "jlat": { 2024-06-10T03:04:19.927 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 295, 2024-06-10T03:04:19.927 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 3.463738233, 2024-06-10T03:04:19.927 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.011741485 2024-06-10T03:04:19.927 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:19.927 INFO:teuthology.orchestra.run.smithi120.stdout: "replayed": 0 2024-06-10T03:04:19.928 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:19.928 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_mem": { 2024-06-10T03:04:19.928 INFO:teuthology.orchestra.run.smithi120.stdout: "ino": 31, 2024-06-10T03:04:19.928 INFO:teuthology.orchestra.run.smithi120.stdout: "ino+": 1553, 2024-06-10T03:04:19.928 INFO:teuthology.orchestra.run.smithi120.stdout: "ino-": 1522, 2024-06-10T03:04:19.928 INFO:teuthology.orchestra.run.smithi120.stdout: "dir": 30, 2024-06-10T03:04:19.928 INFO:teuthology.orchestra.run.smithi120.stdout: "dir+": 1630, 2024-06-10T03:04:19.928 INFO:teuthology.orchestra.run.smithi120.stdout: "dir-": 1600, 2024-06-10T03:04:19.928 INFO:teuthology.orchestra.run.smithi120.stdout: "dn": 26, 2024-06-10T03:04:19.928 INFO:teuthology.orchestra.run.smithi120.stdout: "dn+": 3716, 2024-06-10T03:04:19.928 INFO:teuthology.orchestra.run.smithi120.stdout: "dn-": 3690, 2024-06-10T03:04:19.928 INFO:teuthology.orchestra.run.smithi120.stdout: "cap": 0, 2024-06-10T03:04:19.928 INFO:teuthology.orchestra.run.smithi120.stdout: "cap+": 2107, 2024-06-10T03:04:19.928 INFO:teuthology.orchestra.run.smithi120.stdout: "cap-": 2107, 2024-06-10T03:04:19.929 INFO:teuthology.orchestra.run.smithi120.stdout: "rss": 74084, 2024-06-10T03:04:19.929 INFO:teuthology.orchestra.run.smithi120.stdout: "heap": 223612 2024-06-10T03:04:19.929 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:19.929 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_server": { 2024-06-10T03:04:19.929 INFO:teuthology.orchestra.run.smithi120.stdout: "dispatch_client_request": 22070, 2024-06-10T03:04:19.929 INFO:teuthology.orchestra.run.smithi120.stdout: "dispatch_server_request": 36, 2024-06-10T03:04:19.929 INFO:teuthology.orchestra.run.smithi120.stdout: "handle_client_request": 18662, 2024-06-10T03:04:19.929 INFO:teuthology.orchestra.run.smithi120.stdout: "handle_client_session": 14, 2024-06-10T03:04:19.929 INFO:teuthology.orchestra.run.smithi120.stdout: "handle_peer_request": 2760, 2024-06-10T03:04:19.929 INFO:teuthology.orchestra.run.smithi120.stdout: "req_create_latency": { 2024-06-10T03:04:19.929 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 822, 2024-06-10T03:04:19.929 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 8.127013548, 2024-06-10T03:04:19.929 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.009886877 2024-06-10T03:04:19.929 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:19.930 INFO:teuthology.orchestra.run.smithi120.stdout: "req_getattr_latency": { 2024-06-10T03:04:19.930 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 1, 2024-06-10T03:04:19.930 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.001011165, 2024-06-10T03:04:19.930 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.001011165 2024-06-10T03:04:19.930 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:19.930 INFO:teuthology.orchestra.run.smithi120.stdout: "req_getfilelock_latency": { 2024-06-10T03:04:19.930 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-10T03:04:19.930 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-10T03:04:19.930 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-10T03:04:19.930 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:19.930 INFO:teuthology.orchestra.run.smithi120.stdout: "req_link_latency": { 2024-06-10T03:04:19.930 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-10T03:04:19.930 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-10T03:04:19.930 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-10T03:04:19.931 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:19.931 INFO:teuthology.orchestra.run.smithi120.stdout: "req_lookup_latency": { 2024-06-10T03:04:19.931 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 15338, 2024-06-10T03:04:19.931 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 247.739906583, 2024-06-10T03:04:19.931 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.016152034 2024-06-10T03:04:19.931 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:19.931 INFO:teuthology.orchestra.run.smithi120.stdout: "req_lookuphash_latency": { 2024-06-10T03:04:19.931 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-10T03:04:19.931 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-10T03:04:19.931 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-10T03:04:19.931 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:19.931 INFO:teuthology.orchestra.run.smithi120.stdout: "req_lookupino_latency": { 2024-06-10T03:04:19.931 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-10T03:04:19.931 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-10T03:04:19.932 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-10T03:04:19.932 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:19.932 INFO:teuthology.orchestra.run.smithi120.stdout: "req_lookupname_latency": { 2024-06-10T03:04:19.932 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-10T03:04:19.932 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-10T03:04:19.932 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-10T03:04:19.932 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:19.932 INFO:teuthology.orchestra.run.smithi120.stdout: "req_lookupparent_latency": { 2024-06-10T03:04:19.932 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-10T03:04:19.932 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-10T03:04:19.932 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-10T03:04:19.932 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:19.932 INFO:teuthology.orchestra.run.smithi120.stdout: "req_lookupsnap_latency": { 2024-06-10T03:04:19.932 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-10T03:04:19.932 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-10T03:04:19.933 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-10T03:04:19.933 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:19.933 INFO:teuthology.orchestra.run.smithi120.stdout: "req_lssnap_latency": { 2024-06-10T03:04:19.933 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-10T03:04:19.933 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-10T03:04:19.933 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-10T03:04:19.933 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:19.934 INFO:teuthology.orchestra.run.smithi120.stdout: "req_mkdir_latency": { 2024-06-10T03:04:19.934 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-10T03:04:19.934 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-10T03:04:19.934 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-10T03:04:19.934 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:19.934 INFO:teuthology.orchestra.run.smithi120.stdout: "req_mknod_latency": { 2024-06-10T03:04:19.934 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-10T03:04:19.934 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-10T03:04:19.934 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-10T03:04:19.934 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:19.934 INFO:teuthology.orchestra.run.smithi120.stdout: "req_mksnap_latency": { 2024-06-10T03:04:19.934 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-10T03:04:19.934 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-10T03:04:19.934 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-10T03:04:19.935 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:19.935 INFO:teuthology.orchestra.run.smithi120.stdout: "req_open_latency": { 2024-06-10T03:04:19.935 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 643, 2024-06-10T03:04:19.935 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.685059581, 2024-06-10T03:04:19.935 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.001065411 2024-06-10T03:04:19.935 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:19.935 INFO:teuthology.orchestra.run.smithi120.stdout: "req_readdir_latency": { 2024-06-10T03:04:19.935 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 41, 2024-06-10T03:04:19.935 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.048753639, 2024-06-10T03:04:19.935 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.001189113 2024-06-10T03:04:19.935 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:19.935 INFO:teuthology.orchestra.run.smithi120.stdout: "req_rename_latency": { 2024-06-10T03:04:19.935 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 825, 2024-06-10T03:04:19.935 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 6.118928297, 2024-06-10T03:04:19.936 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.007416882 2024-06-10T03:04:19.936 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:19.936 INFO:teuthology.orchestra.run.smithi120.stdout: "req_renamesnap_latency": { 2024-06-10T03:04:19.936 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-10T03:04:19.936 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-10T03:04:19.936 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-10T03:04:19.936 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:19.936 INFO:teuthology.orchestra.run.smithi120.stdout: "req_snapdiff_latency": { 2024-06-10T03:04:19.936 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-10T03:04:19.936 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-10T03:04:19.936 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-10T03:04:19.936 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:19.936 INFO:teuthology.orchestra.run.smithi120.stdout: "req_rmdir_latency": { 2024-06-10T03:04:19.937 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-10T03:04:19.937 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-10T03:04:19.937 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-10T03:04:19.937 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:19.937 INFO:teuthology.orchestra.run.smithi120.stdout: "req_rmsnap_latency": { 2024-06-10T03:04:19.937 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-10T03:04:19.937 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-10T03:04:19.937 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-10T03:04:19.937 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:19.937 INFO:teuthology.orchestra.run.smithi120.stdout: "req_rmxattr_latency": { 2024-06-10T03:04:19.937 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-10T03:04:19.937 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-10T03:04:19.937 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-10T03:04:19.937 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:19.937 INFO:teuthology.orchestra.run.smithi120.stdout: "req_setattr_latency": { 2024-06-10T03:04:19.938 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-10T03:04:19.938 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-10T03:04:19.938 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-10T03:04:19.938 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:19.938 INFO:teuthology.orchestra.run.smithi120.stdout: "req_setdirlayout_latency": { 2024-06-10T03:04:19.938 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-10T03:04:19.938 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-10T03:04:19.938 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-10T03:04:19.938 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:19.938 INFO:teuthology.orchestra.run.smithi120.stdout: "req_setfilelock_latency": { 2024-06-10T03:04:19.938 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-10T03:04:19.938 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-10T03:04:19.938 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-10T03:04:19.939 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:19.939 INFO:teuthology.orchestra.run.smithi120.stdout: "req_setlayout_latency": { 2024-06-10T03:04:19.939 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-10T03:04:19.939 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-10T03:04:19.939 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-10T03:04:19.939 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:19.939 INFO:teuthology.orchestra.run.smithi120.stdout: "req_setxattr_latency": { 2024-06-10T03:04:19.939 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-10T03:04:19.939 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-10T03:04:19.939 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-10T03:04:19.939 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:19.939 INFO:teuthology.orchestra.run.smithi120.stdout: "req_symlink_latency": { 2024-06-10T03:04:19.939 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-10T03:04:19.939 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-10T03:04:19.940 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-10T03:04:19.940 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:19.940 INFO:teuthology.orchestra.run.smithi120.stdout: "req_unlink_latency": { 2024-06-10T03:04:19.940 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 978, 2024-06-10T03:04:19.940 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 2.250846921, 2024-06-10T03:04:19.940 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.002301479 2024-06-10T03:04:19.940 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:19.940 INFO:teuthology.orchestra.run.smithi120.stdout: "cap_revoke_eviction": 0, 2024-06-10T03:04:19.940 INFO:teuthology.orchestra.run.smithi120.stdout: "cap_acquisition_throttle": 0, 2024-06-10T03:04:19.940 INFO:teuthology.orchestra.run.smithi120.stdout: "req_getvxattr_latency": { 2024-06-10T03:04:19.940 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-10T03:04:19.940 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-10T03:04:19.941 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-10T03:04:19.941 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-06-10T03:04:19.941 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:19.941 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_sessions": { 2024-06-10T03:04:19.941 INFO:teuthology.orchestra.run.smithi120.stdout: "session_count": 1, 2024-06-10T03:04:19.941 INFO:teuthology.orchestra.run.smithi120.stdout: "session_add": 3, 2024-06-10T03:04:19.941 INFO:teuthology.orchestra.run.smithi120.stdout: "session_remove": 2, 2024-06-10T03:04:19.941 INFO:teuthology.orchestra.run.smithi120.stdout: "sessions_open": 1, 2024-06-10T03:04:19.941 INFO:teuthology.orchestra.run.smithi120.stdout: "sessions_stale": 0, 2024-06-10T03:04:19.941 INFO:teuthology.orchestra.run.smithi120.stdout: "total_load": 4441, 2024-06-10T03:04:19.941 INFO:teuthology.orchestra.run.smithi120.stdout: "average_load": 4441, 2024-06-10T03:04:19.941 INFO:teuthology.orchestra.run.smithi120.stdout: "avg_session_uptime": 2343, 2024-06-10T03:04:19.941 INFO:teuthology.orchestra.run.smithi120.stdout: "mdthresh_evicted": 0 2024-06-10T03:04:19.941 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:19.941 INFO:teuthology.orchestra.run.smithi120.stdout: "mempool": { 2024-06-10T03:04:19.942 INFO:teuthology.orchestra.run.smithi120.stdout: "bloom_filter_bytes": 0, 2024-06-10T03:04:19.942 INFO:teuthology.orchestra.run.smithi120.stdout: "bloom_filter_items": 0, 2024-06-10T03:04:19.942 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_alloc_bytes": 0, 2024-06-10T03:04:19.942 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_alloc_items": 0, 2024-06-10T03:04:19.942 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_data_bytes": 0, 2024-06-10T03:04:19.942 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_data_items": 0, 2024-06-10T03:04:19.942 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-10T03:04:19.942 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_onode_items": 0, 2024-06-10T03:04:19.942 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-10T03:04:19.942 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_meta_items": 0, 2024-06-10T03:04:19.942 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_other_bytes": 0, 2024-06-10T03:04:19.942 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_other_items": 0, 2024-06-10T03:04:19.942 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-10T03:04:19.943 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_buffer_items": 0, 2024-06-10T03:04:19.943 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_extent_bytes": 0, 2024-06-10T03:04:19.943 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_extent_items": 0, 2024-06-10T03:04:19.943 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_blob_bytes": 0, 2024-06-10T03:04:19.943 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_blob_items": 0, 2024-06-10T03:04:19.943 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-10T03:04:19.943 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_shared_blob_items": 0, 2024-06-10T03:04:19.943 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-10T03:04:19.943 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_inline_bl_items": 0, 2024-06-10T03:04:19.943 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_fsck_bytes": 0, 2024-06-10T03:04:19.943 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_fsck_items": 0, 2024-06-10T03:04:19.943 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_txc_bytes": 0, 2024-06-10T03:04:19.943 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_txc_items": 0, 2024-06-10T03:04:19.944 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-10T03:04:19.944 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_writing_deferred_items": 0, 2024-06-10T03:04:19.944 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_writing_bytes": 0, 2024-06-10T03:04:19.944 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_writing_items": 0, 2024-06-10T03:04:19.944 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_bytes": 0, 2024-06-10T03:04:19.944 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_items": 0, 2024-06-10T03:04:19.944 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_file_reader_bytes": 0, 2024-06-10T03:04:19.944 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_file_reader_items": 0, 2024-06-10T03:04:19.944 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_file_writer_bytes": 0, 2024-06-10T03:04:19.944 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_file_writer_items": 0, 2024-06-10T03:04:19.944 INFO:teuthology.orchestra.run.smithi120.stdout: "buffer_anon_bytes": 295607, 2024-06-10T03:04:19.944 INFO:teuthology.orchestra.run.smithi120.stdout: "buffer_anon_items": 106, 2024-06-10T03:04:19.944 INFO:teuthology.orchestra.run.smithi120.stdout: "buffer_meta_bytes": 0, 2024-06-10T03:04:19.944 INFO:teuthology.orchestra.run.smithi120.stdout: "buffer_meta_items": 0, 2024-06-10T03:04:19.945 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_bytes": 0, 2024-06-10T03:04:19.945 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_items": 0, 2024-06-10T03:04:19.945 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_mapbl_bytes": 0, 2024-06-10T03:04:19.945 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_mapbl_items": 0, 2024-06-10T03:04:19.945 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_pglog_bytes": 0, 2024-06-10T03:04:19.945 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_pglog_items": 0, 2024-06-10T03:04:19.945 INFO:teuthology.orchestra.run.smithi120.stdout: "osdmap_bytes": 8984, 2024-06-10T03:04:19.945 INFO:teuthology.orchestra.run.smithi120.stdout: "osdmap_items": 172, 2024-06-10T03:04:19.945 INFO:teuthology.orchestra.run.smithi120.stdout: "osdmap_mapping_bytes": 0, 2024-06-10T03:04:19.945 INFO:teuthology.orchestra.run.smithi120.stdout: "osdmap_mapping_items": 0, 2024-06-10T03:04:19.945 INFO:teuthology.orchestra.run.smithi120.stdout: "pgmap_bytes": 0, 2024-06-10T03:04:19.945 INFO:teuthology.orchestra.run.smithi120.stdout: "pgmap_items": 0, 2024-06-10T03:04:19.945 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_co_bytes": 127165, 2024-06-10T03:04:19.946 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_co_items": 535, 2024-06-10T03:04:19.946 INFO:teuthology.orchestra.run.smithi120.stdout: "unittest_1_bytes": 0, 2024-06-10T03:04:19.946 INFO:teuthology.orchestra.run.smithi120.stdout: "unittest_1_items": 0, 2024-06-10T03:04:19.946 INFO:teuthology.orchestra.run.smithi120.stdout: "unittest_2_bytes": 0, 2024-06-10T03:04:19.946 INFO:teuthology.orchestra.run.smithi120.stdout: "unittest_2_items": 0 2024-06-10T03:04:19.946 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:19.946 INFO:teuthology.orchestra.run.smithi120.stdout: "objecter": { 2024-06-10T03:04:19.946 INFO:teuthology.orchestra.run.smithi120.stdout: "op_active": 0, 2024-06-10T03:04:19.946 INFO:teuthology.orchestra.run.smithi120.stdout: "op_laggy": 0, 2024-06-10T03:04:19.946 INFO:teuthology.orchestra.run.smithi120.stdout: "op_send": 5116, 2024-06-10T03:04:19.946 INFO:teuthology.orchestra.run.smithi120.stdout: "op_send_bytes": 10787579, 2024-06-10T03:04:19.946 INFO:teuthology.orchestra.run.smithi120.stdout: "op_resend": 28, 2024-06-10T03:04:19.947 INFO:teuthology.orchestra.run.smithi120.stdout: "op_reply": 5116, 2024-06-10T03:04:19.947 INFO:teuthology.orchestra.run.smithi120.stdout: "op_latency": { 2024-06-10T03:04:19.947 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 5116, 2024-06-10T03:04:19.947 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 30.774310749, 2024-06-10T03:04:19.947 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.006015307 2024-06-10T03:04:19.947 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:19.947 INFO:teuthology.orchestra.run.smithi120.stdout: "op_inflight": 0, 2024-06-10T03:04:19.947 INFO:teuthology.orchestra.run.smithi120.stdout: "oplen_avg": { 2024-06-10T03:04:19.947 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 5116, 2024-06-10T03:04:19.947 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 5325 2024-06-10T03:04:19.947 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:19.947 INFO:teuthology.orchestra.run.smithi120.stdout: "op": 5116, 2024-06-10T03:04:19.947 INFO:teuthology.orchestra.run.smithi120.stdout: "op_r": 1847, 2024-06-10T03:04:19.948 INFO:teuthology.orchestra.run.smithi120.stdout: "op_w": 3269, 2024-06-10T03:04:19.948 INFO:teuthology.orchestra.run.smithi120.stdout: "op_rmw": 0, 2024-06-10T03:04:19.948 INFO:teuthology.orchestra.run.smithi120.stdout: "op_pg": 0, 2024-06-10T03:04:19.948 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_stat": 0, 2024-06-10T03:04:19.948 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_create": 10, 2024-06-10T03:04:19.948 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_read": 71, 2024-06-10T03:04:19.948 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_write": 366, 2024-06-10T03:04:19.948 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_writefull": 65, 2024-06-10T03:04:19.948 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_writesame": 0, 2024-06-10T03:04:19.948 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_append": 0, 2024-06-10T03:04:19.948 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_zero": 0, 2024-06-10T03:04:19.948 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_truncate": 4, 2024-06-10T03:04:19.948 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_delete": 2652, 2024-06-10T03:04:19.948 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_mapext": 0, 2024-06-10T03:04:19.948 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_sparse_read": 0, 2024-06-10T03:04:19.949 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_clonerange": 0, 2024-06-10T03:04:19.949 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_getxattr": 1776, 2024-06-10T03:04:19.949 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_setxattr": 20, 2024-06-10T03:04:19.949 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_cmpxattr": 0, 2024-06-10T03:04:19.949 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_rmxattr": 0, 2024-06-10T03:04:19.949 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_resetxattrs": 0, 2024-06-10T03:04:19.949 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_call": 0, 2024-06-10T03:04:19.949 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_watch": 0, 2024-06-10T03:04:19.949 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_notify": 0, 2024-06-10T03:04:19.949 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_src_cmpxattr": 0, 2024-06-10T03:04:19.949 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_pgls": 0, 2024-06-10T03:04:19.949 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_pgls_filter": 0, 2024-06-10T03:04:19.949 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_other": 0, 2024-06-10T03:04:19.949 INFO:teuthology.orchestra.run.smithi120.stdout: "linger_active": 0, 2024-06-10T03:04:19.949 INFO:teuthology.orchestra.run.smithi120.stdout: "linger_send": 0, 2024-06-10T03:04:19.950 INFO:teuthology.orchestra.run.smithi120.stdout: "linger_resend": 0, 2024-06-10T03:04:19.950 INFO:teuthology.orchestra.run.smithi120.stdout: "linger_ping": 0, 2024-06-10T03:04:19.950 INFO:teuthology.orchestra.run.smithi120.stdout: "poolop_active": 0, 2024-06-10T03:04:19.950 INFO:teuthology.orchestra.run.smithi120.stdout: "poolop_send": 0, 2024-06-10T03:04:19.950 INFO:teuthology.orchestra.run.smithi120.stdout: "poolop_resend": 0, 2024-06-10T03:04:19.950 INFO:teuthology.orchestra.run.smithi120.stdout: "poolstat_active": 0, 2024-06-10T03:04:19.950 INFO:teuthology.orchestra.run.smithi120.stdout: "poolstat_send": 0, 2024-06-10T03:04:19.950 INFO:teuthology.orchestra.run.smithi120.stdout: "poolstat_resend": 0, 2024-06-10T03:04:19.950 INFO:teuthology.orchestra.run.smithi120.stdout: "statfs_active": 0, 2024-06-10T03:04:19.950 INFO:teuthology.orchestra.run.smithi120.stdout: "statfs_send": 0, 2024-06-10T03:04:19.950 INFO:teuthology.orchestra.run.smithi120.stdout: "statfs_resend": 0, 2024-06-10T03:04:19.950 INFO:teuthology.orchestra.run.smithi120.stdout: "command_active": 0, 2024-06-10T03:04:19.950 INFO:teuthology.orchestra.run.smithi120.stdout: "command_send": 0, 2024-06-10T03:04:19.950 INFO:teuthology.orchestra.run.smithi120.stdout: "command_resend": 0, 2024-06-10T03:04:19.951 INFO:teuthology.orchestra.run.smithi120.stdout: "map_epoch": 100, 2024-06-10T03:04:19.951 INFO:teuthology.orchestra.run.smithi120.stdout: "map_full": 0, 2024-06-10T03:04:19.951 INFO:teuthology.orchestra.run.smithi120.stdout: "map_inc": 2, 2024-06-10T03:04:19.951 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_sessions": 12, 2024-06-10T03:04:19.951 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_session_open": 12, 2024-06-10T03:04:19.951 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_session_close": 0, 2024-06-10T03:04:19.951 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_laggy": 0, 2024-06-10T03:04:19.951 INFO:teuthology.orchestra.run.smithi120.stdout: "omap_wr": 299, 2024-06-10T03:04:19.951 INFO:teuthology.orchestra.run.smithi120.stdout: "omap_rd": 0, 2024-06-10T03:04:19.951 INFO:teuthology.orchestra.run.smithi120.stdout: "omap_del": 62 2024-06-10T03:04:19.951 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:19.951 INFO:teuthology.orchestra.run.smithi120.stdout: "oft": { 2024-06-10T03:04:19.951 INFO:teuthology.orchestra.run.smithi120.stdout: "omap_total_objs": 1, 2024-06-10T03:04:19.951 INFO:teuthology.orchestra.run.smithi120.stdout: "omap_total_kv_pairs": 0, 2024-06-10T03:04:19.951 INFO:teuthology.orchestra.run.smithi120.stdout: "omap_total_updates": 0, 2024-06-10T03:04:19.952 INFO:teuthology.orchestra.run.smithi120.stdout: "omap_total_removes": 0 2024-06-10T03:04:19.952 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:19.952 INFO:teuthology.orchestra.run.smithi120.stdout: "osd-slow-ops": { 2024-06-10T03:04:19.952 INFO:teuthology.orchestra.run.smithi120.stdout: "slow_ops_count": 1504 2024-06-10T03:04:19.952 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:19.952 INFO:teuthology.orchestra.run.smithi120.stdout: "purge_queue": { 2024-06-10T03:04:19.952 INFO:teuthology.orchestra.run.smithi120.stdout: "pq_executing_ops": 0, 2024-06-10T03:04:19.952 INFO:teuthology.orchestra.run.smithi120.stdout: "pq_executing_ops_high_water": 10, 2024-06-10T03:04:19.952 INFO:teuthology.orchestra.run.smithi120.stdout: "pq_executing": 0, 2024-06-10T03:04:19.952 INFO:teuthology.orchestra.run.smithi120.stdout: "pq_executing_high_water": 5, 2024-06-10T03:04:19.952 INFO:teuthology.orchestra.run.smithi120.stdout: "pq_executed_ops": 2638, 2024-06-10T03:04:19.952 INFO:teuthology.orchestra.run.smithi120.stdout: "pq_executed": 1319, 2024-06-10T03:04:19.952 INFO:teuthology.orchestra.run.smithi120.stdout: "pq_item_in_journal": 0 2024-06-10T03:04:19.952 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:19.952 INFO:teuthology.orchestra.run.smithi120.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-10T03:04:19.953 INFO:teuthology.orchestra.run.smithi120.stdout: "val": 0, 2024-06-10T03:04:19.953 INFO:teuthology.orchestra.run.smithi120.stdout: "max": 104857600, 2024-06-10T03:04:19.953 INFO:teuthology.orchestra.run.smithi120.stdout: "get_started": 0, 2024-06-10T03:04:19.953 INFO:teuthology.orchestra.run.smithi120.stdout: "get": 46862, 2024-06-10T03:04:19.953 INFO:teuthology.orchestra.run.smithi120.stdout: "get_sum": 13087307, 2024-06-10T03:04:19.953 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_fail": 0, 2024-06-10T03:04:19.953 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_success": 46862, 2024-06-10T03:04:19.953 INFO:teuthology.orchestra.run.smithi120.stdout: "take": 0, 2024-06-10T03:04:19.953 INFO:teuthology.orchestra.run.smithi120.stdout: "take_sum": 0, 2024-06-10T03:04:19.953 INFO:teuthology.orchestra.run.smithi120.stdout: "put": 46862, 2024-06-10T03:04:19.953 INFO:teuthology.orchestra.run.smithi120.stdout: "put_sum": 13087307, 2024-06-10T03:04:19.953 INFO:teuthology.orchestra.run.smithi120.stdout: "wait": { 2024-06-10T03:04:19.953 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-10T03:04:19.953 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-10T03:04:19.954 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-10T03:04:19.954 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-06-10T03:04:19.954 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:19.954 INFO:teuthology.orchestra.run.smithi120.stdout: "throttle-objecter_bytes": { 2024-06-10T03:04:19.954 INFO:teuthology.orchestra.run.smithi120.stdout: "val": 0, 2024-06-10T03:04:19.954 INFO:teuthology.orchestra.run.smithi120.stdout: "max": 104857600, 2024-06-10T03:04:19.954 INFO:teuthology.orchestra.run.smithi120.stdout: "get_started": 0, 2024-06-10T03:04:19.954 INFO:teuthology.orchestra.run.smithi120.stdout: "get": 0, 2024-06-10T03:04:19.954 INFO:teuthology.orchestra.run.smithi120.stdout: "get_sum": 0, 2024-06-10T03:04:19.954 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_fail": 0, 2024-06-10T03:04:19.954 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_success": 0, 2024-06-10T03:04:19.954 INFO:teuthology.orchestra.run.smithi120.stdout: "take": 5116, 2024-06-10T03:04:19.954 INFO:teuthology.orchestra.run.smithi120.stdout: "take_sum": 10931350, 2024-06-10T03:04:19.955 INFO:teuthology.orchestra.run.smithi120.stdout: "put": 5116, 2024-06-10T03:04:19.955 INFO:teuthology.orchestra.run.smithi120.stdout: "put_sum": 10931350, 2024-06-10T03:04:19.955 INFO:teuthology.orchestra.run.smithi120.stdout: "wait": { 2024-06-10T03:04:19.955 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-10T03:04:19.955 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-10T03:04:19.955 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-10T03:04:19.955 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-06-10T03:04:19.955 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:19.955 INFO:teuthology.orchestra.run.smithi120.stdout: "throttle-objecter_ops": { 2024-06-10T03:04:19.955 INFO:teuthology.orchestra.run.smithi120.stdout: "val": 0, 2024-06-10T03:04:19.955 INFO:teuthology.orchestra.run.smithi120.stdout: "max": 1024, 2024-06-10T03:04:19.955 INFO:teuthology.orchestra.run.smithi120.stdout: "get_started": 0, 2024-06-10T03:04:19.955 INFO:teuthology.orchestra.run.smithi120.stdout: "get": 0, 2024-06-10T03:04:19.955 INFO:teuthology.orchestra.run.smithi120.stdout: "get_sum": 0, 2024-06-10T03:04:19.956 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_fail": 0, 2024-06-10T03:04:19.956 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_success": 0, 2024-06-10T03:04:19.956 INFO:teuthology.orchestra.run.smithi120.stdout: "take": 5116, 2024-06-10T03:04:19.956 INFO:teuthology.orchestra.run.smithi120.stdout: "take_sum": 5116, 2024-06-10T03:04:19.956 INFO:teuthology.orchestra.run.smithi120.stdout: "put": 5116, 2024-06-10T03:04:19.956 INFO:teuthology.orchestra.run.smithi120.stdout: "put_sum": 5116, 2024-06-10T03:04:19.956 INFO:teuthology.orchestra.run.smithi120.stdout: "wait": { 2024-06-10T03:04:19.956 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-10T03:04:19.956 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-10T03:04:19.956 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-10T03:04:19.956 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-06-10T03:04:19.956 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:19.956 INFO:teuthology.orchestra.run.smithi120.stdout: "throttle-write_buf_throttle": { 2024-06-10T03:04:19.956 INFO:teuthology.orchestra.run.smithi120.stdout: "val": 0, 2024-06-10T03:04:19.956 INFO:teuthology.orchestra.run.smithi120.stdout: "max": 3758096384, 2024-06-10T03:04:19.957 INFO:teuthology.orchestra.run.smithi120.stdout: "get_started": 0, 2024-06-10T03:04:19.957 INFO:teuthology.orchestra.run.smithi120.stdout: "get": 1319, 2024-06-10T03:04:19.957 INFO:teuthology.orchestra.run.smithi120.stdout: "get_sum": 143771, 2024-06-10T03:04:19.957 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_fail": 0, 2024-06-10T03:04:19.957 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_success": 1319, 2024-06-10T03:04:19.957 INFO:teuthology.orchestra.run.smithi120.stdout: "take": 0, 2024-06-10T03:04:19.957 INFO:teuthology.orchestra.run.smithi120.stdout: "take_sum": 0, 2024-06-10T03:04:19.957 INFO:teuthology.orchestra.run.smithi120.stdout: "put": 71, 2024-06-10T03:04:19.957 INFO:teuthology.orchestra.run.smithi120.stdout: "put_sum": 143771, 2024-06-10T03:04:19.957 INFO:teuthology.orchestra.run.smithi120.stdout: "wait": { 2024-06-10T03:04:19.957 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-10T03:04:19.957 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-10T03:04:19.957 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-10T03:04:19.957 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-06-10T03:04:19.957 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:19.958 INFO:teuthology.orchestra.run.smithi120.stdout: "throttle-write_buf_throttle-0x55abecec2500": { 2024-06-10T03:04:19.958 INFO:teuthology.orchestra.run.smithi120.stdout: "val": 0, 2024-06-10T03:04:19.958 INFO:teuthology.orchestra.run.smithi120.stdout: "max": 3758096384, 2024-06-10T03:04:19.958 INFO:teuthology.orchestra.run.smithi120.stdout: "get_started": 0, 2024-06-10T03:04:19.958 INFO:teuthology.orchestra.run.smithi120.stdout: "get": 6535, 2024-06-10T03:04:19.958 INFO:teuthology.orchestra.run.smithi120.stdout: "get_sum": 9729200, 2024-06-10T03:04:19.958 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_fail": 0, 2024-06-10T03:04:19.958 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_success": 6535, 2024-06-10T03:04:19.958 INFO:teuthology.orchestra.run.smithi120.stdout: "take": 0, 2024-06-10T03:04:19.958 INFO:teuthology.orchestra.run.smithi120.stdout: "take_sum": 0, 2024-06-10T03:04:19.958 INFO:teuthology.orchestra.run.smithi120.stdout: "put": 295, 2024-06-10T03:04:19.958 INFO:teuthology.orchestra.run.smithi120.stdout: "put_sum": 9729200, 2024-06-10T03:04:19.958 INFO:teuthology.orchestra.run.smithi120.stdout: "wait": { 2024-06-10T03:04:19.958 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-10T03:04:19.959 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-10T03:04:19.959 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-10T03:04:19.959 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-06-10T03:04:19.959 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-06-10T03:04:19.959 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-06-10T03:04:20.727 INFO:tasks.check_counter:Daemon mds.h mds.exported=41 2024-06-10T03:04:20.727 INFO:tasks.check_counter:Daemon mds.h mds.imported=13 2024-06-10T03:04:20.728 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update=475 2024-06-10T03:04:20.728 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update_receipt=3362 2024-06-10T03:04:20.728 INFO:tasks.check_counter:Daemon mds.h mds.dir_split=4 2024-06-10T03:04:20.728 DEBUG:tasks.check_counter:Getting stats from k 2024-06-10T03:04:20.728 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph daemon mds.k perf dump 2024-06-10T03:04:20.970 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.b/config 2024-06-10T03:04:21.091 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:04:20 smithi120 ceph-mon[36634]: pgmap v738: 129 pgs: 129 active+clean; 228 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-06-10T03:04:21.204 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:04:20 smithi188 ceph-mon[36599]: pgmap v738: 129 pgs: 129 active+clean; 228 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-06-10T03:04:21.227 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:04:20 smithi052 ceph-mon[32362]: pgmap v738: 129 pgs: 129 active+clean; 228 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-06-10T03:04:22.732 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-06-10T03:04:22.732 INFO:teuthology.orchestra.run.smithi120.stdout: "AsyncMessenger::Worker-0": { 2024-06-10T03:04:22.732 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_messages": 707, 2024-06-10T03:04:22.732 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_messages": 681, 2024-06-10T03:04:22.732 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_bytes": 126868159, 2024-06-10T03:04:22.732 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_bytes": 287902, 2024-06-10T03:04:22.732 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_created_connections": 7, 2024-06-10T03:04:22.732 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_active_connections": 3, 2024-06-10T03:04:22.732 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_total_time": 0.147212853, 2024-06-10T03:04:22.732 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_send_time": 0.024126345, 2024-06-10T03:04:22.732 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_recv_time": 0.105417631, 2024-06-10T03:04:22.733 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_fast_dispatch_time": 0.016343469, 2024-06-10T03:04:22.733 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_messages_queue_lat": { 2024-06-10T03:04:22.733 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 681, 2024-06-10T03:04:22.733 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.025348434, 2024-06-10T03:04:22.733 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000037222 2024-06-10T03:04:22.733 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:22.733 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_handle_ack_lat": { 2024-06-10T03:04:22.733 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-10T03:04:22.733 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-10T03:04:22.733 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-10T03:04:22.733 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:22.733 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_encrypted_bytes": 911552, 2024-06-10T03:04:22.734 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_encrypted_bytes": 151792 2024-06-10T03:04:22.734 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:22.734 INFO:teuthology.orchestra.run.smithi120.stdout: "AsyncMessenger::Worker-1": { 2024-06-10T03:04:22.734 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_messages": 1036, 2024-06-10T03:04:22.734 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_messages": 1220, 2024-06-10T03:04:22.734 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_bytes": 198552422, 2024-06-10T03:04:22.734 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_bytes": 704175, 2024-06-10T03:04:22.734 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_created_connections": 6, 2024-06-10T03:04:22.734 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_active_connections": 5, 2024-06-10T03:04:22.734 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_total_time": 0.202920623, 2024-06-10T03:04:22.734 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_send_time": 0.029142717, 2024-06-10T03:04:22.734 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_recv_time": 0.134088076, 2024-06-10T03:04:22.735 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_fast_dispatch_time": 0.029898196, 2024-06-10T03:04:22.735 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_messages_queue_lat": { 2024-06-10T03:04:22.735 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 1218, 2024-06-10T03:04:22.735 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.040612478, 2024-06-10T03:04:22.735 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000033343 2024-06-10T03:04:22.735 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:22.735 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_handle_ack_lat": { 2024-06-10T03:04:22.735 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-10T03:04:22.735 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-10T03:04:22.735 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-10T03:04:22.735 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:22.735 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_encrypted_bytes": 160, 2024-06-10T03:04:22.735 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_encrypted_bytes": 375040 2024-06-10T03:04:22.735 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:22.736 INFO:teuthology.orchestra.run.smithi120.stdout: "AsyncMessenger::Worker-2": { 2024-06-10T03:04:22.736 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_messages": 1742, 2024-06-10T03:04:22.736 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_messages": 1744, 2024-06-10T03:04:22.736 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_bytes": 176701480, 2024-06-10T03:04:22.736 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_bytes": 554014, 2024-06-10T03:04:22.736 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_created_connections": 6, 2024-06-10T03:04:22.736 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_active_connections": 5, 2024-06-10T03:04:22.736 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_total_time": 0.231249063, 2024-06-10T03:04:22.736 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_send_time": 0.051307776, 2024-06-10T03:04:22.736 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_recv_time": 0.115575943, 2024-06-10T03:04:22.736 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_fast_dispatch_time": 0.050689022, 2024-06-10T03:04:22.736 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_messages_queue_lat": { 2024-06-10T03:04:22.736 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 1742, 2024-06-10T03:04:22.737 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.064804593, 2024-06-10T03:04:22.737 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000037201 2024-06-10T03:04:22.737 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:22.737 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_handle_ack_lat": { 2024-06-10T03:04:22.737 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-10T03:04:22.737 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-10T03:04:22.737 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-10T03:04:22.737 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:22.737 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_encrypted_bytes": 0, 2024-06-10T03:04:22.737 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_encrypted_bytes": 0 2024-06-10T03:04:22.737 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:22.737 INFO:teuthology.orchestra.run.smithi120.stdout: "cct": { 2024-06-10T03:04:22.737 INFO:teuthology.orchestra.run.smithi120.stdout: "total_workers": 1, 2024-06-10T03:04:22.738 INFO:teuthology.orchestra.run.smithi120.stdout: "unhealthy_workers": 0 2024-06-10T03:04:22.738 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:22.738 INFO:teuthology.orchestra.run.smithi120.stdout: "finisher-MDSRank": { 2024-06-10T03:04:22.738 INFO:teuthology.orchestra.run.smithi120.stdout: "queue_len": 0, 2024-06-10T03:04:22.738 INFO:teuthology.orchestra.run.smithi120.stdout: "complete_latency": { 2024-06-10T03:04:22.738 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 5017, 2024-06-10T03:04:22.738 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.239559488, 2024-06-10T03:04:22.738 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000047749 2024-06-10T03:04:22.738 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-06-10T03:04:22.738 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:22.738 INFO:teuthology.orchestra.run.smithi120.stdout: "finisher-PurgeQueue": { 2024-06-10T03:04:22.738 INFO:teuthology.orchestra.run.smithi120.stdout: "queue_len": 0, 2024-06-10T03:04:22.738 INFO:teuthology.orchestra.run.smithi120.stdout: "complete_latency": { 2024-06-10T03:04:22.739 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-10T03:04:22.739 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-10T03:04:22.739 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-10T03:04:22.739 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-06-10T03:04:22.739 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:22.739 INFO:teuthology.orchestra.run.smithi120.stdout: "mds": { 2024-06-10T03:04:22.739 INFO:teuthology.orchestra.run.smithi120.stdout: "request": 0, 2024-06-10T03:04:22.739 INFO:teuthology.orchestra.run.smithi120.stdout: "reply": 0, 2024-06-10T03:04:22.739 INFO:teuthology.orchestra.run.smithi120.stdout: "reply_latency": { 2024-06-10T03:04:22.739 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-10T03:04:22.739 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-10T03:04:22.739 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-10T03:04:22.739 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:22.740 INFO:teuthology.orchestra.run.smithi120.stdout: "slow_reply": 0, 2024-06-10T03:04:22.740 INFO:teuthology.orchestra.run.smithi120.stdout: "forward": 0, 2024-06-10T03:04:22.740 INFO:teuthology.orchestra.run.smithi120.stdout: "dir_fetch_complete": 0, 2024-06-10T03:04:22.740 INFO:teuthology.orchestra.run.smithi120.stdout: "dir_fetch_keys": 0, 2024-06-10T03:04:22.740 INFO:teuthology.orchestra.run.smithi120.stdout: "dir_commit": 0, 2024-06-10T03:04:22.740 INFO:teuthology.orchestra.run.smithi120.stdout: "dir_split": 0, 2024-06-10T03:04:22.740 INFO:teuthology.orchestra.run.smithi120.stdout: "dir_merge": 0, 2024-06-10T03:04:22.740 INFO:teuthology.orchestra.run.smithi120.stdout: "inodes": 36629, 2024-06-10T03:04:22.740 INFO:teuthology.orchestra.run.smithi120.stdout: "inodes_top": 8276, 2024-06-10T03:04:22.740 INFO:teuthology.orchestra.run.smithi120.stdout: "inodes_bottom": 28353, 2024-06-10T03:04:22.740 INFO:teuthology.orchestra.run.smithi120.stdout: "inodes_pin_tail": 0, 2024-06-10T03:04:22.740 INFO:teuthology.orchestra.run.smithi120.stdout: "inodes_pinned": 24806, 2024-06-10T03:04:22.740 INFO:teuthology.orchestra.run.smithi120.stdout: "inodes_expired": 0, 2024-06-10T03:04:22.740 INFO:teuthology.orchestra.run.smithi120.stdout: "inodes_with_caps": 0, 2024-06-10T03:04:22.741 INFO:teuthology.orchestra.run.smithi120.stdout: "caps": 0, 2024-06-10T03:04:22.741 INFO:teuthology.orchestra.run.smithi120.stdout: "subtrees": 1, 2024-06-10T03:04:22.741 INFO:teuthology.orchestra.run.smithi120.stdout: "traverse": 0, 2024-06-10T03:04:22.741 INFO:teuthology.orchestra.run.smithi120.stdout: "traverse_hit": 0, 2024-06-10T03:04:22.741 INFO:teuthology.orchestra.run.smithi120.stdout: "traverse_forward": 0, 2024-06-10T03:04:22.741 INFO:teuthology.orchestra.run.smithi120.stdout: "traverse_discover": 0, 2024-06-10T03:04:22.741 INFO:teuthology.orchestra.run.smithi120.stdout: "traverse_dir_fetch": 0, 2024-06-10T03:04:22.741 INFO:teuthology.orchestra.run.smithi120.stdout: "traverse_remote_ino": 0, 2024-06-10T03:04:22.741 INFO:teuthology.orchestra.run.smithi120.stdout: "traverse_lock": 0, 2024-06-10T03:04:22.741 INFO:teuthology.orchestra.run.smithi120.stdout: "load_cent": 0, 2024-06-10T03:04:22.741 INFO:teuthology.orchestra.run.smithi120.stdout: "q": 0, 2024-06-10T03:04:22.741 INFO:teuthology.orchestra.run.smithi120.stdout: "exported": 0, 2024-06-10T03:04:22.741 INFO:teuthology.orchestra.run.smithi120.stdout: "exported_inodes": 0, 2024-06-10T03:04:22.741 INFO:teuthology.orchestra.run.smithi120.stdout: "imported": 0, 2024-06-10T03:04:22.741 INFO:teuthology.orchestra.run.smithi120.stdout: "imported_inodes": 0, 2024-06-10T03:04:22.742 INFO:teuthology.orchestra.run.smithi120.stdout: "openino_dir_fetch": 0, 2024-06-10T03:04:22.742 INFO:teuthology.orchestra.run.smithi120.stdout: "openino_backtrace_fetch": 0, 2024-06-10T03:04:22.742 INFO:teuthology.orchestra.run.smithi120.stdout: "openino_peer_discover": 0, 2024-06-10T03:04:22.742 INFO:teuthology.orchestra.run.smithi120.stdout: "root_rfiles": 0, 2024-06-10T03:04:22.742 INFO:teuthology.orchestra.run.smithi120.stdout: "root_rbytes": 0, 2024-06-10T03:04:22.742 INFO:teuthology.orchestra.run.smithi120.stdout: "root_rsnaps": 0, 2024-06-10T03:04:22.742 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_backtrace_fetch": 0, 2024-06-10T03:04:22.742 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_set_tag": 0, 2024-06-10T03:04:22.742 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_backtrace_repaired": 0, 2024-06-10T03:04:22.742 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_inotable_repaired": 0, 2024-06-10T03:04:22.742 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_dir_inodes": 0, 2024-06-10T03:04:22.742 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_dir_base_inodes": 0, 2024-06-10T03:04:22.742 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_dirfrag_rstats": 0, 2024-06-10T03:04:22.743 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_file_inodes": 0, 2024-06-10T03:04:22.743 INFO:teuthology.orchestra.run.smithi120.stdout: "handle_inode_file_caps": 0, 2024-06-10T03:04:22.743 INFO:teuthology.orchestra.run.smithi120.stdout: "ceph_cap_op_revoke": 0, 2024-06-10T03:04:22.743 INFO:teuthology.orchestra.run.smithi120.stdout: "ceph_cap_op_grant": 0, 2024-06-10T03:04:22.743 INFO:teuthology.orchestra.run.smithi120.stdout: "ceph_cap_op_trunc": 0, 2024-06-10T03:04:22.743 INFO:teuthology.orchestra.run.smithi120.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-06-10T03:04:22.743 INFO:teuthology.orchestra.run.smithi120.stdout: "ceph_cap_op_flush_ack": 0, 2024-06-10T03:04:22.743 INFO:teuthology.orchestra.run.smithi120.stdout: "handle_client_caps": 0, 2024-06-10T03:04:22.743 INFO:teuthology.orchestra.run.smithi120.stdout: "handle_client_caps_dirty": 0, 2024-06-10T03:04:22.743 INFO:teuthology.orchestra.run.smithi120.stdout: "handle_client_cap_release": 0, 2024-06-10T03:04:22.743 INFO:teuthology.orchestra.run.smithi120.stdout: "process_request_cap_release": 0 2024-06-10T03:04:22.743 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:22.743 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_cache": { 2024-06-10T03:04:22.743 INFO:teuthology.orchestra.run.smithi120.stdout: "dir_update": 0, 2024-06-10T03:04:22.743 INFO:teuthology.orchestra.run.smithi120.stdout: "dir_update_receipt": 0, 2024-06-10T03:04:22.744 INFO:teuthology.orchestra.run.smithi120.stdout: "dir_try_discover": 0, 2024-06-10T03:04:22.744 INFO:teuthology.orchestra.run.smithi120.stdout: "dir_send_discover": 0, 2024-06-10T03:04:22.744 INFO:teuthology.orchestra.run.smithi120.stdout: "dir_handle_discover": 0, 2024-06-10T03:04:22.744 INFO:teuthology.orchestra.run.smithi120.stdout: "num_strays": 0, 2024-06-10T03:04:22.744 INFO:teuthology.orchestra.run.smithi120.stdout: "num_strays_delayed": 0, 2024-06-10T03:04:22.744 INFO:teuthology.orchestra.run.smithi120.stdout: "num_strays_enqueuing": 0, 2024-06-10T03:04:22.744 INFO:teuthology.orchestra.run.smithi120.stdout: "strays_created": 0, 2024-06-10T03:04:22.744 INFO:teuthology.orchestra.run.smithi120.stdout: "strays_enqueued": 0, 2024-06-10T03:04:22.744 INFO:teuthology.orchestra.run.smithi120.stdout: "strays_reintegrated": 0, 2024-06-10T03:04:22.744 INFO:teuthology.orchestra.run.smithi120.stdout: "strays_migrated": 0, 2024-06-10T03:04:22.744 INFO:teuthology.orchestra.run.smithi120.stdout: "num_recovering_processing": 0, 2024-06-10T03:04:22.744 INFO:teuthology.orchestra.run.smithi120.stdout: "num_recovering_enqueued": 0, 2024-06-10T03:04:22.744 INFO:teuthology.orchestra.run.smithi120.stdout: "num_recovering_prioritized": 0, 2024-06-10T03:04:22.745 INFO:teuthology.orchestra.run.smithi120.stdout: "recovery_started": 0, 2024-06-10T03:04:22.745 INFO:teuthology.orchestra.run.smithi120.stdout: "recovery_completed": 0, 2024-06-10T03:04:22.745 INFO:teuthology.orchestra.run.smithi120.stdout: "ireq_quiesce_path": 0, 2024-06-10T03:04:22.745 INFO:teuthology.orchestra.run.smithi120.stdout: "ireq_quiesce_inode": 0, 2024-06-10T03:04:22.745 INFO:teuthology.orchestra.run.smithi120.stdout: "ireq_enqueue_scrub": 0, 2024-06-10T03:04:22.745 INFO:teuthology.orchestra.run.smithi120.stdout: "ireq_exportdir": 0, 2024-06-10T03:04:22.745 INFO:teuthology.orchestra.run.smithi120.stdout: "ireq_flush": 0, 2024-06-10T03:04:22.745 INFO:teuthology.orchestra.run.smithi120.stdout: "ireq_fragmentdir": 0, 2024-06-10T03:04:22.745 INFO:teuthology.orchestra.run.smithi120.stdout: "ireq_fragstats": 0, 2024-06-10T03:04:22.745 INFO:teuthology.orchestra.run.smithi120.stdout: "ireq_inodestats": 0 2024-06-10T03:04:22.745 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:22.745 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_log": { 2024-06-10T03:04:22.745 INFO:teuthology.orchestra.run.smithi120.stdout: "evlrg": 0, 2024-06-10T03:04:22.745 INFO:teuthology.orchestra.run.smithi120.stdout: "evadd": 0, 2024-06-10T03:04:22.746 INFO:teuthology.orchestra.run.smithi120.stdout: "evex": 0, 2024-06-10T03:04:22.746 INFO:teuthology.orchestra.run.smithi120.stdout: "evtrm": 0, 2024-06-10T03:04:22.746 INFO:teuthology.orchestra.run.smithi120.stdout: "ev": 232591, 2024-06-10T03:04:22.746 INFO:teuthology.orchestra.run.smithi120.stdout: "evexg": 0, 2024-06-10T03:04:22.746 INFO:teuthology.orchestra.run.smithi120.stdout: "evexd": 0, 2024-06-10T03:04:22.746 INFO:teuthology.orchestra.run.smithi120.stdout: "segadd": 0, 2024-06-10T03:04:22.746 INFO:teuthology.orchestra.run.smithi120.stdout: "segex": 0, 2024-06-10T03:04:22.746 INFO:teuthology.orchestra.run.smithi120.stdout: "segtrm": 0, 2024-06-10T03:04:22.746 INFO:teuthology.orchestra.run.smithi120.stdout: "seg": 141, 2024-06-10T03:04:22.746 INFO:teuthology.orchestra.run.smithi120.stdout: "segmjr": 20, 2024-06-10T03:04:22.746 INFO:teuthology.orchestra.run.smithi120.stdout: "segexg": 0, 2024-06-10T03:04:22.746 INFO:teuthology.orchestra.run.smithi120.stdout: "segexd": 0, 2024-06-10T03:04:22.746 INFO:teuthology.orchestra.run.smithi120.stdout: "expos": 297215900, 2024-06-10T03:04:22.747 INFO:teuthology.orchestra.run.smithi120.stdout: "wrpos": 504464206, 2024-06-10T03:04:22.747 INFO:teuthology.orchestra.run.smithi120.stdout: "rdpos": 504463068, 2024-06-10T03:04:22.747 INFO:teuthology.orchestra.run.smithi120.stdout: "jlat": { 2024-06-10T03:04:22.747 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-10T03:04:22.747 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-10T03:04:22.747 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-10T03:04:22.747 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:22.747 INFO:teuthology.orchestra.run.smithi120.stdout: "replayed": 232591 2024-06-10T03:04:22.747 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:22.747 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_mem": { 2024-06-10T03:04:22.747 INFO:teuthology.orchestra.run.smithi120.stdout: "ino": 14, 2024-06-10T03:04:22.747 INFO:teuthology.orchestra.run.smithi120.stdout: "ino+": 37505, 2024-06-10T03:04:22.747 INFO:teuthology.orchestra.run.smithi120.stdout: "ino-": 37491, 2024-06-10T03:04:22.747 INFO:teuthology.orchestra.run.smithi120.stdout: "dir": 19, 2024-06-10T03:04:22.748 INFO:teuthology.orchestra.run.smithi120.stdout: "dir+": 1173, 2024-06-10T03:04:22.748 INFO:teuthology.orchestra.run.smithi120.stdout: "dir-": 1154, 2024-06-10T03:04:22.748 INFO:teuthology.orchestra.run.smithi120.stdout: "dn": 36629, 2024-06-10T03:04:22.748 INFO:teuthology.orchestra.run.smithi120.stdout: "dn+": 79885, 2024-06-10T03:04:22.748 INFO:teuthology.orchestra.run.smithi120.stdout: "dn-": 43256, 2024-06-10T03:04:22.748 INFO:teuthology.orchestra.run.smithi120.stdout: "cap": 0, 2024-06-10T03:04:22.748 INFO:teuthology.orchestra.run.smithi120.stdout: "cap+": 0, 2024-06-10T03:04:22.748 INFO:teuthology.orchestra.run.smithi120.stdout: "cap-": 0, 2024-06-10T03:04:22.748 INFO:teuthology.orchestra.run.smithi120.stdout: "rss": 122224, 2024-06-10T03:04:22.748 INFO:teuthology.orchestra.run.smithi120.stdout: "heap": 215420 2024-06-10T03:04:22.748 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:22.748 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_server": { 2024-06-10T03:04:22.748 INFO:teuthology.orchestra.run.smithi120.stdout: "dispatch_client_request": 0, 2024-06-10T03:04:22.748 INFO:teuthology.orchestra.run.smithi120.stdout: "dispatch_server_request": 0, 2024-06-10T03:04:22.749 INFO:teuthology.orchestra.run.smithi120.stdout: "handle_client_request": 0, 2024-06-10T03:04:22.749 INFO:teuthology.orchestra.run.smithi120.stdout: "handle_client_session": 0, 2024-06-10T03:04:22.749 INFO:teuthology.orchestra.run.smithi120.stdout: "handle_peer_request": 0, 2024-06-10T03:04:22.749 INFO:teuthology.orchestra.run.smithi120.stdout: "req_create_latency": { 2024-06-10T03:04:22.749 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-10T03:04:22.749 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-10T03:04:22.749 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-10T03:04:22.749 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:22.749 INFO:teuthology.orchestra.run.smithi120.stdout: "req_getattr_latency": { 2024-06-10T03:04:22.749 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-10T03:04:22.749 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-10T03:04:22.749 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-10T03:04:22.749 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:22.750 INFO:teuthology.orchestra.run.smithi120.stdout: "req_getfilelock_latency": { 2024-06-10T03:04:22.750 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-10T03:04:22.750 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-10T03:04:22.750 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-10T03:04:22.750 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:22.750 INFO:teuthology.orchestra.run.smithi120.stdout: "req_link_latency": { 2024-06-10T03:04:22.750 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-10T03:04:22.750 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-10T03:04:22.750 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-10T03:04:22.750 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:22.750 INFO:teuthology.orchestra.run.smithi120.stdout: "req_lookup_latency": { 2024-06-10T03:04:22.750 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-10T03:04:22.750 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-10T03:04:22.751 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-10T03:04:22.751 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:22.751 INFO:teuthology.orchestra.run.smithi120.stdout: "req_lookuphash_latency": { 2024-06-10T03:04:22.751 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-10T03:04:22.751 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-10T03:04:22.751 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-10T03:04:22.751 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:22.751 INFO:teuthology.orchestra.run.smithi120.stdout: "req_lookupino_latency": { 2024-06-10T03:04:22.751 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-10T03:04:22.751 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-10T03:04:22.751 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-10T03:04:22.751 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:22.751 INFO:teuthology.orchestra.run.smithi120.stdout: "req_lookupname_latency": { 2024-06-10T03:04:22.751 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-10T03:04:22.751 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-10T03:04:22.752 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-10T03:04:22.752 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:22.752 INFO:teuthology.orchestra.run.smithi120.stdout: "req_lookupparent_latency": { 2024-06-10T03:04:22.752 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-10T03:04:22.752 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-10T03:04:22.752 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-10T03:04:22.752 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:22.752 INFO:teuthology.orchestra.run.smithi120.stdout: "req_lookupsnap_latency": { 2024-06-10T03:04:22.752 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-10T03:04:22.752 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-10T03:04:22.752 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-10T03:04:22.752 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:22.752 INFO:teuthology.orchestra.run.smithi120.stdout: "req_lssnap_latency": { 2024-06-10T03:04:22.752 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-10T03:04:22.753 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-10T03:04:22.753 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-10T03:04:22.753 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:22.753 INFO:teuthology.orchestra.run.smithi120.stdout: "req_mkdir_latency": { 2024-06-10T03:04:22.753 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-10T03:04:22.753 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-10T03:04:22.753 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-10T03:04:22.753 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:22.753 INFO:teuthology.orchestra.run.smithi120.stdout: "req_mknod_latency": { 2024-06-10T03:04:22.753 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-10T03:04:22.753 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-10T03:04:22.753 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-10T03:04:22.753 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:22.753 INFO:teuthology.orchestra.run.smithi120.stdout: "req_mksnap_latency": { 2024-06-10T03:04:22.754 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-10T03:04:22.754 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-10T03:04:22.754 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-10T03:04:22.754 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:22.754 INFO:teuthology.orchestra.run.smithi120.stdout: "req_open_latency": { 2024-06-10T03:04:22.754 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-10T03:04:22.754 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-10T03:04:22.754 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-10T03:04:22.754 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:22.754 INFO:teuthology.orchestra.run.smithi120.stdout: "req_readdir_latency": { 2024-06-10T03:04:22.754 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-10T03:04:22.754 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-10T03:04:22.755 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-10T03:04:22.755 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:22.755 INFO:teuthology.orchestra.run.smithi120.stdout: "req_rename_latency": { 2024-06-10T03:04:22.755 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-10T03:04:22.755 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-10T03:04:22.755 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-10T03:04:22.755 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:22.755 INFO:teuthology.orchestra.run.smithi120.stdout: "req_renamesnap_latency": { 2024-06-10T03:04:22.755 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-10T03:04:22.755 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-10T03:04:22.755 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-10T03:04:22.755 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:22.755 INFO:teuthology.orchestra.run.smithi120.stdout: "req_snapdiff_latency": { 2024-06-10T03:04:22.755 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-10T03:04:22.756 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-10T03:04:22.756 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-10T03:04:22.756 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:22.756 INFO:teuthology.orchestra.run.smithi120.stdout: "req_rmdir_latency": { 2024-06-10T03:04:22.756 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-10T03:04:22.756 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-10T03:04:22.756 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-10T03:04:22.756 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:22.756 INFO:teuthology.orchestra.run.smithi120.stdout: "req_rmsnap_latency": { 2024-06-10T03:04:22.756 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-10T03:04:22.756 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-10T03:04:22.756 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-10T03:04:22.756 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:22.756 INFO:teuthology.orchestra.run.smithi120.stdout: "req_rmxattr_latency": { 2024-06-10T03:04:22.757 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-10T03:04:22.757 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-10T03:04:22.757 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-10T03:04:22.757 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:22.757 INFO:teuthology.orchestra.run.smithi120.stdout: "req_setattr_latency": { 2024-06-10T03:04:22.757 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-10T03:04:22.757 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-10T03:04:22.757 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-10T03:04:22.757 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:22.757 INFO:teuthology.orchestra.run.smithi120.stdout: "req_setdirlayout_latency": { 2024-06-10T03:04:22.757 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-10T03:04:22.757 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-10T03:04:22.757 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-10T03:04:22.758 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:22.758 INFO:teuthology.orchestra.run.smithi120.stdout: "req_setfilelock_latency": { 2024-06-10T03:04:22.758 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-10T03:04:22.758 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-10T03:04:22.758 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-10T03:04:22.758 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:22.758 INFO:teuthology.orchestra.run.smithi120.stdout: "req_setlayout_latency": { 2024-06-10T03:04:22.758 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-10T03:04:22.758 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-10T03:04:22.758 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-10T03:04:22.758 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:22.758 INFO:teuthology.orchestra.run.smithi120.stdout: "req_setxattr_latency": { 2024-06-10T03:04:22.758 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-10T03:04:22.759 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-10T03:04:22.759 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-10T03:04:22.759 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:22.759 INFO:teuthology.orchestra.run.smithi120.stdout: "req_symlink_latency": { 2024-06-10T03:04:22.759 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-10T03:04:22.759 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-10T03:04:22.759 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-10T03:04:22.759 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:22.759 INFO:teuthology.orchestra.run.smithi120.stdout: "req_unlink_latency": { 2024-06-10T03:04:22.759 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-10T03:04:22.759 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-10T03:04:22.759 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-10T03:04:22.759 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:22.760 INFO:teuthology.orchestra.run.smithi120.stdout: "cap_revoke_eviction": 0, 2024-06-10T03:04:22.760 INFO:teuthology.orchestra.run.smithi120.stdout: "cap_acquisition_throttle": 0, 2024-06-10T03:04:22.760 INFO:teuthology.orchestra.run.smithi120.stdout: "req_getvxattr_latency": { 2024-06-10T03:04:22.760 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-10T03:04:22.760 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-10T03:04:22.760 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-10T03:04:22.760 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-06-10T03:04:22.760 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:22.760 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_sessions": { 2024-06-10T03:04:22.760 INFO:teuthology.orchestra.run.smithi120.stdout: "session_count": 1, 2024-06-10T03:04:22.760 INFO:teuthology.orchestra.run.smithi120.stdout: "session_add": 2, 2024-06-10T03:04:22.760 INFO:teuthology.orchestra.run.smithi120.stdout: "session_remove": 1, 2024-06-10T03:04:22.760 INFO:teuthology.orchestra.run.smithi120.stdout: "sessions_open": 1, 2024-06-10T03:04:22.761 INFO:teuthology.orchestra.run.smithi120.stdout: "sessions_stale": 0, 2024-06-10T03:04:22.761 INFO:teuthology.orchestra.run.smithi120.stdout: "total_load": 0, 2024-06-10T03:04:22.761 INFO:teuthology.orchestra.run.smithi120.stdout: "average_load": 0, 2024-06-10T03:04:22.761 INFO:teuthology.orchestra.run.smithi120.stdout: "avg_session_uptime": 2343, 2024-06-10T03:04:22.761 INFO:teuthology.orchestra.run.smithi120.stdout: "mdthresh_evicted": 0 2024-06-10T03:04:22.761 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:22.761 INFO:teuthology.orchestra.run.smithi120.stdout: "mempool": { 2024-06-10T03:04:22.761 INFO:teuthology.orchestra.run.smithi120.stdout: "bloom_filter_bytes": 0, 2024-06-10T03:04:22.761 INFO:teuthology.orchestra.run.smithi120.stdout: "bloom_filter_items": 0, 2024-06-10T03:04:22.761 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_alloc_bytes": 0, 2024-06-10T03:04:22.761 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_alloc_items": 0, 2024-06-10T03:04:22.761 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_data_bytes": 0, 2024-06-10T03:04:22.761 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_data_items": 0, 2024-06-10T03:04:22.761 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-10T03:04:22.762 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_onode_items": 0, 2024-06-10T03:04:22.762 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-10T03:04:22.762 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_meta_items": 0, 2024-06-10T03:04:22.762 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_other_bytes": 0, 2024-06-10T03:04:22.762 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_other_items": 0, 2024-06-10T03:04:22.762 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-10T03:04:22.762 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_buffer_items": 0, 2024-06-10T03:04:22.762 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_extent_bytes": 0, 2024-06-10T03:04:22.762 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_extent_items": 0, 2024-06-10T03:04:22.762 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_blob_bytes": 0, 2024-06-10T03:04:22.762 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_blob_items": 0, 2024-06-10T03:04:22.762 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-10T03:04:22.762 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_shared_blob_items": 0, 2024-06-10T03:04:22.763 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-10T03:04:22.763 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_inline_bl_items": 0, 2024-06-10T03:04:22.763 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_fsck_bytes": 0, 2024-06-10T03:04:22.763 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_fsck_items": 0, 2024-06-10T03:04:22.763 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_txc_bytes": 0, 2024-06-10T03:04:22.763 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_txc_items": 0, 2024-06-10T03:04:22.763 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-10T03:04:22.763 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_writing_deferred_items": 0, 2024-06-10T03:04:22.763 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_writing_bytes": 0, 2024-06-10T03:04:22.763 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_writing_items": 0, 2024-06-10T03:04:22.763 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_bytes": 0, 2024-06-10T03:04:22.763 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_items": 0, 2024-06-10T03:04:22.763 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_file_reader_bytes": 0, 2024-06-10T03:04:22.763 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_file_reader_items": 0, 2024-06-10T03:04:22.764 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_file_writer_bytes": 0, 2024-06-10T03:04:22.764 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_file_writer_items": 0, 2024-06-10T03:04:22.764 INFO:teuthology.orchestra.run.smithi120.stdout: "buffer_anon_bytes": 232189, 2024-06-10T03:04:22.764 INFO:teuthology.orchestra.run.smithi120.stdout: "buffer_anon_items": 78, 2024-06-10T03:04:22.764 INFO:teuthology.orchestra.run.smithi120.stdout: "buffer_meta_bytes": 0, 2024-06-10T03:04:22.764 INFO:teuthology.orchestra.run.smithi120.stdout: "buffer_meta_items": 0, 2024-06-10T03:04:22.764 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_bytes": 0, 2024-06-10T03:04:22.764 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_items": 0, 2024-06-10T03:04:22.764 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_mapbl_bytes": 0, 2024-06-10T03:04:22.764 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_mapbl_items": 0, 2024-06-10T03:04:22.764 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_pglog_bytes": 0, 2024-06-10T03:04:22.764 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_pglog_items": 0, 2024-06-10T03:04:22.764 INFO:teuthology.orchestra.run.smithi120.stdout: "osdmap_bytes": 8984, 2024-06-10T03:04:22.764 INFO:teuthology.orchestra.run.smithi120.stdout: "osdmap_items": 172, 2024-06-10T03:04:22.765 INFO:teuthology.orchestra.run.smithi120.stdout: "osdmap_mapping_bytes": 0, 2024-06-10T03:04:22.765 INFO:teuthology.orchestra.run.smithi120.stdout: "osdmap_mapping_items": 0, 2024-06-10T03:04:22.765 INFO:teuthology.orchestra.run.smithi120.stdout: "pgmap_bytes": 0, 2024-06-10T03:04:22.765 INFO:teuthology.orchestra.run.smithi120.stdout: "pgmap_items": 0, 2024-06-10T03:04:22.765 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_co_bytes": 25843344, 2024-06-10T03:04:22.765 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_co_items": 147168, 2024-06-10T03:04:22.765 INFO:teuthology.orchestra.run.smithi120.stdout: "unittest_1_bytes": 0, 2024-06-10T03:04:22.765 INFO:teuthology.orchestra.run.smithi120.stdout: "unittest_1_items": 0, 2024-06-10T03:04:22.765 INFO:teuthology.orchestra.run.smithi120.stdout: "unittest_2_bytes": 0, 2024-06-10T03:04:22.765 INFO:teuthology.orchestra.run.smithi120.stdout: "unittest_2_items": 0 2024-06-10T03:04:22.765 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:22.765 INFO:teuthology.orchestra.run.smithi120.stdout: "objecter": { 2024-06-10T03:04:22.765 INFO:teuthology.orchestra.run.smithi120.stdout: "op_active": 0, 2024-06-10T03:04:22.765 INFO:teuthology.orchestra.run.smithi120.stdout: "op_laggy": 0, 2024-06-10T03:04:22.766 INFO:teuthology.orchestra.run.smithi120.stdout: "op_send": 3205, 2024-06-10T03:04:22.766 INFO:teuthology.orchestra.run.smithi120.stdout: "op_send_bytes": 21, 2024-06-10T03:04:22.766 INFO:teuthology.orchestra.run.smithi120.stdout: "op_resend": 3, 2024-06-10T03:04:22.766 INFO:teuthology.orchestra.run.smithi120.stdout: "op_reply": 3205, 2024-06-10T03:04:22.766 INFO:teuthology.orchestra.run.smithi120.stdout: "op_latency": { 2024-06-10T03:04:22.766 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 3205, 2024-06-10T03:04:22.766 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 7.745826221, 2024-06-10T03:04:22.766 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.002416794 2024-06-10T03:04:22.766 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:22.766 INFO:teuthology.orchestra.run.smithi120.stdout: "op_inflight": 0, 2024-06-10T03:04:22.766 INFO:teuthology.orchestra.run.smithi120.stdout: "oplen_avg": { 2024-06-10T03:04:22.766 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 3205, 2024-06-10T03:04:22.766 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 3206 2024-06-10T03:04:22.766 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:22.767 INFO:teuthology.orchestra.run.smithi120.stdout: "op": 3205, 2024-06-10T03:04:22.767 INFO:teuthology.orchestra.run.smithi120.stdout: "op_r": 3205, 2024-06-10T03:04:22.767 INFO:teuthology.orchestra.run.smithi120.stdout: "op_w": 0, 2024-06-10T03:04:22.767 INFO:teuthology.orchestra.run.smithi120.stdout: "op_rmw": 0, 2024-06-10T03:04:22.767 INFO:teuthology.orchestra.run.smithi120.stdout: "op_pg": 0, 2024-06-10T03:04:22.767 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_stat": 1975, 2024-06-10T03:04:22.767 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_create": 0, 2024-06-10T03:04:22.767 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_read": 1228, 2024-06-10T03:04:22.767 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_write": 0, 2024-06-10T03:04:22.767 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_writefull": 0, 2024-06-10T03:04:22.767 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_writesame": 0, 2024-06-10T03:04:22.767 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_append": 0, 2024-06-10T03:04:22.767 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_zero": 0, 2024-06-10T03:04:22.767 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_truncate": 0, 2024-06-10T03:04:22.768 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_delete": 0, 2024-06-10T03:04:22.768 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_mapext": 0, 2024-06-10T03:04:22.768 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_sparse_read": 0, 2024-06-10T03:04:22.768 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_clonerange": 0, 2024-06-10T03:04:22.768 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_getxattr": 1, 2024-06-10T03:04:22.768 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_setxattr": 0, 2024-06-10T03:04:22.768 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_cmpxattr": 0, 2024-06-10T03:04:22.768 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_rmxattr": 0, 2024-06-10T03:04:22.768 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_resetxattrs": 0, 2024-06-10T03:04:22.768 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_call": 0, 2024-06-10T03:04:22.768 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_watch": 0, 2024-06-10T03:04:22.768 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_notify": 0, 2024-06-10T03:04:22.768 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_src_cmpxattr": 0, 2024-06-10T03:04:22.768 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_pgls": 0, 2024-06-10T03:04:22.769 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_pgls_filter": 0, 2024-06-10T03:04:22.769 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_other": 0, 2024-06-10T03:04:22.769 INFO:teuthology.orchestra.run.smithi120.stdout: "linger_active": 0, 2024-06-10T03:04:22.769 INFO:teuthology.orchestra.run.smithi120.stdout: "linger_send": 0, 2024-06-10T03:04:22.769 INFO:teuthology.orchestra.run.smithi120.stdout: "linger_resend": 0, 2024-06-10T03:04:22.769 INFO:teuthology.orchestra.run.smithi120.stdout: "linger_ping": 0, 2024-06-10T03:04:22.769 INFO:teuthology.orchestra.run.smithi120.stdout: "poolop_active": 0, 2024-06-10T03:04:22.769 INFO:teuthology.orchestra.run.smithi120.stdout: "poolop_send": 0, 2024-06-10T03:04:22.769 INFO:teuthology.orchestra.run.smithi120.stdout: "poolop_resend": 0, 2024-06-10T03:04:22.769 INFO:teuthology.orchestra.run.smithi120.stdout: "poolstat_active": 0, 2024-06-10T03:04:22.769 INFO:teuthology.orchestra.run.smithi120.stdout: "poolstat_send": 0, 2024-06-10T03:04:22.769 INFO:teuthology.orchestra.run.smithi120.stdout: "poolstat_resend": 0, 2024-06-10T03:04:22.769 INFO:teuthology.orchestra.run.smithi120.stdout: "statfs_active": 0, 2024-06-10T03:04:22.769 INFO:teuthology.orchestra.run.smithi120.stdout: "statfs_send": 0, 2024-06-10T03:04:22.770 INFO:teuthology.orchestra.run.smithi120.stdout: "statfs_resend": 0, 2024-06-10T03:04:22.770 INFO:teuthology.orchestra.run.smithi120.stdout: "command_active": 0, 2024-06-10T03:04:22.770 INFO:teuthology.orchestra.run.smithi120.stdout: "command_send": 0, 2024-06-10T03:04:22.770 INFO:teuthology.orchestra.run.smithi120.stdout: "command_resend": 0, 2024-06-10T03:04:22.770 INFO:teuthology.orchestra.run.smithi120.stdout: "map_epoch": 100, 2024-06-10T03:04:22.770 INFO:teuthology.orchestra.run.smithi120.stdout: "map_full": 0, 2024-06-10T03:04:22.770 INFO:teuthology.orchestra.run.smithi120.stdout: "map_inc": 2, 2024-06-10T03:04:22.770 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_sessions": 10, 2024-06-10T03:04:22.770 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_session_open": 10, 2024-06-10T03:04:22.770 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_session_close": 0, 2024-06-10T03:04:22.770 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_laggy": 0, 2024-06-10T03:04:22.770 INFO:teuthology.orchestra.run.smithi120.stdout: "omap_wr": 0, 2024-06-10T03:04:22.770 INFO:teuthology.orchestra.run.smithi120.stdout: "omap_rd": 2, 2024-06-10T03:04:22.771 INFO:teuthology.orchestra.run.smithi120.stdout: "omap_del": 0 2024-06-10T03:04:22.771 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:22.771 INFO:teuthology.orchestra.run.smithi120.stdout: "oft": { 2024-06-10T03:04:22.771 INFO:teuthology.orchestra.run.smithi120.stdout: "omap_total_objs": 0, 2024-06-10T03:04:22.771 INFO:teuthology.orchestra.run.smithi120.stdout: "omap_total_kv_pairs": 0, 2024-06-10T03:04:22.771 INFO:teuthology.orchestra.run.smithi120.stdout: "omap_total_updates": 0, 2024-06-10T03:04:22.771 INFO:teuthology.orchestra.run.smithi120.stdout: "omap_total_removes": 0 2024-06-10T03:04:22.771 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:22.771 INFO:teuthology.orchestra.run.smithi120.stdout: "osd-slow-ops": { 2024-06-10T03:04:22.771 INFO:teuthology.orchestra.run.smithi120.stdout: "slow_ops_count": 0 2024-06-10T03:04:22.771 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:22.771 INFO:teuthology.orchestra.run.smithi120.stdout: "purge_queue": { 2024-06-10T03:04:22.771 INFO:teuthology.orchestra.run.smithi120.stdout: "pq_executing_ops": 0, 2024-06-10T03:04:22.771 INFO:teuthology.orchestra.run.smithi120.stdout: "pq_executing_ops_high_water": 0, 2024-06-10T03:04:22.772 INFO:teuthology.orchestra.run.smithi120.stdout: "pq_executing": 0, 2024-06-10T03:04:22.772 INFO:teuthology.orchestra.run.smithi120.stdout: "pq_executing_high_water": 0, 2024-06-10T03:04:22.772 INFO:teuthology.orchestra.run.smithi120.stdout: "pq_executed_ops": 0, 2024-06-10T03:04:22.772 INFO:teuthology.orchestra.run.smithi120.stdout: "pq_executed": 0, 2024-06-10T03:04:22.772 INFO:teuthology.orchestra.run.smithi120.stdout: "pq_item_in_journal": 0 2024-06-10T03:04:22.772 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:22.772 INFO:teuthology.orchestra.run.smithi120.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-10T03:04:22.772 INFO:teuthology.orchestra.run.smithi120.stdout: "val": 0, 2024-06-10T03:04:22.772 INFO:teuthology.orchestra.run.smithi120.stdout: "max": 104857600, 2024-06-10T03:04:22.772 INFO:teuthology.orchestra.run.smithi120.stdout: "get_started": 0, 2024-06-10T03:04:22.772 INFO:teuthology.orchestra.run.smithi120.stdout: "get": 3481, 2024-06-10T03:04:22.772 INFO:teuthology.orchestra.run.smithi120.stdout: "get_sum": 501791674, 2024-06-10T03:04:22.772 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_fail": 0, 2024-06-10T03:04:22.772 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_success": 3481, 2024-06-10T03:04:22.773 INFO:teuthology.orchestra.run.smithi120.stdout: "take": 0, 2024-06-10T03:04:22.773 INFO:teuthology.orchestra.run.smithi120.stdout: "take_sum": 0, 2024-06-10T03:04:22.773 INFO:teuthology.orchestra.run.smithi120.stdout: "put": 3481, 2024-06-10T03:04:22.773 INFO:teuthology.orchestra.run.smithi120.stdout: "put_sum": 501791674, 2024-06-10T03:04:22.773 INFO:teuthology.orchestra.run.smithi120.stdout: "wait": { 2024-06-10T03:04:22.773 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-10T03:04:22.773 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-10T03:04:22.773 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-10T03:04:22.773 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-06-10T03:04:22.773 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:22.773 INFO:teuthology.orchestra.run.smithi120.stdout: "throttle-objecter_bytes": { 2024-06-10T03:04:22.773 INFO:teuthology.orchestra.run.smithi120.stdout: "val": 0, 2024-06-10T03:04:22.773 INFO:teuthology.orchestra.run.smithi120.stdout: "max": 104857600, 2024-06-10T03:04:22.773 INFO:teuthology.orchestra.run.smithi120.stdout: "get_started": 0, 2024-06-10T03:04:22.774 INFO:teuthology.orchestra.run.smithi120.stdout: "get": 0, 2024-06-10T03:04:22.774 INFO:teuthology.orchestra.run.smithi120.stdout: "get_sum": 0, 2024-06-10T03:04:22.774 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_fail": 0, 2024-06-10T03:04:22.774 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_success": 0, 2024-06-10T03:04:22.774 INFO:teuthology.orchestra.run.smithi120.stdout: "take": 3205, 2024-06-10T03:04:22.774 INFO:teuthology.orchestra.run.smithi120.stdout: "take_sum": 500326999, 2024-06-10T03:04:22.774 INFO:teuthology.orchestra.run.smithi120.stdout: "put": 3205, 2024-06-10T03:04:22.774 INFO:teuthology.orchestra.run.smithi120.stdout: "put_sum": 500326999, 2024-06-10T03:04:22.774 INFO:teuthology.orchestra.run.smithi120.stdout: "wait": { 2024-06-10T03:04:22.774 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-10T03:04:22.774 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-10T03:04:22.774 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-10T03:04:22.775 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-06-10T03:04:22.775 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:22.775 INFO:teuthology.orchestra.run.smithi120.stdout: "throttle-objecter_ops": { 2024-06-10T03:04:22.775 INFO:teuthology.orchestra.run.smithi120.stdout: "val": 0, 2024-06-10T03:04:22.775 INFO:teuthology.orchestra.run.smithi120.stdout: "max": 1024, 2024-06-10T03:04:22.775 INFO:teuthology.orchestra.run.smithi120.stdout: "get_started": 0, 2024-06-10T03:04:22.775 INFO:teuthology.orchestra.run.smithi120.stdout: "get": 0, 2024-06-10T03:04:22.775 INFO:teuthology.orchestra.run.smithi120.stdout: "get_sum": 0, 2024-06-10T03:04:22.775 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_fail": 0, 2024-06-10T03:04:22.775 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_success": 0, 2024-06-10T03:04:22.775 INFO:teuthology.orchestra.run.smithi120.stdout: "take": 3205, 2024-06-10T03:04:22.775 INFO:teuthology.orchestra.run.smithi120.stdout: "take_sum": 3205, 2024-06-10T03:04:22.775 INFO:teuthology.orchestra.run.smithi120.stdout: "put": 3205, 2024-06-10T03:04:22.776 INFO:teuthology.orchestra.run.smithi120.stdout: "put_sum": 3205, 2024-06-10T03:04:22.776 INFO:teuthology.orchestra.run.smithi120.stdout: "wait": { 2024-06-10T03:04:22.776 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-10T03:04:22.776 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-10T03:04:22.776 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-10T03:04:22.776 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-06-10T03:04:22.776 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:22.776 INFO:teuthology.orchestra.run.smithi120.stdout: "throttle-write_buf_throttle": { 2024-06-10T03:04:22.776 INFO:teuthology.orchestra.run.smithi120.stdout: "val": 0, 2024-06-10T03:04:22.776 INFO:teuthology.orchestra.run.smithi120.stdout: "max": 3758096384, 2024-06-10T03:04:22.776 INFO:teuthology.orchestra.run.smithi120.stdout: "get_started": 0, 2024-06-10T03:04:22.776 INFO:teuthology.orchestra.run.smithi120.stdout: "get": 0, 2024-06-10T03:04:22.776 INFO:teuthology.orchestra.run.smithi120.stdout: "get_sum": 0, 2024-06-10T03:04:22.776 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_fail": 0, 2024-06-10T03:04:22.777 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_success": 0, 2024-06-10T03:04:22.777 INFO:teuthology.orchestra.run.smithi120.stdout: "take": 0, 2024-06-10T03:04:22.777 INFO:teuthology.orchestra.run.smithi120.stdout: "take_sum": 0, 2024-06-10T03:04:22.777 INFO:teuthology.orchestra.run.smithi120.stdout: "put": 0, 2024-06-10T03:04:22.777 INFO:teuthology.orchestra.run.smithi120.stdout: "put_sum": 0, 2024-06-10T03:04:22.777 INFO:teuthology.orchestra.run.smithi120.stdout: "wait": { 2024-06-10T03:04:22.777 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-10T03:04:22.777 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-10T03:04:22.777 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-10T03:04:22.777 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-06-10T03:04:22.777 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-10T03:04:22.777 INFO:teuthology.orchestra.run.smithi120.stdout: "throttle-write_buf_throttle-0x55b1fac5a6e0": { 2024-06-10T03:04:22.777 INFO:teuthology.orchestra.run.smithi120.stdout: "val": 0, 2024-06-10T03:04:22.777 INFO:teuthology.orchestra.run.smithi120.stdout: "max": 3758096384, 2024-06-10T03:04:22.777 INFO:teuthology.orchestra.run.smithi120.stdout: "get_started": 0, 2024-06-10T03:04:22.778 INFO:teuthology.orchestra.run.smithi120.stdout: "get": 0, 2024-06-10T03:04:22.778 INFO:teuthology.orchestra.run.smithi120.stdout: "get_sum": 0, 2024-06-10T03:04:22.778 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_fail": 0, 2024-06-10T03:04:22.778 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_success": 0, 2024-06-10T03:04:22.778 INFO:teuthology.orchestra.run.smithi120.stdout: "take": 0, 2024-06-10T03:04:22.778 INFO:teuthology.orchestra.run.smithi120.stdout: "take_sum": 0, 2024-06-10T03:04:22.778 INFO:teuthology.orchestra.run.smithi120.stdout: "put": 0, 2024-06-10T03:04:22.778 INFO:teuthology.orchestra.run.smithi120.stdout: "put_sum": 0, 2024-06-10T03:04:22.778 INFO:teuthology.orchestra.run.smithi120.stdout: "wait": { 2024-06-10T03:04:22.778 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-10T03:04:22.778 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-10T03:04:22.778 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-10T03:04:22.778 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-06-10T03:04:22.778 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-06-10T03:04:22.779 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-06-10T03:04:23.036 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:04:22 smithi120 ceph-mon[36634]: pgmap v739: 129 pgs: 129 active+clean; 228 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-06-10T03:04:23.204 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:04:22 smithi188 ceph-mon[36599]: pgmap v739: 129 pgs: 129 active+clean; 228 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-06-10T03:04:23.227 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:04:22 smithi052 ceph-mon[32362]: pgmap v739: 129 pgs: 129 active+clean; 228 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-06-10T03:04:23.550 INFO:tasks.check_counter:Daemon mds.k mds.exported=0 2024-06-10T03:04:23.550 INFO:tasks.check_counter:Daemon mds.k mds.imported=0 2024-06-10T03:04:23.550 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update=0 2024-06-10T03:04:23.550 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update_receipt=0 2024-06-10T03:04:23.550 INFO:tasks.check_counter:Daemon mds.k mds.dir_split=0 2024-06-10T03:04:23.551 DEBUG:tasks.check_counter:Getting stats from c 2024-06-10T03:04:23.551 DEBUG:teuthology.orchestra.run.smithi188:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph daemon mds.c perf dump 2024-06-10T03:04:23.905 INFO:teuthology.orchestra.run.smithi188.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.c/config 2024-06-10T03:04:25.204 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:04:24 smithi188 ceph-mon[36599]: pgmap v740: 129 pgs: 129 active+clean; 228 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-06-10T03:04:25.227 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:04:24 smithi052 ceph-mon[32362]: pgmap v740: 129 pgs: 129 active+clean; 228 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-06-10T03:04:25.388 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:04:24 smithi120 ceph-mon[36634]: pgmap v740: 129 pgs: 129 active+clean; 228 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-06-10T03:04:25.931 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-06-10T03:04:26.079 INFO:teuthology.orchestra.run.smithi188.stdout: "AsyncMessenger::Worker-0": { 2024-06-10T03:04:26.079 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_messages": 36597, 2024-06-10T03:04:26.079 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_messages": 42022, 2024-06-10T03:04:26.079 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_bytes": 13899749, 2024-06-10T03:04:26.079 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_bytes": 30192325, 2024-06-10T03:04:26.080 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_created_connections": 17, 2024-06-10T03:04:26.080 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_active_connections": 6, 2024-06-10T03:04:26.080 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_total_time": 1.915299112, 2024-06-10T03:04:26.080 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_send_time": 0.845015381, 2024-06-10T03:04:26.080 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_recv_time": 1.719097695, 2024-06-10T03:04:26.080 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_fast_dispatch_time": 0.046693782, 2024-06-10T03:04:26.080 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_messages_queue_lat": { 2024-06-10T03:04:26.080 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 42020, 2024-06-10T03:04:26.080 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 2.898064790, 2024-06-10T03:04:26.080 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000068968 2024-06-10T03:04:26.080 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:26.080 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_handle_ack_lat": { 2024-06-10T03:04:26.080 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 68022, 2024-06-10T03:04:26.080 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.010494281, 2024-06-10T03:04:26.081 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000154 2024-06-10T03:04:26.081 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:26.081 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_encrypted_bytes": 13508960, 2024-06-10T03:04:26.081 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_encrypted_bytes": 22076480 2024-06-10T03:04:26.081 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:26.081 INFO:teuthology.orchestra.run.smithi188.stdout: "AsyncMessenger::Worker-1": { 2024-06-10T03:04:26.081 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_messages": 2975, 2024-06-10T03:04:26.081 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_messages": 3107, 2024-06-10T03:04:26.081 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_bytes": 916714, 2024-06-10T03:04:26.081 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_bytes": 1666790, 2024-06-10T03:04:26.081 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_created_connections": 9, 2024-06-10T03:04:26.081 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_active_connections": 6, 2024-06-10T03:04:26.081 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_total_time": 0.218204023, 2024-06-10T03:04:26.081 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_send_time": 0.093072390, 2024-06-10T03:04:26.081 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_recv_time": 0.107964276, 2024-06-10T03:04:26.082 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_fast_dispatch_time": 0.025553389, 2024-06-10T03:04:26.082 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_messages_queue_lat": { 2024-06-10T03:04:26.082 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 3107, 2024-06-10T03:04:26.082 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.161290861, 2024-06-10T03:04:26.082 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000051912 2024-06-10T03:04:26.082 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:26.082 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_handle_ack_lat": { 2024-06-10T03:04:26.082 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 4092, 2024-06-10T03:04:26.082 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000862543, 2024-06-10T03:04:26.082 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000210 2024-06-10T03:04:26.082 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:26.082 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_encrypted_bytes": 160, 2024-06-10T03:04:26.083 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_encrypted_bytes": 380400 2024-06-10T03:04:26.083 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:26.083 INFO:teuthology.orchestra.run.smithi188.stdout: "AsyncMessenger::Worker-2": { 2024-06-10T03:04:26.083 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_messages": 11953, 2024-06-10T03:04:26.083 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_messages": 10453, 2024-06-10T03:04:26.083 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_bytes": 5566430, 2024-06-10T03:04:26.083 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_bytes": 3210252, 2024-06-10T03:04:26.083 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_created_connections": 24, 2024-06-10T03:04:26.083 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_active_connections": 7, 2024-06-10T03:04:26.083 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_total_time": 0.738658950, 2024-06-10T03:04:26.083 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_send_time": 0.318228141, 2024-06-10T03:04:26.083 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_recv_time": 0.463905813, 2024-06-10T03:04:26.083 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_fast_dispatch_time": 0.048398914, 2024-06-10T03:04:26.083 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_messages_queue_lat": { 2024-06-10T03:04:26.083 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 10451, 2024-06-10T03:04:26.084 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.374535731, 2024-06-10T03:04:26.084 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000035837 2024-06-10T03:04:26.084 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:26.084 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_handle_ack_lat": { 2024-06-10T03:04:26.084 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 18583, 2024-06-10T03:04:26.084 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.003135643, 2024-06-10T03:04:26.084 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000168 2024-06-10T03:04:26.084 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:26.084 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_encrypted_bytes": 0, 2024-06-10T03:04:26.084 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_encrypted_bytes": 0 2024-06-10T03:04:26.084 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:26.084 INFO:teuthology.orchestra.run.smithi188.stdout: "cct": { 2024-06-10T03:04:26.084 INFO:teuthology.orchestra.run.smithi188.stdout: "total_workers": 1, 2024-06-10T03:04:26.085 INFO:teuthology.orchestra.run.smithi188.stdout: "unhealthy_workers": 0 2024-06-10T03:04:26.085 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:26.085 INFO:teuthology.orchestra.run.smithi188.stdout: "finisher-MDSRank": { 2024-06-10T03:04:26.085 INFO:teuthology.orchestra.run.smithi188.stdout: "queue_len": 0, 2024-06-10T03:04:26.085 INFO:teuthology.orchestra.run.smithi188.stdout: "complete_latency": { 2024-06-10T03:04:26.085 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 1704, 2024-06-10T03:04:26.085 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 2.405334778, 2024-06-10T03:04:26.085 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.001411581 2024-06-10T03:04:26.085 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-06-10T03:04:26.085 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:26.085 INFO:teuthology.orchestra.run.smithi188.stdout: "finisher-PurgeQueue": { 2024-06-10T03:04:26.085 INFO:teuthology.orchestra.run.smithi188.stdout: "queue_len": 0, 2024-06-10T03:04:26.085 INFO:teuthology.orchestra.run.smithi188.stdout: "complete_latency": { 2024-06-10T03:04:26.086 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 1870, 2024-06-10T03:04:26.086 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.152335915, 2024-06-10T03:04:26.086 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000081463 2024-06-10T03:04:26.086 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-06-10T03:04:26.086 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:26.086 INFO:teuthology.orchestra.run.smithi188.stdout: "mds": { 2024-06-10T03:04:26.086 INFO:teuthology.orchestra.run.smithi188.stdout: "request": 26467, 2024-06-10T03:04:26.086 INFO:teuthology.orchestra.run.smithi188.stdout: "reply": 16903, 2024-06-10T03:04:26.086 INFO:teuthology.orchestra.run.smithi188.stdout: "reply_latency": { 2024-06-10T03:04:26.086 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 16903, 2024-06-10T03:04:26.086 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 502.815458359, 2024-06-10T03:04:26.086 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.029747113 2024-06-10T03:04:26.087 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:26.087 INFO:teuthology.orchestra.run.smithi188.stdout: "slow_reply": 0, 2024-06-10T03:04:26.087 INFO:teuthology.orchestra.run.smithi188.stdout: "forward": 9564, 2024-06-10T03:04:26.087 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_fetch_complete": 0, 2024-06-10T03:04:26.087 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_fetch_keys": 0, 2024-06-10T03:04:26.087 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_commit": 38, 2024-06-10T03:04:26.087 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_split": 3, 2024-06-10T03:04:26.087 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_merge": 0, 2024-06-10T03:04:26.087 INFO:teuthology.orchestra.run.smithi188.stdout: "inodes": 26, 2024-06-10T03:04:26.087 INFO:teuthology.orchestra.run.smithi188.stdout: "inodes_top": 7, 2024-06-10T03:04:26.087 INFO:teuthology.orchestra.run.smithi188.stdout: "inodes_bottom": 9, 2024-06-10T03:04:26.087 INFO:teuthology.orchestra.run.smithi188.stdout: "inodes_pin_tail": 10, 2024-06-10T03:04:26.087 INFO:teuthology.orchestra.run.smithi188.stdout: "inodes_pinned": 13, 2024-06-10T03:04:26.087 INFO:teuthology.orchestra.run.smithi188.stdout: "inodes_expired": 288, 2024-06-10T03:04:26.088 INFO:teuthology.orchestra.run.smithi188.stdout: "inodes_with_caps": 0, 2024-06-10T03:04:26.088 INFO:teuthology.orchestra.run.smithi188.stdout: "caps": 0, 2024-06-10T03:04:26.088 INFO:teuthology.orchestra.run.smithi188.stdout: "subtrees": 4, 2024-06-10T03:04:26.088 INFO:teuthology.orchestra.run.smithi188.stdout: "traverse": 45569, 2024-06-10T03:04:26.088 INFO:teuthology.orchestra.run.smithi188.stdout: "traverse_hit": 27822, 2024-06-10T03:04:26.088 INFO:teuthology.orchestra.run.smithi188.stdout: "traverse_forward": 9564, 2024-06-10T03:04:26.088 INFO:teuthology.orchestra.run.smithi188.stdout: "traverse_discover": 153, 2024-06-10T03:04:26.088 INFO:teuthology.orchestra.run.smithi188.stdout: "traverse_dir_fetch": 0, 2024-06-10T03:04:26.088 INFO:teuthology.orchestra.run.smithi188.stdout: "traverse_remote_ino": 0, 2024-06-10T03:04:26.088 INFO:teuthology.orchestra.run.smithi188.stdout: "traverse_lock": 242, 2024-06-10T03:04:26.088 INFO:teuthology.orchestra.run.smithi188.stdout: "load_cent": 0, 2024-06-10T03:04:26.088 INFO:teuthology.orchestra.run.smithi188.stdout: "q": 0, 2024-06-10T03:04:26.088 INFO:teuthology.orchestra.run.smithi188.stdout: "exported": 24, 2024-06-10T03:04:26.088 INFO:teuthology.orchestra.run.smithi188.stdout: "exported_inodes": 6, 2024-06-10T03:04:26.089 INFO:teuthology.orchestra.run.smithi188.stdout: "imported": 3, 2024-06-10T03:04:26.089 INFO:teuthology.orchestra.run.smithi188.stdout: "imported_inodes": 806, 2024-06-10T03:04:26.089 INFO:teuthology.orchestra.run.smithi188.stdout: "openino_dir_fetch": 0, 2024-06-10T03:04:26.089 INFO:teuthology.orchestra.run.smithi188.stdout: "openino_backtrace_fetch": 1395, 2024-06-10T03:04:26.089 INFO:teuthology.orchestra.run.smithi188.stdout: "openino_peer_discover": 0, 2024-06-10T03:04:26.089 INFO:teuthology.orchestra.run.smithi188.stdout: "root_rfiles": 3367, 2024-06-10T03:04:26.089 INFO:teuthology.orchestra.run.smithi188.stdout: "root_rbytes": 68050036, 2024-06-10T03:04:26.089 INFO:teuthology.orchestra.run.smithi188.stdout: "root_rsnaps": 0, 2024-06-10T03:04:26.089 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_backtrace_fetch": 1395, 2024-06-10T03:04:26.089 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_set_tag": 0, 2024-06-10T03:04:26.089 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_backtrace_repaired": 0, 2024-06-10T03:04:26.089 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_inotable_repaired": 0, 2024-06-10T03:04:26.089 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_dir_inodes": 0, 2024-06-10T03:04:26.089 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_dir_base_inodes": 0, 2024-06-10T03:04:26.090 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_dirfrag_rstats": 0, 2024-06-10T03:04:26.090 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_file_inodes": 1395, 2024-06-10T03:04:26.090 INFO:teuthology.orchestra.run.smithi188.stdout: "handle_inode_file_caps": 23, 2024-06-10T03:04:26.090 INFO:teuthology.orchestra.run.smithi188.stdout: "ceph_cap_op_revoke": 2799, 2024-06-10T03:04:26.090 INFO:teuthology.orchestra.run.smithi188.stdout: "ceph_cap_op_grant": 3315, 2024-06-10T03:04:26.090 INFO:teuthology.orchestra.run.smithi188.stdout: "ceph_cap_op_trunc": 0, 2024-06-10T03:04:26.090 INFO:teuthology.orchestra.run.smithi188.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-06-10T03:04:26.090 INFO:teuthology.orchestra.run.smithi188.stdout: "ceph_cap_op_flush_ack": 0, 2024-06-10T03:04:26.090 INFO:teuthology.orchestra.run.smithi188.stdout: "handle_client_caps": 3799, 2024-06-10T03:04:26.090 INFO:teuthology.orchestra.run.smithi188.stdout: "handle_client_caps_dirty": 902, 2024-06-10T03:04:26.090 INFO:teuthology.orchestra.run.smithi188.stdout: "handle_client_cap_release": 43, 2024-06-10T03:04:26.090 INFO:teuthology.orchestra.run.smithi188.stdout: "process_request_cap_release": 3304 2024-06-10T03:04:26.090 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:26.091 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_cache": { 2024-06-10T03:04:26.091 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_update": 691, 2024-06-10T03:04:26.091 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_update_receipt": 2070, 2024-06-10T03:04:26.091 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_try_discover": 1143, 2024-06-10T03:04:26.091 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_send_discover": 1352, 2024-06-10T03:04:26.091 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_handle_discover": 517, 2024-06-10T03:04:26.091 INFO:teuthology.orchestra.run.smithi188.stdout: "num_strays": 0, 2024-06-10T03:04:26.091 INFO:teuthology.orchestra.run.smithi188.stdout: "num_strays_delayed": 0, 2024-06-10T03:04:26.091 INFO:teuthology.orchestra.run.smithi188.stdout: "num_strays_enqueuing": 0, 2024-06-10T03:04:26.091 INFO:teuthology.orchestra.run.smithi188.stdout: "strays_created": 902, 2024-06-10T03:04:26.091 INFO:teuthology.orchestra.run.smithi188.stdout: "strays_enqueued": 902, 2024-06-10T03:04:26.091 INFO:teuthology.orchestra.run.smithi188.stdout: "strays_reintegrated": 0, 2024-06-10T03:04:26.091 INFO:teuthology.orchestra.run.smithi188.stdout: "strays_migrated": 0, 2024-06-10T03:04:26.091 INFO:teuthology.orchestra.run.smithi188.stdout: "num_recovering_processing": 0, 2024-06-10T03:04:26.092 INFO:teuthology.orchestra.run.smithi188.stdout: "num_recovering_enqueued": 0, 2024-06-10T03:04:26.092 INFO:teuthology.orchestra.run.smithi188.stdout: "num_recovering_prioritized": 0, 2024-06-10T03:04:26.092 INFO:teuthology.orchestra.run.smithi188.stdout: "recovery_started": 0, 2024-06-10T03:04:26.092 INFO:teuthology.orchestra.run.smithi188.stdout: "recovery_completed": 0, 2024-06-10T03:04:26.092 INFO:teuthology.orchestra.run.smithi188.stdout: "ireq_quiesce_path": 6, 2024-06-10T03:04:26.092 INFO:teuthology.orchestra.run.smithi188.stdout: "ireq_quiesce_inode": 1552, 2024-06-10T03:04:26.092 INFO:teuthology.orchestra.run.smithi188.stdout: "ireq_enqueue_scrub": 0, 2024-06-10T03:04:26.092 INFO:teuthology.orchestra.run.smithi188.stdout: "ireq_exportdir": 24, 2024-06-10T03:04:26.092 INFO:teuthology.orchestra.run.smithi188.stdout: "ireq_flush": 0, 2024-06-10T03:04:26.092 INFO:teuthology.orchestra.run.smithi188.stdout: "ireq_fragmentdir": 3, 2024-06-10T03:04:26.092 INFO:teuthology.orchestra.run.smithi188.stdout: "ireq_fragstats": 0, 2024-06-10T03:04:26.092 INFO:teuthology.orchestra.run.smithi188.stdout: "ireq_inodestats": 0 2024-06-10T03:04:26.092 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:26.092 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_log": { 2024-06-10T03:04:26.093 INFO:teuthology.orchestra.run.smithi188.stdout: "evlrg": 0, 2024-06-10T03:04:26.093 INFO:teuthology.orchestra.run.smithi188.stdout: "evadd": 4935, 2024-06-10T03:04:26.093 INFO:teuthology.orchestra.run.smithi188.stdout: "evex": 0, 2024-06-10T03:04:26.093 INFO:teuthology.orchestra.run.smithi188.stdout: "evtrm": 0, 2024-06-10T03:04:26.093 INFO:teuthology.orchestra.run.smithi188.stdout: "ev": 4935, 2024-06-10T03:04:26.093 INFO:teuthology.orchestra.run.smithi188.stdout: "evexg": 0, 2024-06-10T03:04:26.093 INFO:teuthology.orchestra.run.smithi188.stdout: "evexd": 0, 2024-06-10T03:04:26.093 INFO:teuthology.orchestra.run.smithi188.stdout: "segadd": 6, 2024-06-10T03:04:26.093 INFO:teuthology.orchestra.run.smithi188.stdout: "segex": 0, 2024-06-10T03:04:26.093 INFO:teuthology.orchestra.run.smithi188.stdout: "segtrm": 0, 2024-06-10T03:04:26.093 INFO:teuthology.orchestra.run.smithi188.stdout: "seg": 6, 2024-06-10T03:04:26.093 INFO:teuthology.orchestra.run.smithi188.stdout: "segmjr": 2, 2024-06-10T03:04:26.093 INFO:teuthology.orchestra.run.smithi188.stdout: "segexg": 0, 2024-06-10T03:04:26.093 INFO:teuthology.orchestra.run.smithi188.stdout: "segexd": 0, 2024-06-10T03:04:26.093 INFO:teuthology.orchestra.run.smithi188.stdout: "expos": 4194304, 2024-06-10T03:04:26.094 INFO:teuthology.orchestra.run.smithi188.stdout: "wrpos": 11503137, 2024-06-10T03:04:26.094 INFO:teuthology.orchestra.run.smithi188.stdout: "rdpos": 0, 2024-06-10T03:04:26.094 INFO:teuthology.orchestra.run.smithi188.stdout: "jlat": { 2024-06-10T03:04:26.094 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 319, 2024-06-10T03:04:26.094 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 7.525760754, 2024-06-10T03:04:26.094 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.023591726 2024-06-10T03:04:26.094 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:26.094 INFO:teuthology.orchestra.run.smithi188.stdout: "replayed": 0 2024-06-10T03:04:26.094 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:26.094 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_mem": { 2024-06-10T03:04:26.094 INFO:teuthology.orchestra.run.smithi188.stdout: "ino": 31, 2024-06-10T03:04:26.094 INFO:teuthology.orchestra.run.smithi188.stdout: "ino+": 1077, 2024-06-10T03:04:26.094 INFO:teuthology.orchestra.run.smithi188.stdout: "ino-": 1046, 2024-06-10T03:04:26.095 INFO:teuthology.orchestra.run.smithi188.stdout: "dir": 30, 2024-06-10T03:04:26.095 INFO:teuthology.orchestra.run.smithi188.stdout: "dir+": 1174, 2024-06-10T03:04:26.095 INFO:teuthology.orchestra.run.smithi188.stdout: "dir-": 1144, 2024-06-10T03:04:26.095 INFO:teuthology.orchestra.run.smithi188.stdout: "dn": 26, 2024-06-10T03:04:26.095 INFO:teuthology.orchestra.run.smithi188.stdout: "dn+": 2516, 2024-06-10T03:04:26.095 INFO:teuthology.orchestra.run.smithi188.stdout: "dn-": 2490, 2024-06-10T03:04:26.095 INFO:teuthology.orchestra.run.smithi188.stdout: "cap": 0, 2024-06-10T03:04:26.095 INFO:teuthology.orchestra.run.smithi188.stdout: "cap+": 1339, 2024-06-10T03:04:26.095 INFO:teuthology.orchestra.run.smithi188.stdout: "cap-": 1339, 2024-06-10T03:04:26.095 INFO:teuthology.orchestra.run.smithi188.stdout: "rss": 74488, 2024-06-10T03:04:26.095 INFO:teuthology.orchestra.run.smithi188.stdout: "heap": 223612 2024-06-10T03:04:26.095 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:26.095 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_server": { 2024-06-10T03:04:26.095 INFO:teuthology.orchestra.run.smithi188.stdout: "dispatch_client_request": 28858, 2024-06-10T03:04:26.096 INFO:teuthology.orchestra.run.smithi188.stdout: "dispatch_server_request": 24, 2024-06-10T03:04:26.096 INFO:teuthology.orchestra.run.smithi188.stdout: "handle_client_request": 26467, 2024-06-10T03:04:26.096 INFO:teuthology.orchestra.run.smithi188.stdout: "handle_client_session": 13, 2024-06-10T03:04:26.096 INFO:teuthology.orchestra.run.smithi188.stdout: "handle_peer_request": 1874, 2024-06-10T03:04:26.096 INFO:teuthology.orchestra.run.smithi188.stdout: "req_create_latency": { 2024-06-10T03:04:26.096 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 583, 2024-06-10T03:04:26.096 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 7.734193074, 2024-06-10T03:04:26.096 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.013266197 2024-06-10T03:04:26.096 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:26.096 INFO:teuthology.orchestra.run.smithi188.stdout: "req_getattr_latency": { 2024-06-10T03:04:26.096 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 247, 2024-06-10T03:04:26.096 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.904590057, 2024-06-10T03:04:26.096 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.003662307 2024-06-10T03:04:26.097 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:26.097 INFO:teuthology.orchestra.run.smithi188.stdout: "req_getfilelock_latency": { 2024-06-10T03:04:26.097 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-06-10T03:04:26.097 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-06-10T03:04:26.097 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-06-10T03:04:26.097 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:26.097 INFO:teuthology.orchestra.run.smithi188.stdout: "req_link_latency": { 2024-06-10T03:04:26.097 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-06-10T03:04:26.097 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-06-10T03:04:26.097 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-06-10T03:04:26.097 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:26.097 INFO:teuthology.orchestra.run.smithi188.stdout: "req_lookup_latency": { 2024-06-10T03:04:26.097 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 14367, 2024-06-10T03:04:26.097 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 484.441704958, 2024-06-10T03:04:26.098 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.033719057 2024-06-10T03:04:26.098 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:26.098 INFO:teuthology.orchestra.run.smithi188.stdout: "req_lookuphash_latency": { 2024-06-10T03:04:26.098 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-06-10T03:04:26.098 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-06-10T03:04:26.098 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-06-10T03:04:26.098 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:26.098 INFO:teuthology.orchestra.run.smithi188.stdout: "req_lookupino_latency": { 2024-06-10T03:04:26.098 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-06-10T03:04:26.098 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-06-10T03:04:26.098 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-06-10T03:04:26.098 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:26.098 INFO:teuthology.orchestra.run.smithi188.stdout: "req_lookupname_latency": { 2024-06-10T03:04:26.098 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-06-10T03:04:26.099 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-06-10T03:04:26.099 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-06-10T03:04:26.099 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:26.099 INFO:teuthology.orchestra.run.smithi188.stdout: "req_lookupparent_latency": { 2024-06-10T03:04:26.099 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-06-10T03:04:26.099 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-06-10T03:04:26.099 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-06-10T03:04:26.099 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:26.099 INFO:teuthology.orchestra.run.smithi188.stdout: "req_lookupsnap_latency": { 2024-06-10T03:04:26.099 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-06-10T03:04:26.099 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-06-10T03:04:26.099 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-06-10T03:04:26.099 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:26.100 INFO:teuthology.orchestra.run.smithi188.stdout: "req_lssnap_latency": { 2024-06-10T03:04:26.100 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-06-10T03:04:26.100 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-06-10T03:04:26.101 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-06-10T03:04:26.101 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:26.101 INFO:teuthology.orchestra.run.smithi188.stdout: "req_mkdir_latency": { 2024-06-10T03:04:26.101 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-06-10T03:04:26.101 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-06-10T03:04:26.101 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-06-10T03:04:26.101 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:26.101 INFO:teuthology.orchestra.run.smithi188.stdout: "req_mknod_latency": { 2024-06-10T03:04:26.101 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-06-10T03:04:26.101 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-06-10T03:04:26.101 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-06-10T03:04:26.101 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:26.101 INFO:teuthology.orchestra.run.smithi188.stdout: "req_mksnap_latency": { 2024-06-10T03:04:26.101 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-06-10T03:04:26.102 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-06-10T03:04:26.102 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-06-10T03:04:26.102 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:26.102 INFO:teuthology.orchestra.run.smithi188.stdout: "req_open_latency": { 2024-06-10T03:04:26.102 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 482, 2024-06-10T03:04:26.102 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 1.618287587, 2024-06-10T03:04:26.102 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.003357443 2024-06-10T03:04:26.102 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:26.102 INFO:teuthology.orchestra.run.smithi188.stdout: "req_readdir_latency": { 2024-06-10T03:04:26.102 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 24, 2024-06-10T03:04:26.102 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.029984184, 2024-06-10T03:04:26.102 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.001249341 2024-06-10T03:04:26.102 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:26.103 INFO:teuthology.orchestra.run.smithi188.stdout: "req_rename_latency": { 2024-06-10T03:04:26.103 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 584, 2024-06-10T03:04:26.103 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 7.103706319, 2024-06-10T03:04:26.103 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.012163880 2024-06-10T03:04:26.103 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:26.103 INFO:teuthology.orchestra.run.smithi188.stdout: "req_renamesnap_latency": { 2024-06-10T03:04:26.103 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-06-10T03:04:26.103 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-06-10T03:04:26.103 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-06-10T03:04:26.103 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:26.103 INFO:teuthology.orchestra.run.smithi188.stdout: "req_snapdiff_latency": { 2024-06-10T03:04:26.103 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-06-10T03:04:26.103 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-06-10T03:04:26.103 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-06-10T03:04:26.104 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:26.104 INFO:teuthology.orchestra.run.smithi188.stdout: "req_rmdir_latency": { 2024-06-10T03:04:26.104 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-06-10T03:04:26.104 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-06-10T03:04:26.104 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-06-10T03:04:26.104 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:26.104 INFO:teuthology.orchestra.run.smithi188.stdout: "req_rmsnap_latency": { 2024-06-10T03:04:26.104 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-06-10T03:04:26.104 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-06-10T03:04:26.104 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-06-10T03:04:26.104 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:26.104 INFO:teuthology.orchestra.run.smithi188.stdout: "req_rmxattr_latency": { 2024-06-10T03:04:26.104 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-06-10T03:04:26.104 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-06-10T03:04:26.105 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-06-10T03:04:26.105 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:26.105 INFO:teuthology.orchestra.run.smithi188.stdout: "req_setattr_latency": { 2024-06-10T03:04:26.105 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-06-10T03:04:26.105 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-06-10T03:04:26.105 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-06-10T03:04:26.105 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:26.105 INFO:teuthology.orchestra.run.smithi188.stdout: "req_setdirlayout_latency": { 2024-06-10T03:04:26.105 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-06-10T03:04:26.105 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-06-10T03:04:26.105 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-06-10T03:04:26.105 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:26.105 INFO:teuthology.orchestra.run.smithi188.stdout: "req_setfilelock_latency": { 2024-06-10T03:04:26.105 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-06-10T03:04:26.105 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-06-10T03:04:26.106 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-06-10T03:04:26.106 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:26.106 INFO:teuthology.orchestra.run.smithi188.stdout: "req_setlayout_latency": { 2024-06-10T03:04:26.106 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-06-10T03:04:26.106 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-06-10T03:04:26.106 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-06-10T03:04:26.106 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:26.106 INFO:teuthology.orchestra.run.smithi188.stdout: "req_setxattr_latency": { 2024-06-10T03:04:26.106 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-06-10T03:04:26.106 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-06-10T03:04:26.106 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-06-10T03:04:26.106 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:26.106 INFO:teuthology.orchestra.run.smithi188.stdout: "req_symlink_latency": { 2024-06-10T03:04:26.107 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-06-10T03:04:26.107 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-06-10T03:04:26.107 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-06-10T03:04:26.107 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:26.107 INFO:teuthology.orchestra.run.smithi188.stdout: "req_unlink_latency": { 2024-06-10T03:04:26.107 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 616, 2024-06-10T03:04:26.107 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.982992180, 2024-06-10T03:04:26.107 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.001595766 2024-06-10T03:04:26.107 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:26.107 INFO:teuthology.orchestra.run.smithi188.stdout: "cap_revoke_eviction": 0, 2024-06-10T03:04:26.107 INFO:teuthology.orchestra.run.smithi188.stdout: "cap_acquisition_throttle": 0, 2024-06-10T03:04:26.107 INFO:teuthology.orchestra.run.smithi188.stdout: "req_getvxattr_latency": { 2024-06-10T03:04:26.107 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-06-10T03:04:26.107 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-06-10T03:04:26.108 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-06-10T03:04:26.108 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-06-10T03:04:26.109 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:26.109 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_sessions": { 2024-06-10T03:04:26.109 INFO:teuthology.orchestra.run.smithi188.stdout: "session_count": 1, 2024-06-10T03:04:26.109 INFO:teuthology.orchestra.run.smithi188.stdout: "session_add": 2, 2024-06-10T03:04:26.109 INFO:teuthology.orchestra.run.smithi188.stdout: "session_remove": 1, 2024-06-10T03:04:26.109 INFO:teuthology.orchestra.run.smithi188.stdout: "sessions_open": 1, 2024-06-10T03:04:26.109 INFO:teuthology.orchestra.run.smithi188.stdout: "sessions_stale": 0, 2024-06-10T03:04:26.109 INFO:teuthology.orchestra.run.smithi188.stdout: "total_load": 2292, 2024-06-10T03:04:26.110 INFO:teuthology.orchestra.run.smithi188.stdout: "average_load": 2292, 2024-06-10T03:04:26.110 INFO:teuthology.orchestra.run.smithi188.stdout: "avg_session_uptime": 619, 2024-06-10T03:04:26.110 INFO:teuthology.orchestra.run.smithi188.stdout: "mdthresh_evicted": 0 2024-06-10T03:04:26.110 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:26.110 INFO:teuthology.orchestra.run.smithi188.stdout: "mempool": { 2024-06-10T03:04:26.110 INFO:teuthology.orchestra.run.smithi188.stdout: "bloom_filter_bytes": 0, 2024-06-10T03:04:26.110 INFO:teuthology.orchestra.run.smithi188.stdout: "bloom_filter_items": 0, 2024-06-10T03:04:26.110 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_alloc_bytes": 0, 2024-06-10T03:04:26.110 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_alloc_items": 0, 2024-06-10T03:04:26.110 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_data_bytes": 0, 2024-06-10T03:04:26.110 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_data_items": 0, 2024-06-10T03:04:26.110 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-10T03:04:26.110 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_onode_items": 0, 2024-06-10T03:04:26.111 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-10T03:04:26.111 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_meta_items": 0, 2024-06-10T03:04:26.111 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_other_bytes": 0, 2024-06-10T03:04:26.111 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_other_items": 0, 2024-06-10T03:04:26.111 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-10T03:04:26.111 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_buffer_items": 0, 2024-06-10T03:04:26.111 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_extent_bytes": 0, 2024-06-10T03:04:26.111 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_extent_items": 0, 2024-06-10T03:04:26.111 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_blob_bytes": 0, 2024-06-10T03:04:26.111 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_blob_items": 0, 2024-06-10T03:04:26.111 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-10T03:04:26.111 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_shared_blob_items": 0, 2024-06-10T03:04:26.111 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-10T03:04:26.111 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_inline_bl_items": 0, 2024-06-10T03:04:26.111 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_fsck_bytes": 0, 2024-06-10T03:04:26.112 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_fsck_items": 0, 2024-06-10T03:04:26.112 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_txc_bytes": 0, 2024-06-10T03:04:26.112 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_txc_items": 0, 2024-06-10T03:04:26.112 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-10T03:04:26.112 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_writing_deferred_items": 0, 2024-06-10T03:04:26.112 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_writing_bytes": 0, 2024-06-10T03:04:26.112 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_writing_items": 0, 2024-06-10T03:04:26.112 INFO:teuthology.orchestra.run.smithi188.stdout: "bluefs_bytes": 0, 2024-06-10T03:04:26.112 INFO:teuthology.orchestra.run.smithi188.stdout: "bluefs_items": 0, 2024-06-10T03:04:26.112 INFO:teuthology.orchestra.run.smithi188.stdout: "bluefs_file_reader_bytes": 0, 2024-06-10T03:04:26.112 INFO:teuthology.orchestra.run.smithi188.stdout: "bluefs_file_reader_items": 0, 2024-06-10T03:04:26.112 INFO:teuthology.orchestra.run.smithi188.stdout: "bluefs_file_writer_bytes": 0, 2024-06-10T03:04:26.112 INFO:teuthology.orchestra.run.smithi188.stdout: "bluefs_file_writer_items": 0, 2024-06-10T03:04:26.112 INFO:teuthology.orchestra.run.smithi188.stdout: "buffer_anon_bytes": 279607, 2024-06-10T03:04:26.113 INFO:teuthology.orchestra.run.smithi188.stdout: "buffer_anon_items": 101, 2024-06-10T03:04:26.113 INFO:teuthology.orchestra.run.smithi188.stdout: "buffer_meta_bytes": 0, 2024-06-10T03:04:26.113 INFO:teuthology.orchestra.run.smithi188.stdout: "buffer_meta_items": 0, 2024-06-10T03:04:26.113 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_bytes": 0, 2024-06-10T03:04:26.113 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_items": 0, 2024-06-10T03:04:26.113 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_mapbl_bytes": 0, 2024-06-10T03:04:26.113 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_mapbl_items": 0, 2024-06-10T03:04:26.113 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_pglog_bytes": 0, 2024-06-10T03:04:26.113 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_pglog_items": 0, 2024-06-10T03:04:26.113 INFO:teuthology.orchestra.run.smithi188.stdout: "osdmap_bytes": 8984, 2024-06-10T03:04:26.113 INFO:teuthology.orchestra.run.smithi188.stdout: "osdmap_items": 172, 2024-06-10T03:04:26.113 INFO:teuthology.orchestra.run.smithi188.stdout: "osdmap_mapping_bytes": 0, 2024-06-10T03:04:26.113 INFO:teuthology.orchestra.run.smithi188.stdout: "osdmap_mapping_items": 0, 2024-06-10T03:04:26.113 INFO:teuthology.orchestra.run.smithi188.stdout: "pgmap_bytes": 0, 2024-06-10T03:04:26.114 INFO:teuthology.orchestra.run.smithi188.stdout: "pgmap_items": 0, 2024-06-10T03:04:26.114 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_co_bytes": 126797, 2024-06-10T03:04:26.114 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_co_items": 513, 2024-06-10T03:04:26.114 INFO:teuthology.orchestra.run.smithi188.stdout: "unittest_1_bytes": 0, 2024-06-10T03:04:26.114 INFO:teuthology.orchestra.run.smithi188.stdout: "unittest_1_items": 0, 2024-06-10T03:04:26.114 INFO:teuthology.orchestra.run.smithi188.stdout: "unittest_2_bytes": 0, 2024-06-10T03:04:26.114 INFO:teuthology.orchestra.run.smithi188.stdout: "unittest_2_items": 0 2024-06-10T03:04:26.114 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:26.114 INFO:teuthology.orchestra.run.smithi188.stdout: "objecter": { 2024-06-10T03:04:26.114 INFO:teuthology.orchestra.run.smithi188.stdout: "op_active": 0, 2024-06-10T03:04:26.114 INFO:teuthology.orchestra.run.smithi188.stdout: "op_laggy": 0, 2024-06-10T03:04:26.114 INFO:teuthology.orchestra.run.smithi188.stdout: "op_send": 3880, 2024-06-10T03:04:26.115 INFO:teuthology.orchestra.run.smithi188.stdout: "op_send_bytes": 8123059, 2024-06-10T03:04:26.115 INFO:teuthology.orchestra.run.smithi188.stdout: "op_resend": 29, 2024-06-10T03:04:26.115 INFO:teuthology.orchestra.run.smithi188.stdout: "op_reply": 3880, 2024-06-10T03:04:26.115 INFO:teuthology.orchestra.run.smithi188.stdout: "op_latency": { 2024-06-10T03:04:26.115 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 3880, 2024-06-10T03:04:26.115 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 44.691206123, 2024-06-10T03:04:26.115 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.011518352 2024-06-10T03:04:26.115 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:26.115 INFO:teuthology.orchestra.run.smithi188.stdout: "op_inflight": 0, 2024-06-10T03:04:26.115 INFO:teuthology.orchestra.run.smithi188.stdout: "oplen_avg": { 2024-06-10T03:04:26.115 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 3880, 2024-06-10T03:04:26.115 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 4070 2024-06-10T03:04:26.115 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:26.115 INFO:teuthology.orchestra.run.smithi188.stdout: "op": 3880, 2024-06-10T03:04:26.116 INFO:teuthology.orchestra.run.smithi188.stdout: "op_r": 1455, 2024-06-10T03:04:26.116 INFO:teuthology.orchestra.run.smithi188.stdout: "op_w": 2425, 2024-06-10T03:04:26.116 INFO:teuthology.orchestra.run.smithi188.stdout: "op_rmw": 0, 2024-06-10T03:04:26.116 INFO:teuthology.orchestra.run.smithi188.stdout: "op_pg": 0, 2024-06-10T03:04:26.116 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_stat": 0, 2024-06-10T03:04:26.116 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_create": 10, 2024-06-10T03:04:26.116 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_read": 60, 2024-06-10T03:04:26.116 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_write": 379, 2024-06-10T03:04:26.116 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_writefull": 61, 2024-06-10T03:04:26.116 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_writesame": 0, 2024-06-10T03:04:26.116 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_append": 0, 2024-06-10T03:04:26.116 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_zero": 0, 2024-06-10T03:04:26.116 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_truncate": 3, 2024-06-10T03:04:26.116 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_delete": 1817, 2024-06-10T03:04:26.117 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_mapext": 0, 2024-06-10T03:04:26.117 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_sparse_read": 0, 2024-06-10T03:04:26.117 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_clonerange": 0, 2024-06-10T03:04:26.117 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_getxattr": 1395, 2024-06-10T03:04:26.117 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_setxattr": 20, 2024-06-10T03:04:26.117 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_cmpxattr": 0, 2024-06-10T03:04:26.117 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_rmxattr": 0, 2024-06-10T03:04:26.117 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_resetxattrs": 0, 2024-06-10T03:04:26.117 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_call": 0, 2024-06-10T03:04:26.117 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_watch": 0, 2024-06-10T03:04:26.117 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_notify": 0, 2024-06-10T03:04:26.117 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_src_cmpxattr": 0, 2024-06-10T03:04:26.117 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_pgls": 0, 2024-06-10T03:04:26.117 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_pgls_filter": 0, 2024-06-10T03:04:26.117 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_other": 0, 2024-06-10T03:04:26.118 INFO:teuthology.orchestra.run.smithi188.stdout: "linger_active": 0, 2024-06-10T03:04:26.118 INFO:teuthology.orchestra.run.smithi188.stdout: "linger_send": 0, 2024-06-10T03:04:26.118 INFO:teuthology.orchestra.run.smithi188.stdout: "linger_resend": 0, 2024-06-10T03:04:26.118 INFO:teuthology.orchestra.run.smithi188.stdout: "linger_ping": 0, 2024-06-10T03:04:26.118 INFO:teuthology.orchestra.run.smithi188.stdout: "poolop_active": 0, 2024-06-10T03:04:26.118 INFO:teuthology.orchestra.run.smithi188.stdout: "poolop_send": 0, 2024-06-10T03:04:26.118 INFO:teuthology.orchestra.run.smithi188.stdout: "poolop_resend": 0, 2024-06-10T03:04:26.118 INFO:teuthology.orchestra.run.smithi188.stdout: "poolstat_active": 0, 2024-06-10T03:04:26.118 INFO:teuthology.orchestra.run.smithi188.stdout: "poolstat_send": 0, 2024-06-10T03:04:26.118 INFO:teuthology.orchestra.run.smithi188.stdout: "poolstat_resend": 0, 2024-06-10T03:04:26.118 INFO:teuthology.orchestra.run.smithi188.stdout: "statfs_active": 0, 2024-06-10T03:04:26.118 INFO:teuthology.orchestra.run.smithi188.stdout: "statfs_send": 0, 2024-06-10T03:04:26.118 INFO:teuthology.orchestra.run.smithi188.stdout: "statfs_resend": 0, 2024-06-10T03:04:26.119 INFO:teuthology.orchestra.run.smithi188.stdout: "command_active": 0, 2024-06-10T03:04:26.119 INFO:teuthology.orchestra.run.smithi188.stdout: "command_send": 0, 2024-06-10T03:04:26.119 INFO:teuthology.orchestra.run.smithi188.stdout: "command_resend": 0, 2024-06-10T03:04:26.119 INFO:teuthology.orchestra.run.smithi188.stdout: "map_epoch": 100, 2024-06-10T03:04:26.119 INFO:teuthology.orchestra.run.smithi188.stdout: "map_full": 0, 2024-06-10T03:04:26.119 INFO:teuthology.orchestra.run.smithi188.stdout: "map_inc": 2, 2024-06-10T03:04:26.119 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_sessions": 12, 2024-06-10T03:04:26.119 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_session_open": 12, 2024-06-10T03:04:26.119 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_session_close": 0, 2024-06-10T03:04:26.119 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_laggy": 0, 2024-06-10T03:04:26.119 INFO:teuthology.orchestra.run.smithi188.stdout: "omap_wr": 268, 2024-06-10T03:04:26.119 INFO:teuthology.orchestra.run.smithi188.stdout: "omap_rd": 0, 2024-06-10T03:04:26.119 INFO:teuthology.orchestra.run.smithi188.stdout: "omap_del": 57 2024-06-10T03:04:26.119 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:26.119 INFO:teuthology.orchestra.run.smithi188.stdout: "oft": { 2024-06-10T03:04:26.120 INFO:teuthology.orchestra.run.smithi188.stdout: "omap_total_objs": 1, 2024-06-10T03:04:26.120 INFO:teuthology.orchestra.run.smithi188.stdout: "omap_total_kv_pairs": 0, 2024-06-10T03:04:26.120 INFO:teuthology.orchestra.run.smithi188.stdout: "omap_total_updates": 0, 2024-06-10T03:04:26.120 INFO:teuthology.orchestra.run.smithi188.stdout: "omap_total_removes": 0 2024-06-10T03:04:26.120 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:26.120 INFO:teuthology.orchestra.run.smithi188.stdout: "osd-slow-ops": { 2024-06-10T03:04:26.120 INFO:teuthology.orchestra.run.smithi188.stdout: "slow_ops_count": 1542 2024-06-10T03:04:26.120 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:26.120 INFO:teuthology.orchestra.run.smithi188.stdout: "purge_queue": { 2024-06-10T03:04:26.120 INFO:teuthology.orchestra.run.smithi188.stdout: "pq_executing_ops": 0, 2024-06-10T03:04:26.120 INFO:teuthology.orchestra.run.smithi188.stdout: "pq_executing_ops_high_water": 10, 2024-06-10T03:04:26.120 INFO:teuthology.orchestra.run.smithi188.stdout: "pq_executing": 0, 2024-06-10T03:04:26.120 INFO:teuthology.orchestra.run.smithi188.stdout: "pq_executing_high_water": 5, 2024-06-10T03:04:26.120 INFO:teuthology.orchestra.run.smithi188.stdout: "pq_executed_ops": 1804, 2024-06-10T03:04:26.120 INFO:teuthology.orchestra.run.smithi188.stdout: "pq_executed": 902, 2024-06-10T03:04:26.121 INFO:teuthology.orchestra.run.smithi188.stdout: "pq_item_in_journal": 0 2024-06-10T03:04:26.121 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:26.121 INFO:teuthology.orchestra.run.smithi188.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-10T03:04:26.121 INFO:teuthology.orchestra.run.smithi188.stdout: "val": 0, 2024-06-10T03:04:26.121 INFO:teuthology.orchestra.run.smithi188.stdout: "max": 104857600, 2024-06-10T03:04:26.121 INFO:teuthology.orchestra.run.smithi188.stdout: "get_started": 0, 2024-06-10T03:04:26.121 INFO:teuthology.orchestra.run.smithi188.stdout: "get": 51521, 2024-06-10T03:04:26.121 INFO:teuthology.orchestra.run.smithi188.stdout: "get_sum": 13950516, 2024-06-10T03:04:26.121 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_fail": 0, 2024-06-10T03:04:26.121 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_success": 51521, 2024-06-10T03:04:26.121 INFO:teuthology.orchestra.run.smithi188.stdout: "take": 0, 2024-06-10T03:04:26.121 INFO:teuthology.orchestra.run.smithi188.stdout: "take_sum": 0, 2024-06-10T03:04:26.121 INFO:teuthology.orchestra.run.smithi188.stdout: "put": 51521, 2024-06-10T03:04:26.121 INFO:teuthology.orchestra.run.smithi188.stdout: "put_sum": 13950516, 2024-06-10T03:04:26.121 INFO:teuthology.orchestra.run.smithi188.stdout: "wait": { 2024-06-10T03:04:26.122 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-06-10T03:04:26.122 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-06-10T03:04:26.122 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-06-10T03:04:26.122 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-06-10T03:04:26.122 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:26.122 INFO:teuthology.orchestra.run.smithi188.stdout: "throttle-objecter_bytes": { 2024-06-10T03:04:26.122 INFO:teuthology.orchestra.run.smithi188.stdout: "val": 0, 2024-06-10T03:04:26.122 INFO:teuthology.orchestra.run.smithi188.stdout: "max": 104857600, 2024-06-10T03:04:26.122 INFO:teuthology.orchestra.run.smithi188.stdout: "get_started": 0, 2024-06-10T03:04:26.122 INFO:teuthology.orchestra.run.smithi188.stdout: "get": 0, 2024-06-10T03:04:26.122 INFO:teuthology.orchestra.run.smithi188.stdout: "get_sum": 0, 2024-06-10T03:04:26.122 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_fail": 0, 2024-06-10T03:04:26.122 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_success": 0, 2024-06-10T03:04:26.123 INFO:teuthology.orchestra.run.smithi188.stdout: "take": 3880, 2024-06-10T03:04:26.123 INFO:teuthology.orchestra.run.smithi188.stdout: "take_sum": 8221377, 2024-06-10T03:04:26.123 INFO:teuthology.orchestra.run.smithi188.stdout: "put": 3880, 2024-06-10T03:04:26.123 INFO:teuthology.orchestra.run.smithi188.stdout: "put_sum": 8221377, 2024-06-10T03:04:26.123 INFO:teuthology.orchestra.run.smithi188.stdout: "wait": { 2024-06-10T03:04:26.123 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-06-10T03:04:26.123 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-06-10T03:04:26.123 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-06-10T03:04:26.123 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-06-10T03:04:26.123 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:26.123 INFO:teuthology.orchestra.run.smithi188.stdout: "throttle-objecter_ops": { 2024-06-10T03:04:26.123 INFO:teuthology.orchestra.run.smithi188.stdout: "val": 0, 2024-06-10T03:04:26.123 INFO:teuthology.orchestra.run.smithi188.stdout: "max": 1024, 2024-06-10T03:04:26.123 INFO:teuthology.orchestra.run.smithi188.stdout: "get_started": 0, 2024-06-10T03:04:26.124 INFO:teuthology.orchestra.run.smithi188.stdout: "get": 0, 2024-06-10T03:04:26.124 INFO:teuthology.orchestra.run.smithi188.stdout: "get_sum": 0, 2024-06-10T03:04:26.124 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_fail": 0, 2024-06-10T03:04:26.124 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_success": 0, 2024-06-10T03:04:26.124 INFO:teuthology.orchestra.run.smithi188.stdout: "take": 3880, 2024-06-10T03:04:26.124 INFO:teuthology.orchestra.run.smithi188.stdout: "take_sum": 3880, 2024-06-10T03:04:26.124 INFO:teuthology.orchestra.run.smithi188.stdout: "put": 3880, 2024-06-10T03:04:26.124 INFO:teuthology.orchestra.run.smithi188.stdout: "put_sum": 3880, 2024-06-10T03:04:26.124 INFO:teuthology.orchestra.run.smithi188.stdout: "wait": { 2024-06-10T03:04:26.124 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-06-10T03:04:26.124 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-06-10T03:04:26.124 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-06-10T03:04:26.124 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-06-10T03:04:26.124 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:26.124 INFO:teuthology.orchestra.run.smithi188.stdout: "throttle-write_buf_throttle": { 2024-06-10T03:04:26.125 INFO:teuthology.orchestra.run.smithi188.stdout: "val": 0, 2024-06-10T03:04:26.125 INFO:teuthology.orchestra.run.smithi188.stdout: "max": 3758096384, 2024-06-10T03:04:26.125 INFO:teuthology.orchestra.run.smithi188.stdout: "get_started": 0, 2024-06-10T03:04:26.125 INFO:teuthology.orchestra.run.smithi188.stdout: "get": 902, 2024-06-10T03:04:26.125 INFO:teuthology.orchestra.run.smithi188.stdout: "get_sum": 98318, 2024-06-10T03:04:26.125 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_fail": 0, 2024-06-10T03:04:26.125 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_success": 902, 2024-06-10T03:04:26.125 INFO:teuthology.orchestra.run.smithi188.stdout: "take": 0, 2024-06-10T03:04:26.125 INFO:teuthology.orchestra.run.smithi188.stdout: "take_sum": 0, 2024-06-10T03:04:26.125 INFO:teuthology.orchestra.run.smithi188.stdout: "put": 60, 2024-06-10T03:04:26.125 INFO:teuthology.orchestra.run.smithi188.stdout: "put_sum": 98318, 2024-06-10T03:04:26.125 INFO:teuthology.orchestra.run.smithi188.stdout: "wait": { 2024-06-10T03:04:26.125 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-06-10T03:04:26.125 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-06-10T03:04:26.126 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-06-10T03:04:26.126 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-06-10T03:04:26.126 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:26.126 INFO:teuthology.orchestra.run.smithi188.stdout: "throttle-write_buf_throttle-0x557c76aee500": { 2024-06-10T03:04:26.126 INFO:teuthology.orchestra.run.smithi188.stdout: "val": 0, 2024-06-10T03:04:26.126 INFO:teuthology.orchestra.run.smithi188.stdout: "max": 3758096384, 2024-06-10T03:04:26.126 INFO:teuthology.orchestra.run.smithi188.stdout: "get_started": 0, 2024-06-10T03:04:26.126 INFO:teuthology.orchestra.run.smithi188.stdout: "get": 4935, 2024-06-10T03:04:26.126 INFO:teuthology.orchestra.run.smithi188.stdout: "get_sum": 7308833, 2024-06-10T03:04:26.126 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_fail": 0, 2024-06-10T03:04:26.126 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_success": 4935, 2024-06-10T03:04:26.126 INFO:teuthology.orchestra.run.smithi188.stdout: "take": 0, 2024-06-10T03:04:26.126 INFO:teuthology.orchestra.run.smithi188.stdout: "take_sum": 0, 2024-06-10T03:04:26.126 INFO:teuthology.orchestra.run.smithi188.stdout: "put": 319, 2024-06-10T03:04:26.127 INFO:teuthology.orchestra.run.smithi188.stdout: "put_sum": 7308833, 2024-06-10T03:04:26.127 INFO:teuthology.orchestra.run.smithi188.stdout: "wait": { 2024-06-10T03:04:26.127 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-06-10T03:04:26.127 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-06-10T03:04:26.127 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-06-10T03:04:26.127 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-06-10T03:04:26.127 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-06-10T03:04:26.127 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-06-10T03:04:26.725 INFO:tasks.check_counter:Daemon mds.c mds.exported=24 2024-06-10T03:04:26.725 INFO:tasks.check_counter:Daemon mds.c mds.imported=3 2024-06-10T03:04:26.725 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update=691 2024-06-10T03:04:26.725 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update_receipt=2070 2024-06-10T03:04:26.725 INFO:tasks.check_counter:Daemon mds.c mds.dir_split=3 2024-06-10T03:04:26.725 DEBUG:tasks.check_counter:Getting stats from f 2024-06-10T03:04:26.725 DEBUG:teuthology.orchestra.run.smithi188:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph daemon mds.f perf dump 2024-06-10T03:04:26.974 INFO:teuthology.orchestra.run.smithi188.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.c/config 2024-06-10T03:04:27.204 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:04:26 smithi188 ceph-mon[36599]: pgmap v741: 129 pgs: 129 active+clean; 228 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-10T03:04:27.227 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:04:26 smithi052 ceph-mon[32362]: pgmap v741: 129 pgs: 129 active+clean; 228 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-10T03:04:27.388 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:04:26 smithi120 ceph-mon[36634]: pgmap v741: 129 pgs: 129 active+clean; 228 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-10T03:04:28.227 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:04:27 smithi052 ceph-mon[32362]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-10T03:04:28.237 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:04:27 smithi188 ceph-mon[36599]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-10T03:04:28.388 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:04:27 smithi120 ceph-mon[36634]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-10T03:04:28.741 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-06-10T03:04:28.741 INFO:teuthology.orchestra.run.smithi188.stdout: "AsyncMessenger::Worker-0": { 2024-06-10T03:04:28.742 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_messages": 437631, 2024-06-10T03:04:28.742 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_messages": 625570, 2024-06-10T03:04:28.742 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_bytes": 159787922, 2024-06-10T03:04:28.742 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_bytes": 473936702, 2024-06-10T03:04:28.742 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_created_connections": 19, 2024-06-10T03:04:28.742 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_active_connections": 4, 2024-06-10T03:04:28.742 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_total_time": 23.311583028, 2024-06-10T03:04:28.742 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_send_time": 10.771616386, 2024-06-10T03:04:28.742 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_recv_time": 36.939812547, 2024-06-10T03:04:28.742 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_fast_dispatch_time": 1.477000082, 2024-06-10T03:04:28.742 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_messages_queue_lat": { 2024-06-10T03:04:28.743 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 625569, 2024-06-10T03:04:28.743 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 47.115852806, 2024-06-10T03:04:28.743 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000075316 2024-06-10T03:04:28.743 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:28.743 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_handle_ack_lat": { 2024-06-10T03:04:28.743 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 836107, 2024-06-10T03:04:28.743 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.128415705, 2024-06-10T03:04:28.743 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000153 2024-06-10T03:04:28.743 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:28.743 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_encrypted_bytes": 141446480, 2024-06-10T03:04:28.743 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_encrypted_bytes": 341586848 2024-06-10T03:04:28.743 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:28.743 INFO:teuthology.orchestra.run.smithi188.stdout: "AsyncMessenger::Worker-1": { 2024-06-10T03:04:28.744 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_messages": 84686, 2024-06-10T03:04:28.744 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_messages": 86371, 2024-06-10T03:04:28.744 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_bytes": 25700343, 2024-06-10T03:04:28.744 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_bytes": 316975652, 2024-06-10T03:04:28.744 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_created_connections": 12, 2024-06-10T03:04:28.744 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_active_connections": 7, 2024-06-10T03:04:28.744 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_total_time": 8.554289630, 2024-06-10T03:04:28.744 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_send_time": 4.318914089, 2024-06-10T03:04:28.744 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_recv_time": 1.615415301, 2024-06-10T03:04:28.744 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_fast_dispatch_time": 2.319183148, 2024-06-10T03:04:28.744 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_messages_queue_lat": { 2024-06-10T03:04:28.744 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 86369, 2024-06-10T03:04:28.744 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 31.999116287, 2024-06-10T03:04:28.745 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000370493 2024-06-10T03:04:28.745 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:28.745 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_handle_ack_lat": { 2024-06-10T03:04:28.745 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 18534, 2024-06-10T03:04:28.745 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.003657518, 2024-06-10T03:04:28.745 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000197 2024-06-10T03:04:28.745 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:28.745 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_encrypted_bytes": 320, 2024-06-10T03:04:28.745 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_encrypted_bytes": 468288 2024-06-10T03:04:28.745 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:28.745 INFO:teuthology.orchestra.run.smithi188.stdout: "AsyncMessenger::Worker-2": { 2024-06-10T03:04:28.745 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_messages": 55147, 2024-06-10T03:04:28.745 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_messages": 61830, 2024-06-10T03:04:28.746 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_bytes": 19282370, 2024-06-10T03:04:28.746 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_bytes": 160976638, 2024-06-10T03:04:28.746 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_created_connections": 20, 2024-06-10T03:04:28.746 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_active_connections": 7, 2024-06-10T03:04:28.746 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_total_time": 4.404901129, 2024-06-10T03:04:28.746 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_send_time": 1.625504194, 2024-06-10T03:04:28.746 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_recv_time": 1.223076143, 2024-06-10T03:04:28.746 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_fast_dispatch_time": 1.354547695, 2024-06-10T03:04:28.746 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_messages_queue_lat": { 2024-06-10T03:04:28.746 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 61829, 2024-06-10T03:04:28.746 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 21.624909426, 2024-06-10T03:04:28.746 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000349753 2024-06-10T03:04:28.746 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:28.747 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_handle_ack_lat": { 2024-06-10T03:04:28.747 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 30034, 2024-06-10T03:04:28.747 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.005694847, 2024-06-10T03:04:28.747 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000189 2024-06-10T03:04:28.747 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:28.747 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_encrypted_bytes": 3072, 2024-06-10T03:04:28.747 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_encrypted_bytes": 448 2024-06-10T03:04:28.747 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:28.747 INFO:teuthology.orchestra.run.smithi188.stdout: "cct": { 2024-06-10T03:04:28.747 INFO:teuthology.orchestra.run.smithi188.stdout: "total_workers": 1, 2024-06-10T03:04:28.747 INFO:teuthology.orchestra.run.smithi188.stdout: "unhealthy_workers": 0 2024-06-10T03:04:28.747 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:28.747 INFO:teuthology.orchestra.run.smithi188.stdout: "finisher-MDSRank": { 2024-06-10T03:04:28.748 INFO:teuthology.orchestra.run.smithi188.stdout: "queue_len": 0, 2024-06-10T03:04:28.748 INFO:teuthology.orchestra.run.smithi188.stdout: "complete_latency": { 2024-06-10T03:04:28.748 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 29910, 2024-06-10T03:04:28.748 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 242.578128481, 2024-06-10T03:04:28.748 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.008110268 2024-06-10T03:04:28.748 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-06-10T03:04:28.748 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:28.748 INFO:teuthology.orchestra.run.smithi188.stdout: "finisher-PurgeQueue": { 2024-06-10T03:04:28.748 INFO:teuthology.orchestra.run.smithi188.stdout: "queue_len": 0, 2024-06-10T03:04:28.748 INFO:teuthology.orchestra.run.smithi188.stdout: "complete_latency": { 2024-06-10T03:04:28.748 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 66517, 2024-06-10T03:04:28.748 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 31.915258098, 2024-06-10T03:04:28.748 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000479806 2024-06-10T03:04:28.748 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-06-10T03:04:28.749 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:28.749 INFO:teuthology.orchestra.run.smithi188.stdout: "mds": { 2024-06-10T03:04:28.749 INFO:teuthology.orchestra.run.smithi188.stdout: "request": 237560, 2024-06-10T03:04:28.749 INFO:teuthology.orchestra.run.smithi188.stdout: "reply": 233538, 2024-06-10T03:04:28.749 INFO:teuthology.orchestra.run.smithi188.stdout: "reply_latency": { 2024-06-10T03:04:28.749 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 233538, 2024-06-10T03:04:28.749 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 8501.446265014, 2024-06-10T03:04:28.749 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.036402839 2024-06-10T03:04:28.749 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:28.749 INFO:teuthology.orchestra.run.smithi188.stdout: "slow_reply": 0, 2024-06-10T03:04:28.749 INFO:teuthology.orchestra.run.smithi188.stdout: "forward": 4022, 2024-06-10T03:04:28.749 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_fetch_complete": 16, 2024-06-10T03:04:28.749 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_fetch_keys": 0, 2024-06-10T03:04:28.749 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_commit": 1618, 2024-06-10T03:04:28.750 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_split": 117, 2024-06-10T03:04:28.750 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_merge": 14, 2024-06-10T03:04:28.750 INFO:teuthology.orchestra.run.smithi188.stdout: "inodes": 714, 2024-06-10T03:04:28.750 INFO:teuthology.orchestra.run.smithi188.stdout: "inodes_top": 489, 2024-06-10T03:04:28.750 INFO:teuthology.orchestra.run.smithi188.stdout: "inodes_bottom": 214, 2024-06-10T03:04:28.750 INFO:teuthology.orchestra.run.smithi188.stdout: "inodes_pin_tail": 11, 2024-06-10T03:04:28.750 INFO:teuthology.orchestra.run.smithi188.stdout: "inodes_pinned": 14, 2024-06-10T03:04:28.750 INFO:teuthology.orchestra.run.smithi188.stdout: "inodes_expired": 3822, 2024-06-10T03:04:28.750 INFO:teuthology.orchestra.run.smithi188.stdout: "inodes_with_caps": 0, 2024-06-10T03:04:28.750 INFO:teuthology.orchestra.run.smithi188.stdout: "caps": 0, 2024-06-10T03:04:28.750 INFO:teuthology.orchestra.run.smithi188.stdout: "subtrees": 5, 2024-06-10T03:04:28.750 INFO:teuthology.orchestra.run.smithi188.stdout: "traverse": 395718, 2024-06-10T03:04:28.750 INFO:teuthology.orchestra.run.smithi188.stdout: "traverse_hit": 338828, 2024-06-10T03:04:28.751 INFO:teuthology.orchestra.run.smithi188.stdout: "traverse_forward": 3995, 2024-06-10T03:04:28.751 INFO:teuthology.orchestra.run.smithi188.stdout: "traverse_discover": 37, 2024-06-10T03:04:28.751 INFO:teuthology.orchestra.run.smithi188.stdout: "traverse_dir_fetch": 1, 2024-06-10T03:04:28.751 INFO:teuthology.orchestra.run.smithi188.stdout: "traverse_remote_ino": 0, 2024-06-10T03:04:28.751 INFO:teuthology.orchestra.run.smithi188.stdout: "traverse_lock": 427, 2024-06-10T03:04:28.751 INFO:teuthology.orchestra.run.smithi188.stdout: "load_cent": 0, 2024-06-10T03:04:28.751 INFO:teuthology.orchestra.run.smithi188.stdout: "q": 0, 2024-06-10T03:04:28.751 INFO:teuthology.orchestra.run.smithi188.stdout: "exported": 20, 2024-06-10T03:04:28.751 INFO:teuthology.orchestra.run.smithi188.stdout: "exported_inodes": 2298, 2024-06-10T03:04:28.751 INFO:teuthology.orchestra.run.smithi188.stdout: "imported": 83, 2024-06-10T03:04:28.751 INFO:teuthology.orchestra.run.smithi188.stdout: "imported_inodes": 272, 2024-06-10T03:04:28.751 INFO:teuthology.orchestra.run.smithi188.stdout: "openino_dir_fetch": 0, 2024-06-10T03:04:28.751 INFO:teuthology.orchestra.run.smithi188.stdout: "openino_backtrace_fetch": 57595, 2024-06-10T03:04:28.752 INFO:teuthology.orchestra.run.smithi188.stdout: "openino_peer_discover": 0, 2024-06-10T03:04:28.752 INFO:teuthology.orchestra.run.smithi188.stdout: "root_rfiles": 3367, 2024-06-10T03:04:28.752 INFO:teuthology.orchestra.run.smithi188.stdout: "root_rbytes": 68050036, 2024-06-10T03:04:28.752 INFO:teuthology.orchestra.run.smithi188.stdout: "root_rsnaps": 0, 2024-06-10T03:04:28.752 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_backtrace_fetch": 57595, 2024-06-10T03:04:28.752 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_set_tag": 0, 2024-06-10T03:04:28.752 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_backtrace_repaired": 0, 2024-06-10T03:04:28.752 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_inotable_repaired": 0, 2024-06-10T03:04:28.752 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_dir_inodes": 360, 2024-06-10T03:04:28.752 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_dir_base_inodes": 0, 2024-06-10T03:04:28.752 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_dirfrag_rstats": 360, 2024-06-10T03:04:28.752 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_file_inodes": 57235, 2024-06-10T03:04:28.752 INFO:teuthology.orchestra.run.smithi188.stdout: "handle_inode_file_caps": 30, 2024-06-10T03:04:28.752 INFO:teuthology.orchestra.run.smithi188.stdout: "ceph_cap_op_revoke": 84297, 2024-06-10T03:04:28.753 INFO:teuthology.orchestra.run.smithi188.stdout: "ceph_cap_op_grant": 108046, 2024-06-10T03:04:28.753 INFO:teuthology.orchestra.run.smithi188.stdout: "ceph_cap_op_trunc": 1, 2024-06-10T03:04:28.753 INFO:teuthology.orchestra.run.smithi188.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-06-10T03:04:28.753 INFO:teuthology.orchestra.run.smithi188.stdout: "ceph_cap_op_flush_ack": 0, 2024-06-10T03:04:28.753 INFO:teuthology.orchestra.run.smithi188.stdout: "handle_client_caps": 131688, 2024-06-10T03:04:28.753 INFO:teuthology.orchestra.run.smithi188.stdout: "handle_client_caps_dirty": 38283, 2024-06-10T03:04:28.753 INFO:teuthology.orchestra.run.smithi188.stdout: "handle_client_cap_release": 7550, 2024-06-10T03:04:28.753 INFO:teuthology.orchestra.run.smithi188.stdout: "process_request_cap_release": 75908 2024-06-10T03:04:28.753 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:28.753 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_cache": { 2024-06-10T03:04:28.753 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_update": 3257, 2024-06-10T03:04:28.753 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_update_receipt": 29, 2024-06-10T03:04:28.753 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_try_discover": 19, 2024-06-10T03:04:28.753 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_send_discover": 48, 2024-06-10T03:04:28.754 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_handle_discover": 6155, 2024-06-10T03:04:28.754 INFO:teuthology.orchestra.run.smithi188.stdout: "num_strays": 1, 2024-06-10T03:04:28.754 INFO:teuthology.orchestra.run.smithi188.stdout: "num_strays_delayed": 0, 2024-06-10T03:04:28.754 INFO:teuthology.orchestra.run.smithi188.stdout: "num_strays_enqueuing": 0, 2024-06-10T03:04:28.754 INFO:teuthology.orchestra.run.smithi188.stdout: "strays_created": 36484, 2024-06-10T03:04:28.754 INFO:teuthology.orchestra.run.smithi188.stdout: "strays_enqueued": 36483, 2024-06-10T03:04:28.754 INFO:teuthology.orchestra.run.smithi188.stdout: "strays_reintegrated": 0, 2024-06-10T03:04:28.754 INFO:teuthology.orchestra.run.smithi188.stdout: "strays_migrated": 0, 2024-06-10T03:04:28.754 INFO:teuthology.orchestra.run.smithi188.stdout: "num_recovering_processing": 0, 2024-06-10T03:04:28.754 INFO:teuthology.orchestra.run.smithi188.stdout: "num_recovering_enqueued": 0, 2024-06-10T03:04:28.754 INFO:teuthology.orchestra.run.smithi188.stdout: "num_recovering_prioritized": 0, 2024-06-10T03:04:28.754 INFO:teuthology.orchestra.run.smithi188.stdout: "recovery_started": 0, 2024-06-10T03:04:28.754 INFO:teuthology.orchestra.run.smithi188.stdout: "recovery_completed": 0, 2024-06-10T03:04:28.754 INFO:teuthology.orchestra.run.smithi188.stdout: "ireq_quiesce_path": 6, 2024-06-10T03:04:28.755 INFO:teuthology.orchestra.run.smithi188.stdout: "ireq_quiesce_inode": 49159, 2024-06-10T03:04:28.755 INFO:teuthology.orchestra.run.smithi188.stdout: "ireq_enqueue_scrub": 0, 2024-06-10T03:04:28.755 INFO:teuthology.orchestra.run.smithi188.stdout: "ireq_exportdir": 20, 2024-06-10T03:04:28.755 INFO:teuthology.orchestra.run.smithi188.stdout: "ireq_flush": 0, 2024-06-10T03:04:28.755 INFO:teuthology.orchestra.run.smithi188.stdout: "ireq_fragmentdir": 133, 2024-06-10T03:04:28.755 INFO:teuthology.orchestra.run.smithi188.stdout: "ireq_fragstats": 0, 2024-06-10T03:04:28.755 INFO:teuthology.orchestra.run.smithi188.stdout: "ireq_inodestats": 0 2024-06-10T03:04:28.755 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:28.755 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_log": { 2024-06-10T03:04:28.755 INFO:teuthology.orchestra.run.smithi188.stdout: "evlrg": 0, 2024-06-10T03:04:28.755 INFO:teuthology.orchestra.run.smithi188.stdout: "evadd": 232591, 2024-06-10T03:04:28.755 INFO:teuthology.orchestra.run.smithi188.stdout: "evex": 133829, 2024-06-10T03:04:28.755 INFO:teuthology.orchestra.run.smithi188.stdout: "evtrm": 122901, 2024-06-10T03:04:28.755 INFO:teuthology.orchestra.run.smithi188.stdout: "ev": 109690, 2024-06-10T03:04:28.756 INFO:teuthology.orchestra.run.smithi188.stdout: "evexg": 0, 2024-06-10T03:04:28.756 INFO:teuthology.orchestra.run.smithi188.stdout: "evexd": 10928, 2024-06-10T03:04:28.756 INFO:teuthology.orchestra.run.smithi188.stdout: "segadd": 294, 2024-06-10T03:04:28.756 INFO:teuthology.orchestra.run.smithi188.stdout: "segex": 166, 2024-06-10T03:04:28.756 INFO:teuthology.orchestra.run.smithi188.stdout: "segtrm": 153, 2024-06-10T03:04:28.756 INFO:teuthology.orchestra.run.smithi188.stdout: "seg": 141, 2024-06-10T03:04:28.756 INFO:teuthology.orchestra.run.smithi188.stdout: "segmjr": 9, 2024-06-10T03:04:28.756 INFO:teuthology.orchestra.run.smithi188.stdout: "segexg": 0, 2024-06-10T03:04:28.756 INFO:teuthology.orchestra.run.smithi188.stdout: "segexd": 13, 2024-06-10T03:04:28.756 INFO:teuthology.orchestra.run.smithi188.stdout: "expos": 297215900, 2024-06-10T03:04:28.756 INFO:teuthology.orchestra.run.smithi188.stdout: "wrpos": 504464206, 2024-06-10T03:04:28.756 INFO:teuthology.orchestra.run.smithi188.stdout: "rdpos": 0, 2024-06-10T03:04:28.756 INFO:teuthology.orchestra.run.smithi188.stdout: "jlat": { 2024-06-10T03:04:28.756 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 6953, 2024-06-10T03:04:28.756 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 1428.715287358, 2024-06-10T03:04:28.757 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.205481847 2024-06-10T03:04:28.757 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:28.757 INFO:teuthology.orchestra.run.smithi188.stdout: "replayed": 0 2024-06-10T03:04:28.757 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:28.757 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_mem": { 2024-06-10T03:04:28.757 INFO:teuthology.orchestra.run.smithi188.stdout: "ino": 35, 2024-06-10T03:04:28.757 INFO:teuthology.orchestra.run.smithi188.stdout: "ino+": 37667, 2024-06-10T03:04:28.757 INFO:teuthology.orchestra.run.smithi188.stdout: "ino-": 37632, 2024-06-10T03:04:28.757 INFO:teuthology.orchestra.run.smithi188.stdout: "dir": 41, 2024-06-10T03:04:28.757 INFO:teuthology.orchestra.run.smithi188.stdout: "dir+": 1201, 2024-06-10T03:04:28.757 INFO:teuthology.orchestra.run.smithi188.stdout: "dir-": 1160, 2024-06-10T03:04:28.757 INFO:teuthology.orchestra.run.smithi188.stdout: "dn": 714, 2024-06-10T03:04:28.757 INFO:teuthology.orchestra.run.smithi188.stdout: "dn+": 82357, 2024-06-10T03:04:28.757 INFO:teuthology.orchestra.run.smithi188.stdout: "dn-": 81643, 2024-06-10T03:04:28.758 INFO:teuthology.orchestra.run.smithi188.stdout: "cap": 0, 2024-06-10T03:04:28.758 INFO:teuthology.orchestra.run.smithi188.stdout: "cap+": 54731, 2024-06-10T03:04:28.758 INFO:teuthology.orchestra.run.smithi188.stdout: "cap-": 54731, 2024-06-10T03:04:28.758 INFO:teuthology.orchestra.run.smithi188.stdout: "rss": 175364, 2024-06-10T03:04:28.758 INFO:teuthology.orchestra.run.smithi188.stdout: "heap": 223612 2024-06-10T03:04:28.758 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:28.758 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_server": { 2024-06-10T03:04:28.758 INFO:teuthology.orchestra.run.smithi188.stdout: "dispatch_client_request": 242547, 2024-06-10T03:04:28.758 INFO:teuthology.orchestra.run.smithi188.stdout: "dispatch_server_request": 5338, 2024-06-10T03:04:28.758 INFO:teuthology.orchestra.run.smithi188.stdout: "handle_client_request": 237560, 2024-06-10T03:04:28.758 INFO:teuthology.orchestra.run.smithi188.stdout: "handle_client_session": 31, 2024-06-10T03:04:28.758 INFO:teuthology.orchestra.run.smithi188.stdout: "handle_peer_request": 10799, 2024-06-10T03:04:28.758 INFO:teuthology.orchestra.run.smithi188.stdout: "req_create_latency": { 2024-06-10T03:04:28.758 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 37334, 2024-06-10T03:04:28.759 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 1031.581718105, 2024-06-10T03:04:28.759 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.027631159 2024-06-10T03:04:28.759 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:28.759 INFO:teuthology.orchestra.run.smithi188.stdout: "req_getattr_latency": { 2024-06-10T03:04:28.759 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 3632, 2024-06-10T03:04:28.759 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 528.513450253, 2024-06-10T03:04:28.759 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.145515817 2024-06-10T03:04:28.759 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:28.759 INFO:teuthology.orchestra.run.smithi188.stdout: "req_getfilelock_latency": { 2024-06-10T03:04:28.759 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-06-10T03:04:28.759 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-06-10T03:04:28.759 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-06-10T03:04:28.759 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:28.759 INFO:teuthology.orchestra.run.smithi188.stdout: "req_link_latency": { 2024-06-10T03:04:28.760 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-06-10T03:04:28.760 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-06-10T03:04:28.760 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-06-10T03:04:28.760 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:28.760 INFO:teuthology.orchestra.run.smithi188.stdout: "req_lookup_latency": { 2024-06-10T03:04:28.760 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 115591, 2024-06-10T03:04:28.760 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 5331.953551504, 2024-06-10T03:04:28.760 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.046127756 2024-06-10T03:04:28.760 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:28.760 INFO:teuthology.orchestra.run.smithi188.stdout: "req_lookuphash_latency": { 2024-06-10T03:04:28.760 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-06-10T03:04:28.760 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-06-10T03:04:28.760 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-06-10T03:04:28.760 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:28.760 INFO:teuthology.orchestra.run.smithi188.stdout: "req_lookupino_latency": { 2024-06-10T03:04:28.761 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-06-10T03:04:28.761 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-06-10T03:04:28.761 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-06-10T03:04:28.761 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:28.761 INFO:teuthology.orchestra.run.smithi188.stdout: "req_lookupname_latency": { 2024-06-10T03:04:28.761 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-06-10T03:04:28.761 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-06-10T03:04:28.761 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-06-10T03:04:28.761 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:28.761 INFO:teuthology.orchestra.run.smithi188.stdout: "req_lookupparent_latency": { 2024-06-10T03:04:28.761 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-06-10T03:04:28.761 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-06-10T03:04:28.761 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-06-10T03:04:28.762 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:28.762 INFO:teuthology.orchestra.run.smithi188.stdout: "req_lookupsnap_latency": { 2024-06-10T03:04:28.762 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-06-10T03:04:28.762 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-06-10T03:04:28.762 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-06-10T03:04:28.762 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:28.762 INFO:teuthology.orchestra.run.smithi188.stdout: "req_lssnap_latency": { 2024-06-10T03:04:28.763 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-06-10T03:04:28.763 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-06-10T03:04:28.763 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-06-10T03:04:28.763 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:28.763 INFO:teuthology.orchestra.run.smithi188.stdout: "req_mkdir_latency": { 2024-06-10T03:04:28.763 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 129, 2024-06-10T03:04:28.763 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 10.802835590, 2024-06-10T03:04:28.763 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.083742911 2024-06-10T03:04:28.763 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:28.763 INFO:teuthology.orchestra.run.smithi188.stdout: "req_mknod_latency": { 2024-06-10T03:04:28.763 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-06-10T03:04:28.763 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-06-10T03:04:28.763 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-06-10T03:04:28.763 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:28.764 INFO:teuthology.orchestra.run.smithi188.stdout: "req_mksnap_latency": { 2024-06-10T03:04:28.764 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-06-10T03:04:28.764 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-06-10T03:04:28.764 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-06-10T03:04:28.764 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:28.764 INFO:teuthology.orchestra.run.smithi188.stdout: "req_open_latency": { 2024-06-10T03:04:28.764 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 18037, 2024-06-10T03:04:28.764 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 494.507201888, 2024-06-10T03:04:28.764 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.027416266 2024-06-10T03:04:28.764 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:28.764 INFO:teuthology.orchestra.run.smithi188.stdout: "req_readdir_latency": { 2024-06-10T03:04:28.764 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 857, 2024-06-10T03:04:28.764 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 1.724148273, 2024-06-10T03:04:28.764 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.002011841 2024-06-10T03:04:28.765 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:28.765 INFO:teuthology.orchestra.run.smithi188.stdout: "req_rename_latency": { 2024-06-10T03:04:28.765 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 37310, 2024-06-10T03:04:28.765 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 1031.162521406, 2024-06-10T03:04:28.765 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.027637698 2024-06-10T03:04:28.765 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:28.765 INFO:teuthology.orchestra.run.smithi188.stdout: "req_renamesnap_latency": { 2024-06-10T03:04:28.765 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-06-10T03:04:28.765 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-06-10T03:04:28.765 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-06-10T03:04:28.765 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:28.765 INFO:teuthology.orchestra.run.smithi188.stdout: "req_snapdiff_latency": { 2024-06-10T03:04:28.765 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-06-10T03:04:28.765 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-06-10T03:04:28.766 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-06-10T03:04:28.766 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:28.766 INFO:teuthology.orchestra.run.smithi188.stdout: "req_rmdir_latency": { 2024-06-10T03:04:28.766 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 129, 2024-06-10T03:04:28.766 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 31.939412366, 2024-06-10T03:04:28.766 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.247592343 2024-06-10T03:04:28.766 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:28.766 INFO:teuthology.orchestra.run.smithi188.stdout: "req_rmsnap_latency": { 2024-06-10T03:04:28.766 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-06-10T03:04:28.766 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-06-10T03:04:28.766 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-06-10T03:04:28.766 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:28.766 INFO:teuthology.orchestra.run.smithi188.stdout: "req_rmxattr_latency": { 2024-06-10T03:04:28.766 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-06-10T03:04:28.767 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-06-10T03:04:28.767 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-06-10T03:04:28.767 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:28.767 INFO:teuthology.orchestra.run.smithi188.stdout: "req_setattr_latency": { 2024-06-10T03:04:28.767 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 1, 2024-06-10T03:04:28.767 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 1.307671230, 2024-06-10T03:04:28.767 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 1.307671230 2024-06-10T03:04:28.767 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:28.767 INFO:teuthology.orchestra.run.smithi188.stdout: "req_setdirlayout_latency": { 2024-06-10T03:04:28.767 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-06-10T03:04:28.767 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-06-10T03:04:28.767 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-06-10T03:04:28.767 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:28.767 INFO:teuthology.orchestra.run.smithi188.stdout: "req_setfilelock_latency": { 2024-06-10T03:04:28.767 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-06-10T03:04:28.768 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-06-10T03:04:28.768 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-06-10T03:04:28.768 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:28.768 INFO:teuthology.orchestra.run.smithi188.stdout: "req_setlayout_latency": { 2024-06-10T03:04:28.768 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-06-10T03:04:28.768 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-06-10T03:04:28.768 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-06-10T03:04:28.768 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:28.768 INFO:teuthology.orchestra.run.smithi188.stdout: "req_setxattr_latency": { 2024-06-10T03:04:28.768 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-06-10T03:04:28.768 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-06-10T03:04:28.768 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-06-10T03:04:28.768 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:28.768 INFO:teuthology.orchestra.run.smithi188.stdout: "req_symlink_latency": { 2024-06-10T03:04:28.769 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-06-10T03:04:28.769 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-06-10T03:04:28.769 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-06-10T03:04:28.769 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:28.769 INFO:teuthology.orchestra.run.smithi188.stdout: "req_unlink_latency": { 2024-06-10T03:04:28.769 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 20518, 2024-06-10T03:04:28.769 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 37.953754399, 2024-06-10T03:04:28.769 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.001849778 2024-06-10T03:04:28.769 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:28.769 INFO:teuthology.orchestra.run.smithi188.stdout: "cap_revoke_eviction": 0, 2024-06-10T03:04:28.769 INFO:teuthology.orchestra.run.smithi188.stdout: "cap_acquisition_throttle": 0, 2024-06-10T03:04:28.769 INFO:teuthology.orchestra.run.smithi188.stdout: "req_getvxattr_latency": { 2024-06-10T03:04:28.769 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-06-10T03:04:28.769 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-06-10T03:04:28.770 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-06-10T03:04:28.770 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-06-10T03:04:28.770 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:28.770 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_sessions": { 2024-06-10T03:04:28.770 INFO:teuthology.orchestra.run.smithi188.stdout: "session_count": 1, 2024-06-10T03:04:28.770 INFO:teuthology.orchestra.run.smithi188.stdout: "session_add": 2, 2024-06-10T03:04:28.770 INFO:teuthology.orchestra.run.smithi188.stdout: "session_remove": 1, 2024-06-10T03:04:28.770 INFO:teuthology.orchestra.run.smithi188.stdout: "sessions_open": 1, 2024-06-10T03:04:28.770 INFO:teuthology.orchestra.run.smithi188.stdout: "sessions_stale": 0, 2024-06-10T03:04:28.770 INFO:teuthology.orchestra.run.smithi188.stdout: "total_load": 42305, 2024-06-10T03:04:28.770 INFO:teuthology.orchestra.run.smithi188.stdout: "average_load": 42305, 2024-06-10T03:04:28.770 INFO:teuthology.orchestra.run.smithi188.stdout: "avg_session_uptime": 2350, 2024-06-10T03:04:28.771 INFO:teuthology.orchestra.run.smithi188.stdout: "mdthresh_evicted": 0 2024-06-10T03:04:28.771 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:28.771 INFO:teuthology.orchestra.run.smithi188.stdout: "mempool": { 2024-06-10T03:04:28.771 INFO:teuthology.orchestra.run.smithi188.stdout: "bloom_filter_bytes": 0, 2024-06-10T03:04:28.771 INFO:teuthology.orchestra.run.smithi188.stdout: "bloom_filter_items": 0, 2024-06-10T03:04:28.771 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_alloc_bytes": 0, 2024-06-10T03:04:28.771 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_alloc_items": 0, 2024-06-10T03:04:28.771 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_data_bytes": 0, 2024-06-10T03:04:28.771 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_data_items": 0, 2024-06-10T03:04:28.771 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-10T03:04:28.771 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_onode_items": 0, 2024-06-10T03:04:28.771 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-10T03:04:28.771 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_meta_items": 0, 2024-06-10T03:04:28.771 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_other_bytes": 0, 2024-06-10T03:04:28.772 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_other_items": 0, 2024-06-10T03:04:28.772 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-10T03:04:28.772 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_buffer_items": 0, 2024-06-10T03:04:28.772 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_extent_bytes": 0, 2024-06-10T03:04:28.772 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_extent_items": 0, 2024-06-10T03:04:28.772 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_blob_bytes": 0, 2024-06-10T03:04:28.772 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_blob_items": 0, 2024-06-10T03:04:28.772 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-10T03:04:28.772 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_shared_blob_items": 0, 2024-06-10T03:04:28.772 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-10T03:04:28.772 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_inline_bl_items": 0, 2024-06-10T03:04:28.772 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_fsck_bytes": 0, 2024-06-10T03:04:28.772 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_fsck_items": 0, 2024-06-10T03:04:28.772 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_txc_bytes": 0, 2024-06-10T03:04:28.772 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_txc_items": 0, 2024-06-10T03:04:28.773 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-10T03:04:28.773 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_writing_deferred_items": 0, 2024-06-10T03:04:28.773 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_writing_bytes": 0, 2024-06-10T03:04:28.773 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_writing_items": 0, 2024-06-10T03:04:28.773 INFO:teuthology.orchestra.run.smithi188.stdout: "bluefs_bytes": 0, 2024-06-10T03:04:28.773 INFO:teuthology.orchestra.run.smithi188.stdout: "bluefs_items": 0, 2024-06-10T03:04:28.773 INFO:teuthology.orchestra.run.smithi188.stdout: "bluefs_file_reader_bytes": 0, 2024-06-10T03:04:28.773 INFO:teuthology.orchestra.run.smithi188.stdout: "bluefs_file_reader_items": 0, 2024-06-10T03:04:28.773 INFO:teuthology.orchestra.run.smithi188.stdout: "bluefs_file_writer_bytes": 0, 2024-06-10T03:04:28.773 INFO:teuthology.orchestra.run.smithi188.stdout: "bluefs_file_writer_items": 0, 2024-06-10T03:04:28.773 INFO:teuthology.orchestra.run.smithi188.stdout: "buffer_anon_bytes": 311484, 2024-06-10T03:04:28.773 INFO:teuthology.orchestra.run.smithi188.stdout: "buffer_anon_items": 110, 2024-06-10T03:04:28.773 INFO:teuthology.orchestra.run.smithi188.stdout: "buffer_meta_bytes": 0, 2024-06-10T03:04:28.773 INFO:teuthology.orchestra.run.smithi188.stdout: "buffer_meta_items": 0, 2024-06-10T03:04:28.774 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_bytes": 0, 2024-06-10T03:04:28.774 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_items": 0, 2024-06-10T03:04:28.774 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_mapbl_bytes": 0, 2024-06-10T03:04:28.774 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_mapbl_items": 0, 2024-06-10T03:04:28.774 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_pglog_bytes": 0, 2024-06-10T03:04:28.774 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_pglog_items": 0, 2024-06-10T03:04:28.774 INFO:teuthology.orchestra.run.smithi188.stdout: "osdmap_bytes": 8984, 2024-06-10T03:04:28.774 INFO:teuthology.orchestra.run.smithi188.stdout: "osdmap_items": 172, 2024-06-10T03:04:28.774 INFO:teuthology.orchestra.run.smithi188.stdout: "osdmap_mapping_bytes": 0, 2024-06-10T03:04:28.774 INFO:teuthology.orchestra.run.smithi188.stdout: "osdmap_mapping_items": 0, 2024-06-10T03:04:28.774 INFO:teuthology.orchestra.run.smithi188.stdout: "pgmap_bytes": 0, 2024-06-10T03:04:28.774 INFO:teuthology.orchestra.run.smithi188.stdout: "pgmap_items": 0, 2024-06-10T03:04:28.774 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_co_bytes": 673352, 2024-06-10T03:04:28.774 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_co_items": 7778, 2024-06-10T03:04:28.775 INFO:teuthology.orchestra.run.smithi188.stdout: "unittest_1_bytes": 0, 2024-06-10T03:04:28.775 INFO:teuthology.orchestra.run.smithi188.stdout: "unittest_1_items": 0, 2024-06-10T03:04:28.775 INFO:teuthology.orchestra.run.smithi188.stdout: "unittest_2_bytes": 0, 2024-06-10T03:04:28.775 INFO:teuthology.orchestra.run.smithi188.stdout: "unittest_2_items": 0 2024-06-10T03:04:28.775 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:28.775 INFO:teuthology.orchestra.run.smithi188.stdout: "objecter": { 2024-06-10T03:04:28.775 INFO:teuthology.orchestra.run.smithi188.stdout: "op_active": 0, 2024-06-10T03:04:28.775 INFO:teuthology.orchestra.run.smithi188.stdout: "op_laggy": 0, 2024-06-10T03:04:28.775 INFO:teuthology.orchestra.run.smithi188.stdout: "op_send": 165878, 2024-06-10T03:04:28.775 INFO:teuthology.orchestra.run.smithi188.stdout: "op_send_bytes": 549709361, 2024-06-10T03:04:28.775 INFO:teuthology.orchestra.run.smithi188.stdout: "op_resend": 33, 2024-06-10T03:04:28.775 INFO:teuthology.orchestra.run.smithi188.stdout: "op_reply": 165878, 2024-06-10T03:04:28.775 INFO:teuthology.orchestra.run.smithi188.stdout: "op_latency": { 2024-06-10T03:04:28.775 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 165878, 2024-06-10T03:04:28.776 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 3467.072469581, 2024-06-10T03:04:28.776 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.020901339 2024-06-10T03:04:28.776 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:28.776 INFO:teuthology.orchestra.run.smithi188.stdout: "op_inflight": 0, 2024-06-10T03:04:28.776 INFO:teuthology.orchestra.run.smithi188.stdout: "oplen_avg": { 2024-06-10T03:04:28.776 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 165878, 2024-06-10T03:04:28.776 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 204142 2024-06-10T03:04:28.776 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:28.776 INFO:teuthology.orchestra.run.smithi188.stdout: "op": 165878, 2024-06-10T03:04:28.776 INFO:teuthology.orchestra.run.smithi188.stdout: "op_r": 58416, 2024-06-10T03:04:28.776 INFO:teuthology.orchestra.run.smithi188.stdout: "op_w": 107462, 2024-06-10T03:04:28.776 INFO:teuthology.orchestra.run.smithi188.stdout: "op_rmw": 0, 2024-06-10T03:04:28.777 INFO:teuthology.orchestra.run.smithi188.stdout: "op_pg": 0, 2024-06-10T03:04:28.777 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_stat": 547, 2024-06-10T03:04:28.777 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_create": 23152, 2024-06-10T03:04:28.777 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_read": 805, 2024-06-10T03:04:28.777 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_write": 7758, 2024-06-10T03:04:28.777 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_writefull": 374, 2024-06-10T03:04:28.777 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_writesame": 0, 2024-06-10T03:04:28.777 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_append": 0, 2024-06-10T03:04:28.777 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_zero": 0, 2024-06-10T03:04:28.777 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_truncate": 117, 2024-06-10T03:04:28.777 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_delete": 74041, 2024-06-10T03:04:28.777 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_mapext": 0, 2024-06-10T03:04:28.777 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_sparse_read": 0, 2024-06-10T03:04:28.777 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_clonerange": 0, 2024-06-10T03:04:28.778 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_getxattr": 57596, 2024-06-10T03:04:28.778 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_setxattr": 34782, 2024-06-10T03:04:28.778 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_cmpxattr": 0, 2024-06-10T03:04:28.778 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_rmxattr": 0, 2024-06-10T03:04:28.778 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_resetxattrs": 0, 2024-06-10T03:04:28.778 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_call": 0, 2024-06-10T03:04:28.778 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_watch": 0, 2024-06-10T03:04:28.778 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_notify": 0, 2024-06-10T03:04:28.778 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_src_cmpxattr": 0, 2024-06-10T03:04:28.778 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_pgls": 0, 2024-06-10T03:04:28.778 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_pgls_filter": 0, 2024-06-10T03:04:28.778 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_other": 1, 2024-06-10T03:04:28.778 INFO:teuthology.orchestra.run.smithi188.stdout: "linger_active": 0, 2024-06-10T03:04:28.778 INFO:teuthology.orchestra.run.smithi188.stdout: "linger_send": 0, 2024-06-10T03:04:28.779 INFO:teuthology.orchestra.run.smithi188.stdout: "linger_resend": 0, 2024-06-10T03:04:28.779 INFO:teuthology.orchestra.run.smithi188.stdout: "linger_ping": 0, 2024-06-10T03:04:28.779 INFO:teuthology.orchestra.run.smithi188.stdout: "poolop_active": 0, 2024-06-10T03:04:28.779 INFO:teuthology.orchestra.run.smithi188.stdout: "poolop_send": 0, 2024-06-10T03:04:28.779 INFO:teuthology.orchestra.run.smithi188.stdout: "poolop_resend": 0, 2024-06-10T03:04:28.779 INFO:teuthology.orchestra.run.smithi188.stdout: "poolstat_active": 0, 2024-06-10T03:04:28.779 INFO:teuthology.orchestra.run.smithi188.stdout: "poolstat_send": 0, 2024-06-10T03:04:28.779 INFO:teuthology.orchestra.run.smithi188.stdout: "poolstat_resend": 0, 2024-06-10T03:04:28.779 INFO:teuthology.orchestra.run.smithi188.stdout: "statfs_active": 0, 2024-06-10T03:04:28.779 INFO:teuthology.orchestra.run.smithi188.stdout: "statfs_send": 0, 2024-06-10T03:04:28.779 INFO:teuthology.orchestra.run.smithi188.stdout: "statfs_resend": 0, 2024-06-10T03:04:28.779 INFO:teuthology.orchestra.run.smithi188.stdout: "command_active": 0, 2024-06-10T03:04:28.779 INFO:teuthology.orchestra.run.smithi188.stdout: "command_send": 0, 2024-06-10T03:04:28.780 INFO:teuthology.orchestra.run.smithi188.stdout: "command_resend": 0, 2024-06-10T03:04:28.780 INFO:teuthology.orchestra.run.smithi188.stdout: "map_epoch": 100, 2024-06-10T03:04:28.780 INFO:teuthology.orchestra.run.smithi188.stdout: "map_full": 0, 2024-06-10T03:04:28.780 INFO:teuthology.orchestra.run.smithi188.stdout: "map_inc": 2, 2024-06-10T03:04:28.780 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_sessions": 12, 2024-06-10T03:04:28.780 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_session_open": 12, 2024-06-10T03:04:28.780 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_session_close": 0, 2024-06-10T03:04:28.780 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_laggy": 0, 2024-06-10T03:04:28.780 INFO:teuthology.orchestra.run.smithi188.stdout: "omap_wr": 3961, 2024-06-10T03:04:28.780 INFO:teuthology.orchestra.run.smithi188.stdout: "omap_rd": 32, 2024-06-10T03:04:28.780 INFO:teuthology.orchestra.run.smithi188.stdout: "omap_del": 976 2024-06-10T03:04:28.780 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:28.780 INFO:teuthology.orchestra.run.smithi188.stdout: "oft": { 2024-06-10T03:04:28.781 INFO:teuthology.orchestra.run.smithi188.stdout: "omap_total_objs": 1, 2024-06-10T03:04:28.781 INFO:teuthology.orchestra.run.smithi188.stdout: "omap_total_kv_pairs": 0, 2024-06-10T03:04:28.781 INFO:teuthology.orchestra.run.smithi188.stdout: "omap_total_updates": 0, 2024-06-10T03:04:28.781 INFO:teuthology.orchestra.run.smithi188.stdout: "omap_total_removes": 0 2024-06-10T03:04:28.781 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:28.781 INFO:teuthology.orchestra.run.smithi188.stdout: "osd-slow-ops": { 2024-06-10T03:04:28.781 INFO:teuthology.orchestra.run.smithi188.stdout: "slow_ops_count": 49082 2024-06-10T03:04:28.781 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:28.781 INFO:teuthology.orchestra.run.smithi188.stdout: "purge_queue": { 2024-06-10T03:04:28.781 INFO:teuthology.orchestra.run.smithi188.stdout: "pq_executing_ops": 0, 2024-06-10T03:04:28.781 INFO:teuthology.orchestra.run.smithi188.stdout: "pq_executing_ops_high_water": 17, 2024-06-10T03:04:28.781 INFO:teuthology.orchestra.run.smithi188.stdout: "pq_executing": 0, 2024-06-10T03:04:28.781 INFO:teuthology.orchestra.run.smithi188.stdout: "pq_executing_high_water": 5, 2024-06-10T03:04:28.781 INFO:teuthology.orchestra.run.smithi188.stdout: "pq_executed_ops": 73746, 2024-06-10T03:04:28.781 INFO:teuthology.orchestra.run.smithi188.stdout: "pq_executed": 36483, 2024-06-10T03:04:28.782 INFO:teuthology.orchestra.run.smithi188.stdout: "pq_item_in_journal": 0 2024-06-10T03:04:28.782 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:28.782 INFO:teuthology.orchestra.run.smithi188.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-10T03:04:28.782 INFO:teuthology.orchestra.run.smithi188.stdout: "val": 0, 2024-06-10T03:04:28.782 INFO:teuthology.orchestra.run.smithi188.stdout: "max": 104857600, 2024-06-10T03:04:28.782 INFO:teuthology.orchestra.run.smithi188.stdout: "get_started": 0, 2024-06-10T03:04:28.782 INFO:teuthology.orchestra.run.smithi188.stdout: "get": 577460, 2024-06-10T03:04:28.782 INFO:teuthology.orchestra.run.smithi188.stdout: "get_sum": 133801371, 2024-06-10T03:04:28.782 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_fail": 0, 2024-06-10T03:04:28.782 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_success": 577460, 2024-06-10T03:04:28.782 INFO:teuthology.orchestra.run.smithi188.stdout: "take": 0, 2024-06-10T03:04:28.782 INFO:teuthology.orchestra.run.smithi188.stdout: "take_sum": 0, 2024-06-10T03:04:28.782 INFO:teuthology.orchestra.run.smithi188.stdout: "put": 577460, 2024-06-10T03:04:28.783 INFO:teuthology.orchestra.run.smithi188.stdout: "put_sum": 133801371, 2024-06-10T03:04:28.783 INFO:teuthology.orchestra.run.smithi188.stdout: "wait": { 2024-06-10T03:04:28.783 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-06-10T03:04:28.783 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-06-10T03:04:28.783 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-06-10T03:04:28.783 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-06-10T03:04:28.783 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:28.783 INFO:teuthology.orchestra.run.smithi188.stdout: "throttle-objecter_bytes": { 2024-06-10T03:04:28.783 INFO:teuthology.orchestra.run.smithi188.stdout: "val": 0, 2024-06-10T03:04:28.783 INFO:teuthology.orchestra.run.smithi188.stdout: "max": 104857600, 2024-06-10T03:04:28.783 INFO:teuthology.orchestra.run.smithi188.stdout: "get_started": 0, 2024-06-10T03:04:28.783 INFO:teuthology.orchestra.run.smithi188.stdout: "get": 0, 2024-06-10T03:04:28.783 INFO:teuthology.orchestra.run.smithi188.stdout: "get_sum": 0, 2024-06-10T03:04:28.783 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_fail": 0, 2024-06-10T03:04:28.784 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_success": 0, 2024-06-10T03:04:28.784 INFO:teuthology.orchestra.run.smithi188.stdout: "take": 165878, 2024-06-10T03:04:28.784 INFO:teuthology.orchestra.run.smithi188.stdout: "take_sum": 553685656, 2024-06-10T03:04:28.784 INFO:teuthology.orchestra.run.smithi188.stdout: "put": 165878, 2024-06-10T03:04:28.784 INFO:teuthology.orchestra.run.smithi188.stdout: "put_sum": 553685656, 2024-06-10T03:04:28.784 INFO:teuthology.orchestra.run.smithi188.stdout: "wait": { 2024-06-10T03:04:28.784 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-06-10T03:04:28.784 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-06-10T03:04:28.784 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-06-10T03:04:28.784 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-06-10T03:04:28.784 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:28.784 INFO:teuthology.orchestra.run.smithi188.stdout: "throttle-objecter_ops": { 2024-06-10T03:04:28.784 INFO:teuthology.orchestra.run.smithi188.stdout: "val": 0, 2024-06-10T03:04:28.785 INFO:teuthology.orchestra.run.smithi188.stdout: "max": 1024, 2024-06-10T03:04:28.785 INFO:teuthology.orchestra.run.smithi188.stdout: "get_started": 0, 2024-06-10T03:04:28.785 INFO:teuthology.orchestra.run.smithi188.stdout: "get": 0, 2024-06-10T03:04:28.785 INFO:teuthology.orchestra.run.smithi188.stdout: "get_sum": 0, 2024-06-10T03:04:28.785 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_fail": 0, 2024-06-10T03:04:28.785 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_success": 0, 2024-06-10T03:04:28.785 INFO:teuthology.orchestra.run.smithi188.stdout: "take": 165878, 2024-06-10T03:04:28.785 INFO:teuthology.orchestra.run.smithi188.stdout: "take_sum": 165878, 2024-06-10T03:04:28.785 INFO:teuthology.orchestra.run.smithi188.stdout: "put": 165878, 2024-06-10T03:04:28.785 INFO:teuthology.orchestra.run.smithi188.stdout: "put_sum": 165878, 2024-06-10T03:04:28.785 INFO:teuthology.orchestra.run.smithi188.stdout: "wait": { 2024-06-10T03:04:28.785 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-06-10T03:04:28.785 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-06-10T03:04:28.785 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-06-10T03:04:28.786 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-06-10T03:04:28.786 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:28.786 INFO:teuthology.orchestra.run.smithi188.stdout: "throttle-write_buf_throttle": { 2024-06-10T03:04:28.786 INFO:teuthology.orchestra.run.smithi188.stdout: "val": 0, 2024-06-10T03:04:28.786 INFO:teuthology.orchestra.run.smithi188.stdout: "max": 3758096384, 2024-06-10T03:04:28.786 INFO:teuthology.orchestra.run.smithi188.stdout: "get_started": 0, 2024-06-10T03:04:28.786 INFO:teuthology.orchestra.run.smithi188.stdout: "get": 36483, 2024-06-10T03:04:28.786 INFO:teuthology.orchestra.run.smithi188.stdout: "get_sum": 3976551, 2024-06-10T03:04:28.786 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_fail": 0, 2024-06-10T03:04:28.786 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_success": 36483, 2024-06-10T03:04:28.786 INFO:teuthology.orchestra.run.smithi188.stdout: "take": 0, 2024-06-10T03:04:28.786 INFO:teuthology.orchestra.run.smithi188.stdout: "take_sum": 0, 2024-06-10T03:04:28.786 INFO:teuthology.orchestra.run.smithi188.stdout: "put": 805, 2024-06-10T03:04:28.786 INFO:teuthology.orchestra.run.smithi188.stdout: "put_sum": 3976551, 2024-06-10T03:04:28.787 INFO:teuthology.orchestra.run.smithi188.stdout: "wait": { 2024-06-10T03:04:28.787 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-06-10T03:04:28.787 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-06-10T03:04:28.787 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-06-10T03:04:28.787 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-06-10T03:04:28.787 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:28.787 INFO:teuthology.orchestra.run.smithi188.stdout: "throttle-write_buf_throttle-0x558e1e512500": { 2024-06-10T03:04:28.787 INFO:teuthology.orchestra.run.smithi188.stdout: "val": 0, 2024-06-10T03:04:28.787 INFO:teuthology.orchestra.run.smithi188.stdout: "max": 3758096384, 2024-06-10T03:04:28.787 INFO:teuthology.orchestra.run.smithi188.stdout: "get_started": 0, 2024-06-10T03:04:28.787 INFO:teuthology.orchestra.run.smithi188.stdout: "get": 232591, 2024-06-10T03:04:28.787 INFO:teuthology.orchestra.run.smithi188.stdout: "get_sum": 500269902, 2024-06-10T03:04:28.787 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_fail": 0, 2024-06-10T03:04:28.787 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_success": 232591, 2024-06-10T03:04:28.788 INFO:teuthology.orchestra.run.smithi188.stdout: "take": 0, 2024-06-10T03:04:28.788 INFO:teuthology.orchestra.run.smithi188.stdout: "take_sum": 0, 2024-06-10T03:04:28.788 INFO:teuthology.orchestra.run.smithi188.stdout: "put": 6953, 2024-06-10T03:04:28.788 INFO:teuthology.orchestra.run.smithi188.stdout: "put_sum": 500269902, 2024-06-10T03:04:28.788 INFO:teuthology.orchestra.run.smithi188.stdout: "wait": { 2024-06-10T03:04:28.788 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-06-10T03:04:28.788 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-06-10T03:04:28.788 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-06-10T03:04:28.788 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-06-10T03:04:28.788 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-06-10T03:04:28.788 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-06-10T03:04:29.227 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:04:28 smithi052 ceph-mon[32362]: pgmap v742: 129 pgs: 129 active+clean; 228 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 0 B/s wr, 11 op/s 2024-06-10T03:04:29.237 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:04:28 smithi188 ceph-mon[36599]: pgmap v742: 129 pgs: 129 active+clean; 228 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 0 B/s wr, 11 op/s 2024-06-10T03:04:29.389 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:04:28 smithi120 ceph-mon[36634]: pgmap v742: 129 pgs: 129 active+clean; 228 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 0 B/s wr, 11 op/s 2024-06-10T03:04:29.609 INFO:tasks.check_counter:Daemon mds.f mds.exported=20 2024-06-10T03:04:29.609 INFO:tasks.check_counter:Daemon mds.f mds.imported=83 2024-06-10T03:04:29.609 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update=3257 2024-06-10T03:04:29.609 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update_receipt=29 2024-06-10T03:04:29.610 INFO:tasks.check_counter:Daemon mds.f mds.dir_split=117 2024-06-10T03:04:29.610 DEBUG:tasks.check_counter:Getting stats from i 2024-06-10T03:04:29.610 DEBUG:teuthology.orchestra.run.smithi188:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph daemon mds.i perf dump 2024-06-10T03:04:29.849 INFO:teuthology.orchestra.run.smithi188.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.c/config 2024-06-10T03:04:31.227 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:04:30 smithi052 ceph-mon[32362]: pgmap v743: 129 pgs: 129 active+clean; 228 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 9 op/s 2024-06-10T03:04:31.322 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:04:30 smithi188 ceph-mon[36599]: pgmap v743: 129 pgs: 129 active+clean; 228 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 9 op/s 2024-06-10T03:04:31.389 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:04:30 smithi120 ceph-mon[36634]: pgmap v743: 129 pgs: 129 active+clean; 228 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 9 op/s 2024-06-10T03:04:31.612 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-06-10T03:04:31.613 INFO:teuthology.orchestra.run.smithi188.stdout: "AsyncMessenger::Worker-0": { 2024-06-10T03:04:31.613 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_messages": 3596, 2024-06-10T03:04:31.613 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_messages": 4137, 2024-06-10T03:04:31.613 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_bytes": 1135870, 2024-06-10T03:04:31.613 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_bytes": 7582924, 2024-06-10T03:04:31.613 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_created_connections": 26, 2024-06-10T03:04:31.613 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_active_connections": 5, 2024-06-10T03:04:31.613 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_total_time": 0.270967354, 2024-06-10T03:04:31.613 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_send_time": 0.092873884, 2024-06-10T03:04:31.613 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_recv_time": 0.105113996, 2024-06-10T03:04:31.613 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_fast_dispatch_time": 0.058527036, 2024-06-10T03:04:31.613 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_messages_queue_lat": { 2024-06-10T03:04:31.614 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 4132, 2024-06-10T03:04:31.614 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.154134325, 2024-06-10T03:04:31.614 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000037302 2024-06-10T03:04:31.614 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:31.614 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_handle_ack_lat": { 2024-06-10T03:04:31.614 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 5097, 2024-06-10T03:04:31.614 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000855407, 2024-06-10T03:04:31.614 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000167 2024-06-10T03:04:31.614 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:31.614 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_encrypted_bytes": 0, 2024-06-10T03:04:31.614 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_encrypted_bytes": 0 2024-06-10T03:04:31.614 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:31.615 INFO:teuthology.orchestra.run.smithi188.stdout: "AsyncMessenger::Worker-1": { 2024-06-10T03:04:31.615 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_messages": 11404, 2024-06-10T03:04:31.615 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_messages": 8730, 2024-06-10T03:04:31.615 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_bytes": 5122489, 2024-06-10T03:04:31.615 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_bytes": 2664752, 2024-06-10T03:04:31.615 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_created_connections": 9, 2024-06-10T03:04:31.615 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_active_connections": 6, 2024-06-10T03:04:31.615 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_total_time": 0.692263038, 2024-06-10T03:04:31.615 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_send_time": 0.300025684, 2024-06-10T03:04:31.615 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_recv_time": 0.448809006, 2024-06-10T03:04:31.615 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_fast_dispatch_time": 0.038836967, 2024-06-10T03:04:31.615 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_messages_queue_lat": { 2024-06-10T03:04:31.615 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 8731, 2024-06-10T03:04:31.616 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.305283211, 2024-06-10T03:04:31.616 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000034965 2024-06-10T03:04:31.616 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:31.616 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_handle_ack_lat": { 2024-06-10T03:04:31.616 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 17445, 2024-06-10T03:04:31.616 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.003001801, 2024-06-10T03:04:31.616 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000172 2024-06-10T03:04:31.616 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:31.616 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_encrypted_bytes": 160, 2024-06-10T03:04:31.616 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_encrypted_bytes": 404976 2024-06-10T03:04:31.616 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:31.616 INFO:teuthology.orchestra.run.smithi188.stdout: "AsyncMessenger::Worker-2": { 2024-06-10T03:04:31.616 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_messages": 18793, 2024-06-10T03:04:31.617 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_messages": 21298, 2024-06-10T03:04:31.617 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_bytes": 7506199, 2024-06-10T03:04:31.617 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_bytes": 8548634, 2024-06-10T03:04:31.617 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_created_connections": 99, 2024-06-10T03:04:31.617 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_active_connections": 5, 2024-06-10T03:04:31.617 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_total_time": 1.228521787, 2024-06-10T03:04:31.617 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_send_time": 0.480899833, 2024-06-10T03:04:31.617 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_recv_time": 0.865119998, 2024-06-10T03:04:31.617 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_fast_dispatch_time": 0.081653999, 2024-06-10T03:04:31.617 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_messages_queue_lat": { 2024-06-10T03:04:31.617 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 21298, 2024-06-10T03:04:31.617 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.588414168, 2024-06-10T03:04:31.617 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000027627 2024-06-10T03:04:31.618 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:31.618 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_handle_ack_lat": { 2024-06-10T03:04:31.618 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 35287, 2024-06-10T03:04:31.618 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.005667722, 2024-06-10T03:04:31.618 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000160 2024-06-10T03:04:31.618 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:31.618 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_encrypted_bytes": 6756400, 2024-06-10T03:04:31.618 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_encrypted_bytes": 6567872 2024-06-10T03:04:31.618 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:31.618 INFO:teuthology.orchestra.run.smithi188.stdout: "cct": { 2024-06-10T03:04:31.618 INFO:teuthology.orchestra.run.smithi188.stdout: "total_workers": 1, 2024-06-10T03:04:31.618 INFO:teuthology.orchestra.run.smithi188.stdout: "unhealthy_workers": 0 2024-06-10T03:04:31.619 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:31.619 INFO:teuthology.orchestra.run.smithi188.stdout: "finisher-MDSRank": { 2024-06-10T03:04:31.619 INFO:teuthology.orchestra.run.smithi188.stdout: "queue_len": 0, 2024-06-10T03:04:31.619 INFO:teuthology.orchestra.run.smithi188.stdout: "complete_latency": { 2024-06-10T03:04:31.619 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 1365, 2024-06-10T03:04:31.619 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 1.811479927, 2024-06-10T03:04:31.619 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.001327091 2024-06-10T03:04:31.619 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-06-10T03:04:31.619 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:31.619 INFO:teuthology.orchestra.run.smithi188.stdout: "finisher-PurgeQueue": { 2024-06-10T03:04:31.619 INFO:teuthology.orchestra.run.smithi188.stdout: "queue_len": 0, 2024-06-10T03:04:31.619 INFO:teuthology.orchestra.run.smithi188.stdout: "complete_latency": { 2024-06-10T03:04:31.619 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 1326, 2024-06-10T03:04:31.619 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.140782229, 2024-06-10T03:04:31.620 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000106170 2024-06-10T03:04:31.620 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-06-10T03:04:31.620 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:31.620 INFO:teuthology.orchestra.run.smithi188.stdout: "mds": { 2024-06-10T03:04:31.620 INFO:teuthology.orchestra.run.smithi188.stdout: "request": 12577, 2024-06-10T03:04:31.620 INFO:teuthology.orchestra.run.smithi188.stdout: "reply": 2819, 2024-06-10T03:04:31.620 INFO:teuthology.orchestra.run.smithi188.stdout: "reply_latency": { 2024-06-10T03:04:31.620 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 2819, 2024-06-10T03:04:31.620 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 78.546604189, 2024-06-10T03:04:31.620 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.027863286 2024-06-10T03:04:31.620 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:31.620 INFO:teuthology.orchestra.run.smithi188.stdout: "slow_reply": 0, 2024-06-10T03:04:31.620 INFO:teuthology.orchestra.run.smithi188.stdout: "forward": 9758, 2024-06-10T03:04:31.621 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_fetch_complete": 0, 2024-06-10T03:04:31.621 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_fetch_keys": 0, 2024-06-10T03:04:31.621 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_commit": 30, 2024-06-10T03:04:31.621 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_split": 2, 2024-06-10T03:04:31.621 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_merge": 0, 2024-06-10T03:04:31.621 INFO:teuthology.orchestra.run.smithi188.stdout: "inodes": 38, 2024-06-10T03:04:31.621 INFO:teuthology.orchestra.run.smithi188.stdout: "inodes_top": 9, 2024-06-10T03:04:31.621 INFO:teuthology.orchestra.run.smithi188.stdout: "inodes_bottom": 11, 2024-06-10T03:04:31.621 INFO:teuthology.orchestra.run.smithi188.stdout: "inodes_pin_tail": 18, 2024-06-10T03:04:31.621 INFO:teuthology.orchestra.run.smithi188.stdout: "inodes_pinned": 25, 2024-06-10T03:04:31.621 INFO:teuthology.orchestra.run.smithi188.stdout: "inodes_expired": 282, 2024-06-10T03:04:31.621 INFO:teuthology.orchestra.run.smithi188.stdout: "inodes_with_caps": 2, 2024-06-10T03:04:31.621 INFO:teuthology.orchestra.run.smithi188.stdout: "caps": 2, 2024-06-10T03:04:31.621 INFO:teuthology.orchestra.run.smithi188.stdout: "subtrees": 3, 2024-06-10T03:04:31.622 INFO:teuthology.orchestra.run.smithi188.stdout: "traverse": 16414, 2024-06-10T03:04:31.622 INFO:teuthology.orchestra.run.smithi188.stdout: "traverse_hit": 5415, 2024-06-10T03:04:31.622 INFO:teuthology.orchestra.run.smithi188.stdout: "traverse_forward": 9758, 2024-06-10T03:04:31.622 INFO:teuthology.orchestra.run.smithi188.stdout: "traverse_discover": 142, 2024-06-10T03:04:31.622 INFO:teuthology.orchestra.run.smithi188.stdout: "traverse_dir_fetch": 0, 2024-06-10T03:04:31.622 INFO:teuthology.orchestra.run.smithi188.stdout: "traverse_remote_ino": 0, 2024-06-10T03:04:31.622 INFO:teuthology.orchestra.run.smithi188.stdout: "traverse_lock": 34, 2024-06-10T03:04:31.622 INFO:teuthology.orchestra.run.smithi188.stdout: "load_cent": 0, 2024-06-10T03:04:31.622 INFO:teuthology.orchestra.run.smithi188.stdout: "q": 0, 2024-06-10T03:04:31.622 INFO:teuthology.orchestra.run.smithi188.stdout: "exported": 17, 2024-06-10T03:04:31.622 INFO:teuthology.orchestra.run.smithi188.stdout: "exported_inodes": 20, 2024-06-10T03:04:31.622 INFO:teuthology.orchestra.run.smithi188.stdout: "imported": 3, 2024-06-10T03:04:31.622 INFO:teuthology.orchestra.run.smithi188.stdout: "imported_inodes": 209, 2024-06-10T03:04:31.623 INFO:teuthology.orchestra.run.smithi188.stdout: "openino_dir_fetch": 0, 2024-06-10T03:04:31.623 INFO:teuthology.orchestra.run.smithi188.stdout: "openino_backtrace_fetch": 813, 2024-06-10T03:04:31.623 INFO:teuthology.orchestra.run.smithi188.stdout: "openino_peer_discover": 0, 2024-06-10T03:04:31.623 INFO:teuthology.orchestra.run.smithi188.stdout: "root_rfiles": 4, 2024-06-10T03:04:31.623 INFO:teuthology.orchestra.run.smithi188.stdout: "root_rbytes": 234, 2024-06-10T03:04:31.623 INFO:teuthology.orchestra.run.smithi188.stdout: "root_rsnaps": 0, 2024-06-10T03:04:31.623 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_backtrace_fetch": 813, 2024-06-10T03:04:31.623 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_set_tag": 0, 2024-06-10T03:04:31.623 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_backtrace_repaired": 0, 2024-06-10T03:04:31.623 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_inotable_repaired": 0, 2024-06-10T03:04:31.623 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_dir_inodes": 92, 2024-06-10T03:04:31.623 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_dir_base_inodes": 8, 2024-06-10T03:04:31.623 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_dirfrag_rstats": 84, 2024-06-10T03:04:31.624 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_file_inodes": 721, 2024-06-10T03:04:31.624 INFO:teuthology.orchestra.run.smithi188.stdout: "handle_inode_file_caps": 12, 2024-06-10T03:04:31.624 INFO:teuthology.orchestra.run.smithi188.stdout: "ceph_cap_op_revoke": 913, 2024-06-10T03:04:31.624 INFO:teuthology.orchestra.run.smithi188.stdout: "ceph_cap_op_grant": 1936, 2024-06-10T03:04:31.624 INFO:teuthology.orchestra.run.smithi188.stdout: "ceph_cap_op_trunc": 5, 2024-06-10T03:04:31.624 INFO:teuthology.orchestra.run.smithi188.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-06-10T03:04:31.624 INFO:teuthology.orchestra.run.smithi188.stdout: "ceph_cap_op_flush_ack": 0, 2024-06-10T03:04:31.624 INFO:teuthology.orchestra.run.smithi188.stdout: "handle_client_caps": 1682, 2024-06-10T03:04:31.624 INFO:teuthology.orchestra.run.smithi188.stdout: "handle_client_caps_dirty": 536, 2024-06-10T03:04:31.624 INFO:teuthology.orchestra.run.smithi188.stdout: "handle_client_cap_release": 87, 2024-06-10T03:04:31.624 INFO:teuthology.orchestra.run.smithi188.stdout: "process_request_cap_release": 1738 2024-06-10T03:04:31.624 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:31.624 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_cache": { 2024-06-10T03:04:31.624 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_update": 167, 2024-06-10T03:04:31.625 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_update_receipt": 2603, 2024-06-10T03:04:31.625 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_try_discover": 1516, 2024-06-10T03:04:31.625 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_send_discover": 1848, 2024-06-10T03:04:31.625 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_handle_discover": 201, 2024-06-10T03:04:31.625 INFO:teuthology.orchestra.run.smithi188.stdout: "num_strays": 2, 2024-06-10T03:04:31.625 INFO:teuthology.orchestra.run.smithi188.stdout: "num_strays_delayed": 0, 2024-06-10T03:04:31.625 INFO:teuthology.orchestra.run.smithi188.stdout: "num_strays_enqueuing": 0, 2024-06-10T03:04:31.625 INFO:teuthology.orchestra.run.smithi188.stdout: "strays_created": 671, 2024-06-10T03:04:31.625 INFO:teuthology.orchestra.run.smithi188.stdout: "strays_enqueued": 669, 2024-06-10T03:04:31.625 INFO:teuthology.orchestra.run.smithi188.stdout: "strays_reintegrated": 0, 2024-06-10T03:04:31.625 INFO:teuthology.orchestra.run.smithi188.stdout: "strays_migrated": 0, 2024-06-10T03:04:31.625 INFO:teuthology.orchestra.run.smithi188.stdout: "num_recovering_processing": 0, 2024-06-10T03:04:31.625 INFO:teuthology.orchestra.run.smithi188.stdout: "num_recovering_enqueued": 0, 2024-06-10T03:04:31.626 INFO:teuthology.orchestra.run.smithi188.stdout: "num_recovering_prioritized": 0, 2024-06-10T03:04:31.626 INFO:teuthology.orchestra.run.smithi188.stdout: "recovery_started": 0, 2024-06-10T03:04:31.626 INFO:teuthology.orchestra.run.smithi188.stdout: "recovery_completed": 0, 2024-06-10T03:04:31.626 INFO:teuthology.orchestra.run.smithi188.stdout: "ireq_quiesce_path": 6, 2024-06-10T03:04:31.626 INFO:teuthology.orchestra.run.smithi188.stdout: "ireq_quiesce_inode": 1047, 2024-06-10T03:04:31.626 INFO:teuthology.orchestra.run.smithi188.stdout: "ireq_enqueue_scrub": 8, 2024-06-10T03:04:31.626 INFO:teuthology.orchestra.run.smithi188.stdout: "ireq_exportdir": 17, 2024-06-10T03:04:31.626 INFO:teuthology.orchestra.run.smithi188.stdout: "ireq_flush": 0, 2024-06-10T03:04:31.626 INFO:teuthology.orchestra.run.smithi188.stdout: "ireq_fragmentdir": 2, 2024-06-10T03:04:31.626 INFO:teuthology.orchestra.run.smithi188.stdout: "ireq_fragstats": 0, 2024-06-10T03:04:31.626 INFO:teuthology.orchestra.run.smithi188.stdout: "ireq_inodestats": 0 2024-06-10T03:04:31.626 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:31.626 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_log": { 2024-06-10T03:04:31.627 INFO:teuthology.orchestra.run.smithi188.stdout: "evlrg": 0, 2024-06-10T03:04:31.627 INFO:teuthology.orchestra.run.smithi188.stdout: "evadd": 3331, 2024-06-10T03:04:31.627 INFO:teuthology.orchestra.run.smithi188.stdout: "evex": 0, 2024-06-10T03:04:31.627 INFO:teuthology.orchestra.run.smithi188.stdout: "evtrm": 0, 2024-06-10T03:04:31.627 INFO:teuthology.orchestra.run.smithi188.stdout: "ev": 3331, 2024-06-10T03:04:31.627 INFO:teuthology.orchestra.run.smithi188.stdout: "evexg": 0, 2024-06-10T03:04:31.627 INFO:teuthology.orchestra.run.smithi188.stdout: "evexd": 0, 2024-06-10T03:04:31.627 INFO:teuthology.orchestra.run.smithi188.stdout: "segadd": 7, 2024-06-10T03:04:31.627 INFO:teuthology.orchestra.run.smithi188.stdout: "segex": 0, 2024-06-10T03:04:31.627 INFO:teuthology.orchestra.run.smithi188.stdout: "segtrm": 0, 2024-06-10T03:04:31.627 INFO:teuthology.orchestra.run.smithi188.stdout: "seg": 7, 2024-06-10T03:04:31.627 INFO:teuthology.orchestra.run.smithi188.stdout: "segmjr": 2, 2024-06-10T03:04:31.627 INFO:teuthology.orchestra.run.smithi188.stdout: "segexg": 0, 2024-06-10T03:04:31.627 INFO:teuthology.orchestra.run.smithi188.stdout: "segexd": 0, 2024-06-10T03:04:31.628 INFO:teuthology.orchestra.run.smithi188.stdout: "expos": 4194304, 2024-06-10T03:04:31.628 INFO:teuthology.orchestra.run.smithi188.stdout: "wrpos": 10175017, 2024-06-10T03:04:31.628 INFO:teuthology.orchestra.run.smithi188.stdout: "rdpos": 0, 2024-06-10T03:04:31.628 INFO:teuthology.orchestra.run.smithi188.stdout: "jlat": { 2024-06-10T03:04:31.628 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 321, 2024-06-10T03:04:31.628 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.933915408, 2024-06-10T03:04:31.628 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.002909393 2024-06-10T03:04:31.628 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:31.628 INFO:teuthology.orchestra.run.smithi188.stdout: "replayed": 0 2024-06-10T03:04:31.628 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:31.628 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_mem": { 2024-06-10T03:04:31.628 INFO:teuthology.orchestra.run.smithi188.stdout: "ino": 36, 2024-06-10T03:04:31.628 INFO:teuthology.orchestra.run.smithi188.stdout: "ino+": 861, 2024-06-10T03:04:31.628 INFO:teuthology.orchestra.run.smithi188.stdout: "ino-": 825, 2024-06-10T03:04:31.629 INFO:teuthology.orchestra.run.smithi188.stdout: "dir": 31, 2024-06-10T03:04:31.629 INFO:teuthology.orchestra.run.smithi188.stdout: "dir+": 1271, 2024-06-10T03:04:31.629 INFO:teuthology.orchestra.run.smithi188.stdout: "dir-": 1240, 2024-06-10T03:04:31.629 INFO:teuthology.orchestra.run.smithi188.stdout: "dn": 38, 2024-06-10T03:04:31.629 INFO:teuthology.orchestra.run.smithi188.stdout: "dn+": 1670, 2024-06-10T03:04:31.629 INFO:teuthology.orchestra.run.smithi188.stdout: "dn-": 1632, 2024-06-10T03:04:31.629 INFO:teuthology.orchestra.run.smithi188.stdout: "cap": 2, 2024-06-10T03:04:31.629 INFO:teuthology.orchestra.run.smithi188.stdout: "cap+": 905, 2024-06-10T03:04:31.629 INFO:teuthology.orchestra.run.smithi188.stdout: "cap-": 903, 2024-06-10T03:04:31.629 INFO:teuthology.orchestra.run.smithi188.stdout: "rss": 65992, 2024-06-10T03:04:31.629 INFO:teuthology.orchestra.run.smithi188.stdout: "heap": 231804 2024-06-10T03:04:31.629 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:31.629 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_server": { 2024-06-10T03:04:31.629 INFO:teuthology.orchestra.run.smithi188.stdout: "dispatch_client_request": 13626, 2024-06-10T03:04:31.630 INFO:teuthology.orchestra.run.smithi188.stdout: "dispatch_server_request": 75, 2024-06-10T03:04:31.630 INFO:teuthology.orchestra.run.smithi188.stdout: "handle_client_request": 12577, 2024-06-10T03:04:31.630 INFO:teuthology.orchestra.run.smithi188.stdout: "handle_client_session": 25, 2024-06-10T03:04:31.630 INFO:teuthology.orchestra.run.smithi188.stdout: "handle_peer_request": 870, 2024-06-10T03:04:31.630 INFO:teuthology.orchestra.run.smithi188.stdout: "req_create_latency": { 2024-06-10T03:04:31.630 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 590, 2024-06-10T03:04:31.630 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 63.036851473, 2024-06-10T03:04:31.630 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.106842121 2024-06-10T03:04:31.630 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:31.630 INFO:teuthology.orchestra.run.smithi188.stdout: "req_getattr_latency": { 2024-06-10T03:04:31.630 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 104, 2024-06-10T03:04:31.630 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.197119075, 2024-06-10T03:04:31.630 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.001895375 2024-06-10T03:04:31.631 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:31.631 INFO:teuthology.orchestra.run.smithi188.stdout: "req_getfilelock_latency": { 2024-06-10T03:04:31.631 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-06-10T03:04:31.631 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-06-10T03:04:31.631 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-06-10T03:04:31.631 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:31.631 INFO:teuthology.orchestra.run.smithi188.stdout: "req_link_latency": { 2024-06-10T03:04:31.631 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-06-10T03:04:31.631 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-06-10T03:04:31.631 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-06-10T03:04:31.631 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:31.631 INFO:teuthology.orchestra.run.smithi188.stdout: "req_lookup_latency": { 2024-06-10T03:04:31.631 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 920, 2024-06-10T03:04:31.631 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 2.408709016, 2024-06-10T03:04:31.632 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.002618161 2024-06-10T03:04:31.632 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:31.632 INFO:teuthology.orchestra.run.smithi188.stdout: "req_lookuphash_latency": { 2024-06-10T03:04:31.632 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-06-10T03:04:31.632 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-06-10T03:04:31.632 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-06-10T03:04:31.632 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:31.632 INFO:teuthology.orchestra.run.smithi188.stdout: "req_lookupino_latency": { 2024-06-10T03:04:31.632 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 1, 2024-06-10T03:04:31.632 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000381605, 2024-06-10T03:04:31.632 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000381605 2024-06-10T03:04:31.632 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:31.632 INFO:teuthology.orchestra.run.smithi188.stdout: "req_lookupname_latency": { 2024-06-10T03:04:31.633 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-06-10T03:04:31.633 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-06-10T03:04:31.633 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-06-10T03:04:31.633 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:31.633 INFO:teuthology.orchestra.run.smithi188.stdout: "req_lookupparent_latency": { 2024-06-10T03:04:31.633 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-06-10T03:04:31.633 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-06-10T03:04:31.634 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-06-10T03:04:31.634 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:31.634 INFO:teuthology.orchestra.run.smithi188.stdout: "req_lookupsnap_latency": { 2024-06-10T03:04:31.634 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-06-10T03:04:31.634 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-06-10T03:04:31.634 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-06-10T03:04:31.634 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:31.634 INFO:teuthology.orchestra.run.smithi188.stdout: "req_lssnap_latency": { 2024-06-10T03:04:31.635 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-06-10T03:04:31.635 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-06-10T03:04:31.635 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-06-10T03:04:31.635 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:31.635 INFO:teuthology.orchestra.run.smithi188.stdout: "req_mkdir_latency": { 2024-06-10T03:04:31.635 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 16, 2024-06-10T03:04:31.635 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 9.921470907, 2024-06-10T03:04:31.635 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.620091931 2024-06-10T03:04:31.635 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:31.636 INFO:teuthology.orchestra.run.smithi188.stdout: "req_mknod_latency": { 2024-06-10T03:04:31.636 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-06-10T03:04:31.636 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-06-10T03:04:31.636 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-06-10T03:04:31.636 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:31.636 INFO:teuthology.orchestra.run.smithi188.stdout: "req_mksnap_latency": { 2024-06-10T03:04:31.636 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-06-10T03:04:31.636 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-06-10T03:04:31.636 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-06-10T03:04:31.636 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:31.636 INFO:teuthology.orchestra.run.smithi188.stdout: "req_open_latency": { 2024-06-10T03:04:31.636 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 228, 2024-06-10T03:04:31.637 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.105587516, 2024-06-10T03:04:31.637 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000463103 2024-06-10T03:04:31.637 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:31.637 INFO:teuthology.orchestra.run.smithi188.stdout: "req_readdir_latency": { 2024-06-10T03:04:31.637 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 84, 2024-06-10T03:04:31.637 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.555061478, 2024-06-10T03:04:31.637 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.006607874 2024-06-10T03:04:31.637 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:31.637 INFO:teuthology.orchestra.run.smithi188.stdout: "req_rename_latency": { 2024-06-10T03:04:31.637 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 232, 2024-06-10T03:04:31.637 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 1.237414274, 2024-06-10T03:04:31.637 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.005333682 2024-06-10T03:04:31.637 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:31.637 INFO:teuthology.orchestra.run.smithi188.stdout: "req_renamesnap_latency": { 2024-06-10T03:04:31.638 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-06-10T03:04:31.638 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-06-10T03:04:31.638 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-06-10T03:04:31.638 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:31.638 INFO:teuthology.orchestra.run.smithi188.stdout: "req_snapdiff_latency": { 2024-06-10T03:04:31.638 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-06-10T03:04:31.638 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-06-10T03:04:31.638 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-06-10T03:04:31.638 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:31.638 INFO:teuthology.orchestra.run.smithi188.stdout: "req_rmdir_latency": { 2024-06-10T03:04:31.638 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 10, 2024-06-10T03:04:31.638 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.119667328, 2024-06-10T03:04:31.639 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.011966732 2024-06-10T03:04:31.639 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:31.639 INFO:teuthology.orchestra.run.smithi188.stdout: "req_rmsnap_latency": { 2024-06-10T03:04:31.639 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-06-10T03:04:31.639 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-06-10T03:04:31.639 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-06-10T03:04:31.639 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:31.639 INFO:teuthology.orchestra.run.smithi188.stdout: "req_rmxattr_latency": { 2024-06-10T03:04:31.639 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-06-10T03:04:31.639 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-06-10T03:04:31.639 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-06-10T03:04:31.639 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:31.639 INFO:teuthology.orchestra.run.smithi188.stdout: "req_setattr_latency": { 2024-06-10T03:04:31.640 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 7, 2024-06-10T03:04:31.640 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.029831434, 2024-06-10T03:04:31.640 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.004261633 2024-06-10T03:04:31.640 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:31.640 INFO:teuthology.orchestra.run.smithi188.stdout: "req_setdirlayout_latency": { 2024-06-10T03:04:31.640 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-06-10T03:04:31.640 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-06-10T03:04:31.640 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-06-10T03:04:31.640 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:31.640 INFO:teuthology.orchestra.run.smithi188.stdout: "req_setfilelock_latency": { 2024-06-10T03:04:31.640 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-06-10T03:04:31.640 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-06-10T03:04:31.640 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-06-10T03:04:31.640 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:31.641 INFO:teuthology.orchestra.run.smithi188.stdout: "req_setlayout_latency": { 2024-06-10T03:04:31.641 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-06-10T03:04:31.641 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-06-10T03:04:31.641 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-06-10T03:04:31.641 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:31.641 INFO:teuthology.orchestra.run.smithi188.stdout: "req_setxattr_latency": { 2024-06-10T03:04:31.641 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 9, 2024-06-10T03:04:31.641 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.025330891, 2024-06-10T03:04:31.641 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.002814543 2024-06-10T03:04:31.641 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:31.641 INFO:teuthology.orchestra.run.smithi188.stdout: "req_symlink_latency": { 2024-06-10T03:04:31.641 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 3, 2024-06-10T03:04:31.641 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.001892106, 2024-06-10T03:04:31.641 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000630702 2024-06-10T03:04:31.642 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:31.642 INFO:teuthology.orchestra.run.smithi188.stdout: "req_unlink_latency": { 2024-06-10T03:04:31.642 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 615, 2024-06-10T03:04:31.642 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.907287086, 2024-06-10T03:04:31.642 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.001475263 2024-06-10T03:04:31.642 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:31.642 INFO:teuthology.orchestra.run.smithi188.stdout: "cap_revoke_eviction": 0, 2024-06-10T03:04:31.642 INFO:teuthology.orchestra.run.smithi188.stdout: "cap_acquisition_throttle": 0, 2024-06-10T03:04:31.642 INFO:teuthology.orchestra.run.smithi188.stdout: "req_getvxattr_latency": { 2024-06-10T03:04:31.642 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-06-10T03:04:31.642 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-06-10T03:04:31.642 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-06-10T03:04:31.642 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-06-10T03:04:31.643 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:31.643 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_sessions": { 2024-06-10T03:04:31.643 INFO:teuthology.orchestra.run.smithi188.stdout: "session_count": 1, 2024-06-10T03:04:31.643 INFO:teuthology.orchestra.run.smithi188.stdout: "session_add": 4, 2024-06-10T03:04:31.643 INFO:teuthology.orchestra.run.smithi188.stdout: "session_remove": 3, 2024-06-10T03:04:31.643 INFO:teuthology.orchestra.run.smithi188.stdout: "sessions_open": 1, 2024-06-10T03:04:31.643 INFO:teuthology.orchestra.run.smithi188.stdout: "sessions_stale": 0, 2024-06-10T03:04:31.643 INFO:teuthology.orchestra.run.smithi188.stdout: "total_load": 598, 2024-06-10T03:04:31.643 INFO:teuthology.orchestra.run.smithi188.stdout: "average_load": 598, 2024-06-10T03:04:31.643 INFO:teuthology.orchestra.run.smithi188.stdout: "avg_session_uptime": 624, 2024-06-10T03:04:31.643 INFO:teuthology.orchestra.run.smithi188.stdout: "mdthresh_evicted": 0 2024-06-10T03:04:31.643 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:31.643 INFO:teuthology.orchestra.run.smithi188.stdout: "mempool": { 2024-06-10T03:04:31.644 INFO:teuthology.orchestra.run.smithi188.stdout: "bloom_filter_bytes": 0, 2024-06-10T03:04:31.644 INFO:teuthology.orchestra.run.smithi188.stdout: "bloom_filter_items": 0, 2024-06-10T03:04:31.644 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_alloc_bytes": 0, 2024-06-10T03:04:31.644 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_alloc_items": 0, 2024-06-10T03:04:31.644 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_data_bytes": 0, 2024-06-10T03:04:31.644 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_data_items": 0, 2024-06-10T03:04:31.644 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-10T03:04:31.644 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_onode_items": 0, 2024-06-10T03:04:31.644 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-10T03:04:31.644 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_meta_items": 0, 2024-06-10T03:04:31.644 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_other_bytes": 0, 2024-06-10T03:04:31.644 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_other_items": 0, 2024-06-10T03:04:31.644 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-10T03:04:31.644 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_buffer_items": 0, 2024-06-10T03:04:31.645 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_extent_bytes": 0, 2024-06-10T03:04:31.645 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_extent_items": 0, 2024-06-10T03:04:31.645 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_blob_bytes": 0, 2024-06-10T03:04:31.645 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_blob_items": 0, 2024-06-10T03:04:31.645 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-10T03:04:31.645 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_shared_blob_items": 0, 2024-06-10T03:04:31.645 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-10T03:04:31.645 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_inline_bl_items": 0, 2024-06-10T03:04:31.645 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_fsck_bytes": 0, 2024-06-10T03:04:31.645 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_fsck_items": 0, 2024-06-10T03:04:31.645 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_txc_bytes": 0, 2024-06-10T03:04:31.645 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_txc_items": 0, 2024-06-10T03:04:31.645 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-10T03:04:31.645 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_writing_deferred_items": 0, 2024-06-10T03:04:31.646 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_writing_bytes": 0, 2024-06-10T03:04:31.646 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_writing_items": 0, 2024-06-10T03:04:31.646 INFO:teuthology.orchestra.run.smithi188.stdout: "bluefs_bytes": 0, 2024-06-10T03:04:31.646 INFO:teuthology.orchestra.run.smithi188.stdout: "bluefs_items": 0, 2024-06-10T03:04:31.646 INFO:teuthology.orchestra.run.smithi188.stdout: "bluefs_file_reader_bytes": 0, 2024-06-10T03:04:31.646 INFO:teuthology.orchestra.run.smithi188.stdout: "bluefs_file_reader_items": 0, 2024-06-10T03:04:31.646 INFO:teuthology.orchestra.run.smithi188.stdout: "bluefs_file_writer_bytes": 0, 2024-06-10T03:04:31.646 INFO:teuthology.orchestra.run.smithi188.stdout: "bluefs_file_writer_items": 0, 2024-06-10T03:04:31.646 INFO:teuthology.orchestra.run.smithi188.stdout: "buffer_anon_bytes": 311601, 2024-06-10T03:04:31.646 INFO:teuthology.orchestra.run.smithi188.stdout: "buffer_anon_items": 109, 2024-06-10T03:04:31.646 INFO:teuthology.orchestra.run.smithi188.stdout: "buffer_meta_bytes": 88, 2024-06-10T03:04:31.646 INFO:teuthology.orchestra.run.smithi188.stdout: "buffer_meta_items": 1, 2024-06-10T03:04:31.647 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_bytes": 0, 2024-06-10T03:04:31.647 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_items": 0, 2024-06-10T03:04:31.647 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_mapbl_bytes": 0, 2024-06-10T03:04:31.647 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_mapbl_items": 0, 2024-06-10T03:04:31.647 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_pglog_bytes": 0, 2024-06-10T03:04:31.647 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_pglog_items": 0, 2024-06-10T03:04:31.647 INFO:teuthology.orchestra.run.smithi188.stdout: "osdmap_bytes": 8984, 2024-06-10T03:04:31.647 INFO:teuthology.orchestra.run.smithi188.stdout: "osdmap_items": 172, 2024-06-10T03:04:31.647 INFO:teuthology.orchestra.run.smithi188.stdout: "osdmap_mapping_bytes": 0, 2024-06-10T03:04:31.647 INFO:teuthology.orchestra.run.smithi188.stdout: "osdmap_mapping_items": 0, 2024-06-10T03:04:31.647 INFO:teuthology.orchestra.run.smithi188.stdout: "pgmap_bytes": 0, 2024-06-10T03:04:31.647 INFO:teuthology.orchestra.run.smithi188.stdout: "pgmap_items": 0, 2024-06-10T03:04:31.647 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_co_bytes": 153688, 2024-06-10T03:04:31.648 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_co_items": 1141, 2024-06-10T03:04:31.648 INFO:teuthology.orchestra.run.smithi188.stdout: "unittest_1_bytes": 0, 2024-06-10T03:04:31.648 INFO:teuthology.orchestra.run.smithi188.stdout: "unittest_1_items": 0, 2024-06-10T03:04:31.648 INFO:teuthology.orchestra.run.smithi188.stdout: "unittest_2_bytes": 0, 2024-06-10T03:04:31.648 INFO:teuthology.orchestra.run.smithi188.stdout: "unittest_2_items": 0 2024-06-10T03:04:31.648 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:31.648 INFO:teuthology.orchestra.run.smithi188.stdout: "objecter": { 2024-06-10T03:04:31.648 INFO:teuthology.orchestra.run.smithi188.stdout: "op_active": 0, 2024-06-10T03:04:31.648 INFO:teuthology.orchestra.run.smithi188.stdout: "op_laggy": 0, 2024-06-10T03:04:31.648 INFO:teuthology.orchestra.run.smithi188.stdout: "op_send": 3253, 2024-06-10T03:04:31.648 INFO:teuthology.orchestra.run.smithi188.stdout: "op_send_bytes": 6469372, 2024-06-10T03:04:31.648 INFO:teuthology.orchestra.run.smithi188.stdout: "op_resend": 37, 2024-06-10T03:04:31.648 INFO:teuthology.orchestra.run.smithi188.stdout: "op_reply": 3253, 2024-06-10T03:04:31.649 INFO:teuthology.orchestra.run.smithi188.stdout: "op_latency": { 2024-06-10T03:04:31.649 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 3253, 2024-06-10T03:04:31.649 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 30.395459519, 2024-06-10T03:04:31.649 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.009343824 2024-06-10T03:04:31.649 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:31.649 INFO:teuthology.orchestra.run.smithi188.stdout: "op_inflight": 0, 2024-06-10T03:04:31.649 INFO:teuthology.orchestra.run.smithi188.stdout: "oplen_avg": { 2024-06-10T03:04:31.649 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 3253, 2024-06-10T03:04:31.649 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 3403 2024-06-10T03:04:31.649 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:31.649 INFO:teuthology.orchestra.run.smithi188.stdout: "op": 3253, 2024-06-10T03:04:31.649 INFO:teuthology.orchestra.run.smithi188.stdout: "op_r": 861, 2024-06-10T03:04:31.649 INFO:teuthology.orchestra.run.smithi188.stdout: "op_w": 2392, 2024-06-10T03:04:31.649 INFO:teuthology.orchestra.run.smithi188.stdout: "op_rmw": 0, 2024-06-10T03:04:31.650 INFO:teuthology.orchestra.run.smithi188.stdout: "op_pg": 0, 2024-06-10T03:04:31.650 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_stat": 0, 2024-06-10T03:04:31.650 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_create": 11, 2024-06-10T03:04:31.650 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_read": 40, 2024-06-10T03:04:31.650 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_write": 353, 2024-06-10T03:04:31.650 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_writefull": 60, 2024-06-10T03:04:31.650 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_writesame": 0, 2024-06-10T03:04:31.650 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_append": 0, 2024-06-10T03:04:31.650 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_zero": 0, 2024-06-10T03:04:31.650 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_truncate": 2, 2024-06-10T03:04:31.650 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_delete": 1840, 2024-06-10T03:04:31.650 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_mapext": 0, 2024-06-10T03:04:31.651 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_sparse_read": 0, 2024-06-10T03:04:31.651 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_clonerange": 0, 2024-06-10T03:04:31.651 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_getxattr": 821, 2024-06-10T03:04:31.651 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_setxattr": 22, 2024-06-10T03:04:31.651 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_cmpxattr": 0, 2024-06-10T03:04:31.651 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_rmxattr": 0, 2024-06-10T03:04:31.651 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_resetxattrs": 0, 2024-06-10T03:04:31.651 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_call": 0, 2024-06-10T03:04:31.651 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_watch": 0, 2024-06-10T03:04:31.651 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_notify": 0, 2024-06-10T03:04:31.651 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_src_cmpxattr": 0, 2024-06-10T03:04:31.651 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_pgls": 0, 2024-06-10T03:04:31.651 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_pgls_filter": 0, 2024-06-10T03:04:31.651 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_other": 5, 2024-06-10T03:04:31.652 INFO:teuthology.orchestra.run.smithi188.stdout: "linger_active": 0, 2024-06-10T03:04:31.652 INFO:teuthology.orchestra.run.smithi188.stdout: "linger_send": 0, 2024-06-10T03:04:31.652 INFO:teuthology.orchestra.run.smithi188.stdout: "linger_resend": 0, 2024-06-10T03:04:31.652 INFO:teuthology.orchestra.run.smithi188.stdout: "linger_ping": 0, 2024-06-10T03:04:31.652 INFO:teuthology.orchestra.run.smithi188.stdout: "poolop_active": 0, 2024-06-10T03:04:31.652 INFO:teuthology.orchestra.run.smithi188.stdout: "poolop_send": 0, 2024-06-10T03:04:31.652 INFO:teuthology.orchestra.run.smithi188.stdout: "poolop_resend": 0, 2024-06-10T03:04:31.652 INFO:teuthology.orchestra.run.smithi188.stdout: "poolstat_active": 0, 2024-06-10T03:04:31.652 INFO:teuthology.orchestra.run.smithi188.stdout: "poolstat_send": 0, 2024-06-10T03:04:31.652 INFO:teuthology.orchestra.run.smithi188.stdout: "poolstat_resend": 0, 2024-06-10T03:04:31.652 INFO:teuthology.orchestra.run.smithi188.stdout: "statfs_active": 0, 2024-06-10T03:04:31.652 INFO:teuthology.orchestra.run.smithi188.stdout: "statfs_send": 0, 2024-06-10T03:04:31.652 INFO:teuthology.orchestra.run.smithi188.stdout: "statfs_resend": 0, 2024-06-10T03:04:31.652 INFO:teuthology.orchestra.run.smithi188.stdout: "command_active": 0, 2024-06-10T03:04:31.652 INFO:teuthology.orchestra.run.smithi188.stdout: "command_send": 0, 2024-06-10T03:04:31.653 INFO:teuthology.orchestra.run.smithi188.stdout: "command_resend": 0, 2024-06-10T03:04:31.653 INFO:teuthology.orchestra.run.smithi188.stdout: "map_epoch": 100, 2024-06-10T03:04:31.653 INFO:teuthology.orchestra.run.smithi188.stdout: "map_full": 0, 2024-06-10T03:04:31.653 INFO:teuthology.orchestra.run.smithi188.stdout: "map_inc": 11, 2024-06-10T03:04:31.653 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_sessions": 12, 2024-06-10T03:04:31.653 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_session_open": 12, 2024-06-10T03:04:31.653 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_session_close": 0, 2024-06-10T03:04:31.653 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_laggy": 0, 2024-06-10T03:04:31.653 INFO:teuthology.orchestra.run.smithi188.stdout: "omap_wr": 203, 2024-06-10T03:04:31.653 INFO:teuthology.orchestra.run.smithi188.stdout: "omap_rd": 0, 2024-06-10T03:04:31.653 INFO:teuthology.orchestra.run.smithi188.stdout: "omap_del": 46 2024-06-10T03:04:31.653 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:31.653 INFO:teuthology.orchestra.run.smithi188.stdout: "oft": { 2024-06-10T03:04:31.654 INFO:teuthology.orchestra.run.smithi188.stdout: "omap_total_objs": 1, 2024-06-10T03:04:31.654 INFO:teuthology.orchestra.run.smithi188.stdout: "omap_total_kv_pairs": 0, 2024-06-10T03:04:31.654 INFO:teuthology.orchestra.run.smithi188.stdout: "omap_total_updates": 0, 2024-06-10T03:04:31.654 INFO:teuthology.orchestra.run.smithi188.stdout: "omap_total_removes": 0 2024-06-10T03:04:31.654 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:31.654 INFO:teuthology.orchestra.run.smithi188.stdout: "osd-slow-ops": { 2024-06-10T03:04:31.654 INFO:teuthology.orchestra.run.smithi188.stdout: "slow_ops_count": 1041 2024-06-10T03:04:31.654 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:31.654 INFO:teuthology.orchestra.run.smithi188.stdout: "purge_queue": { 2024-06-10T03:04:31.654 INFO:teuthology.orchestra.run.smithi188.stdout: "pq_executing_ops": 0, 2024-06-10T03:04:31.654 INFO:teuthology.orchestra.run.smithi188.stdout: "pq_executing_ops_high_water": 10, 2024-06-10T03:04:31.654 INFO:teuthology.orchestra.run.smithi188.stdout: "pq_executing": 0, 2024-06-10T03:04:31.654 INFO:teuthology.orchestra.run.smithi188.stdout: "pq_executing_high_water": 5, 2024-06-10T03:04:31.655 INFO:teuthology.orchestra.run.smithi188.stdout: "pq_executed_ops": 1327, 2024-06-10T03:04:31.655 INFO:teuthology.orchestra.run.smithi188.stdout: "pq_executed": 669, 2024-06-10T03:04:31.655 INFO:teuthology.orchestra.run.smithi188.stdout: "pq_item_in_journal": 0 2024-06-10T03:04:31.655 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:31.655 INFO:teuthology.orchestra.run.smithi188.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-10T03:04:31.655 INFO:teuthology.orchestra.run.smithi188.stdout: "val": 0, 2024-06-10T03:04:31.655 INFO:teuthology.orchestra.run.smithi188.stdout: "max": 104857600, 2024-06-10T03:04:31.655 INFO:teuthology.orchestra.run.smithi188.stdout: "get_started": 0, 2024-06-10T03:04:31.655 INFO:teuthology.orchestra.run.smithi188.stdout: "get": 33789, 2024-06-10T03:04:31.655 INFO:teuthology.orchestra.run.smithi188.stdout: "get_sum": 9888580, 2024-06-10T03:04:31.655 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_fail": 0, 2024-06-10T03:04:31.655 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_success": 33789, 2024-06-10T03:04:31.655 INFO:teuthology.orchestra.run.smithi188.stdout: "take": 0, 2024-06-10T03:04:31.655 INFO:teuthology.orchestra.run.smithi188.stdout: "take_sum": 0, 2024-06-10T03:04:31.656 INFO:teuthology.orchestra.run.smithi188.stdout: "put": 33789, 2024-06-10T03:04:31.656 INFO:teuthology.orchestra.run.smithi188.stdout: "put_sum": 9888580, 2024-06-10T03:04:31.656 INFO:teuthology.orchestra.run.smithi188.stdout: "wait": { 2024-06-10T03:04:31.656 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-06-10T03:04:31.656 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-06-10T03:04:31.656 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-06-10T03:04:31.656 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-06-10T03:04:31.656 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:31.656 INFO:teuthology.orchestra.run.smithi188.stdout: "throttle-objecter_bytes": { 2024-06-10T03:04:31.656 INFO:teuthology.orchestra.run.smithi188.stdout: "val": 0, 2024-06-10T03:04:31.656 INFO:teuthology.orchestra.run.smithi188.stdout: "max": 104857600, 2024-06-10T03:04:31.656 INFO:teuthology.orchestra.run.smithi188.stdout: "get_started": 0, 2024-06-10T03:04:31.656 INFO:teuthology.orchestra.run.smithi188.stdout: "get": 0, 2024-06-10T03:04:31.656 INFO:teuthology.orchestra.run.smithi188.stdout: "get_sum": 0, 2024-06-10T03:04:31.657 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_fail": 0, 2024-06-10T03:04:31.657 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_success": 0, 2024-06-10T03:04:31.657 INFO:teuthology.orchestra.run.smithi188.stdout: "take": 3253, 2024-06-10T03:04:31.657 INFO:teuthology.orchestra.run.smithi188.stdout: "take_sum": 6542205, 2024-06-10T03:04:31.657 INFO:teuthology.orchestra.run.smithi188.stdout: "put": 3253, 2024-06-10T03:04:31.657 INFO:teuthology.orchestra.run.smithi188.stdout: "put_sum": 6542205, 2024-06-10T03:04:31.657 INFO:teuthology.orchestra.run.smithi188.stdout: "wait": { 2024-06-10T03:04:31.657 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-06-10T03:04:31.657 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-06-10T03:04:31.657 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-06-10T03:04:31.657 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-06-10T03:04:31.657 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:31.657 INFO:teuthology.orchestra.run.smithi188.stdout: "throttle-objecter_ops": { 2024-06-10T03:04:31.657 INFO:teuthology.orchestra.run.smithi188.stdout: "val": 0, 2024-06-10T03:04:31.658 INFO:teuthology.orchestra.run.smithi188.stdout: "max": 1024, 2024-06-10T03:04:31.658 INFO:teuthology.orchestra.run.smithi188.stdout: "get_started": 0, 2024-06-10T03:04:31.658 INFO:teuthology.orchestra.run.smithi188.stdout: "get": 0, 2024-06-10T03:04:31.658 INFO:teuthology.orchestra.run.smithi188.stdout: "get_sum": 0, 2024-06-10T03:04:31.658 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_fail": 0, 2024-06-10T03:04:31.658 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_success": 0, 2024-06-10T03:04:31.658 INFO:teuthology.orchestra.run.smithi188.stdout: "take": 3253, 2024-06-10T03:04:31.658 INFO:teuthology.orchestra.run.smithi188.stdout: "take_sum": 3253, 2024-06-10T03:04:31.658 INFO:teuthology.orchestra.run.smithi188.stdout: "put": 3253, 2024-06-10T03:04:31.658 INFO:teuthology.orchestra.run.smithi188.stdout: "put_sum": 3253, 2024-06-10T03:04:31.658 INFO:teuthology.orchestra.run.smithi188.stdout: "wait": { 2024-06-10T03:04:31.658 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-06-10T03:04:31.658 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-06-10T03:04:31.659 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-06-10T03:04:31.659 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-06-10T03:04:31.659 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:31.659 INFO:teuthology.orchestra.run.smithi188.stdout: "throttle-write_buf_throttle": { 2024-06-10T03:04:31.659 INFO:teuthology.orchestra.run.smithi188.stdout: "val": 0, 2024-06-10T03:04:31.659 INFO:teuthology.orchestra.run.smithi188.stdout: "max": 3758096384, 2024-06-10T03:04:31.659 INFO:teuthology.orchestra.run.smithi188.stdout: "get_started": 0, 2024-06-10T03:04:31.659 INFO:teuthology.orchestra.run.smithi188.stdout: "get": 669, 2024-06-10T03:04:31.659 INFO:teuthology.orchestra.run.smithi188.stdout: "get_sum": 72833, 2024-06-10T03:04:31.659 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_fail": 0, 2024-06-10T03:04:31.659 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_success": 669, 2024-06-10T03:04:31.659 INFO:teuthology.orchestra.run.smithi188.stdout: "take": 0, 2024-06-10T03:04:31.659 INFO:teuthology.orchestra.run.smithi188.stdout: "take_sum": 0, 2024-06-10T03:04:31.659 INFO:teuthology.orchestra.run.smithi188.stdout: "put": 32, 2024-06-10T03:04:31.660 INFO:teuthology.orchestra.run.smithi188.stdout: "put_sum": 72833, 2024-06-10T03:04:31.660 INFO:teuthology.orchestra.run.smithi188.stdout: "wait": { 2024-06-10T03:04:31.660 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-06-10T03:04:31.660 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-06-10T03:04:31.660 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-06-10T03:04:31.660 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-06-10T03:04:31.660 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:31.660 INFO:teuthology.orchestra.run.smithi188.stdout: "throttle-write_buf_throttle-0x55de9a57a500": { 2024-06-10T03:04:31.660 INFO:teuthology.orchestra.run.smithi188.stdout: "val": 0, 2024-06-10T03:04:31.660 INFO:teuthology.orchestra.run.smithi188.stdout: "max": 3758096384, 2024-06-10T03:04:31.660 INFO:teuthology.orchestra.run.smithi188.stdout: "get_started": 0, 2024-06-10T03:04:31.660 INFO:teuthology.orchestra.run.smithi188.stdout: "get": 3331, 2024-06-10T03:04:31.660 INFO:teuthology.orchestra.run.smithi188.stdout: "get_sum": 5980713, 2024-06-10T03:04:31.661 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_fail": 0, 2024-06-10T03:04:31.661 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_success": 3331, 2024-06-10T03:04:31.661 INFO:teuthology.orchestra.run.smithi188.stdout: "take": 0, 2024-06-10T03:04:31.661 INFO:teuthology.orchestra.run.smithi188.stdout: "take_sum": 0, 2024-06-10T03:04:31.661 INFO:teuthology.orchestra.run.smithi188.stdout: "put": 321, 2024-06-10T03:04:31.661 INFO:teuthology.orchestra.run.smithi188.stdout: "put_sum": 5980713, 2024-06-10T03:04:31.661 INFO:teuthology.orchestra.run.smithi188.stdout: "wait": { 2024-06-10T03:04:31.661 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-06-10T03:04:31.661 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-06-10T03:04:31.661 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-06-10T03:04:31.661 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-06-10T03:04:31.661 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-06-10T03:04:31.661 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-06-10T03:04:32.427 INFO:tasks.check_counter:Daemon mds.i mds.exported=17 2024-06-10T03:04:32.428 INFO:tasks.check_counter:Daemon mds.i mds.imported=3 2024-06-10T03:04:32.428 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update=167 2024-06-10T03:04:32.428 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update_receipt=2603 2024-06-10T03:04:32.428 INFO:tasks.check_counter:Daemon mds.i mds.dir_split=2 2024-06-10T03:04:32.428 DEBUG:tasks.check_counter:Getting stats from l 2024-06-10T03:04:32.428 DEBUG:teuthology.orchestra.run.smithi188:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph daemon mds.l perf dump 2024-06-10T03:04:32.663 INFO:teuthology.orchestra.run.smithi188.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.c/config 2024-06-10T03:04:33.054 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:04:32 smithi188 ceph-mon[36599]: pgmap v744: 129 pgs: 129 active+clean; 228 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 8 op/s 2024-06-10T03:04:33.227 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:04:32 smithi052 ceph-mon[32362]: pgmap v744: 129 pgs: 129 active+clean; 228 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 8 op/s 2024-06-10T03:04:33.388 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:04:32 smithi120 ceph-mon[36634]: pgmap v744: 129 pgs: 129 active+clean; 228 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 8 op/s 2024-06-10T03:04:34.361 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-06-10T03:04:34.361 INFO:teuthology.orchestra.run.smithi188.stdout: "AsyncMessenger::Worker-0": { 2024-06-10T03:04:34.361 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_messages": 1536, 2024-06-10T03:04:34.361 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_messages": 1536, 2024-06-10T03:04:34.361 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_bytes": 4658374, 2024-06-10T03:04:34.361 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_bytes": 487634, 2024-06-10T03:04:34.361 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_created_connections": 6, 2024-06-10T03:04:34.361 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_active_connections": 2, 2024-06-10T03:04:34.361 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_total_time": 0.128359011, 2024-06-10T03:04:34.361 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_send_time": 0.037898635, 2024-06-10T03:04:34.362 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_recv_time": 0.035971389, 2024-06-10T03:04:34.362 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_fast_dispatch_time": 0.043096250, 2024-06-10T03:04:34.362 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_messages_queue_lat": { 2024-06-10T03:04:34.362 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 1535, 2024-06-10T03:04:34.362 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.053321937, 2024-06-10T03:04:34.362 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000034737 2024-06-10T03:04:34.362 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:34.362 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_handle_ack_lat": { 2024-06-10T03:04:34.362 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-06-10T03:04:34.362 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-06-10T03:04:34.362 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-06-10T03:04:34.362 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:34.363 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_encrypted_bytes": 3072, 2024-06-10T03:04:34.363 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_encrypted_bytes": 448 2024-06-10T03:04:34.363 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:34.363 INFO:teuthology.orchestra.run.smithi188.stdout: "AsyncMessenger::Worker-1": { 2024-06-10T03:04:34.363 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_messages": 454, 2024-06-10T03:04:34.363 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_messages": 428, 2024-06-10T03:04:34.363 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_bytes": 966248, 2024-06-10T03:04:34.363 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_bytes": 208826, 2024-06-10T03:04:34.363 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_created_connections": 4, 2024-06-10T03:04:34.363 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_active_connections": 3, 2024-06-10T03:04:34.363 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_total_time": 0.054008508, 2024-06-10T03:04:34.363 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_send_time": 0.018074239, 2024-06-10T03:04:34.363 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_recv_time": 0.027345326, 2024-06-10T03:04:34.363 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_fast_dispatch_time": 0.011044617, 2024-06-10T03:04:34.363 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_messages_queue_lat": { 2024-06-10T03:04:34.364 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 427, 2024-06-10T03:04:34.364 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.020939274, 2024-06-10T03:04:34.364 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000049038 2024-06-10T03:04:34.364 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:34.364 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_handle_ack_lat": { 2024-06-10T03:04:34.364 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-06-10T03:04:34.364 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-06-10T03:04:34.364 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-06-10T03:04:34.364 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:34.364 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_encrypted_bytes": 923264, 2024-06-10T03:04:34.364 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_encrypted_bytes": 154064 2024-06-10T03:04:34.364 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:34.364 INFO:teuthology.orchestra.run.smithi188.stdout: "AsyncMessenger::Worker-2": { 2024-06-10T03:04:34.365 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_messages": 830, 2024-06-10T03:04:34.365 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_messages": 1017, 2024-06-10T03:04:34.365 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_bytes": 1993438, 2024-06-10T03:04:34.365 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_bytes": 644039, 2024-06-10T03:04:34.365 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_created_connections": 4, 2024-06-10T03:04:34.365 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_active_connections": 3, 2024-06-10T03:04:34.365 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_total_time": 0.077299885, 2024-06-10T03:04:34.365 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_send_time": 0.030528417, 2024-06-10T03:04:34.365 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_recv_time": 0.016357420, 2024-06-10T03:04:34.365 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_fast_dispatch_time": 0.022984629, 2024-06-10T03:04:34.365 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_messages_queue_lat": { 2024-06-10T03:04:34.365 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 1015, 2024-06-10T03:04:34.365 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.035368680, 2024-06-10T03:04:34.366 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000034845 2024-06-10T03:04:34.366 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:34.366 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_handle_ack_lat": { 2024-06-10T03:04:34.366 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-06-10T03:04:34.366 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-06-10T03:04:34.366 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-06-10T03:04:34.366 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:34.366 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_encrypted_bytes": 160, 2024-06-10T03:04:34.366 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_encrypted_bytes": 380416 2024-06-10T03:04:34.366 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:34.366 INFO:teuthology.orchestra.run.smithi188.stdout: "cct": { 2024-06-10T03:04:34.366 INFO:teuthology.orchestra.run.smithi188.stdout: "total_workers": 1, 2024-06-10T03:04:34.366 INFO:teuthology.orchestra.run.smithi188.stdout: "unhealthy_workers": 0 2024-06-10T03:04:34.367 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:34.367 INFO:teuthology.orchestra.run.smithi188.stdout: "finisher-MDSRank": { 2024-06-10T03:04:34.367 INFO:teuthology.orchestra.run.smithi188.stdout: "queue_len": 0, 2024-06-10T03:04:34.367 INFO:teuthology.orchestra.run.smithi188.stdout: "complete_latency": { 2024-06-10T03:04:34.367 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 4187, 2024-06-10T03:04:34.367 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.107204765, 2024-06-10T03:04:34.367 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000025604 2024-06-10T03:04:34.367 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-06-10T03:04:34.367 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:34.367 INFO:teuthology.orchestra.run.smithi188.stdout: "finisher-PurgeQueue": { 2024-06-10T03:04:34.367 INFO:teuthology.orchestra.run.smithi188.stdout: "queue_len": 0, 2024-06-10T03:04:34.367 INFO:teuthology.orchestra.run.smithi188.stdout: "complete_latency": { 2024-06-10T03:04:34.367 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-06-10T03:04:34.367 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-06-10T03:04:34.368 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-06-10T03:04:34.368 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-06-10T03:04:34.368 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:34.368 INFO:teuthology.orchestra.run.smithi188.stdout: "mds": { 2024-06-10T03:04:34.368 INFO:teuthology.orchestra.run.smithi188.stdout: "request": 0, 2024-06-10T03:04:34.368 INFO:teuthology.orchestra.run.smithi188.stdout: "reply": 0, 2024-06-10T03:04:34.368 INFO:teuthology.orchestra.run.smithi188.stdout: "reply_latency": { 2024-06-10T03:04:34.368 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-06-10T03:04:34.368 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-06-10T03:04:34.368 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-06-10T03:04:34.368 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:34.368 INFO:teuthology.orchestra.run.smithi188.stdout: "slow_reply": 0, 2024-06-10T03:04:34.368 INFO:teuthology.orchestra.run.smithi188.stdout: "forward": 0, 2024-06-10T03:04:34.369 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_fetch_complete": 0, 2024-06-10T03:04:34.369 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_fetch_keys": 0, 2024-06-10T03:04:34.369 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_commit": 0, 2024-06-10T03:04:34.369 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_split": 0, 2024-06-10T03:04:34.369 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_merge": 0, 2024-06-10T03:04:34.369 INFO:teuthology.orchestra.run.smithi188.stdout: "inodes": 746, 2024-06-10T03:04:34.369 INFO:teuthology.orchestra.run.smithi188.stdout: "inodes_top": 0, 2024-06-10T03:04:34.369 INFO:teuthology.orchestra.run.smithi188.stdout: "inodes_bottom": 746, 2024-06-10T03:04:34.369 INFO:teuthology.orchestra.run.smithi188.stdout: "inodes_pin_tail": 0, 2024-06-10T03:04:34.369 INFO:teuthology.orchestra.run.smithi188.stdout: "inodes_pinned": 746, 2024-06-10T03:04:34.369 INFO:teuthology.orchestra.run.smithi188.stdout: "inodes_expired": 0, 2024-06-10T03:04:34.369 INFO:teuthology.orchestra.run.smithi188.stdout: "inodes_with_caps": 0, 2024-06-10T03:04:34.369 INFO:teuthology.orchestra.run.smithi188.stdout: "caps": 0, 2024-06-10T03:04:34.369 INFO:teuthology.orchestra.run.smithi188.stdout: "subtrees": 2, 2024-06-10T03:04:34.370 INFO:teuthology.orchestra.run.smithi188.stdout: "traverse": 0, 2024-06-10T03:04:34.370 INFO:teuthology.orchestra.run.smithi188.stdout: "traverse_hit": 0, 2024-06-10T03:04:34.370 INFO:teuthology.orchestra.run.smithi188.stdout: "traverse_forward": 0, 2024-06-10T03:04:34.370 INFO:teuthology.orchestra.run.smithi188.stdout: "traverse_discover": 0, 2024-06-10T03:04:34.370 INFO:teuthology.orchestra.run.smithi188.stdout: "traverse_dir_fetch": 0, 2024-06-10T03:04:34.370 INFO:teuthology.orchestra.run.smithi188.stdout: "traverse_remote_ino": 0, 2024-06-10T03:04:34.370 INFO:teuthology.orchestra.run.smithi188.stdout: "traverse_lock": 0, 2024-06-10T03:04:34.370 INFO:teuthology.orchestra.run.smithi188.stdout: "load_cent": 0, 2024-06-10T03:04:34.370 INFO:teuthology.orchestra.run.smithi188.stdout: "q": 0, 2024-06-10T03:04:34.370 INFO:teuthology.orchestra.run.smithi188.stdout: "exported": 0, 2024-06-10T03:04:34.370 INFO:teuthology.orchestra.run.smithi188.stdout: "exported_inodes": 0, 2024-06-10T03:04:34.370 INFO:teuthology.orchestra.run.smithi188.stdout: "imported": 0, 2024-06-10T03:04:34.370 INFO:teuthology.orchestra.run.smithi188.stdout: "imported_inodes": 0, 2024-06-10T03:04:34.371 INFO:teuthology.orchestra.run.smithi188.stdout: "openino_dir_fetch": 0, 2024-06-10T03:04:34.371 INFO:teuthology.orchestra.run.smithi188.stdout: "openino_backtrace_fetch": 0, 2024-06-10T03:04:34.371 INFO:teuthology.orchestra.run.smithi188.stdout: "openino_peer_discover": 0, 2024-06-10T03:04:34.371 INFO:teuthology.orchestra.run.smithi188.stdout: "root_rfiles": 4, 2024-06-10T03:04:34.371 INFO:teuthology.orchestra.run.smithi188.stdout: "root_rbytes": 234, 2024-06-10T03:04:34.371 INFO:teuthology.orchestra.run.smithi188.stdout: "root_rsnaps": 0, 2024-06-10T03:04:34.371 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_backtrace_fetch": 0, 2024-06-10T03:04:34.371 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_set_tag": 0, 2024-06-10T03:04:34.371 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_backtrace_repaired": 0, 2024-06-10T03:04:34.371 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_inotable_repaired": 0, 2024-06-10T03:04:34.371 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_dir_inodes": 0, 2024-06-10T03:04:34.371 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_dir_base_inodes": 0, 2024-06-10T03:04:34.371 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_dirfrag_rstats": 0, 2024-06-10T03:04:34.371 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_file_inodes": 0, 2024-06-10T03:04:34.372 INFO:teuthology.orchestra.run.smithi188.stdout: "handle_inode_file_caps": 0, 2024-06-10T03:04:34.372 INFO:teuthology.orchestra.run.smithi188.stdout: "ceph_cap_op_revoke": 0, 2024-06-10T03:04:34.372 INFO:teuthology.orchestra.run.smithi188.stdout: "ceph_cap_op_grant": 0, 2024-06-10T03:04:34.372 INFO:teuthology.orchestra.run.smithi188.stdout: "ceph_cap_op_trunc": 0, 2024-06-10T03:04:34.372 INFO:teuthology.orchestra.run.smithi188.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-06-10T03:04:34.372 INFO:teuthology.orchestra.run.smithi188.stdout: "ceph_cap_op_flush_ack": 0, 2024-06-10T03:04:34.372 INFO:teuthology.orchestra.run.smithi188.stdout: "handle_client_caps": 0, 2024-06-10T03:04:34.372 INFO:teuthology.orchestra.run.smithi188.stdout: "handle_client_caps_dirty": 0, 2024-06-10T03:04:34.372 INFO:teuthology.orchestra.run.smithi188.stdout: "handle_client_cap_release": 0, 2024-06-10T03:04:34.372 INFO:teuthology.orchestra.run.smithi188.stdout: "process_request_cap_release": 0 2024-06-10T03:04:34.372 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:34.372 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_cache": { 2024-06-10T03:04:34.372 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_update": 0, 2024-06-10T03:04:34.372 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_update_receipt": 0, 2024-06-10T03:04:34.373 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_try_discover": 0, 2024-06-10T03:04:34.373 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_send_discover": 0, 2024-06-10T03:04:34.373 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_handle_discover": 0, 2024-06-10T03:04:34.373 INFO:teuthology.orchestra.run.smithi188.stdout: "num_strays": 0, 2024-06-10T03:04:34.373 INFO:teuthology.orchestra.run.smithi188.stdout: "num_strays_delayed": 0, 2024-06-10T03:04:34.373 INFO:teuthology.orchestra.run.smithi188.stdout: "num_strays_enqueuing": 0, 2024-06-10T03:04:34.373 INFO:teuthology.orchestra.run.smithi188.stdout: "strays_created": 0, 2024-06-10T03:04:34.373 INFO:teuthology.orchestra.run.smithi188.stdout: "strays_enqueued": 0, 2024-06-10T03:04:34.373 INFO:teuthology.orchestra.run.smithi188.stdout: "strays_reintegrated": 0, 2024-06-10T03:04:34.373 INFO:teuthology.orchestra.run.smithi188.stdout: "strays_migrated": 0, 2024-06-10T03:04:34.373 INFO:teuthology.orchestra.run.smithi188.stdout: "num_recovering_processing": 0, 2024-06-10T03:04:34.373 INFO:teuthology.orchestra.run.smithi188.stdout: "num_recovering_enqueued": 0, 2024-06-10T03:04:34.373 INFO:teuthology.orchestra.run.smithi188.stdout: "num_recovering_prioritized": 0, 2024-06-10T03:04:34.373 INFO:teuthology.orchestra.run.smithi188.stdout: "recovery_started": 0, 2024-06-10T03:04:34.374 INFO:teuthology.orchestra.run.smithi188.stdout: "recovery_completed": 0, 2024-06-10T03:04:34.374 INFO:teuthology.orchestra.run.smithi188.stdout: "ireq_quiesce_path": 0, 2024-06-10T03:04:34.374 INFO:teuthology.orchestra.run.smithi188.stdout: "ireq_quiesce_inode": 0, 2024-06-10T03:04:34.374 INFO:teuthology.orchestra.run.smithi188.stdout: "ireq_enqueue_scrub": 0, 2024-06-10T03:04:34.374 INFO:teuthology.orchestra.run.smithi188.stdout: "ireq_exportdir": 0, 2024-06-10T03:04:34.374 INFO:teuthology.orchestra.run.smithi188.stdout: "ireq_flush": 0, 2024-06-10T03:04:34.374 INFO:teuthology.orchestra.run.smithi188.stdout: "ireq_fragmentdir": 0, 2024-06-10T03:04:34.374 INFO:teuthology.orchestra.run.smithi188.stdout: "ireq_fragstats": 0, 2024-06-10T03:04:34.374 INFO:teuthology.orchestra.run.smithi188.stdout: "ireq_inodestats": 0 2024-06-10T03:04:34.374 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:34.374 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_log": { 2024-06-10T03:04:34.374 INFO:teuthology.orchestra.run.smithi188.stdout: "evlrg": 0, 2024-06-10T03:04:34.375 INFO:teuthology.orchestra.run.smithi188.stdout: "evadd": 0, 2024-06-10T03:04:34.375 INFO:teuthology.orchestra.run.smithi188.stdout: "evex": 0, 2024-06-10T03:04:34.375 INFO:teuthology.orchestra.run.smithi188.stdout: "evtrm": 0, 2024-06-10T03:04:34.375 INFO:teuthology.orchestra.run.smithi188.stdout: "ev": 3331, 2024-06-10T03:04:34.375 INFO:teuthology.orchestra.run.smithi188.stdout: "evexg": 0, 2024-06-10T03:04:34.375 INFO:teuthology.orchestra.run.smithi188.stdout: "evexd": 0, 2024-06-10T03:04:34.375 INFO:teuthology.orchestra.run.smithi188.stdout: "segadd": 0, 2024-06-10T03:04:34.375 INFO:teuthology.orchestra.run.smithi188.stdout: "segex": 0, 2024-06-10T03:04:34.375 INFO:teuthology.orchestra.run.smithi188.stdout: "segtrm": 0, 2024-06-10T03:04:34.375 INFO:teuthology.orchestra.run.smithi188.stdout: "seg": 7, 2024-06-10T03:04:34.375 INFO:teuthology.orchestra.run.smithi188.stdout: "segmjr": 2, 2024-06-10T03:04:34.375 INFO:teuthology.orchestra.run.smithi188.stdout: "segexg": 0, 2024-06-10T03:04:34.375 INFO:teuthology.orchestra.run.smithi188.stdout: "segexd": 0, 2024-06-10T03:04:34.375 INFO:teuthology.orchestra.run.smithi188.stdout: "expos": 4194304, 2024-06-10T03:04:34.376 INFO:teuthology.orchestra.run.smithi188.stdout: "wrpos": 10175017, 2024-06-10T03:04:34.376 INFO:teuthology.orchestra.run.smithi188.stdout: "rdpos": 10170716, 2024-06-10T03:04:34.376 INFO:teuthology.orchestra.run.smithi188.stdout: "jlat": { 2024-06-10T03:04:34.376 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-06-10T03:04:34.376 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-06-10T03:04:34.376 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-06-10T03:04:34.376 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:34.376 INFO:teuthology.orchestra.run.smithi188.stdout: "replayed": 3331 2024-06-10T03:04:34.376 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:34.376 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_mem": { 2024-06-10T03:04:34.376 INFO:teuthology.orchestra.run.smithi188.stdout: "ino": 19, 2024-06-10T03:04:34.376 INFO:teuthology.orchestra.run.smithi188.stdout: "ino+": 714, 2024-06-10T03:04:34.376 INFO:teuthology.orchestra.run.smithi188.stdout: "ino-": 695, 2024-06-10T03:04:34.376 INFO:teuthology.orchestra.run.smithi188.stdout: "dir": 14, 2024-06-10T03:04:34.377 INFO:teuthology.orchestra.run.smithi188.stdout: "dir+": 45, 2024-06-10T03:04:34.377 INFO:teuthology.orchestra.run.smithi188.stdout: "dir-": 31, 2024-06-10T03:04:34.377 INFO:teuthology.orchestra.run.smithi188.stdout: "dn": 746, 2024-06-10T03:04:34.377 INFO:teuthology.orchestra.run.smithi188.stdout: "dn+": 1352, 2024-06-10T03:04:34.377 INFO:teuthology.orchestra.run.smithi188.stdout: "dn-": 606, 2024-06-10T03:04:34.377 INFO:teuthology.orchestra.run.smithi188.stdout: "cap": 0, 2024-06-10T03:04:34.377 INFO:teuthology.orchestra.run.smithi188.stdout: "cap+": 0, 2024-06-10T03:04:34.377 INFO:teuthology.orchestra.run.smithi188.stdout: "cap-": 0, 2024-06-10T03:04:34.377 INFO:teuthology.orchestra.run.smithi188.stdout: "rss": 56188, 2024-06-10T03:04:34.377 INFO:teuthology.orchestra.run.smithi188.stdout: "heap": 215420 2024-06-10T03:04:34.377 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:34.377 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_server": { 2024-06-10T03:04:34.377 INFO:teuthology.orchestra.run.smithi188.stdout: "dispatch_client_request": 0, 2024-06-10T03:04:34.378 INFO:teuthology.orchestra.run.smithi188.stdout: "dispatch_server_request": 0, 2024-06-10T03:04:34.378 INFO:teuthology.orchestra.run.smithi188.stdout: "handle_client_request": 0, 2024-06-10T03:04:34.378 INFO:teuthology.orchestra.run.smithi188.stdout: "handle_client_session": 0, 2024-06-10T03:04:34.378 INFO:teuthology.orchestra.run.smithi188.stdout: "handle_peer_request": 0, 2024-06-10T03:04:34.378 INFO:teuthology.orchestra.run.smithi188.stdout: "req_create_latency": { 2024-06-10T03:04:34.378 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-06-10T03:04:34.378 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-06-10T03:04:34.378 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-06-10T03:04:34.378 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:34.378 INFO:teuthology.orchestra.run.smithi188.stdout: "req_getattr_latency": { 2024-06-10T03:04:34.378 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-06-10T03:04:34.378 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-06-10T03:04:34.378 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-06-10T03:04:34.378 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:34.379 INFO:teuthology.orchestra.run.smithi188.stdout: "req_getfilelock_latency": { 2024-06-10T03:04:34.379 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-06-10T03:04:34.379 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-06-10T03:04:34.379 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-06-10T03:04:34.379 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:34.379 INFO:teuthology.orchestra.run.smithi188.stdout: "req_link_latency": { 2024-06-10T03:04:34.379 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-06-10T03:04:34.379 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-06-10T03:04:34.379 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-06-10T03:04:34.379 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:34.379 INFO:teuthology.orchestra.run.smithi188.stdout: "req_lookup_latency": { 2024-06-10T03:04:34.379 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-06-10T03:04:34.379 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-06-10T03:04:34.379 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-06-10T03:04:34.380 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:34.380 INFO:teuthology.orchestra.run.smithi188.stdout: "req_lookuphash_latency": { 2024-06-10T03:04:34.380 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-06-10T03:04:34.380 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-06-10T03:04:34.380 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-06-10T03:04:34.380 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:34.380 INFO:teuthology.orchestra.run.smithi188.stdout: "req_lookupino_latency": { 2024-06-10T03:04:34.380 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-06-10T03:04:34.380 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-06-10T03:04:34.380 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-06-10T03:04:34.380 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:34.380 INFO:teuthology.orchestra.run.smithi188.stdout: "req_lookupname_latency": { 2024-06-10T03:04:34.380 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-06-10T03:04:34.380 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-06-10T03:04:34.381 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-06-10T03:04:34.381 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:34.381 INFO:teuthology.orchestra.run.smithi188.stdout: "req_lookupparent_latency": { 2024-06-10T03:04:34.381 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-06-10T03:04:34.381 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-06-10T03:04:34.381 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-06-10T03:04:34.381 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:34.381 INFO:teuthology.orchestra.run.smithi188.stdout: "req_lookupsnap_latency": { 2024-06-10T03:04:34.381 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-06-10T03:04:34.381 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-06-10T03:04:34.381 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-06-10T03:04:34.381 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:34.381 INFO:teuthology.orchestra.run.smithi188.stdout: "req_lssnap_latency": { 2024-06-10T03:04:34.381 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-06-10T03:04:34.381 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-06-10T03:04:34.382 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-06-10T03:04:34.382 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:34.382 INFO:teuthology.orchestra.run.smithi188.stdout: "req_mkdir_latency": { 2024-06-10T03:04:34.382 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-06-10T03:04:34.382 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-06-10T03:04:34.382 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-06-10T03:04:34.383 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:34.383 INFO:teuthology.orchestra.run.smithi188.stdout: "req_mknod_latency": { 2024-06-10T03:04:34.383 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-06-10T03:04:34.383 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-06-10T03:04:34.383 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-06-10T03:04:34.383 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:34.383 INFO:teuthology.orchestra.run.smithi188.stdout: "req_mksnap_latency": { 2024-06-10T03:04:34.383 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-06-10T03:04:34.383 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-06-10T03:04:34.383 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-06-10T03:04:34.383 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:34.383 INFO:teuthology.orchestra.run.smithi188.stdout: "req_open_latency": { 2024-06-10T03:04:34.383 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-06-10T03:04:34.383 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-06-10T03:04:34.384 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-06-10T03:04:34.384 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:34.384 INFO:teuthology.orchestra.run.smithi188.stdout: "req_readdir_latency": { 2024-06-10T03:04:34.384 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-06-10T03:04:34.384 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-06-10T03:04:34.384 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-06-10T03:04:34.384 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:34.384 INFO:teuthology.orchestra.run.smithi188.stdout: "req_rename_latency": { 2024-06-10T03:04:34.384 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-06-10T03:04:34.384 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-06-10T03:04:34.384 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-06-10T03:04:34.384 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:34.384 INFO:teuthology.orchestra.run.smithi188.stdout: "req_renamesnap_latency": { 2024-06-10T03:04:34.385 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-06-10T03:04:34.385 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-06-10T03:04:34.385 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-06-10T03:04:34.385 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:34.385 INFO:teuthology.orchestra.run.smithi188.stdout: "req_snapdiff_latency": { 2024-06-10T03:04:34.385 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-06-10T03:04:34.385 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-06-10T03:04:34.385 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-06-10T03:04:34.385 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:34.385 INFO:teuthology.orchestra.run.smithi188.stdout: "req_rmdir_latency": { 2024-06-10T03:04:34.385 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-06-10T03:04:34.385 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-06-10T03:04:34.385 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-06-10T03:04:34.385 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:34.386 INFO:teuthology.orchestra.run.smithi188.stdout: "req_rmsnap_latency": { 2024-06-10T03:04:34.386 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-06-10T03:04:34.386 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-06-10T03:04:34.386 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-06-10T03:04:34.386 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:34.386 INFO:teuthology.orchestra.run.smithi188.stdout: "req_rmxattr_latency": { 2024-06-10T03:04:34.386 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-06-10T03:04:34.386 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-06-10T03:04:34.386 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-06-10T03:04:34.386 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:34.386 INFO:teuthology.orchestra.run.smithi188.stdout: "req_setattr_latency": { 2024-06-10T03:04:34.386 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-06-10T03:04:34.386 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-06-10T03:04:34.386 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-06-10T03:04:34.386 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:34.387 INFO:teuthology.orchestra.run.smithi188.stdout: "req_setdirlayout_latency": { 2024-06-10T03:04:34.387 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-06-10T03:04:34.387 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-06-10T03:04:34.387 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-06-10T03:04:34.387 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:34.387 INFO:teuthology.orchestra.run.smithi188.stdout: "req_setfilelock_latency": { 2024-06-10T03:04:34.387 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-06-10T03:04:34.387 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-06-10T03:04:34.387 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-06-10T03:04:34.387 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:34.387 INFO:teuthology.orchestra.run.smithi188.stdout: "req_setlayout_latency": { 2024-06-10T03:04:34.387 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-06-10T03:04:34.387 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-06-10T03:04:34.388 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-06-10T03:04:34.388 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:34.388 INFO:teuthology.orchestra.run.smithi188.stdout: "req_setxattr_latency": { 2024-06-10T03:04:34.388 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-06-10T03:04:34.388 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-06-10T03:04:34.388 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-06-10T03:04:34.388 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:34.388 INFO:teuthology.orchestra.run.smithi188.stdout: "req_symlink_latency": { 2024-06-10T03:04:34.388 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-06-10T03:04:34.388 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-06-10T03:04:34.388 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-06-10T03:04:34.388 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:34.388 INFO:teuthology.orchestra.run.smithi188.stdout: "req_unlink_latency": { 2024-06-10T03:04:34.388 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-06-10T03:04:34.388 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-06-10T03:04:34.389 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-06-10T03:04:34.389 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:34.389 INFO:teuthology.orchestra.run.smithi188.stdout: "cap_revoke_eviction": 0, 2024-06-10T03:04:34.389 INFO:teuthology.orchestra.run.smithi188.stdout: "cap_acquisition_throttle": 0, 2024-06-10T03:04:34.389 INFO:teuthology.orchestra.run.smithi188.stdout: "req_getvxattr_latency": { 2024-06-10T03:04:34.389 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-06-10T03:04:34.389 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-06-10T03:04:34.389 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-06-10T03:04:34.389 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-06-10T03:04:34.389 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:34.389 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_sessions": { 2024-06-10T03:04:34.389 INFO:teuthology.orchestra.run.smithi188.stdout: "session_count": 1, 2024-06-10T03:04:34.389 INFO:teuthology.orchestra.run.smithi188.stdout: "session_add": 4, 2024-06-10T03:04:34.389 INFO:teuthology.orchestra.run.smithi188.stdout: "session_remove": 3, 2024-06-10T03:04:34.390 INFO:teuthology.orchestra.run.smithi188.stdout: "sessions_open": 2, 2024-06-10T03:04:34.390 INFO:teuthology.orchestra.run.smithi188.stdout: "sessions_stale": 0, 2024-06-10T03:04:34.390 INFO:teuthology.orchestra.run.smithi188.stdout: "total_load": 0, 2024-06-10T03:04:34.390 INFO:teuthology.orchestra.run.smithi188.stdout: "average_load": 0, 2024-06-10T03:04:34.390 INFO:teuthology.orchestra.run.smithi188.stdout: "avg_session_uptime": 4081, 2024-06-10T03:04:34.390 INFO:teuthology.orchestra.run.smithi188.stdout: "mdthresh_evicted": 0 2024-06-10T03:04:34.390 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:34.390 INFO:teuthology.orchestra.run.smithi188.stdout: "mempool": { 2024-06-10T03:04:34.390 INFO:teuthology.orchestra.run.smithi188.stdout: "bloom_filter_bytes": 0, 2024-06-10T03:04:34.390 INFO:teuthology.orchestra.run.smithi188.stdout: "bloom_filter_items": 0, 2024-06-10T03:04:34.390 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_alloc_bytes": 0, 2024-06-10T03:04:34.390 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_alloc_items": 0, 2024-06-10T03:04:34.390 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_data_bytes": 0, 2024-06-10T03:04:34.390 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_data_items": 0, 2024-06-10T03:04:34.391 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-10T03:04:34.391 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_onode_items": 0, 2024-06-10T03:04:34.391 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-10T03:04:34.391 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_meta_items": 0, 2024-06-10T03:04:34.391 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_other_bytes": 0, 2024-06-10T03:04:34.391 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_other_items": 0, 2024-06-10T03:04:34.391 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-10T03:04:34.391 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_buffer_items": 0, 2024-06-10T03:04:34.391 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_extent_bytes": 0, 2024-06-10T03:04:34.391 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_extent_items": 0, 2024-06-10T03:04:34.391 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_blob_bytes": 0, 2024-06-10T03:04:34.391 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_blob_items": 0, 2024-06-10T03:04:34.391 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-10T03:04:34.391 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_shared_blob_items": 0, 2024-06-10T03:04:34.392 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-10T03:04:34.392 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_inline_bl_items": 0, 2024-06-10T03:04:34.392 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_fsck_bytes": 0, 2024-06-10T03:04:34.392 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_fsck_items": 0, 2024-06-10T03:04:34.392 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_txc_bytes": 0, 2024-06-10T03:04:34.392 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_txc_items": 0, 2024-06-10T03:04:34.392 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-10T03:04:34.392 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_writing_deferred_items": 0, 2024-06-10T03:04:34.392 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_writing_bytes": 0, 2024-06-10T03:04:34.392 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_writing_items": 0, 2024-06-10T03:04:34.392 INFO:teuthology.orchestra.run.smithi188.stdout: "bluefs_bytes": 0, 2024-06-10T03:04:34.392 INFO:teuthology.orchestra.run.smithi188.stdout: "bluefs_items": 0, 2024-06-10T03:04:34.392 INFO:teuthology.orchestra.run.smithi188.stdout: "bluefs_file_reader_bytes": 0, 2024-06-10T03:04:34.392 INFO:teuthology.orchestra.run.smithi188.stdout: "bluefs_file_reader_items": 0, 2024-06-10T03:04:34.392 INFO:teuthology.orchestra.run.smithi188.stdout: "bluefs_file_writer_bytes": 0, 2024-06-10T03:04:34.393 INFO:teuthology.orchestra.run.smithi188.stdout: "bluefs_file_writer_items": 0, 2024-06-10T03:04:34.393 INFO:teuthology.orchestra.run.smithi188.stdout: "buffer_anon_bytes": 189497, 2024-06-10T03:04:34.393 INFO:teuthology.orchestra.run.smithi188.stdout: "buffer_anon_items": 53, 2024-06-10T03:04:34.393 INFO:teuthology.orchestra.run.smithi188.stdout: "buffer_meta_bytes": 0, 2024-06-10T03:04:34.393 INFO:teuthology.orchestra.run.smithi188.stdout: "buffer_meta_items": 0, 2024-06-10T03:04:34.393 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_bytes": 0, 2024-06-10T03:04:34.393 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_items": 0, 2024-06-10T03:04:34.393 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_mapbl_bytes": 0, 2024-06-10T03:04:34.393 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_mapbl_items": 0, 2024-06-10T03:04:34.393 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_pglog_bytes": 0, 2024-06-10T03:04:34.393 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_pglog_items": 0, 2024-06-10T03:04:34.393 INFO:teuthology.orchestra.run.smithi188.stdout: "osdmap_bytes": 8984, 2024-06-10T03:04:34.393 INFO:teuthology.orchestra.run.smithi188.stdout: "osdmap_items": 172, 2024-06-10T03:04:34.393 INFO:teuthology.orchestra.run.smithi188.stdout: "osdmap_mapping_bytes": 0, 2024-06-10T03:04:34.393 INFO:teuthology.orchestra.run.smithi188.stdout: "osdmap_mapping_items": 0, 2024-06-10T03:04:34.394 INFO:teuthology.orchestra.run.smithi188.stdout: "pgmap_bytes": 0, 2024-06-10T03:04:34.394 INFO:teuthology.orchestra.run.smithi188.stdout: "pgmap_items": 0, 2024-06-10T03:04:34.394 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_co_bytes": 586023, 2024-06-10T03:04:34.394 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_co_items": 3458, 2024-06-10T03:04:34.394 INFO:teuthology.orchestra.run.smithi188.stdout: "unittest_1_bytes": 0, 2024-06-10T03:04:34.394 INFO:teuthology.orchestra.run.smithi188.stdout: "unittest_1_items": 0, 2024-06-10T03:04:34.394 INFO:teuthology.orchestra.run.smithi188.stdout: "unittest_2_bytes": 0, 2024-06-10T03:04:34.394 INFO:teuthology.orchestra.run.smithi188.stdout: "unittest_2_items": 0 2024-06-10T03:04:34.394 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:34.394 INFO:teuthology.orchestra.run.smithi188.stdout: "objecter": { 2024-06-10T03:04:34.394 INFO:teuthology.orchestra.run.smithi188.stdout: "op_active": 0, 2024-06-10T03:04:34.394 INFO:teuthology.orchestra.run.smithi188.stdout: "op_laggy": 0, 2024-06-10T03:04:34.394 INFO:teuthology.orchestra.run.smithi188.stdout: "op_send": 2533, 2024-06-10T03:04:34.394 INFO:teuthology.orchestra.run.smithi188.stdout: "op_send_bytes": 26, 2024-06-10T03:04:34.394 INFO:teuthology.orchestra.run.smithi188.stdout: "op_resend": 4, 2024-06-10T03:04:34.395 INFO:teuthology.orchestra.run.smithi188.stdout: "op_reply": 2533, 2024-06-10T03:04:34.395 INFO:teuthology.orchestra.run.smithi188.stdout: "op_latency": { 2024-06-10T03:04:34.395 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 2533, 2024-06-10T03:04:34.395 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 3.270941816, 2024-06-10T03:04:34.395 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.001291331 2024-06-10T03:04:34.395 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:34.395 INFO:teuthology.orchestra.run.smithi188.stdout: "op_inflight": 0, 2024-06-10T03:04:34.395 INFO:teuthology.orchestra.run.smithi188.stdout: "oplen_avg": { 2024-06-10T03:04:34.395 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 2533, 2024-06-10T03:04:34.395 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 2534 2024-06-10T03:04:34.395 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:34.395 INFO:teuthology.orchestra.run.smithi188.stdout: "op": 2533, 2024-06-10T03:04:34.395 INFO:teuthology.orchestra.run.smithi188.stdout: "op_r": 2533, 2024-06-10T03:04:34.396 INFO:teuthology.orchestra.run.smithi188.stdout: "op_w": 0, 2024-06-10T03:04:34.396 INFO:teuthology.orchestra.run.smithi188.stdout: "op_rmw": 0, 2024-06-10T03:04:34.396 INFO:teuthology.orchestra.run.smithi188.stdout: "op_pg": 0, 2024-06-10T03:04:34.396 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_stat": 1604, 2024-06-10T03:04:34.396 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_create": 0, 2024-06-10T03:04:34.396 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_read": 926, 2024-06-10T03:04:34.396 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_write": 0, 2024-06-10T03:04:34.396 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_writefull": 0, 2024-06-10T03:04:34.396 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_writesame": 0, 2024-06-10T03:04:34.396 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_append": 0, 2024-06-10T03:04:34.396 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_zero": 0, 2024-06-10T03:04:34.396 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_truncate": 0, 2024-06-10T03:04:34.396 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_delete": 0, 2024-06-10T03:04:34.396 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_mapext": 0, 2024-06-10T03:04:34.396 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_sparse_read": 0, 2024-06-10T03:04:34.397 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_clonerange": 0, 2024-06-10T03:04:34.397 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_getxattr": 2, 2024-06-10T03:04:34.397 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_setxattr": 0, 2024-06-10T03:04:34.397 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_cmpxattr": 0, 2024-06-10T03:04:34.397 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_rmxattr": 0, 2024-06-10T03:04:34.397 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_resetxattrs": 0, 2024-06-10T03:04:34.397 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_call": 0, 2024-06-10T03:04:34.397 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_watch": 0, 2024-06-10T03:04:34.397 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_notify": 0, 2024-06-10T03:04:34.397 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_src_cmpxattr": 0, 2024-06-10T03:04:34.397 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_pgls": 0, 2024-06-10T03:04:34.397 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_pgls_filter": 0, 2024-06-10T03:04:34.397 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_other": 0, 2024-06-10T03:04:34.397 INFO:teuthology.orchestra.run.smithi188.stdout: "linger_active": 0, 2024-06-10T03:04:34.398 INFO:teuthology.orchestra.run.smithi188.stdout: "linger_send": 0, 2024-06-10T03:04:34.398 INFO:teuthology.orchestra.run.smithi188.stdout: "linger_resend": 0, 2024-06-10T03:04:34.398 INFO:teuthology.orchestra.run.smithi188.stdout: "linger_ping": 0, 2024-06-10T03:04:34.398 INFO:teuthology.orchestra.run.smithi188.stdout: "poolop_active": 0, 2024-06-10T03:04:34.398 INFO:teuthology.orchestra.run.smithi188.stdout: "poolop_send": 0, 2024-06-10T03:04:34.398 INFO:teuthology.orchestra.run.smithi188.stdout: "poolop_resend": 0, 2024-06-10T03:04:34.398 INFO:teuthology.orchestra.run.smithi188.stdout: "poolstat_active": 0, 2024-06-10T03:04:34.398 INFO:teuthology.orchestra.run.smithi188.stdout: "poolstat_send": 0, 2024-06-10T03:04:34.398 INFO:teuthology.orchestra.run.smithi188.stdout: "poolstat_resend": 0, 2024-06-10T03:04:34.398 INFO:teuthology.orchestra.run.smithi188.stdout: "statfs_active": 0, 2024-06-10T03:04:34.398 INFO:teuthology.orchestra.run.smithi188.stdout: "statfs_send": 0, 2024-06-10T03:04:34.398 INFO:teuthology.orchestra.run.smithi188.stdout: "statfs_resend": 0, 2024-06-10T03:04:34.398 INFO:teuthology.orchestra.run.smithi188.stdout: "command_active": 0, 2024-06-10T03:04:34.398 INFO:teuthology.orchestra.run.smithi188.stdout: "command_send": 0, 2024-06-10T03:04:34.399 INFO:teuthology.orchestra.run.smithi188.stdout: "command_resend": 0, 2024-06-10T03:04:34.399 INFO:teuthology.orchestra.run.smithi188.stdout: "map_epoch": 100, 2024-06-10T03:04:34.399 INFO:teuthology.orchestra.run.smithi188.stdout: "map_full": 0, 2024-06-10T03:04:34.399 INFO:teuthology.orchestra.run.smithi188.stdout: "map_inc": 2, 2024-06-10T03:04:34.399 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_sessions": 5, 2024-06-10T03:04:34.399 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_session_open": 5, 2024-06-10T03:04:34.399 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_session_close": 0, 2024-06-10T03:04:34.399 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_laggy": 0, 2024-06-10T03:04:34.399 INFO:teuthology.orchestra.run.smithi188.stdout: "omap_wr": 0, 2024-06-10T03:04:34.399 INFO:teuthology.orchestra.run.smithi188.stdout: "omap_rd": 2, 2024-06-10T03:04:34.399 INFO:teuthology.orchestra.run.smithi188.stdout: "omap_del": 0 2024-06-10T03:04:34.399 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:34.399 INFO:teuthology.orchestra.run.smithi188.stdout: "oft": { 2024-06-10T03:04:34.399 INFO:teuthology.orchestra.run.smithi188.stdout: "omap_total_objs": 0, 2024-06-10T03:04:34.400 INFO:teuthology.orchestra.run.smithi188.stdout: "omap_total_kv_pairs": 0, 2024-06-10T03:04:34.400 INFO:teuthology.orchestra.run.smithi188.stdout: "omap_total_updates": 0, 2024-06-10T03:04:34.400 INFO:teuthology.orchestra.run.smithi188.stdout: "omap_total_removes": 0 2024-06-10T03:04:34.400 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:34.400 INFO:teuthology.orchestra.run.smithi188.stdout: "osd-slow-ops": { 2024-06-10T03:04:34.400 INFO:teuthology.orchestra.run.smithi188.stdout: "slow_ops_count": 0 2024-06-10T03:04:34.400 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:34.400 INFO:teuthology.orchestra.run.smithi188.stdout: "purge_queue": { 2024-06-10T03:04:34.400 INFO:teuthology.orchestra.run.smithi188.stdout: "pq_executing_ops": 0, 2024-06-10T03:04:34.400 INFO:teuthology.orchestra.run.smithi188.stdout: "pq_executing_ops_high_water": 0, 2024-06-10T03:04:34.400 INFO:teuthology.orchestra.run.smithi188.stdout: "pq_executing": 0, 2024-06-10T03:04:34.400 INFO:teuthology.orchestra.run.smithi188.stdout: "pq_executing_high_water": 0, 2024-06-10T03:04:34.400 INFO:teuthology.orchestra.run.smithi188.stdout: "pq_executed_ops": 0, 2024-06-10T03:04:34.400 INFO:teuthology.orchestra.run.smithi188.stdout: "pq_executed": 0, 2024-06-10T03:04:34.401 INFO:teuthology.orchestra.run.smithi188.stdout: "pq_item_in_journal": 0 2024-06-10T03:04:34.401 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:34.401 INFO:teuthology.orchestra.run.smithi188.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-10T03:04:34.401 INFO:teuthology.orchestra.run.smithi188.stdout: "val": 0, 2024-06-10T03:04:34.401 INFO:teuthology.orchestra.run.smithi188.stdout: "max": 104857600, 2024-06-10T03:04:34.401 INFO:teuthology.orchestra.run.smithi188.stdout: "get_started": 0, 2024-06-10T03:04:34.401 INFO:teuthology.orchestra.run.smithi188.stdout: "get": 2816, 2024-06-10T03:04:34.401 INFO:teuthology.orchestra.run.smithi188.stdout: "get_sum": 7347183, 2024-06-10T03:04:34.401 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_fail": 0, 2024-06-10T03:04:34.401 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_success": 2816, 2024-06-10T03:04:34.401 INFO:teuthology.orchestra.run.smithi188.stdout: "take": 0, 2024-06-10T03:04:34.401 INFO:teuthology.orchestra.run.smithi188.stdout: "take_sum": 0, 2024-06-10T03:04:34.401 INFO:teuthology.orchestra.run.smithi188.stdout: "put": 2816, 2024-06-10T03:04:34.402 INFO:teuthology.orchestra.run.smithi188.stdout: "put_sum": 7347183, 2024-06-10T03:04:34.402 INFO:teuthology.orchestra.run.smithi188.stdout: "wait": { 2024-06-10T03:04:34.402 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-06-10T03:04:34.402 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-06-10T03:04:34.402 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-06-10T03:04:34.402 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-06-10T03:04:34.402 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:34.402 INFO:teuthology.orchestra.run.smithi188.stdout: "throttle-objecter_bytes": { 2024-06-10T03:04:34.402 INFO:teuthology.orchestra.run.smithi188.stdout: "val": 0, 2024-06-10T03:04:34.402 INFO:teuthology.orchestra.run.smithi188.stdout: "max": 104857600, 2024-06-10T03:04:34.402 INFO:teuthology.orchestra.run.smithi188.stdout: "get_started": 0, 2024-06-10T03:04:34.402 INFO:teuthology.orchestra.run.smithi188.stdout: "get": 0, 2024-06-10T03:04:34.402 INFO:teuthology.orchestra.run.smithi188.stdout: "get_sum": 0, 2024-06-10T03:04:34.403 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_fail": 0, 2024-06-10T03:04:34.403 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_success": 0, 2024-06-10T03:04:34.403 INFO:teuthology.orchestra.run.smithi188.stdout: "take": 2533, 2024-06-10T03:04:34.403 INFO:teuthology.orchestra.run.smithi188.stdout: "take_sum": 5981904, 2024-06-10T03:04:34.403 INFO:teuthology.orchestra.run.smithi188.stdout: "put": 2533, 2024-06-10T03:04:34.403 INFO:teuthology.orchestra.run.smithi188.stdout: "put_sum": 5981904, 2024-06-10T03:04:34.403 INFO:teuthology.orchestra.run.smithi188.stdout: "wait": { 2024-06-10T03:04:34.403 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-06-10T03:04:34.403 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-06-10T03:04:34.403 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-06-10T03:04:34.403 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-06-10T03:04:34.403 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:34.403 INFO:teuthology.orchestra.run.smithi188.stdout: "throttle-objecter_ops": { 2024-06-10T03:04:34.403 INFO:teuthology.orchestra.run.smithi188.stdout: "val": 0, 2024-06-10T03:04:34.404 INFO:teuthology.orchestra.run.smithi188.stdout: "max": 1024, 2024-06-10T03:04:34.404 INFO:teuthology.orchestra.run.smithi188.stdout: "get_started": 0, 2024-06-10T03:04:34.404 INFO:teuthology.orchestra.run.smithi188.stdout: "get": 0, 2024-06-10T03:04:34.404 INFO:teuthology.orchestra.run.smithi188.stdout: "get_sum": 0, 2024-06-10T03:04:34.404 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_fail": 0, 2024-06-10T03:04:34.404 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_success": 0, 2024-06-10T03:04:34.404 INFO:teuthology.orchestra.run.smithi188.stdout: "take": 2533, 2024-06-10T03:04:34.404 INFO:teuthology.orchestra.run.smithi188.stdout: "take_sum": 2533, 2024-06-10T03:04:34.404 INFO:teuthology.orchestra.run.smithi188.stdout: "put": 2533, 2024-06-10T03:04:34.404 INFO:teuthology.orchestra.run.smithi188.stdout: "put_sum": 2533, 2024-06-10T03:04:34.404 INFO:teuthology.orchestra.run.smithi188.stdout: "wait": { 2024-06-10T03:04:34.404 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-06-10T03:04:34.404 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-06-10T03:04:34.404 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-06-10T03:04:34.404 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-06-10T03:04:34.405 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:34.405 INFO:teuthology.orchestra.run.smithi188.stdout: "throttle-write_buf_throttle": { 2024-06-10T03:04:34.405 INFO:teuthology.orchestra.run.smithi188.stdout: "val": 0, 2024-06-10T03:04:34.405 INFO:teuthology.orchestra.run.smithi188.stdout: "max": 3758096384, 2024-06-10T03:04:34.405 INFO:teuthology.orchestra.run.smithi188.stdout: "get_started": 0, 2024-06-10T03:04:34.405 INFO:teuthology.orchestra.run.smithi188.stdout: "get": 0, 2024-06-10T03:04:34.405 INFO:teuthology.orchestra.run.smithi188.stdout: "get_sum": 0, 2024-06-10T03:04:34.405 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_fail": 0, 2024-06-10T03:04:34.405 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_success": 0, 2024-06-10T03:04:34.405 INFO:teuthology.orchestra.run.smithi188.stdout: "take": 0, 2024-06-10T03:04:34.405 INFO:teuthology.orchestra.run.smithi188.stdout: "take_sum": 0, 2024-06-10T03:04:34.405 INFO:teuthology.orchestra.run.smithi188.stdout: "put": 0, 2024-06-10T03:04:34.405 INFO:teuthology.orchestra.run.smithi188.stdout: "put_sum": 0, 2024-06-10T03:04:34.405 INFO:teuthology.orchestra.run.smithi188.stdout: "wait": { 2024-06-10T03:04:34.406 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-06-10T03:04:34.406 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-06-10T03:04:34.406 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-06-10T03:04:34.406 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-06-10T03:04:34.406 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-06-10T03:04:34.406 INFO:teuthology.orchestra.run.smithi188.stdout: "throttle-write_buf_throttle-0x55c152f466e0": { 2024-06-10T03:04:34.406 INFO:teuthology.orchestra.run.smithi188.stdout: "val": 0, 2024-06-10T03:04:34.406 INFO:teuthology.orchestra.run.smithi188.stdout: "max": 3758096384, 2024-06-10T03:04:34.406 INFO:teuthology.orchestra.run.smithi188.stdout: "get_started": 0, 2024-06-10T03:04:34.406 INFO:teuthology.orchestra.run.smithi188.stdout: "get": 0, 2024-06-10T03:04:34.406 INFO:teuthology.orchestra.run.smithi188.stdout: "get_sum": 0, 2024-06-10T03:04:34.406 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_fail": 0, 2024-06-10T03:04:34.406 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_success": 0, 2024-06-10T03:04:34.407 INFO:teuthology.orchestra.run.smithi188.stdout: "take": 0, 2024-06-10T03:04:34.407 INFO:teuthology.orchestra.run.smithi188.stdout: "take_sum": 0, 2024-06-10T03:04:34.407 INFO:teuthology.orchestra.run.smithi188.stdout: "put": 0, 2024-06-10T03:04:34.407 INFO:teuthology.orchestra.run.smithi188.stdout: "put_sum": 0, 2024-06-10T03:04:34.407 INFO:teuthology.orchestra.run.smithi188.stdout: "wait": { 2024-06-10T03:04:34.407 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-06-10T03:04:34.407 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-06-10T03:04:34.407 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-06-10T03:04:34.407 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-06-10T03:04:34.407 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-06-10T03:04:34.407 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-06-10T03:04:35.204 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:04:34 smithi188 ceph-mon[36599]: pgmap v745: 129 pgs: 129 active+clean; 228 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 0 B/s wr, 12 op/s 2024-06-10T03:04:35.204 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:04:34 smithi188 ceph-mon[36599]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-10T03:04:35.227 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:04:34 smithi052 ceph-mon[32362]: pgmap v745: 129 pgs: 129 active+clean; 228 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 0 B/s wr, 12 op/s 2024-06-10T03:04:35.227 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:04:34 smithi052 ceph-mon[32362]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-10T03:04:35.305 INFO:tasks.check_counter:Daemon mds.l mds.exported=0 2024-06-10T03:04:35.306 INFO:tasks.check_counter:Daemon mds.l mds.imported=0 2024-06-10T03:04:35.306 INFO:tasks.check_counter:Daemon mds.l mds_cache.dir_update=0 2024-06-10T03:04:35.306 INFO:tasks.check_counter:Daemon mds.l mds_cache.dir_update_receipt=0 2024-06-10T03:04:35.306 INFO:tasks.check_counter:Daemon mds.l mds.dir_split=0 2024-06-10T03:04:35.306 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-06-10T03:04:35.317 INFO:tasks.kclient:Unmounting kernel clients... 2024-06-10T03:04:35.317 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-06-10T03:04:35.317 DEBUG:teuthology.orchestra.run.smithi052:> dd if=/proc/self/mounts of=/dev/stdout 2024-06-10T03:04:35.339 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-06-10T03:04:35.339 DEBUG:teuthology.orchestra.run.smithi052:> dd if=/proc/self/mounts of=/dev/stdout 2024-06-10T03:04:35.388 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:04:34 smithi120 ceph-mon[36634]: pgmap v745: 129 pgs: 129 active+clean; 228 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 0 B/s wr, 12 op/s 2024-06-10T03:04:35.388 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:04:34 smithi120 ceph-mon[36634]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-10T03:04:35.400 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph osd blocklist ls 2024-06-10T03:04:35.705 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T03:04:37.087 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:04:36 smithi052 ceph-mon[32362]: pgmap v746: 129 pgs: 129 active+clean; 228 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 8 op/s 2024-06-10T03:04:37.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:37.336+0000 7f9f2a50d700 1 -- 172.21.15.52:0/363748780 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9f24101390 msgr2=0x7f9f24101810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:04:37.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:37.336+0000 7f9f2a50d700 1 --2- 172.21.15.52:0/363748780 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9f24101390 0x7f9f24101810 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f9f18009990 tx=0x7f9f18009ca0 comp rx=0 tx=0).stop 2024-06-10T03:04:37.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:37.336+0000 7f9f2a50d700 1 -- 172.21.15.52:0/363748780 shutdown_connections 2024-06-10T03:04:37.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:37.336+0000 7f9f2a50d700 1 --2- 172.21.15.52:0/363748780 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9f24101d50 0x7f9f2410e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:04:37.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:37.336+0000 7f9f2a50d700 1 --2- 172.21.15.52:0/363748780 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9f24107400 0x7f9f241077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:04:37.338 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:37.336+0000 7f9f2a50d700 1 --2- 172.21.15.52:0/363748780 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9f24101390 0x7f9f24101810 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:04:37.338 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:37.336+0000 7f9f2a50d700 1 -- 172.21.15.52:0/363748780 >> 172.21.15.52:0/363748780 conn(0x7f9f24078340 msgr2=0x7f9f24078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:04:37.338 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:37.336+0000 7f9f2a50d700 1 -- 172.21.15.52:0/363748780 shutdown_connections 2024-06-10T03:04:37.338 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:37.336+0000 7f9f2a50d700 1 -- 172.21.15.52:0/363748780 wait complete. 2024-06-10T03:04:37.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:37.337+0000 7f9f2a50d700 1 Processor -- start 2024-06-10T03:04:37.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:37.337+0000 7f9f2a50d700 1 -- start start 2024-06-10T03:04:37.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:37.338+0000 7f9f2a50d700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9f24101390 0x7f9f2419e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:04:37.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:37.338+0000 7f9f2a50d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9f24101d50 0x7f9f2419eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:04:37.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:37.338+0000 7f9f2a50d700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9f24107400 0x7f9f24198a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:04:37.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:37.338+0000 7f9f2a50d700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f9f24111670 con 0x7f9f24101d50 2024-06-10T03:04:37.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:37.338+0000 7f9f2a50d700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f9f241114f0 con 0x7f9f24107400 2024-06-10T03:04:37.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:37.338+0000 7f9f2a50d700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f9f241117f0 con 0x7f9f24101390 2024-06-10T03:04:37.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:37.338+0000 7f9f237fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9f24101d50 0x7f9f2419eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:04:37.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:37.338+0000 7f9f237fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9f24101d50 0x7f9f2419eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:41750/0 (socket says 172.21.15.52:41750) 2024-06-10T03:04:37.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:37.338+0000 7f9f237fe700 1 -- 172.21.15.52:0/3385117836 learned_addr learned my addr 172.21.15.52:0/3385117836 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:04:37.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:37.338+0000 7f9f28aaa700 1 --2- 172.21.15.52:0/3385117836 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9f24107400 0x7f9f24198a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:04:37.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:37.338+0000 7f9f23fff700 1 --2- 172.21.15.52:0/3385117836 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9f24101390 0x7f9f2419e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:04:37.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:37.338+0000 7f9f237fe700 1 -- 172.21.15.52:0/3385117836 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9f24101390 msgr2=0x7f9f2419e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:04:37.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:37.338+0000 7f9f237fe700 1 --2- 172.21.15.52:0/3385117836 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9f24101390 0x7f9f2419e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:04:37.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:37.338+0000 7f9f237fe700 1 -- 172.21.15.52:0/3385117836 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9f24107400 msgr2=0x7f9f24198a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:04:37.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:37.338+0000 7f9f237fe700 1 --2- 172.21.15.52:0/3385117836 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9f24107400 0x7f9f24198a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:04:37.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:37.338+0000 7f9f237fe700 1 -- 172.21.15.52:0/3385117836 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f241991d0 con 0x7f9f24101d50 2024-06-10T03:04:37.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:37.338+0000 7f9f28aaa700 1 --2- 172.21.15.52:0/3385117836 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9f24107400 0x7f9f24198a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T03:04:37.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:37.339+0000 7f9f237fe700 1 --2- 172.21.15.52:0/3385117836 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9f24101d50 0x7f9f2419eda0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f9f18000c00 tx=0x7f9f180047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:04:37.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:37.339+0000 7f9f217fa700 1 -- 172.21.15.52:0/3385117836 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f18027070 con 0x7f9f24101d50 2024-06-10T03:04:37.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:37.339+0000 7f9f2a50d700 1 -- 172.21.15.52:0/3385117836 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f24199460 con 0x7f9f24101d50 2024-06-10T03:04:37.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:37.339+0000 7f9f217fa700 1 -- 172.21.15.52:0/3385117836 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f18003e40 con 0x7f9f24101d50 2024-06-10T03:04:37.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:37.339+0000 7f9f217fa700 1 -- 172.21.15.52:0/3385117836 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f18022d10 con 0x7f9f24101d50 2024-06-10T03:04:37.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:37.339+0000 7f9f2a50d700 1 -- 172.21.15.52:0/3385117836 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f241a5730 con 0x7f9f24101d50 2024-06-10T03:04:37.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:37.340+0000 7f9f2a50d700 1 -- 172.21.15.52:0/3385117836 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f24102ab0 con 0x7f9f24101d50 2024-06-10T03:04:37.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:37.341+0000 7f9f217fa700 1 -- 172.21.15.52:0/3385117836 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f9f1802a070 con 0x7f9f24101d50 2024-06-10T03:04:37.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:37.342+0000 7f9f217fa700 1 --2- 172.21.15.52:0/3385117836 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f9f0c0a2c20 0x7f9f0c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:04:37.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:37.342+0000 7f9f23fff700 1 --2- 172.21.15.52:0/3385117836 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f9f0c0a2c20 0x7f9f0c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:04:37.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:37.342+0000 7f9f217fa700 1 -- 172.21.15.52:0/3385117836 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f9f180d01f0 con 0x7f9f24101d50 2024-06-10T03:04:37.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:37.342+0000 7f9f23fff700 1 --2- 172.21.15.52:0/3385117836 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f9f0c0a2c20 0x7f9f0c0a50e0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f9f14005950 tx=0x7f9f1400a300 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:04:37.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:37.347+0000 7f9f217fa700 1 -- 172.21.15.52:0/3385117836 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f9f1805e850 con 0x7f9f24101d50 2024-06-10T03:04:37.388 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:04:36 smithi120 ceph-mon[36634]: pgmap v746: 129 pgs: 129 active+clean; 228 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 8 op/s 2024-06-10T03:04:37.454 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:04:36 smithi188 ceph-mon[36599]: pgmap v746: 129 pgs: 129 active+clean; 228 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 8 op/s 2024-06-10T03:04:37.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:37.567+0000 7f9f2a50d700 1 -- 172.21.15.52:0/3385117836 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f9f241a5900 con 0x7f9f24101d50 2024-06-10T03:04:37.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:37.567+0000 7f9f217fa700 1 -- 172.21.15.52:0/3385117836 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v100) ==== 81+0+590 (secure 0 0 0) 0x7f9f18013070 con 0x7f9f24101d50 2024-06-10T03:04:37.568 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.15.52:0/3535271671 2024-06-11T02:42:42.872312+0000 2024-06-10T03:04:37.568 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.15.52:0/1092934380 2024-06-11T02:42:42.872312+0000 2024-06-10T03:04:37.568 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.15.52:6800/896317057 2024-06-11T02:42:42.872312+0000 2024-06-10T03:04:37.568 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.15.52:6801/3934479642 2024-06-11T02:42:15.498118+0000 2024-06-10T03:04:37.568 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.15.52:6800/3934479642 2024-06-11T02:42:15.498118+0000 2024-06-10T03:04:37.568 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.15.52:0/2759694132 2024-06-11T02:42:15.498118+0000 2024-06-10T03:04:37.568 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.15.52:0/2593665631 2024-06-11T02:42:15.498118+0000 2024-06-10T03:04:37.568 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.15.52:0/3151504634 2024-06-11T02:42:15.498118+0000 2024-06-10T03:04:37.568 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.15.52:6801/896317057 2024-06-11T02:42:42.872312+0000 2024-06-10T03:04:37.568 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.15.52:0/1254480982 2024-06-11T02:42:42.872312+0000 2024-06-10T03:04:37.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:37.570+0000 7f9f2a50d700 1 -- 172.21.15.52:0/3385117836 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f9f0c0a2c20 msgr2=0x7f9f0c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:04:37.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:37.570+0000 7f9f2a50d700 1 --2- 172.21.15.52:0/3385117836 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f9f0c0a2c20 0x7f9f0c0a50e0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f9f14005950 tx=0x7f9f1400a300 comp rx=0 tx=0).stop 2024-06-10T03:04:37.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:37.570+0000 7f9f2a50d700 1 -- 172.21.15.52:0/3385117836 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9f24101d50 msgr2=0x7f9f2419eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:04:37.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:37.570+0000 7f9f2a50d700 1 --2- 172.21.15.52:0/3385117836 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9f24101d50 0x7f9f2419eda0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f9f18000c00 tx=0x7f9f180047b0 comp rx=0 tx=0).stop 2024-06-10T03:04:37.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:37.570+0000 7f9f2a50d700 1 -- 172.21.15.52:0/3385117836 shutdown_connections 2024-06-10T03:04:37.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:37.570+0000 7f9f2a50d700 1 --2- 172.21.15.52:0/3385117836 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9f24107400 0x7f9f24198a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:04:37.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:37.570+0000 7f9f2a50d700 1 --2- 172.21.15.52:0/3385117836 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9f24101390 0x7f9f2419e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:04:37.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:37.570+0000 7f9f2a50d700 1 --2- 172.21.15.52:0/3385117836 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f9f0c0a2c20 0x7f9f0c0a50e0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:04:37.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:37.570+0000 7f9f2a50d700 1 --2- 172.21.15.52:0/3385117836 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9f24101d50 0x7f9f2419eda0 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:04:37.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:37.570+0000 7f9f2a50d700 1 -- 172.21.15.52:0/3385117836 >> 172.21.15.52:0/3385117836 conn(0x7f9f24078340 msgr2=0x7f9f24100670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:04:37.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:37.570+0000 7f9f2a50d700 1 -- 172.21.15.52:0/3385117836 shutdown_connections 2024-06-10T03:04:37.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:37.570+0000 7f9f2a50d700 1 -- 172.21.15.52:0/3385117836 wait complete. 2024-06-10T03:04:37.571 INFO:teuthology.orchestra.run.smithi052.stderr:listed 10 entries 2024-06-10T03:04:38.209 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:04:37 smithi052 ceph-mon[32362]: from='client.? 172.21.15.52:0/3385117836' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-06-10T03:04:38.236 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-06-10T03:04:38.237 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-10T03:04:38.237 DEBUG:teuthology.orchestra.run.smithi052:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-06-10T03:04:38.287 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi052.front.sepia.ceph.com 2024-06-10T03:04:38.287 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-10T03:04:38.288 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-06-10T03:04:38.329 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-10T03:04:38.329 DEBUG:teuthology.orchestra.run.smithi052:> ip netns list 2024-06-10T03:04:38.388 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:04:37 smithi120 ceph-mon[36634]: from='client.? 172.21.15.52:0/3385117836' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-06-10T03:04:38.400 INFO:teuthology.orchestra.run.smithi052.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-06-10T03:04:38.401 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-10T03:04:38.401 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-06-10T03:04:38.453 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:04:37 smithi188 ceph-mon[36599]: from='client.? 172.21.15.52:0/3385117836' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-06-10T03:04:38.489 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-10T03:04:38.489 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip link delete ceph-brx 2024-06-10T03:04:38.542 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-06-10T03:04:38.551 INFO:tasks.cephadm:Teardown begin 2024-06-10T03:04:38.551 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-06-10T03:04:38.552 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T03:04:38.802 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T03:04:39.170 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:04:38 smithi052 ceph-mon[32362]: pgmap v747: 129 pgs: 129 active+clean; 228 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 0 B/s wr, 11 op/s 2024-06-10T03:04:39.388 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:04:38 smithi120 ceph-mon[36634]: pgmap v747: 129 pgs: 129 active+clean; 228 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 0 B/s wr, 11 op/s 2024-06-10T03:04:39.454 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:04:38 smithi188 ceph-mon[36599]: pgmap v747: 129 pgs: 129 active+clean; 228 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 0 B/s wr, 11 op/s 2024-06-10T03:04:40.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:40.446+0000 7fd1c690e700 1 -- 172.21.15.52:0/2270260220 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd1c0101fc0 msgr2=0x7fd1c0106010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:04:40.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:40.446+0000 7fd1c690e700 1 --2- 172.21.15.52:0/2270260220 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd1c0101fc0 0x7fd1c0106010 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fd1a80099c0 tx=0x7fd1a8009cd0 comp rx=0 tx=0).stop 2024-06-10T03:04:40.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:40.446+0000 7fd1c690e700 1 -- 172.21.15.52:0/2270260220 shutdown_connections 2024-06-10T03:04:40.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:40.446+0000 7fd1c690e700 1 --2- 172.21.15.52:0/2270260220 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd1c01066f0 0x7fd1c010e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:04:40.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:40.446+0000 7fd1c690e700 1 --2- 172.21.15.52:0/2270260220 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd1c0101610 0x7fd1c01019f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:04:40.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:40.446+0000 7fd1c690e700 1 --2- 172.21.15.52:0/2270260220 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd1c0101fc0 0x7fd1c0106010 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:04:40.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:40.446+0000 7fd1c690e700 1 -- 172.21.15.52:0/2270260220 >> 172.21.15.52:0/2270260220 conn(0x7fd1c0078430 msgr2=0x7fd1c0078840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:04:40.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:40.446+0000 7fd1c690e700 1 -- 172.21.15.52:0/2270260220 shutdown_connections 2024-06-10T03:04:40.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:40.446+0000 7fd1c690e700 1 -- 172.21.15.52:0/2270260220 wait complete. 2024-06-10T03:04:40.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:40.446+0000 7fd1c690e700 1 Processor -- start 2024-06-10T03:04:40.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:40.446+0000 7fd1c690e700 1 -- start start 2024-06-10T03:04:40.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:40.447+0000 7fd1c690e700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd1c0101610 0x7fd1c01030b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:04:40.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:40.447+0000 7fd1c690e700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd1c0101fc0 0x7fd1c01035f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:04:40.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:40.447+0000 7fd1c690e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd1c01066f0 0x7fd1c0103b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:04:40.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:40.447+0000 7fd1c690e700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fd1c01116b0 con 0x7fd1c01066f0 2024-06-10T03:04:40.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:40.447+0000 7fd1c690e700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd1c0111530 con 0x7fd1c0101610 2024-06-10T03:04:40.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:40.447+0000 7fd1c690e700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fd1c0111830 con 0x7fd1c0101fc0 2024-06-10T03:04:40.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:40.447+0000 7fd1c4eab700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd1c01066f0 0x7fd1c0103b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:04:40.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:40.447+0000 7fd1c4eab700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd1c01066f0 0x7fd1c0103b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:41762/0 (socket says 172.21.15.52:41762) 2024-06-10T03:04:40.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:40.447+0000 7fd1c4eab700 1 -- 172.21.15.52:0/1573785662 learned_addr learned my addr 172.21.15.52:0/1573785662 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:04:40.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:40.447+0000 7fd1bffff700 1 --2- 172.21.15.52:0/1573785662 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd1c0101610 0x7fd1c01030b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:04:40.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:40.447+0000 7fd1b7fff700 1 --2- 172.21.15.52:0/1573785662 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd1c0101fc0 0x7fd1c01035f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:04:40.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:40.447+0000 7fd1c4eab700 1 -- 172.21.15.52:0/1573785662 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd1c0101fc0 msgr2=0x7fd1c01035f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:04:40.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:40.447+0000 7fd1c4eab700 1 --2- 172.21.15.52:0/1573785662 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd1c0101fc0 0x7fd1c01035f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:04:40.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:40.447+0000 7fd1c4eab700 1 -- 172.21.15.52:0/1573785662 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd1c0101610 msgr2=0x7fd1c01030b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:04:40.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:40.447+0000 7fd1c4eab700 1 --2- 172.21.15.52:0/1573785662 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd1c0101610 0x7fd1c01030b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:04:40.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:40.447+0000 7fd1c4eab700 1 -- 172.21.15.52:0/1573785662 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1c01a7100 con 0x7fd1c01066f0 2024-06-10T03:04:40.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:40.447+0000 7fd1b7fff700 1 --2- 172.21.15.52:0/1573785662 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd1c0101fc0 0x7fd1c01035f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T03:04:40.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:40.447+0000 7fd1bffff700 1 --2- 172.21.15.52:0/1573785662 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd1c0101610 0x7fd1c01030b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T03:04:40.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:40.447+0000 7fd1c4eab700 1 --2- 172.21.15.52:0/1573785662 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd1c01066f0 0x7fd1c0103b30 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fd1b8007d70 tx=0x7fd1b800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:04:40.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:40.448+0000 7fd1bdffb700 1 -- 172.21.15.52:0/1573785662 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1b8016020 con 0x7fd1c01066f0 2024-06-10T03:04:40.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:40.448+0000 7fd1bdffb700 1 -- 172.21.15.52:0/1573785662 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1b800ad40 con 0x7fd1c01066f0 2024-06-10T03:04:40.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:40.448+0000 7fd1c690e700 1 -- 172.21.15.52:0/1573785662 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1c01a73f0 con 0x7fd1c01066f0 2024-06-10T03:04:40.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:40.448+0000 7fd1bdffb700 1 -- 172.21.15.52:0/1573785662 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1b8011600 con 0x7fd1c01066f0 2024-06-10T03:04:40.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:40.448+0000 7fd1c690e700 1 -- 172.21.15.52:0/1573785662 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1c01a78b0 con 0x7fd1c01066f0 2024-06-10T03:04:40.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:40.449+0000 7fd1c690e700 1 -- 172.21.15.52:0/1573785662 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1c0069180 con 0x7fd1c01066f0 2024-06-10T03:04:40.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:40.450+0000 7fd1bdffb700 1 -- 172.21.15.52:0/1573785662 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fd1b8019020 con 0x7fd1c01066f0 2024-06-10T03:04:40.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:40.451+0000 7fd1bdffb700 1 --2- 172.21.15.52:0/1573785662 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fd1ac0a2c20 0x7fd1ac0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:04:40.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:40.451+0000 7fd1bffff700 1 --2- 172.21.15.52:0/1573785662 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fd1ac0a2c20 0x7fd1ac0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:04:40.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:40.451+0000 7fd1bdffb700 1 -- 172.21.15.52:0/1573785662 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7fd1b80cf760 con 0x7fd1c01066f0 2024-06-10T03:04:40.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:40.451+0000 7fd1bffff700 1 --2- 172.21.15.52:0/1573785662 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fd1ac0a2c20 0x7fd1ac0a50e0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fd1b0005950 tx=0x7fd1b000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:04:40.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:40.456+0000 7fd1bdffb700 1 -- 172.21.15.52:0/1573785662 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fd1b8020020 con 0x7fd1c01066f0 2024-06-10T03:04:40.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:40.709+0000 7fd1c690e700 1 -- 172.21.15.52:0/1573785662 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd1c002dca0 con 0x7fd1c01066f0 2024-06-10T03:04:40.710 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:40.710+0000 7fd1bdffb700 1 -- 172.21.15.52:0/1573785662 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) ==== 94+0+11948 (secure 0 0 0) 0x7fd1b809c310 con 0x7fd1c01066f0 2024-06-10T03:04:40.710 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-10T03:04:40.710 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":49,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/889673247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":889673247},{"type":"v1","addr":"172.21.15.52:6835","nonce":889673247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/3615663511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":3615663511},{"type":"v1","addr":"172.21.15.52:6837","nonce":3615663511}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":49,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T02:50:03.889766+0000","modified":"2024-06-10T03:04:33.373598+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24484,"mds_1":24469,"mds_2":24463,"mds_3":24449,"mds_4":24428},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14571":{"gid":14571,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6839/3059599042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":3059599042},{"type":"v1","addr":"172.21.15.52:6839","nonce":3059599042}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14583":{"gid":14583,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6841/3039275521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":3039275521},{"type":"v1","addr":"172.21.15.52:6841","nonce":3039275521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24383":{"gid":24383,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6833/2609002896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":2609002896},{"type":"v1","addr":"172.21.15.120:6833","nonce":2609002896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6835/1996370288","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":1996370288},{"type":"v1","addr":"172.21.15.120:6835","nonce":1996370288}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6833/529564979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":529564979},{"type":"v1","addr":"172.21.15.188:6833","nonce":529564979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":25,"addr":"172.21.15.120:6837/4132440952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":4132440952},{"type":"v1","addr":"172.21.15.120:6837","nonce":4132440952}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"b","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.120:6839/1672436487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1672436487},{"type":"v1","addr":"172.21.15.120:6839","nonce":1672436487}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":2,"incarnation":22,"state":"up:active","state_seq":26,"addr":"172.21.15.188:6835/3904544259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":3904544259},{"type":"v1","addr":"172.21.15.188:6835","nonce":3904544259}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.188:6837/819317726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":819317726},{"type":"v1","addr":"172.21.15.188:6837","nonce":819317726}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.188:6839/310444836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":310444836},{"type":"v1","addr":"172.21.15.188:6839","nonce":310444836}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24484,"qdb_cluster":[24469,24449,24428,24463,24484]},"id":1}]} 2024-06-10T03:04:40.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:40.712+0000 7fd1c690e700 1 -- 172.21.15.52:0/1573785662 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fd1ac0a2c20 msgr2=0x7fd1ac0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:04:40.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:40.712+0000 7fd1c690e700 1 --2- 172.21.15.52:0/1573785662 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fd1ac0a2c20 0x7fd1ac0a50e0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fd1b0005950 tx=0x7fd1b000b410 comp rx=0 tx=0).stop 2024-06-10T03:04:40.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:40.712+0000 7fd1c690e700 1 -- 172.21.15.52:0/1573785662 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd1c01066f0 msgr2=0x7fd1c0103b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:04:40.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:40.712+0000 7fd1c690e700 1 --2- 172.21.15.52:0/1573785662 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd1c01066f0 0x7fd1c0103b30 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fd1b8007d70 tx=0x7fd1b800a340 comp rx=0 tx=0).stop 2024-06-10T03:04:40.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:40.712+0000 7fd1c690e700 1 -- 172.21.15.52:0/1573785662 shutdown_connections 2024-06-10T03:04:40.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:40.712+0000 7fd1c690e700 1 --2- 172.21.15.52:0/1573785662 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fd1ac0a2c20 0x7fd1ac0a50e0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:04:40.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:40.712+0000 7fd1c690e700 1 --2- 172.21.15.52:0/1573785662 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd1c01066f0 0x7fd1c0103b30 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:04:40.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:40.712+0000 7fd1c690e700 1 --2- 172.21.15.52:0/1573785662 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd1c0101610 0x7fd1c01030b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:04:40.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:40.712+0000 7fd1c690e700 1 --2- 172.21.15.52:0/1573785662 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd1c0101fc0 0x7fd1c01035f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:04:40.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:40.712+0000 7fd1c690e700 1 -- 172.21.15.52:0/1573785662 >> 172.21.15.52:0/1573785662 conn(0x7fd1c0078430 msgr2=0x7fd1c006ada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:04:40.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:40.712+0000 7fd1c690e700 1 -- 172.21.15.52:0/1573785662 shutdown_connections 2024-06-10T03:04:40.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:40.712+0000 7fd1c690e700 1 -- 172.21.15.52:0/1573785662 wait complete. 2024-06-10T03:04:40.714 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 49 2024-06-10T03:04:41.389 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:04:40 smithi120 ceph-mon[36634]: pgmap v748: 129 pgs: 129 active+clean; 228 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-10T03:04:41.390 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:04:40 smithi120 ceph-mon[36634]: from='client.? 172.21.15.52:0/1573785662' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T03:04:41.454 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:04:40 smithi188 ceph-mon[36599]: pgmap v748: 129 pgs: 129 active+clean; 228 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-10T03:04:41.454 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:04:40 smithi188 ceph-mon[36599]: from='client.? 172.21.15.52:0/1573785662' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T03:04:41.477 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:04:40 smithi052 ceph-mon[32362]: pgmap v748: 129 pgs: 129 active+clean; 228 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-10T03:04:41.477 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:04:40 smithi052 ceph-mon[32362]: from='client.? 172.21.15.52:0/1573785662' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T03:04:41.554 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T03:04:41.814 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T03:04:43.355 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:04:42 smithi052 ceph-mon[32362]: pgmap v749: 129 pgs: 129 active+clean; 228 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 8 op/s 2024-06-10T03:04:43.388 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:04:42 smithi120 ceph-mon[36634]: pgmap v749: 129 pgs: 129 active+clean; 228 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 8 op/s 2024-06-10T03:04:43.454 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:04:42 smithi188 ceph-mon[36599]: pgmap v749: 129 pgs: 129 active+clean; 228 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 8 op/s 2024-06-10T03:04:43.758 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:43.756+0000 7f5506ed0700 1 -- 172.21.15.52:0/3706149446 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5500102ea0 msgr2=0x7f550010f380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:04:43.758 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:43.756+0000 7f5506ed0700 1 --2- 172.21.15.52:0/3706149446 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5500102ea0 0x7f550010f380 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f54f800b0a0 tx=0x7f54f800b3b0 comp rx=0 tx=0).stop 2024-06-10T03:04:43.758 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:43.756+0000 7f5506ed0700 1 -- 172.21.15.52:0/3706149446 shutdown_connections 2024-06-10T03:04:43.759 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:43.756+0000 7f5506ed0700 1 --2- 172.21.15.52:0/3706149446 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5500102ea0 0x7f550010f380 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:04:43.759 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:43.756+0000 7f5506ed0700 1 --2- 172.21.15.52:0/3706149446 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f55001085f0 0x7f55001089d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:04:43.759 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:43.756+0000 7f5506ed0700 1 --2- 172.21.15.52:0/3706149446 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f55001024e0 0x7f5500102960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:04:43.759 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:43.756+0000 7f5506ed0700 1 -- 172.21.15.52:0/3706149446 >> 172.21.15.52:0/3706149446 conn(0x7f5500079640 msgr2=0x7f5500079a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:04:43.759 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:43.756+0000 7f5506ed0700 1 -- 172.21.15.52:0/3706149446 shutdown_connections 2024-06-10T03:04:43.759 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:43.756+0000 7f5506ed0700 1 -- 172.21.15.52:0/3706149446 wait complete. 2024-06-10T03:04:43.759 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:43.757+0000 7f5506ed0700 1 Processor -- start 2024-06-10T03:04:43.759 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:43.757+0000 7f5506ed0700 1 -- start start 2024-06-10T03:04:43.759 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:43.757+0000 7f5506ed0700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f55001024e0 0x7f550019e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:04:43.759 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:43.757+0000 7f5506ed0700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5500102ea0 0x7f550019ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:04:43.759 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:43.757+0000 7f5506ed0700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f55001085f0 0x7f5500198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:04:43.759 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:43.757+0000 7f5506ed0700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f55001127b0 con 0x7f55001085f0 2024-06-10T03:04:43.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:43.757+0000 7f5506ed0700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f5500112630 con 0x7f55001024e0 2024-06-10T03:04:43.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:43.757+0000 7f5506ed0700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f5500112930 con 0x7f5500102ea0 2024-06-10T03:04:43.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:43.757+0000 7f550546d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f55001085f0 0x7f5500198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:04:43.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:43.757+0000 7f550546d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f55001085f0 0x7f5500198930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:39174/0 (socket says 172.21.15.52:39174) 2024-06-10T03:04:43.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:43.757+0000 7f550546d700 1 -- 172.21.15.52:0/2027747356 learned_addr learned my addr 172.21.15.52:0/2027747356 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:04:43.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:43.757+0000 7f5504c6c700 1 --2- 172.21.15.52:0/2027747356 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f55001024e0 0x7f550019e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:04:43.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:43.757+0000 7f54fffff700 1 --2- 172.21.15.52:0/2027747356 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5500102ea0 0x7f550019ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:04:43.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:43.757+0000 7f550546d700 1 -- 172.21.15.52:0/2027747356 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5500102ea0 msgr2=0x7f550019ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:04:43.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:43.757+0000 7f550546d700 1 --2- 172.21.15.52:0/2027747356 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5500102ea0 0x7f550019ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:04:43.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:43.757+0000 7f550546d700 1 -- 172.21.15.52:0/2027747356 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f55001024e0 msgr2=0x7f550019e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:04:43.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:43.757+0000 7f550546d700 1 --2- 172.21.15.52:0/2027747356 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f55001024e0 0x7f550019e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:04:43.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:43.757+0000 7f550546d700 1 -- 172.21.15.52:0/2027747356 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5500199090 con 0x7f55001085f0 2024-06-10T03:04:43.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:43.758+0000 7f550546d700 1 --2- 172.21.15.52:0/2027747356 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f55001085f0 0x7f5500198930 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f54f80062a0 tx=0x7f54f8004e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:04:43.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:43.758+0000 7f54fdffb700 1 -- 172.21.15.52:0/2027747356 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54f8021070 con 0x7f55001085f0 2024-06-10T03:04:43.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:43.758+0000 7f54fdffb700 1 -- 172.21.15.52:0/2027747356 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54f8007c00 con 0x7f55001085f0 2024-06-10T03:04:43.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:43.758+0000 7f5506ed0700 1 -- 172.21.15.52:0/2027747356 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5500199320 con 0x7f55001085f0 2024-06-10T03:04:43.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:43.758+0000 7f54fdffb700 1 -- 172.21.15.52:0/2027747356 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54f80052a0 con 0x7f55001085f0 2024-06-10T03:04:43.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:43.758+0000 7f5506ed0700 1 -- 172.21.15.52:0/2027747356 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55001a55f0 con 0x7f55001085f0 2024-06-10T03:04:43.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:43.759+0000 7f5506ed0700 1 -- 172.21.15.52:0/2027747356 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5500069180 con 0x7f55001085f0 2024-06-10T03:04:43.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:43.760+0000 7f54fdffb700 1 -- 172.21.15.52:0/2027747356 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f54f8024070 con 0x7f55001085f0 2024-06-10T03:04:43.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:43.761+0000 7f54fdffb700 1 --2- 172.21.15.52:0/2027747356 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f54f00a2c20 0x7f54f00a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:04:43.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:43.761+0000 7f5504c6c700 1 --2- 172.21.15.52:0/2027747356 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f54f00a2c20 0x7f54f00a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:04:43.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:43.761+0000 7f54fdffb700 1 -- 172.21.15.52:0/2027747356 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f54f800f020 con 0x7f55001085f0 2024-06-10T03:04:43.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:43.761+0000 7f5504c6c700 1 --2- 172.21.15.52:0/2027747356 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f54f00a2c20 0x7f54f00a50e0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f5500108dc0 tx=0x7f54f400a380 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:04:43.766 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:43.766+0000 7f54fdffb700 1 -- 172.21.15.52:0/2027747356 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f54f802b020 con 0x7f55001085f0 2024-06-10T03:04:44.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:44.019+0000 7f5506ed0700 1 -- 172.21.15.52:0/2027747356 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f55001a57c0 con 0x7f55001085f0 2024-06-10T03:04:44.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:44.020+0000 7f54fdffb700 1 -- 172.21.15.52:0/2027747356 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) ==== 94+0+11948 (secure 0 0 0) 0x7f54f8007db0 con 0x7f55001085f0 2024-06-10T03:04:44.020 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-10T03:04:44.020 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":49,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/889673247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":889673247},{"type":"v1","addr":"172.21.15.52:6835","nonce":889673247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/3615663511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":3615663511},{"type":"v1","addr":"172.21.15.52:6837","nonce":3615663511}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":49,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T02:50:03.889766+0000","modified":"2024-06-10T03:04:33.373598+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24484,"mds_1":24469,"mds_2":24463,"mds_3":24449,"mds_4":24428},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14571":{"gid":14571,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6839/3059599042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":3059599042},{"type":"v1","addr":"172.21.15.52:6839","nonce":3059599042}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14583":{"gid":14583,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6841/3039275521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":3039275521},{"type":"v1","addr":"172.21.15.52:6841","nonce":3039275521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24383":{"gid":24383,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6833/2609002896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":2609002896},{"type":"v1","addr":"172.21.15.120:6833","nonce":2609002896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6835/1996370288","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":1996370288},{"type":"v1","addr":"172.21.15.120:6835","nonce":1996370288}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6833/529564979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":529564979},{"type":"v1","addr":"172.21.15.188:6833","nonce":529564979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":25,"addr":"172.21.15.120:6837/4132440952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":4132440952},{"type":"v1","addr":"172.21.15.120:6837","nonce":4132440952}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"b","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.120:6839/1672436487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1672436487},{"type":"v1","addr":"172.21.15.120:6839","nonce":1672436487}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":2,"incarnation":22,"state":"up:active","state_seq":26,"addr":"172.21.15.188:6835/3904544259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":3904544259},{"type":"v1","addr":"172.21.15.188:6835","nonce":3904544259}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.188:6837/819317726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":819317726},{"type":"v1","addr":"172.21.15.188:6837","nonce":819317726}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.188:6839/310444836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":310444836},{"type":"v1","addr":"172.21.15.188:6839","nonce":310444836}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24484,"qdb_cluster":[24469,24449,24428,24463,24484]},"id":1}]} 2024-06-10T03:04:44.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:44.022+0000 7f5506ed0700 1 -- 172.21.15.52:0/2027747356 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f54f00a2c20 msgr2=0x7f54f00a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:04:44.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:44.022+0000 7f5506ed0700 1 --2- 172.21.15.52:0/2027747356 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f54f00a2c20 0x7f54f00a50e0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f5500108dc0 tx=0x7f54f400a380 comp rx=0 tx=0).stop 2024-06-10T03:04:44.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:44.022+0000 7f5506ed0700 1 -- 172.21.15.52:0/2027747356 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f55001085f0 msgr2=0x7f5500198930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:04:44.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:44.022+0000 7f5506ed0700 1 --2- 172.21.15.52:0/2027747356 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f55001085f0 0x7f5500198930 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f54f80062a0 tx=0x7f54f8004e40 comp rx=0 tx=0).stop 2024-06-10T03:04:44.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:44.022+0000 7f5506ed0700 1 -- 172.21.15.52:0/2027747356 shutdown_connections 2024-06-10T03:04:44.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:44.022+0000 7f5506ed0700 1 --2- 172.21.15.52:0/2027747356 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f54f00a2c20 0x7f54f00a50e0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:04:44.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:44.022+0000 7f5506ed0700 1 --2- 172.21.15.52:0/2027747356 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f55001085f0 0x7f5500198930 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:04:44.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:44.022+0000 7f5506ed0700 1 --2- 172.21.15.52:0/2027747356 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f55001024e0 0x7f550019e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:04:44.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:44.022+0000 7f5506ed0700 1 --2- 172.21.15.52:0/2027747356 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5500102ea0 0x7f550019ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:04:44.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:44.022+0000 7f5506ed0700 1 -- 172.21.15.52:0/2027747356 >> 172.21.15.52:0/2027747356 conn(0x7f5500079640 msgr2=0x7f5500100db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:04:44.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:44.022+0000 7f5506ed0700 1 -- 172.21.15.52:0/2027747356 shutdown_connections 2024-06-10T03:04:44.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:44.022+0000 7f5506ed0700 1 -- 172.21.15.52:0/2027747356 wait complete. 2024-06-10T03:04:44.024 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 49 2024-06-10T03:04:44.766 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-10T03:04:45.023 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T03:04:45.261 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:04:44 smithi052 ceph-mon[32362]: pgmap v750: 129 pgs: 129 active+clean; 228 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.2 KiB/s wr, 13 op/s 2024-06-10T03:04:45.261 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:04:44 smithi052 ceph-mon[32362]: from='client.? 172.21.15.52:0/2027747356' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T03:04:45.388 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:04:44 smithi120 ceph-mon[36634]: pgmap v750: 129 pgs: 129 active+clean; 228 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.2 KiB/s wr, 13 op/s 2024-06-10T03:04:45.389 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:04:44 smithi120 ceph-mon[36634]: from='client.? 172.21.15.52:0/2027747356' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T03:04:45.454 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:04:44 smithi188 ceph-mon[36599]: pgmap v750: 129 pgs: 129 active+clean; 228 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.2 KiB/s wr, 13 op/s 2024-06-10T03:04:45.454 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:04:44 smithi188 ceph-mon[36599]: from='client.? 172.21.15.52:0/2027747356' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T03:04:46.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:46.825+0000 7fa48aa8c700 1 -- 172.21.15.52:0/1766513024 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa48406b0a0 msgr2=0x7fa48406b500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:04:46.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:46.825+0000 7fa48aa8c700 1 --2- 172.21.15.52:0/1766513024 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa48406b0a0 0x7fa48406b500 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fa470009990 tx=0x7fa470009ca0 comp rx=0 tx=0).stop 2024-06-10T03:04:46.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:46.825+0000 7fa48aa8c700 1 -- 172.21.15.52:0/1766513024 shutdown_connections 2024-06-10T03:04:46.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:46.825+0000 7fa48aa8c700 1 --2- 172.21.15.52:0/1766513024 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa48410ede0 0x7fa4841111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:04:46.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:46.825+0000 7fa48aa8c700 1 --2- 172.21.15.52:0/1766513024 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa4841033a0 0x7fa48406ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:04:46.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:46.825+0000 7fa48aa8c700 1 --2- 172.21.15.52:0/1766513024 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa48406b0a0 0x7fa48406b500 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:04:46.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:46.825+0000 7fa48aa8c700 1 -- 172.21.15.52:0/1766513024 >> 172.21.15.52:0/1766513024 conn(0x7fa484078340 msgr2=0x7fa484078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:04:46.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:46.825+0000 7fa48aa8c700 1 -- 172.21.15.52:0/1766513024 shutdown_connections 2024-06-10T03:04:46.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:46.825+0000 7fa48aa8c700 1 -- 172.21.15.52:0/1766513024 wait complete. 2024-06-10T03:04:46.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:46.826+0000 7fa48aa8c700 1 Processor -- start 2024-06-10T03:04:46.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:46.826+0000 7fa48aa8c700 1 -- start start 2024-06-10T03:04:46.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:46.826+0000 7fa48aa8c700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa48406b0a0 0x7fa48419aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:04:46.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:46.826+0000 7fa48aa8c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa4841033a0 0x7fa48419b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:04:46.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:46.826+0000 7fa48aa8c700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa48410ede0 0x7fa48419f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:04:46.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:46.826+0000 7fa48aa8c700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fa484114a10 con 0x7fa4841033a0 2024-06-10T03:04:46.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:46.826+0000 7fa48aa8c700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa484114890 con 0x7fa48410ede0 2024-06-10T03:04:46.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:46.826+0000 7fa48aa8c700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa484114b90 con 0x7fa48406b0a0 2024-06-10T03:04:46.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:46.826+0000 7fa483fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa4841033a0 0x7fa48419b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:04:46.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:46.827+0000 7fa488828700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa48406b0a0 0x7fa48419aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:04:46.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:46.827+0000 7fa483fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa4841033a0 0x7fa48419b3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:39188/0 (socket says 172.21.15.52:39188) 2024-06-10T03:04:46.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:46.827+0000 7fa483fff700 1 -- 172.21.15.52:0/1896691606 learned_addr learned my addr 172.21.15.52:0/1896691606 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:04:46.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:46.827+0000 7fa489029700 1 --2- 172.21.15.52:0/1896691606 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa48410ede0 0x7fa48419f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:04:46.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:46.827+0000 7fa483fff700 1 -- 172.21.15.52:0/1896691606 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa48406b0a0 msgr2=0x7fa48419aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:04:46.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:46.827+0000 7fa483fff700 1 --2- 172.21.15.52:0/1896691606 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa48406b0a0 0x7fa48419aea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:04:46.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:46.827+0000 7fa483fff700 1 -- 172.21.15.52:0/1896691606 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa48410ede0 msgr2=0x7fa48419f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:04:46.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:46.827+0000 7fa483fff700 1 --2- 172.21.15.52:0/1896691606 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa48410ede0 0x7fa48419f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:04:46.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:46.827+0000 7fa483fff700 1 -- 172.21.15.52:0/1896691606 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa48419feb0 con 0x7fa4841033a0 2024-06-10T03:04:46.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:46.827+0000 7fa483fff700 1 --2- 172.21.15.52:0/1896691606 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa4841033a0 0x7fa48419b3e0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fa470000c00 tx=0x7fa470003890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:04:46.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:46.827+0000 7fa481ffb700 1 -- 172.21.15.52:0/1896691606 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa470027070 con 0x7fa4841033a0 2024-06-10T03:04:46.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:46.827+0000 7fa481ffb700 1 -- 172.21.15.52:0/1896691606 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa470003e40 con 0x7fa4841033a0 2024-06-10T03:04:46.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:46.827+0000 7fa48aa8c700 1 -- 172.21.15.52:0/1896691606 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4841a0140 con 0x7fa4841033a0 2024-06-10T03:04:46.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:46.827+0000 7fa481ffb700 1 -- 172.21.15.52:0/1896691606 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa470022c90 con 0x7fa4841033a0 2024-06-10T03:04:46.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:46.828+0000 7fa48aa8c700 1 -- 172.21.15.52:0/1896691606 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4841a7a20 con 0x7fa4841033a0 2024-06-10T03:04:46.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:46.829+0000 7fa48aa8c700 1 -- 172.21.15.52:0/1896691606 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4841019d0 con 0x7fa4841033a0 2024-06-10T03:04:46.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:46.830+0000 7fa481ffb700 1 -- 172.21.15.52:0/1896691606 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fa47002a070 con 0x7fa4841033a0 2024-06-10T03:04:46.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:46.830+0000 7fa481ffb700 1 --2- 172.21.15.52:0/1896691606 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fa4740a2ca0 0x7fa4740a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:04:46.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:46.831+0000 7fa488828700 1 --2- 172.21.15.52:0/1896691606 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fa4740a2ca0 0x7fa4740a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:04:46.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:46.831+0000 7fa481ffb700 1 -- 172.21.15.52:0/1896691606 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7fa4700d0160 con 0x7fa4841033a0 2024-06-10T03:04:46.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:46.831+0000 7fa488828700 1 --2- 172.21.15.52:0/1896691606 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fa4740a2ca0 0x7fa4740a5160 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fa478005950 tx=0x7fa47800a300 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:04:46.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:46.835+0000 7fa481ffb700 1 -- 172.21.15.52:0/1896691606 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fa47005e850 con 0x7fa4841033a0 2024-06-10T03:04:47.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:47.042+0000 7fa48aa8c700 1 -- 172.21.15.52:0/1896691606 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa4841a7bf0 con 0x7fa4841033a0 2024-06-10T03:04:47.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:47.044+0000 7fa481ffb700 1 -- 172.21.15.52:0/1896691606 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v100) ==== 74+0+24739 (secure 0 0 0) 0x7fa470013070 con 0x7fa4841033a0 2024-06-10T03:04:47.044 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-10T03:04:47.044 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":100,"fsid":"d8676e9e-26d2-11ef-bc9b-c7b262605968","created":"2024-06-10T02:41:43.076837+0000","modified":"2024-06-10T02:51:45.708105+0000","last_up_change":"2024-06-10T02:48:57.297829+0000","last_in_change":"2024-06-10T02:48:36.865802+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-10T02:45:07.394690+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-10T02:49:57.926838+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-10T02:50:01.130393+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-10T02:50:10.655588+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e03c0691-cc5a-497a-bbc2-ce18c337f074","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":4255833408},{"type":"v1","addr":"172.21.15.52:6803","nonce":4255833408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":4255833408},{"type":"v1","addr":"172.21.15.52:6805","nonce":4255833408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":4255833408},{"type":"v1","addr":"172.21.15.52:6809","nonce":4255833408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":4255833408},{"type":"v1","addr":"172.21.15.52:6807","nonce":4255833408}]},"public_addr":"172.21.15.52:6803/4255833408","cluster_addr":"172.21.15.52:6805/4255833408","heartbeat_back_addr":"172.21.15.52:6809/4255833408","heartbeat_front_addr":"172.21.15.52:6807/4255833408","state":["exists","up"]},{"osd":1,"uuid":"a24d8da8-c5ec-44d7-ab30-56f6ce3efda7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":4116540053},{"type":"v1","addr":"172.21.15.120:6801","nonce":4116540053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":4116540053},{"type":"v1","addr":"172.21.15.120:6803","nonce":4116540053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":4116540053},{"type":"v1","addr":"172.21.15.120:6807","nonce":4116540053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":4116540053},{"type":"v1","addr":"172.21.15.120:6805","nonce":4116540053}]},"public_addr":"172.21.15.120:6801/4116540053","cluster_addr":"172.21.15.120:6803/4116540053","heartbeat_back_addr":"172.21.15.120:6807/4116540053","heartbeat_front_addr":"172.21.15.120:6805/4116540053","state":["exists","up"]},{"osd":2,"uuid":"a7162106-ccbf-45bd-bc0e-051ee43f4214","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":2624326999},{"type":"v1","addr":"172.21.15.188:6801","nonce":2624326999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":2624326999},{"type":"v1","addr":"172.21.15.188:6803","nonce":2624326999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":2624326999},{"type":"v1","addr":"172.21.15.188:6807","nonce":2624326999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":2624326999},{"type":"v1","addr":"172.21.15.188:6805","nonce":2624326999}]},"public_addr":"172.21.15.188:6801/2624326999","cluster_addr":"172.21.15.188:6803/2624326999","heartbeat_back_addr":"172.21.15.188:6807/2624326999","heartbeat_front_addr":"172.21.15.188:6805/2624326999","state":["exists","up"]},{"osd":3,"uuid":"53680244-b49a-4120-b24a-9ad7a6341dc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":116897011},{"type":"v1","addr":"172.21.15.52:6811","nonce":116897011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":116897011},{"type":"v1","addr":"172.21.15.52:6813","nonce":116897011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":116897011},{"type":"v1","addr":"172.21.15.52:6817","nonce":116897011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":116897011},{"type":"v1","addr":"172.21.15.52:6815","nonce":116897011}]},"public_addr":"172.21.15.52:6811/116897011","cluster_addr":"172.21.15.52:6813/116897011","heartbeat_back_addr":"172.21.15.52:6817/116897011","heartbeat_front_addr":"172.21.15.52:6815/116897011","state":["exists","up"]},{"osd":4,"uuid":"9f72f1f4-2ef3-49c5-8410-c108767d2ef5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":1320690621},{"type":"v1","addr":"172.21.15.120:6809","nonce":1320690621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":1320690621},{"type":"v1","addr":"172.21.15.120:6811","nonce":1320690621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":1320690621},{"type":"v1","addr":"172.21.15.120:6815","nonce":1320690621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":1320690621},{"type":"v1","addr":"172.21.15.120:6813","nonce":1320690621}]},"public_addr":"172.21.15.120:6809/1320690621","cluster_addr":"172.21.15.120:6811/1320690621","heartbeat_back_addr":"172.21.15.120:6815/1320690621","heartbeat_front_addr":"172.21.15.120:6813/1320690621","state":["exists","up"]},{"osd":5,"uuid":"001b5706-7f27-4fa4-9b69-969e6d765abc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3951947483},{"type":"v1","addr":"172.21.15.188:6809","nonce":3951947483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3951947483},{"type":"v1","addr":"172.21.15.188:6811","nonce":3951947483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3951947483},{"type":"v1","addr":"172.21.15.188:6815","nonce":3951947483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3951947483},{"type":"v1","addr":"172.21.15.188:6813","nonce":3951947483}]},"public_addr":"172.21.15.188:6809/3951947483","cluster_addr":"172.21.15.188:6811/3951947483","heartbeat_back_addr":"172.21.15.188:6815/3951947483","heartbeat_front_addr":"172.21.15.188:6813/3951947483","state":["exists","up"]},{"osd":6,"uuid":"140c903e-9b31-4a81-9ccd-2edc0a2decf9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":359084919},{"type":"v1","addr":"172.21.15.52:6819","nonce":359084919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":359084919},{"type":"v1","addr":"172.21.15.52:6821","nonce":359084919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":359084919},{"type":"v1","addr":"172.21.15.52:6825","nonce":359084919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":359084919},{"type":"v1","addr":"172.21.15.52:6823","nonce":359084919}]},"public_addr":"172.21.15.52:6819/359084919","cluster_addr":"172.21.15.52:6821/359084919","heartbeat_back_addr":"172.21.15.52:6825/359084919","heartbeat_front_addr":"172.21.15.52:6823/359084919","state":["exists","up"]},{"osd":7,"uuid":"57b735d4-5072-4ca2-a3ff-3763ad84f15a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":3737722011},{"type":"v1","addr":"172.21.15.120:6817","nonce":3737722011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6818","nonce":3737722011},{"type":"v1","addr":"172.21.15.120:6819","nonce":3737722011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":3737722011},{"type":"v1","addr":"172.21.15.120:6823","nonce":3737722011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":3737722011},{"type":"v1","addr":"172.21.15.120:6821","nonce":3737722011}]},"public_addr":"172.21.15.120:6817/3737722011","cluster_addr":"172.21.15.120:6819/3737722011","heartbeat_back_addr":"172.21.15.120:6823/3737722011","heartbeat_front_addr":"172.21.15.120:6821/3737722011","state":["exists","up"]},{"osd":8,"uuid":"4ffbb43b-854a-4eac-99cf-7af89bc65409","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1444828441},{"type":"v1","addr":"172.21.15.188:6817","nonce":1444828441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1444828441},{"type":"v1","addr":"172.21.15.188:6819","nonce":1444828441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1444828441},{"type":"v1","addr":"172.21.15.188:6823","nonce":1444828441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1444828441},{"type":"v1","addr":"172.21.15.188:6821","nonce":1444828441}]},"public_addr":"172.21.15.188:6817/1444828441","cluster_addr":"172.21.15.188:6819/1444828441","heartbeat_back_addr":"172.21.15.188:6823/1444828441","heartbeat_front_addr":"172.21.15.188:6821/1444828441","state":["exists","up"]},{"osd":9,"uuid":"5bfd985c-8acc-4d00-8561-40bbd541c07b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":924440658},{"type":"v1","addr":"172.21.15.52:6827","nonce":924440658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":924440658},{"type":"v1","addr":"172.21.15.52:6829","nonce":924440658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":924440658},{"type":"v1","addr":"172.21.15.52:6833","nonce":924440658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":924440658},{"type":"v1","addr":"172.21.15.52:6831","nonce":924440658}]},"public_addr":"172.21.15.52:6827/924440658","cluster_addr":"172.21.15.52:6829/924440658","heartbeat_back_addr":"172.21.15.52:6833/924440658","heartbeat_front_addr":"172.21.15.52:6831/924440658","state":["exists","up"]},{"osd":10,"uuid":"0180cddf-e109-42cc-829a-185604276f02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6824","nonce":1531769083},{"type":"v1","addr":"172.21.15.120:6825","nonce":1531769083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":1531769083},{"type":"v1","addr":"172.21.15.120:6827","nonce":1531769083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":1531769083},{"type":"v1","addr":"172.21.15.120:6831","nonce":1531769083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6828","nonce":1531769083},{"type":"v1","addr":"172.21.15.120:6829","nonce":1531769083}]},"public_addr":"172.21.15.120:6825/1531769083","cluster_addr":"172.21.15.120:6827/1531769083","heartbeat_back_addr":"172.21.15.120:6831/1531769083","heartbeat_front_addr":"172.21.15.120:6829/1531769083","state":["exists","up"]},{"osd":11,"uuid":"73aec750-82d1-4800-b6a0-4e305912f9ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":3769050170},{"type":"v1","addr":"172.21.15.188:6825","nonce":3769050170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":3769050170},{"type":"v1","addr":"172.21.15.188:6827","nonce":3769050170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":3769050170},{"type":"v1","addr":"172.21.15.188:6831","nonce":3769050170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":3769050170},{"type":"v1","addr":"172.21.15.188:6829","nonce":3769050170}]},"public_addr":"172.21.15.188:6825/3769050170","cluster_addr":"172.21.15.188:6827/3769050170","heartbeat_back_addr":"172.21.15.188:6831/3769050170","heartbeat_front_addr":"172.21.15.188:6829/3769050170","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:44:16.455752+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:44:40.219735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:45:03.558899+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:45:29.155004+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:45:53.981098+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:46:19.412020+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:46:45.462640+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:47:11.484578+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:47:36.879054+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:48:03.392446+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:48:29.422173+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T02:48:55.523423+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":4}]},{"pgid":"3.6","mappings":[{"from":3,"to":7}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.9","mappings":[{"from":5,"to":8}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.52:0/3535271671":"2024-06-11T02:42:42.872312+0000","172.21.15.52:0/1092934380":"2024-06-11T02:42:42.872312+0000","172.21.15.52:6800/896317057":"2024-06-11T02:42:42.872312+0000","172.21.15.52:6801/3934479642":"2024-06-11T02:42:15.498118+0000","172.21.15.52:6800/3934479642":"2024-06-11T02:42:15.498118+0000","172.21.15.52:0/2759694132":"2024-06-11T02:42:15.498118+0000","172.21.15.52:0/2593665631":"2024-06-11T02:42:15.498118+0000","172.21.15.52:0/3151504634":"2024-06-11T02:42:15.498118+0000","172.21.15.52:6801/896317057":"2024-06-11T02:42:42.872312+0000","172.21.15.52:0/1254480982":"2024-06-11T02:42:42.872312+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-10T03:04:47.046 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:47.046+0000 7fa48aa8c700 1 -- 172.21.15.52:0/1896691606 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fa4740a2ca0 msgr2=0x7fa4740a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:04:47.047 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:47.046+0000 7fa48aa8c700 1 --2- 172.21.15.52:0/1896691606 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fa4740a2ca0 0x7fa4740a5160 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fa478005950 tx=0x7fa47800a300 comp rx=0 tx=0).stop 2024-06-10T03:04:47.047 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:47.046+0000 7fa48aa8c700 1 -- 172.21.15.52:0/1896691606 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa4841033a0 msgr2=0x7fa48419b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:04:47.047 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:47.046+0000 7fa48aa8c700 1 --2- 172.21.15.52:0/1896691606 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa4841033a0 0x7fa48419b3e0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fa470000c00 tx=0x7fa470003890 comp rx=0 tx=0).stop 2024-06-10T03:04:47.047 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:47.046+0000 7fa48aa8c700 1 -- 172.21.15.52:0/1896691606 shutdown_connections 2024-06-10T03:04:47.047 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:47.046+0000 7fa48aa8c700 1 --2- 172.21.15.52:0/1896691606 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa48410ede0 0x7fa48419f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:04:47.047 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:47.046+0000 7fa48aa8c700 1 --2- 172.21.15.52:0/1896691606 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa48406b0a0 0x7fa48419aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:04:47.047 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:47.046+0000 7fa48aa8c700 1 --2- 172.21.15.52:0/1896691606 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fa4740a2ca0 0x7fa4740a5160 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:04:47.047 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:47.046+0000 7fa48aa8c700 1 --2- 172.21.15.52:0/1896691606 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa4841033a0 0x7fa48419b3e0 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:04:47.047 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:47.046+0000 7fa48aa8c700 1 -- 172.21.15.52:0/1896691606 >> 172.21.15.52:0/1896691606 conn(0x7fa484078340 msgr2=0x7fa484100760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:04:47.047 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:47.046+0000 7fa48aa8c700 1 -- 172.21.15.52:0/1896691606 shutdown_connections 2024-06-10T03:04:47.047 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:47.046+0000 7fa48aa8c700 1 -- 172.21.15.52:0/1896691606 wait complete. 2024-06-10T03:04:47.058 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:04:46 smithi052 ceph-mon[32362]: pgmap v751: 129 pgs: 129 active+clean; 228 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-06-10T03:04:47.388 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:04:46 smithi120 ceph-mon[36634]: pgmap v751: 129 pgs: 129 active+clean; 228 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-06-10T03:04:47.454 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:04:46 smithi188 ceph-mon[36599]: pgmap v751: 129 pgs: 129 active+clean; 228 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-06-10T03:04:47.829 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph fs fail cephfs 2024-06-10T03:04:48.084 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T03:04:48.206 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:04:47 smithi052 ceph-mon[32362]: from='client.? 172.21.15.52:0/1896691606' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T03:04:48.388 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:04:47 smithi120 ceph-mon[36634]: from='client.? 172.21.15.52:0/1896691606' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T03:04:48.454 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:04:47 smithi188 ceph-mon[36599]: from='client.? 172.21.15.52:0/1896691606' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T03:04:49.170 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:04:48 smithi052 ceph-mon[32362]: pgmap v752: 129 pgs: 129 active+clean; 228 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.2 KiB/s wr, 13 op/s 2024-06-10T03:04:49.388 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:04:48 smithi120 ceph-mon[36634]: pgmap v752: 129 pgs: 129 active+clean; 228 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.2 KiB/s wr, 13 op/s 2024-06-10T03:04:49.454 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:04:48 smithi188 ceph-mon[36599]: pgmap v752: 129 pgs: 129 active+clean; 228 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.2 KiB/s wr, 13 op/s 2024-06-10T03:04:49.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:49.705+0000 7ff2bbdc9700 1 -- 172.21.15.52:0/1927721573 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff2b4102ed0 msgr2=0x7ff2b410f350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:04:49.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:49.705+0000 7ff2bbdc9700 1 --2- 172.21.15.52:0/1927721573 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff2b4102ed0 0x7ff2b410f350 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7ff2ac00b0a0 tx=0x7ff2ac00b3b0 comp rx=0 tx=0).stop 2024-06-10T03:04:49.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:49.705+0000 7ff2bbdc9700 1 -- 172.21.15.52:0/1927721573 shutdown_connections 2024-06-10T03:04:49.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:49.705+0000 7ff2bbdc9700 1 --2- 172.21.15.52:0/1927721573 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff2b4102ed0 0x7ff2b410f350 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:04:49.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:49.705+0000 7ff2bbdc9700 1 --2- 172.21.15.52:0/1927721573 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff2b4108510 0x7ff2b41088f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:04:49.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:49.705+0000 7ff2bbdc9700 1 --2- 172.21.15.52:0/1927721573 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff2b4102510 0x7ff2b4102990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:04:49.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:49.705+0000 7ff2bbdc9700 1 -- 172.21.15.52:0/1927721573 >> 172.21.15.52:0/1927721573 conn(0x7ff2b40795b0 msgr2=0x7ff2b40799c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:04:49.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:49.705+0000 7ff2bbdc9700 1 -- 172.21.15.52:0/1927721573 shutdown_connections 2024-06-10T03:04:49.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:49.705+0000 7ff2bbdc9700 1 -- 172.21.15.52:0/1927721573 wait complete. 2024-06-10T03:04:49.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:49.706+0000 7ff2bbdc9700 1 Processor -- start 2024-06-10T03:04:49.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:49.706+0000 7ff2bbdc9700 1 -- start start 2024-06-10T03:04:49.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:49.706+0000 7ff2bbdc9700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff2b4102510 0x7ff2b419e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:04:49.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:49.706+0000 7ff2bbdc9700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff2b4102ed0 0x7ff2b419eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:04:49.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:49.706+0000 7ff2bbdc9700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff2b4108510 0x7ff2b4198a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:04:49.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:49.706+0000 7ff2bbdc9700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ff2b41127e0 con 0x7ff2b4102ed0 2024-06-10T03:04:49.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:49.706+0000 7ff2bbdc9700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff2b4112660 con 0x7ff2b4108510 2024-06-10T03:04:49.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:49.706+0000 7ff2bbdc9700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ff2b4112960 con 0x7ff2b4102510 2024-06-10T03:04:49.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:49.706+0000 7ff2b9364700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff2b4102ed0 0x7ff2b419eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:04:49.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:49.706+0000 7ff2b9364700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff2b4102ed0 0x7ff2b419eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:39194/0 (socket says 172.21.15.52:39194) 2024-06-10T03:04:49.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:49.706+0000 7ff2b9364700 1 -- 172.21.15.52:0/3630335614 learned_addr learned my addr 172.21.15.52:0/3630335614 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:04:49.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:49.706+0000 7ff2b9b65700 1 --2- 172.21.15.52:0/3630335614 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff2b4102510 0x7ff2b419e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:04:49.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:49.706+0000 7ff2ba366700 1 --2- 172.21.15.52:0/3630335614 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff2b4108510 0x7ff2b4198a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:04:49.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:49.706+0000 7ff2b9364700 1 -- 172.21.15.52:0/3630335614 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff2b4102510 msgr2=0x7ff2b419e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:04:49.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:49.706+0000 7ff2b9364700 1 --2- 172.21.15.52:0/3630335614 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff2b4102510 0x7ff2b419e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:04:49.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:49.706+0000 7ff2b9364700 1 -- 172.21.15.52:0/3630335614 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff2b4108510 msgr2=0x7ff2b4198a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:04:49.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:49.706+0000 7ff2b9364700 1 --2- 172.21.15.52:0/3630335614 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff2b4108510 0x7ff2b4198a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:04:49.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:49.706+0000 7ff2b9364700 1 -- 172.21.15.52:0/3630335614 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2b4199180 con 0x7ff2b4102ed0 2024-06-10T03:04:49.710 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:49.706+0000 7ff2b9b65700 1 --2- 172.21.15.52:0/3630335614 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff2b4102510 0x7ff2b419e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T03:04:49.710 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:49.707+0000 7ff2b9364700 1 --2- 172.21.15.52:0/3630335614 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff2b4102ed0 0x7ff2b419eda0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7ff2a400b530 tx=0x7ff2a400b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:04:49.710 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:49.707+0000 7ff2aaffd700 1 -- 172.21.15.52:0/3630335614 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2a4015020 con 0x7ff2b4102ed0 2024-06-10T03:04:49.710 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:49.707+0000 7ff2bbdc9700 1 -- 172.21.15.52:0/3630335614 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2b4199470 con 0x7ff2b4102ed0 2024-06-10T03:04:49.710 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:49.707+0000 7ff2aaffd700 1 -- 172.21.15.52:0/3630335614 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff2a4011d10 con 0x7ff2b4102ed0 2024-06-10T03:04:49.710 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:49.707+0000 7ff2aaffd700 1 -- 172.21.15.52:0/3630335614 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2a4010e50 con 0x7ff2b4102ed0 2024-06-10T03:04:49.710 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:49.707+0000 7ff2bbdc9700 1 -- 172.21.15.52:0/3630335614 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2b41a5730 con 0x7ff2b4102ed0 2024-06-10T03:04:49.710 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:49.708+0000 7ff2bbdc9700 1 -- 172.21.15.52:0/3630335614 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2b4103bd0 con 0x7ff2b4102ed0 2024-06-10T03:04:49.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:49.709+0000 7ff2aaffd700 1 -- 172.21.15.52:0/3630335614 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7ff2a4018020 con 0x7ff2b4102ed0 2024-06-10T03:04:49.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:49.710+0000 7ff2aaffd700 1 --2- 172.21.15.52:0/3630335614 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7ff2a00a2ca0 0x7ff2a00a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:04:49.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:49.710+0000 7ff2aaffd700 1 -- 172.21.15.52:0/3630335614 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7ff2a40cebe0 con 0x7ff2b4102ed0 2024-06-10T03:04:49.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:49.710+0000 7ff2b9b65700 1 --2- 172.21.15.52:0/3630335614 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7ff2a00a2ca0 0x7ff2a00a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:04:49.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:49.710+0000 7ff2b9b65700 1 --2- 172.21.15.52:0/3630335614 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7ff2a00a2ca0 0x7ff2a00a5160 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7ff2b0005950 tx=0x7ff2b00058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:04:49.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:49.714+0000 7ff2aaffd700 1 -- 172.21.15.52:0/3630335614 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7ff2a401f020 con 0x7ff2b4102ed0 2024-06-10T03:04:49.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:49.970+0000 7ff2bbdc9700 1 -- 172.21.15.52:0/3630335614 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7ff2b4069180 con 0x7ff2b4102ed0 2024-06-10T03:04:49.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:49.972+0000 7ff2aaffd700 1 -- 172.21.15.52:0/3630335614 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v50) ==== 161+0+0 (secure 0 0 0) 0x7ff2a409b790 con 0x7ff2b4102ed0 2024-06-10T03:04:49.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:49.976+0000 7ff2bbdc9700 1 -- 172.21.15.52:0/3630335614 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7ff2a00a2ca0 msgr2=0x7ff2a00a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:04:49.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:49.976+0000 7ff2bbdc9700 1 --2- 172.21.15.52:0/3630335614 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7ff2a00a2ca0 0x7ff2a00a5160 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7ff2b0005950 tx=0x7ff2b00058e0 comp rx=0 tx=0).stop 2024-06-10T03:04:49.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:49.976+0000 7ff2bbdc9700 1 -- 172.21.15.52:0/3630335614 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff2b4102ed0 msgr2=0x7ff2b419eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:04:49.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:49.976+0000 7ff2bbdc9700 1 --2- 172.21.15.52:0/3630335614 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff2b4102ed0 0x7ff2b419eda0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7ff2a400b530 tx=0x7ff2a400b8f0 comp rx=0 tx=0).stop 2024-06-10T03:04:49.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:49.976+0000 7ff2bbdc9700 1 -- 172.21.15.52:0/3630335614 shutdown_connections 2024-06-10T03:04:49.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:49.976+0000 7ff2bbdc9700 1 --2- 172.21.15.52:0/3630335614 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff2b4108510 0x7ff2b4198a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:04:49.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:49.976+0000 7ff2bbdc9700 1 --2- 172.21.15.52:0/3630335614 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff2b4102510 0x7ff2b419e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:04:49.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:49.976+0000 7ff2bbdc9700 1 --2- 172.21.15.52:0/3630335614 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7ff2a00a2ca0 0x7ff2a00a5160 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:04:49.978 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:49.976+0000 7ff2bbdc9700 1 --2- 172.21.15.52:0/3630335614 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff2b4102ed0 0x7ff2b419eda0 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:04:49.978 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:49.976+0000 7ff2bbdc9700 1 -- 172.21.15.52:0/3630335614 >> 172.21.15.52:0/3630335614 conn(0x7ff2b40795b0 msgr2=0x7ff2b4101160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:04:49.978 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:49.977+0000 7ff2bbdc9700 1 -- 172.21.15.52:0/3630335614 shutdown_connections 2024-06-10T03:04:49.978 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:49.977+0000 7ff2bbdc9700 1 -- 172.21.15.52:0/3630335614 wait complete. 2024-06-10T03:04:49.978 INFO:teuthology.orchestra.run.smithi052.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-06-10T03:04:50.227 INFO:journalctl@ceph.mds.a.smithi052.stdout:Jun 10 03:04:50 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mds-a[74607]: ignoring --setuser ceph since I am not root 2024-06-10T03:04:50.227 INFO:journalctl@ceph.mds.a.smithi052.stdout:Jun 10 03:04:50 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mds-a[74607]: ignoring --setgroup ceph since I am not root 2024-06-10T03:04:50.227 INFO:journalctl@ceph.mds.a.smithi052.stdout:Jun 10 03:04:50 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mds-a[74607]: starting mds.a at 2024-06-10T03:04:50.228 INFO:journalctl@ceph.mds.g.smithi052.stdout:Jun 10 03:04:50 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mds-g[74147]: ignoring --setuser ceph since I am not root 2024-06-10T03:04:50.228 INFO:journalctl@ceph.mds.g.smithi052.stdout:Jun 10 03:04:50 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mds-g[74147]: ignoring --setgroup ceph since I am not root 2024-06-10T03:04:50.228 INFO:journalctl@ceph.mds.g.smithi052.stdout:Jun 10 03:04:50 smithi052 ceph-mds[74169]: -- 172.21.15.52:0/3179346541 <== mon.0 v2:172.21.15.52:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x559391253a40 con 0x559391227000 2024-06-10T03:04:50.228 INFO:journalctl@ceph.mds.g.smithi052.stdout:Jun 10 03:04:50 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mds-g[74147]: starting mds.g at 2024-06-10T03:04:50.228 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:04:49 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mon-a[32358]: 2024-06-10T03:04:49.971+0000 7f2d3ba70700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-06-10T03:04:50.228 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:04:50 smithi052 ceph-mon[32362]: from='client.? 172.21.15.52:0/3630335614' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-06-10T03:04:50.228 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:04:50 smithi052 ceph-mon[32362]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-06-10T03:04:50.228 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:04:50 smithi052 ceph-mon[32362]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-06-10T03:04:50.228 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:04:50 smithi052 ceph-mon[32362]: from='client.? 172.21.15.52:0/3630335614' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-06-10T03:04:50.229 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:04:50 smithi052 ceph-mon[32362]: osdmap e101: 12 total, 12 up, 12 in 2024-06-10T03:04:50.229 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:04:50 smithi052 ceph-mon[32362]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-06-10T03:04:50.389 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:04:50 smithi120 ceph-mon[36634]: from='client.? 172.21.15.52:0/3630335614' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-06-10T03:04:50.389 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:04:50 smithi120 ceph-mon[36634]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-06-10T03:04:50.389 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:04:50 smithi120 ceph-mon[36634]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-06-10T03:04:50.389 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:04:50 smithi120 ceph-mon[36634]: from='client.? 172.21.15.52:0/3630335614' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-06-10T03:04:50.389 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:04:50 smithi120 ceph-mon[36634]: osdmap e101: 12 total, 12 up, 12 in 2024-06-10T03:04:50.389 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:04:50 smithi120 ceph-mon[36634]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-06-10T03:04:50.389 INFO:journalctl@ceph.mds.b.smithi120.stdout:Jun 10 03:04:50 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mds-b[66008]: ignoring --setuser ceph since I am not root 2024-06-10T03:04:50.389 INFO:journalctl@ceph.mds.b.smithi120.stdout:Jun 10 03:04:50 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mds-b[66008]: ignoring --setgroup ceph since I am not root 2024-06-10T03:04:50.389 INFO:journalctl@ceph.mds.b.smithi120.stdout:Jun 10 03:04:50 smithi120 ceph-mds[66012]: -- 172.21.15.120:0/980479373 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x560f0af684e0 con 0x560f0af3e800 2024-06-10T03:04:50.390 INFO:journalctl@ceph.mds.b.smithi120.stdout:Jun 10 03:04:50 smithi120 ceph-mds[66012]: -- 172.21.15.120:0/980479373 <== mon.2 v2:172.21.15.120:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x560f0af6ba40 con 0x560f0af3e800 2024-06-10T03:04:50.390 INFO:journalctl@ceph.mds.b.smithi120.stdout:Jun 10 03:04:50 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mds-b[66008]: starting mds.b at 2024-06-10T03:04:50.390 INFO:journalctl@ceph.mds.k.smithi120.stdout:Jun 10 03:04:50 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mds-k[65075]: ignoring --setuser ceph since I am not root 2024-06-10T03:04:50.390 INFO:journalctl@ceph.mds.k.smithi120.stdout:Jun 10 03:04:50 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mds-k[65075]: ignoring --setgroup ceph since I am not root 2024-06-10T03:04:50.390 INFO:journalctl@ceph.mds.k.smithi120.stdout:Jun 10 03:04:50 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mds-k[65075]: starting mds.k at 2024-06-10T03:04:50.390 INFO:journalctl@ceph.mds.e.smithi120.stdout:Jun 10 03:04:50 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mds-e[65389]: ignoring --setuser ceph since I am not root 2024-06-10T03:04:50.390 INFO:journalctl@ceph.mds.e.smithi120.stdout:Jun 10 03:04:50 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mds-e[65389]: ignoring --setgroup ceph since I am not root 2024-06-10T03:04:50.391 INFO:journalctl@ceph.mds.e.smithi120.stdout:Jun 10 03:04:50 smithi120 ceph-mds[65393]: -- 172.21.15.120:0/2315002130 <== mon.2 v2:172.21.15.120:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56164f981a40 con 0x56164f954800 2024-06-10T03:04:50.391 INFO:journalctl@ceph.mds.e.smithi120.stdout:Jun 10 03:04:50 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mds-e[65389]: starting mds.e at 2024-06-10T03:04:50.391 INFO:journalctl@ceph.mds.h.smithi120.stdout:Jun 10 03:04:50 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mds-h[65703]: ignoring --setuser ceph since I am not root 2024-06-10T03:04:50.391 INFO:journalctl@ceph.mds.h.smithi120.stdout:Jun 10 03:04:50 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mds-h[65703]: ignoring --setgroup ceph since I am not root 2024-06-10T03:04:50.391 INFO:journalctl@ceph.mds.h.smithi120.stdout:Jun 10 03:04:50 smithi120 ceph-mds[65707]: -- 172.21.15.120:0/2549174184 <== mon.1 v2:172.21.15.188:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a6a2cefa40 con 0x55a6a2cc3000 2024-06-10T03:04:50.391 INFO:journalctl@ceph.mds.h.smithi120.stdout:Jun 10 03:04:50 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mds-h[65703]: starting mds.h at 2024-06-10T03:04:50.454 INFO:journalctl@ceph.mds.c.smithi188.stdout:Jun 10 03:04:50 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mds-c[66642]: ignoring --setuser ceph since I am not root 2024-06-10T03:04:50.454 INFO:journalctl@ceph.mds.c.smithi188.stdout:Jun 10 03:04:50 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mds-c[66642]: ignoring --setgroup ceph since I am not root 2024-06-10T03:04:50.454 INFO:journalctl@ceph.mds.c.smithi188.stdout:Jun 10 03:04:50 smithi188 ceph-mds[66646]: -- 172.21.15.188:0/542251800 <== mon.1 v2:172.21.15.188:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5599e712fa40 con 0x5599e7102800 2024-06-10T03:04:50.454 INFO:journalctl@ceph.mds.c.smithi188.stdout:Jun 10 03:04:50 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mds-c[66642]: starting mds.c at 2024-06-10T03:04:50.457 INFO:journalctl@ceph.mds.f.smithi188.stdout:Jun 10 03:04:50 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mds-f[66339]: ignoring --setuser ceph since I am not root 2024-06-10T03:04:50.458 INFO:journalctl@ceph.mds.f.smithi188.stdout:Jun 10 03:04:50 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mds-f[66339]: ignoring --setgroup ceph since I am not root 2024-06-10T03:04:50.458 INFO:journalctl@ceph.mds.f.smithi188.stdout:Jun 10 03:04:50 smithi188 ceph-mds[66343]: -- 172.21.15.188:0/1021611871 <== mon.2 v2:172.21.15.120:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55dd3a785a40 con 0x55dd3a758c00 2024-06-10T03:04:50.458 INFO:journalctl@ceph.mds.f.smithi188.stdout:Jun 10 03:04:50 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mds-f[66339]: starting mds.f at 2024-06-10T03:04:50.458 INFO:journalctl@ceph.mds.i.smithi188.stdout:Jun 10 03:04:50 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mds-i[66946]: ignoring --setuser ceph since I am not root 2024-06-10T03:04:50.458 INFO:journalctl@ceph.mds.i.smithi188.stdout:Jun 10 03:04:50 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mds-i[66946]: ignoring --setgroup ceph since I am not root 2024-06-10T03:04:50.458 INFO:journalctl@ceph.mds.i.smithi188.stdout:Jun 10 03:04:50 smithi188 ceph-mds[66950]: -- 172.21.15.188:0/2792113335 <== mon.2 v2:172.21.15.120:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e943d43a40 con 0x55e943d17000 2024-06-10T03:04:50.458 INFO:journalctl@ceph.mds.i.smithi188.stdout:Jun 10 03:04:50 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mds-i[66946]: starting mds.i at 2024-06-10T03:04:50.459 INFO:journalctl@ceph.mds.l.smithi188.stdout:Jun 10 03:04:50 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mds-l[66025]: ignoring --setuser ceph since I am not root 2024-06-10T03:04:50.459 INFO:journalctl@ceph.mds.l.smithi188.stdout:Jun 10 03:04:50 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mds-l[66025]: ignoring --setgroup ceph since I am not root 2024-06-10T03:04:50.459 INFO:journalctl@ceph.mds.l.smithi188.stdout:Jun 10 03:04:50 smithi188 ceph-mds[66029]: -- 172.21.15.188:0/4191474224 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5624868dc4e0 con 0x5624868b2800 2024-06-10T03:04:50.459 INFO:journalctl@ceph.mds.l.smithi188.stdout:Jun 10 03:04:50 smithi188 ceph-mds[66029]: -- 172.21.15.188:0/4191474224 <== mon.0 v2:172.21.15.52:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5624868dfa40 con 0x5624868b2800 2024-06-10T03:04:50.459 INFO:journalctl@ceph.mds.l.smithi188.stdout:Jun 10 03:04:50 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mds-l[66025]: starting mds.l at 2024-06-10T03:04:50.459 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:04:50 smithi188 ceph-mon[36599]: from='client.? 172.21.15.52:0/3630335614' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-06-10T03:04:50.459 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:04:50 smithi188 ceph-mon[36599]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-06-10T03:04:50.459 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:04:50 smithi188 ceph-mon[36599]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-06-10T03:04:50.459 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:04:50 smithi188 ceph-mon[36599]: from='client.? 172.21.15.52:0/3630335614' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-06-10T03:04:50.459 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:04:50 smithi188 ceph-mon[36599]: osdmap e101: 12 total, 12 up, 12 in 2024-06-10T03:04:50.459 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:04:50 smithi188 ceph-mon[36599]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-06-10T03:04:50.634 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph fs rm cephfs --yes-i-really-mean-it 2024-06-10T03:04:50.886 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T03:04:51.389 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:04:51 smithi120 ceph-mon[36634]: pgmap v753: 129 pgs: 129 active+clean; 228 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-06-10T03:04:51.433 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:04:51 smithi052 ceph-mon[32362]: pgmap v753: 129 pgs: 129 active+clean; 228 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-06-10T03:04:51.454 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:04:51 smithi188 ceph-mon[36599]: pgmap v753: 129 pgs: 129 active+clean; 228 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-06-10T03:04:52.269 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:04:52 smithi052 ceph-mon[32362]: mds.? [v2:172.21.15.120:6833/3031986796,v1:172.21.15.120:6836/3031986796] up:boot 2024-06-10T03:04:52.269 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:04:52 smithi052 ceph-mon[32362]: mds.? [v2:172.21.15.120:6834/2836659195,v1:172.21.15.120:6837/2836659195] up:boot 2024-06-10T03:04:52.269 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:04:52 smithi052 ceph-mon[32362]: mds.? [v2:172.21.15.120:6832/541191450,v1:172.21.15.120:6835/541191450] up:boot 2024-06-10T03:04:52.270 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:04:52 smithi052 ceph-mon[32362]: mds.? [v2:172.21.15.120:6838/1538354971,v1:172.21.15.120:6839/1538354971] up:boot 2024-06-10T03:04:52.270 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:04:52 smithi052 ceph-mon[32362]: mds.? [v2:172.21.15.52:6838/4232172243,v1:172.21.15.52:6839/4232172243] up:boot 2024-06-10T03:04:52.270 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:04:52 smithi052 ceph-mon[32362]: mds.? [v2:172.21.15.52:6840/757245882,v1:172.21.15.52:6841/757245882] up:boot 2024-06-10T03:04:52.270 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:04:52 smithi052 ceph-mon[32362]: mds.? [v2:172.21.15.188:6833/1312425809,v1:172.21.15.188:6837/1312425809] up:boot 2024-06-10T03:04:52.270 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:04:52 smithi052 ceph-mon[32362]: mds.? [v2:172.21.15.188:6832/876198500,v1:172.21.15.188:6836/876198500] up:boot 2024-06-10T03:04:52.270 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:04:52 smithi052 ceph-mon[32362]: mds.? [v2:172.21.15.188:6835/3358861594,v1:172.21.15.188:6839/3358861594] up:boot 2024-06-10T03:04:52.270 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:04:52 smithi052 ceph-mon[32362]: mds.? [v2:172.21.15.188:6834/535558419,v1:172.21.15.188:6838/535558419] up:boot 2024-06-10T03:04:52.270 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:04:52 smithi052 ceph-mon[32362]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-06-10T03:04:52.270 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:04:52 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-06-10T03:04:52.270 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:04:52 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-06-10T03:04:52.270 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:04:52 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-06-10T03:04:52.270 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:04:52 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-06-10T03:04:52.271 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:04:52 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-06-10T03:04:52.271 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:04:52 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-06-10T03:04:52.271 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:04:52 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-06-10T03:04:52.271 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:04:52 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-06-10T03:04:52.271 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:04:52 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-06-10T03:04:52.271 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:04:52 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-06-10T03:04:52.271 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:04:52 smithi052 ceph-mon[32362]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-06-10T03:04:52.271 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:04:52 smithi052 ceph-mon[32362]: pgmap v755: 129 pgs: 129 active+clean; 228 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.3 KiB/s wr, 11 op/s 2024-06-10T03:04:52.389 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:04:52 smithi120 ceph-mon[36634]: mds.? [v2:172.21.15.120:6833/3031986796,v1:172.21.15.120:6836/3031986796] up:boot 2024-06-10T03:04:52.389 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:04:52 smithi120 ceph-mon[36634]: mds.? [v2:172.21.15.120:6834/2836659195,v1:172.21.15.120:6837/2836659195] up:boot 2024-06-10T03:04:52.389 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:04:52 smithi120 ceph-mon[36634]: mds.? [v2:172.21.15.120:6832/541191450,v1:172.21.15.120:6835/541191450] up:boot 2024-06-10T03:04:52.389 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:04:52 smithi120 ceph-mon[36634]: mds.? [v2:172.21.15.120:6838/1538354971,v1:172.21.15.120:6839/1538354971] up:boot 2024-06-10T03:04:52.389 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:04:52 smithi120 ceph-mon[36634]: mds.? [v2:172.21.15.52:6838/4232172243,v1:172.21.15.52:6839/4232172243] up:boot 2024-06-10T03:04:52.389 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:04:52 smithi120 ceph-mon[36634]: mds.? [v2:172.21.15.52:6840/757245882,v1:172.21.15.52:6841/757245882] up:boot 2024-06-10T03:04:52.389 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:04:52 smithi120 ceph-mon[36634]: mds.? [v2:172.21.15.188:6833/1312425809,v1:172.21.15.188:6837/1312425809] up:boot 2024-06-10T03:04:52.389 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:04:52 smithi120 ceph-mon[36634]: mds.? [v2:172.21.15.188:6832/876198500,v1:172.21.15.188:6836/876198500] up:boot 2024-06-10T03:04:52.389 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:04:52 smithi120 ceph-mon[36634]: mds.? [v2:172.21.15.188:6835/3358861594,v1:172.21.15.188:6839/3358861594] up:boot 2024-06-10T03:04:52.390 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:04:52 smithi120 ceph-mon[36634]: mds.? [v2:172.21.15.188:6834/535558419,v1:172.21.15.188:6838/535558419] up:boot 2024-06-10T03:04:52.390 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:04:52 smithi120 ceph-mon[36634]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-06-10T03:04:52.390 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:04:52 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-06-10T03:04:52.390 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:04:52 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-06-10T03:04:52.390 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:04:52 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-06-10T03:04:52.390 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:04:52 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-06-10T03:04:52.390 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:04:52 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-06-10T03:04:52.390 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:04:52 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-06-10T03:04:52.390 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:04:52 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-06-10T03:04:52.390 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:04:52 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-06-10T03:04:52.390 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:04:52 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-06-10T03:04:52.390 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:04:52 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-06-10T03:04:52.390 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:04:52 smithi120 ceph-mon[36634]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-06-10T03:04:52.391 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:04:52 smithi120 ceph-mon[36634]: pgmap v755: 129 pgs: 129 active+clean; 228 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.3 KiB/s wr, 11 op/s 2024-06-10T03:04:52.454 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:04:52 smithi188 ceph-mon[36599]: mds.? [v2:172.21.15.120:6833/3031986796,v1:172.21.15.120:6836/3031986796] up:boot 2024-06-10T03:04:52.454 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:04:52 smithi188 ceph-mon[36599]: mds.? [v2:172.21.15.120:6834/2836659195,v1:172.21.15.120:6837/2836659195] up:boot 2024-06-10T03:04:52.454 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:04:52 smithi188 ceph-mon[36599]: mds.? [v2:172.21.15.120:6832/541191450,v1:172.21.15.120:6835/541191450] up:boot 2024-06-10T03:04:52.454 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:04:52 smithi188 ceph-mon[36599]: mds.? [v2:172.21.15.120:6838/1538354971,v1:172.21.15.120:6839/1538354971] up:boot 2024-06-10T03:04:52.455 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:04:52 smithi188 ceph-mon[36599]: mds.? [v2:172.21.15.52:6838/4232172243,v1:172.21.15.52:6839/4232172243] up:boot 2024-06-10T03:04:52.455 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:04:52 smithi188 ceph-mon[36599]: mds.? [v2:172.21.15.52:6840/757245882,v1:172.21.15.52:6841/757245882] up:boot 2024-06-10T03:04:52.455 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:04:52 smithi188 ceph-mon[36599]: mds.? [v2:172.21.15.188:6833/1312425809,v1:172.21.15.188:6837/1312425809] up:boot 2024-06-10T03:04:52.455 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:04:52 smithi188 ceph-mon[36599]: mds.? [v2:172.21.15.188:6832/876198500,v1:172.21.15.188:6836/876198500] up:boot 2024-06-10T03:04:52.455 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:04:52 smithi188 ceph-mon[36599]: mds.? [v2:172.21.15.188:6835/3358861594,v1:172.21.15.188:6839/3358861594] up:boot 2024-06-10T03:04:52.455 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:04:52 smithi188 ceph-mon[36599]: mds.? [v2:172.21.15.188:6834/535558419,v1:172.21.15.188:6838/535558419] up:boot 2024-06-10T03:04:52.455 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:04:52 smithi188 ceph-mon[36599]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-06-10T03:04:52.455 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:04:52 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-06-10T03:04:52.455 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:04:52 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-06-10T03:04:52.455 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:04:52 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-06-10T03:04:52.455 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:04:52 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-06-10T03:04:52.455 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:04:52 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-06-10T03:04:52.456 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:04:52 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-06-10T03:04:52.456 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:04:52 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-06-10T03:04:52.456 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:04:52 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-06-10T03:04:52.456 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:04:52 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-06-10T03:04:52.456 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:04:52 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-06-10T03:04:52.456 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:04:52 smithi188 ceph-mon[36599]: from='mgr.14150 172.21.15.52:0/2939367345' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-06-10T03:04:52.456 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:04:52 smithi188 ceph-mon[36599]: pgmap v755: 129 pgs: 129 active+clean; 228 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.3 KiB/s wr, 11 op/s 2024-06-10T03:04:52.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:52.641+0000 7fa7e6d33700 1 -- 172.21.15.52:0/2925662188 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa7e006ab30 msgr2=0x7fa7e010f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:04:52.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:52.641+0000 7fa7e6d33700 1 --2- 172.21.15.52:0/2925662188 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa7e006ab30 0x7fa7e010f320 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fa7d800b0a0 tx=0x7fa7d800b3b0 comp rx=0 tx=0).stop 2024-06-10T03:04:52.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:52.641+0000 7fa7e6d33700 1 -- 172.21.15.52:0/2925662188 shutdown_connections 2024-06-10T03:04:52.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:52.641+0000 7fa7e6d33700 1 --2- 172.21.15.52:0/2925662188 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa7e006ab30 0x7fa7e010f320 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:04:52.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:52.641+0000 7fa7e6d33700 1 --2- 172.21.15.52:0/2925662188 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa7e0108500 0x7fa7e01088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:04:52.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:52.641+0000 7fa7e6d33700 1 --2- 172.21.15.52:0/2925662188 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa7e006a170 0x7fa7e006a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:04:52.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:52.641+0000 7fa7e6d33700 1 -- 172.21.15.52:0/2925662188 >> 172.21.15.52:0/2925662188 conn(0x7fa7e0078340 msgr2=0x7fa7e0078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:04:52.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:52.641+0000 7fa7e6d33700 1 -- 172.21.15.52:0/2925662188 shutdown_connections 2024-06-10T03:04:52.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:52.641+0000 7fa7e6d33700 1 -- 172.21.15.52:0/2925662188 wait complete. 2024-06-10T03:04:52.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:52.642+0000 7fa7e6d33700 1 Processor -- start 2024-06-10T03:04:52.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:52.642+0000 7fa7e6d33700 1 -- start start 2024-06-10T03:04:52.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:52.642+0000 7fa7e6d33700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa7e006a170 0x7fa7e019e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:04:52.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:52.642+0000 7fa7e6d33700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa7e006ab30 0x7fa7e019ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:04:52.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:52.642+0000 7fa7e6d33700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa7e0108500 0x7fa7e01989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:04:52.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:52.642+0000 7fa7e6d33700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fa7e0112760 con 0x7fa7e006a170 2024-06-10T03:04:52.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:52.642+0000 7fa7e6d33700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa7e01125e0 con 0x7fa7e0108500 2024-06-10T03:04:52.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:52.642+0000 7fa7e6d33700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa7e01128e0 con 0x7fa7e006ab30 2024-06-10T03:04:52.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:52.642+0000 7fa7e4acf700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa7e006a170 0x7fa7e019e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:04:52.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:52.642+0000 7fa7e4acf700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa7e006a170 0x7fa7e019e800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55374/0 (socket says 172.21.15.52:55374) 2024-06-10T03:04:52.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:52.642+0000 7fa7e4acf700 1 -- 172.21.15.52:0/3476972233 learned_addr learned my addr 172.21.15.52:0/3476972233 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:04:52.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:52.642+0000 7fa7e52d0700 1 --2- 172.21.15.52:0/3476972233 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa7e0108500 0x7fa7e01989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:04:52.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:52.642+0000 7fa7dffff700 1 --2- 172.21.15.52:0/3476972233 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa7e006ab30 0x7fa7e019ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:04:52.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:52.642+0000 7fa7e4acf700 1 -- 172.21.15.52:0/3476972233 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa7e006ab30 msgr2=0x7fa7e019ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:04:52.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:52.642+0000 7fa7e4acf700 1 --2- 172.21.15.52:0/3476972233 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa7e006ab30 0x7fa7e019ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:04:52.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:52.642+0000 7fa7e4acf700 1 -- 172.21.15.52:0/3476972233 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa7e0108500 msgr2=0x7fa7e01989c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:04:52.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:52.642+0000 7fa7e4acf700 1 --2- 172.21.15.52:0/3476972233 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa7e0108500 0x7fa7e01989c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:04:52.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:52.642+0000 7fa7e4acf700 1 -- 172.21.15.52:0/3476972233 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7e0199120 con 0x7fa7e006a170 2024-06-10T03:04:52.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:52.643+0000 7fa7e4acf700 1 --2- 172.21.15.52:0/3476972233 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa7e006a170 0x7fa7e019e800 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fa7d400d980 tx=0x7fa7d400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:04:52.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:52.643+0000 7fa7ddffb700 1 -- 172.21.15.52:0/3476972233 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7d4016070 con 0x7fa7e006a170 2024-06-10T03:04:52.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:52.643+0000 7fa7e6d33700 1 -- 172.21.15.52:0/3476972233 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7e0199410 con 0x7fa7e006a170 2024-06-10T03:04:52.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:52.643+0000 7fa7ddffb700 1 -- 172.21.15.52:0/3476972233 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7d4004d10 con 0x7fa7e006a170 2024-06-10T03:04:52.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:52.643+0000 7fa7ddffb700 1 -- 172.21.15.52:0/3476972233 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7d40055f0 con 0x7fa7e006a170 2024-06-10T03:04:52.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:52.643+0000 7fa7e6d33700 1 -- 172.21.15.52:0/3476972233 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7e01a56d0 con 0x7fa7e006a170 2024-06-10T03:04:52.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:52.644+0000 7fa7e6d33700 1 -- 172.21.15.52:0/3476972233 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7e0069180 con 0x7fa7e006a170 2024-06-10T03:04:52.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:52.645+0000 7fa7ddffb700 1 -- 172.21.15.52:0/3476972233 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fa7d4004750 con 0x7fa7e006a170 2024-06-10T03:04:52.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:52.646+0000 7fa7ddffb700 1 --2- 172.21.15.52:0/3476972233 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fa7d00a2c20 0x7fa7d00a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:04:52.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:52.646+0000 7fa7dffff700 1 --2- 172.21.15.52:0/3476972233 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fa7d00a2c20 0x7fa7d00a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:04:52.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:52.646+0000 7fa7ddffb700 1 -- 172.21.15.52:0/3476972233 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9609+0+0 (secure 0 0 0) 0x7fa7d40cf3b0 con 0x7fa7e006a170 2024-06-10T03:04:52.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:52.647+0000 7fa7dffff700 1 --2- 172.21.15.52:0/3476972233 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fa7d00a2c20 0x7fa7d00a50e0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fa7cc005fd0 tx=0x7fa7cc00a560 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:04:52.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:52.651+0000 7fa7ddffb700 1 -- 172.21.15.52:0/3476972233 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fa7d401f020 con 0x7fa7e006a170 2024-06-10T03:04:52.907 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:52.907+0000 7fa7e6d33700 1 -- 172.21.15.52:0/3476972233 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fa7e002dca0 con 0x7fa7e006a170 2024-06-10T03:04:53.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:53.025+0000 7fa7ddffb700 1 -- 172.21.15.52:0/3476972233 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v52) ==== 104+0+0 (secure 0 0 0) 0x7fa7d409bd50 con 0x7fa7e006a170 2024-06-10T03:04:53.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:53.027+0000 7fa7e6d33700 1 -- 172.21.15.52:0/3476972233 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fa7d00a2c20 msgr2=0x7fa7d00a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:04:53.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:53.027+0000 7fa7e6d33700 1 --2- 172.21.15.52:0/3476972233 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fa7d00a2c20 0x7fa7d00a50e0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fa7cc005fd0 tx=0x7fa7cc00a560 comp rx=0 tx=0).stop 2024-06-10T03:04:53.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:53.027+0000 7fa7e6d33700 1 -- 172.21.15.52:0/3476972233 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa7e006a170 msgr2=0x7fa7e019e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:04:53.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:53.027+0000 7fa7e6d33700 1 --2- 172.21.15.52:0/3476972233 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa7e006a170 0x7fa7e019e800 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fa7d400d980 tx=0x7fa7d400dc90 comp rx=0 tx=0).stop 2024-06-10T03:04:53.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:53.027+0000 7fa7e6d33700 1 -- 172.21.15.52:0/3476972233 shutdown_connections 2024-06-10T03:04:53.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:53.027+0000 7fa7e6d33700 1 --2- 172.21.15.52:0/3476972233 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa7e0108500 0x7fa7e01989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:04:53.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:53.027+0000 7fa7e6d33700 1 --2- 172.21.15.52:0/3476972233 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fa7d00a2c20 0x7fa7d00a50e0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:04:53.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:53.027+0000 7fa7e6d33700 1 --2- 172.21.15.52:0/3476972233 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa7e006a170 0x7fa7e019e800 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:04:53.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:53.027+0000 7fa7e6d33700 1 --2- 172.21.15.52:0/3476972233 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa7e006ab30 0x7fa7e019ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:04:53.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:53.027+0000 7fa7e6d33700 1 -- 172.21.15.52:0/3476972233 >> 172.21.15.52:0/3476972233 conn(0x7fa7e0078340 msgr2=0x7fa7e00ffff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:04:53.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:53.028+0000 7fa7e6d33700 1 -- 172.21.15.52:0/3476972233 shutdown_connections 2024-06-10T03:04:53.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:53.028+0000 7fa7e6d33700 1 -- 172.21.15.52:0/3476972233 wait complete. 2024-06-10T03:04:53.227 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:04:53 smithi052 ceph-mon[32362]: from='client.? 172.21.15.52:0/3476972233' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-06-10T03:04:53.388 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:04:53 smithi120 ceph-mon[36634]: from='client.? 172.21.15.52:0/3476972233' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-06-10T03:04:53.454 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:04:53 smithi188 ceph-mon[36599]: from='client.? 172.21.15.52:0/3476972233' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-06-10T03:04:53.628 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-06-10T03:04:53.884 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T03:04:54.257 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:04:54 smithi052 ceph-mon[32362]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-06-10T03:04:54.257 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:04:54 smithi052 ceph-mon[32362]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-06-10T03:04:54.257 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:04:54 smithi052 ceph-mon[32362]: Cluster is now healthy 2024-06-10T03:04:54.257 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:04:54 smithi052 ceph-mon[32362]: from='client.? 172.21.15.52:0/3476972233' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-06-10T03:04:54.257 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:04:54 smithi052 ceph-mon[32362]: fsmap 12 up:standby 2024-06-10T03:04:54.257 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:04:54 smithi052 ceph-mon[32362]: pgmap v756: 129 pgs: 129 active+clean; 228 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-10T03:04:54.389 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:04:54 smithi120 ceph-mon[36634]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-06-10T03:04:54.389 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:04:54 smithi120 ceph-mon[36634]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-06-10T03:04:54.389 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:04:54 smithi120 ceph-mon[36634]: Cluster is now healthy 2024-06-10T03:04:54.389 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:04:54 smithi120 ceph-mon[36634]: from='client.? 172.21.15.52:0/3476972233' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-06-10T03:04:54.389 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:04:54 smithi120 ceph-mon[36634]: fsmap 12 up:standby 2024-06-10T03:04:54.389 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:04:54 smithi120 ceph-mon[36634]: pgmap v756: 129 pgs: 129 active+clean; 228 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-10T03:04:54.454 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:04:54 smithi188 ceph-mon[36599]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-06-10T03:04:54.454 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:04:54 smithi188 ceph-mon[36599]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-06-10T03:04:54.454 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:04:54 smithi188 ceph-mon[36599]: Cluster is now healthy 2024-06-10T03:04:54.454 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:04:54 smithi188 ceph-mon[36599]: from='client.? 172.21.15.52:0/3476972233' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-06-10T03:04:54.454 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:04:54 smithi188 ceph-mon[36599]: fsmap 12 up:standby 2024-06-10T03:04:54.454 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:04:54 smithi188 ceph-mon[36599]: pgmap v756: 129 pgs: 129 active+clean; 228 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-10T03:04:55.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:55.478+0000 7fee8c1f6700 1 -- 172.21.15.52:0/1936763608 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fee8406a170 msgr2=0x7fee8406a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:04:55.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:55.478+0000 7fee8c1f6700 1 --2- 172.21.15.52:0/1936763608 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fee8406a170 0x7fee8406a5f0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fee74009990 tx=0x7fee74009ca0 comp rx=0 tx=0).stop 2024-06-10T03:04:55.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:55.478+0000 7fee8c1f6700 1 -- 172.21.15.52:0/1936763608 shutdown_connections 2024-06-10T03:04:55.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:55.478+0000 7fee8c1f6700 1 --2- 172.21.15.52:0/1936763608 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fee8406ab30 0x7fee8410f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:04:55.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:55.478+0000 7fee8c1f6700 1 --2- 172.21.15.52:0/1936763608 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fee84108500 0x7fee841088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:04:55.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:55.478+0000 7fee8c1f6700 1 --2- 172.21.15.52:0/1936763608 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fee8406a170 0x7fee8406a5f0 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:04:55.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:55.478+0000 7fee8c1f6700 1 -- 172.21.15.52:0/1936763608 >> 172.21.15.52:0/1936763608 conn(0x7fee84078340 msgr2=0x7fee84078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:04:55.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:55.478+0000 7fee8c1f6700 1 -- 172.21.15.52:0/1936763608 shutdown_connections 2024-06-10T03:04:55.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:55.478+0000 7fee8c1f6700 1 -- 172.21.15.52:0/1936763608 wait complete. 2024-06-10T03:04:55.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:55.479+0000 7fee8c1f6700 1 Processor -- start 2024-06-10T03:04:55.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:55.479+0000 7fee8c1f6700 1 -- start start 2024-06-10T03:04:55.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:55.479+0000 7fee8c1f6700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fee8406a170 0x7fee8419ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:04:55.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:55.479+0000 7fee8c1f6700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fee8406ab30 0x7fee8419b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:04:55.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:55.479+0000 7fee8c1f6700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fee84108500 0x7fee8419f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:04:55.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:55.479+0000 7fee8c1f6700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fee841127e0 con 0x7fee8406a170 2024-06-10T03:04:55.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:55.479+0000 7fee8c1f6700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fee84112660 con 0x7fee8406ab30 2024-06-10T03:04:55.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:55.479+0000 7fee8c1f6700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fee84112960 con 0x7fee84108500 2024-06-10T03:04:55.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:55.480+0000 7fee89791700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fee8406ab30 0x7fee8419b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:04:55.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:55.480+0000 7fee89f92700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fee8406a170 0x7fee8419ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:04:55.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:55.480+0000 7fee89791700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fee8406ab30 0x7fee8419b3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.52:47832/0 (socket says 172.21.15.52:47832) 2024-06-10T03:04:55.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:55.480+0000 7fee89791700 1 -- 172.21.15.52:0/2610450737 learned_addr learned my addr 172.21.15.52:0/2610450737 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:04:55.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:55.480+0000 7fee8a793700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fee84108500 0x7fee8419f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:04:55.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:55.480+0000 7fee89f92700 1 --2- 172.21.15.52:0/2610450737 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fee8406a170 0x7fee8419ae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55382/0 (socket says 172.21.15.52:55382) 2024-06-10T03:04:55.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:55.480+0000 7fee89f92700 1 -- 172.21.15.52:0/2610450737 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fee84108500 msgr2=0x7fee8419f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:04:55.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:55.480+0000 7fee89f92700 1 --2- 172.21.15.52:0/2610450737 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fee84108500 0x7fee8419f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:04:55.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:55.480+0000 7fee89f92700 1 -- 172.21.15.52:0/2610450737 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fee8406ab30 msgr2=0x7fee8419b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:04:55.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:55.480+0000 7fee89f92700 1 --2- 172.21.15.52:0/2610450737 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fee8406ab30 0x7fee8419b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:04:55.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:55.480+0000 7fee89f92700 1 -- 172.21.15.52:0/2610450737 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee8419fe80 con 0x7fee8406a170 2024-06-10T03:04:55.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:55.480+0000 7fee89791700 1 --2- 172.21.15.52:0/2610450737 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fee8406ab30 0x7fee8419b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T03:04:55.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:55.480+0000 7fee89f92700 1 --2- 172.21.15.52:0/2610450737 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fee8406a170 0x7fee8419ae70 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fee8000d950 tx=0x7fee8000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:04:55.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:55.480+0000 7fee7affd700 1 -- 172.21.15.52:0/2610450737 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee80016070 con 0x7fee8406a170 2024-06-10T03:04:55.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:55.480+0000 7fee8c1f6700 1 -- 172.21.15.52:0/2610450737 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee841a0170 con 0x7fee8406a170 2024-06-10T03:04:55.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:55.480+0000 7fee7affd700 1 -- 172.21.15.52:0/2610450737 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee8000fd70 con 0x7fee8406a170 2024-06-10T03:04:55.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:55.480+0000 7fee7affd700 1 -- 172.21.15.52:0/2610450737 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee8000e3b0 con 0x7fee8406a170 2024-06-10T03:04:55.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:55.481+0000 7fee8c1f6700 1 -- 172.21.15.52:0/2610450737 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee841a79f0 con 0x7fee8406a170 2024-06-10T03:04:55.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:55.482+0000 7fee8c1f6700 1 -- 172.21.15.52:0/2610450737 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee84069180 con 0x7fee8406a170 2024-06-10T03:04:55.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:55.484+0000 7fee7affd700 1 -- 172.21.15.52:0/2610450737 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fee80019070 con 0x7fee8406a170 2024-06-10T03:04:55.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:55.484+0000 7fee7affd700 1 --2- 172.21.15.52:0/2610450737 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fee700a2c20 0x7fee700a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:04:55.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:55.485+0000 7fee89791700 1 --2- 172.21.15.52:0/2610450737 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fee700a2c20 0x7fee700a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:04:55.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:55.485+0000 7fee7affd700 1 -- 172.21.15.52:0/2610450737 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9609+0+0 (secure 0 0 0) 0x7fee800cef70 con 0x7fee8406a170 2024-06-10T03:04:55.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:55.485+0000 7fee89791700 1 --2- 172.21.15.52:0/2610450737 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fee700a2c20 0x7fee700a50e0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fee841033f0 tx=0x7fee74005c00 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:04:55.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:55.489+0000 7fee7affd700 1 -- 172.21.15.52:0/2610450737 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fee80010460 con 0x7fee8406a170 2024-06-10T03:04:55.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:55.695+0000 7fee8c1f6700 1 -- 172.21.15.52:0/2610450737 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fee8419c1d0 con 0x7fee8406a170 2024-06-10T03:04:56.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:56.325+0000 7fee7affd700 1 -- 172.21.15.52:0/2610450737 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v102) ==== 181+0+0 (secure 0 0 0) 0x7fee8009b970 con 0x7fee8406a170 2024-06-10T03:04:56.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:56.329+0000 7fee8c1f6700 1 -- 172.21.15.52:0/2610450737 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fee700a2c20 msgr2=0x7fee700a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:04:56.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:56.329+0000 7fee8c1f6700 1 --2- 172.21.15.52:0/2610450737 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fee700a2c20 0x7fee700a50e0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fee841033f0 tx=0x7fee74005c00 comp rx=0 tx=0).stop 2024-06-10T03:04:56.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:56.329+0000 7fee8c1f6700 1 -- 172.21.15.52:0/2610450737 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fee8406a170 msgr2=0x7fee8419ae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:04:56.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:56.329+0000 7fee8c1f6700 1 --2- 172.21.15.52:0/2610450737 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fee8406a170 0x7fee8419ae70 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fee8000d950 tx=0x7fee8000dc60 comp rx=0 tx=0).stop 2024-06-10T03:04:56.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:56.330+0000 7fee8c1f6700 1 -- 172.21.15.52:0/2610450737 shutdown_connections 2024-06-10T03:04:56.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:56.330+0000 7fee8c1f6700 1 --2- 172.21.15.52:0/2610450737 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fee84108500 0x7fee8419f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:04:56.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:56.330+0000 7fee8c1f6700 1 --2- 172.21.15.52:0/2610450737 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fee700a2c20 0x7fee700a50e0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:04:56.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:56.330+0000 7fee8c1f6700 1 --2- 172.21.15.52:0/2610450737 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fee8406a170 0x7fee8419ae70 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:04:56.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:56.330+0000 7fee8c1f6700 1 --2- 172.21.15.52:0/2610450737 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fee8406ab30 0x7fee8419b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:04:56.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:56.330+0000 7fee8c1f6700 1 -- 172.21.15.52:0/2610450737 >> 172.21.15.52:0/2610450737 conn(0x7fee84078340 msgr2=0x7fee840ffdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:04:56.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:56.330+0000 7fee8c1f6700 1 -- 172.21.15.52:0/2610450737 shutdown_connections 2024-06-10T03:04:56.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:56.330+0000 7fee8c1f6700 1 -- 172.21.15.52:0/2610450737 wait complete. 2024-06-10T03:04:56.332 INFO:teuthology.orchestra.run.smithi052.stderr:pool 'cephfs_metadata' removed 2024-06-10T03:04:56.638 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:04:56 smithi120 ceph-mon[36634]: pgmap v757: 129 pgs: 129 active+clean; 228 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-10T03:04:57.166 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:04:56 smithi120 ceph-mon[36634]: from='client.? 172.21.15.52:0/2610450737' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-10T03:04:57.168 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:04:56 smithi188 ceph-mon[36599]: pgmap v757: 129 pgs: 129 active+clean; 228 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-10T03:04:57.168 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:04:56 smithi188 ceph-mon[36599]: from='client.? 172.21.15.52:0/2610450737' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-10T03:04:57.168 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:04:56 smithi052 ceph-mon[32362]: pgmap v757: 129 pgs: 129 active+clean; 228 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-10T03:04:57.168 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:04:56 smithi052 ceph-mon[32362]: from='client.? 172.21.15.52:0/2610450737' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-10T03:04:57.169 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-06-10T03:04:57.426 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T03:04:57.629 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:04:57 smithi052 ceph-mon[32362]: from='client.? 172.21.15.52:0/2610450737' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-06-10T03:04:57.629 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:04:57 smithi052 ceph-mon[32362]: osdmap e102: 12 total, 12 up, 12 in 2024-06-10T03:04:57.638 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:04:57 smithi120 ceph-mon[36634]: from='client.? 172.21.15.52:0/2610450737' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-06-10T03:04:57.638 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:04:57 smithi120 ceph-mon[36634]: osdmap e102: 12 total, 12 up, 12 in 2024-06-10T03:04:57.704 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:04:57 smithi188 ceph-mon[36599]: from='client.? 172.21.15.52:0/2610450737' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-06-10T03:04:57.705 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:04:57 smithi188 ceph-mon[36599]: osdmap e102: 12 total, 12 up, 12 in 2024-06-10T03:04:58.471 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:04:58 smithi052 ceph-mon[32362]: pgmap v759: 97 pgs: 97 active+clean; 577 KiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T03:04:58.638 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:04:58 smithi120 ceph-mon[36634]: pgmap v759: 97 pgs: 97 active+clean; 577 KiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T03:04:58.704 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:04:58 smithi188 ceph-mon[36599]: pgmap v759: 97 pgs: 97 active+clean; 577 KiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T03:04:59.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:58.985+0000 7f998649f700 1 -- 172.21.15.52:0/1345320237 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9980108500 msgr2=0x7f99801088e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:04:59.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:58.985+0000 7f998649f700 1 --2- 172.21.15.52:0/1345320237 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9980108500 0x7f99801088e0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f9970009990 tx=0x7f9970009ca0 comp rx=0 tx=0).stop 2024-06-10T03:04:59.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:58.985+0000 7f998649f700 1 -- 172.21.15.52:0/1345320237 shutdown_connections 2024-06-10T03:04:59.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:58.985+0000 7f998649f700 1 --2- 172.21.15.52:0/1345320237 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f998006ab30 0x7f998010f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:04:59.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:58.985+0000 7f998649f700 1 --2- 172.21.15.52:0/1345320237 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9980108500 0x7f99801088e0 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:04:59.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:58.985+0000 7f998649f700 1 --2- 172.21.15.52:0/1345320237 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f998006a170 0x7f998006a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:04:59.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:58.985+0000 7f998649f700 1 -- 172.21.15.52:0/1345320237 >> 172.21.15.52:0/1345320237 conn(0x7f9980078340 msgr2=0x7f9980078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:04:59.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:58.985+0000 7f998649f700 1 -- 172.21.15.52:0/1345320237 shutdown_connections 2024-06-10T03:04:59.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:58.985+0000 7f998649f700 1 -- 172.21.15.52:0/1345320237 wait complete. 2024-06-10T03:04:59.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:58.985+0000 7f998649f700 1 Processor -- start 2024-06-10T03:04:59.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:58.985+0000 7f998649f700 1 -- start start 2024-06-10T03:04:59.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:58.986+0000 7f998649f700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f998006a170 0x7f9980101460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:04:59.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:58.986+0000 7f998649f700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f998006ab30 0x7f99801019a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:04:59.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:58.986+0000 7f998649f700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9980108500 0x7f9980101ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:04:59.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:58.986+0000 7f998649f700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f9980112830 con 0x7f998006ab30 2024-06-10T03:04:59.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:58.986+0000 7f998649f700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f99801126b0 con 0x7f9980108500 2024-06-10T03:04:59.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:58.986+0000 7f998649f700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f99801129b0 con 0x7f998006a170 2024-06-10T03:04:59.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:58.986+0000 7f997f7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f998006ab30 0x7f99801019a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:04:59.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:58.986+0000 7f997f7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f998006ab30 0x7f99801019a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55396/0 (socket says 172.21.15.52:55396) 2024-06-10T03:04:59.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:58.986+0000 7f997f7fe700 1 -- 172.21.15.52:0/3388364639 learned_addr learned my addr 172.21.15.52:0/3388364639 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:04:59.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:58.986+0000 7f997ffff700 1 --2- 172.21.15.52:0/3388364639 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f998006a170 0x7f9980101460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:04:59.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:58.986+0000 7f9984a3c700 1 --2- 172.21.15.52:0/3388364639 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9980108500 0x7f9980101ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:04:59.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:58.986+0000 7f997f7fe700 1 -- 172.21.15.52:0/3388364639 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f998006a170 msgr2=0x7f9980101460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:04:59.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:58.986+0000 7f997f7fe700 1 --2- 172.21.15.52:0/3388364639 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f998006a170 0x7f9980101460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:04:59.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:58.986+0000 7f997f7fe700 1 -- 172.21.15.52:0/3388364639 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9980108500 msgr2=0x7f9980101ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:04:59.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:58.986+0000 7f997f7fe700 1 --2- 172.21.15.52:0/3388364639 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9980108500 0x7f9980101ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:04:59.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:58.986+0000 7f997f7fe700 1 -- 172.21.15.52:0/3388364639 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99801a7270 con 0x7f998006ab30 2024-06-10T03:04:59.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:58.986+0000 7f997ffff700 1 --2- 172.21.15.52:0/3388364639 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f998006a170 0x7f9980101460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T03:04:59.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:58.986+0000 7f9984a3c700 1 --2- 172.21.15.52:0/3388364639 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9980108500 0x7f9980101ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T03:04:59.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:58.987+0000 7f997f7fe700 1 --2- 172.21.15.52:0/3388364639 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f998006ab30 0x7f99801019a0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f997400c920 tx=0x7f997400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:04:59.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:58.987+0000 7f997d7fa700 1 -- 172.21.15.52:0/3388364639 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9974007b00 con 0x7f998006ab30 2024-06-10T03:04:59.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:58.987+0000 7f997d7fa700 1 -- 172.21.15.52:0/3388364639 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9974004d10 con 0x7f998006ab30 2024-06-10T03:04:59.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:58.987+0000 7f998649f700 1 -- 172.21.15.52:0/3388364639 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99801a7560 con 0x7f998006ab30 2024-06-10T03:04:59.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:58.987+0000 7f997d7fa700 1 -- 172.21.15.52:0/3388364639 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f997400f450 con 0x7f998006ab30 2024-06-10T03:04:59.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:58.987+0000 7f998649f700 1 -- 172.21.15.52:0/3388364639 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99801a7aa0 con 0x7f998006ab30 2024-06-10T03:04:59.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:58.988+0000 7f998649f700 1 -- 172.21.15.52:0/3388364639 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f998010a9e0 con 0x7f998006ab30 2024-06-10T03:04:59.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:58.989+0000 7f997d7fa700 1 -- 172.21.15.52:0/3388364639 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f9974004750 con 0x7f998006ab30 2024-06-10T03:04:59.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:58.989+0000 7f997d7fa700 1 --2- 172.21.15.52:0/3388364639 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f99680a2c20 0x7f99680a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:04:59.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:58.990+0000 7f997ffff700 1 --2- 172.21.15.52:0/3388364639 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f99680a2c20 0x7f99680a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:04:59.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:58.990+0000 7f997d7fa700 1 -- 172.21.15.52:0/3388364639 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9084+0+0 (secure 0 0 0) 0x7f99740cf5a0 con 0x7f998006ab30 2024-06-10T03:04:59.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:58.990+0000 7f997ffff700 1 --2- 172.21.15.52:0/3388364639 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f99680a2c20 0x7f99680a50e0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f9970009750 tx=0x7f9970005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:04:59.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:58.993+0000 7f997d7fa700 1 -- 172.21.15.52:0/3388364639 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f9974014030 con 0x7f998006ab30 2024-06-10T03:04:59.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:59.200+0000 7f998649f700 1 -- 172.21.15.52:0/3388364639 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f9980069180 con 0x7f998006ab30 2024-06-10T03:04:59.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:59.337+0000 7f997d7fa700 1 -- 172.21.15.52:0/3388364639 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v103) ==== 169+0+0 (secure 0 0 0) 0x7f997409c1b0 con 0x7f998006ab30 2024-06-10T03:04:59.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:59.341+0000 7f998649f700 1 -- 172.21.15.52:0/3388364639 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f99680a2c20 msgr2=0x7f99680a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:04:59.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:59.341+0000 7f998649f700 1 --2- 172.21.15.52:0/3388364639 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f99680a2c20 0x7f99680a50e0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f9970009750 tx=0x7f9970005fd0 comp rx=0 tx=0).stop 2024-06-10T03:04:59.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:59.341+0000 7f998649f700 1 -- 172.21.15.52:0/3388364639 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f998006ab30 msgr2=0x7f99801019a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:04:59.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:59.341+0000 7f998649f700 1 --2- 172.21.15.52:0/3388364639 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f998006ab30 0x7f99801019a0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f997400c920 tx=0x7f997400cce0 comp rx=0 tx=0).stop 2024-06-10T03:04:59.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:59.341+0000 7f998649f700 1 -- 172.21.15.52:0/3388364639 shutdown_connections 2024-06-10T03:04:59.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:59.341+0000 7f998649f700 1 --2- 172.21.15.52:0/3388364639 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9980108500 0x7f9980101ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:04:59.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:59.341+0000 7f998649f700 1 --2- 172.21.15.52:0/3388364639 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f998006a170 0x7f9980101460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:04:59.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:59.341+0000 7f998649f700 1 --2- 172.21.15.52:0/3388364639 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7f99680a2c20 0x7f99680a50e0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:04:59.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:59.341+0000 7f998649f700 1 --2- 172.21.15.52:0/3388364639 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f998006ab30 0x7f99801019a0 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:04:59.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:59.341+0000 7f998649f700 1 -- 172.21.15.52:0/3388364639 >> 172.21.15.52:0/3388364639 conn(0x7f9980078340 msgr2=0x7f9980100050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:04:59.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:59.342+0000 7f998649f700 1 -- 172.21.15.52:0/3388364639 shutdown_connections 2024-06-10T03:04:59.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:04:59.342+0000 7f998649f700 1 -- 172.21.15.52:0/3388364639 wait complete. 2024-06-10T03:04:59.343 INFO:teuthology.orchestra.run.smithi052.stderr:pool 'cephfs_data' removed 2024-06-10T03:04:59.639 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:04:59 smithi120 ceph-mon[36634]: from='client.? 172.21.15.52:0/3388364639' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-10T03:04:59.704 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:04:59 smithi188 ceph-mon[36599]: from='client.? 172.21.15.52:0/3388364639' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-10T03:04:59.727 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:04:59 smithi052 ceph-mon[32362]: from='client.? 172.21.15.52:0/3388364639' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-10T03:04:59.984 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 -- ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-06-10T03:05:00.236 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T03:05:00.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:05:00 smithi052 ceph-mon[32362]: pgmap v760: 97 pgs: 97 active+clean; 577 KiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T03:05:00.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:05:00 smithi052 ceph-mon[32362]: from='client.? 172.21.15.52:0/3388364639' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-06-10T03:05:00.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:05:00 smithi052 ceph-mon[32362]: osdmap e103: 12 total, 12 up, 12 in 2024-06-10T03:05:00.638 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:05:00 smithi120 ceph-mon[36634]: pgmap v760: 97 pgs: 97 active+clean; 577 KiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T03:05:00.638 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:05:00 smithi120 ceph-mon[36634]: from='client.? 172.21.15.52:0/3388364639' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-06-10T03:05:00.639 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:05:00 smithi120 ceph-mon[36634]: osdmap e103: 12 total, 12 up, 12 in 2024-06-10T03:05:00.704 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:05:00 smithi188 ceph-mon[36599]: pgmap v760: 97 pgs: 97 active+clean; 577 KiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T03:05:00.704 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:05:00 smithi188 ceph-mon[36599]: from='client.? 172.21.15.52:0/3388364639' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-06-10T03:05:00.704 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:05:00 smithi188 ceph-mon[36599]: osdmap e103: 12 total, 12 up, 12 in 2024-06-10T03:05:01.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:05:01.946+0000 7fb393a28700 1 -- 172.21.15.52:0/1130464617 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb38c107840 msgr2=0x7fb38c10f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:05:01.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:05:01.946+0000 7fb393a28700 1 --2- 172.21.15.52:0/1130464617 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb38c107840 0x7fb38c10f3c0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7fb38400b0d0 tx=0x7fb38400b3e0 comp rx=0 tx=0).stop 2024-06-10T03:05:01.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:05:01.946+0000 7fb393a28700 1 -- 172.21.15.52:0/1130464617 shutdown_connections 2024-06-10T03:05:01.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:05:01.946+0000 7fb393a28700 1 --2- 172.21.15.52:0/1130464617 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb38c107840 0x7fb38c10f3c0 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:05:01.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:05:01.946+0000 7fb393a28700 1 --2- 172.21.15.52:0/1130464617 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb38c102740 0x7fb38c102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:05:01.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:05:01.946+0000 7fb393a28700 1 --2- 172.21.15.52:0/1130464617 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb38c1030f0 0x7fb38c107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:05:01.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:05:01.946+0000 7fb393a28700 1 -- 172.21.15.52:0/1130464617 >> 172.21.15.52:0/1130464617 conn(0x7fb38c0795f0 msgr2=0x7fb38c079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:05:01.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:05:01.946+0000 7fb393a28700 1 -- 172.21.15.52:0/1130464617 shutdown_connections 2024-06-10T03:05:01.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:05:01.946+0000 7fb393a28700 1 -- 172.21.15.52:0/1130464617 wait complete. 2024-06-10T03:05:01.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:05:01.947+0000 7fb393a28700 1 Processor -- start 2024-06-10T03:05:01.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:05:01.947+0000 7fb393a28700 1 -- start start 2024-06-10T03:05:01.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:05:01.947+0000 7fb393a28700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb38c102740 0x7fb38c074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:05:01.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:05:01.947+0000 7fb393a28700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb38c1030f0 0x7fb38c074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:05:01.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:05:01.947+0000 7fb393a28700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb38c107840 0x7fb38c075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:05:01.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:05:01.947+0000 7fb393a28700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb38c112760 con 0x7fb38c1030f0 2024-06-10T03:05:01.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:05:01.947+0000 7fb393a28700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fb38c1125e0 con 0x7fb38c102740 2024-06-10T03:05:01.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:05:01.947+0000 7fb393a28700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fb38c1128e0 con 0x7fb38c107840 2024-06-10T03:05:01.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:05:01.947+0000 7fb390fc3700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb38c1030f0 0x7fb38c074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:05:01.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:05:01.947+0000 7fb390fc3700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb38c1030f0 0x7fb38c074cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36814/0 (socket says 172.21.15.52:36814) 2024-06-10T03:05:01.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:05:01.947+0000 7fb390fc3700 1 -- 172.21.15.52:0/1818298657 learned_addr learned my addr 172.21.15.52:0/1818298657 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-10T03:05:01.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:05:01.947+0000 7fb391fc5700 1 --2- 172.21.15.52:0/1818298657 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb38c107840 0x7fb38c075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:05:01.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:05:01.947+0000 7fb3917c4700 1 --2- 172.21.15.52:0/1818298657 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb38c102740 0x7fb38c074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:05:01.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:05:01.947+0000 7fb390fc3700 1 -- 172.21.15.52:0/1818298657 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb38c107840 msgr2=0x7fb38c075210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:05:01.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:05:01.947+0000 7fb390fc3700 1 --2- 172.21.15.52:0/1818298657 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb38c107840 0x7fb38c075210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:05:01.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:05:01.947+0000 7fb390fc3700 1 -- 172.21.15.52:0/1818298657 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb38c102740 msgr2=0x7fb38c074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:05:01.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:05:01.947+0000 7fb390fc3700 1 --2- 172.21.15.52:0/1818298657 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb38c102740 0x7fb38c074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:05:01.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:05:01.947+0000 7fb390fc3700 1 -- 172.21.15.52:0/1818298657 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb38c1a7080 con 0x7fb38c1030f0 2024-06-10T03:05:01.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:05:01.948+0000 7fb390fc3700 1 --2- 172.21.15.52:0/1818298657 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb38c1030f0 0x7fb38c074cd0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fb37c00d950 tx=0x7fb37c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:05:01.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:05:01.948+0000 7fb3827fc700 1 -- 172.21.15.52:0/1818298657 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb37c016070 con 0x7fb38c1030f0 2024-06-10T03:05:01.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:05:01.948+0000 7fb393a28700 1 -- 172.21.15.52:0/1818298657 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb38c1a7370 con 0x7fb38c1030f0 2024-06-10T03:05:01.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:05:01.948+0000 7fb3827fc700 1 -- 172.21.15.52:0/1818298657 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb37c00fd70 con 0x7fb38c1030f0 2024-06-10T03:05:01.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:05:01.948+0000 7fb3827fc700 1 -- 172.21.15.52:0/1818298657 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb37c00e610 con 0x7fb38c1030f0 2024-06-10T03:05:01.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:05:01.948+0000 7fb393a28700 1 -- 172.21.15.52:0/1818298657 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb38c1a78b0 con 0x7fb38c1030f0 2024-06-10T03:05:01.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:05:01.949+0000 7fb393a28700 1 -- 172.21.15.52:0/1818298657 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb38c069180 con 0x7fb38c1030f0 2024-06-10T03:05:01.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:05:01.950+0000 7fb3827fc700 1 -- 172.21.15.52:0/1818298657 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fb37c019070 con 0x7fb38c1030f0 2024-06-10T03:05:01.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:05:01.951+0000 7fb3827fc700 1 --2- 172.21.15.52:0/1818298657 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fb3780a2c20 0x7fb3780a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T03:05:01.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:05:01.951+0000 7fb3917c4700 1 --2- 172.21.15.52:0/1818298657 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fb3780a2c20 0x7fb3780a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T03:05:01.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:05:01.951+0000 7fb3827fc700 1 -- 172.21.15.52:0/1818298657 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8496+0+0 (secure 0 0 0) 0x7fb37c0ceb90 con 0x7fb38c1030f0 2024-06-10T03:05:01.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:05:01.951+0000 7fb3917c4700 1 --2- 172.21.15.52:0/1818298657 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fb3780a2c20 0x7fb3780a50e0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fb388005950 tx=0x7fb38800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T03:05:01.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:05:01.955+0000 7fb3827fc700 1 -- 172.21.15.52:0/1818298657 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fb37c09b9b0 con 0x7fb38c1030f0 2024-06-10T03:05:02.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:05:02.165+0000 7fb393a28700 1 -- 172.21.15.52:0/1818298657 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) -- 0x7fb38c02dca0 con 0x7fb38c1030f0 2024-06-10T03:05:02.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:05:02.341+0000 7fb3827fc700 1 -- 172.21.15.52:0/1818298657 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v104) ==== 178+0+0 (secure 0 0 0) 0x7fb37c095e20 con 0x7fb38c1030f0 2024-06-10T03:05:02.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:05:02.345+0000 7fb393a28700 1 -- 172.21.15.52:0/1818298657 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fb3780a2c20 msgr2=0x7fb3780a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:05:02.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:05:02.345+0000 7fb393a28700 1 --2- 172.21.15.52:0/1818298657 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fb3780a2c20 0x7fb3780a50e0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fb388005950 tx=0x7fb38800b410 comp rx=0 tx=0).stop 2024-06-10T03:05:02.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:05:02.345+0000 7fb393a28700 1 -- 172.21.15.52:0/1818298657 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb38c1030f0 msgr2=0x7fb38c074cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T03:05:02.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:05:02.345+0000 7fb393a28700 1 --2- 172.21.15.52:0/1818298657 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb38c1030f0 0x7fb38c074cd0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fb37c00d950 tx=0x7fb37c00dc60 comp rx=0 tx=0).stop 2024-06-10T03:05:02.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:05:02.345+0000 7fb393a28700 1 -- 172.21.15.52:0/1818298657 shutdown_connections 2024-06-10T03:05:02.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:05:02.345+0000 7fb393a28700 1 --2- 172.21.15.52:0/1818298657 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb38c107840 0x7fb38c075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:05:02.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:05:02.345+0000 7fb393a28700 1 --2- 172.21.15.52:0/1818298657 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb38c102740 0x7fb38c074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:05:02.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:05:02.345+0000 7fb393a28700 1 --2- 172.21.15.52:0/1818298657 >> [v2:172.21.15.52:6800/3214763969,v1:172.21.15.52:6801/3214763969] conn(0x7fb3780a2c20 0x7fb3780a50e0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:05:02.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:05:02.345+0000 7fb393a28700 1 --2- 172.21.15.52:0/1818298657 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb38c1030f0 0x7fb38c074cd0 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T03:05:02.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:05:02.345+0000 7fb393a28700 1 -- 172.21.15.52:0/1818298657 >> 172.21.15.52:0/1818298657 conn(0x7fb38c0795f0 msgr2=0x7fb38c100490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T03:05:02.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:05:02.346+0000 7fb393a28700 1 -- 172.21.15.52:0/1818298657 shutdown_connections 2024-06-10T03:05:02.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-10T03:05:02.346+0000 7fb393a28700 1 -- 172.21.15.52:0/1818298657 wait complete. 2024-06-10T03:05:02.353 INFO:teuthology.orchestra.run.smithi052.stderr:pool 'cephfs_data_ec' removed 2024-06-10T03:05:02.638 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:05:02 smithi120 ceph-mon[36634]: pgmap v762: 33 pgs: 33 active+clean; 577 KiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T03:05:02.639 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:05:02 smithi120 ceph-mon[36634]: from='client.? 172.21.15.52:0/1818298657' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-10T03:05:02.663 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:05:02 smithi052 ceph-mon[32362]: pgmap v762: 33 pgs: 33 active+clean; 577 KiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T03:05:02.663 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:05:02 smithi052 ceph-mon[32362]: from='client.? 172.21.15.52:0/1818298657' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-10T03:05:02.704 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:05:02 smithi188 ceph-mon[36599]: pgmap v762: 33 pgs: 33 active+clean; 577 KiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T03:05:02.704 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:05:02 smithi188 ceph-mon[36599]: from='client.? 172.21.15.52:0/1818298657' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-10T03:05:02.919 DEBUG:teuthology.orchestra.run.smithi052:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-06-10T03:05:02.950 DEBUG:teuthology.orchestra.run.smithi120:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-06-10T03:05:02.979 DEBUG:teuthology.orchestra.run.smithi188:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-06-10T03:05:03.006 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-06-10T03:05:03.006 DEBUG:teuthology.orchestra.run.smithi052:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-06-10T03:05:03.023 DEBUG:teuthology.orchestra.run.smithi120:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-06-10T03:05:03.039 DEBUG:teuthology.orchestra.run.smithi188:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-06-10T03:05:03.061 INFO:tasks.cephadm:Stopping all daemons... 2024-06-10T03:05:03.062 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-06-10T03:05:03.062 DEBUG:teuthology.orchestra.run.smithi052:> sudo systemctl stop ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968@mon.a 2024-06-10T03:05:03.227 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:05:03 smithi052 systemd[1]: Stopping Ceph mon.a for d8676e9e-26d2-11ef-bc9b-c7b262605968... 2024-06-10T03:05:03.639 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:05:03 smithi120 ceph-mon[36634]: from='client.? 172.21.15.52:0/1818298657' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-06-10T03:05:03.639 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:05:03 smithi120 ceph-mon[36634]: osdmap e104: 12 total, 12 up, 12 in 2024-06-10T03:05:03.704 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:05:03 smithi188 ceph-mon[36599]: from='client.? 172.21.15.52:0/1818298657' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-06-10T03:05:03.704 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:05:03 smithi188 ceph-mon[36599]: osdmap e104: 12 total, 12 up, 12 in 2024-06-10T03:05:03.727 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:05:03 smithi052 ceph-mon[32362]: from='client.? 172.21.15.52:0/1818298657' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-06-10T03:05:03.727 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:05:03 smithi052 ceph-mon[32362]: osdmap e104: 12 total, 12 up, 12 in 2024-06-10T03:05:03.727 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:05:03 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mon-a[32358]: 2024-06-10T03:05:03.393+0000 7f2d4127b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-06-10T03:05:03.727 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:05:03 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mon-a[32358]: 2024-06-10T03:05:03.393+0000 7f2d4127b700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-06-10T03:05:03.727 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:05:03 smithi052 podman[120191]: 2024-06-10 03:05:03.456247357 +0000 UTC m=+0.180732620 container died cb4e54e38ee793a7d4a80b5b4ec7bed9005451170e7a0aaaaa917e8fddd8cc68 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-06-10T03:05:04.054 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:05:03 smithi052 podman[120191]: 2024-06-10 03:05:03.852949908 +0000 UTC m=+0.577435170 container cleanup cb4e54e38ee793a7d4a80b5b4ec7bed9005451170e7a0aaaaa917e8fddd8cc68 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mon-a, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, ceph=True, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-10T03:05:04.054 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:05:03 smithi052 bash[120191]: ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mon-a 2024-06-10T03:05:04.299 DEBUG:teuthology.orchestra.run.smithi052:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968@mon.a.service' 2024-06-10T03:05:04.323 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:05:04 smithi052 podman[120204]: 2024-06-10 03:05:04.05497538 +0000 UTC m=+0.595254176 container remove cb4e54e38ee793a7d4a80b5b4ec7bed9005451170e7a0aaaaa917e8fddd8cc68 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mon-a, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-10T03:05:04.323 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:05:04 smithi052 systemd[1]: ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968@mon.a.service: Deactivated successfully. 2024-06-10T03:05:04.323 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:05:04 smithi052 systemd[1]: Stopped Ceph mon.a for d8676e9e-26d2-11ef-bc9b-c7b262605968. 2024-06-10T03:05:04.323 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 10 03:05:04 smithi052 systemd[1]: ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968@mon.a.service: Consumed 23.005s CPU time. 2024-06-10T03:05:04.353 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-10T03:05:04.354 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-06-10T03:05:04.354 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-06-10T03:05:04.354 DEBUG:teuthology.orchestra.run.smithi120:> sudo systemctl stop ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968@mon.b 2024-06-10T03:05:04.638 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:05:04 smithi120 systemd[1]: Stopping Ceph mon.b for d8676e9e-26d2-11ef-bc9b-c7b262605968... 2024-06-10T03:05:05.019 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:05:04 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mon-b[36630]: 2024-06-10T03:05:04.658+0000 7fe4b21a3700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-06-10T03:05:05.019 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:05:04 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mon-b[36630]: 2024-06-10T03:05:04.658+0000 7fe4b21a3700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-06-10T03:05:05.019 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:05:04 smithi120 podman[70972]: 2024-06-10 03:05:04.720411891 +0000 UTC m=+0.183550605 container died c34e60e12cac7a99d83e99d71989d10cfa4994bfd7deb104b9938eee1f63a7e2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mon-b, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-06-10T03:05:05.389 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:05:05 smithi120 podman[70972]: 2024-06-10 03:05:05.110578456 +0000 UTC m=+0.573717170 container cleanup c34e60e12cac7a99d83e99d71989d10cfa4994bfd7deb104b9938eee1f63a7e2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mon-b, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True) 2024-06-10T03:05:05.389 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 10 03:05:05 smithi120 bash[70972]: ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mon-b 2024-06-10T03:05:05.520 DEBUG:teuthology.orchestra.run.smithi120:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968@mon.b.service' 2024-06-10T03:05:05.566 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-10T03:05:05.567 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-06-10T03:05:05.570 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-06-10T03:05:05.570 DEBUG:teuthology.orchestra.run.smithi188:> sudo systemctl stop ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968@mon.c 2024-06-10T03:05:05.954 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:05:05 smithi188 systemd[1]: Stopping Ceph mon.c for d8676e9e-26d2-11ef-bc9b-c7b262605968... 2024-06-10T03:05:06.432 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:05:05 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mon-c[36584]: 2024-06-10T03:05:05.955+0000 7ff2e4c20700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-06-10T03:05:06.432 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:05:05 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mon-c[36584]: 2024-06-10T03:05:05.955+0000 7ff2e4c20700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-06-10T03:05:06.433 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:05:06 smithi188 podman[71692]: 2024-06-10 03:05:06.098914763 +0000 UTC m=+0.271528599 container died 2e0de8d1db4b3907f56c38ad6174815240014147a85f00d9d3f028ed7209930b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mon-c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=squid-3d89acc) 2024-06-10T03:05:06.704 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:05:06 smithi188 podman[71692]: 2024-06-10 03:05:06.523292035 +0000 UTC m=+0.695905912 container cleanup 2e0de8d1db4b3907f56c38ad6174815240014147a85f00d9d3f028ed7209930b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mon-c, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-10T03:05:06.704 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:05:06 smithi188 bash[71692]: ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mon-c 2024-06-10T03:05:07.017 DEBUG:teuthology.orchestra.run.smithi188:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968@mon.c.service' 2024-06-10T03:05:07.038 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:05:06 smithi188 podman[71706]: 2024-06-10 03:05:06.78341319 +0000 UTC m=+0.677294864 container remove 2e0de8d1db4b3907f56c38ad6174815240014147a85f00d9d3f028ed7209930b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mon-c, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240528, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True) 2024-06-10T03:05:07.038 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:05:07 smithi188 systemd[1]: ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968@mon.c.service: Deactivated successfully. 2024-06-10T03:05:07.038 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:05:07 smithi188 systemd[1]: Stopped Ceph mon.c for d8676e9e-26d2-11ef-bc9b-c7b262605968. 2024-06-10T03:05:07.038 INFO:journalctl@ceph.mon.c.smithi188.stdout:Jun 10 03:05:07 smithi188 systemd[1]: ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968@mon.c.service: Consumed 13.442s CPU time. 2024-06-10T03:05:07.066 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-10T03:05:07.067 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-06-10T03:05:07.067 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-06-10T03:05:07.067 DEBUG:teuthology.orchestra.run.smithi052:> sudo systemctl stop ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968@mgr.x 2024-06-10T03:05:07.426 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 10 03:05:07 smithi052 systemd[1]: Stopping Ceph mgr.x for d8676e9e-26d2-11ef-bc9b-c7b262605968... 2024-06-10T03:05:07.692 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 10 03:05:07 smithi052 podman[120311]: 2024-06-10 03:05:07.431226688 +0000 UTC m=+0.159683186 container died c194861d5c8bae344a78836907d4a118fc2c6cbfd0e5fc5a27e6dc0bb5527d0e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mgr-x, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS) 2024-06-10T03:05:07.977 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 10 03:05:07 smithi052 podman[120311]: 2024-06-10 03:05:07.901016388 +0000 UTC m=+0.629472875 container cleanup c194861d5c8bae344a78836907d4a118fc2c6cbfd0e5fc5a27e6dc0bb5527d0e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mgr-x, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-06-10T03:05:07.977 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 10 03:05:07 smithi052 bash[120311]: ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mgr-x 2024-06-10T03:05:08.342 DEBUG:teuthology.orchestra.run.smithi052:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968@mgr.x.service' 2024-06-10T03:05:08.378 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-10T03:05:08.379 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-06-10T03:05:08.379 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-06-10T03:05:08.379 DEBUG:teuthology.orchestra.run.smithi120:> sudo systemctl stop ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968@mgr.y 2024-06-10T03:05:08.638 INFO:journalctl@ceph.mgr.y.smithi120.stdout:Jun 10 03:05:08 smithi120 systemd[1]: Stopping Ceph mgr.y for d8676e9e-26d2-11ef-bc9b-c7b262605968... 2024-06-10T03:05:09.034 INFO:journalctl@ceph.mgr.y.smithi120.stdout:Jun 10 03:05:08 smithi120 podman[71096]: 2024-06-10 03:05:08.720417898 +0000 UTC m=+0.154727360 container died cb5986bc30395ad0b53fbb7fbb75b1e9fe086e6edc0982ea518727b5a25341a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mgr-y, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-06-10T03:05:09.369 INFO:journalctl@ceph.mgr.y.smithi120.stdout:Jun 10 03:05:09 smithi120 podman[71096]: 2024-06-10 03:05:09.134206819 +0000 UTC m=+0.568516264 container cleanup cb5986bc30395ad0b53fbb7fbb75b1e9fe086e6edc0982ea518727b5a25341a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mgr-y, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, ceph=True, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-06-10T03:05:09.369 INFO:journalctl@ceph.mgr.y.smithi120.stdout:Jun 10 03:05:09 smithi120 bash[71096]: ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mgr-y 2024-06-10T03:05:09.513 DEBUG:teuthology.orchestra.run.smithi120:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968@mgr.y.service' 2024-06-10T03:05:09.546 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-10T03:05:09.546 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-06-10T03:05:09.546 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-06-10T03:05:09.546 DEBUG:teuthology.orchestra.run.smithi188:> sudo systemctl stop ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968@mgr.z 2024-06-10T03:05:09.874 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Jun 10 03:05:09 smithi188 systemd[1]: Stopping Ceph mgr.z for d8676e9e-26d2-11ef-bc9b-c7b262605968... 2024-06-10T03:05:10.189 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Jun 10 03:05:09 smithi188 podman[71815]: 2024-06-10 03:05:09.874283111 +0000 UTC m=+0.133517802 container died 2ede476ce097933a1031babcad5a744c62310e7ba1669bc78aaf5349564a5d40 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mgr-z, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-10T03:05:10.453 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Jun 10 03:05:10 smithi188 podman[71815]: 2024-06-10 03:05:10.296232756 +0000 UTC m=+0.555467437 container cleanup 2ede476ce097933a1031babcad5a744c62310e7ba1669bc78aaf5349564a5d40 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mgr-z, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-10T03:05:10.454 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Jun 10 03:05:10 smithi188 bash[71815]: ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mgr-z 2024-06-10T03:05:10.760 DEBUG:teuthology.orchestra.run.smithi188:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968@mgr.z.service' 2024-06-10T03:05:10.794 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-10T03:05:10.794 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-06-10T03:05:10.794 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-06-10T03:05:10.794 DEBUG:teuthology.orchestra.run.smithi052:> sudo systemctl stop ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968@osd.0 2024-06-10T03:05:11.126 INFO:journalctl@ceph.osd.0.smithi052.stdout:Jun 10 03:05:10 smithi052 systemd[1]: Stopping Ceph osd.0 for d8676e9e-26d2-11ef-bc9b-c7b262605968... 2024-06-10T03:05:11.477 INFO:journalctl@ceph.osd.0.smithi052.stdout:Jun 10 03:05:11 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-0[46981]: 2024-06-10T03:05:11.122+0000 7f280c85a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-10T03:05:11.477 INFO:journalctl@ceph.osd.0.smithi052.stdout:Jun 10 03:05:11 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-0[46981]: 2024-06-10T03:05:11.122+0000 7f280c85a700 -1 osd.0 104 *** Got signal Terminated *** 2024-06-10T03:05:11.477 INFO:journalctl@ceph.osd.0.smithi052.stdout:Jun 10 03:05:11 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-0[46981]: 2024-06-10T03:05:11.122+0000 7f280c85a700 -1 osd.0 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-10T03:05:16.841 INFO:journalctl@ceph.osd.0.smithi052.stdout:Jun 10 03:05:16 smithi052 podman[120433]: 2024-06-10 03:05:16.541403008 +0000 UTC m=+5.546848325 container died c48f3cd30f8808f81669e3207a3359e3d5d98e77d741896c6466ca6d8c8be1f0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-06-10T03:05:17.227 INFO:journalctl@ceph.osd.0.smithi052.stdout:Jun 10 03:05:17 smithi052 podman[120433]: 2024-06-10 03:05:17.040449529 +0000 UTC m=+6.045894845 container cleanup c48f3cd30f8808f81669e3207a3359e3d5d98e77d741896c6466ca6d8c8be1f0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-0, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-10T03:05:17.227 INFO:journalctl@ceph.osd.0.smithi052.stdout:Jun 10 03:05:17 smithi052 bash[120433]: ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-0 2024-06-10T03:05:17.727 INFO:journalctl@ceph.osd.0.smithi052.stdout:Jun 10 03:05:17 smithi052 podman[120449]: 2024-06-10 03:05:17.401228286 +0000 UTC m=+0.858593534 container remove c48f3cd30f8808f81669e3207a3359e3d5d98e77d741896c6466ca6d8c8be1f0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-0, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-06-10T03:05:18.046 INFO:journalctl@ceph.osd.0.smithi052.stdout:Jun 10 03:05:17 smithi052 podman[120541]: 2024-06-10 03:05:17.643626604 +0000 UTC m=+0.017840520 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-10T03:05:18.046 INFO:journalctl@ceph.osd.0.smithi052.stdout:Jun 10 03:05:17 smithi052 podman[120541]: 2024-06-10 03:05:17.743894008 +0000 UTC m=+0.118107913 container create b8a999aa4cdc4b516d62b9808fbe86e0ac14cf866947210da81956de49fb2d00 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-0-deactivate, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240528) 2024-06-10T03:05:18.328 INFO:journalctl@ceph.osd.0.smithi052.stdout:Jun 10 03:05:18 smithi052 podman[120541]: 2024-06-10 03:05:18.046939466 +0000 UTC m=+0.421153371 container init b8a999aa4cdc4b516d62b9808fbe86e0ac14cf866947210da81956de49fb2d00 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-0-deactivate, GIT_CLEAN=True, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5) 2024-06-10T03:05:18.328 INFO:journalctl@ceph.osd.0.smithi052.stdout:Jun 10 03:05:18 smithi052 podman[120541]: 2024-06-10 03:05:18.051381726 +0000 UTC m=+0.425595626 container start b8a999aa4cdc4b516d62b9808fbe86e0ac14cf866947210da81956de49fb2d00 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-0-deactivate, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-06-10T03:05:18.328 INFO:journalctl@ceph.osd.0.smithi052.stdout:Jun 10 03:05:18 smithi052 podman[120541]: 2024-06-10 03:05:18.138779379 +0000 UTC m=+0.512993293 container attach b8a999aa4cdc4b516d62b9808fbe86e0ac14cf866947210da81956de49fb2d00 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-0-deactivate, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528) 2024-06-10T03:05:18.675 INFO:journalctl@ceph.osd.0.smithi052.stdout:Jun 10 03:05:18 smithi052 podman[120541]: 2024-06-10 03:05:18.367009402 +0000 UTC m=+0.741223312 container died b8a999aa4cdc4b516d62b9808fbe86e0ac14cf866947210da81956de49fb2d00 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-0-deactivate, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-06-10T03:05:18.960 DEBUG:teuthology.orchestra.run.smithi052:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968@osd.0.service' 2024-06-10T03:05:19.001 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-10T03:05:19.002 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-06-10T03:05:19.002 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-06-10T03:05:19.002 DEBUG:teuthology.orchestra.run.smithi120:> sudo systemctl stop ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968@osd.1 2024-06-10T03:05:19.322 INFO:journalctl@ceph.osd.1.smithi120.stdout:Jun 10 03:05:19 smithi120 systemd[1]: Stopping Ceph osd.1 for d8676e9e-26d2-11ef-bc9b-c7b262605968... 2024-06-10T03:05:19.638 INFO:journalctl@ceph.osd.1.smithi120.stdout:Jun 10 03:05:19 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-1[41185]: 2024-06-10T03:05:19.322+0000 7fe5a466c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-10T03:05:19.639 INFO:journalctl@ceph.osd.1.smithi120.stdout:Jun 10 03:05:19 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-1[41185]: 2024-06-10T03:05:19.322+0000 7fe5a466c700 -1 osd.1 104 *** Got signal Terminated *** 2024-06-10T03:05:19.639 INFO:journalctl@ceph.osd.1.smithi120.stdout:Jun 10 03:05:19 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-1[41185]: 2024-06-10T03:05:19.322+0000 7fe5a466c700 -1 osd.1 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-10T03:05:24.990 INFO:journalctl@ceph.osd.1.smithi120.stdout:Jun 10 03:05:24 smithi120 podman[71221]: 2024-06-10 03:05:24.697836784 +0000 UTC m=+5.496853774 container died 69c8735fce9e58ad9b07ca60176c78b9fc4c962ef2d35e3dfc504d9fb1674e87 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-1, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-06-10T03:05:25.349 INFO:journalctl@ceph.osd.1.smithi120.stdout:Jun 10 03:05:25 smithi120 podman[71221]: 2024-06-10 03:05:25.089584086 +0000 UTC m=+5.888601078 container cleanup 69c8735fce9e58ad9b07ca60176c78b9fc4c962ef2d35e3dfc504d9fb1674e87 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-1, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD) 2024-06-10T03:05:25.349 INFO:journalctl@ceph.osd.1.smithi120.stdout:Jun 10 03:05:25 smithi120 bash[71221]: ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-1 2024-06-10T03:05:25.638 INFO:journalctl@ceph.osd.1.smithi120.stdout:Jun 10 03:05:25 smithi120 podman[71238]: 2024-06-10 03:05:25.349601005 +0000 UTC m=+0.647581758 container remove 69c8735fce9e58ad9b07ca60176c78b9fc4c962ef2d35e3dfc504d9fb1674e87 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-1, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc) 2024-06-10T03:05:25.945 INFO:journalctl@ceph.osd.1.smithi120.stdout:Jun 10 03:05:25 smithi120 podman[71329]: 2024-06-10 03:05:25.573579155 +0000 UTC m=+0.017471540 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-10T03:05:25.945 INFO:journalctl@ceph.osd.1.smithi120.stdout:Jun 10 03:05:25 smithi120 podman[71329]: 2024-06-10 03:05:25.675651556 +0000 UTC m=+0.119543968 container create d0cec196655b82508320c62be79529ddc8a1b52ec751d5b95d2895c98c57e3bf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-1-deactivate, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True) 2024-06-10T03:05:25.946 INFO:journalctl@ceph.osd.1.smithi120.stdout:Jun 10 03:05:25 smithi120 podman[71329]: 2024-06-10 03:05:25.945638468 +0000 UTC m=+0.389530834 container init d0cec196655b82508320c62be79529ddc8a1b52ec751d5b95d2895c98c57e3bf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-1-deactivate, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528) 2024-06-10T03:05:26.242 INFO:journalctl@ceph.osd.1.smithi120.stdout:Jun 10 03:05:25 smithi120 podman[71329]: 2024-06-10 03:05:25.949106775 +0000 UTC m=+0.392999149 container start d0cec196655b82508320c62be79529ddc8a1b52ec751d5b95d2895c98c57e3bf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-1-deactivate, org.label-schema.build-date=20240528, GIT_CLEAN=True, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-06-10T03:05:26.242 INFO:journalctl@ceph.osd.1.smithi120.stdout:Jun 10 03:05:26 smithi120 podman[71329]: 2024-06-10 03:05:26.054177504 +0000 UTC m=+0.498069881 container attach d0cec196655b82508320c62be79529ddc8a1b52ec751d5b95d2895c98c57e3bf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-1-deactivate, ceph=True, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-06-10T03:05:26.513 INFO:journalctl@ceph.osd.1.smithi120.stdout:Jun 10 03:05:26 smithi120 podman[71329]: 2024-06-10 03:05:26.242569587 +0000 UTC m=+0.686461958 container died d0cec196655b82508320c62be79529ddc8a1b52ec751d5b95d2895c98c57e3bf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-1-deactivate, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240528, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-06-10T03:05:26.918 DEBUG:teuthology.orchestra.run.smithi120:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968@osd.1.service' 2024-06-10T03:05:26.955 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-10T03:05:26.955 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-06-10T03:05:26.955 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-06-10T03:05:26.956 DEBUG:teuthology.orchestra.run.smithi188:> sudo systemctl stop ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968@osd.2 2024-06-10T03:05:27.341 INFO:journalctl@ceph.osd.2.smithi188.stdout:Jun 10 03:05:27 smithi188 systemd[1]: Stopping Ceph osd.2 for d8676e9e-26d2-11ef-bc9b-c7b262605968... 2024-06-10T03:05:27.704 INFO:journalctl@ceph.osd.2.smithi188.stdout:Jun 10 03:05:27 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-2[41721]: 2024-06-10T03:05:27.341+0000 7f9d65450700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-10T03:05:27.704 INFO:journalctl@ceph.osd.2.smithi188.stdout:Jun 10 03:05:27 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-2[41721]: 2024-06-10T03:05:27.341+0000 7f9d65450700 -1 osd.2 104 *** Got signal Terminated *** 2024-06-10T03:05:27.704 INFO:journalctl@ceph.osd.2.smithi188.stdout:Jun 10 03:05:27 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-2[41721]: 2024-06-10T03:05:27.341+0000 7f9d65450700 -1 osd.2 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-10T03:05:33.075 INFO:journalctl@ceph.osd.2.smithi188.stdout:Jun 10 03:05:32 smithi188 podman[71942]: 2024-06-10 03:05:32.763506707 +0000 UTC m=+5.618192914 container died 6126f8078a4a5c38658ee547ec7243f0362decb1206a64141d6e2664e69bdaaf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, ceph=True, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-10T03:05:33.454 INFO:journalctl@ceph.osd.2.smithi188.stdout:Jun 10 03:05:33 smithi188 podman[71942]: 2024-06-10 03:05:33.183033615 +0000 UTC m=+6.037719813 container cleanup 6126f8078a4a5c38658ee547ec7243f0362decb1206a64141d6e2664e69bdaaf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-10T03:05:33.454 INFO:journalctl@ceph.osd.2.smithi188.stdout:Jun 10 03:05:33 smithi188 bash[71942]: ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-2 2024-06-10T03:05:33.954 INFO:journalctl@ceph.osd.2.smithi188.stdout:Jun 10 03:05:33 smithi188 podman[71961]: 2024-06-10 03:05:33.609815081 +0000 UTC m=+0.841652605 container remove 6126f8078a4a5c38658ee547ec7243f0362decb1206a64141d6e2664e69bdaaf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS) 2024-06-10T03:05:34.313 INFO:journalctl@ceph.osd.2.smithi188.stdout:Jun 10 03:05:34 smithi188 podman[72050]: 2024-06-10 03:05:33.959777213 +0000 UTC m=+0.017090269 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-10T03:05:34.313 INFO:journalctl@ceph.osd.2.smithi188.stdout:Jun 10 03:05:34 smithi188 podman[72050]: 2024-06-10 03:05:34.077372588 +0000 UTC m=+0.134685622 container create ffad715cc4a2ec2ca9e88ecb5e4ed9ff49a88e2ffc3c4a88be6e19dc4ca83179 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-2-deactivate, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-06-10T03:05:34.314 INFO:journalctl@ceph.osd.2.smithi188.stdout:Jun 10 03:05:34 smithi188 podman[72050]: 2024-06-10 03:05:34.3136929 +0000 UTC m=+0.371005943 container init ffad715cc4a2ec2ca9e88ecb5e4ed9ff49a88e2ffc3c4a88be6e19dc4ca83179 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-2-deactivate, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-06-10T03:05:34.704 INFO:journalctl@ceph.osd.2.smithi188.stdout:Jun 10 03:05:34 smithi188 podman[72050]: 2024-06-10 03:05:34.317270185 +0000 UTC m=+0.374583230 container start ffad715cc4a2ec2ca9e88ecb5e4ed9ff49a88e2ffc3c4a88be6e19dc4ca83179 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-2-deactivate, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-06-10T03:05:34.704 INFO:journalctl@ceph.osd.2.smithi188.stdout:Jun 10 03:05:34 smithi188 podman[72050]: 2024-06-10 03:05:34.455638688 +0000 UTC m=+0.512951744 container attach ffad715cc4a2ec2ca9e88ecb5e4ed9ff49a88e2ffc3c4a88be6e19dc4ca83179 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-2-deactivate, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-06-10T03:05:35.167 INFO:journalctl@ceph.osd.2.smithi188.stdout:Jun 10 03:05:34 smithi188 podman[72050]: 2024-06-10 03:05:34.857441148 +0000 UTC m=+0.914754190 container died ffad715cc4a2ec2ca9e88ecb5e4ed9ff49a88e2ffc3c4a88be6e19dc4ca83179 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-2-deactivate, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-06-10T03:05:35.476 DEBUG:teuthology.orchestra.run.smithi188:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968@osd.2.service' 2024-06-10T03:05:35.513 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-10T03:05:35.513 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-06-10T03:05:35.513 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-06-10T03:05:35.514 DEBUG:teuthology.orchestra.run.smithi052:> sudo systemctl stop ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968@osd.3 2024-06-10T03:05:35.846 INFO:journalctl@ceph.osd.3.smithi052.stdout:Jun 10 03:05:35 smithi052 systemd[1]: Stopping Ceph osd.3 for d8676e9e-26d2-11ef-bc9b-c7b262605968... 2024-06-10T03:05:36.227 INFO:journalctl@ceph.osd.3.smithi052.stdout:Jun 10 03:05:35 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-3[53862]: 2024-06-10T03:05:35.846+0000 7f3c45272700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-10T03:05:36.227 INFO:journalctl@ceph.osd.3.smithi052.stdout:Jun 10 03:05:35 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-3[53862]: 2024-06-10T03:05:35.846+0000 7f3c45272700 -1 osd.3 104 *** Got signal Terminated *** 2024-06-10T03:05:36.227 INFO:journalctl@ceph.osd.3.smithi052.stdout:Jun 10 03:05:35 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-3[53862]: 2024-06-10T03:05:35.846+0000 7f3c45272700 -1 osd.3 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-10T03:05:37.889 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:05:37 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:05:37.460+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-10T03:05:15.396577+0000 front 2024-06-10T03:05:15.396655+0000 (oldest deadline 2024-06-10T03:05:37.096690+0000) 2024-06-10T03:05:38.477 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:05:38 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:05:38.011+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-10T03:05:12.298026+0000 front 2024-06-10T03:05:12.298086+0000 (oldest deadline 2024-06-10T03:05:37.598149+0000) 2024-06-10T03:05:38.888 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:05:38 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:05:38.462+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-10T03:05:15.396577+0000 front 2024-06-10T03:05:15.396655+0000 (oldest deadline 2024-06-10T03:05:37.096690+0000) 2024-06-10T03:05:38.954 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:05:38 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:05:38.680+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-10T03:05:13.354170+0000 front 2024-06-10T03:05:13.354237+0000 (oldest deadline 2024-06-10T03:05:38.053995+0000) 2024-06-10T03:05:39.477 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:05:39 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:05:39.034+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-10T03:05:12.298026+0000 front 2024-06-10T03:05:12.298086+0000 (oldest deadline 2024-06-10T03:05:37.598149+0000) 2024-06-10T03:05:39.888 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:05:39 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:05:39.483+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-10T03:05:15.396577+0000 front 2024-06-10T03:05:15.396655+0000 (oldest deadline 2024-06-10T03:05:37.096690+0000) 2024-06-10T03:05:39.954 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:05:39 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:05:39.694+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-10T03:05:13.354170+0000 front 2024-06-10T03:05:13.354237+0000 (oldest deadline 2024-06-10T03:05:38.053995+0000) 2024-06-10T03:05:40.477 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:05:39 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:05:39.997+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-10T03:05:12.298026+0000 front 2024-06-10T03:05:12.298086+0000 (oldest deadline 2024-06-10T03:05:37.598149+0000) 2024-06-10T03:05:40.889 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:05:40 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:05:40.498+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-10T03:05:15.396577+0000 front 2024-06-10T03:05:15.396655+0000 (oldest deadline 2024-06-10T03:05:37.096690+0000) 2024-06-10T03:05:40.954 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:05:40 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:05:40.695+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-10T03:05:13.354170+0000 front 2024-06-10T03:05:13.354237+0000 (oldest deadline 2024-06-10T03:05:38.053995+0000) 2024-06-10T03:05:41.226 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:05:40 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:05:40.952+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-10T03:05:12.298026+0000 front 2024-06-10T03:05:12.298086+0000 (oldest deadline 2024-06-10T03:05:37.598149+0000) 2024-06-10T03:05:41.477 INFO:journalctl@ceph.osd.3.smithi052.stdout:Jun 10 03:05:41 smithi052 podman[120669]: 2024-06-10 03:05:41.226706647 +0000 UTC m=+5.507038944 container died 2d725109637acce58c5e588cb0ab292be10f403621b33a31f8ac19552f147d86 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-3, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-10T03:05:41.865 INFO:journalctl@ceph.osd.3.smithi052.stdout:Jun 10 03:05:41 smithi052 podman[120669]: 2024-06-10 03:05:41.663282899 +0000 UTC m=+5.943615184 container cleanup 2d725109637acce58c5e588cb0ab292be10f403621b33a31f8ac19552f147d86 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-3, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-06-10T03:05:41.865 INFO:journalctl@ceph.osd.3.smithi052.stdout:Jun 10 03:05:41 smithi052 bash[120669]: ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-3 2024-06-10T03:05:41.888 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:05:41 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:05:41.518+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-10T03:05:15.396577+0000 front 2024-06-10T03:05:15.396655+0000 (oldest deadline 2024-06-10T03:05:37.096690+0000) 2024-06-10T03:05:41.954 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:05:41 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:05:41.665+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-10T03:05:13.354170+0000 front 2024-06-10T03:05:13.354237+0000 (oldest deadline 2024-06-10T03:05:38.053995+0000) 2024-06-10T03:05:42.227 INFO:journalctl@ceph.osd.3.smithi052.stdout:Jun 10 03:05:41 smithi052 podman[120689]: 2024-06-10 03:05:41.865074591 +0000 UTC m=+0.635145666 container remove 2d725109637acce58c5e588cb0ab292be10f403621b33a31f8ac19552f147d86 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-3, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240528) 2024-06-10T03:05:42.227 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:05:41 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:05:41.941+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-10T03:05:12.298026+0000 front 2024-06-10T03:05:12.298086+0000 (oldest deadline 2024-06-10T03:05:37.598149+0000) 2024-06-10T03:05:42.635 INFO:journalctl@ceph.osd.3.smithi052.stdout:Jun 10 03:05:42 smithi052 podman[120775]: 2024-06-10 03:05:42.209771524 +0000 UTC m=+0.017868809 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-10T03:05:42.635 INFO:journalctl@ceph.osd.3.smithi052.stdout:Jun 10 03:05:42 smithi052 podman[120775]: 2024-06-10 03:05:42.307545405 +0000 UTC m=+0.115642669 container create 51e3cdb3ecf8c2e0d9ceeee4fe2b497b1325d33c09f9c972352f5f4e2da0a5d9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-3-deactivate, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=squid-3d89acc, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-06-10T03:05:42.636 INFO:journalctl@ceph.osd.3.smithi052.stdout:Jun 10 03:05:42 smithi052 podman[120775]: 2024-06-10 03:05:42.543934588 +0000 UTC m=+0.352031874 container init 51e3cdb3ecf8c2e0d9ceeee4fe2b497b1325d33c09f9c972352f5f4e2da0a5d9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-3-deactivate, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-10T03:05:42.636 INFO:journalctl@ceph.osd.3.smithi052.stdout:Jun 10 03:05:42 smithi052 podman[120775]: 2024-06-10 03:05:42.54768656 +0000 UTC m=+0.355783833 container start 51e3cdb3ecf8c2e0d9ceeee4fe2b497b1325d33c09f9c972352f5f4e2da0a5d9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-3-deactivate, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-10T03:05:42.636 INFO:journalctl@ceph.osd.3.smithi052.stdout:Jun 10 03:05:42 smithi052 podman[120775]: 2024-06-10 03:05:42.635518748 +0000 UTC m=+0.443616023 container attach 51e3cdb3ecf8c2e0d9ceeee4fe2b497b1325d33c09f9c972352f5f4e2da0a5d9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-3-deactivate, io.buildah.version=1.33.5, ceph=True, RELEASE=squid-3d89acc, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0) 2024-06-10T03:05:42.888 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:05:42 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:05:42.500+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-10T03:05:15.396577+0000 front 2024-06-10T03:05:15.396655+0000 (oldest deadline 2024-06-10T03:05:37.096690+0000) 2024-06-10T03:05:42.954 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:05:42 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:05:42.655+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-10T03:05:13.354170+0000 front 2024-06-10T03:05:13.354237+0000 (oldest deadline 2024-06-10T03:05:38.053995+0000) 2024-06-10T03:05:42.954 INFO:journalctl@ceph.osd.3.smithi052.stdout:Jun 10 03:05:42 smithi052 podman[120775]: 2024-06-10 03:05:42.848710929 +0000 UTC m=+0.656808212 container died 51e3cdb3ecf8c2e0d9ceeee4fe2b497b1325d33c09f9c972352f5f4e2da0a5d9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-3-deactivate, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528) 2024-06-10T03:05:43.227 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:05:42 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:05:42.954+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-10T03:05:12.298026+0000 front 2024-06-10T03:05:12.298086+0000 (oldest deadline 2024-06-10T03:05:37.598149+0000) 2024-06-10T03:05:43.548 DEBUG:teuthology.orchestra.run.smithi052:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968@osd.3.service' 2024-06-10T03:05:43.593 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-10T03:05:43.593 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-06-10T03:05:43.594 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-06-10T03:05:43.594 DEBUG:teuthology.orchestra.run.smithi120:> sudo systemctl stop ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968@osd.4 2024-06-10T03:05:43.888 INFO:journalctl@ceph.osd.4.smithi120.stdout:Jun 10 03:05:43 smithi120 systemd[1]: Stopping Ceph osd.4 for d8676e9e-26d2-11ef-bc9b-c7b262605968... 2024-06-10T03:05:43.889 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:05:43 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:05:43.470+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-10T03:05:15.396577+0000 front 2024-06-10T03:05:15.396655+0000 (oldest deadline 2024-06-10T03:05:37.096690+0000) 2024-06-10T03:05:43.954 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:05:43 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:05:43.666+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-10T03:05:13.354170+0000 front 2024-06-10T03:05:13.354237+0000 (oldest deadline 2024-06-10T03:05:38.053995+0000) 2024-06-10T03:05:44.227 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:05:43 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:05:43.968+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-10T03:05:12.298026+0000 front 2024-06-10T03:05:12.298086+0000 (oldest deadline 2024-06-10T03:05:37.598149+0000) 2024-06-10T03:05:44.389 INFO:journalctl@ceph.osd.4.smithi120.stdout:Jun 10 03:05:43 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-4[47679]: 2024-06-10T03:05:43.911+0000 7f4dfe711700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-10T03:05:44.389 INFO:journalctl@ceph.osd.4.smithi120.stdout:Jun 10 03:05:43 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-4[47679]: 2024-06-10T03:05:43.911+0000 7f4dfe711700 -1 osd.4 104 *** Got signal Terminated *** 2024-06-10T03:05:44.389 INFO:journalctl@ceph.osd.4.smithi120.stdout:Jun 10 03:05:43 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-4[47679]: 2024-06-10T03:05:43.911+0000 7f4dfe711700 -1 osd.4 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-10T03:05:44.888 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:05:44 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:05:44.468+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-10T03:05:15.396577+0000 front 2024-06-10T03:05:15.396655+0000 (oldest deadline 2024-06-10T03:05:37.096690+0000) 2024-06-10T03:05:44.954 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:05:44 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:05:44.620+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-10T03:05:13.354170+0000 front 2024-06-10T03:05:13.354237+0000 (oldest deadline 2024-06-10T03:05:38.053995+0000) 2024-06-10T03:05:45.477 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:05:45 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:05:45.002+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-10T03:05:12.298026+0000 front 2024-06-10T03:05:12.298086+0000 (oldest deadline 2024-06-10T03:05:37.598149+0000) 2024-06-10T03:05:45.633 INFO:journalctl@ceph.osd.5.smithi188.stdout:Jun 10 03:05:45 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-5[48629]: 2024-06-10T03:05:45.372+0000 7f5d445a7700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:19.483971+0000 front 2024-06-10T03:05:19.484087+0000 (oldest deadline 2024-06-10T03:05:44.784126+0000) 2024-06-10T03:05:45.890 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:05:45 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:05:45.496+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-10T03:05:15.396577+0000 front 2024-06-10T03:05:15.396655+0000 (oldest deadline 2024-06-10T03:05:37.096690+0000) 2024-06-10T03:05:45.954 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:05:45 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:05:45.632+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-10T03:05:13.354170+0000 front 2024-06-10T03:05:13.354237+0000 (oldest deadline 2024-06-10T03:05:38.053995+0000) 2024-06-10T03:05:45.954 INFO:journalctl@ceph.osd.8.smithi188.stdout:Jun 10 03:05:45 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-8[55165]: 2024-06-10T03:05:45.758+0000 7fc330a8a700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:19.415044+0000 front 2024-06-10T03:05:19.415069+0000 (oldest deadline 2024-06-10T03:05:45.315110+0000) 2024-06-10T03:05:46.477 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:05:46 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:05:46.047+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-10T03:05:12.298026+0000 front 2024-06-10T03:05:12.298086+0000 (oldest deadline 2024-06-10T03:05:37.598149+0000) 2024-06-10T03:05:46.704 INFO:journalctl@ceph.osd.5.smithi188.stdout:Jun 10 03:05:46 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-5[48629]: 2024-06-10T03:05:46.400+0000 7f5d445a7700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:19.483971+0000 front 2024-06-10T03:05:19.484087+0000 (oldest deadline 2024-06-10T03:05:44.784126+0000) 2024-06-10T03:05:46.704 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:05:46 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:05:46.604+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-10T03:05:13.354170+0000 front 2024-06-10T03:05:13.354237+0000 (oldest deadline 2024-06-10T03:05:38.053995+0000) 2024-06-10T03:05:46.704 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:05:46 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:05:46.604+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:22.755328+0000 front 2024-06-10T03:05:22.755363+0000 (oldest deadline 2024-06-10T03:05:45.655329+0000) 2024-06-10T03:05:46.888 INFO:journalctl@ceph.osd.4.smithi120.stdout:Jun 10 03:05:46 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-4[47679]: 2024-06-10T03:05:46.591+0000 7f4dfad05700 -1 osd.4 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:23.295685+0000 front 2024-06-10T03:05:23.295641+0000 (oldest deadline 2024-06-10T03:05:46.195878+0000) 2024-06-10T03:05:46.889 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:05:46 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:05:46.530+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-10T03:05:15.396577+0000 front 2024-06-10T03:05:15.396655+0000 (oldest deadline 2024-06-10T03:05:37.096690+0000) 2024-06-10T03:05:46.889 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:05:46 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:05:46.530+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:22.397847+0000 front 2024-06-10T03:05:22.397969+0000 (oldest deadline 2024-06-10T03:05:46.497941+0000) 2024-06-10T03:05:47.204 INFO:journalctl@ceph.osd.8.smithi188.stdout:Jun 10 03:05:46 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-8[55165]: 2024-06-10T03:05:46.724+0000 7fc330a8a700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:19.415044+0000 front 2024-06-10T03:05:19.415069+0000 (oldest deadline 2024-06-10T03:05:45.315110+0000) 2024-06-10T03:05:47.477 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:05:47 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:05:47.009+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-10T03:05:12.298026+0000 front 2024-06-10T03:05:12.298086+0000 (oldest deadline 2024-06-10T03:05:37.598149+0000) 2024-06-10T03:05:47.704 INFO:journalctl@ceph.osd.5.smithi188.stdout:Jun 10 03:05:47 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-5[48629]: 2024-06-10T03:05:47.438+0000 7f5d445a7700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:19.483971+0000 front 2024-06-10T03:05:19.484087+0000 (oldest deadline 2024-06-10T03:05:44.784126+0000) 2024-06-10T03:05:47.704 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:05:47 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:05:47.608+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-10T03:05:13.354170+0000 front 2024-06-10T03:05:13.354237+0000 (oldest deadline 2024-06-10T03:05:38.053995+0000) 2024-06-10T03:05:47.704 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:05:47 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:05:47.608+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:22.755328+0000 front 2024-06-10T03:05:22.755363+0000 (oldest deadline 2024-06-10T03:05:45.655329+0000) 2024-06-10T03:05:47.888 INFO:journalctl@ceph.osd.4.smithi120.stdout:Jun 10 03:05:47 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-4[47679]: 2024-06-10T03:05:47.602+0000 7f4dfad05700 -1 osd.4 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:23.295685+0000 front 2024-06-10T03:05:23.295641+0000 (oldest deadline 2024-06-10T03:05:46.195878+0000) 2024-06-10T03:05:47.889 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:05:47 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:05:47.496+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-10T03:05:15.396577+0000 front 2024-06-10T03:05:15.396655+0000 (oldest deadline 2024-06-10T03:05:37.096690+0000) 2024-06-10T03:05:47.889 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:05:47 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:05:47.496+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:22.397847+0000 front 2024-06-10T03:05:22.397969+0000 (oldest deadline 2024-06-10T03:05:46.497941+0000) 2024-06-10T03:05:48.204 INFO:journalctl@ceph.osd.8.smithi188.stdout:Jun 10 03:05:47 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-8[55165]: 2024-06-10T03:05:47.729+0000 7fc330a8a700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:19.415044+0000 front 2024-06-10T03:05:19.415069+0000 (oldest deadline 2024-06-10T03:05:45.315110+0000) 2024-06-10T03:05:48.477 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:05:48 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:05:48.032+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-10T03:05:12.298026+0000 front 2024-06-10T03:05:12.298086+0000 (oldest deadline 2024-06-10T03:05:37.598149+0000) 2024-06-10T03:05:48.477 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:05:48 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:05:48.032+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:23.499450+0000 front 2024-06-10T03:05:23.499380+0000 (oldest deadline 2024-06-10T03:05:47.599354+0000) 2024-06-10T03:05:48.478 INFO:journalctl@ceph.osd.6.smithi052.stdout:Jun 10 03:05:48 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-6[60649]: 2024-06-10T03:05:48.267+0000 7f4ce3c07700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:23.897917+0000 front 2024-06-10T03:05:23.897967+0000 (oldest deadline 2024-06-10T03:05:47.998039+0000) 2024-06-10T03:05:48.634 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:05:48 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:05:48.502+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-10T03:05:15.396577+0000 front 2024-06-10T03:05:15.396655+0000 (oldest deadline 2024-06-10T03:05:37.096690+0000) 2024-06-10T03:05:48.635 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:05:48 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:05:48.502+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:22.397847+0000 front 2024-06-10T03:05:22.397969+0000 (oldest deadline 2024-06-10T03:05:46.497941+0000) 2024-06-10T03:05:48.635 INFO:journalctl@ceph.osd.7.smithi120.stdout:Jun 10 03:05:48 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-7[54219]: 2024-06-10T03:05:48.375+0000 7f3b24ea2700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:23.717100+0000 front 2024-06-10T03:05:23.717037+0000 (oldest deadline 2024-06-10T03:05:47.817258+0000) 2024-06-10T03:05:48.704 INFO:journalctl@ceph.osd.5.smithi188.stdout:Jun 10 03:05:48 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-5[48629]: 2024-06-10T03:05:48.409+0000 7f5d445a7700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:19.483971+0000 front 2024-06-10T03:05:19.484087+0000 (oldest deadline 2024-06-10T03:05:44.784126+0000) 2024-06-10T03:05:48.704 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:05:48 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:05:48.567+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-10T03:05:13.354170+0000 front 2024-06-10T03:05:13.354237+0000 (oldest deadline 2024-06-10T03:05:38.053995+0000) 2024-06-10T03:05:48.704 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:05:48 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:05:48.567+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:22.755328+0000 front 2024-06-10T03:05:22.755363+0000 (oldest deadline 2024-06-10T03:05:45.655329+0000) 2024-06-10T03:05:48.888 INFO:journalctl@ceph.osd.4.smithi120.stdout:Jun 10 03:05:48 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-4[47679]: 2024-06-10T03:05:48.634+0000 7f4dfad05700 -1 osd.4 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:23.295685+0000 front 2024-06-10T03:05:23.295641+0000 (oldest deadline 2024-06-10T03:05:46.195878+0000) 2024-06-10T03:05:49.204 INFO:journalctl@ceph.osd.8.smithi188.stdout:Jun 10 03:05:48 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-8[55165]: 2024-06-10T03:05:48.737+0000 7fc330a8a700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:19.415044+0000 front 2024-06-10T03:05:19.415069+0000 (oldest deadline 2024-06-10T03:05:45.315110+0000) 2024-06-10T03:05:49.477 INFO:journalctl@ceph.osd.6.smithi052.stdout:Jun 10 03:05:49 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-6[60649]: 2024-06-10T03:05:49.238+0000 7f4ce3c07700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:23.897917+0000 front 2024-06-10T03:05:23.897967+0000 (oldest deadline 2024-06-10T03:05:47.998039+0000) 2024-06-10T03:05:49.477 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:05:49 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:05:49.081+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-10T03:05:12.298026+0000 front 2024-06-10T03:05:12.298086+0000 (oldest deadline 2024-06-10T03:05:37.598149+0000) 2024-06-10T03:05:49.477 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:05:49 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:05:49.081+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:23.499450+0000 front 2024-06-10T03:05:23.499380+0000 (oldest deadline 2024-06-10T03:05:47.599354+0000) 2024-06-10T03:05:49.639 INFO:journalctl@ceph.osd.7.smithi120.stdout:Jun 10 03:05:49 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-7[54219]: 2024-06-10T03:05:49.377+0000 7f3b24ea2700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:23.717100+0000 front 2024-06-10T03:05:23.717037+0000 (oldest deadline 2024-06-10T03:05:47.817258+0000) 2024-06-10T03:05:49.639 INFO:journalctl@ceph.osd.4.smithi120.stdout:Jun 10 03:05:49 smithi120 podman[71449]: 2024-06-10 03:05:49.44034371 +0000 UTC m=+5.652184313 container died 843d778f56e50249aedc41cf754a76a90a497fbdf10994e5b4da21221f78038c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-4, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-06-10T03:05:49.639 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:05:49 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:05:49.479+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-10T03:05:15.396577+0000 front 2024-06-10T03:05:15.396655+0000 (oldest deadline 2024-06-10T03:05:37.096690+0000) 2024-06-10T03:05:49.639 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:05:49 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:05:49.479+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:22.397847+0000 front 2024-06-10T03:05:22.397969+0000 (oldest deadline 2024-06-10T03:05:46.497941+0000) 2024-06-10T03:05:49.704 INFO:journalctl@ceph.osd.5.smithi188.stdout:Jun 10 03:05:49 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-5[48629]: 2024-06-10T03:05:49.374+0000 7f5d445a7700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:19.483971+0000 front 2024-06-10T03:05:19.484087+0000 (oldest deadline 2024-06-10T03:05:44.784126+0000) 2024-06-10T03:05:49.704 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:05:49 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:05:49.567+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-10T03:05:13.354170+0000 front 2024-06-10T03:05:13.354237+0000 (oldest deadline 2024-06-10T03:05:38.053995+0000) 2024-06-10T03:05:49.704 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:05:49 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:05:49.567+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:22.755328+0000 front 2024-06-10T03:05:22.755363+0000 (oldest deadline 2024-06-10T03:05:45.655329+0000) 2024-06-10T03:05:50.105 INFO:journalctl@ceph.osd.4.smithi120.stdout:Jun 10 03:05:49 smithi120 podman[71449]: 2024-06-10 03:05:49.804173867 +0000 UTC m=+6.016014462 container cleanup 843d778f56e50249aedc41cf754a76a90a497fbdf10994e5b4da21221f78038c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-4, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc) 2024-06-10T03:05:50.106 INFO:journalctl@ceph.osd.4.smithi120.stdout:Jun 10 03:05:49 smithi120 bash[71449]: ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-4 2024-06-10T03:05:50.204 INFO:journalctl@ceph.osd.8.smithi188.stdout:Jun 10 03:05:49 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-8[55165]: 2024-06-10T03:05:49.755+0000 7fc330a8a700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:19.415044+0000 front 2024-06-10T03:05:19.415069+0000 (oldest deadline 2024-06-10T03:05:45.315110+0000) 2024-06-10T03:05:50.377 INFO:journalctl@ceph.osd.4.smithi120.stdout:Jun 10 03:05:50 smithi120 podman[71466]: 2024-06-10 03:05:50.106128167 +0000 UTC m=+0.667776390 container remove 843d778f56e50249aedc41cf754a76a90a497fbdf10994e5b4da21221f78038c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-4, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-06-10T03:05:50.477 INFO:journalctl@ceph.osd.6.smithi052.stdout:Jun 10 03:05:50 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-6[60649]: 2024-06-10T03:05:50.285+0000 7f4ce3c07700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:23.897917+0000 front 2024-06-10T03:05:23.897967+0000 (oldest deadline 2024-06-10T03:05:47.998039+0000) 2024-06-10T03:05:50.477 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:05:50 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:05:50.081+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-10T03:05:12.298026+0000 front 2024-06-10T03:05:12.298086+0000 (oldest deadline 2024-06-10T03:05:37.598149+0000) 2024-06-10T03:05:50.478 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:05:50 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:05:50.081+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:23.499450+0000 front 2024-06-10T03:05:23.499380+0000 (oldest deadline 2024-06-10T03:05:47.599354+0000) 2024-06-10T03:05:50.654 INFO:journalctl@ceph.osd.4.smithi120.stdout:Jun 10 03:05:50 smithi120 podman[71553]: 2024-06-10 03:05:50.374984807 +0000 UTC m=+0.017288488 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-10T03:05:50.654 INFO:journalctl@ceph.osd.4.smithi120.stdout:Jun 10 03:05:50 smithi120 podman[71553]: 2024-06-10 03:05:50.481837963 +0000 UTC m=+0.124141634 container create 7f4d218c85d39d1d0e166e029bea7bbe63a6d491a082fb8d940d6b1499fa731b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-4-deactivate, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-06-10T03:05:50.654 INFO:journalctl@ceph.osd.7.smithi120.stdout:Jun 10 03:05:50 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-7[54219]: 2024-06-10T03:05:50.376+0000 7f3b24ea2700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:23.717100+0000 front 2024-06-10T03:05:23.717037+0000 (oldest deadline 2024-06-10T03:05:47.817258+0000) 2024-06-10T03:05:50.654 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:05:50 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:05:50.451+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-10T03:05:15.396577+0000 front 2024-06-10T03:05:15.396655+0000 (oldest deadline 2024-06-10T03:05:37.096690+0000) 2024-06-10T03:05:50.654 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:05:50 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:05:50.451+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:22.397847+0000 front 2024-06-10T03:05:22.397969+0000 (oldest deadline 2024-06-10T03:05:46.497941+0000) 2024-06-10T03:05:50.704 INFO:journalctl@ceph.osd.5.smithi188.stdout:Jun 10 03:05:50 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-5[48629]: 2024-06-10T03:05:50.380+0000 7f5d445a7700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:19.483971+0000 front 2024-06-10T03:05:19.484087+0000 (oldest deadline 2024-06-10T03:05:44.784126+0000) 2024-06-10T03:05:50.704 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:05:50 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:05:50.602+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-10T03:05:13.354170+0000 front 2024-06-10T03:05:13.354237+0000 (oldest deadline 2024-06-10T03:05:38.053995+0000) 2024-06-10T03:05:50.704 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:05:50 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:05:50.602+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:22.755328+0000 front 2024-06-10T03:05:22.755363+0000 (oldest deadline 2024-06-10T03:05:45.655329+0000) 2024-06-10T03:05:51.030 INFO:journalctl@ceph.osd.4.smithi120.stdout:Jun 10 03:05:50 smithi120 podman[71553]: 2024-06-10 03:05:50.743049824 +0000 UTC m=+0.385353511 container init 7f4d218c85d39d1d0e166e029bea7bbe63a6d491a082fb8d940d6b1499fa731b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-4-deactivate, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-10T03:05:51.030 INFO:journalctl@ceph.osd.4.smithi120.stdout:Jun 10 03:05:50 smithi120 podman[71553]: 2024-06-10 03:05:50.746268462 +0000 UTC m=+0.388572143 container start 7f4d218c85d39d1d0e166e029bea7bbe63a6d491a082fb8d940d6b1499fa731b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-4-deactivate, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528) 2024-06-10T03:05:51.030 INFO:journalctl@ceph.osd.4.smithi120.stdout:Jun 10 03:05:50 smithi120 podman[71553]: 2024-06-10 03:05:50.868328429 +0000 UTC m=+0.510632111 container attach 7f4d218c85d39d1d0e166e029bea7bbe63a6d491a082fb8d940d6b1499fa731b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-4-deactivate, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240528) 2024-06-10T03:05:51.204 INFO:journalctl@ceph.osd.8.smithi188.stdout:Jun 10 03:05:50 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-8[55165]: 2024-06-10T03:05:50.784+0000 7fc330a8a700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:19.415044+0000 front 2024-06-10T03:05:19.415069+0000 (oldest deadline 2024-06-10T03:05:45.315110+0000) 2024-06-10T03:05:51.321 INFO:journalctl@ceph.osd.4.smithi120.stdout:Jun 10 03:05:51 smithi120 podman[71553]: 2024-06-10 03:05:51.030883472 +0000 UTC m=+0.673187161 container died 7f4d218c85d39d1d0e166e029bea7bbe63a6d491a082fb8d940d6b1499fa731b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-06-10T03:05:51.477 INFO:journalctl@ceph.osd.6.smithi052.stdout:Jun 10 03:05:51 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-6[60649]: 2024-06-10T03:05:51.306+0000 7f4ce3c07700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:23.897917+0000 front 2024-06-10T03:05:23.897967+0000 (oldest deadline 2024-06-10T03:05:47.998039+0000) 2024-06-10T03:05:51.478 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:05:51 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:05:51.099+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-10T03:05:12.298026+0000 front 2024-06-10T03:05:12.298086+0000 (oldest deadline 2024-06-10T03:05:37.598149+0000) 2024-06-10T03:05:51.478 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:05:51 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:05:51.099+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:23.499450+0000 front 2024-06-10T03:05:23.499380+0000 (oldest deadline 2024-06-10T03:05:47.599354+0000) 2024-06-10T03:05:51.622 INFO:journalctl@ceph.osd.7.smithi120.stdout:Jun 10 03:05:51 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-7[54219]: 2024-06-10T03:05:51.372+0000 7f3b24ea2700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:23.717100+0000 front 2024-06-10T03:05:23.717037+0000 (oldest deadline 2024-06-10T03:05:47.817258+0000) 2024-06-10T03:05:51.622 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:05:51 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:05:51.436+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-10T03:05:15.396577+0000 front 2024-06-10T03:05:15.396655+0000 (oldest deadline 2024-06-10T03:05:37.096690+0000) 2024-06-10T03:05:51.622 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:05:51 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:05:51.436+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:22.397847+0000 front 2024-06-10T03:05:22.397969+0000 (oldest deadline 2024-06-10T03:05:46.497941+0000) 2024-06-10T03:05:51.644 DEBUG:teuthology.orchestra.run.smithi120:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968@osd.4.service' 2024-06-10T03:05:51.681 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-10T03:05:51.681 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-06-10T03:05:51.681 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-06-10T03:05:51.682 DEBUG:teuthology.orchestra.run.smithi188:> sudo systemctl stop ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968@osd.5 2024-06-10T03:05:51.704 INFO:journalctl@ceph.osd.5.smithi188.stdout:Jun 10 03:05:51 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-5[48629]: 2024-06-10T03:05:51.391+0000 7f5d445a7700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:19.483971+0000 front 2024-06-10T03:05:19.484087+0000 (oldest deadline 2024-06-10T03:05:44.784126+0000) 2024-06-10T03:05:51.704 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:05:51 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:05:51.591+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-10T03:05:13.354170+0000 front 2024-06-10T03:05:13.354237+0000 (oldest deadline 2024-06-10T03:05:38.053995+0000) 2024-06-10T03:05:51.704 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:05:51 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:05:51.591+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:22.755328+0000 front 2024-06-10T03:05:22.755363+0000 (oldest deadline 2024-06-10T03:05:45.655329+0000) 2024-06-10T03:05:51.986 INFO:journalctl@ceph.osd.8.smithi188.stdout:Jun 10 03:05:51 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-8[55165]: 2024-06-10T03:05:51.762+0000 7fc330a8a700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:19.415044+0000 front 2024-06-10T03:05:19.415069+0000 (oldest deadline 2024-06-10T03:05:45.315110+0000) 2024-06-10T03:05:51.987 INFO:journalctl@ceph.osd.5.smithi188.stdout:Jun 10 03:05:51 smithi188 systemd[1]: Stopping Ceph osd.5 for d8676e9e-26d2-11ef-bc9b-c7b262605968... 2024-06-10T03:05:51.987 INFO:journalctl@ceph.osd.5.smithi188.stdout:Jun 10 03:05:51 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-5[48629]: 2024-06-10T03:05:51.985+0000 7f5d47fb3700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-10T03:05:52.372 INFO:journalctl@ceph.osd.5.smithi188.stdout:Jun 10 03:05:51 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-5[48629]: 2024-06-10T03:05:51.985+0000 7f5d47fb3700 -1 osd.5 104 *** Got signal Terminated *** 2024-06-10T03:05:52.372 INFO:journalctl@ceph.osd.5.smithi188.stdout:Jun 10 03:05:51 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-5[48629]: 2024-06-10T03:05:51.985+0000 7f5d47fb3700 -1 osd.5 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-10T03:05:52.372 INFO:journalctl@ceph.osd.5.smithi188.stdout:Jun 10 03:05:52 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-5[48629]: 2024-06-10T03:05:52.372+0000 7f5d445a7700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:19.483971+0000 front 2024-06-10T03:05:19.484087+0000 (oldest deadline 2024-06-10T03:05:44.784126+0000) 2024-06-10T03:05:52.477 INFO:journalctl@ceph.osd.6.smithi052.stdout:Jun 10 03:05:52 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-6[60649]: 2024-06-10T03:05:52.306+0000 7f4ce3c07700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:23.897917+0000 front 2024-06-10T03:05:23.897967+0000 (oldest deadline 2024-06-10T03:05:47.998039+0000) 2024-06-10T03:05:52.477 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:05:52 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:05:52.059+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-10T03:05:12.298026+0000 front 2024-06-10T03:05:12.298086+0000 (oldest deadline 2024-06-10T03:05:37.598149+0000) 2024-06-10T03:05:52.477 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:05:52 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:05:52.059+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:23.499450+0000 front 2024-06-10T03:05:23.499380+0000 (oldest deadline 2024-06-10T03:05:47.599354+0000) 2024-06-10T03:05:52.638 INFO:journalctl@ceph.osd.7.smithi120.stdout:Jun 10 03:05:52 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-7[54219]: 2024-06-10T03:05:52.327+0000 7f3b24ea2700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:23.717100+0000 front 2024-06-10T03:05:23.717037+0000 (oldest deadline 2024-06-10T03:05:47.817258+0000) 2024-06-10T03:05:52.639 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:05:52 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:05:52.447+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-10T03:05:15.396577+0000 front 2024-06-10T03:05:15.396655+0000 (oldest deadline 2024-06-10T03:05:37.096690+0000) 2024-06-10T03:05:52.639 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:05:52 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:05:52.447+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:22.397847+0000 front 2024-06-10T03:05:22.397969+0000 (oldest deadline 2024-06-10T03:05:46.497941+0000) 2024-06-10T03:05:52.704 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:05:52 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:05:52.563+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-10T03:05:13.354170+0000 front 2024-06-10T03:05:13.354237+0000 (oldest deadline 2024-06-10T03:05:38.053995+0000) 2024-06-10T03:05:52.704 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:05:52 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:05:52.563+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:22.755328+0000 front 2024-06-10T03:05:22.755363+0000 (oldest deadline 2024-06-10T03:05:45.655329+0000) 2024-06-10T03:05:53.204 INFO:journalctl@ceph.osd.8.smithi188.stdout:Jun 10 03:05:52 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-8[55165]: 2024-06-10T03:05:52.794+0000 7fc330a8a700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:19.415044+0000 front 2024-06-10T03:05:19.415069+0000 (oldest deadline 2024-06-10T03:05:45.315110+0000) 2024-06-10T03:05:53.301 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:05:53 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:05:53.018+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-10T03:05:12.298026+0000 front 2024-06-10T03:05:12.298086+0000 (oldest deadline 2024-06-10T03:05:37.598149+0000) 2024-06-10T03:05:53.301 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:05:53 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:05:53.018+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:23.499450+0000 front 2024-06-10T03:05:23.499380+0000 (oldest deadline 2024-06-10T03:05:47.599354+0000) 2024-06-10T03:05:53.639 INFO:journalctl@ceph.osd.7.smithi120.stdout:Jun 10 03:05:53 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-7[54219]: 2024-06-10T03:05:53.340+0000 7f3b24ea2700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:23.717100+0000 front 2024-06-10T03:05:23.717037+0000 (oldest deadline 2024-06-10T03:05:47.817258+0000) 2024-06-10T03:05:53.639 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:05:53 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:05:53.432+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-10T03:05:15.396577+0000 front 2024-06-10T03:05:15.396655+0000 (oldest deadline 2024-06-10T03:05:37.096690+0000) 2024-06-10T03:05:53.639 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:05:53 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:05:53.432+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:22.397847+0000 front 2024-06-10T03:05:22.397969+0000 (oldest deadline 2024-06-10T03:05:46.497941+0000) 2024-06-10T03:05:53.704 INFO:journalctl@ceph.osd.5.smithi188.stdout:Jun 10 03:05:53 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-5[48629]: 2024-06-10T03:05:53.374+0000 7f5d445a7700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:19.483971+0000 front 2024-06-10T03:05:19.484087+0000 (oldest deadline 2024-06-10T03:05:44.784126+0000) 2024-06-10T03:05:53.704 INFO:journalctl@ceph.osd.5.smithi188.stdout:Jun 10 03:05:53 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-5[48629]: 2024-06-10T03:05:53.374+0000 7f5d445a7700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:31.785826+0000 front 2024-06-10T03:05:31.785773+0000 (oldest deadline 2024-06-10T03:05:52.885940+0000) 2024-06-10T03:05:53.704 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:05:53 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:05:53.545+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-10T03:05:13.354170+0000 front 2024-06-10T03:05:13.354237+0000 (oldest deadline 2024-06-10T03:05:38.053995+0000) 2024-06-10T03:05:53.705 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:05:53 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:05:53.545+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:22.755328+0000 front 2024-06-10T03:05:22.755363+0000 (oldest deadline 2024-06-10T03:05:45.655329+0000) 2024-06-10T03:05:53.730 INFO:journalctl@ceph.osd.6.smithi052.stdout:Jun 10 03:05:53 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-6[60649]: 2024-06-10T03:05:53.301+0000 7f4ce3c07700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:23.897917+0000 front 2024-06-10T03:05:23.897967+0000 (oldest deadline 2024-06-10T03:05:47.998039+0000) 2024-06-10T03:05:54.204 INFO:journalctl@ceph.osd.8.smithi188.stdout:Jun 10 03:05:53 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-8[55165]: 2024-06-10T03:05:53.814+0000 7fc330a8a700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:19.415044+0000 front 2024-06-10T03:05:19.415069+0000 (oldest deadline 2024-06-10T03:05:45.315110+0000) 2024-06-10T03:05:54.279 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:05:54 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:05:54.013+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-10T03:05:12.298026+0000 front 2024-06-10T03:05:12.298086+0000 (oldest deadline 2024-06-10T03:05:37.598149+0000) 2024-06-10T03:05:54.279 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:05:54 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:05:54.013+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:23.499450+0000 front 2024-06-10T03:05:23.499380+0000 (oldest deadline 2024-06-10T03:05:47.599354+0000) 2024-06-10T03:05:54.280 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:05:54 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:05:54.013+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:27.600040+0000 front 2024-06-10T03:05:27.600064+0000 (oldest deadline 2024-06-10T03:05:53.499906+0000) 2024-06-10T03:05:54.639 INFO:journalctl@ceph.osd.7.smithi120.stdout:Jun 10 03:05:54 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-7[54219]: 2024-06-10T03:05:54.387+0000 7f3b24ea2700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:23.717100+0000 front 2024-06-10T03:05:23.717037+0000 (oldest deadline 2024-06-10T03:05:47.817258+0000) 2024-06-10T03:05:54.639 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:05:54 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:05:54.472+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-10T03:05:15.396577+0000 front 2024-06-10T03:05:15.396655+0000 (oldest deadline 2024-06-10T03:05:37.096690+0000) 2024-06-10T03:05:54.639 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:05:54 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:05:54.472+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:22.397847+0000 front 2024-06-10T03:05:22.397969+0000 (oldest deadline 2024-06-10T03:05:46.497941+0000) 2024-06-10T03:05:54.704 INFO:journalctl@ceph.osd.5.smithi188.stdout:Jun 10 03:05:54 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-5[48629]: 2024-06-10T03:05:54.337+0000 7f5d445a7700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:19.483971+0000 front 2024-06-10T03:05:19.484087+0000 (oldest deadline 2024-06-10T03:05:44.784126+0000) 2024-06-10T03:05:54.704 INFO:journalctl@ceph.osd.5.smithi188.stdout:Jun 10 03:05:54 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-5[48629]: 2024-06-10T03:05:54.337+0000 7f5d445a7700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:31.785826+0000 front 2024-06-10T03:05:31.785773+0000 (oldest deadline 2024-06-10T03:05:52.885940+0000) 2024-06-10T03:05:54.704 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:05:54 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:05:54.550+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-10T03:05:13.354170+0000 front 2024-06-10T03:05:13.354237+0000 (oldest deadline 2024-06-10T03:05:38.053995+0000) 2024-06-10T03:05:54.705 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:05:54 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:05:54.550+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:22.755328+0000 front 2024-06-10T03:05:22.755363+0000 (oldest deadline 2024-06-10T03:05:45.655329+0000) 2024-06-10T03:05:54.705 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:05:54 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:05:54.550+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:29.156052+0000 front 2024-06-10T03:05:29.156095+0000 (oldest deadline 2024-06-10T03:05:53.856194+0000) 2024-06-10T03:05:54.727 INFO:journalctl@ceph.osd.6.smithi052.stdout:Jun 10 03:05:54 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-6[60649]: 2024-06-10T03:05:54.279+0000 7f4ce3c07700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:23.897917+0000 front 2024-06-10T03:05:23.897967+0000 (oldest deadline 2024-06-10T03:05:47.998039+0000) 2024-06-10T03:05:54.727 INFO:journalctl@ceph.osd.6.smithi052.stdout:Jun 10 03:05:54 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-6[60649]: 2024-06-10T03:05:54.279+0000 7f4ce3c07700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:27.998615+0000 front 2024-06-10T03:05:27.998568+0000 (oldest deadline 2024-06-10T03:05:53.898652+0000) 2024-06-10T03:05:55.204 INFO:journalctl@ceph.osd.8.smithi188.stdout:Jun 10 03:05:54 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-8[55165]: 2024-06-10T03:05:54.820+0000 7fc330a8a700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:19.415044+0000 front 2024-06-10T03:05:19.415069+0000 (oldest deadline 2024-06-10T03:05:45.315110+0000) 2024-06-10T03:05:55.204 INFO:journalctl@ceph.osd.8.smithi188.stdout:Jun 10 03:05:54 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-8[55165]: 2024-06-10T03:05:54.820+0000 7fc330a8a700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:32.217415+0000 front 2024-06-10T03:05:32.217588+0000 (oldest deadline 2024-06-10T03:05:54.418216+0000) 2024-06-10T03:05:55.477 INFO:journalctl@ceph.osd.6.smithi052.stdout:Jun 10 03:05:55 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-6[60649]: 2024-06-10T03:05:55.274+0000 7f4ce3c07700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:23.897917+0000 front 2024-06-10T03:05:23.897967+0000 (oldest deadline 2024-06-10T03:05:47.998039+0000) 2024-06-10T03:05:55.477 INFO:journalctl@ceph.osd.6.smithi052.stdout:Jun 10 03:05:55 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-6[60649]: 2024-06-10T03:05:55.274+0000 7f4ce3c07700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:27.998615+0000 front 2024-06-10T03:05:27.998568+0000 (oldest deadline 2024-06-10T03:05:53.898652+0000) 2024-06-10T03:05:55.478 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:05:55 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:05:55.061+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-10T03:05:12.298026+0000 front 2024-06-10T03:05:12.298086+0000 (oldest deadline 2024-06-10T03:05:37.598149+0000) 2024-06-10T03:05:55.478 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:05:55 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:05:55.061+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:23.499450+0000 front 2024-06-10T03:05:23.499380+0000 (oldest deadline 2024-06-10T03:05:47.599354+0000) 2024-06-10T03:05:55.478 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:05:55 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:05:55.061+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:27.600040+0000 front 2024-06-10T03:05:27.600064+0000 (oldest deadline 2024-06-10T03:05:53.499906+0000) 2024-06-10T03:05:55.704 INFO:journalctl@ceph.osd.5.smithi188.stdout:Jun 10 03:05:55 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-5[48629]: 2024-06-10T03:05:55.376+0000 7f5d445a7700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:19.483971+0000 front 2024-06-10T03:05:19.484087+0000 (oldest deadline 2024-06-10T03:05:44.784126+0000) 2024-06-10T03:05:55.704 INFO:journalctl@ceph.osd.5.smithi188.stdout:Jun 10 03:05:55 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-5[48629]: 2024-06-10T03:05:55.376+0000 7f5d445a7700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:31.785826+0000 front 2024-06-10T03:05:31.785773+0000 (oldest deadline 2024-06-10T03:05:52.885940+0000) 2024-06-10T03:05:55.704 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:05:55 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:05:55.579+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-10T03:05:13.354170+0000 front 2024-06-10T03:05:13.354237+0000 (oldest deadline 2024-06-10T03:05:38.053995+0000) 2024-06-10T03:05:55.704 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:05:55 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:05:55.579+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:22.755328+0000 front 2024-06-10T03:05:22.755363+0000 (oldest deadline 2024-06-10T03:05:45.655329+0000) 2024-06-10T03:05:55.704 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:05:55 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:05:55.579+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:29.156052+0000 front 2024-06-10T03:05:29.156095+0000 (oldest deadline 2024-06-10T03:05:53.856194+0000) 2024-06-10T03:05:55.889 INFO:journalctl@ceph.osd.7.smithi120.stdout:Jun 10 03:05:55 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-7[54219]: 2024-06-10T03:05:55.435+0000 7f3b24ea2700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:23.717100+0000 front 2024-06-10T03:05:23.717037+0000 (oldest deadline 2024-06-10T03:05:47.817258+0000) 2024-06-10T03:05:55.889 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:05:55 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:05:55.475+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-10T03:05:15.396577+0000 front 2024-06-10T03:05:15.396655+0000 (oldest deadline 2024-06-10T03:05:37.096690+0000) 2024-06-10T03:05:55.889 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:05:55 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:05:55.475+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:22.397847+0000 front 2024-06-10T03:05:22.397969+0000 (oldest deadline 2024-06-10T03:05:46.497941+0000) 2024-06-10T03:05:55.889 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:05:55 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:05:55.475+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:26.498562+0000 front 2024-06-10T03:05:26.498718+0000 (oldest deadline 2024-06-10T03:05:55.298987+0000) 2024-06-10T03:05:56.204 INFO:journalctl@ceph.osd.8.smithi188.stdout:Jun 10 03:05:55 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-8[55165]: 2024-06-10T03:05:55.814+0000 7fc330a8a700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:19.415044+0000 front 2024-06-10T03:05:19.415069+0000 (oldest deadline 2024-06-10T03:05:45.315110+0000) 2024-06-10T03:05:56.204 INFO:journalctl@ceph.osd.8.smithi188.stdout:Jun 10 03:05:55 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-8[55165]: 2024-06-10T03:05:55.814+0000 7fc330a8a700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:32.217415+0000 front 2024-06-10T03:05:32.217588+0000 (oldest deadline 2024-06-10T03:05:54.418216+0000) 2024-06-10T03:05:56.307 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:05:56 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:05:56.048+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-10T03:05:12.298026+0000 front 2024-06-10T03:05:12.298086+0000 (oldest deadline 2024-06-10T03:05:37.598149+0000) 2024-06-10T03:05:56.307 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:05:56 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:05:56.048+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:23.499450+0000 front 2024-06-10T03:05:23.499380+0000 (oldest deadline 2024-06-10T03:05:47.599354+0000) 2024-06-10T03:05:56.307 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:05:56 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:05:56.048+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:27.600040+0000 front 2024-06-10T03:05:27.600064+0000 (oldest deadline 2024-06-10T03:05:53.499906+0000) 2024-06-10T03:05:56.704 INFO:journalctl@ceph.osd.5.smithi188.stdout:Jun 10 03:05:56 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-5[48629]: 2024-06-10T03:05:56.385+0000 7f5d445a7700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:19.483971+0000 front 2024-06-10T03:05:19.484087+0000 (oldest deadline 2024-06-10T03:05:44.784126+0000) 2024-06-10T03:05:56.704 INFO:journalctl@ceph.osd.5.smithi188.stdout:Jun 10 03:05:56 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-5[48629]: 2024-06-10T03:05:56.385+0000 7f5d445a7700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:31.785826+0000 front 2024-06-10T03:05:31.785773+0000 (oldest deadline 2024-06-10T03:05:52.885940+0000) 2024-06-10T03:05:56.704 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:05:56 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:05:56.628+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-10T03:05:13.354170+0000 front 2024-06-10T03:05:13.354237+0000 (oldest deadline 2024-06-10T03:05:38.053995+0000) 2024-06-10T03:05:56.704 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:05:56 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:05:56.628+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:22.755328+0000 front 2024-06-10T03:05:22.755363+0000 (oldest deadline 2024-06-10T03:05:45.655329+0000) 2024-06-10T03:05:56.704 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:05:56 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:05:56.628+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:29.156052+0000 front 2024-06-10T03:05:29.156095+0000 (oldest deadline 2024-06-10T03:05:53.856194+0000) 2024-06-10T03:05:56.727 INFO:journalctl@ceph.osd.6.smithi052.stdout:Jun 10 03:05:56 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-6[60649]: 2024-06-10T03:05:56.306+0000 7f4ce3c07700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:23.897917+0000 front 2024-06-10T03:05:23.897967+0000 (oldest deadline 2024-06-10T03:05:47.998039+0000) 2024-06-10T03:05:56.727 INFO:journalctl@ceph.osd.6.smithi052.stdout:Jun 10 03:05:56 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-6[60649]: 2024-06-10T03:05:56.306+0000 7f4ce3c07700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:27.998615+0000 front 2024-06-10T03:05:27.998568+0000 (oldest deadline 2024-06-10T03:05:53.898652+0000) 2024-06-10T03:05:56.889 INFO:journalctl@ceph.osd.7.smithi120.stdout:Jun 10 03:05:56 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-7[54219]: 2024-06-10T03:05:56.406+0000 7f3b24ea2700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:23.717100+0000 front 2024-06-10T03:05:23.717037+0000 (oldest deadline 2024-06-10T03:05:47.817258+0000) 2024-06-10T03:05:56.889 INFO:journalctl@ceph.osd.7.smithi120.stdout:Jun 10 03:05:56 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-7[54219]: 2024-06-10T03:05:56.406+0000 7f3b24ea2700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:31.819036+0000 front 2024-06-10T03:05:31.819139+0000 (oldest deadline 2024-06-10T03:05:55.919249+0000) 2024-06-10T03:05:56.889 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:05:56 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:05:56.434+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-10T03:05:15.396577+0000 front 2024-06-10T03:05:15.396655+0000 (oldest deadline 2024-06-10T03:05:37.096690+0000) 2024-06-10T03:05:56.889 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:05:56 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:05:56.434+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:22.397847+0000 front 2024-06-10T03:05:22.397969+0000 (oldest deadline 2024-06-10T03:05:46.497941+0000) 2024-06-10T03:05:56.889 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:05:56 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:05:56.434+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:26.498562+0000 front 2024-06-10T03:05:26.498718+0000 (oldest deadline 2024-06-10T03:05:55.298987+0000) 2024-06-10T03:05:57.204 INFO:journalctl@ceph.osd.8.smithi188.stdout:Jun 10 03:05:56 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-8[55165]: 2024-06-10T03:05:56.862+0000 7fc330a8a700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:19.415044+0000 front 2024-06-10T03:05:19.415069+0000 (oldest deadline 2024-06-10T03:05:45.315110+0000) 2024-06-10T03:05:57.204 INFO:journalctl@ceph.osd.8.smithi188.stdout:Jun 10 03:05:56 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-8[55165]: 2024-06-10T03:05:56.862+0000 7fc330a8a700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:32.217415+0000 front 2024-06-10T03:05:32.217588+0000 (oldest deadline 2024-06-10T03:05:54.418216+0000) 2024-06-10T03:05:57.477 INFO:journalctl@ceph.osd.6.smithi052.stdout:Jun 10 03:05:57 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-6[60649]: 2024-06-10T03:05:57.325+0000 7f4ce3c07700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:23.897917+0000 front 2024-06-10T03:05:23.897967+0000 (oldest deadline 2024-06-10T03:05:47.998039+0000) 2024-06-10T03:05:57.477 INFO:journalctl@ceph.osd.6.smithi052.stdout:Jun 10 03:05:57 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-6[60649]: 2024-06-10T03:05:57.325+0000 7f4ce3c07700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:27.998615+0000 front 2024-06-10T03:05:27.998568+0000 (oldest deadline 2024-06-10T03:05:53.898652+0000) 2024-06-10T03:05:57.478 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:05:57 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:05:57.079+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-10T03:05:12.298026+0000 front 2024-06-10T03:05:12.298086+0000 (oldest deadline 2024-06-10T03:05:37.598149+0000) 2024-06-10T03:05:57.478 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:05:57 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:05:57.079+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:23.499450+0000 front 2024-06-10T03:05:23.499380+0000 (oldest deadline 2024-06-10T03:05:47.599354+0000) 2024-06-10T03:05:57.478 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:05:57 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:05:57.079+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:27.600040+0000 front 2024-06-10T03:05:27.600064+0000 (oldest deadline 2024-06-10T03:05:53.499906+0000) 2024-06-10T03:05:57.790 INFO:journalctl@ceph.osd.7.smithi120.stdout:Jun 10 03:05:57 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-7[54219]: 2024-06-10T03:05:57.429+0000 7f3b24ea2700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:23.717100+0000 front 2024-06-10T03:05:23.717037+0000 (oldest deadline 2024-06-10T03:05:47.817258+0000) 2024-06-10T03:05:57.790 INFO:journalctl@ceph.osd.7.smithi120.stdout:Jun 10 03:05:57 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-7[54219]: 2024-06-10T03:05:57.429+0000 7f3b24ea2700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:31.819036+0000 front 2024-06-10T03:05:31.819139+0000 (oldest deadline 2024-06-10T03:05:55.919249+0000) 2024-06-10T03:05:57.791 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:05:57 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:05:57.466+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-10T03:05:15.396577+0000 front 2024-06-10T03:05:15.396655+0000 (oldest deadline 2024-06-10T03:05:37.096690+0000) 2024-06-10T03:05:57.791 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:05:57 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:05:57.466+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:22.397847+0000 front 2024-06-10T03:05:22.397969+0000 (oldest deadline 2024-06-10T03:05:46.497941+0000) 2024-06-10T03:05:57.791 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:05:57 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:05:57.466+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:26.498562+0000 front 2024-06-10T03:05:26.498718+0000 (oldest deadline 2024-06-10T03:05:55.298987+0000) 2024-06-10T03:05:57.829 INFO:journalctl@ceph.osd.5.smithi188.stdout:Jun 10 03:05:57 smithi188 podman[72169]: 2024-06-10 03:05:57.522408807 +0000 UTC m=+5.648040229 container died 3f6cf57d2a0fe16beebea7d37b086ff348bcf9923f2a4b8811c86ee6fbfb1a43 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0) 2024-06-10T03:05:57.830 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:05:57 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:05:57.654+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-10T03:05:13.354170+0000 front 2024-06-10T03:05:13.354237+0000 (oldest deadline 2024-06-10T03:05:38.053995+0000) 2024-06-10T03:05:57.830 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:05:57 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:05:57.654+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:22.755328+0000 front 2024-06-10T03:05:22.755363+0000 (oldest deadline 2024-06-10T03:05:45.655329+0000) 2024-06-10T03:05:57.830 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:05:57 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:05:57.654+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:29.156052+0000 front 2024-06-10T03:05:29.156095+0000 (oldest deadline 2024-06-10T03:05:53.856194+0000) 2024-06-10T03:05:58.196 INFO:journalctl@ceph.osd.8.smithi188.stdout:Jun 10 03:05:57 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-8[55165]: 2024-06-10T03:05:57.910+0000 7fc330a8a700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:19.415044+0000 front 2024-06-10T03:05:19.415069+0000 (oldest deadline 2024-06-10T03:05:45.315110+0000) 2024-06-10T03:05:58.197 INFO:journalctl@ceph.osd.8.smithi188.stdout:Jun 10 03:05:57 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-8[55165]: 2024-06-10T03:05:57.910+0000 7fc330a8a700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:32.217415+0000 front 2024-06-10T03:05:32.217588+0000 (oldest deadline 2024-06-10T03:05:54.418216+0000) 2024-06-10T03:05:58.197 INFO:journalctl@ceph.osd.5.smithi188.stdout:Jun 10 03:05:57 smithi188 podman[72169]: 2024-06-10 03:05:57.953758926 +0000 UTC m=+6.079390345 container cleanup 3f6cf57d2a0fe16beebea7d37b086ff348bcf9923f2a4b8811c86ee6fbfb1a43 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-5, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5) 2024-06-10T03:05:58.197 INFO:journalctl@ceph.osd.5.smithi188.stdout:Jun 10 03:05:57 smithi188 bash[72169]: ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-5 2024-06-10T03:05:58.333 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:05:58 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:05:58.050+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-10T03:05:12.298026+0000 front 2024-06-10T03:05:12.298086+0000 (oldest deadline 2024-06-10T03:05:37.598149+0000) 2024-06-10T03:05:58.333 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:05:58 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:05:58.050+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:23.499450+0000 front 2024-06-10T03:05:23.499380+0000 (oldest deadline 2024-06-10T03:05:47.599354+0000) 2024-06-10T03:05:58.333 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:05:58 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:05:58.050+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:27.600040+0000 front 2024-06-10T03:05:27.600064+0000 (oldest deadline 2024-06-10T03:05:53.499906+0000) 2024-06-10T03:05:58.454 INFO:journalctl@ceph.osd.5.smithi188.stdout:Jun 10 03:05:58 smithi188 podman[72189]: 2024-06-10 03:05:58.197033883 +0000 UTC m=+0.671353733 container remove 3f6cf57d2a0fe16beebea7d37b086ff348bcf9923f2a4b8811c86ee6fbfb1a43 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0) 2024-06-10T03:05:58.727 INFO:journalctl@ceph.osd.6.smithi052.stdout:Jun 10 03:05:58 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-6[60649]: 2024-06-10T03:05:58.332+0000 7f4ce3c07700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:23.897917+0000 front 2024-06-10T03:05:23.897967+0000 (oldest deadline 2024-06-10T03:05:47.998039+0000) 2024-06-10T03:05:58.727 INFO:journalctl@ceph.osd.6.smithi052.stdout:Jun 10 03:05:58 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-6[60649]: 2024-06-10T03:05:58.332+0000 7f4ce3c07700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:27.998615+0000 front 2024-06-10T03:05:27.998568+0000 (oldest deadline 2024-06-10T03:05:53.898652+0000) 2024-06-10T03:05:58.805 INFO:journalctl@ceph.osd.7.smithi120.stdout:Jun 10 03:05:58 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-7[54219]: 2024-06-10T03:05:58.415+0000 7f3b24ea2700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:23.717100+0000 front 2024-06-10T03:05:23.717037+0000 (oldest deadline 2024-06-10T03:05:47.817258+0000) 2024-06-10T03:05:58.805 INFO:journalctl@ceph.osd.7.smithi120.stdout:Jun 10 03:05:58 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-7[54219]: 2024-06-10T03:05:58.415+0000 7f3b24ea2700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:31.819036+0000 front 2024-06-10T03:05:31.819139+0000 (oldest deadline 2024-06-10T03:05:55.919249+0000) 2024-06-10T03:05:58.806 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:05:58 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:05:58.473+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-10T03:05:15.396577+0000 front 2024-06-10T03:05:15.396655+0000 (oldest deadline 2024-06-10T03:05:37.096690+0000) 2024-06-10T03:05:58.806 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:05:58 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:05:58.473+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:22.397847+0000 front 2024-06-10T03:05:22.397969+0000 (oldest deadline 2024-06-10T03:05:46.497941+0000) 2024-06-10T03:05:58.806 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:05:58 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:05:58.473+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:26.498562+0000 front 2024-06-10T03:05:26.498718+0000 (oldest deadline 2024-06-10T03:05:55.298987+0000) 2024-06-10T03:05:58.880 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:05:58 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:05:58.678+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-10T03:05:13.354170+0000 front 2024-06-10T03:05:13.354237+0000 (oldest deadline 2024-06-10T03:05:38.053995+0000) 2024-06-10T03:05:58.880 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:05:58 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:05:58.679+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:22.755328+0000 front 2024-06-10T03:05:22.755363+0000 (oldest deadline 2024-06-10T03:05:45.655329+0000) 2024-06-10T03:05:58.880 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:05:58 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:05:58.679+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:29.156052+0000 front 2024-06-10T03:05:29.156095+0000 (oldest deadline 2024-06-10T03:05:53.856194+0000) 2024-06-10T03:05:58.881 INFO:journalctl@ceph.osd.5.smithi188.stdout:Jun 10 03:05:58 smithi188 podman[72275]: 2024-06-10 03:05:58.529246484 +0000 UTC m=+0.017255484 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-10T03:05:58.881 INFO:journalctl@ceph.osd.5.smithi188.stdout:Jun 10 03:05:58 smithi188 podman[72275]: 2024-06-10 03:05:58.648179834 +0000 UTC m=+0.136188862 container create 33922d0e4f7156b7bc78cb1feddb849bed76456012d48257b84a51ea691a9f21 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-5-deactivate, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, RELEASE=squid-3d89acc, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-10T03:05:58.881 INFO:journalctl@ceph.osd.5.smithi188.stdout:Jun 10 03:05:58 smithi188 podman[72275]: 2024-06-10 03:05:58.876044173 +0000 UTC m=+0.364053170 container init 33922d0e4f7156b7bc78cb1feddb849bed76456012d48257b84a51ea691a9f21 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-5-deactivate, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-06-10T03:05:58.881 INFO:journalctl@ceph.osd.5.smithi188.stdout:Jun 10 03:05:58 smithi188 podman[72275]: 2024-06-10 03:05:58.879289346 +0000 UTC m=+0.367298333 container start 33922d0e4f7156b7bc78cb1feddb849bed76456012d48257b84a51ea691a9f21 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-5-deactivate, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True) 2024-06-10T03:05:59.170 INFO:journalctl@ceph.osd.8.smithi188.stdout:Jun 10 03:05:58 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-8[55165]: 2024-06-10T03:05:58.924+0000 7fc330a8a700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:19.415044+0000 front 2024-06-10T03:05:19.415069+0000 (oldest deadline 2024-06-10T03:05:45.315110+0000) 2024-06-10T03:05:59.170 INFO:journalctl@ceph.osd.8.smithi188.stdout:Jun 10 03:05:58 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-8[55165]: 2024-06-10T03:05:58.924+0000 7fc330a8a700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:32.217415+0000 front 2024-06-10T03:05:32.217588+0000 (oldest deadline 2024-06-10T03:05:54.418216+0000) 2024-06-10T03:05:59.170 INFO:journalctl@ceph.osd.5.smithi188.stdout:Jun 10 03:05:58 smithi188 podman[72275]: 2024-06-10 03:05:58.967644316 +0000 UTC m=+0.455653313 container attach 33922d0e4f7156b7bc78cb1feddb849bed76456012d48257b84a51ea691a9f21 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-5-deactivate, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-06-10T03:05:59.342 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:05:59 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:05:59.049+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-10T03:05:12.298026+0000 front 2024-06-10T03:05:12.298086+0000 (oldest deadline 2024-06-10T03:05:37.598149+0000) 2024-06-10T03:05:59.342 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:05:59 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:05:59.049+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:23.499450+0000 front 2024-06-10T03:05:23.499380+0000 (oldest deadline 2024-06-10T03:05:47.599354+0000) 2024-06-10T03:05:59.342 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:05:59 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:05:59.049+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:27.600040+0000 front 2024-06-10T03:05:27.600064+0000 (oldest deadline 2024-06-10T03:05:53.499906+0000) 2024-06-10T03:05:59.454 INFO:journalctl@ceph.osd.5.smithi188.stdout:Jun 10 03:05:59 smithi188 podman[72275]: 2024-06-10 03:05:59.170284043 +0000 UTC m=+0.658293030 container died 33922d0e4f7156b7bc78cb1feddb849bed76456012d48257b84a51ea691a9f21 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-5-deactivate, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True) 2024-06-10T03:05:59.636 INFO:journalctl@ceph.osd.6.smithi052.stdout:Jun 10 03:05:59 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-6[60649]: 2024-06-10T03:05:59.342+0000 7f4ce3c07700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:23.897917+0000 front 2024-06-10T03:05:23.897967+0000 (oldest deadline 2024-06-10T03:05:47.998039+0000) 2024-06-10T03:05:59.636 INFO:journalctl@ceph.osd.6.smithi052.stdout:Jun 10 03:05:59 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-6[60649]: 2024-06-10T03:05:59.342+0000 7f4ce3c07700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:27.998615+0000 front 2024-06-10T03:05:27.998568+0000 (oldest deadline 2024-06-10T03:05:53.898652+0000) 2024-06-10T03:05:59.821 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:05:59 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:05:59.658+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-10T03:05:13.354170+0000 front 2024-06-10T03:05:13.354237+0000 (oldest deadline 2024-06-10T03:05:38.053995+0000) 2024-06-10T03:05:59.821 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:05:59 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:05:59.658+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:22.755328+0000 front 2024-06-10T03:05:22.755363+0000 (oldest deadline 2024-06-10T03:05:45.655329+0000) 2024-06-10T03:05:59.821 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:05:59 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:05:59.658+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:29.156052+0000 front 2024-06-10T03:05:29.156095+0000 (oldest deadline 2024-06-10T03:05:53.856194+0000) 2024-06-10T03:05:59.849 DEBUG:teuthology.orchestra.run.smithi188:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968@osd.5.service' 2024-06-10T03:05:59.888 INFO:journalctl@ceph.osd.7.smithi120.stdout:Jun 10 03:05:59 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-7[54219]: 2024-06-10T03:05:59.394+0000 7f3b24ea2700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:23.717100+0000 front 2024-06-10T03:05:23.717037+0000 (oldest deadline 2024-06-10T03:05:47.817258+0000) 2024-06-10T03:05:59.889 INFO:journalctl@ceph.osd.7.smithi120.stdout:Jun 10 03:05:59 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-7[54219]: 2024-06-10T03:05:59.394+0000 7f3b24ea2700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:31.819036+0000 front 2024-06-10T03:05:31.819139+0000 (oldest deadline 2024-06-10T03:05:55.919249+0000) 2024-06-10T03:05:59.889 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:05:59 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:05:59.522+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-10T03:05:15.396577+0000 front 2024-06-10T03:05:15.396655+0000 (oldest deadline 2024-06-10T03:05:37.096690+0000) 2024-06-10T03:05:59.889 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:05:59 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:05:59.522+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:22.397847+0000 front 2024-06-10T03:05:22.397969+0000 (oldest deadline 2024-06-10T03:05:46.497941+0000) 2024-06-10T03:05:59.889 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:05:59 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:05:59.522+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:26.498562+0000 front 2024-06-10T03:05:26.498718+0000 (oldest deadline 2024-06-10T03:05:55.298987+0000) 2024-06-10T03:05:59.926 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-10T03:05:59.926 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-06-10T03:05:59.927 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-06-10T03:05:59.927 DEBUG:teuthology.orchestra.run.smithi052:> sudo systemctl stop ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968@osd.6 2024-06-10T03:06:00.204 INFO:journalctl@ceph.osd.8.smithi188.stdout:Jun 10 03:05:59 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-8[55165]: 2024-06-10T03:05:59.966+0000 7fc330a8a700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:19.415044+0000 front 2024-06-10T03:05:19.415069+0000 (oldest deadline 2024-06-10T03:05:45.315110+0000) 2024-06-10T03:06:00.204 INFO:journalctl@ceph.osd.8.smithi188.stdout:Jun 10 03:05:59 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-8[55165]: 2024-06-10T03:05:59.966+0000 7fc330a8a700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:32.217415+0000 front 2024-06-10T03:05:32.217588+0000 (oldest deadline 2024-06-10T03:05:54.418216+0000) 2024-06-10T03:06:00.227 INFO:journalctl@ceph.osd.6.smithi052.stdout:Jun 10 03:05:59 smithi052 systemd[1]: Stopping Ceph osd.6 for d8676e9e-26d2-11ef-bc9b-c7b262605968... 2024-06-10T03:06:00.228 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:06:00 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:06:00.078+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-10T03:05:12.298026+0000 front 2024-06-10T03:05:12.298086+0000 (oldest deadline 2024-06-10T03:05:37.598149+0000) 2024-06-10T03:06:00.228 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:06:00 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:06:00.078+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:23.499450+0000 front 2024-06-10T03:05:23.499380+0000 (oldest deadline 2024-06-10T03:05:47.599354+0000) 2024-06-10T03:06:00.228 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:06:00 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:06:00.078+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:27.600040+0000 front 2024-06-10T03:05:27.600064+0000 (oldest deadline 2024-06-10T03:05:53.499906+0000) 2024-06-10T03:06:00.639 INFO:journalctl@ceph.osd.7.smithi120.stdout:Jun 10 03:06:00 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-7[54219]: 2024-06-10T03:06:00.359+0000 7f3b24ea2700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:23.717100+0000 front 2024-06-10T03:05:23.717037+0000 (oldest deadline 2024-06-10T03:05:47.817258+0000) 2024-06-10T03:06:00.639 INFO:journalctl@ceph.osd.7.smithi120.stdout:Jun 10 03:06:00 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-7[54219]: 2024-06-10T03:06:00.359+0000 7f3b24ea2700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:31.819036+0000 front 2024-06-10T03:05:31.819139+0000 (oldest deadline 2024-06-10T03:05:55.919249+0000) 2024-06-10T03:06:00.639 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:00 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:00.514+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-10T03:05:15.396577+0000 front 2024-06-10T03:05:15.396655+0000 (oldest deadline 2024-06-10T03:05:37.096690+0000) 2024-06-10T03:06:00.639 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:00 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:00.514+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:22.397847+0000 front 2024-06-10T03:05:22.397969+0000 (oldest deadline 2024-06-10T03:05:46.497941+0000) 2024-06-10T03:06:00.639 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:00 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:00.514+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:26.498562+0000 front 2024-06-10T03:05:26.498718+0000 (oldest deadline 2024-06-10T03:05:55.298987+0000) 2024-06-10T03:06:00.661 INFO:journalctl@ceph.osd.6.smithi052.stdout:Jun 10 03:06:00 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-6[60649]: 2024-06-10T03:06:00.292+0000 7f4ceb61b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-10T03:06:00.661 INFO:journalctl@ceph.osd.6.smithi052.stdout:Jun 10 03:06:00 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-6[60649]: 2024-06-10T03:06:00.292+0000 7f4ceb61b700 -1 osd.6 104 *** Got signal Terminated *** 2024-06-10T03:06:00.661 INFO:journalctl@ceph.osd.6.smithi052.stdout:Jun 10 03:06:00 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-6[60649]: 2024-06-10T03:06:00.292+0000 7f4ceb61b700 -1 osd.6 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-10T03:06:00.661 INFO:journalctl@ceph.osd.6.smithi052.stdout:Jun 10 03:06:00 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-6[60649]: 2024-06-10T03:06:00.364+0000 7f4ce3c07700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:23.897917+0000 front 2024-06-10T03:05:23.897967+0000 (oldest deadline 2024-06-10T03:05:47.998039+0000) 2024-06-10T03:06:00.661 INFO:journalctl@ceph.osd.6.smithi052.stdout:Jun 10 03:06:00 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-6[60649]: 2024-06-10T03:06:00.364+0000 7f4ce3c07700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:27.998615+0000 front 2024-06-10T03:05:27.998568+0000 (oldest deadline 2024-06-10T03:05:53.898652+0000) 2024-06-10T03:06:00.954 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:00 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:00.625+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-10T03:05:13.354170+0000 front 2024-06-10T03:05:13.354237+0000 (oldest deadline 2024-06-10T03:05:38.053995+0000) 2024-06-10T03:06:00.954 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:00 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:00.625+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:22.755328+0000 front 2024-06-10T03:05:22.755363+0000 (oldest deadline 2024-06-10T03:05:45.655329+0000) 2024-06-10T03:06:00.954 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:00 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:00.625+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:29.156052+0000 front 2024-06-10T03:05:29.156095+0000 (oldest deadline 2024-06-10T03:05:53.856194+0000) 2024-06-10T03:06:01.402 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:06:01 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:06:01.046+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-10T03:05:12.298026+0000 front 2024-06-10T03:05:12.298086+0000 (oldest deadline 2024-06-10T03:05:37.598149+0000) 2024-06-10T03:06:01.402 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:06:01 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:06:01.046+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:23.499450+0000 front 2024-06-10T03:05:23.499380+0000 (oldest deadline 2024-06-10T03:05:47.599354+0000) 2024-06-10T03:06:01.402 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:06:01 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:06:01.046+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:27.600040+0000 front 2024-06-10T03:05:27.600064+0000 (oldest deadline 2024-06-10T03:05:53.499906+0000) 2024-06-10T03:06:01.454 INFO:journalctl@ceph.osd.8.smithi188.stdout:Jun 10 03:06:00 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-8[55165]: 2024-06-10T03:06:00.985+0000 7fc330a8a700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:19.415044+0000 front 2024-06-10T03:05:19.415069+0000 (oldest deadline 2024-06-10T03:05:45.315110+0000) 2024-06-10T03:06:01.454 INFO:journalctl@ceph.osd.8.smithi188.stdout:Jun 10 03:06:00 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-8[55165]: 2024-06-10T03:06:00.985+0000 7fc330a8a700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:32.217415+0000 front 2024-06-10T03:05:32.217588+0000 (oldest deadline 2024-06-10T03:05:54.418216+0000) 2024-06-10T03:06:01.639 INFO:journalctl@ceph.osd.7.smithi120.stdout:Jun 10 03:06:01 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-7[54219]: 2024-06-10T03:06:01.319+0000 7f3b24ea2700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:23.717100+0000 front 2024-06-10T03:05:23.717037+0000 (oldest deadline 2024-06-10T03:05:47.817258+0000) 2024-06-10T03:06:01.639 INFO:journalctl@ceph.osd.7.smithi120.stdout:Jun 10 03:06:01 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-7[54219]: 2024-06-10T03:06:01.319+0000 7f3b24ea2700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:31.819036+0000 front 2024-06-10T03:05:31.819139+0000 (oldest deadline 2024-06-10T03:05:55.919249+0000) 2024-06-10T03:06:01.639 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:01 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:01.469+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-10T03:05:15.396577+0000 front 2024-06-10T03:05:15.396655+0000 (oldest deadline 2024-06-10T03:05:37.096690+0000) 2024-06-10T03:06:01.639 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:01 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:01.469+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:22.397847+0000 front 2024-06-10T03:05:22.397969+0000 (oldest deadline 2024-06-10T03:05:46.497941+0000) 2024-06-10T03:06:01.639 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:01 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:01.469+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:26.498562+0000 front 2024-06-10T03:05:26.498718+0000 (oldest deadline 2024-06-10T03:05:55.298987+0000) 2024-06-10T03:06:01.727 INFO:journalctl@ceph.osd.6.smithi052.stdout:Jun 10 03:06:01 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-6[60649]: 2024-06-10T03:06:01.402+0000 7f4ce3c07700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:23.897917+0000 front 2024-06-10T03:05:23.897967+0000 (oldest deadline 2024-06-10T03:05:47.998039+0000) 2024-06-10T03:06:01.727 INFO:journalctl@ceph.osd.6.smithi052.stdout:Jun 10 03:06:01 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-6[60649]: 2024-06-10T03:06:01.402+0000 7f4ce3c07700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:27.998615+0000 front 2024-06-10T03:05:27.998568+0000 (oldest deadline 2024-06-10T03:05:53.898652+0000) 2024-06-10T03:06:01.954 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:01 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:01.672+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-10T03:05:13.354170+0000 front 2024-06-10T03:05:13.354237+0000 (oldest deadline 2024-06-10T03:05:38.053995+0000) 2024-06-10T03:06:01.954 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:01 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:01.672+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:22.755328+0000 front 2024-06-10T03:05:22.755363+0000 (oldest deadline 2024-06-10T03:05:45.655329+0000) 2024-06-10T03:06:01.954 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:01 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:01.672+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:29.156052+0000 front 2024-06-10T03:05:29.156095+0000 (oldest deadline 2024-06-10T03:05:53.856194+0000) 2024-06-10T03:06:02.375 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:06:02 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:06:02.063+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-10T03:05:12.298026+0000 front 2024-06-10T03:05:12.298086+0000 (oldest deadline 2024-06-10T03:05:37.598149+0000) 2024-06-10T03:06:02.375 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:06:02 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:06:02.063+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:23.499450+0000 front 2024-06-10T03:05:23.499380+0000 (oldest deadline 2024-06-10T03:05:47.599354+0000) 2024-06-10T03:06:02.375 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:06:02 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:06:02.063+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:27.600040+0000 front 2024-06-10T03:05:27.600064+0000 (oldest deadline 2024-06-10T03:05:53.499906+0000) 2024-06-10T03:06:02.454 INFO:journalctl@ceph.osd.8.smithi188.stdout:Jun 10 03:06:02 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-8[55165]: 2024-06-10T03:06:02.010+0000 7fc330a8a700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:19.415044+0000 front 2024-06-10T03:05:19.415069+0000 (oldest deadline 2024-06-10T03:05:45.315110+0000) 2024-06-10T03:06:02.454 INFO:journalctl@ceph.osd.8.smithi188.stdout:Jun 10 03:06:02 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-8[55165]: 2024-06-10T03:06:02.010+0000 7fc330a8a700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:32.217415+0000 front 2024-06-10T03:05:32.217588+0000 (oldest deadline 2024-06-10T03:05:54.418216+0000) 2024-06-10T03:06:02.639 INFO:journalctl@ceph.osd.7.smithi120.stdout:Jun 10 03:06:02 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-7[54219]: 2024-06-10T03:06:02.294+0000 7f3b24ea2700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:23.717100+0000 front 2024-06-10T03:05:23.717037+0000 (oldest deadline 2024-06-10T03:05:47.817258+0000) 2024-06-10T03:06:02.639 INFO:journalctl@ceph.osd.7.smithi120.stdout:Jun 10 03:06:02 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-7[54219]: 2024-06-10T03:06:02.294+0000 7f3b24ea2700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:31.819036+0000 front 2024-06-10T03:05:31.819139+0000 (oldest deadline 2024-06-10T03:05:55.919249+0000) 2024-06-10T03:06:02.639 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:02 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:02.444+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-10T03:05:15.396577+0000 front 2024-06-10T03:05:15.396655+0000 (oldest deadline 2024-06-10T03:05:37.096690+0000) 2024-06-10T03:06:02.639 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:02 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:02.444+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:22.397847+0000 front 2024-06-10T03:05:22.397969+0000 (oldest deadline 2024-06-10T03:05:46.497941+0000) 2024-06-10T03:06:02.639 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:02 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:02.444+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:26.498562+0000 front 2024-06-10T03:05:26.498718+0000 (oldest deadline 2024-06-10T03:05:55.298987+0000) 2024-06-10T03:06:02.727 INFO:journalctl@ceph.osd.6.smithi052.stdout:Jun 10 03:06:02 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-6[60649]: 2024-06-10T03:06:02.374+0000 7f4ce3c07700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:23.897917+0000 front 2024-06-10T03:05:23.897967+0000 (oldest deadline 2024-06-10T03:05:47.998039+0000) 2024-06-10T03:06:02.727 INFO:journalctl@ceph.osd.6.smithi052.stdout:Jun 10 03:06:02 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-6[60649]: 2024-06-10T03:06:02.374+0000 7f4ce3c07700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:27.998615+0000 front 2024-06-10T03:05:27.998568+0000 (oldest deadline 2024-06-10T03:05:53.898652+0000) 2024-06-10T03:06:02.954 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:02 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:02.690+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-10T03:05:13.354170+0000 front 2024-06-10T03:05:13.354237+0000 (oldest deadline 2024-06-10T03:05:38.053995+0000) 2024-06-10T03:06:02.955 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:02 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:02.690+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:22.755328+0000 front 2024-06-10T03:05:22.755363+0000 (oldest deadline 2024-06-10T03:05:45.655329+0000) 2024-06-10T03:06:02.955 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:02 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:02.690+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:29.156052+0000 front 2024-06-10T03:05:29.156095+0000 (oldest deadline 2024-06-10T03:05:53.856194+0000) 2024-06-10T03:06:03.418 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:06:03 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:06:03.019+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-10T03:05:12.298026+0000 front 2024-06-10T03:05:12.298086+0000 (oldest deadline 2024-06-10T03:05:37.598149+0000) 2024-06-10T03:06:03.418 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:06:03 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:06:03.019+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:23.499450+0000 front 2024-06-10T03:05:23.499380+0000 (oldest deadline 2024-06-10T03:05:47.599354+0000) 2024-06-10T03:06:03.418 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:06:03 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:06:03.019+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:27.600040+0000 front 2024-06-10T03:05:27.600064+0000 (oldest deadline 2024-06-10T03:05:53.499906+0000) 2024-06-10T03:06:03.418 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:06:03 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:06:03.019+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-06-10T03:05:38.800668+0000 front 2024-06-10T03:05:38.800723+0000 (oldest deadline 2024-06-10T03:06:02.900794+0000) 2024-06-10T03:06:03.454 INFO:journalctl@ceph.osd.8.smithi188.stdout:Jun 10 03:06:03 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-8[55165]: 2024-06-10T03:06:03.058+0000 7fc330a8a700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:19.415044+0000 front 2024-06-10T03:05:19.415069+0000 (oldest deadline 2024-06-10T03:05:45.315110+0000) 2024-06-10T03:06:03.454 INFO:journalctl@ceph.osd.8.smithi188.stdout:Jun 10 03:06:03 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-8[55165]: 2024-06-10T03:06:03.058+0000 7fc330a8a700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:32.217415+0000 front 2024-06-10T03:05:32.217588+0000 (oldest deadline 2024-06-10T03:05:54.418216+0000) 2024-06-10T03:06:03.639 INFO:journalctl@ceph.osd.7.smithi120.stdout:Jun 10 03:06:03 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-7[54219]: 2024-06-10T03:06:03.319+0000 7f3b24ea2700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:23.717100+0000 front 2024-06-10T03:05:23.717037+0000 (oldest deadline 2024-06-10T03:05:47.817258+0000) 2024-06-10T03:06:03.639 INFO:journalctl@ceph.osd.7.smithi120.stdout:Jun 10 03:06:03 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-7[54219]: 2024-06-10T03:06:03.319+0000 7f3b24ea2700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:31.819036+0000 front 2024-06-10T03:05:31.819139+0000 (oldest deadline 2024-06-10T03:05:55.919249+0000) 2024-06-10T03:06:03.639 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:03 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:03.414+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-10T03:05:15.396577+0000 front 2024-06-10T03:05:15.396655+0000 (oldest deadline 2024-06-10T03:05:37.096690+0000) 2024-06-10T03:06:03.639 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:03 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:03.414+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:22.397847+0000 front 2024-06-10T03:05:22.397969+0000 (oldest deadline 2024-06-10T03:05:46.497941+0000) 2024-06-10T03:06:03.639 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:03 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:03.414+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:26.498562+0000 front 2024-06-10T03:05:26.498718+0000 (oldest deadline 2024-06-10T03:05:55.298987+0000) 2024-06-10T03:06:03.724 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:03 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:03.680+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-10T03:05:13.354170+0000 front 2024-06-10T03:05:13.354237+0000 (oldest deadline 2024-06-10T03:05:38.053995+0000) 2024-06-10T03:06:03.724 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:03 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:03.680+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:22.755328+0000 front 2024-06-10T03:05:22.755363+0000 (oldest deadline 2024-06-10T03:05:45.655329+0000) 2024-06-10T03:06:03.724 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:03 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:03.680+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:29.156052+0000 front 2024-06-10T03:05:29.156095+0000 (oldest deadline 2024-06-10T03:05:53.856194+0000) 2024-06-10T03:06:03.727 INFO:journalctl@ceph.osd.6.smithi052.stdout:Jun 10 03:06:03 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-6[60649]: 2024-06-10T03:06:03.417+0000 7f4ce3c07700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:23.897917+0000 front 2024-06-10T03:05:23.897967+0000 (oldest deadline 2024-06-10T03:05:47.998039+0000) 2024-06-10T03:06:03.727 INFO:journalctl@ceph.osd.6.smithi052.stdout:Jun 10 03:06:03 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-6[60649]: 2024-06-10T03:06:03.417+0000 7f4ce3c07700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:27.998615+0000 front 2024-06-10T03:05:27.998568+0000 (oldest deadline 2024-06-10T03:05:53.898652+0000) 2024-06-10T03:06:03.727 INFO:journalctl@ceph.osd.6.smithi052.stdout:Jun 10 03:06:03 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-6[60649]: 2024-06-10T03:06:03.417+0000 7f4ce3c07700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-06-10T03:05:37.999602+0000 front 2024-06-10T03:05:37.999631+0000 (oldest deadline 2024-06-10T03:06:03.299805+0000) 2024-06-10T03:06:04.428 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:06:04 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:06:04.062+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-10T03:05:12.298026+0000 front 2024-06-10T03:05:12.298086+0000 (oldest deadline 2024-06-10T03:05:37.598149+0000) 2024-06-10T03:06:04.428 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:06:04 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:06:04.062+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:23.499450+0000 front 2024-06-10T03:05:23.499380+0000 (oldest deadline 2024-06-10T03:05:47.599354+0000) 2024-06-10T03:06:04.428 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:06:04 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:06:04.062+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:27.600040+0000 front 2024-06-10T03:05:27.600064+0000 (oldest deadline 2024-06-10T03:05:53.499906+0000) 2024-06-10T03:06:04.428 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:06:04 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:06:04.062+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-06-10T03:05:38.800668+0000 front 2024-06-10T03:05:38.800723+0000 (oldest deadline 2024-06-10T03:06:02.900794+0000) 2024-06-10T03:06:04.453 INFO:journalctl@ceph.osd.8.smithi188.stdout:Jun 10 03:06:04 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-8[55165]: 2024-06-10T03:06:04.026+0000 7fc330a8a700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:19.415044+0000 front 2024-06-10T03:05:19.415069+0000 (oldest deadline 2024-06-10T03:05:45.315110+0000) 2024-06-10T03:06:04.454 INFO:journalctl@ceph.osd.8.smithi188.stdout:Jun 10 03:06:04 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-8[55165]: 2024-06-10T03:06:04.026+0000 7fc330a8a700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:32.217415+0000 front 2024-06-10T03:05:32.217588+0000 (oldest deadline 2024-06-10T03:05:54.418216+0000) 2024-06-10T03:06:04.454 INFO:journalctl@ceph.osd.8.smithi188.stdout:Jun 10 03:06:04 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-8[55165]: 2024-06-10T03:06:04.026+0000 7fc330a8a700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-06-10T03:05:39.719178+0000 front 2024-06-10T03:05:39.719197+0000 (oldest deadline 2024-06-10T03:06:03.819222+0000) 2024-06-10T03:06:04.639 INFO:journalctl@ceph.osd.7.smithi120.stdout:Jun 10 03:06:04 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-7[54219]: 2024-06-10T03:06:04.297+0000 7f3b24ea2700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:23.717100+0000 front 2024-06-10T03:05:23.717037+0000 (oldest deadline 2024-06-10T03:05:47.817258+0000) 2024-06-10T03:06:04.639 INFO:journalctl@ceph.osd.7.smithi120.stdout:Jun 10 03:06:04 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-7[54219]: 2024-06-10T03:06:04.297+0000 7f3b24ea2700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:31.819036+0000 front 2024-06-10T03:05:31.819139+0000 (oldest deadline 2024-06-10T03:05:55.919249+0000) 2024-06-10T03:06:04.639 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:04 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:04.409+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-10T03:05:15.396577+0000 front 2024-06-10T03:05:15.396655+0000 (oldest deadline 2024-06-10T03:05:37.096690+0000) 2024-06-10T03:06:04.639 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:04 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:04.409+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:22.397847+0000 front 2024-06-10T03:05:22.397969+0000 (oldest deadline 2024-06-10T03:05:46.497941+0000) 2024-06-10T03:06:04.639 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:04 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:04.409+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:26.498562+0000 front 2024-06-10T03:05:26.498718+0000 (oldest deadline 2024-06-10T03:05:55.298987+0000) 2024-06-10T03:06:04.639 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:04 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:04.409+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-06-10T03:05:40.000119+0000 front 2024-06-10T03:05:40.000202+0000 (oldest deadline 2024-06-10T03:06:04.000669+0000) 2024-06-10T03:06:04.727 INFO:journalctl@ceph.osd.6.smithi052.stdout:Jun 10 03:06:04 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-6[60649]: 2024-06-10T03:06:04.427+0000 7f4ce3c07700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:23.897917+0000 front 2024-06-10T03:05:23.897967+0000 (oldest deadline 2024-06-10T03:05:47.998039+0000) 2024-06-10T03:06:04.727 INFO:journalctl@ceph.osd.6.smithi052.stdout:Jun 10 03:06:04 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-6[60649]: 2024-06-10T03:06:04.427+0000 7f4ce3c07700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:27.998615+0000 front 2024-06-10T03:05:27.998568+0000 (oldest deadline 2024-06-10T03:05:53.898652+0000) 2024-06-10T03:06:04.727 INFO:journalctl@ceph.osd.6.smithi052.stdout:Jun 10 03:06:04 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-6[60649]: 2024-06-10T03:06:04.427+0000 7f4ce3c07700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-06-10T03:05:37.999602+0000 front 2024-06-10T03:05:37.999631+0000 (oldest deadline 2024-06-10T03:06:03.299805+0000) 2024-06-10T03:06:04.954 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:04 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:04.708+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-10T03:05:13.354170+0000 front 2024-06-10T03:05:13.354237+0000 (oldest deadline 2024-06-10T03:05:38.053995+0000) 2024-06-10T03:06:04.954 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:04 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:04.708+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:22.755328+0000 front 2024-06-10T03:05:22.755363+0000 (oldest deadline 2024-06-10T03:05:45.655329+0000) 2024-06-10T03:06:04.954 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:04 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:04.708+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:29.156052+0000 front 2024-06-10T03:05:29.156095+0000 (oldest deadline 2024-06-10T03:05:53.856194+0000) 2024-06-10T03:06:05.454 INFO:journalctl@ceph.osd.8.smithi188.stdout:Jun 10 03:06:05 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-8[55165]: 2024-06-10T03:06:05.037+0000 7fc330a8a700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:19.415044+0000 front 2024-06-10T03:05:19.415069+0000 (oldest deadline 2024-06-10T03:05:45.315110+0000) 2024-06-10T03:06:05.454 INFO:journalctl@ceph.osd.8.smithi188.stdout:Jun 10 03:06:05 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-8[55165]: 2024-06-10T03:06:05.037+0000 7fc330a8a700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:32.217415+0000 front 2024-06-10T03:05:32.217588+0000 (oldest deadline 2024-06-10T03:05:54.418216+0000) 2024-06-10T03:06:05.454 INFO:journalctl@ceph.osd.8.smithi188.stdout:Jun 10 03:06:05 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-8[55165]: 2024-06-10T03:06:05.037+0000 7fc330a8a700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-06-10T03:05:39.719178+0000 front 2024-06-10T03:05:39.719197+0000 (oldest deadline 2024-06-10T03:06:03.819222+0000) 2024-06-10T03:06:05.477 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:06:05 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:06:05.059+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-10T03:05:12.298026+0000 front 2024-06-10T03:05:12.298086+0000 (oldest deadline 2024-06-10T03:05:37.598149+0000) 2024-06-10T03:06:05.477 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:06:05 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:06:05.059+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:23.499450+0000 front 2024-06-10T03:05:23.499380+0000 (oldest deadline 2024-06-10T03:05:47.599354+0000) 2024-06-10T03:06:05.477 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:06:05 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:06:05.059+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:27.600040+0000 front 2024-06-10T03:05:27.600064+0000 (oldest deadline 2024-06-10T03:05:53.499906+0000) 2024-06-10T03:06:05.477 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:06:05 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:06:05.059+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-06-10T03:05:38.800668+0000 front 2024-06-10T03:05:38.800723+0000 (oldest deadline 2024-06-10T03:06:02.900794+0000) 2024-06-10T03:06:05.638 INFO:journalctl@ceph.osd.7.smithi120.stdout:Jun 10 03:06:05 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-7[54219]: 2024-06-10T03:06:05.303+0000 7f3b24ea2700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:23.717100+0000 front 2024-06-10T03:05:23.717037+0000 (oldest deadline 2024-06-10T03:05:47.817258+0000) 2024-06-10T03:06:05.639 INFO:journalctl@ceph.osd.7.smithi120.stdout:Jun 10 03:06:05 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-7[54219]: 2024-06-10T03:06:05.303+0000 7f3b24ea2700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:31.819036+0000 front 2024-06-10T03:05:31.819139+0000 (oldest deadline 2024-06-10T03:05:55.919249+0000) 2024-06-10T03:06:05.639 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:05 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:05.447+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-10T03:05:15.396577+0000 front 2024-06-10T03:05:15.396655+0000 (oldest deadline 2024-06-10T03:05:37.096690+0000) 2024-06-10T03:06:05.639 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:05 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:05.447+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:22.397847+0000 front 2024-06-10T03:05:22.397969+0000 (oldest deadline 2024-06-10T03:05:46.497941+0000) 2024-06-10T03:06:05.639 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:05 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:05.447+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:26.498562+0000 front 2024-06-10T03:05:26.498718+0000 (oldest deadline 2024-06-10T03:05:55.298987+0000) 2024-06-10T03:06:05.639 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:05 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:05.447+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-06-10T03:05:40.000119+0000 front 2024-06-10T03:05:40.000202+0000 (oldest deadline 2024-06-10T03:06:04.000669+0000) 2024-06-10T03:06:05.954 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:05 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:05.699+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-10T03:05:13.354170+0000 front 2024-06-10T03:05:13.354237+0000 (oldest deadline 2024-06-10T03:05:38.053995+0000) 2024-06-10T03:06:05.954 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:05 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:05.699+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:22.755328+0000 front 2024-06-10T03:05:22.755363+0000 (oldest deadline 2024-06-10T03:05:45.655329+0000) 2024-06-10T03:06:05.954 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:05 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:05.699+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:29.156052+0000 front 2024-06-10T03:05:29.156095+0000 (oldest deadline 2024-06-10T03:05:53.856194+0000) 2024-06-10T03:06:06.159 INFO:journalctl@ceph.osd.6.smithi052.stdout:Jun 10 03:06:05 smithi052 podman[120890]: 2024-06-10 03:06:05.86172891 +0000 UTC m=+5.700568181 container died ba23282b7e3276e45679e5974ed793776afc294f02f436e8c8818ce7b1888d25 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-6, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-06-10T03:06:06.159 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:06:06 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:06:06.017+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-10T03:05:12.298026+0000 front 2024-06-10T03:05:12.298086+0000 (oldest deadline 2024-06-10T03:05:37.598149+0000) 2024-06-10T03:06:06.160 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:06:06 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:06:06.017+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:23.499450+0000 front 2024-06-10T03:05:23.499380+0000 (oldest deadline 2024-06-10T03:05:47.599354+0000) 2024-06-10T03:06:06.160 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:06:06 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:06:06.017+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:27.600040+0000 front 2024-06-10T03:05:27.600064+0000 (oldest deadline 2024-06-10T03:05:53.499906+0000) 2024-06-10T03:06:06.160 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:06:06 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:06:06.017+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-06-10T03:05:38.800668+0000 front 2024-06-10T03:05:38.800723+0000 (oldest deadline 2024-06-10T03:06:02.900794+0000) 2024-06-10T03:06:06.454 INFO:journalctl@ceph.osd.8.smithi188.stdout:Jun 10 03:06:06 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-8[55165]: 2024-06-10T03:06:06.024+0000 7fc330a8a700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:19.415044+0000 front 2024-06-10T03:05:19.415069+0000 (oldest deadline 2024-06-10T03:05:45.315110+0000) 2024-06-10T03:06:06.454 INFO:journalctl@ceph.osd.8.smithi188.stdout:Jun 10 03:06:06 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-8[55165]: 2024-06-10T03:06:06.024+0000 7fc330a8a700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:32.217415+0000 front 2024-06-10T03:05:32.217588+0000 (oldest deadline 2024-06-10T03:05:54.418216+0000) 2024-06-10T03:06:06.454 INFO:journalctl@ceph.osd.8.smithi188.stdout:Jun 10 03:06:06 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-8[55165]: 2024-06-10T03:06:06.024+0000 7fc330a8a700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-06-10T03:05:39.719178+0000 front 2024-06-10T03:05:39.719197+0000 (oldest deadline 2024-06-10T03:06:03.819222+0000) 2024-06-10T03:06:06.477 INFO:journalctl@ceph.osd.6.smithi052.stdout:Jun 10 03:06:06 smithi052 podman[120890]: 2024-06-10 03:06:06.359470474 +0000 UTC m=+6.198309729 container cleanup ba23282b7e3276e45679e5974ed793776afc294f02f436e8c8818ce7b1888d25 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-6, GIT_CLEAN=True, ceph=True, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-06-10T03:06:06.477 INFO:journalctl@ceph.osd.6.smithi052.stdout:Jun 10 03:06:06 smithi052 bash[120890]: ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-6 2024-06-10T03:06:06.639 INFO:journalctl@ceph.osd.7.smithi120.stdout:Jun 10 03:06:06 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-7[54219]: 2024-06-10T03:06:06.309+0000 7f3b24ea2700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:23.717100+0000 front 2024-06-10T03:05:23.717037+0000 (oldest deadline 2024-06-10T03:05:47.817258+0000) 2024-06-10T03:06:06.639 INFO:journalctl@ceph.osd.7.smithi120.stdout:Jun 10 03:06:06 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-7[54219]: 2024-06-10T03:06:06.309+0000 7f3b24ea2700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:31.819036+0000 front 2024-06-10T03:05:31.819139+0000 (oldest deadline 2024-06-10T03:05:55.919249+0000) 2024-06-10T03:06:06.639 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:06 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:06.461+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-10T03:05:15.396577+0000 front 2024-06-10T03:05:15.396655+0000 (oldest deadline 2024-06-10T03:05:37.096690+0000) 2024-06-10T03:06:06.639 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:06 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:06.461+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:22.397847+0000 front 2024-06-10T03:05:22.397969+0000 (oldest deadline 2024-06-10T03:05:46.497941+0000) 2024-06-10T03:06:06.639 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:06 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:06.461+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:26.498562+0000 front 2024-06-10T03:05:26.498718+0000 (oldest deadline 2024-06-10T03:05:55.298987+0000) 2024-06-10T03:06:06.639 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:06 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:06.461+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-06-10T03:05:40.000119+0000 front 2024-06-10T03:05:40.000202+0000 (oldest deadline 2024-06-10T03:06:04.000669+0000) 2024-06-10T03:06:06.945 INFO:journalctl@ceph.osd.6.smithi052.stdout:Jun 10 03:06:06 smithi052 podman[120907]: 2024-06-10 03:06:06.602567926 +0000 UTC m=+0.738553319 container remove ba23282b7e3276e45679e5974ed793776afc294f02f436e8c8818ce7b1888d25 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-6, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-06-10T03:06:07.052 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:06 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:06.729+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-10T03:05:13.354170+0000 front 2024-06-10T03:05:13.354237+0000 (oldest deadline 2024-06-10T03:05:38.053995+0000) 2024-06-10T03:06:07.052 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:06 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:06.729+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:22.755328+0000 front 2024-06-10T03:05:22.755363+0000 (oldest deadline 2024-06-10T03:05:45.655329+0000) 2024-06-10T03:06:07.052 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:06 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:06.729+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:29.156052+0000 front 2024-06-10T03:05:29.156095+0000 (oldest deadline 2024-06-10T03:05:53.856194+0000) 2024-06-10T03:06:07.206 INFO:journalctl@ceph.osd.6.smithi052.stdout:Jun 10 03:06:06 smithi052 podman[120995]: 2024-06-10 03:06:06.945237764 +0000 UTC m=+0.111776099 container create 3298f7ae237810e645fcc84a1442b0d2543765c0026152fbaee1be964dc28de0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-6-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc) 2024-06-10T03:06:07.206 INFO:journalctl@ceph.osd.6.smithi052.stdout:Jun 10 03:06:06 smithi052 podman[120995]: 2024-06-10 03:06:06.851162095 +0000 UTC m=+0.017700413 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-10T03:06:07.206 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:06:06 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:06:06.999+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-10T03:05:12.298026+0000 front 2024-06-10T03:05:12.298086+0000 (oldest deadline 2024-06-10T03:05:37.598149+0000) 2024-06-10T03:06:07.206 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:06:06 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:06:06.999+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:23.499450+0000 front 2024-06-10T03:05:23.499380+0000 (oldest deadline 2024-06-10T03:05:47.599354+0000) 2024-06-10T03:06:07.206 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:06:06 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:06:06.999+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:27.600040+0000 front 2024-06-10T03:05:27.600064+0000 (oldest deadline 2024-06-10T03:05:53.499906+0000) 2024-06-10T03:06:07.206 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:06:06 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:06:06.999+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-06-10T03:05:38.800668+0000 front 2024-06-10T03:05:38.800723+0000 (oldest deadline 2024-06-10T03:06:02.900794+0000) 2024-06-10T03:06:07.454 INFO:journalctl@ceph.osd.8.smithi188.stdout:Jun 10 03:06:07 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-8[55165]: 2024-06-10T03:06:07.051+0000 7fc330a8a700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:19.415044+0000 front 2024-06-10T03:05:19.415069+0000 (oldest deadline 2024-06-10T03:05:45.315110+0000) 2024-06-10T03:06:07.455 INFO:journalctl@ceph.osd.8.smithi188.stdout:Jun 10 03:06:07 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-8[55165]: 2024-06-10T03:06:07.051+0000 7fc330a8a700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:32.217415+0000 front 2024-06-10T03:05:32.217588+0000 (oldest deadline 2024-06-10T03:05:54.418216+0000) 2024-06-10T03:06:07.455 INFO:journalctl@ceph.osd.8.smithi188.stdout:Jun 10 03:06:07 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-8[55165]: 2024-06-10T03:06:07.051+0000 7fc330a8a700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-06-10T03:05:39.719178+0000 front 2024-06-10T03:05:39.719197+0000 (oldest deadline 2024-06-10T03:06:03.819222+0000) 2024-06-10T03:06:07.477 INFO:journalctl@ceph.osd.6.smithi052.stdout:Jun 10 03:06:07 smithi052 podman[120995]: 2024-06-10 03:06:07.20639085 +0000 UTC m=+0.372929153 container init 3298f7ae237810e645fcc84a1442b0d2543765c0026152fbaee1be964dc28de0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-6-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux ) 2024-06-10T03:06:07.477 INFO:journalctl@ceph.osd.6.smithi052.stdout:Jun 10 03:06:07 smithi052 podman[120995]: 2024-06-10 03:06:07.210353013 +0000 UTC m=+0.376891312 container start 3298f7ae237810e645fcc84a1442b0d2543765c0026152fbaee1be964dc28de0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-6-deactivate, GIT_BRANCH=HEAD, ceph=True, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-10T03:06:07.477 INFO:journalctl@ceph.osd.6.smithi052.stdout:Jun 10 03:06:07 smithi052 podman[120995]: 2024-06-10 03:06:07.306591334 +0000 UTC m=+0.473129639 container attach 3298f7ae237810e645fcc84a1442b0d2543765c0026152fbaee1be964dc28de0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-6-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-10T03:06:07.638 INFO:journalctl@ceph.osd.7.smithi120.stdout:Jun 10 03:06:07 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-7[54219]: 2024-06-10T03:06:07.312+0000 7f3b24ea2700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:23.717100+0000 front 2024-06-10T03:05:23.717037+0000 (oldest deadline 2024-06-10T03:05:47.817258+0000) 2024-06-10T03:06:07.639 INFO:journalctl@ceph.osd.7.smithi120.stdout:Jun 10 03:06:07 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-7[54219]: 2024-06-10T03:06:07.312+0000 7f3b24ea2700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:31.819036+0000 front 2024-06-10T03:05:31.819139+0000 (oldest deadline 2024-06-10T03:05:55.919249+0000) 2024-06-10T03:06:07.639 INFO:journalctl@ceph.osd.7.smithi120.stdout:Jun 10 03:06:07 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-7[54219]: 2024-06-10T03:06:07.312+0000 7f3b24ea2700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-06-10T03:05:40.521059+0000 front 2024-06-10T03:05:40.521090+0000 (oldest deadline 2024-06-10T03:06:06.921348+0000) 2024-06-10T03:06:07.639 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:07 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:07.458+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-10T03:05:15.396577+0000 front 2024-06-10T03:05:15.396655+0000 (oldest deadline 2024-06-10T03:05:37.096690+0000) 2024-06-10T03:06:07.639 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:07 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:07.458+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:22.397847+0000 front 2024-06-10T03:05:22.397969+0000 (oldest deadline 2024-06-10T03:05:46.497941+0000) 2024-06-10T03:06:07.639 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:07 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:07.458+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:26.498562+0000 front 2024-06-10T03:05:26.498718+0000 (oldest deadline 2024-06-10T03:05:55.298987+0000) 2024-06-10T03:06:07.639 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:07 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:07.458+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-06-10T03:05:40.000119+0000 front 2024-06-10T03:05:40.000202+0000 (oldest deadline 2024-06-10T03:06:04.000669+0000) 2024-06-10T03:06:07.801 INFO:journalctl@ceph.osd.6.smithi052.stdout:Jun 10 03:06:07 smithi052 podman[120995]: 2024-06-10 03:06:07.516728454 +0000 UTC m=+0.683266766 container died 3298f7ae237810e645fcc84a1442b0d2543765c0026152fbaee1be964dc28de0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-6-deactivate, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-06-10T03:06:08.040 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:07 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:07.725+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-10T03:05:13.354170+0000 front 2024-06-10T03:05:13.354237+0000 (oldest deadline 2024-06-10T03:05:38.053995+0000) 2024-06-10T03:06:08.040 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:07 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:07.725+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:22.755328+0000 front 2024-06-10T03:05:22.755363+0000 (oldest deadline 2024-06-10T03:05:45.655329+0000) 2024-06-10T03:06:08.040 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:07 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:07.725+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:29.156052+0000 front 2024-06-10T03:05:29.156095+0000 (oldest deadline 2024-06-10T03:05:53.856194+0000) 2024-06-10T03:06:08.110 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:06:07 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:06:07.969+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-10T03:05:12.298026+0000 front 2024-06-10T03:05:12.298086+0000 (oldest deadline 2024-06-10T03:05:37.598149+0000) 2024-06-10T03:06:08.110 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:06:07 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:06:07.969+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:23.499450+0000 front 2024-06-10T03:05:23.499380+0000 (oldest deadline 2024-06-10T03:05:47.599354+0000) 2024-06-10T03:06:08.110 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:06:07 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:06:07.969+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:27.600040+0000 front 2024-06-10T03:05:27.600064+0000 (oldest deadline 2024-06-10T03:05:53.499906+0000) 2024-06-10T03:06:08.110 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:06:07 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:06:07.969+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-06-10T03:05:38.800668+0000 front 2024-06-10T03:05:38.800723+0000 (oldest deadline 2024-06-10T03:06:02.900794+0000) 2024-06-10T03:06:08.132 DEBUG:teuthology.orchestra.run.smithi052:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968@osd.6.service' 2024-06-10T03:06:08.177 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-10T03:06:08.177 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-06-10T03:06:08.177 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-06-10T03:06:08.177 DEBUG:teuthology.orchestra.run.smithi120:> sudo systemctl stop ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968@osd.7 2024-06-10T03:06:08.454 INFO:journalctl@ceph.osd.8.smithi188.stdout:Jun 10 03:06:08 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-8[55165]: 2024-06-10T03:06:08.040+0000 7fc330a8a700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:19.415044+0000 front 2024-06-10T03:05:19.415069+0000 (oldest deadline 2024-06-10T03:05:45.315110+0000) 2024-06-10T03:06:08.454 INFO:journalctl@ceph.osd.8.smithi188.stdout:Jun 10 03:06:08 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-8[55165]: 2024-06-10T03:06:08.040+0000 7fc330a8a700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:32.217415+0000 front 2024-06-10T03:05:32.217588+0000 (oldest deadline 2024-06-10T03:05:54.418216+0000) 2024-06-10T03:06:08.454 INFO:journalctl@ceph.osd.8.smithi188.stdout:Jun 10 03:06:08 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-8[55165]: 2024-06-10T03:06:08.040+0000 7fc330a8a700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-06-10T03:05:39.719178+0000 front 2024-06-10T03:05:39.719197+0000 (oldest deadline 2024-06-10T03:06:03.819222+0000) 2024-06-10T03:06:08.490 INFO:journalctl@ceph.osd.7.smithi120.stdout:Jun 10 03:06:08 smithi120 systemd[1]: Stopping Ceph osd.7 for d8676e9e-26d2-11ef-bc9b-c7b262605968... 2024-06-10T03:06:08.491 INFO:journalctl@ceph.osd.7.smithi120.stdout:Jun 10 03:06:08 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-7[54219]: 2024-06-10T03:06:08.269+0000 7f3b24ea2700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:23.717100+0000 front 2024-06-10T03:05:23.717037+0000 (oldest deadline 2024-06-10T03:05:47.817258+0000) 2024-06-10T03:06:08.491 INFO:journalctl@ceph.osd.7.smithi120.stdout:Jun 10 03:06:08 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-7[54219]: 2024-06-10T03:06:08.269+0000 7f3b24ea2700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:31.819036+0000 front 2024-06-10T03:05:31.819139+0000 (oldest deadline 2024-06-10T03:05:55.919249+0000) 2024-06-10T03:06:08.491 INFO:journalctl@ceph.osd.7.smithi120.stdout:Jun 10 03:06:08 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-7[54219]: 2024-06-10T03:06:08.269+0000 7f3b24ea2700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-06-10T03:05:40.521059+0000 front 2024-06-10T03:05:40.521090+0000 (oldest deadline 2024-06-10T03:06:06.921348+0000) 2024-06-10T03:06:08.889 INFO:journalctl@ceph.osd.7.smithi120.stdout:Jun 10 03:06:08 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-7[54219]: 2024-06-10T03:06:08.500+0000 7f3b288ae700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-10T03:06:08.889 INFO:journalctl@ceph.osd.7.smithi120.stdout:Jun 10 03:06:08 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-7[54219]: 2024-06-10T03:06:08.500+0000 7f3b288ae700 -1 osd.7 104 *** Got signal Terminated *** 2024-06-10T03:06:08.889 INFO:journalctl@ceph.osd.7.smithi120.stdout:Jun 10 03:06:08 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-7[54219]: 2024-06-10T03:06:08.500+0000 7f3b288ae700 -1 osd.7 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-10T03:06:08.889 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:08 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:08.489+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-10T03:05:15.396577+0000 front 2024-06-10T03:05:15.396655+0000 (oldest deadline 2024-06-10T03:05:37.096690+0000) 2024-06-10T03:06:08.889 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:08 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:08.489+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:22.397847+0000 front 2024-06-10T03:05:22.397969+0000 (oldest deadline 2024-06-10T03:05:46.497941+0000) 2024-06-10T03:06:08.889 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:08 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:08.489+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:26.498562+0000 front 2024-06-10T03:05:26.498718+0000 (oldest deadline 2024-06-10T03:05:55.298987+0000) 2024-06-10T03:06:08.889 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:08 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:08.489+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-06-10T03:05:40.000119+0000 front 2024-06-10T03:05:40.000202+0000 (oldest deadline 2024-06-10T03:06:04.000669+0000) 2024-06-10T03:06:09.070 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:08 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:08.767+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-10T03:05:13.354170+0000 front 2024-06-10T03:05:13.354237+0000 (oldest deadline 2024-06-10T03:05:38.053995+0000) 2024-06-10T03:06:09.070 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:08 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:08.767+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:22.755328+0000 front 2024-06-10T03:05:22.755363+0000 (oldest deadline 2024-06-10T03:05:45.655329+0000) 2024-06-10T03:06:09.070 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:08 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:08.767+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:29.156052+0000 front 2024-06-10T03:05:29.156095+0000 (oldest deadline 2024-06-10T03:05:53.856194+0000) 2024-06-10T03:06:09.227 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:06:08 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:06:08.930+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-10T03:05:12.298026+0000 front 2024-06-10T03:05:12.298086+0000 (oldest deadline 2024-06-10T03:05:37.598149+0000) 2024-06-10T03:06:09.227 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:06:08 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:06:08.930+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:23.499450+0000 front 2024-06-10T03:05:23.499380+0000 (oldest deadline 2024-06-10T03:05:47.599354+0000) 2024-06-10T03:06:09.227 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:06:08 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:06:08.930+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:27.600040+0000 front 2024-06-10T03:05:27.600064+0000 (oldest deadline 2024-06-10T03:05:53.499906+0000) 2024-06-10T03:06:09.227 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:06:08 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:06:08.930+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-06-10T03:05:38.800668+0000 front 2024-06-10T03:05:38.800723+0000 (oldest deadline 2024-06-10T03:06:02.900794+0000) 2024-06-10T03:06:09.454 INFO:journalctl@ceph.osd.8.smithi188.stdout:Jun 10 03:06:09 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-8[55165]: 2024-06-10T03:06:09.070+0000 7fc330a8a700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:19.415044+0000 front 2024-06-10T03:05:19.415069+0000 (oldest deadline 2024-06-10T03:05:45.315110+0000) 2024-06-10T03:06:09.454 INFO:journalctl@ceph.osd.8.smithi188.stdout:Jun 10 03:06:09 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-8[55165]: 2024-06-10T03:06:09.070+0000 7fc330a8a700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:32.217415+0000 front 2024-06-10T03:05:32.217588+0000 (oldest deadline 2024-06-10T03:05:54.418216+0000) 2024-06-10T03:06:09.454 INFO:journalctl@ceph.osd.8.smithi188.stdout:Jun 10 03:06:09 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-8[55165]: 2024-06-10T03:06:09.070+0000 7fc330a8a700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-06-10T03:05:39.719178+0000 front 2024-06-10T03:05:39.719197+0000 (oldest deadline 2024-06-10T03:06:03.819222+0000) 2024-06-10T03:06:09.638 INFO:journalctl@ceph.osd.7.smithi120.stdout:Jun 10 03:06:09 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-7[54219]: 2024-06-10T03:06:09.309+0000 7f3b24ea2700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:23.717100+0000 front 2024-06-10T03:05:23.717037+0000 (oldest deadline 2024-06-10T03:05:47.817258+0000) 2024-06-10T03:06:09.639 INFO:journalctl@ceph.osd.7.smithi120.stdout:Jun 10 03:06:09 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-7[54219]: 2024-06-10T03:06:09.309+0000 7f3b24ea2700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:31.819036+0000 front 2024-06-10T03:05:31.819139+0000 (oldest deadline 2024-06-10T03:05:55.919249+0000) 2024-06-10T03:06:09.639 INFO:journalctl@ceph.osd.7.smithi120.stdout:Jun 10 03:06:09 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-7[54219]: 2024-06-10T03:06:09.309+0000 7f3b24ea2700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-06-10T03:05:40.521059+0000 front 2024-06-10T03:05:40.521090+0000 (oldest deadline 2024-06-10T03:06:06.921348+0000) 2024-06-10T03:06:09.639 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:09 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:09.482+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-10T03:05:15.396577+0000 front 2024-06-10T03:05:15.396655+0000 (oldest deadline 2024-06-10T03:05:37.096690+0000) 2024-06-10T03:06:09.639 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:09 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:09.482+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:22.397847+0000 front 2024-06-10T03:05:22.397969+0000 (oldest deadline 2024-06-10T03:05:46.497941+0000) 2024-06-10T03:06:09.639 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:09 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:09.482+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:26.498562+0000 front 2024-06-10T03:05:26.498718+0000 (oldest deadline 2024-06-10T03:05:55.298987+0000) 2024-06-10T03:06:09.639 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:09 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:09.482+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-06-10T03:05:40.000119+0000 front 2024-06-10T03:05:40.000202+0000 (oldest deadline 2024-06-10T03:06:04.000669+0000) 2024-06-10T03:06:10.047 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:09 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:09.756+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-10T03:05:13.354170+0000 front 2024-06-10T03:05:13.354237+0000 (oldest deadline 2024-06-10T03:05:38.053995+0000) 2024-06-10T03:06:10.047 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:09 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:09.756+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:22.755328+0000 front 2024-06-10T03:05:22.755363+0000 (oldest deadline 2024-06-10T03:05:45.655329+0000) 2024-06-10T03:06:10.047 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:09 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:09.756+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:29.156052+0000 front 2024-06-10T03:05:29.156095+0000 (oldest deadline 2024-06-10T03:05:53.856194+0000) 2024-06-10T03:06:10.227 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:06:09 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:06:09.934+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-10T03:05:12.298026+0000 front 2024-06-10T03:05:12.298086+0000 (oldest deadline 2024-06-10T03:05:37.598149+0000) 2024-06-10T03:06:10.227 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:06:09 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:06:09.934+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:23.499450+0000 front 2024-06-10T03:05:23.499380+0000 (oldest deadline 2024-06-10T03:05:47.599354+0000) 2024-06-10T03:06:10.227 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:06:09 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:06:09.934+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:27.600040+0000 front 2024-06-10T03:05:27.600064+0000 (oldest deadline 2024-06-10T03:05:53.499906+0000) 2024-06-10T03:06:10.228 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:06:09 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:06:09.934+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-06-10T03:05:38.800668+0000 front 2024-06-10T03:05:38.800723+0000 (oldest deadline 2024-06-10T03:06:02.900794+0000) 2024-06-10T03:06:10.454 INFO:journalctl@ceph.osd.8.smithi188.stdout:Jun 10 03:06:10 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-8[55165]: 2024-06-10T03:06:10.047+0000 7fc330a8a700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:19.415044+0000 front 2024-06-10T03:05:19.415069+0000 (oldest deadline 2024-06-10T03:05:45.315110+0000) 2024-06-10T03:06:10.454 INFO:journalctl@ceph.osd.8.smithi188.stdout:Jun 10 03:06:10 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-8[55165]: 2024-06-10T03:06:10.047+0000 7fc330a8a700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:32.217415+0000 front 2024-06-10T03:05:32.217588+0000 (oldest deadline 2024-06-10T03:05:54.418216+0000) 2024-06-10T03:06:10.454 INFO:journalctl@ceph.osd.8.smithi188.stdout:Jun 10 03:06:10 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-8[55165]: 2024-06-10T03:06:10.047+0000 7fc330a8a700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-06-10T03:05:39.719178+0000 front 2024-06-10T03:05:39.719197+0000 (oldest deadline 2024-06-10T03:06:03.819222+0000) 2024-06-10T03:06:10.638 INFO:journalctl@ceph.osd.7.smithi120.stdout:Jun 10 03:06:10 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-7[54219]: 2024-06-10T03:06:10.353+0000 7f3b24ea2700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:23.717100+0000 front 2024-06-10T03:05:23.717037+0000 (oldest deadline 2024-06-10T03:05:47.817258+0000) 2024-06-10T03:06:10.639 INFO:journalctl@ceph.osd.7.smithi120.stdout:Jun 10 03:06:10 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-7[54219]: 2024-06-10T03:06:10.353+0000 7f3b24ea2700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:31.819036+0000 front 2024-06-10T03:05:31.819139+0000 (oldest deadline 2024-06-10T03:05:55.919249+0000) 2024-06-10T03:06:10.639 INFO:journalctl@ceph.osd.7.smithi120.stdout:Jun 10 03:06:10 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-7[54219]: 2024-06-10T03:06:10.353+0000 7f3b24ea2700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-06-10T03:05:40.521059+0000 front 2024-06-10T03:05:40.521090+0000 (oldest deadline 2024-06-10T03:06:06.921348+0000) 2024-06-10T03:06:10.639 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:10 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:10.472+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-10T03:05:15.396577+0000 front 2024-06-10T03:05:15.396655+0000 (oldest deadline 2024-06-10T03:05:37.096690+0000) 2024-06-10T03:06:10.639 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:10 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:10.472+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:22.397847+0000 front 2024-06-10T03:05:22.397969+0000 (oldest deadline 2024-06-10T03:05:46.497941+0000) 2024-06-10T03:06:10.639 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:10 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:10.472+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:26.498562+0000 front 2024-06-10T03:05:26.498718+0000 (oldest deadline 2024-06-10T03:05:55.298987+0000) 2024-06-10T03:06:10.639 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:10 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:10.472+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-06-10T03:05:40.000119+0000 front 2024-06-10T03:05:40.000202+0000 (oldest deadline 2024-06-10T03:06:04.000669+0000) 2024-06-10T03:06:10.639 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:10 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:10.472+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-10T03:05:48.101395+0000 front 2024-06-10T03:05:48.101445+0000 (oldest deadline 2024-06-10T03:06:09.801515+0000) 2024-06-10T03:06:11.037 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:10 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:10.763+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-10T03:05:13.354170+0000 front 2024-06-10T03:05:13.354237+0000 (oldest deadline 2024-06-10T03:05:38.053995+0000) 2024-06-10T03:06:11.037 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:10 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:10.763+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:22.755328+0000 front 2024-06-10T03:05:22.755363+0000 (oldest deadline 2024-06-10T03:05:45.655329+0000) 2024-06-10T03:06:11.037 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:10 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:10.763+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:29.156052+0000 front 2024-06-10T03:05:29.156095+0000 (oldest deadline 2024-06-10T03:05:53.856194+0000) 2024-06-10T03:06:11.454 INFO:journalctl@ceph.osd.8.smithi188.stdout:Jun 10 03:06:11 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-8[55165]: 2024-06-10T03:06:11.037+0000 7fc330a8a700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:19.415044+0000 front 2024-06-10T03:05:19.415069+0000 (oldest deadline 2024-06-10T03:05:45.315110+0000) 2024-06-10T03:06:11.454 INFO:journalctl@ceph.osd.8.smithi188.stdout:Jun 10 03:06:11 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-8[55165]: 2024-06-10T03:06:11.037+0000 7fc330a8a700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:32.217415+0000 front 2024-06-10T03:05:32.217588+0000 (oldest deadline 2024-06-10T03:05:54.418216+0000) 2024-06-10T03:06:11.454 INFO:journalctl@ceph.osd.8.smithi188.stdout:Jun 10 03:06:11 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-8[55165]: 2024-06-10T03:06:11.037+0000 7fc330a8a700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-06-10T03:05:39.719178+0000 front 2024-06-10T03:05:39.719197+0000 (oldest deadline 2024-06-10T03:06:03.819222+0000) 2024-06-10T03:06:11.477 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:06:10 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:06:10.982+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-10T03:05:12.298026+0000 front 2024-06-10T03:05:12.298086+0000 (oldest deadline 2024-06-10T03:05:37.598149+0000) 2024-06-10T03:06:11.477 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:06:10 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:06:10.982+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:23.499450+0000 front 2024-06-10T03:05:23.499380+0000 (oldest deadline 2024-06-10T03:05:47.599354+0000) 2024-06-10T03:06:11.477 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:06:10 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:06:10.982+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:27.600040+0000 front 2024-06-10T03:05:27.600064+0000 (oldest deadline 2024-06-10T03:05:53.499906+0000) 2024-06-10T03:06:11.478 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:06:10 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:06:10.982+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-06-10T03:05:38.800668+0000 front 2024-06-10T03:05:38.800723+0000 (oldest deadline 2024-06-10T03:06:02.900794+0000) 2024-06-10T03:06:11.639 INFO:journalctl@ceph.osd.7.smithi120.stdout:Jun 10 03:06:11 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-7[54219]: 2024-06-10T03:06:11.313+0000 7f3b24ea2700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:23.717100+0000 front 2024-06-10T03:05:23.717037+0000 (oldest deadline 2024-06-10T03:05:47.817258+0000) 2024-06-10T03:06:11.639 INFO:journalctl@ceph.osd.7.smithi120.stdout:Jun 10 03:06:11 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-7[54219]: 2024-06-10T03:06:11.313+0000 7f3b24ea2700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:31.819036+0000 front 2024-06-10T03:05:31.819139+0000 (oldest deadline 2024-06-10T03:05:55.919249+0000) 2024-06-10T03:06:11.639 INFO:journalctl@ceph.osd.7.smithi120.stdout:Jun 10 03:06:11 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-7[54219]: 2024-06-10T03:06:11.313+0000 7f3b24ea2700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-06-10T03:05:40.521059+0000 front 2024-06-10T03:05:40.521090+0000 (oldest deadline 2024-06-10T03:06:06.921348+0000) 2024-06-10T03:06:11.639 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:11 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:11.501+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-10T03:05:15.396577+0000 front 2024-06-10T03:05:15.396655+0000 (oldest deadline 2024-06-10T03:05:37.096690+0000) 2024-06-10T03:06:11.639 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:11 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:11.501+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:22.397847+0000 front 2024-06-10T03:05:22.397969+0000 (oldest deadline 2024-06-10T03:05:46.497941+0000) 2024-06-10T03:06:11.639 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:11 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:11.501+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:26.498562+0000 front 2024-06-10T03:05:26.498718+0000 (oldest deadline 2024-06-10T03:05:55.298987+0000) 2024-06-10T03:06:11.639 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:11 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:11.501+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-06-10T03:05:40.000119+0000 front 2024-06-10T03:05:40.000202+0000 (oldest deadline 2024-06-10T03:06:04.000669+0000) 2024-06-10T03:06:11.639 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:11 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:11.501+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-10T03:05:48.101395+0000 front 2024-06-10T03:05:48.101445+0000 (oldest deadline 2024-06-10T03:06:09.801515+0000) 2024-06-10T03:06:12.055 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:11 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:11.722+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-10T03:05:13.354170+0000 front 2024-06-10T03:05:13.354237+0000 (oldest deadline 2024-06-10T03:05:38.053995+0000) 2024-06-10T03:06:12.055 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:11 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:11.722+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:22.755328+0000 front 2024-06-10T03:05:22.755363+0000 (oldest deadline 2024-06-10T03:05:45.655329+0000) 2024-06-10T03:06:12.055 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:11 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:11.722+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:29.156052+0000 front 2024-06-10T03:05:29.156095+0000 (oldest deadline 2024-06-10T03:05:53.856194+0000) 2024-06-10T03:06:12.454 INFO:journalctl@ceph.osd.8.smithi188.stdout:Jun 10 03:06:12 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-8[55165]: 2024-06-10T03:06:12.055+0000 7fc330a8a700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:19.415044+0000 front 2024-06-10T03:05:19.415069+0000 (oldest deadline 2024-06-10T03:05:45.315110+0000) 2024-06-10T03:06:12.454 INFO:journalctl@ceph.osd.8.smithi188.stdout:Jun 10 03:06:12 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-8[55165]: 2024-06-10T03:06:12.055+0000 7fc330a8a700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:32.217415+0000 front 2024-06-10T03:05:32.217588+0000 (oldest deadline 2024-06-10T03:05:54.418216+0000) 2024-06-10T03:06:12.454 INFO:journalctl@ceph.osd.8.smithi188.stdout:Jun 10 03:06:12 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-8[55165]: 2024-06-10T03:06:12.055+0000 7fc330a8a700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-06-10T03:05:39.719178+0000 front 2024-06-10T03:05:39.719197+0000 (oldest deadline 2024-06-10T03:06:03.819222+0000) 2024-06-10T03:06:12.477 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:06:12 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:06:12.008+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-10T03:05:12.298026+0000 front 2024-06-10T03:05:12.298086+0000 (oldest deadline 2024-06-10T03:05:37.598149+0000) 2024-06-10T03:06:12.477 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:06:12 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:06:12.008+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:23.499450+0000 front 2024-06-10T03:05:23.499380+0000 (oldest deadline 2024-06-10T03:05:47.599354+0000) 2024-06-10T03:06:12.477 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:06:12 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:06:12.008+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:27.600040+0000 front 2024-06-10T03:05:27.600064+0000 (oldest deadline 2024-06-10T03:05:53.499906+0000) 2024-06-10T03:06:12.477 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:06:12 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:06:12.008+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-06-10T03:05:38.800668+0000 front 2024-06-10T03:05:38.800723+0000 (oldest deadline 2024-06-10T03:06:02.900794+0000) 2024-06-10T03:06:12.477 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:06:12 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:06:12.008+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-10T03:05:48.101972+0000 front 2024-06-10T03:05:48.102107+0000 (oldest deadline 2024-06-10T03:06:11.002037+0000) 2024-06-10T03:06:12.639 INFO:journalctl@ceph.osd.7.smithi120.stdout:Jun 10 03:06:12 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-7[54219]: 2024-06-10T03:06:12.347+0000 7f3b24ea2700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:23.717100+0000 front 2024-06-10T03:05:23.717037+0000 (oldest deadline 2024-06-10T03:05:47.817258+0000) 2024-06-10T03:06:12.639 INFO:journalctl@ceph.osd.7.smithi120.stdout:Jun 10 03:06:12 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-7[54219]: 2024-06-10T03:06:12.347+0000 7f3b24ea2700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:31.819036+0000 front 2024-06-10T03:05:31.819139+0000 (oldest deadline 2024-06-10T03:05:55.919249+0000) 2024-06-10T03:06:12.639 INFO:journalctl@ceph.osd.7.smithi120.stdout:Jun 10 03:06:12 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-7[54219]: 2024-06-10T03:06:12.347+0000 7f3b24ea2700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-06-10T03:05:40.521059+0000 front 2024-06-10T03:05:40.521090+0000 (oldest deadline 2024-06-10T03:06:06.921348+0000) 2024-06-10T03:06:12.639 INFO:journalctl@ceph.osd.7.smithi120.stdout:Jun 10 03:06:12 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-7[54219]: 2024-06-10T03:06:12.347+0000 7f3b24ea2700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-10T03:05:46.921841+0000 front 2024-06-10T03:05:46.921890+0000 (oldest deadline 2024-06-10T03:06:11.621997+0000) 2024-06-10T03:06:12.639 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:12 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:12.478+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-10T03:05:15.396577+0000 front 2024-06-10T03:05:15.396655+0000 (oldest deadline 2024-06-10T03:05:37.096690+0000) 2024-06-10T03:06:12.639 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:12 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:12.478+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:22.397847+0000 front 2024-06-10T03:05:22.397969+0000 (oldest deadline 2024-06-10T03:05:46.497941+0000) 2024-06-10T03:06:12.639 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:12 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:12.478+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:26.498562+0000 front 2024-06-10T03:05:26.498718+0000 (oldest deadline 2024-06-10T03:05:55.298987+0000) 2024-06-10T03:06:12.640 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:12 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:12.478+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-06-10T03:05:40.000119+0000 front 2024-06-10T03:05:40.000202+0000 (oldest deadline 2024-06-10T03:06:04.000669+0000) 2024-06-10T03:06:12.640 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:12 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:12.478+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-10T03:05:48.101395+0000 front 2024-06-10T03:05:48.101445+0000 (oldest deadline 2024-06-10T03:06:09.801515+0000) 2024-06-10T03:06:13.021 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:12 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:12.745+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-10T03:05:13.354170+0000 front 2024-06-10T03:05:13.354237+0000 (oldest deadline 2024-06-10T03:05:38.053995+0000) 2024-06-10T03:06:13.021 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:12 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:12.745+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:22.755328+0000 front 2024-06-10T03:05:22.755363+0000 (oldest deadline 2024-06-10T03:05:45.655329+0000) 2024-06-10T03:06:13.022 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:12 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:12.745+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:29.156052+0000 front 2024-06-10T03:05:29.156095+0000 (oldest deadline 2024-06-10T03:05:53.856194+0000) 2024-06-10T03:06:13.022 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:12 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:12.745+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-10T03:05:46.158301+0000 front 2024-06-10T03:05:46.158340+0000 (oldest deadline 2024-06-10T03:06:12.058331+0000) 2024-06-10T03:06:13.454 INFO:journalctl@ceph.osd.8.smithi188.stdout:Jun 10 03:06:13 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-8[55165]: 2024-06-10T03:06:13.021+0000 7fc330a8a700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:19.415044+0000 front 2024-06-10T03:05:19.415069+0000 (oldest deadline 2024-06-10T03:05:45.315110+0000) 2024-06-10T03:06:13.454 INFO:journalctl@ceph.osd.8.smithi188.stdout:Jun 10 03:06:13 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-8[55165]: 2024-06-10T03:06:13.021+0000 7fc330a8a700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:32.217415+0000 front 2024-06-10T03:05:32.217588+0000 (oldest deadline 2024-06-10T03:05:54.418216+0000) 2024-06-10T03:06:13.454 INFO:journalctl@ceph.osd.8.smithi188.stdout:Jun 10 03:06:13 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-8[55165]: 2024-06-10T03:06:13.021+0000 7fc330a8a700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-06-10T03:05:39.719178+0000 front 2024-06-10T03:05:39.719197+0000 (oldest deadline 2024-06-10T03:06:03.819222+0000) 2024-06-10T03:06:13.454 INFO:journalctl@ceph.osd.8.smithi188.stdout:Jun 10 03:06:13 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-8[55165]: 2024-06-10T03:06:13.021+0000 7fc330a8a700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-10T03:05:43.819709+0000 front 2024-06-10T03:05:43.819808+0000 (oldest deadline 2024-06-10T03:06:12.620398+0000) 2024-06-10T03:06:13.477 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:06:12 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:06:12.996+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-10T03:05:12.298026+0000 front 2024-06-10T03:05:12.298086+0000 (oldest deadline 2024-06-10T03:05:37.598149+0000) 2024-06-10T03:06:13.477 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:06:12 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:06:12.996+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:23.499450+0000 front 2024-06-10T03:05:23.499380+0000 (oldest deadline 2024-06-10T03:05:47.599354+0000) 2024-06-10T03:06:13.477 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:06:12 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:06:12.996+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:27.600040+0000 front 2024-06-10T03:05:27.600064+0000 (oldest deadline 2024-06-10T03:05:53.499906+0000) 2024-06-10T03:06:13.477 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:06:12 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:06:12.996+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-06-10T03:05:38.800668+0000 front 2024-06-10T03:05:38.800723+0000 (oldest deadline 2024-06-10T03:06:02.900794+0000) 2024-06-10T03:06:13.477 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:06:12 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:06:12.996+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-10T03:05:48.101972+0000 front 2024-06-10T03:05:48.102107+0000 (oldest deadline 2024-06-10T03:06:11.002037+0000) 2024-06-10T03:06:13.639 INFO:journalctl@ceph.osd.7.smithi120.stdout:Jun 10 03:06:13 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-7[54219]: 2024-06-10T03:06:13.310+0000 7f3b24ea2700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:23.717100+0000 front 2024-06-10T03:05:23.717037+0000 (oldest deadline 2024-06-10T03:05:47.817258+0000) 2024-06-10T03:06:13.639 INFO:journalctl@ceph.osd.7.smithi120.stdout:Jun 10 03:06:13 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-7[54219]: 2024-06-10T03:06:13.310+0000 7f3b24ea2700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:31.819036+0000 front 2024-06-10T03:05:31.819139+0000 (oldest deadline 2024-06-10T03:05:55.919249+0000) 2024-06-10T03:06:13.639 INFO:journalctl@ceph.osd.7.smithi120.stdout:Jun 10 03:06:13 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-7[54219]: 2024-06-10T03:06:13.310+0000 7f3b24ea2700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-06-10T03:05:40.521059+0000 front 2024-06-10T03:05:40.521090+0000 (oldest deadline 2024-06-10T03:06:06.921348+0000) 2024-06-10T03:06:13.639 INFO:journalctl@ceph.osd.7.smithi120.stdout:Jun 10 03:06:13 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-7[54219]: 2024-06-10T03:06:13.310+0000 7f3b24ea2700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-10T03:05:46.921841+0000 front 2024-06-10T03:05:46.921890+0000 (oldest deadline 2024-06-10T03:06:11.621997+0000) 2024-06-10T03:06:13.639 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:13 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:13.526+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-10T03:05:15.396577+0000 front 2024-06-10T03:05:15.396655+0000 (oldest deadline 2024-06-10T03:05:37.096690+0000) 2024-06-10T03:06:13.639 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:13 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:13.526+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:22.397847+0000 front 2024-06-10T03:05:22.397969+0000 (oldest deadline 2024-06-10T03:05:46.497941+0000) 2024-06-10T03:06:13.639 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:13 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:13.526+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:26.498562+0000 front 2024-06-10T03:05:26.498718+0000 (oldest deadline 2024-06-10T03:05:55.298987+0000) 2024-06-10T03:06:13.639 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:13 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:13.526+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-06-10T03:05:40.000119+0000 front 2024-06-10T03:05:40.000202+0000 (oldest deadline 2024-06-10T03:06:04.000669+0000) 2024-06-10T03:06:13.640 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:13 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:13.526+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-10T03:05:48.101395+0000 front 2024-06-10T03:05:48.101445+0000 (oldest deadline 2024-06-10T03:06:09.801515+0000) 2024-06-10T03:06:14.158 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:13 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:13.780+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-10T03:05:13.354170+0000 front 2024-06-10T03:05:13.354237+0000 (oldest deadline 2024-06-10T03:05:38.053995+0000) 2024-06-10T03:06:14.158 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:13 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:13.780+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:22.755328+0000 front 2024-06-10T03:05:22.755363+0000 (oldest deadline 2024-06-10T03:05:45.655329+0000) 2024-06-10T03:06:14.158 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:13 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:13.780+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:29.156052+0000 front 2024-06-10T03:05:29.156095+0000 (oldest deadline 2024-06-10T03:05:53.856194+0000) 2024-06-10T03:06:14.158 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:13 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:13.780+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-10T03:05:46.158301+0000 front 2024-06-10T03:05:46.158340+0000 (oldest deadline 2024-06-10T03:06:12.058331+0000) 2024-06-10T03:06:14.158 INFO:journalctl@ceph.osd.8.smithi188.stdout:Jun 10 03:06:14 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-8[55165]: 2024-06-10T03:06:14.016+0000 7fc330a8a700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:19.415044+0000 front 2024-06-10T03:05:19.415069+0000 (oldest deadline 2024-06-10T03:05:45.315110+0000) 2024-06-10T03:06:14.158 INFO:journalctl@ceph.osd.8.smithi188.stdout:Jun 10 03:06:14 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-8[55165]: 2024-06-10T03:06:14.016+0000 7fc330a8a700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:32.217415+0000 front 2024-06-10T03:05:32.217588+0000 (oldest deadline 2024-06-10T03:05:54.418216+0000) 2024-06-10T03:06:14.159 INFO:journalctl@ceph.osd.8.smithi188.stdout:Jun 10 03:06:14 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-8[55165]: 2024-06-10T03:06:14.016+0000 7fc330a8a700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-06-10T03:05:39.719178+0000 front 2024-06-10T03:05:39.719197+0000 (oldest deadline 2024-06-10T03:06:03.819222+0000) 2024-06-10T03:06:14.159 INFO:journalctl@ceph.osd.8.smithi188.stdout:Jun 10 03:06:14 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-8[55165]: 2024-06-10T03:06:14.016+0000 7fc330a8a700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-10T03:05:43.819709+0000 front 2024-06-10T03:05:43.819808+0000 (oldest deadline 2024-06-10T03:06:12.620398+0000) 2024-06-10T03:06:14.201 INFO:journalctl@ceph.osd.7.smithi120.stdout:Jun 10 03:06:13 smithi120 podman[71671]: 2024-06-10 03:06:13.922223594 +0000 UTC m=+5.549398857 container died 9e6d20ad28f83eaf3c790677367fd7477079f75ff71b699ebe4476a400ae7ff8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-7, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-06-10T03:06:14.477 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:06:14 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:06:14.011+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-10T03:05:12.298026+0000 front 2024-06-10T03:05:12.298086+0000 (oldest deadline 2024-06-10T03:05:37.598149+0000) 2024-06-10T03:06:14.477 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:06:14 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:06:14.011+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:23.499450+0000 front 2024-06-10T03:05:23.499380+0000 (oldest deadline 2024-06-10T03:05:47.599354+0000) 2024-06-10T03:06:14.477 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:06:14 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:06:14.011+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:27.600040+0000 front 2024-06-10T03:05:27.600064+0000 (oldest deadline 2024-06-10T03:05:53.499906+0000) 2024-06-10T03:06:14.477 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:06:14 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:06:14.011+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-06-10T03:05:38.800668+0000 front 2024-06-10T03:05:38.800723+0000 (oldest deadline 2024-06-10T03:06:02.900794+0000) 2024-06-10T03:06:14.477 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:06:14 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:06:14.011+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-10T03:05:48.101972+0000 front 2024-06-10T03:05:48.102107+0000 (oldest deadline 2024-06-10T03:06:11.002037+0000) 2024-06-10T03:06:14.536 INFO:journalctl@ceph.osd.7.smithi120.stdout:Jun 10 03:06:14 smithi120 podman[71671]: 2024-06-10 03:06:14.309626562 +0000 UTC m=+5.936801806 container cleanup 9e6d20ad28f83eaf3c790677367fd7477079f75ff71b699ebe4476a400ae7ff8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-7, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-06-10T03:06:14.536 INFO:journalctl@ceph.osd.7.smithi120.stdout:Jun 10 03:06:14 smithi120 bash[71671]: ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-7 2024-06-10T03:06:14.889 INFO:journalctl@ceph.osd.7.smithi120.stdout:Jun 10 03:06:14 smithi120 podman[71687]: 2024-06-10 03:06:14.594380251 +0000 UTC m=+0.673850498 container remove 9e6d20ad28f83eaf3c790677367fd7477079f75ff71b699ebe4476a400ae7ff8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-7, ceph=True, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-06-10T03:06:14.889 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:14 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:14.536+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-10T03:05:15.396577+0000 front 2024-06-10T03:05:15.396655+0000 (oldest deadline 2024-06-10T03:05:37.096690+0000) 2024-06-10T03:06:14.889 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:14 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:14.536+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:22.397847+0000 front 2024-06-10T03:05:22.397969+0000 (oldest deadline 2024-06-10T03:05:46.497941+0000) 2024-06-10T03:06:14.889 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:14 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:14.536+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:26.498562+0000 front 2024-06-10T03:05:26.498718+0000 (oldest deadline 2024-06-10T03:05:55.298987+0000) 2024-06-10T03:06:14.889 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:14 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:14.536+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-06-10T03:05:40.000119+0000 front 2024-06-10T03:05:40.000202+0000 (oldest deadline 2024-06-10T03:06:04.000669+0000) 2024-06-10T03:06:14.889 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:14 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:14.536+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-10T03:05:48.101395+0000 front 2024-06-10T03:05:48.101445+0000 (oldest deadline 2024-06-10T03:06:09.801515+0000) 2024-06-10T03:06:15.018 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:14 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:14.825+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-10T03:05:13.354170+0000 front 2024-06-10T03:05:13.354237+0000 (oldest deadline 2024-06-10T03:05:38.053995+0000) 2024-06-10T03:06:15.019 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:14 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:14.825+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:22.755328+0000 front 2024-06-10T03:05:22.755363+0000 (oldest deadline 2024-06-10T03:05:45.655329+0000) 2024-06-10T03:06:15.019 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:14 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:14.825+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:29.156052+0000 front 2024-06-10T03:05:29.156095+0000 (oldest deadline 2024-06-10T03:05:53.856194+0000) 2024-06-10T03:06:15.019 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:14 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:14.825+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-10T03:05:46.158301+0000 front 2024-06-10T03:05:46.158340+0000 (oldest deadline 2024-06-10T03:06:12.058331+0000) 2024-06-10T03:06:15.281 INFO:journalctl@ceph.osd.7.smithi120.stdout:Jun 10 03:06:14 smithi120 podman[71774]: 2024-06-10 03:06:14.843454513 +0000 UTC m=+0.017354035 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-10T03:06:15.281 INFO:journalctl@ceph.osd.7.smithi120.stdout:Jun 10 03:06:14 smithi120 podman[71774]: 2024-06-10 03:06:14.953839011 +0000 UTC m=+0.127738527 container create 75fa4ae3a64d0e8ebae6c7b83327d26d21a36b0bcf88a0401c067eb771dd9041 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-7-deactivate, org.label-schema.build-date=20240528, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-06-10T03:06:15.281 INFO:journalctl@ceph.osd.7.smithi120.stdout:Jun 10 03:06:15 smithi120 podman[71774]: 2024-06-10 03:06:15.181810624 +0000 UTC m=+0.355710163 container init 75fa4ae3a64d0e8ebae6c7b83327d26d21a36b0bcf88a0401c067eb771dd9041 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-7-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-10T03:06:15.281 INFO:journalctl@ceph.osd.7.smithi120.stdout:Jun 10 03:06:15 smithi120 podman[71774]: 2024-06-10 03:06:15.185515915 +0000 UTC m=+0.359415441 container start 75fa4ae3a64d0e8ebae6c7b83327d26d21a36b0bcf88a0401c067eb771dd9041 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-7-deactivate, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc) 2024-06-10T03:06:15.281 INFO:journalctl@ceph.osd.7.smithi120.stdout:Jun 10 03:06:15 smithi120 podman[71774]: 2024-06-10 03:06:15.281287878 +0000 UTC m=+0.455187416 container attach 75fa4ae3a64d0e8ebae6c7b83327d26d21a36b0bcf88a0401c067eb771dd9041 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-7-deactivate, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc) 2024-06-10T03:06:15.454 INFO:journalctl@ceph.osd.8.smithi188.stdout:Jun 10 03:06:15 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-8[55165]: 2024-06-10T03:06:15.018+0000 7fc330a8a700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:19.415044+0000 front 2024-06-10T03:05:19.415069+0000 (oldest deadline 2024-06-10T03:05:45.315110+0000) 2024-06-10T03:06:15.454 INFO:journalctl@ceph.osd.8.smithi188.stdout:Jun 10 03:06:15 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-8[55165]: 2024-06-10T03:06:15.018+0000 7fc330a8a700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:32.217415+0000 front 2024-06-10T03:05:32.217588+0000 (oldest deadline 2024-06-10T03:05:54.418216+0000) 2024-06-10T03:06:15.454 INFO:journalctl@ceph.osd.8.smithi188.stdout:Jun 10 03:06:15 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-8[55165]: 2024-06-10T03:06:15.018+0000 7fc330a8a700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-06-10T03:05:39.719178+0000 front 2024-06-10T03:05:39.719197+0000 (oldest deadline 2024-06-10T03:06:03.819222+0000) 2024-06-10T03:06:15.454 INFO:journalctl@ceph.osd.8.smithi188.stdout:Jun 10 03:06:15 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-8[55165]: 2024-06-10T03:06:15.018+0000 7fc330a8a700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-10T03:05:43.819709+0000 front 2024-06-10T03:05:43.819808+0000 (oldest deadline 2024-06-10T03:06:12.620398+0000) 2024-06-10T03:06:15.477 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:06:15 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:06:15.014+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-10T03:05:12.298026+0000 front 2024-06-10T03:05:12.298086+0000 (oldest deadline 2024-06-10T03:05:37.598149+0000) 2024-06-10T03:06:15.477 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:06:15 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:06:15.014+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:23.499450+0000 front 2024-06-10T03:05:23.499380+0000 (oldest deadline 2024-06-10T03:05:47.599354+0000) 2024-06-10T03:06:15.477 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:06:15 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:06:15.014+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:27.600040+0000 front 2024-06-10T03:05:27.600064+0000 (oldest deadline 2024-06-10T03:05:53.499906+0000) 2024-06-10T03:06:15.477 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:06:15 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:06:15.014+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-06-10T03:05:38.800668+0000 front 2024-06-10T03:05:38.800723+0000 (oldest deadline 2024-06-10T03:06:02.900794+0000) 2024-06-10T03:06:15.477 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:06:15 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:06:15.014+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-10T03:05:48.101972+0000 front 2024-06-10T03:05:48.102107+0000 (oldest deadline 2024-06-10T03:06:11.002037+0000) 2024-06-10T03:06:15.569 INFO:journalctl@ceph.osd.7.smithi120.stdout:Jun 10 03:06:15 smithi120 podman[71774]: 2024-06-10 03:06:15.473954364 +0000 UTC m=+0.647853888 container died 75fa4ae3a64d0e8ebae6c7b83327d26d21a36b0bcf88a0401c067eb771dd9041 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-7-deactivate, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-06-10T03:06:15.889 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:15 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:15.569+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-10T03:05:15.396577+0000 front 2024-06-10T03:05:15.396655+0000 (oldest deadline 2024-06-10T03:05:37.096690+0000) 2024-06-10T03:06:15.889 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:15 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:15.569+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:22.397847+0000 front 2024-06-10T03:05:22.397969+0000 (oldest deadline 2024-06-10T03:05:46.497941+0000) 2024-06-10T03:06:15.889 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:15 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:15.569+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:26.498562+0000 front 2024-06-10T03:05:26.498718+0000 (oldest deadline 2024-06-10T03:05:55.298987+0000) 2024-06-10T03:06:15.889 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:15 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:15.569+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-06-10T03:05:40.000119+0000 front 2024-06-10T03:05:40.000202+0000 (oldest deadline 2024-06-10T03:06:04.000669+0000) 2024-06-10T03:06:15.889 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:15 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:15.569+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-10T03:05:48.101395+0000 front 2024-06-10T03:05:48.101445+0000 (oldest deadline 2024-06-10T03:06:09.801515+0000) 2024-06-10T03:06:16.090 DEBUG:teuthology.orchestra.run.smithi120:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968@osd.7.service' 2024-06-10T03:06:16.131 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-10T03:06:16.131 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-06-10T03:06:16.131 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-06-10T03:06:16.131 DEBUG:teuthology.orchestra.run.smithi188:> sudo systemctl stop ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968@osd.8 2024-06-10T03:06:16.158 INFO:journalctl@ceph.osd.8.smithi188.stdout:Jun 10 03:06:16 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-8[55165]: 2024-06-10T03:06:16.057+0000 7fc330a8a700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:19.415044+0000 front 2024-06-10T03:05:19.415069+0000 (oldest deadline 2024-06-10T03:05:45.315110+0000) 2024-06-10T03:06:16.158 INFO:journalctl@ceph.osd.8.smithi188.stdout:Jun 10 03:06:16 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-8[55165]: 2024-06-10T03:06:16.057+0000 7fc330a8a700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:32.217415+0000 front 2024-06-10T03:05:32.217588+0000 (oldest deadline 2024-06-10T03:05:54.418216+0000) 2024-06-10T03:06:16.158 INFO:journalctl@ceph.osd.8.smithi188.stdout:Jun 10 03:06:16 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-8[55165]: 2024-06-10T03:06:16.057+0000 7fc330a8a700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-06-10T03:05:39.719178+0000 front 2024-06-10T03:05:39.719197+0000 (oldest deadline 2024-06-10T03:06:03.819222+0000) 2024-06-10T03:06:16.158 INFO:journalctl@ceph.osd.8.smithi188.stdout:Jun 10 03:06:16 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-8[55165]: 2024-06-10T03:06:16.057+0000 7fc330a8a700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-10T03:05:43.819709+0000 front 2024-06-10T03:05:43.819808+0000 (oldest deadline 2024-06-10T03:06:12.620398+0000) 2024-06-10T03:06:16.158 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:15 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:15.861+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-10T03:05:13.354170+0000 front 2024-06-10T03:05:13.354237+0000 (oldest deadline 2024-06-10T03:05:38.053995+0000) 2024-06-10T03:06:16.158 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:15 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:15.861+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:22.755328+0000 front 2024-06-10T03:05:22.755363+0000 (oldest deadline 2024-06-10T03:05:45.655329+0000) 2024-06-10T03:06:16.158 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:15 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:15.861+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:29.156052+0000 front 2024-06-10T03:05:29.156095+0000 (oldest deadline 2024-06-10T03:05:53.856194+0000) 2024-06-10T03:06:16.158 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:15 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:15.861+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-10T03:05:46.158301+0000 front 2024-06-10T03:05:46.158340+0000 (oldest deadline 2024-06-10T03:06:12.058331+0000) 2024-06-10T03:06:16.445 INFO:journalctl@ceph.osd.8.smithi188.stdout:Jun 10 03:06:16 smithi188 systemd[1]: Stopping Ceph osd.8 for d8676e9e-26d2-11ef-bc9b-c7b262605968... 2024-06-10T03:06:16.477 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:06:16 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:06:16.054+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-10T03:05:12.298026+0000 front 2024-06-10T03:05:12.298086+0000 (oldest deadline 2024-06-10T03:05:37.598149+0000) 2024-06-10T03:06:16.477 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:06:16 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:06:16.054+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:23.499450+0000 front 2024-06-10T03:05:23.499380+0000 (oldest deadline 2024-06-10T03:05:47.599354+0000) 2024-06-10T03:06:16.477 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:06:16 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:06:16.054+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:27.600040+0000 front 2024-06-10T03:05:27.600064+0000 (oldest deadline 2024-06-10T03:05:53.499906+0000) 2024-06-10T03:06:16.477 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:06:16 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:06:16.054+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-06-10T03:05:38.800668+0000 front 2024-06-10T03:05:38.800723+0000 (oldest deadline 2024-06-10T03:06:02.900794+0000) 2024-06-10T03:06:16.477 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:06:16 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:06:16.054+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-10T03:05:48.101972+0000 front 2024-06-10T03:05:48.102107+0000 (oldest deadline 2024-06-10T03:06:11.002037+0000) 2024-06-10T03:06:16.643 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:16 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:16.570+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-10T03:05:15.396577+0000 front 2024-06-10T03:05:15.396655+0000 (oldest deadline 2024-06-10T03:05:37.096690+0000) 2024-06-10T03:06:16.643 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:16 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:16.570+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:22.397847+0000 front 2024-06-10T03:05:22.397969+0000 (oldest deadline 2024-06-10T03:05:46.497941+0000) 2024-06-10T03:06:16.643 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:16 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:16.570+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:26.498562+0000 front 2024-06-10T03:05:26.498718+0000 (oldest deadline 2024-06-10T03:05:55.298987+0000) 2024-06-10T03:06:16.643 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:16 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:16.570+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-06-10T03:05:40.000119+0000 front 2024-06-10T03:05:40.000202+0000 (oldest deadline 2024-06-10T03:06:04.000669+0000) 2024-06-10T03:06:16.643 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:16 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:16.570+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-10T03:05:48.101395+0000 front 2024-06-10T03:05:48.101445+0000 (oldest deadline 2024-06-10T03:06:09.801515+0000) 2024-06-10T03:06:16.704 INFO:journalctl@ceph.osd.8.smithi188.stdout:Jun 10 03:06:16 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-8[55165]: 2024-06-10T03:06:16.445+0000 7fc334496700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-10T03:06:16.704 INFO:journalctl@ceph.osd.8.smithi188.stdout:Jun 10 03:06:16 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-8[55165]: 2024-06-10T03:06:16.445+0000 7fc334496700 -1 osd.8 104 *** Got signal Terminated *** 2024-06-10T03:06:16.704 INFO:journalctl@ceph.osd.8.smithi188.stdout:Jun 10 03:06:16 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-8[55165]: 2024-06-10T03:06:16.445+0000 7fc334496700 -1 osd.8 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-10T03:06:17.204 INFO:journalctl@ceph.osd.8.smithi188.stdout:Jun 10 03:06:17 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-8[55165]: 2024-06-10T03:06:17.032+0000 7fc330a8a700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:19.415044+0000 front 2024-06-10T03:05:19.415069+0000 (oldest deadline 2024-06-10T03:05:45.315110+0000) 2024-06-10T03:06:17.204 INFO:journalctl@ceph.osd.8.smithi188.stdout:Jun 10 03:06:17 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-8[55165]: 2024-06-10T03:06:17.032+0000 7fc330a8a700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:32.217415+0000 front 2024-06-10T03:05:32.217588+0000 (oldest deadline 2024-06-10T03:05:54.418216+0000) 2024-06-10T03:06:17.204 INFO:journalctl@ceph.osd.8.smithi188.stdout:Jun 10 03:06:17 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-8[55165]: 2024-06-10T03:06:17.032+0000 7fc330a8a700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-06-10T03:05:39.719178+0000 front 2024-06-10T03:05:39.719197+0000 (oldest deadline 2024-06-10T03:06:03.819222+0000) 2024-06-10T03:06:17.204 INFO:journalctl@ceph.osd.8.smithi188.stdout:Jun 10 03:06:17 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-8[55165]: 2024-06-10T03:06:17.032+0000 7fc330a8a700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-10T03:05:43.819709+0000 front 2024-06-10T03:05:43.819808+0000 (oldest deadline 2024-06-10T03:06:12.620398+0000) 2024-06-10T03:06:17.204 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:16 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:16.831+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-10T03:05:13.354170+0000 front 2024-06-10T03:05:13.354237+0000 (oldest deadline 2024-06-10T03:05:38.053995+0000) 2024-06-10T03:06:17.204 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:16 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:16.831+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:22.755328+0000 front 2024-06-10T03:05:22.755363+0000 (oldest deadline 2024-06-10T03:05:45.655329+0000) 2024-06-10T03:06:17.204 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:16 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:16.831+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:29.156052+0000 front 2024-06-10T03:05:29.156095+0000 (oldest deadline 2024-06-10T03:05:53.856194+0000) 2024-06-10T03:06:17.205 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:16 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:16.831+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-10T03:05:46.158301+0000 front 2024-06-10T03:05:46.158340+0000 (oldest deadline 2024-06-10T03:06:12.058331+0000) 2024-06-10T03:06:17.477 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:06:17 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:06:17.043+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-10T03:05:12.298026+0000 front 2024-06-10T03:05:12.298086+0000 (oldest deadline 2024-06-10T03:05:37.598149+0000) 2024-06-10T03:06:17.477 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:06:17 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:06:17.043+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:23.499450+0000 front 2024-06-10T03:05:23.499380+0000 (oldest deadline 2024-06-10T03:05:47.599354+0000) 2024-06-10T03:06:17.477 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:06:17 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:06:17.043+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:27.600040+0000 front 2024-06-10T03:05:27.600064+0000 (oldest deadline 2024-06-10T03:05:53.499906+0000) 2024-06-10T03:06:17.477 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:06:17 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:06:17.043+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-06-10T03:05:38.800668+0000 front 2024-06-10T03:05:38.800723+0000 (oldest deadline 2024-06-10T03:06:02.900794+0000) 2024-06-10T03:06:17.477 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:06:17 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:06:17.043+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-10T03:05:48.101972+0000 front 2024-06-10T03:05:48.102107+0000 (oldest deadline 2024-06-10T03:06:11.002037+0000) 2024-06-10T03:06:17.556 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:17 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:17.556+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-10T03:05:15.396577+0000 front 2024-06-10T03:05:15.396655+0000 (oldest deadline 2024-06-10T03:05:37.096690+0000) 2024-06-10T03:06:17.557 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:17 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:17.556+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:22.397847+0000 front 2024-06-10T03:05:22.397969+0000 (oldest deadline 2024-06-10T03:05:46.497941+0000) 2024-06-10T03:06:17.557 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:17 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:17.556+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:26.498562+0000 front 2024-06-10T03:05:26.498718+0000 (oldest deadline 2024-06-10T03:05:55.298987+0000) 2024-06-10T03:06:17.557 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:17 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:17.556+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-06-10T03:05:40.000119+0000 front 2024-06-10T03:05:40.000202+0000 (oldest deadline 2024-06-10T03:06:04.000669+0000) 2024-06-10T03:06:17.557 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:17 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:17.556+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-10T03:05:48.101395+0000 front 2024-06-10T03:05:48.101445+0000 (oldest deadline 2024-06-10T03:06:09.801515+0000) 2024-06-10T03:06:18.057 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:17 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:17.798+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-10T03:05:13.354170+0000 front 2024-06-10T03:05:13.354237+0000 (oldest deadline 2024-06-10T03:05:38.053995+0000) 2024-06-10T03:06:18.058 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:17 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:17.798+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:22.755328+0000 front 2024-06-10T03:05:22.755363+0000 (oldest deadline 2024-06-10T03:05:45.655329+0000) 2024-06-10T03:06:18.058 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:17 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:17.798+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:29.156052+0000 front 2024-06-10T03:05:29.156095+0000 (oldest deadline 2024-06-10T03:05:53.856194+0000) 2024-06-10T03:06:18.058 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:17 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:17.798+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-10T03:05:46.158301+0000 front 2024-06-10T03:05:46.158340+0000 (oldest deadline 2024-06-10T03:06:12.058331+0000) 2024-06-10T03:06:18.454 INFO:journalctl@ceph.osd.8.smithi188.stdout:Jun 10 03:06:18 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-8[55165]: 2024-06-10T03:06:18.057+0000 7fc330a8a700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:19.415044+0000 front 2024-06-10T03:05:19.415069+0000 (oldest deadline 2024-06-10T03:05:45.315110+0000) 2024-06-10T03:06:18.454 INFO:journalctl@ceph.osd.8.smithi188.stdout:Jun 10 03:06:18 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-8[55165]: 2024-06-10T03:06:18.057+0000 7fc330a8a700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:32.217415+0000 front 2024-06-10T03:05:32.217588+0000 (oldest deadline 2024-06-10T03:05:54.418216+0000) 2024-06-10T03:06:18.454 INFO:journalctl@ceph.osd.8.smithi188.stdout:Jun 10 03:06:18 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-8[55165]: 2024-06-10T03:06:18.057+0000 7fc330a8a700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-06-10T03:05:39.719178+0000 front 2024-06-10T03:05:39.719197+0000 (oldest deadline 2024-06-10T03:06:03.819222+0000) 2024-06-10T03:06:18.454 INFO:journalctl@ceph.osd.8.smithi188.stdout:Jun 10 03:06:18 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-8[55165]: 2024-06-10T03:06:18.057+0000 7fc330a8a700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-10T03:05:43.819709+0000 front 2024-06-10T03:05:43.819808+0000 (oldest deadline 2024-06-10T03:06:12.620398+0000) 2024-06-10T03:06:18.477 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:06:18 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:06:18.010+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-10T03:05:12.298026+0000 front 2024-06-10T03:05:12.298086+0000 (oldest deadline 2024-06-10T03:05:37.598149+0000) 2024-06-10T03:06:18.477 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:06:18 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:06:18.010+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:23.499450+0000 front 2024-06-10T03:05:23.499380+0000 (oldest deadline 2024-06-10T03:05:47.599354+0000) 2024-06-10T03:06:18.477 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:06:18 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:06:18.010+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:27.600040+0000 front 2024-06-10T03:05:27.600064+0000 (oldest deadline 2024-06-10T03:05:53.499906+0000) 2024-06-10T03:06:18.477 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:06:18 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:06:18.010+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-06-10T03:05:38.800668+0000 front 2024-06-10T03:05:38.800723+0000 (oldest deadline 2024-06-10T03:06:02.900794+0000) 2024-06-10T03:06:18.477 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:06:18 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:06:18.010+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-10T03:05:48.101972+0000 front 2024-06-10T03:05:48.102107+0000 (oldest deadline 2024-06-10T03:06:11.002037+0000) 2024-06-10T03:06:18.477 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:06:18 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:06:18.010+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-06-10T03:05:53.303222+0000 front 2024-06-10T03:05:53.303302+0000 (oldest deadline 2024-06-10T03:06:18.003010+0000) 2024-06-10T03:06:18.889 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:18 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:18.584+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-10T03:05:15.396577+0000 front 2024-06-10T03:05:15.396655+0000 (oldest deadline 2024-06-10T03:05:37.096690+0000) 2024-06-10T03:06:18.889 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:18 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:18.584+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:22.397847+0000 front 2024-06-10T03:05:22.397969+0000 (oldest deadline 2024-06-10T03:05:46.497941+0000) 2024-06-10T03:06:18.889 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:18 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:18.584+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:26.498562+0000 front 2024-06-10T03:05:26.498718+0000 (oldest deadline 2024-06-10T03:05:55.298987+0000) 2024-06-10T03:06:18.889 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:18 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:18.584+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-06-10T03:05:40.000119+0000 front 2024-06-10T03:05:40.000202+0000 (oldest deadline 2024-06-10T03:06:04.000669+0000) 2024-06-10T03:06:18.889 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:18 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:18.584+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-10T03:05:48.101395+0000 front 2024-06-10T03:05:48.101445+0000 (oldest deadline 2024-06-10T03:06:09.801515+0000) 2024-06-10T03:06:19.204 INFO:journalctl@ceph.osd.8.smithi188.stdout:Jun 10 03:06:19 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-8[55165]: 2024-06-10T03:06:19.030+0000 7fc330a8a700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:19.415044+0000 front 2024-06-10T03:05:19.415069+0000 (oldest deadline 2024-06-10T03:05:45.315110+0000) 2024-06-10T03:06:19.204 INFO:journalctl@ceph.osd.8.smithi188.stdout:Jun 10 03:06:19 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-8[55165]: 2024-06-10T03:06:19.030+0000 7fc330a8a700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:32.217415+0000 front 2024-06-10T03:05:32.217588+0000 (oldest deadline 2024-06-10T03:05:54.418216+0000) 2024-06-10T03:06:19.204 INFO:journalctl@ceph.osd.8.smithi188.stdout:Jun 10 03:06:19 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-8[55165]: 2024-06-10T03:06:19.030+0000 7fc330a8a700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-06-10T03:05:39.719178+0000 front 2024-06-10T03:05:39.719197+0000 (oldest deadline 2024-06-10T03:06:03.819222+0000) 2024-06-10T03:06:19.204 INFO:journalctl@ceph.osd.8.smithi188.stdout:Jun 10 03:06:19 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-8[55165]: 2024-06-10T03:06:19.030+0000 7fc330a8a700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-10T03:05:43.819709+0000 front 2024-06-10T03:05:43.819808+0000 (oldest deadline 2024-06-10T03:06:12.620398+0000) 2024-06-10T03:06:19.204 INFO:journalctl@ceph.osd.8.smithi188.stdout:Jun 10 03:06:19 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-8[55165]: 2024-06-10T03:06:19.030+0000 7fc330a8a700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-06-10T03:05:52.620910+0000 front 2024-06-10T03:05:52.620814+0000 (oldest deadline 2024-06-10T03:06:18.520944+0000) 2024-06-10T03:06:19.204 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:18 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:18.832+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-10T03:05:13.354170+0000 front 2024-06-10T03:05:13.354237+0000 (oldest deadline 2024-06-10T03:05:38.053995+0000) 2024-06-10T03:06:19.204 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:18 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:18.832+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:22.755328+0000 front 2024-06-10T03:05:22.755363+0000 (oldest deadline 2024-06-10T03:05:45.655329+0000) 2024-06-10T03:06:19.205 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:18 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:18.832+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:29.156052+0000 front 2024-06-10T03:05:29.156095+0000 (oldest deadline 2024-06-10T03:05:53.856194+0000) 2024-06-10T03:06:19.205 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:18 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:18.832+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-10T03:05:46.158301+0000 front 2024-06-10T03:05:46.158340+0000 (oldest deadline 2024-06-10T03:06:12.058331+0000) 2024-06-10T03:06:19.477 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:06:19 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:06:19.031+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-10T03:05:12.298026+0000 front 2024-06-10T03:05:12.298086+0000 (oldest deadline 2024-06-10T03:05:37.598149+0000) 2024-06-10T03:06:19.477 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:06:19 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:06:19.031+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:23.499450+0000 front 2024-06-10T03:05:23.499380+0000 (oldest deadline 2024-06-10T03:05:47.599354+0000) 2024-06-10T03:06:19.477 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:06:19 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:06:19.031+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:27.600040+0000 front 2024-06-10T03:05:27.600064+0000 (oldest deadline 2024-06-10T03:05:53.499906+0000) 2024-06-10T03:06:19.477 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:06:19 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:06:19.031+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-06-10T03:05:38.800668+0000 front 2024-06-10T03:05:38.800723+0000 (oldest deadline 2024-06-10T03:06:02.900794+0000) 2024-06-10T03:06:19.477 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:06:19 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:06:19.031+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-10T03:05:48.101972+0000 front 2024-06-10T03:05:48.102107+0000 (oldest deadline 2024-06-10T03:06:11.002037+0000) 2024-06-10T03:06:19.478 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:06:19 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:06:19.031+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-06-10T03:05:53.303222+0000 front 2024-06-10T03:05:53.303302+0000 (oldest deadline 2024-06-10T03:06:18.003010+0000) 2024-06-10T03:06:19.889 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:19 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:19.625+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-10T03:05:15.396577+0000 front 2024-06-10T03:05:15.396655+0000 (oldest deadline 2024-06-10T03:05:37.096690+0000) 2024-06-10T03:06:19.889 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:19 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:19.625+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:22.397847+0000 front 2024-06-10T03:05:22.397969+0000 (oldest deadline 2024-06-10T03:05:46.497941+0000) 2024-06-10T03:06:19.889 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:19 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:19.625+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:26.498562+0000 front 2024-06-10T03:05:26.498718+0000 (oldest deadline 2024-06-10T03:05:55.298987+0000) 2024-06-10T03:06:19.889 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:19 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:19.625+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-06-10T03:05:40.000119+0000 front 2024-06-10T03:05:40.000202+0000 (oldest deadline 2024-06-10T03:06:04.000669+0000) 2024-06-10T03:06:19.889 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:19 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:19.625+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-10T03:05:48.101395+0000 front 2024-06-10T03:05:48.101445+0000 (oldest deadline 2024-06-10T03:06:09.801515+0000) 2024-06-10T03:06:19.889 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:19 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:19.625+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-06-10T03:05:53.302965+0000 front 2024-06-10T03:05:53.303155+0000 (oldest deadline 2024-06-10T03:06:19.202851+0000) 2024-06-10T03:06:20.204 INFO:journalctl@ceph.osd.8.smithi188.stdout:Jun 10 03:06:20 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-8[55165]: 2024-06-10T03:06:20.014+0000 7fc330a8a700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:19.415044+0000 front 2024-06-10T03:05:19.415069+0000 (oldest deadline 2024-06-10T03:05:45.315110+0000) 2024-06-10T03:06:20.204 INFO:journalctl@ceph.osd.8.smithi188.stdout:Jun 10 03:06:20 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-8[55165]: 2024-06-10T03:06:20.014+0000 7fc330a8a700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:32.217415+0000 front 2024-06-10T03:05:32.217588+0000 (oldest deadline 2024-06-10T03:05:54.418216+0000) 2024-06-10T03:06:20.204 INFO:journalctl@ceph.osd.8.smithi188.stdout:Jun 10 03:06:20 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-8[55165]: 2024-06-10T03:06:20.014+0000 7fc330a8a700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-06-10T03:05:39.719178+0000 front 2024-06-10T03:05:39.719197+0000 (oldest deadline 2024-06-10T03:06:03.819222+0000) 2024-06-10T03:06:20.204 INFO:journalctl@ceph.osd.8.smithi188.stdout:Jun 10 03:06:20 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-8[55165]: 2024-06-10T03:06:20.014+0000 7fc330a8a700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-10T03:05:43.819709+0000 front 2024-06-10T03:05:43.819808+0000 (oldest deadline 2024-06-10T03:06:12.620398+0000) 2024-06-10T03:06:20.204 INFO:journalctl@ceph.osd.8.smithi188.stdout:Jun 10 03:06:20 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-8[55165]: 2024-06-10T03:06:20.014+0000 7fc330a8a700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-06-10T03:05:52.620910+0000 front 2024-06-10T03:05:52.620814+0000 (oldest deadline 2024-06-10T03:06:18.520944+0000) 2024-06-10T03:06:20.205 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:19 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:19.866+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-10T03:05:13.354170+0000 front 2024-06-10T03:05:13.354237+0000 (oldest deadline 2024-06-10T03:05:38.053995+0000) 2024-06-10T03:06:20.205 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:19 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:19.866+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:22.755328+0000 front 2024-06-10T03:05:22.755363+0000 (oldest deadline 2024-06-10T03:05:45.655329+0000) 2024-06-10T03:06:20.205 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:19 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:19.866+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:29.156052+0000 front 2024-06-10T03:05:29.156095+0000 (oldest deadline 2024-06-10T03:05:53.856194+0000) 2024-06-10T03:06:20.205 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:19 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:19.866+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-10T03:05:46.158301+0000 front 2024-06-10T03:05:46.158340+0000 (oldest deadline 2024-06-10T03:06:12.058331+0000) 2024-06-10T03:06:20.477 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:06:20 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:06:20.026+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-10T03:05:12.298026+0000 front 2024-06-10T03:05:12.298086+0000 (oldest deadline 2024-06-10T03:05:37.598149+0000) 2024-06-10T03:06:20.478 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:06:20 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:06:20.026+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:23.499450+0000 front 2024-06-10T03:05:23.499380+0000 (oldest deadline 2024-06-10T03:05:47.599354+0000) 2024-06-10T03:06:20.478 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:06:20 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:06:20.026+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:27.600040+0000 front 2024-06-10T03:05:27.600064+0000 (oldest deadline 2024-06-10T03:05:53.499906+0000) 2024-06-10T03:06:20.478 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:06:20 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:06:20.026+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-06-10T03:05:38.800668+0000 front 2024-06-10T03:05:38.800723+0000 (oldest deadline 2024-06-10T03:06:02.900794+0000) 2024-06-10T03:06:20.478 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:06:20 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:06:20.026+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-10T03:05:48.101972+0000 front 2024-06-10T03:05:48.102107+0000 (oldest deadline 2024-06-10T03:06:11.002037+0000) 2024-06-10T03:06:20.478 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:06:20 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:06:20.026+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-06-10T03:05:53.303222+0000 front 2024-06-10T03:05:53.303302+0000 (oldest deadline 2024-06-10T03:06:18.003010+0000) 2024-06-10T03:06:20.888 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:20 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:20.618+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-10T03:05:15.396577+0000 front 2024-06-10T03:05:15.396655+0000 (oldest deadline 2024-06-10T03:05:37.096690+0000) 2024-06-10T03:06:20.889 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:20 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:20.618+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:22.397847+0000 front 2024-06-10T03:05:22.397969+0000 (oldest deadline 2024-06-10T03:05:46.497941+0000) 2024-06-10T03:06:20.889 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:20 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:20.618+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:26.498562+0000 front 2024-06-10T03:05:26.498718+0000 (oldest deadline 2024-06-10T03:05:55.298987+0000) 2024-06-10T03:06:20.889 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:20 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:20.618+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-06-10T03:05:40.000119+0000 front 2024-06-10T03:05:40.000202+0000 (oldest deadline 2024-06-10T03:06:04.000669+0000) 2024-06-10T03:06:20.889 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:20 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:20.618+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-10T03:05:48.101395+0000 front 2024-06-10T03:05:48.101445+0000 (oldest deadline 2024-06-10T03:06:09.801515+0000) 2024-06-10T03:06:20.889 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:20 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:20.618+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-06-10T03:05:53.302965+0000 front 2024-06-10T03:05:53.303155+0000 (oldest deadline 2024-06-10T03:06:19.202851+0000) 2024-06-10T03:06:21.204 INFO:journalctl@ceph.osd.8.smithi188.stdout:Jun 10 03:06:21 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-8[55165]: 2024-06-10T03:06:21.007+0000 7fc330a8a700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:19.415044+0000 front 2024-06-10T03:05:19.415069+0000 (oldest deadline 2024-06-10T03:05:45.315110+0000) 2024-06-10T03:06:21.204 INFO:journalctl@ceph.osd.8.smithi188.stdout:Jun 10 03:06:21 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-8[55165]: 2024-06-10T03:06:21.007+0000 7fc330a8a700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:32.217415+0000 front 2024-06-10T03:05:32.217588+0000 (oldest deadline 2024-06-10T03:05:54.418216+0000) 2024-06-10T03:06:21.204 INFO:journalctl@ceph.osd.8.smithi188.stdout:Jun 10 03:06:21 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-8[55165]: 2024-06-10T03:06:21.007+0000 7fc330a8a700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-06-10T03:05:39.719178+0000 front 2024-06-10T03:05:39.719197+0000 (oldest deadline 2024-06-10T03:06:03.819222+0000) 2024-06-10T03:06:21.204 INFO:journalctl@ceph.osd.8.smithi188.stdout:Jun 10 03:06:21 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-8[55165]: 2024-06-10T03:06:21.007+0000 7fc330a8a700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-10T03:05:43.819709+0000 front 2024-06-10T03:05:43.819808+0000 (oldest deadline 2024-06-10T03:06:12.620398+0000) 2024-06-10T03:06:21.204 INFO:journalctl@ceph.osd.8.smithi188.stdout:Jun 10 03:06:21 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-8[55165]: 2024-06-10T03:06:21.007+0000 7fc330a8a700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-06-10T03:05:52.620910+0000 front 2024-06-10T03:05:52.620814+0000 (oldest deadline 2024-06-10T03:06:18.520944+0000) 2024-06-10T03:06:21.205 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:20 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:20.859+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-10T03:05:13.354170+0000 front 2024-06-10T03:05:13.354237+0000 (oldest deadline 2024-06-10T03:05:38.053995+0000) 2024-06-10T03:06:21.205 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:20 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:20.859+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:22.755328+0000 front 2024-06-10T03:05:22.755363+0000 (oldest deadline 2024-06-10T03:05:45.655329+0000) 2024-06-10T03:06:21.205 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:20 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:20.859+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:29.156052+0000 front 2024-06-10T03:05:29.156095+0000 (oldest deadline 2024-06-10T03:05:53.856194+0000) 2024-06-10T03:06:21.205 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:20 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:20.859+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-10T03:05:46.158301+0000 front 2024-06-10T03:05:46.158340+0000 (oldest deadline 2024-06-10T03:06:12.058331+0000) 2024-06-10T03:06:21.477 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:06:21 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:06:21.006+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-10T03:05:12.298026+0000 front 2024-06-10T03:05:12.298086+0000 (oldest deadline 2024-06-10T03:05:37.598149+0000) 2024-06-10T03:06:21.477 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:06:21 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:06:21.006+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:23.499450+0000 front 2024-06-10T03:05:23.499380+0000 (oldest deadline 2024-06-10T03:05:47.599354+0000) 2024-06-10T03:06:21.477 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:06:21 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:06:21.006+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:27.600040+0000 front 2024-06-10T03:05:27.600064+0000 (oldest deadline 2024-06-10T03:05:53.499906+0000) 2024-06-10T03:06:21.477 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:06:21 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:06:21.006+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-06-10T03:05:38.800668+0000 front 2024-06-10T03:05:38.800723+0000 (oldest deadline 2024-06-10T03:06:02.900794+0000) 2024-06-10T03:06:21.477 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:06:21 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:06:21.006+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-10T03:05:48.101972+0000 front 2024-06-10T03:05:48.102107+0000 (oldest deadline 2024-06-10T03:06:11.002037+0000) 2024-06-10T03:06:21.477 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:06:21 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:06:21.006+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-06-10T03:05:53.303222+0000 front 2024-06-10T03:05:53.303302+0000 (oldest deadline 2024-06-10T03:06:18.003010+0000) 2024-06-10T03:06:21.889 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:21 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:21.621+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-10T03:05:15.396577+0000 front 2024-06-10T03:05:15.396655+0000 (oldest deadline 2024-06-10T03:05:37.096690+0000) 2024-06-10T03:06:21.889 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:21 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:21.621+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:22.397847+0000 front 2024-06-10T03:05:22.397969+0000 (oldest deadline 2024-06-10T03:05:46.497941+0000) 2024-06-10T03:06:21.889 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:21 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:21.621+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:26.498562+0000 front 2024-06-10T03:05:26.498718+0000 (oldest deadline 2024-06-10T03:05:55.298987+0000) 2024-06-10T03:06:21.889 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:21 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:21.621+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-06-10T03:05:40.000119+0000 front 2024-06-10T03:05:40.000202+0000 (oldest deadline 2024-06-10T03:06:04.000669+0000) 2024-06-10T03:06:21.889 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:21 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:21.621+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-10T03:05:48.101395+0000 front 2024-06-10T03:05:48.101445+0000 (oldest deadline 2024-06-10T03:06:09.801515+0000) 2024-06-10T03:06:21.889 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:21 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:21.621+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-06-10T03:05:53.302965+0000 front 2024-06-10T03:05:53.303155+0000 (oldest deadline 2024-06-10T03:06:19.202851+0000) 2024-06-10T03:06:22.070 INFO:journalctl@ceph.osd.8.smithi188.stdout:Jun 10 03:06:21 smithi188 podman[72393]: 2024-06-10 03:06:21.810332275 +0000 UTC m=+5.485517213 container died 4eb83d6e0842c4000af655ca7a11eb299b2f6cec7ca29abd5e831c437a10647d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-8, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-06-10T03:06:22.070 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:21 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:21.899+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-10T03:05:13.354170+0000 front 2024-06-10T03:05:13.354237+0000 (oldest deadline 2024-06-10T03:05:38.053995+0000) 2024-06-10T03:06:22.070 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:21 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:21.899+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:22.755328+0000 front 2024-06-10T03:05:22.755363+0000 (oldest deadline 2024-06-10T03:05:45.655329+0000) 2024-06-10T03:06:22.071 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:21 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:21.899+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:29.156052+0000 front 2024-06-10T03:05:29.156095+0000 (oldest deadline 2024-06-10T03:05:53.856194+0000) 2024-06-10T03:06:22.071 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:21 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:21.899+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-10T03:05:46.158301+0000 front 2024-06-10T03:05:46.158340+0000 (oldest deadline 2024-06-10T03:06:12.058331+0000) 2024-06-10T03:06:22.454 INFO:journalctl@ceph.osd.8.smithi188.stdout:Jun 10 03:06:22 smithi188 podman[72393]: 2024-06-10 03:06:22.169878132 +0000 UTC m=+5.845063065 container cleanup 4eb83d6e0842c4000af655ca7a11eb299b2f6cec7ca29abd5e831c437a10647d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-8, RELEASE=squid-3d89acc, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-10T03:06:22.454 INFO:journalctl@ceph.osd.8.smithi188.stdout:Jun 10 03:06:22 smithi188 bash[72393]: ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-8 2024-06-10T03:06:22.477 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:06:21 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:06:21.979+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-10T03:05:12.298026+0000 front 2024-06-10T03:05:12.298086+0000 (oldest deadline 2024-06-10T03:05:37.598149+0000) 2024-06-10T03:06:22.477 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:06:21 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:06:21.979+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:23.499450+0000 front 2024-06-10T03:05:23.499380+0000 (oldest deadline 2024-06-10T03:05:47.599354+0000) 2024-06-10T03:06:22.477 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:06:21 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:06:21.979+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:27.600040+0000 front 2024-06-10T03:05:27.600064+0000 (oldest deadline 2024-06-10T03:05:53.499906+0000) 2024-06-10T03:06:22.477 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:06:21 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:06:21.979+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-06-10T03:05:38.800668+0000 front 2024-06-10T03:05:38.800723+0000 (oldest deadline 2024-06-10T03:06:02.900794+0000) 2024-06-10T03:06:22.477 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:06:21 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:06:21.979+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-10T03:05:48.101972+0000 front 2024-06-10T03:05:48.102107+0000 (oldest deadline 2024-06-10T03:06:11.002037+0000) 2024-06-10T03:06:22.477 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:06:21 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:06:21.979+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-06-10T03:05:53.303222+0000 front 2024-06-10T03:05:53.303302+0000 (oldest deadline 2024-06-10T03:06:18.003010+0000) 2024-06-10T03:06:22.795 INFO:journalctl@ceph.osd.8.smithi188.stdout:Jun 10 03:06:22 smithi188 podman[72408]: 2024-06-10 03:06:22.454723916 +0000 UTC m=+0.642438738 container remove 4eb83d6e0842c4000af655ca7a11eb299b2f6cec7ca29abd5e831c437a10647d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-8, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, io.buildah.version=1.33.5) 2024-06-10T03:06:22.888 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:22 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:22.612+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-10T03:05:15.396577+0000 front 2024-06-10T03:05:15.396655+0000 (oldest deadline 2024-06-10T03:05:37.096690+0000) 2024-06-10T03:06:22.889 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:22 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:22.612+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:22.397847+0000 front 2024-06-10T03:05:22.397969+0000 (oldest deadline 2024-06-10T03:05:46.497941+0000) 2024-06-10T03:06:22.889 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:22 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:22.612+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:26.498562+0000 front 2024-06-10T03:05:26.498718+0000 (oldest deadline 2024-06-10T03:05:55.298987+0000) 2024-06-10T03:06:22.889 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:22 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:22.612+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-06-10T03:05:40.000119+0000 front 2024-06-10T03:05:40.000202+0000 (oldest deadline 2024-06-10T03:06:04.000669+0000) 2024-06-10T03:06:22.889 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:22 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:22.612+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-10T03:05:48.101395+0000 front 2024-06-10T03:05:48.101445+0000 (oldest deadline 2024-06-10T03:06:09.801515+0000) 2024-06-10T03:06:22.889 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:22 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:22.612+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-06-10T03:05:53.302965+0000 front 2024-06-10T03:05:53.303155+0000 (oldest deadline 2024-06-10T03:06:19.202851+0000) 2024-06-10T03:06:23.141 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:22 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:22.960+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-10T03:05:13.354170+0000 front 2024-06-10T03:05:13.354237+0000 (oldest deadline 2024-06-10T03:05:38.053995+0000) 2024-06-10T03:06:23.141 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:22 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:22.960+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:22.755328+0000 front 2024-06-10T03:05:22.755363+0000 (oldest deadline 2024-06-10T03:05:45.655329+0000) 2024-06-10T03:06:23.141 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:22 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:22.960+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:29.156052+0000 front 2024-06-10T03:05:29.156095+0000 (oldest deadline 2024-06-10T03:05:53.856194+0000) 2024-06-10T03:06:23.141 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:22 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:22.960+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-10T03:05:46.158301+0000 front 2024-06-10T03:05:46.158340+0000 (oldest deadline 2024-06-10T03:06:12.058331+0000) 2024-06-10T03:06:23.141 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:22 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:22.960+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-06-10T03:05:52.058700+0000 front 2024-06-10T03:05:52.058771+0000 (oldest deadline 2024-06-10T03:06:22.059421+0000) 2024-06-10T03:06:23.141 INFO:journalctl@ceph.osd.8.smithi188.stdout:Jun 10 03:06:22 smithi188 podman[72497]: 2024-06-10 03:06:22.698982723 +0000 UTC m=+0.017272145 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-10T03:06:23.141 INFO:journalctl@ceph.osd.8.smithi188.stdout:Jun 10 03:06:22 smithi188 podman[72497]: 2024-06-10 03:06:22.796926653 +0000 UTC m=+0.115216053 container create 999951bd61650253883bbcef1d4a6b6c10fde979c719029b4dadd0c5baa5040b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-8-deactivate, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-06-10T03:06:23.141 INFO:journalctl@ceph.osd.8.smithi188.stdout:Jun 10 03:06:23 smithi188 podman[72497]: 2024-06-10 03:06:23.041215503 +0000 UTC m=+0.359504913 container init 999951bd61650253883bbcef1d4a6b6c10fde979c719029b4dadd0c5baa5040b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-8-deactivate, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc) 2024-06-10T03:06:23.142 INFO:journalctl@ceph.osd.8.smithi188.stdout:Jun 10 03:06:23 smithi188 podman[72497]: 2024-06-10 03:06:23.044300109 +0000 UTC m=+0.362589521 container start 999951bd61650253883bbcef1d4a6b6c10fde979c719029b4dadd0c5baa5040b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-8-deactivate, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-06-10T03:06:23.142 INFO:journalctl@ceph.osd.8.smithi188.stdout:Jun 10 03:06:23 smithi188 podman[72497]: 2024-06-10 03:06:23.141269048 +0000 UTC m=+0.459558468 container attach 999951bd61650253883bbcef1d4a6b6c10fde979c719029b4dadd0c5baa5040b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-8-deactivate, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-06-10T03:06:23.227 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:06:22 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:06:22.947+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-10T03:05:12.298026+0000 front 2024-06-10T03:05:12.298086+0000 (oldest deadline 2024-06-10T03:05:37.598149+0000) 2024-06-10T03:06:23.227 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:06:22 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:06:22.947+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:23.499450+0000 front 2024-06-10T03:05:23.499380+0000 (oldest deadline 2024-06-10T03:05:47.599354+0000) 2024-06-10T03:06:23.227 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:06:22 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:06:22.947+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:27.600040+0000 front 2024-06-10T03:05:27.600064+0000 (oldest deadline 2024-06-10T03:05:53.499906+0000) 2024-06-10T03:06:23.227 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:06:22 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:06:22.947+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-06-10T03:05:38.800668+0000 front 2024-06-10T03:05:38.800723+0000 (oldest deadline 2024-06-10T03:06:02.900794+0000) 2024-06-10T03:06:23.227 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:06:22 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:06:22.947+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-10T03:05:48.101972+0000 front 2024-06-10T03:05:48.102107+0000 (oldest deadline 2024-06-10T03:06:11.002037+0000) 2024-06-10T03:06:23.227 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:06:22 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:06:22.947+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-06-10T03:05:53.303222+0000 front 2024-06-10T03:05:53.303302+0000 (oldest deadline 2024-06-10T03:06:18.003010+0000) 2024-06-10T03:06:23.419 INFO:journalctl@ceph.osd.8.smithi188.stdout:Jun 10 03:06:23 smithi188 podman[72497]: 2024-06-10 03:06:23.345932104 +0000 UTC m=+0.664221512 container died 999951bd61650253883bbcef1d4a6b6c10fde979c719029b4dadd0c5baa5040b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-8-deactivate, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-10T03:06:23.889 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:23 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:23.564+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-10T03:05:15.396577+0000 front 2024-06-10T03:05:15.396655+0000 (oldest deadline 2024-06-10T03:05:37.096690+0000) 2024-06-10T03:06:23.889 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:23 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:23.564+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:22.397847+0000 front 2024-06-10T03:05:22.397969+0000 (oldest deadline 2024-06-10T03:05:46.497941+0000) 2024-06-10T03:06:23.889 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:23 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:23.564+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:26.498562+0000 front 2024-06-10T03:05:26.498718+0000 (oldest deadline 2024-06-10T03:05:55.298987+0000) 2024-06-10T03:06:23.889 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:23 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:23.564+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-06-10T03:05:40.000119+0000 front 2024-06-10T03:05:40.000202+0000 (oldest deadline 2024-06-10T03:06:04.000669+0000) 2024-06-10T03:06:23.889 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:23 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:23.564+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-10T03:05:48.101395+0000 front 2024-06-10T03:05:48.101445+0000 (oldest deadline 2024-06-10T03:06:09.801515+0000) 2024-06-10T03:06:23.889 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:23 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:23.564+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-06-10T03:05:53.302965+0000 front 2024-06-10T03:05:53.303155+0000 (oldest deadline 2024-06-10T03:06:19.202851+0000) 2024-06-10T03:06:23.931 DEBUG:teuthology.orchestra.run.smithi188:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968@osd.8.service' 2024-06-10T03:06:24.008 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-10T03:06:24.008 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-06-10T03:06:24.009 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-06-10T03:06:24.009 DEBUG:teuthology.orchestra.run.smithi052:> sudo systemctl stop ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968@osd.9 2024-06-10T03:06:24.206 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:23 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:23.984+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-10T03:05:13.354170+0000 front 2024-06-10T03:05:13.354237+0000 (oldest deadline 2024-06-10T03:05:38.053995+0000) 2024-06-10T03:06:24.206 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:23 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:23.984+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:22.755328+0000 front 2024-06-10T03:05:22.755363+0000 (oldest deadline 2024-06-10T03:05:45.655329+0000) 2024-06-10T03:06:24.206 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:23 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:23.984+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:29.156052+0000 front 2024-06-10T03:05:29.156095+0000 (oldest deadline 2024-06-10T03:05:53.856194+0000) 2024-06-10T03:06:24.206 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:23 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:23.984+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-10T03:05:46.158301+0000 front 2024-06-10T03:05:46.158340+0000 (oldest deadline 2024-06-10T03:06:12.058331+0000) 2024-06-10T03:06:24.206 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:23 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:23.984+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-06-10T03:05:52.058700+0000 front 2024-06-10T03:05:52.058771+0000 (oldest deadline 2024-06-10T03:06:22.059421+0000) 2024-06-10T03:06:24.227 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:06:23 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:06:23.972+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-10T03:05:12.298026+0000 front 2024-06-10T03:05:12.298086+0000 (oldest deadline 2024-06-10T03:05:37.598149+0000) 2024-06-10T03:06:24.227 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:06:23 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:06:23.972+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:23.499450+0000 front 2024-06-10T03:05:23.499380+0000 (oldest deadline 2024-06-10T03:05:47.599354+0000) 2024-06-10T03:06:24.227 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:06:23 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:06:23.972+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:27.600040+0000 front 2024-06-10T03:05:27.600064+0000 (oldest deadline 2024-06-10T03:05:53.499906+0000) 2024-06-10T03:06:24.228 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:06:23 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:06:23.972+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-06-10T03:05:38.800668+0000 front 2024-06-10T03:05:38.800723+0000 (oldest deadline 2024-06-10T03:06:02.900794+0000) 2024-06-10T03:06:24.228 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:06:23 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:06:23.972+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-10T03:05:48.101972+0000 front 2024-06-10T03:05:48.102107+0000 (oldest deadline 2024-06-10T03:06:11.002037+0000) 2024-06-10T03:06:24.228 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:06:23 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:06:23.972+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-06-10T03:05:53.303222+0000 front 2024-06-10T03:05:53.303302+0000 (oldest deadline 2024-06-10T03:06:18.003010+0000) 2024-06-10T03:06:24.228 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:06:24 smithi052 systemd[1]: Stopping Ceph osd.9 for d8676e9e-26d2-11ef-bc9b-c7b262605968... 2024-06-10T03:06:24.727 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:06:24 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:06:24.312+0000 7f98752d1700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-10T03:06:24.727 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:06:24 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:06:24.312+0000 7f98752d1700 -1 osd.9 104 *** Got signal Terminated *** 2024-06-10T03:06:24.727 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:06:24 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:06:24.312+0000 7f98752d1700 -1 osd.9 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-10T03:06:24.889 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:24 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:24.597+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-10T03:05:15.396577+0000 front 2024-06-10T03:05:15.396655+0000 (oldest deadline 2024-06-10T03:05:37.096690+0000) 2024-06-10T03:06:24.889 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:24 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:24.597+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:22.397847+0000 front 2024-06-10T03:05:22.397969+0000 (oldest deadline 2024-06-10T03:05:46.497941+0000) 2024-06-10T03:06:24.889 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:24 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:24.597+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:26.498562+0000 front 2024-06-10T03:05:26.498718+0000 (oldest deadline 2024-06-10T03:05:55.298987+0000) 2024-06-10T03:06:24.889 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:24 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:24.597+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-06-10T03:05:40.000119+0000 front 2024-06-10T03:05:40.000202+0000 (oldest deadline 2024-06-10T03:06:04.000669+0000) 2024-06-10T03:06:24.889 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:24 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:24.597+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-10T03:05:48.101395+0000 front 2024-06-10T03:05:48.101445+0000 (oldest deadline 2024-06-10T03:06:09.801515+0000) 2024-06-10T03:06:24.889 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:24 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:24.597+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-06-10T03:05:53.302965+0000 front 2024-06-10T03:05:53.303155+0000 (oldest deadline 2024-06-10T03:06:19.202851+0000) 2024-06-10T03:06:25.227 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:06:24 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:06:24.941+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-10T03:05:12.298026+0000 front 2024-06-10T03:05:12.298086+0000 (oldest deadline 2024-06-10T03:05:37.598149+0000) 2024-06-10T03:06:25.227 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:06:24 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:06:24.941+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:23.499450+0000 front 2024-06-10T03:05:23.499380+0000 (oldest deadline 2024-06-10T03:05:47.599354+0000) 2024-06-10T03:06:25.227 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:06:24 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:06:24.941+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:27.600040+0000 front 2024-06-10T03:05:27.600064+0000 (oldest deadline 2024-06-10T03:05:53.499906+0000) 2024-06-10T03:06:25.227 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:06:24 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:06:24.941+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-06-10T03:05:38.800668+0000 front 2024-06-10T03:05:38.800723+0000 (oldest deadline 2024-06-10T03:06:02.900794+0000) 2024-06-10T03:06:25.227 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:06:24 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:06:24.941+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-10T03:05:48.101972+0000 front 2024-06-10T03:05:48.102107+0000 (oldest deadline 2024-06-10T03:06:11.002037+0000) 2024-06-10T03:06:25.228 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:06:24 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:06:24.941+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-06-10T03:05:53.303222+0000 front 2024-06-10T03:05:53.303302+0000 (oldest deadline 2024-06-10T03:06:18.003010+0000) 2024-06-10T03:06:25.454 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:25 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:25.017+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-10T03:05:13.354170+0000 front 2024-06-10T03:05:13.354237+0000 (oldest deadline 2024-06-10T03:05:38.053995+0000) 2024-06-10T03:06:25.454 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:25 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:25.017+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:22.755328+0000 front 2024-06-10T03:05:22.755363+0000 (oldest deadline 2024-06-10T03:05:45.655329+0000) 2024-06-10T03:06:25.454 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:25 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:25.017+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:29.156052+0000 front 2024-06-10T03:05:29.156095+0000 (oldest deadline 2024-06-10T03:05:53.856194+0000) 2024-06-10T03:06:25.454 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:25 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:25.017+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-10T03:05:46.158301+0000 front 2024-06-10T03:05:46.158340+0000 (oldest deadline 2024-06-10T03:06:12.058331+0000) 2024-06-10T03:06:25.455 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:25 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:25.017+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-06-10T03:05:52.058700+0000 front 2024-06-10T03:05:52.058771+0000 (oldest deadline 2024-06-10T03:06:22.059421+0000) 2024-06-10T03:06:25.888 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:25 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:25.575+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-10T03:05:15.396577+0000 front 2024-06-10T03:05:15.396655+0000 (oldest deadline 2024-06-10T03:05:37.096690+0000) 2024-06-10T03:06:25.889 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:25 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:25.575+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:22.397847+0000 front 2024-06-10T03:05:22.397969+0000 (oldest deadline 2024-06-10T03:05:46.497941+0000) 2024-06-10T03:06:25.889 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:25 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:25.575+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:26.498562+0000 front 2024-06-10T03:05:26.498718+0000 (oldest deadline 2024-06-10T03:05:55.298987+0000) 2024-06-10T03:06:25.889 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:25 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:25.575+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-06-10T03:05:40.000119+0000 front 2024-06-10T03:05:40.000202+0000 (oldest deadline 2024-06-10T03:06:04.000669+0000) 2024-06-10T03:06:25.889 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:25 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:25.575+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-10T03:05:48.101395+0000 front 2024-06-10T03:05:48.101445+0000 (oldest deadline 2024-06-10T03:06:09.801515+0000) 2024-06-10T03:06:25.889 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:25 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:25.575+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-06-10T03:05:53.302965+0000 front 2024-06-10T03:05:53.303155+0000 (oldest deadline 2024-06-10T03:06:19.202851+0000) 2024-06-10T03:06:26.454 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:26 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:26.033+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-10T03:05:13.354170+0000 front 2024-06-10T03:05:13.354237+0000 (oldest deadline 2024-06-10T03:05:38.053995+0000) 2024-06-10T03:06:26.454 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:26 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:26.033+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:22.755328+0000 front 2024-06-10T03:05:22.755363+0000 (oldest deadline 2024-06-10T03:05:45.655329+0000) 2024-06-10T03:06:26.454 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:26 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:26.033+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:29.156052+0000 front 2024-06-10T03:05:29.156095+0000 (oldest deadline 2024-06-10T03:05:53.856194+0000) 2024-06-10T03:06:26.454 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:26 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:26.033+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-10T03:05:46.158301+0000 front 2024-06-10T03:05:46.158340+0000 (oldest deadline 2024-06-10T03:06:12.058331+0000) 2024-06-10T03:06:26.454 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:26 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:26.033+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-06-10T03:05:52.058700+0000 front 2024-06-10T03:05:52.058771+0000 (oldest deadline 2024-06-10T03:06:22.059421+0000) 2024-06-10T03:06:26.477 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:06:25 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:06:25.978+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-10T03:05:12.298026+0000 front 2024-06-10T03:05:12.298086+0000 (oldest deadline 2024-06-10T03:05:37.598149+0000) 2024-06-10T03:06:26.477 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:06:25 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:06:25.978+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:23.499450+0000 front 2024-06-10T03:05:23.499380+0000 (oldest deadline 2024-06-10T03:05:47.599354+0000) 2024-06-10T03:06:26.477 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:06:25 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:06:25.978+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:27.600040+0000 front 2024-06-10T03:05:27.600064+0000 (oldest deadline 2024-06-10T03:05:53.499906+0000) 2024-06-10T03:06:26.477 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:06:25 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:06:25.978+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-06-10T03:05:38.800668+0000 front 2024-06-10T03:05:38.800723+0000 (oldest deadline 2024-06-10T03:06:02.900794+0000) 2024-06-10T03:06:26.477 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:06:25 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:06:25.978+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-10T03:05:48.101972+0000 front 2024-06-10T03:05:48.102107+0000 (oldest deadline 2024-06-10T03:06:11.002037+0000) 2024-06-10T03:06:26.477 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:06:25 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:06:25.978+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-06-10T03:05:53.303222+0000 front 2024-06-10T03:05:53.303302+0000 (oldest deadline 2024-06-10T03:06:18.003010+0000) 2024-06-10T03:06:26.889 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:26 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:26.607+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-10T03:05:15.396577+0000 front 2024-06-10T03:05:15.396655+0000 (oldest deadline 2024-06-10T03:05:37.096690+0000) 2024-06-10T03:06:26.889 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:26 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:26.607+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:22.397847+0000 front 2024-06-10T03:05:22.397969+0000 (oldest deadline 2024-06-10T03:05:46.497941+0000) 2024-06-10T03:06:26.889 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:26 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:26.607+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:26.498562+0000 front 2024-06-10T03:05:26.498718+0000 (oldest deadline 2024-06-10T03:05:55.298987+0000) 2024-06-10T03:06:26.889 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:26 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:26.607+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-06-10T03:05:40.000119+0000 front 2024-06-10T03:05:40.000202+0000 (oldest deadline 2024-06-10T03:06:04.000669+0000) 2024-06-10T03:06:26.889 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:26 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:26.607+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-10T03:05:48.101395+0000 front 2024-06-10T03:05:48.101445+0000 (oldest deadline 2024-06-10T03:06:09.801515+0000) 2024-06-10T03:06:26.889 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:26 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:26.607+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-06-10T03:05:53.302965+0000 front 2024-06-10T03:05:53.303155+0000 (oldest deadline 2024-06-10T03:06:19.202851+0000) 2024-06-10T03:06:27.227 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:06:26 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:06:26.950+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-10T03:05:12.298026+0000 front 2024-06-10T03:05:12.298086+0000 (oldest deadline 2024-06-10T03:05:37.598149+0000) 2024-06-10T03:06:27.227 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:06:26 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:06:26.950+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:23.499450+0000 front 2024-06-10T03:05:23.499380+0000 (oldest deadline 2024-06-10T03:05:47.599354+0000) 2024-06-10T03:06:27.227 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:06:26 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:06:26.950+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:27.600040+0000 front 2024-06-10T03:05:27.600064+0000 (oldest deadline 2024-06-10T03:05:53.499906+0000) 2024-06-10T03:06:27.227 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:06:26 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:06:26.950+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-06-10T03:05:38.800668+0000 front 2024-06-10T03:05:38.800723+0000 (oldest deadline 2024-06-10T03:06:02.900794+0000) 2024-06-10T03:06:27.227 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:06:26 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:06:26.950+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-10T03:05:48.101972+0000 front 2024-06-10T03:05:48.102107+0000 (oldest deadline 2024-06-10T03:06:11.002037+0000) 2024-06-10T03:06:27.227 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:06:26 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:06:26.950+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-06-10T03:05:53.303222+0000 front 2024-06-10T03:05:53.303302+0000 (oldest deadline 2024-06-10T03:06:18.003010+0000) 2024-06-10T03:06:27.454 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:27 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:27.070+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-10T03:05:13.354170+0000 front 2024-06-10T03:05:13.354237+0000 (oldest deadline 2024-06-10T03:05:38.053995+0000) 2024-06-10T03:06:27.454 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:27 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:27.070+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:22.755328+0000 front 2024-06-10T03:05:22.755363+0000 (oldest deadline 2024-06-10T03:05:45.655329+0000) 2024-06-10T03:06:27.454 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:27 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:27.070+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:29.156052+0000 front 2024-06-10T03:05:29.156095+0000 (oldest deadline 2024-06-10T03:05:53.856194+0000) 2024-06-10T03:06:27.454 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:27 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:27.070+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-10T03:05:46.158301+0000 front 2024-06-10T03:05:46.158340+0000 (oldest deadline 2024-06-10T03:06:12.058331+0000) 2024-06-10T03:06:27.454 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:27 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:27.070+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-06-10T03:05:52.058700+0000 front 2024-06-10T03:05:52.058771+0000 (oldest deadline 2024-06-10T03:06:22.059421+0000) 2024-06-10T03:06:27.639 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:27 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:27.583+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-10T03:05:15.396577+0000 front 2024-06-10T03:05:15.396655+0000 (oldest deadline 2024-06-10T03:05:37.096690+0000) 2024-06-10T03:06:27.639 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:27 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:27.583+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:22.397847+0000 front 2024-06-10T03:05:22.397969+0000 (oldest deadline 2024-06-10T03:05:46.497941+0000) 2024-06-10T03:06:27.639 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:27 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:27.583+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:26.498562+0000 front 2024-06-10T03:05:26.498718+0000 (oldest deadline 2024-06-10T03:05:55.298987+0000) 2024-06-10T03:06:27.640 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:27 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:27.583+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-06-10T03:05:40.000119+0000 front 2024-06-10T03:05:40.000202+0000 (oldest deadline 2024-06-10T03:06:04.000669+0000) 2024-06-10T03:06:27.640 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:27 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:27.583+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-10T03:05:48.101395+0000 front 2024-06-10T03:05:48.101445+0000 (oldest deadline 2024-06-10T03:06:09.801515+0000) 2024-06-10T03:06:27.640 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:27 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:27.583+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-06-10T03:05:53.302965+0000 front 2024-06-10T03:05:53.303155+0000 (oldest deadline 2024-06-10T03:06:19.202851+0000) 2024-06-10T03:06:28.227 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:06:27 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:06:27.941+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-10T03:05:12.298026+0000 front 2024-06-10T03:05:12.298086+0000 (oldest deadline 2024-06-10T03:05:37.598149+0000) 2024-06-10T03:06:28.227 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:06:27 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:06:27.941+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:23.499450+0000 front 2024-06-10T03:05:23.499380+0000 (oldest deadline 2024-06-10T03:05:47.599354+0000) 2024-06-10T03:06:28.227 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:06:27 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:06:27.941+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:27.600040+0000 front 2024-06-10T03:05:27.600064+0000 (oldest deadline 2024-06-10T03:05:53.499906+0000) 2024-06-10T03:06:28.227 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:06:27 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:06:27.941+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-06-10T03:05:38.800668+0000 front 2024-06-10T03:05:38.800723+0000 (oldest deadline 2024-06-10T03:06:02.900794+0000) 2024-06-10T03:06:28.227 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:06:27 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:06:27.941+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-10T03:05:48.101972+0000 front 2024-06-10T03:05:48.102107+0000 (oldest deadline 2024-06-10T03:06:11.002037+0000) 2024-06-10T03:06:28.227 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:06:27 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:06:27.941+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-06-10T03:05:53.303222+0000 front 2024-06-10T03:05:53.303302+0000 (oldest deadline 2024-06-10T03:06:18.003010+0000) 2024-06-10T03:06:28.454 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:28 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:28.085+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-10T03:05:13.354170+0000 front 2024-06-10T03:05:13.354237+0000 (oldest deadline 2024-06-10T03:05:38.053995+0000) 2024-06-10T03:06:28.454 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:28 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:28.085+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:22.755328+0000 front 2024-06-10T03:05:22.755363+0000 (oldest deadline 2024-06-10T03:05:45.655329+0000) 2024-06-10T03:06:28.454 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:28 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:28.085+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:29.156052+0000 front 2024-06-10T03:05:29.156095+0000 (oldest deadline 2024-06-10T03:05:53.856194+0000) 2024-06-10T03:06:28.454 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:28 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:28.085+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-10T03:05:46.158301+0000 front 2024-06-10T03:05:46.158340+0000 (oldest deadline 2024-06-10T03:06:12.058331+0000) 2024-06-10T03:06:28.454 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:28 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:28.085+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-06-10T03:05:52.058700+0000 front 2024-06-10T03:05:52.058771+0000 (oldest deadline 2024-06-10T03:06:22.059421+0000) 2024-06-10T03:06:28.454 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:28 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:28.085+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.52:6822 osd.6 since back 2024-06-10T03:06:02.059890+0000 front 2024-06-10T03:06:02.060010+0000 (oldest deadline 2024-06-10T03:06:27.359857+0000) 2024-06-10T03:06:28.889 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:28 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:28.586+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-10T03:05:15.396577+0000 front 2024-06-10T03:05:15.396655+0000 (oldest deadline 2024-06-10T03:05:37.096690+0000) 2024-06-10T03:06:28.889 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:28 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:28.586+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:22.397847+0000 front 2024-06-10T03:05:22.397969+0000 (oldest deadline 2024-06-10T03:05:46.497941+0000) 2024-06-10T03:06:28.889 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:28 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:28.586+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:26.498562+0000 front 2024-06-10T03:05:26.498718+0000 (oldest deadline 2024-06-10T03:05:55.298987+0000) 2024-06-10T03:06:28.889 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:28 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:28.586+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-06-10T03:05:40.000119+0000 front 2024-06-10T03:05:40.000202+0000 (oldest deadline 2024-06-10T03:06:04.000669+0000) 2024-06-10T03:06:28.889 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:28 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:28.586+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-10T03:05:48.101395+0000 front 2024-06-10T03:05:48.101445+0000 (oldest deadline 2024-06-10T03:06:09.801515+0000) 2024-06-10T03:06:28.889 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:28 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:28.586+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-06-10T03:05:53.302965+0000 front 2024-06-10T03:05:53.303155+0000 (oldest deadline 2024-06-10T03:06:19.202851+0000) 2024-06-10T03:06:29.227 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:06:28 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:06:28.965+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-10T03:05:12.298026+0000 front 2024-06-10T03:05:12.298086+0000 (oldest deadline 2024-06-10T03:05:37.598149+0000) 2024-06-10T03:06:29.227 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:06:28 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:06:28.965+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:23.499450+0000 front 2024-06-10T03:05:23.499380+0000 (oldest deadline 2024-06-10T03:05:47.599354+0000) 2024-06-10T03:06:29.227 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:06:28 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:06:28.965+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:27.600040+0000 front 2024-06-10T03:05:27.600064+0000 (oldest deadline 2024-06-10T03:05:53.499906+0000) 2024-06-10T03:06:29.227 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:06:28 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:06:28.965+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-06-10T03:05:38.800668+0000 front 2024-06-10T03:05:38.800723+0000 (oldest deadline 2024-06-10T03:06:02.900794+0000) 2024-06-10T03:06:29.228 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:06:28 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:06:28.965+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-10T03:05:48.101972+0000 front 2024-06-10T03:05:48.102107+0000 (oldest deadline 2024-06-10T03:06:11.002037+0000) 2024-06-10T03:06:29.228 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:06:28 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9[67378]: 2024-06-10T03:06:28.965+0000 7f986d8bd700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-06-10T03:05:53.303222+0000 front 2024-06-10T03:05:53.303302+0000 (oldest deadline 2024-06-10T03:06:18.003010+0000) 2024-06-10T03:06:29.454 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:29 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:29.120+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-10T03:05:13.354170+0000 front 2024-06-10T03:05:13.354237+0000 (oldest deadline 2024-06-10T03:05:38.053995+0000) 2024-06-10T03:06:29.454 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:29 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:29.120+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:22.755328+0000 front 2024-06-10T03:05:22.755363+0000 (oldest deadline 2024-06-10T03:05:45.655329+0000) 2024-06-10T03:06:29.454 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:29 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:29.120+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:29.156052+0000 front 2024-06-10T03:05:29.156095+0000 (oldest deadline 2024-06-10T03:05:53.856194+0000) 2024-06-10T03:06:29.454 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:29 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:29.120+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-10T03:05:46.158301+0000 front 2024-06-10T03:05:46.158340+0000 (oldest deadline 2024-06-10T03:06:12.058331+0000) 2024-06-10T03:06:29.454 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:29 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:29.120+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-06-10T03:05:52.058700+0000 front 2024-06-10T03:05:52.058771+0000 (oldest deadline 2024-06-10T03:06:22.059421+0000) 2024-06-10T03:06:29.454 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:29 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:29.120+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.52:6822 osd.6 since back 2024-06-10T03:06:02.059890+0000 front 2024-06-10T03:06:02.060010+0000 (oldest deadline 2024-06-10T03:06:27.359857+0000) 2024-06-10T03:06:29.889 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:29 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:29.570+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-10T03:05:15.396577+0000 front 2024-06-10T03:05:15.396655+0000 (oldest deadline 2024-06-10T03:05:37.096690+0000) 2024-06-10T03:06:29.889 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:29 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:29.570+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:22.397847+0000 front 2024-06-10T03:05:22.397969+0000 (oldest deadline 2024-06-10T03:05:46.497941+0000) 2024-06-10T03:06:29.889 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:29 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:29.570+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:26.498562+0000 front 2024-06-10T03:05:26.498718+0000 (oldest deadline 2024-06-10T03:05:55.298987+0000) 2024-06-10T03:06:29.889 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:29 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:29.570+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-06-10T03:05:40.000119+0000 front 2024-06-10T03:05:40.000202+0000 (oldest deadline 2024-06-10T03:06:04.000669+0000) 2024-06-10T03:06:29.889 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:29 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:29.570+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-10T03:05:48.101395+0000 front 2024-06-10T03:05:48.101445+0000 (oldest deadline 2024-06-10T03:06:09.801515+0000) 2024-06-10T03:06:29.889 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:29 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:29.570+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-06-10T03:05:53.302965+0000 front 2024-06-10T03:05:53.303155+0000 (oldest deadline 2024-06-10T03:06:19.202851+0000) 2024-06-10T03:06:29.946 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:06:29 smithi052 podman[121099]: 2024-06-10 03:06:29.682866783 +0000 UTC m=+5.465928768 container died bacb807c288dda6a4ac7546063e024478654fe8ba4e35b3317445df243ab500a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-06-10T03:06:30.227 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:06:30 smithi052 podman[121099]: 2024-06-10 03:06:30.137901856 +0000 UTC m=+5.920963840 container cleanup bacb807c288dda6a4ac7546063e024478654fe8ba4e35b3317445df243ab500a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-06-10T03:06:30.227 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:06:30 smithi052 bash[121099]: ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9 2024-06-10T03:06:30.454 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:30 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:30.140+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-10T03:05:13.354170+0000 front 2024-06-10T03:05:13.354237+0000 (oldest deadline 2024-06-10T03:05:38.053995+0000) 2024-06-10T03:06:30.454 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:30 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:30.140+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:22.755328+0000 front 2024-06-10T03:05:22.755363+0000 (oldest deadline 2024-06-10T03:05:45.655329+0000) 2024-06-10T03:06:30.454 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:30 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:30.140+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:29.156052+0000 front 2024-06-10T03:05:29.156095+0000 (oldest deadline 2024-06-10T03:05:53.856194+0000) 2024-06-10T03:06:30.454 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:30 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:30.140+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-10T03:05:46.158301+0000 front 2024-06-10T03:05:46.158340+0000 (oldest deadline 2024-06-10T03:06:12.058331+0000) 2024-06-10T03:06:30.454 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:30 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:30.140+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-06-10T03:05:52.058700+0000 front 2024-06-10T03:05:52.058771+0000 (oldest deadline 2024-06-10T03:06:22.059421+0000) 2024-06-10T03:06:30.454 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:30 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:30.140+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.52:6822 osd.6 since back 2024-06-10T03:06:02.059890+0000 front 2024-06-10T03:06:02.060010+0000 (oldest deadline 2024-06-10T03:06:27.359857+0000) 2024-06-10T03:06:30.719 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:06:30 smithi052 podman[121127]: 2024-06-10 03:06:30.372578676 +0000 UTC m=+0.682924587 container remove bacb807c288dda6a4ac7546063e024478654fe8ba4e35b3317445df243ab500a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9, RELEASE=squid-3d89acc, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True) 2024-06-10T03:06:30.889 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:30 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:30.614+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-10T03:05:15.396577+0000 front 2024-06-10T03:05:15.396655+0000 (oldest deadline 2024-06-10T03:05:37.096690+0000) 2024-06-10T03:06:30.889 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:30 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:30.614+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:22.397847+0000 front 2024-06-10T03:05:22.397969+0000 (oldest deadline 2024-06-10T03:05:46.497941+0000) 2024-06-10T03:06:30.889 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:30 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:30.614+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:26.498562+0000 front 2024-06-10T03:05:26.498718+0000 (oldest deadline 2024-06-10T03:05:55.298987+0000) 2024-06-10T03:06:30.889 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:30 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:30.614+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-06-10T03:05:40.000119+0000 front 2024-06-10T03:05:40.000202+0000 (oldest deadline 2024-06-10T03:06:04.000669+0000) 2024-06-10T03:06:30.889 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:30 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:30.614+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-10T03:05:48.101395+0000 front 2024-06-10T03:05:48.101445+0000 (oldest deadline 2024-06-10T03:06:09.801515+0000) 2024-06-10T03:06:30.889 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:30 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:30.614+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-06-10T03:05:53.302965+0000 front 2024-06-10T03:05:53.303155+0000 (oldest deadline 2024-06-10T03:06:19.202851+0000) 2024-06-10T03:06:30.889 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:30 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:30.614+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.52:6822 osd.6 since back 2024-06-10T03:06:03.704404+0000 front 2024-06-10T03:06:03.704370+0000 (oldest deadline 2024-06-10T03:06:29.604294+0000) 2024-06-10T03:06:30.970 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:06:30 smithi052 podman[121218]: 2024-06-10 03:06:30.622843811 +0000 UTC m=+0.017852553 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-10T03:06:30.970 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:06:30 smithi052 podman[121218]: 2024-06-10 03:06:30.731929553 +0000 UTC m=+0.126938297 container create 889510f3c2605ce8399e4bbf8a1172dcd13b4008cd0e98e95b86c3d4d7d2e627 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9-deactivate, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5) 2024-06-10T03:06:30.971 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:06:30 smithi052 podman[121218]: 2024-06-10 03:06:30.96748053 +0000 UTC m=+0.362489261 container init 889510f3c2605ce8399e4bbf8a1172dcd13b4008cd0e98e95b86c3d4d7d2e627 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2) 2024-06-10T03:06:30.971 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:06:30 smithi052 podman[121218]: 2024-06-10 03:06:30.970951341 +0000 UTC m=+0.365960069 container start 889510f3c2605ce8399e4bbf8a1172dcd13b4008cd0e98e95b86c3d4d7d2e627 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, GIT_CLEAN=True) 2024-06-10T03:06:31.227 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:06:31 smithi052 podman[121218]: 2024-06-10 03:06:31.059400857 +0000 UTC m=+0.454409581 container attach 889510f3c2605ce8399e4bbf8a1172dcd13b4008cd0e98e95b86c3d4d7d2e627 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9-deactivate, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc) 2024-06-10T03:06:31.454 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:31 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:31.099+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-10T03:05:13.354170+0000 front 2024-06-10T03:05:13.354237+0000 (oldest deadline 2024-06-10T03:05:38.053995+0000) 2024-06-10T03:06:31.454 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:31 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:31.099+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:22.755328+0000 front 2024-06-10T03:05:22.755363+0000 (oldest deadline 2024-06-10T03:05:45.655329+0000) 2024-06-10T03:06:31.454 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:31 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:31.099+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:29.156052+0000 front 2024-06-10T03:05:29.156095+0000 (oldest deadline 2024-06-10T03:05:53.856194+0000) 2024-06-10T03:06:31.454 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:31 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:31.099+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-10T03:05:46.158301+0000 front 2024-06-10T03:05:46.158340+0000 (oldest deadline 2024-06-10T03:06:12.058331+0000) 2024-06-10T03:06:31.454 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:31 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:31.099+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-06-10T03:05:52.058700+0000 front 2024-06-10T03:05:52.058771+0000 (oldest deadline 2024-06-10T03:06:22.059421+0000) 2024-06-10T03:06:31.454 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:31 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:31.099+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.52:6822 osd.6 since back 2024-06-10T03:06:02.059890+0000 front 2024-06-10T03:06:02.060010+0000 (oldest deadline 2024-06-10T03:06:27.359857+0000) 2024-06-10T03:06:31.587 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 10 03:06:31 smithi052 podman[121218]: 2024-06-10 03:06:31.283093001 +0000 UTC m=+0.678101733 container died 889510f3c2605ce8399e4bbf8a1172dcd13b4008cd0e98e95b86c3d4d7d2e627 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-9-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528) 2024-06-10T03:06:31.888 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:31 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:31.596+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-10T03:05:15.396577+0000 front 2024-06-10T03:05:15.396655+0000 (oldest deadline 2024-06-10T03:05:37.096690+0000) 2024-06-10T03:06:31.889 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:31 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:31.596+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:22.397847+0000 front 2024-06-10T03:05:22.397969+0000 (oldest deadline 2024-06-10T03:05:46.497941+0000) 2024-06-10T03:06:31.889 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:31 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:31.596+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:26.498562+0000 front 2024-06-10T03:05:26.498718+0000 (oldest deadline 2024-06-10T03:05:55.298987+0000) 2024-06-10T03:06:31.889 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:31 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:31.596+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-06-10T03:05:40.000119+0000 front 2024-06-10T03:05:40.000202+0000 (oldest deadline 2024-06-10T03:06:04.000669+0000) 2024-06-10T03:06:31.889 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:31 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:31.596+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-10T03:05:48.101395+0000 front 2024-06-10T03:05:48.101445+0000 (oldest deadline 2024-06-10T03:06:09.801515+0000) 2024-06-10T03:06:31.889 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:31 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:31.596+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-06-10T03:05:53.302965+0000 front 2024-06-10T03:05:53.303155+0000 (oldest deadline 2024-06-10T03:06:19.202851+0000) 2024-06-10T03:06:31.889 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:31 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:31.596+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.52:6822 osd.6 since back 2024-06-10T03:06:03.704404+0000 front 2024-06-10T03:06:03.704370+0000 (oldest deadline 2024-06-10T03:06:29.604294+0000) 2024-06-10T03:06:31.962 DEBUG:teuthology.orchestra.run.smithi052:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968@osd.9.service' 2024-06-10T03:06:32.004 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-10T03:06:32.004 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-06-10T03:06:32.005 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-06-10T03:06:32.005 DEBUG:teuthology.orchestra.run.smithi120:> sudo systemctl stop ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968@osd.10 2024-06-10T03:06:32.321 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:32 smithi120 systemd[1]: Stopping Ceph osd.10 for d8676e9e-26d2-11ef-bc9b-c7b262605968... 2024-06-10T03:06:32.454 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:32 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:32.130+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-10T03:05:13.354170+0000 front 2024-06-10T03:05:13.354237+0000 (oldest deadline 2024-06-10T03:05:38.053995+0000) 2024-06-10T03:06:32.454 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:32 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:32.130+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:22.755328+0000 front 2024-06-10T03:05:22.755363+0000 (oldest deadline 2024-06-10T03:05:45.655329+0000) 2024-06-10T03:06:32.454 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:32 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:32.130+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:29.156052+0000 front 2024-06-10T03:05:29.156095+0000 (oldest deadline 2024-06-10T03:05:53.856194+0000) 2024-06-10T03:06:32.454 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:32 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:32.130+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-10T03:05:46.158301+0000 front 2024-06-10T03:05:46.158340+0000 (oldest deadline 2024-06-10T03:06:12.058331+0000) 2024-06-10T03:06:32.454 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:32 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:32.130+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-06-10T03:05:52.058700+0000 front 2024-06-10T03:05:52.058771+0000 (oldest deadline 2024-06-10T03:06:22.059421+0000) 2024-06-10T03:06:32.454 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:32 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:32.130+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.52:6822 osd.6 since back 2024-06-10T03:06:02.059890+0000 front 2024-06-10T03:06:02.060010+0000 (oldest deadline 2024-06-10T03:06:27.359857+0000) 2024-06-10T03:06:32.627 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:32 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:32.321+0000 7ff996c7b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-10T03:06:32.627 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:32 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:32.321+0000 7ff996c7b700 -1 osd.10 104 *** Got signal Terminated *** 2024-06-10T03:06:32.627 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:32 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:32.321+0000 7ff996c7b700 -1 osd.10 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-10T03:06:32.628 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:32 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:32.626+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-10T03:05:15.396577+0000 front 2024-06-10T03:05:15.396655+0000 (oldest deadline 2024-06-10T03:05:37.096690+0000) 2024-06-10T03:06:32.628 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:32 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:32.627+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:22.397847+0000 front 2024-06-10T03:05:22.397969+0000 (oldest deadline 2024-06-10T03:05:46.497941+0000) 2024-06-10T03:06:32.889 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:32 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:32.627+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:26.498562+0000 front 2024-06-10T03:05:26.498718+0000 (oldest deadline 2024-06-10T03:05:55.298987+0000) 2024-06-10T03:06:32.889 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:32 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:32.627+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-06-10T03:05:40.000119+0000 front 2024-06-10T03:05:40.000202+0000 (oldest deadline 2024-06-10T03:06:04.000669+0000) 2024-06-10T03:06:32.889 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:32 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:32.627+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-10T03:05:48.101395+0000 front 2024-06-10T03:05:48.101445+0000 (oldest deadline 2024-06-10T03:06:09.801515+0000) 2024-06-10T03:06:32.889 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:32 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:32.627+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-06-10T03:05:53.302965+0000 front 2024-06-10T03:05:53.303155+0000 (oldest deadline 2024-06-10T03:06:19.202851+0000) 2024-06-10T03:06:32.889 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:32 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:32.627+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.52:6822 osd.6 since back 2024-06-10T03:06:03.704404+0000 front 2024-06-10T03:06:03.704370+0000 (oldest deadline 2024-06-10T03:06:29.604294+0000) 2024-06-10T03:06:33.454 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:33 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:33.140+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-10T03:05:13.354170+0000 front 2024-06-10T03:05:13.354237+0000 (oldest deadline 2024-06-10T03:05:38.053995+0000) 2024-06-10T03:06:33.454 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:33 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:33.140+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:22.755328+0000 front 2024-06-10T03:05:22.755363+0000 (oldest deadline 2024-06-10T03:05:45.655329+0000) 2024-06-10T03:06:33.454 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:33 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:33.140+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:29.156052+0000 front 2024-06-10T03:05:29.156095+0000 (oldest deadline 2024-06-10T03:05:53.856194+0000) 2024-06-10T03:06:33.454 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:33 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:33.140+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-10T03:05:46.158301+0000 front 2024-06-10T03:05:46.158340+0000 (oldest deadline 2024-06-10T03:06:12.058331+0000) 2024-06-10T03:06:33.454 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:33 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:33.140+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-06-10T03:05:52.058700+0000 front 2024-06-10T03:05:52.058771+0000 (oldest deadline 2024-06-10T03:06:22.059421+0000) 2024-06-10T03:06:33.454 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:33 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:33.140+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.52:6822 osd.6 since back 2024-06-10T03:06:02.059890+0000 front 2024-06-10T03:06:02.060010+0000 (oldest deadline 2024-06-10T03:06:27.359857+0000) 2024-06-10T03:06:33.889 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:33 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:33.582+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-10T03:05:15.396577+0000 front 2024-06-10T03:05:15.396655+0000 (oldest deadline 2024-06-10T03:05:37.096690+0000) 2024-06-10T03:06:33.889 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:33 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:33.582+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:22.397847+0000 front 2024-06-10T03:05:22.397969+0000 (oldest deadline 2024-06-10T03:05:46.497941+0000) 2024-06-10T03:06:33.889 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:33 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:33.582+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:26.498562+0000 front 2024-06-10T03:05:26.498718+0000 (oldest deadline 2024-06-10T03:05:55.298987+0000) 2024-06-10T03:06:33.890 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:33 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:33.582+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-06-10T03:05:40.000119+0000 front 2024-06-10T03:05:40.000202+0000 (oldest deadline 2024-06-10T03:06:04.000669+0000) 2024-06-10T03:06:33.890 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:33 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:33.582+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-10T03:05:48.101395+0000 front 2024-06-10T03:05:48.101445+0000 (oldest deadline 2024-06-10T03:06:09.801515+0000) 2024-06-10T03:06:33.890 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:33 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:33.582+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-06-10T03:05:53.302965+0000 front 2024-06-10T03:05:53.303155+0000 (oldest deadline 2024-06-10T03:06:19.202851+0000) 2024-06-10T03:06:33.890 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:33 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:33.582+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.52:6822 osd.6 since back 2024-06-10T03:06:03.704404+0000 front 2024-06-10T03:06:03.704370+0000 (oldest deadline 2024-06-10T03:06:29.604294+0000) 2024-06-10T03:06:34.424 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:34 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:34.126+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-10T03:05:13.354170+0000 front 2024-06-10T03:05:13.354237+0000 (oldest deadline 2024-06-10T03:05:38.053995+0000) 2024-06-10T03:06:34.424 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:34 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:34.126+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:22.755328+0000 front 2024-06-10T03:05:22.755363+0000 (oldest deadline 2024-06-10T03:05:45.655329+0000) 2024-06-10T03:06:34.424 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:34 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:34.126+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:29.156052+0000 front 2024-06-10T03:05:29.156095+0000 (oldest deadline 2024-06-10T03:05:53.856194+0000) 2024-06-10T03:06:34.425 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:34 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:34.126+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-10T03:05:46.158301+0000 front 2024-06-10T03:05:46.158340+0000 (oldest deadline 2024-06-10T03:06:12.058331+0000) 2024-06-10T03:06:34.425 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:34 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:34.126+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-06-10T03:05:52.058700+0000 front 2024-06-10T03:05:52.058771+0000 (oldest deadline 2024-06-10T03:06:22.059421+0000) 2024-06-10T03:06:34.425 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:34 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:34.126+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.52:6822 osd.6 since back 2024-06-10T03:06:02.059890+0000 front 2024-06-10T03:06:02.060010+0000 (oldest deadline 2024-06-10T03:06:27.359857+0000) 2024-06-10T03:06:34.889 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:34 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:34.574+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-10T03:05:15.396577+0000 front 2024-06-10T03:05:15.396655+0000 (oldest deadline 2024-06-10T03:05:37.096690+0000) 2024-06-10T03:06:34.889 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:34 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:34.574+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:22.397847+0000 front 2024-06-10T03:05:22.397969+0000 (oldest deadline 2024-06-10T03:05:46.497941+0000) 2024-06-10T03:06:34.889 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:34 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:34.574+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:26.498562+0000 front 2024-06-10T03:05:26.498718+0000 (oldest deadline 2024-06-10T03:05:55.298987+0000) 2024-06-10T03:06:34.889 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:34 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:34.574+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-06-10T03:05:40.000119+0000 front 2024-06-10T03:05:40.000202+0000 (oldest deadline 2024-06-10T03:06:04.000669+0000) 2024-06-10T03:06:34.889 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:34 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:34.574+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-10T03:05:48.101395+0000 front 2024-06-10T03:05:48.101445+0000 (oldest deadline 2024-06-10T03:06:09.801515+0000) 2024-06-10T03:06:34.889 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:34 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:34.574+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-06-10T03:05:53.302965+0000 front 2024-06-10T03:05:53.303155+0000 (oldest deadline 2024-06-10T03:06:19.202851+0000) 2024-06-10T03:06:34.889 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:34 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:34.574+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.52:6822 osd.6 since back 2024-06-10T03:06:03.704404+0000 front 2024-06-10T03:06:03.704370+0000 (oldest deadline 2024-06-10T03:06:29.604294+0000) 2024-06-10T03:06:35.454 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:35 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:35.119+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-10T03:05:13.354170+0000 front 2024-06-10T03:05:13.354237+0000 (oldest deadline 2024-06-10T03:05:38.053995+0000) 2024-06-10T03:06:35.454 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:35 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:35.119+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:22.755328+0000 front 2024-06-10T03:05:22.755363+0000 (oldest deadline 2024-06-10T03:05:45.655329+0000) 2024-06-10T03:06:35.454 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:35 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:35.119+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:29.156052+0000 front 2024-06-10T03:05:29.156095+0000 (oldest deadline 2024-06-10T03:05:53.856194+0000) 2024-06-10T03:06:35.454 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:35 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:35.119+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-10T03:05:46.158301+0000 front 2024-06-10T03:05:46.158340+0000 (oldest deadline 2024-06-10T03:06:12.058331+0000) 2024-06-10T03:06:35.454 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:35 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:35.119+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-06-10T03:05:52.058700+0000 front 2024-06-10T03:05:52.058771+0000 (oldest deadline 2024-06-10T03:06:22.059421+0000) 2024-06-10T03:06:35.454 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:35 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:35.119+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.52:6822 osd.6 since back 2024-06-10T03:06:02.059890+0000 front 2024-06-10T03:06:02.060010+0000 (oldest deadline 2024-06-10T03:06:27.359857+0000) 2024-06-10T03:06:35.455 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:35 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:35.119+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.120:6820 osd.7 since back 2024-06-10T03:06:10.860915+0000 front 2024-06-10T03:06:10.860961+0000 (oldest deadline 2024-06-10T03:06:34.261434+0000) 2024-06-10T03:06:35.889 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:35 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:35.621+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-10T03:05:15.396577+0000 front 2024-06-10T03:05:15.396655+0000 (oldest deadline 2024-06-10T03:05:37.096690+0000) 2024-06-10T03:06:35.889 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:35 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:35.621+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:22.397847+0000 front 2024-06-10T03:05:22.397969+0000 (oldest deadline 2024-06-10T03:05:46.497941+0000) 2024-06-10T03:06:35.889 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:35 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:35.621+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:26.498562+0000 front 2024-06-10T03:05:26.498718+0000 (oldest deadline 2024-06-10T03:05:55.298987+0000) 2024-06-10T03:06:35.889 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:35 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:35.621+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-06-10T03:05:40.000119+0000 front 2024-06-10T03:05:40.000202+0000 (oldest deadline 2024-06-10T03:06:04.000669+0000) 2024-06-10T03:06:35.889 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:35 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:35.621+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-10T03:05:48.101395+0000 front 2024-06-10T03:05:48.101445+0000 (oldest deadline 2024-06-10T03:06:09.801515+0000) 2024-06-10T03:06:35.889 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:35 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:35.621+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-06-10T03:05:53.302965+0000 front 2024-06-10T03:05:53.303155+0000 (oldest deadline 2024-06-10T03:06:19.202851+0000) 2024-06-10T03:06:35.889 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:35 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:35.621+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.52:6822 osd.6 since back 2024-06-10T03:06:03.704404+0000 front 2024-06-10T03:06:03.704370+0000 (oldest deadline 2024-06-10T03:06:29.604294+0000) 2024-06-10T03:06:36.454 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:36 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:36.152+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-10T03:05:13.354170+0000 front 2024-06-10T03:05:13.354237+0000 (oldest deadline 2024-06-10T03:05:38.053995+0000) 2024-06-10T03:06:36.454 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:36 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:36.152+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:22.755328+0000 front 2024-06-10T03:05:22.755363+0000 (oldest deadline 2024-06-10T03:05:45.655329+0000) 2024-06-10T03:06:36.454 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:36 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:36.152+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:29.156052+0000 front 2024-06-10T03:05:29.156095+0000 (oldest deadline 2024-06-10T03:05:53.856194+0000) 2024-06-10T03:06:36.454 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:36 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:36.152+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-10T03:05:46.158301+0000 front 2024-06-10T03:05:46.158340+0000 (oldest deadline 2024-06-10T03:06:12.058331+0000) 2024-06-10T03:06:36.454 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:36 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:36.152+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-06-10T03:05:52.058700+0000 front 2024-06-10T03:05:52.058771+0000 (oldest deadline 2024-06-10T03:06:22.059421+0000) 2024-06-10T03:06:36.454 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:36 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:36.152+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.52:6822 osd.6 since back 2024-06-10T03:06:02.059890+0000 front 2024-06-10T03:06:02.060010+0000 (oldest deadline 2024-06-10T03:06:27.359857+0000) 2024-06-10T03:06:36.454 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:36 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:36.152+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.120:6820 osd.7 since back 2024-06-10T03:06:10.860915+0000 front 2024-06-10T03:06:10.860961+0000 (oldest deadline 2024-06-10T03:06:34.261434+0000) 2024-06-10T03:06:36.889 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:36 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:36.609+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-10T03:05:15.396577+0000 front 2024-06-10T03:05:15.396655+0000 (oldest deadline 2024-06-10T03:05:37.096690+0000) 2024-06-10T03:06:36.889 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:36 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:36.609+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:22.397847+0000 front 2024-06-10T03:05:22.397969+0000 (oldest deadline 2024-06-10T03:05:46.497941+0000) 2024-06-10T03:06:36.889 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:36 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:36.609+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:26.498562+0000 front 2024-06-10T03:05:26.498718+0000 (oldest deadline 2024-06-10T03:05:55.298987+0000) 2024-06-10T03:06:36.889 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:36 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:36.609+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-06-10T03:05:40.000119+0000 front 2024-06-10T03:05:40.000202+0000 (oldest deadline 2024-06-10T03:06:04.000669+0000) 2024-06-10T03:06:36.889 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:36 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:36.609+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-10T03:05:48.101395+0000 front 2024-06-10T03:05:48.101445+0000 (oldest deadline 2024-06-10T03:06:09.801515+0000) 2024-06-10T03:06:36.889 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:36 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:36.609+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-06-10T03:05:53.302965+0000 front 2024-06-10T03:05:53.303155+0000 (oldest deadline 2024-06-10T03:06:19.202851+0000) 2024-06-10T03:06:36.889 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:36 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10[60795]: 2024-06-10T03:06:36.609+0000 7ff98f267700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.52:6822 osd.6 since back 2024-06-10T03:06:03.704404+0000 front 2024-06-10T03:06:03.704370+0000 (oldest deadline 2024-06-10T03:06:29.604294+0000) 2024-06-10T03:06:37.454 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:37 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:37.126+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-10T03:05:13.354170+0000 front 2024-06-10T03:05:13.354237+0000 (oldest deadline 2024-06-10T03:05:38.053995+0000) 2024-06-10T03:06:37.454 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:37 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:37.126+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:22.755328+0000 front 2024-06-10T03:05:22.755363+0000 (oldest deadline 2024-06-10T03:05:45.655329+0000) 2024-06-10T03:06:37.454 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:37 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:37.126+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:29.156052+0000 front 2024-06-10T03:05:29.156095+0000 (oldest deadline 2024-06-10T03:05:53.856194+0000) 2024-06-10T03:06:37.454 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:37 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:37.126+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-10T03:05:46.158301+0000 front 2024-06-10T03:05:46.158340+0000 (oldest deadline 2024-06-10T03:06:12.058331+0000) 2024-06-10T03:06:37.455 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:37 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:37.126+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-06-10T03:05:52.058700+0000 front 2024-06-10T03:05:52.058771+0000 (oldest deadline 2024-06-10T03:06:22.059421+0000) 2024-06-10T03:06:37.455 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:37 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:37.126+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.52:6822 osd.6 since back 2024-06-10T03:06:02.059890+0000 front 2024-06-10T03:06:02.060010+0000 (oldest deadline 2024-06-10T03:06:27.359857+0000) 2024-06-10T03:06:37.455 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:37 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:37.126+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.120:6820 osd.7 since back 2024-06-10T03:06:10.860915+0000 front 2024-06-10T03:06:10.860961+0000 (oldest deadline 2024-06-10T03:06:34.261434+0000) 2024-06-10T03:06:38.048 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:37 smithi120 podman[71904]: 2024-06-10 03:06:37.78411815 +0000 UTC m=+5.584993021 container died b2c5b5c905b95b8c098dd891fd354f8de3e90548157949973d3ecf693deb3c71 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-06-10T03:06:38.326 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:38 smithi120 podman[71904]: 2024-06-10 03:06:38.12190003 +0000 UTC m=+5.922774890 container cleanup b2c5b5c905b95b8c098dd891fd354f8de3e90548157949973d3ecf693deb3c71 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-06-10T03:06:38.326 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:38 smithi120 bash[71904]: ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10 2024-06-10T03:06:38.454 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:38 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:38.158+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-10T03:05:13.354170+0000 front 2024-06-10T03:05:13.354237+0000 (oldest deadline 2024-06-10T03:05:38.053995+0000) 2024-06-10T03:06:38.454 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:38 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:38.158+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:22.755328+0000 front 2024-06-10T03:05:22.755363+0000 (oldest deadline 2024-06-10T03:05:45.655329+0000) 2024-06-10T03:06:38.454 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:38 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:38.158+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:29.156052+0000 front 2024-06-10T03:05:29.156095+0000 (oldest deadline 2024-06-10T03:05:53.856194+0000) 2024-06-10T03:06:38.454 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:38 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:38.158+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-10T03:05:46.158301+0000 front 2024-06-10T03:05:46.158340+0000 (oldest deadline 2024-06-10T03:06:12.058331+0000) 2024-06-10T03:06:38.454 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:38 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:38.158+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-06-10T03:05:52.058700+0000 front 2024-06-10T03:05:52.058771+0000 (oldest deadline 2024-06-10T03:06:22.059421+0000) 2024-06-10T03:06:38.454 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:38 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:38.158+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.52:6822 osd.6 since back 2024-06-10T03:06:02.059890+0000 front 2024-06-10T03:06:02.060010+0000 (oldest deadline 2024-06-10T03:06:27.359857+0000) 2024-06-10T03:06:38.455 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:38 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:38.158+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.120:6820 osd.7 since back 2024-06-10T03:06:10.860915+0000 front 2024-06-10T03:06:10.860961+0000 (oldest deadline 2024-06-10T03:06:34.261434+0000) 2024-06-10T03:06:38.638 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:38 smithi120 podman[71918]: 2024-06-10 03:06:38.339916251 +0000 UTC m=+0.556619931 container remove b2c5b5c905b95b8c098dd891fd354f8de3e90548157949973d3ecf693deb3c71 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, RELEASE=squid-3d89acc, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-06-10T03:06:39.067 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:38 smithi120 podman[72010]: 2024-06-10 03:06:38.64352738 +0000 UTC m=+0.017873109 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-10T03:06:39.068 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:38 smithi120 podman[72010]: 2024-06-10 03:06:38.740837366 +0000 UTC m=+0.115183087 container create c854485d32e670cb2ae4493fcc47d085fb67edbc8e8b3256e4df59ea6cf6c16b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10-deactivate, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240528) 2024-06-10T03:06:39.068 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:38 smithi120 podman[72010]: 2024-06-10 03:06:38.985318217 +0000 UTC m=+0.359663952 container init c854485d32e670cb2ae4493fcc47d085fb67edbc8e8b3256e4df59ea6cf6c16b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10-deactivate, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-06-10T03:06:39.068 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:38 smithi120 podman[72010]: 2024-06-10 03:06:38.988614257 +0000 UTC m=+0.362959984 container start c854485d32e670cb2ae4493fcc47d085fb67edbc8e8b3256e4df59ea6cf6c16b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD) 2024-06-10T03:06:39.068 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:39 smithi120 podman[72010]: 2024-06-10 03:06:39.06823987 +0000 UTC m=+0.442585610 container attach c854485d32e670cb2ae4493fcc47d085fb67edbc8e8b3256e4df59ea6cf6c16b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-06-10T03:06:39.388 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 10 03:06:39 smithi120 podman[72010]: 2024-06-10 03:06:39.276284052 +0000 UTC m=+0.650629774 container died c854485d32e670cb2ae4493fcc47d085fb67edbc8e8b3256e4df59ea6cf6c16b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-10-deactivate, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux ) 2024-06-10T03:06:39.454 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:39 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:39.195+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-10T03:05:13.354170+0000 front 2024-06-10T03:05:13.354237+0000 (oldest deadline 2024-06-10T03:05:38.053995+0000) 2024-06-10T03:06:39.454 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:39 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:39.195+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:22.755328+0000 front 2024-06-10T03:05:22.755363+0000 (oldest deadline 2024-06-10T03:05:45.655329+0000) 2024-06-10T03:06:39.454 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:39 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:39.195+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:29.156052+0000 front 2024-06-10T03:05:29.156095+0000 (oldest deadline 2024-06-10T03:05:53.856194+0000) 2024-06-10T03:06:39.454 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:39 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:39.195+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-10T03:05:46.158301+0000 front 2024-06-10T03:05:46.158340+0000 (oldest deadline 2024-06-10T03:06:12.058331+0000) 2024-06-10T03:06:39.454 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:39 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:39.195+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-06-10T03:05:52.058700+0000 front 2024-06-10T03:05:52.058771+0000 (oldest deadline 2024-06-10T03:06:22.059421+0000) 2024-06-10T03:06:39.454 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:39 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:39.195+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.52:6822 osd.6 since back 2024-06-10T03:06:02.059890+0000 front 2024-06-10T03:06:02.060010+0000 (oldest deadline 2024-06-10T03:06:27.359857+0000) 2024-06-10T03:06:39.454 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:39 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:39.195+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.120:6820 osd.7 since back 2024-06-10T03:06:10.860915+0000 front 2024-06-10T03:06:10.860961+0000 (oldest deadline 2024-06-10T03:06:34.261434+0000) 2024-06-10T03:06:39.854 DEBUG:teuthology.orchestra.run.smithi120:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968@osd.10.service' 2024-06-10T03:06:39.892 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-10T03:06:39.892 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-06-10T03:06:39.892 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-06-10T03:06:39.892 DEBUG:teuthology.orchestra.run.smithi188:> sudo systemctl stop ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968@osd.11 2024-06-10T03:06:40.196 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:39 smithi188 systemd[1]: Stopping Ceph osd.11 for d8676e9e-26d2-11ef-bc9b-c7b262605968... 2024-06-10T03:06:40.454 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:40 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:40.195+0000 7f1e29a01700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-10T03:06:40.454 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:40 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:40.195+0000 7f1e29a01700 -1 osd.11 104 *** Got signal Terminated *** 2024-06-10T03:06:40.454 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:40 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:40.195+0000 7f1e29a01700 -1 osd.11 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-10T03:06:40.454 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:40 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:40.219+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-10T03:05:13.354170+0000 front 2024-06-10T03:05:13.354237+0000 (oldest deadline 2024-06-10T03:05:38.053995+0000) 2024-06-10T03:06:40.454 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:40 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:40.219+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:22.755328+0000 front 2024-06-10T03:05:22.755363+0000 (oldest deadline 2024-06-10T03:05:45.655329+0000) 2024-06-10T03:06:40.454 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:40 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:40.219+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:29.156052+0000 front 2024-06-10T03:05:29.156095+0000 (oldest deadline 2024-06-10T03:05:53.856194+0000) 2024-06-10T03:06:40.454 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:40 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:40.219+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-10T03:05:46.158301+0000 front 2024-06-10T03:05:46.158340+0000 (oldest deadline 2024-06-10T03:06:12.058331+0000) 2024-06-10T03:06:40.455 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:40 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:40.219+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-06-10T03:05:52.058700+0000 front 2024-06-10T03:05:52.058771+0000 (oldest deadline 2024-06-10T03:06:22.059421+0000) 2024-06-10T03:06:40.455 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:40 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:40.219+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.52:6822 osd.6 since back 2024-06-10T03:06:02.059890+0000 front 2024-06-10T03:06:02.060010+0000 (oldest deadline 2024-06-10T03:06:27.359857+0000) 2024-06-10T03:06:40.455 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:40 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:40.219+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.120:6820 osd.7 since back 2024-06-10T03:06:10.860915+0000 front 2024-06-10T03:06:10.860961+0000 (oldest deadline 2024-06-10T03:06:34.261434+0000) 2024-06-10T03:06:41.704 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:41 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:41.229+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-10T03:05:13.354170+0000 front 2024-06-10T03:05:13.354237+0000 (oldest deadline 2024-06-10T03:05:38.053995+0000) 2024-06-10T03:06:41.704 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:41 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:41.229+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:22.755328+0000 front 2024-06-10T03:05:22.755363+0000 (oldest deadline 2024-06-10T03:05:45.655329+0000) 2024-06-10T03:06:41.704 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:41 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:41.229+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:29.156052+0000 front 2024-06-10T03:05:29.156095+0000 (oldest deadline 2024-06-10T03:05:53.856194+0000) 2024-06-10T03:06:41.704 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:41 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:41.229+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-10T03:05:46.158301+0000 front 2024-06-10T03:05:46.158340+0000 (oldest deadline 2024-06-10T03:06:12.058331+0000) 2024-06-10T03:06:41.704 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:41 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:41.229+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-06-10T03:05:52.058700+0000 front 2024-06-10T03:05:52.058771+0000 (oldest deadline 2024-06-10T03:06:22.059421+0000) 2024-06-10T03:06:41.704 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:41 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:41.229+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.52:6822 osd.6 since back 2024-06-10T03:06:02.059890+0000 front 2024-06-10T03:06:02.060010+0000 (oldest deadline 2024-06-10T03:06:27.359857+0000) 2024-06-10T03:06:41.705 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:41 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:41.229+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.120:6820 osd.7 since back 2024-06-10T03:06:10.860915+0000 front 2024-06-10T03:06:10.860961+0000 (oldest deadline 2024-06-10T03:06:34.261434+0000) 2024-06-10T03:06:42.704 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:42 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:42.249+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-10T03:05:13.354170+0000 front 2024-06-10T03:05:13.354237+0000 (oldest deadline 2024-06-10T03:05:38.053995+0000) 2024-06-10T03:06:42.704 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:42 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:42.249+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:22.755328+0000 front 2024-06-10T03:05:22.755363+0000 (oldest deadline 2024-06-10T03:05:45.655329+0000) 2024-06-10T03:06:42.704 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:42 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:42.249+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:29.156052+0000 front 2024-06-10T03:05:29.156095+0000 (oldest deadline 2024-06-10T03:05:53.856194+0000) 2024-06-10T03:06:42.705 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:42 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:42.249+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-10T03:05:46.158301+0000 front 2024-06-10T03:05:46.158340+0000 (oldest deadline 2024-06-10T03:06:12.058331+0000) 2024-06-10T03:06:42.705 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:42 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:42.249+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-06-10T03:05:52.058700+0000 front 2024-06-10T03:05:52.058771+0000 (oldest deadline 2024-06-10T03:06:22.059421+0000) 2024-06-10T03:06:42.705 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:42 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:42.249+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.52:6822 osd.6 since back 2024-06-10T03:06:02.059890+0000 front 2024-06-10T03:06:02.060010+0000 (oldest deadline 2024-06-10T03:06:27.359857+0000) 2024-06-10T03:06:42.705 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:42 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:42.249+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.120:6820 osd.7 since back 2024-06-10T03:06:10.860915+0000 front 2024-06-10T03:06:10.860961+0000 (oldest deadline 2024-06-10T03:06:34.261434+0000) 2024-06-10T03:06:43.704 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:43 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:43.296+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-10T03:05:13.354170+0000 front 2024-06-10T03:05:13.354237+0000 (oldest deadline 2024-06-10T03:05:38.053995+0000) 2024-06-10T03:06:43.704 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:43 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:43.296+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:22.755328+0000 front 2024-06-10T03:05:22.755363+0000 (oldest deadline 2024-06-10T03:05:45.655329+0000) 2024-06-10T03:06:43.704 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:43 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:43.296+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:29.156052+0000 front 2024-06-10T03:05:29.156095+0000 (oldest deadline 2024-06-10T03:05:53.856194+0000) 2024-06-10T03:06:43.704 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:43 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:43.296+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-10T03:05:46.158301+0000 front 2024-06-10T03:05:46.158340+0000 (oldest deadline 2024-06-10T03:06:12.058331+0000) 2024-06-10T03:06:43.704 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:43 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:43.296+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-06-10T03:05:52.058700+0000 front 2024-06-10T03:05:52.058771+0000 (oldest deadline 2024-06-10T03:06:22.059421+0000) 2024-06-10T03:06:43.704 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:43 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:43.296+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.52:6822 osd.6 since back 2024-06-10T03:06:02.059890+0000 front 2024-06-10T03:06:02.060010+0000 (oldest deadline 2024-06-10T03:06:27.359857+0000) 2024-06-10T03:06:43.704 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:43 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:43.296+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.120:6820 osd.7 since back 2024-06-10T03:06:10.860915+0000 front 2024-06-10T03:06:10.860961+0000 (oldest deadline 2024-06-10T03:06:34.261434+0000) 2024-06-10T03:06:44.704 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:44 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:44.249+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-10T03:05:13.354170+0000 front 2024-06-10T03:05:13.354237+0000 (oldest deadline 2024-06-10T03:05:38.053995+0000) 2024-06-10T03:06:44.704 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:44 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:44.249+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-10T03:05:22.755328+0000 front 2024-06-10T03:05:22.755363+0000 (oldest deadline 2024-06-10T03:05:45.655329+0000) 2024-06-10T03:06:44.704 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:44 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:44.249+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-06-10T03:05:29.156052+0000 front 2024-06-10T03:05:29.156095+0000 (oldest deadline 2024-06-10T03:05:53.856194+0000) 2024-06-10T03:06:44.704 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:44 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:44.249+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-10T03:05:46.158301+0000 front 2024-06-10T03:05:46.158340+0000 (oldest deadline 2024-06-10T03:06:12.058331+0000) 2024-06-10T03:06:44.704 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:44 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:44.249+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-06-10T03:05:52.058700+0000 front 2024-06-10T03:05:52.058771+0000 (oldest deadline 2024-06-10T03:06:22.059421+0000) 2024-06-10T03:06:44.704 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:44 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:44.249+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.52:6822 osd.6 since back 2024-06-10T03:06:02.059890+0000 front 2024-06-10T03:06:02.060010+0000 (oldest deadline 2024-06-10T03:06:27.359857+0000) 2024-06-10T03:06:44.704 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:44 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11[61740]: 2024-06-10T03:06:44.249+0000 7f1e21fed700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.120:6820 osd.7 since back 2024-06-10T03:06:10.860915+0000 front 2024-06-10T03:06:10.860961+0000 (oldest deadline 2024-06-10T03:06:34.261434+0000) 2024-06-10T03:06:45.813 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:45 smithi188 podman[72616]: 2024-06-10 03:06:45.55734543 +0000 UTC m=+5.469614129 container died 9d67c43c665bae39caf534ecf100862ba8360e40a3c5df503b87095abe8df043 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-06-10T03:06:46.163 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:45 smithi188 podman[72616]: 2024-06-10 03:06:45.88713786 +0000 UTC m=+5.799406562 container cleanup 9d67c43c665bae39caf534ecf100862ba8360e40a3c5df503b87095abe8df043 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-10T03:06:46.163 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:45 smithi188 bash[72616]: ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11 2024-06-10T03:06:46.454 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:46 smithi188 podman[72633]: 2024-06-10 03:06:46.163485105 +0000 UTC m=+0.604784475 container remove 9d67c43c665bae39caf534ecf100862ba8360e40a3c5df503b87095abe8df043 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-06-10T03:06:46.808 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:46 smithi188 podman[72725]: 2024-06-10 03:06:46.472477245 +0000 UTC m=+0.109593094 container create 99a4b6f878b24b42e182999368dc4a0fcb6c58f5eb006621c449e5e41c427541 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11-deactivate, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-06-10T03:06:46.808 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:46 smithi188 podman[72725]: 2024-06-10 03:06:46.380066595 +0000 UTC m=+0.017182454 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-10T03:06:46.808 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:46 smithi188 podman[72725]: 2024-06-10 03:06:46.716775821 +0000 UTC m=+0.353891673 container init 99a4b6f878b24b42e182999368dc4a0fcb6c58f5eb006621c449e5e41c427541 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11-deactivate, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-10T03:06:46.808 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:46 smithi188 podman[72725]: 2024-06-10 03:06:46.719917668 +0000 UTC m=+0.357033524 container start 99a4b6f878b24b42e182999368dc4a0fcb6c58f5eb006621c449e5e41c427541 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528) 2024-06-10T03:06:46.808 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:46 smithi188 podman[72725]: 2024-06-10 03:06:46.808453437 +0000 UTC m=+0.445569292 container attach 99a4b6f878b24b42e182999368dc4a0fcb6c58f5eb006621c449e5e41c427541 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11-deactivate, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-06-10T03:06:47.204 INFO:journalctl@ceph.osd.11.smithi188.stdout:Jun 10 03:06:47 smithi188 podman[72725]: 2024-06-10 03:06:47.010846679 +0000 UTC m=+0.647962536 container died 99a4b6f878b24b42e182999368dc4a0fcb6c58f5eb006621c449e5e41c427541 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-osd-11-deactivate, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528) 2024-06-10T03:06:47.574 DEBUG:teuthology.orchestra.run.smithi188:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968@osd.11.service' 2024-06-10T03:06:47.611 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-10T03:06:47.611 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-06-10T03:06:47.611 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-06-10T03:06:47.611 DEBUG:teuthology.orchestra.run.smithi052:> sudo systemctl stop ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968@mds.a 2024-06-10T03:06:47.924 INFO:journalctl@ceph.mds.a.smithi052.stdout:Jun 10 03:06:47 smithi052 systemd[1]: Stopping Ceph mds.a for d8676e9e-26d2-11ef-bc9b-c7b262605968... 2024-06-10T03:06:48.227 INFO:journalctl@ceph.mds.a.smithi052.stdout:Jun 10 03:06:47 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mds-a[74607]: 2024-06-10T03:06:47.923+0000 7ff08768f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-10T03:06:48.227 INFO:journalctl@ceph.mds.a.smithi052.stdout:Jun 10 03:06:47 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mds-a[74607]: 2024-06-10T03:06:47.923+0000 7ff08768f700 -1 mds.a *** got signal Terminated *** 2024-06-10T03:06:50.398 INFO:journalctl@ceph.mds.a.smithi052.stdout:Jun 10 03:06:50 smithi052 podman[121325]: 2024-06-10 03:06:50.057940126 +0000 UTC m=+2.238387299 container died 20535a539d25316c74f9e9fc1c30c803e6a62f9a60c575fa97717355850608cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mds-a, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528) 2024-06-10T03:06:50.398 INFO:journalctl@ceph.mds.a.smithi052.stdout:Jun 10 03:06:50 smithi052 podman[121325]: 2024-06-10 03:06:50.398306339 +0000 UTC m=+2.578753509 container cleanup 20535a539d25316c74f9e9fc1c30c803e6a62f9a60c575fa97717355850608cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mds-a, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, ceph=True, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-06-10T03:06:50.727 INFO:journalctl@ceph.mds.a.smithi052.stdout:Jun 10 03:06:50 smithi052 bash[121325]: ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mds-a 2024-06-10T03:06:50.727 INFO:journalctl@ceph.mds.a.smithi052.stdout:Jun 10 03:06:50 smithi052 podman[121339]: 2024-06-10 03:06:50.583040848 +0000 UTC m=+0.521259837 container remove 20535a539d25316c74f9e9fc1c30c803e6a62f9a60c575fa97717355850608cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mds-a, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-06-10T03:06:50.844 DEBUG:teuthology.orchestra.run.smithi052:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968@mds.a.service' 2024-06-10T03:06:50.879 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-10T03:06:50.879 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-06-10T03:06:50.879 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-06-10T03:06:50.880 DEBUG:teuthology.orchestra.run.smithi052:> sudo systemctl stop ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968@mds.d 2024-06-10T03:06:51.202 INFO:journalctl@ceph.mds.d.smithi052.stdout:Jun 10 03:06:50 smithi052 systemd[1]: Stopping Ceph mds.d for d8676e9e-26d2-11ef-bc9b-c7b262605968... 2024-06-10T03:06:51.477 INFO:journalctl@ceph.mds.d.smithi052.stdout:Jun 10 03:06:51 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mds-d[73349]: 2024-06-10T03:06:51.201+0000 7fa230742700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-10T03:06:51.478 INFO:journalctl@ceph.mds.d.smithi052.stdout:Jun 10 03:06:51 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mds-d[73349]: 2024-06-10T03:06:51.201+0000 7fa230742700 -1 mds.d *** got signal Terminated *** 2024-06-10T03:06:54.295 INFO:journalctl@ceph.mds.d.smithi052.stdout:Jun 10 03:06:54 smithi052 podman[121445]: 2024-06-10 03:06:54.015937289 +0000 UTC m=+2.901426056 container died 51f2513273d2eaf6ce1688beeb24f985b458578f8e748dad6c4d9b2bb0bcb1ac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mds-d, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, ceph=True) 2024-06-10T03:06:54.653 INFO:journalctl@ceph.mds.d.smithi052.stdout:Jun 10 03:06:54 smithi052 podman[121445]: 2024-06-10 03:06:54.452595102 +0000 UTC m=+3.338083852 container cleanup 51f2513273d2eaf6ce1688beeb24f985b458578f8e748dad6c4d9b2bb0bcb1ac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mds-d, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-10T03:06:54.654 INFO:journalctl@ceph.mds.d.smithi052.stdout:Jun 10 03:06:54 smithi052 bash[121445]: ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mds-d 2024-06-10T03:06:54.904 INFO:journalctl@ceph.mds.d.smithi052.stdout:Jun 10 03:06:54 smithi052 podman[121458]: 2024-06-10 03:06:54.654034683 +0000 UTC m=+0.631184875 container remove 51f2513273d2eaf6ce1688beeb24f985b458578f8e748dad6c4d9b2bb0bcb1ac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mds-d, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, ceph=True, org.label-schema.build-date=20240528, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-06-10T03:06:54.904 INFO:journalctl@ceph.mds.d.smithi052.stdout:Jun 10 03:06:54 smithi052 systemd[1]: ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968@mds.d.service: Deactivated successfully. 2024-06-10T03:06:54.905 INFO:journalctl@ceph.mds.d.smithi052.stdout:Jun 10 03:06:54 smithi052 systemd[1]: Stopped Ceph mds.d for d8676e9e-26d2-11ef-bc9b-c7b262605968. 2024-06-10T03:06:54.905 INFO:journalctl@ceph.mds.d.smithi052.stdout:Jun 10 03:06:54 smithi052 systemd[1]: ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968@mds.d.service: Consumed 1.112s CPU time. 2024-06-10T03:06:54.906 DEBUG:teuthology.orchestra.run.smithi052:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968@mds.d.service' 2024-06-10T03:06:54.939 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-10T03:06:54.939 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-06-10T03:06:54.939 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-06-10T03:06:54.939 DEBUG:teuthology.orchestra.run.smithi052:> sudo systemctl stop ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968@mds.g 2024-06-10T03:06:55.227 INFO:journalctl@ceph.mds.g.smithi052.stdout:Jun 10 03:06:55 smithi052 systemd[1]: Stopping Ceph mds.g for d8676e9e-26d2-11ef-bc9b-c7b262605968... 2024-06-10T03:06:55.727 INFO:journalctl@ceph.mds.g.smithi052.stdout:Jun 10 03:06:55 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mds-g[74147]: 2024-06-10T03:06:55.465+0000 7f4b4822f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-10T03:06:55.727 INFO:journalctl@ceph.mds.g.smithi052.stdout:Jun 10 03:06:55 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mds-g[74147]: 2024-06-10T03:06:55.465+0000 7f4b4822f700 -1 mds.g *** got signal Terminated *** 2024-06-10T03:06:58.315 INFO:journalctl@ceph.mds.g.smithi052.stdout:Jun 10 03:06:58 smithi052 podman[121564]: 2024-06-10 03:06:58.044748996 +0000 UTC m=+2.876581720 container died 76f195dccc988bd64f4212ed1badb6021f602f5339e3cffb48a3d0daf32636b3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mds-g, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-10T03:06:58.727 INFO:journalctl@ceph.mds.g.smithi052.stdout:Jun 10 03:06:58 smithi052 podman[121564]: 2024-06-10 03:06:58.498339595 +0000 UTC m=+3.330172302 container cleanup 76f195dccc988bd64f4212ed1badb6021f602f5339e3cffb48a3d0daf32636b3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mds-g, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-06-10T03:06:58.727 INFO:journalctl@ceph.mds.g.smithi052.stdout:Jun 10 03:06:58 smithi052 bash[121564]: ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mds-g 2024-06-10T03:06:58.929 DEBUG:teuthology.orchestra.run.smithi052:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968@mds.g.service' 2024-06-10T03:06:58.963 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-10T03:06:58.963 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-06-10T03:06:58.963 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-06-10T03:06:58.963 DEBUG:teuthology.orchestra.run.smithi052:> sudo systemctl stop ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968@mds.j 2024-06-10T03:06:59.283 INFO:journalctl@ceph.mds.j.smithi052.stdout:Jun 10 03:06:59 smithi052 systemd[1]: Stopping Ceph mds.j for d8676e9e-26d2-11ef-bc9b-c7b262605968... 2024-06-10T03:06:59.727 INFO:journalctl@ceph.mds.j.smithi052.stdout:Jun 10 03:06:59 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mds-j[72907]: 2024-06-10T03:06:59.281+0000 7f9fd0396700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-10T03:06:59.727 INFO:journalctl@ceph.mds.j.smithi052.stdout:Jun 10 03:06:59 smithi052 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mds-j[72907]: 2024-06-10T03:06:59.282+0000 7f9fd0396700 -1 mds.j *** got signal Terminated *** 2024-06-10T03:07:01.996 INFO:journalctl@ceph.mds.j.smithi052.stdout:Jun 10 03:07:01 smithi052 podman[121684]: 2024-06-10 03:07:01.742874625 +0000 UTC m=+2.553271595 container died 6ba576331e94db1d4f7f22b03deba8d69e93243eea22e4980e7abeab01a874a2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mds-j, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-10T03:07:02.337 INFO:journalctl@ceph.mds.j.smithi052.stdout:Jun 10 03:07:02 smithi052 podman[121684]: 2024-06-10 03:07:02.161302709 +0000 UTC m=+2.971699719 container cleanup 6ba576331e94db1d4f7f22b03deba8d69e93243eea22e4980e7abeab01a874a2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mds-j, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-06-10T03:07:02.337 INFO:journalctl@ceph.mds.j.smithi052.stdout:Jun 10 03:07:02 smithi052 bash[121684]: ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mds-j 2024-06-10T03:07:02.600 INFO:journalctl@ceph.mds.j.smithi052.stdout:Jun 10 03:07:02 smithi052 podman[121709]: 2024-06-10 03:07:02.337448786 +0000 UTC m=+0.596167750 container remove 6ba576331e94db1d4f7f22b03deba8d69e93243eea22e4980e7abeab01a874a2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mds-j, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-10T03:07:02.605 DEBUG:teuthology.orchestra.run.smithi052:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968@mds.j.service' 2024-06-10T03:07:02.639 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-10T03:07:02.639 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-06-10T03:07:02.640 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-06-10T03:07:02.640 DEBUG:teuthology.orchestra.run.smithi120:> sudo systemctl stop ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968@mds.b 2024-06-10T03:07:02.972 INFO:journalctl@ceph.mds.b.smithi120.stdout:Jun 10 03:07:02 smithi120 systemd[1]: Stopping Ceph mds.b for d8676e9e-26d2-11ef-bc9b-c7b262605968... 2024-06-10T03:07:03.231 INFO:journalctl@ceph.mds.b.smithi120.stdout:Jun 10 03:07:02 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mds-b[66008]: 2024-06-10T03:07:02.971+0000 7f1c54c6e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-10T03:07:03.231 INFO:journalctl@ceph.mds.b.smithi120.stdout:Jun 10 03:07:02 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mds-b[66008]: 2024-06-10T03:07:02.971+0000 7f1c54c6e700 -1 mds.b *** got signal Terminated *** 2024-06-10T03:07:06.314 INFO:journalctl@ceph.mds.b.smithi120.stdout:Jun 10 03:07:06 smithi120 podman[72116]: 2024-06-10 03:07:06.042440675 +0000 UTC m=+3.212454783 container died 920e913c3ac785cbb9088617e1ef9b3bdd425117a4df8ac647c4f323ba53888c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mds-b, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=) 2024-06-10T03:07:06.639 INFO:journalctl@ceph.mds.b.smithi120.stdout:Jun 10 03:07:06 smithi120 podman[72116]: 2024-06-10 03:07:06.405768456 +0000 UTC m=+3.575782555 container cleanup 920e913c3ac785cbb9088617e1ef9b3bdd425117a4df8ac647c4f323ba53888c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mds-b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-10T03:07:06.639 INFO:journalctl@ceph.mds.b.smithi120.stdout:Jun 10 03:07:06 smithi120 bash[72116]: ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mds-b 2024-06-10T03:07:06.844 DEBUG:teuthology.orchestra.run.smithi120:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968@mds.b.service' 2024-06-10T03:07:06.879 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-10T03:07:06.879 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-06-10T03:07:06.879 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-06-10T03:07:06.880 DEBUG:teuthology.orchestra.run.smithi120:> sudo systemctl stop ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968@mds.e 2024-06-10T03:07:07.210 INFO:journalctl@ceph.mds.e.smithi120.stdout:Jun 10 03:07:06 smithi120 systemd[1]: Stopping Ceph mds.e for d8676e9e-26d2-11ef-bc9b-c7b262605968... 2024-06-10T03:07:07.638 INFO:journalctl@ceph.mds.e.smithi120.stdout:Jun 10 03:07:07 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mds-e[65389]: 2024-06-10T03:07:07.209+0000 7f684e627700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-10T03:07:07.639 INFO:journalctl@ceph.mds.e.smithi120.stdout:Jun 10 03:07:07 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mds-e[65389]: 2024-06-10T03:07:07.209+0000 7f684e627700 -1 mds.e *** got signal Terminated *** 2024-06-10T03:07:10.302 INFO:journalctl@ceph.mds.e.smithi120.stdout:Jun 10 03:07:10 smithi120 podman[72248]: 2024-06-10 03:07:10.035291326 +0000 UTC m=+2.933288888 container died e38b88a6eb058e63a998296d4d14bbe6e4418284f925720c67fdd23acf4f1781 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mds-e, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528) 2024-06-10T03:07:10.638 INFO:journalctl@ceph.mds.e.smithi120.stdout:Jun 10 03:07:10 smithi120 podman[72248]: 2024-06-10 03:07:10.401742858 +0000 UTC m=+3.299740412 container cleanup e38b88a6eb058e63a998296d4d14bbe6e4418284f925720c67fdd23acf4f1781 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mds-e, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-06-10T03:07:10.639 INFO:journalctl@ceph.mds.e.smithi120.stdout:Jun 10 03:07:10 smithi120 bash[72248]: ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mds-e 2024-06-10T03:07:10.807 DEBUG:teuthology.orchestra.run.smithi120:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968@mds.e.service' 2024-06-10T03:07:10.839 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-10T03:07:10.839 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-06-10T03:07:10.839 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-06-10T03:07:10.839 DEBUG:teuthology.orchestra.run.smithi120:> sudo systemctl stop ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968@mds.h 2024-06-10T03:07:11.180 INFO:journalctl@ceph.mds.h.smithi120.stdout:Jun 10 03:07:10 smithi120 systemd[1]: Stopping Ceph mds.h for d8676e9e-26d2-11ef-bc9b-c7b262605968... 2024-06-10T03:07:11.639 INFO:journalctl@ceph.mds.h.smithi120.stdout:Jun 10 03:07:11 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mds-h[65703]: 2024-06-10T03:07:11.179+0000 7fcaad452700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-10T03:07:11.639 INFO:journalctl@ceph.mds.h.smithi120.stdout:Jun 10 03:07:11 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mds-h[65703]: 2024-06-10T03:07:11.180+0000 7fcaad452700 -1 mds.h *** got signal Terminated *** 2024-06-10T03:07:14.313 INFO:journalctl@ceph.mds.h.smithi120.stdout:Jun 10 03:07:14 smithi120 podman[72369]: 2024-06-10 03:07:14.037121168 +0000 UTC m=+2.974568375 container died a7d21c767cd48c5de5b4b362b1c410c35234e590203a343bd16314dc489046ba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mds-h, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-06-10T03:07:14.639 INFO:journalctl@ceph.mds.h.smithi120.stdout:Jun 10 03:07:14 smithi120 podman[72369]: 2024-06-10 03:07:14.413876325 +0000 UTC m=+3.351323530 container cleanup a7d21c767cd48c5de5b4b362b1c410c35234e590203a343bd16314dc489046ba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mds-h, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-10T03:07:14.639 INFO:journalctl@ceph.mds.h.smithi120.stdout:Jun 10 03:07:14 smithi120 bash[72369]: ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mds-h 2024-06-10T03:07:14.793 DEBUG:teuthology.orchestra.run.smithi120:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968@mds.h.service' 2024-06-10T03:07:14.823 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-10T03:07:14.824 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-06-10T03:07:14.824 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-06-10T03:07:14.824 DEBUG:teuthology.orchestra.run.smithi120:> sudo systemctl stop ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968@mds.k 2024-06-10T03:07:15.389 INFO:journalctl@ceph.mds.k.smithi120.stdout:Jun 10 03:07:14 smithi120 systemd[1]: Stopping Ceph mds.k for d8676e9e-26d2-11ef-bc9b-c7b262605968... 2024-06-10T03:07:15.389 INFO:journalctl@ceph.mds.k.smithi120.stdout:Jun 10 03:07:15 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mds-k[65075]: 2024-06-10T03:07:15.142+0000 7fe56a0b3700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-10T03:07:15.389 INFO:journalctl@ceph.mds.k.smithi120.stdout:Jun 10 03:07:15 smithi120 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mds-k[65075]: 2024-06-10T03:07:15.142+0000 7fe56a0b3700 -1 mds.k *** got signal Terminated *** 2024-06-10T03:07:18.378 INFO:journalctl@ceph.mds.k.smithi120.stdout:Jun 10 03:07:18 smithi120 podman[72491]: 2024-06-10 03:07:18.04099301 +0000 UTC m=+3.003041195 container died 87d4e55dc881189ff24119696bed15ea75240f677a08f32425e6adedf06111c6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mds-k, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-06-10T03:07:18.639 INFO:journalctl@ceph.mds.k.smithi120.stdout:Jun 10 03:07:18 smithi120 podman[72491]: 2024-06-10 03:07:18.451596103 +0000 UTC m=+3.413644288 container cleanup 87d4e55dc881189ff24119696bed15ea75240f677a08f32425e6adedf06111c6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mds-k, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-06-10T03:07:18.639 INFO:journalctl@ceph.mds.k.smithi120.stdout:Jun 10 03:07:18 smithi120 bash[72491]: ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mds-k 2024-06-10T03:07:18.820 DEBUG:teuthology.orchestra.run.smithi120:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968@mds.k.service' 2024-06-10T03:07:18.851 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-10T03:07:18.851 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-06-10T03:07:18.851 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-06-10T03:07:18.851 DEBUG:teuthology.orchestra.run.smithi188:> sudo systemctl stop ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968@mds.c 2024-06-10T03:07:19.142 INFO:journalctl@ceph.mds.c.smithi188.stdout:Jun 10 03:07:18 smithi188 systemd[1]: Stopping Ceph mds.c for d8676e9e-26d2-11ef-bc9b-c7b262605968... 2024-06-10T03:07:19.454 INFO:journalctl@ceph.mds.c.smithi188.stdout:Jun 10 03:07:19 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mds-c[66642]: 2024-06-10T03:07:19.142+0000 7f88bb976700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-10T03:07:19.454 INFO:journalctl@ceph.mds.c.smithi188.stdout:Jun 10 03:07:19 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mds-c[66642]: 2024-06-10T03:07:19.142+0000 7f88bb976700 -1 mds.c *** got signal Terminated *** 2024-06-10T03:07:22.442 INFO:journalctl@ceph.mds.c.smithi188.stdout:Jun 10 03:07:22 smithi188 podman[72829]: 2024-06-10 03:07:22.069123022 +0000 UTC m=+3.019616530 container died 714d5a7132bb0e96d2c39b78cd361de88975ce3fb7a258a0d18ddbffee93c38b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mds-c, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True) 2024-06-10T03:07:22.704 INFO:journalctl@ceph.mds.c.smithi188.stdout:Jun 10 03:07:22 smithi188 podman[72829]: 2024-06-10 03:07:22.542041081 +0000 UTC m=+3.492534590 container cleanup 714d5a7132bb0e96d2c39b78cd361de88975ce3fb7a258a0d18ddbffee93c38b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mds-c, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0) 2024-06-10T03:07:22.704 INFO:journalctl@ceph.mds.c.smithi188.stdout:Jun 10 03:07:22 smithi188 bash[72829]: ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mds-c 2024-06-10T03:07:23.027 DEBUG:teuthology.orchestra.run.smithi188:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968@mds.c.service' 2024-06-10T03:07:23.063 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-10T03:07:23.064 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-06-10T03:07:23.064 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-06-10T03:07:23.064 DEBUG:teuthology.orchestra.run.smithi188:> sudo systemctl stop ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968@mds.f 2024-06-10T03:07:23.387 INFO:journalctl@ceph.mds.f.smithi188.stdout:Jun 10 03:07:23 smithi188 systemd[1]: Stopping Ceph mds.f for d8676e9e-26d2-11ef-bc9b-c7b262605968... 2024-06-10T03:07:23.703 INFO:journalctl@ceph.mds.f.smithi188.stdout:Jun 10 03:07:23 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mds-f[66339]: 2024-06-10T03:07:23.387+0000 7fdfd33d9700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-10T03:07:23.704 INFO:journalctl@ceph.mds.f.smithi188.stdout:Jun 10 03:07:23 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mds-f[66339]: 2024-06-10T03:07:23.387+0000 7fdfd33d9700 -1 mds.f *** got signal Terminated *** 2024-06-10T03:07:26.344 INFO:journalctl@ceph.mds.f.smithi188.stdout:Jun 10 03:07:26 smithi188 podman[72958]: 2024-06-10 03:07:26.070786382 +0000 UTC m=+2.790323663 container died d833706531e889d802362fdafb3fd63f13f00b56823861510e78ba58fd1d9a04 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mds-f, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-06-10T03:07:26.679 INFO:journalctl@ceph.mds.f.smithi188.stdout:Jun 10 03:07:26 smithi188 podman[72958]: 2024-06-10 03:07:26.445736226 +0000 UTC m=+3.165273507 container cleanup d833706531e889d802362fdafb3fd63f13f00b56823861510e78ba58fd1d9a04 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mds-f, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2) 2024-06-10T03:07:26.680 INFO:journalctl@ceph.mds.f.smithi188.stdout:Jun 10 03:07:26 smithi188 bash[72958]: ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mds-f 2024-06-10T03:07:26.825 DEBUG:teuthology.orchestra.run.smithi188:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968@mds.f.service' 2024-06-10T03:07:26.859 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-10T03:07:26.859 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-06-10T03:07:26.859 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-06-10T03:07:26.859 DEBUG:teuthology.orchestra.run.smithi188:> sudo systemctl stop ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968@mds.i 2024-06-10T03:07:27.204 INFO:journalctl@ceph.mds.i.smithi188.stdout:Jun 10 03:07:26 smithi188 systemd[1]: Stopping Ceph mds.i for d8676e9e-26d2-11ef-bc9b-c7b262605968... 2024-06-10T03:07:27.204 INFO:journalctl@ceph.mds.i.smithi188.stdout:Jun 10 03:07:27 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mds-i[66946]: 2024-06-10T03:07:27.182+0000 7eff7af44700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-10T03:07:27.204 INFO:journalctl@ceph.mds.i.smithi188.stdout:Jun 10 03:07:27 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mds-i[66946]: 2024-06-10T03:07:27.182+0000 7eff7af44700 -1 mds.i *** got signal Terminated *** 2024-06-10T03:07:30.408 INFO:journalctl@ceph.mds.i.smithi188.stdout:Jun 10 03:07:30 smithi188 podman[73078]: 2024-06-10 03:07:30.070699859 +0000 UTC m=+2.996305381 container died 843470528b5be402a8e6ed580c3d719a17dd4bcfd9fa141870596e14cf8b5e02 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mds-i, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-06-10T03:07:30.683 INFO:journalctl@ceph.mds.i.smithi188.stdout:Jun 10 03:07:30 smithi188 podman[73078]: 2024-06-10 03:07:30.499225188 +0000 UTC m=+3.424830710 container cleanup 843470528b5be402a8e6ed580c3d719a17dd4bcfd9fa141870596e14cf8b5e02 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mds-i, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-10T03:07:30.684 INFO:journalctl@ceph.mds.i.smithi188.stdout:Jun 10 03:07:30 smithi188 bash[73078]: ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mds-i 2024-06-10T03:07:30.927 DEBUG:teuthology.orchestra.run.smithi188:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968@mds.i.service' 2024-06-10T03:07:30.947 INFO:journalctl@ceph.mds.i.smithi188.stdout:Jun 10 03:07:30 smithi188 podman[73090]: 2024-06-10 03:07:30.683956554 +0000 UTC m=+0.613441359 container remove 843470528b5be402a8e6ed580c3d719a17dd4bcfd9fa141870596e14cf8b5e02 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mds-i, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, maintainer=Guillaume Abrioux ) 2024-06-10T03:07:30.947 INFO:journalctl@ceph.mds.i.smithi188.stdout:Jun 10 03:07:30 smithi188 systemd[1]: ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968@mds.i.service: Deactivated successfully. 2024-06-10T03:07:30.947 INFO:journalctl@ceph.mds.i.smithi188.stdout:Jun 10 03:07:30 smithi188 systemd[1]: Stopped Ceph mds.i for d8676e9e-26d2-11ef-bc9b-c7b262605968. 2024-06-10T03:07:30.948 INFO:journalctl@ceph.mds.i.smithi188.stdout:Jun 10 03:07:30 smithi188 systemd[1]: ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968@mds.i.service: Consumed 18.830s CPU time. 2024-06-10T03:07:30.958 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-10T03:07:30.959 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-06-10T03:07:30.959 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-06-10T03:07:30.959 DEBUG:teuthology.orchestra.run.smithi188:> sudo systemctl stop ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968@mds.l 2024-06-10T03:07:31.204 INFO:journalctl@ceph.mds.l.smithi188.stdout:Jun 10 03:07:31 smithi188 systemd[1]: Stopping Ceph mds.l for d8676e9e-26d2-11ef-bc9b-c7b262605968... 2024-06-10T03:07:31.704 INFO:journalctl@ceph.mds.l.smithi188.stdout:Jun 10 03:07:31 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mds-l[66025]: 2024-06-10T03:07:31.302+0000 7f5a43edf700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-10T03:07:31.704 INFO:journalctl@ceph.mds.l.smithi188.stdout:Jun 10 03:07:31 smithi188 ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mds-l[66025]: 2024-06-10T03:07:31.302+0000 7f5a43edf700 -1 mds.l *** got signal Terminated *** 2024-06-10T03:07:34.420 INFO:journalctl@ceph.mds.l.smithi188.stdout:Jun 10 03:07:34 smithi188 podman[73199]: 2024-06-10 03:07:34.071045058 +0000 UTC m=+2.887083518 container died 14557ac164f932444d000162d6bea3fae330fb441b9a1dbb1ed599d0b2786b37 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mds-l, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-06-10T03:07:34.687 INFO:journalctl@ceph.mds.l.smithi188.stdout:Jun 10 03:07:34 smithi188 podman[73199]: 2024-06-10 03:07:34.494767287 +0000 UTC m=+3.310805742 container cleanup 14557ac164f932444d000162d6bea3fae330fb441b9a1dbb1ed599d0b2786b37 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mds-l, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc) 2024-06-10T03:07:34.687 INFO:journalctl@ceph.mds.l.smithi188.stdout:Jun 10 03:07:34 smithi188 bash[73199]: ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968-mds-l 2024-06-10T03:07:34.913 DEBUG:teuthology.orchestra.run.smithi188:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d8676e9e-26d2-11ef-bc9b-c7b262605968@mds.l.service' 2024-06-10T03:07:34.945 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-10T03:07:34.945 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-06-10T03:07:34.945 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 --force --keep-logs 2024-06-10T03:07:35.146 INFO:teuthology.orchestra.run.smithi052.stdout:Deleting cluster with fsid: d8676e9e-26d2-11ef-bc9b-c7b262605968 2024-06-10T03:07:37.676 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 --force --keep-logs 2024-06-10T03:07:37.866 INFO:teuthology.orchestra.run.smithi120.stdout:Deleting cluster with fsid: d8676e9e-26d2-11ef-bc9b-c7b262605968 2024-06-10T03:07:40.239 DEBUG:teuthology.orchestra.run.smithi188:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 --force --keep-logs 2024-06-10T03:07:40.427 INFO:teuthology.orchestra.run.smithi188.stdout:Deleting cluster with fsid: d8676e9e-26d2-11ef-bc9b-c7b262605968 2024-06-10T03:07:42.911 DEBUG:teuthology.orchestra.run.smithi052:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-06-10T03:07:42.946 DEBUG:teuthology.orchestra.run.smithi120:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-06-10T03:07:42.976 DEBUG:teuthology.orchestra.run.smithi188:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-06-10T03:07:43.002 INFO:tasks.cephadm:Archiving crash dumps... 2024-06-10T03:07:43.004 DEBUG:teuthology.misc:Transferring archived files from smithi052:/var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-06-09_21:24:02-fs-squid-distro-default-smithi/7747454/remote/smithi052/crash 2024-06-10T03:07:43.005 DEBUG:teuthology.orchestra.run.smithi052:> sudo tar c -f - -C /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/crash -- . 2024-06-10T03:07:43.032 INFO:teuthology.orchestra.run.smithi052.stderr:tar: /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/crash: Cannot open: No such file or directory 2024-06-10T03:07:43.033 INFO:teuthology.orchestra.run.smithi052.stderr:tar: Error is not recoverable: exiting now 2024-06-10T03:07:43.035 DEBUG:teuthology.misc:Transferring archived files from smithi120:/var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-06-09_21:24:02-fs-squid-distro-default-smithi/7747454/remote/smithi120/crash 2024-06-10T03:07:43.035 DEBUG:teuthology.orchestra.run.smithi120:> sudo tar c -f - -C /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/crash -- . 2024-06-10T03:07:43.077 INFO:teuthology.orchestra.run.smithi120.stderr:tar: /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/crash: Cannot open: No such file or directory 2024-06-10T03:07:43.078 INFO:teuthology.orchestra.run.smithi120.stderr:tar: Error is not recoverable: exiting now 2024-06-10T03:07:43.080 DEBUG:teuthology.misc:Transferring archived files from smithi188:/var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-06-09_21:24:02-fs-squid-distro-default-smithi/7747454/remote/smithi188/crash 2024-06-10T03:07:43.080 DEBUG:teuthology.orchestra.run.smithi188:> sudo tar c -f - -C /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/crash -- . 2024-06-10T03:07:43.115 INFO:teuthology.orchestra.run.smithi188.stderr:tar: /var/lib/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/crash: Cannot open: No such file or directory 2024-06-10T03:07:43.115 INFO:teuthology.orchestra.run.smithi188.stderr:tar: Error is not recoverable: exiting now 2024-06-10T03:07:43.117 INFO:tasks.cephadm:Checking cluster log for badness... 2024-06-10T03:07:43.117 DEBUG:teuthology.orchestra.run.smithi052:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-06-10T03:07:43.156 INFO:teuthology.orchestra.run.smithi052.stdout:2024-06-10T02:50:00.002085+0000 mon.a (mon.0) 1015 : cluster [WRN] Health detail: HEALTH_WARN 1 pool(s) do not have an application enabled 2024-06-10T03:07:43.157 WARNING:tasks.cephadm:Found errors (ERR|WRN|SEC) in cluster log 2024-06-10T03:07:43.157 DEBUG:teuthology.orchestra.run.smithi052:> sudo egrep '\[SEC\]' /var/log/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-06-10T03:07:43.233 DEBUG:teuthology.orchestra.run.smithi052:> sudo egrep '\[ERR\]' /var/log/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-06-10T03:07:43.310 DEBUG:teuthology.orchestra.run.smithi052:> sudo egrep '\[WRN\]' /var/log/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-06-10T03:07:43.384 INFO:teuthology.orchestra.run.smithi052.stdout:2024-06-10T02:50:00.002085+0000 mon.a (mon.0) 1015 : cluster [WRN] Health detail: HEALTH_WARN 1 pool(s) do not have an application enabled 2024-06-10T03:07:43.385 INFO:tasks.cephadm:Compressing logs... 2024-06-10T03:07:43.385 DEBUG:teuthology.orchestra.run.smithi052:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-06-10T03:07:43.427 DEBUG:teuthology.orchestra.run.smithi120:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-06-10T03:07:43.430 DEBUG:teuthology.orchestra.run.smithi188:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-06-10T03:07:43.450 INFO:teuthology.orchestra.run.smithi052.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/ceph.log 2024-06-10T03:07:43.450 INFO:teuthology.orchestra.run.smithi052.stderr:: No such file or directory 2024-06-10T03:07:43.450 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/ceph.audit.log 2024-06-10T03:07:43.451 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/ceph-mds.g.log 2024-06-10T03:07:43.452 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/ceph.log: /var/log/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/ceph-mds.j.log 2024-06-10T03:07:43.452 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/ceph-osd.3.log 2024-06-10T03:07:43.452 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/ceph-osd.0.log 2024-06-10T03:07:43.452 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/ceph-mgr.x.log 2024-06-10T03:07:43.454 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/ceph-mon.a.log 2024-06-10T03:07:43.454 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/ceph-osd.6.log 2024-06-10T03:07:43.454 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/ceph-mon.a.log: 90.2% -- replaced with /var/log/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/ceph.audit.log.gz 2024-06-10T03:07:43.454 INFO:teuthology.orchestra.run.smithi052.stderr: 87.3% -- replaced with /var/log/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/ceph.log.gz 2024-06-10T03:07:43.455 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/ceph.cephadm.log 2024-06-10T03:07:43.455 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/ceph-mds.a.log 2024-06-10T03:07:43.458 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/ceph.cephadm.log: 82.9% -- replaced with /var/log/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/ceph.cephadm.log.gz 2024-06-10T03:07:43.458 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/ceph-volume.log 2024-06-10T03:07:43.458 INFO:teuthology.orchestra.run.smithi052.stderr: 94.6% -- replaced with /var/log/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/ceph-mds.j.log.gz 2024-06-10T03:07:43.460 INFO:teuthology.orchestra.run.smithi120.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/ceph.log 2024-06-10T03:07:43.460 INFO:teuthology.orchestra.run.smithi120.stderr:: No such file or directory 2024-06-10T03:07:43.460 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/ceph.audit.log 2024-06-10T03:07:43.460 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/ceph-osd.7.log 2024-06-10T03:07:43.460 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/ceph.log: /var/log/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/ceph-mds.b.log 2024-06-10T03:07:43.460 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/ceph-osd.4.log 2024-06-10T03:07:43.460 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/ceph-osd.1.log 2024-06-10T03:07:43.460 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/ceph-mds.b.log: /var/log/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/ceph-mgr.y.log 2024-06-10T03:07:43.461 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/ceph-mds.h.log 2024-06-10T03:07:43.461 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/ceph-osd.10.log 2024-06-10T03:07:43.461 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/ceph-mds.h.log: 90.4% -- replaced with /var/log/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/ceph.audit.log.gz 2024-06-10T03:07:43.461 INFO:teuthology.orchestra.run.smithi120.stderr: 87.3% -- replaced with /var/log/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/ceph.log.gz 2024-06-10T03:07:43.461 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/ceph.cephadm.log 2024-06-10T03:07:43.461 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/ceph-mds.e.log 2024-06-10T03:07:43.462 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/ceph.cephadm.log: 82.5% -- replaced with /var/log/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/ceph.cephadm.log.gz 2024-06-10T03:07:43.462 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/ceph-volume.log 2024-06-10T03:07:43.466 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/ceph-mds.e.log: 92.9% -- replaced with /var/log/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/ceph-mgr.y.log.gz 2024-06-10T03:07:43.470 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/ceph-mon.b.log 2024-06-10T03:07:43.472 INFO:teuthology.orchestra.run.smithi188.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-06-10T03:07:43.472 INFO:teuthology.orchestra.run.smithi188.stderr:gzip -5 --verbose -- /var/log/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/ceph.log 2024-06-10T03:07:43.472 INFO:teuthology.orchestra.run.smithi188.stderr:gzip -5 --verbose -- /var/log/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/ceph-osd.2.log 2024-06-10T03:07:43.476 INFO:teuthology.orchestra.run.smithi188.stderr:gzip -5 --verbose -- /var/log/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/ceph-mds.f.log 2024-06-10T03:07:43.476 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/ceph.log: /var/log/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/ceph.audit.log 2024-06-10T03:07:43.476 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/ceph-osd.5.log 2024-06-10T03:07:43.476 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/ceph-mds.c.log 2024-06-10T03:07:43.476 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/ceph-mon.c.log 2024-06-10T03:07:43.476 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/ceph-mgr.z.log 2024-06-10T03:07:43.476 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/ceph-mds.i.log 2024-06-10T03:07:43.476 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/ceph.cephadm.log 2024-06-10T03:07:43.476 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/ceph-osd.11.log 2024-06-10T03:07:43.476 INFO:teuthology.orchestra.run.smithi188.stderr:gzip -5 --verbose -- /var/log/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/ceph-osd.8.log 2024-06-10T03:07:43.476 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/ceph.cephadm.log: /var/log/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/ceph-volume.log 2024-06-10T03:07:43.477 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/ceph-mds.l.log 2024-06-10T03:07:43.477 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-06-10T03:07:43.477 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/ceph-mds.d.log 2024-06-10T03:07:43.483 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/ceph-mds.k.log 2024-06-10T03:07:43.485 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/ceph-mon.b.log: 91.0% -- replaced with /var/log/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/ceph-volume.log.gz 2024-06-10T03:07:43.489 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/ceph-osd.9.log 2024-06-10T03:07:43.490 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-06-10T03:07:43.491 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/ceph-mds.d.log: 91.1% -- replaced with /var/log/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/ceph-volume.log.gz 2024-06-10T03:07:43.492 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-06-10T03:07:43.502 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/ceph-osd.9.log: 94.7%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.log 2024-06-10T03:07:43.502 INFO:teuthology.orchestra.run.smithi052.stderr: -- replaced with /var/log/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/ceph-mds.d.log.gz 2024-06-10T03:07:43.512 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/ceph-mds.k.log: /var/log/ceph/cephadm.log: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-06-10T03:07:43.523 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/cephadm.log: /var/log/ceph/ceph-client.admin.log: 92.8% -- replaced with /var/log/ceph/ceph-client.admin.log.gz 2024-06-10T03:07:43.523 INFO:teuthology.orchestra.run.smithi052.stderr: 94.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-06-10T03:07:43.566 INFO:teuthology.orchestra.run.smithi052.stderr: 94.5% -- replaced with /var/log/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/ceph-mds.a.log.gz 2024-06-10T03:07:43.627 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/ceph-mds.l.log: /var/log/ceph/cephadm.log: 91.1% -- replaced with /var/log/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/ceph-volume.log.gz 2024-06-10T03:07:43.663 INFO:teuthology.orchestra.run.smithi188.stderr: 93.1% -- replaced with /var/log/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/ceph-mgr.z.log.gz 2024-06-10T03:07:43.696 INFO:teuthology.orchestra.run.smithi052.stderr: 90.2% -- replaced with /var/log/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/ceph-mgr.x.log.gz 2024-06-10T03:07:43.762 INFO:teuthology.orchestra.run.smithi188.stderr: 82.4% 87.3% -- replaced with /var/log/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/ceph.log.gz -- replaced with /var/log/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/ceph.cephadm.log.gz 2024-06-10T03:07:43.762 INFO:teuthology.orchestra.run.smithi188.stderr: 2024-06-10T03:07:43.837 INFO:teuthology.orchestra.run.smithi188.stderr: 92.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-06-10T03:07:43.843 INFO:teuthology.orchestra.run.smithi188.stderr: 90.4% -- replaced with /var/log/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/ceph.audit.log.gz 2024-06-10T03:07:43.879 INFO:teuthology.orchestra.run.smithi120.stderr: 95.1% -- replaced with /var/log/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/ceph-mds.e.log.gz 2024-06-10T03:07:44.007 INFO:teuthology.orchestra.run.smithi052.stderr: 95.1% -- replaced with /var/log/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/ceph-mds.g.log.gz 2024-06-10T03:07:44.092 INFO:teuthology.orchestra.run.smithi120.stderr: 91.8% -- replaced with /var/log/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/ceph-mon.b.log.gz 2024-06-10T03:07:44.547 INFO:teuthology.orchestra.run.smithi052.stderr: 90.3% -- replaced with /var/log/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/ceph-mon.a.log.gz 2024-06-10T03:07:45.130 INFO:teuthology.orchestra.run.smithi120.stderr: 94.7% -- replaced with /var/log/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/ceph-mds.b.log.gz 2024-06-10T03:07:46.264 INFO:teuthology.orchestra.run.smithi188.stderr: 91.8% -- replaced with /var/log/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/ceph-mon.c.log.gz 2024-06-10T03:07:48.245 INFO:teuthology.orchestra.run.smithi188.stderr: 94.7% -- replaced with /var/log/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/ceph-mds.l.log.gz 2024-06-10T03:07:55.633 INFO:teuthology.orchestra.run.smithi120.stderr: 95.0% -- replaced with /var/log/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/ceph-mds.k.log.gz 2024-06-10T03:07:56.188 INFO:teuthology.orchestra.run.smithi188.stderr: 94.4% -- replaced with /var/log/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/ceph-mds.i.log.gz 2024-06-10T03:07:56.255 INFO:teuthology.orchestra.run.smithi120.stderr: 94.8% -- replaced with /var/log/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/ceph-mds.h.log.gz 2024-06-10T03:07:56.611 INFO:teuthology.orchestra.run.smithi120.stderr: 91.7% -- replaced with /var/log/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/ceph-osd.10.log.gz 2024-06-10T03:07:58.603 INFO:teuthology.orchestra.run.smithi052.stderr: 91.6% -- replaced with /var/log/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/ceph-osd.0.log.gz 2024-06-10T03:08:00.598 INFO:teuthology.orchestra.run.smithi120.stderr: 91.8% -- replaced with /var/log/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/ceph-osd.1.log.gz 2024-06-10T03:08:00.806 INFO:teuthology.orchestra.run.smithi052.stderr: 91.8% -- replaced with /var/log/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/ceph-osd.9.log.gz 2024-06-10T03:08:00.944 INFO:teuthology.orchestra.run.smithi052.stderr: 91.7% -- replaced with /var/log/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/ceph-osd.3.log.gz 2024-06-10T03:08:01.560 INFO:teuthology.orchestra.run.smithi052.stderr: 91.7% -- replaced with /var/log/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/ceph-osd.6.log.gz 2024-06-10T03:08:01.563 INFO:teuthology.orchestra.run.smithi052.stderr: 2024-06-10T03:08:01.563 INFO:teuthology.orchestra.run.smithi052.stderr:real 0m18.122s 2024-06-10T03:08:01.563 INFO:teuthology.orchestra.run.smithi052.stderr:user 1m7.994s 2024-06-10T03:08:01.563 INFO:teuthology.orchestra.run.smithi052.stderr:sys 0m1.993s 2024-06-10T03:08:02.539 INFO:teuthology.orchestra.run.smithi120.stderr: 91.6% -- replaced with /var/log/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/ceph-osd.7.log.gz 2024-06-10T03:08:09.610 INFO:teuthology.orchestra.run.smithi120.stderr: 91.8% -- replaced with /var/log/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/ceph-osd.4.log.gz 2024-06-10T03:08:09.612 INFO:teuthology.orchestra.run.smithi120.stderr: 2024-06-10T03:08:09.612 INFO:teuthology.orchestra.run.smithi120.stderr:real 0m26.168s 2024-06-10T03:08:09.613 INFO:teuthology.orchestra.run.smithi120.stderr:user 1m39.641s 2024-06-10T03:08:09.613 INFO:teuthology.orchestra.run.smithi120.stderr:sys 0m2.975s 2024-06-10T03:09:16.558 INFO:teuthology.orchestra.run.smithi188.stderr: 95.0% -- replaced with /var/log/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/ceph-mds.c.log.gz 2024-06-10T03:09:49.951 INFO:teuthology.orchestra.run.smithi188.stderr: 91.8% -- replaced with /var/log/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/ceph-osd.8.log.gz 2024-06-10T03:10:24.401 INFO:teuthology.orchestra.run.smithi188.stderr: 91.7% -- replaced with /var/log/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/ceph-osd.2.log.gz 2024-06-10T03:10:35.456 INFO:teuthology.orchestra.run.smithi188.stderr: 91.6% -- replaced with /var/log/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/ceph-osd.5.log.gz 2024-06-10T03:10:46.029 INFO:teuthology.orchestra.run.smithi188.stderr: 91.8% -- replaced with /var/log/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/ceph-osd.11.log.gz 2024-06-10T03:14:14.334 INFO:teuthology.orchestra.run.smithi188.stderr: 93.5% -- replaced with /var/log/ceph/d8676e9e-26d2-11ef-bc9b-c7b262605968/ceph-mds.f.log.gz 2024-06-10T03:14:14.339 INFO:teuthology.orchestra.run.smithi188.stderr: 2024-06-10T03:14:14.339 INFO:teuthology.orchestra.run.smithi188.stderr:real 6m30.891s 2024-06-10T03:14:14.339 INFO:teuthology.orchestra.run.smithi188.stderr:user 4m46.275s 2024-06-10T03:14:14.339 INFO:teuthology.orchestra.run.smithi188.stderr:sys 0m10.745s 2024-06-10T03:14:14.339 INFO:tasks.cephadm:Archiving logs... 2024-06-10T03:14:14.340 DEBUG:teuthology.misc:Transferring archived files from smithi052:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-06-09_21:24:02-fs-squid-distro-default-smithi/7747454/remote/smithi052/log 2024-06-10T03:14:14.341 DEBUG:teuthology.orchestra.run.smithi052:> sudo tar c -f - -C /var/log/ceph -- . 2024-06-10T03:14:25.302 DEBUG:teuthology.misc:Transferring archived files from smithi120:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-06-09_21:24:02-fs-squid-distro-default-smithi/7747454/remote/smithi120/log 2024-06-10T03:14:25.303 DEBUG:teuthology.orchestra.run.smithi120:> sudo tar c -f - -C /var/log/ceph -- . 2024-06-10T03:14:38.713 DEBUG:teuthology.misc:Transferring archived files from smithi188:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-06-09_21:24:02-fs-squid-distro-default-smithi/7747454/remote/smithi188/log 2024-06-10T03:14:38.715 DEBUG:teuthology.orchestra.run.smithi188:> sudo tar c -f - -C /var/log/ceph -- . 2024-06-10T03:15:24.013 INFO:tasks.cephadm:Removing cluster... 2024-06-10T03:15:24.014 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 --force 2024-06-10T03:15:24.216 INFO:teuthology.orchestra.run.smithi052.stdout:Deleting cluster with fsid: d8676e9e-26d2-11ef-bc9b-c7b262605968 2024-06-10T03:15:24.589 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 --force 2024-06-10T03:15:24.781 INFO:teuthology.orchestra.run.smithi120.stdout:Deleting cluster with fsid: d8676e9e-26d2-11ef-bc9b-c7b262605968 2024-06-10T03:15:25.148 DEBUG:teuthology.orchestra.run.smithi188:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid d8676e9e-26d2-11ef-bc9b-c7b262605968 --force 2024-06-10T03:15:25.350 INFO:teuthology.orchestra.run.smithi188.stdout:Deleting cluster with fsid: d8676e9e-26d2-11ef-bc9b-c7b262605968 2024-06-10T03:15:25.888 INFO:tasks.cephadm:Removing cephadm ... 2024-06-10T03:15:25.889 DEBUG:teuthology.orchestra.run.smithi052:> rm -rf /home/ubuntu/cephtest/cephadm 2024-06-10T03:15:25.909 DEBUG:teuthology.orchestra.run.smithi120:> rm -rf /home/ubuntu/cephtest/cephadm 2024-06-10T03:15:25.925 DEBUG:teuthology.orchestra.run.smithi188:> rm -rf /home/ubuntu/cephtest/cephadm 2024-06-10T03:15:25.941 INFO:tasks.cephadm:Teardown complete 2024-06-10T03:15:25.941 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-06-10T03:15:25.951 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-06-10T03:15:25.952 DEBUG:teuthology.orchestra.run.smithi052:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-06-10T03:15:25.955 DEBUG:teuthology.orchestra.run.smithi120:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-06-10T03:15:25.968 DEBUG:teuthology.orchestra.run.smithi188:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-06-10T03:15:26.078 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-06-10T03:15:26.090 INFO:teuthology.task.clock:Checking final clock skew... 2024-06-10T03:15:26.090 DEBUG:teuthology.orchestra.run.smithi052:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-06-10T03:15:26.093 DEBUG:teuthology.orchestra.run.smithi120:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-06-10T03:15:26.096 DEBUG:teuthology.orchestra.run.smithi188:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-06-10T03:15:26.107 INFO:teuthology.orchestra.run.smithi120.stderr:bash: line 1: ntpq: command not found 2024-06-10T03:15:26.108 INFO:teuthology.orchestra.run.smithi052.stderr:bash: line 1: ntpq: command not found 2024-06-10T03:15:26.110 INFO:teuthology.orchestra.run.smithi188.stderr:bash: line 1: ntpq: command not found 2024-06-10T03:15:26.111 INFO:teuthology.orchestra.run.smithi120.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-06-10T03:15:26.111 INFO:teuthology.orchestra.run.smithi120.stdout:=============================================================================== 2024-06-10T03:15:26.112 INFO:teuthology.orchestra.run.smithi120.stdout:^* hv01.front.sepia.ceph.com 3 8 377 92 +1794us[+1822us] +/- 66ms 2024-06-10T03:15:26.112 INFO:teuthology.orchestra.run.smithi120.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 25 -1284us[-1284us] +/- 79ms 2024-06-10T03:15:26.112 INFO:teuthology.orchestra.run.smithi120.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 25 -1711us[-1711us] +/- 79ms 2024-06-10T03:15:26.112 INFO:teuthology.orchestra.run.smithi120.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-06-10T03:15:26.113 INFO:teuthology.orchestra.run.smithi052.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-06-10T03:15:26.113 INFO:teuthology.orchestra.run.smithi052.stdout:=============================================================================== 2024-06-10T03:15:26.113 INFO:teuthology.orchestra.run.smithi052.stdout:^* hv01.front.sepia.ceph.com 3 6 377 34 +2111us[+2135us] +/- 67ms 2024-06-10T03:15:26.114 INFO:teuthology.orchestra.run.smithi052.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 28 -983us[ -983us] +/- 79ms 2024-06-10T03:15:26.114 INFO:teuthology.orchestra.run.smithi052.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 31 -1386us[-1386us] +/- 79ms 2024-06-10T03:15:26.114 INFO:teuthology.orchestra.run.smithi052.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-06-10T03:15:26.116 INFO:teuthology.orchestra.run.smithi188.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-06-10T03:15:26.116 INFO:teuthology.orchestra.run.smithi188.stdout:=============================================================================== 2024-06-10T03:15:26.116 INFO:teuthology.orchestra.run.smithi188.stdout:^* hv01.front.sepia.ceph.com 3 6 377 30 +2023us[+2025us] +/- 67ms 2024-06-10T03:15:26.116 INFO:teuthology.orchestra.run.smithi188.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 28 -1039us[-1039us] +/- 79ms 2024-06-10T03:15:26.117 INFO:teuthology.orchestra.run.smithi188.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 31 -1465us[-1464us] +/- 79ms 2024-06-10T03:15:26.117 INFO:teuthology.orchestra.run.smithi188.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-06-10T03:15:26.118 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-06-10T03:15:26.128 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-06-10T03:15:26.129 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-06-10T03:15:26.139 DEBUG:teuthology.orchestra.run.smithi052:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-06-10T03:15:26.245 DEBUG:teuthology.orchestra.run.smithi120:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-06-10T03:15:26.309 DEBUG:teuthology.orchestra.run.smithi188:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-06-10T03:15:26.401 DEBUG:teuthology.orchestra.run.smithi052:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-06-10T03:15:26.432 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-10T03:15:26.433 DEBUG:teuthology.orchestra.run.smithi120:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-06-10T03:15:26.458 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-10T03:15:26.459 DEBUG:teuthology.orchestra.run.smithi188:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-06-10T03:15:26.485 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-10T03:15:26.485 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-06-10T03:15:26.496 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-06-10T03:15:26.506 INFO:teuthology.task.internal:Duration was 2774.008446 seconds 2024-06-10T03:15:26.506 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-06-10T03:15:26.516 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-06-10T03:15:26.516 DEBUG:teuthology.orchestra.run.smithi052:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-06-10T03:15:26.519 DEBUG:teuthology.orchestra.run.smithi120:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-06-10T03:15:26.522 DEBUG:teuthology.orchestra.run.smithi188:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-06-10T03:15:26.555 INFO:teuthology.orchestra.run.smithi120.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-06-10T03:15:26.560 INFO:teuthology.orchestra.run.smithi052.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-06-10T03:15:26.574 INFO:teuthology.orchestra.run.smithi188.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-06-10T03:15:26.933 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-06-10T03:15:26.933 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi052.front.sepia.ceph.com 2024-06-10T03:15:26.933 DEBUG:teuthology.orchestra.run.smithi052:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-06-10T03:15:26.956 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi120.front.sepia.ceph.com 2024-06-10T03:15:26.956 DEBUG:teuthology.orchestra.run.smithi120:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-06-10T03:15:26.999 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi188.front.sepia.ceph.com 2024-06-10T03:15:27.000 DEBUG:teuthology.orchestra.run.smithi188:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-06-10T03:15:27.023 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-06-10T03:15:27.023 DEBUG:teuthology.orchestra.run.smithi052:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-06-10T03:15:27.026 DEBUG:teuthology.orchestra.run.smithi120:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-06-10T03:15:27.042 DEBUG:teuthology.orchestra.run.smithi188:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-06-10T03:15:27.091 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-06-10T03:15:27.091 DEBUG:teuthology.orchestra.run.smithi052:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-06-10T03:15:27.095 DEBUG:teuthology.orchestra.run.smithi120:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-06-10T03:15:27.107 DEBUG:teuthology.orchestra.run.smithi188:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-06-10T03:15:28.228 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-06-10T03:15:28.240 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-06-10T03:15:28.240 DEBUG:teuthology.orchestra.run.smithi052:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-06-10T03:15:28.266 DEBUG:teuthology.orchestra.run.smithi120:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-06-10T03:15:28.292 DEBUG:teuthology.orchestra.run.smithi188:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-06-10T03:15:28.322 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-06-10T03:15:28.333 DEBUG:teuthology.orchestra.run.smithi052:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-06-10T03:15:28.335 DEBUG:teuthology.orchestra.run.smithi120:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-06-10T03:15:28.338 DEBUG:teuthology.orchestra.run.smithi188:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-06-10T03:15:28.357 INFO:teuthology.orchestra.run.smithi052.stdout:kernel.core_pattern = core 2024-06-10T03:15:28.360 INFO:teuthology.orchestra.run.smithi120.stdout:kernel.core_pattern = core 2024-06-10T03:15:28.388 INFO:teuthology.orchestra.run.smithi188.stdout:kernel.core_pattern = core 2024-06-10T03:15:28.416 DEBUG:teuthology.orchestra.run.smithi052:> test -e /home/ubuntu/cephtest/archive/coredump 2024-06-10T03:15:28.431 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-10T03:15:28.431 DEBUG:teuthology.orchestra.run.smithi120:> test -e /home/ubuntu/cephtest/archive/coredump 2024-06-10T03:15:28.446 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-10T03:15:28.447 DEBUG:teuthology.orchestra.run.smithi188:> test -e /home/ubuntu/cephtest/archive/coredump 2024-06-10T03:15:28.472 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-10T03:15:28.472 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-06-10T03:15:28.483 INFO:teuthology.task.internal:Transferring archived files... 2024-06-10T03:15:28.485 DEBUG:teuthology.misc:Transferring archived files from smithi052:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-06-09_21:24:02-fs-squid-distro-default-smithi/7747454/remote/smithi052 2024-06-10T03:15:28.485 DEBUG:teuthology.orchestra.run.smithi052:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-06-10T03:15:28.548 DEBUG:teuthology.misc:Transferring archived files from smithi120:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-06-09_21:24:02-fs-squid-distro-default-smithi/7747454/remote/smithi120 2024-06-10T03:15:28.548 DEBUG:teuthology.orchestra.run.smithi120:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-06-10T03:15:28.584 DEBUG:teuthology.misc:Transferring archived files from smithi188:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-06-09_21:24:02-fs-squid-distro-default-smithi/7747454/remote/smithi188 2024-06-10T03:15:28.584 DEBUG:teuthology.orchestra.run.smithi188:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-06-10T03:15:28.622 INFO:teuthology.task.internal:Removing archive directory... 2024-06-10T03:15:28.622 DEBUG:teuthology.orchestra.run.smithi052:> rm -rf -- /home/ubuntu/cephtest/archive 2024-06-10T03:15:28.625 DEBUG:teuthology.orchestra.run.smithi120:> rm -rf -- /home/ubuntu/cephtest/archive 2024-06-10T03:15:28.627 DEBUG:teuthology.orchestra.run.smithi188:> rm -rf -- /home/ubuntu/cephtest/archive 2024-06-10T03:15:28.674 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-06-10T03:15:28.687 INFO:teuthology.task.internal:Not uploading archives. 2024-06-10T03:15:28.687 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-06-10T03:15:28.784 INFO:teuthology.task.internal:Tidying up after the test... 2024-06-10T03:15:28.784 DEBUG:teuthology.orchestra.run.smithi052:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-06-10T03:15:28.787 DEBUG:teuthology.orchestra.run.smithi120:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-06-10T03:15:28.789 DEBUG:teuthology.orchestra.run.smithi188:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-06-10T03:15:28.800 INFO:teuthology.orchestra.run.smithi052.stdout: 266037 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jun 10 03:15 /home/ubuntu/cephtest 2024-06-10T03:15:28.802 INFO:teuthology.orchestra.run.smithi120.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jun 10 03:15 /home/ubuntu/cephtest 2024-06-10T03:15:28.805 INFO:teuthology.orchestra.run.smithi188.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jun 10 03:15 /home/ubuntu/cephtest 2024-06-10T03:15:28.806 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-06-10T03:15:28.818 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-06-10T03:15:28.844 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/no} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/no 5-quiesce/with-quiesce 6-workunit/suites/blogbench}} duration: 2774.008446455002 failure_reason: '"2024-06-10T02:50:00.002085+0000 mon.a (mon.0) 1015 : cluster [WRN] Health detail: HEALTH_WARN 1 pool(s) do not have an application enabled" in cluster log' flavor: default owner: scheduled_teuthology@teuthology success: false 2024-06-10T03:15:28.845 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-06-10T03:15:28.928 INFO:teuthology.run:FAIL